Attachment 827386 Details for Bug 1032598 – vdsm.log

vdsm.log

vdsm.log (text/plain), 18.64 MB, created by Pratik Pravin Bandarkar on 2013-11-21 17:20:50 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Pratik Pravin Bandarkar

Created: 2013-11-21 17:20:50 UTC

Size: 18.64 MB

patch

obsolete

>Dummy-15909965::DEBUG::2013-11-19 21:01:01,753::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.04418 s, 23.2 MB/s\n'; <rc> = 0
>Thread-15909988::DEBUG::2013-11-19 21:01:02,171::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15909988::INFO::2013-11-19 21:01:02,172::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15909988::DEBUG::2013-11-19 21:01:02,173::task::495::TaskManager.Task::(_debug) Task 993b65de-5d01-4045-a3d2-4b44e46b13a9: moving from state init -> state preparing
>Thread-15909988::DEBUG::2013-11-19 21:01:02,173::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=274 spmId=2
>Thread-15909988::DEBUG::2013-11-19 21:01:02,173::task::495::TaskManager.Task::(_debug) Task 993b65de-5d01-4045-a3d2-4b44e46b13a9: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15909988::DEBUG::2013-11-19 21:01:02,174::task::495::TaskManager.Task::(_debug) Task 993b65de-5d01-4045-a3d2-4b44e46b13a9: moving from state preparing -> state finished
>Thread-15909988::DEBUG::2013-11-19 21:01:02,175::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15909988::DEBUG::2013-11-19 21:01:02,175::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15909988::DEBUG::2013-11-19 21:01:02,175::task::495::TaskManager.Task::(_debug) Task 993b65de-5d01-4045-a3d2-4b44e46b13a9: ref 0 aborting False
>Thread-15909988::INFO::2013-11-19 21:01:02,176::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15909989::DEBUG::2013-11-19 21:01:02,181::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15909989::INFO::2013-11-19 21:01:02,183::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15909989::DEBUG::2013-11-19 21:01:02,184::task::495::TaskManager.Task::(_debug) Task 613006dc-c03a-4bb1-a590-7c175598d210: moving from state init -> state preparing
>Thread-15909989::DEBUG::2013-11-19 21:01:02,184::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`13a1dfd1-3f3a-442b-8cdc-a2c8668a7cf1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15909989::DEBUG::2013-11-19 21:01:02,185::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15909989::DEBUG::2013-11-19 21:01:02,185::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15909989::DEBUG::2013-11-19 21:01:02,186::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`13a1dfd1-3f3a-442b-8cdc-a2c8668a7cf1`::Granted request
>Thread-15909989::DEBUG::2013-11-19 21:01:02,186::task::495::TaskManager.Task::(_debug) Task 613006dc-c03a-4bb1-a590-7c175598d210: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15909989::DEBUG::2013-11-19 21:01:02,187::task::495::TaskManager.Task::(_debug) Task 613006dc-c03a-4bb1-a590-7c175598d210: ref 1 aborting False
>Thread-15909989::WARNING::2013-11-19 21:01:02,188::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15909989::DEBUG::2013-11-19 21:01:02,189::task::495::TaskManager.Task::(_debug) Task 613006dc-c03a-4bb1-a590-7c175598d210: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15909989::DEBUG::2013-11-19 21:01:02,189::task::495::TaskManager.Task::(_debug) Task 613006dc-c03a-4bb1-a590-7c175598d210: moving from state preparing -> state finished
>Thread-15909989::DEBUG::2013-11-19 21:01:02,189::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15909989::DEBUG::2013-11-19 21:01:02,190::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15909989::DEBUG::2013-11-19 21:01:02,190::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15909989::DEBUG::2013-11-19 21:01:02,191::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15909989::DEBUG::2013-11-19 21:01:02,191::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15909989::DEBUG::2013-11-19 21:01:02,191::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15909989::DEBUG::2013-11-19 21:01:02,191::task::495::TaskManager.Task::(_debug) Task 613006dc-c03a-4bb1-a590-7c175598d210: ref 0 aborting False
>Thread-15909989::INFO::2013-11-19 21:01:02,192::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15909990::INFO::2013-11-19 21:01:02,482::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15909990::DEBUG::2013-11-19 21:01:02,483::task::495::TaskManager.Task::(_debug) Task 6c8b3899-81bc-4fdc-ba48-48e8cdb5b30b: moving from state init -> state preparing
>Thread-15909990::DEBUG::2013-11-19 21:01:02,483::task::495::TaskManager.Task::(_debug) Task 6c8b3899-81bc-4fdc-ba48-48e8cdb5b30b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000282049179077', 'lastCheck': 1384894855.904762, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00129199028015', 'lastCheck': 1384894852.394912, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384894857.41732, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000365018844604', 'lastCheck': 1384894854.3821671, 'valid': True, 'code': 0}}
>Thread-15909990::DEBUG::2013-11-19 21:01:02,484::task::495::TaskManager.Task::(_debug) Task 6c8b3899-81bc-4fdc-ba48-48e8cdb5b30b: moving from state preparing -> state finished
>Thread-15909990::DEBUG::2013-11-19 21:01:02,484::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15909990::DEBUG::2013-11-19 21:01:02,485::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15909990::DEBUG::2013-11-19 21:01:02,485::task::495::TaskManager.Task::(_debug) Task 6c8b3899-81bc-4fdc-ba48-48e8cdb5b30b: ref 0 aborting False
>Thread-15909990::INFO::2013-11-19 21:01:02,485::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000282049179077', 'lastCheck': 1384894855.904762, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00129199028015', 'lastCheck': 1384894852.394912, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384894857.41732, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000365018844604', 'lastCheck': 1384894854.3821671, 'valid': True, 'code': 0}}
>Dummy-15909965::DEBUG::2013-11-19 21:01:03,764::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:01:04,180::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0443889 s, 23.1 MB/s\n'; <rc> = 0
>Dummy-15909965::DEBUG::2013-11-19 21:01:06,190::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:01:06,682::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0772103 s, 13.3 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:01:07,527::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15909965::DEBUG::2013-11-19 21:01:08,697::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:01:09,197::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0623932 s, 16.4 MB/s\n'; <rc> = 0
>Dummy-15909965::DEBUG::2013-11-19 21:01:11,211::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:01:11,788::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0828814 s, 12.4 MB/s\n'; <rc> = 0
>Thread-15909996::DEBUG::2013-11-19 21:01:12,435::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15909996::INFO::2013-11-19 21:01:12,436::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15909996::DEBUG::2013-11-19 21:01:12,437::task::495::TaskManager.Task::(_debug) Task 59171e4f-397b-492b-9b6f-3236be8e6bf4: moving from state init -> state preparing
>Thread-15909996::DEBUG::2013-11-19 21:01:12,446::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=274 spmId=2
>Thread-15909996::DEBUG::2013-11-19 21:01:12,447::task::495::TaskManager.Task::(_debug) Task 59171e4f-397b-492b-9b6f-3236be8e6bf4: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15909996::DEBUG::2013-11-19 21:01:12,447::task::495::TaskManager.Task::(_debug) Task 59171e4f-397b-492b-9b6f-3236be8e6bf4: moving from state preparing -> state finished
>Thread-15909996::DEBUG::2013-11-19 21:01:12,448::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15909996::DEBUG::2013-11-19 21:01:12,448::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15909996::DEBUG::2013-11-19 21:01:12,449::task::495::TaskManager.Task::(_debug) Task 59171e4f-397b-492b-9b6f-3236be8e6bf4: ref 0 aborting False
>Thread-15909996::INFO::2013-11-19 21:01:12,450::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15909997::DEBUG::2013-11-19 21:01:12,462::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15909997::INFO::2013-11-19 21:01:12,463::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15909997::DEBUG::2013-11-19 21:01:12,464::task::495::TaskManager.Task::(_debug) Task 560e81b2-e9ed-4b17-96c1-eddca5cf97cc: moving from state init -> state preparing
>Thread-15909997::DEBUG::2013-11-19 21:01:12,465::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`38cab960-17df-479c-ae61-27f84ed5dd2b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15909997::DEBUG::2013-11-19 21:01:12,466::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15909997::DEBUG::2013-11-19 21:01:12,468::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15909997::DEBUG::2013-11-19 21:01:12,468::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`38cab960-17df-479c-ae61-27f84ed5dd2b`::Granted request
>Thread-15909997::DEBUG::2013-11-19 21:01:12,470::task::495::TaskManager.Task::(_debug) Task 560e81b2-e9ed-4b17-96c1-eddca5cf97cc: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15909997::DEBUG::2013-11-19 21:01:12,471::task::495::TaskManager.Task::(_debug) Task 560e81b2-e9ed-4b17-96c1-eddca5cf97cc: ref 1 aborting False
>Thread-15909997::WARNING::2013-11-19 21:01:12,474::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15909997::DEBUG::2013-11-19 21:01:12,475::task::495::TaskManager.Task::(_debug) Task 560e81b2-e9ed-4b17-96c1-eddca5cf97cc: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15909997::DEBUG::2013-11-19 21:01:12,476::task::495::TaskManager.Task::(_debug) Task 560e81b2-e9ed-4b17-96c1-eddca5cf97cc: moving from state preparing -> state finished
>Thread-15909997::DEBUG::2013-11-19 21:01:12,477::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15909997::DEBUG::2013-11-19 21:01:12,477::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15909997::DEBUG::2013-11-19 21:01:12,478::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15909997::DEBUG::2013-11-19 21:01:12,479::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15909997::DEBUG::2013-11-19 21:01:12,480::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15909997::DEBUG::2013-11-19 21:01:12,480::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15909997::DEBUG::2013-11-19 21:01:12,481::task::495::TaskManager.Task::(_debug) Task 560e81b2-e9ed-4b17-96c1-eddca5cf97cc: ref 0 aborting False
>Thread-15909997::INFO::2013-11-19 21:01:12,482::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15909998::INFO::2013-11-19 21:01:12,688::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15909998::DEBUG::2013-11-19 21:01:12,688::task::495::TaskManager.Task::(_debug) Task ff581736-df30-4ea6-87ae-90afb178060d: moving from state init -> state preparing
>Thread-15909998::DEBUG::2013-11-19 21:01:12,689::task::495::TaskManager.Task::(_debug) Task ff581736-df30-4ea6-87ae-90afb178060d: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000279188156128', 'lastCheck': 1384894865.9636569, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00115513801575', 'lastCheck': 1384894872.5390949, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384894867.5280011, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000637054443359', 'lastCheck': 1384894864.5238931, 'valid': True, 'code': 0}}
>Thread-15909998::DEBUG::2013-11-19 21:01:12,690::task::495::TaskManager.Task::(_debug) Task ff581736-df30-4ea6-87ae-90afb178060d: moving from state preparing -> state finished
>Thread-15909998::DEBUG::2013-11-19 21:01:12,690::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15909998::DEBUG::2013-11-19 21:01:12,691::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15909998::DEBUG::2013-11-19 21:01:12,691::task::495::TaskManager.Task::(_debug) Task ff581736-df30-4ea6-87ae-90afb178060d: ref 0 aborting False
>Thread-15909998::INFO::2013-11-19 21:01:12,692::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000279188156128', 'lastCheck': 1384894865.9636569, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00115513801575', 'lastCheck': 1384894872.5390949, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384894867.5280011, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000637054443359', 'lastCheck': 1384894864.5238931, 'valid': True, 'code': 0}}
>Dummy-15909965::DEBUG::2013-11-19 21:01:13,803::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:01:14,379::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0804903 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15909965::DEBUG::2013-11-19 21:01:16,392::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:01:16,945::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0796164 s, 12.9 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:01:17,624::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15909965::DEBUG::2013-11-19 21:01:18,960::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:01:19,607::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0765105 s, 13.4 MB/s\n'; <rc> = 0
>Thread-4922911::INFO::2013-11-19 21:01:21,196::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-4922911::DEBUG::2013-11-19 21:01:21,197::task::495::TaskManager.Task::(_debug) Task 5bc39542-2cfa-490d-8151-6a459e8a4c98: moving from state init -> state preparing
>Thread-4922911::DEBUG::2013-11-19 21:01:21,199::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`ad7e2cd9-7ffd-4b6b-a244-5aacf97e0c71`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-4922911::DEBUG::2013-11-19 21:01:21,199::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-4922911::DEBUG::2013-11-19 21:01:21,200::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-4922911::DEBUG::2013-11-19 21:01:21,201::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`ad7e2cd9-7ffd-4b6b-a244-5aacf97e0c71`::Granted request
>Thread-4922911::DEBUG::2013-11-19 21:01:21,202::task::495::TaskManager.Task::(_debug) Task 5bc39542-2cfa-490d-8151-6a459e8a4c98: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-4922911::DEBUG::2013-11-19 21:01:21,202::task::495::TaskManager.Task::(_debug) Task 5bc39542-2cfa-490d-8151-6a459e8a4c98: ref 1 aborting False
>Thread-4922911::DEBUG::2013-11-19 21:01:21,203::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-4922911::DEBUG::2013-11-19 21:01:21,215::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:01:21,627::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-4922911::DEBUG::2013-11-19 21:01:22,013::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-4922911::DEBUG::2013-11-19 21:01:22,093::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-4922911::DEBUG::2013-11-19 21:01:22,104::task::495::TaskManager.Task::(_debug) Task 5bc39542-2cfa-490d-8151-6a459e8a4c98: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-4922911::DEBUG::2013-11-19 21:01:22,105::task::495::TaskManager.Task::(_debug) Task 5bc39542-2cfa-490d-8151-6a459e8a4c98: moving from state preparing -> state finished
>Thread-4922911::DEBUG::2013-11-19 21:01:22,106::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-4922911::DEBUG::2013-11-19 21:01:22,106::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-4922911::DEBUG::2013-11-19 21:01:22,108::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-4922911::DEBUG::2013-11-19 21:01:22,109::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-4922911::DEBUG::2013-11-19 21:01:22,110::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-4922911::DEBUG::2013-11-19 21:01:22,110::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-4922911::DEBUG::2013-11-19 21:01:22,112::task::495::TaskManager.Task::(_debug) Task 5bc39542-2cfa-490d-8151-6a459e8a4c98: ref 0 aborting False
>Thread-4922911::INFO::2013-11-19 21:01:22,112::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-4922911::INFO::2013-11-19 21:01:22,114::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-4922911::DEBUG::2013-11-19 21:01:22,115::task::495::TaskManager.Task::(_debug) Task d2c8edc6-d7ef-41a3-9b50-257972b687fd: moving from state init -> state preparing
>Thread-4922911::DEBUG::2013-11-19 21:01:22,116::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`3c0ecf2d-3d46-449f-84dc-3e55cf0cfc0e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-4922911::DEBUG::2013-11-19 21:01:22,117::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-4922911::DEBUG::2013-11-19 21:01:22,118::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-4922911::DEBUG::2013-11-19 21:01:22,119::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`3c0ecf2d-3d46-449f-84dc-3e55cf0cfc0e`::Granted request
>Thread-4922911::DEBUG::2013-11-19 21:01:22,120::task::495::TaskManager.Task::(_debug) Task d2c8edc6-d7ef-41a3-9b50-257972b687fd: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-4922911::DEBUG::2013-11-19 21:01:22,121::task::495::TaskManager.Task::(_debug) Task d2c8edc6-d7ef-41a3-9b50-257972b687fd: ref 1 aborting False
>Thread-4922911::DEBUG::2013-11-19 21:01:22,122::task::495::TaskManager.Task::(_debug) Task d2c8edc6-d7ef-41a3-9b50-257972b687fd: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-4922911::DEBUG::2013-11-19 21:01:22,122::task::495::TaskManager.Task::(_debug) Task d2c8edc6-d7ef-41a3-9b50-257972b687fd: moving from state preparing -> state finished
>Thread-4922911::DEBUG::2013-11-19 21:01:22,123::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-4922911::DEBUG::2013-11-19 21:01:22,124::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-4922911::DEBUG::2013-11-19 21:01:22,125::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-4922911::DEBUG::2013-11-19 21:01:22,126::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-4922911::DEBUG::2013-11-19 21:01:22,126::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-4922911::DEBUG::2013-11-19 21:01:22,127::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-4922911::DEBUG::2013-11-19 21:01:22,128::task::495::TaskManager.Task::(_debug) Task d2c8edc6-d7ef-41a3-9b50-257972b687fd: ref 0 aborting False
>Thread-4922911::INFO::2013-11-19 21:01:22,129::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-4922911::INFO::2013-11-19 21:01:22,130::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-4922911::DEBUG::2013-11-19 21:01:22,131::task::495::TaskManager.Task::(_debug) Task 7464a467-3e8f-4a2d-a8e1-dbff00120867: moving from state init -> state preparing
>Thread-4922911::DEBUG::2013-11-19 21:01:22,132::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f9461eb8-3a82-47ba-b0e4-41deaf87f8c0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-4922911::DEBUG::2013-11-19 21:01:22,133::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-4922911::DEBUG::2013-11-19 21:01:22,134::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-4922911::DEBUG::2013-11-19 21:01:22,134::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f9461eb8-3a82-47ba-b0e4-41deaf87f8c0`::Granted request
>Thread-4922911::DEBUG::2013-11-19 21:01:22,136::task::495::TaskManager.Task::(_debug) Task 7464a467-3e8f-4a2d-a8e1-dbff00120867: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-4922911::DEBUG::2013-11-19 21:01:22,137::task::495::TaskManager.Task::(_debug) Task 7464a467-3e8f-4a2d-a8e1-dbff00120867: ref 1 aborting False
>Thread-4922911::DEBUG::2013-11-19 21:01:22,138::task::495::TaskManager.Task::(_debug) Task 7464a467-3e8f-4a2d-a8e1-dbff00120867: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-4922911::DEBUG::2013-11-19 21:01:22,138::task::495::TaskManager.Task::(_debug) Task 7464a467-3e8f-4a2d-a8e1-dbff00120867: moving from state preparing -> state finished
>Thread-4922911::DEBUG::2013-11-19 21:01:22,139::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-4922911::DEBUG::2013-11-19 21:01:22,140::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-4922911::DEBUG::2013-11-19 21:01:22,141::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-4922911::DEBUG::2013-11-19 21:01:22,142::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-4922911::DEBUG::2013-11-19 21:01:22,142::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-4922911::DEBUG::2013-11-19 21:01:22,143::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-4922911::DEBUG::2013-11-19 21:01:22,144::task::495::TaskManager.Task::(_debug) Task 7464a467-3e8f-4a2d-a8e1-dbff00120867: ref 0 aborting False
>Thread-4922911::INFO::2013-11-19 21:01:22,145::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-4922911::INFO::2013-11-19 21:01:22,146::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-4922911::DEBUG::2013-11-19 21:01:22,146::task::495::TaskManager.Task::(_debug) Task 9a21f246-cb53-4233-a17c-d6454d07d8ee: moving from state init -> state preparing
>Thread-4922911::DEBUG::2013-11-19 21:01:22,148::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`24a3070b-1fb7-4dcd-8645-98f01a10c769`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-4922911::DEBUG::2013-11-19 21:01:22,149::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-4922911::DEBUG::2013-11-19 21:01:22,149::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-4922911::DEBUG::2013-11-19 21:01:22,150::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`24a3070b-1fb7-4dcd-8645-98f01a10c769`::Granted request
>Thread-4922911::DEBUG::2013-11-19 21:01:22,152::task::495::TaskManager.Task::(_debug) Task 9a21f246-cb53-4233-a17c-d6454d07d8ee: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-4922911::DEBUG::2013-11-19 21:01:22,152::task::495::TaskManager.Task::(_debug) Task 9a21f246-cb53-4233-a17c-d6454d07d8ee: ref 1 aborting False
>Thread-4922911::DEBUG::2013-11-19 21:01:22,153::task::495::TaskManager.Task::(_debug) Task 9a21f246-cb53-4233-a17c-d6454d07d8ee: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-4922911::DEBUG::2013-11-19 21:01:22,154::task::495::TaskManager.Task::(_debug) Task 9a21f246-cb53-4233-a17c-d6454d07d8ee: moving from state preparing -> state finished
>Thread-4922911::DEBUG::2013-11-19 21:01:22,155::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-4922911::DEBUG::2013-11-19 21:01:22,155::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-4922911::DEBUG::2013-11-19 21:01:22,156::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-4922911::DEBUG::2013-11-19 21:01:22,157::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-4922911::DEBUG::2013-11-19 21:01:22,158::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-4922911::DEBUG::2013-11-19 21:01:22,158::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-4922911::DEBUG::2013-11-19 21:01:22,159::task::495::TaskManager.Task::(_debug) Task 9a21f246-cb53-4233-a17c-d6454d07d8ee: ref 0 aborting False
>Thread-4922911::INFO::2013-11-19 21:01:22,160::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15909965::DEBUG::2013-11-19 21:01:22,290::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.188912 s, 5.4 MB/s\n'; <rc> = 0
>Thread-15909535::DEBUG::2013-11-19 21:01:22,567::guestIF::97::vm.Vm::(_handleMessage) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::u'AOS1Batch@brd-klee.dk'
>Thread-15910004::DEBUG::2013-11-19 21:01:22,610::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910004::INFO::2013-11-19 21:01:22,611::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910004::DEBUG::2013-11-19 21:01:22,612::task::495::TaskManager.Task::(_debug) Task 04c3b15b-5f11-4801-9334-b7c02ec9353e: moving from state init -> state preparing
>Thread-15910004::DEBUG::2013-11-19 21:01:22,612::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=274 spmId=2
>Thread-15910004::DEBUG::2013-11-19 21:01:22,613::task::495::TaskManager.Task::(_debug) Task 04c3b15b-5f11-4801-9334-b7c02ec9353e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910004::DEBUG::2013-11-19 21:01:22,614::task::495::TaskManager.Task::(_debug) Task 04c3b15b-5f11-4801-9334-b7c02ec9353e: moving from state preparing -> state finished
>Thread-15910004::DEBUG::2013-11-19 21:01:22,614::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910004::DEBUG::2013-11-19 21:01:22,615::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910004::DEBUG::2013-11-19 21:01:22,615::task::495::TaskManager.Task::(_debug) Task 04c3b15b-5f11-4801-9334-b7c02ec9353e: ref 0 aborting False
>Thread-15910004::INFO::2013-11-19 21:01:22,616::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910005::DEBUG::2013-11-19 21:01:22,621::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910005::INFO::2013-11-19 21:01:22,621::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15868524::INFO::2013-11-19 21:01:22,630::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15910005::DEBUG::2013-11-19 21:01:22,632::task::495::TaskManager.Task::(_debug) Task 212cb393-ef4e-4c5d-b925-b91acd034e46: moving from state init -> state preparing
>Thread-15868524::DEBUG::2013-11-19 21:01:22,633::task::495::TaskManager.Task::(_debug) Task a2100830-05f0-4377-add4-d1f6895dab03: moving from state init -> state preparing
>Thread-15910005::DEBUG::2013-11-19 21:01:22,634::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b71149ba-a62c-47e1-885b-fe53799e6680`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15868524::DEBUG::2013-11-19 21:01:22,636::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`737867fc-820c-42ed-af00-580a718a4926`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15910005::DEBUG::2013-11-19 21:01:22,637::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15868524::DEBUG::2013-11-19 21:01:22,638::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15910005::DEBUG::2013-11-19 21:01:22,639::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910005::DEBUG::2013-11-19 21:01:22,640::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b71149ba-a62c-47e1-885b-fe53799e6680`::Granted request
>Thread-15868524::DEBUG::2013-11-19 21:01:22,641::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15910005::DEBUG::2013-11-19 21:01:22,641::task::495::TaskManager.Task::(_debug) Task 212cb393-ef4e-4c5d-b925-b91acd034e46: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15868524::DEBUG::2013-11-19 21:01:22,642::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`737867fc-820c-42ed-af00-580a718a4926`::Granted request
>Thread-15910005::DEBUG::2013-11-19 21:01:22,643::task::495::TaskManager.Task::(_debug) Task 212cb393-ef4e-4c5d-b925-b91acd034e46: ref 1 aborting False
>Thread-15868524::DEBUG::2013-11-19 21:01:22,644::task::495::TaskManager.Task::(_debug) Task a2100830-05f0-4377-add4-d1f6895dab03: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15910005::WARNING::2013-11-19 21:01:22,647::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15868524::DEBUG::2013-11-19 21:01:22,647::task::495::TaskManager.Task::(_debug) Task a2100830-05f0-4377-add4-d1f6895dab03: ref 1 aborting False
>Thread-15910005::DEBUG::2013-11-19 21:01:22,648::task::495::TaskManager.Task::(_debug) Task 212cb393-ef4e-4c5d-b925-b91acd034e46: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15868524::DEBUG::2013-11-19 21:01:22,649::task::495::TaskManager.Task::(_debug) Task a2100830-05f0-4377-add4-d1f6895dab03: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15910005::DEBUG::2013-11-19 21:01:22,650::task::495::TaskManager.Task::(_debug) Task 212cb393-ef4e-4c5d-b925-b91acd034e46: moving from state preparing -> state finished
>Thread-15868524::DEBUG::2013-11-19 21:01:22,651::task::495::TaskManager.Task::(_debug) Task a2100830-05f0-4377-add4-d1f6895dab03: moving from state preparing -> state finished
>Thread-15910005::DEBUG::2013-11-19 21:01:22,651::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15868524::DEBUG::2013-11-19 21:01:22,652::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15910005::DEBUG::2013-11-19 21:01:22,653::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15868524::DEBUG::2013-11-19 21:01:22,653::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910005::DEBUG::2013-11-19 21:01:22,654::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15868524::DEBUG::2013-11-19 21:01:22,655::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15910005::DEBUG::2013-11-19 21:01:22,656::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910005::DEBUG::2013-11-19 21:01:22,657::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910005::DEBUG::2013-11-19 21:01:22,657::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15868524::DEBUG::2013-11-19 21:01:22,658::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15910005::DEBUG::2013-11-19 21:01:22,658::task::495::TaskManager.Task::(_debug) Task 212cb393-ef4e-4c5d-b925-b91acd034e46: ref 0 aborting False
>Thread-15868524::DEBUG::2013-11-19 21:01:22,659::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15910005::INFO::2013-11-19 21:01:22,660::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15868524::DEBUG::2013-11-19 21:01:22,660::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15868524::DEBUG::2013-11-19 21:01:22,662::task::495::TaskManager.Task::(_debug) Task a2100830-05f0-4377-add4-d1f6895dab03: ref 0 aborting False
>Thread-15868524::INFO::2013-11-19 21:01:22,663::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15910006::INFO::2013-11-19 21:01:22,915::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910006::DEBUG::2013-11-19 21:01:22,916::task::495::TaskManager.Task::(_debug) Task 3106f792-1c67-4842-8057-2e9bc26517f9: moving from state init -> state preparing
>Thread-15910006::DEBUG::2013-11-19 21:01:22,917::task::495::TaskManager.Task::(_debug) Task 3106f792-1c67-4842-8057-2e9bc26517f9: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000410079956055', 'lastCheck': 1384894876.0406599, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00194406509399', 'lastCheck': 1384894882.6341989, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384894877.6248951, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000679016113281', 'lastCheck': 1384894874.5350571, 'valid': True, 'code': 0}}
>Thread-15910006::DEBUG::2013-11-19 21:01:22,918::task::495::TaskManager.Task::(_debug) Task 3106f792-1c67-4842-8057-2e9bc26517f9: moving from state preparing -> state finished
>Thread-15910006::DEBUG::2013-11-19 21:01:22,919::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910006::DEBUG::2013-11-19 21:01:22,919::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910006::DEBUG::2013-11-19 21:01:22,920::task::495::TaskManager.Task::(_debug) Task 3106f792-1c67-4842-8057-2e9bc26517f9: ref 0 aborting False
>Thread-15910006::INFO::2013-11-19 21:01:22,920::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000410079956055', 'lastCheck': 1384894876.0406599, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00194406509399', 'lastCheck': 1384894882.6341989, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384894877.6248951, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000679016113281', 'lastCheck': 1384894874.5350571, 'valid': True, 'code': 0}}
>Dummy-15909965::DEBUG::2013-11-19 21:01:24,305::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:01:24,874::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0715697 s, 14.3 MB/s\n'; <rc> = 0
>Dummy-15909965::DEBUG::2013-11-19 21:01:26,889::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:01:27,399::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0796909 s, 12.8 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:01:27,639::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15909965::DEBUG::2013-11-19 21:01:29,415::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:01:30,014::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0822195 s, 12.5 MB/s\n'; <rc> = 0
>Dummy-15909965::DEBUG::2013-11-19 21:01:32,024::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:01:32,540::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0831492 s, 12.3 MB/s\n'; <rc> = 0
>Thread-15910012::DEBUG::2013-11-19 21:01:32,789::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910012::INFO::2013-11-19 21:01:32,792::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910012::DEBUG::2013-11-19 21:01:32,792::task::495::TaskManager.Task::(_debug) Task df93d138-ce77-4017-902e-7f2033781f67: moving from state init -> state preparing
>Thread-15910012::DEBUG::2013-11-19 21:01:32,793::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=274 spmId=2
>Thread-15910012::DEBUG::2013-11-19 21:01:32,794::task::495::TaskManager.Task::(_debug) Task df93d138-ce77-4017-902e-7f2033781f67: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910012::DEBUG::2013-11-19 21:01:32,794::task::495::TaskManager.Task::(_debug) Task df93d138-ce77-4017-902e-7f2033781f67: moving from state preparing -> state finished
>Thread-15910012::DEBUG::2013-11-19 21:01:32,795::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910012::DEBUG::2013-11-19 21:01:32,795::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910012::DEBUG::2013-11-19 21:01:32,796::task::495::TaskManager.Task::(_debug) Task df93d138-ce77-4017-902e-7f2033781f67: ref 0 aborting False
>Thread-15910012::INFO::2013-11-19 21:01:32,796::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910013::DEBUG::2013-11-19 21:01:32,801::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910013::INFO::2013-11-19 21:01:32,802::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910013::DEBUG::2013-11-19 21:01:32,803::task::495::TaskManager.Task::(_debug) Task 386691f5-daa8-49dc-b888-f0cd8c4fc3f3: moving from state init -> state preparing
>Thread-15910013::DEBUG::2013-11-19 21:01:32,804::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`033e88b2-9ecc-4e27-93e9-fe4ccd906126`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910013::DEBUG::2013-11-19 21:01:32,805::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910013::DEBUG::2013-11-19 21:01:32,805::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910013::DEBUG::2013-11-19 21:01:32,806::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`033e88b2-9ecc-4e27-93e9-fe4ccd906126`::Granted request
>Thread-15910013::DEBUG::2013-11-19 21:01:32,807::task::495::TaskManager.Task::(_debug) Task 386691f5-daa8-49dc-b888-f0cd8c4fc3f3: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910013::DEBUG::2013-11-19 21:01:32,807::task::495::TaskManager.Task::(_debug) Task 386691f5-daa8-49dc-b888-f0cd8c4fc3f3: ref 1 aborting False
>Thread-15910013::WARNING::2013-11-19 21:01:32,809::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910013::DEBUG::2013-11-19 21:01:32,810::task::495::TaskManager.Task::(_debug) Task 386691f5-daa8-49dc-b888-f0cd8c4fc3f3: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910013::DEBUG::2013-11-19 21:01:32,811::task::495::TaskManager.Task::(_debug) Task 386691f5-daa8-49dc-b888-f0cd8c4fc3f3: moving from state preparing -> state finished
>Thread-15910013::DEBUG::2013-11-19 21:01:32,812::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910013::DEBUG::2013-11-19 21:01:32,812::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910013::DEBUG::2013-11-19 21:01:32,813::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910013::DEBUG::2013-11-19 21:01:32,813::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910013::DEBUG::2013-11-19 21:01:32,814::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910013::DEBUG::2013-11-19 21:01:32,814::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910013::DEBUG::2013-11-19 21:01:32,815::task::495::TaskManager.Task::(_debug) Task 386691f5-daa8-49dc-b888-f0cd8c4fc3f3: ref 0 aborting False
>Thread-15910013::INFO::2013-11-19 21:01:32,816::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910014::INFO::2013-11-19 21:01:33,193::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910014::DEBUG::2013-11-19 21:01:33,193::task::495::TaskManager.Task::(_debug) Task 9576e14f-d870-49b0-9d7a-fbf1513bb616: moving from state init -> state preparing
>Thread-15910014::DEBUG::2013-11-19 21:01:33,194::task::495::TaskManager.Task::(_debug) Task 9576e14f-d870-49b0-9d7a-fbf1513bb616: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000364065170288', 'lastCheck': 1384894886.2032509, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00187611579895', 'lastCheck': 1384894892.6852951, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384894887.640872, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000503063201904', 'lastCheck': 1384894884.6285191, 'valid': True, 'code': 0}}
>Thread-15910014::DEBUG::2013-11-19 21:01:33,195::task::495::TaskManager.Task::(_debug) Task 9576e14f-d870-49b0-9d7a-fbf1513bb616: moving from state preparing -> state finished
>Thread-15910014::DEBUG::2013-11-19 21:01:33,195::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910014::DEBUG::2013-11-19 21:01:33,196::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910014::DEBUG::2013-11-19 21:01:33,197::task::495::TaskManager.Task::(_debug) Task 9576e14f-d870-49b0-9d7a-fbf1513bb616: ref 0 aborting False
>Thread-15910014::INFO::2013-11-19 21:01:33,198::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000364065170288', 'lastCheck': 1384894886.2032509, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00187611579895', 'lastCheck': 1384894892.6852951, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384894887.640872, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000503063201904', 'lastCheck': 1384894884.6285191, 'valid': True, 'code': 0}}
>Dummy-15909965::DEBUG::2013-11-19 21:01:34,556::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:01:35,123::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0795763 s, 12.9 MB/s\n'; <rc> = 0
>Thread-4922912::DEBUG::2013-11-19 21:01:36,722::guestIF::97::vm.Vm::(_handleMessage) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::u'BRD-KLEE\\Administrator'
>Dummy-15909965::DEBUG::2013-11-19 21:01:37,138::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:01:37,710::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0857649 s, 11.9 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:01:37,790::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15909965::DEBUG::2013-11-19 21:01:39,726::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:01:40,211::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0474575 s, 21.6 MB/s\n'; <rc> = 0
>Thread-15909534::INFO::2013-11-19 21:01:40,233::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15909534::DEBUG::2013-11-19 21:01:40,234::task::495::TaskManager.Task::(_debug) Task 93216059-81a6-473b-901d-9c2dda12ef29: moving from state init -> state preparing
>Thread-15909534::DEBUG::2013-11-19 21:01:40,236::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`aefbb1e0-5b34-4a18-8ad4-950ccf811034`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15909534::DEBUG::2013-11-19 21:01:40,236::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15909534::DEBUG::2013-11-19 21:01:40,237::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15909534::DEBUG::2013-11-19 21:01:40,238::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`aefbb1e0-5b34-4a18-8ad4-950ccf811034`::Granted request
>Thread-15909534::DEBUG::2013-11-19 21:01:40,239::task::495::TaskManager.Task::(_debug) Task 93216059-81a6-473b-901d-9c2dda12ef29: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15909534::DEBUG::2013-11-19 21:01:40,240::task::495::TaskManager.Task::(_debug) Task 93216059-81a6-473b-901d-9c2dda12ef29: ref 1 aborting False
>Thread-15909534::DEBUG::2013-11-19 21:01:40,241::task::495::TaskManager.Task::(_debug) Task 93216059-81a6-473b-901d-9c2dda12ef29: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15909534::DEBUG::2013-11-19 21:01:40,241::task::495::TaskManager.Task::(_debug) Task 93216059-81a6-473b-901d-9c2dda12ef29: moving from state preparing -> state finished
>Thread-15909534::DEBUG::2013-11-19 21:01:40,242::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15909534::DEBUG::2013-11-19 21:01:40,243::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15909534::DEBUG::2013-11-19 21:01:40,244::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15909534::DEBUG::2013-11-19 21:01:40,244::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15909534::DEBUG::2013-11-19 21:01:40,245::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15909534::DEBUG::2013-11-19 21:01:40,245::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15909534::DEBUG::2013-11-19 21:01:40,246::task::495::TaskManager.Task::(_debug) Task 93216059-81a6-473b-901d-9c2dda12ef29: ref 0 aborting False
>Thread-15909534::INFO::2013-11-19 21:01:40,247::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15909965::DEBUG::2013-11-19 21:01:42,221::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:01:42,781::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0753163 s, 13.6 MB/s\n'; <rc> = 0
>Thread-15910020::DEBUG::2013-11-19 21:01:42,932::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910020::INFO::2013-11-19 21:01:42,933::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910020::DEBUG::2013-11-19 21:01:42,933::task::495::TaskManager.Task::(_debug) Task efc1a099-4439-4c60-861c-bc0969c313f3: moving from state init -> state preparing
>Thread-15910020::DEBUG::2013-11-19 21:01:42,934::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=274 spmId=2
>Thread-15910020::DEBUG::2013-11-19 21:01:42,934::task::495::TaskManager.Task::(_debug) Task efc1a099-4439-4c60-861c-bc0969c313f3: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910020::DEBUG::2013-11-19 21:01:42,934::task::495::TaskManager.Task::(_debug) Task efc1a099-4439-4c60-861c-bc0969c313f3: moving from state preparing -> state finished
>Thread-15910020::DEBUG::2013-11-19 21:01:42,935::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910020::DEBUG::2013-11-19 21:01:42,935::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910020::DEBUG::2013-11-19 21:01:42,935::task::495::TaskManager.Task::(_debug) Task efc1a099-4439-4c60-861c-bc0969c313f3: ref 0 aborting False
>Thread-15910020::INFO::2013-11-19 21:01:42,936::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910021::DEBUG::2013-11-19 21:01:42,940::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910021::INFO::2013-11-19 21:01:42,941::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910021::DEBUG::2013-11-19 21:01:42,941::task::495::TaskManager.Task::(_debug) Task dc12650e-2b17-45bc-b50e-d430b33f713c: moving from state init -> state preparing
>Thread-15910021::DEBUG::2013-11-19 21:01:42,943::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e4745502-798a-4569-a3d2-8037c6d095a4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910021::DEBUG::2013-11-19 21:01:42,943::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910021::DEBUG::2013-11-19 21:01:42,944::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910021::DEBUG::2013-11-19 21:01:42,945::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e4745502-798a-4569-a3d2-8037c6d095a4`::Granted request
>Thread-15910021::DEBUG::2013-11-19 21:01:42,946::task::495::TaskManager.Task::(_debug) Task dc12650e-2b17-45bc-b50e-d430b33f713c: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910021::DEBUG::2013-11-19 21:01:42,946::task::495::TaskManager.Task::(_debug) Task dc12650e-2b17-45bc-b50e-d430b33f713c: ref 1 aborting False
>Thread-15910021::WARNING::2013-11-19 21:01:42,948::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910021::DEBUG::2013-11-19 21:01:42,949::task::495::TaskManager.Task::(_debug) Task dc12650e-2b17-45bc-b50e-d430b33f713c: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910021::DEBUG::2013-11-19 21:01:42,950::task::495::TaskManager.Task::(_debug) Task dc12650e-2b17-45bc-b50e-d430b33f713c: moving from state preparing -> state finished
>Thread-15910021::DEBUG::2013-11-19 21:01:42,951::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910021::DEBUG::2013-11-19 21:01:42,951::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910021::DEBUG::2013-11-19 21:01:42,952::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910021::DEBUG::2013-11-19 21:01:42,953::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910021::DEBUG::2013-11-19 21:01:42,953::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910021::DEBUG::2013-11-19 21:01:42,954::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910021::DEBUG::2013-11-19 21:01:42,954::task::495::TaskManager.Task::(_debug) Task dc12650e-2b17-45bc-b50e-d430b33f713c: ref 0 aborting False
>Thread-15910021::INFO::2013-11-19 21:01:42,955::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910022::INFO::2013-11-19 21:01:43,409::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910022::DEBUG::2013-11-19 21:01:43,410::task::495::TaskManager.Task::(_debug) Task ce249973-9c45-4bdf-8947-15a7324c08a7: moving from state init -> state preparing
>Thread-15910022::DEBUG::2013-11-19 21:01:43,410::task::495::TaskManager.Task::(_debug) Task ce249973-9c45-4bdf-8947-15a7324c08a7: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000366926193237', 'lastCheck': 1384894896.230176, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0010130405426', 'lastCheck': 1384894902.931289, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384894897.791348, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000730037689209', 'lastCheck': 1384894894.7851319, 'valid': True, 'code': 0}}
>Thread-15910022::DEBUG::2013-11-19 21:01:43,411::task::495::TaskManager.Task::(_debug) Task ce249973-9c45-4bdf-8947-15a7324c08a7: moving from state preparing -> state finished
>Thread-15910022::DEBUG::2013-11-19 21:01:43,411::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910022::DEBUG::2013-11-19 21:01:43,411::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910022::DEBUG::2013-11-19 21:01:43,412::task::495::TaskManager.Task::(_debug) Task ce249973-9c45-4bdf-8947-15a7324c08a7: ref 0 aborting False
>Thread-15910022::INFO::2013-11-19 21:01:43,412::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000366926193237', 'lastCheck': 1384894896.230176, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0010130405426', 'lastCheck': 1384894902.931289, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384894897.791348, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000730037689209', 'lastCheck': 1384894894.7851319, 'valid': True, 'code': 0}}
>Dummy-15909965::DEBUG::2013-11-19 21:01:44,798::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:01:45,316::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0788518 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15909965::DEBUG::2013-11-19 21:01:47,332::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:01:47,918::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.12047 s, 8.5 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:01:48,015::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15909965::DEBUG::2013-11-19 21:01:49,935::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:01:50,539::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0804068 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15909965::DEBUG::2013-11-19 21:01:52,559::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:01:53,103::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0793667 s, 12.9 MB/s\n'; <rc> = 0
>Thread-15910028::DEBUG::2013-11-19 21:01:53,189::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910028::INFO::2013-11-19 21:01:53,191::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910028::DEBUG::2013-11-19 21:01:53,191::task::495::TaskManager.Task::(_debug) Task a29ad9fe-2b4d-4560-b678-a7c8ddd2c95e: moving from state init -> state preparing
>Thread-15910028::DEBUG::2013-11-19 21:01:53,192::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=274 spmId=2
>Thread-15910028::DEBUG::2013-11-19 21:01:53,192::task::495::TaskManager.Task::(_debug) Task a29ad9fe-2b4d-4560-b678-a7c8ddd2c95e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910028::DEBUG::2013-11-19 21:01:53,193::task::495::TaskManager.Task::(_debug) Task a29ad9fe-2b4d-4560-b678-a7c8ddd2c95e: moving from state preparing -> state finished
>Thread-15910028::DEBUG::2013-11-19 21:01:53,194::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910028::DEBUG::2013-11-19 21:01:53,194::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910028::DEBUG::2013-11-19 21:01:53,195::task::495::TaskManager.Task::(_debug) Task a29ad9fe-2b4d-4560-b678-a7c8ddd2c95e: ref 0 aborting False
>Thread-15910028::INFO::2013-11-19 21:01:53,195::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910029::DEBUG::2013-11-19 21:01:53,200::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910029::INFO::2013-11-19 21:01:53,201::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910029::DEBUG::2013-11-19 21:01:53,202::task::495::TaskManager.Task::(_debug) Task c1921c03-3c39-4aaf-bfb6-7e3fc78d7b26: moving from state init -> state preparing
>Thread-15910029::DEBUG::2013-11-19 21:01:53,203::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ab305262-e39a-4e0d-a11b-4d1165aa9184`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910029::DEBUG::2013-11-19 21:01:53,204::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910029::DEBUG::2013-11-19 21:01:53,205::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910029::DEBUG::2013-11-19 21:01:53,205::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ab305262-e39a-4e0d-a11b-4d1165aa9184`::Granted request
>Thread-15910029::DEBUG::2013-11-19 21:01:53,206::task::495::TaskManager.Task::(_debug) Task c1921c03-3c39-4aaf-bfb6-7e3fc78d7b26: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910029::DEBUG::2013-11-19 21:01:53,207::task::495::TaskManager.Task::(_debug) Task c1921c03-3c39-4aaf-bfb6-7e3fc78d7b26: ref 1 aborting False
>Thread-15910029::WARNING::2013-11-19 21:01:53,209::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910029::DEBUG::2013-11-19 21:01:53,210::task::495::TaskManager.Task::(_debug) Task c1921c03-3c39-4aaf-bfb6-7e3fc78d7b26: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910029::DEBUG::2013-11-19 21:01:53,211::task::495::TaskManager.Task::(_debug) Task c1921c03-3c39-4aaf-bfb6-7e3fc78d7b26: moving from state preparing -> state finished
>Thread-15910029::DEBUG::2013-11-19 21:01:53,211::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910029::DEBUG::2013-11-19 21:01:53,212::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910029::DEBUG::2013-11-19 21:01:53,213::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910029::DEBUG::2013-11-19 21:01:53,213::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910029::DEBUG::2013-11-19 21:01:53,214::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910029::DEBUG::2013-11-19 21:01:53,214::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910029::DEBUG::2013-11-19 21:01:53,215::task::495::TaskManager.Task::(_debug) Task c1921c03-3c39-4aaf-bfb6-7e3fc78d7b26: ref 0 aborting False
>Thread-15910029::INFO::2013-11-19 21:01:53,216::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910030::INFO::2013-11-19 21:01:53,602::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910030::DEBUG::2013-11-19 21:01:53,603::task::495::TaskManager.Task::(_debug) Task dd39d70b-2b6b-4dd4-82dd-32c0b30bd2e3: moving from state init -> state preparing
>Thread-15910030::DEBUG::2013-11-19 21:01:53,604::task::495::TaskManager.Task::(_debug) Task dd39d70b-2b6b-4dd4-82dd-32c0b30bd2e3: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000334024429321', 'lastCheck': 1384894906.4503391, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00132894515991', 'lastCheck': 1384894913.170774, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384894908.0167711, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00113487243652', 'lastCheck': 1384894905.011519, 'valid': True, 'code': 0}}
>Thread-15910030::DEBUG::2013-11-19 21:01:53,605::task::495::TaskManager.Task::(_debug) Task dd39d70b-2b6b-4dd4-82dd-32c0b30bd2e3: moving from state preparing -> state finished
>Thread-15910030::DEBUG::2013-11-19 21:01:53,605::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910030::DEBUG::2013-11-19 21:01:53,606::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910030::DEBUG::2013-11-19 21:01:53,607::task::495::TaskManager.Task::(_debug) Task dd39d70b-2b6b-4dd4-82dd-32c0b30bd2e3: ref 0 aborting False
>Thread-15910030::INFO::2013-11-19 21:01:53,607::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000334024429321', 'lastCheck': 1384894906.4503391, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00132894515991', 'lastCheck': 1384894913.170774, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384894908.0167711, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00113487243652', 'lastCheck': 1384894905.011519, 'valid': True, 'code': 0}}
>Dummy-15909965::DEBUG::2013-11-19 21:01:55,119::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-42840::DEBUG::2013-11-19 21:01:55,329::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:01:55,806::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.086326 s, 11.9 MB/s\n'; <rc> = 0
>Thread-42840::DEBUG::2013-11-19 21:01:56,100::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15909965::DEBUG::2013-11-19 21:01:57,822::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:01:58,337::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15909965::DEBUG::2013-11-19 21:01:58,366::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0712587 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15909965::DEBUG::2013-11-19 21:02:00,381::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:02:00,919::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0780567 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15909965::DEBUG::2013-11-19 21:02:02,935::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:02:03,488::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0758178 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15910036::DEBUG::2013-11-19 21:02:03,663::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910036::INFO::2013-11-19 21:02:03,664::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910036::DEBUG::2013-11-19 21:02:03,665::task::495::TaskManager.Task::(_debug) Task f5b6fd4c-f6b4-4bcc-83e1-4d472d56bed1: moving from state init -> state preparing
>Thread-15910036::DEBUG::2013-11-19 21:02:03,666::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=274 spmId=2
>Thread-15910036::DEBUG::2013-11-19 21:02:03,666::task::495::TaskManager.Task::(_debug) Task f5b6fd4c-f6b4-4bcc-83e1-4d472d56bed1: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910036::DEBUG::2013-11-19 21:02:03,667::task::495::TaskManager.Task::(_debug) Task f5b6fd4c-f6b4-4bcc-83e1-4d472d56bed1: moving from state preparing -> state finished
>Thread-15910036::DEBUG::2013-11-19 21:02:03,667::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910036::DEBUG::2013-11-19 21:02:03,668::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910036::DEBUG::2013-11-19 21:02:03,668::task::495::TaskManager.Task::(_debug) Task f5b6fd4c-f6b4-4bcc-83e1-4d472d56bed1: ref 0 aborting False
>Thread-15910036::INFO::2013-11-19 21:02:03,669::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910037::DEBUG::2013-11-19 21:02:03,674::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910037::INFO::2013-11-19 21:02:03,674::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910037::DEBUG::2013-11-19 21:02:03,675::task::495::TaskManager.Task::(_debug) Task 59ef5f50-a15e-4e13-ba6f-80bec36a97e5: moving from state init -> state preparing
>Thread-15910037::DEBUG::2013-11-19 21:02:03,676::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`239f6a66-d27b-4a30-ad74-75bfa6108018`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910037::DEBUG::2013-11-19 21:02:03,677::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910037::DEBUG::2013-11-19 21:02:03,678::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910037::DEBUG::2013-11-19 21:02:03,678::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`239f6a66-d27b-4a30-ad74-75bfa6108018`::Granted request
>Thread-15910037::DEBUG::2013-11-19 21:02:03,679::task::495::TaskManager.Task::(_debug) Task 59ef5f50-a15e-4e13-ba6f-80bec36a97e5: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910037::DEBUG::2013-11-19 21:02:03,680::task::495::TaskManager.Task::(_debug) Task 59ef5f50-a15e-4e13-ba6f-80bec36a97e5: ref 1 aborting False
>Thread-15910037::WARNING::2013-11-19 21:02:03,682::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910037::DEBUG::2013-11-19 21:02:03,683::task::495::TaskManager.Task::(_debug) Task 59ef5f50-a15e-4e13-ba6f-80bec36a97e5: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910037::DEBUG::2013-11-19 21:02:03,684::task::495::TaskManager.Task::(_debug) Task 59ef5f50-a15e-4e13-ba6f-80bec36a97e5: moving from state preparing -> state finished
>Thread-15910037::DEBUG::2013-11-19 21:02:03,684::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910037::DEBUG::2013-11-19 21:02:03,685::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910037::DEBUG::2013-11-19 21:02:03,686::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910037::DEBUG::2013-11-19 21:02:03,687::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910037::DEBUG::2013-11-19 21:02:03,687::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910037::DEBUG::2013-11-19 21:02:03,688::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910037::DEBUG::2013-11-19 21:02:03,689::task::495::TaskManager.Task::(_debug) Task 59ef5f50-a15e-4e13-ba6f-80bec36a97e5: ref 0 aborting False
>Thread-15910037::INFO::2013-11-19 21:02:03,689::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910038::INFO::2013-11-19 21:02:04,133::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910038::DEBUG::2013-11-19 21:02:04,134::task::495::TaskManager.Task::(_debug) Task f64ca708-9ea8-4ab6-b682-7fc58ed19260: moving from state init -> state preparing
>Thread-15910038::DEBUG::2013-11-19 21:02:04,134::task::495::TaskManager.Task::(_debug) Task f64ca708-9ea8-4ab6-b682-7fc58ed19260: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000420808792114', 'lastCheck': 1384894916.673322, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00155305862427', 'lastCheck': 1384894923.343523, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384894918.3400691, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00031304359436', 'lastCheck': 1384894916.1024711, 'valid': True, 'code': 0}}
>Thread-15910038::DEBUG::2013-11-19 21:02:04,135::task::495::TaskManager.Task::(_debug) Task f64ca708-9ea8-4ab6-b682-7fc58ed19260: moving from state preparing -> state finished
>Thread-15910038::DEBUG::2013-11-19 21:02:04,136::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910038::DEBUG::2013-11-19 21:02:04,136::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910038::DEBUG::2013-11-19 21:02:04,136::task::495::TaskManager.Task::(_debug) Task f64ca708-9ea8-4ab6-b682-7fc58ed19260: ref 0 aborting False
>Thread-15910038::INFO::2013-11-19 21:02:04,137::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000420808792114', 'lastCheck': 1384894916.673322, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00155305862427', 'lastCheck': 1384894923.343523, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384894918.3400691, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00031304359436', 'lastCheck': 1384894916.1024711, 'valid': True, 'code': 0}}
>Dummy-15909965::DEBUG::2013-11-19 21:02:05,502::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:02:05,936::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0474992 s, 21.6 MB/s\n'; <rc> = 0
>Dummy-15909965::DEBUG::2013-11-19 21:02:07,951::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:02:08,353::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15909965::DEBUG::2013-11-19 21:02:08,478::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0803388 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15909965::DEBUG::2013-11-19 21:02:10,493::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:02:11,054::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.088026 s, 11.6 MB/s\n'; <rc> = 0
>Dummy-15909965::DEBUG::2013-11-19 21:02:13,072::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:02:13,588::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0767494 s, 13.3 MB/s\n'; <rc> = 0
>Thread-15910044::DEBUG::2013-11-19 21:02:13,789::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910044::INFO::2013-11-19 21:02:13,791::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910044::DEBUG::2013-11-19 21:02:13,791::task::495::TaskManager.Task::(_debug) Task 15d007fe-ee35-428e-b8e3-748f4c8cfc06: moving from state init -> state preparing
>Thread-15910044::DEBUG::2013-11-19 21:02:13,792::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=274 spmId=2
>Thread-15910044::DEBUG::2013-11-19 21:02:13,793::task::495::TaskManager.Task::(_debug) Task 15d007fe-ee35-428e-b8e3-748f4c8cfc06: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910044::DEBUG::2013-11-19 21:02:13,794::task::495::TaskManager.Task::(_debug) Task 15d007fe-ee35-428e-b8e3-748f4c8cfc06: moving from state preparing -> state finished
>Thread-15910044::DEBUG::2013-11-19 21:02:13,795::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910044::DEBUG::2013-11-19 21:02:13,796::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910044::DEBUG::2013-11-19 21:02:13,796::task::495::TaskManager.Task::(_debug) Task 15d007fe-ee35-428e-b8e3-748f4c8cfc06: ref 0 aborting False
>Thread-15910044::INFO::2013-11-19 21:02:13,797::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910045::DEBUG::2013-11-19 21:02:13,802::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910045::INFO::2013-11-19 21:02:13,803::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910045::DEBUG::2013-11-19 21:02:13,803::task::495::TaskManager.Task::(_debug) Task ae4a902b-0b1c-45ce-bc7a-c060a36e5290: moving from state init -> state preparing
>Thread-15910045::DEBUG::2013-11-19 21:02:13,805::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`328d4193-1369-4730-b0bf-36ded5d047c8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910045::DEBUG::2013-11-19 21:02:13,805::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910045::DEBUG::2013-11-19 21:02:13,806::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910045::DEBUG::2013-11-19 21:02:13,807::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`328d4193-1369-4730-b0bf-36ded5d047c8`::Granted request
>Thread-15910045::DEBUG::2013-11-19 21:02:13,808::task::495::TaskManager.Task::(_debug) Task ae4a902b-0b1c-45ce-bc7a-c060a36e5290: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910045::DEBUG::2013-11-19 21:02:13,808::task::495::TaskManager.Task::(_debug) Task ae4a902b-0b1c-45ce-bc7a-c060a36e5290: ref 1 aborting False
>Thread-15910045::WARNING::2013-11-19 21:02:13,810::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910045::DEBUG::2013-11-19 21:02:13,811::task::495::TaskManager.Task::(_debug) Task ae4a902b-0b1c-45ce-bc7a-c060a36e5290: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910045::DEBUG::2013-11-19 21:02:13,812::task::495::TaskManager.Task::(_debug) Task ae4a902b-0b1c-45ce-bc7a-c060a36e5290: moving from state preparing -> state finished
>Thread-15910045::DEBUG::2013-11-19 21:02:13,813::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910045::DEBUG::2013-11-19 21:02:13,813::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910045::DEBUG::2013-11-19 21:02:13,814::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910045::DEBUG::2013-11-19 21:02:13,815::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910045::DEBUG::2013-11-19 21:02:13,815::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910045::DEBUG::2013-11-19 21:02:13,816::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910045::DEBUG::2013-11-19 21:02:13,816::task::495::TaskManager.Task::(_debug) Task ae4a902b-0b1c-45ce-bc7a-c060a36e5290: ref 0 aborting False
>Thread-15910045::INFO::2013-11-19 21:02:13,817::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910046::INFO::2013-11-19 21:02:14,283::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910046::DEBUG::2013-11-19 21:02:14,284::task::495::TaskManager.Task::(_debug) Task 3529a14c-9fc4-41fe-8924-0fab27f50e1c: moving from state init -> state preparing
>Thread-15910046::DEBUG::2013-11-19 21:02:14,284::task::495::TaskManager.Task::(_debug) Task 3529a14c-9fc4-41fe-8924-0fab27f50e1c: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000343084335327', 'lastCheck': 1384894926.70735, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00137591362', 'lastCheck': 1384894933.360806, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384894928.354538, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000740051269531', 'lastCheck': 1384894926.1610529, 'valid': True, 'code': 0}}
>Thread-15910046::DEBUG::2013-11-19 21:02:14,285::task::495::TaskManager.Task::(_debug) Task 3529a14c-9fc4-41fe-8924-0fab27f50e1c: moving from state preparing -> state finished
>Thread-15910046::DEBUG::2013-11-19 21:02:14,286::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910046::DEBUG::2013-11-19 21:02:14,286::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910046::DEBUG::2013-11-19 21:02:14,287::task::495::TaskManager.Task::(_debug) Task 3529a14c-9fc4-41fe-8924-0fab27f50e1c: ref 0 aborting False
>Thread-15910046::INFO::2013-11-19 21:02:14,288::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000343084335327', 'lastCheck': 1384894926.70735, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00137591362', 'lastCheck': 1384894933.360806, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384894928.354538, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000740051269531', 'lastCheck': 1384894926.1610529, 'valid': True, 'code': 0}}
>Dummy-15909965::DEBUG::2013-11-19 21:02:15,603::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:02:16,136::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0752066 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15909965::DEBUG::2013-11-19 21:02:18,151::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:02:18,394::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15909965::DEBUG::2013-11-19 21:02:18,606::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0455872 s, 22.5 MB/s\n'; <rc> = 0
>Dummy-15909965::DEBUG::2013-11-19 21:02:20,616::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:02:21,142::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0827538 s, 12.4 MB/s\n'; <rc> = 0
>Thread-4922911::INFO::2013-11-19 21:02:22,838::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-4922911::DEBUG::2013-11-19 21:02:22,839::task::495::TaskManager.Task::(_debug) Task 8210e323-d7c1-4275-a9f2-e9de178208a9: moving from state init -> state preparing
>Thread-4922911::DEBUG::2013-11-19 21:02:22,840::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`087fca4a-3fdf-497e-ae98-ed3d823d11b6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-4922911::DEBUG::2013-11-19 21:02:22,841::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-4922911::DEBUG::2013-11-19 21:02:22,842::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-4922911::DEBUG::2013-11-19 21:02:22,842::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`087fca4a-3fdf-497e-ae98-ed3d823d11b6`::Granted request
>Thread-4922911::DEBUG::2013-11-19 21:02:22,843::task::495::TaskManager.Task::(_debug) Task 8210e323-d7c1-4275-a9f2-e9de178208a9: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-4922911::DEBUG::2013-11-19 21:02:22,844::task::495::TaskManager.Task::(_debug) Task 8210e323-d7c1-4275-a9f2-e9de178208a9: ref 1 aborting False
>Thread-4922911::DEBUG::2013-11-19 21:02:22,845::task::495::TaskManager.Task::(_debug) Task 8210e323-d7c1-4275-a9f2-e9de178208a9: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-4922911::DEBUG::2013-11-19 21:02:22,845::task::495::TaskManager.Task::(_debug) Task 8210e323-d7c1-4275-a9f2-e9de178208a9: moving from state preparing -> state finished
>Thread-4922911::DEBUG::2013-11-19 21:02:22,846::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-4922911::DEBUG::2013-11-19 21:02:22,846::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-4922911::DEBUG::2013-11-19 21:02:22,847::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-4922911::DEBUG::2013-11-19 21:02:22,847::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-4922911::DEBUG::2013-11-19 21:02:22,848::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-4922911::DEBUG::2013-11-19 21:02:22,848::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-4922911::DEBUG::2013-11-19 21:02:22,849::task::495::TaskManager.Task::(_debug) Task 8210e323-d7c1-4275-a9f2-e9de178208a9: ref 0 aborting False
>Thread-4922911::INFO::2013-11-19 21:02:22,850::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-4922911::INFO::2013-11-19 21:02:22,850::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-4922911::DEBUG::2013-11-19 21:02:22,851::task::495::TaskManager.Task::(_debug) Task c42eb4e8-df32-4c13-8338-a6b197446d10: moving from state init -> state preparing
>Thread-4922911::DEBUG::2013-11-19 21:02:22,852::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d66e814d-59a7-492c-8e10-8997e272fce1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-4922911::DEBUG::2013-11-19 21:02:22,852::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-4922911::DEBUG::2013-11-19 21:02:22,853::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-4922911::DEBUG::2013-11-19 21:02:22,853::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d66e814d-59a7-492c-8e10-8997e272fce1`::Granted request
>Thread-4922911::DEBUG::2013-11-19 21:02:22,854::task::495::TaskManager.Task::(_debug) Task c42eb4e8-df32-4c13-8338-a6b197446d10: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-4922911::DEBUG::2013-11-19 21:02:22,855::task::495::TaskManager.Task::(_debug) Task c42eb4e8-df32-4c13-8338-a6b197446d10: ref 1 aborting False
>Thread-4922911::DEBUG::2013-11-19 21:02:22,855::task::495::TaskManager.Task::(_debug) Task c42eb4e8-df32-4c13-8338-a6b197446d10: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-4922911::DEBUG::2013-11-19 21:02:22,856::task::495::TaskManager.Task::(_debug) Task c42eb4e8-df32-4c13-8338-a6b197446d10: moving from state preparing -> state finished
>Thread-4922911::DEBUG::2013-11-19 21:02:22,856::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-4922911::DEBUG::2013-11-19 21:02:22,857::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-4922911::DEBUG::2013-11-19 21:02:22,858::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-4922911::DEBUG::2013-11-19 21:02:22,858::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-4922911::DEBUG::2013-11-19 21:02:22,859::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-4922911::DEBUG::2013-11-19 21:02:22,859::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-4922911::DEBUG::2013-11-19 21:02:22,860::task::495::TaskManager.Task::(_debug) Task c42eb4e8-df32-4c13-8338-a6b197446d10: ref 0 aborting False
>Thread-4922911::INFO::2013-11-19 21:02:22,860::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-4922911::INFO::2013-11-19 21:02:22,861::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-4922911::DEBUG::2013-11-19 21:02:22,862::task::495::TaskManager.Task::(_debug) Task 74bc7a8f-e2d8-4598-b287-06594999dc50: moving from state init -> state preparing
>Thread-4922911::DEBUG::2013-11-19 21:02:22,863::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e31dccc7-daad-4ab2-bd70-106c81ba36b8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-4922911::DEBUG::2013-11-19 21:02:22,863::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-4922911::DEBUG::2013-11-19 21:02:22,864::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-4922911::DEBUG::2013-11-19 21:02:22,864::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e31dccc7-daad-4ab2-bd70-106c81ba36b8`::Granted request
>Thread-4922911::DEBUG::2013-11-19 21:02:22,865::task::495::TaskManager.Task::(_debug) Task 74bc7a8f-e2d8-4598-b287-06594999dc50: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-4922911::DEBUG::2013-11-19 21:02:22,866::task::495::TaskManager.Task::(_debug) Task 74bc7a8f-e2d8-4598-b287-06594999dc50: ref 1 aborting False
>Thread-4922911::DEBUG::2013-11-19 21:02:22,866::task::495::TaskManager.Task::(_debug) Task 74bc7a8f-e2d8-4598-b287-06594999dc50: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-4922911::DEBUG::2013-11-19 21:02:22,867::task::495::TaskManager.Task::(_debug) Task 74bc7a8f-e2d8-4598-b287-06594999dc50: moving from state preparing -> state finished
>Thread-4922911::DEBUG::2013-11-19 21:02:22,867::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-4922911::DEBUG::2013-11-19 21:02:22,868::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-4922911::DEBUG::2013-11-19 21:02:22,868::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-4922911::DEBUG::2013-11-19 21:02:22,869::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-4922911::DEBUG::2013-11-19 21:02:22,870::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-4922911::DEBUG::2013-11-19 21:02:22,870::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-4922911::DEBUG::2013-11-19 21:02:22,871::task::495::TaskManager.Task::(_debug) Task 74bc7a8f-e2d8-4598-b287-06594999dc50: ref 0 aborting False
>Thread-4922911::INFO::2013-11-19 21:02:22,871::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-4922911::INFO::2013-11-19 21:02:22,872::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-4922911::DEBUG::2013-11-19 21:02:22,873::task::495::TaskManager.Task::(_debug) Task 0e604aac-817f-4d4c-bebe-d3a3322605b3: moving from state init -> state preparing
>Thread-4922911::DEBUG::2013-11-19 21:02:22,874::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d5c203e5-c864-4f86-9866-95e57fc3e88e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-4922911::DEBUG::2013-11-19 21:02:22,874::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-4922911::DEBUG::2013-11-19 21:02:22,875::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-4922911::DEBUG::2013-11-19 21:02:22,875::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d5c203e5-c864-4f86-9866-95e57fc3e88e`::Granted request
>Thread-4922911::DEBUG::2013-11-19 21:02:22,876::task::495::TaskManager.Task::(_debug) Task 0e604aac-817f-4d4c-bebe-d3a3322605b3: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-4922911::DEBUG::2013-11-19 21:02:22,877::task::495::TaskManager.Task::(_debug) Task 0e604aac-817f-4d4c-bebe-d3a3322605b3: ref 1 aborting False
>Thread-4922911::DEBUG::2013-11-19 21:02:22,877::task::495::TaskManager.Task::(_debug) Task 0e604aac-817f-4d4c-bebe-d3a3322605b3: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-4922911::DEBUG::2013-11-19 21:02:22,878::task::495::TaskManager.Task::(_debug) Task 0e604aac-817f-4d4c-bebe-d3a3322605b3: moving from state preparing -> state finished
>Thread-4922911::DEBUG::2013-11-19 21:02:22,879::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-4922911::DEBUG::2013-11-19 21:02:22,879::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-4922911::DEBUG::2013-11-19 21:02:22,880::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-4922911::DEBUG::2013-11-19 21:02:22,880::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-4922911::DEBUG::2013-11-19 21:02:22,881::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-4922911::DEBUG::2013-11-19 21:02:22,881::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-4922911::DEBUG::2013-11-19 21:02:22,882::task::495::TaskManager.Task::(_debug) Task 0e604aac-817f-4d4c-bebe-d3a3322605b3: ref 0 aborting False
>Thread-4922911::INFO::2013-11-19 21:02:22,883::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15909965::DEBUG::2013-11-19 21:02:23,157::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15868524::INFO::2013-11-19 21:02:23,411::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15868524::DEBUG::2013-11-19 21:02:23,413::task::495::TaskManager.Task::(_debug) Task db0baf9b-0bc3-4c3f-b629-bb1c876e27f5: moving from state init -> state preparing
>Thread-15868524::DEBUG::2013-11-19 21:02:23,423::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`434ccaff-4288-4c11-89b5-545956124eb8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15868524::DEBUG::2013-11-19 21:02:23,424::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15868524::DEBUG::2013-11-19 21:02:23,425::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15868524::DEBUG::2013-11-19 21:02:23,425::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`434ccaff-4288-4c11-89b5-545956124eb8`::Granted request
>Thread-15868524::DEBUG::2013-11-19 21:02:23,427::task::495::TaskManager.Task::(_debug) Task db0baf9b-0bc3-4c3f-b629-bb1c876e27f5: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15868524::DEBUG::2013-11-19 21:02:23,428::task::495::TaskManager.Task::(_debug) Task db0baf9b-0bc3-4c3f-b629-bb1c876e27f5: ref 1 aborting False
>Thread-15868524::DEBUG::2013-11-19 21:02:23,429::task::495::TaskManager.Task::(_debug) Task db0baf9b-0bc3-4c3f-b629-bb1c876e27f5: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15868524::DEBUG::2013-11-19 21:02:23,429::task::495::TaskManager.Task::(_debug) Task db0baf9b-0bc3-4c3f-b629-bb1c876e27f5: moving from state preparing -> state finished
>Thread-15868524::DEBUG::2013-11-19 21:02:23,430::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15868524::DEBUG::2013-11-19 21:02:23,431::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15868524::DEBUG::2013-11-19 21:02:23,432::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15868524::DEBUG::2013-11-19 21:02:23,432::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15868524::DEBUG::2013-11-19 21:02:23,433::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15868524::DEBUG::2013-11-19 21:02:23,433::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15868524::DEBUG::2013-11-19 21:02:23,434::task::495::TaskManager.Task::(_debug) Task db0baf9b-0bc3-4c3f-b629-bb1c876e27f5: ref 0 aborting False
>Thread-15868524::INFO::2013-11-19 21:02:23,435::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15909965::DEBUG::2013-11-19 21:02:23,718::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0747645 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15910052::DEBUG::2013-11-19 21:02:23,928::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910052::INFO::2013-11-19 21:02:23,929::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910052::DEBUG::2013-11-19 21:02:23,929::task::495::TaskManager.Task::(_debug) Task c7b1d90e-ede8-4c9b-9758-fc0c23632976: moving from state init -> state preparing
>Thread-15910052::DEBUG::2013-11-19 21:02:23,930::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=274 spmId=2
>Thread-15910052::DEBUG::2013-11-19 21:02:23,930::task::495::TaskManager.Task::(_debug) Task c7b1d90e-ede8-4c9b-9758-fc0c23632976: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910052::DEBUG::2013-11-19 21:02:23,931::task::495::TaskManager.Task::(_debug) Task c7b1d90e-ede8-4c9b-9758-fc0c23632976: moving from state preparing -> state finished
>Thread-15910052::DEBUG::2013-11-19 21:02:23,932::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910052::DEBUG::2013-11-19 21:02:23,932::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910052::DEBUG::2013-11-19 21:02:23,933::task::495::TaskManager.Task::(_debug) Task c7b1d90e-ede8-4c9b-9758-fc0c23632976: ref 0 aborting False
>Thread-15910052::INFO::2013-11-19 21:02:23,933::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910053::DEBUG::2013-11-19 21:02:23,938::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910053::INFO::2013-11-19 21:02:23,939::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910053::DEBUG::2013-11-19 21:02:23,940::task::495::TaskManager.Task::(_debug) Task be2ecf13-ac9a-44dc-a9c7-6ec94b2dcb92: moving from state init -> state preparing
>Thread-15910053::DEBUG::2013-11-19 21:02:23,941::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6db061ef-503b-4376-89f1-1acebec59819`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910053::DEBUG::2013-11-19 21:02:23,942::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910053::DEBUG::2013-11-19 21:02:23,942::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910053::DEBUG::2013-11-19 21:02:23,943::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6db061ef-503b-4376-89f1-1acebec59819`::Granted request
>Thread-15910053::DEBUG::2013-11-19 21:02:23,944::task::495::TaskManager.Task::(_debug) Task be2ecf13-ac9a-44dc-a9c7-6ec94b2dcb92: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910053::DEBUG::2013-11-19 21:02:23,945::task::495::TaskManager.Task::(_debug) Task be2ecf13-ac9a-44dc-a9c7-6ec94b2dcb92: ref 1 aborting False
>Thread-15910053::WARNING::2013-11-19 21:02:23,947::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910053::DEBUG::2013-11-19 21:02:23,948::task::495::TaskManager.Task::(_debug) Task be2ecf13-ac9a-44dc-a9c7-6ec94b2dcb92: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910053::DEBUG::2013-11-19 21:02:23,949::task::495::TaskManager.Task::(_debug) Task be2ecf13-ac9a-44dc-a9c7-6ec94b2dcb92: moving from state preparing -> state finished
>Thread-15910053::DEBUG::2013-11-19 21:02:23,949::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910053::DEBUG::2013-11-19 21:02:23,950::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910053::DEBUG::2013-11-19 21:02:23,951::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910053::DEBUG::2013-11-19 21:02:23,951::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910053::DEBUG::2013-11-19 21:02:23,952::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910053::DEBUG::2013-11-19 21:02:23,953::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910053::DEBUG::2013-11-19 21:02:23,953::task::495::TaskManager.Task::(_debug) Task be2ecf13-ac9a-44dc-a9c7-6ec94b2dcb92: ref 0 aborting False
>Thread-15910053::INFO::2013-11-19 21:02:23,954::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910054::INFO::2013-11-19 21:02:24,428::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910054::DEBUG::2013-11-19 21:02:24,429::task::495::TaskManager.Task::(_debug) Task fced4c13-5ec8-4956-9fcb-2f8398b3de43: moving from state init -> state preparing
>Thread-15910054::DEBUG::2013-11-19 21:02:24,429::task::495::TaskManager.Task::(_debug) Task fced4c13-5ec8-4956-9fcb-2f8398b3de43: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000293016433716', 'lastCheck': 1384894936.820327, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00141501426697', 'lastCheck': 1384894943.4160161, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384894938.396374, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000478029251099', 'lastCheck': 1384894936.274322, 'valid': True, 'code': 0}}
>Thread-15910054::DEBUG::2013-11-19 21:02:24,430::task::495::TaskManager.Task::(_debug) Task fced4c13-5ec8-4956-9fcb-2f8398b3de43: moving from state preparing -> state finished
>Thread-15910054::DEBUG::2013-11-19 21:02:24,431::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910054::DEBUG::2013-11-19 21:02:24,432::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910054::DEBUG::2013-11-19 21:02:24,432::task::495::TaskManager.Task::(_debug) Task fced4c13-5ec8-4956-9fcb-2f8398b3de43: ref 0 aborting False
>Thread-15910054::INFO::2013-11-19 21:02:24,433::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000293016433716', 'lastCheck': 1384894936.820327, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00141501426697', 'lastCheck': 1384894943.4160161, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384894938.396374, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000478029251099', 'lastCheck': 1384894936.274322, 'valid': True, 'code': 0}}
>Dummy-15909965::DEBUG::2013-11-19 21:02:25,733::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:02:26,284::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0757299 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15909965::DEBUG::2013-11-19 21:02:28,299::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:02:28,526::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15909965::DEBUG::2013-11-19 21:02:28,787::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0575639 s, 17.8 MB/s\n'; <rc> = 0
>Dummy-15909965::DEBUG::2013-11-19 21:02:30,798::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:02:31,360::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0736069 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15909965::DEBUG::2013-11-19 21:02:33,375::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:02:33,908::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0713555 s, 14.4 MB/s\n'; <rc> = 0
>Thread-15910060::DEBUG::2013-11-19 21:02:34,053::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910060::INFO::2013-11-19 21:02:34,054::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910060::DEBUG::2013-11-19 21:02:34,055::task::495::TaskManager.Task::(_debug) Task 5931f955-0efb-4587-a59c-1d7ecc4279f6: moving from state init -> state preparing
>Thread-15910060::DEBUG::2013-11-19 21:02:34,055::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=274 spmId=2
>Thread-15910060::DEBUG::2013-11-19 21:02:34,056::task::495::TaskManager.Task::(_debug) Task 5931f955-0efb-4587-a59c-1d7ecc4279f6: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910060::DEBUG::2013-11-19 21:02:34,056::task::495::TaskManager.Task::(_debug) Task 5931f955-0efb-4587-a59c-1d7ecc4279f6: moving from state preparing -> state finished
>Thread-15910060::DEBUG::2013-11-19 21:02:34,056::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910060::DEBUG::2013-11-19 21:02:34,057::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910060::DEBUG::2013-11-19 21:02:34,057::task::495::TaskManager.Task::(_debug) Task 5931f955-0efb-4587-a59c-1d7ecc4279f6: ref 0 aborting False
>Thread-15910060::INFO::2013-11-19 21:02:34,057::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910061::DEBUG::2013-11-19 21:02:34,062::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910061::INFO::2013-11-19 21:02:34,062::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910061::DEBUG::2013-11-19 21:02:34,063::task::495::TaskManager.Task::(_debug) Task ecac23a4-3ee5-40d3-9987-61ec58adb60b: moving from state init -> state preparing
>Thread-15910061::DEBUG::2013-11-19 21:02:34,064::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`05bf7975-b615-4f40-8fad-d262d30de26c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910061::DEBUG::2013-11-19 21:02:34,065::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910061::DEBUG::2013-11-19 21:02:34,066::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910061::DEBUG::2013-11-19 21:02:34,066::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`05bf7975-b615-4f40-8fad-d262d30de26c`::Granted request
>Thread-15910061::DEBUG::2013-11-19 21:02:34,067::task::495::TaskManager.Task::(_debug) Task ecac23a4-3ee5-40d3-9987-61ec58adb60b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910061::DEBUG::2013-11-19 21:02:34,068::task::495::TaskManager.Task::(_debug) Task ecac23a4-3ee5-40d3-9987-61ec58adb60b: ref 1 aborting False
>Thread-15910061::WARNING::2013-11-19 21:02:34,070::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910061::DEBUG::2013-11-19 21:02:34,070::task::495::TaskManager.Task::(_debug) Task ecac23a4-3ee5-40d3-9987-61ec58adb60b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910061::DEBUG::2013-11-19 21:02:34,071::task::495::TaskManager.Task::(_debug) Task ecac23a4-3ee5-40d3-9987-61ec58adb60b: moving from state preparing -> state finished
>Thread-15910061::DEBUG::2013-11-19 21:02:34,072::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910061::DEBUG::2013-11-19 21:02:34,072::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910061::DEBUG::2013-11-19 21:02:34,073::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910061::DEBUG::2013-11-19 21:02:34,073::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910061::DEBUG::2013-11-19 21:02:34,074::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910061::DEBUG::2013-11-19 21:02:34,074::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910061::DEBUG::2013-11-19 21:02:34,075::task::495::TaskManager.Task::(_debug) Task ecac23a4-3ee5-40d3-9987-61ec58adb60b: ref 0 aborting False
>Thread-15910061::INFO::2013-11-19 21:02:34,076::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910062::INFO::2013-11-19 21:02:34,568::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910062::DEBUG::2013-11-19 21:02:34,569::task::495::TaskManager.Task::(_debug) Task f4441c1c-f5ca-419c-b207-0ec73b1625ac: moving from state init -> state preparing
>Thread-15910062::DEBUG::2013-11-19 21:02:34,569::task::495::TaskManager.Task::(_debug) Task f4441c1c-f5ca-419c-b207-0ec73b1625ac: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000393152236938', 'lastCheck': 1384894946.965574, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00206685066223', 'lastCheck': 1384894953.59971, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384894948.5350389, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000555992126465', 'lastCheck': 1384894946.400502, 'valid': True, 'code': 0}}
>Thread-15910062::DEBUG::2013-11-19 21:02:34,570::task::495::TaskManager.Task::(_debug) Task f4441c1c-f5ca-419c-b207-0ec73b1625ac: moving from state preparing -> state finished
>Thread-15910062::DEBUG::2013-11-19 21:02:34,571::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910062::DEBUG::2013-11-19 21:02:34,571::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910062::DEBUG::2013-11-19 21:02:34,572::task::495::TaskManager.Task::(_debug) Task f4441c1c-f5ca-419c-b207-0ec73b1625ac: ref 0 aborting False
>Thread-15910062::INFO::2013-11-19 21:02:34,572::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000393152236938', 'lastCheck': 1384894946.965574, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00206685066223', 'lastCheck': 1384894953.59971, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384894948.5350389, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000555992126465', 'lastCheck': 1384894946.400502, 'valid': True, 'code': 0}}
>Dummy-15909965::DEBUG::2013-11-19 21:02:35,924::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:02:36,466::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0713031 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15909965::DEBUG::2013-11-19 21:02:38,481::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:02:38,701::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15909965::DEBUG::2013-11-19 21:02:39,028::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0878445 s, 11.7 MB/s\n'; <rc> = 0
>Dummy-15909965::DEBUG::2013-11-19 21:02:41,043::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15909534::INFO::2013-11-19 21:02:41,300::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15909534::DEBUG::2013-11-19 21:02:41,303::task::495::TaskManager.Task::(_debug) Task 0c76e5fb-4faf-4f4e-aef9-99a5dae4aebd: moving from state init -> state preparing
>Thread-15909534::DEBUG::2013-11-19 21:02:41,306::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f209bd64-ddba-4cfb-8858-04f3c851bfe1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15909534::DEBUG::2013-11-19 21:02:41,307::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15909534::DEBUG::2013-11-19 21:02:41,308::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15909534::DEBUG::2013-11-19 21:02:41,309::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f209bd64-ddba-4cfb-8858-04f3c851bfe1`::Granted request
>Thread-15909534::DEBUG::2013-11-19 21:02:41,310::task::495::TaskManager.Task::(_debug) Task 0c76e5fb-4faf-4f4e-aef9-99a5dae4aebd: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15909534::DEBUG::2013-11-19 21:02:41,311::task::495::TaskManager.Task::(_debug) Task 0c76e5fb-4faf-4f4e-aef9-99a5dae4aebd: ref 1 aborting False
>Thread-15909534::DEBUG::2013-11-19 21:02:41,311::task::495::TaskManager.Task::(_debug) Task 0c76e5fb-4faf-4f4e-aef9-99a5dae4aebd: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15909534::DEBUG::2013-11-19 21:02:41,312::task::495::TaskManager.Task::(_debug) Task 0c76e5fb-4faf-4f4e-aef9-99a5dae4aebd: moving from state preparing -> state finished
>Thread-15909534::DEBUG::2013-11-19 21:02:41,313::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15909534::DEBUG::2013-11-19 21:02:41,314::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15909534::DEBUG::2013-11-19 21:02:41,314::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15909534::DEBUG::2013-11-19 21:02:41,315::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15909534::DEBUG::2013-11-19 21:02:41,316::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15909534::DEBUG::2013-11-19 21:02:41,316::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15909534::DEBUG::2013-11-19 21:02:41,317::task::495::TaskManager.Task::(_debug) Task 0c76e5fb-4faf-4f4e-aef9-99a5dae4aebd: ref 0 aborting False
>Thread-15909534::INFO::2013-11-19 21:02:41,318::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15909965::DEBUG::2013-11-19 21:02:41,636::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0835639 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15909965::DEBUG::2013-11-19 21:02:43,646::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15910068::DEBUG::2013-11-19 21:02:44,196::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910068::INFO::2013-11-19 21:02:44,199::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910068::DEBUG::2013-11-19 21:02:44,200::task::495::TaskManager.Task::(_debug) Task a79bfb32-5d40-49b9-baee-afd6d347a6e7: moving from state init -> state preparing
>Thread-15910068::DEBUG::2013-11-19 21:02:44,205::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=274 spmId=2
>Thread-15910068::DEBUG::2013-11-19 21:02:44,209::task::495::TaskManager.Task::(_debug) Task a79bfb32-5d40-49b9-baee-afd6d347a6e7: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910068::DEBUG::2013-11-19 21:02:44,212::task::495::TaskManager.Task::(_debug) Task a79bfb32-5d40-49b9-baee-afd6d347a6e7: moving from state preparing -> state finished
>Thread-15910068::DEBUG::2013-11-19 21:02:44,214::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910068::DEBUG::2013-11-19 21:02:44,215::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910068::DEBUG::2013-11-19 21:02:44,217::task::495::TaskManager.Task::(_debug) Task a79bfb32-5d40-49b9-baee-afd6d347a6e7: ref 0 aborting False
>Thread-15910068::INFO::2013-11-19 21:02:44,218::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910069::DEBUG::2013-11-19 21:02:44,226::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910069::INFO::2013-11-19 21:02:44,228::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910069::DEBUG::2013-11-19 21:02:44,229::task::495::TaskManager.Task::(_debug) Task e89a35e4-e4f6-4f55-b541-aaa2a965b13f: moving from state init -> state preparing
>Thread-15910069::DEBUG::2013-11-19 21:02:44,231::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`28fd7dcc-dc85-47a0-800c-dc5a0a2f5bf9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910069::DEBUG::2013-11-19 21:02:44,232::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910069::DEBUG::2013-11-19 21:02:44,235::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910069::DEBUG::2013-11-19 21:02:44,236::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`28fd7dcc-dc85-47a0-800c-dc5a0a2f5bf9`::Granted request
>Thread-15910069::DEBUG::2013-11-19 21:02:44,239::task::495::TaskManager.Task::(_debug) Task e89a35e4-e4f6-4f55-b541-aaa2a965b13f: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910069::DEBUG::2013-11-19 21:02:44,240::task::495::TaskManager.Task::(_debug) Task e89a35e4-e4f6-4f55-b541-aaa2a965b13f: ref 1 aborting False
>Thread-15910069::WARNING::2013-11-19 21:02:44,243::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910069::DEBUG::2013-11-19 21:02:44,246::task::495::TaskManager.Task::(_debug) Task e89a35e4-e4f6-4f55-b541-aaa2a965b13f: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910069::DEBUG::2013-11-19 21:02:44,247::task::495::TaskManager.Task::(_debug) Task e89a35e4-e4f6-4f55-b541-aaa2a965b13f: moving from state preparing -> state finished
>Thread-15910069::DEBUG::2013-11-19 21:02:44,248::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910069::DEBUG::2013-11-19 21:02:44,249::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910069::DEBUG::2013-11-19 21:02:44,250::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910069::DEBUG::2013-11-19 21:02:44,252::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910069::DEBUG::2013-11-19 21:02:44,253::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910069::DEBUG::2013-11-19 21:02:44,254::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910069::DEBUG::2013-11-19 21:02:44,255::task::495::TaskManager.Task::(_debug) Task e89a35e4-e4f6-4f55-b541-aaa2a965b13f: ref 0 aborting False
>Thread-15910069::INFO::2013-11-19 21:02:44,256::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15909965::DEBUG::2013-11-19 21:02:44,265::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0730093 s, 14.0 MB/s\n'; <rc> = 0
>Thread-15910070::INFO::2013-11-19 21:02:44,753::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910070::DEBUG::2013-11-19 21:02:44,754::task::495::TaskManager.Task::(_debug) Task cb557e13-30e4-4861-aae1-5153e75622d2: moving from state init -> state preparing
>Thread-15910070::DEBUG::2013-11-19 21:02:44,754::task::495::TaskManager.Task::(_debug) Task cb557e13-30e4-4861-aae1-5153e75622d2: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000346899032593', 'lastCheck': 1384894957.149045, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00153994560242', 'lastCheck': 1384894963.896831, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384894958.709393, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000497102737427', 'lastCheck': 1384894956.5929899, 'valid': True, 'code': 0}}
>Thread-15910070::DEBUG::2013-11-19 21:02:44,755::task::495::TaskManager.Task::(_debug) Task cb557e13-30e4-4861-aae1-5153e75622d2: moving from state preparing -> state finished
>Thread-15910070::DEBUG::2013-11-19 21:02:44,756::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910070::DEBUG::2013-11-19 21:02:44,756::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910070::DEBUG::2013-11-19 21:02:44,757::task::495::TaskManager.Task::(_debug) Task cb557e13-30e4-4861-aae1-5153e75622d2: ref 0 aborting False
>Thread-15910070::INFO::2013-11-19 21:02:44,758::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000346899032593', 'lastCheck': 1384894957.149045, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00153994560242', 'lastCheck': 1384894963.896831, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384894958.709393, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000497102737427', 'lastCheck': 1384894956.5929899, 'valid': True, 'code': 0}}
>Dummy-15909965::DEBUG::2013-11-19 21:02:46,280::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:02:46,822::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0963337 s, 10.6 MB/s\n'; <rc> = 0
>Dummy-15909965::DEBUG::2013-11-19 21:02:48,838::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:02:49,064::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15909965::DEBUG::2013-11-19 21:02:49,343::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0716114 s, 14.3 MB/s\n'; <rc> = 0
>Dummy-15909965::DEBUG::2013-11-19 21:02:51,359::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:02:51,895::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0752084 s, 13.6 MB/s\n'; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-19 21:02:53,514::vm::525::vm.Vm::(_rtcUpdate) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::new rtc offset 2
>Dummy-15909965::DEBUG::2013-11-19 21:02:53,910::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15910076::DEBUG::2013-11-19 21:02:54,368::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910076::INFO::2013-11-19 21:02:54,370::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910076::DEBUG::2013-11-19 21:02:54,370::task::495::TaskManager.Task::(_debug) Task d6befbd3-ad71-44ce-9ee9-d9900973f58d: moving from state init -> state preparing
>Thread-15910076::DEBUG::2013-11-19 21:02:54,371::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=274 spmId=2
>Thread-15910076::DEBUG::2013-11-19 21:02:54,372::task::495::TaskManager.Task::(_debug) Task d6befbd3-ad71-44ce-9ee9-d9900973f58d: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910076::DEBUG::2013-11-19 21:02:54,373::task::495::TaskManager.Task::(_debug) Task d6befbd3-ad71-44ce-9ee9-d9900973f58d: moving from state preparing -> state finished
>Thread-15910076::DEBUG::2013-11-19 21:02:54,374::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910076::DEBUG::2013-11-19 21:02:54,374::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910076::DEBUG::2013-11-19 21:02:54,375::task::495::TaskManager.Task::(_debug) Task d6befbd3-ad71-44ce-9ee9-d9900973f58d: ref 0 aborting False
>Thread-15910076::INFO::2013-11-19 21:02:54,376::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910077::DEBUG::2013-11-19 21:02:54,381::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910077::INFO::2013-11-19 21:02:54,382::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910077::DEBUG::2013-11-19 21:02:54,383::task::495::TaskManager.Task::(_debug) Task c07175c6-f7cd-41b0-bc52-dead2a720c09: moving from state init -> state preparing
>Thread-15910077::DEBUG::2013-11-19 21:02:54,385::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`27298ef2-63c2-4ff4-b370-f08e92e416fc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910077::DEBUG::2013-11-19 21:02:54,386::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910077::DEBUG::2013-11-19 21:02:54,387::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910077::DEBUG::2013-11-19 21:02:54,388::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`27298ef2-63c2-4ff4-b370-f08e92e416fc`::Granted request
>Thread-15910077::DEBUG::2013-11-19 21:02:54,390::task::495::TaskManager.Task::(_debug) Task c07175c6-f7cd-41b0-bc52-dead2a720c09: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910077::DEBUG::2013-11-19 21:02:54,391::task::495::TaskManager.Task::(_debug) Task c07175c6-f7cd-41b0-bc52-dead2a720c09: ref 1 aborting False
>Thread-15910077::WARNING::2013-11-19 21:02:54,393::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910077::DEBUG::2013-11-19 21:02:54,395::task::495::TaskManager.Task::(_debug) Task c07175c6-f7cd-41b0-bc52-dead2a720c09: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910077::DEBUG::2013-11-19 21:02:54,396::task::495::TaskManager.Task::(_debug) Task c07175c6-f7cd-41b0-bc52-dead2a720c09: moving from state preparing -> state finished
>Thread-15910077::DEBUG::2013-11-19 21:02:54,397::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910077::DEBUG::2013-11-19 21:02:54,397::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910077::DEBUG::2013-11-19 21:02:54,398::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910077::DEBUG::2013-11-19 21:02:54,399::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910077::DEBUG::2013-11-19 21:02:54,400::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910077::DEBUG::2013-11-19 21:02:54,401::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910077::DEBUG::2013-11-19 21:02:54,402::task::495::TaskManager.Task::(_debug) Task c07175c6-f7cd-41b0-bc52-dead2a720c09: ref 0 aborting False
>Thread-15910077::INFO::2013-11-19 21:02:54,403::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15909965::DEBUG::2013-11-19 21:02:54,502::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.105244 s, 9.7 MB/s\n'; <rc> = 0
>Thread-15910078::INFO::2013-11-19 21:02:55,091::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910078::DEBUG::2013-11-19 21:02:55,091::task::495::TaskManager.Task::(_debug) Task b1e746ae-fb6c-4069-b90b-eba5213a3bef: moving from state init -> state preparing
>Thread-15910078::DEBUG::2013-11-19 21:02:55,092::task::495::TaskManager.Task::(_debug) Task b1e746ae-fb6c-4069-b90b-eba5213a3bef: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000378131866455', 'lastCheck': 1384894967.4729011, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.000960111618042', 'lastCheck': 1384894974.165947, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384894969.068589, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000412940979004', 'lastCheck': 1384894966.890193, 'valid': True, 'code': 0}}
>Thread-15910078::DEBUG::2013-11-19 21:02:55,092::task::495::TaskManager.Task::(_debug) Task b1e746ae-fb6c-4069-b90b-eba5213a3bef: moving from state preparing -> state finished
>Thread-15910078::DEBUG::2013-11-19 21:02:55,093::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910078::DEBUG::2013-11-19 21:02:55,093::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910078::DEBUG::2013-11-19 21:02:55,093::task::495::TaskManager.Task::(_debug) Task b1e746ae-fb6c-4069-b90b-eba5213a3bef: ref 0 aborting False
>Thread-15910078::INFO::2013-11-19 21:02:55,094::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000378131866455', 'lastCheck': 1384894967.4729011, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.000960111618042', 'lastCheck': 1384894974.165947, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384894969.068589, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000412940979004', 'lastCheck': 1384894966.890193, 'valid': True, 'code': 0}}
>Dummy-15909965::DEBUG::2013-11-19 21:02:56,515::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:02:57,035::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0956138 s, 10.7 MB/s\n'; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-19 21:02:58,657::libvirtvm::1401::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::event Saved detail 0 opaque None
>Thread-15868525::ERROR::2013-11-19 21:02:58,977::guestIF::246::vm.Vm::(run) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Run exception: None
>Traceback (most recent call last):
>  File "/usr/share/vdsm/guestIF.py", line 238, in run
>  File "/usr/share/vdsm/guestIF.py", line 214, in _readLine
>  File "/usr/share/vdsm/guestIF.py", line 200, in _readBuffer
>error: [Errno 104] Connection reset by peer
>Dummy-15909965::DEBUG::2013-11-19 21:02:59,051::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:02:59,288::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15868524::DEBUG::2013-11-19 21:02:59,536::libvirtvm::288::vm.Vm::(handleStatsException) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::VM not found, moving to Down
>Traceback (most recent call last):
>  File "/usr/share/vdsm/utils.py", line 373, in collect
>  File "/usr/share/vdsm/utils.py", line 250, in __call__
>  File "/usr/share/vdsm/libvirtvm.py", line 161, in _sampleNet
>  File "/usr/share/vdsm/libvirtvm.py", line 445, in f
>  File "/usr/share/vdsm/libvirtconnection.py", line 63, in wrapper
>  File "/usr/lib64/python2.6/site-packages/libvirt.py", line 1734, in interfaceStats
>libvirtError: Domain not found: no domain with matching uuid '2afdc298-b10b-4bd1-8df2-2a401efd4f9c'
>libvirtEventLoop::DEBUG::2013-11-19 21:02:59,536::libvirtvm::1401::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::event Stopped detail 0 opaque None
>libvirtEventLoop::INFO::2013-11-19 21:02:59,539::vm::445::vm.Vm::(_onQemuDeath) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::underlying process disconnected
>libvirtEventLoop::INFO::2013-11-19 21:02:59,540::libvirtvm::1304::vm.Vm::(releaseVm) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Release VM resources
>Thread-15868524::DEBUG::2013-11-19 21:02:59,549::vm::742::vm.Vm::(setDownStatus) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Changed state to Down: Domain not found: no domain with matching uuid '2afdc298-b10b-4bd1-8df2-2a401efd4f9c'
>Thread-15868524::DEBUG::2013-11-19 21:02:59,550::utils::314::vm.Vm::(stop) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Stop statistics collection
>libvirtEventLoop::DEBUG::2013-11-19 21:02:59,550::utils::314::vm.Vm::(stop) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Stop statistics collection
>libvirtEventLoop::WARNING::2013-11-19 21:02:59,554::libvirtvm::1315::vm.Vm::(releaseVm) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Traceback (most recent call last):
>  File "/usr/share/vdsm/libvirtvm.py", line 1312, in releaseVm
>  File "/usr/share/vdsm/libvirtvm.py", line 445, in f
>  File "/usr/share/vdsm/libvirtconnection.py", line 63, in wrapper
>  File "/usr/lib64/python2.6/site-packages/libvirt.py", line 672, in destroy
>libvirtError: Domain not found: no domain with matching uuid '2afdc298-b10b-4bd1-8df2-2a401efd4f9c'
>
>libvirtEventLoop::DEBUG::2013-11-19 21:02:59,556::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /sbin/service ksmtuned retune' (cwd None)
>Thread-15868524::DEBUG::2013-11-19 21:02:59,744::utils::345::vm.Vm::(run) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Stats thread finished
>Dummy-15909965::DEBUG::2013-11-19 21:02:59,791::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.283326 s, 3.6 MB/s\n'; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-19 21:02:59,996::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-19 21:02:59,999::vm::718::vm.Vm::(_cleanup) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Drive None cleanup
>libvirtEventLoop::DEBUG::2013-11-19 21:03:00,000::vm::718::vm.Vm::(_cleanup) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Drive /rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/e0628852-41b1-4535-8dda-ebc96cb00952/cd4f9324-c91d-405b-983f-62afa4daa8ad cleanup
>libvirtEventLoop::INFO::2013-11-19 21:03:00,002::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: teardownVolume, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>libvirtEventLoop::DEBUG::2013-11-19 21:03:00,002::task::495::TaskManager.Task::(_debug) Task f4526a4c-a2e3-4906-b3d5-ef95f9fcec69: moving from state init -> state preparing
>libvirtEventLoop::DEBUG::2013-11-19 21:03:00,004::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3cd77753-67e2-446a-ac8e-83b58638c447`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1727' at 'public_teardownVolume'
>libvirtEventLoop::DEBUG::2013-11-19 21:03:00,005::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>libvirtEventLoop::DEBUG::2013-11-19 21:03:00,006::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>libvirtEventLoop::DEBUG::2013-11-19 21:03:00,006::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3cd77753-67e2-446a-ac8e-83b58638c447`::Granted request
>libvirtEventLoop::DEBUG::2013-11-19 21:03:00,007::task::495::TaskManager.Task::(_debug) Task f4526a4c-a2e3-4906-b3d5-ef95f9fcec69: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>libvirtEventLoop::DEBUG::2013-11-19 21:03:00,008::task::495::TaskManager.Task::(_debug) Task f4526a4c-a2e3-4906-b3d5-ef95f9fcec69: ref 1 aborting False
>libvirtEventLoop::DEBUG::2013-11-19 21:03:00,009::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.e0628852-41b1-4535-8dda-ebc96cb00952'
>libvirtEventLoop::DEBUG::2013-11-19 21:03:00,010::resourceManager::530::ResourceManager::(releaseResource) Released resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.e0628852-41b1-4535-8dda-ebc96cb00952' (0 active users)
>libvirtEventLoop::DEBUG::2013-11-19 21:03:00,010::resourceManager::535::ResourceManager::(releaseResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.e0628852-41b1-4535-8dda-ebc96cb00952' is free, finding out if anyone is waiting for it.
>libvirtEventLoop::DEBUG::2013-11-19 21:03:00,011::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.cd4f9324-c91d-405b-983f-62afa4daa8ad'
>libvirtEventLoop::DEBUG::2013-11-19 21:03:00,012::resourceManager::530::ResourceManager::(releaseResource) Released resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.cd4f9324-c91d-405b-983f-62afa4daa8ad' (0 active users)
>libvirtEventLoop::DEBUG::2013-11-19 21:03:00,013::resourceManager::535::ResourceManager::(releaseResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.cd4f9324-c91d-405b-983f-62afa4daa8ad' is free, finding out if anyone is waiting for it.
>libvirtEventLoop::DEBUG::2013-11-19 21:03:00,013::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.cd4f9324-c91d-405b-983f-62afa4daa8ad', Clearing records.
>libvirtEventLoop::DEBUG::2013-11-19 21:03:00,014::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.e0628852-41b1-4535-8dda-ebc96cb00952', Clearing records.
>libvirtEventLoop::INFO::2013-11-19 21:03:00,015::blockVolume::386::Storage.Volume::(teardown) Tearing down volume 6b892067-ac7e-4039-a7be-5b7387525b16/cd4f9324-c91d-405b-983f-62afa4daa8ad justme False
>libvirtEventLoop::DEBUG::2013-11-19 21:03:00,015::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.cd4f9324-c91d-405b-983f-62afa4daa8ad'
>libvirtEventLoop::DEBUG::2013-11-19 21:03:00,016::resourceManager::530::ResourceManager::(releaseResource) Released resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.cd4f9324-c91d-405b-983f-62afa4daa8ad' (0 active users)
>libvirtEventLoop::DEBUG::2013-11-19 21:03:00,016::resourceManager::535::ResourceManager::(releaseResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.cd4f9324-c91d-405b-983f-62afa4daa8ad' is free, finding out if anyone is waiting for it.
>libvirtEventLoop::DEBUG::2013-11-19 21:03:00,018::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --autobackup n --available n 6b892067-ac7e-4039-a7be-5b7387525b16/cd4f9324-c91d-405b-983f-62afa4daa8ad' (cwd None)
>libvirtEventLoop::DEBUG::2013-11-19 21:03:00,751::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-19 21:03:00,753::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>libvirtEventLoop::DEBUG::2013-11-19 21:03:00,754::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>libvirtEventLoop::DEBUG::2013-11-19 21:03:00,754::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.cd4f9324-c91d-405b-983f-62afa4daa8ad', Clearing records.
>libvirtEventLoop::DEBUG::2013-11-19 21:03:00,755::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>libvirtEventLoop::DEBUG::2013-11-19 21:03:00,757::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15910083::DEBUG::2013-11-19 21:03:01,320::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('2afdc298-b10b-4bd1-8df2-2a401efd4f9c',) {}
>Thread-15910083::DEBUG::2013-11-19 21:03:01,322::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering down', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '19979', 'displayIp': '0', 'displayPort': u'5900', 'session': 'Unknown', 'displaySecurePort': u'5901', 'timeOffset': 2L, 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet0': {'macAddr': u'00:1a:4a:a8:64:11', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}, u'vnet1': {'macAddr': u'00:1a:4a:a8:64:12', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '100', 'name': u'vnet1'}}, 'vmId': '2afdc298-b10b-4bd1-8df2-2a401efd4f9c', 'monitorResponse': '0', 'cpuUser': '19.41', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '32346472448', 'writeLatency': '360629', 'imageID': 'e0628852-41b1-4535-8dda-ebc96cb00952', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '32346472448', 'writeRate': '17860.71'}}, 'boot': 'c', 'statsAge': '2.04', 'cpuIdle': '80.59', 'elapsedTime': '128783', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>libvirtEventLoop::DEBUG::2013-11-19 21:03:01,519::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-19 21:03:01,775::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>libvirtEventLoop::DEBUG::2013-11-19 21:03:01,776::task::495::TaskManager.Task::(_debug) Task f4526a4c-a2e3-4906-b3d5-ef95f9fcec69: finished: None
>libvirtEventLoop::DEBUG::2013-11-19 21:03:01,776::task::495::TaskManager.Task::(_debug) Task f4526a4c-a2e3-4906-b3d5-ef95f9fcec69: moving from state preparing -> state finished
>libvirtEventLoop::DEBUG::2013-11-19 21:03:01,776::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>libvirtEventLoop::DEBUG::2013-11-19 21:03:01,777::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>libvirtEventLoop::DEBUG::2013-11-19 21:03:01,777::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>libvirtEventLoop::DEBUG::2013-11-19 21:03:01,778::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>libvirtEventLoop::DEBUG::2013-11-19 21:03:01,778::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>libvirtEventLoop::DEBUG::2013-11-19 21:03:01,778::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>libvirtEventLoop::DEBUG::2013-11-19 21:03:01,779::task::495::TaskManager.Task::(_debug) Task f4526a4c-a2e3-4906-b3d5-ef95f9fcec69: ref 0 aborting False
>libvirtEventLoop::INFO::2013-11-19 21:03:01,779::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: teardownVolume, Return response: {'status': {'message': 'OK', 'code': 0}}
>libvirtEventLoop::DEBUG::2013-11-19 21:03:01,781::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/libexec/vdsm/hooks/after_vm_destroy/50_vhostmd' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:03:01,955::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:03:02,593::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0726782 s, 14.1 MB/s\n'; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-19 21:03:02,690::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>libvirtEventLoop::INFO::2013-11-19 21:03:02,691::hooks::51::root::(_runHooksDir) 
>libvirtEventLoop::DEBUG::2013-11-19 21:03:02,698::vm::742::vm.Vm::(setDownStatus) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Changed state to Down: User shut down
>libvirtEventLoop::DEBUG::2013-11-19 21:03:02,698::utils::314::vm.Vm::(stop) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Stop statistics collection
>Thread-15910086::DEBUG::2013-11-19 21:03:03,380::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('2afdc298-b10b-4bd1-8df2-2a401efd4f9c',) {}
>Thread-15910086::DEBUG::2013-11-19 21:03:03,380::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Down', 'timeOffset': 2L, 'vmId': '2afdc298-b10b-4bd1-8df2-2a401efd4f9c', 'exitMessage': 'User shut down', 'exitCode': 0}]}
>Thread-15910087::DEBUG::2013-11-19 21:03:03,465::clientIF::54::vds::(wrapper) [192.168.100.135]::call destroy with ('2afdc298-b10b-4bd1-8df2-2a401efd4f9c',) {}
>Thread-15910087::INFO::2013-11-19 21:03:03,466::clientIF::450::vds::(destroy) vmContainerLock acquired by vm 2afdc298-b10b-4bd1-8df2-2a401efd4f9c
>Thread-15910087::DEBUG::2013-11-19 21:03:03,467::libvirtvm::1344::vm.Vm::(destroy) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::destroy Called
>Thread-15910087::DEBUG::2013-11-19 21:03:03,467::libvirtvm::1339::vm.Vm::(deleteVm) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Total desktops after destroy of 2afdc298-b10b-4bd1-8df2-2a401efd4f9c is 2
>Thread-15910087::DEBUG::2013-11-19 21:03:03,468::clientIF::59::vds::(wrapper) return destroy with {'status': {'message': 'Machine destroyed', 'code': 0}}
>Thread-15910088::DEBUG::2013-11-19 21:03:04,541::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910088::INFO::2013-11-19 21:03:04,543::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910088::DEBUG::2013-11-19 21:03:04,544::task::495::TaskManager.Task::(_debug) Task 8df135f6-4ef4-4235-9123-f1de00141cf6: moving from state init -> state preparing
>Thread-15910088::DEBUG::2013-11-19 21:03:04,544::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=274 spmId=2
>Thread-15910088::DEBUG::2013-11-19 21:03:04,545::task::495::TaskManager.Task::(_debug) Task 8df135f6-4ef4-4235-9123-f1de00141cf6: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910088::DEBUG::2013-11-19 21:03:04,546::task::495::TaskManager.Task::(_debug) Task 8df135f6-4ef4-4235-9123-f1de00141cf6: moving from state preparing -> state finished
>Thread-15910088::DEBUG::2013-11-19 21:03:04,546::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910088::DEBUG::2013-11-19 21:03:04,547::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910088::DEBUG::2013-11-19 21:03:04,547::task::495::TaskManager.Task::(_debug) Task 8df135f6-4ef4-4235-9123-f1de00141cf6: ref 0 aborting False
>Thread-15910088::INFO::2013-11-19 21:03:04,548::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910089::DEBUG::2013-11-19 21:03:04,552::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910089::INFO::2013-11-19 21:03:04,553::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910089::DEBUG::2013-11-19 21:03:04,554::task::495::TaskManager.Task::(_debug) Task e93591a6-de64-47fd-bcbf-1d976e1dd4f7: moving from state init -> state preparing
>Thread-15910089::DEBUG::2013-11-19 21:03:04,555::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`cb8ceef9-13fc-4370-b76a-354716efc87b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910089::DEBUG::2013-11-19 21:03:04,556::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910089::DEBUG::2013-11-19 21:03:04,557::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910089::DEBUG::2013-11-19 21:03:04,557::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`cb8ceef9-13fc-4370-b76a-354716efc87b`::Granted request
>Thread-15910089::DEBUG::2013-11-19 21:03:04,558::task::495::TaskManager.Task::(_debug) Task e93591a6-de64-47fd-bcbf-1d976e1dd4f7: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910089::DEBUG::2013-11-19 21:03:04,559::task::495::TaskManager.Task::(_debug) Task e93591a6-de64-47fd-bcbf-1d976e1dd4f7: ref 1 aborting False
>Thread-15910089::WARNING::2013-11-19 21:03:04,561::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910089::DEBUG::2013-11-19 21:03:04,562::task::495::TaskManager.Task::(_debug) Task e93591a6-de64-47fd-bcbf-1d976e1dd4f7: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910089::DEBUG::2013-11-19 21:03:04,563::task::495::TaskManager.Task::(_debug) Task e93591a6-de64-47fd-bcbf-1d976e1dd4f7: moving from state preparing -> state finished
>Thread-15910089::DEBUG::2013-11-19 21:03:04,563::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910089::DEBUG::2013-11-19 21:03:04,564::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910089::DEBUG::2013-11-19 21:03:04,565::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910089::DEBUG::2013-11-19 21:03:04,565::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910089::DEBUG::2013-11-19 21:03:04,566::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910089::DEBUG::2013-11-19 21:03:04,567::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910089::DEBUG::2013-11-19 21:03:04,567::task::495::TaskManager.Task::(_debug) Task e93591a6-de64-47fd-bcbf-1d976e1dd4f7: ref 0 aborting False
>Thread-15910089::INFO::2013-11-19 21:03:04,568::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15909965::DEBUG::2013-11-19 21:03:04,603::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:03:05,169::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.079084 s, 12.9 MB/s\n'; <rc> = 0
>Thread-15910090::INFO::2013-11-19 21:03:05,580::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910090::DEBUG::2013-11-19 21:03:05,581::task::495::TaskManager.Task::(_debug) Task 833269ef-fc81-4d55-879f-3160655adacb: moving from state init -> state preparing
>Thread-15910090::DEBUG::2013-11-19 21:03:05,582::task::495::TaskManager.Task::(_debug) Task 833269ef-fc81-4d55-879f-3160655adacb: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000433206558228', 'lastCheck': 1384894977.6984739, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148487091064', 'lastCheck': 1384894984.289892, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384894979.2972829, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000394821166992', 'lastCheck': 1384894977.1592281, 'valid': True, 'code': 0}}
>Thread-15910090::DEBUG::2013-11-19 21:03:05,583::task::495::TaskManager.Task::(_debug) Task 833269ef-fc81-4d55-879f-3160655adacb: moving from state preparing -> state finished
>Thread-15910090::DEBUG::2013-11-19 21:03:05,584::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910090::DEBUG::2013-11-19 21:03:05,584::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910090::DEBUG::2013-11-19 21:03:05,585::task::495::TaskManager.Task::(_debug) Task 833269ef-fc81-4d55-879f-3160655adacb: ref 0 aborting False
>Thread-15910090::INFO::2013-11-19 21:03:05,585::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000433206558228', 'lastCheck': 1384894977.6984739, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148487091064', 'lastCheck': 1384894984.289892, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384894979.2972829, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000394821166992', 'lastCheck': 1384894977.1592281, 'valid': True, 'code': 0}}
>Thread-15909535::DEBUG::2013-11-19 21:03:06,735::guestIF::97::vm.Vm::(_handleMessage) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::u'None'
>Dummy-15909965::DEBUG::2013-11-19 21:03:07,186::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:03:07,677::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0776669 s, 13.2 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:03:09,372::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15909965::DEBUG::2013-11-19 21:03:09,693::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:03:10,266::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0756869 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15909535::DEBUG::2013-11-19 21:03:11,423::guestIF::112::vm.Vm::(_handleMessage) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Guest system shuts down.
>libvirtEventLoop::DEBUG::2013-11-19 21:03:11,450::vm::525::vm.Vm::(_rtcUpdate) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::new rtc offset -1
>libvirtEventLoop::DEBUG::2013-11-19 21:03:11,451::vm::525::vm.Vm::(_rtcUpdate) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::new rtc offset -1
>libvirtEventLoop::DEBUG::2013-11-19 21:03:11,452::vm::525::vm.Vm::(_rtcUpdate) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::new rtc offset -1
>libvirtEventLoop::DEBUG::2013-11-19 21:03:11,453::vm::525::vm.Vm::(_rtcUpdate) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::new rtc offset -1
>libvirtEventLoop::DEBUG::2013-11-19 21:03:11,454::vm::525::vm.Vm::(_rtcUpdate) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::new rtc offset -1
>libvirtEventLoop::DEBUG::2013-11-19 21:03:11,455::vm::525::vm.Vm::(_rtcUpdate) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::new rtc offset -1
>libvirtEventLoop::DEBUG::2013-11-19 21:03:11,456::vm::525::vm.Vm::(_rtcUpdate) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::new rtc offset -1
>Dummy-15909965::DEBUG::2013-11-19 21:03:12,280::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:03:12,849::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0810334 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15910096::DEBUG::2013-11-19 21:03:14,681::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910096::INFO::2013-11-19 21:03:14,683::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910096::DEBUG::2013-11-19 21:03:14,684::task::495::TaskManager.Task::(_debug) Task 67e868e0-f4f0-46b2-b2bb-48dab9c2d500: moving from state init -> state preparing
>Thread-15910096::DEBUG::2013-11-19 21:03:14,685::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=274 spmId=2
>Thread-15910096::DEBUG::2013-11-19 21:03:14,685::task::495::TaskManager.Task::(_debug) Task 67e868e0-f4f0-46b2-b2bb-48dab9c2d500: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910096::DEBUG::2013-11-19 21:03:14,686::task::495::TaskManager.Task::(_debug) Task 67e868e0-f4f0-46b2-b2bb-48dab9c2d500: moving from state preparing -> state finished
>Thread-15910096::DEBUG::2013-11-19 21:03:14,687::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910096::DEBUG::2013-11-19 21:03:14,687::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910096::DEBUG::2013-11-19 21:03:14,688::task::495::TaskManager.Task::(_debug) Task 67e868e0-f4f0-46b2-b2bb-48dab9c2d500: ref 0 aborting False
>Thread-15910096::INFO::2013-11-19 21:03:14,689::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910097::DEBUG::2013-11-19 21:03:14,694::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910097::INFO::2013-11-19 21:03:14,696::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910097::DEBUG::2013-11-19 21:03:14,696::task::495::TaskManager.Task::(_debug) Task 6d12caa5-a34f-42c5-8eef-b7c61b11673a: moving from state init -> state preparing
>Thread-15910097::DEBUG::2013-11-19 21:03:14,699::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`19c02e71-b108-4e93-b37b-cdf2bdb395e6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910097::DEBUG::2013-11-19 21:03:14,700::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910097::DEBUG::2013-11-19 21:03:14,701::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910097::DEBUG::2013-11-19 21:03:14,702::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`19c02e71-b108-4e93-b37b-cdf2bdb395e6`::Granted request
>Thread-15910097::DEBUG::2013-11-19 21:03:14,704::task::495::TaskManager.Task::(_debug) Task 6d12caa5-a34f-42c5-8eef-b7c61b11673a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910097::DEBUG::2013-11-19 21:03:14,705::task::495::TaskManager.Task::(_debug) Task 6d12caa5-a34f-42c5-8eef-b7c61b11673a: ref 1 aborting False
>Thread-15910097::WARNING::2013-11-19 21:03:14,709::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910097::DEBUG::2013-11-19 21:03:14,710::task::495::TaskManager.Task::(_debug) Task 6d12caa5-a34f-42c5-8eef-b7c61b11673a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910097::DEBUG::2013-11-19 21:03:14,712::task::495::TaskManager.Task::(_debug) Task 6d12caa5-a34f-42c5-8eef-b7c61b11673a: moving from state preparing -> state finished
>Thread-15910097::DEBUG::2013-11-19 21:03:14,713::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910097::DEBUG::2013-11-19 21:03:14,714::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910097::DEBUG::2013-11-19 21:03:14,715::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910097::DEBUG::2013-11-19 21:03:14,716::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910097::DEBUG::2013-11-19 21:03:14,717::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910097::DEBUG::2013-11-19 21:03:14,718::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910097::DEBUG::2013-11-19 21:03:14,719::task::495::TaskManager.Task::(_debug) Task 6d12caa5-a34f-42c5-8eef-b7c61b11673a: ref 0 aborting False
>Thread-15910097::INFO::2013-11-19 21:03:14,719::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15909965::DEBUG::2013-11-19 21:03:14,866::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:03:15,376::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0756449 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15910098::INFO::2013-11-19 21:03:15,971::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910098::DEBUG::2013-11-19 21:03:15,972::task::495::TaskManager.Task::(_debug) Task ee89203b-9054-4bc2-9e10-7ffe31c6ca06: moving from state init -> state preparing
>Thread-15910098::DEBUG::2013-11-19 21:03:15,972::task::495::TaskManager.Task::(_debug) Task ee89203b-9054-4bc2-9e10-7ffe31c6ca06: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000358819961548', 'lastCheck': 1384894987.8386061, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00100898742676', 'lastCheck': 1384894994.5179939, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384894989.3733709, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000553846359253', 'lastCheck': 1384894987.368474, 'valid': True, 'code': 0}}
>Thread-15910098::DEBUG::2013-11-19 21:03:15,973::task::495::TaskManager.Task::(_debug) Task ee89203b-9054-4bc2-9e10-7ffe31c6ca06: moving from state preparing -> state finished
>Thread-15910098::DEBUG::2013-11-19 21:03:15,974::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910098::DEBUG::2013-11-19 21:03:15,975::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910098::DEBUG::2013-11-19 21:03:15,975::task::495::TaskManager.Task::(_debug) Task ee89203b-9054-4bc2-9e10-7ffe31c6ca06: ref 0 aborting False
>Thread-15910098::INFO::2013-11-19 21:03:15,976::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000358819961548', 'lastCheck': 1384894987.8386061, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00100898742676', 'lastCheck': 1384894994.5179939, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384894989.3733709, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000553846359253', 'lastCheck': 1384894987.368474, 'valid': True, 'code': 0}}
>Dummy-15909965::DEBUG::2013-11-19 21:03:17,392::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:03:17,946::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.101882 s, 10.1 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:03:19,608::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15909965::DEBUG::2013-11-19 21:03:19,962::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:03:20,522::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0804972 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15909965::DEBUG::2013-11-19 21:03:22,537::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:03:23,094::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0718544 s, 14.3 MB/s\n'; <rc> = 0
>Thread-4922911::INFO::2013-11-19 21:03:23,391::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-4922911::DEBUG::2013-11-19 21:03:23,395::task::495::TaskManager.Task::(_debug) Task 40c410dd-e80b-4c52-a285-bc062942733c: moving from state init -> state preparing
>Thread-4922911::DEBUG::2013-11-19 21:03:23,398::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b35ed45a-2221-4efa-b636-5df7f68ec2ab`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-4922911::DEBUG::2013-11-19 21:03:23,399::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-4922911::DEBUG::2013-11-19 21:03:23,403::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-4922911::DEBUG::2013-11-19 21:03:23,408::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b35ed45a-2221-4efa-b636-5df7f68ec2ab`::Granted request
>Thread-4922911::DEBUG::2013-11-19 21:03:23,411::task::495::TaskManager.Task::(_debug) Task 40c410dd-e80b-4c52-a285-bc062942733c: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-4922911::DEBUG::2013-11-19 21:03:23,417::task::495::TaskManager.Task::(_debug) Task 40c410dd-e80b-4c52-a285-bc062942733c: ref 1 aborting False
>Thread-4922911::DEBUG::2013-11-19 21:03:23,424::task::495::TaskManager.Task::(_debug) Task 40c410dd-e80b-4c52-a285-bc062942733c: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-4922911::DEBUG::2013-11-19 21:03:23,430::task::495::TaskManager.Task::(_debug) Task 40c410dd-e80b-4c52-a285-bc062942733c: moving from state preparing -> state finished
>Thread-4922911::DEBUG::2013-11-19 21:03:23,434::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-4922911::DEBUG::2013-11-19 21:03:23,434::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-4922911::DEBUG::2013-11-19 21:03:23,437::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-4922911::DEBUG::2013-11-19 21:03:23,437::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-4922911::DEBUG::2013-11-19 21:03:23,438::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-4922911::DEBUG::2013-11-19 21:03:23,439::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-4922911::DEBUG::2013-11-19 21:03:23,439::task::495::TaskManager.Task::(_debug) Task 40c410dd-e80b-4c52-a285-bc062942733c: ref 0 aborting False
>Thread-4922911::INFO::2013-11-19 21:03:23,440::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-4922911::INFO::2013-11-19 21:03:23,441::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-4922911::DEBUG::2013-11-19 21:03:23,444::task::495::TaskManager.Task::(_debug) Task 0e3e2ae3-83fe-4117-bdc5-1c3c1d8aed45: moving from state init -> state preparing
>Thread-4922911::DEBUG::2013-11-19 21:03:23,446::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`3617c09b-5727-4f65-b01b-11dc9cae8d02`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-4922911::DEBUG::2013-11-19 21:03:23,451::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-4922911::DEBUG::2013-11-19 21:03:23,456::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-4922911::DEBUG::2013-11-19 21:03:23,463::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`3617c09b-5727-4f65-b01b-11dc9cae8d02`::Granted request
>Thread-4922911::DEBUG::2013-11-19 21:03:23,464::task::495::TaskManager.Task::(_debug) Task 0e3e2ae3-83fe-4117-bdc5-1c3c1d8aed45: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-4922911::DEBUG::2013-11-19 21:03:23,468::task::495::TaskManager.Task::(_debug) Task 0e3e2ae3-83fe-4117-bdc5-1c3c1d8aed45: ref 1 aborting False
>Thread-4922911::DEBUG::2013-11-19 21:03:23,471::task::495::TaskManager.Task::(_debug) Task 0e3e2ae3-83fe-4117-bdc5-1c3c1d8aed45: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-4922911::DEBUG::2013-11-19 21:03:23,471::task::495::TaskManager.Task::(_debug) Task 0e3e2ae3-83fe-4117-bdc5-1c3c1d8aed45: moving from state preparing -> state finished
>Thread-4922911::DEBUG::2013-11-19 21:03:23,472::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-4922911::DEBUG::2013-11-19 21:03:23,476::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-4922911::DEBUG::2013-11-19 21:03:23,476::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-4922911::DEBUG::2013-11-19 21:03:23,477::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-4922911::DEBUG::2013-11-19 21:03:23,478::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-4922911::DEBUG::2013-11-19 21:03:23,479::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-4922911::DEBUG::2013-11-19 21:03:23,480::task::495::TaskManager.Task::(_debug) Task 0e3e2ae3-83fe-4117-bdc5-1c3c1d8aed45: ref 0 aborting False
>Thread-4922911::INFO::2013-11-19 21:03:23,485::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-4922911::INFO::2013-11-19 21:03:23,487::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-4922911::DEBUG::2013-11-19 21:03:23,488::task::495::TaskManager.Task::(_debug) Task 28d6a99e-f359-4b5b-b997-8f926d8489d0: moving from state init -> state preparing
>Thread-4922911::DEBUG::2013-11-19 21:03:23,494::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2a0bcb50-b4b3-4f80-9cc9-efb6df195dbc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-4922911::DEBUG::2013-11-19 21:03:23,495::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-4922911::DEBUG::2013-11-19 21:03:23,503::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-4922911::DEBUG::2013-11-19 21:03:23,503::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2a0bcb50-b4b3-4f80-9cc9-efb6df195dbc`::Granted request
>Thread-4922911::DEBUG::2013-11-19 21:03:23,508::task::495::TaskManager.Task::(_debug) Task 28d6a99e-f359-4b5b-b997-8f926d8489d0: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-4922911::DEBUG::2013-11-19 21:03:23,508::task::495::TaskManager.Task::(_debug) Task 28d6a99e-f359-4b5b-b997-8f926d8489d0: ref 1 aborting False
>Thread-4922911::DEBUG::2013-11-19 21:03:23,509::task::495::TaskManager.Task::(_debug) Task 28d6a99e-f359-4b5b-b997-8f926d8489d0: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-4922911::DEBUG::2013-11-19 21:03:23,510::task::495::TaskManager.Task::(_debug) Task 28d6a99e-f359-4b5b-b997-8f926d8489d0: moving from state preparing -> state finished
>Thread-4922911::DEBUG::2013-11-19 21:03:23,511::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-4922911::DEBUG::2013-11-19 21:03:23,516::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-4922911::DEBUG::2013-11-19 21:03:23,521::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-4922911::DEBUG::2013-11-19 21:03:23,522::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-4922911::DEBUG::2013-11-19 21:03:23,522::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-4922911::DEBUG::2013-11-19 21:03:23,523::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-4922911::DEBUG::2013-11-19 21:03:23,523::task::495::TaskManager.Task::(_debug) Task 28d6a99e-f359-4b5b-b997-8f926d8489d0: ref 0 aborting False
>Thread-4922911::INFO::2013-11-19 21:03:23,528::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-4922911::INFO::2013-11-19 21:03:23,531::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-4922911::DEBUG::2013-11-19 21:03:23,534::task::495::TaskManager.Task::(_debug) Task 4f59f614-8116-40cf-8730-200d66774124: moving from state init -> state preparing
>Thread-4922911::DEBUG::2013-11-19 21:03:23,541::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`463cf201-208a-48f2-9690-c12a5c001bb4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-4922911::DEBUG::2013-11-19 21:03:23,542::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-4922911::DEBUG::2013-11-19 21:03:23,543::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-4922911::DEBUG::2013-11-19 21:03:23,551::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`463cf201-208a-48f2-9690-c12a5c001bb4`::Granted request
>Thread-4922911::DEBUG::2013-11-19 21:03:23,556::task::495::TaskManager.Task::(_debug) Task 4f59f614-8116-40cf-8730-200d66774124: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-4922911::DEBUG::2013-11-19 21:03:23,560::task::495::TaskManager.Task::(_debug) Task 4f59f614-8116-40cf-8730-200d66774124: ref 1 aborting False
>Thread-4922911::DEBUG::2013-11-19 21:03:23,561::task::495::TaskManager.Task::(_debug) Task 4f59f614-8116-40cf-8730-200d66774124: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-4922911::DEBUG::2013-11-19 21:03:23,568::task::495::TaskManager.Task::(_debug) Task 4f59f614-8116-40cf-8730-200d66774124: moving from state preparing -> state finished
>Thread-4922911::DEBUG::2013-11-19 21:03:23,568::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-4922911::DEBUG::2013-11-19 21:03:23,569::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-4922911::DEBUG::2013-11-19 21:03:23,569::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-4922911::DEBUG::2013-11-19 21:03:23,572::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-4922911::DEBUG::2013-11-19 21:03:23,572::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-4922911::DEBUG::2013-11-19 21:03:23,578::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-4922911::DEBUG::2013-11-19 21:03:23,579::task::495::TaskManager.Task::(_debug) Task 4f59f614-8116-40cf-8730-200d66774124: ref 0 aborting False
>Thread-4922911::INFO::2013-11-19 21:03:23,579::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>libvirtEventLoop::DEBUG::2013-11-19 21:03:23,729::libvirtvm::1401::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::event Saved detail 0 opaque None
>libvirtEventLoop::DEBUG::2013-11-19 21:03:24,837::libvirtvm::1401::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::event Stopped detail 0 opaque None
>libvirtEventLoop::INFO::2013-11-19 21:03:24,839::vm::445::vm.Vm::(_onQemuDeath) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::underlying process disconnected
>Thread-15910104::DEBUG::2013-11-19 21:03:24,840::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>libvirtEventLoop::INFO::2013-11-19 21:03:24,841::libvirtvm::1304::vm.Vm::(releaseVm) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Release VM resources
>Thread-15910104::INFO::2013-11-19 21:03:24,841::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910104::DEBUG::2013-11-19 21:03:24,843::task::495::TaskManager.Task::(_debug) Task 56f08a01-e922-42d7-ba70-a0e7bcdeecb2: moving from state init -> state preparing
>Thread-15910104::DEBUG::2013-11-19 21:03:24,845::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=274 spmId=2
>Thread-15910104::DEBUG::2013-11-19 21:03:24,846::task::495::TaskManager.Task::(_debug) Task 56f08a01-e922-42d7-ba70-a0e7bcdeecb2: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910104::DEBUG::2013-11-19 21:03:24,847::task::495::TaskManager.Task::(_debug) Task 56f08a01-e922-42d7-ba70-a0e7bcdeecb2: moving from state preparing -> state finished
>Thread-15910104::DEBUG::2013-11-19 21:03:24,849::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910104::DEBUG::2013-11-19 21:03:24,850::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910104::DEBUG::2013-11-19 21:03:24,850::task::495::TaskManager.Task::(_debug) Task 56f08a01-e922-42d7-ba70-a0e7bcdeecb2: ref 0 aborting False
>Thread-15910104::INFO::2013-11-19 21:03:24,851::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>libvirtEventLoop::DEBUG::2013-11-19 21:03:24,852::utils::314::vm.Vm::(stop) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Stop statistics collection
>Thread-15909534::DEBUG::2013-11-19 21:03:24,853::utils::345::vm.Vm::(run) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Stats thread finished
>libvirtEventLoop::WARNING::2013-11-19 21:03:24,856::libvirtvm::1315::vm.Vm::(releaseVm) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Traceback (most recent call last):
>  File "/usr/share/vdsm/libvirtvm.py", line 1312, in releaseVm
>  File "/usr/share/vdsm/libvirtvm.py", line 445, in f
>  File "/usr/share/vdsm/libvirtconnection.py", line 63, in wrapper
>  File "/usr/lib64/python2.6/site-packages/libvirt.py", line 672, in destroy
>libvirtError: Domain not found: no domain with matching uuid '4760db3e-d3a8-4045-9153-12c36ec16ba2'
>
>Thread-15910105::DEBUG::2013-11-19 21:03:24,857::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>libvirtEventLoop::DEBUG::2013-11-19 21:03:24,858::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /sbin/service ksmtuned retune' (cwd None)
>Thread-15910105::INFO::2013-11-19 21:03:24,859::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910105::DEBUG::2013-11-19 21:03:24,860::task::495::TaskManager.Task::(_debug) Task 39066e94-d8f7-4914-84d2-db7ac98e707b: moving from state init -> state preparing
>Thread-15910105::DEBUG::2013-11-19 21:03:25,067::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`26cd29de-63fc-48a8-8b13-a5407383fcce`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910105::DEBUG::2013-11-19 21:03:25,068::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910105::DEBUG::2013-11-19 21:03:25,070::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910105::DEBUG::2013-11-19 21:03:25,070::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`26cd29de-63fc-48a8-8b13-a5407383fcce`::Granted request
>Thread-15910105::DEBUG::2013-11-19 21:03:25,072::task::495::TaskManager.Task::(_debug) Task 39066e94-d8f7-4914-84d2-db7ac98e707b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910105::DEBUG::2013-11-19 21:03:25,072::task::495::TaskManager.Task::(_debug) Task 39066e94-d8f7-4914-84d2-db7ac98e707b: ref 1 aborting False
>Thread-15910105::WARNING::2013-11-19 21:03:25,076::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910105::DEBUG::2013-11-19 21:03:25,077::task::495::TaskManager.Task::(_debug) Task 39066e94-d8f7-4914-84d2-db7ac98e707b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910105::DEBUG::2013-11-19 21:03:25,078::task::495::TaskManager.Task::(_debug) Task 39066e94-d8f7-4914-84d2-db7ac98e707b: moving from state preparing -> state finished
>Thread-15910105::DEBUG::2013-11-19 21:03:25,079::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910105::DEBUG::2013-11-19 21:03:25,079::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910105::DEBUG::2013-11-19 21:03:25,080::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910105::DEBUG::2013-11-19 21:03:25,081::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910105::DEBUG::2013-11-19 21:03:25,081::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910105::DEBUG::2013-11-19 21:03:25,082::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910105::DEBUG::2013-11-19 21:03:25,083::task::495::TaskManager.Task::(_debug) Task 39066e94-d8f7-4914-84d2-db7ac98e707b: ref 0 aborting False
>Thread-15910105::INFO::2013-11-19 21:03:25,083::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15909965::DEBUG::2013-11-19 21:03:25,110::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>libvirtEventLoop::DEBUG::2013-11-19 21:03:25,354::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-19 21:03:25,357::vm::718::vm.Vm::(_cleanup) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Drive None cleanup
>libvirtEventLoop::DEBUG::2013-11-19 21:03:25,358::vm::718::vm.Vm::(_cleanup) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Drive /rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/6f97dfa5-ee35-4f37-8495-1e6b3136b92c/5eaa8dde-10d8-45b5-9e10-5417c297c4be cleanup
>libvirtEventLoop::INFO::2013-11-19 21:03:25,361::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: teardownVolume, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>libvirtEventLoop::DEBUG::2013-11-19 21:03:25,362::task::495::TaskManager.Task::(_debug) Task 6fbeb9b9-3fb8-4453-8a4a-1ed471197e23: moving from state init -> state preparing
>libvirtEventLoop::DEBUG::2013-11-19 21:03:25,363::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5556e88c-0c68-40db-94e7-0bf9a22e58a4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1727' at 'public_teardownVolume'
>libvirtEventLoop::DEBUG::2013-11-19 21:03:25,364::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>libvirtEventLoop::DEBUG::2013-11-19 21:03:25,366::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>libvirtEventLoop::DEBUG::2013-11-19 21:03:25,366::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5556e88c-0c68-40db-94e7-0bf9a22e58a4`::Granted request
>libvirtEventLoop::DEBUG::2013-11-19 21:03:25,368::task::495::TaskManager.Task::(_debug) Task 6fbeb9b9-3fb8-4453-8a4a-1ed471197e23: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>libvirtEventLoop::DEBUG::2013-11-19 21:03:25,369::task::495::TaskManager.Task::(_debug) Task 6fbeb9b9-3fb8-4453-8a4a-1ed471197e23: ref 1 aborting False
>libvirtEventLoop::DEBUG::2013-11-19 21:03:25,370::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.6f97dfa5-ee35-4f37-8495-1e6b3136b92c'
>libvirtEventLoop::DEBUG::2013-11-19 21:03:25,371::resourceManager::530::ResourceManager::(releaseResource) Released resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.6f97dfa5-ee35-4f37-8495-1e6b3136b92c' (0 active users)
>libvirtEventLoop::DEBUG::2013-11-19 21:03:25,372::resourceManager::535::ResourceManager::(releaseResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.6f97dfa5-ee35-4f37-8495-1e6b3136b92c' is free, finding out if anyone is waiting for it.
>libvirtEventLoop::DEBUG::2013-11-19 21:03:25,377::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.5eaa8dde-10d8-45b5-9e10-5417c297c4be'
>libvirtEventLoop::DEBUG::2013-11-19 21:03:25,378::resourceManager::530::ResourceManager::(releaseResource) Released resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.5eaa8dde-10d8-45b5-9e10-5417c297c4be' (0 active users)
>libvirtEventLoop::DEBUG::2013-11-19 21:03:25,379::resourceManager::535::ResourceManager::(releaseResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.5eaa8dde-10d8-45b5-9e10-5417c297c4be' is free, finding out if anyone is waiting for it.
>libvirtEventLoop::DEBUG::2013-11-19 21:03:25,379::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.5eaa8dde-10d8-45b5-9e10-5417c297c4be', Clearing records.
>libvirtEventLoop::DEBUG::2013-11-19 21:03:25,380::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.6f97dfa5-ee35-4f37-8495-1e6b3136b92c', Clearing records.
>libvirtEventLoop::INFO::2013-11-19 21:03:25,381::blockVolume::386::Storage.Volume::(teardown) Tearing down volume 6b892067-ac7e-4039-a7be-5b7387525b16/5eaa8dde-10d8-45b5-9e10-5417c297c4be justme False
>libvirtEventLoop::DEBUG::2013-11-19 21:03:25,381::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.5eaa8dde-10d8-45b5-9e10-5417c297c4be'
>libvirtEventLoop::DEBUG::2013-11-19 21:03:25,382::resourceManager::530::ResourceManager::(releaseResource) Released resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.5eaa8dde-10d8-45b5-9e10-5417c297c4be' (0 active users)
>libvirtEventLoop::DEBUG::2013-11-19 21:03:25,383::resourceManager::535::ResourceManager::(releaseResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.5eaa8dde-10d8-45b5-9e10-5417c297c4be' is free, finding out if anyone is waiting for it.
>libvirtEventLoop::DEBUG::2013-11-19 21:03:25,384::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --autobackup n --available n 6b892067-ac7e-4039-a7be-5b7387525b16/5eaa8dde-10d8-45b5-9e10-5417c297c4be' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:03:25,762::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0883389 s, 11.6 MB/s\n'; <rc> = 0
>Thread-15910106::INFO::2013-11-19 21:03:26,223::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910106::DEBUG::2013-11-19 21:03:26,224::task::495::TaskManager.Task::(_debug) Task 79fa6f6f-4503-4ad5-b0ae-fe25057ee66f: moving from state init -> state preparing
>Thread-15910106::DEBUG::2013-11-19 21:03:26,235::task::495::TaskManager.Task::(_debug) Task 79fa6f6f-4503-4ad5-b0ae-fe25057ee66f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000423192977905', 'lastCheck': 1384894998.062989, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00182700157166', 'lastCheck': 1384895004.77634, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384894999.6098909, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00100803375244', 'lastCheck': 1384894997.6138971, 'valid': True, 'code': 0}}
>Thread-15910106::DEBUG::2013-11-19 21:03:26,247::task::495::TaskManager.Task::(_debug) Task 79fa6f6f-4503-4ad5-b0ae-fe25057ee66f: moving from state preparing -> state finished
>libvirtEventLoop::DEBUG::2013-11-19 21:03:26,248::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15910106::DEBUG::2013-11-19 21:03:26,252::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>libvirtEventLoop::DEBUG::2013-11-19 21:03:26,253::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15910106::DEBUG::2013-11-19 21:03:26,254::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>libvirtEventLoop::DEBUG::2013-11-19 21:03:26,254::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15910106::DEBUG::2013-11-19 21:03:26,255::task::495::TaskManager.Task::(_debug) Task 79fa6f6f-4503-4ad5-b0ae-fe25057ee66f: ref 0 aborting False
>libvirtEventLoop::DEBUG::2013-11-19 21:03:26,256::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.5eaa8dde-10d8-45b5-9e10-5417c297c4be', Clearing records.
>Thread-15910106::INFO::2013-11-19 21:03:26,257::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000423192977905', 'lastCheck': 1384894998.062989, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00182700157166', 'lastCheck': 1384895004.77634, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384894999.6098909, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00100803375244', 'lastCheck': 1384894997.6138971, 'valid': True, 'code': 0}}
>libvirtEventLoop::DEBUG::2013-11-19 21:03:26,259::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>libvirtEventLoop::DEBUG::2013-11-19 21:03:26,261::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>libvirtEventLoop::DEBUG::2013-11-19 21:03:27,135::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-19 21:03:27,341::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>libvirtEventLoop::DEBUG::2013-11-19 21:03:27,342::task::495::TaskManager.Task::(_debug) Task 6fbeb9b9-3fb8-4453-8a4a-1ed471197e23: finished: None
>libvirtEventLoop::DEBUG::2013-11-19 21:03:27,342::task::495::TaskManager.Task::(_debug) Task 6fbeb9b9-3fb8-4453-8a4a-1ed471197e23: moving from state preparing -> state finished
>libvirtEventLoop::DEBUG::2013-11-19 21:03:27,343::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>libvirtEventLoop::DEBUG::2013-11-19 21:03:27,343::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>libvirtEventLoop::DEBUG::2013-11-19 21:03:27,343::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>libvirtEventLoop::DEBUG::2013-11-19 21:03:27,344::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>libvirtEventLoop::DEBUG::2013-11-19 21:03:27,344::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>libvirtEventLoop::DEBUG::2013-11-19 21:03:27,344::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>libvirtEventLoop::DEBUG::2013-11-19 21:03:27,345::task::495::TaskManager.Task::(_debug) Task 6fbeb9b9-3fb8-4453-8a4a-1ed471197e23: ref 0 aborting False
>libvirtEventLoop::INFO::2013-11-19 21:03:27,345::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: teardownVolume, Return response: {'status': {'message': 'OK', 'code': 0}}
>libvirtEventLoop::DEBUG::2013-11-19 21:03:27,346::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/libexec/vdsm/hooks/after_vm_destroy/50_vhostmd' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:03:27,773::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:03:28,231::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0460724 s, 22.2 MB/s\n'; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-19 21:03:28,243::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>libvirtEventLoop::INFO::2013-11-19 21:03:28,244::hooks::51::root::(_runHooksDir) 
>libvirtEventLoop::DEBUG::2013-11-19 21:03:28,248::vm::742::vm.Vm::(setDownStatus) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Changed state to Down: User shut down
>libvirtEventLoop::DEBUG::2013-11-19 21:03:28,248::utils::314::vm.Vm::(stop) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Stop statistics collection
>Thread-15910110::DEBUG::2013-11-19 21:03:28,633::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('4760db3e-d3a8-4045-9153-12c36ec16ba2',) {}
>Thread-15910110::DEBUG::2013-11-19 21:03:28,633::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Down', 'timeOffset': -1L, 'vmId': '4760db3e-d3a8-4045-9153-12c36ec16ba2', 'exitMessage': 'User shut down', 'exitCode': 0}]}
>Thread-15910111::DEBUG::2013-11-19 21:03:28,641::clientIF::54::vds::(wrapper) [192.168.100.135]::call destroy with ('4760db3e-d3a8-4045-9153-12c36ec16ba2',) {}
>Thread-15910111::INFO::2013-11-19 21:03:28,641::clientIF::450::vds::(destroy) vmContainerLock acquired by vm 4760db3e-d3a8-4045-9153-12c36ec16ba2
>Thread-15910111::DEBUG::2013-11-19 21:03:28,642::libvirtvm::1344::vm.Vm::(destroy) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::destroy Called
>Thread-15910111::DEBUG::2013-11-19 21:03:28,643::libvirtvm::1339::vm.Vm::(deleteVm) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Total desktops after destroy of 4760db3e-d3a8-4045-9153-12c36ec16ba2 is 1
>Thread-15910111::DEBUG::2013-11-19 21:03:28,643::clientIF::59::vds::(wrapper) return destroy with {'status': {'message': 'Machine destroyed', 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:03:29,780::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15909965::DEBUG::2013-11-19 21:03:30,241::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:03:30,763::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0787295 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15909965::DEBUG::2013-11-19 21:03:32,777::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:03:33,328::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0752112 s, 13.6 MB/s\n'; <rc> = 0
>Thread-15910115::DEBUG::2013-11-19 21:03:35,186::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910115::INFO::2013-11-19 21:03:35,188::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910115::DEBUG::2013-11-19 21:03:35,188::task::495::TaskManager.Task::(_debug) Task 5c1ac5db-09ad-4fa4-8943-93899127788c: moving from state init -> state preparing
>Thread-15910115::DEBUG::2013-11-19 21:03:35,189::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=274 spmId=2
>Thread-15910115::DEBUG::2013-11-19 21:03:35,190::task::495::TaskManager.Task::(_debug) Task 5c1ac5db-09ad-4fa4-8943-93899127788c: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910115::DEBUG::2013-11-19 21:03:35,190::task::495::TaskManager.Task::(_debug) Task 5c1ac5db-09ad-4fa4-8943-93899127788c: moving from state preparing -> state finished
>Thread-15910115::DEBUG::2013-11-19 21:03:35,191::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910115::DEBUG::2013-11-19 21:03:35,191::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910115::DEBUG::2013-11-19 21:03:35,192::task::495::TaskManager.Task::(_debug) Task 5c1ac5db-09ad-4fa4-8943-93899127788c: ref 0 aborting False
>Thread-15910115::INFO::2013-11-19 21:03:35,192::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910116::DEBUG::2013-11-19 21:03:35,197::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910116::INFO::2013-11-19 21:03:35,197::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910116::DEBUG::2013-11-19 21:03:35,198::task::495::TaskManager.Task::(_debug) Task 9c043bbd-0313-4b7a-943e-f78722ee2b01: moving from state init -> state preparing
>Thread-15910116::DEBUG::2013-11-19 21:03:35,199::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`cc4aba83-e4c6-413c-8e4f-1b404fc65895`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910116::DEBUG::2013-11-19 21:03:35,200::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910116::DEBUG::2013-11-19 21:03:35,201::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910116::DEBUG::2013-11-19 21:03:35,201::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`cc4aba83-e4c6-413c-8e4f-1b404fc65895`::Granted request
>Thread-15910116::DEBUG::2013-11-19 21:03:35,202::task::495::TaskManager.Task::(_debug) Task 9c043bbd-0313-4b7a-943e-f78722ee2b01: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910116::DEBUG::2013-11-19 21:03:35,203::task::495::TaskManager.Task::(_debug) Task 9c043bbd-0313-4b7a-943e-f78722ee2b01: ref 1 aborting False
>Thread-15910116::WARNING::2013-11-19 21:03:35,205::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910116::DEBUG::2013-11-19 21:03:35,205::task::495::TaskManager.Task::(_debug) Task 9c043bbd-0313-4b7a-943e-f78722ee2b01: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910116::DEBUG::2013-11-19 21:03:35,206::task::495::TaskManager.Task::(_debug) Task 9c043bbd-0313-4b7a-943e-f78722ee2b01: moving from state preparing -> state finished
>Thread-15910116::DEBUG::2013-11-19 21:03:35,207::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910116::DEBUG::2013-11-19 21:03:35,207::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910116::DEBUG::2013-11-19 21:03:35,208::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910116::DEBUG::2013-11-19 21:03:35,209::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910116::DEBUG::2013-11-19 21:03:35,209::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910116::DEBUG::2013-11-19 21:03:35,210::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910116::DEBUG::2013-11-19 21:03:35,210::task::495::TaskManager.Task::(_debug) Task 9c043bbd-0313-4b7a-943e-f78722ee2b01: ref 0 aborting False
>Thread-15910116::INFO::2013-11-19 21:03:35,211::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15909965::DEBUG::2013-11-19 21:03:35,344::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:03:35,826::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0469543 s, 21.8 MB/s\n'; <rc> = 0
>Thread-15910117::INFO::2013-11-19 21:03:36,740::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910117::DEBUG::2013-11-19 21:03:36,741::task::495::TaskManager.Task::(_debug) Task 26a5dfc5-f8b8-4403-998a-b5345daa3636: moving from state init -> state preparing
>Thread-15910117::DEBUG::2013-11-19 21:03:36,742::task::495::TaskManager.Task::(_debug) Task 26a5dfc5-f8b8-4403-998a-b5345daa3636: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000458002090454', 'lastCheck': 1384895008.5292549, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00245499610901', 'lastCheck': 1384895015.010802, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895009.780951, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000570058822632', 'lastCheck': 1384895007.776001, 'valid': True, 'code': 0}}
>Thread-15910117::DEBUG::2013-11-19 21:03:36,743::task::495::TaskManager.Task::(_debug) Task 26a5dfc5-f8b8-4403-998a-b5345daa3636: moving from state preparing -> state finished
>Thread-15910117::DEBUG::2013-11-19 21:03:36,743::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910117::DEBUG::2013-11-19 21:03:36,744::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910117::DEBUG::2013-11-19 21:03:36,744::task::495::TaskManager.Task::(_debug) Task 26a5dfc5-f8b8-4403-998a-b5345daa3636: ref 0 aborting False
>Thread-15910117::INFO::2013-11-19 21:03:36,745::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000458002090454', 'lastCheck': 1384895008.5292549, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00245499610901', 'lastCheck': 1384895015.010802, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895009.780951, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000570058822632', 'lastCheck': 1384895007.776001, 'valid': True, 'code': 0}}
>Dummy-15909965::DEBUG::2013-11-19 21:03:37,837::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:03:38,377::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0740086 s, 13.8 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:03:40,064::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15909965::DEBUG::2013-11-19 21:03:40,391::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:03:40,913::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0741586 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15909965::DEBUG::2013-11-19 21:03:42,928::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:03:43,457::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0752537 s, 13.6 MB/s\n'; <rc> = 0
>Thread-15910123::DEBUG::2013-11-19 21:03:45,327::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910123::INFO::2013-11-19 21:03:45,328::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910123::DEBUG::2013-11-19 21:03:45,329::task::495::TaskManager.Task::(_debug) Task b961c98b-347a-4f2b-9c47-82cd8ada15bb: moving from state init -> state preparing
>Thread-15910123::DEBUG::2013-11-19 21:03:45,330::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=274 spmId=2
>Thread-15910123::DEBUG::2013-11-19 21:03:45,331::task::495::TaskManager.Task::(_debug) Task b961c98b-347a-4f2b-9c47-82cd8ada15bb: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910123::DEBUG::2013-11-19 21:03:45,331::task::495::TaskManager.Task::(_debug) Task b961c98b-347a-4f2b-9c47-82cd8ada15bb: moving from state preparing -> state finished
>Thread-15910123::DEBUG::2013-11-19 21:03:45,332::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910123::DEBUG::2013-11-19 21:03:45,332::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910123::DEBUG::2013-11-19 21:03:45,333::task::495::TaskManager.Task::(_debug) Task b961c98b-347a-4f2b-9c47-82cd8ada15bb: ref 0 aborting False
>Thread-15910123::INFO::2013-11-19 21:03:45,334::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910124::DEBUG::2013-11-19 21:03:45,338::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910124::INFO::2013-11-19 21:03:45,339::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910124::DEBUG::2013-11-19 21:03:45,340::task::495::TaskManager.Task::(_debug) Task 962c0d91-044b-47d3-a381-f147cca1e4ca: moving from state init -> state preparing
>Thread-15910124::DEBUG::2013-11-19 21:03:45,341::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3d8588c6-4150-4e8a-a7fc-71efa42a2a9a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910124::DEBUG::2013-11-19 21:03:45,342::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910124::DEBUG::2013-11-19 21:03:45,343::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910124::DEBUG::2013-11-19 21:03:45,343::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3d8588c6-4150-4e8a-a7fc-71efa42a2a9a`::Granted request
>Thread-15910124::DEBUG::2013-11-19 21:03:45,344::task::495::TaskManager.Task::(_debug) Task 962c0d91-044b-47d3-a381-f147cca1e4ca: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910124::DEBUG::2013-11-19 21:03:45,345::task::495::TaskManager.Task::(_debug) Task 962c0d91-044b-47d3-a381-f147cca1e4ca: ref 1 aborting False
>Thread-15910124::WARNING::2013-11-19 21:03:45,347::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910124::DEBUG::2013-11-19 21:03:45,348::task::495::TaskManager.Task::(_debug) Task 962c0d91-044b-47d3-a381-f147cca1e4ca: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910124::DEBUG::2013-11-19 21:03:45,349::task::495::TaskManager.Task::(_debug) Task 962c0d91-044b-47d3-a381-f147cca1e4ca: moving from state preparing -> state finished
>Thread-15910124::DEBUG::2013-11-19 21:03:45,349::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910124::DEBUG::2013-11-19 21:03:45,350::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910124::DEBUG::2013-11-19 21:03:45,350::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910124::DEBUG::2013-11-19 21:03:45,351::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910124::DEBUG::2013-11-19 21:03:45,352::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910124::DEBUG::2013-11-19 21:03:45,352::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910124::DEBUG::2013-11-19 21:03:45,353::task::495::TaskManager.Task::(_debug) Task 962c0d91-044b-47d3-a381-f147cca1e4ca: ref 0 aborting False
>Thread-15910124::INFO::2013-11-19 21:03:45,353::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15909965::DEBUG::2013-11-19 21:03:45,473::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:03:45,972::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0453893 s, 22.6 MB/s\n'; <rc> = 0
>Thread-15910125::INFO::2013-11-19 21:03:46,890::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910125::DEBUG::2013-11-19 21:03:46,890::task::495::TaskManager.Task::(_debug) Task f23c6d0a-d074-4037-b164-ebc5226585f2: moving from state init -> state preparing
>Thread-15910125::DEBUG::2013-11-19 21:03:46,891::task::495::TaskManager.Task::(_debug) Task f23c6d0a-d074-4037-b164-ebc5226585f2: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000286817550659', 'lastCheck': 1384895018.574862, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00144410133362', 'lastCheck': 1384895025.1583879, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895020.0649731, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000739097595215', 'lastCheck': 1384895018.0603249, 'valid': True, 'code': 0}}
>Thread-15910125::DEBUG::2013-11-19 21:03:46,892::task::495::TaskManager.Task::(_debug) Task f23c6d0a-d074-4037-b164-ebc5226585f2: moving from state preparing -> state finished
>Thread-15910125::DEBUG::2013-11-19 21:03:46,893::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910125::DEBUG::2013-11-19 21:03:46,893::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910125::DEBUG::2013-11-19 21:03:46,894::task::495::TaskManager.Task::(_debug) Task f23c6d0a-d074-4037-b164-ebc5226585f2: ref 0 aborting False
>Thread-15910125::INFO::2013-11-19 21:03:46,894::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000286817550659', 'lastCheck': 1384895018.574862, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00144410133362', 'lastCheck': 1384895025.1583879, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895020.0649731, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000739097595215', 'lastCheck': 1384895018.0603249, 'valid': True, 'code': 0}}
>Dummy-15909965::DEBUG::2013-11-19 21:03:47,983::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:03:48,535::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0724856 s, 14.1 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:03:50,223::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15909965::DEBUG::2013-11-19 21:03:50,550::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:03:51,126::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.08746 s, 11.7 MB/s\n'; <rc> = 0
>Dummy-15909965::DEBUG::2013-11-19 21:03:53,140::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:03:53,666::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0705313 s, 14.5 MB/s\n'; <rc> = 0
>Thread-15910131::DEBUG::2013-11-19 21:03:55,459::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910131::INFO::2013-11-19 21:03:55,461::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910131::DEBUG::2013-11-19 21:03:55,461::task::495::TaskManager.Task::(_debug) Task d47a6308-b704-4ccf-aec4-8006623cff11: moving from state init -> state preparing
>Thread-15910131::DEBUG::2013-11-19 21:03:55,462::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=274 spmId=2
>Thread-15910131::DEBUG::2013-11-19 21:03:55,463::task::495::TaskManager.Task::(_debug) Task d47a6308-b704-4ccf-aec4-8006623cff11: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910131::DEBUG::2013-11-19 21:03:55,463::task::495::TaskManager.Task::(_debug) Task d47a6308-b704-4ccf-aec4-8006623cff11: moving from state preparing -> state finished
>Thread-15910131::DEBUG::2013-11-19 21:03:55,464::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910131::DEBUG::2013-11-19 21:03:55,465::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910131::DEBUG::2013-11-19 21:03:55,465::task::495::TaskManager.Task::(_debug) Task d47a6308-b704-4ccf-aec4-8006623cff11: ref 0 aborting False
>Thread-15910131::INFO::2013-11-19 21:03:55,466::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910132::DEBUG::2013-11-19 21:03:55,470::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910132::INFO::2013-11-19 21:03:55,471::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910132::DEBUG::2013-11-19 21:03:55,472::task::495::TaskManager.Task::(_debug) Task 93d5fbf8-3af6-4030-9c27-f2cace062c7a: moving from state init -> state preparing
>Thread-15910132::DEBUG::2013-11-19 21:03:55,473::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`733f130d-f94c-4de0-bfbc-176bf53edef4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910132::DEBUG::2013-11-19 21:03:55,473::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910132::DEBUG::2013-11-19 21:03:55,474::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910132::DEBUG::2013-11-19 21:03:55,475::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`733f130d-f94c-4de0-bfbc-176bf53edef4`::Granted request
>Thread-15910132::DEBUG::2013-11-19 21:03:55,476::task::495::TaskManager.Task::(_debug) Task 93d5fbf8-3af6-4030-9c27-f2cace062c7a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910132::DEBUG::2013-11-19 21:03:55,476::task::495::TaskManager.Task::(_debug) Task 93d5fbf8-3af6-4030-9c27-f2cace062c7a: ref 1 aborting False
>Thread-15910132::WARNING::2013-11-19 21:03:55,479::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910132::DEBUG::2013-11-19 21:03:55,479::task::495::TaskManager.Task::(_debug) Task 93d5fbf8-3af6-4030-9c27-f2cace062c7a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910132::DEBUG::2013-11-19 21:03:55,480::task::495::TaskManager.Task::(_debug) Task 93d5fbf8-3af6-4030-9c27-f2cace062c7a: moving from state preparing -> state finished
>Thread-15910132::DEBUG::2013-11-19 21:03:55,481::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910132::DEBUG::2013-11-19 21:03:55,481::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910132::DEBUG::2013-11-19 21:03:55,482::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910132::DEBUG::2013-11-19 21:03:55,483::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910132::DEBUG::2013-11-19 21:03:55,483::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910132::DEBUG::2013-11-19 21:03:55,484::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910132::DEBUG::2013-11-19 21:03:55,485::task::495::TaskManager.Task::(_debug) Task 93d5fbf8-3af6-4030-9c27-f2cace062c7a: ref 0 aborting False
>Thread-15910132::INFO::2013-11-19 21:03:55,485::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15909965::DEBUG::2013-11-19 21:03:55,681::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:03:56,159::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0499763 s, 20.5 MB/s\n'; <rc> = 0
>Thread-15910133::INFO::2013-11-19 21:03:57,105::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910133::DEBUG::2013-11-19 21:03:57,106::task::495::TaskManager.Task::(_debug) Task 6c59b50d-f575-4ed4-a595-9fa80f3b38d6: moving from state init -> state preparing
>Thread-15910133::DEBUG::2013-11-19 21:03:57,107::task::495::TaskManager.Task::(_debug) Task 6c59b50d-f575-4ed4-a595-9fa80f3b38d6: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000808954238892', 'lastCheck': 1384895028.7077301, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00134301185608', 'lastCheck': 1384895035.3667779, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895030.2249441, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00803112983704', 'lastCheck': 1384895028.241358, 'valid': True, 'code': 0}}
>Thread-15910133::DEBUG::2013-11-19 21:03:57,108::task::495::TaskManager.Task::(_debug) Task 6c59b50d-f575-4ed4-a595-9fa80f3b38d6: moving from state preparing -> state finished
>Thread-15910133::DEBUG::2013-11-19 21:03:57,109::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910133::DEBUG::2013-11-19 21:03:57,109::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910133::DEBUG::2013-11-19 21:03:57,110::task::495::TaskManager.Task::(_debug) Task 6c59b50d-f575-4ed4-a595-9fa80f3b38d6: ref 0 aborting False
>Thread-15910133::INFO::2013-11-19 21:03:57,110::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000808954238892', 'lastCheck': 1384895028.7077301, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00134301185608', 'lastCheck': 1384895035.3667779, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895030.2249441, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00803112983704', 'lastCheck': 1384895028.241358, 'valid': True, 'code': 0}}
>Dummy-15909965::DEBUG::2013-11-19 21:03:58,170::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:03:58,734::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0792493 s, 12.9 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:04:00,396::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15909965::DEBUG::2013-11-19 21:04:00,749::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:04:01,281::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0737496 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15909965::DEBUG::2013-11-19 21:04:03,295::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:04:03,814::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0577656 s, 17.7 MB/s\n'; <rc> = 0
>Thread-15910139::DEBUG::2013-11-19 21:04:05,592::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910139::INFO::2013-11-19 21:04:05,594::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910139::DEBUG::2013-11-19 21:04:05,594::task::495::TaskManager.Task::(_debug) Task 9268ad65-ac27-4976-899b-410714ff06ae: moving from state init -> state preparing
>Thread-15910139::DEBUG::2013-11-19 21:04:05,603::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=274 spmId=2
>Thread-15910139::DEBUG::2013-11-19 21:04:05,603::task::495::TaskManager.Task::(_debug) Task 9268ad65-ac27-4976-899b-410714ff06ae: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910139::DEBUG::2013-11-19 21:04:05,604::task::495::TaskManager.Task::(_debug) Task 9268ad65-ac27-4976-899b-410714ff06ae: moving from state preparing -> state finished
>Thread-15910139::DEBUG::2013-11-19 21:04:05,604::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910139::DEBUG::2013-11-19 21:04:05,605::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910139::DEBUG::2013-11-19 21:04:05,606::task::495::TaskManager.Task::(_debug) Task 9268ad65-ac27-4976-899b-410714ff06ae: ref 0 aborting False
>Thread-15910139::INFO::2013-11-19 21:04:05,606::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910140::DEBUG::2013-11-19 21:04:05,611::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910140::INFO::2013-11-19 21:04:05,611::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910140::DEBUG::2013-11-19 21:04:05,612::task::495::TaskManager.Task::(_debug) Task f7d7b021-6c79-41a6-aee2-71fb1fa65bb7: moving from state init -> state preparing
>Thread-15910140::DEBUG::2013-11-19 21:04:05,613::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1a1b60fb-263b-4793-ac86-80e496ba8b55`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910140::DEBUG::2013-11-19 21:04:05,614::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910140::DEBUG::2013-11-19 21:04:05,615::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910140::DEBUG::2013-11-19 21:04:05,615::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1a1b60fb-263b-4793-ac86-80e496ba8b55`::Granted request
>Thread-15910140::DEBUG::2013-11-19 21:04:05,616::task::495::TaskManager.Task::(_debug) Task f7d7b021-6c79-41a6-aee2-71fb1fa65bb7: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910140::DEBUG::2013-11-19 21:04:05,617::task::495::TaskManager.Task::(_debug) Task f7d7b021-6c79-41a6-aee2-71fb1fa65bb7: ref 1 aborting False
>Thread-15910140::WARNING::2013-11-19 21:04:05,619::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910140::DEBUG::2013-11-19 21:04:05,620::task::495::TaskManager.Task::(_debug) Task f7d7b021-6c79-41a6-aee2-71fb1fa65bb7: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910140::DEBUG::2013-11-19 21:04:05,621::task::495::TaskManager.Task::(_debug) Task f7d7b021-6c79-41a6-aee2-71fb1fa65bb7: moving from state preparing -> state finished
>Thread-15910140::DEBUG::2013-11-19 21:04:05,621::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910140::DEBUG::2013-11-19 21:04:05,622::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910140::DEBUG::2013-11-19 21:04:05,623::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910140::DEBUG::2013-11-19 21:04:05,623::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910140::DEBUG::2013-11-19 21:04:05,624::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910140::DEBUG::2013-11-19 21:04:05,624::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910140::DEBUG::2013-11-19 21:04:05,625::task::495::TaskManager.Task::(_debug) Task f7d7b021-6c79-41a6-aee2-71fb1fa65bb7: ref 0 aborting False
>Thread-15910140::INFO::2013-11-19 21:04:05,626::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15909965::DEBUG::2013-11-19 21:04:05,829::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:04:06,412::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0930995 s, 11.0 MB/s\n'; <rc> = 0
>Thread-4922912::DEBUG::2013-11-19 21:04:06,503::guestIF::97::vm.Vm::(_handleMessage) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::u'None'
>Thread-15910141::INFO::2013-11-19 21:04:07,267::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910141::DEBUG::2013-11-19 21:04:07,268::task::495::TaskManager.Task::(_debug) Task 05bbb3cf-5776-49bb-ac6c-8bb18713e4a4: moving from state init -> state preparing
>Thread-15910141::DEBUG::2013-11-19 21:04:07,269::task::495::TaskManager.Task::(_debug) Task 05bbb3cf-5776-49bb-ac6c-8bb18713e4a4: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00046706199646', 'lastCheck': 1384895038.910903, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00137209892273', 'lastCheck': 1384895045.549921, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895040.396832, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000672817230225', 'lastCheck': 1384895038.3919809, 'valid': True, 'code': 0}}
>Thread-15910141::DEBUG::2013-11-19 21:04:07,269::task::495::TaskManager.Task::(_debug) Task 05bbb3cf-5776-49bb-ac6c-8bb18713e4a4: moving from state preparing -> state finished
>Thread-15910141::DEBUG::2013-11-19 21:04:07,270::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910141::DEBUG::2013-11-19 21:04:07,271::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910141::DEBUG::2013-11-19 21:04:07,271::task::495::TaskManager.Task::(_debug) Task 05bbb3cf-5776-49bb-ac6c-8bb18713e4a4: ref 0 aborting False
>Thread-15910141::INFO::2013-11-19 21:04:07,272::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00046706199646', 'lastCheck': 1384895038.910903, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00137209892273', 'lastCheck': 1384895045.549921, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895040.396832, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000672817230225', 'lastCheck': 1384895038.3919809, 'valid': True, 'code': 0}}
>Dummy-15909965::DEBUG::2013-11-19 21:04:08,427::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:04:08,954::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0708984 s, 14.4 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:04:10,669::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15909965::DEBUG::2013-11-19 21:04:10,970::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:04:11,450::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0632862 s, 16.2 MB/s\n'; <rc> = 0
>Dummy-15909965::DEBUG::2013-11-19 21:04:13,472::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:04:14,010::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0745248 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15910147::DEBUG::2013-11-19 21:04:15,741::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910147::INFO::2013-11-19 21:04:15,742::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910147::DEBUG::2013-11-19 21:04:15,743::task::495::TaskManager.Task::(_debug) Task 430a7f15-06cb-4dd4-971e-59c6afabbb34: moving from state init -> state preparing
>Thread-15910147::DEBUG::2013-11-19 21:04:15,744::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=274 spmId=2
>Thread-15910147::DEBUG::2013-11-19 21:04:15,744::task::495::TaskManager.Task::(_debug) Task 430a7f15-06cb-4dd4-971e-59c6afabbb34: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910147::DEBUG::2013-11-19 21:04:15,745::task::495::TaskManager.Task::(_debug) Task 430a7f15-06cb-4dd4-971e-59c6afabbb34: moving from state preparing -> state finished
>Thread-15910147::DEBUG::2013-11-19 21:04:15,745::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910147::DEBUG::2013-11-19 21:04:15,746::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910147::DEBUG::2013-11-19 21:04:15,747::task::495::TaskManager.Task::(_debug) Task 430a7f15-06cb-4dd4-971e-59c6afabbb34: ref 0 aborting False
>Thread-15910147::INFO::2013-11-19 21:04:15,747::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910148::DEBUG::2013-11-19 21:04:15,751::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910148::INFO::2013-11-19 21:04:15,752::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910148::DEBUG::2013-11-19 21:04:15,753::task::495::TaskManager.Task::(_debug) Task 1d9f9f2d-c587-4cf9-8fd0-a8051b8ede8a: moving from state init -> state preparing
>Thread-15910148::DEBUG::2013-11-19 21:04:15,754::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c587b635-00c3-46cd-a032-47b1e4bb5702`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910148::DEBUG::2013-11-19 21:04:15,755::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910148::DEBUG::2013-11-19 21:04:15,755::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910148::DEBUG::2013-11-19 21:04:15,756::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c587b635-00c3-46cd-a032-47b1e4bb5702`::Granted request
>Thread-15910148::DEBUG::2013-11-19 21:04:15,757::task::495::TaskManager.Task::(_debug) Task 1d9f9f2d-c587-4cf9-8fd0-a8051b8ede8a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910148::DEBUG::2013-11-19 21:04:15,758::task::495::TaskManager.Task::(_debug) Task 1d9f9f2d-c587-4cf9-8fd0-a8051b8ede8a: ref 1 aborting False
>Thread-15910148::WARNING::2013-11-19 21:04:15,760::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910148::DEBUG::2013-11-19 21:04:15,761::task::495::TaskManager.Task::(_debug) Task 1d9f9f2d-c587-4cf9-8fd0-a8051b8ede8a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910148::DEBUG::2013-11-19 21:04:15,762::task::495::TaskManager.Task::(_debug) Task 1d9f9f2d-c587-4cf9-8fd0-a8051b8ede8a: moving from state preparing -> state finished
>Thread-15910148::DEBUG::2013-11-19 21:04:15,762::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910148::DEBUG::2013-11-19 21:04:15,763::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910148::DEBUG::2013-11-19 21:04:15,763::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910148::DEBUG::2013-11-19 21:04:15,764::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910148::DEBUG::2013-11-19 21:04:15,765::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910148::DEBUG::2013-11-19 21:04:15,765::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910148::DEBUG::2013-11-19 21:04:15,766::task::495::TaskManager.Task::(_debug) Task 1d9f9f2d-c587-4cf9-8fd0-a8051b8ede8a: ref 0 aborting False
>Thread-15910148::INFO::2013-11-19 21:04:15,766::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15909965::DEBUG::2013-11-19 21:04:16,025::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:04:16,555::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0733971 s, 14.0 MB/s\n'; <rc> = 0
>Thread-15910149::INFO::2013-11-19 21:04:17,734::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910149::DEBUG::2013-11-19 21:04:17,735::task::495::TaskManager.Task::(_debug) Task 5a8672fd-e8d3-487f-a4d4-86df34edd0cd: moving from state init -> state preparing
>Thread-15910149::DEBUG::2013-11-19 21:04:17,736::task::495::TaskManager.Task::(_debug) Task 5a8672fd-e8d3-487f-a4d4-86df34edd0cd: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000461101531982', 'lastCheck': 1384895049.085433, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00112199783325', 'lastCheck': 1384895055.69243, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895050.6699841, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000598192214966', 'lastCheck': 1384895048.6668911, 'valid': True, 'code': 0}}
>Thread-15910149::DEBUG::2013-11-19 21:04:17,737::task::495::TaskManager.Task::(_debug) Task 5a8672fd-e8d3-487f-a4d4-86df34edd0cd: moving from state preparing -> state finished
>Thread-15910149::DEBUG::2013-11-19 21:04:17,738::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910149::DEBUG::2013-11-19 21:04:17,739::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910149::DEBUG::2013-11-19 21:04:17,739::task::495::TaskManager.Task::(_debug) Task 5a8672fd-e8d3-487f-a4d4-86df34edd0cd: ref 0 aborting False
>Thread-15910149::INFO::2013-11-19 21:04:17,740::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000461101531982', 'lastCheck': 1384895049.085433, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00112199783325', 'lastCheck': 1384895055.69243, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895050.6699841, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000598192214966', 'lastCheck': 1384895048.6668911, 'valid': True, 'code': 0}}
>Dummy-15909965::DEBUG::2013-11-19 21:04:18,569::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:04:19,132::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0743848 s, 13.8 MB/s\n'; <rc> = 0
>Thread-42838::DEBUG::2013-11-19 21:04:19,256::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 21:04:19,900::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:04:20,817::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15909965::DEBUG::2013-11-19 21:04:21,147::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15910152::DEBUG::2013-11-19 21:04:21,438::clientIF::54::vds::(wrapper) [192.168.100.135]::call shutdown with ('6574d88c-00ac-46f4-99ce-a2726932bfda', '30', 'The Red Hat Enterprise Virtualization Manager is shutting down this Virtual Machine') {}
>Thread-15910152::DEBUG::2013-11-19 21:04:21,441::vm::673::vm.Vm::(shutdown) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::guestAgent shutdown called
>Thread-15910152::DEBUG::2013-11-19 21:04:21,442::guestIF::179::vm.Vm::(desktopShutdown) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::desktopShutdown called
>Thread-15910152::DEBUG::2013-11-19 21:04:21,444::clientIF::59::vds::(wrapper) return shutdown with {'status': {'message': 'Machine shut down', 'code': 0}}
>Dummy-15909965::DEBUG::2013-11-19 21:04:21,710::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.071086 s, 14.4 MB/s\n'; <rc> = 0
>Thread-15910155::DEBUG::2013-11-19 21:04:21,945::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('6574d88c-00ac-46f4-99ce-a2726932bfda',) {}
>Thread-15910155::DEBUG::2013-11-19 21:04:21,946::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': u'None', 'memUsage': '92', 'acpiEnable': 'true', 'pid': '57291', 'displayPort': u'5908', 'session': 'UserLoggedOn', 'netIfaces': [{u'inet6': [], u'hw': u'00-1a-4a-a8-64-08', u'inet': [u'192.168.100.70'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': -1L, 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '38686343168', u'fs': u'NTFS'}, {u'path': u'h:\\', u'total': '107237863424', u'used': '99742478336', u'fs': u'NTFS'}, {u'path': u'l:\\', u'total': '150187474944', u'used': '130882863104', u'fs': u'NTFS'}, {u'path': u'x:\\', u'total': '429360349184', u'used': '330617978880', u'fs': u'NTFS'}], 'network': {u'vnet5': {'macAddr': u'00:1a:4a:a8:64:08', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet5'}}, 'displayType': 'qxl', 'guestName': u'KLEE-AXDB01', 'boot': 'cd', 'elapsedTime': '18663671', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Microsoft Report Viewer Redistributable 2008 SP1', u'Zabbix Agent 2.0.8', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'Microsoft SQL Server Compact 3.5 SP2 ENU', u'RHEV-Spice-Agent64', u'Microsoft Visual Studio Tools for Applications 2.0 - ENU', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'Microsoft Office 2003 Web Components', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10', u'Microsoft SQL Server Browser', u'Microsoft Report Viewer Redistributable 2008 (KB971119)', u'Microsoft SQL Server 2008 R2 Policies', u'Microsoft SQL Server Compact 3.5 SP2 Query Tools ENU'], 'guestOs': u'Unknown', 'nice': '', 'status': 'Powering down', 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'displayIp': '0', 'lastLogout': 1366231390.568368, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '64424509440', 'writeLatency': '306111', 'imageID': 'c4460480-32d3-469f-b0f4-c03bb6d74e41', 'flushLatency': '197517', 'readRate': '0.00', 'truesize': '64424509440', 'writeRate': '2352.91'}, u'vdc': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '107374182400', 'writeRate': '0.00'}, u'vdb': {'readLatency': '0', 'apparentsize': '150323855360', 'writeLatency': '371802', 'imageID': 'df8b790c-c762-4feb-9378-d64a321ce43c', 'flushLatency': '181637', 'readRate': '0.00', 'truesize': '150323855360', 'writeRate': '936.10'}, u'vdd': {'readLatency': '8479202', 'apparentsize': '429496729600', 'writeLatency': '0', 'imageID': 'd368307e-425e-423a-85b7-266c1e4a99ec', 'flushLatency': '0', 'readRate': '404.80', 'truesize': '429496729600', 'writeRate': '0.00'}}, 'monitorResponse': '0', 'statsAge': '1.49', 'cpuUser': '162.62', 'cpuIdle': '0.00', 'lastLogin': 1384895046.5037899, 'guestIPs': u'192.168.100.70', 'displaySecurePort': u'5909'}]}
>Dummy-15909965::DEBUG::2013-11-19 21:04:23,725::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15910157::DEBUG::2013-11-19 21:04:24,019::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('6574d88c-00ac-46f4-99ce-a2726932bfda',) {}
>Thread-15910157::DEBUG::2013-11-19 21:04:24,021::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': u'None', 'memUsage': '92', 'acpiEnable': 'true', 'pid': '57291', 'displayPort': u'5908', 'session': 'UserLoggedOn', 'netIfaces': [{u'inet6': [], u'hw': u'00-1a-4a-a8-64-08', u'inet': [u'192.168.100.70'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': -1L, 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '38686343168', u'fs': u'NTFS'}, {u'path': u'h:\\', u'total': '107237863424', u'used': '99742478336', u'fs': u'NTFS'}, {u'path': u'l:\\', u'total': '150187474944', u'used': '130882863104', u'fs': u'NTFS'}, {u'path': u'x:\\', u'total': '429360349184', u'used': '330617978880', u'fs': u'NTFS'}], 'network': {u'vnet5': {'macAddr': u'00:1a:4a:a8:64:08', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet5'}}, 'displayType': 'qxl', 'guestName': u'KLEE-AXDB01', 'boot': 'cd', 'elapsedTime': '18663673', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Microsoft Report Viewer Redistributable 2008 SP1', u'Zabbix Agent 2.0.8', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'Microsoft SQL Server Compact 3.5 SP2 ENU', u'RHEV-Spice-Agent64', u'Microsoft Visual Studio Tools for Applications 2.0 - ENU', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'Microsoft Office 2003 Web Components', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10', u'Microsoft SQL Server Browser', u'Microsoft Report Viewer Redistributable 2008 (KB971119)', u'Microsoft SQL Server 2008 R2 Policies', u'Microsoft SQL Server Compact 3.5 SP2 Query Tools ENU'], 'guestOs': u'Unknown', 'nice': '', 'status': 'Powering down', 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'displayIp': '0', 'lastLogout': 1366231390.568368, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '64424509440', 'writeLatency': '306111', 'imageID': 'c4460480-32d3-469f-b0f4-c03bb6d74e41', 'flushLatency': '197517', 'readRate': '0.00', 'truesize': '64424509440', 'writeRate': '2352.91'}, u'vdc': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '107374182400', 'writeRate': '0.00'}, u'vdb': {'readLatency': '0', 'apparentsize': '150323855360', 'writeLatency': '371802', 'imageID': 'df8b790c-c762-4feb-9378-d64a321ce43c', 'flushLatency': '181637', 'readRate': '0.00', 'truesize': '150323855360', 'writeRate': '936.10'}, u'vdd': {'readLatency': '8479202', 'apparentsize': '429496729600', 'writeLatency': '0', 'imageID': 'd368307e-425e-423a-85b7-266c1e4a99ec', 'flushLatency': '0', 'readRate': '404.80', 'truesize': '429496729600', 'writeRate': '0.00'}}, 'monitorResponse': '0', 'statsAge': '1.56', 'cpuUser': '162.62', 'cpuIdle': '0.00', 'lastLogin': 1384895046.5037899, 'guestIPs': u'192.168.100.70', 'displaySecurePort': u'5909'}]}
>Dummy-15909965::DEBUG::2013-11-19 21:04:24,264::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0739422 s, 13.8 MB/s\n'; <rc> = 0
>Thread-4922911::INFO::2013-11-19 21:04:24,460::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-4922911::DEBUG::2013-11-19 21:04:24,461::task::495::TaskManager.Task::(_debug) Task 7e9fb6db-dce4-4dce-b896-424fda534495: moving from state init -> state preparing
>Thread-4922911::DEBUG::2013-11-19 21:04:24,462::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fac99520-b517-4649-8aec-3a83cb430c41`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-4922911::DEBUG::2013-11-19 21:04:24,463::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-4922911::DEBUG::2013-11-19 21:04:24,464::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-4922911::DEBUG::2013-11-19 21:04:24,464::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fac99520-b517-4649-8aec-3a83cb430c41`::Granted request
>Thread-4922911::DEBUG::2013-11-19 21:04:24,465::task::495::TaskManager.Task::(_debug) Task 7e9fb6db-dce4-4dce-b896-424fda534495: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-4922911::DEBUG::2013-11-19 21:04:24,466::task::495::TaskManager.Task::(_debug) Task 7e9fb6db-dce4-4dce-b896-424fda534495: ref 1 aborting False
>Thread-4922911::DEBUG::2013-11-19 21:04:24,467::task::495::TaskManager.Task::(_debug) Task 7e9fb6db-dce4-4dce-b896-424fda534495: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-4922911::DEBUG::2013-11-19 21:04:24,467::task::495::TaskManager.Task::(_debug) Task 7e9fb6db-dce4-4dce-b896-424fda534495: moving from state preparing -> state finished
>Thread-4922911::DEBUG::2013-11-19 21:04:24,468::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-4922911::DEBUG::2013-11-19 21:04:24,468::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-4922911::DEBUG::2013-11-19 21:04:24,469::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-4922911::DEBUG::2013-11-19 21:04:24,470::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-4922911::DEBUG::2013-11-19 21:04:24,470::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-4922911::DEBUG::2013-11-19 21:04:24,470::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-4922911::DEBUG::2013-11-19 21:04:24,471::task::495::TaskManager.Task::(_debug) Task 7e9fb6db-dce4-4dce-b896-424fda534495: ref 0 aborting False
>Thread-4922911::INFO::2013-11-19 21:04:24,472::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-4922911::INFO::2013-11-19 21:04:24,472::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-4922911::DEBUG::2013-11-19 21:04:24,473::task::495::TaskManager.Task::(_debug) Task 49269880-962c-43c8-b8f0-8e0a4833dc74: moving from state init -> state preparing
>Thread-4922911::DEBUG::2013-11-19 21:04:24,474::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e2cc679f-b0dd-4944-9595-798b3a3993d4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-4922911::DEBUG::2013-11-19 21:04:24,475::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-4922911::DEBUG::2013-11-19 21:04:24,475::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-4922911::DEBUG::2013-11-19 21:04:24,476::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e2cc679f-b0dd-4944-9595-798b3a3993d4`::Granted request
>Thread-4922911::DEBUG::2013-11-19 21:04:24,477::task::495::TaskManager.Task::(_debug) Task 49269880-962c-43c8-b8f0-8e0a4833dc74: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-4922911::DEBUG::2013-11-19 21:04:24,477::task::495::TaskManager.Task::(_debug) Task 49269880-962c-43c8-b8f0-8e0a4833dc74: ref 1 aborting False
>Thread-4922911::DEBUG::2013-11-19 21:04:24,478::task::495::TaskManager.Task::(_debug) Task 49269880-962c-43c8-b8f0-8e0a4833dc74: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-4922911::DEBUG::2013-11-19 21:04:24,478::task::495::TaskManager.Task::(_debug) Task 49269880-962c-43c8-b8f0-8e0a4833dc74: moving from state preparing -> state finished
>Thread-4922911::DEBUG::2013-11-19 21:04:24,479::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-4922911::DEBUG::2013-11-19 21:04:24,479::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-4922911::DEBUG::2013-11-19 21:04:24,480::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-4922911::DEBUG::2013-11-19 21:04:24,481::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-4922911::DEBUG::2013-11-19 21:04:24,481::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-4922911::DEBUG::2013-11-19 21:04:24,482::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-4922911::DEBUG::2013-11-19 21:04:24,482::task::495::TaskManager.Task::(_debug) Task 49269880-962c-43c8-b8f0-8e0a4833dc74: ref 0 aborting False
>Thread-4922911::INFO::2013-11-19 21:04:24,483::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-4922911::INFO::2013-11-19 21:04:24,484::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-4922911::DEBUG::2013-11-19 21:04:24,484::task::495::TaskManager.Task::(_debug) Task 4f37c8fd-7353-4c83-b509-63cb9c2250de: moving from state init -> state preparing
>Thread-4922911::DEBUG::2013-11-19 21:04:24,485::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b7ad225b-954f-40fe-9cbc-791d92fd418c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-4922911::DEBUG::2013-11-19 21:04:24,486::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-4922911::DEBUG::2013-11-19 21:04:24,486::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-4922911::DEBUG::2013-11-19 21:04:24,487::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b7ad225b-954f-40fe-9cbc-791d92fd418c`::Granted request
>Thread-4922911::DEBUG::2013-11-19 21:04:24,488::task::495::TaskManager.Task::(_debug) Task 4f37c8fd-7353-4c83-b509-63cb9c2250de: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-4922911::DEBUG::2013-11-19 21:04:24,488::task::495::TaskManager.Task::(_debug) Task 4f37c8fd-7353-4c83-b509-63cb9c2250de: ref 1 aborting False
>Thread-4922911::DEBUG::2013-11-19 21:04:24,489::task::495::TaskManager.Task::(_debug) Task 4f37c8fd-7353-4c83-b509-63cb9c2250de: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-4922911::DEBUG::2013-11-19 21:04:24,489::task::495::TaskManager.Task::(_debug) Task 4f37c8fd-7353-4c83-b509-63cb9c2250de: moving from state preparing -> state finished
>Thread-4922911::DEBUG::2013-11-19 21:04:24,490::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-4922911::DEBUG::2013-11-19 21:04:24,490::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-4922911::DEBUG::2013-11-19 21:04:24,491::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-4922911::DEBUG::2013-11-19 21:04:24,492::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-4922911::DEBUG::2013-11-19 21:04:24,492::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-4922911::DEBUG::2013-11-19 21:04:24,493::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-4922911::DEBUG::2013-11-19 21:04:24,493::task::495::TaskManager.Task::(_debug) Task 4f37c8fd-7353-4c83-b509-63cb9c2250de: ref 0 aborting False
>Thread-4922911::INFO::2013-11-19 21:04:24,494::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-4922911::INFO::2013-11-19 21:04:24,494::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-4922911::DEBUG::2013-11-19 21:04:24,495::task::495::TaskManager.Task::(_debug) Task fc6ca8ba-2241-4bf5-bde8-06accb662ca0: moving from state init -> state preparing
>Thread-4922911::DEBUG::2013-11-19 21:04:24,496::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`bcea7f96-f18a-4957-a2be-a029d1c43b6e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-4922911::DEBUG::2013-11-19 21:04:24,497::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-4922911::DEBUG::2013-11-19 21:04:24,497::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-4922911::DEBUG::2013-11-19 21:04:24,498::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`bcea7f96-f18a-4957-a2be-a029d1c43b6e`::Granted request
>Thread-4922911::DEBUG::2013-11-19 21:04:24,499::task::495::TaskManager.Task::(_debug) Task fc6ca8ba-2241-4bf5-bde8-06accb662ca0: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-4922911::DEBUG::2013-11-19 21:04:24,499::task::495::TaskManager.Task::(_debug) Task fc6ca8ba-2241-4bf5-bde8-06accb662ca0: ref 1 aborting False
>Thread-4922911::DEBUG::2013-11-19 21:04:24,500::task::495::TaskManager.Task::(_debug) Task fc6ca8ba-2241-4bf5-bde8-06accb662ca0: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-4922911::DEBUG::2013-11-19 21:04:24,500::task::495::TaskManager.Task::(_debug) Task fc6ca8ba-2241-4bf5-bde8-06accb662ca0: moving from state preparing -> state finished
>Thread-4922911::DEBUG::2013-11-19 21:04:24,501::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-4922911::DEBUG::2013-11-19 21:04:24,501::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-4922911::DEBUG::2013-11-19 21:04:24,502::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-4922911::DEBUG::2013-11-19 21:04:24,503::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-4922911::DEBUG::2013-11-19 21:04:24,503::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-4922911::DEBUG::2013-11-19 21:04:24,504::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-4922911::DEBUG::2013-11-19 21:04:24,504::task::495::TaskManager.Task::(_debug) Task fc6ca8ba-2241-4bf5-bde8-06accb662ca0: ref 0 aborting False
>Thread-4922911::INFO::2013-11-19 21:04:24,505::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15910158::DEBUG::2013-11-19 21:04:25,891::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910158::INFO::2013-11-19 21:04:25,892::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910158::DEBUG::2013-11-19 21:04:25,892::task::495::TaskManager.Task::(_debug) Task 455407ee-994f-4170-88e4-25a4d5c75416: moving from state init -> state preparing
>Thread-15910158::DEBUG::2013-11-19 21:04:25,893::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=274 spmId=2
>Thread-15910158::DEBUG::2013-11-19 21:04:25,894::task::495::TaskManager.Task::(_debug) Task 455407ee-994f-4170-88e4-25a4d5c75416: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910158::DEBUG::2013-11-19 21:04:25,894::task::495::TaskManager.Task::(_debug) Task 455407ee-994f-4170-88e4-25a4d5c75416: moving from state preparing -> state finished
>Thread-15910158::DEBUG::2013-11-19 21:04:25,895::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910158::DEBUG::2013-11-19 21:04:25,895::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910158::DEBUG::2013-11-19 21:04:25,896::task::495::TaskManager.Task::(_debug) Task 455407ee-994f-4170-88e4-25a4d5c75416: ref 0 aborting False
>Thread-15910158::INFO::2013-11-19 21:04:25,896::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910159::DEBUG::2013-11-19 21:04:25,901::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910159::INFO::2013-11-19 21:04:25,902::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910159::DEBUG::2013-11-19 21:04:25,903::task::495::TaskManager.Task::(_debug) Task 85c6b465-aca0-464e-ad83-29b81421b5cf: moving from state init -> state preparing
>Thread-15910159::DEBUG::2013-11-19 21:04:25,905::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3acfe5f9-4a5f-46ee-bb2c-627f4be4f36e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910159::DEBUG::2013-11-19 21:04:25,906::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910159::DEBUG::2013-11-19 21:04:25,906::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910159::DEBUG::2013-11-19 21:04:25,907::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3acfe5f9-4a5f-46ee-bb2c-627f4be4f36e`::Granted request
>Thread-15910159::DEBUG::2013-11-19 21:04:25,908::task::495::TaskManager.Task::(_debug) Task 85c6b465-aca0-464e-ad83-29b81421b5cf: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910159::DEBUG::2013-11-19 21:04:25,908::task::495::TaskManager.Task::(_debug) Task 85c6b465-aca0-464e-ad83-29b81421b5cf: ref 1 aborting False
>Thread-15910159::WARNING::2013-11-19 21:04:25,910::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910159::DEBUG::2013-11-19 21:04:25,911::task::495::TaskManager.Task::(_debug) Task 85c6b465-aca0-464e-ad83-29b81421b5cf: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910159::DEBUG::2013-11-19 21:04:25,912::task::495::TaskManager.Task::(_debug) Task 85c6b465-aca0-464e-ad83-29b81421b5cf: moving from state preparing -> state finished
>Thread-15910159::DEBUG::2013-11-19 21:04:25,913::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910159::DEBUG::2013-11-19 21:04:25,913::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910159::DEBUG::2013-11-19 21:04:25,914::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910159::DEBUG::2013-11-19 21:04:25,915::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910159::DEBUG::2013-11-19 21:04:25,915::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910159::DEBUG::2013-11-19 21:04:25,916::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910159::DEBUG::2013-11-19 21:04:25,916::task::495::TaskManager.Task::(_debug) Task 85c6b465-aca0-464e-ad83-29b81421b5cf: ref 0 aborting False
>Thread-15910159::INFO::2013-11-19 21:04:25,917::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910161::DEBUG::2013-11-19 21:04:26,093::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('6574d88c-00ac-46f4-99ce-a2726932bfda',) {}
>Thread-15910161::DEBUG::2013-11-19 21:04:26,094::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': u'None', 'memUsage': '92', 'acpiEnable': 'true', 'pid': '57291', 'displayPort': u'5908', 'session': 'UserLoggedOn', 'netIfaces': [{u'inet6': [], u'hw': u'00-1a-4a-a8-64-08', u'inet': [u'192.168.100.70'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': -1L, 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '38686343168', u'fs': u'NTFS'}, {u'path': u'h:\\', u'total': '107237863424', u'used': '99742478336', u'fs': u'NTFS'}, {u'path': u'l:\\', u'total': '150187474944', u'used': '130882863104', u'fs': u'NTFS'}, {u'path': u'x:\\', u'total': '429360349184', u'used': '330617978880', u'fs': u'NTFS'}], 'network': {u'vnet5': {'macAddr': u'00:1a:4a:a8:64:08', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet5'}}, 'displayType': 'qxl', 'guestName': u'KLEE-AXDB01', 'boot': 'cd', 'elapsedTime': '18663675', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Microsoft Report Viewer Redistributable 2008 SP1', u'Zabbix Agent 2.0.8', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'Microsoft SQL Server Compact 3.5 SP2 ENU', u'RHEV-Spice-Agent64', u'Microsoft Visual Studio Tools for Applications 2.0 - ENU', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'Microsoft Office 2003 Web Components', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10', u'Microsoft SQL Server Browser', u'Microsoft Report Viewer Redistributable 2008 (KB971119)', u'Microsoft SQL Server 2008 R2 Policies', u'Microsoft SQL Server Compact 3.5 SP2 Query Tools ENU'], 'guestOs': u'Unknown', 'nice': '', 'status': 'Powering down', 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'displayIp': '0', 'lastLogout': 1366231390.568368, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '3652045', 'apparentsize': '64424509440', 'writeLatency': '312295', 'imageID': 'c4460480-32d3-469f-b0f4-c03bb6d74e41', 'flushLatency': '183455', 'readRate': '145382.38', 'truesize': '64424509440', 'writeRate': '189200.51'}, u'vdc': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '107374182400', 'writeRate': '0.00'}, u'vdb': {'readLatency': '0', 'apparentsize': '150323855360', 'writeLatency': '0', 'imageID': 'df8b790c-c762-4feb-9378-d64a321ce43c', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '150323855360', 'writeRate': '0.00'}, u'vdd': {'readLatency': '0', 'apparentsize': '429496729600', 'writeLatency': '0', 'imageID': 'd368307e-425e-423a-85b7-266c1e4a99ec', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '429496729600', 'writeRate': '0.00'}}, 'monitorResponse': '0', 'statsAge': '1.64', 'cpuUser': '207.50', 'cpuIdle': '0.00', 'lastLogin': 1384895046.5037899, 'guestIPs': u'192.168.100.70', 'displaySecurePort': u'5909'}]}
>Dummy-15909965::DEBUG::2013-11-19 21:04:26,276::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:04:26,879::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0759159 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15910162::INFO::2013-11-19 21:04:28,192::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910162::DEBUG::2013-11-19 21:04:28,193::task::495::TaskManager.Task::(_debug) Task a0f975c5-68ea-4894-aea7-32a21ee3ee5f: moving from state init -> state preparing
>Thread-15910162::DEBUG::2013-11-19 21:04:28,194::task::495::TaskManager.Task::(_debug) Task a0f975c5-68ea-4894-aea7-32a21ee3ee5f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000324010848999', 'lastCheck': 1384895059.9078059, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00121212005615', 'lastCheck': 1384895065.95099, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895060.8186281, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00952196121216', 'lastCheck': 1384895058.8261299, 'valid': True, 'code': 0}}
>Thread-15910162::DEBUG::2013-11-19 21:04:28,194::task::495::TaskManager.Task::(_debug) Task a0f975c5-68ea-4894-aea7-32a21ee3ee5f: moving from state preparing -> state finished
>Thread-15910162::DEBUG::2013-11-19 21:04:28,195::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910162::DEBUG::2013-11-19 21:04:28,195::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910162::DEBUG::2013-11-19 21:04:28,196::task::495::TaskManager.Task::(_debug) Task a0f975c5-68ea-4894-aea7-32a21ee3ee5f: ref 0 aborting False
>Thread-15910162::INFO::2013-11-19 21:04:28,197::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000324010848999', 'lastCheck': 1384895059.9078059, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00121212005615', 'lastCheck': 1384895065.95099, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895060.8186281, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00952196121216', 'lastCheck': 1384895058.8261299, 'valid': True, 'code': 0}}
>Dummy-15909965::DEBUG::2013-11-19 21:04:28,892::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:04:29,424::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0737402 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15910165::DEBUG::2013-11-19 21:04:30,338::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('6574d88c-00ac-46f4-99ce-a2726932bfda',) {}
>Thread-15910165::DEBUG::2013-11-19 21:04:30,339::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': u'None', 'memUsage': '92', 'acpiEnable': 'true', 'pid': '57291', 'displayPort': u'5908', 'session': 'UserLoggedOn', 'netIfaces': [{u'inet6': [], u'hw': u'00-1a-4a-a8-64-08', u'inet': [u'192.168.100.70'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': -1L, 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '38686343168', u'fs': u'NTFS'}, {u'path': u'h:\\', u'total': '107237863424', u'used': '99742478336', u'fs': u'NTFS'}, {u'path': u'l:\\', u'total': '150187474944', u'used': '130882863104', u'fs': u'NTFS'}, {u'path': u'x:\\', u'total': '429360349184', u'used': '330617978880', u'fs': u'NTFS'}], 'network': {u'vnet5': {'macAddr': u'00:1a:4a:a8:64:08', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet5'}}, 'displayType': 'qxl', 'guestName': u'KLEE-AXDB01', 'boot': 'cd', 'elapsedTime': '18663679', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Microsoft Report Viewer Redistributable 2008 SP1', u'Zabbix Agent 2.0.8', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'Microsoft SQL Server Compact 3.5 SP2 ENU', u'RHEV-Spice-Agent64', u'Microsoft Visual Studio Tools for Applications 2.0 - ENU', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'Microsoft Office 2003 Web Components', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10', u'Microsoft SQL Server Browser', u'Microsoft Report Viewer Redistributable 2008 (KB971119)', u'Microsoft SQL Server 2008 R2 Policies', u'Microsoft SQL Server Compact 3.5 SP2 Query Tools ENU'], 'guestOs': u'Unknown', 'nice': '', 'status': 'Powering down', 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'displayIp': '0', 'lastLogout': 1366231390.568368, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '3652045', 'apparentsize': '64424509440', 'writeLatency': '312295', 'imageID': 'c4460480-32d3-469f-b0f4-c03bb6d74e41', 'flushLatency': '183455', 'readRate': '145382.38', 'truesize': '64424509440', 'writeRate': '189200.51'}, u'vdc': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '107374182400', 'writeRate': '0.00'}, u'vdb': {'readLatency': '0', 'apparentsize': '150323855360', 'writeLatency': '0', 'imageID': 'df8b790c-c762-4feb-9378-d64a321ce43c', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '150323855360', 'writeRate': '0.00'}, u'vdd': {'readLatency': '0', 'apparentsize': '429496729600', 'writeLatency': '0', 'imageID': 'd368307e-425e-423a-85b7-266c1e4a99ec', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '429496729600', 'writeRate': '0.00'}}, 'monitorResponse': '0', 'statsAge': '0.77', 'cpuUser': '207.50', 'cpuIdle': '0.00', 'lastLogin': 1384895046.5037899, 'guestIPs': u'192.168.100.70', 'displaySecurePort': u'5909'}]}
>Thread-1024767::ERROR::2013-11-19 21:04:31,116::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15909965::DEBUG::2013-11-19 21:04:31,439::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:04:31,985::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0736101 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15910167::DEBUG::2013-11-19 21:04:32,416::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('6574d88c-00ac-46f4-99ce-a2726932bfda',) {}
>Thread-15910167::DEBUG::2013-11-19 21:04:32,417::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': u'None', 'memUsage': '92', 'acpiEnable': 'true', 'pid': '57291', 'displayPort': u'5908', 'session': 'UserLoggedOn', 'netIfaces': [{u'inet6': [], u'hw': u'00-1a-4a-a8-64-08', u'inet': [u'192.168.100.70'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': -1L, 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '38686343168', u'fs': u'NTFS'}, {u'path': u'h:\\', u'total': '107237863424', u'used': '99742478336', u'fs': u'NTFS'}, {u'path': u'l:\\', u'total': '150187474944', u'used': '130882863104', u'fs': u'NTFS'}, {u'path': u'x:\\', u'total': '429360349184', u'used': '330617978880', u'fs': u'NTFS'}], 'network': {u'vnet5': {'macAddr': u'00:1a:4a:a8:64:08', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet5'}}, 'displayType': 'qxl', 'guestName': u'KLEE-AXDB01', 'boot': 'cd', 'elapsedTime': '18663681', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Microsoft Report Viewer Redistributable 2008 SP1', u'Zabbix Agent 2.0.8', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'Microsoft SQL Server Compact 3.5 SP2 ENU', u'RHEV-Spice-Agent64', u'Microsoft Visual Studio Tools for Applications 2.0 - ENU', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'Microsoft Office 2003 Web Components', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10', u'Microsoft SQL Server Browser', u'Microsoft Report Viewer Redistributable 2008 (KB971119)', u'Microsoft SQL Server 2008 R2 Policies', u'Microsoft SQL Server Compact 3.5 SP2 Query Tools ENU'], 'guestOs': u'Unknown', 'nice': '', 'status': 'Powering down', 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'displayIp': '0', 'lastLogout': 1366231390.568368, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '3652045', 'apparentsize': '64424509440', 'writeLatency': '312295', 'imageID': 'c4460480-32d3-469f-b0f4-c03bb6d74e41', 'flushLatency': '183455', 'readRate': '145382.38', 'truesize': '64424509440', 'writeRate': '189200.51'}, u'vdc': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '107374182400', 'writeRate': '0.00'}, u'vdb': {'readLatency': '0', 'apparentsize': '150323855360', 'writeLatency': '0', 'imageID': 'df8b790c-c762-4feb-9378-d64a321ce43c', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '150323855360', 'writeRate': '0.00'}, u'vdd': {'readLatency': '0', 'apparentsize': '429496729600', 'writeLatency': '0', 'imageID': 'd368307e-425e-423a-85b7-266c1e4a99ec', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '429496729600', 'writeRate': '0.00'}}, 'monitorResponse': '0', 'statsAge': '1.85', 'cpuUser': '207.50', 'cpuIdle': '0.00', 'lastLogin': 1384895046.5037899, 'guestIPs': u'192.168.100.70', 'displaySecurePort': u'5909'}]}
>Dummy-15909965::DEBUG::2013-11-19 21:04:34,001::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15910169::DEBUG::2013-11-19 21:04:34,502::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('6574d88c-00ac-46f4-99ce-a2726932bfda',) {}
>Thread-15910169::DEBUG::2013-11-19 21:04:34,505::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': u'None', 'memUsage': '92', 'acpiEnable': 'true', 'pid': '57291', 'displayPort': u'5908', 'session': 'UserLoggedOn', 'netIfaces': [{u'inet6': [], u'hw': u'00-1a-4a-a8-64-08', u'inet': [u'192.168.100.70'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': -1L, 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '38686343168', u'fs': u'NTFS'}, {u'path': u'h:\\', u'total': '107237863424', u'used': '99742478336', u'fs': u'NTFS'}, {u'path': u'l:\\', u'total': '150187474944', u'used': '130882863104', u'fs': u'NTFS'}, {u'path': u'x:\\', u'total': '429360349184', u'used': '330617978880', u'fs': u'NTFS'}], 'network': {u'vnet5': {'macAddr': u'00:1a:4a:a8:64:08', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet5'}}, 'displayType': 'qxl', 'guestName': u'KLEE-AXDB01', 'boot': 'cd', 'elapsedTime': '18663683', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Microsoft Report Viewer Redistributable 2008 SP1', u'Zabbix Agent 2.0.8', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'Microsoft SQL Server Compact 3.5 SP2 ENU', u'RHEV-Spice-Agent64', u'Microsoft Visual Studio Tools for Applications 2.0 - ENU', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'Microsoft Office 2003 Web Components', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10', u'Microsoft SQL Server Browser', u'Microsoft Report Viewer Redistributable 2008 (KB971119)', u'Microsoft SQL Server 2008 R2 Policies', u'Microsoft SQL Server Compact 3.5 SP2 Query Tools ENU'], 'guestOs': u'Unknown', 'nice': '', 'status': 'Powering down', 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'displayIp': '0', 'lastLogout': 1366231390.568368, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '3652045', 'apparentsize': '64424509440', 'writeLatency': '312295', 'imageID': 'c4460480-32d3-469f-b0f4-c03bb6d74e41', 'flushLatency': '183455', 'readRate': '145382.38', 'truesize': '64424509440', 'writeRate': '189200.51'}, u'vdc': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '107374182400', 'writeRate': '0.00'}, u'vdb': {'readLatency': '0', 'apparentsize': '150323855360', 'writeLatency': '0', 'imageID': 'df8b790c-c762-4feb-9378-d64a321ce43c', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '150323855360', 'writeRate': '0.00'}, u'vdd': {'readLatency': '0', 'apparentsize': '429496729600', 'writeLatency': '0', 'imageID': 'd368307e-425e-423a-85b7-266c1e4a99ec', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '429496729600', 'writeRate': '0.00'}}, 'monitorResponse': '0', 'statsAge': '1.93', 'cpuUser': '207.50', 'cpuIdle': '0.00', 'lastLogin': 1384895046.5037899, 'guestIPs': u'192.168.100.70', 'displaySecurePort': u'5909'}]}
>Dummy-15909965::DEBUG::2013-11-19 21:04:34,555::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0734953 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15910170::DEBUG::2013-11-19 21:04:36,115::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910170::INFO::2013-11-19 21:04:36,116::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910170::DEBUG::2013-11-19 21:04:36,117::task::495::TaskManager.Task::(_debug) Task 2c72cbfc-3617-48d5-802f-46e587d912aa: moving from state init -> state preparing
>Thread-15910170::DEBUG::2013-11-19 21:04:36,117::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=274 spmId=2
>Thread-15910170::DEBUG::2013-11-19 21:04:36,118::task::495::TaskManager.Task::(_debug) Task 2c72cbfc-3617-48d5-802f-46e587d912aa: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910170::DEBUG::2013-11-19 21:04:36,119::task::495::TaskManager.Task::(_debug) Task 2c72cbfc-3617-48d5-802f-46e587d912aa: moving from state preparing -> state finished
>Thread-15910170::DEBUG::2013-11-19 21:04:36,119::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910170::DEBUG::2013-11-19 21:04:36,120::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910170::DEBUG::2013-11-19 21:04:36,120::task::495::TaskManager.Task::(_debug) Task 2c72cbfc-3617-48d5-802f-46e587d912aa: ref 0 aborting False
>Thread-15910170::INFO::2013-11-19 21:04:36,121::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910171::DEBUG::2013-11-19 21:04:36,126::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910171::INFO::2013-11-19 21:04:36,127::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910171::DEBUG::2013-11-19 21:04:36,127::task::495::TaskManager.Task::(_debug) Task 55f38594-d585-4238-936c-2f07856b4af6: moving from state init -> state preparing
>Thread-15910171::DEBUG::2013-11-19 21:04:36,128::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8e9f9448-1763-4880-ab4d-1455d79856e7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910171::DEBUG::2013-11-19 21:04:36,129::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910171::DEBUG::2013-11-19 21:04:36,130::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910171::DEBUG::2013-11-19 21:04:36,130::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8e9f9448-1763-4880-ab4d-1455d79856e7`::Granted request
>Thread-15910171::DEBUG::2013-11-19 21:04:36,131::task::495::TaskManager.Task::(_debug) Task 55f38594-d585-4238-936c-2f07856b4af6: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910171::DEBUG::2013-11-19 21:04:36,132::task::495::TaskManager.Task::(_debug) Task 55f38594-d585-4238-936c-2f07856b4af6: ref 1 aborting False
>Thread-15910171::WARNING::2013-11-19 21:04:36,134::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910171::DEBUG::2013-11-19 21:04:36,135::task::495::TaskManager.Task::(_debug) Task 55f38594-d585-4238-936c-2f07856b4af6: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910171::DEBUG::2013-11-19 21:04:36,136::task::495::TaskManager.Task::(_debug) Task 55f38594-d585-4238-936c-2f07856b4af6: moving from state preparing -> state finished
>Thread-15910171::DEBUG::2013-11-19 21:04:36,136::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910171::DEBUG::2013-11-19 21:04:36,137::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910171::DEBUG::2013-11-19 21:04:36,138::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910171::DEBUG::2013-11-19 21:04:36,138::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910171::DEBUG::2013-11-19 21:04:36,139::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910171::DEBUG::2013-11-19 21:04:36,139::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910171::DEBUG::2013-11-19 21:04:36,140::task::495::TaskManager.Task::(_debug) Task 55f38594-d585-4238-936c-2f07856b4af6: ref 0 aborting False
>Thread-15910171::INFO::2013-11-19 21:04:36,141::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15909965::DEBUG::2013-11-19 21:04:36,573::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15910173::DEBUG::2013-11-19 21:04:36,803::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('6574d88c-00ac-46f4-99ce-a2726932bfda',) {}
>Thread-15910173::DEBUG::2013-11-19 21:04:36,807::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': u'None', 'memUsage': '92', 'acpiEnable': 'true', 'pid': '57291', 'displayPort': u'5908', 'session': 'UserLoggedOn', 'netIfaces': [{u'inet6': [], u'hw': u'00-1a-4a-a8-64-08', u'inet': [u'192.168.100.70'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': -1L, 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '38686343168', u'fs': u'NTFS'}, {u'path': u'h:\\', u'total': '107237863424', u'used': '99742478336', u'fs': u'NTFS'}, {u'path': u'l:\\', u'total': '150187474944', u'used': '130882863104', u'fs': u'NTFS'}, {u'path': u'x:\\', u'total': '429360349184', u'used': '330617978880', u'fs': u'NTFS'}], 'network': {u'vnet5': {'macAddr': u'00:1a:4a:a8:64:08', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet5'}}, 'displayType': 'qxl', 'guestName': u'KLEE-AXDB01', 'boot': 'cd', 'elapsedTime': '18663686', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Microsoft Report Viewer Redistributable 2008 SP1', u'Zabbix Agent 2.0.8', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'Microsoft SQL Server Compact 3.5 SP2 ENU', u'RHEV-Spice-Agent64', u'Microsoft Visual Studio Tools for Applications 2.0 - ENU', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'Microsoft Office 2003 Web Components', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10', u'Microsoft SQL Server Browser', u'Microsoft Report Viewer Redistributable 2008 (KB971119)', u'Microsoft SQL Server 2008 R2 Policies', u'Microsoft SQL Server Compact 3.5 SP2 Query Tools ENU'], 'guestOs': u'Unknown', 'nice': '', 'status': 'Powering down', 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'displayIp': '0', 'lastLogout': 1366231390.568368, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '3652045', 'apparentsize': '64424509440', 'writeLatency': '312295', 'imageID': 'c4460480-32d3-469f-b0f4-c03bb6d74e41', 'flushLatency': '183455', 'readRate': '145382.38', 'truesize': '64424509440', 'writeRate': '189200.51'}, u'vdc': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '107374182400', 'writeRate': '0.00'}, u'vdb': {'readLatency': '0', 'apparentsize': '150323855360', 'writeLatency': '0', 'imageID': 'df8b790c-c762-4feb-9378-d64a321ce43c', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '150323855360', 'writeRate': '0.00'}, u'vdd': {'readLatency': '0', 'apparentsize': '429496729600', 'writeLatency': '0', 'imageID': 'd368307e-425e-423a-85b7-266c1e4a99ec', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '429496729600', 'writeRate': '0.00'}}, 'monitorResponse': '0', 'statsAge': '0.23', 'cpuUser': '207.50', 'cpuIdle': '0.00', 'lastLogin': 1384895046.5037899, 'guestIPs': u'192.168.100.70', 'displaySecurePort': u'5909'}]}
>Dummy-15909965::DEBUG::2013-11-19 21:04:37,115::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0742137 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15910174::INFO::2013-11-19 21:04:38,864::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910174::DEBUG::2013-11-19 21:04:38,865::task::495::TaskManager.Task::(_debug) Task 5aec09de-ea39-4c40-ab59-92e954b3a0d8: moving from state init -> state preparing
>Thread-15910174::DEBUG::2013-11-19 21:04:38,866::task::495::TaskManager.Task::(_debug) Task 5aec09de-ea39-4c40-ab59-92e954b3a0d8: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000329971313477', 'lastCheck': 1384895070.1167641, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00170803070068', 'lastCheck': 1384895076.2175169, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895071.1171169, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000751972198486', 'lastCheck': 1384895069.11534, 'valid': True, 'code': 0}}
>Thread-15910174::DEBUG::2013-11-19 21:04:38,867::task::495::TaskManager.Task::(_debug) Task 5aec09de-ea39-4c40-ab59-92e954b3a0d8: moving from state preparing -> state finished
>Thread-15910174::DEBUG::2013-11-19 21:04:38,867::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910174::DEBUG::2013-11-19 21:04:38,868::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910174::DEBUG::2013-11-19 21:04:38,868::task::495::TaskManager.Task::(_debug) Task 5aec09de-ea39-4c40-ab59-92e954b3a0d8: ref 0 aborting False
>Thread-15910174::INFO::2013-11-19 21:04:38,869::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000329971313477', 'lastCheck': 1384895070.1167641, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00170803070068', 'lastCheck': 1384895076.2175169, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895071.1171169, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000751972198486', 'lastCheck': 1384895069.11534, 'valid': True, 'code': 0}}
>Dummy-15909965::DEBUG::2013-11-19 21:04:39,130::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:04:39,686::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0958235 s, 10.7 MB/s\n'; <rc> = 0
>Thread-15910177::DEBUG::2013-11-19 21:04:41,065::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('6574d88c-00ac-46f4-99ce-a2726932bfda',) {}
>Thread-15910177::DEBUG::2013-11-19 21:04:41,067::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': u'None', 'memUsage': '92', 'acpiEnable': 'true', 'pid': '57291', 'displayPort': u'5908', 'session': 'UserLoggedOn', 'netIfaces': [{u'inet6': [], u'hw': u'00-1a-4a-a8-64-08', u'inet': [u'192.168.100.70'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': -1L, 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '38686343168', u'fs': u'NTFS'}, {u'path': u'h:\\', u'total': '107237863424', u'used': '99742478336', u'fs': u'NTFS'}, {u'path': u'l:\\', u'total': '150187474944', u'used': '130882863104', u'fs': u'NTFS'}, {u'path': u'x:\\', u'total': '429360349184', u'used': '330617978880', u'fs': u'NTFS'}], 'network': {u'vnet5': {'macAddr': u'00:1a:4a:a8:64:08', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet5'}}, 'displayType': 'qxl', 'guestName': u'KLEE-AXDB01', 'boot': 'cd', 'elapsedTime': '18663690', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Microsoft Report Viewer Redistributable 2008 SP1', u'Zabbix Agent 2.0.8', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'Microsoft SQL Server Compact 3.5 SP2 ENU', u'RHEV-Spice-Agent64', u'Microsoft Visual Studio Tools for Applications 2.0 - ENU', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'Microsoft Office 2003 Web Components', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10', u'Microsoft SQL Server Browser', u'Microsoft Report Viewer Redistributable 2008 (KB971119)', u'Microsoft SQL Server 2008 R2 Policies', u'Microsoft SQL Server Compact 3.5 SP2 Query Tools ENU'], 'guestOs': u'Unknown', 'nice': '', 'status': 'Powering down', 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'displayIp': '0', 'lastLogout': 1366231390.568368, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '3652045', 'apparentsize': '64424509440', 'writeLatency': '312295', 'imageID': 'c4460480-32d3-469f-b0f4-c03bb6d74e41', 'flushLatency': '183455', 'readRate': '145382.38', 'truesize': '64424509440', 'writeRate': '189200.51'}, u'vdc': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '107374182400', 'writeRate': '0.00'}, u'vdb': {'readLatency': '0', 'apparentsize': '150323855360', 'writeLatency': '0', 'imageID': 'df8b790c-c762-4feb-9378-d64a321ce43c', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '150323855360', 'writeRate': '0.00'}, u'vdd': {'readLatency': '0', 'apparentsize': '429496729600', 'writeLatency': '0', 'imageID': 'd368307e-425e-423a-85b7-266c1e4a99ec', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '429496729600', 'writeRate': '0.00'}}, 'monitorResponse': '0', 'statsAge': '0.48', 'cpuUser': '139.33', 'cpuIdle': '0.00', 'lastLogin': 1384895046.5037899, 'guestIPs': u'192.168.100.70', 'displaySecurePort': u'5909'}]}
>Thread-1024767::ERROR::2013-11-19 21:04:41,337::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15909965::DEBUG::2013-11-19 21:04:41,700::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:04:42,205::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0509604 s, 20.1 MB/s\n'; <rc> = 0
>Thread-15910179::DEBUG::2013-11-19 21:04:43,198::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('6574d88c-00ac-46f4-99ce-a2726932bfda',) {}
>Thread-15910179::DEBUG::2013-11-19 21:04:43,200::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': u'None', 'memUsage': '92', 'acpiEnable': 'true', 'pid': '57291', 'displayPort': u'5908', 'session': 'UserLoggedOn', 'netIfaces': [{u'inet6': [], u'hw': u'00-1a-4a-a8-64-08', u'inet': [u'192.168.100.70'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': -1L, 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '38686343168', u'fs': u'NTFS'}, {u'path': u'h:\\', u'total': '107237863424', u'used': '99742478336', u'fs': u'NTFS'}, {u'path': u'l:\\', u'total': '150187474944', u'used': '130882863104', u'fs': u'NTFS'}, {u'path': u'x:\\', u'total': '429360349184', u'used': '330617978880', u'fs': u'NTFS'}], 'network': {u'vnet5': {'macAddr': u'00:1a:4a:a8:64:08', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet5'}}, 'displayType': 'qxl', 'guestName': u'KLEE-AXDB01', 'boot': 'cd', 'elapsedTime': '18663692', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Microsoft Report Viewer Redistributable 2008 SP1', u'Zabbix Agent 2.0.8', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'Microsoft SQL Server Compact 3.5 SP2 ENU', u'RHEV-Spice-Agent64', u'Microsoft Visual Studio Tools for Applications 2.0 - ENU', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'Microsoft Office 2003 Web Components', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10', u'Microsoft SQL Server Browser', u'Microsoft Report Viewer Redistributable 2008 (KB971119)', u'Microsoft SQL Server 2008 R2 Policies', u'Microsoft SQL Server Compact 3.5 SP2 Query Tools ENU'], 'guestOs': u'Unknown', 'nice': '', 'status': 'Powering down', 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'displayIp': '0', 'lastLogout': 1366231390.568368, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '3652045', 'apparentsize': '64424509440', 'writeLatency': '312295', 'imageID': 'c4460480-32d3-469f-b0f4-c03bb6d74e41', 'flushLatency': '183455', 'readRate': '145382.38', 'truesize': '64424509440', 'writeRate': '189200.51'}, u'vdc': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '107374182400', 'writeRate': '0.00'}, u'vdb': {'readLatency': '0', 'apparentsize': '150323855360', 'writeLatency': '0', 'imageID': 'df8b790c-c762-4feb-9378-d64a321ce43c', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '150323855360', 'writeRate': '0.00'}, u'vdd': {'readLatency': '0', 'apparentsize': '429496729600', 'writeLatency': '0', 'imageID': 'd368307e-425e-423a-85b7-266c1e4a99ec', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '429496729600', 'writeRate': '0.00'}}, 'monitorResponse': '0', 'statsAge': '0.62', 'cpuUser': '139.33', 'cpuIdle': '0.00', 'lastLogin': 1384895046.5037899, 'guestIPs': u'192.168.100.70', 'displaySecurePort': u'5909'}]}
>Dummy-15909965::DEBUG::2013-11-19 21:04:44,216::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:04:44,776::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0778722 s, 13.1 MB/s\n'; <rc> = 0
>Thread-15910181::DEBUG::2013-11-19 21:04:45,287::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('6574d88c-00ac-46f4-99ce-a2726932bfda',) {}
>Thread-15910181::DEBUG::2013-11-19 21:04:45,289::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': u'None', 'memUsage': '92', 'acpiEnable': 'true', 'pid': '57291', 'displayPort': u'5908', 'session': 'UserLoggedOn', 'netIfaces': [{u'inet6': [], u'hw': u'00-1a-4a-a8-64-08', u'inet': [u'192.168.100.70'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': -1L, 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '38686343168', u'fs': u'NTFS'}, {u'path': u'h:\\', u'total': '107237863424', u'used': '99742478336', u'fs': u'NTFS'}, {u'path': u'l:\\', u'total': '150187474944', u'used': '130882863104', u'fs': u'NTFS'}, {u'path': u'x:\\', u'total': '429360349184', u'used': '330617978880', u'fs': u'NTFS'}], 'network': {u'vnet5': {'macAddr': u'00:1a:4a:a8:64:08', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet5'}}, 'displayType': 'qxl', 'guestName': u'KLEE-AXDB01', 'boot': 'cd', 'elapsedTime': '18663694', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Microsoft Report Viewer Redistributable 2008 SP1', u'Zabbix Agent 2.0.8', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'Microsoft SQL Server Compact 3.5 SP2 ENU', u'RHEV-Spice-Agent64', u'Microsoft Visual Studio Tools for Applications 2.0 - ENU', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'Microsoft Office 2003 Web Components', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10', u'Microsoft SQL Server Browser', u'Microsoft Report Viewer Redistributable 2008 (KB971119)', u'Microsoft SQL Server 2008 R2 Policies', u'Microsoft SQL Server Compact 3.5 SP2 Query Tools ENU'], 'guestOs': u'Unknown', 'nice': '', 'status': 'Powering down', 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'displayIp': '0', 'lastLogout': 1366231390.568368, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '3652045', 'apparentsize': '64424509440', 'writeLatency': '312295', 'imageID': 'c4460480-32d3-469f-b0f4-c03bb6d74e41', 'flushLatency': '183455', 'readRate': '145382.38', 'truesize': '64424509440', 'writeRate': '189200.51'}, u'vdc': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '107374182400', 'writeRate': '0.00'}, u'vdb': {'readLatency': '0', 'apparentsize': '150323855360', 'writeLatency': '0', 'imageID': 'df8b790c-c762-4feb-9378-d64a321ce43c', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '150323855360', 'writeRate': '0.00'}, u'vdd': {'readLatency': '0', 'apparentsize': '429496729600', 'writeLatency': '0', 'imageID': 'd368307e-425e-423a-85b7-266c1e4a99ec', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '429496729600', 'writeRate': '0.00'}}, 'monitorResponse': '0', 'statsAge': '0.70', 'cpuUser': '139.33', 'cpuIdle': '0.00', 'lastLogin': 1384895046.5037899, 'guestIPs': u'192.168.100.70', 'displaySecurePort': u'5909'}]}
>Thread-15910182::DEBUG::2013-11-19 21:04:46,221::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910182::INFO::2013-11-19 21:04:46,223::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910182::DEBUG::2013-11-19 21:04:46,224::task::495::TaskManager.Task::(_debug) Task 9a709b40-7d55-4988-ba50-1a6792c777ad: moving from state init -> state preparing
>Thread-15910182::DEBUG::2013-11-19 21:04:46,224::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=274 spmId=2
>Thread-15910182::DEBUG::2013-11-19 21:04:46,225::task::495::TaskManager.Task::(_debug) Task 9a709b40-7d55-4988-ba50-1a6792c777ad: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910182::DEBUG::2013-11-19 21:04:46,226::task::495::TaskManager.Task::(_debug) Task 9a709b40-7d55-4988-ba50-1a6792c777ad: moving from state preparing -> state finished
>Thread-15910182::DEBUG::2013-11-19 21:04:46,226::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910182::DEBUG::2013-11-19 21:04:46,227::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910182::DEBUG::2013-11-19 21:04:46,227::task::495::TaskManager.Task::(_debug) Task 9a709b40-7d55-4988-ba50-1a6792c777ad: ref 0 aborting False
>Thread-15910182::INFO::2013-11-19 21:04:46,228::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910183::DEBUG::2013-11-19 21:04:46,233::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910183::INFO::2013-11-19 21:04:46,233::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910183::DEBUG::2013-11-19 21:04:46,234::task::495::TaskManager.Task::(_debug) Task e977c7b9-e2a5-44ac-9ffb-12d647528273: moving from state init -> state preparing
>Thread-15910183::DEBUG::2013-11-19 21:04:46,235::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ccd4e4fc-5391-4aff-89f8-744bd9377f01`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910183::DEBUG::2013-11-19 21:04:46,236::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910183::DEBUG::2013-11-19 21:04:46,237::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910183::DEBUG::2013-11-19 21:04:46,237::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ccd4e4fc-5391-4aff-89f8-744bd9377f01`::Granted request
>Thread-15910183::DEBUG::2013-11-19 21:04:46,242::task::495::TaskManager.Task::(_debug) Task e977c7b9-e2a5-44ac-9ffb-12d647528273: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910183::DEBUG::2013-11-19 21:04:46,243::task::495::TaskManager.Task::(_debug) Task e977c7b9-e2a5-44ac-9ffb-12d647528273: ref 1 aborting False
>Thread-15910183::WARNING::2013-11-19 21:04:46,245::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910183::DEBUG::2013-11-19 21:04:46,246::task::495::TaskManager.Task::(_debug) Task e977c7b9-e2a5-44ac-9ffb-12d647528273: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910183::DEBUG::2013-11-19 21:04:46,247::task::495::TaskManager.Task::(_debug) Task e977c7b9-e2a5-44ac-9ffb-12d647528273: moving from state preparing -> state finished
>Thread-15910183::DEBUG::2013-11-19 21:04:46,247::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910183::DEBUG::2013-11-19 21:04:46,248::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910183::DEBUG::2013-11-19 21:04:46,249::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910183::DEBUG::2013-11-19 21:04:46,249::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910183::DEBUG::2013-11-19 21:04:46,250::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910183::DEBUG::2013-11-19 21:04:46,250::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910183::DEBUG::2013-11-19 21:04:46,251::task::495::TaskManager.Task::(_debug) Task e977c7b9-e2a5-44ac-9ffb-12d647528273: ref 0 aborting False
>Thread-15910183::INFO::2013-11-19 21:04:46,252::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15909965::DEBUG::2013-11-19 21:04:46,791::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:04:47,338::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0775395 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15910185::DEBUG::2013-11-19 21:04:47,385::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('6574d88c-00ac-46f4-99ce-a2726932bfda',) {}
>Thread-15910185::DEBUG::2013-11-19 21:04:47,387::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': u'None', 'memUsage': '92', 'acpiEnable': 'true', 'pid': '57291', 'displayPort': u'5908', 'session': 'UserLoggedOn', 'netIfaces': [{u'inet6': [], u'hw': u'00-1a-4a-a8-64-08', u'inet': [u'192.168.100.70'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': -1L, 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '38686343168', u'fs': u'NTFS'}, {u'path': u'h:\\', u'total': '107237863424', u'used': '99742478336', u'fs': u'NTFS'}, {u'path': u'l:\\', u'total': '150187474944', u'used': '130882863104', u'fs': u'NTFS'}, {u'path': u'x:\\', u'total': '429360349184', u'used': '330617978880', u'fs': u'NTFS'}], 'network': {u'vnet5': {'macAddr': u'00:1a:4a:a8:64:08', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet5'}}, 'displayType': 'qxl', 'guestName': u'KLEE-AXDB01', 'boot': 'cd', 'elapsedTime': '18663696', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Microsoft Report Viewer Redistributable 2008 SP1', u'Zabbix Agent 2.0.8', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'Microsoft SQL Server Compact 3.5 SP2 ENU', u'RHEV-Spice-Agent64', u'Microsoft Visual Studio Tools for Applications 2.0 - ENU', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'Microsoft Office 2003 Web Components', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10', u'Microsoft SQL Server Browser', u'Microsoft Report Viewer Redistributable 2008 (KB971119)', u'Microsoft SQL Server 2008 R2 Policies', u'Microsoft SQL Server Compact 3.5 SP2 Query Tools ENU'], 'guestOs': u'Unknown', 'nice': '', 'status': 'Powering down', 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'displayIp': '0', 'lastLogout': 1366231390.568368, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '3652045', 'apparentsize': '64424509440', 'writeLatency': '312295', 'imageID': 'c4460480-32d3-469f-b0f4-c03bb6d74e41', 'flushLatency': '183455', 'readRate': '145382.38', 'truesize': '64424509440', 'writeRate': '189200.51'}, u'vdc': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '107374182400', 'writeRate': '0.00'}, u'vdb': {'readLatency': '0', 'apparentsize': '150323855360', 'writeLatency': '0', 'imageID': 'df8b790c-c762-4feb-9378-d64a321ce43c', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '150323855360', 'writeRate': '0.00'}, u'vdd': {'readLatency': '0', 'apparentsize': '429496729600', 'writeLatency': '0', 'imageID': 'd368307e-425e-423a-85b7-266c1e4a99ec', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '429496729600', 'writeRate': '0.00'}}, 'monitorResponse': '0', 'statsAge': '0.80', 'cpuUser': '139.33', 'cpuIdle': '0.00', 'lastLogin': 1384895046.5037899, 'guestIPs': u'192.168.100.70', 'displaySecurePort': u'5909'}]}
>Dummy-15909965::DEBUG::2013-11-19 21:04:49,353::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15910186::INFO::2013-11-19 21:04:49,544::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910186::DEBUG::2013-11-19 21:04:49,545::task::495::TaskManager.Task::(_debug) Task 5fa2b204-50a4-40ed-9c96-ed7a65909080: moving from state init -> state preparing
>Thread-15910186::DEBUG::2013-11-19 21:04:49,546::task::495::TaskManager.Task::(_debug) Task 5fa2b204-50a4-40ed-9c96-ed7a65909080: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000360012054443', 'lastCheck': 1384895080.338366, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00137996673584', 'lastCheck': 1384895086.4436231, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895081.338659, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000998020172119', 'lastCheck': 1384895089.5396891, 'valid': True, 'code': 0}}
>Thread-15910186::DEBUG::2013-11-19 21:04:49,547::task::495::TaskManager.Task::(_debug) Task 5fa2b204-50a4-40ed-9c96-ed7a65909080: moving from state preparing -> state finished
>Thread-15910186::DEBUG::2013-11-19 21:04:49,548::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910186::DEBUG::2013-11-19 21:04:49,549::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910186::DEBUG::2013-11-19 21:04:49,549::task::495::TaskManager.Task::(_debug) Task 5fa2b204-50a4-40ed-9c96-ed7a65909080: ref 0 aborting False
>Thread-15910186::INFO::2013-11-19 21:04:49,550::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000360012054443', 'lastCheck': 1384895080.338366, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00137996673584', 'lastCheck': 1384895086.4436231, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895081.338659, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000998020172119', 'lastCheck': 1384895089.5396891, 'valid': True, 'code': 0}}
>Dummy-15909965::DEBUG::2013-11-19 21:04:49,820::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0579583 s, 17.7 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:04:51,544::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910189::DEBUG::2013-11-19 21:04:51,665::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('6574d88c-00ac-46f4-99ce-a2726932bfda',) {}
>Thread-15910189::DEBUG::2013-11-19 21:04:51,666::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': u'None', 'memUsage': '92', 'acpiEnable': 'true', 'pid': '57291', 'displayPort': u'5908', 'session': 'UserLoggedOn', 'netIfaces': [{u'inet6': [], u'hw': u'00-1a-4a-a8-64-08', u'inet': [u'192.168.100.70'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': -1L, 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '38686343168', u'fs': u'NTFS'}, {u'path': u'h:\\', u'total': '107237863424', u'used': '99742478336', u'fs': u'NTFS'}, {u'path': u'l:\\', u'total': '150187474944', u'used': '130882863104', u'fs': u'NTFS'}, {u'path': u'x:\\', u'total': '429360349184', u'used': '330617978880', u'fs': u'NTFS'}], 'network': {u'vnet5': {'macAddr': u'00:1a:4a:a8:64:08', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet5'}}, 'displayType': 'qxl', 'guestName': u'KLEE-AXDB01', 'boot': 'cd', 'elapsedTime': '18663701', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Microsoft Report Viewer Redistributable 2008 SP1', u'Zabbix Agent 2.0.8', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'Microsoft SQL Server Compact 3.5 SP2 ENU', u'RHEV-Spice-Agent64', u'Microsoft Visual Studio Tools for Applications 2.0 - ENU', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'Microsoft Office 2003 Web Components', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10', u'Microsoft SQL Server Browser', u'Microsoft Report Viewer Redistributable 2008 (KB971119)', u'Microsoft SQL Server 2008 R2 Policies', u'Microsoft SQL Server Compact 3.5 SP2 Query Tools ENU'], 'guestOs': u'Unknown', 'nice': '', 'status': 'Powering down', 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'displayIp': '0', 'lastLogout': 1366231390.568368, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '3652045', 'apparentsize': '64424509440', 'writeLatency': '312295', 'imageID': 'c4460480-32d3-469f-b0f4-c03bb6d74e41', 'flushLatency': '183455', 'readRate': '145382.38', 'truesize': '64424509440', 'writeRate': '189200.51'}, u'vdc': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '107374182400', 'writeRate': '0.00'}, u'vdb': {'readLatency': '0', 'apparentsize': '150323855360', 'writeLatency': '0', 'imageID': 'df8b790c-c762-4feb-9378-d64a321ce43c', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '150323855360', 'writeRate': '0.00'}, u'vdd': {'readLatency': '0', 'apparentsize': '429496729600', 'writeLatency': '0', 'imageID': 'd368307e-425e-423a-85b7-266c1e4a99ec', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '429496729600', 'writeRate': '0.00'}}, 'monitorResponse': '0', 'statsAge': '1.08', 'cpuUser': '139.33', 'cpuIdle': '0.00', 'lastLogin': 1384895046.5037899, 'guestIPs': u'192.168.100.70', 'displaySecurePort': u'5909'}]}
>Dummy-15909965::DEBUG::2013-11-19 21:04:51,830::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:04:52,353::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0768156 s, 13.3 MB/s\n'; <rc> = 0
>Thread-15910191::DEBUG::2013-11-19 21:04:53,790::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('6574d88c-00ac-46f4-99ce-a2726932bfda',) {}
>Thread-15910191::DEBUG::2013-11-19 21:04:53,791::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': u'None', 'memUsage': '92', 'acpiEnable': 'true', 'pid': '57291', 'displayPort': u'5908', 'session': 'UserLoggedOn', 'netIfaces': [{u'inet6': [], u'hw': u'00-1a-4a-a8-64-08', u'inet': [u'192.168.100.70'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': -1L, 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '38686343168', u'fs': u'NTFS'}, {u'path': u'h:\\', u'total': '107237863424', u'used': '99742478336', u'fs': u'NTFS'}, {u'path': u'l:\\', u'total': '150187474944', u'used': '130882863104', u'fs': u'NTFS'}, {u'path': u'x:\\', u'total': '429360349184', u'used': '330617978880', u'fs': u'NTFS'}], 'network': {u'vnet5': {'macAddr': u'00:1a:4a:a8:64:08', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet5'}}, 'displayType': 'qxl', 'guestName': u'KLEE-AXDB01', 'boot': 'cd', 'elapsedTime': '18663703', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Microsoft Report Viewer Redistributable 2008 SP1', u'Zabbix Agent 2.0.8', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'Microsoft SQL Server Compact 3.5 SP2 ENU', u'RHEV-Spice-Agent64', u'Microsoft Visual Studio Tools for Applications 2.0 - ENU', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'Microsoft Office 2003 Web Components', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10', u'Microsoft SQL Server Browser', u'Microsoft Report Viewer Redistributable 2008 (KB971119)', u'Microsoft SQL Server 2008 R2 Policies', u'Microsoft SQL Server Compact 3.5 SP2 Query Tools ENU'], 'guestOs': u'Unknown', 'nice': '', 'status': 'Powering down', 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'displayIp': '0', 'lastLogout': 1366231390.568368, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '3652045', 'apparentsize': '64424509440', 'writeLatency': '312295', 'imageID': 'c4460480-32d3-469f-b0f4-c03bb6d74e41', 'flushLatency': '183455', 'readRate': '145382.38', 'truesize': '64424509440', 'writeRate': '189200.51'}, u'vdc': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '107374182400', 'writeRate': '0.00'}, u'vdb': {'readLatency': '0', 'apparentsize': '150323855360', 'writeLatency': '0', 'imageID': 'df8b790c-c762-4feb-9378-d64a321ce43c', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '150323855360', 'writeRate': '0.00'}, u'vdd': {'readLatency': '0', 'apparentsize': '429496729600', 'writeLatency': '0', 'imageID': 'd368307e-425e-423a-85b7-266c1e4a99ec', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '429496729600', 'writeRate': '0.00'}}, 'monitorResponse': '0', 'statsAge': '1.20', 'cpuUser': '139.33', 'cpuIdle': '0.00', 'lastLogin': 1384895046.5037899, 'guestIPs': u'192.168.100.70', 'displaySecurePort': u'5909'}]}
>Dummy-15909965::DEBUG::2013-11-19 21:04:54,369::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:04:54,870::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0546511 s, 18.7 MB/s\n'; <rc> = 0
>Thread-15910193::DEBUG::2013-11-19 21:04:55,851::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('6574d88c-00ac-46f4-99ce-a2726932bfda',) {}
>Thread-15910193::DEBUG::2013-11-19 21:04:55,852::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': u'None', 'memUsage': '92', 'acpiEnable': 'true', 'pid': '57291', 'displayPort': u'5908', 'session': 'UserLoggedOn', 'netIfaces': [{u'inet6': [], u'hw': u'00-1a-4a-a8-64-08', u'inet': [u'192.168.100.70'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': -1L, 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '38686343168', u'fs': u'NTFS'}, {u'path': u'h:\\', u'total': '107237863424', u'used': '99742478336', u'fs': u'NTFS'}, {u'path': u'l:\\', u'total': '150187474944', u'used': '130882863104', u'fs': u'NTFS'}, {u'path': u'x:\\', u'total': '429360349184', u'used': '330617978880', u'fs': u'NTFS'}], 'network': {u'vnet5': {'macAddr': u'00:1a:4a:a8:64:08', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet5'}}, 'displayType': 'qxl', 'guestName': u'KLEE-AXDB01', 'boot': 'cd', 'elapsedTime': '18663705', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Microsoft Report Viewer Redistributable 2008 SP1', u'Zabbix Agent 2.0.8', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'Microsoft SQL Server Compact 3.5 SP2 ENU', u'RHEV-Spice-Agent64', u'Microsoft Visual Studio Tools for Applications 2.0 - ENU', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'Microsoft Office 2003 Web Components', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10', u'Microsoft SQL Server Browser', u'Microsoft Report Viewer Redistributable 2008 (KB971119)', u'Microsoft SQL Server 2008 R2 Policies', u'Microsoft SQL Server Compact 3.5 SP2 Query Tools ENU'], 'guestOs': u'Unknown', 'nice': '', 'status': 'Powering down', 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'displayIp': '0', 'lastLogout': 1366231390.568368, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '3652045', 'apparentsize': '64424509440', 'writeLatency': '312295', 'imageID': 'c4460480-32d3-469f-b0f4-c03bb6d74e41', 'flushLatency': '183455', 'readRate': '145382.38', 'truesize': '64424509440', 'writeRate': '189200.51'}, u'vdc': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '107374182400', 'writeRate': '0.00'}, u'vdb': {'readLatency': '0', 'apparentsize': '150323855360', 'writeLatency': '0', 'imageID': 'df8b790c-c762-4feb-9378-d64a321ce43c', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '150323855360', 'writeRate': '0.00'}, u'vdd': {'readLatency': '0', 'apparentsize': '429496729600', 'writeLatency': '0', 'imageID': 'd368307e-425e-423a-85b7-266c1e4a99ec', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '429496729600', 'writeRate': '0.00'}}, 'monitorResponse': '0', 'statsAge': '1.23', 'cpuUser': '172.48', 'cpuIdle': '0.00', 'lastLogin': 1384895046.5037899, 'guestIPs': u'192.168.100.70', 'displaySecurePort': u'5909'}]}
>Thread-15910194::DEBUG::2013-11-19 21:04:56,412::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910194::INFO::2013-11-19 21:04:56,414::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910194::DEBUG::2013-11-19 21:04:56,415::task::495::TaskManager.Task::(_debug) Task b6a2055c-7c9e-4aef-8a86-0ff071e32cd8: moving from state init -> state preparing
>Thread-15910194::DEBUG::2013-11-19 21:04:56,416::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=274 spmId=2
>Thread-15910194::DEBUG::2013-11-19 21:04:56,416::task::495::TaskManager.Task::(_debug) Task b6a2055c-7c9e-4aef-8a86-0ff071e32cd8: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910194::DEBUG::2013-11-19 21:04:56,417::task::495::TaskManager.Task::(_debug) Task b6a2055c-7c9e-4aef-8a86-0ff071e32cd8: moving from state preparing -> state finished
>Thread-15910194::DEBUG::2013-11-19 21:04:56,418::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910194::DEBUG::2013-11-19 21:04:56,418::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910194::DEBUG::2013-11-19 21:04:56,419::task::495::TaskManager.Task::(_debug) Task b6a2055c-7c9e-4aef-8a86-0ff071e32cd8: ref 0 aborting False
>Thread-15910194::INFO::2013-11-19 21:04:56,420::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910195::DEBUG::2013-11-19 21:04:56,424::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910195::INFO::2013-11-19 21:04:56,425::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910195::DEBUG::2013-11-19 21:04:56,426::task::495::TaskManager.Task::(_debug) Task 39f8fc9c-aa50-4258-bc4a-861b38ae973a: moving from state init -> state preparing
>Thread-15910195::DEBUG::2013-11-19 21:04:56,427::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`88abad86-ca7a-403c-bc59-f165a52ba377`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910195::DEBUG::2013-11-19 21:04:56,428::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910195::DEBUG::2013-11-19 21:04:56,429::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910195::DEBUG::2013-11-19 21:04:56,429::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`88abad86-ca7a-403c-bc59-f165a52ba377`::Granted request
>Thread-15910195::DEBUG::2013-11-19 21:04:56,434::task::495::TaskManager.Task::(_debug) Task 39f8fc9c-aa50-4258-bc4a-861b38ae973a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910195::DEBUG::2013-11-19 21:04:56,435::task::495::TaskManager.Task::(_debug) Task 39f8fc9c-aa50-4258-bc4a-861b38ae973a: ref 1 aborting False
>Thread-15910195::WARNING::2013-11-19 21:04:56,438::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910195::DEBUG::2013-11-19 21:04:56,439::task::495::TaskManager.Task::(_debug) Task 39f8fc9c-aa50-4258-bc4a-861b38ae973a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910195::DEBUG::2013-11-19 21:04:56,440::task::495::TaskManager.Task::(_debug) Task 39f8fc9c-aa50-4258-bc4a-861b38ae973a: moving from state preparing -> state finished
>Thread-15910195::DEBUG::2013-11-19 21:04:56,441::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910195::DEBUG::2013-11-19 21:04:56,441::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910195::DEBUG::2013-11-19 21:04:56,442::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910195::DEBUG::2013-11-19 21:04:56,443::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910195::DEBUG::2013-11-19 21:04:56,443::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910195::DEBUG::2013-11-19 21:04:56,444::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910195::DEBUG::2013-11-19 21:04:56,445::task::495::TaskManager.Task::(_debug) Task 39f8fc9c-aa50-4258-bc4a-861b38ae973a: ref 0 aborting False
>Thread-15910195::INFO::2013-11-19 21:04:56,446::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-4922912::DEBUG::2013-11-19 21:04:56,691::guestIF::97::vm.Vm::(_handleMessage) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::u'None'
>Dummy-15909965::DEBUG::2013-11-19 21:04:56,881::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:04:57,513::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0886279 s, 11.6 MB/s\n'; <rc> = 0
>Thread-15910197::DEBUG::2013-11-19 21:04:57,942::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('6574d88c-00ac-46f4-99ce-a2726932bfda',) {}
>Thread-15910197::DEBUG::2013-11-19 21:04:57,944::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': u'None', 'memUsage': '92', 'acpiEnable': 'true', 'pid': '57291', 'displayPort': u'5908', 'session': 'LoggedOff', 'netIfaces': [{u'inet6': [], u'hw': u'00-1a-4a-a8-64-08', u'inet': [u'192.168.100.70'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': -1L, 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '38686343168', u'fs': u'NTFS'}, {u'path': u'h:\\', u'total': '107237863424', u'used': '99742478336', u'fs': u'NTFS'}, {u'path': u'l:\\', u'total': '150187474944', u'used': '130882863104', u'fs': u'NTFS'}, {u'path': u'x:\\', u'total': '429360349184', u'used': '330617978880', u'fs': u'NTFS'}], 'network': {u'vnet5': {'macAddr': u'00:1a:4a:a8:64:08', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet5'}}, 'displayType': 'qxl', 'guestName': u'KLEE-AXDB01', 'boot': 'cd', 'elapsedTime': '18663707', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Microsoft Report Viewer Redistributable 2008 SP1', u'Zabbix Agent 2.0.8', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'Microsoft SQL Server Compact 3.5 SP2 ENU', u'RHEV-Spice-Agent64', u'Microsoft Visual Studio Tools for Applications 2.0 - ENU', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'Microsoft Office 2003 Web Components', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10', u'Microsoft SQL Server Browser', u'Microsoft Report Viewer Redistributable 2008 (KB971119)', u'Microsoft SQL Server 2008 R2 Policies', u'Microsoft SQL Server Compact 3.5 SP2 Query Tools ENU'], 'guestOs': u'Unknown', 'nice': '', 'status': 'Powering down', 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'displayIp': '0', 'lastLogout': 1366231390.568368, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '3652045', 'apparentsize': '64424509440', 'writeLatency': '312295', 'imageID': 'c4460480-32d3-469f-b0f4-c03bb6d74e41', 'flushLatency': '183455', 'readRate': '145382.38', 'truesize': '64424509440', 'writeRate': '189200.51'}, u'vdc': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '107374182400', 'writeRate': '0.00'}, u'vdb': {'readLatency': '0', 'apparentsize': '150323855360', 'writeLatency': '0', 'imageID': 'df8b790c-c762-4feb-9378-d64a321ce43c', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '150323855360', 'writeRate': '0.00'}, u'vdd': {'readLatency': '0', 'apparentsize': '429496729600', 'writeLatency': '0', 'imageID': 'd368307e-425e-423a-85b7-266c1e4a99ec', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '429496729600', 'writeRate': '0.00'}}, 'monitorResponse': '0', 'statsAge': '1.28', 'cpuUser': '172.48', 'cpuIdle': '0.00', 'lastLogin': 1384895046.5037899, 'guestIPs': u'192.168.100.70', 'displaySecurePort': u'5909'}]}
>Dummy-15909965::DEBUG::2013-11-19 21:04:59,523::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:05:00,039::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0747381 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15910198::INFO::2013-11-19 21:05:00,074::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910198::DEBUG::2013-11-19 21:05:00,075::task::495::TaskManager.Task::(_debug) Task 6720d63b-bbde-4736-94f3-aa7ae5afac72: moving from state init -> state preparing
>Thread-15910198::DEBUG::2013-11-19 21:05:00,076::task::495::TaskManager.Task::(_debug) Task 6720d63b-bbde-4736-94f3-aa7ae5afac72: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000296115875244', 'lastCheck': 1384895090.5389211, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00112795829773', 'lastCheck': 1384895096.6210091, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895091.545125, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00135493278503', 'lastCheck': 1384895099.738771, 'valid': True, 'code': 0}}
>Thread-15910198::DEBUG::2013-11-19 21:05:00,077::task::495::TaskManager.Task::(_debug) Task 6720d63b-bbde-4736-94f3-aa7ae5afac72: moving from state preparing -> state finished
>Thread-15910198::DEBUG::2013-11-19 21:05:00,078::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910198::DEBUG::2013-11-19 21:05:00,078::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910198::DEBUG::2013-11-19 21:05:00,079::task::495::TaskManager.Task::(_debug) Task 6720d63b-bbde-4736-94f3-aa7ae5afac72: ref 0 aborting False
>Thread-15910198::INFO::2013-11-19 21:05:00,080::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000296115875244', 'lastCheck': 1384895090.5389211, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00112795829773', 'lastCheck': 1384895096.6210091, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895091.545125, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00135493278503', 'lastCheck': 1384895099.738771, 'valid': True, 'code': 0}}
>Thread-15910153::DEBUG::2013-11-19 21:05:01,445::vm::692::vm.Vm::(_timedShutdown) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::_timedShutdown Called
>Thread-1024767::ERROR::2013-11-19 21:05:01,742::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15909965::DEBUG::2013-11-19 21:05:02,055::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15910201::DEBUG::2013-11-19 21:05:02,327::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('6574d88c-00ac-46f4-99ce-a2726932bfda',) {}
>Thread-15910201::DEBUG::2013-11-19 21:05:02,342::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': u'None', 'memUsage': '91', 'acpiEnable': 'true', 'pid': '57291', 'displayPort': u'5908', 'session': 'LoggedOff', 'netIfaces': [{u'inet6': [], u'hw': u'00-1a-4a-a8-64-08', u'inet': [u'192.168.100.70'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': -1L, 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '38686343168', u'fs': u'NTFS'}, {u'path': u'h:\\', u'total': '107237863424', u'used': '99742478336', u'fs': u'NTFS'}, {u'path': u'l:\\', u'total': '150187474944', u'used': '130882863104', u'fs': u'NTFS'}, {u'path': u'x:\\', u'total': '429360349184', u'used': '330617978880', u'fs': u'NTFS'}], 'network': {u'vnet5': {'macAddr': u'00:1a:4a:a8:64:08', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet5'}}, 'displayType': 'qxl', 'guestName': u'KLEE-AXDB01', 'boot': 'cd', 'elapsedTime': '18663711', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Microsoft Report Viewer Redistributable 2008 SP1', u'Zabbix Agent 2.0.8', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'Microsoft SQL Server Compact 3.5 SP2 ENU', u'RHEV-Spice-Agent64', u'Microsoft Visual Studio Tools for Applications 2.0 - ENU', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'Microsoft Office 2003 Web Components', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10', u'Microsoft SQL Server Browser', u'Microsoft Report Viewer Redistributable 2008 (KB971119)', u'Microsoft SQL Server 2008 R2 Policies', u'Microsoft SQL Server Compact 3.5 SP2 Query Tools ENU'], 'guestOs': u'Unknown', 'nice': '', 'status': 'Powering down', 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'displayIp': '0', 'lastLogout': 1366231390.568368, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '3652045', 'apparentsize': '64424509440', 'writeLatency': '312295', 'imageID': 'c4460480-32d3-469f-b0f4-c03bb6d74e41', 'flushLatency': '183455', 'readRate': '145382.38', 'truesize': '64424509440', 'writeRate': '189200.51'}, u'vdc': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '107374182400', 'writeRate': '0.00'}, u'vdb': {'readLatency': '0', 'apparentsize': '150323855360', 'writeLatency': '0', 'imageID': 'df8b790c-c762-4feb-9378-d64a321ce43c', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '150323855360', 'writeRate': '0.00'}, u'vdd': {'readLatency': '0', 'apparentsize': '429496729600', 'writeLatency': '0', 'imageID': 'd368307e-425e-423a-85b7-266c1e4a99ec', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '429496729600', 'writeRate': '0.00'}}, 'monitorResponse': '0', 'statsAge': '1.59', 'cpuUser': '172.48', 'cpuIdle': '0.00', 'lastLogin': 1384895046.5037899, 'guestIPs': u'192.168.100.70', 'displaySecurePort': u'5909'}]}
>Dummy-15909965::DEBUG::2013-11-19 21:05:02,648::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0822316 s, 12.5 MB/s\n'; <rc> = 0
>Thread-4922912::DEBUG::2013-11-19 21:05:03,671::guestIF::97::vm.Vm::(_handleMessage) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::u'None'
>Thread-15910203::DEBUG::2013-11-19 21:05:04,491::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('6574d88c-00ac-46f4-99ce-a2726932bfda',) {}
>Thread-15910203::DEBUG::2013-11-19 21:05:04,492::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': u'None', 'memUsage': '91', 'acpiEnable': 'true', 'pid': '57291', 'displayPort': u'5908', 'session': 'LoggedOff', 'netIfaces': [{u'inet6': [], u'hw': u'00-1a-4a-a8-64-08', u'inet': [u'192.168.100.70'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': -1L, 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '38686343168', u'fs': u'NTFS'}, {u'path': u'h:\\', u'total': '107237863424', u'used': '99742478336', u'fs': u'NTFS'}, {u'path': u'l:\\', u'total': '150187474944', u'used': '130882863104', u'fs': u'NTFS'}, {u'path': u'x:\\', u'total': '429360349184', u'used': '330617978880', u'fs': u'NTFS'}], 'network': {u'vnet5': {'macAddr': u'00:1a:4a:a8:64:08', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet5'}}, 'displayType': 'qxl', 'guestName': u'KLEE-AXDB01', 'boot': 'cd', 'elapsedTime': '18663713', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Microsoft Report Viewer Redistributable 2008 SP1', u'Zabbix Agent 2.0.8', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'Microsoft SQL Server Compact 3.5 SP2 ENU', u'RHEV-Spice-Agent64', u'Microsoft Visual Studio Tools for Applications 2.0 - ENU', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'Microsoft Office 2003 Web Components', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10', u'Microsoft SQL Server Browser', u'Microsoft Report Viewer Redistributable 2008 (KB971119)', u'Microsoft SQL Server 2008 R2 Policies', u'Microsoft SQL Server Compact 3.5 SP2 Query Tools ENU'], 'guestOs': u'Unknown', 'nice': '', 'status': 'Powering down', 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'displayIp': '0', 'lastLogout': 1366231390.568368, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '3652045', 'apparentsize': '64424509440', 'writeLatency': '312295', 'imageID': 'c4460480-32d3-469f-b0f4-c03bb6d74e41', 'flushLatency': '183455', 'readRate': '145382.38', 'truesize': '64424509440', 'writeRate': '189200.51'}, u'vdc': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '107374182400', 'writeRate': '0.00'}, u'vdb': {'readLatency': '0', 'apparentsize': '150323855360', 'writeLatency': '0', 'imageID': 'df8b790c-c762-4feb-9378-d64a321ce43c', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '150323855360', 'writeRate': '0.00'}, u'vdd': {'readLatency': '0', 'apparentsize': '429496729600', 'writeLatency': '0', 'imageID': 'd368307e-425e-423a-85b7-266c1e4a99ec', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '429496729600', 'writeRate': '0.00'}}, 'monitorResponse': '0', 'statsAge': '1.75', 'cpuUser': '172.48', 'cpuIdle': '0.00', 'lastLogin': 1384895046.5037899, 'guestIPs': u'192.168.100.70', 'displaySecurePort': u'5909'}]}
>Dummy-15909965::DEBUG::2013-11-19 21:05:04,662::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:05:05,261::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0812764 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15910204::DEBUG::2013-11-19 21:05:06,614::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910204::INFO::2013-11-19 21:05:06,617::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910204::DEBUG::2013-11-19 21:05:06,623::task::495::TaskManager.Task::(_debug) Task a6196c2a-5b31-41b0-8296-51a3985f525c: moving from state init -> state preparing
>Thread-15910204::DEBUG::2013-11-19 21:05:06,627::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=274 spmId=2
>Thread-15910204::DEBUG::2013-11-19 21:05:06,629::task::495::TaskManager.Task::(_debug) Task a6196c2a-5b31-41b0-8296-51a3985f525c: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910204::DEBUG::2013-11-19 21:05:06,631::task::495::TaskManager.Task::(_debug) Task a6196c2a-5b31-41b0-8296-51a3985f525c: moving from state preparing -> state finished
>Thread-15910204::DEBUG::2013-11-19 21:05:06,636::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910204::DEBUG::2013-11-19 21:05:06,639::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910204::DEBUG::2013-11-19 21:05:06,642::task::495::TaskManager.Task::(_debug) Task a6196c2a-5b31-41b0-8296-51a3985f525c: ref 0 aborting False
>Thread-15910204::INFO::2013-11-19 21:05:06,646::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910207::DEBUG::2013-11-19 21:05:06,697::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('6574d88c-00ac-46f4-99ce-a2726932bfda',) {}
>Thread-15910207::DEBUG::2013-11-19 21:05:06,698::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': u'None', 'memUsage': '91', 'acpiEnable': 'true', 'pid': '57291', 'displayPort': u'5908', 'session': 'LoggedOff', 'netIfaces': [{u'inet6': [], u'hw': u'00-1a-4a-a8-64-08', u'inet': [u'192.168.100.70'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': -1L, 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '38686343168', u'fs': u'NTFS'}, {u'path': u'h:\\', u'total': '107237863424', u'used': '99742478336', u'fs': u'NTFS'}, {u'path': u'l:\\', u'total': '150187474944', u'used': '130882863104', u'fs': u'NTFS'}, {u'path': u'x:\\', u'total': '429360349184', u'used': '330617978880', u'fs': u'NTFS'}], 'network': {u'vnet5': {'macAddr': u'00:1a:4a:a8:64:08', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet5'}}, 'displayType': 'qxl', 'guestName': u'KLEE-AXDB01', 'boot': 'cd', 'elapsedTime': '18663716', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Microsoft Report Viewer Redistributable 2008 SP1', u'Zabbix Agent 2.0.8', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'Microsoft SQL Server Compact 3.5 SP2 ENU', u'RHEV-Spice-Agent64', u'Microsoft Visual Studio Tools for Applications 2.0 - ENU', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'Microsoft Office 2003 Web Components', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10', u'Microsoft SQL Server Browser', u'Microsoft Report Viewer Redistributable 2008 (KB971119)', u'Microsoft SQL Server 2008 R2 Policies', u'Microsoft SQL Server Compact 3.5 SP2 Query Tools ENU'], 'guestOs': u'Unknown', 'nice': '', 'status': 'Powering down', 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'displayIp': '0', 'lastLogout': 1366231390.568368, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '3652045', 'apparentsize': '64424509440', 'writeLatency': '312295', 'imageID': 'c4460480-32d3-469f-b0f4-c03bb6d74e41', 'flushLatency': '183455', 'readRate': '145382.38', 'truesize': '64424509440', 'writeRate': '189200.51'}, u'vdc': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '107374182400', 'writeRate': '0.00'}, u'vdb': {'readLatency': '0', 'apparentsize': '150323855360', 'writeLatency': '0', 'imageID': 'df8b790c-c762-4feb-9378-d64a321ce43c', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '150323855360', 'writeRate': '0.00'}, u'vdd': {'readLatency': '0', 'apparentsize': '429496729600', 'writeLatency': '0', 'imageID': 'd368307e-425e-423a-85b7-266c1e4a99ec', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '429496729600', 'writeRate': '0.00'}}, 'monitorResponse': '0', 'statsAge': '1.79', 'cpuUser': '172.48', 'cpuIdle': '0.00', 'lastLogin': 1384895046.5037899, 'guestIPs': u'192.168.100.70', 'displaySecurePort': u'5909'}]}
>Thread-15910206::DEBUG::2013-11-19 21:05:06,703::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910206::INFO::2013-11-19 21:05:06,704::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910206::DEBUG::2013-11-19 21:05:06,705::task::495::TaskManager.Task::(_debug) Task bc69bf5c-e9ce-4f38-9913-41eefa127ee9: moving from state init -> state preparing
>Thread-15910206::DEBUG::2013-11-19 21:05:06,711::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`32804ff6-4298-471b-aeda-8d77e2f02f98`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910206::DEBUG::2013-11-19 21:05:06,716::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910206::DEBUG::2013-11-19 21:05:06,717::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910206::DEBUG::2013-11-19 21:05:06,718::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`32804ff6-4298-471b-aeda-8d77e2f02f98`::Granted request
>Thread-15910206::DEBUG::2013-11-19 21:05:06,719::task::495::TaskManager.Task::(_debug) Task bc69bf5c-e9ce-4f38-9913-41eefa127ee9: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910206::DEBUG::2013-11-19 21:05:06,722::task::495::TaskManager.Task::(_debug) Task bc69bf5c-e9ce-4f38-9913-41eefa127ee9: ref 1 aborting False
>Thread-15910206::WARNING::2013-11-19 21:05:06,729::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910206::DEBUG::2013-11-19 21:05:06,748::task::495::TaskManager.Task::(_debug) Task bc69bf5c-e9ce-4f38-9913-41eefa127ee9: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910206::DEBUG::2013-11-19 21:05:06,768::task::495::TaskManager.Task::(_debug) Task bc69bf5c-e9ce-4f38-9913-41eefa127ee9: moving from state preparing -> state finished
>Thread-15910206::DEBUG::2013-11-19 21:05:06,768::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910206::DEBUG::2013-11-19 21:05:06,774::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910206::DEBUG::2013-11-19 21:05:06,775::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910206::DEBUG::2013-11-19 21:05:06,784::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910206::DEBUG::2013-11-19 21:05:06,785::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910206::DEBUG::2013-11-19 21:05:06,785::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910206::DEBUG::2013-11-19 21:05:06,786::task::495::TaskManager.Task::(_debug) Task bc69bf5c-e9ce-4f38-9913-41eefa127ee9: ref 0 aborting False
>Thread-15910206::INFO::2013-11-19 21:05:06,795::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15909965::DEBUG::2013-11-19 21:05:07,276::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:05:07,855::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0780894 s, 13.1 MB/s\n'; <rc> = 0
>Thread-15910209::DEBUG::2013-11-19 21:05:08,794::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('6574d88c-00ac-46f4-99ce-a2726932bfda',) {}
>Thread-15910209::DEBUG::2013-11-19 21:05:08,797::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': u'None', 'memUsage': '91', 'acpiEnable': 'true', 'pid': '57291', 'displayPort': u'5908', 'session': 'LoggedOff', 'netIfaces': [{u'inet6': [], u'hw': u'00-1a-4a-a8-64-08', u'inet': [u'192.168.100.70'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': -1L, 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '38686343168', u'fs': u'NTFS'}, {u'path': u'h:\\', u'total': '107237863424', u'used': '99742478336', u'fs': u'NTFS'}, {u'path': u'l:\\', u'total': '150187474944', u'used': '130882863104', u'fs': u'NTFS'}, {u'path': u'x:\\', u'total': '429360349184', u'used': '330617978880', u'fs': u'NTFS'}], 'network': {u'vnet5': {'macAddr': u'00:1a:4a:a8:64:08', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet5'}}, 'displayType': 'qxl', 'guestName': u'KLEE-AXDB01', 'boot': 'cd', 'elapsedTime': '18663718', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Microsoft Report Viewer Redistributable 2008 SP1', u'Zabbix Agent 2.0.8', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'Microsoft SQL Server Compact 3.5 SP2 ENU', u'RHEV-Spice-Agent64', u'Microsoft Visual Studio Tools for Applications 2.0 - ENU', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'Microsoft Office 2003 Web Components', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10', u'Microsoft SQL Server Browser', u'Microsoft Report Viewer Redistributable 2008 (KB971119)', u'Microsoft SQL Server 2008 R2 Policies', u'Microsoft SQL Server Compact 3.5 SP2 Query Tools ENU'], 'guestOs': u'Unknown', 'nice': '', 'status': 'Powering down', 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'displayIp': '0', 'lastLogout': 1366231390.568368, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '3652045', 'apparentsize': '64424509440', 'writeLatency': '312295', 'imageID': 'c4460480-32d3-469f-b0f4-c03bb6d74e41', 'flushLatency': '183455', 'readRate': '145382.38', 'truesize': '64424509440', 'writeRate': '189200.51'}, u'vdc': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '107374182400', 'writeRate': '0.00'}, u'vdb': {'readLatency': '0', 'apparentsize': '150323855360', 'writeLatency': '0', 'imageID': 'df8b790c-c762-4feb-9378-d64a321ce43c', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '150323855360', 'writeRate': '0.00'}, u'vdd': {'readLatency': '0', 'apparentsize': '429496729600', 'writeLatency': '0', 'imageID': 'd368307e-425e-423a-85b7-266c1e4a99ec', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '429496729600', 'writeRate': '0.00'}}, 'monitorResponse': '0', 'statsAge': '1.88', 'cpuUser': '172.48', 'cpuIdle': '0.00', 'lastLogin': 1384895046.5037899, 'guestIPs': u'192.168.100.70', 'displaySecurePort': u'5909'}]}
>Dummy-15909965::DEBUG::2013-11-19 21:05:09,871::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:05:10,247::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0436375 s, 23.5 MB/s\n'; <rc> = 0
>Thread-15910210::INFO::2013-11-19 21:05:10,948::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910210::DEBUG::2013-11-19 21:05:10,949::task::495::TaskManager.Task::(_debug) Task 2332005f-5c81-41a1-9fa8-a768b95a0e00: moving from state init -> state preparing
>Thread-15910210::DEBUG::2013-11-19 21:05:10,951::task::495::TaskManager.Task::(_debug) Task 2332005f-5c81-41a1-9fa8-a768b95a0e00: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000334978103638', 'lastCheck': 1384895100.739496, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148987770081', 'lastCheck': 1384895106.9134469, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895101.7436941, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000579833984375', 'lastCheck': 1384895110.038691, 'valid': True, 'code': 0}}
>Thread-15910210::DEBUG::2013-11-19 21:05:10,967::task::495::TaskManager.Task::(_debug) Task 2332005f-5c81-41a1-9fa8-a768b95a0e00: moving from state preparing -> state finished
>Thread-15910210::DEBUG::2013-11-19 21:05:10,968::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910210::DEBUG::2013-11-19 21:05:10,969::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910210::DEBUG::2013-11-19 21:05:10,970::task::495::TaskManager.Task::(_debug) Task 2332005f-5c81-41a1-9fa8-a768b95a0e00: ref 0 aborting False
>Thread-15910210::INFO::2013-11-19 21:05:10,975::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000334978103638', 'lastCheck': 1384895100.739496, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148987770081', 'lastCheck': 1384895106.9134469, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895101.7436941, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000579833984375', 'lastCheck': 1384895110.038691, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:05:12,039::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15909965::DEBUG::2013-11-19 21:05:12,257::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:05:12,818::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.109251 s, 9.4 MB/s\n'; <rc> = 0
>Thread-15910213::DEBUG::2013-11-19 21:05:13,132::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('6574d88c-00ac-46f4-99ce-a2726932bfda',) {}
>Thread-15910213::DEBUG::2013-11-19 21:05:13,134::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': u'None', 'memUsage': '91', 'acpiEnable': 'true', 'pid': '57291', 'displayPort': u'5908', 'session': 'LoggedOff', 'netIfaces': [{u'inet6': [], u'hw': u'00-1a-4a-a8-64-08', u'inet': [u'192.168.100.70'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': -1L, 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '38686343168', u'fs': u'NTFS'}, {u'path': u'h:\\', u'total': '107237863424', u'used': '99742478336', u'fs': u'NTFS'}, {u'path': u'l:\\', u'total': '150187474944', u'used': '130882863104', u'fs': u'NTFS'}, {u'path': u'x:\\', u'total': '429360349184', u'used': '330617978880', u'fs': u'NTFS'}], 'network': {u'vnet5': {'macAddr': u'00:1a:4a:a8:64:08', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet5'}}, 'displayType': 'qxl', 'guestName': u'KLEE-AXDB01', 'boot': 'cd', 'elapsedTime': '18663722', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Microsoft Report Viewer Redistributable 2008 SP1', u'Zabbix Agent 2.0.8', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'Microsoft SQL Server Compact 3.5 SP2 ENU', u'RHEV-Spice-Agent64', u'Microsoft Visual Studio Tools for Applications 2.0 - ENU', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'Microsoft Office 2003 Web Components', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10', u'Microsoft SQL Server Browser', u'Microsoft Report Viewer Redistributable 2008 (KB971119)', u'Microsoft SQL Server 2008 R2 Policies', u'Microsoft SQL Server Compact 3.5 SP2 Query Tools ENU'], 'guestOs': u'Unknown', 'nice': '', 'status': 'Powering down', 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'displayIp': '0', 'lastLogout': 1366231390.568368, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '3652045', 'apparentsize': '64424509440', 'writeLatency': '312295', 'imageID': 'c4460480-32d3-469f-b0f4-c03bb6d74e41', 'flushLatency': '183455', 'readRate': '145382.38', 'truesize': '64424509440', 'writeRate': '189200.51'}, u'vdc': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '107374182400', 'writeRate': '0.00'}, u'vdb': {'readLatency': '0', 'apparentsize': '150323855360', 'writeLatency': '0', 'imageID': 'df8b790c-c762-4feb-9378-d64a321ce43c', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '150323855360', 'writeRate': '0.00'}, u'vdd': {'readLatency': '0', 'apparentsize': '429496729600', 'writeLatency': '0', 'imageID': 'd368307e-425e-423a-85b7-266c1e4a99ec', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '429496729600', 'writeRate': '0.00'}}, 'monitorResponse': '0', 'statsAge': '0.09', 'cpuUser': '175.49', 'cpuIdle': '0.00', 'lastLogin': 1384895046.5037899, 'guestIPs': u'192.168.100.70', 'displaySecurePort': u'5909'}]}
>Thread-4922912::DEBUG::2013-11-19 21:05:14,508::guestIF::112::vm.Vm::(_handleMessage) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Guest system shuts down.
>libvirtEventLoop::DEBUG::2013-11-19 21:05:14,595::vm::525::vm.Vm::(_rtcUpdate) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::new rtc offset -1
>libvirtEventLoop::DEBUG::2013-11-19 21:05:14,598::vm::525::vm.Vm::(_rtcUpdate) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::new rtc offset -1
>libvirtEventLoop::DEBUG::2013-11-19 21:05:14,603::vm::525::vm.Vm::(_rtcUpdate) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::new rtc offset -1
>libvirtEventLoop::DEBUG::2013-11-19 21:05:14,605::vm::525::vm.Vm::(_rtcUpdate) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::new rtc offset -1
>libvirtEventLoop::DEBUG::2013-11-19 21:05:14,606::vm::525::vm.Vm::(_rtcUpdate) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::new rtc offset -1
>libvirtEventLoop::DEBUG::2013-11-19 21:05:14,608::vm::525::vm.Vm::(_rtcUpdate) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::new rtc offset -1
>libvirtEventLoop::DEBUG::2013-11-19 21:05:14,609::vm::525::vm.Vm::(_rtcUpdate) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::new rtc offset -1
>Dummy-15909965::DEBUG::2013-11-19 21:05:14,846::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15910215::DEBUG::2013-11-19 21:05:15,262::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('6574d88c-00ac-46f4-99ce-a2726932bfda',) {}
>Thread-15910215::DEBUG::2013-11-19 21:05:15,264::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': u'None', 'memUsage': '91', 'acpiEnable': 'true', 'pid': '57291', 'displayPort': u'5908', 'session': 'LoggedOff', 'netIfaces': [{u'inet6': [], u'hw': u'00-1a-4a-a8-64-08', u'inet': [u'192.168.100.70'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': -1L, 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '38686343168', u'fs': u'NTFS'}, {u'path': u'h:\\', u'total': '107237863424', u'used': '99742478336', u'fs': u'NTFS'}, {u'path': u'l:\\', u'total': '150187474944', u'used': '130882863104', u'fs': u'NTFS'}, {u'path': u'x:\\', u'total': '429360349184', u'used': '330617978880', u'fs': u'NTFS'}], 'network': {u'vnet5': {'macAddr': u'00:1a:4a:a8:64:08', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet5'}}, 'displayType': 'qxl', 'guestName': u'KLEE-AXDB01', 'boot': 'cd', 'elapsedTime': '18663724', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Microsoft Report Viewer Redistributable 2008 SP1', u'Zabbix Agent 2.0.8', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'Microsoft SQL Server Compact 3.5 SP2 ENU', u'RHEV-Spice-Agent64', u'Microsoft Visual Studio Tools for Applications 2.0 - ENU', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'Microsoft Office 2003 Web Components', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10', u'Microsoft SQL Server Browser', u'Microsoft Report Viewer Redistributable 2008 (KB971119)', u'Microsoft SQL Server 2008 R2 Policies', u'Microsoft SQL Server Compact 3.5 SP2 Query Tools ENU'], 'guestOs': u'Unknown', 'nice': '', 'status': 'Powering down', 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'displayIp': '0', 'lastLogout': 1366231390.568368, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '3652045', 'apparentsize': '64424509440', 'writeLatency': '312295', 'imageID': 'c4460480-32d3-469f-b0f4-c03bb6d74e41', 'flushLatency': '183455', 'readRate': '145382.38', 'truesize': '64424509440', 'writeRate': '189200.51'}, u'vdc': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '107374182400', 'writeRate': '0.00'}, u'vdb': {'readLatency': '0', 'apparentsize': '150323855360', 'writeLatency': '0', 'imageID': 'df8b790c-c762-4feb-9378-d64a321ce43c', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '150323855360', 'writeRate': '0.00'}, u'vdd': {'readLatency': '0', 'apparentsize': '429496729600', 'writeLatency': '0', 'imageID': 'd368307e-425e-423a-85b7-266c1e4a99ec', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '429496729600', 'writeRate': '0.00'}}, 'monitorResponse': '0', 'statsAge': '0.19', 'cpuUser': '175.49', 'cpuIdle': '0.00', 'lastLogin': 1384895046.5037899, 'guestIPs': u'192.168.100.70', 'displaySecurePort': u'5909'}]}
>Dummy-15909965::DEBUG::2013-11-19 21:05:15,512::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.148877 s, 6.9 MB/s\n'; <rc> = 0
>Thread-15910216::DEBUG::2013-11-19 21:05:16,925::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910216::INFO::2013-11-19 21:05:16,926::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910216::DEBUG::2013-11-19 21:05:16,927::task::495::TaskManager.Task::(_debug) Task 5c65d653-481c-4bf8-a3e6-6ece3a326f75: moving from state init -> state preparing
>Thread-15910216::DEBUG::2013-11-19 21:05:16,928::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=274 spmId=2
>Thread-15910216::DEBUG::2013-11-19 21:05:16,929::task::495::TaskManager.Task::(_debug) Task 5c65d653-481c-4bf8-a3e6-6ece3a326f75: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910216::DEBUG::2013-11-19 21:05:16,936::task::495::TaskManager.Task::(_debug) Task 5c65d653-481c-4bf8-a3e6-6ece3a326f75: moving from state preparing -> state finished
>Thread-15910216::DEBUG::2013-11-19 21:05:16,940::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910216::DEBUG::2013-11-19 21:05:16,940::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910216::DEBUG::2013-11-19 21:05:16,941::task::495::TaskManager.Task::(_debug) Task 5c65d653-481c-4bf8-a3e6-6ece3a326f75: ref 0 aborting False
>Thread-15910216::INFO::2013-11-19 21:05:16,941::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910217::DEBUG::2013-11-19 21:05:16,955::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910217::INFO::2013-11-19 21:05:16,956::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910217::DEBUG::2013-11-19 21:05:16,957::task::495::TaskManager.Task::(_debug) Task 80481dac-54e5-4b92-ab0e-a4f91b9dad41: moving from state init -> state preparing
>Thread-15910217::DEBUG::2013-11-19 21:05:16,960::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c2d9d2d7-53c4-449e-b4b3-a17435af21b0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910217::DEBUG::2013-11-19 21:05:16,966::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910217::DEBUG::2013-11-19 21:05:16,967::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910217::DEBUG::2013-11-19 21:05:16,971::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c2d9d2d7-53c4-449e-b4b3-a17435af21b0`::Granted request
>Thread-15910217::DEBUG::2013-11-19 21:05:16,980::task::495::TaskManager.Task::(_debug) Task 80481dac-54e5-4b92-ab0e-a4f91b9dad41: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910217::DEBUG::2013-11-19 21:05:16,981::task::495::TaskManager.Task::(_debug) Task 80481dac-54e5-4b92-ab0e-a4f91b9dad41: ref 1 aborting False
>Thread-15910217::WARNING::2013-11-19 21:05:16,991::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910217::DEBUG::2013-11-19 21:05:17,001::task::495::TaskManager.Task::(_debug) Task 80481dac-54e5-4b92-ab0e-a4f91b9dad41: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910217::DEBUG::2013-11-19 21:05:17,013::task::495::TaskManager.Task::(_debug) Task 80481dac-54e5-4b92-ab0e-a4f91b9dad41: moving from state preparing -> state finished
>Thread-15910217::DEBUG::2013-11-19 21:05:17,014::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910217::DEBUG::2013-11-19 21:05:17,023::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910217::DEBUG::2013-11-19 21:05:17,024::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910217::DEBUG::2013-11-19 21:05:17,025::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910217::DEBUG::2013-11-19 21:05:17,026::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910217::DEBUG::2013-11-19 21:05:17,027::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910217::DEBUG::2013-11-19 21:05:17,028::task::495::TaskManager.Task::(_debug) Task 80481dac-54e5-4b92-ab0e-a4f91b9dad41: ref 0 aborting False
>Thread-15910217::INFO::2013-11-19 21:05:17,028::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910219::DEBUG::2013-11-19 21:05:17,445::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('6574d88c-00ac-46f4-99ce-a2726932bfda',) {}
>Thread-15910219::DEBUG::2013-11-19 21:05:17,449::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': u'None', 'memUsage': '91', 'acpiEnable': 'true', 'pid': '57291', 'displayPort': u'5908', 'session': 'LoggedOff', 'netIfaces': [{u'inet6': [], u'hw': u'00-1a-4a-a8-64-08', u'inet': [u'192.168.100.70'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': -1L, 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '38686343168', u'fs': u'NTFS'}, {u'path': u'h:\\', u'total': '107237863424', u'used': '99742478336', u'fs': u'NTFS'}, {u'path': u'l:\\', u'total': '150187474944', u'used': '130882863104', u'fs': u'NTFS'}, {u'path': u'x:\\', u'total': '429360349184', u'used': '330617978880', u'fs': u'NTFS'}], 'network': {u'vnet5': {'macAddr': u'00:1a:4a:a8:64:08', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet5'}}, 'displayType': 'qxl', 'guestName': u'KLEE-AXDB01', 'boot': 'cd', 'elapsedTime': '18663726', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Microsoft Report Viewer Redistributable 2008 SP1', u'Zabbix Agent 2.0.8', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'Microsoft SQL Server Compact 3.5 SP2 ENU', u'RHEV-Spice-Agent64', u'Microsoft Visual Studio Tools for Applications 2.0 - ENU', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'Microsoft Office 2003 Web Components', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10', u'Microsoft SQL Server Browser', u'Microsoft Report Viewer Redistributable 2008 (KB971119)', u'Microsoft SQL Server 2008 R2 Policies', u'Microsoft SQL Server Compact 3.5 SP2 Query Tools ENU'], 'guestOs': u'Unknown', 'nice': '', 'status': 'Powering down', 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'displayIp': '0', 'lastLogout': 1366231390.568368, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '3652045', 'apparentsize': '64424509440', 'writeLatency': '312295', 'imageID': 'c4460480-32d3-469f-b0f4-c03bb6d74e41', 'flushLatency': '183455', 'readRate': '145382.38', 'truesize': '64424509440', 'writeRate': '189200.51'}, u'vdc': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '107374182400', 'writeRate': '0.00'}, u'vdb': {'readLatency': '0', 'apparentsize': '150323855360', 'writeLatency': '0', 'imageID': 'df8b790c-c762-4feb-9378-d64a321ce43c', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '150323855360', 'writeRate': '0.00'}, u'vdd': {'readLatency': '0', 'apparentsize': '429496729600', 'writeLatency': '0', 'imageID': 'd368307e-425e-423a-85b7-266c1e4a99ec', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '429496729600', 'writeRate': '0.00'}}, 'monitorResponse': '0', 'statsAge': '0.37', 'cpuUser': '175.49', 'cpuIdle': '0.00', 'lastLogin': 1384895046.5037899, 'guestIPs': u'192.168.100.70', 'displaySecurePort': u'5909'}]}
>Dummy-15909965::DEBUG::2013-11-19 21:05:17,526::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:05:18,129::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0786111 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15910221::DEBUG::2013-11-19 21:05:19,574::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('6574d88c-00ac-46f4-99ce-a2726932bfda',) {}
>Thread-15910221::DEBUG::2013-11-19 21:05:19,576::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': u'None', 'memUsage': '91', 'acpiEnable': 'true', 'pid': '57291', 'displayPort': u'5908', 'session': 'LoggedOff', 'netIfaces': [{u'inet6': [], u'hw': u'00-1a-4a-a8-64-08', u'inet': [u'192.168.100.70'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': -1L, 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '38686343168', u'fs': u'NTFS'}, {u'path': u'h:\\', u'total': '107237863424', u'used': '99742478336', u'fs': u'NTFS'}, {u'path': u'l:\\', u'total': '150187474944', u'used': '130882863104', u'fs': u'NTFS'}, {u'path': u'x:\\', u'total': '429360349184', u'used': '330617978880', u'fs': u'NTFS'}], 'network': {u'vnet5': {'macAddr': u'00:1a:4a:a8:64:08', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet5'}}, 'displayType': 'qxl', 'guestName': u'KLEE-AXDB01', 'boot': 'cd', 'elapsedTime': '18663729', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Microsoft Report Viewer Redistributable 2008 SP1', u'Zabbix Agent 2.0.8', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'Microsoft SQL Server Compact 3.5 SP2 ENU', u'RHEV-Spice-Agent64', u'Microsoft Visual Studio Tools for Applications 2.0 - ENU', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'Microsoft Office 2003 Web Components', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10', u'Microsoft SQL Server Browser', u'Microsoft Report Viewer Redistributable 2008 (KB971119)', u'Microsoft SQL Server 2008 R2 Policies', u'Microsoft SQL Server Compact 3.5 SP2 Query Tools ENU'], 'guestOs': u'Unknown', 'nice': '', 'status': 'Powering down', 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'displayIp': '0', 'lastLogout': 1366231390.568368, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '3652045', 'apparentsize': '64424509440', 'writeLatency': '312295', 'imageID': 'c4460480-32d3-469f-b0f4-c03bb6d74e41', 'flushLatency': '183455', 'readRate': '145382.38', 'truesize': '64424509440', 'writeRate': '189200.51'}, u'vdc': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '107374182400', 'writeRate': '0.00'}, u'vdb': {'readLatency': '0', 'apparentsize': '150323855360', 'writeLatency': '0', 'imageID': 'df8b790c-c762-4feb-9378-d64a321ce43c', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '150323855360', 'writeRate': '0.00'}, u'vdd': {'readLatency': '0', 'apparentsize': '429496729600', 'writeLatency': '0', 'imageID': 'd368307e-425e-423a-85b7-266c1e4a99ec', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '429496729600', 'writeRate': '0.00'}}, 'monitorResponse': '0', 'statsAge': '0.49', 'cpuUser': '175.49', 'cpuIdle': '0.00', 'lastLogin': 1384895046.5037899, 'guestIPs': u'192.168.100.70', 'displaySecurePort': u'5909'}]}
>Dummy-15909965::DEBUG::2013-11-19 21:05:20,141::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:05:20,787::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0945618 s, 10.8 MB/s\n'; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-19 21:05:21,211::libvirtvm::1401::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::event Saved detail 0 opaque None
>Thread-15910222::INFO::2013-11-19 21:05:21,715::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910222::DEBUG::2013-11-19 21:05:21,716::task::495::TaskManager.Task::(_debug) Task 53f89f69-0c8c-4125-b49a-44c1f892b333: moving from state init -> state preparing
>Thread-15910222::DEBUG::2013-11-19 21:05:21,717::task::495::TaskManager.Task::(_debug) Task 53f89f69-0c8c-4125-b49a-44c1f892b333: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000465869903564', 'lastCheck': 1384895121.08833, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00144600868225', 'lastCheck': 1384895117.086225, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895112.040729, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000355958938599', 'lastCheck': 1384895120.082762, 'valid': True, 'code': 0}}
>Thread-15910222::DEBUG::2013-11-19 21:05:21,743::task::495::TaskManager.Task::(_debug) Task 53f89f69-0c8c-4125-b49a-44c1f892b333: moving from state preparing -> state finished
>Thread-15910222::DEBUG::2013-11-19 21:05:21,744::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910222::DEBUG::2013-11-19 21:05:21,744::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910222::DEBUG::2013-11-19 21:05:21,745::task::495::TaskManager.Task::(_debug) Task 53f89f69-0c8c-4125-b49a-44c1f892b333: ref 0 aborting False
>Thread-15910222::INFO::2013-11-19 21:05:21,746::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000465869903564', 'lastCheck': 1384895121.08833, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00144600868225', 'lastCheck': 1384895117.086225, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895112.040729, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000355958938599', 'lastCheck': 1384895120.082762, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:05:22,085::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15909965::DEBUG::2013-11-19 21:05:22,803::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:05:23,346::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0828931 s, 12.4 MB/s\n'; <rc> = 0
>Thread-15910225::DEBUG::2013-11-19 21:05:24,017::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('6574d88c-00ac-46f4-99ce-a2726932bfda',) {}
>Thread-15910225::DEBUG::2013-11-19 21:05:24,021::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': u'None', 'memUsage': '91', 'acpiEnable': 'true', 'pid': '57291', 'displayPort': u'5908', 'session': 'LoggedOff', 'netIfaces': [{u'inet6': [], u'hw': u'00-1a-4a-a8-64-08', u'inet': [u'192.168.100.70'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': -1L, 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '38686343168', u'fs': u'NTFS'}, {u'path': u'h:\\', u'total': '107237863424', u'used': '99742478336', u'fs': u'NTFS'}, {u'path': u'l:\\', u'total': '150187474944', u'used': '130882863104', u'fs': u'NTFS'}, {u'path': u'x:\\', u'total': '429360349184', u'used': '330617978880', u'fs': u'NTFS'}], 'network': {u'vnet5': {'macAddr': u'00:1a:4a:a8:64:08', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet5'}}, 'displayType': 'qxl', 'guestName': u'KLEE-AXDB01', 'boot': 'cd', 'elapsedTime': '18663733', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Microsoft Report Viewer Redistributable 2008 SP1', u'Zabbix Agent 2.0.8', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'Microsoft SQL Server Compact 3.5 SP2 ENU', u'RHEV-Spice-Agent64', u'Microsoft Visual Studio Tools for Applications 2.0 - ENU', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'Microsoft Office 2003 Web Components', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10', u'Microsoft SQL Server Browser', u'Microsoft Report Viewer Redistributable 2008 (KB971119)', u'Microsoft SQL Server 2008 R2 Policies', u'Microsoft SQL Server Compact 3.5 SP2 Query Tools ENU'], 'guestOs': u'Unknown', 'nice': '', 'status': 'Powering down', 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'displayIp': '0', 'lastLogout': 1366231390.568368, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '3652045', 'apparentsize': '64424509440', 'writeLatency': '312295', 'imageID': 'c4460480-32d3-469f-b0f4-c03bb6d74e41', 'flushLatency': '183455', 'readRate': '145382.38', 'truesize': '64424509440', 'writeRate': '189200.51'}, u'vdc': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '107374182400', 'writeRate': '0.00'}, u'vdb': {'readLatency': '0', 'apparentsize': '150323855360', 'writeLatency': '0', 'imageID': 'df8b790c-c762-4feb-9378-d64a321ce43c', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '150323855360', 'writeRate': '0.00'}, u'vdd': {'readLatency': '0', 'apparentsize': '429496729600', 'writeLatency': '0', 'imageID': 'd368307e-425e-423a-85b7-266c1e4a99ec', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '429496729600', 'writeRate': '0.00'}}, 'monitorResponse': '0', 'statsAge': '0.93', 'cpuUser': '175.49', 'cpuIdle': '0.00', 'lastLogin': 1384895046.5037899, 'guestIPs': u'192.168.100.70', 'displaySecurePort': u'5909'}]}
>Thread-4922911::INFO::2013-11-19 21:05:25,089::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-4922911::DEBUG::2013-11-19 21:05:25,095::task::495::TaskManager.Task::(_debug) Task 93c1a13e-4053-490f-a818-be632d9d5483: moving from state init -> state preparing
>Thread-4922911::DEBUG::2013-11-19 21:05:25,101::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`407955dd-5dd9-40f6-922e-2ec4897c5e11`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-4922911::DEBUG::2013-11-19 21:05:25,101::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-4922911::DEBUG::2013-11-19 21:05:25,102::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-4922911::DEBUG::2013-11-19 21:05:25,102::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`407955dd-5dd9-40f6-922e-2ec4897c5e11`::Granted request
>Thread-4922911::DEBUG::2013-11-19 21:05:25,108::task::495::TaskManager.Task::(_debug) Task 93c1a13e-4053-490f-a818-be632d9d5483: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-4922911::DEBUG::2013-11-19 21:05:25,112::task::495::TaskManager.Task::(_debug) Task 93c1a13e-4053-490f-a818-be632d9d5483: ref 1 aborting False
>Thread-4922911::DEBUG::2013-11-19 21:05:25,118::task::495::TaskManager.Task::(_debug) Task 93c1a13e-4053-490f-a818-be632d9d5483: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-4922911::DEBUG::2013-11-19 21:05:25,118::task::495::TaskManager.Task::(_debug) Task 93c1a13e-4053-490f-a818-be632d9d5483: moving from state preparing -> state finished
>Thread-4922911::DEBUG::2013-11-19 21:05:25,118::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-4922911::DEBUG::2013-11-19 21:05:25,119::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-4922911::DEBUG::2013-11-19 21:05:25,121::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-4922911::DEBUG::2013-11-19 21:05:25,123::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-4922911::DEBUG::2013-11-19 21:05:25,124::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-4922911::DEBUG::2013-11-19 21:05:25,129::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-4922911::DEBUG::2013-11-19 21:05:25,129::task::495::TaskManager.Task::(_debug) Task 93c1a13e-4053-490f-a818-be632d9d5483: ref 0 aborting False
>Thread-4922911::INFO::2013-11-19 21:05:25,131::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-4922911::INFO::2013-11-19 21:05:25,132::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-4922911::DEBUG::2013-11-19 21:05:25,137::task::495::TaskManager.Task::(_debug) Task 73ea90f1-dbd6-4b94-95fe-f93a97b80759: moving from state init -> state preparing
>Thread-4922911::DEBUG::2013-11-19 21:05:25,138::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a3b2c6f6-6fa2-4ee9-9505-252912cb6da1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-4922911::DEBUG::2013-11-19 21:05:25,143::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-4922911::DEBUG::2013-11-19 21:05:25,143::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-4922911::DEBUG::2013-11-19 21:05:25,146::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a3b2c6f6-6fa2-4ee9-9505-252912cb6da1`::Granted request
>Thread-4922911::DEBUG::2013-11-19 21:05:25,152::task::495::TaskManager.Task::(_debug) Task 73ea90f1-dbd6-4b94-95fe-f93a97b80759: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-4922911::DEBUG::2013-11-19 21:05:25,155::task::495::TaskManager.Task::(_debug) Task 73ea90f1-dbd6-4b94-95fe-f93a97b80759: ref 1 aborting False
>Thread-4922911::DEBUG::2013-11-19 21:05:25,155::task::495::TaskManager.Task::(_debug) Task 73ea90f1-dbd6-4b94-95fe-f93a97b80759: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-4922911::DEBUG::2013-11-19 21:05:25,158::task::495::TaskManager.Task::(_debug) Task 73ea90f1-dbd6-4b94-95fe-f93a97b80759: moving from state preparing -> state finished
>Thread-4922911::DEBUG::2013-11-19 21:05:25,163::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-4922911::DEBUG::2013-11-19 21:05:25,163::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-4922911::DEBUG::2013-11-19 21:05:25,167::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-4922911::DEBUG::2013-11-19 21:05:25,173::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-4922911::DEBUG::2013-11-19 21:05:25,176::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-4922911::DEBUG::2013-11-19 21:05:25,176::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-4922911::DEBUG::2013-11-19 21:05:25,181::task::495::TaskManager.Task::(_debug) Task 73ea90f1-dbd6-4b94-95fe-f93a97b80759: ref 0 aborting False
>Thread-4922911::INFO::2013-11-19 21:05:25,181::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-4922911::INFO::2013-11-19 21:05:25,182::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-4922911::DEBUG::2013-11-19 21:05:25,187::task::495::TaskManager.Task::(_debug) Task 31d6c011-6cbf-443e-970a-811caf9b7bb7: moving from state init -> state preparing
>Thread-4922911::DEBUG::2013-11-19 21:05:25,189::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a8eabbd5-ed9a-4e68-af4c-5f60dd031919`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-4922911::DEBUG::2013-11-19 21:05:25,195::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-4922911::DEBUG::2013-11-19 21:05:25,199::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-4922911::DEBUG::2013-11-19 21:05:25,200::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a8eabbd5-ed9a-4e68-af4c-5f60dd031919`::Granted request
>Thread-4922911::DEBUG::2013-11-19 21:05:25,207::task::495::TaskManager.Task::(_debug) Task 31d6c011-6cbf-443e-970a-811caf9b7bb7: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-4922911::DEBUG::2013-11-19 21:05:25,210::task::495::TaskManager.Task::(_debug) Task 31d6c011-6cbf-443e-970a-811caf9b7bb7: ref 1 aborting False
>Thread-4922911::DEBUG::2013-11-19 21:05:25,210::task::495::TaskManager.Task::(_debug) Task 31d6c011-6cbf-443e-970a-811caf9b7bb7: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-4922911::DEBUG::2013-11-19 21:05:25,211::task::495::TaskManager.Task::(_debug) Task 31d6c011-6cbf-443e-970a-811caf9b7bb7: moving from state preparing -> state finished
>Thread-4922911::DEBUG::2013-11-19 21:05:25,211::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-4922911::DEBUG::2013-11-19 21:05:25,211::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-4922911::DEBUG::2013-11-19 21:05:25,212::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-4922911::DEBUG::2013-11-19 21:05:25,217::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-4922911::DEBUG::2013-11-19 21:05:25,221::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-4922911::DEBUG::2013-11-19 21:05:25,221::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-4922911::DEBUG::2013-11-19 21:05:25,222::task::495::TaskManager.Task::(_debug) Task 31d6c011-6cbf-443e-970a-811caf9b7bb7: ref 0 aborting False
>Thread-4922911::INFO::2013-11-19 21:05:25,222::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-4922911::INFO::2013-11-19 21:05:25,222::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-4922911::DEBUG::2013-11-19 21:05:25,223::task::495::TaskManager.Task::(_debug) Task e8c8710c-a140-4ed4-89a0-14718fbbeafb: moving from state init -> state preparing
>Thread-4922911::DEBUG::2013-11-19 21:05:25,225::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b42d073e-4255-4883-822e-9502c4345fbb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-4922911::DEBUG::2013-11-19 21:05:25,231::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-4922911::DEBUG::2013-11-19 21:05:25,236::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-4922911::DEBUG::2013-11-19 21:05:25,242::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b42d073e-4255-4883-822e-9502c4345fbb`::Granted request
>Thread-4922911::DEBUG::2013-11-19 21:05:25,242::task::495::TaskManager.Task::(_debug) Task e8c8710c-a140-4ed4-89a0-14718fbbeafb: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-4922911::DEBUG::2013-11-19 21:05:25,243::task::495::TaskManager.Task::(_debug) Task e8c8710c-a140-4ed4-89a0-14718fbbeafb: ref 1 aborting False
>Thread-4922911::DEBUG::2013-11-19 21:05:25,247::task::495::TaskManager.Task::(_debug) Task e8c8710c-a140-4ed4-89a0-14718fbbeafb: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-4922911::DEBUG::2013-11-19 21:05:25,248::task::495::TaskManager.Task::(_debug) Task e8c8710c-a140-4ed4-89a0-14718fbbeafb: moving from state preparing -> state finished
>Thread-4922911::DEBUG::2013-11-19 21:05:25,248::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-4922911::DEBUG::2013-11-19 21:05:25,249::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-4922911::DEBUG::2013-11-19 21:05:25,250::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-4922911::DEBUG::2013-11-19 21:05:25,253::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-4922911::DEBUG::2013-11-19 21:05:25,258::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-4922911::DEBUG::2013-11-19 21:05:25,258::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-4922911::DEBUG::2013-11-19 21:05:25,259::task::495::TaskManager.Task::(_debug) Task e8c8710c-a140-4ed4-89a0-14718fbbeafb: ref 0 aborting False
>Thread-4922911::INFO::2013-11-19 21:05:25,260::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15909965::DEBUG::2013-11-19 21:05:25,360::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:05:25,876::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0743445 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15910227::DEBUG::2013-11-19 21:05:26,106::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('6574d88c-00ac-46f4-99ce-a2726932bfda',) {}
>Thread-15910227::DEBUG::2013-11-19 21:05:26,107::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': u'None', 'memUsage': '91', 'acpiEnable': 'true', 'pid': '57291', 'displayPort': u'5908', 'session': 'LoggedOff', 'netIfaces': [{u'inet6': [], u'hw': u'00-1a-4a-a8-64-08', u'inet': [u'192.168.100.70'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': -1L, 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '38686343168', u'fs': u'NTFS'}, {u'path': u'h:\\', u'total': '107237863424', u'used': '99742478336', u'fs': u'NTFS'}, {u'path': u'l:\\', u'total': '150187474944', u'used': '130882863104', u'fs': u'NTFS'}, {u'path': u'x:\\', u'total': '429360349184', u'used': '330617978880', u'fs': u'NTFS'}], 'network': {u'vnet5': {'macAddr': u'00:1a:4a:a8:64:08', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet5'}}, 'displayType': 'qxl', 'guestName': u'KLEE-AXDB01', 'boot': 'cd', 'elapsedTime': '18663735', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Microsoft Report Viewer Redistributable 2008 SP1', u'Zabbix Agent 2.0.8', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'Microsoft SQL Server Compact 3.5 SP2 ENU', u'RHEV-Spice-Agent64', u'Microsoft Visual Studio Tools for Applications 2.0 - ENU', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'Microsoft Office 2003 Web Components', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10', u'Microsoft SQL Server Browser', u'Microsoft Report Viewer Redistributable 2008 (KB971119)', u'Microsoft SQL Server 2008 R2 Policies', u'Microsoft SQL Server Compact 3.5 SP2 Query Tools ENU'], 'guestOs': u'Unknown', 'nice': '', 'status': 'Powering down', 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'displayIp': '0', 'lastLogout': 1366231390.568368, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '3652045', 'apparentsize': '64424509440', 'writeLatency': '312295', 'imageID': 'c4460480-32d3-469f-b0f4-c03bb6d74e41', 'flushLatency': '183455', 'readRate': '145382.38', 'truesize': '64424509440', 'writeRate': '189200.51'}, u'vdc': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '107374182400', 'writeRate': '0.00'}, u'vdb': {'readLatency': '0', 'apparentsize': '150323855360', 'writeLatency': '0', 'imageID': 'df8b790c-c762-4feb-9378-d64a321ce43c', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '150323855360', 'writeRate': '0.00'}, u'vdd': {'readLatency': '0', 'apparentsize': '429496729600', 'writeLatency': '0', 'imageID': 'd368307e-425e-423a-85b7-266c1e4a99ec', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '429496729600', 'writeRate': '0.00'}}, 'monitorResponse': '0', 'statsAge': '1.02', 'cpuUser': '144.52', 'cpuIdle': '0.00', 'lastLogin': 1384895046.5037899, 'guestIPs': u'192.168.100.70', 'displaySecurePort': u'5909'}]}
>Thread-15910228::DEBUG::2013-11-19 21:05:27,160::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910228::INFO::2013-11-19 21:05:27,162::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910228::DEBUG::2013-11-19 21:05:27,163::task::495::TaskManager.Task::(_debug) Task 491c6fda-03b3-427a-9589-1dd587c8b2b7: moving from state init -> state preparing
>Thread-15910228::DEBUG::2013-11-19 21:05:27,165::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=274 spmId=2
>Thread-15910228::DEBUG::2013-11-19 21:05:27,165::task::495::TaskManager.Task::(_debug) Task 491c6fda-03b3-427a-9589-1dd587c8b2b7: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910228::DEBUG::2013-11-19 21:05:27,166::task::495::TaskManager.Task::(_debug) Task 491c6fda-03b3-427a-9589-1dd587c8b2b7: moving from state preparing -> state finished
>Thread-15910228::DEBUG::2013-11-19 21:05:27,167::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910228::DEBUG::2013-11-19 21:05:27,167::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910228::DEBUG::2013-11-19 21:05:27,168::task::495::TaskManager.Task::(_debug) Task 491c6fda-03b3-427a-9589-1dd587c8b2b7: ref 0 aborting False
>Thread-15910228::INFO::2013-11-19 21:05:27,168::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910229::DEBUG::2013-11-19 21:05:27,175::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910229::INFO::2013-11-19 21:05:27,175::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910229::DEBUG::2013-11-19 21:05:27,175::task::495::TaskManager.Task::(_debug) Task 77e0d661-181c-4680-b626-825fa748d29d: moving from state init -> state preparing
>Thread-15910229::DEBUG::2013-11-19 21:05:27,181::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4aa04163-746a-4e7d-9308-53c63073de63`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910229::DEBUG::2013-11-19 21:05:27,181::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910229::DEBUG::2013-11-19 21:05:27,182::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910229::DEBUG::2013-11-19 21:05:27,182::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4aa04163-746a-4e7d-9308-53c63073de63`::Granted request
>Thread-15910229::DEBUG::2013-11-19 21:05:27,183::task::495::TaskManager.Task::(_debug) Task 77e0d661-181c-4680-b626-825fa748d29d: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910229::DEBUG::2013-11-19 21:05:27,183::task::495::TaskManager.Task::(_debug) Task 77e0d661-181c-4680-b626-825fa748d29d: ref 1 aborting False
>Thread-15910229::WARNING::2013-11-19 21:05:27,190::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910229::DEBUG::2013-11-19 21:05:27,198::task::495::TaskManager.Task::(_debug) Task 77e0d661-181c-4680-b626-825fa748d29d: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910229::DEBUG::2013-11-19 21:05:27,211::task::495::TaskManager.Task::(_debug) Task 77e0d661-181c-4680-b626-825fa748d29d: moving from state preparing -> state finished
>Thread-15910229::DEBUG::2013-11-19 21:05:27,212::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910229::DEBUG::2013-11-19 21:05:27,212::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910229::DEBUG::2013-11-19 21:05:27,217::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910229::DEBUG::2013-11-19 21:05:27,218::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910229::DEBUG::2013-11-19 21:05:27,218::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910229::DEBUG::2013-11-19 21:05:27,222::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910229::DEBUG::2013-11-19 21:05:27,223::task::495::TaskManager.Task::(_debug) Task 77e0d661-181c-4680-b626-825fa748d29d: ref 0 aborting False
>Thread-15910229::INFO::2013-11-19 21:05:27,223::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15909965::DEBUG::2013-11-19 21:05:27,889::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15910231::DEBUG::2013-11-19 21:05:28,258::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('6574d88c-00ac-46f4-99ce-a2726932bfda',) {}
>Thread-15910231::DEBUG::2013-11-19 21:05:28,263::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': u'None', 'memUsage': '91', 'acpiEnable': 'true', 'pid': '57291', 'displayPort': u'5908', 'session': 'LoggedOff', 'netIfaces': [{u'inet6': [], u'hw': u'00-1a-4a-a8-64-08', u'inet': [u'192.168.100.70'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': -1L, 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '38686343168', u'fs': u'NTFS'}, {u'path': u'h:\\', u'total': '107237863424', u'used': '99742478336', u'fs': u'NTFS'}, {u'path': u'l:\\', u'total': '150187474944', u'used': '130882863104', u'fs': u'NTFS'}, {u'path': u'x:\\', u'total': '429360349184', u'used': '330617978880', u'fs': u'NTFS'}], 'network': {u'vnet5': {'macAddr': u'00:1a:4a:a8:64:08', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet5'}}, 'displayType': 'qxl', 'guestName': u'KLEE-AXDB01', 'boot': 'cd', 'elapsedTime': '18663737', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Microsoft Report Viewer Redistributable 2008 SP1', u'Zabbix Agent 2.0.8', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'Microsoft SQL Server Compact 3.5 SP2 ENU', u'RHEV-Spice-Agent64', u'Microsoft Visual Studio Tools for Applications 2.0 - ENU', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'Microsoft Office 2003 Web Components', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10', u'Microsoft SQL Server Browser', u'Microsoft Report Viewer Redistributable 2008 (KB971119)', u'Microsoft SQL Server 2008 R2 Policies', u'Microsoft SQL Server Compact 3.5 SP2 Query Tools ENU'], 'guestOs': u'Unknown', 'nice': '', 'status': 'Powering down', 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'displayIp': '0', 'lastLogout': 1366231390.568368, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '3652045', 'apparentsize': '64424509440', 'writeLatency': '312295', 'imageID': 'c4460480-32d3-469f-b0f4-c03bb6d74e41', 'flushLatency': '183455', 'readRate': '145382.38', 'truesize': '64424509440', 'writeRate': '189200.51'}, u'vdc': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '107374182400', 'writeRate': '0.00'}, u'vdb': {'readLatency': '0', 'apparentsize': '150323855360', 'writeLatency': '0', 'imageID': 'df8b790c-c762-4feb-9378-d64a321ce43c', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '150323855360', 'writeRate': '0.00'}, u'vdd': {'readLatency': '0', 'apparentsize': '429496729600', 'writeLatency': '0', 'imageID': 'd368307e-425e-423a-85b7-266c1e4a99ec', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '429496729600', 'writeRate': '0.00'}}, 'monitorResponse': '0', 'statsAge': '3.17', 'cpuUser': '144.52', 'cpuIdle': '0.00', 'lastLogin': 1384895046.5037899, 'guestIPs': u'192.168.100.70', 'displaySecurePort': u'5909'}]}
>Dummy-15909965::DEBUG::2013-11-19 21:05:28,417::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0979412 s, 10.5 MB/s\n'; <rc> = 0
>Dummy-15909965::DEBUG::2013-11-19 21:05:30,433::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15910233::DEBUG::2013-11-19 21:05:30,724::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('6574d88c-00ac-46f4-99ce-a2726932bfda',) {}
>Thread-15910233::DEBUG::2013-11-19 21:05:30,736::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': u'None', 'memUsage': '91', 'acpiEnable': 'true', 'pid': '57291', 'displayPort': u'5908', 'session': 'LoggedOff', 'netIfaces': [{u'inet6': [], u'hw': u'00-1a-4a-a8-64-08', u'inet': [u'192.168.100.70'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': -1L, 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '38686343168', u'fs': u'NTFS'}, {u'path': u'h:\\', u'total': '107237863424', u'used': '99742478336', u'fs': u'NTFS'}, {u'path': u'l:\\', u'total': '150187474944', u'used': '130882863104', u'fs': u'NTFS'}, {u'path': u'x:\\', u'total': '429360349184', u'used': '330617978880', u'fs': u'NTFS'}], 'network': {u'vnet5': {'macAddr': u'00:1a:4a:a8:64:08', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet5'}}, 'displayType': 'qxl', 'guestName': u'KLEE-AXDB01', 'boot': 'cd', 'elapsedTime': '18663740', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Microsoft Report Viewer Redistributable 2008 SP1', u'Zabbix Agent 2.0.8', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'Microsoft SQL Server Compact 3.5 SP2 ENU', u'RHEV-Spice-Agent64', u'Microsoft Visual Studio Tools for Applications 2.0 - ENU', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'Microsoft Office 2003 Web Components', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10', u'Microsoft SQL Server Browser', u'Microsoft Report Viewer Redistributable 2008 (KB971119)', u'Microsoft SQL Server 2008 R2 Policies', u'Microsoft SQL Server Compact 3.5 SP2 Query Tools ENU'], 'guestOs': u'Unknown', 'nice': '', 'status': 'Powering down', 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'displayIp': '0', 'lastLogout': 1366231390.568368, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '3652045', 'apparentsize': '64424509440', 'writeLatency': '312295', 'imageID': 'c4460480-32d3-469f-b0f4-c03bb6d74e41', 'flushLatency': '183455', 'readRate': '145382.38', 'truesize': '64424509440', 'writeRate': '189200.51'}, u'vdc': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '107374182400', 'writeRate': '0.00'}, u'vdb': {'readLatency': '0', 'apparentsize': '150323855360', 'writeLatency': '0', 'imageID': 'df8b790c-c762-4feb-9378-d64a321ce43c', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '150323855360', 'writeRate': '0.00'}, u'vdd': {'readLatency': '0', 'apparentsize': '429496729600', 'writeLatency': '0', 'imageID': 'd368307e-425e-423a-85b7-266c1e4a99ec', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '429496729600', 'writeRate': '0.00'}}, 'monitorResponse': '0', 'statsAge': '5.65', 'cpuUser': '144.52', 'cpuIdle': '0.00', 'lastLogin': 1384895046.5037899, 'guestIPs': u'192.168.100.70', 'displaySecurePort': u'5909'}]}
>Dummy-15909965::DEBUG::2013-11-19 21:05:30,982::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0741267 s, 13.8 MB/s\n'; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-19 21:05:31,176::libvirtvm::1401::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::event Stopped detail 0 opaque None
>Thread-4922911::ERROR::2013-11-19 21:05:31,177::utils::377::vm.Vm::(collect) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Stats function failed: <AdvancedStatsFunction _sampleDisk at 0x1b96148>
>Traceback (most recent call last):
>  File "/usr/share/vdsm/utils.py", line 373, in collect
>  File "/usr/share/vdsm/utils.py", line 250, in __call__
>  File "/usr/share/vdsm/libvirtvm.py", line 101, in _sampleDisk
>  File "/usr/share/vdsm/libvirtvm.py", line 445, in f
>  File "/usr/share/vdsm/libvirtconnection.py", line 63, in wrapper
>  File "/usr/lib64/python2.6/site-packages/libvirt.py", line 1687, in blockStats
>libvirtError: Unable to read from monitor: Connection reset by peer
>libvirtEventLoop::INFO::2013-11-19 21:05:31,178::vm::445::vm.Vm::(_onQemuDeath) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::underlying process disconnected
>libvirtEventLoop::INFO::2013-11-19 21:05:31,187::libvirtvm::1304::vm.Vm::(releaseVm) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Release VM resources
>libvirtEventLoop::DEBUG::2013-11-19 21:05:31,205::utils::314::vm.Vm::(stop) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Stop statistics collection
>Thread-4922911::DEBUG::2013-11-19 21:05:31,207::utils::345::vm.Vm::(run) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Stats thread finished
>libvirtEventLoop::WARNING::2013-11-19 21:05:31,210::libvirtvm::1315::vm.Vm::(releaseVm) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Traceback (most recent call last):
>  File "/usr/share/vdsm/libvirtvm.py", line 1312, in releaseVm
>  File "/usr/share/vdsm/libvirtvm.py", line 445, in f
>  File "/usr/share/vdsm/libvirtconnection.py", line 63, in wrapper
>  File "/usr/lib64/python2.6/site-packages/libvirt.py", line 672, in destroy
>libvirtError: Domain not found: no domain with matching uuid '6574d88c-00ac-46f4-99ce-a2726932bfda'
>
>libvirtEventLoop::DEBUG::2013-11-19 21:05:31,212::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /sbin/service ksmtuned retune' (cwd None)
>libvirtEventLoop::DEBUG::2013-11-19 21:05:31,661::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-19 21:05:31,663::vm::718::vm.Vm::(_cleanup) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Drive /rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/772472cb-2429-466d-96cd-00b3301ef78a/images/d368307e-425e-423a-85b7-266c1e4a99ec/fdc91117-476f-4b82-8fc4-d682fa283ecc cleanup
>libvirtEventLoop::INFO::2013-11-19 21:05:31,665::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: teardownVolume, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>libvirtEventLoop::DEBUG::2013-11-19 21:05:31,666::task::495::TaskManager.Task::(_debug) Task 03a96d28-dbe3-4697-aeab-759d31bc7387: moving from state init -> state preparing
>libvirtEventLoop::DEBUG::2013-11-19 21:05:31,667::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b56d8e44-8e7a-413c-9f3a-00a32b453478`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1727' at 'public_teardownVolume'
>libvirtEventLoop::DEBUG::2013-11-19 21:05:31,667::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>libvirtEventLoop::DEBUG::2013-11-19 21:05:31,668::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>libvirtEventLoop::DEBUG::2013-11-19 21:05:31,669::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b56d8e44-8e7a-413c-9f3a-00a32b453478`::Granted request
>libvirtEventLoop::DEBUG::2013-11-19 21:05:31,670::task::495::TaskManager.Task::(_debug) Task 03a96d28-dbe3-4697-aeab-759d31bc7387: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>libvirtEventLoop::DEBUG::2013-11-19 21:05:31,670::task::495::TaskManager.Task::(_debug) Task 03a96d28-dbe3-4697-aeab-759d31bc7387: ref 1 aborting False
>libvirtEventLoop::DEBUG::2013-11-19 21:05:31,671::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource '772472cb-2429-466d-96cd-00b3301ef78a_imageNS.d368307e-425e-423a-85b7-266c1e4a99ec'
>libvirtEventLoop::DEBUG::2013-11-19 21:05:31,672::resourceManager::530::ResourceManager::(releaseResource) Released resource '772472cb-2429-466d-96cd-00b3301ef78a_imageNS.d368307e-425e-423a-85b7-266c1e4a99ec' (0 active users)
>libvirtEventLoop::DEBUG::2013-11-19 21:05:31,672::resourceManager::535::ResourceManager::(releaseResource) Resource '772472cb-2429-466d-96cd-00b3301ef78a_imageNS.d368307e-425e-423a-85b7-266c1e4a99ec' is free, finding out if anyone is waiting for it.
>libvirtEventLoop::DEBUG::2013-11-19 21:05:31,673::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource '772472cb-2429-466d-96cd-00b3301ef78a_volumeNS.fdc91117-476f-4b82-8fc4-d682fa283ecc'
>libvirtEventLoop::DEBUG::2013-11-19 21:05:31,673::resourceManager::530::ResourceManager::(releaseResource) Released resource '772472cb-2429-466d-96cd-00b3301ef78a_volumeNS.fdc91117-476f-4b82-8fc4-d682fa283ecc' (0 active users)
>libvirtEventLoop::DEBUG::2013-11-19 21:05:31,674::resourceManager::535::ResourceManager::(releaseResource) Resource '772472cb-2429-466d-96cd-00b3301ef78a_volumeNS.fdc91117-476f-4b82-8fc4-d682fa283ecc' is free, finding out if anyone is waiting for it.
>libvirtEventLoop::DEBUG::2013-11-19 21:05:31,674::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource '772472cb-2429-466d-96cd-00b3301ef78a_volumeNS.fdc91117-476f-4b82-8fc4-d682fa283ecc', Clearing records.
>libvirtEventLoop::DEBUG::2013-11-19 21:05:31,675::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource '772472cb-2429-466d-96cd-00b3301ef78a_imageNS.d368307e-425e-423a-85b7-266c1e4a99ec', Clearing records.
>libvirtEventLoop::INFO::2013-11-19 21:05:31,676::blockVolume::386::Storage.Volume::(teardown) Tearing down volume 772472cb-2429-466d-96cd-00b3301ef78a/fdc91117-476f-4b82-8fc4-d682fa283ecc justme False
>libvirtEventLoop::DEBUG::2013-11-19 21:05:31,676::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource '772472cb-2429-466d-96cd-00b3301ef78a_lvmActivationNS.fdc91117-476f-4b82-8fc4-d682fa283ecc'
>libvirtEventLoop::DEBUG::2013-11-19 21:05:31,677::resourceManager::530::ResourceManager::(releaseResource) Released resource '772472cb-2429-466d-96cd-00b3301ef78a_lvmActivationNS.fdc91117-476f-4b82-8fc4-d682fa283ecc' (0 active users)
>libvirtEventLoop::DEBUG::2013-11-19 21:05:31,677::resourceManager::535::ResourceManager::(releaseResource) Resource '772472cb-2429-466d-96cd-00b3301ef78a_lvmActivationNS.fdc91117-476f-4b82-8fc4-d682fa283ecc' is free, finding out if anyone is waiting for it.
>libvirtEventLoop::DEBUG::2013-11-19 21:05:31,678::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --autobackup n --available n 772472cb-2429-466d-96cd-00b3301ef78a/fdc91117-476f-4b82-8fc4-d682fa283ecc' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:05:32,100::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>libvirtEventLoop::DEBUG::2013-11-19 21:05:32,445::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-19 21:05:32,446::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>libvirtEventLoop::DEBUG::2013-11-19 21:05:32,447::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>libvirtEventLoop::DEBUG::2013-11-19 21:05:32,447::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource '772472cb-2429-466d-96cd-00b3301ef78a_lvmActivationNS.fdc91117-476f-4b82-8fc4-d682fa283ecc', Clearing records.
>libvirtEventLoop::DEBUG::2013-11-19 21:05:32,448::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>libvirtEventLoop::DEBUG::2013-11-19 21:05:32,449::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-15910234::INFO::2013-11-19 21:05:32,855::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910234::DEBUG::2013-11-19 21:05:32,856::task::495::TaskManager.Task::(_debug) Task 5e9b5241-84a7-4099-920d-aa05f95cd98a: moving from state init -> state preparing
>Thread-15910234::DEBUG::2013-11-19 21:05:32,857::task::495::TaskManager.Task::(_debug) Task 5e9b5241-84a7-4099-920d-aa05f95cd98a: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000362873077393', 'lastCheck': 1384895131.101028, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00131106376648', 'lastCheck': 1384895127.1006629, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895132.101685, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000398874282837', 'lastCheck': 1384895130.094244, 'valid': True, 'code': 0}}
>Thread-15910234::DEBUG::2013-11-19 21:05:32,858::task::495::TaskManager.Task::(_debug) Task 5e9b5241-84a7-4099-920d-aa05f95cd98a: moving from state preparing -> state finished
>Thread-15910234::DEBUG::2013-11-19 21:05:32,858::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910234::DEBUG::2013-11-19 21:05:32,859::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910234::DEBUG::2013-11-19 21:05:32,859::task::495::TaskManager.Task::(_debug) Task 5e9b5241-84a7-4099-920d-aa05f95cd98a: ref 0 aborting False
>Thread-15910234::INFO::2013-11-19 21:05:32,860::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000362873077393', 'lastCheck': 1384895131.101028, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00131106376648', 'lastCheck': 1384895127.1006629, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895132.101685, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000398874282837', 'lastCheck': 1384895130.094244, 'valid': True, 'code': 0}}
>Dummy-15909965::DEBUG::2013-11-19 21:05:32,999::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,228::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,248::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,249::task::495::TaskManager.Task::(_debug) Task 03a96d28-dbe3-4697-aeab-759d31bc7387: finished: None
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,250::task::495::TaskManager.Task::(_debug) Task 03a96d28-dbe3-4697-aeab-759d31bc7387: moving from state preparing -> state finished
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,250::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,251::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,252::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,253::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,253::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,254::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,254::task::495::TaskManager.Task::(_debug) Task 03a96d28-dbe3-4697-aeab-759d31bc7387: ref 0 aborting False
>libvirtEventLoop::INFO::2013-11-19 21:05:33,255::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: teardownVolume, Return response: {'status': {'message': 'OK', 'code': 0}}
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,256::vm::718::vm.Vm::(_cleanup) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Drive None cleanup
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,256::vm::718::vm.Vm::(_cleanup) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Drive /rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/772472cb-2429-466d-96cd-00b3301ef78a/images/df8b790c-c762-4feb-9378-d64a321ce43c/2e903e43-1016-4339-a7f3-c5723a995725 cleanup
>libvirtEventLoop::INFO::2013-11-19 21:05:33,257::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: teardownVolume, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,258::task::495::TaskManager.Task::(_debug) Task 0ac4afa1-2ce3-431f-ad53-3cdb138cc7fa: moving from state init -> state preparing
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,259::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`98bba4a6-eff2-4264-9f84-f4462c37bf00`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1727' at 'public_teardownVolume'
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,260::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,261::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,261::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`98bba4a6-eff2-4264-9f84-f4462c37bf00`::Granted request
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,262::task::495::TaskManager.Task::(_debug) Task 0ac4afa1-2ce3-431f-ad53-3cdb138cc7fa: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,263::task::495::TaskManager.Task::(_debug) Task 0ac4afa1-2ce3-431f-ad53-3cdb138cc7fa: ref 1 aborting False
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,263::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource '772472cb-2429-466d-96cd-00b3301ef78a_imageNS.df8b790c-c762-4feb-9378-d64a321ce43c'
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,264::resourceManager::530::ResourceManager::(releaseResource) Released resource '772472cb-2429-466d-96cd-00b3301ef78a_imageNS.df8b790c-c762-4feb-9378-d64a321ce43c' (0 active users)
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,265::resourceManager::535::ResourceManager::(releaseResource) Resource '772472cb-2429-466d-96cd-00b3301ef78a_imageNS.df8b790c-c762-4feb-9378-d64a321ce43c' is free, finding out if anyone is waiting for it.
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,265::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource '772472cb-2429-466d-96cd-00b3301ef78a_volumeNS.2e903e43-1016-4339-a7f3-c5723a995725'
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,266::resourceManager::530::ResourceManager::(releaseResource) Released resource '772472cb-2429-466d-96cd-00b3301ef78a_volumeNS.2e903e43-1016-4339-a7f3-c5723a995725' (0 active users)
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,266::resourceManager::535::ResourceManager::(releaseResource) Resource '772472cb-2429-466d-96cd-00b3301ef78a_volumeNS.2e903e43-1016-4339-a7f3-c5723a995725' is free, finding out if anyone is waiting for it.
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,267::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource '772472cb-2429-466d-96cd-00b3301ef78a_volumeNS.2e903e43-1016-4339-a7f3-c5723a995725', Clearing records.
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,267::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource '772472cb-2429-466d-96cd-00b3301ef78a_imageNS.df8b790c-c762-4feb-9378-d64a321ce43c', Clearing records.
>libvirtEventLoop::INFO::2013-11-19 21:05:33,268::blockVolume::386::Storage.Volume::(teardown) Tearing down volume 772472cb-2429-466d-96cd-00b3301ef78a/2e903e43-1016-4339-a7f3-c5723a995725 justme False
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,269::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource '772472cb-2429-466d-96cd-00b3301ef78a_lvmActivationNS.2e903e43-1016-4339-a7f3-c5723a995725'
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,269::resourceManager::530::ResourceManager::(releaseResource) Released resource '772472cb-2429-466d-96cd-00b3301ef78a_lvmActivationNS.2e903e43-1016-4339-a7f3-c5723a995725' (0 active users)
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,270::resourceManager::535::ResourceManager::(releaseResource) Resource '772472cb-2429-466d-96cd-00b3301ef78a_lvmActivationNS.2e903e43-1016-4339-a7f3-c5723a995725' is free, finding out if anyone is waiting for it.
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,271::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --autobackup n --available n 772472cb-2429-466d-96cd-00b3301ef78a/2e903e43-1016-4339-a7f3-c5723a995725' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:05:33,666::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.152164 s, 6.7 MB/s\n'; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,971::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,973::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,973::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,974::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource '772472cb-2429-466d-96cd-00b3301ef78a_lvmActivationNS.2e903e43-1016-4339-a7f3-c5723a995725', Clearing records.
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,975::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>libvirtEventLoop::DEBUG::2013-11-19 21:05:33,976::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>libvirtEventLoop::DEBUG::2013-11-19 21:05:34,624::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-19 21:05:34,651::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>libvirtEventLoop::DEBUG::2013-11-19 21:05:34,652::task::495::TaskManager.Task::(_debug) Task 0ac4afa1-2ce3-431f-ad53-3cdb138cc7fa: finished: None
>libvirtEventLoop::DEBUG::2013-11-19 21:05:34,652::task::495::TaskManager.Task::(_debug) Task 0ac4afa1-2ce3-431f-ad53-3cdb138cc7fa: moving from state preparing -> state finished
>libvirtEventLoop::DEBUG::2013-11-19 21:05:34,653::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>libvirtEventLoop::DEBUG::2013-11-19 21:05:34,653::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>libvirtEventLoop::DEBUG::2013-11-19 21:05:34,654::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>libvirtEventLoop::DEBUG::2013-11-19 21:05:34,655::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>libvirtEventLoop::DEBUG::2013-11-19 21:05:34,655::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>libvirtEventLoop::DEBUG::2013-11-19 21:05:34,656::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>libvirtEventLoop::DEBUG::2013-11-19 21:05:34,657::task::495::TaskManager.Task::(_debug) Task 0ac4afa1-2ce3-431f-ad53-3cdb138cc7fa: ref 0 aborting False
>libvirtEventLoop::INFO::2013-11-19 21:05:34,657::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: teardownVolume, Return response: {'status': {'message': 'OK', 'code': 0}}
>libvirtEventLoop::DEBUG::2013-11-19 21:05:34,658::vm::718::vm.Vm::(_cleanup) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Drive /rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/772472cb-2429-466d-96cd-00b3301ef78a/images/c4460480-32d3-469f-b0f4-c03bb6d74e41/e6984064-e09e-4c0c-8b51-a8a1711994cf cleanup
>libvirtEventLoop::INFO::2013-11-19 21:05:34,659::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: teardownVolume, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>libvirtEventLoop::DEBUG::2013-11-19 21:05:34,659::task::495::TaskManager.Task::(_debug) Task 7d493234-f77b-418f-9ed2-0a24a3b9c15d: moving from state init -> state preparing
>libvirtEventLoop::DEBUG::2013-11-19 21:05:34,660::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c0f840eb-fb9a-4bf8-966b-eff58b6b3b37`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1727' at 'public_teardownVolume'
>libvirtEventLoop::DEBUG::2013-11-19 21:05:34,661::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>libvirtEventLoop::DEBUG::2013-11-19 21:05:34,661::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>libvirtEventLoop::DEBUG::2013-11-19 21:05:34,662::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c0f840eb-fb9a-4bf8-966b-eff58b6b3b37`::Granted request
>libvirtEventLoop::DEBUG::2013-11-19 21:05:34,663::task::495::TaskManager.Task::(_debug) Task 7d493234-f77b-418f-9ed2-0a24a3b9c15d: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>libvirtEventLoop::DEBUG::2013-11-19 21:05:34,663::task::495::TaskManager.Task::(_debug) Task 7d493234-f77b-418f-9ed2-0a24a3b9c15d: ref 1 aborting False
>libvirtEventLoop::DEBUG::2013-11-19 21:05:34,664::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource '772472cb-2429-466d-96cd-00b3301ef78a_imageNS.c4460480-32d3-469f-b0f4-c03bb6d74e41'
>libvirtEventLoop::DEBUG::2013-11-19 21:05:34,665::resourceManager::530::ResourceManager::(releaseResource) Released resource '772472cb-2429-466d-96cd-00b3301ef78a_imageNS.c4460480-32d3-469f-b0f4-c03bb6d74e41' (0 active users)
>libvirtEventLoop::DEBUG::2013-11-19 21:05:34,665::resourceManager::535::ResourceManager::(releaseResource) Resource '772472cb-2429-466d-96cd-00b3301ef78a_imageNS.c4460480-32d3-469f-b0f4-c03bb6d74e41' is free, finding out if anyone is waiting for it.
>libvirtEventLoop::DEBUG::2013-11-19 21:05:34,666::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource '772472cb-2429-466d-96cd-00b3301ef78a_volumeNS.e6984064-e09e-4c0c-8b51-a8a1711994cf'
>libvirtEventLoop::DEBUG::2013-11-19 21:05:34,666::resourceManager::530::ResourceManager::(releaseResource) Released resource '772472cb-2429-466d-96cd-00b3301ef78a_volumeNS.e6984064-e09e-4c0c-8b51-a8a1711994cf' (0 active users)
>libvirtEventLoop::DEBUG::2013-11-19 21:05:34,667::resourceManager::535::ResourceManager::(releaseResource) Resource '772472cb-2429-466d-96cd-00b3301ef78a_volumeNS.e6984064-e09e-4c0c-8b51-a8a1711994cf' is free, finding out if anyone is waiting for it.
>libvirtEventLoop::DEBUG::2013-11-19 21:05:34,667::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource '772472cb-2429-466d-96cd-00b3301ef78a_volumeNS.e6984064-e09e-4c0c-8b51-a8a1711994cf', Clearing records.
>libvirtEventLoop::DEBUG::2013-11-19 21:05:34,668::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource '772472cb-2429-466d-96cd-00b3301ef78a_imageNS.c4460480-32d3-469f-b0f4-c03bb6d74e41', Clearing records.
>libvirtEventLoop::INFO::2013-11-19 21:05:34,668::blockVolume::386::Storage.Volume::(teardown) Tearing down volume 772472cb-2429-466d-96cd-00b3301ef78a/e6984064-e09e-4c0c-8b51-a8a1711994cf justme False
>libvirtEventLoop::DEBUG::2013-11-19 21:05:34,669::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource '772472cb-2429-466d-96cd-00b3301ef78a_lvmActivationNS.e6984064-e09e-4c0c-8b51-a8a1711994cf'
>libvirtEventLoop::DEBUG::2013-11-19 21:05:34,669::resourceManager::530::ResourceManager::(releaseResource) Released resource '772472cb-2429-466d-96cd-00b3301ef78a_lvmActivationNS.e6984064-e09e-4c0c-8b51-a8a1711994cf' (0 active users)
>libvirtEventLoop::DEBUG::2013-11-19 21:05:34,670::resourceManager::535::ResourceManager::(releaseResource) Resource '772472cb-2429-466d-96cd-00b3301ef78a_lvmActivationNS.e6984064-e09e-4c0c-8b51-a8a1711994cf' is free, finding out if anyone is waiting for it.
>libvirtEventLoop::DEBUG::2013-11-19 21:05:34,671::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --autobackup n --available n 772472cb-2429-466d-96cd-00b3301ef78a/e6984064-e09e-4c0c-8b51-a8a1711994cf' (cwd None)
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,283::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,285::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,285::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,286::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource '772472cb-2429-466d-96cd-00b3301ef78a_lvmActivationNS.e6984064-e09e-4c0c-8b51-a8a1711994cf', Clearing records.
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,287::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,288::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:05:35,723::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,943::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,963::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,964::task::495::TaskManager.Task::(_debug) Task 7d493234-f77b-418f-9ed2-0a24a3b9c15d: finished: None
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,964::task::495::TaskManager.Task::(_debug) Task 7d493234-f77b-418f-9ed2-0a24a3b9c15d: moving from state preparing -> state finished
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,965::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,966::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,967::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,967::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,968::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,968::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,969::task::495::TaskManager.Task::(_debug) Task 7d493234-f77b-418f-9ed2-0a24a3b9c15d: ref 0 aborting False
>libvirtEventLoop::INFO::2013-11-19 21:05:35,970::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: teardownVolume, Return response: {'status': {'message': 'OK', 'code': 0}}
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,970::vm::718::vm.Vm::(_cleanup) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Drive /rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/772472cb-2429-466d-96cd-00b3301ef78a/images/13efb45b-f4d2-4c94-bbdb-91ce9b9b7321/088640f7-316c-4e2b-bfdc-0f64cf2913c8 cleanup
>libvirtEventLoop::INFO::2013-11-19 21:05:35,972::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: teardownVolume, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,972::task::495::TaskManager.Task::(_debug) Task f9dcf002-cb98-43cd-a054-32a0ca4f488b: moving from state init -> state preparing
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,973::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`20abb79d-7689-4795-9e64-7343d3683033`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1727' at 'public_teardownVolume'
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,974::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,974::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,975::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`20abb79d-7689-4795-9e64-7343d3683033`::Granted request
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,976::task::495::TaskManager.Task::(_debug) Task f9dcf002-cb98-43cd-a054-32a0ca4f488b: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,976::task::495::TaskManager.Task::(_debug) Task f9dcf002-cb98-43cd-a054-32a0ca4f488b: ref 1 aborting False
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,977::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource '772472cb-2429-466d-96cd-00b3301ef78a_imageNS.13efb45b-f4d2-4c94-bbdb-91ce9b9b7321'
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,978::resourceManager::530::ResourceManager::(releaseResource) Released resource '772472cb-2429-466d-96cd-00b3301ef78a_imageNS.13efb45b-f4d2-4c94-bbdb-91ce9b9b7321' (0 active users)
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,978::resourceManager::535::ResourceManager::(releaseResource) Resource '772472cb-2429-466d-96cd-00b3301ef78a_imageNS.13efb45b-f4d2-4c94-bbdb-91ce9b9b7321' is free, finding out if anyone is waiting for it.
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,979::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource '772472cb-2429-466d-96cd-00b3301ef78a_volumeNS.088640f7-316c-4e2b-bfdc-0f64cf2913c8'
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,979::resourceManager::530::ResourceManager::(releaseResource) Released resource '772472cb-2429-466d-96cd-00b3301ef78a_volumeNS.088640f7-316c-4e2b-bfdc-0f64cf2913c8' (0 active users)
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,980::resourceManager::535::ResourceManager::(releaseResource) Resource '772472cb-2429-466d-96cd-00b3301ef78a_volumeNS.088640f7-316c-4e2b-bfdc-0f64cf2913c8' is free, finding out if anyone is waiting for it.
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,980::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource '772472cb-2429-466d-96cd-00b3301ef78a_volumeNS.088640f7-316c-4e2b-bfdc-0f64cf2913c8', Clearing records.
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,981::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource '772472cb-2429-466d-96cd-00b3301ef78a_imageNS.13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', Clearing records.
>libvirtEventLoop::INFO::2013-11-19 21:05:35,982::blockVolume::386::Storage.Volume::(teardown) Tearing down volume 772472cb-2429-466d-96cd-00b3301ef78a/088640f7-316c-4e2b-bfdc-0f64cf2913c8 justme False
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,982::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource '772472cb-2429-466d-96cd-00b3301ef78a_lvmActivationNS.088640f7-316c-4e2b-bfdc-0f64cf2913c8'
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,983::resourceManager::530::ResourceManager::(releaseResource) Released resource '772472cb-2429-466d-96cd-00b3301ef78a_lvmActivationNS.088640f7-316c-4e2b-bfdc-0f64cf2913c8' (0 active users)
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,983::resourceManager::535::ResourceManager::(releaseResource) Resource '772472cb-2429-466d-96cd-00b3301ef78a_lvmActivationNS.088640f7-316c-4e2b-bfdc-0f64cf2913c8' is free, finding out if anyone is waiting for it.
>libvirtEventLoop::DEBUG::2013-11-19 21:05:35,984::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --autobackup n --available n 772472cb-2429-466d-96cd-00b3301ef78a/088640f7-316c-4e2b-bfdc-0f64cf2913c8' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:05:36,445::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.155637 s, 6.6 MB/s\n'; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-19 21:05:36,757::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-19 21:05:36,758::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>libvirtEventLoop::DEBUG::2013-11-19 21:05:36,759::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>libvirtEventLoop::DEBUG::2013-11-19 21:05:36,760::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource '772472cb-2429-466d-96cd-00b3301ef78a_lvmActivationNS.088640f7-316c-4e2b-bfdc-0f64cf2913c8', Clearing records.
>libvirtEventLoop::DEBUG::2013-11-19 21:05:36,761::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>libvirtEventLoop::DEBUG::2013-11-19 21:05:36,762::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>libvirtEventLoop::DEBUG::2013-11-19 21:05:37,367::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-19 21:05:37,393::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>libvirtEventLoop::DEBUG::2013-11-19 21:05:37,394::task::495::TaskManager.Task::(_debug) Task f9dcf002-cb98-43cd-a054-32a0ca4f488b: finished: None
>libvirtEventLoop::DEBUG::2013-11-19 21:05:37,395::task::495::TaskManager.Task::(_debug) Task f9dcf002-cb98-43cd-a054-32a0ca4f488b: moving from state preparing -> state finished
>Thread-15910238::DEBUG::2013-11-19 21:05:37,396::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>libvirtEventLoop::DEBUG::2013-11-19 21:05:37,397::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>libvirtEventLoop::DEBUG::2013-11-19 21:05:37,402::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910238::INFO::2013-11-19 21:05:37,403::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>libvirtEventLoop::DEBUG::2013-11-19 21:05:37,404::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15910238::DEBUG::2013-11-19 21:05:37,410::task::495::TaskManager.Task::(_debug) Task 4e53534e-ac5a-4a8c-9b1f-b65da1fc8a0a: moving from state init -> state preparing
>Thread-15910238::DEBUG::2013-11-19 21:05:37,411::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=274 spmId=2
>libvirtEventLoop::DEBUG::2013-11-19 21:05:37,412::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15910238::DEBUG::2013-11-19 21:05:37,412::task::495::TaskManager.Task::(_debug) Task 4e53534e-ac5a-4a8c-9b1f-b65da1fc8a0a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>libvirtEventLoop::DEBUG::2013-11-19 21:05:37,413::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15910238::DEBUG::2013-11-19 21:05:37,414::task::495::TaskManager.Task::(_debug) Task 4e53534e-ac5a-4a8c-9b1f-b65da1fc8a0a: moving from state preparing -> state finished
>libvirtEventLoop::DEBUG::2013-11-19 21:05:37,414::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15910238::DEBUG::2013-11-19 21:05:37,415::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>libvirtEventLoop::DEBUG::2013-11-19 21:05:37,416::task::495::TaskManager.Task::(_debug) Task f9dcf002-cb98-43cd-a054-32a0ca4f488b: ref 0 aborting False
>Thread-15910238::DEBUG::2013-11-19 21:05:37,416::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>libvirtEventLoop::INFO::2013-11-19 21:05:37,417::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: teardownVolume, Return response: {'status': {'message': 'OK', 'code': 0}}
>Thread-15910238::DEBUG::2013-11-19 21:05:37,418::task::495::TaskManager.Task::(_debug) Task 4e53534e-ac5a-4a8c-9b1f-b65da1fc8a0a: ref 0 aborting False
>Thread-15910238::INFO::2013-11-19 21:05:37,419::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>libvirtEventLoop::DEBUG::2013-11-19 21:05:37,421::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/libexec/vdsm/hooks/after_vm_destroy/50_vhostmd' (cwd None)
>Thread-15910239::DEBUG::2013-11-19 21:05:37,656::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910239::INFO::2013-11-19 21:05:37,659::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910239::DEBUG::2013-11-19 21:05:37,660::task::495::TaskManager.Task::(_debug) Task 4d602ac7-9093-448a-83ae-df5181f00118: moving from state init -> state preparing
>Thread-15910239::DEBUG::2013-11-19 21:05:37,662::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3edf6fee-a3a2-405b-9b01-226b5a9e71fe`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910239::DEBUG::2013-11-19 21:05:37,663::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910239::DEBUG::2013-11-19 21:05:37,665::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910239::DEBUG::2013-11-19 21:05:37,666::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3edf6fee-a3a2-405b-9b01-226b5a9e71fe`::Granted request
>Thread-15910239::DEBUG::2013-11-19 21:05:37,668::task::495::TaskManager.Task::(_debug) Task 4d602ac7-9093-448a-83ae-df5181f00118: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910239::DEBUG::2013-11-19 21:05:37,669::task::495::TaskManager.Task::(_debug) Task 4d602ac7-9093-448a-83ae-df5181f00118: ref 1 aborting False
>Thread-15910239::WARNING::2013-11-19 21:05:37,673::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910239::DEBUG::2013-11-19 21:05:37,681::task::495::TaskManager.Task::(_debug) Task 4d602ac7-9093-448a-83ae-df5181f00118: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910239::DEBUG::2013-11-19 21:05:37,690::task::495::TaskManager.Task::(_debug) Task 4d602ac7-9093-448a-83ae-df5181f00118: moving from state preparing -> state finished
>Thread-15910239::DEBUG::2013-11-19 21:05:37,691::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910239::DEBUG::2013-11-19 21:05:37,693::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910239::DEBUG::2013-11-19 21:05:37,693::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910239::DEBUG::2013-11-19 21:05:37,694::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910239::DEBUG::2013-11-19 21:05:37,695::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910239::DEBUG::2013-11-19 21:05:37,695::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910239::DEBUG::2013-11-19 21:05:37,696::task::495::TaskManager.Task::(_debug) Task 4d602ac7-9093-448a-83ae-df5181f00118: ref 0 aborting False
>Thread-15910239::INFO::2013-11-19 21:05:37,697::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>libvirtEventLoop::DEBUG::2013-11-19 21:05:38,308::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>libvirtEventLoop::INFO::2013-11-19 21:05:38,309::hooks::51::root::(_runHooksDir) 
>libvirtEventLoop::DEBUG::2013-11-19 21:05:38,317::vm::742::vm.Vm::(setDownStatus) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Changed state to Down: User shut down
>libvirtEventLoop::DEBUG::2013-11-19 21:05:38,318::utils::314::vm.Vm::(stop) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Stop statistics collection
>Dummy-15909965::DEBUG::2013-11-19 21:05:38,462::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:05:39,005::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0738998 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15910242::DEBUG::2013-11-19 21:05:39,145::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('6574d88c-00ac-46f4-99ce-a2726932bfda',) {}
>Thread-15910242::DEBUG::2013-11-19 21:05:39,146::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Down', 'timeOffset': -1L, 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'exitMessage': 'User shut down', 'exitCode': 0}]}
>Thread-15910243::DEBUG::2013-11-19 21:05:39,159::clientIF::54::vds::(wrapper) [192.168.100.135]::call destroy with ('6574d88c-00ac-46f4-99ce-a2726932bfda',) {}
>Thread-15910243::INFO::2013-11-19 21:05:39,160::clientIF::450::vds::(destroy) vmContainerLock acquired by vm 6574d88c-00ac-46f4-99ce-a2726932bfda
>Thread-15910243::DEBUG::2013-11-19 21:05:39,160::libvirtvm::1344::vm.Vm::(destroy) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::destroy Called
>Thread-15910243::DEBUG::2013-11-19 21:05:39,161::libvirtvm::1339::vm.Vm::(deleteVm) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Total desktops after destroy of 6574d88c-00ac-46f4-99ce-a2726932bfda is 0
>Thread-15910243::DEBUG::2013-11-19 21:05:39,162::clientIF::59::vds::(wrapper) return destroy with {'status': {'message': 'Machine destroyed', 'code': 0}}
>Dummy-15909965::DEBUG::2013-11-19 21:05:41,020::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:05:41,573::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0739456 s, 13.8 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:05:42,316::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910245::INFO::2013-11-19 21:05:43,310::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910245::DEBUG::2013-11-19 21:05:43,311::task::495::TaskManager.Task::(_debug) Task 853ba3b9-f6d8-4d1e-935d-75445f77a73d: moving from state init -> state preparing
>Thread-15910245::DEBUG::2013-11-19 21:05:43,311::task::495::TaskManager.Task::(_debug) Task 853ba3b9-f6d8-4d1e-935d-75445f77a73d: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000452995300293', 'lastCheck': 1384895141.3012021, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00221800804138', 'lastCheck': 1384895137.3147531, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895142.317908, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000633001327515', 'lastCheck': 1384895140.305815, 'valid': True, 'code': 0}}
>Thread-15910245::DEBUG::2013-11-19 21:05:43,312::task::495::TaskManager.Task::(_debug) Task 853ba3b9-f6d8-4d1e-935d-75445f77a73d: moving from state preparing -> state finished
>Thread-15910245::DEBUG::2013-11-19 21:05:43,313::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910245::DEBUG::2013-11-19 21:05:43,313::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910245::DEBUG::2013-11-19 21:05:43,314::task::495::TaskManager.Task::(_debug) Task 853ba3b9-f6d8-4d1e-935d-75445f77a73d: ref 0 aborting False
>Thread-15910245::INFO::2013-11-19 21:05:43,315::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000452995300293', 'lastCheck': 1384895141.3012021, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00221800804138', 'lastCheck': 1384895137.3147531, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895142.317908, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000633001327515', 'lastCheck': 1384895140.305815, 'valid': True, 'code': 0}}
>Dummy-15909965::DEBUG::2013-11-19 21:05:43,589::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:05:44,120::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0742872 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15909965::DEBUG::2013-11-19 21:05:46,135::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:05:46,651::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0732189 s, 14.0 MB/s\n'; <rc> = 0
>Thread-15910249::DEBUG::2013-11-19 21:05:47,816::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910249::INFO::2013-11-19 21:05:47,818::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910249::DEBUG::2013-11-19 21:05:47,819::task::495::TaskManager.Task::(_debug) Task 883eca27-1678-4328-ad8d-cedcc347cfca: moving from state init -> state preparing
>Thread-15910249::DEBUG::2013-11-19 21:05:47,820::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=274 spmId=2
>Thread-15910249::DEBUG::2013-11-19 21:05:47,821::task::495::TaskManager.Task::(_debug) Task 883eca27-1678-4328-ad8d-cedcc347cfca: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910249::DEBUG::2013-11-19 21:05:47,822::task::495::TaskManager.Task::(_debug) Task 883eca27-1678-4328-ad8d-cedcc347cfca: moving from state preparing -> state finished
>Thread-15910249::DEBUG::2013-11-19 21:05:47,822::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910249::DEBUG::2013-11-19 21:05:47,823::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910249::DEBUG::2013-11-19 21:05:47,824::task::495::TaskManager.Task::(_debug) Task 883eca27-1678-4328-ad8d-cedcc347cfca: ref 0 aborting False
>Thread-15910249::INFO::2013-11-19 21:05:47,825::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910250::DEBUG::2013-11-19 21:05:47,836::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910250::INFO::2013-11-19 21:05:47,837::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910250::DEBUG::2013-11-19 21:05:47,838::task::495::TaskManager.Task::(_debug) Task 1aa6642b-2929-4be2-8144-17b048d21f0e: moving from state init -> state preparing
>Thread-15910250::DEBUG::2013-11-19 21:05:47,839::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`04aea70c-f457-4184-8700-044528fb10c9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910250::DEBUG::2013-11-19 21:05:47,839::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910250::DEBUG::2013-11-19 21:05:47,840::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910250::DEBUG::2013-11-19 21:05:47,841::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`04aea70c-f457-4184-8700-044528fb10c9`::Granted request
>Thread-15910250::DEBUG::2013-11-19 21:05:47,844::task::495::TaskManager.Task::(_debug) Task 1aa6642b-2929-4be2-8144-17b048d21f0e: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910250::DEBUG::2013-11-19 21:05:47,845::task::495::TaskManager.Task::(_debug) Task 1aa6642b-2929-4be2-8144-17b048d21f0e: ref 1 aborting False
>Thread-15910250::WARNING::2013-11-19 21:05:47,847::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910250::DEBUG::2013-11-19 21:05:47,855::task::495::TaskManager.Task::(_debug) Task 1aa6642b-2929-4be2-8144-17b048d21f0e: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910250::DEBUG::2013-11-19 21:05:47,859::task::495::TaskManager.Task::(_debug) Task 1aa6642b-2929-4be2-8144-17b048d21f0e: moving from state preparing -> state finished
>Thread-15910250::DEBUG::2013-11-19 21:05:47,859::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910250::DEBUG::2013-11-19 21:05:47,860::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910250::DEBUG::2013-11-19 21:05:47,860::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910250::DEBUG::2013-11-19 21:05:47,861::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910250::DEBUG::2013-11-19 21:05:47,862::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910250::DEBUG::2013-11-19 21:05:47,862::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910250::DEBUG::2013-11-19 21:05:47,863::task::495::TaskManager.Task::(_debug) Task 1aa6642b-2929-4be2-8144-17b048d21f0e: ref 0 aborting False
>Thread-15910250::INFO::2013-11-19 21:05:47,863::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 274}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15909965::DEBUG::2013-11-19 21:05:48,666::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:05:49,215::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0743069 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15909965::DEBUG::2013-11-19 21:05:51,230::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15909965::DEBUG::2013-11-19 21:05:51,754::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0750319 s, 13.6 MB/s\n'; <rc> = 0
>Thread-15910253::DEBUG::2013-11-19 21:05:52,345::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910253::INFO::2013-11-19 21:05:52,352::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksStatuses, args: ()
>Thread-15910253::DEBUG::2013-11-19 21:05:52,353::task::495::TaskManager.Task::(_debug) Task f73f2d85-3c96-4de2-b35a-30b89271318b: moving from state init -> state preparing
>Thread-15910253::DEBUG::2013-11-19 21:05:52,354::taskManager::92::TaskManager::(getAllTasksStatuses) Entry.
>Thread-15910253::DEBUG::2013-11-19 21:05:52,354::taskManager::97::TaskManager::(getAllTasksStatuses) Return: {}
>Thread-15910253::DEBUG::2013-11-19 21:05:52,355::task::495::TaskManager.Task::(_debug) Task f73f2d85-3c96-4de2-b35a-30b89271318b: finished: {'allTasksStatus': {}}
>Thread-15910253::DEBUG::2013-11-19 21:05:52,355::task::495::TaskManager.Task::(_debug) Task f73f2d85-3c96-4de2-b35a-30b89271318b: moving from state preparing -> state finished
>Thread-15910253::DEBUG::2013-11-19 21:05:52,356::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910253::DEBUG::2013-11-19 21:05:52,356::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910253::DEBUG::2013-11-19 21:05:52,357::task::495::TaskManager.Task::(_debug) Task f73f2d85-3c96-4de2-b35a-30b89271318b: ref 0 aborting False
>Thread-15910253::INFO::2013-11-19 21:05:52,357::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksStatuses, Return response: {'status': {'message': 'OK', 'code': 0}, 'allTasksStatus': {}}
>Thread-15910254::DEBUG::2013-11-19 21:05:52,384::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910254::INFO::2013-11-19 21:05:52,385::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: deactivateStorageDomain, args: ( sdUUID=5d7b5178-3790-472c-a13b-b481eb27a61e spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 msdUUID=00000000-0000-0000-0000-000000000000 masterVersion=1)
>Thread-15910254::DEBUG::2013-11-19 21:05:52,386::task::495::TaskManager.Task::(_debug) Task fc43eb4b-607e-44ef-b0dc-a0fa60171dd9: moving from state init -> state preparing
>Thread-15910254::DEBUG::2013-11-19 21:05:52,387::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`506b49c6-d267-4d73-a8d6-def453f3268f`::Request was made in '/usr/share/vdsm/storage/spm.py' line '1112' at 'public_deactivateStorageDomain'
>Thread-15910254::DEBUG::2013-11-19 21:05:52,388::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'exclusive'
>Thread-15910254::DEBUG::2013-11-19 21:05:52,389::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'exclusive' (1 active user)
>Thread-15910254::DEBUG::2013-11-19 21:05:52,389::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`506b49c6-d267-4d73-a8d6-def453f3268f`::Granted request
>Thread-15910254::DEBUG::2013-11-19 21:05:52,395::task::495::TaskManager.Task::(_debug) Task fc43eb4b-607e-44ef-b0dc-a0fa60171dd9: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (exclusive)
>Thread-15910254::DEBUG::2013-11-19 21:05:52,395::task::495::TaskManager.Task::(_debug) Task fc43eb4b-607e-44ef-b0dc-a0fa60171dd9: ref 1 aborting False
>Thread-15910254::DEBUG::2013-11-19 21:05:52,400::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.5d7b5178-3790-472c-a13b-b481eb27a61e`ReqID=`501cbbc7-efde-463f-917f-61d451f94c7d`::Request was made in '/usr/share/vdsm/storage/spm.py' line '1113' at 'public_deactivateStorageDomain'
>Thread-15910254::DEBUG::2013-11-19 21:05:52,400::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e' for lock type 'exclusive'
>Thread-15910254::DEBUG::2013-11-19 21:05:52,401::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e' is free. Now locking as 'exclusive' (1 active user)
>Thread-15910254::DEBUG::2013-11-19 21:05:52,402::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.5d7b5178-3790-472c-a13b-b481eb27a61e`ReqID=`501cbbc7-efde-463f-917f-61d451f94c7d`::Granted request
>Thread-15910254::DEBUG::2013-11-19 21:05:52,410::task::495::TaskManager.Task::(_debug) Task fc43eb4b-607e-44ef-b0dc-a0fa60171dd9: _resourcesAcquired: Storage.5d7b5178-3790-472c-a13b-b481eb27a61e (exclusive)
>Thread-15910254::DEBUG::2013-11-19 21:05:52,412::task::495::TaskManager.Task::(_debug) Task fc43eb4b-607e-44ef-b0dc-a0fa60171dd9: ref 1 aborting False
>Thread-15910254::INFO::2013-11-19 21:05:52,413::sp::775::Storage.StoragePool::(deactivateSD) sdUUID=5d7b5178-3790-472c-a13b-b481eb27a61e spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 new_msdUUID=00000000-0000-0000-0000-000000000000
>Thread-15910254::INFO::2013-11-19 21:05:52,414::fileSD::176::Storage.StorageDomain::(validate) sdUUID=5d7b5178-3790-472c-a13b-b481eb27a61e
>Thread-15910254::DEBUG::2013-11-19 21:05:52,415::persistentDict::204::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=[]
>Thread-15910254::WARNING::2013-11-19 21:05:52,415::persistentDict::226::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is
>Thread-15910254::ERROR::2013-11-19 21:05:52,416::task::868::TaskManager.Task::(_setError) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/task.py", line 876, in _run
>  File "/usr/share/vdsm/storage/spm.py", line 115, in run
>  File "/usr/share/vdsm/storage/spm.py", line 1115, in public_deactivateStorageDomain
>  File "/usr/share/vdsm/storage/sp.py", line 798, in deactivateSD
>  File "/usr/share/vdsm/storage/sd.py", line 666, in isBackup
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910254::DEBUG::2013-11-19 21:05:52,420::task::495::TaskManager.Task::(_debug) Task fc43eb4b-607e-44ef-b0dc-a0fa60171dd9: Task._run: fc43eb4b-607e-44ef-b0dc-a0fa60171dd9 ('5d7b5178-3790-472c-a13b-b481eb27a61e', 'c4e5889a-348d-11e2-83fe-9f72369eab30', '00000000-0000-0000-0000-000000000000', 1) {} failed - stopping task
>Thread-15910254::DEBUG::2013-11-19 21:05:52,421::task::495::TaskManager.Task::(_debug) Task fc43eb4b-607e-44ef-b0dc-a0fa60171dd9: stopping in state preparing (force False)
>Thread-15910254::DEBUG::2013-11-19 21:05:52,422::task::495::TaskManager.Task::(_debug) Task fc43eb4b-607e-44ef-b0dc-a0fa60171dd9: ref 1 aborting True
>Thread-15910254::INFO::2013-11-19 21:05:52,422::task::1171::TaskManager.Task::(prepare) aborting: Task is aborted: "'CLASS'" - code 100
>Thread-15910254::DEBUG::2013-11-19 21:05:52,423::task::495::TaskManager.Task::(_debug) Task fc43eb4b-607e-44ef-b0dc-a0fa60171dd9: Prepare: aborted: 'CLASS'
>Thread-15910254::DEBUG::2013-11-19 21:05:52,423::task::495::TaskManager.Task::(_debug) Task fc43eb4b-607e-44ef-b0dc-a0fa60171dd9: ref 0 aborting True
>Thread-15910254::DEBUG::2013-11-19 21:05:52,424::task::495::TaskManager.Task::(_debug) Task fc43eb4b-607e-44ef-b0dc-a0fa60171dd9: Task._doAbort: force False
>Thread-15910254::DEBUG::2013-11-19 21:05:52,424::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910254::DEBUG::2013-11-19 21:05:52,425::task::495::TaskManager.Task::(_debug) Task fc43eb4b-607e-44ef-b0dc-a0fa60171dd9: moving from state preparing -> state aborting
>Thread-15910254::DEBUG::2013-11-19 21:05:52,425::task::495::TaskManager.Task::(_debug) Task fc43eb4b-607e-44ef-b0dc-a0fa60171dd9: _aborting: recover policy none
>Thread-15910254::DEBUG::2013-11-19 21:05:52,426::task::495::TaskManager.Task::(_debug) Task fc43eb4b-607e-44ef-b0dc-a0fa60171dd9: moving from state aborting -> state failed
>Thread-15910254::DEBUG::2013-11-19 21:05:52,426::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>, 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e': < ResourceRef 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e', isValid: 'True' obj: 'None'>}
>Thread-15910254::DEBUG::2013-11-19 21:05:52,430::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910254::DEBUG::2013-11-19 21:05:52,431::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910254::DEBUG::2013-11-19 21:05:52,432::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910254::DEBUG::2013-11-19 21:05:52,432::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910254::DEBUG::2013-11-19 21:05:52,433::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910254::DEBUG::2013-11-19 21:05:52,433::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e'
>Thread-15910254::DEBUG::2013-11-19 21:05:52,434::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e' (0 active users)
>Thread-15910254::DEBUG::2013-11-19 21:05:52,434::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e' is free, finding out if anyone is waiting for it.
>Thread-15910254::DEBUG::2013-11-19 21:05:52,435::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e', Clearing records.
>Thread-15910254::ERROR::2013-11-19 21:05:52,436::dispatcher::106::Storage.Dispatcher.Protect::(run) 'CLASS'
>Thread-15910254::ERROR::2013-11-19 21:05:52,436::dispatcher::107::Storage.Dispatcher.Protect::(run) Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/dispatcher.py", line 96, in run
>  File "/usr/share/vdsm/storage/task.py", line 1178, in prepare
>KeyError: 'CLASS'
>
>Thread-1024767::ERROR::2013-11-19 21:05:52,441::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910255::DEBUG::2013-11-19 21:05:52,452::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910255::INFO::2013-11-19 21:05:52,453::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910255::DEBUG::2013-11-19 21:05:52,453::task::495::TaskManager.Task::(_debug) Task 123bfd63-731b-4596-b335-c5c1f254af6e: moving from state init -> state preparing
>Thread-15910255::DEBUG::2013-11-19 21:05:52,454::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=274 spmId=2
>Thread-15910255::DEBUG::2013-11-19 21:05:52,455::task::495::TaskManager.Task::(_debug) Task 123bfd63-731b-4596-b335-c5c1f254af6e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910255::DEBUG::2013-11-19 21:05:52,455::task::495::TaskManager.Task::(_debug) Task 123bfd63-731b-4596-b335-c5c1f254af6e: moving from state preparing -> state finished
>Thread-15910255::DEBUG::2013-11-19 21:05:52,456::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910255::DEBUG::2013-11-19 21:05:52,456::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910255::DEBUG::2013-11-19 21:05:52,457::task::495::TaskManager.Task::(_debug) Task 123bfd63-731b-4596-b335-c5c1f254af6e: ref 0 aborting False
>Thread-15910255::INFO::2013-11-19 21:05:52,457::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 274}}
>Thread-15910256::DEBUG::2013-11-19 21:05:52,468::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910256::INFO::2013-11-19 21:05:52,469::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksStatuses, args: ()
>Thread-15910256::DEBUG::2013-11-19 21:05:52,470::task::495::TaskManager.Task::(_debug) Task 327e0066-91e6-4354-a297-d7bcdcd10123: moving from state init -> state preparing
>Thread-15910256::DEBUG::2013-11-19 21:05:52,470::taskManager::92::TaskManager::(getAllTasksStatuses) Entry.
>Thread-15910256::DEBUG::2013-11-19 21:05:52,471::taskManager::97::TaskManager::(getAllTasksStatuses) Return: {}
>Thread-15910256::DEBUG::2013-11-19 21:05:52,471::task::495::TaskManager.Task::(_debug) Task 327e0066-91e6-4354-a297-d7bcdcd10123: finished: {'allTasksStatus': {}}
>Thread-15910256::DEBUG::2013-11-19 21:05:52,472::task::495::TaskManager.Task::(_debug) Task 327e0066-91e6-4354-a297-d7bcdcd10123: moving from state preparing -> state finished
>Thread-15910256::DEBUG::2013-11-19 21:05:52,473::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910256::DEBUG::2013-11-19 21:05:52,473::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910256::DEBUG::2013-11-19 21:05:52,474::task::495::TaskManager.Task::(_debug) Task 327e0066-91e6-4354-a297-d7bcdcd10123: ref 0 aborting False
>Thread-15910256::INFO::2013-11-19 21:05:52,474::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksStatuses, Return response: {'status': {'message': 'OK', 'code': 0}, 'allTasksStatus': {}}
>Thread-15910257::DEBUG::2013-11-19 21:05:52,482::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910257::INFO::2013-11-19 21:05:52,487::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: spmStop, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910257::DEBUG::2013-11-19 21:05:52,488::task::495::TaskManager.Task::(_debug) Task 1cee9ac6-7302-4a86-abf3-67b0ce0fabdb: moving from state init -> state preparing
>Thread-15910257::DEBUG::2013-11-19 21:05:52,488::spm::547::Storage.SPM::(public_spmStop) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30
>Thread-15910257::DEBUG::2013-11-19 21:05:52,489::taskManager::105::TaskManager::(getAllTasks) Entry.
>Thread-15910257::DEBUG::2013-11-19 21:05:52,489::taskManager::110::TaskManager::(getAllTasks) Return: {}
>Thread-15910257::DEBUG::2013-11-19 21:05:52,490::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f225f88c-6d25-445e-81af-17a5da96a59a`::Request was made in '/usr/share/vdsm/storage/spm.py' line '573' at 'public_spmStop'
>Thread-15910257::DEBUG::2013-11-19 21:05:52,491::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'exclusive'
>Thread-15910257::DEBUG::2013-11-19 21:05:52,492::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'exclusive' (1 active user)
>Thread-15910257::DEBUG::2013-11-19 21:05:52,492::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f225f88c-6d25-445e-81af-17a5da96a59a`::Granted request
>Thread-15910257::DEBUG::2013-11-19 21:05:52,499::task::495::TaskManager.Task::(_debug) Task 1cee9ac6-7302-4a86-abf3-67b0ce0fabdb: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (exclusive)
>Thread-15910257::DEBUG::2013-11-19 21:05:52,499::task::495::TaskManager.Task::(_debug) Task 1cee9ac6-7302-4a86-abf3-67b0ce0fabdb: ref 1 aborting False
>Thread-15910257::DEBUG::2013-11-19 21:05:52,500::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`cd3e0a6d-6148-4742-bb1f-6c9ee5fdb6d6`::Request was made in '/usr/share/vdsm/storage/spm.py' line '584' at '_stop'
>Thread-15910257::DEBUG::2013-11-19 21:05:52,501::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'exclusive'
>Thread-15910257::DEBUG::2013-11-19 21:05:52,501::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'exclusive' (1 active user)
>Thread-15910257::DEBUG::2013-11-19 21:05:52,502::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`cd3e0a6d-6148-4742-bb1f-6c9ee5fdb6d6`::Granted request
>Thread-15910257::DEBUG::2013-11-19 21:05:52,505::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e`ReqID=`4243a79a-03e3-41d0-8176-fa99c9e084e8`::Request was made in '/usr/share/vdsm/storage/spm.py' line '601' at '_stop'
>Thread-15910257::DEBUG::2013-11-19 21:05:52,506::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e' for lock type 'exclusive'
>Thread-15910257::DEBUG::2013-11-19 21:05:52,506::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e' is free. Now locking as 'exclusive' (1 active user)
>Thread-15910257::DEBUG::2013-11-19 21:05:52,513::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e`ReqID=`4243a79a-03e3-41d0-8176-fa99c9e084e8`::Granted request
>Thread-15910257::DEBUG::2013-11-19 21:05:52,520::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e'
>Thread-15910257::DEBUG::2013-11-19 21:05:52,523::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e' (0 active users)
>Thread-15910257::DEBUG::2013-11-19 21:05:52,523::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e' is free, finding out if anyone is waiting for it.
>Thread-15910257::DEBUG::2013-11-19 21:05:52,524::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e', Clearing records.
>Thread-15910257::DEBUG::2013-11-19 21:05:52,525::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910257::DEBUG::2013-11-19 21:05:52,525::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910257::DEBUG::2013-11-19 21:05:52,526::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910257::DEBUG::2013-11-19 21:05:52,526::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910257::DEBUG::2013-11-19 21:05:52,527::taskManager::118::TaskManager::(unloadTasks) Entry.
>Thread-15910257::DEBUG::2013-11-19 21:05:52,527::taskManager::122::TaskManager::(unloadTasks) Return
>Thread-15910257::DEBUG::2013-11-19 21:05:52,529::spm::195::Storage.SPM::(__cleanupMasterMount) unmounting /rhev/data-center/mnt/blockSD/6b892067-ac7e-4039-a7be-5b7387525b16/master
>Thread-15910257::DEBUG::2013-11-19 21:05:52,655::fileUtils::112::Storage.Misc.excCmd::(umount) '/usr/bin/sudo -n /bin/umount -f /rhev/data-center/mnt/blockSD/6b892067-ac7e-4039-a7be-5b7387525b16/master' (cwd None)
>Thread-15910257::DEBUG::2013-11-19 21:05:53,057::fileUtils::112::Storage.Misc.excCmd::(umount) SUCCESS: <err> = ''; <rc> = 0
>Thread-15910257::DEBUG::2013-11-19 21:05:53,060::spm::290::Storage.SPM::(__cleanupSPM) cleaning up SPM: c4e5889a-348d-11e2-83fe-9f72369eab30
>Thread-15910257::DEBUG::2013-11-19 21:05:53,061::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
>Thread-15910257::DEBUG::2013-11-19 21:05:53,062::persistentDict::146::Storage.PersistentDict::(transaction) Flushing changes
>Thread-15910257::DEBUG::2013-11-19 21:05:53,063::persistentDict::265::Storage.PersistentDict::(flush) about to write lines (VGTagMetadataRW)=[u'CLASS=Data', u'DESCRIPTION=OS', u'IOOPTIMEOUTSEC=10', u'LEASERETRIES=3', u'LEASETIMESEC=60', u'LOCKPOLICY=', u'LOCKRENEWALINTERVALSEC=5', u'LOGBLKSIZE=512', u'MASTER_VERSION=1', u'PHYBLKSIZE=512', u'POOL_DESCRIPTION=Default', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_ID=-1', u'POOL_SPM_LVER=274', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'ROLE=Master', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'TYPE=FCP', u'VERSION=2', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', '_SHA_CKSUM=a631aa7004ad6a1f7b79a293dfe44576de53980f']
>Thread-15910257::DEBUG::2013-11-19 21:05:53,075::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15910257::DEBUG::2013-11-19 21:05:53,076::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15910257::DEBUG::2013-11-19 21:05:53,077::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15910257::DEBUG::2013-11-19 21:05:53,078::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15910257::DEBUG::2013-11-19 21:05:53,079::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-15910257::DEBUG::2013-11-19 21:05:53,086::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15910258::INFO::2013-11-19 21:05:53,482::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910258::DEBUG::2013-11-19 21:05:53,483::task::495::TaskManager.Task::(_debug) Task ccfafc69-876c-4daf-abf7-72b838fe597f: moving from state init -> state preparing
>Thread-15910258::DEBUG::2013-11-19 21:05:53,484::task::495::TaskManager.Task::(_debug) Task ccfafc69-876c-4daf-abf7-72b838fe597f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000458955764771', 'lastCheck': 1384895151.474242, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00147604942322', 'lastCheck': 1384895147.3423131, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895152.442081, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000319004058838', 'lastCheck': 1384895150.340637, 'valid': True, 'code': 0}}
>Thread-15910258::DEBUG::2013-11-19 21:05:53,490::task::495::TaskManager.Task::(_debug) Task ccfafc69-876c-4daf-abf7-72b838fe597f: moving from state preparing -> state finished
>Thread-15910258::DEBUG::2013-11-19 21:05:53,491::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910258::DEBUG::2013-11-19 21:05:53,495::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910258::DEBUG::2013-11-19 21:05:53,496::task::495::TaskManager.Task::(_debug) Task ccfafc69-876c-4daf-abf7-72b838fe597f: ref 0 aborting False
>Thread-15910258::INFO::2013-11-19 21:05:53,500::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000458955764771', 'lastCheck': 1384895151.474242, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00147604942322', 'lastCheck': 1384895147.3423131, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895152.442081, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000319004058838', 'lastCheck': 1384895150.340637, 'valid': True, 'code': 0}}
>Thread-15910257::DEBUG::2013-11-19 21:05:53,764::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15909965::INFO::2013-11-19 21:05:53,766::storage_mailbox::673::Storage.MailBox.SpmMailMonitor::(run) SPM_MailMonitor - Incoming mail monitoring thread stopped
>Thread-15910257::DEBUG::2013-11-19 21:05:53,790::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-15910257::DEBUG::2013-11-19 21:05:53,792::blockSD::145::storage.Metadata.VGTagMetadataRW::(writelines) Updating metadata adding=MDT_POOL_SPM_ID=-1, MDT__SHA_CKSUM=a631aa7004ad6a1f7b79a293dfe44576de53980f removing=MDT__SHA_CKSUM=e643d303df7878b6d71c193b5f1f2469245114b5, MDT_POOL_SPM_ID=2
>Thread-15910257::DEBUG::2013-11-19 21:05:53,793::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --deltag MDT_POOL_SPM_ID=2 --deltag MDT__SHA_CKSUM=e643d303df7878b6d71c193b5f1f2469245114b5 --addtag MDT_POOL_SPM_ID=-1 --addtag MDT__SHA_CKSUM=a631aa7004ad6a1f7b79a293dfe44576de53980f 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15910257::DEBUG::2013-11-19 21:05:54,449::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15910257::DEBUG::2013-11-19 21:05:54,451::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15910257::DEBUG::2013-11-19 21:05:54,452::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15910257::DEBUG::2013-11-19 21:05:54,453::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction
>Thread-15910257::INFO::2013-11-19 21:05:54,454::safelease::83::ClusterLock::(release) Releasing cluster lock for domain 6b892067-ac7e-4039-a7be-5b7387525b16
>Thread-15910257::DEBUG::2013-11-19 21:05:54,459::safelease::84::Storage.Misc.excCmd::(release) '/usr/libexec/vdsm/spmstop.sh 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd /usr/libexec/vdsm/)
>Thread-15910257::DEBUG::2013-11-19 21:05:55,946::safelease::84::Storage.Misc.excCmd::(release) SUCCESS: <err> = ''; <rc> = 0
>Thread-15910257::DEBUG::2013-11-19 21:05:55,947::safelease::87::ClusterLock::(release) Cluster lock released successfully
>Thread-15910257::DEBUG::2013-11-19 21:05:55,948::task::495::TaskManager.Task::(_debug) Task 1cee9ac6-7302-4a86-abf3-67b0ce0fabdb: finished: None
>Thread-15910257::DEBUG::2013-11-19 21:05:55,949::task::495::TaskManager.Task::(_debug) Task 1cee9ac6-7302-4a86-abf3-67b0ce0fabdb: moving from state preparing -> state finished
>Thread-15910257::DEBUG::2013-11-19 21:05:55,949::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910257::DEBUG::2013-11-19 21:05:55,950::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910257::DEBUG::2013-11-19 21:05:55,951::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910257::DEBUG::2013-11-19 21:05:55,952::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910257::DEBUG::2013-11-19 21:05:55,952::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910257::DEBUG::2013-11-19 21:05:55,953::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910257::DEBUG::2013-11-19 21:05:55,953::task::495::TaskManager.Task::(_debug) Task 1cee9ac6-7302-4a86-abf3-67b0ce0fabdb: ref 0 aborting False
>Thread-15910257::INFO::2013-11-19 21:05:55,954::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: spmStop, Return response: {'status': {'message': 'OK', 'code': 0}}
>Thread-42838::DEBUG::2013-11-19 21:06:01,681::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-42838::DEBUG::2013-11-19 21:06:01,683::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 21:06:02,295::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-42838::DEBUG::2013-11-19 21:06:02,299::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1024767::ERROR::2013-11-19 21:06:02,454::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:06:12,468::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:06:22,482::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:06:32,496::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:06:42,510::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:06:52,524::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-19 21:07:00,523::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-19 21:07:01,247::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:07:02,751::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:07:12,765::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:07:22,779::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:07:32,795::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:07:42,809::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:07:52,822::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:08:02,837::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:08:12,851::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:08:22,864::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:08:32,888::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910264::INFO::2013-11-19 21:08:42,470::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910264::DEBUG::2013-11-19 21:08:42,473::task::495::TaskManager.Task::(_debug) Task ab4b5d65-7852-4b9d-abc1-59466c2d300e: moving from state init -> state preparing
>Thread-15910264::DEBUG::2013-11-19 21:08:42,475::task::495::TaskManager.Task::(_debug) Task ab4b5d65-7852-4b9d-abc1-59466c2d300e: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000408172607422', 'lastCheck': 1384895312.5005269, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00215601921082', 'lastCheck': 1384895317.9003539, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895312.889158, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00042986869812', 'lastCheck': 1384895321.363343, 'valid': True, 'code': 0}}
>Thread-15910264::DEBUG::2013-11-19 21:08:42,476::task::495::TaskManager.Task::(_debug) Task ab4b5d65-7852-4b9d-abc1-59466c2d300e: moving from state preparing -> state finished
>Thread-15910264::DEBUG::2013-11-19 21:08:42,478::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910264::DEBUG::2013-11-19 21:08:42,479::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910264::DEBUG::2013-11-19 21:08:42,479::task::495::TaskManager.Task::(_debug) Task ab4b5d65-7852-4b9d-abc1-59466c2d300e: ref 0 aborting False
>Thread-15910264::INFO::2013-11-19 21:08:42,480::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000408172607422', 'lastCheck': 1384895312.5005269, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00215601921082', 'lastCheck': 1384895317.9003539, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895312.889158, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00042986869812', 'lastCheck': 1384895321.363343, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:08:42,902::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910270::INFO::2013-11-19 21:08:52,612::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910270::DEBUG::2013-11-19 21:08:52,613::task::495::TaskManager.Task::(_debug) Task b91d27b9-3fdc-4d79-ac58-6040fb8ed9fd: moving from state init -> state preparing
>Thread-15910270::DEBUG::2013-11-19 21:08:52,614::task::495::TaskManager.Task::(_debug) Task b91d27b9-3fdc-4d79-ac58-6040fb8ed9fd: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000324964523315', 'lastCheck': 1384895332.5263181, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00136399269104', 'lastCheck': 1384895327.915976, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895322.903131, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000418186187744', 'lastCheck': 1384895331.3750341, 'valid': True, 'code': 0}}
>Thread-15910270::DEBUG::2013-11-19 21:08:52,615::task::495::TaskManager.Task::(_debug) Task b91d27b9-3fdc-4d79-ac58-6040fb8ed9fd: moving from state preparing -> state finished
>Thread-15910270::DEBUG::2013-11-19 21:08:52,615::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910270::DEBUG::2013-11-19 21:08:52,616::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910270::DEBUG::2013-11-19 21:08:52,616::task::495::TaskManager.Task::(_debug) Task b91d27b9-3fdc-4d79-ac58-6040fb8ed9fd: ref 0 aborting False
>Thread-15910270::INFO::2013-11-19 21:08:52,617::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000324964523315', 'lastCheck': 1384895332.5263181, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00136399269104', 'lastCheck': 1384895327.915976, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895322.903131, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000418186187744', 'lastCheck': 1384895331.3750341, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:08:52,916::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910276::INFO::2013-11-19 21:09:02,734::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910276::DEBUG::2013-11-19 21:09:02,735::task::495::TaskManager.Task::(_debug) Task b0e26d18-3490-43e4-b7c6-e9b3c7452586: moving from state init -> state preparing
>Thread-15910276::DEBUG::2013-11-19 21:09:02,735::task::495::TaskManager.Task::(_debug) Task b0e26d18-3490-43e4-b7c6-e9b3c7452586: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000302076339722', 'lastCheck': 1384895342.5392871, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00125598907471', 'lastCheck': 1384895337.9319, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895332.9171131, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000450134277344', 'lastCheck': 1384895341.3871281, 'valid': True, 'code': 0}}
>Thread-15910276::DEBUG::2013-11-19 21:09:02,736::task::495::TaskManager.Task::(_debug) Task b0e26d18-3490-43e4-b7c6-e9b3c7452586: moving from state preparing -> state finished
>Thread-15910276::DEBUG::2013-11-19 21:09:02,737::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910276::DEBUG::2013-11-19 21:09:02,737::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910276::DEBUG::2013-11-19 21:09:02,738::task::495::TaskManager.Task::(_debug) Task b0e26d18-3490-43e4-b7c6-e9b3c7452586: ref 0 aborting False
>Thread-15910276::INFO::2013-11-19 21:09:02,739::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000302076339722', 'lastCheck': 1384895342.5392871, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00125598907471', 'lastCheck': 1384895337.9319, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895332.9171131, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000450134277344', 'lastCheck': 1384895341.3871281, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:09:02,930::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910282::INFO::2013-11-19 21:09:12,899::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910282::DEBUG::2013-11-19 21:09:12,900::task::495::TaskManager.Task::(_debug) Task 7326c44d-23b2-4aad-be1b-4aa9265459a7: moving from state init -> state preparing
>Thread-15910282::DEBUG::2013-11-19 21:09:12,900::task::495::TaskManager.Task::(_debug) Task 7326c44d-23b2-4aad-be1b-4aa9265459a7: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000398874282837', 'lastCheck': 1384895352.5523, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00137686729431', 'lastCheck': 1384895347.9473341, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895342.931057, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000427007675171', 'lastCheck': 1384895351.398876, 'valid': True, 'code': 0}}
>Thread-15910282::DEBUG::2013-11-19 21:09:12,901::task::495::TaskManager.Task::(_debug) Task 7326c44d-23b2-4aad-be1b-4aa9265459a7: moving from state preparing -> state finished
>Thread-15910282::DEBUG::2013-11-19 21:09:12,902::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910282::DEBUG::2013-11-19 21:09:12,902::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910282::DEBUG::2013-11-19 21:09:12,903::task::495::TaskManager.Task::(_debug) Task 7326c44d-23b2-4aad-be1b-4aa9265459a7: ref 0 aborting False
>Thread-15910282::INFO::2013-11-19 21:09:12,903::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000398874282837', 'lastCheck': 1384895352.5523, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00137686729431', 'lastCheck': 1384895347.9473341, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895342.931057, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000427007675171', 'lastCheck': 1384895351.398876, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:09:12,944::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-19 21:09:22,563::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:09:22,960::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910288::INFO::2013-11-19 21:09:23,055::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910288::DEBUG::2013-11-19 21:09:23,055::task::495::TaskManager.Task::(_debug) Task deebe857-c003-45ea-a57f-4ef99642b367: moving from state init -> state preparing
>Thread-15910288::DEBUG::2013-11-19 21:09:23,056::task::495::TaskManager.Task::(_debug) Task deebe857-c003-45ea-a57f-4ef99642b367: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000398874282837', 'lastCheck': 1384895352.5523, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00136208534241', 'lastCheck': 1384895357.962534, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895362.9622829, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000322103500366', 'lastCheck': 1384895361.4105051, 'valid': True, 'code': 0}}
>Thread-15910288::DEBUG::2013-11-19 21:09:23,057::task::495::TaskManager.Task::(_debug) Task deebe857-c003-45ea-a57f-4ef99642b367: moving from state preparing -> state finished
>Thread-15910288::DEBUG::2013-11-19 21:09:23,058::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910288::DEBUG::2013-11-19 21:09:23,058::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910288::DEBUG::2013-11-19 21:09:23,059::task::495::TaskManager.Task::(_debug) Task deebe857-c003-45ea-a57f-4ef99642b367: ref 0 aborting False
>Thread-15910288::INFO::2013-11-19 21:09:23,059::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000398874282837', 'lastCheck': 1384895352.5523, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00136208534241', 'lastCheck': 1384895357.962534, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895362.9622829, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000322103500366', 'lastCheck': 1384895361.4105051, 'valid': True, 'code': 0}}
>Thread-42838::DEBUG::2013-11-19 21:09:23,198::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:09:32,975::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910294::INFO::2013-11-19 21:09:33,189::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910294::DEBUG::2013-11-19 21:09:33,190::task::495::TaskManager.Task::(_debug) Task d69fbd7c-1698-4d00-9207-13609135a8e3: moving from state init -> state preparing
>Thread-15910294::DEBUG::2013-11-19 21:09:33,191::task::495::TaskManager.Task::(_debug) Task d69fbd7c-1698-4d00-9207-13609135a8e3: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000361919403076', 'lastCheck': 1384895363.2013409, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00292801856995', 'lastCheck': 1384895367.984206, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895372.9765589, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000463962554932', 'lastCheck': 1384895371.4224789, 'valid': True, 'code': 0}}
>Thread-15910294::DEBUG::2013-11-19 21:09:33,192::task::495::TaskManager.Task::(_debug) Task d69fbd7c-1698-4d00-9207-13609135a8e3: moving from state preparing -> state finished
>Thread-15910294::DEBUG::2013-11-19 21:09:33,192::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910294::DEBUG::2013-11-19 21:09:33,193::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910294::DEBUG::2013-11-19 21:09:33,193::task::495::TaskManager.Task::(_debug) Task d69fbd7c-1698-4d00-9207-13609135a8e3: ref 0 aborting False
>Thread-15910294::INFO::2013-11-19 21:09:33,194::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000361919403076', 'lastCheck': 1384895363.2013409, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00292801856995', 'lastCheck': 1384895367.984206, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895372.9765589, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000463962554932', 'lastCheck': 1384895371.4224789, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:09:42,992::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910300::INFO::2013-11-19 21:09:43,363::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910300::DEBUG::2013-11-19 21:09:43,364::task::495::TaskManager.Task::(_debug) Task baafe264-10fa-46ea-afa3-fffe3bf1a188: moving from state init -> state preparing
>Thread-15910300::DEBUG::2013-11-19 21:09:43,365::task::495::TaskManager.Task::(_debug) Task baafe264-10fa-46ea-afa3-fffe3bf1a188: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000962018966675', 'lastCheck': 1384895383.226661, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00221610069275', 'lastCheck': 1384895378.001792, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895382.9941361, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000440835952759', 'lastCheck': 1384895381.434324, 'valid': True, 'code': 0}}
>Thread-15910300::DEBUG::2013-11-19 21:09:43,365::task::495::TaskManager.Task::(_debug) Task baafe264-10fa-46ea-afa3-fffe3bf1a188: moving from state preparing -> state finished
>Thread-15910300::DEBUG::2013-11-19 21:09:43,366::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910300::DEBUG::2013-11-19 21:09:43,367::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910300::DEBUG::2013-11-19 21:09:43,367::task::495::TaskManager.Task::(_debug) Task baafe264-10fa-46ea-afa3-fffe3bf1a188: ref 0 aborting False
>Thread-15910300::INFO::2013-11-19 21:09:43,368::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000962018966675', 'lastCheck': 1384895383.226661, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00221610069275', 'lastCheck': 1384895378.001792, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895382.9941361, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000440835952759', 'lastCheck': 1384895381.434324, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:09:53,005::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910306::INFO::2013-11-19 21:09:53,543::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910306::DEBUG::2013-11-19 21:09:53,544::task::495::TaskManager.Task::(_debug) Task 231dcf0a-1bec-4674-9ac3-e1465f549f8b: moving from state init -> state preparing
>Thread-15910306::DEBUG::2013-11-19 21:09:53,544::task::495::TaskManager.Task::(_debug) Task 231dcf0a-1bec-4674-9ac3-e1465f549f8b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000321865081787', 'lastCheck': 1384895393.2395229, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00140905380249', 'lastCheck': 1384895388.0171759, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895393.006654, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000323057174683', 'lastCheck': 1384895391.445971, 'valid': True, 'code': 0}}
>Thread-15910306::DEBUG::2013-11-19 21:09:53,545::task::495::TaskManager.Task::(_debug) Task 231dcf0a-1bec-4674-9ac3-e1465f549f8b: moving from state preparing -> state finished
>Thread-15910306::DEBUG::2013-11-19 21:09:53,546::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910306::DEBUG::2013-11-19 21:09:53,546::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910306::DEBUG::2013-11-19 21:09:53,547::task::495::TaskManager.Task::(_debug) Task 231dcf0a-1bec-4674-9ac3-e1465f549f8b: ref 0 aborting False
>Thread-15910306::INFO::2013-11-19 21:09:53,547::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000321865081787', 'lastCheck': 1384895393.2395229, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00140905380249', 'lastCheck': 1384895388.0171759, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895393.006654, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000323057174683', 'lastCheck': 1384895391.445971, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:10:03,020::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910312::INFO::2013-11-19 21:10:03,723::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910312::DEBUG::2013-11-19 21:10:03,724::task::495::TaskManager.Task::(_debug) Task e9f40114-a5f4-47b2-8fce-e78b781acc9f: moving from state init -> state preparing
>Thread-15910312::DEBUG::2013-11-19 21:10:03,733::task::495::TaskManager.Task::(_debug) Task e9f40114-a5f4-47b2-8fce-e78b781acc9f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000370979309082', 'lastCheck': 1384895403.2527349, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00118780136108', 'lastCheck': 1384895398.03247, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895403.0211179, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000417947769165', 'lastCheck': 1384895401.4578071, 'valid': True, 'code': 0}}
>Thread-15910312::DEBUG::2013-11-19 21:10:03,734::task::495::TaskManager.Task::(_debug) Task e9f40114-a5f4-47b2-8fce-e78b781acc9f: moving from state preparing -> state finished
>Thread-15910312::DEBUG::2013-11-19 21:10:03,735::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910312::DEBUG::2013-11-19 21:10:03,735::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910312::DEBUG::2013-11-19 21:10:03,736::task::495::TaskManager.Task::(_debug) Task e9f40114-a5f4-47b2-8fce-e78b781acc9f: ref 0 aborting False
>Thread-15910312::INFO::2013-11-19 21:10:03,736::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000370979309082', 'lastCheck': 1384895403.2527349, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00118780136108', 'lastCheck': 1384895398.03247, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895403.0211179, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000417947769165', 'lastCheck': 1384895401.4578071, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:10:13,034::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910318::INFO::2013-11-19 21:10:13,871::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910318::DEBUG::2013-11-19 21:10:13,872::task::495::TaskManager.Task::(_debug) Task e4e303f5-5190-429d-bcb7-b1a1b05b0e01: moving from state init -> state preparing
>Thread-15910318::DEBUG::2013-11-19 21:10:13,881::task::495::TaskManager.Task::(_debug) Task e4e303f5-5190-429d-bcb7-b1a1b05b0e01: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000344038009644', 'lastCheck': 1384895413.2657571, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00155305862427', 'lastCheck': 1384895408.0483191, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895413.034914, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000426054000854', 'lastCheck': 1384895411.4693689, 'valid': True, 'code': 0}}
>Thread-15910318::DEBUG::2013-11-19 21:10:13,882::task::495::TaskManager.Task::(_debug) Task e4e303f5-5190-429d-bcb7-b1a1b05b0e01: moving from state preparing -> state finished
>Thread-15910318::DEBUG::2013-11-19 21:10:13,882::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910318::DEBUG::2013-11-19 21:10:13,883::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910318::DEBUG::2013-11-19 21:10:13,884::task::495::TaskManager.Task::(_debug) Task e4e303f5-5190-429d-bcb7-b1a1b05b0e01: ref 0 aborting False
>Thread-15910318::INFO::2013-11-19 21:10:13,884::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000344038009644', 'lastCheck': 1384895413.2657571, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00155305862427', 'lastCheck': 1384895408.0483191, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895413.034914, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000426054000854', 'lastCheck': 1384895411.4693689, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:10:23,047::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910324::INFO::2013-11-19 21:10:23,999::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910324::DEBUG::2013-11-19 21:10:24,000::task::495::TaskManager.Task::(_debug) Task b209c23f-e4e8-4ffb-a380-09863d825916: moving from state init -> state preparing
>Thread-15910324::DEBUG::2013-11-19 21:10:24,001::task::495::TaskManager.Task::(_debug) Task b209c23f-e4e8-4ffb-a380-09863d825916: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000339984893799', 'lastCheck': 1384895423.2786729, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00136399269104', 'lastCheck': 1384895418.063956, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895423.048322, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000330924987793', 'lastCheck': 1384895421.4808969, 'valid': True, 'code': 0}}
>Thread-15910324::DEBUG::2013-11-19 21:10:24,001::task::495::TaskManager.Task::(_debug) Task b209c23f-e4e8-4ffb-a380-09863d825916: moving from state preparing -> state finished
>Thread-15910324::DEBUG::2013-11-19 21:10:24,002::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910324::DEBUG::2013-11-19 21:10:24,002::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910324::DEBUG::2013-11-19 21:10:24,003::task::495::TaskManager.Task::(_debug) Task b209c23f-e4e8-4ffb-a380-09863d825916: ref 0 aborting False
>Thread-15910324::INFO::2013-11-19 21:10:24,004::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000339984893799', 'lastCheck': 1384895423.2786729, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00136399269104', 'lastCheck': 1384895418.063956, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895423.048322, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000330924987793', 'lastCheck': 1384895421.4808969, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:10:33,061::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910330::INFO::2013-11-19 21:10:34,166::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910330::DEBUG::2013-11-19 21:10:34,167::task::495::TaskManager.Task::(_debug) Task bec9db4b-e505-43ac-88da-3ddee52b18f3: moving from state init -> state preparing
>Thread-15910330::DEBUG::2013-11-19 21:10:34,167::task::495::TaskManager.Task::(_debug) Task bec9db4b-e505-43ac-88da-3ddee52b18f3: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000358104705811', 'lastCheck': 1384895433.2918279, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00137400627136', 'lastCheck': 1384895428.079247, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895433.062007, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000414848327637', 'lastCheck': 1384895431.4925799, 'valid': True, 'code': 0}}
>Thread-15910330::DEBUG::2013-11-19 21:10:34,168::task::495::TaskManager.Task::(_debug) Task bec9db4b-e505-43ac-88da-3ddee52b18f3: moving from state preparing -> state finished
>Thread-15910330::DEBUG::2013-11-19 21:10:34,169::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910330::DEBUG::2013-11-19 21:10:34,169::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910330::DEBUG::2013-11-19 21:10:34,170::task::495::TaskManager.Task::(_debug) Task bec9db4b-e505-43ac-88da-3ddee52b18f3: ref 0 aborting False
>Thread-15910330::INFO::2013-11-19 21:10:34,170::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000358104705811', 'lastCheck': 1384895433.2918279, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00137400627136', 'lastCheck': 1384895428.079247, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895433.062007, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000414848327637', 'lastCheck': 1384895431.4925799, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:10:43,075::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910336::INFO::2013-11-19 21:10:44,305::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910336::DEBUG::2013-11-19 21:10:44,306::task::495::TaskManager.Task::(_debug) Task 32eae202-3c9d-46d7-aa60-484f22cf6401: moving from state init -> state preparing
>Thread-15910336::DEBUG::2013-11-19 21:10:44,307::task::495::TaskManager.Task::(_debug) Task 32eae202-3c9d-46d7-aa60-484f22cf6401: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000369071960449', 'lastCheck': 1384895443.3044679, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00225591659546', 'lastCheck': 1384895438.0960181, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895443.075911, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000434875488281', 'lastCheck': 1384895441.5043221, 'valid': True, 'code': 0}}
>Thread-15910336::DEBUG::2013-11-19 21:10:44,308::task::495::TaskManager.Task::(_debug) Task 32eae202-3c9d-46d7-aa60-484f22cf6401: moving from state preparing -> state finished
>Thread-15910336::DEBUG::2013-11-19 21:10:44,308::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910336::DEBUG::2013-11-19 21:10:44,309::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910336::DEBUG::2013-11-19 21:10:44,309::task::495::TaskManager.Task::(_debug) Task 32eae202-3c9d-46d7-aa60-484f22cf6401: ref 0 aborting False
>Thread-15910336::INFO::2013-11-19 21:10:44,310::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000369071960449', 'lastCheck': 1384895443.3044679, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00225591659546', 'lastCheck': 1384895438.0960181, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895443.075911, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000434875488281', 'lastCheck': 1384895441.5043221, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:10:53,088::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910342::INFO::2013-11-19 21:10:54,432::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910342::DEBUG::2013-11-19 21:10:54,432::task::495::TaskManager.Task::(_debug) Task d0a3799a-6bab-451e-9b83-2442d2ac378f: moving from state init -> state preparing
>Thread-15910342::DEBUG::2013-11-19 21:10:54,433::task::495::TaskManager.Task::(_debug) Task d0a3799a-6bab-451e-9b83-2442d2ac378f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000319957733154', 'lastCheck': 1384895453.3178811, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00140619277954', 'lastCheck': 1384895448.1113989, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895453.0896139, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000417947769165', 'lastCheck': 1384895451.5159969, 'valid': True, 'code': 0}}
>Thread-15910342::DEBUG::2013-11-19 21:10:54,434::task::495::TaskManager.Task::(_debug) Task d0a3799a-6bab-451e-9b83-2442d2ac378f: moving from state preparing -> state finished
>Thread-15910342::DEBUG::2013-11-19 21:10:54,434::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910342::DEBUG::2013-11-19 21:10:54,435::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910342::DEBUG::2013-11-19 21:10:54,435::task::495::TaskManager.Task::(_debug) Task d0a3799a-6bab-451e-9b83-2442d2ac378f: ref 0 aborting False
>Thread-15910342::INFO::2013-11-19 21:10:54,436::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000319957733154', 'lastCheck': 1384895453.3178811, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00140619277954', 'lastCheck': 1384895448.1113989, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895453.0896139, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000417947769165', 'lastCheck': 1384895451.5159969, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:11:03,102::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910348::INFO::2013-11-19 21:11:04,669::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910348::DEBUG::2013-11-19 21:11:04,669::task::495::TaskManager.Task::(_debug) Task a5ea5607-815a-473a-92bd-cd00e35ee5ec: moving from state init -> state preparing
>Thread-15910348::DEBUG::2013-11-19 21:11:04,670::task::495::TaskManager.Task::(_debug) Task a5ea5607-815a-473a-92bd-cd00e35ee5ec: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000321865081787', 'lastCheck': 1384895463.3308849, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00136303901672', 'lastCheck': 1384895458.126931, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895463.1031461, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000327110290527', 'lastCheck': 1384895461.5276811, 'valid': True, 'code': 0}}
>Thread-15910348::DEBUG::2013-11-19 21:11:04,671::task::495::TaskManager.Task::(_debug) Task a5ea5607-815a-473a-92bd-cd00e35ee5ec: moving from state preparing -> state finished
>Thread-15910348::DEBUG::2013-11-19 21:11:04,671::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910348::DEBUG::2013-11-19 21:11:04,672::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910348::DEBUG::2013-11-19 21:11:04,672::task::495::TaskManager.Task::(_debug) Task a5ea5607-815a-473a-92bd-cd00e35ee5ec: ref 0 aborting False
>Thread-15910348::INFO::2013-11-19 21:11:04,673::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000321865081787', 'lastCheck': 1384895463.3308849, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00136303901672', 'lastCheck': 1384895458.126931, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895463.1031461, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000327110290527', 'lastCheck': 1384895461.5276811, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:11:13,115::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910354::INFO::2013-11-19 21:11:14,825::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910354::DEBUG::2013-11-19 21:11:14,825::task::495::TaskManager.Task::(_debug) Task f48a06fe-2197-4a5f-9282-40347a0a5b94: moving from state init -> state preparing
>Thread-15910354::DEBUG::2013-11-19 21:11:14,826::task::495::TaskManager.Task::(_debug) Task f48a06fe-2197-4a5f-9282-40347a0a5b94: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000309944152832', 'lastCheck': 1384895473.343817, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00147008895874', 'lastCheck': 1384895468.1423869, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895473.1166539, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000341176986694', 'lastCheck': 1384895471.5393701, 'valid': True, 'code': 0}}
>Thread-15910354::DEBUG::2013-11-19 21:11:14,827::task::495::TaskManager.Task::(_debug) Task f48a06fe-2197-4a5f-9282-40347a0a5b94: moving from state preparing -> state finished
>Thread-15910354::DEBUG::2013-11-19 21:11:14,827::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910354::DEBUG::2013-11-19 21:11:14,828::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910354::DEBUG::2013-11-19 21:11:14,828::task::495::TaskManager.Task::(_debug) Task f48a06fe-2197-4a5f-9282-40347a0a5b94: ref 0 aborting False
>Thread-15910354::INFO::2013-11-19 21:11:14,836::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000309944152832', 'lastCheck': 1384895473.343817, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00147008895874', 'lastCheck': 1384895468.1423869, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895473.1166539, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000341176986694', 'lastCheck': 1384895471.5393701, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:11:23,128::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910360::INFO::2013-11-19 21:11:25,071::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910360::DEBUG::2013-11-19 21:11:25,072::task::495::TaskManager.Task::(_debug) Task c3c39101-5f46-4ae8-b9be-71242af0bbbf: moving from state init -> state preparing
>Thread-15910360::DEBUG::2013-11-19 21:11:25,072::task::495::TaskManager.Task::(_debug) Task c3c39101-5f46-4ae8-b9be-71242af0bbbf: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000325918197632', 'lastCheck': 1384895483.3567221, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00137090682983', 'lastCheck': 1384895478.158097, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895483.1291201, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000318050384521', 'lastCheck': 1384895481.5509479, 'valid': True, 'code': 0}}
>Thread-15910360::DEBUG::2013-11-19 21:11:25,073::task::495::TaskManager.Task::(_debug) Task c3c39101-5f46-4ae8-b9be-71242af0bbbf: moving from state preparing -> state finished
>Thread-15910360::DEBUG::2013-11-19 21:11:25,074::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910360::DEBUG::2013-11-19 21:11:25,074::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910360::DEBUG::2013-11-19 21:11:25,075::task::495::TaskManager.Task::(_debug) Task c3c39101-5f46-4ae8-b9be-71242af0bbbf: ref 0 aborting False
>Thread-15910360::INFO::2013-11-19 21:11:25,075::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000325918197632', 'lastCheck': 1384895483.3567221, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00137090682983', 'lastCheck': 1384895478.158097, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895483.1291201, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000318050384521', 'lastCheck': 1384895481.5509479, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:11:33,142::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910366::INFO::2013-11-19 21:11:35,189::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910366::DEBUG::2013-11-19 21:11:35,190::task::495::TaskManager.Task::(_debug) Task eb2365a4-a243-40a7-a2dc-ed2e411ecd5b: moving from state init -> state preparing
>Thread-15910366::DEBUG::2013-11-19 21:11:35,191::task::495::TaskManager.Task::(_debug) Task eb2365a4-a243-40a7-a2dc-ed2e411ecd5b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000344038009644', 'lastCheck': 1384895493.3696339, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00138211250305', 'lastCheck': 1384895488.17326, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895493.142915, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000350952148438', 'lastCheck': 1384895491.562614, 'valid': True, 'code': 0}}
>Thread-15910366::DEBUG::2013-11-19 21:11:35,191::task::495::TaskManager.Task::(_debug) Task eb2365a4-a243-40a7-a2dc-ed2e411ecd5b: moving from state preparing -> state finished
>Thread-15910366::DEBUG::2013-11-19 21:11:35,192::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910366::DEBUG::2013-11-19 21:11:35,193::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910366::DEBUG::2013-11-19 21:11:35,193::task::495::TaskManager.Task::(_debug) Task eb2365a4-a243-40a7-a2dc-ed2e411ecd5b: ref 0 aborting False
>Thread-15910366::INFO::2013-11-19 21:11:35,194::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000344038009644', 'lastCheck': 1384895493.3696339, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00138211250305', 'lastCheck': 1384895488.17326, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895493.142915, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000350952148438', 'lastCheck': 1384895491.562614, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:11:43,155::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910372::INFO::2013-11-19 21:11:45,334::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910372::DEBUG::2013-11-19 21:11:45,334::task::495::TaskManager.Task::(_debug) Task ef3f0ed1-f27b-4979-8da1-f5be3b56ddda: moving from state init -> state preparing
>Thread-15910372::DEBUG::2013-11-19 21:11:45,335::task::495::TaskManager.Task::(_debug) Task ef3f0ed1-f27b-4979-8da1-f5be3b56ddda: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000320911407471', 'lastCheck': 1384895503.3827341, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00226998329163', 'lastCheck': 1384895498.1895151, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895503.156425, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000459909439087', 'lastCheck': 1384895501.5744121, 'valid': True, 'code': 0}}
>Thread-15910372::DEBUG::2013-11-19 21:11:45,336::task::495::TaskManager.Task::(_debug) Task ef3f0ed1-f27b-4979-8da1-f5be3b56ddda: moving from state preparing -> state finished
>Thread-15910372::DEBUG::2013-11-19 21:11:45,336::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910372::DEBUG::2013-11-19 21:11:45,337::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910372::DEBUG::2013-11-19 21:11:45,337::task::495::TaskManager.Task::(_debug) Task ef3f0ed1-f27b-4979-8da1-f5be3b56ddda: ref 0 aborting False
>Thread-15910372::INFO::2013-11-19 21:11:45,338::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000320911407471', 'lastCheck': 1384895503.3827341, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00226998329163', 'lastCheck': 1384895498.1895151, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895503.156425, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000459909439087', 'lastCheck': 1384895501.5744121, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:11:53,168::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910378::INFO::2013-11-19 21:11:55,459::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910378::DEBUG::2013-11-19 21:11:55,459::task::495::TaskManager.Task::(_debug) Task 2753572b-2b91-4b35-8272-ad5c6f214e04: moving from state init -> state preparing
>Thread-15910378::DEBUG::2013-11-19 21:11:55,460::task::495::TaskManager.Task::(_debug) Task 2753572b-2b91-4b35-8272-ad5c6f214e04: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000318050384521', 'lastCheck': 1384895513.395658, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00134205818176', 'lastCheck': 1384895508.2047901, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895513.169647, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000324010848999', 'lastCheck': 1384895511.585988, 'valid': True, 'code': 0}}
>Thread-15910378::DEBUG::2013-11-19 21:11:55,461::task::495::TaskManager.Task::(_debug) Task 2753572b-2b91-4b35-8272-ad5c6f214e04: moving from state preparing -> state finished
>Thread-15910378::DEBUG::2013-11-19 21:11:55,461::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910378::DEBUG::2013-11-19 21:11:55,462::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910378::DEBUG::2013-11-19 21:11:55,462::task::495::TaskManager.Task::(_debug) Task 2753572b-2b91-4b35-8272-ad5c6f214e04: ref 0 aborting False
>Thread-15910378::INFO::2013-11-19 21:11:55,463::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000318050384521', 'lastCheck': 1384895513.395658, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00134205818176', 'lastCheck': 1384895508.2047901, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895513.169647, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000324010848999', 'lastCheck': 1384895511.585988, 'valid': True, 'code': 0}}
>Thread-42840::DEBUG::2013-11-19 21:12:01,597::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-19 21:12:02,214::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:12:03,181::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910384::INFO::2013-11-19 21:12:05,626::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910384::DEBUG::2013-11-19 21:12:05,627::task::495::TaskManager.Task::(_debug) Task 1f943a22-502b-4eb5-9969-a304790c98b9: moving from state init -> state preparing
>Thread-15910384::DEBUG::2013-11-19 21:12:05,628::task::495::TaskManager.Task::(_debug) Task 1f943a22-502b-4eb5-9969-a304790c98b9: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00042200088501', 'lastCheck': 1384895523.4093571, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00135207176208', 'lastCheck': 1384895518.2201409, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895523.1826639, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000316143035889', 'lastCheck': 1384895522.2175879, 'valid': True, 'code': 0}}
>Thread-15910384::DEBUG::2013-11-19 21:12:05,629::task::495::TaskManager.Task::(_debug) Task 1f943a22-502b-4eb5-9969-a304790c98b9: moving from state preparing -> state finished
>Thread-15910384::DEBUG::2013-11-19 21:12:05,629::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910384::DEBUG::2013-11-19 21:12:05,630::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910384::DEBUG::2013-11-19 21:12:05,630::task::495::TaskManager.Task::(_debug) Task 1f943a22-502b-4eb5-9969-a304790c98b9: ref 0 aborting False
>Thread-15910384::INFO::2013-11-19 21:12:05,631::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00042200088501', 'lastCheck': 1384895523.4093571, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00135207176208', 'lastCheck': 1384895518.2201409, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895523.1826639, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000316143035889', 'lastCheck': 1384895522.2175879, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:12:13,195::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910390::INFO::2013-11-19 21:12:15,797::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910390::DEBUG::2013-11-19 21:12:15,797::task::495::TaskManager.Task::(_debug) Task e3e71470-c449-49c8-a8d4-71616dc5ded7: moving from state init -> state preparing
>Thread-15910390::DEBUG::2013-11-19 21:12:15,798::task::495::TaskManager.Task::(_debug) Task e3e71470-c449-49c8-a8d4-71616dc5ded7: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000313997268677', 'lastCheck': 1384895533.4224761, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00152707099915', 'lastCheck': 1384895528.2353671, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895533.196115, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000421047210693', 'lastCheck': 1384895532.229032, 'valid': True, 'code': 0}}
>Thread-15910390::DEBUG::2013-11-19 21:12:15,799::task::495::TaskManager.Task::(_debug) Task e3e71470-c449-49c8-a8d4-71616dc5ded7: moving from state preparing -> state finished
>Thread-15910390::DEBUG::2013-11-19 21:12:15,799::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910390::DEBUG::2013-11-19 21:12:15,800::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910390::DEBUG::2013-11-19 21:12:15,800::task::495::TaskManager.Task::(_debug) Task e3e71470-c449-49c8-a8d4-71616dc5ded7: ref 0 aborting False
>Thread-15910390::INFO::2013-11-19 21:12:15,805::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000313997268677', 'lastCheck': 1384895533.4224761, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00152707099915', 'lastCheck': 1384895528.2353671, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895533.196115, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000421047210693', 'lastCheck': 1384895532.229032, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:12:23,208::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910396::INFO::2013-11-19 21:12:25,984::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910396::DEBUG::2013-11-19 21:12:25,985::task::495::TaskManager.Task::(_debug) Task 1192731e-a6c8-48ab-9dd9-8b661e651d49: moving from state init -> state preparing
>Thread-15910396::DEBUG::2013-11-19 21:12:25,985::task::495::TaskManager.Task::(_debug) Task 1192731e-a6c8-48ab-9dd9-8b661e651d49: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000319004058838', 'lastCheck': 1384895543.4355631, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00137305259705', 'lastCheck': 1384895538.2509029, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895543.2094221, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000339984893799', 'lastCheck': 1384895542.24038, 'valid': True, 'code': 0}}
>Thread-15910396::DEBUG::2013-11-19 21:12:25,986::task::495::TaskManager.Task::(_debug) Task 1192731e-a6c8-48ab-9dd9-8b661e651d49: moving from state preparing -> state finished
>Thread-15910396::DEBUG::2013-11-19 21:12:25,987::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910396::DEBUG::2013-11-19 21:12:25,987::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910396::DEBUG::2013-11-19 21:12:25,988::task::495::TaskManager.Task::(_debug) Task 1192731e-a6c8-48ab-9dd9-8b661e651d49: ref 0 aborting False
>Thread-15910396::INFO::2013-11-19 21:12:25,992::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000319004058838', 'lastCheck': 1384895543.4355631, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00137305259705', 'lastCheck': 1384895538.2509029, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895543.2094221, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000339984893799', 'lastCheck': 1384895542.24038, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:12:33,222::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910402::INFO::2013-11-19 21:12:36,122::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910402::DEBUG::2013-11-19 21:12:36,123::task::495::TaskManager.Task::(_debug) Task b413f516-df14-41f8-b476-45dcc838844c: moving from state init -> state preparing
>Thread-15910402::DEBUG::2013-11-19 21:12:36,124::task::495::TaskManager.Task::(_debug) Task b413f516-df14-41f8-b476-45dcc838844c: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000382900238037', 'lastCheck': 1384895553.4489319, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00140404701233', 'lastCheck': 1384895548.2661879, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895553.222811, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000334024429321', 'lastCheck': 1384895552.251694, 'valid': True, 'code': 0}}
>Thread-15910402::DEBUG::2013-11-19 21:12:36,124::task::495::TaskManager.Task::(_debug) Task b413f516-df14-41f8-b476-45dcc838844c: moving from state preparing -> state finished
>Thread-15910402::DEBUG::2013-11-19 21:12:36,125::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910402::DEBUG::2013-11-19 21:12:36,126::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910402::DEBUG::2013-11-19 21:12:36,126::task::495::TaskManager.Task::(_debug) Task b413f516-df14-41f8-b476-45dcc838844c: ref 0 aborting False
>Thread-15910402::INFO::2013-11-19 21:12:36,127::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000382900238037', 'lastCheck': 1384895553.4489319, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00140404701233', 'lastCheck': 1384895548.2661879, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895553.222811, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000334024429321', 'lastCheck': 1384895552.251694, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:12:43,235::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910408::INFO::2013-11-19 21:12:46,238::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910408::DEBUG::2013-11-19 21:12:46,238::task::495::TaskManager.Task::(_debug) Task d8e2ed60-cdf7-4cc6-a4ba-991e77cb8453: moving from state init -> state preparing
>Thread-15910408::DEBUG::2013-11-19 21:12:46,239::task::495::TaskManager.Task::(_debug) Task d8e2ed60-cdf7-4cc6-a4ba-991e77cb8453: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000336885452271', 'lastCheck': 1384895563.4620769, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00220012664795', 'lastCheck': 1384895558.282846, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895563.2362289, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000450134277344', 'lastCheck': 1384895562.263191, 'valid': True, 'code': 0}}
>Thread-15910408::DEBUG::2013-11-19 21:12:46,240::task::495::TaskManager.Task::(_debug) Task d8e2ed60-cdf7-4cc6-a4ba-991e77cb8453: moving from state preparing -> state finished
>Thread-15910408::DEBUG::2013-11-19 21:12:46,241::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910408::DEBUG::2013-11-19 21:12:46,241::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910408::DEBUG::2013-11-19 21:12:46,242::task::495::TaskManager.Task::(_debug) Task d8e2ed60-cdf7-4cc6-a4ba-991e77cb8453: ref 0 aborting False
>Thread-15910408::INFO::2013-11-19 21:12:46,242::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000336885452271', 'lastCheck': 1384895563.4620769, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00220012664795', 'lastCheck': 1384895558.282846, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895563.2362289, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000450134277344', 'lastCheck': 1384895562.263191, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:12:53,249::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910414::INFO::2013-11-19 21:12:56,349::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910414::DEBUG::2013-11-19 21:12:56,350::task::495::TaskManager.Task::(_debug) Task 9ea9a0c5-d016-4499-ad81-6ab25a38a53b: moving from state init -> state preparing
>Thread-15910414::DEBUG::2013-11-19 21:12:56,351::task::495::TaskManager.Task::(_debug) Task 9ea9a0c5-d016-4499-ad81-6ab25a38a53b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000330924987793', 'lastCheck': 1384895573.475178, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00134682655334', 'lastCheck': 1384895568.2990451, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895573.2495561, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000304937362671', 'lastCheck': 1384895572.274518, 'valid': True, 'code': 0}}
>Thread-15910414::DEBUG::2013-11-19 21:12:56,351::task::495::TaskManager.Task::(_debug) Task 9ea9a0c5-d016-4499-ad81-6ab25a38a53b: moving from state preparing -> state finished
>Thread-15910414::DEBUG::2013-11-19 21:12:56,352::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910414::DEBUG::2013-11-19 21:12:56,352::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910414::DEBUG::2013-11-19 21:12:56,353::task::495::TaskManager.Task::(_debug) Task 9ea9a0c5-d016-4499-ad81-6ab25a38a53b: ref 0 aborting False
>Thread-15910414::INFO::2013-11-19 21:12:56,354::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000330924987793', 'lastCheck': 1384895573.475178, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00134682655334', 'lastCheck': 1384895568.2990451, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895573.2495561, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000304937362671', 'lastCheck': 1384895572.274518, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:13:03,262::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910420::INFO::2013-11-19 21:13:06,488::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910420::DEBUG::2013-11-19 21:13:06,489::task::495::TaskManager.Task::(_debug) Task e28b579c-e1fe-455d-8bab-128df95d5243: moving from state init -> state preparing
>Thread-15910420::DEBUG::2013-11-19 21:13:06,490::task::495::TaskManager.Task::(_debug) Task e28b579c-e1fe-455d-8bab-128df95d5243: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000352144241333', 'lastCheck': 1384895583.4882181, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00139284133911', 'lastCheck': 1384895578.314255, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895583.2632959, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000420093536377', 'lastCheck': 1384895582.2859011, 'valid': True, 'code': 0}}
>Thread-15910420::DEBUG::2013-11-19 21:13:06,492::task::495::TaskManager.Task::(_debug) Task e28b579c-e1fe-455d-8bab-128df95d5243: moving from state preparing -> state finished
>Thread-15910420::DEBUG::2013-11-19 21:13:06,492::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910420::DEBUG::2013-11-19 21:13:06,493::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910420::DEBUG::2013-11-19 21:13:06,494::task::495::TaskManager.Task::(_debug) Task e28b579c-e1fe-455d-8bab-128df95d5243: ref 0 aborting False
>Thread-15910420::INFO::2013-11-19 21:13:06,496::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000352144241333', 'lastCheck': 1384895583.4882181, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00139284133911', 'lastCheck': 1384895578.314255, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895583.2632959, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000420093536377', 'lastCheck': 1384895582.2859011, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:13:13,275::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910426::INFO::2013-11-19 21:13:16,583::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910426::DEBUG::2013-11-19 21:13:16,584::task::495::TaskManager.Task::(_debug) Task fff5f27d-7fdf-46b7-b996-bf9a751a82d4: moving from state init -> state preparing
>Thread-15910426::DEBUG::2013-11-19 21:13:16,585::task::495::TaskManager.Task::(_debug) Task fff5f27d-7fdf-46b7-b996-bf9a751a82d4: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000400066375732', 'lastCheck': 1384895593.501384, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00136494636536', 'lastCheck': 1384895588.329561, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895593.276644, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000404834747314', 'lastCheck': 1384895592.297287, 'valid': True, 'code': 0}}
>Thread-15910426::DEBUG::2013-11-19 21:13:16,586::task::495::TaskManager.Task::(_debug) Task fff5f27d-7fdf-46b7-b996-bf9a751a82d4: moving from state preparing -> state finished
>Thread-15910426::DEBUG::2013-11-19 21:13:16,586::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910426::DEBUG::2013-11-19 21:13:16,587::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910426::DEBUG::2013-11-19 21:13:16,587::task::495::TaskManager.Task::(_debug) Task fff5f27d-7fdf-46b7-b996-bf9a751a82d4: ref 0 aborting False
>Thread-15910426::INFO::2013-11-19 21:13:16,588::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000400066375732', 'lastCheck': 1384895593.501384, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00136494636536', 'lastCheck': 1384895588.329561, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895593.276644, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000404834747314', 'lastCheck': 1384895592.297287, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:13:23,289::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910432::INFO::2013-11-19 21:13:26,733::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910432::DEBUG::2013-11-19 21:13:26,734::task::495::TaskManager.Task::(_debug) Task 9e87bf66-b718-46e9-990f-3326e938cd8d: moving from state init -> state preparing
>Thread-15910432::DEBUG::2013-11-19 21:13:26,735::task::495::TaskManager.Task::(_debug) Task 9e87bf66-b718-46e9-990f-3326e938cd8d: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000308990478516', 'lastCheck': 1384895603.514235, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00139784812927', 'lastCheck': 1384895598.345098, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895603.290184, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000298976898193', 'lastCheck': 1384895602.308568, 'valid': True, 'code': 0}}
>Thread-15910432::DEBUG::2013-11-19 21:13:26,735::task::495::TaskManager.Task::(_debug) Task 9e87bf66-b718-46e9-990f-3326e938cd8d: moving from state preparing -> state finished
>Thread-15910432::DEBUG::2013-11-19 21:13:26,736::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910432::DEBUG::2013-11-19 21:13:26,737::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910432::DEBUG::2013-11-19 21:13:26,737::task::495::TaskManager.Task::(_debug) Task 9e87bf66-b718-46e9-990f-3326e938cd8d: ref 0 aborting False
>Thread-15910432::INFO::2013-11-19 21:13:26,738::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000308990478516', 'lastCheck': 1384895603.514235, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00139784812927', 'lastCheck': 1384895598.345098, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895603.290184, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000298976898193', 'lastCheck': 1384895602.308568, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:13:33,302::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910438::INFO::2013-11-19 21:13:36,820::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910438::DEBUG::2013-11-19 21:13:36,821::task::495::TaskManager.Task::(_debug) Task 6db58ab8-804a-41f7-832e-0758b2d19bdb: moving from state init -> state preparing
>Thread-15910438::DEBUG::2013-11-19 21:13:36,822::task::495::TaskManager.Task::(_debug) Task 6db58ab8-804a-41f7-832e-0758b2d19bdb: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000391960144043', 'lastCheck': 1384895613.5273261, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00134181976318', 'lastCheck': 1384895608.36025, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895613.3036339, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000415086746216', 'lastCheck': 1384895612.3199761, 'valid': True, 'code': 0}}
>Thread-15910438::DEBUG::2013-11-19 21:13:36,823::task::495::TaskManager.Task::(_debug) Task 6db58ab8-804a-41f7-832e-0758b2d19bdb: moving from state preparing -> state finished
>Thread-15910438::DEBUG::2013-11-19 21:13:36,823::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910438::DEBUG::2013-11-19 21:13:36,824::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910438::DEBUG::2013-11-19 21:13:36,824::task::495::TaskManager.Task::(_debug) Task 6db58ab8-804a-41f7-832e-0758b2d19bdb: ref 0 aborting False
>Thread-15910438::INFO::2013-11-19 21:13:36,825::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000391960144043', 'lastCheck': 1384895613.5273261, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00134181976318', 'lastCheck': 1384895608.36025, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895613.3036339, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000415086746216', 'lastCheck': 1384895612.3199761, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:13:43,316::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910444::INFO::2013-11-19 21:13:46,954::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910444::DEBUG::2013-11-19 21:13:46,955::task::495::TaskManager.Task::(_debug) Task 1ae61d88-0bee-4aaa-b98d-eb696c39a390: moving from state init -> state preparing
>Thread-15910444::DEBUG::2013-11-19 21:13:46,956::task::495::TaskManager.Task::(_debug) Task 1ae61d88-0bee-4aaa-b98d-eb696c39a390: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000324010848999', 'lastCheck': 1384895623.540354, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00223207473755', 'lastCheck': 1384895618.3769441, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895623.3171301, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000436067581177', 'lastCheck': 1384895622.331425, 'valid': True, 'code': 0}}
>Thread-15910444::DEBUG::2013-11-19 21:13:46,957::task::495::TaskManager.Task::(_debug) Task 1ae61d88-0bee-4aaa-b98d-eb696c39a390: moving from state preparing -> state finished
>Thread-15910444::DEBUG::2013-11-19 21:13:46,957::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910444::DEBUG::2013-11-19 21:13:46,958::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910444::DEBUG::2013-11-19 21:13:46,958::task::495::TaskManager.Task::(_debug) Task 1ae61d88-0bee-4aaa-b98d-eb696c39a390: ref 0 aborting False
>Thread-15910444::INFO::2013-11-19 21:13:46,959::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000324010848999', 'lastCheck': 1384895623.540354, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00223207473755', 'lastCheck': 1384895618.3769441, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895623.3171301, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000436067581177', 'lastCheck': 1384895622.331425, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:13:53,329::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910450::INFO::2013-11-19 21:13:57,086::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910450::DEBUG::2013-11-19 21:13:57,086::task::495::TaskManager.Task::(_debug) Task 0ae757a5-c2c1-4c74-9767-47ee2783777e: moving from state init -> state preparing
>Thread-15910450::DEBUG::2013-11-19 21:13:57,087::task::495::TaskManager.Task::(_debug) Task 0ae757a5-c2c1-4c74-9767-47ee2783777e: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000345945358276', 'lastCheck': 1384895633.553391, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142908096313', 'lastCheck': 1384895628.3922441, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895633.3305371, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000321865081787', 'lastCheck': 1384895632.342752, 'valid': True, 'code': 0}}
>Thread-15910450::DEBUG::2013-11-19 21:13:57,088::task::495::TaskManager.Task::(_debug) Task 0ae757a5-c2c1-4c74-9767-47ee2783777e: moving from state preparing -> state finished
>Thread-15910450::DEBUG::2013-11-19 21:13:57,089::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910450::DEBUG::2013-11-19 21:13:57,089::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910450::DEBUG::2013-11-19 21:13:57,090::task::495::TaskManager.Task::(_debug) Task 0ae757a5-c2c1-4c74-9767-47ee2783777e: ref 0 aborting False
>Thread-15910450::INFO::2013-11-19 21:13:57,091::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000345945358276', 'lastCheck': 1384895633.553391, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142908096313', 'lastCheck': 1384895628.3922441, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895633.3305371, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000321865081787', 'lastCheck': 1384895632.342752, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:14:03,343::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910456::INFO::2013-11-19 21:14:07,193::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910456::DEBUG::2013-11-19 21:14:07,194::task::495::TaskManager.Task::(_debug) Task b88d54c4-c7d0-4208-b83b-0de4e0ffd361: moving from state init -> state preparing
>Thread-15910456::DEBUG::2013-11-19 21:14:07,195::task::495::TaskManager.Task::(_debug) Task b88d54c4-c7d0-4208-b83b-0de4e0ffd361: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00037693977356', 'lastCheck': 1384895643.5665641, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00138711929321', 'lastCheck': 1384895638.4074831, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895643.3443849, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000458955764771', 'lastCheck': 1384895642.3542509, 'valid': True, 'code': 0}}
>Thread-15910456::DEBUG::2013-11-19 21:14:07,195::task::495::TaskManager.Task::(_debug) Task b88d54c4-c7d0-4208-b83b-0de4e0ffd361: moving from state preparing -> state finished
>Thread-15910456::DEBUG::2013-11-19 21:14:07,196::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910456::DEBUG::2013-11-19 21:14:07,197::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910456::DEBUG::2013-11-19 21:14:07,197::task::495::TaskManager.Task::(_debug) Task b88d54c4-c7d0-4208-b83b-0de4e0ffd361: ref 0 aborting False
>Thread-15910456::INFO::2013-11-19 21:14:07,198::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00037693977356', 'lastCheck': 1384895643.5665641, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00138711929321', 'lastCheck': 1384895638.4074831, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895643.3443849, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000458955764771', 'lastCheck': 1384895642.3542509, 'valid': True, 'code': 0}}
>Thread-15910458::DEBUG::2013-11-19 21:14:12,764::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910458::INFO::2013-11-19 21:14:12,765::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910458::DEBUG::2013-11-19 21:14:12,775::task::495::TaskManager.Task::(_debug) Task 9715bbe1-1522-4c9f-8e63-2c2beffe8d45: moving from state init -> state preparing
>Thread-15910458::DEBUG::2013-11-19 21:14:12,776::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=Free spmLver=274 spmId=-1
>Thread-15910458::DEBUG::2013-11-19 21:14:12,776::task::495::TaskManager.Task::(_debug) Task 9715bbe1-1522-4c9f-8e63-2c2beffe8d45: finished: {'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': 274}}
>Thread-15910458::DEBUG::2013-11-19 21:14:12,777::task::495::TaskManager.Task::(_debug) Task 9715bbe1-1522-4c9f-8e63-2c2beffe8d45: moving from state preparing -> state finished
>Thread-15910458::DEBUG::2013-11-19 21:14:12,778::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910458::DEBUG::2013-11-19 21:14:12,779::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910458::DEBUG::2013-11-19 21:14:12,779::task::495::TaskManager.Task::(_debug) Task 9715bbe1-1522-4c9f-8e63-2c2beffe8d45: ref 0 aborting False
>Thread-15910458::INFO::2013-11-19 21:14:12,780::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': 274}}
>Thread-15910459::DEBUG::2013-11-19 21:14:12,809::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910459::INFO::2013-11-19 21:14:12,812::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: spmStart, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 prevID=-1 prevLVER=274 recoveryMode=0 scsiFencing=false maxHostID=250 domVersion=2)
>Thread-15910459::DEBUG::2013-11-19 21:14:12,813::task::495::TaskManager.Task::(_debug) Task 88cc2f68-e802-4615-bced-c7a8c4aefa33: moving from state init -> state preparing
>Thread-15910459::DEBUG::2013-11-19 21:14:12,814::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5b6b8e38-b3df-4bd9-a75b-bba6ca4b10ea`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '330' at 'public_spmStart'
>Thread-15910459::DEBUG::2013-11-19 21:14:12,815::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'exclusive'
>Thread-15910459::DEBUG::2013-11-19 21:14:12,815::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'exclusive' (1 active user)
>Thread-15910459::DEBUG::2013-11-19 21:14:12,816::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5b6b8e38-b3df-4bd9-a75b-bba6ca4b10ea`::Granted request
>Thread-15910459::DEBUG::2013-11-19 21:14:12,817::task::495::TaskManager.Task::(_debug) Task 88cc2f68-e802-4615-bced-c7a8c4aefa33: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (exclusive)
>Thread-15910459::DEBUG::2013-11-19 21:14:12,817::task::495::TaskManager.Task::(_debug) Task 88cc2f68-e802-4615-bced-c7a8c4aefa33: ref 1 aborting False
>Thread-15910459::DEBUG::2013-11-19 21:14:12,818::task::495::TaskManager.Task::(_debug) Task 88cc2f68-e802-4615-bced-c7a8c4aefa33: Prepare: 1 jobs exist, move to acquiring
>Thread-15910459::DEBUG::2013-11-19 21:14:12,819::task::495::TaskManager.Task::(_debug) Task 88cc2f68-e802-4615-bced-c7a8c4aefa33: moving from state preparing -> state acquiring
>Thread-15910459::DEBUG::2013-11-19 21:14:12,819::task::495::TaskManager.Task::(_debug) Task 88cc2f68-e802-4615-bced-c7a8c4aefa33: moving from state acquiring -> state queued
>Thread-15910459::DEBUG::2013-11-19 21:14:12,820::taskManager::37::TaskManager::(_queueTask) queueing task: 88cc2f68-e802-4615-bced-c7a8c4aefa33
>Thread-15910459::DEBUG::2013-11-19 21:14:12,820::taskManager::43::TaskManager::(_queueTask) task queued: 88cc2f68-e802-4615-bced-c7a8c4aefa33
>57fe3c04-4010-4c2b-b99c-03efcc17c092::DEBUG::2013-11-19 21:14:12,821::threadPool::47::Misc.ThreadPool::(setRunningTask) Number of running tasks: 1
>Thread-15910459::DEBUG::2013-11-19 21:14:12,821::task::1183::TaskManager.Task::(prepare) returning
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:12,822::threadPool::192::Misc.ThreadPool.WorkerThread::(run) Task: 88cc2f68-e802-4615-bced-c7a8c4aefa33 running: <bound method Task.commit of <storage.task.Task instance at 0x7f487c13ad40>> with: None
>Thread-15910459::DEBUG::2013-11-19 21:14:12,823::task::495::TaskManager.Task::(_debug) Task 88cc2f68-e802-4615-bced-c7a8c4aefa33: ref 0 aborting False
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:12,823::task::1195::TaskManager.Task::(commit) Commiting task: 88cc2f68-e802-4615-bced-c7a8c4aefa33
>Thread-15910459::INFO::2013-11-19 21:14:12,824::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: spmStart, Return response: {'status': {'message': 'OK', 'code': 0}, 'uuid': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:12,825::task::495::TaskManager.Task::(_debug) Task 88cc2f68-e802-4615-bced-c7a8c4aefa33: moving from state queued -> state running
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:12,828::task::907::TaskManager.Task::(_runJobs) Task.run: running job 0: spmStart: <bound method SPM.start of <storage.spm.SPM instance at 0x1bf9f38>> (args: ('c4e5889a-348d-11e2-83fe-9f72369eab30', -1, '274', 0, 'false', 250, 2) kwargs: {})
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:12,829::task::300::TaskManager.Task::(run) Job.run: running spmStart: <bound method SPM.start of <storage.spm.SPM instance at 0x1bf9f38>> (args: ('c4e5889a-348d-11e2-83fe-9f72369eab30', -1, '274', 0, 'false', 250, 2) kwargs: {}) callback None
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:12,830::spm::433::Storage.SPM::(start) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:12,830::misc::1010::SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads)
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:12,831::misc::1012::SamplingMethod::(__call__) Got in to sampling method
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:12,831::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:12,832::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:12,833::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:12,833::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:12,834::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:12,835::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:14:13,358::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:13,548::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:13,551::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:13,552::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_LVER=276', u'POOL_SPM_ID=-1', u'_SHA_CKSUM=d1ce634f167d7f0ddd58cf2ebbdf434d45f6ef67']
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:13,553::misc::1020::SamplingMethod::(__call__) Returning last result
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:13,554::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:13,555::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:13,555::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:13,556::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:13,557::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:13,558::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 21:14:13,775::lvm::439::OperationMutex::(_reloadvgs) Got the operational mutex
>Thread-42838::DEBUG::2013-11-19 21:14:13,781::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15910460::DEBUG::2013-11-19 21:14:14,130::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910460::INFO::2013-11-19 21:14:14,133::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=88cc2f68-e802-4615-bced-c7a8c4aefa33)
>Thread-15910460::DEBUG::2013-11-19 21:14:14,133::task::495::TaskManager.Task::(_debug) Task c64690c8-6eb8-444a-ac09-334f327564a6: moving from state init -> state preparing
>Thread-15910460::DEBUG::2013-11-19 21:14:14,134::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 88cc2f68-e802-4615-bced-c7a8c4aefa33
>Thread-15910460::DEBUG::2013-11-19 21:14:14,135::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}
>Thread-15910460::DEBUG::2013-11-19 21:14:14,136::task::495::TaskManager.Task::(_debug) Task c64690c8-6eb8-444a-ac09-334f327564a6: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910460::DEBUG::2013-11-19 21:14:14,136::task::495::TaskManager.Task::(_debug) Task c64690c8-6eb8-444a-ac09-334f327564a6: moving from state preparing -> state finished
>Thread-15910460::DEBUG::2013-11-19 21:14:14,137::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910460::DEBUG::2013-11-19 21:14:14,137::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910460::DEBUG::2013-11-19 21:14:14,138::task::495::TaskManager.Task::(_debug) Task c64690c8-6eb8-444a-ac09-334f327564a6: ref 0 aborting False
>Thread-15910460::INFO::2013-11-19 21:14:14,139::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:14,333::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:14,336::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_LVER=276', u'POOL_SPM_ID=-1', u'_SHA_CKSUM=d1ce634f167d7f0ddd58cf2ebbdf434d45f6ef67']
>88cc2f68-e802-4615-bced-c7a8c4aefa33::INFO::2013-11-19 21:14:14,338::spm::457::Storage.SPM::(start) expected previd:-1 lver:276 got request for previd:-1 lver:274
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:14,339::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm reload operation' is holding the operation mutex, waiting...
>Thread-42838::DEBUG::2013-11-19 21:14:14,567::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-42838::DEBUG::2013-11-19 21:14:14,570::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:14,571::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:14,572::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:14,574::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:14,576::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:14,576::persistentDict::163::Storage.PersistentDict::(__init__) Created a persistant dict with LvMetadataRW backend
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:14,577::blockSD::164::Storage.Misc.excCmd::(readlines) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:14,973::blockSD::164::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000205295 s, 10.0 MB/s\n'; <rc> = 0
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:14,976::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000205295 s, 10.0 MB/s'], size: 2048
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:14,977::persistentDict::204::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[]
>88cc2f68-e802-4615-bced-c7a8c4aefa33::WARNING::2013-11-19 21:14:14,977::persistentDict::226::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:14,978::persistentDict::163::Storage.PersistentDict::(__init__) Created a persistant dict with VGTagMetadataRW backend
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:14,979::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:14,979::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:14,980::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:14,981::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:14,981::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:14,983::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15910461::DEBUG::2013-11-19 21:14:15,188::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910461::INFO::2013-11-19 21:14:15,191::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=88cc2f68-e802-4615-bced-c7a8c4aefa33)
>Thread-15910461::DEBUG::2013-11-19 21:14:15,192::task::495::TaskManager.Task::(_debug) Task f10b365d-7645-4780-baec-05ea83407463: moving from state init -> state preparing
>Thread-15910461::DEBUG::2013-11-19 21:14:15,193::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 88cc2f68-e802-4615-bced-c7a8c4aefa33
>Thread-15910461::DEBUG::2013-11-19 21:14:15,193::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}
>Thread-15910461::DEBUG::2013-11-19 21:14:15,194::task::495::TaskManager.Task::(_debug) Task f10b365d-7645-4780-baec-05ea83407463: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910461::DEBUG::2013-11-19 21:14:15,195::task::495::TaskManager.Task::(_debug) Task f10b365d-7645-4780-baec-05ea83407463: moving from state preparing -> state finished
>Thread-15910461::DEBUG::2013-11-19 21:14:15,196::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910461::DEBUG::2013-11-19 21:14:15,197::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910461::DEBUG::2013-11-19 21:14:15,198::task::495::TaskManager.Task::(_debug) Task f10b365d-7645-4780-baec-05ea83407463: ref 0 aborting False
>Thread-15910461::INFO::2013-11-19 21:14:15,199::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:15,638::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:15,641::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:15,642::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_LVER=276', u'POOL_SPM_ID=-1', u'_SHA_CKSUM=d1ce634f167d7f0ddd58cf2ebbdf434d45f6ef67']
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:15,643::safelease::52::ClusterLock::(acquire) Acquiring cluster lock for domain 6b892067-ac7e-4039-a7be-5b7387525b16
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:15,644::safelease::61::Storage.Misc.excCmd::(acquire) '/usr/bin/sudo -n /usr/bin/setsid /usr/bin/ionice -c1 -n0 /bin/su vdsm -s /bin/sh -c "/usr/libexec/vdsm/spmprotect.sh start 6b892067-ac7e-4039-a7be-5b7387525b16 2 5 /dev/6b892067-ac7e-4039-a7be-5b7387525b16/leases 60000 10000 3"' (cwd /usr/libexec/vdsm/)
>Thread-15910462::DEBUG::2013-11-19 21:14:16,223::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910462::INFO::2013-11-19 21:14:16,225::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=88cc2f68-e802-4615-bced-c7a8c4aefa33)
>Thread-15910462::DEBUG::2013-11-19 21:14:16,226::task::495::TaskManager.Task::(_debug) Task 9aa45c3d-79c7-451a-ad8a-018cb1eb5810: moving from state init -> state preparing
>Thread-15910462::DEBUG::2013-11-19 21:14:16,226::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 88cc2f68-e802-4615-bced-c7a8c4aefa33
>Thread-15910462::DEBUG::2013-11-19 21:14:16,227::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}
>Thread-15910462::DEBUG::2013-11-19 21:14:16,227::task::495::TaskManager.Task::(_debug) Task 9aa45c3d-79c7-451a-ad8a-018cb1eb5810: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910462::DEBUG::2013-11-19 21:14:16,228::task::495::TaskManager.Task::(_debug) Task 9aa45c3d-79c7-451a-ad8a-018cb1eb5810: moving from state preparing -> state finished
>Thread-15910462::DEBUG::2013-11-19 21:14:16,229::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910462::DEBUG::2013-11-19 21:14:16,229::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910462::DEBUG::2013-11-19 21:14:16,230::task::495::TaskManager.Task::(_debug) Task 9aa45c3d-79c7-451a-ad8a-018cb1eb5810: ref 0 aborting False
>Thread-15910462::INFO::2013-11-19 21:14:16,230::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910463::DEBUG::2013-11-19 21:14:17,239::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910463::INFO::2013-11-19 21:14:17,240::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=88cc2f68-e802-4615-bced-c7a8c4aefa33)
>Thread-15910463::DEBUG::2013-11-19 21:14:17,240::task::495::TaskManager.Task::(_debug) Task 55e70696-5317-4733-b499-880e08c0de0c: moving from state init -> state preparing
>Thread-15910463::DEBUG::2013-11-19 21:14:17,241::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 88cc2f68-e802-4615-bced-c7a8c4aefa33
>Thread-15910463::DEBUG::2013-11-19 21:14:17,241::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}
>Thread-15910463::DEBUG::2013-11-19 21:14:17,242::task::495::TaskManager.Task::(_debug) Task 55e70696-5317-4733-b499-880e08c0de0c: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910463::DEBUG::2013-11-19 21:14:17,243::task::495::TaskManager.Task::(_debug) Task 55e70696-5317-4733-b499-880e08c0de0c: moving from state preparing -> state finished
>Thread-15910463::DEBUG::2013-11-19 21:14:17,243::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910463::DEBUG::2013-11-19 21:14:17,244::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910463::DEBUG::2013-11-19 21:14:17,244::task::495::TaskManager.Task::(_debug) Task 55e70696-5317-4733-b499-880e08c0de0c: ref 0 aborting False
>Thread-15910463::INFO::2013-11-19 21:14:17,245::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910464::DEBUG::2013-11-19 21:14:18,263::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910464::INFO::2013-11-19 21:14:18,263::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=88cc2f68-e802-4615-bced-c7a8c4aefa33)
>Thread-15910464::DEBUG::2013-11-19 21:14:18,264::task::495::TaskManager.Task::(_debug) Task cf61f899-ed74-4e0d-ab6f-0f51e9a9be92: moving from state init -> state preparing
>Thread-15910464::DEBUG::2013-11-19 21:14:18,265::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 88cc2f68-e802-4615-bced-c7a8c4aefa33
>Thread-15910464::DEBUG::2013-11-19 21:14:18,265::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}
>Thread-15910464::DEBUG::2013-11-19 21:14:18,266::task::495::TaskManager.Task::(_debug) Task cf61f899-ed74-4e0d-ab6f-0f51e9a9be92: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910464::DEBUG::2013-11-19 21:14:18,266::task::495::TaskManager.Task::(_debug) Task cf61f899-ed74-4e0d-ab6f-0f51e9a9be92: moving from state preparing -> state finished
>Thread-15910464::DEBUG::2013-11-19 21:14:18,267::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910464::DEBUG::2013-11-19 21:14:18,267::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910464::DEBUG::2013-11-19 21:14:18,268::task::495::TaskManager.Task::(_debug) Task cf61f899-ed74-4e0d-ab6f-0f51e9a9be92: ref 0 aborting False
>Thread-15910464::INFO::2013-11-19 21:14:18,268::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910465::DEBUG::2013-11-19 21:14:19,281::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910465::INFO::2013-11-19 21:14:19,282::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=88cc2f68-e802-4615-bced-c7a8c4aefa33)
>Thread-15910465::DEBUG::2013-11-19 21:14:19,283::task::495::TaskManager.Task::(_debug) Task 4113e920-ed0b-4391-b3aa-051f91318fc2: moving from state init -> state preparing
>Thread-15910465::DEBUG::2013-11-19 21:14:19,284::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 88cc2f68-e802-4615-bced-c7a8c4aefa33
>Thread-15910465::DEBUG::2013-11-19 21:14:19,284::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}
>Thread-15910465::DEBUG::2013-11-19 21:14:19,285::task::495::TaskManager.Task::(_debug) Task 4113e920-ed0b-4391-b3aa-051f91318fc2: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910465::DEBUG::2013-11-19 21:14:19,285::task::495::TaskManager.Task::(_debug) Task 4113e920-ed0b-4391-b3aa-051f91318fc2: moving from state preparing -> state finished
>Thread-15910465::DEBUG::2013-11-19 21:14:19,286::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910465::DEBUG::2013-11-19 21:14:19,286::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910465::DEBUG::2013-11-19 21:14:19,287::task::495::TaskManager.Task::(_debug) Task 4113e920-ed0b-4391-b3aa-051f91318fc2: ref 0 aborting False
>Thread-15910465::INFO::2013-11-19 21:14:19,287::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910466::DEBUG::2013-11-19 21:14:20,304::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910466::INFO::2013-11-19 21:14:20,305::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=88cc2f68-e802-4615-bced-c7a8c4aefa33)
>Thread-15910466::DEBUG::2013-11-19 21:14:20,306::task::495::TaskManager.Task::(_debug) Task f0a3828e-3564-474b-a6e5-4421549393ae: moving from state init -> state preparing
>Thread-15910466::DEBUG::2013-11-19 21:14:20,306::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 88cc2f68-e802-4615-bced-c7a8c4aefa33
>Thread-15910466::DEBUG::2013-11-19 21:14:20,307::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}
>Thread-15910466::DEBUG::2013-11-19 21:14:20,307::task::495::TaskManager.Task::(_debug) Task f0a3828e-3564-474b-a6e5-4421549393ae: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910466::DEBUG::2013-11-19 21:14:20,308::task::495::TaskManager.Task::(_debug) Task f0a3828e-3564-474b-a6e5-4421549393ae: moving from state preparing -> state finished
>Thread-15910466::DEBUG::2013-11-19 21:14:20,308::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910466::DEBUG::2013-11-19 21:14:20,309::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910466::DEBUG::2013-11-19 21:14:20,310::task::495::TaskManager.Task::(_debug) Task f0a3828e-3564-474b-a6e5-4421549393ae: ref 0 aborting False
>Thread-15910466::INFO::2013-11-19 21:14:20,310::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910467::DEBUG::2013-11-19 21:14:21,329::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910467::INFO::2013-11-19 21:14:21,330::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=88cc2f68-e802-4615-bced-c7a8c4aefa33)
>Thread-15910467::DEBUG::2013-11-19 21:14:21,331::task::495::TaskManager.Task::(_debug) Task d575f139-6d0a-4b85-a560-5e4549635c81: moving from state init -> state preparing
>Thread-15910467::DEBUG::2013-11-19 21:14:21,331::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 88cc2f68-e802-4615-bced-c7a8c4aefa33
>Thread-15910467::DEBUG::2013-11-19 21:14:21,332::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}
>Thread-15910467::DEBUG::2013-11-19 21:14:21,332::task::495::TaskManager.Task::(_debug) Task d575f139-6d0a-4b85-a560-5e4549635c81: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910467::DEBUG::2013-11-19 21:14:21,333::task::495::TaskManager.Task::(_debug) Task d575f139-6d0a-4b85-a560-5e4549635c81: moving from state preparing -> state finished
>Thread-15910467::DEBUG::2013-11-19 21:14:21,334::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910467::DEBUG::2013-11-19 21:14:21,334::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910467::DEBUG::2013-11-19 21:14:21,335::task::495::TaskManager.Task::(_debug) Task d575f139-6d0a-4b85-a560-5e4549635c81: ref 0 aborting False
>Thread-15910467::INFO::2013-11-19 21:14:21,335::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910468::DEBUG::2013-11-19 21:14:22,341::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910468::INFO::2013-11-19 21:14:22,342::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=88cc2f68-e802-4615-bced-c7a8c4aefa33)
>Thread-15910468::DEBUG::2013-11-19 21:14:22,342::task::495::TaskManager.Task::(_debug) Task eca382ad-7607-4261-ab0c-62de58c1d69f: moving from state init -> state preparing
>Thread-15910468::DEBUG::2013-11-19 21:14:22,343::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 88cc2f68-e802-4615-bced-c7a8c4aefa33
>Thread-15910468::DEBUG::2013-11-19 21:14:22,344::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}
>Thread-15910468::DEBUG::2013-11-19 21:14:22,347::task::495::TaskManager.Task::(_debug) Task eca382ad-7607-4261-ab0c-62de58c1d69f: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910468::DEBUG::2013-11-19 21:14:22,348::task::495::TaskManager.Task::(_debug) Task eca382ad-7607-4261-ab0c-62de58c1d69f: moving from state preparing -> state finished
>Thread-15910468::DEBUG::2013-11-19 21:14:22,348::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910468::DEBUG::2013-11-19 21:14:22,349::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910468::DEBUG::2013-11-19 21:14:22,349::task::495::TaskManager.Task::(_debug) Task eca382ad-7607-4261-ab0c-62de58c1d69f: ref 0 aborting False
>Thread-15910468::INFO::2013-11-19 21:14:22,350::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910469::DEBUG::2013-11-19 21:14:23,367::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910469::INFO::2013-11-19 21:14:23,368::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=88cc2f68-e802-4615-bced-c7a8c4aefa33)
>Thread-15910469::DEBUG::2013-11-19 21:14:23,369::task::495::TaskManager.Task::(_debug) Task 3253eb4f-7c21-4a45-a99e-9d9deedb8290: moving from state init -> state preparing
>Thread-15910469::DEBUG::2013-11-19 21:14:23,369::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 88cc2f68-e802-4615-bced-c7a8c4aefa33
>Thread-15910469::DEBUG::2013-11-19 21:14:23,370::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}
>Thread-15910469::DEBUG::2013-11-19 21:14:23,372::task::495::TaskManager.Task::(_debug) Task 3253eb4f-7c21-4a45-a99e-9d9deedb8290: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910469::DEBUG::2013-11-19 21:14:23,373::task::495::TaskManager.Task::(_debug) Task 3253eb4f-7c21-4a45-a99e-9d9deedb8290: moving from state preparing -> state finished
>Thread-1024767::ERROR::2013-11-19 21:14:23,373::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910469::DEBUG::2013-11-19 21:14:23,374::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910469::DEBUG::2013-11-19 21:14:23,375::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910469::DEBUG::2013-11-19 21:14:23,376::task::495::TaskManager.Task::(_debug) Task 3253eb4f-7c21-4a45-a99e-9d9deedb8290: ref 0 aborting False
>Thread-15910469::INFO::2013-11-19 21:14:23,376::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910470::DEBUG::2013-11-19 21:14:24,382::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910470::INFO::2013-11-19 21:14:24,383::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=88cc2f68-e802-4615-bced-c7a8c4aefa33)
>Thread-15910470::DEBUG::2013-11-19 21:14:24,384::task::495::TaskManager.Task::(_debug) Task d19617ad-3633-4722-818d-3ee568334099: moving from state init -> state preparing
>Thread-15910470::DEBUG::2013-11-19 21:14:24,384::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 88cc2f68-e802-4615-bced-c7a8c4aefa33
>Thread-15910470::DEBUG::2013-11-19 21:14:24,385::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}
>Thread-15910470::DEBUG::2013-11-19 21:14:24,386::task::495::TaskManager.Task::(_debug) Task d19617ad-3633-4722-818d-3ee568334099: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910470::DEBUG::2013-11-19 21:14:24,386::task::495::TaskManager.Task::(_debug) Task d19617ad-3633-4722-818d-3ee568334099: moving from state preparing -> state finished
>Thread-15910470::DEBUG::2013-11-19 21:14:24,387::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910470::DEBUG::2013-11-19 21:14:24,387::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910470::DEBUG::2013-11-19 21:14:24,388::task::495::TaskManager.Task::(_debug) Task d19617ad-3633-4722-818d-3ee568334099: ref 0 aborting False
>Thread-15910470::INFO::2013-11-19 21:14:24,388::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-42838::DEBUG::2013-11-19 21:14:24,587::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 21:14:25,221::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15910471::DEBUG::2013-11-19 21:14:25,401::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910471::INFO::2013-11-19 21:14:25,402::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=88cc2f68-e802-4615-bced-c7a8c4aefa33)
>Thread-15910471::DEBUG::2013-11-19 21:14:25,403::task::495::TaskManager.Task::(_debug) Task b89e0ab3-9b58-4116-b596-e353742c3fe7: moving from state init -> state preparing
>Thread-15910471::DEBUG::2013-11-19 21:14:25,403::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 88cc2f68-e802-4615-bced-c7a8c4aefa33
>Thread-15910471::DEBUG::2013-11-19 21:14:25,404::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}
>Thread-15910471::DEBUG::2013-11-19 21:14:25,405::task::495::TaskManager.Task::(_debug) Task b89e0ab3-9b58-4116-b596-e353742c3fe7: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910471::DEBUG::2013-11-19 21:14:25,405::task::495::TaskManager.Task::(_debug) Task b89e0ab3-9b58-4116-b596-e353742c3fe7: moving from state preparing -> state finished
>Thread-15910471::DEBUG::2013-11-19 21:14:25,406::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910471::DEBUG::2013-11-19 21:14:25,406::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910471::DEBUG::2013-11-19 21:14:25,407::task::495::TaskManager.Task::(_debug) Task b89e0ab3-9b58-4116-b596-e353742c3fe7: ref 0 aborting False
>Thread-15910471::INFO::2013-11-19 21:14:25,407::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910472::DEBUG::2013-11-19 21:14:26,418::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910472::INFO::2013-11-19 21:14:26,419::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=88cc2f68-e802-4615-bced-c7a8c4aefa33)
>Thread-15910472::DEBUG::2013-11-19 21:14:26,426::task::495::TaskManager.Task::(_debug) Task c20af5db-d874-4bfa-aa5d-0c3db7e24e87: moving from state init -> state preparing
>Thread-15910472::DEBUG::2013-11-19 21:14:26,427::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 88cc2f68-e802-4615-bced-c7a8c4aefa33
>Thread-15910472::DEBUG::2013-11-19 21:14:26,427::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}
>Thread-15910472::DEBUG::2013-11-19 21:14:26,428::task::495::TaskManager.Task::(_debug) Task c20af5db-d874-4bfa-aa5d-0c3db7e24e87: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910472::DEBUG::2013-11-19 21:14:26,429::task::495::TaskManager.Task::(_debug) Task c20af5db-d874-4bfa-aa5d-0c3db7e24e87: moving from state preparing -> state finished
>Thread-15910472::DEBUG::2013-11-19 21:14:26,429::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910472::DEBUG::2013-11-19 21:14:26,430::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910472::DEBUG::2013-11-19 21:14:26,430::task::495::TaskManager.Task::(_debug) Task c20af5db-d874-4bfa-aa5d-0c3db7e24e87: ref 0 aborting False
>Thread-15910472::INFO::2013-11-19 21:14:26,431::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910473::DEBUG::2013-11-19 21:14:27,437::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910473::INFO::2013-11-19 21:14:27,438::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=88cc2f68-e802-4615-bced-c7a8c4aefa33)
>Thread-15910473::DEBUG::2013-11-19 21:14:27,438::task::495::TaskManager.Task::(_debug) Task 7bbd9c7f-0b7b-420a-8220-a07f38a2c6e4: moving from state init -> state preparing
>Thread-15910473::DEBUG::2013-11-19 21:14:27,439::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 88cc2f68-e802-4615-bced-c7a8c4aefa33
>Thread-15910473::DEBUG::2013-11-19 21:14:27,439::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}
>Thread-15910473::DEBUG::2013-11-19 21:14:27,440::task::495::TaskManager.Task::(_debug) Task 7bbd9c7f-0b7b-420a-8220-a07f38a2c6e4: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910473::DEBUG::2013-11-19 21:14:27,441::task::495::TaskManager.Task::(_debug) Task 7bbd9c7f-0b7b-420a-8220-a07f38a2c6e4: moving from state preparing -> state finished
>Thread-15910473::DEBUG::2013-11-19 21:14:27,441::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910473::DEBUG::2013-11-19 21:14:27,442::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910473::DEBUG::2013-11-19 21:14:27,442::task::495::TaskManager.Task::(_debug) Task 7bbd9c7f-0b7b-420a-8220-a07f38a2c6e4: ref 0 aborting False
>Thread-15910473::INFO::2013-11-19 21:14:27,443::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910474::DEBUG::2013-11-19 21:14:28,460::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910474::INFO::2013-11-19 21:14:28,461::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=88cc2f68-e802-4615-bced-c7a8c4aefa33)
>Thread-15910474::DEBUG::2013-11-19 21:14:28,461::task::495::TaskManager.Task::(_debug) Task 689ceecd-8576-40f5-9ce8-c91121bb6be3: moving from state init -> state preparing
>Thread-15910474::DEBUG::2013-11-19 21:14:28,462::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 88cc2f68-e802-4615-bced-c7a8c4aefa33
>Thread-15910474::DEBUG::2013-11-19 21:14:28,462::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}
>Thread-15910474::DEBUG::2013-11-19 21:14:28,463::task::495::TaskManager.Task::(_debug) Task 689ceecd-8576-40f5-9ce8-c91121bb6be3: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910474::DEBUG::2013-11-19 21:14:28,464::task::495::TaskManager.Task::(_debug) Task 689ceecd-8576-40f5-9ce8-c91121bb6be3: moving from state preparing -> state finished
>Thread-15910474::DEBUG::2013-11-19 21:14:28,464::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910474::DEBUG::2013-11-19 21:14:28,465::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910474::DEBUG::2013-11-19 21:14:28,465::task::495::TaskManager.Task::(_debug) Task 689ceecd-8576-40f5-9ce8-c91121bb6be3: ref 0 aborting False
>Thread-15910474::INFO::2013-11-19 21:14:28,466::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910475::DEBUG::2013-11-19 21:14:29,480::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910475::INFO::2013-11-19 21:14:29,481::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=88cc2f68-e802-4615-bced-c7a8c4aefa33)
>Thread-15910475::DEBUG::2013-11-19 21:14:29,481::task::495::TaskManager.Task::(_debug) Task dae02429-78b9-453f-94f3-01231893f908: moving from state init -> state preparing
>Thread-15910475::DEBUG::2013-11-19 21:14:29,482::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 88cc2f68-e802-4615-bced-c7a8c4aefa33
>Thread-15910475::DEBUG::2013-11-19 21:14:29,482::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}
>Thread-15910475::DEBUG::2013-11-19 21:14:29,483::task::495::TaskManager.Task::(_debug) Task dae02429-78b9-453f-94f3-01231893f908: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910475::DEBUG::2013-11-19 21:14:29,484::task::495::TaskManager.Task::(_debug) Task dae02429-78b9-453f-94f3-01231893f908: moving from state preparing -> state finished
>Thread-15910475::DEBUG::2013-11-19 21:14:29,484::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910475::DEBUG::2013-11-19 21:14:29,485::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910475::DEBUG::2013-11-19 21:14:29,485::task::495::TaskManager.Task::(_debug) Task dae02429-78b9-453f-94f3-01231893f908: ref 0 aborting False
>Thread-15910475::INFO::2013-11-19 21:14:29,486::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910476::DEBUG::2013-11-19 21:14:30,492::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910476::INFO::2013-11-19 21:14:30,494::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=88cc2f68-e802-4615-bced-c7a8c4aefa33)
>Thread-15910476::DEBUG::2013-11-19 21:14:30,494::task::495::TaskManager.Task::(_debug) Task 2cea97ee-4bc7-4fb2-9463-66879a8c2d1a: moving from state init -> state preparing
>Thread-15910476::DEBUG::2013-11-19 21:14:30,495::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 88cc2f68-e802-4615-bced-c7a8c4aefa33
>Thread-15910476::DEBUG::2013-11-19 21:14:30,496::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}
>Thread-15910476::DEBUG::2013-11-19 21:14:30,497::task::495::TaskManager.Task::(_debug) Task 2cea97ee-4bc7-4fb2-9463-66879a8c2d1a: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910476::DEBUG::2013-11-19 21:14:30,499::task::495::TaskManager.Task::(_debug) Task 2cea97ee-4bc7-4fb2-9463-66879a8c2d1a: moving from state preparing -> state finished
>Thread-15910476::DEBUG::2013-11-19 21:14:30,500::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910476::DEBUG::2013-11-19 21:14:30,501::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910476::DEBUG::2013-11-19 21:14:30,503::task::495::TaskManager.Task::(_debug) Task 2cea97ee-4bc7-4fb2-9463-66879a8c2d1a: ref 0 aborting False
>Thread-15910476::INFO::2013-11-19 21:14:30,505::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910477::DEBUG::2013-11-19 21:14:31,518::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910477::INFO::2013-11-19 21:14:31,519::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=88cc2f68-e802-4615-bced-c7a8c4aefa33)
>Thread-15910477::DEBUG::2013-11-19 21:14:31,524::task::495::TaskManager.Task::(_debug) Task 507bfd38-35ee-4e88-86c0-8b1a3c5c9c9c: moving from state init -> state preparing
>Thread-15910477::DEBUG::2013-11-19 21:14:31,524::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 88cc2f68-e802-4615-bced-c7a8c4aefa33
>Thread-15910477::DEBUG::2013-11-19 21:14:31,525::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}
>Thread-15910477::DEBUG::2013-11-19 21:14:31,525::task::495::TaskManager.Task::(_debug) Task 507bfd38-35ee-4e88-86c0-8b1a3c5c9c9c: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910477::DEBUG::2013-11-19 21:14:31,526::task::495::TaskManager.Task::(_debug) Task 507bfd38-35ee-4e88-86c0-8b1a3c5c9c9c: moving from state preparing -> state finished
>Thread-15910477::DEBUG::2013-11-19 21:14:31,527::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910477::DEBUG::2013-11-19 21:14:31,527::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910477::DEBUG::2013-11-19 21:14:31,528::task::495::TaskManager.Task::(_debug) Task 507bfd38-35ee-4e88-86c0-8b1a3c5c9c9c: ref 0 aborting False
>Thread-15910477::INFO::2013-11-19 21:14:31,528::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910478::DEBUG::2013-11-19 21:14:32,534::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910478::INFO::2013-11-19 21:14:32,535::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=88cc2f68-e802-4615-bced-c7a8c4aefa33)
>Thread-15910478::DEBUG::2013-11-19 21:14:32,536::task::495::TaskManager.Task::(_debug) Task 7eca6f46-ddfc-4986-ac2d-d0e9cdcd4d4c: moving from state init -> state preparing
>Thread-15910478::DEBUG::2013-11-19 21:14:32,537::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 88cc2f68-e802-4615-bced-c7a8c4aefa33
>Thread-15910478::DEBUG::2013-11-19 21:14:32,537::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}
>Thread-15910478::DEBUG::2013-11-19 21:14:32,538::task::495::TaskManager.Task::(_debug) Task 7eca6f46-ddfc-4986-ac2d-d0e9cdcd4d4c: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910478::DEBUG::2013-11-19 21:14:32,538::task::495::TaskManager.Task::(_debug) Task 7eca6f46-ddfc-4986-ac2d-d0e9cdcd4d4c: moving from state preparing -> state finished
>Thread-15910478::DEBUG::2013-11-19 21:14:32,539::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910478::DEBUG::2013-11-19 21:14:32,539::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910478::DEBUG::2013-11-19 21:14:32,540::task::495::TaskManager.Task::(_debug) Task 7eca6f46-ddfc-4986-ac2d-d0e9cdcd4d4c: ref 0 aborting False
>Thread-15910478::INFO::2013-11-19 21:14:32,541::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-1024767::ERROR::2013-11-19 21:14:33,388::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910479::DEBUG::2013-11-19 21:14:33,546::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910479::INFO::2013-11-19 21:14:33,547::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=88cc2f68-e802-4615-bced-c7a8c4aefa33)
>Thread-15910479::DEBUG::2013-11-19 21:14:33,548::task::495::TaskManager.Task::(_debug) Task 364d24e3-d387-4238-9b7d-bdb88d49528f: moving from state init -> state preparing
>Thread-15910479::DEBUG::2013-11-19 21:14:33,549::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 88cc2f68-e802-4615-bced-c7a8c4aefa33
>Thread-15910479::DEBUG::2013-11-19 21:14:33,549::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}
>Thread-15910479::DEBUG::2013-11-19 21:14:33,550::task::495::TaskManager.Task::(_debug) Task 364d24e3-d387-4238-9b7d-bdb88d49528f: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910479::DEBUG::2013-11-19 21:14:33,550::task::495::TaskManager.Task::(_debug) Task 364d24e3-d387-4238-9b7d-bdb88d49528f: moving from state preparing -> state finished
>Thread-15910479::DEBUG::2013-11-19 21:14:33,551::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910479::DEBUG::2013-11-19 21:14:33,551::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910479::DEBUG::2013-11-19 21:14:33,552::task::495::TaskManager.Task::(_debug) Task 364d24e3-d387-4238-9b7d-bdb88d49528f: ref 0 aborting False
>Thread-15910479::INFO::2013-11-19 21:14:33,553::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910480::DEBUG::2013-11-19 21:14:34,564::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910480::INFO::2013-11-19 21:14:34,565::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=88cc2f68-e802-4615-bced-c7a8c4aefa33)
>Thread-15910480::DEBUG::2013-11-19 21:14:34,566::task::495::TaskManager.Task::(_debug) Task 06c4428a-99fe-4902-bcd7-5fdfa886a738: moving from state init -> state preparing
>Thread-15910480::DEBUG::2013-11-19 21:14:34,566::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 88cc2f68-e802-4615-bced-c7a8c4aefa33
>Thread-15910480::DEBUG::2013-11-19 21:14:34,567::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}
>Thread-15910480::DEBUG::2013-11-19 21:14:34,567::task::495::TaskManager.Task::(_debug) Task 06c4428a-99fe-4902-bcd7-5fdfa886a738: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910480::DEBUG::2013-11-19 21:14:34,568::task::495::TaskManager.Task::(_debug) Task 06c4428a-99fe-4902-bcd7-5fdfa886a738: moving from state preparing -> state finished
>Thread-15910480::DEBUG::2013-11-19 21:14:34,568::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910480::DEBUG::2013-11-19 21:14:34,569::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910480::DEBUG::2013-11-19 21:14:34,570::task::495::TaskManager.Task::(_debug) Task 06c4428a-99fe-4902-bcd7-5fdfa886a738: ref 0 aborting False
>Thread-15910480::INFO::2013-11-19 21:14:34,570::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910481::DEBUG::2013-11-19 21:14:35,576::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910481::INFO::2013-11-19 21:14:35,577::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=88cc2f68-e802-4615-bced-c7a8c4aefa33)
>Thread-15910481::DEBUG::2013-11-19 21:14:35,578::task::495::TaskManager.Task::(_debug) Task 45ed58d2-bcd1-4d94-8a88-0d15204429fc: moving from state init -> state preparing
>Thread-15910481::DEBUG::2013-11-19 21:14:35,578::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 88cc2f68-e802-4615-bced-c7a8c4aefa33
>Thread-15910481::DEBUG::2013-11-19 21:14:35,579::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}
>Thread-15910481::DEBUG::2013-11-19 21:14:35,579::task::495::TaskManager.Task::(_debug) Task 45ed58d2-bcd1-4d94-8a88-0d15204429fc: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910481::DEBUG::2013-11-19 21:14:35,580::task::495::TaskManager.Task::(_debug) Task 45ed58d2-bcd1-4d94-8a88-0d15204429fc: moving from state preparing -> state finished
>Thread-15910481::DEBUG::2013-11-19 21:14:35,580::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910481::DEBUG::2013-11-19 21:14:35,581::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910481::DEBUG::2013-11-19 21:14:35,581::task::495::TaskManager.Task::(_debug) Task 45ed58d2-bcd1-4d94-8a88-0d15204429fc: ref 0 aborting False
>Thread-15910481::INFO::2013-11-19 21:14:35,582::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:36,082::safelease::61::Storage.Misc.excCmd::(acquire) SUCCESS: <err> = ''; <rc> = 0
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:36,083::safelease::65::ClusterLock::(acquire) Clustered lock acquired successfully
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:36,084::spm::468::Storage.SPM::(start) spm lock acquired successfully
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:36,084::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:36,085::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:36,086::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:36,086::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:36,087::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:36,088::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15910482::DEBUG::2013-11-19 21:14:36,594::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910482::INFO::2013-11-19 21:14:36,596::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=88cc2f68-e802-4615-bced-c7a8c4aefa33)
>Thread-15910482::DEBUG::2013-11-19 21:14:36,612::task::495::TaskManager.Task::(_debug) Task 31ae7293-5be4-47cb-83e8-da0d6335f8de: moving from state init -> state preparing
>Thread-15910482::DEBUG::2013-11-19 21:14:36,613::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 88cc2f68-e802-4615-bced-c7a8c4aefa33
>Thread-15910482::DEBUG::2013-11-19 21:14:36,613::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}
>Thread-15910482::DEBUG::2013-11-19 21:14:36,614::task::495::TaskManager.Task::(_debug) Task 31ae7293-5be4-47cb-83e8-da0d6335f8de: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910482::DEBUG::2013-11-19 21:14:36,615::task::495::TaskManager.Task::(_debug) Task 31ae7293-5be4-47cb-83e8-da0d6335f8de: moving from state preparing -> state finished
>Thread-15910482::DEBUG::2013-11-19 21:14:36,615::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910482::DEBUG::2013-11-19 21:14:36,616::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910482::DEBUG::2013-11-19 21:14:36,616::task::495::TaskManager.Task::(_debug) Task 31ae7293-5be4-47cb-83e8-da0d6335f8de: ref 0 aborting False
>Thread-15910482::INFO::2013-11-19 21:14:36,617::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:36,752::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:36,755::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:36,756::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_LVER=276', u'POOL_SPM_ID=-1', u'_SHA_CKSUM=d1ce634f167d7f0ddd58cf2ebbdf434d45f6ef67']
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:36,757::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:36,758::persistentDict::146::Storage.PersistentDict::(transaction) Flushing changes
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:36,759::persistentDict::265::Storage.PersistentDict::(flush) about to write lines (VGTagMetadataRW)=[u'CLASS=Data', u'DESCRIPTION=OS', u'IOOPTIMEOUTSEC=10', u'LEASERETRIES=3', u'LEASETIMESEC=60', u'LOCKPOLICY=', u'LOCKRENEWALINTERVALSEC=5', u'LOGBLKSIZE=512', u'MASTER_VERSION=1', u'PHYBLKSIZE=512', u'POOL_DESCRIPTION=Default', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_ID=2', u'POOL_SPM_LVER=277', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'ROLE=Master', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'TYPE=FCP', u'VERSION=2', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', '_SHA_CKSUM=a3ed62b0970fa02db432bafe8169c17849a2719b']
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:36,759::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:36,760::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:36,761::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:36,761::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:36,762::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:36,763::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:37,416::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:37,419::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:37,420::blockSD::145::storage.Metadata.VGTagMetadataRW::(writelines) Updating metadata adding=MDT_POOL_SPM_LVER=277, MDT__SHA_CKSUM=a3ed62b0970fa02db432bafe8169c17849a2719b, MDT_POOL_SPM_ID=2 removing=MDT_POOL_SPM_ID=-1, MDT_POOL_SPM_LVER=276, MDT__SHA_CKSUM=d1ce634f167d7f0ddd58cf2ebbdf434d45f6ef67
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:37,421::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --deltag MDT_POOL_SPM_LVER=276 --deltag MDT_POOL_SPM_ID=-1 --deltag MDT__SHA_CKSUM=d1ce634f167d7f0ddd58cf2ebbdf434d45f6ef67 --addtag MDT__SHA_CKSUM=a3ed62b0970fa02db432bafe8169c17849a2719b --addtag MDT_POOL_SPM_ID=2 --addtag MDT_POOL_SPM_LVER=277 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15910483::DEBUG::2013-11-19 21:14:37,645::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910483::INFO::2013-11-19 21:14:37,648::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=88cc2f68-e802-4615-bced-c7a8c4aefa33)
>Thread-15910483::DEBUG::2013-11-19 21:14:37,649::task::495::TaskManager.Task::(_debug) Task 03a78abe-6dc0-40fe-b840-bb7b1fae9ef7: moving from state init -> state preparing
>Thread-15910483::DEBUG::2013-11-19 21:14:37,649::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 88cc2f68-e802-4615-bced-c7a8c4aefa33
>Thread-15910483::DEBUG::2013-11-19 21:14:37,650::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}
>Thread-15910483::DEBUG::2013-11-19 21:14:37,651::task::495::TaskManager.Task::(_debug) Task 03a78abe-6dc0-40fe-b840-bb7b1fae9ef7: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910483::DEBUG::2013-11-19 21:14:37,651::task::495::TaskManager.Task::(_debug) Task 03a78abe-6dc0-40fe-b840-bb7b1fae9ef7: moving from state preparing -> state finished
>Thread-15910483::DEBUG::2013-11-19 21:14:37,652::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910483::DEBUG::2013-11-19 21:14:37,652::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910483::DEBUG::2013-11-19 21:14:37,653::task::495::TaskManager.Task::(_debug) Task 03a78abe-6dc0-40fe-b840-bb7b1fae9ef7: ref 0 aborting False
>Thread-15910483::INFO::2013-11-19 21:14:37,654::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:38,127::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:38,128::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:38,129::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:38,130::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:38,131::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`67122734-677a-4233-ba17-e4c9cde2c78c`::Request was made in '/usr/share/vdsm/storage/spm.py' line '330' at '_upgradePool'
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:38,132::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'exclusive'
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:38,133::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'exclusive' (1 active user)
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:38,133::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`67122734-677a-4233-ba17-e4c9cde2c78c`::Granted request
>88cc2f68-e802-4615-bced-c7a8c4aefa33::INFO::2013-11-19 21:14:38,134::spm::336::Storage.SPM::(_upgradePool) Trying to upgrade master domain `6b892067-ac7e-4039-a7be-5b7387525b16`
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:38,135::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ce6bff3b-5f97-48a2-b4f1-b2b55f16516b`::Request was made in '/usr/share/vdsm/storage/spm.py' line '337' at '_upgradePool'
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:38,135::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'exclusive'
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:38,136::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'exclusive' (1 active user)
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:38,136::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ce6bff3b-5f97-48a2-b4f1-b2b55f16516b`::Granted request
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:38,138::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:38,138::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:38,139::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:38,139::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:38,140::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:38,141::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15910484::DEBUG::2013-11-19 21:14:38,667::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910484::INFO::2013-11-19 21:14:38,669::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=88cc2f68-e802-4615-bced-c7a8c4aefa33)
>Thread-15910484::DEBUG::2013-11-19 21:14:38,670::task::495::TaskManager.Task::(_debug) Task d25b5b9b-4767-4c17-ba1a-a942fdcd80de: moving from state init -> state preparing
>Thread-15910484::DEBUG::2013-11-19 21:14:38,670::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 88cc2f68-e802-4615-bced-c7a8c4aefa33
>Thread-15910484::DEBUG::2013-11-19 21:14:38,671::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}
>Thread-15910484::DEBUG::2013-11-19 21:14:38,671::task::495::TaskManager.Task::(_debug) Task d25b5b9b-4767-4c17-ba1a-a942fdcd80de: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910484::DEBUG::2013-11-19 21:14:38,672::task::495::TaskManager.Task::(_debug) Task d25b5b9b-4767-4c17-ba1a-a942fdcd80de: moving from state preparing -> state finished
>Thread-15910484::DEBUG::2013-11-19 21:14:38,673::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910484::DEBUG::2013-11-19 21:14:38,673::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910484::DEBUG::2013-11-19 21:14:38,674::task::495::TaskManager.Task::(_debug) Task d25b5b9b-4767-4c17-ba1a-a942fdcd80de: ref 0 aborting False
>Thread-15910484::INFO::2013-11-19 21:14:38,674::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:38,790::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:38,793::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:38,794::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'_SHA_CKSUM=a3ed62b0970fa02db432bafe8169c17849a2719b', u'POOL_SPM_ID=2', u'POOL_SPM_LVER=277']
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:38,795::blockSD::625::Storage.StorageDomain::(upgrade) Trying to upgrade domain `6b892067-ac7e-4039-a7be-5b7387525b16` from version 2 to version 2
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:38,795::blockSD::631::Storage.StorageDomain::(upgrade) No need to upgrade domain `6b892067-ac7e-4039-a7be-5b7387525b16`, leaving unchanged
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:38,796::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:38,797::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:38,797::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:38,798::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:38,798::spm::340::Storage.SPM::(_upgradePool) Marking all domains for upgrade
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:38,799::spm::347::Storage.SPM::(_upgradePool) Registering with state change event
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:38,800::spm::349::Storage.SPM::(_upgradePool) Running initial domain upgrade threads
>Thread-15910485::DEBUG::2013-11-19 21:14:38,800::spm::365::Storage.SPM::(_upgradePoolDomain) Preparing to upgrade domain 45cdff8b-89c0-4926-a8c7-80354064bfa6
>Thread-15910485::DEBUG::2013-11-19 21:14:38,805::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6`ReqID=`b8588e7b-60b6-4f67-af89-a20223c7b28d`::Request was made in '/usr/share/vdsm/storage/spm.py' line '376' at '_upgradePoolDomain'
>Thread-15910486::DEBUG::2013-11-19 21:14:38,806::spm::365::Storage.SPM::(_upgradePoolDomain) Preparing to upgrade domain 772472cb-2429-466d-96cd-00b3301ef78a
>Thread-15910485::DEBUG::2013-11-19 21:14:38,807::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6' for lock type 'exclusive'
>Thread-15910486::DEBUG::2013-11-19 21:14:38,808::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8fc8ef6f-f04a-48c0-bfb3-64a869b65da2`::Request was made in '/usr/share/vdsm/storage/spm.py' line '376' at '_upgradePoolDomain'
>Thread-15910487::DEBUG::2013-11-19 21:14:38,809::spm::365::Storage.SPM::(_upgradePoolDomain) Preparing to upgrade domain 5d7b5178-3790-472c-a13b-b481eb27a61e
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:38,809::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910485::DEBUG::2013-11-19 21:14:38,810::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6' is free. Now locking as 'exclusive' (1 active user)
>Thread-15910486::DEBUG::2013-11-19 21:14:38,811::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'exclusive'
>Thread-15910487::ERROR::2013-11-19 21:14:38,814::spm::373::Storage.SPM::(_upgradePoolDomain) Error while preparing domain `5d7b5178-3790-472c-a13b-b481eb27a61e` upgrade
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/spm.py", line 369, in _upgradePoolDomain
>IndexError: list index out of range
>Thread-15910485::DEBUG::2013-11-19 21:14:38,815::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6`ReqID=`b8588e7b-60b6-4f67-af89-a20223c7b28d`::Granted request
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:38,818::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910485::DEBUG::2013-11-19 21:14:38,819::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6`ReqID=`176bc5dc-18d7-4e40-9250-e1b5ac7d93d2`::Request was made in '/usr/share/vdsm/storage/spm.py' line '377' at '_upgradePoolDomain'
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:38,820::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910485::DEBUG::2013-11-19 21:14:38,820::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6' for lock type 'exclusive'
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:38,821::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910486::DEBUG::2013-11-19 21:14:38,822::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'exclusive' (1 active user)
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:38,823::blockSD::791::Storage.Misc.excCmd::(mountMaster) '/usr/bin/sudo -n /sbin/e2fsck -p /dev/6b892067-ac7e-4039-a7be-5b7387525b16/master' (cwd None)
>Thread-15910486::DEBUG::2013-11-19 21:14:38,824::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8fc8ef6f-f04a-48c0-bfb3-64a869b65da2`::Granted request
>Thread-15910485::DEBUG::2013-11-19 21:14:38,825::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6' is free. Now locking as 'exclusive' (1 active user)
>Thread-15910486::DEBUG::2013-11-19 21:14:39,033::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`9efd327d-86d6-446b-9413-3a93bd99b0de`::Request was made in '/usr/share/vdsm/storage/spm.py' line '377' at '_upgradePoolDomain'
>Thread-15910486::DEBUG::2013-11-19 21:14:39,036::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'exclusive'
>Thread-15910485::DEBUG::2013-11-19 21:14:39,036::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6`ReqID=`176bc5dc-18d7-4e40-9250-e1b5ac7d93d2`::Granted request
>Thread-15910486::DEBUG::2013-11-19 21:14:39,040::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'exclusive' (1 active user)
>Thread-15910486::DEBUG::2013-11-19 21:14:39,042::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`9efd327d-86d6-446b-9413-3a93bd99b0de`::Granted request
>Thread-15910485::DEBUG::2013-11-19 21:14:39,042::spm::385::Storage.SPM::(_upgradePoolDomain) Domain `45cdff8b-89c0-4926-a8c7-80354064bfa6` is not a data domain it is an 2 domain, not upgrading
>Thread-15910485::DEBUG::2013-11-19 21:14:39,052::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6'
>Thread-15910485::DEBUG::2013-11-19 21:14:39,054::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6' (0 active users)
>Thread-15910486::DEBUG::2013-11-19 21:14:39,055::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15910485::DEBUG::2013-11-19 21:14:39,055::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6' is free, finding out if anyone is waiting for it.
>Thread-15910486::DEBUG::2013-11-19 21:14:39,057::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15910485::DEBUG::2013-11-19 21:14:39,060::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6', Clearing records.
>Thread-15910486::DEBUG::2013-11-19 21:14:39,066::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15910486::DEBUG::2013-11-19 21:14:39,069::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15910485::DEBUG::2013-11-19 21:14:39,069::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6'
>Thread-15910486::DEBUG::2013-11-19 21:14:39,070::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-15910485::DEBUG::2013-11-19 21:14:39,071::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6' (0 active users)
>Thread-15910486::DEBUG::2013-11-19 21:14:39,073::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-15910485::DEBUG::2013-11-19 21:14:39,074::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6' is free, finding out if anyone is waiting for it.
>Thread-15910485::DEBUG::2013-11-19 21:14:39,079::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6', Clearing records.
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:39,395::blockSD::791::Storage.Misc.excCmd::(mountMaster) SUCCESS: <err> = ''; <rc> = 0
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:39,396::blockSD::816::Storage.Misc.excCmd::(mountMaster) '/usr/bin/sudo -n /sbin/tune2fs -j /dev/6b892067-ac7e-4039-a7be-5b7387525b16/master' (cwd None)
>Thread-15910488::DEBUG::2013-11-19 21:14:39,800::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910488::INFO::2013-11-19 21:14:39,802::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=88cc2f68-e802-4615-bced-c7a8c4aefa33)
>Thread-15910488::DEBUG::2013-11-19 21:14:39,802::task::495::TaskManager.Task::(_debug) Task f15d3920-3338-4349-8a04-900826b13937: moving from state init -> state preparing
>Thread-15910488::DEBUG::2013-11-19 21:14:39,803::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 88cc2f68-e802-4615-bced-c7a8c4aefa33
>Thread-15910488::DEBUG::2013-11-19 21:14:39,803::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}
>Thread-15910488::DEBUG::2013-11-19 21:14:39,804::task::495::TaskManager.Task::(_debug) Task f15d3920-3338-4349-8a04-900826b13937: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910488::DEBUG::2013-11-19 21:14:39,804::task::495::TaskManager.Task::(_debug) Task f15d3920-3338-4349-8a04-900826b13937: moving from state preparing -> state finished
>Thread-15910488::DEBUG::2013-11-19 21:14:39,804::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910488::DEBUG::2013-11-19 21:14:39,805::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910488::DEBUG::2013-11-19 21:14:39,805::task::495::TaskManager.Task::(_debug) Task f15d3920-3338-4349-8a04-900826b13937: ref 0 aborting False
>Thread-15910488::INFO::2013-11-19 21:14:39,805::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:39,935::blockSD::816::Storage.Misc.excCmd::(mountMaster) FAILED: <err> = 'The filesystem already has a journal.\n'; <rc> = 1
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:39,938::fileUtils::83::Storage.Misc.excCmd::(mount) '/usr/bin/sudo -n /bin/mount /dev/6b892067-ac7e-4039-a7be-5b7387525b16/master /rhev/data-center/mnt/blockSD/6b892067-ac7e-4039-a7be-5b7387525b16/master' (cwd None)
>Thread-15910486::DEBUG::2013-11-19 21:14:40,230::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15910486::DEBUG::2013-11-19 21:14:40,233::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-15910486::DEBUG::2013-11-19 21:14:40,234::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'SDUUID=772472cb-2429-466d-96cd-00b3301ef78a', u'VERSION=2', u'LOCKPOLICY=', u'VGUUID=XTb294-NYv6-dVpO-bB1n-4f9P-d0GL-dhd2DI', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'LEASETIMESEC=5', u'LOCKRENEWALINTERVALSEC=5', u'ROLE=Regular', u'PHYBLKSIZE=512', u'DESCRIPTION=Database', u'IOOPTIMEOUTSEC=1', u'CLASS=Data', u'TYPE=FCP', u'PV0=pv:3600c0ff0001520c7414f605001000000,uuid:NujM33-vNiD-sBHY-mrPo-EQ2f-1qsz-HXfn0q,pestart:0,pecount:8929,mapoffset:0', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'MASTER_VERSION=0', u'_SHA_CKSUM=ef8840d9fdef24516a56fd970bed1960d2b1ae92']
>Thread-15910486::DEBUG::2013-11-19 21:14:40,235::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15910486::DEBUG::2013-11-19 21:14:40,236::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15910486::DEBUG::2013-11-19 21:14:40,236::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15910486::DEBUG::2013-11-19 21:14:40,236::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15910486::DEBUG::2013-11-19 21:14:40,237::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15910486::DEBUG::2013-11-19 21:14:40,237::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15910486::DEBUG::2013-11-19 21:14:40,238::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15910486::DEBUG::2013-11-19 21:14:40,238::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,415::fileUtils::83::Storage.Misc.excCmd::(mount) SUCCESS: <err> = ''; <rc> = 0
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,416::blockSD::833::Storage.Misc.excCmd::(mountMaster) '/usr/bin/sudo -n /bin/chown vdsm:kvm /rhev/data-center/mnt/blockSD/6b892067-ac7e-4039-a7be-5b7387525b16/master' (cwd None)
>Thread-15910489::DEBUG::2013-11-19 21:14:40,814::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910489::INFO::2013-11-19 21:14:40,816::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=88cc2f68-e802-4615-bced-c7a8c4aefa33)
>Thread-15910489::DEBUG::2013-11-19 21:14:40,817::task::495::TaskManager.Task::(_debug) Task 19474d41-a570-4cc4-af58-f9ef6607c694: moving from state init -> state preparing
>Thread-15910489::DEBUG::2013-11-19 21:14:40,818::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 88cc2f68-e802-4615-bced-c7a8c4aefa33
>Thread-15910489::DEBUG::2013-11-19 21:14:40,818::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}
>Thread-15910489::DEBUG::2013-11-19 21:14:40,819::task::495::TaskManager.Task::(_debug) Task 19474d41-a570-4cc4-af58-f9ef6607c694: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910489::DEBUG::2013-11-19 21:14:40,820::task::495::TaskManager.Task::(_debug) Task 19474d41-a570-4cc4-af58-f9ef6607c694: moving from state preparing -> state finished
>Thread-15910489::DEBUG::2013-11-19 21:14:40,820::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910489::DEBUG::2013-11-19 21:14:40,821::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910489::DEBUG::2013-11-19 21:14:40,821::task::495::TaskManager.Task::(_debug) Task 19474d41-a570-4cc4-af58-f9ef6607c694: ref 0 aborting False
>Thread-15910489::INFO::2013-11-19 21:14:40,822::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,843::blockSD::833::Storage.Misc.excCmd::(mountMaster) SUCCESS: <err> = ''; <rc> = 0
>88cc2f68-e802-4615-bced-c7a8c4aefa33::WARNING::2013-11-19 21:14:40,871::spm::490::Storage.SPM::(start) Backup domain validation failed
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/spm.py", line 488, in start
>  File "/usr/share/vdsm/storage/sp.py", line 1208, in checkBackupDomain
>  File "/usr/share/vdsm/storage/sd.py", line 666, in isBackup
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,872::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,873::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,873::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,874::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,874::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,875::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,875::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,876::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,876::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,877::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,877::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,878::persistentDict::146::Storage.PersistentDict::(transaction) Flushing changes
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,878::persistentDict::265::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['MASTER_VERSION=0', 'ROLE=Regular', '_SHA_CKSUM=1deb73127f89496e2cb17e51bbfae1df6c1879d1']
>88cc2f68-e802-4615-bced-c7a8c4aefa33::WARNING::2013-11-19 21:14:40,880::persistentDict::150::Storage.PersistentDict::(transaction) Error in transaction, rolling back changes
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/persistentDict.py", line 147, in transaction
>  File "/usr/share/vdsm/storage/persistentDict.py", line 266, in flush
>  File "/usr/share/vdsm/storage/fileSD.py", line 58, in writelines
>  File "/usr/share/vdsm/storage/processPool.py", line 33, in wrapper
>  File "/usr/share/vdsm/storage/processPool.py", line 44, in runExternally
>  File "/usr/share/vdsm/storage/processPool.py", line 127, in runExternally
>IOError: [Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata.new'
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,881::persistentDict::265::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['_SHA_CKSUM=da39a3ee5e6b4b0d3255bfef95601890afd80709']
>88cc2f68-e802-4615-bced-c7a8c4aefa33::ERROR::2013-11-19 21:14:40,882::spm::526::Storage.SPM::(start) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/spm.py", line 493, in start
>  File "/usr/share/vdsm/storage/sp.py", line 637, in _regulariseOutdatedMasters
>  File "/usr/share/vdsm/storage/sp.py", line 631, in _setRegularSD
>  File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 152, in transaction
>  File "/usr/share/vdsm/storage/persistentDict.py", line 266, in flush
>  File "/usr/share/vdsm/storage/fileSD.py", line 58, in writelines
>  File "/usr/share/vdsm/storage/processPool.py", line 33, in wrapper
>  File "/usr/share/vdsm/storage/processPool.py", line 44, in runExternally
>  File "/usr/share/vdsm/storage/processPool.py", line 127, in runExternally
>IOError: [Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata.new'
>88cc2f68-e802-4615-bced-c7a8c4aefa33::ERROR::2013-11-19 21:14:40,883::spm::527::Storage.SPM::(start) failed: [Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata.new'
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,884::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`61d8fc27-466f-443d-aa88-41b16bda908b`::Request was made in '/usr/share/vdsm/storage/spm.py' line '584' at '_stop'
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,884::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'exclusive'
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,885::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'exclusive' (1 active user)
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,885::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`61d8fc27-466f-443d-aa88-41b16bda908b`::Granted request
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,887::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e`ReqID=`1908c5be-2e2c-4523-9f6c-f2705c921ff4`::Request was made in '/usr/share/vdsm/storage/spm.py' line '601' at '_stop'
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,887::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e' for lock type 'exclusive'
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,888::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e' is free. Now locking as 'exclusive' (1 active user)
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,889::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e`ReqID=`1908c5be-2e2c-4523-9f6c-f2705c921ff4`::Granted request
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,889::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e'
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,890::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e' (0 active users)
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,891::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e' is free, finding out if anyone is waiting for it.
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,891::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e', Clearing records.
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,892::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30'
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,893::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,893::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,893::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,894::taskManager::118::TaskManager::(unloadTasks) Entry.
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,895::taskManager::122::TaskManager::(unloadTasks) Return
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:40,897::spm::195::Storage.SPM::(__cleanupMasterMount) unmounting /rhev/data-center/mnt/blockSD/6b892067-ac7e-4039-a7be-5b7387525b16/master
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:41,011::fileUtils::112::Storage.Misc.excCmd::(umount) '/usr/bin/sudo -n /bin/umount -f /rhev/data-center/mnt/blockSD/6b892067-ac7e-4039-a7be-5b7387525b16/master' (cwd None)
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:41,392::fileUtils::112::Storage.Misc.excCmd::(umount) SUCCESS: <err> = ''; <rc> = 0
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:41,396::spm::290::Storage.SPM::(__cleanupSPM) cleaning up SPM: c4e5889a-348d-11e2-83fe-9f72369eab30
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:41,396::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:41,397::persistentDict::146::Storage.PersistentDict::(transaction) Flushing changes
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:41,398::persistentDict::265::Storage.PersistentDict::(flush) about to write lines (VGTagMetadataRW)=[u'CLASS=Data', u'DESCRIPTION=OS', u'IOOPTIMEOUTSEC=10', u'LEASERETRIES=3', u'LEASETIMESEC=60', u'LOCKPOLICY=', u'LOCKRENEWALINTERVALSEC=5', u'LOGBLKSIZE=512', u'MASTER_VERSION=1', u'PHYBLKSIZE=512', u'POOL_DESCRIPTION=Default', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_ID=-1', u'POOL_SPM_LVER=277', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'ROLE=Master', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'TYPE=FCP', u'VERSION=2', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', '_SHA_CKSUM=44c47fe1672a8deeb5c0f134ce9469f0ec72525b']
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:41,399::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:41,400::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:41,400::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:41,401::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:41,402::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:41,403::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15910490::DEBUG::2013-11-19 21:14:41,830::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910490::INFO::2013-11-19 21:14:41,831::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=88cc2f68-e802-4615-bced-c7a8c4aefa33)
>Thread-15910490::DEBUG::2013-11-19 21:14:41,832::task::495::TaskManager.Task::(_debug) Task 4fd4c0ae-07da-4e8d-aeae-03d2a80a80b0: moving from state init -> state preparing
>Thread-15910490::DEBUG::2013-11-19 21:14:41,833::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 88cc2f68-e802-4615-bced-c7a8c4aefa33
>Thread-15910490::DEBUG::2013-11-19 21:14:41,833::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}
>Thread-15910490::DEBUG::2013-11-19 21:14:41,834::task::495::TaskManager.Task::(_debug) Task 4fd4c0ae-07da-4e8d-aeae-03d2a80a80b0: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910490::DEBUG::2013-11-19 21:14:41,835::task::495::TaskManager.Task::(_debug) Task 4fd4c0ae-07da-4e8d-aeae-03d2a80a80b0: moving from state preparing -> state finished
>Thread-15910490::DEBUG::2013-11-19 21:14:41,835::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910490::DEBUG::2013-11-19 21:14:41,836::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910490::DEBUG::2013-11-19 21:14:41,836::task::495::TaskManager.Task::(_debug) Task 4fd4c0ae-07da-4e8d-aeae-03d2a80a80b0: ref 0 aborting False
>Thread-15910490::INFO::2013-11-19 21:14:41,837::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:42,075::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:42,078::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:42,079::blockSD::145::storage.Metadata.VGTagMetadataRW::(writelines) Updating metadata adding=MDT__SHA_CKSUM=44c47fe1672a8deeb5c0f134ce9469f0ec72525b, MDT_POOL_SPM_ID=-1 removing=MDT__SHA_CKSUM=a3ed62b0970fa02db432bafe8169c17849a2719b, MDT_POOL_SPM_ID=2
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:42,080::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --deltag MDT__SHA_CKSUM=a3ed62b0970fa02db432bafe8169c17849a2719b --deltag MDT_POOL_SPM_ID=2 --addtag MDT_POOL_SPM_ID=-1 --addtag MDT__SHA_CKSUM=44c47fe1672a8deeb5c0f134ce9469f0ec72525b 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:42,776::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:42,778::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:42,779::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:42,781::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction
>88cc2f68-e802-4615-bced-c7a8c4aefa33::INFO::2013-11-19 21:14:42,782::safelease::83::ClusterLock::(release) Releasing cluster lock for domain 6b892067-ac7e-4039-a7be-5b7387525b16
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:42,784::safelease::84::Storage.Misc.excCmd::(release) '/usr/libexec/vdsm/spmstop.sh 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd /usr/libexec/vdsm/)
>Thread-15910491::DEBUG::2013-11-19 21:14:43,017::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910491::INFO::2013-11-19 21:14:43,020::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=88cc2f68-e802-4615-bced-c7a8c4aefa33)
>Thread-15910491::DEBUG::2013-11-19 21:14:43,020::task::495::TaskManager.Task::(_debug) Task e35dce8b-8aa7-42a4-98e4-7d7f32c4dac9: moving from state init -> state preparing
>Thread-15910491::DEBUG::2013-11-19 21:14:43,021::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 88cc2f68-e802-4615-bced-c7a8c4aefa33
>Thread-15910491::DEBUG::2013-11-19 21:14:43,022::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}
>Thread-15910491::DEBUG::2013-11-19 21:14:43,023::task::495::TaskManager.Task::(_debug) Task e35dce8b-8aa7-42a4-98e4-7d7f32c4dac9: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910491::DEBUG::2013-11-19 21:14:43,023::task::495::TaskManager.Task::(_debug) Task e35dce8b-8aa7-42a4-98e4-7d7f32c4dac9: moving from state preparing -> state finished
>Thread-15910491::DEBUG::2013-11-19 21:14:43,024::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910491::DEBUG::2013-11-19 21:14:43,025::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910491::DEBUG::2013-11-19 21:14:43,025::task::495::TaskManager.Task::(_debug) Task e35dce8b-8aa7-42a4-98e4-7d7f32c4dac9: ref 0 aborting False
>Thread-15910491::INFO::2013-11-19 21:14:43,026::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-1024767::ERROR::2013-11-19 21:14:43,403::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910492::DEBUG::2013-11-19 21:14:44,032::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910492::INFO::2013-11-19 21:14:44,034::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=88cc2f68-e802-4615-bced-c7a8c4aefa33)
>Thread-15910492::DEBUG::2013-11-19 21:14:44,034::task::495::TaskManager.Task::(_debug) Task fd5fbdf2-c480-4d37-97ee-c42712c1c6f9: moving from state init -> state preparing
>Thread-15910492::DEBUG::2013-11-19 21:14:44,035::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 88cc2f68-e802-4615-bced-c7a8c4aefa33
>Thread-15910492::DEBUG::2013-11-19 21:14:44,035::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}
>Thread-15910492::DEBUG::2013-11-19 21:14:44,036::task::495::TaskManager.Task::(_debug) Task fd5fbdf2-c480-4d37-97ee-c42712c1c6f9: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910492::DEBUG::2013-11-19 21:14:44,037::task::495::TaskManager.Task::(_debug) Task fd5fbdf2-c480-4d37-97ee-c42712c1c6f9: moving from state preparing -> state finished
>Thread-15910492::DEBUG::2013-11-19 21:14:44,037::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910492::DEBUG::2013-11-19 21:14:44,038::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910492::DEBUG::2013-11-19 21:14:44,038::task::495::TaskManager.Task::(_debug) Task fd5fbdf2-c480-4d37-97ee-c42712c1c6f9: ref 0 aborting False
>Thread-15910492::INFO::2013-11-19 21:14:44,039::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:44,300::safelease::84::Storage.Misc.excCmd::(release) SUCCESS: <err> = ''; <rc> = 0
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:44,301::safelease::87::ClusterLock::(release) Cluster lock released successfully
>88cc2f68-e802-4615-bced-c7a8c4aefa33::ERROR::2013-11-19 21:14:44,301::task::868::TaskManager.Task::(_setError) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/task.py", line 876, in _run
>  File "/usr/share/vdsm/storage/task.py", line 303, in run
>  File "/usr/share/vdsm/storage/spm.py", line 493, in start
>  File "/usr/share/vdsm/storage/sp.py", line 637, in _regulariseOutdatedMasters
>  File "/usr/share/vdsm/storage/sp.py", line 631, in _setRegularSD
>  File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 152, in transaction
>  File "/usr/share/vdsm/storage/persistentDict.py", line 266, in flush
>  File "/usr/share/vdsm/storage/fileSD.py", line 58, in writelines
>  File "/usr/share/vdsm/storage/processPool.py", line 33, in wrapper
>  File "/usr/share/vdsm/storage/processPool.py", line 44, in runExternally
>  File "/usr/share/vdsm/storage/processPool.py", line 127, in runExternally
>IOError: [Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata.new'
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:44,302::task::495::TaskManager.Task::(_debug) Task 88cc2f68-e802-4615-bced-c7a8c4aefa33: Task._run: 88cc2f68-e802-4615-bced-c7a8c4aefa33 () {} failed - stopping task
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:44,303::task::495::TaskManager.Task::(_debug) Task 88cc2f68-e802-4615-bced-c7a8c4aefa33: stopping in state running (force False)
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:44,304::task::495::TaskManager.Task::(_debug) Task 88cc2f68-e802-4615-bced-c7a8c4aefa33: ref 1 aborting True
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:44,304::task::918::TaskManager.Task::(_runJobs) aborting: Task is aborted: "[Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata.new'" - code 100
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:44,305::task::495::TaskManager.Task::(_debug) Task 88cc2f68-e802-4615-bced-c7a8c4aefa33: ref 0 aborting True
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:44,305::task::495::TaskManager.Task::(_debug) Task 88cc2f68-e802-4615-bced-c7a8c4aefa33: Task._doAbort: force False
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:44,306::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:44,306::task::495::TaskManager.Task::(_debug) Task 88cc2f68-e802-4615-bced-c7a8c4aefa33: moving from state running -> state aborting
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:44,307::task::495::TaskManager.Task::(_debug) Task 88cc2f68-e802-4615-bced-c7a8c4aefa33: _aborting: recover policy auto
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:44,307::task::495::TaskManager.Task::(_debug) Task 88cc2f68-e802-4615-bced-c7a8c4aefa33: moving from state aborting -> state racquiring
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:44,308::task::495::TaskManager.Task::(_debug) Task 88cc2f68-e802-4615-bced-c7a8c4aefa33: moving from state racquiring -> state recovering
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:44,308::task::495::TaskManager.Task::(_debug) Task 88cc2f68-e802-4615-bced-c7a8c4aefa33: _recover
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:44,309::task::495::TaskManager.Task::(_debug) Task 88cc2f68-e802-4615-bced-c7a8c4aefa33: running recovery None
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:44,309::task::495::TaskManager.Task::(_debug) Task 88cc2f68-e802-4615-bced-c7a8c4aefa33: Recover Done: state recovering
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:44,310::task::495::TaskManager.Task::(_debug) Task 88cc2f68-e802-4615-bced-c7a8c4aefa33: moving from state recovering -> state recovered
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:44,310::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:44,311::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:44,312::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:44,312::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:44,313::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:44,313::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>88cc2f68-e802-4615-bced-c7a8c4aefa33::DEBUG::2013-11-19 21:14:44,314::threadPool::47::Misc.ThreadPool::(setRunningTask) Number of running tasks: 0
>Thread-15910493::DEBUG::2013-11-19 21:14:45,045::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910493::INFO::2013-11-19 21:14:45,045::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=88cc2f68-e802-4615-bced-c7a8c4aefa33)
>Thread-15910493::DEBUG::2013-11-19 21:14:45,046::task::495::TaskManager.Task::(_debug) Task 237abb41-7fb6-40d5-97c4-c7273261f8ee: moving from state init -> state preparing
>Thread-15910493::DEBUG::2013-11-19 21:14:45,047::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 88cc2f68-e802-4615-bced-c7a8c4aefa33
>Thread-15910493::DEBUG::2013-11-19 21:14:45,047::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 100, 'message': "[Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata.new'", 'taskState': 'finished', 'taskResult': 'cleanSuccess', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}
>Thread-15910493::DEBUG::2013-11-19 21:14:45,048::task::495::TaskManager.Task::(_debug) Task 237abb41-7fb6-40d5-97c4-c7273261f8ee: finished: {'taskStatus': {'code': 100, 'message': "[Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata.new'", 'taskState': 'finished', 'taskResult': 'cleanSuccess', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910493::DEBUG::2013-11-19 21:14:45,048::task::495::TaskManager.Task::(_debug) Task 237abb41-7fb6-40d5-97c4-c7273261f8ee: moving from state preparing -> state finished
>Thread-15910493::DEBUG::2013-11-19 21:14:45,049::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910493::DEBUG::2013-11-19 21:14:45,049::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910493::DEBUG::2013-11-19 21:14:45,050::task::495::TaskManager.Task::(_debug) Task 237abb41-7fb6-40d5-97c4-c7273261f8ee: ref 0 aborting False
>Thread-15910493::INFO::2013-11-19 21:14:45,051::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 100, 'message': "[Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata.new'", 'taskState': 'finished', 'taskResult': 'cleanSuccess', 'taskID': '88cc2f68-e802-4615-bced-c7a8c4aefa33'}}
>Thread-15910494::DEBUG::2013-11-19 21:14:45,055::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910494::INFO::2013-11-19 21:14:45,056::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910494::DEBUG::2013-11-19 21:14:45,057::task::495::TaskManager.Task::(_debug) Task 1ded1f08-bbb7-48db-b736-c5bc95488ef9: moving from state init -> state preparing
>Thread-15910494::DEBUG::2013-11-19 21:14:45,057::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=Free spmLver=277 spmId=-1
>Thread-15910494::DEBUG::2013-11-19 21:14:45,058::task::495::TaskManager.Task::(_debug) Task 1ded1f08-bbb7-48db-b736-c5bc95488ef9: finished: {'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': 277}}
>Thread-15910494::DEBUG::2013-11-19 21:14:45,059::task::495::TaskManager.Task::(_debug) Task 1ded1f08-bbb7-48db-b736-c5bc95488ef9: moving from state preparing -> state finished
>Thread-15910494::DEBUG::2013-11-19 21:14:45,059::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910494::DEBUG::2013-11-19 21:14:45,060::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910494::DEBUG::2013-11-19 21:14:45,060::task::495::TaskManager.Task::(_debug) Task 1ded1f08-bbb7-48db-b736-c5bc95488ef9: ref 0 aborting False
>Thread-15910494::INFO::2013-11-19 21:14:45,061::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': 277}}
>Thread-15910495::DEBUG::2013-11-19 21:14:45,066::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910495::INFO::2013-11-19 21:14:45,067::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: clearTask, args: ( taskID=88cc2f68-e802-4615-bced-c7a8c4aefa33)
>Thread-15910495::DEBUG::2013-11-19 21:14:45,068::task::495::TaskManager.Task::(_debug) Task 7cf64625-d991-4e01-b46a-2377c6bb7ddf: moving from state init -> state preparing
>Thread-15910495::DEBUG::2013-11-19 21:14:45,068::taskManager::146::TaskManager::(clearTask) Entry. taskID: 88cc2f68-e802-4615-bced-c7a8c4aefa33
>Thread-15910495::DEBUG::2013-11-19 21:14:45,069::taskManager::151::TaskManager::(clearTask) Return.
>Thread-15910495::DEBUG::2013-11-19 21:14:45,069::task::495::TaskManager.Task::(_debug) Task 7cf64625-d991-4e01-b46a-2377c6bb7ddf: finished: None
>Thread-15910495::DEBUG::2013-11-19 21:14:45,070::task::495::TaskManager.Task::(_debug) Task 7cf64625-d991-4e01-b46a-2377c6bb7ddf: moving from state preparing -> state finished
>Thread-15910495::DEBUG::2013-11-19 21:14:45,070::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910495::DEBUG::2013-11-19 21:14:45,071::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910495::DEBUG::2013-11-19 21:14:45,071::task::495::TaskManager.Task::(_debug) Task 7cf64625-d991-4e01-b46a-2377c6bb7ddf: ref 0 aborting False
>Thread-15910495::INFO::2013-11-19 21:14:45,072::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: clearTask, Return response: {'status': {'message': 'OK', 'code': 0}}
>Thread-15910496::DEBUG::2013-11-19 21:14:45,161::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910496::INFO::2013-11-19 21:14:45,162::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910496::DEBUG::2013-11-19 21:14:45,162::task::495::TaskManager.Task::(_debug) Task 9b1e06af-b390-42fd-a143-377c9ba478b5: moving from state init -> state preparing
>Thread-15910496::DEBUG::2013-11-19 21:14:45,163::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=Free spmLver=277 spmId=-1
>Thread-15910496::DEBUG::2013-11-19 21:14:45,163::task::495::TaskManager.Task::(_debug) Task 9b1e06af-b390-42fd-a143-377c9ba478b5: finished: {'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': 277}}
>Thread-15910496::DEBUG::2013-11-19 21:14:45,164::task::495::TaskManager.Task::(_debug) Task 9b1e06af-b390-42fd-a143-377c9ba478b5: moving from state preparing -> state finished
>Thread-15910496::DEBUG::2013-11-19 21:14:45,164::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910496::DEBUG::2013-11-19 21:14:45,165::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910496::DEBUG::2013-11-19 21:14:45,165::task::495::TaskManager.Task::(_debug) Task 9b1e06af-b390-42fd-a143-377c9ba478b5: ref 0 aborting False
>Thread-15910496::INFO::2013-11-19 21:14:45,166::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': 277}}
>Thread-15910497::DEBUG::2013-11-19 21:14:45,189::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910497::INFO::2013-11-19 21:14:45,190::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: spmStart, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 prevID=-1 prevLVER=277 recoveryMode=0 scsiFencing=false maxHostID=250 domVersion=2)
>Thread-15910497::DEBUG::2013-11-19 21:14:45,190::task::495::TaskManager.Task::(_debug) Task 21b4b46d-58c8-40ca-8f03-a8063e50f1b9: moving from state init -> state preparing
>Thread-15910497::DEBUG::2013-11-19 21:14:45,191::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`979b0547-8ba9-4f75-9403-87c9442ca01f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '330' at 'public_spmStart'
>Thread-15910497::DEBUG::2013-11-19 21:14:45,192::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'exclusive'
>Thread-15910497::DEBUG::2013-11-19 21:14:45,193::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'exclusive' (1 active user)
>Thread-15910497::DEBUG::2013-11-19 21:14:45,193::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`979b0547-8ba9-4f75-9403-87c9442ca01f`::Granted request
>Thread-15910497::DEBUG::2013-11-19 21:14:45,194::task::495::TaskManager.Task::(_debug) Task 21b4b46d-58c8-40ca-8f03-a8063e50f1b9: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (exclusive)
>Thread-15910497::DEBUG::2013-11-19 21:14:45,195::task::495::TaskManager.Task::(_debug) Task 21b4b46d-58c8-40ca-8f03-a8063e50f1b9: ref 1 aborting False
>Thread-15910497::DEBUG::2013-11-19 21:14:45,195::task::495::TaskManager.Task::(_debug) Task 21b4b46d-58c8-40ca-8f03-a8063e50f1b9: Prepare: 1 jobs exist, move to acquiring
>Thread-15910497::DEBUG::2013-11-19 21:14:45,196::task::495::TaskManager.Task::(_debug) Task 21b4b46d-58c8-40ca-8f03-a8063e50f1b9: moving from state preparing -> state acquiring
>Thread-15910497::DEBUG::2013-11-19 21:14:45,196::task::495::TaskManager.Task::(_debug) Task 21b4b46d-58c8-40ca-8f03-a8063e50f1b9: moving from state acquiring -> state queued
>Thread-15910497::DEBUG::2013-11-19 21:14:45,197::taskManager::37::TaskManager::(_queueTask) queueing task: 21b4b46d-58c8-40ca-8f03-a8063e50f1b9
>Thread-15910497::DEBUG::2013-11-19 21:14:45,197::taskManager::43::TaskManager::(_queueTask) task queued: 21b4b46d-58c8-40ca-8f03-a8063e50f1b9
>16fdd9ed-6b31-4b43-9b39-df445f20d9f6::DEBUG::2013-11-19 21:14:45,198::threadPool::47::Misc.ThreadPool::(setRunningTask) Number of running tasks: 1
>Thread-15910497::DEBUG::2013-11-19 21:14:45,199::task::1183::TaskManager.Task::(prepare) returning
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:45,199::threadPool::192::Misc.ThreadPool.WorkerThread::(run) Task: 21b4b46d-58c8-40ca-8f03-a8063e50f1b9 running: <bound method Task.commit of <storage.task.Task instance at 0x7f487c121ef0>> with: None
>Thread-15910497::DEBUG::2013-11-19 21:14:45,200::task::495::TaskManager.Task::(_debug) Task 21b4b46d-58c8-40ca-8f03-a8063e50f1b9: ref 0 aborting False
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:45,200::task::1195::TaskManager.Task::(commit) Commiting task: 21b4b46d-58c8-40ca-8f03-a8063e50f1b9
>Thread-15910497::INFO::2013-11-19 21:14:45,201::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: spmStart, Return response: {'status': {'message': 'OK', 'code': 0}, 'uuid': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:45,202::task::495::TaskManager.Task::(_debug) Task 21b4b46d-58c8-40ca-8f03-a8063e50f1b9: moving from state queued -> state running
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:45,203::task::907::TaskManager.Task::(_runJobs) Task.run: running job 0: spmStart: <bound method SPM.start of <storage.spm.SPM instance at 0x1bf9f38>> (args: ('c4e5889a-348d-11e2-83fe-9f72369eab30', -1, '277', 0, 'false', 250, 2) kwargs: {})
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:45,204::task::300::TaskManager.Task::(run) Job.run: running spmStart: <bound method SPM.start of <storage.spm.SPM instance at 0x1bf9f38>> (args: ('c4e5889a-348d-11e2-83fe-9f72369eab30', -1, '277', 0, 'false', 250, 2) kwargs: {}) callback None
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:45,204::spm::433::Storage.SPM::(start) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:45,205::misc::1010::SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads)
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:45,205::misc::1012::SamplingMethod::(__call__) Got in to sampling method
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:45,206::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:45,207::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:45,207::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:45,208::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:45,209::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:45,210::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 21:14:45,418::lvm::439::OperationMutex::(_reloadvgs) Got the operational mutex
>Thread-42838::DEBUG::2013-11-19 21:14:45,420::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:46,002::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:46,006::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_LVER=277', u'POOL_SPM_ID=-1', u'_SHA_CKSUM=44c47fe1672a8deeb5c0f134ce9469f0ec72525b']
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:46,008::misc::1020::SamplingMethod::(__call__) Returning last result
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:46,008::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm reload operation' is holding the operation mutex, waiting...
>Thread-42838::DEBUG::2013-11-19 21:14:46,190::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-42838::DEBUG::2013-11-19 21:14:46,194::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:46,196::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:46,199::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:46,201::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:46,202::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:46,204::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:46,206::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15910498::DEBUG::2013-11-19 21:14:46,216::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910498::INFO::2013-11-19 21:14:46,220::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=21b4b46d-58c8-40ca-8f03-a8063e50f1b9)
>Thread-15910498::DEBUG::2013-11-19 21:14:46,461::task::495::TaskManager.Task::(_debug) Task 6a853d10-609d-42ec-a8a8-7d4f0ca40aeb: moving from state init -> state preparing
>Thread-15910498::DEBUG::2013-11-19 21:14:46,482::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 21b4b46d-58c8-40ca-8f03-a8063e50f1b9
>Thread-15910498::DEBUG::2013-11-19 21:14:46,483::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}
>Thread-15910498::DEBUG::2013-11-19 21:14:46,484::task::495::TaskManager.Task::(_debug) Task 6a853d10-609d-42ec-a8a8-7d4f0ca40aeb: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910498::DEBUG::2013-11-19 21:14:46,485::task::495::TaskManager.Task::(_debug) Task 6a853d10-609d-42ec-a8a8-7d4f0ca40aeb: moving from state preparing -> state finished
>Thread-15910498::DEBUG::2013-11-19 21:14:46,485::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910498::DEBUG::2013-11-19 21:14:46,486::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910498::DEBUG::2013-11-19 21:14:46,486::task::495::TaskManager.Task::(_debug) Task 6a853d10-609d-42ec-a8a8-7d4f0ca40aeb: ref 0 aborting False
>Thread-15910498::INFO::2013-11-19 21:14:46,487::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:46,848::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:46,851::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:46,852::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_LVER=277', u'POOL_SPM_ID=-1', u'_SHA_CKSUM=44c47fe1672a8deeb5c0f134ce9469f0ec72525b']
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:46,855::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:46,857::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:46,858::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:46,859::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:46,859::persistentDict::163::Storage.PersistentDict::(__init__) Created a persistant dict with LvMetadataRW backend
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:46,860::blockSD::164::Storage.Misc.excCmd::(readlines) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:47,261::blockSD::164::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000302292 s, 6.8 MB/s\n'; <rc> = 0
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:47,262::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000302292 s, 6.8 MB/s'], size: 2048
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:47,263::persistentDict::204::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[]
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::WARNING::2013-11-19 21:14:47,264::persistentDict::226::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:47,264::persistentDict::163::Storage.PersistentDict::(__init__) Created a persistant dict with VGTagMetadataRW backend
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:47,265::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:47,266::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:47,266::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:47,267::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:47,268::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:47,269::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15910500::DEBUG::2013-11-19 21:14:47,497::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910500::INFO::2013-11-19 21:14:47,500::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=21b4b46d-58c8-40ca-8f03-a8063e50f1b9)
>Thread-15910500::DEBUG::2013-11-19 21:14:47,500::task::495::TaskManager.Task::(_debug) Task c24a96c2-2059-46ad-acbf-636f7fda04ce: moving from state init -> state preparing
>Thread-15910500::DEBUG::2013-11-19 21:14:47,501::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 21b4b46d-58c8-40ca-8f03-a8063e50f1b9
>Thread-15910500::DEBUG::2013-11-19 21:14:47,502::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}
>Thread-15910500::DEBUG::2013-11-19 21:14:47,502::task::495::TaskManager.Task::(_debug) Task c24a96c2-2059-46ad-acbf-636f7fda04ce: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910500::DEBUG::2013-11-19 21:14:47,503::task::495::TaskManager.Task::(_debug) Task c24a96c2-2059-46ad-acbf-636f7fda04ce: moving from state preparing -> state finished
>Thread-15910500::DEBUG::2013-11-19 21:14:47,504::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910500::DEBUG::2013-11-19 21:14:47,504::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910500::DEBUG::2013-11-19 21:14:47,505::task::495::TaskManager.Task::(_debug) Task c24a96c2-2059-46ad-acbf-636f7fda04ce: ref 0 aborting False
>Thread-15910500::INFO::2013-11-19 21:14:47,506::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:47,931::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:47,938::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:47,939::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_LVER=277', u'POOL_SPM_ID=-1', u'_SHA_CKSUM=44c47fe1672a8deeb5c0f134ce9469f0ec72525b']
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:47,940::safelease::52::ClusterLock::(acquire) Acquiring cluster lock for domain 6b892067-ac7e-4039-a7be-5b7387525b16
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:14:47,941::safelease::61::Storage.Misc.excCmd::(acquire) '/usr/bin/sudo -n /usr/bin/setsid /usr/bin/ionice -c1 -n0 /bin/su vdsm -s /bin/sh -c "/usr/libexec/vdsm/spmprotect.sh start 6b892067-ac7e-4039-a7be-5b7387525b16 2 5 /dev/6b892067-ac7e-4039-a7be-5b7387525b16/leases 60000 10000 3"' (cwd /usr/libexec/vdsm/)
>Thread-15910501::DEBUG::2013-11-19 21:14:48,514::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910501::INFO::2013-11-19 21:14:48,515::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=21b4b46d-58c8-40ca-8f03-a8063e50f1b9)
>Thread-15910501::DEBUG::2013-11-19 21:14:48,516::task::495::TaskManager.Task::(_debug) Task 354b09fe-08cd-449b-be62-b5a08279cea7: moving from state init -> state preparing
>Thread-15910501::DEBUG::2013-11-19 21:14:48,517::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 21b4b46d-58c8-40ca-8f03-a8063e50f1b9
>Thread-15910501::DEBUG::2013-11-19 21:14:48,517::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}
>Thread-15910501::DEBUG::2013-11-19 21:14:48,518::task::495::TaskManager.Task::(_debug) Task 354b09fe-08cd-449b-be62-b5a08279cea7: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910501::DEBUG::2013-11-19 21:14:48,519::task::495::TaskManager.Task::(_debug) Task 354b09fe-08cd-449b-be62-b5a08279cea7: moving from state preparing -> state finished
>Thread-15910501::DEBUG::2013-11-19 21:14:48,519::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910501::DEBUG::2013-11-19 21:14:48,520::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910501::DEBUG::2013-11-19 21:14:48,520::task::495::TaskManager.Task::(_debug) Task 354b09fe-08cd-449b-be62-b5a08279cea7: ref 0 aborting False
>Thread-15910501::INFO::2013-11-19 21:14:48,521::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910503::DEBUG::2013-11-19 21:14:49,527::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910503::INFO::2013-11-19 21:14:49,528::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=21b4b46d-58c8-40ca-8f03-a8063e50f1b9)
>Thread-15910503::DEBUG::2013-11-19 21:14:49,528::task::495::TaskManager.Task::(_debug) Task 96dcbc6c-6587-403a-8941-ebae2206d48e: moving from state init -> state preparing
>Thread-15910503::DEBUG::2013-11-19 21:14:49,529::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 21b4b46d-58c8-40ca-8f03-a8063e50f1b9
>Thread-15910503::DEBUG::2013-11-19 21:14:49,529::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}
>Thread-15910503::DEBUG::2013-11-19 21:14:49,530::task::495::TaskManager.Task::(_debug) Task 96dcbc6c-6587-403a-8941-ebae2206d48e: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910503::DEBUG::2013-11-19 21:14:49,531::task::495::TaskManager.Task::(_debug) Task 96dcbc6c-6587-403a-8941-ebae2206d48e: moving from state preparing -> state finished
>Thread-15910503::DEBUG::2013-11-19 21:14:49,531::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910503::DEBUG::2013-11-19 21:14:49,532::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910503::DEBUG::2013-11-19 21:14:49,532::task::495::TaskManager.Task::(_debug) Task 96dcbc6c-6587-403a-8941-ebae2206d48e: ref 0 aborting False
>Thread-15910503::INFO::2013-11-19 21:14:49,533::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910504::DEBUG::2013-11-19 21:14:50,539::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910504::INFO::2013-11-19 21:14:50,539::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=21b4b46d-58c8-40ca-8f03-a8063e50f1b9)
>Thread-15910504::DEBUG::2013-11-19 21:14:50,540::task::495::TaskManager.Task::(_debug) Task eb36fad6-180a-4abc-8f7b-9c3d950412ef: moving from state init -> state preparing
>Thread-15910504::DEBUG::2013-11-19 21:14:50,541::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 21b4b46d-58c8-40ca-8f03-a8063e50f1b9
>Thread-15910504::DEBUG::2013-11-19 21:14:50,541::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}
>Thread-15910504::DEBUG::2013-11-19 21:14:50,542::task::495::TaskManager.Task::(_debug) Task eb36fad6-180a-4abc-8f7b-9c3d950412ef: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910504::DEBUG::2013-11-19 21:14:50,542::task::495::TaskManager.Task::(_debug) Task eb36fad6-180a-4abc-8f7b-9c3d950412ef: moving from state preparing -> state finished
>Thread-15910504::DEBUG::2013-11-19 21:14:50,543::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910504::DEBUG::2013-11-19 21:14:50,543::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910504::DEBUG::2013-11-19 21:14:50,544::task::495::TaskManager.Task::(_debug) Task eb36fad6-180a-4abc-8f7b-9c3d950412ef: ref 0 aborting False
>Thread-15910504::INFO::2013-11-19 21:14:50,544::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910506::DEBUG::2013-11-19 21:14:51,550::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910506::INFO::2013-11-19 21:14:51,551::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=21b4b46d-58c8-40ca-8f03-a8063e50f1b9)
>Thread-15910506::DEBUG::2013-11-19 21:14:51,552::task::495::TaskManager.Task::(_debug) Task 639fed28-b7e5-4827-bd65-ffa22b298a6d: moving from state init -> state preparing
>Thread-15910506::DEBUG::2013-11-19 21:14:51,552::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 21b4b46d-58c8-40ca-8f03-a8063e50f1b9
>Thread-15910506::DEBUG::2013-11-19 21:14:51,553::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}
>Thread-15910506::DEBUG::2013-11-19 21:14:51,554::task::495::TaskManager.Task::(_debug) Task 639fed28-b7e5-4827-bd65-ffa22b298a6d: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910506::DEBUG::2013-11-19 21:14:51,554::task::495::TaskManager.Task::(_debug) Task 639fed28-b7e5-4827-bd65-ffa22b298a6d: moving from state preparing -> state finished
>Thread-15910506::DEBUG::2013-11-19 21:14:51,555::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910506::DEBUG::2013-11-19 21:14:51,555::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910506::DEBUG::2013-11-19 21:14:51,556::task::495::TaskManager.Task::(_debug) Task 639fed28-b7e5-4827-bd65-ffa22b298a6d: ref 0 aborting False
>Thread-15910506::INFO::2013-11-19 21:14:51,556::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910507::DEBUG::2013-11-19 21:14:52,562::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910507::INFO::2013-11-19 21:14:52,563::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=21b4b46d-58c8-40ca-8f03-a8063e50f1b9)
>Thread-15910507::DEBUG::2013-11-19 21:14:52,564::task::495::TaskManager.Task::(_debug) Task db428b00-7540-40b1-a649-ecc2fb7d3ce8: moving from state init -> state preparing
>Thread-15910507::DEBUG::2013-11-19 21:14:52,564::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 21b4b46d-58c8-40ca-8f03-a8063e50f1b9
>Thread-15910507::DEBUG::2013-11-19 21:14:52,565::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}
>Thread-15910507::DEBUG::2013-11-19 21:14:52,565::task::495::TaskManager.Task::(_debug) Task db428b00-7540-40b1-a649-ecc2fb7d3ce8: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910507::DEBUG::2013-11-19 21:14:52,566::task::495::TaskManager.Task::(_debug) Task db428b00-7540-40b1-a649-ecc2fb7d3ce8: moving from state preparing -> state finished
>Thread-15910507::DEBUG::2013-11-19 21:14:52,566::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910507::DEBUG::2013-11-19 21:14:52,567::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910507::DEBUG::2013-11-19 21:14:52,568::task::495::TaskManager.Task::(_debug) Task db428b00-7540-40b1-a649-ecc2fb7d3ce8: ref 0 aborting False
>Thread-15910507::INFO::2013-11-19 21:14:52,568::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-1024767::ERROR::2013-11-19 21:14:53,497::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910509::DEBUG::2013-11-19 21:14:53,574::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910509::INFO::2013-11-19 21:14:53,575::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=21b4b46d-58c8-40ca-8f03-a8063e50f1b9)
>Thread-15910509::DEBUG::2013-11-19 21:14:53,575::task::495::TaskManager.Task::(_debug) Task 01ab3932-9a46-45bf-be00-bf197ed23e32: moving from state init -> state preparing
>Thread-15910509::DEBUG::2013-11-19 21:14:53,576::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 21b4b46d-58c8-40ca-8f03-a8063e50f1b9
>Thread-15910509::DEBUG::2013-11-19 21:14:53,576::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}
>Thread-15910509::DEBUG::2013-11-19 21:14:53,577::task::495::TaskManager.Task::(_debug) Task 01ab3932-9a46-45bf-be00-bf197ed23e32: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910509::DEBUG::2013-11-19 21:14:53,577::task::495::TaskManager.Task::(_debug) Task 01ab3932-9a46-45bf-be00-bf197ed23e32: moving from state preparing -> state finished
>Thread-15910509::DEBUG::2013-11-19 21:14:53,578::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910509::DEBUG::2013-11-19 21:14:53,578::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910509::DEBUG::2013-11-19 21:14:53,579::task::495::TaskManager.Task::(_debug) Task 01ab3932-9a46-45bf-be00-bf197ed23e32: ref 0 aborting False
>Thread-15910509::INFO::2013-11-19 21:14:53,580::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910510::DEBUG::2013-11-19 21:14:54,585::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910510::INFO::2013-11-19 21:14:54,586::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=21b4b46d-58c8-40ca-8f03-a8063e50f1b9)
>Thread-15910510::DEBUG::2013-11-19 21:14:54,587::task::495::TaskManager.Task::(_debug) Task 1b87215e-b204-45de-a131-b8140bd477f0: moving from state init -> state preparing
>Thread-15910510::DEBUG::2013-11-19 21:14:54,587::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 21b4b46d-58c8-40ca-8f03-a8063e50f1b9
>Thread-15910510::DEBUG::2013-11-19 21:14:54,588::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}
>Thread-15910510::DEBUG::2013-11-19 21:14:54,588::task::495::TaskManager.Task::(_debug) Task 1b87215e-b204-45de-a131-b8140bd477f0: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910510::DEBUG::2013-11-19 21:14:54,589::task::495::TaskManager.Task::(_debug) Task 1b87215e-b204-45de-a131-b8140bd477f0: moving from state preparing -> state finished
>Thread-15910510::DEBUG::2013-11-19 21:14:54,589::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910510::DEBUG::2013-11-19 21:14:54,590::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910510::DEBUG::2013-11-19 21:14:54,590::task::495::TaskManager.Task::(_debug) Task 1b87215e-b204-45de-a131-b8140bd477f0: ref 0 aborting False
>Thread-15910510::INFO::2013-11-19 21:14:54,591::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910511::INFO::2013-11-19 21:14:55,219::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910511::DEBUG::2013-11-19 21:14:55,222::task::495::TaskManager.Task::(_debug) Task 83cb05c0-5d22-449c-801c-f89c983b9794: moving from state init -> state preparing
>Thread-15910511::DEBUG::2013-11-19 21:14:55,224::task::495::TaskManager.Task::(_debug) Task 83cb05c0-5d22-449c-801c-f89c983b9794: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000824928283691', 'lastCheck': 1384895686.858032, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00170588493347', 'lastCheck': 1384895688.7861741, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895693.498703, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0021698474884', 'lastCheck': 1384895692.501792, 'valid': True, 'code': 0}}
>Thread-15910511::DEBUG::2013-11-19 21:14:55,225::task::495::TaskManager.Task::(_debug) Task 83cb05c0-5d22-449c-801c-f89c983b9794: moving from state preparing -> state finished
>Thread-15910511::DEBUG::2013-11-19 21:14:55,226::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910511::DEBUG::2013-11-19 21:14:55,227::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910511::DEBUG::2013-11-19 21:14:55,227::task::495::TaskManager.Task::(_debug) Task 83cb05c0-5d22-449c-801c-f89c983b9794: ref 0 aborting False
>Thread-15910511::INFO::2013-11-19 21:14:55,228::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000824928283691', 'lastCheck': 1384895686.858032, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00170588493347', 'lastCheck': 1384895688.7861741, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895693.498703, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0021698474884', 'lastCheck': 1384895692.501792, 'valid': True, 'code': 0}}
>Thread-15910513::DEBUG::2013-11-19 21:14:55,598::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910513::INFO::2013-11-19 21:14:55,599::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=21b4b46d-58c8-40ca-8f03-a8063e50f1b9)
>Thread-15910513::DEBUG::2013-11-19 21:14:55,600::task::495::TaskManager.Task::(_debug) Task 6559708c-0be9-4fcd-a750-ed411969e554: moving from state init -> state preparing
>Thread-15910513::DEBUG::2013-11-19 21:14:55,600::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 21b4b46d-58c8-40ca-8f03-a8063e50f1b9
>Thread-15910513::DEBUG::2013-11-19 21:14:55,601::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}
>Thread-15910513::DEBUG::2013-11-19 21:14:55,601::task::495::TaskManager.Task::(_debug) Task 6559708c-0be9-4fcd-a750-ed411969e554: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910513::DEBUG::2013-11-19 21:14:55,602::task::495::TaskManager.Task::(_debug) Task 6559708c-0be9-4fcd-a750-ed411969e554: moving from state preparing -> state finished
>Thread-15910513::DEBUG::2013-11-19 21:14:55,602::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910513::DEBUG::2013-11-19 21:14:55,603::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910513::DEBUG::2013-11-19 21:14:55,603::task::495::TaskManager.Task::(_debug) Task 6559708c-0be9-4fcd-a750-ed411969e554: ref 0 aborting False
>Thread-15910513::INFO::2013-11-19 21:14:55,604::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910514::DEBUG::2013-11-19 21:14:56,610::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910514::INFO::2013-11-19 21:14:56,611::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=21b4b46d-58c8-40ca-8f03-a8063e50f1b9)
>Thread-15910514::DEBUG::2013-11-19 21:14:56,611::task::495::TaskManager.Task::(_debug) Task 54bb086e-0d6b-4584-9107-7631194df15f: moving from state init -> state preparing
>Thread-15910514::DEBUG::2013-11-19 21:14:56,612::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 21b4b46d-58c8-40ca-8f03-a8063e50f1b9
>Thread-15910514::DEBUG::2013-11-19 21:14:56,612::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}
>Thread-15910514::DEBUG::2013-11-19 21:14:56,613::task::495::TaskManager.Task::(_debug) Task 54bb086e-0d6b-4584-9107-7631194df15f: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910514::DEBUG::2013-11-19 21:14:56,613::task::495::TaskManager.Task::(_debug) Task 54bb086e-0d6b-4584-9107-7631194df15f: moving from state preparing -> state finished
>Thread-15910514::DEBUG::2013-11-19 21:14:56,614::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910514::DEBUG::2013-11-19 21:14:56,614::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910514::DEBUG::2013-11-19 21:14:56,615::task::495::TaskManager.Task::(_debug) Task 54bb086e-0d6b-4584-9107-7631194df15f: ref 0 aborting False
>Thread-15910514::INFO::2013-11-19 21:14:56,616::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910516::DEBUG::2013-11-19 21:14:57,621::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910516::INFO::2013-11-19 21:14:57,622::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=21b4b46d-58c8-40ca-8f03-a8063e50f1b9)
>Thread-15910516::DEBUG::2013-11-19 21:14:57,623::task::495::TaskManager.Task::(_debug) Task 02038fd4-0929-46bd-8979-dc485ee17628: moving from state init -> state preparing
>Thread-15910516::DEBUG::2013-11-19 21:14:57,623::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 21b4b46d-58c8-40ca-8f03-a8063e50f1b9
>Thread-15910516::DEBUG::2013-11-19 21:14:57,624::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}
>Thread-15910516::DEBUG::2013-11-19 21:14:57,625::task::495::TaskManager.Task::(_debug) Task 02038fd4-0929-46bd-8979-dc485ee17628: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910516::DEBUG::2013-11-19 21:14:57,625::task::495::TaskManager.Task::(_debug) Task 02038fd4-0929-46bd-8979-dc485ee17628: moving from state preparing -> state finished
>Thread-15910516::DEBUG::2013-11-19 21:14:57,626::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910516::DEBUG::2013-11-19 21:14:57,626::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910516::DEBUG::2013-11-19 21:14:57,627::task::495::TaskManager.Task::(_debug) Task 02038fd4-0929-46bd-8979-dc485ee17628: ref 0 aborting False
>Thread-15910516::INFO::2013-11-19 21:14:57,627::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910517::DEBUG::2013-11-19 21:14:58,633::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910517::INFO::2013-11-19 21:14:58,634::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=21b4b46d-58c8-40ca-8f03-a8063e50f1b9)
>Thread-15910517::DEBUG::2013-11-19 21:14:58,643::task::495::TaskManager.Task::(_debug) Task 22ca50db-aed2-478c-8366-b471cd13a14d: moving from state init -> state preparing
>Thread-15910517::DEBUG::2013-11-19 21:14:58,644::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 21b4b46d-58c8-40ca-8f03-a8063e50f1b9
>Thread-15910517::DEBUG::2013-11-19 21:14:58,644::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}
>Thread-15910517::DEBUG::2013-11-19 21:14:58,645::task::495::TaskManager.Task::(_debug) Task 22ca50db-aed2-478c-8366-b471cd13a14d: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910517::DEBUG::2013-11-19 21:14:58,646::task::495::TaskManager.Task::(_debug) Task 22ca50db-aed2-478c-8366-b471cd13a14d: moving from state preparing -> state finished
>Thread-15910517::DEBUG::2013-11-19 21:14:58,646::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910517::DEBUG::2013-11-19 21:14:58,647::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910517::DEBUG::2013-11-19 21:14:58,647::task::495::TaskManager.Task::(_debug) Task 22ca50db-aed2-478c-8366-b471cd13a14d: ref 0 aborting False
>Thread-15910517::INFO::2013-11-19 21:14:58,648::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910519::DEBUG::2013-11-19 21:14:59,654::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910519::INFO::2013-11-19 21:14:59,654::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=21b4b46d-58c8-40ca-8f03-a8063e50f1b9)
>Thread-15910519::DEBUG::2013-11-19 21:14:59,655::task::495::TaskManager.Task::(_debug) Task 2faa9699-8774-43da-97a9-c4e9e81c1d5a: moving from state init -> state preparing
>Thread-15910519::DEBUG::2013-11-19 21:14:59,656::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 21b4b46d-58c8-40ca-8f03-a8063e50f1b9
>Thread-15910519::DEBUG::2013-11-19 21:14:59,656::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}
>Thread-15910519::DEBUG::2013-11-19 21:14:59,657::task::495::TaskManager.Task::(_debug) Task 2faa9699-8774-43da-97a9-c4e9e81c1d5a: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910519::DEBUG::2013-11-19 21:14:59,657::task::495::TaskManager.Task::(_debug) Task 2faa9699-8774-43da-97a9-c4e9e81c1d5a: moving from state preparing -> state finished
>Thread-15910519::DEBUG::2013-11-19 21:14:59,658::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910519::DEBUG::2013-11-19 21:14:59,658::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910519::DEBUG::2013-11-19 21:14:59,659::task::495::TaskManager.Task::(_debug) Task 2faa9699-8774-43da-97a9-c4e9e81c1d5a: ref 0 aborting False
>Thread-15910519::INFO::2013-11-19 21:14:59,659::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910520::DEBUG::2013-11-19 21:15:00,665::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910520::INFO::2013-11-19 21:15:00,666::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=21b4b46d-58c8-40ca-8f03-a8063e50f1b9)
>Thread-15910520::DEBUG::2013-11-19 21:15:00,667::task::495::TaskManager.Task::(_debug) Task dc6bd7a0-b338-4e44-939b-50b173ddda18: moving from state init -> state preparing
>Thread-15910520::DEBUG::2013-11-19 21:15:00,667::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 21b4b46d-58c8-40ca-8f03-a8063e50f1b9
>Thread-15910520::DEBUG::2013-11-19 21:15:00,668::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}
>Thread-15910520::DEBUG::2013-11-19 21:15:00,669::task::495::TaskManager.Task::(_debug) Task dc6bd7a0-b338-4e44-939b-50b173ddda18: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910520::DEBUG::2013-11-19 21:15:00,669::task::495::TaskManager.Task::(_debug) Task dc6bd7a0-b338-4e44-939b-50b173ddda18: moving from state preparing -> state finished
>Thread-15910520::DEBUG::2013-11-19 21:15:00,670::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910520::DEBUG::2013-11-19 21:15:00,670::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910520::DEBUG::2013-11-19 21:15:00,671::task::495::TaskManager.Task::(_debug) Task dc6bd7a0-b338-4e44-939b-50b173ddda18: ref 0 aborting False
>Thread-15910520::INFO::2013-11-19 21:15:00,671::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910522::DEBUG::2013-11-19 21:15:01,677::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910522::INFO::2013-11-19 21:15:01,678::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=21b4b46d-58c8-40ca-8f03-a8063e50f1b9)
>Thread-15910522::DEBUG::2013-11-19 21:15:01,679::task::495::TaskManager.Task::(_debug) Task b55ea658-4986-420e-914e-2442013aa4f8: moving from state init -> state preparing
>Thread-15910522::DEBUG::2013-11-19 21:15:01,679::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 21b4b46d-58c8-40ca-8f03-a8063e50f1b9
>Thread-15910522::DEBUG::2013-11-19 21:15:01,680::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}
>Thread-15910522::DEBUG::2013-11-19 21:15:01,680::task::495::TaskManager.Task::(_debug) Task b55ea658-4986-420e-914e-2442013aa4f8: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910522::DEBUG::2013-11-19 21:15:01,681::task::495::TaskManager.Task::(_debug) Task b55ea658-4986-420e-914e-2442013aa4f8: moving from state preparing -> state finished
>Thread-15910522::DEBUG::2013-11-19 21:15:01,681::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910522::DEBUG::2013-11-19 21:15:01,682::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910522::DEBUG::2013-11-19 21:15:01,682::task::495::TaskManager.Task::(_debug) Task b55ea658-4986-420e-914e-2442013aa4f8: ref 0 aborting False
>Thread-15910522::INFO::2013-11-19 21:15:01,683::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910523::DEBUG::2013-11-19 21:15:02,689::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910523::INFO::2013-11-19 21:15:02,690::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=21b4b46d-58c8-40ca-8f03-a8063e50f1b9)
>Thread-15910523::DEBUG::2013-11-19 21:15:02,690::task::495::TaskManager.Task::(_debug) Task f3d7ad6c-7ab1-489e-bcb8-33eb51c577da: moving from state init -> state preparing
>Thread-15910523::DEBUG::2013-11-19 21:15:02,691::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 21b4b46d-58c8-40ca-8f03-a8063e50f1b9
>Thread-15910523::DEBUG::2013-11-19 21:15:02,691::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}
>Thread-15910523::DEBUG::2013-11-19 21:15:02,692::task::495::TaskManager.Task::(_debug) Task f3d7ad6c-7ab1-489e-bcb8-33eb51c577da: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910523::DEBUG::2013-11-19 21:15:02,692::task::495::TaskManager.Task::(_debug) Task f3d7ad6c-7ab1-489e-bcb8-33eb51c577da: moving from state preparing -> state finished
>Thread-15910523::DEBUG::2013-11-19 21:15:02,693::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910523::DEBUG::2013-11-19 21:15:02,694::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910523::DEBUG::2013-11-19 21:15:02,694::task::495::TaskManager.Task::(_debug) Task f3d7ad6c-7ab1-489e-bcb8-33eb51c577da: ref 0 aborting False
>Thread-15910523::INFO::2013-11-19 21:15:02,695::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-1024767::ERROR::2013-11-19 21:15:03,511::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910525::DEBUG::2013-11-19 21:15:03,700::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910525::INFO::2013-11-19 21:15:03,701::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=21b4b46d-58c8-40ca-8f03-a8063e50f1b9)
>Thread-15910525::DEBUG::2013-11-19 21:15:03,713::task::495::TaskManager.Task::(_debug) Task be899098-1413-45b3-988f-8eec5f1747d6: moving from state init -> state preparing
>Thread-15910525::DEBUG::2013-11-19 21:15:03,713::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 21b4b46d-58c8-40ca-8f03-a8063e50f1b9
>Thread-15910525::DEBUG::2013-11-19 21:15:03,714::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}
>Thread-15910525::DEBUG::2013-11-19 21:15:03,714::task::495::TaskManager.Task::(_debug) Task be899098-1413-45b3-988f-8eec5f1747d6: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910525::DEBUG::2013-11-19 21:15:03,715::task::495::TaskManager.Task::(_debug) Task be899098-1413-45b3-988f-8eec5f1747d6: moving from state preparing -> state finished
>Thread-15910525::DEBUG::2013-11-19 21:15:03,716::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910525::DEBUG::2013-11-19 21:15:03,716::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910525::DEBUG::2013-11-19 21:15:03,717::task::495::TaskManager.Task::(_debug) Task be899098-1413-45b3-988f-8eec5f1747d6: ref 0 aborting False
>Thread-15910525::INFO::2013-11-19 21:15:03,717::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910526::DEBUG::2013-11-19 21:15:04,723::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910526::INFO::2013-11-19 21:15:04,724::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=21b4b46d-58c8-40ca-8f03-a8063e50f1b9)
>Thread-15910526::DEBUG::2013-11-19 21:15:04,724::task::495::TaskManager.Task::(_debug) Task e9800a1f-1535-4037-a9b0-03203dc22042: moving from state init -> state preparing
>Thread-15910526::DEBUG::2013-11-19 21:15:04,725::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 21b4b46d-58c8-40ca-8f03-a8063e50f1b9
>Thread-15910526::DEBUG::2013-11-19 21:15:04,725::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}
>Thread-15910526::DEBUG::2013-11-19 21:15:04,726::task::495::TaskManager.Task::(_debug) Task e9800a1f-1535-4037-a9b0-03203dc22042: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910526::DEBUG::2013-11-19 21:15:04,726::task::495::TaskManager.Task::(_debug) Task e9800a1f-1535-4037-a9b0-03203dc22042: moving from state preparing -> state finished
>Thread-15910526::DEBUG::2013-11-19 21:15:04,727::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910526::DEBUG::2013-11-19 21:15:04,727::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910526::DEBUG::2013-11-19 21:15:04,728::task::495::TaskManager.Task::(_debug) Task e9800a1f-1535-4037-a9b0-03203dc22042: ref 0 aborting False
>Thread-15910526::INFO::2013-11-19 21:15:04,729::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910527::INFO::2013-11-19 21:15:05,353::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910527::DEBUG::2013-11-19 21:15:05,354::task::495::TaskManager.Task::(_debug) Task b6530d88-7669-4dfe-afd9-4d0aa325f1a8: moving from state init -> state preparing
>Thread-15910527::DEBUG::2013-11-19 21:15:05,354::task::495::TaskManager.Task::(_debug) Task b6530d88-7669-4dfe-afd9-4d0aa325f1a8: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000344038009644', 'lastCheck': 1384895696.870894, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00145101547241', 'lastCheck': 1384895698.8019259, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895703.512305, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000396966934204', 'lastCheck': 1384895702.5128241, 'valid': True, 'code': 0}}
>Thread-15910527::DEBUG::2013-11-19 21:15:05,355::task::495::TaskManager.Task::(_debug) Task b6530d88-7669-4dfe-afd9-4d0aa325f1a8: moving from state preparing -> state finished
>Thread-15910527::DEBUG::2013-11-19 21:15:05,356::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910527::DEBUG::2013-11-19 21:15:05,356::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910527::DEBUG::2013-11-19 21:15:05,357::task::495::TaskManager.Task::(_debug) Task b6530d88-7669-4dfe-afd9-4d0aa325f1a8: ref 0 aborting False
>Thread-15910527::INFO::2013-11-19 21:15:05,357::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000344038009644', 'lastCheck': 1384895696.870894, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00145101547241', 'lastCheck': 1384895698.8019259, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895703.512305, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000396966934204', 'lastCheck': 1384895702.5128241, 'valid': True, 'code': 0}}
>Thread-15910529::DEBUG::2013-11-19 21:15:05,734::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910529::INFO::2013-11-19 21:15:05,735::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=21b4b46d-58c8-40ca-8f03-a8063e50f1b9)
>Thread-15910529::DEBUG::2013-11-19 21:15:05,736::task::495::TaskManager.Task::(_debug) Task 75bdbfec-4c24-4d97-89df-8c16f7bc0fae: moving from state init -> state preparing
>Thread-15910529::DEBUG::2013-11-19 21:15:05,736::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 21b4b46d-58c8-40ca-8f03-a8063e50f1b9
>Thread-15910529::DEBUG::2013-11-19 21:15:05,737::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}
>Thread-15910529::DEBUG::2013-11-19 21:15:05,737::task::495::TaskManager.Task::(_debug) Task 75bdbfec-4c24-4d97-89df-8c16f7bc0fae: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910529::DEBUG::2013-11-19 21:15:05,738::task::495::TaskManager.Task::(_debug) Task 75bdbfec-4c24-4d97-89df-8c16f7bc0fae: moving from state preparing -> state finished
>Thread-15910529::DEBUG::2013-11-19 21:15:05,738::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910529::DEBUG::2013-11-19 21:15:05,739::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910529::DEBUG::2013-11-19 21:15:05,739::task::495::TaskManager.Task::(_debug) Task 75bdbfec-4c24-4d97-89df-8c16f7bc0fae: ref 0 aborting False
>Thread-15910529::INFO::2013-11-19 21:15:05,740::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910530::DEBUG::2013-11-19 21:15:06,746::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910530::INFO::2013-11-19 21:15:06,747::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=21b4b46d-58c8-40ca-8f03-a8063e50f1b9)
>Thread-15910530::DEBUG::2013-11-19 21:15:06,747::task::495::TaskManager.Task::(_debug) Task bb3a4a71-7b30-475c-a53a-ac8f17a22b8d: moving from state init -> state preparing
>Thread-15910530::DEBUG::2013-11-19 21:15:06,748::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 21b4b46d-58c8-40ca-8f03-a8063e50f1b9
>Thread-15910530::DEBUG::2013-11-19 21:15:06,748::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}
>Thread-15910530::DEBUG::2013-11-19 21:15:06,749::task::495::TaskManager.Task::(_debug) Task bb3a4a71-7b30-475c-a53a-ac8f17a22b8d: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910530::DEBUG::2013-11-19 21:15:06,749::task::495::TaskManager.Task::(_debug) Task bb3a4a71-7b30-475c-a53a-ac8f17a22b8d: moving from state preparing -> state finished
>Thread-15910530::DEBUG::2013-11-19 21:15:06,750::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910530::DEBUG::2013-11-19 21:15:06,750::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910530::DEBUG::2013-11-19 21:15:06,751::task::495::TaskManager.Task::(_debug) Task bb3a4a71-7b30-475c-a53a-ac8f17a22b8d: ref 0 aborting False
>Thread-15910530::INFO::2013-11-19 21:15:06,752::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910532::DEBUG::2013-11-19 21:15:07,757::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910532::INFO::2013-11-19 21:15:07,758::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=21b4b46d-58c8-40ca-8f03-a8063e50f1b9)
>Thread-15910532::DEBUG::2013-11-19 21:15:07,759::task::495::TaskManager.Task::(_debug) Task 0b05d2fc-e8ba-4364-93b4-91c26fbc96db: moving from state init -> state preparing
>Thread-15910532::DEBUG::2013-11-19 21:15:07,759::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 21b4b46d-58c8-40ca-8f03-a8063e50f1b9
>Thread-15910532::DEBUG::2013-11-19 21:15:07,760::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}
>Thread-15910532::DEBUG::2013-11-19 21:15:07,760::task::495::TaskManager.Task::(_debug) Task 0b05d2fc-e8ba-4364-93b4-91c26fbc96db: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910532::DEBUG::2013-11-19 21:15:07,761::task::495::TaskManager.Task::(_debug) Task 0b05d2fc-e8ba-4364-93b4-91c26fbc96db: moving from state preparing -> state finished
>Thread-15910532::DEBUG::2013-11-19 21:15:07,762::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910532::DEBUG::2013-11-19 21:15:07,762::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910532::DEBUG::2013-11-19 21:15:07,763::task::495::TaskManager.Task::(_debug) Task 0b05d2fc-e8ba-4364-93b4-91c26fbc96db: ref 0 aborting False
>Thread-15910532::INFO::2013-11-19 21:15:07,763::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:08,396::safelease::61::Storage.Misc.excCmd::(acquire) SUCCESS: <err> = ''; <rc> = 0
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:08,397::safelease::65::ClusterLock::(acquire) Clustered lock acquired successfully
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:08,397::spm::468::Storage.SPM::(start) spm lock acquired successfully
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:08,398::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:08,399::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:08,399::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:08,400::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:08,401::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:08,402::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15910533::DEBUG::2013-11-19 21:15:08,771::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910533::INFO::2013-11-19 21:15:08,773::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=21b4b46d-58c8-40ca-8f03-a8063e50f1b9)
>Thread-15910533::DEBUG::2013-11-19 21:15:08,792::task::495::TaskManager.Task::(_debug) Task 74016fe4-21ab-4c64-9e75-68921daeabfb: moving from state init -> state preparing
>Thread-15910533::DEBUG::2013-11-19 21:15:08,792::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 21b4b46d-58c8-40ca-8f03-a8063e50f1b9
>Thread-15910533::DEBUG::2013-11-19 21:15:08,793::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}
>Thread-15910533::DEBUG::2013-11-19 21:15:08,794::task::495::TaskManager.Task::(_debug) Task 74016fe4-21ab-4c64-9e75-68921daeabfb: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910533::DEBUG::2013-11-19 21:15:08,794::task::495::TaskManager.Task::(_debug) Task 74016fe4-21ab-4c64-9e75-68921daeabfb: moving from state preparing -> state finished
>Thread-15910533::DEBUG::2013-11-19 21:15:08,795::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910533::DEBUG::2013-11-19 21:15:08,795::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910533::DEBUG::2013-11-19 21:15:08,796::task::495::TaskManager.Task::(_debug) Task 74016fe4-21ab-4c64-9e75-68921daeabfb: ref 0 aborting False
>Thread-15910533::INFO::2013-11-19 21:15:08,797::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:08,990::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:08,993::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:08,994::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_LVER=277', u'POOL_SPM_ID=-1', u'_SHA_CKSUM=44c47fe1672a8deeb5c0f134ce9469f0ec72525b']
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:08,995::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:08,996::persistentDict::146::Storage.PersistentDict::(transaction) Flushing changes
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:08,997::persistentDict::265::Storage.PersistentDict::(flush) about to write lines (VGTagMetadataRW)=[u'CLASS=Data', u'DESCRIPTION=OS', u'IOOPTIMEOUTSEC=10', u'LEASERETRIES=3', u'LEASETIMESEC=60', u'LOCKPOLICY=', u'LOCKRENEWALINTERVALSEC=5', u'LOGBLKSIZE=512', u'MASTER_VERSION=1', u'PHYBLKSIZE=512', u'POOL_DESCRIPTION=Default', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_ID=2', u'POOL_SPM_LVER=278', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'ROLE=Master', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'TYPE=FCP', u'VERSION=2', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', '_SHA_CKSUM=913ce85a0b55e60c3f92c982d8a15ae168c40596']
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:08,997::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:08,998::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:08,999::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:08,999::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:09,000::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:09,001::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:09,565::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:09,573::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:09,574::blockSD::145::storage.Metadata.VGTagMetadataRW::(writelines) Updating metadata adding=MDT_POOL_SPM_ID=2, MDT_POOL_SPM_LVER=278, MDT__SHA_CKSUM=913ce85a0b55e60c3f92c982d8a15ae168c40596 removing=MDT__SHA_CKSUM=44c47fe1672a8deeb5c0f134ce9469f0ec72525b, MDT_POOL_SPM_ID=-1, MDT_POOL_SPM_LVER=277
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:09,576::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --deltag MDT_POOL_SPM_ID=-1 --deltag MDT__SHA_CKSUM=44c47fe1672a8deeb5c0f134ce9469f0ec72525b --deltag MDT_POOL_SPM_LVER=277 --addtag MDT_POOL_SPM_LVER=278 --addtag MDT_POOL_SPM_ID=2 --addtag MDT__SHA_CKSUM=913ce85a0b55e60c3f92c982d8a15ae168c40596 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15910535::DEBUG::2013-11-19 21:15:09,807::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910535::INFO::2013-11-19 21:15:09,810::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=21b4b46d-58c8-40ca-8f03-a8063e50f1b9)
>Thread-15910535::DEBUG::2013-11-19 21:15:09,811::task::495::TaskManager.Task::(_debug) Task 4ce1fcf1-d025-4027-b973-96b18f551db6: moving from state init -> state preparing
>Thread-15910535::DEBUG::2013-11-19 21:15:09,812::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 21b4b46d-58c8-40ca-8f03-a8063e50f1b9
>Thread-15910535::DEBUG::2013-11-19 21:15:09,812::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}
>Thread-15910535::DEBUG::2013-11-19 21:15:09,813::task::495::TaskManager.Task::(_debug) Task 4ce1fcf1-d025-4027-b973-96b18f551db6: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910535::DEBUG::2013-11-19 21:15:09,814::task::495::TaskManager.Task::(_debug) Task 4ce1fcf1-d025-4027-b973-96b18f551db6: moving from state preparing -> state finished
>Thread-15910535::DEBUG::2013-11-19 21:15:09,815::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910535::DEBUG::2013-11-19 21:15:09,816::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910535::DEBUG::2013-11-19 21:15:09,817::task::495::TaskManager.Task::(_debug) Task 4ce1fcf1-d025-4027-b973-96b18f551db6: ref 0 aborting False
>Thread-15910535::INFO::2013-11-19 21:15:09,817::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:10,261::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:10,262::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:10,263::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:10,264::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:10,265::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`646117f5-1e62-471a-97fc-246bf947cb9f`::Request was made in '/usr/share/vdsm/storage/spm.py' line '330' at '_upgradePool'
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:10,266::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'exclusive'
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:10,266::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'exclusive' (1 active user)
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:10,267::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`646117f5-1e62-471a-97fc-246bf947cb9f`::Granted request
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::INFO::2013-11-19 21:15:10,268::spm::336::Storage.SPM::(_upgradePool) Trying to upgrade master domain `6b892067-ac7e-4039-a7be-5b7387525b16`
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:10,269::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b6d7fe31-1d2f-4141-85fa-2f051914b317`::Request was made in '/usr/share/vdsm/storage/spm.py' line '337' at '_upgradePool'
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:10,269::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'exclusive'
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:10,270::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'exclusive' (1 active user)
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:10,270::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b6d7fe31-1d2f-4141-85fa-2f051914b317`::Granted request
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:10,271::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:10,272::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:10,273::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:10,273::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:10,274::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:10,275::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15910536::DEBUG::2013-11-19 21:15:10,825::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910536::INFO::2013-11-19 21:15:10,827::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=21b4b46d-58c8-40ca-8f03-a8063e50f1b9)
>Thread-15910536::DEBUG::2013-11-19 21:15:10,828::task::495::TaskManager.Task::(_debug) Task 721ed015-cfb7-4aca-b2bc-eb7c0079bc89: moving from state init -> state preparing
>Thread-15910536::DEBUG::2013-11-19 21:15:10,828::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 21b4b46d-58c8-40ca-8f03-a8063e50f1b9
>Thread-15910536::DEBUG::2013-11-19 21:15:10,829::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}
>Thread-15910536::DEBUG::2013-11-19 21:15:10,830::task::495::TaskManager.Task::(_debug) Task 721ed015-cfb7-4aca-b2bc-eb7c0079bc89: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910536::DEBUG::2013-11-19 21:15:10,830::task::495::TaskManager.Task::(_debug) Task 721ed015-cfb7-4aca-b2bc-eb7c0079bc89: moving from state preparing -> state finished
>Thread-15910536::DEBUG::2013-11-19 21:15:10,831::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910536::DEBUG::2013-11-19 21:15:10,831::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910536::DEBUG::2013-11-19 21:15:10,832::task::495::TaskManager.Task::(_debug) Task 721ed015-cfb7-4aca-b2bc-eb7c0079bc89: ref 0 aborting False
>Thread-15910536::INFO::2013-11-19 21:15:10,832::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:10,921::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:10,924::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:10,925::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_LVER=278', u'POOL_SPM_ID=2', u'_SHA_CKSUM=913ce85a0b55e60c3f92c982d8a15ae168c40596']
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:10,927::blockSD::625::Storage.StorageDomain::(upgrade) Trying to upgrade domain `6b892067-ac7e-4039-a7be-5b7387525b16` from version 2 to version 2
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:10,927::blockSD::631::Storage.StorageDomain::(upgrade) No need to upgrade domain `6b892067-ac7e-4039-a7be-5b7387525b16`, leaving unchanged
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:10,928::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:10,928::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:10,929::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:10,929::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:10,930::spm::340::Storage.SPM::(_upgradePool) Marking all domains for upgrade
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:10,931::spm::347::Storage.SPM::(_upgradePool) Registering with state change event
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:10,931::spm::349::Storage.SPM::(_upgradePool) Running initial domain upgrade threads
>Thread-15910537::DEBUG::2013-11-19 21:15:10,932::spm::365::Storage.SPM::(_upgradePoolDomain) Preparing to upgrade domain 45cdff8b-89c0-4926-a8c7-80354064bfa6
>Thread-15910537::DEBUG::2013-11-19 21:15:10,934::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6`ReqID=`6d31b91c-c6f3-4a78-8c7a-6001ea6b2467`::Request was made in '/usr/share/vdsm/storage/spm.py' line '376' at '_upgradePoolDomain'
>Thread-15910538::DEBUG::2013-11-19 21:15:10,935::spm::365::Storage.SPM::(_upgradePoolDomain) Preparing to upgrade domain 772472cb-2429-466d-96cd-00b3301ef78a
>Thread-15910539::DEBUG::2013-11-19 21:15:10,936::spm::365::Storage.SPM::(_upgradePoolDomain) Preparing to upgrade domain 5d7b5178-3790-472c-a13b-b481eb27a61e
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:10,936::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910537::DEBUG::2013-11-19 21:15:10,944::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6' for lock type 'exclusive'
>Thread-15910538::DEBUG::2013-11-19 21:15:10,948::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2c65d13e-9f93-4f6d-9178-2c03158c43c0`::Request was made in '/usr/share/vdsm/storage/spm.py' line '376' at '_upgradePoolDomain'
>Thread-15910539::ERROR::2013-11-19 21:15:10,952::spm::373::Storage.SPM::(_upgradePoolDomain) Error while preparing domain `5d7b5178-3790-472c-a13b-b481eb27a61e` upgrade
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/spm.py", line 369, in _upgradePoolDomain
>IndexError: list index out of range
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:10,953::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910538::DEBUG::2013-11-19 21:15:10,958::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'exclusive'
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:10,960::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:10,961::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910537::DEBUG::2013-11-19 21:15:10,961::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6' is free. Now locking as 'exclusive' (1 active user)
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:10,962::blockSD::791::Storage.Misc.excCmd::(mountMaster) '/usr/bin/sudo -n /sbin/e2fsck -p /dev/6b892067-ac7e-4039-a7be-5b7387525b16/master' (cwd None)
>Thread-15910537::DEBUG::2013-11-19 21:15:10,963::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6`ReqID=`6d31b91c-c6f3-4a78-8c7a-6001ea6b2467`::Granted request
>Thread-15910538::DEBUG::2013-11-19 21:15:10,964::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'exclusive' (1 active user)
>Thread-15910538::DEBUG::2013-11-19 21:15:11,169::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2c65d13e-9f93-4f6d-9178-2c03158c43c0`::Granted request
>Thread-15910537::DEBUG::2013-11-19 21:15:11,173::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6`ReqID=`4d1fc42b-09c4-4001-ad7f-613dcb89e13d`::Request was made in '/usr/share/vdsm/storage/spm.py' line '377' at '_upgradePoolDomain'
>Thread-15910537::DEBUG::2013-11-19 21:15:11,180::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6' for lock type 'exclusive'
>Thread-15910538::DEBUG::2013-11-19 21:15:11,187::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`441cfe1f-5d54-45e8-b117-aa7c65252dae`::Request was made in '/usr/share/vdsm/storage/spm.py' line '377' at '_upgradePoolDomain'
>Thread-15910537::DEBUG::2013-11-19 21:15:11,187::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6' is free. Now locking as 'exclusive' (1 active user)
>Thread-15910538::DEBUG::2013-11-19 21:15:11,188::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'exclusive'
>Thread-15910537::DEBUG::2013-11-19 21:15:11,189::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6`ReqID=`4d1fc42b-09c4-4001-ad7f-613dcb89e13d`::Granted request
>Thread-15910538::DEBUG::2013-11-19 21:15:11,191::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'exclusive' (1 active user)
>Thread-15910537::DEBUG::2013-11-19 21:15:11,192::spm::385::Storage.SPM::(_upgradePoolDomain) Domain `45cdff8b-89c0-4926-a8c7-80354064bfa6` is not a data domain it is an 2 domain, not upgrading
>Thread-15910538::DEBUG::2013-11-19 21:15:11,193::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`441cfe1f-5d54-45e8-b117-aa7c65252dae`::Granted request
>Thread-15910537::DEBUG::2013-11-19 21:15:11,194::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6'
>Thread-15910538::DEBUG::2013-11-19 21:15:11,196::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15910537::DEBUG::2013-11-19 21:15:11,197::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6' (0 active users)
>Thread-15910538::DEBUG::2013-11-19 21:15:11,198::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15910537::DEBUG::2013-11-19 21:15:11,199::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6' is free, finding out if anyone is waiting for it.
>Thread-15910538::DEBUG::2013-11-19 21:15:11,200::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15910537::DEBUG::2013-11-19 21:15:11,200::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6', Clearing records.
>Thread-15910538::DEBUG::2013-11-19 21:15:11,201::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15910538::DEBUG::2013-11-19 21:15:11,203::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-15910537::DEBUG::2013-11-19 21:15:11,204::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6'
>Thread-15910538::DEBUG::2013-11-19 21:15:11,205::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-15910537::DEBUG::2013-11-19 21:15:11,206::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6' (0 active users)
>Thread-15910537::DEBUG::2013-11-19 21:15:11,208::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6' is free, finding out if anyone is waiting for it.
>Thread-15910537::DEBUG::2013-11-19 21:15:11,521::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6', Clearing records.
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:11,529::blockSD::791::Storage.Misc.excCmd::(mountMaster) SUCCESS: <err> = ''; <rc> = 0
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:11,530::blockSD::816::Storage.Misc.excCmd::(mountMaster) '/usr/bin/sudo -n /sbin/tune2fs -j /dev/6b892067-ac7e-4039-a7be-5b7387525b16/master' (cwd None)
>Thread-15910541::DEBUG::2013-11-19 21:15:11,881::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910541::INFO::2013-11-19 21:15:11,883::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=21b4b46d-58c8-40ca-8f03-a8063e50f1b9)
>Thread-15910541::DEBUG::2013-11-19 21:15:11,885::task::495::TaskManager.Task::(_debug) Task 60934217-81cd-49cb-8bd5-b8fa63953eb4: moving from state init -> state preparing
>Thread-15910541::DEBUG::2013-11-19 21:15:11,885::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 21b4b46d-58c8-40ca-8f03-a8063e50f1b9
>Thread-15910541::DEBUG::2013-11-19 21:15:11,886::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}
>Thread-15910541::DEBUG::2013-11-19 21:15:11,887::task::495::TaskManager.Task::(_debug) Task 60934217-81cd-49cb-8bd5-b8fa63953eb4: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910541::DEBUG::2013-11-19 21:15:11,888::task::495::TaskManager.Task::(_debug) Task 60934217-81cd-49cb-8bd5-b8fa63953eb4: moving from state preparing -> state finished
>Thread-15910541::DEBUG::2013-11-19 21:15:11,889::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910541::DEBUG::2013-11-19 21:15:11,890::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910541::DEBUG::2013-11-19 21:15:11,891::task::495::TaskManager.Task::(_debug) Task 60934217-81cd-49cb-8bd5-b8fa63953eb4: ref 0 aborting False
>Thread-15910541::INFO::2013-11-19 21:15:11,892::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:12,075::blockSD::816::Storage.Misc.excCmd::(mountMaster) FAILED: <err> = 'The filesystem already has a journal.\n'; <rc> = 1
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:12,078::fileUtils::83::Storage.Misc.excCmd::(mount) '/usr/bin/sudo -n /bin/mount /dev/6b892067-ac7e-4039-a7be-5b7387525b16/master /rhev/data-center/mnt/blockSD/6b892067-ac7e-4039-a7be-5b7387525b16/master' (cwd None)
>Thread-15910538::DEBUG::2013-11-19 21:15:12,080::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15910538::DEBUG::2013-11-19 21:15:12,304::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-15910538::DEBUG::2013-11-19 21:15:12,306::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'SDUUID=772472cb-2429-466d-96cd-00b3301ef78a', u'VERSION=2', u'LOCKPOLICY=', u'VGUUID=XTb294-NYv6-dVpO-bB1n-4f9P-d0GL-dhd2DI', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'LEASETIMESEC=5', u'LOCKRENEWALINTERVALSEC=5', u'ROLE=Regular', u'PHYBLKSIZE=512', u'DESCRIPTION=Database', u'IOOPTIMEOUTSEC=1', u'CLASS=Data', u'TYPE=FCP', u'PV0=pv:3600c0ff0001520c7414f605001000000,uuid:NujM33-vNiD-sBHY-mrPo-EQ2f-1qsz-HXfn0q,pestart:0,pecount:8929,mapoffset:0', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'MASTER_VERSION=0', u'_SHA_CKSUM=ef8840d9fdef24516a56fd970bed1960d2b1ae92']
>Thread-15910538::DEBUG::2013-11-19 21:15:12,311::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15910538::DEBUG::2013-11-19 21:15:12,315::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15910538::DEBUG::2013-11-19 21:15:12,315::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15910538::DEBUG::2013-11-19 21:15:12,319::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15910538::DEBUG::2013-11-19 21:15:12,321::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15910538::DEBUG::2013-11-19 21:15:12,321::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15910538::DEBUG::2013-11-19 21:15:12,325::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15910538::DEBUG::2013-11-19 21:15:12,331::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:12,499::fileUtils::83::Storage.Misc.excCmd::(mount) SUCCESS: <err> = ''; <rc> = 0
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:12,501::blockSD::833::Storage.Misc.excCmd::(mountMaster) '/usr/bin/sudo -n /bin/chown vdsm:kvm /rhev/data-center/mnt/blockSD/6b892067-ac7e-4039-a7be-5b7387525b16/master' (cwd None)
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:12,893::blockSD::833::Storage.Misc.excCmd::(mountMaster) SUCCESS: <err> = ''; <rc> = 0
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::WARNING::2013-11-19 21:15:12,896::spm::490::Storage.SPM::(start) Backup domain validation failed
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/spm.py", line 488, in start
>  File "/usr/share/vdsm/storage/sp.py", line 1208, in checkBackupDomain
>  File "/usr/share/vdsm/storage/sd.py", line 666, in isBackup
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:12,898::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:12,899::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:12,900::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:12,900::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:12,901::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:12,901::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:12,902::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:12,902::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:12,903::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
>Thread-15910542::DEBUG::2013-11-19 21:15:12,905::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:12,905::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:12,907::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>Thread-15910542::INFO::2013-11-19 21:15:12,907::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=21b4b46d-58c8-40ca-8f03-a8063e50f1b9)
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:12,908::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction
>Thread-15910542::DEBUG::2013-11-19 21:15:12,909::task::495::TaskManager.Task::(_debug) Task d0f39847-12cc-49ab-b3cc-b58d33b4eb7d: moving from state init -> state preparing
>Thread-15910542::DEBUG::2013-11-19 21:15:12,910::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 21b4b46d-58c8-40ca-8f03-a8063e50f1b9
>Thread-15910542::DEBUG::2013-11-19 21:15:12,910::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}
>Thread-15910542::DEBUG::2013-11-19 21:15:12,911::task::495::TaskManager.Task::(_debug) Task d0f39847-12cc-49ab-b3cc-b58d33b4eb7d: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910542::DEBUG::2013-11-19 21:15:12,912::task::495::TaskManager.Task::(_debug) Task d0f39847-12cc-49ab-b3cc-b58d33b4eb7d: moving from state preparing -> state finished
>Thread-15910542::DEBUG::2013-11-19 21:15:12,912::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910542::DEBUG::2013-11-19 21:15:12,913::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910542::DEBUG::2013-11-19 21:15:12,913::task::495::TaskManager.Task::(_debug) Task d0f39847-12cc-49ab-b3cc-b58d33b4eb7d: ref 0 aborting False
>Thread-15910542::INFO::2013-11-19 21:15:12,914::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:12,919::threadPool::25::Misc.ThreadPool::(__init__) Enter - numThreads: 5.0, waitTimeout: 3, maxTasks: 500.0
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:12,924::storage_mailbox::486::Storage.MailBox.SpmMailMonitor::(__init__) SPM_MailMonitor - clearing outgoing mail, command is: ['dd', 'of=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/outbox', 'oflag=direct', 'iflag=fullblock', 'conv=notrunc', 'count=1']
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:12,925::storage_mailbox::488::Storage.Misc.excCmd::(__init__) 'dd of=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/outbox oflag=direct iflag=fullblock conv=notrunc count=1 bs=1024000' (cwd None)
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:13,371::storage_mailbox::488::Storage.Misc.excCmd::(__init__) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0406752 s, 25.2 MB/s\n'; <rc> = 0
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:13,373::storage_mailbox::493::Storage.MailBox.SpmMailMonitor::(__init__) SPM_MailMonitor created for pool c4e5889a-348d-11e2-83fe-9f72369eab30
>Dummy-15910548::DEBUG::2013-11-19 21:15:13,374::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:13,376::spm::523::Storage.SPM::(start) ended.
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:13,378::task::495::TaskManager.Task::(_debug) Task 21b4b46d-58c8-40ca-8f03-a8063e50f1b9: moving from state running -> state finished
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:13,624::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:13,647::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:13,649::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:13,649::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:13,650::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:13,650::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:13,651::task::915::TaskManager.Task::(_runJobs) Task.run: exit - success: result 
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:13,652::task::495::TaskManager.Task::(_debug) Task 21b4b46d-58c8-40ca-8f03-a8063e50f1b9: ref 0 aborting False
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:15:13,653::threadPool::47::Misc.ThreadPool::(setRunningTask) Number of running tasks: 0
>Thread-1024767::ERROR::2013-11-19 21:15:13,881::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15910548::DEBUG::2013-11-19 21:15:13,907::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0703364 s, 14.6 MB/s\n'; <rc> = 0
>Thread-15910550::DEBUG::2013-11-19 21:15:13,929::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910550::INFO::2013-11-19 21:15:13,930::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=21b4b46d-58c8-40ca-8f03-a8063e50f1b9)
>Thread-15910550::DEBUG::2013-11-19 21:15:13,931::task::495::TaskManager.Task::(_debug) Task 52511ad9-48e4-4a5b-b02b-e2b6551c98e5: moving from state init -> state preparing
>Thread-15910550::DEBUG::2013-11-19 21:15:13,932::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 21b4b46d-58c8-40ca-8f03-a8063e50f1b9
>Thread-15910550::DEBUG::2013-11-19 21:15:13,933::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': '1 jobs completed successfuly', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}
>Thread-15910550::DEBUG::2013-11-19 21:15:13,934::task::495::TaskManager.Task::(_debug) Task 52511ad9-48e4-4a5b-b02b-e2b6551c98e5: finished: {'taskStatus': {'code': 0, 'message': '1 jobs completed successfuly', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910550::DEBUG::2013-11-19 21:15:13,935::task::495::TaskManager.Task::(_debug) Task 52511ad9-48e4-4a5b-b02b-e2b6551c98e5: moving from state preparing -> state finished
>Thread-15910550::DEBUG::2013-11-19 21:15:13,935::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910550::DEBUG::2013-11-19 21:15:13,936::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910550::DEBUG::2013-11-19 21:15:13,937::task::495::TaskManager.Task::(_debug) Task 52511ad9-48e4-4a5b-b02b-e2b6551c98e5: ref 0 aborting False
>Thread-15910550::INFO::2013-11-19 21:15:13,938::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': '1 jobs completed successfuly', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '21b4b46d-58c8-40ca-8f03-a8063e50f1b9'}}
>Thread-15910551::DEBUG::2013-11-19 21:15:13,942::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910551::INFO::2013-11-19 21:15:13,943::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910551::DEBUG::2013-11-19 21:15:13,943::task::495::TaskManager.Task::(_debug) Task 58ff7d91-906a-455b-88e6-1d4e83222e18: moving from state init -> state preparing
>Thread-15910551::DEBUG::2013-11-19 21:15:13,944::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=278 spmId=2
>Thread-15910551::DEBUG::2013-11-19 21:15:13,945::task::495::TaskManager.Task::(_debug) Task 58ff7d91-906a-455b-88e6-1d4e83222e18: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910551::DEBUG::2013-11-19 21:15:13,945::task::495::TaskManager.Task::(_debug) Task 58ff7d91-906a-455b-88e6-1d4e83222e18: moving from state preparing -> state finished
>Thread-15910551::DEBUG::2013-11-19 21:15:13,946::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910551::DEBUG::2013-11-19 21:15:13,946::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910551::DEBUG::2013-11-19 21:15:13,947::task::495::TaskManager.Task::(_debug) Task 58ff7d91-906a-455b-88e6-1d4e83222e18: ref 0 aborting False
>Thread-15910551::INFO::2013-11-19 21:15:13,947::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910552::DEBUG::2013-11-19 21:15:13,953::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910552::INFO::2013-11-19 21:15:13,954::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: clearTask, args: ( taskID=21b4b46d-58c8-40ca-8f03-a8063e50f1b9)
>Thread-15910552::DEBUG::2013-11-19 21:15:13,954::task::495::TaskManager.Task::(_debug) Task da521451-7213-4dbd-8c00-e09e6cd3a39a: moving from state init -> state preparing
>Thread-15910552::DEBUG::2013-11-19 21:15:13,955::taskManager::146::TaskManager::(clearTask) Entry. taskID: 21b4b46d-58c8-40ca-8f03-a8063e50f1b9
>Thread-15910552::DEBUG::2013-11-19 21:15:13,955::taskManager::151::TaskManager::(clearTask) Return.
>Thread-15910552::DEBUG::2013-11-19 21:15:13,956::task::495::TaskManager.Task::(_debug) Task da521451-7213-4dbd-8c00-e09e6cd3a39a: finished: None
>Thread-15910552::DEBUG::2013-11-19 21:15:13,956::task::495::TaskManager.Task::(_debug) Task da521451-7213-4dbd-8c00-e09e6cd3a39a: moving from state preparing -> state finished
>Thread-15910552::DEBUG::2013-11-19 21:15:13,957::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910552::DEBUG::2013-11-19 21:15:13,957::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910552::DEBUG::2013-11-19 21:15:13,958::task::495::TaskManager.Task::(_debug) Task da521451-7213-4dbd-8c00-e09e6cd3a39a: ref 0 aborting False
>Thread-15910552::INFO::2013-11-19 21:15:13,958::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: clearTask, Return response: {'status': {'message': 'OK', 'code': 0}}
>Thread-15910553::DEBUG::2013-11-19 21:15:13,995::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910553::INFO::2013-11-19 21:15:13,996::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910553::DEBUG::2013-11-19 21:15:13,997::task::495::TaskManager.Task::(_debug) Task 803f1de4-c8a7-420f-8294-06c3ec01c2a0: moving from state init -> state preparing
>Thread-15910553::DEBUG::2013-11-19 21:15:13,998::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`079e5672-ac83-4781-96a1-6a6b44b1af78`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910553::DEBUG::2013-11-19 21:15:13,998::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910553::DEBUG::2013-11-19 21:15:13,999::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910553::DEBUG::2013-11-19 21:15:14,000::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`079e5672-ac83-4781-96a1-6a6b44b1af78`::Granted request
>Thread-15910553::DEBUG::2013-11-19 21:15:14,001::task::495::TaskManager.Task::(_debug) Task 803f1de4-c8a7-420f-8294-06c3ec01c2a0: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910553::DEBUG::2013-11-19 21:15:14,001::task::495::TaskManager.Task::(_debug) Task 803f1de4-c8a7-420f-8294-06c3ec01c2a0: ref 1 aborting False
>Thread-15910553::WARNING::2013-11-19 21:15:14,003::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910553::DEBUG::2013-11-19 21:15:14,004::task::495::TaskManager.Task::(_debug) Task 803f1de4-c8a7-420f-8294-06c3ec01c2a0: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910553::DEBUG::2013-11-19 21:15:14,005::task::495::TaskManager.Task::(_debug) Task 803f1de4-c8a7-420f-8294-06c3ec01c2a0: moving from state preparing -> state finished
>Thread-15910553::DEBUG::2013-11-19 21:15:14,006::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910553::DEBUG::2013-11-19 21:15:14,006::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910553::DEBUG::2013-11-19 21:15:14,007::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910553::DEBUG::2013-11-19 21:15:14,008::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910553::DEBUG::2013-11-19 21:15:14,008::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910553::DEBUG::2013-11-19 21:15:14,009::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910553::DEBUG::2013-11-19 21:15:14,009::task::495::TaskManager.Task::(_debug) Task 803f1de4-c8a7-420f-8294-06c3ec01c2a0: ref 0 aborting False
>Thread-15910553::INFO::2013-11-19 21:15:14,010::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910554::DEBUG::2013-11-19 21:15:14,126::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910554::INFO::2013-11-19 21:15:14,127::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksInfo, args: ()
>Thread-15910554::DEBUG::2013-11-19 21:15:14,128::task::495::TaskManager.Task::(_debug) Task c33acd6c-e73a-4738-bb2d-67f8fe29448d: moving from state init -> state preparing
>Thread-15910554::DEBUG::2013-11-19 21:15:14,128::taskManager::168::TaskManager::(getAllTasksInfo) Entry.
>Thread-15910554::DEBUG::2013-11-19 21:15:14,129::taskManager::173::TaskManager::(getAllTasksInfo) Return. Response: {}
>Thread-15910554::DEBUG::2013-11-19 21:15:14,129::task::495::TaskManager.Task::(_debug) Task c33acd6c-e73a-4738-bb2d-67f8fe29448d: finished: {'allTasksInfo': {}}
>Thread-15910554::DEBUG::2013-11-19 21:15:14,130::task::495::TaskManager.Task::(_debug) Task c33acd6c-e73a-4738-bb2d-67f8fe29448d: moving from state preparing -> state finished
>Thread-15910554::DEBUG::2013-11-19 21:15:14,131::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910554::DEBUG::2013-11-19 21:15:14,131::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910554::DEBUG::2013-11-19 21:15:14,132::task::495::TaskManager.Task::(_debug) Task c33acd6c-e73a-4738-bb2d-67f8fe29448d: ref 0 aborting False
>Thread-15910554::INFO::2013-11-19 21:15:14,132::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'allTasksInfo': {}}
>Thread-15910555::DEBUG::2013-11-19 21:15:14,139::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910555::INFO::2013-11-19 21:15:14,140::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksInfo, args: ()
>Thread-15910555::DEBUG::2013-11-19 21:15:14,140::task::495::TaskManager.Task::(_debug) Task fe73aa27-87d9-4467-b066-695a4fac0638: moving from state init -> state preparing
>Thread-15910555::DEBUG::2013-11-19 21:15:14,141::taskManager::168::TaskManager::(getAllTasksInfo) Entry.
>Thread-15910555::DEBUG::2013-11-19 21:15:14,141::taskManager::173::TaskManager::(getAllTasksInfo) Return. Response: {}
>Thread-15910555::DEBUG::2013-11-19 21:15:14,142::task::495::TaskManager.Task::(_debug) Task fe73aa27-87d9-4467-b066-695a4fac0638: finished: {'allTasksInfo': {}}
>Thread-15910555::DEBUG::2013-11-19 21:15:14,143::task::495::TaskManager.Task::(_debug) Task fe73aa27-87d9-4467-b066-695a4fac0638: moving from state preparing -> state finished
>Thread-15910555::DEBUG::2013-11-19 21:15:14,143::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910555::DEBUG::2013-11-19 21:15:14,144::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910555::DEBUG::2013-11-19 21:15:14,144::task::495::TaskManager.Task::(_debug) Task fe73aa27-87d9-4467-b066-695a4fac0638: ref 0 aborting False
>Thread-15910555::INFO::2013-11-19 21:15:14,145::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'allTasksInfo': {}}
>Thread-15910556::INFO::2013-11-19 21:15:15,652::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910556::DEBUG::2013-11-19 21:15:15,653::task::495::TaskManager.Task::(_debug) Task 18cc9775-b91c-442c-a192-18e66179b787: moving from state init -> state preparing
>Thread-15910556::WARNING::2013-11-19 21:15:15,653::hsm::1895::Storage.HSM::(public_repoStats) repoStats detected invalid master:6b892067-ac7e-4039-a7be-5b7387525b16 {'finish': 1384895706.8834889, 'mdasize': '134479872', 'mdathreshold': True, 'mdavalid': True, 'diskfree': '1142595518464', 'disktotal': '3595827150848', 'result': {'delay': '0.000278949737549', 'lastCheck': 1384895706.8834889, 'valid': True, 'code': 0}, 'masterValidate': {'mount': False, 'valid': True}, 'mdafree': '67205632'}
>Thread-15910556::DEBUG::2013-11-19 21:15:15,654::task::495::TaskManager.Task::(_debug) Task 18cc9775-b91c-442c-a192-18e66179b787: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000278949737549', 'lastCheck': 1384895706.8834889, 'valid': True, 'code': 355}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00109696388245', 'lastCheck': 1384895708.8166831, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895713.8826809, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000510931015015', 'lastCheck': 1384895712.876039, 'valid': True, 'code': 0}}
>Thread-15910556::DEBUG::2013-11-19 21:15:15,655::task::495::TaskManager.Task::(_debug) Task 18cc9775-b91c-442c-a192-18e66179b787: moving from state preparing -> state finished
>Thread-15910556::DEBUG::2013-11-19 21:15:15,655::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910556::DEBUG::2013-11-19 21:15:15,656::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910556::DEBUG::2013-11-19 21:15:15,657::task::495::TaskManager.Task::(_debug) Task 18cc9775-b91c-442c-a192-18e66179b787: ref 0 aborting False
>Thread-15910556::INFO::2013-11-19 21:15:15,657::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000278949737549', 'lastCheck': 1384895706.8834889, 'valid': True, 'code': 355}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00109696388245', 'lastCheck': 1384895708.8166831, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895713.8826809, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000510931015015', 'lastCheck': 1384895712.876039, 'valid': True, 'code': 0}}
>Dummy-15910548::DEBUG::2013-11-19 21:15:15,930::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:15:16,461::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0709684 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:15:18,477::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:15:19,007::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0743424 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:15:21,022::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:15:21,569::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0713424 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:15:23,585::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:15:23,899::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15910548::DEBUG::2013-11-19 21:15:24,118::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0711656 s, 14.4 MB/s\n'; <rc> = 0
>Thread-15910562::DEBUG::2013-11-19 21:15:24,133::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910562::INFO::2013-11-19 21:15:24,134::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910562::DEBUG::2013-11-19 21:15:24,134::task::495::TaskManager.Task::(_debug) Task 9ab733fe-d1c8-4bbf-b4d3-e606453a84fb: moving from state init -> state preparing
>Thread-15910562::DEBUG::2013-11-19 21:15:24,135::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=278 spmId=2
>Thread-15910562::DEBUG::2013-11-19 21:15:24,135::task::495::TaskManager.Task::(_debug) Task 9ab733fe-d1c8-4bbf-b4d3-e606453a84fb: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910562::DEBUG::2013-11-19 21:15:24,136::task::495::TaskManager.Task::(_debug) Task 9ab733fe-d1c8-4bbf-b4d3-e606453a84fb: moving from state preparing -> state finished
>Thread-15910562::DEBUG::2013-11-19 21:15:24,136::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910562::DEBUG::2013-11-19 21:15:24,136::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910562::DEBUG::2013-11-19 21:15:24,137::task::495::TaskManager.Task::(_debug) Task 9ab733fe-d1c8-4bbf-b4d3-e606453a84fb: ref 0 aborting False
>Thread-15910562::INFO::2013-11-19 21:15:24,137::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910563::DEBUG::2013-11-19 21:15:24,141::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910563::INFO::2013-11-19 21:15:24,142::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910563::DEBUG::2013-11-19 21:15:24,142::task::495::TaskManager.Task::(_debug) Task dbcfcf97-940b-42e1-8f8e-7823c170677f: moving from state init -> state preparing
>Thread-15910563::DEBUG::2013-11-19 21:15:24,143::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`dacf4bd5-93e9-4054-a2ea-1dd8cb3759cc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910563::DEBUG::2013-11-19 21:15:24,143::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910563::DEBUG::2013-11-19 21:15:24,143::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910563::DEBUG::2013-11-19 21:15:24,144::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`dacf4bd5-93e9-4054-a2ea-1dd8cb3759cc`::Granted request
>Thread-15910563::DEBUG::2013-11-19 21:15:24,144::task::495::TaskManager.Task::(_debug) Task dbcfcf97-940b-42e1-8f8e-7823c170677f: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910563::DEBUG::2013-11-19 21:15:24,145::task::495::TaskManager.Task::(_debug) Task dbcfcf97-940b-42e1-8f8e-7823c170677f: ref 1 aborting False
>Thread-15910563::WARNING::2013-11-19 21:15:24,146::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910563::DEBUG::2013-11-19 21:15:24,147::task::495::TaskManager.Task::(_debug) Task dbcfcf97-940b-42e1-8f8e-7823c170677f: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910563::DEBUG::2013-11-19 21:15:24,147::task::495::TaskManager.Task::(_debug) Task dbcfcf97-940b-42e1-8f8e-7823c170677f: moving from state preparing -> state finished
>Thread-15910563::DEBUG::2013-11-19 21:15:24,147::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910563::DEBUG::2013-11-19 21:15:24,148::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910563::DEBUG::2013-11-19 21:15:24,148::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910563::DEBUG::2013-11-19 21:15:24,149::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910563::DEBUG::2013-11-19 21:15:24,149::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910563::DEBUG::2013-11-19 21:15:24,149::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910563::DEBUG::2013-11-19 21:15:24,150::task::495::TaskManager.Task::(_debug) Task dbcfcf97-940b-42e1-8f8e-7823c170677f: ref 0 aborting False
>Thread-15910563::INFO::2013-11-19 21:15:24,150::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910564::INFO::2013-11-19 21:15:25,814::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910564::DEBUG::2013-11-19 21:15:25,814::task::495::TaskManager.Task::(_debug) Task 27b8f6b4-2800-4ea4-b1aa-4bd0c0b25827: moving from state init -> state preparing
>Thread-15910564::DEBUG::2013-11-19 21:15:25,815::task::495::TaskManager.Task::(_debug) Task 27b8f6b4-2800-4ea4-b1aa-4bd0c0b25827: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000325202941895', 'lastCheck': 1384895716.897404, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00138807296753', 'lastCheck': 1384895718.888042, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895723.900955, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000407934188843', 'lastCheck': 1384895722.887794, 'valid': True, 'code': 0}}
>Thread-15910564::DEBUG::2013-11-19 21:15:25,816::task::495::TaskManager.Task::(_debug) Task 27b8f6b4-2800-4ea4-b1aa-4bd0c0b25827: moving from state preparing -> state finished
>Thread-15910564::DEBUG::2013-11-19 21:15:25,816::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910564::DEBUG::2013-11-19 21:15:25,817::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910564::DEBUG::2013-11-19 21:15:25,817::task::495::TaskManager.Task::(_debug) Task 27b8f6b4-2800-4ea4-b1aa-4bd0c0b25827: ref 0 aborting False
>Thread-15910564::INFO::2013-11-19 21:15:25,818::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000325202941895', 'lastCheck': 1384895716.897404, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00138807296753', 'lastCheck': 1384895718.888042, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895723.900955, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000407934188843', 'lastCheck': 1384895722.887794, 'valid': True, 'code': 0}}
>Dummy-15910548::DEBUG::2013-11-19 21:15:26,134::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:15:26,642::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0707014 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:15:28,658::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:15:29,194::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0711509 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:15:31,209::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:15:31,770::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.107993 s, 9.5 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:15:33,785::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:15:34,035::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910570::DEBUG::2013-11-19 21:15:34,275::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910570::INFO::2013-11-19 21:15:34,277::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910570::DEBUG::2013-11-19 21:15:34,293::task::495::TaskManager.Task::(_debug) Task 45e3aadf-4441-4cc2-bca7-306d7d7c53cd: moving from state init -> state preparing
>Thread-15910570::DEBUG::2013-11-19 21:15:34,295::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=278 spmId=2
>Thread-15910570::DEBUG::2013-11-19 21:15:34,296::task::495::TaskManager.Task::(_debug) Task 45e3aadf-4441-4cc2-bca7-306d7d7c53cd: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910570::DEBUG::2013-11-19 21:15:34,298::task::495::TaskManager.Task::(_debug) Task 45e3aadf-4441-4cc2-bca7-306d7d7c53cd: moving from state preparing -> state finished
>Thread-15910570::DEBUG::2013-11-19 21:15:34,299::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910570::DEBUG::2013-11-19 21:15:34,300::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910570::DEBUG::2013-11-19 21:15:34,301::task::495::TaskManager.Task::(_debug) Task 45e3aadf-4441-4cc2-bca7-306d7d7c53cd: ref 0 aborting False
>Thread-15910570::INFO::2013-11-19 21:15:34,303::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910571::DEBUG::2013-11-19 21:15:34,309::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910571::INFO::2013-11-19 21:15:34,311::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910571::DEBUG::2013-11-19 21:15:34,312::task::495::TaskManager.Task::(_debug) Task 2d0c5157-346c-4281-a6bf-a66baa0a96ee: moving from state init -> state preparing
>Thread-15910571::DEBUG::2013-11-19 21:15:34,314::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f9281132-3f21-474e-a3af-2d70e9119254`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910571::DEBUG::2013-11-19 21:15:34,316::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910571::DEBUG::2013-11-19 21:15:34,319::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910571::DEBUG::2013-11-19 21:15:34,320::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f9281132-3f21-474e-a3af-2d70e9119254`::Granted request
>Thread-15910571::DEBUG::2013-11-19 21:15:34,322::task::495::TaskManager.Task::(_debug) Task 2d0c5157-346c-4281-a6bf-a66baa0a96ee: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910571::DEBUG::2013-11-19 21:15:34,324::task::495::TaskManager.Task::(_debug) Task 2d0c5157-346c-4281-a6bf-a66baa0a96ee: ref 1 aborting False
>Thread-15910571::WARNING::2013-11-19 21:15:34,328::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910571::DEBUG::2013-11-19 21:15:34,330::task::495::TaskManager.Task::(_debug) Task 2d0c5157-346c-4281-a6bf-a66baa0a96ee: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910571::DEBUG::2013-11-19 21:15:34,332::task::495::TaskManager.Task::(_debug) Task 2d0c5157-346c-4281-a6bf-a66baa0a96ee: moving from state preparing -> state finished
>Thread-15910571::DEBUG::2013-11-19 21:15:34,333::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910571::DEBUG::2013-11-19 21:15:34,335::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910571::DEBUG::2013-11-19 21:15:34,337::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910571::DEBUG::2013-11-19 21:15:34,338::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910571::DEBUG::2013-11-19 21:15:34,339::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910571::DEBUG::2013-11-19 21:15:34,340::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910571::DEBUG::2013-11-19 21:15:34,342::task::495::TaskManager.Task::(_debug) Task 2d0c5157-346c-4281-a6bf-a66baa0a96ee: ref 0 aborting False
>Thread-15910571::INFO::2013-11-19 21:15:34,344::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15910548::DEBUG::2013-11-19 21:15:34,401::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0781844 s, 13.1 MB/s\n'; <rc> = 0
>Thread-15910572::INFO::2013-11-19 21:15:36,039::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910572::DEBUG::2013-11-19 21:15:36,040::task::495::TaskManager.Task::(_debug) Task b1299791-f110-4f0d-ae2e-fa2835db9abf: moving from state init -> state preparing
>Thread-15910572::DEBUG::2013-11-19 21:15:36,041::task::495::TaskManager.Task::(_debug) Task b1299791-f110-4f0d-ae2e-fa2835db9abf: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000499963760376', 'lastCheck': 1384895726.9124539, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00159597396851', 'lastCheck': 1384895728.9060919, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895734.0372019, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000357151031494', 'lastCheck': 1384895732.900136, 'valid': True, 'code': 0}}
>Thread-15910572::DEBUG::2013-11-19 21:15:36,042::task::495::TaskManager.Task::(_debug) Task b1299791-f110-4f0d-ae2e-fa2835db9abf: moving from state preparing -> state finished
>Thread-15910572::DEBUG::2013-11-19 21:15:36,042::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910572::DEBUG::2013-11-19 21:15:36,043::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910572::DEBUG::2013-11-19 21:15:36,043::task::495::TaskManager.Task::(_debug) Task b1299791-f110-4f0d-ae2e-fa2835db9abf: ref 0 aborting False
>Thread-15910572::INFO::2013-11-19 21:15:36,044::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000499963760376', 'lastCheck': 1384895726.9124539, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00159597396851', 'lastCheck': 1384895728.9060919, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895734.0372019, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000357151031494', 'lastCheck': 1384895732.900136, 'valid': True, 'code': 0}}
>Dummy-15910548::DEBUG::2013-11-19 21:15:36,425::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:15:36,962::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0712188 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:15:38,977::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:15:39,496::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0715435 s, 14.3 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:15:41,510::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:15:42,003::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0679397 s, 15.1 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:15:44,018::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:15:44,247::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910578::DEBUG::2013-11-19 21:15:44,465::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910578::INFO::2013-11-19 21:15:44,467::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910578::DEBUG::2013-11-19 21:15:44,468::task::495::TaskManager.Task::(_debug) Task 4f73bb23-75fa-48e7-ac58-ca86145246fe: moving from state init -> state preparing
>Thread-15910578::DEBUG::2013-11-19 21:15:44,469::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=278 spmId=2
>Thread-15910578::DEBUG::2013-11-19 21:15:44,470::task::495::TaskManager.Task::(_debug) Task 4f73bb23-75fa-48e7-ac58-ca86145246fe: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910578::DEBUG::2013-11-19 21:15:44,471::task::495::TaskManager.Task::(_debug) Task 4f73bb23-75fa-48e7-ac58-ca86145246fe: moving from state preparing -> state finished
>Thread-15910578::DEBUG::2013-11-19 21:15:44,472::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910578::DEBUG::2013-11-19 21:15:44,472::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910578::DEBUG::2013-11-19 21:15:44,473::task::495::TaskManager.Task::(_debug) Task 4f73bb23-75fa-48e7-ac58-ca86145246fe: ref 0 aborting False
>Thread-15910578::INFO::2013-11-19 21:15:44,474::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910579::DEBUG::2013-11-19 21:15:44,479::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910579::INFO::2013-11-19 21:15:44,481::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910579::DEBUG::2013-11-19 21:15:44,481::task::495::TaskManager.Task::(_debug) Task a98df52e-340d-424a-ae04-f8e78d0d66d0: moving from state init -> state preparing
>Thread-15910579::DEBUG::2013-11-19 21:15:44,483::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c5ebb70d-dae9-4da6-9ee9-bf4751a53c0c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910579::DEBUG::2013-11-19 21:15:44,484::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910579::DEBUG::2013-11-19 21:15:44,485::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910579::DEBUG::2013-11-19 21:15:44,486::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c5ebb70d-dae9-4da6-9ee9-bf4751a53c0c`::Granted request
>Thread-15910579::DEBUG::2013-11-19 21:15:44,488::task::495::TaskManager.Task::(_debug) Task a98df52e-340d-424a-ae04-f8e78d0d66d0: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910579::DEBUG::2013-11-19 21:15:44,489::task::495::TaskManager.Task::(_debug) Task a98df52e-340d-424a-ae04-f8e78d0d66d0: ref 1 aborting False
>Thread-15910579::WARNING::2013-11-19 21:15:44,492::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910579::DEBUG::2013-11-19 21:15:44,493::task::495::TaskManager.Task::(_debug) Task a98df52e-340d-424a-ae04-f8e78d0d66d0: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910579::DEBUG::2013-11-19 21:15:44,494::task::495::TaskManager.Task::(_debug) Task a98df52e-340d-424a-ae04-f8e78d0d66d0: moving from state preparing -> state finished
>Thread-15910579::DEBUG::2013-11-19 21:15:44,495::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910579::DEBUG::2013-11-19 21:15:44,496::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910579::DEBUG::2013-11-19 21:15:44,497::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910579::DEBUG::2013-11-19 21:15:44,498::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910579::DEBUG::2013-11-19 21:15:44,499::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910579::DEBUG::2013-11-19 21:15:44,500::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910579::DEBUG::2013-11-19 21:15:44,501::task::495::TaskManager.Task::(_debug) Task a98df52e-340d-424a-ae04-f8e78d0d66d0: ref 0 aborting False
>Thread-15910579::INFO::2013-11-19 21:15:44,502::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15910548::DEBUG::2013-11-19 21:15:44,568::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.100855 s, 10.2 MB/s\n'; <rc> = 0
>Thread-15910580::INFO::2013-11-19 21:15:46,258::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910580::DEBUG::2013-11-19 21:15:46,259::task::495::TaskManager.Task::(_debug) Task 52720f41-0b65-4395-8a24-4dd1986b8d24: moving from state init -> state preparing
>Thread-15910580::DEBUG::2013-11-19 21:15:46,260::task::495::TaskManager.Task::(_debug) Task 52720f41-0b65-4395-8a24-4dd1986b8d24: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000674962997437', 'lastCheck': 1384895737.020926, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00257802009583', 'lastCheck': 1384895739.192733, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895744.25155, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000842809677124', 'lastCheck': 1384895743.1882861, 'valid': True, 'code': 0}}
>Thread-15910580::DEBUG::2013-11-19 21:15:46,261::task::495::TaskManager.Task::(_debug) Task 52720f41-0b65-4395-8a24-4dd1986b8d24: moving from state preparing -> state finished
>Thread-15910580::DEBUG::2013-11-19 21:15:46,261::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910580::DEBUG::2013-11-19 21:15:46,262::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910580::DEBUG::2013-11-19 21:15:46,262::task::495::TaskManager.Task::(_debug) Task 52720f41-0b65-4395-8a24-4dd1986b8d24: ref 0 aborting False
>Thread-15910580::INFO::2013-11-19 21:15:46,263::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000674962997437', 'lastCheck': 1384895737.020926, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00257802009583', 'lastCheck': 1384895739.192733, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895744.25155, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000842809677124', 'lastCheck': 1384895743.1882861, 'valid': True, 'code': 0}}
>Dummy-15910548::DEBUG::2013-11-19 21:15:46,583::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:15:47,120::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0717487 s, 14.3 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:15:49,134::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:15:49,648::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0699884 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:15:51,663::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:15:52,196::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0706343 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:15:54,212::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:15:54,427::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910586::DEBUG::2013-11-19 21:15:54,615::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910586::INFO::2013-11-19 21:15:54,616::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910586::DEBUG::2013-11-19 21:15:54,617::task::495::TaskManager.Task::(_debug) Task 75876aa4-6f2b-4e46-9af3-60a33ab18928: moving from state init -> state preparing
>Thread-15910586::DEBUG::2013-11-19 21:15:54,619::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=278 spmId=2
>Thread-15910586::DEBUG::2013-11-19 21:15:54,620::task::495::TaskManager.Task::(_debug) Task 75876aa4-6f2b-4e46-9af3-60a33ab18928: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910586::DEBUG::2013-11-19 21:15:54,620::task::495::TaskManager.Task::(_debug) Task 75876aa4-6f2b-4e46-9af3-60a33ab18928: moving from state preparing -> state finished
>Thread-15910586::DEBUG::2013-11-19 21:15:54,621::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910586::DEBUG::2013-11-19 21:15:54,622::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910586::DEBUG::2013-11-19 21:15:54,623::task::495::TaskManager.Task::(_debug) Task 75876aa4-6f2b-4e46-9af3-60a33ab18928: ref 0 aborting False
>Thread-15910586::INFO::2013-11-19 21:15:54,624::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910587::DEBUG::2013-11-19 21:15:54,630::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910587::INFO::2013-11-19 21:15:54,631::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910587::DEBUG::2013-11-19 21:15:54,632::task::495::TaskManager.Task::(_debug) Task 6141c5c2-28fd-47f6-9ae8-3bedee3185bd: moving from state init -> state preparing
>Thread-15910587::DEBUG::2013-11-19 21:15:54,634::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b94eed69-1342-4c35-bdb1-7c3b216f94a0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910587::DEBUG::2013-11-19 21:15:54,635::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910587::DEBUG::2013-11-19 21:15:54,636::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910587::DEBUG::2013-11-19 21:15:54,637::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b94eed69-1342-4c35-bdb1-7c3b216f94a0`::Granted request
>Thread-15910587::DEBUG::2013-11-19 21:15:54,639::task::495::TaskManager.Task::(_debug) Task 6141c5c2-28fd-47f6-9ae8-3bedee3185bd: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910587::DEBUG::2013-11-19 21:15:54,640::task::495::TaskManager.Task::(_debug) Task 6141c5c2-28fd-47f6-9ae8-3bedee3185bd: ref 1 aborting False
>Thread-15910587::WARNING::2013-11-19 21:15:54,643::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910587::DEBUG::2013-11-19 21:15:54,644::task::495::TaskManager.Task::(_debug) Task 6141c5c2-28fd-47f6-9ae8-3bedee3185bd: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910587::DEBUG::2013-11-19 21:15:54,645::task::495::TaskManager.Task::(_debug) Task 6141c5c2-28fd-47f6-9ae8-3bedee3185bd: moving from state preparing -> state finished
>Thread-15910587::DEBUG::2013-11-19 21:15:54,646::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910587::DEBUG::2013-11-19 21:15:54,647::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910587::DEBUG::2013-11-19 21:15:54,648::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910587::DEBUG::2013-11-19 21:15:54,649::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910587::DEBUG::2013-11-19 21:15:54,650::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910587::DEBUG::2013-11-19 21:15:54,651::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910587::DEBUG::2013-11-19 21:15:54,652::task::495::TaskManager.Task::(_debug) Task 6141c5c2-28fd-47f6-9ae8-3bedee3185bd: ref 0 aborting False
>Thread-15910587::INFO::2013-11-19 21:15:54,653::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15910548::DEBUG::2013-11-19 21:15:54,765::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.100751 s, 10.2 MB/s\n'; <rc> = 0
>Thread-15910588::INFO::2013-11-19 21:15:56,440::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910588::DEBUG::2013-11-19 21:15:56,441::task::495::TaskManager.Task::(_debug) Task d3d0bebf-bff1-418c-b645-1aee647bcec7: moving from state init -> state preparing
>Thread-15910588::DEBUG::2013-11-19 21:15:56,441::task::495::TaskManager.Task::(_debug) Task d3d0bebf-bff1-418c-b645-1aee647bcec7: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000349998474121', 'lastCheck': 1384895747.2468829, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00129795074463', 'lastCheck': 1384895749.3710151, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895754.431953, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000501871109009', 'lastCheck': 1384895753.355706, 'valid': True, 'code': 0}}
>Thread-15910588::DEBUG::2013-11-19 21:15:56,442::task::495::TaskManager.Task::(_debug) Task d3d0bebf-bff1-418c-b645-1aee647bcec7: moving from state preparing -> state finished
>Thread-15910588::DEBUG::2013-11-19 21:15:56,443::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910588::DEBUG::2013-11-19 21:15:56,443::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910588::DEBUG::2013-11-19 21:15:56,444::task::495::TaskManager.Task::(_debug) Task d3d0bebf-bff1-418c-b645-1aee647bcec7: ref 0 aborting False
>Thread-15910588::INFO::2013-11-19 21:15:56,444::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000349998474121', 'lastCheck': 1384895747.2468829, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00129795074463', 'lastCheck': 1384895749.3710151, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895754.431953, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000501871109009', 'lastCheck': 1384895753.355706, 'valid': True, 'code': 0}}
>Dummy-15910548::DEBUG::2013-11-19 21:15:56,780::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:15:57,314::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0708345 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:15:59,329::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:15:59,854::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0711629 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:16:01,868::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:16:02,390::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0705285 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:16:04,405::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:16:04,659::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910594::DEBUG::2013-11-19 21:16:04,762::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910594::INFO::2013-11-19 21:16:04,764::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910594::DEBUG::2013-11-19 21:16:04,764::task::495::TaskManager.Task::(_debug) Task 54bd5431-26a4-4ea4-9318-e318f0e839ac: moving from state init -> state preparing
>Thread-15910594::DEBUG::2013-11-19 21:16:04,765::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=278 spmId=2
>Thread-15910594::DEBUG::2013-11-19 21:16:04,766::task::495::TaskManager.Task::(_debug) Task 54bd5431-26a4-4ea4-9318-e318f0e839ac: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910594::DEBUG::2013-11-19 21:16:04,766::task::495::TaskManager.Task::(_debug) Task 54bd5431-26a4-4ea4-9318-e318f0e839ac: moving from state preparing -> state finished
>Thread-15910594::DEBUG::2013-11-19 21:16:04,767::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910594::DEBUG::2013-11-19 21:16:04,768::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910594::DEBUG::2013-11-19 21:16:04,768::task::495::TaskManager.Task::(_debug) Task 54bd5431-26a4-4ea4-9318-e318f0e839ac: ref 0 aborting False
>Thread-15910594::INFO::2013-11-19 21:16:04,769::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910595::DEBUG::2013-11-19 21:16:04,774::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910595::INFO::2013-11-19 21:16:04,775::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910595::DEBUG::2013-11-19 21:16:04,775::task::495::TaskManager.Task::(_debug) Task a184a53e-c62f-4ce9-8ef5-772eabf31f99: moving from state init -> state preparing
>Thread-15910595::DEBUG::2013-11-19 21:16:04,776::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`71595538-d15d-4687-bf96-82895f02bee2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910595::DEBUG::2013-11-19 21:16:04,777::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910595::DEBUG::2013-11-19 21:16:04,778::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910595::DEBUG::2013-11-19 21:16:04,780::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`71595538-d15d-4687-bf96-82895f02bee2`::Granted request
>Thread-15910595::DEBUG::2013-11-19 21:16:04,781::task::495::TaskManager.Task::(_debug) Task a184a53e-c62f-4ce9-8ef5-772eabf31f99: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910595::DEBUG::2013-11-19 21:16:04,782::task::495::TaskManager.Task::(_debug) Task a184a53e-c62f-4ce9-8ef5-772eabf31f99: ref 1 aborting False
>Thread-15910595::WARNING::2013-11-19 21:16:04,784::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910595::DEBUG::2013-11-19 21:16:04,786::task::495::TaskManager.Task::(_debug) Task a184a53e-c62f-4ce9-8ef5-772eabf31f99: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910595::DEBUG::2013-11-19 21:16:04,787::task::495::TaskManager.Task::(_debug) Task a184a53e-c62f-4ce9-8ef5-772eabf31f99: moving from state preparing -> state finished
>Thread-15910595::DEBUG::2013-11-19 21:16:04,788::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910595::DEBUG::2013-11-19 21:16:04,789::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910595::DEBUG::2013-11-19 21:16:04,790::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910595::DEBUG::2013-11-19 21:16:04,792::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910595::DEBUG::2013-11-19 21:16:04,792::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910595::DEBUG::2013-11-19 21:16:04,793::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910595::DEBUG::2013-11-19 21:16:04,795::task::495::TaskManager.Task::(_debug) Task a184a53e-c62f-4ce9-8ef5-772eabf31f99: ref 0 aborting False
>Thread-15910595::INFO::2013-11-19 21:16:04,796::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15910548::DEBUG::2013-11-19 21:16:04,936::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.106169 s, 9.6 MB/s\n'; <rc> = 0
>Thread-15910596::INFO::2013-11-19 21:16:06,671::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910596::DEBUG::2013-11-19 21:16:06,672::task::495::TaskManager.Task::(_debug) Task bf3ecb62-1d20-442e-ba8d-24633a23789b: moving from state init -> state preparing
>Thread-15910596::DEBUG::2013-11-19 21:16:06,672::task::495::TaskManager.Task::(_debug) Task bf3ecb62-1d20-442e-ba8d-24633a23789b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000333070755005', 'lastCheck': 1384895757.429872, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00161910057068', 'lastCheck': 1384895759.5464809, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895764.663202, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00043511390686', 'lastCheck': 1384895763.544986, 'valid': True, 'code': 0}}
>Thread-15910596::DEBUG::2013-11-19 21:16:06,673::task::495::TaskManager.Task::(_debug) Task bf3ecb62-1d20-442e-ba8d-24633a23789b: moving from state preparing -> state finished
>Thread-15910596::DEBUG::2013-11-19 21:16:06,674::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910596::DEBUG::2013-11-19 21:16:06,674::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910596::DEBUG::2013-11-19 21:16:06,675::task::495::TaskManager.Task::(_debug) Task bf3ecb62-1d20-442e-ba8d-24633a23789b: ref 0 aborting False
>Thread-15910596::INFO::2013-11-19 21:16:06,676::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000333070755005', 'lastCheck': 1384895757.429872, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00161910057068', 'lastCheck': 1384895759.5464809, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895764.663202, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00043511390686', 'lastCheck': 1384895763.544986, 'valid': True, 'code': 0}}
>Dummy-15910548::DEBUG::2013-11-19 21:16:06,952::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:16:07,495::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0750174 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:16:09,509::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:16:10,015::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0707134 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:16:12,031::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:16:12,564::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0709703 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:16:14,580::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:16:14,798::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910602::DEBUG::2013-11-19 21:16:14,920::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910602::INFO::2013-11-19 21:16:14,922::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910602::DEBUG::2013-11-19 21:16:14,923::task::495::TaskManager.Task::(_debug) Task 8584ffaa-9d60-4a55-9edb-fdbf5953856a: moving from state init -> state preparing
>Thread-15910602::DEBUG::2013-11-19 21:16:14,924::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=278 spmId=2
>Thread-15910602::DEBUG::2013-11-19 21:16:14,924::task::495::TaskManager.Task::(_debug) Task 8584ffaa-9d60-4a55-9edb-fdbf5953856a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910602::DEBUG::2013-11-19 21:16:14,925::task::495::TaskManager.Task::(_debug) Task 8584ffaa-9d60-4a55-9edb-fdbf5953856a: moving from state preparing -> state finished
>Thread-15910602::DEBUG::2013-11-19 21:16:14,926::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910602::DEBUG::2013-11-19 21:16:14,926::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910602::DEBUG::2013-11-19 21:16:14,927::task::495::TaskManager.Task::(_debug) Task 8584ffaa-9d60-4a55-9edb-fdbf5953856a: ref 0 aborting False
>Thread-15910602::INFO::2013-11-19 21:16:14,927::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910603::DEBUG::2013-11-19 21:16:14,932::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910603::INFO::2013-11-19 21:16:14,933::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910603::DEBUG::2013-11-19 21:16:14,934::task::495::TaskManager.Task::(_debug) Task 828e2416-f6af-40dd-ab36-9d27d77323fe: moving from state init -> state preparing
>Thread-15910603::DEBUG::2013-11-19 21:16:14,936::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2a7b7918-da39-4c02-9913-f87819b3de26`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910603::DEBUG::2013-11-19 21:16:14,936::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910603::DEBUG::2013-11-19 21:16:14,937::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910603::DEBUG::2013-11-19 21:16:14,938::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2a7b7918-da39-4c02-9913-f87819b3de26`::Granted request
>Thread-15910603::DEBUG::2013-11-19 21:16:14,939::task::495::TaskManager.Task::(_debug) Task 828e2416-f6af-40dd-ab36-9d27d77323fe: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910603::DEBUG::2013-11-19 21:16:14,939::task::495::TaskManager.Task::(_debug) Task 828e2416-f6af-40dd-ab36-9d27d77323fe: ref 1 aborting False
>Thread-15910603::WARNING::2013-11-19 21:16:14,942::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910603::DEBUG::2013-11-19 21:16:14,943::task::495::TaskManager.Task::(_debug) Task 828e2416-f6af-40dd-ab36-9d27d77323fe: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910603::DEBUG::2013-11-19 21:16:14,944::task::495::TaskManager.Task::(_debug) Task 828e2416-f6af-40dd-ab36-9d27d77323fe: moving from state preparing -> state finished
>Thread-15910603::DEBUG::2013-11-19 21:16:14,944::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910603::DEBUG::2013-11-19 21:16:14,945::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910603::DEBUG::2013-11-19 21:16:14,946::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910603::DEBUG::2013-11-19 21:16:14,946::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910603::DEBUG::2013-11-19 21:16:14,947::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910603::DEBUG::2013-11-19 21:16:14,947::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910603::DEBUG::2013-11-19 21:16:14,948::task::495::TaskManager.Task::(_debug) Task 828e2416-f6af-40dd-ab36-9d27d77323fe: ref 0 aborting False
>Thread-15910603::INFO::2013-11-19 21:16:14,949::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15910548::DEBUG::2013-11-19 21:16:15,106::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0713685 s, 14.3 MB/s\n'; <rc> = 0
>Thread-15910604::INFO::2013-11-19 21:16:16,810::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910604::DEBUG::2013-11-19 21:16:16,811::task::495::TaskManager.Task::(_debug) Task 9ff9cddb-23d7-4c08-a9fc-480471740f7e: moving from state init -> state preparing
>Thread-15910604::DEBUG::2013-11-19 21:16:16,811::task::495::TaskManager.Task::(_debug) Task 9ff9cddb-23d7-4c08-a9fc-480471740f7e: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00046706199646', 'lastCheck': 1384895767.6651831, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00163722038269', 'lastCheck': 1384895769.7116101, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895774.802418, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000565052032471', 'lastCheck': 1384895773.709693, 'valid': True, 'code': 0}}
>Thread-15910604::DEBUG::2013-11-19 21:16:16,813::task::495::TaskManager.Task::(_debug) Task 9ff9cddb-23d7-4c08-a9fc-480471740f7e: moving from state preparing -> state finished
>Thread-15910604::DEBUG::2013-11-19 21:16:16,813::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910604::DEBUG::2013-11-19 21:16:16,814::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910604::DEBUG::2013-11-19 21:16:16,815::task::495::TaskManager.Task::(_debug) Task 9ff9cddb-23d7-4c08-a9fc-480471740f7e: ref 0 aborting False
>Thread-15910604::INFO::2013-11-19 21:16:16,815::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00046706199646', 'lastCheck': 1384895767.6651831, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00163722038269', 'lastCheck': 1384895769.7116101, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895774.802418, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000565052032471', 'lastCheck': 1384895773.709693, 'valid': True, 'code': 0}}
>Dummy-15910548::DEBUG::2013-11-19 21:16:17,121::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:16:17,634::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.070076 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:16:19,649::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:16:20,210::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0717554 s, 14.3 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:16:22,226::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:16:22,763::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0723858 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:16:24,778::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:16:25,017::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910610::DEBUG::2013-11-19 21:16:25,060::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910610::INFO::2013-11-19 21:16:25,061::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910610::DEBUG::2013-11-19 21:16:25,062::task::495::TaskManager.Task::(_debug) Task df23ec8a-4061-438f-9c23-4465902f2873: moving from state init -> state preparing
>Thread-15910610::DEBUG::2013-11-19 21:16:25,063::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=278 spmId=2
>Thread-15910610::DEBUG::2013-11-19 21:16:25,064::task::495::TaskManager.Task::(_debug) Task df23ec8a-4061-438f-9c23-4465902f2873: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910610::DEBUG::2013-11-19 21:16:25,064::task::495::TaskManager.Task::(_debug) Task df23ec8a-4061-438f-9c23-4465902f2873: moving from state preparing -> state finished
>Thread-15910610::DEBUG::2013-11-19 21:16:25,065::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910610::DEBUG::2013-11-19 21:16:25,066::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910610::DEBUG::2013-11-19 21:16:25,066::task::495::TaskManager.Task::(_debug) Task df23ec8a-4061-438f-9c23-4465902f2873: ref 0 aborting False
>Thread-15910610::INFO::2013-11-19 21:16:25,067::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910611::DEBUG::2013-11-19 21:16:25,072::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910611::INFO::2013-11-19 21:16:25,073::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910611::DEBUG::2013-11-19 21:16:25,073::task::495::TaskManager.Task::(_debug) Task 443cadef-28a0-4905-bf5d-e7c8692abb9a: moving from state init -> state preparing
>Thread-15910611::DEBUG::2013-11-19 21:16:25,075::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`26e8e958-c795-4d1c-9ec6-c0ecc427f3f7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910611::DEBUG::2013-11-19 21:16:25,075::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910611::DEBUG::2013-11-19 21:16:25,076::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910611::DEBUG::2013-11-19 21:16:25,077::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`26e8e958-c795-4d1c-9ec6-c0ecc427f3f7`::Granted request
>Thread-15910611::DEBUG::2013-11-19 21:16:25,078::task::495::TaskManager.Task::(_debug) Task 443cadef-28a0-4905-bf5d-e7c8692abb9a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910611::DEBUG::2013-11-19 21:16:25,079::task::495::TaskManager.Task::(_debug) Task 443cadef-28a0-4905-bf5d-e7c8692abb9a: ref 1 aborting False
>Thread-15910611::WARNING::2013-11-19 21:16:25,081::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910611::DEBUG::2013-11-19 21:16:25,082::task::495::TaskManager.Task::(_debug) Task 443cadef-28a0-4905-bf5d-e7c8692abb9a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910611::DEBUG::2013-11-19 21:16:25,083::task::495::TaskManager.Task::(_debug) Task 443cadef-28a0-4905-bf5d-e7c8692abb9a: moving from state preparing -> state finished
>Thread-15910611::DEBUG::2013-11-19 21:16:25,083::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910611::DEBUG::2013-11-19 21:16:25,084::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910611::DEBUG::2013-11-19 21:16:25,085::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910611::DEBUG::2013-11-19 21:16:25,085::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910611::DEBUG::2013-11-19 21:16:25,086::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910611::DEBUG::2013-11-19 21:16:25,086::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910611::DEBUG::2013-11-19 21:16:25,087::task::495::TaskManager.Task::(_debug) Task 443cadef-28a0-4905-bf5d-e7c8692abb9a: ref 0 aborting False
>Thread-15910611::INFO::2013-11-19 21:16:25,088::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15910548::DEBUG::2013-11-19 21:16:25,326::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0714091 s, 14.3 MB/s\n'; <rc> = 0
>Thread-15910612::INFO::2013-11-19 21:16:27,027::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910612::DEBUG::2013-11-19 21:16:27,028::task::495::TaskManager.Task::(_debug) Task cafbae51-4acf-4c81-a21a-3bd298753610: moving from state init -> state preparing
>Thread-15910612::DEBUG::2013-11-19 21:16:27,028::task::495::TaskManager.Task::(_debug) Task cafbae51-4acf-4c81-a21a-3bd298753610: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000403165817261', 'lastCheck': 1384895777.7982261, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00130200386047', 'lastCheck': 1384895779.9537661, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895785.0196431, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000472068786621', 'lastCheck': 1384895783.897423, 'valid': True, 'code': 0}}
>Thread-15910612::DEBUG::2013-11-19 21:16:27,029::task::495::TaskManager.Task::(_debug) Task cafbae51-4acf-4c81-a21a-3bd298753610: moving from state preparing -> state finished
>Thread-15910612::DEBUG::2013-11-19 21:16:27,030::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910612::DEBUG::2013-11-19 21:16:27,030::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910612::DEBUG::2013-11-19 21:16:27,031::task::495::TaskManager.Task::(_debug) Task cafbae51-4acf-4c81-a21a-3bd298753610: ref 0 aborting False
>Thread-15910612::INFO::2013-11-19 21:16:27,032::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000403165817261', 'lastCheck': 1384895777.7982261, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00130200386047', 'lastCheck': 1384895779.9537661, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895785.0196431, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000472068786621', 'lastCheck': 1384895783.897423, 'valid': True, 'code': 0}}
>Dummy-15910548::DEBUG::2013-11-19 21:16:27,342::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:16:27,873::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.070541 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:16:29,888::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:16:30,409::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0701698 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:16:32,424::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:16:32,960::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0708102 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:16:34,975::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:16:35,193::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15910548::DEBUG::2013-11-19 21:16:35,501::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0707617 s, 14.5 MB/s\n'; <rc> = 0
>Thread-15910618::DEBUG::2013-11-19 21:16:35,536::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910618::INFO::2013-11-19 21:16:35,537::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910618::DEBUG::2013-11-19 21:16:35,538::task::495::TaskManager.Task::(_debug) Task 015fec1f-4fa0-4c87-ac4d-85c985b78c0a: moving from state init -> state preparing
>Thread-15910618::DEBUG::2013-11-19 21:16:35,539::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=278 spmId=2
>Thread-15910618::DEBUG::2013-11-19 21:16:35,539::task::495::TaskManager.Task::(_debug) Task 015fec1f-4fa0-4c87-ac4d-85c985b78c0a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910618::DEBUG::2013-11-19 21:16:35,540::task::495::TaskManager.Task::(_debug) Task 015fec1f-4fa0-4c87-ac4d-85c985b78c0a: moving from state preparing -> state finished
>Thread-15910618::DEBUG::2013-11-19 21:16:35,541::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910618::DEBUG::2013-11-19 21:16:35,541::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910618::DEBUG::2013-11-19 21:16:35,542::task::495::TaskManager.Task::(_debug) Task 015fec1f-4fa0-4c87-ac4d-85c985b78c0a: ref 0 aborting False
>Thread-15910618::INFO::2013-11-19 21:16:35,542::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910619::DEBUG::2013-11-19 21:16:35,547::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910619::INFO::2013-11-19 21:16:35,548::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910619::DEBUG::2013-11-19 21:16:35,549::task::495::TaskManager.Task::(_debug) Task d5f62c79-a02f-4a2a-a55a-c28b67db1fe2: moving from state init -> state preparing
>Thread-15910619::DEBUG::2013-11-19 21:16:35,563::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b0149772-b837-4b82-969c-ffcbba40661e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910619::DEBUG::2013-11-19 21:16:35,564::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910619::DEBUG::2013-11-19 21:16:35,564::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910619::DEBUG::2013-11-19 21:16:35,565::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b0149772-b837-4b82-969c-ffcbba40661e`::Granted request
>Thread-15910619::DEBUG::2013-11-19 21:16:35,566::task::495::TaskManager.Task::(_debug) Task d5f62c79-a02f-4a2a-a55a-c28b67db1fe2: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910619::DEBUG::2013-11-19 21:16:35,567::task::495::TaskManager.Task::(_debug) Task d5f62c79-a02f-4a2a-a55a-c28b67db1fe2: ref 1 aborting False
>Thread-15910619::WARNING::2013-11-19 21:16:35,569::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910619::DEBUG::2013-11-19 21:16:35,569::task::495::TaskManager.Task::(_debug) Task d5f62c79-a02f-4a2a-a55a-c28b67db1fe2: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910619::DEBUG::2013-11-19 21:16:35,570::task::495::TaskManager.Task::(_debug) Task d5f62c79-a02f-4a2a-a55a-c28b67db1fe2: moving from state preparing -> state finished
>Thread-15910619::DEBUG::2013-11-19 21:16:35,571::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910619::DEBUG::2013-11-19 21:16:35,571::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910619::DEBUG::2013-11-19 21:16:35,572::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910619::DEBUG::2013-11-19 21:16:35,573::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910619::DEBUG::2013-11-19 21:16:35,573::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910619::DEBUG::2013-11-19 21:16:35,574::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910619::DEBUG::2013-11-19 21:16:35,574::task::495::TaskManager.Task::(_debug) Task d5f62c79-a02f-4a2a-a55a-c28b67db1fe2: ref 0 aborting False
>Thread-15910619::INFO::2013-11-19 21:16:35,575::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910620::INFO::2013-11-19 21:16:37,205::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910620::DEBUG::2013-11-19 21:16:37,206::task::495::TaskManager.Task::(_debug) Task 673882be-4045-4f77-ac8a-6bf5bec06d38: moving from state init -> state preparing
>Thread-15910620::DEBUG::2013-11-19 21:16:37,206::task::495::TaskManager.Task::(_debug) Task 673882be-4045-4f77-ac8a-6bf5bec06d38: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000420808792114', 'lastCheck': 1384895788.0150459, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0015881061554', 'lastCheck': 1384895790.1179559, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895795.1972849, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000478029251099', 'lastCheck': 1384895794.1134479, 'valid': True, 'code': 0}}
>Thread-15910620::DEBUG::2013-11-19 21:16:37,207::task::495::TaskManager.Task::(_debug) Task 673882be-4045-4f77-ac8a-6bf5bec06d38: moving from state preparing -> state finished
>Thread-15910620::DEBUG::2013-11-19 21:16:37,208::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910620::DEBUG::2013-11-19 21:16:37,208::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910620::DEBUG::2013-11-19 21:16:37,209::task::495::TaskManager.Task::(_debug) Task 673882be-4045-4f77-ac8a-6bf5bec06d38: ref 0 aborting False
>Thread-15910620::INFO::2013-11-19 21:16:37,209::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000420808792114', 'lastCheck': 1384895788.0150459, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0015881061554', 'lastCheck': 1384895790.1179559, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895795.1972849, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000478029251099', 'lastCheck': 1384895794.1134479, 'valid': True, 'code': 0}}
>Dummy-15910548::DEBUG::2013-11-19 21:16:37,515::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:16:38,083::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.108334 s, 9.5 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:16:40,099::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:16:40,627::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.071435 s, 14.3 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:16:42,642::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:16:43,162::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0699412 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:16:45,177::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:16:45,404::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15910548::DEBUG::2013-11-19 21:16:45,710::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0715822 s, 14.3 MB/s\n'; <rc> = 0
>Thread-15910626::DEBUG::2013-11-19 21:16:46,058::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910626::INFO::2013-11-19 21:16:46,060::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910626::DEBUG::2013-11-19 21:16:46,060::task::495::TaskManager.Task::(_debug) Task a2c429d9-bbaa-4793-b32b-9ea7340a57d5: moving from state init -> state preparing
>Thread-15910626::DEBUG::2013-11-19 21:16:46,061::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=278 spmId=2
>Thread-15910626::DEBUG::2013-11-19 21:16:46,062::task::495::TaskManager.Task::(_debug) Task a2c429d9-bbaa-4793-b32b-9ea7340a57d5: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910626::DEBUG::2013-11-19 21:16:46,062::task::495::TaskManager.Task::(_debug) Task a2c429d9-bbaa-4793-b32b-9ea7340a57d5: moving from state preparing -> state finished
>Thread-15910626::DEBUG::2013-11-19 21:16:46,063::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910626::DEBUG::2013-11-19 21:16:46,063::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910626::DEBUG::2013-11-19 21:16:46,064::task::495::TaskManager.Task::(_debug) Task a2c429d9-bbaa-4793-b32b-9ea7340a57d5: ref 0 aborting False
>Thread-15910626::INFO::2013-11-19 21:16:46,065::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910627::DEBUG::2013-11-19 21:16:46,069::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910627::INFO::2013-11-19 21:16:46,070::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910627::DEBUG::2013-11-19 21:16:46,071::task::495::TaskManager.Task::(_debug) Task 35f17e52-73ef-4be7-aa04-e9b4479df747: moving from state init -> state preparing
>Thread-15910627::DEBUG::2013-11-19 21:16:46,072::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`09a913e6-1961-4c59-bbd0-fb24793819f9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910627::DEBUG::2013-11-19 21:16:46,073::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910627::DEBUG::2013-11-19 21:16:46,073::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910627::DEBUG::2013-11-19 21:16:46,074::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`09a913e6-1961-4c59-bbd0-fb24793819f9`::Granted request
>Thread-15910627::DEBUG::2013-11-19 21:16:46,075::task::495::TaskManager.Task::(_debug) Task 35f17e52-73ef-4be7-aa04-e9b4479df747: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910627::DEBUG::2013-11-19 21:16:46,076::task::495::TaskManager.Task::(_debug) Task 35f17e52-73ef-4be7-aa04-e9b4479df747: ref 1 aborting False
>Thread-15910627::WARNING::2013-11-19 21:16:46,078::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910627::DEBUG::2013-11-19 21:16:46,079::task::495::TaskManager.Task::(_debug) Task 35f17e52-73ef-4be7-aa04-e9b4479df747: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910627::DEBUG::2013-11-19 21:16:46,079::task::495::TaskManager.Task::(_debug) Task 35f17e52-73ef-4be7-aa04-e9b4479df747: moving from state preparing -> state finished
>Thread-15910627::DEBUG::2013-11-19 21:16:46,080::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910627::DEBUG::2013-11-19 21:16:46,080::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910627::DEBUG::2013-11-19 21:16:46,081::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910627::DEBUG::2013-11-19 21:16:46,082::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910627::DEBUG::2013-11-19 21:16:46,082::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910627::DEBUG::2013-11-19 21:16:46,083::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910627::DEBUG::2013-11-19 21:16:46,083::task::495::TaskManager.Task::(_debug) Task 35f17e52-73ef-4be7-aa04-e9b4479df747: ref 0 aborting False
>Thread-15910627::INFO::2013-11-19 21:16:46,084::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910628::INFO::2013-11-19 21:16:47,414::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910628::DEBUG::2013-11-19 21:16:47,414::task::495::TaskManager.Task::(_debug) Task afec0397-2d43-45c3-bcaf-1c01c6afc652: moving from state init -> state preparing
>Thread-15910628::DEBUG::2013-11-19 21:16:47,415::task::495::TaskManager.Task::(_debug) Task afec0397-2d43-45c3-bcaf-1c01c6afc652: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000419139862061', 'lastCheck': 1384895798.18977, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00225496292114', 'lastCheck': 1384895800.338099, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895805.4073839, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00051212310791', 'lastCheck': 1384895804.3251081, 'valid': True, 'code': 0}}
>Thread-15910628::DEBUG::2013-11-19 21:16:47,416::task::495::TaskManager.Task::(_debug) Task afec0397-2d43-45c3-bcaf-1c01c6afc652: moving from state preparing -> state finished
>Thread-15910628::DEBUG::2013-11-19 21:16:47,416::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910628::DEBUG::2013-11-19 21:16:47,417::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910628::DEBUG::2013-11-19 21:16:47,418::task::495::TaskManager.Task::(_debug) Task afec0397-2d43-45c3-bcaf-1c01c6afc652: ref 0 aborting False
>Thread-15910628::INFO::2013-11-19 21:16:47,418::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000419139862061', 'lastCheck': 1384895798.18977, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00225496292114', 'lastCheck': 1384895800.338099, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895805.4073839, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00051212310791', 'lastCheck': 1384895804.3251081, 'valid': True, 'code': 0}}
>Dummy-15910548::DEBUG::2013-11-19 21:16:47,725::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:16:48,251::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.070392 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:16:50,266::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:16:50,786::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0703785 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:16:52,802::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:16:53,353::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.071789 s, 14.3 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:16:55,368::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:16:55,602::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15910548::DEBUG::2013-11-19 21:16:55,890::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0707244 s, 14.5 MB/s\n'; <rc> = 0
>Thread-15910634::DEBUG::2013-11-19 21:16:56,190::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910634::INFO::2013-11-19 21:16:56,191::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910634::DEBUG::2013-11-19 21:16:56,192::task::495::TaskManager.Task::(_debug) Task 4b052b01-732f-4fda-a0e1-33b65449a4ba: moving from state init -> state preparing
>Thread-15910634::DEBUG::2013-11-19 21:16:56,194::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=278 spmId=2
>Thread-15910634::DEBUG::2013-11-19 21:16:56,195::task::495::TaskManager.Task::(_debug) Task 4b052b01-732f-4fda-a0e1-33b65449a4ba: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910634::DEBUG::2013-11-19 21:16:56,196::task::495::TaskManager.Task::(_debug) Task 4b052b01-732f-4fda-a0e1-33b65449a4ba: moving from state preparing -> state finished
>Thread-15910634::DEBUG::2013-11-19 21:16:56,197::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910634::DEBUG::2013-11-19 21:16:56,197::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910634::DEBUG::2013-11-19 21:16:56,198::task::495::TaskManager.Task::(_debug) Task 4b052b01-732f-4fda-a0e1-33b65449a4ba: ref 0 aborting False
>Thread-15910634::INFO::2013-11-19 21:16:56,198::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910635::DEBUG::2013-11-19 21:16:56,203::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910635::INFO::2013-11-19 21:16:56,204::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910635::DEBUG::2013-11-19 21:16:56,205::task::495::TaskManager.Task::(_debug) Task 50ac24df-7d97-4689-b865-62ea27e93e91: moving from state init -> state preparing
>Thread-15910635::DEBUG::2013-11-19 21:16:56,206::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d1d2f717-9fdf-4e2b-a6ce-797e40fd931c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910635::DEBUG::2013-11-19 21:16:56,206::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910635::DEBUG::2013-11-19 21:16:56,207::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910635::DEBUG::2013-11-19 21:16:56,208::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d1d2f717-9fdf-4e2b-a6ce-797e40fd931c`::Granted request
>Thread-15910635::DEBUG::2013-11-19 21:16:56,209::task::495::TaskManager.Task::(_debug) Task 50ac24df-7d97-4689-b865-62ea27e93e91: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910635::DEBUG::2013-11-19 21:16:56,209::task::495::TaskManager.Task::(_debug) Task 50ac24df-7d97-4689-b865-62ea27e93e91: ref 1 aborting False
>Thread-15910635::WARNING::2013-11-19 21:16:56,211::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910635::DEBUG::2013-11-19 21:16:56,212::task::495::TaskManager.Task::(_debug) Task 50ac24df-7d97-4689-b865-62ea27e93e91: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910635::DEBUG::2013-11-19 21:16:56,213::task::495::TaskManager.Task::(_debug) Task 50ac24df-7d97-4689-b865-62ea27e93e91: moving from state preparing -> state finished
>Thread-15910635::DEBUG::2013-11-19 21:16:56,214::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910635::DEBUG::2013-11-19 21:16:56,214::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910635::DEBUG::2013-11-19 21:16:56,215::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910635::DEBUG::2013-11-19 21:16:56,216::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910635::DEBUG::2013-11-19 21:16:56,216::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910635::DEBUG::2013-11-19 21:16:56,217::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910635::DEBUG::2013-11-19 21:16:56,217::task::495::TaskManager.Task::(_debug) Task 50ac24df-7d97-4689-b865-62ea27e93e91: ref 0 aborting False
>Thread-15910635::INFO::2013-11-19 21:16:56,218::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910636::INFO::2013-11-19 21:16:57,613::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910636::DEBUG::2013-11-19 21:16:57,614::task::495::TaskManager.Task::(_debug) Task 4f29e1a0-a441-43fe-8f1f-b82cce5cc745: moving from state init -> state preparing
>Thread-15910636::DEBUG::2013-11-19 21:16:57,614::task::495::TaskManager.Task::(_debug) Task 4f29e1a0-a441-43fe-8f1f-b82cce5cc745: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000356912612915', 'lastCheck': 1384895808.401027, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00161600112915', 'lastCheck': 1384895810.4942491, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895815.6064429, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000406980514526', 'lastCheck': 1384895814.491873, 'valid': True, 'code': 0}}
>Thread-15910636::DEBUG::2013-11-19 21:16:57,615::task::495::TaskManager.Task::(_debug) Task 4f29e1a0-a441-43fe-8f1f-b82cce5cc745: moving from state preparing -> state finished
>Thread-15910636::DEBUG::2013-11-19 21:16:57,616::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910636::DEBUG::2013-11-19 21:16:57,616::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910636::DEBUG::2013-11-19 21:16:57,617::task::495::TaskManager.Task::(_debug) Task 4f29e1a0-a441-43fe-8f1f-b82cce5cc745: ref 0 aborting False
>Thread-15910636::INFO::2013-11-19 21:16:57,617::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000356912612915', 'lastCheck': 1384895808.401027, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00161600112915', 'lastCheck': 1384895810.4942491, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895815.6064429, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000406980514526', 'lastCheck': 1384895814.491873, 'valid': True, 'code': 0}}
>Dummy-15910548::DEBUG::2013-11-19 21:16:57,906::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:16:58,453::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0713689 s, 14.3 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:17:00,468::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:17:00,985::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0680891 s, 15.0 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:17:03,000::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:17:03,517::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0705446 s, 14.5 MB/s\n'; <rc> = 0
>Thread-42840::DEBUG::2013-11-19 21:17:04,679::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-19 21:17:05,366::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:17:05,533::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:17:05,765::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15910548::DEBUG::2013-11-19 21:17:06,081::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0714527 s, 14.3 MB/s\n'; <rc> = 0
>Thread-15910642::DEBUG::2013-11-19 21:17:06,339::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910642::INFO::2013-11-19 21:17:06,340::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910642::DEBUG::2013-11-19 21:17:06,341::task::495::TaskManager.Task::(_debug) Task c83301f2-645b-4044-aaf6-aaae537cb59a: moving from state init -> state preparing
>Thread-15910642::DEBUG::2013-11-19 21:17:06,342::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=278 spmId=2
>Thread-15910642::DEBUG::2013-11-19 21:17:06,342::task::495::TaskManager.Task::(_debug) Task c83301f2-645b-4044-aaf6-aaae537cb59a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910642::DEBUG::2013-11-19 21:17:06,343::task::495::TaskManager.Task::(_debug) Task c83301f2-645b-4044-aaf6-aaae537cb59a: moving from state preparing -> state finished
>Thread-15910642::DEBUG::2013-11-19 21:17:06,343::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910642::DEBUG::2013-11-19 21:17:06,344::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910642::DEBUG::2013-11-19 21:17:06,344::task::495::TaskManager.Task::(_debug) Task c83301f2-645b-4044-aaf6-aaae537cb59a: ref 0 aborting False
>Thread-15910642::INFO::2013-11-19 21:17:06,345::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910643::DEBUG::2013-11-19 21:17:06,350::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910643::INFO::2013-11-19 21:17:06,351::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910643::DEBUG::2013-11-19 21:17:06,352::task::495::TaskManager.Task::(_debug) Task cc57e2a2-d62a-4125-a63b-e55356b9b47b: moving from state init -> state preparing
>Thread-15910643::DEBUG::2013-11-19 21:17:06,353::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`312bbdf6-0e0d-445d-a2a8-4a4ba538b42a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910643::DEBUG::2013-11-19 21:17:06,353::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910643::DEBUG::2013-11-19 21:17:06,354::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910643::DEBUG::2013-11-19 21:17:06,355::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`312bbdf6-0e0d-445d-a2a8-4a4ba538b42a`::Granted request
>Thread-15910643::DEBUG::2013-11-19 21:17:06,356::task::495::TaskManager.Task::(_debug) Task cc57e2a2-d62a-4125-a63b-e55356b9b47b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910643::DEBUG::2013-11-19 21:17:06,356::task::495::TaskManager.Task::(_debug) Task cc57e2a2-d62a-4125-a63b-e55356b9b47b: ref 1 aborting False
>Thread-15910643::WARNING::2013-11-19 21:17:06,358::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910643::DEBUG::2013-11-19 21:17:06,359::task::495::TaskManager.Task::(_debug) Task cc57e2a2-d62a-4125-a63b-e55356b9b47b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910643::DEBUG::2013-11-19 21:17:06,360::task::495::TaskManager.Task::(_debug) Task cc57e2a2-d62a-4125-a63b-e55356b9b47b: moving from state preparing -> state finished
>Thread-15910643::DEBUG::2013-11-19 21:17:06,361::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910643::DEBUG::2013-11-19 21:17:06,361::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910643::DEBUG::2013-11-19 21:17:06,362::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910643::DEBUG::2013-11-19 21:17:06,362::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910643::DEBUG::2013-11-19 21:17:06,363::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910643::DEBUG::2013-11-19 21:17:06,364::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910643::DEBUG::2013-11-19 21:17:06,364::task::495::TaskManager.Task::(_debug) Task cc57e2a2-d62a-4125-a63b-e55356b9b47b: ref 0 aborting False
>Thread-15910643::INFO::2013-11-19 21:17:06,365::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910644::INFO::2013-11-19 21:17:07,801::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910644::DEBUG::2013-11-19 21:17:07,802::task::495::TaskManager.Task::(_debug) Task f4a3f23b-c005-4622-99bd-521fef818601: moving from state init -> state preparing
>Thread-15910644::DEBUG::2013-11-19 21:17:07,803::task::495::TaskManager.Task::(_debug) Task f4a3f23b-c005-4622-99bd-521fef818601: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00232195854187', 'lastCheck': 1384895818.5982511, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00161409378052', 'lastCheck': 1384895820.683538, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895825.767473, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000328063964844', 'lastCheck': 1384895825.3684521, 'valid': True, 'code': 0}}
>Thread-15910644::DEBUG::2013-11-19 21:17:07,804::task::495::TaskManager.Task::(_debug) Task f4a3f23b-c005-4622-99bd-521fef818601: moving from state preparing -> state finished
>Thread-15910644::DEBUG::2013-11-19 21:17:07,804::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910644::DEBUG::2013-11-19 21:17:07,805::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910644::DEBUG::2013-11-19 21:17:07,805::task::495::TaskManager.Task::(_debug) Task f4a3f23b-c005-4622-99bd-521fef818601: ref 0 aborting False
>Thread-15910644::INFO::2013-11-19 21:17:07,806::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00232195854187', 'lastCheck': 1384895818.5982511, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00161409378052', 'lastCheck': 1384895820.683538, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895825.767473, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000328063964844', 'lastCheck': 1384895825.3684521, 'valid': True, 'code': 0}}
>Dummy-15910548::DEBUG::2013-11-19 21:17:08,096::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:17:08,630::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0707999 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:17:10,645::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:17:11,215::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0932177 s, 11.0 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:17:13,230::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:17:13,760::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0704161 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:17:15,777::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:17:16,014::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15910548::DEBUG::2013-11-19 21:17:16,322::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.071351 s, 14.4 MB/s\n'; <rc> = 0
>Thread-15910650::DEBUG::2013-11-19 21:17:16,480::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910650::INFO::2013-11-19 21:17:16,482::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910650::DEBUG::2013-11-19 21:17:16,482::task::495::TaskManager.Task::(_debug) Task 14cb84da-7b59-4851-a88e-89a5aa76e7db: moving from state init -> state preparing
>Thread-15910650::DEBUG::2013-11-19 21:17:16,483::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=278 spmId=2
>Thread-15910650::DEBUG::2013-11-19 21:17:16,484::task::495::TaskManager.Task::(_debug) Task 14cb84da-7b59-4851-a88e-89a5aa76e7db: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910650::DEBUG::2013-11-19 21:17:16,484::task::495::TaskManager.Task::(_debug) Task 14cb84da-7b59-4851-a88e-89a5aa76e7db: moving from state preparing -> state finished
>Thread-15910650::DEBUG::2013-11-19 21:17:16,485::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910650::DEBUG::2013-11-19 21:17:16,485::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910650::DEBUG::2013-11-19 21:17:16,486::task::495::TaskManager.Task::(_debug) Task 14cb84da-7b59-4851-a88e-89a5aa76e7db: ref 0 aborting False
>Thread-15910650::INFO::2013-11-19 21:17:16,487::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910651::DEBUG::2013-11-19 21:17:16,492::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910651::INFO::2013-11-19 21:17:16,493::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910651::DEBUG::2013-11-19 21:17:16,493::task::495::TaskManager.Task::(_debug) Task 038c4f13-e1a7-4962-8455-cf339f1441bf: moving from state init -> state preparing
>Thread-15910651::DEBUG::2013-11-19 21:17:16,495::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3f3a50fd-7c17-4e40-80d9-f0379448f2ef`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910651::DEBUG::2013-11-19 21:17:16,495::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910651::DEBUG::2013-11-19 21:17:16,496::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910651::DEBUG::2013-11-19 21:17:16,497::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3f3a50fd-7c17-4e40-80d9-f0379448f2ef`::Granted request
>Thread-15910651::DEBUG::2013-11-19 21:17:16,498::task::495::TaskManager.Task::(_debug) Task 038c4f13-e1a7-4962-8455-cf339f1441bf: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910651::DEBUG::2013-11-19 21:17:16,498::task::495::TaskManager.Task::(_debug) Task 038c4f13-e1a7-4962-8455-cf339f1441bf: ref 1 aborting False
>Thread-15910651::WARNING::2013-11-19 21:17:16,500::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910651::DEBUG::2013-11-19 21:17:16,501::task::495::TaskManager.Task::(_debug) Task 038c4f13-e1a7-4962-8455-cf339f1441bf: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910651::DEBUG::2013-11-19 21:17:16,502::task::495::TaskManager.Task::(_debug) Task 038c4f13-e1a7-4962-8455-cf339f1441bf: moving from state preparing -> state finished
>Thread-15910651::DEBUG::2013-11-19 21:17:16,502::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910651::DEBUG::2013-11-19 21:17:16,503::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910651::DEBUG::2013-11-19 21:17:16,504::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910651::DEBUG::2013-11-19 21:17:16,505::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910651::DEBUG::2013-11-19 21:17:16,505::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910651::DEBUG::2013-11-19 21:17:16,506::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910651::DEBUG::2013-11-19 21:17:16,506::task::495::TaskManager.Task::(_debug) Task 038c4f13-e1a7-4962-8455-cf339f1441bf: ref 0 aborting False
>Thread-15910651::INFO::2013-11-19 21:17:16,507::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910652::INFO::2013-11-19 21:17:18,023::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910652::DEBUG::2013-11-19 21:17:18,024::task::495::TaskManager.Task::(_debug) Task b8a8f579-8822-4783-98e5-63ce3e9bcc58: moving from state init -> state preparing
>Thread-15910652::DEBUG::2013-11-19 21:17:18,024::task::495::TaskManager.Task::(_debug) Task b8a8f579-8822-4783-98e5-63ce3e9bcc58: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000414133071899', 'lastCheck': 1384895828.7669859, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00159597396851', 'lastCheck': 1384895830.9146409, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895836.016191, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000412940979004', 'lastCheck': 1384895835.44748, 'valid': True, 'code': 0}}
>Thread-15910652::DEBUG::2013-11-19 21:17:18,025::task::495::TaskManager.Task::(_debug) Task b8a8f579-8822-4783-98e5-63ce3e9bcc58: moving from state preparing -> state finished
>Thread-15910652::DEBUG::2013-11-19 21:17:18,026::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910652::DEBUG::2013-11-19 21:17:18,026::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910652::DEBUG::2013-11-19 21:17:18,027::task::495::TaskManager.Task::(_debug) Task b8a8f579-8822-4783-98e5-63ce3e9bcc58: ref 0 aborting False
>Thread-15910652::INFO::2013-11-19 21:17:18,027::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000414133071899', 'lastCheck': 1384895828.7669859, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00159597396851', 'lastCheck': 1384895830.9146409, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895836.016191, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000412940979004', 'lastCheck': 1384895835.44748, 'valid': True, 'code': 0}}
>Dummy-15910548::DEBUG::2013-11-19 21:17:18,337::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:17:18,864::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.071148 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:17:20,880::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:17:21,391::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0705971 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:17:23,406::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:17:23,952::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0716661 s, 14.3 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:17:25,967::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:17:26,238::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15910548::DEBUG::2013-11-19 21:17:26,499::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.071093 s, 14.4 MB/s\n'; <rc> = 0
>Thread-15910658::DEBUG::2013-11-19 21:17:26,620::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910658::INFO::2013-11-19 21:17:26,621::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910658::DEBUG::2013-11-19 21:17:26,622::task::495::TaskManager.Task::(_debug) Task 6664076c-33fb-4e03-939a-ea5340a974a8: moving from state init -> state preparing
>Thread-15910658::DEBUG::2013-11-19 21:17:26,623::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=278 spmId=2
>Thread-15910658::DEBUG::2013-11-19 21:17:26,623::task::495::TaskManager.Task::(_debug) Task 6664076c-33fb-4e03-939a-ea5340a974a8: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910658::DEBUG::2013-11-19 21:17:26,624::task::495::TaskManager.Task::(_debug) Task 6664076c-33fb-4e03-939a-ea5340a974a8: moving from state preparing -> state finished
>Thread-15910658::DEBUG::2013-11-19 21:17:26,624::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910658::DEBUG::2013-11-19 21:17:26,625::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910658::DEBUG::2013-11-19 21:17:26,626::task::495::TaskManager.Task::(_debug) Task 6664076c-33fb-4e03-939a-ea5340a974a8: ref 0 aborting False
>Thread-15910658::INFO::2013-11-19 21:17:26,626::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910659::DEBUG::2013-11-19 21:17:26,631::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910659::INFO::2013-11-19 21:17:26,632::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910659::DEBUG::2013-11-19 21:17:26,633::task::495::TaskManager.Task::(_debug) Task 91419e3c-3b5c-4fe8-b1a4-1968e95829e6: moving from state init -> state preparing
>Thread-15910659::DEBUG::2013-11-19 21:17:26,634::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f68ce179-2830-49e2-aefc-1268b3977925`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910659::DEBUG::2013-11-19 21:17:26,635::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910659::DEBUG::2013-11-19 21:17:26,635::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910659::DEBUG::2013-11-19 21:17:26,636::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f68ce179-2830-49e2-aefc-1268b3977925`::Granted request
>Thread-15910659::DEBUG::2013-11-19 21:17:26,637::task::495::TaskManager.Task::(_debug) Task 91419e3c-3b5c-4fe8-b1a4-1968e95829e6: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910659::DEBUG::2013-11-19 21:17:26,638::task::495::TaskManager.Task::(_debug) Task 91419e3c-3b5c-4fe8-b1a4-1968e95829e6: ref 1 aborting False
>Thread-15910659::WARNING::2013-11-19 21:17:26,640::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910659::DEBUG::2013-11-19 21:17:26,641::task::495::TaskManager.Task::(_debug) Task 91419e3c-3b5c-4fe8-b1a4-1968e95829e6: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910659::DEBUG::2013-11-19 21:17:26,641::task::495::TaskManager.Task::(_debug) Task 91419e3c-3b5c-4fe8-b1a4-1968e95829e6: moving from state preparing -> state finished
>Thread-15910659::DEBUG::2013-11-19 21:17:26,642::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910659::DEBUG::2013-11-19 21:17:26,643::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910659::DEBUG::2013-11-19 21:17:26,643::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910659::DEBUG::2013-11-19 21:17:26,644::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910659::DEBUG::2013-11-19 21:17:26,644::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910659::DEBUG::2013-11-19 21:17:26,645::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910659::DEBUG::2013-11-19 21:17:26,646::task::495::TaskManager.Task::(_debug) Task 91419e3c-3b5c-4fe8-b1a4-1968e95829e6: ref 0 aborting False
>Thread-15910659::INFO::2013-11-19 21:17:26,646::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910660::INFO::2013-11-19 21:17:28,249::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910660::DEBUG::2013-11-19 21:17:28,249::task::495::TaskManager.Task::(_debug) Task 72d1de26-d6da-44e8-a5d7-768435acd2c6: moving from state init -> state preparing
>Thread-15910660::DEBUG::2013-11-19 21:17:28,250::task::495::TaskManager.Task::(_debug) Task 72d1de26-d6da-44e8-a5d7-768435acd2c6: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000483989715576', 'lastCheck': 1384895839.010541, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00164389610291', 'lastCheck': 1384895841.0841999, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895846.242121, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0003981590271', 'lastCheck': 1384895845.6361229, 'valid': True, 'code': 0}}
>Thread-15910660::DEBUG::2013-11-19 21:17:28,251::task::495::TaskManager.Task::(_debug) Task 72d1de26-d6da-44e8-a5d7-768435acd2c6: moving from state preparing -> state finished
>Thread-15910660::DEBUG::2013-11-19 21:17:28,251::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910660::DEBUG::2013-11-19 21:17:28,252::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910660::DEBUG::2013-11-19 21:17:28,252::task::495::TaskManager.Task::(_debug) Task 72d1de26-d6da-44e8-a5d7-768435acd2c6: ref 0 aborting False
>Thread-15910660::INFO::2013-11-19 21:17:28,253::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000483989715576', 'lastCheck': 1384895839.010541, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00164389610291', 'lastCheck': 1384895841.0841999, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895846.242121, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0003981590271', 'lastCheck': 1384895845.6361229, 'valid': True, 'code': 0}}
>Dummy-15910548::DEBUG::2013-11-19 21:17:28,514::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:17:29,041::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0707649 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:17:31,057::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:17:31,591::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0711923 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:17:33,607::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:17:34,123::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0711648 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:17:36,138::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:17:36,368::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15910548::DEBUG::2013-11-19 21:17:36,667::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0703491 s, 14.6 MB/s\n'; <rc> = 0
>Thread-15910666::DEBUG::2013-11-19 21:17:36,769::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910666::INFO::2013-11-19 21:17:36,770::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910666::DEBUG::2013-11-19 21:17:36,771::task::495::TaskManager.Task::(_debug) Task 1c5429f7-bed2-44ee-bcb6-031022411ec4: moving from state init -> state preparing
>Thread-15910666::DEBUG::2013-11-19 21:17:36,772::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=278 spmId=2
>Thread-15910666::DEBUG::2013-11-19 21:17:36,772::task::495::TaskManager.Task::(_debug) Task 1c5429f7-bed2-44ee-bcb6-031022411ec4: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910666::DEBUG::2013-11-19 21:17:36,773::task::495::TaskManager.Task::(_debug) Task 1c5429f7-bed2-44ee-bcb6-031022411ec4: moving from state preparing -> state finished
>Thread-15910666::DEBUG::2013-11-19 21:17:36,774::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910666::DEBUG::2013-11-19 21:17:36,774::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910666::DEBUG::2013-11-19 21:17:36,775::task::495::TaskManager.Task::(_debug) Task 1c5429f7-bed2-44ee-bcb6-031022411ec4: ref 0 aborting False
>Thread-15910666::INFO::2013-11-19 21:17:36,775::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910667::DEBUG::2013-11-19 21:17:36,780::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910667::INFO::2013-11-19 21:17:36,781::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910667::DEBUG::2013-11-19 21:17:36,782::task::495::TaskManager.Task::(_debug) Task aa47e312-4ffa-4e0f-8c75-703a9f557fdc: moving from state init -> state preparing
>Thread-15910667::DEBUG::2013-11-19 21:17:36,784::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`086902f1-f9cf-41db-8df9-0520be01c4bc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910667::DEBUG::2013-11-19 21:17:36,785::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910667::DEBUG::2013-11-19 21:17:36,786::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910667::DEBUG::2013-11-19 21:17:36,787::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`086902f1-f9cf-41db-8df9-0520be01c4bc`::Granted request
>Thread-15910667::DEBUG::2013-11-19 21:17:36,797::task::495::TaskManager.Task::(_debug) Task aa47e312-4ffa-4e0f-8c75-703a9f557fdc: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910667::DEBUG::2013-11-19 21:17:36,798::task::495::TaskManager.Task::(_debug) Task aa47e312-4ffa-4e0f-8c75-703a9f557fdc: ref 1 aborting False
>Thread-15910667::WARNING::2013-11-19 21:17:36,801::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910667::DEBUG::2013-11-19 21:17:36,802::task::495::TaskManager.Task::(_debug) Task aa47e312-4ffa-4e0f-8c75-703a9f557fdc: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910667::DEBUG::2013-11-19 21:17:36,803::task::495::TaskManager.Task::(_debug) Task aa47e312-4ffa-4e0f-8c75-703a9f557fdc: moving from state preparing -> state finished
>Thread-15910667::DEBUG::2013-11-19 21:17:36,804::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910667::DEBUG::2013-11-19 21:17:36,804::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910667::DEBUG::2013-11-19 21:17:36,806::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910667::DEBUG::2013-11-19 21:17:36,806::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910667::DEBUG::2013-11-19 21:17:36,807::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910667::DEBUG::2013-11-19 21:17:36,808::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910667::DEBUG::2013-11-19 21:17:36,809::task::495::TaskManager.Task::(_debug) Task aa47e312-4ffa-4e0f-8c75-703a9f557fdc: ref 0 aborting False
>Thread-15910667::INFO::2013-11-19 21:17:36,810::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910668::INFO::2013-11-19 21:17:38,380::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910668::DEBUG::2013-11-19 21:17:38,381::task::495::TaskManager.Task::(_debug) Task 939593e5-bfef-48ea-8833-4aa4f808e03a: moving from state init -> state preparing
>Thread-15910668::DEBUG::2013-11-19 21:17:38,382::task::495::TaskManager.Task::(_debug) Task 939593e5-bfef-48ea-8833-4aa4f808e03a: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00218486785889', 'lastCheck': 1384895849.2401719, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00158596038818', 'lastCheck': 1384895851.2864039, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895856.3723309, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000452995300293', 'lastCheck': 1384895855.812453, 'valid': True, 'code': 0}}
>Thread-15910668::DEBUG::2013-11-19 21:17:38,382::task::495::TaskManager.Task::(_debug) Task 939593e5-bfef-48ea-8833-4aa4f808e03a: moving from state preparing -> state finished
>Thread-15910668::DEBUG::2013-11-19 21:17:38,383::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910668::DEBUG::2013-11-19 21:17:38,384::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910668::DEBUG::2013-11-19 21:17:38,384::task::495::TaskManager.Task::(_debug) Task 939593e5-bfef-48ea-8833-4aa4f808e03a: ref 0 aborting False
>Thread-15910668::INFO::2013-11-19 21:17:38,385::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00218486785889', 'lastCheck': 1384895849.2401719, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00158596038818', 'lastCheck': 1384895851.2864039, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895856.3723309, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000452995300293', 'lastCheck': 1384895855.812453, 'valid': True, 'code': 0}}
>Dummy-15910548::DEBUG::2013-11-19 21:17:38,682::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:17:39,216::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0712462 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:17:41,231::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:17:41,760::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0705107 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:17:43,776::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:17:44,340::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0940987 s, 10.9 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:17:46,355::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:17:46,586::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15910548::DEBUG::2013-11-19 21:17:46,887::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0775026 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15910674::DEBUG::2013-11-19 21:17:46,942::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910674::INFO::2013-11-19 21:17:46,943::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910674::DEBUG::2013-11-19 21:17:46,944::task::495::TaskManager.Task::(_debug) Task 0b827f33-9dd6-4334-ad80-8c5326f36efe: moving from state init -> state preparing
>Thread-15910674::DEBUG::2013-11-19 21:17:46,945::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=278 spmId=2
>Thread-15910674::DEBUG::2013-11-19 21:17:46,945::task::495::TaskManager.Task::(_debug) Task 0b827f33-9dd6-4334-ad80-8c5326f36efe: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910674::DEBUG::2013-11-19 21:17:46,946::task::495::TaskManager.Task::(_debug) Task 0b827f33-9dd6-4334-ad80-8c5326f36efe: moving from state preparing -> state finished
>Thread-15910674::DEBUG::2013-11-19 21:17:46,947::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910674::DEBUG::2013-11-19 21:17:46,947::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910674::DEBUG::2013-11-19 21:17:46,948::task::495::TaskManager.Task::(_debug) Task 0b827f33-9dd6-4334-ad80-8c5326f36efe: ref 0 aborting False
>Thread-15910674::INFO::2013-11-19 21:17:46,948::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910675::DEBUG::2013-11-19 21:17:46,953::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910675::INFO::2013-11-19 21:17:46,954::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910675::DEBUG::2013-11-19 21:17:46,955::task::495::TaskManager.Task::(_debug) Task 709fe604-d374-4163-8491-3270198aa6ba: moving from state init -> state preparing
>Thread-15910675::DEBUG::2013-11-19 21:17:46,956::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c4b4ac7c-7feb-4713-b494-12d9772f2c22`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910675::DEBUG::2013-11-19 21:17:46,956::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910675::DEBUG::2013-11-19 21:17:46,957::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910675::DEBUG::2013-11-19 21:17:46,958::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c4b4ac7c-7feb-4713-b494-12d9772f2c22`::Granted request
>Thread-15910675::DEBUG::2013-11-19 21:17:46,959::task::495::TaskManager.Task::(_debug) Task 709fe604-d374-4163-8491-3270198aa6ba: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910675::DEBUG::2013-11-19 21:17:46,959::task::495::TaskManager.Task::(_debug) Task 709fe604-d374-4163-8491-3270198aa6ba: ref 1 aborting False
>Thread-15910675::WARNING::2013-11-19 21:17:46,961::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910675::DEBUG::2013-11-19 21:17:46,962::task::495::TaskManager.Task::(_debug) Task 709fe604-d374-4163-8491-3270198aa6ba: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910675::DEBUG::2013-11-19 21:17:46,963::task::495::TaskManager.Task::(_debug) Task 709fe604-d374-4163-8491-3270198aa6ba: moving from state preparing -> state finished
>Thread-15910675::DEBUG::2013-11-19 21:17:46,964::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910675::DEBUG::2013-11-19 21:17:46,964::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910675::DEBUG::2013-11-19 21:17:46,965::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910675::DEBUG::2013-11-19 21:17:46,966::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910675::DEBUG::2013-11-19 21:17:46,966::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910675::DEBUG::2013-11-19 21:17:46,967::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910675::DEBUG::2013-11-19 21:17:46,967::task::495::TaskManager.Task::(_debug) Task 709fe604-d374-4163-8491-3270198aa6ba: ref 0 aborting False
>Thread-15910675::INFO::2013-11-19 21:17:46,968::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910676::INFO::2013-11-19 21:17:48,595::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910676::DEBUG::2013-11-19 21:17:48,596::task::495::TaskManager.Task::(_debug) Task 83574dfb-3557-463a-8353-590835da7f57: moving from state init -> state preparing
>Thread-15910676::DEBUG::2013-11-19 21:17:48,596::task::495::TaskManager.Task::(_debug) Task 83574dfb-3557-463a-8353-590835da7f57: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000460863113403', 'lastCheck': 1384895859.3681841, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00215697288513', 'lastCheck': 1384895861.485774, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895866.5877221, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000427007675171', 'lastCheck': 1384895866.014514, 'valid': True, 'code': 0}}
>Thread-15910676::DEBUG::2013-11-19 21:17:48,597::task::495::TaskManager.Task::(_debug) Task 83574dfb-3557-463a-8353-590835da7f57: moving from state preparing -> state finished
>Thread-15910676::DEBUG::2013-11-19 21:17:48,598::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910676::DEBUG::2013-11-19 21:17:48,598::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910676::DEBUG::2013-11-19 21:17:48,599::task::495::TaskManager.Task::(_debug) Task 83574dfb-3557-463a-8353-590835da7f57: ref 0 aborting False
>Thread-15910676::INFO::2013-11-19 21:17:48,599::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000460863113403', 'lastCheck': 1384895859.3681841, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00215697288513', 'lastCheck': 1384895861.485774, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895866.5877221, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000427007675171', 'lastCheck': 1384895866.014514, 'valid': True, 'code': 0}}
>Dummy-15910548::DEBUG::2013-11-19 21:17:48,902::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:17:49,443::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0713667 s, 14.3 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:17:51,459::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:17:51,981::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0707233 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:17:53,996::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:17:54,506::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0703108 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:17:56,521::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:17:56,757::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15910548::DEBUG::2013-11-19 21:17:57,061::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0713639 s, 14.3 MB/s\n'; <rc> = 0
>Thread-15910682::DEBUG::2013-11-19 21:17:57,109::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910682::INFO::2013-11-19 21:17:57,110::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910682::DEBUG::2013-11-19 21:17:57,111::task::495::TaskManager.Task::(_debug) Task 42079000-a12b-41fb-8ddc-7d8f10d37098: moving from state init -> state preparing
>Thread-15910682::DEBUG::2013-11-19 21:17:57,111::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=278 spmId=2
>Thread-15910682::DEBUG::2013-11-19 21:17:57,112::task::495::TaskManager.Task::(_debug) Task 42079000-a12b-41fb-8ddc-7d8f10d37098: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910682::DEBUG::2013-11-19 21:17:57,113::task::495::TaskManager.Task::(_debug) Task 42079000-a12b-41fb-8ddc-7d8f10d37098: moving from state preparing -> state finished
>Thread-15910682::DEBUG::2013-11-19 21:17:57,113::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910682::DEBUG::2013-11-19 21:17:57,114::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910682::DEBUG::2013-11-19 21:17:57,114::task::495::TaskManager.Task::(_debug) Task 42079000-a12b-41fb-8ddc-7d8f10d37098: ref 0 aborting False
>Thread-15910682::INFO::2013-11-19 21:17:57,115::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910683::DEBUG::2013-11-19 21:17:57,120::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910683::INFO::2013-11-19 21:17:57,121::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910683::DEBUG::2013-11-19 21:17:57,121::task::495::TaskManager.Task::(_debug) Task 614a71fb-db5c-4010-b4f3-9e53f4887dd6: moving from state init -> state preparing
>Thread-15910683::DEBUG::2013-11-19 21:17:57,123::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bdd6267a-26c9-4d0e-a91d-ed90d54ffe32`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910683::DEBUG::2013-11-19 21:17:57,123::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910683::DEBUG::2013-11-19 21:17:57,124::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910683::DEBUG::2013-11-19 21:17:57,125::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bdd6267a-26c9-4d0e-a91d-ed90d54ffe32`::Granted request
>Thread-15910683::DEBUG::2013-11-19 21:17:57,126::task::495::TaskManager.Task::(_debug) Task 614a71fb-db5c-4010-b4f3-9e53f4887dd6: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910683::DEBUG::2013-11-19 21:17:57,126::task::495::TaskManager.Task::(_debug) Task 614a71fb-db5c-4010-b4f3-9e53f4887dd6: ref 1 aborting False
>Thread-15910683::WARNING::2013-11-19 21:17:57,128::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910683::DEBUG::2013-11-19 21:17:57,129::task::495::TaskManager.Task::(_debug) Task 614a71fb-db5c-4010-b4f3-9e53f4887dd6: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910683::DEBUG::2013-11-19 21:17:57,130::task::495::TaskManager.Task::(_debug) Task 614a71fb-db5c-4010-b4f3-9e53f4887dd6: moving from state preparing -> state finished
>Thread-15910683::DEBUG::2013-11-19 21:17:57,131::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910683::DEBUG::2013-11-19 21:17:57,131::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910683::DEBUG::2013-11-19 21:17:57,132::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910683::DEBUG::2013-11-19 21:17:57,133::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910683::DEBUG::2013-11-19 21:17:57,133::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910683::DEBUG::2013-11-19 21:17:57,134::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910683::DEBUG::2013-11-19 21:17:57,134::task::495::TaskManager.Task::(_debug) Task 614a71fb-db5c-4010-b4f3-9e53f4887dd6: ref 0 aborting False
>Thread-15910683::INFO::2013-11-19 21:17:57,135::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910684::INFO::2013-11-19 21:17:58,771::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910684::DEBUG::2013-11-19 21:17:58,772::task::495::TaskManager.Task::(_debug) Task 3663cc8c-83c4-4e10-84bc-618744192e74: moving from state init -> state preparing
>Thread-15910684::DEBUG::2013-11-19 21:17:58,772::task::495::TaskManager.Task::(_debug) Task 3663cc8c-83c4-4e10-84bc-618744192e74: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000458002090454', 'lastCheck': 1384895869.588304, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00163602828979', 'lastCheck': 1384895871.6756339, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895876.7621729, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000884056091309', 'lastCheck': 1384895876.197823, 'valid': True, 'code': 0}}
>Thread-15910684::DEBUG::2013-11-19 21:17:58,773::task::495::TaskManager.Task::(_debug) Task 3663cc8c-83c4-4e10-84bc-618744192e74: moving from state preparing -> state finished
>Thread-15910684::DEBUG::2013-11-19 21:17:58,774::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910684::DEBUG::2013-11-19 21:17:58,774::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910684::DEBUG::2013-11-19 21:17:58,775::task::495::TaskManager.Task::(_debug) Task 3663cc8c-83c4-4e10-84bc-618744192e74: ref 0 aborting False
>Thread-15910684::INFO::2013-11-19 21:17:58,775::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000458002090454', 'lastCheck': 1384895869.588304, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00163602828979', 'lastCheck': 1384895871.6756339, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895876.7621729, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000884056091309', 'lastCheck': 1384895876.197823, 'valid': True, 'code': 0}}
>Dummy-15910548::DEBUG::2013-11-19 21:17:59,076::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:17:59,609::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0679936 s, 15.1 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:18:01,625::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:18:02,153::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0709541 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:18:04,168::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:18:04,707::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0730574 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:18:06,722::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:18:06,935::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15910548::DEBUG::2013-11-19 21:18:07,241::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0706304 s, 14.5 MB/s\n'; <rc> = 0
>Thread-15910690::DEBUG::2013-11-19 21:18:07,257::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910690::INFO::2013-11-19 21:18:07,258::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910690::DEBUG::2013-11-19 21:18:07,258::task::495::TaskManager.Task::(_debug) Task e781fdf5-a84b-4839-9694-ee541c43e071: moving from state init -> state preparing
>Thread-15910690::DEBUG::2013-11-19 21:18:07,259::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=278 spmId=2
>Thread-15910690::DEBUG::2013-11-19 21:18:07,260::task::495::TaskManager.Task::(_debug) Task e781fdf5-a84b-4839-9694-ee541c43e071: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910690::DEBUG::2013-11-19 21:18:07,260::task::495::TaskManager.Task::(_debug) Task e781fdf5-a84b-4839-9694-ee541c43e071: moving from state preparing -> state finished
>Thread-15910690::DEBUG::2013-11-19 21:18:07,261::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910690::DEBUG::2013-11-19 21:18:07,262::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910690::DEBUG::2013-11-19 21:18:07,262::task::495::TaskManager.Task::(_debug) Task e781fdf5-a84b-4839-9694-ee541c43e071: ref 0 aborting False
>Thread-15910690::INFO::2013-11-19 21:18:07,263::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910691::DEBUG::2013-11-19 21:18:07,268::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910691::INFO::2013-11-19 21:18:07,269::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910691::DEBUG::2013-11-19 21:18:07,269::task::495::TaskManager.Task::(_debug) Task 7796564e-4e24-4b13-9332-7c81d37d383b: moving from state init -> state preparing
>Thread-15910691::DEBUG::2013-11-19 21:18:07,270::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3dd5f24e-ad2e-4112-a3d8-38b23aa91224`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910691::DEBUG::2013-11-19 21:18:07,271::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910691::DEBUG::2013-11-19 21:18:07,272::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910691::DEBUG::2013-11-19 21:18:07,272::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3dd5f24e-ad2e-4112-a3d8-38b23aa91224`::Granted request
>Thread-15910691::DEBUG::2013-11-19 21:18:07,273::task::495::TaskManager.Task::(_debug) Task 7796564e-4e24-4b13-9332-7c81d37d383b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910691::DEBUG::2013-11-19 21:18:07,274::task::495::TaskManager.Task::(_debug) Task 7796564e-4e24-4b13-9332-7c81d37d383b: ref 1 aborting False
>Thread-15910691::WARNING::2013-11-19 21:18:07,276::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910691::DEBUG::2013-11-19 21:18:07,277::task::495::TaskManager.Task::(_debug) Task 7796564e-4e24-4b13-9332-7c81d37d383b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910691::DEBUG::2013-11-19 21:18:07,278::task::495::TaskManager.Task::(_debug) Task 7796564e-4e24-4b13-9332-7c81d37d383b: moving from state preparing -> state finished
>Thread-15910691::DEBUG::2013-11-19 21:18:07,278::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910691::DEBUG::2013-11-19 21:18:07,279::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910691::DEBUG::2013-11-19 21:18:07,279::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910691::DEBUG::2013-11-19 21:18:07,280::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910691::DEBUG::2013-11-19 21:18:07,281::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910691::DEBUG::2013-11-19 21:18:07,281::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910691::DEBUG::2013-11-19 21:18:07,282::task::495::TaskManager.Task::(_debug) Task 7796564e-4e24-4b13-9332-7c81d37d383b: ref 0 aborting False
>Thread-15910691::INFO::2013-11-19 21:18:07,282::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910692::INFO::2013-11-19 21:18:08,948::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910692::DEBUG::2013-11-19 21:18:08,949::task::495::TaskManager.Task::(_debug) Task bd7f6801-8e0e-4216-a220-732019880dde: moving from state init -> state preparing
>Thread-15910692::DEBUG::2013-11-19 21:18:08,949::task::495::TaskManager.Task::(_debug) Task bd7f6801-8e0e-4216-a220-732019880dde: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00040602684021', 'lastCheck': 1384895879.7578211, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00130486488342', 'lastCheck': 1384895881.8657351, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895886.9401901, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000515937805176', 'lastCheck': 1384895886.396148, 'valid': True, 'code': 0}}
>Thread-15910692::DEBUG::2013-11-19 21:18:08,950::task::495::TaskManager.Task::(_debug) Task bd7f6801-8e0e-4216-a220-732019880dde: moving from state preparing -> state finished
>Thread-15910692::DEBUG::2013-11-19 21:18:08,951::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910692::DEBUG::2013-11-19 21:18:08,951::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910692::DEBUG::2013-11-19 21:18:08,952::task::495::TaskManager.Task::(_debug) Task bd7f6801-8e0e-4216-a220-732019880dde: ref 0 aborting False
>Thread-15910692::INFO::2013-11-19 21:18:08,952::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00040602684021', 'lastCheck': 1384895879.7578211, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00130486488342', 'lastCheck': 1384895881.8657351, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895886.9401901, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000515937805176', 'lastCheck': 1384895886.396148, 'valid': True, 'code': 0}}
>Dummy-15910548::DEBUG::2013-11-19 21:18:09,257::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:18:09,764::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0705474 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:18:11,780::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:18:12,323::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0712999 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:18:14,338::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:18:14,825::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0450488 s, 22.7 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:18:16,836::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:18:17,090::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15910548::DEBUG::2013-11-19 21:18:17,399::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0974499 s, 10.5 MB/s\n'; <rc> = 0
>Thread-15910698::DEBUG::2013-11-19 21:18:17,416::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910698::INFO::2013-11-19 21:18:17,417::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910698::DEBUG::2013-11-19 21:18:17,418::task::495::TaskManager.Task::(_debug) Task 07ee5071-88f4-4f64-9a52-6a9be414a7b8: moving from state init -> state preparing
>Thread-15910698::DEBUG::2013-11-19 21:18:17,419::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=278 spmId=2
>Thread-15910698::DEBUG::2013-11-19 21:18:17,419::task::495::TaskManager.Task::(_debug) Task 07ee5071-88f4-4f64-9a52-6a9be414a7b8: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910698::DEBUG::2013-11-19 21:18:17,420::task::495::TaskManager.Task::(_debug) Task 07ee5071-88f4-4f64-9a52-6a9be414a7b8: moving from state preparing -> state finished
>Thread-15910698::DEBUG::2013-11-19 21:18:17,420::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910698::DEBUG::2013-11-19 21:18:17,421::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910698::DEBUG::2013-11-19 21:18:17,421::task::495::TaskManager.Task::(_debug) Task 07ee5071-88f4-4f64-9a52-6a9be414a7b8: ref 0 aborting False
>Thread-15910698::INFO::2013-11-19 21:18:17,422::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910699::DEBUG::2013-11-19 21:18:17,427::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910699::INFO::2013-11-19 21:18:17,428::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910699::DEBUG::2013-11-19 21:18:17,428::task::495::TaskManager.Task::(_debug) Task 0e0894ec-3222-4ed0-b9ef-913ba2a482a9: moving from state init -> state preparing
>Thread-15910699::DEBUG::2013-11-19 21:18:17,429::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`84745321-fbe7-40a7-af5d-cac3eb3a1257`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910699::DEBUG::2013-11-19 21:18:17,430::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910699::DEBUG::2013-11-19 21:18:17,431::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910699::DEBUG::2013-11-19 21:18:17,431::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`84745321-fbe7-40a7-af5d-cac3eb3a1257`::Granted request
>Thread-15910699::DEBUG::2013-11-19 21:18:17,432::task::495::TaskManager.Task::(_debug) Task 0e0894ec-3222-4ed0-b9ef-913ba2a482a9: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910699::DEBUG::2013-11-19 21:18:17,433::task::495::TaskManager.Task::(_debug) Task 0e0894ec-3222-4ed0-b9ef-913ba2a482a9: ref 1 aborting False
>Thread-15910699::WARNING::2013-11-19 21:18:17,435::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910699::DEBUG::2013-11-19 21:18:17,436::task::495::TaskManager.Task::(_debug) Task 0e0894ec-3222-4ed0-b9ef-913ba2a482a9: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910699::DEBUG::2013-11-19 21:18:17,437::task::495::TaskManager.Task::(_debug) Task 0e0894ec-3222-4ed0-b9ef-913ba2a482a9: moving from state preparing -> state finished
>Thread-15910699::DEBUG::2013-11-19 21:18:17,437::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910699::DEBUG::2013-11-19 21:18:17,438::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910699::DEBUG::2013-11-19 21:18:17,438::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910699::DEBUG::2013-11-19 21:18:17,439::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910699::DEBUG::2013-11-19 21:18:17,440::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910699::DEBUG::2013-11-19 21:18:17,440::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910699::DEBUG::2013-11-19 21:18:17,441::task::495::TaskManager.Task::(_debug) Task 0e0894ec-3222-4ed0-b9ef-913ba2a482a9: ref 0 aborting False
>Thread-15910699::INFO::2013-11-19 21:18:17,441::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910700::INFO::2013-11-19 21:18:19,102::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910700::DEBUG::2013-11-19 21:18:19,103::task::495::TaskManager.Task::(_debug) Task 2c053d05-60ca-4904-9a2f-af77bc04b288: moving from state init -> state preparing
>Thread-15910700::DEBUG::2013-11-19 21:18:19,103::task::495::TaskManager.Task::(_debug) Task 2c053d05-60ca-4904-9a2f-af77bc04b288: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000356197357178', 'lastCheck': 1384895889.9362769, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0015389919281', 'lastCheck': 1384895892.0160811, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895897.094624, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000440120697021', 'lastCheck': 1384895896.556711, 'valid': True, 'code': 0}}
>Thread-15910700::DEBUG::2013-11-19 21:18:19,104::task::495::TaskManager.Task::(_debug) Task 2c053d05-60ca-4904-9a2f-af77bc04b288: moving from state preparing -> state finished
>Thread-15910700::DEBUG::2013-11-19 21:18:19,105::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910700::DEBUG::2013-11-19 21:18:19,105::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910700::DEBUG::2013-11-19 21:18:19,106::task::495::TaskManager.Task::(_debug) Task 2c053d05-60ca-4904-9a2f-af77bc04b288: ref 0 aborting False
>Thread-15910700::INFO::2013-11-19 21:18:19,106::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000356197357178', 'lastCheck': 1384895889.9362769, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0015389919281', 'lastCheck': 1384895892.0160811, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895897.094624, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000440120697021', 'lastCheck': 1384895896.556711, 'valid': True, 'code': 0}}
>Dummy-15910548::DEBUG::2013-11-19 21:18:19,416::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:18:19,958::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0712801 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:18:21,973::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:18:22,455::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0439418 s, 23.3 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:18:24,466::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:18:24,986::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0711741 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:18:27,001::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:18:27,202::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15910548::DEBUG::2013-11-19 21:18:27,507::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0706307 s, 14.5 MB/s\n'; <rc> = 0
>Thread-15910706::DEBUG::2013-11-19 21:18:27,554::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910706::INFO::2013-11-19 21:18:27,556::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910706::DEBUG::2013-11-19 21:18:27,556::task::495::TaskManager.Task::(_debug) Task a313c662-be96-46a0-acff-b7142d79bbcc: moving from state init -> state preparing
>Thread-15910706::DEBUG::2013-11-19 21:18:27,557::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=278 spmId=2
>Thread-15910706::DEBUG::2013-11-19 21:18:27,558::task::495::TaskManager.Task::(_debug) Task a313c662-be96-46a0-acff-b7142d79bbcc: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910706::DEBUG::2013-11-19 21:18:27,558::task::495::TaskManager.Task::(_debug) Task a313c662-be96-46a0-acff-b7142d79bbcc: moving from state preparing -> state finished
>Thread-15910706::DEBUG::2013-11-19 21:18:27,559::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910706::DEBUG::2013-11-19 21:18:27,559::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910706::DEBUG::2013-11-19 21:18:27,560::task::495::TaskManager.Task::(_debug) Task a313c662-be96-46a0-acff-b7142d79bbcc: ref 0 aborting False
>Thread-15910706::INFO::2013-11-19 21:18:27,560::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910707::DEBUG::2013-11-19 21:18:27,565::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910707::INFO::2013-11-19 21:18:27,566::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910707::DEBUG::2013-11-19 21:18:27,567::task::495::TaskManager.Task::(_debug) Task a5d79a68-a879-478b-962e-32a55cae3345: moving from state init -> state preparing
>Thread-15910707::DEBUG::2013-11-19 21:18:27,568::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`309996ca-db33-4a1c-b31d-391ed3835c12`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910707::DEBUG::2013-11-19 21:18:27,569::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910707::DEBUG::2013-11-19 21:18:27,570::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910707::DEBUG::2013-11-19 21:18:27,570::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`309996ca-db33-4a1c-b31d-391ed3835c12`::Granted request
>Thread-15910707::DEBUG::2013-11-19 21:18:27,571::task::495::TaskManager.Task::(_debug) Task a5d79a68-a879-478b-962e-32a55cae3345: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910707::DEBUG::2013-11-19 21:18:27,572::task::495::TaskManager.Task::(_debug) Task a5d79a68-a879-478b-962e-32a55cae3345: ref 1 aborting False
>Thread-15910707::WARNING::2013-11-19 21:18:27,574::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910707::DEBUG::2013-11-19 21:18:27,575::task::495::TaskManager.Task::(_debug) Task a5d79a68-a879-478b-962e-32a55cae3345: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910707::DEBUG::2013-11-19 21:18:27,575::task::495::TaskManager.Task::(_debug) Task a5d79a68-a879-478b-962e-32a55cae3345: moving from state preparing -> state finished
>Thread-15910707::DEBUG::2013-11-19 21:18:27,576::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910707::DEBUG::2013-11-19 21:18:27,576::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910707::DEBUG::2013-11-19 21:18:27,577::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910707::DEBUG::2013-11-19 21:18:27,578::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910707::DEBUG::2013-11-19 21:18:27,578::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910707::DEBUG::2013-11-19 21:18:27,579::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910707::DEBUG::2013-11-19 21:18:27,579::task::495::TaskManager.Task::(_debug) Task a5d79a68-a879-478b-962e-32a55cae3345: ref 0 aborting False
>Thread-15910707::INFO::2013-11-19 21:18:27,580::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910708::INFO::2013-11-19 21:18:29,253::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910708::DEBUG::2013-11-19 21:18:29,254::task::495::TaskManager.Task::(_debug) Task 95c9af97-f729-43e6-a9c0-c55ab4861ba3: moving from state init -> state preparing
>Thread-15910708::DEBUG::2013-11-19 21:18:29,255::task::495::TaskManager.Task::(_debug) Task 95c9af97-f729-43e6-a9c0-c55ab4861ba3: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00062108039856', 'lastCheck': 1384895900.0783701, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00151419639587', 'lastCheck': 1384895902.1980031, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895907.2046001, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000526189804077', 'lastCheck': 1384895906.6760631, 'valid': True, 'code': 0}}
>Thread-15910708::DEBUG::2013-11-19 21:18:29,255::task::495::TaskManager.Task::(_debug) Task 95c9af97-f729-43e6-a9c0-c55ab4861ba3: moving from state preparing -> state finished
>Thread-15910708::DEBUG::2013-11-19 21:18:29,256::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910708::DEBUG::2013-11-19 21:18:29,256::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910708::DEBUG::2013-11-19 21:18:29,257::task::495::TaskManager.Task::(_debug) Task 95c9af97-f729-43e6-a9c0-c55ab4861ba3: ref 0 aborting False
>Thread-15910708::INFO::2013-11-19 21:18:29,258::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00062108039856', 'lastCheck': 1384895900.0783701, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00151419639587', 'lastCheck': 1384895902.1980031, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895907.2046001, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000526189804077', 'lastCheck': 1384895906.6760631, 'valid': True, 'code': 0}}
>Dummy-15910548::DEBUG::2013-11-19 21:18:29,523::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:18:30,050::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0705775 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:18:32,066::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:18:32,615::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0711785 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:18:34,630::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:18:35,144::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0743011 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:18:37,159::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:18:37,390::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910714::DEBUG::2013-11-19 21:18:37,695::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910714::INFO::2013-11-19 21:18:37,697::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910714::DEBUG::2013-11-19 21:18:37,704::task::495::TaskManager.Task::(_debug) Task a4a01bb9-dbc7-450e-b07e-0d2ae8e8931d: moving from state init -> state preparing
>Thread-15910714::DEBUG::2013-11-19 21:18:37,705::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=278 spmId=2
>Thread-15910714::DEBUG::2013-11-19 21:18:37,706::task::495::TaskManager.Task::(_debug) Task a4a01bb9-dbc7-450e-b07e-0d2ae8e8931d: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910714::DEBUG::2013-11-19 21:18:37,707::task::495::TaskManager.Task::(_debug) Task a4a01bb9-dbc7-450e-b07e-0d2ae8e8931d: moving from state preparing -> state finished
>Thread-15910714::DEBUG::2013-11-19 21:18:37,708::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910714::DEBUG::2013-11-19 21:18:37,709::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910714::DEBUG::2013-11-19 21:18:37,710::task::495::TaskManager.Task::(_debug) Task a4a01bb9-dbc7-450e-b07e-0d2ae8e8931d: ref 0 aborting False
>Thread-15910714::INFO::2013-11-19 21:18:37,711::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910715::DEBUG::2013-11-19 21:18:37,717::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910715::INFO::2013-11-19 21:18:37,718::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910715::DEBUG::2013-11-19 21:18:37,719::task::495::TaskManager.Task::(_debug) Task a36f92f0-29a9-4063-a1dd-09c328d79a92: moving from state init -> state preparing
>Thread-15910715::DEBUG::2013-11-19 21:18:37,720::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`04df802a-e0d6-4957-846f-47722aeace92`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910715::DEBUG::2013-11-19 21:18:37,721::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910715::DEBUG::2013-11-19 21:18:37,723::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910715::DEBUG::2013-11-19 21:18:37,723::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`04df802a-e0d6-4957-846f-47722aeace92`::Granted request
>Thread-15910715::DEBUG::2013-11-19 21:18:37,725::task::495::TaskManager.Task::(_debug) Task a36f92f0-29a9-4063-a1dd-09c328d79a92: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910715::DEBUG::2013-11-19 21:18:37,726::task::495::TaskManager.Task::(_debug) Task a36f92f0-29a9-4063-a1dd-09c328d79a92: ref 1 aborting False
>Thread-15910715::WARNING::2013-11-19 21:18:37,729::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910715::DEBUG::2013-11-19 21:18:37,730::task::495::TaskManager.Task::(_debug) Task a36f92f0-29a9-4063-a1dd-09c328d79a92: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910715::DEBUG::2013-11-19 21:18:37,732::task::495::TaskManager.Task::(_debug) Task a36f92f0-29a9-4063-a1dd-09c328d79a92: moving from state preparing -> state finished
>Thread-15910715::DEBUG::2013-11-19 21:18:37,733::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910715::DEBUG::2013-11-19 21:18:37,733::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910715::DEBUG::2013-11-19 21:18:37,734::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Dummy-15910548::DEBUG::2013-11-19 21:18:37,736::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0713171 s, 14.4 MB/s\n'; <rc> = 0
>Thread-15910715::DEBUG::2013-11-19 21:18:37,737::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910715::DEBUG::2013-11-19 21:18:37,738::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910715::DEBUG::2013-11-19 21:18:37,753::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910715::DEBUG::2013-11-19 21:18:37,754::task::495::TaskManager.Task::(_debug) Task a36f92f0-29a9-4063-a1dd-09c328d79a92: ref 0 aborting False
>Thread-15910715::INFO::2013-11-19 21:18:37,754::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910716::INFO::2013-11-19 21:18:39,408::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910716::DEBUG::2013-11-19 21:18:39,409::task::495::TaskManager.Task::(_debug) Task e2850f1b-695b-4267-a4b9-e9e5efe519fb: moving from state init -> state preparing
>Thread-15910716::DEBUG::2013-11-19 21:18:39,410::task::495::TaskManager.Task::(_debug) Task e2850f1b-695b-4267-a4b9-e9e5efe519fb: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000373840332031', 'lastCheck': 1384895910.205627, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014910697937', 'lastCheck': 1384895912.3529811, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895917.391942, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000524997711182', 'lastCheck': 1384895916.839061, 'valid': True, 'code': 0}}
>Thread-15910716::DEBUG::2013-11-19 21:18:39,410::task::495::TaskManager.Task::(_debug) Task e2850f1b-695b-4267-a4b9-e9e5efe519fb: moving from state preparing -> state finished
>Thread-15910716::DEBUG::2013-11-19 21:18:39,411::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910716::DEBUG::2013-11-19 21:18:39,411::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910716::DEBUG::2013-11-19 21:18:39,412::task::495::TaskManager.Task::(_debug) Task e2850f1b-695b-4267-a4b9-e9e5efe519fb: ref 0 aborting False
>Thread-15910716::INFO::2013-11-19 21:18:39,413::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000373840332031', 'lastCheck': 1384895910.205627, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014910697937', 'lastCheck': 1384895912.3529811, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895917.391942, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000524997711182', 'lastCheck': 1384895916.839061, 'valid': True, 'code': 0}}
>Dummy-15910548::DEBUG::2013-11-19 21:18:39,755::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:18:40,264::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0702919 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:18:42,279::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:18:42,820::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.071019 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:18:44,835::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:18:45,357::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0706501 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:18:47,373::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:18:47,578::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910722::DEBUG::2013-11-19 21:18:47,898::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910722::INFO::2013-11-19 21:18:47,900::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910722::DEBUG::2013-11-19 21:18:47,901::task::495::TaskManager.Task::(_debug) Task 67ae2a64-1fcd-44f1-80b8-398b8e5e622e: moving from state init -> state preparing
>Thread-15910722::DEBUG::2013-11-19 21:18:47,903::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=278 spmId=2
>Thread-15910722::DEBUG::2013-11-19 21:18:47,903::task::495::TaskManager.Task::(_debug) Task 67ae2a64-1fcd-44f1-80b8-398b8e5e622e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910722::DEBUG::2013-11-19 21:18:47,905::task::495::TaskManager.Task::(_debug) Task 67ae2a64-1fcd-44f1-80b8-398b8e5e622e: moving from state preparing -> state finished
>Thread-15910722::DEBUG::2013-11-19 21:18:47,906::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910722::DEBUG::2013-11-19 21:18:47,907::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910722::DEBUG::2013-11-19 21:18:47,908::task::495::TaskManager.Task::(_debug) Task 67ae2a64-1fcd-44f1-80b8-398b8e5e622e: ref 0 aborting False
>Thread-15910722::INFO::2013-11-19 21:18:47,910::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910723::DEBUG::2013-11-19 21:18:47,917::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910723::INFO::2013-11-19 21:18:47,918::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910723::DEBUG::2013-11-19 21:18:47,919::task::495::TaskManager.Task::(_debug) Task 6ed31f6e-e688-4224-b6fa-9bc0e2010f32: moving from state init -> state preparing
>Thread-15910723::DEBUG::2013-11-19 21:18:47,921::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9425fe80-20d0-408a-a58b-577aaab43dad`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910723::DEBUG::2013-11-19 21:18:47,922::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910723::DEBUG::2013-11-19 21:18:47,924::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910723::DEBUG::2013-11-19 21:18:47,925::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9425fe80-20d0-408a-a58b-577aaab43dad`::Granted request
>Thread-15910723::DEBUG::2013-11-19 21:18:47,926::task::495::TaskManager.Task::(_debug) Task 6ed31f6e-e688-4224-b6fa-9bc0e2010f32: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910723::DEBUG::2013-11-19 21:18:47,927::task::495::TaskManager.Task::(_debug) Task 6ed31f6e-e688-4224-b6fa-9bc0e2010f32: ref 1 aborting False
>Thread-15910723::WARNING::2013-11-19 21:18:47,930::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910723::DEBUG::2013-11-19 21:18:47,931::task::495::TaskManager.Task::(_debug) Task 6ed31f6e-e688-4224-b6fa-9bc0e2010f32: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910723::DEBUG::2013-11-19 21:18:47,932::task::495::TaskManager.Task::(_debug) Task 6ed31f6e-e688-4224-b6fa-9bc0e2010f32: moving from state preparing -> state finished
>Thread-15910723::DEBUG::2013-11-19 21:18:47,933::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910723::DEBUG::2013-11-19 21:18:47,934::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910723::DEBUG::2013-11-19 21:18:47,935::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910723::DEBUG::2013-11-19 21:18:47,936::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910723::DEBUG::2013-11-19 21:18:47,937::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910723::DEBUG::2013-11-19 21:18:47,938::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910723::DEBUG::2013-11-19 21:18:47,939::task::495::TaskManager.Task::(_debug) Task 6ed31f6e-e688-4224-b6fa-9bc0e2010f32: ref 0 aborting False
>Thread-15910723::INFO::2013-11-19 21:18:47,940::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15910548::DEBUG::2013-11-19 21:18:47,945::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0933358 s, 11.0 MB/s\n'; <rc> = 0
>Thread-15910724::INFO::2013-11-19 21:18:49,590::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910724::DEBUG::2013-11-19 21:18:49,591::task::495::TaskManager.Task::(_debug) Task 1554ab68-db5c-4120-a6cf-91c905e7e359: moving from state init -> state preparing
>Thread-15910724::DEBUG::2013-11-19 21:18:49,592::task::495::TaskManager.Task::(_debug) Task 1554ab68-db5c-4120-a6cf-91c905e7e359: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000409126281738', 'lastCheck': 1384895920.3937111, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00245785713196', 'lastCheck': 1384895922.515836, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895927.5822909, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000391006469727', 'lastCheck': 1384895927.046561, 'valid': True, 'code': 0}}
>Thread-15910724::DEBUG::2013-11-19 21:18:49,592::task::495::TaskManager.Task::(_debug) Task 1554ab68-db5c-4120-a6cf-91c905e7e359: moving from state preparing -> state finished
>Thread-15910724::DEBUG::2013-11-19 21:18:49,593::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910724::DEBUG::2013-11-19 21:18:49,593::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910724::DEBUG::2013-11-19 21:18:49,594::task::495::TaskManager.Task::(_debug) Task 1554ab68-db5c-4120-a6cf-91c905e7e359: ref 0 aborting False
>Thread-15910724::INFO::2013-11-19 21:18:49,595::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000409126281738', 'lastCheck': 1384895920.3937111, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00245785713196', 'lastCheck': 1384895922.515836, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895927.5822909, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000391006469727', 'lastCheck': 1384895927.046561, 'valid': True, 'code': 0}}
>Dummy-15910548::DEBUG::2013-11-19 21:18:49,959::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:18:50,490::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0712066 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:18:52,505::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:18:53,016::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0712308 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:18:55,031::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:18:55,564::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0712034 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:18:57,579::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:18:57,794::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15910548::DEBUG::2013-11-19 21:18:58,103::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0709988 s, 14.4 MB/s\n'; <rc> = 0
>Thread-15910730::DEBUG::2013-11-19 21:18:58,160::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910730::INFO::2013-11-19 21:18:58,162::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910730::DEBUG::2013-11-19 21:18:58,163::task::495::TaskManager.Task::(_debug) Task d4c3d198-e642-4045-b16f-2f2bf40f6901: moving from state init -> state preparing
>Thread-15910730::DEBUG::2013-11-19 21:18:58,163::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=278 spmId=2
>Thread-15910730::DEBUG::2013-11-19 21:18:58,164::task::495::TaskManager.Task::(_debug) Task d4c3d198-e642-4045-b16f-2f2bf40f6901: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910730::DEBUG::2013-11-19 21:18:58,165::task::495::TaskManager.Task::(_debug) Task d4c3d198-e642-4045-b16f-2f2bf40f6901: moving from state preparing -> state finished
>Thread-15910730::DEBUG::2013-11-19 21:18:58,165::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910730::DEBUG::2013-11-19 21:18:58,166::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910730::DEBUG::2013-11-19 21:18:58,166::task::495::TaskManager.Task::(_debug) Task d4c3d198-e642-4045-b16f-2f2bf40f6901: ref 0 aborting False
>Thread-15910730::INFO::2013-11-19 21:18:58,167::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910731::DEBUG::2013-11-19 21:18:58,172::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910731::INFO::2013-11-19 21:18:58,173::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910731::DEBUG::2013-11-19 21:18:58,173::task::495::TaskManager.Task::(_debug) Task c20cb585-742a-49b4-9c59-176f2d61c983: moving from state init -> state preparing
>Thread-15910731::DEBUG::2013-11-19 21:18:58,175::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`fb91a55c-40ea-4632-b71c-c04586ddaa32`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910731::DEBUG::2013-11-19 21:18:58,175::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910731::DEBUG::2013-11-19 21:18:58,176::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910731::DEBUG::2013-11-19 21:18:58,177::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`fb91a55c-40ea-4632-b71c-c04586ddaa32`::Granted request
>Thread-15910731::DEBUG::2013-11-19 21:18:58,178::task::495::TaskManager.Task::(_debug) Task c20cb585-742a-49b4-9c59-176f2d61c983: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910731::DEBUG::2013-11-19 21:18:58,178::task::495::TaskManager.Task::(_debug) Task c20cb585-742a-49b4-9c59-176f2d61c983: ref 1 aborting False
>Thread-15910731::WARNING::2013-11-19 21:18:58,180::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910731::DEBUG::2013-11-19 21:18:58,181::task::495::TaskManager.Task::(_debug) Task c20cb585-742a-49b4-9c59-176f2d61c983: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910731::DEBUG::2013-11-19 21:18:58,182::task::495::TaskManager.Task::(_debug) Task c20cb585-742a-49b4-9c59-176f2d61c983: moving from state preparing -> state finished
>Thread-15910731::DEBUG::2013-11-19 21:18:58,182::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910731::DEBUG::2013-11-19 21:18:58,183::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910731::DEBUG::2013-11-19 21:18:58,184::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910731::DEBUG::2013-11-19 21:18:58,184::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910731::DEBUG::2013-11-19 21:18:58,185::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910731::DEBUG::2013-11-19 21:18:58,185::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910731::DEBUG::2013-11-19 21:18:58,186::task::495::TaskManager.Task::(_debug) Task c20cb585-742a-49b4-9c59-176f2d61c983: ref 0 aborting False
>Thread-15910731::INFO::2013-11-19 21:18:58,187::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910732::INFO::2013-11-19 21:18:59,806::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910732::DEBUG::2013-11-19 21:18:59,807::task::495::TaskManager.Task::(_debug) Task dea2a88f-9b9e-4649-99f4-39feda83c740: moving from state init -> state preparing
>Thread-15910732::DEBUG::2013-11-19 21:18:59,808::task::495::TaskManager.Task::(_debug) Task dea2a88f-9b9e-4649-99f4-39feda83c740: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00049090385437', 'lastCheck': 1384895930.580384, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00163197517395', 'lastCheck': 1384895932.709038, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895937.795506, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000383853912354', 'lastCheck': 1384895937.254693, 'valid': True, 'code': 0}}
>Thread-15910732::DEBUG::2013-11-19 21:18:59,808::task::495::TaskManager.Task::(_debug) Task dea2a88f-9b9e-4649-99f4-39feda83c740: moving from state preparing -> state finished
>Thread-15910732::DEBUG::2013-11-19 21:18:59,809::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910732::DEBUG::2013-11-19 21:18:59,809::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910732::DEBUG::2013-11-19 21:18:59,810::task::495::TaskManager.Task::(_debug) Task dea2a88f-9b9e-4649-99f4-39feda83c740: ref 0 aborting False
>Thread-15910732::INFO::2013-11-19 21:18:59,811::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00049090385437', 'lastCheck': 1384895930.580384, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00163197517395', 'lastCheck': 1384895932.709038, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895937.795506, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000383853912354', 'lastCheck': 1384895937.254693, 'valid': True, 'code': 0}}
>Dummy-15910548::DEBUG::2013-11-19 21:19:00,119::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:19:00,634::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0700107 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:19:02,649::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:19:03,169::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0703466 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:19:05,184::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:19:05,725::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0710145 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:19:07,740::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:19:07,971::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15910548::DEBUG::2013-11-19 21:19:08,264::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0740591 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15910738::DEBUG::2013-11-19 21:19:08,318::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910738::INFO::2013-11-19 21:19:08,319::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910738::DEBUG::2013-11-19 21:19:08,320::task::495::TaskManager.Task::(_debug) Task 02e9448b-65e3-4ebb-ac5e-fdbc1013cf04: moving from state init -> state preparing
>Thread-15910738::DEBUG::2013-11-19 21:19:08,321::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=278 spmId=2
>Thread-15910738::DEBUG::2013-11-19 21:19:08,321::task::495::TaskManager.Task::(_debug) Task 02e9448b-65e3-4ebb-ac5e-fdbc1013cf04: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910738::DEBUG::2013-11-19 21:19:08,322::task::495::TaskManager.Task::(_debug) Task 02e9448b-65e3-4ebb-ac5e-fdbc1013cf04: moving from state preparing -> state finished
>Thread-15910738::DEBUG::2013-11-19 21:19:08,322::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910738::DEBUG::2013-11-19 21:19:08,323::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910738::DEBUG::2013-11-19 21:19:08,324::task::495::TaskManager.Task::(_debug) Task 02e9448b-65e3-4ebb-ac5e-fdbc1013cf04: ref 0 aborting False
>Thread-15910738::INFO::2013-11-19 21:19:08,324::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910739::DEBUG::2013-11-19 21:19:08,329::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910739::INFO::2013-11-19 21:19:08,330::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910739::DEBUG::2013-11-19 21:19:08,330::task::495::TaskManager.Task::(_debug) Task a3d74178-47b0-4a37-8644-b88b7dc47ed3: moving from state init -> state preparing
>Thread-15910739::DEBUG::2013-11-19 21:19:08,332::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6fb20b9c-6572-4438-93c3-5bed9237aecd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910739::DEBUG::2013-11-19 21:19:08,332::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910739::DEBUG::2013-11-19 21:19:08,333::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910739::DEBUG::2013-11-19 21:19:08,333::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6fb20b9c-6572-4438-93c3-5bed9237aecd`::Granted request
>Thread-15910739::DEBUG::2013-11-19 21:19:08,334::task::495::TaskManager.Task::(_debug) Task a3d74178-47b0-4a37-8644-b88b7dc47ed3: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910739::DEBUG::2013-11-19 21:19:08,335::task::495::TaskManager.Task::(_debug) Task a3d74178-47b0-4a37-8644-b88b7dc47ed3: ref 1 aborting False
>Thread-15910739::WARNING::2013-11-19 21:19:08,337::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910739::DEBUG::2013-11-19 21:19:08,338::task::495::TaskManager.Task::(_debug) Task a3d74178-47b0-4a37-8644-b88b7dc47ed3: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910739::DEBUG::2013-11-19 21:19:08,339::task::495::TaskManager.Task::(_debug) Task a3d74178-47b0-4a37-8644-b88b7dc47ed3: moving from state preparing -> state finished
>Thread-15910739::DEBUG::2013-11-19 21:19:08,339::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910739::DEBUG::2013-11-19 21:19:08,340::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910739::DEBUG::2013-11-19 21:19:08,341::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910739::DEBUG::2013-11-19 21:19:08,341::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910739::DEBUG::2013-11-19 21:19:08,342::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910739::DEBUG::2013-11-19 21:19:08,342::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910739::DEBUG::2013-11-19 21:19:08,343::task::495::TaskManager.Task::(_debug) Task a3d74178-47b0-4a37-8644-b88b7dc47ed3: ref 0 aborting False
>Thread-15910739::INFO::2013-11-19 21:19:08,343::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910740::INFO::2013-11-19 21:19:09,983::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910740::DEBUG::2013-11-19 21:19:09,984::task::495::TaskManager.Task::(_debug) Task 3c618fc0-171b-4bce-8641-0a02949f98b1: moving from state init -> state preparing
>Thread-15910740::DEBUG::2013-11-19 21:19:09,984::task::495::TaskManager.Task::(_debug) Task 3c618fc0-171b-4bce-8641-0a02949f98b1: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000504016876221', 'lastCheck': 1384895940.7969739, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00129890441895', 'lastCheck': 1384895942.9170351, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895947.975529, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000461101531982', 'lastCheck': 1384895947.4119179, 'valid': True, 'code': 0}}
>Thread-15910740::DEBUG::2013-11-19 21:19:09,985::task::495::TaskManager.Task::(_debug) Task 3c618fc0-171b-4bce-8641-0a02949f98b1: moving from state preparing -> state finished
>Thread-15910740::DEBUG::2013-11-19 21:19:09,986::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910740::DEBUG::2013-11-19 21:19:09,986::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910740::DEBUG::2013-11-19 21:19:09,987::task::495::TaskManager.Task::(_debug) Task 3c618fc0-171b-4bce-8641-0a02949f98b1: ref 0 aborting False
>Thread-15910740::INFO::2013-11-19 21:19:09,988::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000504016876221', 'lastCheck': 1384895940.7969739, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00129890441895', 'lastCheck': 1384895942.9170351, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895947.975529, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000461101531982', 'lastCheck': 1384895947.4119179, 'valid': True, 'code': 0}}
>Dummy-15910548::DEBUG::2013-11-19 21:19:10,279::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:19:10,819::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0709502 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:19:12,834::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:19:13,355::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0705378 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:19:15,370::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:19:15,880::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0700522 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:19:17,894::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:19:18,158::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15910548::DEBUG::2013-11-19 21:19:18,418::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0701327 s, 14.6 MB/s\n'; <rc> = 0
>Thread-15910746::DEBUG::2013-11-19 21:19:18,507::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910746::INFO::2013-11-19 21:19:18,508::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910746::DEBUG::2013-11-19 21:19:18,508::task::495::TaskManager.Task::(_debug) Task d246b964-66af-4aba-8e00-224d546835f6: moving from state init -> state preparing
>Thread-15910746::DEBUG::2013-11-19 21:19:18,509::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=278 spmId=2
>Thread-15910746::DEBUG::2013-11-19 21:19:18,510::task::495::TaskManager.Task::(_debug) Task d246b964-66af-4aba-8e00-224d546835f6: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910746::DEBUG::2013-11-19 21:19:18,511::task::495::TaskManager.Task::(_debug) Task d246b964-66af-4aba-8e00-224d546835f6: moving from state preparing -> state finished
>Thread-15910746::DEBUG::2013-11-19 21:19:18,511::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910746::DEBUG::2013-11-19 21:19:18,512::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910746::DEBUG::2013-11-19 21:19:18,512::task::495::TaskManager.Task::(_debug) Task d246b964-66af-4aba-8e00-224d546835f6: ref 0 aborting False
>Thread-15910746::INFO::2013-11-19 21:19:18,513::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910747::DEBUG::2013-11-19 21:19:18,518::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910747::INFO::2013-11-19 21:19:18,519::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910747::DEBUG::2013-11-19 21:19:18,519::task::495::TaskManager.Task::(_debug) Task 5170d1f2-e27c-4f39-ad28-381dad3185ac: moving from state init -> state preparing
>Thread-15910747::DEBUG::2013-11-19 21:19:18,520::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bf3ac7cd-6b60-4e87-84b5-193f5f116c65`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910747::DEBUG::2013-11-19 21:19:18,521::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910747::DEBUG::2013-11-19 21:19:18,522::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910747::DEBUG::2013-11-19 21:19:18,522::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bf3ac7cd-6b60-4e87-84b5-193f5f116c65`::Granted request
>Thread-15910747::DEBUG::2013-11-19 21:19:18,523::task::495::TaskManager.Task::(_debug) Task 5170d1f2-e27c-4f39-ad28-381dad3185ac: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910747::DEBUG::2013-11-19 21:19:18,524::task::495::TaskManager.Task::(_debug) Task 5170d1f2-e27c-4f39-ad28-381dad3185ac: ref 1 aborting False
>Thread-15910747::WARNING::2013-11-19 21:19:18,526::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910747::DEBUG::2013-11-19 21:19:18,527::task::495::TaskManager.Task::(_debug) Task 5170d1f2-e27c-4f39-ad28-381dad3185ac: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910747::DEBUG::2013-11-19 21:19:18,527::task::495::TaskManager.Task::(_debug) Task 5170d1f2-e27c-4f39-ad28-381dad3185ac: moving from state preparing -> state finished
>Thread-15910747::DEBUG::2013-11-19 21:19:18,528::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910747::DEBUG::2013-11-19 21:19:18,529::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910747::DEBUG::2013-11-19 21:19:18,530::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910747::DEBUG::2013-11-19 21:19:18,531::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910747::DEBUG::2013-11-19 21:19:18,532::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910747::DEBUG::2013-11-19 21:19:18,532::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910747::DEBUG::2013-11-19 21:19:18,533::task::495::TaskManager.Task::(_debug) Task 5170d1f2-e27c-4f39-ad28-381dad3185ac: ref 0 aborting False
>Thread-15910747::INFO::2013-11-19 21:19:18,534::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910748::INFO::2013-11-19 21:19:20,169::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910748::DEBUG::2013-11-19 21:19:20,170::task::495::TaskManager.Task::(_debug) Task 6df45cec-0074-43fa-957c-6a20851e89ef: moving from state init -> state preparing
>Thread-15910748::DEBUG::2013-11-19 21:19:20,171::task::495::TaskManager.Task::(_debug) Task 6df45cec-0074-43fa-957c-6a20851e89ef: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000424861907959', 'lastCheck': 1384895950.9653339, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00172710418701', 'lastCheck': 1384895953.050432, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895958.1620331, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000381946563721', 'lastCheck': 1384895957.5719521, 'valid': True, 'code': 0}}
>Thread-15910748::DEBUG::2013-11-19 21:19:20,171::task::495::TaskManager.Task::(_debug) Task 6df45cec-0074-43fa-957c-6a20851e89ef: moving from state preparing -> state finished
>Thread-15910748::DEBUG::2013-11-19 21:19:20,172::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910748::DEBUG::2013-11-19 21:19:20,173::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910748::DEBUG::2013-11-19 21:19:20,173::task::495::TaskManager.Task::(_debug) Task 6df45cec-0074-43fa-957c-6a20851e89ef: ref 0 aborting False
>Thread-15910748::INFO::2013-11-19 21:19:20,174::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000424861907959', 'lastCheck': 1384895950.9653339, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00172710418701', 'lastCheck': 1384895953.050432, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895958.1620331, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000381946563721', 'lastCheck': 1384895957.5719521, 'valid': True, 'code': 0}}
>Dummy-15910548::DEBUG::2013-11-19 21:19:20,434::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:19:20,982::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.077425 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:19:22,997::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:19:23,521::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0807402 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:19:25,537::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:19:26,082::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0842655 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:19:28,098::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:19:28,310::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15910548::DEBUG::2013-11-19 21:19:28,615::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0712861 s, 14.4 MB/s\n'; <rc> = 0
>Thread-15910754::DEBUG::2013-11-19 21:19:28,683::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910754::INFO::2013-11-19 21:19:28,685::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910754::DEBUG::2013-11-19 21:19:28,685::task::495::TaskManager.Task::(_debug) Task 2583795b-a0e7-4f8b-8931-eb96cece773c: moving from state init -> state preparing
>Thread-15910754::DEBUG::2013-11-19 21:19:28,686::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=278 spmId=2
>Thread-15910754::DEBUG::2013-11-19 21:19:28,687::task::495::TaskManager.Task::(_debug) Task 2583795b-a0e7-4f8b-8931-eb96cece773c: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910754::DEBUG::2013-11-19 21:19:28,687::task::495::TaskManager.Task::(_debug) Task 2583795b-a0e7-4f8b-8931-eb96cece773c: moving from state preparing -> state finished
>Thread-15910754::DEBUG::2013-11-19 21:19:28,688::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910754::DEBUG::2013-11-19 21:19:28,688::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910754::DEBUG::2013-11-19 21:19:28,689::task::495::TaskManager.Task::(_debug) Task 2583795b-a0e7-4f8b-8931-eb96cece773c: ref 0 aborting False
>Thread-15910754::INFO::2013-11-19 21:19:28,689::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910755::DEBUG::2013-11-19 21:19:28,694::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910755::INFO::2013-11-19 21:19:28,695::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910755::DEBUG::2013-11-19 21:19:28,696::task::495::TaskManager.Task::(_debug) Task e9d3655d-d8ad-41a9-b77f-554543e3d9e3: moving from state init -> state preparing
>Thread-15910755::DEBUG::2013-11-19 21:19:28,697::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0b45ba67-5968-4c36-a059-375051e8ef82`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910755::DEBUG::2013-11-19 21:19:28,698::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910755::DEBUG::2013-11-19 21:19:28,698::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910755::DEBUG::2013-11-19 21:19:28,699::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0b45ba67-5968-4c36-a059-375051e8ef82`::Granted request
>Thread-15910755::DEBUG::2013-11-19 21:19:28,700::task::495::TaskManager.Task::(_debug) Task e9d3655d-d8ad-41a9-b77f-554543e3d9e3: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910755::DEBUG::2013-11-19 21:19:28,700::task::495::TaskManager.Task::(_debug) Task e9d3655d-d8ad-41a9-b77f-554543e3d9e3: ref 1 aborting False
>Thread-15910755::WARNING::2013-11-19 21:19:28,702::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910755::DEBUG::2013-11-19 21:19:28,703::task::495::TaskManager.Task::(_debug) Task e9d3655d-d8ad-41a9-b77f-554543e3d9e3: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910755::DEBUG::2013-11-19 21:19:28,704::task::495::TaskManager.Task::(_debug) Task e9d3655d-d8ad-41a9-b77f-554543e3d9e3: moving from state preparing -> state finished
>Thread-15910755::DEBUG::2013-11-19 21:19:28,705::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910755::DEBUG::2013-11-19 21:19:28,705::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910755::DEBUG::2013-11-19 21:19:28,706::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910755::DEBUG::2013-11-19 21:19:28,707::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910755::DEBUG::2013-11-19 21:19:28,707::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910755::DEBUG::2013-11-19 21:19:28,708::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910755::DEBUG::2013-11-19 21:19:28,708::task::495::TaskManager.Task::(_debug) Task e9d3655d-d8ad-41a9-b77f-554543e3d9e3: ref 0 aborting False
>Thread-15910755::INFO::2013-11-19 21:19:28,709::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910756::INFO::2013-11-19 21:19:30,322::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910756::DEBUG::2013-11-19 21:19:30,323::task::495::TaskManager.Task::(_debug) Task 1fe0a430-b741-49e8-af77-625c59295a88: moving from state init -> state preparing
>Thread-15910756::DEBUG::2013-11-19 21:19:30,323::task::495::TaskManager.Task::(_debug) Task 1fe0a430-b741-49e8-af77-625c59295a88: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00119400024414', 'lastCheck': 1384895961.1499009, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00159883499146', 'lastCheck': 1384895963.214612, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895968.314203, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000491142272949', 'lastCheck': 1384895967.76038, 'valid': True, 'code': 0}}
>Thread-15910756::DEBUG::2013-11-19 21:19:30,324::task::495::TaskManager.Task::(_debug) Task 1fe0a430-b741-49e8-af77-625c59295a88: moving from state preparing -> state finished
>Thread-15910756::DEBUG::2013-11-19 21:19:30,325::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910756::DEBUG::2013-11-19 21:19:30,325::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910756::DEBUG::2013-11-19 21:19:30,326::task::495::TaskManager.Task::(_debug) Task 1fe0a430-b741-49e8-af77-625c59295a88: ref 0 aborting False
>Thread-15910756::INFO::2013-11-19 21:19:30,326::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00119400024414', 'lastCheck': 1384895961.1499009, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00159883499146', 'lastCheck': 1384895963.214612, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895968.314203, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000491142272949', 'lastCheck': 1384895967.76038, 'valid': True, 'code': 0}}
>Dummy-15910548::DEBUG::2013-11-19 21:19:30,630::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:19:31,138::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0706305 s, 14.5 MB/s\n'; <rc> = 0
>Thread-42838::DEBUG::2013-11-19 21:19:31,307::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 21:19:31,995::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:19:33,154::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:19:33,690::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0710608 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:19:35,705::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:19:36,262::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0883131 s, 11.6 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:19:38,277::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:19:38,535::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15910548::DEBUG::2013-11-19 21:19:38,829::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0711798 s, 14.4 MB/s\n'; <rc> = 0
>Thread-15910762::DEBUG::2013-11-19 21:19:38,846::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910762::INFO::2013-11-19 21:19:38,848::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910762::DEBUG::2013-11-19 21:19:38,849::task::495::TaskManager.Task::(_debug) Task efa6b18a-79a2-4406-a58e-c4d1c9125668: moving from state init -> state preparing
>Thread-15910762::DEBUG::2013-11-19 21:19:38,850::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=278 spmId=2
>Thread-15910762::DEBUG::2013-11-19 21:19:38,850::task::495::TaskManager.Task::(_debug) Task efa6b18a-79a2-4406-a58e-c4d1c9125668: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910762::DEBUG::2013-11-19 21:19:38,851::task::495::TaskManager.Task::(_debug) Task efa6b18a-79a2-4406-a58e-c4d1c9125668: moving from state preparing -> state finished
>Thread-15910762::DEBUG::2013-11-19 21:19:38,852::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910762::DEBUG::2013-11-19 21:19:38,852::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910762::DEBUG::2013-11-19 21:19:38,853::task::495::TaskManager.Task::(_debug) Task efa6b18a-79a2-4406-a58e-c4d1c9125668: ref 0 aborting False
>Thread-15910762::INFO::2013-11-19 21:19:38,854::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910763::DEBUG::2013-11-19 21:19:38,860::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910763::INFO::2013-11-19 21:19:38,861::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910763::DEBUG::2013-11-19 21:19:38,861::task::495::TaskManager.Task::(_debug) Task 9ea96967-358f-43df-9ad9-a7e214b7f406: moving from state init -> state preparing
>Thread-15910763::DEBUG::2013-11-19 21:19:38,862::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`032d2894-05d1-4c88-b4e2-9d4ec2524768`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910763::DEBUG::2013-11-19 21:19:38,863::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910763::DEBUG::2013-11-19 21:19:38,864::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910763::DEBUG::2013-11-19 21:19:38,864::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`032d2894-05d1-4c88-b4e2-9d4ec2524768`::Granted request
>Thread-15910763::DEBUG::2013-11-19 21:19:38,865::task::495::TaskManager.Task::(_debug) Task 9ea96967-358f-43df-9ad9-a7e214b7f406: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910763::DEBUG::2013-11-19 21:19:38,866::task::495::TaskManager.Task::(_debug) Task 9ea96967-358f-43df-9ad9-a7e214b7f406: ref 1 aborting False
>Thread-15910763::WARNING::2013-11-19 21:19:38,868::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910763::DEBUG::2013-11-19 21:19:38,869::task::495::TaskManager.Task::(_debug) Task 9ea96967-358f-43df-9ad9-a7e214b7f406: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910763::DEBUG::2013-11-19 21:19:38,869::task::495::TaskManager.Task::(_debug) Task 9ea96967-358f-43df-9ad9-a7e214b7f406: moving from state preparing -> state finished
>Thread-15910763::DEBUG::2013-11-19 21:19:38,870::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910763::DEBUG::2013-11-19 21:19:38,870::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910763::DEBUG::2013-11-19 21:19:38,871::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910763::DEBUG::2013-11-19 21:19:38,872::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910763::DEBUG::2013-11-19 21:19:38,872::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910763::DEBUG::2013-11-19 21:19:38,873::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910763::DEBUG::2013-11-19 21:19:38,873::task::495::TaskManager.Task::(_debug) Task 9ea96967-358f-43df-9ad9-a7e214b7f406: ref 0 aborting False
>Thread-15910763::INFO::2013-11-19 21:19:38,874::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910764::INFO::2013-11-19 21:19:40,539::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910764::DEBUG::2013-11-19 21:19:40,540::task::495::TaskManager.Task::(_debug) Task 135fabaa-9efd-4650-b825-23a7508179c7: moving from state init -> state preparing
>Thread-15910764::DEBUG::2013-11-19 21:19:40,541::task::495::TaskManager.Task::(_debug) Task 135fabaa-9efd-4650-b825-23a7508179c7: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000313997268677', 'lastCheck': 1384895971.9998729, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00165104866028', 'lastCheck': 1384895973.3947361, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895978.5368869, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000418901443481', 'lastCheck': 1384895977.9438889, 'valid': True, 'code': 0}}
>Thread-15910764::DEBUG::2013-11-19 21:19:40,541::task::495::TaskManager.Task::(_debug) Task 135fabaa-9efd-4650-b825-23a7508179c7: moving from state preparing -> state finished
>Thread-15910764::DEBUG::2013-11-19 21:19:40,542::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910764::DEBUG::2013-11-19 21:19:40,542::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910764::DEBUG::2013-11-19 21:19:40,543::task::495::TaskManager.Task::(_debug) Task 135fabaa-9efd-4650-b825-23a7508179c7: ref 0 aborting False
>Thread-15910764::INFO::2013-11-19 21:19:40,544::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000313997268677', 'lastCheck': 1384895971.9998729, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00165104866028', 'lastCheck': 1384895973.3947361, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895978.5368869, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000418901443481', 'lastCheck': 1384895977.9438889, 'valid': True, 'code': 0}}
>Dummy-15910548::DEBUG::2013-11-19 21:19:40,847::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:19:41,376::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0711464 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:19:43,390::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:19:43,915::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0704753 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:19:45,931::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:19:46,459::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.071543 s, 14.3 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:19:48,474::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:19:48,687::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15910548::DEBUG::2013-11-19 21:19:49,001::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.070849 s, 14.5 MB/s\n'; <rc> = 0
>Thread-15910770::DEBUG::2013-11-19 21:19:49,017::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910770::INFO::2013-11-19 21:19:49,019::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910770::DEBUG::2013-11-19 21:19:49,019::task::495::TaskManager.Task::(_debug) Task d589b847-6736-4e5d-8838-3c4061d6f00b: moving from state init -> state preparing
>Thread-15910770::DEBUG::2013-11-19 21:19:49,020::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=278 spmId=2
>Thread-15910770::DEBUG::2013-11-19 21:19:49,021::task::495::TaskManager.Task::(_debug) Task d589b847-6736-4e5d-8838-3c4061d6f00b: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910770::DEBUG::2013-11-19 21:19:49,021::task::495::TaskManager.Task::(_debug) Task d589b847-6736-4e5d-8838-3c4061d6f00b: moving from state preparing -> state finished
>Thread-15910770::DEBUG::2013-11-19 21:19:49,022::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910770::DEBUG::2013-11-19 21:19:49,022::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910770::DEBUG::2013-11-19 21:19:49,023::task::495::TaskManager.Task::(_debug) Task d589b847-6736-4e5d-8838-3c4061d6f00b: ref 0 aborting False
>Thread-15910770::INFO::2013-11-19 21:19:49,023::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910771::DEBUG::2013-11-19 21:19:49,028::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910771::INFO::2013-11-19 21:19:49,029::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910771::DEBUG::2013-11-19 21:19:49,029::task::495::TaskManager.Task::(_debug) Task e674a020-c38a-4d3b-873e-5b106a505864: moving from state init -> state preparing
>Thread-15910771::DEBUG::2013-11-19 21:19:49,031::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d0fd3e4e-56ab-4bb4-98ab-8d11bc865a31`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910771::DEBUG::2013-11-19 21:19:49,031::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910771::DEBUG::2013-11-19 21:19:49,032::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910771::DEBUG::2013-11-19 21:19:49,033::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d0fd3e4e-56ab-4bb4-98ab-8d11bc865a31`::Granted request
>Thread-15910771::DEBUG::2013-11-19 21:19:49,034::task::495::TaskManager.Task::(_debug) Task e674a020-c38a-4d3b-873e-5b106a505864: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910771::DEBUG::2013-11-19 21:19:49,034::task::495::TaskManager.Task::(_debug) Task e674a020-c38a-4d3b-873e-5b106a505864: ref 1 aborting False
>Thread-15910771::WARNING::2013-11-19 21:19:49,036::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910771::DEBUG::2013-11-19 21:19:49,037::task::495::TaskManager.Task::(_debug) Task e674a020-c38a-4d3b-873e-5b106a505864: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910771::DEBUG::2013-11-19 21:19:49,038::task::495::TaskManager.Task::(_debug) Task e674a020-c38a-4d3b-873e-5b106a505864: moving from state preparing -> state finished
>Thread-15910771::DEBUG::2013-11-19 21:19:49,038::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910771::DEBUG::2013-11-19 21:19:49,039::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910771::DEBUG::2013-11-19 21:19:49,040::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910771::DEBUG::2013-11-19 21:19:49,040::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910771::DEBUG::2013-11-19 21:19:49,041::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910771::DEBUG::2013-11-19 21:19:49,041::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910771::DEBUG::2013-11-19 21:19:49,042::task::495::TaskManager.Task::(_debug) Task e674a020-c38a-4d3b-873e-5b106a505864: ref 0 aborting False
>Thread-15910771::INFO::2013-11-19 21:19:49,043::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910772::INFO::2013-11-19 21:19:50,703::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910772::DEBUG::2013-11-19 21:19:50,705::task::495::TaskManager.Task::(_debug) Task 55fb2545-44ed-42c3-b95a-696e0294058a: moving from state init -> state preparing
>Thread-15910772::DEBUG::2013-11-19 21:19:50,706::task::495::TaskManager.Task::(_debug) Task 55fb2545-44ed-42c3-b95a-696e0294058a: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000441074371338', 'lastCheck': 1384895982.065028, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00247097015381', 'lastCheck': 1384895983.6277189, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895988.691354, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000871896743774', 'lastCheck': 1384895988.1502531, 'valid': True, 'code': 0}}
>Thread-15910772::DEBUG::2013-11-19 21:19:50,707::task::495::TaskManager.Task::(_debug) Task 55fb2545-44ed-42c3-b95a-696e0294058a: moving from state preparing -> state finished
>Thread-15910772::DEBUG::2013-11-19 21:19:50,709::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910772::DEBUG::2013-11-19 21:19:50,710::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910772::DEBUG::2013-11-19 21:19:50,711::task::495::TaskManager.Task::(_debug) Task 55fb2545-44ed-42c3-b95a-696e0294058a: ref 0 aborting False
>Thread-15910772::INFO::2013-11-19 21:19:50,712::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000441074371338', 'lastCheck': 1384895982.065028, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00247097015381', 'lastCheck': 1384895983.6277189, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895988.691354, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000871896743774', 'lastCheck': 1384895988.1502531, 'valid': True, 'code': 0}}
>Dummy-15910548::DEBUG::2013-11-19 21:19:51,019::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:19:51,562::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0705119 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:19:53,577::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:19:54,105::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0707819 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:19:56,120::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:19:56,644::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0804743 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:19:58,659::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:19:58,891::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910778::DEBUG::2013-11-19 21:19:59,193::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910778::INFO::2013-11-19 21:19:59,196::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910778::DEBUG::2013-11-19 21:19:59,197::task::495::TaskManager.Task::(_debug) Task 719f33c1-b0ed-4de8-8a82-449d4b1f0e3d: moving from state init -> state preparing
>Thread-15910778::DEBUG::2013-11-19 21:19:59,198::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=278 spmId=2
>Thread-15910778::DEBUG::2013-11-19 21:19:59,199::task::495::TaskManager.Task::(_debug) Task 719f33c1-b0ed-4de8-8a82-449d4b1f0e3d: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910778::DEBUG::2013-11-19 21:19:59,201::task::495::TaskManager.Task::(_debug) Task 719f33c1-b0ed-4de8-8a82-449d4b1f0e3d: moving from state preparing -> state finished
>Thread-15910778::DEBUG::2013-11-19 21:19:59,203::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910778::DEBUG::2013-11-19 21:19:59,204::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910778::DEBUG::2013-11-19 21:19:59,205::task::495::TaskManager.Task::(_debug) Task 719f33c1-b0ed-4de8-8a82-449d4b1f0e3d: ref 0 aborting False
>Thread-15910778::INFO::2013-11-19 21:19:59,206::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910779::DEBUG::2013-11-19 21:19:59,214::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910779::INFO::2013-11-19 21:19:59,216::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910779::DEBUG::2013-11-19 21:19:59,216::task::495::TaskManager.Task::(_debug) Task d64858dd-d362-4b97-a53f-c3715f509884: moving from state init -> state preparing
>Thread-15910779::DEBUG::2013-11-19 21:19:59,219::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`168c7599-a437-4628-8308-fc6ad7c81915`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910779::DEBUG::2013-11-19 21:19:59,220::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910779::DEBUG::2013-11-19 21:19:59,223::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910779::DEBUG::2013-11-19 21:19:59,224::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`168c7599-a437-4628-8308-fc6ad7c81915`::Granted request
>Thread-15910779::DEBUG::2013-11-19 21:19:59,242::task::495::TaskManager.Task::(_debug) Task d64858dd-d362-4b97-a53f-c3715f509884: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910779::DEBUG::2013-11-19 21:19:59,243::task::495::TaskManager.Task::(_debug) Task d64858dd-d362-4b97-a53f-c3715f509884: ref 1 aborting False
>Thread-15910779::WARNING::2013-11-19 21:19:59,247::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910779::DEBUG::2013-11-19 21:19:59,249::task::495::TaskManager.Task::(_debug) Task d64858dd-d362-4b97-a53f-c3715f509884: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910779::DEBUG::2013-11-19 21:19:59,250::task::495::TaskManager.Task::(_debug) Task d64858dd-d362-4b97-a53f-c3715f509884: moving from state preparing -> state finished
>Thread-15910779::DEBUG::2013-11-19 21:19:59,250::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910779::DEBUG::2013-11-19 21:19:59,252::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Dummy-15910548::DEBUG::2013-11-19 21:19:59,255::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0709386 s, 14.4 MB/s\n'; <rc> = 0
>Thread-15910779::DEBUG::2013-11-19 21:19:59,255::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910779::DEBUG::2013-11-19 21:19:59,262::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910779::DEBUG::2013-11-19 21:19:59,272::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910779::DEBUG::2013-11-19 21:19:59,274::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910779::DEBUG::2013-11-19 21:19:59,275::task::495::TaskManager.Task::(_debug) Task d64858dd-d362-4b97-a53f-c3715f509884: ref 0 aborting False
>Thread-15910779::INFO::2013-11-19 21:19:59,275::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910780::INFO::2013-11-19 21:20:00,904::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910780::DEBUG::2013-11-19 21:20:00,905::task::495::TaskManager.Task::(_debug) Task a3b42a13-3c6e-4b2e-98f8-cc0cee22cf4f: moving from state init -> state preparing
>Thread-15910780::DEBUG::2013-11-19 21:20:00,905::task::495::TaskManager.Task::(_debug) Task a3b42a13-3c6e-4b2e-98f8-cc0cee22cf4f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00055193901062', 'lastCheck': 1384895992.2627511, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00130295753479', 'lastCheck': 1384895993.8322611, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895998.895349, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000479936599731', 'lastCheck': 1384895998.329006, 'valid': True, 'code': 0}}
>Thread-15910780::DEBUG::2013-11-19 21:20:00,906::task::495::TaskManager.Task::(_debug) Task a3b42a13-3c6e-4b2e-98f8-cc0cee22cf4f: moving from state preparing -> state finished
>Thread-15910780::DEBUG::2013-11-19 21:20:00,907::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910780::DEBUG::2013-11-19 21:20:00,907::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910780::DEBUG::2013-11-19 21:20:00,908::task::495::TaskManager.Task::(_debug) Task a3b42a13-3c6e-4b2e-98f8-cc0cee22cf4f: ref 0 aborting False
>Thread-15910780::INFO::2013-11-19 21:20:00,908::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00055193901062', 'lastCheck': 1384895992.2627511, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00130295753479', 'lastCheck': 1384895993.8322611, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384895998.895349, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000479936599731', 'lastCheck': 1384895998.329006, 'valid': True, 'code': 0}}
>Dummy-15910548::DEBUG::2013-11-19 21:20:01,276::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:20:01,818::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0672367 s, 15.2 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:20:03,834::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:20:04,356::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.070988 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:20:06,371::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:20:06,881::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0703209 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:20:08,896::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:20:09,147::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910786::DEBUG::2013-11-19 21:20:09,385::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910786::INFO::2013-11-19 21:20:09,388::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910786::DEBUG::2013-11-19 21:20:09,389::task::495::TaskManager.Task::(_debug) Task 06221832-3351-475b-a398-a2dd2328eed6: moving from state init -> state preparing
>Thread-15910786::DEBUG::2013-11-19 21:20:09,391::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=278 spmId=2
>Thread-15910786::DEBUG::2013-11-19 21:20:09,392::task::495::TaskManager.Task::(_debug) Task 06221832-3351-475b-a398-a2dd2328eed6: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910786::DEBUG::2013-11-19 21:20:09,393::task::495::TaskManager.Task::(_debug) Task 06221832-3351-475b-a398-a2dd2328eed6: moving from state preparing -> state finished
>Thread-15910786::DEBUG::2013-11-19 21:20:09,394::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910786::DEBUG::2013-11-19 21:20:09,395::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910786::DEBUG::2013-11-19 21:20:09,397::task::495::TaskManager.Task::(_debug) Task 06221832-3351-475b-a398-a2dd2328eed6: ref 0 aborting False
>Thread-15910786::INFO::2013-11-19 21:20:09,398::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910787::DEBUG::2013-11-19 21:20:09,406::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910787::INFO::2013-11-19 21:20:09,407::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910787::DEBUG::2013-11-19 21:20:09,408::task::495::TaskManager.Task::(_debug) Task 08abcc71-d7c3-4b03-b2a7-49bc47e6152d: moving from state init -> state preparing
>Thread-15910787::DEBUG::2013-11-19 21:20:09,410::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a6a79b7c-f2d2-4a15-bbb2-0e5de5518ccd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15910787::DEBUG::2013-11-19 21:20:09,411::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15910787::DEBUG::2013-11-19 21:20:09,414::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15910787::DEBUG::2013-11-19 21:20:09,416::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a6a79b7c-f2d2-4a15-bbb2-0e5de5518ccd`::Granted request
>Thread-15910787::DEBUG::2013-11-19 21:20:09,418::task::495::TaskManager.Task::(_debug) Task 08abcc71-d7c3-4b03-b2a7-49bc47e6152d: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15910787::DEBUG::2013-11-19 21:20:09,419::task::495::TaskManager.Task::(_debug) Task 08abcc71-d7c3-4b03-b2a7-49bc47e6152d: ref 1 aborting False
>Thread-15910787::WARNING::2013-11-19 21:20:09,423::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910787::DEBUG::2013-11-19 21:20:09,425::task::495::TaskManager.Task::(_debug) Task 08abcc71-d7c3-4b03-b2a7-49bc47e6152d: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15910787::DEBUG::2013-11-19 21:20:09,426::task::495::TaskManager.Task::(_debug) Task 08abcc71-d7c3-4b03-b2a7-49bc47e6152d: moving from state preparing -> state finished
>Thread-15910787::DEBUG::2013-11-19 21:20:09,427::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910787::DEBUG::2013-11-19 21:20:09,429::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910787::DEBUG::2013-11-19 21:20:09,431::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910787::DEBUG::2013-11-19 21:20:09,432::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910787::DEBUG::2013-11-19 21:20:09,433::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910787::DEBUG::2013-11-19 21:20:09,434::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910787::DEBUG::2013-11-19 21:20:09,436::task::495::TaskManager.Task::(_debug) Task 08abcc71-d7c3-4b03-b2a7-49bc47e6152d: ref 0 aborting False
>Thread-15910787::INFO::2013-11-19 21:20:09,437::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 278}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15910548::DEBUG::2013-11-19 21:20:09,462::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0701178 s, 14.6 MB/s\n'; <rc> = 0
>Thread-15910788::INFO::2013-11-19 21:20:11,159::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910788::DEBUG::2013-11-19 21:20:11,159::task::495::TaskManager.Task::(_debug) Task 20b7c2d8-2788-49fb-83b0-768087f2b751: moving from state init -> state preparing
>Thread-15910788::DEBUG::2013-11-19 21:20:11,160::task::495::TaskManager.Task::(_debug) Task 20b7c2d8-2788-49fb-83b0-768087f2b751: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000427007675171', 'lastCheck': 1384896002.5115809, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148916244507', 'lastCheck': 1384896004.05053, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896009.151531, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000463008880615', 'lastCheck': 1384896008.5775249, 'valid': True, 'code': 0}}
>Thread-15910788::DEBUG::2013-11-19 21:20:11,161::task::495::TaskManager.Task::(_debug) Task 20b7c2d8-2788-49fb-83b0-768087f2b751: moving from state preparing -> state finished
>Thread-15910788::DEBUG::2013-11-19 21:20:11,161::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910788::DEBUG::2013-11-19 21:20:11,162::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910788::DEBUG::2013-11-19 21:20:11,162::task::495::TaskManager.Task::(_debug) Task 20b7c2d8-2788-49fb-83b0-768087f2b751: ref 0 aborting False
>Thread-15910788::INFO::2013-11-19 21:20:11,163::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000427007675171', 'lastCheck': 1384896002.5115809, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148916244507', 'lastCheck': 1384896004.05053, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896009.151531, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000463008880615', 'lastCheck': 1384896008.5775249, 'valid': True, 'code': 0}}
>Dummy-15910548::DEBUG::2013-11-19 21:20:11,477::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:20:12,007::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0704396 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15910548::DEBUG::2013-11-19 21:20:14,022::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:20:14,571::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0712785 s, 14.4 MB/s\n'; <rc> = 0
>Thread-15910792::DEBUG::2013-11-19 21:20:16,204::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910792::INFO::2013-11-19 21:20:16,205::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksStatuses, args: ()
>Thread-15910792::DEBUG::2013-11-19 21:20:16,206::task::495::TaskManager.Task::(_debug) Task 00790317-3f6e-4891-b438-6a18cd85170a: moving from state init -> state preparing
>Thread-15910792::DEBUG::2013-11-19 21:20:16,206::taskManager::92::TaskManager::(getAllTasksStatuses) Entry.
>Thread-15910792::DEBUG::2013-11-19 21:20:16,207::taskManager::97::TaskManager::(getAllTasksStatuses) Return: {}
>Thread-15910792::DEBUG::2013-11-19 21:20:16,207::task::495::TaskManager.Task::(_debug) Task 00790317-3f6e-4891-b438-6a18cd85170a: finished: {'allTasksStatus': {}}
>Thread-15910792::DEBUG::2013-11-19 21:20:16,208::task::495::TaskManager.Task::(_debug) Task 00790317-3f6e-4891-b438-6a18cd85170a: moving from state preparing -> state finished
>Thread-15910792::DEBUG::2013-11-19 21:20:16,209::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910792::DEBUG::2013-11-19 21:20:16,209::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910792::DEBUG::2013-11-19 21:20:16,210::task::495::TaskManager.Task::(_debug) Task 00790317-3f6e-4891-b438-6a18cd85170a: ref 0 aborting False
>Thread-15910792::INFO::2013-11-19 21:20:16,210::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksStatuses, Return response: {'status': {'message': 'OK', 'code': 0}, 'allTasksStatus': {}}
>Thread-15910793::DEBUG::2013-11-19 21:20:16,246::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910793::INFO::2013-11-19 21:20:16,247::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: deactivateStorageDomain, args: ( sdUUID=5d7b5178-3790-472c-a13b-b481eb27a61e spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 msdUUID=00000000-0000-0000-0000-000000000000 masterVersion=1)
>Thread-15910793::DEBUG::2013-11-19 21:20:16,247::task::495::TaskManager.Task::(_debug) Task 5fdbc2a9-93a6-491e-afd0-c8300046a427: moving from state init -> state preparing
>Thread-15910793::DEBUG::2013-11-19 21:20:16,249::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1e129772-23b3-4a80-9bc9-d8254465c6db`::Request was made in '/usr/share/vdsm/storage/spm.py' line '1112' at 'public_deactivateStorageDomain'
>Thread-15910793::DEBUG::2013-11-19 21:20:16,249::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'exclusive'
>Thread-15910793::DEBUG::2013-11-19 21:20:16,250::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'exclusive' (1 active user)
>Thread-15910793::DEBUG::2013-11-19 21:20:16,251::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1e129772-23b3-4a80-9bc9-d8254465c6db`::Granted request
>Thread-15910793::DEBUG::2013-11-19 21:20:16,252::task::495::TaskManager.Task::(_debug) Task 5fdbc2a9-93a6-491e-afd0-c8300046a427: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (exclusive)
>Thread-15910793::DEBUG::2013-11-19 21:20:16,252::task::495::TaskManager.Task::(_debug) Task 5fdbc2a9-93a6-491e-afd0-c8300046a427: ref 1 aborting False
>Thread-15910793::DEBUG::2013-11-19 21:20:16,253::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.5d7b5178-3790-472c-a13b-b481eb27a61e`ReqID=`8e68d170-b8f9-47cd-91ad-84886a61cf50`::Request was made in '/usr/share/vdsm/storage/spm.py' line '1113' at 'public_deactivateStorageDomain'
>Thread-15910793::DEBUG::2013-11-19 21:20:16,254::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e' for lock type 'exclusive'
>Thread-15910793::DEBUG::2013-11-19 21:20:16,255::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e' is free. Now locking as 'exclusive' (1 active user)
>Thread-15910793::DEBUG::2013-11-19 21:20:16,255::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.5d7b5178-3790-472c-a13b-b481eb27a61e`ReqID=`8e68d170-b8f9-47cd-91ad-84886a61cf50`::Granted request
>Thread-15910793::DEBUG::2013-11-19 21:20:16,256::task::495::TaskManager.Task::(_debug) Task 5fdbc2a9-93a6-491e-afd0-c8300046a427: _resourcesAcquired: Storage.5d7b5178-3790-472c-a13b-b481eb27a61e (exclusive)
>Thread-15910793::DEBUG::2013-11-19 21:20:16,256::task::495::TaskManager.Task::(_debug) Task 5fdbc2a9-93a6-491e-afd0-c8300046a427: ref 1 aborting False
>Thread-15910793::INFO::2013-11-19 21:20:16,257::sp::775::Storage.StoragePool::(deactivateSD) sdUUID=5d7b5178-3790-472c-a13b-b481eb27a61e spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 new_msdUUID=00000000-0000-0000-0000-000000000000
>Thread-15910793::INFO::2013-11-19 21:20:16,258::fileSD::176::Storage.StorageDomain::(validate) sdUUID=5d7b5178-3790-472c-a13b-b481eb27a61e
>Thread-15910793::DEBUG::2013-11-19 21:20:16,259::persistentDict::204::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=[]
>Thread-15910793::WARNING::2013-11-19 21:20:16,259::persistentDict::226::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is
>Thread-15910793::ERROR::2013-11-19 21:20:16,260::task::868::TaskManager.Task::(_setError) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/task.py", line 876, in _run
>  File "/usr/share/vdsm/storage/spm.py", line 115, in run
>  File "/usr/share/vdsm/storage/spm.py", line 1115, in public_deactivateStorageDomain
>  File "/usr/share/vdsm/storage/sp.py", line 798, in deactivateSD
>  File "/usr/share/vdsm/storage/sd.py", line 666, in isBackup
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15910793::DEBUG::2013-11-19 21:20:16,261::task::495::TaskManager.Task::(_debug) Task 5fdbc2a9-93a6-491e-afd0-c8300046a427: Task._run: 5fdbc2a9-93a6-491e-afd0-c8300046a427 ('5d7b5178-3790-472c-a13b-b481eb27a61e', 'c4e5889a-348d-11e2-83fe-9f72369eab30', '00000000-0000-0000-0000-000000000000', 1) {} failed - stopping task
>Thread-15910793::DEBUG::2013-11-19 21:20:16,262::task::495::TaskManager.Task::(_debug) Task 5fdbc2a9-93a6-491e-afd0-c8300046a427: stopping in state preparing (force False)
>Thread-15910793::DEBUG::2013-11-19 21:20:16,262::task::495::TaskManager.Task::(_debug) Task 5fdbc2a9-93a6-491e-afd0-c8300046a427: ref 1 aborting True
>Thread-15910793::INFO::2013-11-19 21:20:16,263::task::1171::TaskManager.Task::(prepare) aborting: Task is aborted: "'CLASS'" - code 100
>Thread-15910793::DEBUG::2013-11-19 21:20:16,263::task::495::TaskManager.Task::(_debug) Task 5fdbc2a9-93a6-491e-afd0-c8300046a427: Prepare: aborted: 'CLASS'
>Thread-15910793::DEBUG::2013-11-19 21:20:16,264::task::495::TaskManager.Task::(_debug) Task 5fdbc2a9-93a6-491e-afd0-c8300046a427: ref 0 aborting True
>Thread-15910793::DEBUG::2013-11-19 21:20:16,264::task::495::TaskManager.Task::(_debug) Task 5fdbc2a9-93a6-491e-afd0-c8300046a427: Task._doAbort: force False
>Thread-15910793::DEBUG::2013-11-19 21:20:16,265::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910793::DEBUG::2013-11-19 21:20:16,265::task::495::TaskManager.Task::(_debug) Task 5fdbc2a9-93a6-491e-afd0-c8300046a427: moving from state preparing -> state aborting
>Thread-15910793::DEBUG::2013-11-19 21:20:16,266::task::495::TaskManager.Task::(_debug) Task 5fdbc2a9-93a6-491e-afd0-c8300046a427: _aborting: recover policy none
>Thread-15910793::DEBUG::2013-11-19 21:20:16,266::task::495::TaskManager.Task::(_debug) Task 5fdbc2a9-93a6-491e-afd0-c8300046a427: moving from state aborting -> state failed
>Thread-15910793::DEBUG::2013-11-19 21:20:16,267::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>, 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e': < ResourceRef 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e', isValid: 'True' obj: 'None'>}
>Thread-15910793::DEBUG::2013-11-19 21:20:16,267::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910793::DEBUG::2013-11-19 21:20:16,268::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910793::DEBUG::2013-11-19 21:20:16,269::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910793::DEBUG::2013-11-19 21:20:16,269::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910793::DEBUG::2013-11-19 21:20:16,270::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910793::DEBUG::2013-11-19 21:20:16,270::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e'
>Thread-15910793::DEBUG::2013-11-19 21:20:16,271::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e' (0 active users)
>Thread-15910793::DEBUG::2013-11-19 21:20:16,271::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e' is free, finding out if anyone is waiting for it.
>Thread-15910793::DEBUG::2013-11-19 21:20:16,272::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e', Clearing records.
>Thread-15910793::ERROR::2013-11-19 21:20:16,273::dispatcher::106::Storage.Dispatcher.Protect::(run) 'CLASS'
>Thread-15910793::ERROR::2013-11-19 21:20:16,273::dispatcher::107::Storage.Dispatcher.Protect::(run) Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/dispatcher.py", line 96, in run
>  File "/usr/share/vdsm/storage/task.py", line 1178, in prepare
>KeyError: 'CLASS'
>
>Thread-15910794::DEBUG::2013-11-19 21:20:16,287::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910794::INFO::2013-11-19 21:20:16,288::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910794::DEBUG::2013-11-19 21:20:16,288::task::495::TaskManager.Task::(_debug) Task fc7eb1aa-14af-4b50-9a9f-4b2943f7e58f: moving from state init -> state preparing
>Thread-15910794::DEBUG::2013-11-19 21:20:16,289::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=278 spmId=2
>Thread-15910794::DEBUG::2013-11-19 21:20:16,290::task::495::TaskManager.Task::(_debug) Task fc7eb1aa-14af-4b50-9a9f-4b2943f7e58f: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910794::DEBUG::2013-11-19 21:20:16,290::task::495::TaskManager.Task::(_debug) Task fc7eb1aa-14af-4b50-9a9f-4b2943f7e58f: moving from state preparing -> state finished
>Thread-15910794::DEBUG::2013-11-19 21:20:16,291::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910794::DEBUG::2013-11-19 21:20:16,291::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910794::DEBUG::2013-11-19 21:20:16,292::task::495::TaskManager.Task::(_debug) Task fc7eb1aa-14af-4b50-9a9f-4b2943f7e58f: ref 0 aborting False
>Thread-15910794::INFO::2013-11-19 21:20:16,292::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 278}}
>Thread-15910795::DEBUG::2013-11-19 21:20:16,299::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910795::INFO::2013-11-19 21:20:16,300::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksStatuses, args: ()
>Thread-15910795::DEBUG::2013-11-19 21:20:16,301::task::495::TaskManager.Task::(_debug) Task c0d88e0a-6cf4-418e-b7e6-aca6061398a9: moving from state init -> state preparing
>Thread-15910795::DEBUG::2013-11-19 21:20:16,301::taskManager::92::TaskManager::(getAllTasksStatuses) Entry.
>Thread-15910795::DEBUG::2013-11-19 21:20:16,302::taskManager::97::TaskManager::(getAllTasksStatuses) Return: {}
>Thread-15910795::DEBUG::2013-11-19 21:20:16,302::task::495::TaskManager.Task::(_debug) Task c0d88e0a-6cf4-418e-b7e6-aca6061398a9: finished: {'allTasksStatus': {}}
>Thread-15910795::DEBUG::2013-11-19 21:20:16,303::task::495::TaskManager.Task::(_debug) Task c0d88e0a-6cf4-418e-b7e6-aca6061398a9: moving from state preparing -> state finished
>Thread-15910795::DEBUG::2013-11-19 21:20:16,304::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910795::DEBUG::2013-11-19 21:20:16,304::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910795::DEBUG::2013-11-19 21:20:16,305::task::495::TaskManager.Task::(_debug) Task c0d88e0a-6cf4-418e-b7e6-aca6061398a9: ref 0 aborting False
>Thread-15910795::INFO::2013-11-19 21:20:16,305::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksStatuses, Return response: {'status': {'message': 'OK', 'code': 0}, 'allTasksStatus': {}}
>Thread-15910796::DEBUG::2013-11-19 21:20:16,309::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910796::INFO::2013-11-19 21:20:16,310::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: spmStop, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910796::DEBUG::2013-11-19 21:20:16,310::task::495::TaskManager.Task::(_debug) Task f40f092b-3e0b-4e58-b570-13ecf7022426: moving from state init -> state preparing
>Thread-15910796::DEBUG::2013-11-19 21:20:16,311::spm::547::Storage.SPM::(public_spmStop) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30
>Thread-15910796::DEBUG::2013-11-19 21:20:16,311::taskManager::105::TaskManager::(getAllTasks) Entry.
>Thread-15910796::DEBUG::2013-11-19 21:20:16,312::taskManager::110::TaskManager::(getAllTasks) Return: {}
>Thread-15910796::DEBUG::2013-11-19 21:20:16,313::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`efeb1bbf-ea4e-41f2-bbd3-8d3969d320a5`::Request was made in '/usr/share/vdsm/storage/spm.py' line '573' at 'public_spmStop'
>Thread-15910796::DEBUG::2013-11-19 21:20:16,313::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'exclusive'
>Thread-15910796::DEBUG::2013-11-19 21:20:16,314::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'exclusive' (1 active user)
>Thread-15910796::DEBUG::2013-11-19 21:20:16,315::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`efeb1bbf-ea4e-41f2-bbd3-8d3969d320a5`::Granted request
>Thread-15910796::DEBUG::2013-11-19 21:20:16,316::task::495::TaskManager.Task::(_debug) Task f40f092b-3e0b-4e58-b570-13ecf7022426: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (exclusive)
>Thread-15910796::DEBUG::2013-11-19 21:20:16,316::task::495::TaskManager.Task::(_debug) Task f40f092b-3e0b-4e58-b570-13ecf7022426: ref 1 aborting False
>Thread-15910796::DEBUG::2013-11-19 21:20:16,317::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bbd5b86e-8d30-4418-b7b3-fb4db6c7aa32`::Request was made in '/usr/share/vdsm/storage/spm.py' line '584' at '_stop'
>Thread-15910796::DEBUG::2013-11-19 21:20:16,318::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'exclusive'
>Thread-15910796::DEBUG::2013-11-19 21:20:16,318::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'exclusive' (1 active user)
>Thread-15910796::DEBUG::2013-11-19 21:20:16,319::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bbd5b86e-8d30-4418-b7b3-fb4db6c7aa32`::Granted request
>Thread-15910796::DEBUG::2013-11-19 21:20:16,320::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e`ReqID=`34dcfc2e-6993-4439-8809-c361f883ca4c`::Request was made in '/usr/share/vdsm/storage/spm.py' line '601' at '_stop'
>Thread-15910796::DEBUG::2013-11-19 21:20:16,321::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e' for lock type 'exclusive'
>Thread-15910796::DEBUG::2013-11-19 21:20:16,321::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e' is free. Now locking as 'exclusive' (1 active user)
>Thread-15910796::DEBUG::2013-11-19 21:20:16,322::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e`ReqID=`34dcfc2e-6993-4439-8809-c361f883ca4c`::Granted request
>Thread-15910796::DEBUG::2013-11-19 21:20:16,323::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e'
>Thread-15910796::DEBUG::2013-11-19 21:20:16,323::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e' (0 active users)
>Thread-15910796::DEBUG::2013-11-19 21:20:16,324::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e' is free, finding out if anyone is waiting for it.
>Thread-15910796::DEBUG::2013-11-19 21:20:16,324::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e', Clearing records.
>Thread-15910796::DEBUG::2013-11-19 21:20:16,325::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910796::DEBUG::2013-11-19 21:20:16,326::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910796::DEBUG::2013-11-19 21:20:16,326::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910796::DEBUG::2013-11-19 21:20:16,327::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910796::DEBUG::2013-11-19 21:20:16,327::taskManager::118::TaskManager::(unloadTasks) Entry.
>Thread-15910796::DEBUG::2013-11-19 21:20:16,328::taskManager::122::TaskManager::(unloadTasks) Return
>Thread-15910796::DEBUG::2013-11-19 21:20:16,330::spm::195::Storage.SPM::(__cleanupMasterMount) unmounting /rhev/data-center/mnt/blockSD/6b892067-ac7e-4039-a7be-5b7387525b16/master
>Thread-15910796::DEBUG::2013-11-19 21:20:16,449::fileUtils::112::Storage.Misc.excCmd::(umount) '/usr/bin/sudo -n /bin/umount -f /rhev/data-center/mnt/blockSD/6b892067-ac7e-4039-a7be-5b7387525b16/master' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:20:16,635::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15910796::DEBUG::2013-11-19 21:20:16,944::fileUtils::112::Storage.Misc.excCmd::(umount) SUCCESS: <err> = ''; <rc> = 0
>Thread-15910796::DEBUG::2013-11-19 21:20:16,948::spm::290::Storage.SPM::(__cleanupSPM) cleaning up SPM: c4e5889a-348d-11e2-83fe-9f72369eab30
>Thread-15910796::DEBUG::2013-11-19 21:20:16,949::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
>Thread-15910796::DEBUG::2013-11-19 21:20:16,950::persistentDict::146::Storage.PersistentDict::(transaction) Flushing changes
>Thread-15910796::DEBUG::2013-11-19 21:20:16,951::persistentDict::265::Storage.PersistentDict::(flush) about to write lines (VGTagMetadataRW)=[u'CLASS=Data', u'DESCRIPTION=OS', u'IOOPTIMEOUTSEC=10', u'LEASERETRIES=3', u'LEASETIMESEC=60', u'LOCKPOLICY=', u'LOCKRENEWALINTERVALSEC=5', u'LOGBLKSIZE=512', u'MASTER_VERSION=1', u'PHYBLKSIZE=512', u'POOL_DESCRIPTION=Default', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_ID=-1', u'POOL_SPM_LVER=278', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'ROLE=Master', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'TYPE=FCP', u'VERSION=2', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', '_SHA_CKSUM=9f9e428092a2f2a5a408c3df7cae07b957db8170']
>Thread-15910796::DEBUG::2013-11-19 21:20:16,952::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15910796::DEBUG::2013-11-19 21:20:16,953::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15910796::DEBUG::2013-11-19 21:20:16,954::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15910796::DEBUG::2013-11-19 21:20:16,954::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15910796::DEBUG::2013-11-19 21:20:16,955::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-15910796::DEBUG::2013-11-19 21:20:16,957::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15910548::DEBUG::2013-11-19 21:20:17,440::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.137761 s, 7.4 MB/s\n'; <rc> = 0
>Thread-15910796::DEBUG::2013-11-19 21:20:17,763::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15910796::DEBUG::2013-11-19 21:20:17,766::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-15910796::DEBUG::2013-11-19 21:20:17,768::blockSD::145::storage.Metadata.VGTagMetadataRW::(writelines) Updating metadata adding=MDT_POOL_SPM_ID=-1, MDT__SHA_CKSUM=9f9e428092a2f2a5a408c3df7cae07b957db8170 removing=MDT__SHA_CKSUM=913ce85a0b55e60c3f92c982d8a15ae168c40596, MDT_POOL_SPM_ID=2
>Thread-15910796::DEBUG::2013-11-19 21:20:17,769::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --deltag MDT__SHA_CKSUM=913ce85a0b55e60c3f92c982d8a15ae168c40596 --deltag MDT_POOL_SPM_ID=2 --addtag MDT__SHA_CKSUM=9f9e428092a2f2a5a408c3df7cae07b957db8170 --addtag MDT_POOL_SPM_ID=-1 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15910796::DEBUG::2013-11-19 21:20:18,493::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15910796::DEBUG::2013-11-19 21:20:18,494::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15910796::DEBUG::2013-11-19 21:20:18,495::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15910796::DEBUG::2013-11-19 21:20:18,496::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction
>Thread-15910796::INFO::2013-11-19 21:20:18,497::safelease::83::ClusterLock::(release) Releasing cluster lock for domain 6b892067-ac7e-4039-a7be-5b7387525b16
>Thread-15910796::DEBUG::2013-11-19 21:20:18,497::safelease::84::Storage.Misc.excCmd::(release) '/usr/libexec/vdsm/spmstop.sh 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd /usr/libexec/vdsm/)
>Thread-1024767::ERROR::2013-11-19 21:20:19,302::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15910548::INFO::2013-11-19 21:20:19,456::storage_mailbox::673::Storage.MailBox.SpmMailMonitor::(run) SPM_MailMonitor - Incoming mail monitoring thread stopped
>Thread-15910796::DEBUG::2013-11-19 21:20:19,970::safelease::84::Storage.Misc.excCmd::(release) SUCCESS: <err> = ''; <rc> = 0
>Thread-15910796::DEBUG::2013-11-19 21:20:19,971::safelease::87::ClusterLock::(release) Cluster lock released successfully
>Thread-15910796::DEBUG::2013-11-19 21:20:19,972::task::495::TaskManager.Task::(_debug) Task f40f092b-3e0b-4e58-b570-13ecf7022426: finished: None
>Thread-15910796::DEBUG::2013-11-19 21:20:19,972::task::495::TaskManager.Task::(_debug) Task f40f092b-3e0b-4e58-b570-13ecf7022426: moving from state preparing -> state finished
>Thread-15910796::DEBUG::2013-11-19 21:20:19,973::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15910796::DEBUG::2013-11-19 21:20:19,974::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910796::DEBUG::2013-11-19 21:20:19,974::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910796::DEBUG::2013-11-19 21:20:19,975::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910796::DEBUG::2013-11-19 21:20:19,976::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910796::DEBUG::2013-11-19 21:20:19,976::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910796::DEBUG::2013-11-19 21:20:19,977::task::495::TaskManager.Task::(_debug) Task f40f092b-3e0b-4e58-b570-13ecf7022426: ref 0 aborting False
>Thread-15910796::INFO::2013-11-19 21:20:19,978::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: spmStop, Return response: {'status': {'message': 'OK', 'code': 0}}
>Thread-15910799::INFO::2013-11-19 21:20:21,328::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910799::DEBUG::2013-11-19 21:20:21,329::task::495::TaskManager.Task::(_debug) Task 9b6481ea-707a-40fd-840d-bbbd3a096195: moving from state init -> state preparing
>Thread-15910799::WARNING::2013-11-19 21:20:21,329::hsm::1895::Storage.HSM::(public_repoStats) repoStats detected invalid master:6b892067-ac7e-4039-a7be-5b7387525b16 {'finish': 1384896012.6998551, 'mdasize': '134479872', 'mdathreshold': True, 'mdavalid': True, 'diskfree': '1142595518464', 'disktotal': '3595827150848', 'result': {'delay': '0.000464916229248', 'lastCheck': 1384896012.6998551, 'valid': True, 'code': 0}, 'masterValidate': {'mount': True, 'valid': True}, 'mdafree': '67205632'}
>Thread-15910799::DEBUG::2013-11-19 21:20:21,330::task::495::TaskManager.Task::(_debug) Task 9b6481ea-707a-40fd-840d-bbbd3a096195: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000464916229248', 'lastCheck': 1384896012.6998551, 'valid': True, 'code': 355}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00149393081665', 'lastCheck': 1384896014.275703, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896019.3036189, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00049901008606', 'lastCheck': 1384896019.007432, 'valid': True, 'code': 0}}
>Thread-15910799::DEBUG::2013-11-19 21:20:21,331::task::495::TaskManager.Task::(_debug) Task 9b6481ea-707a-40fd-840d-bbbd3a096195: moving from state preparing -> state finished
>Thread-15910799::DEBUG::2013-11-19 21:20:21,331::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910799::DEBUG::2013-11-19 21:20:21,332::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910799::DEBUG::2013-11-19 21:20:21,333::task::495::TaskManager.Task::(_debug) Task 9b6481ea-707a-40fd-840d-bbbd3a096195: ref 0 aborting False
>Thread-15910799::INFO::2013-11-19 21:20:21,333::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000464916229248', 'lastCheck': 1384896012.6998551, 'valid': True, 'code': 355}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00149393081665', 'lastCheck': 1384896014.275703, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896019.3036189, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00049901008606', 'lastCheck': 1384896019.007432, 'valid': True, 'code': 0}}
>Thread-42838::DEBUG::2013-11-19 21:20:23,015::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-42838::DEBUG::2013-11-19 21:20:23,016::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 21:20:23,593::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-42838::DEBUG::2013-11-19 21:20:23,596::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1024767::ERROR::2013-11-19 21:20:29,317::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910805::INFO::2013-11-19 21:20:31,433::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910805::DEBUG::2013-11-19 21:20:31,434::task::495::TaskManager.Task::(_debug) Task 34412443-9cf3-4d9a-a169-2e5f98c5e011: moving from state init -> state preparing
>Thread-15910805::DEBUG::2013-11-19 21:20:31,435::task::495::TaskManager.Task::(_debug) Task 34412443-9cf3-4d9a-a169-2e5f98c5e011: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000357151031494', 'lastCheck': 1384896023.5994561, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148010253906', 'lastCheck': 1384896024.3090329, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896029.318846, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000409126281738', 'lastCheck': 1384896029.0193429, 'valid': True, 'code': 0}}
>Thread-15910805::DEBUG::2013-11-19 21:20:31,435::task::495::TaskManager.Task::(_debug) Task 34412443-9cf3-4d9a-a169-2e5f98c5e011: moving from state preparing -> state finished
>Thread-15910805::DEBUG::2013-11-19 21:20:31,436::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910805::DEBUG::2013-11-19 21:20:31,437::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910805::DEBUG::2013-11-19 21:20:31,437::task::495::TaskManager.Task::(_debug) Task 34412443-9cf3-4d9a-a169-2e5f98c5e011: ref 0 aborting False
>Thread-15910805::INFO::2013-11-19 21:20:31,438::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000357151031494', 'lastCheck': 1384896023.5994561, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148010253906', 'lastCheck': 1384896024.3090329, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896029.318846, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000409126281738', 'lastCheck': 1384896029.0193429, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:20:39,331::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910811::INFO::2013-11-19 21:20:41,562::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910811::DEBUG::2013-11-19 21:20:41,562::task::495::TaskManager.Task::(_debug) Task 0f09536d-2115-484e-be99-4bacf636bae6: moving from state init -> state preparing
>Thread-15910811::DEBUG::2013-11-19 21:20:41,563::task::495::TaskManager.Task::(_debug) Task 0f09536d-2115-484e-be99-4bacf636bae6: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000357866287231', 'lastCheck': 1384896033.6125431, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00150203704834', 'lastCheck': 1384896034.3245611, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896039.3427711, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00388693809509', 'lastCheck': 1384896039.035394, 'valid': True, 'code': 0}}
>Thread-15910811::DEBUG::2013-11-19 21:20:41,564::task::495::TaskManager.Task::(_debug) Task 0f09536d-2115-484e-be99-4bacf636bae6: moving from state preparing -> state finished
>Thread-15910811::DEBUG::2013-11-19 21:20:41,564::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910811::DEBUG::2013-11-19 21:20:41,565::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910811::DEBUG::2013-11-19 21:20:41,565::task::495::TaskManager.Task::(_debug) Task 0f09536d-2115-484e-be99-4bacf636bae6: ref 0 aborting False
>Thread-15910811::INFO::2013-11-19 21:20:41,566::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000357866287231', 'lastCheck': 1384896033.6125431, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00150203704834', 'lastCheck': 1384896034.3245611, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896039.3427711, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00388693809509', 'lastCheck': 1384896039.035394, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:20:49,355::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910817::INFO::2013-11-19 21:20:51,690::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910817::DEBUG::2013-11-19 21:20:51,691::task::495::TaskManager.Task::(_debug) Task 75574db4-63ef-448c-8788-09c2a345bbea: moving from state init -> state preparing
>Thread-15910817::DEBUG::2013-11-19 21:20:51,692::task::495::TaskManager.Task::(_debug) Task 75574db4-63ef-448c-8788-09c2a345bbea: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000338077545166', 'lastCheck': 1384896043.6254671, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0022280216217', 'lastCheck': 1384896044.3410809, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896049.35642, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00588321685791', 'lastCheck': 1384896049.051645, 'valid': True, 'code': 0}}
>Thread-15910817::DEBUG::2013-11-19 21:20:51,693::task::495::TaskManager.Task::(_debug) Task 75574db4-63ef-448c-8788-09c2a345bbea: moving from state preparing -> state finished
>Thread-15910817::DEBUG::2013-11-19 21:20:51,693::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910817::DEBUG::2013-11-19 21:20:51,694::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910817::DEBUG::2013-11-19 21:20:51,694::task::495::TaskManager.Task::(_debug) Task 75574db4-63ef-448c-8788-09c2a345bbea: ref 0 aborting False
>Thread-15910817::INFO::2013-11-19 21:20:51,695::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000338077545166', 'lastCheck': 1384896043.6254671, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0022280216217', 'lastCheck': 1384896044.3410809, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896049.35642, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00588321685791', 'lastCheck': 1384896049.051645, 'valid': True, 'code': 0}}
>Thread-15910821::DEBUG::2013-11-19 21:20:56,381::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910821::INFO::2013-11-19 21:20:56,382::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910821::DEBUG::2013-11-19 21:20:56,382::task::495::TaskManager.Task::(_debug) Task a2e49274-8c78-484c-99f1-5925ccab7cfd: moving from state init -> state preparing
>Thread-15910821::DEBUG::2013-11-19 21:20:56,383::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=Free spmLver=278 spmId=-1
>Thread-15910821::DEBUG::2013-11-19 21:20:56,383::task::495::TaskManager.Task::(_debug) Task a2e49274-8c78-484c-99f1-5925ccab7cfd: finished: {'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': 278}}
>Thread-15910821::DEBUG::2013-11-19 21:20:56,384::task::495::TaskManager.Task::(_debug) Task a2e49274-8c78-484c-99f1-5925ccab7cfd: moving from state preparing -> state finished
>Thread-15910821::DEBUG::2013-11-19 21:20:56,385::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910821::DEBUG::2013-11-19 21:20:56,385::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910821::DEBUG::2013-11-19 21:20:56,386::task::495::TaskManager.Task::(_debug) Task a2e49274-8c78-484c-99f1-5925ccab7cfd: ref 0 aborting False
>Thread-15910821::INFO::2013-11-19 21:20:56,386::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': 278}}
>Thread-15910822::DEBUG::2013-11-19 21:20:56,409::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910822::INFO::2013-11-19 21:20:56,410::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: spmStart, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 prevID=-1 prevLVER=278 recoveryMode=0 scsiFencing=false maxHostID=250 domVersion=2)
>Thread-15910822::DEBUG::2013-11-19 21:20:56,411::task::495::TaskManager.Task::(_debug) Task 934c4d35-aab6-4fba-9996-9cc07dd7d991: moving from state init -> state preparing
>Thread-15910822::DEBUG::2013-11-19 21:20:56,412::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f2292637-8eb5-4d89-b0e9-66555f85b956`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '330' at 'public_spmStart'
>Thread-15910822::DEBUG::2013-11-19 21:20:56,412::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'exclusive'
>Thread-15910822::DEBUG::2013-11-19 21:20:56,413::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'exclusive' (1 active user)
>Thread-15910822::DEBUG::2013-11-19 21:20:56,414::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f2292637-8eb5-4d89-b0e9-66555f85b956`::Granted request
>Thread-15910822::DEBUG::2013-11-19 21:20:56,415::task::495::TaskManager.Task::(_debug) Task 934c4d35-aab6-4fba-9996-9cc07dd7d991: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (exclusive)
>Thread-15910822::DEBUG::2013-11-19 21:20:56,415::task::495::TaskManager.Task::(_debug) Task 934c4d35-aab6-4fba-9996-9cc07dd7d991: ref 1 aborting False
>Thread-15910822::DEBUG::2013-11-19 21:20:56,416::task::495::TaskManager.Task::(_debug) Task 934c4d35-aab6-4fba-9996-9cc07dd7d991: Prepare: 1 jobs exist, move to acquiring
>Thread-15910822::DEBUG::2013-11-19 21:20:56,416::task::495::TaskManager.Task::(_debug) Task 934c4d35-aab6-4fba-9996-9cc07dd7d991: moving from state preparing -> state acquiring
>Thread-15910822::DEBUG::2013-11-19 21:20:56,417::task::495::TaskManager.Task::(_debug) Task 934c4d35-aab6-4fba-9996-9cc07dd7d991: moving from state acquiring -> state queued
>Thread-15910822::DEBUG::2013-11-19 21:20:56,417::taskManager::37::TaskManager::(_queueTask) queueing task: 934c4d35-aab6-4fba-9996-9cc07dd7d991
>Thread-15910822::DEBUG::2013-11-19 21:20:56,418::taskManager::43::TaskManager::(_queueTask) task queued: 934c4d35-aab6-4fba-9996-9cc07dd7d991
>eca29df8-c91e-46f6-8df8-b5d172c88b9f::DEBUG::2013-11-19 21:20:56,418::threadPool::47::Misc.ThreadPool::(setRunningTask) Number of running tasks: 1
>Thread-15910822::DEBUG::2013-11-19 21:20:56,419::task::1183::TaskManager.Task::(prepare) returning
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:56,420::threadPool::192::Misc.ThreadPool.WorkerThread::(run) Task: 934c4d35-aab6-4fba-9996-9cc07dd7d991 running: <bound method Task.commit of <storage.task.Task instance at 0x7f487c1379e0>> with: None
>Thread-15910822::DEBUG::2013-11-19 21:20:56,420::task::495::TaskManager.Task::(_debug) Task 934c4d35-aab6-4fba-9996-9cc07dd7d991: ref 0 aborting False
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:56,421::task::1195::TaskManager.Task::(commit) Commiting task: 934c4d35-aab6-4fba-9996-9cc07dd7d991
>Thread-15910822::INFO::2013-11-19 21:20:56,421::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: spmStart, Return response: {'status': {'message': 'OK', 'code': 0}, 'uuid': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:56,422::task::495::TaskManager.Task::(_debug) Task 934c4d35-aab6-4fba-9996-9cc07dd7d991: moving from state queued -> state running
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:56,423::task::907::TaskManager.Task::(_runJobs) Task.run: running job 0: spmStart: <bound method SPM.start of <storage.spm.SPM instance at 0x1bf9f38>> (args: ('c4e5889a-348d-11e2-83fe-9f72369eab30', -1, '278', 0, 'false', 250, 2) kwargs: {})
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:56,424::task::300::TaskManager.Task::(run) Job.run: running spmStart: <bound method SPM.start of <storage.spm.SPM instance at 0x1bf9f38>> (args: ('c4e5889a-348d-11e2-83fe-9f72369eab30', -1, '278', 0, 'false', 250, 2) kwargs: {}) callback None
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:56,424::spm::433::Storage.SPM::(start) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:56,425::misc::1010::SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads)
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:56,425::misc::1012::SamplingMethod::(__call__) Got in to sampling method
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:56,426::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:56,426::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:56,427::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:56,428::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:56,428::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:56,429::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:57,119::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:57,122::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:57,124::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_LVER=279', u'_SHA_CKSUM=b5dee775aac07995010b8c6a5db186617df84cf4', u'POOL_SPM_ID=-1']
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:57,125::misc::1020::SamplingMethod::(__call__) Returning last result
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:57,126::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:57,128::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:57,129::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:57,129::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:57,130::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:57,131::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15910823::DEBUG::2013-11-19 21:20:57,430::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910823::INFO::2013-11-19 21:20:57,432::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=934c4d35-aab6-4fba-9996-9cc07dd7d991)
>Thread-15910823::DEBUG::2013-11-19 21:20:57,433::task::495::TaskManager.Task::(_debug) Task 4f9b8411-2d56-494e-b3fb-0510e0fd7c57: moving from state init -> state preparing
>Thread-15910823::DEBUG::2013-11-19 21:20:57,433::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 934c4d35-aab6-4fba-9996-9cc07dd7d991
>Thread-15910823::DEBUG::2013-11-19 21:20:57,434::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}
>Thread-15910823::DEBUG::2013-11-19 21:20:57,435::task::495::TaskManager.Task::(_debug) Task 4f9b8411-2d56-494e-b3fb-0510e0fd7c57: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910823::DEBUG::2013-11-19 21:20:57,435::task::495::TaskManager.Task::(_debug) Task 4f9b8411-2d56-494e-b3fb-0510e0fd7c57: moving from state preparing -> state finished
>Thread-15910823::DEBUG::2013-11-19 21:20:57,436::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910823::DEBUG::2013-11-19 21:20:57,436::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910823::DEBUG::2013-11-19 21:20:57,437::task::495::TaskManager.Task::(_debug) Task 4f9b8411-2d56-494e-b3fb-0510e0fd7c57: ref 0 aborting False
>Thread-15910823::INFO::2013-11-19 21:20:57,438::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:57,788::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:57,791::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:57,792::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_LVER=279', u'_SHA_CKSUM=b5dee775aac07995010b8c6a5db186617df84cf4', u'POOL_SPM_ID=-1']
>934c4d35-aab6-4fba-9996-9cc07dd7d991::INFO::2013-11-19 21:20:57,793::spm::457::Storage.SPM::(start) expected previd:-1 lver:279 got request for previd:-1 lver:278
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:57,794::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:57,795::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:57,796::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:57,796::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:57,797::persistentDict::163::Storage.PersistentDict::(__init__) Created a persistant dict with LvMetadataRW backend
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:57,798::blockSD::164::Storage.Misc.excCmd::(readlines) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:58,203::blockSD::164::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000230349 s, 8.9 MB/s\n'; <rc> = 0
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:58,204::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000230349 s, 8.9 MB/s'], size: 2048
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:58,205::persistentDict::204::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[]
>934c4d35-aab6-4fba-9996-9cc07dd7d991::WARNING::2013-11-19 21:20:58,205::persistentDict::226::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:58,206::persistentDict::163::Storage.PersistentDict::(__init__) Created a persistant dict with VGTagMetadataRW backend
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:58,207::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:58,207::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:58,208::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:58,209::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:58,209::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:58,210::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15910825::DEBUG::2013-11-19 21:20:58,447::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910825::INFO::2013-11-19 21:20:58,450::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=934c4d35-aab6-4fba-9996-9cc07dd7d991)
>Thread-15910825::DEBUG::2013-11-19 21:20:58,450::task::495::TaskManager.Task::(_debug) Task 7c11ac28-8b29-419d-bfe0-68cfca23a5b9: moving from state init -> state preparing
>Thread-15910825::DEBUG::2013-11-19 21:20:58,451::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 934c4d35-aab6-4fba-9996-9cc07dd7d991
>Thread-15910825::DEBUG::2013-11-19 21:20:58,452::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}
>Thread-15910825::DEBUG::2013-11-19 21:20:58,452::task::495::TaskManager.Task::(_debug) Task 7c11ac28-8b29-419d-bfe0-68cfca23a5b9: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910825::DEBUG::2013-11-19 21:20:58,453::task::495::TaskManager.Task::(_debug) Task 7c11ac28-8b29-419d-bfe0-68cfca23a5b9: moving from state preparing -> state finished
>Thread-15910825::DEBUG::2013-11-19 21:20:58,453::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910825::DEBUG::2013-11-19 21:20:58,454::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910825::DEBUG::2013-11-19 21:20:58,455::task::495::TaskManager.Task::(_debug) Task 7c11ac28-8b29-419d-bfe0-68cfca23a5b9: ref 0 aborting False
>Thread-15910825::INFO::2013-11-19 21:20:58,455::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:58,845::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:58,848::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:58,849::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_LVER=279', u'_SHA_CKSUM=b5dee775aac07995010b8c6a5db186617df84cf4', u'POOL_SPM_ID=-1']
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:58,850::safelease::52::ClusterLock::(acquire) Acquiring cluster lock for domain 6b892067-ac7e-4039-a7be-5b7387525b16
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:20:58,851::safelease::61::Storage.Misc.excCmd::(acquire) '/usr/bin/sudo -n /usr/bin/setsid /usr/bin/ionice -c1 -n0 /bin/su vdsm -s /bin/sh -c "/usr/libexec/vdsm/spmprotect.sh start 6b892067-ac7e-4039-a7be-5b7387525b16 2 5 /dev/6b892067-ac7e-4039-a7be-5b7387525b16/leases 60000 10000 3"' (cwd /usr/libexec/vdsm/)
>Thread-1024767::ERROR::2013-11-19 21:20:59,418::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910826::DEBUG::2013-11-19 21:20:59,463::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910826::INFO::2013-11-19 21:20:59,464::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=934c4d35-aab6-4fba-9996-9cc07dd7d991)
>Thread-15910826::DEBUG::2013-11-19 21:20:59,465::task::495::TaskManager.Task::(_debug) Task f9db044d-9d2b-4481-a225-8522427dd4ad: moving from state init -> state preparing
>Thread-15910826::DEBUG::2013-11-19 21:20:59,465::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 934c4d35-aab6-4fba-9996-9cc07dd7d991
>Thread-15910826::DEBUG::2013-11-19 21:20:59,466::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}
>Thread-15910826::DEBUG::2013-11-19 21:20:59,467::task::495::TaskManager.Task::(_debug) Task f9db044d-9d2b-4481-a225-8522427dd4ad: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910826::DEBUG::2013-11-19 21:20:59,467::task::495::TaskManager.Task::(_debug) Task f9db044d-9d2b-4481-a225-8522427dd4ad: moving from state preparing -> state finished
>Thread-15910826::DEBUG::2013-11-19 21:20:59,468::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910826::DEBUG::2013-11-19 21:20:59,468::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910826::DEBUG::2013-11-19 21:20:59,469::task::495::TaskManager.Task::(_debug) Task f9db044d-9d2b-4481-a225-8522427dd4ad: ref 0 aborting False
>Thread-15910826::INFO::2013-11-19 21:20:59,469::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910828::DEBUG::2013-11-19 21:21:00,475::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910828::INFO::2013-11-19 21:21:00,476::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=934c4d35-aab6-4fba-9996-9cc07dd7d991)
>Thread-15910828::DEBUG::2013-11-19 21:21:00,477::task::495::TaskManager.Task::(_debug) Task b7645369-6ad2-47fa-a153-f9100e0b9331: moving from state init -> state preparing
>Thread-15910828::DEBUG::2013-11-19 21:21:00,477::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 934c4d35-aab6-4fba-9996-9cc07dd7d991
>Thread-15910828::DEBUG::2013-11-19 21:21:00,478::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}
>Thread-15910828::DEBUG::2013-11-19 21:21:00,478::task::495::TaskManager.Task::(_debug) Task b7645369-6ad2-47fa-a153-f9100e0b9331: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910828::DEBUG::2013-11-19 21:21:00,479::task::495::TaskManager.Task::(_debug) Task b7645369-6ad2-47fa-a153-f9100e0b9331: moving from state preparing -> state finished
>Thread-15910828::DEBUG::2013-11-19 21:21:00,479::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910828::DEBUG::2013-11-19 21:21:00,480::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910828::DEBUG::2013-11-19 21:21:00,481::task::495::TaskManager.Task::(_debug) Task b7645369-6ad2-47fa-a153-f9100e0b9331: ref 0 aborting False
>Thread-15910828::INFO::2013-11-19 21:21:00,481::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910829::DEBUG::2013-11-19 21:21:01,487::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910829::INFO::2013-11-19 21:21:01,487::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=934c4d35-aab6-4fba-9996-9cc07dd7d991)
>Thread-15910829::DEBUG::2013-11-19 21:21:01,488::task::495::TaskManager.Task::(_debug) Task ca945ac7-f9b6-4ab2-8e0c-3689483d1db1: moving from state init -> state preparing
>Thread-15910829::DEBUG::2013-11-19 21:21:01,489::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 934c4d35-aab6-4fba-9996-9cc07dd7d991
>Thread-15910829::DEBUG::2013-11-19 21:21:01,489::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}
>Thread-15910829::DEBUG::2013-11-19 21:21:01,490::task::495::TaskManager.Task::(_debug) Task ca945ac7-f9b6-4ab2-8e0c-3689483d1db1: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910829::DEBUG::2013-11-19 21:21:01,490::task::495::TaskManager.Task::(_debug) Task ca945ac7-f9b6-4ab2-8e0c-3689483d1db1: moving from state preparing -> state finished
>Thread-15910829::DEBUG::2013-11-19 21:21:01,491::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910829::DEBUG::2013-11-19 21:21:01,491::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910829::DEBUG::2013-11-19 21:21:01,492::task::495::TaskManager.Task::(_debug) Task ca945ac7-f9b6-4ab2-8e0c-3689483d1db1: ref 0 aborting False
>Thread-15910829::INFO::2013-11-19 21:21:01,492::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910830::INFO::2013-11-19 21:21:01,799::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910830::DEBUG::2013-11-19 21:21:01,799::task::495::TaskManager.Task::(_debug) Task 6abd6968-5998-4fac-a45f-9c016d4518c5: moving from state init -> state preparing
>Thread-15910830::DEBUG::2013-11-19 21:21:01,800::task::495::TaskManager.Task::(_debug) Task 6abd6968-5998-4fac-a45f-9c016d4518c5: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000318050384521', 'lastCheck': 1384896053.638303, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00149297714233', 'lastCheck': 1384896054.3564379, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896059.4198689, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000597953796387', 'lastCheck': 1384896059.064239, 'valid': True, 'code': 0}}
>Thread-15910830::DEBUG::2013-11-19 21:21:01,801::task::495::TaskManager.Task::(_debug) Task 6abd6968-5998-4fac-a45f-9c016d4518c5: moving from state preparing -> state finished
>Thread-15910830::DEBUG::2013-11-19 21:21:01,801::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910830::DEBUG::2013-11-19 21:21:01,802::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910830::DEBUG::2013-11-19 21:21:01,802::task::495::TaskManager.Task::(_debug) Task 6abd6968-5998-4fac-a45f-9c016d4518c5: ref 0 aborting False
>Thread-15910830::INFO::2013-11-19 21:21:01,803::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000318050384521', 'lastCheck': 1384896053.638303, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00149297714233', 'lastCheck': 1384896054.3564379, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896059.4198689, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000597953796387', 'lastCheck': 1384896059.064239, 'valid': True, 'code': 0}}
>Thread-15910832::DEBUG::2013-11-19 21:21:02,498::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910832::INFO::2013-11-19 21:21:02,499::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=934c4d35-aab6-4fba-9996-9cc07dd7d991)
>Thread-15910832::DEBUG::2013-11-19 21:21:02,499::task::495::TaskManager.Task::(_debug) Task f44d5c0c-113b-47b0-b5fe-6e1997e83206: moving from state init -> state preparing
>Thread-15910832::DEBUG::2013-11-19 21:21:02,500::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 934c4d35-aab6-4fba-9996-9cc07dd7d991
>Thread-15910832::DEBUG::2013-11-19 21:21:02,501::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}
>Thread-15910832::DEBUG::2013-11-19 21:21:02,501::task::495::TaskManager.Task::(_debug) Task f44d5c0c-113b-47b0-b5fe-6e1997e83206: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910832::DEBUG::2013-11-19 21:21:02,502::task::495::TaskManager.Task::(_debug) Task f44d5c0c-113b-47b0-b5fe-6e1997e83206: moving from state preparing -> state finished
>Thread-15910832::DEBUG::2013-11-19 21:21:02,503::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910832::DEBUG::2013-11-19 21:21:02,503::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910832::DEBUG::2013-11-19 21:21:02,504::task::495::TaskManager.Task::(_debug) Task f44d5c0c-113b-47b0-b5fe-6e1997e83206: ref 0 aborting False
>Thread-15910832::INFO::2013-11-19 21:21:02,504::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910833::DEBUG::2013-11-19 21:21:03,510::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910833::INFO::2013-11-19 21:21:03,511::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=934c4d35-aab6-4fba-9996-9cc07dd7d991)
>Thread-15910833::DEBUG::2013-11-19 21:21:03,511::task::495::TaskManager.Task::(_debug) Task 5b7dafa1-9877-4ca8-8baf-fecf9b06b7df: moving from state init -> state preparing
>Thread-15910833::DEBUG::2013-11-19 21:21:03,512::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 934c4d35-aab6-4fba-9996-9cc07dd7d991
>Thread-15910833::DEBUG::2013-11-19 21:21:03,512::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}
>Thread-15910833::DEBUG::2013-11-19 21:21:03,513::task::495::TaskManager.Task::(_debug) Task 5b7dafa1-9877-4ca8-8baf-fecf9b06b7df: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910833::DEBUG::2013-11-19 21:21:03,514::task::495::TaskManager.Task::(_debug) Task 5b7dafa1-9877-4ca8-8baf-fecf9b06b7df: moving from state preparing -> state finished
>Thread-15910833::DEBUG::2013-11-19 21:21:03,514::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910833::DEBUG::2013-11-19 21:21:03,515::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910833::DEBUG::2013-11-19 21:21:03,515::task::495::TaskManager.Task::(_debug) Task 5b7dafa1-9877-4ca8-8baf-fecf9b06b7df: ref 0 aborting False
>Thread-15910833::INFO::2013-11-19 21:21:03,516::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910835::DEBUG::2013-11-19 21:21:04,521::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910835::INFO::2013-11-19 21:21:04,522::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=934c4d35-aab6-4fba-9996-9cc07dd7d991)
>Thread-15910835::DEBUG::2013-11-19 21:21:04,539::task::495::TaskManager.Task::(_debug) Task 932131cf-bf82-48ea-a368-89f89a89c97b: moving from state init -> state preparing
>Thread-15910835::DEBUG::2013-11-19 21:21:04,539::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 934c4d35-aab6-4fba-9996-9cc07dd7d991
>Thread-15910835::DEBUG::2013-11-19 21:21:04,540::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}
>Thread-15910835::DEBUG::2013-11-19 21:21:04,540::task::495::TaskManager.Task::(_debug) Task 932131cf-bf82-48ea-a368-89f89a89c97b: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910835::DEBUG::2013-11-19 21:21:04,541::task::495::TaskManager.Task::(_debug) Task 932131cf-bf82-48ea-a368-89f89a89c97b: moving from state preparing -> state finished
>Thread-15910835::DEBUG::2013-11-19 21:21:04,542::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910835::DEBUG::2013-11-19 21:21:04,542::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910835::DEBUG::2013-11-19 21:21:04,543::task::495::TaskManager.Task::(_debug) Task 932131cf-bf82-48ea-a368-89f89a89c97b: ref 0 aborting False
>Thread-15910835::INFO::2013-11-19 21:21:04,543::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910836::DEBUG::2013-11-19 21:21:05,549::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910836::INFO::2013-11-19 21:21:05,550::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=934c4d35-aab6-4fba-9996-9cc07dd7d991)
>Thread-15910836::DEBUG::2013-11-19 21:21:05,550::task::495::TaskManager.Task::(_debug) Task 70e52133-8750-46d4-9d53-3944f78bddc2: moving from state init -> state preparing
>Thread-15910836::DEBUG::2013-11-19 21:21:05,551::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 934c4d35-aab6-4fba-9996-9cc07dd7d991
>Thread-15910836::DEBUG::2013-11-19 21:21:05,551::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}
>Thread-15910836::DEBUG::2013-11-19 21:21:05,552::task::495::TaskManager.Task::(_debug) Task 70e52133-8750-46d4-9d53-3944f78bddc2: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910836::DEBUG::2013-11-19 21:21:05,553::task::495::TaskManager.Task::(_debug) Task 70e52133-8750-46d4-9d53-3944f78bddc2: moving from state preparing -> state finished
>Thread-15910836::DEBUG::2013-11-19 21:21:05,553::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910836::DEBUG::2013-11-19 21:21:05,554::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910836::DEBUG::2013-11-19 21:21:05,554::task::495::TaskManager.Task::(_debug) Task 70e52133-8750-46d4-9d53-3944f78bddc2: ref 0 aborting False
>Thread-15910836::INFO::2013-11-19 21:21:05,555::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910838::DEBUG::2013-11-19 21:21:06,560::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910838::INFO::2013-11-19 21:21:06,561::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=934c4d35-aab6-4fba-9996-9cc07dd7d991)
>Thread-15910838::DEBUG::2013-11-19 21:21:06,562::task::495::TaskManager.Task::(_debug) Task 81fcc1d9-f17b-46fe-b428-f989ad5ca468: moving from state init -> state preparing
>Thread-15910838::DEBUG::2013-11-19 21:21:06,562::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 934c4d35-aab6-4fba-9996-9cc07dd7d991
>Thread-15910838::DEBUG::2013-11-19 21:21:06,563::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}
>Thread-15910838::DEBUG::2013-11-19 21:21:06,563::task::495::TaskManager.Task::(_debug) Task 81fcc1d9-f17b-46fe-b428-f989ad5ca468: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910838::DEBUG::2013-11-19 21:21:06,564::task::495::TaskManager.Task::(_debug) Task 81fcc1d9-f17b-46fe-b428-f989ad5ca468: moving from state preparing -> state finished
>Thread-15910838::DEBUG::2013-11-19 21:21:06,565::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910838::DEBUG::2013-11-19 21:21:06,565::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910838::DEBUG::2013-11-19 21:21:06,566::task::495::TaskManager.Task::(_debug) Task 81fcc1d9-f17b-46fe-b428-f989ad5ca468: ref 0 aborting False
>Thread-15910838::INFO::2013-11-19 21:21:06,566::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910839::DEBUG::2013-11-19 21:21:07,572::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910839::INFO::2013-11-19 21:21:07,573::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=934c4d35-aab6-4fba-9996-9cc07dd7d991)
>Thread-15910839::DEBUG::2013-11-19 21:21:07,573::task::495::TaskManager.Task::(_debug) Task 65f08743-8242-4360-868f-2f520a1b6134: moving from state init -> state preparing
>Thread-15910839::DEBUG::2013-11-19 21:21:07,574::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 934c4d35-aab6-4fba-9996-9cc07dd7d991
>Thread-15910839::DEBUG::2013-11-19 21:21:07,574::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}
>Thread-15910839::DEBUG::2013-11-19 21:21:07,575::task::495::TaskManager.Task::(_debug) Task 65f08743-8242-4360-868f-2f520a1b6134: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910839::DEBUG::2013-11-19 21:21:07,576::task::495::TaskManager.Task::(_debug) Task 65f08743-8242-4360-868f-2f520a1b6134: moving from state preparing -> state finished
>Thread-15910839::DEBUG::2013-11-19 21:21:07,576::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910839::DEBUG::2013-11-19 21:21:07,577::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910839::DEBUG::2013-11-19 21:21:07,577::task::495::TaskManager.Task::(_debug) Task 65f08743-8242-4360-868f-2f520a1b6134: ref 0 aborting False
>Thread-15910839::INFO::2013-11-19 21:21:07,578::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910841::DEBUG::2013-11-19 21:21:08,584::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910841::INFO::2013-11-19 21:21:08,584::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=934c4d35-aab6-4fba-9996-9cc07dd7d991)
>Thread-15910841::DEBUG::2013-11-19 21:21:08,585::task::495::TaskManager.Task::(_debug) Task c13be546-789b-4d28-9200-404e56a11585: moving from state init -> state preparing
>Thread-15910841::DEBUG::2013-11-19 21:21:08,585::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 934c4d35-aab6-4fba-9996-9cc07dd7d991
>Thread-15910841::DEBUG::2013-11-19 21:21:08,586::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}
>Thread-15910841::DEBUG::2013-11-19 21:21:08,587::task::495::TaskManager.Task::(_debug) Task c13be546-789b-4d28-9200-404e56a11585: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910841::DEBUG::2013-11-19 21:21:08,587::task::495::TaskManager.Task::(_debug) Task c13be546-789b-4d28-9200-404e56a11585: moving from state preparing -> state finished
>Thread-15910841::DEBUG::2013-11-19 21:21:08,588::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910841::DEBUG::2013-11-19 21:21:08,588::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910841::DEBUG::2013-11-19 21:21:08,589::task::495::TaskManager.Task::(_debug) Task c13be546-789b-4d28-9200-404e56a11585: ref 0 aborting False
>Thread-15910841::INFO::2013-11-19 21:21:08,589::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-1024767::ERROR::2013-11-19 21:21:09,432::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910842::DEBUG::2013-11-19 21:21:09,595::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910842::INFO::2013-11-19 21:21:09,605::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=934c4d35-aab6-4fba-9996-9cc07dd7d991)
>Thread-15910842::DEBUG::2013-11-19 21:21:09,606::task::495::TaskManager.Task::(_debug) Task a6b4799a-fcb5-4269-b313-e6f25a54a456: moving from state init -> state preparing
>Thread-15910842::DEBUG::2013-11-19 21:21:09,607::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 934c4d35-aab6-4fba-9996-9cc07dd7d991
>Thread-15910842::DEBUG::2013-11-19 21:21:09,607::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}
>Thread-15910842::DEBUG::2013-11-19 21:21:09,608::task::495::TaskManager.Task::(_debug) Task a6b4799a-fcb5-4269-b313-e6f25a54a456: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910842::DEBUG::2013-11-19 21:21:09,608::task::495::TaskManager.Task::(_debug) Task a6b4799a-fcb5-4269-b313-e6f25a54a456: moving from state preparing -> state finished
>Thread-15910842::DEBUG::2013-11-19 21:21:09,609::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910842::DEBUG::2013-11-19 21:21:09,609::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910842::DEBUG::2013-11-19 21:21:09,610::task::495::TaskManager.Task::(_debug) Task a6b4799a-fcb5-4269-b313-e6f25a54a456: ref 0 aborting False
>Thread-15910842::INFO::2013-11-19 21:21:09,610::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910844::DEBUG::2013-11-19 21:21:10,616::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910844::INFO::2013-11-19 21:21:10,617::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=934c4d35-aab6-4fba-9996-9cc07dd7d991)
>Thread-15910844::DEBUG::2013-11-19 21:21:10,617::task::495::TaskManager.Task::(_debug) Task a3602667-dfdb-49c2-baff-085b1cda1eb3: moving from state init -> state preparing
>Thread-15910844::DEBUG::2013-11-19 21:21:10,618::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 934c4d35-aab6-4fba-9996-9cc07dd7d991
>Thread-15910844::DEBUG::2013-11-19 21:21:10,618::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}
>Thread-15910844::DEBUG::2013-11-19 21:21:10,619::task::495::TaskManager.Task::(_debug) Task a3602667-dfdb-49c2-baff-085b1cda1eb3: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910844::DEBUG::2013-11-19 21:21:10,620::task::495::TaskManager.Task::(_debug) Task a3602667-dfdb-49c2-baff-085b1cda1eb3: moving from state preparing -> state finished
>Thread-15910844::DEBUG::2013-11-19 21:21:10,620::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910844::DEBUG::2013-11-19 21:21:10,621::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910844::DEBUG::2013-11-19 21:21:10,621::task::495::TaskManager.Task::(_debug) Task a3602667-dfdb-49c2-baff-085b1cda1eb3: ref 0 aborting False
>Thread-15910844::INFO::2013-11-19 21:21:10,622::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910845::DEBUG::2013-11-19 21:21:11,628::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910845::INFO::2013-11-19 21:21:11,629::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=934c4d35-aab6-4fba-9996-9cc07dd7d991)
>Thread-15910845::DEBUG::2013-11-19 21:21:11,629::task::495::TaskManager.Task::(_debug) Task 71fef9c5-5b25-4ec5-ac5d-f1264df35c06: moving from state init -> state preparing
>Thread-15910845::DEBUG::2013-11-19 21:21:11,630::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 934c4d35-aab6-4fba-9996-9cc07dd7d991
>Thread-15910845::DEBUG::2013-11-19 21:21:11,630::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}
>Thread-15910845::DEBUG::2013-11-19 21:21:11,631::task::495::TaskManager.Task::(_debug) Task 71fef9c5-5b25-4ec5-ac5d-f1264df35c06: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910845::DEBUG::2013-11-19 21:21:11,631::task::495::TaskManager.Task::(_debug) Task 71fef9c5-5b25-4ec5-ac5d-f1264df35c06: moving from state preparing -> state finished
>Thread-15910845::DEBUG::2013-11-19 21:21:11,632::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910845::DEBUG::2013-11-19 21:21:11,632::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910845::DEBUG::2013-11-19 21:21:11,633::task::495::TaskManager.Task::(_debug) Task 71fef9c5-5b25-4ec5-ac5d-f1264df35c06: ref 0 aborting False
>Thread-15910845::INFO::2013-11-19 21:21:11,633::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910846::INFO::2013-11-19 21:21:11,924::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910846::DEBUG::2013-11-19 21:21:11,924::task::495::TaskManager.Task::(_debug) Task 127b21cd-b650-4a72-8ea2-f5552996048f: moving from state init -> state preparing
>Thread-15910846::DEBUG::2013-11-19 21:21:11,925::task::495::TaskManager.Task::(_debug) Task 127b21cd-b650-4a72-8ea2-f5552996048f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000339984893799', 'lastCheck': 1384896063.6513791, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00150203704834', 'lastCheck': 1384896064.4249959, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896069.4335749, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000384092330933', 'lastCheck': 1384896069.075789, 'valid': True, 'code': 0}}
>Thread-15910846::DEBUG::2013-11-19 21:21:11,926::task::495::TaskManager.Task::(_debug) Task 127b21cd-b650-4a72-8ea2-f5552996048f: moving from state preparing -> state finished
>Thread-15910846::DEBUG::2013-11-19 21:21:11,926::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910846::DEBUG::2013-11-19 21:21:11,927::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910846::DEBUG::2013-11-19 21:21:11,927::task::495::TaskManager.Task::(_debug) Task 127b21cd-b650-4a72-8ea2-f5552996048f: ref 0 aborting False
>Thread-15910846::INFO::2013-11-19 21:21:11,928::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000339984893799', 'lastCheck': 1384896063.6513791, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00150203704834', 'lastCheck': 1384896064.4249959, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896069.4335749, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000384092330933', 'lastCheck': 1384896069.075789, 'valid': True, 'code': 0}}
>Thread-15910848::DEBUG::2013-11-19 21:21:12,639::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910848::INFO::2013-11-19 21:21:12,640::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=934c4d35-aab6-4fba-9996-9cc07dd7d991)
>Thread-15910848::DEBUG::2013-11-19 21:21:12,641::task::495::TaskManager.Task::(_debug) Task e6f0a08d-81b7-4ae5-95d3-cc992b50e941: moving from state init -> state preparing
>Thread-15910848::DEBUG::2013-11-19 21:21:12,641::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 934c4d35-aab6-4fba-9996-9cc07dd7d991
>Thread-15910848::DEBUG::2013-11-19 21:21:12,642::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}
>Thread-15910848::DEBUG::2013-11-19 21:21:12,642::task::495::TaskManager.Task::(_debug) Task e6f0a08d-81b7-4ae5-95d3-cc992b50e941: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910848::DEBUG::2013-11-19 21:21:12,643::task::495::TaskManager.Task::(_debug) Task e6f0a08d-81b7-4ae5-95d3-cc992b50e941: moving from state preparing -> state finished
>Thread-15910848::DEBUG::2013-11-19 21:21:12,643::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910848::DEBUG::2013-11-19 21:21:12,644::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910848::DEBUG::2013-11-19 21:21:12,644::task::495::TaskManager.Task::(_debug) Task e6f0a08d-81b7-4ae5-95d3-cc992b50e941: ref 0 aborting False
>Thread-15910848::INFO::2013-11-19 21:21:12,645::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910849::DEBUG::2013-11-19 21:21:13,651::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910849::INFO::2013-11-19 21:21:13,651::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=934c4d35-aab6-4fba-9996-9cc07dd7d991)
>Thread-15910849::DEBUG::2013-11-19 21:21:13,652::task::495::TaskManager.Task::(_debug) Task ed8e0f56-ff8a-46fc-bdbf-82605d433e45: moving from state init -> state preparing
>Thread-15910849::DEBUG::2013-11-19 21:21:13,653::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 934c4d35-aab6-4fba-9996-9cc07dd7d991
>Thread-15910849::DEBUG::2013-11-19 21:21:13,653::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}
>Thread-15910849::DEBUG::2013-11-19 21:21:13,654::task::495::TaskManager.Task::(_debug) Task ed8e0f56-ff8a-46fc-bdbf-82605d433e45: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910849::DEBUG::2013-11-19 21:21:13,654::task::495::TaskManager.Task::(_debug) Task ed8e0f56-ff8a-46fc-bdbf-82605d433e45: moving from state preparing -> state finished
>Thread-15910849::DEBUG::2013-11-19 21:21:13,655::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910849::DEBUG::2013-11-19 21:21:13,655::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910849::DEBUG::2013-11-19 21:21:13,656::task::495::TaskManager.Task::(_debug) Task ed8e0f56-ff8a-46fc-bdbf-82605d433e45: ref 0 aborting False
>Thread-15910849::INFO::2013-11-19 21:21:13,656::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910851::DEBUG::2013-11-19 21:21:14,662::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910851::INFO::2013-11-19 21:21:14,663::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=934c4d35-aab6-4fba-9996-9cc07dd7d991)
>Thread-15910851::DEBUG::2013-11-19 21:21:14,671::task::495::TaskManager.Task::(_debug) Task 8ddee017-4e5f-4b13-8426-6455023a6a52: moving from state init -> state preparing
>Thread-15910851::DEBUG::2013-11-19 21:21:14,671::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 934c4d35-aab6-4fba-9996-9cc07dd7d991
>Thread-15910851::DEBUG::2013-11-19 21:21:14,672::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}
>Thread-15910851::DEBUG::2013-11-19 21:21:14,672::task::495::TaskManager.Task::(_debug) Task 8ddee017-4e5f-4b13-8426-6455023a6a52: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910851::DEBUG::2013-11-19 21:21:14,673::task::495::TaskManager.Task::(_debug) Task 8ddee017-4e5f-4b13-8426-6455023a6a52: moving from state preparing -> state finished
>Thread-15910851::DEBUG::2013-11-19 21:21:14,673::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910851::DEBUG::2013-11-19 21:21:14,674::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910851::DEBUG::2013-11-19 21:21:14,674::task::495::TaskManager.Task::(_debug) Task 8ddee017-4e5f-4b13-8426-6455023a6a52: ref 0 aborting False
>Thread-15910851::INFO::2013-11-19 21:21:14,675::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910852::DEBUG::2013-11-19 21:21:15,681::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910852::INFO::2013-11-19 21:21:15,682::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=934c4d35-aab6-4fba-9996-9cc07dd7d991)
>Thread-15910852::DEBUG::2013-11-19 21:21:15,682::task::495::TaskManager.Task::(_debug) Task 75905a5c-ec67-459a-bc6a-e7693355faa5: moving from state init -> state preparing
>Thread-15910852::DEBUG::2013-11-19 21:21:15,683::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 934c4d35-aab6-4fba-9996-9cc07dd7d991
>Thread-15910852::DEBUG::2013-11-19 21:21:15,683::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}
>Thread-15910852::DEBUG::2013-11-19 21:21:15,684::task::495::TaskManager.Task::(_debug) Task 75905a5c-ec67-459a-bc6a-e7693355faa5: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910852::DEBUG::2013-11-19 21:21:15,684::task::495::TaskManager.Task::(_debug) Task 75905a5c-ec67-459a-bc6a-e7693355faa5: moving from state preparing -> state finished
>Thread-15910852::DEBUG::2013-11-19 21:21:15,685::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910852::DEBUG::2013-11-19 21:21:15,685::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910852::DEBUG::2013-11-19 21:21:15,686::task::495::TaskManager.Task::(_debug) Task 75905a5c-ec67-459a-bc6a-e7693355faa5: ref 0 aborting False
>Thread-15910852::INFO::2013-11-19 21:21:15,687::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910854::DEBUG::2013-11-19 21:21:16,692::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910854::INFO::2013-11-19 21:21:16,693::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=934c4d35-aab6-4fba-9996-9cc07dd7d991)
>Thread-15910854::DEBUG::2013-11-19 21:21:16,694::task::495::TaskManager.Task::(_debug) Task 9bf03bef-4319-4f3d-9f05-8b6accf80b0b: moving from state init -> state preparing
>Thread-15910854::DEBUG::2013-11-19 21:21:16,694::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 934c4d35-aab6-4fba-9996-9cc07dd7d991
>Thread-15910854::DEBUG::2013-11-19 21:21:16,695::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}
>Thread-15910854::DEBUG::2013-11-19 21:21:16,695::task::495::TaskManager.Task::(_debug) Task 9bf03bef-4319-4f3d-9f05-8b6accf80b0b: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910854::DEBUG::2013-11-19 21:21:16,696::task::495::TaskManager.Task::(_debug) Task 9bf03bef-4319-4f3d-9f05-8b6accf80b0b: moving from state preparing -> state finished
>Thread-15910854::DEBUG::2013-11-19 21:21:16,696::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910854::DEBUG::2013-11-19 21:21:16,697::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910854::DEBUG::2013-11-19 21:21:16,697::task::495::TaskManager.Task::(_debug) Task 9bf03bef-4319-4f3d-9f05-8b6accf80b0b: ref 0 aborting False
>Thread-15910854::INFO::2013-11-19 21:21:16,698::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910855::DEBUG::2013-11-19 21:21:17,704::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910855::INFO::2013-11-19 21:21:17,705::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=934c4d35-aab6-4fba-9996-9cc07dd7d991)
>Thread-15910855::DEBUG::2013-11-19 21:21:17,707::task::495::TaskManager.Task::(_debug) Task 9350c3ed-1fb6-43e4-ae60-f6899f296482: moving from state init -> state preparing
>Thread-15910855::DEBUG::2013-11-19 21:21:17,708::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 934c4d35-aab6-4fba-9996-9cc07dd7d991
>Thread-15910855::DEBUG::2013-11-19 21:21:17,708::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}
>Thread-15910855::DEBUG::2013-11-19 21:21:17,709::task::495::TaskManager.Task::(_debug) Task 9350c3ed-1fb6-43e4-ae60-f6899f296482: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910855::DEBUG::2013-11-19 21:21:17,710::task::495::TaskManager.Task::(_debug) Task 9350c3ed-1fb6-43e4-ae60-f6899f296482: moving from state preparing -> state finished
>Thread-15910855::DEBUG::2013-11-19 21:21:17,712::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910855::DEBUG::2013-11-19 21:21:17,712::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910855::DEBUG::2013-11-19 21:21:17,713::task::495::TaskManager.Task::(_debug) Task 9350c3ed-1fb6-43e4-ae60-f6899f296482: ref 0 aborting False
>Thread-15910855::INFO::2013-11-19 21:21:17,714::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910857::DEBUG::2013-11-19 21:21:18,719::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910857::INFO::2013-11-19 21:21:18,720::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=934c4d35-aab6-4fba-9996-9cc07dd7d991)
>Thread-15910857::DEBUG::2013-11-19 21:21:18,721::task::495::TaskManager.Task::(_debug) Task eba498f3-84a0-46f7-87a1-201ce1e5cf24: moving from state init -> state preparing
>Thread-15910857::DEBUG::2013-11-19 21:21:18,721::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 934c4d35-aab6-4fba-9996-9cc07dd7d991
>Thread-15910857::DEBUG::2013-11-19 21:21:18,722::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}
>Thread-15910857::DEBUG::2013-11-19 21:21:18,722::task::495::TaskManager.Task::(_debug) Task eba498f3-84a0-46f7-87a1-201ce1e5cf24: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910857::DEBUG::2013-11-19 21:21:18,723::task::495::TaskManager.Task::(_debug) Task eba498f3-84a0-46f7-87a1-201ce1e5cf24: moving from state preparing -> state finished
>Thread-15910857::DEBUG::2013-11-19 21:21:18,724::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910857::DEBUG::2013-11-19 21:21:18,724::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910857::DEBUG::2013-11-19 21:21:18,725::task::495::TaskManager.Task::(_debug) Task eba498f3-84a0-46f7-87a1-201ce1e5cf24: ref 0 aborting False
>Thread-15910857::INFO::2013-11-19 21:21:18,725::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:19,281::safelease::61::Storage.Misc.excCmd::(acquire) SUCCESS: <err> = ''; <rc> = 0
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:19,281::safelease::65::ClusterLock::(acquire) Clustered lock acquired successfully
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:19,282::spm::468::Storage.SPM::(start) spm lock acquired successfully
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:19,283::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:19,283::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:19,284::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:19,285::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:19,285::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:19,286::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:21:19,506::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910858::DEBUG::2013-11-19 21:21:19,732::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910858::INFO::2013-11-19 21:21:19,733::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=934c4d35-aab6-4fba-9996-9cc07dd7d991)
>Thread-15910858::DEBUG::2013-11-19 21:21:19,734::task::495::TaskManager.Task::(_debug) Task c7f3110f-d7e4-4fa3-9b8e-36462ddde7b2: moving from state init -> state preparing
>Thread-15910858::DEBUG::2013-11-19 21:21:19,734::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 934c4d35-aab6-4fba-9996-9cc07dd7d991
>Thread-15910858::DEBUG::2013-11-19 21:21:19,735::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}
>Thread-15910858::DEBUG::2013-11-19 21:21:19,736::task::495::TaskManager.Task::(_debug) Task c7f3110f-d7e4-4fa3-9b8e-36462ddde7b2: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910858::DEBUG::2013-11-19 21:21:19,736::task::495::TaskManager.Task::(_debug) Task c7f3110f-d7e4-4fa3-9b8e-36462ddde7b2: moving from state preparing -> state finished
>Thread-15910858::DEBUG::2013-11-19 21:21:19,737::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910858::DEBUG::2013-11-19 21:21:19,737::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910858::DEBUG::2013-11-19 21:21:19,738::task::495::TaskManager.Task::(_debug) Task c7f3110f-d7e4-4fa3-9b8e-36462ddde7b2: ref 0 aborting False
>Thread-15910858::INFO::2013-11-19 21:21:19,738::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:19,924::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:19,927::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:19,928::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_LVER=279', u'_SHA_CKSUM=b5dee775aac07995010b8c6a5db186617df84cf4', u'POOL_SPM_ID=-1']
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:19,929::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:19,930::persistentDict::146::Storage.PersistentDict::(transaction) Flushing changes
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:19,930::persistentDict::265::Storage.PersistentDict::(flush) about to write lines (VGTagMetadataRW)=[u'CLASS=Data', u'DESCRIPTION=OS', u'IOOPTIMEOUTSEC=10', u'LEASERETRIES=3', u'LEASETIMESEC=60', u'LOCKPOLICY=', u'LOCKRENEWALINTERVALSEC=5', u'LOGBLKSIZE=512', u'MASTER_VERSION=1', u'PHYBLKSIZE=512', u'POOL_DESCRIPTION=Default', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_ID=2', u'POOL_SPM_LVER=280', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'ROLE=Master', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'TYPE=FCP', u'VERSION=2', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', '_SHA_CKSUM=215540c631d1cfd943bc1c89ef09ec9e3251e9bd']
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:19,931::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:19,932::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:19,932::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:19,933::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:19,934::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:19,935::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:20,584::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:20,587::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:20,588::blockSD::145::storage.Metadata.VGTagMetadataRW::(writelines) Updating metadata adding=MDT__SHA_CKSUM=215540c631d1cfd943bc1c89ef09ec9e3251e9bd, MDT_POOL_SPM_LVER=280, MDT_POOL_SPM_ID=2 removing=MDT_POOL_SPM_ID=-1, MDT__SHA_CKSUM=b5dee775aac07995010b8c6a5db186617df84cf4, MDT_POOL_SPM_LVER=279
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:20,590::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --deltag MDT__SHA_CKSUM=b5dee775aac07995010b8c6a5db186617df84cf4 --deltag MDT_POOL_SPM_LVER=279 --deltag MDT_POOL_SPM_ID=-1 --addtag MDT_POOL_SPM_LVER=280 --addtag MDT__SHA_CKSUM=215540c631d1cfd943bc1c89ef09ec9e3251e9bd --addtag MDT_POOL_SPM_ID=2 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15910860::DEBUG::2013-11-19 21:21:20,812::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910860::INFO::2013-11-19 21:21:20,814::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=934c4d35-aab6-4fba-9996-9cc07dd7d991)
>Thread-15910860::DEBUG::2013-11-19 21:21:20,815::task::495::TaskManager.Task::(_debug) Task d16c47e8-705d-4004-a4d5-061fb071ccc7: moving from state init -> state preparing
>Thread-15910860::DEBUG::2013-11-19 21:21:20,816::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 934c4d35-aab6-4fba-9996-9cc07dd7d991
>Thread-15910860::DEBUG::2013-11-19 21:21:20,816::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}
>Thread-15910860::DEBUG::2013-11-19 21:21:20,817::task::495::TaskManager.Task::(_debug) Task d16c47e8-705d-4004-a4d5-061fb071ccc7: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910860::DEBUG::2013-11-19 21:21:20,818::task::495::TaskManager.Task::(_debug) Task d16c47e8-705d-4004-a4d5-061fb071ccc7: moving from state preparing -> state finished
>Thread-15910860::DEBUG::2013-11-19 21:21:20,818::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910860::DEBUG::2013-11-19 21:21:20,819::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910860::DEBUG::2013-11-19 21:21:20,819::task::495::TaskManager.Task::(_debug) Task d16c47e8-705d-4004-a4d5-061fb071ccc7: ref 0 aborting False
>Thread-15910860::INFO::2013-11-19 21:21:20,820::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:21,220::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:21,221::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:21,222::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:21,223::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:21,224::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f39bd185-4535-444c-bdd3-e7744f66c881`::Request was made in '/usr/share/vdsm/storage/spm.py' line '330' at '_upgradePool'
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:21,225::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'exclusive'
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:21,225::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'exclusive' (1 active user)
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:21,226::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f39bd185-4535-444c-bdd3-e7744f66c881`::Granted request
>934c4d35-aab6-4fba-9996-9cc07dd7d991::INFO::2013-11-19 21:21:21,227::spm::336::Storage.SPM::(_upgradePool) Trying to upgrade master domain `6b892067-ac7e-4039-a7be-5b7387525b16`
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:21,228::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`38d6bcc4-c986-47d8-a5e3-25f05bb46151`::Request was made in '/usr/share/vdsm/storage/spm.py' line '337' at '_upgradePool'
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:21,228::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'exclusive'
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:21,229::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'exclusive' (1 active user)
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:21,229::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`38d6bcc4-c986-47d8-a5e3-25f05bb46151`::Granted request
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:21,230::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:21,231::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:21,232::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:21,232::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:21,233::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:21,234::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:21,813::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:21,817::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:21,818::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_LVER=280', u'_SHA_CKSUM=215540c631d1cfd943bc1c89ef09ec9e3251e9bd', u'POOL_SPM_ID=2']
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:21,819::blockSD::625::Storage.StorageDomain::(upgrade) Trying to upgrade domain `6b892067-ac7e-4039-a7be-5b7387525b16` from version 2 to version 2
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:21,819::blockSD::631::Storage.StorageDomain::(upgrade) No need to upgrade domain `6b892067-ac7e-4039-a7be-5b7387525b16`, leaving unchanged
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:21,820::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:21,821::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:21,821::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:21,822::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:21,822::spm::340::Storage.SPM::(_upgradePool) Marking all domains for upgrade
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:21,823::spm::347::Storage.SPM::(_upgradePool) Registering with state change event
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:21,824::spm::349::Storage.SPM::(_upgradePool) Running initial domain upgrade threads
>Thread-15910861::DEBUG::2013-11-19 21:21:21,825::spm::365::Storage.SPM::(_upgradePoolDomain) Preparing to upgrade domain 45cdff8b-89c0-4926-a8c7-80354064bfa6
>Thread-15910863::DEBUG::2013-11-19 21:21:21,827::spm::365::Storage.SPM::(_upgradePoolDomain) Preparing to upgrade domain 772472cb-2429-466d-96cd-00b3301ef78a
>Thread-15910862::DEBUG::2013-11-19 21:21:21,830::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910864::DEBUG::2013-11-19 21:21:21,831::spm::365::Storage.SPM::(_upgradePoolDomain) Preparing to upgrade domain 5d7b5178-3790-472c-a13b-b481eb27a61e
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:21,832::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15910864::ERROR::2013-11-19 21:21:21,841::spm::373::Storage.SPM::(_upgradePoolDomain) Error while preparing domain `5d7b5178-3790-472c-a13b-b481eb27a61e` upgrade
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/spm.py", line 369, in _upgradePoolDomain
>IndexError: list index out of range
>Thread-15910863::DEBUG::2013-11-19 21:21:21,843::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2d7a3930-4fae-4578-beba-389d757607b1`::Request was made in '/usr/share/vdsm/storage/spm.py' line '376' at '_upgradePoolDomain'
>Thread-15910862::INFO::2013-11-19 21:21:21,843::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=934c4d35-aab6-4fba-9996-9cc07dd7d991)
>Thread-15910863::DEBUG::2013-11-19 21:21:21,851::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'exclusive'
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:21,852::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15910862::DEBUG::2013-11-19 21:21:21,852::task::495::TaskManager.Task::(_debug) Task c41982af-17ac-434d-a324-37cd95930140: moving from state init -> state preparing
>Thread-15910861::DEBUG::2013-11-19 21:21:21,854::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6`ReqID=`0ac4c005-ee7d-4a90-b20c-f95d924b4099`::Request was made in '/usr/share/vdsm/storage/spm.py' line '376' at '_upgradePoolDomain'
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:21,854::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15910862::DEBUG::2013-11-19 21:21:21,855::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 934c4d35-aab6-4fba-9996-9cc07dd7d991
>Thread-15910861::DEBUG::2013-11-19 21:21:21,855::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6' for lock type 'exclusive'
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:21,856::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15910862::DEBUG::2013-11-19 21:21:21,856::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}
>Thread-15910863::DEBUG::2013-11-19 21:21:21,857::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'exclusive' (1 active user)
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:21,858::blockSD::791::Storage.Misc.excCmd::(mountMaster) '/usr/bin/sudo -n /sbin/e2fsck -p /dev/6b892067-ac7e-4039-a7be-5b7387525b16/master' (cwd None)
>Thread-15910862::DEBUG::2013-11-19 21:21:21,859::task::495::TaskManager.Task::(_debug) Task c41982af-17ac-434d-a324-37cd95930140: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910863::DEBUG::2013-11-19 21:21:21,859::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2d7a3930-4fae-4578-beba-389d757607b1`::Granted request
>Thread-15910862::DEBUG::2013-11-19 21:21:21,861::task::495::TaskManager.Task::(_debug) Task c41982af-17ac-434d-a324-37cd95930140: moving from state preparing -> state finished
>Thread-15910861::DEBUG::2013-11-19 21:21:21,862::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6' is free. Now locking as 'exclusive' (1 active user)
>Thread-15910862::DEBUG::2013-11-19 21:21:21,863::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910863::DEBUG::2013-11-19 21:21:22,065::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`62e8e300-bc1a-48cf-8a2c-10534928ef5e`::Request was made in '/usr/share/vdsm/storage/spm.py' line '377' at '_upgradePoolDomain'
>Thread-15910861::DEBUG::2013-11-19 21:21:22,067::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6`ReqID=`0ac4c005-ee7d-4a90-b20c-f95d924b4099`::Granted request
>Thread-15910862::DEBUG::2013-11-19 21:21:22,068::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910863::DEBUG::2013-11-19 21:21:22,069::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'exclusive'
>Thread-15910862::DEBUG::2013-11-19 21:21:22,072::task::495::TaskManager.Task::(_debug) Task c41982af-17ac-434d-a324-37cd95930140: ref 0 aborting False
>Thread-15910862::INFO::2013-11-19 21:21:22,074::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910861::DEBUG::2013-11-19 21:21:22,077::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6`ReqID=`db93c7a6-5b28-4ce1-b14c-b9177c970d7a`::Request was made in '/usr/share/vdsm/storage/spm.py' line '377' at '_upgradePoolDomain'
>Thread-15910863::DEBUG::2013-11-19 21:21:22,078::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'exclusive' (1 active user)
>Thread-15910861::DEBUG::2013-11-19 21:21:22,079::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6' for lock type 'exclusive'
>Thread-15910863::DEBUG::2013-11-19 21:21:22,081::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`62e8e300-bc1a-48cf-8a2c-10534928ef5e`::Granted request
>Thread-15910861::DEBUG::2013-11-19 21:21:22,083::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6' is free. Now locking as 'exclusive' (1 active user)
>Thread-15910863::DEBUG::2013-11-19 21:21:22,084::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15910861::DEBUG::2013-11-19 21:21:22,085::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6`ReqID=`db93c7a6-5b28-4ce1-b14c-b9177c970d7a`::Granted request
>Thread-15910863::DEBUG::2013-11-19 21:21:22,086::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15910861::DEBUG::2013-11-19 21:21:22,087::spm::385::Storage.SPM::(_upgradePoolDomain) Domain `45cdff8b-89c0-4926-a8c7-80354064bfa6` is not a data domain it is an 2 domain, not upgrading
>Thread-15910863::DEBUG::2013-11-19 21:21:22,088::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15910861::DEBUG::2013-11-19 21:21:22,089::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6'
>Thread-15910863::DEBUG::2013-11-19 21:21:22,089::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15910861::DEBUG::2013-11-19 21:21:22,090::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6' (0 active users)
>Thread-15910863::DEBUG::2013-11-19 21:21:22,091::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-15910861::DEBUG::2013-11-19 21:21:22,092::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6' is free, finding out if anyone is waiting for it.
>Thread-15910863::DEBUG::2013-11-19 21:21:22,093::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-15910861::DEBUG::2013-11-19 21:21:22,094::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6', Clearing records.
>Thread-15910861::DEBUG::2013-11-19 21:21:22,095::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6'
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:22,355::blockSD::791::Storage.Misc.excCmd::(mountMaster) SUCCESS: <err> = ''; <rc> = 0
>Thread-15910861::DEBUG::2013-11-19 21:21:22,357::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6' (0 active users)
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:22,358::blockSD::816::Storage.Misc.excCmd::(mountMaster) '/usr/bin/sudo -n /sbin/tune2fs -j /dev/6b892067-ac7e-4039-a7be-5b7387525b16/master' (cwd None)
>Thread-15910865::INFO::2013-11-19 21:21:22,363::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910861::DEBUG::2013-11-19 21:21:22,364::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6' is free, finding out if anyone is waiting for it.
>Thread-15910865::DEBUG::2013-11-19 21:21:22,364::task::495::TaskManager.Task::(_debug) Task cba94721-251d-4191-9da3-55bd755ba0bf: moving from state init -> state preparing
>Thread-15910861::DEBUG::2013-11-19 21:21:22,714::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6', Clearing records.
>Thread-15910865::DEBUG::2013-11-19 21:21:22,716::task::495::TaskManager.Task::(_debug) Task cba94721-251d-4191-9da3-55bd755ba0bf: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000332117080688', 'lastCheck': 1384896073.6639979, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00149703025818', 'lastCheck': 1384896074.440325, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896079.507699, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000433921813965', 'lastCheck': 1384896079.087245, 'valid': True, 'code': 0}}
>Thread-15910865::DEBUG::2013-11-19 21:21:22,718::task::495::TaskManager.Task::(_debug) Task cba94721-251d-4191-9da3-55bd755ba0bf: moving from state preparing -> state finished
>Thread-15910865::DEBUG::2013-11-19 21:21:22,720::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910865::DEBUG::2013-11-19 21:21:22,721::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910865::DEBUG::2013-11-19 21:21:22,722::task::495::TaskManager.Task::(_debug) Task cba94721-251d-4191-9da3-55bd755ba0bf: ref 0 aborting False
>Thread-15910865::INFO::2013-11-19 21:21:22,723::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000332117080688', 'lastCheck': 1384896073.6639979, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00149703025818', 'lastCheck': 1384896074.440325, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896079.507699, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000433921813965', 'lastCheck': 1384896079.087245, 'valid': True, 'code': 0}}
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:22,912::blockSD::816::Storage.Misc.excCmd::(mountMaster) FAILED: <err> = 'The filesystem already has a journal.\n'; <rc> = 1
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:22,914::fileUtils::83::Storage.Misc.excCmd::(mount) '/usr/bin/sudo -n /bin/mount /dev/6b892067-ac7e-4039-a7be-5b7387525b16/master /rhev/data-center/mnt/blockSD/6b892067-ac7e-4039-a7be-5b7387525b16/master' (cwd None)
>Thread-15910863::DEBUG::2013-11-19 21:21:23,134::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15910867::DEBUG::2013-11-19 21:21:23,137::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910863::DEBUG::2013-11-19 21:21:23,142::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-15910867::INFO::2013-11-19 21:21:23,143::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=934c4d35-aab6-4fba-9996-9cc07dd7d991)
>Thread-15910863::DEBUG::2013-11-19 21:21:23,144::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'SDUUID=772472cb-2429-466d-96cd-00b3301ef78a', u'VERSION=2', u'LOCKPOLICY=', u'VGUUID=XTb294-NYv6-dVpO-bB1n-4f9P-d0GL-dhd2DI', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'LEASETIMESEC=5', u'LOCKRENEWALINTERVALSEC=5', u'ROLE=Regular', u'PHYBLKSIZE=512', u'DESCRIPTION=Database', u'IOOPTIMEOUTSEC=1', u'CLASS=Data', u'TYPE=FCP', u'PV0=pv:3600c0ff0001520c7414f605001000000,uuid:NujM33-vNiD-sBHY-mrPo-EQ2f-1qsz-HXfn0q,pestart:0,pecount:8929,mapoffset:0', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'MASTER_VERSION=0', u'_SHA_CKSUM=ef8840d9fdef24516a56fd970bed1960d2b1ae92']
>Thread-15910867::DEBUG::2013-11-19 21:21:23,145::task::495::TaskManager.Task::(_debug) Task 45951473-db59-4fb6-bb10-bdb5f17e0a2a: moving from state init -> state preparing
>Thread-15910863::DEBUG::2013-11-19 21:21:23,147::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15910867::DEBUG::2013-11-19 21:21:23,148::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 934c4d35-aab6-4fba-9996-9cc07dd7d991
>Thread-15910867::DEBUG::2013-11-19 21:21:23,149::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}
>Thread-15910863::DEBUG::2013-11-19 21:21:23,150::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15910867::DEBUG::2013-11-19 21:21:23,150::task::495::TaskManager.Task::(_debug) Task 45951473-db59-4fb6-bb10-bdb5f17e0a2a: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910863::DEBUG::2013-11-19 21:21:23,151::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15910867::DEBUG::2013-11-19 21:21:23,152::task::495::TaskManager.Task::(_debug) Task 45951473-db59-4fb6-bb10-bdb5f17e0a2a: moving from state preparing -> state finished
>Thread-15910863::DEBUG::2013-11-19 21:21:23,152::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15910867::DEBUG::2013-11-19 21:21:23,153::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910863::DEBUG::2013-11-19 21:21:23,154::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15910867::DEBUG::2013-11-19 21:21:23,155::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910863::DEBUG::2013-11-19 21:21:23,156::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15910867::DEBUG::2013-11-19 21:21:23,157::task::495::TaskManager.Task::(_debug) Task 45951473-db59-4fb6-bb10-bdb5f17e0a2a: ref 0 aborting False
>Thread-15910863::DEBUG::2013-11-19 21:21:23,157::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15910867::INFO::2013-11-19 21:21:23,158::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910863::DEBUG::2013-11-19 21:21:23,158::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,343::fileUtils::83::Storage.Misc.excCmd::(mount) SUCCESS: <err> = ''; <rc> = 0
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,344::blockSD::833::Storage.Misc.excCmd::(mountMaster) '/usr/bin/sudo -n /bin/chown vdsm:kvm /rhev/data-center/mnt/blockSD/6b892067-ac7e-4039-a7be-5b7387525b16/master' (cwd None)
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,746::blockSD::833::Storage.Misc.excCmd::(mountMaster) SUCCESS: <err> = ''; <rc> = 0
>934c4d35-aab6-4fba-9996-9cc07dd7d991::WARNING::2013-11-19 21:21:23,749::spm::490::Storage.SPM::(start) Backup domain validation failed
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/spm.py", line 488, in start
>  File "/usr/share/vdsm/storage/sp.py", line 1208, in checkBackupDomain
>  File "/usr/share/vdsm/storage/sd.py", line 666, in isBackup
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,750::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,751::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,752::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,752::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,753::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,754::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,754::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,755::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,755::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,756::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,756::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,757::persistentDict::146::Storage.PersistentDict::(transaction) Flushing changes
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,757::persistentDict::265::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['MASTER_VERSION=0', 'ROLE=Regular', '_SHA_CKSUM=1deb73127f89496e2cb17e51bbfae1df6c1879d1']
>934c4d35-aab6-4fba-9996-9cc07dd7d991::WARNING::2013-11-19 21:21:23,758::persistentDict::150::Storage.PersistentDict::(transaction) Error in transaction, rolling back changes
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/persistentDict.py", line 147, in transaction
>  File "/usr/share/vdsm/storage/persistentDict.py", line 266, in flush
>  File "/usr/share/vdsm/storage/fileSD.py", line 58, in writelines
>  File "/usr/share/vdsm/storage/processPool.py", line 33, in wrapper
>  File "/usr/share/vdsm/storage/processPool.py", line 44, in runExternally
>  File "/usr/share/vdsm/storage/processPool.py", line 127, in runExternally
>IOError: [Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata.new'
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,759::persistentDict::265::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['_SHA_CKSUM=da39a3ee5e6b4b0d3255bfef95601890afd80709']
>934c4d35-aab6-4fba-9996-9cc07dd7d991::ERROR::2013-11-19 21:21:23,760::spm::526::Storage.SPM::(start) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/spm.py", line 493, in start
>  File "/usr/share/vdsm/storage/sp.py", line 637, in _regulariseOutdatedMasters
>  File "/usr/share/vdsm/storage/sp.py", line 631, in _setRegularSD
>  File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 152, in transaction
>  File "/usr/share/vdsm/storage/persistentDict.py", line 266, in flush
>  File "/usr/share/vdsm/storage/fileSD.py", line 58, in writelines
>  File "/usr/share/vdsm/storage/processPool.py", line 33, in wrapper
>  File "/usr/share/vdsm/storage/processPool.py", line 44, in runExternally
>  File "/usr/share/vdsm/storage/processPool.py", line 127, in runExternally
>IOError: [Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata.new'
>934c4d35-aab6-4fba-9996-9cc07dd7d991::ERROR::2013-11-19 21:21:23,761::spm::527::Storage.SPM::(start) failed: [Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata.new'
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,762::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`84330296-faf6-4d34-af3e-638a082fbdb4`::Request was made in '/usr/share/vdsm/storage/spm.py' line '584' at '_stop'
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,763::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'exclusive'
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,763::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'exclusive' (1 active user)
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,764::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`84330296-faf6-4d34-af3e-638a082fbdb4`::Granted request
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,765::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e`ReqID=`b2fb8e7d-7020-469a-9cc1-2d350516713b`::Request was made in '/usr/share/vdsm/storage/spm.py' line '601' at '_stop'
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,766::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e' for lock type 'exclusive'
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,766::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e' is free. Now locking as 'exclusive' (1 active user)
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,767::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e`ReqID=`b2fb8e7d-7020-469a-9cc1-2d350516713b`::Granted request
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,768::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e'
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,769::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e' (0 active users)
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,769::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e' is free, finding out if anyone is waiting for it.
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,770::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e', Clearing records.
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,770::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30'
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,771::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,771::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,772::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,773::taskManager::118::TaskManager::(unloadTasks) Entry.
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,773::taskManager::122::TaskManager::(unloadTasks) Return
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,775::spm::195::Storage.SPM::(__cleanupMasterMount) unmounting /rhev/data-center/mnt/blockSD/6b892067-ac7e-4039-a7be-5b7387525b16/master
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:23,907::fileUtils::112::Storage.Misc.excCmd::(umount) '/usr/bin/sudo -n /bin/umount -f /rhev/data-center/mnt/blockSD/6b892067-ac7e-4039-a7be-5b7387525b16/master' (cwd None)
>Thread-15910868::DEBUG::2013-11-19 21:21:24,168::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910868::INFO::2013-11-19 21:21:24,170::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=934c4d35-aab6-4fba-9996-9cc07dd7d991)
>Thread-15910868::DEBUG::2013-11-19 21:21:24,171::task::495::TaskManager.Task::(_debug) Task 10e2f76d-7dd0-468e-8bb5-3a615e3a7915: moving from state init -> state preparing
>Thread-15910868::DEBUG::2013-11-19 21:21:24,171::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 934c4d35-aab6-4fba-9996-9cc07dd7d991
>Thread-15910868::DEBUG::2013-11-19 21:21:24,172::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}
>Thread-15910868::DEBUG::2013-11-19 21:21:24,173::task::495::TaskManager.Task::(_debug) Task 10e2f76d-7dd0-468e-8bb5-3a615e3a7915: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910868::DEBUG::2013-11-19 21:21:24,173::task::495::TaskManager.Task::(_debug) Task 10e2f76d-7dd0-468e-8bb5-3a615e3a7915: moving from state preparing -> state finished
>Thread-15910868::DEBUG::2013-11-19 21:21:24,174::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910868::DEBUG::2013-11-19 21:21:24,174::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910868::DEBUG::2013-11-19 21:21:24,175::task::495::TaskManager.Task::(_debug) Task 10e2f76d-7dd0-468e-8bb5-3a615e3a7915: ref 0 aborting False
>Thread-15910868::INFO::2013-11-19 21:21:24,176::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:24,301::fileUtils::112::Storage.Misc.excCmd::(umount) SUCCESS: <err> = ''; <rc> = 0
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:24,304::spm::290::Storage.SPM::(__cleanupSPM) cleaning up SPM: c4e5889a-348d-11e2-83fe-9f72369eab30
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:24,305::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:24,305::persistentDict::146::Storage.PersistentDict::(transaction) Flushing changes
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:24,306::persistentDict::265::Storage.PersistentDict::(flush) about to write lines (VGTagMetadataRW)=[u'CLASS=Data', u'DESCRIPTION=OS', u'IOOPTIMEOUTSEC=10', u'LEASERETRIES=3', u'LEASETIMESEC=60', u'LOCKPOLICY=', u'LOCKRENEWALINTERVALSEC=5', u'LOGBLKSIZE=512', u'MASTER_VERSION=1', u'PHYBLKSIZE=512', u'POOL_DESCRIPTION=Default', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_ID=-1', u'POOL_SPM_LVER=280', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'ROLE=Master', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'TYPE=FCP', u'VERSION=2', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', '_SHA_CKSUM=1cdcb520a5614f101fe172bdfa0b0c9276978abe']
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:24,307::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:24,308::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:24,308::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:24,309::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:24,310::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:24,311::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:24,918::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:24,921::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:24,922::blockSD::145::storage.Metadata.VGTagMetadataRW::(writelines) Updating metadata adding=MDT_POOL_SPM_ID=-1, MDT__SHA_CKSUM=1cdcb520a5614f101fe172bdfa0b0c9276978abe removing=MDT__SHA_CKSUM=215540c631d1cfd943bc1c89ef09ec9e3251e9bd, MDT_POOL_SPM_ID=2
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:24,923::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --deltag MDT_POOL_SPM_ID=2 --deltag MDT__SHA_CKSUM=215540c631d1cfd943bc1c89ef09ec9e3251e9bd --addtag MDT__SHA_CKSUM=1cdcb520a5614f101fe172bdfa0b0c9276978abe --addtag MDT_POOL_SPM_ID=-1 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15910870::DEBUG::2013-11-19 21:21:25,185::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910870::INFO::2013-11-19 21:21:25,188::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=934c4d35-aab6-4fba-9996-9cc07dd7d991)
>Thread-15910870::DEBUG::2013-11-19 21:21:25,188::task::495::TaskManager.Task::(_debug) Task 15f3ec47-5ff8-41d8-8e0c-bbd7cfdf0c65: moving from state init -> state preparing
>Thread-15910870::DEBUG::2013-11-19 21:21:25,189::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 934c4d35-aab6-4fba-9996-9cc07dd7d991
>Thread-15910870::DEBUG::2013-11-19 21:21:25,190::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}
>Thread-15910870::DEBUG::2013-11-19 21:21:25,190::task::495::TaskManager.Task::(_debug) Task 15f3ec47-5ff8-41d8-8e0c-bbd7cfdf0c65: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910870::DEBUG::2013-11-19 21:21:25,191::task::495::TaskManager.Task::(_debug) Task 15f3ec47-5ff8-41d8-8e0c-bbd7cfdf0c65: moving from state preparing -> state finished
>Thread-15910870::DEBUG::2013-11-19 21:21:25,192::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910870::DEBUG::2013-11-19 21:21:25,192::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910870::DEBUG::2013-11-19 21:21:25,193::task::495::TaskManager.Task::(_debug) Task 15f3ec47-5ff8-41d8-8e0c-bbd7cfdf0c65: ref 0 aborting False
>Thread-15910870::INFO::2013-11-19 21:21:25,193::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:25,619::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:25,620::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:25,621::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:25,622::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction
>934c4d35-aab6-4fba-9996-9cc07dd7d991::INFO::2013-11-19 21:21:25,622::safelease::83::ClusterLock::(release) Releasing cluster lock for domain 6b892067-ac7e-4039-a7be-5b7387525b16
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:25,623::safelease::84::Storage.Misc.excCmd::(release) '/usr/libexec/vdsm/spmstop.sh 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd /usr/libexec/vdsm/)
>Thread-15910871::DEBUG::2013-11-19 21:21:26,201::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910871::INFO::2013-11-19 21:21:26,203::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=934c4d35-aab6-4fba-9996-9cc07dd7d991)
>Thread-15910871::DEBUG::2013-11-19 21:21:26,204::task::495::TaskManager.Task::(_debug) Task ad570ec9-799e-439f-b749-80d14bf6e0d9: moving from state init -> state preparing
>Thread-15910871::DEBUG::2013-11-19 21:21:26,204::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 934c4d35-aab6-4fba-9996-9cc07dd7d991
>Thread-15910871::DEBUG::2013-11-19 21:21:26,205::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}
>Thread-15910871::DEBUG::2013-11-19 21:21:26,205::task::495::TaskManager.Task::(_debug) Task ad570ec9-799e-439f-b749-80d14bf6e0d9: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910871::DEBUG::2013-11-19 21:21:26,206::task::495::TaskManager.Task::(_debug) Task ad570ec9-799e-439f-b749-80d14bf6e0d9: moving from state preparing -> state finished
>Thread-15910871::DEBUG::2013-11-19 21:21:26,207::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910871::DEBUG::2013-11-19 21:21:26,207::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910871::DEBUG::2013-11-19 21:21:26,208::task::495::TaskManager.Task::(_debug) Task ad570ec9-799e-439f-b749-80d14bf6e0d9: ref 0 aborting False
>Thread-15910871::INFO::2013-11-19 21:21:26,208::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:27,095::safelease::84::Storage.Misc.excCmd::(release) SUCCESS: <err> = ''; <rc> = 0
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:27,095::safelease::87::ClusterLock::(release) Cluster lock released successfully
>934c4d35-aab6-4fba-9996-9cc07dd7d991::ERROR::2013-11-19 21:21:27,096::task::868::TaskManager.Task::(_setError) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/task.py", line 876, in _run
>  File "/usr/share/vdsm/storage/task.py", line 303, in run
>  File "/usr/share/vdsm/storage/spm.py", line 493, in start
>  File "/usr/share/vdsm/storage/sp.py", line 637, in _regulariseOutdatedMasters
>  File "/usr/share/vdsm/storage/sp.py", line 631, in _setRegularSD
>  File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 152, in transaction
>  File "/usr/share/vdsm/storage/persistentDict.py", line 266, in flush
>  File "/usr/share/vdsm/storage/fileSD.py", line 58, in writelines
>  File "/usr/share/vdsm/storage/processPool.py", line 33, in wrapper
>  File "/usr/share/vdsm/storage/processPool.py", line 44, in runExternally
>  File "/usr/share/vdsm/storage/processPool.py", line 127, in runExternally
>IOError: [Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata.new'
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:27,098::task::495::TaskManager.Task::(_debug) Task 934c4d35-aab6-4fba-9996-9cc07dd7d991: Task._run: 934c4d35-aab6-4fba-9996-9cc07dd7d991 () {} failed - stopping task
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:27,099::task::495::TaskManager.Task::(_debug) Task 934c4d35-aab6-4fba-9996-9cc07dd7d991: stopping in state running (force False)
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:27,100::task::495::TaskManager.Task::(_debug) Task 934c4d35-aab6-4fba-9996-9cc07dd7d991: ref 1 aborting True
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:27,100::task::918::TaskManager.Task::(_runJobs) aborting: Task is aborted: "[Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata.new'" - code 100
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:27,101::task::495::TaskManager.Task::(_debug) Task 934c4d35-aab6-4fba-9996-9cc07dd7d991: ref 0 aborting True
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:27,101::task::495::TaskManager.Task::(_debug) Task 934c4d35-aab6-4fba-9996-9cc07dd7d991: Task._doAbort: force False
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:27,102::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:27,102::task::495::TaskManager.Task::(_debug) Task 934c4d35-aab6-4fba-9996-9cc07dd7d991: moving from state running -> state aborting
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:27,103::task::495::TaskManager.Task::(_debug) Task 934c4d35-aab6-4fba-9996-9cc07dd7d991: _aborting: recover policy auto
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:27,104::task::495::TaskManager.Task::(_debug) Task 934c4d35-aab6-4fba-9996-9cc07dd7d991: moving from state aborting -> state racquiring
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:27,104::task::495::TaskManager.Task::(_debug) Task 934c4d35-aab6-4fba-9996-9cc07dd7d991: moving from state racquiring -> state recovering
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:27,105::task::495::TaskManager.Task::(_debug) Task 934c4d35-aab6-4fba-9996-9cc07dd7d991: _recover
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:27,105::task::495::TaskManager.Task::(_debug) Task 934c4d35-aab6-4fba-9996-9cc07dd7d991: running recovery None
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:27,106::task::495::TaskManager.Task::(_debug) Task 934c4d35-aab6-4fba-9996-9cc07dd7d991: Recover Done: state recovering
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:27,107::task::495::TaskManager.Task::(_debug) Task 934c4d35-aab6-4fba-9996-9cc07dd7d991: moving from state recovering -> state recovered
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:27,107::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:27,108::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:27,109::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:27,111::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:27,113::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:27,116::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:21:27,118::threadPool::47::Misc.ThreadPool::(setRunningTask) Number of running tasks: 0
>Thread-15910873::DEBUG::2013-11-19 21:21:27,214::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910873::INFO::2013-11-19 21:21:27,215::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=934c4d35-aab6-4fba-9996-9cc07dd7d991)
>Thread-15910873::DEBUG::2013-11-19 21:21:27,216::task::495::TaskManager.Task::(_debug) Task 85a272df-1c72-450e-85df-99cb98eb625c: moving from state init -> state preparing
>Thread-15910873::DEBUG::2013-11-19 21:21:27,216::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 934c4d35-aab6-4fba-9996-9cc07dd7d991
>Thread-15910873::DEBUG::2013-11-19 21:21:27,217::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 100, 'message': "[Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata.new'", 'taskState': 'finished', 'taskResult': 'cleanSuccess', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}
>Thread-15910873::DEBUG::2013-11-19 21:21:27,217::task::495::TaskManager.Task::(_debug) Task 85a272df-1c72-450e-85df-99cb98eb625c: finished: {'taskStatus': {'code': 100, 'message': "[Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata.new'", 'taskState': 'finished', 'taskResult': 'cleanSuccess', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910873::DEBUG::2013-11-19 21:21:27,218::task::495::TaskManager.Task::(_debug) Task 85a272df-1c72-450e-85df-99cb98eb625c: moving from state preparing -> state finished
>Thread-15910873::DEBUG::2013-11-19 21:21:27,218::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910873::DEBUG::2013-11-19 21:21:27,219::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910873::DEBUG::2013-11-19 21:21:27,219::task::495::TaskManager.Task::(_debug) Task 85a272df-1c72-450e-85df-99cb98eb625c: ref 0 aborting False
>Thread-15910873::INFO::2013-11-19 21:21:27,220::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 100, 'message': "[Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata.new'", 'taskState': 'finished', 'taskResult': 'cleanSuccess', 'taskID': '934c4d35-aab6-4fba-9996-9cc07dd7d991'}}
>Thread-15910874::DEBUG::2013-11-19 21:21:27,225::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910874::INFO::2013-11-19 21:21:27,225::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910874::DEBUG::2013-11-19 21:21:27,226::task::495::TaskManager.Task::(_debug) Task 0ffaa8d6-92ab-4b25-bdf5-95d4a05af8b4: moving from state init -> state preparing
>Thread-15910874::DEBUG::2013-11-19 21:21:27,227::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=Free spmLver=280 spmId=-1
>Thread-15910874::DEBUG::2013-11-19 21:21:27,227::task::495::TaskManager.Task::(_debug) Task 0ffaa8d6-92ab-4b25-bdf5-95d4a05af8b4: finished: {'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': 280}}
>Thread-15910874::DEBUG::2013-11-19 21:21:27,228::task::495::TaskManager.Task::(_debug) Task 0ffaa8d6-92ab-4b25-bdf5-95d4a05af8b4: moving from state preparing -> state finished
>Thread-15910874::DEBUG::2013-11-19 21:21:27,228::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910874::DEBUG::2013-11-19 21:21:27,229::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910874::DEBUG::2013-11-19 21:21:27,229::task::495::TaskManager.Task::(_debug) Task 0ffaa8d6-92ab-4b25-bdf5-95d4a05af8b4: ref 0 aborting False
>Thread-15910874::INFO::2013-11-19 21:21:27,230::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': 280}}
>Thread-15910875::DEBUG::2013-11-19 21:21:27,236::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910875::INFO::2013-11-19 21:21:27,236::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: clearTask, args: ( taskID=934c4d35-aab6-4fba-9996-9cc07dd7d991)
>Thread-15910875::DEBUG::2013-11-19 21:21:27,237::task::495::TaskManager.Task::(_debug) Task 2cd55f3e-3421-4030-85b2-577d769c8868: moving from state init -> state preparing
>Thread-15910875::DEBUG::2013-11-19 21:21:27,237::taskManager::146::TaskManager::(clearTask) Entry. taskID: 934c4d35-aab6-4fba-9996-9cc07dd7d991
>Thread-15910875::DEBUG::2013-11-19 21:21:27,238::taskManager::151::TaskManager::(clearTask) Return.
>Thread-15910875::DEBUG::2013-11-19 21:21:27,238::task::495::TaskManager.Task::(_debug) Task 2cd55f3e-3421-4030-85b2-577d769c8868: finished: None
>Thread-15910875::DEBUG::2013-11-19 21:21:27,239::task::495::TaskManager.Task::(_debug) Task 2cd55f3e-3421-4030-85b2-577d769c8868: moving from state preparing -> state finished
>Thread-15910875::DEBUG::2013-11-19 21:21:27,239::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910875::DEBUG::2013-11-19 21:21:27,240::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910875::DEBUG::2013-11-19 21:21:27,240::task::495::TaskManager.Task::(_debug) Task 2cd55f3e-3421-4030-85b2-577d769c8868: ref 0 aborting False
>Thread-15910875::INFO::2013-11-19 21:21:27,241::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: clearTask, Return response: {'status': {'message': 'OK', 'code': 0}}
>Thread-15910876::DEBUG::2013-11-19 21:21:27,315::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15910876::INFO::2013-11-19 21:21:27,316::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15910876::DEBUG::2013-11-19 21:21:27,316::task::495::TaskManager.Task::(_debug) Task 7eae8942-543b-42cb-b181-230ba74e7650: moving from state init -> state preparing
>Thread-15910876::DEBUG::2013-11-19 21:21:27,317::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=Free spmLver=280 spmId=-1
>Thread-15910876::DEBUG::2013-11-19 21:21:27,318::task::495::TaskManager.Task::(_debug) Task 7eae8942-543b-42cb-b181-230ba74e7650: finished: {'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': 280}}
>Thread-15910876::DEBUG::2013-11-19 21:21:27,318::task::495::TaskManager.Task::(_debug) Task 7eae8942-543b-42cb-b181-230ba74e7650: moving from state preparing -> state finished
>Thread-15910876::DEBUG::2013-11-19 21:21:27,319::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910876::DEBUG::2013-11-19 21:21:27,319::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910876::DEBUG::2013-11-19 21:21:27,320::task::495::TaskManager.Task::(_debug) Task 7eae8942-543b-42cb-b181-230ba74e7650: ref 0 aborting False
>Thread-15910876::INFO::2013-11-19 21:21:27,320::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': 280}}
>Thread-1024767::ERROR::2013-11-19 21:21:29,830::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910879::INFO::2013-11-19 21:21:32,880::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910879::DEBUG::2013-11-19 21:21:32,881::task::495::TaskManager.Task::(_debug) Task a077b9c7-d3d2-4e49-8384-36050d5bb046: moving from state init -> state preparing
>Thread-15910879::WARNING::2013-11-19 21:21:32,882::hsm::1895::Storage.HSM::(public_repoStats) repoStats detected invalid master:6b892067-ac7e-4039-a7be-5b7387525b16 {'finish': 1384896083.8064029, 'mdasize': '134479872', 'mdathreshold': True, 'mdavalid': True, 'diskfree': '1142595518464', 'disktotal': '3595827150848', 'result': {'delay': '0.000607013702393', 'lastCheck': 1384896083.8064029, 'valid': True, 'code': 0}, 'masterValidate': {'mount': True, 'valid': True}, 'mdafree': '67205632'}
>Thread-15910879::DEBUG::2013-11-19 21:21:32,882::task::495::TaskManager.Task::(_debug) Task a077b9c7-d3d2-4e49-8384-36050d5bb046: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000607013702393', 'lastCheck': 1384896083.8064029, 'valid': True, 'code': 355}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00144696235657', 'lastCheck': 1384896084.721595, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896089.831394, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000514984130859', 'lastCheck': 1384896089.5651569, 'valid': True, 'code': 0}}
>Thread-15910879::DEBUG::2013-11-19 21:21:32,883::task::495::TaskManager.Task::(_debug) Task a077b9c7-d3d2-4e49-8384-36050d5bb046: moving from state preparing -> state finished
>Thread-15910879::DEBUG::2013-11-19 21:21:32,884::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910879::DEBUG::2013-11-19 21:21:32,884::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910879::DEBUG::2013-11-19 21:21:32,885::task::495::TaskManager.Task::(_debug) Task a077b9c7-d3d2-4e49-8384-36050d5bb046: ref 0 aborting False
>Thread-15910879::INFO::2013-11-19 21:21:32,885::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000607013702393', 'lastCheck': 1384896083.8064029, 'valid': True, 'code': 355}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00144696235657', 'lastCheck': 1384896084.721595, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896089.831394, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000514984130859', 'lastCheck': 1384896089.5651569, 'valid': True, 'code': 0}}
>Thread-42838::DEBUG::2013-11-19 21:21:33,832::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-42838::DEBUG::2013-11-19 21:21:33,834::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 21:21:34,452::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-42838::DEBUG::2013-11-19 21:21:34,455::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1024767::ERROR::2013-11-19 21:21:40,051::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910885::INFO::2013-11-19 21:21:43,001::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910885::DEBUG::2013-11-19 21:21:43,002::task::495::TaskManager.Task::(_debug) Task b92fffed-1113-4788-8806-ddfd2f2a4116: moving from state init -> state preparing
>Thread-15910885::DEBUG::2013-11-19 21:21:43,003::task::495::TaskManager.Task::(_debug) Task b92fffed-1113-4788-8806-ddfd2f2a4116: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000415086746216', 'lastCheck': 1384896094.459101, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00159406661987', 'lastCheck': 1384896094.8393171, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896100.0524311, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000833988189697', 'lastCheck': 1384896099.5780461, 'valid': True, 'code': 0}}
>Thread-15910885::DEBUG::2013-11-19 21:21:43,003::task::495::TaskManager.Task::(_debug) Task b92fffed-1113-4788-8806-ddfd2f2a4116: moving from state preparing -> state finished
>Thread-15910885::DEBUG::2013-11-19 21:21:43,004::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910885::DEBUG::2013-11-19 21:21:43,005::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910885::DEBUG::2013-11-19 21:21:43,005::task::495::TaskManager.Task::(_debug) Task b92fffed-1113-4788-8806-ddfd2f2a4116: ref 0 aborting False
>Thread-15910885::INFO::2013-11-19 21:21:43,006::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000415086746216', 'lastCheck': 1384896094.459101, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00159406661987', 'lastCheck': 1384896094.8393171, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896100.0524311, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000833988189697', 'lastCheck': 1384896099.5780461, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:21:50,065::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910891::INFO::2013-11-19 21:21:53,110::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910891::DEBUG::2013-11-19 21:21:53,111::task::495::TaskManager.Task::(_debug) Task 5d659818-85ad-4bbc-91f7-901d6200c867: moving from state init -> state preparing
>Thread-15910891::DEBUG::2013-11-19 21:21:53,111::task::495::TaskManager.Task::(_debug) Task 5d659818-85ad-4bbc-91f7-901d6200c867: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000335931777954', 'lastCheck': 1384896104.471967, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00227117538452', 'lastCheck': 1384896104.856185, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896110.0663061, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000463962554932', 'lastCheck': 1384896109.588975, 'valid': True, 'code': 0}}
>Thread-15910891::DEBUG::2013-11-19 21:21:53,112::task::495::TaskManager.Task::(_debug) Task 5d659818-85ad-4bbc-91f7-901d6200c867: moving from state preparing -> state finished
>Thread-15910891::DEBUG::2013-11-19 21:21:53,113::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910891::DEBUG::2013-11-19 21:21:53,113::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910891::DEBUG::2013-11-19 21:21:53,114::task::495::TaskManager.Task::(_debug) Task 5d659818-85ad-4bbc-91f7-901d6200c867: ref 0 aborting False
>Thread-15910891::INFO::2013-11-19 21:21:53,114::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000335931777954', 'lastCheck': 1384896104.471967, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00227117538452', 'lastCheck': 1384896104.856185, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896110.0663061, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000463962554932', 'lastCheck': 1384896109.588975, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:22:00,079::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910897::INFO::2013-11-19 21:22:03,219::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910897::DEBUG::2013-11-19 21:22:03,220::task::495::TaskManager.Task::(_debug) Task 6deeb172-3836-4f59-85e0-b9ed64c774df: moving from state init -> state preparing
>Thread-15910897::DEBUG::2013-11-19 21:22:03,220::task::495::TaskManager.Task::(_debug) Task 6deeb172-3836-4f59-85e0-b9ed64c774df: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000320911407471', 'lastCheck': 1384896114.4847939, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142407417297', 'lastCheck': 1384896114.8720989, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896120.0802431, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000452041625977', 'lastCheck': 1384896119.600466, 'valid': True, 'code': 0}}
>Thread-15910897::DEBUG::2013-11-19 21:22:03,221::task::495::TaskManager.Task::(_debug) Task 6deeb172-3836-4f59-85e0-b9ed64c774df: moving from state preparing -> state finished
>Thread-15910897::DEBUG::2013-11-19 21:22:03,222::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910897::DEBUG::2013-11-19 21:22:03,222::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910897::DEBUG::2013-11-19 21:22:03,223::task::495::TaskManager.Task::(_debug) Task 6deeb172-3836-4f59-85e0-b9ed64c774df: ref 0 aborting False
>Thread-15910897::INFO::2013-11-19 21:22:03,223::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000320911407471', 'lastCheck': 1384896114.4847939, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142407417297', 'lastCheck': 1384896114.8720989, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896120.0802431, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000452041625977', 'lastCheck': 1384896119.600466, 'valid': True, 'code': 0}}
>Thread-42840::DEBUG::2013-11-19 21:22:09,611::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:22:10,094::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-19 21:22:10,335::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15910903::INFO::2013-11-19 21:22:13,403::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910903::DEBUG::2013-11-19 21:22:13,404::task::495::TaskManager.Task::(_debug) Task 6466cc32-feb7-41f7-80d5-d7a9827acb2b: moving from state init -> state preparing
>Thread-15910903::DEBUG::2013-11-19 21:22:13,404::task::495::TaskManager.Task::(_debug) Task 6466cc32-feb7-41f7-80d5-d7a9827acb2b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000392913818359', 'lastCheck': 1384896124.4979379, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00154399871826', 'lastCheck': 1384896124.887991, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896130.0957601, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000317096710205', 'lastCheck': 1384896130.3372171, 'valid': True, 'code': 0}}
>Thread-15910903::DEBUG::2013-11-19 21:22:13,405::task::495::TaskManager.Task::(_debug) Task 6466cc32-feb7-41f7-80d5-d7a9827acb2b: moving from state preparing -> state finished
>Thread-15910903::DEBUG::2013-11-19 21:22:13,406::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910903::DEBUG::2013-11-19 21:22:13,406::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910903::DEBUG::2013-11-19 21:22:13,407::task::495::TaskManager.Task::(_debug) Task 6466cc32-feb7-41f7-80d5-d7a9827acb2b: ref 0 aborting False
>Thread-15910903::INFO::2013-11-19 21:22:13,408::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000392913818359', 'lastCheck': 1384896124.4979379, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00154399871826', 'lastCheck': 1384896124.887991, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896130.0957601, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000317096710205', 'lastCheck': 1384896130.3372171, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:22:20,109::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910909::INFO::2013-11-19 21:22:23,539::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910909::DEBUG::2013-11-19 21:22:23,540::task::495::TaskManager.Task::(_debug) Task 56ba6387-fc78-46b8-8319-16f461b57c6d: moving from state init -> state preparing
>Thread-15910909::DEBUG::2013-11-19 21:22:23,540::task::495::TaskManager.Task::(_debug) Task 56ba6387-fc78-46b8-8319-16f461b57c6d: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000443935394287', 'lastCheck': 1384896134.5114851, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00145697593689', 'lastCheck': 1384896134.9042151, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896140.109766, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000392198562622', 'lastCheck': 1384896140.348901, 'valid': True, 'code': 0}}
>Thread-15910909::DEBUG::2013-11-19 21:22:23,541::task::495::TaskManager.Task::(_debug) Task 56ba6387-fc78-46b8-8319-16f461b57c6d: moving from state preparing -> state finished
>Thread-15910909::DEBUG::2013-11-19 21:22:23,542::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910909::DEBUG::2013-11-19 21:22:23,542::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910909::DEBUG::2013-11-19 21:22:23,543::task::495::TaskManager.Task::(_debug) Task 56ba6387-fc78-46b8-8319-16f461b57c6d: ref 0 aborting False
>Thread-15910909::INFO::2013-11-19 21:22:23,543::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000443935394287', 'lastCheck': 1384896134.5114851, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00145697593689', 'lastCheck': 1384896134.9042151, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896140.109766, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000392198562622', 'lastCheck': 1384896140.348901, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:22:30,124::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910915::INFO::2013-11-19 21:22:33,650::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910915::DEBUG::2013-11-19 21:22:33,651::task::495::TaskManager.Task::(_debug) Task 13ecfc05-990e-47b7-a348-4d9fe9ebdcc4: moving from state init -> state preparing
>Thread-15910915::DEBUG::2013-11-19 21:22:33,652::task::495::TaskManager.Task::(_debug) Task 13ecfc05-990e-47b7-a348-4d9fe9ebdcc4: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000395774841309', 'lastCheck': 1384896144.5244119, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00144600868225', 'lastCheck': 1384896144.9200881, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896150.1260059, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000379085540771', 'lastCheck': 1384896150.360539, 'valid': True, 'code': 0}}
>Thread-15910915::DEBUG::2013-11-19 21:22:33,653::task::495::TaskManager.Task::(_debug) Task 13ecfc05-990e-47b7-a348-4d9fe9ebdcc4: moving from state preparing -> state finished
>Thread-15910915::DEBUG::2013-11-19 21:22:33,653::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910915::DEBUG::2013-11-19 21:22:33,654::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910915::DEBUG::2013-11-19 21:22:33,654::task::495::TaskManager.Task::(_debug) Task 13ecfc05-990e-47b7-a348-4d9fe9ebdcc4: ref 0 aborting False
>Thread-15910915::INFO::2013-11-19 21:22:33,655::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000395774841309', 'lastCheck': 1384896144.5244119, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00144600868225', 'lastCheck': 1384896144.9200881, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896150.1260059, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000379085540771', 'lastCheck': 1384896150.360539, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:22:40,139::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910921::INFO::2013-11-19 21:22:43,757::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910921::DEBUG::2013-11-19 21:22:43,757::task::495::TaskManager.Task::(_debug) Task 0d52b733-9f6b-4a23-b41a-3d540ff33ef0: moving from state init -> state preparing
>Thread-15910921::DEBUG::2013-11-19 21:22:43,758::task::495::TaskManager.Task::(_debug) Task 0d52b733-9f6b-4a23-b41a-3d540ff33ef0: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000375032424927', 'lastCheck': 1384896154.537642, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00153112411499', 'lastCheck': 1384896154.9361269, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896160.140373, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000463008880615', 'lastCheck': 1384896160.372328, 'valid': True, 'code': 0}}
>Thread-15910921::DEBUG::2013-11-19 21:22:43,759::task::495::TaskManager.Task::(_debug) Task 0d52b733-9f6b-4a23-b41a-3d540ff33ef0: moving from state preparing -> state finished
>Thread-15910921::DEBUG::2013-11-19 21:22:43,759::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910921::DEBUG::2013-11-19 21:22:43,760::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910921::DEBUG::2013-11-19 21:22:43,760::task::495::TaskManager.Task::(_debug) Task 0d52b733-9f6b-4a23-b41a-3d540ff33ef0: ref 0 aborting False
>Thread-15910921::INFO::2013-11-19 21:22:43,761::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000375032424927', 'lastCheck': 1384896154.537642, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00153112411499', 'lastCheck': 1384896154.9361269, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896160.140373, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000463008880615', 'lastCheck': 1384896160.372328, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:22:50,153::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910927::INFO::2013-11-19 21:22:53,876::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910927::DEBUG::2013-11-19 21:22:53,876::task::495::TaskManager.Task::(_debug) Task 0a44c657-0190-447d-a802-8d6c2ba6863e: moving from state init -> state preparing
>Thread-15910927::DEBUG::2013-11-19 21:22:53,877::task::495::TaskManager.Task::(_debug) Task 0a44c657-0190-447d-a802-8d6c2ba6863e: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.0003821849823', 'lastCheck': 1384896164.5506649, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00227904319763', 'lastCheck': 1384896164.952817, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896170.1539831, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000408887863159', 'lastCheck': 1384896170.3839791, 'valid': True, 'code': 0}}
>Thread-15910927::DEBUG::2013-11-19 21:22:53,878::task::495::TaskManager.Task::(_debug) Task 0a44c657-0190-447d-a802-8d6c2ba6863e: moving from state preparing -> state finished
>Thread-15910927::DEBUG::2013-11-19 21:22:53,878::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910927::DEBUG::2013-11-19 21:22:53,879::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910927::DEBUG::2013-11-19 21:22:53,879::task::495::TaskManager.Task::(_debug) Task 0a44c657-0190-447d-a802-8d6c2ba6863e: ref 0 aborting False
>Thread-15910927::INFO::2013-11-19 21:22:53,880::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.0003821849823', 'lastCheck': 1384896164.5506649, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00227904319763', 'lastCheck': 1384896164.952817, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896170.1539831, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000408887863159', 'lastCheck': 1384896170.3839791, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:23:00,166::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910933::INFO::2013-11-19 21:23:03,985::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910933::DEBUG::2013-11-19 21:23:03,986::task::495::TaskManager.Task::(_debug) Task 3e68bc61-b148-4202-8a20-01a2fce952bc: moving from state init -> state preparing
>Thread-15910933::DEBUG::2013-11-19 21:23:03,987::task::495::TaskManager.Task::(_debug) Task 3e68bc61-b148-4202-8a20-01a2fce952bc: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00349116325378', 'lastCheck': 1384896174.567354, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00141096115112', 'lastCheck': 1384896174.9681931, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896180.167511, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000316143035889', 'lastCheck': 1384896180.3956001, 'valid': True, 'code': 0}}
>Thread-15910933::DEBUG::2013-11-19 21:23:03,987::task::495::TaskManager.Task::(_debug) Task 3e68bc61-b148-4202-8a20-01a2fce952bc: moving from state preparing -> state finished
>Thread-15910933::DEBUG::2013-11-19 21:23:03,988::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910933::DEBUG::2013-11-19 21:23:03,989::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910933::DEBUG::2013-11-19 21:23:03,989::task::495::TaskManager.Task::(_debug) Task 3e68bc61-b148-4202-8a20-01a2fce952bc: ref 0 aborting False
>Thread-15910933::INFO::2013-11-19 21:23:03,990::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00349116325378', 'lastCheck': 1384896174.567354, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00141096115112', 'lastCheck': 1384896174.9681931, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896180.167511, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000316143035889', 'lastCheck': 1384896180.3956001, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:23:10,180::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910939::INFO::2013-11-19 21:23:14,103::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910939::DEBUG::2013-11-19 21:23:14,104::task::495::TaskManager.Task::(_debug) Task 07394d7d-4cca-49cf-9765-44b850e508c4: moving from state init -> state preparing
>Thread-15910939::DEBUG::2013-11-19 21:23:14,105::task::495::TaskManager.Task::(_debug) Task 07394d7d-4cca-49cf-9765-44b850e508c4: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000401973724365', 'lastCheck': 1384896184.580133, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142002105713', 'lastCheck': 1384896184.9836559, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896190.180896, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000381946563721', 'lastCheck': 1384896190.4072521, 'valid': True, 'code': 0}}
>Thread-15910939::DEBUG::2013-11-19 21:23:14,105::task::495::TaskManager.Task::(_debug) Task 07394d7d-4cca-49cf-9765-44b850e508c4: moving from state preparing -> state finished
>Thread-15910939::DEBUG::2013-11-19 21:23:14,106::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910939::DEBUG::2013-11-19 21:23:14,107::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910939::DEBUG::2013-11-19 21:23:14,107::task::495::TaskManager.Task::(_debug) Task 07394d7d-4cca-49cf-9765-44b850e508c4: ref 0 aborting False
>Thread-15910939::INFO::2013-11-19 21:23:14,108::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000401973724365', 'lastCheck': 1384896184.580133, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142002105713', 'lastCheck': 1384896184.9836559, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896190.180896, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000381946563721', 'lastCheck': 1384896190.4072521, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:23:20,194::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910945::INFO::2013-11-19 21:23:24,211::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910945::DEBUG::2013-11-19 21:23:24,211::task::495::TaskManager.Task::(_debug) Task 80d5bec1-7256-43d6-ab0b-e89ea1e988fc: moving from state init -> state preparing
>Thread-15910945::DEBUG::2013-11-19 21:23:24,212::task::495::TaskManager.Task::(_debug) Task 80d5bec1-7256-43d6-ab0b-e89ea1e988fc: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00031590461731', 'lastCheck': 1384896194.5912099, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00140500068665', 'lastCheck': 1384896194.9991951, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896200.195117, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000307083129883', 'lastCheck': 1384896200.418812, 'valid': True, 'code': 0}}
>Thread-15910945::DEBUG::2013-11-19 21:23:24,213::task::495::TaskManager.Task::(_debug) Task 80d5bec1-7256-43d6-ab0b-e89ea1e988fc: moving from state preparing -> state finished
>Thread-15910945::DEBUG::2013-11-19 21:23:24,213::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910945::DEBUG::2013-11-19 21:23:24,214::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910945::DEBUG::2013-11-19 21:23:24,214::task::495::TaskManager.Task::(_debug) Task 80d5bec1-7256-43d6-ab0b-e89ea1e988fc: ref 0 aborting False
>Thread-15910945::INFO::2013-11-19 21:23:24,215::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00031590461731', 'lastCheck': 1384896194.5912099, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00140500068665', 'lastCheck': 1384896194.9991951, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896200.195117, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000307083129883', 'lastCheck': 1384896200.418812, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:23:30,208::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910951::INFO::2013-11-19 21:23:34,360::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910951::DEBUG::2013-11-19 21:23:34,361::task::495::TaskManager.Task::(_debug) Task 21369c81-34ba-473b-8274-fa3ec7f0b785: moving from state init -> state preparing
>Thread-15910951::DEBUG::2013-11-19 21:23:34,361::task::495::TaskManager.Task::(_debug) Task 21369c81-34ba-473b-8274-fa3ec7f0b785: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000311136245728', 'lastCheck': 1384896204.6038909, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00144982337952', 'lastCheck': 1384896205.014539, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896210.208775, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000296115875244', 'lastCheck': 1384896210.430301, 'valid': True, 'code': 0}}
>Thread-15910951::DEBUG::2013-11-19 21:23:34,362::task::495::TaskManager.Task::(_debug) Task 21369c81-34ba-473b-8274-fa3ec7f0b785: moving from state preparing -> state finished
>Thread-15910951::DEBUG::2013-11-19 21:23:34,363::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910951::DEBUG::2013-11-19 21:23:34,363::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910951::DEBUG::2013-11-19 21:23:34,364::task::495::TaskManager.Task::(_debug) Task 21369c81-34ba-473b-8274-fa3ec7f0b785: ref 0 aborting False
>Thread-15910951::INFO::2013-11-19 21:23:34,364::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000311136245728', 'lastCheck': 1384896204.6038909, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00144982337952', 'lastCheck': 1384896205.014539, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896210.208775, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000296115875244', 'lastCheck': 1384896210.430301, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:23:40,221::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910957::INFO::2013-11-19 21:23:44,464::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910957::DEBUG::2013-11-19 21:23:44,465::task::495::TaskManager.Task::(_debug) Task 7eaaad37-7428-487e-ac28-1fe6bba66769: moving from state init -> state preparing
>Thread-15910957::DEBUG::2013-11-19 21:23:44,465::task::495::TaskManager.Task::(_debug) Task 7eaaad37-7428-487e-ac28-1fe6bba66769: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00039005279541', 'lastCheck': 1384896214.6168921, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00134086608887', 'lastCheck': 1384896215.030549, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896220.222296, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000423908233643', 'lastCheck': 1384896220.4417379, 'valid': True, 'code': 0}}
>Thread-15910957::DEBUG::2013-11-19 21:23:44,466::task::495::TaskManager.Task::(_debug) Task 7eaaad37-7428-487e-ac28-1fe6bba66769: moving from state preparing -> state finished
>Thread-15910957::DEBUG::2013-11-19 21:23:44,467::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910957::DEBUG::2013-11-19 21:23:44,467::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910957::DEBUG::2013-11-19 21:23:44,468::task::495::TaskManager.Task::(_debug) Task 7eaaad37-7428-487e-ac28-1fe6bba66769: ref 0 aborting False
>Thread-15910957::INFO::2013-11-19 21:23:44,468::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00039005279541', 'lastCheck': 1384896214.6168921, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00134086608887', 'lastCheck': 1384896215.030549, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896220.222296, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000423908233643', 'lastCheck': 1384896220.4417379, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:23:50,235::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910963::INFO::2013-11-19 21:23:54,574::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910963::DEBUG::2013-11-19 21:23:54,580::task::495::TaskManager.Task::(_debug) Task 25c06ea8-5164-43e7-ab0f-59b1cac68971: moving from state init -> state preparing
>Thread-15910963::DEBUG::2013-11-19 21:23:54,582::task::495::TaskManager.Task::(_debug) Task 25c06ea8-5164-43e7-ab0f-59b1cac68971: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000354051589966', 'lastCheck': 1384896224.629945, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00227999687195', 'lastCheck': 1384896225.047545, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896230.236021, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000418186187744', 'lastCheck': 1384896230.4534709, 'valid': True, 'code': 0}}
>Thread-15910963::DEBUG::2013-11-19 21:23:54,583::task::495::TaskManager.Task::(_debug) Task 25c06ea8-5164-43e7-ab0f-59b1cac68971: moving from state preparing -> state finished
>Thread-15910963::DEBUG::2013-11-19 21:23:54,584::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910963::DEBUG::2013-11-19 21:23:54,585::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910963::DEBUG::2013-11-19 21:23:54,587::task::495::TaskManager.Task::(_debug) Task 25c06ea8-5164-43e7-ab0f-59b1cac68971: ref 0 aborting False
>Thread-15910963::INFO::2013-11-19 21:23:54,587::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000354051589966', 'lastCheck': 1384896224.629945, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00227999687195', 'lastCheck': 1384896225.047545, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896230.236021, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000418186187744', 'lastCheck': 1384896230.4534709, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:24:00,249::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910969::INFO::2013-11-19 21:24:04,692::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910969::DEBUG::2013-11-19 21:24:04,693::task::495::TaskManager.Task::(_debug) Task 10c66dc0-3d4c-4eff-bd3b-800dba1440b5: moving from state init -> state preparing
>Thread-15910969::DEBUG::2013-11-19 21:24:04,693::task::495::TaskManager.Task::(_debug) Task 10c66dc0-3d4c-4eff-bd3b-800dba1440b5: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000418901443481', 'lastCheck': 1384896244.654932, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00141406059265', 'lastCheck': 1384896235.0631771, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896240.2496951, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000428915023804', 'lastCheck': 1384896240.465271, 'valid': True, 'code': 0}}
>Thread-15910969::DEBUG::2013-11-19 21:24:04,694::task::495::TaskManager.Task::(_debug) Task 10c66dc0-3d4c-4eff-bd3b-800dba1440b5: moving from state preparing -> state finished
>Thread-15910969::DEBUG::2013-11-19 21:24:04,695::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910969::DEBUG::2013-11-19 21:24:04,695::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910969::DEBUG::2013-11-19 21:24:04,696::task::495::TaskManager.Task::(_debug) Task 10c66dc0-3d4c-4eff-bd3b-800dba1440b5: ref 0 aborting False
>Thread-15910969::INFO::2013-11-19 21:24:04,696::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000418901443481', 'lastCheck': 1384896244.654932, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00141406059265', 'lastCheck': 1384896235.0631771, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896240.2496951, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000428915023804', 'lastCheck': 1384896240.465271, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:24:10,262::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910975::INFO::2013-11-19 21:24:14,798::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910975::DEBUG::2013-11-19 21:24:14,799::task::495::TaskManager.Task::(_debug) Task d380e398-88f5-497a-8e9c-6ac8cb67e86e: moving from state init -> state preparing
>Thread-15910975::DEBUG::2013-11-19 21:24:14,799::task::495::TaskManager.Task::(_debug) Task d380e398-88f5-497a-8e9c-6ac8cb67e86e: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000379085540771', 'lastCheck': 1384896254.667459, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00147199630737', 'lastCheck': 1384896245.0791991, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896250.263453, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000432968139648', 'lastCheck': 1384896250.4769809, 'valid': True, 'code': 0}}
>Thread-15910975::DEBUG::2013-11-19 21:24:14,800::task::495::TaskManager.Task::(_debug) Task d380e398-88f5-497a-8e9c-6ac8cb67e86e: moving from state preparing -> state finished
>Thread-15910975::DEBUG::2013-11-19 21:24:14,801::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910975::DEBUG::2013-11-19 21:24:14,801::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910975::DEBUG::2013-11-19 21:24:14,802::task::495::TaskManager.Task::(_debug) Task d380e398-88f5-497a-8e9c-6ac8cb67e86e: ref 0 aborting False
>Thread-15910975::INFO::2013-11-19 21:24:14,802::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000379085540771', 'lastCheck': 1384896254.667459, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00147199630737', 'lastCheck': 1384896245.0791991, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896250.263453, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000432968139648', 'lastCheck': 1384896250.4769809, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:24:20,276::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910981::INFO::2013-11-19 21:24:24,956::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910981::DEBUG::2013-11-19 21:24:24,957::task::495::TaskManager.Task::(_debug) Task dda289da-37c7-485b-8731-149e7de23922: moving from state init -> state preparing
>Thread-15910981::DEBUG::2013-11-19 21:24:24,957::task::495::TaskManager.Task::(_debug) Task dda289da-37c7-485b-8731-149e7de23922: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000313997268677', 'lastCheck': 1384896264.6799359, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142598152161', 'lastCheck': 1384896255.095057, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896260.2771511, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000421047210693', 'lastCheck': 1384896260.4887309, 'valid': True, 'code': 0}}
>Thread-15910981::DEBUG::2013-11-19 21:24:24,958::task::495::TaskManager.Task::(_debug) Task dda289da-37c7-485b-8731-149e7de23922: moving from state preparing -> state finished
>Thread-15910981::DEBUG::2013-11-19 21:24:24,958::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910981::DEBUG::2013-11-19 21:24:24,959::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910981::DEBUG::2013-11-19 21:24:24,959::task::495::TaskManager.Task::(_debug) Task dda289da-37c7-485b-8731-149e7de23922: ref 0 aborting False
>Thread-15910981::INFO::2013-11-19 21:24:24,960::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000313997268677', 'lastCheck': 1384896264.6799359, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142598152161', 'lastCheck': 1384896255.095057, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896260.2771511, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000421047210693', 'lastCheck': 1384896260.4887309, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:24:30,290::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-19 21:24:34,691::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15910987::INFO::2013-11-19 21:24:35,067::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910987::DEBUG::2013-11-19 21:24:35,068::task::495::TaskManager.Task::(_debug) Task 2be574eb-57aa-4c63-93b6-b949e5de9c88: moving from state init -> state preparing
>Thread-15910987::DEBUG::2013-11-19 21:24:35,069::task::495::TaskManager.Task::(_debug) Task 2be574eb-57aa-4c63-93b6-b949e5de9c88: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000313997268677', 'lastCheck': 1384896264.6799359, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00144600868225', 'lastCheck': 1384896265.111063, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896270.2910099, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000360012054443', 'lastCheck': 1384896270.5004399, 'valid': True, 'code': 0}}
>Thread-15910987::DEBUG::2013-11-19 21:24:35,070::task::495::TaskManager.Task::(_debug) Task 2be574eb-57aa-4c63-93b6-b949e5de9c88: moving from state preparing -> state finished
>Thread-15910987::DEBUG::2013-11-19 21:24:35,071::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910987::DEBUG::2013-11-19 21:24:35,071::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910987::DEBUG::2013-11-19 21:24:35,072::task::495::TaskManager.Task::(_debug) Task 2be574eb-57aa-4c63-93b6-b949e5de9c88: ref 0 aborting False
>Thread-15910987::INFO::2013-11-19 21:24:35,072::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000313997268677', 'lastCheck': 1384896264.6799359, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00144600868225', 'lastCheck': 1384896265.111063, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896270.2910099, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000360012054443', 'lastCheck': 1384896270.5004399, 'valid': True, 'code': 0}}
>Thread-42838::DEBUG::2013-11-19 21:24:35,399::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:24:40,304::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910993::INFO::2013-11-19 21:24:45,204::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910993::DEBUG::2013-11-19 21:24:45,204::task::495::TaskManager.Task::(_debug) Task d8d9df0b-164d-4280-a366-55d2e1601171: moving from state init -> state preparing
>Thread-15910993::DEBUG::2013-11-19 21:24:45,205::task::495::TaskManager.Task::(_debug) Task d8d9df0b-164d-4280-a366-55d2e1601171: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000316143035889', 'lastCheck': 1384896275.402132, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00249290466309', 'lastCheck': 1384896285.145678, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896280.3052809, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000444173812866', 'lastCheck': 1384896280.5122001, 'valid': True, 'code': 0}}
>Thread-15910993::DEBUG::2013-11-19 21:24:45,206::task::495::TaskManager.Task::(_debug) Task d8d9df0b-164d-4280-a366-55d2e1601171: moving from state preparing -> state finished
>Thread-15910993::DEBUG::2013-11-19 21:24:45,206::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910993::DEBUG::2013-11-19 21:24:45,207::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910993::DEBUG::2013-11-19 21:24:45,207::task::495::TaskManager.Task::(_debug) Task d8d9df0b-164d-4280-a366-55d2e1601171: ref 0 aborting False
>Thread-15910993::INFO::2013-11-19 21:24:45,208::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000316143035889', 'lastCheck': 1384896275.402132, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00249290466309', 'lastCheck': 1384896285.145678, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896280.3052809, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000444173812866', 'lastCheck': 1384896280.5122001, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:24:50,317::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15910999::INFO::2013-11-19 21:24:55,315::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15910999::DEBUG::2013-11-19 21:24:55,315::task::495::TaskManager.Task::(_debug) Task ca555fbf-150a-42b4-967a-89d475b807b9: moving from state init -> state preparing
>Thread-15910999::DEBUG::2013-11-19 21:24:55,316::task::495::TaskManager.Task::(_debug) Task ca555fbf-150a-42b4-967a-89d475b807b9: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000219106674194', 'lastCheck': 1384896285.4141891, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00143218040466', 'lastCheck': 1384896295.1599159, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896290.31884, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000364065170288', 'lastCheck': 1384896290.5236421, 'valid': True, 'code': 0}}
>Thread-15910999::DEBUG::2013-11-19 21:24:55,317::task::495::TaskManager.Task::(_debug) Task ca555fbf-150a-42b4-967a-89d475b807b9: moving from state preparing -> state finished
>Thread-15910999::DEBUG::2013-11-19 21:24:55,317::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15910999::DEBUG::2013-11-19 21:24:55,318::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15910999::DEBUG::2013-11-19 21:24:55,318::task::495::TaskManager.Task::(_debug) Task ca555fbf-150a-42b4-967a-89d475b807b9: ref 0 aborting False
>Thread-15910999::INFO::2013-11-19 21:24:55,327::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000219106674194', 'lastCheck': 1384896285.4141891, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00143218040466', 'lastCheck': 1384896295.1599159, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896290.31884, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000364065170288', 'lastCheck': 1384896290.5236421, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:25:00,331::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911005::INFO::2013-11-19 21:25:05,438::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911005::DEBUG::2013-11-19 21:25:05,440::task::495::TaskManager.Task::(_debug) Task 7e705f0f-13cb-46e7-8f5e-59bba9d5b1b0: moving from state init -> state preparing
>Thread-15911005::DEBUG::2013-11-19 21:25:05,440::task::495::TaskManager.Task::(_debug) Task 7e705f0f-13cb-46e7-8f5e-59bba9d5b1b0: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000633955001831', 'lastCheck': 1384896305.4404781, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00149202346802', 'lastCheck': 1384896305.176949, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896300.3321509, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000327110290527', 'lastCheck': 1384896300.5350029, 'valid': True, 'code': 0}}
>Thread-15911005::DEBUG::2013-11-19 21:25:05,441::task::495::TaskManager.Task::(_debug) Task 7e705f0f-13cb-46e7-8f5e-59bba9d5b1b0: moving from state preparing -> state finished
>Thread-15911005::DEBUG::2013-11-19 21:25:05,442::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911005::DEBUG::2013-11-19 21:25:05,442::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911005::DEBUG::2013-11-19 21:25:05,443::task::495::TaskManager.Task::(_debug) Task 7e705f0f-13cb-46e7-8f5e-59bba9d5b1b0: ref 0 aborting False
>Thread-15911005::INFO::2013-11-19 21:25:05,443::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000633955001831', 'lastCheck': 1384896305.4404781, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00149202346802', 'lastCheck': 1384896305.176949, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896300.3321509, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000327110290527', 'lastCheck': 1384896300.5350029, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:25:10,345::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911011::INFO::2013-11-19 21:25:15,549::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911011::DEBUG::2013-11-19 21:25:15,550::task::495::TaskManager.Task::(_debug) Task 42b01541-e880-41c0-97e3-cf2f45b981f2: moving from state init -> state preparing
>Thread-15911011::DEBUG::2013-11-19 21:25:15,550::task::495::TaskManager.Task::(_debug) Task 42b01541-e880-41c0-97e3-cf2f45b981f2: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000323057174683', 'lastCheck': 1384896315.453151, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00143718719482', 'lastCheck': 1384896315.1922719, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896310.3458619, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000452041625977', 'lastCheck': 1384896310.5464771, 'valid': True, 'code': 0}}
>Thread-15911011::DEBUG::2013-11-19 21:25:15,551::task::495::TaskManager.Task::(_debug) Task 42b01541-e880-41c0-97e3-cf2f45b981f2: moving from state preparing -> state finished
>Thread-15911011::DEBUG::2013-11-19 21:25:15,552::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911011::DEBUG::2013-11-19 21:25:15,552::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911011::DEBUG::2013-11-19 21:25:15,553::task::495::TaskManager.Task::(_debug) Task 42b01541-e880-41c0-97e3-cf2f45b981f2: ref 0 aborting False
>Thread-15911011::INFO::2013-11-19 21:25:15,553::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000323057174683', 'lastCheck': 1384896315.453151, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00143718719482', 'lastCheck': 1384896315.1922719, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896310.3458619, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000452041625977', 'lastCheck': 1384896310.5464771, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:25:20,358::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911017::INFO::2013-11-19 21:25:25,665::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911017::DEBUG::2013-11-19 21:25:25,666::task::495::TaskManager.Task::(_debug) Task 397d33ce-a1c4-4b2a-9537-853782052a17: moving from state init -> state preparing
>Thread-15911017::DEBUG::2013-11-19 21:25:25,676::task::495::TaskManager.Task::(_debug) Task 397d33ce-a1c4-4b2a-9537-853782052a17: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000262022018433', 'lastCheck': 1384896325.465745, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00149488449097', 'lastCheck': 1384896325.2081001, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896320.3593431, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000322818756104', 'lastCheck': 1384896320.558249, 'valid': True, 'code': 0}}
>Thread-15911017::DEBUG::2013-11-19 21:25:25,677::task::495::TaskManager.Task::(_debug) Task 397d33ce-a1c4-4b2a-9537-853782052a17: moving from state preparing -> state finished
>Thread-15911017::DEBUG::2013-11-19 21:25:25,677::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911017::DEBUG::2013-11-19 21:25:25,678::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911017::DEBUG::2013-11-19 21:25:25,678::task::495::TaskManager.Task::(_debug) Task 397d33ce-a1c4-4b2a-9537-853782052a17: ref 0 aborting False
>Thread-15911017::INFO::2013-11-19 21:25:25,679::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000262022018433', 'lastCheck': 1384896325.465745, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00149488449097', 'lastCheck': 1384896325.2081001, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896320.3593431, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000322818756104', 'lastCheck': 1384896320.558249, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:25:30,372::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911023::INFO::2013-11-19 21:25:35,785::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911023::DEBUG::2013-11-19 21:25:35,786::task::495::TaskManager.Task::(_debug) Task 522eae04-da3e-4e10-b195-cdb0fc8b8026: moving from state init -> state preparing
>Thread-15911023::DEBUG::2013-11-19 21:25:35,786::task::495::TaskManager.Task::(_debug) Task 522eae04-da3e-4e10-b195-cdb0fc8b8026: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000318050384521', 'lastCheck': 1384896335.4784651, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00152707099915', 'lastCheck': 1384896335.2240939, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896330.3729451, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000387907028198', 'lastCheck': 1384896330.569663, 'valid': True, 'code': 0}}
>Thread-15911023::DEBUG::2013-11-19 21:25:35,787::task::495::TaskManager.Task::(_debug) Task 522eae04-da3e-4e10-b195-cdb0fc8b8026: moving from state preparing -> state finished
>Thread-15911023::DEBUG::2013-11-19 21:25:35,788::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911023::DEBUG::2013-11-19 21:25:35,788::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911023::DEBUG::2013-11-19 21:25:35,789::task::495::TaskManager.Task::(_debug) Task 522eae04-da3e-4e10-b195-cdb0fc8b8026: ref 0 aborting False
>Thread-15911023::INFO::2013-11-19 21:25:35,789::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000318050384521', 'lastCheck': 1384896335.4784651, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00152707099915', 'lastCheck': 1384896335.2240939, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896330.3729451, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000387907028198', 'lastCheck': 1384896330.569663, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:25:40,385::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911029::INFO::2013-11-19 21:25:45,901::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911029::DEBUG::2013-11-19 21:25:45,902::task::495::TaskManager.Task::(_debug) Task 6345fee6-bb46-45c2-bb98-985b33b18f08: moving from state init -> state preparing
>Thread-15911029::DEBUG::2013-11-19 21:25:45,911::task::495::TaskManager.Task::(_debug) Task 6345fee6-bb46-45c2-bb98-985b33b18f08: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000273942947388', 'lastCheck': 1384896345.4910541, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00222897529602', 'lastCheck': 1384896345.2406721, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896340.3866911, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000345945358276', 'lastCheck': 1384896340.580718, 'valid': True, 'code': 0}}
>Thread-15911029::DEBUG::2013-11-19 21:25:45,912::task::495::TaskManager.Task::(_debug) Task 6345fee6-bb46-45c2-bb98-985b33b18f08: moving from state preparing -> state finished
>Thread-15911029::DEBUG::2013-11-19 21:25:45,913::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911029::DEBUG::2013-11-19 21:25:45,913::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911029::DEBUG::2013-11-19 21:25:45,914::task::495::TaskManager.Task::(_debug) Task 6345fee6-bb46-45c2-bb98-985b33b18f08: ref 0 aborting False
>Thread-15911029::INFO::2013-11-19 21:25:45,914::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000273942947388', 'lastCheck': 1384896345.4910541, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00222897529602', 'lastCheck': 1384896345.2406721, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896340.3866911, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000345945358276', 'lastCheck': 1384896340.580718, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:25:50,399::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911035::INFO::2013-11-19 21:25:56,021::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911035::DEBUG::2013-11-19 21:25:56,022::task::495::TaskManager.Task::(_debug) Task 6a5c5a5f-6a45-4470-858e-e0caff9574d5: moving from state init -> state preparing
>Thread-15911035::DEBUG::2013-11-19 21:25:56,022::task::495::TaskManager.Task::(_debug) Task 6a5c5a5f-6a45-4470-858e-e0caff9574d5: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000296115875244', 'lastCheck': 1384896355.503885, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142002105713', 'lastCheck': 1384896355.2560771, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896350.400012, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000488996505737', 'lastCheck': 1384896350.5918469, 'valid': True, 'code': 0}}
>Thread-15911035::DEBUG::2013-11-19 21:25:56,023::task::495::TaskManager.Task::(_debug) Task 6a5c5a5f-6a45-4470-858e-e0caff9574d5: moving from state preparing -> state finished
>Thread-15911035::DEBUG::2013-11-19 21:25:56,024::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911035::DEBUG::2013-11-19 21:25:56,024::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911035::DEBUG::2013-11-19 21:25:56,025::task::495::TaskManager.Task::(_debug) Task 6a5c5a5f-6a45-4470-858e-e0caff9574d5: ref 0 aborting False
>Thread-15911035::INFO::2013-11-19 21:25:56,025::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000296115875244', 'lastCheck': 1384896355.503885, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142002105713', 'lastCheck': 1384896355.2560771, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896350.400012, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000488996505737', 'lastCheck': 1384896350.5918469, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:26:00,412::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911041::INFO::2013-11-19 21:26:06,160::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911041::DEBUG::2013-11-19 21:26:06,161::task::495::TaskManager.Task::(_debug) Task be4e21fe-1b39-4996-b929-069a1e84e955: moving from state init -> state preparing
>Thread-15911041::DEBUG::2013-11-19 21:26:06,162::task::495::TaskManager.Task::(_debug) Task be4e21fe-1b39-4996-b929-069a1e84e955: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000296115875244', 'lastCheck': 1384896365.5164969, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00206708908081', 'lastCheck': 1384896365.272527, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896360.413337, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000367879867554', 'lastCheck': 1384896360.602582, 'valid': True, 'code': 0}}
>Thread-15911041::DEBUG::2013-11-19 21:26:06,163::task::495::TaskManager.Task::(_debug) Task be4e21fe-1b39-4996-b929-069a1e84e955: moving from state preparing -> state finished
>Thread-15911041::DEBUG::2013-11-19 21:26:06,163::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911041::DEBUG::2013-11-19 21:26:06,164::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911041::DEBUG::2013-11-19 21:26:06,164::task::495::TaskManager.Task::(_debug) Task be4e21fe-1b39-4996-b929-069a1e84e955: ref 0 aborting False
>Thread-15911041::INFO::2013-11-19 21:26:06,165::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000296115875244', 'lastCheck': 1384896365.5164969, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00206708908081', 'lastCheck': 1384896365.272527, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896360.413337, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000367879867554', 'lastCheck': 1384896360.602582, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:26:10,426::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911047::INFO::2013-11-19 21:26:16,243::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911047::DEBUG::2013-11-19 21:26:16,244::task::495::TaskManager.Task::(_debug) Task e95e7d70-5209-44d0-84db-48c9cd0c82b1: moving from state init -> state preparing
>Thread-15911047::DEBUG::2013-11-19 21:26:16,244::task::495::TaskManager.Task::(_debug) Task e95e7d70-5209-44d0-84db-48c9cd0c82b1: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000295162200928', 'lastCheck': 1384896375.5289631, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00150609016418', 'lastCheck': 1384896375.288049, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896370.426806, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000501155853271', 'lastCheck': 1384896370.6144941, 'valid': True, 'code': 0}}
>Thread-15911047::DEBUG::2013-11-19 21:26:16,245::task::495::TaskManager.Task::(_debug) Task e95e7d70-5209-44d0-84db-48c9cd0c82b1: moving from state preparing -> state finished
>Thread-15911047::DEBUG::2013-11-19 21:26:16,245::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911047::DEBUG::2013-11-19 21:26:16,246::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911047::DEBUG::2013-11-19 21:26:16,246::task::495::TaskManager.Task::(_debug) Task e95e7d70-5209-44d0-84db-48c9cd0c82b1: ref 0 aborting False
>Thread-15911047::INFO::2013-11-19 21:26:16,247::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000295162200928', 'lastCheck': 1384896375.5289631, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00150609016418', 'lastCheck': 1384896375.288049, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896370.426806, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000501155853271', 'lastCheck': 1384896370.6144941, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:26:20,439::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911053::INFO::2013-11-19 21:26:26,358::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911053::DEBUG::2013-11-19 21:26:26,358::task::495::TaskManager.Task::(_debug) Task 49a7c2bc-0d7f-4baf-8ce8-f9ccd2591e6a: moving from state init -> state preparing
>Thread-15911053::DEBUG::2013-11-19 21:26:26,359::task::495::TaskManager.Task::(_debug) Task 49a7c2bc-0d7f-4baf-8ce8-f9ccd2591e6a: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000281095504761', 'lastCheck': 1384896385.5414951, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00146007537842', 'lastCheck': 1384896385.303916, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896380.440177, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000383853912354', 'lastCheck': 1384896380.626647, 'valid': True, 'code': 0}}
>Thread-15911053::DEBUG::2013-11-19 21:26:26,360::task::495::TaskManager.Task::(_debug) Task 49a7c2bc-0d7f-4baf-8ce8-f9ccd2591e6a: moving from state preparing -> state finished
>Thread-15911053::DEBUG::2013-11-19 21:26:26,360::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911053::DEBUG::2013-11-19 21:26:26,361::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911053::DEBUG::2013-11-19 21:26:26,361::task::495::TaskManager.Task::(_debug) Task 49a7c2bc-0d7f-4baf-8ce8-f9ccd2591e6a: ref 0 aborting False
>Thread-15911053::INFO::2013-11-19 21:26:26,362::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000281095504761', 'lastCheck': 1384896385.5414951, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00146007537842', 'lastCheck': 1384896385.303916, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896380.440177, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000383853912354', 'lastCheck': 1384896380.626647, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:26:30,452::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911059::INFO::2013-11-19 21:26:36,573::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911059::DEBUG::2013-11-19 21:26:36,574::task::495::TaskManager.Task::(_debug) Task ba0094fa-0099-47bd-ba40-bc7e8e7ecb21: moving from state init -> state preparing
>Thread-15911059::DEBUG::2013-11-19 21:26:36,575::task::495::TaskManager.Task::(_debug) Task ba0094fa-0099-47bd-ba40-bc7e8e7ecb21: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000355005264282', 'lastCheck': 1384896395.5538001, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00133395195007', 'lastCheck': 1384896395.319736, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896390.453069, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000386953353882', 'lastCheck': 1384896390.6383491, 'valid': True, 'code': 0}}
>Thread-15911059::DEBUG::2013-11-19 21:26:36,576::task::495::TaskManager.Task::(_debug) Task ba0094fa-0099-47bd-ba40-bc7e8e7ecb21: moving from state preparing -> state finished
>Thread-15911059::DEBUG::2013-11-19 21:26:36,577::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911059::DEBUG::2013-11-19 21:26:36,578::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911059::DEBUG::2013-11-19 21:26:36,579::task::495::TaskManager.Task::(_debug) Task ba0094fa-0099-47bd-ba40-bc7e8e7ecb21: ref 0 aborting False
>Thread-15911059::INFO::2013-11-19 21:26:36,581::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000355005264282', 'lastCheck': 1384896395.5538001, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00133395195007', 'lastCheck': 1384896395.319736, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896390.453069, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000386953353882', 'lastCheck': 1384896390.6383491, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:26:40,465::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911065::INFO::2013-11-19 21:26:46,696::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911065::DEBUG::2013-11-19 21:26:46,697::task::495::TaskManager.Task::(_debug) Task 60af5da1-e709-4b33-b69b-19fa6b5af200: moving from state init -> state preparing
>Thread-15911065::DEBUG::2013-11-19 21:26:46,698::task::495::TaskManager.Task::(_debug) Task 60af5da1-e709-4b33-b69b-19fa6b5af200: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000277042388916', 'lastCheck': 1384896405.566467, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00227808952332', 'lastCheck': 1384896405.3356709, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896400.46649, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00047492980957', 'lastCheck': 1384896400.6497071, 'valid': True, 'code': 0}}
>Thread-15911065::DEBUG::2013-11-19 21:26:46,698::task::495::TaskManager.Task::(_debug) Task 60af5da1-e709-4b33-b69b-19fa6b5af200: moving from state preparing -> state finished
>Thread-15911065::DEBUG::2013-11-19 21:26:46,699::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911065::DEBUG::2013-11-19 21:26:46,699::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911065::DEBUG::2013-11-19 21:26:46,700::task::495::TaskManager.Task::(_debug) Task 60af5da1-e709-4b33-b69b-19fa6b5af200: ref 0 aborting False
>Thread-15911065::INFO::2013-11-19 21:26:46,701::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000277042388916', 'lastCheck': 1384896405.566467, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00227808952332', 'lastCheck': 1384896405.3356709, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896400.46649, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00047492980957', 'lastCheck': 1384896400.6497071, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:26:50,479::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911071::INFO::2013-11-19 21:26:56,909::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911071::DEBUG::2013-11-19 21:26:56,909::task::495::TaskManager.Task::(_debug) Task 8b429815-ffab-4f55-8917-2787d5102ed0: moving from state init -> state preparing
>Thread-15911071::DEBUG::2013-11-19 21:26:56,910::task::495::TaskManager.Task::(_debug) Task 8b429815-ffab-4f55-8917-2787d5102ed0: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000290155410767', 'lastCheck': 1384896415.5789361, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142598152161', 'lastCheck': 1384896415.3514221, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896410.4797771, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000448942184448', 'lastCheck': 1384896410.6611559, 'valid': True, 'code': 0}}
>Thread-15911071::DEBUG::2013-11-19 21:26:56,911::task::495::TaskManager.Task::(_debug) Task 8b429815-ffab-4f55-8917-2787d5102ed0: moving from state preparing -> state finished
>Thread-15911071::DEBUG::2013-11-19 21:26:56,911::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911071::DEBUG::2013-11-19 21:26:56,912::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911071::DEBUG::2013-11-19 21:26:56,912::task::495::TaskManager.Task::(_debug) Task 8b429815-ffab-4f55-8917-2787d5102ed0: ref 0 aborting False
>Thread-15911071::INFO::2013-11-19 21:26:56,913::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000290155410767', 'lastCheck': 1384896415.5789361, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142598152161', 'lastCheck': 1384896415.3514221, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896410.4797771, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000448942184448', 'lastCheck': 1384896410.6611559, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:27:00,492::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911077::INFO::2013-11-19 21:27:07,021::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911077::DEBUG::2013-11-19 21:27:07,021::task::495::TaskManager.Task::(_debug) Task 165c30d0-b19d-4716-bf84-00ade4ebc32f: moving from state init -> state preparing
>Thread-15911077::DEBUG::2013-11-19 21:27:07,022::task::495::TaskManager.Task::(_debug) Task 165c30d0-b19d-4716-bf84-00ade4ebc32f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000279903411865', 'lastCheck': 1384896425.591224, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00145196914673', 'lastCheck': 1384896425.3669081, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896420.4930091, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000477075576782', 'lastCheck': 1384896420.672648, 'valid': True, 'code': 0}}
>Thread-15911077::DEBUG::2013-11-19 21:27:07,023::task::495::TaskManager.Task::(_debug) Task 165c30d0-b19d-4716-bf84-00ade4ebc32f: moving from state preparing -> state finished
>Thread-15911077::DEBUG::2013-11-19 21:27:07,023::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911077::DEBUG::2013-11-19 21:27:07,024::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911077::DEBUG::2013-11-19 21:27:07,024::task::495::TaskManager.Task::(_debug) Task 165c30d0-b19d-4716-bf84-00ade4ebc32f: ref 0 aborting False
>Thread-15911077::INFO::2013-11-19 21:27:07,025::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000279903411865', 'lastCheck': 1384896425.591224, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00145196914673', 'lastCheck': 1384896425.3669081, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896420.4930091, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000477075576782', 'lastCheck': 1384896420.672648, 'valid': True, 'code': 0}}
>Thread-15911079::DEBUG::2013-11-19 21:27:08,656::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911079::INFO::2013-11-19 21:27:08,657::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911079::DEBUG::2013-11-19 21:27:08,657::task::495::TaskManager.Task::(_debug) Task 39b2886b-616c-455c-9f88-f859600fe2e7: moving from state init -> state preparing
>Thread-15911079::DEBUG::2013-11-19 21:27:08,658::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=Free spmLver=280 spmId=-1
>Thread-15911079::DEBUG::2013-11-19 21:27:08,659::task::495::TaskManager.Task::(_debug) Task 39b2886b-616c-455c-9f88-f859600fe2e7: finished: {'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': 280}}
>Thread-15911079::DEBUG::2013-11-19 21:27:08,659::task::495::TaskManager.Task::(_debug) Task 39b2886b-616c-455c-9f88-f859600fe2e7: moving from state preparing -> state finished
>Thread-15911079::DEBUG::2013-11-19 21:27:08,660::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911079::DEBUG::2013-11-19 21:27:08,660::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911079::DEBUG::2013-11-19 21:27:08,661::task::495::TaskManager.Task::(_debug) Task 39b2886b-616c-455c-9f88-f859600fe2e7: ref 0 aborting False
>Thread-15911079::INFO::2013-11-19 21:27:08,661::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': 280}}
>Thread-15911080::DEBUG::2013-11-19 21:27:08,691::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911080::INFO::2013-11-19 21:27:08,692::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: spmStart, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 prevID=-1 prevLVER=280 recoveryMode=0 scsiFencing=false maxHostID=250 domVersion=2)
>Thread-15911080::DEBUG::2013-11-19 21:27:08,692::task::495::TaskManager.Task::(_debug) Task 3c078476-ceba-49ba-8c4f-cfc47f187488: moving from state init -> state preparing
>Thread-15911080::DEBUG::2013-11-19 21:27:08,694::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e0040fbc-ff91-4bc3-826a-a8d60258cf99`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '330' at 'public_spmStart'
>Thread-15911080::DEBUG::2013-11-19 21:27:08,694::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'exclusive'
>Thread-15911080::DEBUG::2013-11-19 21:27:08,695::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'exclusive' (1 active user)
>Thread-15911080::DEBUG::2013-11-19 21:27:08,695::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e0040fbc-ff91-4bc3-826a-a8d60258cf99`::Granted request
>Thread-15911080::DEBUG::2013-11-19 21:27:08,696::task::495::TaskManager.Task::(_debug) Task 3c078476-ceba-49ba-8c4f-cfc47f187488: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (exclusive)
>Thread-15911080::DEBUG::2013-11-19 21:27:08,697::task::495::TaskManager.Task::(_debug) Task 3c078476-ceba-49ba-8c4f-cfc47f187488: ref 1 aborting False
>Thread-15911080::DEBUG::2013-11-19 21:27:08,697::task::495::TaskManager.Task::(_debug) Task 3c078476-ceba-49ba-8c4f-cfc47f187488: Prepare: 1 jobs exist, move to acquiring
>Thread-15911080::DEBUG::2013-11-19 21:27:08,698::task::495::TaskManager.Task::(_debug) Task 3c078476-ceba-49ba-8c4f-cfc47f187488: moving from state preparing -> state acquiring
>Thread-15911080::DEBUG::2013-11-19 21:27:08,698::task::495::TaskManager.Task::(_debug) Task 3c078476-ceba-49ba-8c4f-cfc47f187488: moving from state acquiring -> state queued
>Thread-15911080::DEBUG::2013-11-19 21:27:08,699::taskManager::37::TaskManager::(_queueTask) queueing task: 3c078476-ceba-49ba-8c4f-cfc47f187488
>Thread-15911080::DEBUG::2013-11-19 21:27:08,700::taskManager::43::TaskManager::(_queueTask) task queued: 3c078476-ceba-49ba-8c4f-cfc47f187488
>21b4b46d-58c8-40ca-8f03-a8063e50f1b9::DEBUG::2013-11-19 21:27:08,700::threadPool::47::Misc.ThreadPool::(setRunningTask) Number of running tasks: 1
>Thread-15911080::DEBUG::2013-11-19 21:27:08,700::task::1183::TaskManager.Task::(prepare) returning
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:08,701::threadPool::192::Misc.ThreadPool.WorkerThread::(run) Task: 3c078476-ceba-49ba-8c4f-cfc47f187488 running: <bound method Task.commit of <storage.task.Task instance at 0x7f487c12bdd0>> with: None
>Thread-15911080::DEBUG::2013-11-19 21:27:08,702::task::495::TaskManager.Task::(_debug) Task 3c078476-ceba-49ba-8c4f-cfc47f187488: ref 0 aborting False
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:08,703::task::1195::TaskManager.Task::(commit) Commiting task: 3c078476-ceba-49ba-8c4f-cfc47f187488
>Thread-15911080::INFO::2013-11-19 21:27:08,704::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: spmStart, Return response: {'status': {'message': 'OK', 'code': 0}, 'uuid': '3c078476-ceba-49ba-8c4f-cfc47f187488'}
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:08,704::task::495::TaskManager.Task::(_debug) Task 3c078476-ceba-49ba-8c4f-cfc47f187488: moving from state queued -> state running
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:08,706::task::907::TaskManager.Task::(_runJobs) Task.run: running job 0: spmStart: <bound method SPM.start of <storage.spm.SPM instance at 0x1bf9f38>> (args: ('c4e5889a-348d-11e2-83fe-9f72369eab30', -1, '280', 0, 'false', 250, 2) kwargs: {})
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:08,706::task::300::TaskManager.Task::(run) Job.run: running spmStart: <bound method SPM.start of <storage.spm.SPM instance at 0x1bf9f38>> (args: ('c4e5889a-348d-11e2-83fe-9f72369eab30', -1, '280', 0, 'false', 250, 2) kwargs: {}) callback None
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:08,707::spm::433::Storage.SPM::(start) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:08,707::misc::1010::SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads)
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:08,708::misc::1012::SamplingMethod::(__call__) Got in to sampling method
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:08,709::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:08,709::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:08,710::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:08,711::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:08,711::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:08,712::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:09,398::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:09,401::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:09,402::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_LVER=281', u'POOL_SPM_ID=-1', u'_SHA_CKSUM=52809f588a9aef1f1ae8cf2b75391d6f337f51aa']
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:09,404::misc::1020::SamplingMethod::(__call__) Returning last result
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:09,405::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:09,405::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:09,406::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:09,406::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:09,407::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:09,408::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15911081::DEBUG::2013-11-19 21:27:09,713::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911081::INFO::2013-11-19 21:27:09,715::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=3c078476-ceba-49ba-8c4f-cfc47f187488)
>Thread-15911081::DEBUG::2013-11-19 21:27:09,715::task::495::TaskManager.Task::(_debug) Task ab0fff4e-6da3-4f90-93a9-296d0f2a869a: moving from state init -> state preparing
>Thread-15911081::DEBUG::2013-11-19 21:27:09,716::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 3c078476-ceba-49ba-8c4f-cfc47f187488
>Thread-15911081::DEBUG::2013-11-19 21:27:09,717::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}
>Thread-15911081::DEBUG::2013-11-19 21:27:09,717::task::495::TaskManager.Task::(_debug) Task ab0fff4e-6da3-4f90-93a9-296d0f2a869a: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911081::DEBUG::2013-11-19 21:27:09,718::task::495::TaskManager.Task::(_debug) Task ab0fff4e-6da3-4f90-93a9-296d0f2a869a: moving from state preparing -> state finished
>Thread-15911081::DEBUG::2013-11-19 21:27:09,719::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911081::DEBUG::2013-11-19 21:27:09,719::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911081::DEBUG::2013-11-19 21:27:09,720::task::495::TaskManager.Task::(_debug) Task ab0fff4e-6da3-4f90-93a9-296d0f2a869a: ref 0 aborting False
>Thread-15911081::INFO::2013-11-19 21:27:09,720::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:10,071::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:10,074::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:10,075::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_LVER=281', u'POOL_SPM_ID=-1', u'_SHA_CKSUM=52809f588a9aef1f1ae8cf2b75391d6f337f51aa']
>3c078476-ceba-49ba-8c4f-cfc47f187488::INFO::2013-11-19 21:27:10,077::spm::457::Storage.SPM::(start) expected previd:-1 lver:281 got request for previd:-1 lver:280
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:10,078::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:10,078::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:10,079::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:10,080::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:10,080::persistentDict::163::Storage.PersistentDict::(__init__) Created a persistant dict with LvMetadataRW backend
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:10,081::blockSD::164::Storage.Misc.excCmd::(readlines) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:10,476::blockSD::164::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000231298 s, 8.9 MB/s\n'; <rc> = 0
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:10,477::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000231298 s, 8.9 MB/s'], size: 2048
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:10,478::persistentDict::204::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[]
>3c078476-ceba-49ba-8c4f-cfc47f187488::WARNING::2013-11-19 21:27:10,478::persistentDict::226::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:10,479::persistentDict::163::Storage.PersistentDict::(__init__) Created a persistant dict with VGTagMetadataRW backend
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:10,480::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:10,480::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:10,481::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:10,482::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:10,482::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:10,484::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42840::DEBUG::2013-11-19 21:27:10,686::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:27:10,687::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911082::DEBUG::2013-11-19 21:27:11,044::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911082::INFO::2013-11-19 21:27:11,046::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=3c078476-ceba-49ba-8c4f-cfc47f187488)
>Thread-15911082::DEBUG::2013-11-19 21:27:11,046::task::495::TaskManager.Task::(_debug) Task 7a90951f-c45e-403b-ba18-ccd649397564: moving from state init -> state preparing
>Thread-15911082::DEBUG::2013-11-19 21:27:11,047::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 3c078476-ceba-49ba-8c4f-cfc47f187488
>Thread-15911082::DEBUG::2013-11-19 21:27:11,047::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}
>Thread-15911082::DEBUG::2013-11-19 21:27:11,047::task::495::TaskManager.Task::(_debug) Task 7a90951f-c45e-403b-ba18-ccd649397564: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911082::DEBUG::2013-11-19 21:27:11,048::task::495::TaskManager.Task::(_debug) Task 7a90951f-c45e-403b-ba18-ccd649397564: moving from state preparing -> state finished
>Thread-15911082::DEBUG::2013-11-19 21:27:11,048::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911082::DEBUG::2013-11-19 21:27:11,048::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911082::DEBUG::2013-11-19 21:27:11,049::task::495::TaskManager.Task::(_debug) Task 7a90951f-c45e-403b-ba18-ccd649397564: ref 0 aborting False
>Thread-15911082::INFO::2013-11-19 21:27:11,049::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:11,246::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:11,249::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:11,250::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_LVER=281', u'POOL_SPM_ID=-1', u'_SHA_CKSUM=52809f588a9aef1f1ae8cf2b75391d6f337f51aa']
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:11,252::safelease::52::ClusterLock::(acquire) Acquiring cluster lock for domain 6b892067-ac7e-4039-a7be-5b7387525b16
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:11,253::safelease::61::Storage.Misc.excCmd::(acquire) '/usr/bin/sudo -n /usr/bin/setsid /usr/bin/ionice -c1 -n0 /bin/su vdsm -s /bin/sh -c "/usr/libexec/vdsm/spmprotect.sh start 6b892067-ac7e-4039-a7be-5b7387525b16 2 5 /dev/6b892067-ac7e-4039-a7be-5b7387525b16/leases 60000 10000 3"' (cwd /usr/libexec/vdsm/)
>Thread-42840::DEBUG::2013-11-19 21:27:11,466::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15911083::DEBUG::2013-11-19 21:27:12,056::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911083::INFO::2013-11-19 21:27:12,058::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=3c078476-ceba-49ba-8c4f-cfc47f187488)
>Thread-15911083::DEBUG::2013-11-19 21:27:12,058::task::495::TaskManager.Task::(_debug) Task a6241060-9310-442a-90c3-5c6e778f6cdf: moving from state init -> state preparing
>Thread-15911083::DEBUG::2013-11-19 21:27:12,059::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 3c078476-ceba-49ba-8c4f-cfc47f187488
>Thread-15911083::DEBUG::2013-11-19 21:27:12,059::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}
>Thread-15911083::DEBUG::2013-11-19 21:27:12,060::task::495::TaskManager.Task::(_debug) Task a6241060-9310-442a-90c3-5c6e778f6cdf: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911083::DEBUG::2013-11-19 21:27:12,061::task::495::TaskManager.Task::(_debug) Task a6241060-9310-442a-90c3-5c6e778f6cdf: moving from state preparing -> state finished
>Thread-15911083::DEBUG::2013-11-19 21:27:12,061::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911083::DEBUG::2013-11-19 21:27:12,062::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911083::DEBUG::2013-11-19 21:27:12,062::task::495::TaskManager.Task::(_debug) Task a6241060-9310-442a-90c3-5c6e778f6cdf: ref 0 aborting False
>Thread-15911083::INFO::2013-11-19 21:27:12,063::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911084::DEBUG::2013-11-19 21:27:13,069::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911084::INFO::2013-11-19 21:27:13,070::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=3c078476-ceba-49ba-8c4f-cfc47f187488)
>Thread-15911084::DEBUG::2013-11-19 21:27:13,070::task::495::TaskManager.Task::(_debug) Task 039ed82c-1b8c-4acd-b5f9-cfadd5deb8a4: moving from state init -> state preparing
>Thread-15911084::DEBUG::2013-11-19 21:27:13,071::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 3c078476-ceba-49ba-8c4f-cfc47f187488
>Thread-15911084::DEBUG::2013-11-19 21:27:13,071::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}
>Thread-15911084::DEBUG::2013-11-19 21:27:13,072::task::495::TaskManager.Task::(_debug) Task 039ed82c-1b8c-4acd-b5f9-cfadd5deb8a4: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911084::DEBUG::2013-11-19 21:27:13,072::task::495::TaskManager.Task::(_debug) Task 039ed82c-1b8c-4acd-b5f9-cfadd5deb8a4: moving from state preparing -> state finished
>Thread-15911084::DEBUG::2013-11-19 21:27:13,073::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911084::DEBUG::2013-11-19 21:27:13,073::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911084::DEBUG::2013-11-19 21:27:13,074::task::495::TaskManager.Task::(_debug) Task 039ed82c-1b8c-4acd-b5f9-cfadd5deb8a4: ref 0 aborting False
>Thread-15911084::INFO::2013-11-19 21:27:13,075::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911085::DEBUG::2013-11-19 21:27:14,080::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911085::INFO::2013-11-19 21:27:14,081::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=3c078476-ceba-49ba-8c4f-cfc47f187488)
>Thread-15911085::DEBUG::2013-11-19 21:27:14,082::task::495::TaskManager.Task::(_debug) Task 05a308c5-a979-441f-990f-2a7cedf85ccd: moving from state init -> state preparing
>Thread-15911085::DEBUG::2013-11-19 21:27:14,082::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 3c078476-ceba-49ba-8c4f-cfc47f187488
>Thread-15911085::DEBUG::2013-11-19 21:27:14,083::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}
>Thread-15911085::DEBUG::2013-11-19 21:27:14,083::task::495::TaskManager.Task::(_debug) Task 05a308c5-a979-441f-990f-2a7cedf85ccd: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911085::DEBUG::2013-11-19 21:27:14,084::task::495::TaskManager.Task::(_debug) Task 05a308c5-a979-441f-990f-2a7cedf85ccd: moving from state preparing -> state finished
>Thread-15911085::DEBUG::2013-11-19 21:27:14,084::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911085::DEBUG::2013-11-19 21:27:14,085::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911085::DEBUG::2013-11-19 21:27:14,085::task::495::TaskManager.Task::(_debug) Task 05a308c5-a979-441f-990f-2a7cedf85ccd: ref 0 aborting False
>Thread-15911085::INFO::2013-11-19 21:27:14,086::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911086::DEBUG::2013-11-19 21:27:15,092::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911086::INFO::2013-11-19 21:27:15,093::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=3c078476-ceba-49ba-8c4f-cfc47f187488)
>Thread-15911086::DEBUG::2013-11-19 21:27:15,093::task::495::TaskManager.Task::(_debug) Task 2e93a975-b7cd-4192-a643-c031252f361b: moving from state init -> state preparing
>Thread-15911086::DEBUG::2013-11-19 21:27:15,094::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 3c078476-ceba-49ba-8c4f-cfc47f187488
>Thread-15911086::DEBUG::2013-11-19 21:27:15,094::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}
>Thread-15911086::DEBUG::2013-11-19 21:27:15,095::task::495::TaskManager.Task::(_debug) Task 2e93a975-b7cd-4192-a643-c031252f361b: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911086::DEBUG::2013-11-19 21:27:15,095::task::495::TaskManager.Task::(_debug) Task 2e93a975-b7cd-4192-a643-c031252f361b: moving from state preparing -> state finished
>Thread-15911086::DEBUG::2013-11-19 21:27:15,096::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911086::DEBUG::2013-11-19 21:27:15,096::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911086::DEBUG::2013-11-19 21:27:15,097::task::495::TaskManager.Task::(_debug) Task 2e93a975-b7cd-4192-a643-c031252f361b: ref 0 aborting False
>Thread-15911086::INFO::2013-11-19 21:27:15,098::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911087::DEBUG::2013-11-19 21:27:16,103::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911087::INFO::2013-11-19 21:27:16,104::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=3c078476-ceba-49ba-8c4f-cfc47f187488)
>Thread-15911087::DEBUG::2013-11-19 21:27:16,105::task::495::TaskManager.Task::(_debug) Task f84929d7-0215-4ea5-848f-b5bb33bd30a4: moving from state init -> state preparing
>Thread-15911087::DEBUG::2013-11-19 21:27:16,105::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 3c078476-ceba-49ba-8c4f-cfc47f187488
>Thread-15911087::DEBUG::2013-11-19 21:27:16,106::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}
>Thread-15911087::DEBUG::2013-11-19 21:27:16,106::task::495::TaskManager.Task::(_debug) Task f84929d7-0215-4ea5-848f-b5bb33bd30a4: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911087::DEBUG::2013-11-19 21:27:16,107::task::495::TaskManager.Task::(_debug) Task f84929d7-0215-4ea5-848f-b5bb33bd30a4: moving from state preparing -> state finished
>Thread-15911087::DEBUG::2013-11-19 21:27:16,107::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911087::DEBUG::2013-11-19 21:27:16,108::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911087::DEBUG::2013-11-19 21:27:16,108::task::495::TaskManager.Task::(_debug) Task f84929d7-0215-4ea5-848f-b5bb33bd30a4: ref 0 aborting False
>Thread-15911087::INFO::2013-11-19 21:27:16,109::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911088::DEBUG::2013-11-19 21:27:17,114::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911088::INFO::2013-11-19 21:27:17,115::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=3c078476-ceba-49ba-8c4f-cfc47f187488)
>Thread-15911088::DEBUG::2013-11-19 21:27:17,116::task::495::TaskManager.Task::(_debug) Task dfbed9f2-8c7c-4642-a516-6d4748cc7bd5: moving from state init -> state preparing
>Thread-15911088::DEBUG::2013-11-19 21:27:17,116::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 3c078476-ceba-49ba-8c4f-cfc47f187488
>Thread-15911088::DEBUG::2013-11-19 21:27:17,117::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}
>Thread-15911088::DEBUG::2013-11-19 21:27:17,117::task::495::TaskManager.Task::(_debug) Task dfbed9f2-8c7c-4642-a516-6d4748cc7bd5: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911088::DEBUG::2013-11-19 21:27:17,118::task::495::TaskManager.Task::(_debug) Task dfbed9f2-8c7c-4642-a516-6d4748cc7bd5: moving from state preparing -> state finished
>Thread-15911088::DEBUG::2013-11-19 21:27:17,118::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911088::DEBUG::2013-11-19 21:27:17,119::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911088::DEBUG::2013-11-19 21:27:17,119::task::495::TaskManager.Task::(_debug) Task dfbed9f2-8c7c-4642-a516-6d4748cc7bd5: ref 0 aborting False
>Thread-15911088::INFO::2013-11-19 21:27:17,120::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911089::DEBUG::2013-11-19 21:27:18,126::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911089::INFO::2013-11-19 21:27:18,127::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=3c078476-ceba-49ba-8c4f-cfc47f187488)
>Thread-15911089::DEBUG::2013-11-19 21:27:18,127::task::495::TaskManager.Task::(_debug) Task a1f9cc6f-636c-4679-8340-f92c7ceb48cc: moving from state init -> state preparing
>Thread-15911089::DEBUG::2013-11-19 21:27:18,128::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 3c078476-ceba-49ba-8c4f-cfc47f187488
>Thread-15911089::DEBUG::2013-11-19 21:27:18,128::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}
>Thread-15911089::DEBUG::2013-11-19 21:27:18,129::task::495::TaskManager.Task::(_debug) Task a1f9cc6f-636c-4679-8340-f92c7ceb48cc: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911089::DEBUG::2013-11-19 21:27:18,130::task::495::TaskManager.Task::(_debug) Task a1f9cc6f-636c-4679-8340-f92c7ceb48cc: moving from state preparing -> state finished
>Thread-15911089::DEBUG::2013-11-19 21:27:18,131::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911089::DEBUG::2013-11-19 21:27:18,131::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911089::DEBUG::2013-11-19 21:27:18,132::task::495::TaskManager.Task::(_debug) Task a1f9cc6f-636c-4679-8340-f92c7ceb48cc: ref 0 aborting False
>Thread-15911089::INFO::2013-11-19 21:27:18,133::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911090::DEBUG::2013-11-19 21:27:19,138::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911090::INFO::2013-11-19 21:27:19,139::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=3c078476-ceba-49ba-8c4f-cfc47f187488)
>Thread-15911090::DEBUG::2013-11-19 21:27:19,140::task::495::TaskManager.Task::(_debug) Task c9555e53-0709-4811-a547-dd8a8ebb6885: moving from state init -> state preparing
>Thread-15911090::DEBUG::2013-11-19 21:27:19,140::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 3c078476-ceba-49ba-8c4f-cfc47f187488
>Thread-15911090::DEBUG::2013-11-19 21:27:19,141::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}
>Thread-15911090::DEBUG::2013-11-19 21:27:19,141::task::495::TaskManager.Task::(_debug) Task c9555e53-0709-4811-a547-dd8a8ebb6885: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911090::DEBUG::2013-11-19 21:27:19,142::task::495::TaskManager.Task::(_debug) Task c9555e53-0709-4811-a547-dd8a8ebb6885: moving from state preparing -> state finished
>Thread-15911090::DEBUG::2013-11-19 21:27:19,142::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911090::DEBUG::2013-11-19 21:27:19,143::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911090::DEBUG::2013-11-19 21:27:19,143::task::495::TaskManager.Task::(_debug) Task c9555e53-0709-4811-a547-dd8a8ebb6885: ref 0 aborting False
>Thread-15911090::INFO::2013-11-19 21:27:19,144::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911091::DEBUG::2013-11-19 21:27:20,150::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911091::INFO::2013-11-19 21:27:20,150::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=3c078476-ceba-49ba-8c4f-cfc47f187488)
>Thread-15911091::DEBUG::2013-11-19 21:27:20,151::task::495::TaskManager.Task::(_debug) Task 953d93aa-bc07-492d-8edb-914b9923877a: moving from state init -> state preparing
>Thread-15911091::DEBUG::2013-11-19 21:27:20,151::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 3c078476-ceba-49ba-8c4f-cfc47f187488
>Thread-15911091::DEBUG::2013-11-19 21:27:20,152::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}
>Thread-15911091::DEBUG::2013-11-19 21:27:20,153::task::495::TaskManager.Task::(_debug) Task 953d93aa-bc07-492d-8edb-914b9923877a: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911091::DEBUG::2013-11-19 21:27:20,153::task::495::TaskManager.Task::(_debug) Task 953d93aa-bc07-492d-8edb-914b9923877a: moving from state preparing -> state finished
>Thread-15911091::DEBUG::2013-11-19 21:27:20,154::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911091::DEBUG::2013-11-19 21:27:20,154::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911091::DEBUG::2013-11-19 21:27:20,155::task::495::TaskManager.Task::(_debug) Task 953d93aa-bc07-492d-8edb-914b9923877a: ref 0 aborting False
>Thread-15911091::INFO::2013-11-19 21:27:20,155::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-1024767::ERROR::2013-11-19 21:27:20,702::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911092::DEBUG::2013-11-19 21:27:21,161::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911092::INFO::2013-11-19 21:27:21,163::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=3c078476-ceba-49ba-8c4f-cfc47f187488)
>Thread-15911092::DEBUG::2013-11-19 21:27:21,163::task::495::TaskManager.Task::(_debug) Task 75d82b22-5362-4fe0-b810-39b53fbebb78: moving from state init -> state preparing
>Thread-15911092::DEBUG::2013-11-19 21:27:21,164::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 3c078476-ceba-49ba-8c4f-cfc47f187488
>Thread-15911092::DEBUG::2013-11-19 21:27:21,164::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}
>Thread-15911092::DEBUG::2013-11-19 21:27:21,165::task::495::TaskManager.Task::(_debug) Task 75d82b22-5362-4fe0-b810-39b53fbebb78: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911092::DEBUG::2013-11-19 21:27:21,166::task::495::TaskManager.Task::(_debug) Task 75d82b22-5362-4fe0-b810-39b53fbebb78: moving from state preparing -> state finished
>Thread-15911092::DEBUG::2013-11-19 21:27:21,166::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911092::DEBUG::2013-11-19 21:27:21,167::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911092::DEBUG::2013-11-19 21:27:21,167::task::495::TaskManager.Task::(_debug) Task 75d82b22-5362-4fe0-b810-39b53fbebb78: ref 0 aborting False
>Thread-15911092::INFO::2013-11-19 21:27:21,168::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911093::DEBUG::2013-11-19 21:27:22,174::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911093::INFO::2013-11-19 21:27:22,174::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=3c078476-ceba-49ba-8c4f-cfc47f187488)
>Thread-15911093::DEBUG::2013-11-19 21:27:22,175::task::495::TaskManager.Task::(_debug) Task 532ffec1-75cc-4714-88de-aa16d29258d0: moving from state init -> state preparing
>Thread-15911093::DEBUG::2013-11-19 21:27:22,175::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 3c078476-ceba-49ba-8c4f-cfc47f187488
>Thread-15911093::DEBUG::2013-11-19 21:27:22,176::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}
>Thread-15911093::DEBUG::2013-11-19 21:27:22,177::task::495::TaskManager.Task::(_debug) Task 532ffec1-75cc-4714-88de-aa16d29258d0: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911093::DEBUG::2013-11-19 21:27:22,177::task::495::TaskManager.Task::(_debug) Task 532ffec1-75cc-4714-88de-aa16d29258d0: moving from state preparing -> state finished
>Thread-15911093::DEBUG::2013-11-19 21:27:22,178::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911093::DEBUG::2013-11-19 21:27:22,178::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911093::DEBUG::2013-11-19 21:27:22,179::task::495::TaskManager.Task::(_debug) Task 532ffec1-75cc-4714-88de-aa16d29258d0: ref 0 aborting False
>Thread-15911093::INFO::2013-11-19 21:27:22,179::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911094::DEBUG::2013-11-19 21:27:23,185::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911094::INFO::2013-11-19 21:27:23,186::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=3c078476-ceba-49ba-8c4f-cfc47f187488)
>Thread-15911094::DEBUG::2013-11-19 21:27:23,186::task::495::TaskManager.Task::(_debug) Task fed80f6f-3740-49c6-9f48-32b6846d6401: moving from state init -> state preparing
>Thread-15911094::DEBUG::2013-11-19 21:27:23,187::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 3c078476-ceba-49ba-8c4f-cfc47f187488
>Thread-15911094::DEBUG::2013-11-19 21:27:23,187::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}
>Thread-15911094::DEBUG::2013-11-19 21:27:23,188::task::495::TaskManager.Task::(_debug) Task fed80f6f-3740-49c6-9f48-32b6846d6401: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911094::DEBUG::2013-11-19 21:27:23,188::task::495::TaskManager.Task::(_debug) Task fed80f6f-3740-49c6-9f48-32b6846d6401: moving from state preparing -> state finished
>Thread-15911094::DEBUG::2013-11-19 21:27:23,189::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911094::DEBUG::2013-11-19 21:27:23,189::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911094::DEBUG::2013-11-19 21:27:23,190::task::495::TaskManager.Task::(_debug) Task fed80f6f-3740-49c6-9f48-32b6846d6401: ref 0 aborting False
>Thread-15911094::INFO::2013-11-19 21:27:23,190::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911095::DEBUG::2013-11-19 21:27:24,196::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911095::INFO::2013-11-19 21:27:24,197::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=3c078476-ceba-49ba-8c4f-cfc47f187488)
>Thread-15911095::DEBUG::2013-11-19 21:27:24,197::task::495::TaskManager.Task::(_debug) Task ced09e13-e1c5-41a6-9b59-dd9f059963f9: moving from state init -> state preparing
>Thread-15911095::DEBUG::2013-11-19 21:27:24,198::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 3c078476-ceba-49ba-8c4f-cfc47f187488
>Thread-15911095::DEBUG::2013-11-19 21:27:24,198::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}
>Thread-15911095::DEBUG::2013-11-19 21:27:24,199::task::495::TaskManager.Task::(_debug) Task ced09e13-e1c5-41a6-9b59-dd9f059963f9: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911095::DEBUG::2013-11-19 21:27:24,200::task::495::TaskManager.Task::(_debug) Task ced09e13-e1c5-41a6-9b59-dd9f059963f9: moving from state preparing -> state finished
>Thread-15911095::DEBUG::2013-11-19 21:27:24,200::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911095::DEBUG::2013-11-19 21:27:24,201::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911095::DEBUG::2013-11-19 21:27:24,201::task::495::TaskManager.Task::(_debug) Task ced09e13-e1c5-41a6-9b59-dd9f059963f9: ref 0 aborting False
>Thread-15911095::INFO::2013-11-19 21:27:24,202::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911096::DEBUG::2013-11-19 21:27:25,207::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911096::INFO::2013-11-19 21:27:25,208::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=3c078476-ceba-49ba-8c4f-cfc47f187488)
>Thread-15911096::DEBUG::2013-11-19 21:27:25,217::task::495::TaskManager.Task::(_debug) Task 2d893693-686d-4780-a573-9d6145122ec8: moving from state init -> state preparing
>Thread-15911096::DEBUG::2013-11-19 21:27:25,217::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 3c078476-ceba-49ba-8c4f-cfc47f187488
>Thread-15911096::DEBUG::2013-11-19 21:27:25,218::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}
>Thread-15911096::DEBUG::2013-11-19 21:27:25,218::task::495::TaskManager.Task::(_debug) Task 2d893693-686d-4780-a573-9d6145122ec8: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911096::DEBUG::2013-11-19 21:27:25,219::task::495::TaskManager.Task::(_debug) Task 2d893693-686d-4780-a573-9d6145122ec8: moving from state preparing -> state finished
>Thread-15911096::DEBUG::2013-11-19 21:27:25,219::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911096::DEBUG::2013-11-19 21:27:25,220::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911096::DEBUG::2013-11-19 21:27:25,220::task::495::TaskManager.Task::(_debug) Task 2d893693-686d-4780-a573-9d6145122ec8: ref 0 aborting False
>Thread-15911096::INFO::2013-11-19 21:27:25,221::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911097::DEBUG::2013-11-19 21:27:26,226::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911097::INFO::2013-11-19 21:27:26,227::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=3c078476-ceba-49ba-8c4f-cfc47f187488)
>Thread-15911097::DEBUG::2013-11-19 21:27:26,228::task::495::TaskManager.Task::(_debug) Task 8d08fd7f-84f8-4058-a655-ce3bc8fc91e2: moving from state init -> state preparing
>Thread-15911097::DEBUG::2013-11-19 21:27:26,228::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 3c078476-ceba-49ba-8c4f-cfc47f187488
>Thread-15911097::DEBUG::2013-11-19 21:27:26,229::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}
>Thread-15911097::DEBUG::2013-11-19 21:27:26,229::task::495::TaskManager.Task::(_debug) Task 8d08fd7f-84f8-4058-a655-ce3bc8fc91e2: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911097::DEBUG::2013-11-19 21:27:26,230::task::495::TaskManager.Task::(_debug) Task 8d08fd7f-84f8-4058-a655-ce3bc8fc91e2: moving from state preparing -> state finished
>Thread-15911097::DEBUG::2013-11-19 21:27:26,231::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911097::DEBUG::2013-11-19 21:27:26,231::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911097::DEBUG::2013-11-19 21:27:26,232::task::495::TaskManager.Task::(_debug) Task 8d08fd7f-84f8-4058-a655-ce3bc8fc91e2: ref 0 aborting False
>Thread-15911097::INFO::2013-11-19 21:27:26,232::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911098::DEBUG::2013-11-19 21:27:27,238::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911098::INFO::2013-11-19 21:27:27,238::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=3c078476-ceba-49ba-8c4f-cfc47f187488)
>Thread-15911098::DEBUG::2013-11-19 21:27:27,239::task::495::TaskManager.Task::(_debug) Task f1bc18e5-e890-4a66-9f42-98fa552f29bd: moving from state init -> state preparing
>Thread-15911098::DEBUG::2013-11-19 21:27:27,240::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 3c078476-ceba-49ba-8c4f-cfc47f187488
>Thread-15911098::DEBUG::2013-11-19 21:27:27,240::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}
>Thread-15911098::DEBUG::2013-11-19 21:27:27,241::task::495::TaskManager.Task::(_debug) Task f1bc18e5-e890-4a66-9f42-98fa552f29bd: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911098::DEBUG::2013-11-19 21:27:27,241::task::495::TaskManager.Task::(_debug) Task f1bc18e5-e890-4a66-9f42-98fa552f29bd: moving from state preparing -> state finished
>Thread-15911098::DEBUG::2013-11-19 21:27:27,242::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911098::DEBUG::2013-11-19 21:27:27,242::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911098::DEBUG::2013-11-19 21:27:27,243::task::495::TaskManager.Task::(_debug) Task f1bc18e5-e890-4a66-9f42-98fa552f29bd: ref 0 aborting False
>Thread-15911098::INFO::2013-11-19 21:27:27,243::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911099::DEBUG::2013-11-19 21:27:28,249::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911099::INFO::2013-11-19 21:27:28,250::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=3c078476-ceba-49ba-8c4f-cfc47f187488)
>Thread-15911099::DEBUG::2013-11-19 21:27:28,251::task::495::TaskManager.Task::(_debug) Task 801c9fdd-86f9-4882-a7f9-7267f2a1233a: moving from state init -> state preparing
>Thread-15911099::DEBUG::2013-11-19 21:27:28,251::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 3c078476-ceba-49ba-8c4f-cfc47f187488
>Thread-15911099::DEBUG::2013-11-19 21:27:28,252::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}
>Thread-15911099::DEBUG::2013-11-19 21:27:28,252::task::495::TaskManager.Task::(_debug) Task 801c9fdd-86f9-4882-a7f9-7267f2a1233a: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911099::DEBUG::2013-11-19 21:27:28,253::task::495::TaskManager.Task::(_debug) Task 801c9fdd-86f9-4882-a7f9-7267f2a1233a: moving from state preparing -> state finished
>Thread-15911099::DEBUG::2013-11-19 21:27:28,253::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911099::DEBUG::2013-11-19 21:27:28,254::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911099::DEBUG::2013-11-19 21:27:28,254::task::495::TaskManager.Task::(_debug) Task 801c9fdd-86f9-4882-a7f9-7267f2a1233a: ref 0 aborting False
>Thread-15911099::INFO::2013-11-19 21:27:28,255::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911100::DEBUG::2013-11-19 21:27:29,261::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911100::INFO::2013-11-19 21:27:29,262::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=3c078476-ceba-49ba-8c4f-cfc47f187488)
>Thread-15911100::DEBUG::2013-11-19 21:27:29,262::task::495::TaskManager.Task::(_debug) Task 036550bd-c864-4d2b-9feb-516c072d876a: moving from state init -> state preparing
>Thread-15911100::DEBUG::2013-11-19 21:27:29,263::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 3c078476-ceba-49ba-8c4f-cfc47f187488
>Thread-15911100::DEBUG::2013-11-19 21:27:29,263::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}
>Thread-15911100::DEBUG::2013-11-19 21:27:29,264::task::495::TaskManager.Task::(_debug) Task 036550bd-c864-4d2b-9feb-516c072d876a: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911100::DEBUG::2013-11-19 21:27:29,265::task::495::TaskManager.Task::(_debug) Task 036550bd-c864-4d2b-9feb-516c072d876a: moving from state preparing -> state finished
>Thread-15911100::DEBUG::2013-11-19 21:27:29,265::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911100::DEBUG::2013-11-19 21:27:29,266::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911100::DEBUG::2013-11-19 21:27:29,266::task::495::TaskManager.Task::(_debug) Task 036550bd-c864-4d2b-9feb-516c072d876a: ref 0 aborting False
>Thread-15911100::INFO::2013-11-19 21:27:29,267::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911101::DEBUG::2013-11-19 21:27:30,272::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911101::INFO::2013-11-19 21:27:30,273::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=3c078476-ceba-49ba-8c4f-cfc47f187488)
>Thread-15911101::DEBUG::2013-11-19 21:27:30,280::task::495::TaskManager.Task::(_debug) Task 5f4c2174-0019-45f1-849a-e6159da182a7: moving from state init -> state preparing
>Thread-15911101::DEBUG::2013-11-19 21:27:30,280::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 3c078476-ceba-49ba-8c4f-cfc47f187488
>Thread-15911101::DEBUG::2013-11-19 21:27:30,281::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}
>Thread-15911101::DEBUG::2013-11-19 21:27:30,281::task::495::TaskManager.Task::(_debug) Task 5f4c2174-0019-45f1-849a-e6159da182a7: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911101::DEBUG::2013-11-19 21:27:30,282::task::495::TaskManager.Task::(_debug) Task 5f4c2174-0019-45f1-849a-e6159da182a7: moving from state preparing -> state finished
>Thread-15911101::DEBUG::2013-11-19 21:27:30,283::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911101::DEBUG::2013-11-19 21:27:30,283::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911101::DEBUG::2013-11-19 21:27:30,284::task::495::TaskManager.Task::(_debug) Task 5f4c2174-0019-45f1-849a-e6159da182a7: ref 0 aborting False
>Thread-15911101::INFO::2013-11-19 21:27:30,284::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-1024767::ERROR::2013-11-19 21:27:30,716::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911102::DEBUG::2013-11-19 21:27:31,290::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911102::INFO::2013-11-19 21:27:31,291::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=3c078476-ceba-49ba-8c4f-cfc47f187488)
>Thread-15911102::DEBUG::2013-11-19 21:27:31,291::task::495::TaskManager.Task::(_debug) Task 8f091748-5407-4968-8751-98d23afbd6f6: moving from state init -> state preparing
>Thread-15911102::DEBUG::2013-11-19 21:27:31,292::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 3c078476-ceba-49ba-8c4f-cfc47f187488
>Thread-15911102::DEBUG::2013-11-19 21:27:31,292::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}
>Thread-15911102::DEBUG::2013-11-19 21:27:31,293::task::495::TaskManager.Task::(_debug) Task 8f091748-5407-4968-8751-98d23afbd6f6: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911102::DEBUG::2013-11-19 21:27:31,293::task::495::TaskManager.Task::(_debug) Task 8f091748-5407-4968-8751-98d23afbd6f6: moving from state preparing -> state finished
>Thread-15911102::DEBUG::2013-11-19 21:27:31,294::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911102::DEBUG::2013-11-19 21:27:31,294::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911102::DEBUG::2013-11-19 21:27:31,295::task::495::TaskManager.Task::(_debug) Task 8f091748-5407-4968-8751-98d23afbd6f6: ref 0 aborting False
>Thread-15911102::INFO::2013-11-19 21:27:31,295::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:31,654::safelease::61::Storage.Misc.excCmd::(acquire) SUCCESS: <err> = ''; <rc> = 0
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:31,655::safelease::65::ClusterLock::(acquire) Clustered lock acquired successfully
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:31,655::spm::468::Storage.SPM::(start) spm lock acquired successfully
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:31,656::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:31,657::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:31,657::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:31,658::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:31,659::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:31,660::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:32,293::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:32,300::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:32,302::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_LVER=281', u'POOL_SPM_ID=-1', u'_SHA_CKSUM=52809f588a9aef1f1ae8cf2b75391d6f337f51aa']
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:32,304::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
>Thread-15911103::DEBUG::2013-11-19 21:27:32,306::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:32,307::persistentDict::146::Storage.PersistentDict::(transaction) Flushing changes
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:32,308::persistentDict::265::Storage.PersistentDict::(flush) about to write lines (VGTagMetadataRW)=[u'CLASS=Data', u'DESCRIPTION=OS', u'IOOPTIMEOUTSEC=10', u'LEASERETRIES=3', u'LEASETIMESEC=60', u'LOCKPOLICY=', u'LOCKRENEWALINTERVALSEC=5', u'LOGBLKSIZE=512', u'MASTER_VERSION=1', u'PHYBLKSIZE=512', u'POOL_DESCRIPTION=Default', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_ID=2', u'POOL_SPM_LVER=282', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'ROLE=Master', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'TYPE=FCP', u'VERSION=2', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', '_SHA_CKSUM=de56f40c15838d78cba198e8c045a60fe83f87c3']
>Thread-15911103::INFO::2013-11-19 21:27:32,309::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=3c078476-ceba-49ba-8c4f-cfc47f187488)
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:32,310::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15911103::DEBUG::2013-11-19 21:27:32,310::task::495::TaskManager.Task::(_debug) Task 5d2cc08e-eff7-4bd1-83c9-4480c598674a: moving from state init -> state preparing
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:32,311::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15911103::DEBUG::2013-11-19 21:27:32,312::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 3c078476-ceba-49ba-8c4f-cfc47f187488
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:32,313::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15911103::DEBUG::2013-11-19 21:27:32,313::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:32,314::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15911103::DEBUG::2013-11-19 21:27:32,315::task::495::TaskManager.Task::(_debug) Task 5d2cc08e-eff7-4bd1-83c9-4480c598674a: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:32,316::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-15911103::DEBUG::2013-11-19 21:27:32,316::task::495::TaskManager.Task::(_debug) Task 5d2cc08e-eff7-4bd1-83c9-4480c598674a: moving from state preparing -> state finished
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:32,317::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15911103::DEBUG::2013-11-19 21:27:32,318::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911103::DEBUG::2013-11-19 21:27:32,319::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911103::DEBUG::2013-11-19 21:27:32,548::task::495::TaskManager.Task::(_debug) Task 5d2cc08e-eff7-4bd1-83c9-4480c598674a: ref 0 aborting False
>Thread-15911103::INFO::2013-11-19 21:27:32,552::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:32,968::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:32,982::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:32,983::blockSD::145::storage.Metadata.VGTagMetadataRW::(writelines) Updating metadata adding=MDT__SHA_CKSUM=de56f40c15838d78cba198e8c045a60fe83f87c3, MDT_POOL_SPM_LVER=282, MDT_POOL_SPM_ID=2 removing=MDT_POOL_SPM_ID=-1, MDT__SHA_CKSUM=52809f588a9aef1f1ae8cf2b75391d6f337f51aa, MDT_POOL_SPM_LVER=281
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:32,985::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --deltag MDT_POOL_SPM_LVER=281 --deltag MDT_POOL_SPM_ID=-1 --deltag MDT__SHA_CKSUM=52809f588a9aef1f1ae8cf2b75391d6f337f51aa --addtag MDT_POOL_SPM_LVER=282 --addtag MDT__SHA_CKSUM=de56f40c15838d78cba198e8c045a60fe83f87c3 --addtag MDT_POOL_SPM_ID=2 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15911104::DEBUG::2013-11-19 21:27:33,561::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911104::INFO::2013-11-19 21:27:33,562::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=3c078476-ceba-49ba-8c4f-cfc47f187488)
>Thread-15911104::DEBUG::2013-11-19 21:27:33,563::task::495::TaskManager.Task::(_debug) Task 96b0234c-e8c5-42d6-93ab-15a508d61288: moving from state init -> state preparing
>Thread-15911104::DEBUG::2013-11-19 21:27:33,564::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 3c078476-ceba-49ba-8c4f-cfc47f187488
>Thread-15911104::DEBUG::2013-11-19 21:27:33,564::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}
>Thread-15911104::DEBUG::2013-11-19 21:27:33,565::task::495::TaskManager.Task::(_debug) Task 96b0234c-e8c5-42d6-93ab-15a508d61288: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911104::DEBUG::2013-11-19 21:27:33,565::task::495::TaskManager.Task::(_debug) Task 96b0234c-e8c5-42d6-93ab-15a508d61288: moving from state preparing -> state finished
>Thread-15911104::DEBUG::2013-11-19 21:27:33,566::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911104::DEBUG::2013-11-19 21:27:33,567::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911104::DEBUG::2013-11-19 21:27:33,567::task::495::TaskManager.Task::(_debug) Task 96b0234c-e8c5-42d6-93ab-15a508d61288: ref 0 aborting False
>Thread-15911104::INFO::2013-11-19 21:27:33,568::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:33,618::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:33,619::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:33,620::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:33,620::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:33,621::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d70ce20c-ee3a-43fe-8e64-cebbf35fc887`::Request was made in '/usr/share/vdsm/storage/spm.py' line '330' at '_upgradePool'
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:33,622::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'exclusive'
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:33,623::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'exclusive' (1 active user)
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:33,623::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d70ce20c-ee3a-43fe-8e64-cebbf35fc887`::Granted request
>3c078476-ceba-49ba-8c4f-cfc47f187488::INFO::2013-11-19 21:27:33,624::spm::336::Storage.SPM::(_upgradePool) Trying to upgrade master domain `6b892067-ac7e-4039-a7be-5b7387525b16`
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:33,625::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8093ac85-edeb-4915-9436-da96688a5090`::Request was made in '/usr/share/vdsm/storage/spm.py' line '337' at '_upgradePool'
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:33,626::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'exclusive'
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:33,626::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'exclusive' (1 active user)
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:33,627::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8093ac85-edeb-4915-9436-da96688a5090`::Granted request
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:33,628::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:33,629::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:33,629::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:33,630::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:33,630::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:33,632::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:34,280::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:34,283::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:34,284::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_LVER=282', u'_SHA_CKSUM=de56f40c15838d78cba198e8c045a60fe83f87c3', u'POOL_SPM_ID=2']
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:34,285::blockSD::625::Storage.StorageDomain::(upgrade) Trying to upgrade domain `6b892067-ac7e-4039-a7be-5b7387525b16` from version 2 to version 2
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:34,286::blockSD::631::Storage.StorageDomain::(upgrade) No need to upgrade domain `6b892067-ac7e-4039-a7be-5b7387525b16`, leaving unchanged
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:34,287::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:34,287::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:34,288::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:34,288::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:34,289::spm::340::Storage.SPM::(_upgradePool) Marking all domains for upgrade
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:34,290::spm::347::Storage.SPM::(_upgradePool) Registering with state change event
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:34,290::spm::349::Storage.SPM::(_upgradePool) Running initial domain upgrade threads
>Thread-15911105::DEBUG::2013-11-19 21:27:34,291::spm::365::Storage.SPM::(_upgradePoolDomain) Preparing to upgrade domain 45cdff8b-89c0-4926-a8c7-80354064bfa6
>Thread-15911107::DEBUG::2013-11-19 21:27:34,293::spm::365::Storage.SPM::(_upgradePoolDomain) Preparing to upgrade domain 5d7b5178-3790-472c-a13b-b481eb27a61e
>Thread-15911106::DEBUG::2013-11-19 21:27:34,294::spm::365::Storage.SPM::(_upgradePoolDomain) Preparing to upgrade domain 772472cb-2429-466d-96cd-00b3301ef78a
>Thread-15911107::ERROR::2013-11-19 21:27:34,296::spm::373::Storage.SPM::(_upgradePoolDomain) Error while preparing domain `5d7b5178-3790-472c-a13b-b481eb27a61e` upgrade
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/spm.py", line 369, in _upgradePoolDomain
>IndexError: list index out of range
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:34,296::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911105::DEBUG::2013-11-19 21:27:34,298::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6`ReqID=`d6b8afd5-fcdf-43b2-b88a-b1e755f3435d`::Request was made in '/usr/share/vdsm/storage/spm.py' line '376' at '_upgradePoolDomain'
>Thread-15911106::DEBUG::2013-11-19 21:27:34,299::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a3eb70c5-f6c4-43b6-9376-0207e30b5eeb`::Request was made in '/usr/share/vdsm/storage/spm.py' line '376' at '_upgradePoolDomain'
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:34,300::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911105::DEBUG::2013-11-19 21:27:34,301::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6' for lock type 'exclusive'
>Thread-15911106::DEBUG::2013-11-19 21:27:34,302::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'exclusive'
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:34,302::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:34,304::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911105::DEBUG::2013-11-19 21:27:34,305::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6' is free. Now locking as 'exclusive' (1 active user)
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:34,306::blockSD::791::Storage.Misc.excCmd::(mountMaster) '/usr/bin/sudo -n /sbin/e2fsck -p /dev/6b892067-ac7e-4039-a7be-5b7387525b16/master' (cwd None)
>Thread-15911105::DEBUG::2013-11-19 21:27:34,306::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6`ReqID=`d6b8afd5-fcdf-43b2-b88a-b1e755f3435d`::Granted request
>Thread-15911106::DEBUG::2013-11-19 21:27:34,308::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'exclusive' (1 active user)
>Thread-15911106::DEBUG::2013-11-19 21:27:34,520::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a3eb70c5-f6c4-43b6-9376-0207e30b5eeb`::Granted request
>Thread-15911105::DEBUG::2013-11-19 21:27:34,525::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6`ReqID=`536b7109-8f80-44f2-83a5-c58b6fefa2e7`::Request was made in '/usr/share/vdsm/storage/spm.py' line '377' at '_upgradePoolDomain'
>Thread-15911106::DEBUG::2013-11-19 21:27:34,525::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d2e38165-3a6b-4ca1-b044-fa09fbe056c1`::Request was made in '/usr/share/vdsm/storage/spm.py' line '377' at '_upgradePoolDomain'
>Thread-15911105::DEBUG::2013-11-19 21:27:34,527::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6' for lock type 'exclusive'
>Thread-15911106::DEBUG::2013-11-19 21:27:34,527::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'exclusive'
>Thread-15911105::DEBUG::2013-11-19 21:27:34,529::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6' is free. Now locking as 'exclusive' (1 active user)
>Thread-15911105::DEBUG::2013-11-19 21:27:34,530::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6`ReqID=`536b7109-8f80-44f2-83a5-c58b6fefa2e7`::Granted request
>Thread-15911106::DEBUG::2013-11-19 21:27:34,531::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'exclusive' (1 active user)
>Thread-15911105::DEBUG::2013-11-19 21:27:34,533::spm::385::Storage.SPM::(_upgradePoolDomain) Domain `45cdff8b-89c0-4926-a8c7-80354064bfa6` is not a data domain it is an 2 domain, not upgrading
>Thread-15911106::DEBUG::2013-11-19 21:27:34,534::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d2e38165-3a6b-4ca1-b044-fa09fbe056c1`::Granted request
>Thread-15911105::DEBUG::2013-11-19 21:27:34,535::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6'
>Thread-15911106::DEBUG::2013-11-19 21:27:34,537::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15911105::DEBUG::2013-11-19 21:27:34,537::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6' (0 active users)
>Thread-15911106::DEBUG::2013-11-19 21:27:34,539::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15911105::DEBUG::2013-11-19 21:27:34,539::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6' is free, finding out if anyone is waiting for it.
>Thread-15911106::DEBUG::2013-11-19 21:27:34,540::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15911105::DEBUG::2013-11-19 21:27:34,541::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6', Clearing records.
>Thread-15911106::DEBUG::2013-11-19 21:27:34,542::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15911105::DEBUG::2013-11-19 21:27:34,543::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6'
>Thread-15911106::DEBUG::2013-11-19 21:27:34,544::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-15911105::DEBUG::2013-11-19 21:27:34,545::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6' (0 active users)
>Thread-15911106::DEBUG::2013-11-19 21:27:34,547::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-15911105::DEBUG::2013-11-19 21:27:34,547::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6' is free, finding out if anyone is waiting for it.
>Thread-15911105::DEBUG::2013-11-19 21:27:34,548::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6', Clearing records.
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:34,896::blockSD::791::Storage.Misc.excCmd::(mountMaster) SUCCESS: <err> = ''; <rc> = 0
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:34,897::blockSD::816::Storage.Misc.excCmd::(mountMaster) '/usr/bin/sudo -n /sbin/tune2fs -j /dev/6b892067-ac7e-4039-a7be-5b7387525b16/master' (cwd None)
>Thread-15911108::DEBUG::2013-11-19 21:27:34,899::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911108::INFO::2013-11-19 21:27:35,254::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=3c078476-ceba-49ba-8c4f-cfc47f187488)
>Thread-15911108::DEBUG::2013-11-19 21:27:35,256::task::495::TaskManager.Task::(_debug) Task 1e66ef65-7ef4-47c3-8b3e-4ceb9cfd384d: moving from state init -> state preparing
>Thread-15911108::DEBUG::2013-11-19 21:27:35,257::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 3c078476-ceba-49ba-8c4f-cfc47f187488
>Thread-15911108::DEBUG::2013-11-19 21:27:35,258::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}
>Thread-15911108::DEBUG::2013-11-19 21:27:35,258::task::495::TaskManager.Task::(_debug) Task 1e66ef65-7ef4-47c3-8b3e-4ceb9cfd384d: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911108::DEBUG::2013-11-19 21:27:35,259::task::495::TaskManager.Task::(_debug) Task 1e66ef65-7ef4-47c3-8b3e-4ceb9cfd384d: moving from state preparing -> state finished
>Thread-15911108::DEBUG::2013-11-19 21:27:35,260::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911108::DEBUG::2013-11-19 21:27:35,260::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911108::DEBUG::2013-11-19 21:27:35,261::task::495::TaskManager.Task::(_debug) Task 1e66ef65-7ef4-47c3-8b3e-4ceb9cfd384d: ref 0 aborting False
>Thread-15911108::INFO::2013-11-19 21:27:35,262::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:35,455::blockSD::816::Storage.Misc.excCmd::(mountMaster) FAILED: <err> = 'The filesystem already has a journal.\n'; <rc> = 1
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:35,457::fileUtils::83::Storage.Misc.excCmd::(mount) '/usr/bin/sudo -n /bin/mount /dev/6b892067-ac7e-4039-a7be-5b7387525b16/master /rhev/data-center/mnt/blockSD/6b892067-ac7e-4039-a7be-5b7387525b16/master' (cwd None)
>Thread-15911106::DEBUG::2013-11-19 21:27:35,691::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15911106::DEBUG::2013-11-19 21:27:35,696::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-15911106::DEBUG::2013-11-19 21:27:35,697::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'SDUUID=772472cb-2429-466d-96cd-00b3301ef78a', u'VERSION=2', u'LOCKPOLICY=', u'VGUUID=XTb294-NYv6-dVpO-bB1n-4f9P-d0GL-dhd2DI', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'LEASETIMESEC=5', u'LOCKRENEWALINTERVALSEC=5', u'ROLE=Regular', u'PHYBLKSIZE=512', u'DESCRIPTION=Database', u'IOOPTIMEOUTSEC=1', u'CLASS=Data', u'TYPE=FCP', u'PV0=pv:3600c0ff0001520c7414f605001000000,uuid:NujM33-vNiD-sBHY-mrPo-EQ2f-1qsz-HXfn0q,pestart:0,pecount:8929,mapoffset:0', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'MASTER_VERSION=0', u'_SHA_CKSUM=ef8840d9fdef24516a56fd970bed1960d2b1ae92']
>Thread-15911106::DEBUG::2013-11-19 21:27:35,699::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15911106::DEBUG::2013-11-19 21:27:35,700::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15911106::DEBUG::2013-11-19 21:27:35,700::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15911106::DEBUG::2013-11-19 21:27:35,701::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15911106::DEBUG::2013-11-19 21:27:35,702::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15911106::DEBUG::2013-11-19 21:27:35,702::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15911106::DEBUG::2013-11-19 21:27:35,703::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15911106::DEBUG::2013-11-19 21:27:35,704::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:35,878::fileUtils::83::Storage.Misc.excCmd::(mount) SUCCESS: <err> = ''; <rc> = 0
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:35,879::blockSD::833::Storage.Misc.excCmd::(mountMaster) '/usr/bin/sudo -n /bin/chown vdsm:kvm /rhev/data-center/mnt/blockSD/6b892067-ac7e-4039-a7be-5b7387525b16/master' (cwd None)
>Thread-15911109::DEBUG::2013-11-19 21:27:36,269::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911109::INFO::2013-11-19 21:27:36,271::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=3c078476-ceba-49ba-8c4f-cfc47f187488)
>Thread-15911109::DEBUG::2013-11-19 21:27:36,271::task::495::TaskManager.Task::(_debug) Task 40c77a93-bebc-4b76-9f60-0fa7e7d2746c: moving from state init -> state preparing
>Thread-15911109::DEBUG::2013-11-19 21:27:36,272::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 3c078476-ceba-49ba-8c4f-cfc47f187488
>Thread-15911109::DEBUG::2013-11-19 21:27:36,272::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}
>Thread-15911109::DEBUG::2013-11-19 21:27:36,273::task::495::TaskManager.Task::(_debug) Task 40c77a93-bebc-4b76-9f60-0fa7e7d2746c: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911109::DEBUG::2013-11-19 21:27:36,274::task::495::TaskManager.Task::(_debug) Task 40c77a93-bebc-4b76-9f60-0fa7e7d2746c: moving from state preparing -> state finished
>Thread-15911109::DEBUG::2013-11-19 21:27:36,274::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911109::DEBUG::2013-11-19 21:27:36,275::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:36,276::blockSD::833::Storage.Misc.excCmd::(mountMaster) SUCCESS: <err> = ''; <rc> = 0
>Thread-15911109::DEBUG::2013-11-19 21:27:36,277::task::495::TaskManager.Task::(_debug) Task 40c77a93-bebc-4b76-9f60-0fa7e7d2746c: ref 0 aborting False
>Thread-15911109::INFO::2013-11-19 21:27:36,279::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>3c078476-ceba-49ba-8c4f-cfc47f187488::WARNING::2013-11-19 21:27:36,279::spm::490::Storage.SPM::(start) Backup domain validation failed
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/spm.py", line 488, in start
>  File "/usr/share/vdsm/storage/sp.py", line 1208, in checkBackupDomain
>  File "/usr/share/vdsm/storage/sd.py", line 666, in isBackup
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:36,282::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:36,282::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:36,283::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:36,283::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:36,284::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:36,285::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:36,285::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:36,286::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:36,286::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:36,287::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:36,287::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:36,287::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:36,289::threadPool::25::Misc.ThreadPool::(__init__) Enter - numThreads: 5.0, waitTimeout: 3, maxTasks: 500.0
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:36,293::storage_mailbox::486::Storage.MailBox.SpmMailMonitor::(__init__) SPM_MailMonitor - clearing outgoing mail, command is: ['dd', 'of=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/outbox', 'oflag=direct', 'iflag=fullblock', 'conv=notrunc', 'count=1']
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:36,294::storage_mailbox::488::Storage.Misc.excCmd::(__init__) 'dd of=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/outbox oflag=direct iflag=fullblock conv=notrunc count=1 bs=1024000' (cwd None)
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:36,752::storage_mailbox::488::Storage.Misc.excCmd::(__init__) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0409812 s, 25.0 MB/s\n'; <rc> = 0
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:36,753::storage_mailbox::493::Storage.MailBox.SpmMailMonitor::(__init__) SPM_MailMonitor created for pool c4e5889a-348d-11e2-83fe-9f72369eab30
>Dummy-15911115::DEBUG::2013-11-19 21:27:36,754::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:36,756::spm::523::Storage.SPM::(start) ended.
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:36,758::task::495::TaskManager.Task::(_debug) Task 3c078476-ceba-49ba-8c4f-cfc47f187488: moving from state running -> state finished
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:36,997::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:37,002::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:37,003::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:37,004::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:37,005::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:37,005::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:37,006::task::915::TaskManager.Task::(_runJobs) Task.run: exit - success: result 
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:37,007::task::495::TaskManager.Task::(_debug) Task 3c078476-ceba-49ba-8c4f-cfc47f187488: ref 0 aborting False
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-19 21:27:37,007::threadPool::47::Misc.ThreadPool::(setRunningTask) Number of running tasks: 0
>Dummy-15911115::DEBUG::2013-11-19 21:27:37,273::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0726459 s, 14.1 MB/s\n'; <rc> = 0
>Thread-15911116::DEBUG::2013-11-19 21:27:37,311::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911116::INFO::2013-11-19 21:27:37,312::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=3c078476-ceba-49ba-8c4f-cfc47f187488)
>Thread-15911116::DEBUG::2013-11-19 21:27:37,323::task::495::TaskManager.Task::(_debug) Task 82a2b289-f4ed-4475-91fa-6d21eb4e1365: moving from state init -> state preparing
>Thread-15911116::DEBUG::2013-11-19 21:27:37,323::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 3c078476-ceba-49ba-8c4f-cfc47f187488
>Thread-15911116::DEBUG::2013-11-19 21:27:37,327::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': '1 jobs completed successfuly', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}
>Thread-15911116::DEBUG::2013-11-19 21:27:37,328::task::495::TaskManager.Task::(_debug) Task 82a2b289-f4ed-4475-91fa-6d21eb4e1365: finished: {'taskStatus': {'code': 0, 'message': '1 jobs completed successfuly', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911116::DEBUG::2013-11-19 21:27:37,337::task::495::TaskManager.Task::(_debug) Task 82a2b289-f4ed-4475-91fa-6d21eb4e1365: moving from state preparing -> state finished
>Thread-15911116::DEBUG::2013-11-19 21:27:37,348::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911116::DEBUG::2013-11-19 21:27:37,351::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911116::DEBUG::2013-11-19 21:27:37,352::task::495::TaskManager.Task::(_debug) Task 82a2b289-f4ed-4475-91fa-6d21eb4e1365: ref 0 aborting False
>Thread-15911116::INFO::2013-11-19 21:27:37,352::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': '1 jobs completed successfuly', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '3c078476-ceba-49ba-8c4f-cfc47f187488'}}
>Thread-15911117::DEBUG::2013-11-19 21:27:37,392::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911117::INFO::2013-11-19 21:27:37,396::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911117::DEBUG::2013-11-19 21:27:37,400::task::495::TaskManager.Task::(_debug) Task aac575fb-e9f8-40b9-90a7-d9cc469b549e: moving from state init -> state preparing
>Thread-15911117::DEBUG::2013-11-19 21:27:37,412::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=282 spmId=2
>Thread-15911117::DEBUG::2013-11-19 21:27:37,417::task::495::TaskManager.Task::(_debug) Task aac575fb-e9f8-40b9-90a7-d9cc469b549e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911117::DEBUG::2013-11-19 21:27:37,434::task::495::TaskManager.Task::(_debug) Task aac575fb-e9f8-40b9-90a7-d9cc469b549e: moving from state preparing -> state finished
>Thread-15911117::DEBUG::2013-11-19 21:27:37,448::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911117::DEBUG::2013-11-19 21:27:37,457::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911117::DEBUG::2013-11-19 21:27:37,457::task::495::TaskManager.Task::(_debug) Task aac575fb-e9f8-40b9-90a7-d9cc469b549e: ref 0 aborting False
>Thread-15911117::INFO::2013-11-19 21:27:37,458::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911118::DEBUG::2013-11-19 21:27:37,502::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911118::INFO::2013-11-19 21:27:37,512::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: clearTask, args: ( taskID=3c078476-ceba-49ba-8c4f-cfc47f187488)
>Thread-15911118::DEBUG::2013-11-19 21:27:37,514::task::495::TaskManager.Task::(_debug) Task abf8fba0-a9c2-4710-8ea0-cbca5cef098e: moving from state init -> state preparing
>Thread-15911118::DEBUG::2013-11-19 21:27:37,517::taskManager::146::TaskManager::(clearTask) Entry. taskID: 3c078476-ceba-49ba-8c4f-cfc47f187488
>Thread-15911118::DEBUG::2013-11-19 21:27:37,520::taskManager::151::TaskManager::(clearTask) Return.
>Thread-15911118::DEBUG::2013-11-19 21:27:37,526::task::495::TaskManager.Task::(_debug) Task abf8fba0-a9c2-4710-8ea0-cbca5cef098e: finished: None
>Thread-15911118::DEBUG::2013-11-19 21:27:37,527::task::495::TaskManager.Task::(_debug) Task abf8fba0-a9c2-4710-8ea0-cbca5cef098e: moving from state preparing -> state finished
>Thread-15911118::DEBUG::2013-11-19 21:27:37,531::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911118::DEBUG::2013-11-19 21:27:37,539::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911118::DEBUG::2013-11-19 21:27:37,547::task::495::TaskManager.Task::(_debug) Task abf8fba0-a9c2-4710-8ea0-cbca5cef098e: ref 0 aborting False
>Thread-15911118::INFO::2013-11-19 21:27:37,550::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: clearTask, Return response: {'status': {'message': 'OK', 'code': 0}}
>Thread-15911119::DEBUG::2013-11-19 21:27:37,646::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911119::INFO::2013-11-19 21:27:37,655::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: deactivateStorageDomain, args: ( sdUUID=5d7b5178-3790-472c-a13b-b481eb27a61e spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 msdUUID=00000000-0000-0000-0000-000000000000 masterVersion=1)
>Thread-15911119::DEBUG::2013-11-19 21:27:37,665::task::495::TaskManager.Task::(_debug) Task 332ce334-59da-4235-95f9-b0b20113b537: moving from state init -> state preparing
>Thread-15911119::DEBUG::2013-11-19 21:27:37,683::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1beaebe7-35e9-4c58-bfe7-5ebfcbb82552`::Request was made in '/usr/share/vdsm/storage/spm.py' line '1112' at 'public_deactivateStorageDomain'
>Thread-15911119::DEBUG::2013-11-19 21:27:37,697::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'exclusive'
>Thread-15911119::DEBUG::2013-11-19 21:27:37,698::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'exclusive' (1 active user)
>Thread-15911119::DEBUG::2013-11-19 21:27:37,698::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1beaebe7-35e9-4c58-bfe7-5ebfcbb82552`::Granted request
>Thread-15911119::DEBUG::2013-11-19 21:27:37,705::task::495::TaskManager.Task::(_debug) Task 332ce334-59da-4235-95f9-b0b20113b537: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (exclusive)
>Thread-15911119::DEBUG::2013-11-19 21:27:37,710::task::495::TaskManager.Task::(_debug) Task 332ce334-59da-4235-95f9-b0b20113b537: ref 1 aborting False
>Thread-15911119::DEBUG::2013-11-19 21:27:37,719::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.5d7b5178-3790-472c-a13b-b481eb27a61e`ReqID=`7702d961-8576-4465-b53a-6dbfab16f035`::Request was made in '/usr/share/vdsm/storage/spm.py' line '1113' at 'public_deactivateStorageDomain'
>Thread-15911119::DEBUG::2013-11-19 21:27:37,744::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e' for lock type 'exclusive'
>Thread-15911119::DEBUG::2013-11-19 21:27:37,750::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e' is free. Now locking as 'exclusive' (1 active user)
>Thread-15911119::DEBUG::2013-11-19 21:27:37,755::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.5d7b5178-3790-472c-a13b-b481eb27a61e`ReqID=`7702d961-8576-4465-b53a-6dbfab16f035`::Granted request
>Thread-15911119::DEBUG::2013-11-19 21:27:37,764::task::495::TaskManager.Task::(_debug) Task 332ce334-59da-4235-95f9-b0b20113b537: _resourcesAcquired: Storage.5d7b5178-3790-472c-a13b-b481eb27a61e (exclusive)
>Thread-15911119::DEBUG::2013-11-19 21:27:37,775::task::495::TaskManager.Task::(_debug) Task 332ce334-59da-4235-95f9-b0b20113b537: ref 1 aborting False
>Thread-15911119::INFO::2013-11-19 21:27:37,785::sp::775::Storage.StoragePool::(deactivateSD) sdUUID=5d7b5178-3790-472c-a13b-b481eb27a61e spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 new_msdUUID=00000000-0000-0000-0000-000000000000
>Thread-15911119::INFO::2013-11-19 21:27:37,797::fileSD::176::Storage.StorageDomain::(validate) sdUUID=5d7b5178-3790-472c-a13b-b481eb27a61e
>Thread-15911119::DEBUG::2013-11-19 21:27:37,808::persistentDict::204::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=[]
>Thread-15911119::WARNING::2013-11-19 21:27:37,811::persistentDict::226::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is
>Thread-15911119::ERROR::2013-11-19 21:27:37,811::task::868::TaskManager.Task::(_setError) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/task.py", line 876, in _run
>  File "/usr/share/vdsm/storage/spm.py", line 115, in run
>  File "/usr/share/vdsm/storage/spm.py", line 1115, in public_deactivateStorageDomain
>  File "/usr/share/vdsm/storage/sp.py", line 798, in deactivateSD
>  File "/usr/share/vdsm/storage/sd.py", line 666, in isBackup
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911119::DEBUG::2013-11-19 21:27:37,830::task::495::TaskManager.Task::(_debug) Task 332ce334-59da-4235-95f9-b0b20113b537: Task._run: 332ce334-59da-4235-95f9-b0b20113b537 ('5d7b5178-3790-472c-a13b-b481eb27a61e', 'c4e5889a-348d-11e2-83fe-9f72369eab30', '00000000-0000-0000-0000-000000000000', 1) {} failed - stopping task
>Thread-15911119::DEBUG::2013-11-19 21:27:37,846::task::495::TaskManager.Task::(_debug) Task 332ce334-59da-4235-95f9-b0b20113b537: stopping in state preparing (force False)
>Thread-15911119::DEBUG::2013-11-19 21:27:37,846::task::495::TaskManager.Task::(_debug) Task 332ce334-59da-4235-95f9-b0b20113b537: ref 1 aborting True
>Thread-15911119::INFO::2013-11-19 21:27:37,853::task::1171::TaskManager.Task::(prepare) aborting: Task is aborted: "'CLASS'" - code 100
>Thread-15911119::DEBUG::2013-11-19 21:27:37,854::task::495::TaskManager.Task::(_debug) Task 332ce334-59da-4235-95f9-b0b20113b537: Prepare: aborted: 'CLASS'
>Thread-15911119::DEBUG::2013-11-19 21:27:37,854::task::495::TaskManager.Task::(_debug) Task 332ce334-59da-4235-95f9-b0b20113b537: ref 0 aborting True
>Thread-15911119::DEBUG::2013-11-19 21:27:37,857::task::495::TaskManager.Task::(_debug) Task 332ce334-59da-4235-95f9-b0b20113b537: Task._doAbort: force False
>Thread-15911119::DEBUG::2013-11-19 21:27:37,858::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911119::DEBUG::2013-11-19 21:27:37,861::task::495::TaskManager.Task::(_debug) Task 332ce334-59da-4235-95f9-b0b20113b537: moving from state preparing -> state aborting
>Thread-15911119::DEBUG::2013-11-19 21:27:37,861::task::495::TaskManager.Task::(_debug) Task 332ce334-59da-4235-95f9-b0b20113b537: _aborting: recover policy none
>Thread-15911119::DEBUG::2013-11-19 21:27:37,862::task::495::TaskManager.Task::(_debug) Task 332ce334-59da-4235-95f9-b0b20113b537: moving from state aborting -> state failed
>Thread-15911119::DEBUG::2013-11-19 21:27:37,863::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>, 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e': < ResourceRef 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e', isValid: 'True' obj: 'None'>}
>Thread-15911119::DEBUG::2013-11-19 21:27:37,866::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911119::DEBUG::2013-11-19 21:27:37,869::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911119::DEBUG::2013-11-19 21:27:37,873::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911119::DEBUG::2013-11-19 21:27:37,879::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911119::DEBUG::2013-11-19 21:27:37,879::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911119::DEBUG::2013-11-19 21:27:37,883::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e'
>Thread-15911119::DEBUG::2013-11-19 21:27:37,884::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e' (0 active users)
>Thread-15911119::DEBUG::2013-11-19 21:27:37,889::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e' is free, finding out if anyone is waiting for it.
>Thread-15911119::DEBUG::2013-11-19 21:27:37,889::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e', Clearing records.
>Thread-15911119::ERROR::2013-11-19 21:27:37,890::dispatcher::106::Storage.Dispatcher.Protect::(run) 'CLASS'
>Thread-15911119::ERROR::2013-11-19 21:27:37,891::dispatcher::107::Storage.Dispatcher.Protect::(run) Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/dispatcher.py", line 96, in run
>  File "/usr/share/vdsm/storage/task.py", line 1178, in prepare
>KeyError: 'CLASS'
>
>Thread-15911120::DEBUG::2013-11-19 21:27:37,917::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911120::INFO::2013-11-19 21:27:37,918::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksInfo, args: ()
>Thread-15911120::DEBUG::2013-11-19 21:27:37,918::task::495::TaskManager.Task::(_debug) Task de0e06d1-14d7-4e6c-a16e-c621d6b7ab2a: moving from state init -> state preparing
>Thread-15911120::DEBUG::2013-11-19 21:27:37,919::taskManager::168::TaskManager::(getAllTasksInfo) Entry.
>Thread-15911120::DEBUG::2013-11-19 21:27:37,924::taskManager::173::TaskManager::(getAllTasksInfo) Return. Response: {}
>Thread-15911120::DEBUG::2013-11-19 21:27:37,924::task::495::TaskManager.Task::(_debug) Task de0e06d1-14d7-4e6c-a16e-c621d6b7ab2a: finished: {'allTasksInfo': {}}
>Thread-15911120::DEBUG::2013-11-19 21:27:37,925::task::495::TaskManager.Task::(_debug) Task de0e06d1-14d7-4e6c-a16e-c621d6b7ab2a: moving from state preparing -> state finished
>Thread-15911120::DEBUG::2013-11-19 21:27:37,930::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911120::DEBUG::2013-11-19 21:27:37,935::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911120::DEBUG::2013-11-19 21:27:37,936::task::495::TaskManager.Task::(_debug) Task de0e06d1-14d7-4e6c-a16e-c621d6b7ab2a: ref 0 aborting False
>Thread-15911120::INFO::2013-11-19 21:27:37,936::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'allTasksInfo': {}}
>Thread-15911121::DEBUG::2013-11-19 21:27:37,976::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911121::INFO::2013-11-19 21:27:37,980::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911121::DEBUG::2013-11-19 21:27:37,981::task::495::TaskManager.Task::(_debug) Task 946f43a2-a4d7-4e9b-8c22-885ae40b10f2: moving from state init -> state preparing
>Thread-15911121::DEBUG::2013-11-19 21:27:37,986::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=282 spmId=2
>Thread-15911121::DEBUG::2013-11-19 21:27:37,986::task::495::TaskManager.Task::(_debug) Task 946f43a2-a4d7-4e9b-8c22-885ae40b10f2: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911121::DEBUG::2013-11-19 21:27:37,987::task::495::TaskManager.Task::(_debug) Task 946f43a2-a4d7-4e9b-8c22-885ae40b10f2: moving from state preparing -> state finished
>Thread-15911121::DEBUG::2013-11-19 21:27:37,993::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911121::DEBUG::2013-11-19 21:27:37,993::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911121::DEBUG::2013-11-19 21:27:37,994::task::495::TaskManager.Task::(_debug) Task 946f43a2-a4d7-4e9b-8c22-885ae40b10f2: ref 0 aborting False
>Thread-15911121::INFO::2013-11-19 21:27:37,995::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911122::DEBUG::2013-11-19 21:27:38,026::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911122::INFO::2013-11-19 21:27:38,031::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911122::DEBUG::2013-11-19 21:27:38,031::task::495::TaskManager.Task::(_debug) Task a4962d61-42e7-46d8-a26f-1360e43651e9: moving from state init -> state preparing
>Thread-15911122::DEBUG::2013-11-19 21:27:38,042::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9a8bbb37-59ab-48ea-bb3a-5df4da3436a4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911122::DEBUG::2013-11-19 21:27:38,046::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911122::DEBUG::2013-11-19 21:27:38,047::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911122::DEBUG::2013-11-19 21:27:38,050::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9a8bbb37-59ab-48ea-bb3a-5df4da3436a4`::Granted request
>Thread-15911122::DEBUG::2013-11-19 21:27:38,059::task::495::TaskManager.Task::(_debug) Task a4962d61-42e7-46d8-a26f-1360e43651e9: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911122::DEBUG::2013-11-19 21:27:38,060::task::495::TaskManager.Task::(_debug) Task a4962d61-42e7-46d8-a26f-1360e43651e9: ref 1 aborting False
>Thread-15911122::WARNING::2013-11-19 21:27:38,083::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911122::DEBUG::2013-11-19 21:27:38,101::task::495::TaskManager.Task::(_debug) Task a4962d61-42e7-46d8-a26f-1360e43651e9: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911122::DEBUG::2013-11-19 21:27:38,118::task::495::TaskManager.Task::(_debug) Task a4962d61-42e7-46d8-a26f-1360e43651e9: moving from state preparing -> state finished
>Thread-15911122::DEBUG::2013-11-19 21:27:38,129::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911122::DEBUG::2013-11-19 21:27:38,137::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911122::DEBUG::2013-11-19 21:27:38,138::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911122::DEBUG::2013-11-19 21:27:38,141::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911122::DEBUG::2013-11-19 21:27:38,142::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911122::DEBUG::2013-11-19 21:27:38,147::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911122::DEBUG::2013-11-19 21:27:38,148::task::495::TaskManager.Task::(_debug) Task a4962d61-42e7-46d8-a26f-1360e43651e9: ref 0 aborting False
>Thread-15911122::INFO::2013-11-19 21:27:38,156::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911123::DEBUG::2013-11-19 21:27:38,330::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911123::INFO::2013-11-19 21:27:38,339::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksInfo, args: ()
>Thread-15911123::DEBUG::2013-11-19 21:27:38,339::task::495::TaskManager.Task::(_debug) Task 59fbbf91-b166-437f-9b13-5e671437cbc6: moving from state init -> state preparing
>Thread-15911123::DEBUG::2013-11-19 21:27:38,340::taskManager::168::TaskManager::(getAllTasksInfo) Entry.
>Thread-15911123::DEBUG::2013-11-19 21:27:38,340::taskManager::173::TaskManager::(getAllTasksInfo) Return. Response: {}
>Thread-15911123::DEBUG::2013-11-19 21:27:38,341::task::495::TaskManager.Task::(_debug) Task 59fbbf91-b166-437f-9b13-5e671437cbc6: finished: {'allTasksInfo': {}}
>Thread-15911123::DEBUG::2013-11-19 21:27:38,342::task::495::TaskManager.Task::(_debug) Task 59fbbf91-b166-437f-9b13-5e671437cbc6: moving from state preparing -> state finished
>Thread-15911123::DEBUG::2013-11-19 21:27:38,342::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911123::DEBUG::2013-11-19 21:27:38,352::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911123::DEBUG::2013-11-19 21:27:38,355::task::495::TaskManager.Task::(_debug) Task 59fbbf91-b166-437f-9b13-5e671437cbc6: ref 0 aborting False
>Thread-15911123::INFO::2013-11-19 21:27:38,356::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'allTasksInfo': {}}
>Dummy-15911115::DEBUG::2013-11-19 21:27:39,289::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:27:39,830::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0741424 s, 13.8 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:27:41,096::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911115::DEBUG::2013-11-19 21:27:41,846::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:27:42,377::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0732462 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:27:44,391::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:27:44,938::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0733825 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:27:46,954::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:27:47,486::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0733239 s, 14.0 MB/s\n'; <rc> = 0
>Thread-15911128::INFO::2013-11-19 21:27:48,188::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911128::DEBUG::2013-11-19 21:27:48,190::task::495::TaskManager.Task::(_debug) Task 0dac1488-d47d-4625-9408-4765b1a597c7: moving from state init -> state preparing
>Thread-15911128::DEBUG::2013-11-19 21:27:48,190::task::495::TaskManager.Task::(_debug) Task 0dac1488-d47d-4625-9408-4765b1a597c7: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000728845596313', 'lastCheck': 1384896466.1345589, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00233507156372', 'lastCheck': 1384896465.7145, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896461.0970981, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000458955764771', 'lastCheck': 1384896461.6780951, 'valid': True, 'code': 0}}
>Thread-15911128::DEBUG::2013-11-19 21:27:48,207::task::495::TaskManager.Task::(_debug) Task 0dac1488-d47d-4625-9408-4765b1a597c7: moving from state preparing -> state finished
>Thread-15911128::DEBUG::2013-11-19 21:27:48,217::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911128::DEBUG::2013-11-19 21:27:48,228::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911128::DEBUG::2013-11-19 21:27:48,234::task::495::TaskManager.Task::(_debug) Task 0dac1488-d47d-4625-9408-4765b1a597c7: ref 0 aborting False
>Thread-15911128::INFO::2013-11-19 21:27:48,244::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000728845596313', 'lastCheck': 1384896466.1345589, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00233507156372', 'lastCheck': 1384896465.7145, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896461.0970981, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000458955764771', 'lastCheck': 1384896461.6780951, 'valid': True, 'code': 0}}
>Thread-15911130::DEBUG::2013-11-19 21:27:48,323::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911130::INFO::2013-11-19 21:27:48,330::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911130::DEBUG::2013-11-19 21:27:48,334::task::495::TaskManager.Task::(_debug) Task 8eb88e78-826a-46cd-bd99-79ab1b47fb8e: moving from state init -> state preparing
>Thread-15911130::DEBUG::2013-11-19 21:27:48,335::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=282 spmId=2
>Thread-15911130::DEBUG::2013-11-19 21:27:48,336::task::495::TaskManager.Task::(_debug) Task 8eb88e78-826a-46cd-bd99-79ab1b47fb8e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911130::DEBUG::2013-11-19 21:27:48,336::task::495::TaskManager.Task::(_debug) Task 8eb88e78-826a-46cd-bd99-79ab1b47fb8e: moving from state preparing -> state finished
>Thread-15911130::DEBUG::2013-11-19 21:27:48,337::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911130::DEBUG::2013-11-19 21:27:48,343::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911130::DEBUG::2013-11-19 21:27:48,347::task::495::TaskManager.Task::(_debug) Task 8eb88e78-826a-46cd-bd99-79ab1b47fb8e: ref 0 aborting False
>Thread-15911130::INFO::2013-11-19 21:27:48,351::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911131::DEBUG::2013-11-19 21:27:48,394::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911131::INFO::2013-11-19 21:27:48,398::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911131::DEBUG::2013-11-19 21:27:48,398::task::495::TaskManager.Task::(_debug) Task 09f5de93-ff56-4db5-bd43-84697aa36d28: moving from state init -> state preparing
>Thread-15911131::DEBUG::2013-11-19 21:27:48,399::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6f1334bb-a250-4d4f-97ad-bbb35aa91bbc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911131::DEBUG::2013-11-19 21:27:48,415::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911131::DEBUG::2013-11-19 21:27:48,421::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911131::DEBUG::2013-11-19 21:27:48,427::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6f1334bb-a250-4d4f-97ad-bbb35aa91bbc`::Granted request
>Thread-15911131::DEBUG::2013-11-19 21:27:48,435::task::495::TaskManager.Task::(_debug) Task 09f5de93-ff56-4db5-bd43-84697aa36d28: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911131::DEBUG::2013-11-19 21:27:48,450::task::495::TaskManager.Task::(_debug) Task 09f5de93-ff56-4db5-bd43-84697aa36d28: ref 1 aborting False
>Thread-15911131::WARNING::2013-11-19 21:27:48,465::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911131::DEBUG::2013-11-19 21:27:48,479::task::495::TaskManager.Task::(_debug) Task 09f5de93-ff56-4db5-bd43-84697aa36d28: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911131::DEBUG::2013-11-19 21:27:48,515::task::495::TaskManager.Task::(_debug) Task 09f5de93-ff56-4db5-bd43-84697aa36d28: moving from state preparing -> state finished
>Thread-15911131::DEBUG::2013-11-19 21:27:48,517::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911131::DEBUG::2013-11-19 21:27:48,517::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911131::DEBUG::2013-11-19 21:27:48,518::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911131::DEBUG::2013-11-19 21:27:48,519::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911131::DEBUG::2013-11-19 21:27:48,519::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911131::DEBUG::2013-11-19 21:27:48,525::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911131::DEBUG::2013-11-19 21:27:48,525::task::495::TaskManager.Task::(_debug) Task 09f5de93-ff56-4db5-bd43-84697aa36d28: ref 0 aborting False
>Thread-15911131::INFO::2013-11-19 21:27:48,533::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911115::DEBUG::2013-11-19 21:27:49,502::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:27:50,019::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0732388 s, 14.0 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:27:51,194::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911115::DEBUG::2013-11-19 21:27:52,034::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:27:52,572::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0737843 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:27:54,588::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:27:55,142::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.102934 s, 9.9 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:27:57,157::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:27:57,644::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0704051 s, 14.5 MB/s\n'; <rc> = 0
>Thread-15911136::INFO::2013-11-19 21:27:58,509::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911136::DEBUG::2013-11-19 21:27:58,510::task::495::TaskManager.Task::(_debug) Task f605972b-9bda-4649-9c32-156f938935ef: moving from state init -> state preparing
>Thread-15911136::DEBUG::2013-11-19 21:27:58,516::task::495::TaskManager.Task::(_debug) Task f605972b-9bda-4649-9c32-156f938935ef: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000345945358276', 'lastCheck': 1384896476.2618029, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00156188011169', 'lastCheck': 1384896475.801497, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896471.1948969, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000381946563721', 'lastCheck': 1384896471.7057941, 'valid': True, 'code': 0}}
>Thread-15911136::DEBUG::2013-11-19 21:27:58,525::task::495::TaskManager.Task::(_debug) Task f605972b-9bda-4649-9c32-156f938935ef: moving from state preparing -> state finished
>Thread-15911136::DEBUG::2013-11-19 21:27:58,531::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911136::DEBUG::2013-11-19 21:27:58,535::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911136::DEBUG::2013-11-19 21:27:58,538::task::495::TaskManager.Task::(_debug) Task f605972b-9bda-4649-9c32-156f938935ef: ref 0 aborting False
>Thread-15911136::INFO::2013-11-19 21:27:58,541::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000345945358276', 'lastCheck': 1384896476.2618029, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00156188011169', 'lastCheck': 1384896475.801497, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896471.1948969, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000381946563721', 'lastCheck': 1384896471.7057941, 'valid': True, 'code': 0}}
>Thread-15911138::DEBUG::2013-11-19 21:27:58,689::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911138::INFO::2013-11-19 21:27:58,694::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911138::DEBUG::2013-11-19 21:27:58,695::task::495::TaskManager.Task::(_debug) Task 2276a252-8fe7-4f3c-bb89-a13183e151f5: moving from state init -> state preparing
>Thread-15911138::DEBUG::2013-11-19 21:27:58,696::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=282 spmId=2
>Thread-15911138::DEBUG::2013-11-19 21:27:58,700::task::495::TaskManager.Task::(_debug) Task 2276a252-8fe7-4f3c-bb89-a13183e151f5: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911138::DEBUG::2013-11-19 21:27:58,700::task::495::TaskManager.Task::(_debug) Task 2276a252-8fe7-4f3c-bb89-a13183e151f5: moving from state preparing -> state finished
>Thread-15911138::DEBUG::2013-11-19 21:27:58,701::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911138::DEBUG::2013-11-19 21:27:58,706::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911138::DEBUG::2013-11-19 21:27:58,707::task::495::TaskManager.Task::(_debug) Task 2276a252-8fe7-4f3c-bb89-a13183e151f5: ref 0 aborting False
>Thread-15911138::INFO::2013-11-19 21:27:58,707::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911139::DEBUG::2013-11-19 21:27:58,760::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911139::INFO::2013-11-19 21:27:58,761::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911139::DEBUG::2013-11-19 21:27:58,761::task::495::TaskManager.Task::(_debug) Task 9026fdae-aae0-4932-bd74-55d362093df7: moving from state init -> state preparing
>Thread-15911139::DEBUG::2013-11-19 21:27:58,771::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c0a24abb-9dab-460b-a53f-9034aa569527`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911139::DEBUG::2013-11-19 21:27:58,777::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911139::DEBUG::2013-11-19 21:27:58,778::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911139::DEBUG::2013-11-19 21:27:58,778::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c0a24abb-9dab-460b-a53f-9034aa569527`::Granted request
>Thread-15911139::DEBUG::2013-11-19 21:27:58,782::task::495::TaskManager.Task::(_debug) Task 9026fdae-aae0-4932-bd74-55d362093df7: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911139::DEBUG::2013-11-19 21:27:58,783::task::495::TaskManager.Task::(_debug) Task 9026fdae-aae0-4932-bd74-55d362093df7: ref 1 aborting False
>Thread-15911139::WARNING::2013-11-19 21:27:58,794::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911139::DEBUG::2013-11-19 21:27:58,809::task::495::TaskManager.Task::(_debug) Task 9026fdae-aae0-4932-bd74-55d362093df7: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911139::DEBUG::2013-11-19 21:27:58,817::task::495::TaskManager.Task::(_debug) Task 9026fdae-aae0-4932-bd74-55d362093df7: moving from state preparing -> state finished
>Thread-15911139::DEBUG::2013-11-19 21:27:58,821::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911139::DEBUG::2013-11-19 21:27:58,824::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911139::DEBUG::2013-11-19 21:27:58,829::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911139::DEBUG::2013-11-19 21:27:58,840::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911139::DEBUG::2013-11-19 21:27:58,841::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911139::DEBUG::2013-11-19 21:27:58,841::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911139::DEBUG::2013-11-19 21:27:58,842::task::495::TaskManager.Task::(_debug) Task 9026fdae-aae0-4932-bd74-55d362093df7: ref 0 aborting False
>Thread-15911139::INFO::2013-11-19 21:27:58,847::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911115::DEBUG::2013-11-19 21:27:59,660::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:28:00,218::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0834281 s, 12.3 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:28:01,399::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911115::DEBUG::2013-11-19 21:28:02,233::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:28:02,772::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0897622 s, 11.4 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:28:04,787::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:28:05,322::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0736538 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:28:07,338::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:28:07,863::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0739876 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15911144::INFO::2013-11-19 21:28:08,762::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911144::DEBUG::2013-11-19 21:28:08,763::task::495::TaskManager.Task::(_debug) Task 51dd015a-5eb7-4ca4-805d-9ff18190059a: moving from state init -> state preparing
>Thread-15911144::DEBUG::2013-11-19 21:28:08,764::task::495::TaskManager.Task::(_debug) Task 51dd015a-5eb7-4ca4-805d-9ff18190059a: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000401973724365', 'lastCheck': 1384896486.4470141, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00157189369202', 'lastCheck': 1384896486.0126951, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896481.4001191, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000561952590942', 'lastCheck': 1384896481.8916891, 'valid': True, 'code': 0}}
>Thread-15911144::DEBUG::2013-11-19 21:28:08,770::task::495::TaskManager.Task::(_debug) Task 51dd015a-5eb7-4ca4-805d-9ff18190059a: moving from state preparing -> state finished
>Thread-15911144::DEBUG::2013-11-19 21:28:08,770::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911144::DEBUG::2013-11-19 21:28:08,777::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911144::DEBUG::2013-11-19 21:28:08,777::task::495::TaskManager.Task::(_debug) Task 51dd015a-5eb7-4ca4-805d-9ff18190059a: ref 0 aborting False
>Thread-15911144::INFO::2013-11-19 21:28:08,778::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000401973724365', 'lastCheck': 1384896486.4470141, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00157189369202', 'lastCheck': 1384896486.0126951, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896481.4001191, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000561952590942', 'lastCheck': 1384896481.8916891, 'valid': True, 'code': 0}}
>Thread-15911146::DEBUG::2013-11-19 21:28:08,990::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911146::INFO::2013-11-19 21:28:08,991::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911146::DEBUG::2013-11-19 21:28:08,991::task::495::TaskManager.Task::(_debug) Task d0c028cf-f34a-44b2-b271-6ced113b08ac: moving from state init -> state preparing
>Thread-15911146::DEBUG::2013-11-19 21:28:08,992::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=282 spmId=2
>Thread-15911146::DEBUG::2013-11-19 21:28:08,993::task::495::TaskManager.Task::(_debug) Task d0c028cf-f34a-44b2-b271-6ced113b08ac: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911146::DEBUG::2013-11-19 21:28:08,997::task::495::TaskManager.Task::(_debug) Task d0c028cf-f34a-44b2-b271-6ced113b08ac: moving from state preparing -> state finished
>Thread-15911146::DEBUG::2013-11-19 21:28:09,003::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911146::DEBUG::2013-11-19 21:28:09,003::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911146::DEBUG::2013-11-19 21:28:09,004::task::495::TaskManager.Task::(_debug) Task d0c028cf-f34a-44b2-b271-6ced113b08ac: ref 0 aborting False
>Thread-15911146::INFO::2013-11-19 21:28:09,009::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911147::DEBUG::2013-11-19 21:28:09,039::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911147::INFO::2013-11-19 21:28:09,040::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911147::DEBUG::2013-11-19 21:28:09,041::task::495::TaskManager.Task::(_debug) Task 8b2dbed3-cf6f-4a57-a71e-290614f99032: moving from state init -> state preparing
>Thread-15911147::DEBUG::2013-11-19 21:28:09,044::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ac163f13-782f-45fb-be97-e3266ae29eba`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911147::DEBUG::2013-11-19 21:28:09,055::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911147::DEBUG::2013-11-19 21:28:09,056::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911147::DEBUG::2013-11-19 21:28:09,057::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ac163f13-782f-45fb-be97-e3266ae29eba`::Granted request
>Thread-15911147::DEBUG::2013-11-19 21:28:09,058::task::495::TaskManager.Task::(_debug) Task 8b2dbed3-cf6f-4a57-a71e-290614f99032: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911147::DEBUG::2013-11-19 21:28:09,064::task::495::TaskManager.Task::(_debug) Task 8b2dbed3-cf6f-4a57-a71e-290614f99032: ref 1 aborting False
>Thread-15911147::WARNING::2013-11-19 21:28:09,081::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911147::DEBUG::2013-11-19 21:28:09,102::task::495::TaskManager.Task::(_debug) Task 8b2dbed3-cf6f-4a57-a71e-290614f99032: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911147::DEBUG::2013-11-19 21:28:09,134::task::495::TaskManager.Task::(_debug) Task 8b2dbed3-cf6f-4a57-a71e-290614f99032: moving from state preparing -> state finished
>Thread-15911147::DEBUG::2013-11-19 21:28:09,135::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911147::DEBUG::2013-11-19 21:28:09,135::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911147::DEBUG::2013-11-19 21:28:09,143::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911147::DEBUG::2013-11-19 21:28:09,144::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911147::DEBUG::2013-11-19 21:28:09,150::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911147::DEBUG::2013-11-19 21:28:09,151::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911147::DEBUG::2013-11-19 21:28:09,151::task::495::TaskManager.Task::(_debug) Task 8b2dbed3-cf6f-4a57-a71e-290614f99032: ref 0 aborting False
>Thread-15911147::INFO::2013-11-19 21:28:09,152::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911115::DEBUG::2013-11-19 21:28:09,878::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:28:10,395::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0726088 s, 14.1 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:28:11,551::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911115::DEBUG::2013-11-19 21:28:12,410::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:28:12,986::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.09985 s, 10.3 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:28:15,001::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:28:15,533::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0836241 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:28:17,549::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:28:18,094::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0799157 s, 12.8 MB/s\n'; <rc> = 0
>Thread-15911152::INFO::2013-11-19 21:28:18,987::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911152::DEBUG::2013-11-19 21:28:18,993::task::495::TaskManager.Task::(_debug) Task a5b6d3e1-77fe-40b8-9758-ca8f155e576e: moving from state init -> state preparing
>Thread-15911152::DEBUG::2013-11-19 21:28:18,994::task::495::TaskManager.Task::(_debug) Task a5b6d3e1-77fe-40b8-9758-ca8f155e576e: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00037693977356', 'lastCheck': 1384896496.64905, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00151991844177', 'lastCheck': 1384896496.2208431, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896491.5521519, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00053596496582', 'lastCheck': 1384896492.0845261, 'valid': True, 'code': 0}}
>Thread-15911152::DEBUG::2013-11-19 21:28:19,001::task::495::TaskManager.Task::(_debug) Task a5b6d3e1-77fe-40b8-9758-ca8f155e576e: moving from state preparing -> state finished
>Thread-15911152::DEBUG::2013-11-19 21:28:19,001::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911152::DEBUG::2013-11-19 21:28:19,006::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911152::DEBUG::2013-11-19 21:28:19,007::task::495::TaskManager.Task::(_debug) Task a5b6d3e1-77fe-40b8-9758-ca8f155e576e: ref 0 aborting False
>Thread-15911152::INFO::2013-11-19 21:28:19,007::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00037693977356', 'lastCheck': 1384896496.64905, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00151991844177', 'lastCheck': 1384896496.2208431, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896491.5521519, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00053596496582', 'lastCheck': 1384896492.0845261, 'valid': True, 'code': 0}}
>Thread-15911154::DEBUG::2013-11-19 21:28:19,282::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911154::INFO::2013-11-19 21:28:19,283::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911154::DEBUG::2013-11-19 21:28:19,287::task::495::TaskManager.Task::(_debug) Task b7db5e3d-df52-43e6-8908-506215ad9cf6: moving from state init -> state preparing
>Thread-15911154::DEBUG::2013-11-19 21:28:19,293::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=282 spmId=2
>Thread-15911154::DEBUG::2013-11-19 21:28:19,293::task::495::TaskManager.Task::(_debug) Task b7db5e3d-df52-43e6-8908-506215ad9cf6: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911154::DEBUG::2013-11-19 21:28:19,294::task::495::TaskManager.Task::(_debug) Task b7db5e3d-df52-43e6-8908-506215ad9cf6: moving from state preparing -> state finished
>Thread-15911154::DEBUG::2013-11-19 21:28:19,294::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911154::DEBUG::2013-11-19 21:28:19,295::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911154::DEBUG::2013-11-19 21:28:19,295::task::495::TaskManager.Task::(_debug) Task b7db5e3d-df52-43e6-8908-506215ad9cf6: ref 0 aborting False
>Thread-15911154::INFO::2013-11-19 21:28:19,300::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911155::DEBUG::2013-11-19 21:28:19,341::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911155::INFO::2013-11-19 21:28:19,347::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911155::DEBUG::2013-11-19 21:28:19,351::task::495::TaskManager.Task::(_debug) Task 92e56e6d-50d1-493e-a388-92c9ed9f6264: moving from state init -> state preparing
>Thread-15911155::DEBUG::2013-11-19 21:28:19,356::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f9dbfcce-3181-4bca-a718-37abd97baf8f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911155::DEBUG::2013-11-19 21:28:19,357::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911155::DEBUG::2013-11-19 21:28:19,363::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911155::DEBUG::2013-11-19 21:28:19,364::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f9dbfcce-3181-4bca-a718-37abd97baf8f`::Granted request
>Thread-15911155::DEBUG::2013-11-19 21:28:19,370::task::495::TaskManager.Task::(_debug) Task 92e56e6d-50d1-493e-a388-92c9ed9f6264: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911155::DEBUG::2013-11-19 21:28:19,371::task::495::TaskManager.Task::(_debug) Task 92e56e6d-50d1-493e-a388-92c9ed9f6264: ref 1 aborting False
>Thread-15911155::WARNING::2013-11-19 21:28:19,388::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911155::DEBUG::2013-11-19 21:28:19,392::task::495::TaskManager.Task::(_debug) Task 92e56e6d-50d1-493e-a388-92c9ed9f6264: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911155::DEBUG::2013-11-19 21:28:19,402::task::495::TaskManager.Task::(_debug) Task 92e56e6d-50d1-493e-a388-92c9ed9f6264: moving from state preparing -> state finished
>Thread-15911155::DEBUG::2013-11-19 21:28:19,405::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911155::DEBUG::2013-11-19 21:28:19,408::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911155::DEBUG::2013-11-19 21:28:19,416::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911155::DEBUG::2013-11-19 21:28:19,417::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911155::DEBUG::2013-11-19 21:28:19,418::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911155::DEBUG::2013-11-19 21:28:19,418::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911155::DEBUG::2013-11-19 21:28:19,419::task::495::TaskManager.Task::(_debug) Task 92e56e6d-50d1-493e-a388-92c9ed9f6264: ref 0 aborting False
>Thread-15911155::INFO::2013-11-19 21:28:19,425::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911115::DEBUG::2013-11-19 21:28:20,110::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:28:20,628::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0731599 s, 14.0 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:28:21,776::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911115::DEBUG::2013-11-19 21:28:22,642::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:28:23,206::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0925146 s, 11.1 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:28:25,222::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:28:25,744::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0740442 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:28:27,759::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:28:28,288::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0741539 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15911160::INFO::2013-11-19 21:28:29,198::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911160::DEBUG::2013-11-19 21:28:29,199::task::495::TaskManager.Task::(_debug) Task a5d06c92-2c98-45b8-ba91-22909f6867cb: moving from state init -> state preparing
>Thread-15911160::DEBUG::2013-11-19 21:28:29,201::task::495::TaskManager.Task::(_debug) Task a5d06c92-2c98-45b8-ba91-22909f6867cb: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000377893447876', 'lastCheck': 1384896506.878119, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00151705741882', 'lastCheck': 1384896506.4331961, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896501.7776141, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000483989715576', 'lastCheck': 1384896502.3138571, 'valid': True, 'code': 0}}
>Thread-15911160::DEBUG::2013-11-19 21:28:29,211::task::495::TaskManager.Task::(_debug) Task a5d06c92-2c98-45b8-ba91-22909f6867cb: moving from state preparing -> state finished
>Thread-15911160::DEBUG::2013-11-19 21:28:29,211::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911160::DEBUG::2013-11-19 21:28:29,212::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911160::DEBUG::2013-11-19 21:28:29,212::task::495::TaskManager.Task::(_debug) Task a5d06c92-2c98-45b8-ba91-22909f6867cb: ref 0 aborting False
>Thread-15911160::INFO::2013-11-19 21:28:29,213::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000377893447876', 'lastCheck': 1384896506.878119, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00151705741882', 'lastCheck': 1384896506.4331961, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896501.7776141, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000483989715576', 'lastCheck': 1384896502.3138571, 'valid': True, 'code': 0}}
>Thread-15911162::DEBUG::2013-11-19 21:28:29,567::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911162::INFO::2013-11-19 21:28:29,568::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911162::DEBUG::2013-11-19 21:28:29,571::task::495::TaskManager.Task::(_debug) Task 921dd74e-f1e2-443b-9eee-fca423e5551e: moving from state init -> state preparing
>Thread-15911162::DEBUG::2013-11-19 21:28:29,571::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=282 spmId=2
>Thread-15911162::DEBUG::2013-11-19 21:28:29,574::task::495::TaskManager.Task::(_debug) Task 921dd74e-f1e2-443b-9eee-fca423e5551e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911162::DEBUG::2013-11-19 21:28:29,575::task::495::TaskManager.Task::(_debug) Task 921dd74e-f1e2-443b-9eee-fca423e5551e: moving from state preparing -> state finished
>Thread-15911162::DEBUG::2013-11-19 21:28:29,575::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911162::DEBUG::2013-11-19 21:28:29,576::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911162::DEBUG::2013-11-19 21:28:29,576::task::495::TaskManager.Task::(_debug) Task 921dd74e-f1e2-443b-9eee-fca423e5551e: ref 0 aborting False
>Thread-15911162::INFO::2013-11-19 21:28:29,582::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911163::DEBUG::2013-11-19 21:28:29,596::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911163::INFO::2013-11-19 21:28:29,597::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911163::DEBUG::2013-11-19 21:28:29,601::task::495::TaskManager.Task::(_debug) Task fc13eb4c-38f4-48fb-bcdb-8f0d3cbafcd1: moving from state init -> state preparing
>Thread-15911163::DEBUG::2013-11-19 21:28:29,602::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f62d47b4-e79a-4af4-a801-57b6bcdbcc8b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911163::DEBUG::2013-11-19 21:28:29,603::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911163::DEBUG::2013-11-19 21:28:29,605::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911163::DEBUG::2013-11-19 21:28:29,605::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f62d47b4-e79a-4af4-a801-57b6bcdbcc8b`::Granted request
>Thread-15911163::DEBUG::2013-11-19 21:28:29,612::task::495::TaskManager.Task::(_debug) Task fc13eb4c-38f4-48fb-bcdb-8f0d3cbafcd1: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911163::DEBUG::2013-11-19 21:28:29,613::task::495::TaskManager.Task::(_debug) Task fc13eb4c-38f4-48fb-bcdb-8f0d3cbafcd1: ref 1 aborting False
>Thread-15911163::WARNING::2013-11-19 21:28:29,620::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911163::DEBUG::2013-11-19 21:28:29,639::task::495::TaskManager.Task::(_debug) Task fc13eb4c-38f4-48fb-bcdb-8f0d3cbafcd1: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911163::DEBUG::2013-11-19 21:28:29,646::task::495::TaskManager.Task::(_debug) Task fc13eb4c-38f4-48fb-bcdb-8f0d3cbafcd1: moving from state preparing -> state finished
>Thread-15911163::DEBUG::2013-11-19 21:28:29,652::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911163::DEBUG::2013-11-19 21:28:29,652::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911163::DEBUG::2013-11-19 21:28:29,653::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911163::DEBUG::2013-11-19 21:28:29,654::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911163::DEBUG::2013-11-19 21:28:29,654::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911163::DEBUG::2013-11-19 21:28:29,655::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911163::DEBUG::2013-11-19 21:28:29,655::task::495::TaskManager.Task::(_debug) Task fc13eb4c-38f4-48fb-bcdb-8f0d3cbafcd1: ref 0 aborting False
>Thread-15911163::INFO::2013-11-19 21:28:29,661::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911115::DEBUG::2013-11-19 21:28:30,303::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:28:30,850::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0736112 s, 13.9 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:28:31,986::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911115::DEBUG::2013-11-19 21:28:32,866::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:28:33,396::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0870685 s, 11.8 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:28:35,411::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:28:35,948::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0740842 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:28:37,963::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:28:38,487::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0745398 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15911168::INFO::2013-11-19 21:28:39,430::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911168::DEBUG::2013-11-19 21:28:39,435::task::495::TaskManager.Task::(_debug) Task b2c5ddee-812b-4824-98e6-9a896575d49d: moving from state init -> state preparing
>Thread-15911168::DEBUG::2013-11-19 21:28:39,441::task::495::TaskManager.Task::(_debug) Task b2c5ddee-812b-4824-98e6-9a896575d49d: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000410079956055', 'lastCheck': 1384896517.0772691, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00151991844177', 'lastCheck': 1384896516.6382909, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896511.987448, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000568866729736', 'lastCheck': 1384896512.5396371, 'valid': True, 'code': 0}}
>Thread-15911168::DEBUG::2013-11-19 21:28:39,444::task::495::TaskManager.Task::(_debug) Task b2c5ddee-812b-4824-98e6-9a896575d49d: moving from state preparing -> state finished
>Thread-15911168::DEBUG::2013-11-19 21:28:39,450::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911168::DEBUG::2013-11-19 21:28:39,451::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911168::DEBUG::2013-11-19 21:28:39,451::task::495::TaskManager.Task::(_debug) Task b2c5ddee-812b-4824-98e6-9a896575d49d: ref 0 aborting False
>Thread-15911168::INFO::2013-11-19 21:28:39,452::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000410079956055', 'lastCheck': 1384896517.0772691, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00151991844177', 'lastCheck': 1384896516.6382909, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896511.987448, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000568866729736', 'lastCheck': 1384896512.5396371, 'valid': True, 'code': 0}}
>Thread-15911170::DEBUG::2013-11-19 21:28:39,799::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911170::INFO::2013-11-19 21:28:39,800::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911170::DEBUG::2013-11-19 21:28:39,801::task::495::TaskManager.Task::(_debug) Task 1db20086-86fd-4812-818d-ede90b828d10: moving from state init -> state preparing
>Thread-15911170::DEBUG::2013-11-19 21:28:39,807::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=282 spmId=2
>Thread-15911170::DEBUG::2013-11-19 21:28:39,811::task::495::TaskManager.Task::(_debug) Task 1db20086-86fd-4812-818d-ede90b828d10: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911170::DEBUG::2013-11-19 21:28:39,818::task::495::TaskManager.Task::(_debug) Task 1db20086-86fd-4812-818d-ede90b828d10: moving from state preparing -> state finished
>Thread-15911170::DEBUG::2013-11-19 21:28:39,818::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911170::DEBUG::2013-11-19 21:28:39,822::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911170::DEBUG::2013-11-19 21:28:39,823::task::495::TaskManager.Task::(_debug) Task 1db20086-86fd-4812-818d-ede90b828d10: ref 0 aborting False
>Thread-15911170::INFO::2013-11-19 21:28:39,825::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911171::DEBUG::2013-11-19 21:28:39,851::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911171::INFO::2013-11-19 21:28:39,852::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911171::DEBUG::2013-11-19 21:28:39,856::task::495::TaskManager.Task::(_debug) Task e3752623-146b-45a1-906a-5784aa8a2f43: moving from state init -> state preparing
>Thread-15911171::DEBUG::2013-11-19 21:28:39,869::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1cceb5c8-b95f-4136-8518-462874e5942e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911171::DEBUG::2013-11-19 21:28:39,875::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911171::DEBUG::2013-11-19 21:28:39,876::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911171::DEBUG::2013-11-19 21:28:39,877::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1cceb5c8-b95f-4136-8518-462874e5942e`::Granted request
>Thread-15911171::DEBUG::2013-11-19 21:28:39,888::task::495::TaskManager.Task::(_debug) Task e3752623-146b-45a1-906a-5784aa8a2f43: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911171::DEBUG::2013-11-19 21:28:39,889::task::495::TaskManager.Task::(_debug) Task e3752623-146b-45a1-906a-5784aa8a2f43: ref 1 aborting False
>Thread-15911171::WARNING::2013-11-19 21:28:39,899::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911171::DEBUG::2013-11-19 21:28:39,910::task::495::TaskManager.Task::(_debug) Task e3752623-146b-45a1-906a-5784aa8a2f43: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911171::DEBUG::2013-11-19 21:28:39,913::task::495::TaskManager.Task::(_debug) Task e3752623-146b-45a1-906a-5784aa8a2f43: moving from state preparing -> state finished
>Thread-15911171::DEBUG::2013-11-19 21:28:39,914::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911171::DEBUG::2013-11-19 21:28:39,917::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911171::DEBUG::2013-11-19 21:28:39,917::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911171::DEBUG::2013-11-19 21:28:39,918::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911171::DEBUG::2013-11-19 21:28:39,918::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911171::DEBUG::2013-11-19 21:28:39,919::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911171::DEBUG::2013-11-19 21:28:39,920::task::495::TaskManager.Task::(_debug) Task e3752623-146b-45a1-906a-5784aa8a2f43: ref 0 aborting False
>Thread-15911171::INFO::2013-11-19 21:28:39,920::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911115::DEBUG::2013-11-19 21:28:40,502::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:28:41,014::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0732723 s, 14.0 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:28:42,177::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911115::DEBUG::2013-11-19 21:28:43,030::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:28:43,583::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0739955 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:28:45,599::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:28:46,127::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0727015 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:28:48,150::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:28:48,692::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0745462 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15911176::INFO::2013-11-19 21:28:49,632::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911176::DEBUG::2013-11-19 21:28:49,633::task::495::TaskManager.Task::(_debug) Task 6e4def5b-4ce5-4e83-a53c-d278d19ac7cf: moving from state init -> state preparing
>Thread-15911176::DEBUG::2013-11-19 21:28:49,634::task::495::TaskManager.Task::(_debug) Task 6e4def5b-4ce5-4e83-a53c-d278d19ac7cf: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00042200088501', 'lastCheck': 1384896527.27424, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00242805480957', 'lastCheck': 1384896526.8190501, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896522.1777599, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000473976135254', 'lastCheck': 1384896522.700588, 'valid': True, 'code': 0}}
>Thread-15911176::DEBUG::2013-11-19 21:28:49,641::task::495::TaskManager.Task::(_debug) Task 6e4def5b-4ce5-4e83-a53c-d278d19ac7cf: moving from state preparing -> state finished
>Thread-15911176::DEBUG::2013-11-19 21:28:49,642::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911176::DEBUG::2013-11-19 21:28:49,643::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911176::DEBUG::2013-11-19 21:28:49,643::task::495::TaskManager.Task::(_debug) Task 6e4def5b-4ce5-4e83-a53c-d278d19ac7cf: ref 0 aborting False
>Thread-15911176::INFO::2013-11-19 21:28:49,644::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00042200088501', 'lastCheck': 1384896527.27424, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00242805480957', 'lastCheck': 1384896526.8190501, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896522.1777599, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000473976135254', 'lastCheck': 1384896522.700588, 'valid': True, 'code': 0}}
>Thread-15911178::DEBUG::2013-11-19 21:28:50,057::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911178::INFO::2013-11-19 21:28:50,058::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911178::DEBUG::2013-11-19 21:28:50,059::task::495::TaskManager.Task::(_debug) Task 84b2db63-119a-4b4a-b386-285178004e2a: moving from state init -> state preparing
>Thread-15911178::DEBUG::2013-11-19 21:28:50,059::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=282 spmId=2
>Thread-15911178::DEBUG::2013-11-19 21:28:50,060::task::495::TaskManager.Task::(_debug) Task 84b2db63-119a-4b4a-b386-285178004e2a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911178::DEBUG::2013-11-19 21:28:50,061::task::495::TaskManager.Task::(_debug) Task 84b2db63-119a-4b4a-b386-285178004e2a: moving from state preparing -> state finished
>Thread-15911178::DEBUG::2013-11-19 21:28:50,061::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911178::DEBUG::2013-11-19 21:28:50,062::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911178::DEBUG::2013-11-19 21:28:50,062::task::495::TaskManager.Task::(_debug) Task 84b2db63-119a-4b4a-b386-285178004e2a: ref 0 aborting False
>Thread-15911178::INFO::2013-11-19 21:28:50,063::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911179::DEBUG::2013-11-19 21:28:50,092::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911179::INFO::2013-11-19 21:28:50,093::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911179::DEBUG::2013-11-19 21:28:50,094::task::495::TaskManager.Task::(_debug) Task 5b9d8ec2-0640-41b9-a9e8-26d1db3ba8bd: moving from state init -> state preparing
>Thread-15911179::DEBUG::2013-11-19 21:28:50,095::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6030f192-0342-405e-a85f-609c3b41125f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911179::DEBUG::2013-11-19 21:28:50,096::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911179::DEBUG::2013-11-19 21:28:50,097::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911179::DEBUG::2013-11-19 21:28:50,097::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6030f192-0342-405e-a85f-609c3b41125f`::Granted request
>Thread-15911179::DEBUG::2013-11-19 21:28:50,098::task::495::TaskManager.Task::(_debug) Task 5b9d8ec2-0640-41b9-a9e8-26d1db3ba8bd: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911179::DEBUG::2013-11-19 21:28:50,099::task::495::TaskManager.Task::(_debug) Task 5b9d8ec2-0640-41b9-a9e8-26d1db3ba8bd: ref 1 aborting False
>Thread-15911179::WARNING::2013-11-19 21:28:50,119::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911179::DEBUG::2013-11-19 21:28:50,135::task::495::TaskManager.Task::(_debug) Task 5b9d8ec2-0640-41b9-a9e8-26d1db3ba8bd: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911179::DEBUG::2013-11-19 21:28:50,137::task::495::TaskManager.Task::(_debug) Task 5b9d8ec2-0640-41b9-a9e8-26d1db3ba8bd: moving from state preparing -> state finished
>Thread-15911179::DEBUG::2013-11-19 21:28:50,138::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911179::DEBUG::2013-11-19 21:28:50,139::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911179::DEBUG::2013-11-19 21:28:50,139::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911179::DEBUG::2013-11-19 21:28:50,140::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911179::DEBUG::2013-11-19 21:28:50,140::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911179::DEBUG::2013-11-19 21:28:50,141::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911179::DEBUG::2013-11-19 21:28:50,142::task::495::TaskManager.Task::(_debug) Task 5b9d8ec2-0640-41b9-a9e8-26d1db3ba8bd: ref 0 aborting False
>Thread-15911179::INFO::2013-11-19 21:28:50,142::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911115::DEBUG::2013-11-19 21:28:50,708::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:28:51,215::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.078533 s, 13.0 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:28:52,385::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911115::DEBUG::2013-11-19 21:28:53,231::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:28:53,760::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.072534 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:28:55,779::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:28:56,328::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0741464 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:28:58,344::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:28:58,866::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.073965 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15911184::INFO::2013-11-19 21:29:00,098::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911184::DEBUG::2013-11-19 21:29:00,100::task::495::TaskManager.Task::(_debug) Task 17bbdf95-6cae-4d79-bbc7-039c05254b85: moving from state init -> state preparing
>Thread-15911184::DEBUG::2013-11-19 21:29:00,100::task::495::TaskManager.Task::(_debug) Task 17bbdf95-6cae-4d79-bbc7-039c05254b85: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000375032424927', 'lastCheck': 1384896537.4444809, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00154209136963', 'lastCheck': 1384896537.0167899, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896532.3859661, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000632047653198', 'lastCheck': 1384896532.9112451, 'valid': True, 'code': 0}}
>Thread-15911184::DEBUG::2013-11-19 21:29:00,106::task::495::TaskManager.Task::(_debug) Task 17bbdf95-6cae-4d79-bbc7-039c05254b85: moving from state preparing -> state finished
>Thread-15911184::DEBUG::2013-11-19 21:29:00,111::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911184::DEBUG::2013-11-19 21:29:00,112::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911184::DEBUG::2013-11-19 21:29:00,112::task::495::TaskManager.Task::(_debug) Task 17bbdf95-6cae-4d79-bbc7-039c05254b85: ref 0 aborting False
>Thread-15911184::INFO::2013-11-19 21:29:00,113::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000375032424927', 'lastCheck': 1384896537.4444809, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00154209136963', 'lastCheck': 1384896537.0167899, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896532.3859661, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000632047653198', 'lastCheck': 1384896532.9112451, 'valid': True, 'code': 0}}
>Thread-15911186::DEBUG::2013-11-19 21:29:00,264::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911186::INFO::2013-11-19 21:29:00,265::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911186::DEBUG::2013-11-19 21:29:00,266::task::495::TaskManager.Task::(_debug) Task 50bbbbfc-b217-4797-9b9d-97aea2a38f7c: moving from state init -> state preparing
>Thread-15911186::DEBUG::2013-11-19 21:29:00,266::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=282 spmId=2
>Thread-15911186::DEBUG::2013-11-19 21:29:00,272::task::495::TaskManager.Task::(_debug) Task 50bbbbfc-b217-4797-9b9d-97aea2a38f7c: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911186::DEBUG::2013-11-19 21:29:00,272::task::495::TaskManager.Task::(_debug) Task 50bbbbfc-b217-4797-9b9d-97aea2a38f7c: moving from state preparing -> state finished
>Thread-15911186::DEBUG::2013-11-19 21:29:00,273::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911186::DEBUG::2013-11-19 21:29:00,273::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911186::DEBUG::2013-11-19 21:29:00,274::task::495::TaskManager.Task::(_debug) Task 50bbbbfc-b217-4797-9b9d-97aea2a38f7c: ref 0 aborting False
>Thread-15911186::INFO::2013-11-19 21:29:00,275::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911187::DEBUG::2013-11-19 21:29:00,296::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911187::INFO::2013-11-19 21:29:00,297::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911187::DEBUG::2013-11-19 21:29:00,298::task::495::TaskManager.Task::(_debug) Task bbd37f0b-32ea-45f6-9044-5965ca1aaa1e: moving from state init -> state preparing
>Thread-15911187::DEBUG::2013-11-19 21:29:00,302::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e23cfff4-c5b0-4719-8d0b-7190e2bc59dc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911187::DEBUG::2013-11-19 21:29:00,303::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911187::DEBUG::2013-11-19 21:29:00,304::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911187::DEBUG::2013-11-19 21:29:00,304::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e23cfff4-c5b0-4719-8d0b-7190e2bc59dc`::Granted request
>Thread-15911187::DEBUG::2013-11-19 21:29:00,312::task::495::TaskManager.Task::(_debug) Task bbd37f0b-32ea-45f6-9044-5965ca1aaa1e: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911187::DEBUG::2013-11-19 21:29:00,313::task::495::TaskManager.Task::(_debug) Task bbd37f0b-32ea-45f6-9044-5965ca1aaa1e: ref 1 aborting False
>Thread-15911187::WARNING::2013-11-19 21:29:00,320::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911187::DEBUG::2013-11-19 21:29:00,321::task::495::TaskManager.Task::(_debug) Task bbd37f0b-32ea-45f6-9044-5965ca1aaa1e: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911187::DEBUG::2013-11-19 21:29:00,328::task::495::TaskManager.Task::(_debug) Task bbd37f0b-32ea-45f6-9044-5965ca1aaa1e: moving from state preparing -> state finished
>Thread-15911187::DEBUG::2013-11-19 21:29:00,328::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911187::DEBUG::2013-11-19 21:29:00,333::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911187::DEBUG::2013-11-19 21:29:00,334::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911187::DEBUG::2013-11-19 21:29:00,336::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911187::DEBUG::2013-11-19 21:29:00,337::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911187::DEBUG::2013-11-19 21:29:00,337::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911187::DEBUG::2013-11-19 21:29:00,338::task::495::TaskManager.Task::(_debug) Task bbd37f0b-32ea-45f6-9044-5965ca1aaa1e: ref 0 aborting False
>Thread-15911187::INFO::2013-11-19 21:29:00,338::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911115::DEBUG::2013-11-19 21:29:00,880::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:29:01,431::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.113469 s, 9.0 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:29:02,555::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911115::DEBUG::2013-11-19 21:29:03,446::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:29:03,942::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0504075 s, 20.3 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:29:05,952::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:29:06,497::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0834511 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:29:08,513::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:29:09,060::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0742441 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15911192::INFO::2013-11-19 21:29:10,301::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911192::DEBUG::2013-11-19 21:29:10,310::task::495::TaskManager.Task::(_debug) Task c870f476-ede0-4b64-89ee-78415e6ae56b: moving from state init -> state preparing
>Thread-15911192::DEBUG::2013-11-19 21:29:10,321::task::495::TaskManager.Task::(_debug) Task c870f476-ede0-4b64-89ee-78415e6ae56b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000420093536377', 'lastCheck': 1384896547.6862071, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00173401832581', 'lastCheck': 1384896547.1792181, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896542.556654, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00122213363647', 'lastCheck': 1384896543.0787611, 'valid': True, 'code': 0}}
>Thread-15911192::DEBUG::2013-11-19 21:29:10,330::task::495::TaskManager.Task::(_debug) Task c870f476-ede0-4b64-89ee-78415e6ae56b: moving from state preparing -> state finished
>Thread-15911192::DEBUG::2013-11-19 21:29:10,331::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911192::DEBUG::2013-11-19 21:29:10,336::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911192::DEBUG::2013-11-19 21:29:10,337::task::495::TaskManager.Task::(_debug) Task c870f476-ede0-4b64-89ee-78415e6ae56b: ref 0 aborting False
>Thread-15911192::INFO::2013-11-19 21:29:10,338::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000420093536377', 'lastCheck': 1384896547.6862071, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00173401832581', 'lastCheck': 1384896547.1792181, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896542.556654, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00122213363647', 'lastCheck': 1384896543.0787611, 'valid': True, 'code': 0}}
>Thread-15911194::DEBUG::2013-11-19 21:29:10,467::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911194::INFO::2013-11-19 21:29:10,468::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911194::DEBUG::2013-11-19 21:29:10,469::task::495::TaskManager.Task::(_debug) Task 2faa87e3-7948-40b8-bad3-b8c871590e18: moving from state init -> state preparing
>Thread-15911194::DEBUG::2013-11-19 21:29:10,470::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=282 spmId=2
>Thread-15911194::DEBUG::2013-11-19 21:29:10,470::task::495::TaskManager.Task::(_debug) Task 2faa87e3-7948-40b8-bad3-b8c871590e18: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911194::DEBUG::2013-11-19 21:29:10,471::task::495::TaskManager.Task::(_debug) Task 2faa87e3-7948-40b8-bad3-b8c871590e18: moving from state preparing -> state finished
>Thread-15911194::DEBUG::2013-11-19 21:29:10,471::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911194::DEBUG::2013-11-19 21:29:10,472::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911194::DEBUG::2013-11-19 21:29:10,472::task::495::TaskManager.Task::(_debug) Task 2faa87e3-7948-40b8-bad3-b8c871590e18: ref 0 aborting False
>Thread-15911194::INFO::2013-11-19 21:29:10,473::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911195::DEBUG::2013-11-19 21:29:10,502::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911195::INFO::2013-11-19 21:29:10,503::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911195::DEBUG::2013-11-19 21:29:10,503::task::495::TaskManager.Task::(_debug) Task 9455834e-1402-434e-bb55-0c4de5d479eb: moving from state init -> state preparing
>Thread-15911195::DEBUG::2013-11-19 21:29:10,508::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`fbe21839-a72a-4de5-8a10-e4df67e61427`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911195::DEBUG::2013-11-19 21:29:10,509::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911195::DEBUG::2013-11-19 21:29:10,510::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911195::DEBUG::2013-11-19 21:29:10,510::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`fbe21839-a72a-4de5-8a10-e4df67e61427`::Granted request
>Thread-15911195::DEBUG::2013-11-19 21:29:10,515::task::495::TaskManager.Task::(_debug) Task 9455834e-1402-434e-bb55-0c4de5d479eb: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911195::DEBUG::2013-11-19 21:29:10,524::task::495::TaskManager.Task::(_debug) Task 9455834e-1402-434e-bb55-0c4de5d479eb: ref 1 aborting False
>Thread-15911195::WARNING::2013-11-19 21:29:10,535::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911195::DEBUG::2013-11-19 21:29:10,542::task::495::TaskManager.Task::(_debug) Task 9455834e-1402-434e-bb55-0c4de5d479eb: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911195::DEBUG::2013-11-19 21:29:10,543::task::495::TaskManager.Task::(_debug) Task 9455834e-1402-434e-bb55-0c4de5d479eb: moving from state preparing -> state finished
>Thread-15911195::DEBUG::2013-11-19 21:29:10,543::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911195::DEBUG::2013-11-19 21:29:10,544::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911195::DEBUG::2013-11-19 21:29:10,544::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911195::DEBUG::2013-11-19 21:29:10,545::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911195::DEBUG::2013-11-19 21:29:10,546::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911195::DEBUG::2013-11-19 21:29:10,546::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911195::DEBUG::2013-11-19 21:29:10,547::task::495::TaskManager.Task::(_debug) Task 9455834e-1402-434e-bb55-0c4de5d479eb: ref 0 aborting False
>Thread-15911195::INFO::2013-11-19 21:29:10,551::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911115::DEBUG::2013-11-19 21:29:11,075::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:29:11,605::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0744361 s, 13.8 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:29:12,750::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911115::DEBUG::2013-11-19 21:29:13,621::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:29:14,144::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0737296 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:29:16,159::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:29:16,713::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0741607 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:29:18,729::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:29:19,259::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0732013 s, 14.0 MB/s\n'; <rc> = 0
>Thread-15911200::INFO::2013-11-19 21:29:20,513::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911200::DEBUG::2013-11-19 21:29:20,513::task::495::TaskManager.Task::(_debug) Task 243c7b4d-497d-4671-96aa-8086923f68d8: moving from state init -> state preparing
>Thread-15911200::DEBUG::2013-11-19 21:29:20,514::task::495::TaskManager.Task::(_debug) Task 243c7b4d-497d-4671-96aa-8086923f68d8: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000432968139648', 'lastCheck': 1384896557.831037, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00150394439697', 'lastCheck': 1384896557.398869, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896552.751441, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000511884689331', 'lastCheck': 1384896553.289361, 'valid': True, 'code': 0}}
>Thread-15911200::DEBUG::2013-11-19 21:29:20,521::task::495::TaskManager.Task::(_debug) Task 243c7b4d-497d-4671-96aa-8086923f68d8: moving from state preparing -> state finished
>Thread-15911200::DEBUG::2013-11-19 21:29:20,522::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911200::DEBUG::2013-11-19 21:29:20,522::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911200::DEBUG::2013-11-19 21:29:20,523::task::495::TaskManager.Task::(_debug) Task 243c7b4d-497d-4671-96aa-8086923f68d8: ref 0 aborting False
>Thread-15911200::INFO::2013-11-19 21:29:20,524::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000432968139648', 'lastCheck': 1384896557.831037, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00150394439697', 'lastCheck': 1384896557.398869, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896552.751441, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000511884689331', 'lastCheck': 1384896553.289361, 'valid': True, 'code': 0}}
>Thread-15911202::DEBUG::2013-11-19 21:29:20,648::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911202::INFO::2013-11-19 21:29:20,649::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911202::DEBUG::2013-11-19 21:29:20,649::task::495::TaskManager.Task::(_debug) Task 8b995671-e105-4ba6-aa69-2c03d896064e: moving from state init -> state preparing
>Thread-15911202::DEBUG::2013-11-19 21:29:20,650::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=282 spmId=2
>Thread-15911202::DEBUG::2013-11-19 21:29:20,651::task::495::TaskManager.Task::(_debug) Task 8b995671-e105-4ba6-aa69-2c03d896064e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911202::DEBUG::2013-11-19 21:29:20,651::task::495::TaskManager.Task::(_debug) Task 8b995671-e105-4ba6-aa69-2c03d896064e: moving from state preparing -> state finished
>Thread-15911202::DEBUG::2013-11-19 21:29:20,652::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911202::DEBUG::2013-11-19 21:29:20,652::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911202::DEBUG::2013-11-19 21:29:20,653::task::495::TaskManager.Task::(_debug) Task 8b995671-e105-4ba6-aa69-2c03d896064e: ref 0 aborting False
>Thread-15911202::INFO::2013-11-19 21:29:20,654::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911203::DEBUG::2013-11-19 21:29:20,671::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911203::INFO::2013-11-19 21:29:20,671::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911203::DEBUG::2013-11-19 21:29:20,672::task::495::TaskManager.Task::(_debug) Task 05a52452-6fc4-4232-95e0-7f4221ab9baf: moving from state init -> state preparing
>Thread-15911203::DEBUG::2013-11-19 21:29:20,676::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0769aada-d4f7-465d-b487-047ed99e1206`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911203::DEBUG::2013-11-19 21:29:20,677::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911203::DEBUG::2013-11-19 21:29:20,678::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911203::DEBUG::2013-11-19 21:29:20,678::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0769aada-d4f7-465d-b487-047ed99e1206`::Granted request
>Thread-15911203::DEBUG::2013-11-19 21:29:20,682::task::495::TaskManager.Task::(_debug) Task 05a52452-6fc4-4232-95e0-7f4221ab9baf: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911203::DEBUG::2013-11-19 21:29:20,682::task::495::TaskManager.Task::(_debug) Task 05a52452-6fc4-4232-95e0-7f4221ab9baf: ref 1 aborting False
>Thread-15911203::WARNING::2013-11-19 21:29:20,688::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911203::DEBUG::2013-11-19 21:29:20,692::task::495::TaskManager.Task::(_debug) Task 05a52452-6fc4-4232-95e0-7f4221ab9baf: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911203::DEBUG::2013-11-19 21:29:20,693::task::495::TaskManager.Task::(_debug) Task 05a52452-6fc4-4232-95e0-7f4221ab9baf: moving from state preparing -> state finished
>Thread-15911203::DEBUG::2013-11-19 21:29:20,694::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911203::DEBUG::2013-11-19 21:29:20,694::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911203::DEBUG::2013-11-19 21:29:20,700::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911203::DEBUG::2013-11-19 21:29:20,700::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911203::DEBUG::2013-11-19 21:29:20,701::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911203::DEBUG::2013-11-19 21:29:20,701::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911203::DEBUG::2013-11-19 21:29:20,702::task::495::TaskManager.Task::(_debug) Task 05a52452-6fc4-4232-95e0-7f4221ab9baf: ref 0 aborting False
>Thread-15911203::INFO::2013-11-19 21:29:20,703::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911115::DEBUG::2013-11-19 21:29:21,275::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:29:21,822::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0737275 s, 13.9 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:29:22,946::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911115::DEBUG::2013-11-19 21:29:23,837::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:29:24,369::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0742101 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:29:26,385::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:29:26,915::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0732197 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:29:28,930::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:29:29,473::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.073984 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15911208::INFO::2013-11-19 21:29:30,706::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911208::DEBUG::2013-11-19 21:29:30,707::task::495::TaskManager.Task::(_debug) Task 4fef6051-efeb-4955-9a6a-808a3dd50d17: moving from state init -> state preparing
>Thread-15911208::DEBUG::2013-11-19 21:29:30,708::task::495::TaskManager.Task::(_debug) Task 4fef6051-efeb-4955-9a6a-808a3dd50d17: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000375986099243', 'lastCheck': 1384896568.057802, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00149703025818', 'lastCheck': 1384896567.612828, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896562.947335, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000591039657593', 'lastCheck': 1384896563.514246, 'valid': True, 'code': 0}}
>Thread-15911208::DEBUG::2013-11-19 21:29:30,720::task::495::TaskManager.Task::(_debug) Task 4fef6051-efeb-4955-9a6a-808a3dd50d17: moving from state preparing -> state finished
>Thread-15911208::DEBUG::2013-11-19 21:29:30,721::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911208::DEBUG::2013-11-19 21:29:30,721::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911208::DEBUG::2013-11-19 21:29:30,722::task::495::TaskManager.Task::(_debug) Task 4fef6051-efeb-4955-9a6a-808a3dd50d17: ref 0 aborting False
>Thread-15911208::INFO::2013-11-19 21:29:30,723::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000375986099243', 'lastCheck': 1384896568.057802, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00149703025818', 'lastCheck': 1384896567.612828, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896562.947335, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000591039657593', 'lastCheck': 1384896563.514246, 'valid': True, 'code': 0}}
>Thread-15911210::DEBUG::2013-11-19 21:29:30,818::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911210::INFO::2013-11-19 21:29:30,819::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911210::DEBUG::2013-11-19 21:29:30,820::task::495::TaskManager.Task::(_debug) Task ec6b77b9-995a-4132-8b5f-63b9c202c290: moving from state init -> state preparing
>Thread-15911210::DEBUG::2013-11-19 21:29:30,821::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=282 spmId=2
>Thread-15911210::DEBUG::2013-11-19 21:29:30,821::task::495::TaskManager.Task::(_debug) Task ec6b77b9-995a-4132-8b5f-63b9c202c290: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911210::DEBUG::2013-11-19 21:29:30,826::task::495::TaskManager.Task::(_debug) Task ec6b77b9-995a-4132-8b5f-63b9c202c290: moving from state preparing -> state finished
>Thread-15911210::DEBUG::2013-11-19 21:29:30,826::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911210::DEBUG::2013-11-19 21:29:30,827::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911210::DEBUG::2013-11-19 21:29:30,827::task::495::TaskManager.Task::(_debug) Task ec6b77b9-995a-4132-8b5f-63b9c202c290: ref 0 aborting False
>Thread-15911210::INFO::2013-11-19 21:29:30,828::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911211::DEBUG::2013-11-19 21:29:30,836::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911211::INFO::2013-11-19 21:29:30,837::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911211::DEBUG::2013-11-19 21:29:30,837::task::495::TaskManager.Task::(_debug) Task 46aeb565-bf01-4954-9fcf-b753dc54889e: moving from state init -> state preparing
>Thread-15911211::DEBUG::2013-11-19 21:29:30,842::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`914e6be4-e0db-49bd-843a-2e0128a87ba7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911211::DEBUG::2013-11-19 21:29:30,843::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911211::DEBUG::2013-11-19 21:29:30,843::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911211::DEBUG::2013-11-19 21:29:30,844::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`914e6be4-e0db-49bd-843a-2e0128a87ba7`::Granted request
>Thread-15911211::DEBUG::2013-11-19 21:29:30,850::task::495::TaskManager.Task::(_debug) Task 46aeb565-bf01-4954-9fcf-b753dc54889e: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911211::DEBUG::2013-11-19 21:29:30,851::task::495::TaskManager.Task::(_debug) Task 46aeb565-bf01-4954-9fcf-b753dc54889e: ref 1 aborting False
>Thread-15911211::WARNING::2013-11-19 21:29:30,867::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911211::DEBUG::2013-11-19 21:29:30,878::task::495::TaskManager.Task::(_debug) Task 46aeb565-bf01-4954-9fcf-b753dc54889e: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911211::DEBUG::2013-11-19 21:29:30,883::task::495::TaskManager.Task::(_debug) Task 46aeb565-bf01-4954-9fcf-b753dc54889e: moving from state preparing -> state finished
>Thread-15911211::DEBUG::2013-11-19 21:29:30,884::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911211::DEBUG::2013-11-19 21:29:30,889::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911211::DEBUG::2013-11-19 21:29:30,890::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911211::DEBUG::2013-11-19 21:29:30,891::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911211::DEBUG::2013-11-19 21:29:30,891::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911211::DEBUG::2013-11-19 21:29:30,892::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911211::DEBUG::2013-11-19 21:29:30,892::task::495::TaskManager.Task::(_debug) Task 46aeb565-bf01-4954-9fcf-b753dc54889e: ref 0 aborting False
>Thread-15911211::INFO::2013-11-19 21:29:30,893::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911115::DEBUG::2013-11-19 21:29:31,488::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:29:32,007::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.073101 s, 14.0 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:29:33,163::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911115::DEBUG::2013-11-19 21:29:34,022::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:29:34,577::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0810506 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:29:36,592::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:29:37,124::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0733157 s, 14.0 MB/s\n'; <rc> = 0
>Thread-42838::DEBUG::2013-11-19 21:29:38,256::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 21:29:38,949::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:29:39,148::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:29:39,693::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0737704 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15911216::INFO::2013-11-19 21:29:40,906::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911216::DEBUG::2013-11-19 21:29:40,907::task::495::TaskManager.Task::(_debug) Task bb5bb847-f7eb-47da-8d7f-c541e071ac60: moving from state init -> state preparing
>Thread-15911216::DEBUG::2013-11-19 21:29:40,907::task::495::TaskManager.Task::(_debug) Task bb5bb847-f7eb-47da-8d7f-c541e071ac60: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000382900238037', 'lastCheck': 1384896578.954037, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00160193443298', 'lastCheck': 1384896577.8213, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896573.1647429, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00143504142761', 'lastCheck': 1384896573.6980071, 'valid': True, 'code': 0}}
>Thread-15911216::DEBUG::2013-11-19 21:29:40,919::task::495::TaskManager.Task::(_debug) Task bb5bb847-f7eb-47da-8d7f-c541e071ac60: moving from state preparing -> state finished
>Thread-15911216::DEBUG::2013-11-19 21:29:40,920::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911216::DEBUG::2013-11-19 21:29:40,920::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911216::DEBUG::2013-11-19 21:29:40,921::task::495::TaskManager.Task::(_debug) Task bb5bb847-f7eb-47da-8d7f-c541e071ac60: ref 0 aborting False
>Thread-15911216::INFO::2013-11-19 21:29:40,922::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000382900238037', 'lastCheck': 1384896578.954037, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00160193443298', 'lastCheck': 1384896577.8213, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896573.1647429, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00143504142761', 'lastCheck': 1384896573.6980071, 'valid': True, 'code': 0}}
>Thread-15911218::DEBUG::2013-11-19 21:29:40,990::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911218::INFO::2013-11-19 21:29:40,991::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911218::DEBUG::2013-11-19 21:29:40,992::task::495::TaskManager.Task::(_debug) Task 551a0333-816a-42db-aaa5-492441828195: moving from state init -> state preparing
>Thread-15911218::DEBUG::2013-11-19 21:29:40,992::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=282 spmId=2
>Thread-15911218::DEBUG::2013-11-19 21:29:40,993::task::495::TaskManager.Task::(_debug) Task 551a0333-816a-42db-aaa5-492441828195: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911218::DEBUG::2013-11-19 21:29:40,994::task::495::TaskManager.Task::(_debug) Task 551a0333-816a-42db-aaa5-492441828195: moving from state preparing -> state finished
>Thread-15911218::DEBUG::2013-11-19 21:29:40,994::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911218::DEBUG::2013-11-19 21:29:40,995::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911218::DEBUG::2013-11-19 21:29:40,995::task::495::TaskManager.Task::(_debug) Task 551a0333-816a-42db-aaa5-492441828195: ref 0 aborting False
>Thread-15911218::INFO::2013-11-19 21:29:40,996::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911219::DEBUG::2013-11-19 21:29:41,005::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911219::INFO::2013-11-19 21:29:41,006::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911219::DEBUG::2013-11-19 21:29:41,007::task::495::TaskManager.Task::(_debug) Task fed045f3-2aa8-4897-a23f-2eb35bfdd660: moving from state init -> state preparing
>Thread-15911219::DEBUG::2013-11-19 21:29:41,008::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`48b4dd5f-3154-4561-a99a-fe0dc0e74914`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911219::DEBUG::2013-11-19 21:29:41,013::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911219::DEBUG::2013-11-19 21:29:41,014::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911219::DEBUG::2013-11-19 21:29:41,015::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`48b4dd5f-3154-4561-a99a-fe0dc0e74914`::Granted request
>Thread-15911219::DEBUG::2013-11-19 21:29:41,022::task::495::TaskManager.Task::(_debug) Task fed045f3-2aa8-4897-a23f-2eb35bfdd660: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911219::DEBUG::2013-11-19 21:29:41,027::task::495::TaskManager.Task::(_debug) Task fed045f3-2aa8-4897-a23f-2eb35bfdd660: ref 1 aborting False
>Thread-15911219::WARNING::2013-11-19 21:29:41,029::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911219::DEBUG::2013-11-19 21:29:41,044::task::495::TaskManager.Task::(_debug) Task fed045f3-2aa8-4897-a23f-2eb35bfdd660: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911219::DEBUG::2013-11-19 21:29:41,050::task::495::TaskManager.Task::(_debug) Task fed045f3-2aa8-4897-a23f-2eb35bfdd660: moving from state preparing -> state finished
>Thread-15911219::DEBUG::2013-11-19 21:29:41,051::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911219::DEBUG::2013-11-19 21:29:41,051::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911219::DEBUG::2013-11-19 21:29:41,052::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911219::DEBUG::2013-11-19 21:29:41,053::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911219::DEBUG::2013-11-19 21:29:41,053::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911219::DEBUG::2013-11-19 21:29:41,054::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911219::DEBUG::2013-11-19 21:29:41,054::task::495::TaskManager.Task::(_debug) Task fed045f3-2aa8-4897-a23f-2eb35bfdd660: ref 0 aborting False
>Thread-15911219::INFO::2013-11-19 21:29:41,055::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911115::DEBUG::2013-11-19 21:29:41,708::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:29:42,245::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0745469 s, 13.7 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:29:43,384::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911115::DEBUG::2013-11-19 21:29:44,261::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:29:44,792::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0734955 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:29:46,808::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:29:47,343::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0744038 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:29:49,359::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:29:49,882::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0730842 s, 14.0 MB/s\n'; <rc> = 0
>Thread-15911225::INFO::2013-11-19 21:29:51,161::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911224::DEBUG::2013-11-19 21:29:51,181::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911225::DEBUG::2013-11-19 21:29:51,182::task::495::TaskManager.Task::(_debug) Task 5b2c4d23-b753-4469-9ebc-cb303e68dde5: moving from state init -> state preparing
>Thread-15911224::INFO::2013-11-19 21:29:51,187::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911225::DEBUG::2013-11-19 21:29:51,190::task::495::TaskManager.Task::(_debug) Task 5b2c4d23-b753-4469-9ebc-cb303e68dde5: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000433921813965', 'lastCheck': 1384896589.0285299, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00238299369812', 'lastCheck': 1384896588.03268, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896583.384994, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00048303604126', 'lastCheck': 1384896583.92977, 'valid': True, 'code': 0}}
>Thread-15911224::DEBUG::2013-11-19 21:29:51,191::task::495::TaskManager.Task::(_debug) Task 82fe740a-f930-4f24-b1a1-2a7c9d94d3de: moving from state init -> state preparing
>Thread-15911225::DEBUG::2013-11-19 21:29:51,192::task::495::TaskManager.Task::(_debug) Task 5b2c4d23-b753-4469-9ebc-cb303e68dde5: moving from state preparing -> state finished
>Thread-15911224::DEBUG::2013-11-19 21:29:51,193::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=282 spmId=2
>Thread-15911225::DEBUG::2013-11-19 21:29:51,195::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911224::DEBUG::2013-11-19 21:29:51,196::task::495::TaskManager.Task::(_debug) Task 82fe740a-f930-4f24-b1a1-2a7c9d94d3de: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911225::DEBUG::2013-11-19 21:29:51,203::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911224::DEBUG::2013-11-19 21:29:51,204::task::495::TaskManager.Task::(_debug) Task 82fe740a-f930-4f24-b1a1-2a7c9d94d3de: moving from state preparing -> state finished
>Thread-15911225::DEBUG::2013-11-19 21:29:51,208::task::495::TaskManager.Task::(_debug) Task 5b2c4d23-b753-4469-9ebc-cb303e68dde5: ref 0 aborting False
>Thread-15911224::DEBUG::2013-11-19 21:29:51,208::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911225::INFO::2013-11-19 21:29:51,209::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000433921813965', 'lastCheck': 1384896589.0285299, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00238299369812', 'lastCheck': 1384896588.03268, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896583.384994, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00048303604126', 'lastCheck': 1384896583.92977, 'valid': True, 'code': 0}}
>Thread-15911224::DEBUG::2013-11-19 21:29:51,210::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911224::DEBUG::2013-11-19 21:29:51,215::task::495::TaskManager.Task::(_debug) Task 82fe740a-f930-4f24-b1a1-2a7c9d94d3de: ref 0 aborting False
>Thread-15911224::INFO::2013-11-19 21:29:51,224::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911226::DEBUG::2013-11-19 21:29:51,243::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911226::INFO::2013-11-19 21:29:51,245::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911226::DEBUG::2013-11-19 21:29:51,245::task::495::TaskManager.Task::(_debug) Task 4d136391-383a-4cde-9508-5755f3a7e225: moving from state init -> state preparing
>Thread-15911226::DEBUG::2013-11-19 21:29:51,252::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1d42ae31-8d97-4dee-8793-781dfdff419b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911226::DEBUG::2013-11-19 21:29:51,253::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911226::DEBUG::2013-11-19 21:29:51,254::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911226::DEBUG::2013-11-19 21:29:51,254::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1d42ae31-8d97-4dee-8793-781dfdff419b`::Granted request
>Thread-15911226::DEBUG::2013-11-19 21:29:51,260::task::495::TaskManager.Task::(_debug) Task 4d136391-383a-4cde-9508-5755f3a7e225: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911226::DEBUG::2013-11-19 21:29:51,264::task::495::TaskManager.Task::(_debug) Task 4d136391-383a-4cde-9508-5755f3a7e225: ref 1 aborting False
>Thread-15911226::WARNING::2013-11-19 21:29:51,266::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911226::DEBUG::2013-11-19 21:29:51,276::task::495::TaskManager.Task::(_debug) Task 4d136391-383a-4cde-9508-5755f3a7e225: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911226::DEBUG::2013-11-19 21:29:51,283::task::495::TaskManager.Task::(_debug) Task 4d136391-383a-4cde-9508-5755f3a7e225: moving from state preparing -> state finished
>Thread-15911226::DEBUG::2013-11-19 21:29:51,283::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911226::DEBUG::2013-11-19 21:29:51,284::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911226::DEBUG::2013-11-19 21:29:51,284::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911226::DEBUG::2013-11-19 21:29:51,285::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911226::DEBUG::2013-11-19 21:29:51,285::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911226::DEBUG::2013-11-19 21:29:51,286::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911226::DEBUG::2013-11-19 21:29:51,287::task::495::TaskManager.Task::(_debug) Task 4d136391-383a-4cde-9508-5755f3a7e225: ref 0 aborting False
>Thread-15911226::INFO::2013-11-19 21:29:51,287::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911115::DEBUG::2013-11-19 21:29:51,897::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:29:52,436::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.073784 s, 13.9 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:29:53,576::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911115::DEBUG::2013-11-19 21:29:54,451::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:29:54,988::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.07357 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:29:57,003::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:29:57,527::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0732544 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:29:59,543::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:30:00,083::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0738056 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15911232::DEBUG::2013-11-19 21:30:01,383::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911232::INFO::2013-11-19 21:30:01,384::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911232::DEBUG::2013-11-19 21:30:01,385::task::495::TaskManager.Task::(_debug) Task 65d430e8-8ff4-4085-bc2a-517a2098a680: moving from state init -> state preparing
>Thread-15911232::DEBUG::2013-11-19 21:30:01,386::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=282 spmId=2
>Thread-15911232::DEBUG::2013-11-19 21:30:01,387::task::495::TaskManager.Task::(_debug) Task 65d430e8-8ff4-4085-bc2a-517a2098a680: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911232::DEBUG::2013-11-19 21:30:01,387::task::495::TaskManager.Task::(_debug) Task 65d430e8-8ff4-4085-bc2a-517a2098a680: moving from state preparing -> state finished
>Thread-15911232::DEBUG::2013-11-19 21:30:01,388::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911232::DEBUG::2013-11-19 21:30:01,388::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911232::DEBUG::2013-11-19 21:30:01,389::task::495::TaskManager.Task::(_debug) Task 65d430e8-8ff4-4085-bc2a-517a2098a680: ref 0 aborting False
>Thread-15911232::INFO::2013-11-19 21:30:01,389::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911234::INFO::2013-11-19 21:30:01,400::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911233::DEBUG::2013-11-19 21:30:01,411::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911234::DEBUG::2013-11-19 21:30:01,412::task::495::TaskManager.Task::(_debug) Task 4e6b7e1d-b865-4b06-b340-7baf10488b03: moving from state init -> state preparing
>Thread-15911233::INFO::2013-11-19 21:30:01,413::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911234::DEBUG::2013-11-19 21:30:01,417::task::495::TaskManager.Task::(_debug) Task 4e6b7e1d-b865-4b06-b340-7baf10488b03: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000426054000854', 'lastCheck': 1384896599.2087009, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00154185295105', 'lastCheck': 1384896598.210664, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896593.577342, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000489950180054', 'lastCheck': 1384896594.1274059, 'valid': True, 'code': 0}}
>Thread-15911233::DEBUG::2013-11-19 21:30:01,418::task::495::TaskManager.Task::(_debug) Task 6cd5d3f8-1470-47c9-8d67-b79e22b4541b: moving from state init -> state preparing
>Thread-15911234::DEBUG::2013-11-19 21:30:01,424::task::495::TaskManager.Task::(_debug) Task 4e6b7e1d-b865-4b06-b340-7baf10488b03: moving from state preparing -> state finished
>Thread-15911233::DEBUG::2013-11-19 21:30:01,425::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`56490edc-e929-461c-a167-646fa7b729c2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911234::DEBUG::2013-11-19 21:30:01,426::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911233::DEBUG::2013-11-19 21:30:01,427::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911234::DEBUG::2013-11-19 21:30:01,430::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911234::DEBUG::2013-11-19 21:30:01,431::task::495::TaskManager.Task::(_debug) Task 4e6b7e1d-b865-4b06-b340-7baf10488b03: ref 0 aborting False
>Thread-15911233::DEBUG::2013-11-19 21:30:01,432::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911234::INFO::2013-11-19 21:30:01,432::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000426054000854', 'lastCheck': 1384896599.2087009, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00154185295105', 'lastCheck': 1384896598.210664, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896593.577342, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000489950180054', 'lastCheck': 1384896594.1274059, 'valid': True, 'code': 0}}
>Thread-15911233::DEBUG::2013-11-19 21:30:01,433::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`56490edc-e929-461c-a167-646fa7b729c2`::Granted request
>Thread-15911233::DEBUG::2013-11-19 21:30:01,439::task::495::TaskManager.Task::(_debug) Task 6cd5d3f8-1470-47c9-8d67-b79e22b4541b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911233::DEBUG::2013-11-19 21:30:01,446::task::495::TaskManager.Task::(_debug) Task 6cd5d3f8-1470-47c9-8d67-b79e22b4541b: ref 1 aborting False
>Thread-15911233::WARNING::2013-11-19 21:30:01,449::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911233::DEBUG::2013-11-19 21:30:01,453::task::495::TaskManager.Task::(_debug) Task 6cd5d3f8-1470-47c9-8d67-b79e22b4541b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911233::DEBUG::2013-11-19 21:30:01,455::task::495::TaskManager.Task::(_debug) Task 6cd5d3f8-1470-47c9-8d67-b79e22b4541b: moving from state preparing -> state finished
>Thread-15911233::DEBUG::2013-11-19 21:30:01,455::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911233::DEBUG::2013-11-19 21:30:01,456::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911233::DEBUG::2013-11-19 21:30:01,463::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911233::DEBUG::2013-11-19 21:30:01,464::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911233::DEBUG::2013-11-19 21:30:01,464::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911233::DEBUG::2013-11-19 21:30:01,465::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911233::DEBUG::2013-11-19 21:30:01,465::task::495::TaskManager.Task::(_debug) Task 6cd5d3f8-1470-47c9-8d67-b79e22b4541b: ref 0 aborting False
>Thread-15911233::INFO::2013-11-19 21:30:01,466::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911115::DEBUG::2013-11-19 21:30:02,099::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:30:02,620::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0739173 s, 13.9 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:30:03,771::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911115::DEBUG::2013-11-19 21:30:04,635::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:30:05,159::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0729323 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:30:07,175::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:30:07,741::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.108079 s, 9.5 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:30:09,755::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:30:10,275::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0733282 s, 14.0 MB/s\n'; <rc> = 0
>Thread-15911240::DEBUG::2013-11-19 21:30:11,591::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911240::INFO::2013-11-19 21:30:11,592::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911240::DEBUG::2013-11-19 21:30:11,593::task::495::TaskManager.Task::(_debug) Task be565fa4-5847-4507-8101-d2a2078a1d4a: moving from state init -> state preparing
>Thread-15911240::DEBUG::2013-11-19 21:30:11,603::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=282 spmId=2
>Thread-15911240::DEBUG::2013-11-19 21:30:11,603::task::495::TaskManager.Task::(_debug) Task be565fa4-5847-4507-8101-d2a2078a1d4a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911240::DEBUG::2013-11-19 21:30:11,604::task::495::TaskManager.Task::(_debug) Task be565fa4-5847-4507-8101-d2a2078a1d4a: moving from state preparing -> state finished
>Thread-15911240::DEBUG::2013-11-19 21:30:11,605::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911240::DEBUG::2013-11-19 21:30:11,605::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911240::DEBUG::2013-11-19 21:30:11,606::task::495::TaskManager.Task::(_debug) Task be565fa4-5847-4507-8101-d2a2078a1d4a: ref 0 aborting False
>Thread-15911240::INFO::2013-11-19 21:30:11,606::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911241::DEBUG::2013-11-19 21:30:11,622::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911241::INFO::2013-11-19 21:30:11,623::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911241::DEBUG::2013-11-19 21:30:11,624::task::495::TaskManager.Task::(_debug) Task ddcd7a5b-3db5-4f0a-b3d1-6a0ad628fa0a: moving from state init -> state preparing
>Thread-15911241::DEBUG::2013-11-19 21:30:11,628::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`14b5c650-e8f7-43ed-81cc-f2da01d0b5c1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911242::INFO::2013-11-19 21:30:11,630::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911241::DEBUG::2013-11-19 21:30:11,631::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911242::DEBUG::2013-11-19 21:30:11,640::task::495::TaskManager.Task::(_debug) Task 1909219e-2299-499b-9d54-451f9e61a39b: moving from state init -> state preparing
>Thread-15911242::DEBUG::2013-11-19 21:30:11,641::task::495::TaskManager.Task::(_debug) Task 1909219e-2299-499b-9d54-451f9e61a39b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000590085983276', 'lastCheck': 1384896609.3994031, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00155782699585', 'lastCheck': 1384896608.397249, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896603.771982, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000539064407349', 'lastCheck': 1384896604.305979, 'valid': True, 'code': 0}}
>Thread-15911241::DEBUG::2013-11-19 21:30:11,642::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911242::DEBUG::2013-11-19 21:30:11,643::task::495::TaskManager.Task::(_debug) Task 1909219e-2299-499b-9d54-451f9e61a39b: moving from state preparing -> state finished
>Thread-15911241::DEBUG::2013-11-19 21:30:11,644::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`14b5c650-e8f7-43ed-81cc-f2da01d0b5c1`::Granted request
>Thread-15911242::DEBUG::2013-11-19 21:30:11,648::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911241::DEBUG::2013-11-19 21:30:11,654::task::495::TaskManager.Task::(_debug) Task ddcd7a5b-3db5-4f0a-b3d1-6a0ad628fa0a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911242::DEBUG::2013-11-19 21:30:11,654::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911241::DEBUG::2013-11-19 21:30:11,659::task::495::TaskManager.Task::(_debug) Task ddcd7a5b-3db5-4f0a-b3d1-6a0ad628fa0a: ref 1 aborting False
>Thread-15911242::DEBUG::2013-11-19 21:30:11,662::task::495::TaskManager.Task::(_debug) Task 1909219e-2299-499b-9d54-451f9e61a39b: ref 0 aborting False
>Thread-15911241::WARNING::2013-11-19 21:30:11,669::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911242::INFO::2013-11-19 21:30:11,671::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000590085983276', 'lastCheck': 1384896609.3994031, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00155782699585', 'lastCheck': 1384896608.397249, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896603.771982, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000539064407349', 'lastCheck': 1384896604.305979, 'valid': True, 'code': 0}}
>Thread-15911241::DEBUG::2013-11-19 21:30:11,680::task::495::TaskManager.Task::(_debug) Task ddcd7a5b-3db5-4f0a-b3d1-6a0ad628fa0a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911241::DEBUG::2013-11-19 21:30:11,693::task::495::TaskManager.Task::(_debug) Task ddcd7a5b-3db5-4f0a-b3d1-6a0ad628fa0a: moving from state preparing -> state finished
>Thread-15911241::DEBUG::2013-11-19 21:30:11,702::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911241::DEBUG::2013-11-19 21:30:11,703::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911241::DEBUG::2013-11-19 21:30:11,704::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911241::DEBUG::2013-11-19 21:30:11,705::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911241::DEBUG::2013-11-19 21:30:11,705::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911241::DEBUG::2013-11-19 21:30:11,706::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911241::DEBUG::2013-11-19 21:30:11,707::task::495::TaskManager.Task::(_debug) Task ddcd7a5b-3db5-4f0a-b3d1-6a0ad628fa0a: ref 0 aborting False
>Thread-15911241::INFO::2013-11-19 21:30:11,708::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911115::DEBUG::2013-11-19 21:30:12,290::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:30:12,776::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.072332 s, 14.2 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:30:13,980::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911115::DEBUG::2013-11-19 21:30:14,791::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:30:15,327::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0747574 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:30:17,343::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:30:17,766::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0449244 s, 22.8 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:30:19,776::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:30:20,313::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0716356 s, 14.3 MB/s\n'; <rc> = 0
>Thread-15911248::DEBUG::2013-11-19 21:30:21,844::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911248::INFO::2013-11-19 21:30:21,845::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911248::DEBUG::2013-11-19 21:30:21,853::task::495::TaskManager.Task::(_debug) Task 986df438-0433-4f65-8e9a-7dc70dff5195: moving from state init -> state preparing
>Thread-15911248::DEBUG::2013-11-19 21:30:21,858::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=282 spmId=2
>Thread-15911248::DEBUG::2013-11-19 21:30:21,862::task::495::TaskManager.Task::(_debug) Task 986df438-0433-4f65-8e9a-7dc70dff5195: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911248::DEBUG::2013-11-19 21:30:21,863::task::495::TaskManager.Task::(_debug) Task 986df438-0433-4f65-8e9a-7dc70dff5195: moving from state preparing -> state finished
>Thread-15911248::DEBUG::2013-11-19 21:30:21,867::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911248::DEBUG::2013-11-19 21:30:21,868::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911248::DEBUG::2013-11-19 21:30:21,868::task::495::TaskManager.Task::(_debug) Task 986df438-0433-4f65-8e9a-7dc70dff5195: ref 0 aborting False
>Thread-15911248::INFO::2013-11-19 21:30:21,869::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911249::DEBUG::2013-11-19 21:30:21,876::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911249::INFO::2013-11-19 21:30:21,877::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911249::DEBUG::2013-11-19 21:30:21,878::task::495::TaskManager.Task::(_debug) Task f6118835-431d-4d2e-b3b5-7a633ce62a07: moving from state init -> state preparing
>Thread-15911249::DEBUG::2013-11-19 21:30:21,879::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2737d3bb-a430-4e4a-99e9-aa31755e835a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911249::DEBUG::2013-11-19 21:30:21,884::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911249::DEBUG::2013-11-19 21:30:21,884::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911249::DEBUG::2013-11-19 21:30:21,885::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2737d3bb-a430-4e4a-99e9-aa31755e835a`::Granted request
>Thread-15911249::DEBUG::2013-11-19 21:30:21,886::task::495::TaskManager.Task::(_debug) Task f6118835-431d-4d2e-b3b5-7a633ce62a07: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911249::DEBUG::2013-11-19 21:30:21,887::task::495::TaskManager.Task::(_debug) Task f6118835-431d-4d2e-b3b5-7a633ce62a07: ref 1 aborting False
>Thread-15911249::WARNING::2013-11-19 21:30:21,889::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911249::DEBUG::2013-11-19 21:30:21,893::task::495::TaskManager.Task::(_debug) Task f6118835-431d-4d2e-b3b5-7a633ce62a07: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911249::DEBUG::2013-11-19 21:30:21,898::task::495::TaskManager.Task::(_debug) Task f6118835-431d-4d2e-b3b5-7a633ce62a07: moving from state preparing -> state finished
>Thread-15911249::DEBUG::2013-11-19 21:30:21,907::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911249::DEBUG::2013-11-19 21:30:21,913::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911249::DEBUG::2013-11-19 21:30:21,917::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911249::DEBUG::2013-11-19 21:30:21,917::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911249::DEBUG::2013-11-19 21:30:21,921::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911249::DEBUG::2013-11-19 21:30:21,922::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911249::DEBUG::2013-11-19 21:30:21,927::task::495::TaskManager.Task::(_debug) Task f6118835-431d-4d2e-b3b5-7a633ce62a07: ref 0 aborting False
>Thread-15911249::INFO::2013-11-19 21:30:21,933::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911250::INFO::2013-11-19 21:30:22,021::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911250::DEBUG::2013-11-19 21:30:22,021::task::495::TaskManager.Task::(_debug) Task 45a3321e-a585-4c7e-ae61-e53bb6fa61a9: moving from state init -> state preparing
>Thread-15911250::DEBUG::2013-11-19 21:30:22,022::task::495::TaskManager.Task::(_debug) Task 45a3321e-a585-4c7e-ae61-e53bb6fa61a9: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000437021255493', 'lastCheck': 1384896619.554714, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00155997276306', 'lastCheck': 1384896618.556078, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896613.9808481, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000546932220459', 'lastCheck': 1384896614.5314109, 'valid': True, 'code': 0}}
>Thread-15911250::DEBUG::2013-11-19 21:30:22,023::task::495::TaskManager.Task::(_debug) Task 45a3321e-a585-4c7e-ae61-e53bb6fa61a9: moving from state preparing -> state finished
>Thread-15911250::DEBUG::2013-11-19 21:30:22,028::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911250::DEBUG::2013-11-19 21:30:22,031::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911250::DEBUG::2013-11-19 21:30:22,037::task::495::TaskManager.Task::(_debug) Task 45a3321e-a585-4c7e-ae61-e53bb6fa61a9: ref 0 aborting False
>Thread-15911250::INFO::2013-11-19 21:30:22,041::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000437021255493', 'lastCheck': 1384896619.554714, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00155997276306', 'lastCheck': 1384896618.556078, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896613.9808481, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000546932220459', 'lastCheck': 1384896614.5314109, 'valid': True, 'code': 0}}
>Dummy-15911115::DEBUG::2013-11-19 21:30:22,328::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:30:22,863::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0716847 s, 14.3 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:30:24,022::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911115::DEBUG::2013-11-19 21:30:24,877::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:30:25,389::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0706514 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:30:27,404::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:30:27,953::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0740763 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:30:29,969::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:30:30,498::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0738606 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15911256::DEBUG::2013-11-19 21:30:32,054::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911256::INFO::2013-11-19 21:30:32,064::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911256::DEBUG::2013-11-19 21:30:32,065::task::495::TaskManager.Task::(_debug) Task 68828aa5-6635-4490-997a-b773e1cc716e: moving from state init -> state preparing
>Thread-15911256::DEBUG::2013-11-19 21:30:32,070::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=282 spmId=2
>Thread-15911256::DEBUG::2013-11-19 21:30:32,074::task::495::TaskManager.Task::(_debug) Task 68828aa5-6635-4490-997a-b773e1cc716e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911256::DEBUG::2013-11-19 21:30:32,077::task::495::TaskManager.Task::(_debug) Task 68828aa5-6635-4490-997a-b773e1cc716e: moving from state preparing -> state finished
>Thread-15911256::DEBUG::2013-11-19 21:30:32,089::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911256::DEBUG::2013-11-19 21:30:32,094::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911256::DEBUG::2013-11-19 21:30:32,095::task::495::TaskManager.Task::(_debug) Task 68828aa5-6635-4490-997a-b773e1cc716e: ref 0 aborting False
>Thread-15911256::INFO::2013-11-19 21:30:32,096::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911257::DEBUG::2013-11-19 21:30:32,101::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911257::INFO::2013-11-19 21:30:32,102::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911257::DEBUG::2013-11-19 21:30:32,103::task::495::TaskManager.Task::(_debug) Task 1d52166f-58fd-4800-af92-b41a4fbdb019: moving from state init -> state preparing
>Thread-15911257::DEBUG::2013-11-19 21:30:32,104::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`06360c5e-2a3b-4c55-87e0-21508008b549`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911257::DEBUG::2013-11-19 21:30:32,104::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911257::DEBUG::2013-11-19 21:30:32,105::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911257::DEBUG::2013-11-19 21:30:32,106::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`06360c5e-2a3b-4c55-87e0-21508008b549`::Granted request
>Thread-15911257::DEBUG::2013-11-19 21:30:32,107::task::495::TaskManager.Task::(_debug) Task 1d52166f-58fd-4800-af92-b41a4fbdb019: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911257::DEBUG::2013-11-19 21:30:32,108::task::495::TaskManager.Task::(_debug) Task 1d52166f-58fd-4800-af92-b41a4fbdb019: ref 1 aborting False
>Thread-15911257::WARNING::2013-11-19 21:30:32,114::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911257::DEBUG::2013-11-19 21:30:32,122::task::495::TaskManager.Task::(_debug) Task 1d52166f-58fd-4800-af92-b41a4fbdb019: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911257::DEBUG::2013-11-19 21:30:32,126::task::495::TaskManager.Task::(_debug) Task 1d52166f-58fd-4800-af92-b41a4fbdb019: moving from state preparing -> state finished
>Thread-15911257::DEBUG::2013-11-19 21:30:32,127::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911257::DEBUG::2013-11-19 21:30:32,127::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911257::DEBUG::2013-11-19 21:30:32,128::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911257::DEBUG::2013-11-19 21:30:32,129::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911257::DEBUG::2013-11-19 21:30:32,129::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911257::DEBUG::2013-11-19 21:30:32,130::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911257::DEBUG::2013-11-19 21:30:32,131::task::495::TaskManager.Task::(_debug) Task 1d52166f-58fd-4800-af92-b41a4fbdb019: ref 0 aborting False
>Thread-15911257::INFO::2013-11-19 21:30:32,131::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911258::INFO::2013-11-19 21:30:32,194::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911258::DEBUG::2013-11-19 21:30:32,195::task::495::TaskManager.Task::(_debug) Task 429e3c1a-2344-4c2c-97ef-4659ac6d5abe: moving from state init -> state preparing
>Thread-15911258::DEBUG::2013-11-19 21:30:32,202::task::495::TaskManager.Task::(_debug) Task 429e3c1a-2344-4c2c-97ef-4659ac6d5abe: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000518083572388', 'lastCheck': 1384896629.640168, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00154185295105', 'lastCheck': 1384896628.641227, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896624.0231359, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00044584274292', 'lastCheck': 1384896624.5577459, 'valid': True, 'code': 0}}
>Thread-15911258::DEBUG::2013-11-19 21:30:32,206::task::495::TaskManager.Task::(_debug) Task 429e3c1a-2344-4c2c-97ef-4659ac6d5abe: moving from state preparing -> state finished
>Thread-15911258::DEBUG::2013-11-19 21:30:32,209::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911258::DEBUG::2013-11-19 21:30:32,210::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911258::DEBUG::2013-11-19 21:30:32,211::task::495::TaskManager.Task::(_debug) Task 429e3c1a-2344-4c2c-97ef-4659ac6d5abe: ref 0 aborting False
>Thread-15911258::INFO::2013-11-19 21:30:32,214::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000518083572388', 'lastCheck': 1384896629.640168, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00154185295105', 'lastCheck': 1384896628.641227, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896624.0231359, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00044584274292', 'lastCheck': 1384896624.5577459, 'valid': True, 'code': 0}}
>Dummy-15911115::DEBUG::2013-11-19 21:30:32,513::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:30:33,056::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0730939 s, 14.0 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:30:34,189::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911115::DEBUG::2013-11-19 21:30:35,071::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:30:35,604::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0737681 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:30:37,620::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:30:38,130::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.070309 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:30:40,146::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:30:40,714::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0955083 s, 10.7 MB/s\n'; <rc> = 0
>Thread-15911264::DEBUG::2013-11-19 21:30:42,258::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911264::INFO::2013-11-19 21:30:42,259::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911264::DEBUG::2013-11-19 21:30:42,260::task::495::TaskManager.Task::(_debug) Task 9c090c21-1137-4cb7-bd40-56846cd1e5d4: moving from state init -> state preparing
>Thread-15911264::DEBUG::2013-11-19 21:30:42,261::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=282 spmId=2
>Thread-15911264::DEBUG::2013-11-19 21:30:42,267::task::495::TaskManager.Task::(_debug) Task 9c090c21-1137-4cb7-bd40-56846cd1e5d4: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911264::DEBUG::2013-11-19 21:30:42,273::task::495::TaskManager.Task::(_debug) Task 9c090c21-1137-4cb7-bd40-56846cd1e5d4: moving from state preparing -> state finished
>Thread-15911264::DEBUG::2013-11-19 21:30:42,274::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911264::DEBUG::2013-11-19 21:30:42,281::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911264::DEBUG::2013-11-19 21:30:42,288::task::495::TaskManager.Task::(_debug) Task 9c090c21-1137-4cb7-bd40-56846cd1e5d4: ref 0 aborting False
>Thread-15911264::INFO::2013-11-19 21:30:42,288::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911265::DEBUG::2013-11-19 21:30:42,294::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911265::INFO::2013-11-19 21:30:42,295::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911265::DEBUG::2013-11-19 21:30:42,296::task::495::TaskManager.Task::(_debug) Task 3f39b349-ef89-4425-b719-b89c1a5e6ec4: moving from state init -> state preparing
>Thread-15911265::DEBUG::2013-11-19 21:30:42,297::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`709580a8-34c0-404e-8cde-23f779b76bc4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911265::DEBUG::2013-11-19 21:30:42,298::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911265::DEBUG::2013-11-19 21:30:42,299::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911265::DEBUG::2013-11-19 21:30:42,300::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`709580a8-34c0-404e-8cde-23f779b76bc4`::Granted request
>Thread-15911265::DEBUG::2013-11-19 21:30:42,301::task::495::TaskManager.Task::(_debug) Task 3f39b349-ef89-4425-b719-b89c1a5e6ec4: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911265::DEBUG::2013-11-19 21:30:42,301::task::495::TaskManager.Task::(_debug) Task 3f39b349-ef89-4425-b719-b89c1a5e6ec4: ref 1 aborting False
>Thread-15911265::WARNING::2013-11-19 21:30:42,304::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911265::DEBUG::2013-11-19 21:30:42,305::task::495::TaskManager.Task::(_debug) Task 3f39b349-ef89-4425-b719-b89c1a5e6ec4: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911265::DEBUG::2013-11-19 21:30:42,306::task::495::TaskManager.Task::(_debug) Task 3f39b349-ef89-4425-b719-b89c1a5e6ec4: moving from state preparing -> state finished
>Thread-15911265::DEBUG::2013-11-19 21:30:42,306::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911265::DEBUG::2013-11-19 21:30:42,307::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911265::DEBUG::2013-11-19 21:30:42,307::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911265::DEBUG::2013-11-19 21:30:42,308::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911265::DEBUG::2013-11-19 21:30:42,309::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911265::DEBUG::2013-11-19 21:30:42,309::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911265::DEBUG::2013-11-19 21:30:42,314::task::495::TaskManager.Task::(_debug) Task 3f39b349-ef89-4425-b719-b89c1a5e6ec4: ref 0 aborting False
>Thread-15911265::INFO::2013-11-19 21:30:42,317::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911266::INFO::2013-11-19 21:30:42,402::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911266::DEBUG::2013-11-19 21:30:42,403::task::495::TaskManager.Task::(_debug) Task 3b99929b-b481-4d73-b10c-e939ff9ea6ef: moving from state init -> state preparing
>Thread-15911266::DEBUG::2013-11-19 21:30:42,406::task::495::TaskManager.Task::(_debug) Task 3b99929b-b481-4d73-b10c-e939ff9ea6ef: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000479221343994', 'lastCheck': 1384896639.827662, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00146698951721', 'lastCheck': 1384896638.8318191, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896634.1899741, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00066089630127', 'lastCheck': 1384896634.737155, 'valid': True, 'code': 0}}
>Thread-15911266::DEBUG::2013-11-19 21:30:42,407::task::495::TaskManager.Task::(_debug) Task 3b99929b-b481-4d73-b10c-e939ff9ea6ef: moving from state preparing -> state finished
>Thread-15911266::DEBUG::2013-11-19 21:30:42,412::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911266::DEBUG::2013-11-19 21:30:42,417::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911266::DEBUG::2013-11-19 21:30:42,422::task::495::TaskManager.Task::(_debug) Task 3b99929b-b481-4d73-b10c-e939ff9ea6ef: ref 0 aborting False
>Thread-15911266::INFO::2013-11-19 21:30:42,423::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000479221343994', 'lastCheck': 1384896639.827662, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00146698951721', 'lastCheck': 1384896638.8318191, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896634.1899741, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00066089630127', 'lastCheck': 1384896634.737155, 'valid': True, 'code': 0}}
>Dummy-15911115::DEBUG::2013-11-19 21:30:42,730::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:30:43,260::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0788588 s, 13.0 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:30:44,381::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911115::DEBUG::2013-11-19 21:30:45,276::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:30:45,814::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0710354 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:30:47,830::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:30:48,354::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0713813 s, 14.3 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:30:50,370::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:30:50,888::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0711102 s, 14.4 MB/s\n'; <rc> = 0
>Thread-15911272::DEBUG::2013-11-19 21:30:52,469::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911272::INFO::2013-11-19 21:30:52,471::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911272::DEBUG::2013-11-19 21:30:52,479::task::495::TaskManager.Task::(_debug) Task e1bad405-3e36-44b9-b727-b6e0b1d07e50: moving from state init -> state preparing
>Thread-15911272::DEBUG::2013-11-19 21:30:52,485::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=282 spmId=2
>Thread-15911272::DEBUG::2013-11-19 21:30:52,488::task::495::TaskManager.Task::(_debug) Task e1bad405-3e36-44b9-b727-b6e0b1d07e50: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911272::DEBUG::2013-11-19 21:30:52,493::task::495::TaskManager.Task::(_debug) Task e1bad405-3e36-44b9-b727-b6e0b1d07e50: moving from state preparing -> state finished
>Thread-15911272::DEBUG::2013-11-19 21:30:52,503::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911272::DEBUG::2013-11-19 21:30:52,506::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911272::DEBUG::2013-11-19 21:30:52,512::task::495::TaskManager.Task::(_debug) Task e1bad405-3e36-44b9-b727-b6e0b1d07e50: ref 0 aborting False
>Thread-15911272::INFO::2013-11-19 21:30:52,514::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911273::DEBUG::2013-11-19 21:30:52,520::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911273::INFO::2013-11-19 21:30:52,521::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911273::DEBUG::2013-11-19 21:30:52,521::task::495::TaskManager.Task::(_debug) Task 5957c26e-571b-4e71-a3bc-824e081e6228: moving from state init -> state preparing
>Thread-15911273::DEBUG::2013-11-19 21:30:52,528::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`cdbf29da-a91f-448b-a13a-1b392f2ff3ec`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911273::DEBUG::2013-11-19 21:30:52,534::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911273::DEBUG::2013-11-19 21:30:52,536::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911273::DEBUG::2013-11-19 21:30:52,537::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`cdbf29da-a91f-448b-a13a-1b392f2ff3ec`::Granted request
>Thread-15911273::DEBUG::2013-11-19 21:30:52,538::task::495::TaskManager.Task::(_debug) Task 5957c26e-571b-4e71-a3bc-824e081e6228: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911273::DEBUG::2013-11-19 21:30:52,544::task::495::TaskManager.Task::(_debug) Task 5957c26e-571b-4e71-a3bc-824e081e6228: ref 1 aborting False
>Thread-15911273::WARNING::2013-11-19 21:30:52,546::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911273::DEBUG::2013-11-19 21:30:52,547::task::495::TaskManager.Task::(_debug) Task 5957c26e-571b-4e71-a3bc-824e081e6228: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911273::DEBUG::2013-11-19 21:30:52,548::task::495::TaskManager.Task::(_debug) Task 5957c26e-571b-4e71-a3bc-824e081e6228: moving from state preparing -> state finished
>Thread-15911273::DEBUG::2013-11-19 21:30:52,549::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911273::DEBUG::2013-11-19 21:30:52,549::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911273::DEBUG::2013-11-19 21:30:52,552::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911273::DEBUG::2013-11-19 21:30:52,557::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911273::DEBUG::2013-11-19 21:30:52,558::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911273::DEBUG::2013-11-19 21:30:52,561::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911273::DEBUG::2013-11-19 21:30:52,561::task::495::TaskManager.Task::(_debug) Task 5957c26e-571b-4e71-a3bc-824e081e6228: ref 0 aborting False
>Thread-15911273::INFO::2013-11-19 21:30:52,565::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911274::INFO::2013-11-19 21:30:52,585::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911274::DEBUG::2013-11-19 21:30:52,586::task::495::TaskManager.Task::(_debug) Task 5cd1e240-25fd-4a2f-a025-3ab9df75f5a6: moving from state init -> state preparing
>Thread-15911274::DEBUG::2013-11-19 21:30:52,587::task::495::TaskManager.Task::(_debug) Task 5cd1e240-25fd-4a2f-a025-3ab9df75f5a6: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000415086746216', 'lastCheck': 1384896650.044364, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00239205360413', 'lastCheck': 1384896649.0466111, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896644.382061, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000519990921021', 'lastCheck': 1384896644.939728, 'valid': True, 'code': 0}}
>Thread-15911274::DEBUG::2013-11-19 21:30:52,588::task::495::TaskManager.Task::(_debug) Task 5cd1e240-25fd-4a2f-a025-3ab9df75f5a6: moving from state preparing -> state finished
>Thread-15911274::DEBUG::2013-11-19 21:30:52,592::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911274::DEBUG::2013-11-19 21:30:52,592::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911274::DEBUG::2013-11-19 21:30:52,593::task::495::TaskManager.Task::(_debug) Task 5cd1e240-25fd-4a2f-a025-3ab9df75f5a6: ref 0 aborting False
>Thread-15911274::INFO::2013-11-19 21:30:52,598::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000415086746216', 'lastCheck': 1384896650.044364, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00239205360413', 'lastCheck': 1384896649.0466111, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896644.382061, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000519990921021', 'lastCheck': 1384896644.939728, 'valid': True, 'code': 0}}
>Dummy-15911115::DEBUG::2013-11-19 21:30:52,903::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:30:53,447::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0720865 s, 14.2 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:30:54,579::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911115::DEBUG::2013-11-19 21:30:55,463::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:30:55,949::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0454066 s, 22.6 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:30:57,960::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:30:58,497::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0740901 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:31:00,512::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:31:01,044::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0730413 s, 14.0 MB/s\n'; <rc> = 0
>Thread-15911280::DEBUG::2013-11-19 21:31:02,722::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911280::INFO::2013-11-19 21:31:02,728::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911280::DEBUG::2013-11-19 21:31:02,729::task::495::TaskManager.Task::(_debug) Task 5ad184f2-d95b-44a0-a8d9-a29ad354b7eb: moving from state init -> state preparing
>Thread-15911280::DEBUG::2013-11-19 21:31:02,730::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=282 spmId=2
>Thread-15911280::DEBUG::2013-11-19 21:31:02,730::task::495::TaskManager.Task::(_debug) Task 5ad184f2-d95b-44a0-a8d9-a29ad354b7eb: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911280::DEBUG::2013-11-19 21:31:02,740::task::495::TaskManager.Task::(_debug) Task 5ad184f2-d95b-44a0-a8d9-a29ad354b7eb: moving from state preparing -> state finished
>Thread-15911280::DEBUG::2013-11-19 21:31:02,744::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911280::DEBUG::2013-11-19 21:31:02,754::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911280::DEBUG::2013-11-19 21:31:02,758::task::495::TaskManager.Task::(_debug) Task 5ad184f2-d95b-44a0-a8d9-a29ad354b7eb: ref 0 aborting False
>Thread-15911280::INFO::2013-11-19 21:31:02,759::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911281::DEBUG::2013-11-19 21:31:02,767::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911281::INFO::2013-11-19 21:31:02,771::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911281::DEBUG::2013-11-19 21:31:02,771::task::495::TaskManager.Task::(_debug) Task 29e9ee31-458f-4c77-8198-ba5271afa4c9: moving from state init -> state preparing
>Thread-15911281::DEBUG::2013-11-19 21:31:02,779::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a536898a-de29-41f5-9b8a-edd0ceac1484`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911281::DEBUG::2013-11-19 21:31:02,781::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911281::DEBUG::2013-11-19 21:31:02,785::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911281::DEBUG::2013-11-19 21:31:02,790::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a536898a-de29-41f5-9b8a-edd0ceac1484`::Granted request
>Thread-15911282::INFO::2013-11-19 21:31:02,810::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911281::DEBUG::2013-11-19 21:31:02,811::task::495::TaskManager.Task::(_debug) Task 29e9ee31-458f-4c77-8198-ba5271afa4c9: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911282::DEBUG::2013-11-19 21:31:02,811::task::495::TaskManager.Task::(_debug) Task d4653dad-07ca-4f23-b689-4fadf02a6941: moving from state init -> state preparing
>Thread-15911281::DEBUG::2013-11-19 21:31:02,816::task::495::TaskManager.Task::(_debug) Task 29e9ee31-458f-4c77-8198-ba5271afa4c9: ref 1 aborting False
>Thread-15911282::DEBUG::2013-11-19 21:31:02,817::task::495::TaskManager.Task::(_debug) Task d4653dad-07ca-4f23-b689-4fadf02a6941: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000400066375732', 'lastCheck': 1384896660.1900761, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00151205062866', 'lastCheck': 1384896659.1837609, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896654.5803289, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000839948654175', 'lastCheck': 1384896655.135042, 'valid': True, 'code': 0}}
>Thread-15911282::DEBUG::2013-11-19 21:31:02,831::task::495::TaskManager.Task::(_debug) Task d4653dad-07ca-4f23-b689-4fadf02a6941: moving from state preparing -> state finished
>Thread-15911281::WARNING::2013-11-19 21:31:02,832::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911282::DEBUG::2013-11-19 21:31:02,833::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911281::DEBUG::2013-11-19 21:31:02,834::task::495::TaskManager.Task::(_debug) Task 29e9ee31-458f-4c77-8198-ba5271afa4c9: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911282::DEBUG::2013-11-19 21:31:02,835::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911281::DEBUG::2013-11-19 21:31:02,836::task::495::TaskManager.Task::(_debug) Task 29e9ee31-458f-4c77-8198-ba5271afa4c9: moving from state preparing -> state finished
>Thread-15911282::DEBUG::2013-11-19 21:31:02,836::task::495::TaskManager.Task::(_debug) Task d4653dad-07ca-4f23-b689-4fadf02a6941: ref 0 aborting False
>Thread-15911281::DEBUG::2013-11-19 21:31:02,841::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911282::INFO::2013-11-19 21:31:02,848::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000400066375732', 'lastCheck': 1384896660.1900761, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00151205062866', 'lastCheck': 1384896659.1837609, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896654.5803289, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000839948654175', 'lastCheck': 1384896655.135042, 'valid': True, 'code': 0}}
>Thread-15911281::DEBUG::2013-11-19 21:31:02,848::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911281::DEBUG::2013-11-19 21:31:02,863::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911281::DEBUG::2013-11-19 21:31:02,864::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911281::DEBUG::2013-11-19 21:31:02,867::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911281::DEBUG::2013-11-19 21:31:02,872::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911281::DEBUG::2013-11-19 21:31:02,877::task::495::TaskManager.Task::(_debug) Task 29e9ee31-458f-4c77-8198-ba5271afa4c9: ref 0 aborting False
>Thread-15911281::INFO::2013-11-19 21:31:02,878::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911115::DEBUG::2013-11-19 21:31:03,059::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:31:03,597::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0736866 s, 13.9 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:31:04,752::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911115::DEBUG::2013-11-19 21:31:05,612::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:31:06,127::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0727431 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:31:08,141::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:31:08,663::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0735321 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:31:10,679::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:31:11,224::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0738136 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15911288::DEBUG::2013-11-19 21:31:13,037::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911288::INFO::2013-11-19 21:31:13,047::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911288::DEBUG::2013-11-19 21:31:13,048::task::495::TaskManager.Task::(_debug) Task e7f4c79d-7a77-4746-866c-aebdd23adb0d: moving from state init -> state preparing
>Thread-15911288::DEBUG::2013-11-19 21:31:13,049::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=282 spmId=2
>Thread-15911288::DEBUG::2013-11-19 21:31:13,050::task::495::TaskManager.Task::(_debug) Task e7f4c79d-7a77-4746-866c-aebdd23adb0d: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911288::DEBUG::2013-11-19 21:31:13,051::task::495::TaskManager.Task::(_debug) Task e7f4c79d-7a77-4746-866c-aebdd23adb0d: moving from state preparing -> state finished
>Thread-15911288::DEBUG::2013-11-19 21:31:13,054::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911289::INFO::2013-11-19 21:31:13,063::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911288::DEBUG::2013-11-19 21:31:13,064::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911289::DEBUG::2013-11-19 21:31:13,064::task::495::TaskManager.Task::(_debug) Task 3d4b33a5-4356-48c5-8c8f-24324f7b4077: moving from state init -> state preparing
>Thread-15911288::DEBUG::2013-11-19 21:31:13,069::task::495::TaskManager.Task::(_debug) Task e7f4c79d-7a77-4746-866c-aebdd23adb0d: ref 0 aborting False
>Thread-15911289::DEBUG::2013-11-19 21:31:13,070::task::495::TaskManager.Task::(_debug) Task 3d4b33a5-4356-48c5-8c8f-24324f7b4077: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000416994094849', 'lastCheck': 1384896670.3685949, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00151395797729', 'lastCheck': 1384896669.3689001, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896664.7532351, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000532150268555', 'lastCheck': 1384896665.2833481, 'valid': True, 'code': 0}}
>Thread-15911288::INFO::2013-11-19 21:31:13,074::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911289::DEBUG::2013-11-19 21:31:13,085::task::495::TaskManager.Task::(_debug) Task 3d4b33a5-4356-48c5-8c8f-24324f7b4077: moving from state preparing -> state finished
>Thread-15911289::DEBUG::2013-11-19 21:31:13,097::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911289::DEBUG::2013-11-19 21:31:13,098::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911289::DEBUG::2013-11-19 21:31:13,098::task::495::TaskManager.Task::(_debug) Task 3d4b33a5-4356-48c5-8c8f-24324f7b4077: ref 0 aborting False
>Thread-15911289::INFO::2013-11-19 21:31:13,099::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000416994094849', 'lastCheck': 1384896670.3685949, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00151395797729', 'lastCheck': 1384896669.3689001, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896664.7532351, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000532150268555', 'lastCheck': 1384896665.2833481, 'valid': True, 'code': 0}}
>Thread-15911290::DEBUG::2013-11-19 21:31:13,105::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911290::INFO::2013-11-19 21:31:13,123::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911290::DEBUG::2013-11-19 21:31:13,124::task::495::TaskManager.Task::(_debug) Task 5d037206-4dd1-442b-ad79-76735c1158c4: moving from state init -> state preparing
>Thread-15911290::DEBUG::2013-11-19 21:31:13,125::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`71ec479d-deae-4c5c-8f40-e3674335f62d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911290::DEBUG::2013-11-19 21:31:13,131::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911290::DEBUG::2013-11-19 21:31:13,136::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911290::DEBUG::2013-11-19 21:31:13,139::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`71ec479d-deae-4c5c-8f40-e3674335f62d`::Granted request
>Thread-15911290::DEBUG::2013-11-19 21:31:13,142::task::495::TaskManager.Task::(_debug) Task 5d037206-4dd1-442b-ad79-76735c1158c4: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911290::DEBUG::2013-11-19 21:31:13,146::task::495::TaskManager.Task::(_debug) Task 5d037206-4dd1-442b-ad79-76735c1158c4: ref 1 aborting False
>Thread-15911290::WARNING::2013-11-19 21:31:13,152::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911290::DEBUG::2013-11-19 21:31:13,157::task::495::TaskManager.Task::(_debug) Task 5d037206-4dd1-442b-ad79-76735c1158c4: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911290::DEBUG::2013-11-19 21:31:13,161::task::495::TaskManager.Task::(_debug) Task 5d037206-4dd1-442b-ad79-76735c1158c4: moving from state preparing -> state finished
>Thread-15911290::DEBUG::2013-11-19 21:31:13,167::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911290::DEBUG::2013-11-19 21:31:13,168::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911290::DEBUG::2013-11-19 21:31:13,169::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911290::DEBUG::2013-11-19 21:31:13,174::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911290::DEBUG::2013-11-19 21:31:13,175::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911290::DEBUG::2013-11-19 21:31:13,175::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911290::DEBUG::2013-11-19 21:31:13,176::task::495::TaskManager.Task::(_debug) Task 5d037206-4dd1-442b-ad79-76735c1158c4: ref 0 aborting False
>Thread-15911290::INFO::2013-11-19 21:31:13,177::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911115::DEBUG::2013-11-19 21:31:13,239::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:31:13,773::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0748828 s, 13.7 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:31:14,927::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911115::DEBUG::2013-11-19 21:31:15,788::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:31:16,348::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0904018 s, 11.3 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:31:18,363::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:31:18,897::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0919825 s, 11.1 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:31:20,912::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:31:21,449::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0739486 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15911296::DEBUG::2013-11-19 21:31:23,289::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911296::INFO::2013-11-19 21:31:23,296::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911296::DEBUG::2013-11-19 21:31:23,299::task::495::TaskManager.Task::(_debug) Task 0a426eb1-364c-42e3-bc28-aa91f626746c: moving from state init -> state preparing
>Thread-15911296::DEBUG::2013-11-19 21:31:23,300::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=282 spmId=2
>Thread-15911296::DEBUG::2013-11-19 21:31:23,305::task::495::TaskManager.Task::(_debug) Task 0a426eb1-364c-42e3-bc28-aa91f626746c: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911297::INFO::2013-11-19 21:31:23,308::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911296::DEBUG::2013-11-19 21:31:23,310::task::495::TaskManager.Task::(_debug) Task 0a426eb1-364c-42e3-bc28-aa91f626746c: moving from state preparing -> state finished
>Thread-15911297::DEBUG::2013-11-19 21:31:23,311::task::495::TaskManager.Task::(_debug) Task 90d7c265-97be-4a00-ac88-f919976144d9: moving from state init -> state preparing
>Thread-15911296::DEBUG::2013-11-19 21:31:23,319::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911297::DEBUG::2013-11-19 21:31:23,320::task::495::TaskManager.Task::(_debug) Task 90d7c265-97be-4a00-ac88-f919976144d9: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000410079956055', 'lastCheck': 1384896680.567976, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00147795677185', 'lastCheck': 1384896679.5693519, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896674.9286611, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000479936599731', 'lastCheck': 1384896675.451674, 'valid': True, 'code': 0}}
>Thread-15911296::DEBUG::2013-11-19 21:31:23,329::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911297::DEBUG::2013-11-19 21:31:23,330::task::495::TaskManager.Task::(_debug) Task 90d7c265-97be-4a00-ac88-f919976144d9: moving from state preparing -> state finished
>Thread-15911296::DEBUG::2013-11-19 21:31:23,331::task::495::TaskManager.Task::(_debug) Task 0a426eb1-364c-42e3-bc28-aa91f626746c: ref 0 aborting False
>Thread-15911297::DEBUG::2013-11-19 21:31:23,331::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911296::INFO::2013-11-19 21:31:23,337::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911297::DEBUG::2013-11-19 21:31:23,338::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911297::DEBUG::2013-11-19 21:31:23,342::task::495::TaskManager.Task::(_debug) Task 90d7c265-97be-4a00-ac88-f919976144d9: ref 0 aborting False
>Thread-15911297::INFO::2013-11-19 21:31:23,345::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000410079956055', 'lastCheck': 1384896680.567976, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00147795677185', 'lastCheck': 1384896679.5693519, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896674.9286611, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000479936599731', 'lastCheck': 1384896675.451674, 'valid': True, 'code': 0}}
>Thread-15911298::DEBUG::2013-11-19 21:31:23,368::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911298::INFO::2013-11-19 21:31:23,378::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911298::DEBUG::2013-11-19 21:31:23,397::task::495::TaskManager.Task::(_debug) Task 19488453-461a-477c-bb71-521d21aca2e0: moving from state init -> state preparing
>Thread-15911298::DEBUG::2013-11-19 21:31:23,413::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`147a4ef5-1421-4e97-9180-0e4f12fb3e3a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911298::DEBUG::2013-11-19 21:31:23,414::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911298::DEBUG::2013-11-19 21:31:23,418::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911298::DEBUG::2013-11-19 21:31:23,421::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`147a4ef5-1421-4e97-9180-0e4f12fb3e3a`::Granted request
>Thread-15911298::DEBUG::2013-11-19 21:31:23,427::task::495::TaskManager.Task::(_debug) Task 19488453-461a-477c-bb71-521d21aca2e0: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911298::DEBUG::2013-11-19 21:31:23,432::task::495::TaskManager.Task::(_debug) Task 19488453-461a-477c-bb71-521d21aca2e0: ref 1 aborting False
>Thread-15911298::WARNING::2013-11-19 21:31:23,438::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911298::DEBUG::2013-11-19 21:31:23,439::task::495::TaskManager.Task::(_debug) Task 19488453-461a-477c-bb71-521d21aca2e0: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911298::DEBUG::2013-11-19 21:31:23,441::task::495::TaskManager.Task::(_debug) Task 19488453-461a-477c-bb71-521d21aca2e0: moving from state preparing -> state finished
>Thread-15911298::DEBUG::2013-11-19 21:31:23,444::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911298::DEBUG::2013-11-19 21:31:23,445::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911298::DEBUG::2013-11-19 21:31:23,446::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911298::DEBUG::2013-11-19 21:31:23,452::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911298::DEBUG::2013-11-19 21:31:23,452::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911298::DEBUG::2013-11-19 21:31:23,458::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911298::DEBUG::2013-11-19 21:31:23,460::task::495::TaskManager.Task::(_debug) Task 19488453-461a-477c-bb71-521d21aca2e0: ref 0 aborting False
>Dummy-15911115::DEBUG::2013-11-19 21:31:23,464::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15911298::INFO::2013-11-19 21:31:23,466::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911115::DEBUG::2013-11-19 21:31:23,994::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0741411 s, 13.8 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:31:25,137::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911115::DEBUG::2013-11-19 21:31:26,009::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:31:26,541::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0733732 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:31:28,557::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:31:29,101::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0751608 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:31:31,116::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:31:31,640::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0732934 s, 14.0 MB/s\n'; <rc> = 0
>Thread-15911304::INFO::2013-11-19 21:31:33,520::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911304::DEBUG::2013-11-19 21:31:33,521::task::495::TaskManager.Task::(_debug) Task 8d918c69-fc09-42fb-b03c-1183dc22ede1: moving from state init -> state preparing
>Thread-15911304::DEBUG::2013-11-19 21:31:33,532::task::495::TaskManager.Task::(_debug) Task 8d918c69-fc09-42fb-b03c-1183dc22ede1: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00038480758667', 'lastCheck': 1384896690.7822969, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00155901908875', 'lastCheck': 1384896689.7859249, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896685.138437, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000769853591919', 'lastCheck': 1384896685.6800931, 'valid': True, 'code': 0}}
>Thread-15911304::DEBUG::2013-11-19 21:31:33,538::task::495::TaskManager.Task::(_debug) Task 8d918c69-fc09-42fb-b03c-1183dc22ede1: moving from state preparing -> state finished
>Thread-15911304::DEBUG::2013-11-19 21:31:33,538::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911304::DEBUG::2013-11-19 21:31:33,541::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911304::DEBUG::2013-11-19 21:31:33,547::task::495::TaskManager.Task::(_debug) Task 8d918c69-fc09-42fb-b03c-1183dc22ede1: ref 0 aborting False
>Thread-15911304::INFO::2013-11-19 21:31:33,549::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00038480758667', 'lastCheck': 1384896690.7822969, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00155901908875', 'lastCheck': 1384896689.7859249, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896685.138437, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000769853591919', 'lastCheck': 1384896685.6800931, 'valid': True, 'code': 0}}
>Dummy-15911115::DEBUG::2013-11-19 21:31:33,655::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15911306::DEBUG::2013-11-19 21:31:33,910::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911306::INFO::2013-11-19 21:31:33,921::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911306::DEBUG::2013-11-19 21:31:33,931::task::495::TaskManager.Task::(_debug) Task 203780d8-f6a8-4384-bdaf-48ca4e57aac4: moving from state init -> state preparing
>Thread-15911306::DEBUG::2013-11-19 21:31:33,937::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=282 spmId=2
>Thread-15911306::DEBUG::2013-11-19 21:31:33,948::task::495::TaskManager.Task::(_debug) Task 203780d8-f6a8-4384-bdaf-48ca4e57aac4: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911306::DEBUG::2013-11-19 21:31:33,950::task::495::TaskManager.Task::(_debug) Task 203780d8-f6a8-4384-bdaf-48ca4e57aac4: moving from state preparing -> state finished
>Thread-15911306::DEBUG::2013-11-19 21:31:33,951::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911306::DEBUG::2013-11-19 21:31:33,956::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911306::DEBUG::2013-11-19 21:31:33,962::task::495::TaskManager.Task::(_debug) Task 203780d8-f6a8-4384-bdaf-48ca4e57aac4: ref 0 aborting False
>Thread-15911306::INFO::2013-11-19 21:31:33,963::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911307::DEBUG::2013-11-19 21:31:33,973::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911307::INFO::2013-11-19 21:31:33,977::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911307::DEBUG::2013-11-19 21:31:33,986::task::495::TaskManager.Task::(_debug) Task bd1c5134-4903-4214-8a5c-f618cecc43f8: moving from state init -> state preparing
>Thread-15911307::DEBUG::2013-11-19 21:31:33,988::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ca2f3a68-98bb-423b-8f9c-5dbe7c16f333`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911307::DEBUG::2013-11-19 21:31:33,993::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911307::DEBUG::2013-11-19 21:31:33,999::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911307::DEBUG::2013-11-19 21:31:34,005::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ca2f3a68-98bb-423b-8f9c-5dbe7c16f333`::Granted request
>Thread-15911307::DEBUG::2013-11-19 21:31:34,007::task::495::TaskManager.Task::(_debug) Task bd1c5134-4903-4214-8a5c-f618cecc43f8: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911307::DEBUG::2013-11-19 21:31:34,008::task::495::TaskManager.Task::(_debug) Task bd1c5134-4903-4214-8a5c-f618cecc43f8: ref 1 aborting False
>Thread-15911307::WARNING::2013-11-19 21:31:34,015::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911307::DEBUG::2013-11-19 21:31:34,016::task::495::TaskManager.Task::(_debug) Task bd1c5134-4903-4214-8a5c-f618cecc43f8: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911307::DEBUG::2013-11-19 21:31:34,017::task::495::TaskManager.Task::(_debug) Task bd1c5134-4903-4214-8a5c-f618cecc43f8: moving from state preparing -> state finished
>Thread-15911307::DEBUG::2013-11-19 21:31:34,020::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911307::DEBUG::2013-11-19 21:31:34,026::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911307::DEBUG::2013-11-19 21:31:34,030::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911307::DEBUG::2013-11-19 21:31:34,033::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911307::DEBUG::2013-11-19 21:31:34,037::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911307::DEBUG::2013-11-19 21:31:34,041::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911307::DEBUG::2013-11-19 21:31:34,045::task::495::TaskManager.Task::(_debug) Task bd1c5134-4903-4214-8a5c-f618cecc43f8: ref 0 aborting False
>Thread-15911307::INFO::2013-11-19 21:31:34,050::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911115::DEBUG::2013-11-19 21:31:34,200::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0744408 s, 13.8 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:31:35,341::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911115::DEBUG::2013-11-19 21:31:36,215::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:31:36,744::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0736149 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:31:38,760::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:31:39,290::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0725406 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:31:41,306::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:31:41,853::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0898653 s, 11.4 MB/s\n'; <rc> = 0
>Thread-15911312::INFO::2013-11-19 21:31:43,752::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911312::DEBUG::2013-11-19 21:31:43,753::task::495::TaskManager.Task::(_debug) Task aea4d7b0-7f0e-4151-8e83-94581bd5e5e6: moving from state init -> state preparing
>Thread-15911312::DEBUG::2013-11-19 21:31:43,760::task::495::TaskManager.Task::(_debug) Task aea4d7b0-7f0e-4151-8e83-94581bd5e5e6: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000498056411743', 'lastCheck': 1384896700.9948461, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00154805183411', 'lastCheck': 1384896699.995724, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896695.34273, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000533103942871', 'lastCheck': 1384896695.8908579, 'valid': True, 'code': 0}}
>Thread-15911312::DEBUG::2013-11-19 21:31:43,761::task::495::TaskManager.Task::(_debug) Task aea4d7b0-7f0e-4151-8e83-94581bd5e5e6: moving from state preparing -> state finished
>Thread-15911312::DEBUG::2013-11-19 21:31:43,764::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911312::DEBUG::2013-11-19 21:31:43,767::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911312::DEBUG::2013-11-19 21:31:43,777::task::495::TaskManager.Task::(_debug) Task aea4d7b0-7f0e-4151-8e83-94581bd5e5e6: ref 0 aborting False
>Thread-15911312::INFO::2013-11-19 21:31:43,781::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000498056411743', 'lastCheck': 1384896700.9948461, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00154805183411', 'lastCheck': 1384896699.995724, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896695.34273, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000533103942871', 'lastCheck': 1384896695.8908579, 'valid': True, 'code': 0}}
>Dummy-15911115::DEBUG::2013-11-19 21:31:43,868::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15911314::DEBUG::2013-11-19 21:31:44,168::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911314::INFO::2013-11-19 21:31:44,175::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911314::DEBUG::2013-11-19 21:31:44,186::task::495::TaskManager.Task::(_debug) Task f2d666ba-ae83-4717-bd07-cd0e3e8a0e82: moving from state init -> state preparing
>Thread-15911314::DEBUG::2013-11-19 21:31:44,193::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=282 spmId=2
>Thread-15911314::DEBUG::2013-11-19 21:31:44,203::task::495::TaskManager.Task::(_debug) Task f2d666ba-ae83-4717-bd07-cd0e3e8a0e82: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911314::DEBUG::2013-11-19 21:31:44,209::task::495::TaskManager.Task::(_debug) Task f2d666ba-ae83-4717-bd07-cd0e3e8a0e82: moving from state preparing -> state finished
>Thread-15911314::DEBUG::2013-11-19 21:31:44,213::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911314::DEBUG::2013-11-19 21:31:44,218::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911314::DEBUG::2013-11-19 21:31:44,218::task::495::TaskManager.Task::(_debug) Task f2d666ba-ae83-4717-bd07-cd0e3e8a0e82: ref 0 aborting False
>Thread-15911314::INFO::2013-11-19 21:31:44,220::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911315::DEBUG::2013-11-19 21:31:44,230::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911315::INFO::2013-11-19 21:31:44,236::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911315::DEBUG::2013-11-19 21:31:44,241::task::495::TaskManager.Task::(_debug) Task bcf480b9-87ac-40c3-9d5a-b6c229e12a55: moving from state init -> state preparing
>Thread-15911315::DEBUG::2013-11-19 21:31:44,243::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a551880b-8bd6-4c24-92e0-e1cab09ef4f9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911315::DEBUG::2013-11-19 21:31:44,248::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911315::DEBUG::2013-11-19 21:31:44,255::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911315::DEBUG::2013-11-19 21:31:44,259::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a551880b-8bd6-4c24-92e0-e1cab09ef4f9`::Granted request
>Thread-15911315::DEBUG::2013-11-19 21:31:44,260::task::495::TaskManager.Task::(_debug) Task bcf480b9-87ac-40c3-9d5a-b6c229e12a55: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911315::DEBUG::2013-11-19 21:31:44,263::task::495::TaskManager.Task::(_debug) Task bcf480b9-87ac-40c3-9d5a-b6c229e12a55: ref 1 aborting False
>Thread-15911315::WARNING::2013-11-19 21:31:44,265::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911315::DEBUG::2013-11-19 21:31:44,266::task::495::TaskManager.Task::(_debug) Task bcf480b9-87ac-40c3-9d5a-b6c229e12a55: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911315::DEBUG::2013-11-19 21:31:44,267::task::495::TaskManager.Task::(_debug) Task bcf480b9-87ac-40c3-9d5a-b6c229e12a55: moving from state preparing -> state finished
>Thread-15911315::DEBUG::2013-11-19 21:31:44,271::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911315::DEBUG::2013-11-19 21:31:44,275::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911315::DEBUG::2013-11-19 21:31:44,280::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911315::DEBUG::2013-11-19 21:31:44,282::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911315::DEBUG::2013-11-19 21:31:44,285::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911315::DEBUG::2013-11-19 21:31:44,285::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911315::DEBUG::2013-11-19 21:31:44,287::task::495::TaskManager.Task::(_debug) Task bcf480b9-87ac-40c3-9d5a-b6c229e12a55: ref 0 aborting False
>Thread-15911315::INFO::2013-11-19 21:31:44,288::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911115::DEBUG::2013-11-19 21:31:44,361::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0562281 s, 18.2 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:31:45,526::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911115::DEBUG::2013-11-19 21:31:46,371::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:31:46,892::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0747638 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:31:48,908::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:31:49,470::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0976026 s, 10.5 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:31:51,485::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:31:52,023::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0937152 s, 10.9 MB/s\n'; <rc> = 0
>Thread-15911320::INFO::2013-11-19 21:31:53,952::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911320::DEBUG::2013-11-19 21:31:53,957::task::495::TaskManager.Task::(_debug) Task 0462a362-af71-4510-851f-a20c00d33896: moving from state init -> state preparing
>Thread-15911320::DEBUG::2013-11-19 21:31:53,962::task::495::TaskManager.Task::(_debug) Task 0462a362-af71-4510-851f-a20c00d33896: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000231027603149', 'lastCheck': 1384896711.144629, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00188899040222', 'lastCheck': 1384896710.150712, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896705.527746, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000523090362549', 'lastCheck': 1384896706.089349, 'valid': True, 'code': 0}}
>Thread-15911320::DEBUG::2013-11-19 21:31:53,972::task::495::TaskManager.Task::(_debug) Task 0462a362-af71-4510-851f-a20c00d33896: moving from state preparing -> state finished
>Thread-15911320::DEBUG::2013-11-19 21:31:53,976::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911320::DEBUG::2013-11-19 21:31:53,979::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911320::DEBUG::2013-11-19 21:31:53,979::task::495::TaskManager.Task::(_debug) Task 0462a362-af71-4510-851f-a20c00d33896: ref 0 aborting False
>Thread-15911320::INFO::2013-11-19 21:31:53,980::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000231027603149', 'lastCheck': 1384896711.144629, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00188899040222', 'lastCheck': 1384896710.150712, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896705.527746, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000523090362549', 'lastCheck': 1384896706.089349, 'valid': True, 'code': 0}}
>Dummy-15911115::DEBUG::2013-11-19 21:31:54,038::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15911322::DEBUG::2013-11-19 21:31:54,400::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911322::INFO::2013-11-19 21:31:54,406::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911322::DEBUG::2013-11-19 21:31:54,407::task::495::TaskManager.Task::(_debug) Task 8e4cab5b-3068-482f-b635-d0632d6974b1: moving from state init -> state preparing
>Thread-15911322::DEBUG::2013-11-19 21:31:54,411::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=282 spmId=2
>Thread-15911322::DEBUG::2013-11-19 21:31:54,415::task::495::TaskManager.Task::(_debug) Task 8e4cab5b-3068-482f-b635-d0632d6974b1: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911322::DEBUG::2013-11-19 21:31:54,422::task::495::TaskManager.Task::(_debug) Task 8e4cab5b-3068-482f-b635-d0632d6974b1: moving from state preparing -> state finished
>Thread-15911322::DEBUG::2013-11-19 21:31:54,428::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911322::DEBUG::2013-11-19 21:31:54,434::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911322::DEBUG::2013-11-19 21:31:54,439::task::495::TaskManager.Task::(_debug) Task 8e4cab5b-3068-482f-b635-d0632d6974b1: ref 0 aborting False
>Thread-15911322::INFO::2013-11-19 21:31:54,444::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911323::DEBUG::2013-11-19 21:31:54,456::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911323::INFO::2013-11-19 21:31:54,458::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911323::DEBUG::2013-11-19 21:31:54,462::task::495::TaskManager.Task::(_debug) Task 43513123-8c22-4d68-a96d-8442ede0103f: moving from state init -> state preparing
>Thread-15911323::DEBUG::2013-11-19 21:31:54,468::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e312e8e4-c5d5-4227-ac17-65882aa64857`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911323::DEBUG::2013-11-19 21:31:54,475::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911323::DEBUG::2013-11-19 21:31:54,485::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911323::DEBUG::2013-11-19 21:31:54,491::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e312e8e4-c5d5-4227-ac17-65882aa64857`::Granted request
>Thread-15911323::DEBUG::2013-11-19 21:31:54,496::task::495::TaskManager.Task::(_debug) Task 43513123-8c22-4d68-a96d-8442ede0103f: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911323::DEBUG::2013-11-19 21:31:54,500::task::495::TaskManager.Task::(_debug) Task 43513123-8c22-4d68-a96d-8442ede0103f: ref 1 aborting False
>Thread-15911323::WARNING::2013-11-19 21:31:54,505::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911323::DEBUG::2013-11-19 21:31:54,510::task::495::TaskManager.Task::(_debug) Task 43513123-8c22-4d68-a96d-8442ede0103f: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911323::DEBUG::2013-11-19 21:31:54,513::task::495::TaskManager.Task::(_debug) Task 43513123-8c22-4d68-a96d-8442ede0103f: moving from state preparing -> state finished
>Thread-15911323::DEBUG::2013-11-19 21:31:54,521::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911323::DEBUG::2013-11-19 21:31:54,527::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911323::DEBUG::2013-11-19 21:31:54,528::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911323::DEBUG::2013-11-19 21:31:54,533::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911323::DEBUG::2013-11-19 21:31:54,534::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911323::DEBUG::2013-11-19 21:31:54,535::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911323::DEBUG::2013-11-19 21:31:54,539::task::495::TaskManager.Task::(_debug) Task 43513123-8c22-4d68-a96d-8442ede0103f: ref 0 aborting False
>Thread-15911323::INFO::2013-11-19 21:31:54,544::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911115::DEBUG::2013-11-19 21:31:54,598::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.092226 s, 11.1 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:31:55,691::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911115::DEBUG::2013-11-19 21:31:56,615::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:31:57,127::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0709177 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:31:59,141::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:31:59,685::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0709683 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:32:01,701::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:32:02,265::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.109372 s, 9.4 MB/s\n'; <rc> = 0
>Thread-15911328::INFO::2013-11-19 21:32:04,152::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911328::DEBUG::2013-11-19 21:32:04,153::task::495::TaskManager.Task::(_debug) Task 6b21e019-3a01-4da4-819b-f04b66e61113: moving from state init -> state preparing
>Thread-15911328::DEBUG::2013-11-19 21:32:04,159::task::495::TaskManager.Task::(_debug) Task 6b21e019-3a01-4da4-819b-f04b66e61113: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000411033630371', 'lastCheck': 1384896721.3771479, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00153112411499', 'lastCheck': 1384896720.377507, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896715.69279, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000561952590942', 'lastCheck': 1384896716.2595651, 'valid': True, 'code': 0}}
>Thread-15911328::DEBUG::2013-11-19 21:32:04,161::task::495::TaskManager.Task::(_debug) Task 6b21e019-3a01-4da4-819b-f04b66e61113: moving from state preparing -> state finished
>Thread-15911328::DEBUG::2013-11-19 21:32:04,161::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911328::DEBUG::2013-11-19 21:32:04,167::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911328::DEBUG::2013-11-19 21:32:04,167::task::495::TaskManager.Task::(_debug) Task 6b21e019-3a01-4da4-819b-f04b66e61113: ref 0 aborting False
>Thread-15911328::INFO::2013-11-19 21:32:04,173::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000411033630371', 'lastCheck': 1384896721.3771479, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00153112411499', 'lastCheck': 1384896720.377507, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896715.69279, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000561952590942', 'lastCheck': 1384896716.2595651, 'valid': True, 'code': 0}}
>Dummy-15911115::DEBUG::2013-11-19 21:32:04,280::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15911330::DEBUG::2013-11-19 21:32:04,657::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911330::INFO::2013-11-19 21:32:04,661::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911330::DEBUG::2013-11-19 21:32:04,665::task::495::TaskManager.Task::(_debug) Task 068bae4c-73f8-4637-a18c-b8826878a90d: moving from state init -> state preparing
>Thread-15911330::DEBUG::2013-11-19 21:32:04,672::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=282 spmId=2
>Thread-15911330::DEBUG::2013-11-19 21:32:04,673::task::495::TaskManager.Task::(_debug) Task 068bae4c-73f8-4637-a18c-b8826878a90d: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911330::DEBUG::2013-11-19 21:32:04,674::task::495::TaskManager.Task::(_debug) Task 068bae4c-73f8-4637-a18c-b8826878a90d: moving from state preparing -> state finished
>Thread-15911330::DEBUG::2013-11-19 21:32:04,679::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911330::DEBUG::2013-11-19 21:32:04,680::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911330::DEBUG::2013-11-19 21:32:04,680::task::495::TaskManager.Task::(_debug) Task 068bae4c-73f8-4637-a18c-b8826878a90d: ref 0 aborting False
>Thread-15911330::INFO::2013-11-19 21:32:04,681::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911331::DEBUG::2013-11-19 21:32:04,689::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911331::INFO::2013-11-19 21:32:04,691::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911331::DEBUG::2013-11-19 21:32:04,697::task::495::TaskManager.Task::(_debug) Task 621a2743-8d68-40a1-9f95-b17c476f11a9: moving from state init -> state preparing
>Thread-15911331::DEBUG::2013-11-19 21:32:04,704::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b6cfa35a-874d-4e6b-a6fe-5cc25e58bb9b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911331::DEBUG::2013-11-19 21:32:04,705::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911331::DEBUG::2013-11-19 21:32:04,706::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911331::DEBUG::2013-11-19 21:32:04,711::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b6cfa35a-874d-4e6b-a6fe-5cc25e58bb9b`::Granted request
>Thread-15911331::DEBUG::2013-11-19 21:32:04,718::task::495::TaskManager.Task::(_debug) Task 621a2743-8d68-40a1-9f95-b17c476f11a9: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911331::DEBUG::2013-11-19 21:32:04,724::task::495::TaskManager.Task::(_debug) Task 621a2743-8d68-40a1-9f95-b17c476f11a9: ref 1 aborting False
>Thread-15911331::WARNING::2013-11-19 21:32:04,731::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911331::DEBUG::2013-11-19 21:32:04,737::task::495::TaskManager.Task::(_debug) Task 621a2743-8d68-40a1-9f95-b17c476f11a9: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911331::DEBUG::2013-11-19 21:32:04,738::task::495::TaskManager.Task::(_debug) Task 621a2743-8d68-40a1-9f95-b17c476f11a9: moving from state preparing -> state finished
>Thread-15911331::DEBUG::2013-11-19 21:32:04,743::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911331::DEBUG::2013-11-19 21:32:04,747::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911331::DEBUG::2013-11-19 21:32:04,752::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911331::DEBUG::2013-11-19 21:32:04,756::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911331::DEBUG::2013-11-19 21:32:04,763::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911331::DEBUG::2013-11-19 21:32:04,763::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911331::DEBUG::2013-11-19 21:32:04,767::task::495::TaskManager.Task::(_debug) Task 621a2743-8d68-40a1-9f95-b17c476f11a9: ref 0 aborting False
>Thread-15911331::INFO::2013-11-19 21:32:04,773::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911115::DEBUG::2013-11-19 21:32:04,867::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0948396 s, 10.8 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:32:05,918::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911115::DEBUG::2013-11-19 21:32:06,887::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:32:07,420::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0730991 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:32:09,436::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:32:09,987::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.076453 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:32:12,003::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:32:12,520::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0738053 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15911336::INFO::2013-11-19 21:32:14,299::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911336::DEBUG::2013-11-19 21:32:14,307::task::495::TaskManager.Task::(_debug) Task d59b038e-5e2c-4ad1-8214-b781deeb0c2d: moving from state init -> state preparing
>Thread-15911336::DEBUG::2013-11-19 21:32:14,310::task::495::TaskManager.Task::(_debug) Task d59b038e-5e2c-4ad1-8214-b781deeb0c2d: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000458955764771', 'lastCheck': 1384896731.6659329, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00158905982971', 'lastCheck': 1384896730.6680319, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896725.9189889, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00053596496582', 'lastCheck': 1384896726.51808, 'valid': True, 'code': 0}}
>Thread-15911336::DEBUG::2013-11-19 21:32:14,311::task::495::TaskManager.Task::(_debug) Task d59b038e-5e2c-4ad1-8214-b781deeb0c2d: moving from state preparing -> state finished
>Thread-15911336::DEBUG::2013-11-19 21:32:14,318::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911336::DEBUG::2013-11-19 21:32:14,320::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911336::DEBUG::2013-11-19 21:32:14,328::task::495::TaskManager.Task::(_debug) Task d59b038e-5e2c-4ad1-8214-b781deeb0c2d: ref 0 aborting False
>Thread-15911336::INFO::2013-11-19 21:32:14,332::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000458955764771', 'lastCheck': 1384896731.6659329, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00158905982971', 'lastCheck': 1384896730.6680319, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896725.9189889, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00053596496582', 'lastCheck': 1384896726.51808, 'valid': True, 'code': 0}}
>Dummy-15911115::DEBUG::2013-11-19 21:32:14,535::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15911338::DEBUG::2013-11-19 21:32:14,902::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911338::INFO::2013-11-19 21:32:14,904::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911338::DEBUG::2013-11-19 21:32:14,908::task::495::TaskManager.Task::(_debug) Task 26b709da-742a-4ba9-b6a0-1470bc4baa98: moving from state init -> state preparing
>Thread-15911338::DEBUG::2013-11-19 21:32:14,909::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=282 spmId=2
>Thread-15911338::DEBUG::2013-11-19 21:32:14,910::task::495::TaskManager.Task::(_debug) Task 26b709da-742a-4ba9-b6a0-1470bc4baa98: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911338::DEBUG::2013-11-19 21:32:14,911::task::495::TaskManager.Task::(_debug) Task 26b709da-742a-4ba9-b6a0-1470bc4baa98: moving from state preparing -> state finished
>Thread-15911338::DEBUG::2013-11-19 21:32:14,916::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911338::DEBUG::2013-11-19 21:32:14,916::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911338::DEBUG::2013-11-19 21:32:14,921::task::495::TaskManager.Task::(_debug) Task 26b709da-742a-4ba9-b6a0-1470bc4baa98: ref 0 aborting False
>Thread-15911338::INFO::2013-11-19 21:32:14,926::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911339::DEBUG::2013-11-19 21:32:14,934::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911339::INFO::2013-11-19 21:32:14,937::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911339::DEBUG::2013-11-19 21:32:14,943::task::495::TaskManager.Task::(_debug) Task 5a499037-7a78-4695-b67d-a333b122d336: moving from state init -> state preparing
>Thread-15911339::DEBUG::2013-11-19 21:32:14,947::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6c0dd485-0f9d-401f-a6a4-a4238345bad2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911339::DEBUG::2013-11-19 21:32:14,955::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911339::DEBUG::2013-11-19 21:32:14,962::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911339::DEBUG::2013-11-19 21:32:14,968::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6c0dd485-0f9d-401f-a6a4-a4238345bad2`::Granted request
>Thread-15911339::DEBUG::2013-11-19 21:32:14,970::task::495::TaskManager.Task::(_debug) Task 5a499037-7a78-4695-b67d-a333b122d336: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911339::DEBUG::2013-11-19 21:32:14,981::task::495::TaskManager.Task::(_debug) Task 5a499037-7a78-4695-b67d-a333b122d336: ref 1 aborting False
>Thread-15911339::WARNING::2013-11-19 21:32:14,986::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911339::DEBUG::2013-11-19 21:32:14,991::task::495::TaskManager.Task::(_debug) Task 5a499037-7a78-4695-b67d-a333b122d336: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911339::DEBUG::2013-11-19 21:32:14,999::task::495::TaskManager.Task::(_debug) Task 5a499037-7a78-4695-b67d-a333b122d336: moving from state preparing -> state finished
>Thread-15911339::DEBUG::2013-11-19 21:32:15,001::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911339::DEBUG::2013-11-19 21:32:15,002::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911339::DEBUG::2013-11-19 21:32:15,007::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911339::DEBUG::2013-11-19 21:32:15,012::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911339::DEBUG::2013-11-19 21:32:15,018::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911339::DEBUG::2013-11-19 21:32:15,022::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911339::DEBUG::2013-11-19 21:32:15,026::task::495::TaskManager.Task::(_debug) Task 5a499037-7a78-4695-b67d-a333b122d336: ref 0 aborting False
>Thread-15911339::INFO::2013-11-19 21:32:15,031::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911115::DEBUG::2013-11-19 21:32:15,092::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0893432 s, 11.5 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:32:16,214::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-19 21:32:16,757::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:32:17,113::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-42840::DEBUG::2013-11-19 21:32:17,466::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:32:17,659::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0736515 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:32:19,675::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:32:20,236::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.100983 s, 10.1 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:32:22,252::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:32:22,770::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0703223 s, 14.6 MB/s\n'; <rc> = 0
>Thread-15911344::INFO::2013-11-19 21:32:24,500::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911344::DEBUG::2013-11-19 21:32:24,500::task::495::TaskManager.Task::(_debug) Task 5f20bfb2-20e6-4ff3-9182-3e4f0964fc70: moving from state init -> state preparing
>Thread-15911344::DEBUG::2013-11-19 21:32:24,501::task::495::TaskManager.Task::(_debug) Task 5f20bfb2-20e6-4ff3-9182-3e4f0964fc70: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00041389465332', 'lastCheck': 1384896741.8957131, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0015389919281', 'lastCheck': 1384896740.896699, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896736.215724, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000415086746216', 'lastCheck': 1384896737.46929, 'valid': True, 'code': 0}}
>Thread-15911344::DEBUG::2013-11-19 21:32:24,511::task::495::TaskManager.Task::(_debug) Task 5f20bfb2-20e6-4ff3-9182-3e4f0964fc70: moving from state preparing -> state finished
>Thread-15911344::DEBUG::2013-11-19 21:32:24,513::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911344::DEBUG::2013-11-19 21:32:24,520::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911344::DEBUG::2013-11-19 21:32:24,522::task::495::TaskManager.Task::(_debug) Task 5f20bfb2-20e6-4ff3-9182-3e4f0964fc70: ref 0 aborting False
>Thread-15911344::INFO::2013-11-19 21:32:24,525::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00041389465332', 'lastCheck': 1384896741.8957131, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0015389919281', 'lastCheck': 1384896740.896699, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896736.215724, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000415086746216', 'lastCheck': 1384896737.46929, 'valid': True, 'code': 0}}
>Dummy-15911115::DEBUG::2013-11-19 21:32:24,786::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15911346::DEBUG::2013-11-19 21:32:25,168::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911346::INFO::2013-11-19 21:32:25,172::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911346::DEBUG::2013-11-19 21:32:25,173::task::495::TaskManager.Task::(_debug) Task e41c018f-9c08-4cb8-9f50-7835dc3edacc: moving from state init -> state preparing
>Thread-15911346::DEBUG::2013-11-19 21:32:25,179::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=282 spmId=2
>Thread-15911346::DEBUG::2013-11-19 21:32:25,184::task::495::TaskManager.Task::(_debug) Task e41c018f-9c08-4cb8-9f50-7835dc3edacc: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911346::DEBUG::2013-11-19 21:32:25,189::task::495::TaskManager.Task::(_debug) Task e41c018f-9c08-4cb8-9f50-7835dc3edacc: moving from state preparing -> state finished
>Thread-15911346::DEBUG::2013-11-19 21:32:25,191::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911346::DEBUG::2013-11-19 21:32:25,195::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911346::DEBUG::2013-11-19 21:32:25,197::task::495::TaskManager.Task::(_debug) Task e41c018f-9c08-4cb8-9f50-7835dc3edacc: ref 0 aborting False
>Thread-15911346::INFO::2013-11-19 21:32:25,200::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911347::DEBUG::2013-11-19 21:32:25,209::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911347::INFO::2013-11-19 21:32:25,216::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911347::DEBUG::2013-11-19 21:32:25,216::task::495::TaskManager.Task::(_debug) Task 4055a242-d0f7-4c8b-8032-c34c411e1274: moving from state init -> state preparing
>Thread-15911347::DEBUG::2013-11-19 21:32:25,223::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`72d39545-1747-4894-abe1-2aadba9be500`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911347::DEBUG::2013-11-19 21:32:25,225::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911347::DEBUG::2013-11-19 21:32:25,232::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911347::DEBUG::2013-11-19 21:32:25,235::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`72d39545-1747-4894-abe1-2aadba9be500`::Granted request
>Thread-15911347::DEBUG::2013-11-19 21:32:25,240::task::495::TaskManager.Task::(_debug) Task 4055a242-d0f7-4c8b-8032-c34c411e1274: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911347::DEBUG::2013-11-19 21:32:25,241::task::495::TaskManager.Task::(_debug) Task 4055a242-d0f7-4c8b-8032-c34c411e1274: ref 1 aborting False
>Thread-15911347::WARNING::2013-11-19 21:32:25,249::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911347::DEBUG::2013-11-19 21:32:25,256::task::495::TaskManager.Task::(_debug) Task 4055a242-d0f7-4c8b-8032-c34c411e1274: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911347::DEBUG::2013-11-19 21:32:25,259::task::495::TaskManager.Task::(_debug) Task 4055a242-d0f7-4c8b-8032-c34c411e1274: moving from state preparing -> state finished
>Thread-15911347::DEBUG::2013-11-19 21:32:25,264::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911347::DEBUG::2013-11-19 21:32:25,271::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911347::DEBUG::2013-11-19 21:32:25,274::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911347::DEBUG::2013-11-19 21:32:25,277::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911347::DEBUG::2013-11-19 21:32:25,282::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911347::DEBUG::2013-11-19 21:32:25,286::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911347::DEBUG::2013-11-19 21:32:25,287::task::495::TaskManager.Task::(_debug) Task 4055a242-d0f7-4c8b-8032-c34c411e1274: ref 0 aborting False
>Thread-15911347::INFO::2013-11-19 21:32:25,297::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911115::DEBUG::2013-11-19 21:32:25,349::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0938593 s, 10.9 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:32:26,458::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911115::DEBUG::2013-11-19 21:32:27,364::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:32:27,882::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0727621 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:32:29,897::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:32:30,435::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0733308 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:32:32,450::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:32:32,977::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0731734 s, 14.0 MB/s\n'; <rc> = 0
>Thread-15911352::INFO::2013-11-19 21:32:34,702::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911352::DEBUG::2013-11-19 21:32:34,707::task::495::TaskManager.Task::(_debug) Task ab65de2c-f0b9-44f2-89e8-9cf87fe014d3: moving from state init -> state preparing
>Thread-15911352::DEBUG::2013-11-19 21:32:34,713::task::495::TaskManager.Task::(_debug) Task ab65de2c-f0b9-44f2-89e8-9cf87fe014d3: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000406980514526', 'lastCheck': 1384896752.1252999, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0022599697113', 'lastCheck': 1384896751.129323, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896746.459507, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00099515914917', 'lastCheck': 1384896747.573915, 'valid': True, 'code': 0}}
>Thread-15911352::DEBUG::2013-11-19 21:32:34,714::task::495::TaskManager.Task::(_debug) Task ab65de2c-f0b9-44f2-89e8-9cf87fe014d3: moving from state preparing -> state finished
>Thread-15911352::DEBUG::2013-11-19 21:32:34,718::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911352::DEBUG::2013-11-19 21:32:34,720::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911352::DEBUG::2013-11-19 21:32:34,724::task::495::TaskManager.Task::(_debug) Task ab65de2c-f0b9-44f2-89e8-9cf87fe014d3: ref 0 aborting False
>Thread-15911352::INFO::2013-11-19 21:32:34,725::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000406980514526', 'lastCheck': 1384896752.1252999, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0022599697113', 'lastCheck': 1384896751.129323, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896746.459507, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00099515914917', 'lastCheck': 1384896747.573915, 'valid': True, 'code': 0}}
>Dummy-15911115::DEBUG::2013-11-19 21:32:34,992::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15911354::DEBUG::2013-11-19 21:32:35,415::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911354::INFO::2013-11-19 21:32:35,420::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911354::DEBUG::2013-11-19 21:32:35,421::task::495::TaskManager.Task::(_debug) Task eba7c324-dca9-4b0a-9c9f-36d040f680d5: moving from state init -> state preparing
>Thread-15911354::DEBUG::2013-11-19 21:32:35,429::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=282 spmId=2
>Thread-15911354::DEBUG::2013-11-19 21:32:35,435::task::495::TaskManager.Task::(_debug) Task eba7c324-dca9-4b0a-9c9f-36d040f680d5: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911354::DEBUG::2013-11-19 21:32:35,440::task::495::TaskManager.Task::(_debug) Task eba7c324-dca9-4b0a-9c9f-36d040f680d5: moving from state preparing -> state finished
>Thread-15911354::DEBUG::2013-11-19 21:32:35,450::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911354::DEBUG::2013-11-19 21:32:35,458::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911354::DEBUG::2013-11-19 21:32:35,463::task::495::TaskManager.Task::(_debug) Task eba7c324-dca9-4b0a-9c9f-36d040f680d5: ref 0 aborting False
>Thread-15911354::INFO::2013-11-19 21:32:35,463::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911355::DEBUG::2013-11-19 21:32:35,474::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911355::INFO::2013-11-19 21:32:35,484::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911355::DEBUG::2013-11-19 21:32:35,486::task::495::TaskManager.Task::(_debug) Task f89d0027-f096-40ff-b4d2-6b9baa3c0fd9: moving from state init -> state preparing
>Thread-15911355::DEBUG::2013-11-19 21:32:35,489::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d80abfec-8735-43dc-9775-6ea84bcf0b46`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911355::DEBUG::2013-11-19 21:32:35,490::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911355::DEBUG::2013-11-19 21:32:35,496::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911355::DEBUG::2013-11-19 21:32:35,500::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d80abfec-8735-43dc-9775-6ea84bcf0b46`::Granted request
>Thread-15911355::DEBUG::2013-11-19 21:32:35,503::task::495::TaskManager.Task::(_debug) Task f89d0027-f096-40ff-b4d2-6b9baa3c0fd9: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911355::DEBUG::2013-11-19 21:32:35,506::task::495::TaskManager.Task::(_debug) Task f89d0027-f096-40ff-b4d2-6b9baa3c0fd9: ref 1 aborting False
>Thread-15911355::WARNING::2013-11-19 21:32:35,513::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911355::DEBUG::2013-11-19 21:32:35,519::task::495::TaskManager.Task::(_debug) Task f89d0027-f096-40ff-b4d2-6b9baa3c0fd9: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911355::DEBUG::2013-11-19 21:32:35,525::task::495::TaskManager.Task::(_debug) Task f89d0027-f096-40ff-b4d2-6b9baa3c0fd9: moving from state preparing -> state finished
>Thread-15911355::DEBUG::2013-11-19 21:32:35,532::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911355::DEBUG::2013-11-19 21:32:35,533::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911355::DEBUG::2013-11-19 21:32:35,535::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911355::DEBUG::2013-11-19 21:32:35,539::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911355::DEBUG::2013-11-19 21:32:35,544::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Dummy-15911115::DEBUG::2013-11-19 21:32:35,545::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.087863 s, 11.7 MB/s\n'; <rc> = 0
>Thread-15911355::DEBUG::2013-11-19 21:32:35,545::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911355::DEBUG::2013-11-19 21:32:35,561::task::495::TaskManager.Task::(_debug) Task f89d0027-f096-40ff-b4d2-6b9baa3c0fd9: ref 0 aborting False
>Thread-15911355::INFO::2013-11-19 21:32:35,567::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-1024767::ERROR::2013-11-19 21:32:36,667::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911115::DEBUG::2013-11-19 21:32:37,563::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:32:38,101::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0751734 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:32:40,117::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:32:40,663::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0940199 s, 10.9 MB/s\n'; <rc> = 0
>Dummy-15911115::DEBUG::2013-11-19 21:32:42,678::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:32:43,225::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0746352 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15911360::INFO::2013-11-19 21:32:44,920::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911360::DEBUG::2013-11-19 21:32:44,926::task::495::TaskManager.Task::(_debug) Task ba1e2c87-f876-4d8d-b6f6-a0842c5b9cd2: moving from state init -> state preparing
>Thread-15911360::DEBUG::2013-11-19 21:32:44,929::task::495::TaskManager.Task::(_debug) Task ba1e2c87-f876-4d8d-b6f6-a0842c5b9cd2: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000515222549438', 'lastCheck': 1384896762.3370249, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00155186653137', 'lastCheck': 1384896761.345922, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896756.668355, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00338411331177', 'lastCheck': 1384896757.789011, 'valid': True, 'code': 0}}
>Thread-15911360::DEBUG::2013-11-19 21:32:44,929::task::495::TaskManager.Task::(_debug) Task ba1e2c87-f876-4d8d-b6f6-a0842c5b9cd2: moving from state preparing -> state finished
>Thread-15911360::DEBUG::2013-11-19 21:32:44,930::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911360::DEBUG::2013-11-19 21:32:44,931::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911360::DEBUG::2013-11-19 21:32:44,934::task::495::TaskManager.Task::(_debug) Task ba1e2c87-f876-4d8d-b6f6-a0842c5b9cd2: ref 0 aborting False
>Thread-15911360::INFO::2013-11-19 21:32:44,943::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000515222549438', 'lastCheck': 1384896762.3370249, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00155186653137', 'lastCheck': 1384896761.345922, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896756.668355, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00338411331177', 'lastCheck': 1384896757.789011, 'valid': True, 'code': 0}}
>Dummy-15911115::DEBUG::2013-11-19 21:32:45,240::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15911362::DEBUG::2013-11-19 21:32:45,690::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911362::INFO::2013-11-19 21:32:45,698::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911362::DEBUG::2013-11-19 21:32:45,699::task::495::TaskManager.Task::(_debug) Task 16eef668-0538-409a-ad13-ab28123776e8: moving from state init -> state preparing
>Thread-15911362::DEBUG::2013-11-19 21:32:45,706::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=282 spmId=2
>Thread-15911362::DEBUG::2013-11-19 21:32:45,709::task::495::TaskManager.Task::(_debug) Task 16eef668-0538-409a-ad13-ab28123776e8: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911362::DEBUG::2013-11-19 21:32:45,715::task::495::TaskManager.Task::(_debug) Task 16eef668-0538-409a-ad13-ab28123776e8: moving from state preparing -> state finished
>Thread-15911362::DEBUG::2013-11-19 21:32:45,726::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911362::DEBUG::2013-11-19 21:32:45,732::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911362::DEBUG::2013-11-19 21:32:45,733::task::495::TaskManager.Task::(_debug) Task 16eef668-0538-409a-ad13-ab28123776e8: ref 0 aborting False
>Thread-15911362::INFO::2013-11-19 21:32:45,737::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911363::DEBUG::2013-11-19 21:32:45,755::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911363::INFO::2013-11-19 21:32:45,760::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911363::DEBUG::2013-11-19 21:32:45,761::task::495::TaskManager.Task::(_debug) Task 591dd447-ba13-4da3-83fc-f5d24aae10e5: moving from state init -> state preparing
>Thread-15911363::DEBUG::2013-11-19 21:32:45,764::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6e2f4d37-9d34-4c61-a005-e9f77ef402eb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911363::DEBUG::2013-11-19 21:32:45,765::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911363::DEBUG::2013-11-19 21:32:45,771::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911363::DEBUG::2013-11-19 21:32:45,772::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6e2f4d37-9d34-4c61-a005-e9f77ef402eb`::Granted request
>Thread-15911363::DEBUG::2013-11-19 21:32:45,774::task::495::TaskManager.Task::(_debug) Task 591dd447-ba13-4da3-83fc-f5d24aae10e5: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911363::DEBUG::2013-11-19 21:32:45,780::task::495::TaskManager.Task::(_debug) Task 591dd447-ba13-4da3-83fc-f5d24aae10e5: ref 1 aborting False
>Dummy-15911115::DEBUG::2013-11-19 21:32:45,786::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0804732 s, 12.7 MB/s\n'; <rc> = 0
>Thread-15911363::WARNING::2013-11-19 21:32:45,789::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911363::DEBUG::2013-11-19 21:32:45,804::task::495::TaskManager.Task::(_debug) Task 591dd447-ba13-4da3-83fc-f5d24aae10e5: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911363::DEBUG::2013-11-19 21:32:45,810::task::495::TaskManager.Task::(_debug) Task 591dd447-ba13-4da3-83fc-f5d24aae10e5: moving from state preparing -> state finished
>Thread-15911363::DEBUG::2013-11-19 21:32:45,813::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911363::DEBUG::2013-11-19 21:32:45,816::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911363::DEBUG::2013-11-19 21:32:45,816::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911363::DEBUG::2013-11-19 21:32:45,820::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911363::DEBUG::2013-11-19 21:32:45,824::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911363::DEBUG::2013-11-19 21:32:45,829::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911363::DEBUG::2013-11-19 21:32:45,832::task::495::TaskManager.Task::(_debug) Task 591dd447-ba13-4da3-83fc-f5d24aae10e5: ref 0 aborting False
>Thread-15911363::INFO::2013-11-19 21:32:45,837::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 282}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-1024767::ERROR::2013-11-19 21:32:46,912::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911115::DEBUG::2013-11-19 21:32:47,806::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15911365::DEBUG::2013-11-19 21:32:48,240::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911365::INFO::2013-11-19 21:32:48,247::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksStatuses, args: ()
>Thread-15911365::DEBUG::2013-11-19 21:32:48,250::task::495::TaskManager.Task::(_debug) Task eaf94ff5-229a-4e8e-a5b0-f3f0eeba6293: moving from state init -> state preparing
>Thread-15911365::DEBUG::2013-11-19 21:32:48,255::taskManager::92::TaskManager::(getAllTasksStatuses) Entry.
>Thread-15911365::DEBUG::2013-11-19 21:32:48,258::taskManager::97::TaskManager::(getAllTasksStatuses) Return: {}
>Thread-15911365::DEBUG::2013-11-19 21:32:48,261::task::495::TaskManager.Task::(_debug) Task eaf94ff5-229a-4e8e-a5b0-f3f0eeba6293: finished: {'allTasksStatus': {}}
>Thread-15911365::DEBUG::2013-11-19 21:32:48,265::task::495::TaskManager.Task::(_debug) Task eaf94ff5-229a-4e8e-a5b0-f3f0eeba6293: moving from state preparing -> state finished
>Thread-15911365::DEBUG::2013-11-19 21:32:48,266::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911365::DEBUG::2013-11-19 21:32:48,269::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911365::DEBUG::2013-11-19 21:32:48,277::task::495::TaskManager.Task::(_debug) Task eaf94ff5-229a-4e8e-a5b0-f3f0eeba6293: ref 0 aborting False
>Thread-15911365::INFO::2013-11-19 21:32:48,282::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksStatuses, Return response: {'status': {'message': 'OK', 'code': 0}, 'allTasksStatus': {}}
>Thread-15911366::DEBUG::2013-11-19 21:32:48,325::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911366::INFO::2013-11-19 21:32:48,334::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: deactivateStorageDomain, args: ( sdUUID=5d7b5178-3790-472c-a13b-b481eb27a61e spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 msdUUID=00000000-0000-0000-0000-000000000000 masterVersion=1)
>Thread-15911366::DEBUG::2013-11-19 21:32:48,341::task::495::TaskManager.Task::(_debug) Task 03c2cd07-d237-43f3-9bb6-2337dd99ef48: moving from state init -> state preparing
>Thread-15911366::DEBUG::2013-11-19 21:32:48,344::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a6176029-37b3-4249-9978-33dabf666afc`::Request was made in '/usr/share/vdsm/storage/spm.py' line '1112' at 'public_deactivateStorageDomain'
>Thread-15911366::DEBUG::2013-11-19 21:32:48,348::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'exclusive'
>Thread-15911366::DEBUG::2013-11-19 21:32:48,354::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'exclusive' (1 active user)
>Thread-15911366::DEBUG::2013-11-19 21:32:48,360::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a6176029-37b3-4249-9978-33dabf666afc`::Granted request
>Thread-15911366::DEBUG::2013-11-19 21:32:48,386::task::495::TaskManager.Task::(_debug) Task 03c2cd07-d237-43f3-9bb6-2337dd99ef48: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (exclusive)
>Thread-15911366::DEBUG::2013-11-19 21:32:48,391::task::495::TaskManager.Task::(_debug) Task 03c2cd07-d237-43f3-9bb6-2337dd99ef48: ref 1 aborting False
>Thread-15911366::DEBUG::2013-11-19 21:32:48,396::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.5d7b5178-3790-472c-a13b-b481eb27a61e`ReqID=`1188c516-4f4b-42af-bb27-4e0902c22262`::Request was made in '/usr/share/vdsm/storage/spm.py' line '1113' at 'public_deactivateStorageDomain'
>Thread-15911366::DEBUG::2013-11-19 21:32:48,401::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e' for lock type 'exclusive'
>Thread-15911366::DEBUG::2013-11-19 21:32:48,404::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e' is free. Now locking as 'exclusive' (1 active user)
>Dummy-15911115::DEBUG::2013-11-19 21:32:48,406::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0890451 s, 11.5 MB/s\n'; <rc> = 0
>Thread-15911366::DEBUG::2013-11-19 21:32:48,409::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.5d7b5178-3790-472c-a13b-b481eb27a61e`ReqID=`1188c516-4f4b-42af-bb27-4e0902c22262`::Granted request
>Thread-15911366::DEBUG::2013-11-19 21:32:48,426::task::495::TaskManager.Task::(_debug) Task 03c2cd07-d237-43f3-9bb6-2337dd99ef48: _resourcesAcquired: Storage.5d7b5178-3790-472c-a13b-b481eb27a61e (exclusive)
>Thread-15911366::DEBUG::2013-11-19 21:32:48,431::task::495::TaskManager.Task::(_debug) Task 03c2cd07-d237-43f3-9bb6-2337dd99ef48: ref 1 aborting False
>Thread-15911366::INFO::2013-11-19 21:32:48,437::sp::775::Storage.StoragePool::(deactivateSD) sdUUID=5d7b5178-3790-472c-a13b-b481eb27a61e spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 new_msdUUID=00000000-0000-0000-0000-000000000000
>Thread-15911366::INFO::2013-11-19 21:32:48,443::fileSD::176::Storage.StorageDomain::(validate) sdUUID=5d7b5178-3790-472c-a13b-b481eb27a61e
>Thread-15911366::DEBUG::2013-11-19 21:32:48,452::persistentDict::204::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=[]
>Thread-15911366::WARNING::2013-11-19 21:32:48,455::persistentDict::226::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is
>Thread-15911366::ERROR::2013-11-19 21:32:48,458::task::868::TaskManager.Task::(_setError) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/task.py", line 876, in _run
>  File "/usr/share/vdsm/storage/spm.py", line 115, in run
>  File "/usr/share/vdsm/storage/spm.py", line 1115, in public_deactivateStorageDomain
>  File "/usr/share/vdsm/storage/sp.py", line 798, in deactivateSD
>  File "/usr/share/vdsm/storage/sd.py", line 666, in isBackup
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911366::DEBUG::2013-11-19 21:32:48,466::task::495::TaskManager.Task::(_debug) Task 03c2cd07-d237-43f3-9bb6-2337dd99ef48: Task._run: 03c2cd07-d237-43f3-9bb6-2337dd99ef48 ('5d7b5178-3790-472c-a13b-b481eb27a61e', 'c4e5889a-348d-11e2-83fe-9f72369eab30', '00000000-0000-0000-0000-000000000000', 1) {} failed - stopping task
>Thread-15911366::DEBUG::2013-11-19 21:32:48,469::task::495::TaskManager.Task::(_debug) Task 03c2cd07-d237-43f3-9bb6-2337dd99ef48: stopping in state preparing (force False)
>Thread-15911366::DEBUG::2013-11-19 21:32:48,474::task::495::TaskManager.Task::(_debug) Task 03c2cd07-d237-43f3-9bb6-2337dd99ef48: ref 1 aborting True
>Thread-15911366::INFO::2013-11-19 21:32:48,477::task::1171::TaskManager.Task::(prepare) aborting: Task is aborted: "'CLASS'" - code 100
>Thread-15911366::DEBUG::2013-11-19 21:32:48,483::task::495::TaskManager.Task::(_debug) Task 03c2cd07-d237-43f3-9bb6-2337dd99ef48: Prepare: aborted: 'CLASS'
>Thread-15911366::DEBUG::2013-11-19 21:32:48,484::task::495::TaskManager.Task::(_debug) Task 03c2cd07-d237-43f3-9bb6-2337dd99ef48: ref 0 aborting True
>Thread-15911366::DEBUG::2013-11-19 21:32:48,486::task::495::TaskManager.Task::(_debug) Task 03c2cd07-d237-43f3-9bb6-2337dd99ef48: Task._doAbort: force False
>Thread-15911366::DEBUG::2013-11-19 21:32:48,486::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911366::DEBUG::2013-11-19 21:32:48,490::task::495::TaskManager.Task::(_debug) Task 03c2cd07-d237-43f3-9bb6-2337dd99ef48: moving from state preparing -> state aborting
>Thread-15911366::DEBUG::2013-11-19 21:32:48,492::task::495::TaskManager.Task::(_debug) Task 03c2cd07-d237-43f3-9bb6-2337dd99ef48: _aborting: recover policy none
>Thread-15911366::DEBUG::2013-11-19 21:32:48,497::task::495::TaskManager.Task::(_debug) Task 03c2cd07-d237-43f3-9bb6-2337dd99ef48: moving from state aborting -> state failed
>Thread-15911366::DEBUG::2013-11-19 21:32:48,502::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>, 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e': < ResourceRef 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e', isValid: 'True' obj: 'None'>}
>Thread-15911366::DEBUG::2013-11-19 21:32:48,510::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911366::DEBUG::2013-11-19 21:32:48,516::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911366::DEBUG::2013-11-19 21:32:48,522::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911366::DEBUG::2013-11-19 21:32:48,526::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911366::DEBUG::2013-11-19 21:32:48,530::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911366::DEBUG::2013-11-19 21:32:48,533::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e'
>Thread-15911366::DEBUG::2013-11-19 21:32:48,533::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e' (0 active users)
>Thread-15911366::DEBUG::2013-11-19 21:32:48,538::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e' is free, finding out if anyone is waiting for it.
>Thread-15911366::DEBUG::2013-11-19 21:32:48,542::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e', Clearing records.
>Thread-15911366::ERROR::2013-11-19 21:32:48,551::dispatcher::106::Storage.Dispatcher.Protect::(run) 'CLASS'
>Thread-15911366::ERROR::2013-11-19 21:32:48,552::dispatcher::107::Storage.Dispatcher.Protect::(run) Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/dispatcher.py", line 96, in run
>  File "/usr/share/vdsm/storage/task.py", line 1178, in prepare
>KeyError: 'CLASS'
>
>Thread-15911367::DEBUG::2013-11-19 21:32:48,566::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911367::INFO::2013-11-19 21:32:48,568::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911367::DEBUG::2013-11-19 21:32:48,569::task::495::TaskManager.Task::(_debug) Task 03e8fcfb-b639-43c7-8a15-a3301ae789e7: moving from state init -> state preparing
>Thread-15911367::DEBUG::2013-11-19 21:32:48,570::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=282 spmId=2
>Thread-15911367::DEBUG::2013-11-19 21:32:48,573::task::495::TaskManager.Task::(_debug) Task 03e8fcfb-b639-43c7-8a15-a3301ae789e7: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911367::DEBUG::2013-11-19 21:32:48,580::task::495::TaskManager.Task::(_debug) Task 03e8fcfb-b639-43c7-8a15-a3301ae789e7: moving from state preparing -> state finished
>Thread-15911367::DEBUG::2013-11-19 21:32:48,584::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911367::DEBUG::2013-11-19 21:32:48,589::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911367::DEBUG::2013-11-19 21:32:48,594::task::495::TaskManager.Task::(_debug) Task 03e8fcfb-b639-43c7-8a15-a3301ae789e7: ref 0 aborting False
>Thread-15911367::INFO::2013-11-19 21:32:48,596::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 282}}
>Thread-15911368::DEBUG::2013-11-19 21:32:48,611::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911368::INFO::2013-11-19 21:32:48,612::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksStatuses, args: ()
>Thread-15911368::DEBUG::2013-11-19 21:32:48,616::task::495::TaskManager.Task::(_debug) Task f9e3cf38-af8c-4cfa-b45d-0b592e8eb94c: moving from state init -> state preparing
>Thread-15911368::DEBUG::2013-11-19 21:32:48,620::taskManager::92::TaskManager::(getAllTasksStatuses) Entry.
>Thread-15911368::DEBUG::2013-11-19 21:32:48,627::taskManager::97::TaskManager::(getAllTasksStatuses) Return: {}
>Thread-15911368::DEBUG::2013-11-19 21:32:48,628::task::495::TaskManager.Task::(_debug) Task f9e3cf38-af8c-4cfa-b45d-0b592e8eb94c: finished: {'allTasksStatus': {}}
>Thread-15911368::DEBUG::2013-11-19 21:32:48,632::task::495::TaskManager.Task::(_debug) Task f9e3cf38-af8c-4cfa-b45d-0b592e8eb94c: moving from state preparing -> state finished
>Thread-15911368::DEBUG::2013-11-19 21:32:48,637::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911368::DEBUG::2013-11-19 21:32:48,643::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911368::DEBUG::2013-11-19 21:32:48,652::task::495::TaskManager.Task::(_debug) Task f9e3cf38-af8c-4cfa-b45d-0b592e8eb94c: ref 0 aborting False
>Thread-15911368::INFO::2013-11-19 21:32:48,652::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksStatuses, Return response: {'status': {'message': 'OK', 'code': 0}, 'allTasksStatus': {}}
>Thread-15911369::DEBUG::2013-11-19 21:32:48,666::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911369::INFO::2013-11-19 21:32:48,672::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: spmStop, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911369::DEBUG::2013-11-19 21:32:48,677::task::495::TaskManager.Task::(_debug) Task fa928307-b43e-4461-a204-968268ffd7bc: moving from state init -> state preparing
>Thread-15911369::DEBUG::2013-11-19 21:32:48,681::spm::547::Storage.SPM::(public_spmStop) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30
>Thread-15911369::DEBUG::2013-11-19 21:32:48,686::taskManager::105::TaskManager::(getAllTasks) Entry.
>Thread-15911369::DEBUG::2013-11-19 21:32:48,686::taskManager::110::TaskManager::(getAllTasks) Return: {}
>Thread-15911369::DEBUG::2013-11-19 21:32:48,693::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`14cf12b2-c543-4f20-864d-94e57c02488f`::Request was made in '/usr/share/vdsm/storage/spm.py' line '573' at 'public_spmStop'
>Thread-15911369::DEBUG::2013-11-19 21:32:48,702::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'exclusive'
>Thread-15911369::DEBUG::2013-11-19 21:32:48,705::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'exclusive' (1 active user)
>Thread-15911369::DEBUG::2013-11-19 21:32:48,705::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`14cf12b2-c543-4f20-864d-94e57c02488f`::Granted request
>Thread-15911369::DEBUG::2013-11-19 21:32:48,709::task::495::TaskManager.Task::(_debug) Task fa928307-b43e-4461-a204-968268ffd7bc: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (exclusive)
>Thread-15911369::DEBUG::2013-11-19 21:32:48,713::task::495::TaskManager.Task::(_debug) Task fa928307-b43e-4461-a204-968268ffd7bc: ref 1 aborting False
>Thread-15911369::DEBUG::2013-11-19 21:32:48,717::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`354ddf24-151c-458e-9af8-0698e5e152c2`::Request was made in '/usr/share/vdsm/storage/spm.py' line '584' at '_stop'
>Thread-15911369::DEBUG::2013-11-19 21:32:48,717::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'exclusive'
>Thread-15911369::DEBUG::2013-11-19 21:32:48,722::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'exclusive' (1 active user)
>Thread-15911369::DEBUG::2013-11-19 21:32:48,727::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`354ddf24-151c-458e-9af8-0698e5e152c2`::Granted request
>Thread-15911369::DEBUG::2013-11-19 21:32:48,730::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e`ReqID=`270247ba-3217-4bcb-a230-cbb83730fba8`::Request was made in '/usr/share/vdsm/storage/spm.py' line '601' at '_stop'
>Thread-15911369::DEBUG::2013-11-19 21:32:48,735::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e' for lock type 'exclusive'
>Thread-15911369::DEBUG::2013-11-19 21:32:48,741::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e' is free. Now locking as 'exclusive' (1 active user)
>Thread-15911369::DEBUG::2013-11-19 21:32:48,747::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e`ReqID=`270247ba-3217-4bcb-a230-cbb83730fba8`::Granted request
>Thread-15911369::DEBUG::2013-11-19 21:32:48,755::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e'
>Thread-15911369::DEBUG::2013-11-19 21:32:48,758::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e' (0 active users)
>Thread-15911369::DEBUG::2013-11-19 21:32:48,764::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e' is free, finding out if anyone is waiting for it.
>Thread-15911369::DEBUG::2013-11-19 21:32:48,765::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e', Clearing records.
>Thread-15911369::DEBUG::2013-11-19 21:32:48,777::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911369::DEBUG::2013-11-19 21:32:48,780::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911369::DEBUG::2013-11-19 21:32:48,784::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911369::DEBUG::2013-11-19 21:32:48,789::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911369::DEBUG::2013-11-19 21:32:48,793::taskManager::118::TaskManager::(unloadTasks) Entry.
>Thread-15911369::DEBUG::2013-11-19 21:32:48,796::taskManager::122::TaskManager::(unloadTasks) Return
>Thread-15911369::DEBUG::2013-11-19 21:32:48,810::spm::195::Storage.SPM::(__cleanupMasterMount) unmounting /rhev/data-center/mnt/blockSD/6b892067-ac7e-4039-a7be-5b7387525b16/master
>Dummy-15911115::DEBUG::2013-11-19 21:32:50,431::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15911369::DEBUG::2013-11-19 21:32:51,412::fileUtils::112::Storage.Misc.excCmd::(umount) '/usr/bin/sudo -n /bin/umount -f /rhev/data-center/mnt/blockSD/6b892067-ac7e-4039-a7be-5b7387525b16/master' (cwd None)
>Dummy-15911115::DEBUG::2013-11-19 21:32:51,759::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.846124 s, 1.2 MB/s\n'; <rc> = 0
>Thread-15911369::DEBUG::2013-11-19 21:32:51,853::fileUtils::112::Storage.Misc.excCmd::(umount) SUCCESS: <err> = ''; <rc> = 0
>Thread-15911369::DEBUG::2013-11-19 21:32:51,870::spm::290::Storage.SPM::(__cleanupSPM) cleaning up SPM: c4e5889a-348d-11e2-83fe-9f72369eab30
>Thread-15911369::DEBUG::2013-11-19 21:32:51,875::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
>Thread-15911369::DEBUG::2013-11-19 21:32:51,876::persistentDict::146::Storage.PersistentDict::(transaction) Flushing changes
>Thread-15911369::DEBUG::2013-11-19 21:32:51,885::persistentDict::265::Storage.PersistentDict::(flush) about to write lines (VGTagMetadataRW)=[u'CLASS=Data', u'DESCRIPTION=OS', u'IOOPTIMEOUTSEC=10', u'LEASERETRIES=3', u'LEASETIMESEC=60', u'LOCKPOLICY=', u'LOCKRENEWALINTERVALSEC=5', u'LOGBLKSIZE=512', u'MASTER_VERSION=1', u'PHYBLKSIZE=512', u'POOL_DESCRIPTION=Default', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_ID=-1', u'POOL_SPM_LVER=282', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'ROLE=Master', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'TYPE=FCP', u'VERSION=2', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', '_SHA_CKSUM=f2989d9b2e8ee39a10c4cf9b6999adb068743a1b']
>Thread-15911369::DEBUG::2013-11-19 21:32:51,892::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15911369::DEBUG::2013-11-19 21:32:51,893::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15911369::DEBUG::2013-11-19 21:32:51,898::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15911369::DEBUG::2013-11-19 21:32:51,903::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15911369::DEBUG::2013-11-19 21:32:51,909::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-15911369::DEBUG::2013-11-19 21:32:51,915::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15911369::DEBUG::2013-11-19 21:32:52,565::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15911369::DEBUG::2013-11-19 21:32:52,574::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-15911369::DEBUG::2013-11-19 21:32:52,584::blockSD::145::storage.Metadata.VGTagMetadataRW::(writelines) Updating metadata adding=MDT_POOL_SPM_ID=-1, MDT__SHA_CKSUM=f2989d9b2e8ee39a10c4cf9b6999adb068743a1b removing=MDT__SHA_CKSUM=de56f40c15838d78cba198e8c045a60fe83f87c3, MDT_POOL_SPM_ID=2
>Thread-15911369::DEBUG::2013-11-19 21:32:52,585::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --deltag MDT__SHA_CKSUM=de56f40c15838d78cba198e8c045a60fe83f87c3 --deltag MDT_POOL_SPM_ID=2 --addtag MDT_POOL_SPM_ID=-1 --addtag MDT__SHA_CKSUM=f2989d9b2e8ee39a10c4cf9b6999adb068743a1b 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15911369::DEBUG::2013-11-19 21:32:53,267::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15911369::DEBUG::2013-11-19 21:32:53,272::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15911369::DEBUG::2013-11-19 21:32:53,278::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15911369::DEBUG::2013-11-19 21:32:53,281::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction
>Thread-15911369::INFO::2013-11-19 21:32:53,282::safelease::83::ClusterLock::(release) Releasing cluster lock for domain 6b892067-ac7e-4039-a7be-5b7387525b16
>Thread-15911369::DEBUG::2013-11-19 21:32:53,287::safelease::84::Storage.Misc.excCmd::(release) '/usr/libexec/vdsm/spmstop.sh 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd /usr/libexec/vdsm/)
>Dummy-15911115::INFO::2013-11-19 21:32:53,774::storage_mailbox::673::Storage.MailBox.SpmMailMonitor::(run) SPM_MailMonitor - Incoming mail monitoring thread stopped
>Thread-15911369::DEBUG::2013-11-19 21:32:54,781::safelease::84::Storage.Misc.excCmd::(release) SUCCESS: <err> = ''; <rc> = 0
>Thread-15911369::DEBUG::2013-11-19 21:32:54,794::safelease::87::ClusterLock::(release) Cluster lock released successfully
>Thread-15911369::DEBUG::2013-11-19 21:32:54,795::task::495::TaskManager.Task::(_debug) Task fa928307-b43e-4461-a204-968268ffd7bc: finished: None
>Thread-15911369::DEBUG::2013-11-19 21:32:54,797::task::495::TaskManager.Task::(_debug) Task fa928307-b43e-4461-a204-968268ffd7bc: moving from state preparing -> state finished
>Thread-15911369::DEBUG::2013-11-19 21:32:54,798::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911369::DEBUG::2013-11-19 21:32:54,802::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911369::DEBUG::2013-11-19 21:32:54,803::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911369::DEBUG::2013-11-19 21:32:54,809::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911369::DEBUG::2013-11-19 21:32:54,814::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911369::DEBUG::2013-11-19 21:32:54,817::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911369::DEBUG::2013-11-19 21:32:54,822::task::495::TaskManager.Task::(_debug) Task fa928307-b43e-4461-a204-968268ffd7bc: ref 0 aborting False
>Thread-15911369::INFO::2013-11-19 21:32:54,828::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: spmStop, Return response: {'status': {'message': 'OK', 'code': 0}}
>Thread-15911373::INFO::2013-11-19 21:32:55,167::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911373::DEBUG::2013-11-19 21:32:55,171::task::495::TaskManager.Task::(_debug) Task 342b82f4-0839-4a93-9ef9-88a6574ebf7f: moving from state init -> state preparing
>Thread-15911373::DEBUG::2013-11-19 21:32:55,172::task::495::TaskManager.Task::(_debug) Task 342b82f4-0839-4a93-9ef9-88a6574ebf7f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000354051589966', 'lastCheck': 1384896773.285223, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00261998176575', 'lastCheck': 1384896771.677285, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896766.9137299, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00110697746277', 'lastCheck': 1384896768.029747, 'valid': True, 'code': 0}}
>Thread-15911373::DEBUG::2013-11-19 21:32:55,177::task::495::TaskManager.Task::(_debug) Task 342b82f4-0839-4a93-9ef9-88a6574ebf7f: moving from state preparing -> state finished
>Thread-15911373::DEBUG::2013-11-19 21:32:55,178::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911373::DEBUG::2013-11-19 21:32:55,179::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911373::DEBUG::2013-11-19 21:32:55,179::task::495::TaskManager.Task::(_debug) Task 342b82f4-0839-4a93-9ef9-88a6574ebf7f: ref 0 aborting False
>Thread-15911373::INFO::2013-11-19 21:32:55,186::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000354051589966', 'lastCheck': 1384896773.285223, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00261998176575', 'lastCheck': 1384896771.677285, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896766.9137299, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00110697746277', 'lastCheck': 1384896768.029747, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:32:57,151::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-19 21:33:03,296::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-42838::DEBUG::2013-11-19 21:33:03,297::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 21:33:04,010::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-42838::DEBUG::2013-11-19 21:33:04,014::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-15911379::INFO::2013-11-19 21:33:05,530::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911379::DEBUG::2013-11-19 21:33:05,541::task::495::TaskManager.Task::(_debug) Task 51ab0f8b-4f17-458c-9255-dce94fc44939: moving from state init -> state preparing
>Thread-15911379::DEBUG::2013-11-19 21:33:05,546::task::495::TaskManager.Task::(_debug) Task 51ab0f8b-4f17-458c-9255-dce94fc44939: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000343084335327', 'lastCheck': 1384896784.017015, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00156593322754', 'lastCheck': 1384896781.816519, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896777.1525199, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000456809997559', 'lastCheck': 1384896778.1475229, 'valid': True, 'code': 0}}
>Thread-15911379::DEBUG::2013-11-19 21:33:05,551::task::495::TaskManager.Task::(_debug) Task 51ab0f8b-4f17-458c-9255-dce94fc44939: moving from state preparing -> state finished
>Thread-15911379::DEBUG::2013-11-19 21:33:05,553::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911379::DEBUG::2013-11-19 21:33:05,555::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911379::DEBUG::2013-11-19 21:33:05,560::task::495::TaskManager.Task::(_debug) Task 51ab0f8b-4f17-458c-9255-dce94fc44939: ref 0 aborting False
>Thread-15911379::INFO::2013-11-19 21:33:05,563::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000343084335327', 'lastCheck': 1384896784.017015, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00156593322754', 'lastCheck': 1384896781.816519, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896777.1525199, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000456809997559', 'lastCheck': 1384896778.1475229, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:33:07,165::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911385::INFO::2013-11-19 21:33:15,710::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911385::DEBUG::2013-11-19 21:33:15,716::task::495::TaskManager.Task::(_debug) Task 1275dd79-bad0-435e-ac2e-344d1511461a: moving from state init -> state preparing
>Thread-15911385::DEBUG::2013-11-19 21:33:15,719::task::495::TaskManager.Task::(_debug) Task 1275dd79-bad0-435e-ac2e-344d1511461a: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000350952148438', 'lastCheck': 1384896794.0297301, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00152111053467', 'lastCheck': 1384896791.832377, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896787.165859, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000514030456543', 'lastCheck': 1384896788.1593089, 'valid': True, 'code': 0}}
>Thread-15911385::DEBUG::2013-11-19 21:33:15,723::task::495::TaskManager.Task::(_debug) Task 1275dd79-bad0-435e-ac2e-344d1511461a: moving from state preparing -> state finished
>Thread-15911385::DEBUG::2013-11-19 21:33:15,726::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911385::DEBUG::2013-11-19 21:33:15,732::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911385::DEBUG::2013-11-19 21:33:15,732::task::495::TaskManager.Task::(_debug) Task 1275dd79-bad0-435e-ac2e-344d1511461a: ref 0 aborting False
>Thread-15911385::INFO::2013-11-19 21:33:15,733::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000350952148438', 'lastCheck': 1384896794.0297301, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00152111053467', 'lastCheck': 1384896791.832377, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896787.165859, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000514030456543', 'lastCheck': 1384896788.1593089, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:33:17,178::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911391::INFO::2013-11-19 21:33:25,902::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911391::DEBUG::2013-11-19 21:33:25,903::task::495::TaskManager.Task::(_debug) Task b0cccb68-1763-4cc3-91de-c638d6b6e87c: moving from state init -> state preparing
>Thread-15911391::DEBUG::2013-11-19 21:33:25,914::task::495::TaskManager.Task::(_debug) Task b0cccb68-1763-4cc3-91de-c638d6b6e87c: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000314950942993', 'lastCheck': 1384896804.042403, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00152802467346', 'lastCheck': 1384896801.848088, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896797.1790421, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000376224517822', 'lastCheck': 1384896798.170836, 'valid': True, 'code': 0}}
>Thread-15911391::DEBUG::2013-11-19 21:33:25,917::task::495::TaskManager.Task::(_debug) Task b0cccb68-1763-4cc3-91de-c638d6b6e87c: moving from state preparing -> state finished
>Thread-15911391::DEBUG::2013-11-19 21:33:25,921::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911391::DEBUG::2013-11-19 21:33:25,921::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911391::DEBUG::2013-11-19 21:33:25,924::task::495::TaskManager.Task::(_debug) Task b0cccb68-1763-4cc3-91de-c638d6b6e87c: ref 0 aborting False
>Thread-15911391::INFO::2013-11-19 21:33:25,925::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000314950942993', 'lastCheck': 1384896804.042403, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00152802467346', 'lastCheck': 1384896801.848088, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896797.1790421, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000376224517822', 'lastCheck': 1384896798.170836, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:33:27,191::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911395::DEBUG::2013-11-19 21:33:30,845::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911395::INFO::2013-11-19 21:33:30,851::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911395::DEBUG::2013-11-19 21:33:30,852::task::495::TaskManager.Task::(_debug) Task 2d064909-e822-488a-94cf-29edada0c506: moving from state init -> state preparing
>Thread-15911395::DEBUG::2013-11-19 21:33:30,856::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=Free spmLver=282 spmId=-1
>Thread-15911395::DEBUG::2013-11-19 21:33:30,857::task::495::TaskManager.Task::(_debug) Task 2d064909-e822-488a-94cf-29edada0c506: finished: {'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': 282}}
>Thread-15911395::DEBUG::2013-11-19 21:33:30,859::task::495::TaskManager.Task::(_debug) Task 2d064909-e822-488a-94cf-29edada0c506: moving from state preparing -> state finished
>Thread-15911395::DEBUG::2013-11-19 21:33:30,859::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911395::DEBUG::2013-11-19 21:33:30,860::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911395::DEBUG::2013-11-19 21:33:30,860::task::495::TaskManager.Task::(_debug) Task 2d064909-e822-488a-94cf-29edada0c506: ref 0 aborting False
>Thread-15911395::INFO::2013-11-19 21:33:30,861::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': 282}}
>Thread-15911396::DEBUG::2013-11-19 21:33:30,889::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911396::INFO::2013-11-19 21:33:30,897::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: spmStart, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 prevID=-1 prevLVER=282 recoveryMode=0 scsiFencing=false maxHostID=250 domVersion=2)
>Thread-15911396::DEBUG::2013-11-19 21:33:30,900::task::495::TaskManager.Task::(_debug) Task d3f111cf-7b31-41dc-bcdf-c918a6697f62: moving from state init -> state preparing
>Thread-15911396::DEBUG::2013-11-19 21:33:30,905::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`471856ca-84e1-48c9-a828-58db130cc847`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '330' at 'public_spmStart'
>Thread-15911396::DEBUG::2013-11-19 21:33:30,905::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'exclusive'
>Thread-15911396::DEBUG::2013-11-19 21:33:30,906::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'exclusive' (1 active user)
>Thread-15911396::DEBUG::2013-11-19 21:33:30,909::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`471856ca-84e1-48c9-a828-58db130cc847`::Granted request
>Thread-15911396::DEBUG::2013-11-19 21:33:30,910::task::495::TaskManager.Task::(_debug) Task d3f111cf-7b31-41dc-bcdf-c918a6697f62: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (exclusive)
>Thread-15911396::DEBUG::2013-11-19 21:33:30,910::task::495::TaskManager.Task::(_debug) Task d3f111cf-7b31-41dc-bcdf-c918a6697f62: ref 1 aborting False
>Thread-15911396::DEBUG::2013-11-19 21:33:30,911::task::495::TaskManager.Task::(_debug) Task d3f111cf-7b31-41dc-bcdf-c918a6697f62: Prepare: 1 jobs exist, move to acquiring
>Thread-15911396::DEBUG::2013-11-19 21:33:30,914::task::495::TaskManager.Task::(_debug) Task d3f111cf-7b31-41dc-bcdf-c918a6697f62: moving from state preparing -> state acquiring
>Thread-15911396::DEBUG::2013-11-19 21:33:30,914::task::495::TaskManager.Task::(_debug) Task d3f111cf-7b31-41dc-bcdf-c918a6697f62: moving from state acquiring -> state queued
>Thread-15911396::DEBUG::2013-11-19 21:33:30,917::taskManager::37::TaskManager::(_queueTask) queueing task: d3f111cf-7b31-41dc-bcdf-c918a6697f62
>Thread-15911396::DEBUG::2013-11-19 21:33:30,919::taskManager::43::TaskManager::(_queueTask) task queued: d3f111cf-7b31-41dc-bcdf-c918a6697f62
>af42b7a2-b5f4-4f0d-b67c-ff9c021074aa::DEBUG::2013-11-19 21:33:30,919::threadPool::47::Misc.ThreadPool::(setRunningTask) Number of running tasks: 1
>Thread-15911396::DEBUG::2013-11-19 21:33:30,923::task::1183::TaskManager.Task::(prepare) returning
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:30,923::threadPool::192::Misc.ThreadPool.WorkerThread::(run) Task: d3f111cf-7b31-41dc-bcdf-c918a6697f62 running: <bound method Task.commit of <storage.task.Task instance at 0x7f4878072170>> with: None
>Thread-15911396::DEBUG::2013-11-19 21:33:30,928::task::495::TaskManager.Task::(_debug) Task d3f111cf-7b31-41dc-bcdf-c918a6697f62: ref 0 aborting False
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:30,929::task::1195::TaskManager.Task::(commit) Commiting task: d3f111cf-7b31-41dc-bcdf-c918a6697f62
>Thread-15911396::INFO::2013-11-19 21:33:30,932::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: spmStart, Return response: {'status': {'message': 'OK', 'code': 0}, 'uuid': 'd3f111cf-7b31-41dc-bcdf-c918a6697f62'}
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:30,933::task::495::TaskManager.Task::(_debug) Task d3f111cf-7b31-41dc-bcdf-c918a6697f62: moving from state queued -> state running
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:30,934::task::907::TaskManager.Task::(_runJobs) Task.run: running job 0: spmStart: <bound method SPM.start of <storage.spm.SPM instance at 0x1bf9f38>> (args: ('c4e5889a-348d-11e2-83fe-9f72369eab30', -1, '282', 0, 'false', 250, 2) kwargs: {})
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:30,935::task::300::TaskManager.Task::(run) Job.run: running spmStart: <bound method SPM.start of <storage.spm.SPM instance at 0x1bf9f38>> (args: ('c4e5889a-348d-11e2-83fe-9f72369eab30', -1, '282', 0, 'false', 250, 2) kwargs: {}) callback None
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:30,935::spm::433::Storage.SPM::(start) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:30,936::misc::1010::SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads)
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:30,936::misc::1012::SamplingMethod::(__call__) Got in to sampling method
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:30,937::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:30,938::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:30,938::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:30,939::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:30,940::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:30,941::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:31,628::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:31,632::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:31,633::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_LVER=283', u'_SHA_CKSUM=a0430374bee4cdb3438125b8b1df9b26332bc740', u'POOL_SPM_ID=-1']
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:31,634::misc::1020::SamplingMethod::(__call__) Returning last result
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:31,635::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:31,636::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:31,636::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:31,637::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:31,638::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:31,639::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15911397::DEBUG::2013-11-19 21:33:31,941::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911397::INFO::2013-11-19 21:33:31,946::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=d3f111cf-7b31-41dc-bcdf-c918a6697f62)
>Thread-15911397::DEBUG::2013-11-19 21:33:31,949::task::495::TaskManager.Task::(_debug) Task 7c6ed653-5afd-417e-804a-31b28d2935df: moving from state init -> state preparing
>Thread-15911397::DEBUG::2013-11-19 21:33:31,952::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: d3f111cf-7b31-41dc-bcdf-c918a6697f62
>Thread-15911397::DEBUG::2013-11-19 21:33:31,953::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': 'd3f111cf-7b31-41dc-bcdf-c918a6697f62'}
>Thread-15911397::DEBUG::2013-11-19 21:33:31,958::task::495::TaskManager.Task::(_debug) Task 7c6ed653-5afd-417e-804a-31b28d2935df: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': 'd3f111cf-7b31-41dc-bcdf-c918a6697f62'}}
>Thread-15911397::DEBUG::2013-11-19 21:33:31,959::task::495::TaskManager.Task::(_debug) Task 7c6ed653-5afd-417e-804a-31b28d2935df: moving from state preparing -> state finished
>Thread-15911397::DEBUG::2013-11-19 21:33:31,964::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911397::DEBUG::2013-11-19 21:33:31,965::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911397::DEBUG::2013-11-19 21:33:31,966::task::495::TaskManager.Task::(_debug) Task 7c6ed653-5afd-417e-804a-31b28d2935df: ref 0 aborting False
>Thread-15911397::INFO::2013-11-19 21:33:31,967::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': 'd3f111cf-7b31-41dc-bcdf-c918a6697f62'}}
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:32,300::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:32,303::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:32,304::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_LVER=283', u'_SHA_CKSUM=a0430374bee4cdb3438125b8b1df9b26332bc740', u'POOL_SPM_ID=-1']
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::INFO::2013-11-19 21:33:32,306::spm::457::Storage.SPM::(start) expected previd:-1 lver:283 got request for previd:-1 lver:282
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:32,307::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:32,307::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:32,308::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:32,309::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:32,309::persistentDict::163::Storage.PersistentDict::(__init__) Created a persistant dict with LvMetadataRW backend
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:32,310::blockSD::164::Storage.Misc.excCmd::(readlines) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:32,714::blockSD::164::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000321387 s, 6.4 MB/s\n'; <rc> = 0
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:32,715::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000321387 s, 6.4 MB/s'], size: 2048
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:32,716::persistentDict::204::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[]
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::WARNING::2013-11-19 21:33:32,716::persistentDict::226::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:32,717::persistentDict::163::Storage.PersistentDict::(__init__) Created a persistant dict with VGTagMetadataRW backend
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:32,718::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:32,719::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:32,719::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:32,720::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:32,720::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:32,722::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15911399::DEBUG::2013-11-19 21:33:32,984::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911399::INFO::2013-11-19 21:33:32,989::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=d3f111cf-7b31-41dc-bcdf-c918a6697f62)
>Thread-15911399::DEBUG::2013-11-19 21:33:32,990::task::495::TaskManager.Task::(_debug) Task b0097669-c869-44d3-b698-f1a9ed107b8c: moving from state init -> state preparing
>Thread-15911399::DEBUG::2013-11-19 21:33:32,991::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: d3f111cf-7b31-41dc-bcdf-c918a6697f62
>Thread-15911399::DEBUG::2013-11-19 21:33:32,999::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': 'd3f111cf-7b31-41dc-bcdf-c918a6697f62'}
>Thread-15911399::DEBUG::2013-11-19 21:33:33,002::task::495::TaskManager.Task::(_debug) Task b0097669-c869-44d3-b698-f1a9ed107b8c: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': 'd3f111cf-7b31-41dc-bcdf-c918a6697f62'}}
>Thread-15911399::DEBUG::2013-11-19 21:33:33,004::task::495::TaskManager.Task::(_debug) Task b0097669-c869-44d3-b698-f1a9ed107b8c: moving from state preparing -> state finished
>Thread-15911399::DEBUG::2013-11-19 21:33:33,004::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911399::DEBUG::2013-11-19 21:33:33,005::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911399::DEBUG::2013-11-19 21:33:33,009::task::495::TaskManager.Task::(_debug) Task b0097669-c869-44d3-b698-f1a9ed107b8c: ref 0 aborting False
>Thread-15911399::INFO::2013-11-19 21:33:33,017::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': 'd3f111cf-7b31-41dc-bcdf-c918a6697f62'}}
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:33,383::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:33,386::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:33,387::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_LVER=283', u'_SHA_CKSUM=a0430374bee4cdb3438125b8b1df9b26332bc740', u'POOL_SPM_ID=-1']
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:33,389::safelease::52::ClusterLock::(acquire) Acquiring cluster lock for domain 6b892067-ac7e-4039-a7be-5b7387525b16
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:33,390::safelease::61::Storage.Misc.excCmd::(acquire) '/usr/bin/sudo -n /usr/bin/setsid /usr/bin/ionice -c1 -n0 /bin/su vdsm -s /bin/sh -c "/usr/libexec/vdsm/spmprotect.sh start 6b892067-ac7e-4039-a7be-5b7387525b16 2 5 /dev/6b892067-ac7e-4039-a7be-5b7387525b16/leases 60000 10000 3"' (cwd /usr/libexec/vdsm/)
>Thread-15911400::DEBUG::2013-11-19 21:33:34,029::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911400::INFO::2013-11-19 21:33:34,038::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=d3f111cf-7b31-41dc-bcdf-c918a6697f62)
>Thread-15911400::DEBUG::2013-11-19 21:33:34,039::task::495::TaskManager.Task::(_debug) Task 9b767371-4fee-4b59-a382-57fd90d1f87b: moving from state init -> state preparing
>Thread-15911400::DEBUG::2013-11-19 21:33:34,044::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: d3f111cf-7b31-41dc-bcdf-c918a6697f62
>Thread-15911400::DEBUG::2013-11-19 21:33:34,046::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': 'd3f111cf-7b31-41dc-bcdf-c918a6697f62'}
>Thread-15911400::DEBUG::2013-11-19 21:33:34,053::task::495::TaskManager.Task::(_debug) Task 9b767371-4fee-4b59-a382-57fd90d1f87b: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': 'd3f111cf-7b31-41dc-bcdf-c918a6697f62'}}
>Thread-15911400::DEBUG::2013-11-19 21:33:34,055::task::495::TaskManager.Task::(_debug) Task 9b767371-4fee-4b59-a382-57fd90d1f87b: moving from state preparing -> state finished
>Thread-15911400::DEBUG::2013-11-19 21:33:34,065::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911400::DEBUG::2013-11-19 21:33:34,066::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911400::DEBUG::2013-11-19 21:33:34,070::task::495::TaskManager.Task::(_debug) Task 9b767371-4fee-4b59-a382-57fd90d1f87b: ref 0 aborting False
>Thread-15911400::INFO::2013-11-19 21:33:34,076::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': 'd3f111cf-7b31-41dc-bcdf-c918a6697f62'}}
>Thread-15911402::DEBUG::2013-11-19 21:33:35,087::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911402::INFO::2013-11-19 21:33:35,093::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=d3f111cf-7b31-41dc-bcdf-c918a6697f62)
>Thread-15911402::DEBUG::2013-11-19 21:33:35,094::task::495::TaskManager.Task::(_debug) Task 19eb3224-05b1-4373-9542-443a57b6b74b: moving from state init -> state preparing
>Thread-15911402::DEBUG::2013-11-19 21:33:35,097::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: d3f111cf-7b31-41dc-bcdf-c918a6697f62
>Thread-15911402::DEBUG::2013-11-19 21:33:35,097::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': 'd3f111cf-7b31-41dc-bcdf-c918a6697f62'}
>Thread-15911402::DEBUG::2013-11-19 21:33:35,098::task::495::TaskManager.Task::(_debug) Task 19eb3224-05b1-4373-9542-443a57b6b74b: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': 'd3f111cf-7b31-41dc-bcdf-c918a6697f62'}}
>Thread-15911402::DEBUG::2013-11-19 21:33:35,103::task::495::TaskManager.Task::(_debug) Task 19eb3224-05b1-4373-9542-443a57b6b74b: moving from state preparing -> state finished
>Thread-15911402::DEBUG::2013-11-19 21:33:35,108::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911402::DEBUG::2013-11-19 21:33:35,111::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911402::DEBUG::2013-11-19 21:33:35,114::task::495::TaskManager.Task::(_debug) Task 19eb3224-05b1-4373-9542-443a57b6b74b: ref 0 aborting False
>Thread-15911402::INFO::2013-11-19 21:33:35,117::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': 'd3f111cf-7b31-41dc-bcdf-c918a6697f62'}}
>Thread-1024767::ERROR::2013-11-19 21:33:37,205::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:33:47,219::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:53,822::safelease::61::Storage.Misc.excCmd::(acquire) SUCCESS: <err> = ''; <rc> = 0
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:53,823::safelease::65::ClusterLock::(acquire) Clustered lock acquired successfully
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:53,823::spm::468::Storage.SPM::(start) spm lock acquired successfully
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:53,824::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:53,825::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:53,825::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:53,826::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:53,827::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:53,828::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 21:33:54,179::lvm::439::OperationMutex::(_reloadvgs) Got the operational mutex
>Thread-42838::DEBUG::2013-11-19 21:33:54,181::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:54,502::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:54,508::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_LVER=283', u'_SHA_CKSUM=a0430374bee4cdb3438125b8b1df9b26332bc740', u'POOL_SPM_ID=-1']
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:54,510::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:54,511::persistentDict::146::Storage.PersistentDict::(transaction) Flushing changes
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:54,512::persistentDict::265::Storage.PersistentDict::(flush) about to write lines (VGTagMetadataRW)=[u'CLASS=Data', u'DESCRIPTION=OS', u'IOOPTIMEOUTSEC=10', u'LEASERETRIES=3', u'LEASETIMESEC=60', u'LOCKPOLICY=', u'LOCKRENEWALINTERVALSEC=5', u'LOGBLKSIZE=512', u'MASTER_VERSION=1', u'PHYBLKSIZE=512', u'POOL_DESCRIPTION=Default', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_ID=2', u'POOL_SPM_LVER=284', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'ROLE=Master', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'TYPE=FCP', u'VERSION=2', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', '_SHA_CKSUM=444b3b54067466f11e0599712ab301365a6be2aa']
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:54,513::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm reload operation' is holding the operation mutex, waiting...
>Thread-42838::DEBUG::2013-11-19 21:33:54,786::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-42838::DEBUG::2013-11-19 21:33:54,789::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:54,790::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:54,791::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:54,792::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:54,792::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:54,793::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:54,794::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:55,402::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:55,405::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:55,406::blockSD::145::storage.Metadata.VGTagMetadataRW::(writelines) Updating metadata adding=MDT_POOL_SPM_ID=2, MDT_POOL_SPM_LVER=284, MDT__SHA_CKSUM=444b3b54067466f11e0599712ab301365a6be2aa removing=MDT_POOL_SPM_ID=-1, MDT_POOL_SPM_LVER=283, MDT__SHA_CKSUM=a0430374bee4cdb3438125b8b1df9b26332bc740
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:55,408::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --deltag MDT_POOL_SPM_LVER=283 --deltag MDT__SHA_CKSUM=a0430374bee4cdb3438125b8b1df9b26332bc740 --deltag MDT_POOL_SPM_ID=-1 --addtag MDT_POOL_SPM_ID=2 --addtag MDT__SHA_CKSUM=444b3b54067466f11e0599712ab301365a6be2aa --addtag MDT_POOL_SPM_LVER=284 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:56,147::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:56,148::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:56,149::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:56,150::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:56,152::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`54b0ff01-1808-474c-b733-777c94c948e3`::Request was made in '/usr/share/vdsm/storage/spm.py' line '330' at '_upgradePool'
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:56,152::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'exclusive'
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:56,155::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'exclusive' (1 active user)
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:56,155::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`54b0ff01-1808-474c-b733-777c94c948e3`::Granted request
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::INFO::2013-11-19 21:33:56,156::spm::336::Storage.SPM::(_upgradePool) Trying to upgrade master domain `6b892067-ac7e-4039-a7be-5b7387525b16`
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:56,157::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ffd71c28-f399-4057-92d8-ecc590861d53`::Request was made in '/usr/share/vdsm/storage/spm.py' line '337' at '_upgradePool'
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:56,158::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'exclusive'
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:56,158::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'exclusive' (1 active user)
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:56,159::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ffd71c28-f399-4057-92d8-ecc590861d53`::Granted request
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:56,160::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:56,160::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:56,161::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:56,162::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:56,162::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:56,163::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:56,759::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:56,762::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:56,763::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_ID=2', u'_SHA_CKSUM=444b3b54067466f11e0599712ab301365a6be2aa', u'POOL_SPM_LVER=284']
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:56,764::blockSD::625::Storage.StorageDomain::(upgrade) Trying to upgrade domain `6b892067-ac7e-4039-a7be-5b7387525b16` from version 2 to version 2
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:56,765::blockSD::631::Storage.StorageDomain::(upgrade) No need to upgrade domain `6b892067-ac7e-4039-a7be-5b7387525b16`, leaving unchanged
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:56,766::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:56,766::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:56,767::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:56,767::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:56,768::spm::340::Storage.SPM::(_upgradePool) Marking all domains for upgrade
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:56,769::spm::347::Storage.SPM::(_upgradePool) Registering with state change event
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:56,769::spm::349::Storage.SPM::(_upgradePool) Running initial domain upgrade threads
>Thread-15911403::DEBUG::2013-11-19 21:33:56,770::spm::365::Storage.SPM::(_upgradePoolDomain) Preparing to upgrade domain 45cdff8b-89c0-4926-a8c7-80354064bfa6
>Thread-15911404::DEBUG::2013-11-19 21:33:56,771::spm::365::Storage.SPM::(_upgradePoolDomain) Preparing to upgrade domain 772472cb-2429-466d-96cd-00b3301ef78a
>Thread-15911405::DEBUG::2013-11-19 21:33:56,772::spm::365::Storage.SPM::(_upgradePoolDomain) Preparing to upgrade domain 5d7b5178-3790-472c-a13b-b481eb27a61e
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:56,782::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911403::DEBUG::2013-11-19 21:33:56,783::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6`ReqID=`3563c0a1-5cdc-49c0-b62d-2d1a126a690a`::Request was made in '/usr/share/vdsm/storage/spm.py' line '376' at '_upgradePoolDomain'
>Thread-15911404::DEBUG::2013-11-19 21:33:56,790::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8257dd3e-29f6-42cc-b5a0-a52a75c5c917`::Request was made in '/usr/share/vdsm/storage/spm.py' line '376' at '_upgradePoolDomain'
>Thread-15911405::ERROR::2013-11-19 21:33:56,790::spm::373::Storage.SPM::(_upgradePoolDomain) Error while preparing domain `5d7b5178-3790-472c-a13b-b481eb27a61e` upgrade
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/spm.py", line 369, in _upgradePoolDomain
>IndexError: list index out of range
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:56,791::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911403::DEBUG::2013-11-19 21:33:56,798::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6' for lock type 'exclusive'
>Thread-15911404::DEBUG::2013-11-19 21:33:56,805::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'exclusive'
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:56,807::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:56,812::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911403::DEBUG::2013-11-19 21:33:56,813::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6' is free. Now locking as 'exclusive' (1 active user)
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:56,814::blockSD::791::Storage.Misc.excCmd::(mountMaster) '/usr/bin/sudo -n /sbin/e2fsck -p /dev/6b892067-ac7e-4039-a7be-5b7387525b16/master' (cwd None)
>Thread-15911403::DEBUG::2013-11-19 21:33:56,814::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6`ReqID=`3563c0a1-5cdc-49c0-b62d-2d1a126a690a`::Granted request
>Thread-15911404::DEBUG::2013-11-19 21:33:57,034::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'exclusive' (1 active user)
>Thread-15911404::DEBUG::2013-11-19 21:33:57,036::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8257dd3e-29f6-42cc-b5a0-a52a75c5c917`::Granted request
>Thread-15911403::DEBUG::2013-11-19 21:33:57,038::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6`ReqID=`232a75f0-99b2-4d73-aff6-f7e9d1a1f5c4`::Request was made in '/usr/share/vdsm/storage/spm.py' line '377' at '_upgradePoolDomain'
>Thread-15911404::DEBUG::2013-11-19 21:33:57,040::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2e031752-9d1a-4dd2-80a3-76c2c699d4b3`::Request was made in '/usr/share/vdsm/storage/spm.py' line '377' at '_upgradePoolDomain'
>Thread-15911403::DEBUG::2013-11-19 21:33:57,043::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6' for lock type 'exclusive'
>Thread-15911404::DEBUG::2013-11-19 21:33:57,044::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'exclusive'
>Thread-15911403::DEBUG::2013-11-19 21:33:57,045::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6' is free. Now locking as 'exclusive' (1 active user)
>Thread-15911403::DEBUG::2013-11-19 21:33:57,051::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6`ReqID=`232a75f0-99b2-4d73-aff6-f7e9d1a1f5c4`::Granted request
>Thread-15911404::DEBUG::2013-11-19 21:33:57,055::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'exclusive' (1 active user)
>Thread-15911403::DEBUG::2013-11-19 21:33:57,055::spm::385::Storage.SPM::(_upgradePoolDomain) Domain `45cdff8b-89c0-4926-a8c7-80354064bfa6` is not a data domain it is an 2 domain, not upgrading
>Thread-15911404::DEBUG::2013-11-19 21:33:57,056::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2e031752-9d1a-4dd2-80a3-76c2c699d4b3`::Granted request
>Thread-15911403::DEBUG::2013-11-19 21:33:57,057::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6'
>Thread-15911404::DEBUG::2013-11-19 21:33:57,063::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15911403::DEBUG::2013-11-19 21:33:57,067::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6' (0 active users)
>Thread-15911404::DEBUG::2013-11-19 21:33:57,067::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15911403::DEBUG::2013-11-19 21:33:57,068::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6' is free, finding out if anyone is waiting for it.
>Thread-15911404::DEBUG::2013-11-19 21:33:57,069::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15911403::DEBUG::2013-11-19 21:33:57,069::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6', Clearing records.
>Thread-15911404::DEBUG::2013-11-19 21:33:57,070::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15911403::DEBUG::2013-11-19 21:33:57,071::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6'
>Thread-15911404::DEBUG::2013-11-19 21:33:57,072::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-15911403::DEBUG::2013-11-19 21:33:57,077::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6' (0 active users)
>Thread-15911404::DEBUG::2013-11-19 21:33:57,078::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-15911403::DEBUG::2013-11-19 21:33:57,079::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6' is free, finding out if anyone is waiting for it.
>Thread-15911403::DEBUG::2013-11-19 21:33:57,080::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6', Clearing records.
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:57,392::blockSD::791::Storage.Misc.excCmd::(mountMaster) SUCCESS: <err> = ''; <rc> = 0
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:57,393::blockSD::816::Storage.Misc.excCmd::(mountMaster) '/usr/bin/sudo -n /sbin/tune2fs -j /dev/6b892067-ac7e-4039-a7be-5b7387525b16/master' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:33:57,700::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911404::DEBUG::2013-11-19 21:33:57,873::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15911404::DEBUG::2013-11-19 21:33:57,882::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-15911404::DEBUG::2013-11-19 21:33:57,883::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'SDUUID=772472cb-2429-466d-96cd-00b3301ef78a', u'VERSION=2', u'LOCKPOLICY=', u'VGUUID=XTb294-NYv6-dVpO-bB1n-4f9P-d0GL-dhd2DI', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'LEASETIMESEC=5', u'LOCKRENEWALINTERVALSEC=5', u'ROLE=Regular', u'PHYBLKSIZE=512', u'DESCRIPTION=Database', u'IOOPTIMEOUTSEC=1', u'CLASS=Data', u'TYPE=FCP', u'PV0=pv:3600c0ff0001520c7414f605001000000,uuid:NujM33-vNiD-sBHY-mrPo-EQ2f-1qsz-HXfn0q,pestart:0,pecount:8929,mapoffset:0', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'MASTER_VERSION=0', u'_SHA_CKSUM=ef8840d9fdef24516a56fd970bed1960d2b1ae92']
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:57,884::blockSD::816::Storage.Misc.excCmd::(mountMaster) FAILED: <err> = 'The filesystem already has a journal.\n'; <rc> = 1
>Thread-15911404::DEBUG::2013-11-19 21:33:57,886::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15911404::DEBUG::2013-11-19 21:33:57,892::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15911404::DEBUG::2013-11-19 21:33:57,893::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:57,895::fileUtils::83::Storage.Misc.excCmd::(mount) '/usr/bin/sudo -n /bin/mount /dev/6b892067-ac7e-4039-a7be-5b7387525b16/master /rhev/data-center/mnt/blockSD/6b892067-ac7e-4039-a7be-5b7387525b16/master' (cwd None)
>Thread-15911404::DEBUG::2013-11-19 21:33:57,896::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15911404::DEBUG::2013-11-19 21:33:58,115::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15911404::DEBUG::2013-11-19 21:33:58,117::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15911404::DEBUG::2013-11-19 21:33:58,118::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15911404::DEBUG::2013-11-19 21:33:58,119::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,325::fileUtils::83::Storage.Misc.excCmd::(mount) SUCCESS: <err> = ''; <rc> = 0
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,326::blockSD::833::Storage.Misc.excCmd::(mountMaster) '/usr/bin/sudo -n /bin/chown vdsm:kvm /rhev/data-center/mnt/blockSD/6b892067-ac7e-4039-a7be-5b7387525b16/master' (cwd None)
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,746::blockSD::833::Storage.Misc.excCmd::(mountMaster) SUCCESS: <err> = ''; <rc> = 0
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::WARNING::2013-11-19 21:33:58,749::spm::490::Storage.SPM::(start) Backup domain validation failed
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/spm.py", line 488, in start
>  File "/usr/share/vdsm/storage/sp.py", line 1208, in checkBackupDomain
>  File "/usr/share/vdsm/storage/sd.py", line 666, in isBackup
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,751::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,752::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,752::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,753::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,754::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,754::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,755::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,756::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,756::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,757::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,757::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,758::persistentDict::146::Storage.PersistentDict::(transaction) Flushing changes
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,759::persistentDict::265::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['MASTER_VERSION=0', 'ROLE=Regular', '_SHA_CKSUM=1deb73127f89496e2cb17e51bbfae1df6c1879d1']
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::WARNING::2013-11-19 21:33:58,760::persistentDict::150::Storage.PersistentDict::(transaction) Error in transaction, rolling back changes
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/persistentDict.py", line 147, in transaction
>  File "/usr/share/vdsm/storage/persistentDict.py", line 266, in flush
>  File "/usr/share/vdsm/storage/fileSD.py", line 58, in writelines
>  File "/usr/share/vdsm/storage/processPool.py", line 33, in wrapper
>  File "/usr/share/vdsm/storage/processPool.py", line 44, in runExternally
>  File "/usr/share/vdsm/storage/processPool.py", line 127, in runExternally
>IOError: [Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata.new'
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,761::persistentDict::265::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['_SHA_CKSUM=da39a3ee5e6b4b0d3255bfef95601890afd80709']
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::ERROR::2013-11-19 21:33:58,762::spm::526::Storage.SPM::(start) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/spm.py", line 493, in start
>  File "/usr/share/vdsm/storage/sp.py", line 637, in _regulariseOutdatedMasters
>  File "/usr/share/vdsm/storage/sp.py", line 631, in _setRegularSD
>  File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 152, in transaction
>  File "/usr/share/vdsm/storage/persistentDict.py", line 266, in flush
>  File "/usr/share/vdsm/storage/fileSD.py", line 58, in writelines
>  File "/usr/share/vdsm/storage/processPool.py", line 33, in wrapper
>  File "/usr/share/vdsm/storage/processPool.py", line 44, in runExternally
>  File "/usr/share/vdsm/storage/processPool.py", line 127, in runExternally
>IOError: [Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata.new'
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::ERROR::2013-11-19 21:33:58,763::spm::527::Storage.SPM::(start) failed: [Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata.new'
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,765::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6301b211-13c4-49fe-8aaa-74bb58f8a3e1`::Request was made in '/usr/share/vdsm/storage/spm.py' line '584' at '_stop'
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,765::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'exclusive'
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,766::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'exclusive' (1 active user)
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,767::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6301b211-13c4-49fe-8aaa-74bb58f8a3e1`::Granted request
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,769::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e`ReqID=`1e585214-39b8-407f-94b4-a7beb4836335`::Request was made in '/usr/share/vdsm/storage/spm.py' line '601' at '_stop'
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,769::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e' for lock type 'exclusive'
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,770::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e' is free. Now locking as 'exclusive' (1 active user)
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,771::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e`ReqID=`1e585214-39b8-407f-94b4-a7beb4836335`::Granted request
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,772::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e'
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,773::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e' (0 active users)
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,773::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e' is free, finding out if anyone is waiting for it.
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,774::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e', Clearing records.
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,775::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30'
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,776::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,777::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,777::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,778::taskManager::118::TaskManager::(unloadTasks) Entry.
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,779::taskManager::122::TaskManager::(unloadTasks) Return
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,782::spm::195::Storage.SPM::(__cleanupMasterMount) unmounting /rhev/data-center/mnt/blockSD/6b892067-ac7e-4039-a7be-5b7387525b16/master
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:58,901::fileUtils::112::Storage.Misc.excCmd::(umount) '/usr/bin/sudo -n /bin/umount -f /rhev/data-center/mnt/blockSD/6b892067-ac7e-4039-a7be-5b7387525b16/master' (cwd None)
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:59,269::fileUtils::112::Storage.Misc.excCmd::(umount) SUCCESS: <err> = ''; <rc> = 0
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:59,272::spm::290::Storage.SPM::(__cleanupSPM) cleaning up SPM: c4e5889a-348d-11e2-83fe-9f72369eab30
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:59,273::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:59,273::persistentDict::146::Storage.PersistentDict::(transaction) Flushing changes
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:59,274::persistentDict::265::Storage.PersistentDict::(flush) about to write lines (VGTagMetadataRW)=[u'CLASS=Data', u'DESCRIPTION=OS', u'IOOPTIMEOUTSEC=10', u'LEASERETRIES=3', u'LEASETIMESEC=60', u'LOCKPOLICY=', u'LOCKRENEWALINTERVALSEC=5', u'LOGBLKSIZE=512', u'MASTER_VERSION=1', u'PHYBLKSIZE=512', u'POOL_DESCRIPTION=Default', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_ID=-1', u'POOL_SPM_LVER=284', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'ROLE=Master', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'TYPE=FCP', u'VERSION=2', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', '_SHA_CKSUM=fa1ddc0ebaad4a3d815671fdef2d27b7d9f8a82e']
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:59,275::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:59,276::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:59,277::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:59,277::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:59,278::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:59,279::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:59,949::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:59,952::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:59,953::blockSD::145::storage.Metadata.VGTagMetadataRW::(writelines) Updating metadata adding=MDT_POOL_SPM_ID=-1, MDT__SHA_CKSUM=fa1ddc0ebaad4a3d815671fdef2d27b7d9f8a82e removing=MDT_POOL_SPM_ID=2, MDT__SHA_CKSUM=444b3b54067466f11e0599712ab301365a6be2aa
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:33:59,954::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --deltag MDT_POOL_SPM_ID=2 --deltag MDT__SHA_CKSUM=444b3b54067466f11e0599712ab301365a6be2aa --addtag MDT__SHA_CKSUM=fa1ddc0ebaad4a3d815671fdef2d27b7d9f8a82e --addtag MDT_POOL_SPM_ID=-1 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:34:00,637::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:34:00,638::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:34:00,639::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:34:00,639::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::INFO::2013-11-19 21:34:00,640::safelease::83::ClusterLock::(release) Releasing cluster lock for domain 6b892067-ac7e-4039-a7be-5b7387525b16
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:34:00,641::safelease::84::Storage.Misc.excCmd::(release) '/usr/libexec/vdsm/spmstop.sh 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd /usr/libexec/vdsm/)
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:34:02,110::safelease::84::Storage.Misc.excCmd::(release) SUCCESS: <err> = ''; <rc> = 0
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:34:02,111::safelease::87::ClusterLock::(release) Cluster lock released successfully
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::ERROR::2013-11-19 21:34:02,112::task::868::TaskManager.Task::(_setError) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/task.py", line 876, in _run
>  File "/usr/share/vdsm/storage/task.py", line 303, in run
>  File "/usr/share/vdsm/storage/spm.py", line 493, in start
>  File "/usr/share/vdsm/storage/sp.py", line 637, in _regulariseOutdatedMasters
>  File "/usr/share/vdsm/storage/sp.py", line 631, in _setRegularSD
>  File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 152, in transaction
>  File "/usr/share/vdsm/storage/persistentDict.py", line 266, in flush
>  File "/usr/share/vdsm/storage/fileSD.py", line 58, in writelines
>  File "/usr/share/vdsm/storage/processPool.py", line 33, in wrapper
>  File "/usr/share/vdsm/storage/processPool.py", line 44, in runExternally
>  File "/usr/share/vdsm/storage/processPool.py", line 127, in runExternally
>IOError: [Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata.new'
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:34:02,113::task::495::TaskManager.Task::(_debug) Task d3f111cf-7b31-41dc-bcdf-c918a6697f62: Task._run: d3f111cf-7b31-41dc-bcdf-c918a6697f62 () {} failed - stopping task
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:34:02,113::task::495::TaskManager.Task::(_debug) Task d3f111cf-7b31-41dc-bcdf-c918a6697f62: stopping in state running (force False)
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:34:02,114::task::495::TaskManager.Task::(_debug) Task d3f111cf-7b31-41dc-bcdf-c918a6697f62: ref 1 aborting True
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:34:02,115::task::918::TaskManager.Task::(_runJobs) aborting: Task is aborted: "[Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata.new'" - code 100
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:34:02,115::task::495::TaskManager.Task::(_debug) Task d3f111cf-7b31-41dc-bcdf-c918a6697f62: ref 0 aborting True
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:34:02,116::task::495::TaskManager.Task::(_debug) Task d3f111cf-7b31-41dc-bcdf-c918a6697f62: Task._doAbort: force False
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:34:02,116::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:34:02,117::task::495::TaskManager.Task::(_debug) Task d3f111cf-7b31-41dc-bcdf-c918a6697f62: moving from state running -> state aborting
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:34:02,117::task::495::TaskManager.Task::(_debug) Task d3f111cf-7b31-41dc-bcdf-c918a6697f62: _aborting: recover policy auto
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:34:02,118::task::495::TaskManager.Task::(_debug) Task d3f111cf-7b31-41dc-bcdf-c918a6697f62: moving from state aborting -> state racquiring
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:34:02,118::task::495::TaskManager.Task::(_debug) Task d3f111cf-7b31-41dc-bcdf-c918a6697f62: moving from state racquiring -> state recovering
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:34:02,119::task::495::TaskManager.Task::(_debug) Task d3f111cf-7b31-41dc-bcdf-c918a6697f62: _recover
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:34:02,119::task::495::TaskManager.Task::(_debug) Task d3f111cf-7b31-41dc-bcdf-c918a6697f62: running recovery None
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:34:02,120::task::495::TaskManager.Task::(_debug) Task d3f111cf-7b31-41dc-bcdf-c918a6697f62: Recover Done: state recovering
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:34:02,120::task::495::TaskManager.Task::(_debug) Task d3f111cf-7b31-41dc-bcdf-c918a6697f62: moving from state recovering -> state recovered
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:34:02,121::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:34:02,121::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:34:02,123::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:34:02,123::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:34:02,124::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:34:02,124::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>d3f111cf-7b31-41dc-bcdf-c918a6697f62::DEBUG::2013-11-19 21:34:02,125::threadPool::47::Misc.ThreadPool::(setRunningTask) Number of running tasks: 0
>Thread-42838::DEBUG::2013-11-19 21:34:06,548::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-42838::DEBUG::2013-11-19 21:34:06,550::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 21:34:07,209::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-42838::DEBUG::2013-11-19 21:34:07,212::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1024767::ERROR::2013-11-19 21:34:07,866::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:34:17,879::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:34:27,893::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911406::INFO::2013-11-19 21:34:34,768::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911406::DEBUG::2013-11-19 21:34:34,769::task::495::TaskManager.Task::(_debug) Task 74fecb06-5d4f-4bbf-9783-2945d9e62584: moving from state init -> state preparing
>Thread-15911406::DEBUG::2013-11-19 21:34:34,770::task::495::TaskManager.Task::(_debug) Task 74fecb06-5d4f-4bbf-9783-2945d9e62584: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000329971313477', 'lastCheck': 1384896867.2409911, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142693519592', 'lastCheck': 1384896872.21348, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896867.894151, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000482082366943', 'lastCheck': 1384896868.8891749, 'valid': True, 'code': 0}}
>Thread-15911406::DEBUG::2013-11-19 21:34:34,771::task::495::TaskManager.Task::(_debug) Task 74fecb06-5d4f-4bbf-9783-2945d9e62584: moving from state preparing -> state finished
>Thread-15911406::DEBUG::2013-11-19 21:34:34,771::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911406::DEBUG::2013-11-19 21:34:34,772::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911406::DEBUG::2013-11-19 21:34:34,773::task::495::TaskManager.Task::(_debug) Task 74fecb06-5d4f-4bbf-9783-2945d9e62584: ref 0 aborting False
>Thread-15911406::INFO::2013-11-19 21:34:34,773::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000329971313477', 'lastCheck': 1384896867.2409911, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142693519592', 'lastCheck': 1384896872.21348, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896867.894151, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000482082366943', 'lastCheck': 1384896868.8891749, 'valid': True, 'code': 0}}
>Thread-15911407::DEBUG::2013-11-19 21:34:34,906::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVdsCapabilities with () {}
>Thread-15911407::DEBUG::2013-11-19 21:34:35,375::utils::579::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-img-rhev' (cwd None)
>Thread-15911407::DEBUG::2013-11-19 21:34:35,896::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15911407::DEBUG::2013-11-19 21:34:35,897::utils::579::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-kvm-rhev' (cwd None)
>Thread-15911407::DEBUG::2013-11-19 21:34:36,385::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15911407::DEBUG::2013-11-19 21:34:36,386::utils::579::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" vdsm' (cwd None)
>Thread-15911407::DEBUG::2013-11-19 21:34:36,804::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15911407::DEBUG::2013-11-19 21:34:36,806::utils::579::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" spice-server' (cwd None)
>Thread-15911407::DEBUG::2013-11-19 21:34:37,238::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15911407::DEBUG::2013-11-19 21:34:37,239::utils::579::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" libvirt' (cwd None)
>Thread-15911407::DEBUG::2013-11-19 21:34:37,656::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15911407::DEBUG::2013-11-19 21:34:37,670::clientIF::59::vds::(wrapper) return getVdsCapabilities with {'status': {'message': 'Done', 'code': 0}, 'info': {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:c936ebb147a8'}], 'FC': []}, 'packages2': {'kernel': {'release': '279.11.1.el6.x86_64', 'buildtime': 1348297826.0, 'version': '2.6.32'}, 'spice-server': {'release': '10.el6', 'buildtime': '1337611492', 'version': '0.10.1'}, 'vdsm': {'release': '113.4.el6_3', 'buildtime': '1347534072', 'version': '4.9'}, 'qemu-kvm': {'release': '2.295.el6_3.2', 'buildtime': '1346673796', 'version': '0.12.1.2'}, 'libvirt': {'release': '21.el6_3.5', 'buildtime': '1348687577', 'version': '0.9.10'}, 'qemu-img': {'release': '2.295.el6_3.2', 'buildtime': '1346673796', 'version': '0.12.1.2'}}, 'cpuModel': 'Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz', 'hooks': {'before_vm_start': {'50_vhostmd': {'md5': '1635e45584995d9ad2629939cb0bcdbf'}}, 'after_vm_destroy': {'50_vhostmd': {'md5': '7e7509430304099f38e4a9de966af090'}}, 'before_vm_dehibernate': {'50_vhostmd': {'md5': '1635e45584995d9ad2629939cb0bcdbf'}}, 'before_vm_migrate_destination': {'50_vhostmd': {'md5': '1635e45584995d9ad2629939cb0bcdbf'}}}, 'vmTypes': ['kvm'], 'supportedProtocols': ['2.2', '2.3'], 'networks': {'rhevm': {'addr': '192.168.100.133', 'cfg': {'IPADDR': '192.168.100.133', 'GATEWAY': '192.168.100.1', 'DELAY': '0', 'NETMASK': '255.255.128.0', 'STP': 'no', 'DEVICE': 'rhevm', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'ports': ['bond0'], 'netmask': '255.255.128.0', 'stp': 'off', 'gateway': '192.168.100.1'}, 'DMZ': {'addr': '172.16.1.133', 'cfg': {'IPADDR': '172.16.1.133', 'DELAY': '0', 'NETMASK': '255.255.255.0', 'STP': 'no', 'DEVICE': 'DMZ', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'ports': ['eth0'], 'netmask': '255.255.255.0', 'stp': 'off', 'gateway': '0.0.0.0'}}, 'uuid': '30343536-3138-5A43-4A32-333630373442_AC:16:2D:76:EB:F0', 'lastClientIface': 'rhevm', 'nics': {'eth3': {'hwaddr': 'AC:16:2D:76:EB:F2', 'netmask': '', 'speed': 1000, 'addr': '', 'permhwaddr': 'AC:16:2D:76:EB:F3'}, 'eth2': {'hwaddr': 'AC:16:2D:76:EB:F2', 'netmask': '', 'speed': 1000, 'addr': '', 'permhwaddr': 'AC:16:2D:76:EB:F2'}, 'eth1': {'hwaddr': 'AC:16:2D:76:EB:F1', 'netmask': '', 'speed': 0, 'addr': ''}, 'eth0': {'hwaddr': 'AC:16:2D:76:EB:F0', 'netmask': '', 'speed': 1000, 'addr': ''}}, 'software_revision': '113.4', 'management_ip': '192.168.100.133', 'clusterLevels': ['3.0'], 'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,constant_tsc,arch_perfmon,pebs,bts,rep_good,xtopology,nonstop_tsc,aperfmperf,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,dca,sse4_1,sse4_2,x2apic,popcnt,aes,xsave,avx,lahf_lm,ida,arat,epb,xsaveopt,pln,pts,dts,tpr_shadow,vnmi,flexpriority,ept,vpid,model_486,model_pentium,model_pentium2,model_pentium3,model_pentiumpro,model_qemu32,model_cpu64-rhel5,model_cpu64-rhel6,model_kvm32,model_coreduo,model_kvm64,model_core2duo,model_n270,model_Conroe,model_Penryn,model_Nehalem,model_Westmere,model_SandyBridge,model_Opteron_G1', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:c936ebb147a8', 'memSize': '64392', 'reservedMem': '256', 'bondings': {'bond4': {'hwaddr': '00:00:00:00:00:00', 'cfg': {}, 'netmask': '', 'addr': '', 'slaves': []}, 'bond0': {'hwaddr': 'AC:16:2D:76:EB:F2', 'cfg': {'DEVICE': 'bond0', 'BRIDGE': 'rhevm', 'BOOTPROTO': 'none', 'ONBOOT': 'yes', 'BONDING_OPTS': 'mode=1 miimon=100'}, 'netmask': '', 'addr': '', 'slaves': ['eth2', 'eth3']}, 'bond1': {'hwaddr': '00:00:00:00:00:00', 'cfg': {}, 'netmask': '', 'addr': '', 'slaves': []}, 'bond2': {'hwaddr': '00:00:00:00:00:00', 'cfg': {}, 'netmask': '', 'addr': '', 'slaves': []}, 'bond3': {'hwaddr': '00:00:00:00:00:00', 'cfg': {}, 'netmask': '', 'addr': '', 'slaves': []}}, 'software_version': '4.9', 'cpuSpeed': '1995.126', 'cpuSockets': '2', 'vlans': {}, 'cpuCores': '12', 'kvmEnabled': 'true', 'guestOverhead': '65', 'supportedRHEVMs': ['3.0'], 'version_name': 'Snow Man', 'emulatedMachines': [u'rhel6.3.0', u'pc', u'rhel6.2.0', u'rhel6.1.0', u'rhel6.0.0', u'rhel5.5.0', u'rhel5.4.4', u'rhel5.4.0'], 'operatingSystem': {'release': '20121012.0.el6_3', 'version': '6.3', 'name': 'RHEV Hypervisor'}, 'lastClient': '192.168.100.135'}}
>Thread-1024767::ERROR::2013-11-19 21:34:38,135::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911412::DEBUG::2013-11-19 21:34:44,937::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911412::INFO::2013-11-19 21:34:44,939::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911412::DEBUG::2013-11-19 21:34:44,939::task::495::TaskManager.Task::(_debug) Task f5fc2942-aa8e-4768-b5b7-1ef311448185: moving from state init -> state preparing
>Thread-15911412::DEBUG::2013-11-19 21:34:44,940::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=Free spmLver=284 spmId=-1
>Thread-15911412::DEBUG::2013-11-19 21:34:44,941::task::495::TaskManager.Task::(_debug) Task f5fc2942-aa8e-4768-b5b7-1ef311448185: finished: {'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': 284}}
>Thread-15911412::DEBUG::2013-11-19 21:34:44,941::task::495::TaskManager.Task::(_debug) Task f5fc2942-aa8e-4768-b5b7-1ef311448185: moving from state preparing -> state finished
>Thread-15911412::DEBUG::2013-11-19 21:34:44,942::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911412::DEBUG::2013-11-19 21:34:44,942::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911412::DEBUG::2013-11-19 21:34:44,943::task::495::TaskManager.Task::(_debug) Task f5fc2942-aa8e-4768-b5b7-1ef311448185: ref 0 aborting False
>Thread-15911412::INFO::2013-11-19 21:34:44,943::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': 284}}
>Thread-15911413::DEBUG::2013-11-19 21:34:44,993::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911413::INFO::2013-11-19 21:34:44,994::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: spmStart, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 prevID=-1 prevLVER=284 recoveryMode=0 scsiFencing=false maxHostID=250 domVersion=2)
>Thread-15911413::DEBUG::2013-11-19 21:34:44,995::task::495::TaskManager.Task::(_debug) Task 44a1be5a-8538-4745-8f2c-b3e70865e4e7: moving from state init -> state preparing
>Thread-15911413::DEBUG::2013-11-19 21:34:44,996::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`33916515-dffe-468a-be56-a8223b622f4a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '330' at 'public_spmStart'
>Thread-15911413::DEBUG::2013-11-19 21:34:44,997::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'exclusive'
>Thread-15911413::DEBUG::2013-11-19 21:34:44,998::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'exclusive' (1 active user)
>Thread-15911413::DEBUG::2013-11-19 21:34:44,999::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`33916515-dffe-468a-be56-a8223b622f4a`::Granted request
>Thread-15911413::DEBUG::2013-11-19 21:34:45,000::task::495::TaskManager.Task::(_debug) Task 44a1be5a-8538-4745-8f2c-b3e70865e4e7: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (exclusive)
>Thread-15911413::DEBUG::2013-11-19 21:34:45,000::task::495::TaskManager.Task::(_debug) Task 44a1be5a-8538-4745-8f2c-b3e70865e4e7: ref 1 aborting False
>Thread-15911413::DEBUG::2013-11-19 21:34:45,001::task::495::TaskManager.Task::(_debug) Task 44a1be5a-8538-4745-8f2c-b3e70865e4e7: Prepare: 1 jobs exist, move to acquiring
>Thread-15911413::DEBUG::2013-11-19 21:34:45,001::task::495::TaskManager.Task::(_debug) Task 44a1be5a-8538-4745-8f2c-b3e70865e4e7: moving from state preparing -> state acquiring
>Thread-15911413::DEBUG::2013-11-19 21:34:45,002::task::495::TaskManager.Task::(_debug) Task 44a1be5a-8538-4745-8f2c-b3e70865e4e7: moving from state acquiring -> state queued
>Thread-15911413::DEBUG::2013-11-19 21:34:45,002::taskManager::37::TaskManager::(_queueTask) queueing task: 44a1be5a-8538-4745-8f2c-b3e70865e4e7
>Thread-15911413::DEBUG::2013-11-19 21:34:45,003::taskManager::43::TaskManager::(_queueTask) task queued: 44a1be5a-8538-4745-8f2c-b3e70865e4e7
>934c4d35-aab6-4fba-9996-9cc07dd7d991::DEBUG::2013-11-19 21:34:45,003::threadPool::47::Misc.ThreadPool::(setRunningTask) Number of running tasks: 1
>Thread-15911413::DEBUG::2013-11-19 21:34:45,004::task::1183::TaskManager.Task::(prepare) returning
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:45,005::threadPool::192::Misc.ThreadPool.WorkerThread::(run) Task: 44a1be5a-8538-4745-8f2c-b3e70865e4e7 running: <bound method Task.commit of <storage.task.Task instance at 0x7f47180ba488>> with: None
>Thread-15911413::DEBUG::2013-11-19 21:34:45,005::task::495::TaskManager.Task::(_debug) Task 44a1be5a-8538-4745-8f2c-b3e70865e4e7: ref 0 aborting False
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:45,006::task::1195::TaskManager.Task::(commit) Commiting task: 44a1be5a-8538-4745-8f2c-b3e70865e4e7
>Thread-15911413::INFO::2013-11-19 21:34:45,006::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: spmStart, Return response: {'status': {'message': 'OK', 'code': 0}, 'uuid': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:45,007::task::495::TaskManager.Task::(_debug) Task 44a1be5a-8538-4745-8f2c-b3e70865e4e7: moving from state queued -> state running
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:45,008::task::907::TaskManager.Task::(_runJobs) Task.run: running job 0: spmStart: <bound method SPM.start of <storage.spm.SPM instance at 0x1bf9f38>> (args: ('c4e5889a-348d-11e2-83fe-9f72369eab30', -1, '284', 0, 'false', 250, 2) kwargs: {})
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:45,009::task::300::TaskManager.Task::(run) Job.run: running spmStart: <bound method SPM.start of <storage.spm.SPM instance at 0x1bf9f38>> (args: ('c4e5889a-348d-11e2-83fe-9f72369eab30', -1, '284', 0, 'false', 250, 2) kwargs: {}) callback None
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:45,009::spm::433::Storage.SPM::(start) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:45,010::misc::1010::SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads)
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:45,010::misc::1012::SamplingMethod::(__call__) Got in to sampling method
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:45,011::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:45,011::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:45,012::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:45,013::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:45,013::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:45,014::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:45,664::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:45,668::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:45,669::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_LVER=284', u'_SHA_CKSUM=fa1ddc0ebaad4a3d815671fdef2d27b7d9f8a82e', u'POOL_SPM_ID=-1']
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:45,670::misc::1020::SamplingMethod::(__call__) Returning last result
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:45,671::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:45,672::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:45,672::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:45,673::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:45,674::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:45,675::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15911414::DEBUG::2013-11-19 21:34:46,028::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911414::INFO::2013-11-19 21:34:46,030::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=44a1be5a-8538-4745-8f2c-b3e70865e4e7)
>Thread-15911414::DEBUG::2013-11-19 21:34:46,031::task::495::TaskManager.Task::(_debug) Task 1409ee5c-9802-489c-ab73-13069d3ce063: moving from state init -> state preparing
>Thread-15911414::DEBUG::2013-11-19 21:34:46,032::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 44a1be5a-8538-4745-8f2c-b3e70865e4e7
>Thread-15911414::DEBUG::2013-11-19 21:34:46,032::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}
>Thread-15911414::DEBUG::2013-11-19 21:34:46,033::task::495::TaskManager.Task::(_debug) Task 1409ee5c-9802-489c-ab73-13069d3ce063: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911414::DEBUG::2013-11-19 21:34:46,034::task::495::TaskManager.Task::(_debug) Task 1409ee5c-9802-489c-ab73-13069d3ce063: moving from state preparing -> state finished
>Thread-15911414::DEBUG::2013-11-19 21:34:46,034::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911414::DEBUG::2013-11-19 21:34:46,035::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911414::DEBUG::2013-11-19 21:34:46,035::task::495::TaskManager.Task::(_debug) Task 1409ee5c-9802-489c-ab73-13069d3ce063: ref 0 aborting False
>Thread-15911414::INFO::2013-11-19 21:34:46,036::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:46,281::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:46,284::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:46,285::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_LVER=284', u'_SHA_CKSUM=fa1ddc0ebaad4a3d815671fdef2d27b7d9f8a82e', u'POOL_SPM_ID=-1']
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:46,287::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:46,288::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:46,289::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:46,289::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:46,290::persistentDict::163::Storage.PersistentDict::(__init__) Created a persistant dict with LvMetadataRW backend
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:46,291::blockSD::164::Storage.Misc.excCmd::(readlines) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:46,701::blockSD::164::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000226921 s, 9.0 MB/s\n'; <rc> = 0
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:46,703::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000226921 s, 9.0 MB/s'], size: 2048
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:46,703::persistentDict::204::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[]
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::WARNING::2013-11-19 21:34:46,704::persistentDict::226::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:46,704::persistentDict::163::Storage.PersistentDict::(__init__) Created a persistant dict with VGTagMetadataRW backend
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:46,705::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:46,706::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:46,706::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:46,707::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:46,708::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:46,709::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15911416::DEBUG::2013-11-19 21:34:47,059::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911416::INFO::2013-11-19 21:34:47,061::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=44a1be5a-8538-4745-8f2c-b3e70865e4e7)
>Thread-15911416::DEBUG::2013-11-19 21:34:47,062::task::495::TaskManager.Task::(_debug) Task 3956c1b1-80a9-4b7f-8d57-eee5239d1809: moving from state init -> state preparing
>Thread-15911416::DEBUG::2013-11-19 21:34:47,063::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 44a1be5a-8538-4745-8f2c-b3e70865e4e7
>Thread-15911416::DEBUG::2013-11-19 21:34:47,063::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}
>Thread-15911416::DEBUG::2013-11-19 21:34:47,064::task::495::TaskManager.Task::(_debug) Task 3956c1b1-80a9-4b7f-8d57-eee5239d1809: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911416::DEBUG::2013-11-19 21:34:47,065::task::495::TaskManager.Task::(_debug) Task 3956c1b1-80a9-4b7f-8d57-eee5239d1809: moving from state preparing -> state finished
>Thread-15911416::DEBUG::2013-11-19 21:34:47,065::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911416::DEBUG::2013-11-19 21:34:47,066::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911416::DEBUG::2013-11-19 21:34:47,066::task::495::TaskManager.Task::(_debug) Task 3956c1b1-80a9-4b7f-8d57-eee5239d1809: ref 0 aborting False
>Thread-15911416::INFO::2013-11-19 21:34:47,067::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:47,296::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:47,299::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:47,300::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_LVER=284', u'_SHA_CKSUM=fa1ddc0ebaad4a3d815671fdef2d27b7d9f8a82e', u'POOL_SPM_ID=-1']
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:47,301::safelease::52::ClusterLock::(acquire) Acquiring cluster lock for domain 6b892067-ac7e-4039-a7be-5b7387525b16
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:34:47,302::safelease::61::Storage.Misc.excCmd::(acquire) '/usr/bin/sudo -n /usr/bin/setsid /usr/bin/ionice -c1 -n0 /bin/su vdsm -s /bin/sh -c "/usr/libexec/vdsm/spmprotect.sh start 6b892067-ac7e-4039-a7be-5b7387525b16 2 5 /dev/6b892067-ac7e-4039-a7be-5b7387525b16/leases 60000 10000 3"' (cwd /usr/libexec/vdsm/)
>Thread-42838::DEBUG::2013-11-19 21:34:47,516::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15911417::DEBUG::2013-11-19 21:34:48,079::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911417::INFO::2013-11-19 21:34:48,081::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=44a1be5a-8538-4745-8f2c-b3e70865e4e7)
>Thread-15911417::DEBUG::2013-11-19 21:34:48,081::task::495::TaskManager.Task::(_debug) Task 5b664412-99ef-423e-bdd2-c34bb4a26f29: moving from state init -> state preparing
>Thread-15911417::DEBUG::2013-11-19 21:34:48,082::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 44a1be5a-8538-4745-8f2c-b3e70865e4e7
>Thread-15911417::DEBUG::2013-11-19 21:34:48,085::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}
>Thread-15911417::DEBUG::2013-11-19 21:34:48,086::task::495::TaskManager.Task::(_debug) Task 5b664412-99ef-423e-bdd2-c34bb4a26f29: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911418::INFO::2013-11-19 21:34:48,088::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911417::DEBUG::2013-11-19 21:34:48,088::task::495::TaskManager.Task::(_debug) Task 5b664412-99ef-423e-bdd2-c34bb4a26f29: moving from state preparing -> state finished
>Thread-15911418::DEBUG::2013-11-19 21:34:48,089::task::495::TaskManager.Task::(_debug) Task 146a0e76-4d15-4451-89b4-b4bd4ddf67a0: moving from state init -> state preparing
>Thread-15911417::DEBUG::2013-11-19 21:34:48,089::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911418::DEBUG::2013-11-19 21:34:48,097::task::495::TaskManager.Task::(_debug) Task 146a0e76-4d15-4451-89b4-b4bd4ddf67a0: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000545978546143', 'lastCheck': 1384896877.4420099, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00162196159363', 'lastCheck': 1384896882.2284839, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896878.1361749, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00449299812317', 'lastCheck': 1384896879.028841, 'valid': True, 'code': 0}}
>Thread-15911417::DEBUG::2013-11-19 21:34:48,097::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911418::DEBUG::2013-11-19 21:34:48,106::task::495::TaskManager.Task::(_debug) Task 146a0e76-4d15-4451-89b4-b4bd4ddf67a0: moving from state preparing -> state finished
>Thread-15911417::DEBUG::2013-11-19 21:34:48,110::task::495::TaskManager.Task::(_debug) Task 5b664412-99ef-423e-bdd2-c34bb4a26f29: ref 0 aborting False
>Thread-15911418::DEBUG::2013-11-19 21:34:48,110::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911417::INFO::2013-11-19 21:34:48,111::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911418::DEBUG::2013-11-19 21:34:48,113::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911418::DEBUG::2013-11-19 21:34:48,117::task::495::TaskManager.Task::(_debug) Task 146a0e76-4d15-4451-89b4-b4bd4ddf67a0: ref 0 aborting False
>Thread-15911418::INFO::2013-11-19 21:34:48,118::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000545978546143', 'lastCheck': 1384896877.4420099, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00162196159363', 'lastCheck': 1384896882.2284839, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896878.1361749, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00449299812317', 'lastCheck': 1384896879.028841, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:34:48,242::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-19 21:34:48,331::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15911420::DEBUG::2013-11-19 21:34:49,125::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911420::INFO::2013-11-19 21:34:49,132::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=44a1be5a-8538-4745-8f2c-b3e70865e4e7)
>Thread-15911420::DEBUG::2013-11-19 21:34:49,138::task::495::TaskManager.Task::(_debug) Task 2e2ce7dc-84b3-44b5-9caa-512848c7b094: moving from state init -> state preparing
>Thread-15911420::DEBUG::2013-11-19 21:34:49,142::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 44a1be5a-8538-4745-8f2c-b3e70865e4e7
>Thread-15911420::DEBUG::2013-11-19 21:34:49,145::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}
>Thread-15911420::DEBUG::2013-11-19 21:34:49,148::task::495::TaskManager.Task::(_debug) Task 2e2ce7dc-84b3-44b5-9caa-512848c7b094: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911420::DEBUG::2013-11-19 21:34:49,152::task::495::TaskManager.Task::(_debug) Task 2e2ce7dc-84b3-44b5-9caa-512848c7b094: moving from state preparing -> state finished
>Thread-15911420::DEBUG::2013-11-19 21:34:49,155::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911420::DEBUG::2013-11-19 21:34:49,159::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911420::DEBUG::2013-11-19 21:34:49,159::task::495::TaskManager.Task::(_debug) Task 2e2ce7dc-84b3-44b5-9caa-512848c7b094: ref 0 aborting False
>Thread-15911420::INFO::2013-11-19 21:34:49,160::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911421::DEBUG::2013-11-19 21:34:50,180::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911421::INFO::2013-11-19 21:34:50,184::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=44a1be5a-8538-4745-8f2c-b3e70865e4e7)
>Thread-15911421::DEBUG::2013-11-19 21:34:50,188::task::495::TaskManager.Task::(_debug) Task 0c59133d-6af2-4fff-9e88-196d2f31d71b: moving from state init -> state preparing
>Thread-15911421::DEBUG::2013-11-19 21:34:50,194::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 44a1be5a-8538-4745-8f2c-b3e70865e4e7
>Thread-15911421::DEBUG::2013-11-19 21:34:50,194::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}
>Thread-15911421::DEBUG::2013-11-19 21:34:50,200::task::495::TaskManager.Task::(_debug) Task 0c59133d-6af2-4fff-9e88-196d2f31d71b: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911421::DEBUG::2013-11-19 21:34:50,201::task::495::TaskManager.Task::(_debug) Task 0c59133d-6af2-4fff-9e88-196d2f31d71b: moving from state preparing -> state finished
>Thread-15911421::DEBUG::2013-11-19 21:34:50,205::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911421::DEBUG::2013-11-19 21:34:50,205::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911421::DEBUG::2013-11-19 21:34:50,211::task::495::TaskManager.Task::(_debug) Task 0c59133d-6af2-4fff-9e88-196d2f31d71b: ref 0 aborting False
>Thread-15911421::INFO::2013-11-19 21:34:50,216::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911423::DEBUG::2013-11-19 21:34:51,229::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911423::INFO::2013-11-19 21:34:51,230::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=44a1be5a-8538-4745-8f2c-b3e70865e4e7)
>Thread-15911423::DEBUG::2013-11-19 21:34:51,231::task::495::TaskManager.Task::(_debug) Task 9966fa7d-6086-4367-80cb-ca783f1488fa: moving from state init -> state preparing
>Thread-15911423::DEBUG::2013-11-19 21:34:51,234::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 44a1be5a-8538-4745-8f2c-b3e70865e4e7
>Thread-15911423::DEBUG::2013-11-19 21:34:51,239::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}
>Thread-15911423::DEBUG::2013-11-19 21:34:51,244::task::495::TaskManager.Task::(_debug) Task 9966fa7d-6086-4367-80cb-ca783f1488fa: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911423::DEBUG::2013-11-19 21:34:51,244::task::495::TaskManager.Task::(_debug) Task 9966fa7d-6086-4367-80cb-ca783f1488fa: moving from state preparing -> state finished
>Thread-15911423::DEBUG::2013-11-19 21:34:51,247::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911423::DEBUG::2013-11-19 21:34:51,253::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911423::DEBUG::2013-11-19 21:34:51,256::task::495::TaskManager.Task::(_debug) Task 9966fa7d-6086-4367-80cb-ca783f1488fa: ref 0 aborting False
>Thread-15911423::INFO::2013-11-19 21:34:51,263::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911424::DEBUG::2013-11-19 21:34:52,574::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911424::INFO::2013-11-19 21:34:52,576::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=44a1be5a-8538-4745-8f2c-b3e70865e4e7)
>Thread-15911424::DEBUG::2013-11-19 21:34:52,586::task::495::TaskManager.Task::(_debug) Task 987662e6-d2ad-46d5-89cc-96c9825fef0f: moving from state init -> state preparing
>Thread-15911424::DEBUG::2013-11-19 21:34:52,592::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 44a1be5a-8538-4745-8f2c-b3e70865e4e7
>Thread-15911424::DEBUG::2013-11-19 21:34:52,595::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}
>Thread-15911424::DEBUG::2013-11-19 21:34:52,600::task::495::TaskManager.Task::(_debug) Task 987662e6-d2ad-46d5-89cc-96c9825fef0f: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911424::DEBUG::2013-11-19 21:34:52,603::task::495::TaskManager.Task::(_debug) Task 987662e6-d2ad-46d5-89cc-96c9825fef0f: moving from state preparing -> state finished
>Thread-15911424::DEBUG::2013-11-19 21:34:52,606::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911424::DEBUG::2013-11-19 21:34:52,609::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911424::DEBUG::2013-11-19 21:34:52,616::task::495::TaskManager.Task::(_debug) Task 987662e6-d2ad-46d5-89cc-96c9825fef0f: ref 0 aborting False
>Thread-15911424::INFO::2013-11-19 21:34:52,622::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911426::DEBUG::2013-11-19 21:34:53,639::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911426::INFO::2013-11-19 21:34:53,646::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=44a1be5a-8538-4745-8f2c-b3e70865e4e7)
>Thread-15911426::DEBUG::2013-11-19 21:34:53,648::task::495::TaskManager.Task::(_debug) Task 64439d5a-03fd-420a-a9aa-d6d829380eb8: moving from state init -> state preparing
>Thread-15911426::DEBUG::2013-11-19 21:34:53,654::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 44a1be5a-8538-4745-8f2c-b3e70865e4e7
>Thread-15911426::DEBUG::2013-11-19 21:34:53,655::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}
>Thread-15911426::DEBUG::2013-11-19 21:34:53,661::task::495::TaskManager.Task::(_debug) Task 64439d5a-03fd-420a-a9aa-d6d829380eb8: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911426::DEBUG::2013-11-19 21:34:53,661::task::495::TaskManager.Task::(_debug) Task 64439d5a-03fd-420a-a9aa-d6d829380eb8: moving from state preparing -> state finished
>Thread-15911426::DEBUG::2013-11-19 21:34:53,666::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911426::DEBUG::2013-11-19 21:34:53,667::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911426::DEBUG::2013-11-19 21:34:53,674::task::495::TaskManager.Task::(_debug) Task 64439d5a-03fd-420a-a9aa-d6d829380eb8: ref 0 aborting False
>Thread-15911426::INFO::2013-11-19 21:34:53,678::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911428::DEBUG::2013-11-19 21:34:54,697::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911428::INFO::2013-11-19 21:34:54,701::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=44a1be5a-8538-4745-8f2c-b3e70865e4e7)
>Thread-15911428::DEBUG::2013-11-19 21:34:54,704::task::495::TaskManager.Task::(_debug) Task 4e20e2d5-a1b4-4e91-91b1-c3380541d604: moving from state init -> state preparing
>Thread-15911428::DEBUG::2013-11-19 21:34:54,708::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 44a1be5a-8538-4745-8f2c-b3e70865e4e7
>Thread-15911428::DEBUG::2013-11-19 21:34:54,709::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}
>Thread-15911428::DEBUG::2013-11-19 21:34:54,715::task::495::TaskManager.Task::(_debug) Task 4e20e2d5-a1b4-4e91-91b1-c3380541d604: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911428::DEBUG::2013-11-19 21:34:54,720::task::495::TaskManager.Task::(_debug) Task 4e20e2d5-a1b4-4e91-91b1-c3380541d604: moving from state preparing -> state finished
>Thread-15911428::DEBUG::2013-11-19 21:34:54,725::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911428::DEBUG::2013-11-19 21:34:54,731::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911428::DEBUG::2013-11-19 21:34:54,731::task::495::TaskManager.Task::(_debug) Task 4e20e2d5-a1b4-4e91-91b1-c3380541d604: ref 0 aborting False
>Thread-15911428::INFO::2013-11-19 21:34:54,737::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911429::DEBUG::2013-11-19 21:34:55,753::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911429::INFO::2013-11-19 21:34:55,754::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=44a1be5a-8538-4745-8f2c-b3e70865e4e7)
>Thread-15911429::DEBUG::2013-11-19 21:34:55,758::task::495::TaskManager.Task::(_debug) Task de5fd53d-2c16-4223-b696-d1cba209d2ca: moving from state init -> state preparing
>Thread-15911429::DEBUG::2013-11-19 21:34:55,759::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 44a1be5a-8538-4745-8f2c-b3e70865e4e7
>Thread-15911429::DEBUG::2013-11-19 21:34:55,762::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}
>Thread-15911429::DEBUG::2013-11-19 21:34:55,764::task::495::TaskManager.Task::(_debug) Task de5fd53d-2c16-4223-b696-d1cba209d2ca: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911429::DEBUG::2013-11-19 21:34:55,768::task::495::TaskManager.Task::(_debug) Task de5fd53d-2c16-4223-b696-d1cba209d2ca: moving from state preparing -> state finished
>Thread-15911429::DEBUG::2013-11-19 21:34:55,768::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911429::DEBUG::2013-11-19 21:34:55,774::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911429::DEBUG::2013-11-19 21:34:55,781::task::495::TaskManager.Task::(_debug) Task de5fd53d-2c16-4223-b696-d1cba209d2ca: ref 0 aborting False
>Thread-15911429::INFO::2013-11-19 21:34:55,781::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911431::DEBUG::2013-11-19 21:34:56,795::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911431::INFO::2013-11-19 21:34:56,796::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=44a1be5a-8538-4745-8f2c-b3e70865e4e7)
>Thread-15911431::DEBUG::2013-11-19 21:34:56,799::task::495::TaskManager.Task::(_debug) Task 049ed4f2-198e-42a4-a3d5-d705ce864fe8: moving from state init -> state preparing
>Thread-15911431::DEBUG::2013-11-19 21:34:56,800::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 44a1be5a-8538-4745-8f2c-b3e70865e4e7
>Thread-15911431::DEBUG::2013-11-19 21:34:56,800::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}
>Thread-15911431::DEBUG::2013-11-19 21:34:56,803::task::495::TaskManager.Task::(_debug) Task 049ed4f2-198e-42a4-a3d5-d705ce864fe8: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911431::DEBUG::2013-11-19 21:34:56,806::task::495::TaskManager.Task::(_debug) Task 049ed4f2-198e-42a4-a3d5-d705ce864fe8: moving from state preparing -> state finished
>Thread-15911431::DEBUG::2013-11-19 21:34:56,811::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911431::DEBUG::2013-11-19 21:34:56,812::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911431::DEBUG::2013-11-19 21:34:56,815::task::495::TaskManager.Task::(_debug) Task 049ed4f2-198e-42a4-a3d5-d705ce864fe8: ref 0 aborting False
>Thread-15911431::INFO::2013-11-19 21:34:56,817::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911432::DEBUG::2013-11-19 21:34:57,837::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911432::INFO::2013-11-19 21:34:57,843::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=44a1be5a-8538-4745-8f2c-b3e70865e4e7)
>Thread-15911432::DEBUG::2013-11-19 21:34:57,852::task::495::TaskManager.Task::(_debug) Task c7985d5e-c173-489a-9176-6bf5b58bc0c6: moving from state init -> state preparing
>Thread-15911432::DEBUG::2013-11-19 21:34:57,855::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 44a1be5a-8538-4745-8f2c-b3e70865e4e7
>Thread-15911432::DEBUG::2013-11-19 21:34:57,861::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}
>Thread-15911432::DEBUG::2013-11-19 21:34:57,867::task::495::TaskManager.Task::(_debug) Task c7985d5e-c173-489a-9176-6bf5b58bc0c6: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911432::DEBUG::2013-11-19 21:34:57,868::task::495::TaskManager.Task::(_debug) Task c7985d5e-c173-489a-9176-6bf5b58bc0c6: moving from state preparing -> state finished
>Thread-15911432::DEBUG::2013-11-19 21:34:57,871::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911432::DEBUG::2013-11-19 21:34:57,875::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911432::DEBUG::2013-11-19 21:34:57,884::task::495::TaskManager.Task::(_debug) Task c7985d5e-c173-489a-9176-6bf5b58bc0c6: ref 0 aborting False
>Thread-15911432::INFO::2013-11-19 21:34:57,887::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-1024767::ERROR::2013-11-19 21:34:58,255::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911433::INFO::2013-11-19 21:34:58,633::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911433::DEBUG::2013-11-19 21:34:58,641::task::495::TaskManager.Task::(_debug) Task 914ff2a9-4653-485a-bdb3-852b5d15688e: moving from state init -> state preparing
>Thread-15911433::DEBUG::2013-11-19 21:34:58,642::task::495::TaskManager.Task::(_debug) Task 914ff2a9-4653-485a-bdb3-852b5d15688e: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00032901763916', 'lastCheck': 1384896898.346746, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00251817703247', 'lastCheck': 1384896892.2490489, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896898.256459, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000525951385498', 'lastCheck': 1384896889.2436521, 'valid': True, 'code': 0}}
>Thread-15911433::DEBUG::2013-11-19 21:34:58,651::task::495::TaskManager.Task::(_debug) Task 914ff2a9-4653-485a-bdb3-852b5d15688e: moving from state preparing -> state finished
>Thread-15911433::DEBUG::2013-11-19 21:34:58,661::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911433::DEBUG::2013-11-19 21:34:58,670::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911433::DEBUG::2013-11-19 21:34:58,676::task::495::TaskManager.Task::(_debug) Task 914ff2a9-4653-485a-bdb3-852b5d15688e: ref 0 aborting False
>Thread-15911433::INFO::2013-11-19 21:34:58,682::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00032901763916', 'lastCheck': 1384896898.346746, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00251817703247', 'lastCheck': 1384896892.2490489, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896898.256459, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000525951385498', 'lastCheck': 1384896889.2436521, 'valid': True, 'code': 0}}
>Thread-15911435::DEBUG::2013-11-19 21:34:58,908::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911435::INFO::2013-11-19 21:34:58,909::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=44a1be5a-8538-4745-8f2c-b3e70865e4e7)
>Thread-15911435::DEBUG::2013-11-19 21:34:58,915::task::495::TaskManager.Task::(_debug) Task d86eddd2-4092-45d9-8a96-c8fd6371b116: moving from state init -> state preparing
>Thread-15911435::DEBUG::2013-11-19 21:34:58,919::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 44a1be5a-8538-4745-8f2c-b3e70865e4e7
>Thread-15911435::DEBUG::2013-11-19 21:34:58,925::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}
>Thread-15911435::DEBUG::2013-11-19 21:34:58,930::task::495::TaskManager.Task::(_debug) Task d86eddd2-4092-45d9-8a96-c8fd6371b116: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911435::DEBUG::2013-11-19 21:34:58,930::task::495::TaskManager.Task::(_debug) Task d86eddd2-4092-45d9-8a96-c8fd6371b116: moving from state preparing -> state finished
>Thread-15911435::DEBUG::2013-11-19 21:34:58,931::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911435::DEBUG::2013-11-19 21:34:58,941::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911435::DEBUG::2013-11-19 21:34:58,941::task::495::TaskManager.Task::(_debug) Task d86eddd2-4092-45d9-8a96-c8fd6371b116: ref 0 aborting False
>Thread-15911435::INFO::2013-11-19 21:34:58,945::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911436::DEBUG::2013-11-19 21:34:59,963::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911436::INFO::2013-11-19 21:34:59,967::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=44a1be5a-8538-4745-8f2c-b3e70865e4e7)
>Thread-15911436::DEBUG::2013-11-19 21:34:59,968::task::495::TaskManager.Task::(_debug) Task 42efc547-9c33-4b9a-854f-682cdf7a486c: moving from state init -> state preparing
>Thread-15911436::DEBUG::2013-11-19 21:34:59,971::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 44a1be5a-8538-4745-8f2c-b3e70865e4e7
>Thread-15911436::DEBUG::2013-11-19 21:34:59,972::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}
>Thread-15911436::DEBUG::2013-11-19 21:34:59,972::task::495::TaskManager.Task::(_debug) Task 42efc547-9c33-4b9a-854f-682cdf7a486c: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911436::DEBUG::2013-11-19 21:34:59,974::task::495::TaskManager.Task::(_debug) Task 42efc547-9c33-4b9a-854f-682cdf7a486c: moving from state preparing -> state finished
>Thread-15911436::DEBUG::2013-11-19 21:34:59,975::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911436::DEBUG::2013-11-19 21:34:59,978::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911436::DEBUG::2013-11-19 21:34:59,978::task::495::TaskManager.Task::(_debug) Task 42efc547-9c33-4b9a-854f-682cdf7a486c: ref 0 aborting False
>Thread-15911436::INFO::2013-11-19 21:34:59,979::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911438::DEBUG::2013-11-19 21:35:00,998::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911438::INFO::2013-11-19 21:35:01,003::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=44a1be5a-8538-4745-8f2c-b3e70865e4e7)
>Thread-15911438::DEBUG::2013-11-19 21:35:01,010::task::495::TaskManager.Task::(_debug) Task 02ff22b0-6da5-4a96-bf7c-fdc1fc4faec1: moving from state init -> state preparing
>Thread-15911438::DEBUG::2013-11-19 21:35:01,013::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 44a1be5a-8538-4745-8f2c-b3e70865e4e7
>Thread-15911438::DEBUG::2013-11-19 21:35:01,018::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}
>Thread-15911438::DEBUG::2013-11-19 21:35:01,021::task::495::TaskManager.Task::(_debug) Task 02ff22b0-6da5-4a96-bf7c-fdc1fc4faec1: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911438::DEBUG::2013-11-19 21:35:01,026::task::495::TaskManager.Task::(_debug) Task 02ff22b0-6da5-4a96-bf7c-fdc1fc4faec1: moving from state preparing -> state finished
>Thread-15911438::DEBUG::2013-11-19 21:35:01,029::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911438::DEBUG::2013-11-19 21:35:01,036::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911438::DEBUG::2013-11-19 21:35:01,041::task::495::TaskManager.Task::(_debug) Task 02ff22b0-6da5-4a96-bf7c-fdc1fc4faec1: ref 0 aborting False
>Thread-15911438::INFO::2013-11-19 21:35:01,044::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911439::DEBUG::2013-11-19 21:35:02,069::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911439::INFO::2013-11-19 21:35:02,076::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=44a1be5a-8538-4745-8f2c-b3e70865e4e7)
>Thread-15911439::DEBUG::2013-11-19 21:35:02,081::task::495::TaskManager.Task::(_debug) Task 3abc0e92-9957-4ebc-b4e6-3275f7f32f6d: moving from state init -> state preparing
>Thread-15911439::DEBUG::2013-11-19 21:35:02,085::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 44a1be5a-8538-4745-8f2c-b3e70865e4e7
>Thread-15911439::DEBUG::2013-11-19 21:35:02,090::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}
>Thread-15911439::DEBUG::2013-11-19 21:35:02,092::task::495::TaskManager.Task::(_debug) Task 3abc0e92-9957-4ebc-b4e6-3275f7f32f6d: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911439::DEBUG::2013-11-19 21:35:02,093::task::495::TaskManager.Task::(_debug) Task 3abc0e92-9957-4ebc-b4e6-3275f7f32f6d: moving from state preparing -> state finished
>Thread-15911439::DEBUG::2013-11-19 21:35:02,098::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911439::DEBUG::2013-11-19 21:35:02,103::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911439::DEBUG::2013-11-19 21:35:02,109::task::495::TaskManager.Task::(_debug) Task 3abc0e92-9957-4ebc-b4e6-3275f7f32f6d: ref 0 aborting False
>Thread-15911439::INFO::2013-11-19 21:35:02,111::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911441::DEBUG::2013-11-19 21:35:03,127::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911441::INFO::2013-11-19 21:35:03,131::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=44a1be5a-8538-4745-8f2c-b3e70865e4e7)
>Thread-15911441::DEBUG::2013-11-19 21:35:03,134::task::495::TaskManager.Task::(_debug) Task d9a39074-3189-4555-9a89-d0b8bf0e6a9e: moving from state init -> state preparing
>Thread-15911441::DEBUG::2013-11-19 21:35:03,139::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 44a1be5a-8538-4745-8f2c-b3e70865e4e7
>Thread-15911441::DEBUG::2013-11-19 21:35:03,140::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}
>Thread-15911441::DEBUG::2013-11-19 21:35:03,144::task::495::TaskManager.Task::(_debug) Task d9a39074-3189-4555-9a89-d0b8bf0e6a9e: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911441::DEBUG::2013-11-19 21:35:03,149::task::495::TaskManager.Task::(_debug) Task d9a39074-3189-4555-9a89-d0b8bf0e6a9e: moving from state preparing -> state finished
>Thread-15911441::DEBUG::2013-11-19 21:35:03,149::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911441::DEBUG::2013-11-19 21:35:03,158::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911441::DEBUG::2013-11-19 21:35:03,159::task::495::TaskManager.Task::(_debug) Task d9a39074-3189-4555-9a89-d0b8bf0e6a9e: ref 0 aborting False
>Thread-15911441::INFO::2013-11-19 21:35:03,170::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911442::DEBUG::2013-11-19 21:35:04,184::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911442::INFO::2013-11-19 21:35:04,185::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=44a1be5a-8538-4745-8f2c-b3e70865e4e7)
>Thread-15911442::DEBUG::2013-11-19 21:35:04,193::task::495::TaskManager.Task::(_debug) Task 4379b735-bf62-4a24-9707-0e83bff324fc: moving from state init -> state preparing
>Thread-15911442::DEBUG::2013-11-19 21:35:04,198::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 44a1be5a-8538-4745-8f2c-b3e70865e4e7
>Thread-15911442::DEBUG::2013-11-19 21:35:04,203::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}
>Thread-15911442::DEBUG::2013-11-19 21:35:04,207::task::495::TaskManager.Task::(_debug) Task 4379b735-bf62-4a24-9707-0e83bff324fc: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911442::DEBUG::2013-11-19 21:35:04,214::task::495::TaskManager.Task::(_debug) Task 4379b735-bf62-4a24-9707-0e83bff324fc: moving from state preparing -> state finished
>Thread-15911442::DEBUG::2013-11-19 21:35:04,215::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911442::DEBUG::2013-11-19 21:35:04,218::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911442::DEBUG::2013-11-19 21:35:04,218::task::495::TaskManager.Task::(_debug) Task 4379b735-bf62-4a24-9707-0e83bff324fc: ref 0 aborting False
>Thread-15911442::INFO::2013-11-19 21:35:04,221::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911444::DEBUG::2013-11-19 21:35:05,231::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911444::INFO::2013-11-19 21:35:05,233::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=44a1be5a-8538-4745-8f2c-b3e70865e4e7)
>Thread-15911444::DEBUG::2013-11-19 21:35:05,234::task::495::TaskManager.Task::(_debug) Task 1d28f981-ab63-4aef-b873-d498cafd6c66: moving from state init -> state preparing
>Thread-15911444::DEBUG::2013-11-19 21:35:05,234::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 44a1be5a-8538-4745-8f2c-b3e70865e4e7
>Thread-15911444::DEBUG::2013-11-19 21:35:05,235::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}
>Thread-15911444::DEBUG::2013-11-19 21:35:05,239::task::495::TaskManager.Task::(_debug) Task 1d28f981-ab63-4aef-b873-d498cafd6c66: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911444::DEBUG::2013-11-19 21:35:05,248::task::495::TaskManager.Task::(_debug) Task 1d28f981-ab63-4aef-b873-d498cafd6c66: moving from state preparing -> state finished
>Thread-15911444::DEBUG::2013-11-19 21:35:05,249::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911444::DEBUG::2013-11-19 21:35:05,255::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911444::DEBUG::2013-11-19 21:35:05,257::task::495::TaskManager.Task::(_debug) Task 1d28f981-ab63-4aef-b873-d498cafd6c66: ref 0 aborting False
>Thread-15911444::INFO::2013-11-19 21:35:05,258::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911445::DEBUG::2013-11-19 21:35:06,277::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911445::INFO::2013-11-19 21:35:06,285::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=44a1be5a-8538-4745-8f2c-b3e70865e4e7)
>Thread-15911445::DEBUG::2013-11-19 21:35:06,290::task::495::TaskManager.Task::(_debug) Task 47f3b5eb-1f8c-4a6f-a715-6cf0fd684bde: moving from state init -> state preparing
>Thread-15911445::DEBUG::2013-11-19 21:35:06,291::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 44a1be5a-8538-4745-8f2c-b3e70865e4e7
>Thread-15911445::DEBUG::2013-11-19 21:35:06,291::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}
>Thread-15911445::DEBUG::2013-11-19 21:35:06,292::task::495::TaskManager.Task::(_debug) Task 47f3b5eb-1f8c-4a6f-a715-6cf0fd684bde: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911445::DEBUG::2013-11-19 21:35:06,292::task::495::TaskManager.Task::(_debug) Task 47f3b5eb-1f8c-4a6f-a715-6cf0fd684bde: moving from state preparing -> state finished
>Thread-15911445::DEBUG::2013-11-19 21:35:06,293::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911445::DEBUG::2013-11-19 21:35:06,299::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911445::DEBUG::2013-11-19 21:35:06,305::task::495::TaskManager.Task::(_debug) Task 47f3b5eb-1f8c-4a6f-a715-6cf0fd684bde: ref 0 aborting False
>Thread-15911445::INFO::2013-11-19 21:35:06,311::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911447::DEBUG::2013-11-19 21:35:07,327::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911447::INFO::2013-11-19 21:35:07,337::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=44a1be5a-8538-4745-8f2c-b3e70865e4e7)
>Thread-15911447::DEBUG::2013-11-19 21:35:07,341::task::495::TaskManager.Task::(_debug) Task 1afaac2d-d76b-4122-bf5a-1f787e836fe4: moving from state init -> state preparing
>Thread-15911447::DEBUG::2013-11-19 21:35:07,349::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 44a1be5a-8538-4745-8f2c-b3e70865e4e7
>Thread-15911447::DEBUG::2013-11-19 21:35:07,352::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}
>Thread-15911447::DEBUG::2013-11-19 21:35:07,358::task::495::TaskManager.Task::(_debug) Task 1afaac2d-d76b-4122-bf5a-1f787e836fe4: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911447::DEBUG::2013-11-19 21:35:07,359::task::495::TaskManager.Task::(_debug) Task 1afaac2d-d76b-4122-bf5a-1f787e836fe4: moving from state preparing -> state finished
>Thread-15911447::DEBUG::2013-11-19 21:35:07,364::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911447::DEBUG::2013-11-19 21:35:07,368::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911447::DEBUG::2013-11-19 21:35:07,373::task::495::TaskManager.Task::(_debug) Task 1afaac2d-d76b-4122-bf5a-1f787e836fe4: ref 0 aborting False
>Thread-15911447::INFO::2013-11-19 21:35:07,379::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:07,908::safelease::61::Storage.Misc.excCmd::(acquire) SUCCESS: <err> = ''; <rc> = 0
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:07,908::safelease::65::ClusterLock::(acquire) Clustered lock acquired successfully
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:07,909::spm::468::Storage.SPM::(start) spm lock acquired successfully
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:07,910::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:07,910::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:07,911::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:07,912::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:07,912::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:07,913::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-1024767::ERROR::2013-11-19 21:35:08,279::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-19 21:35:08,358::lvm::439::OperationMutex::(_reloadvgs) Got the operational mutex
>Thread-42838::DEBUG::2013-11-19 21:35:08,359::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15911448::DEBUG::2013-11-19 21:35:08,562::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911448::INFO::2013-11-19 21:35:08,568::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=44a1be5a-8538-4745-8f2c-b3e70865e4e7)
>Thread-15911448::DEBUG::2013-11-19 21:35:08,571::task::495::TaskManager.Task::(_debug) Task d14c31b7-efaf-474a-8bd4-35148ea36dbb: moving from state init -> state preparing
>Thread-15911448::DEBUG::2013-11-19 21:35:08,575::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 44a1be5a-8538-4745-8f2c-b3e70865e4e7
>Thread-15911448::DEBUG::2013-11-19 21:35:08,579::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}
>Thread-15911448::DEBUG::2013-11-19 21:35:08,585::task::495::TaskManager.Task::(_debug) Task d14c31b7-efaf-474a-8bd4-35148ea36dbb: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:08,586::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15911448::DEBUG::2013-11-19 21:35:08,588::task::495::TaskManager.Task::(_debug) Task d14c31b7-efaf-474a-8bd4-35148ea36dbb: moving from state preparing -> state finished
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:08,592::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_LVER=284', u'_SHA_CKSUM=fa1ddc0ebaad4a3d815671fdef2d27b7d9f8a82e', u'POOL_SPM_ID=-1']
>Thread-15911448::DEBUG::2013-11-19 21:35:08,596::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:08,597::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
>Thread-15911448::DEBUG::2013-11-19 21:35:08,599::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:08,600::persistentDict::146::Storage.PersistentDict::(transaction) Flushing changes
>Thread-15911448::DEBUG::2013-11-19 21:35:08,602::task::495::TaskManager.Task::(_debug) Task d14c31b7-efaf-474a-8bd4-35148ea36dbb: ref 0 aborting False
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:08,603::persistentDict::265::Storage.PersistentDict::(flush) about to write lines (VGTagMetadataRW)=[u'CLASS=Data', u'DESCRIPTION=OS', u'IOOPTIMEOUTSEC=10', u'LEASERETRIES=3', u'LEASETIMESEC=60', u'LOCKPOLICY=', u'LOCKRENEWALINTERVALSEC=5', u'LOGBLKSIZE=512', u'MASTER_VERSION=1', u'PHYBLKSIZE=512', u'POOL_DESCRIPTION=Default', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_ID=2', u'POOL_SPM_LVER=285', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'ROLE=Master', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'TYPE=FCP', u'VERSION=2', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', '_SHA_CKSUM=901e51f2c0f3aefcf8d2e501fb4543e2352ea1bc']
>Thread-15911448::INFO::2013-11-19 21:35:08,606::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:08,607::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm reload operation' is holding the operation mutex, waiting...
>Thread-15911449::INFO::2013-11-19 21:35:08,872::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911449::DEBUG::2013-11-19 21:35:08,880::task::495::TaskManager.Task::(_debug) Task c2ba812e-d618-429b-90fb-150f9af4b999: moving from state init -> state preparing
>Thread-15911449::DEBUG::2013-11-19 21:35:08,887::task::495::TaskManager.Task::(_debug) Task c2ba812e-d618-429b-90fb-150f9af4b999: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00032901763916', 'lastCheck': 1384896898.346746, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00134897232056', 'lastCheck': 1384896902.2645841, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896908.2812121, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00050687789917', 'lastCheck': 1384896899.254674, 'valid': True, 'code': 0}}
>Thread-15911449::DEBUG::2013-11-19 21:35:08,888::task::495::TaskManager.Task::(_debug) Task c2ba812e-d618-429b-90fb-150f9af4b999: moving from state preparing -> state finished
>Thread-15911449::DEBUG::2013-11-19 21:35:08,898::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911449::DEBUG::2013-11-19 21:35:08,903::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911449::DEBUG::2013-11-19 21:35:08,912::task::495::TaskManager.Task::(_debug) Task c2ba812e-d618-429b-90fb-150f9af4b999: ref 0 aborting False
>Thread-15911449::INFO::2013-11-19 21:35:08,917::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00032901763916', 'lastCheck': 1384896898.346746, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00134897232056', 'lastCheck': 1384896902.2645841, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896908.2812121, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00050687789917', 'lastCheck': 1384896899.254674, 'valid': True, 'code': 0}}
>Thread-42838::DEBUG::2013-11-19 21:35:09,045::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-42838::DEBUG::2013-11-19 21:35:09,048::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:09,049::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:09,050::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:09,050::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:09,051::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:09,052::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:09,053::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15911451::DEBUG::2013-11-19 21:35:09,620::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911451::INFO::2013-11-19 21:35:09,624::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=44a1be5a-8538-4745-8f2c-b3e70865e4e7)
>Thread-15911451::DEBUG::2013-11-19 21:35:09,625::task::495::TaskManager.Task::(_debug) Task eea33e2c-d6c2-4dd1-8227-64975bdae578: moving from state init -> state preparing
>Thread-15911451::DEBUG::2013-11-19 21:35:09,626::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 44a1be5a-8538-4745-8f2c-b3e70865e4e7
>Thread-15911451::DEBUG::2013-11-19 21:35:09,626::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}
>Thread-15911451::DEBUG::2013-11-19 21:35:09,636::task::495::TaskManager.Task::(_debug) Task eea33e2c-d6c2-4dd1-8227-64975bdae578: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911451::DEBUG::2013-11-19 21:35:09,643::task::495::TaskManager.Task::(_debug) Task eea33e2c-d6c2-4dd1-8227-64975bdae578: moving from state preparing -> state finished
>Thread-15911451::DEBUG::2013-11-19 21:35:09,643::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911451::DEBUG::2013-11-19 21:35:09,661::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911451::DEBUG::2013-11-19 21:35:09,674::task::495::TaskManager.Task::(_debug) Task eea33e2c-d6c2-4dd1-8227-64975bdae578: ref 0 aborting False
>Thread-15911451::INFO::2013-11-19 21:35:09,683::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:09,744::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:09,748::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:09,749::blockSD::145::storage.Metadata.VGTagMetadataRW::(writelines) Updating metadata adding=MDT__SHA_CKSUM=901e51f2c0f3aefcf8d2e501fb4543e2352ea1bc, MDT_POOL_SPM_LVER=285, MDT_POOL_SPM_ID=2 removing=MDT_POOL_SPM_ID=-1, MDT_POOL_SPM_LVER=284, MDT__SHA_CKSUM=fa1ddc0ebaad4a3d815671fdef2d27b7d9f8a82e
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:09,750::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --deltag MDT__SHA_CKSUM=fa1ddc0ebaad4a3d815671fdef2d27b7d9f8a82e --deltag MDT_POOL_SPM_ID=-1 --deltag MDT_POOL_SPM_LVER=284 --addtag MDT_POOL_SPM_ID=2 --addtag MDT__SHA_CKSUM=901e51f2c0f3aefcf8d2e501fb4543e2352ea1bc --addtag MDT_POOL_SPM_LVER=285 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:10,465::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:10,466::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:10,467::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:10,468::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:10,470::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`201777e5-cf9c-49e9-b517-96c0f26fafb4`::Request was made in '/usr/share/vdsm/storage/spm.py' line '330' at '_upgradePool'
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:10,472::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'exclusive'
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:10,473::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'exclusive' (1 active user)
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:10,474::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`201777e5-cf9c-49e9-b517-96c0f26fafb4`::Granted request
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::INFO::2013-11-19 21:35:10,475::spm::336::Storage.SPM::(_upgradePool) Trying to upgrade master domain `6b892067-ac7e-4039-a7be-5b7387525b16`
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:10,476::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3dc875ce-7dc3-43e9-b81c-7fc08564bce9`::Request was made in '/usr/share/vdsm/storage/spm.py' line '337' at '_upgradePool'
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:10,476::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'exclusive'
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:10,477::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'exclusive' (1 active user)
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:10,477::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3dc875ce-7dc3-43e9-b81c-7fc08564bce9`::Granted request
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:10,479::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:10,479::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:10,480::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:10,480::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:10,481::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:10,482::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15911452::DEBUG::2013-11-19 21:35:10,744::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911452::INFO::2013-11-19 21:35:10,748::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=44a1be5a-8538-4745-8f2c-b3e70865e4e7)
>Thread-15911452::DEBUG::2013-11-19 21:35:10,752::task::495::TaskManager.Task::(_debug) Task de0467ae-0023-4b90-860f-9ef088f0a445: moving from state init -> state preparing
>Thread-15911452::DEBUG::2013-11-19 21:35:10,759::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 44a1be5a-8538-4745-8f2c-b3e70865e4e7
>Thread-15911452::DEBUG::2013-11-19 21:35:10,764::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}
>Thread-15911452::DEBUG::2013-11-19 21:35:10,772::task::495::TaskManager.Task::(_debug) Task de0467ae-0023-4b90-860f-9ef088f0a445: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911452::DEBUG::2013-11-19 21:35:10,773::task::495::TaskManager.Task::(_debug) Task de0467ae-0023-4b90-860f-9ef088f0a445: moving from state preparing -> state finished
>Thread-15911452::DEBUG::2013-11-19 21:35:10,779::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911452::DEBUG::2013-11-19 21:35:10,783::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911452::DEBUG::2013-11-19 21:35:10,790::task::495::TaskManager.Task::(_debug) Task de0467ae-0023-4b90-860f-9ef088f0a445: ref 0 aborting False
>Thread-15911452::INFO::2013-11-19 21:35:10,795::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:11,087::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:11,090::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:11,091::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_ID=2', u'_SHA_CKSUM=901e51f2c0f3aefcf8d2e501fb4543e2352ea1bc', u'POOL_SPM_LVER=285']
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:11,092::blockSD::625::Storage.StorageDomain::(upgrade) Trying to upgrade domain `6b892067-ac7e-4039-a7be-5b7387525b16` from version 2 to version 2
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:11,093::blockSD::631::Storage.StorageDomain::(upgrade) No need to upgrade domain `6b892067-ac7e-4039-a7be-5b7387525b16`, leaving unchanged
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:11,094::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:11,094::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:11,095::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:11,095::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:11,096::spm::340::Storage.SPM::(_upgradePool) Marking all domains for upgrade
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:11,097::spm::347::Storage.SPM::(_upgradePool) Registering with state change event
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:11,097::spm::349::Storage.SPM::(_upgradePool) Running initial domain upgrade threads
>Thread-15911453::DEBUG::2013-11-19 21:35:11,098::spm::365::Storage.SPM::(_upgradePoolDomain) Preparing to upgrade domain 45cdff8b-89c0-4926-a8c7-80354064bfa6
>Thread-15911454::DEBUG::2013-11-19 21:35:11,102::spm::365::Storage.SPM::(_upgradePoolDomain) Preparing to upgrade domain 772472cb-2429-466d-96cd-00b3301ef78a
>Thread-15911455::DEBUG::2013-11-19 21:35:11,104::spm::365::Storage.SPM::(_upgradePoolDomain) Preparing to upgrade domain 5d7b5178-3790-472c-a13b-b481eb27a61e
>Thread-15911453::DEBUG::2013-11-19 21:35:11,105::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6`ReqID=`4b3764fe-2219-4c03-b8db-6ecd3c6d5eb3`::Request was made in '/usr/share/vdsm/storage/spm.py' line '376' at '_upgradePoolDomain'
>Thread-15911454::DEBUG::2013-11-19 21:35:11,106::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`96bae22c-bbc3-4c3e-a656-96b2523b87c8`::Request was made in '/usr/share/vdsm/storage/spm.py' line '376' at '_upgradePoolDomain'
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:11,107::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911455::ERROR::2013-11-19 21:35:11,114::spm::373::Storage.SPM::(_upgradePoolDomain) Error while preparing domain `5d7b5178-3790-472c-a13b-b481eb27a61e` upgrade
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/spm.py", line 369, in _upgradePoolDomain
>IndexError: list index out of range
>Thread-15911453::DEBUG::2013-11-19 21:35:11,117::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6' for lock type 'exclusive'
>Thread-15911454::DEBUG::2013-11-19 21:35:11,118::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'exclusive'
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:11,119::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:11,134::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:11,135::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911453::DEBUG::2013-11-19 21:35:11,136::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6' is free. Now locking as 'exclusive' (1 active user)
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:11,137::blockSD::791::Storage.Misc.excCmd::(mountMaster) '/usr/bin/sudo -n /sbin/e2fsck -p /dev/6b892067-ac7e-4039-a7be-5b7387525b16/master' (cwd None)
>Thread-15911453::DEBUG::2013-11-19 21:35:11,142::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6`ReqID=`4b3764fe-2219-4c03-b8db-6ecd3c6d5eb3`::Granted request
>Thread-15911454::DEBUG::2013-11-19 21:35:11,376::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'exclusive' (1 active user)
>Thread-15911454::DEBUG::2013-11-19 21:35:11,378::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`96bae22c-bbc3-4c3e-a656-96b2523b87c8`::Granted request
>Thread-15911453::DEBUG::2013-11-19 21:35:11,381::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6`ReqID=`d5a88e0f-dd3d-4ceb-be98-d5389402a101`::Request was made in '/usr/share/vdsm/storage/spm.py' line '377' at '_upgradePoolDomain'
>Thread-15911453::DEBUG::2013-11-19 21:35:11,387::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6' for lock type 'exclusive'
>Thread-15911454::DEBUG::2013-11-19 21:35:11,388::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`1c5c7d84-8041-4da3-9920-6644cbc6aac4`::Request was made in '/usr/share/vdsm/storage/spm.py' line '377' at '_upgradePoolDomain'
>Thread-15911453::DEBUG::2013-11-19 21:35:11,401::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6' is free. Now locking as 'exclusive' (1 active user)
>Thread-15911454::DEBUG::2013-11-19 21:35:11,401::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'exclusive'
>Thread-15911453::DEBUG::2013-11-19 21:35:11,404::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6`ReqID=`d5a88e0f-dd3d-4ceb-be98-d5389402a101`::Granted request
>Thread-15911454::DEBUG::2013-11-19 21:35:11,407::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'exclusive' (1 active user)
>Thread-15911453::DEBUG::2013-11-19 21:35:11,407::spm::385::Storage.SPM::(_upgradePoolDomain) Domain `45cdff8b-89c0-4926-a8c7-80354064bfa6` is not a data domain it is an 2 domain, not upgrading
>Thread-15911454::DEBUG::2013-11-19 21:35:11,408::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`1c5c7d84-8041-4da3-9920-6644cbc6aac4`::Granted request
>Thread-15911453::DEBUG::2013-11-19 21:35:11,412::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6'
>Thread-15911454::DEBUG::2013-11-19 21:35:11,418::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15911453::DEBUG::2013-11-19 21:35:11,423::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6' (0 active users)
>Thread-15911454::DEBUG::2013-11-19 21:35:11,424::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15911453::DEBUG::2013-11-19 21:35:11,428::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6' is free, finding out if anyone is waiting for it.
>Thread-15911454::DEBUG::2013-11-19 21:35:11,429::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15911453::DEBUG::2013-11-19 21:35:11,434::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6', Clearing records.
>Thread-15911454::DEBUG::2013-11-19 21:35:11,435::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15911454::DEBUG::2013-11-19 21:35:11,439::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-15911453::DEBUG::2013-11-19 21:35:11,439::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6'
>Thread-15911454::DEBUG::2013-11-19 21:35:11,440::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-15911453::DEBUG::2013-11-19 21:35:11,444::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6' (0 active users)
>Thread-15911453::DEBUG::2013-11-19 21:35:11,705::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6' is free, finding out if anyone is waiting for it.
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:11,706::blockSD::791::Storage.Misc.excCmd::(mountMaster) SUCCESS: <err> = ''; <rc> = 0
>Thread-15911453::DEBUG::2013-11-19 21:35:11,709::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6', Clearing records.
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:11,710::blockSD::816::Storage.Misc.excCmd::(mountMaster) '/usr/bin/sudo -n /sbin/tune2fs -j /dev/6b892067-ac7e-4039-a7be-5b7387525b16/master' (cwd None)
>Thread-15911457::DEBUG::2013-11-19 21:35:12,082::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911457::INFO::2013-11-19 21:35:12,096::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=44a1be5a-8538-4745-8f2c-b3e70865e4e7)
>Thread-15911457::DEBUG::2013-11-19 21:35:12,097::task::495::TaskManager.Task::(_debug) Task 35d88432-def4-4052-90fb-6ca028e2f45c: moving from state init -> state preparing
>Thread-15911457::DEBUG::2013-11-19 21:35:12,101::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 44a1be5a-8538-4745-8f2c-b3e70865e4e7
>Thread-15911457::DEBUG::2013-11-19 21:35:12,103::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}
>Thread-15911457::DEBUG::2013-11-19 21:35:12,104::task::495::TaskManager.Task::(_debug) Task 35d88432-def4-4052-90fb-6ca028e2f45c: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911457::DEBUG::2013-11-19 21:35:12,108::task::495::TaskManager.Task::(_debug) Task 35d88432-def4-4052-90fb-6ca028e2f45c: moving from state preparing -> state finished
>Thread-15911457::DEBUG::2013-11-19 21:35:12,109::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911457::DEBUG::2013-11-19 21:35:12,109::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911457::DEBUG::2013-11-19 21:35:12,110::task::495::TaskManager.Task::(_debug) Task 35d88432-def4-4052-90fb-6ca028e2f45c: ref 0 aborting False
>Thread-15911457::INFO::2013-11-19 21:35:12,118::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:12,227::blockSD::816::Storage.Misc.excCmd::(mountMaster) FAILED: <err> = 'The filesystem already has a journal.\n'; <rc> = 1
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:12,230::fileUtils::83::Storage.Misc.excCmd::(mount) '/usr/bin/sudo -n /bin/mount /dev/6b892067-ac7e-4039-a7be-5b7387525b16/master /rhev/data-center/mnt/blockSD/6b892067-ac7e-4039-a7be-5b7387525b16/master' (cwd None)
>Thread-15911454::DEBUG::2013-11-19 21:35:12,435::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15911454::DEBUG::2013-11-19 21:35:12,442::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-15911454::DEBUG::2013-11-19 21:35:12,444::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'SDUUID=772472cb-2429-466d-96cd-00b3301ef78a', u'VERSION=2', u'LOCKPOLICY=', u'VGUUID=XTb294-NYv6-dVpO-bB1n-4f9P-d0GL-dhd2DI', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'LEASETIMESEC=5', u'LOCKRENEWALINTERVALSEC=5', u'ROLE=Regular', u'PHYBLKSIZE=512', u'DESCRIPTION=Database', u'IOOPTIMEOUTSEC=1', u'CLASS=Data', u'TYPE=FCP', u'PV0=pv:3600c0ff0001520c7414f605001000000,uuid:NujM33-vNiD-sBHY-mrPo-EQ2f-1qsz-HXfn0q,pestart:0,pecount:8929,mapoffset:0', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'MASTER_VERSION=0', u'_SHA_CKSUM=ef8840d9fdef24516a56fd970bed1960d2b1ae92']
>Thread-15911454::DEBUG::2013-11-19 21:35:12,446::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15911454::DEBUG::2013-11-19 21:35:12,447::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15911454::DEBUG::2013-11-19 21:35:12,448::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15911454::DEBUG::2013-11-19 21:35:12,448::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15911454::DEBUG::2013-11-19 21:35:12,449::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15911454::DEBUG::2013-11-19 21:35:12,450::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15911454::DEBUG::2013-11-19 21:35:12,451::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15911454::DEBUG::2013-11-19 21:35:12,451::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:12,641::fileUtils::83::Storage.Misc.excCmd::(mount) SUCCESS: <err> = ''; <rc> = 0
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:12,642::blockSD::833::Storage.Misc.excCmd::(mountMaster) '/usr/bin/sudo -n /bin/chown vdsm:kvm /rhev/data-center/mnt/blockSD/6b892067-ac7e-4039-a7be-5b7387525b16/master' (cwd None)
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:13,042::blockSD::833::Storage.Misc.excCmd::(mountMaster) SUCCESS: <err> = ''; <rc> = 0
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::WARNING::2013-11-19 21:35:13,045::spm::490::Storage.SPM::(start) Backup domain validation failed
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/spm.py", line 488, in start
>  File "/usr/share/vdsm/storage/sp.py", line 1208, in checkBackupDomain
>  File "/usr/share/vdsm/storage/sd.py", line 666, in isBackup
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:13,046::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:13,047::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:13,047::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:13,048::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:13,048::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:13,049::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:13,050::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:13,050::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:13,051::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:13,051::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:13,052::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:13,052::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:13,053::threadPool::25::Misc.ThreadPool::(__init__) Enter - numThreads: 5.0, waitTimeout: 3, maxTasks: 500.0
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:13,067::storage_mailbox::486::Storage.MailBox.SpmMailMonitor::(__init__) SPM_MailMonitor - clearing outgoing mail, command is: ['dd', 'of=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/outbox', 'oflag=direct', 'iflag=fullblock', 'conv=notrunc', 'count=1']
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:13,068::storage_mailbox::488::Storage.Misc.excCmd::(__init__) 'dd of=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/outbox oflag=direct iflag=fullblock conv=notrunc count=1 bs=1024000' (cwd None)
>Thread-15911463::DEBUG::2013-11-19 21:35:13,285::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911463::INFO::2013-11-19 21:35:13,288::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=44a1be5a-8538-4745-8f2c-b3e70865e4e7)
>Thread-15911463::DEBUG::2013-11-19 21:35:13,289::task::495::TaskManager.Task::(_debug) Task 42424bc7-7227-47f7-ba8a-dc8d765a8620: moving from state init -> state preparing
>Thread-15911463::DEBUG::2013-11-19 21:35:13,289::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 44a1be5a-8538-4745-8f2c-b3e70865e4e7
>Thread-15911463::DEBUG::2013-11-19 21:35:13,290::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}
>Thread-15911463::DEBUG::2013-11-19 21:35:13,290::task::495::TaskManager.Task::(_debug) Task 42424bc7-7227-47f7-ba8a-dc8d765a8620: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911463::DEBUG::2013-11-19 21:35:13,291::task::495::TaskManager.Task::(_debug) Task 42424bc7-7227-47f7-ba8a-dc8d765a8620: moving from state preparing -> state finished
>Thread-15911463::DEBUG::2013-11-19 21:35:13,292::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911463::DEBUG::2013-11-19 21:35:13,292::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911463::DEBUG::2013-11-19 21:35:13,293::task::495::TaskManager.Task::(_debug) Task 42424bc7-7227-47f7-ba8a-dc8d765a8620: ref 0 aborting False
>Thread-15911463::INFO::2013-11-19 21:35:13,293::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:13,510::storage_mailbox::488::Storage.Misc.excCmd::(__init__) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.040803 s, 25.1 MB/s\n'; <rc> = 0
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:13,511::storage_mailbox::493::Storage.MailBox.SpmMailMonitor::(__init__) SPM_MailMonitor created for pool c4e5889a-348d-11e2-83fe-9f72369eab30
>Dummy-15911465::DEBUG::2013-11-19 21:35:13,511::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:13,513::spm::523::Storage.SPM::(start) ended.
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:13,515::task::495::TaskManager.Task::(_debug) Task 44a1be5a-8538-4745-8f2c-b3e70865e4e7: moving from state running -> state finished
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:13,796::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:13,809::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:13,817::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:13,818::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:13,818::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:13,819::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:13,820::task::915::TaskManager.Task::(_runJobs) Task.run: exit - success: result 
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:13,820::task::495::TaskManager.Task::(_debug) Task 44a1be5a-8538-4745-8f2c-b3e70865e4e7: ref 0 aborting False
>44a1be5a-8538-4745-8f2c-b3e70865e4e7::DEBUG::2013-11-19 21:35:13,821::threadPool::47::Misc.ThreadPool::(setRunningTask) Number of running tasks: 0
>Dummy-15911465::DEBUG::2013-11-19 21:35:14,071::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0707385 s, 14.5 MB/s\n'; <rc> = 0
>Thread-15911466::DEBUG::2013-11-19 21:35:14,305::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911466::INFO::2013-11-19 21:35:14,306::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=44a1be5a-8538-4745-8f2c-b3e70865e4e7)
>Thread-15911466::DEBUG::2013-11-19 21:35:14,307::task::495::TaskManager.Task::(_debug) Task e3c2bb0e-ec58-4702-969c-0c72482fb8b0: moving from state init -> state preparing
>Thread-15911466::DEBUG::2013-11-19 21:35:14,308::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 44a1be5a-8538-4745-8f2c-b3e70865e4e7
>Thread-15911466::DEBUG::2013-11-19 21:35:14,308::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': '1 jobs completed successfuly', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}
>Thread-15911466::DEBUG::2013-11-19 21:35:14,309::task::495::TaskManager.Task::(_debug) Task e3c2bb0e-ec58-4702-969c-0c72482fb8b0: finished: {'taskStatus': {'code': 0, 'message': '1 jobs completed successfuly', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911466::DEBUG::2013-11-19 21:35:14,310::task::495::TaskManager.Task::(_debug) Task e3c2bb0e-ec58-4702-969c-0c72482fb8b0: moving from state preparing -> state finished
>Thread-15911466::DEBUG::2013-11-19 21:35:14,310::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911466::DEBUG::2013-11-19 21:35:14,311::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911466::DEBUG::2013-11-19 21:35:14,311::task::495::TaskManager.Task::(_debug) Task e3c2bb0e-ec58-4702-969c-0c72482fb8b0: ref 0 aborting False
>Thread-15911466::INFO::2013-11-19 21:35:14,312::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': '1 jobs completed successfuly', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '44a1be5a-8538-4745-8f2c-b3e70865e4e7'}}
>Thread-15911467::DEBUG::2013-11-19 21:35:14,319::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911467::INFO::2013-11-19 21:35:14,320::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911467::DEBUG::2013-11-19 21:35:14,320::task::495::TaskManager.Task::(_debug) Task 99c637b6-9c6c-46a4-8b2a-1c37dac491c0: moving from state init -> state preparing
>Thread-15911467::DEBUG::2013-11-19 21:35:14,321::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=285 spmId=2
>Thread-15911467::DEBUG::2013-11-19 21:35:14,322::task::495::TaskManager.Task::(_debug) Task 99c637b6-9c6c-46a4-8b2a-1c37dac491c0: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911467::DEBUG::2013-11-19 21:35:14,322::task::495::TaskManager.Task::(_debug) Task 99c637b6-9c6c-46a4-8b2a-1c37dac491c0: moving from state preparing -> state finished
>Thread-15911467::DEBUG::2013-11-19 21:35:14,323::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911467::DEBUG::2013-11-19 21:35:14,323::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911467::DEBUG::2013-11-19 21:35:14,324::task::495::TaskManager.Task::(_debug) Task 99c637b6-9c6c-46a4-8b2a-1c37dac491c0: ref 0 aborting False
>Thread-15911467::INFO::2013-11-19 21:35:14,324::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911468::DEBUG::2013-11-19 21:35:14,340::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911468::INFO::2013-11-19 21:35:14,341::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: clearTask, args: ( taskID=44a1be5a-8538-4745-8f2c-b3e70865e4e7)
>Thread-15911468::DEBUG::2013-11-19 21:35:14,341::task::495::TaskManager.Task::(_debug) Task a9abba42-2b11-4e0c-b0ae-da44dd8049ea: moving from state init -> state preparing
>Thread-15911468::DEBUG::2013-11-19 21:35:14,342::taskManager::146::TaskManager::(clearTask) Entry. taskID: 44a1be5a-8538-4745-8f2c-b3e70865e4e7
>Thread-15911468::DEBUG::2013-11-19 21:35:14,343::taskManager::151::TaskManager::(clearTask) Return.
>Thread-15911468::DEBUG::2013-11-19 21:35:14,343::task::495::TaskManager.Task::(_debug) Task a9abba42-2b11-4e0c-b0ae-da44dd8049ea: finished: None
>Thread-15911468::DEBUG::2013-11-19 21:35:14,344::task::495::TaskManager.Task::(_debug) Task a9abba42-2b11-4e0c-b0ae-da44dd8049ea: moving from state preparing -> state finished
>Thread-15911468::DEBUG::2013-11-19 21:35:14,344::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911468::DEBUG::2013-11-19 21:35:14,345::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911468::DEBUG::2013-11-19 21:35:14,345::task::495::TaskManager.Task::(_debug) Task a9abba42-2b11-4e0c-b0ae-da44dd8049ea: ref 0 aborting False
>Thread-15911468::INFO::2013-11-19 21:35:14,346::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: clearTask, Return response: {'status': {'message': 'OK', 'code': 0}}
>Thread-15911469::DEBUG::2013-11-19 21:35:14,411::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911469::INFO::2013-11-19 21:35:14,412::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911469::DEBUG::2013-11-19 21:35:14,413::task::495::TaskManager.Task::(_debug) Task 2bff2eef-c59b-4069-b260-a7877b83e086: moving from state init -> state preparing
>Thread-15911469::DEBUG::2013-11-19 21:35:14,414::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3ec94a27-f0a9-4c60-b2a8-f2051c6b2363`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911469::DEBUG::2013-11-19 21:35:14,415::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911469::DEBUG::2013-11-19 21:35:14,415::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911469::DEBUG::2013-11-19 21:35:14,416::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3ec94a27-f0a9-4c60-b2a8-f2051c6b2363`::Granted request
>Thread-15911469::DEBUG::2013-11-19 21:35:14,417::task::495::TaskManager.Task::(_debug) Task 2bff2eef-c59b-4069-b260-a7877b83e086: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911469::DEBUG::2013-11-19 21:35:14,417::task::495::TaskManager.Task::(_debug) Task 2bff2eef-c59b-4069-b260-a7877b83e086: ref 1 aborting False
>Thread-15911469::WARNING::2013-11-19 21:35:14,419::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911469::DEBUG::2013-11-19 21:35:14,420::task::495::TaskManager.Task::(_debug) Task 2bff2eef-c59b-4069-b260-a7877b83e086: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911469::DEBUG::2013-11-19 21:35:14,421::task::495::TaskManager.Task::(_debug) Task 2bff2eef-c59b-4069-b260-a7877b83e086: moving from state preparing -> state finished
>Thread-15911469::DEBUG::2013-11-19 21:35:14,422::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911469::DEBUG::2013-11-19 21:35:14,423::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911469::DEBUG::2013-11-19 21:35:14,423::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911469::DEBUG::2013-11-19 21:35:14,424::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911469::DEBUG::2013-11-19 21:35:14,424::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911469::DEBUG::2013-11-19 21:35:14,425::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911469::DEBUG::2013-11-19 21:35:14,426::task::495::TaskManager.Task::(_debug) Task 2bff2eef-c59b-4069-b260-a7877b83e086: ref 0 aborting False
>Thread-15911469::INFO::2013-11-19 21:35:14,426::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911470::DEBUG::2013-11-19 21:35:14,586::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911470::INFO::2013-11-19 21:35:14,587::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksInfo, args: ()
>Thread-15911470::DEBUG::2013-11-19 21:35:14,587::task::495::TaskManager.Task::(_debug) Task 9b58b6ed-7cb0-4fa3-aada-fa3c28e1dd12: moving from state init -> state preparing
>Thread-15911470::DEBUG::2013-11-19 21:35:14,588::taskManager::168::TaskManager::(getAllTasksInfo) Entry.
>Thread-15911470::DEBUG::2013-11-19 21:35:14,588::taskManager::173::TaskManager::(getAllTasksInfo) Return. Response: {}
>Thread-15911470::DEBUG::2013-11-19 21:35:14,589::task::495::TaskManager.Task::(_debug) Task 9b58b6ed-7cb0-4fa3-aada-fa3c28e1dd12: finished: {'allTasksInfo': {}}
>Thread-15911470::DEBUG::2013-11-19 21:35:14,589::task::495::TaskManager.Task::(_debug) Task 9b58b6ed-7cb0-4fa3-aada-fa3c28e1dd12: moving from state preparing -> state finished
>Thread-15911470::DEBUG::2013-11-19 21:35:14,590::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911470::DEBUG::2013-11-19 21:35:14,590::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911470::DEBUG::2013-11-19 21:35:14,591::task::495::TaskManager.Task::(_debug) Task 9b58b6ed-7cb0-4fa3-aada-fa3c28e1dd12: ref 0 aborting False
>Thread-15911470::INFO::2013-11-19 21:35:14,592::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'allTasksInfo': {}}
>Thread-15911471::DEBUG::2013-11-19 21:35:14,612::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911471::INFO::2013-11-19 21:35:14,612::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksInfo, args: ()
>Thread-15911471::DEBUG::2013-11-19 21:35:14,613::task::495::TaskManager.Task::(_debug) Task b20c9d99-8954-451c-b44d-ef3e60f29934: moving from state init -> state preparing
>Thread-15911471::DEBUG::2013-11-19 21:35:14,614::taskManager::168::TaskManager::(getAllTasksInfo) Entry.
>Thread-15911471::DEBUG::2013-11-19 21:35:14,614::taskManager::173::TaskManager::(getAllTasksInfo) Return. Response: {}
>Thread-15911471::DEBUG::2013-11-19 21:35:14,615::task::495::TaskManager.Task::(_debug) Task b20c9d99-8954-451c-b44d-ef3e60f29934: finished: {'allTasksInfo': {}}
>Thread-15911471::DEBUG::2013-11-19 21:35:14,615::task::495::TaskManager.Task::(_debug) Task b20c9d99-8954-451c-b44d-ef3e60f29934: moving from state preparing -> state finished
>Thread-15911471::DEBUG::2013-11-19 21:35:14,616::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911471::DEBUG::2013-11-19 21:35:14,616::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911471::DEBUG::2013-11-19 21:35:14,617::task::495::TaskManager.Task::(_debug) Task b20c9d99-8954-451c-b44d-ef3e60f29934: ref 0 aborting False
>Thread-15911471::INFO::2013-11-19 21:35:14,617::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'allTasksInfo': {}}
>Dummy-15911465::DEBUG::2013-11-19 21:35:16,086::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:35:16,611::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.070185 s, 14.6 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:35:18,438::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911465::DEBUG::2013-11-19 21:35:18,626::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:35:19,141::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0724285 s, 14.1 MB/s\n'; <rc> = 0
>Thread-15911474::INFO::2013-11-19 21:35:19,426::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911474::DEBUG::2013-11-19 21:35:19,427::task::495::TaskManager.Task::(_debug) Task a740853c-738c-4a8b-b3de-2df5d7ceb3fa: moving from state init -> state preparing
>Thread-15911474::WARNING::2013-11-19 21:35:19,428::hsm::1895::Storage.HSM::(public_repoStats) repoStats detected invalid master:6b892067-ac7e-4039-a7be-5b7387525b16 {'finish': 1384896910.473666, 'mdasize': '134479872', 'mdathreshold': True, 'mdavalid': True, 'diskfree': '1142595518464', 'disktotal': '3595827150848', 'result': {'delay': '0.000699043273926', 'lastCheck': 1384896910.473666, 'valid': True, 'code': 0}, 'masterValidate': {'mount': False, 'valid': True}, 'mdafree': '67205632'}
>Thread-15911474::DEBUG::2013-11-19 21:35:19,429::task::495::TaskManager.Task::(_debug) Task a740853c-738c-4a8b-b3de-2df5d7ceb3fa: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000699043273926', 'lastCheck': 1384896910.473666, 'valid': True, 'code': 355}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00635194778442', 'lastCheck': 1384896912.444699, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896918.438978, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00066089630127', 'lastCheck': 1384896909.266923, 'valid': True, 'code': 0}}
>Thread-15911474::DEBUG::2013-11-19 21:35:19,439::task::495::TaskManager.Task::(_debug) Task a740853c-738c-4a8b-b3de-2df5d7ceb3fa: moving from state preparing -> state finished
>Thread-15911474::DEBUG::2013-11-19 21:35:19,440::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911474::DEBUG::2013-11-19 21:35:19,441::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911474::DEBUG::2013-11-19 21:35:19,441::task::495::TaskManager.Task::(_debug) Task a740853c-738c-4a8b-b3de-2df5d7ceb3fa: ref 0 aborting False
>Thread-15911474::INFO::2013-11-19 21:35:19,442::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000699043273926', 'lastCheck': 1384896910.473666, 'valid': True, 'code': 355}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00635194778442', 'lastCheck': 1384896912.444699, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896918.438978, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00066089630127', 'lastCheck': 1384896909.266923, 'valid': True, 'code': 0}}
>Dummy-15911465::DEBUG::2013-11-19 21:35:21,157::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:35:21,689::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.070086 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15911465::DEBUG::2013-11-19 21:35:23,704::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:35:24,235::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0710186 s, 14.4 MB/s\n'; <rc> = 0
>Thread-15911478::DEBUG::2013-11-19 21:35:24,583::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911478::INFO::2013-11-19 21:35:24,584::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911478::DEBUG::2013-11-19 21:35:24,585::task::495::TaskManager.Task::(_debug) Task 1bc4d5a1-55ec-431a-a4ef-b2b4008bf67f: moving from state init -> state preparing
>Thread-15911478::DEBUG::2013-11-19 21:35:24,586::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=285 spmId=2
>Thread-15911478::DEBUG::2013-11-19 21:35:24,587::task::495::TaskManager.Task::(_debug) Task 1bc4d5a1-55ec-431a-a4ef-b2b4008bf67f: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911478::DEBUG::2013-11-19 21:35:24,587::task::495::TaskManager.Task::(_debug) Task 1bc4d5a1-55ec-431a-a4ef-b2b4008bf67f: moving from state preparing -> state finished
>Thread-15911478::DEBUG::2013-11-19 21:35:24,588::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911478::DEBUG::2013-11-19 21:35:24,588::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911478::DEBUG::2013-11-19 21:35:24,589::task::495::TaskManager.Task::(_debug) Task 1bc4d5a1-55ec-431a-a4ef-b2b4008bf67f: ref 0 aborting False
>Thread-15911478::INFO::2013-11-19 21:35:24,590::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911479::DEBUG::2013-11-19 21:35:24,596::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911479::INFO::2013-11-19 21:35:24,597::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911479::DEBUG::2013-11-19 21:35:24,597::task::495::TaskManager.Task::(_debug) Task ad2e8349-7f14-4d9f-a511-08e4c53f2592: moving from state init -> state preparing
>Thread-15911479::DEBUG::2013-11-19 21:35:24,598::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e721a359-5a68-4f71-be55-a46892b361f3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911479::DEBUG::2013-11-19 21:35:24,599::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911479::DEBUG::2013-11-19 21:35:24,600::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911479::DEBUG::2013-11-19 21:35:24,600::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e721a359-5a68-4f71-be55-a46892b361f3`::Granted request
>Thread-15911479::DEBUG::2013-11-19 21:35:24,602::task::495::TaskManager.Task::(_debug) Task ad2e8349-7f14-4d9f-a511-08e4c53f2592: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911479::DEBUG::2013-11-19 21:35:24,602::task::495::TaskManager.Task::(_debug) Task ad2e8349-7f14-4d9f-a511-08e4c53f2592: ref 1 aborting False
>Thread-15911479::WARNING::2013-11-19 21:35:24,604::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911479::DEBUG::2013-11-19 21:35:24,605::task::495::TaskManager.Task::(_debug) Task ad2e8349-7f14-4d9f-a511-08e4c53f2592: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911479::DEBUG::2013-11-19 21:35:24,606::task::495::TaskManager.Task::(_debug) Task ad2e8349-7f14-4d9f-a511-08e4c53f2592: moving from state preparing -> state finished
>Thread-15911479::DEBUG::2013-11-19 21:35:24,607::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911479::DEBUG::2013-11-19 21:35:24,607::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911479::DEBUG::2013-11-19 21:35:24,608::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911479::DEBUG::2013-11-19 21:35:24,609::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911479::DEBUG::2013-11-19 21:35:24,609::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911479::DEBUG::2013-11-19 21:35:24,610::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911479::DEBUG::2013-11-19 21:35:24,610::task::495::TaskManager.Task::(_debug) Task ad2e8349-7f14-4d9f-a511-08e4c53f2592: ref 0 aborting False
>Thread-15911479::INFO::2013-11-19 21:35:24,611::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911465::DEBUG::2013-11-19 21:35:26,250::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:35:26,764::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0699634 s, 14.6 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:35:28,455::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911465::DEBUG::2013-11-19 21:35:28,780::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:35:29,316::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0700693 s, 14.6 MB/s\n'; <rc> = 0
>Thread-15911482::INFO::2013-11-19 21:35:29,577::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911482::DEBUG::2013-11-19 21:35:29,578::task::495::TaskManager.Task::(_debug) Task 4fb253b1-736e-4279-86dc-1b984d3cbb2f: moving from state init -> state preparing
>Thread-15911482::DEBUG::2013-11-19 21:35:29,579::task::495::TaskManager.Task::(_debug) Task 4fb253b1-736e-4279-86dc-1b984d3cbb2f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000339984893799', 'lastCheck': 1384896920.869468, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00151491165161', 'lastCheck': 1384896922.4618061, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896928.4563911, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000466108322144', 'lastCheck': 1384896929.457324, 'valid': True, 'code': 0}}
>Thread-15911482::DEBUG::2013-11-19 21:35:29,583::task::495::TaskManager.Task::(_debug) Task 4fb253b1-736e-4279-86dc-1b984d3cbb2f: moving from state preparing -> state finished
>Thread-15911482::DEBUG::2013-11-19 21:35:29,584::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911482::DEBUG::2013-11-19 21:35:29,584::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911482::DEBUG::2013-11-19 21:35:29,585::task::495::TaskManager.Task::(_debug) Task 4fb253b1-736e-4279-86dc-1b984d3cbb2f: ref 0 aborting False
>Thread-15911482::INFO::2013-11-19 21:35:29,587::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000339984893799', 'lastCheck': 1384896920.869468, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00151491165161', 'lastCheck': 1384896922.4618061, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896928.4563911, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000466108322144', 'lastCheck': 1384896929.457324, 'valid': True, 'code': 0}}
>Dummy-15911465::DEBUG::2013-11-19 21:35:31,331::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:35:31,868::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.079768 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15911465::DEBUG::2013-11-19 21:35:33,883::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:35:34,427::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0703569 s, 14.6 MB/s\n'; <rc> = 0
>Thread-15911486::DEBUG::2013-11-19 21:35:34,790::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911486::INFO::2013-11-19 21:35:34,791::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911486::DEBUG::2013-11-19 21:35:34,792::task::495::TaskManager.Task::(_debug) Task 35b5af62-61c6-4923-a86b-2ed001072389: moving from state init -> state preparing
>Thread-15911486::DEBUG::2013-11-19 21:35:34,793::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=285 spmId=2
>Thread-15911486::DEBUG::2013-11-19 21:35:34,793::task::495::TaskManager.Task::(_debug) Task 35b5af62-61c6-4923-a86b-2ed001072389: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911486::DEBUG::2013-11-19 21:35:34,794::task::495::TaskManager.Task::(_debug) Task 35b5af62-61c6-4923-a86b-2ed001072389: moving from state preparing -> state finished
>Thread-15911486::DEBUG::2013-11-19 21:35:34,794::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911486::DEBUG::2013-11-19 21:35:34,795::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911486::DEBUG::2013-11-19 21:35:34,796::task::495::TaskManager.Task::(_debug) Task 35b5af62-61c6-4923-a86b-2ed001072389: ref 0 aborting False
>Thread-15911486::INFO::2013-11-19 21:35:34,796::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911487::DEBUG::2013-11-19 21:35:34,802::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911487::INFO::2013-11-19 21:35:34,803::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911487::DEBUG::2013-11-19 21:35:34,804::task::495::TaskManager.Task::(_debug) Task 47fc1bfe-5aa0-4864-a2b2-1a344ff277c6: moving from state init -> state preparing
>Thread-15911487::DEBUG::2013-11-19 21:35:34,805::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`088623de-3cc9-4d3a-ba46-4c15cabf668d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911487::DEBUG::2013-11-19 21:35:34,806::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911487::DEBUG::2013-11-19 21:35:34,807::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911487::DEBUG::2013-11-19 21:35:34,807::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`088623de-3cc9-4d3a-ba46-4c15cabf668d`::Granted request
>Thread-15911487::DEBUG::2013-11-19 21:35:34,808::task::495::TaskManager.Task::(_debug) Task 47fc1bfe-5aa0-4864-a2b2-1a344ff277c6: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911487::DEBUG::2013-11-19 21:35:34,809::task::495::TaskManager.Task::(_debug) Task 47fc1bfe-5aa0-4864-a2b2-1a344ff277c6: ref 1 aborting False
>Thread-15911487::WARNING::2013-11-19 21:35:34,811::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911487::DEBUG::2013-11-19 21:35:34,812::task::495::TaskManager.Task::(_debug) Task 47fc1bfe-5aa0-4864-a2b2-1a344ff277c6: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911487::DEBUG::2013-11-19 21:35:34,813::task::495::TaskManager.Task::(_debug) Task 47fc1bfe-5aa0-4864-a2b2-1a344ff277c6: moving from state preparing -> state finished
>Thread-15911487::DEBUG::2013-11-19 21:35:34,813::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911487::DEBUG::2013-11-19 21:35:34,814::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911487::DEBUG::2013-11-19 21:35:34,815::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911487::DEBUG::2013-11-19 21:35:34,815::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911487::DEBUG::2013-11-19 21:35:34,816::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911487::DEBUG::2013-11-19 21:35:34,816::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911487::DEBUG::2013-11-19 21:35:34,817::task::495::TaskManager.Task::(_debug) Task 47fc1bfe-5aa0-4864-a2b2-1a344ff277c6: ref 0 aborting False
>Thread-15911487::INFO::2013-11-19 21:35:34,818::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911465::DEBUG::2013-11-19 21:35:36,443::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:35:36,967::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0704573 s, 14.5 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:35:38,659::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911465::DEBUG::2013-11-19 21:35:38,982::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:35:39,410::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0446349 s, 22.9 MB/s\n'; <rc> = 0
>Thread-15911490::INFO::2013-11-19 21:35:39,783::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911490::DEBUG::2013-11-19 21:35:39,784::task::495::TaskManager.Task::(_debug) Task d91b8791-1ae1-4d19-9edf-c1529e30af8e: moving from state init -> state preparing
>Thread-15911490::DEBUG::2013-11-19 21:35:39,794::task::495::TaskManager.Task::(_debug) Task d91b8791-1ae1-4d19-9edf-c1529e30af8e: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000402927398682', 'lastCheck': 1384896931.0225, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00147008895874', 'lastCheck': 1384896932.5518439, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896938.660331, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000509977340698', 'lastCheck': 1384896939.658987, 'valid': True, 'code': 0}}
>Thread-15911490::DEBUG::2013-11-19 21:35:39,804::task::495::TaskManager.Task::(_debug) Task d91b8791-1ae1-4d19-9edf-c1529e30af8e: moving from state preparing -> state finished
>Thread-15911490::DEBUG::2013-11-19 21:35:39,805::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911490::DEBUG::2013-11-19 21:35:39,806::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911490::DEBUG::2013-11-19 21:35:39,806::task::495::TaskManager.Task::(_debug) Task d91b8791-1ae1-4d19-9edf-c1529e30af8e: ref 0 aborting False
>Thread-15911490::INFO::2013-11-19 21:35:39,807::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000402927398682', 'lastCheck': 1384896931.0225, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00147008895874', 'lastCheck': 1384896932.5518439, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896938.660331, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000509977340698', 'lastCheck': 1384896939.658987, 'valid': True, 'code': 0}}
>Dummy-15911465::DEBUG::2013-11-19 21:35:41,420::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:35:41,961::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0746533 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15911465::DEBUG::2013-11-19 21:35:43,976::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:35:44,499::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0692762 s, 14.8 MB/s\n'; <rc> = 0
>Thread-15911494::DEBUG::2013-11-19 21:35:44,923::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911494::INFO::2013-11-19 21:35:44,924::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911494::DEBUG::2013-11-19 21:35:44,933::task::495::TaskManager.Task::(_debug) Task 8db61e71-e341-44eb-a2a9-880192e86fb0: moving from state init -> state preparing
>Thread-15911494::DEBUG::2013-11-19 21:35:44,934::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=285 spmId=2
>Thread-15911494::DEBUG::2013-11-19 21:35:44,934::task::495::TaskManager.Task::(_debug) Task 8db61e71-e341-44eb-a2a9-880192e86fb0: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911494::DEBUG::2013-11-19 21:35:44,935::task::495::TaskManager.Task::(_debug) Task 8db61e71-e341-44eb-a2a9-880192e86fb0: moving from state preparing -> state finished
>Thread-15911494::DEBUG::2013-11-19 21:35:44,936::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911494::DEBUG::2013-11-19 21:35:44,936::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911494::DEBUG::2013-11-19 21:35:44,937::task::495::TaskManager.Task::(_debug) Task 8db61e71-e341-44eb-a2a9-880192e86fb0: ref 0 aborting False
>Thread-15911494::INFO::2013-11-19 21:35:44,937::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911495::DEBUG::2013-11-19 21:35:44,947::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911495::INFO::2013-11-19 21:35:44,948::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911495::DEBUG::2013-11-19 21:35:44,949::task::495::TaskManager.Task::(_debug) Task f247bfd2-31ca-463a-99aa-840558a703e7: moving from state init -> state preparing
>Thread-15911495::DEBUG::2013-11-19 21:35:44,950::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5164ae1d-8348-4ad3-a550-646a1159a63e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911495::DEBUG::2013-11-19 21:35:44,951::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911495::DEBUG::2013-11-19 21:35:44,952::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911495::DEBUG::2013-11-19 21:35:44,952::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5164ae1d-8348-4ad3-a550-646a1159a63e`::Granted request
>Thread-15911495::DEBUG::2013-11-19 21:35:44,953::task::495::TaskManager.Task::(_debug) Task f247bfd2-31ca-463a-99aa-840558a703e7: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911495::DEBUG::2013-11-19 21:35:44,954::task::495::TaskManager.Task::(_debug) Task f247bfd2-31ca-463a-99aa-840558a703e7: ref 1 aborting False
>Thread-15911495::WARNING::2013-11-19 21:35:44,956::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911495::DEBUG::2013-11-19 21:35:44,957::task::495::TaskManager.Task::(_debug) Task f247bfd2-31ca-463a-99aa-840558a703e7: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911495::DEBUG::2013-11-19 21:35:44,968::task::495::TaskManager.Task::(_debug) Task f247bfd2-31ca-463a-99aa-840558a703e7: moving from state preparing -> state finished
>Thread-15911495::DEBUG::2013-11-19 21:35:44,969::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911495::DEBUG::2013-11-19 21:35:44,969::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911495::DEBUG::2013-11-19 21:35:44,970::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911495::DEBUG::2013-11-19 21:35:44,970::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911495::DEBUG::2013-11-19 21:35:44,971::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911495::DEBUG::2013-11-19 21:35:44,971::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911495::DEBUG::2013-11-19 21:35:44,972::task::495::TaskManager.Task::(_debug) Task f247bfd2-31ca-463a-99aa-840558a703e7: ref 0 aborting False
>Thread-15911495::INFO::2013-11-19 21:35:44,973::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911465::DEBUG::2013-11-19 21:35:46,515::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:35:47,033::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0692758 s, 14.8 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:35:48,740::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911465::DEBUG::2013-11-19 21:35:49,048::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:35:49,591::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0704397 s, 14.5 MB/s\n'; <rc> = 0
>Thread-15911498::INFO::2013-11-19 21:35:49,897::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911498::DEBUG::2013-11-19 21:35:49,898::task::495::TaskManager.Task::(_debug) Task 600ba0b8-e302-4f8d-b38e-bfb2f54c2188: moving from state init -> state preparing
>Thread-15911498::DEBUG::2013-11-19 21:35:49,898::task::495::TaskManager.Task::(_debug) Task 600ba0b8-e302-4f8d-b38e-bfb2f54c2188: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000434160232544', 'lastCheck': 1384896941.219434, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00144791603088', 'lastCheck': 1384896942.666486, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896948.7416871, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00051212310791', 'lastCheck': 1384896949.7396619, 'valid': True, 'code': 0}}
>Thread-15911498::DEBUG::2013-11-19 21:35:49,905::task::495::TaskManager.Task::(_debug) Task 600ba0b8-e302-4f8d-b38e-bfb2f54c2188: moving from state preparing -> state finished
>Thread-15911498::DEBUG::2013-11-19 21:35:49,906::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911498::DEBUG::2013-11-19 21:35:49,906::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911498::DEBUG::2013-11-19 21:35:49,907::task::495::TaskManager.Task::(_debug) Task 600ba0b8-e302-4f8d-b38e-bfb2f54c2188: ref 0 aborting False
>Thread-15911498::INFO::2013-11-19 21:35:49,908::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000434160232544', 'lastCheck': 1384896941.219434, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00144791603088', 'lastCheck': 1384896942.666486, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896948.7416871, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00051212310791', 'lastCheck': 1384896949.7396619, 'valid': True, 'code': 0}}
>Dummy-15911465::DEBUG::2013-11-19 21:35:51,607::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:35:52,124::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0696205 s, 14.7 MB/s\n'; <rc> = 0
>Dummy-15911465::DEBUG::2013-11-19 21:35:54,139::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:35:54,694::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0704281 s, 14.5 MB/s\n'; <rc> = 0
>Thread-15911502::DEBUG::2013-11-19 21:35:55,063::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911502::INFO::2013-11-19 21:35:55,064::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911502::DEBUG::2013-11-19 21:35:55,065::task::495::TaskManager.Task::(_debug) Task 8388d43d-6abf-490d-8748-ed7ab41d2ae4: moving from state init -> state preparing
>Thread-15911502::DEBUG::2013-11-19 21:35:55,066::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=285 spmId=2
>Thread-15911502::DEBUG::2013-11-19 21:35:55,066::task::495::TaskManager.Task::(_debug) Task 8388d43d-6abf-490d-8748-ed7ab41d2ae4: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911502::DEBUG::2013-11-19 21:35:55,067::task::495::TaskManager.Task::(_debug) Task 8388d43d-6abf-490d-8748-ed7ab41d2ae4: moving from state preparing -> state finished
>Thread-15911502::DEBUG::2013-11-19 21:35:55,067::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911502::DEBUG::2013-11-19 21:35:55,068::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911502::DEBUG::2013-11-19 21:35:55,068::task::495::TaskManager.Task::(_debug) Task 8388d43d-6abf-490d-8748-ed7ab41d2ae4: ref 0 aborting False
>Thread-15911502::INFO::2013-11-19 21:35:55,069::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911503::DEBUG::2013-11-19 21:35:55,080::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911503::INFO::2013-11-19 21:35:55,081::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911503::DEBUG::2013-11-19 21:35:55,081::task::495::TaskManager.Task::(_debug) Task cdd03187-c99c-42ee-9401-b17ff24811f6: moving from state init -> state preparing
>Thread-15911503::DEBUG::2013-11-19 21:35:55,083::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`904ae441-f0e3-42a6-9910-f9c00095af37`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911503::DEBUG::2013-11-19 21:35:55,083::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911503::DEBUG::2013-11-19 21:35:55,084::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911503::DEBUG::2013-11-19 21:35:55,085::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`904ae441-f0e3-42a6-9910-f9c00095af37`::Granted request
>Thread-15911503::DEBUG::2013-11-19 21:35:55,086::task::495::TaskManager.Task::(_debug) Task cdd03187-c99c-42ee-9401-b17ff24811f6: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911503::DEBUG::2013-11-19 21:35:55,086::task::495::TaskManager.Task::(_debug) Task cdd03187-c99c-42ee-9401-b17ff24811f6: ref 1 aborting False
>Thread-15911503::WARNING::2013-11-19 21:35:55,088::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911503::DEBUG::2013-11-19 21:35:55,089::task::495::TaskManager.Task::(_debug) Task cdd03187-c99c-42ee-9401-b17ff24811f6: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911503::DEBUG::2013-11-19 21:35:55,090::task::495::TaskManager.Task::(_debug) Task cdd03187-c99c-42ee-9401-b17ff24811f6: moving from state preparing -> state finished
>Thread-15911503::DEBUG::2013-11-19 21:35:55,091::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911503::DEBUG::2013-11-19 21:35:55,092::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911503::DEBUG::2013-11-19 21:35:55,092::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911503::DEBUG::2013-11-19 21:35:55,093::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911503::DEBUG::2013-11-19 21:35:55,093::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911503::DEBUG::2013-11-19 21:35:55,094::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911503::DEBUG::2013-11-19 21:35:55,095::task::495::TaskManager.Task::(_debug) Task cdd03187-c99c-42ee-9401-b17ff24811f6: ref 0 aborting False
>Thread-15911503::INFO::2013-11-19 21:35:55,098::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911465::DEBUG::2013-11-19 21:35:56,709::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:35:57,241::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0696661 s, 14.7 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:35:58,931::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911465::DEBUG::2013-11-19 21:35:59,255::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:35:59,812::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0702366 s, 14.6 MB/s\n'; <rc> = 0
>Thread-15911506::INFO::2013-11-19 21:36:00,015::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911506::DEBUG::2013-11-19 21:36:00,015::task::495::TaskManager.Task::(_debug) Task 9663cdd5-7fde-42ed-8761-51d99f720662: moving from state init -> state preparing
>Thread-15911506::DEBUG::2013-11-19 21:36:00,016::task::495::TaskManager.Task::(_debug) Task 9663cdd5-7fde-42ed-8761-51d99f720662: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000297069549561', 'lastCheck': 1384896951.276562, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00225305557251', 'lastCheck': 1384896952.821914, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896958.9328351, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000590085983276', 'lastCheck': 1384896959.9311881, 'valid': True, 'code': 0}}
>Thread-15911506::DEBUG::2013-11-19 21:36:00,022::task::495::TaskManager.Task::(_debug) Task 9663cdd5-7fde-42ed-8761-51d99f720662: moving from state preparing -> state finished
>Thread-15911506::DEBUG::2013-11-19 21:36:00,023::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911506::DEBUG::2013-11-19 21:36:00,024::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911506::DEBUG::2013-11-19 21:36:00,024::task::495::TaskManager.Task::(_debug) Task 9663cdd5-7fde-42ed-8761-51d99f720662: ref 0 aborting False
>Thread-15911506::INFO::2013-11-19 21:36:00,025::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000297069549561', 'lastCheck': 1384896951.276562, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00225305557251', 'lastCheck': 1384896952.821914, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896958.9328351, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000590085983276', 'lastCheck': 1384896959.9311881, 'valid': True, 'code': 0}}
>Dummy-15911465::DEBUG::2013-11-19 21:36:01,829::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:36:02,375::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0844066 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-15911465::DEBUG::2013-11-19 21:36:04,390::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:36:04,942::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0704773 s, 14.5 MB/s\n'; <rc> = 0
>Thread-15911510::DEBUG::2013-11-19 21:36:05,230::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911510::INFO::2013-11-19 21:36:05,237::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911510::DEBUG::2013-11-19 21:36:05,240::task::495::TaskManager.Task::(_debug) Task 35c27000-ee94-4725-8cf5-e8a2a80031bd: moving from state init -> state preparing
>Thread-15911510::DEBUG::2013-11-19 21:36:05,248::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=285 spmId=2
>Thread-15911510::DEBUG::2013-11-19 21:36:05,252::task::495::TaskManager.Task::(_debug) Task 35c27000-ee94-4725-8cf5-e8a2a80031bd: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911510::DEBUG::2013-11-19 21:36:05,255::task::495::TaskManager.Task::(_debug) Task 35c27000-ee94-4725-8cf5-e8a2a80031bd: moving from state preparing -> state finished
>Thread-15911510::DEBUG::2013-11-19 21:36:05,256::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911510::DEBUG::2013-11-19 21:36:05,257::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911510::DEBUG::2013-11-19 21:36:05,257::task::495::TaskManager.Task::(_debug) Task 35c27000-ee94-4725-8cf5-e8a2a80031bd: ref 0 aborting False
>Thread-15911510::INFO::2013-11-19 21:36:05,258::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911511::DEBUG::2013-11-19 21:36:05,267::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911511::INFO::2013-11-19 21:36:05,268::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911511::DEBUG::2013-11-19 21:36:05,268::task::495::TaskManager.Task::(_debug) Task 416aaf45-e0d6-468a-8212-68664d4e78c3: moving from state init -> state preparing
>Thread-15911511::DEBUG::2013-11-19 21:36:05,270::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2cf421fc-0ef0-41c2-8cc3-27f7f876e83d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911511::DEBUG::2013-11-19 21:36:05,270::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911511::DEBUG::2013-11-19 21:36:05,271::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911511::DEBUG::2013-11-19 21:36:05,272::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2cf421fc-0ef0-41c2-8cc3-27f7f876e83d`::Granted request
>Thread-15911511::DEBUG::2013-11-19 21:36:05,273::task::495::TaskManager.Task::(_debug) Task 416aaf45-e0d6-468a-8212-68664d4e78c3: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911511::DEBUG::2013-11-19 21:36:05,273::task::495::TaskManager.Task::(_debug) Task 416aaf45-e0d6-468a-8212-68664d4e78c3: ref 1 aborting False
>Thread-15911511::WARNING::2013-11-19 21:36:05,275::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911511::DEBUG::2013-11-19 21:36:05,276::task::495::TaskManager.Task::(_debug) Task 416aaf45-e0d6-468a-8212-68664d4e78c3: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911511::DEBUG::2013-11-19 21:36:05,277::task::495::TaskManager.Task::(_debug) Task 416aaf45-e0d6-468a-8212-68664d4e78c3: moving from state preparing -> state finished
>Thread-15911511::DEBUG::2013-11-19 21:36:05,278::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911511::DEBUG::2013-11-19 21:36:05,278::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911511::DEBUG::2013-11-19 21:36:05,279::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911511::DEBUG::2013-11-19 21:36:05,280::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911511::DEBUG::2013-11-19 21:36:05,280::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911511::DEBUG::2013-11-19 21:36:05,281::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911511::DEBUG::2013-11-19 21:36:05,281::task::495::TaskManager.Task::(_debug) Task 416aaf45-e0d6-468a-8212-68664d4e78c3: ref 0 aborting False
>Thread-15911511::INFO::2013-11-19 21:36:05,282::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911465::DEBUG::2013-11-19 21:36:06,958::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:36:07,486::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0703399 s, 14.6 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:36:09,175::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911465::DEBUG::2013-11-19 21:36:09,500::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:36:10,011::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0700202 s, 14.6 MB/s\n'; <rc> = 0
>Thread-15911514::INFO::2013-11-19 21:36:10,212::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911514::DEBUG::2013-11-19 21:36:10,212::task::495::TaskManager.Task::(_debug) Task a56927d7-7a63-435f-bf22-c5301270d369: moving from state init -> state preparing
>Thread-15911514::DEBUG::2013-11-19 21:36:10,213::task::495::TaskManager.Task::(_debug) Task a56927d7-7a63-435f-bf22-c5301270d369: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000431060791016', 'lastCheck': 1384896961.5056629, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00147986412048', 'lastCheck': 1384896963.049371, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896969.175921, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000556945800781', 'lastCheck': 1384896970.1740961, 'valid': True, 'code': 0}}
>Thread-15911514::DEBUG::2013-11-19 21:36:10,220::task::495::TaskManager.Task::(_debug) Task a56927d7-7a63-435f-bf22-c5301270d369: moving from state preparing -> state finished
>Thread-15911514::DEBUG::2013-11-19 21:36:10,221::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911514::DEBUG::2013-11-19 21:36:10,221::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911514::DEBUG::2013-11-19 21:36:10,222::task::495::TaskManager.Task::(_debug) Task a56927d7-7a63-435f-bf22-c5301270d369: ref 0 aborting False
>Thread-15911514::INFO::2013-11-19 21:36:10,223::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000431060791016', 'lastCheck': 1384896961.5056629, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00147986412048', 'lastCheck': 1384896963.049371, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896969.175921, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000556945800781', 'lastCheck': 1384896970.1740961, 'valid': True, 'code': 0}}
>Dummy-15911465::DEBUG::2013-11-19 21:36:12,026::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:36:12,584::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0709732 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15911465::DEBUG::2013-11-19 21:36:14,599::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:36:15,124::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0699282 s, 14.6 MB/s\n'; <rc> = 0
>Thread-15911518::DEBUG::2013-11-19 21:36:15,438::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911518::INFO::2013-11-19 21:36:15,442::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911518::DEBUG::2013-11-19 21:36:15,443::task::495::TaskManager.Task::(_debug) Task 45ab61d4-b262-400f-b3d4-0f0d74ed4b86: moving from state init -> state preparing
>Thread-15911518::DEBUG::2013-11-19 21:36:15,444::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=285 spmId=2
>Thread-15911518::DEBUG::2013-11-19 21:36:15,445::task::495::TaskManager.Task::(_debug) Task 45ab61d4-b262-400f-b3d4-0f0d74ed4b86: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911518::DEBUG::2013-11-19 21:36:15,445::task::495::TaskManager.Task::(_debug) Task 45ab61d4-b262-400f-b3d4-0f0d74ed4b86: moving from state preparing -> state finished
>Thread-15911518::DEBUG::2013-11-19 21:36:15,446::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911518::DEBUG::2013-11-19 21:36:15,446::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911518::DEBUG::2013-11-19 21:36:15,447::task::495::TaskManager.Task::(_debug) Task 45ab61d4-b262-400f-b3d4-0f0d74ed4b86: ref 0 aborting False
>Thread-15911518::INFO::2013-11-19 21:36:15,448::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911519::DEBUG::2013-11-19 21:36:15,454::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911519::INFO::2013-11-19 21:36:15,455::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911519::DEBUG::2013-11-19 21:36:15,455::task::495::TaskManager.Task::(_debug) Task c903ad96-62d4-4f92-82ab-74367775ebf1: moving from state init -> state preparing
>Thread-15911519::DEBUG::2013-11-19 21:36:15,457::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8275ee56-83e1-441a-b08a-f7671697f2a0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911519::DEBUG::2013-11-19 21:36:15,457::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911519::DEBUG::2013-11-19 21:36:15,458::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911519::DEBUG::2013-11-19 21:36:15,459::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8275ee56-83e1-441a-b08a-f7671697f2a0`::Granted request
>Thread-15911519::DEBUG::2013-11-19 21:36:15,460::task::495::TaskManager.Task::(_debug) Task c903ad96-62d4-4f92-82ab-74367775ebf1: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911519::DEBUG::2013-11-19 21:36:15,460::task::495::TaskManager.Task::(_debug) Task c903ad96-62d4-4f92-82ab-74367775ebf1: ref 1 aborting False
>Thread-15911519::WARNING::2013-11-19 21:36:15,462::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911519::DEBUG::2013-11-19 21:36:15,463::task::495::TaskManager.Task::(_debug) Task c903ad96-62d4-4f92-82ab-74367775ebf1: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911519::DEBUG::2013-11-19 21:36:15,464::task::495::TaskManager.Task::(_debug) Task c903ad96-62d4-4f92-82ab-74367775ebf1: moving from state preparing -> state finished
>Thread-15911519::DEBUG::2013-11-19 21:36:15,465::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911519::DEBUG::2013-11-19 21:36:15,465::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911519::DEBUG::2013-11-19 21:36:15,466::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911519::DEBUG::2013-11-19 21:36:15,467::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911519::DEBUG::2013-11-19 21:36:15,467::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911519::DEBUG::2013-11-19 21:36:15,468::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911519::DEBUG::2013-11-19 21:36:15,468::task::495::TaskManager.Task::(_debug) Task c903ad96-62d4-4f92-82ab-74367775ebf1: ref 0 aborting False
>Thread-15911519::INFO::2013-11-19 21:36:15,469::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911465::DEBUG::2013-11-19 21:36:17,142::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:36:17,689::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0703454 s, 14.6 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:36:19,403::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911465::DEBUG::2013-11-19 21:36:19,705::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:36:20,236::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.070886 s, 14.4 MB/s\n'; <rc> = 0
>Thread-15911522::INFO::2013-11-19 21:36:20,334::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911522::DEBUG::2013-11-19 21:36:20,335::task::495::TaskManager.Task::(_debug) Task d50313f5-7fee-4a0b-b3fa-85f9e1865ec9: moving from state init -> state preparing
>Thread-15911522::DEBUG::2013-11-19 21:36:20,336::task::495::TaskManager.Task::(_debug) Task d50313f5-7fee-4a0b-b3fa-85f9e1865ec9: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000438928604126', 'lastCheck': 1384896971.7007599, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00143098831177', 'lastCheck': 1384896973.264729, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896979.4038291, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000556945800781', 'lastCheck': 1384896970.1740961, 'valid': True, 'code': 0}}
>Thread-15911522::DEBUG::2013-11-19 21:36:20,337::task::495::TaskManager.Task::(_debug) Task d50313f5-7fee-4a0b-b3fa-85f9e1865ec9: moving from state preparing -> state finished
>Thread-15911522::DEBUG::2013-11-19 21:36:20,338::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911522::DEBUG::2013-11-19 21:36:20,338::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911522::DEBUG::2013-11-19 21:36:20,339::task::495::TaskManager.Task::(_debug) Task d50313f5-7fee-4a0b-b3fa-85f9e1865ec9: ref 0 aborting False
>Thread-15911522::INFO::2013-11-19 21:36:20,340::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000438928604126', 'lastCheck': 1384896971.7007599, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00143098831177', 'lastCheck': 1384896973.264729, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896979.4038291, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000556945800781', 'lastCheck': 1384896970.1740961, 'valid': True, 'code': 0}}
>Dummy-15911465::DEBUG::2013-11-19 21:36:22,250::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:36:22,760::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0699207 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15911465::DEBUG::2013-11-19 21:36:24,776::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:36:25,319::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0706312 s, 14.5 MB/s\n'; <rc> = 0
>Thread-15911526::DEBUG::2013-11-19 21:36:25,577::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911526::INFO::2013-11-19 21:36:25,578::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911526::DEBUG::2013-11-19 21:36:25,579::task::495::TaskManager.Task::(_debug) Task bfa01b48-11a3-468a-9b17-157c58ed5d75: moving from state init -> state preparing
>Thread-15911526::DEBUG::2013-11-19 21:36:25,580::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=285 spmId=2
>Thread-15911526::DEBUG::2013-11-19 21:36:25,580::task::495::TaskManager.Task::(_debug) Task bfa01b48-11a3-468a-9b17-157c58ed5d75: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911526::DEBUG::2013-11-19 21:36:25,581::task::495::TaskManager.Task::(_debug) Task bfa01b48-11a3-468a-9b17-157c58ed5d75: moving from state preparing -> state finished
>Thread-15911526::DEBUG::2013-11-19 21:36:25,582::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911526::DEBUG::2013-11-19 21:36:25,582::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911526::DEBUG::2013-11-19 21:36:25,583::task::495::TaskManager.Task::(_debug) Task bfa01b48-11a3-468a-9b17-157c58ed5d75: ref 0 aborting False
>Thread-15911526::INFO::2013-11-19 21:36:25,583::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911527::DEBUG::2013-11-19 21:36:25,589::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911527::INFO::2013-11-19 21:36:25,590::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911527::DEBUG::2013-11-19 21:36:25,591::task::495::TaskManager.Task::(_debug) Task 1666d51d-2307-41ee-bfa5-ee9b114042dc: moving from state init -> state preparing
>Thread-15911527::DEBUG::2013-11-19 21:36:25,592::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c35dfaf8-0e8b-4867-bc91-c0846f13889e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911527::DEBUG::2013-11-19 21:36:25,593::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911527::DEBUG::2013-11-19 21:36:25,594::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911527::DEBUG::2013-11-19 21:36:25,594::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c35dfaf8-0e8b-4867-bc91-c0846f13889e`::Granted request
>Thread-15911527::DEBUG::2013-11-19 21:36:25,604::task::495::TaskManager.Task::(_debug) Task 1666d51d-2307-41ee-bfa5-ee9b114042dc: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911527::DEBUG::2013-11-19 21:36:25,605::task::495::TaskManager.Task::(_debug) Task 1666d51d-2307-41ee-bfa5-ee9b114042dc: ref 1 aborting False
>Thread-15911527::WARNING::2013-11-19 21:36:25,607::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911527::DEBUG::2013-11-19 21:36:25,608::task::495::TaskManager.Task::(_debug) Task 1666d51d-2307-41ee-bfa5-ee9b114042dc: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911527::DEBUG::2013-11-19 21:36:25,609::task::495::TaskManager.Task::(_debug) Task 1666d51d-2307-41ee-bfa5-ee9b114042dc: moving from state preparing -> state finished
>Thread-15911527::DEBUG::2013-11-19 21:36:25,609::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911527::DEBUG::2013-11-19 21:36:25,610::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911527::DEBUG::2013-11-19 21:36:25,611::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911527::DEBUG::2013-11-19 21:36:25,611::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911527::DEBUG::2013-11-19 21:36:25,612::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911527::DEBUG::2013-11-19 21:36:25,612::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911527::DEBUG::2013-11-19 21:36:25,613::task::495::TaskManager.Task::(_debug) Task 1666d51d-2307-41ee-bfa5-ee9b114042dc: ref 0 aborting False
>Thread-15911527::INFO::2013-11-19 21:36:25,614::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911465::DEBUG::2013-11-19 21:36:27,334::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:36:27,865::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0704832 s, 14.5 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:36:29,576::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911465::DEBUG::2013-11-19 21:36:29,880::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:36:30,408::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0703108 s, 14.6 MB/s\n'; <rc> = 0
>Thread-15911530::INFO::2013-11-19 21:36:30,514::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911530::DEBUG::2013-11-19 21:36:30,515::task::495::TaskManager.Task::(_debug) Task 7a65f44c-1ee6-47f4-9f16-c21ded082352: moving from state init -> state preparing
>Thread-15911530::DEBUG::2013-11-19 21:36:30,516::task::495::TaskManager.Task::(_debug) Task 7a65f44c-1ee6-47f4-9f16-c21ded082352: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000430107116699', 'lastCheck': 1384896981.922365, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00120496749878', 'lastCheck': 1384896983.4630129, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896989.5774319, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000617027282715', 'lastCheck': 1384896980.3921421, 'valid': True, 'code': 0}}
>Thread-15911530::DEBUG::2013-11-19 21:36:30,521::task::495::TaskManager.Task::(_debug) Task 7a65f44c-1ee6-47f4-9f16-c21ded082352: moving from state preparing -> state finished
>Thread-15911530::DEBUG::2013-11-19 21:36:30,522::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911530::DEBUG::2013-11-19 21:36:30,522::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911530::DEBUG::2013-11-19 21:36:30,523::task::495::TaskManager.Task::(_debug) Task 7a65f44c-1ee6-47f4-9f16-c21ded082352: ref 0 aborting False
>Thread-15911530::INFO::2013-11-19 21:36:30,523::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000430107116699', 'lastCheck': 1384896981.922365, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00120496749878', 'lastCheck': 1384896983.4630129, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896989.5774319, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000617027282715', 'lastCheck': 1384896980.3921421, 'valid': True, 'code': 0}}
>Dummy-15911465::DEBUG::2013-11-19 21:36:32,422::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:36:32,963::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0705334 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15911465::DEBUG::2013-11-19 21:36:34,978::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:36:35,525::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0952947 s, 10.7 MB/s\n'; <rc> = 0
>Thread-15911534::DEBUG::2013-11-19 21:36:35,743::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911534::INFO::2013-11-19 21:36:35,744::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911534::DEBUG::2013-11-19 21:36:35,745::task::495::TaskManager.Task::(_debug) Task 8b6366a5-ad71-4704-a10a-7e71e28f1fb1: moving from state init -> state preparing
>Thread-15911534::DEBUG::2013-11-19 21:36:35,746::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=285 spmId=2
>Thread-15911534::DEBUG::2013-11-19 21:36:35,747::task::495::TaskManager.Task::(_debug) Task 8b6366a5-ad71-4704-a10a-7e71e28f1fb1: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911534::DEBUG::2013-11-19 21:36:35,747::task::495::TaskManager.Task::(_debug) Task 8b6366a5-ad71-4704-a10a-7e71e28f1fb1: moving from state preparing -> state finished
>Thread-15911534::DEBUG::2013-11-19 21:36:35,748::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911534::DEBUG::2013-11-19 21:36:35,748::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911534::DEBUG::2013-11-19 21:36:35,749::task::495::TaskManager.Task::(_debug) Task 8b6366a5-ad71-4704-a10a-7e71e28f1fb1: ref 0 aborting False
>Thread-15911534::INFO::2013-11-19 21:36:35,750::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911535::DEBUG::2013-11-19 21:36:35,764::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911535::INFO::2013-11-19 21:36:35,765::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911535::DEBUG::2013-11-19 21:36:35,765::task::495::TaskManager.Task::(_debug) Task dda0638a-1667-4f2a-bf48-d0751c7970ee: moving from state init -> state preparing
>Thread-15911535::DEBUG::2013-11-19 21:36:35,767::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4adb4385-ee98-44e4-8a38-3fdd36aecfbd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911535::DEBUG::2013-11-19 21:36:35,767::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911535::DEBUG::2013-11-19 21:36:35,768::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911535::DEBUG::2013-11-19 21:36:35,769::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4adb4385-ee98-44e4-8a38-3fdd36aecfbd`::Granted request
>Thread-15911535::DEBUG::2013-11-19 21:36:35,770::task::495::TaskManager.Task::(_debug) Task dda0638a-1667-4f2a-bf48-d0751c7970ee: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911535::DEBUG::2013-11-19 21:36:35,770::task::495::TaskManager.Task::(_debug) Task dda0638a-1667-4f2a-bf48-d0751c7970ee: ref 1 aborting False
>Thread-15911535::WARNING::2013-11-19 21:36:35,772::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911535::DEBUG::2013-11-19 21:36:35,773::task::495::TaskManager.Task::(_debug) Task dda0638a-1667-4f2a-bf48-d0751c7970ee: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911535::DEBUG::2013-11-19 21:36:35,774::task::495::TaskManager.Task::(_debug) Task dda0638a-1667-4f2a-bf48-d0751c7970ee: moving from state preparing -> state finished
>Thread-15911535::DEBUG::2013-11-19 21:36:35,775::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911535::DEBUG::2013-11-19 21:36:35,775::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911535::DEBUG::2013-11-19 21:36:35,776::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911535::DEBUG::2013-11-19 21:36:35,777::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911535::DEBUG::2013-11-19 21:36:35,777::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911535::DEBUG::2013-11-19 21:36:35,778::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911535::DEBUG::2013-11-19 21:36:35,778::task::495::TaskManager.Task::(_debug) Task dda0638a-1667-4f2a-bf48-d0751c7970ee: ref 0 aborting False
>Thread-15911535::INFO::2013-11-19 21:36:35,779::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911465::DEBUG::2013-11-19 21:36:37,540::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:36:38,091::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0709523 s, 14.4 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:36:39,774::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911465::DEBUG::2013-11-19 21:36:40,107::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:36:40,629::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0695903 s, 14.7 MB/s\n'; <rc> = 0
>Thread-15911538::INFO::2013-11-19 21:36:40,728::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911538::DEBUG::2013-11-19 21:36:40,729::task::495::TaskManager.Task::(_debug) Task 67fe47f6-b987-486c-80bc-547ffa448560: moving from state init -> state preparing
>Thread-15911538::DEBUG::2013-11-19 21:36:40,730::task::495::TaskManager.Task::(_debug) Task 67fe47f6-b987-486c-80bc-547ffa448560: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000355958938599', 'lastCheck': 1384896992.1142321, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00146102905273', 'lastCheck': 1384896993.6575191, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896999.775434, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00046706199646', 'lastCheck': 1384896990.5561421, 'valid': True, 'code': 0}}
>Thread-15911538::DEBUG::2013-11-19 21:36:40,738::task::495::TaskManager.Task::(_debug) Task 67fe47f6-b987-486c-80bc-547ffa448560: moving from state preparing -> state finished
>Thread-15911538::DEBUG::2013-11-19 21:36:40,738::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911538::DEBUG::2013-11-19 21:36:40,739::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911538::DEBUG::2013-11-19 21:36:40,740::task::495::TaskManager.Task::(_debug) Task 67fe47f6-b987-486c-80bc-547ffa448560: ref 0 aborting False
>Thread-15911538::INFO::2013-11-19 21:36:40,740::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000355958938599', 'lastCheck': 1384896992.1142321, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00146102905273', 'lastCheck': 1384896993.6575191, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384896999.775434, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00046706199646', 'lastCheck': 1384896990.5561421, 'valid': True, 'code': 0}}
>Dummy-15911465::DEBUG::2013-11-19 21:36:42,645::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:36:43,192::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0703361 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15911465::DEBUG::2013-11-19 21:36:45,207::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:36:45,749::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0698755 s, 14.7 MB/s\n'; <rc> = 0
>Thread-15911542::DEBUG::2013-11-19 21:36:45,900::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911542::INFO::2013-11-19 21:36:45,902::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911542::DEBUG::2013-11-19 21:36:45,902::task::495::TaskManager.Task::(_debug) Task d81a27e7-0167-44de-8dd3-226c936aee4d: moving from state init -> state preparing
>Thread-15911542::DEBUG::2013-11-19 21:36:45,903::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=285 spmId=2
>Thread-15911542::DEBUG::2013-11-19 21:36:45,904::task::495::TaskManager.Task::(_debug) Task d81a27e7-0167-44de-8dd3-226c936aee4d: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911542::DEBUG::2013-11-19 21:36:45,904::task::495::TaskManager.Task::(_debug) Task d81a27e7-0167-44de-8dd3-226c936aee4d: moving from state preparing -> state finished
>Thread-15911542::DEBUG::2013-11-19 21:36:45,905::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911542::DEBUG::2013-11-19 21:36:45,906::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911542::DEBUG::2013-11-19 21:36:45,906::task::495::TaskManager.Task::(_debug) Task d81a27e7-0167-44de-8dd3-226c936aee4d: ref 0 aborting False
>Thread-15911542::INFO::2013-11-19 21:36:45,907::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911543::DEBUG::2013-11-19 21:36:45,913::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911543::INFO::2013-11-19 21:36:45,913::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911543::DEBUG::2013-11-19 21:36:45,914::task::495::TaskManager.Task::(_debug) Task a0d33a37-467c-44ff-989d-c1d48a5e0d81: moving from state init -> state preparing
>Thread-15911543::DEBUG::2013-11-19 21:36:45,915::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4c9af96e-5c06-4ec5-ab29-5dbd9756a261`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911543::DEBUG::2013-11-19 21:36:45,916::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911543::DEBUG::2013-11-19 21:36:45,917::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911543::DEBUG::2013-11-19 21:36:45,917::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4c9af96e-5c06-4ec5-ab29-5dbd9756a261`::Granted request
>Thread-15911543::DEBUG::2013-11-19 21:36:45,918::task::495::TaskManager.Task::(_debug) Task a0d33a37-467c-44ff-989d-c1d48a5e0d81: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911543::DEBUG::2013-11-19 21:36:45,919::task::495::TaskManager.Task::(_debug) Task a0d33a37-467c-44ff-989d-c1d48a5e0d81: ref 1 aborting False
>Thread-15911543::WARNING::2013-11-19 21:36:45,921::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911543::DEBUG::2013-11-19 21:36:45,922::task::495::TaskManager.Task::(_debug) Task a0d33a37-467c-44ff-989d-c1d48a5e0d81: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911543::DEBUG::2013-11-19 21:36:45,923::task::495::TaskManager.Task::(_debug) Task a0d33a37-467c-44ff-989d-c1d48a5e0d81: moving from state preparing -> state finished
>Thread-15911543::DEBUG::2013-11-19 21:36:45,924::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911543::DEBUG::2013-11-19 21:36:45,924::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911543::DEBUG::2013-11-19 21:36:45,925::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911543::DEBUG::2013-11-19 21:36:45,926::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911543::DEBUG::2013-11-19 21:36:45,926::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911543::DEBUG::2013-11-19 21:36:45,927::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911543::DEBUG::2013-11-19 21:36:45,927::task::495::TaskManager.Task::(_debug) Task a0d33a37-467c-44ff-989d-c1d48a5e0d81: ref 0 aborting False
>Thread-15911543::INFO::2013-11-19 21:36:45,928::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911465::DEBUG::2013-11-19 21:36:47,764::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:36:48,320::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0703347 s, 14.6 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:36:50,004::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911465::DEBUG::2013-11-19 21:36:50,335::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:36:50,863::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0702332 s, 14.6 MB/s\n'; <rc> = 0
>Thread-15911546::INFO::2013-11-19 21:36:50,919::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911546::DEBUG::2013-11-19 21:36:50,920::task::495::TaskManager.Task::(_debug) Task 02c78b33-c7c2-46c9-b36e-133f539a51ea: moving from state init -> state preparing
>Thread-15911546::DEBUG::2013-11-19 21:36:50,921::task::495::TaskManager.Task::(_debug) Task 02c78b33-c7c2-46c9-b36e-133f539a51ea: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000331163406372', 'lastCheck': 1384897002.3182349, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00147199630737', 'lastCheck': 1384897003.879705, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897010.0051179, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000424861907959', 'lastCheck': 1384897000.7737031, 'valid': True, 'code': 0}}
>Thread-15911546::DEBUG::2013-11-19 21:36:50,922::task::495::TaskManager.Task::(_debug) Task 02c78b33-c7c2-46c9-b36e-133f539a51ea: moving from state preparing -> state finished
>Thread-15911546::DEBUG::2013-11-19 21:36:50,923::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911546::DEBUG::2013-11-19 21:36:50,923::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911546::DEBUG::2013-11-19 21:36:50,924::task::495::TaskManager.Task::(_debug) Task 02c78b33-c7c2-46c9-b36e-133f539a51ea: ref 0 aborting False
>Thread-15911546::INFO::2013-11-19 21:36:50,929::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000331163406372', 'lastCheck': 1384897002.3182349, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00147199630737', 'lastCheck': 1384897003.879705, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897010.0051179, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000424861907959', 'lastCheck': 1384897000.7737031, 'valid': True, 'code': 0}}
>Dummy-15911465::DEBUG::2013-11-19 21:36:52,878::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:36:53,399::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0691949 s, 14.8 MB/s\n'; <rc> = 0
>Dummy-15911465::DEBUG::2013-11-19 21:36:55,414::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:36:55,962::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0712482 s, 14.4 MB/s\n'; <rc> = 0
>Thread-15911550::DEBUG::2013-11-19 21:36:56,032::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911550::INFO::2013-11-19 21:36:56,034::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911550::DEBUG::2013-11-19 21:36:56,034::task::495::TaskManager.Task::(_debug) Task 216b2af3-11bf-4b2e-b10c-1c5060835987: moving from state init -> state preparing
>Thread-15911550::DEBUG::2013-11-19 21:36:56,035::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=285 spmId=2
>Thread-15911550::DEBUG::2013-11-19 21:36:56,036::task::495::TaskManager.Task::(_debug) Task 216b2af3-11bf-4b2e-b10c-1c5060835987: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911550::DEBUG::2013-11-19 21:36:56,036::task::495::TaskManager.Task::(_debug) Task 216b2af3-11bf-4b2e-b10c-1c5060835987: moving from state preparing -> state finished
>Thread-15911550::DEBUG::2013-11-19 21:36:56,037::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911550::DEBUG::2013-11-19 21:36:56,038::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911550::DEBUG::2013-11-19 21:36:56,038::task::495::TaskManager.Task::(_debug) Task 216b2af3-11bf-4b2e-b10c-1c5060835987: ref 0 aborting False
>Thread-15911550::INFO::2013-11-19 21:36:56,039::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911551::DEBUG::2013-11-19 21:36:56,051::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911551::INFO::2013-11-19 21:36:56,052::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911551::DEBUG::2013-11-19 21:36:56,053::task::495::TaskManager.Task::(_debug) Task c3d2852b-a664-419d-8c59-100f16e74990: moving from state init -> state preparing
>Thread-15911551::DEBUG::2013-11-19 21:36:56,054::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`96615427-72a9-4bee-9dc1-d1670993ae33`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911551::DEBUG::2013-11-19 21:36:56,055::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911551::DEBUG::2013-11-19 21:36:56,055::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911551::DEBUG::2013-11-19 21:36:56,056::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`96615427-72a9-4bee-9dc1-d1670993ae33`::Granted request
>Thread-15911551::DEBUG::2013-11-19 21:36:56,057::task::495::TaskManager.Task::(_debug) Task c3d2852b-a664-419d-8c59-100f16e74990: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911551::DEBUG::2013-11-19 21:36:56,058::task::495::TaskManager.Task::(_debug) Task c3d2852b-a664-419d-8c59-100f16e74990: ref 1 aborting False
>Thread-15911551::WARNING::2013-11-19 21:36:56,060::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911551::DEBUG::2013-11-19 21:36:56,061::task::495::TaskManager.Task::(_debug) Task c3d2852b-a664-419d-8c59-100f16e74990: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911551::DEBUG::2013-11-19 21:36:56,062::task::495::TaskManager.Task::(_debug) Task c3d2852b-a664-419d-8c59-100f16e74990: moving from state preparing -> state finished
>Thread-15911551::DEBUG::2013-11-19 21:36:56,062::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911551::DEBUG::2013-11-19 21:36:56,063::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911551::DEBUG::2013-11-19 21:36:56,063::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911551::DEBUG::2013-11-19 21:36:56,064::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911551::DEBUG::2013-11-19 21:36:56,065::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911551::DEBUG::2013-11-19 21:36:56,065::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911551::DEBUG::2013-11-19 21:36:56,066::task::495::TaskManager.Task::(_debug) Task c3d2852b-a664-419d-8c59-100f16e74990: ref 0 aborting False
>Thread-15911551::INFO::2013-11-19 21:36:56,067::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911465::DEBUG::2013-11-19 21:36:57,977::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:36:58,501::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0700069 s, 14.6 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:37:00,190::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911465::DEBUG::2013-11-19 21:37:00,517::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:37:01,068::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0703909 s, 14.5 MB/s\n'; <rc> = 0
>Thread-15911554::INFO::2013-11-19 21:37:01,151::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911554::DEBUG::2013-11-19 21:37:01,152::task::495::TaskManager.Task::(_debug) Task eb20e2f6-3f89-4a36-8c4b-c4165bf2b7cc: moving from state init -> state preparing
>Thread-15911554::DEBUG::2013-11-19 21:37:01,153::task::495::TaskManager.Task::(_debug) Task eb20e2f6-3f89-4a36-8c4b-c4165bf2b7cc: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000578165054321', 'lastCheck': 1384897012.5628331, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00189709663391', 'lastCheck': 1384897014.113179, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897020.190887, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000512838363647', 'lastCheck': 1384897011.0029211, 'valid': True, 'code': 0}}
>Thread-15911554::DEBUG::2013-11-19 21:37:01,162::task::495::TaskManager.Task::(_debug) Task eb20e2f6-3f89-4a36-8c4b-c4165bf2b7cc: moving from state preparing -> state finished
>Thread-15911554::DEBUG::2013-11-19 21:37:01,163::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911554::DEBUG::2013-11-19 21:37:01,164::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911554::DEBUG::2013-11-19 21:37:01,164::task::495::TaskManager.Task::(_debug) Task eb20e2f6-3f89-4a36-8c4b-c4165bf2b7cc: ref 0 aborting False
>Thread-15911554::INFO::2013-11-19 21:37:01,166::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000578165054321', 'lastCheck': 1384897012.5628331, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00189709663391', 'lastCheck': 1384897014.113179, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897020.190887, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000512838363647', 'lastCheck': 1384897011.0029211, 'valid': True, 'code': 0}}
>Dummy-15911465::DEBUG::2013-11-19 21:37:03,083::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:37:03,614::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0706885 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15911465::DEBUG::2013-11-19 21:37:05,629::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:37:06,160::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.070228 s, 14.6 MB/s\n'; <rc> = 0
>Thread-15911558::DEBUG::2013-11-19 21:37:06,239::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911558::INFO::2013-11-19 21:37:06,241::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911558::DEBUG::2013-11-19 21:37:06,241::task::495::TaskManager.Task::(_debug) Task 37d78de6-b775-4a4a-9daf-d9cc3216d1e5: moving from state init -> state preparing
>Thread-15911558::DEBUG::2013-11-19 21:37:06,242::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=285 spmId=2
>Thread-15911558::DEBUG::2013-11-19 21:37:06,243::task::495::TaskManager.Task::(_debug) Task 37d78de6-b775-4a4a-9daf-d9cc3216d1e5: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911558::DEBUG::2013-11-19 21:37:06,243::task::495::TaskManager.Task::(_debug) Task 37d78de6-b775-4a4a-9daf-d9cc3216d1e5: moving from state preparing -> state finished
>Thread-15911558::DEBUG::2013-11-19 21:37:06,244::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911558::DEBUG::2013-11-19 21:37:06,244::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911558::DEBUG::2013-11-19 21:37:06,245::task::495::TaskManager.Task::(_debug) Task 37d78de6-b775-4a4a-9daf-d9cc3216d1e5: ref 0 aborting False
>Thread-15911558::INFO::2013-11-19 21:37:06,246::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911559::DEBUG::2013-11-19 21:37:06,252::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911559::INFO::2013-11-19 21:37:06,253::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911559::DEBUG::2013-11-19 21:37:06,253::task::495::TaskManager.Task::(_debug) Task 1c6ebe12-eb8e-4591-bec6-5720ef196d0f: moving from state init -> state preparing
>Thread-15911559::DEBUG::2013-11-19 21:37:06,254::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a69b1614-e0eb-4687-8787-9af00b9388b4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911559::DEBUG::2013-11-19 21:37:06,255::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911559::DEBUG::2013-11-19 21:37:06,256::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911559::DEBUG::2013-11-19 21:37:06,256::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a69b1614-e0eb-4687-8787-9af00b9388b4`::Granted request
>Thread-15911559::DEBUG::2013-11-19 21:37:06,257::task::495::TaskManager.Task::(_debug) Task 1c6ebe12-eb8e-4591-bec6-5720ef196d0f: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911559::DEBUG::2013-11-19 21:37:06,258::task::495::TaskManager.Task::(_debug) Task 1c6ebe12-eb8e-4591-bec6-5720ef196d0f: ref 1 aborting False
>Thread-15911559::WARNING::2013-11-19 21:37:06,260::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911559::DEBUG::2013-11-19 21:37:06,261::task::495::TaskManager.Task::(_debug) Task 1c6ebe12-eb8e-4591-bec6-5720ef196d0f: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911559::DEBUG::2013-11-19 21:37:06,262::task::495::TaskManager.Task::(_debug) Task 1c6ebe12-eb8e-4591-bec6-5720ef196d0f: moving from state preparing -> state finished
>Thread-15911559::DEBUG::2013-11-19 21:37:06,263::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911559::DEBUG::2013-11-19 21:37:06,263::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911559::DEBUG::2013-11-19 21:37:06,264::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911559::DEBUG::2013-11-19 21:37:06,265::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911559::DEBUG::2013-11-19 21:37:06,265::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911559::DEBUG::2013-11-19 21:37:06,266::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911559::DEBUG::2013-11-19 21:37:06,266::task::495::TaskManager.Task::(_debug) Task 1c6ebe12-eb8e-4591-bec6-5720ef196d0f: ref 0 aborting False
>Thread-15911559::INFO::2013-11-19 21:37:06,267::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911465::DEBUG::2013-11-19 21:37:08,175::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:37:08,712::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.070183 s, 14.6 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:37:10,401::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911465::DEBUG::2013-11-19 21:37:10,726::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:37:11,242::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0702783 s, 14.6 MB/s\n'; <rc> = 0
>Thread-15911562::INFO::2013-11-19 21:37:11,374::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911562::DEBUG::2013-11-19 21:37:11,375::task::495::TaskManager.Task::(_debug) Task d88f5615-f4e8-45b1-971f-c8830f50c978: moving from state init -> state preparing
>Thread-15911562::DEBUG::2013-11-19 21:37:11,375::task::495::TaskManager.Task::(_debug) Task d88f5615-f4e8-45b1-971f-c8830f50c978: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000396966934204', 'lastCheck': 1384897022.775394, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00134110450745', 'lastCheck': 1384897024.2985539, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897030.401911, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000432014465332', 'lastCheck': 1384897021.189374, 'valid': True, 'code': 0}}
>Thread-15911562::DEBUG::2013-11-19 21:37:11,381::task::495::TaskManager.Task::(_debug) Task d88f5615-f4e8-45b1-971f-c8830f50c978: moving from state preparing -> state finished
>Thread-15911562::DEBUG::2013-11-19 21:37:11,381::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911562::DEBUG::2013-11-19 21:37:11,382::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911562::DEBUG::2013-11-19 21:37:11,383::task::495::TaskManager.Task::(_debug) Task d88f5615-f4e8-45b1-971f-c8830f50c978: ref 0 aborting False
>Thread-15911562::INFO::2013-11-19 21:37:11,386::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000396966934204', 'lastCheck': 1384897022.775394, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00134110450745', 'lastCheck': 1384897024.2985539, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897030.401911, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000432014465332', 'lastCheck': 1384897021.189374, 'valid': True, 'code': 0}}
>Dummy-15911465::DEBUG::2013-11-19 21:37:13,257::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:37:13,778::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0693848 s, 14.8 MB/s\n'; <rc> = 0
>Dummy-15911465::DEBUG::2013-11-19 21:37:15,793::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:37:16,335::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0702338 s, 14.6 MB/s\n'; <rc> = 0
>Thread-15911566::DEBUG::2013-11-19 21:37:16,439::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911566::INFO::2013-11-19 21:37:16,440::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911566::DEBUG::2013-11-19 21:37:16,451::task::495::TaskManager.Task::(_debug) Task 522f552f-8eff-4d0b-9c6a-088723de4a40: moving from state init -> state preparing
>Thread-15911566::DEBUG::2013-11-19 21:37:16,452::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=285 spmId=2
>Thread-15911566::DEBUG::2013-11-19 21:37:16,453::task::495::TaskManager.Task::(_debug) Task 522f552f-8eff-4d0b-9c6a-088723de4a40: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911566::DEBUG::2013-11-19 21:37:16,453::task::495::TaskManager.Task::(_debug) Task 522f552f-8eff-4d0b-9c6a-088723de4a40: moving from state preparing -> state finished
>Thread-15911566::DEBUG::2013-11-19 21:37:16,454::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911566::DEBUG::2013-11-19 21:37:16,454::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911566::DEBUG::2013-11-19 21:37:16,455::task::495::TaskManager.Task::(_debug) Task 522f552f-8eff-4d0b-9c6a-088723de4a40: ref 0 aborting False
>Thread-15911566::INFO::2013-11-19 21:37:16,456::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911567::DEBUG::2013-11-19 21:37:16,461::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911567::INFO::2013-11-19 21:37:16,462::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911567::DEBUG::2013-11-19 21:37:16,463::task::495::TaskManager.Task::(_debug) Task 449c5bff-aab9-4d89-8f8c-8567f24e1ba0: moving from state init -> state preparing
>Thread-15911567::DEBUG::2013-11-19 21:37:16,464::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f408f6a5-1a9e-4136-b031-8cc2670b97b7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911567::DEBUG::2013-11-19 21:37:16,465::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911567::DEBUG::2013-11-19 21:37:16,466::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911567::DEBUG::2013-11-19 21:37:16,466::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f408f6a5-1a9e-4136-b031-8cc2670b97b7`::Granted request
>Thread-15911567::DEBUG::2013-11-19 21:37:16,467::task::495::TaskManager.Task::(_debug) Task 449c5bff-aab9-4d89-8f8c-8567f24e1ba0: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911567::DEBUG::2013-11-19 21:37:16,468::task::495::TaskManager.Task::(_debug) Task 449c5bff-aab9-4d89-8f8c-8567f24e1ba0: ref 1 aborting False
>Thread-15911567::WARNING::2013-11-19 21:37:16,470::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911567::DEBUG::2013-11-19 21:37:16,471::task::495::TaskManager.Task::(_debug) Task 449c5bff-aab9-4d89-8f8c-8567f24e1ba0: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911567::DEBUG::2013-11-19 21:37:16,472::task::495::TaskManager.Task::(_debug) Task 449c5bff-aab9-4d89-8f8c-8567f24e1ba0: moving from state preparing -> state finished
>Thread-15911567::DEBUG::2013-11-19 21:37:16,472::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911567::DEBUG::2013-11-19 21:37:16,473::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911567::DEBUG::2013-11-19 21:37:16,474::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911567::DEBUG::2013-11-19 21:37:16,474::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911567::DEBUG::2013-11-19 21:37:16,475::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911567::DEBUG::2013-11-19 21:37:16,475::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911567::DEBUG::2013-11-19 21:37:16,476::task::495::TaskManager.Task::(_debug) Task 449c5bff-aab9-4d89-8f8c-8567f24e1ba0: ref 0 aborting False
>Thread-15911567::INFO::2013-11-19 21:37:16,477::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911465::DEBUG::2013-11-19 21:37:18,350::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:37:18,853::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0783585 s, 13.1 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:37:20,559::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911465::DEBUG::2013-11-19 21:37:20,864::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:37:21,329::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0675208 s, 15.2 MB/s\n'; <rc> = 0
>Thread-15911570::INFO::2013-11-19 21:37:21,546::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911570::DEBUG::2013-11-19 21:37:21,547::task::495::TaskManager.Task::(_debug) Task a50a5412-590a-4580-873d-d5c6eadb8675: moving from state init -> state preparing
>Thread-42840::DEBUG::2013-11-19 21:37:21,560::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-15911570::DEBUG::2013-11-19 21:37:21,561::task::495::TaskManager.Task::(_debug) Task a50a5412-590a-4580-873d-d5c6eadb8675: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000391006469727', 'lastCheck': 1384897032.933435, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00149297714233', 'lastCheck': 1384897034.484406, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897040.5614519, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000517129898071', 'lastCheck': 1384897031.4028809, 'valid': True, 'code': 0}}
>Thread-15911570::DEBUG::2013-11-19 21:37:21,813::task::495::TaskManager.Task::(_debug) Task a50a5412-590a-4580-873d-d5c6eadb8675: moving from state preparing -> state finished
>Thread-15911570::DEBUG::2013-11-19 21:37:21,815::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911570::DEBUG::2013-11-19 21:37:21,816::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911570::DEBUG::2013-11-19 21:37:21,817::task::495::TaskManager.Task::(_debug) Task a50a5412-590a-4580-873d-d5c6eadb8675: ref 0 aborting False
>Thread-15911570::INFO::2013-11-19 21:37:21,817::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000391006469727', 'lastCheck': 1384897032.933435, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00149297714233', 'lastCheck': 1384897034.484406, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897040.5614519, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000517129898071', 'lastCheck': 1384897031.4028809, 'valid': True, 'code': 0}}
>Thread-42840::DEBUG::2013-11-19 21:37:22,200::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15911465::DEBUG::2013-11-19 21:37:23,345::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:37:23,870::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0684624 s, 15.0 MB/s\n'; <rc> = 0
>Dummy-15911465::DEBUG::2013-11-19 21:37:25,885::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:37:26,404::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0694182 s, 14.8 MB/s\n'; <rc> = 0
>Thread-15911574::DEBUG::2013-11-19 21:37:26,653::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911574::INFO::2013-11-19 21:37:26,655::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911574::DEBUG::2013-11-19 21:37:26,664::task::495::TaskManager.Task::(_debug) Task da35be39-100f-4434-81c4-92b9569351bb: moving from state init -> state preparing
>Thread-15911574::DEBUG::2013-11-19 21:37:26,665::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=285 spmId=2
>Thread-15911574::DEBUG::2013-11-19 21:37:26,665::task::495::TaskManager.Task::(_debug) Task da35be39-100f-4434-81c4-92b9569351bb: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911574::DEBUG::2013-11-19 21:37:26,666::task::495::TaskManager.Task::(_debug) Task da35be39-100f-4434-81c4-92b9569351bb: moving from state preparing -> state finished
>Thread-15911574::DEBUG::2013-11-19 21:37:26,667::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911574::DEBUG::2013-11-19 21:37:26,667::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911574::DEBUG::2013-11-19 21:37:26,668::task::495::TaskManager.Task::(_debug) Task da35be39-100f-4434-81c4-92b9569351bb: ref 0 aborting False
>Thread-15911574::INFO::2013-11-19 21:37:26,668::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911575::DEBUG::2013-11-19 21:37:26,674::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911575::INFO::2013-11-19 21:37:26,675::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911575::DEBUG::2013-11-19 21:37:26,676::task::495::TaskManager.Task::(_debug) Task a300ab53-7971-48d4-9c88-2b0aa277f707: moving from state init -> state preparing
>Thread-15911575::DEBUG::2013-11-19 21:37:26,677::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a98774e2-1eac-49e8-8cfd-44507f48a1a5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911575::DEBUG::2013-11-19 21:37:26,677::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911575::DEBUG::2013-11-19 21:37:26,678::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911575::DEBUG::2013-11-19 21:37:26,679::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a98774e2-1eac-49e8-8cfd-44507f48a1a5`::Granted request
>Thread-15911575::DEBUG::2013-11-19 21:37:26,680::task::495::TaskManager.Task::(_debug) Task a300ab53-7971-48d4-9c88-2b0aa277f707: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911575::DEBUG::2013-11-19 21:37:26,680::task::495::TaskManager.Task::(_debug) Task a300ab53-7971-48d4-9c88-2b0aa277f707: ref 1 aborting False
>Thread-15911575::WARNING::2013-11-19 21:37:26,682::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911575::DEBUG::2013-11-19 21:37:26,683::task::495::TaskManager.Task::(_debug) Task a300ab53-7971-48d4-9c88-2b0aa277f707: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911575::DEBUG::2013-11-19 21:37:26,684::task::495::TaskManager.Task::(_debug) Task a300ab53-7971-48d4-9c88-2b0aa277f707: moving from state preparing -> state finished
>Thread-15911575::DEBUG::2013-11-19 21:37:26,685::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911575::DEBUG::2013-11-19 21:37:26,686::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911575::DEBUG::2013-11-19 21:37:26,686::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911575::DEBUG::2013-11-19 21:37:26,687::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911575::DEBUG::2013-11-19 21:37:26,687::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911575::DEBUG::2013-11-19 21:37:26,688::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911575::DEBUG::2013-11-19 21:37:26,689::task::495::TaskManager.Task::(_debug) Task a300ab53-7971-48d4-9c88-2b0aa277f707: ref 0 aborting False
>Thread-15911575::INFO::2013-11-19 21:37:26,689::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911465::DEBUG::2013-11-19 21:37:28,419::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:37:28,952::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0705261 s, 14.5 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:37:30,641::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911465::DEBUG::2013-11-19 21:37:30,967::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:37:31,497::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0700654 s, 14.6 MB/s\n'; <rc> = 0
>Thread-15911578::INFO::2013-11-19 21:37:32,144::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911578::DEBUG::2013-11-19 21:37:32,145::task::495::TaskManager.Task::(_debug) Task 9bf2d344-b6d0-4c66-83d9-e86d99488c96: moving from state init -> state preparing
>Thread-15911578::DEBUG::2013-11-19 21:37:32,145::task::495::TaskManager.Task::(_debug) Task 9bf2d344-b6d0-4c66-83d9-e86d99488c96: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000404119491577', 'lastCheck': 1384897043.0767901, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00154399871826', 'lastCheck': 1384897044.565289, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897050.6426411, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000310897827148', 'lastCheck': 1384897042.2026429, 'valid': True, 'code': 0}}
>Thread-15911578::DEBUG::2013-11-19 21:37:32,151::task::495::TaskManager.Task::(_debug) Task 9bf2d344-b6d0-4c66-83d9-e86d99488c96: moving from state preparing -> state finished
>Thread-15911578::DEBUG::2013-11-19 21:37:32,151::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911578::DEBUG::2013-11-19 21:37:32,152::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911578::DEBUG::2013-11-19 21:37:32,153::task::495::TaskManager.Task::(_debug) Task 9bf2d344-b6d0-4c66-83d9-e86d99488c96: ref 0 aborting False
>Thread-15911578::INFO::2013-11-19 21:37:32,153::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000404119491577', 'lastCheck': 1384897043.0767901, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00154399871826', 'lastCheck': 1384897044.565289, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897050.6426411, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000310897827148', 'lastCheck': 1384897042.2026429, 'valid': True, 'code': 0}}
>Dummy-15911465::DEBUG::2013-11-19 21:37:33,511::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:37:34,057::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0703092 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15911465::DEBUG::2013-11-19 21:37:36,072::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:37:36,600::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0706245 s, 14.5 MB/s\n'; <rc> = 0
>Thread-15911582::DEBUG::2013-11-19 21:37:36,815::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911582::INFO::2013-11-19 21:37:36,820::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911582::DEBUG::2013-11-19 21:37:36,821::task::495::TaskManager.Task::(_debug) Task 5cfd2524-6b2c-4e58-8854-44d6dc212df8: moving from state init -> state preparing
>Thread-15911582::DEBUG::2013-11-19 21:37:36,822::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=285 spmId=2
>Thread-15911582::DEBUG::2013-11-19 21:37:36,823::task::495::TaskManager.Task::(_debug) Task 5cfd2524-6b2c-4e58-8854-44d6dc212df8: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911582::DEBUG::2013-11-19 21:37:36,823::task::495::TaskManager.Task::(_debug) Task 5cfd2524-6b2c-4e58-8854-44d6dc212df8: moving from state preparing -> state finished
>Thread-15911582::DEBUG::2013-11-19 21:37:36,824::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911582::DEBUG::2013-11-19 21:37:36,824::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911582::DEBUG::2013-11-19 21:37:36,825::task::495::TaskManager.Task::(_debug) Task 5cfd2524-6b2c-4e58-8854-44d6dc212df8: ref 0 aborting False
>Thread-15911582::INFO::2013-11-19 21:37:36,825::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911583::DEBUG::2013-11-19 21:37:36,833::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911583::INFO::2013-11-19 21:37:36,834::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911583::DEBUG::2013-11-19 21:37:36,834::task::495::TaskManager.Task::(_debug) Task d2caa9fd-d879-43bc-9770-9aef11f0ac48: moving from state init -> state preparing
>Thread-15911583::DEBUG::2013-11-19 21:37:36,836::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`66a70e0c-0e21-41a6-b426-7b2aec8e0914`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911583::DEBUG::2013-11-19 21:37:36,836::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911583::DEBUG::2013-11-19 21:37:36,837::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911583::DEBUG::2013-11-19 21:37:36,838::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`66a70e0c-0e21-41a6-b426-7b2aec8e0914`::Granted request
>Thread-15911583::DEBUG::2013-11-19 21:37:36,839::task::495::TaskManager.Task::(_debug) Task d2caa9fd-d879-43bc-9770-9aef11f0ac48: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911583::DEBUG::2013-11-19 21:37:36,839::task::495::TaskManager.Task::(_debug) Task d2caa9fd-d879-43bc-9770-9aef11f0ac48: ref 1 aborting False
>Thread-15911583::WARNING::2013-11-19 21:37:36,842::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911583::DEBUG::2013-11-19 21:37:36,843::task::495::TaskManager.Task::(_debug) Task d2caa9fd-d879-43bc-9770-9aef11f0ac48: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911583::DEBUG::2013-11-19 21:37:36,844::task::495::TaskManager.Task::(_debug) Task d2caa9fd-d879-43bc-9770-9aef11f0ac48: moving from state preparing -> state finished
>Thread-15911583::DEBUG::2013-11-19 21:37:36,844::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911583::DEBUG::2013-11-19 21:37:36,845::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911583::DEBUG::2013-11-19 21:37:36,845::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911583::DEBUG::2013-11-19 21:37:36,846::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911583::DEBUG::2013-11-19 21:37:36,847::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911583::DEBUG::2013-11-19 21:37:36,847::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911583::DEBUG::2013-11-19 21:37:36,848::task::495::TaskManager.Task::(_debug) Task d2caa9fd-d879-43bc-9770-9aef11f0ac48: ref 0 aborting False
>Thread-15911583::INFO::2013-11-19 21:37:36,849::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911465::DEBUG::2013-11-19 21:37:38,615::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:37:39,125::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0704312 s, 14.5 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:37:40,818::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911465::DEBUG::2013-11-19 21:37:41,140::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:37:41,703::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0899135 s, 11.4 MB/s\n'; <rc> = 0
>Thread-15911586::INFO::2013-11-19 21:37:42,344::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911586::DEBUG::2013-11-19 21:37:42,345::task::495::TaskManager.Task::(_debug) Task ca410588-02c4-4970-8a7c-dd222e67f528: moving from state init -> state preparing
>Thread-15911586::DEBUG::2013-11-19 21:37:42,345::task::495::TaskManager.Task::(_debug) Task ca410588-02c4-4970-8a7c-dd222e67f528: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000373840332031', 'lastCheck': 1384897053.181349, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00166392326355', 'lastCheck': 1384897054.7461939, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897060.8196089, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000485897064209', 'lastCheck': 1384897052.2140279, 'valid': True, 'code': 0}}
>Thread-15911586::DEBUG::2013-11-19 21:37:42,357::task::495::TaskManager.Task::(_debug) Task ca410588-02c4-4970-8a7c-dd222e67f528: moving from state preparing -> state finished
>Thread-15911586::DEBUG::2013-11-19 21:37:42,357::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911586::DEBUG::2013-11-19 21:37:42,358::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911586::DEBUG::2013-11-19 21:37:42,358::task::495::TaskManager.Task::(_debug) Task ca410588-02c4-4970-8a7c-dd222e67f528: ref 0 aborting False
>Thread-15911586::INFO::2013-11-19 21:37:42,359::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000373840332031', 'lastCheck': 1384897053.181349, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00166392326355', 'lastCheck': 1384897054.7461939, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897060.8196089, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000485897064209', 'lastCheck': 1384897052.2140279, 'valid': True, 'code': 0}}
>Dummy-15911465::DEBUG::2013-11-19 21:37:43,719::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:37:44,236::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.068329 s, 15.0 MB/s\n'; <rc> = 0
>Dummy-15911465::DEBUG::2013-11-19 21:37:46,250::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:37:46,767::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0734635 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15911590::DEBUG::2013-11-19 21:37:46,993::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911590::INFO::2013-11-19 21:37:46,995::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911590::DEBUG::2013-11-19 21:37:46,996::task::495::TaskManager.Task::(_debug) Task d49d2336-24f8-4a8f-9dd4-06020f160542: moving from state init -> state preparing
>Thread-15911590::DEBUG::2013-11-19 21:37:46,997::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=285 spmId=2
>Thread-15911590::DEBUG::2013-11-19 21:37:46,997::task::495::TaskManager.Task::(_debug) Task d49d2336-24f8-4a8f-9dd4-06020f160542: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911590::DEBUG::2013-11-19 21:37:46,998::task::495::TaskManager.Task::(_debug) Task d49d2336-24f8-4a8f-9dd4-06020f160542: moving from state preparing -> state finished
>Thread-15911590::DEBUG::2013-11-19 21:37:46,998::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911590::DEBUG::2013-11-19 21:37:46,999::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911590::DEBUG::2013-11-19 21:37:47,000::task::495::TaskManager.Task::(_debug) Task d49d2336-24f8-4a8f-9dd4-06020f160542: ref 0 aborting False
>Thread-15911590::INFO::2013-11-19 21:37:47,000::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911591::DEBUG::2013-11-19 21:37:47,006::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911591::INFO::2013-11-19 21:37:47,007::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911591::DEBUG::2013-11-19 21:37:47,008::task::495::TaskManager.Task::(_debug) Task df4594fa-432a-4fe5-9f22-94572ab1562a: moving from state init -> state preparing
>Thread-15911591::DEBUG::2013-11-19 21:37:47,009::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2674479d-4651-4689-bfa1-14f7fca26d68`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911591::DEBUG::2013-11-19 21:37:47,010::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911591::DEBUG::2013-11-19 21:37:47,011::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911591::DEBUG::2013-11-19 21:37:47,011::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2674479d-4651-4689-bfa1-14f7fca26d68`::Granted request
>Thread-15911591::DEBUG::2013-11-19 21:37:47,012::task::495::TaskManager.Task::(_debug) Task df4594fa-432a-4fe5-9f22-94572ab1562a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911591::DEBUG::2013-11-19 21:37:47,013::task::495::TaskManager.Task::(_debug) Task df4594fa-432a-4fe5-9f22-94572ab1562a: ref 1 aborting False
>Thread-15911591::WARNING::2013-11-19 21:37:47,015::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911591::DEBUG::2013-11-19 21:37:47,016::task::495::TaskManager.Task::(_debug) Task df4594fa-432a-4fe5-9f22-94572ab1562a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911591::DEBUG::2013-11-19 21:37:47,017::task::495::TaskManager.Task::(_debug) Task df4594fa-432a-4fe5-9f22-94572ab1562a: moving from state preparing -> state finished
>Thread-15911591::DEBUG::2013-11-19 21:37:47,017::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911591::DEBUG::2013-11-19 21:37:47,018::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911591::DEBUG::2013-11-19 21:37:47,019::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911591::DEBUG::2013-11-19 21:37:47,019::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911591::DEBUG::2013-11-19 21:37:47,020::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911591::DEBUG::2013-11-19 21:37:47,020::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911591::DEBUG::2013-11-19 21:37:47,021::task::495::TaskManager.Task::(_debug) Task df4594fa-432a-4fe5-9f22-94572ab1562a: ref 0 aborting False
>Thread-15911591::INFO::2013-11-19 21:37:47,022::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911465::DEBUG::2013-11-19 21:37:48,783::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:37:49,321::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.075561 s, 13.6 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:37:51,011::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911465::DEBUG::2013-11-19 21:37:51,336::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:37:51,865::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.070863 s, 14.5 MB/s\n'; <rc> = 0
>Thread-15911594::INFO::2013-11-19 21:37:52,484::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911594::DEBUG::2013-11-19 21:37:52,485::task::495::TaskManager.Task::(_debug) Task fb83372f-cf89-462c-83ae-c794cb19ff66: moving from state init -> state preparing
>Thread-15911594::DEBUG::2013-11-19 21:37:52,486::task::495::TaskManager.Task::(_debug) Task fb83372f-cf89-462c-83ae-c794cb19ff66: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000411987304688', 'lastCheck': 1384897063.376287, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00150299072266', 'lastCheck': 1384897064.9300129, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897071.0119519, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000520944595337', 'lastCheck': 1384897062.3721781, 'valid': True, 'code': 0}}
>Thread-15911594::DEBUG::2013-11-19 21:37:52,487::task::495::TaskManager.Task::(_debug) Task fb83372f-cf89-462c-83ae-c794cb19ff66: moving from state preparing -> state finished
>Thread-15911594::DEBUG::2013-11-19 21:37:52,487::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911594::DEBUG::2013-11-19 21:37:52,488::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911594::DEBUG::2013-11-19 21:37:52,488::task::495::TaskManager.Task::(_debug) Task fb83372f-cf89-462c-83ae-c794cb19ff66: ref 0 aborting False
>Thread-15911594::INFO::2013-11-19 21:37:52,496::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000411987304688', 'lastCheck': 1384897063.376287, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00150299072266', 'lastCheck': 1384897064.9300129, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897071.0119519, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000520944595337', 'lastCheck': 1384897062.3721781, 'valid': True, 'code': 0}}
>Dummy-15911465::DEBUG::2013-11-19 21:37:53,880::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:37:54,428::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0917731 s, 11.2 MB/s\n'; <rc> = 0
>Dummy-15911465::DEBUG::2013-11-19 21:37:56,443::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:37:56,980::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0721752 s, 14.2 MB/s\n'; <rc> = 0
>Thread-15911598::DEBUG::2013-11-19 21:37:57,161::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911598::INFO::2013-11-19 21:37:57,162::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911598::DEBUG::2013-11-19 21:37:57,163::task::495::TaskManager.Task::(_debug) Task bfc372a4-21ab-4174-bc33-ed933221f3f7: moving from state init -> state preparing
>Thread-15911598::DEBUG::2013-11-19 21:37:57,164::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=285 spmId=2
>Thread-15911598::DEBUG::2013-11-19 21:37:57,164::task::495::TaskManager.Task::(_debug) Task bfc372a4-21ab-4174-bc33-ed933221f3f7: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911598::DEBUG::2013-11-19 21:37:57,165::task::495::TaskManager.Task::(_debug) Task bfc372a4-21ab-4174-bc33-ed933221f3f7: moving from state preparing -> state finished
>Thread-15911598::DEBUG::2013-11-19 21:37:57,165::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911598::DEBUG::2013-11-19 21:37:57,166::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911598::DEBUG::2013-11-19 21:37:57,167::task::495::TaskManager.Task::(_debug) Task bfc372a4-21ab-4174-bc33-ed933221f3f7: ref 0 aborting False
>Thread-15911598::INFO::2013-11-19 21:37:57,167::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911599::DEBUG::2013-11-19 21:37:57,173::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911599::INFO::2013-11-19 21:37:57,174::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911599::DEBUG::2013-11-19 21:37:57,174::task::495::TaskManager.Task::(_debug) Task 44a5ef15-d697-4373-bd4e-7d6860b518e6: moving from state init -> state preparing
>Thread-15911599::DEBUG::2013-11-19 21:37:57,176::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d280ddae-7682-446a-8ef1-35d68a4dd7f4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911599::DEBUG::2013-11-19 21:37:57,176::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911599::DEBUG::2013-11-19 21:37:57,177::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911599::DEBUG::2013-11-19 21:37:57,178::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d280ddae-7682-446a-8ef1-35d68a4dd7f4`::Granted request
>Thread-15911599::DEBUG::2013-11-19 21:37:57,179::task::495::TaskManager.Task::(_debug) Task 44a5ef15-d697-4373-bd4e-7d6860b518e6: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911599::DEBUG::2013-11-19 21:37:57,179::task::495::TaskManager.Task::(_debug) Task 44a5ef15-d697-4373-bd4e-7d6860b518e6: ref 1 aborting False
>Thread-15911599::WARNING::2013-11-19 21:37:57,181::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911599::DEBUG::2013-11-19 21:37:57,182::task::495::TaskManager.Task::(_debug) Task 44a5ef15-d697-4373-bd4e-7d6860b518e6: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911599::DEBUG::2013-11-19 21:37:57,183::task::495::TaskManager.Task::(_debug) Task 44a5ef15-d697-4373-bd4e-7d6860b518e6: moving from state preparing -> state finished
>Thread-15911599::DEBUG::2013-11-19 21:37:57,184::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911599::DEBUG::2013-11-19 21:37:57,184::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911599::DEBUG::2013-11-19 21:37:57,185::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911599::DEBUG::2013-11-19 21:37:57,186::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911599::DEBUG::2013-11-19 21:37:57,186::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911599::DEBUG::2013-11-19 21:37:57,187::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911599::DEBUG::2013-11-19 21:37:57,187::task::495::TaskManager.Task::(_debug) Task 44a5ef15-d697-4373-bd4e-7d6860b518e6: ref 0 aborting False
>Thread-15911599::INFO::2013-11-19 21:37:57,188::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911465::DEBUG::2013-11-19 21:37:58,996::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:37:59,506::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0693539 s, 14.8 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:38:01,206::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911465::DEBUG::2013-11-19 21:38:01,521::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:38:02,065::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0718432 s, 14.3 MB/s\n'; <rc> = 0
>Thread-15911602::INFO::2013-11-19 21:38:02,724::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911602::DEBUG::2013-11-19 21:38:02,725::task::495::TaskManager.Task::(_debug) Task 6707ec49-aaa9-4f32-aa29-00b56277220f: moving from state init -> state preparing
>Thread-15911602::DEBUG::2013-11-19 21:38:02,725::task::495::TaskManager.Task::(_debug) Task 6707ec49-aaa9-4f32-aa29-00b56277220f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000331878662109', 'lastCheck': 1384897073.553632, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00213408470154', 'lastCheck': 1384897075.1159239, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897081.206986, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000448942184448', 'lastCheck': 1384897072.550302, 'valid': True, 'code': 0}}
>Thread-15911602::DEBUG::2013-11-19 21:38:02,736::task::495::TaskManager.Task::(_debug) Task 6707ec49-aaa9-4f32-aa29-00b56277220f: moving from state preparing -> state finished
>Thread-15911602::DEBUG::2013-11-19 21:38:02,736::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911602::DEBUG::2013-11-19 21:38:02,737::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911602::DEBUG::2013-11-19 21:38:02,737::task::495::TaskManager.Task::(_debug) Task 6707ec49-aaa9-4f32-aa29-00b56277220f: ref 0 aborting False
>Thread-15911602::INFO::2013-11-19 21:38:02,738::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000331878662109', 'lastCheck': 1384897073.553632, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00213408470154', 'lastCheck': 1384897075.1159239, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897081.206986, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000448942184448', 'lastCheck': 1384897072.550302, 'valid': True, 'code': 0}}
>Dummy-15911465::DEBUG::2013-11-19 21:38:04,081::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:38:04,616::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0699461 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15911465::DEBUG::2013-11-19 21:38:06,631::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:38:07,165::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0757589 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15911606::DEBUG::2013-11-19 21:38:07,314::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911606::INFO::2013-11-19 21:38:07,316::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911606::DEBUG::2013-11-19 21:38:07,317::task::495::TaskManager.Task::(_debug) Task 2f1ce627-5979-4d39-9ec4-254e03a260ce: moving from state init -> state preparing
>Thread-15911606::DEBUG::2013-11-19 21:38:07,318::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=285 spmId=2
>Thread-15911606::DEBUG::2013-11-19 21:38:07,320::task::495::TaskManager.Task::(_debug) Task 2f1ce627-5979-4d39-9ec4-254e03a260ce: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911606::DEBUG::2013-11-19 21:38:07,326::task::495::TaskManager.Task::(_debug) Task 2f1ce627-5979-4d39-9ec4-254e03a260ce: moving from state preparing -> state finished
>Thread-15911606::DEBUG::2013-11-19 21:38:07,329::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911606::DEBUG::2013-11-19 21:38:07,331::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911606::DEBUG::2013-11-19 21:38:07,332::task::495::TaskManager.Task::(_debug) Task 2f1ce627-5979-4d39-9ec4-254e03a260ce: ref 0 aborting False
>Thread-15911606::INFO::2013-11-19 21:38:07,333::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911607::DEBUG::2013-11-19 21:38:07,346::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911607::INFO::2013-11-19 21:38:07,347::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911607::DEBUG::2013-11-19 21:38:07,347::task::495::TaskManager.Task::(_debug) Task 5373b3d9-9178-481e-a32d-6fb8fc324e05: moving from state init -> state preparing
>Thread-15911607::DEBUG::2013-11-19 21:38:07,349::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`97eb92f9-18fa-408c-b1fc-8b51c36a7a1f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911607::DEBUG::2013-11-19 21:38:07,349::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911607::DEBUG::2013-11-19 21:38:07,350::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911607::DEBUG::2013-11-19 21:38:07,351::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`97eb92f9-18fa-408c-b1fc-8b51c36a7a1f`::Granted request
>Thread-15911607::DEBUG::2013-11-19 21:38:07,352::task::495::TaskManager.Task::(_debug) Task 5373b3d9-9178-481e-a32d-6fb8fc324e05: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911607::DEBUG::2013-11-19 21:38:07,352::task::495::TaskManager.Task::(_debug) Task 5373b3d9-9178-481e-a32d-6fb8fc324e05: ref 1 aborting False
>Thread-15911607::WARNING::2013-11-19 21:38:07,354::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911607::DEBUG::2013-11-19 21:38:07,355::task::495::TaskManager.Task::(_debug) Task 5373b3d9-9178-481e-a32d-6fb8fc324e05: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911607::DEBUG::2013-11-19 21:38:07,356::task::495::TaskManager.Task::(_debug) Task 5373b3d9-9178-481e-a32d-6fb8fc324e05: moving from state preparing -> state finished
>Thread-15911607::DEBUG::2013-11-19 21:38:07,357::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911607::DEBUG::2013-11-19 21:38:07,357::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911607::DEBUG::2013-11-19 21:38:07,358::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911607::DEBUG::2013-11-19 21:38:07,359::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911607::DEBUG::2013-11-19 21:38:07,359::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911607::DEBUG::2013-11-19 21:38:07,360::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911607::DEBUG::2013-11-19 21:38:07,361::task::495::TaskManager.Task::(_debug) Task 5373b3d9-9178-481e-a32d-6fb8fc324e05: ref 0 aborting False
>Thread-15911607::INFO::2013-11-19 21:38:07,361::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911465::DEBUG::2013-11-19 21:38:09,187::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:38:09,725::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0706175 s, 14.5 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:38:11,410::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911465::DEBUG::2013-11-19 21:38:11,741::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:38:12,263::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0702749 s, 14.6 MB/s\n'; <rc> = 0
>Thread-15911610::INFO::2013-11-19 21:38:12,911::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911610::DEBUG::2013-11-19 21:38:12,912::task::495::TaskManager.Task::(_debug) Task 8eb48158-64da-4aac-a6b0-8f07b324fe1b: moving from state init -> state preparing
>Thread-15911610::DEBUG::2013-11-19 21:38:12,913::task::495::TaskManager.Task::(_debug) Task 8eb48158-64da-4aac-a6b0-8f07b324fe1b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000566005706787', 'lastCheck': 1384897083.752562, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00152397155762', 'lastCheck': 1384897085.310354, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897091.4113779, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000468969345093', 'lastCheck': 1384897082.7467189, 'valid': True, 'code': 0}}
>Thread-15911610::DEBUG::2013-11-19 21:38:12,914::task::495::TaskManager.Task::(_debug) Task 8eb48158-64da-4aac-a6b0-8f07b324fe1b: moving from state preparing -> state finished
>Thread-15911610::DEBUG::2013-11-19 21:38:12,914::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911610::DEBUG::2013-11-19 21:38:12,915::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911610::DEBUG::2013-11-19 21:38:12,916::task::495::TaskManager.Task::(_debug) Task 8eb48158-64da-4aac-a6b0-8f07b324fe1b: ref 0 aborting False
>Thread-15911610::INFO::2013-11-19 21:38:12,916::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000566005706787', 'lastCheck': 1384897083.752562, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00152397155762', 'lastCheck': 1384897085.310354, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897091.4113779, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000468969345093', 'lastCheck': 1384897082.7467189, 'valid': True, 'code': 0}}
>Dummy-15911465::DEBUG::2013-11-19 21:38:14,279::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:38:14,826::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0705791 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15911465::DEBUG::2013-11-19 21:38:16,841::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:38:17,367::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0701666 s, 14.6 MB/s\n'; <rc> = 0
>Thread-15911614::DEBUG::2013-11-19 21:38:17,472::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911614::INFO::2013-11-19 21:38:17,474::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911614::DEBUG::2013-11-19 21:38:17,475::task::495::TaskManager.Task::(_debug) Task 86040e83-6102-4055-ab64-3ce65dd99598: moving from state init -> state preparing
>Thread-15911614::DEBUG::2013-11-19 21:38:17,476::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=285 spmId=2
>Thread-15911614::DEBUG::2013-11-19 21:38:17,476::task::495::TaskManager.Task::(_debug) Task 86040e83-6102-4055-ab64-3ce65dd99598: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911614::DEBUG::2013-11-19 21:38:17,481::task::495::TaskManager.Task::(_debug) Task 86040e83-6102-4055-ab64-3ce65dd99598: moving from state preparing -> state finished
>Thread-15911614::DEBUG::2013-11-19 21:38:17,482::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911614::DEBUG::2013-11-19 21:38:17,482::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911614::DEBUG::2013-11-19 21:38:17,483::task::495::TaskManager.Task::(_debug) Task 86040e83-6102-4055-ab64-3ce65dd99598: ref 0 aborting False
>Thread-15911614::INFO::2013-11-19 21:38:17,484::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911615::DEBUG::2013-11-19 21:38:17,492::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911615::INFO::2013-11-19 21:38:17,493::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911615::DEBUG::2013-11-19 21:38:17,494::task::495::TaskManager.Task::(_debug) Task ee2442e8-5e97-4aec-ba85-bf1180626a34: moving from state init -> state preparing
>Thread-15911615::DEBUG::2013-11-19 21:38:17,495::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`72e11759-2a2a-4bff-a2c7-2d29e43ea2ac`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911615::DEBUG::2013-11-19 21:38:17,496::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911615::DEBUG::2013-11-19 21:38:17,497::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911615::DEBUG::2013-11-19 21:38:17,498::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`72e11759-2a2a-4bff-a2c7-2d29e43ea2ac`::Granted request
>Thread-15911615::DEBUG::2013-11-19 21:38:17,507::task::495::TaskManager.Task::(_debug) Task ee2442e8-5e97-4aec-ba85-bf1180626a34: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911615::DEBUG::2013-11-19 21:38:17,508::task::495::TaskManager.Task::(_debug) Task ee2442e8-5e97-4aec-ba85-bf1180626a34: ref 1 aborting False
>Thread-15911615::WARNING::2013-11-19 21:38:17,510::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911615::DEBUG::2013-11-19 21:38:17,511::task::495::TaskManager.Task::(_debug) Task ee2442e8-5e97-4aec-ba85-bf1180626a34: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911615::DEBUG::2013-11-19 21:38:17,512::task::495::TaskManager.Task::(_debug) Task ee2442e8-5e97-4aec-ba85-bf1180626a34: moving from state preparing -> state finished
>Thread-15911615::DEBUG::2013-11-19 21:38:17,513::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911615::DEBUG::2013-11-19 21:38:17,513::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911615::DEBUG::2013-11-19 21:38:17,514::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911615::DEBUG::2013-11-19 21:38:17,515::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911615::DEBUG::2013-11-19 21:38:17,515::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911615::DEBUG::2013-11-19 21:38:17,516::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911615::DEBUG::2013-11-19 21:38:17,516::task::495::TaskManager.Task::(_debug) Task ee2442e8-5e97-4aec-ba85-bf1180626a34: ref 0 aborting False
>Thread-15911615::INFO::2013-11-19 21:38:17,517::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911465::DEBUG::2013-11-19 21:38:19,382::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:38:19,909::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0691234 s, 14.8 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:38:21,615::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911465::DEBUG::2013-11-19 21:38:21,924::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:38:22,469::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0702838 s, 14.6 MB/s\n'; <rc> = 0
>Thread-15911618::INFO::2013-11-19 21:38:23,084::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911618::DEBUG::2013-11-19 21:38:23,085::task::495::TaskManager.Task::(_debug) Task 55fa5182-4b64-4d70-a4cd-09cbd3a94388: moving from state init -> state preparing
>Thread-15911618::DEBUG::2013-11-19 21:38:23,086::task::495::TaskManager.Task::(_debug) Task 55fa5182-4b64-4d70-a4cd-09cbd3a94388: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000472068786621', 'lastCheck': 1384897093.950063, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00147795677185', 'lastCheck': 1384897095.5166919, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897101.6159141, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000579833984375', 'lastCheck': 1384897092.945426, 'valid': True, 'code': 0}}
>Thread-15911618::DEBUG::2013-11-19 21:38:23,094::task::495::TaskManager.Task::(_debug) Task 55fa5182-4b64-4d70-a4cd-09cbd3a94388: moving from state preparing -> state finished
>Thread-15911618::DEBUG::2013-11-19 21:38:23,095::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911618::DEBUG::2013-11-19 21:38:23,095::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911618::DEBUG::2013-11-19 21:38:23,096::task::495::TaskManager.Task::(_debug) Task 55fa5182-4b64-4d70-a4cd-09cbd3a94388: ref 0 aborting False
>Thread-15911618::INFO::2013-11-19 21:38:23,099::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000472068786621', 'lastCheck': 1384897093.950063, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00147795677185', 'lastCheck': 1384897095.5166919, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897101.6159141, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000579833984375', 'lastCheck': 1384897092.945426, 'valid': True, 'code': 0}}
>Dummy-15911465::DEBUG::2013-11-19 21:38:24,484::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:38:25,003::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0695959 s, 14.7 MB/s\n'; <rc> = 0
>Dummy-15911465::DEBUG::2013-11-19 21:38:27,018::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:38:27,566::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0723218 s, 14.2 MB/s\n'; <rc> = 0
>Thread-15911622::DEBUG::2013-11-19 21:38:27,655::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911622::INFO::2013-11-19 21:38:27,656::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911622::DEBUG::2013-11-19 21:38:27,657::task::495::TaskManager.Task::(_debug) Task de92b221-539d-4566-8530-b1bd803a4c03: moving from state init -> state preparing
>Thread-15911622::DEBUG::2013-11-19 21:38:27,657::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=285 spmId=2
>Thread-15911622::DEBUG::2013-11-19 21:38:27,658::task::495::TaskManager.Task::(_debug) Task de92b221-539d-4566-8530-b1bd803a4c03: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911622::DEBUG::2013-11-19 21:38:27,659::task::495::TaskManager.Task::(_debug) Task de92b221-539d-4566-8530-b1bd803a4c03: moving from state preparing -> state finished
>Thread-15911622::DEBUG::2013-11-19 21:38:27,659::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911622::DEBUG::2013-11-19 21:38:27,660::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911622::DEBUG::2013-11-19 21:38:27,660::task::495::TaskManager.Task::(_debug) Task de92b221-539d-4566-8530-b1bd803a4c03: ref 0 aborting False
>Thread-15911622::INFO::2013-11-19 21:38:27,661::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911623::DEBUG::2013-11-19 21:38:27,667::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911623::INFO::2013-11-19 21:38:27,668::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911623::DEBUG::2013-11-19 21:38:27,668::task::495::TaskManager.Task::(_debug) Task 939426b8-3427-4444-b5fc-5f2163b86c5c: moving from state init -> state preparing
>Thread-15911623::DEBUG::2013-11-19 21:38:27,669::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d80f834b-b6cd-4aac-83fc-11f52569fce5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911623::DEBUG::2013-11-19 21:38:27,670::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911623::DEBUG::2013-11-19 21:38:27,671::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911623::DEBUG::2013-11-19 21:38:27,672::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d80f834b-b6cd-4aac-83fc-11f52569fce5`::Granted request
>Thread-15911623::DEBUG::2013-11-19 21:38:27,673::task::495::TaskManager.Task::(_debug) Task 939426b8-3427-4444-b5fc-5f2163b86c5c: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911623::DEBUG::2013-11-19 21:38:27,673::task::495::TaskManager.Task::(_debug) Task 939426b8-3427-4444-b5fc-5f2163b86c5c: ref 1 aborting False
>Thread-15911623::WARNING::2013-11-19 21:38:27,675::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911623::DEBUG::2013-11-19 21:38:27,676::task::495::TaskManager.Task::(_debug) Task 939426b8-3427-4444-b5fc-5f2163b86c5c: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911623::DEBUG::2013-11-19 21:38:27,677::task::495::TaskManager.Task::(_debug) Task 939426b8-3427-4444-b5fc-5f2163b86c5c: moving from state preparing -> state finished
>Thread-15911623::DEBUG::2013-11-19 21:38:27,678::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911623::DEBUG::2013-11-19 21:38:27,678::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911623::DEBUG::2013-11-19 21:38:27,679::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911623::DEBUG::2013-11-19 21:38:27,680::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911623::DEBUG::2013-11-19 21:38:27,680::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911623::DEBUG::2013-11-19 21:38:27,681::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911623::DEBUG::2013-11-19 21:38:27,681::task::495::TaskManager.Task::(_debug) Task 939426b8-3427-4444-b5fc-5f2163b86c5c: ref 0 aborting False
>Thread-15911623::INFO::2013-11-19 21:38:27,682::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911465::DEBUG::2013-11-19 21:38:29,581::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:38:30,116::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0698918 s, 14.7 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:38:31,827::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911465::DEBUG::2013-11-19 21:38:32,131::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:38:32,659::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0701147 s, 14.6 MB/s\n'; <rc> = 0
>Thread-15911626::INFO::2013-11-19 21:38:33,367::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911626::DEBUG::2013-11-19 21:38:33,367::task::495::TaskManager.Task::(_debug) Task f05e481a-523f-48b1-9055-1b8f42bc07a1: moving from state init -> state preparing
>Thread-15911626::DEBUG::2013-11-19 21:38:33,368::task::495::TaskManager.Task::(_debug) Task f05e481a-523f-48b1-9055-1b8f42bc07a1: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000432014465332', 'lastCheck': 1384897104.174964, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00143790245056', 'lastCheck': 1384897105.70068, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897111.8288031, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000478982925415', 'lastCheck': 1384897113.357759, 'valid': True, 'code': 0}}
>Thread-15911626::DEBUG::2013-11-19 21:38:33,374::task::495::TaskManager.Task::(_debug) Task f05e481a-523f-48b1-9055-1b8f42bc07a1: moving from state preparing -> state finished
>Thread-15911626::DEBUG::2013-11-19 21:38:33,375::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911626::DEBUG::2013-11-19 21:38:33,375::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911626::DEBUG::2013-11-19 21:38:33,376::task::495::TaskManager.Task::(_debug) Task f05e481a-523f-48b1-9055-1b8f42bc07a1: ref 0 aborting False
>Thread-15911626::INFO::2013-11-19 21:38:33,378::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000432014465332', 'lastCheck': 1384897104.174964, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00143790245056', 'lastCheck': 1384897105.70068, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897111.8288031, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000478982925415', 'lastCheck': 1384897113.357759, 'valid': True, 'code': 0}}
>Dummy-15911465::DEBUG::2013-11-19 21:38:34,675::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:38:35,215::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0702658 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15911465::DEBUG::2013-11-19 21:38:37,230::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:38:37,744::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0696683 s, 14.7 MB/s\n'; <rc> = 0
>Thread-15911630::DEBUG::2013-11-19 21:38:37,891::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911630::INFO::2013-11-19 21:38:37,892::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911630::DEBUG::2013-11-19 21:38:37,893::task::495::TaskManager.Task::(_debug) Task 95edae5c-b7d8-48f0-8f17-b0cc8ffbf0f1: moving from state init -> state preparing
>Thread-15911630::DEBUG::2013-11-19 21:38:37,894::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=285 spmId=2
>Thread-15911630::DEBUG::2013-11-19 21:38:37,894::task::495::TaskManager.Task::(_debug) Task 95edae5c-b7d8-48f0-8f17-b0cc8ffbf0f1: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911630::DEBUG::2013-11-19 21:38:37,895::task::495::TaskManager.Task::(_debug) Task 95edae5c-b7d8-48f0-8f17-b0cc8ffbf0f1: moving from state preparing -> state finished
>Thread-15911630::DEBUG::2013-11-19 21:38:37,896::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911630::DEBUG::2013-11-19 21:38:37,896::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911630::DEBUG::2013-11-19 21:38:37,897::task::495::TaskManager.Task::(_debug) Task 95edae5c-b7d8-48f0-8f17-b0cc8ffbf0f1: ref 0 aborting False
>Thread-15911630::INFO::2013-11-19 21:38:37,897::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911631::DEBUG::2013-11-19 21:38:37,903::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911631::INFO::2013-11-19 21:38:37,904::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911631::DEBUG::2013-11-19 21:38:37,905::task::495::TaskManager.Task::(_debug) Task 68a69219-678b-482f-a8bf-3657ed6b2029: moving from state init -> state preparing
>Thread-15911631::DEBUG::2013-11-19 21:38:37,908::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`18e0c9b0-64dd-4bd3-949d-2f0c9ae9527b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911631::DEBUG::2013-11-19 21:38:37,909::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911631::DEBUG::2013-11-19 21:38:37,911::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911631::DEBUG::2013-11-19 21:38:37,912::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`18e0c9b0-64dd-4bd3-949d-2f0c9ae9527b`::Granted request
>Thread-15911631::DEBUG::2013-11-19 21:38:37,913::task::495::TaskManager.Task::(_debug) Task 68a69219-678b-482f-a8bf-3657ed6b2029: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911631::DEBUG::2013-11-19 21:38:37,914::task::495::TaskManager.Task::(_debug) Task 68a69219-678b-482f-a8bf-3657ed6b2029: ref 1 aborting False
>Thread-15911631::WARNING::2013-11-19 21:38:37,916::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911631::DEBUG::2013-11-19 21:38:37,917::task::495::TaskManager.Task::(_debug) Task 68a69219-678b-482f-a8bf-3657ed6b2029: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911631::DEBUG::2013-11-19 21:38:37,918::task::495::TaskManager.Task::(_debug) Task 68a69219-678b-482f-a8bf-3657ed6b2029: moving from state preparing -> state finished
>Thread-15911631::DEBUG::2013-11-19 21:38:37,918::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911631::DEBUG::2013-11-19 21:38:37,919::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911631::DEBUG::2013-11-19 21:38:37,919::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911631::DEBUG::2013-11-19 21:38:37,920::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911631::DEBUG::2013-11-19 21:38:37,921::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911631::DEBUG::2013-11-19 21:38:37,921::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911631::DEBUG::2013-11-19 21:38:37,922::task::495::TaskManager.Task::(_debug) Task 68a69219-678b-482f-a8bf-3657ed6b2029: ref 0 aborting False
>Thread-15911631::INFO::2013-11-19 21:38:37,922::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911465::DEBUG::2013-11-19 21:38:39,758::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:38:40,302::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0932622 s, 11.0 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:38:41,986::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911465::DEBUG::2013-11-19 21:38:42,317::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:38:42,844::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0702656 s, 14.6 MB/s\n'; <rc> = 0
>Thread-15911634::INFO::2013-11-19 21:38:43,529::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911634::DEBUG::2013-11-19 21:38:43,530::task::495::TaskManager.Task::(_debug) Task 1c16cf1f-e0c5-4b94-aa0a-fbb122c25f52: moving from state init -> state preparing
>Thread-15911634::DEBUG::2013-11-19 21:38:43,531::task::495::TaskManager.Task::(_debug) Task 1c16cf1f-e0c5-4b94-aa0a-fbb122c25f52: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000480890274048', 'lastCheck': 1384897114.3604419, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00146198272705', 'lastCheck': 1384897115.9116211, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897121.9870801, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000478982925415', 'lastCheck': 1384897113.357759, 'valid': True, 'code': 0}}
>Thread-15911634::DEBUG::2013-11-19 21:38:43,532::task::495::TaskManager.Task::(_debug) Task 1c16cf1f-e0c5-4b94-aa0a-fbb122c25f52: moving from state preparing -> state finished
>Thread-15911634::DEBUG::2013-11-19 21:38:43,532::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911634::DEBUG::2013-11-19 21:38:43,533::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911634::DEBUG::2013-11-19 21:38:43,534::task::495::TaskManager.Task::(_debug) Task 1c16cf1f-e0c5-4b94-aa0a-fbb122c25f52: ref 0 aborting False
>Thread-15911634::INFO::2013-11-19 21:38:43,537::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000480890274048', 'lastCheck': 1384897114.3604419, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00146198272705', 'lastCheck': 1384897115.9116211, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897121.9870801, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000478982925415', 'lastCheck': 1384897113.357759, 'valid': True, 'code': 0}}
>Dummy-15911465::DEBUG::2013-11-19 21:38:44,860::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:38:45,376::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.069632 s, 14.7 MB/s\n'; <rc> = 0
>Dummy-15911465::DEBUG::2013-11-19 21:38:47,391::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:38:47,961::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.092682 s, 11.0 MB/s\n'; <rc> = 0
>Thread-15911638::DEBUG::2013-11-19 21:38:48,035::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911638::INFO::2013-11-19 21:38:48,036::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911638::DEBUG::2013-11-19 21:38:48,037::task::495::TaskManager.Task::(_debug) Task 24310a4a-f35b-487a-83b9-838675708daf: moving from state init -> state preparing
>Thread-15911638::DEBUG::2013-11-19 21:38:48,038::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=285 spmId=2
>Thread-15911638::DEBUG::2013-11-19 21:38:48,038::task::495::TaskManager.Task::(_debug) Task 24310a4a-f35b-487a-83b9-838675708daf: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911638::DEBUG::2013-11-19 21:38:48,039::task::495::TaskManager.Task::(_debug) Task 24310a4a-f35b-487a-83b9-838675708daf: moving from state preparing -> state finished
>Thread-15911638::DEBUG::2013-11-19 21:38:48,040::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911638::DEBUG::2013-11-19 21:38:48,040::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911638::DEBUG::2013-11-19 21:38:48,041::task::495::TaskManager.Task::(_debug) Task 24310a4a-f35b-487a-83b9-838675708daf: ref 0 aborting False
>Thread-15911638::INFO::2013-11-19 21:38:48,041::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911639::DEBUG::2013-11-19 21:38:48,047::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911639::INFO::2013-11-19 21:38:48,048::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911639::DEBUG::2013-11-19 21:38:48,049::task::495::TaskManager.Task::(_debug) Task ad658633-005b-47e4-9d76-060379e09e66: moving from state init -> state preparing
>Thread-15911639::DEBUG::2013-11-19 21:38:48,050::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c8705395-8c55-4c3c-b842-bbef989c6079`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911639::DEBUG::2013-11-19 21:38:48,050::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911639::DEBUG::2013-11-19 21:38:48,051::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911639::DEBUG::2013-11-19 21:38:48,052::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c8705395-8c55-4c3c-b842-bbef989c6079`::Granted request
>Thread-15911639::DEBUG::2013-11-19 21:38:48,053::task::495::TaskManager.Task::(_debug) Task ad658633-005b-47e4-9d76-060379e09e66: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911639::DEBUG::2013-11-19 21:38:48,053::task::495::TaskManager.Task::(_debug) Task ad658633-005b-47e4-9d76-060379e09e66: ref 1 aborting False
>Thread-15911639::WARNING::2013-11-19 21:38:48,056::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911639::DEBUG::2013-11-19 21:38:48,056::task::495::TaskManager.Task::(_debug) Task ad658633-005b-47e4-9d76-060379e09e66: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911639::DEBUG::2013-11-19 21:38:48,057::task::495::TaskManager.Task::(_debug) Task ad658633-005b-47e4-9d76-060379e09e66: moving from state preparing -> state finished
>Thread-15911639::DEBUG::2013-11-19 21:38:48,058::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911639::DEBUG::2013-11-19 21:38:48,059::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911639::DEBUG::2013-11-19 21:38:48,059::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911639::DEBUG::2013-11-19 21:38:48,060::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911639::DEBUG::2013-11-19 21:38:48,061::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911639::DEBUG::2013-11-19 21:38:48,061::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911639::DEBUG::2013-11-19 21:38:48,063::task::495::TaskManager.Task::(_debug) Task ad658633-005b-47e4-9d76-060379e09e66: ref 0 aborting False
>Thread-15911639::INFO::2013-11-19 21:38:48,063::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911465::DEBUG::2013-11-19 21:38:49,977::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:38:50,498::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0702064 s, 14.6 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:38:52,193::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911465::DEBUG::2013-11-19 21:38:52,513::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:38:53,038::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0703057 s, 14.6 MB/s\n'; <rc> = 0
>Thread-15911642::INFO::2013-11-19 21:38:53,696::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911642::DEBUG::2013-11-19 21:38:53,697::task::495::TaskManager.Task::(_debug) Task b6741232-ad8a-409e-a5ed-6c59b9d6ffd5: moving from state init -> state preparing
>Thread-15911642::DEBUG::2013-11-19 21:38:53,697::task::495::TaskManager.Task::(_debug) Task b6741232-ad8a-409e-a5ed-6c59b9d6ffd5: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000468969345093', 'lastCheck': 1384897124.5587959, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00146889686584', 'lastCheck': 1384897126.065712, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897132.1946211, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0010199546814', 'lastCheck': 1384897123.5402679, 'valid': True, 'code': 0}}
>Thread-15911642::DEBUG::2013-11-19 21:38:53,698::task::495::TaskManager.Task::(_debug) Task b6741232-ad8a-409e-a5ed-6c59b9d6ffd5: moving from state preparing -> state finished
>Thread-15911642::DEBUG::2013-11-19 21:38:53,699::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911642::DEBUG::2013-11-19 21:38:53,699::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911642::DEBUG::2013-11-19 21:38:53,700::task::495::TaskManager.Task::(_debug) Task b6741232-ad8a-409e-a5ed-6c59b9d6ffd5: ref 0 aborting False
>Thread-15911642::INFO::2013-11-19 21:38:53,701::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000468969345093', 'lastCheck': 1384897124.5587959, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00146889686584', 'lastCheck': 1384897126.065712, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897132.1946211, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0010199546814', 'lastCheck': 1384897123.5402679, 'valid': True, 'code': 0}}
>Dummy-15911465::DEBUG::2013-11-19 21:38:55,053::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:38:55,601::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0707513 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15911465::DEBUG::2013-11-19 21:38:57,615::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:38:58,144::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.070456 s, 14.5 MB/s\n'; <rc> = 0
>Thread-15911646::DEBUG::2013-11-19 21:38:58,171::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911646::INFO::2013-11-19 21:38:58,172::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911646::DEBUG::2013-11-19 21:38:58,173::task::495::TaskManager.Task::(_debug) Task 28d670b0-062b-4f77-9c70-637663158907: moving from state init -> state preparing
>Thread-15911646::DEBUG::2013-11-19 21:38:58,174::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=285 spmId=2
>Thread-15911646::DEBUG::2013-11-19 21:38:58,174::task::495::TaskManager.Task::(_debug) Task 28d670b0-062b-4f77-9c70-637663158907: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911646::DEBUG::2013-11-19 21:38:58,175::task::495::TaskManager.Task::(_debug) Task 28d670b0-062b-4f77-9c70-637663158907: moving from state preparing -> state finished
>Thread-15911646::DEBUG::2013-11-19 21:38:58,175::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911646::DEBUG::2013-11-19 21:38:58,176::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911646::DEBUG::2013-11-19 21:38:58,177::task::495::TaskManager.Task::(_debug) Task 28d670b0-062b-4f77-9c70-637663158907: ref 0 aborting False
>Thread-15911646::INFO::2013-11-19 21:38:58,177::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911647::DEBUG::2013-11-19 21:38:58,187::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911647::INFO::2013-11-19 21:38:58,188::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911647::DEBUG::2013-11-19 21:38:58,189::task::495::TaskManager.Task::(_debug) Task 97b0512e-f85e-4ac5-8eb5-8ac744c78bc0: moving from state init -> state preparing
>Thread-15911647::DEBUG::2013-11-19 21:38:58,190::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2710bc18-f98c-4574-8f71-af717ec01372`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911647::DEBUG::2013-11-19 21:38:58,190::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911647::DEBUG::2013-11-19 21:38:58,191::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911647::DEBUG::2013-11-19 21:38:58,192::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2710bc18-f98c-4574-8f71-af717ec01372`::Granted request
>Thread-15911647::DEBUG::2013-11-19 21:38:58,193::task::495::TaskManager.Task::(_debug) Task 97b0512e-f85e-4ac5-8eb5-8ac744c78bc0: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911647::DEBUG::2013-11-19 21:38:58,193::task::495::TaskManager.Task::(_debug) Task 97b0512e-f85e-4ac5-8eb5-8ac744c78bc0: ref 1 aborting False
>Thread-15911647::WARNING::2013-11-19 21:38:58,195::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911647::DEBUG::2013-11-19 21:38:58,196::task::495::TaskManager.Task::(_debug) Task 97b0512e-f85e-4ac5-8eb5-8ac744c78bc0: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911647::DEBUG::2013-11-19 21:38:58,197::task::495::TaskManager.Task::(_debug) Task 97b0512e-f85e-4ac5-8eb5-8ac744c78bc0: moving from state preparing -> state finished
>Thread-15911647::DEBUG::2013-11-19 21:38:58,198::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911647::DEBUG::2013-11-19 21:38:58,199::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911647::DEBUG::2013-11-19 21:38:58,199::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911647::DEBUG::2013-11-19 21:38:58,200::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911647::DEBUG::2013-11-19 21:38:58,200::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911647::DEBUG::2013-11-19 21:38:58,201::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911647::DEBUG::2013-11-19 21:38:58,202::task::495::TaskManager.Task::(_debug) Task 97b0512e-f85e-4ac5-8eb5-8ac744c78bc0: ref 0 aborting False
>Thread-15911647::INFO::2013-11-19 21:38:58,202::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133593825280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911465::DEBUG::2013-11-19 21:39:00,159::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:39:00,717::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0742684 s, 13.8 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:39:02,394::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911465::DEBUG::2013-11-19 21:39:02,732::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:39:03,252::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0700733 s, 14.6 MB/s\n'; <rc> = 0
>Thread-15911650::INFO::2013-11-19 21:39:03,861::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911650::DEBUG::2013-11-19 21:39:03,862::task::495::TaskManager.Task::(_debug) Task 21ae5293-2afe-47f7-9ad4-7eb692e7e58a: moving from state init -> state preparing
>Thread-15911650::DEBUG::2013-11-19 21:39:03,862::task::495::TaskManager.Task::(_debug) Task 21ae5293-2afe-47f7-9ad4-7eb692e7e58a: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000375032424927', 'lastCheck': 1384897134.7402501, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00218987464905', 'lastCheck': 1384897136.29813, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897142.3955081, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000626087188721', 'lastCheck': 1384897133.73611, 'valid': True, 'code': 0}}
>Thread-15911650::DEBUG::2013-11-19 21:39:03,868::task::495::TaskManager.Task::(_debug) Task 21ae5293-2afe-47f7-9ad4-7eb692e7e58a: moving from state preparing -> state finished
>Thread-15911650::DEBUG::2013-11-19 21:39:03,869::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911650::DEBUG::2013-11-19 21:39:03,869::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911650::DEBUG::2013-11-19 21:39:03,870::task::495::TaskManager.Task::(_debug) Task 21ae5293-2afe-47f7-9ad4-7eb692e7e58a: ref 0 aborting False
>Thread-15911650::INFO::2013-11-19 21:39:03,871::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000375032424927', 'lastCheck': 1384897134.7402501, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00218987464905', 'lastCheck': 1384897136.29813, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897142.3955081, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000626087188721', 'lastCheck': 1384897133.73611, 'valid': True, 'code': 0}}
>Dummy-15911465::DEBUG::2013-11-19 21:39:05,267::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:39:05,814::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0708154 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15911465::DEBUG::2013-11-19 21:39:07,830::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:39:08,364::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0702079 s, 14.6 MB/s\n'; <rc> = 0
>Thread-15911654::DEBUG::2013-11-19 21:39:09,037::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911654::INFO::2013-11-19 21:39:09,038::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911654::DEBUG::2013-11-19 21:39:09,039::task::495::TaskManager.Task::(_debug) Task 7e03dc26-7808-44f4-9d49-3cccd5518b04: moving from state init -> state preparing
>Thread-15911654::DEBUG::2013-11-19 21:39:09,040::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=285 spmId=2
>Thread-15911654::DEBUG::2013-11-19 21:39:09,040::task::495::TaskManager.Task::(_debug) Task 7e03dc26-7808-44f4-9d49-3cccd5518b04: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911654::DEBUG::2013-11-19 21:39:09,041::task::495::TaskManager.Task::(_debug) Task 7e03dc26-7808-44f4-9d49-3cccd5518b04: moving from state preparing -> state finished
>Thread-15911654::DEBUG::2013-11-19 21:39:09,041::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911654::DEBUG::2013-11-19 21:39:09,042::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911654::DEBUG::2013-11-19 21:39:09,043::task::495::TaskManager.Task::(_debug) Task 7e03dc26-7808-44f4-9d49-3cccd5518b04: ref 0 aborting False
>Thread-15911654::INFO::2013-11-19 21:39:09,043::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911655::DEBUG::2013-11-19 21:39:09,056::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911655::INFO::2013-11-19 21:39:09,057::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911655::DEBUG::2013-11-19 21:39:09,058::task::495::TaskManager.Task::(_debug) Task c17cdaa8-918b-46df-956c-5fbc9f859c8f: moving from state init -> state preparing
>Thread-15911655::DEBUG::2013-11-19 21:39:09,059::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`fe563f78-4cf6-4edf-be88-bd503a13b822`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911655::DEBUG::2013-11-19 21:39:09,060::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911655::DEBUG::2013-11-19 21:39:09,060::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911655::DEBUG::2013-11-19 21:39:09,061::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`fe563f78-4cf6-4edf-be88-bd503a13b822`::Granted request
>Thread-15911655::DEBUG::2013-11-19 21:39:09,062::task::495::TaskManager.Task::(_debug) Task c17cdaa8-918b-46df-956c-5fbc9f859c8f: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911655::DEBUG::2013-11-19 21:39:09,063::task::495::TaskManager.Task::(_debug) Task c17cdaa8-918b-46df-956c-5fbc9f859c8f: ref 1 aborting False
>Thread-15911655::WARNING::2013-11-19 21:39:09,065::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911655::DEBUG::2013-11-19 21:39:09,067::task::495::TaskManager.Task::(_debug) Task c17cdaa8-918b-46df-956c-5fbc9f859c8f: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133970264064', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911655::DEBUG::2013-11-19 21:39:09,068::task::495::TaskManager.Task::(_debug) Task c17cdaa8-918b-46df-956c-5fbc9f859c8f: moving from state preparing -> state finished
>Thread-15911655::DEBUG::2013-11-19 21:39:09,068::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911655::DEBUG::2013-11-19 21:39:09,069::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911655::DEBUG::2013-11-19 21:39:09,070::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911655::DEBUG::2013-11-19 21:39:09,070::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911655::DEBUG::2013-11-19 21:39:09,071::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911655::DEBUG::2013-11-19 21:39:09,071::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911655::DEBUG::2013-11-19 21:39:09,072::task::495::TaskManager.Task::(_debug) Task c17cdaa8-918b-46df-956c-5fbc9f859c8f: ref 0 aborting False
>Thread-15911655::INFO::2013-11-19 21:39:09,073::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133970264064', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911465::DEBUG::2013-11-19 21:39:10,379::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:39:10,913::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0701843 s, 14.6 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:39:12,619::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911465::DEBUG::2013-11-19 21:39:12,929::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:39:13,472::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0705942 s, 14.5 MB/s\n'; <rc> = 0
>Thread-15911658::INFO::2013-11-19 21:39:14,086::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911658::DEBUG::2013-11-19 21:39:14,087::task::495::TaskManager.Task::(_debug) Task feb0cce4-e1ae-4175-b44e-467ed44f8765: moving from state init -> state preparing
>Thread-15911658::DEBUG::2013-11-19 21:39:14,098::task::495::TaskManager.Task::(_debug) Task feb0cce4-e1ae-4175-b44e-467ed44f8765: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000389099121094', 'lastCheck': 1384897144.9570501, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014750957489', 'lastCheck': 1384897146.5063679, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897152.6211431, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000453948974609', 'lastCheck': 1384897143.9518919, 'valid': True, 'code': 0}}
>Thread-15911658::DEBUG::2013-11-19 21:39:14,105::task::495::TaskManager.Task::(_debug) Task feb0cce4-e1ae-4175-b44e-467ed44f8765: moving from state preparing -> state finished
>Thread-15911658::DEBUG::2013-11-19 21:39:14,105::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911658::DEBUG::2013-11-19 21:39:14,106::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911658::DEBUG::2013-11-19 21:39:14,106::task::495::TaskManager.Task::(_debug) Task feb0cce4-e1ae-4175-b44e-467ed44f8765: ref 0 aborting False
>Thread-15911658::INFO::2013-11-19 21:39:14,107::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000389099121094', 'lastCheck': 1384897144.9570501, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014750957489', 'lastCheck': 1384897146.5063679, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897152.6211431, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000453948974609', 'lastCheck': 1384897143.9518919, 'valid': True, 'code': 0}}
>Dummy-15911465::DEBUG::2013-11-19 21:39:15,487::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:39:16,014::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0705622 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15911465::DEBUG::2013-11-19 21:39:18,029::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:39:18,574::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0705349 s, 14.5 MB/s\n'; <rc> = 0
>Thread-15911662::DEBUG::2013-11-19 21:39:19,193::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911662::INFO::2013-11-19 21:39:19,194::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911662::DEBUG::2013-11-19 21:39:19,195::task::495::TaskManager.Task::(_debug) Task de884029-eabe-4e28-a808-358292ce349d: moving from state init -> state preparing
>Thread-15911662::DEBUG::2013-11-19 21:39:19,196::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=285 spmId=2
>Thread-15911662::DEBUG::2013-11-19 21:39:19,197::task::495::TaskManager.Task::(_debug) Task de884029-eabe-4e28-a808-358292ce349d: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911662::DEBUG::2013-11-19 21:39:19,197::task::495::TaskManager.Task::(_debug) Task de884029-eabe-4e28-a808-358292ce349d: moving from state preparing -> state finished
>Thread-15911662::DEBUG::2013-11-19 21:39:19,198::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911662::DEBUG::2013-11-19 21:39:19,198::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911662::DEBUG::2013-11-19 21:39:19,199::task::495::TaskManager.Task::(_debug) Task de884029-eabe-4e28-a808-358292ce349d: ref 0 aborting False
>Thread-15911662::INFO::2013-11-19 21:39:19,199::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911663::DEBUG::2013-11-19 21:39:19,205::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911663::INFO::2013-11-19 21:39:19,206::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911663::DEBUG::2013-11-19 21:39:19,207::task::495::TaskManager.Task::(_debug) Task 43817885-0a6a-46c1-938e-996a34bac0f0: moving from state init -> state preparing
>Thread-15911663::DEBUG::2013-11-19 21:39:19,208::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`868d70d8-9b97-4680-9f38-6785ec70b9f7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911663::DEBUG::2013-11-19 21:39:19,208::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911663::DEBUG::2013-11-19 21:39:19,209::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911663::DEBUG::2013-11-19 21:39:19,210::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`868d70d8-9b97-4680-9f38-6785ec70b9f7`::Granted request
>Thread-15911663::DEBUG::2013-11-19 21:39:19,211::task::495::TaskManager.Task::(_debug) Task 43817885-0a6a-46c1-938e-996a34bac0f0: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911663::DEBUG::2013-11-19 21:39:19,211::task::495::TaskManager.Task::(_debug) Task 43817885-0a6a-46c1-938e-996a34bac0f0: ref 1 aborting False
>Thread-15911663::WARNING::2013-11-19 21:39:19,214::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911663::DEBUG::2013-11-19 21:39:19,214::task::495::TaskManager.Task::(_debug) Task 43817885-0a6a-46c1-938e-996a34bac0f0: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133842337792', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911663::DEBUG::2013-11-19 21:39:19,215::task::495::TaskManager.Task::(_debug) Task 43817885-0a6a-46c1-938e-996a34bac0f0: moving from state preparing -> state finished
>Thread-15911663::DEBUG::2013-11-19 21:39:19,216::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911663::DEBUG::2013-11-19 21:39:19,217::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911663::DEBUG::2013-11-19 21:39:19,217::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911663::DEBUG::2013-11-19 21:39:19,218::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911663::DEBUG::2013-11-19 21:39:19,218::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911663::DEBUG::2013-11-19 21:39:19,219::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911663::DEBUG::2013-11-19 21:39:19,220::task::495::TaskManager.Task::(_debug) Task 43817885-0a6a-46c1-938e-996a34bac0f0: ref 0 aborting False
>Thread-15911663::INFO::2013-11-19 21:39:19,220::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133842337792', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911465::DEBUG::2013-11-19 21:39:20,590::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:39:21,117::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0700825 s, 14.6 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:39:22,803::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911465::DEBUG::2013-11-19 21:39:23,132::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:39:23,652::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0695724 s, 14.7 MB/s\n'; <rc> = 0
>Thread-15911666::INFO::2013-11-19 21:39:24,286::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911666::DEBUG::2013-11-19 21:39:24,287::task::495::TaskManager.Task::(_debug) Task 51f57657-f5cc-4782-85b8-94603d939b1b: moving from state init -> state preparing
>Thread-15911666::DEBUG::2013-11-19 21:39:24,297::task::495::TaskManager.Task::(_debug) Task 51f57657-f5cc-4782-85b8-94603d939b1b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000333070755005', 'lastCheck': 1384897155.1538999, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00152492523193', 'lastCheck': 1384897156.7083681, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897162.804759, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000536203384399', 'lastCheck': 1384897154.1495919, 'valid': True, 'code': 0}}
>Thread-15911666::DEBUG::2013-11-19 21:39:24,298::task::495::TaskManager.Task::(_debug) Task 51f57657-f5cc-4782-85b8-94603d939b1b: moving from state preparing -> state finished
>Thread-15911666::DEBUG::2013-11-19 21:39:24,298::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911666::DEBUG::2013-11-19 21:39:24,299::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911666::DEBUG::2013-11-19 21:39:24,299::task::495::TaskManager.Task::(_debug) Task 51f57657-f5cc-4782-85b8-94603d939b1b: ref 0 aborting False
>Thread-15911666::INFO::2013-11-19 21:39:24,300::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000333070755005', 'lastCheck': 1384897155.1538999, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00152492523193', 'lastCheck': 1384897156.7083681, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897162.804759, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000536203384399', 'lastCheck': 1384897154.1495919, 'valid': True, 'code': 0}}
>Dummy-15911465::DEBUG::2013-11-19 21:39:25,667::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:39:26,212::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0706747 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15911465::DEBUG::2013-11-19 21:39:28,228::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:39:28,753::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0710031 s, 14.4 MB/s\n'; <rc> = 0
>Thread-15911670::DEBUG::2013-11-19 21:39:29,346::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911670::INFO::2013-11-19 21:39:29,348::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911670::DEBUG::2013-11-19 21:39:29,356::task::495::TaskManager.Task::(_debug) Task c6c4c91c-47f1-4e21-9ed9-18c4f2d753ee: moving from state init -> state preparing
>Thread-15911670::DEBUG::2013-11-19 21:39:29,357::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=285 spmId=2
>Thread-15911670::DEBUG::2013-11-19 21:39:29,357::task::495::TaskManager.Task::(_debug) Task c6c4c91c-47f1-4e21-9ed9-18c4f2d753ee: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911670::DEBUG::2013-11-19 21:39:29,358::task::495::TaskManager.Task::(_debug) Task c6c4c91c-47f1-4e21-9ed9-18c4f2d753ee: moving from state preparing -> state finished
>Thread-15911670::DEBUG::2013-11-19 21:39:29,358::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911670::DEBUG::2013-11-19 21:39:29,359::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911670::DEBUG::2013-11-19 21:39:29,359::task::495::TaskManager.Task::(_debug) Task c6c4c91c-47f1-4e21-9ed9-18c4f2d753ee: ref 0 aborting False
>Thread-15911670::INFO::2013-11-19 21:39:29,360::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911671::DEBUG::2013-11-19 21:39:29,366::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911671::INFO::2013-11-19 21:39:29,367::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911671::DEBUG::2013-11-19 21:39:29,367::task::495::TaskManager.Task::(_debug) Task b7410507-5454-441e-a9eb-be07049e27da: moving from state init -> state preparing
>Thread-15911671::DEBUG::2013-11-19 21:39:29,369::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f1a0ccee-1b7a-4942-b5fb-5980f5421e45`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911671::DEBUG::2013-11-19 21:39:29,369::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911671::DEBUG::2013-11-19 21:39:29,370::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911671::DEBUG::2013-11-19 21:39:29,371::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f1a0ccee-1b7a-4942-b5fb-5980f5421e45`::Granted request
>Thread-15911671::DEBUG::2013-11-19 21:39:29,372::task::495::TaskManager.Task::(_debug) Task b7410507-5454-441e-a9eb-be07049e27da: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911671::DEBUG::2013-11-19 21:39:29,373::task::495::TaskManager.Task::(_debug) Task b7410507-5454-441e-a9eb-be07049e27da: ref 1 aborting False
>Thread-15911671::WARNING::2013-11-19 21:39:29,375::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911671::DEBUG::2013-11-19 21:39:29,376::task::495::TaskManager.Task::(_debug) Task b7410507-5454-441e-a9eb-be07049e27da: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133804589056', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911671::DEBUG::2013-11-19 21:39:29,377::task::495::TaskManager.Task::(_debug) Task b7410507-5454-441e-a9eb-be07049e27da: moving from state preparing -> state finished
>Thread-15911671::DEBUG::2013-11-19 21:39:29,377::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911671::DEBUG::2013-11-19 21:39:29,378::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911671::DEBUG::2013-11-19 21:39:29,378::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911671::DEBUG::2013-11-19 21:39:29,379::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911671::DEBUG::2013-11-19 21:39:29,380::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911671::DEBUG::2013-11-19 21:39:29,380::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911671::DEBUG::2013-11-19 21:39:29,381::task::495::TaskManager.Task::(_debug) Task b7410507-5454-441e-a9eb-be07049e27da: ref 0 aborting False
>Thread-15911671::INFO::2013-11-19 21:39:29,381::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133804589056', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911465::DEBUG::2013-11-19 21:39:30,768::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:39:31,314::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0705928 s, 14.5 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:39:33,005::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911465::DEBUG::2013-11-19 21:39:33,330::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:39:33,855::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0744026 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15911674::INFO::2013-11-19 21:39:34,499::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911674::DEBUG::2013-11-19 21:39:34,500::task::495::TaskManager.Task::(_debug) Task 98d62e51-14c3-4309-941d-140b4ffb05e8: moving from state init -> state preparing
>Thread-15911674::DEBUG::2013-11-19 21:39:34,511::task::495::TaskManager.Task::(_debug) Task 98d62e51-14c3-4309-941d-140b4ffb05e8: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000287055969238', 'lastCheck': 1384897165.3577609, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148582458496', 'lastCheck': 1384897166.896399, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897173.006453, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000479936599731', 'lastCheck': 1384897164.354795, 'valid': True, 'code': 0}}
>Thread-15911674::DEBUG::2013-11-19 21:39:34,512::task::495::TaskManager.Task::(_debug) Task 98d62e51-14c3-4309-941d-140b4ffb05e8: moving from state preparing -> state finished
>Thread-15911674::DEBUG::2013-11-19 21:39:34,513::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911674::DEBUG::2013-11-19 21:39:34,513::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911674::DEBUG::2013-11-19 21:39:34,514::task::495::TaskManager.Task::(_debug) Task 98d62e51-14c3-4309-941d-140b4ffb05e8: ref 0 aborting False
>Thread-15911674::INFO::2013-11-19 21:39:34,515::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000287055969238', 'lastCheck': 1384897165.3577609, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148582458496', 'lastCheck': 1384897166.896399, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897173.006453, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000479936599731', 'lastCheck': 1384897164.354795, 'valid': True, 'code': 0}}
>Dummy-15911465::DEBUG::2013-11-19 21:39:35,871::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:39:36,382::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0694911 s, 14.7 MB/s\n'; <rc> = 0
>Dummy-15911465::DEBUG::2013-11-19 21:39:38,397::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:39:38,932::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0701873 s, 14.6 MB/s\n'; <rc> = 0
>Thread-15911678::DEBUG::2013-11-19 21:39:39,523::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911678::INFO::2013-11-19 21:39:39,524::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911678::DEBUG::2013-11-19 21:39:39,534::task::495::TaskManager.Task::(_debug) Task 9b1d5a60-2722-40eb-b5b6-98d37090a666: moving from state init -> state preparing
>Thread-15911678::DEBUG::2013-11-19 21:39:39,535::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=285 spmId=2
>Thread-15911678::DEBUG::2013-11-19 21:39:39,536::task::495::TaskManager.Task::(_debug) Task 9b1d5a60-2722-40eb-b5b6-98d37090a666: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911678::DEBUG::2013-11-19 21:39:39,536::task::495::TaskManager.Task::(_debug) Task 9b1d5a60-2722-40eb-b5b6-98d37090a666: moving from state preparing -> state finished
>Thread-15911678::DEBUG::2013-11-19 21:39:39,537::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911678::DEBUG::2013-11-19 21:39:39,537::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911678::DEBUG::2013-11-19 21:39:39,538::task::495::TaskManager.Task::(_debug) Task 9b1d5a60-2722-40eb-b5b6-98d37090a666: ref 0 aborting False
>Thread-15911678::INFO::2013-11-19 21:39:39,539::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911679::DEBUG::2013-11-19 21:39:39,550::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911679::INFO::2013-11-19 21:39:39,551::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911679::DEBUG::2013-11-19 21:39:39,552::task::495::TaskManager.Task::(_debug) Task 824e2afb-d628-4688-88df-e090356c6f9f: moving from state init -> state preparing
>Thread-15911679::DEBUG::2013-11-19 21:39:39,553::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9215f267-ddf3-41e9-8367-51338108095a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911679::DEBUG::2013-11-19 21:39:39,554::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911679::DEBUG::2013-11-19 21:39:39,554::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911679::DEBUG::2013-11-19 21:39:39,555::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9215f267-ddf3-41e9-8367-51338108095a`::Granted request
>Thread-15911679::DEBUG::2013-11-19 21:39:39,556::task::495::TaskManager.Task::(_debug) Task 824e2afb-d628-4688-88df-e090356c6f9f: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911679::DEBUG::2013-11-19 21:39:39,557::task::495::TaskManager.Task::(_debug) Task 824e2afb-d628-4688-88df-e090356c6f9f: ref 1 aborting False
>Thread-15911679::WARNING::2013-11-19 21:39:39,559::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911679::DEBUG::2013-11-19 21:39:39,560::task::495::TaskManager.Task::(_debug) Task 824e2afb-d628-4688-88df-e090356c6f9f: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133801443328', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911679::DEBUG::2013-11-19 21:39:39,561::task::495::TaskManager.Task::(_debug) Task 824e2afb-d628-4688-88df-e090356c6f9f: moving from state preparing -> state finished
>Thread-15911679::DEBUG::2013-11-19 21:39:39,561::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911679::DEBUG::2013-11-19 21:39:39,562::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911679::DEBUG::2013-11-19 21:39:39,563::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911679::DEBUG::2013-11-19 21:39:39,563::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911679::DEBUG::2013-11-19 21:39:39,564::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911679::DEBUG::2013-11-19 21:39:39,564::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911679::DEBUG::2013-11-19 21:39:39,565::task::495::TaskManager.Task::(_debug) Task 824e2afb-d628-4688-88df-e090356c6f9f: ref 0 aborting False
>Thread-15911679::INFO::2013-11-19 21:39:39,566::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133801443328', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911465::DEBUG::2013-11-19 21:39:40,947::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:39:41,479::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0707925 s, 14.5 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:39:43,165::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911465::DEBUG::2013-11-19 21:39:43,494::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:39:44,009::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.06958 s, 14.7 MB/s\n'; <rc> = 0
>Thread-15911682::INFO::2013-11-19 21:39:44,658::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911682::DEBUG::2013-11-19 21:39:44,659::task::495::TaskManager.Task::(_debug) Task ee95603b-097b-447c-836d-3a4a21c0387e: moving from state init -> state preparing
>Thread-15911682::DEBUG::2013-11-19 21:39:44,666::task::495::TaskManager.Task::(_debug) Task ee95603b-097b-447c-836d-3a4a21c0387e: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000414133071899', 'lastCheck': 1384897175.5449901, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142288208008', 'lastCheck': 1384897177.0786681, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897183.1659491, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000552892684937', 'lastCheck': 1384897174.5397, 'valid': True, 'code': 0}}
>Thread-15911682::DEBUG::2013-11-19 21:39:44,681::task::495::TaskManager.Task::(_debug) Task ee95603b-097b-447c-836d-3a4a21c0387e: moving from state preparing -> state finished
>Thread-15911682::DEBUG::2013-11-19 21:39:44,681::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911682::DEBUG::2013-11-19 21:39:44,682::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911682::DEBUG::2013-11-19 21:39:44,682::task::495::TaskManager.Task::(_debug) Task ee95603b-097b-447c-836d-3a4a21c0387e: ref 0 aborting False
>Thread-15911682::INFO::2013-11-19 21:39:44,683::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000414133071899', 'lastCheck': 1384897175.5449901, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142288208008', 'lastCheck': 1384897177.0786681, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897183.1659491, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000552892684937', 'lastCheck': 1384897174.5397, 'valid': True, 'code': 0}}
>Dummy-15911465::DEBUG::2013-11-19 21:39:46,025::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:39:46,583::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0945474 s, 10.8 MB/s\n'; <rc> = 0
>Dummy-15911465::DEBUG::2013-11-19 21:39:48,599::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:39:49,123::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0699839 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15911465::DEBUG::2013-11-19 21:39:51,139::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15911687::DEBUG::2013-11-19 21:39:51,375::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911687::INFO::2013-11-19 21:39:51,377::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911687::DEBUG::2013-11-19 21:39:51,378::task::495::TaskManager.Task::(_debug) Task 1031e0e4-8771-413a-a451-867a1e552512: moving from state init -> state preparing
>Thread-15911687::DEBUG::2013-11-19 21:39:51,379::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=285 spmId=2
>Thread-15911687::DEBUG::2013-11-19 21:39:51,380::task::495::TaskManager.Task::(_debug) Task 1031e0e4-8771-413a-a451-867a1e552512: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911687::DEBUG::2013-11-19 21:39:51,381::task::495::TaskManager.Task::(_debug) Task 1031e0e4-8771-413a-a451-867a1e552512: moving from state preparing -> state finished
>Thread-15911687::DEBUG::2013-11-19 21:39:51,381::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911687::DEBUG::2013-11-19 21:39:51,382::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911687::DEBUG::2013-11-19 21:39:51,383::task::495::TaskManager.Task::(_debug) Task 1031e0e4-8771-413a-a451-867a1e552512: ref 0 aborting False
>Thread-15911687::INFO::2013-11-19 21:39:51,383::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911688::DEBUG::2013-11-19 21:39:51,395::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911688::INFO::2013-11-19 21:39:51,396::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911688::DEBUG::2013-11-19 21:39:51,397::task::495::TaskManager.Task::(_debug) Task 294f0695-5f71-4c5d-b221-53914532f235: moving from state init -> state preparing
>Thread-15911688::DEBUG::2013-11-19 21:39:51,398::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3f116f03-09dc-4920-8474-ff6807372bda`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911688::DEBUG::2013-11-19 21:39:51,399::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911688::DEBUG::2013-11-19 21:39:51,400::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911688::DEBUG::2013-11-19 21:39:51,401::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3f116f03-09dc-4920-8474-ff6807372bda`::Granted request
>Thread-15911688::DEBUG::2013-11-19 21:39:51,402::task::495::TaskManager.Task::(_debug) Task 294f0695-5f71-4c5d-b221-53914532f235: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911688::DEBUG::2013-11-19 21:39:51,403::task::495::TaskManager.Task::(_debug) Task 294f0695-5f71-4c5d-b221-53914532f235: ref 1 aborting False
>Thread-15911688::WARNING::2013-11-19 21:39:51,405::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911688::DEBUG::2013-11-19 21:39:51,406::task::495::TaskManager.Task::(_debug) Task 294f0695-5f71-4c5d-b221-53914532f235: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133809831936', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911688::DEBUG::2013-11-19 21:39:51,407::task::495::TaskManager.Task::(_debug) Task 294f0695-5f71-4c5d-b221-53914532f235: moving from state preparing -> state finished
>Thread-15911688::DEBUG::2013-11-19 21:39:51,408::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911688::DEBUG::2013-11-19 21:39:51,408::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911688::DEBUG::2013-11-19 21:39:51,409::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911688::DEBUG::2013-11-19 21:39:51,410::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911688::DEBUG::2013-11-19 21:39:51,410::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911688::DEBUG::2013-11-19 21:39:51,411::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911688::DEBUG::2013-11-19 21:39:51,412::task::495::TaskManager.Task::(_debug) Task 294f0695-5f71-4c5d-b221-53914532f235: ref 0 aborting False
>Thread-15911688::INFO::2013-11-19 21:39:51,412::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133809831936', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911465::DEBUG::2013-11-19 21:39:51,682::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0704483 s, 14.5 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:39:53,372::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911465::DEBUG::2013-11-19 21:39:53,697::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:39:54,233::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0745386 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15911690::INFO::2013-11-19 21:39:54,882::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911690::DEBUG::2013-11-19 21:39:54,883::task::495::TaskManager.Task::(_debug) Task 694c8009-8a74-468c-a436-6ba2338cd86f: moving from state init -> state preparing
>Thread-15911690::DEBUG::2013-11-19 21:39:54,884::task::495::TaskManager.Task::(_debug) Task 694c8009-8a74-468c-a436-6ba2338cd86f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000414133071899', 'lastCheck': 1384897185.7120061, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00150799751282', 'lastCheck': 1384897187.25246, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897193.3728571, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000556945800781', 'lastCheck': 1384897184.7166281, 'valid': True, 'code': 0}}
>Thread-15911690::DEBUG::2013-11-19 21:39:54,885::task::495::TaskManager.Task::(_debug) Task 694c8009-8a74-468c-a436-6ba2338cd86f: moving from state preparing -> state finished
>Thread-15911690::DEBUG::2013-11-19 21:39:54,885::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911690::DEBUG::2013-11-19 21:39:54,886::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911690::DEBUG::2013-11-19 21:39:54,886::task::495::TaskManager.Task::(_debug) Task 694c8009-8a74-468c-a436-6ba2338cd86f: ref 0 aborting False
>Thread-15911690::INFO::2013-11-19 21:39:54,887::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000414133071899', 'lastCheck': 1384897185.7120061, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00150799751282', 'lastCheck': 1384897187.25246, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897193.3728571, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000556945800781', 'lastCheck': 1384897184.7166281, 'valid': True, 'code': 0}}
>Thread-42838::DEBUG::2013-11-19 21:39:55,910::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:39:56,249::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-42838::DEBUG::2013-11-19 21:39:56,610::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15911465::DEBUG::2013-11-19 21:39:56,774::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0704507 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15911465::DEBUG::2013-11-19 21:39:58,789::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:39:59,333::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0707952 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15911465::DEBUG::2013-11-19 21:40:01,348::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15911695::DEBUG::2013-11-19 21:40:01,600::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911695::INFO::2013-11-19 21:40:01,607::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911695::DEBUG::2013-11-19 21:40:01,608::task::495::TaskManager.Task::(_debug) Task cf2b4ef0-d921-48a0-893a-84afe6656ae1: moving from state init -> state preparing
>Thread-15911695::DEBUG::2013-11-19 21:40:01,609::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=285 spmId=2
>Thread-15911695::DEBUG::2013-11-19 21:40:01,610::task::495::TaskManager.Task::(_debug) Task cf2b4ef0-d921-48a0-893a-84afe6656ae1: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911695::DEBUG::2013-11-19 21:40:01,611::task::495::TaskManager.Task::(_debug) Task cf2b4ef0-d921-48a0-893a-84afe6656ae1: moving from state preparing -> state finished
>Thread-15911695::DEBUG::2013-11-19 21:40:01,611::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911695::DEBUG::2013-11-19 21:40:01,612::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911695::DEBUG::2013-11-19 21:40:01,612::task::495::TaskManager.Task::(_debug) Task cf2b4ef0-d921-48a0-893a-84afe6656ae1: ref 0 aborting False
>Thread-15911695::INFO::2013-11-19 21:40:01,613::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911696::DEBUG::2013-11-19 21:40:01,625::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911696::INFO::2013-11-19 21:40:01,626::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911696::DEBUG::2013-11-19 21:40:01,627::task::495::TaskManager.Task::(_debug) Task 96a28e5e-c3be-4d80-b750-cf210486cd8d: moving from state init -> state preparing
>Thread-15911696::DEBUG::2013-11-19 21:40:01,628::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a5939890-f5cf-417d-854b-faca580b5560`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911696::DEBUG::2013-11-19 21:40:01,629::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911696::DEBUG::2013-11-19 21:40:01,630::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911696::DEBUG::2013-11-19 21:40:01,630::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a5939890-f5cf-417d-854b-faca580b5560`::Granted request
>Thread-15911696::DEBUG::2013-11-19 21:40:01,632::task::495::TaskManager.Task::(_debug) Task 96a28e5e-c3be-4d80-b750-cf210486cd8d: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911696::DEBUG::2013-11-19 21:40:01,632::task::495::TaskManager.Task::(_debug) Task 96a28e5e-c3be-4d80-b750-cf210486cd8d: ref 1 aborting False
>Thread-15911696::WARNING::2013-11-19 21:40:01,635::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911696::DEBUG::2013-11-19 21:40:01,636::task::495::TaskManager.Task::(_debug) Task 96a28e5e-c3be-4d80-b750-cf210486cd8d: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133811929088', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911696::DEBUG::2013-11-19 21:40:01,637::task::495::TaskManager.Task::(_debug) Task 96a28e5e-c3be-4d80-b750-cf210486cd8d: moving from state preparing -> state finished
>Thread-15911696::DEBUG::2013-11-19 21:40:01,638::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911696::DEBUG::2013-11-19 21:40:01,638::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911696::DEBUG::2013-11-19 21:40:01,639::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911696::DEBUG::2013-11-19 21:40:01,640::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911696::DEBUG::2013-11-19 21:40:01,640::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911696::DEBUG::2013-11-19 21:40:01,641::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911696::DEBUG::2013-11-19 21:40:01,641::task::495::TaskManager.Task::(_debug) Task 96a28e5e-c3be-4d80-b750-cf210486cd8d: ref 0 aborting False
>Thread-15911696::INFO::2013-11-19 21:40:01,642::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133811929088', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15911465::DEBUG::2013-11-19 21:40:01,893::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0702969 s, 14.6 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:40:03,596::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911465::DEBUG::2013-11-19 21:40:03,908::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:40:04,451::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0703995 s, 14.5 MB/s\n'; <rc> = 0
>Thread-15911698::INFO::2013-11-19 21:40:05,042::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911698::DEBUG::2013-11-19 21:40:05,043::task::495::TaskManager.Task::(_debug) Task e091f210-9f0e-4b61-9362-e76bc557f95e: moving from state init -> state preparing
>Thread-15911698::DEBUG::2013-11-19 21:40:05,044::task::495::TaskManager.Task::(_debug) Task e091f210-9f0e-4b61-9362-e76bc557f95e: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000407934188843', 'lastCheck': 1384897196.6160581, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0021800994873', 'lastCheck': 1384897197.4660299, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897203.597055, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000550985336304', 'lastCheck': 1384897194.909667, 'valid': True, 'code': 0}}
>Thread-15911698::DEBUG::2013-11-19 21:40:05,054::task::495::TaskManager.Task::(_debug) Task e091f210-9f0e-4b61-9362-e76bc557f95e: moving from state preparing -> state finished
>Thread-15911698::DEBUG::2013-11-19 21:40:05,055::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911698::DEBUG::2013-11-19 21:40:05,055::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911698::DEBUG::2013-11-19 21:40:05,056::task::495::TaskManager.Task::(_debug) Task e091f210-9f0e-4b61-9362-e76bc557f95e: ref 0 aborting False
>Thread-15911698::INFO::2013-11-19 21:40:05,056::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000407934188843', 'lastCheck': 1384897196.6160581, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0021800994873', 'lastCheck': 1384897197.4660299, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897203.597055, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000550985336304', 'lastCheck': 1384897194.909667, 'valid': True, 'code': 0}}
>Dummy-15911465::DEBUG::2013-11-19 21:40:06,466::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:40:06,992::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0701655 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15911465::DEBUG::2013-11-19 21:40:09,006::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:40:09,537::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0700583 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15911465::DEBUG::2013-11-19 21:40:11,552::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:40:12,100::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0713147 s, 14.4 MB/s\n'; <rc> = 0
>Thread-15911703::DEBUG::2013-11-19 21:40:12,195::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911703::INFO::2013-11-19 21:40:12,196::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911703::DEBUG::2013-11-19 21:40:12,197::task::495::TaskManager.Task::(_debug) Task 916e831c-c6fc-491e-82b5-b842f59ab446: moving from state init -> state preparing
>Thread-15911703::DEBUG::2013-11-19 21:40:12,198::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=285 spmId=2
>Thread-15911703::DEBUG::2013-11-19 21:40:12,198::task::495::TaskManager.Task::(_debug) Task 916e831c-c6fc-491e-82b5-b842f59ab446: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911703::DEBUG::2013-11-19 21:40:12,199::task::495::TaskManager.Task::(_debug) Task 916e831c-c6fc-491e-82b5-b842f59ab446: moving from state preparing -> state finished
>Thread-15911703::DEBUG::2013-11-19 21:40:12,199::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911703::DEBUG::2013-11-19 21:40:12,200::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911703::DEBUG::2013-11-19 21:40:12,200::task::495::TaskManager.Task::(_debug) Task 916e831c-c6fc-491e-82b5-b842f59ab446: ref 0 aborting False
>Thread-15911703::INFO::2013-11-19 21:40:12,201::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911704::DEBUG::2013-11-19 21:40:12,207::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911704::INFO::2013-11-19 21:40:12,208::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911704::DEBUG::2013-11-19 21:40:12,208::task::495::TaskManager.Task::(_debug) Task abbce858-a49b-45e5-8b28-d47549b46481: moving from state init -> state preparing
>Thread-15911704::DEBUG::2013-11-19 21:40:12,210::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c4bfc036-f5e7-46ad-b748-f9501104c94c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15911704::DEBUG::2013-11-19 21:40:12,210::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15911704::DEBUG::2013-11-19 21:40:12,211::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15911704::DEBUG::2013-11-19 21:40:12,212::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c4bfc036-f5e7-46ad-b748-f9501104c94c`::Granted request
>Thread-15911704::DEBUG::2013-11-19 21:40:12,213::task::495::TaskManager.Task::(_debug) Task abbce858-a49b-45e5-8b28-d47549b46481: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15911704::DEBUG::2013-11-19 21:40:12,213::task::495::TaskManager.Task::(_debug) Task abbce858-a49b-45e5-8b28-d47549b46481: ref 1 aborting False
>Thread-15911704::WARNING::2013-11-19 21:40:12,215::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911704::DEBUG::2013-11-19 21:40:12,216::task::495::TaskManager.Task::(_debug) Task abbce858-a49b-45e5-8b28-d47549b46481: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133811929088', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15911704::DEBUG::2013-11-19 21:40:12,217::task::495::TaskManager.Task::(_debug) Task abbce858-a49b-45e5-8b28-d47549b46481: moving from state preparing -> state finished
>Thread-15911704::DEBUG::2013-11-19 21:40:12,218::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911704::DEBUG::2013-11-19 21:40:12,218::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911704::DEBUG::2013-11-19 21:40:12,219::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911704::DEBUG::2013-11-19 21:40:12,220::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911704::DEBUG::2013-11-19 21:40:12,220::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911704::DEBUG::2013-11-19 21:40:12,221::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911704::DEBUG::2013-11-19 21:40:12,221::task::495::TaskManager.Task::(_debug) Task abbce858-a49b-45e5-8b28-d47549b46481: ref 0 aborting False
>Thread-15911704::INFO::2013-11-19 21:40:12,222::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 285}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '133811929088', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-1024767::ERROR::2013-11-19 21:40:13,789::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Dummy-15911465::DEBUG::2013-11-19 21:40:14,116::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:40:14,639::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0705105 s, 14.5 MB/s\n'; <rc> = 0
>Thread-15911706::INFO::2013-11-19 21:40:15,295::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911706::DEBUG::2013-11-19 21:40:15,296::task::495::TaskManager.Task::(_debug) Task 17934326-f1fe-4f62-9658-59611c9286c4: moving from state init -> state preparing
>Thread-15911706::DEBUG::2013-11-19 21:40:15,297::task::495::TaskManager.Task::(_debug) Task 17934326-f1fe-4f62-9658-59611c9286c4: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000596046447754', 'lastCheck': 1384897206.684478, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148606300354', 'lastCheck': 1384897207.687403, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897213.7906649, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000538110733032', 'lastCheck': 1384897205.133152, 'valid': True, 'code': 0}}
>Thread-15911706::DEBUG::2013-11-19 21:40:15,297::task::495::TaskManager.Task::(_debug) Task 17934326-f1fe-4f62-9658-59611c9286c4: moving from state preparing -> state finished
>Thread-15911706::DEBUG::2013-11-19 21:40:15,298::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911706::DEBUG::2013-11-19 21:40:15,299::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911706::DEBUG::2013-11-19 21:40:15,299::task::495::TaskManager.Task::(_debug) Task 17934326-f1fe-4f62-9658-59611c9286c4: ref 0 aborting False
>Thread-15911706::INFO::2013-11-19 21:40:15,300::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000596046447754', 'lastCheck': 1384897206.684478, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148606300354', 'lastCheck': 1384897207.687403, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897213.7906649, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000538110733032', 'lastCheck': 1384897205.133152, 'valid': True, 'code': 0}}
>Dummy-15911465::DEBUG::2013-11-19 21:40:16,654::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15911465::DEBUG::2013-11-19 21:40:17,110::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0450239 s, 22.7 MB/s\n'; <rc> = 0
>Dummy-15911465::DEBUG::2013-11-19 21:40:19,130::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15911710::DEBUG::2013-11-19 21:40:19,641::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911710::INFO::2013-11-19 21:40:19,644::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksStatuses, args: ()
>Thread-15911710::DEBUG::2013-11-19 21:40:19,644::task::495::TaskManager.Task::(_debug) Task 711c83ea-155c-453d-9404-c8069898bba6: moving from state init -> state preparing
>Thread-15911710::DEBUG::2013-11-19 21:40:19,645::taskManager::92::TaskManager::(getAllTasksStatuses) Entry.
>Thread-15911710::DEBUG::2013-11-19 21:40:19,647::taskManager::97::TaskManager::(getAllTasksStatuses) Return: {}
>Thread-15911710::DEBUG::2013-11-19 21:40:19,648::task::495::TaskManager.Task::(_debug) Task 711c83ea-155c-453d-9404-c8069898bba6: finished: {'allTasksStatus': {}}
>Thread-15911710::DEBUG::2013-11-19 21:40:19,649::task::495::TaskManager.Task::(_debug) Task 711c83ea-155c-453d-9404-c8069898bba6: moving from state preparing -> state finished
>Thread-15911710::DEBUG::2013-11-19 21:40:19,650::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911710::DEBUG::2013-11-19 21:40:19,652::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911710::DEBUG::2013-11-19 21:40:19,653::task::495::TaskManager.Task::(_debug) Task 711c83ea-155c-453d-9404-c8069898bba6: ref 0 aborting False
>Thread-15911710::INFO::2013-11-19 21:40:19,654::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksStatuses, Return response: {'status': {'message': 'OK', 'code': 0}, 'allTasksStatus': {}}
>Dummy-15911465::DEBUG::2013-11-19 21:40:19,695::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0918885 s, 11.1 MB/s\n'; <rc> = 0
>Thread-15911711::DEBUG::2013-11-19 21:40:19,813::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911711::INFO::2013-11-19 21:40:19,814::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: deactivateStorageDomain, args: ( sdUUID=5d7b5178-3790-472c-a13b-b481eb27a61e spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 msdUUID=00000000-0000-0000-0000-000000000000 masterVersion=1)
>Thread-15911711::DEBUG::2013-11-19 21:40:19,814::task::495::TaskManager.Task::(_debug) Task 8ac9ff13-f628-40fa-a83e-a0a57719328a: moving from state init -> state preparing
>Thread-15911711::DEBUG::2013-11-19 21:40:19,816::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a3a47e79-d594-4803-b2b1-6d6b2d53afb4`::Request was made in '/usr/share/vdsm/storage/spm.py' line '1112' at 'public_deactivateStorageDomain'
>Thread-15911711::DEBUG::2013-11-19 21:40:19,816::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'exclusive'
>Thread-15911711::DEBUG::2013-11-19 21:40:19,817::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'exclusive' (1 active user)
>Thread-15911711::DEBUG::2013-11-19 21:40:19,818::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a3a47e79-d594-4803-b2b1-6d6b2d53afb4`::Granted request
>Thread-15911711::DEBUG::2013-11-19 21:40:19,819::task::495::TaskManager.Task::(_debug) Task 8ac9ff13-f628-40fa-a83e-a0a57719328a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (exclusive)
>Thread-15911711::DEBUG::2013-11-19 21:40:19,819::task::495::TaskManager.Task::(_debug) Task 8ac9ff13-f628-40fa-a83e-a0a57719328a: ref 1 aborting False
>Thread-15911711::DEBUG::2013-11-19 21:40:19,820::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.5d7b5178-3790-472c-a13b-b481eb27a61e`ReqID=`d0c9aece-3b3a-4892-92d4-5c54c0a80a50`::Request was made in '/usr/share/vdsm/storage/spm.py' line '1113' at 'public_deactivateStorageDomain'
>Thread-15911711::DEBUG::2013-11-19 21:40:19,821::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e' for lock type 'exclusive'
>Thread-15911711::DEBUG::2013-11-19 21:40:19,822::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e' is free. Now locking as 'exclusive' (1 active user)
>Thread-15911711::DEBUG::2013-11-19 21:40:19,822::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.5d7b5178-3790-472c-a13b-b481eb27a61e`ReqID=`d0c9aece-3b3a-4892-92d4-5c54c0a80a50`::Granted request
>Thread-15911711::DEBUG::2013-11-19 21:40:19,823::task::495::TaskManager.Task::(_debug) Task 8ac9ff13-f628-40fa-a83e-a0a57719328a: _resourcesAcquired: Storage.5d7b5178-3790-472c-a13b-b481eb27a61e (exclusive)
>Thread-15911711::DEBUG::2013-11-19 21:40:19,824::task::495::TaskManager.Task::(_debug) Task 8ac9ff13-f628-40fa-a83e-a0a57719328a: ref 1 aborting False
>Thread-15911711::INFO::2013-11-19 21:40:19,824::sp::775::Storage.StoragePool::(deactivateSD) sdUUID=5d7b5178-3790-472c-a13b-b481eb27a61e spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 new_msdUUID=00000000-0000-0000-0000-000000000000
>Thread-15911711::INFO::2013-11-19 21:40:19,825::fileSD::176::Storage.StorageDomain::(validate) sdUUID=5d7b5178-3790-472c-a13b-b481eb27a61e
>Thread-15911711::DEBUG::2013-11-19 21:40:19,826::persistentDict::204::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=[]
>Thread-15911711::WARNING::2013-11-19 21:40:19,827::persistentDict::226::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is
>Thread-15911711::ERROR::2013-11-19 21:40:19,827::task::868::TaskManager.Task::(_setError) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/task.py", line 876, in _run
>  File "/usr/share/vdsm/storage/spm.py", line 115, in run
>  File "/usr/share/vdsm/storage/spm.py", line 1115, in public_deactivateStorageDomain
>  File "/usr/share/vdsm/storage/sp.py", line 798, in deactivateSD
>  File "/usr/share/vdsm/storage/sd.py", line 666, in isBackup
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15911711::DEBUG::2013-11-19 21:40:19,836::task::495::TaskManager.Task::(_debug) Task 8ac9ff13-f628-40fa-a83e-a0a57719328a: Task._run: 8ac9ff13-f628-40fa-a83e-a0a57719328a ('5d7b5178-3790-472c-a13b-b481eb27a61e', 'c4e5889a-348d-11e2-83fe-9f72369eab30', '00000000-0000-0000-0000-000000000000', 1) {} failed - stopping task
>Thread-15911711::DEBUG::2013-11-19 21:40:19,837::task::495::TaskManager.Task::(_debug) Task 8ac9ff13-f628-40fa-a83e-a0a57719328a: stopping in state preparing (force False)
>Thread-15911711::DEBUG::2013-11-19 21:40:19,837::task::495::TaskManager.Task::(_debug) Task 8ac9ff13-f628-40fa-a83e-a0a57719328a: ref 1 aborting True
>Thread-15911711::INFO::2013-11-19 21:40:19,838::task::1171::TaskManager.Task::(prepare) aborting: Task is aborted: "'CLASS'" - code 100
>Thread-15911711::DEBUG::2013-11-19 21:40:19,838::task::495::TaskManager.Task::(_debug) Task 8ac9ff13-f628-40fa-a83e-a0a57719328a: Prepare: aborted: 'CLASS'
>Thread-15911711::DEBUG::2013-11-19 21:40:19,839::task::495::TaskManager.Task::(_debug) Task 8ac9ff13-f628-40fa-a83e-a0a57719328a: ref 0 aborting True
>Thread-15911711::DEBUG::2013-11-19 21:40:19,839::task::495::TaskManager.Task::(_debug) Task 8ac9ff13-f628-40fa-a83e-a0a57719328a: Task._doAbort: force False
>Thread-15911711::DEBUG::2013-11-19 21:40:19,840::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911711::DEBUG::2013-11-19 21:40:19,841::task::495::TaskManager.Task::(_debug) Task 8ac9ff13-f628-40fa-a83e-a0a57719328a: moving from state preparing -> state aborting
>Thread-15911711::DEBUG::2013-11-19 21:40:19,841::task::495::TaskManager.Task::(_debug) Task 8ac9ff13-f628-40fa-a83e-a0a57719328a: _aborting: recover policy none
>Thread-15911711::DEBUG::2013-11-19 21:40:19,842::task::495::TaskManager.Task::(_debug) Task 8ac9ff13-f628-40fa-a83e-a0a57719328a: moving from state aborting -> state failed
>Thread-15911711::DEBUG::2013-11-19 21:40:19,842::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>, 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e': < ResourceRef 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e', isValid: 'True' obj: 'None'>}
>Thread-15911711::DEBUG::2013-11-19 21:40:19,843::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911711::DEBUG::2013-11-19 21:40:19,843::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911711::DEBUG::2013-11-19 21:40:19,844::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911711::DEBUG::2013-11-19 21:40:19,845::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911711::DEBUG::2013-11-19 21:40:19,845::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911711::DEBUG::2013-11-19 21:40:19,846::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e'
>Thread-15911711::DEBUG::2013-11-19 21:40:19,846::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e' (0 active users)
>Thread-15911711::DEBUG::2013-11-19 21:40:19,847::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e' is free, finding out if anyone is waiting for it.
>Thread-15911711::DEBUG::2013-11-19 21:40:19,847::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.5d7b5178-3790-472c-a13b-b481eb27a61e', Clearing records.
>Thread-15911711::ERROR::2013-11-19 21:40:19,848::dispatcher::106::Storage.Dispatcher.Protect::(run) 'CLASS'
>Thread-15911711::ERROR::2013-11-19 21:40:19,849::dispatcher::107::Storage.Dispatcher.Protect::(run) Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/dispatcher.py", line 96, in run
>  File "/usr/share/vdsm/storage/task.py", line 1178, in prepare
>KeyError: 'CLASS'
>
>Thread-15911712::DEBUG::2013-11-19 21:40:19,880::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911712::INFO::2013-11-19 21:40:19,881::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911712::DEBUG::2013-11-19 21:40:19,881::task::495::TaskManager.Task::(_debug) Task 8a7a83cc-688c-4dd2-a325-c81b39a5eb0f: moving from state init -> state preparing
>Thread-15911712::DEBUG::2013-11-19 21:40:19,882::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=285 spmId=2
>Thread-15911712::DEBUG::2013-11-19 21:40:19,883::task::495::TaskManager.Task::(_debug) Task 8a7a83cc-688c-4dd2-a325-c81b39a5eb0f: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911712::DEBUG::2013-11-19 21:40:19,883::task::495::TaskManager.Task::(_debug) Task 8a7a83cc-688c-4dd2-a325-c81b39a5eb0f: moving from state preparing -> state finished
>Thread-15911712::DEBUG::2013-11-19 21:40:19,884::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911712::DEBUG::2013-11-19 21:40:19,884::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911712::DEBUG::2013-11-19 21:40:19,885::task::495::TaskManager.Task::(_debug) Task 8a7a83cc-688c-4dd2-a325-c81b39a5eb0f: ref 0 aborting False
>Thread-15911712::INFO::2013-11-19 21:40:19,885::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 285}}
>Thread-15911713::DEBUG::2013-11-19 21:40:19,903::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911713::INFO::2013-11-19 21:40:19,908::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksStatuses, args: ()
>Thread-15911713::DEBUG::2013-11-19 21:40:19,910::task::495::TaskManager.Task::(_debug) Task 37e2980c-7b77-44ec-801c-595371bee316: moving from state init -> state preparing
>Thread-15911713::DEBUG::2013-11-19 21:40:19,913::taskManager::92::TaskManager::(getAllTasksStatuses) Entry.
>Thread-15911713::DEBUG::2013-11-19 21:40:19,915::taskManager::97::TaskManager::(getAllTasksStatuses) Return: {}
>Thread-15911713::DEBUG::2013-11-19 21:40:19,916::task::495::TaskManager.Task::(_debug) Task 37e2980c-7b77-44ec-801c-595371bee316: finished: {'allTasksStatus': {}}
>Thread-15911713::DEBUG::2013-11-19 21:40:19,916::task::495::TaskManager.Task::(_debug) Task 37e2980c-7b77-44ec-801c-595371bee316: moving from state preparing -> state finished
>Thread-15911713::DEBUG::2013-11-19 21:40:19,917::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911713::DEBUG::2013-11-19 21:40:19,917::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911713::DEBUG::2013-11-19 21:40:19,918::task::495::TaskManager.Task::(_debug) Task 37e2980c-7b77-44ec-801c-595371bee316: ref 0 aborting False
>Thread-15911713::INFO::2013-11-19 21:40:19,919::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksStatuses, Return response: {'status': {'message': 'OK', 'code': 0}, 'allTasksStatus': {}}
>Thread-15911714::DEBUG::2013-11-19 21:40:19,924::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15911714::INFO::2013-11-19 21:40:19,925::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: spmStop, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15911714::DEBUG::2013-11-19 21:40:19,925::task::495::TaskManager.Task::(_debug) Task d1c0b138-4d14-4d13-81d5-17d287e303d5: moving from state init -> state preparing
>Thread-15911714::DEBUG::2013-11-19 21:40:19,926::spm::547::Storage.SPM::(public_spmStop) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30
>Thread-15911714::DEBUG::2013-11-19 21:40:19,926::taskManager::105::TaskManager::(getAllTasks) Entry.
>Thread-15911714::DEBUG::2013-11-19 21:40:19,927::taskManager::110::TaskManager::(getAllTasks) Return: {}
>Thread-15911714::DEBUG::2013-11-19 21:40:19,928::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c6ae2c6a-fbfb-4191-833f-d4f2ed7ee2be`::Request was made in '/usr/share/vdsm/storage/spm.py' line '573' at 'public_spmStop'
>Thread-15911714::DEBUG::2013-11-19 21:40:19,928::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'exclusive'
>Thread-15911714::DEBUG::2013-11-19 21:40:19,929::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'exclusive' (1 active user)
>Thread-15911714::DEBUG::2013-11-19 21:40:19,930::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c6ae2c6a-fbfb-4191-833f-d4f2ed7ee2be`::Granted request
>Thread-15911714::DEBUG::2013-11-19 21:40:19,931::task::495::TaskManager.Task::(_debug) Task d1c0b138-4d14-4d13-81d5-17d287e303d5: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (exclusive)
>Thread-15911714::DEBUG::2013-11-19 21:40:19,931::task::495::TaskManager.Task::(_debug) Task d1c0b138-4d14-4d13-81d5-17d287e303d5: ref 1 aborting False
>Thread-15911714::DEBUG::2013-11-19 21:40:19,932::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a4315281-a910-4449-8ad0-8b33d2cec479`::Request was made in '/usr/share/vdsm/storage/spm.py' line '584' at '_stop'
>Thread-15911714::DEBUG::2013-11-19 21:40:19,933::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'exclusive'
>Thread-15911714::DEBUG::2013-11-19 21:40:19,933::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'exclusive' (1 active user)
>Thread-15911714::DEBUG::2013-11-19 21:40:19,934::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a4315281-a910-4449-8ad0-8b33d2cec479`::Granted request
>Thread-15911714::DEBUG::2013-11-19 21:40:19,945::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e`ReqID=`000c26ed-f9bf-4449-94f4-6f72df1d0563`::Request was made in '/usr/share/vdsm/storage/spm.py' line '601' at '_stop'
>Thread-15911714::DEBUG::2013-11-19 21:40:19,945::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e' for lock type 'exclusive'
>Thread-15911714::DEBUG::2013-11-19 21:40:19,946::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e' is free. Now locking as 'exclusive' (1 active user)
>Thread-15911714::DEBUG::2013-11-19 21:40:19,947::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e`ReqID=`000c26ed-f9bf-4449-94f4-6f72df1d0563`::Granted request
>Thread-15911714::DEBUG::2013-11-19 21:40:19,948::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e'
>Thread-15911714::DEBUG::2013-11-19 21:40:19,949::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e' (0 active users)
>Thread-15911714::DEBUG::2013-11-19 21:40:19,949::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e' is free, finding out if anyone is waiting for it.
>Thread-15911714::DEBUG::2013-11-19 21:40:19,950::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_5d7b5178-3790-472c-a13b-b481eb27a61e', Clearing records.
>Thread-15911714::DEBUG::2013-11-19 21:40:19,951::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911714::DEBUG::2013-11-19 21:40:19,951::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911714::DEBUG::2013-11-19 21:40:19,952::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911714::DEBUG::2013-11-19 21:40:19,952::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911714::DEBUG::2013-11-19 21:40:19,953::taskManager::118::TaskManager::(unloadTasks) Entry.
>Thread-15911714::DEBUG::2013-11-19 21:40:19,954::taskManager::122::TaskManager::(unloadTasks) Return
>Thread-15911714::DEBUG::2013-11-19 21:40:19,956::spm::195::Storage.SPM::(__cleanupMasterMount) unmounting /rhev/data-center/mnt/blockSD/6b892067-ac7e-4039-a7be-5b7387525b16/master
>Thread-15911714::DEBUG::2013-11-19 21:40:20,096::fileUtils::112::Storage.Misc.excCmd::(umount) '/usr/bin/sudo -n /bin/umount -f /rhev/data-center/mnt/blockSD/6b892067-ac7e-4039-a7be-5b7387525b16/master' (cwd None)
>Thread-15911714::DEBUG::2013-11-19 21:40:20,442::fileUtils::112::Storage.Misc.excCmd::(umount) SUCCESS: <err> = ''; <rc> = 0
>Thread-15911714::DEBUG::2013-11-19 21:40:20,445::spm::290::Storage.SPM::(__cleanupSPM) cleaning up SPM: c4e5889a-348d-11e2-83fe-9f72369eab30
>Thread-15911714::DEBUG::2013-11-19 21:40:20,446::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
>Thread-15911714::DEBUG::2013-11-19 21:40:20,447::persistentDict::146::Storage.PersistentDict::(transaction) Flushing changes
>Thread-15911714::DEBUG::2013-11-19 21:40:20,448::persistentDict::265::Storage.PersistentDict::(flush) about to write lines (VGTagMetadataRW)=[u'CLASS=Data', u'DESCRIPTION=OS', u'IOOPTIMEOUTSEC=10', u'LEASERETRIES=3', u'LEASETIMESEC=60', u'LOCKPOLICY=', u'LOCKRENEWALINTERVALSEC=5', u'LOGBLKSIZE=512', u'MASTER_VERSION=1', u'PHYBLKSIZE=512', u'POOL_DESCRIPTION=Default', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_ID=-1', u'POOL_SPM_LVER=285', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'ROLE=Master', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'TYPE=FCP', u'VERSION=2', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', '_SHA_CKSUM=90a91eb94d806053a3001427d0dfd2783a6d5bb4']
>Thread-15911714::DEBUG::2013-11-19 21:40:20,449::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15911714::DEBUG::2013-11-19 21:40:20,450::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15911714::DEBUG::2013-11-19 21:40:20,451::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15911714::DEBUG::2013-11-19 21:40:20,451::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15911714::DEBUG::2013-11-19 21:40:20,452::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-15911714::DEBUG::2013-11-19 21:40:20,453::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15911714::DEBUG::2013-11-19 21:40:21,090::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15911714::DEBUG::2013-11-19 21:40:21,094::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-15911714::DEBUG::2013-11-19 21:40:21,096::blockSD::145::storage.Metadata.VGTagMetadataRW::(writelines) Updating metadata adding=MDT_POOL_SPM_ID=-1, MDT__SHA_CKSUM=90a91eb94d806053a3001427d0dfd2783a6d5bb4 removing=MDT__SHA_CKSUM=901e51f2c0f3aefcf8d2e501fb4543e2352ea1bc, MDT_POOL_SPM_ID=2
>Thread-15911714::DEBUG::2013-11-19 21:40:21,097::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --deltag MDT_POOL_SPM_ID=2 --deltag MDT__SHA_CKSUM=901e51f2c0f3aefcf8d2e501fb4543e2352ea1bc --addtag MDT__SHA_CKSUM=90a91eb94d806053a3001427d0dfd2783a6d5bb4 --addtag MDT_POOL_SPM_ID=-1 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15911465::INFO::2013-11-19 21:40:21,710::storage_mailbox::673::Storage.MailBox.SpmMailMonitor::(run) SPM_MailMonitor - Incoming mail monitoring thread stopped
>Thread-15911714::DEBUG::2013-11-19 21:40:21,749::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15911714::DEBUG::2013-11-19 21:40:21,751::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15911714::DEBUG::2013-11-19 21:40:21,751::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15911714::DEBUG::2013-11-19 21:40:21,752::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction
>Thread-15911714::INFO::2013-11-19 21:40:21,753::safelease::83::ClusterLock::(release) Releasing cluster lock for domain 6b892067-ac7e-4039-a7be-5b7387525b16
>Thread-15911714::DEBUG::2013-11-19 21:40:21,754::safelease::84::Storage.Misc.excCmd::(release) '/usr/libexec/vdsm/spmstop.sh 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd /usr/libexec/vdsm/)
>Thread-15911714::DEBUG::2013-11-19 21:40:23,224::safelease::84::Storage.Misc.excCmd::(release) SUCCESS: <err> = ''; <rc> = 0
>Thread-15911714::DEBUG::2013-11-19 21:40:23,226::safelease::87::ClusterLock::(release) Cluster lock released successfully
>Thread-15911714::DEBUG::2013-11-19 21:40:23,227::task::495::TaskManager.Task::(_debug) Task d1c0b138-4d14-4d13-81d5-17d287e303d5: finished: None
>Thread-15911714::DEBUG::2013-11-19 21:40:23,227::task::495::TaskManager.Task::(_debug) Task d1c0b138-4d14-4d13-81d5-17d287e303d5: moving from state preparing -> state finished
>Thread-15911714::DEBUG::2013-11-19 21:40:23,228::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15911714::DEBUG::2013-11-19 21:40:23,228::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911714::DEBUG::2013-11-19 21:40:23,229::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15911714::DEBUG::2013-11-19 21:40:23,230::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15911714::DEBUG::2013-11-19 21:40:23,231::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15911714::DEBUG::2013-11-19 21:40:23,231::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15911714::DEBUG::2013-11-19 21:40:23,232::task::495::TaskManager.Task::(_debug) Task d1c0b138-4d14-4d13-81d5-17d287e303d5: ref 0 aborting False
>Thread-15911714::INFO::2013-11-19 21:40:23,233::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: spmStop, Return response: {'status': {'message': 'OK', 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:40:23,966::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911717::INFO::2013-11-19 21:40:25,415::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911717::DEBUG::2013-11-19 21:40:25,416::task::495::TaskManager.Task::(_debug) Task 67bda3b9-ee27-4629-a306-270cc0fbbf0b: moving from state init -> state preparing
>Thread-15911717::WARNING::2013-11-19 21:40:25,417::hsm::1895::Storage.HSM::(public_repoStats) repoStats detected invalid master:6b892067-ac7e-4039-a7be-5b7387525b16 {'finish': 1384897216.911788, 'mdasize': '134479872', 'mdathreshold': True, 'mdavalid': True, 'diskfree': '1142595518464', 'disktotal': '3595827150848', 'result': {'delay': '0.000508069992065', 'lastCheck': 1384897216.911788, 'valid': True, 'code': 0}, 'masterValidate': {'mount': True, 'valid': True}, 'mdafree': '67205632'}
>Thread-15911717::DEBUG::2013-11-19 21:40:25,417::task::495::TaskManager.Task::(_debug) Task 67bda3b9-ee27-4629-a306-270cc0fbbf0b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000508069992065', 'lastCheck': 1384897216.911788, 'valid': True, 'code': 355}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00146198272705', 'lastCheck': 1384897217.9052651, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897223.9675469, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000401973724365', 'lastCheck': 1384897225.364517, 'valid': True, 'code': 0}}
>Thread-15911717::DEBUG::2013-11-19 21:40:25,428::task::495::TaskManager.Task::(_debug) Task 67bda3b9-ee27-4629-a306-270cc0fbbf0b: moving from state preparing -> state finished
>Thread-15911717::DEBUG::2013-11-19 21:40:25,429::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911717::DEBUG::2013-11-19 21:40:25,429::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911717::DEBUG::2013-11-19 21:40:25,430::task::495::TaskManager.Task::(_debug) Task 67bda3b9-ee27-4629-a306-270cc0fbbf0b: ref 0 aborting False
>Thread-15911717::INFO::2013-11-19 21:40:25,431::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000508069992065', 'lastCheck': 1384897216.911788, 'valid': True, 'code': 355}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00146198272705', 'lastCheck': 1384897217.9052651, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897223.9675469, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000401973724365', 'lastCheck': 1384897225.364517, 'valid': True, 'code': 0}}
>Thread-42838::DEBUG::2013-11-19 21:40:26,964::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-42838::DEBUG::2013-11-19 21:40:26,965::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 21:40:27,554::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-42838::DEBUG::2013-11-19 21:40:27,558::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1024767::ERROR::2013-11-19 21:40:34,186::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911723::INFO::2013-11-19 21:40:35,578::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911723::DEBUG::2013-11-19 21:40:35,579::task::495::TaskManager.Task::(_debug) Task 98788fe2-be2d-4d56-abf4-92907deab943: moving from state init -> state preparing
>Thread-15911723::DEBUG::2013-11-19 21:40:35,580::task::495::TaskManager.Task::(_debug) Task 98788fe2-be2d-4d56-abf4-92907deab943: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000333070755005', 'lastCheck': 1384897227.560931, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00130605697632', 'lastCheck': 1384897227.969564, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897234.1881471, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000515937805176', 'lastCheck': 1384897235.375993, 'valid': True, 'code': 0}}
>Thread-15911723::DEBUG::2013-11-19 21:40:35,585::task::495::TaskManager.Task::(_debug) Task 98788fe2-be2d-4d56-abf4-92907deab943: moving from state preparing -> state finished
>Thread-15911723::DEBUG::2013-11-19 21:40:35,585::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911723::DEBUG::2013-11-19 21:40:35,586::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911723::DEBUG::2013-11-19 21:40:35,587::task::495::TaskManager.Task::(_debug) Task 98788fe2-be2d-4d56-abf4-92907deab943: ref 0 aborting False
>Thread-15911723::INFO::2013-11-19 21:40:35,587::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000333070755005', 'lastCheck': 1384897227.560931, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00130605697632', 'lastCheck': 1384897227.969564, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897234.1881471, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000515937805176', 'lastCheck': 1384897235.375993, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:40:44,200::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911729::INFO::2013-11-19 21:40:45,689::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911729::DEBUG::2013-11-19 21:40:45,690::task::495::TaskManager.Task::(_debug) Task 6747af37-85c5-45de-aa15-397ae797f486: moving from state init -> state preparing
>Thread-15911729::DEBUG::2013-11-19 21:40:45,691::task::495::TaskManager.Task::(_debug) Task 6747af37-85c5-45de-aa15-397ae797f486: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000402927398682', 'lastCheck': 1384897237.574053, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00143504142761', 'lastCheck': 1384897237.9851861, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897244.201607, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000426054000854', 'lastCheck': 1384897245.387563, 'valid': True, 'code': 0}}
>Thread-15911729::DEBUG::2013-11-19 21:40:45,697::task::495::TaskManager.Task::(_debug) Task 6747af37-85c5-45de-aa15-397ae797f486: moving from state preparing -> state finished
>Thread-15911729::DEBUG::2013-11-19 21:40:45,697::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911729::DEBUG::2013-11-19 21:40:45,698::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911729::DEBUG::2013-11-19 21:40:45,698::task::495::TaskManager.Task::(_debug) Task 6747af37-85c5-45de-aa15-397ae797f486: ref 0 aborting False
>Thread-15911729::INFO::2013-11-19 21:40:45,699::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000402927398682', 'lastCheck': 1384897237.574053, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00143504142761', 'lastCheck': 1384897237.9851861, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897244.201607, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000426054000854', 'lastCheck': 1384897245.387563, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:40:54,214::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911735::INFO::2013-11-19 21:40:55,846::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911735::DEBUG::2013-11-19 21:40:55,846::task::495::TaskManager.Task::(_debug) Task 89ffaf2f-9202-4e27-bb27-98b9387e7b94: moving from state init -> state preparing
>Thread-15911735::DEBUG::2013-11-19 21:40:55,847::task::495::TaskManager.Task::(_debug) Task 89ffaf2f-9202-4e27-bb27-98b9387e7b94: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000283002853394', 'lastCheck': 1384897247.586884, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00138282775879', 'lastCheck': 1384897247.999289, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897254.215116, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000448942184448', 'lastCheck': 1384897255.3991621, 'valid': True, 'code': 0}}
>Thread-15911735::DEBUG::2013-11-19 21:40:55,856::task::495::TaskManager.Task::(_debug) Task 89ffaf2f-9202-4e27-bb27-98b9387e7b94: moving from state preparing -> state finished
>Thread-15911735::DEBUG::2013-11-19 21:40:55,857::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911735::DEBUG::2013-11-19 21:40:55,858::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911735::DEBUG::2013-11-19 21:40:55,858::task::495::TaskManager.Task::(_debug) Task 89ffaf2f-9202-4e27-bb27-98b9387e7b94: ref 0 aborting False
>Thread-15911735::INFO::2013-11-19 21:40:55,859::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000283002853394', 'lastCheck': 1384897247.586884, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00138282775879', 'lastCheck': 1384897247.999289, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897254.215116, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000448942184448', 'lastCheck': 1384897255.3991621, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:41:04,227::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911741::INFO::2013-11-19 21:41:06,233::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911741::DEBUG::2013-11-19 21:41:06,234::task::495::TaskManager.Task::(_debug) Task 6e52a5a9-5519-427c-af78-f596a06c3024: moving from state init -> state preparing
>Thread-15911741::DEBUG::2013-11-19 21:41:06,235::task::495::TaskManager.Task::(_debug) Task 6e52a5a9-5519-427c-af78-f596a06c3024: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000331878662109', 'lastCheck': 1384897257.5987539, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00206112861633', 'lastCheck': 1384897258.0155859, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897264.2284961, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000365972518921', 'lastCheck': 1384897265.4106901, 'valid': True, 'code': 0}}
>Thread-15911741::DEBUG::2013-11-19 21:41:06,243::task::495::TaskManager.Task::(_debug) Task 6e52a5a9-5519-427c-af78-f596a06c3024: moving from state preparing -> state finished
>Thread-15911741::DEBUG::2013-11-19 21:41:06,243::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911741::DEBUG::2013-11-19 21:41:06,244::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911741::DEBUG::2013-11-19 21:41:06,245::task::495::TaskManager.Task::(_debug) Task 6e52a5a9-5519-427c-af78-f596a06c3024: ref 0 aborting False
>Thread-15911741::INFO::2013-11-19 21:41:06,246::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000331878662109', 'lastCheck': 1384897257.5987539, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00206112861633', 'lastCheck': 1384897258.0155859, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897264.2284961, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000365972518921', 'lastCheck': 1384897265.4106901, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:41:14,241::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911747::INFO::2013-11-19 21:41:17,290::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911747::DEBUG::2013-11-19 21:41:17,290::task::495::TaskManager.Task::(_debug) Task 9aac206d-f5c0-42da-8df8-401652ad34b8: moving from state init -> state preparing
>Thread-15911747::DEBUG::2013-11-19 21:41:17,291::task::495::TaskManager.Task::(_debug) Task 9aac206d-f5c0-42da-8df8-401652ad34b8: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00109004974365', 'lastCheck': 1384897267.613065, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0494320392609', 'lastCheck': 1384897268.079035, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897274.2418351, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000361204147339', 'lastCheck': 1384897275.422224, 'valid': True, 'code': 0}}
>Thread-15911747::DEBUG::2013-11-19 21:41:17,300::task::495::TaskManager.Task::(_debug) Task 9aac206d-f5c0-42da-8df8-401652ad34b8: moving from state preparing -> state finished
>Thread-15911747::DEBUG::2013-11-19 21:41:17,300::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911747::DEBUG::2013-11-19 21:41:17,301::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911747::DEBUG::2013-11-19 21:41:17,301::task::495::TaskManager.Task::(_debug) Task 9aac206d-f5c0-42da-8df8-401652ad34b8: ref 0 aborting False
>Thread-15911747::INFO::2013-11-19 21:41:17,306::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00109004974365', 'lastCheck': 1384897267.613065, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0494320392609', 'lastCheck': 1384897268.079035, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897274.2418351, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000361204147339', 'lastCheck': 1384897275.422224, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:41:24,254::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911753::INFO::2013-11-19 21:41:27,447::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911753::DEBUG::2013-11-19 21:41:27,447::task::495::TaskManager.Task::(_debug) Task 2b899450-9168-4b0d-b984-cbfd9aabdcab: moving from state init -> state preparing
>Thread-15911753::DEBUG::2013-11-19 21:41:27,448::task::495::TaskManager.Task::(_debug) Task 2b899450-9168-4b0d-b984-cbfd9aabdcab: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000324964523315', 'lastCheck': 1384897277.6257949, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00133109092712', 'lastCheck': 1384897278.0942831, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897284.255214, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000363111495972', 'lastCheck': 1384897285.4337831, 'valid': True, 'code': 0}}
>Thread-15911753::DEBUG::2013-11-19 21:41:27,449::task::495::TaskManager.Task::(_debug) Task 2b899450-9168-4b0d-b984-cbfd9aabdcab: moving from state preparing -> state finished
>Thread-15911753::DEBUG::2013-11-19 21:41:27,449::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911753::DEBUG::2013-11-19 21:41:27,450::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911753::DEBUG::2013-11-19 21:41:27,451::task::495::TaskManager.Task::(_debug) Task 2b899450-9168-4b0d-b984-cbfd9aabdcab: ref 0 aborting False
>Thread-15911753::INFO::2013-11-19 21:41:27,457::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000324964523315', 'lastCheck': 1384897277.6257949, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00133109092712', 'lastCheck': 1384897278.0942831, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897284.255214, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000363111495972', 'lastCheck': 1384897285.4337831, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:41:34,268::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911759::INFO::2013-11-19 21:41:37,601::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911759::DEBUG::2013-11-19 21:41:37,605::task::495::TaskManager.Task::(_debug) Task 50747220-a0bb-42d4-81c6-4e1392b05369: moving from state init -> state preparing
>Thread-15911759::DEBUG::2013-11-19 21:41:37,607::task::495::TaskManager.Task::(_debug) Task 50747220-a0bb-42d4-81c6-4e1392b05369: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000346183776855', 'lastCheck': 1384897287.638407, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142002105713', 'lastCheck': 1384897288.109844, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897294.2691131, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000498056411743', 'lastCheck': 1384897295.4455531, 'valid': True, 'code': 0}}
>Thread-15911759::DEBUG::2013-11-19 21:41:37,629::task::495::TaskManager.Task::(_debug) Task 50747220-a0bb-42d4-81c6-4e1392b05369: moving from state preparing -> state finished
>Thread-15911759::DEBUG::2013-11-19 21:41:37,631::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911759::DEBUG::2013-11-19 21:41:37,632::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911759::DEBUG::2013-11-19 21:41:37,632::task::495::TaskManager.Task::(_debug) Task 50747220-a0bb-42d4-81c6-4e1392b05369: ref 0 aborting False
>Thread-15911759::INFO::2013-11-19 21:41:37,638::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000346183776855', 'lastCheck': 1384897287.638407, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142002105713', 'lastCheck': 1384897288.109844, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897294.2691131, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000498056411743', 'lastCheck': 1384897295.4455531, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:41:44,281::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911765::INFO::2013-11-19 21:41:47,756::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911765::DEBUG::2013-11-19 21:41:47,757::task::495::TaskManager.Task::(_debug) Task 47205ac4-e502-484a-b7b7-c8019e172b96: moving from state init -> state preparing
>Thread-15911765::DEBUG::2013-11-19 21:41:47,757::task::495::TaskManager.Task::(_debug) Task 47205ac4-e502-484a-b7b7-c8019e172b96: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000342130661011', 'lastCheck': 1384897307.6631291, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00136780738831', 'lastCheck': 1384897298.125643, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897304.2824919, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000363826751709', 'lastCheck': 1384897305.4571559, 'valid': True, 'code': 0}}
>Thread-15911765::DEBUG::2013-11-19 21:41:47,769::task::495::TaskManager.Task::(_debug) Task 47205ac4-e502-484a-b7b7-c8019e172b96: moving from state preparing -> state finished
>Thread-15911765::DEBUG::2013-11-19 21:41:47,769::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911765::DEBUG::2013-11-19 21:41:47,770::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911765::DEBUG::2013-11-19 21:41:47,770::task::495::TaskManager.Task::(_debug) Task 47205ac4-e502-484a-b7b7-c8019e172b96: ref 0 aborting False
>Thread-15911765::INFO::2013-11-19 21:41:47,776::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000342130661011', 'lastCheck': 1384897307.6631291, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00136780738831', 'lastCheck': 1384897298.125643, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897304.2824919, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000363826751709', 'lastCheck': 1384897305.4571559, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:41:54,295::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911771::INFO::2013-11-19 21:41:57,905::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911771::DEBUG::2013-11-19 21:41:57,906::task::495::TaskManager.Task::(_debug) Task 1aa36bf0-1e54-460a-8c93-f8bafcf11506: moving from state init -> state preparing
>Thread-15911771::DEBUG::2013-11-19 21:41:57,906::task::495::TaskManager.Task::(_debug) Task 1aa36bf0-1e54-460a-8c93-f8bafcf11506: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000350952148438', 'lastCheck': 1384897317.6756811, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0013530254364', 'lastCheck': 1384897308.1410539, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897314.295814, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000368118286133', 'lastCheck': 1384897315.4687469, 'valid': True, 'code': 0}}
>Thread-15911771::DEBUG::2013-11-19 21:41:57,917::task::495::TaskManager.Task::(_debug) Task 1aa36bf0-1e54-460a-8c93-f8bafcf11506: moving from state preparing -> state finished
>Thread-15911771::DEBUG::2013-11-19 21:41:57,917::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911771::DEBUG::2013-11-19 21:41:57,918::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911771::DEBUG::2013-11-19 21:41:57,918::task::495::TaskManager.Task::(_debug) Task 1aa36bf0-1e54-460a-8c93-f8bafcf11506: ref 0 aborting False
>Thread-15911771::INFO::2013-11-19 21:41:57,919::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000350952148438', 'lastCheck': 1384897317.6756811, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0013530254364', 'lastCheck': 1384897308.1410539, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897314.295814, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000368118286133', 'lastCheck': 1384897315.4687469, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:42:04,308::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911777::INFO::2013-11-19 21:42:09,667::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911777::DEBUG::2013-11-19 21:42:09,668::task::495::TaskManager.Task::(_debug) Task 77deb013-fada-4bdf-88e8-6ea60cbe6548: moving from state init -> state preparing
>Thread-15911777::DEBUG::2013-11-19 21:42:09,669::task::495::TaskManager.Task::(_debug) Task 77deb013-fada-4bdf-88e8-6ea60cbe6548: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00036883354187', 'lastCheck': 1384897327.6883531, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00139307975769', 'lastCheck': 1384897328.1722939, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897324.30955, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000443935394287', 'lastCheck': 1384897325.4805, 'valid': True, 'code': 0}}
>Thread-15911777::DEBUG::2013-11-19 21:42:09,677::task::495::TaskManager.Task::(_debug) Task 77deb013-fada-4bdf-88e8-6ea60cbe6548: moving from state preparing -> state finished
>Thread-15911777::DEBUG::2013-11-19 21:42:09,678::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911777::DEBUG::2013-11-19 21:42:09,679::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911777::DEBUG::2013-11-19 21:42:09,679::task::495::TaskManager.Task::(_debug) Task 77deb013-fada-4bdf-88e8-6ea60cbe6548: ref 0 aborting False
>Thread-15911777::INFO::2013-11-19 21:42:09,680::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00036883354187', 'lastCheck': 1384897327.6883531, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00139307975769', 'lastCheck': 1384897328.1722939, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897324.30955, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000443935394287', 'lastCheck': 1384897325.4805, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:42:14,322::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911783::INFO::2013-11-19 21:42:19,938::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911783::DEBUG::2013-11-19 21:42:19,938::task::495::TaskManager.Task::(_debug) Task 7645550b-6d8a-4b79-bb18-5801c0686217: moving from state init -> state preparing
>Thread-15911783::DEBUG::2013-11-19 21:42:19,954::task::495::TaskManager.Task::(_debug) Task 7645550b-6d8a-4b79-bb18-5801c0686217: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00035285949707', 'lastCheck': 1384897337.700597, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00138115882874', 'lastCheck': 1384897338.1876769, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897334.3231189, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000401020050049', 'lastCheck': 1384897335.4921689, 'valid': True, 'code': 0}}
>Thread-15911783::DEBUG::2013-11-19 21:42:19,966::task::495::TaskManager.Task::(_debug) Task 7645550b-6d8a-4b79-bb18-5801c0686217: moving from state preparing -> state finished
>Thread-15911783::DEBUG::2013-11-19 21:42:19,966::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911783::DEBUG::2013-11-19 21:42:19,967::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911783::DEBUG::2013-11-19 21:42:19,967::task::495::TaskManager.Task::(_debug) Task 7645550b-6d8a-4b79-bb18-5801c0686217: ref 0 aborting False
>Thread-15911783::INFO::2013-11-19 21:42:19,971::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00035285949707', 'lastCheck': 1384897337.700597, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00138115882874', 'lastCheck': 1384897338.1876769, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897334.3231189, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000401020050049', 'lastCheck': 1384897335.4921689, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:42:24,336::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-19 21:42:25,503::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-19 21:42:26,207::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15911789::INFO::2013-11-19 21:42:30,067::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911789::DEBUG::2013-11-19 21:42:30,068::task::495::TaskManager.Task::(_debug) Task 24900dba-b140-421b-b45c-f2c0e36fe3b4: moving from state init -> state preparing
>Thread-15911789::DEBUG::2013-11-19 21:42:30,068::task::495::TaskManager.Task::(_debug) Task 24900dba-b140-421b-b45c-f2c0e36fe3b4: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000453948974609', 'lastCheck': 1384897347.746563, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00146317481995', 'lastCheck': 1384897348.2034869, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897344.3370881, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000348091125488', 'lastCheck': 1384897346.2093611, 'valid': True, 'code': 0}}
>Thread-15911789::DEBUG::2013-11-19 21:42:30,069::task::495::TaskManager.Task::(_debug) Task 24900dba-b140-421b-b45c-f2c0e36fe3b4: moving from state preparing -> state finished
>Thread-15911789::DEBUG::2013-11-19 21:42:30,070::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911789::DEBUG::2013-11-19 21:42:30,070::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911789::DEBUG::2013-11-19 21:42:30,071::task::495::TaskManager.Task::(_debug) Task 24900dba-b140-421b-b45c-f2c0e36fe3b4: ref 0 aborting False
>Thread-15911789::INFO::2013-11-19 21:42:30,072::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000453948974609', 'lastCheck': 1384897347.746563, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00146317481995', 'lastCheck': 1384897348.2034869, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897344.3370881, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000348091125488', 'lastCheck': 1384897346.2093611, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:42:34,349::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911795::INFO::2013-11-19 21:42:40,169::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911795::DEBUG::2013-11-19 21:42:40,170::task::495::TaskManager.Task::(_debug) Task d733679b-ad49-4d5d-b943-8d8beec92835: moving from state init -> state preparing
>Thread-15911795::DEBUG::2013-11-19 21:42:40,171::task::495::TaskManager.Task::(_debug) Task d733679b-ad49-4d5d-b943-8d8beec92835: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000780820846558', 'lastCheck': 1384897357.7631941, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00143909454346', 'lastCheck': 1384897358.2191319, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897354.350445, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00037693977356', 'lastCheck': 1384897356.2208469, 'valid': True, 'code': 0}}
>Thread-15911795::DEBUG::2013-11-19 21:42:40,171::task::495::TaskManager.Task::(_debug) Task d733679b-ad49-4d5d-b943-8d8beec92835: moving from state preparing -> state finished
>Thread-15911795::DEBUG::2013-11-19 21:42:40,172::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911795::DEBUG::2013-11-19 21:42:40,173::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911795::DEBUG::2013-11-19 21:42:40,173::task::495::TaskManager.Task::(_debug) Task d733679b-ad49-4d5d-b943-8d8beec92835: ref 0 aborting False
>Thread-15911795::INFO::2013-11-19 21:42:40,174::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000780820846558', 'lastCheck': 1384897357.7631941, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00143909454346', 'lastCheck': 1384897358.2191319, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897354.350445, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00037693977356', 'lastCheck': 1384897356.2208469, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:42:44,363::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911801::INFO::2013-11-19 21:42:50,324::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911801::DEBUG::2013-11-19 21:42:50,325::task::495::TaskManager.Task::(_debug) Task 7c0632c9-77db-4014-bf09-016f24d29584: moving from state init -> state preparing
>Thread-15911801::DEBUG::2013-11-19 21:42:50,326::task::495::TaskManager.Task::(_debug) Task 7c0632c9-77db-4014-bf09-016f24d29584: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000335931777954', 'lastCheck': 1384897367.776026, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00139713287354', 'lastCheck': 1384897368.2342999, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897364.3641069, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000427961349487', 'lastCheck': 1384897366.231375, 'valid': True, 'code': 0}}
>Thread-15911801::DEBUG::2013-11-19 21:42:50,336::task::495::TaskManager.Task::(_debug) Task 7c0632c9-77db-4014-bf09-016f24d29584: moving from state preparing -> state finished
>Thread-15911801::DEBUG::2013-11-19 21:42:50,337::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911801::DEBUG::2013-11-19 21:42:50,338::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911801::DEBUG::2013-11-19 21:42:50,339::task::495::TaskManager.Task::(_debug) Task 7c0632c9-77db-4014-bf09-016f24d29584: ref 0 aborting False
>Thread-15911801::INFO::2013-11-19 21:42:50,344::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000335931777954', 'lastCheck': 1384897367.776026, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00139713287354', 'lastCheck': 1384897368.2342999, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897364.3641069, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000427961349487', 'lastCheck': 1384897366.231375, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:42:54,376::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911807::INFO::2013-11-19 21:43:00,505::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911807::DEBUG::2013-11-19 21:43:00,506::task::495::TaskManager.Task::(_debug) Task b6d29b79-487f-4814-b86c-1a60e8ed2014: moving from state init -> state preparing
>Thread-15911807::DEBUG::2013-11-19 21:43:00,507::task::495::TaskManager.Task::(_debug) Task b6d29b79-487f-4814-b86c-1a60e8ed2014: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000265121459961', 'lastCheck': 1384897377.7888069, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00211405754089', 'lastCheck': 1384897378.2504871, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897374.377619, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000397205352783', 'lastCheck': 1384897376.242481, 'valid': True, 'code': 0}}
>Thread-15911807::DEBUG::2013-11-19 21:43:00,514::task::495::TaskManager.Task::(_debug) Task b6d29b79-487f-4814-b86c-1a60e8ed2014: moving from state preparing -> state finished
>Thread-15911807::DEBUG::2013-11-19 21:43:00,515::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911807::DEBUG::2013-11-19 21:43:00,515::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911807::DEBUG::2013-11-19 21:43:00,516::task::495::TaskManager.Task::(_debug) Task b6d29b79-487f-4814-b86c-1a60e8ed2014: ref 0 aborting False
>Thread-15911807::INFO::2013-11-19 21:43:00,516::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000265121459961', 'lastCheck': 1384897377.7888069, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00211405754089', 'lastCheck': 1384897378.2504871, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897374.377619, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000397205352783', 'lastCheck': 1384897376.242481, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:43:04,390::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911813::INFO::2013-11-19 21:43:10,624::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911813::DEBUG::2013-11-19 21:43:10,625::task::495::TaskManager.Task::(_debug) Task e27d59a6-f41a-43e4-af04-d755461e9090: moving from state init -> state preparing
>Thread-15911813::DEBUG::2013-11-19 21:43:10,625::task::495::TaskManager.Task::(_debug) Task e27d59a6-f41a-43e4-af04-d755461e9090: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000328063964844', 'lastCheck': 1384897387.801584, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00139093399048', 'lastCheck': 1384897388.2647991, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897384.391139, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000361919403076', 'lastCheck': 1384897386.2542601, 'valid': True, 'code': 0}}
>Thread-15911813::DEBUG::2013-11-19 21:43:10,626::task::495::TaskManager.Task::(_debug) Task e27d59a6-f41a-43e4-af04-d755461e9090: moving from state preparing -> state finished
>Thread-15911813::DEBUG::2013-11-19 21:43:10,627::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911813::DEBUG::2013-11-19 21:43:10,627::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911813::DEBUG::2013-11-19 21:43:10,628::task::495::TaskManager.Task::(_debug) Task e27d59a6-f41a-43e4-af04-d755461e9090: ref 0 aborting False
>Thread-15911813::INFO::2013-11-19 21:43:10,629::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000328063964844', 'lastCheck': 1384897387.801584, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00139093399048', 'lastCheck': 1384897388.2647991, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897384.391139, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000361919403076', 'lastCheck': 1384897386.2542601, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:43:14,403::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911819::INFO::2013-11-19 21:43:20,739::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911819::DEBUG::2013-11-19 21:43:20,740::task::495::TaskManager.Task::(_debug) Task 2237a240-fc94-42f0-88a4-be31b39400d7: moving from state init -> state preparing
>Thread-15911819::DEBUG::2013-11-19 21:43:20,741::task::495::TaskManager.Task::(_debug) Task 2237a240-fc94-42f0-88a4-be31b39400d7: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000323057174683', 'lastCheck': 1384897397.814194, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00140595436096', 'lastCheck': 1384897398.2798901, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897394.404515, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000301122665405', 'lastCheck': 1384897396.2657509, 'valid': True, 'code': 0}}
>Thread-15911819::DEBUG::2013-11-19 21:43:20,748::task::495::TaskManager.Task::(_debug) Task 2237a240-fc94-42f0-88a4-be31b39400d7: moving from state preparing -> state finished
>Thread-15911819::DEBUG::2013-11-19 21:43:20,748::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911819::DEBUG::2013-11-19 21:43:20,749::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911819::DEBUG::2013-11-19 21:43:20,749::task::495::TaskManager.Task::(_debug) Task 2237a240-fc94-42f0-88a4-be31b39400d7: ref 0 aborting False
>Thread-15911819::INFO::2013-11-19 21:43:20,754::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000323057174683', 'lastCheck': 1384897397.814194, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00140595436096', 'lastCheck': 1384897398.2798901, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897394.404515, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000301122665405', 'lastCheck': 1384897396.2657509, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:43:24,417::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911825::INFO::2013-11-19 21:43:30,871::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911825::DEBUG::2013-11-19 21:43:30,872::task::495::TaskManager.Task::(_debug) Task cbe2c401-410c-4398-9518-41138fe91265: moving from state init -> state preparing
>Thread-15911825::DEBUG::2013-11-19 21:43:30,873::task::495::TaskManager.Task::(_debug) Task cbe2c401-410c-4398-9518-41138fe91265: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000292778015137', 'lastCheck': 1384897407.826875, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0013439655304', 'lastCheck': 1384897408.2949741, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897404.4179709, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000377893447876', 'lastCheck': 1384897406.2773399, 'valid': True, 'code': 0}}
>Thread-15911825::DEBUG::2013-11-19 21:43:30,878::task::495::TaskManager.Task::(_debug) Task cbe2c401-410c-4398-9518-41138fe91265: moving from state preparing -> state finished
>Thread-15911825::DEBUG::2013-11-19 21:43:30,879::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911825::DEBUG::2013-11-19 21:43:30,879::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911825::DEBUG::2013-11-19 21:43:30,880::task::495::TaskManager.Task::(_debug) Task cbe2c401-410c-4398-9518-41138fe91265: ref 0 aborting False
>Thread-15911825::INFO::2013-11-19 21:43:30,885::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000292778015137', 'lastCheck': 1384897407.826875, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0013439655304', 'lastCheck': 1384897408.2949741, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897404.4179709, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000377893447876', 'lastCheck': 1384897406.2773399, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:43:34,431::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911831::INFO::2013-11-19 21:43:40,995::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911831::DEBUG::2013-11-19 21:43:40,996::task::495::TaskManager.Task::(_debug) Task 2bc1ac6a-55da-448e-ba3b-eb98b44ba5d1: moving from state init -> state preparing
>Thread-15911831::DEBUG::2013-11-19 21:43:40,997::task::495::TaskManager.Task::(_debug) Task 2bc1ac6a-55da-448e-ba3b-eb98b44ba5d1: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000269889831543', 'lastCheck': 1384897417.839484, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00145602226257', 'lastCheck': 1384897418.3104081, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897414.4318049, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000456809997559', 'lastCheck': 1384897416.2890429, 'valid': True, 'code': 0}}
>Thread-15911831::DEBUG::2013-11-19 21:43:40,998::task::495::TaskManager.Task::(_debug) Task 2bc1ac6a-55da-448e-ba3b-eb98b44ba5d1: moving from state preparing -> state finished
>Thread-15911831::DEBUG::2013-11-19 21:43:40,998::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911831::DEBUG::2013-11-19 21:43:40,999::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911831::DEBUG::2013-11-19 21:43:41,000::task::495::TaskManager.Task::(_debug) Task 2bc1ac6a-55da-448e-ba3b-eb98b44ba5d1: ref 0 aborting False
>Thread-15911831::INFO::2013-11-19 21:43:41,008::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000269889831543', 'lastCheck': 1384897417.839484, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00145602226257', 'lastCheck': 1384897418.3104081, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897414.4318049, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000456809997559', 'lastCheck': 1384897416.2890429, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:43:44,444::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911837::INFO::2013-11-19 21:43:51,099::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911837::DEBUG::2013-11-19 21:43:51,099::task::495::TaskManager.Task::(_debug) Task ce564904-2f04-474a-bd0f-1dfa3a32a839: moving from state init -> state preparing
>Thread-15911837::DEBUG::2013-11-19 21:43:51,100::task::495::TaskManager.Task::(_debug) Task ce564904-2f04-474a-bd0f-1dfa3a32a839: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00029182434082', 'lastCheck': 1384897427.8520401, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00132393836975', 'lastCheck': 1384897428.3255551, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897424.4452541, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000329971313477', 'lastCheck': 1384897426.300627, 'valid': True, 'code': 0}}
>Thread-15911837::DEBUG::2013-11-19 21:43:51,101::task::495::TaskManager.Task::(_debug) Task ce564904-2f04-474a-bd0f-1dfa3a32a839: moving from state preparing -> state finished
>Thread-15911837::DEBUG::2013-11-19 21:43:51,102::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911837::DEBUG::2013-11-19 21:43:51,103::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911837::DEBUG::2013-11-19 21:43:51,103::task::495::TaskManager.Task::(_debug) Task ce564904-2f04-474a-bd0f-1dfa3a32a839: ref 0 aborting False
>Thread-15911837::INFO::2013-11-19 21:43:51,104::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00029182434082', 'lastCheck': 1384897427.8520401, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00132393836975', 'lastCheck': 1384897428.3255551, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897424.4452541, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000329971313477', 'lastCheck': 1384897426.300627, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:43:54,458::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911843::INFO::2013-11-19 21:44:01,358::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911843::DEBUG::2013-11-19 21:44:01,359::task::495::TaskManager.Task::(_debug) Task 78b94a99-6bc4-4efb-95b8-c884e8cbf401: moving from state init -> state preparing
>Thread-15911843::DEBUG::2013-11-19 21:44:01,359::task::495::TaskManager.Task::(_debug) Task 78b94a99-6bc4-4efb-95b8-c884e8cbf401: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00026798248291', 'lastCheck': 1384897437.864686, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00208711624146', 'lastCheck': 1384897438.342711, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897434.458694, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000418901443481', 'lastCheck': 1384897436.3123901, 'valid': True, 'code': 0}}
>Thread-15911843::DEBUG::2013-11-19 21:44:01,370::task::495::TaskManager.Task::(_debug) Task 78b94a99-6bc4-4efb-95b8-c884e8cbf401: moving from state preparing -> state finished
>Thread-15911843::DEBUG::2013-11-19 21:44:01,370::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911843::DEBUG::2013-11-19 21:44:01,371::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911843::DEBUG::2013-11-19 21:44:01,371::task::495::TaskManager.Task::(_debug) Task 78b94a99-6bc4-4efb-95b8-c884e8cbf401: ref 0 aborting False
>Thread-15911843::INFO::2013-11-19 21:44:01,374::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00026798248291', 'lastCheck': 1384897437.864686, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00208711624146', 'lastCheck': 1384897438.342711, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384897434.458694, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000418901443481', 'lastCheck': 1384897436.3123901, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 21:44:04,471::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:44:14,485::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:44:24,498::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:44:34,512::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:44:44,525::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:44:54,538::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-19 21:44:57,937::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 21:44:58,609::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:45:04,552::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:45:14,566::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:45:24,579::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:45:34,593::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:45:44,607::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:45:54,620::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:46:04,634::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:46:14,648::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:46:24,661::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:46:34,675::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:46:44,689::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:46:54,702::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:47:04,716::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:47:14,729::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:47:24,743::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-19 21:47:26,556::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-19 21:47:27,218::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:47:34,778::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:47:44,792::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:47:54,806::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:48:04,820::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:48:14,834::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:48:24,847::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:48:34,861::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:48:44,875::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:48:54,888::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:49:04,902::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:49:14,915::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:49:24,929::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:49:34,943::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:49:44,957::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:49:54,970::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-19 21:49:59,005::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 21:49:59,650::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:50:04,985::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:50:14,999::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:50:25,013::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:50:35,027::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:50:45,041::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:50:55,055::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:51:05,068::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:51:15,082::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:51:25,096::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:51:35,110::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:51:45,124::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:51:55,138::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:52:05,151::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:52:15,165::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:52:25,179::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-19 21:52:27,567::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-19 21:52:28,221::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:52:35,193::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:52:45,206::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:52:55,220::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:53:05,234::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:53:15,247::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:53:25,261::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:53:35,274::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:53:45,287::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:53:55,300::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:54:05,314::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:54:15,327::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:54:25,340::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:54:35,354::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:54:45,368::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:54:55,381::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-19 21:55:00,032::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 21:55:00,680::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:55:05,395::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:55:15,409::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:55:25,422::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:55:35,435::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:55:45,448::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:55:55,462::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:56:05,476::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:56:15,489::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:56:25,502::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:56:35,516::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:56:45,529::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:56:55,543::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:57:05,556::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:57:15,569::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:57:25,583::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-19 21:57:28,576::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-19 21:57:29,203::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 21:57:35,790::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:57:45,804::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:57:55,817::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:58:05,830::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:58:15,844::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:58:25,857::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:58:35,872::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:58:45,886::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:58:55,899::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:59:05,912::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:59:15,925::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:59:25,938::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:59:35,952::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:59:45,965::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 21:59:55,980::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-19 22:00:01,064::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 22:00:01,726::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 22:00:05,994::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:00:16,007::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:00:26,021::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:00:36,035::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:00:46,048::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:00:56,061::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:01:06,075::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:01:16,089::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:01:26,102::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:01:36,116::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:01:46,130::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:01:56,143::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:02:06,157::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:02:16,170::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:02:26,184::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-19 22:02:29,558::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-19 22:02:30,272::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 22:02:36,197::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:02:46,210::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:02:56,224::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:03:06,237::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:03:16,250::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:03:26,264::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:03:36,277::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:03:46,291::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:03:56,305::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:04:06,319::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:04:16,332::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:04:26,346::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:04:36,360::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:04:46,373::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:04:56,387::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-19 22:05:02,113::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 22:05:02,794::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 22:05:06,401::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:05:16,414::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:05:26,428::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:05:36,443::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:05:46,457::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:05:56,470::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:06:06,484::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:06:16,498::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:06:26,511::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:06:36,525::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:06:46,538::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:06:56,552::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:07:06,565::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:07:16,579::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:07:26,592::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-19 22:07:30,618::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-19 22:07:31,265::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 22:07:36,606::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:07:46,620::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:07:56,634::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:08:06,647::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:08:16,661::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:08:26,675::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:08:36,688::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:08:46,702::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:08:56,716::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:09:06,729::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:09:16,743::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:09:26,757::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:09:36,771::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:09:46,785::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:09:56,798::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-19 22:10:03,180::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 22:10:03,821::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 22:10:06,813::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:10:16,827::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911847::INFO::2013-11-19 22:10:24,510::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911847::DEBUG::2013-11-19 22:10:24,511::task::495::TaskManager.Task::(_debug) Task 14271aee-57d1-483e-97e6-a7e7aee6e90f: moving from state init -> state preparing
>Thread-15911847::DEBUG::2013-11-19 22:10:24,519::task::495::TaskManager.Task::(_debug) Task 14271aee-57d1-483e-97e6-a7e7aee6e90f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000361204147339', 'lastCheck': 1384899023.850594, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142097473145', 'lastCheck': 1384899021.4222579, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899016.827668, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000737905502319', 'lastCheck': 1384899021.4634449, 'valid': True, 'code': 0}}
>Thread-15911847::DEBUG::2013-11-19 22:10:24,526::task::495::TaskManager.Task::(_debug) Task 14271aee-57d1-483e-97e6-a7e7aee6e90f: moving from state preparing -> state finished
>Thread-15911847::DEBUG::2013-11-19 22:10:24,527::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911847::DEBUG::2013-11-19 22:10:24,528::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911847::DEBUG::2013-11-19 22:10:24,528::task::495::TaskManager.Task::(_debug) Task 14271aee-57d1-483e-97e6-a7e7aee6e90f: ref 0 aborting False
>Thread-15911847::INFO::2013-11-19 22:10:24,531::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000361204147339', 'lastCheck': 1384899023.850594, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142097473145', 'lastCheck': 1384899021.4222579, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899016.827668, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000737905502319', 'lastCheck': 1384899021.4634449, 'valid': True, 'code': 0}}
>Thread-15911848::DEBUG::2013-11-19 22:10:24,564::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVdsCapabilities with () {}
>Thread-15911848::DEBUG::2013-11-19 22:10:25,000::utils::579::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-img-rhev' (cwd None)
>Thread-15911848::DEBUG::2013-11-19 22:10:25,418::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15911848::DEBUG::2013-11-19 22:10:25,420::utils::579::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-kvm-rhev' (cwd None)
>Thread-15911848::DEBUG::2013-11-19 22:10:25,863::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15911848::DEBUG::2013-11-19 22:10:25,865::utils::579::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" vdsm' (cwd None)
>Thread-15911848::DEBUG::2013-11-19 22:10:26,321::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15911848::DEBUG::2013-11-19 22:10:26,323::utils::579::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" spice-server' (cwd None)
>Thread-15911848::DEBUG::2013-11-19 22:10:26,750::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15911848::DEBUG::2013-11-19 22:10:26,751::utils::579::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" libvirt' (cwd None)
>Thread-1024767::ERROR::2013-11-19 22:10:26,997::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911848::DEBUG::2013-11-19 22:10:27,212::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15911848::DEBUG::2013-11-19 22:10:27,215::clientIF::59::vds::(wrapper) return getVdsCapabilities with {'status': {'message': 'Done', 'code': 0}, 'info': {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:c936ebb147a8'}], 'FC': []}, 'packages2': {'kernel': {'release': '279.11.1.el6.x86_64', 'buildtime': 1348297826.0, 'version': '2.6.32'}, 'spice-server': {'release': '10.el6', 'buildtime': '1337611492', 'version': '0.10.1'}, 'vdsm': {'release': '113.4.el6_3', 'buildtime': '1347534072', 'version': '4.9'}, 'qemu-kvm': {'release': '2.295.el6_3.2', 'buildtime': '1346673796', 'version': '0.12.1.2'}, 'libvirt': {'release': '21.el6_3.5', 'buildtime': '1348687577', 'version': '0.9.10'}, 'qemu-img': {'release': '2.295.el6_3.2', 'buildtime': '1346673796', 'version': '0.12.1.2'}}, 'cpuModel': 'Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz', 'hooks': {'before_vm_start': {'50_vhostmd': {'md5': '1635e45584995d9ad2629939cb0bcdbf'}}, 'after_vm_destroy': {'50_vhostmd': {'md5': '7e7509430304099f38e4a9de966af090'}}, 'before_vm_dehibernate': {'50_vhostmd': {'md5': '1635e45584995d9ad2629939cb0bcdbf'}}, 'before_vm_migrate_destination': {'50_vhostmd': {'md5': '1635e45584995d9ad2629939cb0bcdbf'}}}, 'vmTypes': ['kvm'], 'supportedProtocols': ['2.2', '2.3'], 'networks': {'rhevm': {'addr': '192.168.100.133', 'cfg': {'IPADDR': '192.168.100.133', 'GATEWAY': '192.168.100.1', 'DELAY': '0', 'NETMASK': '255.255.128.0', 'STP': 'no', 'DEVICE': 'rhevm', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'ports': ['bond0'], 'netmask': '255.255.128.0', 'stp': 'off', 'gateway': '192.168.100.1'}, 'DMZ': {'addr': '172.16.1.133', 'cfg': {'IPADDR': '172.16.1.133', 'DELAY': '0', 'NETMASK': '255.255.255.0', 'STP': 'no', 'DEVICE': 'DMZ', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'ports': ['eth0'], 'netmask': '255.255.255.0', 'stp': 'off', 'gateway': '0.0.0.0'}}, 'uuid': '30343536-3138-5A43-4A32-333630373442_AC:16:2D:76:EB:F0', 'lastClientIface': 'rhevm', 'nics': {'eth3': {'hwaddr': 'AC:16:2D:76:EB:F2', 'netmask': '', 'speed': 1000, 'addr': '', 'permhwaddr': 'AC:16:2D:76:EB:F3'}, 'eth2': {'hwaddr': 'AC:16:2D:76:EB:F2', 'netmask': '', 'speed': 1000, 'addr': '', 'permhwaddr': 'AC:16:2D:76:EB:F2'}, 'eth1': {'hwaddr': 'AC:16:2D:76:EB:F1', 'netmask': '', 'speed': 0, 'addr': ''}, 'eth0': {'hwaddr': 'AC:16:2D:76:EB:F0', 'netmask': '', 'speed': 1000, 'addr': ''}}, 'software_revision': '113.4', 'management_ip': '192.168.100.133', 'clusterLevels': ['3.0'], 'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,constant_tsc,arch_perfmon,pebs,bts,rep_good,xtopology,nonstop_tsc,aperfmperf,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,dca,sse4_1,sse4_2,x2apic,popcnt,aes,xsave,avx,lahf_lm,ida,arat,epb,xsaveopt,pln,pts,dts,tpr_shadow,vnmi,flexpriority,ept,vpid,model_486,model_pentium,model_pentium2,model_pentium3,model_pentiumpro,model_qemu32,model_cpu64-rhel5,model_cpu64-rhel6,model_kvm32,model_coreduo,model_kvm64,model_core2duo,model_n270,model_Conroe,model_Penryn,model_Nehalem,model_Westmere,model_SandyBridge,model_Opteron_G1', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:c936ebb147a8', 'memSize': '64392', 'reservedMem': '256', 'bondings': {'bond4': {'hwaddr': '00:00:00:00:00:00', 'cfg': {}, 'netmask': '', 'addr': '', 'slaves': []}, 'bond0': {'hwaddr': 'AC:16:2D:76:EB:F2', 'cfg': {'DEVICE': 'bond0', 'BRIDGE': 'rhevm', 'BOOTPROTO': 'none', 'ONBOOT': 'yes', 'BONDING_OPTS': 'mode=1 miimon=100'}, 'netmask': '', 'addr': '', 'slaves': ['eth2', 'eth3']}, 'bond1': {'hwaddr': '00:00:00:00:00:00', 'cfg': {}, 'netmask': '', 'addr': '', 'slaves': []}, 'bond2': {'hwaddr': '00:00:00:00:00:00', 'cfg': {}, 'netmask': '', 'addr': '', 'slaves': []}, 'bond3': {'hwaddr': '00:00:00:00:00:00', 'cfg': {}, 'netmask': '', 'addr': '', 'slaves': []}}, 'software_version': '4.9', 'cpuSpeed': '1995.126', 'cpuSockets': '2', 'vlans': {}, 'cpuCores': '12', 'kvmEnabled': 'true', 'guestOverhead': '65', 'supportedRHEVMs': ['3.0'], 'version_name': 'Snow Man', 'emulatedMachines': [u'rhel6.3.0', u'pc', u'rhel6.2.0', u'rhel6.1.0', u'rhel6.0.0', u'rhel5.5.0', u'rhel5.4.4', u'rhel5.4.0'], 'operatingSystem': {'release': '20121012.0.el6_3', 'version': '6.3', 'name': 'RHEV Hypervisor'}, 'lastClient': '192.168.100.135'}}
>Thread-1024767::ERROR::2013-11-19 22:10:37,012::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911854::INFO::2013-11-19 22:10:37,583::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911854::DEBUG::2013-11-19 22:10:37,584::task::495::TaskManager.Task::(_debug) Task 9d4fff69-7262-4d57-9f74-ac44ccaf3f60: moving from state init -> state preparing
>Thread-15911854::DEBUG::2013-11-19 22:10:37,585::task::495::TaskManager.Task::(_debug) Task 9d4fff69-7262-4d57-9f74-ac44ccaf3f60: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000437021255493', 'lastCheck': 1384899034.0024991, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00153803825378', 'lastCheck': 1384899031.6465781, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899037.013145, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000581979751587', 'lastCheck': 1384899031.6420341, 'valid': True, 'code': 0}}
>Thread-15911854::DEBUG::2013-11-19 22:10:37,593::task::495::TaskManager.Task::(_debug) Task 9d4fff69-7262-4d57-9f74-ac44ccaf3f60: moving from state preparing -> state finished
>Thread-15911854::DEBUG::2013-11-19 22:10:37,594::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911854::DEBUG::2013-11-19 22:10:37,594::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911854::DEBUG::2013-11-19 22:10:37,595::task::495::TaskManager.Task::(_debug) Task 9d4fff69-7262-4d57-9f74-ac44ccaf3f60: ref 0 aborting False
>Thread-15911854::INFO::2013-11-19 22:10:37,595::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000437021255493', 'lastCheck': 1384899034.0024991, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00153803825378', 'lastCheck': 1384899031.6465781, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899037.013145, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000581979751587', 'lastCheck': 1384899031.6420341, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:10:47,025::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911860::INFO::2013-11-19 22:10:48,160::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911860::DEBUG::2013-11-19 22:10:48,160::task::495::TaskManager.Task::(_debug) Task 0ef391c0-feb6-40f0-bf80-6cb26d28a8ad: moving from state init -> state preparing
>Thread-15911860::DEBUG::2013-11-19 22:10:48,161::task::495::TaskManager.Task::(_debug) Task 0ef391c0-feb6-40f0-bf80-6cb26d28a8ad: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000416040420532', 'lastCheck': 1384899044.0154419, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00152993202209', 'lastCheck': 1384899041.662467, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899047.026602, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000422954559326', 'lastCheck': 1384899041.6535621, 'valid': True, 'code': 0}}
>Thread-15911860::DEBUG::2013-11-19 22:10:48,170::task::495::TaskManager.Task::(_debug) Task 0ef391c0-feb6-40f0-bf80-6cb26d28a8ad: moving from state preparing -> state finished
>Thread-15911860::DEBUG::2013-11-19 22:10:48,171::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911860::DEBUG::2013-11-19 22:10:48,171::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911860::DEBUG::2013-11-19 22:10:48,172::task::495::TaskManager.Task::(_debug) Task 0ef391c0-feb6-40f0-bf80-6cb26d28a8ad: ref 0 aborting False
>Thread-15911860::INFO::2013-11-19 22:10:48,173::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000416040420532', 'lastCheck': 1384899044.0154419, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00152993202209', 'lastCheck': 1384899041.662467, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899047.026602, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000422954559326', 'lastCheck': 1384899041.6535621, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:10:57,039::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911866::INFO::2013-11-19 22:10:58,351::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911866::DEBUG::2013-11-19 22:10:58,353::task::495::TaskManager.Task::(_debug) Task 1b5c8efe-97a7-4293-8775-db0a84abe06e: moving from state init -> state preparing
>Thread-15911866::DEBUG::2013-11-19 22:10:58,355::task::495::TaskManager.Task::(_debug) Task 1b5c8efe-97a7-4293-8775-db0a84abe06e: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000334024429321', 'lastCheck': 1384899054.0281579, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148105621338', 'lastCheck': 1384899051.6782041, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899057.040169, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000379085540771', 'lastCheck': 1384899051.6648729, 'valid': True, 'code': 0}}
>Thread-15911866::DEBUG::2013-11-19 22:10:58,368::task::495::TaskManager.Task::(_debug) Task 1b5c8efe-97a7-4293-8775-db0a84abe06e: moving from state preparing -> state finished
>Thread-15911866::DEBUG::2013-11-19 22:10:58,370::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911866::DEBUG::2013-11-19 22:10:58,371::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911866::DEBUG::2013-11-19 22:10:58,372::task::495::TaskManager.Task::(_debug) Task 1b5c8efe-97a7-4293-8775-db0a84abe06e: ref 0 aborting False
>Thread-15911866::INFO::2013-11-19 22:10:58,372::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000334024429321', 'lastCheck': 1384899054.0281579, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148105621338', 'lastCheck': 1384899051.6782041, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899057.040169, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000379085540771', 'lastCheck': 1384899051.6648729, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:11:07,053::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911872::INFO::2013-11-19 22:11:08,568::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911872::DEBUG::2013-11-19 22:11:08,569::task::495::TaskManager.Task::(_debug) Task 764ac90a-ae2d-4a4b-bfed-a32db3c1529c: moving from state init -> state preparing
>Thread-15911872::DEBUG::2013-11-19 22:11:08,570::task::495::TaskManager.Task::(_debug) Task 764ac90a-ae2d-4a4b-bfed-a32db3c1529c: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00036096572876', 'lastCheck': 1384899064.0409529, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.023845911026', 'lastCheck': 1384899061.7165401, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899067.053674, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000464916229248', 'lastCheck': 1384899061.6762969, 'valid': True, 'code': 0}}
>Thread-15911872::DEBUG::2013-11-19 22:11:08,581::task::495::TaskManager.Task::(_debug) Task 764ac90a-ae2d-4a4b-bfed-a32db3c1529c: moving from state preparing -> state finished
>Thread-15911872::DEBUG::2013-11-19 22:11:08,581::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911872::DEBUG::2013-11-19 22:11:08,582::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911872::DEBUG::2013-11-19 22:11:08,582::task::495::TaskManager.Task::(_debug) Task 764ac90a-ae2d-4a4b-bfed-a32db3c1529c: ref 0 aborting False
>Thread-15911872::INFO::2013-11-19 22:11:08,583::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00036096572876', 'lastCheck': 1384899064.0409529, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.023845911026', 'lastCheck': 1384899061.7165401, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899067.053674, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000464916229248', 'lastCheck': 1384899061.6762969, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:11:17,066::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911878::INFO::2013-11-19 22:11:18,745::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911878::DEBUG::2013-11-19 22:11:18,746::task::495::TaskManager.Task::(_debug) Task a4eb19d0-5836-4f24-b9e9-8ca481cca8c0: moving from state init -> state preparing
>Thread-15911878::DEBUG::2013-11-19 22:11:18,746::task::495::TaskManager.Task::(_debug) Task a4eb19d0-5836-4f24-b9e9-8ca481cca8c0: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000389099121094', 'lastCheck': 1384899074.0537479, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00166797637939', 'lastCheck': 1384899071.73228, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899077.067194, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00043511390686', 'lastCheck': 1384899071.6877, 'valid': True, 'code': 0}}
>Thread-15911878::DEBUG::2013-11-19 22:11:18,762::task::495::TaskManager.Task::(_debug) Task a4eb19d0-5836-4f24-b9e9-8ca481cca8c0: moving from state preparing -> state finished
>Thread-15911878::DEBUG::2013-11-19 22:11:18,763::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911878::DEBUG::2013-11-19 22:11:18,763::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911878::DEBUG::2013-11-19 22:11:18,764::task::495::TaskManager.Task::(_debug) Task a4eb19d0-5836-4f24-b9e9-8ca481cca8c0: ref 0 aborting False
>Thread-15911878::INFO::2013-11-19 22:11:18,769::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000389099121094', 'lastCheck': 1384899074.0537479, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00166797637939', 'lastCheck': 1384899071.73228, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899077.067194, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00043511390686', 'lastCheck': 1384899071.6877, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:11:27,080::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911884::INFO::2013-11-19 22:11:28,893::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911884::DEBUG::2013-11-19 22:11:28,894::task::495::TaskManager.Task::(_debug) Task 9343605a-7b6c-4f22-8078-b2d80f380842: moving from state init -> state preparing
>Thread-15911884::DEBUG::2013-11-19 22:11:28,894::task::495::TaskManager.Task::(_debug) Task 9343605a-7b6c-4f22-8078-b2d80f380842: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000363826751709', 'lastCheck': 1384899084.066514, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00139093399048', 'lastCheck': 1384899081.7479241, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899087.0807371, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000458002090454', 'lastCheck': 1384899081.699137, 'valid': True, 'code': 0}}
>Thread-15911884::DEBUG::2013-11-19 22:11:28,907::task::495::TaskManager.Task::(_debug) Task 9343605a-7b6c-4f22-8078-b2d80f380842: moving from state preparing -> state finished
>Thread-15911884::DEBUG::2013-11-19 22:11:28,907::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911884::DEBUG::2013-11-19 22:11:28,908::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911884::DEBUG::2013-11-19 22:11:28,908::task::495::TaskManager.Task::(_debug) Task 9343605a-7b6c-4f22-8078-b2d80f380842: ref 0 aborting False
>Thread-15911884::INFO::2013-11-19 22:11:28,909::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000363826751709', 'lastCheck': 1384899084.066514, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00139093399048', 'lastCheck': 1384899081.7479241, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899087.0807371, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000458002090454', 'lastCheck': 1384899081.699137, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:11:37,093::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911890::INFO::2013-11-19 22:11:39,039::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911890::DEBUG::2013-11-19 22:11:39,039::task::495::TaskManager.Task::(_debug) Task d0b99a9b-c854-4de9-8305-e7e38153dee8: moving from state init -> state preparing
>Thread-15911890::DEBUG::2013-11-19 22:11:39,057::task::495::TaskManager.Task::(_debug) Task d0b99a9b-c854-4de9-8305-e7e38153dee8: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000380039215088', 'lastCheck': 1384899094.0793469, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00135087966919', 'lastCheck': 1384899091.763551, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899097.0946281, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00042200088501', 'lastCheck': 1384899091.710542, 'valid': True, 'code': 0}}
>Thread-15911890::DEBUG::2013-11-19 22:11:39,063::task::495::TaskManager.Task::(_debug) Task d0b99a9b-c854-4de9-8305-e7e38153dee8: moving from state preparing -> state finished
>Thread-15911890::DEBUG::2013-11-19 22:11:39,064::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911890::DEBUG::2013-11-19 22:11:39,064::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911890::DEBUG::2013-11-19 22:11:39,065::task::495::TaskManager.Task::(_debug) Task d0b99a9b-c854-4de9-8305-e7e38153dee8: ref 0 aborting False
>Thread-15911890::INFO::2013-11-19 22:11:39,069::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000380039215088', 'lastCheck': 1384899094.0793469, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00135087966919', 'lastCheck': 1384899091.763551, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899097.0946281, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00042200088501', 'lastCheck': 1384899091.710542, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:11:47,107::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911896::INFO::2013-11-19 22:11:49,214::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911896::DEBUG::2013-11-19 22:11:49,215::task::495::TaskManager.Task::(_debug) Task 48ee9297-c5bf-4cb6-9301-c2318cd056ba: moving from state init -> state preparing
>Thread-15911896::DEBUG::2013-11-19 22:11:49,215::task::495::TaskManager.Task::(_debug) Task 48ee9297-c5bf-4cb6-9301-c2318cd056ba: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000313997268677', 'lastCheck': 1384899104.0921569, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00143098831177', 'lastCheck': 1384899101.7794211, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899107.108434, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00046181678772', 'lastCheck': 1384899101.7220521, 'valid': True, 'code': 0}}
>Thread-15911896::DEBUG::2013-11-19 22:11:49,222::task::495::TaskManager.Task::(_debug) Task 48ee9297-c5bf-4cb6-9301-c2318cd056ba: moving from state preparing -> state finished
>Thread-15911896::DEBUG::2013-11-19 22:11:49,223::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911896::DEBUG::2013-11-19 22:11:49,224::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911896::DEBUG::2013-11-19 22:11:49,224::task::495::TaskManager.Task::(_debug) Task 48ee9297-c5bf-4cb6-9301-c2318cd056ba: ref 0 aborting False
>Thread-15911896::INFO::2013-11-19 22:11:49,227::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000313997268677', 'lastCheck': 1384899104.0921569, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00143098831177', 'lastCheck': 1384899101.7794211, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899107.108434, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00046181678772', 'lastCheck': 1384899101.7220521, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:11:57,121::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911902::INFO::2013-11-19 22:11:59,527::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911902::DEBUG::2013-11-19 22:11:59,528::task::495::TaskManager.Task::(_debug) Task d94339b9-fa3a-49d8-8380-ee69f1942961: moving from state init -> state preparing
>Thread-15911902::DEBUG::2013-11-19 22:11:59,528::task::495::TaskManager.Task::(_debug) Task d94339b9-fa3a-49d8-8380-ee69f1942961: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000380039215088', 'lastCheck': 1384899114.105006, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00133895874023', 'lastCheck': 1384899111.795001, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899117.1221471, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000370025634766', 'lastCheck': 1384899111.733439, 'valid': True, 'code': 0}}
>Thread-15911902::DEBUG::2013-11-19 22:11:59,539::task::495::TaskManager.Task::(_debug) Task d94339b9-fa3a-49d8-8380-ee69f1942961: moving from state preparing -> state finished
>Thread-15911902::DEBUG::2013-11-19 22:11:59,540::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911902::DEBUG::2013-11-19 22:11:59,540::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911902::DEBUG::2013-11-19 22:11:59,541::task::495::TaskManager.Task::(_debug) Task d94339b9-fa3a-49d8-8380-ee69f1942961: ref 0 aborting False
>Thread-15911902::INFO::2013-11-19 22:11:59,544::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000380039215088', 'lastCheck': 1384899114.105006, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00133895874023', 'lastCheck': 1384899111.795001, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899117.1221471, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000370025634766', 'lastCheck': 1384899111.733439, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:12:07,134::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911908::INFO::2013-11-19 22:12:09,681::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911908::DEBUG::2013-11-19 22:12:09,682::task::495::TaskManager.Task::(_debug) Task f3ad77ac-f656-44f7-a19f-f68ea5f5d722: moving from state init -> state preparing
>Thread-15911908::DEBUG::2013-11-19 22:12:09,682::task::495::TaskManager.Task::(_debug) Task f3ad77ac-f656-44f7-a19f-f68ea5f5d722: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000478029251099', 'lastCheck': 1384899124.118196, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00212097167969', 'lastCheck': 1384899121.81181, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899127.135011, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000392913818359', 'lastCheck': 1384899121.7448759, 'valid': True, 'code': 0}}
>Thread-15911908::DEBUG::2013-11-19 22:12:09,690::task::495::TaskManager.Task::(_debug) Task f3ad77ac-f656-44f7-a19f-f68ea5f5d722: moving from state preparing -> state finished
>Thread-15911908::DEBUG::2013-11-19 22:12:09,690::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911908::DEBUG::2013-11-19 22:12:09,691::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911908::DEBUG::2013-11-19 22:12:09,691::task::495::TaskManager.Task::(_debug) Task f3ad77ac-f656-44f7-a19f-f68ea5f5d722: ref 0 aborting False
>Thread-15911908::INFO::2013-11-19 22:12:09,692::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000478029251099', 'lastCheck': 1384899124.118196, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00212097167969', 'lastCheck': 1384899121.81181, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899127.135011, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000392913818359', 'lastCheck': 1384899121.7448759, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:12:17,151::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911914::INFO::2013-11-19 22:12:19,853::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911914::DEBUG::2013-11-19 22:12:19,854::task::495::TaskManager.Task::(_debug) Task 9a1c691c-0bd1-4c8d-afb4-12ac058301fd: moving from state init -> state preparing
>Thread-15911914::DEBUG::2013-11-19 22:12:19,854::task::495::TaskManager.Task::(_debug) Task 9a1c691c-0bd1-4c8d-afb4-12ac058301fd: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00041389465332', 'lastCheck': 1384899134.1315229, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00146389007568', 'lastCheck': 1384899131.8273871, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899137.1518691, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00037693977356', 'lastCheck': 1384899131.7563429, 'valid': True, 'code': 0}}
>Thread-15911914::DEBUG::2013-11-19 22:12:19,860::task::495::TaskManager.Task::(_debug) Task 9a1c691c-0bd1-4c8d-afb4-12ac058301fd: moving from state preparing -> state finished
>Thread-15911914::DEBUG::2013-11-19 22:12:19,861::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911914::DEBUG::2013-11-19 22:12:19,862::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911914::DEBUG::2013-11-19 22:12:19,862::task::495::TaskManager.Task::(_debug) Task 9a1c691c-0bd1-4c8d-afb4-12ac058301fd: ref 0 aborting False
>Thread-15911914::INFO::2013-11-19 22:12:19,867::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00041389465332', 'lastCheck': 1384899134.1315229, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00146389007568', 'lastCheck': 1384899131.8273871, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899137.1518691, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00037693977356', 'lastCheck': 1384899131.7563429, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:12:27,164::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911920::INFO::2013-11-19 22:12:30,060::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911920::DEBUG::2013-11-19 22:12:30,061::task::495::TaskManager.Task::(_debug) Task a480542f-aed7-4f9f-b8e4-03e53b967d9b: moving from state init -> state preparing
>Thread-15911920::DEBUG::2013-11-19 22:12:30,062::task::495::TaskManager.Task::(_debug) Task a480542f-aed7-4f9f-b8e4-03e53b967d9b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000303030014038', 'lastCheck': 1384899144.1441309, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148105621338', 'lastCheck': 1384899141.8429649, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899147.165333, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000361204147339', 'lastCheck': 1384899141.767771, 'valid': True, 'code': 0}}
>Thread-15911920::DEBUG::2013-11-19 22:12:30,072::task::495::TaskManager.Task::(_debug) Task a480542f-aed7-4f9f-b8e4-03e53b967d9b: moving from state preparing -> state finished
>Thread-15911920::DEBUG::2013-11-19 22:12:30,072::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911920::DEBUG::2013-11-19 22:12:30,073::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911920::DEBUG::2013-11-19 22:12:30,073::task::495::TaskManager.Task::(_debug) Task a480542f-aed7-4f9f-b8e4-03e53b967d9b: ref 0 aborting False
>Thread-15911920::INFO::2013-11-19 22:12:30,074::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000303030014038', 'lastCheck': 1384899144.1441309, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148105621338', 'lastCheck': 1384899141.8429649, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899147.165333, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000361204147339', 'lastCheck': 1384899141.767771, 'valid': True, 'code': 0}}
>Thread-42840::DEBUG::2013-11-19 22:12:31,779::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-19 22:12:32,458::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 22:12:37,178::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911926::INFO::2013-11-19 22:12:40,226::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911926::DEBUG::2013-11-19 22:12:40,227::task::495::TaskManager.Task::(_debug) Task 3cbfbf30-d530-42b5-8ec2-5d987e4b22e1: moving from state init -> state preparing
>Thread-15911926::DEBUG::2013-11-19 22:12:40,228::task::495::TaskManager.Task::(_debug) Task 3cbfbf30-d530-42b5-8ec2-5d987e4b22e1: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000408887863159', 'lastCheck': 1384899154.157372, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00137615203857', 'lastCheck': 1384899152.0079639, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899157.1797559, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000333070755005', 'lastCheck': 1384899152.4606471, 'valid': True, 'code': 0}}
>Thread-15911926::DEBUG::2013-11-19 22:12:40,234::task::495::TaskManager.Task::(_debug) Task 3cbfbf30-d530-42b5-8ec2-5d987e4b22e1: moving from state preparing -> state finished
>Thread-15911926::DEBUG::2013-11-19 22:12:40,235::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911926::DEBUG::2013-11-19 22:12:40,235::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911926::DEBUG::2013-11-19 22:12:40,236::task::495::TaskManager.Task::(_debug) Task 3cbfbf30-d530-42b5-8ec2-5d987e4b22e1: ref 0 aborting False
>Thread-15911926::INFO::2013-11-19 22:12:40,237::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000408887863159', 'lastCheck': 1384899154.157372, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00137615203857', 'lastCheck': 1384899152.0079639, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899157.1797559, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000333070755005', 'lastCheck': 1384899152.4606471, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:12:47,192::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911932::INFO::2013-11-19 22:12:50,419::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911932::DEBUG::2013-11-19 22:12:50,420::task::495::TaskManager.Task::(_debug) Task dc75ed79-322a-45a4-bad2-8cbf4f8b52f5: moving from state init -> state preparing
>Thread-15911932::DEBUG::2013-11-19 22:12:50,420::task::495::TaskManager.Task::(_debug) Task dc75ed79-322a-45a4-bad2-8cbf4f8b52f5: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000321865081787', 'lastCheck': 1384899164.1702571, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00154685974121', 'lastCheck': 1384899162.02426, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899167.193222, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000554084777832', 'lastCheck': 1384899162.472271, 'valid': True, 'code': 0}}
>Thread-15911932::DEBUG::2013-11-19 22:12:50,431::task::495::TaskManager.Task::(_debug) Task dc75ed79-322a-45a4-bad2-8cbf4f8b52f5: moving from state preparing -> state finished
>Thread-15911932::DEBUG::2013-11-19 22:12:50,432::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911932::DEBUG::2013-11-19 22:12:50,433::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911932::DEBUG::2013-11-19 22:12:50,433::task::495::TaskManager.Task::(_debug) Task dc75ed79-322a-45a4-bad2-8cbf4f8b52f5: ref 0 aborting False
>Thread-15911932::INFO::2013-11-19 22:12:50,434::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000321865081787', 'lastCheck': 1384899164.1702571, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00154685974121', 'lastCheck': 1384899162.02426, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899167.193222, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000554084777832', 'lastCheck': 1384899162.472271, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:12:57,205::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911938::INFO::2013-11-19 22:13:00,569::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911938::DEBUG::2013-11-19 22:13:00,570::task::495::TaskManager.Task::(_debug) Task 5aa2b9f4-980a-4213-80e3-6ad78d099c69: moving from state init -> state preparing
>Thread-15911938::DEBUG::2013-11-19 22:13:00,571::task::495::TaskManager.Task::(_debug) Task 5aa2b9f4-980a-4213-80e3-6ad78d099c69: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000407934188843', 'lastCheck': 1384899174.1829889, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00139498710632', 'lastCheck': 1384899172.0396459, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899177.206548, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000410079956055', 'lastCheck': 1384899172.4825909, 'valid': True, 'code': 0}}
>Thread-15911938::DEBUG::2013-11-19 22:13:00,583::task::495::TaskManager.Task::(_debug) Task 5aa2b9f4-980a-4213-80e3-6ad78d099c69: moving from state preparing -> state finished
>Thread-15911938::DEBUG::2013-11-19 22:13:00,584::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911938::DEBUG::2013-11-19 22:13:00,584::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911938::DEBUG::2013-11-19 22:13:00,585::task::495::TaskManager.Task::(_debug) Task 5aa2b9f4-980a-4213-80e3-6ad78d099c69: ref 0 aborting False
>Thread-15911938::INFO::2013-11-19 22:13:00,586::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000407934188843', 'lastCheck': 1384899174.1829889, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00139498710632', 'lastCheck': 1384899172.0396459, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899177.206548, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000410079956055', 'lastCheck': 1384899172.4825909, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:13:07,219::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911944::INFO::2013-11-19 22:13:10,857::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911944::DEBUG::2013-11-19 22:13:10,858::task::495::TaskManager.Task::(_debug) Task c72b8ad6-2771-4b99-bfcc-0178f4281772: moving from state init -> state preparing
>Thread-15911944::DEBUG::2013-11-19 22:13:10,858::task::495::TaskManager.Task::(_debug) Task c72b8ad6-2771-4b99-bfcc-0178f4281772: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000325918197632', 'lastCheck': 1384899184.1956429, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00226497650146', 'lastCheck': 1384899182.0561559, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899187.22001, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000416040420532', 'lastCheck': 1384899182.4938991, 'valid': True, 'code': 0}}
>Thread-15911944::DEBUG::2013-11-19 22:13:10,859::task::495::TaskManager.Task::(_debug) Task c72b8ad6-2771-4b99-bfcc-0178f4281772: moving from state preparing -> state finished
>Thread-15911944::DEBUG::2013-11-19 22:13:10,860::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911944::DEBUG::2013-11-19 22:13:10,860::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911944::DEBUG::2013-11-19 22:13:10,861::task::495::TaskManager.Task::(_debug) Task c72b8ad6-2771-4b99-bfcc-0178f4281772: ref 0 aborting False
>Thread-15911944::INFO::2013-11-19 22:13:10,862::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000325918197632', 'lastCheck': 1384899184.1956429, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00226497650146', 'lastCheck': 1384899182.0561559, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899187.22001, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000416040420532', 'lastCheck': 1384899182.4938991, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:13:17,233::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911950::INFO::2013-11-19 22:13:21,150::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911950::DEBUG::2013-11-19 22:13:21,151::task::495::TaskManager.Task::(_debug) Task a8a49307-2165-4976-886c-2a27a72ba179: moving from state init -> state preparing
>Thread-15911950::DEBUG::2013-11-19 22:13:21,152::task::495::TaskManager.Task::(_debug) Task a8a49307-2165-4976-886c-2a27a72ba179: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000347852706909', 'lastCheck': 1384899194.2083759, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00133109092712', 'lastCheck': 1384899192.071286, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899197.2336969, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000395059585571', 'lastCheck': 1384899192.50489, 'valid': True, 'code': 0}}
>Thread-15911950::DEBUG::2013-11-19 22:13:21,156::task::495::TaskManager.Task::(_debug) Task a8a49307-2165-4976-886c-2a27a72ba179: moving from state preparing -> state finished
>Thread-15911950::DEBUG::2013-11-19 22:13:21,157::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911950::DEBUG::2013-11-19 22:13:21,157::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911950::DEBUG::2013-11-19 22:13:21,158::task::495::TaskManager.Task::(_debug) Task a8a49307-2165-4976-886c-2a27a72ba179: ref 0 aborting False
>Thread-15911950::INFO::2013-11-19 22:13:21,162::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000347852706909', 'lastCheck': 1384899194.2083759, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00133109092712', 'lastCheck': 1384899192.071286, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899197.2336969, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000395059585571', 'lastCheck': 1384899192.50489, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:13:27,246::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911956::INFO::2013-11-19 22:13:31,378::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911956::DEBUG::2013-11-19 22:13:31,379::task::495::TaskManager.Task::(_debug) Task be82b700-5c86-4f15-80e7-8f57c541f7e3: moving from state init -> state preparing
>Thread-15911956::DEBUG::2013-11-19 22:13:31,381::task::495::TaskManager.Task::(_debug) Task be82b700-5c86-4f15-80e7-8f57c541f7e3: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000309944152832', 'lastCheck': 1384899204.2210031, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00146412849426', 'lastCheck': 1384899202.0867479, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899207.2472041, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00043797492981', 'lastCheck': 1384899202.5165651, 'valid': True, 'code': 0}}
>Thread-15911956::DEBUG::2013-11-19 22:13:31,389::task::495::TaskManager.Task::(_debug) Task be82b700-5c86-4f15-80e7-8f57c541f7e3: moving from state preparing -> state finished
>Thread-15911956::DEBUG::2013-11-19 22:13:31,391::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911956::DEBUG::2013-11-19 22:13:31,391::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911956::DEBUG::2013-11-19 22:13:31,392::task::495::TaskManager.Task::(_debug) Task be82b700-5c86-4f15-80e7-8f57c541f7e3: ref 0 aborting False
>Thread-15911956::INFO::2013-11-19 22:13:31,396::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000309944152832', 'lastCheck': 1384899204.2210031, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00146412849426', 'lastCheck': 1384899202.0867479, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899207.2472041, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00043797492981', 'lastCheck': 1384899202.5165651, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:13:37,260::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911962::INFO::2013-11-19 22:13:41,548::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911962::DEBUG::2013-11-19 22:13:41,549::task::495::TaskManager.Task::(_debug) Task f3690dd7-a504-4806-a9e5-d2fdec55e834: moving from state init -> state preparing
>Thread-15911962::DEBUG::2013-11-19 22:13:41,549::task::495::TaskManager.Task::(_debug) Task f3690dd7-a504-4806-a9e5-d2fdec55e834: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000406980514526', 'lastCheck': 1384899214.234071, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00143694877625', 'lastCheck': 1384899212.1022091, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899217.260875, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000401973724365', 'lastCheck': 1384899212.528116, 'valid': True, 'code': 0}}
>Thread-15911962::DEBUG::2013-11-19 22:13:41,556::task::495::TaskManager.Task::(_debug) Task f3690dd7-a504-4806-a9e5-d2fdec55e834: moving from state preparing -> state finished
>Thread-15911962::DEBUG::2013-11-19 22:13:41,557::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911962::DEBUG::2013-11-19 22:13:41,558::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911962::DEBUG::2013-11-19 22:13:41,558::task::495::TaskManager.Task::(_debug) Task f3690dd7-a504-4806-a9e5-d2fdec55e834: ref 0 aborting False
>Thread-15911962::INFO::2013-11-19 22:13:41,559::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000406980514526', 'lastCheck': 1384899214.234071, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00143694877625', 'lastCheck': 1384899212.1022091, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899217.260875, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000401973724365', 'lastCheck': 1384899212.528116, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:13:47,273::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911968::INFO::2013-11-19 22:13:51,706::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911968::DEBUG::2013-11-19 22:13:51,706::task::495::TaskManager.Task::(_debug) Task 6a10380e-280b-4739-8df1-20f16d7a989b: moving from state init -> state preparing
>Thread-15911968::DEBUG::2013-11-19 22:13:51,707::task::495::TaskManager.Task::(_debug) Task 6a10380e-280b-4739-8df1-20f16d7a989b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000365972518921', 'lastCheck': 1384899224.2470429, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00133895874023', 'lastCheck': 1384899222.117789, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899227.2744, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000395059585571', 'lastCheck': 1384899222.5395119, 'valid': True, 'code': 0}}
>Thread-15911968::DEBUG::2013-11-19 22:13:51,718::task::495::TaskManager.Task::(_debug) Task 6a10380e-280b-4739-8df1-20f16d7a989b: moving from state preparing -> state finished
>Thread-15911968::DEBUG::2013-11-19 22:13:51,718::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911968::DEBUG::2013-11-19 22:13:51,719::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911968::DEBUG::2013-11-19 22:13:51,719::task::495::TaskManager.Task::(_debug) Task 6a10380e-280b-4739-8df1-20f16d7a989b: ref 0 aborting False
>Thread-15911968::INFO::2013-11-19 22:13:51,722::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000365972518921', 'lastCheck': 1384899224.2470429, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00133895874023', 'lastCheck': 1384899222.117789, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899227.2744, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000395059585571', 'lastCheck': 1384899222.5395119, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:13:57,287::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911974::INFO::2013-11-19 22:14:01,876::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911974::DEBUG::2013-11-19 22:14:01,876::task::495::TaskManager.Task::(_debug) Task dd52bec4-13ff-492a-bcf7-a715b40c6ab7: moving from state init -> state preparing
>Thread-15911974::DEBUG::2013-11-19 22:14:01,877::task::495::TaskManager.Task::(_debug) Task dd52bec4-13ff-492a-bcf7-a715b40c6ab7: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000384092330933', 'lastCheck': 1384899234.2598851, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00144505500793', 'lastCheck': 1384899232.1332829, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899237.2878251, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000345945358276', 'lastCheck': 1384899232.5510621, 'valid': True, 'code': 0}}
>Thread-15911974::DEBUG::2013-11-19 22:14:01,885::task::495::TaskManager.Task::(_debug) Task dd52bec4-13ff-492a-bcf7-a715b40c6ab7: moving from state preparing -> state finished
>Thread-15911974::DEBUG::2013-11-19 22:14:01,886::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911974::DEBUG::2013-11-19 22:14:01,886::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911974::DEBUG::2013-11-19 22:14:01,887::task::495::TaskManager.Task::(_debug) Task dd52bec4-13ff-492a-bcf7-a715b40c6ab7: ref 0 aborting False
>Thread-15911974::INFO::2013-11-19 22:14:01,888::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000384092330933', 'lastCheck': 1384899234.2598851, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00144505500793', 'lastCheck': 1384899232.1332829, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899237.2878251, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000345945358276', 'lastCheck': 1384899232.5510621, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:14:07,300::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911980::INFO::2013-11-19 22:14:12,019::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911980::DEBUG::2013-11-19 22:14:12,020::task::495::TaskManager.Task::(_debug) Task 4f5adafd-9d1c-4b7f-83ca-38a80f30535b: moving from state init -> state preparing
>Thread-15911980::DEBUG::2013-11-19 22:14:12,020::task::495::TaskManager.Task::(_debug) Task 4f5adafd-9d1c-4b7f-83ca-38a80f30535b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000388860702515', 'lastCheck': 1384899244.2729459, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00193905830383', 'lastCheck': 1384899242.1497071, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899247.301384, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00049901008606', 'lastCheck': 1384899242.5630801, 'valid': True, 'code': 0}}
>Thread-15911980::DEBUG::2013-11-19 22:14:12,024::task::495::TaskManager.Task::(_debug) Task 4f5adafd-9d1c-4b7f-83ca-38a80f30535b: moving from state preparing -> state finished
>Thread-15911980::DEBUG::2013-11-19 22:14:12,025::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911980::DEBUG::2013-11-19 22:14:12,025::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911980::DEBUG::2013-11-19 22:14:12,026::task::495::TaskManager.Task::(_debug) Task 4f5adafd-9d1c-4b7f-83ca-38a80f30535b: ref 0 aborting False
>Thread-15911980::INFO::2013-11-19 22:14:12,032::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000388860702515', 'lastCheck': 1384899244.2729459, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00193905830383', 'lastCheck': 1384899242.1497071, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899247.301384, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00049901008606', 'lastCheck': 1384899242.5630801, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:14:17,314::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911986::INFO::2013-11-19 22:14:22,160::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911986::DEBUG::2013-11-19 22:14:22,161::task::495::TaskManager.Task::(_debug) Task 52429d11-bba9-4703-a63c-28e3ddd345f6: moving from state init -> state preparing
>Thread-15911986::DEBUG::2013-11-19 22:14:22,161::task::495::TaskManager.Task::(_debug) Task 52429d11-bba9-4703-a63c-28e3ddd345f6: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000406980514526', 'lastCheck': 1384899254.2859001, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00138401985168', 'lastCheck': 1384899252.163806, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899257.3148201, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000422954559326', 'lastCheck': 1384899252.5746009, 'valid': True, 'code': 0}}
>Thread-15911986::DEBUG::2013-11-19 22:14:22,162::task::495::TaskManager.Task::(_debug) Task 52429d11-bba9-4703-a63c-28e3ddd345f6: moving from state preparing -> state finished
>Thread-15911986::DEBUG::2013-11-19 22:14:22,163::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911986::DEBUG::2013-11-19 22:14:22,163::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911986::DEBUG::2013-11-19 22:14:22,164::task::495::TaskManager.Task::(_debug) Task 52429d11-bba9-4703-a63c-28e3ddd345f6: ref 0 aborting False
>Thread-15911986::INFO::2013-11-19 22:14:22,169::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000406980514526', 'lastCheck': 1384899254.2859001, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00138401985168', 'lastCheck': 1384899252.163806, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899257.3148201, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000422954559326', 'lastCheck': 1384899252.5746009, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:14:27,327::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911992::INFO::2013-11-19 22:14:32,348::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911992::DEBUG::2013-11-19 22:14:32,349::task::495::TaskManager.Task::(_debug) Task 41afc542-8218-4160-afef-dd4bf49be72a: moving from state init -> state preparing
>Thread-15911992::DEBUG::2013-11-19 22:14:32,349::task::495::TaskManager.Task::(_debug) Task 41afc542-8218-4160-afef-dd4bf49be72a: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00439286231995', 'lastCheck': 1384899264.3027229, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00135207176208', 'lastCheck': 1384899272.1942649, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899267.3283451, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000445127487183', 'lastCheck': 1384899262.586148, 'valid': True, 'code': 0}}
>Thread-15911992::DEBUG::2013-11-19 22:14:32,360::task::495::TaskManager.Task::(_debug) Task 41afc542-8218-4160-afef-dd4bf49be72a: moving from state preparing -> state finished
>Thread-15911992::DEBUG::2013-11-19 22:14:32,361::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911992::DEBUG::2013-11-19 22:14:32,361::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911992::DEBUG::2013-11-19 22:14:32,362::task::495::TaskManager.Task::(_debug) Task 41afc542-8218-4160-afef-dd4bf49be72a: ref 0 aborting False
>Thread-15911992::INFO::2013-11-19 22:14:32,363::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00439286231995', 'lastCheck': 1384899264.3027229, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00135207176208', 'lastCheck': 1384899272.1942649, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899267.3283451, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000445127487183', 'lastCheck': 1384899262.586148, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:14:37,341::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15911998::INFO::2013-11-19 22:14:42,514::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15911998::DEBUG::2013-11-19 22:14:42,514::task::495::TaskManager.Task::(_debug) Task f7c0777e-8564-4295-a768-2be142847d84: moving from state init -> state preparing
>Thread-15911998::DEBUG::2013-11-19 22:14:42,515::task::495::TaskManager.Task::(_debug) Task f7c0777e-8564-4295-a768-2be142847d84: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000399112701416', 'lastCheck': 1384899274.3156011, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00152206420898', 'lastCheck': 1384899282.2101851, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899277.3423209, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000415086746216', 'lastCheck': 1384899272.597693, 'valid': True, 'code': 0}}
>Thread-15911998::DEBUG::2013-11-19 22:14:42,517::task::495::TaskManager.Task::(_debug) Task f7c0777e-8564-4295-a768-2be142847d84: moving from state preparing -> state finished
>Thread-15911998::DEBUG::2013-11-19 22:14:42,517::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15911998::DEBUG::2013-11-19 22:14:42,518::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15911998::DEBUG::2013-11-19 22:14:42,518::task::495::TaskManager.Task::(_debug) Task f7c0777e-8564-4295-a768-2be142847d84: ref 0 aborting False
>Thread-15911998::INFO::2013-11-19 22:14:42,519::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000399112701416', 'lastCheck': 1384899274.3156011, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00152206420898', 'lastCheck': 1384899282.2101851, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899277.3423209, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000415086746216', 'lastCheck': 1384899272.597693, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:14:47,355::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912004::INFO::2013-11-19 22:14:52,711::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912004::DEBUG::2013-11-19 22:14:52,711::task::495::TaskManager.Task::(_debug) Task 6ca27f82-b103-465f-8337-313a006141ed: moving from state init -> state preparing
>Thread-15912004::DEBUG::2013-11-19 22:14:52,712::task::495::TaskManager.Task::(_debug) Task 6ca27f82-b103-465f-8337-313a006141ed: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000327110290527', 'lastCheck': 1384899284.328531, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00147795677185', 'lastCheck': 1384899292.2256589, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899287.3562629, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000423908233643', 'lastCheck': 1384899292.621212, 'valid': True, 'code': 0}}
>Thread-15912004::DEBUG::2013-11-19 22:14:52,714::task::495::TaskManager.Task::(_debug) Task 6ca27f82-b103-465f-8337-313a006141ed: moving from state preparing -> state finished
>Thread-15912004::DEBUG::2013-11-19 22:14:52,714::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912004::DEBUG::2013-11-19 22:14:52,715::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912004::DEBUG::2013-11-19 22:14:52,715::task::495::TaskManager.Task::(_debug) Task 6ca27f82-b103-465f-8337-313a006141ed: ref 0 aborting False
>Thread-15912004::INFO::2013-11-19 22:14:52,723::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000327110290527', 'lastCheck': 1384899284.328531, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00147795677185', 'lastCheck': 1384899292.2256589, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899287.3562629, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000423908233643', 'lastCheck': 1384899292.621212, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:14:57,368::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912010::INFO::2013-11-19 22:15:02,895::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912010::DEBUG::2013-11-19 22:15:02,896::task::495::TaskManager.Task::(_debug) Task 6eb13251-e051-4bb2-9c6f-5c836f80c99b: moving from state init -> state preparing
>Thread-15912010::DEBUG::2013-11-19 22:15:02,903::task::495::TaskManager.Task::(_debug) Task 6eb13251-e051-4bb2-9c6f-5c836f80c99b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000336170196533', 'lastCheck': 1384899294.3413451, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00226807594299', 'lastCheck': 1384899302.242744, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899297.3695071, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000341892242432', 'lastCheck': 1384899302.632704, 'valid': True, 'code': 0}}
>Thread-15912010::DEBUG::2013-11-19 22:15:02,904::task::495::TaskManager.Task::(_debug) Task 6eb13251-e051-4bb2-9c6f-5c836f80c99b: moving from state preparing -> state finished
>Thread-15912010::DEBUG::2013-11-19 22:15:02,905::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912010::DEBUG::2013-11-19 22:15:02,906::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912010::DEBUG::2013-11-19 22:15:02,906::task::495::TaskManager.Task::(_debug) Task 6eb13251-e051-4bb2-9c6f-5c836f80c99b: ref 0 aborting False
>Thread-15912010::INFO::2013-11-19 22:15:02,907::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000336170196533', 'lastCheck': 1384899294.3413451, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00226807594299', 'lastCheck': 1384899302.242744, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899297.3695071, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000341892242432', 'lastCheck': 1384899302.632704, 'valid': True, 'code': 0}}
>Thread-42838::DEBUG::2013-11-19 22:15:04,352::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 22:15:04,998::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 22:15:07,560::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912016::INFO::2013-11-19 22:15:13,071::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912016::DEBUG::2013-11-19 22:15:13,072::task::495::TaskManager.Task::(_debug) Task 5a1ed58f-8631-410b-84db-9d71ed23b7b0: moving from state init -> state preparing
>Thread-15912016::DEBUG::2013-11-19 22:15:13,073::task::495::TaskManager.Task::(_debug) Task 5a1ed58f-8631-410b-84db-9d71ed23b7b0: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000308990478516', 'lastCheck': 1384899305.002492, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00146317481995', 'lastCheck': 1384899312.258251, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899307.561516, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000353097915649', 'lastCheck': 1384899312.6443, 'valid': True, 'code': 0}}
>Thread-15912016::DEBUG::2013-11-19 22:15:13,074::task::495::TaskManager.Task::(_debug) Task 5a1ed58f-8631-410b-84db-9d71ed23b7b0: moving from state preparing -> state finished
>Thread-15912016::DEBUG::2013-11-19 22:15:13,074::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912016::DEBUG::2013-11-19 22:15:13,075::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912016::DEBUG::2013-11-19 22:15:13,076::task::495::TaskManager.Task::(_debug) Task 5a1ed58f-8631-410b-84db-9d71ed23b7b0: ref 0 aborting False
>Thread-15912016::INFO::2013-11-19 22:15:13,081::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000308990478516', 'lastCheck': 1384899305.002492, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00146317481995', 'lastCheck': 1384899312.258251, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899307.561516, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000353097915649', 'lastCheck': 1384899312.6443, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:15:17,573::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912022::INFO::2013-11-19 22:15:23,248::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912022::DEBUG::2013-11-19 22:15:23,249::task::495::TaskManager.Task::(_debug) Task d219e4f9-d6db-497b-bc6f-7d14f1319d44: moving from state init -> state preparing
>Thread-15912022::DEBUG::2013-11-19 22:15:23,249::task::495::TaskManager.Task::(_debug) Task d219e4f9-d6db-497b-bc6f-7d14f1319d44: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000308990478516', 'lastCheck': 1384899315.015337, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014591217041', 'lastCheck': 1384899322.273685, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899317.5743849, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000325918197632', 'lastCheck': 1384899322.6557441, 'valid': True, 'code': 0}}
>Thread-15912022::DEBUG::2013-11-19 22:15:23,261::task::495::TaskManager.Task::(_debug) Task d219e4f9-d6db-497b-bc6f-7d14f1319d44: moving from state preparing -> state finished
>Thread-15912022::DEBUG::2013-11-19 22:15:23,262::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912022::DEBUG::2013-11-19 22:15:23,262::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912022::DEBUG::2013-11-19 22:15:23,263::task::495::TaskManager.Task::(_debug) Task d219e4f9-d6db-497b-bc6f-7d14f1319d44: ref 0 aborting False
>Thread-15912022::INFO::2013-11-19 22:15:23,263::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000308990478516', 'lastCheck': 1384899315.015337, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014591217041', 'lastCheck': 1384899322.273685, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899317.5743849, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000325918197632', 'lastCheck': 1384899322.6557441, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:15:27,587::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912028::INFO::2013-11-19 22:15:33,404::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912028::DEBUG::2013-11-19 22:15:33,405::task::495::TaskManager.Task::(_debug) Task 82067f4e-fe11-4d7b-aa09-5283afd6580a: moving from state init -> state preparing
>Thread-15912028::DEBUG::2013-11-19 22:15:33,405::task::495::TaskManager.Task::(_debug) Task 82067f4e-fe11-4d7b-aa09-5283afd6580a: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000300168991089', 'lastCheck': 1384899325.027838, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00135898590088', 'lastCheck': 1384899332.2889011, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899327.5881629, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000427007675171', 'lastCheck': 1384899332.6672339, 'valid': True, 'code': 0}}
>Thread-15912028::DEBUG::2013-11-19 22:15:33,419::task::495::TaskManager.Task::(_debug) Task 82067f4e-fe11-4d7b-aa09-5283afd6580a: moving from state preparing -> state finished
>Thread-15912028::DEBUG::2013-11-19 22:15:33,420::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912028::DEBUG::2013-11-19 22:15:33,420::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912028::DEBUG::2013-11-19 22:15:33,421::task::495::TaskManager.Task::(_debug) Task 82067f4e-fe11-4d7b-aa09-5283afd6580a: ref 0 aborting False
>Thread-15912028::INFO::2013-11-19 22:15:33,421::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000300168991089', 'lastCheck': 1384899325.027838, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00135898590088', 'lastCheck': 1384899332.2889011, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899327.5881629, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000427007675171', 'lastCheck': 1384899332.6672339, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:15:37,601::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912034::INFO::2013-11-19 22:15:43,573::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912034::DEBUG::2013-11-19 22:15:43,574::task::495::TaskManager.Task::(_debug) Task 76010c56-a674-4529-82cb-c37dd1ac1c5c: moving from state init -> state preparing
>Thread-15912034::DEBUG::2013-11-19 22:15:43,575::task::495::TaskManager.Task::(_debug) Task 76010c56-a674-4529-82cb-c37dd1ac1c5c: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000317096710205', 'lastCheck': 1384899335.04055, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00141906738281', 'lastCheck': 1384899342.3046041, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899337.601974, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000339984893799', 'lastCheck': 1384899342.6787241, 'valid': True, 'code': 0}}
>Thread-15912034::DEBUG::2013-11-19 22:15:43,587::task::495::TaskManager.Task::(_debug) Task 76010c56-a674-4529-82cb-c37dd1ac1c5c: moving from state preparing -> state finished
>Thread-15912034::DEBUG::2013-11-19 22:15:43,593::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912034::DEBUG::2013-11-19 22:15:43,594::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912034::DEBUG::2013-11-19 22:15:43,594::task::495::TaskManager.Task::(_debug) Task 76010c56-a674-4529-82cb-c37dd1ac1c5c: ref 0 aborting False
>Thread-15912034::INFO::2013-11-19 22:15:43,595::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000317096710205', 'lastCheck': 1384899335.04055, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00141906738281', 'lastCheck': 1384899342.3046041, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899337.601974, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000339984893799', 'lastCheck': 1384899342.6787241, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:15:47,614::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912040::INFO::2013-11-19 22:15:53,883::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912040::DEBUG::2013-11-19 22:15:53,884::task::495::TaskManager.Task::(_debug) Task adcd5630-0a0d-4724-9d21-8035f0dccadc: moving from state init -> state preparing
>Thread-15912040::DEBUG::2013-11-19 22:15:53,884::task::495::TaskManager.Task::(_debug) Task adcd5630-0a0d-4724-9d21-8035f0dccadc: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000272035598755', 'lastCheck': 1384899345.0530601, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00141286849976', 'lastCheck': 1384899352.3201671, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899347.615418, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000418901443481', 'lastCheck': 1384899352.6902349, 'valid': True, 'code': 0}}
>Thread-15912040::DEBUG::2013-11-19 22:15:53,896::task::495::TaskManager.Task::(_debug) Task adcd5630-0a0d-4724-9d21-8035f0dccadc: moving from state preparing -> state finished
>Thread-15912040::DEBUG::2013-11-19 22:15:53,897::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912040::DEBUG::2013-11-19 22:15:53,897::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912040::DEBUG::2013-11-19 22:15:53,898::task::495::TaskManager.Task::(_debug) Task adcd5630-0a0d-4724-9d21-8035f0dccadc: ref 0 aborting False
>Thread-15912040::INFO::2013-11-19 22:15:53,898::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000272035598755', 'lastCheck': 1384899345.0530601, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00141286849976', 'lastCheck': 1384899352.3201671, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899347.615418, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000418901443481', 'lastCheck': 1384899352.6902349, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:15:57,628::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912046::INFO::2013-11-19 22:16:04,421::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912046::DEBUG::2013-11-19 22:16:04,422::task::495::TaskManager.Task::(_debug) Task 8dd6fe8d-010b-4f35-a5fb-8001b82949a8: moving from state init -> state preparing
>Thread-15912046::DEBUG::2013-11-19 22:16:04,423::task::495::TaskManager.Task::(_debug) Task 8dd6fe8d-010b-4f35-a5fb-8001b82949a8: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000263929367065', 'lastCheck': 1384899355.0655861, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00213098526001', 'lastCheck': 1384899362.3366809, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899357.628926, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000370025634766', 'lastCheck': 1384899362.70175, 'valid': True, 'code': 0}}
>Thread-15912046::DEBUG::2013-11-19 22:16:04,424::task::495::TaskManager.Task::(_debug) Task 8dd6fe8d-010b-4f35-a5fb-8001b82949a8: moving from state preparing -> state finished
>Thread-15912046::DEBUG::2013-11-19 22:16:04,424::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912046::DEBUG::2013-11-19 22:16:04,425::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912046::DEBUG::2013-11-19 22:16:04,425::task::495::TaskManager.Task::(_debug) Task 8dd6fe8d-010b-4f35-a5fb-8001b82949a8: ref 0 aborting False
>Thread-15912046::INFO::2013-11-19 22:16:04,426::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000263929367065', 'lastCheck': 1384899355.0655861, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00213098526001', 'lastCheck': 1384899362.3366809, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899357.628926, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000370025634766', 'lastCheck': 1384899362.70175, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:16:07,641::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912052::INFO::2013-11-19 22:16:15,280::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912052::DEBUG::2013-11-19 22:16:15,281::task::495::TaskManager.Task::(_debug) Task 77d006e6-eb45-461e-ab13-2847ba63a842: moving from state init -> state preparing
>Thread-15912052::DEBUG::2013-11-19 22:16:15,282::task::495::TaskManager.Task::(_debug) Task 77d006e6-eb45-461e-ab13-2847ba63a842: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000263214111328', 'lastCheck': 1384899375.090596, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00138807296753', 'lastCheck': 1384899372.3520391, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899367.642592, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000328063964844', 'lastCheck': 1384899372.7132051, 'valid': True, 'code': 0}}
>Thread-15912052::DEBUG::2013-11-19 22:16:15,288::task::495::TaskManager.Task::(_debug) Task 77d006e6-eb45-461e-ab13-2847ba63a842: moving from state preparing -> state finished
>Thread-15912052::DEBUG::2013-11-19 22:16:15,288::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912052::DEBUG::2013-11-19 22:16:15,289::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912052::DEBUG::2013-11-19 22:16:15,290::task::495::TaskManager.Task::(_debug) Task 77d006e6-eb45-461e-ab13-2847ba63a842: ref 0 aborting False
>Thread-15912052::INFO::2013-11-19 22:16:15,290::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000263214111328', 'lastCheck': 1384899375.090596, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00138807296753', 'lastCheck': 1384899372.3520391, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899367.642592, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000328063964844', 'lastCheck': 1384899372.7132051, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:16:17,655::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912058::INFO::2013-11-19 22:16:25,398::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912058::DEBUG::2013-11-19 22:16:25,399::task::495::TaskManager.Task::(_debug) Task df99c9b3-5eea-43bb-a7ad-7cc2bc65fb77: moving from state init -> state preparing
>Thread-15912058::DEBUG::2013-11-19 22:16:25,399::task::495::TaskManager.Task::(_debug) Task df99c9b3-5eea-43bb-a7ad-7cc2bc65fb77: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000309944152832', 'lastCheck': 1384899385.1031539, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00135612487793', 'lastCheck': 1384899382.367857, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899377.656112, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000344038009644', 'lastCheck': 1384899382.7247281, 'valid': True, 'code': 0}}
>Thread-15912058::DEBUG::2013-11-19 22:16:25,407::task::495::TaskManager.Task::(_debug) Task df99c9b3-5eea-43bb-a7ad-7cc2bc65fb77: moving from state preparing -> state finished
>Thread-15912058::DEBUG::2013-11-19 22:16:25,408::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912058::DEBUG::2013-11-19 22:16:25,408::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912058::DEBUG::2013-11-19 22:16:25,409::task::495::TaskManager.Task::(_debug) Task df99c9b3-5eea-43bb-a7ad-7cc2bc65fb77: ref 0 aborting False
>Thread-15912058::INFO::2013-11-19 22:16:25,410::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000309944152832', 'lastCheck': 1384899385.1031539, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00135612487793', 'lastCheck': 1384899382.367857, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899377.656112, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000344038009644', 'lastCheck': 1384899382.7247281, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:16:27,669::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912064::INFO::2013-11-19 22:16:35,583::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912064::DEBUG::2013-11-19 22:16:35,584::task::495::TaskManager.Task::(_debug) Task a80a3137-9e6d-40ac-b71d-06f5e8c7ba77: moving from state init -> state preparing
>Thread-15912064::DEBUG::2013-11-19 22:16:35,585::task::495::TaskManager.Task::(_debug) Task a80a3137-9e6d-40ac-b71d-06f5e8c7ba77: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000319004058838', 'lastCheck': 1384899395.11589, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00136184692383', 'lastCheck': 1384899392.383126, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899387.6697431, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000412940979004', 'lastCheck': 1384899392.7363341, 'valid': True, 'code': 0}}
>Thread-15912064::DEBUG::2013-11-19 22:16:35,592::task::495::TaskManager.Task::(_debug) Task a80a3137-9e6d-40ac-b71d-06f5e8c7ba77: moving from state preparing -> state finished
>Thread-15912064::DEBUG::2013-11-19 22:16:35,592::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912064::DEBUG::2013-11-19 22:16:35,593::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912064::DEBUG::2013-11-19 22:16:35,593::task::495::TaskManager.Task::(_debug) Task a80a3137-9e6d-40ac-b71d-06f5e8c7ba77: ref 0 aborting False
>Thread-15912064::INFO::2013-11-19 22:16:35,596::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000319004058838', 'lastCheck': 1384899395.11589, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00136184692383', 'lastCheck': 1384899392.383126, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899387.6697431, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000412940979004', 'lastCheck': 1384899392.7363341, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:16:37,683::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912070::INFO::2013-11-19 22:16:45,741::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912070::DEBUG::2013-11-19 22:16:45,742::task::495::TaskManager.Task::(_debug) Task d616bc14-e563-40e6-b281-edc31c2d3ca6: moving from state init -> state preparing
>Thread-15912070::DEBUG::2013-11-19 22:16:45,742::task::495::TaskManager.Task::(_debug) Task d616bc14-e563-40e6-b281-edc31c2d3ca6: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000339031219482', 'lastCheck': 1384899405.1285341, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014009475708', 'lastCheck': 1384899402.3989811, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899397.683785, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000348806381226', 'lastCheck': 1384899402.7478061, 'valid': True, 'code': 0}}
>Thread-15912070::DEBUG::2013-11-19 22:16:45,753::task::495::TaskManager.Task::(_debug) Task d616bc14-e563-40e6-b281-edc31c2d3ca6: moving from state preparing -> state finished
>Thread-15912070::DEBUG::2013-11-19 22:16:45,754::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912070::DEBUG::2013-11-19 22:16:45,754::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912070::DEBUG::2013-11-19 22:16:45,755::task::495::TaskManager.Task::(_debug) Task d616bc14-e563-40e6-b281-edc31c2d3ca6: ref 0 aborting False
>Thread-15912070::INFO::2013-11-19 22:16:45,756::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000339031219482', 'lastCheck': 1384899405.1285341, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014009475708', 'lastCheck': 1384899402.3989811, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899397.683785, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000348806381226', 'lastCheck': 1384899402.7478061, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:16:47,696::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912076::INFO::2013-11-19 22:16:57,188::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912076::DEBUG::2013-11-19 22:16:57,189::task::495::TaskManager.Task::(_debug) Task c90c6426-4327-408e-87a8-fd503b15ec55: moving from state init -> state preparing
>Thread-15912076::DEBUG::2013-11-19 22:16:57,190::task::495::TaskManager.Task::(_debug) Task c90c6426-4327-408e-87a8-fd503b15ec55: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000268220901489', 'lastCheck': 1384899415.141047, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00141191482544', 'lastCheck': 1384899412.414417, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899407.6973529, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00032901763916', 'lastCheck': 1384899412.7615421, 'valid': True, 'code': 0}}
>Thread-15912076::DEBUG::2013-11-19 22:16:57,197::task::495::TaskManager.Task::(_debug) Task c90c6426-4327-408e-87a8-fd503b15ec55: moving from state preparing -> state finished
>Thread-15912076::DEBUG::2013-11-19 22:16:57,198::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912076::DEBUG::2013-11-19 22:16:57,198::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912076::DEBUG::2013-11-19 22:16:57,199::task::495::TaskManager.Task::(_debug) Task c90c6426-4327-408e-87a8-fd503b15ec55: ref 0 aborting False
>Thread-15912076::INFO::2013-11-19 22:16:57,200::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000268220901489', 'lastCheck': 1384899415.141047, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00141191482544', 'lastCheck': 1384899412.414417, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899407.6973529, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00032901763916', 'lastCheck': 1384899412.7615421, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:16:57,710::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912082::INFO::2013-11-19 22:17:07,335::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912082::DEBUG::2013-11-19 22:17:07,335::task::495::TaskManager.Task::(_debug) Task 2f6bf88f-f4d0-472f-a44f-ee821980a76d: moving from state init -> state preparing
>Thread-15912082::DEBUG::2013-11-19 22:17:07,336::task::495::TaskManager.Task::(_debug) Task 2f6bf88f-f4d0-472f-a44f-ee821980a76d: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000919818878174', 'lastCheck': 1384899425.1547389, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0023238658905', 'lastCheck': 1384899422.431107, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899417.711009, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000437021255493', 'lastCheck': 1384899422.773242, 'valid': True, 'code': 0}}
>Thread-15912082::DEBUG::2013-11-19 22:17:07,356::task::495::TaskManager.Task::(_debug) Task 2f6bf88f-f4d0-472f-a44f-ee821980a76d: moving from state preparing -> state finished
>Thread-15912082::DEBUG::2013-11-19 22:17:07,356::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912082::DEBUG::2013-11-19 22:17:07,357::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912082::DEBUG::2013-11-19 22:17:07,358::task::495::TaskManager.Task::(_debug) Task 2f6bf88f-f4d0-472f-a44f-ee821980a76d: ref 0 aborting False
>Thread-15912082::INFO::2013-11-19 22:17:07,362::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000919818878174', 'lastCheck': 1384899425.1547389, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0023238658905', 'lastCheck': 1384899422.431107, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899417.711009, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000437021255493', 'lastCheck': 1384899422.773242, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:17:07,724::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912088::INFO::2013-11-19 22:17:17,492::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912088::DEBUG::2013-11-19 22:17:17,492::task::495::TaskManager.Task::(_debug) Task 52e4651a-9f84-4dd0-b538-51cee427a12d: moving from state init -> state preparing
>Thread-15912088::DEBUG::2013-11-19 22:17:17,493::task::495::TaskManager.Task::(_debug) Task 52e4651a-9f84-4dd0-b538-51cee427a12d: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000259876251221', 'lastCheck': 1384899435.167232, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00140595436096', 'lastCheck': 1384899432.446408, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899427.7247379, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000328063964844', 'lastCheck': 1384899432.7848229, 'valid': True, 'code': 0}}
>Thread-15912088::DEBUG::2013-11-19 22:17:17,502::task::495::TaskManager.Task::(_debug) Task 52e4651a-9f84-4dd0-b538-51cee427a12d: moving from state preparing -> state finished
>Thread-15912088::DEBUG::2013-11-19 22:17:17,503::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912088::DEBUG::2013-11-19 22:17:17,504::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912088::DEBUG::2013-11-19 22:17:17,504::task::495::TaskManager.Task::(_debug) Task 52e4651a-9f84-4dd0-b538-51cee427a12d: ref 0 aborting False
>Thread-15912088::INFO::2013-11-19 22:17:17,505::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000259876251221', 'lastCheck': 1384899435.167232, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00140595436096', 'lastCheck': 1384899432.446408, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899427.7247379, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000328063964844', 'lastCheck': 1384899432.7848229, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:17:17,737::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912094::INFO::2013-11-19 22:17:27,694::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912094::DEBUG::2013-11-19 22:17:27,695::task::495::TaskManager.Task::(_debug) Task 0e84c412-00ed-4ff8-bdef-6113c8e555cb: moving from state init -> state preparing
>Thread-15912094::DEBUG::2013-11-19 22:17:27,696::task::495::TaskManager.Task::(_debug) Task 0e84c412-00ed-4ff8-bdef-6113c8e555cb: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000349044799805', 'lastCheck': 1384899445.1791861, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00147795677185', 'lastCheck': 1384899442.4622481, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899437.7386169, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000340938568115', 'lastCheck': 1384899442.79636, 'valid': True, 'code': 0}}
>Thread-15912094::DEBUG::2013-11-19 22:17:27,704::task::495::TaskManager.Task::(_debug) Task 0e84c412-00ed-4ff8-bdef-6113c8e555cb: moving from state preparing -> state finished
>Thread-15912094::DEBUG::2013-11-19 22:17:27,704::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912094::DEBUG::2013-11-19 22:17:27,705::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912094::DEBUG::2013-11-19 22:17:27,705::task::495::TaskManager.Task::(_debug) Task 0e84c412-00ed-4ff8-bdef-6113c8e555cb: ref 0 aborting False
>Thread-15912094::INFO::2013-11-19 22:17:27,709::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000349044799805', 'lastCheck': 1384899445.1791861, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00147795677185', 'lastCheck': 1384899442.4622481, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899437.7386169, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000340938568115', 'lastCheck': 1384899442.79636, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:17:27,751::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-19 22:17:32,807::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-19 22:17:33,429::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 22:17:37,766::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912100::INFO::2013-11-19 22:17:37,900::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912100::DEBUG::2013-11-19 22:17:37,901::task::495::TaskManager.Task::(_debug) Task 476e9905-e3a3-4147-ad7a-16f533f697e0: moving from state init -> state preparing
>Thread-15912100::DEBUG::2013-11-19 22:17:37,902::task::495::TaskManager.Task::(_debug) Task 476e9905-e3a3-4147-ad7a-16f533f697e0: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000566005706787', 'lastCheck': 1384899455.1925609, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00143504142761', 'lastCheck': 1384899452.477886, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899457.7668591, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000300884246826', 'lastCheck': 1384899453.4319601, 'valid': True, 'code': 0}}
>Thread-15912100::DEBUG::2013-11-19 22:17:37,903::task::495::TaskManager.Task::(_debug) Task 476e9905-e3a3-4147-ad7a-16f533f697e0: moving from state preparing -> state finished
>Thread-15912100::DEBUG::2013-11-19 22:17:37,903::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912100::DEBUG::2013-11-19 22:17:37,904::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912100::DEBUG::2013-11-19 22:17:37,904::task::495::TaskManager.Task::(_debug) Task 476e9905-e3a3-4147-ad7a-16f533f697e0: ref 0 aborting False
>Thread-15912100::INFO::2013-11-19 22:17:37,905::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000566005706787', 'lastCheck': 1384899455.1925609, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00143504142761', 'lastCheck': 1384899452.477886, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899457.7668591, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000300884246826', 'lastCheck': 1384899453.4319601, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:17:47,779::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912106::INFO::2013-11-19 22:17:48,044::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912106::DEBUG::2013-11-19 22:17:48,044::task::495::TaskManager.Task::(_debug) Task edc323eb-d6d6-4569-9aa4-6fbb7721876a: moving from state init -> state preparing
>Thread-15912106::DEBUG::2013-11-19 22:17:48,045::task::495::TaskManager.Task::(_debug) Task edc323eb-d6d6-4569-9aa4-6fbb7721876a: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000302076339722', 'lastCheck': 1384899465.2050431, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00219297409058', 'lastCheck': 1384899462.49615, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899467.7805221, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000438928604126', 'lastCheck': 1384899463.4437339, 'valid': True, 'code': 0}}
>Thread-15912106::DEBUG::2013-11-19 22:17:48,049::task::495::TaskManager.Task::(_debug) Task edc323eb-d6d6-4569-9aa4-6fbb7721876a: moving from state preparing -> state finished
>Thread-15912106::DEBUG::2013-11-19 22:17:48,050::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912106::DEBUG::2013-11-19 22:17:48,050::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912106::DEBUG::2013-11-19 22:17:48,051::task::495::TaskManager.Task::(_debug) Task edc323eb-d6d6-4569-9aa4-6fbb7721876a: ref 0 aborting False
>Thread-15912106::INFO::2013-11-19 22:17:48,056::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000302076339722', 'lastCheck': 1384899465.2050431, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00219297409058', 'lastCheck': 1384899462.49615, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899467.7805221, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000438928604126', 'lastCheck': 1384899463.4437339, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:17:57,793::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912112::INFO::2013-11-19 22:17:58,198::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912112::DEBUG::2013-11-19 22:17:58,198::task::495::TaskManager.Task::(_debug) Task 3a8cf4d9-9e77-47ec-b28f-7164ae3fd1b8: moving from state init -> state preparing
>Thread-15912112::DEBUG::2013-11-19 22:17:58,199::task::495::TaskManager.Task::(_debug) Task 3a8cf4d9-9e77-47ec-b28f-7164ae3fd1b8: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00026798248291', 'lastCheck': 1384899475.21753, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0012218952179', 'lastCheck': 1384899472.51017, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899477.794246, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000416994094849', 'lastCheck': 1384899473.4554241, 'valid': True, 'code': 0}}
>Thread-15912112::DEBUG::2013-11-19 22:17:58,213::task::495::TaskManager.Task::(_debug) Task 3a8cf4d9-9e77-47ec-b28f-7164ae3fd1b8: moving from state preparing -> state finished
>Thread-15912112::DEBUG::2013-11-19 22:17:58,213::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912112::DEBUG::2013-11-19 22:17:58,214::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912112::DEBUG::2013-11-19 22:17:58,214::task::495::TaskManager.Task::(_debug) Task 3a8cf4d9-9e77-47ec-b28f-7164ae3fd1b8: ref 0 aborting False
>Thread-15912112::INFO::2013-11-19 22:17:58,219::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00026798248291', 'lastCheck': 1384899475.21753, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0012218952179', 'lastCheck': 1384899472.51017, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899477.794246, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000416994094849', 'lastCheck': 1384899473.4554241, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:18:07,807::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912118::INFO::2013-11-19 22:18:08,356::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912118::DEBUG::2013-11-19 22:18:08,357::task::495::TaskManager.Task::(_debug) Task f5d12dbc-a265-4c34-b5c0-d465ddb30744: moving from state init -> state preparing
>Thread-15912118::DEBUG::2013-11-19 22:18:08,357::task::495::TaskManager.Task::(_debug) Task f5d12dbc-a265-4c34-b5c0-d465ddb30744: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000292062759399', 'lastCheck': 1384899485.2325189, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00216388702393', 'lastCheck': 1384899482.526546, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899487.808006, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000427961349487', 'lastCheck': 1384899483.467, 'valid': True, 'code': 0}}
>Thread-15912118::DEBUG::2013-11-19 22:18:08,358::task::495::TaskManager.Task::(_debug) Task f5d12dbc-a265-4c34-b5c0-d465ddb30744: moving from state preparing -> state finished
>Thread-15912118::DEBUG::2013-11-19 22:18:08,359::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912118::DEBUG::2013-11-19 22:18:08,359::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912118::DEBUG::2013-11-19 22:18:08,360::task::495::TaskManager.Task::(_debug) Task f5d12dbc-a265-4c34-b5c0-d465ddb30744: ref 0 aborting False
>Thread-15912118::INFO::2013-11-19 22:18:08,364::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000292062759399', 'lastCheck': 1384899485.2325189, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00216388702393', 'lastCheck': 1384899482.526546, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899487.808006, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000427961349487', 'lastCheck': 1384899483.467, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:18:17,820::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912124::INFO::2013-11-19 22:18:18,522::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912124::DEBUG::2013-11-19 22:18:18,523::task::495::TaskManager.Task::(_debug) Task d1288538-90cb-46ff-ab5f-f66387c99922: moving from state init -> state preparing
>Thread-15912124::DEBUG::2013-11-19 22:18:18,524::task::495::TaskManager.Task::(_debug) Task d1288538-90cb-46ff-ab5f-f66387c99922: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000270128250122', 'lastCheck': 1384899495.2450039, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00139284133911', 'lastCheck': 1384899492.541883, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899497.821532, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000416994094849', 'lastCheck': 1384899493.478533, 'valid': True, 'code': 0}}
>Thread-15912124::DEBUG::2013-11-19 22:18:18,535::task::495::TaskManager.Task::(_debug) Task d1288538-90cb-46ff-ab5f-f66387c99922: moving from state preparing -> state finished
>Thread-15912124::DEBUG::2013-11-19 22:18:18,536::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912124::DEBUG::2013-11-19 22:18:18,536::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912124::DEBUG::2013-11-19 22:18:18,537::task::495::TaskManager.Task::(_debug) Task d1288538-90cb-46ff-ab5f-f66387c99922: ref 0 aborting False
>Thread-15912124::INFO::2013-11-19 22:18:18,537::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000270128250122', 'lastCheck': 1384899495.2450039, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00139284133911', 'lastCheck': 1384899492.541883, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899497.821532, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000416994094849', 'lastCheck': 1384899493.478533, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:18:27,834::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912130::INFO::2013-11-19 22:18:29,433::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912130::DEBUG::2013-11-19 22:18:29,434::task::495::TaskManager.Task::(_debug) Task 5b55d6bb-1199-4997-8925-251cc901e01d: moving from state init -> state preparing
>Thread-15912130::DEBUG::2013-11-19 22:18:29,435::task::495::TaskManager.Task::(_debug) Task 5b55d6bb-1199-4997-8925-251cc901e01d: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000297069549561', 'lastCheck': 1384899505.2576079, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00141501426697', 'lastCheck': 1384899502.5570641, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899507.835078, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000444889068604', 'lastCheck': 1384899503.4905341, 'valid': True, 'code': 0}}
>Thread-15912130::DEBUG::2013-11-19 22:18:29,435::task::495::TaskManager.Task::(_debug) Task 5b55d6bb-1199-4997-8925-251cc901e01d: moving from state preparing -> state finished
>Thread-15912130::DEBUG::2013-11-19 22:18:29,436::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912130::DEBUG::2013-11-19 22:18:29,437::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912130::DEBUG::2013-11-19 22:18:29,437::task::495::TaskManager.Task::(_debug) Task 5b55d6bb-1199-4997-8925-251cc901e01d: ref 0 aborting False
>Thread-15912130::INFO::2013-11-19 22:18:29,438::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000297069549561', 'lastCheck': 1384899505.2576079, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00141501426697', 'lastCheck': 1384899502.5570641, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899507.835078, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000444889068604', 'lastCheck': 1384899503.4905341, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:18:37,848::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912136::INFO::2013-11-19 22:18:39,559::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912136::DEBUG::2013-11-19 22:18:39,560::task::495::TaskManager.Task::(_debug) Task 1fd652de-e184-4468-af59-8354af2964a9: moving from state init -> state preparing
>Thread-15912136::DEBUG::2013-11-19 22:18:39,560::task::495::TaskManager.Task::(_debug) Task 1fd652de-e184-4468-af59-8354af2964a9: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000331878662109', 'lastCheck': 1384899515.2703471, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00134015083313', 'lastCheck': 1384899512.5721719, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899517.84884, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000470876693726', 'lastCheck': 1384899513.5021441, 'valid': True, 'code': 0}}
>Thread-15912136::DEBUG::2013-11-19 22:18:39,561::task::495::TaskManager.Task::(_debug) Task 1fd652de-e184-4468-af59-8354af2964a9: moving from state preparing -> state finished
>Thread-15912136::DEBUG::2013-11-19 22:18:39,562::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912136::DEBUG::2013-11-19 22:18:39,563::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912136::DEBUG::2013-11-19 22:18:39,563::task::495::TaskManager.Task::(_debug) Task 1fd652de-e184-4468-af59-8354af2964a9: ref 0 aborting False
>Thread-15912136::INFO::2013-11-19 22:18:39,569::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000331878662109', 'lastCheck': 1384899515.2703471, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00134015083313', 'lastCheck': 1384899512.5721719, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899517.84884, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000470876693726', 'lastCheck': 1384899513.5021441, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:18:47,861::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912142::INFO::2013-11-19 22:18:50,340::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912142::DEBUG::2013-11-19 22:18:50,340::task::495::TaskManager.Task::(_debug) Task 1c3a0894-e91f-4ec2-9b02-0e6da2c02a8a: moving from state init -> state preparing
>Thread-15912142::DEBUG::2013-11-19 22:18:50,341::task::495::TaskManager.Task::(_debug) Task 1c3a0894-e91f-4ec2-9b02-0e6da2c02a8a: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000277042388916', 'lastCheck': 1384899525.2828851, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00138711929321', 'lastCheck': 1384899522.5883319, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899527.862237, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000442028045654', 'lastCheck': 1384899523.5135751, 'valid': True, 'code': 0}}
>Thread-15912142::DEBUG::2013-11-19 22:18:50,346::task::495::TaskManager.Task::(_debug) Task 1c3a0894-e91f-4ec2-9b02-0e6da2c02a8a: moving from state preparing -> state finished
>Thread-15912142::DEBUG::2013-11-19 22:18:50,347::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912142::DEBUG::2013-11-19 22:18:50,347::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912142::DEBUG::2013-11-19 22:18:50,348::task::495::TaskManager.Task::(_debug) Task 1c3a0894-e91f-4ec2-9b02-0e6da2c02a8a: ref 0 aborting False
>Thread-15912142::INFO::2013-11-19 22:18:50,349::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000277042388916', 'lastCheck': 1384899525.2828851, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00138711929321', 'lastCheck': 1384899522.5883319, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899527.862237, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000442028045654', 'lastCheck': 1384899523.5135751, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:18:57,874::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912148::INFO::2013-11-19 22:19:00,492::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912148::DEBUG::2013-11-19 22:19:00,493::task::495::TaskManager.Task::(_debug) Task a466f55e-c069-457a-8fa5-9db3b1e618f4: moving from state init -> state preparing
>Thread-15912148::DEBUG::2013-11-19 22:19:00,503::task::495::TaskManager.Task::(_debug) Task a466f55e-c069-457a-8fa5-9db3b1e618f4: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000269174575806', 'lastCheck': 1384899535.295486, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00187802314758', 'lastCheck': 1384899532.6040151, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899537.875514, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000416994094849', 'lastCheck': 1384899533.524935, 'valid': True, 'code': 0}}
>Thread-15912148::DEBUG::2013-11-19 22:19:00,505::task::495::TaskManager.Task::(_debug) Task a466f55e-c069-457a-8fa5-9db3b1e618f4: moving from state preparing -> state finished
>Thread-15912148::DEBUG::2013-11-19 22:19:00,507::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912148::DEBUG::2013-11-19 22:19:00,509::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912148::DEBUG::2013-11-19 22:19:00,510::task::495::TaskManager.Task::(_debug) Task a466f55e-c069-457a-8fa5-9db3b1e618f4: ref 0 aborting False
>Thread-15912148::INFO::2013-11-19 22:19:00,518::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000269174575806', 'lastCheck': 1384899535.295486, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00187802314758', 'lastCheck': 1384899532.6040151, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899537.875514, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000416994094849', 'lastCheck': 1384899533.524935, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:19:07,888::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912154::INFO::2013-11-19 22:19:10,678::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912154::DEBUG::2013-11-19 22:19:10,679::task::495::TaskManager.Task::(_debug) Task 0687b899-9b9e-4f4c-ae90-cdcb51b152f8: moving from state init -> state preparing
>Thread-15912154::DEBUG::2013-11-19 22:19:10,680::task::495::TaskManager.Task::(_debug) Task 0687b899-9b9e-4f4c-ae90-cdcb51b152f8: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.0002601146698', 'lastCheck': 1384899545.3079841, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00216698646545', 'lastCheck': 1384899542.6189549, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899547.8887539, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000435829162598', 'lastCheck': 1384899543.5361271, 'valid': True, 'code': 0}}
>Thread-15912154::DEBUG::2013-11-19 22:19:10,687::task::495::TaskManager.Task::(_debug) Task 0687b899-9b9e-4f4c-ae90-cdcb51b152f8: moving from state preparing -> state finished
>Thread-15912154::DEBUG::2013-11-19 22:19:10,688::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912154::DEBUG::2013-11-19 22:19:10,688::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912154::DEBUG::2013-11-19 22:19:10,689::task::495::TaskManager.Task::(_debug) Task 0687b899-9b9e-4f4c-ae90-cdcb51b152f8: ref 0 aborting False
>Thread-15912154::INFO::2013-11-19 22:19:10,692::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.0002601146698', 'lastCheck': 1384899545.3079841, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00216698646545', 'lastCheck': 1384899542.6189549, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899547.8887539, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000435829162598', 'lastCheck': 1384899543.5361271, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:19:17,901::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912160::INFO::2013-11-19 22:19:20,857::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912160::DEBUG::2013-11-19 22:19:20,858::task::495::TaskManager.Task::(_debug) Task 931fb449-27e3-4428-beb4-292adcd1b745: moving from state init -> state preparing
>Thread-15912160::DEBUG::2013-11-19 22:19:20,858::task::495::TaskManager.Task::(_debug) Task 931fb449-27e3-4428-beb4-292adcd1b745: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000334024429321', 'lastCheck': 1384899555.320451, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00131106376648', 'lastCheck': 1384899552.634002, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899557.90201, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000376224517822', 'lastCheck': 1384899553.547652, 'valid': True, 'code': 0}}
>Thread-15912160::DEBUG::2013-11-19 22:19:20,868::task::495::TaskManager.Task::(_debug) Task 931fb449-27e3-4428-beb4-292adcd1b745: moving from state preparing -> state finished
>Thread-15912160::DEBUG::2013-11-19 22:19:20,869::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912160::DEBUG::2013-11-19 22:19:20,869::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912160::DEBUG::2013-11-19 22:19:20,870::task::495::TaskManager.Task::(_debug) Task 931fb449-27e3-4428-beb4-292adcd1b745: ref 0 aborting False
>Thread-15912160::INFO::2013-11-19 22:19:20,875::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000334024429321', 'lastCheck': 1384899555.320451, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00131106376648', 'lastCheck': 1384899552.634002, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899557.90201, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000376224517822', 'lastCheck': 1384899553.547652, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:19:27,914::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912166::INFO::2013-11-19 22:19:31,087::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912166::DEBUG::2013-11-19 22:19:31,088::task::495::TaskManager.Task::(_debug) Task e120a92d-ea55-4304-9ee9-469396fa9037: moving from state init -> state preparing
>Thread-15912166::DEBUG::2013-11-19 22:19:31,099::task::495::TaskManager.Task::(_debug) Task e120a92d-ea55-4304-9ee9-469396fa9037: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000263929367065', 'lastCheck': 1384899565.332953, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00140619277954', 'lastCheck': 1384899562.649209, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899567.9150159, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000338077545166', 'lastCheck': 1384899563.55913, 'valid': True, 'code': 0}}
>Thread-15912166::DEBUG::2013-11-19 22:19:31,109::task::495::TaskManager.Task::(_debug) Task e120a92d-ea55-4304-9ee9-469396fa9037: moving from state preparing -> state finished
>Thread-15912166::DEBUG::2013-11-19 22:19:31,109::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912166::DEBUG::2013-11-19 22:19:31,110::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912166::DEBUG::2013-11-19 22:19:31,110::task::495::TaskManager.Task::(_debug) Task e120a92d-ea55-4304-9ee9-469396fa9037: ref 0 aborting False
>Thread-15912166::INFO::2013-11-19 22:19:31,111::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000263929367065', 'lastCheck': 1384899565.332953, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00140619277954', 'lastCheck': 1384899562.649209, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899567.9150159, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000338077545166', 'lastCheck': 1384899563.55913, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:19:37,928::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912172::INFO::2013-11-19 22:19:41,245::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912172::DEBUG::2013-11-19 22:19:41,246::task::495::TaskManager.Task::(_debug) Task 11ea9bf7-8821-4ffe-bf78-4cc2c615fffc: moving from state init -> state preparing
>Thread-15912172::DEBUG::2013-11-19 22:19:41,246::task::495::TaskManager.Task::(_debug) Task 11ea9bf7-8821-4ffe-bf78-4cc2c615fffc: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000371932983398', 'lastCheck': 1384899575.3456371, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00134205818176', 'lastCheck': 1384899572.664288, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899577.9289351, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000322103500366', 'lastCheck': 1384899573.57059, 'valid': True, 'code': 0}}
>Thread-15912172::DEBUG::2013-11-19 22:19:41,247::task::495::TaskManager.Task::(_debug) Task 11ea9bf7-8821-4ffe-bf78-4cc2c615fffc: moving from state preparing -> state finished
>Thread-15912172::DEBUG::2013-11-19 22:19:41,248::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912172::DEBUG::2013-11-19 22:19:41,248::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912172::DEBUG::2013-11-19 22:19:41,249::task::495::TaskManager.Task::(_debug) Task 11ea9bf7-8821-4ffe-bf78-4cc2c615fffc: ref 0 aborting False
>Thread-15912172::INFO::2013-11-19 22:19:41,258::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000371932983398', 'lastCheck': 1384899575.3456371, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00134205818176', 'lastCheck': 1384899572.664288, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899577.9289351, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000322103500366', 'lastCheck': 1384899573.57059, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:19:47,941::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912178::INFO::2013-11-19 22:19:51,382::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912178::DEBUG::2013-11-19 22:19:51,383::task::495::TaskManager.Task::(_debug) Task 7220fbeb-9a3c-468e-8a09-7da0f9d0599f: moving from state init -> state preparing
>Thread-15912178::DEBUG::2013-11-19 22:19:51,383::task::495::TaskManager.Task::(_debug) Task 7220fbeb-9a3c-468e-8a09-7da0f9d0599f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000255823135376', 'lastCheck': 1384899585.358366, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00151896476746', 'lastCheck': 1384899582.6801791, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899587.942589, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000344038009644', 'lastCheck': 1384899583.5820351, 'valid': True, 'code': 0}}
>Thread-15912178::DEBUG::2013-11-19 22:19:51,392::task::495::TaskManager.Task::(_debug) Task 7220fbeb-9a3c-468e-8a09-7da0f9d0599f: moving from state preparing -> state finished
>Thread-15912178::DEBUG::2013-11-19 22:19:51,392::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912178::DEBUG::2013-11-19 22:19:51,393::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912178::DEBUG::2013-11-19 22:19:51,393::task::495::TaskManager.Task::(_debug) Task 7220fbeb-9a3c-468e-8a09-7da0f9d0599f: ref 0 aborting False
>Thread-15912178::INFO::2013-11-19 22:19:51,394::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000255823135376', 'lastCheck': 1384899585.358366, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00151896476746', 'lastCheck': 1384899582.6801791, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384899587.942589, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000344038009644', 'lastCheck': 1384899583.5820351, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-19 22:19:57,955::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-19 22:20:05,382::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 22:20:06,037::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 22:20:07,969::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:20:17,983::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:20:27,997::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:20:38,011::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:20:48,025::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:20:58,038::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:21:08,052::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:21:18,066::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:21:28,080::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:21:38,094::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:21:48,107::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:21:58,121::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:22:08,134::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:22:18,148::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:22:28,161::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-19 22:22:33,778::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-19 22:22:34,450::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 22:22:38,179::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:22:48,193::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:22:58,206::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:23:08,220::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:23:18,233::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:23:28,247::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:23:38,260::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:23:48,274::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:23:58,287::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:24:08,301::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:24:18,314::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:24:28,328::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:24:38,341::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:24:48,355::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:24:58,369::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-19 22:25:06,422::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 22:25:07,142::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 22:25:08,383::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:25:18,397::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:25:28,410::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:25:38,424::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:25:48,438::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:25:58,452::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:26:08,465::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:26:18,479::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:26:28,493::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:26:38,507::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:26:48,521::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:26:58,534::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:27:08,548::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:27:18,562::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:27:28,575::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-19 22:27:34,802::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-19 22:27:35,463::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 22:27:38,590::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:27:48,604::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:27:58,618::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:28:08,632::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:28:18,645::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:28:28,658::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:28:38,672::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:28:48,685::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:28:58,699::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:29:08,712::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:29:18,726::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:29:28,739::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:29:38,753::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:29:48,766::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:29:58,779::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-19 22:30:07,524::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 22:30:08,226::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 22:30:08,792::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:30:18,806::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:30:28,819::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:30:38,832::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:30:48,846::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:30:58,859::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:31:08,872::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:31:18,885::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:31:28,898::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:31:38,911::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:31:48,924::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:31:58,937::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:32:08,950::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:32:18,963::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:32:28,976::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-19 22:32:35,914::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-19 22:32:36,569::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 22:32:39,128::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:32:49,143::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:32:59,157::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:33:09,169::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:33:19,182::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:33:29,195::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:33:39,209::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:33:49,223::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:33:59,239::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:34:09,252::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:34:19,265::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:34:29,279::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:34:39,292::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:34:49,306::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:34:59,320::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-19 22:35:08,612::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 22:35:09,273::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 22:35:09,334::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:35:19,348::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:35:29,361::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:35:39,375::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:35:49,389::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:35:59,403::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:36:09,416::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:36:19,430::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:36:29,444::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:36:39,457::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:36:49,471::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:36:59,485::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:37:09,499::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:37:19,512::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:37:29,526::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-19 22:37:37,002::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-19 22:37:37,682::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 22:37:39,540::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:37:49,554::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:37:59,568::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:38:09,582::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:38:19,596::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:38:29,610::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:38:39,624::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:38:49,638::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:38:59,652::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:39:09,666::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:39:19,680::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:39:29,694::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:39:39,708::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:39:49,721::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:39:59,735::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-19 22:40:09,664::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-1024767::ERROR::2013-11-19 22:40:09,920::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-19 22:40:10,301::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 22:40:19,934::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:40:29,948::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:40:39,962::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:40:49,975::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:40:59,989::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:41:10,003::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:41:20,016::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:41:30,030::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:41:40,044::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:41:50,058::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:42:00,072::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:42:10,086::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:42:20,100::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:42:30,114::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-19 22:42:38,054::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-19 22:42:38,708::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 22:42:40,274::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:42:50,289::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:43:00,303::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:43:10,316::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:43:20,330::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:43:30,344::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:43:40,358::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:43:50,371::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:44:00,385::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:44:10,398::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:44:20,412::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:44:30,426::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:44:40,439::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:44:50,453::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:45:00,467::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:45:10,481::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-19 22:45:10,689::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 22:45:11,333::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 22:45:20,495::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:45:30,508::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:45:40,522::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:45:50,536::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:46:00,549::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:46:10,563::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:46:20,576::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:46:30,590::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:46:40,603::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:46:50,617::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:47:00,630::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:47:10,644::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:47:20,657::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:47:30,670::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-19 22:47:39,072::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-19 22:47:39,713::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 22:47:40,684::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:47:50,697::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:48:00,710::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:48:10,724::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:48:20,737::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:48:30,751::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:48:40,764::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:48:50,778::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:49:00,791::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:49:10,805::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:49:20,818::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:49:30,832::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:49:40,846::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:49:50,859::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:50:00,873::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:50:10,886::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-19 22:50:11,729::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 22:50:12,331::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 22:50:20,957::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:50:30,970::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:50:40,984::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:50:50,997::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:51:01,011::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:51:11,024::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:51:21,038::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:51:31,051::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:51:41,064::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:51:51,078::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:52:01,092::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:52:11,105::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:52:21,119::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:52:31,132::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-19 22:52:40,130::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-19 22:52:40,795::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 22:52:41,359::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:52:51,373::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:53:01,386::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:53:11,399::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:53:21,411::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:53:31,424::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:53:41,439::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:53:51,452::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:54:01,466::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:54:11,479::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:54:21,493::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:54:31,507::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:54:41,520::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:54:51,534::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:55:01,548::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:55:11,561::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-19 22:55:12,716::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 22:55:13,387::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 22:55:21,575::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:55:31,589::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:55:41,603::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:55:51,617::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:56:01,631::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:56:11,645::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:56:21,658::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:56:31,672::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:56:41,686::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:56:51,700::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:57:01,714::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:57:11,727::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:57:21,741::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:57:31,755::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-19 22:57:41,139::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-1024767::ERROR::2013-11-19 22:57:41,770::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-19 22:57:41,861::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 22:57:51,785::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:58:01,799::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:58:11,813::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:58:21,826::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:58:31,841::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:58:41,855::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:58:51,869::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:59:01,883::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:59:11,897::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:59:21,911::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:59:31,925::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:59:41,939::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 22:59:51,953::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:00:01,967::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:00:11,980::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-19 23:00:13,772::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 23:00:14,454::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 23:00:22,002::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:00:32,020::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:00:42,034::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:00:52,047::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:01:02,060::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:01:12,074::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:01:22,087::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:01:32,100::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:01:42,114::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:01:52,127::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:02:02,141::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:02:12,155::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:02:22,168::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:02:32,181::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:02:42,194::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-19 23:02:42,207::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-19 23:02:42,839::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 23:02:52,208::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:03:02,222::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:03:12,235::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:03:22,248::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:03:32,262::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:03:42,275::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:03:52,289::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:04:02,302::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:04:12,316::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:04:22,329::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:04:32,343::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:04:42,356::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:04:52,369::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:05:02,383::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:05:12,397::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-19 23:05:14,842::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 23:05:15,453::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 23:05:22,410::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:05:32,425::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:05:42,439::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:05:52,452::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:06:02,466::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:06:12,480::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:06:22,493::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:06:32,507::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:06:42,520::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:06:52,534::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:07:02,547::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:07:12,561::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:07:22,574::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:07:32,588::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:07:42,601::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-19 23:07:43,234::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-19 23:07:43,956::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 23:07:52,615::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:08:02,629::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:08:12,643::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:08:22,656::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:08:32,670::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:08:42,683::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:08:52,697::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:09:02,710::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:09:12,724::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:09:22,738::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:09:32,751::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:09:42,765::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:09:52,779::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:10:02,792::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:10:12,805::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-19 23:10:15,826::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 23:10:16,472::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 23:10:22,819::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:10:32,833::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:10:42,847::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:10:52,860::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:11:02,873::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:11:12,887::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:11:22,900::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:11:32,913::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:11:42,927::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:11:52,940::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:12:02,954::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:12:12,967::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:12:22,981::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:12:32,995::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:12:43,009::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-19 23:12:44,302::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-19 23:12:45,025::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 23:12:53,023::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:13:03,037::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:13:13,054::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:13:23,068::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:13:33,083::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:13:43,096::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:13:53,110::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:14:03,124::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:14:13,137::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:14:23,150::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:14:33,163::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:14:43,176::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:14:53,189::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:15:03,203::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:15:13,216::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-19 23:15:16,862::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 23:15:17,569::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 23:15:23,229::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:15:33,243::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:15:43,257::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:15:53,270::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:16:03,283::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:16:13,297::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:16:23,310::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:16:33,323::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:16:43,336::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:16:53,350::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:17:03,363::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:17:13,376::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:17:23,389::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:17:33,403::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:17:43,417::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-19 23:17:45,372::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-19 23:17:46,047::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 23:17:53,606::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:18:03,619::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:18:13,633::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:18:23,646::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:18:33,660::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:18:43,673::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:18:53,687::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:19:03,700::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:19:13,714::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:19:23,727::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:19:33,741::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:19:43,755::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:19:53,768::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:20:03,782::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:20:13,796::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-19 23:20:17,953::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 23:20:18,562::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 23:20:23,810::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:20:33,824::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:20:43,838::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:20:53,852::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:21:03,865::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:21:13,879::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:21:23,892::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:21:33,906::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:21:43,918::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:21:53,931::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:22:03,945::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:22:13,958::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:22:23,971::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:22:33,984::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:22:43,998::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-19 23:22:46,390::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-19 23:22:47,048::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 23:22:54,012::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:23:04,025::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:23:14,039::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:23:24,052::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:23:34,066::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:23:44,079::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:23:54,093::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:24:04,108::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:24:14,119::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:24:24,133::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:24:34,145::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:24:44,159::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:24:54,172::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:25:04,185::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:25:14,199::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-19 23:25:18,949::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 23:25:19,592::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 23:25:24,212::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:25:34,226::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:25:44,239::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:25:54,253::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:26:04,266::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:26:14,280::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:26:24,293::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:26:34,307::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:26:44,320::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:26:54,333::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:27:04,346::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:27:14,359::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:27:24,372::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:27:34,386::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:27:44,398::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-19 23:27:47,390::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-19 23:27:48,050::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 23:27:54,618::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:28:04,632::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:28:14,650::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:28:24,666::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:28:34,680::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:28:44,693::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:28:54,707::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:29:04,720::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:29:14,734::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:29:24,747::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:29:34,761::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:29:44,775::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:29:54,788::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:30:04,802::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:30:14,816::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-19 23:30:19,986::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 23:30:20,671::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 23:30:24,830::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:30:34,844::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:30:44,858::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:30:54,872::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:31:04,886::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:31:14,899::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:31:24,913::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:31:34,927::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:31:44,940::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:31:54,954::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:32:04,968::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:32:14,982::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:32:24,996::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:32:35,010::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:32:45,024::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-19 23:32:48,399::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-19 23:32:49,051::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 23:32:55,038::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:33:05,052::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:33:15,066::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:33:25,080::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:33:35,094::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:33:45,108::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:33:55,123::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:34:05,136::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:34:15,150::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:34:25,163::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:34:35,176::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:34:45,189::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:34:55,202::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:35:05,215::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:35:15,228::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-19 23:35:21,054::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 23:35:21,695::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 23:35:25,259::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:35:35,273::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:35:45,286::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:35:55,299::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:36:05,312::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:36:15,325::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:36:25,338::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:36:35,352::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:36:45,365::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:36:55,378::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:37:05,391::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:37:15,404::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:37:25,417::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:37:35,431::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:37:45,445::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-19 23:37:49,427::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-19 23:37:50,137::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 23:37:55,634::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:38:05,648::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:38:15,661::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:38:25,674::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:38:35,687::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:38:45,700::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:38:55,713::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:39:05,726::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:39:15,740::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:39:25,753::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:39:35,766::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:39:45,780::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:39:55,793::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:40:05,806::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:40:15,819::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-19 23:40:22,081::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 23:40:22,732::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 23:40:25,833::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:40:35,847::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:40:45,860::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:40:55,873::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:41:05,886::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:41:15,899::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:41:25,912::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:41:35,926::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:41:45,939::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:41:55,953::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:42:05,966::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:42:15,979::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:42:25,993::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:42:36,006::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:42:46,019::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-19 23:42:50,494::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-19 23:42:51,203::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 23:42:56,033::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:43:06,047::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:43:16,060::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:43:26,073::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:43:36,087::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:43:46,100::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:43:56,114::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:44:06,127::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:44:16,141::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:44:26,154::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:44:36,167::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:44:46,180::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:44:56,194::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:45:06,207::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:45:16,220::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-19 23:45:23,116::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 23:45:23,811::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 23:45:26,340::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:45:36,354::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:45:46,369::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:45:56,384::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:46:06,397::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:46:16,410::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:46:26,424::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:46:36,437::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:46:46,451::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:46:56,464::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:47:06,477::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:47:16,491::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:47:26,504::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:47:36,518::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:47:46,531::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-19 23:47:51,548::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-19 23:47:52,184::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 23:47:56,545::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:48:06,559::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:48:16,573::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:48:26,586::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:48:36,600::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:48:46,614::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:48:56,627::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:49:06,641::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:49:16,654::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:49:26,668::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:49:36,681::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:49:46,695::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:49:56,709::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:50:06,723::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:50:16,736::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-19 23:50:24,197::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 23:50:24,844::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 23:50:26,751::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:50:36,765::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:50:46,778::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:50:56,793::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:51:06,806::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:51:16,820::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:51:26,833::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:51:36,846::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:51:46,860::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:51:56,874::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:52:06,887::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:52:16,901::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:52:26,914::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:52:36,928::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:52:46,942::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-19 23:52:52,591::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-19 23:52:53,330::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 23:52:56,956::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:53:06,970::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:53:16,984::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:53:26,997::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:53:37,011::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:53:47,024::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:53:57,038::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:54:07,052::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:54:17,066::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:54:27,080::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:54:37,093::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:54:47,107::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:54:57,121::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:55:07,134::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:55:17,147::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-19 23:55:25,222::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-19 23:55:25,954::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 23:55:27,161::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:55:37,175::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:55:47,189::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:55:57,202::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:56:07,216::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:56:17,229::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:56:27,242::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:56:37,256::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:56:47,270::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:56:57,283::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:57:07,296::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:57:17,309::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:57:27,322::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:57:37,336::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:57:47,349::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-19 23:57:53,675::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-19 23:57:54,415::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-19 23:57:57,363::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:58:07,377::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:58:17,390::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:58:27,404::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:58:37,416::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:58:47,431::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:58:57,445::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:59:07,459::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:59:17,472::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:59:27,485::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:59:37,498::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:59:47,511::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-19 23:59:57,524::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:00:07,538::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:00:17,551::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-20 00:00:26,340::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 00:00:27,075::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 00:00:27,579::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:00:37,591::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:00:47,605::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:00:57,619::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:01:07,632::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:01:17,646::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:01:27,659::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:01:37,673::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:01:47,686::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:01:57,700::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:02:07,713::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:02:17,726::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:02:27,740::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:02:37,753::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:02:47,766::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-20 00:02:54,761::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 00:02:55,412::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 00:02:57,979::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:03:07,993::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:03:18,007::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:03:28,021::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:03:38,035::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:03:48,048::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:03:58,062::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:04:08,076::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:04:18,089::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:04:28,103::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:04:38,117::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:04:48,130::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:04:58,144::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:05:08,158::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:05:18,171::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-20 00:05:27,459::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-1024767::ERROR::2013-11-20 00:05:28,187::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-20 00:05:28,193::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 00:05:38,201::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:05:48,215::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:05:58,228::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:06:08,242::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:06:18,256::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:06:28,269::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:06:38,283::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:06:48,298::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:06:58,313::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:07:08,327::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:07:18,341::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:07:28,355::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:07:38,369::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:07:48,382::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-20 00:07:55,848::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 00:07:56,488::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 00:07:58,396::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:08:08,411::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:08:18,424::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:08:28,438::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:08:38,452::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:08:48,465::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:08:58,478::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:09:08,490::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:09:18,502::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:09:28,515::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:09:38,527::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:09:48,540::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:09:58,553::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:10:08,566::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:10:18,579::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-20 00:10:28,583::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-1024767::ERROR::2013-11-20 00:10:28,805::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-20 00:10:29,236::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 00:10:38,819::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:10:48,832::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:10:58,846::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:11:08,858::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:11:18,872::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:11:28,885::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:11:38,898::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:11:48,911::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:11:58,924::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:12:08,938::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:12:18,951::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:12:28,964::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:12:38,977::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:12:48,991::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-20 00:12:56,971::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 00:12:57,695::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 00:12:59,198::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:13:09,213::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:13:19,226::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:13:29,240::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:13:39,253::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:13:49,267::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:13:59,280::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:14:09,294::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:14:19,307::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:14:29,321::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:14:39,334::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:14:49,348::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:14:59,361::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:15:09,375::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:15:19,388::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:15:29,402::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-20 00:15:29,617::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 00:15:30,285::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 00:15:39,415::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:15:49,429::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:15:59,442::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:16:09,456::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:16:19,469::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:16:29,482::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:16:39,496::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:16:49,509::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:16:59,522::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:17:09,536::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:17:19,549::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:17:29,563::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:17:39,576::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:17:49,590::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-20 00:17:58,040::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 00:17:58,773::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 00:17:59,604::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:18:09,618::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:18:19,631::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:18:29,644::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:18:39,658::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:18:49,671::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:18:59,685::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:19:09,698::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:19:19,712::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:19:29,725::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:19:39,738::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:19:49,752::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:19:59,765::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:20:09,778::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:20:19,792::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:20:29,805::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-20 00:20:30,671::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 00:20:31,307::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 00:20:39,878::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:20:49,891::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:20:59,904::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:21:09,918::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:21:19,931::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:21:29,945::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:21:39,958::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:21:49,971::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:21:59,985::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:22:09,998::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:22:20,012::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:22:30,025::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:22:40,039::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:22:50,053::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-20 00:22:59,113::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 00:22:59,822::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 00:23:00,067::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:23:10,081::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:23:20,095::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:23:30,108::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:23:40,122::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:23:50,136::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:24:00,149::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:24:10,162::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:24:20,175::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:24:30,189::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:24:40,202::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:24:50,216::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:25:00,229::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:25:10,242::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:25:20,256::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:25:30,272::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-20 00:25:31,688::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 00:25:32,406::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 00:25:40,286::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:25:50,300::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:26:00,313::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:26:10,327::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:26:20,340::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:26:30,354::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:26:40,367::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:26:50,380::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:27:00,394::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:27:10,407::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:27:20,420::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:27:30,433::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:27:40,447::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:27:50,459::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-20 00:28:00,177::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-1024767::ERROR::2013-11-20 00:28:00,475::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-20 00:28:00,899::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 00:28:10,489::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:28:20,503::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:28:30,516::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:28:40,530::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:28:50,543::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:29:00,557::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:29:10,570::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:29:20,584::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:29:30,597::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:29:40,610::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:29:50,624::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:30:00,637::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:30:10,650::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:30:20,663::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:30:30,677::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-20 00:30:32,785::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 00:30:33,511::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 00:30:40,690::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:30:50,704::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:31:00,717::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:31:10,731::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:31:20,744::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:31:30,757::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:31:40,771::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:31:50,783::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:32:00,797::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:32:10,810::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:32:20,823::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:32:30,837::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:32:40,850::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:32:50,863::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:33:00,877::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-20 00:33:01,245::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 00:33:01,957::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 00:33:10,890::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:33:20,904::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:33:30,917::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:33:40,931::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:33:50,945::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:34:00,958::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:34:10,972::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:34:20,985::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:34:30,999::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:34:41,012::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:34:51,026::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:35:01,039::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:35:11,053::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:35:21,066::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:35:31,080::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-20 00:35:33,890::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 00:35:34,545::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 00:35:41,107::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:35:51,121::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:36:01,135::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:36:11,147::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:36:21,161::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:36:31,173::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:36:41,186::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:36:51,200::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:37:01,213::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:37:11,226::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:37:21,240::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:37:31,253::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:37:41,266::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:37:51,280::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:38:01,293::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-20 00:38:02,302::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 00:38:02,959::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 00:38:11,307::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:38:21,321::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:38:31,334::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:38:41,348::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:38:51,361::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:39:01,375::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:39:11,388::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:39:21,402::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:39:31,415::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:39:41,429::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:39:51,443::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:40:01,457::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:40:11,470::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:40:21,483::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:40:31,496::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-20 00:40:34,930::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 00:40:35,666::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 00:40:41,510::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:40:51,523::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:41:01,537::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:41:11,550::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:41:21,563::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:41:31,577::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:41:41,590::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:41:51,603::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:42:01,617::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:42:11,630::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:42:21,644::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:42:31,656::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:42:41,668::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:42:51,681::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:43:01,694::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-20 00:43:03,315::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 00:43:04,052::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 00:43:11,708::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:43:21,721::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:43:31,735::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:43:41,748::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:43:51,761::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:44:01,775::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:44:11,788::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:44:21,802::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:44:31,815::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:44:41,829::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:44:51,843::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:45:01,856::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:45:11,869::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:45:21,883::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:45:31,896::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-20 00:45:36,050::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 00:45:36,779::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 00:45:41,910::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:45:51,923::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:46:01,937::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:46:11,950::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:46:21,964::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:46:31,977::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:46:41,990::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:46:52,004::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:47:02,017::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:47:12,030::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:47:22,044::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:47:32,057::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:47:42,070::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:47:52,083::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:48:02,097::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-20 00:48:04,468::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 00:48:05,158::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 00:48:12,110::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:48:22,124::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:48:32,137::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:48:42,150::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:48:52,164::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:49:02,177::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:49:12,190::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:49:22,203::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:49:32,216::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:49:42,229::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:49:52,242::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:50:02,256::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:50:12,270::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:50:22,283::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:50:32,297::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-20 00:50:37,165::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 00:50:37,793::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 00:50:42,370::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:50:52,384::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:51:02,397::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:51:12,411::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:51:22,424::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:51:32,438::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:51:42,451::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:51:52,465::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:52:02,479::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:52:12,492::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:52:22,505::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:52:32,519::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:52:42,532::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:52:52,546::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:53:02,560::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-20 00:53:05,536::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 00:53:06,211::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 00:53:12,745::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:53:22,758::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:53:32,771::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:53:42,785::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:53:52,799::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:54:02,813::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:54:12,826::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:54:22,840::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:54:32,854::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:54:42,867::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:54:52,881::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:55:02,894::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:55:12,908::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:55:22,922::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:55:32,935::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-20 00:55:38,183::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 00:55:38,826::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 00:55:42,949::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:55:52,962::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:56:02,976::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:56:12,989::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:56:23,002::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:56:33,016::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:56:43,029::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:56:53,042::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:57:03,056::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:57:13,069::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:57:23,082::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:57:33,096::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:57:43,109::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:57:53,122::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:58:03,136::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-20 00:58:06,570::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 00:58:07,256::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 00:58:13,150::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:58:23,163::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:58:33,175::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:58:43,188::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:58:53,201::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:59:03,214::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:59:13,227::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:59:23,240::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:59:33,253::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:59:43,266::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 00:59:53,280::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:00:03,293::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:00:13,306::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:00:23,320::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:00:33,333::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-20 01:00:39,215::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 01:00:39,935::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 01:00:43,440::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:00:53,453::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:01:03,470::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:01:13,486::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:01:23,498::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:01:33,511::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:01:43,524::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:01:53,537::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:02:03,551::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:02:13,564::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:02:23,577::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:02:33,590::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:02:43,604::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:02:53,618::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:03:03,631::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-20 01:03:07,632::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 01:03:08,369::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 01:03:13,858::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:03:23,872::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:03:33,885::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:03:43,898::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:03:53,912::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:04:03,925::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:04:13,938::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:04:23,952::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:04:33,965::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:04:43,979::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:04:53,993::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:05:04,006::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:05:14,019::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:05:24,033::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:05:34,046::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-20 01:05:40,318::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 01:05:40,965::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 01:05:44,060::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:05:54,073::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:06:04,087::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:06:14,100::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:06:24,114::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:06:34,127::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:06:44,141::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:06:54,155::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:07:04,168::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:07:14,181::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:07:24,195::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:07:34,208::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:07:44,221::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:07:54,235::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:08:04,248::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-20 01:08:08,720::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 01:08:09,377::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 01:08:14,262::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:08:24,275::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:08:34,289::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:08:44,302::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:08:54,316::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:09:04,329::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:09:14,343::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:09:24,356::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:09:34,369::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:09:44,383::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:09:54,396::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:10:04,410::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:10:14,424::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:10:24,437::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:10:34,451::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-20 01:10:41,348::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 01:10:41,979::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 01:10:44,565::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:10:54,579::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:11:04,593::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:11:14,606::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:11:24,620::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:11:34,633::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:11:44,647::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:11:54,661::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:12:04,674::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:12:14,688::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:12:24,702::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:12:34,716::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:12:44,729::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:12:54,743::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:13:04,756::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-20 01:13:09,729::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 01:13:10,435::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 01:13:14,955::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:13:24,972::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:13:34,987::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:13:45,001::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:13:55,014::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:14:05,028::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:14:15,041::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:14:25,055::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:14:35,068::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:14:45,081::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:14:55,095::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:15:05,109::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:15:15,122::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:15:25,135::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:15:35,148::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-20 01:15:42,363::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 01:15:43,077::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 01:15:45,162::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:15:55,176::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:16:05,189::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:16:15,202::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:16:25,215::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:16:35,228::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:16:45,242::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:16:55,255::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:17:05,282::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:17:15,295::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:17:25,309::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:17:35,322::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:17:45,335::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:17:55,349::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:18:05,362::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-20 01:18:10,783::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 01:18:11,509::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 01:18:15,376::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:18:25,390::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:18:35,403::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:18:45,416::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:18:55,430::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:19:05,444::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:19:15,457::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:19:25,470::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:19:35,484::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:19:45,497::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:19:55,510::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:20:05,524::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:20:15,537::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:20:25,550::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:20:35,564::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-20 01:20:43,460::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 01:20:44,155::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 01:20:45,682::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:20:55,696::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:21:05,709::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:21:15,722::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:21:25,736::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:21:35,750::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:21:45,763::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:21:55,776::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:22:05,790::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:22:15,803::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:22:25,816::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:22:35,830::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:22:45,844::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:22:55,857::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:23:05,871::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-20 01:23:11,853::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 01:23:12,484::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 01:23:16,070::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:23:26,084::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:23:36,097::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:23:46,111::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:23:56,124::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:24:06,138::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:24:16,152::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:24:26,165::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:24:36,178::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:24:46,191::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:24:56,204::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:25:06,218::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:25:16,231::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:25:26,244::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:25:36,258::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-20 01:25:44,537::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 01:25:45,262::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 01:25:46,271::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:25:56,284::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:26:06,297::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:26:16,310::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:26:26,323::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:26:36,336::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:26:46,349::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:26:56,362::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:27:06,375::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:27:16,388::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:27:26,401::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:27:36,414::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:27:46,427::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:27:56,440::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:28:06,453::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-20 01:28:12,930::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 01:28:13,660::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 01:28:16,467::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:28:26,480::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:28:36,493::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:28:46,507::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:28:56,520::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:29:06,533::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:29:16,546::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:29:26,559::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:29:36,573::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:29:46,586::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:29:56,599::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:30:06,612::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:30:16,625::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:30:26,639::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:30:36,652::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-20 01:30:45,644::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 01:30:46,368::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 01:30:46,855::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:30:56,867::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:31:06,883::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:31:16,897::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:31:26,910::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:31:36,923::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:31:46,937::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:31:56,950::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:32:06,964::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:32:16,977::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:32:26,991::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:32:37,004::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:32:47,018::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:32:57,031::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:33:07,044::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-20 01:33:14,030::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 01:33:14,754::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 01:33:17,256::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:33:27,270::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:33:37,283::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:33:47,295::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:33:57,308::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:34:07,322::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:34:17,335::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:34:27,349::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:34:37,362::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:34:47,376::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:34:57,389::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:35:07,403::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:35:17,416::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:35:27,430::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:35:37,443::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-20 01:35:46,753::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 01:35:47,398::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 01:35:47,458::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:35:57,471::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:36:07,485::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:36:17,499::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:36:27,512::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:36:37,526::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:36:47,540::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:36:57,553::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:37:07,567::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:37:17,580::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:37:27,594::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:37:37,608::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:37:47,621::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:37:57,635::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:38:07,649::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-20 01:38:15,127::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 01:38:15,767::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 01:38:17,663::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:38:27,677::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:38:37,691::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:38:47,705::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:38:57,718::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:39:07,732::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:39:17,745::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:39:27,759::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:39:37,773::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:39:47,787::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:39:57,800::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:40:07,814::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:40:17,828::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:40:27,842::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:40:37,855::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-20 01:40:47,783::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-1024767::ERROR::2013-11-20 01:40:48,009::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-20 01:40:48,478::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 01:40:58,023::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:41:08,037::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:41:18,050::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:41:28,064::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:41:38,078::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:41:48,092::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:41:58,105::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:42:08,119::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:42:18,133::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:42:28,146::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:42:38,160::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:42:48,174::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:42:58,187::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:43:08,201::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-20 01:43:16,177::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 01:43:16,920::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 01:43:18,405::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:43:28,419::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:43:38,434::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:43:48,447::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:43:58,461::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:44:08,474::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:44:18,488::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:44:28,502::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:44:38,515::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:44:48,529::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:44:58,543::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:45:08,556::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:45:18,570::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:45:28,583::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:45:38,597::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:45:48,610::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-20 01:45:48,863::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 01:45:49,611::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 01:45:58,624::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:46:08,638::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:46:18,652::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:46:28,665::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:46:38,678::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:46:48,692::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:46:58,705::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:47:08,718::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:47:18,731::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:47:28,745::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:47:38,758::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:47:48,771::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:47:58,785::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:48:08,798::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-20 01:48:17,271::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 01:48:17,924::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 01:48:18,812::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:48:28,826::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:48:38,839::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:48:48,853::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:48:58,867::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:49:08,880::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:49:18,894::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:49:28,908::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:49:38,921::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:49:48,935::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:49:58,949::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:50:08,962::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:50:18,976::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:50:28,990::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:50:39,004::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:50:49,017::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-20 01:50:49,994::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 01:50:50,599::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 01:50:59,202::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:51:09,215::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:51:19,229::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:51:29,242::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:51:39,255::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:51:49,268::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:51:59,282::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:52:09,295::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:52:19,308::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:52:29,321::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:52:39,334::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:52:49,348::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:52:59,361::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:53:09,374::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-20 01:53:18,365::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 01:53:19,061::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 01:53:19,572::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:53:29,586::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:53:39,599::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:53:49,613::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:53:59,627::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:54:09,641::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:54:19,654::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:54:29,667::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:54:39,681::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:54:49,695::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:54:59,708::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:55:09,722::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:55:19,735::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:55:29,749::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:55:39,763::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:55:49,776::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-20 01:55:50,984::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 01:55:51,703::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 01:55:59,790::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:56:09,803::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:56:19,816::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:56:29,829::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:56:39,842::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:56:49,856::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:56:59,869::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:57:09,882::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:57:19,896::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:57:29,908::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:57:39,922::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:57:49,936::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:57:59,949::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:58:09,963::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-20 01:58:19,416::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-1024767::ERROR::2013-11-20 01:58:19,980::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-20 01:58:20,132::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 01:58:29,994::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:58:40,007::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:58:50,020::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:59:00,034::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:59:10,048::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:59:20,061::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:59:30,075::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:59:40,089::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 01:59:50,103::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:00:00,116::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:00:10,130::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:00:20,144::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:00:30,158::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:00:40,171::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:00:50,183::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-20 02:00:52,087::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 02:00:52,812::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 02:01:00,329::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:01:10,343::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:01:20,356::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:01:30,369::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:01:40,383::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:01:50,396::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:02:00,409::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:02:10,423::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:02:20,436::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:02:30,449::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:02:40,463::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:02:50,476::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:03:00,489::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:03:10,502::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-20 02:03:20,485::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-1024767::ERROR::2013-11-20 02:03:20,738::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-20 02:03:21,130::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 02:03:30,752::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:03:40,766::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:03:50,779::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:04:00,792::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:04:10,805::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:04:20,818::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:04:30,831::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:04:40,844::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:04:50,857::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:05:00,870::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:05:10,883::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:05:20,896::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:05:30,909::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:05:40,923::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:05:50,936::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-20 02:05:53,201::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 02:05:53,841::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 02:06:00,949::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:06:10,963::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:06:20,976::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:06:30,989::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:06:41,003::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:06:51,017::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:07:01,030::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:07:11,043::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:07:21,057::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:07:31,070::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:07:41,084::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:07:51,097::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:08:01,111::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:08:11,124::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:08:21,138::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-20 02:08:21,579::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 02:08:22,223::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 02:08:31,152::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:08:41,165::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:08:51,178::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:09:01,192::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:09:11,205::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:09:21,218::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:09:31,231::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:09:41,245::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:09:51,258::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:10:01,271::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:10:11,284::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:10:21,297::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:10:31,311::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:10:41,324::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:10:51,337::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-20 02:10:54,225::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 02:10:54,926::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 02:11:01,434::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:11:11,447::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:11:21,461::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:11:31,474::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:11:41,488::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:11:51,501::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:12:01,515::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:12:11,528::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:12:21,542::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:12:31,555::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:12:41,569::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:12:51,582::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:13:01,595::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:13:11,609::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:13:21,623::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-20 02:13:22,599::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 02:13:23,241::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 02:13:31,847::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:13:41,861::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:13:51,875::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:14:01,888::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:14:11,902::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:14:21,916::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:14:31,929::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:14:41,943::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:14:51,956::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:15:01,970::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:15:11,984::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:15:21,998::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:15:32,011::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:15:42,025::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:15:52,039::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-20 02:15:55,312::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 02:15:55,972::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 02:16:02,052::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:16:12,066::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:16:22,080::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:16:32,094::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:16:42,108::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:16:52,122::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:17:02,135::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:17:12,148::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:17:22,162::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:17:32,175::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:17:42,188::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:17:52,201::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:18:02,215::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:18:12,228::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:18:22,241::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-20 02:18:23,707::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 02:18:24,430::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 02:18:32,255::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:18:42,268::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:18:52,282::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:19:02,295::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:19:12,309::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:19:22,322::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:19:32,335::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:19:42,349::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:19:52,362::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:20:02,376::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:20:12,390::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:20:22,403::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:20:32,417::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:20:42,430::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:20:52,444::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42838::DEBUG::2013-11-20 02:20:56,353::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 02:20:56,969::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 02:21:02,558::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:21:12,572::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:21:22,588::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:21:32,604::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:21:42,617::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:21:52,631::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:22:02,645::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:22:12,659::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:22:22,672::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:22:32,686::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:22:42,700::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:22:52,713::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:23:02,727::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:23:12,741::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-1024767::ERROR::2013-11-20 02:23:22,754::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-42840::DEBUG::2013-11-20 02:23:24,787::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 02:23:25,516::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912180::INFO::2013-11-20 02:23:30,364::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912180::DEBUG::2013-11-20 02:23:30,365::task::495::TaskManager.Task::(_debug) Task 2b684da7-ca08-42df-a66c-91527cfb2c77: moving from state init -> state preparing
>Thread-15912180::DEBUG::2013-11-20 02:23:30,374::task::495::TaskManager.Task::(_debug) Task 2b684da7-ca08-42df-a66c-91527cfb2c77: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000353097915649', 'lastCheck': 1384914207.1672649, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00137400627136', 'lastCheck': 1384914201.457849, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384914202.7555621, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000349998474121', 'lastCheck': 1384914205.5183051, 'valid': True, 'code': 0}}
>Thread-15912180::DEBUG::2013-11-20 02:23:30,380::task::495::TaskManager.Task::(_debug) Task 2b684da7-ca08-42df-a66c-91527cfb2c77: moving from state preparing -> state finished
>Thread-15912180::DEBUG::2013-11-20 02:23:30,381::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912180::DEBUG::2013-11-20 02:23:30,381::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912180::DEBUG::2013-11-20 02:23:30,382::task::495::TaskManager.Task::(_debug) Task 2b684da7-ca08-42df-a66c-91527cfb2c77: ref 0 aborting False
>Thread-15912180::INFO::2013-11-20 02:23:30,388::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000353097915649', 'lastCheck': 1384914207.1672649, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00137400627136', 'lastCheck': 1384914201.457849, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384914202.7555621, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000349998474121', 'lastCheck': 1384914205.5183051, 'valid': True, 'code': 0}}
>Thread-15912181::DEBUG::2013-11-20 02:23:30,623::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVdsCapabilities with () {}
>Thread-15912181::DEBUG::2013-11-20 02:23:31,078::utils::579::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-img-rhev' (cwd None)
>Thread-15912181::DEBUG::2013-11-20 02:23:31,507::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912181::DEBUG::2013-11-20 02:23:31,509::utils::579::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-kvm-rhev' (cwd None)
>Thread-15912181::DEBUG::2013-11-20 02:23:31,932::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912181::DEBUG::2013-11-20 02:23:31,934::utils::579::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" vdsm' (cwd None)
>Thread-15912181::DEBUG::2013-11-20 02:23:32,367::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912181::DEBUG::2013-11-20 02:23:32,369::utils::579::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" spice-server' (cwd None)
>Thread-15912181::DEBUG::2013-11-20 02:23:32,781::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912181::DEBUG::2013-11-20 02:23:32,783::utils::579::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" libvirt' (cwd None)
>Thread-1024767::ERROR::2013-11-20 02:23:33,049::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912181::DEBUG::2013-11-20 02:23:33,232::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912181::DEBUG::2013-11-20 02:23:33,235::clientIF::59::vds::(wrapper) return getVdsCapabilities with {'status': {'message': 'Done', 'code': 0}, 'info': {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:c936ebb147a8'}], 'FC': []}, 'packages2': {'kernel': {'release': '279.11.1.el6.x86_64', 'buildtime': 1348297826.0, 'version': '2.6.32'}, 'spice-server': {'release': '10.el6', 'buildtime': '1337611492', 'version': '0.10.1'}, 'vdsm': {'release': '113.4.el6_3', 'buildtime': '1347534072', 'version': '4.9'}, 'qemu-kvm': {'release': '2.295.el6_3.2', 'buildtime': '1346673796', 'version': '0.12.1.2'}, 'libvirt': {'release': '21.el6_3.5', 'buildtime': '1348687577', 'version': '0.9.10'}, 'qemu-img': {'release': '2.295.el6_3.2', 'buildtime': '1346673796', 'version': '0.12.1.2'}}, 'cpuModel': 'Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz', 'hooks': {'before_vm_start': {'50_vhostmd': {'md5': '1635e45584995d9ad2629939cb0bcdbf'}}, 'after_vm_destroy': {'50_vhostmd': {'md5': '7e7509430304099f38e4a9de966af090'}}, 'before_vm_dehibernate': {'50_vhostmd': {'md5': '1635e45584995d9ad2629939cb0bcdbf'}}, 'before_vm_migrate_destination': {'50_vhostmd': {'md5': '1635e45584995d9ad2629939cb0bcdbf'}}}, 'vmTypes': ['kvm'], 'supportedProtocols': ['2.2', '2.3'], 'networks': {'rhevm': {'addr': '192.168.100.133', 'cfg': {'IPADDR': '192.168.100.133', 'GATEWAY': '192.168.100.1', 'DELAY': '0', 'NETMASK': '255.255.128.0', 'STP': 'no', 'DEVICE': 'rhevm', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'ports': ['bond0'], 'netmask': '255.255.128.0', 'stp': 'off', 'gateway': '192.168.100.1'}, 'DMZ': {'addr': '172.16.1.133', 'cfg': {'IPADDR': '172.16.1.133', 'DELAY': '0', 'NETMASK': '255.255.255.0', 'STP': 'no', 'DEVICE': 'DMZ', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'ports': ['eth0'], 'netmask': '255.255.255.0', 'stp': 'off', 'gateway': '0.0.0.0'}}, 'uuid': '30343536-3138-5A43-4A32-333630373442_AC:16:2D:76:EB:F0', 'lastClientIface': 'rhevm', 'nics': {'eth3': {'hwaddr': 'AC:16:2D:76:EB:F2', 'netmask': '', 'speed': 1000, 'addr': '', 'permhwaddr': 'AC:16:2D:76:EB:F3'}, 'eth2': {'hwaddr': 'AC:16:2D:76:EB:F2', 'netmask': '', 'speed': 1000, 'addr': '', 'permhwaddr': 'AC:16:2D:76:EB:F2'}, 'eth1': {'hwaddr': 'AC:16:2D:76:EB:F1', 'netmask': '', 'speed': 0, 'addr': ''}, 'eth0': {'hwaddr': 'AC:16:2D:76:EB:F0', 'netmask': '', 'speed': 1000, 'addr': ''}}, 'software_revision': '113.4', 'management_ip': '192.168.100.133', 'clusterLevels': ['3.0'], 'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,constant_tsc,arch_perfmon,pebs,bts,rep_good,xtopology,nonstop_tsc,aperfmperf,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,dca,sse4_1,sse4_2,x2apic,popcnt,aes,xsave,avx,lahf_lm,ida,arat,epb,xsaveopt,pln,pts,dts,tpr_shadow,vnmi,flexpriority,ept,vpid,model_486,model_pentium,model_pentium2,model_pentium3,model_pentiumpro,model_qemu32,model_cpu64-rhel5,model_cpu64-rhel6,model_kvm32,model_coreduo,model_kvm64,model_core2duo,model_n270,model_Conroe,model_Penryn,model_Nehalem,model_Westmere,model_SandyBridge,model_Opteron_G1', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:c936ebb147a8', 'memSize': '64392', 'reservedMem': '256', 'bondings': {'bond4': {'hwaddr': '00:00:00:00:00:00', 'cfg': {}, 'netmask': '', 'addr': '', 'slaves': []}, 'bond0': {'hwaddr': 'AC:16:2D:76:EB:F2', 'cfg': {'DEVICE': 'bond0', 'BRIDGE': 'rhevm', 'BOOTPROTO': 'none', 'ONBOOT': 'yes', 'BONDING_OPTS': 'mode=1 miimon=100'}, 'netmask': '', 'addr': '', 'slaves': ['eth2', 'eth3']}, 'bond1': {'hwaddr': '00:00:00:00:00:00', 'cfg': {}, 'netmask': '', 'addr': '', 'slaves': []}, 'bond2': {'hwaddr': '00:00:00:00:00:00', 'cfg': {}, 'netmask': '', 'addr': '', 'slaves': []}, 'bond3': {'hwaddr': '00:00:00:00:00:00', 'cfg': {}, 'netmask': '', 'addr': '', 'slaves': []}}, 'software_version': '4.9', 'cpuSpeed': '1995.126', 'cpuSockets': '2', 'vlans': {}, 'cpuCores': '12', 'kvmEnabled': 'true', 'guestOverhead': '65', 'supportedRHEVMs': ['3.0'], 'version_name': 'Snow Man', 'emulatedMachines': [u'rhel6.3.0', u'pc', u'rhel6.2.0', u'rhel6.1.0', u'rhel6.0.0', u'rhel5.5.0', u'rhel5.4.4', u'rhel5.4.0'], 'operatingSystem': {'release': '20121012.0.el6_3', 'version': '6.3', 'name': 'RHEV Hypervisor'}, 'lastClient': '192.168.100.135'}}
>Thread-15912184::DEBUG::2013-11-20 02:23:37,366::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912184::INFO::2013-11-20 02:23:37,368::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912184::DEBUG::2013-11-20 02:23:37,369::task::495::TaskManager.Task::(_debug) Task 7c3c0c07-af15-47ed-92cb-859532c0d677: moving from state init -> state preparing
>Thread-15912184::DEBUG::2013-11-20 02:23:37,369::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=Free spmLver=285 spmId=-1
>Thread-15912184::DEBUG::2013-11-20 02:23:37,370::task::495::TaskManager.Task::(_debug) Task 7c3c0c07-af15-47ed-92cb-859532c0d677: finished: {'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': 285}}
>Thread-15912184::DEBUG::2013-11-20 02:23:37,371::task::495::TaskManager.Task::(_debug) Task 7c3c0c07-af15-47ed-92cb-859532c0d677: moving from state preparing -> state finished
>Thread-15912184::DEBUG::2013-11-20 02:23:37,371::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912184::DEBUG::2013-11-20 02:23:37,372::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912184::DEBUG::2013-11-20 02:23:37,372::task::495::TaskManager.Task::(_debug) Task 7c3c0c07-af15-47ed-92cb-859532c0d677: ref 0 aborting False
>Thread-15912184::INFO::2013-11-20 02:23:37,373::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': 285}}
>Thread-15912185::DEBUG::2013-11-20 02:23:37,424::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912185::INFO::2013-11-20 02:23:37,425::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: spmStart, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 prevID=-1 prevLVER=285 recoveryMode=0 scsiFencing=false maxHostID=250 domVersion=2)
>Thread-15912185::DEBUG::2013-11-20 02:23:37,425::task::495::TaskManager.Task::(_debug) Task 378b55ad-db2a-4978-8944-3175e4685c11: moving from state init -> state preparing
>Thread-15912185::DEBUG::2013-11-20 02:23:37,427::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3eb220e1-a94e-4fe7-bbe4-68dc9b877d0b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '330' at 'public_spmStart'
>Thread-15912185::DEBUG::2013-11-20 02:23:37,427::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'exclusive'
>Thread-15912185::DEBUG::2013-11-20 02:23:37,428::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'exclusive' (1 active user)
>Thread-15912185::DEBUG::2013-11-20 02:23:37,429::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3eb220e1-a94e-4fe7-bbe4-68dc9b877d0b`::Granted request
>Thread-15912185::DEBUG::2013-11-20 02:23:37,430::task::495::TaskManager.Task::(_debug) Task 378b55ad-db2a-4978-8944-3175e4685c11: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (exclusive)
>Thread-15912185::DEBUG::2013-11-20 02:23:37,430::task::495::TaskManager.Task::(_debug) Task 378b55ad-db2a-4978-8944-3175e4685c11: ref 1 aborting False
>Thread-15912185::DEBUG::2013-11-20 02:23:37,431::task::495::TaskManager.Task::(_debug) Task 378b55ad-db2a-4978-8944-3175e4685c11: Prepare: 1 jobs exist, move to acquiring
>Thread-15912185::DEBUG::2013-11-20 02:23:37,432::task::495::TaskManager.Task::(_debug) Task 378b55ad-db2a-4978-8944-3175e4685c11: moving from state preparing -> state acquiring
>Thread-15912185::DEBUG::2013-11-20 02:23:37,432::task::495::TaskManager.Task::(_debug) Task 378b55ad-db2a-4978-8944-3175e4685c11: moving from state acquiring -> state queued
>Thread-15912185::DEBUG::2013-11-20 02:23:37,433::taskManager::37::TaskManager::(_queueTask) queueing task: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912185::DEBUG::2013-11-20 02:23:37,433::taskManager::43::TaskManager::(_queueTask) task queued: 378b55ad-db2a-4978-8944-3175e4685c11
>baa736ad-461c-4f0e-89d8-6875408fdc80::DEBUG::2013-11-20 02:23:37,434::threadPool::47::Misc.ThreadPool::(setRunningTask) Number of running tasks: 1
>Thread-15912185::DEBUG::2013-11-20 02:23:37,434::task::1183::TaskManager.Task::(prepare) returning
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:37,435::threadPool::192::Misc.ThreadPool.WorkerThread::(run) Task: 378b55ad-db2a-4978-8944-3175e4685c11 running: <bound method Task.commit of <storage.task.Task instance at 0x7f487c044cf8>> with: None
>Thread-15912185::DEBUG::2013-11-20 02:23:37,436::task::495::TaskManager.Task::(_debug) Task 378b55ad-db2a-4978-8944-3175e4685c11: ref 0 aborting False
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:37,436::task::1195::TaskManager.Task::(commit) Commiting task: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912185::INFO::2013-11-20 02:23:37,437::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: spmStart, Return response: {'status': {'message': 'OK', 'code': 0}, 'uuid': '378b55ad-db2a-4978-8944-3175e4685c11'}
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:37,438::task::495::TaskManager.Task::(_debug) Task 378b55ad-db2a-4978-8944-3175e4685c11: moving from state queued -> state running
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:37,439::task::907::TaskManager.Task::(_runJobs) Task.run: running job 0: spmStart: <bound method SPM.start of <storage.spm.SPM instance at 0x1bf9f38>> (args: ('c4e5889a-348d-11e2-83fe-9f72369eab30', -1, '285', 0, 'false', 250, 2) kwargs: {})
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:37,440::task::300::TaskManager.Task::(run) Job.run: running spmStart: <bound method SPM.start of <storage.spm.SPM instance at 0x1bf9f38>> (args: ('c4e5889a-348d-11e2-83fe-9f72369eab30', -1, '285', 0, 'false', 250, 2) kwargs: {}) callback None
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:37,441::spm::433::Storage.SPM::(start) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:37,441::misc::1010::SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads)
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:37,442::misc::1012::SamplingMethod::(__call__) Got in to sampling method
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:37,442::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:37,443::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:37,444::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:37,444::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:37,445::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:37,446::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:38,162::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:38,165::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:38,166::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_ID=1', u'POOL_SPM_LVER=286', u'_SHA_CKSUM=09323c06acf399057c7d77c795a76a028bc8ee27']
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:38,168::misc::1020::SamplingMethod::(__call__) Returning last result
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:38,169::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:38,169::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:38,170::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:38,170::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:38,171::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:38,172::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15912186::DEBUG::2013-11-20 02:23:38,462::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912186::INFO::2013-11-20 02:23:38,465::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912186::DEBUG::2013-11-20 02:23:38,465::task::495::TaskManager.Task::(_debug) Task b528e0ad-1118-47da-ba4b-bb7a60108173: moving from state init -> state preparing
>Thread-15912186::DEBUG::2013-11-20 02:23:38,466::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912186::DEBUG::2013-11-20 02:23:38,467::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912186::DEBUG::2013-11-20 02:23:38,467::task::495::TaskManager.Task::(_debug) Task b528e0ad-1118-47da-ba4b-bb7a60108173: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912186::DEBUG::2013-11-20 02:23:38,468::task::495::TaskManager.Task::(_debug) Task b528e0ad-1118-47da-ba4b-bb7a60108173: moving from state preparing -> state finished
>Thread-15912186::DEBUG::2013-11-20 02:23:38,469::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912186::DEBUG::2013-11-20 02:23:38,469::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912186::DEBUG::2013-11-20 02:23:38,470::task::495::TaskManager.Task::(_debug) Task b528e0ad-1118-47da-ba4b-bb7a60108173: ref 0 aborting False
>Thread-15912186::INFO::2013-11-20 02:23:38,471::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:38,774::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:38,777::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:38,778::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_ID=1', u'POOL_SPM_LVER=286', u'_SHA_CKSUM=09323c06acf399057c7d77c795a76a028bc8ee27']
>378b55ad-db2a-4978-8944-3175e4685c11::INFO::2013-11-20 02:23:38,780::spm::457::Storage.SPM::(start) expected previd:1 lver:286 got request for previd:-1 lver:285
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:38,781::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:38,782::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:38,782::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:38,783::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:38,784::persistentDict::163::Storage.PersistentDict::(__init__) Created a persistant dict with LvMetadataRW backend
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:38,784::blockSD::164::Storage.Misc.excCmd::(readlines) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:39,203::blockSD::164::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000377853 s, 5.4 MB/s\n'; <rc> = 0
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:39,205::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000377853 s, 5.4 MB/s'], size: 2048
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:39,205::persistentDict::204::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[]
>378b55ad-db2a-4978-8944-3175e4685c11::WARNING::2013-11-20 02:23:39,206::persistentDict::226::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:39,206::persistentDict::163::Storage.PersistentDict::(__init__) Created a persistant dict with VGTagMetadataRW backend
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:39,207::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:39,208::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:39,208::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:39,209::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:39,210::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:39,211::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15912187::DEBUG::2013-11-20 02:23:39,495::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912187::INFO::2013-11-20 02:23:39,497::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912187::DEBUG::2013-11-20 02:23:39,498::task::495::TaskManager.Task::(_debug) Task 496de84f-15ea-456f-b27a-82a25d45f34b: moving from state init -> state preparing
>Thread-15912187::DEBUG::2013-11-20 02:23:39,499::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912187::DEBUG::2013-11-20 02:23:39,499::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912187::DEBUG::2013-11-20 02:23:39,500::task::495::TaskManager.Task::(_debug) Task 496de84f-15ea-456f-b27a-82a25d45f34b: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912187::DEBUG::2013-11-20 02:23:39,501::task::495::TaskManager.Task::(_debug) Task 496de84f-15ea-456f-b27a-82a25d45f34b: moving from state preparing -> state finished
>Thread-15912187::DEBUG::2013-11-20 02:23:39,501::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912187::DEBUG::2013-11-20 02:23:39,502::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912187::DEBUG::2013-11-20 02:23:39,503::task::495::TaskManager.Task::(_debug) Task 496de84f-15ea-456f-b27a-82a25d45f34b: ref 0 aborting False
>Thread-15912187::INFO::2013-11-20 02:23:39,503::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:39,796::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:39,799::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:39,800::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_ID=1', u'POOL_SPM_LVER=286', u'_SHA_CKSUM=09323c06acf399057c7d77c795a76a028bc8ee27']
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:39,801::safelease::52::ClusterLock::(acquire) Acquiring cluster lock for domain 6b892067-ac7e-4039-a7be-5b7387525b16
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:23:39,802::safelease::61::Storage.Misc.excCmd::(acquire) '/usr/bin/sudo -n /usr/bin/setsid /usr/bin/ionice -c1 -n0 /bin/su vdsm -s /bin/sh -c "/usr/libexec/vdsm/spmprotect.sh start 6b892067-ac7e-4039-a7be-5b7387525b16 2 5 /dev/6b892067-ac7e-4039-a7be-5b7387525b16/leases 60000 10000 3"' (cwd /usr/libexec/vdsm/)
>Thread-15912189::DEBUG::2013-11-20 02:23:40,523::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912189::INFO::2013-11-20 02:23:40,525::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912189::DEBUG::2013-11-20 02:23:40,525::task::495::TaskManager.Task::(_debug) Task 4119e7e8-0761-47b0-8eff-0c6f7a74af1c: moving from state init -> state preparing
>Thread-15912189::DEBUG::2013-11-20 02:23:40,526::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912189::DEBUG::2013-11-20 02:23:40,526::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912189::DEBUG::2013-11-20 02:23:40,527::task::495::TaskManager.Task::(_debug) Task 4119e7e8-0761-47b0-8eff-0c6f7a74af1c: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912189::DEBUG::2013-11-20 02:23:40,528::task::495::TaskManager.Task::(_debug) Task 4119e7e8-0761-47b0-8eff-0c6f7a74af1c: moving from state preparing -> state finished
>Thread-15912189::DEBUG::2013-11-20 02:23:40,528::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912189::DEBUG::2013-11-20 02:23:40,529::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912189::DEBUG::2013-11-20 02:23:40,529::task::495::TaskManager.Task::(_debug) Task 4119e7e8-0761-47b0-8eff-0c6f7a74af1c: ref 0 aborting False
>Thread-15912189::INFO::2013-11-20 02:23:40,530::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912190::DEBUG::2013-11-20 02:23:41,551::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912190::INFO::2013-11-20 02:23:41,552::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912190::DEBUG::2013-11-20 02:23:41,552::task::495::TaskManager.Task::(_debug) Task 64ce2bc1-7ab6-4ebd-9f8e-3a73b0687057: moving from state init -> state preparing
>Thread-15912190::DEBUG::2013-11-20 02:23:41,553::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912190::DEBUG::2013-11-20 02:23:41,553::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912190::DEBUG::2013-11-20 02:23:41,554::task::495::TaskManager.Task::(_debug) Task 64ce2bc1-7ab6-4ebd-9f8e-3a73b0687057: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912190::DEBUG::2013-11-20 02:23:41,554::task::495::TaskManager.Task::(_debug) Task 64ce2bc1-7ab6-4ebd-9f8e-3a73b0687057: moving from state preparing -> state finished
>Thread-15912190::DEBUG::2013-11-20 02:23:41,555::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912190::DEBUG::2013-11-20 02:23:41,556::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912190::DEBUG::2013-11-20 02:23:41,556::task::495::TaskManager.Task::(_debug) Task 64ce2bc1-7ab6-4ebd-9f8e-3a73b0687057: ref 0 aborting False
>Thread-15912190::INFO::2013-11-20 02:23:41,557::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912191::DEBUG::2013-11-20 02:23:42,577::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912191::INFO::2013-11-20 02:23:42,578::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912191::DEBUG::2013-11-20 02:23:42,578::task::495::TaskManager.Task::(_debug) Task 73529b0e-3dd9-46d7-9969-dc1f4a0bae86: moving from state init -> state preparing
>Thread-15912191::DEBUG::2013-11-20 02:23:42,579::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912191::DEBUG::2013-11-20 02:23:42,579::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912191::DEBUG::2013-11-20 02:23:42,580::task::495::TaskManager.Task::(_debug) Task 73529b0e-3dd9-46d7-9969-dc1f4a0bae86: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912191::DEBUG::2013-11-20 02:23:42,581::task::495::TaskManager.Task::(_debug) Task 73529b0e-3dd9-46d7-9969-dc1f4a0bae86: moving from state preparing -> state finished
>Thread-15912191::DEBUG::2013-11-20 02:23:42,581::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912191::DEBUG::2013-11-20 02:23:42,582::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912191::DEBUG::2013-11-20 02:23:42,582::task::495::TaskManager.Task::(_debug) Task 73529b0e-3dd9-46d7-9969-dc1f4a0bae86: ref 0 aborting False
>Thread-15912191::INFO::2013-11-20 02:23:42,583::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-1024767::ERROR::2013-11-20 02:23:43,064::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912193::DEBUG::2013-11-20 02:23:43,601::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912193::INFO::2013-11-20 02:23:43,602::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912193::DEBUG::2013-11-20 02:23:43,603::task::495::TaskManager.Task::(_debug) Task f23bc60b-df39-47cf-8f5c-7c57f016bff0: moving from state init -> state preparing
>Thread-15912193::DEBUG::2013-11-20 02:23:43,603::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912193::DEBUG::2013-11-20 02:23:43,604::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912193::DEBUG::2013-11-20 02:23:43,605::task::495::TaskManager.Task::(_debug) Task f23bc60b-df39-47cf-8f5c-7c57f016bff0: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912193::DEBUG::2013-11-20 02:23:43,605::task::495::TaskManager.Task::(_debug) Task f23bc60b-df39-47cf-8f5c-7c57f016bff0: moving from state preparing -> state finished
>Thread-15912193::DEBUG::2013-11-20 02:23:43,606::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912193::DEBUG::2013-11-20 02:23:43,606::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912193::DEBUG::2013-11-20 02:23:43,607::task::495::TaskManager.Task::(_debug) Task f23bc60b-df39-47cf-8f5c-7c57f016bff0: ref 0 aborting False
>Thread-15912193::INFO::2013-11-20 02:23:43,607::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912194::DEBUG::2013-11-20 02:23:44,626::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912194::INFO::2013-11-20 02:23:44,627::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912194::DEBUG::2013-11-20 02:23:44,627::task::495::TaskManager.Task::(_debug) Task 8edd6109-40c5-4482-81ea-db3e563f08f2: moving from state init -> state preparing
>Thread-15912194::DEBUG::2013-11-20 02:23:44,628::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912194::DEBUG::2013-11-20 02:23:44,628::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912194::DEBUG::2013-11-20 02:23:44,629::task::495::TaskManager.Task::(_debug) Task 8edd6109-40c5-4482-81ea-db3e563f08f2: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912194::DEBUG::2013-11-20 02:23:44,630::task::495::TaskManager.Task::(_debug) Task 8edd6109-40c5-4482-81ea-db3e563f08f2: moving from state preparing -> state finished
>Thread-15912194::DEBUG::2013-11-20 02:23:44,630::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912194::DEBUG::2013-11-20 02:23:44,631::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912194::DEBUG::2013-11-20 02:23:44,631::task::495::TaskManager.Task::(_debug) Task 8edd6109-40c5-4482-81ea-db3e563f08f2: ref 0 aborting False
>Thread-15912194::INFO::2013-11-20 02:23:44,632::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912195::DEBUG::2013-11-20 02:23:45,656::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912195::INFO::2013-11-20 02:23:45,657::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912195::DEBUG::2013-11-20 02:23:45,657::task::495::TaskManager.Task::(_debug) Task 418a03b6-fc4f-43c5-9292-1fa607a62cd9: moving from state init -> state preparing
>Thread-15912195::DEBUG::2013-11-20 02:23:45,658::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912195::DEBUG::2013-11-20 02:23:45,658::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912195::DEBUG::2013-11-20 02:23:45,659::task::495::TaskManager.Task::(_debug) Task 418a03b6-fc4f-43c5-9292-1fa607a62cd9: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912195::DEBUG::2013-11-20 02:23:45,659::task::495::TaskManager.Task::(_debug) Task 418a03b6-fc4f-43c5-9292-1fa607a62cd9: moving from state preparing -> state finished
>Thread-15912195::DEBUG::2013-11-20 02:23:45,660::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912195::DEBUG::2013-11-20 02:23:45,661::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912195::DEBUG::2013-11-20 02:23:45,661::task::495::TaskManager.Task::(_debug) Task 418a03b6-fc4f-43c5-9292-1fa607a62cd9: ref 0 aborting False
>Thread-15912195::INFO::2013-11-20 02:23:45,662::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912197::DEBUG::2013-11-20 02:23:46,680::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912197::INFO::2013-11-20 02:23:46,681::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912197::DEBUG::2013-11-20 02:23:46,682::task::495::TaskManager.Task::(_debug) Task afc0fd8f-339f-4a51-bee9-4c8fe6239eee: moving from state init -> state preparing
>Thread-15912197::DEBUG::2013-11-20 02:23:46,682::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912197::DEBUG::2013-11-20 02:23:46,683::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912197::DEBUG::2013-11-20 02:23:46,683::task::495::TaskManager.Task::(_debug) Task afc0fd8f-339f-4a51-bee9-4c8fe6239eee: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912197::DEBUG::2013-11-20 02:23:46,684::task::495::TaskManager.Task::(_debug) Task afc0fd8f-339f-4a51-bee9-4c8fe6239eee: moving from state preparing -> state finished
>Thread-15912197::DEBUG::2013-11-20 02:23:46,685::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912197::DEBUG::2013-11-20 02:23:46,685::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912197::DEBUG::2013-11-20 02:23:46,686::task::495::TaskManager.Task::(_debug) Task afc0fd8f-339f-4a51-bee9-4c8fe6239eee: ref 0 aborting False
>Thread-15912197::INFO::2013-11-20 02:23:46,686::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912198::DEBUG::2013-11-20 02:23:47,705::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912198::INFO::2013-11-20 02:23:47,706::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912198::DEBUG::2013-11-20 02:23:47,706::task::495::TaskManager.Task::(_debug) Task 0471cc38-fa01-4cf5-99e5-50b8fcd668b9: moving from state init -> state preparing
>Thread-15912198::DEBUG::2013-11-20 02:23:47,707::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912198::DEBUG::2013-11-20 02:23:47,707::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912198::DEBUG::2013-11-20 02:23:47,708::task::495::TaskManager.Task::(_debug) Task 0471cc38-fa01-4cf5-99e5-50b8fcd668b9: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912198::DEBUG::2013-11-20 02:23:47,709::task::495::TaskManager.Task::(_debug) Task 0471cc38-fa01-4cf5-99e5-50b8fcd668b9: moving from state preparing -> state finished
>Thread-15912198::DEBUG::2013-11-20 02:23:47,709::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912198::DEBUG::2013-11-20 02:23:47,710::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912198::DEBUG::2013-11-20 02:23:47,710::task::495::TaskManager.Task::(_debug) Task 0471cc38-fa01-4cf5-99e5-50b8fcd668b9: ref 0 aborting False
>Thread-15912198::INFO::2013-11-20 02:23:47,711::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912199::DEBUG::2013-11-20 02:23:48,729::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912199::INFO::2013-11-20 02:23:48,730::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912199::DEBUG::2013-11-20 02:23:48,730::task::495::TaskManager.Task::(_debug) Task 79e57ed0-fe38-4671-8ecc-25cb845d147a: moving from state init -> state preparing
>Thread-15912199::DEBUG::2013-11-20 02:23:48,731::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912199::DEBUG::2013-11-20 02:23:48,731::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912199::DEBUG::2013-11-20 02:23:48,732::task::495::TaskManager.Task::(_debug) Task 79e57ed0-fe38-4671-8ecc-25cb845d147a: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912199::DEBUG::2013-11-20 02:23:48,733::task::495::TaskManager.Task::(_debug) Task 79e57ed0-fe38-4671-8ecc-25cb845d147a: moving from state preparing -> state finished
>Thread-15912199::DEBUG::2013-11-20 02:23:48,733::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912199::DEBUG::2013-11-20 02:23:48,734::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912199::DEBUG::2013-11-20 02:23:48,734::task::495::TaskManager.Task::(_debug) Task 79e57ed0-fe38-4671-8ecc-25cb845d147a: ref 0 aborting False
>Thread-15912199::INFO::2013-11-20 02:23:48,735::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912200::INFO::2013-11-20 02:23:49,273::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912200::DEBUG::2013-11-20 02:23:49,274::task::495::TaskManager.Task::(_debug) Task af9d74fb-5b58-4565-b6ec-53259db6f509: moving from state init -> state preparing
>Thread-15912200::DEBUG::2013-11-20 02:23:49,274::task::495::TaskManager.Task::(_debug) Task af9d74fb-5b58-4565-b6ec-53259db6f509: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00028920173645', 'lastCheck': 1384914227.4294109, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00163197517395', 'lastCheck': 1384914221.662358, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384914223.065506, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00045108795166', 'lastCheck': 1384914225.7559669, 'valid': True, 'code': 0}}
>Thread-15912200::DEBUG::2013-11-20 02:23:49,282::task::495::TaskManager.Task::(_debug) Task af9d74fb-5b58-4565-b6ec-53259db6f509: moving from state preparing -> state finished
>Thread-15912200::DEBUG::2013-11-20 02:23:49,282::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912200::DEBUG::2013-11-20 02:23:49,283::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912200::DEBUG::2013-11-20 02:23:49,283::task::495::TaskManager.Task::(_debug) Task af9d74fb-5b58-4565-b6ec-53259db6f509: ref 0 aborting False
>Thread-15912200::INFO::2013-11-20 02:23:49,286::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00028920173645', 'lastCheck': 1384914227.4294109, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00163197517395', 'lastCheck': 1384914221.662358, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384914223.065506, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00045108795166', 'lastCheck': 1384914225.7559669, 'valid': True, 'code': 0}}
>Thread-15912202::DEBUG::2013-11-20 02:23:49,758::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912202::INFO::2013-11-20 02:23:49,764::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912202::DEBUG::2013-11-20 02:23:49,766::task::495::TaskManager.Task::(_debug) Task 5a416a22-d534-454c-b6c0-6fce64c077ff: moving from state init -> state preparing
>Thread-15912202::DEBUG::2013-11-20 02:23:49,768::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912202::DEBUG::2013-11-20 02:23:49,769::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912202::DEBUG::2013-11-20 02:23:49,771::task::495::TaskManager.Task::(_debug) Task 5a416a22-d534-454c-b6c0-6fce64c077ff: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912202::DEBUG::2013-11-20 02:23:49,772::task::495::TaskManager.Task::(_debug) Task 5a416a22-d534-454c-b6c0-6fce64c077ff: moving from state preparing -> state finished
>Thread-15912202::DEBUG::2013-11-20 02:23:49,774::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912202::DEBUG::2013-11-20 02:23:49,775::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912202::DEBUG::2013-11-20 02:23:49,776::task::495::TaskManager.Task::(_debug) Task 5a416a22-d534-454c-b6c0-6fce64c077ff: ref 0 aborting False
>Thread-15912202::INFO::2013-11-20 02:23:49,776::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912203::DEBUG::2013-11-20 02:23:50,794::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912203::INFO::2013-11-20 02:23:50,795::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912203::DEBUG::2013-11-20 02:23:50,805::task::495::TaskManager.Task::(_debug) Task 0e5dbc65-b30b-435c-996d-90b50f0830f0: moving from state init -> state preparing
>Thread-15912203::DEBUG::2013-11-20 02:23:50,806::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912203::DEBUG::2013-11-20 02:23:50,807::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912203::DEBUG::2013-11-20 02:23:50,807::task::495::TaskManager.Task::(_debug) Task 0e5dbc65-b30b-435c-996d-90b50f0830f0: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912203::DEBUG::2013-11-20 02:23:50,808::task::495::TaskManager.Task::(_debug) Task 0e5dbc65-b30b-435c-996d-90b50f0830f0: moving from state preparing -> state finished
>Thread-15912203::DEBUG::2013-11-20 02:23:50,808::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912203::DEBUG::2013-11-20 02:23:50,809::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912203::DEBUG::2013-11-20 02:23:50,809::task::495::TaskManager.Task::(_debug) Task 0e5dbc65-b30b-435c-996d-90b50f0830f0: ref 0 aborting False
>Thread-15912203::INFO::2013-11-20 02:23:50,810::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912204::DEBUG::2013-11-20 02:23:51,828::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912204::INFO::2013-11-20 02:23:51,829::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912204::DEBUG::2013-11-20 02:23:51,830::task::495::TaskManager.Task::(_debug) Task ca9dd415-6388-4d77-bda7-033942af1490: moving from state init -> state preparing
>Thread-15912204::DEBUG::2013-11-20 02:23:51,830::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912204::DEBUG::2013-11-20 02:23:51,831::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912204::DEBUG::2013-11-20 02:23:51,831::task::495::TaskManager.Task::(_debug) Task ca9dd415-6388-4d77-bda7-033942af1490: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912204::DEBUG::2013-11-20 02:23:51,832::task::495::TaskManager.Task::(_debug) Task ca9dd415-6388-4d77-bda7-033942af1490: moving from state preparing -> state finished
>Thread-15912204::DEBUG::2013-11-20 02:23:51,832::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912204::DEBUG::2013-11-20 02:23:51,833::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912204::DEBUG::2013-11-20 02:23:51,834::task::495::TaskManager.Task::(_debug) Task ca9dd415-6388-4d77-bda7-033942af1490: ref 0 aborting False
>Thread-15912204::INFO::2013-11-20 02:23:51,834::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912206::DEBUG::2013-11-20 02:23:52,853::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912206::INFO::2013-11-20 02:23:52,854::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912206::DEBUG::2013-11-20 02:23:52,854::task::495::TaskManager.Task::(_debug) Task a1769181-bd6a-4adc-b241-6f63246139b3: moving from state init -> state preparing
>Thread-15912206::DEBUG::2013-11-20 02:23:52,855::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912206::DEBUG::2013-11-20 02:23:52,856::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912206::DEBUG::2013-11-20 02:23:52,857::task::495::TaskManager.Task::(_debug) Task a1769181-bd6a-4adc-b241-6f63246139b3: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912206::DEBUG::2013-11-20 02:23:52,857::task::495::TaskManager.Task::(_debug) Task a1769181-bd6a-4adc-b241-6f63246139b3: moving from state preparing -> state finished
>Thread-15912206::DEBUG::2013-11-20 02:23:52,858::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912206::DEBUG::2013-11-20 02:23:52,858::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912206::DEBUG::2013-11-20 02:23:52,859::task::495::TaskManager.Task::(_debug) Task a1769181-bd6a-4adc-b241-6f63246139b3: ref 0 aborting False
>Thread-15912206::INFO::2013-11-20 02:23:52,860::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-1024767::ERROR::2013-11-20 02:23:53,078::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912207::DEBUG::2013-11-20 02:23:53,878::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912207::INFO::2013-11-20 02:23:53,879::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912207::DEBUG::2013-11-20 02:23:53,880::task::495::TaskManager.Task::(_debug) Task 068dd90f-f5d1-4c98-a5c4-1415ea869f87: moving from state init -> state preparing
>Thread-15912207::DEBUG::2013-11-20 02:23:53,880::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912207::DEBUG::2013-11-20 02:23:53,881::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912207::DEBUG::2013-11-20 02:23:53,881::task::495::TaskManager.Task::(_debug) Task 068dd90f-f5d1-4c98-a5c4-1415ea869f87: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912207::DEBUG::2013-11-20 02:23:53,882::task::495::TaskManager.Task::(_debug) Task 068dd90f-f5d1-4c98-a5c4-1415ea869f87: moving from state preparing -> state finished
>Thread-15912207::DEBUG::2013-11-20 02:23:53,883::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912207::DEBUG::2013-11-20 02:23:53,883::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912207::DEBUG::2013-11-20 02:23:53,884::task::495::TaskManager.Task::(_debug) Task 068dd90f-f5d1-4c98-a5c4-1415ea869f87: ref 0 aborting False
>Thread-15912207::INFO::2013-11-20 02:23:53,884::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912208::DEBUG::2013-11-20 02:23:54,902::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912208::INFO::2013-11-20 02:23:54,903::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912208::DEBUG::2013-11-20 02:23:54,904::task::495::TaskManager.Task::(_debug) Task d0e3b0ac-47c5-490f-99f1-af04391ec6d4: moving from state init -> state preparing
>Thread-15912208::DEBUG::2013-11-20 02:23:54,904::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912208::DEBUG::2013-11-20 02:23:54,905::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912208::DEBUG::2013-11-20 02:23:54,906::task::495::TaskManager.Task::(_debug) Task d0e3b0ac-47c5-490f-99f1-af04391ec6d4: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912208::DEBUG::2013-11-20 02:23:54,906::task::495::TaskManager.Task::(_debug) Task d0e3b0ac-47c5-490f-99f1-af04391ec6d4: moving from state preparing -> state finished
>Thread-15912208::DEBUG::2013-11-20 02:23:54,907::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912208::DEBUG::2013-11-20 02:23:54,907::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912208::DEBUG::2013-11-20 02:23:54,908::task::495::TaskManager.Task::(_debug) Task d0e3b0ac-47c5-490f-99f1-af04391ec6d4: ref 0 aborting False
>Thread-15912208::INFO::2013-11-20 02:23:54,908::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912210::DEBUG::2013-11-20 02:23:55,926::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912210::INFO::2013-11-20 02:23:55,927::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912210::DEBUG::2013-11-20 02:23:55,935::task::495::TaskManager.Task::(_debug) Task 39070c10-d9ec-4677-ac13-dbfa81947b3c: moving from state init -> state preparing
>Thread-15912210::DEBUG::2013-11-20 02:23:55,936::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912210::DEBUG::2013-11-20 02:23:55,936::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912210::DEBUG::2013-11-20 02:23:55,937::task::495::TaskManager.Task::(_debug) Task 39070c10-d9ec-4677-ac13-dbfa81947b3c: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912210::DEBUG::2013-11-20 02:23:55,937::task::495::TaskManager.Task::(_debug) Task 39070c10-d9ec-4677-ac13-dbfa81947b3c: moving from state preparing -> state finished
>Thread-15912210::DEBUG::2013-11-20 02:23:55,938::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912210::DEBUG::2013-11-20 02:23:55,939::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912210::DEBUG::2013-11-20 02:23:55,939::task::495::TaskManager.Task::(_debug) Task 39070c10-d9ec-4677-ac13-dbfa81947b3c: ref 0 aborting False
>Thread-15912210::INFO::2013-11-20 02:23:55,940::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912211::DEBUG::2013-11-20 02:23:57,047::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912211::INFO::2013-11-20 02:23:57,050::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912211::DEBUG::2013-11-20 02:23:57,051::task::495::TaskManager.Task::(_debug) Task 33f6663f-b21c-4613-949b-2345537b14b1: moving from state init -> state preparing
>Thread-15912211::DEBUG::2013-11-20 02:23:57,052::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912211::DEBUG::2013-11-20 02:23:57,053::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912211::DEBUG::2013-11-20 02:23:57,055::task::495::TaskManager.Task::(_debug) Task 33f6663f-b21c-4613-949b-2345537b14b1: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912211::DEBUG::2013-11-20 02:23:57,057::task::495::TaskManager.Task::(_debug) Task 33f6663f-b21c-4613-949b-2345537b14b1: moving from state preparing -> state finished
>Thread-15912211::DEBUG::2013-11-20 02:23:57,058::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912211::DEBUG::2013-11-20 02:23:57,060::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912211::DEBUG::2013-11-20 02:23:57,061::task::495::TaskManager.Task::(_debug) Task 33f6663f-b21c-4613-949b-2345537b14b1: ref 0 aborting False
>Thread-15912211::INFO::2013-11-20 02:23:57,062::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912212::DEBUG::2013-11-20 02:23:58,077::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912212::INFO::2013-11-20 02:23:58,077::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912212::DEBUG::2013-11-20 02:23:58,078::task::495::TaskManager.Task::(_debug) Task 50ada3f8-8403-440c-84de-ecb85b453a17: moving from state init -> state preparing
>Thread-15912212::DEBUG::2013-11-20 02:23:58,079::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912212::DEBUG::2013-11-20 02:23:58,079::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912212::DEBUG::2013-11-20 02:23:58,080::task::495::TaskManager.Task::(_debug) Task 50ada3f8-8403-440c-84de-ecb85b453a17: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912212::DEBUG::2013-11-20 02:23:58,080::task::495::TaskManager.Task::(_debug) Task 50ada3f8-8403-440c-84de-ecb85b453a17: moving from state preparing -> state finished
>Thread-15912212::DEBUG::2013-11-20 02:23:58,081::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912212::DEBUG::2013-11-20 02:23:58,081::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912212::DEBUG::2013-11-20 02:23:58,082::task::495::TaskManager.Task::(_debug) Task 50ada3f8-8403-440c-84de-ecb85b453a17: ref 0 aborting False
>Thread-15912212::INFO::2013-11-20 02:23:58,083::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912214::DEBUG::2013-11-20 02:23:59,099::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912214::INFO::2013-11-20 02:23:59,100::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912214::DEBUG::2013-11-20 02:23:59,101::task::495::TaskManager.Task::(_debug) Task c6c45ce2-5979-455f-8a0a-334cc2426a90: moving from state init -> state preparing
>Thread-15912214::DEBUG::2013-11-20 02:23:59,101::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912214::DEBUG::2013-11-20 02:23:59,102::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912214::DEBUG::2013-11-20 02:23:59,103::task::495::TaskManager.Task::(_debug) Task c6c45ce2-5979-455f-8a0a-334cc2426a90: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912214::DEBUG::2013-11-20 02:23:59,103::task::495::TaskManager.Task::(_debug) Task c6c45ce2-5979-455f-8a0a-334cc2426a90: moving from state preparing -> state finished
>Thread-15912214::DEBUG::2013-11-20 02:23:59,104::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912214::DEBUG::2013-11-20 02:23:59,104::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912214::DEBUG::2013-11-20 02:23:59,105::task::495::TaskManager.Task::(_debug) Task c6c45ce2-5979-455f-8a0a-334cc2426a90: ref 0 aborting False
>Thread-15912214::INFO::2013-11-20 02:23:59,105::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912215::DEBUG::2013-11-20 02:24:00,123::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912215::INFO::2013-11-20 02:24:00,123::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912215::DEBUG::2013-11-20 02:24:00,124::task::495::TaskManager.Task::(_debug) Task 91577527-f981-4e23-92c1-44f505cbdb53: moving from state init -> state preparing
>Thread-15912215::DEBUG::2013-11-20 02:24:00,125::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912215::DEBUG::2013-11-20 02:24:00,125::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912215::DEBUG::2013-11-20 02:24:00,126::task::495::TaskManager.Task::(_debug) Task 91577527-f981-4e23-92c1-44f505cbdb53: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912215::DEBUG::2013-11-20 02:24:00,126::task::495::TaskManager.Task::(_debug) Task 91577527-f981-4e23-92c1-44f505cbdb53: moving from state preparing -> state finished
>Thread-15912215::DEBUG::2013-11-20 02:24:00,127::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912215::DEBUG::2013-11-20 02:24:00,127::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912215::DEBUG::2013-11-20 02:24:00,128::task::495::TaskManager.Task::(_debug) Task 91577527-f981-4e23-92c1-44f505cbdb53: ref 0 aborting False
>Thread-15912215::INFO::2013-11-20 02:24:00,129::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912216::DEBUG::2013-11-20 02:24:01,146::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912216::INFO::2013-11-20 02:24:01,156::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912216::DEBUG::2013-11-20 02:24:01,157::task::495::TaskManager.Task::(_debug) Task 3513f87b-a1aa-4ebb-bd35-621250a497d7: moving from state init -> state preparing
>Thread-15912216::DEBUG::2013-11-20 02:24:01,157::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912216::DEBUG::2013-11-20 02:24:01,158::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912216::DEBUG::2013-11-20 02:24:01,158::task::495::TaskManager.Task::(_debug) Task 3513f87b-a1aa-4ebb-bd35-621250a497d7: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912216::DEBUG::2013-11-20 02:24:01,159::task::495::TaskManager.Task::(_debug) Task 3513f87b-a1aa-4ebb-bd35-621250a497d7: moving from state preparing -> state finished
>Thread-15912216::DEBUG::2013-11-20 02:24:01,160::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912216::DEBUG::2013-11-20 02:24:01,160::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912216::DEBUG::2013-11-20 02:24:01,161::task::495::TaskManager.Task::(_debug) Task 3513f87b-a1aa-4ebb-bd35-621250a497d7: ref 0 aborting False
>Thread-15912216::INFO::2013-11-20 02:24:01,161::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912218::DEBUG::2013-11-20 02:24:02,178::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912218::INFO::2013-11-20 02:24:02,179::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912218::DEBUG::2013-11-20 02:24:02,180::task::495::TaskManager.Task::(_debug) Task 1908030a-d4d6-411d-89ac-3c10a465011d: moving from state init -> state preparing
>Thread-15912218::DEBUG::2013-11-20 02:24:02,180::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912218::DEBUG::2013-11-20 02:24:02,181::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912218::DEBUG::2013-11-20 02:24:02,182::task::495::TaskManager.Task::(_debug) Task 1908030a-d4d6-411d-89ac-3c10a465011d: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912218::DEBUG::2013-11-20 02:24:02,182::task::495::TaskManager.Task::(_debug) Task 1908030a-d4d6-411d-89ac-3c10a465011d: moving from state preparing -> state finished
>Thread-15912218::DEBUG::2013-11-20 02:24:02,183::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912218::DEBUG::2013-11-20 02:24:02,183::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912218::DEBUG::2013-11-20 02:24:02,184::task::495::TaskManager.Task::(_debug) Task 1908030a-d4d6-411d-89ac-3c10a465011d: ref 0 aborting False
>Thread-15912218::INFO::2013-11-20 02:24:02,184::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-1024767::ERROR::2013-11-20 02:24:03,091::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912219::DEBUG::2013-11-20 02:24:03,201::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912219::INFO::2013-11-20 02:24:03,202::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912219::DEBUG::2013-11-20 02:24:03,202::task::495::TaskManager.Task::(_debug) Task 81fe65b5-aadf-482f-859f-8f57ecfd6792: moving from state init -> state preparing
>Thread-15912219::DEBUG::2013-11-20 02:24:03,203::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912219::DEBUG::2013-11-20 02:24:03,203::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912219::DEBUG::2013-11-20 02:24:03,204::task::495::TaskManager.Task::(_debug) Task 81fe65b5-aadf-482f-859f-8f57ecfd6792: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912219::DEBUG::2013-11-20 02:24:03,205::task::495::TaskManager.Task::(_debug) Task 81fe65b5-aadf-482f-859f-8f57ecfd6792: moving from state preparing -> state finished
>Thread-15912219::DEBUG::2013-11-20 02:24:03,205::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912219::DEBUG::2013-11-20 02:24:03,206::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912219::DEBUG::2013-11-20 02:24:03,206::task::495::TaskManager.Task::(_debug) Task 81fe65b5-aadf-482f-859f-8f57ecfd6792: ref 0 aborting False
>Thread-15912219::INFO::2013-11-20 02:24:03,207::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912220::DEBUG::2013-11-20 02:24:04,224::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912220::INFO::2013-11-20 02:24:04,224::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912220::DEBUG::2013-11-20 02:24:04,225::task::495::TaskManager.Task::(_debug) Task 59a840ba-3ca0-43e1-a37c-91846f476bca: moving from state init -> state preparing
>Thread-15912220::DEBUG::2013-11-20 02:24:04,226::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912220::DEBUG::2013-11-20 02:24:04,226::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912220::DEBUG::2013-11-20 02:24:04,227::task::495::TaskManager.Task::(_debug) Task 59a840ba-3ca0-43e1-a37c-91846f476bca: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912220::DEBUG::2013-11-20 02:24:04,227::task::495::TaskManager.Task::(_debug) Task 59a840ba-3ca0-43e1-a37c-91846f476bca: moving from state preparing -> state finished
>Thread-15912220::DEBUG::2013-11-20 02:24:04,228::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912220::DEBUG::2013-11-20 02:24:04,229::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912220::DEBUG::2013-11-20 02:24:04,229::task::495::TaskManager.Task::(_debug) Task 59a840ba-3ca0-43e1-a37c-91846f476bca: ref 0 aborting False
>Thread-15912220::INFO::2013-11-20 02:24:04,230::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912221::INFO::2013-11-20 02:24:04,689::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912221::DEBUG::2013-11-20 02:24:04,689::task::495::TaskManager.Task::(_debug) Task 7755a3c3-14e2-4700-9ecd-43b64446c35c: moving from state init -> state preparing
>Thread-15912221::DEBUG::2013-11-20 02:24:04,690::task::495::TaskManager.Task::(_debug) Task 7755a3c3-14e2-4700-9ecd-43b64446c35c: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000368118286133', 'lastCheck': 1384914237.44241, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00153207778931', 'lastCheck': 1384914241.6933701, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384914243.092361, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00133299827576', 'lastCheck': 1384914235.768889, 'valid': True, 'code': 0}}
>Thread-15912221::DEBUG::2013-11-20 02:24:04,696::task::495::TaskManager.Task::(_debug) Task 7755a3c3-14e2-4700-9ecd-43b64446c35c: moving from state preparing -> state finished
>Thread-15912221::DEBUG::2013-11-20 02:24:04,697::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912221::DEBUG::2013-11-20 02:24:04,698::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912221::DEBUG::2013-11-20 02:24:04,698::task::495::TaskManager.Task::(_debug) Task 7755a3c3-14e2-4700-9ecd-43b64446c35c: ref 0 aborting False
>Thread-15912221::INFO::2013-11-20 02:24:04,699::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000368118286133', 'lastCheck': 1384914237.44241, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00153207778931', 'lastCheck': 1384914241.6933701, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384914243.092361, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00133299827576', 'lastCheck': 1384914235.768889, 'valid': True, 'code': 0}}
>Thread-15912223::DEBUG::2013-11-20 02:24:05,247::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912223::INFO::2013-11-20 02:24:05,248::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912223::DEBUG::2013-11-20 02:24:05,248::task::495::TaskManager.Task::(_debug) Task 87bad5af-01e0-43a8-a2f9-596b2caeef03: moving from state init -> state preparing
>Thread-15912223::DEBUG::2013-11-20 02:24:05,249::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912223::DEBUG::2013-11-20 02:24:05,249::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912223::DEBUG::2013-11-20 02:24:05,250::task::495::TaskManager.Task::(_debug) Task 87bad5af-01e0-43a8-a2f9-596b2caeef03: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912223::DEBUG::2013-11-20 02:24:05,250::task::495::TaskManager.Task::(_debug) Task 87bad5af-01e0-43a8-a2f9-596b2caeef03: moving from state preparing -> state finished
>Thread-15912223::DEBUG::2013-11-20 02:24:05,251::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912223::DEBUG::2013-11-20 02:24:05,251::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912223::DEBUG::2013-11-20 02:24:05,252::task::495::TaskManager.Task::(_debug) Task 87bad5af-01e0-43a8-a2f9-596b2caeef03: ref 0 aborting False
>Thread-15912223::INFO::2013-11-20 02:24:05,253::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912224::DEBUG::2013-11-20 02:24:06,269::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912224::INFO::2013-11-20 02:24:06,270::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912224::DEBUG::2013-11-20 02:24:06,270::task::495::TaskManager.Task::(_debug) Task a0e052e6-1fa9-48a2-b39d-3732fbef8a47: moving from state init -> state preparing
>Thread-15912224::DEBUG::2013-11-20 02:24:06,271::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912224::DEBUG::2013-11-20 02:24:06,271::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912224::DEBUG::2013-11-20 02:24:06,272::task::495::TaskManager.Task::(_debug) Task a0e052e6-1fa9-48a2-b39d-3732fbef8a47: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912224::DEBUG::2013-11-20 02:24:06,272::task::495::TaskManager.Task::(_debug) Task a0e052e6-1fa9-48a2-b39d-3732fbef8a47: moving from state preparing -> state finished
>Thread-15912224::DEBUG::2013-11-20 02:24:06,273::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912224::DEBUG::2013-11-20 02:24:06,274::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912224::DEBUG::2013-11-20 02:24:06,274::task::495::TaskManager.Task::(_debug) Task a0e052e6-1fa9-48a2-b39d-3732fbef8a47: ref 0 aborting False
>Thread-15912224::INFO::2013-11-20 02:24:06,275::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912225::DEBUG::2013-11-20 02:24:07,291::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912225::INFO::2013-11-20 02:24:07,292::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912225::DEBUG::2013-11-20 02:24:07,293::task::495::TaskManager.Task::(_debug) Task b8ed8649-6aff-42f2-b4ea-61f7270352c3: moving from state init -> state preparing
>Thread-15912225::DEBUG::2013-11-20 02:24:07,293::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912225::DEBUG::2013-11-20 02:24:07,294::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912225::DEBUG::2013-11-20 02:24:07,295::task::495::TaskManager.Task::(_debug) Task b8ed8649-6aff-42f2-b4ea-61f7270352c3: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912225::DEBUG::2013-11-20 02:24:07,295::task::495::TaskManager.Task::(_debug) Task b8ed8649-6aff-42f2-b4ea-61f7270352c3: moving from state preparing -> state finished
>Thread-15912225::DEBUG::2013-11-20 02:24:07,296::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912225::DEBUG::2013-11-20 02:24:07,296::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912225::DEBUG::2013-11-20 02:24:07,297::task::495::TaskManager.Task::(_debug) Task b8ed8649-6aff-42f2-b4ea-61f7270352c3: ref 0 aborting False
>Thread-15912225::INFO::2013-11-20 02:24:07,297::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912227::DEBUG::2013-11-20 02:24:08,370::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912227::INFO::2013-11-20 02:24:08,371::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912227::DEBUG::2013-11-20 02:24:08,371::task::495::TaskManager.Task::(_debug) Task 48fbbb8c-6324-49e6-b1c9-3e98e67127d9: moving from state init -> state preparing
>Thread-15912227::DEBUG::2013-11-20 02:24:08,372::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912227::DEBUG::2013-11-20 02:24:08,373::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912227::DEBUG::2013-11-20 02:24:08,373::task::495::TaskManager.Task::(_debug) Task 48fbbb8c-6324-49e6-b1c9-3e98e67127d9: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912227::DEBUG::2013-11-20 02:24:08,374::task::495::TaskManager.Task::(_debug) Task 48fbbb8c-6324-49e6-b1c9-3e98e67127d9: moving from state preparing -> state finished
>Thread-15912227::DEBUG::2013-11-20 02:24:08,374::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912227::DEBUG::2013-11-20 02:24:08,375::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912227::DEBUG::2013-11-20 02:24:08,375::task::495::TaskManager.Task::(_debug) Task 48fbbb8c-6324-49e6-b1c9-3e98e67127d9: ref 0 aborting False
>Thread-15912227::INFO::2013-11-20 02:24:08,376::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912228::DEBUG::2013-11-20 02:24:09,392::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912228::INFO::2013-11-20 02:24:09,393::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912228::DEBUG::2013-11-20 02:24:09,394::task::495::TaskManager.Task::(_debug) Task 04aecaaa-e288-4a00-9539-77f459f57cca: moving from state init -> state preparing
>Thread-15912228::DEBUG::2013-11-20 02:24:09,394::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912228::DEBUG::2013-11-20 02:24:09,395::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912228::DEBUG::2013-11-20 02:24:09,396::task::495::TaskManager.Task::(_debug) Task 04aecaaa-e288-4a00-9539-77f459f57cca: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912228::DEBUG::2013-11-20 02:24:09,396::task::495::TaskManager.Task::(_debug) Task 04aecaaa-e288-4a00-9539-77f459f57cca: moving from state preparing -> state finished
>Thread-15912228::DEBUG::2013-11-20 02:24:09,397::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912228::DEBUG::2013-11-20 02:24:09,397::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912228::DEBUG::2013-11-20 02:24:09,398::task::495::TaskManager.Task::(_debug) Task 04aecaaa-e288-4a00-9539-77f459f57cca: ref 0 aborting False
>Thread-15912228::INFO::2013-11-20 02:24:09,399::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912229::DEBUG::2013-11-20 02:24:10,426::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912229::INFO::2013-11-20 02:24:10,427::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912229::DEBUG::2013-11-20 02:24:10,427::task::495::TaskManager.Task::(_debug) Task 615bafca-d21d-4994-aa7f-514b30a33af2: moving from state init -> state preparing
>Thread-15912229::DEBUG::2013-11-20 02:24:10,428::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912229::DEBUG::2013-11-20 02:24:10,429::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912229::DEBUG::2013-11-20 02:24:10,429::task::495::TaskManager.Task::(_debug) Task 615bafca-d21d-4994-aa7f-514b30a33af2: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912229::DEBUG::2013-11-20 02:24:10,430::task::495::TaskManager.Task::(_debug) Task 615bafca-d21d-4994-aa7f-514b30a33af2: moving from state preparing -> state finished
>Thread-15912229::DEBUG::2013-11-20 02:24:10,430::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912229::DEBUG::2013-11-20 02:24:10,431::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912229::DEBUG::2013-11-20 02:24:10,431::task::495::TaskManager.Task::(_debug) Task 615bafca-d21d-4994-aa7f-514b30a33af2: ref 0 aborting False
>Thread-15912229::INFO::2013-11-20 02:24:10,432::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912231::DEBUG::2013-11-20 02:24:11,448::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912231::INFO::2013-11-20 02:24:11,449::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912231::DEBUG::2013-11-20 02:24:11,462::task::495::TaskManager.Task::(_debug) Task 471c6f10-0654-4081-8f65-054754b14c6d: moving from state init -> state preparing
>Thread-15912231::DEBUG::2013-11-20 02:24:11,463::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912231::DEBUG::2013-11-20 02:24:11,464::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912231::DEBUG::2013-11-20 02:24:11,464::task::495::TaskManager.Task::(_debug) Task 471c6f10-0654-4081-8f65-054754b14c6d: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912231::DEBUG::2013-11-20 02:24:11,465::task::495::TaskManager.Task::(_debug) Task 471c6f10-0654-4081-8f65-054754b14c6d: moving from state preparing -> state finished
>Thread-15912231::DEBUG::2013-11-20 02:24:11,465::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912231::DEBUG::2013-11-20 02:24:11,466::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912231::DEBUG::2013-11-20 02:24:11,466::task::495::TaskManager.Task::(_debug) Task 471c6f10-0654-4081-8f65-054754b14c6d: ref 0 aborting False
>Thread-15912231::INFO::2013-11-20 02:24:11,467::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912232::DEBUG::2013-11-20 02:24:12,483::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912232::INFO::2013-11-20 02:24:12,484::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912232::DEBUG::2013-11-20 02:24:12,485::task::495::TaskManager.Task::(_debug) Task 61e4d095-8d63-4c8a-9b0f-5a4b7d83048d: moving from state init -> state preparing
>Thread-15912232::DEBUG::2013-11-20 02:24:12,485::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912232::DEBUG::2013-11-20 02:24:12,486::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912232::DEBUG::2013-11-20 02:24:12,486::task::495::TaskManager.Task::(_debug) Task 61e4d095-8d63-4c8a-9b0f-5a4b7d83048d: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912232::DEBUG::2013-11-20 02:24:12,487::task::495::TaskManager.Task::(_debug) Task 61e4d095-8d63-4c8a-9b0f-5a4b7d83048d: moving from state preparing -> state finished
>Thread-15912232::DEBUG::2013-11-20 02:24:12,487::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912232::DEBUG::2013-11-20 02:24:12,488::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912232::DEBUG::2013-11-20 02:24:12,489::task::495::TaskManager.Task::(_debug) Task 61e4d095-8d63-4c8a-9b0f-5a4b7d83048d: ref 0 aborting False
>Thread-15912232::INFO::2013-11-20 02:24:12,489::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-1024767::ERROR::2013-11-20 02:24:13,104::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912233::DEBUG::2013-11-20 02:24:13,506::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912233::INFO::2013-11-20 02:24:13,507::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912233::DEBUG::2013-11-20 02:24:13,507::task::495::TaskManager.Task::(_debug) Task 0360ec9e-a963-482a-8133-771bbb8897b4: moving from state init -> state preparing
>Thread-15912233::DEBUG::2013-11-20 02:24:13,508::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912233::DEBUG::2013-11-20 02:24:13,508::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912233::DEBUG::2013-11-20 02:24:13,509::task::495::TaskManager.Task::(_debug) Task 0360ec9e-a963-482a-8133-771bbb8897b4: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912233::DEBUG::2013-11-20 02:24:13,510::task::495::TaskManager.Task::(_debug) Task 0360ec9e-a963-482a-8133-771bbb8897b4: moving from state preparing -> state finished
>Thread-15912233::DEBUG::2013-11-20 02:24:13,510::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912233::DEBUG::2013-11-20 02:24:13,511::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912233::DEBUG::2013-11-20 02:24:13,511::task::495::TaskManager.Task::(_debug) Task 0360ec9e-a963-482a-8133-771bbb8897b4: ref 0 aborting False
>Thread-15912233::INFO::2013-11-20 02:24:13,512::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912235::DEBUG::2013-11-20 02:24:14,528::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912235::INFO::2013-11-20 02:24:14,528::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912235::DEBUG::2013-11-20 02:24:14,529::task::495::TaskManager.Task::(_debug) Task 033d1e7f-1f8f-4382-93a3-200bdf508f3b: moving from state init -> state preparing
>Thread-15912235::DEBUG::2013-11-20 02:24:14,530::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912235::DEBUG::2013-11-20 02:24:14,530::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912235::DEBUG::2013-11-20 02:24:14,531::task::495::TaskManager.Task::(_debug) Task 033d1e7f-1f8f-4382-93a3-200bdf508f3b: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912235::DEBUG::2013-11-20 02:24:14,531::task::495::TaskManager.Task::(_debug) Task 033d1e7f-1f8f-4382-93a3-200bdf508f3b: moving from state preparing -> state finished
>Thread-15912235::DEBUG::2013-11-20 02:24:14,532::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912235::DEBUG::2013-11-20 02:24:14,532::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912235::DEBUG::2013-11-20 02:24:14,533::task::495::TaskManager.Task::(_debug) Task 033d1e7f-1f8f-4382-93a3-200bdf508f3b: ref 0 aborting False
>Thread-15912235::INFO::2013-11-20 02:24:14,534::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912236::DEBUG::2013-11-20 02:24:15,550::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912236::INFO::2013-11-20 02:24:15,551::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912236::DEBUG::2013-11-20 02:24:15,552::task::495::TaskManager.Task::(_debug) Task 0ba1c62a-efb8-4c6b-851a-eccd49e770f6: moving from state init -> state preparing
>Thread-15912236::DEBUG::2013-11-20 02:24:15,552::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912236::DEBUG::2013-11-20 02:24:15,553::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912236::DEBUG::2013-11-20 02:24:15,553::task::495::TaskManager.Task::(_debug) Task 0ba1c62a-efb8-4c6b-851a-eccd49e770f6: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912236::DEBUG::2013-11-20 02:24:15,554::task::495::TaskManager.Task::(_debug) Task 0ba1c62a-efb8-4c6b-851a-eccd49e770f6: moving from state preparing -> state finished
>Thread-15912236::DEBUG::2013-11-20 02:24:15,554::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912236::DEBUG::2013-11-20 02:24:15,555::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912236::DEBUG::2013-11-20 02:24:15,556::task::495::TaskManager.Task::(_debug) Task 0ba1c62a-efb8-4c6b-851a-eccd49e770f6: ref 0 aborting False
>Thread-15912236::INFO::2013-11-20 02:24:15,556::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912237::DEBUG::2013-11-20 02:24:16,572::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912237::INFO::2013-11-20 02:24:16,573::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912237::DEBUG::2013-11-20 02:24:16,583::task::495::TaskManager.Task::(_debug) Task b8c87626-2967-46dd-b7c0-130ac7b73b0d: moving from state init -> state preparing
>Thread-15912237::DEBUG::2013-11-20 02:24:16,584::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912237::DEBUG::2013-11-20 02:24:16,584::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912237::DEBUG::2013-11-20 02:24:16,585::task::495::TaskManager.Task::(_debug) Task b8c87626-2967-46dd-b7c0-130ac7b73b0d: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912237::DEBUG::2013-11-20 02:24:16,586::task::495::TaskManager.Task::(_debug) Task b8c87626-2967-46dd-b7c0-130ac7b73b0d: moving from state preparing -> state finished
>Thread-15912237::DEBUG::2013-11-20 02:24:16,586::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912237::DEBUG::2013-11-20 02:24:16,587::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912237::DEBUG::2013-11-20 02:24:16,587::task::495::TaskManager.Task::(_debug) Task b8c87626-2967-46dd-b7c0-130ac7b73b0d: ref 0 aborting False
>Thread-15912237::INFO::2013-11-20 02:24:16,588::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912239::DEBUG::2013-11-20 02:24:17,604::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912239::INFO::2013-11-20 02:24:17,605::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912239::DEBUG::2013-11-20 02:24:17,605::task::495::TaskManager.Task::(_debug) Task 0e607d07-16fe-4918-81d4-d82bf9004ff5: moving from state init -> state preparing
>Thread-15912239::DEBUG::2013-11-20 02:24:17,606::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912239::DEBUG::2013-11-20 02:24:17,606::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912239::DEBUG::2013-11-20 02:24:17,607::task::495::TaskManager.Task::(_debug) Task 0e607d07-16fe-4918-81d4-d82bf9004ff5: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912239::DEBUG::2013-11-20 02:24:17,608::task::495::TaskManager.Task::(_debug) Task 0e607d07-16fe-4918-81d4-d82bf9004ff5: moving from state preparing -> state finished
>Thread-15912239::DEBUG::2013-11-20 02:24:17,608::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912239::DEBUG::2013-11-20 02:24:17,609::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912239::DEBUG::2013-11-20 02:24:17,609::task::495::TaskManager.Task::(_debug) Task 0e607d07-16fe-4918-81d4-d82bf9004ff5: ref 0 aborting False
>Thread-15912239::INFO::2013-11-20 02:24:17,610::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912240::DEBUG::2013-11-20 02:24:18,626::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912240::INFO::2013-11-20 02:24:18,627::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912240::DEBUG::2013-11-20 02:24:18,627::task::495::TaskManager.Task::(_debug) Task 82255666-02c8-4863-a52b-19f8789a9ff2: moving from state init -> state preparing
>Thread-15912240::DEBUG::2013-11-20 02:24:18,628::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912240::DEBUG::2013-11-20 02:24:18,629::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912240::DEBUG::2013-11-20 02:24:18,629::task::495::TaskManager.Task::(_debug) Task 82255666-02c8-4863-a52b-19f8789a9ff2: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912240::DEBUG::2013-11-20 02:24:18,630::task::495::TaskManager.Task::(_debug) Task 82255666-02c8-4863-a52b-19f8789a9ff2: moving from state preparing -> state finished
>Thread-15912240::DEBUG::2013-11-20 02:24:18,630::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912240::DEBUG::2013-11-20 02:24:18,631::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912240::DEBUG::2013-11-20 02:24:18,631::task::495::TaskManager.Task::(_debug) Task 82255666-02c8-4863-a52b-19f8789a9ff2: ref 0 aborting False
>Thread-15912240::INFO::2013-11-20 02:24:18,632::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912241::DEBUG::2013-11-20 02:24:19,648::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912241::INFO::2013-11-20 02:24:19,649::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912241::DEBUG::2013-11-20 02:24:19,650::task::495::TaskManager.Task::(_debug) Task 2e5c3fef-f318-486b-a5d6-9752ae9db30a: moving from state init -> state preparing
>Thread-15912241::DEBUG::2013-11-20 02:24:19,650::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912241::DEBUG::2013-11-20 02:24:19,651::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912241::DEBUG::2013-11-20 02:24:19,651::task::495::TaskManager.Task::(_debug) Task 2e5c3fef-f318-486b-a5d6-9752ae9db30a: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912241::DEBUG::2013-11-20 02:24:19,652::task::495::TaskManager.Task::(_debug) Task 2e5c3fef-f318-486b-a5d6-9752ae9db30a: moving from state preparing -> state finished
>Thread-15912241::DEBUG::2013-11-20 02:24:19,653::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912241::DEBUG::2013-11-20 02:24:19,653::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912241::DEBUG::2013-11-20 02:24:19,654::task::495::TaskManager.Task::(_debug) Task 2e5c3fef-f318-486b-a5d6-9752ae9db30a: ref 0 aborting False
>Thread-15912241::INFO::2013-11-20 02:24:19,654::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912242::INFO::2013-11-20 02:24:19,974::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912242::DEBUG::2013-11-20 02:24:19,975::task::495::TaskManager.Task::(_debug) Task 89d49a38-d7dc-46a0-9765-24868f00ee0f: moving from state init -> state preparing
>Thread-15912242::DEBUG::2013-11-20 02:24:19,976::task::495::TaskManager.Task::(_debug) Task 89d49a38-d7dc-46a0-9765-24868f00ee0f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000293970108032', 'lastCheck': 1384914257.467984, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00145292282104', 'lastCheck': 1384914251.7094791, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384914253.1056781, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000391006469727', 'lastCheck': 1384914255.791477, 'valid': True, 'code': 0}}
>Thread-15912242::DEBUG::2013-11-20 02:24:19,992::task::495::TaskManager.Task::(_debug) Task 89d49a38-d7dc-46a0-9765-24868f00ee0f: moving from state preparing -> state finished
>Thread-15912242::DEBUG::2013-11-20 02:24:19,993::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912242::DEBUG::2013-11-20 02:24:19,993::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912242::DEBUG::2013-11-20 02:24:19,994::task::495::TaskManager.Task::(_debug) Task 89d49a38-d7dc-46a0-9765-24868f00ee0f: ref 0 aborting False
>Thread-15912242::INFO::2013-11-20 02:24:19,995::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000293970108032', 'lastCheck': 1384914257.467984, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00145292282104', 'lastCheck': 1384914251.7094791, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384914253.1056781, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000391006469727', 'lastCheck': 1384914255.791477, 'valid': True, 'code': 0}}
>Thread-15912244::DEBUG::2013-11-20 02:24:20,671::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912244::INFO::2013-11-20 02:24:20,671::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912244::DEBUG::2013-11-20 02:24:20,672::task::495::TaskManager.Task::(_debug) Task fe2e264c-5938-4ca3-a40e-34fe3209d985: moving from state init -> state preparing
>Thread-15912244::DEBUG::2013-11-20 02:24:20,673::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912244::DEBUG::2013-11-20 02:24:20,673::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912244::DEBUG::2013-11-20 02:24:20,674::task::495::TaskManager.Task::(_debug) Task fe2e264c-5938-4ca3-a40e-34fe3209d985: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912244::DEBUG::2013-11-20 02:24:20,674::task::495::TaskManager.Task::(_debug) Task fe2e264c-5938-4ca3-a40e-34fe3209d985: moving from state preparing -> state finished
>Thread-15912244::DEBUG::2013-11-20 02:24:20,675::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912244::DEBUG::2013-11-20 02:24:20,675::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912244::DEBUG::2013-11-20 02:24:20,676::task::495::TaskManager.Task::(_debug) Task fe2e264c-5938-4ca3-a40e-34fe3209d985: ref 0 aborting False
>Thread-15912244::INFO::2013-11-20 02:24:20,676::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912245::DEBUG::2013-11-20 02:24:21,692::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912245::INFO::2013-11-20 02:24:21,693::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912245::DEBUG::2013-11-20 02:24:21,704::task::495::TaskManager.Task::(_debug) Task dddc1e7b-e597-46b6-bb77-512fbe36fa10: moving from state init -> state preparing
>Thread-15912245::DEBUG::2013-11-20 02:24:21,704::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912245::DEBUG::2013-11-20 02:24:21,705::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912245::DEBUG::2013-11-20 02:24:21,705::task::495::TaskManager.Task::(_debug) Task dddc1e7b-e597-46b6-bb77-512fbe36fa10: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912245::DEBUG::2013-11-20 02:24:21,706::task::495::TaskManager.Task::(_debug) Task dddc1e7b-e597-46b6-bb77-512fbe36fa10: moving from state preparing -> state finished
>Thread-15912245::DEBUG::2013-11-20 02:24:21,707::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912245::DEBUG::2013-11-20 02:24:21,707::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912245::DEBUG::2013-11-20 02:24:21,708::task::495::TaskManager.Task::(_debug) Task dddc1e7b-e597-46b6-bb77-512fbe36fa10: ref 0 aborting False
>Thread-15912245::INFO::2013-11-20 02:24:21,708::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912246::DEBUG::2013-11-20 02:24:22,724::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912246::INFO::2013-11-20 02:24:22,725::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912246::DEBUG::2013-11-20 02:24:22,725::task::495::TaskManager.Task::(_debug) Task 0a03f74d-9698-4f19-871b-9ea76ff3db07: moving from state init -> state preparing
>Thread-15912246::DEBUG::2013-11-20 02:24:22,726::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912246::DEBUG::2013-11-20 02:24:22,727::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912246::DEBUG::2013-11-20 02:24:22,727::task::495::TaskManager.Task::(_debug) Task 0a03f74d-9698-4f19-871b-9ea76ff3db07: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912246::DEBUG::2013-11-20 02:24:22,728::task::495::TaskManager.Task::(_debug) Task 0a03f74d-9698-4f19-871b-9ea76ff3db07: moving from state preparing -> state finished
>Thread-15912246::DEBUG::2013-11-20 02:24:22,729::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912246::DEBUG::2013-11-20 02:24:22,729::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912246::DEBUG::2013-11-20 02:24:22,730::task::495::TaskManager.Task::(_debug) Task 0a03f74d-9698-4f19-871b-9ea76ff3db07: ref 0 aborting False
>Thread-15912246::INFO::2013-11-20 02:24:22,730::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-1024767::ERROR::2013-11-20 02:24:23,118::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912248::DEBUG::2013-11-20 02:24:23,747::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912248::INFO::2013-11-20 02:24:23,748::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912248::DEBUG::2013-11-20 02:24:23,749::task::495::TaskManager.Task::(_debug) Task 12ce0d53-c02f-402a-88e1-5d82b6d3f605: moving from state init -> state preparing
>Thread-15912248::DEBUG::2013-11-20 02:24:23,749::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912248::DEBUG::2013-11-20 02:24:23,750::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912248::DEBUG::2013-11-20 02:24:23,750::task::495::TaskManager.Task::(_debug) Task 12ce0d53-c02f-402a-88e1-5d82b6d3f605: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912248::DEBUG::2013-11-20 02:24:23,751::task::495::TaskManager.Task::(_debug) Task 12ce0d53-c02f-402a-88e1-5d82b6d3f605: moving from state preparing -> state finished
>Thread-15912248::DEBUG::2013-11-20 02:24:23,752::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912248::DEBUG::2013-11-20 02:24:23,752::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912248::DEBUG::2013-11-20 02:24:23,753::task::495::TaskManager.Task::(_debug) Task 12ce0d53-c02f-402a-88e1-5d82b6d3f605: ref 0 aborting False
>Thread-15912248::INFO::2013-11-20 02:24:23,753::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912249::DEBUG::2013-11-20 02:24:24,770::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912249::INFO::2013-11-20 02:24:24,771::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912249::DEBUG::2013-11-20 02:24:24,771::task::495::TaskManager.Task::(_debug) Task 48028d05-bb67-4182-99f3-750d218cd0b2: moving from state init -> state preparing
>Thread-15912249::DEBUG::2013-11-20 02:24:24,772::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912249::DEBUG::2013-11-20 02:24:24,772::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912249::DEBUG::2013-11-20 02:24:24,773::task::495::TaskManager.Task::(_debug) Task 48028d05-bb67-4182-99f3-750d218cd0b2: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912249::DEBUG::2013-11-20 02:24:24,774::task::495::TaskManager.Task::(_debug) Task 48028d05-bb67-4182-99f3-750d218cd0b2: moving from state preparing -> state finished
>Thread-15912249::DEBUG::2013-11-20 02:24:24,774::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912249::DEBUG::2013-11-20 02:24:24,775::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912249::DEBUG::2013-11-20 02:24:24,775::task::495::TaskManager.Task::(_debug) Task 48028d05-bb67-4182-99f3-750d218cd0b2: ref 0 aborting False
>Thread-15912249::INFO::2013-11-20 02:24:24,776::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912250::DEBUG::2013-11-20 02:24:25,792::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912250::INFO::2013-11-20 02:24:25,793::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912250::DEBUG::2013-11-20 02:24:25,793::task::495::TaskManager.Task::(_debug) Task c4f9e3b8-ed14-48d8-8bc1-fb41bb114084: moving from state init -> state preparing
>Thread-15912250::DEBUG::2013-11-20 02:24:25,794::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912250::DEBUG::2013-11-20 02:24:25,794::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912250::DEBUG::2013-11-20 02:24:25,795::task::495::TaskManager.Task::(_debug) Task c4f9e3b8-ed14-48d8-8bc1-fb41bb114084: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912250::DEBUG::2013-11-20 02:24:25,796::task::495::TaskManager.Task::(_debug) Task c4f9e3b8-ed14-48d8-8bc1-fb41bb114084: moving from state preparing -> state finished
>Thread-15912250::DEBUG::2013-11-20 02:24:25,796::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912250::DEBUG::2013-11-20 02:24:25,797::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912250::DEBUG::2013-11-20 02:24:25,797::task::495::TaskManager.Task::(_debug) Task c4f9e3b8-ed14-48d8-8bc1-fb41bb114084: ref 0 aborting False
>Thread-15912250::INFO::2013-11-20 02:24:25,798::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912252::DEBUG::2013-11-20 02:24:26,814::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912252::INFO::2013-11-20 02:24:26,815::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912252::DEBUG::2013-11-20 02:24:26,826::task::495::TaskManager.Task::(_debug) Task d09f706e-0dd7-40e5-8905-4e653cd32c5b: moving from state init -> state preparing
>Thread-15912252::DEBUG::2013-11-20 02:24:26,826::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912252::DEBUG::2013-11-20 02:24:26,827::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912252::DEBUG::2013-11-20 02:24:26,828::task::495::TaskManager.Task::(_debug) Task d09f706e-0dd7-40e5-8905-4e653cd32c5b: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912252::DEBUG::2013-11-20 02:24:26,828::task::495::TaskManager.Task::(_debug) Task d09f706e-0dd7-40e5-8905-4e653cd32c5b: moving from state preparing -> state finished
>Thread-15912252::DEBUG::2013-11-20 02:24:26,829::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912252::DEBUG::2013-11-20 02:24:26,829::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912252::DEBUG::2013-11-20 02:24:26,830::task::495::TaskManager.Task::(_debug) Task d09f706e-0dd7-40e5-8905-4e653cd32c5b: ref 0 aborting False
>Thread-15912252::INFO::2013-11-20 02:24:26,830::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912253::DEBUG::2013-11-20 02:24:27,846::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912253::INFO::2013-11-20 02:24:27,847::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912253::DEBUG::2013-11-20 02:24:27,848::task::495::TaskManager.Task::(_debug) Task 894a6175-4f1a-45b8-b2f9-5dddb3442783: moving from state init -> state preparing
>Thread-15912253::DEBUG::2013-11-20 02:24:27,848::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912253::DEBUG::2013-11-20 02:24:27,849::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912253::DEBUG::2013-11-20 02:24:27,850::task::495::TaskManager.Task::(_debug) Task 894a6175-4f1a-45b8-b2f9-5dddb3442783: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912253::DEBUG::2013-11-20 02:24:27,850::task::495::TaskManager.Task::(_debug) Task 894a6175-4f1a-45b8-b2f9-5dddb3442783: moving from state preparing -> state finished
>Thread-15912253::DEBUG::2013-11-20 02:24:27,851::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912253::DEBUG::2013-11-20 02:24:27,851::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912253::DEBUG::2013-11-20 02:24:27,852::task::495::TaskManager.Task::(_debug) Task 894a6175-4f1a-45b8-b2f9-5dddb3442783: ref 0 aborting False
>Thread-15912253::INFO::2013-11-20 02:24:27,852::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912254::DEBUG::2013-11-20 02:24:28,870::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912254::INFO::2013-11-20 02:24:28,871::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912254::DEBUG::2013-11-20 02:24:28,872::task::495::TaskManager.Task::(_debug) Task a66c3c2b-6890-48c3-917d-d6c0e80a9222: moving from state init -> state preparing
>Thread-15912254::DEBUG::2013-11-20 02:24:28,872::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912254::DEBUG::2013-11-20 02:24:28,873::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912254::DEBUG::2013-11-20 02:24:28,874::task::495::TaskManager.Task::(_debug) Task a66c3c2b-6890-48c3-917d-d6c0e80a9222: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912254::DEBUG::2013-11-20 02:24:28,874::task::495::TaskManager.Task::(_debug) Task a66c3c2b-6890-48c3-917d-d6c0e80a9222: moving from state preparing -> state finished
>Thread-15912254::DEBUG::2013-11-20 02:24:28,875::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912254::DEBUG::2013-11-20 02:24:28,875::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912254::DEBUG::2013-11-20 02:24:28,878::task::495::TaskManager.Task::(_debug) Task a66c3c2b-6890-48c3-917d-d6c0e80a9222: ref 0 aborting False
>Thread-15912254::INFO::2013-11-20 02:24:28,880::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912256::DEBUG::2013-11-20 02:24:29,898::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912256::INFO::2013-11-20 02:24:29,899::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912256::DEBUG::2013-11-20 02:24:29,900::task::495::TaskManager.Task::(_debug) Task 49549051-9435-44b8-b3a7-ec774e6978a6: moving from state init -> state preparing
>Thread-15912256::DEBUG::2013-11-20 02:24:29,900::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912256::DEBUG::2013-11-20 02:24:29,901::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912256::DEBUG::2013-11-20 02:24:29,902::task::495::TaskManager.Task::(_debug) Task 49549051-9435-44b8-b3a7-ec774e6978a6: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912256::DEBUG::2013-11-20 02:24:29,902::task::495::TaskManager.Task::(_debug) Task 49549051-9435-44b8-b3a7-ec774e6978a6: moving from state preparing -> state finished
>Thread-15912256::DEBUG::2013-11-20 02:24:29,903::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912256::DEBUG::2013-11-20 02:24:29,903::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912256::DEBUG::2013-11-20 02:24:29,904::task::495::TaskManager.Task::(_debug) Task 49549051-9435-44b8-b3a7-ec774e6978a6: ref 0 aborting False
>Thread-15912256::INFO::2013-11-20 02:24:29,904::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912257::DEBUG::2013-11-20 02:24:30,921::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912257::INFO::2013-11-20 02:24:30,922::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912257::DEBUG::2013-11-20 02:24:30,922::task::495::TaskManager.Task::(_debug) Task 2b1921fc-c669-41d4-9411-be82d1e09264: moving from state init -> state preparing
>Thread-15912257::DEBUG::2013-11-20 02:24:30,923::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912257::DEBUG::2013-11-20 02:24:30,923::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912257::DEBUG::2013-11-20 02:24:30,924::task::495::TaskManager.Task::(_debug) Task 2b1921fc-c669-41d4-9411-be82d1e09264: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912257::DEBUG::2013-11-20 02:24:30,925::task::495::TaskManager.Task::(_debug) Task 2b1921fc-c669-41d4-9411-be82d1e09264: moving from state preparing -> state finished
>Thread-15912257::DEBUG::2013-11-20 02:24:30,925::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912257::DEBUG::2013-11-20 02:24:30,926::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912257::DEBUG::2013-11-20 02:24:30,926::task::495::TaskManager.Task::(_debug) Task 2b1921fc-c669-41d4-9411-be82d1e09264: ref 0 aborting False
>Thread-15912257::INFO::2013-11-20 02:24:30,927::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912258::DEBUG::2013-11-20 02:24:31,942::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912258::INFO::2013-11-20 02:24:31,943::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912258::DEBUG::2013-11-20 02:24:31,955::task::495::TaskManager.Task::(_debug) Task 6c067dae-f0c0-4ffd-bf43-7f540bcc99e2: moving from state init -> state preparing
>Thread-15912258::DEBUG::2013-11-20 02:24:31,956::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912258::DEBUG::2013-11-20 02:24:31,956::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912258::DEBUG::2013-11-20 02:24:31,957::task::495::TaskManager.Task::(_debug) Task 6c067dae-f0c0-4ffd-bf43-7f540bcc99e2: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912258::DEBUG::2013-11-20 02:24:31,957::task::495::TaskManager.Task::(_debug) Task 6c067dae-f0c0-4ffd-bf43-7f540bcc99e2: moving from state preparing -> state finished
>Thread-15912258::DEBUG::2013-11-20 02:24:31,958::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912258::DEBUG::2013-11-20 02:24:31,958::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912258::DEBUG::2013-11-20 02:24:31,959::task::495::TaskManager.Task::(_debug) Task 6c067dae-f0c0-4ffd-bf43-7f540bcc99e2: ref 0 aborting False
>Thread-15912258::INFO::2013-11-20 02:24:31,959::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912260::DEBUG::2013-11-20 02:24:32,975::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912260::INFO::2013-11-20 02:24:32,976::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912260::DEBUG::2013-11-20 02:24:32,976::task::495::TaskManager.Task::(_debug) Task 6840ef92-ae04-4583-81b6-7a9dcc8792cd: moving from state init -> state preparing
>Thread-15912260::DEBUG::2013-11-20 02:24:32,977::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912260::DEBUG::2013-11-20 02:24:32,977::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912260::DEBUG::2013-11-20 02:24:32,978::task::495::TaskManager.Task::(_debug) Task 6840ef92-ae04-4583-81b6-7a9dcc8792cd: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912260::DEBUG::2013-11-20 02:24:32,979::task::495::TaskManager.Task::(_debug) Task 6840ef92-ae04-4583-81b6-7a9dcc8792cd: moving from state preparing -> state finished
>Thread-15912260::DEBUG::2013-11-20 02:24:32,979::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912260::DEBUG::2013-11-20 02:24:32,980::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912260::DEBUG::2013-11-20 02:24:32,980::task::495::TaskManager.Task::(_debug) Task 6840ef92-ae04-4583-81b6-7a9dcc8792cd: ref 0 aborting False
>Thread-15912260::INFO::2013-11-20 02:24:32,981::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-1024767::ERROR::2013-11-20 02:24:33,131::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912261::DEBUG::2013-11-20 02:24:34,000::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912261::INFO::2013-11-20 02:24:34,001::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912261::DEBUG::2013-11-20 02:24:34,002::task::495::TaskManager.Task::(_debug) Task 6bcdc9fd-29e7-4ba8-987a-5ee632d1ecf7: moving from state init -> state preparing
>Thread-15912261::DEBUG::2013-11-20 02:24:34,002::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912261::DEBUG::2013-11-20 02:24:34,003::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912261::DEBUG::2013-11-20 02:24:34,004::task::495::TaskManager.Task::(_debug) Task 6bcdc9fd-29e7-4ba8-987a-5ee632d1ecf7: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912261::DEBUG::2013-11-20 02:24:34,004::task::495::TaskManager.Task::(_debug) Task 6bcdc9fd-29e7-4ba8-987a-5ee632d1ecf7: moving from state preparing -> state finished
>Thread-15912261::DEBUG::2013-11-20 02:24:34,005::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912261::DEBUG::2013-11-20 02:24:34,005::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912261::DEBUG::2013-11-20 02:24:34,006::task::495::TaskManager.Task::(_debug) Task 6bcdc9fd-29e7-4ba8-987a-5ee632d1ecf7: ref 0 aborting False
>Thread-15912261::INFO::2013-11-20 02:24:34,006::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912262::DEBUG::2013-11-20 02:24:35,022::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912262::INFO::2013-11-20 02:24:35,023::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912262::DEBUG::2013-11-20 02:24:35,024::task::495::TaskManager.Task::(_debug) Task 309ac11c-8b36-4a27-b512-b5b09bb83254: moving from state init -> state preparing
>Thread-15912262::DEBUG::2013-11-20 02:24:35,024::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912262::DEBUG::2013-11-20 02:24:35,025::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912262::DEBUG::2013-11-20 02:24:35,025::task::495::TaskManager.Task::(_debug) Task 309ac11c-8b36-4a27-b512-b5b09bb83254: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912262::DEBUG::2013-11-20 02:24:35,026::task::495::TaskManager.Task::(_debug) Task 309ac11c-8b36-4a27-b512-b5b09bb83254: moving from state preparing -> state finished
>Thread-15912262::DEBUG::2013-11-20 02:24:35,027::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912262::DEBUG::2013-11-20 02:24:35,027::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912262::DEBUG::2013-11-20 02:24:35,028::task::495::TaskManager.Task::(_debug) Task 309ac11c-8b36-4a27-b512-b5b09bb83254: ref 0 aborting False
>Thread-15912262::INFO::2013-11-20 02:24:35,028::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912263::INFO::2013-11-20 02:24:35,214::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912263::DEBUG::2013-11-20 02:24:35,214::task::495::TaskManager.Task::(_debug) Task 65e28958-3e92-4ea9-b539-3264a396bb93: moving from state init -> state preparing
>Thread-15912263::DEBUG::2013-11-20 02:24:35,215::task::495::TaskManager.Task::(_debug) Task 65e28958-3e92-4ea9-b539-3264a396bb93: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000334024429321', 'lastCheck': 1384914267.480865, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00227499008179', 'lastCheck': 1384914271.7418101, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384914273.132628, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000362157821655', 'lastCheck': 1384914265.8027561, 'valid': True, 'code': 0}}
>Thread-15912263::DEBUG::2013-11-20 02:24:35,219::task::495::TaskManager.Task::(_debug) Task 65e28958-3e92-4ea9-b539-3264a396bb93: moving from state preparing -> state finished
>Thread-15912263::DEBUG::2013-11-20 02:24:35,219::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912263::DEBUG::2013-11-20 02:24:35,220::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912263::DEBUG::2013-11-20 02:24:35,221::task::495::TaskManager.Task::(_debug) Task 65e28958-3e92-4ea9-b539-3264a396bb93: ref 0 aborting False
>Thread-15912263::INFO::2013-11-20 02:24:35,221::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000334024429321', 'lastCheck': 1384914267.480865, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00227499008179', 'lastCheck': 1384914271.7418101, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384914273.132628, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000362157821655', 'lastCheck': 1384914265.8027561, 'valid': True, 'code': 0}}
>Thread-15912265::DEBUG::2013-11-20 02:24:36,046::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912265::INFO::2013-11-20 02:24:36,049::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912265::DEBUG::2013-11-20 02:24:36,049::task::495::TaskManager.Task::(_debug) Task bd8b8521-c9ba-4a81-8f0f-875ce2dfc3fe: moving from state init -> state preparing
>Thread-15912265::DEBUG::2013-11-20 02:24:36,050::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912265::DEBUG::2013-11-20 02:24:36,051::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912265::DEBUG::2013-11-20 02:24:36,052::task::495::TaskManager.Task::(_debug) Task bd8b8521-c9ba-4a81-8f0f-875ce2dfc3fe: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912265::DEBUG::2013-11-20 02:24:36,053::task::495::TaskManager.Task::(_debug) Task bd8b8521-c9ba-4a81-8f0f-875ce2dfc3fe: moving from state preparing -> state finished
>Thread-15912265::DEBUG::2013-11-20 02:24:36,054::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912265::DEBUG::2013-11-20 02:24:36,055::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912265::DEBUG::2013-11-20 02:24:36,056::task::495::TaskManager.Task::(_debug) Task bd8b8521-c9ba-4a81-8f0f-875ce2dfc3fe: ref 0 aborting False
>Thread-15912265::INFO::2013-11-20 02:24:36,057::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912266::DEBUG::2013-11-20 02:24:37,091::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912266::INFO::2013-11-20 02:24:37,092::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912266::DEBUG::2013-11-20 02:24:37,104::task::495::TaskManager.Task::(_debug) Task e6c42770-bd6c-4434-97e6-85981917707a: moving from state init -> state preparing
>Thread-15912266::DEBUG::2013-11-20 02:24:37,105::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912266::DEBUG::2013-11-20 02:24:37,105::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912266::DEBUG::2013-11-20 02:24:37,106::task::495::TaskManager.Task::(_debug) Task e6c42770-bd6c-4434-97e6-85981917707a: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912266::DEBUG::2013-11-20 02:24:37,107::task::495::TaskManager.Task::(_debug) Task e6c42770-bd6c-4434-97e6-85981917707a: moving from state preparing -> state finished
>Thread-15912266::DEBUG::2013-11-20 02:24:37,107::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912266::DEBUG::2013-11-20 02:24:37,108::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912266::DEBUG::2013-11-20 02:24:37,108::task::495::TaskManager.Task::(_debug) Task e6c42770-bd6c-4434-97e6-85981917707a: ref 0 aborting False
>Thread-15912266::INFO::2013-11-20 02:24:37,109::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912267::DEBUG::2013-11-20 02:24:38,125::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912267::INFO::2013-11-20 02:24:38,125::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912267::DEBUG::2013-11-20 02:24:38,126::task::495::TaskManager.Task::(_debug) Task 935199fb-db37-43c9-aed6-4e61f6cec6b2: moving from state init -> state preparing
>Thread-15912267::DEBUG::2013-11-20 02:24:38,127::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912267::DEBUG::2013-11-20 02:24:38,127::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912267::DEBUG::2013-11-20 02:24:38,128::task::495::TaskManager.Task::(_debug) Task 935199fb-db37-43c9-aed6-4e61f6cec6b2: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912267::DEBUG::2013-11-20 02:24:38,128::task::495::TaskManager.Task::(_debug) Task 935199fb-db37-43c9-aed6-4e61f6cec6b2: moving from state preparing -> state finished
>Thread-15912267::DEBUG::2013-11-20 02:24:38,129::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912267::DEBUG::2013-11-20 02:24:38,129::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912267::DEBUG::2013-11-20 02:24:38,130::task::495::TaskManager.Task::(_debug) Task 935199fb-db37-43c9-aed6-4e61f6cec6b2: ref 0 aborting False
>Thread-15912267::INFO::2013-11-20 02:24:38,130::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912269::DEBUG::2013-11-20 02:24:39,146::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912269::INFO::2013-11-20 02:24:39,147::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912269::DEBUG::2013-11-20 02:24:39,148::task::495::TaskManager.Task::(_debug) Task c88ce247-5d55-44e4-905f-100e1e70b18d: moving from state init -> state preparing
>Thread-15912269::DEBUG::2013-11-20 02:24:39,148::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912269::DEBUG::2013-11-20 02:24:39,149::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912269::DEBUG::2013-11-20 02:24:39,150::task::495::TaskManager.Task::(_debug) Task c88ce247-5d55-44e4-905f-100e1e70b18d: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912269::DEBUG::2013-11-20 02:24:39,150::task::495::TaskManager.Task::(_debug) Task c88ce247-5d55-44e4-905f-100e1e70b18d: moving from state preparing -> state finished
>Thread-15912269::DEBUG::2013-11-20 02:24:39,151::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912269::DEBUG::2013-11-20 02:24:39,151::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912269::DEBUG::2013-11-20 02:24:39,152::task::495::TaskManager.Task::(_debug) Task c88ce247-5d55-44e4-905f-100e1e70b18d: ref 0 aborting False
>Thread-15912269::INFO::2013-11-20 02:24:39,152::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912270::DEBUG::2013-11-20 02:24:40,168::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912270::INFO::2013-11-20 02:24:40,169::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912270::DEBUG::2013-11-20 02:24:40,170::task::495::TaskManager.Task::(_debug) Task 2ea8fc1c-dec6-45d0-91b6-52e3688a1b01: moving from state init -> state preparing
>Thread-15912270::DEBUG::2013-11-20 02:24:40,170::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912270::DEBUG::2013-11-20 02:24:40,171::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912270::DEBUG::2013-11-20 02:24:40,171::task::495::TaskManager.Task::(_debug) Task 2ea8fc1c-dec6-45d0-91b6-52e3688a1b01: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912270::DEBUG::2013-11-20 02:24:40,172::task::495::TaskManager.Task::(_debug) Task 2ea8fc1c-dec6-45d0-91b6-52e3688a1b01: moving from state preparing -> state finished
>Thread-15912270::DEBUG::2013-11-20 02:24:40,172::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912270::DEBUG::2013-11-20 02:24:40,173::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912270::DEBUG::2013-11-20 02:24:40,174::task::495::TaskManager.Task::(_debug) Task 2ea8fc1c-dec6-45d0-91b6-52e3688a1b01: ref 0 aborting False
>Thread-15912270::INFO::2013-11-20 02:24:40,174::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912271::DEBUG::2013-11-20 02:24:41,194::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912271::INFO::2013-11-20 02:24:41,195::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912271::DEBUG::2013-11-20 02:24:41,195::task::495::TaskManager.Task::(_debug) Task 48151d9f-c225-4d09-b8a0-341975f07787: moving from state init -> state preparing
>Thread-15912271::DEBUG::2013-11-20 02:24:41,196::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912271::DEBUG::2013-11-20 02:24:41,196::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912271::DEBUG::2013-11-20 02:24:41,197::task::495::TaskManager.Task::(_debug) Task 48151d9f-c225-4d09-b8a0-341975f07787: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912271::DEBUG::2013-11-20 02:24:41,198::task::495::TaskManager.Task::(_debug) Task 48151d9f-c225-4d09-b8a0-341975f07787: moving from state preparing -> state finished
>Thread-15912271::DEBUG::2013-11-20 02:24:41,198::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912271::DEBUG::2013-11-20 02:24:41,199::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912271::DEBUG::2013-11-20 02:24:41,199::task::495::TaskManager.Task::(_debug) Task 48151d9f-c225-4d09-b8a0-341975f07787: ref 0 aborting False
>Thread-15912271::INFO::2013-11-20 02:24:41,200::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912273::DEBUG::2013-11-20 02:24:42,215::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912273::INFO::2013-11-20 02:24:42,216::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912273::DEBUG::2013-11-20 02:24:42,217::task::495::TaskManager.Task::(_debug) Task 280c5ea7-0281-4afb-ae5a-ddfba21eb14b: moving from state init -> state preparing
>Thread-15912273::DEBUG::2013-11-20 02:24:42,217::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912273::DEBUG::2013-11-20 02:24:42,218::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912273::DEBUG::2013-11-20 02:24:42,219::task::495::TaskManager.Task::(_debug) Task 280c5ea7-0281-4afb-ae5a-ddfba21eb14b: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912273::DEBUG::2013-11-20 02:24:42,219::task::495::TaskManager.Task::(_debug) Task 280c5ea7-0281-4afb-ae5a-ddfba21eb14b: moving from state preparing -> state finished
>Thread-15912273::DEBUG::2013-11-20 02:24:42,220::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912273::DEBUG::2013-11-20 02:24:42,220::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912273::DEBUG::2013-11-20 02:24:42,221::task::495::TaskManager.Task::(_debug) Task 280c5ea7-0281-4afb-ae5a-ddfba21eb14b: ref 0 aborting False
>Thread-15912273::INFO::2013-11-20 02:24:42,221::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-1024767::ERROR::2013-11-20 02:24:43,145::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912274::DEBUG::2013-11-20 02:24:43,239::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912274::INFO::2013-11-20 02:24:43,240::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912274::DEBUG::2013-11-20 02:24:43,241::task::495::TaskManager.Task::(_debug) Task 70615572-118e-4228-ad0d-7e329b526fea: moving from state init -> state preparing
>Thread-15912274::DEBUG::2013-11-20 02:24:43,241::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912274::DEBUG::2013-11-20 02:24:43,242::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912274::DEBUG::2013-11-20 02:24:43,243::task::495::TaskManager.Task::(_debug) Task 70615572-118e-4228-ad0d-7e329b526fea: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912274::DEBUG::2013-11-20 02:24:43,243::task::495::TaskManager.Task::(_debug) Task 70615572-118e-4228-ad0d-7e329b526fea: moving from state preparing -> state finished
>Thread-15912274::DEBUG::2013-11-20 02:24:43,244::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912274::DEBUG::2013-11-20 02:24:43,244::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912274::DEBUG::2013-11-20 02:24:43,245::task::495::TaskManager.Task::(_debug) Task 70615572-118e-4228-ad0d-7e329b526fea: ref 0 aborting False
>Thread-15912274::INFO::2013-11-20 02:24:43,245::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912275::DEBUG::2013-11-20 02:24:44,261::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912275::INFO::2013-11-20 02:24:44,262::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912275::DEBUG::2013-11-20 02:24:44,263::task::495::TaskManager.Task::(_debug) Task 849c03ee-6c79-4e3a-9f02-57f6a697bfda: moving from state init -> state preparing
>Thread-15912275::DEBUG::2013-11-20 02:24:44,263::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912275::DEBUG::2013-11-20 02:24:44,264::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912275::DEBUG::2013-11-20 02:24:44,264::task::495::TaskManager.Task::(_debug) Task 849c03ee-6c79-4e3a-9f02-57f6a697bfda: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912275::DEBUG::2013-11-20 02:24:44,265::task::495::TaskManager.Task::(_debug) Task 849c03ee-6c79-4e3a-9f02-57f6a697bfda: moving from state preparing -> state finished
>Thread-15912275::DEBUG::2013-11-20 02:24:44,266::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912275::DEBUG::2013-11-20 02:24:44,266::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912275::DEBUG::2013-11-20 02:24:44,267::task::495::TaskManager.Task::(_debug) Task 849c03ee-6c79-4e3a-9f02-57f6a697bfda: ref 0 aborting False
>Thread-15912275::INFO::2013-11-20 02:24:44,267::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912277::DEBUG::2013-11-20 02:24:45,283::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912277::INFO::2013-11-20 02:24:45,284::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912277::DEBUG::2013-11-20 02:24:45,285::task::495::TaskManager.Task::(_debug) Task 47295853-bdb2-4d50-b02c-1ff290b06197: moving from state init -> state preparing
>Thread-15912277::DEBUG::2013-11-20 02:24:45,285::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912277::DEBUG::2013-11-20 02:24:45,286::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912277::DEBUG::2013-11-20 02:24:45,286::task::495::TaskManager.Task::(_debug) Task 47295853-bdb2-4d50-b02c-1ff290b06197: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912277::DEBUG::2013-11-20 02:24:45,287::task::495::TaskManager.Task::(_debug) Task 47295853-bdb2-4d50-b02c-1ff290b06197: moving from state preparing -> state finished
>Thread-15912277::DEBUG::2013-11-20 02:24:45,288::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912277::DEBUG::2013-11-20 02:24:45,288::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912277::DEBUG::2013-11-20 02:24:45,289::task::495::TaskManager.Task::(_debug) Task 47295853-bdb2-4d50-b02c-1ff290b06197: ref 0 aborting False
>Thread-15912277::INFO::2013-11-20 02:24:45,289::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912278::DEBUG::2013-11-20 02:24:46,305::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912278::INFO::2013-11-20 02:24:46,306::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912278::DEBUG::2013-11-20 02:24:46,307::task::495::TaskManager.Task::(_debug) Task ba6296fa-8eb2-451c-869b-7c742722fe38: moving from state init -> state preparing
>Thread-15912278::DEBUG::2013-11-20 02:24:46,307::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912278::DEBUG::2013-11-20 02:24:46,308::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912278::DEBUG::2013-11-20 02:24:46,308::task::495::TaskManager.Task::(_debug) Task ba6296fa-8eb2-451c-869b-7c742722fe38: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912278::DEBUG::2013-11-20 02:24:46,309::task::495::TaskManager.Task::(_debug) Task ba6296fa-8eb2-451c-869b-7c742722fe38: moving from state preparing -> state finished
>Thread-15912278::DEBUG::2013-11-20 02:24:46,309::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912278::DEBUG::2013-11-20 02:24:46,310::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912278::DEBUG::2013-11-20 02:24:46,310::task::495::TaskManager.Task::(_debug) Task ba6296fa-8eb2-451c-869b-7c742722fe38: ref 0 aborting False
>Thread-15912278::INFO::2013-11-20 02:24:46,311::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912279::DEBUG::2013-11-20 02:24:47,327::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912279::INFO::2013-11-20 02:24:47,327::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912279::DEBUG::2013-11-20 02:24:47,338::task::495::TaskManager.Task::(_debug) Task ec92d3c8-33bf-425d-989c-6aea6ed1e9a5: moving from state init -> state preparing
>Thread-15912279::DEBUG::2013-11-20 02:24:47,339::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912279::DEBUG::2013-11-20 02:24:47,340::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912279::DEBUG::2013-11-20 02:24:47,340::task::495::TaskManager.Task::(_debug) Task ec92d3c8-33bf-425d-989c-6aea6ed1e9a5: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912279::DEBUG::2013-11-20 02:24:47,341::task::495::TaskManager.Task::(_debug) Task ec92d3c8-33bf-425d-989c-6aea6ed1e9a5: moving from state preparing -> state finished
>Thread-15912279::DEBUG::2013-11-20 02:24:47,341::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912279::DEBUG::2013-11-20 02:24:47,342::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912279::DEBUG::2013-11-20 02:24:47,342::task::495::TaskManager.Task::(_debug) Task ec92d3c8-33bf-425d-989c-6aea6ed1e9a5: ref 0 aborting False
>Thread-15912279::INFO::2013-11-20 02:24:47,343::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912281::DEBUG::2013-11-20 02:24:48,358::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912281::INFO::2013-11-20 02:24:48,359::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912281::DEBUG::2013-11-20 02:24:48,360::task::495::TaskManager.Task::(_debug) Task 97ab8a29-8039-42bf-abe3-c6bbebbe45db: moving from state init -> state preparing
>Thread-15912281::DEBUG::2013-11-20 02:24:48,360::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912281::DEBUG::2013-11-20 02:24:48,361::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912281::DEBUG::2013-11-20 02:24:48,362::task::495::TaskManager.Task::(_debug) Task 97ab8a29-8039-42bf-abe3-c6bbebbe45db: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912281::DEBUG::2013-11-20 02:24:48,362::task::495::TaskManager.Task::(_debug) Task 97ab8a29-8039-42bf-abe3-c6bbebbe45db: moving from state preparing -> state finished
>Thread-15912281::DEBUG::2013-11-20 02:24:48,363::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912281::DEBUG::2013-11-20 02:24:48,363::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912281::DEBUG::2013-11-20 02:24:48,364::task::495::TaskManager.Task::(_debug) Task 97ab8a29-8039-42bf-abe3-c6bbebbe45db: ref 0 aborting False
>Thread-15912281::INFO::2013-11-20 02:24:48,364::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912282::DEBUG::2013-11-20 02:24:49,380::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912282::INFO::2013-11-20 02:24:49,381::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912282::DEBUG::2013-11-20 02:24:49,382::task::495::TaskManager.Task::(_debug) Task 95ef8e57-950b-430a-b5b1-299343041332: moving from state init -> state preparing
>Thread-15912282::DEBUG::2013-11-20 02:24:49,382::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912282::DEBUG::2013-11-20 02:24:49,383::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912282::DEBUG::2013-11-20 02:24:49,383::task::495::TaskManager.Task::(_debug) Task 95ef8e57-950b-430a-b5b1-299343041332: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912282::DEBUG::2013-11-20 02:24:49,384::task::495::TaskManager.Task::(_debug) Task 95ef8e57-950b-430a-b5b1-299343041332: moving from state preparing -> state finished
>Thread-15912282::DEBUG::2013-11-20 02:24:49,385::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912282::DEBUG::2013-11-20 02:24:49,385::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912282::DEBUG::2013-11-20 02:24:49,386::task::495::TaskManager.Task::(_debug) Task 95ef8e57-950b-430a-b5b1-299343041332: ref 0 aborting False
>Thread-15912282::INFO::2013-11-20 02:24:49,386::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912283::DEBUG::2013-11-20 02:24:50,402::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912283::INFO::2013-11-20 02:24:50,402::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912283::DEBUG::2013-11-20 02:24:50,403::task::495::TaskManager.Task::(_debug) Task ebdf7d62-c332-4b45-86fb-d40227ae4a90: moving from state init -> state preparing
>Thread-15912283::DEBUG::2013-11-20 02:24:50,403::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912283::DEBUG::2013-11-20 02:24:50,404::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912283::DEBUG::2013-11-20 02:24:50,405::task::495::TaskManager.Task::(_debug) Task ebdf7d62-c332-4b45-86fb-d40227ae4a90: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912283::DEBUG::2013-11-20 02:24:50,405::task::495::TaskManager.Task::(_debug) Task ebdf7d62-c332-4b45-86fb-d40227ae4a90: moving from state preparing -> state finished
>Thread-15912283::DEBUG::2013-11-20 02:24:50,406::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912283::DEBUG::2013-11-20 02:24:50,406::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912283::DEBUG::2013-11-20 02:24:50,407::task::495::TaskManager.Task::(_debug) Task ebdf7d62-c332-4b45-86fb-d40227ae4a90: ref 0 aborting False
>Thread-15912283::INFO::2013-11-20 02:24:50,407::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912284::INFO::2013-11-20 02:24:50,431::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912284::DEBUG::2013-11-20 02:24:50,431::task::495::TaskManager.Task::(_debug) Task 0a7413ae-1464-4319-8d92-3ec2960ea7ec: moving from state init -> state preparing
>Thread-15912284::DEBUG::2013-11-20 02:24:50,432::task::495::TaskManager.Task::(_debug) Task 0a7413ae-1464-4319-8d92-3ec2960ea7ec: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000280141830444', 'lastCheck': 1384914287.5072241, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00153803825378', 'lastCheck': 1384914281.7575071, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384914283.1459229, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000362873077393', 'lastCheck': 1384914285.825582, 'valid': True, 'code': 0}}
>Thread-15912284::DEBUG::2013-11-20 02:24:50,443::task::495::TaskManager.Task::(_debug) Task 0a7413ae-1464-4319-8d92-3ec2960ea7ec: moving from state preparing -> state finished
>Thread-15912284::DEBUG::2013-11-20 02:24:50,443::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912284::DEBUG::2013-11-20 02:24:50,444::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912284::DEBUG::2013-11-20 02:24:50,444::task::495::TaskManager.Task::(_debug) Task 0a7413ae-1464-4319-8d92-3ec2960ea7ec: ref 0 aborting False
>Thread-15912284::INFO::2013-11-20 02:24:50,445::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000280141830444', 'lastCheck': 1384914287.5072241, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00153803825378', 'lastCheck': 1384914281.7575071, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0', 'lastCheck': 1384914283.1459229, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000362873077393', 'lastCheck': 1384914285.825582, 'valid': True, 'code': 0}}
>Thread-15912286::DEBUG::2013-11-20 02:24:51,465::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912286::INFO::2013-11-20 02:24:51,466::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912286::DEBUG::2013-11-20 02:24:51,467::task::495::TaskManager.Task::(_debug) Task 9ded8c59-9984-452e-bade-c6939ce25bf3: moving from state init -> state preparing
>Thread-15912286::DEBUG::2013-11-20 02:24:51,467::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912286::DEBUG::2013-11-20 02:24:51,468::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912286::DEBUG::2013-11-20 02:24:51,469::task::495::TaskManager.Task::(_debug) Task 9ded8c59-9984-452e-bade-c6939ce25bf3: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912286::DEBUG::2013-11-20 02:24:51,469::task::495::TaskManager.Task::(_debug) Task 9ded8c59-9984-452e-bade-c6939ce25bf3: moving from state preparing -> state finished
>Thread-15912286::DEBUG::2013-11-20 02:24:51,470::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912286::DEBUG::2013-11-20 02:24:51,470::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912286::DEBUG::2013-11-20 02:24:51,471::task::495::TaskManager.Task::(_debug) Task 9ded8c59-9984-452e-bade-c6939ce25bf3: ref 0 aborting False
>Thread-15912286::INFO::2013-11-20 02:24:51,471::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912287::DEBUG::2013-11-20 02:24:52,486::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912287::INFO::2013-11-20 02:24:52,487::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912287::DEBUG::2013-11-20 02:24:52,495::task::495::TaskManager.Task::(_debug) Task 81828ba9-dfae-4b2d-bfb2-d9a70d7984d3: moving from state init -> state preparing
>Thread-15912287::DEBUG::2013-11-20 02:24:52,495::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912287::DEBUG::2013-11-20 02:24:52,496::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912287::DEBUG::2013-11-20 02:24:52,496::task::495::TaskManager.Task::(_debug) Task 81828ba9-dfae-4b2d-bfb2-d9a70d7984d3: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912287::DEBUG::2013-11-20 02:24:52,497::task::495::TaskManager.Task::(_debug) Task 81828ba9-dfae-4b2d-bfb2-d9a70d7984d3: moving from state preparing -> state finished
>Thread-15912287::DEBUG::2013-11-20 02:24:52,498::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912287::DEBUG::2013-11-20 02:24:52,498::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912287::DEBUG::2013-11-20 02:24:52,499::task::495::TaskManager.Task::(_debug) Task 81828ba9-dfae-4b2d-bfb2-d9a70d7984d3: ref 0 aborting False
>Thread-15912287::INFO::2013-11-20 02:24:52,499::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-1024767::ERROR::2013-11-20 02:24:53,158::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` ([Errno 2] No such file or directory: '/rhev/data-center/mnt/kairo.brd-klee.dk:_export/5d7b5178-3790-472c-a13b-b481eb27a61e/dom_md/metadata')
>Thread-15912288::DEBUG::2013-11-20 02:24:53,515::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912288::INFO::2013-11-20 02:24:53,516::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912288::DEBUG::2013-11-20 02:24:53,516::task::495::TaskManager.Task::(_debug) Task 499c46e5-41d5-4422-a17f-b008a0619c3d: moving from state init -> state preparing
>Thread-15912288::DEBUG::2013-11-20 02:24:53,517::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912288::DEBUG::2013-11-20 02:24:53,517::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912288::DEBUG::2013-11-20 02:24:53,518::task::495::TaskManager.Task::(_debug) Task 499c46e5-41d5-4422-a17f-b008a0619c3d: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912288::DEBUG::2013-11-20 02:24:53,519::task::495::TaskManager.Task::(_debug) Task 499c46e5-41d5-4422-a17f-b008a0619c3d: moving from state preparing -> state finished
>Thread-15912288::DEBUG::2013-11-20 02:24:53,519::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912288::DEBUG::2013-11-20 02:24:53,520::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912288::DEBUG::2013-11-20 02:24:53,520::task::495::TaskManager.Task::(_debug) Task 499c46e5-41d5-4422-a17f-b008a0619c3d: ref 0 aborting False
>Thread-15912288::INFO::2013-11-20 02:24:53,521::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912290::DEBUG::2013-11-20 02:24:54,536::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912290::INFO::2013-11-20 02:24:54,537::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912290::DEBUG::2013-11-20 02:24:54,538::task::495::TaskManager.Task::(_debug) Task 55bb00d8-52ec-4ef9-a5af-94ed9f57b66f: moving from state init -> state preparing
>Thread-15912290::DEBUG::2013-11-20 02:24:54,538::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912290::DEBUG::2013-11-20 02:24:54,539::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912290::DEBUG::2013-11-20 02:24:54,539::task::495::TaskManager.Task::(_debug) Task 55bb00d8-52ec-4ef9-a5af-94ed9f57b66f: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912290::DEBUG::2013-11-20 02:24:54,540::task::495::TaskManager.Task::(_debug) Task 55bb00d8-52ec-4ef9-a5af-94ed9f57b66f: moving from state preparing -> state finished
>Thread-15912290::DEBUG::2013-11-20 02:24:54,541::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912290::DEBUG::2013-11-20 02:24:54,541::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912290::DEBUG::2013-11-20 02:24:54,542::task::495::TaskManager.Task::(_debug) Task 55bb00d8-52ec-4ef9-a5af-94ed9f57b66f: ref 0 aborting False
>Thread-15912290::INFO::2013-11-20 02:24:54,542::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912291::DEBUG::2013-11-20 02:24:55,558::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912291::INFO::2013-11-20 02:24:55,559::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912291::DEBUG::2013-11-20 02:24:55,560::task::495::TaskManager.Task::(_debug) Task fe9f98e9-ec15-4baf-8554-09e46fee438a: moving from state init -> state preparing
>Thread-15912291::DEBUG::2013-11-20 02:24:55,560::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912291::DEBUG::2013-11-20 02:24:55,561::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912291::DEBUG::2013-11-20 02:24:55,561::task::495::TaskManager.Task::(_debug) Task fe9f98e9-ec15-4baf-8554-09e46fee438a: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912291::DEBUG::2013-11-20 02:24:55,562::task::495::TaskManager.Task::(_debug) Task fe9f98e9-ec15-4baf-8554-09e46fee438a: moving from state preparing -> state finished
>Thread-15912291::DEBUG::2013-11-20 02:24:55,563::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912291::DEBUG::2013-11-20 02:24:55,563::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912291::DEBUG::2013-11-20 02:24:55,564::task::495::TaskManager.Task::(_debug) Task fe9f98e9-ec15-4baf-8554-09e46fee438a: ref 0 aborting False
>Thread-15912291::INFO::2013-11-20 02:24:55,564::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912293::DEBUG::2013-11-20 02:24:56,580::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912293::INFO::2013-11-20 02:24:56,581::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912293::DEBUG::2013-11-20 02:24:56,581::task::495::TaskManager.Task::(_debug) Task 65955b54-f7cb-47c9-8811-476144c80bde: moving from state init -> state preparing
>Thread-15912293::DEBUG::2013-11-20 02:24:56,582::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912293::DEBUG::2013-11-20 02:24:56,582::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912293::DEBUG::2013-11-20 02:24:56,583::task::495::TaskManager.Task::(_debug) Task 65955b54-f7cb-47c9-8811-476144c80bde: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912293::DEBUG::2013-11-20 02:24:56,583::task::495::TaskManager.Task::(_debug) Task 65955b54-f7cb-47c9-8811-476144c80bde: moving from state preparing -> state finished
>Thread-15912293::DEBUG::2013-11-20 02:24:56,584::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912293::DEBUG::2013-11-20 02:24:56,584::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912293::DEBUG::2013-11-20 02:24:56,585::task::495::TaskManager.Task::(_debug) Task 65955b54-f7cb-47c9-8811-476144c80bde: ref 0 aborting False
>Thread-15912293::INFO::2013-11-20 02:24:56,586::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912294::DEBUG::2013-11-20 02:24:57,601::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912294::INFO::2013-11-20 02:24:57,602::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912294::DEBUG::2013-11-20 02:24:57,613::task::495::TaskManager.Task::(_debug) Task 11b3bab9-c008-4bbc-9340-cbe0bae697ed: moving from state init -> state preparing
>Thread-15912294::DEBUG::2013-11-20 02:24:57,614::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912294::DEBUG::2013-11-20 02:24:57,614::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912294::DEBUG::2013-11-20 02:24:57,615::task::495::TaskManager.Task::(_debug) Task 11b3bab9-c008-4bbc-9340-cbe0bae697ed: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912294::DEBUG::2013-11-20 02:24:57,615::task::495::TaskManager.Task::(_debug) Task 11b3bab9-c008-4bbc-9340-cbe0bae697ed: moving from state preparing -> state finished
>Thread-15912294::DEBUG::2013-11-20 02:24:57,616::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912294::DEBUG::2013-11-20 02:24:57,617::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912294::DEBUG::2013-11-20 02:24:57,617::task::495::TaskManager.Task::(_debug) Task 11b3bab9-c008-4bbc-9340-cbe0bae697ed: ref 0 aborting False
>Thread-15912294::INFO::2013-11-20 02:24:57,618::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912295::DEBUG::2013-11-20 02:24:58,633::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912295::INFO::2013-11-20 02:24:58,634::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912295::DEBUG::2013-11-20 02:24:58,634::task::495::TaskManager.Task::(_debug) Task 6b406e14-7fe3-4235-aa39-9de713af0968: moving from state init -> state preparing
>Thread-15912295::DEBUG::2013-11-20 02:24:58,635::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912295::DEBUG::2013-11-20 02:24:58,635::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912295::DEBUG::2013-11-20 02:24:58,636::task::495::TaskManager.Task::(_debug) Task 6b406e14-7fe3-4235-aa39-9de713af0968: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912295::DEBUG::2013-11-20 02:24:58,637::task::495::TaskManager.Task::(_debug) Task 6b406e14-7fe3-4235-aa39-9de713af0968: moving from state preparing -> state finished
>Thread-15912295::DEBUG::2013-11-20 02:24:58,637::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912295::DEBUG::2013-11-20 02:24:58,638::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912295::DEBUG::2013-11-20 02:24:58,638::task::495::TaskManager.Task::(_debug) Task 6b406e14-7fe3-4235-aa39-9de713af0968: ref 0 aborting False
>Thread-15912295::INFO::2013-11-20 02:24:58,639::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912297::DEBUG::2013-11-20 02:24:59,654::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912297::INFO::2013-11-20 02:24:59,655::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912297::DEBUG::2013-11-20 02:24:59,656::task::495::TaskManager.Task::(_debug) Task d47d456f-7856-4473-9d58-29dc8391311a: moving from state init -> state preparing
>Thread-15912297::DEBUG::2013-11-20 02:24:59,656::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912297::DEBUG::2013-11-20 02:24:59,657::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912297::DEBUG::2013-11-20 02:24:59,657::task::495::TaskManager.Task::(_debug) Task d47d456f-7856-4473-9d58-29dc8391311a: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912297::DEBUG::2013-11-20 02:24:59,658::task::495::TaskManager.Task::(_debug) Task d47d456f-7856-4473-9d58-29dc8391311a: moving from state preparing -> state finished
>Thread-15912297::DEBUG::2013-11-20 02:24:59,658::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912297::DEBUG::2013-11-20 02:24:59,659::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912297::DEBUG::2013-11-20 02:24:59,659::task::495::TaskManager.Task::(_debug) Task d47d456f-7856-4473-9d58-29dc8391311a: ref 0 aborting False
>Thread-15912297::INFO::2013-11-20 02:24:59,660::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912298::DEBUG::2013-11-20 02:25:00,247::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912298::INFO::2013-11-20 02:25:00,248::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: validateStorageServerConnection, args: (domType=1, spUUID=00000000-0000-0000-0000-000000000000, conList=[{'connection': 'kairo.brd-klee.dk:/export', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'ec9380b7-cfc1-41d8-9237-d5c7c505ef36', 'port': ''}])
>Thread-15912298::DEBUG::2013-11-20 02:25:00,248::task::495::TaskManager.Task::(_debug) Task 9429da2b-2c38-4b6c-8593-9b9fae6aca89: moving from state init -> state preparing
>Thread-15912298::INFO::2013-11-20 02:25:00,249::storage_connection::127::Storage.ServerConnection::(validate) Request to validate NFS storage server
>Thread-15912298::INFO::2013-11-20 02:25:00,250::storage_connection::53::Storage.ServerConnection::(__validateConnectionParams) conList=[{'connection': 'kairo.brd-klee.dk:/export', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'ec9380b7-cfc1-41d8-9237-d5c7c505ef36', 'port': ''}]
>Thread-15912298::DEBUG::2013-11-20 02:25:00,254::fileUtils::83::Storage.Misc.excCmd::(mount) '/usr/bin/sudo -n /bin/mount -o soft,timeo=600,retrans=6,nosharecache,vers=3 -t nfs kairo.brd-klee.dk:/export /tmp/tmpWUQ14N' (cwd None)
>Thread-15912299::DEBUG::2013-11-20 02:25:00,676::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912299::INFO::2013-11-20 02:25:00,677::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912299::DEBUG::2013-11-20 02:25:00,677::task::495::TaskManager.Task::(_debug) Task 1729bdc2-a130-4258-8b70-9334530fce68: moving from state init -> state preparing
>Thread-15912299::DEBUG::2013-11-20 02:25:00,677::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912299::DEBUG::2013-11-20 02:25:00,678::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912299::DEBUG::2013-11-20 02:25:00,678::task::495::TaskManager.Task::(_debug) Task 1729bdc2-a130-4258-8b70-9334530fce68: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912299::DEBUG::2013-11-20 02:25:00,678::task::495::TaskManager.Task::(_debug) Task 1729bdc2-a130-4258-8b70-9334530fce68: moving from state preparing -> state finished
>Thread-15912299::DEBUG::2013-11-20 02:25:00,679::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912299::DEBUG::2013-11-20 02:25:00,679::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912299::DEBUG::2013-11-20 02:25:00,679::task::495::TaskManager.Task::(_debug) Task 1729bdc2-a130-4258-8b70-9334530fce68: ref 0 aborting False
>Thread-15912299::INFO::2013-11-20 02:25:00,680::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912298::DEBUG::2013-11-20 02:25:01,017::fileUtils::83::Storage.Misc.excCmd::(mount) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912298::DEBUG::2013-11-20 02:25:01,352::fileUtils::112::Storage.Misc.excCmd::(umount) '/usr/bin/sudo -n /bin/umount -t nfs -f /tmp/tmpWUQ14N' (cwd None)
>Thread-15912300::DEBUG::2013-11-20 02:25:01,695::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912298::DEBUG::2013-11-20 02:25:01,694::fileUtils::112::Storage.Misc.excCmd::(umount) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912300::INFO::2013-11-20 02:25:01,696::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912300::DEBUG::2013-11-20 02:25:01,697::task::495::TaskManager.Task::(_debug) Task f96e9d8d-cdca-4c98-b0ce-ac4e1f5cf2d1: moving from state init -> state preparing
>Thread-15912300::DEBUG::2013-11-20 02:25:01,698::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912298::DEBUG::2013-11-20 02:25:01,698::task::495::TaskManager.Task::(_debug) Task 9429da2b-2c38-4b6c-8593-9b9fae6aca89: finished: {'statuslist': [{'status': 0, 'id': 'ec9380b7-cfc1-41d8-9237-d5c7c505ef36'}]}
>Thread-15912300::DEBUG::2013-11-20 02:25:01,698::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912298::DEBUG::2013-11-20 02:25:01,699::task::495::TaskManager.Task::(_debug) Task 9429da2b-2c38-4b6c-8593-9b9fae6aca89: moving from state preparing -> state finished
>Thread-15912300::DEBUG::2013-11-20 02:25:01,699::task::495::TaskManager.Task::(_debug) Task f96e9d8d-cdca-4c98-b0ce-ac4e1f5cf2d1: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912298::DEBUG::2013-11-20 02:25:01,700::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912300::DEBUG::2013-11-20 02:25:01,700::task::495::TaskManager.Task::(_debug) Task f96e9d8d-cdca-4c98-b0ce-ac4e1f5cf2d1: moving from state preparing -> state finished
>Thread-15912298::DEBUG::2013-11-20 02:25:01,701::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912300::DEBUG::2013-11-20 02:25:01,701::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912298::DEBUG::2013-11-20 02:25:01,701::task::495::TaskManager.Task::(_debug) Task 9429da2b-2c38-4b6c-8593-9b9fae6aca89: ref 0 aborting False
>Thread-15912300::DEBUG::2013-11-20 02:25:01,702::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912298::INFO::2013-11-20 02:25:01,702::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: validateStorageServerConnection, Return response: {'status': {'message': 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': 'ec9380b7-cfc1-41d8-9237-d5c7c505ef36'}]}
>Thread-15912300::DEBUG::2013-11-20 02:25:01,703::task::495::TaskManager.Task::(_debug) Task f96e9d8d-cdca-4c98-b0ce-ac4e1f5cf2d1: ref 0 aborting False
>Thread-15912300::INFO::2013-11-20 02:25:01,709::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912301::DEBUG::2013-11-20 02:25:01,729::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912301::INFO::2013-11-20 02:25:01,730::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: connectStorageServer, args: (domType=1, spUUID=00000000-0000-0000-0000-000000000000, conList=[{'connection': 'kairo.brd-klee.dk:/export', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'ec9380b7-cfc1-41d8-9237-d5c7c505ef36', 'port': ''}])
>Thread-15912301::DEBUG::2013-11-20 02:25:01,731::task::495::TaskManager.Task::(_debug) Task e48a8e9b-62e8-4414-9428-86fb4eaeb3a7: moving from state init -> state preparing
>Thread-15912301::INFO::2013-11-20 02:25:01,731::storage_connection::95::Storage.ServerConnection::(connect) Request to connect NFS storage server
>Thread-15912301::INFO::2013-11-20 02:25:01,732::storage_connection::53::Storage.ServerConnection::(__validateConnectionParams) conList=[{'connection': 'kairo.brd-klee.dk:/export', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'ec9380b7-cfc1-41d8-9237-d5c7c505ef36', 'port': ''}]
>Thread-15912301::DEBUG::2013-11-20 02:25:01,735::fileUtils::83::Storage.Misc.excCmd::(mount) '/usr/bin/sudo -n /bin/mount -o soft,timeo=600,retrans=6,nosharecache,vers=3 -t nfs kairo.brd-klee.dk:/export /rhev/data-center/mnt/kairo.brd-klee.dk:_export' (cwd None)
>Thread-15912301::DEBUG::2013-11-20 02:25:02,125::fileUtils::83::Storage.Misc.excCmd::(mount) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912301::DEBUG::2013-11-20 02:25:02,135::lvm::547::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15912301::DEBUG::2013-11-20 02:25:02,136::lvm::549::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15912301::DEBUG::2013-11-20 02:25:02,137::lvm::559::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15912301::DEBUG::2013-11-20 02:25:02,137::lvm::561::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15912301::DEBUG::2013-11-20 02:25:02,138::lvm::580::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15912301::DEBUG::2013-11-20 02:25:02,139::lvm::582::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15912301::DEBUG::2013-11-20 02:25:02,139::task::495::TaskManager.Task::(_debug) Task e48a8e9b-62e8-4414-9428-86fb4eaeb3a7: finished: {'statuslist': [{'status': 0, 'id': 'ec9380b7-cfc1-41d8-9237-d5c7c505ef36'}]}
>Thread-15912301::DEBUG::2013-11-20 02:25:02,140::task::495::TaskManager.Task::(_debug) Task e48a8e9b-62e8-4414-9428-86fb4eaeb3a7: moving from state preparing -> state finished
>Thread-15912301::DEBUG::2013-11-20 02:25:02,141::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912301::DEBUG::2013-11-20 02:25:02,141::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912301::DEBUG::2013-11-20 02:25:02,142::task::495::TaskManager.Task::(_debug) Task e48a8e9b-62e8-4414-9428-86fb4eaeb3a7: ref 0 aborting False
>Thread-15912301::INFO::2013-11-20 02:25:02,142::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: connectStorageServer, Return response: {'status': {'message': 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': 'ec9380b7-cfc1-41d8-9237-d5c7c505ef36'}]}
>Thread-15912303::DEBUG::2013-11-20 02:25:02,724::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912303::INFO::2013-11-20 02:25:02,725::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912303::DEBUG::2013-11-20 02:25:02,726::task::495::TaskManager.Task::(_debug) Task fc393201-a466-4491-af0d-38c5f25b1747: moving from state init -> state preparing
>Thread-15912303::DEBUG::2013-11-20 02:25:02,726::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912303::DEBUG::2013-11-20 02:25:02,727::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912303::DEBUG::2013-11-20 02:25:02,728::task::495::TaskManager.Task::(_debug) Task fc393201-a466-4491-af0d-38c5f25b1747: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912303::DEBUG::2013-11-20 02:25:02,728::task::495::TaskManager.Task::(_debug) Task fc393201-a466-4491-af0d-38c5f25b1747: moving from state preparing -> state finished
>Thread-15912303::DEBUG::2013-11-20 02:25:02,729::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912303::DEBUG::2013-11-20 02:25:02,729::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912303::DEBUG::2013-11-20 02:25:02,730::task::495::TaskManager.Task::(_debug) Task fc393201-a466-4491-af0d-38c5f25b1747: ref 0 aborting False
>Thread-15912303::INFO::2013-11-20 02:25:02,730::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-1024767::ERROR::2013-11-20 02:25:03,187::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912304::DEBUG::2013-11-20 02:25:03,750::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912304::INFO::2013-11-20 02:25:03,751::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912304::DEBUG::2013-11-20 02:25:03,752::task::495::TaskManager.Task::(_debug) Task 54e796b1-8186-4caa-8e4e-c949fe80a01f: moving from state init -> state preparing
>Thread-15912304::DEBUG::2013-11-20 02:25:03,752::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912304::DEBUG::2013-11-20 02:25:03,753::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912304::DEBUG::2013-11-20 02:25:03,753::task::495::TaskManager.Task::(_debug) Task 54e796b1-8186-4caa-8e4e-c949fe80a01f: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912304::DEBUG::2013-11-20 02:25:03,754::task::495::TaskManager.Task::(_debug) Task 54e796b1-8186-4caa-8e4e-c949fe80a01f: moving from state preparing -> state finished
>Thread-15912304::DEBUG::2013-11-20 02:25:03,754::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912304::DEBUG::2013-11-20 02:25:03,755::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912304::DEBUG::2013-11-20 02:25:03,755::task::495::TaskManager.Task::(_debug) Task 54e796b1-8186-4caa-8e4e-c949fe80a01f: ref 0 aborting False
>Thread-15912304::INFO::2013-11-20 02:25:03,756::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912305::DEBUG::2013-11-20 02:25:04,776::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912305::INFO::2013-11-20 02:25:04,778::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912305::DEBUG::2013-11-20 02:25:04,780::task::495::TaskManager.Task::(_debug) Task f882fff1-98d3-4968-be64-76547d8c0e97: moving from state init -> state preparing
>Thread-15912305::DEBUG::2013-11-20 02:25:04,781::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912305::DEBUG::2013-11-20 02:25:04,781::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912305::DEBUG::2013-11-20 02:25:04,782::task::495::TaskManager.Task::(_debug) Task f882fff1-98d3-4968-be64-76547d8c0e97: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912305::DEBUG::2013-11-20 02:25:04,783::task::495::TaskManager.Task::(_debug) Task f882fff1-98d3-4968-be64-76547d8c0e97: moving from state preparing -> state finished
>Thread-15912305::DEBUG::2013-11-20 02:25:04,785::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912305::DEBUG::2013-11-20 02:25:04,785::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912305::DEBUG::2013-11-20 02:25:04,786::task::495::TaskManager.Task::(_debug) Task f882fff1-98d3-4968-be64-76547d8c0e97: ref 0 aborting False
>Thread-15912305::INFO::2013-11-20 02:25:04,787::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912306::INFO::2013-11-20 02:25:05,634::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912306::DEBUG::2013-11-20 02:25:05,635::task::495::TaskManager.Task::(_debug) Task a5363760-d2e1-4e78-9017-1bffe30bcef9: moving from state init -> state preparing
>Thread-15912306::DEBUG::2013-11-20 02:25:05,635::task::495::TaskManager.Task::(_debug) Task a5363760-d2e1-4e78-9017-1bffe30bcef9: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000324010848999', 'lastCheck': 1384914297.5202911, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148391723633', 'lastCheck': 1384914301.7890749, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0251250267029', 'lastCheck': 1384914303.2140191, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000370025634766', 'lastCheck': 1384914295.8369949, 'valid': True, 'code': 0}}
>Thread-15912306::DEBUG::2013-11-20 02:25:05,636::task::495::TaskManager.Task::(_debug) Task a5363760-d2e1-4e78-9017-1bffe30bcef9: moving from state preparing -> state finished
>Thread-15912306::DEBUG::2013-11-20 02:25:05,636::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912306::DEBUG::2013-11-20 02:25:05,637::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912306::DEBUG::2013-11-20 02:25:05,638::task::495::TaskManager.Task::(_debug) Task a5363760-d2e1-4e78-9017-1bffe30bcef9: ref 0 aborting False
>Thread-15912306::INFO::2013-11-20 02:25:05,638::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000324010848999', 'lastCheck': 1384914297.5202911, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148391723633', 'lastCheck': 1384914301.7890749, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0251250267029', 'lastCheck': 1384914303.2140191, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000370025634766', 'lastCheck': 1384914295.8369949, 'valid': True, 'code': 0}}
>Thread-15912308::DEBUG::2013-11-20 02:25:05,805::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912308::INFO::2013-11-20 02:25:05,805::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912308::DEBUG::2013-11-20 02:25:05,806::task::495::TaskManager.Task::(_debug) Task 30e5c063-315b-4894-94f6-5f793ac8649d: moving from state init -> state preparing
>Thread-15912308::DEBUG::2013-11-20 02:25:05,806::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912308::DEBUG::2013-11-20 02:25:05,807::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912308::DEBUG::2013-11-20 02:25:05,808::task::495::TaskManager.Task::(_debug) Task 30e5c063-315b-4894-94f6-5f793ac8649d: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912308::DEBUG::2013-11-20 02:25:05,808::task::495::TaskManager.Task::(_debug) Task 30e5c063-315b-4894-94f6-5f793ac8649d: moving from state preparing -> state finished
>Thread-15912308::DEBUG::2013-11-20 02:25:05,809::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912308::DEBUG::2013-11-20 02:25:05,809::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912308::DEBUG::2013-11-20 02:25:05,810::task::495::TaskManager.Task::(_debug) Task 30e5c063-315b-4894-94f6-5f793ac8649d: ref 0 aborting False
>Thread-15912308::INFO::2013-11-20 02:25:05,810::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-42840::DEBUG::2013-11-20 02:25:05,847::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-42840::DEBUG::2013-11-20 02:25:05,854::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 02:25:06,564::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-42840::DEBUG::2013-11-20 02:25:06,570::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-15912309::DEBUG::2013-11-20 02:25:06,827::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912309::INFO::2013-11-20 02:25:06,828::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912309::DEBUG::2013-11-20 02:25:06,845::task::495::TaskManager.Task::(_debug) Task 20cf55b2-3a7e-419a-9ceb-e51ed146d4e6: moving from state init -> state preparing
>Thread-15912309::DEBUG::2013-11-20 02:25:06,846::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912309::DEBUG::2013-11-20 02:25:06,846::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912309::DEBUG::2013-11-20 02:25:06,847::task::495::TaskManager.Task::(_debug) Task 20cf55b2-3a7e-419a-9ceb-e51ed146d4e6: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912309::DEBUG::2013-11-20 02:25:06,847::task::495::TaskManager.Task::(_debug) Task 20cf55b2-3a7e-419a-9ceb-e51ed146d4e6: moving from state preparing -> state finished
>Thread-15912309::DEBUG::2013-11-20 02:25:06,848::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912309::DEBUG::2013-11-20 02:25:06,848::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912309::DEBUG::2013-11-20 02:25:06,849::task::495::TaskManager.Task::(_debug) Task 20cf55b2-3a7e-419a-9ceb-e51ed146d4e6: ref 0 aborting False
>Thread-15912309::INFO::2013-11-20 02:25:06,849::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-42838::DEBUG::2013-11-20 02:25:07,530::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-42838::DEBUG::2013-11-20 02:25:07,531::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15912310::DEBUG::2013-11-20 02:25:07,867::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912310::INFO::2013-11-20 02:25:07,869::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912310::DEBUG::2013-11-20 02:25:07,872::task::495::TaskManager.Task::(_debug) Task 7fac510f-67f6-41b1-92fa-a4bfd57e63ab: moving from state init -> state preparing
>Thread-15912310::DEBUG::2013-11-20 02:25:07,874::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912310::DEBUG::2013-11-20 02:25:07,875::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912310::DEBUG::2013-11-20 02:25:07,876::task::495::TaskManager.Task::(_debug) Task 7fac510f-67f6-41b1-92fa-a4bfd57e63ab: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912310::DEBUG::2013-11-20 02:25:07,876::task::495::TaskManager.Task::(_debug) Task 7fac510f-67f6-41b1-92fa-a4bfd57e63ab: moving from state preparing -> state finished
>Thread-15912310::DEBUG::2013-11-20 02:25:07,877::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912310::DEBUG::2013-11-20 02:25:07,877::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912310::DEBUG::2013-11-20 02:25:07,878::task::495::TaskManager.Task::(_debug) Task 7fac510f-67f6-41b1-92fa-a4bfd57e63ab: ref 0 aborting False
>Thread-15912310::INFO::2013-11-20 02:25:07,884::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-42838::DEBUG::2013-11-20 02:25:08,193::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-42838::DEBUG::2013-11-20 02:25:08,196::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-15912312::DEBUG::2013-11-20 02:25:08,903::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912312::INFO::2013-11-20 02:25:08,904::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912312::DEBUG::2013-11-20 02:25:08,905::task::495::TaskManager.Task::(_debug) Task d79303c4-4029-46a5-b288-20a9f0467e37: moving from state init -> state preparing
>Thread-15912312::DEBUG::2013-11-20 02:25:08,905::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912312::DEBUG::2013-11-20 02:25:08,906::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912312::DEBUG::2013-11-20 02:25:08,906::task::495::TaskManager.Task::(_debug) Task d79303c4-4029-46a5-b288-20a9f0467e37: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912312::DEBUG::2013-11-20 02:25:08,907::task::495::TaskManager.Task::(_debug) Task d79303c4-4029-46a5-b288-20a9f0467e37: moving from state preparing -> state finished
>Thread-15912312::DEBUG::2013-11-20 02:25:08,907::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912312::DEBUG::2013-11-20 02:25:08,908::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912312::DEBUG::2013-11-20 02:25:08,908::task::495::TaskManager.Task::(_debug) Task d79303c4-4029-46a5-b288-20a9f0467e37: ref 0 aborting False
>Thread-15912312::INFO::2013-11-20 02:25:08,909::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912313::DEBUG::2013-11-20 02:25:09,924::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912313::INFO::2013-11-20 02:25:09,925::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912313::DEBUG::2013-11-20 02:25:09,925::task::495::TaskManager.Task::(_debug) Task ac8f9089-6df9-4fb4-aabf-4dd8137c94c9: moving from state init -> state preparing
>Thread-15912313::DEBUG::2013-11-20 02:25:09,926::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912313::DEBUG::2013-11-20 02:25:09,927::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912313::DEBUG::2013-11-20 02:25:09,927::task::495::TaskManager.Task::(_debug) Task ac8f9089-6df9-4fb4-aabf-4dd8137c94c9: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912313::DEBUG::2013-11-20 02:25:09,928::task::495::TaskManager.Task::(_debug) Task ac8f9089-6df9-4fb4-aabf-4dd8137c94c9: moving from state preparing -> state finished
>Thread-15912313::DEBUG::2013-11-20 02:25:09,928::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912313::DEBUG::2013-11-20 02:25:09,929::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912313::DEBUG::2013-11-20 02:25:09,929::task::495::TaskManager.Task::(_debug) Task ac8f9089-6df9-4fb4-aabf-4dd8137c94c9: ref 0 aborting False
>Thread-15912313::INFO::2013-11-20 02:25:09,930::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912314::DEBUG::2013-11-20 02:25:10,945::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912314::INFO::2013-11-20 02:25:10,946::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912314::DEBUG::2013-11-20 02:25:10,946::task::495::TaskManager.Task::(_debug) Task 7fcd52ca-9bcd-4c83-98f9-dcc8efb1086f: moving from state init -> state preparing
>Thread-15912314::DEBUG::2013-11-20 02:25:10,947::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912314::DEBUG::2013-11-20 02:25:10,947::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912314::DEBUG::2013-11-20 02:25:10,948::task::495::TaskManager.Task::(_debug) Task 7fcd52ca-9bcd-4c83-98f9-dcc8efb1086f: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912314::DEBUG::2013-11-20 02:25:10,948::task::495::TaskManager.Task::(_debug) Task 7fcd52ca-9bcd-4c83-98f9-dcc8efb1086f: moving from state preparing -> state finished
>Thread-15912314::DEBUG::2013-11-20 02:25:10,949::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912314::DEBUG::2013-11-20 02:25:10,949::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912314::DEBUG::2013-11-20 02:25:10,950::task::495::TaskManager.Task::(_debug) Task 7fcd52ca-9bcd-4c83-98f9-dcc8efb1086f: ref 0 aborting False
>Thread-15912314::INFO::2013-11-20 02:25:10,950::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912316::DEBUG::2013-11-20 02:25:11,965::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912316::INFO::2013-11-20 02:25:11,966::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912316::DEBUG::2013-11-20 02:25:11,979::task::495::TaskManager.Task::(_debug) Task 0995682d-648a-4def-9f23-e3a50beac70d: moving from state init -> state preparing
>Thread-15912316::DEBUG::2013-11-20 02:25:11,980::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912316::DEBUG::2013-11-20 02:25:11,980::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912316::DEBUG::2013-11-20 02:25:11,981::task::495::TaskManager.Task::(_debug) Task 0995682d-648a-4def-9f23-e3a50beac70d: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912316::DEBUG::2013-11-20 02:25:11,981::task::495::TaskManager.Task::(_debug) Task 0995682d-648a-4def-9f23-e3a50beac70d: moving from state preparing -> state finished
>Thread-15912316::DEBUG::2013-11-20 02:25:11,982::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912316::DEBUG::2013-11-20 02:25:11,982::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912316::DEBUG::2013-11-20 02:25:11,983::task::495::TaskManager.Task::(_debug) Task 0995682d-648a-4def-9f23-e3a50beac70d: ref 0 aborting False
>Thread-15912316::INFO::2013-11-20 02:25:11,984::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912317::DEBUG::2013-11-20 02:25:12,999::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912317::INFO::2013-11-20 02:25:12,999::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912317::DEBUG::2013-11-20 02:25:13,000::task::495::TaskManager.Task::(_debug) Task 90586aeb-34e6-4028-b43c-618496c53c2f: moving from state init -> state preparing
>Thread-15912317::DEBUG::2013-11-20 02:25:13,000::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912317::DEBUG::2013-11-20 02:25:13,001::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912317::DEBUG::2013-11-20 02:25:13,002::task::495::TaskManager.Task::(_debug) Task 90586aeb-34e6-4028-b43c-618496c53c2f: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912317::DEBUG::2013-11-20 02:25:13,002::task::495::TaskManager.Task::(_debug) Task 90586aeb-34e6-4028-b43c-618496c53c2f: moving from state preparing -> state finished
>Thread-15912317::DEBUG::2013-11-20 02:25:13,003::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912317::DEBUG::2013-11-20 02:25:13,003::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912317::DEBUG::2013-11-20 02:25:13,004::task::495::TaskManager.Task::(_debug) Task 90586aeb-34e6-4028-b43c-618496c53c2f: ref 0 aborting False
>Thread-15912317::INFO::2013-11-20 02:25:13,004::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-1024767::ERROR::2013-11-20 02:25:13,228::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912318::DEBUG::2013-11-20 02:25:14,019::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912318::INFO::2013-11-20 02:25:14,020::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912318::DEBUG::2013-11-20 02:25:14,021::task::495::TaskManager.Task::(_debug) Task b77f9237-bc39-4fed-8e08-b4a710cee321: moving from state init -> state preparing
>Thread-15912318::DEBUG::2013-11-20 02:25:14,021::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912318::DEBUG::2013-11-20 02:25:14,022::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912318::DEBUG::2013-11-20 02:25:14,022::task::495::TaskManager.Task::(_debug) Task b77f9237-bc39-4fed-8e08-b4a710cee321: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912318::DEBUG::2013-11-20 02:25:14,023::task::495::TaskManager.Task::(_debug) Task b77f9237-bc39-4fed-8e08-b4a710cee321: moving from state preparing -> state finished
>Thread-15912318::DEBUG::2013-11-20 02:25:14,023::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912318::DEBUG::2013-11-20 02:25:14,024::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912318::DEBUG::2013-11-20 02:25:14,024::task::495::TaskManager.Task::(_debug) Task b77f9237-bc39-4fed-8e08-b4a710cee321: ref 0 aborting False
>Thread-15912318::INFO::2013-11-20 02:25:14,025::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912320::DEBUG::2013-11-20 02:25:15,040::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912320::INFO::2013-11-20 02:25:15,041::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912320::DEBUG::2013-11-20 02:25:15,042::task::495::TaskManager.Task::(_debug) Task 68b2d574-2a14-482f-aead-e87309927392: moving from state init -> state preparing
>Thread-15912320::DEBUG::2013-11-20 02:25:15,042::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912320::DEBUG::2013-11-20 02:25:15,043::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912320::DEBUG::2013-11-20 02:25:15,043::task::495::TaskManager.Task::(_debug) Task 68b2d574-2a14-482f-aead-e87309927392: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912320::DEBUG::2013-11-20 02:25:15,044::task::495::TaskManager.Task::(_debug) Task 68b2d574-2a14-482f-aead-e87309927392: moving from state preparing -> state finished
>Thread-15912320::DEBUG::2013-11-20 02:25:15,044::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912320::DEBUG::2013-11-20 02:25:15,045::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912320::DEBUG::2013-11-20 02:25:15,045::task::495::TaskManager.Task::(_debug) Task 68b2d574-2a14-482f-aead-e87309927392: ref 0 aborting False
>Thread-15912320::INFO::2013-11-20 02:25:15,046::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912321::DEBUG::2013-11-20 02:25:16,061::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912321::INFO::2013-11-20 02:25:16,062::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912321::DEBUG::2013-11-20 02:25:16,063::task::495::TaskManager.Task::(_debug) Task 35b28066-4451-4da4-aac6-6ab4e8ef620e: moving from state init -> state preparing
>Thread-15912321::DEBUG::2013-11-20 02:25:16,063::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912321::DEBUG::2013-11-20 02:25:16,064::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912321::DEBUG::2013-11-20 02:25:16,064::task::495::TaskManager.Task::(_debug) Task 35b28066-4451-4da4-aac6-6ab4e8ef620e: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912321::DEBUG::2013-11-20 02:25:16,065::task::495::TaskManager.Task::(_debug) Task 35b28066-4451-4da4-aac6-6ab4e8ef620e: moving from state preparing -> state finished
>Thread-15912321::DEBUG::2013-11-20 02:25:16,065::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912321::DEBUG::2013-11-20 02:25:16,066::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912321::DEBUG::2013-11-20 02:25:16,066::task::495::TaskManager.Task::(_debug) Task 35b28066-4451-4da4-aac6-6ab4e8ef620e: ref 0 aborting False
>Thread-15912321::INFO::2013-11-20 02:25:16,067::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912322::DEBUG::2013-11-20 02:25:17,082::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912322::INFO::2013-11-20 02:25:17,083::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912322::DEBUG::2013-11-20 02:25:17,103::task::495::TaskManager.Task::(_debug) Task 1dbf3482-6fec-472f-84ce-b083df4a7f7f: moving from state init -> state preparing
>Thread-15912322::DEBUG::2013-11-20 02:25:17,104::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912322::DEBUG::2013-11-20 02:25:17,104::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912322::DEBUG::2013-11-20 02:25:17,105::task::495::TaskManager.Task::(_debug) Task 1dbf3482-6fec-472f-84ce-b083df4a7f7f: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912322::DEBUG::2013-11-20 02:25:17,105::task::495::TaskManager.Task::(_debug) Task 1dbf3482-6fec-472f-84ce-b083df4a7f7f: moving from state preparing -> state finished
>Thread-15912322::DEBUG::2013-11-20 02:25:17,106::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912322::DEBUG::2013-11-20 02:25:17,106::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912322::DEBUG::2013-11-20 02:25:17,107::task::495::TaskManager.Task::(_debug) Task 1dbf3482-6fec-472f-84ce-b083df4a7f7f: ref 0 aborting False
>Thread-15912322::INFO::2013-11-20 02:25:17,107::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912324::DEBUG::2013-11-20 02:25:18,122::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912324::INFO::2013-11-20 02:25:18,123::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=378b55ad-db2a-4978-8944-3175e4685c11)
>Thread-15912324::DEBUG::2013-11-20 02:25:18,124::task::495::TaskManager.Task::(_debug) Task d708170d-2708-47ae-b6a5-30bdd8cd32a2: moving from state init -> state preparing
>Thread-15912324::DEBUG::2013-11-20 02:25:18,124::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 378b55ad-db2a-4978-8944-3175e4685c11
>Thread-15912324::DEBUG::2013-11-20 02:25:18,125::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}
>Thread-15912324::DEBUG::2013-11-20 02:25:18,125::task::495::TaskManager.Task::(_debug) Task d708170d-2708-47ae-b6a5-30bdd8cd32a2: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>Thread-15912324::DEBUG::2013-11-20 02:25:18,126::task::495::TaskManager.Task::(_debug) Task d708170d-2708-47ae-b6a5-30bdd8cd32a2: moving from state preparing -> state finished
>Thread-15912324::DEBUG::2013-11-20 02:25:18,127::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912324::DEBUG::2013-11-20 02:25:18,127::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912324::DEBUG::2013-11-20 02:25:18,128::task::495::TaskManager.Task::(_debug) Task d708170d-2708-47ae-b6a5-30bdd8cd32a2: ref 0 aborting False
>Thread-15912324::INFO::2013-11-20 02:25:18,128::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '378b55ad-db2a-4978-8944-3175e4685c11'}}
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:25:40,259::safelease::61::Storage.Misc.excCmd::(acquire) FAILED: <err> = ''; <rc> = 1
>378b55ad-db2a-4978-8944-3175e4685c11::ERROR::2013-11-20 02:25:40,260::task::868::TaskManager.Task::(_setError) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/task.py", line 876, in _run
>  File "/usr/share/vdsm/storage/task.py", line 303, in run
>  File "/usr/share/vdsm/storage/spm.py", line 463, in start
>  File "/usr/share/vdsm/storage/sp.py", line 203, in acquireClusterLock
>  File "/usr/share/vdsm/storage/sd.py", line 376, in acquireClusterLock
>  File "/usr/share/vdsm/storage/safelease.py", line 63, in acquire
>AcquireLockFailure: Could not obtain lock: 'id=6b892067-ac7e-4039-a7be-5b7387525b16, rc=1, out=[], err=[]'
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:25:40,279::task::495::TaskManager.Task::(_debug) Task 378b55ad-db2a-4978-8944-3175e4685c11: Task._run: 378b55ad-db2a-4978-8944-3175e4685c11 () {} failed - stopping task
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:25:40,279::task::495::TaskManager.Task::(_debug) Task 378b55ad-db2a-4978-8944-3175e4685c11: stopping in state running (force False)
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:25:40,280::task::495::TaskManager.Task::(_debug) Task 378b55ad-db2a-4978-8944-3175e4685c11: ref 1 aborting True
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:25:40,280::task::918::TaskManager.Task::(_runJobs) aborting: Task is aborted: 'Could not obtain lock' - code 651
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:25:40,281::task::495::TaskManager.Task::(_debug) Task 378b55ad-db2a-4978-8944-3175e4685c11: ref 0 aborting True
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:25:40,281::task::495::TaskManager.Task::(_debug) Task 378b55ad-db2a-4978-8944-3175e4685c11: Task._doAbort: force False
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:25:40,282::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:25:40,282::task::495::TaskManager.Task::(_debug) Task 378b55ad-db2a-4978-8944-3175e4685c11: moving from state running -> state aborting
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:25:40,283::task::495::TaskManager.Task::(_debug) Task 378b55ad-db2a-4978-8944-3175e4685c11: _aborting: recover policy auto
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:25:40,283::task::495::TaskManager.Task::(_debug) Task 378b55ad-db2a-4978-8944-3175e4685c11: moving from state aborting -> state racquiring
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:25:40,284::task::495::TaskManager.Task::(_debug) Task 378b55ad-db2a-4978-8944-3175e4685c11: moving from state racquiring -> state recovering
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:25:40,284::task::495::TaskManager.Task::(_debug) Task 378b55ad-db2a-4978-8944-3175e4685c11: _recover
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:25:40,285::task::495::TaskManager.Task::(_debug) Task 378b55ad-db2a-4978-8944-3175e4685c11: running recovery None
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:25:40,285::task::495::TaskManager.Task::(_debug) Task 378b55ad-db2a-4978-8944-3175e4685c11: Recover Done: state recovering
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:25:40,286::task::495::TaskManager.Task::(_debug) Task 378b55ad-db2a-4978-8944-3175e4685c11: moving from state recovering -> state recovered
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:25:40,286::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:25:40,287::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:25:40,287::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:25:40,288::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:25:40,289::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:25:40,289::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>378b55ad-db2a-4978-8944-3175e4685c11::DEBUG::2013-11-20 02:25:40,290::threadPool::47::Misc.ThreadPool::(setRunningTask) Number of running tasks: 0
>Thread-42838::DEBUG::2013-11-20 02:25:58,261::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 02:25:58,907::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 02:27:27,590::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` (Operation Stuck)
>Thread-42839::ERROR::2013-11-20 02:27:36,100::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `45cdff8b-89c0-4926-a8c7-80354064bfa6` (Operation Stuck)
>Thread-1024767::ERROR::2013-11-20 02:28:02,248::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-1024767::ERROR::2013-11-20 02:28:12,268::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-1024767::ERROR::2013-11-20 02:28:22,285::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-42840::DEBUG::2013-11-20 02:28:26,983::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 02:28:27,671::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 02:28:32,302::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-1024767::ERROR::2013-11-20 02:28:42,318::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-1024767::ERROR::2013-11-20 02:28:52,335::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-1024767::ERROR::2013-11-20 02:29:02,353::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-1024767::ERROR::2013-11-20 02:29:12,370::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-1024767::ERROR::2013-11-20 02:29:22,387::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912329::INFO::2013-11-20 02:29:29,144::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912329::DEBUG::2013-11-20 02:29:29,145::task::495::TaskManager.Task::(_debug) Task 0fcb44b0-69b2-47ca-be34-9392c01f5df8: moving from state init -> state preparing
>Thread-15912329::DEBUG::2013-11-20 02:29:29,145::task::495::TaskManager.Task::(_debug) Task 0fcb44b0-69b2-47ca-be34-9392c01f5df8: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000340938568115', 'lastCheck': 1384914559.4371641, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00183796882629', 'lastCheck': 1384914562.3889771, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00139403343201', 'lastCheck': 1384914562.3900311, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000348091125488', 'lastCheck': 1384914567.7438021, 'valid': True, 'code': 0}}
>Thread-15912329::DEBUG::2013-11-20 02:29:29,146::task::495::TaskManager.Task::(_debug) Task 0fcb44b0-69b2-47ca-be34-9392c01f5df8: moving from state preparing -> state finished
>Thread-15912329::DEBUG::2013-11-20 02:29:29,147::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912329::DEBUG::2013-11-20 02:29:29,147::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912329::DEBUG::2013-11-20 02:29:29,148::task::495::TaskManager.Task::(_debug) Task 0fcb44b0-69b2-47ca-be34-9392c01f5df8: ref 0 aborting False
>Thread-15912329::INFO::2013-11-20 02:29:29,148::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000340938568115', 'lastCheck': 1384914559.4371641, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00183796882629', 'lastCheck': 1384914562.3889771, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00139403343201', 'lastCheck': 1384914562.3900311, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000348091125488', 'lastCheck': 1384914567.7438021, 'valid': True, 'code': 0}}
>Thread-15912330::DEBUG::2013-11-20 02:29:29,402::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVdsCapabilities with () {}
>Thread-15912330::DEBUG::2013-11-20 02:29:29,781::utils::579::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-img-rhev' (cwd None)
>Thread-15912330::DEBUG::2013-11-20 02:29:30,236::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912330::DEBUG::2013-11-20 02:29:30,238::utils::579::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-kvm-rhev' (cwd None)
>Thread-15912330::DEBUG::2013-11-20 02:29:30,645::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912330::DEBUG::2013-11-20 02:29:30,646::utils::579::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" vdsm' (cwd None)
>Thread-15912330::DEBUG::2013-11-20 02:29:31,088::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912330::DEBUG::2013-11-20 02:29:31,090::utils::579::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" spice-server' (cwd None)
>Thread-15912330::DEBUG::2013-11-20 02:29:31,502::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912330::DEBUG::2013-11-20 02:29:31,503::utils::579::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" libvirt' (cwd None)
>Thread-15912330::DEBUG::2013-11-20 02:29:31,928::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912330::DEBUG::2013-11-20 02:29:31,930::clientIF::59::vds::(wrapper) return getVdsCapabilities with {'status': {'message': 'Done', 'code': 0}, 'info': {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:c936ebb147a8'}], 'FC': []}, 'packages2': {'kernel': {'release': '279.11.1.el6.x86_64', 'buildtime': 1348297826.0, 'version': '2.6.32'}, 'spice-server': {'release': '10.el6', 'buildtime': '1337611492', 'version': '0.10.1'}, 'vdsm': {'release': '113.4.el6_3', 'buildtime': '1347534072', 'version': '4.9'}, 'qemu-kvm': {'release': '2.295.el6_3.2', 'buildtime': '1346673796', 'version': '0.12.1.2'}, 'libvirt': {'release': '21.el6_3.5', 'buildtime': '1348687577', 'version': '0.9.10'}, 'qemu-img': {'release': '2.295.el6_3.2', 'buildtime': '1346673796', 'version': '0.12.1.2'}}, 'cpuModel': 'Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz', 'hooks': {'before_vm_start': {'50_vhostmd': {'md5': '1635e45584995d9ad2629939cb0bcdbf'}}, 'after_vm_destroy': {'50_vhostmd': {'md5': '7e7509430304099f38e4a9de966af090'}}, 'before_vm_dehibernate': {'50_vhostmd': {'md5': '1635e45584995d9ad2629939cb0bcdbf'}}, 'before_vm_migrate_destination': {'50_vhostmd': {'md5': '1635e45584995d9ad2629939cb0bcdbf'}}}, 'vmTypes': ['kvm'], 'supportedProtocols': ['2.2', '2.3'], 'networks': {'rhevm': {'addr': '192.168.100.133', 'cfg': {'IPADDR': '192.168.100.133', 'GATEWAY': '192.168.100.1', 'DELAY': '0', 'NETMASK': '255.255.128.0', 'STP': 'no', 'DEVICE': 'rhevm', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'ports': ['bond0'], 'netmask': '255.255.128.0', 'stp': 'off', 'gateway': '192.168.100.1'}, 'DMZ': {'addr': '172.16.1.133', 'cfg': {'IPADDR': '172.16.1.133', 'DELAY': '0', 'NETMASK': '255.255.255.0', 'STP': 'no', 'DEVICE': 'DMZ', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'ports': ['eth0'], 'netmask': '255.255.255.0', 'stp': 'off', 'gateway': '0.0.0.0'}}, 'uuid': '30343536-3138-5A43-4A32-333630373442_AC:16:2D:76:EB:F0', 'lastClientIface': 'rhevm', 'nics': {'eth3': {'hwaddr': 'AC:16:2D:76:EB:F2', 'netmask': '', 'speed': 1000, 'addr': '', 'permhwaddr': 'AC:16:2D:76:EB:F3'}, 'eth2': {'hwaddr': 'AC:16:2D:76:EB:F2', 'netmask': '', 'speed': 1000, 'addr': '', 'permhwaddr': 'AC:16:2D:76:EB:F2'}, 'eth1': {'hwaddr': 'AC:16:2D:76:EB:F1', 'netmask': '', 'speed': 0, 'addr': ''}, 'eth0': {'hwaddr': 'AC:16:2D:76:EB:F0', 'netmask': '', 'speed': 1000, 'addr': ''}}, 'software_revision': '113.4', 'management_ip': '192.168.100.133', 'clusterLevels': ['3.0'], 'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,constant_tsc,arch_perfmon,pebs,bts,rep_good,xtopology,nonstop_tsc,aperfmperf,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,dca,sse4_1,sse4_2,x2apic,popcnt,aes,xsave,avx,lahf_lm,ida,arat,epb,xsaveopt,pln,pts,dts,tpr_shadow,vnmi,flexpriority,ept,vpid,model_486,model_pentium,model_pentium2,model_pentium3,model_pentiumpro,model_qemu32,model_cpu64-rhel5,model_cpu64-rhel6,model_kvm32,model_coreduo,model_kvm64,model_core2duo,model_n270,model_Conroe,model_Penryn,model_Nehalem,model_Westmere,model_SandyBridge,model_Opteron_G1', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:c936ebb147a8', 'memSize': '64392', 'reservedMem': '256', 'bondings': {'bond4': {'hwaddr': '00:00:00:00:00:00', 'cfg': {}, 'netmask': '', 'addr': '', 'slaves': []}, 'bond0': {'hwaddr': 'AC:16:2D:76:EB:F2', 'cfg': {'DEVICE': 'bond0', 'BRIDGE': 'rhevm', 'BOOTPROTO': 'none', 'ONBOOT': 'yes', 'BONDING_OPTS': 'mode=1 miimon=100'}, 'netmask': '', 'addr': '', 'slaves': ['eth2', 'eth3']}, 'bond1': {'hwaddr': '00:00:00:00:00:00', 'cfg': {}, 'netmask': '', 'addr': '', 'slaves': []}, 'bond2': {'hwaddr': '00:00:00:00:00:00', 'cfg': {}, 'netmask': '', 'addr': '', 'slaves': []}, 'bond3': {'hwaddr': '00:00:00:00:00:00', 'cfg': {}, 'netmask': '', 'addr': '', 'slaves': []}}, 'software_version': '4.9', 'cpuSpeed': '1995.126', 'cpuSockets': '2', 'vlans': {}, 'cpuCores': '12', 'kvmEnabled': 'true', 'guestOverhead': '65', 'supportedRHEVMs': ['3.0'], 'version_name': 'Snow Man', 'emulatedMachines': [u'rhel6.3.0', u'pc', u'rhel6.2.0', u'rhel6.1.0', u'rhel6.0.0', u'rhel5.5.0', u'rhel5.4.4', u'rhel5.4.0'], 'operatingSystem': {'release': '20121012.0.el6_3', 'version': '6.3', 'name': 'RHEV Hypervisor'}, 'lastClient': '192.168.100.135'}}
>Thread-1024767::ERROR::2013-11-20 02:29:32,443::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912332::DEBUG::2013-11-20 02:29:36,180::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912332::INFO::2013-11-20 02:29:36,181::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912332::DEBUG::2013-11-20 02:29:36,182::task::495::TaskManager.Task::(_debug) Task c5c3141c-20c3-4b79-af5a-0d149ca1316e: moving from state init -> state preparing
>Thread-15912332::DEBUG::2013-11-20 02:29:36,183::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=Free spmLver=286 spmId=1
>Thread-15912332::DEBUG::2013-11-20 02:29:36,183::task::495::TaskManager.Task::(_debug) Task c5c3141c-20c3-4b79-af5a-0d149ca1316e: finished: {'spm_st': {'spmId': 1, 'spmStatus': 'Free', 'spmLver': 286}}
>Thread-15912332::DEBUG::2013-11-20 02:29:36,184::task::495::TaskManager.Task::(_debug) Task c5c3141c-20c3-4b79-af5a-0d149ca1316e: moving from state preparing -> state finished
>Thread-15912332::DEBUG::2013-11-20 02:29:36,184::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912332::DEBUG::2013-11-20 02:29:36,185::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912332::DEBUG::2013-11-20 02:29:36,186::task::495::TaskManager.Task::(_debug) Task c5c3141c-20c3-4b79-af5a-0d149ca1316e: ref 0 aborting False
>Thread-15912332::INFO::2013-11-20 02:29:36,186::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 1, 'spmStatus': 'Free', 'spmLver': 286}}
>Thread-1024767::ERROR::2013-11-20 02:29:42,461::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912337::INFO::2013-11-20 02:29:48,344::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912337::DEBUG::2013-11-20 02:29:48,345::task::495::TaskManager.Task::(_debug) Task 5fbf9b8e-4d22-4348-b3ac-f95ce131f0b6: moving from state init -> state preparing
>Thread-15912337::DEBUG::2013-11-20 02:29:48,346::task::495::TaskManager.Task::(_debug) Task 5fbf9b8e-4d22-4348-b3ac-f95ce131f0b6: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000386953353882', 'lastCheck': 1384914579.736455, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00133490562439', 'lastCheck': 1384914582.4611671, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00112891197205', 'lastCheck': 1384914582.4634669, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00032114982605', 'lastCheck': 1384914587.8878109, 'valid': True, 'code': 0}}
>Thread-15912337::DEBUG::2013-11-20 02:29:48,346::task::495::TaskManager.Task::(_debug) Task 5fbf9b8e-4d22-4348-b3ac-f95ce131f0b6: moving from state preparing -> state finished
>Thread-15912337::DEBUG::2013-11-20 02:29:48,347::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912337::DEBUG::2013-11-20 02:29:48,347::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912337::DEBUG::2013-11-20 02:29:48,348::task::495::TaskManager.Task::(_debug) Task 5fbf9b8e-4d22-4348-b3ac-f95ce131f0b6: ref 0 aborting False
>Thread-15912337::INFO::2013-11-20 02:29:48,349::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000386953353882', 'lastCheck': 1384914579.736455, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00133490562439', 'lastCheck': 1384914582.4611671, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00112891197205', 'lastCheck': 1384914582.4634669, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00032114982605', 'lastCheck': 1384914587.8878109, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-20 02:29:52,482::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912342::DEBUG::2013-11-20 02:30:00,607::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912342::INFO::2013-11-20 02:30:00,608::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: validateStorageServerConnection, args: (domType=1, spUUID=00000000-0000-0000-0000-000000000000, conList=[{'connection': 'kairo.brd-klee.dk:/export', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'ec9380b7-cfc1-41d8-9237-d5c7c505ef36', 'port': ''}])
>Thread-15912342::DEBUG::2013-11-20 02:30:00,609::task::495::TaskManager.Task::(_debug) Task 95b15c31-2160-4309-8000-70e25468915c: moving from state init -> state preparing
>Thread-15912342::INFO::2013-11-20 02:30:00,609::storage_connection::127::Storage.ServerConnection::(validate) Request to validate NFS storage server
>Thread-15912342::INFO::2013-11-20 02:30:00,610::storage_connection::53::Storage.ServerConnection::(__validateConnectionParams) conList=[{'connection': 'kairo.brd-klee.dk:/export', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'ec9380b7-cfc1-41d8-9237-d5c7c505ef36', 'port': ''}]
>Thread-15912342::DEBUG::2013-11-20 02:30:00,614::fileUtils::83::Storage.Misc.excCmd::(mount) '/usr/bin/sudo -n /bin/mount -o soft,timeo=600,retrans=6,nosharecache,vers=3 -t nfs kairo.brd-klee.dk:/export /tmp/tmpffZdt0' (cwd None)
>Thread-15912342::DEBUG::2013-11-20 02:30:01,089::fileUtils::83::Storage.Misc.excCmd::(mount) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912342::DEBUG::2013-11-20 02:30:01,102::fileUtils::112::Storage.Misc.excCmd::(umount) '/usr/bin/sudo -n /bin/umount -t nfs -f /tmp/tmpffZdt0' (cwd None)
>Thread-15912342::DEBUG::2013-11-20 02:30:01,522::fileUtils::112::Storage.Misc.excCmd::(umount) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912342::DEBUG::2013-11-20 02:30:01,526::task::495::TaskManager.Task::(_debug) Task 95b15c31-2160-4309-8000-70e25468915c: finished: {'statuslist': [{'status': 0, 'id': 'ec9380b7-cfc1-41d8-9237-d5c7c505ef36'}]}
>Thread-15912342::DEBUG::2013-11-20 02:30:01,533::task::495::TaskManager.Task::(_debug) Task 95b15c31-2160-4309-8000-70e25468915c: moving from state preparing -> state finished
>Thread-15912342::DEBUG::2013-11-20 02:30:01,533::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912342::DEBUG::2013-11-20 02:30:01,534::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912342::DEBUG::2013-11-20 02:30:01,534::task::495::TaskManager.Task::(_debug) Task 95b15c31-2160-4309-8000-70e25468915c: ref 0 aborting False
>Thread-15912342::INFO::2013-11-20 02:30:01,535::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: validateStorageServerConnection, Return response: {'status': {'message': 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': 'ec9380b7-cfc1-41d8-9237-d5c7c505ef36'}]}
>Thread-15912344::DEBUG::2013-11-20 02:30:01,565::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912344::INFO::2013-11-20 02:30:01,566::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: connectStorageServer, args: (domType=1, spUUID=00000000-0000-0000-0000-000000000000, conList=[{'connection': 'kairo.brd-klee.dk:/export', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'ec9380b7-cfc1-41d8-9237-d5c7c505ef36', 'port': ''}])
>Thread-15912344::DEBUG::2013-11-20 02:30:01,566::task::495::TaskManager.Task::(_debug) Task dcffbae9-8513-4685-8229-b33daf6d5b29: moving from state init -> state preparing
>Thread-15912344::INFO::2013-11-20 02:30:01,567::storage_connection::95::Storage.ServerConnection::(connect) Request to connect NFS storage server
>Thread-15912344::INFO::2013-11-20 02:30:01,567::storage_connection::53::Storage.ServerConnection::(__validateConnectionParams) conList=[{'connection': 'kairo.brd-klee.dk:/export', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'ec9380b7-cfc1-41d8-9237-d5c7c505ef36', 'port': ''}]
>Thread-15912344::DEBUG::2013-11-20 02:30:01,581::lvm::547::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15912344::DEBUG::2013-11-20 02:30:01,582::lvm::549::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15912344::DEBUG::2013-11-20 02:30:01,583::lvm::559::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15912344::DEBUG::2013-11-20 02:30:01,583::lvm::561::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15912344::DEBUG::2013-11-20 02:30:01,584::lvm::580::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15912344::DEBUG::2013-11-20 02:30:01,585::lvm::582::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15912344::DEBUG::2013-11-20 02:30:01,585::task::495::TaskManager.Task::(_debug) Task dcffbae9-8513-4685-8229-b33daf6d5b29: finished: {'statuslist': [{'status': 0, 'id': 'ec9380b7-cfc1-41d8-9237-d5c7c505ef36'}]}
>Thread-15912344::DEBUG::2013-11-20 02:30:01,586::task::495::TaskManager.Task::(_debug) Task dcffbae9-8513-4685-8229-b33daf6d5b29: moving from state preparing -> state finished
>Thread-15912344::DEBUG::2013-11-20 02:30:01,586::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912344::DEBUG::2013-11-20 02:30:01,587::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912344::DEBUG::2013-11-20 02:30:01,587::task::495::TaskManager.Task::(_debug) Task dcffbae9-8513-4685-8229-b33daf6d5b29: ref 0 aborting False
>Thread-15912344::INFO::2013-11-20 02:30:01,588::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: connectStorageServer, Return response: {'status': {'message': 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': 'ec9380b7-cfc1-41d8-9237-d5c7c505ef36'}]}
>Thread-1024767::ERROR::2013-11-20 02:30:02,501::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912345::INFO::2013-11-20 02:30:03,719::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912345::DEBUG::2013-11-20 02:30:03,720::task::495::TaskManager.Task::(_debug) Task 8d33322f-577e-4b7b-9479-b12d805f7651: moving from state init -> state preparing
>Thread-15912345::DEBUG::2013-11-20 02:30:03,720::task::495::TaskManager.Task::(_debug) Task 8d33322f-577e-4b7b-9479-b12d805f7651: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000412940979004', 'lastCheck': 1384914599.7623041, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00247693061829', 'lastCheck': 1384914602.5034449, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00165390968323', 'lastCheck': 1384914602.504374, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000339031219482', 'lastCheck': 1384914597.899472, 'valid': True, 'code': 0}}
>Thread-15912345::DEBUG::2013-11-20 02:30:03,721::task::495::TaskManager.Task::(_debug) Task 8d33322f-577e-4b7b-9479-b12d805f7651: moving from state preparing -> state finished
>Thread-15912345::DEBUG::2013-11-20 02:30:03,722::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912345::DEBUG::2013-11-20 02:30:03,722::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912345::DEBUG::2013-11-20 02:30:03,723::task::495::TaskManager.Task::(_debug) Task 8d33322f-577e-4b7b-9479-b12d805f7651: ref 0 aborting False
>Thread-15912345::INFO::2013-11-20 02:30:03,723::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000412940979004', 'lastCheck': 1384914599.7623041, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00247693061829', 'lastCheck': 1384914602.5034449, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00165390968323', 'lastCheck': 1384914602.504374, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000339031219482', 'lastCheck': 1384914597.899472, 'valid': True, 'code': 0}}
>Thread-42840::DEBUG::2013-11-20 02:30:07,911::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-42840::DEBUG::2013-11-20 02:30:07,919::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 02:30:08,570::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-42840::DEBUG::2013-11-20 02:30:08,575::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-42838::DEBUG::2013-11-20 02:30:09,773::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-42838::DEBUG::2013-11-20 02:30:09,774::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 02:30:10,393::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-42838::DEBUG::2013-11-20 02:30:10,396::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1024767::ERROR::2013-11-20 02:30:12,520::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912351::INFO::2013-11-20 02:30:19,069::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912351::DEBUG::2013-11-20 02:30:19,070::task::495::TaskManager.Task::(_debug) Task 9c55bdce-6f76-44bc-9b1c-5fb5b7bf7de5: moving from state init -> state preparing
>Thread-15912351::DEBUG::2013-11-20 02:30:19,070::task::495::TaskManager.Task::(_debug) Task 9c55bdce-6f76-44bc-9b1c-5fb5b7bf7de5: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000403881072998', 'lastCheck': 1384914610.3995609, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00163793563843', 'lastCheck': 1384914612.5220251, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00124192237854', 'lastCheck': 1384914612.523622, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000380039215088', 'lastCheck': 1384914618.5887921, 'valid': True, 'code': 0}}
>Thread-15912351::DEBUG::2013-11-20 02:30:19,071::task::495::TaskManager.Task::(_debug) Task 9c55bdce-6f76-44bc-9b1c-5fb5b7bf7de5: moving from state preparing -> state finished
>Thread-15912351::DEBUG::2013-11-20 02:30:19,072::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912351::DEBUG::2013-11-20 02:30:19,072::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912351::DEBUG::2013-11-20 02:30:19,073::task::495::TaskManager.Task::(_debug) Task 9c55bdce-6f76-44bc-9b1c-5fb5b7bf7de5: ref 0 aborting False
>Thread-15912351::INFO::2013-11-20 02:30:19,073::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000403881072998', 'lastCheck': 1384914610.3995609, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00163793563843', 'lastCheck': 1384914612.5220251, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00124192237854', 'lastCheck': 1384914612.523622, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000380039215088', 'lastCheck': 1384914618.5887921, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-20 02:30:22,538::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-1024767::ERROR::2013-11-20 02:30:32,555::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912357::INFO::2013-11-20 02:30:34,536::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912357::DEBUG::2013-11-20 02:30:34,537::task::495::TaskManager.Task::(_debug) Task 186a33ad-b9c9-40d4-8078-c1d6cc06d437: moving from state init -> state preparing
>Thread-15912357::DEBUG::2013-11-20 02:30:34,538::task::495::TaskManager.Task::(_debug) Task 186a33ad-b9c9-40d4-8078-c1d6cc06d437: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000308036804199', 'lastCheck': 1384914630.4250901, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014340877533', 'lastCheck': 1384914632.5565829, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0014340877533', 'lastCheck': 1384914632.5583401, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000425815582275', 'lastCheck': 1384914628.6003871, 'valid': True, 'code': 0}}
>Thread-15912357::DEBUG::2013-11-20 02:30:34,538::task::495::TaskManager.Task::(_debug) Task 186a33ad-b9c9-40d4-8078-c1d6cc06d437: moving from state preparing -> state finished
>Thread-15912357::DEBUG::2013-11-20 02:30:34,539::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912357::DEBUG::2013-11-20 02:30:34,540::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912357::DEBUG::2013-11-20 02:30:34,540::task::495::TaskManager.Task::(_debug) Task 186a33ad-b9c9-40d4-8078-c1d6cc06d437: ref 0 aborting False
>Thread-15912357::INFO::2013-11-20 02:30:34,541::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000308036804199', 'lastCheck': 1384914630.4250901, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014340877533', 'lastCheck': 1384914632.5565829, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0014340877533', 'lastCheck': 1384914632.5583401, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000425815582275', 'lastCheck': 1384914628.6003871, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-20 02:30:42,573::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912363::INFO::2013-11-20 02:30:49,885::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912363::DEBUG::2013-11-20 02:30:49,886::task::495::TaskManager.Task::(_debug) Task fde00388-bc36-4f2a-b3e7-e313b8e79f26: moving from state init -> state preparing
>Thread-15912363::DEBUG::2013-11-20 02:30:49,887::task::495::TaskManager.Task::(_debug) Task fde00388-bc36-4f2a-b3e7-e313b8e79f26: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00033712387085', 'lastCheck': 1384914640.437886, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00182199478149', 'lastCheck': 1384914642.5733891, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00113701820374', 'lastCheck': 1384914642.5751951, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000377893447876', 'lastCheck': 1384914648.6234379, 'valid': True, 'code': 0}}
>Thread-15912363::DEBUG::2013-11-20 02:30:49,888::task::495::TaskManager.Task::(_debug) Task fde00388-bc36-4f2a-b3e7-e313b8e79f26: moving from state preparing -> state finished
>Thread-15912363::DEBUG::2013-11-20 02:30:49,888::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912363::DEBUG::2013-11-20 02:30:49,889::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912363::DEBUG::2013-11-20 02:30:49,889::task::495::TaskManager.Task::(_debug) Task fde00388-bc36-4f2a-b3e7-e313b8e79f26: ref 0 aborting False
>Thread-15912363::INFO::2013-11-20 02:30:49,890::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00033712387085', 'lastCheck': 1384914640.437886, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00182199478149', 'lastCheck': 1384914642.5733891, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00113701820374', 'lastCheck': 1384914642.5751951, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000377893447876', 'lastCheck': 1384914648.6234379, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-20 02:30:52,589::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-42838::DEBUG::2013-11-20 02:31:00,462::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 02:31:01,104::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 02:31:02,677::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912369::INFO::2013-11-20 02:31:05,385::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912369::DEBUG::2013-11-20 02:31:05,386::task::495::TaskManager.Task::(_debug) Task ae2075bc-10d8-4cf8-ad05-2bed325797df: moving from state init -> state preparing
>Thread-15912369::DEBUG::2013-11-20 02:31:05,387::task::495::TaskManager.Task::(_debug) Task ae2075bc-10d8-4cf8-ad05-2bed325797df: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000346899032593', 'lastCheck': 1384914661.108465, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00259017944336', 'lastCheck': 1384914662.6792359, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00160193443298', 'lastCheck': 1384914662.680382, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000383853912354', 'lastCheck': 1384914658.6350751, 'valid': True, 'code': 0}}
>Thread-15912369::DEBUG::2013-11-20 02:31:05,388::task::495::TaskManager.Task::(_debug) Task ae2075bc-10d8-4cf8-ad05-2bed325797df: moving from state preparing -> state finished
>Thread-15912369::DEBUG::2013-11-20 02:31:05,388::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912369::DEBUG::2013-11-20 02:31:05,389::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912369::DEBUG::2013-11-20 02:31:05,389::task::495::TaskManager.Task::(_debug) Task ae2075bc-10d8-4cf8-ad05-2bed325797df: ref 0 aborting False
>Thread-15912369::INFO::2013-11-20 02:31:05,390::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000346899032593', 'lastCheck': 1384914661.108465, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00259017944336', 'lastCheck': 1384914662.6792359, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00160193443298', 'lastCheck': 1384914662.680382, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000383853912354', 'lastCheck': 1384914658.6350751, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-20 02:31:12,695::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912375::INFO::2013-11-20 02:31:20,852::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912375::DEBUG::2013-11-20 02:31:20,853::task::495::TaskManager.Task::(_debug) Task f4076b7c-d76c-4790-aeb9-7ed3b3f01c08: moving from state init -> state preparing
>Thread-15912375::DEBUG::2013-11-20 02:31:20,853::task::495::TaskManager.Task::(_debug) Task f4076b7c-d76c-4790-aeb9-7ed3b3f01c08: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00031304359436', 'lastCheck': 1384914671.1212211, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00165295600891', 'lastCheck': 1384914672.6964979, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00109505653381', 'lastCheck': 1384914672.6976659, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000471115112305', 'lastCheck': 1384914678.6884501, 'valid': True, 'code': 0}}
>Thread-15912375::DEBUG::2013-11-20 02:31:20,854::task::495::TaskManager.Task::(_debug) Task f4076b7c-d76c-4790-aeb9-7ed3b3f01c08: moving from state preparing -> state finished
>Thread-15912375::DEBUG::2013-11-20 02:31:20,855::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912375::DEBUG::2013-11-20 02:31:20,855::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912375::DEBUG::2013-11-20 02:31:20,856::task::495::TaskManager.Task::(_debug) Task f4076b7c-d76c-4790-aeb9-7ed3b3f01c08: ref 0 aborting False
>Thread-15912375::INFO::2013-11-20 02:31:20,856::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00031304359436', 'lastCheck': 1384914671.1212211, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00165295600891', 'lastCheck': 1384914672.6964979, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00109505653381', 'lastCheck': 1384914672.6976659, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000471115112305', 'lastCheck': 1384914678.6884501, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-20 02:31:22,712::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-1024767::ERROR::2013-11-20 02:31:32,731::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912381::INFO::2013-11-20 02:31:36,293::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912381::DEBUG::2013-11-20 02:31:36,294::task::495::TaskManager.Task::(_debug) Task c31b5b96-0b63-4b77-b8be-5e0dc1008d7b: moving from state init -> state preparing
>Thread-15912381::DEBUG::2013-11-20 02:31:36,295::task::495::TaskManager.Task::(_debug) Task c31b5b96-0b63-4b77-b8be-5e0dc1008d7b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000322103500366', 'lastCheck': 1384914691.1461561, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00139403343201', 'lastCheck': 1384914692.7330041, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00122117996216', 'lastCheck': 1384914692.733649, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000478029251099', 'lastCheck': 1384914688.6999321, 'valid': True, 'code': 0}}
>Thread-15912381::DEBUG::2013-11-20 02:31:36,296::task::495::TaskManager.Task::(_debug) Task c31b5b96-0b63-4b77-b8be-5e0dc1008d7b: moving from state preparing -> state finished
>Thread-15912381::DEBUG::2013-11-20 02:31:36,296::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912381::DEBUG::2013-11-20 02:31:36,297::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912381::DEBUG::2013-11-20 02:31:36,297::task::495::TaskManager.Task::(_debug) Task c31b5b96-0b63-4b77-b8be-5e0dc1008d7b: ref 0 aborting False
>Thread-15912381::INFO::2013-11-20 02:31:36,298::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000322103500366', 'lastCheck': 1384914691.1461561, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00139403343201', 'lastCheck': 1384914692.7330041, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00122117996216', 'lastCheck': 1384914692.733649, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000478029251099', 'lastCheck': 1384914688.6999321, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-20 02:31:42,749::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912385::DEBUG::2013-11-20 02:31:44,350::clientIF::54::vds::(wrapper) [192.168.100.135]::call create with ({'bridge': 'rhevm', 'acpiEnable': 'true', 'emulatedMachine': 'rhel6.2.0', 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'transparentHugePages': 'true', 'spiceSslCipherSuite': 'DEFAULT', 'cpuType': 'Westmere', 'smp': '4', 'macAddr': '00:1a:4a:a8:64:06', 'boot': 'c', 'custom': {}, 'vmType': 'kvm', 'memSize': 2048, 'smpCoresPerSocket': '2', 'vmName': 'KLEE-DC01', 'spiceMonitors': '1', 'nice': '0', 'drives': [{'index': '0', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'format': 'raw', 'boot': 'true', 'volumeID': '038d75ec-9421-4d79-9865-0dbebd5271d9', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'propagateErrors': 'off', 'if': 'virtio'}], 'smartcardEnable': 'false', 'nicModel': 'pv', 'keyboardLayout': 'en-us', 'kvmEnable': 'true', 'timeOffset': '3600', 'spiceSecureChannels': 'smain,sinputs', 'display': 'qxl'},) {}
>Thread-15912385::INFO::2013-11-20 02:31:44,699::clientIF::523::vds::(_getNetworkIp) network None: using 0
>Thread-15912385::INFO::2013-11-20 02:31:44,700::clientIF::656::vds::(create) vmContainerLock acquired by vm 98ec88b5-2c08-4ff2-b607-c3f4e281f6af
>Thread-15912386::DEBUG::2013-11-20 02:31:44,705::vm::342::vm.Vm::(_startUnderlyingVm) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Start
>Thread-15912385::DEBUG::2013-11-20 02:31:44,706::clientIF::669::vds::(create) Total desktops after creation of 98ec88b5-2c08-4ff2-b607-c3f4e281f6af is 1
>Thread-15912386::DEBUG::2013-11-20 02:31:44,707::vm::346::vm.Vm::(_startUnderlyingVm) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::_ongoingCreations acquired
>Thread-15912385::DEBUG::2013-11-20 02:31:44,708::clientIF::59::vds::(wrapper) return create with {'status': {'message': 'Done', 'code': 0}, 'vmList': {'status': 'WaitForLaunch', 'bridge': 'rhevm', 'acpiEnable': 'true', 'emulatedMachine': 'rhel6.2.0', 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'pid': '0', 'drives': [{'index': '0', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'format': 'raw', 'boot': 'true', 'volumeID': '038d75ec-9421-4d79-9865-0dbebd5271d9', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'propagateErrors': 'off', 'if': 'virtio'}], 'transparentHugePages': 'true', 'keyboardLayout': 'en-us', 'displayPort': '-1', 'displaySecurePort': '-1', 'spiceSslCipherSuite': 'DEFAULT', 'cpuType': 'Westmere', 'custom': {}, 'clientIp': '', 'nicModel': 'pv', 'macAddr': '00:1a:4a:a8:64:06', 'kvmEnable': 'true', 'boot': 'c', 'smp': '4', 'vmType': 'kvm', 'smartcardEnable': 'false', 'timeOffset': '3600', 'memSize': 2048, 'displayIp': '0', 'spiceSecureChannels': 'smain,sinputs', 'smpCoresPerSocket': '2', 'vmName': 'KLEE-DC01', 'display': 'qxl', 'spiceMonitors': '1', 'nice': '0'}}
>Thread-15912386::INFO::2013-11-20 02:31:44,708::libvirtvm::1061::vm.Vm::(_run) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::VM wrapper has started
>Thread-15912386::INFO::2013-11-20 02:31:44,712::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912386::DEBUG::2013-11-20 02:31:44,713::task::495::TaskManager.Task::(_debug) Task c01207ea-4d77-41d5-859c-65b0b57669a2: moving from state init -> state preparing
>Thread-15912386::DEBUG::2013-11-20 02:31:44,715::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`36a6b7e3-d132-42d8-8104-2ddfdad4c775`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1690' at 'public_prepareVolume'
>Thread-15912386::DEBUG::2013-11-20 02:31:44,716::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912386::DEBUG::2013-11-20 02:31:44,717::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912386::DEBUG::2013-11-20 02:31:44,718::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`36a6b7e3-d132-42d8-8104-2ddfdad4c775`::Granted request
>Thread-15912386::DEBUG::2013-11-20 02:31:44,719::task::495::TaskManager.Task::(_debug) Task c01207ea-4d77-41d5-859c-65b0b57669a2: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912386::DEBUG::2013-11-20 02:31:44,719::task::495::TaskManager.Task::(_debug) Task c01207ea-4d77-41d5-859c-65b0b57669a2: ref 1 aborting False
>Thread-15912386::DEBUG::2013-11-20 02:31:44,721::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.d8be6e76-2f7f-4b56-9ada-c3fd543a9794`ReqID=`2e86c340-e8bc-435e-9fa0-514cd4843568`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1695' at 'public_prepareVolume'
>Thread-15912386::DEBUG::2013-11-20 02:31:44,721::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.d8be6e76-2f7f-4b56-9ada-c3fd543a9794' for lock type 'exclusive'
>Thread-15912386::DEBUG::2013-11-20 02:31:44,723::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15912386::DEBUG::2013-11-20 02:31:44,724::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15912386::DEBUG::2013-11-20 02:31:45,447::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912386::DEBUG::2013-11-20 02:31:45,898::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15912386::DEBUG::2013-11-20 02:31:45,899::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=15 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15912386::DEBUG::2013-11-20 02:31:46,263::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.00787782 s, 65.0 kB/s\n'; <rc> = 0
>Thread-15912386::DEBUG::2013-11-20 02:31:46,265::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.00787782 s, 65.0 kB/s'], size: 512
>Thread-15912386::DEBUG::2013-11-20 02:31:46,271::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=15 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15912386::DEBUG::2013-11-20 02:31:46,690::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.0003415 s, 1.5 MB/s\n'; <rc> = 0
>Thread-15912386::DEBUG::2013-11-20 02:31:46,691::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.0003415 s, 1.5 MB/s'], size: 512
>Thread-15912386::INFO::2013-11-20 02:31:46,692::image::340::Storage.Image::(getChain) sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 chain=[<storage.blockVolume.BlockVolume instance at 0x1b9bbd8>] 
>Thread-15912386::DEBUG::2013-11-20 02:31:46,694::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --autobackup n --available y 6b892067-ac7e-4039-a7be-5b7387525b16/038d75ec-9421-4d79-9865-0dbebd5271d9' (cwd None)
>Thread-15912386::DEBUG::2013-11-20 02:31:47,643::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912386::DEBUG::2013-11-20 02:31:47,645::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15912386::DEBUG::2013-11-20 02:31:47,645::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15912386::DEBUG::2013-11-20 02:31:47,647::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.038d75ec-9421-4d79-9865-0dbebd5271d9`ReqID=`6eda7ba4-2742-4867-8644-99f3cc0eab4b`::Request was made in '/usr/share/vdsm/storage/resourceFactories.py' line '149' at '__getResourceCandidatesList'
>Thread-15912386::DEBUG::2013-11-20 02:31:47,648::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.038d75ec-9421-4d79-9865-0dbebd5271d9' for lock type 'exclusive'
>Thread-15912386::DEBUG::2013-11-20 02:31:47,649::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.038d75ec-9421-4d79-9865-0dbebd5271d9' is free. Now locking as 'exclusive' (1 active user)
>Thread-15912386::DEBUG::2013-11-20 02:31:47,649::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.038d75ec-9421-4d79-9865-0dbebd5271d9`ReqID=`6eda7ba4-2742-4867-8644-99f3cc0eab4b`::Granted request
>Thread-15912386::DEBUG::2013-11-20 02:31:47,650::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.d8be6e76-2f7f-4b56-9ada-c3fd543a9794' is free. Now locking as 'exclusive' (1 active user)
>Thread-15912386::DEBUG::2013-11-20 02:31:47,651::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.d8be6e76-2f7f-4b56-9ada-c3fd543a9794`ReqID=`2e86c340-e8bc-435e-9fa0-514cd4843568`::Granted request
>Thread-15912386::DEBUG::2013-11-20 02:31:47,653::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15912386::DEBUG::2013-11-20 02:31:47,654::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15912386::DEBUG::2013-11-20 02:31:48,355::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912386::DEBUG::2013-11-20 02:31:48,609::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15912386::DEBUG::2013-11-20 02:31:48,612::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --refresh 6b892067-ac7e-4039-a7be-5b7387525b16/038d75ec-9421-4d79-9865-0dbebd5271d9' (cwd None)
>Thread-15912386::DEBUG::2013-11-20 02:31:49,218::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912386::DEBUG::2013-11-20 02:31:49,220::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15912386::DEBUG::2013-11-20 02:31:49,220::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15912386::INFO::2013-11-20 02:31:49,221::volume::517::Storage.Volume::(prepare) Volume: preparing volume 6b892067-ac7e-4039-a7be-5b7387525b16/038d75ec-9421-4d79-9865-0dbebd5271d9
>Thread-15912386::DEBUG::2013-11-20 02:31:49,222::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15912386::DEBUG::2013-11-20 02:31:49,223::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15912386::DEBUG::2013-11-20 02:31:49,889::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912386::DEBUG::2013-11-20 02:31:50,262::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15912386::DEBUG::2013-11-20 02:31:50,263::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=15 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15912386::DEBUG::2013-11-20 02:31:50,589::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000412951 s, 1.2 MB/s\n'; <rc> = 0
>Thread-15912386::DEBUG::2013-11-20 02:31:50,590::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000412951 s, 1.2 MB/s'], size: 512
>Thread-15912386::DEBUG::2013-11-20 02:31:50,591::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=15 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15912386::DEBUG::2013-11-20 02:31:51,004::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000301411 s, 1.7 MB/s\n'; <rc> = 0
>Thread-15912386::DEBUG::2013-11-20 02:31:51,005::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000301411 s, 1.7 MB/s'], size: 512
>Thread-15912386::DEBUG::2013-11-20 02:31:51,007::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.038d75ec-9421-4d79-9865-0dbebd5271d9`ReqID=`b0a176bb-cd08-4284-b53c-f5f72cee42a9`::Request was made in '/usr/share/vdsm/storage/volume.py' line '533' at 'prepare'
>Thread-15912386::DEBUG::2013-11-20 02:31:51,007::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.038d75ec-9421-4d79-9865-0dbebd5271d9' for lock type 'exclusive'
>Thread-15912386::DEBUG::2013-11-20 02:31:51,008::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.038d75ec-9421-4d79-9865-0dbebd5271d9' is free. Now locking as 'exclusive' (1 active user)
>Thread-15912386::DEBUG::2013-11-20 02:31:51,009::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.038d75ec-9421-4d79-9865-0dbebd5271d9`ReqID=`b0a176bb-cd08-4284-b53c-f5f72cee42a9`::Granted request
>Thread-15912386::DEBUG::2013-11-20 02:31:51,011::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=15 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15912386::DEBUG::2013-11-20 02:31:51,407::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000200469 s, 2.6 MB/s\n'; <rc> = 0
>Thread-15912386::DEBUG::2013-11-20 02:31:51,408::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000200469 s, 2.6 MB/s'], size: 512
>Thread-15912386::DEBUG::2013-11-20 02:31:51,410::task::495::TaskManager.Task::(_debug) Task c01207ea-4d77-41d5-859c-65b0b57669a2: finished: None
>Thread-15912386::DEBUG::2013-11-20 02:31:51,411::task::495::TaskManager.Task::(_debug) Task c01207ea-4d77-41d5-859c-65b0b57669a2: moving from state preparing -> state finished
>Thread-15912386::DEBUG::2013-11-20 02:31:51,411::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912386::DEBUG::2013-11-20 02:31:51,412::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912386::DEBUG::2013-11-20 02:31:51,413::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912386::DEBUG::2013-11-20 02:31:51,414::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912386::DEBUG::2013-11-20 02:31:51,414::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912386::DEBUG::2013-11-20 02:31:51,415::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912386::DEBUG::2013-11-20 02:31:51,415::task::495::TaskManager.Task::(_debug) Task c01207ea-4d77-41d5-859c-65b0b57669a2: ref 0 aborting False
>Thread-15912386::INFO::2013-11-20 02:31:51,416::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, Return response: {'status': {'message': 'OK', 'code': 0}}
>Thread-15912386::INFO::2013-11-20 02:31:51,417::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912386::DEBUG::2013-11-20 02:31:51,418::task::495::TaskManager.Task::(_debug) Task 7a69231e-d457-476f-a56c-492479e16b94: moving from state init -> state preparing
>Thread-15912386::DEBUG::2013-11-20 02:31:51,419::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d4cd1488-9ec8-4d77-b278-e5c67ae84502`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1666' at 'public_getVolumePath'
>Thread-15912386::DEBUG::2013-11-20 02:31:51,420::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912386::DEBUG::2013-11-20 02:31:51,420::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912386::DEBUG::2013-11-20 02:31:51,421::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d4cd1488-9ec8-4d77-b278-e5c67ae84502`::Granted request
>Thread-15912386::DEBUG::2013-11-20 02:31:51,422::task::495::TaskManager.Task::(_debug) Task 7a69231e-d457-476f-a56c-492479e16b94: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912386::DEBUG::2013-11-20 02:31:51,422::task::495::TaskManager.Task::(_debug) Task 7a69231e-d457-476f-a56c-492479e16b94: ref 1 aborting False
>Thread-15912386::DEBUG::2013-11-20 02:31:51,423::task::495::TaskManager.Task::(_debug) Task 7a69231e-d457-476f-a56c-492479e16b94: finished: {'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/d8be6e76-2f7f-4b56-9ada-c3fd543a9794/038d75ec-9421-4d79-9865-0dbebd5271d9'}
>Thread-15912386::DEBUG::2013-11-20 02:31:51,424::task::495::TaskManager.Task::(_debug) Task 7a69231e-d457-476f-a56c-492479e16b94: moving from state preparing -> state finished
>Thread-15912386::DEBUG::2013-11-20 02:31:51,425::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912386::DEBUG::2013-11-20 02:31:51,425::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912386::DEBUG::2013-11-20 02:31:51,426::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912386::DEBUG::2013-11-20 02:31:51,427::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912386::DEBUG::2013-11-20 02:31:51,427::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912386::DEBUG::2013-11-20 02:31:51,428::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912386::DEBUG::2013-11-20 02:31:51,428::task::495::TaskManager.Task::(_debug) Task 7a69231e-d457-476f-a56c-492479e16b94: ref 0 aborting False
>Thread-15912386::INFO::2013-11-20 02:31:51,429::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, Return response: {'status': {'message': 'OK', 'code': 0}, 'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/d8be6e76-2f7f-4b56-9ada-c3fd543a9794/038d75ec-9421-4d79-9865-0dbebd5271d9'}
>Thread-15912386::INFO::2013-11-20 02:31:51,430::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912386::DEBUG::2013-11-20 02:31:51,430::task::495::TaskManager.Task::(_debug) Task f168676e-b6df-466b-9158-c3b5104cf873: moving from state init -> state preparing
>Thread-15912386::DEBUG::2013-11-20 02:31:51,431::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`23b99c0f-72b1-4949-ac5e-cb830c2aa9d6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912386::DEBUG::2013-11-20 02:31:51,432::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912386::DEBUG::2013-11-20 02:31:51,433::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912386::DEBUG::2013-11-20 02:31:51,433::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`23b99c0f-72b1-4949-ac5e-cb830c2aa9d6`::Granted request
>Thread-15912386::DEBUG::2013-11-20 02:31:51,434::task::495::TaskManager.Task::(_debug) Task f168676e-b6df-466b-9158-c3b5104cf873: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912386::DEBUG::2013-11-20 02:31:51,435::task::495::TaskManager.Task::(_debug) Task f168676e-b6df-466b-9158-c3b5104cf873: ref 1 aborting False
>Thread-15912386::DEBUG::2013-11-20 02:31:51,435::task::495::TaskManager.Task::(_debug) Task f168676e-b6df-466b-9158-c3b5104cf873: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912386::DEBUG::2013-11-20 02:31:51,436::task::495::TaskManager.Task::(_debug) Task f168676e-b6df-466b-9158-c3b5104cf873: moving from state preparing -> state finished
>Thread-15912386::DEBUG::2013-11-20 02:31:51,436::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912386::DEBUG::2013-11-20 02:31:51,437::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912386::DEBUG::2013-11-20 02:31:51,438::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912386::DEBUG::2013-11-20 02:31:51,438::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912386::DEBUG::2013-11-20 02:31:51,439::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912386::DEBUG::2013-11-20 02:31:51,439::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912386::DEBUG::2013-11-20 02:31:51,440::task::495::TaskManager.Task::(_debug) Task f168676e-b6df-466b-9158-c3b5104cf873: ref 0 aborting False
>Thread-15912386::INFO::2013-11-20 02:31:51,440::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912386::INFO::2013-11-20 02:31:51,441::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16)
>Thread-15912386::DEBUG::2013-11-20 02:31:51,442::task::495::TaskManager.Task::(_debug) Task 395eff4f-2812-47ae-8751-927442be49d6: moving from state init -> state preparing
>Thread-15912386::INFO::2013-11-20 02:31:51,442::blockSD::674::Storage.StorageDomain::(validate) sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16
>Thread-15912386::DEBUG::2013-11-20 02:31:51,443::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15912386::DEBUG::2013-11-20 02:31:51,444::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15912386::DEBUG::2013-11-20 02:31:51,444::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15912386::DEBUG::2013-11-20 02:31:51,446::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15912386::DEBUG::2013-11-20 02:31:51,446::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-15912386::DEBUG::2013-11-20 02:31:51,448::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15912390::INFO::2013-11-20 02:31:51,660::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912390::DEBUG::2013-11-20 02:31:51,662::task::495::TaskManager.Task::(_debug) Task 9de89b0b-3080-4d8f-a540-9de3237eb831: moving from state init -> state preparing
>Thread-15912390::DEBUG::2013-11-20 02:31:51,663::task::495::TaskManager.Task::(_debug) Task 9de89b0b-3080-4d8f-a540-9de3237eb831: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000427007675171', 'lastCheck': 1384914711.2783451, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148391723633', 'lastCheck': 1384914702.7506361, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00133585929871', 'lastCheck': 1384914702.752203, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000707864761353', 'lastCheck': 1384914708.9284389, 'valid': True, 'code': 0}}
>Thread-15912390::DEBUG::2013-11-20 02:31:51,663::task::495::TaskManager.Task::(_debug) Task 9de89b0b-3080-4d8f-a540-9de3237eb831: moving from state preparing -> state finished
>Thread-15912390::DEBUG::2013-11-20 02:31:51,664::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912390::DEBUG::2013-11-20 02:31:51,665::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912390::DEBUG::2013-11-20 02:31:51,665::task::495::TaskManager.Task::(_debug) Task 9de89b0b-3080-4d8f-a540-9de3237eb831: ref 0 aborting False
>Thread-15912390::INFO::2013-11-20 02:31:51,666::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000427007675171', 'lastCheck': 1384914711.2783451, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148391723633', 'lastCheck': 1384914702.7506361, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00133585929871', 'lastCheck': 1384914702.752203, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000707864761353', 'lastCheck': 1384914708.9284389, 'valid': True, 'code': 0}}
>Thread-15912386::DEBUG::2013-11-20 02:31:52,085::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912386::DEBUG::2013-11-20 02:31:52,089::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-15912386::DEBUG::2013-11-20 02:31:52,090::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_ID=1', u'POOL_SPM_LVER=286', u'_SHA_CKSUM=09323c06acf399057c7d77c795a76a028bc8ee27']
>Thread-15912386::DEBUG::2013-11-20 02:31:52,097::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5ada4500-006f-4070-9787-d02938cfad60`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1297' at 'public_getStorageDomainInfo'
>Thread-15912386::DEBUG::2013-11-20 02:31:52,098::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912386::DEBUG::2013-11-20 02:31:52,099::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912386::DEBUG::2013-11-20 02:31:52,100::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5ada4500-006f-4070-9787-d02938cfad60`::Granted request
>Thread-15912386::DEBUG::2013-11-20 02:31:52,101::task::495::TaskManager.Task::(_debug) Task 395eff4f-2812-47ae-8751-927442be49d6: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912386::DEBUG::2013-11-20 02:31:52,102::task::495::TaskManager.Task::(_debug) Task 395eff4f-2812-47ae-8751-927442be49d6: ref 1 aborting False
>Thread-15912386::WARNING::2013-11-20 02:31:52,104::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912386::DEBUG::2013-11-20 02:31:52,105::task::495::TaskManager.Task::(_debug) Task 395eff4f-2812-47ae-8751-927442be49d6: finished: {'info': {'uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'vguuid': 'aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', 'lver': 286, 'state': 'OK', 'version': '2', 'role': 'Master', 'master_ver': 1, 'spm_id': 1, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'OS'}}
>Thread-15912386::DEBUG::2013-11-20 02:31:52,106::task::495::TaskManager.Task::(_debug) Task 395eff4f-2812-47ae-8751-927442be49d6: moving from state preparing -> state finished
>Thread-15912386::DEBUG::2013-11-20 02:31:52,106::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912386::DEBUG::2013-11-20 02:31:52,107::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912386::DEBUG::2013-11-20 02:31:52,108::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912386::DEBUG::2013-11-20 02:31:52,108::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912386::DEBUG::2013-11-20 02:31:52,109::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912386::DEBUG::2013-11-20 02:31:52,109::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912386::DEBUG::2013-11-20 02:31:52,110::task::495::TaskManager.Task::(_debug) Task 395eff4f-2812-47ae-8751-927442be49d6: ref 0 aborting False
>Thread-15912386::INFO::2013-11-20 02:31:52,111::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'vguuid': 'aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', 'lver': 286, 'state': 'OK', 'version': '2', 'role': 'Master', 'master_ver': 1, 'spm_id': 1, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'OS'}}
>Thread-15912386::DEBUG::2013-11-20 02:31:52,125::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/libexec/vdsm/hooks/before_vm_start/50_vhostmd' (cwd None)
>Thread-15912386::DEBUG::2013-11-20 02:31:52,598::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912386::INFO::2013-11-20 02:31:52,599::hooks::51::root::(_runHooksDir) 
>Thread-15912386::DEBUG::2013-11-20 02:31:52,600::libvirtvm::1077::vm.Vm::(_run) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::<?xml version="1.0" encoding="utf-8"?>
><domain type="kvm">
>	<name>KLEE-DC01</name>
>	<uuid>98ec88b5-2c08-4ff2-b607-c3f4e281f6af</uuid>
>	<memory>2097152</memory>
>	<currentMemory>2097152</currentMemory>
>	<vcpu>4</vcpu>
>	<devices>
>		<disk device="disk" type="block">
>			<source dev="/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/d8be6e76-2f7f-4b56-9ada-c3fd543a9794/038d75ec-9421-4d79-9865-0dbebd5271d9"/>
>			<target bus="virtio" dev="vda"/>
>			<serial>56-9ada-c3fd543a9794</serial>
>			<driver cache="none" error_policy="stop" io="native" name="qemu" type="raw"/>
>		</disk>
>		<disk device="cdrom" type="file">
>			<target bus="ide" dev="hdc"/>
>		</disk>
>		<controller index="0" ports="16" type="virtio-serial"/>
>		<channel type="unix">
>			<target name="com.redhat.rhevm.vdsm" type="virtio"/>
>			<source mode="bind" path="/var/lib/libvirt/qemu/channels/KLEE-DC01.com.redhat.rhevm.vdsm"/>
>		</channel>
>		<memballoon model="none"/>
>		<interface type="bridge">
>			<mac address="00:1a:4a:a8:64:06"/>
>			<model type="virtio"/>
>			<source bridge="rhevm"/>
>		</interface>
>		<input bus="ps2" type="mouse"/>
>		<video>
>			<model heads="1" type="qxl" vram="65536"/>
>		</video>
>		<channel type="spicevmc">
>			<target name="com.redhat.spice.0" type="virtio"/>
>		</channel>
>		<graphics autoport="yes" keymap="en-us" listen="0" passwd="*****" passwdValidTo="1970-01-01T00:00:01" port="-1" tlsPort="-1" type="spice">
>			<channel mode="secure" name="main"/>
>			<channel mode="secure" name="inputs"/>
>		</graphics>
>	</devices>
>	<os>
>		<type arch="x86_64" machine="rhel6.2.0">hvm</type>
>		<boot dev="hd"/>
>		<smbios mode="sysinfo"/>
>	</os>
>	<sysinfo type="smbios">
>		<system>
>			<entry name="manufacturer">Red Hat</entry>
>			<entry name="product">RHEV Hypervisor</entry>
>			<entry name="version">6.3-20121012.0.el6_3</entry>
>			<entry name="serial">30343536-3138-5A43-4A32-333630373442_AC:16:2D:76:EB:F0</entry>
>			<entry name="uuid">98ec88b5-2c08-4ff2-b607-c3f4e281f6af</entry>
>		</system>
>	</sysinfo>
>	<clock adjustment="3600" offset="variable">
>		<timer name="rtc" tickpolicy="catchup"/>
>	</clock>
>	<features>
>		<acpi/>
>	</features>
>	<cpu match="exact">
>		<model>Westmere</model>
>		<topology cores="2" sockets="2" threads="1"/>
>	</cpu>
></domain>
>
>Thread-1024767::ERROR::2013-11-20 02:31:53,344::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>libvirtEventLoop::DEBUG::2013-11-20 02:31:53,606::libvirtvm::1401::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::event Started detail 0 opaque None
>Thread-15912386::DEBUG::2013-11-20 02:31:53,653::utils::307::vm.Vm::(start) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Start statistics collection
>Thread-15912392::DEBUG::2013-11-20 02:31:53,654::utils::336::vm.Vm::(run) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Stats thread started
>Thread-15912393::DEBUG::2013-11-20 02:31:53,656::guestIF::47::root::(_connect) Attempting connection to /var/lib/libvirt/qemu/channels/KLEE-DC01.com.redhat.rhevm.vdsm
>Thread-15912392::INFO::2013-11-20 02:31:53,658::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912393::DEBUG::2013-11-20 02:31:53,659::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /usr/share/vdsm/prepare-vmchannel /var/lib/libvirt/qemu/channels/KLEE-DC01.com.redhat.rhevm.vdsm' (cwd None)
>Thread-15912392::DEBUG::2013-11-20 02:31:53,660::task::495::TaskManager.Task::(_debug) Task 6ec0436c-c099-4b94-aebe-f8a3e780adc1: moving from state init -> state preparing
>Thread-15912386::WARNING::2013-11-20 02:31:53,661::libvirtvm::1124::vm.Vm::(_readPauseCode) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::_readPauseCode unsupported by libvirt vm
>Thread-15912392::DEBUG::2013-11-20 02:31:53,883::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5ffdd12f-f610-4c1c-bc65-3f7982027deb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912386::DEBUG::2013-11-20 02:31:53,891::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /usr/share/vdsm/get-vm-pid KLEE-DC01' (cwd None)
>Thread-15912392::DEBUG::2013-11-20 02:31:53,896::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 02:31:54,157::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 02:31:54,159::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5ffdd12f-f610-4c1c-bc65-3f7982027deb`::Granted request
>Thread-15912392::DEBUG::2013-11-20 02:31:54,160::task::495::TaskManager.Task::(_debug) Task 6ec0436c-c099-4b94-aebe-f8a3e780adc1: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 02:31:54,161::task::495::TaskManager.Task::(_debug) Task 6ec0436c-c099-4b94-aebe-f8a3e780adc1: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 02:31:54,162::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15912392::DEBUG::2013-11-20 02:31:54,162::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15912393::DEBUG::2013-11-20 02:31:54,549::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912386::DEBUG::2013-11-20 02:31:54,552::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912393::DEBUG::2013-11-20 02:31:54,555::guestIF::53::root::(_connect) connected to /var/lib/libvirt/qemu/channels/KLEE-DC01.com.redhat.rhevm.vdsm
>Thread-15912386::DEBUG::2013-11-20 02:31:54,566::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /sbin/service ksmtuned retune' (cwd None)
>Thread-15912386::DEBUG::2013-11-20 02:31:55,092::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912386::DEBUG::2013-11-20 02:31:55,095::vm::358::vm.Vm::(_startUnderlyingVm) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::_ongoingCreations released
>Thread-15912392::DEBUG::2013-11-20 02:31:55,167::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912392::DEBUG::2013-11-20 02:31:55,408::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15912392::DEBUG::2013-11-20 02:31:55,409::task::495::TaskManager.Task::(_debug) Task 6ec0436c-c099-4b94-aebe-f8a3e780adc1: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 02:31:55,409::task::495::TaskManager.Task::(_debug) Task 6ec0436c-c099-4b94-aebe-f8a3e780adc1: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 02:31:55,410::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 02:31:55,410::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 02:31:55,410::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 02:31:55,411::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 02:31:55,411::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 02:31:55,411::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 02:31:55,412::task::495::TaskManager.Task::(_debug) Task 6ec0436c-c099-4b94-aebe-f8a3e780adc1: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 02:31:55,412::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912396::DEBUG::2013-11-20 02:31:57,913::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af',) {}
>Thread-15912396::DEBUG::2013-11-20 02:31:57,914::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::CPU stats not available
>Thread-15912396::DEBUG::2013-11-20 02:31:57,914::libvirtvm::202::vm.Vm::(_getNetworkStats) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Network stats not available
>Thread-15912396::DEBUG::2013-11-20 02:31:57,921::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk stats not available
>Thread-15912396::DEBUG::2013-11-20 02:31:57,922::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk latency not available
>Thread-15912396::DEBUG::2013-11-20 02:31:57,923::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '19721', 'displayIp': '0', 'displayPort': u'5900', 'session': 'Unknown', 'displaySecurePort': u'5901', 'timeOffset': '3600', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet0': {'state': 'unknown', 'macAddr': u'00:1a:4a:a8:64:06', 'speed': '1000', 'name': u'vnet0'}}, 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '64424509440', 'writeLatency': '0', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'flushLatency': '0', 'truesize': '64424509440'}}, 'boot': 'c', 'statsAge': '0.47', 'cpuIdle': '100.00', 'elapsedTime': '13', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15912398::DEBUG::2013-11-20 02:32:01,014::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af',) {}
>Thread-15912398::DEBUG::2013-11-20 02:32:01,015::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::CPU stats not available
>Thread-15912398::DEBUG::2013-11-20 02:32:01,015::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk stats not available
>Thread-15912398::DEBUG::2013-11-20 02:32:01,016::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk latency not available
>Thread-15912398::DEBUG::2013-11-20 02:32:01,016::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '19721', 'displayIp': '0', 'displayPort': u'5900', 'session': 'Unknown', 'displaySecurePort': u'5901', 'timeOffset': '3600', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet0': {'macAddr': u'00:1a:4a:a8:64:06', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '64424509440', 'writeLatency': '0', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'flushLatency': '0', 'truesize': '64424509440'}}, 'boot': 'c', 'statsAge': '0.57', 'cpuIdle': '100.00', 'elapsedTime': '16', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15912399::DEBUG::2013-11-20 02:32:02,089::clientIF::54::vds::(wrapper) [192.168.100.135]::call setVmTicket with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'kcPKiiNPOkVa', '120') {}
>Thread-15912399::DEBUG::2013-11-20 02:32:02,124::clientIF::59::vds::(wrapper) return setVmTicket with {'status': {'message': 'Done', 'code': 0}}
>Thread-1024767::ERROR::2013-11-20 02:32:03,531::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912401::DEBUG::2013-11-20 02:32:04,058::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af',) {}
>Thread-15912401::DEBUG::2013-11-20 02:32:04,059::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::CPU stats not available
>Thread-15912401::DEBUG::2013-11-20 02:32:04,060::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk stats not available
>Thread-15912401::DEBUG::2013-11-20 02:32:04,061::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk latency not available
>Thread-15912401::DEBUG::2013-11-20 02:32:04,061::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '19721', 'displayIp': '0', 'displayPort': u'5900', 'session': 'Unknown', 'displaySecurePort': u'5901', 'timeOffset': '3600', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet0': {'macAddr': u'00:1a:4a:a8:64:06', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '64424509440', 'writeLatency': '0', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'flushLatency': '0', 'truesize': '64424509440'}}, 'boot': 'c', 'statsAge': '0.62', 'cpuIdle': '100.00', 'elapsedTime': '19', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>libvirtEventLoop::DEBUG::2013-11-20 02:32:05,922::libvirtconnection::43::vm.Vm::(__eventCallback) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::graphics event phase 0 localAddr {'node': '192.168.100.133', 'service': '', 'family': 0} remoteAddr {'node': '192.168.50.51', 'service': '', 'family': 0}authScheme spice subject []
>libvirtEventLoop::DEBUG::2013-11-20 02:32:05,923::libvirtconnection::43::vm.Vm::(__eventCallback) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::graphics event phase 1 localAddr {'node': '192.168.100.133', 'service': '', 'family': 0} remoteAddr {'node': '192.168.50.51', 'service': '', 'family': 0}authScheme spice subject []
>libvirtEventLoop::DEBUG::2013-11-20 02:32:05,924::libvirtconnection::43::vm.Vm::(__eventCallback) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::graphics event phase 0 localAddr {'node': '192.168.100.133', 'service': '', 'family': 0} remoteAddr {'node': '192.168.50.51', 'service': '', 'family': 0}authScheme spice subject []
>libvirtEventLoop::DEBUG::2013-11-20 02:32:05,925::libvirtconnection::43::vm.Vm::(__eventCallback) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::graphics event phase 1 localAddr {'node': '192.168.100.133', 'service': '', 'family': 0} remoteAddr {'node': '192.168.50.51', 'service': '', 'family': 0}authScheme spice subject []
>libvirtEventLoop::DEBUG::2013-11-20 02:32:06,007::libvirtconnection::43::vm.Vm::(__eventCallback) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::graphics event phase 0 localAddr {'node': '192.168.100.133', 'service': '', 'family': 0} remoteAddr {'node': '192.168.50.51', 'service': '', 'family': 0}authScheme spice subject []
>libvirtEventLoop::DEBUG::2013-11-20 02:32:06,008::libvirtconnection::43::vm.Vm::(__eventCallback) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::graphics event phase 1 localAddr {'node': '192.168.100.133', 'service': '', 'family': 0} remoteAddr {'node': '192.168.50.51', 'service': '', 'family': 0}authScheme spice subject []
>libvirtEventLoop::DEBUG::2013-11-20 02:32:06,072::libvirtconnection::43::vm.Vm::(__eventCallback) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::graphics event phase 0 localAddr {'node': '192.168.100.133', 'service': '', 'family': 0} remoteAddr {'node': '192.168.50.51', 'service': '', 'family': 0}authScheme spice subject []
>libvirtEventLoop::DEBUG::2013-11-20 02:32:06,073::libvirtconnection::43::vm.Vm::(__eventCallback) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::graphics event phase 1 localAddr {'node': '192.168.100.133', 'service': '', 'family': 0} remoteAddr {'node': '192.168.50.51', 'service': '', 'family': 0}authScheme spice subject []
>libvirtEventLoop::DEBUG::2013-11-20 02:32:06,154::libvirtconnection::43::vm.Vm::(__eventCallback) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::graphics event phase 0 localAddr {'node': '192.168.100.133', 'service': '', 'family': 0} remoteAddr {'node': '192.168.50.51', 'service': '', 'family': 0}authScheme spice subject []
>libvirtEventLoop::DEBUG::2013-11-20 02:32:06,155::libvirtconnection::43::vm.Vm::(__eventCallback) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::graphics event phase 1 localAddr {'node': '192.168.100.133', 'service': '', 'family': 0} remoteAddr {'node': '192.168.50.51', 'service': '', 'family': 0}authScheme spice subject []
>Thread-15912402::INFO::2013-11-20 02:32:07,126::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912402::DEBUG::2013-11-20 02:32:07,127::task::495::TaskManager.Task::(_debug) Task e8d78412-afae-46d9-ac69-0a1dd0ad5094: moving from state init -> state preparing
>Thread-15912402::DEBUG::2013-11-20 02:32:07,127::task::495::TaskManager.Task::(_debug) Task e8d78412-afae-46d9-ac69-0a1dd0ad5094: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000308036804199', 'lastCheck': 1384914721.5232639, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00181698799133', 'lastCheck': 1384914723.5588739, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00177907943726', 'lastCheck': 1384914723.5344419, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00046706199646', 'lastCheck': 1384914719.1628461, 'valid': True, 'code': 0}}
>Thread-15912402::DEBUG::2013-11-20 02:32:07,128::task::495::TaskManager.Task::(_debug) Task e8d78412-afae-46d9-ac69-0a1dd0ad5094: moving from state preparing -> state finished
>Thread-15912402::DEBUG::2013-11-20 02:32:07,129::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912402::DEBUG::2013-11-20 02:32:07,130::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912402::DEBUG::2013-11-20 02:32:07,131::task::495::TaskManager.Task::(_debug) Task e8d78412-afae-46d9-ac69-0a1dd0ad5094: ref 0 aborting False
>Thread-15912402::INFO::2013-11-20 02:32:07,131::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000308036804199', 'lastCheck': 1384914721.5232639, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00181698799133', 'lastCheck': 1384914723.5588739, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00177907943726', 'lastCheck': 1384914723.5344419, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00046706199646', 'lastCheck': 1384914719.1628461, 'valid': True, 'code': 0}}
>Thread-15912403::DEBUG::2013-11-20 02:32:07,146::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::CPU stats not available
>Thread-15912403::DEBUG::2013-11-20 02:32:07,147::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk stats not available
>Thread-15912403::DEBUG::2013-11-20 02:32:07,148::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk latency not available
>Thread-15912405::DEBUG::2013-11-20 02:32:10,406::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af',) {}
>Thread-15912405::DEBUG::2013-11-20 02:32:10,407::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::CPU stats not available
>Thread-15912405::DEBUG::2013-11-20 02:32:10,407::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk stats not available
>Thread-15912405::DEBUG::2013-11-20 02:32:10,408::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk latency not available
>Thread-15912405::DEBUG::2013-11-20 02:32:10,409::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '19721', 'displayIp': '0', 'displayPort': u'5900', 'session': 'Unknown', 'displaySecurePort': u'5901', 'timeOffset': '3600', 'pauseCode': 'NOERR', 'clientIp': '192.168.50.51', 'kvmEnable': 'true', 'network': {u'vnet0': {'macAddr': u'00:1a:4a:a8:64:06', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '64424509440', 'writeLatency': '0', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'flushLatency': '0', 'truesize': '64424509440'}}, 'boot': 'c', 'statsAge': '0.96', 'cpuIdle': '100.00', 'elapsedTime': '25', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15912407::DEBUG::2013-11-20 02:32:13,524::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af',) {}
>Thread-15912407::DEBUG::2013-11-20 02:32:13,525::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk stats not available
>Thread-15912407::DEBUG::2013-11-20 02:32:13,528::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk latency not available
>Thread-15912407::DEBUG::2013-11-20 02:32:13,529::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '19721', 'displayIp': '0', 'displayPort': u'5900', 'session': 'Unknown', 'displaySecurePort': u'5901', 'timeOffset': '3600', 'pauseCode': 'NOERR', 'clientIp': '192.168.50.51', 'kvmEnable': 'true', 'network': {u'vnet0': {'macAddr': u'00:1a:4a:a8:64:06', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'monitorResponse': '0', 'cpuUser': '79.82', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '64424509440', 'writeLatency': '0', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'flushLatency': '0', 'truesize': '64424509440'}}, 'boot': 'c', 'statsAge': '0.08', 'cpuIdle': '20.18', 'elapsedTime': '28', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-1024767::ERROR::2013-11-20 02:32:13,549::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912409::DEBUG::2013-11-20 02:32:16,571::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af',) {}
>Thread-15912409::DEBUG::2013-11-20 02:32:16,572::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk stats not available
>Thread-15912409::DEBUG::2013-11-20 02:32:16,573::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk latency not available
>Thread-15912409::DEBUG::2013-11-20 02:32:16,574::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '19721', 'displayIp': '0', 'displayPort': u'5900', 'session': 'Unknown', 'displaySecurePort': u'5901', 'timeOffset': '3600', 'pauseCode': 'NOERR', 'clientIp': '192.168.50.51', 'kvmEnable': 'true', 'network': {u'vnet0': {'macAddr': u'00:1a:4a:a8:64:06', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'monitorResponse': '0', 'cpuUser': '79.82', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '64424509440', 'writeLatency': '0', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'flushLatency': '0', 'truesize': '64424509440'}}, 'boot': 'c', 'statsAge': '1.12', 'cpuIdle': '20.18', 'elapsedTime': '31', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15912411::DEBUG::2013-11-20 02:32:19,658::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af',) {}
>Thread-15912411::DEBUG::2013-11-20 02:32:19,659::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk stats not available
>Thread-15912411::DEBUG::2013-11-20 02:32:19,660::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk latency not available
>Thread-15912411::DEBUG::2013-11-20 02:32:19,660::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '19721', 'displayIp': '0', 'displayPort': u'5900', 'session': 'Unknown', 'displaySecurePort': u'5901', 'timeOffset': '3600', 'pauseCode': 'NOERR', 'clientIp': '192.168.50.51', 'kvmEnable': 'true', 'network': {u'vnet0': {'macAddr': u'00:1a:4a:a8:64:06', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'monitorResponse': '0', 'cpuUser': '79.82', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '64424509440', 'writeLatency': '0', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'flushLatency': '0', 'truesize': '64424509440'}}, 'boot': 'c', 'statsAge': '0.21', 'cpuIdle': '20.18', 'elapsedTime': '34', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15912412::INFO::2013-11-20 02:32:22,735::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912412::DEBUG::2013-11-20 02:32:22,736::task::495::TaskManager.Task::(_debug) Task 94604849-0316-4b70-a7e5-106dcb5ae511: moving from state init -> state preparing
>Thread-15912412::DEBUG::2013-11-20 02:32:22,737::task::495::TaskManager.Task::(_debug) Task 94604849-0316-4b70-a7e5-106dcb5ae511: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00124311447144', 'lastCheck': 1384914741.5506051, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00160789489746', 'lastCheck': 1384914733.574085, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.001296043396', 'lastCheck': 1384914733.55141, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000448942184448', 'lastCheck': 1384914739.184212, 'valid': True, 'code': 0}}
>Thread-15912412::DEBUG::2013-11-20 02:32:22,738::task::495::TaskManager.Task::(_debug) Task 94604849-0316-4b70-a7e5-106dcb5ae511: moving from state preparing -> state finished
>Thread-15912412::DEBUG::2013-11-20 02:32:22,738::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912412::DEBUG::2013-11-20 02:32:22,739::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912412::DEBUG::2013-11-20 02:32:22,739::task::495::TaskManager.Task::(_debug) Task 94604849-0316-4b70-a7e5-106dcb5ae511: ref 0 aborting False
>Thread-15912412::INFO::2013-11-20 02:32:22,740::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00124311447144', 'lastCheck': 1384914741.5506051, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00160789489746', 'lastCheck': 1384914733.574085, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.001296043396', 'lastCheck': 1384914733.55141, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000448942184448', 'lastCheck': 1384914739.184212, 'valid': True, 'code': 0}}
>Thread-15912413::DEBUG::2013-11-20 02:32:22,762::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk stats not available
>Thread-15912413::DEBUG::2013-11-20 02:32:22,763::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk latency not available
>Thread-1024767::ERROR::2013-11-20 02:32:23,565::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912415::DEBUG::2013-11-20 02:32:25,952::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af',) {}
>Thread-15912415::DEBUG::2013-11-20 02:32:25,953::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk stats not available
>Thread-15912415::DEBUG::2013-11-20 02:32:25,954::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk latency not available
>Thread-15912415::DEBUG::2013-11-20 02:32:25,955::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '19721', 'displayIp': '0', 'displayPort': u'5900', 'session': 'Unknown', 'displaySecurePort': u'5901', 'timeOffset': '3600', 'pauseCode': 'NOERR', 'clientIp': '192.168.50.51', 'kvmEnable': 'true', 'network': {u'vnet0': {'macAddr': u'00:1a:4a:a8:64:06', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'monitorResponse': '0', 'cpuUser': '79.95', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '64424509440', 'writeLatency': '0', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'flushLatency': '0', 'truesize': '64424509440'}}, 'boot': 'c', 'statsAge': '0.50', 'cpuIdle': '20.05', 'elapsedTime': '41', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15912417::DEBUG::2013-11-20 02:32:29,043::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af',) {}
>Thread-15912417::DEBUG::2013-11-20 02:32:29,044::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk stats not available
>Thread-15912417::DEBUG::2013-11-20 02:32:29,044::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk latency not available
>Thread-15912417::DEBUG::2013-11-20 02:32:29,045::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '19721', 'displayIp': '0', 'displayPort': u'5900', 'session': 'Unknown', 'displaySecurePort': u'5901', 'timeOffset': '3600', 'pauseCode': 'NOERR', 'clientIp': '192.168.50.51', 'kvmEnable': 'true', 'network': {u'vnet0': {'macAddr': u'00:1a:4a:a8:64:06', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'monitorResponse': '0', 'cpuUser': '79.95', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '64424509440', 'writeLatency': '0', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'flushLatency': '0', 'truesize': '64424509440'}}, 'boot': 'c', 'statsAge': '1.58', 'cpuIdle': '20.05', 'elapsedTime': '44', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15912419::DEBUG::2013-11-20 02:32:32,114::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af',) {}
>Thread-15912419::DEBUG::2013-11-20 02:32:32,115::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk stats not available
>Thread-15912419::DEBUG::2013-11-20 02:32:32,115::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk latency not available
>Thread-15912419::DEBUG::2013-11-20 02:32:32,116::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '19721', 'displayIp': '0', 'displayPort': u'5900', 'session': 'Unknown', 'displaySecurePort': u'5901', 'timeOffset': '3600', 'pauseCode': 'NOERR', 'clientIp': '192.168.50.51', 'kvmEnable': 'true', 'network': {u'vnet0': {'macAddr': u'00:1a:4a:a8:64:06', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'monitorResponse': '0', 'cpuUser': '79.95', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '64424509440', 'writeLatency': '0', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'flushLatency': '0', 'truesize': '64424509440'}}, 'boot': 'c', 'statsAge': '0.65', 'cpuIdle': '20.05', 'elapsedTime': '47', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-1024767::ERROR::2013-11-20 02:32:33,582::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912421::DEBUG::2013-11-20 02:32:35,203::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af',) {}
>Thread-15912421::DEBUG::2013-11-20 02:32:35,204::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk stats not available
>Thread-15912421::DEBUG::2013-11-20 02:32:35,205::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk latency not available
>Thread-15912421::DEBUG::2013-11-20 02:32:35,205::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '19721', 'displayIp': '0', 'displayPort': u'5900', 'session': 'Unknown', 'displaySecurePort': u'5901', 'timeOffset': '3600', 'pauseCode': 'NOERR', 'clientIp': '192.168.50.51', 'kvmEnable': 'true', 'network': {u'vnet0': {'macAddr': u'00:1a:4a:a8:64:06', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'monitorResponse': '0', 'cpuUser': '79.95', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '64424509440', 'writeLatency': '0', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'flushLatency': '0', 'truesize': '64424509440'}}, 'boot': 'c', 'statsAge': '1.74', 'cpuIdle': '20.05', 'elapsedTime': '50', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15912422::INFO::2013-11-20 02:32:38,271::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912422::DEBUG::2013-11-20 02:32:38,272::task::495::TaskManager.Task::(_debug) Task 57fa1083-97d6-4e0b-97a5-b0ef9b5658d3: moving from state init -> state preparing
>Thread-15912422::DEBUG::2013-11-20 02:32:38,273::task::495::TaskManager.Task::(_debug) Task 57fa1083-97d6-4e0b-97a5-b0ef9b5658d3: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000263929367065', 'lastCheck': 1384914751.562943, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00131702423096', 'lastCheck': 1384914753.6043501, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00151205062866', 'lastCheck': 1384914753.5854781, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000293016433716', 'lastCheck': 1384914749.194684, 'valid': True, 'code': 0}}
>Thread-15912422::DEBUG::2013-11-20 02:32:38,274::task::495::TaskManager.Task::(_debug) Task 57fa1083-97d6-4e0b-97a5-b0ef9b5658d3: moving from state preparing -> state finished
>Thread-15912422::DEBUG::2013-11-20 02:32:38,274::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912422::DEBUG::2013-11-20 02:32:38,275::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912422::DEBUG::2013-11-20 02:32:38,275::task::495::TaskManager.Task::(_debug) Task 57fa1083-97d6-4e0b-97a5-b0ef9b5658d3: ref 0 aborting False
>Thread-15912422::INFO::2013-11-20 02:32:38,276::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000263929367065', 'lastCheck': 1384914751.562943, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00131702423096', 'lastCheck': 1384914753.6043501, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00151205062866', 'lastCheck': 1384914753.5854781, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000293016433716', 'lastCheck': 1384914749.194684, 'valid': True, 'code': 0}}
>Thread-15912423::DEBUG::2013-11-20 02:32:38,310::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk stats not available
>Thread-15912423::DEBUG::2013-11-20 02:32:38,311::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk latency not available
>Thread-15912425::DEBUG::2013-11-20 02:32:41,486::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af',) {}
>Thread-15912425::DEBUG::2013-11-20 02:32:41,487::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk stats not available
>Thread-15912425::DEBUG::2013-11-20 02:32:41,487::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk latency not available
>Thread-15912425::DEBUG::2013-11-20 02:32:41,488::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '19721', 'displayIp': '0', 'displayPort': u'5900', 'session': 'Unknown', 'displaySecurePort': u'5901', 'timeOffset': '3600', 'pauseCode': 'NOERR', 'clientIp': '192.168.50.51', 'kvmEnable': 'true', 'network': {u'vnet0': {'macAddr': u'00:1a:4a:a8:64:06', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'monitorResponse': '0', 'cpuUser': '79.94', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '64424509440', 'writeLatency': '0', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'flushLatency': '0', 'truesize': '64424509440'}}, 'boot': 'c', 'statsAge': '0.02', 'cpuIdle': '20.06', 'elapsedTime': '56', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-1024767::ERROR::2013-11-20 02:32:43,599::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912427::DEBUG::2013-11-20 02:32:44,544::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af',) {}
>Thread-15912427::DEBUG::2013-11-20 02:32:44,545::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk stats not available
>Thread-15912427::DEBUG::2013-11-20 02:32:44,546::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk latency not available
>Thread-15912427::DEBUG::2013-11-20 02:32:44,546::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '19721', 'displayIp': '0', 'displayPort': u'5900', 'session': 'Unknown', 'displaySecurePort': u'5901', 'timeOffset': '3600', 'pauseCode': 'NOERR', 'clientIp': '192.168.50.51', 'kvmEnable': 'true', 'network': {u'vnet0': {'macAddr': u'00:1a:4a:a8:64:06', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'monitorResponse': '0', 'cpuUser': '79.94', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '64424509440', 'writeLatency': '0', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'flushLatency': '0', 'truesize': '64424509440'}}, 'boot': 'c', 'statsAge': '1.07', 'cpuIdle': '20.06', 'elapsedTime': '59', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15912429::DEBUG::2013-11-20 02:32:47,623::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af',) {}
>Thread-15912429::DEBUG::2013-11-20 02:32:47,624::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk stats not available
>Thread-15912429::DEBUG::2013-11-20 02:32:47,625::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk latency not available
>Thread-15912429::DEBUG::2013-11-20 02:32:47,625::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '19721', 'displayIp': '0', 'displayPort': u'5900', 'session': 'Unknown', 'displaySecurePort': u'5901', 'timeOffset': '3600', 'pauseCode': 'NOERR', 'clientIp': '192.168.50.51', 'kvmEnable': 'true', 'network': {u'vnet0': {'macAddr': u'00:1a:4a:a8:64:06', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'monitorResponse': '0', 'cpuUser': '79.94', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '64424509440', 'writeLatency': '0', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'flushLatency': '0', 'truesize': '64424509440'}}, 'boot': 'c', 'statsAge': '0.15', 'cpuIdle': '20.06', 'elapsedTime': '62', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-1024767::ERROR::2013-11-20 02:32:53,616::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912431::INFO::2013-11-20 02:32:53,745::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912431::DEBUG::2013-11-20 02:32:53,746::task::495::TaskManager.Task::(_debug) Task 839f2a56-ae75-45e9-9300-c08989030ea6: moving from state init -> state preparing
>Thread-15912431::DEBUG::2013-11-20 02:32:53,746::task::495::TaskManager.Task::(_debug) Task 839f2a56-ae75-45e9-9300-c08989030ea6: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000406980514526', 'lastCheck': 1384914771.5869241, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00141286849976', 'lastCheck': 1384914773.6329861, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00130796432495', 'lastCheck': 1384914773.618912, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000362157821655', 'lastCheck': 1384914769.2168159, 'valid': True, 'code': 0}}
>Thread-15912431::DEBUG::2013-11-20 02:32:53,747::task::495::TaskManager.Task::(_debug) Task 839f2a56-ae75-45e9-9300-c08989030ea6: moving from state preparing -> state finished
>Thread-15912431::DEBUG::2013-11-20 02:32:53,748::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912431::DEBUG::2013-11-20 02:32:53,748::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912431::DEBUG::2013-11-20 02:32:53,749::task::495::TaskManager.Task::(_debug) Task 839f2a56-ae75-45e9-9300-c08989030ea6: ref 0 aborting False
>Thread-15912431::INFO::2013-11-20 02:32:53,749::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000406980514526', 'lastCheck': 1384914771.5869241, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00141286849976', 'lastCheck': 1384914773.6329861, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00130796432495', 'lastCheck': 1384914773.618912, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000362157821655', 'lastCheck': 1384914769.2168159, 'valid': True, 'code': 0}}
>Thread-15912432::DEBUG::2013-11-20 02:32:53,764::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk stats not available
>Thread-15912432::DEBUG::2013-11-20 02:32:53,764::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Disk latency not available
>Thread-15912392::INFO::2013-11-20 02:32:55,477::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 02:32:55,478::task::495::TaskManager.Task::(_debug) Task 803f2594-927a-4b41-b978-8656e4599ce5: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 02:32:55,485::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1270859c-8d70-489e-8636-3bee97269223`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 02:32:55,485::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 02:32:55,486::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 02:32:55,487::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1270859c-8d70-489e-8636-3bee97269223`::Granted request
>Thread-15912392::DEBUG::2013-11-20 02:32:55,487::task::495::TaskManager.Task::(_debug) Task 803f2594-927a-4b41-b978-8656e4599ce5: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 02:32:55,488::task::495::TaskManager.Task::(_debug) Task 803f2594-927a-4b41-b978-8656e4599ce5: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 02:32:55,489::task::495::TaskManager.Task::(_debug) Task 803f2594-927a-4b41-b978-8656e4599ce5: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 02:32:55,489::task::495::TaskManager.Task::(_debug) Task 803f2594-927a-4b41-b978-8656e4599ce5: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 02:32:55,490::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 02:32:55,491::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 02:32:55,491::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 02:32:55,492::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 02:32:55,492::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 02:32:55,493::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 02:32:55,493::task::495::TaskManager.Task::(_debug) Task 803f2594-927a-4b41-b978-8656e4599ce5: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 02:32:55,494::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-1024767::ERROR::2013-11-20 02:33:03,633::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912437::INFO::2013-11-20 02:33:08,946::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912437::DEBUG::2013-11-20 02:33:08,947::task::495::TaskManager.Task::(_debug) Task ac85250e-89fa-43fa-a348-3fdf093a6bf4: moving from state init -> state preparing
>Thread-15912437::DEBUG::2013-11-20 02:33:08,947::task::495::TaskManager.Task::(_debug) Task ac85250e-89fa-43fa-a348-3fdf093a6bf4: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000337839126587', 'lastCheck': 1384914781.5997419, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00168299674988', 'lastCheck': 1384914783.645865, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00189614295959', 'lastCheck': 1384914783.636493, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000407934188843', 'lastCheck': 1384914779.227442, 'valid': True, 'code': 0}}
>Thread-15912437::DEBUG::2013-11-20 02:33:08,948::task::495::TaskManager.Task::(_debug) Task ac85250e-89fa-43fa-a348-3fdf093a6bf4: moving from state preparing -> state finished
>Thread-15912437::DEBUG::2013-11-20 02:33:08,949::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912437::DEBUG::2013-11-20 02:33:08,949::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912437::DEBUG::2013-11-20 02:33:08,950::task::495::TaskManager.Task::(_debug) Task ac85250e-89fa-43fa-a348-3fdf093a6bf4: ref 0 aborting False
>Thread-15912437::INFO::2013-11-20 02:33:08,950::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000337839126587', 'lastCheck': 1384914781.5997419, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00168299674988', 'lastCheck': 1384914783.645865, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00189614295959', 'lastCheck': 1384914783.636493, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000407934188843', 'lastCheck': 1384914779.227442, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-20 02:33:13,649::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-1024767::ERROR::2013-11-20 02:33:23,666::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912443::INFO::2013-11-20 02:33:24,197::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912443::DEBUG::2013-11-20 02:33:24,198::task::495::TaskManager.Task::(_debug) Task 521f764b-780b-4729-bd13-1413a0ee506c: moving from state init -> state preparing
>Thread-15912443::DEBUG::2013-11-20 02:33:24,198::task::495::TaskManager.Task::(_debug) Task 521f764b-780b-4729-bd13-1413a0ee506c: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000343084335327', 'lastCheck': 1384914801.6248159, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00121307373047', 'lastCheck': 1384914803.6749339, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00188398361206', 'lastCheck': 1384914803.669872, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000363111495972', 'lastCheck': 1384914799.250772, 'valid': True, 'code': 0}}
>Thread-15912443::DEBUG::2013-11-20 02:33:24,199::task::495::TaskManager.Task::(_debug) Task 521f764b-780b-4729-bd13-1413a0ee506c: moving from state preparing -> state finished
>Thread-15912443::DEBUG::2013-11-20 02:33:24,200::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912443::DEBUG::2013-11-20 02:33:24,200::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912443::DEBUG::2013-11-20 02:33:24,201::task::495::TaskManager.Task::(_debug) Task 521f764b-780b-4729-bd13-1413a0ee506c: ref 0 aborting False
>Thread-15912443::INFO::2013-11-20 02:33:24,201::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000343084335327', 'lastCheck': 1384914801.6248159, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00121307373047', 'lastCheck': 1384914803.6749339, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00188398361206', 'lastCheck': 1384914803.669872, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000363111495972', 'lastCheck': 1384914799.250772, 'valid': True, 'code': 0}}
>Thread-15912393::DEBUG::2013-11-20 02:33:28,992::guestIF::97::vm.Vm::(_handleMessage) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::u'None'
>Thread-42840::DEBUG::2013-11-20 02:33:29,262::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 02:33:30,013::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 02:33:33,683::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912449::INFO::2013-11-20 02:33:39,458::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912449::DEBUG::2013-11-20 02:33:39,459::task::495::TaskManager.Task::(_debug) Task 266d9c5b-5aa3-4ba2-9ef8-417c7f70594c: moving from state init -> state preparing
>Thread-15912449::DEBUG::2013-11-20 02:33:39,460::task::495::TaskManager.Task::(_debug) Task 266d9c5b-5aa3-4ba2-9ef8-417c7f70594c: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000313997268677', 'lastCheck': 1384914811.6371779, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142097473145', 'lastCheck': 1384914813.6899159, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00150394439697', 'lastCheck': 1384914813.686748, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000325202941895', 'lastCheck': 1384914810.015805, 'valid': True, 'code': 0}}
>Thread-15912449::DEBUG::2013-11-20 02:33:39,461::task::495::TaskManager.Task::(_debug) Task 266d9c5b-5aa3-4ba2-9ef8-417c7f70594c: moving from state preparing -> state finished
>Thread-15912449::DEBUG::2013-11-20 02:33:39,461::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912449::DEBUG::2013-11-20 02:33:39,462::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912449::DEBUG::2013-11-20 02:33:39,462::task::495::TaskManager.Task::(_debug) Task 266d9c5b-5aa3-4ba2-9ef8-417c7f70594c: ref 0 aborting False
>Thread-15912449::INFO::2013-11-20 02:33:39,463::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000313997268677', 'lastCheck': 1384914811.6371779, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142097473145', 'lastCheck': 1384914813.6899159, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00150394439697', 'lastCheck': 1384914813.686748, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000325202941895', 'lastCheck': 1384914810.015805, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-20 02:33:43,702::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-1024767::ERROR::2013-11-20 02:33:53,722::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912455::INFO::2013-11-20 02:33:54,846::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912455::DEBUG::2013-11-20 02:33:54,847::task::495::TaskManager.Task::(_debug) Task df8171a5-2c4e-4ee9-b779-2fc8e979e8ce: moving from state init -> state preparing
>Thread-15912455::DEBUG::2013-11-20 02:33:54,848::task::495::TaskManager.Task::(_debug) Task df8171a5-2c4e-4ee9-b779-2fc8e979e8ce: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000281095504761', 'lastCheck': 1384914831.660831, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00275802612305', 'lastCheck': 1384914833.7244861, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00133013725281', 'lastCheck': 1384914833.7249589, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000338792800903', 'lastCheck': 1384914830.038132, 'valid': True, 'code': 0}}
>Thread-15912455::DEBUG::2013-11-20 02:33:54,849::task::495::TaskManager.Task::(_debug) Task df8171a5-2c4e-4ee9-b779-2fc8e979e8ce: moving from state preparing -> state finished
>Thread-15912455::DEBUG::2013-11-20 02:33:54,849::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912455::DEBUG::2013-11-20 02:33:54,850::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912455::DEBUG::2013-11-20 02:33:54,850::task::495::TaskManager.Task::(_debug) Task df8171a5-2c4e-4ee9-b779-2fc8e979e8ce: ref 0 aborting False
>Thread-15912455::INFO::2013-11-20 02:33:54,851::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000281095504761', 'lastCheck': 1384914831.660831, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00275802612305', 'lastCheck': 1384914833.7244861, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00133013725281', 'lastCheck': 1384914833.7249589, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000338792800903', 'lastCheck': 1384914830.038132, 'valid': True, 'code': 0}}
>Thread-15912392::INFO::2013-11-20 02:33:55,552::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 02:33:55,553::task::495::TaskManager.Task::(_debug) Task 3a72fc7b-8c83-4122-8cfa-77e3af4a3bd6: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 02:33:55,555::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0b14751a-61af-4086-81f8-3af496e15141`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 02:33:55,555::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 02:33:55,556::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 02:33:55,556::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0b14751a-61af-4086-81f8-3af496e15141`::Granted request
>Thread-15912392::DEBUG::2013-11-20 02:33:55,557::task::495::TaskManager.Task::(_debug) Task 3a72fc7b-8c83-4122-8cfa-77e3af4a3bd6: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 02:33:55,558::task::495::TaskManager.Task::(_debug) Task 3a72fc7b-8c83-4122-8cfa-77e3af4a3bd6: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 02:33:55,559::task::495::TaskManager.Task::(_debug) Task 3a72fc7b-8c83-4122-8cfa-77e3af4a3bd6: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 02:33:55,559::task::495::TaskManager.Task::(_debug) Task 3a72fc7b-8c83-4122-8cfa-77e3af4a3bd6: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 02:33:55,560::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 02:33:55,560::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 02:33:55,561::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 02:33:55,562::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 02:33:55,562::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 02:33:55,563::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 02:33:55,563::task::495::TaskManager.Task::(_debug) Task 3a72fc7b-8c83-4122-8cfa-77e3af4a3bd6: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 02:33:55,564::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-1024767::ERROR::2013-11-20 02:34:03,741::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912461::INFO::2013-11-20 02:34:10,086::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912461::DEBUG::2013-11-20 02:34:10,087::task::495::TaskManager.Task::(_debug) Task 3256a0b4-1cce-42ef-96d0-5dae021708f9: moving from state init -> state preparing
>Thread-15912461::DEBUG::2013-11-20 02:34:10,088::task::495::TaskManager.Task::(_debug) Task 3256a0b4-1cce-42ef-96d0-5dae021708f9: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000285148620605', 'lastCheck': 1384914841.6738191, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00238108634949', 'lastCheck': 1384914843.74283, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00187206268311', 'lastCheck': 1384914843.7442269, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000447988510132', 'lastCheck': 1384914850.060374, 'valid': True, 'code': 0}}
>Thread-15912461::DEBUG::2013-11-20 02:34:10,089::task::495::TaskManager.Task::(_debug) Task 3256a0b4-1cce-42ef-96d0-5dae021708f9: moving from state preparing -> state finished
>Thread-15912461::DEBUG::2013-11-20 02:34:10,089::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912461::DEBUG::2013-11-20 02:34:10,090::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912461::DEBUG::2013-11-20 02:34:10,090::task::495::TaskManager.Task::(_debug) Task 3256a0b4-1cce-42ef-96d0-5dae021708f9: ref 0 aborting False
>Thread-15912461::INFO::2013-11-20 02:34:10,091::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000285148620605', 'lastCheck': 1384914841.6738191, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00238108634949', 'lastCheck': 1384914843.74283, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00187206268311', 'lastCheck': 1384914843.7442269, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000447988510132', 'lastCheck': 1384914850.060374, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-20 02:34:13,759::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-1024767::ERROR::2013-11-20 02:34:23,775::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912467::INFO::2013-11-20 02:34:25,354::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912467::DEBUG::2013-11-20 02:34:25,355::task::495::TaskManager.Task::(_debug) Task b0f19f19-3fb7-4945-b938-f01441268f58: moving from state init -> state preparing
>Thread-15912467::DEBUG::2013-11-20 02:34:25,355::task::495::TaskManager.Task::(_debug) Task b0f19f19-3fb7-4945-b938-f01441268f58: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000262975692749', 'lastCheck': 1384914861.6985321, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00192618370056', 'lastCheck': 1384914863.7757189, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00140309333801', 'lastCheck': 1384914863.777935, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000370979309082', 'lastCheck': 1384914860.0713151, 'valid': True, 'code': 0}}
>Thread-15912467::DEBUG::2013-11-20 02:34:25,356::task::495::TaskManager.Task::(_debug) Task b0f19f19-3fb7-4945-b938-f01441268f58: moving from state preparing -> state finished
>Thread-15912467::DEBUG::2013-11-20 02:34:25,356::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912467::DEBUG::2013-11-20 02:34:25,357::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912467::DEBUG::2013-11-20 02:34:25,358::task::495::TaskManager.Task::(_debug) Task b0f19f19-3fb7-4945-b938-f01441268f58: ref 0 aborting False
>Thread-15912467::INFO::2013-11-20 02:34:25,358::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000262975692749', 'lastCheck': 1384914861.6985321, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00192618370056', 'lastCheck': 1384914863.7757189, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00140309333801', 'lastCheck': 1384914863.777935, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000370979309082', 'lastCheck': 1384914860.0713151, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-20 02:34:33,792::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912473::INFO::2013-11-20 02:34:40,646::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912473::DEBUG::2013-11-20 02:34:40,646::task::495::TaskManager.Task::(_debug) Task 7e0d2356-615a-4752-8139-6a0851e2be65: moving from state init -> state preparing
>Thread-15912473::DEBUG::2013-11-20 02:34:40,647::task::495::TaskManager.Task::(_debug) Task 7e0d2356-615a-4752-8139-6a0851e2be65: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000269889831543', 'lastCheck': 1384914871.7096601, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00178003311157', 'lastCheck': 1384914873.790519, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00152397155762', 'lastCheck': 1384914873.794816, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000419855117798', 'lastCheck': 1384914880.0925889, 'valid': True, 'code': 0}}
>Thread-15912473::DEBUG::2013-11-20 02:34:40,648::task::495::TaskManager.Task::(_debug) Task 7e0d2356-615a-4752-8139-6a0851e2be65: moving from state preparing -> state finished
>Thread-15912473::DEBUG::2013-11-20 02:34:40,648::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912473::DEBUG::2013-11-20 02:34:40,649::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912473::DEBUG::2013-11-20 02:34:40,649::task::495::TaskManager.Task::(_debug) Task 7e0d2356-615a-4752-8139-6a0851e2be65: ref 0 aborting False
>Thread-15912473::INFO::2013-11-20 02:34:40,650::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000269889831543', 'lastCheck': 1384914871.7096601, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00178003311157', 'lastCheck': 1384914873.790519, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00152397155762', 'lastCheck': 1384914873.794816, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000419855117798', 'lastCheck': 1384914880.0925889, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-20 02:34:43,808::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-1024767::ERROR::2013-11-20 02:34:53,823::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912392::INFO::2013-11-20 02:34:55,633::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 02:34:55,634::task::495::TaskManager.Task::(_debug) Task 0301642b-ca01-41a6-80f1-548b430aec07: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 02:34:55,646::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2da0ccb8-8479-495c-8297-8de06ace0cdd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 02:34:55,646::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 02:34:55,647::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 02:34:55,647::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2da0ccb8-8479-495c-8297-8de06ace0cdd`::Granted request
>Thread-15912392::DEBUG::2013-11-20 02:34:55,648::task::495::TaskManager.Task::(_debug) Task 0301642b-ca01-41a6-80f1-548b430aec07: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 02:34:55,649::task::495::TaskManager.Task::(_debug) Task 0301642b-ca01-41a6-80f1-548b430aec07: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 02:34:55,649::task::495::TaskManager.Task::(_debug) Task 0301642b-ca01-41a6-80f1-548b430aec07: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 02:34:55,650::task::495::TaskManager.Task::(_debug) Task 0301642b-ca01-41a6-80f1-548b430aec07: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 02:34:55,650::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 02:34:55,650::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 02:34:55,651::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 02:34:55,651::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 02:34:55,652::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 02:34:55,652::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 02:34:55,653::task::495::TaskManager.Task::(_debug) Task 0301642b-ca01-41a6-80f1-548b430aec07: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 02:34:55,653::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912479::INFO::2013-11-20 02:34:55,902::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912479::DEBUG::2013-11-20 02:34:55,903::task::495::TaskManager.Task::(_debug) Task 72abc95c-79c5-4bff-85e1-6107a228d688: moving from state init -> state preparing
>Thread-15912479::DEBUG::2013-11-20 02:34:55,904::task::495::TaskManager.Task::(_debug) Task 72abc95c-79c5-4bff-85e1-6107a228d688: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000383853912354', 'lastCheck': 1384914891.735106, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00287294387817', 'lastCheck': 1384914893.8232291, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00151920318604', 'lastCheck': 1384914893.8261559, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000380992889404', 'lastCheck': 1384914890.1039081, 'valid': True, 'code': 0}}
>Thread-15912479::DEBUG::2013-11-20 02:34:55,905::task::495::TaskManager.Task::(_debug) Task 72abc95c-79c5-4bff-85e1-6107a228d688: moving from state preparing -> state finished
>Thread-15912479::DEBUG::2013-11-20 02:34:55,905::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912479::DEBUG::2013-11-20 02:34:55,906::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912479::DEBUG::2013-11-20 02:34:55,906::task::495::TaskManager.Task::(_debug) Task 72abc95c-79c5-4bff-85e1-6107a228d688: ref 0 aborting False
>Thread-15912479::INFO::2013-11-20 02:34:55,907::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000383853912354', 'lastCheck': 1384914891.735106, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00287294387817', 'lastCheck': 1384914893.8232291, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00151920318604', 'lastCheck': 1384914893.8261559, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000380992889404', 'lastCheck': 1384914890.1039081, 'valid': True, 'code': 0}}
>Thread-15912482::DEBUG::2013-11-20 02:35:00,525::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912482::INFO::2013-11-20 02:35:00,526::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: validateStorageServerConnection, args: (domType=1, spUUID=00000000-0000-0000-0000-000000000000, conList=[{'connection': 'kairo.brd-klee.dk:/export', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'ec9380b7-cfc1-41d8-9237-d5c7c505ef36', 'port': ''}])
>Thread-15912482::DEBUG::2013-11-20 02:35:00,527::task::495::TaskManager.Task::(_debug) Task ca8dacce-45eb-4fc8-8240-1211413468f1: moving from state init -> state preparing
>Thread-15912482::INFO::2013-11-20 02:35:00,527::storage_connection::127::Storage.ServerConnection::(validate) Request to validate NFS storage server
>Thread-15912482::INFO::2013-11-20 02:35:00,528::storage_connection::53::Storage.ServerConnection::(__validateConnectionParams) conList=[{'connection': 'kairo.brd-klee.dk:/export', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'ec9380b7-cfc1-41d8-9237-d5c7c505ef36', 'port': ''}]
>Thread-15912482::DEBUG::2013-11-20 02:35:00,532::fileUtils::83::Storage.Misc.excCmd::(mount) '/usr/bin/sudo -n /bin/mount -o soft,timeo=600,retrans=6,nosharecache,vers=3 -t nfs kairo.brd-klee.dk:/export /tmp/tmpXQYbRu' (cwd None)
>Thread-15912482::DEBUG::2013-11-20 02:35:00,970::fileUtils::83::Storage.Misc.excCmd::(mount) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912482::DEBUG::2013-11-20 02:35:00,982::fileUtils::112::Storage.Misc.excCmd::(umount) '/usr/bin/sudo -n /bin/umount -t nfs -f /tmp/tmpXQYbRu' (cwd None)
>Thread-15912482::DEBUG::2013-11-20 02:35:01,409::fileUtils::112::Storage.Misc.excCmd::(umount) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912482::DEBUG::2013-11-20 02:35:01,412::task::495::TaskManager.Task::(_debug) Task ca8dacce-45eb-4fc8-8240-1211413468f1: finished: {'statuslist': [{'status': 0, 'id': 'ec9380b7-cfc1-41d8-9237-d5c7c505ef36'}]}
>Thread-15912482::DEBUG::2013-11-20 02:35:01,413::task::495::TaskManager.Task::(_debug) Task ca8dacce-45eb-4fc8-8240-1211413468f1: moving from state preparing -> state finished
>Thread-15912482::DEBUG::2013-11-20 02:35:01,413::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912482::DEBUG::2013-11-20 02:35:01,414::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912482::DEBUG::2013-11-20 02:35:01,414::task::495::TaskManager.Task::(_debug) Task ca8dacce-45eb-4fc8-8240-1211413468f1: ref 0 aborting False
>Thread-15912482::INFO::2013-11-20 02:35:01,415::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: validateStorageServerConnection, Return response: {'status': {'message': 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': 'ec9380b7-cfc1-41d8-9237-d5c7c505ef36'}]}
>Thread-15912483::DEBUG::2013-11-20 02:35:01,432::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912483::INFO::2013-11-20 02:35:01,432::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: connectStorageServer, args: (domType=1, spUUID=00000000-0000-0000-0000-000000000000, conList=[{'connection': 'kairo.brd-klee.dk:/export', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'ec9380b7-cfc1-41d8-9237-d5c7c505ef36', 'port': ''}])
>Thread-15912483::DEBUG::2013-11-20 02:35:01,433::task::495::TaskManager.Task::(_debug) Task 2202f945-2d24-4cbe-aade-94f1d0d17fa5: moving from state init -> state preparing
>Thread-15912483::INFO::2013-11-20 02:35:01,433::storage_connection::95::Storage.ServerConnection::(connect) Request to connect NFS storage server
>Thread-15912483::INFO::2013-11-20 02:35:01,433::storage_connection::53::Storage.ServerConnection::(__validateConnectionParams) conList=[{'connection': 'kairo.brd-klee.dk:/export', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'ec9380b7-cfc1-41d8-9237-d5c7c505ef36', 'port': ''}]
>Thread-15912483::DEBUG::2013-11-20 02:35:01,445::lvm::547::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15912483::DEBUG::2013-11-20 02:35:01,445::lvm::549::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15912483::DEBUG::2013-11-20 02:35:01,446::lvm::559::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15912483::DEBUG::2013-11-20 02:35:01,447::lvm::561::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15912483::DEBUG::2013-11-20 02:35:01,447::lvm::580::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15912483::DEBUG::2013-11-20 02:35:01,448::lvm::582::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15912483::DEBUG::2013-11-20 02:35:01,449::task::495::TaskManager.Task::(_debug) Task 2202f945-2d24-4cbe-aade-94f1d0d17fa5: finished: {'statuslist': [{'status': 0, 'id': 'ec9380b7-cfc1-41d8-9237-d5c7c505ef36'}]}
>Thread-15912483::DEBUG::2013-11-20 02:35:01,449::task::495::TaskManager.Task::(_debug) Task 2202f945-2d24-4cbe-aade-94f1d0d17fa5: moving from state preparing -> state finished
>Thread-15912483::DEBUG::2013-11-20 02:35:01,450::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912483::DEBUG::2013-11-20 02:35:01,450::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912483::DEBUG::2013-11-20 02:35:01,451::task::495::TaskManager.Task::(_debug) Task 2202f945-2d24-4cbe-aade-94f1d0d17fa5: ref 0 aborting False
>Thread-15912483::INFO::2013-11-20 02:35:01,452::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: connectStorageServer, Return response: {'status': {'message': 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': 'ec9380b7-cfc1-41d8-9237-d5c7c505ef36'}]}
>Thread-42838::DEBUG::2013-11-20 02:35:01,744::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-42838::DEBUG::2013-11-20 02:35:01,749::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 02:35:02,443::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-42838::DEBUG::2013-11-20 02:35:02,454::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-1024767::ERROR::2013-11-20 02:35:03,994::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-42840::DEBUG::2013-11-20 02:35:10,125::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-42840::DEBUG::2013-11-20 02:35:10,126::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 02:35:10,730::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-42840::DEBUG::2013-11-20 02:35:10,733::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-15912487::INFO::2013-11-20 02:35:11,106::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912487::DEBUG::2013-11-20 02:35:11,107::task::495::TaskManager.Task::(_debug) Task bf50aa77-85fc-45f8-8056-16d1ade45536: moving from state init -> state preparing
>Thread-15912487::DEBUG::2013-11-20 02:35:11,108::task::495::TaskManager.Task::(_debug) Task bf50aa77-85fc-45f8-8056-16d1ade45536: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000394821166992', 'lastCheck': 1384914902.457823, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0022439956665', 'lastCheck': 1384914903.9991341, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00251817703247', 'lastCheck': 1384914903.9989159, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000380039215088', 'lastCheck': 1384914910.735188, 'valid': True, 'code': 0}}
>Thread-15912487::DEBUG::2013-11-20 02:35:11,109::task::495::TaskManager.Task::(_debug) Task bf50aa77-85fc-45f8-8056-16d1ade45536: moving from state preparing -> state finished
>Thread-15912487::DEBUG::2013-11-20 02:35:11,109::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912487::DEBUG::2013-11-20 02:35:11,110::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912487::DEBUG::2013-11-20 02:35:11,111::task::495::TaskManager.Task::(_debug) Task bf50aa77-85fc-45f8-8056-16d1ade45536: ref 0 aborting False
>Thread-15912487::INFO::2013-11-20 02:35:11,112::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000394821166992', 'lastCheck': 1384914902.457823, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0022439956665', 'lastCheck': 1384914903.9991341, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00251817703247', 'lastCheck': 1384914903.9989159, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000380039215088', 'lastCheck': 1384914910.735188, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-20 02:35:14,015::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912393::DEBUG::2013-11-20 02:35:19,191::guestIF::112::vm.Vm::(_handleMessage) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::Guest system shuts down.
>Thread-1024767::ERROR::2013-11-20 02:35:24,031::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>libvirtEventLoop::DEBUG::2013-11-20 02:35:25,062::vm::498::vm.Vm::(onReboot) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::reboot event
>Thread-15912493::INFO::2013-11-20 02:35:26,341::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912493::DEBUG::2013-11-20 02:35:26,342::task::495::TaskManager.Task::(_debug) Task f14bfc78-1ad6-464a-9da5-d92d14f7549d: moving from state init -> state preparing
>Thread-15912493::DEBUG::2013-11-20 02:35:26,343::task::495::TaskManager.Task::(_debug) Task f14bfc78-1ad6-464a-9da5-d92d14f7549d: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000282049179077', 'lastCheck': 1384914922.482832, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00250315666199', 'lastCheck': 1384914924.0334179, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00144815444946', 'lastCheck': 1384914924.034229, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000388860702515', 'lastCheck': 1384914920.7463801, 'valid': True, 'code': 0}}
>Thread-15912493::DEBUG::2013-11-20 02:35:26,343::task::495::TaskManager.Task::(_debug) Task f14bfc78-1ad6-464a-9da5-d92d14f7549d: moving from state preparing -> state finished
>Thread-15912493::DEBUG::2013-11-20 02:35:26,344::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912493::DEBUG::2013-11-20 02:35:26,345::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912493::DEBUG::2013-11-20 02:35:26,345::task::495::TaskManager.Task::(_debug) Task f14bfc78-1ad6-464a-9da5-d92d14f7549d: ref 0 aborting False
>Thread-15912493::INFO::2013-11-20 02:35:26,346::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000282049179077', 'lastCheck': 1384914922.482832, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00250315666199', 'lastCheck': 1384914924.0334179, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00144815444946', 'lastCheck': 1384914924.034229, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000388860702515', 'lastCheck': 1384914920.7463801, 'valid': True, 'code': 0}}
>Thread-15912496::DEBUG::2013-11-20 02:35:29,531::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af',) {}
>Thread-15912496::DEBUG::2013-11-20 02:35:29,532::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': 'Unknown', 'memUsage': '35', 'acpiEnable': 'true', 'pid': '19721', 'displayPort': u'5900', 'session': 'Unknown', 'netIfaces': [{u'inet6': [u'fe80::7d2d:e139:619b:7edc'], u'hw': u'00-1a-4a-a8-64-06', u'inet': [u'192.168.100.168'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': '3600', 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '22318374912', u'fs': u'NTFS'}], 'network': {u'vnet0': {'macAddr': u'00:1a:4a:a8:64:06', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'displayType': 'qxl', 'guestName': u'KLEE-DC01', 'boot': 'c', 'elapsedTime': '4', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Zabbix Agent 2.0.2', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'RHEV-Spice-Agent64', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10'], 'guestOs': u'Unknown', 'nice': '', 'status': 'RebootInProgress', 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'displayIp': '0', 'lastLogout': 1384914925.0776689, 'lastUser': u'None', 'clientIp': '192.168.50.51', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '5085719', 'apparentsize': '64424509440', 'writeLatency': '480185', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'flushLatency': '240100', 'readRate': '3254049.10', 'truesize': '64424509440', 'writeRate': '1605522.44'}}, 'monitorResponse': '0', 'statsAge': '1.84', 'cpuUser': '99.93', 'cpuIdle': '0.07', 'lastLogin': 1384914808.992449, 'guestIPs': u'192.168.100.168', 'displaySecurePort': u'5901'}]}
>Thread-15912498::DEBUG::2013-11-20 02:35:32,624::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af',) {}
>Thread-15912498::DEBUG::2013-11-20 02:35:32,625::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': 'Unknown', 'memUsage': '35', 'acpiEnable': 'true', 'pid': '19721', 'displayPort': u'5900', 'session': 'Unknown', 'netIfaces': [{u'inet6': [u'fe80::7d2d:e139:619b:7edc'], u'hw': u'00-1a-4a-a8-64-06', u'inet': [u'192.168.100.168'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': '3600', 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '22318374912', u'fs': u'NTFS'}], 'network': {u'vnet0': {'macAddr': u'00:1a:4a:a8:64:06', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'displayType': 'qxl', 'guestName': u'KLEE-DC01', 'boot': 'c', 'elapsedTime': '7', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Zabbix Agent 2.0.2', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'RHEV-Spice-Agent64', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10'], 'guestOs': u'Unknown', 'nice': '', 'status': 'RebootInProgress', 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'displayIp': '0', 'lastLogout': 1384914925.0776689, 'lastUser': u'None', 'clientIp': '192.168.50.51', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '5085719', 'apparentsize': '64424509440', 'writeLatency': '480185', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'flushLatency': '240100', 'readRate': '3254049.10', 'truesize': '64424509440', 'writeRate': '1605522.44'}}, 'monitorResponse': '0', 'statsAge': '0.93', 'cpuUser': '99.93', 'cpuIdle': '0.07', 'lastLogin': 1384914808.992449, 'guestIPs': u'192.168.100.168', 'displaySecurePort': u'5901'}]}
>Thread-1024767::ERROR::2013-11-20 02:35:34,050::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>libvirtEventLoop::DEBUG::2013-11-20 02:35:34,315::libvirtconnection::43::vm.Vm::(__eventCallback) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::graphics event phase 2 localAddr {'node': '192.168.100.133', 'service': '', 'family': 0} remoteAddr {'node': '192.168.50.51', 'service': '', 'family': 0}authScheme spice subject []
>libvirtEventLoop::DEBUG::2013-11-20 02:35:34,316::guestIF::154::vm.Vm::(desktopLock) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::desktopLock called
>Thread-15912500::DEBUG::2013-11-20 02:35:35,716::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af',) {}
>Thread-15912500::DEBUG::2013-11-20 02:35:35,717::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': 'Unknown', 'memUsage': '35', 'acpiEnable': 'true', 'pid': '19721', 'displayPort': u'5900', 'session': 'Unknown', 'netIfaces': [{u'inet6': [u'fe80::7d2d:e139:619b:7edc'], u'hw': u'00-1a-4a-a8-64-06', u'inet': [u'192.168.100.168'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': '3600', 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '22318374912', u'fs': u'NTFS'}], 'network': {u'vnet0': {'macAddr': u'00:1a:4a:a8:64:06', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'displayType': 'qxl', 'guestName': u'KLEE-DC01', 'boot': 'c', 'elapsedTime': '10', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Zabbix Agent 2.0.2', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'RHEV-Spice-Agent64', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10'], 'guestOs': u'Unknown', 'nice': '', 'status': 'RebootInProgress', 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'displayIp': '0', 'lastLogout': 1384914925.0776689, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '5085719', 'apparentsize': '64424509440', 'writeLatency': '480185', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'flushLatency': '240100', 'readRate': '3254049.10', 'truesize': '64424509440', 'writeRate': '1605522.44'}}, 'monitorResponse': '0', 'statsAge': '0.02', 'cpuUser': '99.93', 'cpuIdle': '0.07', 'lastLogin': 1384914808.992449, 'guestIPs': u'192.168.100.168', 'displaySecurePort': u'5901'}]}
>Thread-15912502::DEBUG::2013-11-20 02:35:38,900::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af',) {}
>Thread-15912502::DEBUG::2013-11-20 02:35:38,902::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': 'Unknown', 'memUsage': '35', 'acpiEnable': 'true', 'pid': '19721', 'displayPort': u'5900', 'session': 'Unknown', 'netIfaces': [{u'inet6': [u'fe80::7d2d:e139:619b:7edc'], u'hw': u'00-1a-4a-a8-64-06', u'inet': [u'192.168.100.168'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': '3600', 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '22318374912', u'fs': u'NTFS'}], 'network': {u'vnet0': {'macAddr': u'00:1a:4a:a8:64:06', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'displayType': 'qxl', 'guestName': u'KLEE-DC01', 'boot': 'c', 'elapsedTime': '13', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Zabbix Agent 2.0.2', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'RHEV-Spice-Agent64', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10'], 'guestOs': u'Unknown', 'nice': '', 'status': 'RebootInProgress', 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'displayIp': '0', 'lastLogout': 1384914925.0776689, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '5085719', 'apparentsize': '64424509440', 'writeLatency': '480185', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'flushLatency': '240100', 'readRate': '3254049.10', 'truesize': '64424509440', 'writeRate': '1605522.44'}}, 'monitorResponse': '0', 'statsAge': '1.20', 'cpuUser': '99.93', 'cpuIdle': '0.07', 'lastLogin': 1384914808.992449, 'guestIPs': u'192.168.100.168', 'displaySecurePort': u'5901'}]}
>Thread-15912503::INFO::2013-11-20 02:35:42,060::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912503::DEBUG::2013-11-20 02:35:42,061::task::495::TaskManager.Task::(_debug) Task 0339dc05-04fc-466a-a39d-5575018f2fa5: moving from state init -> state preparing
>Thread-15912503::DEBUG::2013-11-20 02:35:42,061::task::495::TaskManager.Task::(_debug) Task 0339dc05-04fc-466a-a39d-5575018f2fa5: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000271081924438', 'lastCheck': 1384914932.4941659, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00178790092468', 'lastCheck': 1384914934.052454, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00137805938721', 'lastCheck': 1384914934.0533841, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000355005264282', 'lastCheck': 1384914940.7691929, 'valid': True, 'code': 0}}
>Thread-15912503::DEBUG::2013-11-20 02:35:42,062::task::495::TaskManager.Task::(_debug) Task 0339dc05-04fc-466a-a39d-5575018f2fa5: moving from state preparing -> state finished
>Thread-15912503::DEBUG::2013-11-20 02:35:42,063::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912503::DEBUG::2013-11-20 02:35:42,063::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912503::DEBUG::2013-11-20 02:35:42,064::task::495::TaskManager.Task::(_debug) Task 0339dc05-04fc-466a-a39d-5575018f2fa5: ref 0 aborting False
>Thread-15912503::INFO::2013-11-20 02:35:42,065::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000271081924438', 'lastCheck': 1384914932.4941659, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00178790092468', 'lastCheck': 1384914934.052454, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00137805938721', 'lastCheck': 1384914934.0533841, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000355005264282', 'lastCheck': 1384914940.7691929, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-20 02:35:44,066::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912506::DEBUG::2013-11-20 02:35:45,993::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af',) {}
>Thread-15912506::DEBUG::2013-11-20 02:35:45,995::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': 'Unknown', 'memUsage': '35', 'acpiEnable': 'true', 'pid': '19721', 'displayPort': u'5900', 'session': 'Unknown', 'netIfaces': [{u'inet6': [u'fe80::7d2d:e139:619b:7edc'], u'hw': u'00-1a-4a-a8-64-06', u'inet': [u'192.168.100.168'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': '3600', 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '22318374912', u'fs': u'NTFS'}], 'network': {u'vnet0': {'macAddr': u'00:1a:4a:a8:64:06', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'displayType': 'qxl', 'guestName': u'KLEE-DC01', 'boot': 'c', 'elapsedTime': '20', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Zabbix Agent 2.0.2', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'RHEV-Spice-Agent64', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10'], 'guestOs': u'Unknown', 'nice': '', 'status': 'RebootInProgress', 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'displayIp': '0', 'lastLogout': 1384914925.0776689, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '5085719', 'apparentsize': '64424509440', 'writeLatency': '480185', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'flushLatency': '240100', 'readRate': '3254049.10', 'truesize': '64424509440', 'writeRate': '1605522.44'}}, 'monitorResponse': '0', 'statsAge': '0.29', 'cpuUser': '66.62', 'cpuIdle': '33.38', 'lastLogin': 1384914808.992449, 'guestIPs': u'192.168.100.168', 'displaySecurePort': u'5901'}]}
>Thread-15912508::DEBUG::2013-11-20 02:35:49,302::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af',) {}
>Thread-15912508::DEBUG::2013-11-20 02:35:49,303::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '19721', 'displayPort': u'5900', 'session': 'Unknown', 'netIfaces': [{u'inet6': [u'fe80::7d2d:e139:619b:7edc'], u'hw': u'00-1a-4a-a8-64-06', u'inet': [u'192.168.100.168'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': '3600', 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '22318374912', u'fs': u'NTFS'}], 'network': {u'vnet0': {'macAddr': u'00:1a:4a:a8:64:06', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'displayType': 'qxl', 'guestName': u'KLEE-DC01', 'boot': 'c', 'elapsedTime': '24', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Zabbix Agent 2.0.2', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'RHEV-Spice-Agent64', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10'], 'guestOs': u'Unknown', 'nice': '', 'status': 'RebootInProgress', 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'displayIp': '0', 'lastLogout': 1384914925.0776689, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '5085719', 'apparentsize': '64424509440', 'writeLatency': '480185', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'flushLatency': '240100', 'readRate': '3254049.10', 'truesize': '64424509440', 'writeRate': '1605522.44'}}, 'monitorResponse': '0', 'statsAge': '1.60', 'cpuUser': '66.62', 'cpuIdle': '33.38', 'lastLogin': 1384914808.992449, 'guestIPs': u'192.168.100.168', 'displaySecurePort': u'5901'}]}
>Thread-15912510::DEBUG::2013-11-20 02:35:52,790::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af',) {}
>Thread-15912510::DEBUG::2013-11-20 02:35:52,791::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '19721', 'displayPort': u'5900', 'session': 'Unknown', 'netIfaces': [{u'inet6': [u'fe80::7d2d:e139:619b:7edc'], u'hw': u'00-1a-4a-a8-64-06', u'inet': [u'192.168.100.168'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': '3600', 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '22318374912', u'fs': u'NTFS'}], 'network': {u'vnet0': {'macAddr': u'00:1a:4a:a8:64:06', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'displayType': 'qxl', 'guestName': u'KLEE-DC01', 'boot': 'c', 'elapsedTime': '27', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Zabbix Agent 2.0.2', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'RHEV-Spice-Agent64', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10'], 'guestOs': u'Unknown', 'nice': '', 'status': 'RebootInProgress', 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'displayIp': '0', 'lastLogout': 1384914925.0776689, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '5085719', 'apparentsize': '64424509440', 'writeLatency': '480185', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'flushLatency': '240100', 'readRate': '3254049.10', 'truesize': '64424509440', 'writeRate': '1605522.44'}}, 'monitorResponse': '0', 'statsAge': '1.08', 'cpuUser': '66.62', 'cpuIdle': '33.38', 'lastLogin': 1384914808.992449, 'guestIPs': u'192.168.100.168', 'displaySecurePort': u'5901'}]}
>Thread-1024767::ERROR::2013-11-20 02:35:54,084::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912392::INFO::2013-11-20 02:35:55,708::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 02:35:55,709::task::495::TaskManager.Task::(_debug) Task 0f74a0d8-17c5-4bfc-ad1a-d803f5306ea3: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 02:35:55,711::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ba945ca1-0e8e-47fc-be64-20f909b0d432`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 02:35:55,711::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 02:35:55,712::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 02:35:55,713::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ba945ca1-0e8e-47fc-be64-20f909b0d432`::Granted request
>Thread-15912392::DEBUG::2013-11-20 02:35:55,714::task::495::TaskManager.Task::(_debug) Task 0f74a0d8-17c5-4bfc-ad1a-d803f5306ea3: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 02:35:55,715::task::495::TaskManager.Task::(_debug) Task 0f74a0d8-17c5-4bfc-ad1a-d803f5306ea3: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 02:35:55,716::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15912392::DEBUG::2013-11-20 02:35:55,717::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15912512::DEBUG::2013-11-20 02:35:56,128::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af',) {}
>Thread-15912512::DEBUG::2013-11-20 02:35:56,130::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '19721', 'displayPort': u'5900', 'session': 'Unknown', 'netIfaces': [{u'inet6': [u'fe80::7d2d:e139:619b:7edc'], u'hw': u'00-1a-4a-a8-64-06', u'inet': [u'192.168.100.168'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': '3600', 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '22318374912', u'fs': u'NTFS'}], 'network': {u'vnet0': {'macAddr': u'00:1a:4a:a8:64:06', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'displayType': 'qxl', 'guestName': u'KLEE-DC01', 'boot': 'c', 'elapsedTime': '31', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Zabbix Agent 2.0.2', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'RHEV-Spice-Agent64', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10'], 'guestOs': u'Unknown', 'nice': '', 'status': 'RebootInProgress', 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'displayIp': '0', 'lastLogout': 1384914925.0776689, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '5085719', 'apparentsize': '64424509440', 'writeLatency': '480185', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'flushLatency': '240100', 'readRate': '3254049.10', 'truesize': '64424509440', 'writeRate': '1605522.44'}}, 'monitorResponse': '0', 'statsAge': '0.42', 'cpuUser': '66.62', 'cpuIdle': '33.38', 'lastLogin': 1384914808.992449, 'guestIPs': u'192.168.100.168', 'displaySecurePort': u'5901'}]}
>Thread-15912392::DEBUG::2013-11-20 02:35:56,637::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912392::DEBUG::2013-11-20 02:35:56,879::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15912392::DEBUG::2013-11-20 02:35:56,880::task::495::TaskManager.Task::(_debug) Task 0f74a0d8-17c5-4bfc-ad1a-d803f5306ea3: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 02:35:56,880::task::495::TaskManager.Task::(_debug) Task 0f74a0d8-17c5-4bfc-ad1a-d803f5306ea3: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 02:35:56,881::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 02:35:56,881::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 02:35:56,882::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 02:35:56,882::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 02:35:56,882::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 02:35:56,883::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 02:35:56,883::task::495::TaskManager.Task::(_debug) Task 0f74a0d8-17c5-4bfc-ad1a-d803f5306ea3: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 02:35:56,883::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912513::INFO::2013-11-20 02:36:00,981::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912513::DEBUG::2013-11-20 02:36:00,982::task::495::TaskManager.Task::(_debug) Task e78f88dd-4492-4e07-b729-9f48d5c7bc22: moving from state init -> state preparing
>Thread-15912513::DEBUG::2013-11-20 02:36:00,983::task::495::TaskManager.Task::(_debug) Task e78f88dd-4492-4e07-b729-9f48d5c7bc22: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000350952148438', 'lastCheck': 1384914952.517581, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00125908851624', 'lastCheck': 1384914954.0859089, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00121402740479', 'lastCheck': 1384914954.0868239, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00431394577026', 'lastCheck': 1384914960.939889, 'valid': True, 'code': 0}}
>Thread-15912513::DEBUG::2013-11-20 02:36:00,983::task::495::TaskManager.Task::(_debug) Task e78f88dd-4492-4e07-b729-9f48d5c7bc22: moving from state preparing -> state finished
>Thread-15912513::DEBUG::2013-11-20 02:36:00,984::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912513::DEBUG::2013-11-20 02:36:00,984::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912513::DEBUG::2013-11-20 02:36:00,985::task::495::TaskManager.Task::(_debug) Task e78f88dd-4492-4e07-b729-9f48d5c7bc22: ref 0 aborting False
>Thread-15912513::INFO::2013-11-20 02:36:00,986::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000350952148438', 'lastCheck': 1384914952.517581, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00125908851624', 'lastCheck': 1384914954.0859089, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00121402740479', 'lastCheck': 1384914954.0868239, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00431394577026', 'lastCheck': 1384914960.939889, 'valid': True, 'code': 0}}
>Thread-42838::DEBUG::2013-11-20 02:36:02,528::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 02:36:03,158::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 02:36:04,103::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912516::DEBUG::2013-11-20 02:36:04,722::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af',) {}
>Thread-15912516::DEBUG::2013-11-20 02:36:04,723::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '19721', 'displayPort': u'5900', 'session': 'Unknown', 'netIfaces': [{u'inet6': [u'fe80::7d2d:e139:619b:7edc'], u'hw': u'00-1a-4a-a8-64-06', u'inet': [u'192.168.100.168'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': '3600', 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '22318374912', u'fs': u'NTFS'}], 'network': {u'vnet0': {'macAddr': u'00:1a:4a:a8:64:06', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'displayType': 'qxl', 'guestName': u'KLEE-DC01', 'boot': 'c', 'elapsedTime': '39', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Zabbix Agent 2.0.2', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'RHEV-Spice-Agent64', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10'], 'guestOs': u'Unknown', 'nice': '', 'status': 'RebootInProgress', 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'displayIp': '0', 'lastLogout': 1384914925.0776689, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '2358866', 'apparentsize': '64424509440', 'writeLatency': '1259847', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'flushLatency': '238791', 'readRate': '9215864.07', 'truesize': '64424509440', 'writeRate': '1143350.56'}}, 'monitorResponse': '0', 'statsAge': '1.82', 'cpuUser': '67.96', 'cpuIdle': '32.04', 'lastLogin': 1384914808.992449, 'guestIPs': u'192.168.100.168', 'displaySecurePort': u'5901'}]}
>Thread-15912518::DEBUG::2013-11-20 02:36:08,606::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af',) {}
>Thread-15912518::DEBUG::2013-11-20 02:36:08,607::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '19721', 'displayPort': u'5900', 'session': 'Unknown', 'netIfaces': [{u'inet6': [u'fe80::7d2d:e139:619b:7edc'], u'hw': u'00-1a-4a-a8-64-06', u'inet': [u'192.168.100.168'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': '3600', 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '22318374912', u'fs': u'NTFS'}], 'network': {u'vnet0': {'macAddr': u'00:1a:4a:a8:64:06', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'displayType': 'qxl', 'guestName': u'KLEE-DC01', 'boot': 'c', 'elapsedTime': '43', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Zabbix Agent 2.0.2', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'RHEV-Spice-Agent64', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10'], 'guestOs': u'Unknown', 'nice': '', 'status': 'RebootInProgress', 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'displayIp': '0', 'lastLogout': 1384914925.0776689, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '2358866', 'apparentsize': '64424509440', 'writeLatency': '1259847', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'flushLatency': '238791', 'readRate': '9215864.07', 'truesize': '64424509440', 'writeRate': '1143350.56'}}, 'monitorResponse': '0', 'statsAge': '1.70', 'cpuUser': '67.96', 'cpuIdle': '32.04', 'lastLogin': 1384914808.992449, 'guestIPs': u'192.168.100.168', 'displaySecurePort': u'5901'}]}
>Thread-15912520::DEBUG::2013-11-20 02:36:12,273::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af',) {}
>Thread-15912520::DEBUG::2013-11-20 02:36:12,273::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '19721', 'displayPort': u'5900', 'session': 'Unknown', 'netIfaces': [{u'inet6': [u'fe80::7d2d:e139:619b:7edc'], u'hw': u'00-1a-4a-a8-64-06', u'inet': [u'192.168.100.168'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': '3600', 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '22318374912', u'fs': u'NTFS'}], 'network': {u'vnet0': {'macAddr': u'00:1a:4a:a8:64:06', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'displayType': 'qxl', 'guestName': u'KLEE-DC01', 'boot': 'c', 'elapsedTime': '47', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Zabbix Agent 2.0.2', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'RHEV-Spice-Agent64', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10'], 'guestOs': u'Unknown', 'nice': '', 'status': 'RebootInProgress', 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'displayIp': '0', 'lastLogout': 1384914925.0776689, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '2358866', 'apparentsize': '64424509440', 'writeLatency': '1259847', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'flushLatency': '238791', 'readRate': '9215864.07', 'truesize': '64424509440', 'writeRate': '1143350.56'}}, 'monitorResponse': '0', 'statsAge': '0.36', 'cpuUser': '99.83', 'cpuIdle': '0.17', 'lastLogin': 1384914808.992449, 'guestIPs': u'192.168.100.168', 'displaySecurePort': u'5901'}]}
>Thread-1024767::ERROR::2013-11-20 02:36:14,262::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912522::DEBUG::2013-11-20 02:36:15,595::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af',) {}
>Thread-15912522::DEBUG::2013-11-20 02:36:15,597::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '19721', 'displayPort': u'5900', 'session': 'Unknown', 'netIfaces': [{u'inet6': [u'fe80::7d2d:e139:619b:7edc'], u'hw': u'00-1a-4a-a8-64-06', u'inet': [u'192.168.100.168'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': '3600', 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '22318374912', u'fs': u'NTFS'}], 'network': {u'vnet0': {'macAddr': u'00:1a:4a:a8:64:06', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'displayType': 'qxl', 'guestName': u'KLEE-DC01', 'boot': 'c', 'elapsedTime': '50', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Zabbix Agent 2.0.2', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'RHEV-Spice-Agent64', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10'], 'guestOs': u'Unknown', 'nice': '', 'status': 'RebootInProgress', 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'displayIp': '0', 'lastLogout': 1384914925.0776689, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '2358866', 'apparentsize': '64424509440', 'writeLatency': '1259847', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'flushLatency': '238791', 'readRate': '9215864.07', 'truesize': '64424509440', 'writeRate': '1143350.56'}}, 'monitorResponse': '0', 'statsAge': '0.68', 'cpuUser': '99.83', 'cpuIdle': '0.17', 'lastLogin': 1384914808.992449, 'guestIPs': u'192.168.100.168', 'displaySecurePort': u'5901'}]}
>Thread-15912523::INFO::2013-11-20 02:36:19,096::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912523::DEBUG::2013-11-20 02:36:19,097::task::495::TaskManager.Task::(_debug) Task 3c31ce24-b2b8-4ea2-9c81-9ad70d17a724: moving from state init -> state preparing
>Thread-15912523::DEBUG::2013-11-20 02:36:19,098::task::495::TaskManager.Task::(_debug) Task 3c31ce24-b2b8-4ea2-9c81-9ad70d17a724: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000200033187866', 'lastCheck': 1384914973.1737111, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00108599662781', 'lastCheck': 1384914974.243679, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000864028930664', 'lastCheck': 1384914974.2636421, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00048303604126', 'lastCheck': 1384914970.9516399, 'valid': True, 'code': 0}}
>Thread-15912523::DEBUG::2013-11-20 02:36:19,098::task::495::TaskManager.Task::(_debug) Task 3c31ce24-b2b8-4ea2-9c81-9ad70d17a724: moving from state preparing -> state finished
>Thread-15912523::DEBUG::2013-11-20 02:36:19,099::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912523::DEBUG::2013-11-20 02:36:19,100::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912523::DEBUG::2013-11-20 02:36:19,100::task::495::TaskManager.Task::(_debug) Task 3c31ce24-b2b8-4ea2-9c81-9ad70d17a724: ref 0 aborting False
>Thread-15912523::INFO::2013-11-20 02:36:19,101::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000200033187866', 'lastCheck': 1384914973.1737111, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00108599662781', 'lastCheck': 1384914974.243679, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000864028930664', 'lastCheck': 1384914974.2636421, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00048303604126', 'lastCheck': 1384914970.9516399, 'valid': True, 'code': 0}}
>Thread-15912526::DEBUG::2013-11-20 02:36:22,747::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af',) {}
>Thread-15912526::DEBUG::2013-11-20 02:36:22,749::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '19721', 'displayPort': u'5900', 'session': 'Unknown', 'netIfaces': [{u'inet6': [u'fe80::7d2d:e139:619b:7edc'], u'hw': u'00-1a-4a-a8-64-06', u'inet': [u'192.168.100.168'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': '3600', 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '22318374912', u'fs': u'NTFS'}], 'network': {u'vnet0': {'macAddr': u'00:1a:4a:a8:64:06', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'displayType': 'qxl', 'guestName': u'KLEE-DC01', 'boot': 'c', 'elapsedTime': '57', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Zabbix Agent 2.0.2', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'RHEV-Spice-Agent64', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10'], 'guestOs': u'Unknown', 'nice': '', 'status': 'RebootInProgress', 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'displayIp': '0', 'lastLogout': 1384914925.0776689, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '2358866', 'apparentsize': '64424509440', 'writeLatency': '1259847', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'flushLatency': '238791', 'readRate': '9215864.07', 'truesize': '64424509440', 'writeRate': '1143350.56'}}, 'monitorResponse': '0', 'statsAge': '0.83', 'cpuUser': '99.83', 'cpuIdle': '0.17', 'lastLogin': 1384914808.992449, 'guestIPs': u'192.168.100.168', 'displaySecurePort': u'5901'}]}
>Thread-1024767::ERROR::2013-11-20 02:36:24,276::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912528::DEBUG::2013-11-20 02:36:25,932::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af',) {}
>Thread-15912528::DEBUG::2013-11-20 02:36:25,933::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '19721', 'displayPort': u'5900', 'session': 'Unknown', 'netIfaces': [{u'inet6': [u'fe80::7d2d:e139:619b:7edc'], u'hw': u'00-1a-4a-a8-64-06', u'inet': [u'192.168.100.168'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': '3600', 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '22318374912', u'fs': u'NTFS'}], 'network': {u'vnet0': {'macAddr': u'00:1a:4a:a8:64:06', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'displayType': 'qxl', 'guestName': u'KLEE-DC01', 'boot': 'c', 'elapsedTime': '60', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Zabbix Agent 2.0.2', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'RHEV-Spice-Agent64', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10'], 'guestOs': u'Unknown', 'nice': '', 'status': 'RebootInProgress', 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'displayIp': '0', 'lastLogout': 1384914925.0776689, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '2358866', 'apparentsize': '64424509440', 'writeLatency': '1259847', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'flushLatency': '238791', 'readRate': '9215864.07', 'truesize': '64424509440', 'writeRate': '1143350.56'}}, 'monitorResponse': '0', 'statsAge': '1.01', 'cpuUser': '99.83', 'cpuIdle': '0.17', 'lastLogin': 1384914808.992449, 'guestIPs': u'192.168.100.168', 'displaySecurePort': u'5901'}]}
>Thread-15912530::DEBUG::2013-11-20 02:36:29,787::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af',) {}
>Thread-15912530::DEBUG::2013-11-20 02:36:29,788::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '19721', 'displayPort': u'5900', 'session': 'Unknown', 'netIfaces': [{u'inet6': [u'fe80::7d2d:e139:619b:7edc'], u'hw': u'00-1a-4a-a8-64-06', u'inet': [u'192.168.100.168'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': '3600', 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '22318374912', u'fs': u'NTFS'}], 'network': {u'vnet0': {'macAddr': u'00:1a:4a:a8:64:06', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'displayType': 'qxl', 'guestName': u'KLEE-DC01', 'boot': 'c', 'elapsedTime': '64', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Zabbix Agent 2.0.2', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'RHEV-Spice-Agent64', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10'], 'guestOs': u'Unknown', 'nice': '', 'status': 'RebootInProgress', 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'displayIp': '0', 'lastLogout': 1384914925.0776689, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '2358866', 'apparentsize': '64424509440', 'writeLatency': '1259847', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'flushLatency': '238791', 'readRate': '9215864.07', 'truesize': '64424509440', 'writeRate': '1143350.56'}}, 'monitorResponse': '0', 'statsAge': '0.86', 'cpuUser': '153.23', 'cpuIdle': '0.00', 'lastLogin': 1384914808.992449, 'guestIPs': u'192.168.100.168', 'displaySecurePort': u'5901'}]}
>Thread-15912532::DEBUG::2013-11-20 02:36:33,897::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af',) {}
>Thread-15912532::DEBUG::2013-11-20 02:36:33,898::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '19721', 'displayPort': u'5900', 'session': 'Unknown', 'netIfaces': [{u'inet6': [u'fe80::7d2d:e139:619b:7edc'], u'hw': u'00-1a-4a-a8-64-06', u'inet': [u'192.168.100.168'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': '3600', 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '22318374912', u'fs': u'NTFS'}], 'network': {u'vnet0': {'macAddr': u'00:1a:4a:a8:64:06', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'displayType': 'qxl', 'guestName': u'KLEE-DC01', 'boot': 'c', 'elapsedTime': '68', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Zabbix Agent 2.0.2', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'RHEV-Spice-Agent64', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10'], 'guestOs': u'Unknown', 'nice': '', 'status': 'RebootInProgress', 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'displayIp': '0', 'lastLogout': 1384914925.0776689, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '2358866', 'apparentsize': '64424509440', 'writeLatency': '1259847', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'flushLatency': '238791', 'readRate': '9215864.07', 'truesize': '64424509440', 'writeRate': '1143350.56'}}, 'monitorResponse': '0', 'statsAge': '0.96', 'cpuUser': '153.23', 'cpuIdle': '0.00', 'lastLogin': 1384914808.992449, 'guestIPs': u'192.168.100.168', 'displaySecurePort': u'5901'}]}
>Thread-1024767::ERROR::2013-11-20 02:36:34,291::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912533::INFO::2013-11-20 02:36:36,978::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912533::DEBUG::2013-11-20 02:36:36,978::task::495::TaskManager.Task::(_debug) Task 0cc5872d-984d-46e9-8b4f-74480c882146: moving from state init -> state preparing
>Thread-15912533::DEBUG::2013-11-20 02:36:36,979::task::495::TaskManager.Task::(_debug) Task 0cc5872d-984d-46e9-8b4f-74480c882146: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000270128250122', 'lastCheck': 1384914993.1972289, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00135087966919', 'lastCheck': 1384914994.2721529, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00153589248657', 'lastCheck': 1384914994.294167, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000453948974609', 'lastCheck': 1384914990.9750359, 'valid': True, 'code': 0}}
>Thread-15912533::DEBUG::2013-11-20 02:36:36,980::task::495::TaskManager.Task::(_debug) Task 0cc5872d-984d-46e9-8b4f-74480c882146: moving from state preparing -> state finished
>Thread-15912533::DEBUG::2013-11-20 02:36:36,980::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912533::DEBUG::2013-11-20 02:36:36,981::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912533::DEBUG::2013-11-20 02:36:36,981::task::495::TaskManager.Task::(_debug) Task 0cc5872d-984d-46e9-8b4f-74480c882146: ref 0 aborting False
>Thread-15912533::INFO::2013-11-20 02:36:36,982::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000270128250122', 'lastCheck': 1384914993.1972289, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00135087966919', 'lastCheck': 1384914994.2721529, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00153589248657', 'lastCheck': 1384914994.294167, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000453948974609', 'lastCheck': 1384914990.9750359, 'valid': True, 'code': 0}}
>Thread-15912536::DEBUG::2013-11-20 02:36:40,548::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af',) {}
>Thread-15912536::DEBUG::2013-11-20 02:36:40,549::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '19721', 'displayPort': u'5900', 'session': 'Unknown', 'netIfaces': [{u'inet6': [u'fe80::7d2d:e139:619b:7edc'], u'hw': u'00-1a-4a-a8-64-06', u'inet': [u'192.168.100.168'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': '3600', 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '22318374912', u'fs': u'NTFS'}], 'network': {u'vnet0': {'macAddr': u'00:1a:4a:a8:64:06', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'displayType': 'qxl', 'guestName': u'KLEE-DC01', 'boot': 'c', 'elapsedTime': '75', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Zabbix Agent 2.0.2', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'RHEV-Spice-Agent64', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10'], 'guestOs': u'Unknown', 'nice': '', 'status': 'RebootInProgress', 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'displayIp': '0', 'lastLogout': 1384914925.0776689, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '2358866', 'apparentsize': '64424509440', 'writeLatency': '1259847', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'flushLatency': '238791', 'readRate': '9215864.07', 'truesize': '64424509440', 'writeRate': '1143350.56'}}, 'monitorResponse': '0', 'statsAge': '1.61', 'cpuUser': '153.23', 'cpuIdle': '0.00', 'lastLogin': 1384914808.992449, 'guestIPs': u'192.168.100.168', 'displaySecurePort': u'5901'}]}
>Thread-1024767::ERROR::2013-11-20 02:36:44,307::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912538::DEBUG::2013-11-20 02:36:45,248::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af',) {}
>Thread-15912538::DEBUG::2013-11-20 02:36:45,249::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '19721', 'displayPort': u'5900', 'session': 'Unknown', 'netIfaces': [{u'inet6': [u'fe80::7d2d:e139:619b:7edc'], u'hw': u'00-1a-4a-a8-64-06', u'inet': [u'192.168.100.168'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': '3600', 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '22318374912', u'fs': u'NTFS'}], 'network': {u'vnet0': {'macAddr': u'00:1a:4a:a8:64:06', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'displayType': 'qxl', 'guestName': u'KLEE-DC01', 'boot': 'c', 'elapsedTime': '80', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Zabbix Agent 2.0.2', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'RHEV-Spice-Agent64', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10'], 'guestOs': u'Unknown', 'nice': '', 'status': 'RebootInProgress', 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'displayIp': '0', 'lastLogout': 1384914925.0776689, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '2358866', 'apparentsize': '64424509440', 'writeLatency': '1259847', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'flushLatency': '238791', 'readRate': '9215864.07', 'truesize': '64424509440', 'writeRate': '1143350.56'}}, 'monitorResponse': '0', 'statsAge': '0.31', 'cpuUser': '186.46', 'cpuIdle': '0.00', 'lastLogin': 1384914808.992449, 'guestIPs': u'192.168.100.168', 'displaySecurePort': u'5901'}]}
>Thread-15912540::DEBUG::2013-11-20 02:36:50,644::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af',) {}
>Thread-15912540::DEBUG::2013-11-20 02:36:50,645::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '19721', 'displayPort': u'5900', 'session': 'Unknown', 'netIfaces': [{u'inet6': [u'fe80::7d2d:e139:619b:7edc'], u'hw': u'00-1a-4a-a8-64-06', u'inet': [u'192.168.100.168'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': '3600', 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '22318374912', u'fs': u'NTFS'}], 'network': {u'vnet0': {'macAddr': u'00:1a:4a:a8:64:06', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'displayType': 'qxl', 'guestName': u'KLEE-DC01', 'boot': 'c', 'elapsedTime': '85', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Zabbix Agent 2.0.2', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'RHEV-Spice-Agent64', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10'], 'guestOs': u'Unknown', 'nice': '', 'status': 'RebootInProgress', 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'displayIp': '0', 'lastLogout': 1384914925.0776689, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '2358866', 'apparentsize': '64424509440', 'writeLatency': '1259847', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'flushLatency': '238791', 'readRate': '9215864.07', 'truesize': '64424509440', 'writeRate': '1143350.56'}}, 'monitorResponse': '0', 'statsAge': '1.70', 'cpuUser': '186.46', 'cpuIdle': '0.00', 'lastLogin': 1384914808.992449, 'guestIPs': u'192.168.100.168', 'displaySecurePort': u'5901'}]}
>Thread-1024767::ERROR::2013-11-20 02:36:54,322::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912542::DEBUG::2013-11-20 02:36:54,394::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af',) {}
>Thread-15912542::DEBUG::2013-11-20 02:36:54,395::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '19721', 'displayPort': u'5900', 'session': 'Unknown', 'netIfaces': [{u'inet6': [u'fe80::7d2d:e139:619b:7edc'], u'hw': u'00-1a-4a-a8-64-06', u'inet': [u'192.168.100.168'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': '3600', 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '22318374912', u'fs': u'NTFS'}], 'network': {u'vnet0': {'macAddr': u'00:1a:4a:a8:64:06', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'displayType': 'qxl', 'guestName': u'KLEE-DC01', 'boot': 'c', 'elapsedTime': '89', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Zabbix Agent 2.0.2', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'RHEV-Spice-Agent64', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10'], 'guestOs': u'Unknown', 'nice': '', 'status': 'RebootInProgress', 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'displayIp': '0', 'lastLogout': 1384914925.0776689, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '2358866', 'apparentsize': '64424509440', 'writeLatency': '1259847', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'flushLatency': '238791', 'readRate': '9215864.07', 'truesize': '64424509440', 'writeRate': '1143350.56'}}, 'monitorResponse': '0', 'statsAge': '1.45', 'cpuUser': '186.46', 'cpuIdle': '0.00', 'lastLogin': 1384914808.992449, 'guestIPs': u'192.168.100.168', 'displaySecurePort': u'5901'}]}
>Thread-15912393::DEBUG::2013-11-20 02:36:54,482::guestIF::97::vm.Vm::(_handleMessage) vmId=`98ec88b5-2c08-4ff2-b607-c3f4e281f6af`::u'None'
>Thread-15912392::INFO::2013-11-20 02:36:56,946::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 02:36:56,946::task::495::TaskManager.Task::(_debug) Task cd74541d-3305-4916-898a-cd1262583a30: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 02:36:56,962::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d100d61e-d980-44dc-95c5-abfa941657fd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 02:36:56,962::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 02:36:56,963::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 02:36:56,963::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d100d61e-d980-44dc-95c5-abfa941657fd`::Granted request
>Thread-15912392::DEBUG::2013-11-20 02:36:56,964::task::495::TaskManager.Task::(_debug) Task cd74541d-3305-4916-898a-cd1262583a30: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 02:36:56,964::task::495::TaskManager.Task::(_debug) Task cd74541d-3305-4916-898a-cd1262583a30: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 02:36:56,965::task::495::TaskManager.Task::(_debug) Task cd74541d-3305-4916-898a-cd1262583a30: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 02:36:56,965::task::495::TaskManager.Task::(_debug) Task cd74541d-3305-4916-898a-cd1262583a30: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 02:36:56,965::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 02:36:56,966::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 02:36:56,966::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 02:36:56,967::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 02:36:56,967::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 02:36:56,968::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 02:36:56,968::task::495::TaskManager.Task::(_debug) Task cd74541d-3305-4916-898a-cd1262583a30: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 02:36:56,969::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912543::INFO::2013-11-20 02:36:57,902::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912543::DEBUG::2013-11-20 02:36:57,903::task::495::TaskManager.Task::(_debug) Task c084be9e-03a3-4e94-8315-15e9ec9b4faf: moving from state init -> state preparing
>Thread-15912543::DEBUG::2013-11-20 02:36:57,904::task::495::TaskManager.Task::(_debug) Task c084be9e-03a3-4e94-8315-15e9ec9b4faf: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000298023223877', 'lastCheck': 1384915013.221884, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00126886367798', 'lastCheck': 1384915014.303246, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00125193595886', 'lastCheck': 1384915014.324827, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000454902648926', 'lastCheck': 1384915010.99804, 'valid': True, 'code': 0}}
>Thread-15912543::DEBUG::2013-11-20 02:36:57,904::task::495::TaskManager.Task::(_debug) Task c084be9e-03a3-4e94-8315-15e9ec9b4faf: moving from state preparing -> state finished
>Thread-15912543::DEBUG::2013-11-20 02:36:57,905::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912543::DEBUG::2013-11-20 02:36:57,905::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912543::DEBUG::2013-11-20 02:36:57,906::task::495::TaskManager.Task::(_debug) Task c084be9e-03a3-4e94-8315-15e9ec9b4faf: ref 0 aborting False
>Thread-15912543::INFO::2013-11-20 02:36:57,907::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000298023223877', 'lastCheck': 1384915013.221884, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00126886367798', 'lastCheck': 1384915014.303246, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00125193595886', 'lastCheck': 1384915014.324827, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000454902648926', 'lastCheck': 1384915010.99804, 'valid': True, 'code': 0}}
>Thread-15912546::DEBUG::2013-11-20 02:37:03,157::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('98ec88b5-2c08-4ff2-b607-c3f4e281f6af',) {}
>Thread-15912546::DEBUG::2013-11-20 02:37:03,158::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'username': u'None', 'memUsage': '38', 'acpiEnable': 'true', 'pid': '19721', 'displayPort': u'5900', 'session': 'Unknown', 'netIfaces': [{u'inet6': [u'fe80::7d2d:e139:619b:7edc'], u'hw': u'00-1a-4a-a8-64-06', u'inet': [u'192.168.100.168'], u'name': u'Red Hat VirtIO Ethernet Adapter'}], 'timeOffset': '3600', 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'c:\\', u'total': '64055406592', u'used': '22318374912', u'fs': u'NTFS'}], 'network': {u'vnet0': {'macAddr': u'00:1a:4a:a8:64:06', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'displayType': 'qxl', 'guestName': u'KLEE-DC01', 'boot': 'c', 'elapsedTime': '98', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [u'Zabbix Agent 2.0.2', u'RHEV-Tools 3.0.39', u'RHEV-Network64 3.0.7', u'RHEV-Block64 3.0.9', u'RHEV-Spice-Agent64', u'Acronis Backup & Recovery 11.5 Agent Core', u'Acronis Backup & Recovery 11.5\xa0Agent for Windows', u'RHEV-Serial64 3.0.6', u'RHEV-Agent64 3.0.10'], 'guestOs': u'Unknown', 'nice': '', 'status': 'Running', 'vmId': '98ec88b5-2c08-4ff2-b607-c3f4e281f6af', 'displayIp': '0', 'lastLogout': 1384914925.0776689, 'lastUser': u'None', 'clientIp': '', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '2793268', 'apparentsize': '64424509440', 'writeLatency': '1563827', 'imageID': 'd8be6e76-2f7f-4b56-9ada-c3fd543a9794', 'flushLatency': '226682', 'readRate': '7366975.14', 'truesize': '64424509440', 'writeRate': '1650446.35'}}, 'monitorResponse': '0', 'statsAge': '0.17', 'cpuUser': '153.00', 'cpuIdle': '0.00', 'lastLogin': 1384915014.482013, 'guestIPs': u'192.168.100.168', 'displaySecurePort': u'5901'}]}
>Thread-1024767::ERROR::2013-11-20 02:37:04,337::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-1024767::ERROR::2013-11-20 02:37:14,352::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912550::INFO::2013-11-20 02:37:16,103::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912550::DEBUG::2013-11-20 02:37:16,104::task::495::TaskManager.Task::(_debug) Task d7611c87-e4e9-461b-89a6-1c16eb4e85d7: moving from state init -> state preparing
>Thread-15912550::DEBUG::2013-11-20 02:37:16,105::task::495::TaskManager.Task::(_debug) Task d7611c87-e4e9-461b-89a6-1c16eb4e85d7: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000288009643555', 'lastCheck': 1384915033.2466469, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00121998786926', 'lastCheck': 1384915034.3350379, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00104999542236', 'lastCheck': 1384915034.3543899, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000444889068604', 'lastCheck': 1384915031.019691, 'valid': True, 'code': 0}}
>Thread-15912550::DEBUG::2013-11-20 02:37:16,106::task::495::TaskManager.Task::(_debug) Task d7611c87-e4e9-461b-89a6-1c16eb4e85d7: moving from state preparing -> state finished
>Thread-15912550::DEBUG::2013-11-20 02:37:16,106::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912550::DEBUG::2013-11-20 02:37:16,107::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912550::DEBUG::2013-11-20 02:37:16,107::task::495::TaskManager.Task::(_debug) Task d7611c87-e4e9-461b-89a6-1c16eb4e85d7: ref 0 aborting False
>Thread-15912550::INFO::2013-11-20 02:37:16,108::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000288009643555', 'lastCheck': 1384915033.2466469, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00121998786926', 'lastCheck': 1384915034.3350379, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00104999542236', 'lastCheck': 1384915034.3543899, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000444889068604', 'lastCheck': 1384915031.019691, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-20 02:37:24,366::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912556::INFO::2013-11-20 02:37:31,761::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912556::DEBUG::2013-11-20 02:37:31,762::task::495::TaskManager.Task::(_debug) Task 128ea5c2-1618-4707-8728-324dd246f50b: moving from state init -> state preparing
>Thread-15912556::DEBUG::2013-11-20 02:37:31,763::task::495::TaskManager.Task::(_debug) Task 128ea5c2-1618-4707-8728-324dd246f50b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000309944152832', 'lastCheck': 1384915043.2583799, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00108313560486', 'lastCheck': 1384915044.3484759, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00147294998169', 'lastCheck': 1384915044.369036, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00042986869812', 'lastCheck': 1384915051.040086, 'valid': True, 'code': 0}}
>Thread-15912556::DEBUG::2013-11-20 02:37:31,763::task::495::TaskManager.Task::(_debug) Task 128ea5c2-1618-4707-8728-324dd246f50b: moving from state preparing -> state finished
>Thread-15912556::DEBUG::2013-11-20 02:37:31,764::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912556::DEBUG::2013-11-20 02:37:31,764::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912556::DEBUG::2013-11-20 02:37:31,765::task::495::TaskManager.Task::(_debug) Task 128ea5c2-1618-4707-8728-324dd246f50b: ref 0 aborting False
>Thread-15912556::INFO::2013-11-20 02:37:31,766::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000309944152832', 'lastCheck': 1384915043.2583799, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00108313560486', 'lastCheck': 1384915044.3484759, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00147294998169', 'lastCheck': 1384915044.369036, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00042986869812', 'lastCheck': 1384915051.040086, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-20 02:37:34,382::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-1024767::ERROR::2013-11-20 02:37:44,397::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912562::INFO::2013-11-20 02:37:48,882::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912562::DEBUG::2013-11-20 02:37:48,883::task::495::TaskManager.Task::(_debug) Task 13de12d3-742a-4933-b86d-06a9b19130e9: moving from state init -> state preparing
>Thread-15912562::DEBUG::2013-11-20 02:37:48,884::task::495::TaskManager.Task::(_debug) Task 13de12d3-742a-4933-b86d-06a9b19130e9: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000377178192139', 'lastCheck': 1384915063.283426, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.000998020172119', 'lastCheck': 1384915064.3790269, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000948190689087', 'lastCheck': 1384915064.399308, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000685930252075', 'lastCheck': 1384915061.0519381, 'valid': True, 'code': 0}}
>Thread-15912562::DEBUG::2013-11-20 02:37:48,885::task::495::TaskManager.Task::(_debug) Task 13de12d3-742a-4933-b86d-06a9b19130e9: moving from state preparing -> state finished
>Thread-15912562::DEBUG::2013-11-20 02:37:48,885::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912562::DEBUG::2013-11-20 02:37:48,886::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912562::DEBUG::2013-11-20 02:37:48,886::task::495::TaskManager.Task::(_debug) Task 13de12d3-742a-4933-b86d-06a9b19130e9: ref 0 aborting False
>Thread-15912562::INFO::2013-11-20 02:37:48,887::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000377178192139', 'lastCheck': 1384915063.283426, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.000998020172119', 'lastCheck': 1384915064.3790269, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000948190689087', 'lastCheck': 1384915064.399308, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000685930252075', 'lastCheck': 1384915061.0519381, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-20 02:37:54,412::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912392::INFO::2013-11-20 02:37:57,029::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 02:37:57,030::task::495::TaskManager.Task::(_debug) Task 53fb9720-289a-4733-b246-4cda3c27a894: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 02:37:57,031::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7863bc82-0a9a-4ab9-805b-76170e39a471`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 02:37:57,032::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 02:37:57,033::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 02:37:57,033::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7863bc82-0a9a-4ab9-805b-76170e39a471`::Granted request
>Thread-15912392::DEBUG::2013-11-20 02:37:57,034::task::495::TaskManager.Task::(_debug) Task 53fb9720-289a-4733-b246-4cda3c27a894: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 02:37:57,035::task::495::TaskManager.Task::(_debug) Task 53fb9720-289a-4733-b246-4cda3c27a894: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 02:37:57,035::task::495::TaskManager.Task::(_debug) Task 53fb9720-289a-4733-b246-4cda3c27a894: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 02:37:57,036::task::495::TaskManager.Task::(_debug) Task 53fb9720-289a-4733-b246-4cda3c27a894: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 02:37:57,036::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 02:37:57,037::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 02:37:57,037::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 02:37:57,038::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 02:37:57,038::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 02:37:57,039::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 02:37:57,040::task::495::TaskManager.Task::(_debug) Task 53fb9720-289a-4733-b246-4cda3c27a894: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 02:37:57,040::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-1024767::ERROR::2013-11-20 02:38:04,427::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912568::INFO::2013-11-20 02:38:04,497::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912568::DEBUG::2013-11-20 02:38:04,497::task::495::TaskManager.Task::(_debug) Task 5d869cba-1d01-49cb-b2a5-2ab56285eef6: moving from state init -> state preparing
>Thread-15912568::DEBUG::2013-11-20 02:38:04,498::task::495::TaskManager.Task::(_debug) Task 5d869cba-1d01-49cb-b2a5-2ab56285eef6: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000263929367065', 'lastCheck': 1384915083.3068709, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00174117088318', 'lastCheck': 1384915084.409812, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00137686729431', 'lastCheck': 1384915084.4300411, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00045919418335', 'lastCheck': 1384915081.0736721, 'valid': True, 'code': 0}}
>Thread-15912568::DEBUG::2013-11-20 02:38:04,499::task::495::TaskManager.Task::(_debug) Task 5d869cba-1d01-49cb-b2a5-2ab56285eef6: moving from state preparing -> state finished
>Thread-15912568::DEBUG::2013-11-20 02:38:04,499::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912568::DEBUG::2013-11-20 02:38:04,500::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912568::DEBUG::2013-11-20 02:38:04,500::task::495::TaskManager.Task::(_debug) Task 5d869cba-1d01-49cb-b2a5-2ab56285eef6: ref 0 aborting False
>Thread-15912568::INFO::2013-11-20 02:38:04,501::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000263929367065', 'lastCheck': 1384915083.3068709, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00174117088318', 'lastCheck': 1384915084.409812, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00137686729431', 'lastCheck': 1384915084.4300411, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00045919418335', 'lastCheck': 1384915081.0736721, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-20 02:38:14,442::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912574::INFO::2013-11-20 02:38:21,730::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912574::DEBUG::2013-11-20 02:38:21,730::task::495::TaskManager.Task::(_debug) Task 47edaa8b-150c-4078-a071-bc9ac46cbc95: moving from state init -> state preparing
>Thread-15912574::DEBUG::2013-11-20 02:38:21,731::task::495::TaskManager.Task::(_debug) Task 47edaa8b-150c-4078-a071-bc9ac46cbc95: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000293970108032', 'lastCheck': 1384915093.3193631, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00116300582886', 'lastCheck': 1384915094.4245601, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000926971435547', 'lastCheck': 1384915094.444376, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000432014465332', 'lastCheck': 1384915101.0949349, 'valid': True, 'code': 0}}
>Thread-15912574::DEBUG::2013-11-20 02:38:21,732::task::495::TaskManager.Task::(_debug) Task 47edaa8b-150c-4078-a071-bc9ac46cbc95: moving from state preparing -> state finished
>Thread-15912574::DEBUG::2013-11-20 02:38:21,732::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912574::DEBUG::2013-11-20 02:38:21,733::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912574::DEBUG::2013-11-20 02:38:21,733::task::495::TaskManager.Task::(_debug) Task 47edaa8b-150c-4078-a071-bc9ac46cbc95: ref 0 aborting False
>Thread-15912574::INFO::2013-11-20 02:38:21,734::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000293970108032', 'lastCheck': 1384915093.3193631, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00116300582886', 'lastCheck': 1384915094.4245601, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000926971435547', 'lastCheck': 1384915094.444376, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000432014465332', 'lastCheck': 1384915101.0949349, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-20 02:38:24,456::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-42840::DEBUG::2013-11-20 02:38:31,105::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 02:38:31,746::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 02:38:34,472::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912580::INFO::2013-11-20 02:38:38,089::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912580::DEBUG::2013-11-20 02:38:38,091::task::495::TaskManager.Task::(_debug) Task 687f51b5-2bd8-46e6-b56c-da9defb6a1fc: moving from state init -> state preparing
>Thread-15912580::DEBUG::2013-11-20 02:38:38,093::task::495::TaskManager.Task::(_debug) Task 687f51b5-2bd8-46e6-b56c-da9defb6a1fc: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000397920608521', 'lastCheck': 1384915113.3434701, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00105786323547', 'lastCheck': 1384915114.454313, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00091814994812', 'lastCheck': 1384915114.473804, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000346899032593', 'lastCheck': 1384915111.7482569, 'valid': True, 'code': 0}}
>Thread-15912580::DEBUG::2013-11-20 02:38:38,095::task::495::TaskManager.Task::(_debug) Task 687f51b5-2bd8-46e6-b56c-da9defb6a1fc: moving from state preparing -> state finished
>Thread-15912580::DEBUG::2013-11-20 02:38:38,096::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912580::DEBUG::2013-11-20 02:38:38,097::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912580::DEBUG::2013-11-20 02:38:38,098::task::495::TaskManager.Task::(_debug) Task 687f51b5-2bd8-46e6-b56c-da9defb6a1fc: ref 0 aborting False
>Thread-15912580::INFO::2013-11-20 02:38:38,099::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000397920608521', 'lastCheck': 1384915113.3434701, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00105786323547', 'lastCheck': 1384915114.454313, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00091814994812', 'lastCheck': 1384915114.473804, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000346899032593', 'lastCheck': 1384915111.7482569, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-20 02:38:44,488::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912586::INFO::2013-11-20 02:38:54,383::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912586::DEBUG::2013-11-20 02:38:54,384::task::495::TaskManager.Task::(_debug) Task c055f16f-d89d-4799-b4bc-012f9b87bd4b: moving from state init -> state preparing
>Thread-15912586::DEBUG::2013-11-20 02:38:54,384::task::495::TaskManager.Task::(_debug) Task c055f16f-d89d-4799-b4bc-012f9b87bd4b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000313997268677', 'lastCheck': 1384915133.3674231, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00156497955322', 'lastCheck': 1384915124.4701829, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00134205818176', 'lastCheck': 1384915124.490989, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00035285949707', 'lastCheck': 1384915131.7678101, 'valid': True, 'code': 0}}
>Thread-15912586::DEBUG::2013-11-20 02:38:54,385::task::495::TaskManager.Task::(_debug) Task c055f16f-d89d-4799-b4bc-012f9b87bd4b: moving from state preparing -> state finished
>Thread-15912586::DEBUG::2013-11-20 02:38:54,385::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912586::DEBUG::2013-11-20 02:38:54,386::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912586::DEBUG::2013-11-20 02:38:54,387::task::495::TaskManager.Task::(_debug) Task c055f16f-d89d-4799-b4bc-012f9b87bd4b: ref 0 aborting False
>Thread-15912586::INFO::2013-11-20 02:38:54,387::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000313997268677', 'lastCheck': 1384915133.3674231, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00156497955322', 'lastCheck': 1384915124.4701829, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00134205818176', 'lastCheck': 1384915124.490989, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00035285949707', 'lastCheck': 1384915131.7678101, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-20 02:38:54,505::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912392::INFO::2013-11-20 02:38:57,125::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 02:38:57,126::task::495::TaskManager.Task::(_debug) Task fb5789f1-c06d-4ea7-9c00-d62640fcfa29: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 02:38:57,127::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b9369e88-1b33-498e-b151-44f7cf84d387`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 02:38:57,128::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 02:38:57,128::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 02:38:57,129::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b9369e88-1b33-498e-b151-44f7cf84d387`::Granted request
>Thread-15912392::DEBUG::2013-11-20 02:38:57,130::task::495::TaskManager.Task::(_debug) Task fb5789f1-c06d-4ea7-9c00-d62640fcfa29: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 02:38:57,130::task::495::TaskManager.Task::(_debug) Task fb5789f1-c06d-4ea7-9c00-d62640fcfa29: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 02:38:57,131::task::495::TaskManager.Task::(_debug) Task fb5789f1-c06d-4ea7-9c00-d62640fcfa29: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 02:38:57,132::task::495::TaskManager.Task::(_debug) Task fb5789f1-c06d-4ea7-9c00-d62640fcfa29: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 02:38:57,132::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 02:38:57,133::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 02:38:57,133::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 02:38:57,134::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 02:38:57,134::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 02:38:57,135::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 02:38:57,135::task::495::TaskManager.Task::(_debug) Task fb5789f1-c06d-4ea7-9c00-d62640fcfa29: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 02:38:57,136::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-1024767::ERROR::2013-11-20 02:39:04,520::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912592::INFO::2013-11-20 02:39:10,435::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912592::DEBUG::2013-11-20 02:39:10,436::task::495::TaskManager.Task::(_debug) Task c875f357-a99f-47f6-8bc4-21b2fd3894b2: moving from state init -> state preparing
>Thread-15912592::DEBUG::2013-11-20 02:39:10,436::task::495::TaskManager.Task::(_debug) Task c875f357-a99f-47f6-8bc4-21b2fd3894b2: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000282049179077', 'lastCheck': 1384915143.3783841, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.131078004837', 'lastCheck': 1384915144.631851, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.397634029388', 'lastCheck': 1384915144.918833, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000341892242432', 'lastCheck': 1384915141.7792151, 'valid': True, 'code': 0}}
>Thread-15912592::DEBUG::2013-11-20 02:39:10,437::task::495::TaskManager.Task::(_debug) Task c875f357-a99f-47f6-8bc4-21b2fd3894b2: moving from state preparing -> state finished
>Thread-15912592::DEBUG::2013-11-20 02:39:10,438::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912592::DEBUG::2013-11-20 02:39:10,438::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912592::DEBUG::2013-11-20 02:39:10,439::task::495::TaskManager.Task::(_debug) Task c875f357-a99f-47f6-8bc4-21b2fd3894b2: ref 0 aborting False
>Thread-15912592::INFO::2013-11-20 02:39:10,439::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000282049179077', 'lastCheck': 1384915143.3783841, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.131078004837', 'lastCheck': 1384915144.631851, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.397634029388', 'lastCheck': 1384915144.918833, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000341892242432', 'lastCheck': 1384915141.7792151, 'valid': True, 'code': 0}}
>Thread-1024767::ERROR::2013-11-20 02:39:14,930::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-1024767::ERROR::2013-11-20 02:39:24,944::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912598::INFO::2013-11-20 02:39:26,537::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912598::DEBUG::2013-11-20 02:39:26,537::task::495::TaskManager.Task::(_debug) Task 27fbe1ad-bb0d-4e2a-af3d-e63330d5e293: moving from state init -> state preparing
>Thread-15912598::DEBUG::2013-11-20 02:39:26,538::task::495::TaskManager.Task::(_debug) Task 27fbe1ad-bb0d-4e2a-af3d-e63330d5e293: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000373840332031', 'lastCheck': 1384915163.4015219, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142788887024', 'lastCheck': 1384915164.660593, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00155401229858', 'lastCheck': 1384915164.947165, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000473022460938', 'lastCheck': 1384915161.8012991, 'valid': True, 'code': 0}}
>Thread-15912598::DEBUG::2013-11-20 02:39:26,538::task::495::TaskManager.Task::(_debug) Task 27fbe1ad-bb0d-4e2a-af3d-e63330d5e293: moving from state preparing -> state finished
>Thread-15912598::DEBUG::2013-11-20 02:39:26,538::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912598::DEBUG::2013-11-20 02:39:26,539::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912598::DEBUG::2013-11-20 02:39:26,539::task::495::TaskManager.Task::(_debug) Task 27fbe1ad-bb0d-4e2a-af3d-e63330d5e293: ref 0 aborting False
>Thread-15912598::INFO::2013-11-20 02:39:26,539::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000373840332031', 'lastCheck': 1384915163.4015219, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142788887024', 'lastCheck': 1384915164.660593, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00155401229858', 'lastCheck': 1384915164.947165, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000473022460938', 'lastCheck': 1384915161.8012991, 'valid': True, 'code': 0}}
>Thread-15912602::DEBUG::2013-11-20 02:39:34,417::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912602::INFO::2013-11-20 02:39:34,418::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912602::DEBUG::2013-11-20 02:39:34,418::task::495::TaskManager.Task::(_debug) Task de602967-a974-4eae-aa53-bbafddde71fa: moving from state init -> state preparing
>Thread-15912602::DEBUG::2013-11-20 02:39:34,419::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=Free spmLver=286 spmId=1
>Thread-15912602::DEBUG::2013-11-20 02:39:34,419::task::495::TaskManager.Task::(_debug) Task de602967-a974-4eae-aa53-bbafddde71fa: finished: {'spm_st': {'spmId': 1, 'spmStatus': 'Free', 'spmLver': 286}}
>Thread-15912602::DEBUG::2013-11-20 02:39:34,419::task::495::TaskManager.Task::(_debug) Task de602967-a974-4eae-aa53-bbafddde71fa: moving from state preparing -> state finished
>Thread-15912602::DEBUG::2013-11-20 02:39:34,420::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912602::DEBUG::2013-11-20 02:39:34,420::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912602::DEBUG::2013-11-20 02:39:34,420::task::495::TaskManager.Task::(_debug) Task de602967-a974-4eae-aa53-bbafddde71fa: ref 0 aborting False
>Thread-15912602::INFO::2013-11-20 02:39:34,421::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 1, 'spmStatus': 'Free', 'spmLver': 286}}
>Thread-1024767::ERROR::2013-11-20 02:39:34,960::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912603::DEBUG::2013-11-20 02:39:35,280::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912603::INFO::2013-11-20 02:39:35,281::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: spmStart, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 prevID=1 prevLVER=286 recoveryMode=0 scsiFencing=true maxHostID=250 domVersion=2)
>Thread-15912603::DEBUG::2013-11-20 02:39:35,281::task::495::TaskManager.Task::(_debug) Task 59c70706-854b-4ae3-91cc-4a2cfb6f46d4: moving from state init -> state preparing
>Thread-15912603::DEBUG::2013-11-20 02:39:35,283::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f0cb8553-bebe-44fd-9325-4c4c1d7bf6f6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '330' at 'public_spmStart'
>Thread-15912603::DEBUG::2013-11-20 02:39:35,283::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'exclusive'
>Thread-15912603::DEBUG::2013-11-20 02:39:35,284::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'exclusive' (1 active user)
>Thread-15912603::DEBUG::2013-11-20 02:39:35,284::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f0cb8553-bebe-44fd-9325-4c4c1d7bf6f6`::Granted request
>Thread-15912603::DEBUG::2013-11-20 02:39:35,285::task::495::TaskManager.Task::(_debug) Task 59c70706-854b-4ae3-91cc-4a2cfb6f46d4: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (exclusive)
>Thread-15912603::DEBUG::2013-11-20 02:39:35,286::task::495::TaskManager.Task::(_debug) Task 59c70706-854b-4ae3-91cc-4a2cfb6f46d4: ref 1 aborting False
>Thread-15912603::DEBUG::2013-11-20 02:39:35,286::task::495::TaskManager.Task::(_debug) Task 59c70706-854b-4ae3-91cc-4a2cfb6f46d4: Prepare: 1 jobs exist, move to acquiring
>Thread-15912603::DEBUG::2013-11-20 02:39:35,287::task::495::TaskManager.Task::(_debug) Task 59c70706-854b-4ae3-91cc-4a2cfb6f46d4: moving from state preparing -> state acquiring
>Thread-15912603::DEBUG::2013-11-20 02:39:35,287::task::495::TaskManager.Task::(_debug) Task 59c70706-854b-4ae3-91cc-4a2cfb6f46d4: moving from state acquiring -> state queued
>Thread-15912603::DEBUG::2013-11-20 02:39:35,288::taskManager::37::TaskManager::(_queueTask) queueing task: 59c70706-854b-4ae3-91cc-4a2cfb6f46d4
>Thread-15912603::DEBUG::2013-11-20 02:39:35,288::taskManager::43::TaskManager::(_queueTask) task queued: 59c70706-854b-4ae3-91cc-4a2cfb6f46d4
>3c078476-ceba-49ba-8c4f-cfc47f187488::DEBUG::2013-11-20 02:39:35,289::threadPool::47::Misc.ThreadPool::(setRunningTask) Number of running tasks: 1
>Thread-15912603::DEBUG::2013-11-20 02:39:35,289::task::1183::TaskManager.Task::(prepare) returning
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:35,290::threadPool::192::Misc.ThreadPool.WorkerThread::(run) Task: 59c70706-854b-4ae3-91cc-4a2cfb6f46d4 running: <bound method Task.commit of <storage.task.Task instance at 0x7f4870055290>> with: None
>Thread-15912603::DEBUG::2013-11-20 02:39:35,291::task::495::TaskManager.Task::(_debug) Task 59c70706-854b-4ae3-91cc-4a2cfb6f46d4: ref 0 aborting False
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:35,291::task::1195::TaskManager.Task::(commit) Commiting task: 59c70706-854b-4ae3-91cc-4a2cfb6f46d4
>Thread-15912603::INFO::2013-11-20 02:39:35,292::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: spmStart, Return response: {'status': {'message': 'OK', 'code': 0}, 'uuid': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:35,293::task::495::TaskManager.Task::(_debug) Task 59c70706-854b-4ae3-91cc-4a2cfb6f46d4: moving from state queued -> state running
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:35,294::task::907::TaskManager.Task::(_runJobs) Task.run: running job 0: spmStart: <bound method SPM.start of <storage.spm.SPM instance at 0x1bf9f38>> (args: ('c4e5889a-348d-11e2-83fe-9f72369eab30', 1, '286', 0, 'true', 250, 2) kwargs: {})
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:35,295::task::300::TaskManager.Task::(run) Job.run: running spmStart: <bound method SPM.start of <storage.spm.SPM instance at 0x1bf9f38>> (args: ('c4e5889a-348d-11e2-83fe-9f72369eab30', 1, '286', 0, 'true', 250, 2) kwargs: {}) callback None
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:35,295::spm::433::Storage.SPM::(start) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:35,295::misc::1010::SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads)
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:35,296::misc::1012::SamplingMethod::(__call__) Got in to sampling method
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:35,297::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:35,298::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:35,298::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:35,299::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:35,300::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:35,301::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:35,994::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:35,997::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:35,998::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_LVER=286', u'_SHA_CKSUM=d45d960da2c767fa0cdea03c6c2e2fb94c077f58', u'POOL_SPM_ID=-1']
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:36,000::misc::1020::SamplingMethod::(__call__) Returning last result
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:36,000::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:36,001::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:36,002::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:36,002::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:36,003::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:36,004::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15912604::DEBUG::2013-11-20 02:39:36,390::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912604::INFO::2013-11-20 02:39:36,392::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=59c70706-854b-4ae3-91cc-4a2cfb6f46d4)
>Thread-15912604::DEBUG::2013-11-20 02:39:36,392::task::495::TaskManager.Task::(_debug) Task a61e3400-2125-44ad-b442-9dab4b0f9eb5: moving from state init -> state preparing
>Thread-15912604::DEBUG::2013-11-20 02:39:36,393::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 59c70706-854b-4ae3-91cc-4a2cfb6f46d4
>Thread-15912604::DEBUG::2013-11-20 02:39:36,393::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}
>Thread-15912604::DEBUG::2013-11-20 02:39:36,394::task::495::TaskManager.Task::(_debug) Task a61e3400-2125-44ad-b442-9dab4b0f9eb5: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912604::DEBUG::2013-11-20 02:39:36,395::task::495::TaskManager.Task::(_debug) Task a61e3400-2125-44ad-b442-9dab4b0f9eb5: moving from state preparing -> state finished
>Thread-15912604::DEBUG::2013-11-20 02:39:36,395::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912604::DEBUG::2013-11-20 02:39:36,396::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912604::DEBUG::2013-11-20 02:39:36,396::task::495::TaskManager.Task::(_debug) Task a61e3400-2125-44ad-b442-9dab4b0f9eb5: ref 0 aborting False
>Thread-15912604::INFO::2013-11-20 02:39:36,397::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:36,528::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:36,531::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:36,532::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_LVER=286', u'_SHA_CKSUM=d45d960da2c767fa0cdea03c6c2e2fb94c077f58', u'POOL_SPM_ID=-1']
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::INFO::2013-11-20 02:39:36,533::spm::457::Storage.SPM::(start) expected previd:-1 lver:286 got request for previd:1 lver:286
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:36,535::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:36,535::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:36,536::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:36,536::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:36,537::persistentDict::163::Storage.PersistentDict::(__init__) Created a persistant dict with LvMetadataRW backend
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:36,538::blockSD::164::Storage.Misc.excCmd::(readlines) '/bin/dd iflag=direct skip=0 bs=2048 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:36,894::blockSD::164::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000227041 s, 9.0 MB/s\n'; <rc> = 0
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:36,895::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000227041 s, 9.0 MB/s'], size: 2048
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:36,895::persistentDict::204::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[]
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::WARNING::2013-11-20 02:39:36,895::persistentDict::226::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:36,896::persistentDict::163::Storage.PersistentDict::(__init__) Created a persistant dict with VGTagMetadataRW backend
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:36,896::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:36,897::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:36,897::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:36,898::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:36,898::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:36,899::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:37,420::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:37,423::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:37,424::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_LVER=286', u'_SHA_CKSUM=d45d960da2c767fa0cdea03c6c2e2fb94c077f58', u'POOL_SPM_ID=-1']
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:37,426::safelease::52::ClusterLock::(acquire) Acquiring cluster lock for domain 6b892067-ac7e-4039-a7be-5b7387525b16
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:37,427::safelease::61::Storage.Misc.excCmd::(acquire) '/usr/bin/sudo -n /usr/bin/setsid /usr/bin/ionice -c1 -n0 /bin/su vdsm -s /bin/sh -c "/usr/libexec/vdsm/spmprotect.sh start 6b892067-ac7e-4039-a7be-5b7387525b16 2 5 /dev/6b892067-ac7e-4039-a7be-5b7387525b16/leases 60000 10000 3"' (cwd /usr/libexec/vdsm/)
>Thread-15912606::DEBUG::2013-11-20 02:39:37,641::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912606::INFO::2013-11-20 02:39:37,644::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=59c70706-854b-4ae3-91cc-4a2cfb6f46d4)
>Thread-15912606::DEBUG::2013-11-20 02:39:37,645::task::495::TaskManager.Task::(_debug) Task 53894cc8-834f-42fd-bf22-07f48aa62fec: moving from state init -> state preparing
>Thread-15912606::DEBUG::2013-11-20 02:39:37,645::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 59c70706-854b-4ae3-91cc-4a2cfb6f46d4
>Thread-15912606::DEBUG::2013-11-20 02:39:37,646::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}
>Thread-15912606::DEBUG::2013-11-20 02:39:37,646::task::495::TaskManager.Task::(_debug) Task 53894cc8-834f-42fd-bf22-07f48aa62fec: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912606::DEBUG::2013-11-20 02:39:37,647::task::495::TaskManager.Task::(_debug) Task 53894cc8-834f-42fd-bf22-07f48aa62fec: moving from state preparing -> state finished
>Thread-15912606::DEBUG::2013-11-20 02:39:37,648::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912606::DEBUG::2013-11-20 02:39:37,648::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912606::DEBUG::2013-11-20 02:39:37,649::task::495::TaskManager.Task::(_debug) Task 53894cc8-834f-42fd-bf22-07f48aa62fec: ref 0 aborting False
>Thread-15912606::INFO::2013-11-20 02:39:37,649::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912607::DEBUG::2013-11-20 02:39:38,659::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912607::INFO::2013-11-20 02:39:38,660::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=59c70706-854b-4ae3-91cc-4a2cfb6f46d4)
>Thread-15912607::DEBUG::2013-11-20 02:39:38,661::task::495::TaskManager.Task::(_debug) Task 142e3c84-91d4-420b-ac86-49fead251ac6: moving from state init -> state preparing
>Thread-15912607::DEBUG::2013-11-20 02:39:38,661::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 59c70706-854b-4ae3-91cc-4a2cfb6f46d4
>Thread-15912607::DEBUG::2013-11-20 02:39:38,662::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}
>Thread-15912607::DEBUG::2013-11-20 02:39:38,662::task::495::TaskManager.Task::(_debug) Task 142e3c84-91d4-420b-ac86-49fead251ac6: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912607::DEBUG::2013-11-20 02:39:38,663::task::495::TaskManager.Task::(_debug) Task 142e3c84-91d4-420b-ac86-49fead251ac6: moving from state preparing -> state finished
>Thread-15912607::DEBUG::2013-11-20 02:39:38,663::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912607::DEBUG::2013-11-20 02:39:38,664::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912607::DEBUG::2013-11-20 02:39:38,665::task::495::TaskManager.Task::(_debug) Task 142e3c84-91d4-420b-ac86-49fead251ac6: ref 0 aborting False
>Thread-15912607::INFO::2013-11-20 02:39:38,665::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912608::DEBUG::2013-11-20 02:39:39,679::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912608::INFO::2013-11-20 02:39:39,680::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=59c70706-854b-4ae3-91cc-4a2cfb6f46d4)
>Thread-15912608::DEBUG::2013-11-20 02:39:39,681::task::495::TaskManager.Task::(_debug) Task 5789907b-890e-4fd1-9787-cef71af4b513: moving from state init -> state preparing
>Thread-15912608::DEBUG::2013-11-20 02:39:39,681::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 59c70706-854b-4ae3-91cc-4a2cfb6f46d4
>Thread-15912608::DEBUG::2013-11-20 02:39:39,682::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}
>Thread-15912608::DEBUG::2013-11-20 02:39:39,682::task::495::TaskManager.Task::(_debug) Task 5789907b-890e-4fd1-9787-cef71af4b513: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912608::DEBUG::2013-11-20 02:39:39,683::task::495::TaskManager.Task::(_debug) Task 5789907b-890e-4fd1-9787-cef71af4b513: moving from state preparing -> state finished
>Thread-15912608::DEBUG::2013-11-20 02:39:39,684::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912608::DEBUG::2013-11-20 02:39:39,684::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912608::DEBUG::2013-11-20 02:39:39,685::task::495::TaskManager.Task::(_debug) Task 5789907b-890e-4fd1-9787-cef71af4b513: ref 0 aborting False
>Thread-15912608::INFO::2013-11-20 02:39:39,685::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912610::DEBUG::2013-11-20 02:39:42,225::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912610::INFO::2013-11-20 02:39:42,226::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=59c70706-854b-4ae3-91cc-4a2cfb6f46d4)
>Thread-15912610::DEBUG::2013-11-20 02:39:42,226::task::495::TaskManager.Task::(_debug) Task a7ca7cb0-f868-45e7-8bbb-0398f815e0ad: moving from state init -> state preparing
>Thread-15912610::DEBUG::2013-11-20 02:39:42,227::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 59c70706-854b-4ae3-91cc-4a2cfb6f46d4
>Thread-15912610::DEBUG::2013-11-20 02:39:42,227::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}
>Thread-15912610::DEBUG::2013-11-20 02:39:42,228::task::495::TaskManager.Task::(_debug) Task a7ca7cb0-f868-45e7-8bbb-0398f815e0ad: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912610::DEBUG::2013-11-20 02:39:42,229::task::495::TaskManager.Task::(_debug) Task a7ca7cb0-f868-45e7-8bbb-0398f815e0ad: moving from state preparing -> state finished
>Thread-15912610::DEBUG::2013-11-20 02:39:42,229::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912610::DEBUG::2013-11-20 02:39:42,230::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912610::DEBUG::2013-11-20 02:39:42,230::task::495::TaskManager.Task::(_debug) Task a7ca7cb0-f868-45e7-8bbb-0398f815e0ad: ref 0 aborting False
>Thread-15912610::INFO::2013-11-20 02:39:42,231::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912611::INFO::2013-11-20 02:39:42,806::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912611::DEBUG::2013-11-20 02:39:42,807::task::495::TaskManager.Task::(_debug) Task 8e19232c-40c5-4222-86bc-dcb8452728d9: moving from state init -> state preparing
>Thread-15912611::DEBUG::2013-11-20 02:39:42,830::task::495::TaskManager.Task::(_debug) Task 8e19232c-40c5-4222-86bc-dcb8452728d9: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000296115875244', 'lastCheck': 1384915173.4126129, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00160193443298', 'lastCheck': 1384915174.6759131, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00136685371399', 'lastCheck': 1384915174.963186, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000437021255493', 'lastCheck': 1384915181.8235841, 'valid': True, 'code': 0}}
>Thread-15912611::DEBUG::2013-11-20 02:39:42,830::task::495::TaskManager.Task::(_debug) Task 8e19232c-40c5-4222-86bc-dcb8452728d9: moving from state preparing -> state finished
>Thread-15912611::DEBUG::2013-11-20 02:39:42,831::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912611::DEBUG::2013-11-20 02:39:42,831::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912611::DEBUG::2013-11-20 02:39:42,832::task::495::TaskManager.Task::(_debug) Task 8e19232c-40c5-4222-86bc-dcb8452728d9: ref 0 aborting False
>Thread-15912611::INFO::2013-11-20 02:39:42,833::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000296115875244', 'lastCheck': 1384915173.4126129, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00160193443298', 'lastCheck': 1384915174.6759131, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00136685371399', 'lastCheck': 1384915174.963186, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000437021255493', 'lastCheck': 1384915181.8235841, 'valid': True, 'code': 0}}
>Thread-15912613::DEBUG::2013-11-20 02:39:43,240::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912613::INFO::2013-11-20 02:39:43,241::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=59c70706-854b-4ae3-91cc-4a2cfb6f46d4)
>Thread-15912613::DEBUG::2013-11-20 02:39:43,241::task::495::TaskManager.Task::(_debug) Task 95aba5e7-b920-4002-8035-4bba0bed81e7: moving from state init -> state preparing
>Thread-15912613::DEBUG::2013-11-20 02:39:43,242::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 59c70706-854b-4ae3-91cc-4a2cfb6f46d4
>Thread-15912613::DEBUG::2013-11-20 02:39:43,242::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}
>Thread-15912613::DEBUG::2013-11-20 02:39:43,243::task::495::TaskManager.Task::(_debug) Task 95aba5e7-b920-4002-8035-4bba0bed81e7: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912613::DEBUG::2013-11-20 02:39:43,243::task::495::TaskManager.Task::(_debug) Task 95aba5e7-b920-4002-8035-4bba0bed81e7: moving from state preparing -> state finished
>Thread-15912613::DEBUG::2013-11-20 02:39:43,244::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912613::DEBUG::2013-11-20 02:39:43,244::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912613::DEBUG::2013-11-20 02:39:43,245::task::495::TaskManager.Task::(_debug) Task 95aba5e7-b920-4002-8035-4bba0bed81e7: ref 0 aborting False
>Thread-15912613::INFO::2013-11-20 02:39:43,245::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912614::DEBUG::2013-11-20 02:39:44,259::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912614::INFO::2013-11-20 02:39:44,260::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=59c70706-854b-4ae3-91cc-4a2cfb6f46d4)
>Thread-15912614::DEBUG::2013-11-20 02:39:44,261::task::495::TaskManager.Task::(_debug) Task 4655205c-51ab-4d13-9b72-5cbe9726e343: moving from state init -> state preparing
>Thread-15912614::DEBUG::2013-11-20 02:39:44,261::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 59c70706-854b-4ae3-91cc-4a2cfb6f46d4
>Thread-15912614::DEBUG::2013-11-20 02:39:44,262::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}
>Thread-15912614::DEBUG::2013-11-20 02:39:44,262::task::495::TaskManager.Task::(_debug) Task 4655205c-51ab-4d13-9b72-5cbe9726e343: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912614::DEBUG::2013-11-20 02:39:44,263::task::495::TaskManager.Task::(_debug) Task 4655205c-51ab-4d13-9b72-5cbe9726e343: moving from state preparing -> state finished
>Thread-15912614::DEBUG::2013-11-20 02:39:44,263::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912614::DEBUG::2013-11-20 02:39:44,264::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912614::DEBUG::2013-11-20 02:39:44,264::task::495::TaskManager.Task::(_debug) Task 4655205c-51ab-4d13-9b72-5cbe9726e343: ref 0 aborting False
>Thread-15912614::INFO::2013-11-20 02:39:44,265::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-1024767::ERROR::2013-11-20 02:39:45,090::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912615::DEBUG::2013-11-20 02:39:45,274::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912615::INFO::2013-11-20 02:39:45,275::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=59c70706-854b-4ae3-91cc-4a2cfb6f46d4)
>Thread-15912615::DEBUG::2013-11-20 02:39:45,276::task::495::TaskManager.Task::(_debug) Task 1dcf3e44-4f98-4e53-be84-b439574c56d0: moving from state init -> state preparing
>Thread-15912615::DEBUG::2013-11-20 02:39:45,276::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 59c70706-854b-4ae3-91cc-4a2cfb6f46d4
>Thread-15912615::DEBUG::2013-11-20 02:39:45,277::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}
>Thread-15912615::DEBUG::2013-11-20 02:39:45,277::task::495::TaskManager.Task::(_debug) Task 1dcf3e44-4f98-4e53-be84-b439574c56d0: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912615::DEBUG::2013-11-20 02:39:45,278::task::495::TaskManager.Task::(_debug) Task 1dcf3e44-4f98-4e53-be84-b439574c56d0: moving from state preparing -> state finished
>Thread-15912615::DEBUG::2013-11-20 02:39:45,278::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912615::DEBUG::2013-11-20 02:39:45,279::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912615::DEBUG::2013-11-20 02:39:45,279::task::495::TaskManager.Task::(_debug) Task 1dcf3e44-4f98-4e53-be84-b439574c56d0: ref 0 aborting False
>Thread-15912615::INFO::2013-11-20 02:39:45,280::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912616::DEBUG::2013-11-20 02:39:46,289::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912616::INFO::2013-11-20 02:39:46,291::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=59c70706-854b-4ae3-91cc-4a2cfb6f46d4)
>Thread-15912616::DEBUG::2013-11-20 02:39:46,291::task::495::TaskManager.Task::(_debug) Task c14a2c5d-be78-4cf5-a17f-e26a5d98b616: moving from state init -> state preparing
>Thread-15912616::DEBUG::2013-11-20 02:39:46,292::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 59c70706-854b-4ae3-91cc-4a2cfb6f46d4
>Thread-15912616::DEBUG::2013-11-20 02:39:46,292::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}
>Thread-15912616::DEBUG::2013-11-20 02:39:46,293::task::495::TaskManager.Task::(_debug) Task c14a2c5d-be78-4cf5-a17f-e26a5d98b616: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912616::DEBUG::2013-11-20 02:39:46,293::task::495::TaskManager.Task::(_debug) Task c14a2c5d-be78-4cf5-a17f-e26a5d98b616: moving from state preparing -> state finished
>Thread-15912616::DEBUG::2013-11-20 02:39:46,294::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912616::DEBUG::2013-11-20 02:39:46,294::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912616::DEBUG::2013-11-20 02:39:46,295::task::495::TaskManager.Task::(_debug) Task c14a2c5d-be78-4cf5-a17f-e26a5d98b616: ref 0 aborting False
>Thread-15912616::INFO::2013-11-20 02:39:46,295::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912618::DEBUG::2013-11-20 02:39:47,306::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912618::INFO::2013-11-20 02:39:47,308::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=59c70706-854b-4ae3-91cc-4a2cfb6f46d4)
>Thread-15912618::DEBUG::2013-11-20 02:39:47,308::task::495::TaskManager.Task::(_debug) Task 8dce0a05-777f-4595-9c98-09b25f7f8d15: moving from state init -> state preparing
>Thread-15912618::DEBUG::2013-11-20 02:39:47,309::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 59c70706-854b-4ae3-91cc-4a2cfb6f46d4
>Thread-15912618::DEBUG::2013-11-20 02:39:47,309::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}
>Thread-15912618::DEBUG::2013-11-20 02:39:47,310::task::495::TaskManager.Task::(_debug) Task 8dce0a05-777f-4595-9c98-09b25f7f8d15: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912618::DEBUG::2013-11-20 02:39:47,310::task::495::TaskManager.Task::(_debug) Task 8dce0a05-777f-4595-9c98-09b25f7f8d15: moving from state preparing -> state finished
>Thread-15912618::DEBUG::2013-11-20 02:39:47,311::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912618::DEBUG::2013-11-20 02:39:47,311::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912618::DEBUG::2013-11-20 02:39:47,312::task::495::TaskManager.Task::(_debug) Task 8dce0a05-777f-4595-9c98-09b25f7f8d15: ref 0 aborting False
>Thread-15912618::INFO::2013-11-20 02:39:47,312::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912619::DEBUG::2013-11-20 02:39:48,322::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912619::INFO::2013-11-20 02:39:48,323::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=59c70706-854b-4ae3-91cc-4a2cfb6f46d4)
>Thread-15912619::DEBUG::2013-11-20 02:39:48,324::task::495::TaskManager.Task::(_debug) Task d03618af-c27a-4f8e-82a0-484dcf3f3419: moving from state init -> state preparing
>Thread-15912619::DEBUG::2013-11-20 02:39:48,325::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 59c70706-854b-4ae3-91cc-4a2cfb6f46d4
>Thread-15912619::DEBUG::2013-11-20 02:39:48,325::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}
>Thread-15912619::DEBUG::2013-11-20 02:39:48,326::task::495::TaskManager.Task::(_debug) Task d03618af-c27a-4f8e-82a0-484dcf3f3419: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912619::DEBUG::2013-11-20 02:39:48,326::task::495::TaskManager.Task::(_debug) Task d03618af-c27a-4f8e-82a0-484dcf3f3419: moving from state preparing -> state finished
>Thread-15912619::DEBUG::2013-11-20 02:39:48,327::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912619::DEBUG::2013-11-20 02:39:48,327::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912619::DEBUG::2013-11-20 02:39:48,328::task::495::TaskManager.Task::(_debug) Task d03618af-c27a-4f8e-82a0-484dcf3f3419: ref 0 aborting False
>Thread-15912619::INFO::2013-11-20 02:39:48,328::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912620::DEBUG::2013-11-20 02:39:49,339::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912620::INFO::2013-11-20 02:39:49,341::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=59c70706-854b-4ae3-91cc-4a2cfb6f46d4)
>Thread-15912620::DEBUG::2013-11-20 02:39:49,342::task::495::TaskManager.Task::(_debug) Task 96cd8d6a-2eb7-4232-a523-4bc4fba46fd6: moving from state init -> state preparing
>Thread-15912620::DEBUG::2013-11-20 02:39:49,342::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 59c70706-854b-4ae3-91cc-4a2cfb6f46d4
>Thread-15912620::DEBUG::2013-11-20 02:39:49,343::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}
>Thread-15912620::DEBUG::2013-11-20 02:39:49,344::task::495::TaskManager.Task::(_debug) Task 96cd8d6a-2eb7-4232-a523-4bc4fba46fd6: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912620::DEBUG::2013-11-20 02:39:49,345::task::495::TaskManager.Task::(_debug) Task 96cd8d6a-2eb7-4232-a523-4bc4fba46fd6: moving from state preparing -> state finished
>Thread-15912620::DEBUG::2013-11-20 02:39:49,345::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912620::DEBUG::2013-11-20 02:39:49,346::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912620::DEBUG::2013-11-20 02:39:49,347::task::495::TaskManager.Task::(_debug) Task 96cd8d6a-2eb7-4232-a523-4bc4fba46fd6: ref 0 aborting False
>Thread-15912620::INFO::2013-11-20 02:39:49,348::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912622::DEBUG::2013-11-20 02:39:50,490::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912622::INFO::2013-11-20 02:39:50,492::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=59c70706-854b-4ae3-91cc-4a2cfb6f46d4)
>Thread-15912622::DEBUG::2013-11-20 02:39:50,493::task::495::TaskManager.Task::(_debug) Task 07da0950-bb80-4ca1-b18d-25538c2205ca: moving from state init -> state preparing
>Thread-15912622::DEBUG::2013-11-20 02:39:50,494::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 59c70706-854b-4ae3-91cc-4a2cfb6f46d4
>Thread-15912622::DEBUG::2013-11-20 02:39:50,495::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}
>Thread-15912622::DEBUG::2013-11-20 02:39:50,496::task::495::TaskManager.Task::(_debug) Task 07da0950-bb80-4ca1-b18d-25538c2205ca: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912622::DEBUG::2013-11-20 02:39:50,497::task::495::TaskManager.Task::(_debug) Task 07da0950-bb80-4ca1-b18d-25538c2205ca: moving from state preparing -> state finished
>Thread-15912622::DEBUG::2013-11-20 02:39:50,497::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912622::DEBUG::2013-11-20 02:39:50,498::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912622::DEBUG::2013-11-20 02:39:50,499::task::495::TaskManager.Task::(_debug) Task 07da0950-bb80-4ca1-b18d-25538c2205ca: ref 0 aborting False
>Thread-15912622::INFO::2013-11-20 02:39:50,500::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912623::DEBUG::2013-11-20 02:39:51,511::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912623::INFO::2013-11-20 02:39:51,512::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=59c70706-854b-4ae3-91cc-4a2cfb6f46d4)
>Thread-15912623::DEBUG::2013-11-20 02:39:51,513::task::495::TaskManager.Task::(_debug) Task fa4abc5d-8f60-4dcb-bfcd-9f3be8cf234b: moving from state init -> state preparing
>Thread-15912623::DEBUG::2013-11-20 02:39:51,513::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 59c70706-854b-4ae3-91cc-4a2cfb6f46d4
>Thread-15912623::DEBUG::2013-11-20 02:39:51,514::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}
>Thread-15912623::DEBUG::2013-11-20 02:39:51,514::task::495::TaskManager.Task::(_debug) Task fa4abc5d-8f60-4dcb-bfcd-9f3be8cf234b: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912623::DEBUG::2013-11-20 02:39:51,515::task::495::TaskManager.Task::(_debug) Task fa4abc5d-8f60-4dcb-bfcd-9f3be8cf234b: moving from state preparing -> state finished
>Thread-15912623::DEBUG::2013-11-20 02:39:51,515::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912623::DEBUG::2013-11-20 02:39:51,516::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912623::DEBUG::2013-11-20 02:39:51,516::task::495::TaskManager.Task::(_debug) Task fa4abc5d-8f60-4dcb-bfcd-9f3be8cf234b: ref 0 aborting False
>Thread-15912623::INFO::2013-11-20 02:39:51,517::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912624::DEBUG::2013-11-20 02:39:52,528::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912624::INFO::2013-11-20 02:39:52,530::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=59c70706-854b-4ae3-91cc-4a2cfb6f46d4)
>Thread-15912624::DEBUG::2013-11-20 02:39:52,531::task::495::TaskManager.Task::(_debug) Task 568c0a63-a955-4623-a205-eeae068cc5e6: moving from state init -> state preparing
>Thread-15912624::DEBUG::2013-11-20 02:39:52,531::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 59c70706-854b-4ae3-91cc-4a2cfb6f46d4
>Thread-15912624::DEBUG::2013-11-20 02:39:52,532::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}
>Thread-15912624::DEBUG::2013-11-20 02:39:52,533::task::495::TaskManager.Task::(_debug) Task 568c0a63-a955-4623-a205-eeae068cc5e6: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912624::DEBUG::2013-11-20 02:39:52,533::task::495::TaskManager.Task::(_debug) Task 568c0a63-a955-4623-a205-eeae068cc5e6: moving from state preparing -> state finished
>Thread-15912624::DEBUG::2013-11-20 02:39:52,534::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912624::DEBUG::2013-11-20 02:39:52,534::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912624::DEBUG::2013-11-20 02:39:52,535::task::495::TaskManager.Task::(_debug) Task 568c0a63-a955-4623-a205-eeae068cc5e6: ref 0 aborting False
>Thread-15912624::INFO::2013-11-20 02:39:52,535::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912626::DEBUG::2013-11-20 02:39:53,546::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912626::INFO::2013-11-20 02:39:53,547::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=59c70706-854b-4ae3-91cc-4a2cfb6f46d4)
>Thread-15912626::DEBUG::2013-11-20 02:39:53,547::task::495::TaskManager.Task::(_debug) Task c24771bb-9f92-414f-9405-dd013dca9d06: moving from state init -> state preparing
>Thread-15912626::DEBUG::2013-11-20 02:39:53,548::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 59c70706-854b-4ae3-91cc-4a2cfb6f46d4
>Thread-15912626::DEBUG::2013-11-20 02:39:53,548::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}
>Thread-15912626::DEBUG::2013-11-20 02:39:53,549::task::495::TaskManager.Task::(_debug) Task c24771bb-9f92-414f-9405-dd013dca9d06: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912626::DEBUG::2013-11-20 02:39:53,550::task::495::TaskManager.Task::(_debug) Task c24771bb-9f92-414f-9405-dd013dca9d06: moving from state preparing -> state finished
>Thread-15912626::DEBUG::2013-11-20 02:39:53,550::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912626::DEBUG::2013-11-20 02:39:53,551::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912626::DEBUG::2013-11-20 02:39:53,551::task::495::TaskManager.Task::(_debug) Task c24771bb-9f92-414f-9405-dd013dca9d06: ref 0 aborting False
>Thread-15912626::INFO::2013-11-20 02:39:53,552::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912627::DEBUG::2013-11-20 02:39:54,561::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912627::INFO::2013-11-20 02:39:54,562::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=59c70706-854b-4ae3-91cc-4a2cfb6f46d4)
>Thread-15912627::DEBUG::2013-11-20 02:39:54,563::task::495::TaskManager.Task::(_debug) Task 1bb32cdb-ca93-4f62-be78-dbac6519f049: moving from state init -> state preparing
>Thread-15912627::DEBUG::2013-11-20 02:39:54,563::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 59c70706-854b-4ae3-91cc-4a2cfb6f46d4
>Thread-15912627::DEBUG::2013-11-20 02:39:54,564::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}
>Thread-15912627::DEBUG::2013-11-20 02:39:54,564::task::495::TaskManager.Task::(_debug) Task 1bb32cdb-ca93-4f62-be78-dbac6519f049: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912627::DEBUG::2013-11-20 02:39:54,565::task::495::TaskManager.Task::(_debug) Task 1bb32cdb-ca93-4f62-be78-dbac6519f049: moving from state preparing -> state finished
>Thread-15912627::DEBUG::2013-11-20 02:39:54,565::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912627::DEBUG::2013-11-20 02:39:54,566::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912627::DEBUG::2013-11-20 02:39:54,566::task::495::TaskManager.Task::(_debug) Task 1bb32cdb-ca93-4f62-be78-dbac6519f049: ref 0 aborting False
>Thread-15912627::INFO::2013-11-20 02:39:54,567::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-1024767::ERROR::2013-11-20 02:39:55,105::sp::103::Storage.StatsThread::(run) Unexpected error
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 98, in run
>  File "/usr/share/vdsm/storage/sp.py", line 1314, in _repostats
>  File "/usr/share/vdsm/storage/sd.py", line 437, in validateMaster
>  File "/usr/share/vdsm/storage/sd.py", line 660, in isMaster
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'ROLE'
>Thread-15912628::DEBUG::2013-11-20 02:39:55,577::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912628::INFO::2013-11-20 02:39:55,578::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=59c70706-854b-4ae3-91cc-4a2cfb6f46d4)
>Thread-15912628::DEBUG::2013-11-20 02:39:55,579::task::495::TaskManager.Task::(_debug) Task 995ecd5f-eeb9-401a-b7a0-0bfc25fc3033: moving from state init -> state preparing
>Thread-15912628::DEBUG::2013-11-20 02:39:55,579::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 59c70706-854b-4ae3-91cc-4a2cfb6f46d4
>Thread-15912628::DEBUG::2013-11-20 02:39:55,580::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}
>Thread-15912628::DEBUG::2013-11-20 02:39:55,580::task::495::TaskManager.Task::(_debug) Task 995ecd5f-eeb9-401a-b7a0-0bfc25fc3033: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912628::DEBUG::2013-11-20 02:39:55,581::task::495::TaskManager.Task::(_debug) Task 995ecd5f-eeb9-401a-b7a0-0bfc25fc3033: moving from state preparing -> state finished
>Thread-15912628::DEBUG::2013-11-20 02:39:55,581::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912628::DEBUG::2013-11-20 02:39:55,582::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912628::DEBUG::2013-11-20 02:39:55,582::task::495::TaskManager.Task::(_debug) Task 995ecd5f-eeb9-401a-b7a0-0bfc25fc3033: ref 0 aborting False
>Thread-15912628::INFO::2013-11-20 02:39:55,583::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912630::DEBUG::2013-11-20 02:39:56,593::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912630::INFO::2013-11-20 02:39:56,594::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=59c70706-854b-4ae3-91cc-4a2cfb6f46d4)
>Thread-15912630::DEBUG::2013-11-20 02:39:56,594::task::495::TaskManager.Task::(_debug) Task dcfc9085-d39e-4af9-9db9-eab11c342223: moving from state init -> state preparing
>Thread-15912630::DEBUG::2013-11-20 02:39:56,595::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 59c70706-854b-4ae3-91cc-4a2cfb6f46d4
>Thread-15912630::DEBUG::2013-11-20 02:39:56,595::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}
>Thread-15912630::DEBUG::2013-11-20 02:39:56,596::task::495::TaskManager.Task::(_debug) Task dcfc9085-d39e-4af9-9db9-eab11c342223: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912630::DEBUG::2013-11-20 02:39:56,597::task::495::TaskManager.Task::(_debug) Task dcfc9085-d39e-4af9-9db9-eab11c342223: moving from state preparing -> state finished
>Thread-15912630::DEBUG::2013-11-20 02:39:56,597::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912630::DEBUG::2013-11-20 02:39:56,598::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912630::DEBUG::2013-11-20 02:39:56,598::task::495::TaskManager.Task::(_debug) Task dcfc9085-d39e-4af9-9db9-eab11c342223: ref 0 aborting False
>Thread-15912630::INFO::2013-11-20 02:39:56,599::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912392::INFO::2013-11-20 02:39:57,201::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 02:39:57,202::task::495::TaskManager.Task::(_debug) Task f384f876-1b1f-4256-9f64-d8e923528cb7: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 02:39:57,204::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`22653d9c-c200-4331-850a-3e998a2d5a62`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 02:39:57,204::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 02:39:57,205::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 02:39:57,206::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`22653d9c-c200-4331-850a-3e998a2d5a62`::Granted request
>Thread-15912392::DEBUG::2013-11-20 02:39:57,207::task::495::TaskManager.Task::(_debug) Task f384f876-1b1f-4256-9f64-d8e923528cb7: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 02:39:57,207::task::495::TaskManager.Task::(_debug) Task f384f876-1b1f-4256-9f64-d8e923528cb7: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 02:39:57,208::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15912392::DEBUG::2013-11-20 02:39:57,209::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15912631::DEBUG::2013-11-20 02:39:57,608::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912631::INFO::2013-11-20 02:39:57,609::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=59c70706-854b-4ae3-91cc-4a2cfb6f46d4)
>Thread-15912631::DEBUG::2013-11-20 02:39:57,610::task::495::TaskManager.Task::(_debug) Task 3f9ac69d-e188-4618-926a-5022c9955b40: moving from state init -> state preparing
>Thread-15912631::DEBUG::2013-11-20 02:39:57,610::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 59c70706-854b-4ae3-91cc-4a2cfb6f46d4
>Thread-15912631::DEBUG::2013-11-20 02:39:57,610::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}
>Thread-15912631::DEBUG::2013-11-20 02:39:57,611::task::495::TaskManager.Task::(_debug) Task 3f9ac69d-e188-4618-926a-5022c9955b40: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912631::DEBUG::2013-11-20 02:39:57,611::task::495::TaskManager.Task::(_debug) Task 3f9ac69d-e188-4618-926a-5022c9955b40: moving from state preparing -> state finished
>Thread-15912631::DEBUG::2013-11-20 02:39:57,612::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912631::DEBUG::2013-11-20 02:39:57,612::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912631::DEBUG::2013-11-20 02:39:57,612::task::495::TaskManager.Task::(_debug) Task 3f9ac69d-e188-4618-926a-5022c9955b40: ref 0 aborting False
>Thread-15912631::INFO::2013-11-20 02:39:57,613::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:57,869::safelease::61::Storage.Misc.excCmd::(acquire) SUCCESS: <err> = ''; <rc> = 0
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:57,872::safelease::65::ClusterLock::(acquire) Clustered lock acquired successfully
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:57,874::spm::468::Storage.SPM::(start) spm lock acquired successfully
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:57,876::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm reload operation' is holding the operation mutex, waiting...
>Thread-15912392::DEBUG::2013-11-20 02:39:57,883::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912392::DEBUG::2013-11-20 02:39:58,129::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15912392::DEBUG::2013-11-20 02:39:58,130::task::495::TaskManager.Task::(_debug) Task f384f876-1b1f-4256-9f64-d8e923528cb7: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:58,131::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15912392::DEBUG::2013-11-20 02:39:58,131::task::495::TaskManager.Task::(_debug) Task f384f876-1b1f-4256-9f64-d8e923528cb7: moving from state preparing -> state finished
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:58,133::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15912392::DEBUG::2013-11-20 02:39:58,133::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:58,134::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15912392::DEBUG::2013-11-20 02:39:58,135::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:58,136::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15912392::DEBUG::2013-11-20 02:39:58,137::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:58,138::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-15912392::DEBUG::2013-11-20 02:39:58,139::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:58,141::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15912392::DEBUG::2013-11-20 02:39:58,141::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 02:39:58,142::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 02:39:58,379::task::495::TaskManager.Task::(_debug) Task f384f876-1b1f-4256-9f64-d8e923528cb7: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 02:39:58,381::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912632::DEBUG::2013-11-20 02:39:58,627::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912632::INFO::2013-11-20 02:39:58,629::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=59c70706-854b-4ae3-91cc-4a2cfb6f46d4)
>Thread-15912632::DEBUG::2013-11-20 02:39:58,629::task::495::TaskManager.Task::(_debug) Task 9ba0234a-e9a5-4bfb-ba9e-1ae9b92fde4b: moving from state init -> state preparing
>Thread-15912632::DEBUG::2013-11-20 02:39:58,630::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 59c70706-854b-4ae3-91cc-4a2cfb6f46d4
>Thread-15912632::DEBUG::2013-11-20 02:39:58,631::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}
>Thread-15912632::DEBUG::2013-11-20 02:39:58,631::task::495::TaskManager.Task::(_debug) Task 9ba0234a-e9a5-4bfb-ba9e-1ae9b92fde4b: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912632::DEBUG::2013-11-20 02:39:58,632::task::495::TaskManager.Task::(_debug) Task 9ba0234a-e9a5-4bfb-ba9e-1ae9b92fde4b: moving from state preparing -> state finished
>Thread-15912632::DEBUG::2013-11-20 02:39:58,632::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912632::DEBUG::2013-11-20 02:39:58,633::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912632::DEBUG::2013-11-20 02:39:58,633::task::495::TaskManager.Task::(_debug) Task 9ba0234a-e9a5-4bfb-ba9e-1ae9b92fde4b: ref 0 aborting False
>Thread-15912632::INFO::2013-11-20 02:39:58,634::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:58,821::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:58,824::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:58,825::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_LVER=286', u'_SHA_CKSUM=d45d960da2c767fa0cdea03c6c2e2fb94c077f58', u'POOL_SPM_ID=-1']
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:58,826::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:58,827::persistentDict::146::Storage.PersistentDict::(transaction) Flushing changes
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:58,827::persistentDict::265::Storage.PersistentDict::(flush) about to write lines (VGTagMetadataRW)=[u'CLASS=Data', u'DESCRIPTION=OS', u'IOOPTIMEOUTSEC=10', u'LEASERETRIES=3', u'LEASETIMESEC=60', u'LOCKPOLICY=', u'LOCKRENEWALINTERVALSEC=5', u'LOGBLKSIZE=512', u'MASTER_VERSION=1', u'PHYBLKSIZE=512', u'POOL_DESCRIPTION=Default', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'POOL_SPM_ID=2', u'POOL_SPM_LVER=287', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'ROLE=Master', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'TYPE=FCP', u'VERSION=2', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', '_SHA_CKSUM=716ea7492d9134a4b13de27ac61a336ad474727c']
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:58,828::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:58,829::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:58,830::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:58,830::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:58,831::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:58,832::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15912633::INFO::2013-11-20 02:39:59,092::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912633::DEBUG::2013-11-20 02:39:59,094::task::495::TaskManager.Task::(_debug) Task 22dda694-6fc7-42eb-9b6b-8c417ea6e60f: moving from state init -> state preparing
>Thread-15912633::DEBUG::2013-11-20 02:39:59,097::task::495::TaskManager.Task::(_debug) Task 22dda694-6fc7-42eb-9b6b-8c417ea6e60f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000309944152832', 'lastCheck': 1384915193.6592431, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0448310375214', 'lastCheck': 1384915194.825798, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0895879268646', 'lastCheck': 1384915195.1958771, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000488996505737', 'lastCheck': 1384915191.8335409, 'valid': True, 'code': 0}}
>Thread-15912633::DEBUG::2013-11-20 02:39:59,106::task::495::TaskManager.Task::(_debug) Task 22dda694-6fc7-42eb-9b6b-8c417ea6e60f: moving from state preparing -> state finished
>Thread-15912633::DEBUG::2013-11-20 02:39:59,107::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912633::DEBUG::2013-11-20 02:39:59,108::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912633::DEBUG::2013-11-20 02:39:59,109::task::495::TaskManager.Task::(_debug) Task 22dda694-6fc7-42eb-9b6b-8c417ea6e60f: ref 0 aborting False
>Thread-15912633::INFO::2013-11-20 02:39:59,109::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000309944152832', 'lastCheck': 1384915193.6592431, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0448310375214', 'lastCheck': 1384915194.825798, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0895879268646', 'lastCheck': 1384915195.1958771, 'valid': False, 'code': 200}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000488996505737', 'lastCheck': 1384915191.8335409, 'valid': True, 'code': 0}}
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:59,436::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:59,439::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:59,440::blockSD::145::storage.Metadata.VGTagMetadataRW::(writelines) Updating metadata adding=MDT_POOL_SPM_LVER=287, MDT__SHA_CKSUM=716ea7492d9134a4b13de27ac61a336ad474727c, MDT_POOL_SPM_ID=2 removing=MDT_POOL_SPM_ID=-1, MDT_POOL_SPM_LVER=286, MDT__SHA_CKSUM=d45d960da2c767fa0cdea03c6c2e2fb94c077f58
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:39:59,441::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --deltag MDT__SHA_CKSUM=d45d960da2c767fa0cdea03c6c2e2fb94c077f58 --deltag MDT_POOL_SPM_ID=-1 --deltag MDT_POOL_SPM_LVER=286 --addtag MDT__SHA_CKSUM=716ea7492d9134a4b13de27ac61a336ad474727c --addtag MDT_POOL_SPM_ID=2 --addtag MDT_POOL_SPM_LVER=287 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15912635::DEBUG::2013-11-20 02:39:59,663::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912635::INFO::2013-11-20 02:39:59,666::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=59c70706-854b-4ae3-91cc-4a2cfb6f46d4)
>Thread-15912635::DEBUG::2013-11-20 02:39:59,666::task::495::TaskManager.Task::(_debug) Task a5b1348d-20f4-4822-98ff-027d04c83619: moving from state init -> state preparing
>Thread-15912635::DEBUG::2013-11-20 02:39:59,667::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 59c70706-854b-4ae3-91cc-4a2cfb6f46d4
>Thread-15912635::DEBUG::2013-11-20 02:39:59,667::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}
>Thread-15912635::DEBUG::2013-11-20 02:39:59,668::task::495::TaskManager.Task::(_debug) Task a5b1348d-20f4-4822-98ff-027d04c83619: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912635::DEBUG::2013-11-20 02:39:59,669::task::495::TaskManager.Task::(_debug) Task a5b1348d-20f4-4822-98ff-027d04c83619: moving from state preparing -> state finished
>Thread-15912635::DEBUG::2013-11-20 02:39:59,669::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912635::DEBUG::2013-11-20 02:39:59,670::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912635::DEBUG::2013-11-20 02:39:59,670::task::495::TaskManager.Task::(_debug) Task a5b1348d-20f4-4822-98ff-027d04c83619: ref 0 aborting False
>Thread-15912635::INFO::2013-11-20 02:39:59,671::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:00,166::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:00,168::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:00,169::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:00,169::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:00,170::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f6d18b26-248b-4d01-a2c1-b87667e945f7`::Request was made in '/usr/share/vdsm/storage/spm.py' line '330' at '_upgradePool'
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:00,171::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'exclusive'
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:00,172::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'exclusive' (1 active user)
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:00,172::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f6d18b26-248b-4d01-a2c1-b87667e945f7`::Granted request
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::INFO::2013-11-20 02:40:00,173::spm::336::Storage.SPM::(_upgradePool) Trying to upgrade master domain `6b892067-ac7e-4039-a7be-5b7387525b16`
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:00,174::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fdf6264a-962e-4fa0-a9c7-9bef6c23ba02`::Request was made in '/usr/share/vdsm/storage/spm.py' line '337' at '_upgradePool'
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:00,175::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'exclusive'
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:00,175::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'exclusive' (1 active user)
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:00,176::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fdf6264a-962e-4fa0-a9c7-9bef6c23ba02`::Granted request
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:00,177::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:00,177::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:00,178::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:00,179::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:00,179::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:00,180::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15912636::DEBUG::2013-11-20 02:40:00,682::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912636::INFO::2013-11-20 02:40:00,683::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=59c70706-854b-4ae3-91cc-4a2cfb6f46d4)
>Thread-15912636::DEBUG::2013-11-20 02:40:00,684::task::495::TaskManager.Task::(_debug) Task 71c47a34-3fc8-415d-8f00-332a2ffd587f: moving from state init -> state preparing
>Thread-15912636::DEBUG::2013-11-20 02:40:00,684::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 59c70706-854b-4ae3-91cc-4a2cfb6f46d4
>Thread-15912636::DEBUG::2013-11-20 02:40:00,685::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}
>Thread-15912636::DEBUG::2013-11-20 02:40:00,686::task::495::TaskManager.Task::(_debug) Task 71c47a34-3fc8-415d-8f00-332a2ffd587f: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912636::DEBUG::2013-11-20 02:40:00,686::task::495::TaskManager.Task::(_debug) Task 71c47a34-3fc8-415d-8f00-332a2ffd587f: moving from state preparing -> state finished
>Thread-15912636::DEBUG::2013-11-20 02:40:00,687::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912636::DEBUG::2013-11-20 02:40:00,687::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912636::DEBUG::2013-11-20 02:40:00,688::task::495::TaskManager.Task::(_debug) Task 71c47a34-3fc8-415d-8f00-332a2ffd587f: ref 0 aborting False
>Thread-15912636::INFO::2013-11-20 02:40:00,688::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912637::DEBUG::2013-11-20 02:40:00,811::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912637::INFO::2013-11-20 02:40:00,811::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: validateStorageServerConnection, args: (domType=1, spUUID=00000000-0000-0000-0000-000000000000, conList=[{'connection': 'kairo.brd-klee.dk:/export', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'ec9380b7-cfc1-41d8-9237-d5c7c505ef36', 'port': ''}])
>Thread-15912637::DEBUG::2013-11-20 02:40:00,812::task::495::TaskManager.Task::(_debug) Task c068ce1b-3a5e-4369-ab64-0cd423cbc79f: moving from state init -> state preparing
>Thread-15912637::INFO::2013-11-20 02:40:00,812::storage_connection::127::Storage.ServerConnection::(validate) Request to validate NFS storage server
>Thread-15912637::INFO::2013-11-20 02:40:00,812::storage_connection::53::Storage.ServerConnection::(__validateConnectionParams) conList=[{'connection': 'kairo.brd-klee.dk:/export', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'ec9380b7-cfc1-41d8-9237-d5c7c505ef36', 'port': ''}]
>Thread-15912637::DEBUG::2013-11-20 02:40:00,815::fileUtils::83::Storage.Misc.excCmd::(mount) '/usr/bin/sudo -n /bin/mount -o soft,timeo=600,retrans=6,nosharecache,vers=3 -t nfs kairo.brd-klee.dk:/export /tmp/tmpgimtBa' (cwd None)
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:00,841::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:00,844::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:00,845::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'_SHA_CKSUM=716ea7492d9134a4b13de27ac61a336ad474727c', u'POOL_SPM_ID=2', u'POOL_SPM_LVER=287']
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:00,846::blockSD::625::Storage.StorageDomain::(upgrade) Trying to upgrade domain `6b892067-ac7e-4039-a7be-5b7387525b16` from version 2 to version 2
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:00,847::blockSD::631::Storage.StorageDomain::(upgrade) No need to upgrade domain `6b892067-ac7e-4039-a7be-5b7387525b16`, leaving unchanged
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:00,847::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:00,848::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:00,849::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:00,849::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:00,850::spm::340::Storage.SPM::(_upgradePool) Marking all domains for upgrade
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:00,850::spm::347::Storage.SPM::(_upgradePool) Registering with state change event
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:00,851::spm::349::Storage.SPM::(_upgradePool) Running initial domain upgrade threads
>Thread-15912638::DEBUG::2013-11-20 02:40:00,852::spm::365::Storage.SPM::(_upgradePoolDomain) Preparing to upgrade domain 45cdff8b-89c0-4926-a8c7-80354064bfa6
>Thread-15912638::DEBUG::2013-11-20 02:40:00,856::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6`ReqID=`c72e1389-8910-4ceb-9eaa-b8511b783f6d`::Request was made in '/usr/share/vdsm/storage/spm.py' line '376' at '_upgradePoolDomain'
>Thread-15912639::DEBUG::2013-11-20 02:40:00,857::spm::365::Storage.SPM::(_upgradePoolDomain) Preparing to upgrade domain 772472cb-2429-466d-96cd-00b3301ef78a
>Thread-15912638::DEBUG::2013-11-20 02:40:00,858::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6' for lock type 'exclusive'
>Thread-15912640::DEBUG::2013-11-20 02:40:00,858::spm::365::Storage.SPM::(_upgradePoolDomain) Preparing to upgrade domain 5d7b5178-3790-472c-a13b-b481eb27a61e
>Thread-15912639::DEBUG::2013-11-20 02:40:00,861::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`daa5fb58-109f-4f40-b8a9-d4afee706da5`::Request was made in '/usr/share/vdsm/storage/spm.py' line '376' at '_upgradePoolDomain'
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:00,861::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912638::DEBUG::2013-11-20 02:40:00,862::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6' is free. Now locking as 'exclusive' (1 active user)
>Thread-15912640::ERROR::2013-11-20 02:40:00,863::spm::373::Storage.SPM::(_upgradePoolDomain) Error while preparing domain `5d7b5178-3790-472c-a13b-b481eb27a61e` upgrade
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/spm.py", line 369, in _upgradePoolDomain
>IndexError: list index out of range
>Thread-15912639::DEBUG::2013-11-20 02:40:00,864::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'exclusive'
>Thread-15912638::DEBUG::2013-11-20 02:40:00,866::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6`ReqID=`c72e1389-8910-4ceb-9eaa-b8511b783f6d`::Granted request
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:00,867::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912638::DEBUG::2013-11-20 02:40:00,868::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6`ReqID=`c9d5d78e-7c56-4c4b-a669-67afb34711df`::Request was made in '/usr/share/vdsm/storage/spm.py' line '377' at '_upgradePoolDomain'
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:00,869::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912638::DEBUG::2013-11-20 02:40:00,875::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6' for lock type 'exclusive'
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:00,876::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912639::DEBUG::2013-11-20 02:40:00,877::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'exclusive' (1 active user)
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:00,878::blockSD::791::Storage.Misc.excCmd::(mountMaster) '/usr/bin/sudo -n /sbin/e2fsck -p /dev/6b892067-ac7e-4039-a7be-5b7387525b16/master' (cwd None)
>Thread-15912639::DEBUG::2013-11-20 02:40:00,879::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`daa5fb58-109f-4f40-b8a9-d4afee706da5`::Granted request
>Thread-15912638::DEBUG::2013-11-20 02:40:00,880::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6' is free. Now locking as 'exclusive' (1 active user)
>Thread-15912638::DEBUG::2013-11-20 02:40:01,124::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6`ReqID=`c9d5d78e-7c56-4c4b-a669-67afb34711df`::Granted request
>Thread-15912639::DEBUG::2013-11-20 02:40:01,127::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`4761fdaa-e6b3-4f32-88e9-d00933ea94e4`::Request was made in '/usr/share/vdsm/storage/spm.py' line '377' at '_upgradePoolDomain'
>Thread-15912639::DEBUG::2013-11-20 02:40:01,129::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'exclusive'
>Thread-15912638::DEBUG::2013-11-20 02:40:01,131::spm::385::Storage.SPM::(_upgradePoolDomain) Domain `45cdff8b-89c0-4926-a8c7-80354064bfa6` is not a data domain it is an 2 domain, not upgrading
>Thread-15912639::DEBUG::2013-11-20 02:40:01,132::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'exclusive' (1 active user)
>Thread-15912638::DEBUG::2013-11-20 02:40:01,133::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6'
>Thread-15912639::DEBUG::2013-11-20 02:40:01,133::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`4761fdaa-e6b3-4f32-88e9-d00933ea94e4`::Granted request
>Thread-15912638::DEBUG::2013-11-20 02:40:01,135::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6' (0 active users)
>Thread-15912639::DEBUG::2013-11-20 02:40:01,137::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15912638::DEBUG::2013-11-20 02:40:01,137::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6' is free, finding out if anyone is waiting for it.
>Thread-15912639::DEBUG::2013-11-20 02:40:01,138::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15912638::DEBUG::2013-11-20 02:40:01,139::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.45cdff8b-89c0-4926-a8c7-80354064bfa6', Clearing records.
>Thread-15912639::DEBUG::2013-11-20 02:40:01,140::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15912639::DEBUG::2013-11-20 02:40:01,142::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15912638::DEBUG::2013-11-20 02:40:01,143::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6'
>Thread-15912639::DEBUG::2013-11-20 02:40:01,143::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-15912638::DEBUG::2013-11-20 02:40:01,144::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6' (0 active users)
>Thread-15912639::DEBUG::2013-11-20 02:40:01,146::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-15912638::DEBUG::2013-11-20 02:40:01,148::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6' is free, finding out if anyone is waiting for it.
>Thread-15912638::DEBUG::2013-11-20 02:40:01,605::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_45cdff8b-89c0-4926-a8c7-80354064bfa6', Clearing records.
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:01,633::blockSD::791::Storage.Misc.excCmd::(mountMaster) SUCCESS: <err> = ''; <rc> = 0
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:01,634::blockSD::816::Storage.Misc.excCmd::(mountMaster) '/usr/bin/sudo -n /sbin/tune2fs -j /dev/6b892067-ac7e-4039-a7be-5b7387525b16/master' (cwd None)
>Thread-15912637::DEBUG::2013-11-20 02:40:01,650::fileUtils::83::Storage.Misc.excCmd::(mount) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912641::DEBUG::2013-11-20 02:40:01,975::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912641::INFO::2013-11-20 02:40:01,984::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=59c70706-854b-4ae3-91cc-4a2cfb6f46d4)
>Thread-15912641::DEBUG::2013-11-20 02:40:01,985::task::495::TaskManager.Task::(_debug) Task f01de43b-cccb-48e1-8bb4-15802fa89a97: moving from state init -> state preparing
>Thread-15912637::DEBUG::2013-11-20 02:40:01,985::fileUtils::112::Storage.Misc.excCmd::(umount) '/usr/bin/sudo -n /bin/umount -t nfs -f /tmp/tmpgimtBa' (cwd None)
>Thread-15912641::DEBUG::2013-11-20 02:40:01,986::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 59c70706-854b-4ae3-91cc-4a2cfb6f46d4
>Thread-15912641::DEBUG::2013-11-20 02:40:01,987::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}
>Thread-15912641::DEBUG::2013-11-20 02:40:01,992::task::495::TaskManager.Task::(_debug) Task f01de43b-cccb-48e1-8bb4-15802fa89a97: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912641::DEBUG::2013-11-20 02:40:01,993::task::495::TaskManager.Task::(_debug) Task f01de43b-cccb-48e1-8bb4-15802fa89a97: moving from state preparing -> state finished
>Thread-15912641::DEBUG::2013-11-20 02:40:01,994::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912641::DEBUG::2013-11-20 02:40:01,994::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912641::DEBUG::2013-11-20 02:40:01,995::task::495::TaskManager.Task::(_debug) Task f01de43b-cccb-48e1-8bb4-15802fa89a97: ref 0 aborting False
>Thread-15912641::INFO::2013-11-20 02:40:01,995::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-42840::DEBUG::2013-11-20 02:40:02,122::lvm::439::OperationMutex::(_reloadvgs) Got the operational mutex
>Thread-42840::DEBUG::2013-11-20 02:40:02,124::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-15912639::DEBUG::2013-11-20 02:40:02,510::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:02,513::blockSD::816::Storage.Misc.excCmd::(mountMaster) FAILED: <err> = 'The filesystem already has a journal.\n'; <rc> = 1
>Thread-15912639::DEBUG::2013-11-20 02:40:02,516::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'SDUUID=772472cb-2429-466d-96cd-00b3301ef78a', u'VERSION=2', u'LOCKPOLICY=', u'VGUUID=XTb294-NYv6-dVpO-bB1n-4f9P-d0GL-dhd2DI', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'LEASETIMESEC=5', u'LOCKRENEWALINTERVALSEC=5', u'ROLE=Regular', u'PHYBLKSIZE=512', u'DESCRIPTION=Database', u'IOOPTIMEOUTSEC=1', u'CLASS=Data', u'TYPE=FCP', u'PV0=pv:3600c0ff0001520c7414f605001000000,uuid:NujM33-vNiD-sBHY-mrPo-EQ2f-1qsz-HXfn0q,pestart:0,pecount:8929,mapoffset:0', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'MASTER_VERSION=0', u'_SHA_CKSUM=ef8840d9fdef24516a56fd970bed1960d2b1ae92']
>Thread-15912639::DEBUG::2013-11-20 02:40:02,519::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:02,522::fileUtils::83::Storage.Misc.excCmd::(mount) '/usr/bin/sudo -n /bin/mount /dev/6b892067-ac7e-4039-a7be-5b7387525b16/master /rhev/data-center/mnt/blockSD/6b892067-ac7e-4039-a7be-5b7387525b16/master' (cwd None)
>Thread-15912639::DEBUG::2013-11-20 02:40:02,523::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15912639::DEBUG::2013-11-20 02:40:02,524::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15912637::DEBUG::2013-11-20 02:40:02,916::fileUtils::112::Storage.Misc.excCmd::(umount) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912639::DEBUG::2013-11-20 02:40:03,007::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15912637::DEBUG::2013-11-20 02:40:03,008::task::495::TaskManager.Task::(_debug) Task c068ce1b-3a5e-4369-ab64-0cd423cbc79f: finished: {'statuslist': [{'status': 0, 'id': 'ec9380b7-cfc1-41d8-9237-d5c7c505ef36'}]}
>Thread-15912637::DEBUG::2013-11-20 02:40:03,014::task::495::TaskManager.Task::(_debug) Task c068ce1b-3a5e-4369-ab64-0cd423cbc79f: moving from state preparing -> state finished
>Thread-15912639::DEBUG::2013-11-20 02:40:03,016::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15912637::DEBUG::2013-11-20 02:40:03,017::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912637::DEBUG::2013-11-20 02:40:03,018::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912639::DEBUG::2013-11-20 02:40:03,019::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15912637::DEBUG::2013-11-20 02:40:03,020::task::495::TaskManager.Task::(_debug) Task c068ce1b-3a5e-4369-ab64-0cd423cbc79f: ref 0 aborting False
>Thread-15912639::DEBUG::2013-11-20 02:40:03,020::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15912637::INFO::2013-11-20 02:40:03,021::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: validateStorageServerConnection, Return response: {'status': {'message': 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': 'ec9380b7-cfc1-41d8-9237-d5c7c505ef36'}]}
>Thread-15912639::DEBUG::2013-11-20 02:40:03,022::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15912643::DEBUG::2013-11-20 02:40:03,182::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912643::INFO::2013-11-20 02:40:03,184::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=59c70706-854b-4ae3-91cc-4a2cfb6f46d4)
>Thread-15912643::DEBUG::2013-11-20 02:40:03,184::task::495::TaskManager.Task::(_debug) Task cc03b74d-37b5-4411-9ca5-70ef5db383d2: moving from state init -> state preparing
>Thread-15912643::DEBUG::2013-11-20 02:40:03,185::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 59c70706-854b-4ae3-91cc-4a2cfb6f46d4
>Thread-15912643::DEBUG::2013-11-20 02:40:03,185::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}
>Thread-15912643::DEBUG::2013-11-20 02:40:03,186::task::495::TaskManager.Task::(_debug) Task cc03b74d-37b5-4411-9ca5-70ef5db383d2: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912643::DEBUG::2013-11-20 02:40:03,186::task::495::TaskManager.Task::(_debug) Task cc03b74d-37b5-4411-9ca5-70ef5db383d2: moving from state preparing -> state finished
>Thread-15912643::DEBUG::2013-11-20 02:40:03,187::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912643::DEBUG::2013-11-20 02:40:03,187::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912643::DEBUG::2013-11-20 02:40:03,188::task::495::TaskManager.Task::(_debug) Task cc03b74d-37b5-4411-9ca5-70ef5db383d2: ref 0 aborting False
>Thread-15912643::INFO::2013-11-20 02:40:03,189::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912644::DEBUG::2013-11-20 02:40:03,194::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912644::INFO::2013-11-20 02:40:03,195::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: connectStorageServer, args: (domType=1, spUUID=00000000-0000-0000-0000-000000000000, conList=[{'connection': 'kairo.brd-klee.dk:/export', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'ec9380b7-cfc1-41d8-9237-d5c7c505ef36', 'port': ''}])
>Thread-15912644::DEBUG::2013-11-20 02:40:03,196::task::495::TaskManager.Task::(_debug) Task 40fc9d51-2146-46c1-b5ac-38c6a7012dd5: moving from state init -> state preparing
>Thread-15912644::INFO::2013-11-20 02:40:03,197::storage_connection::95::Storage.ServerConnection::(connect) Request to connect NFS storage server
>Thread-15912644::INFO::2013-11-20 02:40:03,197::storage_connection::53::Storage.ServerConnection::(__validateConnectionParams) conList=[{'connection': 'kairo.brd-klee.dk:/export', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'ec9380b7-cfc1-41d8-9237-d5c7c505ef36', 'port': ''}]
>Thread-15912644::DEBUG::2013-11-20 02:40:03,210::lvm::547::OperationMutex::(_invalidateAllPvs) Operation 'lvm reload operation' is holding the operation mutex, waiting...
>Thread-42840::DEBUG::2013-11-20 02:40:03,237::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-42840::DEBUG::2013-11-20 02:40:03,240::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-15912644::DEBUG::2013-11-20 02:40:03,242::lvm::547::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15912644::DEBUG::2013-11-20 02:40:03,243::lvm::549::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15912644::DEBUG::2013-11-20 02:40:03,244::lvm::559::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15912644::DEBUG::2013-11-20 02:40:03,244::lvm::561::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15912644::DEBUG::2013-11-20 02:40:03,245::lvm::580::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15912644::DEBUG::2013-11-20 02:40:03,246::lvm::582::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15912644::DEBUG::2013-11-20 02:40:03,246::task::495::TaskManager.Task::(_debug) Task 40fc9d51-2146-46c1-b5ac-38c6a7012dd5: finished: {'statuslist': [{'status': 0, 'id': 'ec9380b7-cfc1-41d8-9237-d5c7c505ef36'}]}
>Thread-15912644::DEBUG::2013-11-20 02:40:03,247::task::495::TaskManager.Task::(_debug) Task 40fc9d51-2146-46c1-b5ac-38c6a7012dd5: moving from state preparing -> state finished
>Thread-15912644::DEBUG::2013-11-20 02:40:03,247::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912644::DEBUG::2013-11-20 02:40:03,248::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912644::DEBUG::2013-11-20 02:40:03,248::task::495::TaskManager.Task::(_debug) Task 40fc9d51-2146-46c1-b5ac-38c6a7012dd5: ref 0 aborting False
>Thread-15912644::INFO::2013-11-20 02:40:03,249::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: connectStorageServer, Return response: {'status': {'message': 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': 'ec9380b7-cfc1-41d8-9237-d5c7c505ef36'}]}
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:03,268::fileUtils::83::Storage.Misc.excCmd::(mount) SUCCESS: <err> = ''; <rc> = 0
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:03,269::blockSD::833::Storage.Misc.excCmd::(mountMaster) '/usr/bin/sudo -n /bin/chown vdsm:kvm /rhev/data-center/mnt/blockSD/6b892067-ac7e-4039-a7be-5b7387525b16/master' (cwd None)
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:03,688::blockSD::833::Storage.Misc.excCmd::(mountMaster) SUCCESS: <err> = ''; <rc> = 0
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::WARNING::2013-11-20 02:40:03,699::spm::490::Storage.SPM::(start) Backup domain validation failed
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/spm.py", line 488, in start
>  File "/usr/share/vdsm/storage/sp.py", line 1208, in checkBackupDomain
>  File "/usr/share/vdsm/storage/sd.py", line 666, in isBackup
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:03,700::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:03,701::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:03,702::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:03,702::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:03,703::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:03,703::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:03,704::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:03,704::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:03,705::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:03,705::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:03,706::persistentDict::133::Storage.PersistentDict::(transaction) Reusing active transaction
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:03,706::persistentDict::146::Storage.PersistentDict::(transaction) Flushing changes
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:03,707::persistentDict::265::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['MASTER_VERSION=0', 'ROLE=Regular', '_SHA_CKSUM=1deb73127f89496e2cb17e51bbfae1df6c1879d1']
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:03,940::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:03,943::threadPool::25::Misc.ThreadPool::(__init__) Enter - numThreads: 5.0, waitTimeout: 3, maxTasks: 500.0
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:03,948::storage_mailbox::486::Storage.MailBox.SpmMailMonitor::(__init__) SPM_MailMonitor - clearing outgoing mail, command is: ['dd', 'of=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/outbox', 'oflag=direct', 'iflag=fullblock', 'conv=notrunc', 'count=1']
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:03,949::storage_mailbox::488::Storage.Misc.excCmd::(__init__) 'dd of=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/outbox oflag=direct iflag=fullblock conv=notrunc count=1 bs=1024000' (cwd None)
>Thread-42838::DEBUG::2013-11-20 02:40:04,154::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-42838::DEBUG::2013-11-20 02:40:04,161::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15912650::DEBUG::2013-11-20 02:40:04,478::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912650::INFO::2013-11-20 02:40:04,485::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=59c70706-854b-4ae3-91cc-4a2cfb6f46d4)
>Thread-15912650::DEBUG::2013-11-20 02:40:04,486::task::495::TaskManager.Task::(_debug) Task c1fc4616-b3bb-472d-aa21-d7386e34d9d6: moving from state init -> state preparing
>Thread-15912650::DEBUG::2013-11-20 02:40:04,488::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 59c70706-854b-4ae3-91cc-4a2cfb6f46d4
>Thread-15912650::DEBUG::2013-11-20 02:40:04,489::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}
>Thread-15912650::DEBUG::2013-11-20 02:40:04,490::task::495::TaskManager.Task::(_debug) Task c1fc4616-b3bb-472d-aa21-d7386e34d9d6: finished: {'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912650::DEBUG::2013-11-20 02:40:04,492::task::495::TaskManager.Task::(_debug) Task c1fc4616-b3bb-472d-aa21-d7386e34d9d6: moving from state preparing -> state finished
>Thread-15912650::DEBUG::2013-11-20 02:40:04,494::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912650::DEBUG::2013-11-20 02:40:04,495::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912650::DEBUG::2013-11-20 02:40:04,496::task::495::TaskManager.Task::(_debug) Task c1fc4616-b3bb-472d-aa21-d7386e34d9d6: ref 0 aborting False
>Thread-15912650::INFO::2013-11-20 02:40:04,498::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': 'Task is initializing', 'taskState': 'running', 'taskResult': '', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:04,538::storage_mailbox::488::Storage.Misc.excCmd::(__init__) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0572943 s, 17.9 MB/s\n'; <rc> = 0
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:04,539::storage_mailbox::493::Storage.MailBox.SpmMailMonitor::(__init__) SPM_MailMonitor created for pool c4e5889a-348d-11e2-83fe-9f72369eab30
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:04,541::spm::523::Storage.SPM::(start) ended.
>Dummy-15912651::DEBUG::2013-11-20 02:40:04,540::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:04,542::task::495::TaskManager.Task::(_debug) Task 59c70706-854b-4ae3-91cc-4a2cfb6f46d4: moving from state running -> state finished
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:04,809::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:04,817::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:04,819::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:04,820::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:04,820::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:04,821::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:04,822::task::915::TaskManager.Task::(_runJobs) Task.run: exit - success: result 
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:04,822::task::495::TaskManager.Task::(_debug) Task 59c70706-854b-4ae3-91cc-4a2cfb6f46d4: ref 0 aborting False
>59c70706-854b-4ae3-91cc-4a2cfb6f46d4::DEBUG::2013-11-20 02:40:04,823::threadPool::47::Misc.ThreadPool::(setRunningTask) Number of running tasks: 0
>Thread-42838::DEBUG::2013-11-20 02:40:04,959::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-42838::DEBUG::2013-11-20 02:40:04,963::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Dummy-15912651::DEBUG::2013-11-20 02:40:05,173::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.131043 s, 7.8 MB/s\n'; <rc> = 0
>Thread-15912652::DEBUG::2013-11-20 02:40:05,517::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912652::INFO::2013-11-20 02:40:05,519::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, args: ( taskID=59c70706-854b-4ae3-91cc-4a2cfb6f46d4)
>Thread-15912652::DEBUG::2013-11-20 02:40:05,520::task::495::TaskManager.Task::(_debug) Task 11533244-3712-4617-a3fb-9a81b8711f90: moving from state init -> state preparing
>Thread-15912652::DEBUG::2013-11-20 02:40:05,520::taskManager::82::TaskManager::(getTaskStatus) Entry. taskID: 59c70706-854b-4ae3-91cc-4a2cfb6f46d4
>Thread-15912652::DEBUG::2013-11-20 02:40:05,521::taskManager::85::TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': '1 jobs completed successfuly', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}
>Thread-15912652::DEBUG::2013-11-20 02:40:05,522::task::495::TaskManager.Task::(_debug) Task 11533244-3712-4617-a3fb-9a81b8711f90: finished: {'taskStatus': {'code': 0, 'message': '1 jobs completed successfuly', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912652::DEBUG::2013-11-20 02:40:05,522::task::495::TaskManager.Task::(_debug) Task 11533244-3712-4617-a3fb-9a81b8711f90: moving from state preparing -> state finished
>Thread-15912652::DEBUG::2013-11-20 02:40:05,523::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912652::DEBUG::2013-11-20 02:40:05,523::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912652::DEBUG::2013-11-20 02:40:05,524::task::495::TaskManager.Task::(_debug) Task 11533244-3712-4617-a3fb-9a81b8711f90: ref 0 aborting False
>Thread-15912652::INFO::2013-11-20 02:40:05,525::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getTaskStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'taskStatus': {'code': 0, 'message': '1 jobs completed successfuly', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '59c70706-854b-4ae3-91cc-4a2cfb6f46d4'}}
>Thread-15912653::DEBUG::2013-11-20 02:40:05,531::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912653::INFO::2013-11-20 02:40:05,532::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912653::DEBUG::2013-11-20 02:40:05,533::task::495::TaskManager.Task::(_debug) Task 5aaf0398-bb64-485c-9209-2713b6f1c57d: moving from state init -> state preparing
>Thread-15912653::DEBUG::2013-11-20 02:40:05,534::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912653::DEBUG::2013-11-20 02:40:05,534::task::495::TaskManager.Task::(_debug) Task 5aaf0398-bb64-485c-9209-2713b6f1c57d: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912653::DEBUG::2013-11-20 02:40:05,535::task::495::TaskManager.Task::(_debug) Task 5aaf0398-bb64-485c-9209-2713b6f1c57d: moving from state preparing -> state finished
>Thread-15912653::DEBUG::2013-11-20 02:40:05,536::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912653::DEBUG::2013-11-20 02:40:05,536::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912653::DEBUG::2013-11-20 02:40:05,537::task::495::TaskManager.Task::(_debug) Task 5aaf0398-bb64-485c-9209-2713b6f1c57d: ref 0 aborting False
>Thread-15912653::INFO::2013-11-20 02:40:05,537::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912654::DEBUG::2013-11-20 02:40:05,677::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912654::INFO::2013-11-20 02:40:05,678::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: clearTask, args: ( taskID=59c70706-854b-4ae3-91cc-4a2cfb6f46d4)
>Thread-15912654::DEBUG::2013-11-20 02:40:05,678::task::495::TaskManager.Task::(_debug) Task 7cce22c7-921e-46c9-b0cd-5db46111da1b: moving from state init -> state preparing
>Thread-15912654::DEBUG::2013-11-20 02:40:05,679::taskManager::146::TaskManager::(clearTask) Entry. taskID: 59c70706-854b-4ae3-91cc-4a2cfb6f46d4
>Thread-15912654::DEBUG::2013-11-20 02:40:05,680::taskManager::151::TaskManager::(clearTask) Return.
>Thread-15912654::DEBUG::2013-11-20 02:40:05,680::task::495::TaskManager.Task::(_debug) Task 7cce22c7-921e-46c9-b0cd-5db46111da1b: finished: None
>Thread-15912654::DEBUG::2013-11-20 02:40:05,681::task::495::TaskManager.Task::(_debug) Task 7cce22c7-921e-46c9-b0cd-5db46111da1b: moving from state preparing -> state finished
>Thread-15912654::DEBUG::2013-11-20 02:40:05,681::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912654::DEBUG::2013-11-20 02:40:05,682::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912654::DEBUG::2013-11-20 02:40:05,682::task::495::TaskManager.Task::(_debug) Task 7cce22c7-921e-46c9-b0cd-5db46111da1b: ref 0 aborting False
>Thread-15912654::INFO::2013-11-20 02:40:05,683::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: clearTask, Return response: {'status': {'message': 'OK', 'code': 0}}
>Thread-15912655::DEBUG::2013-11-20 02:40:06,164::misc::1156::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Emitting event
>Thread-15912655::DEBUG::2013-11-20 02:40:06,165::misc::1166::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Calling registered method `run`
>Thread-15912656::DEBUG::2013-11-20 02:40:06,167::spm::365::Storage.SPM::(_upgradePoolDomain) Preparing to upgrade domain 5d7b5178-3790-472c-a13b-b481eb27a61e
>Thread-15912655::DEBUG::2013-11-20 02:40:06,174::misc::1174::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Event emitted
>Thread-15912656::ERROR::2013-11-20 02:40:06,176::spm::373::Storage.SPM::(_upgradePoolDomain) Error while preparing domain `5d7b5178-3790-472c-a13b-b481eb27a61e` upgrade
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/spm.py", line 369, in _upgradePoolDomain
>IndexError: list index out of range
>Dummy-15912651::DEBUG::2013-11-20 02:40:07,187::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:40:07,719::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0788495 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15912658::DEBUG::2013-11-20 02:40:07,870::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912658::INFO::2013-11-20 02:40:07,872::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912658::DEBUG::2013-11-20 02:40:07,873::task::495::TaskManager.Task::(_debug) Task 59c8e6a2-993f-43a1-aff9-573924f88106: moving from state init -> state preparing
>Thread-15912658::DEBUG::2013-11-20 02:40:07,874::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`62d951f8-11c0-438a-99aa-63dc8d4b9086`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912658::DEBUG::2013-11-20 02:40:07,881::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912658::DEBUG::2013-11-20 02:40:07,882::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912658::DEBUG::2013-11-20 02:40:07,893::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`62d951f8-11c0-438a-99aa-63dc8d4b9086`::Granted request
>Thread-15912658::DEBUG::2013-11-20 02:40:07,894::task::495::TaskManager.Task::(_debug) Task 59c8e6a2-993f-43a1-aff9-573924f88106: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912658::DEBUG::2013-11-20 02:40:07,894::task::495::TaskManager.Task::(_debug) Task 59c8e6a2-993f-43a1-aff9-573924f88106: ref 1 aborting False
>Thread-15912658::WARNING::2013-11-20 02:40:07,899::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912658::DEBUG::2013-11-20 02:40:07,900::task::495::TaskManager.Task::(_debug) Task 59c8e6a2-993f-43a1-aff9-573924f88106: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912658::DEBUG::2013-11-20 02:40:07,903::task::495::TaskManager.Task::(_debug) Task 59c8e6a2-993f-43a1-aff9-573924f88106: moving from state preparing -> state finished
>Thread-15912658::DEBUG::2013-11-20 02:40:07,904::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912658::DEBUG::2013-11-20 02:40:07,904::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912658::DEBUG::2013-11-20 02:40:07,911::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912658::DEBUG::2013-11-20 02:40:07,912::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912658::DEBUG::2013-11-20 02:40:07,912::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912658::DEBUG::2013-11-20 02:40:07,919::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912658::DEBUG::2013-11-20 02:40:07,923::task::495::TaskManager.Task::(_debug) Task 59c8e6a2-993f-43a1-aff9-573924f88106: ref 0 aborting False
>Thread-15912658::INFO::2013-11-20 02:40:07,924::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912659::DEBUG::2013-11-20 02:40:08,412::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912659::INFO::2013-11-20 02:40:08,413::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksInfo, args: ()
>Thread-15912659::DEBUG::2013-11-20 02:40:08,414::task::495::TaskManager.Task::(_debug) Task e5cdcaca-dfa9-4a97-9e11-c5a14019288f: moving from state init -> state preparing
>Thread-15912659::DEBUG::2013-11-20 02:40:08,415::taskManager::168::TaskManager::(getAllTasksInfo) Entry.
>Thread-15912659::DEBUG::2013-11-20 02:40:08,416::taskManager::173::TaskManager::(getAllTasksInfo) Return. Response: {}
>Thread-15912659::DEBUG::2013-11-20 02:40:08,417::task::495::TaskManager.Task::(_debug) Task e5cdcaca-dfa9-4a97-9e11-c5a14019288f: finished: {'allTasksInfo': {}}
>Thread-15912659::DEBUG::2013-11-20 02:40:08,417::task::495::TaskManager.Task::(_debug) Task e5cdcaca-dfa9-4a97-9e11-c5a14019288f: moving from state preparing -> state finished
>Thread-15912659::DEBUG::2013-11-20 02:40:08,418::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912659::DEBUG::2013-11-20 02:40:08,419::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912659::DEBUG::2013-11-20 02:40:08,420::task::495::TaskManager.Task::(_debug) Task e5cdcaca-dfa9-4a97-9e11-c5a14019288f: ref 0 aborting False
>Thread-15912659::INFO::2013-11-20 02:40:08,424::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'allTasksInfo': {}}
>Dummy-15912651::DEBUG::2013-11-20 02:40:09,735::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:40:10,261::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0727364 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:40:12,276::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:40:12,827::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0726634 s, 14.1 MB/s\n'; <rc> = 0
>Thread-42840::DEBUG::2013-11-20 02:40:13,501::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-42840::DEBUG::2013-11-20 02:40:13,503::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 02:40:14,185::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-42840::DEBUG::2013-11-20 02:40:14,188::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Dummy-15912651::DEBUG::2013-11-20 02:40:14,843::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:40:15,372::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0755846 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15912662::INFO::2013-11-20 02:40:16,037::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912662::DEBUG::2013-11-20 02:40:16,038::task::495::TaskManager.Task::(_debug) Task dad08f6c-2a15-4dd8-84dc-12f3ceba85ab: moving from state init -> state preparing
>Thread-15912662::DEBUG::2013-11-20 02:40:16,039::task::495::TaskManager.Task::(_debug) Task dad08f6c-2a15-4dd8-84dc-12f3ceba85ab: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000531911849976', 'lastCheck': 1384915215.063504, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00179600715637', 'lastCheck': 1384915215.1816781, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00251483917236', 'lastCheck': 1384915206.1634541, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000345230102539', 'lastCheck': 1384915214.1902399, 'valid': True, 'code': 0}}
>Thread-15912662::DEBUG::2013-11-20 02:40:16,040::task::495::TaskManager.Task::(_debug) Task dad08f6c-2a15-4dd8-84dc-12f3ceba85ab: moving from state preparing -> state finished
>Thread-15912662::DEBUG::2013-11-20 02:40:16,040::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912662::DEBUG::2013-11-20 02:40:16,041::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912662::DEBUG::2013-11-20 02:40:16,041::task::495::TaskManager.Task::(_debug) Task dad08f6c-2a15-4dd8-84dc-12f3ceba85ab: ref 0 aborting False
>Thread-15912662::INFO::2013-11-20 02:40:16,042::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000531911849976', 'lastCheck': 1384915215.063504, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00179600715637', 'lastCheck': 1384915215.1816781, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00251483917236', 'lastCheck': 1384915206.1634541, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000345230102539', 'lastCheck': 1384915214.1902399, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:40:17,405::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:40:17,962::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.07667 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15912664::DEBUG::2013-11-20 02:40:18,399::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912664::INFO::2013-11-20 02:40:18,400::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912664::DEBUG::2013-11-20 02:40:18,401::task::495::TaskManager.Task::(_debug) Task 41e28ba9-ef74-4011-80d8-cd1536382651: moving from state init -> state preparing
>Thread-15912664::DEBUG::2013-11-20 02:40:18,402::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912664::DEBUG::2013-11-20 02:40:18,402::task::495::TaskManager.Task::(_debug) Task 41e28ba9-ef74-4011-80d8-cd1536382651: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912664::DEBUG::2013-11-20 02:40:18,403::task::495::TaskManager.Task::(_debug) Task 41e28ba9-ef74-4011-80d8-cd1536382651: moving from state preparing -> state finished
>Thread-15912664::DEBUG::2013-11-20 02:40:18,403::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912664::DEBUG::2013-11-20 02:40:18,404::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912664::DEBUG::2013-11-20 02:40:18,405::task::495::TaskManager.Task::(_debug) Task 41e28ba9-ef74-4011-80d8-cd1536382651: ref 0 aborting False
>Thread-15912664::INFO::2013-11-20 02:40:18,411::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912665::DEBUG::2013-11-20 02:40:18,422::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912665::INFO::2013-11-20 02:40:18,426::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912665::DEBUG::2013-11-20 02:40:18,427::task::495::TaskManager.Task::(_debug) Task 86fe80bb-891e-44a0-922c-e09d50d6ab26: moving from state init -> state preparing
>Thread-15912665::DEBUG::2013-11-20 02:40:18,428::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`06dd6e41-2e74-41a8-b64a-ee4331c9ee04`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912665::DEBUG::2013-11-20 02:40:18,434::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912665::DEBUG::2013-11-20 02:40:18,435::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912665::DEBUG::2013-11-20 02:40:18,436::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`06dd6e41-2e74-41a8-b64a-ee4331c9ee04`::Granted request
>Thread-15912665::DEBUG::2013-11-20 02:40:18,437::task::495::TaskManager.Task::(_debug) Task 86fe80bb-891e-44a0-922c-e09d50d6ab26: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912665::DEBUG::2013-11-20 02:40:18,438::task::495::TaskManager.Task::(_debug) Task 86fe80bb-891e-44a0-922c-e09d50d6ab26: ref 1 aborting False
>Thread-15912665::WARNING::2013-11-20 02:40:18,440::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912665::DEBUG::2013-11-20 02:40:18,441::task::495::TaskManager.Task::(_debug) Task 86fe80bb-891e-44a0-922c-e09d50d6ab26: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912665::DEBUG::2013-11-20 02:40:18,446::task::495::TaskManager.Task::(_debug) Task 86fe80bb-891e-44a0-922c-e09d50d6ab26: moving from state preparing -> state finished
>Thread-15912665::DEBUG::2013-11-20 02:40:18,446::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912665::DEBUG::2013-11-20 02:40:18,447::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912665::DEBUG::2013-11-20 02:40:18,447::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912665::DEBUG::2013-11-20 02:40:18,448::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912665::DEBUG::2013-11-20 02:40:18,449::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912665::DEBUG::2013-11-20 02:40:18,449::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912665::DEBUG::2013-11-20 02:40:18,450::task::495::TaskManager.Task::(_debug) Task 86fe80bb-891e-44a0-922c-e09d50d6ab26: ref 0 aborting False
>Thread-15912665::INFO::2013-11-20 02:40:18,450::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:40:19,979::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:40:20,532::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0744498 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:40:22,547::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:40:23,137::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0745944 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:40:25,150::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:40:25,721::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0797565 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:40:27,735::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:40:28,189::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0447813 s, 22.9 MB/s\n'; <rc> = 0
>Thread-15912670::DEBUG::2013-11-20 02:40:29,254::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912670::INFO::2013-11-20 02:40:29,267::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912670::DEBUG::2013-11-20 02:40:29,273::task::495::TaskManager.Task::(_debug) Task 56e36ab3-fd13-48ba-8c16-fa595e25245c: moving from state init -> state preparing
>Thread-15912670::DEBUG::2013-11-20 02:40:29,276::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912670::DEBUG::2013-11-20 02:40:29,276::task::495::TaskManager.Task::(_debug) Task 56e36ab3-fd13-48ba-8c16-fa595e25245c: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912670::DEBUG::2013-11-20 02:40:29,282::task::495::TaskManager.Task::(_debug) Task 56e36ab3-fd13-48ba-8c16-fa595e25245c: moving from state preparing -> state finished
>Thread-15912670::DEBUG::2013-11-20 02:40:29,282::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912670::DEBUG::2013-11-20 02:40:29,283::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912670::DEBUG::2013-11-20 02:40:29,283::task::495::TaskManager.Task::(_debug) Task 56e36ab3-fd13-48ba-8c16-fa595e25245c: ref 0 aborting False
>Thread-15912670::INFO::2013-11-20 02:40:29,284::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912671::DEBUG::2013-11-20 02:40:29,297::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912671::INFO::2013-11-20 02:40:29,298::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912671::DEBUG::2013-11-20 02:40:29,298::task::495::TaskManager.Task::(_debug) Task 719ffbc2-9d8d-42ce-998f-06fcb4936794: moving from state init -> state preparing
>Thread-15912671::DEBUG::2013-11-20 02:40:29,299::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bdcf08cc-f57b-4be6-a9ad-2aa47ef0a5d7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912671::DEBUG::2013-11-20 02:40:29,300::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912671::DEBUG::2013-11-20 02:40:29,306::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912671::DEBUG::2013-11-20 02:40:29,307::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bdcf08cc-f57b-4be6-a9ad-2aa47ef0a5d7`::Granted request
>Thread-15912671::DEBUG::2013-11-20 02:40:29,308::task::495::TaskManager.Task::(_debug) Task 719ffbc2-9d8d-42ce-998f-06fcb4936794: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912671::DEBUG::2013-11-20 02:40:29,308::task::495::TaskManager.Task::(_debug) Task 719ffbc2-9d8d-42ce-998f-06fcb4936794: ref 1 aborting False
>Thread-15912671::WARNING::2013-11-20 02:40:29,310::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912671::DEBUG::2013-11-20 02:40:29,314::task::495::TaskManager.Task::(_debug) Task 719ffbc2-9d8d-42ce-998f-06fcb4936794: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912671::DEBUG::2013-11-20 02:40:29,315::task::495::TaskManager.Task::(_debug) Task 719ffbc2-9d8d-42ce-998f-06fcb4936794: moving from state preparing -> state finished
>Thread-15912671::DEBUG::2013-11-20 02:40:29,316::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912671::DEBUG::2013-11-20 02:40:29,316::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912671::DEBUG::2013-11-20 02:40:29,317::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912671::DEBUG::2013-11-20 02:40:29,317::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912671::DEBUG::2013-11-20 02:40:29,318::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912671::DEBUG::2013-11-20 02:40:29,318::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912671::DEBUG::2013-11-20 02:40:29,319::task::495::TaskManager.Task::(_debug) Task 719ffbc2-9d8d-42ce-998f-06fcb4936794: ref 0 aborting False
>Thread-15912671::INFO::2013-11-20 02:40:29,320::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:40:30,199::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:40:30,702::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0758841 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15912672::INFO::2013-11-20 02:40:32,025::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912672::DEBUG::2013-11-20 02:40:32,026::task::495::TaskManager.Task::(_debug) Task defccff8-9554-40a7-84cb-36430f7de42c: moving from state init -> state preparing
>Thread-15912672::DEBUG::2013-11-20 02:40:32,027::task::495::TaskManager.Task::(_debug) Task defccff8-9554-40a7-84cb-36430f7de42c: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000823020935059', 'lastCheck': 1384915225.39729, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0166461467743', 'lastCheck': 1384915225.413501, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00139307975769', 'lastCheck': 1384915226.3934691, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000494003295898', 'lastCheck': 1384915224.209826, 'valid': True, 'code': 0}}
>Thread-15912672::DEBUG::2013-11-20 02:40:32,028::task::495::TaskManager.Task::(_debug) Task defccff8-9554-40a7-84cb-36430f7de42c: moving from state preparing -> state finished
>Thread-15912672::DEBUG::2013-11-20 02:40:32,028::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912672::DEBUG::2013-11-20 02:40:32,029::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912672::DEBUG::2013-11-20 02:40:32,029::task::495::TaskManager.Task::(_debug) Task defccff8-9554-40a7-84cb-36430f7de42c: ref 0 aborting False
>Thread-15912672::INFO::2013-11-20 02:40:32,030::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000823020935059', 'lastCheck': 1384915225.39729, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0166461467743', 'lastCheck': 1384915225.413501, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00139307975769', 'lastCheck': 1384915226.3934691, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000494003295898', 'lastCheck': 1384915224.209826, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:40:32,718::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:40:33,246::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0793916 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:40:35,262::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:40:35,821::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0712237 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:40:37,836::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:40:38,337::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0617664 s, 16.6 MB/s\n'; <rc> = 0
>Thread-15912676::DEBUG::2013-11-20 02:40:39,779::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912676::INFO::2013-11-20 02:40:39,783::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912676::DEBUG::2013-11-20 02:40:39,783::task::495::TaskManager.Task::(_debug) Task 0aa12d4c-faa2-4b28-9fee-ae6bf1997a45: moving from state init -> state preparing
>Thread-15912676::DEBUG::2013-11-20 02:40:39,784::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912676::DEBUG::2013-11-20 02:40:39,785::task::495::TaskManager.Task::(_debug) Task 0aa12d4c-faa2-4b28-9fee-ae6bf1997a45: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912676::DEBUG::2013-11-20 02:40:39,785::task::495::TaskManager.Task::(_debug) Task 0aa12d4c-faa2-4b28-9fee-ae6bf1997a45: moving from state preparing -> state finished
>Thread-15912676::DEBUG::2013-11-20 02:40:39,786::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912676::DEBUG::2013-11-20 02:40:39,786::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912676::DEBUG::2013-11-20 02:40:39,787::task::495::TaskManager.Task::(_debug) Task 0aa12d4c-faa2-4b28-9fee-ae6bf1997a45: ref 0 aborting False
>Thread-15912676::INFO::2013-11-20 02:40:39,787::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912677::DEBUG::2013-11-20 02:40:39,797::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912677::INFO::2013-11-20 02:40:39,798::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912677::DEBUG::2013-11-20 02:40:39,799::task::495::TaskManager.Task::(_debug) Task 38efb9a1-bf16-4373-93ea-6bcdb16b97a8: moving from state init -> state preparing
>Thread-15912677::DEBUG::2013-11-20 02:40:39,800::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c5492dd3-ef3f-44de-be86-1f5d9b20ef64`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912677::DEBUG::2013-11-20 02:40:39,800::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912677::DEBUG::2013-11-20 02:40:39,801::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912677::DEBUG::2013-11-20 02:40:39,802::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c5492dd3-ef3f-44de-be86-1f5d9b20ef64`::Granted request
>Thread-15912677::DEBUG::2013-11-20 02:40:39,803::task::495::TaskManager.Task::(_debug) Task 38efb9a1-bf16-4373-93ea-6bcdb16b97a8: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912677::DEBUG::2013-11-20 02:40:39,803::task::495::TaskManager.Task::(_debug) Task 38efb9a1-bf16-4373-93ea-6bcdb16b97a8: ref 1 aborting False
>Thread-15912677::WARNING::2013-11-20 02:40:39,805::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912677::DEBUG::2013-11-20 02:40:39,806::task::495::TaskManager.Task::(_debug) Task 38efb9a1-bf16-4373-93ea-6bcdb16b97a8: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115766984704', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115766984704', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912677::DEBUG::2013-11-20 02:40:39,807::task::495::TaskManager.Task::(_debug) Task 38efb9a1-bf16-4373-93ea-6bcdb16b97a8: moving from state preparing -> state finished
>Thread-15912677::DEBUG::2013-11-20 02:40:39,807::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912677::DEBUG::2013-11-20 02:40:39,808::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912677::DEBUG::2013-11-20 02:40:39,809::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912677::DEBUG::2013-11-20 02:40:39,809::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912677::DEBUG::2013-11-20 02:40:39,810::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912677::DEBUG::2013-11-20 02:40:39,810::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912677::DEBUG::2013-11-20 02:40:39,811::task::495::TaskManager.Task::(_debug) Task 38efb9a1-bf16-4373-93ea-6bcdb16b97a8: ref 0 aborting False
>Thread-15912677::INFO::2013-11-20 02:40:39,812::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115766984704', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115766984704', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:40:40,347::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:40:40,879::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0800273 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:40:42,893::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:40:43,456::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0761718 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:40:45,472::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:40:46,009::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0758962 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:40:48,024::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912680::INFO::2013-11-20 02:40:48,390::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912680::DEBUG::2013-11-20 02:40:48,391::task::495::TaskManager.Task::(_debug) Task 3708e432-1ac1-4697-a40f-38a3371d39e4: moving from state init -> state preparing
>Thread-15912680::DEBUG::2013-11-20 02:40:48,392::task::495::TaskManager.Task::(_debug) Task 3708e432-1ac1-4697-a40f-38a3371d39e4: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000463962554932', 'lastCheck': 1384915245.6879389, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00145602226257', 'lastCheck': 1384915245.691154, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00155782699585', 'lastCheck': 1384915246.6898489, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000530004501343', 'lastCheck': 1384915244.559463, 'valid': True, 'code': 0}}
>Thread-15912680::DEBUG::2013-11-20 02:40:48,393::task::495::TaskManager.Task::(_debug) Task 3708e432-1ac1-4697-a40f-38a3371d39e4: moving from state preparing -> state finished
>Thread-15912680::DEBUG::2013-11-20 02:40:48,394::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912680::DEBUG::2013-11-20 02:40:48,394::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912680::DEBUG::2013-11-20 02:40:48,395::task::495::TaskManager.Task::(_debug) Task 3708e432-1ac1-4697-a40f-38a3371d39e4: ref 0 aborting False
>Thread-15912680::INFO::2013-11-20 02:40:48,396::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000463962554932', 'lastCheck': 1384915245.6879389, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00145602226257', 'lastCheck': 1384915245.691154, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00155782699585', 'lastCheck': 1384915246.6898489, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000530004501343', 'lastCheck': 1384915244.559463, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:40:48,573::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0735003 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:40:50,588::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:40:51,113::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0736667 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15912682::DEBUG::2013-11-20 02:40:51,608::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912682::INFO::2013-11-20 02:40:51,614::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912682::DEBUG::2013-11-20 02:40:51,616::task::495::TaskManager.Task::(_debug) Task 78709e12-23ec-438e-b3ac-2471ec9429d2: moving from state init -> state preparing
>Thread-15912682::DEBUG::2013-11-20 02:40:51,617::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912682::DEBUG::2013-11-20 02:40:51,618::task::495::TaskManager.Task::(_debug) Task 78709e12-23ec-438e-b3ac-2471ec9429d2: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912682::DEBUG::2013-11-20 02:40:51,618::task::495::TaskManager.Task::(_debug) Task 78709e12-23ec-438e-b3ac-2471ec9429d2: moving from state preparing -> state finished
>Thread-15912682::DEBUG::2013-11-20 02:40:51,619::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912682::DEBUG::2013-11-20 02:40:51,619::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912682::DEBUG::2013-11-20 02:40:51,620::task::495::TaskManager.Task::(_debug) Task 78709e12-23ec-438e-b3ac-2471ec9429d2: ref 0 aborting False
>Thread-15912682::INFO::2013-11-20 02:40:51,621::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912683::DEBUG::2013-11-20 02:40:51,625::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912683::INFO::2013-11-20 02:40:51,626::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912683::DEBUG::2013-11-20 02:40:51,627::task::495::TaskManager.Task::(_debug) Task 2095afd3-b952-4581-a453-7514eaed8e40: moving from state init -> state preparing
>Thread-15912683::DEBUG::2013-11-20 02:40:51,628::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`25099e1b-8049-4631-9579-3cd56598e998`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912683::DEBUG::2013-11-20 02:40:51,629::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912683::DEBUG::2013-11-20 02:40:51,629::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912683::DEBUG::2013-11-20 02:40:51,630::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`25099e1b-8049-4631-9579-3cd56598e998`::Granted request
>Thread-15912683::DEBUG::2013-11-20 02:40:51,631::task::495::TaskManager.Task::(_debug) Task 2095afd3-b952-4581-a453-7514eaed8e40: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912683::DEBUG::2013-11-20 02:40:51,631::task::495::TaskManager.Task::(_debug) Task 2095afd3-b952-4581-a453-7514eaed8e40: ref 1 aborting False
>Thread-15912683::WARNING::2013-11-20 02:40:51,633::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912683::DEBUG::2013-11-20 02:40:51,634::task::495::TaskManager.Task::(_debug) Task 2095afd3-b952-4581-a453-7514eaed8e40: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115766984704', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115766984704', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912683::DEBUG::2013-11-20 02:40:51,635::task::495::TaskManager.Task::(_debug) Task 2095afd3-b952-4581-a453-7514eaed8e40: moving from state preparing -> state finished
>Thread-15912683::DEBUG::2013-11-20 02:40:51,636::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912683::DEBUG::2013-11-20 02:40:51,636::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912683::DEBUG::2013-11-20 02:40:51,637::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912683::DEBUG::2013-11-20 02:40:51,638::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912683::DEBUG::2013-11-20 02:40:51,638::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912683::DEBUG::2013-11-20 02:40:51,639::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912683::DEBUG::2013-11-20 02:40:51,639::task::495::TaskManager.Task::(_debug) Task 2095afd3-b952-4581-a453-7514eaed8e40: ref 0 aborting False
>Thread-15912683::INFO::2013-11-20 02:40:51,643::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115766984704', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115766984704', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:40:53,128::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:40:53,671::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0719806 s, 14.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:40:55,687::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:40:56,240::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0851882 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:40:58,265::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:40:58,820::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0758267 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 02:40:59,374::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 02:40:59,375::task::495::TaskManager.Task::(_debug) Task f2e1455a-f461-49cd-84fe-46e4fc8d1a20: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 02:40:59,380::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e288a725-3541-4c84-ad17-941fb4da2ece`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 02:40:59,385::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 02:40:59,392::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 02:40:59,394::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e288a725-3541-4c84-ad17-941fb4da2ece`::Granted request
>Thread-15912392::DEBUG::2013-11-20 02:40:59,400::task::495::TaskManager.Task::(_debug) Task f2e1455a-f461-49cd-84fe-46e4fc8d1a20: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 02:40:59,401::task::495::TaskManager.Task::(_debug) Task f2e1455a-f461-49cd-84fe-46e4fc8d1a20: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 02:40:59,403::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15912392::DEBUG::2013-11-20 02:40:59,406::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15912392::DEBUG::2013-11-20 02:41:00,358::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912392::DEBUG::2013-11-20 02:41:00,562::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15912392::DEBUG::2013-11-20 02:41:00,562::task::495::TaskManager.Task::(_debug) Task f2e1455a-f461-49cd-84fe-46e4fc8d1a20: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 02:41:00,563::task::495::TaskManager.Task::(_debug) Task f2e1455a-f461-49cd-84fe-46e4fc8d1a20: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 02:41:00,563::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 02:41:00,563::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 02:41:00,564::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 02:41:00,564::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 02:41:00,565::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 02:41:00,565::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 02:41:00,565::task::495::TaskManager.Task::(_debug) Task f2e1455a-f461-49cd-84fe-46e4fc8d1a20: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 02:41:00,566::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 02:41:00,835::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:41:01,333::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0709116 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:41:03,347::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:41:03,858::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0707051 s, 14.5 MB/s\n'; <rc> = 0
>Thread-15912688::DEBUG::2013-11-20 02:41:03,924::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912688::INFO::2013-11-20 02:41:03,925::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912688::DEBUG::2013-11-20 02:41:03,926::task::495::TaskManager.Task::(_debug) Task 55d6403a-8044-4267-be3a-2f2a02699a17: moving from state init -> state preparing
>Thread-15912688::DEBUG::2013-11-20 02:41:03,927::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912688::DEBUG::2013-11-20 02:41:03,927::task::495::TaskManager.Task::(_debug) Task 55d6403a-8044-4267-be3a-2f2a02699a17: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912688::DEBUG::2013-11-20 02:41:03,928::task::495::TaskManager.Task::(_debug) Task 55d6403a-8044-4267-be3a-2f2a02699a17: moving from state preparing -> state finished
>Thread-15912688::DEBUG::2013-11-20 02:41:03,928::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912688::DEBUG::2013-11-20 02:41:03,929::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912688::DEBUG::2013-11-20 02:41:03,930::task::495::TaskManager.Task::(_debug) Task 55d6403a-8044-4267-be3a-2f2a02699a17: ref 0 aborting False
>Thread-15912688::INFO::2013-11-20 02:41:03,930::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912689::DEBUG::2013-11-20 02:41:03,935::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912689::INFO::2013-11-20 02:41:03,936::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912689::DEBUG::2013-11-20 02:41:03,936::task::495::TaskManager.Task::(_debug) Task c2d1d8cb-6467-4583-812d-3ab5cd4878de: moving from state init -> state preparing
>Thread-15912689::DEBUG::2013-11-20 02:41:03,937::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`670b4f02-049d-4c9c-868d-cd12ea5fb4b5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912689::DEBUG::2013-11-20 02:41:03,938::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912689::DEBUG::2013-11-20 02:41:03,939::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912689::DEBUG::2013-11-20 02:41:03,939::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`670b4f02-049d-4c9c-868d-cd12ea5fb4b5`::Granted request
>Thread-15912689::DEBUG::2013-11-20 02:41:03,950::task::495::TaskManager.Task::(_debug) Task c2d1d8cb-6467-4583-812d-3ab5cd4878de: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912689::DEBUG::2013-11-20 02:41:03,950::task::495::TaskManager.Task::(_debug) Task c2d1d8cb-6467-4583-812d-3ab5cd4878de: ref 1 aborting False
>Thread-15912689::WARNING::2013-11-20 02:41:03,952::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912689::DEBUG::2013-11-20 02:41:03,953::task::495::TaskManager.Task::(_debug) Task c2d1d8cb-6467-4583-812d-3ab5cd4878de: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115766984704', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115766984704', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912689::DEBUG::2013-11-20 02:41:03,960::task::495::TaskManager.Task::(_debug) Task c2d1d8cb-6467-4583-812d-3ab5cd4878de: moving from state preparing -> state finished
>Thread-15912689::DEBUG::2013-11-20 02:41:03,961::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912689::DEBUG::2013-11-20 02:41:03,961::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912689::DEBUG::2013-11-20 02:41:03,962::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912689::DEBUG::2013-11-20 02:41:03,962::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912689::DEBUG::2013-11-20 02:41:03,963::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912689::DEBUG::2013-11-20 02:41:03,963::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912689::DEBUG::2013-11-20 02:41:03,964::task::495::TaskManager.Task::(_debug) Task c2d1d8cb-6467-4583-812d-3ab5cd4878de: ref 0 aborting False
>Thread-15912689::INFO::2013-11-20 02:41:03,965::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115766984704', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115766984704', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912690::INFO::2013-11-20 02:41:04,197::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912690::DEBUG::2013-11-20 02:41:04,198::task::495::TaskManager.Task::(_debug) Task 28cdbd1e-adf3-4234-ba10-156da287ed99: moving from state init -> state preparing
>Thread-15912690::DEBUG::2013-11-20 02:41:04,198::task::495::TaskManager.Task::(_debug) Task 28cdbd1e-adf3-4234-ba10-156da287ed99: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000423908233643', 'lastCheck': 1384915255.918988, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00137495994568', 'lastCheck': 1384915255.9221311, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0519759654999', 'lastCheck': 1384915256.962745, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00055193901062', 'lastCheck': 1384915254.807621, 'valid': True, 'code': 0}}
>Thread-15912690::DEBUG::2013-11-20 02:41:04,199::task::495::TaskManager.Task::(_debug) Task 28cdbd1e-adf3-4234-ba10-156da287ed99: moving from state preparing -> state finished
>Thread-15912690::DEBUG::2013-11-20 02:41:04,200::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912690::DEBUG::2013-11-20 02:41:04,200::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912690::DEBUG::2013-11-20 02:41:04,201::task::495::TaskManager.Task::(_debug) Task 28cdbd1e-adf3-4234-ba10-156da287ed99: ref 0 aborting False
>Thread-15912690::INFO::2013-11-20 02:41:04,201::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000423908233643', 'lastCheck': 1384915255.918988, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00137495994568', 'lastCheck': 1384915255.9221311, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0519759654999', 'lastCheck': 1384915256.962745, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00055193901062', 'lastCheck': 1384915254.807621, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:41:05,873::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-42838::DEBUG::2013-11-20 02:41:06,104::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:41:06,560::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0779848 s, 13.1 MB/s\n'; <rc> = 0
>Thread-42838::DEBUG::2013-11-20 02:41:06,856::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:41:08,575::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:41:09,136::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0860558 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:41:11,152::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:41:11,717::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0788987 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:41:13,732::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:41:14,291::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0754048 s, 13.6 MB/s\n'; <rc> = 0
>Thread-15912695::DEBUG::2013-11-20 02:41:15,939::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912695::INFO::2013-11-20 02:41:15,941::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912695::DEBUG::2013-11-20 02:41:15,941::task::495::TaskManager.Task::(_debug) Task e0aa5f83-b9e7-49ba-b821-956540166695: moving from state init -> state preparing
>Thread-15912695::DEBUG::2013-11-20 02:41:15,942::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912695::DEBUG::2013-11-20 02:41:15,943::task::495::TaskManager.Task::(_debug) Task e0aa5f83-b9e7-49ba-b821-956540166695: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912695::DEBUG::2013-11-20 02:41:15,943::task::495::TaskManager.Task::(_debug) Task e0aa5f83-b9e7-49ba-b821-956540166695: moving from state preparing -> state finished
>Thread-15912695::DEBUG::2013-11-20 02:41:15,944::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912695::DEBUG::2013-11-20 02:41:15,944::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912695::DEBUG::2013-11-20 02:41:15,945::task::495::TaskManager.Task::(_debug) Task e0aa5f83-b9e7-49ba-b821-956540166695: ref 0 aborting False
>Thread-15912695::INFO::2013-11-20 02:41:15,946::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912696::DEBUG::2013-11-20 02:41:15,950::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912696::INFO::2013-11-20 02:41:15,951::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912696::DEBUG::2013-11-20 02:41:15,951::task::495::TaskManager.Task::(_debug) Task 4102de90-a2dd-4059-90d1-5bccc68aa1a5: moving from state init -> state preparing
>Thread-15912696::DEBUG::2013-11-20 02:41:15,953::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`78673e99-b3dd-4552-9933-092f1a901ce2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912696::DEBUG::2013-11-20 02:41:15,953::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912696::DEBUG::2013-11-20 02:41:15,954::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912696::DEBUG::2013-11-20 02:41:15,955::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`78673e99-b3dd-4552-9933-092f1a901ce2`::Granted request
>Thread-15912696::DEBUG::2013-11-20 02:41:15,956::task::495::TaskManager.Task::(_debug) Task 4102de90-a2dd-4059-90d1-5bccc68aa1a5: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912696::DEBUG::2013-11-20 02:41:15,956::task::495::TaskManager.Task::(_debug) Task 4102de90-a2dd-4059-90d1-5bccc68aa1a5: ref 1 aborting False
>Thread-15912696::WARNING::2013-11-20 02:41:15,958::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912696::DEBUG::2013-11-20 02:41:15,959::task::495::TaskManager.Task::(_debug) Task 4102de90-a2dd-4059-90d1-5bccc68aa1a5: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115766984704', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115766984704', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912696::DEBUG::2013-11-20 02:41:15,960::task::495::TaskManager.Task::(_debug) Task 4102de90-a2dd-4059-90d1-5bccc68aa1a5: moving from state preparing -> state finished
>Thread-15912696::DEBUG::2013-11-20 02:41:15,961::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912696::DEBUG::2013-11-20 02:41:15,961::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912696::DEBUG::2013-11-20 02:41:15,962::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912696::DEBUG::2013-11-20 02:41:15,962::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912696::DEBUG::2013-11-20 02:41:15,963::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912696::DEBUG::2013-11-20 02:41:15,963::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912696::DEBUG::2013-11-20 02:41:15,964::task::495::TaskManager.Task::(_debug) Task 4102de90-a2dd-4059-90d1-5bccc68aa1a5: ref 0 aborting False
>Thread-15912696::INFO::2013-11-20 02:41:15,965::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115766984704', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115766984704', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:41:16,307::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:41:16,857::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0699548 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:41:18,872::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:41:19,409::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0741993 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15912698::INFO::2013-11-20 02:41:21,324::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912698::DEBUG::2013-11-20 02:41:21,325::task::495::TaskManager.Task::(_debug) Task 2f452b6b-adce-4f62-b399-2dd95d6dd468: moving from state init -> state preparing
>Thread-15912698::DEBUG::2013-11-20 02:41:21,326::task::495::TaskManager.Task::(_debug) Task 2f452b6b-adce-4f62-b399-2dd95d6dd468: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000442028045654', 'lastCheck': 1384915276.9627709, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00165891647339', 'lastCheck': 1384915276.5488529, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00156903266907', 'lastCheck': 1384915277.1301949, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000427007675171', 'lastCheck': 1384915275.113627, 'valid': True, 'code': 0}}
>Thread-15912698::DEBUG::2013-11-20 02:41:21,326::task::495::TaskManager.Task::(_debug) Task 2f452b6b-adce-4f62-b399-2dd95d6dd468: moving from state preparing -> state finished
>Thread-15912698::DEBUG::2013-11-20 02:41:21,327::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912698::DEBUG::2013-11-20 02:41:21,328::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912698::DEBUG::2013-11-20 02:41:21,328::task::495::TaskManager.Task::(_debug) Task 2f452b6b-adce-4f62-b399-2dd95d6dd468: ref 0 aborting False
>Thread-15912698::INFO::2013-11-20 02:41:21,329::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000442028045654', 'lastCheck': 1384915276.9627709, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00165891647339', 'lastCheck': 1384915276.5488529, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00156903266907', 'lastCheck': 1384915277.1301949, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000427007675171', 'lastCheck': 1384915275.113627, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:41:21,424::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:41:21,968::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.07415 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:41:23,980::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:41:24,519::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0748645 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:41:26,534::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:41:27,088::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0744492 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15912701::DEBUG::2013-11-20 02:41:28,887::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912701::INFO::2013-11-20 02:41:28,889::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912701::DEBUG::2013-11-20 02:41:28,889::task::495::TaskManager.Task::(_debug) Task c39d1344-00d7-4b69-b3a3-a58d9326a606: moving from state init -> state preparing
>Thread-15912701::DEBUG::2013-11-20 02:41:28,890::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912701::DEBUG::2013-11-20 02:41:28,891::task::495::TaskManager.Task::(_debug) Task c39d1344-00d7-4b69-b3a3-a58d9326a606: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912701::DEBUG::2013-11-20 02:41:28,891::task::495::TaskManager.Task::(_debug) Task c39d1344-00d7-4b69-b3a3-a58d9326a606: moving from state preparing -> state finished
>Thread-15912701::DEBUG::2013-11-20 02:41:28,892::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912701::DEBUG::2013-11-20 02:41:28,892::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912701::DEBUG::2013-11-20 02:41:28,893::task::495::TaskManager.Task::(_debug) Task c39d1344-00d7-4b69-b3a3-a58d9326a606: ref 0 aborting False
>Thread-15912701::INFO::2013-11-20 02:41:28,894::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912702::DEBUG::2013-11-20 02:41:28,898::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912702::INFO::2013-11-20 02:41:28,899::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912702::DEBUG::2013-11-20 02:41:28,900::task::495::TaskManager.Task::(_debug) Task 41898ff0-8700-4d03-a6b5-7d51dff93274: moving from state init -> state preparing
>Thread-15912702::DEBUG::2013-11-20 02:41:28,901::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f9b2081f-9a31-451d-96a7-ddfa5edaeb07`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912702::DEBUG::2013-11-20 02:41:28,901::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912702::DEBUG::2013-11-20 02:41:28,902::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912702::DEBUG::2013-11-20 02:41:28,903::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f9b2081f-9a31-451d-96a7-ddfa5edaeb07`::Granted request
>Thread-15912702::DEBUG::2013-11-20 02:41:28,904::task::495::TaskManager.Task::(_debug) Task 41898ff0-8700-4d03-a6b5-7d51dff93274: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912702::DEBUG::2013-11-20 02:41:28,904::task::495::TaskManager.Task::(_debug) Task 41898ff0-8700-4d03-a6b5-7d51dff93274: ref 1 aborting False
>Thread-15912702::WARNING::2013-11-20 02:41:28,906::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912702::DEBUG::2013-11-20 02:41:28,907::task::495::TaskManager.Task::(_debug) Task 41898ff0-8700-4d03-a6b5-7d51dff93274: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115766984704', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115766984704', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912702::DEBUG::2013-11-20 02:41:28,908::task::495::TaskManager.Task::(_debug) Task 41898ff0-8700-4d03-a6b5-7d51dff93274: moving from state preparing -> state finished
>Thread-15912702::DEBUG::2013-11-20 02:41:28,908::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912702::DEBUG::2013-11-20 02:41:28,909::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912702::DEBUG::2013-11-20 02:41:28,910::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912702::DEBUG::2013-11-20 02:41:28,910::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912702::DEBUG::2013-11-20 02:41:28,911::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912702::DEBUG::2013-11-20 02:41:28,911::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912702::DEBUG::2013-11-20 02:41:28,912::task::495::TaskManager.Task::(_debug) Task 41898ff0-8700-4d03-a6b5-7d51dff93274: ref 0 aborting False
>Thread-15912702::INFO::2013-11-20 02:41:28,912::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115766984704', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115766984704', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:41:29,104::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:41:29,634::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0773064 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:41:31,649::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:41:32,232::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.075587 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:41:34,247::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:41:34,791::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.075739 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:41:36,807::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:41:37,369::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0765814 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15912706::INFO::2013-11-20 02:41:38,861::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912706::DEBUG::2013-11-20 02:41:38,862::task::495::TaskManager.Task::(_debug) Task 11cf2abf-c45a-4353-99d7-3969b9a818c9: moving from state init -> state preparing
>Thread-15912706::DEBUG::2013-11-20 02:41:38,863::task::495::TaskManager.Task::(_debug) Task 11cf2abf-c45a-4353-99d7-3969b9a818c9: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000447034835815', 'lastCheck': 1384915297.4855809, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00154900550842', 'lastCheck': 1384915297.0441411, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00154519081116', 'lastCheck': 1384915297.4886761, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000396013259888', 'lastCheck': 1384915295.4777319, 'valid': True, 'code': 0}}
>Thread-15912706::DEBUG::2013-11-20 02:41:38,864::task::495::TaskManager.Task::(_debug) Task 11cf2abf-c45a-4353-99d7-3969b9a818c9: moving from state preparing -> state finished
>Thread-15912706::DEBUG::2013-11-20 02:41:38,864::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912706::DEBUG::2013-11-20 02:41:38,865::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912706::DEBUG::2013-11-20 02:41:38,865::task::495::TaskManager.Task::(_debug) Task 11cf2abf-c45a-4353-99d7-3969b9a818c9: ref 0 aborting False
>Thread-15912706::INFO::2013-11-20 02:41:38,866::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000447034835815', 'lastCheck': 1384915297.4855809, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00154900550842', 'lastCheck': 1384915297.0441411, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00154519081116', 'lastCheck': 1384915297.4886761, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000396013259888', 'lastCheck': 1384915295.4777319, 'valid': True, 'code': 0}}
>Thread-15912708::DEBUG::2013-11-20 02:41:39,352::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912708::INFO::2013-11-20 02:41:39,353::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912708::DEBUG::2013-11-20 02:41:39,354::task::495::TaskManager.Task::(_debug) Task f1bec3e4-e7cd-4739-b278-0d05f0288d0a: moving from state init -> state preparing
>Thread-15912708::DEBUG::2013-11-20 02:41:39,354::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912708::DEBUG::2013-11-20 02:41:39,355::task::495::TaskManager.Task::(_debug) Task f1bec3e4-e7cd-4739-b278-0d05f0288d0a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912708::DEBUG::2013-11-20 02:41:39,355::task::495::TaskManager.Task::(_debug) Task f1bec3e4-e7cd-4739-b278-0d05f0288d0a: moving from state preparing -> state finished
>Thread-15912708::DEBUG::2013-11-20 02:41:39,356::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912708::DEBUG::2013-11-20 02:41:39,356::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912708::DEBUG::2013-11-20 02:41:39,357::task::495::TaskManager.Task::(_debug) Task f1bec3e4-e7cd-4739-b278-0d05f0288d0a: ref 0 aborting False
>Thread-15912708::INFO::2013-11-20 02:41:39,367::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912709::DEBUG::2013-11-20 02:41:39,371::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912709::INFO::2013-11-20 02:41:39,372::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912709::DEBUG::2013-11-20 02:41:39,373::task::495::TaskManager.Task::(_debug) Task 43d11c4c-b07e-49af-a1e5-10f9c67822e8: moving from state init -> state preparing
>Thread-15912709::DEBUG::2013-11-20 02:41:39,374::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ea6fd0b8-ed24-4023-bfae-011ddf9ecbf5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912709::DEBUG::2013-11-20 02:41:39,375::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912709::DEBUG::2013-11-20 02:41:39,376::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912709::DEBUG::2013-11-20 02:41:39,376::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ea6fd0b8-ed24-4023-bfae-011ddf9ecbf5`::Granted request
>Thread-15912709::DEBUG::2013-11-20 02:41:39,377::task::495::TaskManager.Task::(_debug) Task 43d11c4c-b07e-49af-a1e5-10f9c67822e8: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912709::DEBUG::2013-11-20 02:41:39,378::task::495::TaskManager.Task::(_debug) Task 43d11c4c-b07e-49af-a1e5-10f9c67822e8: ref 1 aborting False
>Thread-15912709::WARNING::2013-11-20 02:41:39,379::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912709::DEBUG::2013-11-20 02:41:39,380::task::495::TaskManager.Task::(_debug) Task 43d11c4c-b07e-49af-a1e5-10f9c67822e8: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912709::DEBUG::2013-11-20 02:41:39,381::task::495::TaskManager.Task::(_debug) Task 43d11c4c-b07e-49af-a1e5-10f9c67822e8: moving from state preparing -> state finished
>Thread-15912709::DEBUG::2013-11-20 02:41:39,382::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912709::DEBUG::2013-11-20 02:41:39,382::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912709::DEBUG::2013-11-20 02:41:39,383::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912709::DEBUG::2013-11-20 02:41:39,383::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912709::DEBUG::2013-11-20 02:41:39,384::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912709::DEBUG::2013-11-20 02:41:39,384::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912709::DEBUG::2013-11-20 02:41:39,385::task::495::TaskManager.Task::(_debug) Task 43d11c4c-b07e-49af-a1e5-10f9c67822e8: ref 0 aborting False
>Dummy-15912651::DEBUG::2013-11-20 02:41:39,386::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912709::INFO::2013-11-20 02:41:39,386::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:41:39,983::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0775364 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:41:41,993::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:41:42,593::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0737248 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:41:44,608::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:41:45,149::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0701212 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:41:47,164::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:41:47,625::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0472678 s, 21.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:41:49,635::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912713::DEBUG::2013-11-20 02:41:50,065::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912713::INFO::2013-11-20 02:41:50,067::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912713::DEBUG::2013-11-20 02:41:50,068::task::495::TaskManager.Task::(_debug) Task 8cabe733-be44-4862-b6e2-ce2bfb9e3e8d: moving from state init -> state preparing
>Thread-15912713::DEBUG::2013-11-20 02:41:50,069::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912713::DEBUG::2013-11-20 02:41:50,070::task::495::TaskManager.Task::(_debug) Task 8cabe733-be44-4862-b6e2-ce2bfb9e3e8d: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912713::DEBUG::2013-11-20 02:41:50,071::task::495::TaskManager.Task::(_debug) Task 8cabe733-be44-4862-b6e2-ce2bfb9e3e8d: moving from state preparing -> state finished
>Thread-15912713::DEBUG::2013-11-20 02:41:50,071::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912713::DEBUG::2013-11-20 02:41:50,072::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912713::DEBUG::2013-11-20 02:41:50,078::task::495::TaskManager.Task::(_debug) Task 8cabe733-be44-4862-b6e2-ce2bfb9e3e8d: ref 0 aborting False
>Thread-15912713::INFO::2013-11-20 02:41:50,079::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912714::DEBUG::2013-11-20 02:41:50,084::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912714::INFO::2013-11-20 02:41:50,085::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912714::DEBUG::2013-11-20 02:41:50,086::task::495::TaskManager.Task::(_debug) Task 4f50c36e-c486-4b5d-8d3a-906704787593: moving from state init -> state preparing
>Thread-15912714::DEBUG::2013-11-20 02:41:50,087::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9b694b36-0a08-4b40-a1d6-d1878f5779cc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912714::DEBUG::2013-11-20 02:41:50,088::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912714::DEBUG::2013-11-20 02:41:50,089::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912714::DEBUG::2013-11-20 02:41:50,090::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9b694b36-0a08-4b40-a1d6-d1878f5779cc`::Granted request
>Thread-15912714::DEBUG::2013-11-20 02:41:50,092::task::495::TaskManager.Task::(_debug) Task 4f50c36e-c486-4b5d-8d3a-906704787593: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912714::DEBUG::2013-11-20 02:41:50,092::task::495::TaskManager.Task::(_debug) Task 4f50c36e-c486-4b5d-8d3a-906704787593: ref 1 aborting False
>Thread-15912714::WARNING::2013-11-20 02:41:50,095::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912714::DEBUG::2013-11-20 02:41:50,096::task::495::TaskManager.Task::(_debug) Task 4f50c36e-c486-4b5d-8d3a-906704787593: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912714::DEBUG::2013-11-20 02:41:50,098::task::495::TaskManager.Task::(_debug) Task 4f50c36e-c486-4b5d-8d3a-906704787593: moving from state preparing -> state finished
>Thread-15912714::DEBUG::2013-11-20 02:41:50,098::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912714::DEBUG::2013-11-20 02:41:50,099::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912714::DEBUG::2013-11-20 02:41:50,101::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912714::DEBUG::2013-11-20 02:41:50,102::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912714::DEBUG::2013-11-20 02:41:50,103::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912714::DEBUG::2013-11-20 02:41:50,103::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912714::DEBUG::2013-11-20 02:41:50,104::task::495::TaskManager.Task::(_debug) Task 4f50c36e-c486-4b5d-8d3a-906704787593: ref 0 aborting False
>Thread-15912714::INFO::2013-11-20 02:41:50,107::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:41:50,222::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0931993 s, 11.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:41:52,237::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:41:52,779::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0721959 s, 14.2 MB/s\n'; <rc> = 0
>Thread-15912716::INFO::2013-11-20 02:41:54,317::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912716::DEBUG::2013-11-20 02:41:54,318::task::495::TaskManager.Task::(_debug) Task 39624bcf-4acf-44a5-af1e-5d4995fd9f78: moving from state init -> state preparing
>Thread-15912716::DEBUG::2013-11-20 02:41:54,319::task::495::TaskManager.Task::(_debug) Task 39624bcf-4acf-44a5-af1e-5d4995fd9f78: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000433921813965', 'lastCheck': 1384915307.825901, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0012309551239', 'lastCheck': 1384915307.4356811, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00122404098511', 'lastCheck': 1384915307.8317511, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000479936599731', 'lastCheck': 1384915305.826436, 'valid': True, 'code': 0}}
>Thread-15912716::DEBUG::2013-11-20 02:41:54,320::task::495::TaskManager.Task::(_debug) Task 39624bcf-4acf-44a5-af1e-5d4995fd9f78: moving from state preparing -> state finished
>Thread-15912716::DEBUG::2013-11-20 02:41:54,320::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912716::DEBUG::2013-11-20 02:41:54,321::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912716::DEBUG::2013-11-20 02:41:54,322::task::495::TaskManager.Task::(_debug) Task 39624bcf-4acf-44a5-af1e-5d4995fd9f78: ref 0 aborting False
>Thread-15912716::INFO::2013-11-20 02:41:54,322::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000433921813965', 'lastCheck': 1384915307.825901, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0012309551239', 'lastCheck': 1384915307.4356811, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00122404098511', 'lastCheck': 1384915307.8317511, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000479936599731', 'lastCheck': 1384915305.826436, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:41:54,794::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:41:55,359::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0749692 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:41:57,374::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:41:57,941::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0758348 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:41:59,956::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:42:00,496::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.074524 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15912719::DEBUG::2013-11-20 02:42:00,850::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912719::INFO::2013-11-20 02:42:00,851::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912719::DEBUG::2013-11-20 02:42:00,852::task::495::TaskManager.Task::(_debug) Task 8a2e89f9-df06-4ce8-b33d-e467be94f11a: moving from state init -> state preparing
>Thread-15912719::DEBUG::2013-11-20 02:42:00,853::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912719::DEBUG::2013-11-20 02:42:00,854::task::495::TaskManager.Task::(_debug) Task 8a2e89f9-df06-4ce8-b33d-e467be94f11a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912719::DEBUG::2013-11-20 02:42:00,854::task::495::TaskManager.Task::(_debug) Task 8a2e89f9-df06-4ce8-b33d-e467be94f11a: moving from state preparing -> state finished
>Thread-15912719::DEBUG::2013-11-20 02:42:00,855::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912719::DEBUG::2013-11-20 02:42:00,855::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912719::DEBUG::2013-11-20 02:42:00,856::task::495::TaskManager.Task::(_debug) Task 8a2e89f9-df06-4ce8-b33d-e467be94f11a: ref 0 aborting False
>Thread-15912719::INFO::2013-11-20 02:42:00,856::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912720::DEBUG::2013-11-20 02:42:00,861::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912720::INFO::2013-11-20 02:42:00,862::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912720::DEBUG::2013-11-20 02:42:00,862::task::495::TaskManager.Task::(_debug) Task 944d892f-683d-4646-8b08-61a0b7c25914: moving from state init -> state preparing
>Thread-15912720::DEBUG::2013-11-20 02:42:00,864::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d4725b73-6630-49ff-a9ba-29bb988efa51`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912720::DEBUG::2013-11-20 02:42:00,864::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912720::DEBUG::2013-11-20 02:42:00,865::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912720::DEBUG::2013-11-20 02:42:00,866::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d4725b73-6630-49ff-a9ba-29bb988efa51`::Granted request
>Thread-15912720::DEBUG::2013-11-20 02:42:00,867::task::495::TaskManager.Task::(_debug) Task 944d892f-683d-4646-8b08-61a0b7c25914: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912720::DEBUG::2013-11-20 02:42:00,867::task::495::TaskManager.Task::(_debug) Task 944d892f-683d-4646-8b08-61a0b7c25914: ref 1 aborting False
>Thread-15912720::WARNING::2013-11-20 02:42:00,869::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912720::DEBUG::2013-11-20 02:42:00,870::task::495::TaskManager.Task::(_debug) Task 944d892f-683d-4646-8b08-61a0b7c25914: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912720::DEBUG::2013-11-20 02:42:00,871::task::495::TaskManager.Task::(_debug) Task 944d892f-683d-4646-8b08-61a0b7c25914: moving from state preparing -> state finished
>Thread-15912720::DEBUG::2013-11-20 02:42:00,872::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912720::DEBUG::2013-11-20 02:42:00,872::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912720::DEBUG::2013-11-20 02:42:00,873::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912720::DEBUG::2013-11-20 02:42:00,873::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912720::DEBUG::2013-11-20 02:42:00,874::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912720::DEBUG::2013-11-20 02:42:00,874::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912720::DEBUG::2013-11-20 02:42:00,875::task::495::TaskManager.Task::(_debug) Task 944d892f-683d-4646-8b08-61a0b7c25914: ref 0 aborting False
>Thread-15912720::INFO::2013-11-20 02:42:00,876::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912392::INFO::2013-11-20 02:42:01,053::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 02:42:01,054::task::495::TaskManager.Task::(_debug) Task d1871768-f8e6-4240-b157-6668c7c760c1: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 02:42:01,056::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f7a2d776-3e23-439e-af22-3c2719b5a3f2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 02:42:01,056::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 02:42:01,057::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 02:42:01,057::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f7a2d776-3e23-439e-af22-3c2719b5a3f2`::Granted request
>Thread-15912392::DEBUG::2013-11-20 02:42:01,058::task::495::TaskManager.Task::(_debug) Task d1871768-f8e6-4240-b157-6668c7c760c1: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 02:42:01,059::task::495::TaskManager.Task::(_debug) Task d1871768-f8e6-4240-b157-6668c7c760c1: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 02:42:01,059::task::495::TaskManager.Task::(_debug) Task d1871768-f8e6-4240-b157-6668c7c760c1: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 02:42:01,060::task::495::TaskManager.Task::(_debug) Task d1871768-f8e6-4240-b157-6668c7c760c1: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 02:42:01,061::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 02:42:01,061::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 02:42:01,062::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 02:42:01,062::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 02:42:01,063::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 02:42:01,064::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 02:42:01,064::task::495::TaskManager.Task::(_debug) Task d1871768-f8e6-4240-b157-6668c7c760c1: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 02:42:01,065::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 02:42:02,512::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:42:03,099::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0800868 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:42:05,116::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:42:05,657::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0902047 s, 11.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:42:07,672::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:42:08,238::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0746545 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15912724::INFO::2013-11-20 02:42:10,008::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912724::DEBUG::2013-11-20 02:42:10,008::task::495::TaskManager.Task::(_debug) Task 06a5c44d-436c-44ae-9826-d329bb55075a: moving from state init -> state preparing
>Thread-15912724::DEBUG::2013-11-20 02:42:10,009::task::495::TaskManager.Task::(_debug) Task 06a5c44d-436c-44ae-9826-d329bb55075a: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000664949417114', 'lastCheck': 1384915328.3280859, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00224781036377', 'lastCheck': 1384915327.90611, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00123286247253', 'lastCheck': 1384915328.3297441, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000549077987671', 'lastCheck': 1384915326.322731, 'valid': True, 'code': 0}}
>Thread-15912724::DEBUG::2013-11-20 02:42:10,010::task::495::TaskManager.Task::(_debug) Task 06a5c44d-436c-44ae-9826-d329bb55075a: moving from state preparing -> state finished
>Thread-15912724::DEBUG::2013-11-20 02:42:10,011::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912724::DEBUG::2013-11-20 02:42:10,011::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912724::DEBUG::2013-11-20 02:42:10,012::task::495::TaskManager.Task::(_debug) Task 06a5c44d-436c-44ae-9826-d329bb55075a: ref 0 aborting False
>Thread-15912724::INFO::2013-11-20 02:42:10,012::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000664949417114', 'lastCheck': 1384915328.3280859, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00224781036377', 'lastCheck': 1384915327.90611, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00123286247253', 'lastCheck': 1384915328.3297441, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000549077987671', 'lastCheck': 1384915326.322731, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:42:10,247::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:42:10,768::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0700235 s, 14.6 MB/s\n'; <rc> = 0
>Thread-15912726::DEBUG::2013-11-20 02:42:11,918::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912726::INFO::2013-11-20 02:42:11,919::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912726::DEBUG::2013-11-20 02:42:11,931::task::495::TaskManager.Task::(_debug) Task 65ea6596-3e20-4ce1-b030-bb2cbe486498: moving from state init -> state preparing
>Thread-15912726::DEBUG::2013-11-20 02:42:11,932::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912726::DEBUG::2013-11-20 02:42:11,933::task::495::TaskManager.Task::(_debug) Task 65ea6596-3e20-4ce1-b030-bb2cbe486498: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912726::DEBUG::2013-11-20 02:42:11,933::task::495::TaskManager.Task::(_debug) Task 65ea6596-3e20-4ce1-b030-bb2cbe486498: moving from state preparing -> state finished
>Thread-15912726::DEBUG::2013-11-20 02:42:11,934::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912726::DEBUG::2013-11-20 02:42:11,934::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912726::DEBUG::2013-11-20 02:42:11,935::task::495::TaskManager.Task::(_debug) Task 65ea6596-3e20-4ce1-b030-bb2cbe486498: ref 0 aborting False
>Thread-15912726::INFO::2013-11-20 02:42:11,936::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912727::DEBUG::2013-11-20 02:42:11,940::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912727::INFO::2013-11-20 02:42:11,941::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912727::DEBUG::2013-11-20 02:42:11,942::task::495::TaskManager.Task::(_debug) Task eec90911-c950-4316-8209-ce35378be8f1: moving from state init -> state preparing
>Thread-15912727::DEBUG::2013-11-20 02:42:11,943::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1f52d38a-3ec9-464e-885c-395dead1e9ff`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912727::DEBUG::2013-11-20 02:42:11,943::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912727::DEBUG::2013-11-20 02:42:11,944::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912727::DEBUG::2013-11-20 02:42:11,945::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1f52d38a-3ec9-464e-885c-395dead1e9ff`::Granted request
>Thread-15912727::DEBUG::2013-11-20 02:42:11,946::task::495::TaskManager.Task::(_debug) Task eec90911-c950-4316-8209-ce35378be8f1: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912727::DEBUG::2013-11-20 02:42:11,946::task::495::TaskManager.Task::(_debug) Task eec90911-c950-4316-8209-ce35378be8f1: ref 1 aborting False
>Thread-15912727::WARNING::2013-11-20 02:42:11,948::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912727::DEBUG::2013-11-20 02:42:11,949::task::495::TaskManager.Task::(_debug) Task eec90911-c950-4316-8209-ce35378be8f1: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912727::DEBUG::2013-11-20 02:42:11,950::task::495::TaskManager.Task::(_debug) Task eec90911-c950-4316-8209-ce35378be8f1: moving from state preparing -> state finished
>Thread-15912727::DEBUG::2013-11-20 02:42:11,950::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912727::DEBUG::2013-11-20 02:42:11,951::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912727::DEBUG::2013-11-20 02:42:11,952::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912727::DEBUG::2013-11-20 02:42:11,952::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912727::DEBUG::2013-11-20 02:42:11,953::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912727::DEBUG::2013-11-20 02:42:11,953::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912727::DEBUG::2013-11-20 02:42:11,954::task::495::TaskManager.Task::(_debug) Task eec90911-c950-4316-8209-ce35378be8f1: ref 0 aborting False
>Thread-15912727::INFO::2013-11-20 02:42:11,955::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:42:12,784::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:42:13,320::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0713513 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:42:15,335::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:42:15,854::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0701752 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:42:17,869::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:42:18,413::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0748935 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:42:20,428::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:42:20,989::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0843428 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:42:23,004::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:42:23,557::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0796514 s, 12.9 MB/s\n'; <rc> = 0
>Thread-15912732::DEBUG::2013-11-20 02:42:24,502::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912732::INFO::2013-11-20 02:42:24,504::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912732::DEBUG::2013-11-20 02:42:24,504::task::495::TaskManager.Task::(_debug) Task 78dbdfb2-fe4e-4fe9-878a-754713c40455: moving from state init -> state preparing
>Thread-15912732::DEBUG::2013-11-20 02:42:24,505::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912732::DEBUG::2013-11-20 02:42:24,506::task::495::TaskManager.Task::(_debug) Task 78dbdfb2-fe4e-4fe9-878a-754713c40455: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912732::DEBUG::2013-11-20 02:42:24,506::task::495::TaskManager.Task::(_debug) Task 78dbdfb2-fe4e-4fe9-878a-754713c40455: moving from state preparing -> state finished
>Thread-15912732::DEBUG::2013-11-20 02:42:24,507::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912732::DEBUG::2013-11-20 02:42:24,508::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912732::DEBUG::2013-11-20 02:42:24,508::task::495::TaskManager.Task::(_debug) Task 78dbdfb2-fe4e-4fe9-878a-754713c40455: ref 0 aborting False
>Thread-15912732::INFO::2013-11-20 02:42:24,509::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912733::DEBUG::2013-11-20 02:42:24,513::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912733::INFO::2013-11-20 02:42:24,514::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912733::DEBUG::2013-11-20 02:42:24,515::task::495::TaskManager.Task::(_debug) Task a043f461-2e1e-4b59-8c25-b6c55bf17ad5: moving from state init -> state preparing
>Thread-15912733::DEBUG::2013-11-20 02:42:24,516::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e108d009-8c79-4cb1-81e0-c4d467414342`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912733::DEBUG::2013-11-20 02:42:24,517::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912733::DEBUG::2013-11-20 02:42:24,517::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912733::DEBUG::2013-11-20 02:42:24,518::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e108d009-8c79-4cb1-81e0-c4d467414342`::Granted request
>Thread-15912733::DEBUG::2013-11-20 02:42:24,519::task::495::TaskManager.Task::(_debug) Task a043f461-2e1e-4b59-8c25-b6c55bf17ad5: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912733::DEBUG::2013-11-20 02:42:24,520::task::495::TaskManager.Task::(_debug) Task a043f461-2e1e-4b59-8c25-b6c55bf17ad5: ref 1 aborting False
>Thread-15912733::WARNING::2013-11-20 02:42:24,521::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912733::DEBUG::2013-11-20 02:42:24,522::task::495::TaskManager.Task::(_debug) Task a043f461-2e1e-4b59-8c25-b6c55bf17ad5: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912733::DEBUG::2013-11-20 02:42:24,523::task::495::TaskManager.Task::(_debug) Task a043f461-2e1e-4b59-8c25-b6c55bf17ad5: moving from state preparing -> state finished
>Thread-15912733::DEBUG::2013-11-20 02:42:24,524::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912733::DEBUG::2013-11-20 02:42:24,524::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912733::DEBUG::2013-11-20 02:42:24,525::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912733::DEBUG::2013-11-20 02:42:24,526::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912733::DEBUG::2013-11-20 02:42:24,526::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912733::DEBUG::2013-11-20 02:42:24,527::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912733::DEBUG::2013-11-20 02:42:24,527::task::495::TaskManager.Task::(_debug) Task a043f461-2e1e-4b59-8c25-b6c55bf17ad5: ref 0 aborting False
>Thread-15912733::INFO::2013-11-20 02:42:24,528::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:42:25,572::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912734::INFO::2013-11-20 02:42:25,793::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912734::DEBUG::2013-11-20 02:42:25,794::task::495::TaskManager.Task::(_debug) Task a3d7befd-81fc-46e6-a8f5-94ff5f25899e: moving from state init -> state preparing
>Thread-15912734::DEBUG::2013-11-20 02:42:25,795::task::495::TaskManager.Task::(_debug) Task a3d7befd-81fc-46e6-a8f5-94ff5f25899e: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000395059585571', 'lastCheck': 1384915338.5476961, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.124522924423', 'lastCheck': 1384915338.222285, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00137615203857', 'lastCheck': 1384915338.5493159, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000427007675171', 'lastCheck': 1384915336.5404551, 'valid': True, 'code': 0}}
>Thread-15912734::DEBUG::2013-11-20 02:42:25,796::task::495::TaskManager.Task::(_debug) Task a3d7befd-81fc-46e6-a8f5-94ff5f25899e: moving from state preparing -> state finished
>Thread-15912734::DEBUG::2013-11-20 02:42:25,797::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912734::DEBUG::2013-11-20 02:42:25,797::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912734::DEBUG::2013-11-20 02:42:25,798::task::495::TaskManager.Task::(_debug) Task a3d7befd-81fc-46e6-a8f5-94ff5f25899e: ref 0 aborting False
>Thread-15912734::INFO::2013-11-20 02:42:25,799::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000395059585571', 'lastCheck': 1384915338.5476961, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.124522924423', 'lastCheck': 1384915338.222285, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00137615203857', 'lastCheck': 1384915338.5493159, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000427007675171', 'lastCheck': 1384915336.5404551, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:42:26,099::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.070764 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:42:28,114::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:42:28,643::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0762249 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:42:30,657::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:42:31,219::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0763 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:42:33,236::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:42:33,773::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0704052 s, 14.5 MB/s\n'; <rc> = 0
>Thread-15912738::DEBUG::2013-11-20 02:42:35,324::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912738::INFO::2013-11-20 02:42:35,326::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912738::DEBUG::2013-11-20 02:42:35,326::task::495::TaskManager.Task::(_debug) Task 1ccbbf53-3f84-45b0-b087-b46d906f0b65: moving from state init -> state preparing
>Thread-15912738::DEBUG::2013-11-20 02:42:35,327::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912738::DEBUG::2013-11-20 02:42:35,328::task::495::TaskManager.Task::(_debug) Task 1ccbbf53-3f84-45b0-b087-b46d906f0b65: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912738::DEBUG::2013-11-20 02:42:35,328::task::495::TaskManager.Task::(_debug) Task 1ccbbf53-3f84-45b0-b087-b46d906f0b65: moving from state preparing -> state finished
>Thread-15912738::DEBUG::2013-11-20 02:42:35,329::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912738::DEBUG::2013-11-20 02:42:35,330::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912738::DEBUG::2013-11-20 02:42:35,331::task::495::TaskManager.Task::(_debug) Task 1ccbbf53-3f84-45b0-b087-b46d906f0b65: ref 0 aborting False
>Thread-15912738::INFO::2013-11-20 02:42:35,332::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912739::DEBUG::2013-11-20 02:42:35,337::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912739::INFO::2013-11-20 02:42:35,338::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912739::DEBUG::2013-11-20 02:42:35,338::task::495::TaskManager.Task::(_debug) Task 7ee15d31-5158-461d-ae73-7828ca7a6aa7: moving from state init -> state preparing
>Thread-15912739::DEBUG::2013-11-20 02:42:35,339::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`54ce41e1-64d1-48d3-bc7f-c2a97110a908`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912739::DEBUG::2013-11-20 02:42:35,340::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912739::DEBUG::2013-11-20 02:42:35,341::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912739::DEBUG::2013-11-20 02:42:35,341::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`54ce41e1-64d1-48d3-bc7f-c2a97110a908`::Granted request
>Thread-15912739::DEBUG::2013-11-20 02:42:35,342::task::495::TaskManager.Task::(_debug) Task 7ee15d31-5158-461d-ae73-7828ca7a6aa7: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912739::DEBUG::2013-11-20 02:42:35,343::task::495::TaskManager.Task::(_debug) Task 7ee15d31-5158-461d-ae73-7828ca7a6aa7: ref 1 aborting False
>Thread-15912739::WARNING::2013-11-20 02:42:35,345::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912739::DEBUG::2013-11-20 02:42:35,346::task::495::TaskManager.Task::(_debug) Task 7ee15d31-5158-461d-ae73-7828ca7a6aa7: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912739::DEBUG::2013-11-20 02:42:35,347::task::495::TaskManager.Task::(_debug) Task 7ee15d31-5158-461d-ae73-7828ca7a6aa7: moving from state preparing -> state finished
>Thread-15912739::DEBUG::2013-11-20 02:42:35,348::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912739::DEBUG::2013-11-20 02:42:35,348::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912739::DEBUG::2013-11-20 02:42:35,349::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912739::DEBUG::2013-11-20 02:42:35,350::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912739::DEBUG::2013-11-20 02:42:35,350::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912739::DEBUG::2013-11-20 02:42:35,351::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912739::DEBUG::2013-11-20 02:42:35,351::task::495::TaskManager.Task::(_debug) Task 7ee15d31-5158-461d-ae73-7828ca7a6aa7: ref 0 aborting False
>Thread-15912739::INFO::2013-11-20 02:42:35,352::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:42:35,788::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:42:36,322::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.070235 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:42:38,338::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:42:38,872::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0819654 s, 12.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:42:40,887::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:42:41,445::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0766569 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:42:43,471::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912742::INFO::2013-11-20 02:42:43,720::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912742::DEBUG::2013-11-20 02:42:43,722::task::495::TaskManager.Task::(_debug) Task 61b61543-d296-4eba-b5d8-3591263f4582: moving from state init -> state preparing
>Thread-15912742::DEBUG::2013-11-20 02:42:43,723::task::495::TaskManager.Task::(_debug) Task 61b61543-d296-4eba-b5d8-3591263f4582: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000471830368042', 'lastCheck': 1384915359.0243089, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00143384933472', 'lastCheck': 1384915358.581356, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00233507156372', 'lastCheck': 1384915359.0230131, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00065803527832', 'lastCheck': 1384915357.0111251, 'valid': True, 'code': 0}}
>Thread-15912742::DEBUG::2013-11-20 02:42:43,724::task::495::TaskManager.Task::(_debug) Task 61b61543-d296-4eba-b5d8-3591263f4582: moving from state preparing -> state finished
>Thread-15912742::DEBUG::2013-11-20 02:42:43,724::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912742::DEBUG::2013-11-20 02:42:43,725::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912742::DEBUG::2013-11-20 02:42:43,726::task::495::TaskManager.Task::(_debug) Task 61b61543-d296-4eba-b5d8-3591263f4582: ref 0 aborting False
>Thread-15912742::INFO::2013-11-20 02:42:43,726::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000471830368042', 'lastCheck': 1384915359.0243089, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00143384933472', 'lastCheck': 1384915358.581356, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00233507156372', 'lastCheck': 1384915359.0230131, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00065803527832', 'lastCheck': 1384915357.0111251, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:42:43,979::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0641547 s, 16.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:42:45,993::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912744::DEBUG::2013-11-20 02:42:46,225::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912744::INFO::2013-11-20 02:42:46,227::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912744::DEBUG::2013-11-20 02:42:46,228::task::495::TaskManager.Task::(_debug) Task e4d28eb3-4182-4576-8707-5ab4378c94f6: moving from state init -> state preparing
>Thread-15912744::DEBUG::2013-11-20 02:42:46,229::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912744::DEBUG::2013-11-20 02:42:46,230::task::495::TaskManager.Task::(_debug) Task e4d28eb3-4182-4576-8707-5ab4378c94f6: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912744::DEBUG::2013-11-20 02:42:46,230::task::495::TaskManager.Task::(_debug) Task e4d28eb3-4182-4576-8707-5ab4378c94f6: moving from state preparing -> state finished
>Thread-15912744::DEBUG::2013-11-20 02:42:46,231::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912744::DEBUG::2013-11-20 02:42:46,232::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912744::DEBUG::2013-11-20 02:42:46,232::task::495::TaskManager.Task::(_debug) Task e4d28eb3-4182-4576-8707-5ab4378c94f6: ref 0 aborting False
>Thread-15912744::INFO::2013-11-20 02:42:46,233::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912745::DEBUG::2013-11-20 02:42:46,239::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912745::INFO::2013-11-20 02:42:46,240::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912745::DEBUG::2013-11-20 02:42:46,241::task::495::TaskManager.Task::(_debug) Task 1feb7869-6b83-43b2-be2a-2eb62509d8de: moving from state init -> state preparing
>Thread-15912745::DEBUG::2013-11-20 02:42:46,242::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2658b0d4-190a-4bed-a05c-5038709070ad`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912745::DEBUG::2013-11-20 02:42:46,243::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912745::DEBUG::2013-11-20 02:42:46,244::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912745::DEBUG::2013-11-20 02:42:46,245::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2658b0d4-190a-4bed-a05c-5038709070ad`::Granted request
>Thread-15912745::DEBUG::2013-11-20 02:42:46,246::task::495::TaskManager.Task::(_debug) Task 1feb7869-6b83-43b2-be2a-2eb62509d8de: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912745::DEBUG::2013-11-20 02:42:46,246::task::495::TaskManager.Task::(_debug) Task 1feb7869-6b83-43b2-be2a-2eb62509d8de: ref 1 aborting False
>Thread-15912745::WARNING::2013-11-20 02:42:46,249::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912745::DEBUG::2013-11-20 02:42:46,250::task::495::TaskManager.Task::(_debug) Task 1feb7869-6b83-43b2-be2a-2eb62509d8de: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912745::DEBUG::2013-11-20 02:42:46,251::task::495::TaskManager.Task::(_debug) Task 1feb7869-6b83-43b2-be2a-2eb62509d8de: moving from state preparing -> state finished
>Thread-15912745::DEBUG::2013-11-20 02:42:46,251::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912745::DEBUG::2013-11-20 02:42:46,252::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912745::DEBUG::2013-11-20 02:42:46,253::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912745::DEBUG::2013-11-20 02:42:46,253::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912745::DEBUG::2013-11-20 02:42:46,254::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912745::DEBUG::2013-11-20 02:42:46,254::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912745::DEBUG::2013-11-20 02:42:46,255::task::495::TaskManager.Task::(_debug) Task 1feb7869-6b83-43b2-be2a-2eb62509d8de: ref 0 aborting False
>Thread-15912745::INFO::2013-11-20 02:42:46,256::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:42:46,535::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0771923 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:42:48,551::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:42:49,092::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0781407 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:42:51,109::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:42:51,628::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0753141 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:42:53,643::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:42:54,193::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.074161 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:42:56,208::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:42:56,722::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0758035 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15912750::DEBUG::2013-11-20 02:42:57,407::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912750::INFO::2013-11-20 02:42:57,409::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912750::DEBUG::2013-11-20 02:42:57,409::task::495::TaskManager.Task::(_debug) Task 89f301a3-785c-47af-bcda-a621e8ea41e7: moving from state init -> state preparing
>Thread-15912750::DEBUG::2013-11-20 02:42:57,410::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912750::DEBUG::2013-11-20 02:42:57,411::task::495::TaskManager.Task::(_debug) Task 89f301a3-785c-47af-bcda-a621e8ea41e7: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912750::DEBUG::2013-11-20 02:42:57,411::task::495::TaskManager.Task::(_debug) Task 89f301a3-785c-47af-bcda-a621e8ea41e7: moving from state preparing -> state finished
>Thread-15912750::DEBUG::2013-11-20 02:42:57,412::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912750::DEBUG::2013-11-20 02:42:57,412::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912750::DEBUG::2013-11-20 02:42:57,413::task::495::TaskManager.Task::(_debug) Task 89f301a3-785c-47af-bcda-a621e8ea41e7: ref 0 aborting False
>Thread-15912750::INFO::2013-11-20 02:42:57,414::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912751::DEBUG::2013-11-20 02:42:57,425::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912751::INFO::2013-11-20 02:42:57,426::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912751::DEBUG::2013-11-20 02:42:57,427::task::495::TaskManager.Task::(_debug) Task a2834101-b195-4030-b5f9-8ca771e7b8f6: moving from state init -> state preparing
>Thread-15912751::DEBUG::2013-11-20 02:42:57,428::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`da26024c-3411-4463-b659-c9f4ce7c72b3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912751::DEBUG::2013-11-20 02:42:57,428::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912751::DEBUG::2013-11-20 02:42:57,429::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912751::DEBUG::2013-11-20 02:42:57,430::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`da26024c-3411-4463-b659-c9f4ce7c72b3`::Granted request
>Thread-15912751::DEBUG::2013-11-20 02:42:57,431::task::495::TaskManager.Task::(_debug) Task a2834101-b195-4030-b5f9-8ca771e7b8f6: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912751::DEBUG::2013-11-20 02:42:57,431::task::495::TaskManager.Task::(_debug) Task a2834101-b195-4030-b5f9-8ca771e7b8f6: ref 1 aborting False
>Thread-15912751::WARNING::2013-11-20 02:42:57,433::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912751::DEBUG::2013-11-20 02:42:57,434::task::495::TaskManager.Task::(_debug) Task a2834101-b195-4030-b5f9-8ca771e7b8f6: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912751::DEBUG::2013-11-20 02:42:57,435::task::495::TaskManager.Task::(_debug) Task a2834101-b195-4030-b5f9-8ca771e7b8f6: moving from state preparing -> state finished
>Thread-15912751::DEBUG::2013-11-20 02:42:57,435::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912751::DEBUG::2013-11-20 02:42:57,436::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912751::DEBUG::2013-11-20 02:42:57,436::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912751::DEBUG::2013-11-20 02:42:57,437::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912751::DEBUG::2013-11-20 02:42:57,437::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912751::DEBUG::2013-11-20 02:42:57,438::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912751::DEBUG::2013-11-20 02:42:57,439::task::495::TaskManager.Task::(_debug) Task a2834101-b195-4030-b5f9-8ca771e7b8f6: ref 0 aborting False
>Thread-15912751::INFO::2013-11-20 02:42:57,439::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:42:58,738::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:42:59,160::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0608333 s, 16.8 MB/s\n'; <rc> = 0
>Thread-15912752::INFO::2013-11-20 02:42:59,187::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912752::DEBUG::2013-11-20 02:42:59,188::task::495::TaskManager.Task::(_debug) Task c969cfda-c264-48bc-890a-c2a07de03c7c: moving from state init -> state preparing
>Thread-15912752::DEBUG::2013-11-20 02:42:59,188::task::495::TaskManager.Task::(_debug) Task c969cfda-c264-48bc-890a-c2a07de03c7c: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000367879867554', 'lastCheck': 1384915369.2288799, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00118899345398', 'lastCheck': 1384915378.9247739, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00131702423096', 'lastCheck': 1384915369.231473, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00059986114502', 'lastCheck': 1384915377.416348, 'valid': True, 'code': 0}}
>Thread-15912752::DEBUG::2013-11-20 02:42:59,189::task::495::TaskManager.Task::(_debug) Task c969cfda-c264-48bc-890a-c2a07de03c7c: moving from state preparing -> state finished
>Thread-15912752::DEBUG::2013-11-20 02:42:59,190::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912752::DEBUG::2013-11-20 02:42:59,190::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912752::DEBUG::2013-11-20 02:42:59,191::task::495::TaskManager.Task::(_debug) Task c969cfda-c264-48bc-890a-c2a07de03c7c: ref 0 aborting False
>Thread-15912752::INFO::2013-11-20 02:42:59,192::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000367879867554', 'lastCheck': 1384915369.2288799, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00118899345398', 'lastCheck': 1384915378.9247739, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00131702423096', 'lastCheck': 1384915369.231473, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00059986114502', 'lastCheck': 1384915377.416348, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:43:01,171::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 02:43:01,503::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 02:43:01,504::task::495::TaskManager.Task::(_debug) Task 5540029b-c8a9-4248-9f82-b622954b2e15: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 02:43:01,506::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`17bcd170-89de-42a3-9009-94daf227da06`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 02:43:01,507::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 02:43:01,507::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 02:43:01,508::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`17bcd170-89de-42a3-9009-94daf227da06`::Granted request
>Thread-15912392::DEBUG::2013-11-20 02:43:01,510::task::495::TaskManager.Task::(_debug) Task 5540029b-c8a9-4248-9f82-b622954b2e15: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 02:43:01,511::task::495::TaskManager.Task::(_debug) Task 5540029b-c8a9-4248-9f82-b622954b2e15: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 02:43:01,512::task::495::TaskManager.Task::(_debug) Task 5540029b-c8a9-4248-9f82-b622954b2e15: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 02:43:01,513::task::495::TaskManager.Task::(_debug) Task 5540029b-c8a9-4248-9f82-b622954b2e15: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 02:43:01,514::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 02:43:01,515::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 02:43:01,516::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 02:43:01,517::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 02:43:01,518::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 02:43:01,518::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 02:43:01,519::task::495::TaskManager.Task::(_debug) Task 5540029b-c8a9-4248-9f82-b622954b2e15: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 02:43:01,520::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 02:43:01,760::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0878887 s, 11.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:43:03,775::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:43:04,309::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0749967 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:43:06,325::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:43:06,835::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.049113 s, 20.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:43:08,849::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:43:09,344::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0756869 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15912757::DEBUG::2013-11-20 02:43:09,589::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912757::INFO::2013-11-20 02:43:09,591::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912757::DEBUG::2013-11-20 02:43:09,591::task::495::TaskManager.Task::(_debug) Task d6f3855c-d892-4c91-a665-815c20e83b99: moving from state init -> state preparing
>Thread-15912757::DEBUG::2013-11-20 02:43:09,592::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912757::DEBUG::2013-11-20 02:43:09,592::task::495::TaskManager.Task::(_debug) Task d6f3855c-d892-4c91-a665-815c20e83b99: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912757::DEBUG::2013-11-20 02:43:09,593::task::495::TaskManager.Task::(_debug) Task d6f3855c-d892-4c91-a665-815c20e83b99: moving from state preparing -> state finished
>Thread-15912757::DEBUG::2013-11-20 02:43:09,594::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912757::DEBUG::2013-11-20 02:43:09,594::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912757::DEBUG::2013-11-20 02:43:09,595::task::495::TaskManager.Task::(_debug) Task d6f3855c-d892-4c91-a665-815c20e83b99: ref 0 aborting False
>Thread-15912757::INFO::2013-11-20 02:43:09,595::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912758::DEBUG::2013-11-20 02:43:09,599::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912758::INFO::2013-11-20 02:43:09,600::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912758::DEBUG::2013-11-20 02:43:09,601::task::495::TaskManager.Task::(_debug) Task 58c39ccd-eb48-4bc2-87d3-63a14d302f3b: moving from state init -> state preparing
>Thread-15912758::DEBUG::2013-11-20 02:43:09,602::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4918b0ff-90a6-4ff2-8451-f75bbb75393c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912758::DEBUG::2013-11-20 02:43:09,603::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912758::DEBUG::2013-11-20 02:43:09,603::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912758::DEBUG::2013-11-20 02:43:09,604::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4918b0ff-90a6-4ff2-8451-f75bbb75393c`::Granted request
>Thread-15912758::DEBUG::2013-11-20 02:43:09,605::task::495::TaskManager.Task::(_debug) Task 58c39ccd-eb48-4bc2-87d3-63a14d302f3b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912758::DEBUG::2013-11-20 02:43:09,605::task::495::TaskManager.Task::(_debug) Task 58c39ccd-eb48-4bc2-87d3-63a14d302f3b: ref 1 aborting False
>Thread-15912758::WARNING::2013-11-20 02:43:09,607::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912758::DEBUG::2013-11-20 02:43:09,608::task::495::TaskManager.Task::(_debug) Task 58c39ccd-eb48-4bc2-87d3-63a14d302f3b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912758::DEBUG::2013-11-20 02:43:09,609::task::495::TaskManager.Task::(_debug) Task 58c39ccd-eb48-4bc2-87d3-63a14d302f3b: moving from state preparing -> state finished
>Thread-15912758::DEBUG::2013-11-20 02:43:09,610::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912758::DEBUG::2013-11-20 02:43:09,610::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912758::DEBUG::2013-11-20 02:43:09,611::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912758::DEBUG::2013-11-20 02:43:09,612::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912758::DEBUG::2013-11-20 02:43:09,612::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912758::DEBUG::2013-11-20 02:43:09,613::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912758::DEBUG::2013-11-20 02:43:09,613::task::495::TaskManager.Task::(_debug) Task 58c39ccd-eb48-4bc2-87d3-63a14d302f3b: ref 0 aborting False
>Thread-15912758::INFO::2013-11-20 02:43:09,614::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:43:11,359::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:43:11,801::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0486161 s, 21.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:43:13,811::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:43:14,219::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0474025 s, 21.6 MB/s\n'; <rc> = 0
>Thread-15912760::INFO::2013-11-20 02:43:15,157::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912760::DEBUG::2013-11-20 02:43:15,158::task::495::TaskManager.Task::(_debug) Task 6cd41819-d98e-4959-a961-6903c9efa119: moving from state init -> state preparing
>Thread-15912760::DEBUG::2013-11-20 02:43:15,159::task::495::TaskManager.Task::(_debug) Task 6cd41819-d98e-4959-a961-6903c9efa119: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000289916992188', 'lastCheck': 1384915389.544889, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00190305709839', 'lastCheck': 1384915389.0541351, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00107383728027', 'lastCheck': 1384915389.545351, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000518083572388', 'lastCheck': 1384915387.5389841, 'valid': True, 'code': 0}}
>Thread-15912760::DEBUG::2013-11-20 02:43:15,160::task::495::TaskManager.Task::(_debug) Task 6cd41819-d98e-4959-a961-6903c9efa119: moving from state preparing -> state finished
>Thread-15912760::DEBUG::2013-11-20 02:43:15,161::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912760::DEBUG::2013-11-20 02:43:15,161::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912760::DEBUG::2013-11-20 02:43:15,162::task::495::TaskManager.Task::(_debug) Task 6cd41819-d98e-4959-a961-6903c9efa119: ref 0 aborting False
>Thread-15912760::INFO::2013-11-20 02:43:15,162::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000289916992188', 'lastCheck': 1384915389.544889, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00190305709839', 'lastCheck': 1384915389.0541351, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00107383728027', 'lastCheck': 1384915389.545351, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000518083572388', 'lastCheck': 1384915387.5389841, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:43:16,230::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:43:16,763::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0745511 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:43:18,778::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:43:19,271::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0704296 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:43:21,286::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:43:21,813::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0732069 s, 14.0 MB/s\n'; <rc> = 0
>Thread-15912763::DEBUG::2013-11-20 02:43:22,758::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912763::INFO::2013-11-20 02:43:22,759::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912763::DEBUG::2013-11-20 02:43:22,760::task::495::TaskManager.Task::(_debug) Task 5c18aaae-08cb-4a8a-90f3-0cc972afcd12: moving from state init -> state preparing
>Thread-15912763::DEBUG::2013-11-20 02:43:22,761::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912763::DEBUG::2013-11-20 02:43:22,761::task::495::TaskManager.Task::(_debug) Task 5c18aaae-08cb-4a8a-90f3-0cc972afcd12: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912763::DEBUG::2013-11-20 02:43:22,762::task::495::TaskManager.Task::(_debug) Task 5c18aaae-08cb-4a8a-90f3-0cc972afcd12: moving from state preparing -> state finished
>Thread-15912763::DEBUG::2013-11-20 02:43:22,762::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912763::DEBUG::2013-11-20 02:43:22,763::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912763::DEBUG::2013-11-20 02:43:22,764::task::495::TaskManager.Task::(_debug) Task 5c18aaae-08cb-4a8a-90f3-0cc972afcd12: ref 0 aborting False
>Thread-15912763::INFO::2013-11-20 02:43:22,764::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912764::DEBUG::2013-11-20 02:43:22,769::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912764::INFO::2013-11-20 02:43:22,769::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912764::DEBUG::2013-11-20 02:43:22,770::task::495::TaskManager.Task::(_debug) Task 5b39ee8c-89b2-4362-adc9-faa20d2322f7: moving from state init -> state preparing
>Thread-15912764::DEBUG::2013-11-20 02:43:22,771::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c3e4df4f-5c3a-49e4-99ba-ed01abf64e1d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912764::DEBUG::2013-11-20 02:43:22,772::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912764::DEBUG::2013-11-20 02:43:22,773::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912764::DEBUG::2013-11-20 02:43:22,773::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c3e4df4f-5c3a-49e4-99ba-ed01abf64e1d`::Granted request
>Thread-15912764::DEBUG::2013-11-20 02:43:22,781::task::495::TaskManager.Task::(_debug) Task 5b39ee8c-89b2-4362-adc9-faa20d2322f7: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912764::DEBUG::2013-11-20 02:43:22,781::task::495::TaskManager.Task::(_debug) Task 5b39ee8c-89b2-4362-adc9-faa20d2322f7: ref 1 aborting False
>Thread-15912764::WARNING::2013-11-20 02:43:22,783::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912764::DEBUG::2013-11-20 02:43:22,784::task::495::TaskManager.Task::(_debug) Task 5b39ee8c-89b2-4362-adc9-faa20d2322f7: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912764::DEBUG::2013-11-20 02:43:22,785::task::495::TaskManager.Task::(_debug) Task 5b39ee8c-89b2-4362-adc9-faa20d2322f7: moving from state preparing -> state finished
>Thread-15912764::DEBUG::2013-11-20 02:43:22,786::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912764::DEBUG::2013-11-20 02:43:22,786::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912764::DEBUG::2013-11-20 02:43:22,787::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912764::DEBUG::2013-11-20 02:43:22,787::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912764::DEBUG::2013-11-20 02:43:22,788::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912764::DEBUG::2013-11-20 02:43:22,788::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912764::DEBUG::2013-11-20 02:43:22,789::task::495::TaskManager.Task::(_debug) Task 5b39ee8c-89b2-4362-adc9-faa20d2322f7: ref 0 aborting False
>Thread-15912764::INFO::2013-11-20 02:43:22,790::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:43:23,828::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:43:24,375::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0736866 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:43:26,389::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:43:26,929::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0746287 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:43:28,944::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:43:29,486::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0778107 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:43:31,500::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:43:32,017::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0723419 s, 14.2 MB/s\n'; <rc> = 0
>Thread-15912768::INFO::2013-11-20 02:43:32,683::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912768::DEBUG::2013-11-20 02:43:32,683::task::495::TaskManager.Task::(_debug) Task fd4569dc-f4ee-4175-a75e-39297c8e197c: moving from state init -> state preparing
>Thread-15912768::DEBUG::2013-11-20 02:43:32,684::task::495::TaskManager.Task::(_debug) Task fd4569dc-f4ee-4175-a75e-39297c8e197c: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000468015670776', 'lastCheck': 1384915409.6229081, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.111585140228', 'lastCheck': 1384915409.2822199, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.1806640625', 'lastCheck': 1384915409.8253901, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000432968139648', 'lastCheck': 1384915407.623462, 'valid': True, 'code': 0}}
>Thread-15912768::DEBUG::2013-11-20 02:43:32,684::task::495::TaskManager.Task::(_debug) Task fd4569dc-f4ee-4175-a75e-39297c8e197c: moving from state preparing -> state finished
>Thread-15912768::DEBUG::2013-11-20 02:43:32,685::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912768::DEBUG::2013-11-20 02:43:32,685::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912768::DEBUG::2013-11-20 02:43:32,686::task::495::TaskManager.Task::(_debug) Task fd4569dc-f4ee-4175-a75e-39297c8e197c: ref 0 aborting False
>Thread-15912768::INFO::2013-11-20 02:43:32,686::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000468015670776', 'lastCheck': 1384915409.6229081, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.111585140228', 'lastCheck': 1384915409.2822199, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.1806640625', 'lastCheck': 1384915409.8253901, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000432968139648', 'lastCheck': 1384915407.623462, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:43:34,032::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912770::DEBUG::2013-11-20 02:43:34,473::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912770::INFO::2013-11-20 02:43:34,478::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912770::DEBUG::2013-11-20 02:43:34,481::task::495::TaskManager.Task::(_debug) Task 7e4bd729-208c-49a2-b1a5-194cefcffb73: moving from state init -> state preparing
>Thread-15912770::DEBUG::2013-11-20 02:43:34,486::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912770::DEBUG::2013-11-20 02:43:34,486::task::495::TaskManager.Task::(_debug) Task 7e4bd729-208c-49a2-b1a5-194cefcffb73: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912770::DEBUG::2013-11-20 02:43:34,487::task::495::TaskManager.Task::(_debug) Task 7e4bd729-208c-49a2-b1a5-194cefcffb73: moving from state preparing -> state finished
>Thread-15912770::DEBUG::2013-11-20 02:43:34,488::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912770::DEBUG::2013-11-20 02:43:34,488::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912770::DEBUG::2013-11-20 02:43:34,489::task::495::TaskManager.Task::(_debug) Task 7e4bd729-208c-49a2-b1a5-194cefcffb73: ref 0 aborting False
>Thread-15912770::INFO::2013-11-20 02:43:34,490::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912771::DEBUG::2013-11-20 02:43:34,494::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912771::INFO::2013-11-20 02:43:34,495::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912771::DEBUG::2013-11-20 02:43:34,496::task::495::TaskManager.Task::(_debug) Task 2f4ea8c3-89e6-41d7-b798-c11379916d31: moving from state init -> state preparing
>Thread-15912771::DEBUG::2013-11-20 02:43:34,497::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f0afc5fa-0110-4c13-a584-14d99e383d54`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912771::DEBUG::2013-11-20 02:43:34,497::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912771::DEBUG::2013-11-20 02:43:34,498::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912771::DEBUG::2013-11-20 02:43:34,499::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f0afc5fa-0110-4c13-a584-14d99e383d54`::Granted request
>Thread-15912771::DEBUG::2013-11-20 02:43:34,500::task::495::TaskManager.Task::(_debug) Task 2f4ea8c3-89e6-41d7-b798-c11379916d31: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912771::DEBUG::2013-11-20 02:43:34,500::task::495::TaskManager.Task::(_debug) Task 2f4ea8c3-89e6-41d7-b798-c11379916d31: ref 1 aborting False
>Thread-15912771::WARNING::2013-11-20 02:43:34,502::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912771::DEBUG::2013-11-20 02:43:34,503::task::495::TaskManager.Task::(_debug) Task 2f4ea8c3-89e6-41d7-b798-c11379916d31: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912771::DEBUG::2013-11-20 02:43:34,504::task::495::TaskManager.Task::(_debug) Task 2f4ea8c3-89e6-41d7-b798-c11379916d31: moving from state preparing -> state finished
>Thread-15912771::DEBUG::2013-11-20 02:43:34,505::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912771::DEBUG::2013-11-20 02:43:34,508::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912771::DEBUG::2013-11-20 02:43:34,510::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912771::DEBUG::2013-11-20 02:43:34,513::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912771::DEBUG::2013-11-20 02:43:34,514::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912771::DEBUG::2013-11-20 02:43:34,515::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912771::DEBUG::2013-11-20 02:43:34,517::task::495::TaskManager.Task::(_debug) Task 2f4ea8c3-89e6-41d7-b798-c11379916d31: ref 0 aborting False
>Thread-15912771::INFO::2013-11-20 02:43:34,518::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:43:34,548::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.112431 s, 9.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:43:36,557::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:43:37,118::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.076404 s, 13.4 MB/s\n'; <rc> = 0
>Thread-42840::DEBUG::2013-11-20 02:43:37,803::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 02:43:38,456::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:43:39,134::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:43:39,649::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0737767 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:43:41,664::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:43:42,197::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0866151 s, 11.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:43:44,212::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:43:44,743::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.092527 s, 11.1 MB/s\n'; <rc> = 0
>Thread-15912776::DEBUG::2013-11-20 02:43:45,885::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912776::INFO::2013-11-20 02:43:45,886::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912776::DEBUG::2013-11-20 02:43:45,886::task::495::TaskManager.Task::(_debug) Task 6deb5dbb-89fc-4472-a4fa-312a5b0b7687: moving from state init -> state preparing
>Thread-15912776::DEBUG::2013-11-20 02:43:45,887::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912776::DEBUG::2013-11-20 02:43:45,888::task::495::TaskManager.Task::(_debug) Task 6deb5dbb-89fc-4472-a4fa-312a5b0b7687: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912776::DEBUG::2013-11-20 02:43:45,888::task::495::TaskManager.Task::(_debug) Task 6deb5dbb-89fc-4472-a4fa-312a5b0b7687: moving from state preparing -> state finished
>Thread-15912776::DEBUG::2013-11-20 02:43:45,889::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912776::DEBUG::2013-11-20 02:43:45,889::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912776::DEBUG::2013-11-20 02:43:45,889::task::495::TaskManager.Task::(_debug) Task 6deb5dbb-89fc-4472-a4fa-312a5b0b7687: ref 0 aborting False
>Thread-15912776::INFO::2013-11-20 02:43:45,890::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912777::DEBUG::2013-11-20 02:43:45,894::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912777::INFO::2013-11-20 02:43:45,895::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912777::DEBUG::2013-11-20 02:43:45,896::task::495::TaskManager.Task::(_debug) Task 1cf633c4-41f5-46e3-bc12-80a24f9c0904: moving from state init -> state preparing
>Thread-15912777::DEBUG::2013-11-20 02:43:45,897::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3911d822-6a24-4798-9ae5-913598813732`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912777::DEBUG::2013-11-20 02:43:45,897::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912777::DEBUG::2013-11-20 02:43:45,898::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912777::DEBUG::2013-11-20 02:43:45,898::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3911d822-6a24-4798-9ae5-913598813732`::Granted request
>Thread-15912777::DEBUG::2013-11-20 02:43:45,899::task::495::TaskManager.Task::(_debug) Task 1cf633c4-41f5-46e3-bc12-80a24f9c0904: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912777::DEBUG::2013-11-20 02:43:45,899::task::495::TaskManager.Task::(_debug) Task 1cf633c4-41f5-46e3-bc12-80a24f9c0904: ref 1 aborting False
>Thread-15912777::WARNING::2013-11-20 02:43:45,901::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912777::DEBUG::2013-11-20 02:43:45,902::task::495::TaskManager.Task::(_debug) Task 1cf633c4-41f5-46e3-bc12-80a24f9c0904: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115764887552', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115764887552', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912777::DEBUG::2013-11-20 02:43:45,902::task::495::TaskManager.Task::(_debug) Task 1cf633c4-41f5-46e3-bc12-80a24f9c0904: moving from state preparing -> state finished
>Thread-15912777::DEBUG::2013-11-20 02:43:45,903::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912777::DEBUG::2013-11-20 02:43:45,903::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912777::DEBUG::2013-11-20 02:43:45,904::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912777::DEBUG::2013-11-20 02:43:45,904::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912777::DEBUG::2013-11-20 02:43:45,905::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912777::DEBUG::2013-11-20 02:43:45,905::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912777::DEBUG::2013-11-20 02:43:45,906::task::495::TaskManager.Task::(_debug) Task 1cf633c4-41f5-46e3-bc12-80a24f9c0904: ref 0 aborting False
>Thread-15912777::INFO::2013-11-20 02:43:45,906::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115764887552', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115764887552', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:43:46,759::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:43:47,287::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0770389 s, 13.3 MB/s\n'; <rc> = 0
>Thread-15912778::INFO::2013-11-20 02:43:48,133::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912778::DEBUG::2013-11-20 02:43:48,134::task::495::TaskManager.Task::(_debug) Task 49536ba2-14ef-4c16-b530-70c8e5e29376: moving from state init -> state preparing
>Thread-15912778::DEBUG::2013-11-20 02:43:48,134::task::495::TaskManager.Task::(_debug) Task 49536ba2-14ef-4c16-b530-70c8e5e29376: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000349044799805', 'lastCheck': 1384915419.8084481, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014169216156', 'lastCheck': 1384915419.357512, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00179100036621', 'lastCheck': 1384915420.0153861, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00040602684021', 'lastCheck': 1384915418.458756, 'valid': True, 'code': 0}}
>Thread-15912778::DEBUG::2013-11-20 02:43:48,135::task::495::TaskManager.Task::(_debug) Task 49536ba2-14ef-4c16-b530-70c8e5e29376: moving from state preparing -> state finished
>Thread-15912778::DEBUG::2013-11-20 02:43:48,136::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912778::DEBUG::2013-11-20 02:43:48,136::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912778::DEBUG::2013-11-20 02:43:48,136::task::495::TaskManager.Task::(_debug) Task 49536ba2-14ef-4c16-b530-70c8e5e29376: ref 0 aborting False
>Thread-15912778::INFO::2013-11-20 02:43:48,137::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000349044799805', 'lastCheck': 1384915419.8084481, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014169216156', 'lastCheck': 1384915419.357512, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00179100036621', 'lastCheck': 1384915420.0153861, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00040602684021', 'lastCheck': 1384915418.458756, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:43:49,302::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:43:49,831::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0743811 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:43:51,846::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:43:52,350::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0661062 s, 15.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:43:54,365::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:43:54,879::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0539989 s, 19.0 MB/s\n'; <rc> = 0
>Thread-15912782::DEBUG::2013-11-20 02:43:56,620::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912782::INFO::2013-11-20 02:43:56,624::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912782::DEBUG::2013-11-20 02:43:56,625::task::495::TaskManager.Task::(_debug) Task c670da17-46e8-41e0-8695-8e76f5306fe6: moving from state init -> state preparing
>Thread-15912782::DEBUG::2013-11-20 02:43:56,629::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912782::DEBUG::2013-11-20 02:43:56,634::task::495::TaskManager.Task::(_debug) Task c670da17-46e8-41e0-8695-8e76f5306fe6: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912782::DEBUG::2013-11-20 02:43:56,636::task::495::TaskManager.Task::(_debug) Task c670da17-46e8-41e0-8695-8e76f5306fe6: moving from state preparing -> state finished
>Thread-15912782::DEBUG::2013-11-20 02:43:56,638::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912782::DEBUG::2013-11-20 02:43:56,640::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912782::DEBUG::2013-11-20 02:43:56,642::task::495::TaskManager.Task::(_debug) Task c670da17-46e8-41e0-8695-8e76f5306fe6: ref 0 aborting False
>Thread-15912782::INFO::2013-11-20 02:43:56,643::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912783::DEBUG::2013-11-20 02:43:56,650::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912783::INFO::2013-11-20 02:43:56,651::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912783::DEBUG::2013-11-20 02:43:56,651::task::495::TaskManager.Task::(_debug) Task c311c567-9daa-4334-8290-7d48f1088e25: moving from state init -> state preparing
>Thread-15912783::DEBUG::2013-11-20 02:43:56,652::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`868aeb49-5b22-47d1-a185-e152c1e77b94`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912783::DEBUG::2013-11-20 02:43:56,653::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912783::DEBUG::2013-11-20 02:43:56,654::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912783::DEBUG::2013-11-20 02:43:56,654::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`868aeb49-5b22-47d1-a185-e152c1e77b94`::Granted request
>Thread-15912783::DEBUG::2013-11-20 02:43:56,655::task::495::TaskManager.Task::(_debug) Task c311c567-9daa-4334-8290-7d48f1088e25: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912783::DEBUG::2013-11-20 02:43:56,656::task::495::TaskManager.Task::(_debug) Task c311c567-9daa-4334-8290-7d48f1088e25: ref 1 aborting False
>Thread-15912783::WARNING::2013-11-20 02:43:56,658::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912783::DEBUG::2013-11-20 02:43:56,659::task::495::TaskManager.Task::(_debug) Task c311c567-9daa-4334-8290-7d48f1088e25: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115764887552', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115764887552', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912783::DEBUG::2013-11-20 02:43:56,660::task::495::TaskManager.Task::(_debug) Task c311c567-9daa-4334-8290-7d48f1088e25: moving from state preparing -> state finished
>Thread-15912783::DEBUG::2013-11-20 02:43:56,660::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912783::DEBUG::2013-11-20 02:43:56,661::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912783::DEBUG::2013-11-20 02:43:56,661::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912783::DEBUG::2013-11-20 02:43:56,662::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912783::DEBUG::2013-11-20 02:43:56,662::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912783::DEBUG::2013-11-20 02:43:56,663::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912783::DEBUG::2013-11-20 02:43:56,664::task::495::TaskManager.Task::(_debug) Task c311c567-9daa-4334-8290-7d48f1088e25: ref 0 aborting False
>Thread-15912783::INFO::2013-11-20 02:43:56,664::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115764887552', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115764887552', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:43:56,891::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:43:57,372::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.070034 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:43:59,387::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:43:59,929::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.070444 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:44:01,943::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 02:44:02,144::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 02:44:02,146::task::495::TaskManager.Task::(_debug) Task 806d697c-65b7-4090-9327-1435734d92ff: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 02:44:02,149::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3dcdce6b-d67c-42f1-97f7-0e69fefa57b9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 02:44:02,149::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 02:44:02,151::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 02:44:02,151::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3dcdce6b-d67c-42f1-97f7-0e69fefa57b9`::Granted request
>Thread-15912392::DEBUG::2013-11-20 02:44:02,152::task::495::TaskManager.Task::(_debug) Task 806d697c-65b7-4090-9327-1435734d92ff: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 02:44:02,153::task::495::TaskManager.Task::(_debug) Task 806d697c-65b7-4090-9327-1435734d92ff: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 02:44:02,154::task::495::TaskManager.Task::(_debug) Task 806d697c-65b7-4090-9327-1435734d92ff: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 02:44:02,154::task::495::TaskManager.Task::(_debug) Task 806d697c-65b7-4090-9327-1435734d92ff: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 02:44:02,155::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 02:44:02,155::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 02:44:02,156::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 02:44:02,157::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 02:44:02,157::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 02:44:02,158::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 02:44:02,158::task::495::TaskManager.Task::(_debug) Task 806d697c-65b7-4090-9327-1435734d92ff: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 02:44:02,159::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 02:44:02,468::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0703908 s, 14.5 MB/s\n'; <rc> = 0
>Thread-15912786::INFO::2013-11-20 02:44:04,170::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912786::DEBUG::2013-11-20 02:44:04,171::task::495::TaskManager.Task::(_debug) Task 0475393c-3481-46a6-a15c-f6a6f13056c5: moving from state init -> state preparing
>Thread-15912786::DEBUG::2013-11-20 02:44:04,171::task::495::TaskManager.Task::(_debug) Task 0475393c-3481-46a6-a15c-f6a6f13056c5: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000381946563721', 'lastCheck': 1384915440.0831151, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0015721321106', 'lastCheck': 1384915439.6194639, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00213003158569', 'lastCheck': 1384915440.0817189, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000475883483887', 'lastCheck': 1384915438.5876279, 'valid': True, 'code': 0}}
>Thread-15912786::DEBUG::2013-11-20 02:44:04,172::task::495::TaskManager.Task::(_debug) Task 0475393c-3481-46a6-a15c-f6a6f13056c5: moving from state preparing -> state finished
>Thread-15912786::DEBUG::2013-11-20 02:44:04,173::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912786::DEBUG::2013-11-20 02:44:04,173::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912786::DEBUG::2013-11-20 02:44:04,174::task::495::TaskManager.Task::(_debug) Task 0475393c-3481-46a6-a15c-f6a6f13056c5: ref 0 aborting False
>Thread-15912786::INFO::2013-11-20 02:44:04,174::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000381946563721', 'lastCheck': 1384915440.0831151, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0015721321106', 'lastCheck': 1384915439.6194639, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00213003158569', 'lastCheck': 1384915440.0817189, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000475883483887', 'lastCheck': 1384915438.5876279, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:44:04,483::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:44:05,040::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.102558 s, 10.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:44:07,055::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:44:07,577::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0766399 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15912788::DEBUG::2013-11-20 02:44:09,008::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912788::INFO::2013-11-20 02:44:09,010::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912788::DEBUG::2013-11-20 02:44:09,010::task::495::TaskManager.Task::(_debug) Task 76983ea6-c98f-493e-9570-b783d4c544fb: moving from state init -> state preparing
>Thread-15912788::DEBUG::2013-11-20 02:44:09,011::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912788::DEBUG::2013-11-20 02:44:09,012::task::495::TaskManager.Task::(_debug) Task 76983ea6-c98f-493e-9570-b783d4c544fb: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912788::DEBUG::2013-11-20 02:44:09,012::task::495::TaskManager.Task::(_debug) Task 76983ea6-c98f-493e-9570-b783d4c544fb: moving from state preparing -> state finished
>Thread-15912788::DEBUG::2013-11-20 02:44:09,013::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912788::DEBUG::2013-11-20 02:44:09,013::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912788::DEBUG::2013-11-20 02:44:09,014::task::495::TaskManager.Task::(_debug) Task 76983ea6-c98f-493e-9570-b783d4c544fb: ref 0 aborting False
>Thread-15912788::INFO::2013-11-20 02:44:09,015::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912789::DEBUG::2013-11-20 02:44:09,019::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912789::INFO::2013-11-20 02:44:09,019::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912789::DEBUG::2013-11-20 02:44:09,020::task::495::TaskManager.Task::(_debug) Task 244815f9-8550-4948-86eb-2208b92e88e5: moving from state init -> state preparing
>Thread-15912789::DEBUG::2013-11-20 02:44:09,021::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b4bceb6d-8890-46be-a78d-a3abb7d856c6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912789::DEBUG::2013-11-20 02:44:09,022::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912789::DEBUG::2013-11-20 02:44:09,023::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912789::DEBUG::2013-11-20 02:44:09,023::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b4bceb6d-8890-46be-a78d-a3abb7d856c6`::Granted request
>Thread-15912789::DEBUG::2013-11-20 02:44:09,024::task::495::TaskManager.Task::(_debug) Task 244815f9-8550-4948-86eb-2208b92e88e5: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912789::DEBUG::2013-11-20 02:44:09,025::task::495::TaskManager.Task::(_debug) Task 244815f9-8550-4948-86eb-2208b92e88e5: ref 1 aborting False
>Thread-15912789::WARNING::2013-11-20 02:44:09,027::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912789::DEBUG::2013-11-20 02:44:09,027::task::495::TaskManager.Task::(_debug) Task 244815f9-8550-4948-86eb-2208b92e88e5: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912789::DEBUG::2013-11-20 02:44:09,028::task::495::TaskManager.Task::(_debug) Task 244815f9-8550-4948-86eb-2208b92e88e5: moving from state preparing -> state finished
>Thread-15912789::DEBUG::2013-11-20 02:44:09,029::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912789::DEBUG::2013-11-20 02:44:09,029::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912789::DEBUG::2013-11-20 02:44:09,030::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912789::DEBUG::2013-11-20 02:44:09,031::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912789::DEBUG::2013-11-20 02:44:09,031::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912789::DEBUG::2013-11-20 02:44:09,032::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912789::DEBUG::2013-11-20 02:44:09,032::task::495::TaskManager.Task::(_debug) Task 244815f9-8550-4948-86eb-2208b92e88e5: ref 0 aborting False
>Thread-15912789::INFO::2013-11-20 02:44:09,033::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:44:09,592::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:44:10,023::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0469762 s, 21.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:44:12,036::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:44:12,568::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0739549 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:44:14,583::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:44:15,148::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.103527 s, 9.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:44:17,170::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:44:17,709::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0709951 s, 14.4 MB/s\n'; <rc> = 0
>Thread-15912794::DEBUG::2013-11-20 02:44:19,611::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912794::INFO::2013-11-20 02:44:19,612::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912794::DEBUG::2013-11-20 02:44:19,613::task::495::TaskManager.Task::(_debug) Task b5d7058a-b206-4b88-b089-1b1267bbc317: moving from state init -> state preparing
>Thread-15912794::DEBUG::2013-11-20 02:44:19,614::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912794::DEBUG::2013-11-20 02:44:19,614::task::495::TaskManager.Task::(_debug) Task b5d7058a-b206-4b88-b089-1b1267bbc317: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912794::DEBUG::2013-11-20 02:44:19,615::task::495::TaskManager.Task::(_debug) Task b5d7058a-b206-4b88-b089-1b1267bbc317: moving from state preparing -> state finished
>Thread-15912794::DEBUG::2013-11-20 02:44:19,615::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912794::DEBUG::2013-11-20 02:44:19,616::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912794::DEBUG::2013-11-20 02:44:19,616::task::495::TaskManager.Task::(_debug) Task b5d7058a-b206-4b88-b089-1b1267bbc317: ref 0 aborting False
>Thread-15912794::INFO::2013-11-20 02:44:19,617::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912795::DEBUG::2013-11-20 02:44:19,621::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912795::INFO::2013-11-20 02:44:19,621::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912795::DEBUG::2013-11-20 02:44:19,622::task::495::TaskManager.Task::(_debug) Task d70953cd-d9b7-48b6-880f-2487bb2377f9: moving from state init -> state preparing
>Thread-15912795::DEBUG::2013-11-20 02:44:19,623::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c087b8d5-ca5b-4b83-b877-3f21e7b4aa20`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912795::DEBUG::2013-11-20 02:44:19,624::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912795::DEBUG::2013-11-20 02:44:19,625::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912795::DEBUG::2013-11-20 02:44:19,625::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c087b8d5-ca5b-4b83-b877-3f21e7b4aa20`::Granted request
>Thread-15912795::DEBUG::2013-11-20 02:44:19,626::task::495::TaskManager.Task::(_debug) Task d70953cd-d9b7-48b6-880f-2487bb2377f9: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912795::DEBUG::2013-11-20 02:44:19,627::task::495::TaskManager.Task::(_debug) Task d70953cd-d9b7-48b6-880f-2487bb2377f9: ref 1 aborting False
>Thread-15912795::WARNING::2013-11-20 02:44:19,629::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912795::DEBUG::2013-11-20 02:44:19,629::task::495::TaskManager.Task::(_debug) Task d70953cd-d9b7-48b6-880f-2487bb2377f9: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912795::DEBUG::2013-11-20 02:44:19,630::task::495::TaskManager.Task::(_debug) Task d70953cd-d9b7-48b6-880f-2487bb2377f9: moving from state preparing -> state finished
>Thread-15912795::DEBUG::2013-11-20 02:44:19,631::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912795::DEBUG::2013-11-20 02:44:19,631::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912795::DEBUG::2013-11-20 02:44:19,632::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912795::DEBUG::2013-11-20 02:44:19,633::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912795::DEBUG::2013-11-20 02:44:19,633::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912795::DEBUG::2013-11-20 02:44:19,634::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912795::DEBUG::2013-11-20 02:44:19,634::task::495::TaskManager.Task::(_debug) Task d70953cd-d9b7-48b6-880f-2487bb2377f9: ref 0 aborting False
>Thread-15912795::INFO::2013-11-20 02:44:19,635::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:44:19,725::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:44:20,142::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0425894 s, 24.0 MB/s\n'; <rc> = 0
>Thread-15912796::INFO::2013-11-20 02:44:21,887::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912796::DEBUG::2013-11-20 02:44:21,888::task::495::TaskManager.Task::(_debug) Task 455ef442-6356-4bad-adb5-6e895eed6fdb: moving from state init -> state preparing
>Thread-15912796::DEBUG::2013-11-20 02:44:21,889::task::495::TaskManager.Task::(_debug) Task 455ef442-6356-4bad-adb5-6e895eed6fdb: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000364065170288', 'lastCheck': 1384915460.4077201, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00102591514587', 'lastCheck': 1384915459.9261141, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00135397911072', 'lastCheck': 1384915460.409332, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000445127487183', 'lastCheck': 1384915458.81552, 'valid': True, 'code': 0}}
>Thread-15912796::DEBUG::2013-11-20 02:44:21,890::task::495::TaskManager.Task::(_debug) Task 455ef442-6356-4bad-adb5-6e895eed6fdb: moving from state preparing -> state finished
>Thread-15912796::DEBUG::2013-11-20 02:44:21,890::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912796::DEBUG::2013-11-20 02:44:21,891::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912796::DEBUG::2013-11-20 02:44:21,891::task::495::TaskManager.Task::(_debug) Task 455ef442-6356-4bad-adb5-6e895eed6fdb: ref 0 aborting False
>Thread-15912796::INFO::2013-11-20 02:44:21,892::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000364065170288', 'lastCheck': 1384915460.4077201, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00102591514587', 'lastCheck': 1384915459.9261141, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00135397911072', 'lastCheck': 1384915460.409332, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000445127487183', 'lastCheck': 1384915458.81552, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:44:22,156::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:44:22,687::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0671611 s, 15.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:44:24,703::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:44:25,142::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0433918 s, 23.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:44:27,152::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:44:27,570::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0529798 s, 19.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:44:29,584::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:44:30,101::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0736466 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15912800::DEBUG::2013-11-20 02:44:30,123::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912800::INFO::2013-11-20 02:44:30,125::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912800::DEBUG::2013-11-20 02:44:30,125::task::495::TaskManager.Task::(_debug) Task d1406a35-6c0e-4b84-a917-12137c414c24: moving from state init -> state preparing
>Thread-15912800::DEBUG::2013-11-20 02:44:30,126::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912800::DEBUG::2013-11-20 02:44:30,127::task::495::TaskManager.Task::(_debug) Task d1406a35-6c0e-4b84-a917-12137c414c24: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912800::DEBUG::2013-11-20 02:44:30,127::task::495::TaskManager.Task::(_debug) Task d1406a35-6c0e-4b84-a917-12137c414c24: moving from state preparing -> state finished
>Thread-15912800::DEBUG::2013-11-20 02:44:30,128::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912800::DEBUG::2013-11-20 02:44:30,128::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912800::DEBUG::2013-11-20 02:44:30,129::task::495::TaskManager.Task::(_debug) Task d1406a35-6c0e-4b84-a917-12137c414c24: ref 0 aborting False
>Thread-15912800::INFO::2013-11-20 02:44:30,129::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912801::DEBUG::2013-11-20 02:44:30,134::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912801::INFO::2013-11-20 02:44:30,135::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912801::DEBUG::2013-11-20 02:44:30,135::task::495::TaskManager.Task::(_debug) Task 1d16eced-e816-4618-9b88-1ff6dd810b8c: moving from state init -> state preparing
>Thread-15912801::DEBUG::2013-11-20 02:44:30,136::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5392d610-c7a1-48da-a643-098270325696`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912801::DEBUG::2013-11-20 02:44:30,137::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912801::DEBUG::2013-11-20 02:44:30,138::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912801::DEBUG::2013-11-20 02:44:30,138::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5392d610-c7a1-48da-a643-098270325696`::Granted request
>Thread-15912801::DEBUG::2013-11-20 02:44:30,139::task::495::TaskManager.Task::(_debug) Task 1d16eced-e816-4618-9b88-1ff6dd810b8c: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912801::DEBUG::2013-11-20 02:44:30,140::task::495::TaskManager.Task::(_debug) Task 1d16eced-e816-4618-9b88-1ff6dd810b8c: ref 1 aborting False
>Thread-15912801::WARNING::2013-11-20 02:44:30,142::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912801::DEBUG::2013-11-20 02:44:30,143::task::495::TaskManager.Task::(_debug) Task 1d16eced-e816-4618-9b88-1ff6dd810b8c: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912801::DEBUG::2013-11-20 02:44:30,143::task::495::TaskManager.Task::(_debug) Task 1d16eced-e816-4618-9b88-1ff6dd810b8c: moving from state preparing -> state finished
>Thread-15912801::DEBUG::2013-11-20 02:44:30,144::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912801::DEBUG::2013-11-20 02:44:30,145::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912801::DEBUG::2013-11-20 02:44:30,145::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912801::DEBUG::2013-11-20 02:44:30,146::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912801::DEBUG::2013-11-20 02:44:30,146::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912801::DEBUG::2013-11-20 02:44:30,147::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912801::DEBUG::2013-11-20 02:44:30,147::task::495::TaskManager.Task::(_debug) Task 1d16eced-e816-4618-9b88-1ff6dd810b8c: ref 0 aborting False
>Thread-15912801::INFO::2013-11-20 02:44:30,148::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:44:32,116::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:44:32,639::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0728841 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:44:34,654::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:44:35,168::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.05375 s, 19.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:44:37,180::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:44:37,708::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0741235 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15912804::INFO::2013-11-20 02:44:38,487::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912804::DEBUG::2013-11-20 02:44:38,489::task::495::TaskManager.Task::(_debug) Task fff36ac9-b2af-4d4b-8f36-d356f2e43e84: moving from state init -> state preparing
>Thread-15912804::DEBUG::2013-11-20 02:44:38,489::task::495::TaskManager.Task::(_debug) Task fff36ac9-b2af-4d4b-8f36-d356f2e43e84: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000385999679565', 'lastCheck': 1384915470.421278, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0140869617462', 'lastCheck': 1384915469.954282, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00330495834351', 'lastCheck': 1384915470.4252591, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000514030456543', 'lastCheck': 1384915468.930141, 'valid': True, 'code': 0}}
>Thread-15912804::DEBUG::2013-11-20 02:44:38,490::task::495::TaskManager.Task::(_debug) Task fff36ac9-b2af-4d4b-8f36-d356f2e43e84: moving from state preparing -> state finished
>Thread-15912804::DEBUG::2013-11-20 02:44:38,491::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912804::DEBUG::2013-11-20 02:44:38,491::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912804::DEBUG::2013-11-20 02:44:38,492::task::495::TaskManager.Task::(_debug) Task fff36ac9-b2af-4d4b-8f36-d356f2e43e84: ref 0 aborting False
>Thread-15912804::INFO::2013-11-20 02:44:38,493::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000385999679565', 'lastCheck': 1384915470.421278, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0140869617462', 'lastCheck': 1384915469.954282, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00330495834351', 'lastCheck': 1384915470.4252591, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000514030456543', 'lastCheck': 1384915468.930141, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:44:39,723::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:44:40,228::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0614619 s, 16.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:44:42,241::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912806::DEBUG::2013-11-20 02:44:42,521::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912806::INFO::2013-11-20 02:44:42,523::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912806::DEBUG::2013-11-20 02:44:42,523::task::495::TaskManager.Task::(_debug) Task a10de214-05f8-4754-9e41-2684e3418bbe: moving from state init -> state preparing
>Thread-15912806::DEBUG::2013-11-20 02:44:42,525::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912806::DEBUG::2013-11-20 02:44:42,525::task::495::TaskManager.Task::(_debug) Task a10de214-05f8-4754-9e41-2684e3418bbe: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912806::DEBUG::2013-11-20 02:44:42,526::task::495::TaskManager.Task::(_debug) Task a10de214-05f8-4754-9e41-2684e3418bbe: moving from state preparing -> state finished
>Thread-15912806::DEBUG::2013-11-20 02:44:42,526::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912806::DEBUG::2013-11-20 02:44:42,527::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912806::DEBUG::2013-11-20 02:44:42,527::task::495::TaskManager.Task::(_debug) Task a10de214-05f8-4754-9e41-2684e3418bbe: ref 0 aborting False
>Thread-15912806::INFO::2013-11-20 02:44:42,528::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912807::DEBUG::2013-11-20 02:44:42,534::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912807::INFO::2013-11-20 02:44:42,535::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912807::DEBUG::2013-11-20 02:44:42,535::task::495::TaskManager.Task::(_debug) Task 2d1a5657-268b-4fc0-a4e4-6b9e07ca9baf: moving from state init -> state preparing
>Thread-15912807::DEBUG::2013-11-20 02:44:42,537::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bf095424-05d9-4d2b-8947-f40842145c87`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912807::DEBUG::2013-11-20 02:44:42,537::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912807::DEBUG::2013-11-20 02:44:42,538::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912807::DEBUG::2013-11-20 02:44:42,539::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bf095424-05d9-4d2b-8947-f40842145c87`::Granted request
>Thread-15912807::DEBUG::2013-11-20 02:44:42,540::task::495::TaskManager.Task::(_debug) Task 2d1a5657-268b-4fc0-a4e4-6b9e07ca9baf: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912807::DEBUG::2013-11-20 02:44:42,540::task::495::TaskManager.Task::(_debug) Task 2d1a5657-268b-4fc0-a4e4-6b9e07ca9baf: ref 1 aborting False
>Thread-15912807::WARNING::2013-11-20 02:44:42,543::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912807::DEBUG::2013-11-20 02:44:42,544::task::495::TaskManager.Task::(_debug) Task 2d1a5657-268b-4fc0-a4e4-6b9e07ca9baf: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912807::DEBUG::2013-11-20 02:44:42,545::task::495::TaskManager.Task::(_debug) Task 2d1a5657-268b-4fc0-a4e4-6b9e07ca9baf: moving from state preparing -> state finished
>Thread-15912807::DEBUG::2013-11-20 02:44:42,545::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912807::DEBUG::2013-11-20 02:44:42,546::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912807::DEBUG::2013-11-20 02:44:42,547::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912807::DEBUG::2013-11-20 02:44:42,547::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912807::DEBUG::2013-11-20 02:44:42,548::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912807::DEBUG::2013-11-20 02:44:42,548::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912807::DEBUG::2013-11-20 02:44:42,549::task::495::TaskManager.Task::(_debug) Task 2d1a5657-268b-4fc0-a4e4-6b9e07ca9baf: ref 0 aborting False
>Thread-15912807::INFO::2013-11-20 02:44:42,550::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:44:42,777::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0701455 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:44:44,792::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:44:45,336::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0703028 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:44:47,350::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:44:47,859::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0696081 s, 14.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:44:49,873::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:44:50,399::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0743392 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:44:52,414::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:44:52,908::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0543772 s, 18.8 MB/s\n'; <rc> = 0
>Thread-15912812::DEBUG::2013-11-20 02:44:54,646::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912812::INFO::2013-11-20 02:44:54,647::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912812::DEBUG::2013-11-20 02:44:54,648::task::495::TaskManager.Task::(_debug) Task 8ac5f586-60d3-458f-bd99-fe9a63dea8d1: moving from state init -> state preparing
>Thread-15912812::DEBUG::2013-11-20 02:44:54,649::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912812::DEBUG::2013-11-20 02:44:54,649::task::495::TaskManager.Task::(_debug) Task 8ac5f586-60d3-458f-bd99-fe9a63dea8d1: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912812::DEBUG::2013-11-20 02:44:54,650::task::495::TaskManager.Task::(_debug) Task 8ac5f586-60d3-458f-bd99-fe9a63dea8d1: moving from state preparing -> state finished
>Thread-15912812::DEBUG::2013-11-20 02:44:54,650::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912812::DEBUG::2013-11-20 02:44:54,651::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912812::DEBUG::2013-11-20 02:44:54,651::task::495::TaskManager.Task::(_debug) Task 8ac5f586-60d3-458f-bd99-fe9a63dea8d1: ref 0 aborting False
>Thread-15912812::INFO::2013-11-20 02:44:54,652::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912813::DEBUG::2013-11-20 02:44:54,656::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912813::INFO::2013-11-20 02:44:54,657::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912813::DEBUG::2013-11-20 02:44:54,658::task::495::TaskManager.Task::(_debug) Task 15661c7e-0cd9-460d-94f3-71e7faf16f77: moving from state init -> state preparing
>Thread-15912813::DEBUG::2013-11-20 02:44:54,659::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8f162de4-5dc9-4ba8-adcb-1387e45aa826`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912813::DEBUG::2013-11-20 02:44:54,660::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912813::DEBUG::2013-11-20 02:44:54,660::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912813::DEBUG::2013-11-20 02:44:54,661::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8f162de4-5dc9-4ba8-adcb-1387e45aa826`::Granted request
>Thread-15912813::DEBUG::2013-11-20 02:44:54,662::task::495::TaskManager.Task::(_debug) Task 15661c7e-0cd9-460d-94f3-71e7faf16f77: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912813::DEBUG::2013-11-20 02:44:54,662::task::495::TaskManager.Task::(_debug) Task 15661c7e-0cd9-460d-94f3-71e7faf16f77: ref 1 aborting False
>Thread-15912813::WARNING::2013-11-20 02:44:54,664::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912813::DEBUG::2013-11-20 02:44:54,665::task::495::TaskManager.Task::(_debug) Task 15661c7e-0cd9-460d-94f3-71e7faf16f77: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912813::DEBUG::2013-11-20 02:44:54,666::task::495::TaskManager.Task::(_debug) Task 15661c7e-0cd9-460d-94f3-71e7faf16f77: moving from state preparing -> state finished
>Thread-15912813::DEBUG::2013-11-20 02:44:54,666::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912813::DEBUG::2013-11-20 02:44:54,667::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912813::DEBUG::2013-11-20 02:44:54,668::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912813::DEBUG::2013-11-20 02:44:54,668::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912813::DEBUG::2013-11-20 02:44:54,669::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912813::DEBUG::2013-11-20 02:44:54,669::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912813::DEBUG::2013-11-20 02:44:54,670::task::495::TaskManager.Task::(_debug) Task 15661c7e-0cd9-460d-94f3-71e7faf16f77: ref 0 aborting False
>Thread-15912813::INFO::2013-11-20 02:44:54,671::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:44:54,918::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:44:55,367::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0484985 s, 21.1 MB/s\n'; <rc> = 0
>Thread-15912814::INFO::2013-11-20 02:44:55,419::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912814::DEBUG::2013-11-20 02:44:55,419::task::495::TaskManager.Task::(_debug) Task ebf84b1c-dbec-4378-bc66-2d9ca93884d7: moving from state init -> state preparing
>Thread-15912814::DEBUG::2013-11-20 02:44:55,420::task::495::TaskManager.Task::(_debug) Task ebf84b1c-dbec-4378-bc66-2d9ca93884d7: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00062108039856', 'lastCheck': 1384915490.5634999, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00124907493591', 'lastCheck': 1384915490.1269619, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00130486488342', 'lastCheck': 1384915490.5643971, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000457048416138', 'lastCheck': 1384915489.0234511, 'valid': True, 'code': 0}}
>Thread-15912814::DEBUG::2013-11-20 02:44:55,421::task::495::TaskManager.Task::(_debug) Task ebf84b1c-dbec-4378-bc66-2d9ca93884d7: moving from state preparing -> state finished
>Thread-15912814::DEBUG::2013-11-20 02:44:55,422::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912814::DEBUG::2013-11-20 02:44:55,422::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912814::DEBUG::2013-11-20 02:44:55,423::task::495::TaskManager.Task::(_debug) Task ebf84b1c-dbec-4378-bc66-2d9ca93884d7: ref 0 aborting False
>Thread-15912814::INFO::2013-11-20 02:44:55,423::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00062108039856', 'lastCheck': 1384915490.5634999, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00124907493591', 'lastCheck': 1384915490.1269619, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00130486488342', 'lastCheck': 1384915490.5643971, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000457048416138', 'lastCheck': 1384915489.0234511, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:44:57,376::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:44:57,849::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0445948 s, 23.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:44:59,862::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:45:00,361::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0698657 s, 14.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:45:02,376::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 02:45:02,653::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 02:45:02,654::task::495::TaskManager.Task::(_debug) Task fd65ae80-5fe9-43ed-b4c0-5fd3a4351d68: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 02:45:02,657::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`99a75870-310e-4ce9-8800-24783d13817c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 02:45:02,657::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 02:45:02,658::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 02:45:02,659::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`99a75870-310e-4ce9-8800-24783d13817c`::Granted request
>Thread-15912392::DEBUG::2013-11-20 02:45:02,660::task::495::TaskManager.Task::(_debug) Task fd65ae80-5fe9-43ed-b4c0-5fd3a4351d68: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 02:45:02,661::task::495::TaskManager.Task::(_debug) Task fd65ae80-5fe9-43ed-b4c0-5fd3a4351d68: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 02:45:02,661::task::495::TaskManager.Task::(_debug) Task fd65ae80-5fe9-43ed-b4c0-5fd3a4351d68: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 02:45:02,662::task::495::TaskManager.Task::(_debug) Task fd65ae80-5fe9-43ed-b4c0-5fd3a4351d68: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 02:45:02,663::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 02:45:02,663::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 02:45:02,664::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 02:45:02,665::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 02:45:02,665::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 02:45:02,666::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 02:45:02,666::task::495::TaskManager.Task::(_debug) Task fd65ae80-5fe9-43ed-b4c0-5fd3a4351d68: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 02:45:02,667::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 02:45:02,915::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.066473 s, 15.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:45:04,925::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:45:05,416::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.054498 s, 18.8 MB/s\n'; <rc> = 0
>Thread-15912819::DEBUG::2013-11-20 02:45:05,753::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912819::INFO::2013-11-20 02:45:05,754::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912819::DEBUG::2013-11-20 02:45:05,755::task::495::TaskManager.Task::(_debug) Task 6fe300fc-19c5-4bb6-91d5-379e67321ba0: moving from state init -> state preparing
>Thread-15912819::DEBUG::2013-11-20 02:45:05,755::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912819::DEBUG::2013-11-20 02:45:05,756::task::495::TaskManager.Task::(_debug) Task 6fe300fc-19c5-4bb6-91d5-379e67321ba0: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912819::DEBUG::2013-11-20 02:45:05,756::task::495::TaskManager.Task::(_debug) Task 6fe300fc-19c5-4bb6-91d5-379e67321ba0: moving from state preparing -> state finished
>Thread-15912819::DEBUG::2013-11-20 02:45:05,756::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912819::DEBUG::2013-11-20 02:45:05,757::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912819::DEBUG::2013-11-20 02:45:05,757::task::495::TaskManager.Task::(_debug) Task 6fe300fc-19c5-4bb6-91d5-379e67321ba0: ref 0 aborting False
>Thread-15912819::INFO::2013-11-20 02:45:05,757::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912820::DEBUG::2013-11-20 02:45:05,762::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912820::INFO::2013-11-20 02:45:05,763::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912820::DEBUG::2013-11-20 02:45:05,763::task::495::TaskManager.Task::(_debug) Task 2ca042a2-9a17-4291-a173-b52405bdbbbc: moving from state init -> state preparing
>Thread-15912820::DEBUG::2013-11-20 02:45:05,765::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a64aad69-a16b-4448-a3c3-5b4365d0c809`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912820::DEBUG::2013-11-20 02:45:05,765::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912820::DEBUG::2013-11-20 02:45:05,766::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912820::DEBUG::2013-11-20 02:45:05,766::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a64aad69-a16b-4448-a3c3-5b4365d0c809`::Granted request
>Thread-15912820::DEBUG::2013-11-20 02:45:05,767::task::495::TaskManager.Task::(_debug) Task 2ca042a2-9a17-4291-a173-b52405bdbbbc: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912820::DEBUG::2013-11-20 02:45:05,768::task::495::TaskManager.Task::(_debug) Task 2ca042a2-9a17-4291-a173-b52405bdbbbc: ref 1 aborting False
>Thread-15912820::WARNING::2013-11-20 02:45:05,770::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912820::DEBUG::2013-11-20 02:45:05,770::task::495::TaskManager.Task::(_debug) Task 2ca042a2-9a17-4291-a173-b52405bdbbbc: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912820::DEBUG::2013-11-20 02:45:05,771::task::495::TaskManager.Task::(_debug) Task 2ca042a2-9a17-4291-a173-b52405bdbbbc: moving from state preparing -> state finished
>Thread-15912820::DEBUG::2013-11-20 02:45:05,772::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912820::DEBUG::2013-11-20 02:45:05,772::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912820::DEBUG::2013-11-20 02:45:05,773::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912820::DEBUG::2013-11-20 02:45:05,773::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912820::DEBUG::2013-11-20 02:45:05,774::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912820::DEBUG::2013-11-20 02:45:05,774::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912820::DEBUG::2013-11-20 02:45:05,775::task::495::TaskManager.Task::(_debug) Task 2ca042a2-9a17-4291-a173-b52405bdbbbc: ref 0 aborting False
>Thread-15912820::INFO::2013-11-20 02:45:05,775::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:45:07,426::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:45:07,937::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0666421 s, 15.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:45:09,952::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:45:10,459::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0742397 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15912822::INFO::2013-11-20 02:45:11,241::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912822::DEBUG::2013-11-20 02:45:11,242::task::495::TaskManager.Task::(_debug) Task 4b3bb88b-8373-42ce-a762-493d268f5a8f: moving from state init -> state preparing
>Thread-15912822::DEBUG::2013-11-20 02:45:11,243::task::495::TaskManager.Task::(_debug) Task 4b3bb88b-8373-42ce-a762-493d268f5a8f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000240802764893', 'lastCheck': 1384915510.6686139, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00186586380005', 'lastCheck': 1384915510.182543, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.001296043396', 'lastCheck': 1384915510.6710401, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000378847122192', 'lastCheck': 1384915509.1792741, 'valid': True, 'code': 0}}
>Thread-15912822::DEBUG::2013-11-20 02:45:11,244::task::495::TaskManager.Task::(_debug) Task 4b3bb88b-8373-42ce-a762-493d268f5a8f: moving from state preparing -> state finished
>Thread-15912822::DEBUG::2013-11-20 02:45:11,244::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912822::DEBUG::2013-11-20 02:45:11,245::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912822::DEBUG::2013-11-20 02:45:11,245::task::495::TaskManager.Task::(_debug) Task 4b3bb88b-8373-42ce-a762-493d268f5a8f: ref 0 aborting False
>Thread-15912822::INFO::2013-11-20 02:45:11,246::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000240802764893', 'lastCheck': 1384915510.6686139, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00186586380005', 'lastCheck': 1384915510.182543, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.001296043396', 'lastCheck': 1384915510.6710401, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000378847122192', 'lastCheck': 1384915509.1792741, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:45:12,474::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:45:12,998::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0745649 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:45:15,012::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:45:15,501::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.073274 s, 14.0 MB/s\n'; <rc> = 0
>Thread-15912825::DEBUG::2013-11-20 02:45:17,103::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912825::INFO::2013-11-20 02:45:17,104::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912825::DEBUG::2013-11-20 02:45:17,105::task::495::TaskManager.Task::(_debug) Task b6c0c97c-554f-46ac-8e3e-adc7c74f49ca: moving from state init -> state preparing
>Thread-15912825::DEBUG::2013-11-20 02:45:17,106::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912825::DEBUG::2013-11-20 02:45:17,106::task::495::TaskManager.Task::(_debug) Task b6c0c97c-554f-46ac-8e3e-adc7c74f49ca: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912825::DEBUG::2013-11-20 02:45:17,107::task::495::TaskManager.Task::(_debug) Task b6c0c97c-554f-46ac-8e3e-adc7c74f49ca: moving from state preparing -> state finished
>Thread-15912825::DEBUG::2013-11-20 02:45:17,108::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912825::DEBUG::2013-11-20 02:45:17,108::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912825::DEBUG::2013-11-20 02:45:17,109::task::495::TaskManager.Task::(_debug) Task b6c0c97c-554f-46ac-8e3e-adc7c74f49ca: ref 0 aborting False
>Thread-15912825::INFO::2013-11-20 02:45:17,109::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912826::DEBUG::2013-11-20 02:45:17,114::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912826::INFO::2013-11-20 02:45:17,114::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912826::DEBUG::2013-11-20 02:45:17,115::task::495::TaskManager.Task::(_debug) Task 56604f22-46e8-4335-b442-5d6316512ce5: moving from state init -> state preparing
>Thread-15912826::DEBUG::2013-11-20 02:45:17,116::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8241a27d-0ded-4024-8143-033b8d086c78`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912826::DEBUG::2013-11-20 02:45:17,117::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912826::DEBUG::2013-11-20 02:45:17,118::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912826::DEBUG::2013-11-20 02:45:17,118::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8241a27d-0ded-4024-8143-033b8d086c78`::Granted request
>Thread-15912826::DEBUG::2013-11-20 02:45:17,119::task::495::TaskManager.Task::(_debug) Task 56604f22-46e8-4335-b442-5d6316512ce5: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912826::DEBUG::2013-11-20 02:45:17,120::task::495::TaskManager.Task::(_debug) Task 56604f22-46e8-4335-b442-5d6316512ce5: ref 1 aborting False
>Thread-15912826::WARNING::2013-11-20 02:45:17,122::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912826::DEBUG::2013-11-20 02:45:17,122::task::495::TaskManager.Task::(_debug) Task 56604f22-46e8-4335-b442-5d6316512ce5: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912826::DEBUG::2013-11-20 02:45:17,123::task::495::TaskManager.Task::(_debug) Task 56604f22-46e8-4335-b442-5d6316512ce5: moving from state preparing -> state finished
>Thread-15912826::DEBUG::2013-11-20 02:45:17,124::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912826::DEBUG::2013-11-20 02:45:17,124::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912826::DEBUG::2013-11-20 02:45:17,125::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912826::DEBUG::2013-11-20 02:45:17,126::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912826::DEBUG::2013-11-20 02:45:17,126::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912826::DEBUG::2013-11-20 02:45:17,127::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912826::DEBUG::2013-11-20 02:45:17,127::task::495::TaskManager.Task::(_debug) Task 56604f22-46e8-4335-b442-5d6316512ce5: ref 0 aborting False
>Thread-15912826::INFO::2013-11-20 02:45:17,128::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:45:17,524::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:45:18,111::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0616165 s, 16.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:45:20,133::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:45:20,637::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0538786 s, 19.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:45:22,648::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:45:23,141::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0675249 s, 15.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:45:25,156::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:45:25,695::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.07556 s, 13.6 MB/s\n'; <rc> = 0
>Thread-15912830::INFO::2013-11-20 02:45:27,387::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912830::DEBUG::2013-11-20 02:45:27,388::task::495::TaskManager.Task::(_debug) Task fcc56de3-1a38-4504-9296-95c2d91b5acd: moving from state init -> state preparing
>Thread-15912830::DEBUG::2013-11-20 02:45:27,389::task::495::TaskManager.Task::(_debug) Task fcc56de3-1a38-4504-9296-95c2d91b5acd: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000333070755005', 'lastCheck': 1384915520.758769, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00108504295349', 'lastCheck': 1384915520.4330201, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000983953475952', 'lastCheck': 1384915520.7590971, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000549077987671', 'lastCheck': 1384915519.200763, 'valid': True, 'code': 0}}
>Thread-15912830::DEBUG::2013-11-20 02:45:27,389::task::495::TaskManager.Task::(_debug) Task fcc56de3-1a38-4504-9296-95c2d91b5acd: moving from state preparing -> state finished
>Thread-15912830::DEBUG::2013-11-20 02:45:27,389::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912830::DEBUG::2013-11-20 02:45:27,390::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912830::DEBUG::2013-11-20 02:45:27,390::task::495::TaskManager.Task::(_debug) Task fcc56de3-1a38-4504-9296-95c2d91b5acd: ref 0 aborting False
>Thread-15912830::INFO::2013-11-20 02:45:27,390::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000333070755005', 'lastCheck': 1384915520.758769, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00108504295349', 'lastCheck': 1384915520.4330201, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000983953475952', 'lastCheck': 1384915520.7590971, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000549077987671', 'lastCheck': 1384915519.200763, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:45:27,711::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:45:28,292::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0788565 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:45:30,309::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912832::DEBUG::2013-11-20 02:45:30,678::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912832::INFO::2013-11-20 02:45:30,680::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912832::DEBUG::2013-11-20 02:45:30,680::task::495::TaskManager.Task::(_debug) Task 8032906d-6447-4863-8ad2-38653c12ca6a: moving from state init -> state preparing
>Thread-15912832::DEBUG::2013-11-20 02:45:30,681::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912832::DEBUG::2013-11-20 02:45:30,682::task::495::TaskManager.Task::(_debug) Task 8032906d-6447-4863-8ad2-38653c12ca6a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912832::DEBUG::2013-11-20 02:45:30,683::task::495::TaskManager.Task::(_debug) Task 8032906d-6447-4863-8ad2-38653c12ca6a: moving from state preparing -> state finished
>Thread-15912832::DEBUG::2013-11-20 02:45:30,684::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912832::DEBUG::2013-11-20 02:45:30,684::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912832::DEBUG::2013-11-20 02:45:30,685::task::495::TaskManager.Task::(_debug) Task 8032906d-6447-4863-8ad2-38653c12ca6a: ref 0 aborting False
>Thread-15912832::INFO::2013-11-20 02:45:30,686::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912833::DEBUG::2013-11-20 02:45:30,691::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912833::INFO::2013-11-20 02:45:30,693::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912833::DEBUG::2013-11-20 02:45:30,693::task::495::TaskManager.Task::(_debug) Task 191b9b58-ffe8-4f7f-b1f2-47cd5d630f86: moving from state init -> state preparing
>Thread-15912833::DEBUG::2013-11-20 02:45:30,695::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`36bac967-ee27-4b1f-8b38-987c57194668`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912833::DEBUG::2013-11-20 02:45:30,696::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912833::DEBUG::2013-11-20 02:45:30,697::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912833::DEBUG::2013-11-20 02:45:30,698::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`36bac967-ee27-4b1f-8b38-987c57194668`::Granted request
>Thread-15912833::DEBUG::2013-11-20 02:45:30,699::task::495::TaskManager.Task::(_debug) Task 191b9b58-ffe8-4f7f-b1f2-47cd5d630f86: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912833::DEBUG::2013-11-20 02:45:30,700::task::495::TaskManager.Task::(_debug) Task 191b9b58-ffe8-4f7f-b1f2-47cd5d630f86: ref 1 aborting False
>Thread-15912833::WARNING::2013-11-20 02:45:30,706::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912833::DEBUG::2013-11-20 02:45:30,710::task::495::TaskManager.Task::(_debug) Task 191b9b58-ffe8-4f7f-b1f2-47cd5d630f86: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912833::DEBUG::2013-11-20 02:45:30,715::task::495::TaskManager.Task::(_debug) Task 191b9b58-ffe8-4f7f-b1f2-47cd5d630f86: moving from state preparing -> state finished
>Thread-15912833::DEBUG::2013-11-20 02:45:30,716::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912833::DEBUG::2013-11-20 02:45:30,716::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912833::DEBUG::2013-11-20 02:45:30,717::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912833::DEBUG::2013-11-20 02:45:30,718::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912833::DEBUG::2013-11-20 02:45:30,718::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912833::DEBUG::2013-11-20 02:45:30,718::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912833::DEBUG::2013-11-20 02:45:30,719::task::495::TaskManager.Task::(_debug) Task 191b9b58-ffe8-4f7f-b1f2-47cd5d630f86: ref 0 aborting False
>Thread-15912833::INFO::2013-11-20 02:45:30,720::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:45:30,847::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0739534 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:45:32,861::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:45:33,371::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0732209 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:45:35,387::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:45:35,830::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0467744 s, 21.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:45:37,841::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:45:38,364::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0726019 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:45:40,379::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:45:40,895::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0723493 s, 14.2 MB/s\n'; <rc> = 0
>Thread-15912837::DEBUG::2013-11-20 02:45:41,217::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912837::INFO::2013-11-20 02:45:41,219::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912837::DEBUG::2013-11-20 02:45:41,219::task::495::TaskManager.Task::(_debug) Task 971cc26c-f17f-4e4b-9bd3-6e4b07d3c897: moving from state init -> state preparing
>Thread-15912837::DEBUG::2013-11-20 02:45:41,220::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912837::DEBUG::2013-11-20 02:45:41,220::task::495::TaskManager.Task::(_debug) Task 971cc26c-f17f-4e4b-9bd3-6e4b07d3c897: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912837::DEBUG::2013-11-20 02:45:41,220::task::495::TaskManager.Task::(_debug) Task 971cc26c-f17f-4e4b-9bd3-6e4b07d3c897: moving from state preparing -> state finished
>Thread-15912837::DEBUG::2013-11-20 02:45:41,221::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912837::DEBUG::2013-11-20 02:45:41,221::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912837::DEBUG::2013-11-20 02:45:41,221::task::495::TaskManager.Task::(_debug) Task 971cc26c-f17f-4e4b-9bd3-6e4b07d3c897: ref 0 aborting False
>Thread-15912837::INFO::2013-11-20 02:45:41,222::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912838::DEBUG::2013-11-20 02:45:41,226::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912838::INFO::2013-11-20 02:45:41,227::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912838::DEBUG::2013-11-20 02:45:41,227::task::495::TaskManager.Task::(_debug) Task 60bab87f-d26c-49ed-abd4-c76dbca39428: moving from state init -> state preparing
>Thread-15912838::DEBUG::2013-11-20 02:45:41,228::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c3089f57-078b-46c4-9d64-874da9006894`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912838::DEBUG::2013-11-20 02:45:41,229::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912838::DEBUG::2013-11-20 02:45:41,229::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912838::DEBUG::2013-11-20 02:45:41,230::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c3089f57-078b-46c4-9d64-874da9006894`::Granted request
>Thread-15912838::DEBUG::2013-11-20 02:45:41,230::task::495::TaskManager.Task::(_debug) Task 60bab87f-d26c-49ed-abd4-c76dbca39428: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912838::DEBUG::2013-11-20 02:45:41,231::task::495::TaskManager.Task::(_debug) Task 60bab87f-d26c-49ed-abd4-c76dbca39428: ref 1 aborting False
>Thread-15912838::WARNING::2013-11-20 02:45:41,233::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912838::DEBUG::2013-11-20 02:45:41,234::task::495::TaskManager.Task::(_debug) Task 60bab87f-d26c-49ed-abd4-c76dbca39428: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912838::DEBUG::2013-11-20 02:45:41,235::task::495::TaskManager.Task::(_debug) Task 60bab87f-d26c-49ed-abd4-c76dbca39428: moving from state preparing -> state finished
>Thread-15912838::DEBUG::2013-11-20 02:45:41,236::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912838::DEBUG::2013-11-20 02:45:41,236::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912838::DEBUG::2013-11-20 02:45:41,237::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912838::DEBUG::2013-11-20 02:45:41,238::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912838::DEBUG::2013-11-20 02:45:41,238::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912838::DEBUG::2013-11-20 02:45:41,239::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912838::DEBUG::2013-11-20 02:45:41,239::task::495::TaskManager.Task::(_debug) Task 60bab87f-d26c-49ed-abd4-c76dbca39428: ref 0 aborting False
>Thread-15912838::INFO::2013-11-20 02:45:41,240::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:45:42,910::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:45:43,346::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0469231 s, 21.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:45:45,357::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912840::INFO::2013-11-20 02:45:45,637::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912840::DEBUG::2013-11-20 02:45:45,638::task::495::TaskManager.Task::(_debug) Task 41db289e-df1d-4ee1-9075-2386718bef6c: moving from state init -> state preparing
>Thread-15912840::DEBUG::2013-11-20 02:45:45,639::task::495::TaskManager.Task::(_debug) Task 41db289e-df1d-4ee1-9075-2386718bef6c: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000832080841064', 'lastCheck': 1384915541.0748219, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00128793716431', 'lastCheck': 1384915540.629492, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00155401229858', 'lastCheck': 1384915541.0769689, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000463008880615', 'lastCheck': 1384915539.6233411, 'valid': True, 'code': 0}}
>Thread-15912840::DEBUG::2013-11-20 02:45:45,640::task::495::TaskManager.Task::(_debug) Task 41db289e-df1d-4ee1-9075-2386718bef6c: moving from state preparing -> state finished
>Thread-15912840::DEBUG::2013-11-20 02:45:45,641::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912840::DEBUG::2013-11-20 02:45:45,641::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912840::DEBUG::2013-11-20 02:45:45,642::task::495::TaskManager.Task::(_debug) Task 41db289e-df1d-4ee1-9075-2386718bef6c: ref 0 aborting False
>Thread-15912840::INFO::2013-11-20 02:45:45,642::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000832080841064', 'lastCheck': 1384915541.0748219, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00128793716431', 'lastCheck': 1384915540.629492, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00155401229858', 'lastCheck': 1384915541.0769689, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000463008880615', 'lastCheck': 1384915539.6233411, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:45:45,894::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0748896 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:45:47,909::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:45:48,460::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0734297 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:45:50,475::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:45:50,994::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0728742 s, 14.1 MB/s\n'; <rc> = 0
>Thread-15912843::DEBUG::2013-11-20 02:45:52,949::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912843::INFO::2013-11-20 02:45:52,950::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912843::DEBUG::2013-11-20 02:45:52,951::task::495::TaskManager.Task::(_debug) Task 9528f3ef-fc42-4afd-ad88-8c266d7b767e: moving from state init -> state preparing
>Thread-15912843::DEBUG::2013-11-20 02:45:52,952::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912843::DEBUG::2013-11-20 02:45:52,952::task::495::TaskManager.Task::(_debug) Task 9528f3ef-fc42-4afd-ad88-8c266d7b767e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912843::DEBUG::2013-11-20 02:45:52,953::task::495::TaskManager.Task::(_debug) Task 9528f3ef-fc42-4afd-ad88-8c266d7b767e: moving from state preparing -> state finished
>Thread-15912843::DEBUG::2013-11-20 02:45:52,953::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912843::DEBUG::2013-11-20 02:45:52,954::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912843::DEBUG::2013-11-20 02:45:52,954::task::495::TaskManager.Task::(_debug) Task 9528f3ef-fc42-4afd-ad88-8c266d7b767e: ref 0 aborting False
>Thread-15912843::INFO::2013-11-20 02:45:52,955::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912844::DEBUG::2013-11-20 02:45:52,959::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912844::INFO::2013-11-20 02:45:52,960::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912844::DEBUG::2013-11-20 02:45:52,961::task::495::TaskManager.Task::(_debug) Task d55cb8b0-1a3a-4c36-85c4-abc21d84e545: moving from state init -> state preparing
>Thread-15912844::DEBUG::2013-11-20 02:45:52,962::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0dd5a9ac-465d-4ab4-b7c1-a82de7661ee3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912844::DEBUG::2013-11-20 02:45:52,963::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912844::DEBUG::2013-11-20 02:45:52,963::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912844::DEBUG::2013-11-20 02:45:52,964::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0dd5a9ac-465d-4ab4-b7c1-a82de7661ee3`::Granted request
>Thread-15912844::DEBUG::2013-11-20 02:45:52,965::task::495::TaskManager.Task::(_debug) Task d55cb8b0-1a3a-4c36-85c4-abc21d84e545: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912844::DEBUG::2013-11-20 02:45:52,965::task::495::TaskManager.Task::(_debug) Task d55cb8b0-1a3a-4c36-85c4-abc21d84e545: ref 1 aborting False
>Thread-15912844::WARNING::2013-11-20 02:45:52,967::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912844::DEBUG::2013-11-20 02:45:52,968::task::495::TaskManager.Task::(_debug) Task d55cb8b0-1a3a-4c36-85c4-abc21d84e545: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912844::DEBUG::2013-11-20 02:45:52,969::task::495::TaskManager.Task::(_debug) Task d55cb8b0-1a3a-4c36-85c4-abc21d84e545: moving from state preparing -> state finished
>Thread-15912844::DEBUG::2013-11-20 02:45:52,970::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912844::DEBUG::2013-11-20 02:45:52,970::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912844::DEBUG::2013-11-20 02:45:52,971::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912844::DEBUG::2013-11-20 02:45:52,971::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912844::DEBUG::2013-11-20 02:45:52,972::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912844::DEBUG::2013-11-20 02:45:52,973::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912844::DEBUG::2013-11-20 02:45:52,973::task::495::TaskManager.Task::(_debug) Task d55cb8b0-1a3a-4c36-85c4-abc21d84e545: ref 0 aborting False
>Thread-15912844::INFO::2013-11-20 02:45:52,974::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:45:53,010::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:45:53,546::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0734219 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:45:55,561::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:45:56,108::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.075483 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:45:58,122::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:45:58,665::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0729437 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:46:00,680::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:46:01,203::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0734643 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:46:03,219::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 02:46:03,442::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 02:46:03,444::task::495::TaskManager.Task::(_debug) Task b8ebbdd8-e1d8-4a45-b9b2-8371a1dab171: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 02:46:03,446::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8db79504-b786-4f7b-991d-a769b8018b65`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 02:46:03,446::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 02:46:03,447::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 02:46:03,448::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8db79504-b786-4f7b-991d-a769b8018b65`::Granted request
>Thread-15912392::DEBUG::2013-11-20 02:46:03,449::task::495::TaskManager.Task::(_debug) Task b8ebbdd8-e1d8-4a45-b9b2-8371a1dab171: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 02:46:03,450::task::495::TaskManager.Task::(_debug) Task b8ebbdd8-e1d8-4a45-b9b2-8371a1dab171: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 02:46:03,450::task::495::TaskManager.Task::(_debug) Task b8ebbdd8-e1d8-4a45-b9b2-8371a1dab171: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 02:46:03,451::task::495::TaskManager.Task::(_debug) Task b8ebbdd8-e1d8-4a45-b9b2-8371a1dab171: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 02:46:03,452::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 02:46:03,452::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 02:46:03,453::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 02:46:03,453::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 02:46:03,454::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 02:46:03,454::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 02:46:03,455::task::495::TaskManager.Task::(_debug) Task b8ebbdd8-e1d8-4a45-b9b2-8371a1dab171: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 02:46:03,455::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 02:46:03,742::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0746276 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15912848::INFO::2013-11-20 02:46:04,428::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912848::DEBUG::2013-11-20 02:46:04,429::task::495::TaskManager.Task::(_debug) Task ca35a9b1-983c-4a05-b562-115bfc9a531c: moving from state init -> state preparing
>Thread-15912848::DEBUG::2013-11-20 02:46:04,429::task::495::TaskManager.Task::(_debug) Task ca35a9b1-983c-4a05-b562-115bfc9a531c: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000766038894653', 'lastCheck': 1384915561.37098, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00126099586487', 'lastCheck': 1384915560.9136021, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00150203704834', 'lastCheck': 1384915561.3631539, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000483989715576', 'lastCheck': 1384915559.821532, 'valid': True, 'code': 0}}
>Thread-15912848::DEBUG::2013-11-20 02:46:04,430::task::495::TaskManager.Task::(_debug) Task ca35a9b1-983c-4a05-b562-115bfc9a531c: moving from state preparing -> state finished
>Thread-15912848::DEBUG::2013-11-20 02:46:04,431::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912848::DEBUG::2013-11-20 02:46:04,431::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912848::DEBUG::2013-11-20 02:46:04,432::task::495::TaskManager.Task::(_debug) Task ca35a9b1-983c-4a05-b562-115bfc9a531c: ref 0 aborting False
>Thread-15912848::INFO::2013-11-20 02:46:04,432::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000766038894653', 'lastCheck': 1384915561.37098, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00126099586487', 'lastCheck': 1384915560.9136021, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00150203704834', 'lastCheck': 1384915561.3631539, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000483989715576', 'lastCheck': 1384915559.821532, 'valid': True, 'code': 0}}
>Thread-15912850::DEBUG::2013-11-20 02:46:04,848::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912850::INFO::2013-11-20 02:46:04,850::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912850::DEBUG::2013-11-20 02:46:04,850::task::495::TaskManager.Task::(_debug) Task 01f5600c-57ee-488b-a51a-512b251e1c4f: moving from state init -> state preparing
>Thread-15912850::DEBUG::2013-11-20 02:46:04,851::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912850::DEBUG::2013-11-20 02:46:04,851::task::495::TaskManager.Task::(_debug) Task 01f5600c-57ee-488b-a51a-512b251e1c4f: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912850::DEBUG::2013-11-20 02:46:04,852::task::495::TaskManager.Task::(_debug) Task 01f5600c-57ee-488b-a51a-512b251e1c4f: moving from state preparing -> state finished
>Thread-15912850::DEBUG::2013-11-20 02:46:04,853::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912850::DEBUG::2013-11-20 02:46:04,853::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912850::DEBUG::2013-11-20 02:46:04,854::task::495::TaskManager.Task::(_debug) Task 01f5600c-57ee-488b-a51a-512b251e1c4f: ref 0 aborting False
>Thread-15912850::INFO::2013-11-20 02:46:04,854::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912851::DEBUG::2013-11-20 02:46:04,859::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912851::INFO::2013-11-20 02:46:04,860::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912851::DEBUG::2013-11-20 02:46:04,860::task::495::TaskManager.Task::(_debug) Task acfdb1e3-0b98-4212-b701-4097fb79c903: moving from state init -> state preparing
>Thread-15912851::DEBUG::2013-11-20 02:46:04,867::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7367a4a0-17e6-4675-8bd0-41a782e44ca4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912851::DEBUG::2013-11-20 02:46:04,868::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912851::DEBUG::2013-11-20 02:46:04,869::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912851::DEBUG::2013-11-20 02:46:04,869::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7367a4a0-17e6-4675-8bd0-41a782e44ca4`::Granted request
>Thread-15912851::DEBUG::2013-11-20 02:46:04,870::task::495::TaskManager.Task::(_debug) Task acfdb1e3-0b98-4212-b701-4097fb79c903: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912851::DEBUG::2013-11-20 02:46:04,871::task::495::TaskManager.Task::(_debug) Task acfdb1e3-0b98-4212-b701-4097fb79c903: ref 1 aborting False
>Thread-15912851::WARNING::2013-11-20 02:46:04,872::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912851::DEBUG::2013-11-20 02:46:04,873::task::495::TaskManager.Task::(_debug) Task acfdb1e3-0b98-4212-b701-4097fb79c903: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912851::DEBUG::2013-11-20 02:46:04,874::task::495::TaskManager.Task::(_debug) Task acfdb1e3-0b98-4212-b701-4097fb79c903: moving from state preparing -> state finished
>Thread-15912851::DEBUG::2013-11-20 02:46:04,875::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912851::DEBUG::2013-11-20 02:46:04,875::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912851::DEBUG::2013-11-20 02:46:04,876::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912851::DEBUG::2013-11-20 02:46:04,876::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912851::DEBUG::2013-11-20 02:46:04,877::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912851::DEBUG::2013-11-20 02:46:04,877::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912851::DEBUG::2013-11-20 02:46:04,878::task::495::TaskManager.Task::(_debug) Task acfdb1e3-0b98-4212-b701-4097fb79c903: ref 0 aborting False
>Thread-15912851::INFO::2013-11-20 02:46:04,879::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:46:05,757::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:46:06,233::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0428606 s, 23.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:46:08,243::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:46:08,816::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.109477 s, 9.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:46:10,831::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:46:11,351::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0702245 s, 14.6 MB/s\n'; <rc> = 0
>Thread-42838::DEBUG::2013-11-20 02:46:11,475::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 02:46:12,143::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:46:13,366::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:46:13,903::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0728261 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:46:15,918::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:46:16,486::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0773797 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15912855::DEBUG::2013-11-20 02:46:18,288::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912855::INFO::2013-11-20 02:46:18,289::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912855::DEBUG::2013-11-20 02:46:18,290::task::495::TaskManager.Task::(_debug) Task af3b0cbe-cab7-45e2-b0ac-e15b3d701386: moving from state init -> state preparing
>Thread-15912855::DEBUG::2013-11-20 02:46:18,290::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912855::DEBUG::2013-11-20 02:46:18,291::task::495::TaskManager.Task::(_debug) Task af3b0cbe-cab7-45e2-b0ac-e15b3d701386: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912855::DEBUG::2013-11-20 02:46:18,291::task::495::TaskManager.Task::(_debug) Task af3b0cbe-cab7-45e2-b0ac-e15b3d701386: moving from state preparing -> state finished
>Thread-15912855::DEBUG::2013-11-20 02:46:18,292::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912855::DEBUG::2013-11-20 02:46:18,292::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912855::DEBUG::2013-11-20 02:46:18,293::task::495::TaskManager.Task::(_debug) Task af3b0cbe-cab7-45e2-b0ac-e15b3d701386: ref 0 aborting False
>Thread-15912855::INFO::2013-11-20 02:46:18,293::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912856::DEBUG::2013-11-20 02:46:18,298::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912856::INFO::2013-11-20 02:46:18,298::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912856::DEBUG::2013-11-20 02:46:18,299::task::495::TaskManager.Task::(_debug) Task 9e3da743-565c-48df-a268-05759809d474: moving from state init -> state preparing
>Thread-15912856::DEBUG::2013-11-20 02:46:18,300::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9e07a869-63df-48c1-b000-3f76261fa9e5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912856::DEBUG::2013-11-20 02:46:18,300::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912856::DEBUG::2013-11-20 02:46:18,301::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912856::DEBUG::2013-11-20 02:46:18,301::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9e07a869-63df-48c1-b000-3f76261fa9e5`::Granted request
>Thread-15912856::DEBUG::2013-11-20 02:46:18,302::task::495::TaskManager.Task::(_debug) Task 9e3da743-565c-48df-a268-05759809d474: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912856::DEBUG::2013-11-20 02:46:18,303::task::495::TaskManager.Task::(_debug) Task 9e3da743-565c-48df-a268-05759809d474: ref 1 aborting False
>Thread-15912856::WARNING::2013-11-20 02:46:18,304::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912856::DEBUG::2013-11-20 02:46:18,305::task::495::TaskManager.Task::(_debug) Task 9e3da743-565c-48df-a268-05759809d474: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115941048320', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115941048320', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912856::DEBUG::2013-11-20 02:46:18,306::task::495::TaskManager.Task::(_debug) Task 9e3da743-565c-48df-a268-05759809d474: moving from state preparing -> state finished
>Thread-15912856::DEBUG::2013-11-20 02:46:18,306::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912856::DEBUG::2013-11-20 02:46:18,307::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912856::DEBUG::2013-11-20 02:46:18,307::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912856::DEBUG::2013-11-20 02:46:18,308::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912856::DEBUG::2013-11-20 02:46:18,308::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912856::DEBUG::2013-11-20 02:46:18,309::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912856::DEBUG::2013-11-20 02:46:18,309::task::495::TaskManager.Task::(_debug) Task 9e3da743-565c-48df-a268-05759809d474: ref 0 aborting False
>Thread-15912856::INFO::2013-11-20 02:46:18,310::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115941048320', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115941048320', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:46:18,501::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:46:19,045::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0707807 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:46:21,059::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:46:21,623::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.10777 s, 9.5 MB/s\n'; <rc> = 0
>Thread-15912858::INFO::2013-11-20 02:46:23,121::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912858::DEBUG::2013-11-20 02:46:23,122::task::495::TaskManager.Task::(_debug) Task 4f854ef0-10c3-437e-83de-d3f79fa61235: moving from state init -> state preparing
>Thread-15912858::DEBUG::2013-11-20 02:46:23,123::task::495::TaskManager.Task::(_debug) Task 4f854ef0-10c3-437e-83de-d3f79fa61235: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000284194946289', 'lastCheck': 1384915582.282527, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00133800506592', 'lastCheck': 1384915581.2967811, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00146818161011', 'lastCheck': 1384915581.741168, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000514030456543', 'lastCheck': 1384915580.153661, 'valid': True, 'code': 0}}
>Thread-15912858::DEBUG::2013-11-20 02:46:23,124::task::495::TaskManager.Task::(_debug) Task 4f854ef0-10c3-437e-83de-d3f79fa61235: moving from state preparing -> state finished
>Thread-15912858::DEBUG::2013-11-20 02:46:23,124::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912858::DEBUG::2013-11-20 02:46:23,125::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912858::DEBUG::2013-11-20 02:46:23,125::task::495::TaskManager.Task::(_debug) Task 4f854ef0-10c3-437e-83de-d3f79fa61235: ref 0 aborting False
>Thread-15912858::INFO::2013-11-20 02:46:23,126::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000284194946289', 'lastCheck': 1384915582.282527, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00133800506592', 'lastCheck': 1384915581.2967811, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00146818161011', 'lastCheck': 1384915581.741168, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000514030456543', 'lastCheck': 1384915580.153661, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:46:23,636::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:46:24,193::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0757027 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:46:26,209::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:46:26,739::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0743503 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:46:28,754::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912861::DEBUG::2013-11-20 02:46:29,220::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912861::INFO::2013-11-20 02:46:29,224::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912861::DEBUG::2013-11-20 02:46:29,226::task::495::TaskManager.Task::(_debug) Task 5349e46f-92ac-4e79-a9b5-7730e1e1d854: moving from state init -> state preparing
>Thread-15912861::DEBUG::2013-11-20 02:46:29,228::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912861::DEBUG::2013-11-20 02:46:29,230::task::495::TaskManager.Task::(_debug) Task 5349e46f-92ac-4e79-a9b5-7730e1e1d854: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912861::DEBUG::2013-11-20 02:46:29,231::task::495::TaskManager.Task::(_debug) Task 5349e46f-92ac-4e79-a9b5-7730e1e1d854: moving from state preparing -> state finished
>Thread-15912861::DEBUG::2013-11-20 02:46:29,234::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912861::DEBUG::2013-11-20 02:46:29,236::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912861::DEBUG::2013-11-20 02:46:29,237::task::495::TaskManager.Task::(_debug) Task 5349e46f-92ac-4e79-a9b5-7730e1e1d854: ref 0 aborting False
>Thread-15912861::INFO::2013-11-20 02:46:29,240::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912862::DEBUG::2013-11-20 02:46:29,247::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912862::INFO::2013-11-20 02:46:29,248::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912862::DEBUG::2013-11-20 02:46:29,249::task::495::TaskManager.Task::(_debug) Task 295ed114-d44a-485c-8ce1-381b93573bab: moving from state init -> state preparing
>Thread-15912862::DEBUG::2013-11-20 02:46:29,252::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`81ebe5b3-f246-4b3d-be75-bb925bb7f279`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912862::DEBUG::2013-11-20 02:46:29,253::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912862::DEBUG::2013-11-20 02:46:29,256::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912862::DEBUG::2013-11-20 02:46:29,257::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`81ebe5b3-f246-4b3d-be75-bb925bb7f279`::Granted request
>Thread-15912862::DEBUG::2013-11-20 02:46:29,259::task::495::TaskManager.Task::(_debug) Task 295ed114-d44a-485c-8ce1-381b93573bab: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912862::DEBUG::2013-11-20 02:46:29,260::task::495::TaskManager.Task::(_debug) Task 295ed114-d44a-485c-8ce1-381b93573bab: ref 1 aborting False
>Thread-15912862::WARNING::2013-11-20 02:46:29,262::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912862::DEBUG::2013-11-20 02:46:29,264::task::495::TaskManager.Task::(_debug) Task 295ed114-d44a-485c-8ce1-381b93573bab: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115941048320', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115941048320', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912862::DEBUG::2013-11-20 02:46:29,264::task::495::TaskManager.Task::(_debug) Task 295ed114-d44a-485c-8ce1-381b93573bab: moving from state preparing -> state finished
>Thread-15912862::DEBUG::2013-11-20 02:46:29,265::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912862::DEBUG::2013-11-20 02:46:29,266::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912862::DEBUG::2013-11-20 02:46:29,268::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912862::DEBUG::2013-11-20 02:46:29,270::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912862::DEBUG::2013-11-20 02:46:29,270::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912862::DEBUG::2013-11-20 02:46:29,272::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912862::DEBUG::2013-11-20 02:46:29,274::task::495::TaskManager.Task::(_debug) Task 295ed114-d44a-485c-8ce1-381b93573bab: ref 0 aborting False
>Thread-15912862::INFO::2013-11-20 02:46:29,275::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115941048320', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115941048320', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:46:29,311::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0726379 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:46:31,330::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:46:31,845::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0729434 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:46:33,860::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:46:34,318::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0516183 s, 19.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:46:36,334::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:46:36,876::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0648102 s, 15.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:46:38,890::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:46:39,397::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0873119 s, 11.7 MB/s\n'; <rc> = 0
>Thread-15912866::DEBUG::2013-11-20 02:46:40,796::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912866::INFO::2013-11-20 02:46:40,798::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912866::DEBUG::2013-11-20 02:46:40,798::task::495::TaskManager.Task::(_debug) Task 76cb0ec9-b492-4599-85be-796d73351f71: moving from state init -> state preparing
>Thread-15912866::DEBUG::2013-11-20 02:46:40,799::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912866::DEBUG::2013-11-20 02:46:40,800::task::495::TaskManager.Task::(_debug) Task 76cb0ec9-b492-4599-85be-796d73351f71: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912866::DEBUG::2013-11-20 02:46:40,801::task::495::TaskManager.Task::(_debug) Task 76cb0ec9-b492-4599-85be-796d73351f71: moving from state preparing -> state finished
>Thread-15912866::DEBUG::2013-11-20 02:46:40,801::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912866::DEBUG::2013-11-20 02:46:40,802::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912866::DEBUG::2013-11-20 02:46:40,802::task::495::TaskManager.Task::(_debug) Task 76cb0ec9-b492-4599-85be-796d73351f71: ref 0 aborting False
>Thread-15912866::INFO::2013-11-20 02:46:40,803::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912867::DEBUG::2013-11-20 02:46:40,808::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912867::INFO::2013-11-20 02:46:40,808::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912867::DEBUG::2013-11-20 02:46:40,809::task::495::TaskManager.Task::(_debug) Task fe194583-ac28-46e1-b7d4-6bf719f6431d: moving from state init -> state preparing
>Thread-15912867::DEBUG::2013-11-20 02:46:40,810::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bd817009-7aa5-47c6-8fae-31b55b4b2f98`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912867::DEBUG::2013-11-20 02:46:40,811::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912867::DEBUG::2013-11-20 02:46:40,812::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912867::DEBUG::2013-11-20 02:46:40,812::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bd817009-7aa5-47c6-8fae-31b55b4b2f98`::Granted request
>Thread-15912867::DEBUG::2013-11-20 02:46:40,813::task::495::TaskManager.Task::(_debug) Task fe194583-ac28-46e1-b7d4-6bf719f6431d: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912867::DEBUG::2013-11-20 02:46:40,814::task::495::TaskManager.Task::(_debug) Task fe194583-ac28-46e1-b7d4-6bf719f6431d: ref 1 aborting False
>Thread-15912867::WARNING::2013-11-20 02:46:40,816::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912867::DEBUG::2013-11-20 02:46:40,816::task::495::TaskManager.Task::(_debug) Task fe194583-ac28-46e1-b7d4-6bf719f6431d: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115924271104', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115923222528', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912867::DEBUG::2013-11-20 02:46:40,817::task::495::TaskManager.Task::(_debug) Task fe194583-ac28-46e1-b7d4-6bf719f6431d: moving from state preparing -> state finished
>Thread-15912867::DEBUG::2013-11-20 02:46:40,818::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912867::DEBUG::2013-11-20 02:46:40,818::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912867::DEBUG::2013-11-20 02:46:40,819::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912867::DEBUG::2013-11-20 02:46:40,820::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912867::DEBUG::2013-11-20 02:46:40,820::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912867::DEBUG::2013-11-20 02:46:40,821::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912867::DEBUG::2013-11-20 02:46:40,821::task::495::TaskManager.Task::(_debug) Task fe194583-ac28-46e1-b7d4-6bf719f6431d: ref 0 aborting False
>Thread-15912867::INFO::2013-11-20 02:46:40,822::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115924271104', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115923222528', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912868::INFO::2013-11-20 02:46:41,338::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912868::DEBUG::2013-11-20 02:46:41,339::task::495::TaskManager.Task::(_debug) Task fdfffb56-5876-4bdd-8b96-f7fccc60c4d2: moving from state init -> state preparing
>Thread-15912868::DEBUG::2013-11-20 02:46:41,340::task::495::TaskManager.Task::(_debug) Task fdfffb56-5876-4bdd-8b96-f7fccc60c4d2: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000371932983398', 'lastCheck': 1384915592.5432799, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00159811973572', 'lastCheck': 1384915591.5499549, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00140309333801', 'lastCheck': 1384915591.9769869, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000420093536377', 'lastCheck': 1384915600.5503819, 'valid': True, 'code': 0}}
>Thread-15912868::DEBUG::2013-11-20 02:46:41,340::task::495::TaskManager.Task::(_debug) Task fdfffb56-5876-4bdd-8b96-f7fccc60c4d2: moving from state preparing -> state finished
>Thread-15912868::DEBUG::2013-11-20 02:46:41,341::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912868::DEBUG::2013-11-20 02:46:41,341::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912868::DEBUG::2013-11-20 02:46:41,342::task::495::TaskManager.Task::(_debug) Task fdfffb56-5876-4bdd-8b96-f7fccc60c4d2: ref 0 aborting False
>Thread-15912868::INFO::2013-11-20 02:46:41,343::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000371932983398', 'lastCheck': 1384915592.5432799, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00159811973572', 'lastCheck': 1384915591.5499549, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00140309333801', 'lastCheck': 1384915591.9769869, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000420093536377', 'lastCheck': 1384915600.5503819, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:46:41,412::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:46:41,960::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0820122 s, 12.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:46:43,976::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:46:44,493::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0755298 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:46:46,508::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:46:46,940::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0533042 s, 19.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:46:48,953::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:46:49,492::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0744508 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:46:51,508::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:46:52,069::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0762855 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:46:54,084::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912873::DEBUG::2013-11-20 02:46:54,354::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912873::INFO::2013-11-20 02:46:54,361::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912873::DEBUG::2013-11-20 02:46:54,361::task::495::TaskManager.Task::(_debug) Task a51244df-c9b1-4a7b-8508-0cb0e78b5f3b: moving from state init -> state preparing
>Thread-15912873::DEBUG::2013-11-20 02:46:54,362::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912873::DEBUG::2013-11-20 02:46:54,362::task::495::TaskManager.Task::(_debug) Task a51244df-c9b1-4a7b-8508-0cb0e78b5f3b: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912873::DEBUG::2013-11-20 02:46:54,362::task::495::TaskManager.Task::(_debug) Task a51244df-c9b1-4a7b-8508-0cb0e78b5f3b: moving from state preparing -> state finished
>Thread-15912873::DEBUG::2013-11-20 02:46:54,363::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912873::DEBUG::2013-11-20 02:46:54,363::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912873::DEBUG::2013-11-20 02:46:54,363::task::495::TaskManager.Task::(_debug) Task a51244df-c9b1-4a7b-8508-0cb0e78b5f3b: ref 0 aborting False
>Thread-15912873::INFO::2013-11-20 02:46:54,364::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912874::DEBUG::2013-11-20 02:46:54,368::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912874::INFO::2013-11-20 02:46:54,369::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912874::DEBUG::2013-11-20 02:46:54,370::task::495::TaskManager.Task::(_debug) Task 9756e5e4-5ce8-48b4-8989-7e26e4955b53: moving from state init -> state preparing
>Thread-15912874::DEBUG::2013-11-20 02:46:54,371::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`89e7b645-937e-496f-9f97-3ddd7d30f7dc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912874::DEBUG::2013-11-20 02:46:54,372::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912874::DEBUG::2013-11-20 02:46:54,373::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912874::DEBUG::2013-11-20 02:46:54,373::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`89e7b645-937e-496f-9f97-3ddd7d30f7dc`::Granted request
>Thread-15912874::DEBUG::2013-11-20 02:46:54,375::task::495::TaskManager.Task::(_debug) Task 9756e5e4-5ce8-48b4-8989-7e26e4955b53: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912874::DEBUG::2013-11-20 02:46:54,375::task::495::TaskManager.Task::(_debug) Task 9756e5e4-5ce8-48b4-8989-7e26e4955b53: ref 1 aborting False
>Thread-15912874::WARNING::2013-11-20 02:46:54,377::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912874::DEBUG::2013-11-20 02:46:54,378::task::495::TaskManager.Task::(_debug) Task 9756e5e4-5ce8-48b4-8989-7e26e4955b53: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115825704960', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115822559232', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912874::DEBUG::2013-11-20 02:46:54,379::task::495::TaskManager.Task::(_debug) Task 9756e5e4-5ce8-48b4-8989-7e26e4955b53: moving from state preparing -> state finished
>Thread-15912874::DEBUG::2013-11-20 02:46:54,380::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912874::DEBUG::2013-11-20 02:46:54,380::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912874::DEBUG::2013-11-20 02:46:54,381::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912874::DEBUG::2013-11-20 02:46:54,382::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912874::DEBUG::2013-11-20 02:46:54,382::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912874::DEBUG::2013-11-20 02:46:54,383::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912874::DEBUG::2013-11-20 02:46:54,383::task::495::TaskManager.Task::(_debug) Task 9756e5e4-5ce8-48b4-8989-7e26e4955b53: ref 0 aborting False
>Thread-15912874::INFO::2013-11-20 02:46:54,384::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115825704960', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115822559232', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:46:54,625::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0735197 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:46:56,641::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:46:57,186::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0738327 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:46:59,202::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:46:59,625::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.045713 s, 22.4 MB/s\n'; <rc> = 0
>Thread-15912876::INFO::2013-11-20 02:46:59,894::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912876::DEBUG::2013-11-20 02:46:59,896::task::495::TaskManager.Task::(_debug) Task c2a85e7a-d0fa-409f-8430-71598aed3432: moving from state init -> state preparing
>Thread-15912876::DEBUG::2013-11-20 02:46:59,897::task::495::TaskManager.Task::(_debug) Task c2a85e7a-d0fa-409f-8430-71598aed3432: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000380039215088', 'lastCheck': 1384915612.758204, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00153183937073', 'lastCheck': 1384915611.761709, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '1.17933297157', 'lastCheck': 1384915613.3739481, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000397920608521', 'lastCheck': 1384915610.692369, 'valid': True, 'code': 0}}
>Thread-15912876::DEBUG::2013-11-20 02:46:59,898::task::495::TaskManager.Task::(_debug) Task c2a85e7a-d0fa-409f-8430-71598aed3432: moving from state preparing -> state finished
>Thread-15912876::DEBUG::2013-11-20 02:46:59,899::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912876::DEBUG::2013-11-20 02:46:59,899::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912876::DEBUG::2013-11-20 02:46:59,900::task::495::TaskManager.Task::(_debug) Task c2a85e7a-d0fa-409f-8430-71598aed3432: ref 0 aborting False
>Thread-15912876::INFO::2013-11-20 02:46:59,900::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000380039215088', 'lastCheck': 1384915612.758204, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00153183937073', 'lastCheck': 1384915611.761709, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '1.17933297157', 'lastCheck': 1384915613.3739481, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000397920608521', 'lastCheck': 1384915610.692369, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:47:01,635::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:47:02,135::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0729208 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:47:04,150::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 02:47:04,409::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 02:47:04,410::task::495::TaskManager.Task::(_debug) Task 06f9e858-8170-4cf6-80c8-eaef06d471f6: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 02:47:04,411::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c7e41c64-53cd-498f-bb6e-e268712df33d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 02:47:04,412::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 02:47:04,412::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 02:47:04,413::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c7e41c64-53cd-498f-bb6e-e268712df33d`::Granted request
>Thread-15912392::DEBUG::2013-11-20 02:47:04,413::task::495::TaskManager.Task::(_debug) Task 06f9e858-8170-4cf6-80c8-eaef06d471f6: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 02:47:04,414::task::495::TaskManager.Task::(_debug) Task 06f9e858-8170-4cf6-80c8-eaef06d471f6: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 02:47:04,414::task::495::TaskManager.Task::(_debug) Task 06f9e858-8170-4cf6-80c8-eaef06d471f6: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 02:47:04,415::task::495::TaskManager.Task::(_debug) Task 06f9e858-8170-4cf6-80c8-eaef06d471f6: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 02:47:04,415::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 02:47:04,415::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 02:47:04,416::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 02:47:04,416::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 02:47:04,416::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 02:47:04,417::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 02:47:04,417::task::495::TaskManager.Task::(_debug) Task 06f9e858-8170-4cf6-80c8-eaef06d471f6: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 02:47:04,417::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 02:47:04,703::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0743489 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15912878::DEBUG::2013-11-20 02:47:05,774::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912878::INFO::2013-11-20 02:47:05,775::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912878::DEBUG::2013-11-20 02:47:05,776::task::495::TaskManager.Task::(_debug) Task 0b62fdff-2762-415f-9e1d-53eed2147812: moving from state init -> state preparing
>Thread-15912878::DEBUG::2013-11-20 02:47:05,777::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912878::DEBUG::2013-11-20 02:47:05,777::task::495::TaskManager.Task::(_debug) Task 0b62fdff-2762-415f-9e1d-53eed2147812: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912878::DEBUG::2013-11-20 02:47:05,778::task::495::TaskManager.Task::(_debug) Task 0b62fdff-2762-415f-9e1d-53eed2147812: moving from state preparing -> state finished
>Thread-15912878::DEBUG::2013-11-20 02:47:05,779::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912878::DEBUG::2013-11-20 02:47:05,779::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912878::DEBUG::2013-11-20 02:47:05,780::task::495::TaskManager.Task::(_debug) Task 0b62fdff-2762-415f-9e1d-53eed2147812: ref 0 aborting False
>Thread-15912878::INFO::2013-11-20 02:47:05,780::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912879::DEBUG::2013-11-20 02:47:05,784::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912879::INFO::2013-11-20 02:47:05,785::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912879::DEBUG::2013-11-20 02:47:05,786::task::495::TaskManager.Task::(_debug) Task bba29169-369a-42a5-958a-2030a1668038: moving from state init -> state preparing
>Thread-15912879::DEBUG::2013-11-20 02:47:05,787::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b376131e-a630-4fc4-801a-d482b30bdcc6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912879::DEBUG::2013-11-20 02:47:05,788::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912879::DEBUG::2013-11-20 02:47:05,788::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912879::DEBUG::2013-11-20 02:47:05,789::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b376131e-a630-4fc4-801a-d482b30bdcc6`::Granted request
>Thread-15912879::DEBUG::2013-11-20 02:47:05,790::task::495::TaskManager.Task::(_debug) Task bba29169-369a-42a5-958a-2030a1668038: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912879::DEBUG::2013-11-20 02:47:05,790::task::495::TaskManager.Task::(_debug) Task bba29169-369a-42a5-958a-2030a1668038: ref 1 aborting False
>Thread-15912879::WARNING::2013-11-20 02:47:05,792::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912879::DEBUG::2013-11-20 02:47:05,793::task::495::TaskManager.Task::(_debug) Task bba29169-369a-42a5-958a-2030a1668038: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115777470464', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115777470464', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912879::DEBUG::2013-11-20 02:47:05,794::task::495::TaskManager.Task::(_debug) Task bba29169-369a-42a5-958a-2030a1668038: moving from state preparing -> state finished
>Thread-15912879::DEBUG::2013-11-20 02:47:05,794::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912879::DEBUG::2013-11-20 02:47:05,795::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912879::DEBUG::2013-11-20 02:47:05,796::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912879::DEBUG::2013-11-20 02:47:05,796::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912879::DEBUG::2013-11-20 02:47:05,797::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912879::DEBUG::2013-11-20 02:47:05,797::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912879::DEBUG::2013-11-20 02:47:05,798::task::495::TaskManager.Task::(_debug) Task bba29169-369a-42a5-958a-2030a1668038: ref 0 aborting False
>Thread-15912879::INFO::2013-11-20 02:47:05,798::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115777470464', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115777470464', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:47:06,718::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:47:07,163::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0436064 s, 23.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:47:09,176::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:47:09,711::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0733351 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:47:11,727::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:47:12,262::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0745845 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:47:14,278::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:47:14,830::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.071703 s, 14.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:47:16,846::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:47:17,472::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0705565 s, 14.5 MB/s\n'; <rc> = 0
>Thread-15912884::INFO::2013-11-20 02:47:19,097::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912884::DEBUG::2013-11-20 02:47:19,098::task::495::TaskManager.Task::(_debug) Task 242c6a79-a890-4e62-bc65-4c568979f9cd: moving from state init -> state preparing
>Thread-15912884::DEBUG::2013-11-20 02:47:19,099::task::495::TaskManager.Task::(_debug) Task 242c6a79-a890-4e62-bc65-4c568979f9cd: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00434994697571', 'lastCheck': 1384915632.95312, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00172710418701', 'lastCheck': 1384915631.9526539, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00142192840576', 'lastCheck': 1384915633.4315341, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00501394271851', 'lastCheck': 1384915630.9430161, 'valid': True, 'code': 0}}
>Thread-15912884::DEBUG::2013-11-20 02:47:19,100::task::495::TaskManager.Task::(_debug) Task 242c6a79-a890-4e62-bc65-4c568979f9cd: moving from state preparing -> state finished
>Thread-15912884::DEBUG::2013-11-20 02:47:19,100::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912884::DEBUG::2013-11-20 02:47:19,101::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912884::DEBUG::2013-11-20 02:47:19,101::task::495::TaskManager.Task::(_debug) Task 242c6a79-a890-4e62-bc65-4c568979f9cd: ref 0 aborting False
>Thread-15912884::INFO::2013-11-20 02:47:19,102::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00434994697571', 'lastCheck': 1384915632.95312, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00172710418701', 'lastCheck': 1384915631.9526539, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00142192840576', 'lastCheck': 1384915633.4315341, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00501394271851', 'lastCheck': 1384915630.9430161, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:47:19,487::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:47:20,015::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0746915 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15912886::DEBUG::2013-11-20 02:47:20,205::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912886::INFO::2013-11-20 02:47:20,206::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912886::DEBUG::2013-11-20 02:47:20,207::task::495::TaskManager.Task::(_debug) Task f02995b8-2ffa-4cae-bcdc-c00cb58b0111: moving from state init -> state preparing
>Thread-15912886::DEBUG::2013-11-20 02:47:20,208::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912886::DEBUG::2013-11-20 02:47:20,208::task::495::TaskManager.Task::(_debug) Task f02995b8-2ffa-4cae-bcdc-c00cb58b0111: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912886::DEBUG::2013-11-20 02:47:20,209::task::495::TaskManager.Task::(_debug) Task f02995b8-2ffa-4cae-bcdc-c00cb58b0111: moving from state preparing -> state finished
>Thread-15912886::DEBUG::2013-11-20 02:47:20,209::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912886::DEBUG::2013-11-20 02:47:20,210::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912886::DEBUG::2013-11-20 02:47:20,210::task::495::TaskManager.Task::(_debug) Task f02995b8-2ffa-4cae-bcdc-c00cb58b0111: ref 0 aborting False
>Thread-15912886::INFO::2013-11-20 02:47:20,211::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912887::DEBUG::2013-11-20 02:47:20,215::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912887::INFO::2013-11-20 02:47:20,216::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912887::DEBUG::2013-11-20 02:47:20,217::task::495::TaskManager.Task::(_debug) Task 96ffdde3-d0f5-49a4-9f0b-08f2e9d96fef: moving from state init -> state preparing
>Thread-15912887::DEBUG::2013-11-20 02:47:20,218::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5f4d666f-25a9-4e0c-b566-570db0171daa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912887::DEBUG::2013-11-20 02:47:20,218::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912887::DEBUG::2013-11-20 02:47:20,219::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912887::DEBUG::2013-11-20 02:47:20,220::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5f4d666f-25a9-4e0c-b566-570db0171daa`::Granted request
>Thread-15912887::DEBUG::2013-11-20 02:47:20,221::task::495::TaskManager.Task::(_debug) Task 96ffdde3-d0f5-49a4-9f0b-08f2e9d96fef: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912887::DEBUG::2013-11-20 02:47:20,221::task::495::TaskManager.Task::(_debug) Task 96ffdde3-d0f5-49a4-9f0b-08f2e9d96fef: ref 1 aborting False
>Thread-15912887::WARNING::2013-11-20 02:47:20,223::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912887::DEBUG::2013-11-20 02:47:20,224::task::495::TaskManager.Task::(_debug) Task 96ffdde3-d0f5-49a4-9f0b-08f2e9d96fef: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115786907648', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115786907648', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912887::DEBUG::2013-11-20 02:47:20,225::task::495::TaskManager.Task::(_debug) Task 96ffdde3-d0f5-49a4-9f0b-08f2e9d96fef: moving from state preparing -> state finished
>Thread-15912887::DEBUG::2013-11-20 02:47:20,226::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912887::DEBUG::2013-11-20 02:47:20,226::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912887::DEBUG::2013-11-20 02:47:20,227::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912887::DEBUG::2013-11-20 02:47:20,227::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912887::DEBUG::2013-11-20 02:47:20,228::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912887::DEBUG::2013-11-20 02:47:20,228::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912887::DEBUG::2013-11-20 02:47:20,229::task::495::TaskManager.Task::(_debug) Task 96ffdde3-d0f5-49a4-9f0b-08f2e9d96fef: ref 0 aborting False
>Thread-15912887::INFO::2013-11-20 02:47:20,230::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115786907648', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115786907648', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:47:22,031::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:47:22,534::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.053826 s, 19.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:47:24,544::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:47:25,028::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0571899 s, 17.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:47:27,042::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:47:27,562::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.107634 s, 9.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:47:29,577::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:47:30,031::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0470991 s, 21.7 MB/s\n'; <rc> = 0
>Thread-15912891::DEBUG::2013-11-20 02:47:31,129::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912891::INFO::2013-11-20 02:47:31,130::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912891::DEBUG::2013-11-20 02:47:31,131::task::495::TaskManager.Task::(_debug) Task 6a232e9d-9ef7-44d2-8bb5-f8e6a10070f4: moving from state init -> state preparing
>Thread-15912891::DEBUG::2013-11-20 02:47:31,132::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912891::DEBUG::2013-11-20 02:47:31,132::task::495::TaskManager.Task::(_debug) Task 6a232e9d-9ef7-44d2-8bb5-f8e6a10070f4: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912891::DEBUG::2013-11-20 02:47:31,133::task::495::TaskManager.Task::(_debug) Task 6a232e9d-9ef7-44d2-8bb5-f8e6a10070f4: moving from state preparing -> state finished
>Thread-15912891::DEBUG::2013-11-20 02:47:31,134::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912891::DEBUG::2013-11-20 02:47:31,134::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912891::DEBUG::2013-11-20 02:47:31,135::task::495::TaskManager.Task::(_debug) Task 6a232e9d-9ef7-44d2-8bb5-f8e6a10070f4: ref 0 aborting False
>Thread-15912891::INFO::2013-11-20 02:47:31,135::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912892::DEBUG::2013-11-20 02:47:31,140::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912892::INFO::2013-11-20 02:47:31,141::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912892::DEBUG::2013-11-20 02:47:31,142::task::495::TaskManager.Task::(_debug) Task 18b43929-e141-491a-bace-04debc8fd802: moving from state init -> state preparing
>Thread-15912892::DEBUG::2013-11-20 02:47:31,143::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ca458766-2145-44ae-8f5c-5630700c0f7f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912892::DEBUG::2013-11-20 02:47:31,143::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912892::DEBUG::2013-11-20 02:47:31,144::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912892::DEBUG::2013-11-20 02:47:31,145::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ca458766-2145-44ae-8f5c-5630700c0f7f`::Granted request
>Thread-15912892::DEBUG::2013-11-20 02:47:31,146::task::495::TaskManager.Task::(_debug) Task 18b43929-e141-491a-bace-04debc8fd802: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912892::DEBUG::2013-11-20 02:47:31,146::task::495::TaskManager.Task::(_debug) Task 18b43929-e141-491a-bace-04debc8fd802: ref 1 aborting False
>Thread-15912892::WARNING::2013-11-20 02:47:31,149::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912892::DEBUG::2013-11-20 02:47:31,149::task::495::TaskManager.Task::(_debug) Task 18b43929-e141-491a-bace-04debc8fd802: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115785859072', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115785859072', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912892::DEBUG::2013-11-20 02:47:31,150::task::495::TaskManager.Task::(_debug) Task 18b43929-e141-491a-bace-04debc8fd802: moving from state preparing -> state finished
>Thread-15912892::DEBUG::2013-11-20 02:47:31,151::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912892::DEBUG::2013-11-20 02:47:31,151::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912892::DEBUG::2013-11-20 02:47:31,152::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912892::DEBUG::2013-11-20 02:47:31,153::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912892::DEBUG::2013-11-20 02:47:31,153::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912892::DEBUG::2013-11-20 02:47:31,154::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912892::DEBUG::2013-11-20 02:47:31,154::task::495::TaskManager.Task::(_debug) Task 18b43929-e141-491a-bace-04debc8fd802: ref 0 aborting False
>Thread-15912892::INFO::2013-11-20 02:47:31,155::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115785859072', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115785859072', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:47:32,041::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:47:32,523::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.079343 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:47:34,540::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:47:35,104::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0823106 s, 12.4 MB/s\n'; <rc> = 0
>Thread-15912894::INFO::2013-11-20 02:47:35,839::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912894::DEBUG::2013-11-20 02:47:35,840::task::495::TaskManager.Task::(_debug) Task c5f15aab-44d1-4d35-821f-20f99b15cb10: moving from state init -> state preparing
>Thread-15912894::DEBUG::2013-11-20 02:47:35,841::task::495::TaskManager.Task::(_debug) Task c5f15aab-44d1-4d35-821f-20f99b15cb10: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000355005264282', 'lastCheck': 1384915653.2712369, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0010039806366', 'lastCheck': 1384915652.277986, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00103306770325', 'lastCheck': 1384915653.803283, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000432968139648', 'lastCheck': 1384915651.264478, 'valid': True, 'code': 0}}
>Thread-15912894::DEBUG::2013-11-20 02:47:35,842::task::495::TaskManager.Task::(_debug) Task c5f15aab-44d1-4d35-821f-20f99b15cb10: moving from state preparing -> state finished
>Thread-15912894::DEBUG::2013-11-20 02:47:35,842::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912894::DEBUG::2013-11-20 02:47:35,843::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912894::DEBUG::2013-11-20 02:47:35,843::task::495::TaskManager.Task::(_debug) Task c5f15aab-44d1-4d35-821f-20f99b15cb10: ref 0 aborting False
>Thread-15912894::INFO::2013-11-20 02:47:35,844::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000355005264282', 'lastCheck': 1384915653.2712369, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0010039806366', 'lastCheck': 1384915652.277986, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00103306770325', 'lastCheck': 1384915653.803283, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000432968139648', 'lastCheck': 1384915651.264478, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:47:37,116::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:47:37,687::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0718353 s, 14.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:47:39,703::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:47:40,271::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0583764 s, 17.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:47:42,282::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:47:42,758::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0736055 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:47:44,772::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:47:45,305::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0748272 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15912898::DEBUG::2013-11-20 02:47:47,271::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912898::INFO::2013-11-20 02:47:47,272::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912898::DEBUG::2013-11-20 02:47:47,273::task::495::TaskManager.Task::(_debug) Task 82c53c6e-9ee9-433d-8e17-bf6604fe0dfc: moving from state init -> state preparing
>Thread-15912898::DEBUG::2013-11-20 02:47:47,274::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912898::DEBUG::2013-11-20 02:47:47,274::task::495::TaskManager.Task::(_debug) Task 82c53c6e-9ee9-433d-8e17-bf6604fe0dfc: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912898::DEBUG::2013-11-20 02:47:47,275::task::495::TaskManager.Task::(_debug) Task 82c53c6e-9ee9-433d-8e17-bf6604fe0dfc: moving from state preparing -> state finished
>Thread-15912898::DEBUG::2013-11-20 02:47:47,276::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912898::DEBUG::2013-11-20 02:47:47,276::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912898::DEBUG::2013-11-20 02:47:47,277::task::495::TaskManager.Task::(_debug) Task 82c53c6e-9ee9-433d-8e17-bf6604fe0dfc: ref 0 aborting False
>Thread-15912898::INFO::2013-11-20 02:47:47,277::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912899::DEBUG::2013-11-20 02:47:47,281::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912899::INFO::2013-11-20 02:47:47,282::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912899::DEBUG::2013-11-20 02:47:47,283::task::495::TaskManager.Task::(_debug) Task 1ab95e21-4ab8-445d-a39f-1f18b7386cdc: moving from state init -> state preparing
>Thread-15912899::DEBUG::2013-11-20 02:47:47,284::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`233643ba-8c17-4474-8431-f067d909ccb8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912899::DEBUG::2013-11-20 02:47:47,284::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912899::DEBUG::2013-11-20 02:47:47,285::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912899::DEBUG::2013-11-20 02:47:47,286::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`233643ba-8c17-4474-8431-f067d909ccb8`::Granted request
>Thread-15912899::DEBUG::2013-11-20 02:47:47,287::task::495::TaskManager.Task::(_debug) Task 1ab95e21-4ab8-445d-a39f-1f18b7386cdc: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912899::DEBUG::2013-11-20 02:47:47,287::task::495::TaskManager.Task::(_debug) Task 1ab95e21-4ab8-445d-a39f-1f18b7386cdc: ref 1 aborting False
>Thread-15912899::WARNING::2013-11-20 02:47:47,289::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912899::DEBUG::2013-11-20 02:47:47,290::task::495::TaskManager.Task::(_debug) Task 1ab95e21-4ab8-445d-a39f-1f18b7386cdc: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115774324736', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115768033280', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912899::DEBUG::2013-11-20 02:47:47,291::task::495::TaskManager.Task::(_debug) Task 1ab95e21-4ab8-445d-a39f-1f18b7386cdc: moving from state preparing -> state finished
>Thread-15912899::DEBUG::2013-11-20 02:47:47,291::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912899::DEBUG::2013-11-20 02:47:47,292::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912899::DEBUG::2013-11-20 02:47:47,293::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912899::DEBUG::2013-11-20 02:47:47,293::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912899::DEBUG::2013-11-20 02:47:47,294::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912899::DEBUG::2013-11-20 02:47:47,294::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912899::DEBUG::2013-11-20 02:47:47,295::task::495::TaskManager.Task::(_debug) Task 1ab95e21-4ab8-445d-a39f-1f18b7386cdc: ref 0 aborting False
>Thread-15912899::INFO::2013-11-20 02:47:47,295::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115774324736', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115768033280', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:47:47,320::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:47:47,869::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.069993 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:47:49,885::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:47:50,383::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0745916 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:47:52,396::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:47:52,938::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0825444 s, 12.4 MB/s\n'; <rc> = 0
>Thread-15912902::INFO::2013-11-20 02:47:54,087::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912902::DEBUG::2013-11-20 02:47:54,087::task::495::TaskManager.Task::(_debug) Task 835c7ddc-46b8-45d8-ad69-ad94b2adacd3: moving from state init -> state preparing
>Thread-15912902::DEBUG::2013-11-20 02:47:54,088::task::495::TaskManager.Task::(_debug) Task 835c7ddc-46b8-45d8-ad69-ad94b2adacd3: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000506162643433', 'lastCheck': 1384915673.6298251, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00144195556641', 'lastCheck': 1384915672.634712, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00142002105713', 'lastCheck': 1384915663.9306581, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000601053237915', 'lastCheck': 1384915671.5508289, 'valid': True, 'code': 0}}
>Thread-15912902::DEBUG::2013-11-20 02:47:54,088::task::495::TaskManager.Task::(_debug) Task 835c7ddc-46b8-45d8-ad69-ad94b2adacd3: moving from state preparing -> state finished
>Thread-15912902::DEBUG::2013-11-20 02:47:54,088::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912902::DEBUG::2013-11-20 02:47:54,089::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912902::DEBUG::2013-11-20 02:47:54,089::task::495::TaskManager.Task::(_debug) Task 835c7ddc-46b8-45d8-ad69-ad94b2adacd3: ref 0 aborting False
>Thread-15912902::INFO::2013-11-20 02:47:54,090::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000506162643433', 'lastCheck': 1384915673.6298251, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00144195556641', 'lastCheck': 1384915672.634712, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00142002105713', 'lastCheck': 1384915663.9306581, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000601053237915', 'lastCheck': 1384915671.5508289, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:47:54,953::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:47:55,467::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0774528 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:47:57,482::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:47:57,977::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0462891 s, 22.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:47:59,988::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:48:00,484::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0589922 s, 17.4 MB/s\n'; <rc> = 0
>Thread-15912904::DEBUG::2013-11-20 02:48:00,887::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912904::INFO::2013-11-20 02:48:00,888::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912904::DEBUG::2013-11-20 02:48:00,888::task::495::TaskManager.Task::(_debug) Task a538fad0-f296-4630-b29e-2586f9f9f828: moving from state init -> state preparing
>Thread-15912904::DEBUG::2013-11-20 02:48:00,889::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912904::DEBUG::2013-11-20 02:48:00,889::task::495::TaskManager.Task::(_debug) Task a538fad0-f296-4630-b29e-2586f9f9f828: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912904::DEBUG::2013-11-20 02:48:00,889::task::495::TaskManager.Task::(_debug) Task a538fad0-f296-4630-b29e-2586f9f9f828: moving from state preparing -> state finished
>Thread-15912904::DEBUG::2013-11-20 02:48:00,890::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912904::DEBUG::2013-11-20 02:48:00,890::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912904::DEBUG::2013-11-20 02:48:00,890::task::495::TaskManager.Task::(_debug) Task a538fad0-f296-4630-b29e-2586f9f9f828: ref 0 aborting False
>Thread-15912904::INFO::2013-11-20 02:48:00,891::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912905::DEBUG::2013-11-20 02:48:00,895::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912905::INFO::2013-11-20 02:48:00,896::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912905::DEBUG::2013-11-20 02:48:00,896::task::495::TaskManager.Task::(_debug) Task 8bbea363-570e-49df-8870-00a63372e6dd: moving from state init -> state preparing
>Thread-15912905::DEBUG::2013-11-20 02:48:00,897::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`637aee6f-b9d4-494f-9cd5-a5e6431aa574`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912905::DEBUG::2013-11-20 02:48:00,898::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912905::DEBUG::2013-11-20 02:48:00,899::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912905::DEBUG::2013-11-20 02:48:00,899::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`637aee6f-b9d4-494f-9cd5-a5e6431aa574`::Granted request
>Thread-15912905::DEBUG::2013-11-20 02:48:00,900::task::495::TaskManager.Task::(_debug) Task 8bbea363-570e-49df-8870-00a63372e6dd: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912905::DEBUG::2013-11-20 02:48:00,901::task::495::TaskManager.Task::(_debug) Task 8bbea363-570e-49df-8870-00a63372e6dd: ref 1 aborting False
>Thread-15912905::WARNING::2013-11-20 02:48:00,903::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912905::DEBUG::2013-11-20 02:48:00,904::task::495::TaskManager.Task::(_debug) Task 8bbea363-570e-49df-8870-00a63372e6dd: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115768033280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115768033280', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912905::DEBUG::2013-11-20 02:48:00,904::task::495::TaskManager.Task::(_debug) Task 8bbea363-570e-49df-8870-00a63372e6dd: moving from state preparing -> state finished
>Thread-15912905::DEBUG::2013-11-20 02:48:00,905::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912905::DEBUG::2013-11-20 02:48:00,906::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912905::DEBUG::2013-11-20 02:48:00,906::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912905::DEBUG::2013-11-20 02:48:00,907::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912905::DEBUG::2013-11-20 02:48:00,907::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912905::DEBUG::2013-11-20 02:48:00,908::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912905::DEBUG::2013-11-20 02:48:00,908::task::495::TaskManager.Task::(_debug) Task 8bbea363-570e-49df-8870-00a63372e6dd: ref 0 aborting False
>Thread-15912905::INFO::2013-11-20 02:48:00,909::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115768033280', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115768033280', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:48:02,494::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:48:03,001::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0749928 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:48:05,016::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 02:48:05,206::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 02:48:05,207::task::495::TaskManager.Task::(_debug) Task b0155bf9-3620-43b0-bf64-848332995c9f: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 02:48:05,209::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e35b3af3-cca0-42a4-9923-a523df63c811`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 02:48:05,209::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 02:48:05,210::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 02:48:05,210::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e35b3af3-cca0-42a4-9923-a523df63c811`::Granted request
>Thread-15912392::DEBUG::2013-11-20 02:48:05,211::task::495::TaskManager.Task::(_debug) Task b0155bf9-3620-43b0-bf64-848332995c9f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 02:48:05,212::task::495::TaskManager.Task::(_debug) Task b0155bf9-3620-43b0-bf64-848332995c9f: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 02:48:05,213::task::495::TaskManager.Task::(_debug) Task b0155bf9-3620-43b0-bf64-848332995c9f: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 02:48:05,214::task::495::TaskManager.Task::(_debug) Task b0155bf9-3620-43b0-bf64-848332995c9f: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 02:48:05,214::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 02:48:05,214::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 02:48:05,215::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 02:48:05,215::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 02:48:05,216::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 02:48:05,216::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 02:48:05,217::task::495::TaskManager.Task::(_debug) Task b0155bf9-3620-43b0-bf64-848332995c9f: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 02:48:05,217::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 02:48:05,427::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0460429 s, 22.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:48:07,437::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:48:07,944::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0718424 s, 14.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:48:09,958::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:48:10,422::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0602684 s, 17.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:48:12,434::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:48:12,901::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0585298 s, 17.5 MB/s\n'; <rc> = 0
>Thread-15912910::INFO::2013-11-20 02:48:13,720::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912910::DEBUG::2013-11-20 02:48:13,722::task::495::TaskManager.Task::(_debug) Task 9e302beb-bfbe-4f48-837f-6a7b4cdc12b9: moving from state init -> state preparing
>Thread-15912910::DEBUG::2013-11-20 02:48:13,724::task::495::TaskManager.Task::(_debug) Task 9e302beb-bfbe-4f48-837f-6a7b4cdc12b9: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000957012176514', 'lastCheck': 1384915683.707859, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00185990333557', 'lastCheck': 1384915692.7397521, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0121049880981', 'lastCheck': 1384915684.2129991, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000515937805176', 'lastCheck': 1384915691.718925, 'valid': True, 'code': 0}}
>Thread-15912910::DEBUG::2013-11-20 02:48:13,725::task::495::TaskManager.Task::(_debug) Task 9e302beb-bfbe-4f48-837f-6a7b4cdc12b9: moving from state preparing -> state finished
>Thread-15912910::DEBUG::2013-11-20 02:48:13,726::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912910::DEBUG::2013-11-20 02:48:13,727::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912910::DEBUG::2013-11-20 02:48:13,727::task::495::TaskManager.Task::(_debug) Task 9e302beb-bfbe-4f48-837f-6a7b4cdc12b9: ref 0 aborting False
>Thread-15912910::INFO::2013-11-20 02:48:13,728::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000957012176514', 'lastCheck': 1384915683.707859, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00185990333557', 'lastCheck': 1384915692.7397521, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0121049880981', 'lastCheck': 1384915684.2129991, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000515937805176', 'lastCheck': 1384915691.718925, 'valid': True, 'code': 0}}
>Thread-15912912::DEBUG::2013-11-20 02:48:13,939::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912912::INFO::2013-11-20 02:48:13,940::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912912::DEBUG::2013-11-20 02:48:13,941::task::495::TaskManager.Task::(_debug) Task b3490019-88a8-44e2-a888-d1f45e4ce219: moving from state init -> state preparing
>Thread-15912912::DEBUG::2013-11-20 02:48:13,942::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912912::DEBUG::2013-11-20 02:48:13,942::task::495::TaskManager.Task::(_debug) Task b3490019-88a8-44e2-a888-d1f45e4ce219: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912912::DEBUG::2013-11-20 02:48:13,943::task::495::TaskManager.Task::(_debug) Task b3490019-88a8-44e2-a888-d1f45e4ce219: moving from state preparing -> state finished
>Thread-15912912::DEBUG::2013-11-20 02:48:13,944::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912912::DEBUG::2013-11-20 02:48:13,944::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912912::DEBUG::2013-11-20 02:48:13,945::task::495::TaskManager.Task::(_debug) Task b3490019-88a8-44e2-a888-d1f45e4ce219: ref 0 aborting False
>Thread-15912912::INFO::2013-11-20 02:48:13,946::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912913::DEBUG::2013-11-20 02:48:13,951::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912913::INFO::2013-11-20 02:48:13,952::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912913::DEBUG::2013-11-20 02:48:13,952::task::495::TaskManager.Task::(_debug) Task 7f78aaa5-50fd-4d38-80c2-3f7bd94cfde5: moving from state init -> state preparing
>Thread-15912913::DEBUG::2013-11-20 02:48:13,954::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ff693d3a-65cd-4b91-9119-d702a61bf76b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912913::DEBUG::2013-11-20 02:48:13,954::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912913::DEBUG::2013-11-20 02:48:13,955::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912913::DEBUG::2013-11-20 02:48:13,956::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ff693d3a-65cd-4b91-9119-d702a61bf76b`::Granted request
>Thread-15912913::DEBUG::2013-11-20 02:48:13,957::task::495::TaskManager.Task::(_debug) Task 7f78aaa5-50fd-4d38-80c2-3f7bd94cfde5: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912913::DEBUG::2013-11-20 02:48:13,958::task::495::TaskManager.Task::(_debug) Task 7f78aaa5-50fd-4d38-80c2-3f7bd94cfde5: ref 1 aborting False
>Thread-15912913::WARNING::2013-11-20 02:48:13,960::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912913::DEBUG::2013-11-20 02:48:13,961::task::495::TaskManager.Task::(_debug) Task 7f78aaa5-50fd-4d38-80c2-3f7bd94cfde5: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912913::DEBUG::2013-11-20 02:48:13,962::task::495::TaskManager.Task::(_debug) Task 7f78aaa5-50fd-4d38-80c2-3f7bd94cfde5: moving from state preparing -> state finished
>Thread-15912913::DEBUG::2013-11-20 02:48:13,963::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912913::DEBUG::2013-11-20 02:48:13,963::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912913::DEBUG::2013-11-20 02:48:13,964::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912913::DEBUG::2013-11-20 02:48:13,965::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912913::DEBUG::2013-11-20 02:48:13,965::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912913::DEBUG::2013-11-20 02:48:13,966::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912913::DEBUG::2013-11-20 02:48:13,967::task::495::TaskManager.Task::(_debug) Task 7f78aaa5-50fd-4d38-80c2-3f7bd94cfde5: ref 0 aborting False
>Thread-15912913::INFO::2013-11-20 02:48:13,967::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115765936128', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115763838976', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:48:14,916::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:48:15,430::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0645707 s, 15.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:48:17,442::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:48:17,958::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0694972 s, 14.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:48:19,973::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:48:20,444::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0438032 s, 23.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:48:22,454::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:48:22,987::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0728875 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:48:25,002::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:48:25,517::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0769763 s, 13.3 MB/s\n'; <rc> = 0
>Thread-15912916::DEBUG::2013-11-20 02:48:26,911::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912916::INFO::2013-11-20 02:48:26,912::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912916::DEBUG::2013-11-20 02:48:26,912::task::495::TaskManager.Task::(_debug) Task 88c979f3-ec82-49ca-8c42-f7b94cb4b99a: moving from state init -> state preparing
>Thread-15912916::DEBUG::2013-11-20 02:48:26,913::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912916::DEBUG::2013-11-20 02:48:26,913::task::495::TaskManager.Task::(_debug) Task 88c979f3-ec82-49ca-8c42-f7b94cb4b99a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912916::DEBUG::2013-11-20 02:48:26,913::task::495::TaskManager.Task::(_debug) Task 88c979f3-ec82-49ca-8c42-f7b94cb4b99a: moving from state preparing -> state finished
>Thread-15912916::DEBUG::2013-11-20 02:48:26,914::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912916::DEBUG::2013-11-20 02:48:26,914::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912916::DEBUG::2013-11-20 02:48:26,914::task::495::TaskManager.Task::(_debug) Task 88c979f3-ec82-49ca-8c42-f7b94cb4b99a: ref 0 aborting False
>Thread-15912916::INFO::2013-11-20 02:48:26,915::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912917::DEBUG::2013-11-20 02:48:26,918::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912917::INFO::2013-11-20 02:48:26,919::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912917::DEBUG::2013-11-20 02:48:26,919::task::495::TaskManager.Task::(_debug) Task 7354f00c-a1be-4a16-b7bb-32eb806d4d66: moving from state init -> state preparing
>Thread-15912917::DEBUG::2013-11-20 02:48:26,920::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ba7bec4f-d4e3-446c-99a1-234455b888cc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912917::DEBUG::2013-11-20 02:48:26,920::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912917::DEBUG::2013-11-20 02:48:26,921::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912917::DEBUG::2013-11-20 02:48:26,921::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ba7bec4f-d4e3-446c-99a1-234455b888cc`::Granted request
>Thread-15912917::DEBUG::2013-11-20 02:48:26,926::task::495::TaskManager.Task::(_debug) Task 7354f00c-a1be-4a16-b7bb-32eb806d4d66: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912917::DEBUG::2013-11-20 02:48:26,926::task::495::TaskManager.Task::(_debug) Task 7354f00c-a1be-4a16-b7bb-32eb806d4d66: ref 1 aborting False
>Thread-15912917::WARNING::2013-11-20 02:48:26,927::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912917::DEBUG::2013-11-20 02:48:26,928::task::495::TaskManager.Task::(_debug) Task 7354f00c-a1be-4a16-b7bb-32eb806d4d66: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115761741824', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115764887552', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912917::DEBUG::2013-11-20 02:48:26,928::task::495::TaskManager.Task::(_debug) Task 7354f00c-a1be-4a16-b7bb-32eb806d4d66: moving from state preparing -> state finished
>Thread-15912917::DEBUG::2013-11-20 02:48:26,929::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912917::DEBUG::2013-11-20 02:48:26,929::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912917::DEBUG::2013-11-20 02:48:26,929::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912917::DEBUG::2013-11-20 02:48:26,930::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912917::DEBUG::2013-11-20 02:48:26,930::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912917::DEBUG::2013-11-20 02:48:26,930::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912917::DEBUG::2013-11-20 02:48:26,931::task::495::TaskManager.Task::(_debug) Task 7354f00c-a1be-4a16-b7bb-32eb806d4d66: ref 0 aborting False
>Thread-15912917::INFO::2013-11-20 02:48:26,931::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115761741824', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115764887552', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:48:27,533::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:48:28,076::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0712208 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:48:30,091::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:48:30,513::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0453064 s, 22.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:48:32,523::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:48:33,024::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0708747 s, 14.4 MB/s\n'; <rc> = 0
>Thread-15912920::INFO::2013-11-20 02:48:33,043::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912920::DEBUG::2013-11-20 02:48:33,044::task::495::TaskManager.Task::(_debug) Task eccaea5c-8065-48e2-a598-444bc06a5d09: moving from state init -> state preparing
>Thread-15912920::DEBUG::2013-11-20 02:48:33,045::task::495::TaskManager.Task::(_debug) Task eccaea5c-8065-48e2-a598-444bc06a5d09: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000381946563721', 'lastCheck': 1384915703.741992, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00163292884827', 'lastCheck': 1384915712.780148, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00176095962524', 'lastCheck': 1384915704.251327, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000546932220459', 'lastCheck': 1384915711.796504, 'valid': True, 'code': 0}}
>Thread-15912920::DEBUG::2013-11-20 02:48:33,046::task::495::TaskManager.Task::(_debug) Task eccaea5c-8065-48e2-a598-444bc06a5d09: moving from state preparing -> state finished
>Thread-15912920::DEBUG::2013-11-20 02:48:33,046::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912920::DEBUG::2013-11-20 02:48:33,047::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912920::DEBUG::2013-11-20 02:48:33,047::task::495::TaskManager.Task::(_debug) Task eccaea5c-8065-48e2-a598-444bc06a5d09: ref 0 aborting False
>Thread-15912920::INFO::2013-11-20 02:48:33,048::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000381946563721', 'lastCheck': 1384915703.741992, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00163292884827', 'lastCheck': 1384915712.780148, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00176095962524', 'lastCheck': 1384915704.251327, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000546932220459', 'lastCheck': 1384915711.796504, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:48:35,042::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:48:35,573::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0729577 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:48:37,588::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:48:38,061::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0475533 s, 21.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:48:40,072::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:48:40,561::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0675272 s, 15.2 MB/s\n'; <rc> = 0
>Thread-42840::DEBUG::2013-11-20 02:48:41,808::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 02:48:42,430::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:48:42,577::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:48:43,111::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0739316 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:48:45,126::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:48:45,566::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0457288 s, 22.4 MB/s\n'; <rc> = 0
>Thread-15912925::DEBUG::2013-11-20 02:48:47,094::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912925::INFO::2013-11-20 02:48:47,095::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912925::DEBUG::2013-11-20 02:48:47,096::task::495::TaskManager.Task::(_debug) Task 18591aa5-7403-478f-b3c4-32bc3dc3a620: moving from state init -> state preparing
>Thread-15912925::DEBUG::2013-11-20 02:48:47,097::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912925::DEBUG::2013-11-20 02:48:47,097::task::495::TaskManager.Task::(_debug) Task 18591aa5-7403-478f-b3c4-32bc3dc3a620: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912925::DEBUG::2013-11-20 02:48:47,098::task::495::TaskManager.Task::(_debug) Task 18591aa5-7403-478f-b3c4-32bc3dc3a620: moving from state preparing -> state finished
>Thread-15912925::DEBUG::2013-11-20 02:48:47,098::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912925::DEBUG::2013-11-20 02:48:47,099::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912925::DEBUG::2013-11-20 02:48:47,099::task::495::TaskManager.Task::(_debug) Task 18591aa5-7403-478f-b3c4-32bc3dc3a620: ref 0 aborting False
>Thread-15912925::INFO::2013-11-20 02:48:47,100::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912926::DEBUG::2013-11-20 02:48:47,104::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912926::INFO::2013-11-20 02:48:47,105::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912926::DEBUG::2013-11-20 02:48:47,105::task::495::TaskManager.Task::(_debug) Task d6c20981-d3ac-4e3c-9518-1584340dd6ce: moving from state init -> state preparing
>Thread-15912926::DEBUG::2013-11-20 02:48:47,107::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3aa8c541-dc85-4240-9991-65358ba51633`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912926::DEBUG::2013-11-20 02:48:47,107::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912926::DEBUG::2013-11-20 02:48:47,107::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912926::DEBUG::2013-11-20 02:48:47,108::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3aa8c541-dc85-4240-9991-65358ba51633`::Granted request
>Thread-15912926::DEBUG::2013-11-20 02:48:47,108::task::495::TaskManager.Task::(_debug) Task d6c20981-d3ac-4e3c-9518-1584340dd6ce: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912926::DEBUG::2013-11-20 02:48:47,109::task::495::TaskManager.Task::(_debug) Task d6c20981-d3ac-4e3c-9518-1584340dd6ce: ref 1 aborting False
>Thread-15912926::WARNING::2013-11-20 02:48:47,110::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912926::DEBUG::2013-11-20 02:48:47,110::task::495::TaskManager.Task::(_debug) Task d6c20981-d3ac-4e3c-9518-1584340dd6ce: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115559366656', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115559366656', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912926::DEBUG::2013-11-20 02:48:47,111::task::495::TaskManager.Task::(_debug) Task d6c20981-d3ac-4e3c-9518-1584340dd6ce: moving from state preparing -> state finished
>Thread-15912926::DEBUG::2013-11-20 02:48:47,111::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912926::DEBUG::2013-11-20 02:48:47,112::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912926::DEBUG::2013-11-20 02:48:47,112::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912926::DEBUG::2013-11-20 02:48:47,112::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912926::DEBUG::2013-11-20 02:48:47,113::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912926::DEBUG::2013-11-20 02:48:47,113::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912926::DEBUG::2013-11-20 02:48:47,113::task::495::TaskManager.Task::(_debug) Task d6c20981-d3ac-4e3c-9518-1584340dd6ce: ref 0 aborting False
>Thread-15912926::INFO::2013-11-20 02:48:47,114::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115559366656', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115559366656', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:48:47,576::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:48:48,081::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0523581 s, 19.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:48:50,097::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:48:50,625::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0695586 s, 14.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:48:52,638::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912928::INFO::2013-11-20 02:48:52,913::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912928::DEBUG::2013-11-20 02:48:52,915::task::495::TaskManager.Task::(_debug) Task 9e158a02-a890-43ab-8a48-2d561ced011e: moving from state init -> state preparing
>Thread-15912928::DEBUG::2013-11-20 02:48:52,916::task::495::TaskManager.Task::(_debug) Task 9e158a02-a890-43ab-8a48-2d561ced011e: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000432014465332', 'lastCheck': 1384915724.0141771, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00150203704834', 'lastCheck': 1384915722.8505931, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.002277135849', 'lastCheck': 1384915724.3242149, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000415802001953', 'lastCheck': 1384915732.4435279, 'valid': True, 'code': 0}}
>Thread-15912928::DEBUG::2013-11-20 02:48:52,917::task::495::TaskManager.Task::(_debug) Task 9e158a02-a890-43ab-8a48-2d561ced011e: moving from state preparing -> state finished
>Thread-15912928::DEBUG::2013-11-20 02:48:52,918::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912928::DEBUG::2013-11-20 02:48:52,918::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912928::DEBUG::2013-11-20 02:48:52,919::task::495::TaskManager.Task::(_debug) Task 9e158a02-a890-43ab-8a48-2d561ced011e: ref 0 aborting False
>Thread-15912928::INFO::2013-11-20 02:48:52,919::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000432014465332', 'lastCheck': 1384915724.0141771, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00150203704834', 'lastCheck': 1384915722.8505931, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.002277135849', 'lastCheck': 1384915724.3242149, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000415802001953', 'lastCheck': 1384915732.4435279, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:48:53,207::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0738321 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:48:55,222::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:48:55,653::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0456371 s, 22.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:48:57,663::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:48:58,215::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.074524 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:49:00,231::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:49:00,768::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0751032 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:49:02,783::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:49:03,323::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.072361 s, 14.2 MB/s\n'; <rc> = 0
>Thread-15912933::DEBUG::2013-11-20 02:49:04,438::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912933::INFO::2013-11-20 02:49:04,440::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912933::DEBUG::2013-11-20 02:49:04,441::task::495::TaskManager.Task::(_debug) Task 59e09a00-f76e-4bcd-b34f-5f55e3b8297f: moving from state init -> state preparing
>Thread-15912933::DEBUG::2013-11-20 02:49:04,441::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912933::DEBUG::2013-11-20 02:49:04,442::task::495::TaskManager.Task::(_debug) Task 59e09a00-f76e-4bcd-b34f-5f55e3b8297f: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912933::DEBUG::2013-11-20 02:49:04,443::task::495::TaskManager.Task::(_debug) Task 59e09a00-f76e-4bcd-b34f-5f55e3b8297f: moving from state preparing -> state finished
>Thread-15912933::DEBUG::2013-11-20 02:49:04,443::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912933::DEBUG::2013-11-20 02:49:04,444::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912933::DEBUG::2013-11-20 02:49:04,444::task::495::TaskManager.Task::(_debug) Task 59e09a00-f76e-4bcd-b34f-5f55e3b8297f: ref 0 aborting False
>Thread-15912933::INFO::2013-11-20 02:49:04,445::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912934::DEBUG::2013-11-20 02:49:04,565::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912934::INFO::2013-11-20 02:49:04,566::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912934::DEBUG::2013-11-20 02:49:04,566::task::495::TaskManager.Task::(_debug) Task c8144f8a-3f57-44c7-8dd8-62b60cdad1b3: moving from state init -> state preparing
>Thread-15912934::DEBUG::2013-11-20 02:49:04,567::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f3542e8a-e2b7-4c46-b08d-f9e299c5f364`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912934::DEBUG::2013-11-20 02:49:04,567::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912934::DEBUG::2013-11-20 02:49:04,568::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912934::DEBUG::2013-11-20 02:49:04,568::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f3542e8a-e2b7-4c46-b08d-f9e299c5f364`::Granted request
>Thread-15912934::DEBUG::2013-11-20 02:49:04,569::task::495::TaskManager.Task::(_debug) Task c8144f8a-3f57-44c7-8dd8-62b60cdad1b3: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912934::DEBUG::2013-11-20 02:49:04,569::task::495::TaskManager.Task::(_debug) Task c8144f8a-3f57-44c7-8dd8-62b60cdad1b3: ref 1 aborting False
>Thread-15912934::WARNING::2013-11-20 02:49:04,570::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912934::DEBUG::2013-11-20 02:49:04,571::task::495::TaskManager.Task::(_debug) Task c8144f8a-3f57-44c7-8dd8-62b60cdad1b3: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115559366656', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115559366656', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912934::DEBUG::2013-11-20 02:49:04,571::task::495::TaskManager.Task::(_debug) Task c8144f8a-3f57-44c7-8dd8-62b60cdad1b3: moving from state preparing -> state finished
>Thread-15912934::DEBUG::2013-11-20 02:49:04,572::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912934::DEBUG::2013-11-20 02:49:04,572::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912934::DEBUG::2013-11-20 02:49:04,573::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912934::DEBUG::2013-11-20 02:49:04,573::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912934::DEBUG::2013-11-20 02:49:04,573::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912934::DEBUG::2013-11-20 02:49:04,574::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912934::DEBUG::2013-11-20 02:49:04,574::task::495::TaskManager.Task::(_debug) Task c8144f8a-3f57-44c7-8dd8-62b60cdad1b3: ref 0 aborting False
>Thread-15912934::INFO::2013-11-20 02:49:04,575::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115559366656', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115559366656', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:49:05,338::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 02:49:05,547::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 02:49:05,549::task::495::TaskManager.Task::(_debug) Task e6b34105-9a41-4e12-b8f1-a55370e01b44: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 02:49:05,551::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c712803f-1805-4b83-a27f-34c8e103a453`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 02:49:05,552::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 02:49:05,553::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 02:49:05,554::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c712803f-1805-4b83-a27f-34c8e103a453`::Granted request
>Thread-15912392::DEBUG::2013-11-20 02:49:05,555::task::495::TaskManager.Task::(_debug) Task e6b34105-9a41-4e12-b8f1-a55370e01b44: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 02:49:05,555::task::495::TaskManager.Task::(_debug) Task e6b34105-9a41-4e12-b8f1-a55370e01b44: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 02:49:05,556::task::495::TaskManager.Task::(_debug) Task e6b34105-9a41-4e12-b8f1-a55370e01b44: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 02:49:05,557::task::495::TaskManager.Task::(_debug) Task e6b34105-9a41-4e12-b8f1-a55370e01b44: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 02:49:05,557::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 02:49:05,558::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 02:49:05,559::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 02:49:05,559::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 02:49:05,560::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 02:49:05,560::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 02:49:05,561::task::495::TaskManager.Task::(_debug) Task e6b34105-9a41-4e12-b8f1-a55370e01b44: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 02:49:05,561::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 02:49:05,858::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0724051 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:49:07,872::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:49:08,397::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0844596 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:49:10,412::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912936::INFO::2013-11-20 02:49:10,658::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912936::DEBUG::2013-11-20 02:49:10,659::task::495::TaskManager.Task::(_debug) Task 7d68041a-f284-4e10-a73f-562e0979b132: moving from state init -> state preparing
>Thread-15912936::DEBUG::2013-11-20 02:49:10,659::task::495::TaskManager.Task::(_debug) Task 7d68041a-f284-4e10-a73f-562e0979b132: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000304937362671', 'lastCheck': 1384915744.039022, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00146698951721', 'lastCheck': 1384915743.015085, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00161600112915', 'lastCheck': 1384915744.4627481, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000468969345093', 'lastCheck': 1384915742.4597249, 'valid': True, 'code': 0}}
>Thread-15912936::DEBUG::2013-11-20 02:49:10,660::task::495::TaskManager.Task::(_debug) Task 7d68041a-f284-4e10-a73f-562e0979b132: moving from state preparing -> state finished
>Thread-15912936::DEBUG::2013-11-20 02:49:10,661::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912936::DEBUG::2013-11-20 02:49:10,661::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912936::DEBUG::2013-11-20 02:49:10,662::task::495::TaskManager.Task::(_debug) Task 7d68041a-f284-4e10-a73f-562e0979b132: ref 0 aborting False
>Thread-15912936::INFO::2013-11-20 02:49:10,662::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000304937362671', 'lastCheck': 1384915744.039022, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00146698951721', 'lastCheck': 1384915743.015085, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00161600112915', 'lastCheck': 1384915744.4627481, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000468969345093', 'lastCheck': 1384915742.4597249, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:49:10,952::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0736196 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:49:12,967::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:49:13,485::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0725883 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:49:15,500::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:49:16,082::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.1084 s, 9.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:49:18,098::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:49:18,644::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0757122 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:49:20,660::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:49:21,217::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0747138 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15912941::DEBUG::2013-11-20 02:49:22,075::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912941::INFO::2013-11-20 02:49:22,076::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912941::DEBUG::2013-11-20 02:49:22,077::task::495::TaskManager.Task::(_debug) Task cb5f460f-b180-4bcf-a164-a6cad1d6b9c2: moving from state init -> state preparing
>Thread-15912941::DEBUG::2013-11-20 02:49:22,077::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912941::DEBUG::2013-11-20 02:49:22,078::task::495::TaskManager.Task::(_debug) Task cb5f460f-b180-4bcf-a164-a6cad1d6b9c2: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912941::DEBUG::2013-11-20 02:49:22,078::task::495::TaskManager.Task::(_debug) Task cb5f460f-b180-4bcf-a164-a6cad1d6b9c2: moving from state preparing -> state finished
>Thread-15912941::DEBUG::2013-11-20 02:49:22,079::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912941::DEBUG::2013-11-20 02:49:22,079::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912941::DEBUG::2013-11-20 02:49:22,080::task::495::TaskManager.Task::(_debug) Task cb5f460f-b180-4bcf-a164-a6cad1d6b9c2: ref 0 aborting False
>Thread-15912941::INFO::2013-11-20 02:49:22,080::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912942::DEBUG::2013-11-20 02:49:22,085::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912942::INFO::2013-11-20 02:49:22,086::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912942::DEBUG::2013-11-20 02:49:22,087::task::495::TaskManager.Task::(_debug) Task ea7ee8bf-8b11-46af-b1c2-c2f958950dc2: moving from state init -> state preparing
>Thread-15912942::DEBUG::2013-11-20 02:49:22,089::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`01a04fd9-0160-4a87-9466-2da04b4a8cea`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912942::DEBUG::2013-11-20 02:49:22,089::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912942::DEBUG::2013-11-20 02:49:22,090::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912942::DEBUG::2013-11-20 02:49:22,091::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`01a04fd9-0160-4a87-9466-2da04b4a8cea`::Granted request
>Thread-15912942::DEBUG::2013-11-20 02:49:22,093::task::495::TaskManager.Task::(_debug) Task ea7ee8bf-8b11-46af-b1c2-c2f958950dc2: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912942::DEBUG::2013-11-20 02:49:22,093::task::495::TaskManager.Task::(_debug) Task ea7ee8bf-8b11-46af-b1c2-c2f958950dc2: ref 1 aborting False
>Thread-15912942::WARNING::2013-11-20 02:49:22,096::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912942::DEBUG::2013-11-20 02:49:22,097::task::495::TaskManager.Task::(_debug) Task ea7ee8bf-8b11-46af-b1c2-c2f958950dc2: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115559366656', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115559366656', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912942::DEBUG::2013-11-20 02:49:22,099::task::495::TaskManager.Task::(_debug) Task ea7ee8bf-8b11-46af-b1c2-c2f958950dc2: moving from state preparing -> state finished
>Thread-15912942::DEBUG::2013-11-20 02:49:22,100::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912942::DEBUG::2013-11-20 02:49:22,101::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912942::DEBUG::2013-11-20 02:49:22,102::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912942::DEBUG::2013-11-20 02:49:22,103::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912942::DEBUG::2013-11-20 02:49:22,104::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912942::DEBUG::2013-11-20 02:49:22,105::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912942::DEBUG::2013-11-20 02:49:22,106::task::495::TaskManager.Task::(_debug) Task ea7ee8bf-8b11-46af-b1c2-c2f958950dc2: ref 0 aborting False
>Thread-15912942::INFO::2013-11-20 02:49:22,107::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115559366656', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115559366656', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:49:23,233::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:49:23,763::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0751059 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:49:25,779::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:49:26,347::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0764128 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15912944::INFO::2013-11-20 02:49:26,924::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912944::DEBUG::2013-11-20 02:49:26,925::task::495::TaskManager.Task::(_debug) Task 4f037215-8796-46ff-b2b9-68905fb15c60: moving from state init -> state preparing
>Thread-15912944::DEBUG::2013-11-20 02:49:26,926::task::495::TaskManager.Task::(_debug) Task 4f037215-8796-46ff-b2b9-68905fb15c60: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000745058059692', 'lastCheck': 1384915764.456069, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0013439655304', 'lastCheck': 1384915763.471374, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00153684616089', 'lastCheck': 1384915764.893177, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00042200088501', 'lastCheck': 1384915762.8853631, 'valid': True, 'code': 0}}
>Thread-15912944::DEBUG::2013-11-20 02:49:26,927::task::495::TaskManager.Task::(_debug) Task 4f037215-8796-46ff-b2b9-68905fb15c60: moving from state preparing -> state finished
>Thread-15912944::DEBUG::2013-11-20 02:49:26,928::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912944::DEBUG::2013-11-20 02:49:26,928::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912944::DEBUG::2013-11-20 02:49:26,929::task::495::TaskManager.Task::(_debug) Task 4f037215-8796-46ff-b2b9-68905fb15c60: ref 0 aborting False
>Thread-15912944::INFO::2013-11-20 02:49:26,929::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000745058059692', 'lastCheck': 1384915764.456069, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0013439655304', 'lastCheck': 1384915763.471374, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00153684616089', 'lastCheck': 1384915764.893177, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00042200088501', 'lastCheck': 1384915762.8853631, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:49:28,361::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:49:28,870::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0758551 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:49:30,885::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:49:31,427::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0640628 s, 16.0 MB/s\n'; <rc> = 0
>Thread-15912947::DEBUG::2013-11-20 02:49:32,860::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912947::INFO::2013-11-20 02:49:32,861::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912947::DEBUG::2013-11-20 02:49:32,862::task::495::TaskManager.Task::(_debug) Task 55f67a0b-55bd-46c5-8bc5-27fcbcd14af1: moving from state init -> state preparing
>Thread-15912947::DEBUG::2013-11-20 02:49:32,863::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912947::DEBUG::2013-11-20 02:49:32,863::task::495::TaskManager.Task::(_debug) Task 55f67a0b-55bd-46c5-8bc5-27fcbcd14af1: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912947::DEBUG::2013-11-20 02:49:32,864::task::495::TaskManager.Task::(_debug) Task 55f67a0b-55bd-46c5-8bc5-27fcbcd14af1: moving from state preparing -> state finished
>Thread-15912947::DEBUG::2013-11-20 02:49:32,864::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912947::DEBUG::2013-11-20 02:49:32,864::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912947::DEBUG::2013-11-20 02:49:32,865::task::495::TaskManager.Task::(_debug) Task 55f67a0b-55bd-46c5-8bc5-27fcbcd14af1: ref 0 aborting False
>Thread-15912947::INFO::2013-11-20 02:49:32,865::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912948::DEBUG::2013-11-20 02:49:32,869::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912948::INFO::2013-11-20 02:49:32,870::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912948::DEBUG::2013-11-20 02:49:32,870::task::495::TaskManager.Task::(_debug) Task ae99fd1d-4045-4811-95af-e92ff6cfb162: moving from state init -> state preparing
>Thread-15912948::DEBUG::2013-11-20 02:49:32,871::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d01ca06d-d53f-42f3-a27c-527f34dda141`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912948::DEBUG::2013-11-20 02:49:32,872::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912948::DEBUG::2013-11-20 02:49:32,872::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912948::DEBUG::2013-11-20 02:49:32,873::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d01ca06d-d53f-42f3-a27c-527f34dda141`::Granted request
>Thread-15912948::DEBUG::2013-11-20 02:49:32,874::task::495::TaskManager.Task::(_debug) Task ae99fd1d-4045-4811-95af-e92ff6cfb162: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912948::DEBUG::2013-11-20 02:49:32,874::task::495::TaskManager.Task::(_debug) Task ae99fd1d-4045-4811-95af-e92ff6cfb162: ref 1 aborting False
>Thread-15912948::WARNING::2013-11-20 02:49:32,876::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912948::DEBUG::2013-11-20 02:49:32,876::task::495::TaskManager.Task::(_debug) Task ae99fd1d-4045-4811-95af-e92ff6cfb162: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115559366656', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115559366656', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912948::DEBUG::2013-11-20 02:49:32,877::task::495::TaskManager.Task::(_debug) Task ae99fd1d-4045-4811-95af-e92ff6cfb162: moving from state preparing -> state finished
>Thread-15912948::DEBUG::2013-11-20 02:49:32,877::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912948::DEBUG::2013-11-20 02:49:32,878::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912948::DEBUG::2013-11-20 02:49:32,878::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912948::DEBUG::2013-11-20 02:49:32,879::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912948::DEBUG::2013-11-20 02:49:32,879::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912948::DEBUG::2013-11-20 02:49:32,879::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912948::DEBUG::2013-11-20 02:49:32,880::task::495::TaskManager.Task::(_debug) Task ae99fd1d-4045-4811-95af-e92ff6cfb162: ref 0 aborting False
>Thread-15912948::INFO::2013-11-20 02:49:32,880::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115559366656', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115559366656', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:49:33,442::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:49:33,997::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0725894 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:49:36,013::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:49:36,544::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0728346 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:49:38,559::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:49:39,086::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0738477 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:49:41,100::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:49:41,613::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0737127 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:49:43,628::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:49:44,174::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0648222 s, 15.8 MB/s\n'; <rc> = 0
>Thread-15912952::INFO::2013-11-20 02:49:44,351::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912952::DEBUG::2013-11-20 02:49:44,352::task::495::TaskManager.Task::(_debug) Task e3d7ade4-548f-4bd9-a797-4d608caab67a: moving from state init -> state preparing
>Thread-15912952::DEBUG::2013-11-20 02:49:44,353::task::495::TaskManager.Task::(_debug) Task e3d7ade4-548f-4bd9-a797-4d608caab67a: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000440835952759', 'lastCheck': 1384915774.6865561, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00125598907471', 'lastCheck': 1384915783.8640771, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00144410133362', 'lastCheck': 1384915775.1432559, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000496864318848', 'lastCheck': 1384915783.313942, 'valid': True, 'code': 0}}
>Thread-15912952::DEBUG::2013-11-20 02:49:44,354::task::495::TaskManager.Task::(_debug) Task e3d7ade4-548f-4bd9-a797-4d608caab67a: moving from state preparing -> state finished
>Thread-15912952::DEBUG::2013-11-20 02:49:44,354::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912952::DEBUG::2013-11-20 02:49:44,355::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912952::DEBUG::2013-11-20 02:49:44,356::task::495::TaskManager.Task::(_debug) Task e3d7ade4-548f-4bd9-a797-4d608caab67a: ref 0 aborting False
>Thread-15912952::INFO::2013-11-20 02:49:44,356::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000440835952759', 'lastCheck': 1384915774.6865561, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00125598907471', 'lastCheck': 1384915783.8640771, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00144410133362', 'lastCheck': 1384915775.1432559, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000496864318848', 'lastCheck': 1384915783.313942, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:49:46,187::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:49:46,668::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0594174 s, 17.2 MB/s\n'; <rc> = 0
>Thread-15912954::DEBUG::2013-11-20 02:49:46,858::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912954::INFO::2013-11-20 02:49:46,859::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912954::DEBUG::2013-11-20 02:49:46,859::task::495::TaskManager.Task::(_debug) Task 97fee814-ea6e-49c1-8448-257bd4f8b24d: moving from state init -> state preparing
>Thread-15912954::DEBUG::2013-11-20 02:49:46,861::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912954::DEBUG::2013-11-20 02:49:46,862::task::495::TaskManager.Task::(_debug) Task 97fee814-ea6e-49c1-8448-257bd4f8b24d: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912954::DEBUG::2013-11-20 02:49:46,862::task::495::TaskManager.Task::(_debug) Task 97fee814-ea6e-49c1-8448-257bd4f8b24d: moving from state preparing -> state finished
>Thread-15912954::DEBUG::2013-11-20 02:49:46,863::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912954::DEBUG::2013-11-20 02:49:46,863::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912954::DEBUG::2013-11-20 02:49:46,863::task::495::TaskManager.Task::(_debug) Task 97fee814-ea6e-49c1-8448-257bd4f8b24d: ref 0 aborting False
>Thread-15912954::INFO::2013-11-20 02:49:46,864::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912955::DEBUG::2013-11-20 02:49:46,867::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912955::INFO::2013-11-20 02:49:46,868::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912955::DEBUG::2013-11-20 02:49:46,868::task::495::TaskManager.Task::(_debug) Task bceaee84-dbf2-4d9e-8caa-b4f5cf155a86: moving from state init -> state preparing
>Thread-15912955::DEBUG::2013-11-20 02:49:46,869::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`fa1dc8b0-5669-44db-a8f0-b41921ae28af`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912955::DEBUG::2013-11-20 02:49:46,869::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912955::DEBUG::2013-11-20 02:49:46,870::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912955::DEBUG::2013-11-20 02:49:46,870::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`fa1dc8b0-5669-44db-a8f0-b41921ae28af`::Granted request
>Thread-15912955::DEBUG::2013-11-20 02:49:46,871::task::495::TaskManager.Task::(_debug) Task bceaee84-dbf2-4d9e-8caa-b4f5cf155a86: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912955::DEBUG::2013-11-20 02:49:46,871::task::495::TaskManager.Task::(_debug) Task bceaee84-dbf2-4d9e-8caa-b4f5cf155a86: ref 1 aborting False
>Thread-15912955::WARNING::2013-11-20 02:49:46,872::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912955::DEBUG::2013-11-20 02:49:46,873::task::495::TaskManager.Task::(_debug) Task bceaee84-dbf2-4d9e-8caa-b4f5cf155a86: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115559366656', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115559366656', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912955::DEBUG::2013-11-20 02:49:46,874::task::495::TaskManager.Task::(_debug) Task bceaee84-dbf2-4d9e-8caa-b4f5cf155a86: moving from state preparing -> state finished
>Thread-15912955::DEBUG::2013-11-20 02:49:46,874::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912955::DEBUG::2013-11-20 02:49:46,874::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912955::DEBUG::2013-11-20 02:49:46,875::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912955::DEBUG::2013-11-20 02:49:46,875::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912955::DEBUG::2013-11-20 02:49:46,875::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912955::DEBUG::2013-11-20 02:49:46,876::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912955::DEBUG::2013-11-20 02:49:46,876::task::495::TaskManager.Task::(_debug) Task bceaee84-dbf2-4d9e-8caa-b4f5cf155a86: ref 0 aborting False
>Thread-15912955::INFO::2013-11-20 02:49:46,876::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115559366656', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115559366656', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:49:48,684::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:49:49,246::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.100836 s, 10.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:49:51,260::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:49:51,799::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0526211 s, 19.5 MB/s\n'; <rc> = 0
>Thread-15912956::DEBUG::2013-11-20 02:49:53,492::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912956::INFO::2013-11-20 02:49:53,494::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getIsoList, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912956::DEBUG::2013-11-20 02:49:53,495::task::495::TaskManager.Task::(_debug) Task 13e93aa2-4d9e-4e54-a24f-5b7e8d86f62d: moving from state init -> state preparing
>Thread-15912956::DEBUG::2013-11-20 02:49:53,496::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`281d013a-516f-4374-800e-178204a5e851`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1032' at 'public_getIsoList'
>Thread-15912956::DEBUG::2013-11-20 02:49:53,497::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912956::DEBUG::2013-11-20 02:49:53,498::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912956::DEBUG::2013-11-20 02:49:53,499::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`281d013a-516f-4374-800e-178204a5e851`::Granted request
>Thread-15912956::DEBUG::2013-11-20 02:49:53,500::task::495::TaskManager.Task::(_debug) Task 13e93aa2-4d9e-4e54-a24f-5b7e8d86f62d: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912956::DEBUG::2013-11-20 02:49:53,501::task::495::TaskManager.Task::(_debug) Task 13e93aa2-4d9e-4e54-a24f-5b7e8d86f62d: ref 1 aborting False
>Thread-15912956::DEBUG::2013-11-20 02:49:53,522::task::495::TaskManager.Task::(_debug) Task 13e93aa2-4d9e-4e54-a24f-5b7e8d86f62d: finished: {'isolist': ['Acronis Boot Image 31-10-2013.iso', 'Windows.2000.SP3.iso', 'ubuntu-13.04-desktop-amd64.iso', 'virtio-win-1.5.3.iso', 'rhev-tools-setup.iso', 'ubuntu-12.04.1-server-amd64.iso', 'ubuntu-13.04-server-amd64.iso', 'ubuntu-12.04-mini.iso', 'SW_DVD5_Win_Pro_8_64BIT_English_MLF_X18-16134.ISO', '.rhev-hypervisor6-6.4-20131016.0.iso', 'RHEV-toolsSetup_3.0_39.iso', 'SW_DVD5_Windows_Svr_2008w_SP2_English__x86_DC_EE_SE_X15-41116.ISO', 'SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO']}
>Thread-15912956::DEBUG::2013-11-20 02:49:53,523::task::495::TaskManager.Task::(_debug) Task 13e93aa2-4d9e-4e54-a24f-5b7e8d86f62d: moving from state preparing -> state finished
>Thread-15912956::DEBUG::2013-11-20 02:49:53,523::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912956::DEBUG::2013-11-20 02:49:53,524::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912956::DEBUG::2013-11-20 02:49:53,525::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912956::DEBUG::2013-11-20 02:49:53,525::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912956::DEBUG::2013-11-20 02:49:53,526::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912956::DEBUG::2013-11-20 02:49:53,526::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912956::DEBUG::2013-11-20 02:49:53,527::task::495::TaskManager.Task::(_debug) Task 13e93aa2-4d9e-4e54-a24f-5b7e8d86f62d: ref 0 aborting False
>Thread-15912956::INFO::2013-11-20 02:49:53,528::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getIsoList, Return response: {'status': {'message': 'OK', 'code': 0}, 'isolist': ['Acronis Boot Image 31-10-2013.iso', 'Windows.2000.SP3.iso', 'ubuntu-13.04-desktop-amd64.iso', 'virtio-win-1.5.3.iso', 'rhev-tools-setup.iso', 'ubuntu-12.04.1-server-amd64.iso', 'ubuntu-13.04-server-amd64.iso', 'ubuntu-12.04-mini.iso', 'SW_DVD5_Win_Pro_8_64BIT_English_MLF_X18-16134.ISO', '.rhev-hypervisor6-6.4-20131016.0.iso', 'RHEV-toolsSetup_3.0_39.iso', 'SW_DVD5_Windows_Svr_2008w_SP2_English__x86_DC_EE_SE_X15-41116.ISO', 'SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO']}
>Dummy-15912651::DEBUG::2013-11-20 02:49:53,814::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:49:54,396::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0743406 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:49:56,412::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:49:56,982::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0740924 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:49:58,997::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:49:59,516::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0755148 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:50:01,532::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:50:01,935::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.044599 s, 23.0 MB/s\n'; <rc> = 0
>Thread-15912960::DEBUG::2013-11-20 02:50:02,022::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912960::INFO::2013-11-20 02:50:02,024::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912960::DEBUG::2013-11-20 02:50:02,025::task::495::TaskManager.Task::(_debug) Task 5760bc15-80b7-423e-8bd2-4f3c5ac791e8: moving from state init -> state preparing
>Thread-15912960::DEBUG::2013-11-20 02:50:02,035::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912960::DEBUG::2013-11-20 02:50:02,036::task::495::TaskManager.Task::(_debug) Task 5760bc15-80b7-423e-8bd2-4f3c5ac791e8: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912960::DEBUG::2013-11-20 02:50:02,037::task::495::TaskManager.Task::(_debug) Task 5760bc15-80b7-423e-8bd2-4f3c5ac791e8: moving from state preparing -> state finished
>Thread-15912960::DEBUG::2013-11-20 02:50:02,037::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912960::DEBUG::2013-11-20 02:50:02,038::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912960::DEBUG::2013-11-20 02:50:02,038::task::495::TaskManager.Task::(_debug) Task 5760bc15-80b7-423e-8bd2-4f3c5ac791e8: ref 0 aborting False
>Thread-15912960::INFO::2013-11-20 02:50:02,039::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912961::DEBUG::2013-11-20 02:50:02,044::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912961::INFO::2013-11-20 02:50:02,045::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912961::DEBUG::2013-11-20 02:50:02,045::task::495::TaskManager.Task::(_debug) Task 22aefeca-db44-4a4a-aba4-b26ce7321f22: moving from state init -> state preparing
>Thread-15912961::DEBUG::2013-11-20 02:50:02,047::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`97196260-3678-4883-91a7-d7c45d37c351`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912961::DEBUG::2013-11-20 02:50:02,047::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912961::DEBUG::2013-11-20 02:50:02,048::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912961::DEBUG::2013-11-20 02:50:02,049::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`97196260-3678-4883-91a7-d7c45d37c351`::Granted request
>Thread-15912961::DEBUG::2013-11-20 02:50:02,050::task::495::TaskManager.Task::(_debug) Task 22aefeca-db44-4a4a-aba4-b26ce7321f22: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912961::DEBUG::2013-11-20 02:50:02,051::task::495::TaskManager.Task::(_debug) Task 22aefeca-db44-4a4a-aba4-b26ce7321f22: ref 1 aborting False
>Thread-15912961::WARNING::2013-11-20 02:50:02,053::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912961::DEBUG::2013-11-20 02:50:02,054::task::495::TaskManager.Task::(_debug) Task 22aefeca-db44-4a4a-aba4-b26ce7321f22: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115559366656', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115559366656', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912961::DEBUG::2013-11-20 02:50:02,055::task::495::TaskManager.Task::(_debug) Task 22aefeca-db44-4a4a-aba4-b26ce7321f22: moving from state preparing -> state finished
>Thread-15912961::DEBUG::2013-11-20 02:50:02,055::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912961::DEBUG::2013-11-20 02:50:02,056::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912961::DEBUG::2013-11-20 02:50:02,057::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912961::DEBUG::2013-11-20 02:50:02,057::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912961::DEBUG::2013-11-20 02:50:02,058::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912961::DEBUG::2013-11-20 02:50:02,058::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912961::DEBUG::2013-11-20 02:50:02,059::task::495::TaskManager.Task::(_debug) Task 22aefeca-db44-4a4a-aba4-b26ce7321f22: ref 0 aborting False
>Thread-15912961::INFO::2013-11-20 02:50:02,060::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115559366656', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115559366656', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:50:03,945::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:50:04,542::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.071034 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:50:06,557::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 02:50:06,790::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 02:50:06,791::task::495::TaskManager.Task::(_debug) Task 9afcd146-965f-4938-a69a-ddf5f4b89b30: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 02:50:06,793::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bd2c11d9-bc4b-4bab-b630-2798f4f59188`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 02:50:06,793::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 02:50:06,794::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 02:50:06,794::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bd2c11d9-bc4b-4bab-b630-2798f4f59188`::Granted request
>Thread-15912392::DEBUG::2013-11-20 02:50:06,795::task::495::TaskManager.Task::(_debug) Task 9afcd146-965f-4938-a69a-ddf5f4b89b30: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 02:50:06,795::task::495::TaskManager.Task::(_debug) Task 9afcd146-965f-4938-a69a-ddf5f4b89b30: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 02:50:06,796::task::495::TaskManager.Task::(_debug) Task 9afcd146-965f-4938-a69a-ddf5f4b89b30: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 02:50:06,796::task::495::TaskManager.Task::(_debug) Task 9afcd146-965f-4938-a69a-ddf5f4b89b30: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 02:50:06,796::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 02:50:06,797::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 02:50:06,797::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 02:50:06,798::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 02:50:06,798::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 02:50:06,799::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 02:50:06,799::task::495::TaskManager.Task::(_debug) Task 9afcd146-965f-4938-a69a-ddf5f4b89b30: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 02:50:06,800::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 02:50:07,117::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0736143 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15912963::INFO::2013-11-20 02:50:08,771::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912963::DEBUG::2013-11-20 02:50:08,771::task::495::TaskManager.Task::(_debug) Task 9f16b3d9-faf2-40c8-893c-2137f97a6f86: moving from state init -> state preparing
>Thread-15912963::DEBUG::2013-11-20 02:50:08,772::task::495::TaskManager.Task::(_debug) Task 9f16b3d9-faf2-40c8-893c-2137f97a6f86: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000396966934204', 'lastCheck': 1384915805.24564, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00146889686584', 'lastCheck': 1384915804.27742, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00161981582642', 'lastCheck': 1384915805.727365, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00048303604126', 'lastCheck': 1384915803.720016, 'valid': True, 'code': 0}}
>Thread-15912963::DEBUG::2013-11-20 02:50:08,772::task::495::TaskManager.Task::(_debug) Task 9f16b3d9-faf2-40c8-893c-2137f97a6f86: moving from state preparing -> state finished
>Thread-15912963::DEBUG::2013-11-20 02:50:08,773::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912963::DEBUG::2013-11-20 02:50:08,773::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912963::DEBUG::2013-11-20 02:50:08,774::task::495::TaskManager.Task::(_debug) Task 9f16b3d9-faf2-40c8-893c-2137f97a6f86: ref 0 aborting False
>Thread-15912963::INFO::2013-11-20 02:50:08,774::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000396966934204', 'lastCheck': 1384915805.24564, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00146889686584', 'lastCheck': 1384915804.27742, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00161981582642', 'lastCheck': 1384915805.727365, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00048303604126', 'lastCheck': 1384915803.720016, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:50:09,131::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:50:09,670::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0749571 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15912965::DEBUG::2013-11-20 02:50:09,824::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912965::INFO::2013-11-20 02:50:09,825::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getFloppyList, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912965::DEBUG::2013-11-20 02:50:09,826::task::495::TaskManager.Task::(_debug) Task 65653789-7fb0-40eb-ba55-c74ac6124e00: moving from state init -> state preparing
>Thread-15912965::DEBUG::2013-11-20 02:50:09,835::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`752ec471-53ea-4b02-ba0c-d3878948c2e1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1032' at 'public_getIsoList'
>Thread-15912965::DEBUG::2013-11-20 02:50:09,836::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912965::DEBUG::2013-11-20 02:50:09,837::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912965::DEBUG::2013-11-20 02:50:09,837::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`752ec471-53ea-4b02-ba0c-d3878948c2e1`::Granted request
>Thread-15912965::DEBUG::2013-11-20 02:50:09,838::task::495::TaskManager.Task::(_debug) Task 65653789-7fb0-40eb-ba55-c74ac6124e00: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912965::DEBUG::2013-11-20 02:50:09,839::task::495::TaskManager.Task::(_debug) Task 65653789-7fb0-40eb-ba55-c74ac6124e00: ref 1 aborting False
>Thread-15912965::DEBUG::2013-11-20 02:50:09,846::task::495::TaskManager.Task::(_debug) Task 65653789-7fb0-40eb-ba55-c74ac6124e00: finished: {'isolist': ['virtio-win-1.5.3.vfd']}
>Thread-15912965::DEBUG::2013-11-20 02:50:09,846::task::495::TaskManager.Task::(_debug) Task 65653789-7fb0-40eb-ba55-c74ac6124e00: moving from state preparing -> state finished
>Thread-15912965::DEBUG::2013-11-20 02:50:09,847::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912965::DEBUG::2013-11-20 02:50:09,847::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912965::DEBUG::2013-11-20 02:50:09,847::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912965::DEBUG::2013-11-20 02:50:09,848::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912965::DEBUG::2013-11-20 02:50:09,848::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912965::DEBUG::2013-11-20 02:50:09,848::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912965::DEBUG::2013-11-20 02:50:09,849::task::495::TaskManager.Task::(_debug) Task 65653789-7fb0-40eb-ba55-c74ac6124e00: ref 0 aborting False
>Thread-15912965::INFO::2013-11-20 02:50:09,849::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getFloppyList, Return response: {'status': {'message': 'OK', 'code': 0}, 'isolist': ['virtio-win-1.5.3.vfd']}
>Thread-15912966::DEBUG::2013-11-20 02:50:10,236::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912966::INFO::2013-11-20 02:50:10,238::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getIsoList, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912966::DEBUG::2013-11-20 02:50:10,238::task::495::TaskManager.Task::(_debug) Task b8a4e780-fa0f-4b47-9b98-335055bffd1a: moving from state init -> state preparing
>Thread-15912966::DEBUG::2013-11-20 02:50:10,240::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5e9b0c2c-8f30-483a-883d-0307bcf2e3e6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1032' at 'public_getIsoList'
>Thread-15912966::DEBUG::2013-11-20 02:50:10,240::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912966::DEBUG::2013-11-20 02:50:10,241::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912966::DEBUG::2013-11-20 02:50:10,243::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5e9b0c2c-8f30-483a-883d-0307bcf2e3e6`::Granted request
>Thread-15912966::DEBUG::2013-11-20 02:50:10,244::task::495::TaskManager.Task::(_debug) Task b8a4e780-fa0f-4b47-9b98-335055bffd1a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912966::DEBUG::2013-11-20 02:50:10,245::task::495::TaskManager.Task::(_debug) Task b8a4e780-fa0f-4b47-9b98-335055bffd1a: ref 1 aborting False
>Thread-15912966::DEBUG::2013-11-20 02:50:10,277::task::495::TaskManager.Task::(_debug) Task b8a4e780-fa0f-4b47-9b98-335055bffd1a: finished: {'isolist': ['Acronis Boot Image 31-10-2013.iso', 'Windows.2000.SP3.iso', 'ubuntu-13.04-desktop-amd64.iso', 'virtio-win-1.5.3.iso', 'rhev-tools-setup.iso', 'rhev-hypervisor6-6.4-20131016.0.iso', 'ubuntu-12.04.1-server-amd64.iso', 'ubuntu-13.04-server-amd64.iso', 'ubuntu-12.04-mini.iso', 'SW_DVD5_Win_Pro_8_64BIT_English_MLF_X18-16134.ISO', 'RHEV-toolsSetup_3.0_39.iso', 'SW_DVD5_Windows_Svr_2008w_SP2_English__x86_DC_EE_SE_X15-41116.ISO', 'SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO']}
>Thread-15912966::DEBUG::2013-11-20 02:50:10,279::task::495::TaskManager.Task::(_debug) Task b8a4e780-fa0f-4b47-9b98-335055bffd1a: moving from state preparing -> state finished
>Thread-15912966::DEBUG::2013-11-20 02:50:10,282::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912966::DEBUG::2013-11-20 02:50:10,283::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912966::DEBUG::2013-11-20 02:50:10,286::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912966::DEBUG::2013-11-20 02:50:10,287::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912966::DEBUG::2013-11-20 02:50:10,287::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912966::DEBUG::2013-11-20 02:50:10,287::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912966::DEBUG::2013-11-20 02:50:10,288::task::495::TaskManager.Task::(_debug) Task b8a4e780-fa0f-4b47-9b98-335055bffd1a: ref 0 aborting False
>Thread-15912966::INFO::2013-11-20 02:50:10,289::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getIsoList, Return response: {'status': {'message': 'OK', 'code': 0}, 'isolist': ['Acronis Boot Image 31-10-2013.iso', 'Windows.2000.SP3.iso', 'ubuntu-13.04-desktop-amd64.iso', 'virtio-win-1.5.3.iso', 'rhev-tools-setup.iso', 'rhev-hypervisor6-6.4-20131016.0.iso', 'ubuntu-12.04.1-server-amd64.iso', 'ubuntu-13.04-server-amd64.iso', 'ubuntu-12.04-mini.iso', 'SW_DVD5_Win_Pro_8_64BIT_English_MLF_X18-16134.ISO', 'RHEV-toolsSetup_3.0_39.iso', 'SW_DVD5_Windows_Svr_2008w_SP2_English__x86_DC_EE_SE_X15-41116.ISO', 'SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO']}
>Thread-15912967::DEBUG::2013-11-20 02:50:10,981::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912967::INFO::2013-11-20 02:50:10,982::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getFloppyList, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912967::DEBUG::2013-11-20 02:50:10,983::task::495::TaskManager.Task::(_debug) Task daea1a81-4d89-4c07-8a0b-716e9b5a68b0: moving from state init -> state preparing
>Thread-15912967::DEBUG::2013-11-20 02:50:10,984::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ccf2345c-a27d-4736-9bd9-78c9ec3124bc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1032' at 'public_getIsoList'
>Thread-15912967::DEBUG::2013-11-20 02:50:10,984::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912967::DEBUG::2013-11-20 02:50:10,985::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912967::DEBUG::2013-11-20 02:50:10,986::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ccf2345c-a27d-4736-9bd9-78c9ec3124bc`::Granted request
>Thread-15912967::DEBUG::2013-11-20 02:50:10,987::task::495::TaskManager.Task::(_debug) Task daea1a81-4d89-4c07-8a0b-716e9b5a68b0: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912967::DEBUG::2013-11-20 02:50:10,987::task::495::TaskManager.Task::(_debug) Task daea1a81-4d89-4c07-8a0b-716e9b5a68b0: ref 1 aborting False
>Thread-15912967::DEBUG::2013-11-20 02:50:10,991::task::495::TaskManager.Task::(_debug) Task daea1a81-4d89-4c07-8a0b-716e9b5a68b0: finished: {'isolist': ['virtio-win-1.5.3.vfd']}
>Thread-15912967::DEBUG::2013-11-20 02:50:10,992::task::495::TaskManager.Task::(_debug) Task daea1a81-4d89-4c07-8a0b-716e9b5a68b0: moving from state preparing -> state finished
>Thread-15912967::DEBUG::2013-11-20 02:50:10,992::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912967::DEBUG::2013-11-20 02:50:10,993::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912967::DEBUG::2013-11-20 02:50:10,993::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912967::DEBUG::2013-11-20 02:50:10,994::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912967::DEBUG::2013-11-20 02:50:10,994::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912967::DEBUG::2013-11-20 02:50:10,995::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912967::DEBUG::2013-11-20 02:50:10,995::task::495::TaskManager.Task::(_debug) Task daea1a81-4d89-4c07-8a0b-716e9b5a68b0: ref 0 aborting False
>Thread-15912967::INFO::2013-11-20 02:50:10,996::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getFloppyList, Return response: {'status': {'message': 'OK', 'code': 0}, 'isolist': ['virtio-win-1.5.3.vfd']}
>Dummy-15912651::DEBUG::2013-11-20 02:50:11,692::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:50:12,208::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0720437 s, 14.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:50:14,224::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:50:14,756::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0719609 s, 14.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:50:16,772::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:50:17,321::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0802047 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:50:19,336::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912971::DEBUG::2013-11-20 02:50:19,826::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912971::INFO::2013-11-20 02:50:19,829::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912971::DEBUG::2013-11-20 02:50:19,829::task::495::TaskManager.Task::(_debug) Task 9a8b9ac0-7ea9-445a-9b1f-267cbbeae45e: moving from state init -> state preparing
>Thread-15912971::DEBUG::2013-11-20 02:50:19,832::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912971::DEBUG::2013-11-20 02:50:19,832::task::495::TaskManager.Task::(_debug) Task 9a8b9ac0-7ea9-445a-9b1f-267cbbeae45e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912971::DEBUG::2013-11-20 02:50:19,834::task::495::TaskManager.Task::(_debug) Task 9a8b9ac0-7ea9-445a-9b1f-267cbbeae45e: moving from state preparing -> state finished
>Thread-15912971::DEBUG::2013-11-20 02:50:19,835::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912971::DEBUG::2013-11-20 02:50:19,836::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912971::DEBUG::2013-11-20 02:50:19,837::task::495::TaskManager.Task::(_debug) Task 9a8b9ac0-7ea9-445a-9b1f-267cbbeae45e: ref 0 aborting False
>Thread-15912971::INFO::2013-11-20 02:50:19,838::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912972::DEBUG::2013-11-20 02:50:19,844::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912972::INFO::2013-11-20 02:50:19,846::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912972::DEBUG::2013-11-20 02:50:19,847::task::495::TaskManager.Task::(_debug) Task 8804d2d0-4766-4a21-bde0-b8472c77d9e5: moving from state init -> state preparing
>Thread-15912972::DEBUG::2013-11-20 02:50:19,849::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a38af112-98a9-4dc3-aa43-f1981c6fa186`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912972::DEBUG::2013-11-20 02:50:19,850::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912972::DEBUG::2013-11-20 02:50:19,853::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912972::DEBUG::2013-11-20 02:50:19,854::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a38af112-98a9-4dc3-aa43-f1981c6fa186`::Granted request
>Thread-15912972::DEBUG::2013-11-20 02:50:19,857::task::495::TaskManager.Task::(_debug) Task 8804d2d0-4766-4a21-bde0-b8472c77d9e5: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912972::DEBUG::2013-11-20 02:50:19,859::task::495::TaskManager.Task::(_debug) Task 8804d2d0-4766-4a21-bde0-b8472c77d9e5: ref 1 aborting False
>Dummy-15912651::DEBUG::2013-11-20 02:50:19,859::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0530862 s, 19.3 MB/s\n'; <rc> = 0
>Thread-15912972::WARNING::2013-11-20 02:50:19,862::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912972::DEBUG::2013-11-20 02:50:19,874::task::495::TaskManager.Task::(_debug) Task 8804d2d0-4766-4a21-bde0-b8472c77d9e5: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115370622976', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115370622976', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912972::DEBUG::2013-11-20 02:50:19,875::task::495::TaskManager.Task::(_debug) Task 8804d2d0-4766-4a21-bde0-b8472c77d9e5: moving from state preparing -> state finished
>Thread-15912972::DEBUG::2013-11-20 02:50:19,875::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912972::DEBUG::2013-11-20 02:50:19,876::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912972::DEBUG::2013-11-20 02:50:19,877::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912972::DEBUG::2013-11-20 02:50:19,877::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912972::DEBUG::2013-11-20 02:50:19,878::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912972::DEBUG::2013-11-20 02:50:19,878::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912972::DEBUG::2013-11-20 02:50:19,879::task::495::TaskManager.Task::(_debug) Task 8804d2d0-4766-4a21-bde0-b8472c77d9e5: ref 0 aborting False
>Thread-15912972::INFO::2013-11-20 02:50:19,880::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115370622976', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115370622976', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:50:21,876::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:50:22,396::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0599446 s, 17.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:50:24,406::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:50:24,950::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.072663 s, 14.1 MB/s\n'; <rc> = 0
>Thread-15912974::INFO::2013-11-20 02:50:25,619::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912974::DEBUG::2013-11-20 02:50:25,619::task::495::TaskManager.Task::(_debug) Task 87321db5-55df-459d-b569-3c3669ec18db: moving from state init -> state preparing
>Thread-15912974::DEBUG::2013-11-20 02:50:25,620::task::495::TaskManager.Task::(_debug) Task 87321db5-55df-459d-b569-3c3669ec18db: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000339031219482', 'lastCheck': 1384915815.42944, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00152492523193', 'lastCheck': 1384915824.633805, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0139360427856', 'lastCheck': 1384915815.9381449, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000513076782227', 'lastCheck': 1384915824.1183629, 'valid': True, 'code': 0}}
>Thread-15912974::DEBUG::2013-11-20 02:50:25,620::task::495::TaskManager.Task::(_debug) Task 87321db5-55df-459d-b569-3c3669ec18db: moving from state preparing -> state finished
>Thread-15912974::DEBUG::2013-11-20 02:50:25,621::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912974::DEBUG::2013-11-20 02:50:25,621::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912974::DEBUG::2013-11-20 02:50:25,621::task::495::TaskManager.Task::(_debug) Task 87321db5-55df-459d-b569-3c3669ec18db: ref 0 aborting False
>Thread-15912974::INFO::2013-11-20 02:50:25,622::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000339031219482', 'lastCheck': 1384915815.42944, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00152492523193', 'lastCheck': 1384915824.633805, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0139360427856', 'lastCheck': 1384915815.9381449, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000513076782227', 'lastCheck': 1384915824.1183629, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:50:26,965::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:50:27,483::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0742179 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:50:29,497::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:50:29,968::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0461171 s, 22.2 MB/s\n'; <rc> = 0
>Thread-15912976::DEBUG::2013-11-20 02:50:31,677::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912976::INFO::2013-11-20 02:50:31,678::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912976::DEBUG::2013-11-20 02:50:31,679::task::495::TaskManager.Task::(_debug) Task bd6fd4b4-c2e6-41a5-9c6c-94e4be15d9e8: moving from state init -> state preparing
>Thread-15912976::DEBUG::2013-11-20 02:50:31,680::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912976::DEBUG::2013-11-20 02:50:31,680::task::495::TaskManager.Task::(_debug) Task bd6fd4b4-c2e6-41a5-9c6c-94e4be15d9e8: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912976::DEBUG::2013-11-20 02:50:31,681::task::495::TaskManager.Task::(_debug) Task bd6fd4b4-c2e6-41a5-9c6c-94e4be15d9e8: moving from state preparing -> state finished
>Thread-15912976::DEBUG::2013-11-20 02:50:31,682::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912976::DEBUG::2013-11-20 02:50:31,682::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912976::DEBUG::2013-11-20 02:50:31,683::task::495::TaskManager.Task::(_debug) Task bd6fd4b4-c2e6-41a5-9c6c-94e4be15d9e8: ref 0 aborting False
>Thread-15912976::INFO::2013-11-20 02:50:31,683::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912977::DEBUG::2013-11-20 02:50:31,688::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912977::INFO::2013-11-20 02:50:31,689::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912977::DEBUG::2013-11-20 02:50:31,689::task::495::TaskManager.Task::(_debug) Task 87772d03-2325-4216-b3d7-bfd08f57a520: moving from state init -> state preparing
>Thread-15912977::DEBUG::2013-11-20 02:50:31,691::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f5bcf253-0519-4a75-b481-1670409bddeb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912977::DEBUG::2013-11-20 02:50:31,691::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912977::DEBUG::2013-11-20 02:50:31,692::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912977::DEBUG::2013-11-20 02:50:31,693::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f5bcf253-0519-4a75-b481-1670409bddeb`::Granted request
>Thread-15912977::DEBUG::2013-11-20 02:50:31,694::task::495::TaskManager.Task::(_debug) Task 87772d03-2325-4216-b3d7-bfd08f57a520: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912977::DEBUG::2013-11-20 02:50:31,694::task::495::TaskManager.Task::(_debug) Task 87772d03-2325-4216-b3d7-bfd08f57a520: ref 1 aborting False
>Thread-15912977::WARNING::2013-11-20 02:50:31,696::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912977::DEBUG::2013-11-20 02:50:31,697::task::495::TaskManager.Task::(_debug) Task 87772d03-2325-4216-b3d7-bfd08f57a520: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115370622976', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115370622976', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912977::DEBUG::2013-11-20 02:50:31,698::task::495::TaskManager.Task::(_debug) Task 87772d03-2325-4216-b3d7-bfd08f57a520: moving from state preparing -> state finished
>Thread-15912977::DEBUG::2013-11-20 02:50:31,699::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912977::DEBUG::2013-11-20 02:50:31,699::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912977::DEBUG::2013-11-20 02:50:31,700::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912977::DEBUG::2013-11-20 02:50:31,700::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912977::DEBUG::2013-11-20 02:50:31,701::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912977::DEBUG::2013-11-20 02:50:31,701::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912977::DEBUG::2013-11-20 02:50:31,702::task::495::TaskManager.Task::(_debug) Task 87772d03-2325-4216-b3d7-bfd08f57a520: ref 0 aborting False
>Thread-15912977::INFO::2013-11-20 02:50:31,703::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115370622976', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115370622976', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:50:31,982::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:50:32,499::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0743677 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15912978::DEBUG::2013-11-20 02:50:34,403::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912978::INFO::2013-11-20 02:50:34,404::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getIsoList, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912978::DEBUG::2013-11-20 02:50:34,405::task::495::TaskManager.Task::(_debug) Task cf3e47fa-1799-4383-a1d9-00cf86437374: moving from state init -> state preparing
>Thread-15912978::DEBUG::2013-11-20 02:50:34,406::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`53623afa-fae5-45c8-a485-0ec742963590`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1032' at 'public_getIsoList'
>Thread-15912978::DEBUG::2013-11-20 02:50:34,407::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912978::DEBUG::2013-11-20 02:50:34,407::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912978::DEBUG::2013-11-20 02:50:34,408::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`53623afa-fae5-45c8-a485-0ec742963590`::Granted request
>Thread-15912978::DEBUG::2013-11-20 02:50:34,409::task::495::TaskManager.Task::(_debug) Task cf3e47fa-1799-4383-a1d9-00cf86437374: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912978::DEBUG::2013-11-20 02:50:34,410::task::495::TaskManager.Task::(_debug) Task cf3e47fa-1799-4383-a1d9-00cf86437374: ref 1 aborting False
>Thread-15912978::DEBUG::2013-11-20 02:50:34,425::task::495::TaskManager.Task::(_debug) Task cf3e47fa-1799-4383-a1d9-00cf86437374: finished: {'isolist': ['Acronis Boot Image 31-10-2013.iso', 'Windows.2000.SP3.iso', 'ubuntu-13.04-desktop-amd64.iso', 'virtio-win-1.5.3.iso', 'rhev-tools-setup.iso', 'rhev-hypervisor6-6.4-20131016.0.iso', 'ubuntu-12.04.1-server-amd64.iso', 'ubuntu-13.04-server-amd64.iso', 'ubuntu-12.04-mini.iso', 'SW_DVD5_Win_Pro_8_64BIT_English_MLF_X18-16134.ISO', 'RHEV-toolsSetup_3.0_39.iso', 'SW_DVD5_Windows_Svr_2008w_SP2_English__x86_DC_EE_SE_X15-41116.ISO', 'SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO']}
>Thread-15912978::DEBUG::2013-11-20 02:50:34,426::task::495::TaskManager.Task::(_debug) Task cf3e47fa-1799-4383-a1d9-00cf86437374: moving from state preparing -> state finished
>Thread-15912978::DEBUG::2013-11-20 02:50:34,427::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912978::DEBUG::2013-11-20 02:50:34,427::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912978::DEBUG::2013-11-20 02:50:34,428::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912978::DEBUG::2013-11-20 02:50:34,429::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912978::DEBUG::2013-11-20 02:50:34,429::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912978::DEBUG::2013-11-20 02:50:34,430::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912978::DEBUG::2013-11-20 02:50:34,431::task::495::TaskManager.Task::(_debug) Task cf3e47fa-1799-4383-a1d9-00cf86437374: ref 0 aborting False
>Thread-15912978::INFO::2013-11-20 02:50:34,431::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getIsoList, Return response: {'status': {'message': 'OK', 'code': 0}, 'isolist': ['Acronis Boot Image 31-10-2013.iso', 'Windows.2000.SP3.iso', 'ubuntu-13.04-desktop-amd64.iso', 'virtio-win-1.5.3.iso', 'rhev-tools-setup.iso', 'rhev-hypervisor6-6.4-20131016.0.iso', 'ubuntu-12.04.1-server-amd64.iso', 'ubuntu-13.04-server-amd64.iso', 'ubuntu-12.04-mini.iso', 'SW_DVD5_Win_Pro_8_64BIT_English_MLF_X18-16134.ISO', 'RHEV-toolsSetup_3.0_39.iso', 'SW_DVD5_Windows_Svr_2008w_SP2_English__x86_DC_EE_SE_X15-41116.ISO', 'SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO']}
>Dummy-15912651::DEBUG::2013-11-20 02:50:34,514::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:50:35,066::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0708826 s, 14.4 MB/s\n'; <rc> = 0
>Thread-15912980::DEBUG::2013-11-20 02:50:35,774::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912980::INFO::2013-11-20 02:50:35,775::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getFloppyList, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912980::DEBUG::2013-11-20 02:50:35,776::task::495::TaskManager.Task::(_debug) Task 7607c3ad-96c2-43ea-9099-1987174732d9: moving from state init -> state preparing
>Thread-15912980::DEBUG::2013-11-20 02:50:35,777::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4e473fb2-0927-442c-a084-3ca323568f03`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1032' at 'public_getIsoList'
>Thread-15912980::DEBUG::2013-11-20 02:50:35,777::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912980::DEBUG::2013-11-20 02:50:35,778::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912980::DEBUG::2013-11-20 02:50:35,779::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4e473fb2-0927-442c-a084-3ca323568f03`::Granted request
>Thread-15912980::DEBUG::2013-11-20 02:50:35,780::task::495::TaskManager.Task::(_debug) Task 7607c3ad-96c2-43ea-9099-1987174732d9: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912980::DEBUG::2013-11-20 02:50:35,780::task::495::TaskManager.Task::(_debug) Task 7607c3ad-96c2-43ea-9099-1987174732d9: ref 1 aborting False
>Thread-15912980::DEBUG::2013-11-20 02:50:35,785::task::495::TaskManager.Task::(_debug) Task 7607c3ad-96c2-43ea-9099-1987174732d9: finished: {'isolist': ['virtio-win-1.5.3.vfd']}
>Thread-15912980::DEBUG::2013-11-20 02:50:35,785::task::495::TaskManager.Task::(_debug) Task 7607c3ad-96c2-43ea-9099-1987174732d9: moving from state preparing -> state finished
>Thread-15912980::DEBUG::2013-11-20 02:50:35,786::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912980::DEBUG::2013-11-20 02:50:35,787::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912980::DEBUG::2013-11-20 02:50:35,787::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912980::DEBUG::2013-11-20 02:50:35,788::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912980::DEBUG::2013-11-20 02:50:35,788::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912980::DEBUG::2013-11-20 02:50:35,789::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912980::DEBUG::2013-11-20 02:50:35,790::task::495::TaskManager.Task::(_debug) Task 7607c3ad-96c2-43ea-9099-1987174732d9: ref 0 aborting False
>Thread-15912980::INFO::2013-11-20 02:50:35,790::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getFloppyList, Return response: {'status': {'message': 'OK', 'code': 0}, 'isolist': ['virtio-win-1.5.3.vfd']}
>Dummy-15912651::DEBUG::2013-11-20 02:50:37,082::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:50:37,605::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0699288 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:50:39,619::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:50:40,164::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0729455 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:50:42,177::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:50:42,727::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0751999 s, 13.6 MB/s\n'; <rc> = 0
>Thread-15912983::DEBUG::2013-11-20 02:50:43,051::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912983::INFO::2013-11-20 02:50:43,052::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912983::DEBUG::2013-11-20 02:50:43,053::task::495::TaskManager.Task::(_debug) Task a10376d1-62ef-4c7f-9dcb-363a43e7a936: moving from state init -> state preparing
>Thread-15912983::DEBUG::2013-11-20 02:50:43,054::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912983::DEBUG::2013-11-20 02:50:43,054::task::495::TaskManager.Task::(_debug) Task a10376d1-62ef-4c7f-9dcb-363a43e7a936: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912983::DEBUG::2013-11-20 02:50:43,055::task::495::TaskManager.Task::(_debug) Task a10376d1-62ef-4c7f-9dcb-363a43e7a936: moving from state preparing -> state finished
>Thread-15912983::DEBUG::2013-11-20 02:50:43,056::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912983::DEBUG::2013-11-20 02:50:43,056::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912983::DEBUG::2013-11-20 02:50:43,057::task::495::TaskManager.Task::(_debug) Task a10376d1-62ef-4c7f-9dcb-363a43e7a936: ref 0 aborting False
>Thread-15912983::INFO::2013-11-20 02:50:43,057::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912984::DEBUG::2013-11-20 02:50:43,064::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912984::INFO::2013-11-20 02:50:43,065::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912984::DEBUG::2013-11-20 02:50:43,066::task::495::TaskManager.Task::(_debug) Task 41784cff-1825-46a1-b9a0-4013c1cc5cd5: moving from state init -> state preparing
>Thread-15912984::DEBUG::2013-11-20 02:50:43,067::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5f87567b-4013-4f2e-bd3e-65b54cc8b920`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912984::DEBUG::2013-11-20 02:50:43,068::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912984::DEBUG::2013-11-20 02:50:43,069::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912984::DEBUG::2013-11-20 02:50:43,069::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5f87567b-4013-4f2e-bd3e-65b54cc8b920`::Granted request
>Thread-15912984::DEBUG::2013-11-20 02:50:43,070::task::495::TaskManager.Task::(_debug) Task 41784cff-1825-46a1-b9a0-4013c1cc5cd5: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912984::DEBUG::2013-11-20 02:50:43,071::task::495::TaskManager.Task::(_debug) Task 41784cff-1825-46a1-b9a0-4013c1cc5cd5: ref 1 aborting False
>Thread-15912984::WARNING::2013-11-20 02:50:43,073::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912984::DEBUG::2013-11-20 02:50:43,074::task::495::TaskManager.Task::(_debug) Task 41784cff-1825-46a1-b9a0-4013c1cc5cd5: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115370622976', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115370622976', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912984::DEBUG::2013-11-20 02:50:43,074::task::495::TaskManager.Task::(_debug) Task 41784cff-1825-46a1-b9a0-4013c1cc5cd5: moving from state preparing -> state finished
>Thread-15912984::DEBUG::2013-11-20 02:50:43,075::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912984::DEBUG::2013-11-20 02:50:43,076::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912984::DEBUG::2013-11-20 02:50:43,076::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912984::DEBUG::2013-11-20 02:50:43,077::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912984::DEBUG::2013-11-20 02:50:43,077::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912984::DEBUG::2013-11-20 02:50:43,078::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912984::DEBUG::2013-11-20 02:50:43,079::task::495::TaskManager.Task::(_debug) Task 41784cff-1825-46a1-b9a0-4013c1cc5cd5: ref 0 aborting False
>Thread-15912984::INFO::2013-11-20 02:50:43,079::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115370622976', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115370622976', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:50:44,742::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:50:45,272::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0746065 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:50:47,287::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:50:47,839::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0784803 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15912986::INFO::2013-11-20 02:50:49,204::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912986::DEBUG::2013-11-20 02:50:49,205::task::495::TaskManager.Task::(_debug) Task 70d49232-bb31-45be-9d21-74d96e903f90: moving from state init -> state preparing
>Thread-15912986::DEBUG::2013-11-20 02:50:49,206::task::495::TaskManager.Task::(_debug) Task 70d49232-bb31-45be-9d21-74d96e903f90: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000391960144043', 'lastCheck': 1384915845.971807, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014820098877', 'lastCheck': 1384915844.9842529, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00178289413452', 'lastCheck': 1384915846.414712, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000411033630371', 'lastCheck': 1384915844.407243, 'valid': True, 'code': 0}}
>Thread-15912986::DEBUG::2013-11-20 02:50:49,207::task::495::TaskManager.Task::(_debug) Task 70d49232-bb31-45be-9d21-74d96e903f90: moving from state preparing -> state finished
>Thread-15912986::DEBUG::2013-11-20 02:50:49,208::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912986::DEBUG::2013-11-20 02:50:49,208::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912986::DEBUG::2013-11-20 02:50:49,209::task::495::TaskManager.Task::(_debug) Task 70d49232-bb31-45be-9d21-74d96e903f90: ref 0 aborting False
>Thread-15912986::INFO::2013-11-20 02:50:49,210::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000391960144043', 'lastCheck': 1384915845.971807, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014820098877', 'lastCheck': 1384915844.9842529, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00178289413452', 'lastCheck': 1384915846.414712, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000411033630371', 'lastCheck': 1384915844.407243, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:50:49,855::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:50:50,373::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.082751 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:50:52,388::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:50:52,972::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0748192 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15912988::DEBUG::2013-11-20 02:50:53,831::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912988::INFO::2013-11-20 02:50:53,832::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912988::DEBUG::2013-11-20 02:50:53,833::task::495::TaskManager.Task::(_debug) Task ebcebeb3-823d-4d13-b0c4-bfe7dddc969e: moving from state init -> state preparing
>Thread-15912988::DEBUG::2013-11-20 02:50:53,834::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912988::DEBUG::2013-11-20 02:50:53,834::task::495::TaskManager.Task::(_debug) Task ebcebeb3-823d-4d13-b0c4-bfe7dddc969e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912988::DEBUG::2013-11-20 02:50:53,835::task::495::TaskManager.Task::(_debug) Task ebcebeb3-823d-4d13-b0c4-bfe7dddc969e: moving from state preparing -> state finished
>Thread-15912988::DEBUG::2013-11-20 02:50:53,836::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912988::DEBUG::2013-11-20 02:50:53,836::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912988::DEBUG::2013-11-20 02:50:53,837::task::495::TaskManager.Task::(_debug) Task ebcebeb3-823d-4d13-b0c4-bfe7dddc969e: ref 0 aborting False
>Thread-15912988::INFO::2013-11-20 02:50:53,837::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912989::DEBUG::2013-11-20 02:50:53,846::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912989::INFO::2013-11-20 02:50:53,847::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912989::DEBUG::2013-11-20 02:50:53,847::task::495::TaskManager.Task::(_debug) Task d5ebe618-1278-4104-a464-ac09e2f38f88: moving from state init -> state preparing
>Thread-15912989::DEBUG::2013-11-20 02:50:53,849::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6216bfd8-6efc-47af-bfe0-d81571a53e0b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912989::DEBUG::2013-11-20 02:50:53,850::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912989::DEBUG::2013-11-20 02:50:53,851::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912989::DEBUG::2013-11-20 02:50:53,852::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6216bfd8-6efc-47af-bfe0-d81571a53e0b`::Granted request
>Thread-15912989::DEBUG::2013-11-20 02:50:53,854::task::495::TaskManager.Task::(_debug) Task d5ebe618-1278-4104-a464-ac09e2f38f88: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912989::DEBUG::2013-11-20 02:50:53,855::task::495::TaskManager.Task::(_debug) Task d5ebe618-1278-4104-a464-ac09e2f38f88: ref 1 aborting False
>Thread-15912989::WARNING::2013-11-20 02:50:53,858::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912989::DEBUG::2013-11-20 02:50:53,860::task::495::TaskManager.Task::(_debug) Task d5ebe618-1278-4104-a464-ac09e2f38f88: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115370622976', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115370622976', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912989::DEBUG::2013-11-20 02:50:53,861::task::495::TaskManager.Task::(_debug) Task d5ebe618-1278-4104-a464-ac09e2f38f88: moving from state preparing -> state finished
>Thread-15912989::DEBUG::2013-11-20 02:50:53,862::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912989::DEBUG::2013-11-20 02:50:53,862::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912989::DEBUG::2013-11-20 02:50:53,864::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912989::DEBUG::2013-11-20 02:50:53,864::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912989::DEBUG::2013-11-20 02:50:53,865::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912989::DEBUG::2013-11-20 02:50:53,866::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912989::DEBUG::2013-11-20 02:50:53,867::task::495::TaskManager.Task::(_debug) Task d5ebe618-1278-4104-a464-ac09e2f38f88: ref 0 aborting False
>Thread-15912989::INFO::2013-11-20 02:50:53,868::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115370622976', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115370622976', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:50:54,986::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:50:55,529::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0734923 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:50:57,545::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:50:58,096::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.07443 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:51:00,110::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:51:00,643::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0733602 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:51:02,658::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:51:03,125::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0450394 s, 22.7 MB/s\n'; <rc> = 0
>Thread-15912993::DEBUG::2013-11-20 02:51:04,438::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912993::INFO::2013-11-20 02:51:04,440::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912993::DEBUG::2013-11-20 02:51:04,441::task::495::TaskManager.Task::(_debug) Task 6530d2f3-8579-445e-976d-c69e2a2010fc: moving from state init -> state preparing
>Thread-15912993::DEBUG::2013-11-20 02:51:04,442::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912993::DEBUG::2013-11-20 02:51:04,442::task::495::TaskManager.Task::(_debug) Task 6530d2f3-8579-445e-976d-c69e2a2010fc: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912993::DEBUG::2013-11-20 02:51:04,443::task::495::TaskManager.Task::(_debug) Task 6530d2f3-8579-445e-976d-c69e2a2010fc: moving from state preparing -> state finished
>Thread-15912993::DEBUG::2013-11-20 02:51:04,444::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912993::DEBUG::2013-11-20 02:51:04,444::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912993::DEBUG::2013-11-20 02:51:04,445::task::495::TaskManager.Task::(_debug) Task 6530d2f3-8579-445e-976d-c69e2a2010fc: ref 0 aborting False
>Thread-15912993::INFO::2013-11-20 02:51:04,445::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912994::DEBUG::2013-11-20 02:51:04,450::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912994::INFO::2013-11-20 02:51:04,451::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912994::DEBUG::2013-11-20 02:51:04,451::task::495::TaskManager.Task::(_debug) Task f4babad3-9f64-4f4a-ae02-0d9b1f18bf29: moving from state init -> state preparing
>Thread-15912994::DEBUG::2013-11-20 02:51:04,453::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4e2f7a5d-4971-46cd-b77d-38b893f7e004`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15912994::DEBUG::2013-11-20 02:51:04,453::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15912994::DEBUG::2013-11-20 02:51:04,454::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15912994::DEBUG::2013-11-20 02:51:04,455::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4e2f7a5d-4971-46cd-b77d-38b893f7e004`::Granted request
>Thread-15912994::DEBUG::2013-11-20 02:51:04,456::task::495::TaskManager.Task::(_debug) Task f4babad3-9f64-4f4a-ae02-0d9b1f18bf29: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15912994::DEBUG::2013-11-20 02:51:04,456::task::495::TaskManager.Task::(_debug) Task f4babad3-9f64-4f4a-ae02-0d9b1f18bf29: ref 1 aborting False
>Thread-15912994::WARNING::2013-11-20 02:51:04,458::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912994::DEBUG::2013-11-20 02:51:04,459::task::495::TaskManager.Task::(_debug) Task f4babad3-9f64-4f4a-ae02-0d9b1f18bf29: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115370622976', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115370622976', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912994::DEBUG::2013-11-20 02:51:04,460::task::495::TaskManager.Task::(_debug) Task f4babad3-9f64-4f4a-ae02-0d9b1f18bf29: moving from state preparing -> state finished
>Thread-15912994::DEBUG::2013-11-20 02:51:04,461::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912994::DEBUG::2013-11-20 02:51:04,461::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912994::DEBUG::2013-11-20 02:51:04,462::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912994::DEBUG::2013-11-20 02:51:04,463::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912994::DEBUG::2013-11-20 02:51:04,463::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912994::DEBUG::2013-11-20 02:51:04,464::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912994::DEBUG::2013-11-20 02:51:04,465::task::495::TaskManager.Task::(_debug) Task f4babad3-9f64-4f4a-ae02-0d9b1f18bf29: ref 0 aborting False
>Thread-15912994::INFO::2013-11-20 02:51:04,465::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115370622976', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115370622976', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:51:05,150::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:51:05,660::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0857552 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:51:07,675::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 02:51:07,900::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 02:51:07,905::task::495::TaskManager.Task::(_debug) Task f1819e31-dfba-490e-8db0-be8973913c41: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 02:51:07,911::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a706504c-289f-4187-8297-d10f4133b2c5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912996::INFO::2013-11-20 02:51:07,914::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912392::DEBUG::2013-11-20 02:51:07,915::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912996::DEBUG::2013-11-20 02:51:07,915::task::495::TaskManager.Task::(_debug) Task 00e30dac-2a20-4d0b-a73d-12b5666162ae: moving from state init -> state preparing
>Thread-15912996::DEBUG::2013-11-20 02:51:07,916::task::495::TaskManager.Task::(_debug) Task 00e30dac-2a20-4d0b-a73d-12b5666162ae: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000488996505737', 'lastCheck': 1384915866.358283, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0215401649475', 'lastCheck': 1384915865.3875771, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00170207023621', 'lastCheck': 1384915866.8855109, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000532865524292', 'lastCheck': 1384915864.881032, 'valid': True, 'code': 0}}
>Thread-15912392::DEBUG::2013-11-20 02:51:07,917::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912996::DEBUG::2013-11-20 02:51:07,918::task::495::TaskManager.Task::(_debug) Task 00e30dac-2a20-4d0b-a73d-12b5666162ae: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 02:51:07,918::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a706504c-289f-4187-8297-d10f4133b2c5`::Granted request
>Thread-15912996::DEBUG::2013-11-20 02:51:07,919::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912996::DEBUG::2013-11-20 02:51:07,920::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 02:51:07,921::task::495::TaskManager.Task::(_debug) Task f1819e31-dfba-490e-8db0-be8973913c41: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912996::DEBUG::2013-11-20 02:51:07,921::task::495::TaskManager.Task::(_debug) Task 00e30dac-2a20-4d0b-a73d-12b5666162ae: ref 0 aborting False
>Thread-15912392::DEBUG::2013-11-20 02:51:07,922::task::495::TaskManager.Task::(_debug) Task f1819e31-dfba-490e-8db0-be8973913c41: ref 1 aborting False
>Thread-15912996::INFO::2013-11-20 02:51:07,922::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000488996505737', 'lastCheck': 1384915866.358283, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0215401649475', 'lastCheck': 1384915865.3875771, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00170207023621', 'lastCheck': 1384915866.8855109, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000532865524292', 'lastCheck': 1384915864.881032, 'valid': True, 'code': 0}}
>Thread-15912392::DEBUG::2013-11-20 02:51:07,923::task::495::TaskManager.Task::(_debug) Task f1819e31-dfba-490e-8db0-be8973913c41: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 02:51:07,924::task::495::TaskManager.Task::(_debug) Task f1819e31-dfba-490e-8db0-be8973913c41: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 02:51:07,925::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 02:51:07,928::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 02:51:07,929::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 02:51:07,930::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 02:51:07,931::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 02:51:07,931::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 02:51:07,932::task::495::TaskManager.Task::(_debug) Task f1819e31-dfba-490e-8db0-be8973913c41: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 02:51:07,932::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 02:51:08,185::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0725852 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:51:10,200::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:51:10,684::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0754084 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:51:12,700::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:51:13,235::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0748208 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15912999::DEBUG::2013-11-20 02:51:14,642::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912999::INFO::2013-11-20 02:51:14,643::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912999::DEBUG::2013-11-20 02:51:14,644::task::495::TaskManager.Task::(_debug) Task 232309b9-e3ad-44e0-80c4-8ecc073091ba: moving from state init -> state preparing
>Thread-15912999::DEBUG::2013-11-20 02:51:14,645::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912999::DEBUG::2013-11-20 02:51:14,645::task::495::TaskManager.Task::(_debug) Task 232309b9-e3ad-44e0-80c4-8ecc073091ba: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912999::DEBUG::2013-11-20 02:51:14,646::task::495::TaskManager.Task::(_debug) Task 232309b9-e3ad-44e0-80c4-8ecc073091ba: moving from state preparing -> state finished
>Thread-15912999::DEBUG::2013-11-20 02:51:14,646::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912999::DEBUG::2013-11-20 02:51:14,647::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912999::DEBUG::2013-11-20 02:51:14,647::task::495::TaskManager.Task::(_debug) Task 232309b9-e3ad-44e0-80c4-8ecc073091ba: ref 0 aborting False
>Thread-15912999::INFO::2013-11-20 02:51:14,648::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913000::DEBUG::2013-11-20 02:51:14,652::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913000::INFO::2013-11-20 02:51:14,653::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913000::DEBUG::2013-11-20 02:51:14,653::task::495::TaskManager.Task::(_debug) Task 2c428e33-8e88-4d56-afe1-2aba56953da7: moving from state init -> state preparing
>Thread-15913000::DEBUG::2013-11-20 02:51:14,654::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`447de0e3-628b-4e82-a788-a1445a5e76d9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913000::DEBUG::2013-11-20 02:51:14,655::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913000::DEBUG::2013-11-20 02:51:14,655::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913000::DEBUG::2013-11-20 02:51:14,656::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`447de0e3-628b-4e82-a788-a1445a5e76d9`::Granted request
>Thread-15913000::DEBUG::2013-11-20 02:51:14,657::task::495::TaskManager.Task::(_debug) Task 2c428e33-8e88-4d56-afe1-2aba56953da7: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913000::DEBUG::2013-11-20 02:51:14,658::task::495::TaskManager.Task::(_debug) Task 2c428e33-8e88-4d56-afe1-2aba56953da7: ref 1 aborting False
>Thread-15913000::WARNING::2013-11-20 02:51:14,659::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913000::DEBUG::2013-11-20 02:51:14,661::task::495::TaskManager.Task::(_debug) Task 2c428e33-8e88-4d56-afe1-2aba56953da7: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115387400192', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115387400192', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913000::DEBUG::2013-11-20 02:51:14,662::task::495::TaskManager.Task::(_debug) Task 2c428e33-8e88-4d56-afe1-2aba56953da7: moving from state preparing -> state finished
>Thread-15913000::DEBUG::2013-11-20 02:51:14,663::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913000::DEBUG::2013-11-20 02:51:14,664::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913000::DEBUG::2013-11-20 02:51:14,665::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913000::DEBUG::2013-11-20 02:51:14,666::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913000::DEBUG::2013-11-20 02:51:14,666::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913000::DEBUG::2013-11-20 02:51:14,667::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913000::DEBUG::2013-11-20 02:51:14,667::task::495::TaskManager.Task::(_debug) Task 2c428e33-8e88-4d56-afe1-2aba56953da7: ref 0 aborting False
>Thread-15913000::INFO::2013-11-20 02:51:14,668::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115387400192', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115387400192', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:51:15,249::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:51:15,809::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.073965 s, 13.8 MB/s\n'; <rc> = 0
>Thread-42838::DEBUG::2013-11-20 02:51:16,491::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 02:51:17,141::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:51:17,825::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:51:18,256::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.044855 s, 22.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:51:20,266::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:51:20,765::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0781857 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:51:22,781::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:51:23,248::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0450195 s, 22.7 MB/s\n'; <rc> = 0
>Thread-15913004::INFO::2013-11-20 02:51:23,708::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913004::DEBUG::2013-11-20 02:51:23,708::task::495::TaskManager.Task::(_debug) Task d71a8633-92e6-4262-ad93-d1b86bc6a312: moving from state init -> state preparing
>Thread-15913004::DEBUG::2013-11-20 02:51:23,709::task::495::TaskManager.Task::(_debug) Task d71a8633-92e6-4262-ad93-d1b86bc6a312: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000319004058838', 'lastCheck': 1384915877.144207, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00219392776489', 'lastCheck': 1384915875.499285, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00141215324402', 'lastCheck': 1384915876.9372849, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000462055206299', 'lastCheck': 1384915874.9258659, 'valid': True, 'code': 0}}
>Thread-15913004::DEBUG::2013-11-20 02:51:23,709::task::495::TaskManager.Task::(_debug) Task d71a8633-92e6-4262-ad93-d1b86bc6a312: moving from state preparing -> state finished
>Thread-15913004::DEBUG::2013-11-20 02:51:23,710::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913004::DEBUG::2013-11-20 02:51:23,710::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913004::DEBUG::2013-11-20 02:51:23,710::task::495::TaskManager.Task::(_debug) Task d71a8633-92e6-4262-ad93-d1b86bc6a312: ref 0 aborting False
>Thread-15913004::INFO::2013-11-20 02:51:23,711::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000319004058838', 'lastCheck': 1384915877.144207, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00219392776489', 'lastCheck': 1384915875.499285, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00141215324402', 'lastCheck': 1384915876.9372849, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000462055206299', 'lastCheck': 1384915874.9258659, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:51:25,258::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913006::DEBUG::2013-11-20 02:51:25,402::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913006::INFO::2013-11-20 02:51:25,404::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913006::DEBUG::2013-11-20 02:51:25,414::task::495::TaskManager.Task::(_debug) Task 508b6f43-1c7c-492e-9dee-6cbac9c10c7e: moving from state init -> state preparing
>Thread-15913006::DEBUG::2013-11-20 02:51:25,415::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913006::DEBUG::2013-11-20 02:51:25,415::task::495::TaskManager.Task::(_debug) Task 508b6f43-1c7c-492e-9dee-6cbac9c10c7e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913006::DEBUG::2013-11-20 02:51:25,416::task::495::TaskManager.Task::(_debug) Task 508b6f43-1c7c-492e-9dee-6cbac9c10c7e: moving from state preparing -> state finished
>Thread-15913006::DEBUG::2013-11-20 02:51:25,416::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913006::DEBUG::2013-11-20 02:51:25,416::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913006::DEBUG::2013-11-20 02:51:25,417::task::495::TaskManager.Task::(_debug) Task 508b6f43-1c7c-492e-9dee-6cbac9c10c7e: ref 0 aborting False
>Thread-15913006::INFO::2013-11-20 02:51:25,417::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913007::DEBUG::2013-11-20 02:51:25,421::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913007::INFO::2013-11-20 02:51:25,422::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913007::DEBUG::2013-11-20 02:51:25,423::task::495::TaskManager.Task::(_debug) Task 74e269ed-14c4-458b-8617-6c4f199d97fc: moving from state init -> state preparing
>Thread-15913007::DEBUG::2013-11-20 02:51:25,424::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e32f8dca-806c-4822-a68b-80c000bf1b36`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913007::DEBUG::2013-11-20 02:51:25,425::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913007::DEBUG::2013-11-20 02:51:25,426::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913007::DEBUG::2013-11-20 02:51:25,426::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e32f8dca-806c-4822-a68b-80c000bf1b36`::Granted request
>Thread-15913007::DEBUG::2013-11-20 02:51:25,428::task::495::TaskManager.Task::(_debug) Task 74e269ed-14c4-458b-8617-6c4f199d97fc: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913007::DEBUG::2013-11-20 02:51:25,428::task::495::TaskManager.Task::(_debug) Task 74e269ed-14c4-458b-8617-6c4f199d97fc: ref 1 aborting False
>Thread-15913007::WARNING::2013-11-20 02:51:25,430::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913007::DEBUG::2013-11-20 02:51:25,431::task::495::TaskManager.Task::(_debug) Task 74e269ed-14c4-458b-8617-6c4f199d97fc: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913007::DEBUG::2013-11-20 02:51:25,432::task::495::TaskManager.Task::(_debug) Task 74e269ed-14c4-458b-8617-6c4f199d97fc: moving from state preparing -> state finished
>Thread-15913007::DEBUG::2013-11-20 02:51:25,432::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913007::DEBUG::2013-11-20 02:51:25,433::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913007::DEBUG::2013-11-20 02:51:25,434::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913007::DEBUG::2013-11-20 02:51:25,435::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913007::DEBUG::2013-11-20 02:51:25,435::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913007::DEBUG::2013-11-20 02:51:25,435::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913007::DEBUG::2013-11-20 02:51:25,436::task::495::TaskManager.Task::(_debug) Task 74e269ed-14c4-458b-8617-6c4f199d97fc: ref 0 aborting False
>Thread-15913007::INFO::2013-11-20 02:51:25,437::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:51:25,591::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0374406 s, 27.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:51:27,601::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:51:28,118::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0752216 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:51:30,133::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:51:30,673::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0702479 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:51:32,688::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:51:33,227::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0779434 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:51:35,238::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:51:35,696::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.045861 s, 22.3 MB/s\n'; <rc> = 0
>Thread-15913011::DEBUG::2013-11-20 02:51:36,172::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913011::INFO::2013-11-20 02:51:36,173::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913011::DEBUG::2013-11-20 02:51:36,174::task::495::TaskManager.Task::(_debug) Task 039bd3db-2603-4503-b22b-22a40fbb0770: moving from state init -> state preparing
>Thread-15913011::DEBUG::2013-11-20 02:51:36,175::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913011::DEBUG::2013-11-20 02:51:36,175::task::495::TaskManager.Task::(_debug) Task 039bd3db-2603-4503-b22b-22a40fbb0770: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913011::DEBUG::2013-11-20 02:51:36,176::task::495::TaskManager.Task::(_debug) Task 039bd3db-2603-4503-b22b-22a40fbb0770: moving from state preparing -> state finished
>Thread-15913011::DEBUG::2013-11-20 02:51:36,176::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913011::DEBUG::2013-11-20 02:51:36,177::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913011::DEBUG::2013-11-20 02:51:36,177::task::495::TaskManager.Task::(_debug) Task 039bd3db-2603-4503-b22b-22a40fbb0770: ref 0 aborting False
>Thread-15913011::INFO::2013-11-20 02:51:36,178::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913012::DEBUG::2013-11-20 02:51:36,182::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913012::INFO::2013-11-20 02:51:36,183::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913012::DEBUG::2013-11-20 02:51:36,184::task::495::TaskManager.Task::(_debug) Task 113f9d53-a7d4-4c13-8128-9b3933245d25: moving from state init -> state preparing
>Thread-15913012::DEBUG::2013-11-20 02:51:36,185::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c3ccb082-cea4-4735-87d3-286fba701a53`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913012::DEBUG::2013-11-20 02:51:36,186::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913012::DEBUG::2013-11-20 02:51:36,186::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913012::DEBUG::2013-11-20 02:51:36,187::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c3ccb082-cea4-4735-87d3-286fba701a53`::Granted request
>Thread-15913012::DEBUG::2013-11-20 02:51:36,188::task::495::TaskManager.Task::(_debug) Task 113f9d53-a7d4-4c13-8128-9b3933245d25: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913012::DEBUG::2013-11-20 02:51:36,188::task::495::TaskManager.Task::(_debug) Task 113f9d53-a7d4-4c13-8128-9b3933245d25: ref 1 aborting False
>Thread-15913012::WARNING::2013-11-20 02:51:36,190::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913012::DEBUG::2013-11-20 02:51:36,191::task::495::TaskManager.Task::(_debug) Task 113f9d53-a7d4-4c13-8128-9b3933245d25: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115405225984', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913012::DEBUG::2013-11-20 02:51:36,192::task::495::TaskManager.Task::(_debug) Task 113f9d53-a7d4-4c13-8128-9b3933245d25: moving from state preparing -> state finished
>Thread-15913012::DEBUG::2013-11-20 02:51:36,193::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913012::DEBUG::2013-11-20 02:51:36,193::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913012::DEBUG::2013-11-20 02:51:36,194::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913012::DEBUG::2013-11-20 02:51:36,194::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913012::DEBUG::2013-11-20 02:51:36,195::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913012::DEBUG::2013-11-20 02:51:36,195::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913012::DEBUG::2013-11-20 02:51:36,196::task::495::TaskManager.Task::(_debug) Task 113f9d53-a7d4-4c13-8128-9b3933245d25: ref 0 aborting False
>Thread-15913012::INFO::2013-11-20 02:51:36,197::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115405225984', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:51:37,707::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:51:38,160::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0456979 s, 22.4 MB/s\n'; <rc> = 0
>Thread-15913014::INFO::2013-11-20 02:51:39,420::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913014::DEBUG::2013-11-20 02:51:39,421::task::495::TaskManager.Task::(_debug) Task 2a0a2b40-3b8f-474c-a522-74a212838566: moving from state init -> state preparing
>Thread-15913014::DEBUG::2013-11-20 02:51:39,421::task::495::TaskManager.Task::(_debug) Task 2a0a2b40-3b8f-474c-a522-74a212838566: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000358104705811', 'lastCheck': 1384915897.432658, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00147008895874', 'lastCheck': 1384915895.9054141, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00116896629333', 'lastCheck': 1384915897.0758171, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000489950180054', 'lastCheck': 1384915895.0517759, 'valid': True, 'code': 0}}
>Thread-15913014::DEBUG::2013-11-20 02:51:39,422::task::495::TaskManager.Task::(_debug) Task 2a0a2b40-3b8f-474c-a522-74a212838566: moving from state preparing -> state finished
>Thread-15913014::DEBUG::2013-11-20 02:51:39,422::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913014::DEBUG::2013-11-20 02:51:39,422::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913014::DEBUG::2013-11-20 02:51:39,423::task::495::TaskManager.Task::(_debug) Task 2a0a2b40-3b8f-474c-a522-74a212838566: ref 0 aborting False
>Thread-15913014::INFO::2013-11-20 02:51:39,423::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000358104705811', 'lastCheck': 1384915897.432658, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00147008895874', 'lastCheck': 1384915895.9054141, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00116896629333', 'lastCheck': 1384915897.0758171, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000489950180054', 'lastCheck': 1384915895.0517759, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:51:40,170::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:51:40,696::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0591702 s, 17.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:51:42,715::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:51:43,239::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0598479 s, 17.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:51:45,253::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:51:45,770::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0638774 s, 16.0 MB/s\n'; <rc> = 0
>Thread-15913018::DEBUG::2013-11-20 02:51:46,743::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913018::INFO::2013-11-20 02:51:46,744::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913018::DEBUG::2013-11-20 02:51:46,745::task::495::TaskManager.Task::(_debug) Task cad36d53-9112-43af-b285-ae4a9f885737: moving from state init -> state preparing
>Thread-15913018::DEBUG::2013-11-20 02:51:46,746::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913018::DEBUG::2013-11-20 02:51:46,746::task::495::TaskManager.Task::(_debug) Task cad36d53-9112-43af-b285-ae4a9f885737: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913018::DEBUG::2013-11-20 02:51:46,747::task::495::TaskManager.Task::(_debug) Task cad36d53-9112-43af-b285-ae4a9f885737: moving from state preparing -> state finished
>Thread-15913018::DEBUG::2013-11-20 02:51:46,748::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913018::DEBUG::2013-11-20 02:51:46,748::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913018::DEBUG::2013-11-20 02:51:46,749::task::495::TaskManager.Task::(_debug) Task cad36d53-9112-43af-b285-ae4a9f885737: ref 0 aborting False
>Thread-15913018::INFO::2013-11-20 02:51:46,749::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913019::DEBUG::2013-11-20 02:51:46,754::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913019::INFO::2013-11-20 02:51:46,755::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913019::DEBUG::2013-11-20 02:51:46,756::task::495::TaskManager.Task::(_debug) Task 39c464d8-c62a-4810-8ad2-1fb5e592a7de: moving from state init -> state preparing
>Thread-15913019::DEBUG::2013-11-20 02:51:46,758::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`754ed8b4-afac-4c9e-9952-3cde9c9ddc63`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913019::DEBUG::2013-11-20 02:51:46,758::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913019::DEBUG::2013-11-20 02:51:46,759::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913019::DEBUG::2013-11-20 02:51:46,760::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`754ed8b4-afac-4c9e-9952-3cde9c9ddc63`::Granted request
>Thread-15913019::DEBUG::2013-11-20 02:51:46,761::task::495::TaskManager.Task::(_debug) Task 39c464d8-c62a-4810-8ad2-1fb5e592a7de: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913019::DEBUG::2013-11-20 02:51:46,761::task::495::TaskManager.Task::(_debug) Task 39c464d8-c62a-4810-8ad2-1fb5e592a7de: ref 1 aborting False
>Thread-15913019::WARNING::2013-11-20 02:51:46,764::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913019::DEBUG::2013-11-20 02:51:46,765::task::495::TaskManager.Task::(_debug) Task 39c464d8-c62a-4810-8ad2-1fb5e592a7de: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115405225984', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115405225984', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913019::DEBUG::2013-11-20 02:51:46,767::task::495::TaskManager.Task::(_debug) Task 39c464d8-c62a-4810-8ad2-1fb5e592a7de: moving from state preparing -> state finished
>Thread-15913019::DEBUG::2013-11-20 02:51:46,768::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913019::DEBUG::2013-11-20 02:51:46,769::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913019::DEBUG::2013-11-20 02:51:46,769::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913019::DEBUG::2013-11-20 02:51:46,770::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913019::DEBUG::2013-11-20 02:51:46,771::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913019::DEBUG::2013-11-20 02:51:46,771::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913019::DEBUG::2013-11-20 02:51:46,772::task::495::TaskManager.Task::(_debug) Task 39c464d8-c62a-4810-8ad2-1fb5e592a7de: ref 0 aborting False
>Thread-15913019::INFO::2013-11-20 02:51:46,773::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115405225984', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115405225984', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:51:47,780::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:51:48,266::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0699758 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:51:50,281::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:51:50,739::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.051175 s, 20.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:51:52,749::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:51:53,326::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0810723 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15913022::INFO::2013-11-20 02:51:55,188::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913022::DEBUG::2013-11-20 02:51:55,189::task::495::TaskManager.Task::(_debug) Task 41534ae6-6ccf-44aa-8ff2-40feba14f2a9: moving from state init -> state preparing
>Thread-15913022::DEBUG::2013-11-20 02:51:55,190::task::495::TaskManager.Task::(_debug) Task 41534ae6-6ccf-44aa-8ff2-40feba14f2a9: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000339031219482', 'lastCheck': 1384915907.4881401, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00815320014954', 'lastCheck': 1384915905.965147, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00165319442749', 'lastCheck': 1384915907.0908949, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000507831573486', 'lastCheck': 1384915915.0746379, 'valid': True, 'code': 0}}
>Thread-15913022::DEBUG::2013-11-20 02:51:55,191::task::495::TaskManager.Task::(_debug) Task 41534ae6-6ccf-44aa-8ff2-40feba14f2a9: moving from state preparing -> state finished
>Thread-15913022::DEBUG::2013-11-20 02:51:55,191::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913022::DEBUG::2013-11-20 02:51:55,192::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913022::DEBUG::2013-11-20 02:51:55,192::task::495::TaskManager.Task::(_debug) Task 41534ae6-6ccf-44aa-8ff2-40feba14f2a9: ref 0 aborting False
>Thread-15913022::INFO::2013-11-20 02:51:55,193::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000339031219482', 'lastCheck': 1384915907.4881401, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00815320014954', 'lastCheck': 1384915905.965147, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00165319442749', 'lastCheck': 1384915907.0908949, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000507831573486', 'lastCheck': 1384915915.0746379, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:51:55,342::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:51:55,892::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0729512 s, 14.0 MB/s\n'; <rc> = 0
>Thread-15913024::DEBUG::2013-11-20 02:51:56,938::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913024::INFO::2013-11-20 02:51:56,940::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913024::DEBUG::2013-11-20 02:51:56,940::task::495::TaskManager.Task::(_debug) Task 09c29049-7eeb-421d-b671-52faddbe1a0e: moving from state init -> state preparing
>Thread-15913024::DEBUG::2013-11-20 02:51:56,941::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913024::DEBUG::2013-11-20 02:51:56,942::task::495::TaskManager.Task::(_debug) Task 09c29049-7eeb-421d-b671-52faddbe1a0e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913024::DEBUG::2013-11-20 02:51:56,942::task::495::TaskManager.Task::(_debug) Task 09c29049-7eeb-421d-b671-52faddbe1a0e: moving from state preparing -> state finished
>Thread-15913024::DEBUG::2013-11-20 02:51:56,943::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913024::DEBUG::2013-11-20 02:51:56,943::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913024::DEBUG::2013-11-20 02:51:56,944::task::495::TaskManager.Task::(_debug) Task 09c29049-7eeb-421d-b671-52faddbe1a0e: ref 0 aborting False
>Thread-15913024::INFO::2013-11-20 02:51:56,944::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913025::DEBUG::2013-11-20 02:51:56,949::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913025::INFO::2013-11-20 02:51:56,950::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913025::DEBUG::2013-11-20 02:51:56,951::task::495::TaskManager.Task::(_debug) Task 68a3b8cf-7045-4349-87f2-220edc0081af: moving from state init -> state preparing
>Thread-15913025::DEBUG::2013-11-20 02:51:56,952::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e3b2e296-120d-4d68-9dd3-711f18ebff10`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913025::DEBUG::2013-11-20 02:51:56,953::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913025::DEBUG::2013-11-20 02:51:56,953::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913025::DEBUG::2013-11-20 02:51:56,954::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e3b2e296-120d-4d68-9dd3-711f18ebff10`::Granted request
>Thread-15913025::DEBUG::2013-11-20 02:51:56,960::task::495::TaskManager.Task::(_debug) Task 68a3b8cf-7045-4349-87f2-220edc0081af: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913025::DEBUG::2013-11-20 02:51:56,961::task::495::TaskManager.Task::(_debug) Task 68a3b8cf-7045-4349-87f2-220edc0081af: ref 1 aborting False
>Thread-15913025::WARNING::2013-11-20 02:51:56,963::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913025::DEBUG::2013-11-20 02:51:56,964::task::495::TaskManager.Task::(_debug) Task 68a3b8cf-7045-4349-87f2-220edc0081af: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115405225984', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115405225984', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913025::DEBUG::2013-11-20 02:51:56,965::task::495::TaskManager.Task::(_debug) Task 68a3b8cf-7045-4349-87f2-220edc0081af: moving from state preparing -> state finished
>Thread-15913025::DEBUG::2013-11-20 02:51:56,965::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913025::DEBUG::2013-11-20 02:51:56,966::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913025::DEBUG::2013-11-20 02:51:56,967::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913025::DEBUG::2013-11-20 02:51:56,967::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913025::DEBUG::2013-11-20 02:51:56,968::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913025::DEBUG::2013-11-20 02:51:56,968::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913025::DEBUG::2013-11-20 02:51:56,969::task::495::TaskManager.Task::(_debug) Task 68a3b8cf-7045-4349-87f2-220edc0081af: ref 0 aborting False
>Thread-15913025::INFO::2013-11-20 02:51:56,970::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115405225984', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115405225984', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:51:57,907::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:51:58,427::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.071833 s, 14.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:52:00,442::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:52:00,970::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0734917 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:52:02,984::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:52:03,493::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.072761 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:52:05,509::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:52:06,070::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0751324 s, 13.6 MB/s\n'; <rc> = 0
>Thread-15913029::DEBUG::2013-11-20 02:52:07,678::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913029::INFO::2013-11-20 02:52:07,679::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913029::DEBUG::2013-11-20 02:52:07,679::task::495::TaskManager.Task::(_debug) Task 2c3ceecd-0a99-4dcd-9489-cdb27fe8578a: moving from state init -> state preparing
>Thread-15913029::DEBUG::2013-11-20 02:52:07,680::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913029::DEBUG::2013-11-20 02:52:07,680::task::495::TaskManager.Task::(_debug) Task 2c3ceecd-0a99-4dcd-9489-cdb27fe8578a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913029::DEBUG::2013-11-20 02:52:07,681::task::495::TaskManager.Task::(_debug) Task 2c3ceecd-0a99-4dcd-9489-cdb27fe8578a: moving from state preparing -> state finished
>Thread-15913029::DEBUG::2013-11-20 02:52:07,681::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913029::DEBUG::2013-11-20 02:52:07,681::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913029::DEBUG::2013-11-20 02:52:07,682::task::495::TaskManager.Task::(_debug) Task 2c3ceecd-0a99-4dcd-9489-cdb27fe8578a: ref 0 aborting False
>Thread-15913029::INFO::2013-11-20 02:52:07,682::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913030::DEBUG::2013-11-20 02:52:07,685::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913030::INFO::2013-11-20 02:52:07,686::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913030::DEBUG::2013-11-20 02:52:07,686::task::495::TaskManager.Task::(_debug) Task f6dcdee7-216c-435f-be8a-49de29bcd494: moving from state init -> state preparing
>Thread-15913030::DEBUG::2013-11-20 02:52:07,687::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`da59c914-1930-4e77-8424-4535732880cc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913030::DEBUG::2013-11-20 02:52:07,687::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913030::DEBUG::2013-11-20 02:52:07,688::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913030::DEBUG::2013-11-20 02:52:07,688::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`da59c914-1930-4e77-8424-4535732880cc`::Granted request
>Thread-15913030::DEBUG::2013-11-20 02:52:07,689::task::495::TaskManager.Task::(_debug) Task f6dcdee7-216c-435f-be8a-49de29bcd494: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913030::DEBUG::2013-11-20 02:52:07,689::task::495::TaskManager.Task::(_debug) Task f6dcdee7-216c-435f-be8a-49de29bcd494: ref 1 aborting False
>Thread-15913030::WARNING::2013-11-20 02:52:07,690::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913030::DEBUG::2013-11-20 02:52:07,691::task::495::TaskManager.Task::(_debug) Task f6dcdee7-216c-435f-be8a-49de29bcd494: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115405225984', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115405225984', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913030::DEBUG::2013-11-20 02:52:07,691::task::495::TaskManager.Task::(_debug) Task f6dcdee7-216c-435f-be8a-49de29bcd494: moving from state preparing -> state finished
>Thread-15913030::DEBUG::2013-11-20 02:52:07,692::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913030::DEBUG::2013-11-20 02:52:07,692::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913030::DEBUG::2013-11-20 02:52:07,692::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913030::DEBUG::2013-11-20 02:52:07,693::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913030::DEBUG::2013-11-20 02:52:07,693::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913030::DEBUG::2013-11-20 02:52:07,693::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913030::DEBUG::2013-11-20 02:52:07,694::task::495::TaskManager.Task::(_debug) Task f6dcdee7-216c-435f-be8a-49de29bcd494: ref 0 aborting False
>Thread-15913030::INFO::2013-11-20 02:52:07,694::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115405225984', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115405225984', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:52:08,086::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 02:52:08,305::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 02:52:08,307::task::495::TaskManager.Task::(_debug) Task e27e764f-4238-46ff-8226-ef3a5ae59a87: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 02:52:08,309::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b5dc8790-ea23-4363-8968-35edc81db0d0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 02:52:08,310::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 02:52:08,311::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 02:52:08,311::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b5dc8790-ea23-4363-8968-35edc81db0d0`::Granted request
>Thread-15912392::DEBUG::2013-11-20 02:52:08,313::task::495::TaskManager.Task::(_debug) Task e27e764f-4238-46ff-8226-ef3a5ae59a87: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 02:52:08,313::task::495::TaskManager.Task::(_debug) Task e27e764f-4238-46ff-8226-ef3a5ae59a87: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 02:52:08,314::task::495::TaskManager.Task::(_debug) Task e27e764f-4238-46ff-8226-ef3a5ae59a87: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 02:52:08,315::task::495::TaskManager.Task::(_debug) Task e27e764f-4238-46ff-8226-ef3a5ae59a87: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 02:52:08,315::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 02:52:08,316::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 02:52:08,316::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 02:52:08,317::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 02:52:08,318::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 02:52:08,318::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 02:52:08,319::task::495::TaskManager.Task::(_debug) Task e27e764f-4238-46ff-8226-ef3a5ae59a87: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 02:52:08,319::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 02:52:08,628::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.082776 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:52:10,642::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913032::INFO::2013-11-20 02:52:10,873::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913032::DEBUG::2013-11-20 02:52:10,874::task::495::TaskManager.Task::(_debug) Task 96f42030-20dd-4cc6-9c71-35dffb780d3d: moving from state init -> state preparing
>Thread-15913032::DEBUG::2013-11-20 02:52:10,875::task::495::TaskManager.Task::(_debug) Task 96f42030-20dd-4cc6-9c71-35dffb780d3d: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000420808792114', 'lastCheck': 1384915927.7614231, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00159597396851', 'lastCheck': 1384915926.1905799, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0158338546753', 'lastCheck': 1384915927.214159, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000525951385498', 'lastCheck': 1384915925.190167, 'valid': True, 'code': 0}}
>Thread-15913032::DEBUG::2013-11-20 02:52:10,876::task::495::TaskManager.Task::(_debug) Task 96f42030-20dd-4cc6-9c71-35dffb780d3d: moving from state preparing -> state finished
>Thread-15913032::DEBUG::2013-11-20 02:52:10,876::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913032::DEBUG::2013-11-20 02:52:10,877::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913032::DEBUG::2013-11-20 02:52:10,878::task::495::TaskManager.Task::(_debug) Task 96f42030-20dd-4cc6-9c71-35dffb780d3d: ref 0 aborting False
>Thread-15913032::INFO::2013-11-20 02:52:10,878::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000420808792114', 'lastCheck': 1384915927.7614231, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00159597396851', 'lastCheck': 1384915926.1905799, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0158338546753', 'lastCheck': 1384915927.214159, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000525951385498', 'lastCheck': 1384915925.190167, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:52:11,169::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0735897 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:52:13,184::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:52:13,741::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.10415 s, 9.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:52:15,756::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:52:16,296::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0710003 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:52:18,312::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913036::DEBUG::2013-11-20 02:52:18,521::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913036::INFO::2013-11-20 02:52:18,523::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913036::DEBUG::2013-11-20 02:52:18,523::task::495::TaskManager.Task::(_debug) Task 09ea68e9-d734-4ae3-b573-23f8e51d3e50: moving from state init -> state preparing
>Thread-15913036::DEBUG::2013-11-20 02:52:18,524::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913036::DEBUG::2013-11-20 02:52:18,524::task::495::TaskManager.Task::(_debug) Task 09ea68e9-d734-4ae3-b573-23f8e51d3e50: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913036::DEBUG::2013-11-20 02:52:18,525::task::495::TaskManager.Task::(_debug) Task 09ea68e9-d734-4ae3-b573-23f8e51d3e50: moving from state preparing -> state finished
>Thread-15913036::DEBUG::2013-11-20 02:52:18,525::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913036::DEBUG::2013-11-20 02:52:18,525::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913036::DEBUG::2013-11-20 02:52:18,526::task::495::TaskManager.Task::(_debug) Task 09ea68e9-d734-4ae3-b573-23f8e51d3e50: ref 0 aborting False
>Thread-15913036::INFO::2013-11-20 02:52:18,526::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913037::DEBUG::2013-11-20 02:52:18,530::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913037::INFO::2013-11-20 02:52:18,531::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913037::DEBUG::2013-11-20 02:52:18,531::task::495::TaskManager.Task::(_debug) Task c13b1a2c-b627-4197-b064-1f4752aa019a: moving from state init -> state preparing
>Thread-15913037::DEBUG::2013-11-20 02:52:18,533::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f497ec5c-a766-407a-8d44-827b6939f5c8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913037::DEBUG::2013-11-20 02:52:18,533::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913037::DEBUG::2013-11-20 02:52:18,534::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913037::DEBUG::2013-11-20 02:52:18,534::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f497ec5c-a766-407a-8d44-827b6939f5c8`::Granted request
>Thread-15913037::DEBUG::2013-11-20 02:52:18,535::task::495::TaskManager.Task::(_debug) Task c13b1a2c-b627-4197-b064-1f4752aa019a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913037::DEBUG::2013-11-20 02:52:18,535::task::495::TaskManager.Task::(_debug) Task c13b1a2c-b627-4197-b064-1f4752aa019a: ref 1 aborting False
>Thread-15913037::WARNING::2013-11-20 02:52:18,537::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913037::DEBUG::2013-11-20 02:52:18,537::task::495::TaskManager.Task::(_debug) Task c13b1a2c-b627-4197-b064-1f4752aa019a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115405225984', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115405225984', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913037::DEBUG::2013-11-20 02:52:18,538::task::495::TaskManager.Task::(_debug) Task c13b1a2c-b627-4197-b064-1f4752aa019a: moving from state preparing -> state finished
>Thread-15913037::DEBUG::2013-11-20 02:52:18,538::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913037::DEBUG::2013-11-20 02:52:18,538::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913037::DEBUG::2013-11-20 02:52:18,539::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913037::DEBUG::2013-11-20 02:52:18,539::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913037::DEBUG::2013-11-20 02:52:18,540::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913037::DEBUG::2013-11-20 02:52:18,540::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913037::DEBUG::2013-11-20 02:52:18,540::task::495::TaskManager.Task::(_debug) Task c13b1a2c-b627-4197-b064-1f4752aa019a: ref 0 aborting False
>Thread-15913037::INFO::2013-11-20 02:52:18,541::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115405225984', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115405225984', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:52:18,818::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0728216 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:52:20,833::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:52:21,351::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0717271 s, 14.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:52:23,366::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:52:23,894::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0768041 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:52:25,909::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913040::INFO::2013-11-20 02:52:26,271::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913040::DEBUG::2013-11-20 02:52:26,271::task::495::TaskManager.Task::(_debug) Task 1520f0a6-49b2-42b2-97e8-5a1b8eb05e0f: moving from state init -> state preparing
>Thread-15913040::DEBUG::2013-11-20 02:52:26,272::task::495::TaskManager.Task::(_debug) Task 1520f0a6-49b2-42b2-97e8-5a1b8eb05e0f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000324010848999', 'lastCheck': 1384915937.982949, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00210285186768', 'lastCheck': 1384915936.404006, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00155901908875', 'lastCheck': 1384915937.4039919, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000532150268555', 'lastCheck': 1384915945.596802, 'valid': True, 'code': 0}}
>Thread-15913040::DEBUG::2013-11-20 02:52:26,273::task::495::TaskManager.Task::(_debug) Task 1520f0a6-49b2-42b2-97e8-5a1b8eb05e0f: moving from state preparing -> state finished
>Thread-15913040::DEBUG::2013-11-20 02:52:26,273::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913040::DEBUG::2013-11-20 02:52:26,274::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913040::DEBUG::2013-11-20 02:52:26,274::task::495::TaskManager.Task::(_debug) Task 1520f0a6-49b2-42b2-97e8-5a1b8eb05e0f: ref 0 aborting False
>Thread-15913040::INFO::2013-11-20 02:52:26,274::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000324010848999', 'lastCheck': 1384915937.982949, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00210285186768', 'lastCheck': 1384915936.404006, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00155901908875', 'lastCheck': 1384915937.4039919, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000532150268555', 'lastCheck': 1384915945.596802, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:52:26,435::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0709268 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:52:28,450::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:52:28,986::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0787868 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15913042::DEBUG::2013-11-20 02:52:29,046::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913042::INFO::2013-11-20 02:52:29,047::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913042::DEBUG::2013-11-20 02:52:29,047::task::495::TaskManager.Task::(_debug) Task f45f94c2-a889-4f42-9b1f-11a5b5e742be: moving from state init -> state preparing
>Thread-15913042::DEBUG::2013-11-20 02:52:29,048::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913042::DEBUG::2013-11-20 02:52:29,048::task::495::TaskManager.Task::(_debug) Task f45f94c2-a889-4f42-9b1f-11a5b5e742be: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913042::DEBUG::2013-11-20 02:52:29,048::task::495::TaskManager.Task::(_debug) Task f45f94c2-a889-4f42-9b1f-11a5b5e742be: moving from state preparing -> state finished
>Thread-15913042::DEBUG::2013-11-20 02:52:29,049::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913042::DEBUG::2013-11-20 02:52:29,049::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913042::DEBUG::2013-11-20 02:52:29,049::task::495::TaskManager.Task::(_debug) Task f45f94c2-a889-4f42-9b1f-11a5b5e742be: ref 0 aborting False
>Thread-15913042::INFO::2013-11-20 02:52:29,050::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913043::DEBUG::2013-11-20 02:52:29,053::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913043::INFO::2013-11-20 02:52:29,053::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913043::DEBUG::2013-11-20 02:52:29,054::task::495::TaskManager.Task::(_debug) Task 381d1af5-a4b8-4286-aae5-1d86c008dbde: moving from state init -> state preparing
>Thread-15913043::DEBUG::2013-11-20 02:52:29,055::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`dd194b59-31ac-4a6f-bd2d-e1b20982c866`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913043::DEBUG::2013-11-20 02:52:29,055::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913043::DEBUG::2013-11-20 02:52:29,055::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913043::DEBUG::2013-11-20 02:52:29,056::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`dd194b59-31ac-4a6f-bd2d-e1b20982c866`::Granted request
>Thread-15913043::DEBUG::2013-11-20 02:52:29,056::task::495::TaskManager.Task::(_debug) Task 381d1af5-a4b8-4286-aae5-1d86c008dbde: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913043::DEBUG::2013-11-20 02:52:29,057::task::495::TaskManager.Task::(_debug) Task 381d1af5-a4b8-4286-aae5-1d86c008dbde: ref 1 aborting False
>Thread-15913043::WARNING::2013-11-20 02:52:29,058::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913043::DEBUG::2013-11-20 02:52:29,059::task::495::TaskManager.Task::(_debug) Task 381d1af5-a4b8-4286-aae5-1d86c008dbde: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115405225984', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913043::DEBUG::2013-11-20 02:52:29,059::task::495::TaskManager.Task::(_debug) Task 381d1af5-a4b8-4286-aae5-1d86c008dbde: moving from state preparing -> state finished
>Thread-15913043::DEBUG::2013-11-20 02:52:29,059::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913043::DEBUG::2013-11-20 02:52:29,060::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913043::DEBUG::2013-11-20 02:52:29,060::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913043::DEBUG::2013-11-20 02:52:29,061::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913043::DEBUG::2013-11-20 02:52:29,061::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913043::DEBUG::2013-11-20 02:52:29,062::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913043::DEBUG::2013-11-20 02:52:29,062::task::495::TaskManager.Task::(_debug) Task 381d1af5-a4b8-4286-aae5-1d86c008dbde: ref 0 aborting False
>Thread-15913043::INFO::2013-11-20 02:52:29,062::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115405225984', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:52:31,001::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:52:31,539::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.075363 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:52:33,554::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:52:34,102::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0738119 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:52:36,117::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:52:36,615::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0730562 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:52:38,630::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:52:39,151::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0644409 s, 15.9 MB/s\n'; <rc> = 0
>Thread-15913048::DEBUG::2013-11-20 02:52:39,349::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913048::INFO::2013-11-20 02:52:39,350::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913048::DEBUG::2013-11-20 02:52:39,351::task::495::TaskManager.Task::(_debug) Task fec0356d-de1f-4d59-9a67-29528bfd60fd: moving from state init -> state preparing
>Thread-15913048::DEBUG::2013-11-20 02:52:39,352::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913048::DEBUG::2013-11-20 02:52:39,352::task::495::TaskManager.Task::(_debug) Task fec0356d-de1f-4d59-9a67-29528bfd60fd: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913048::DEBUG::2013-11-20 02:52:39,353::task::495::TaskManager.Task::(_debug) Task fec0356d-de1f-4d59-9a67-29528bfd60fd: moving from state preparing -> state finished
>Thread-15913048::DEBUG::2013-11-20 02:52:39,354::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913048::DEBUG::2013-11-20 02:52:39,354::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913048::DEBUG::2013-11-20 02:52:39,355::task::495::TaskManager.Task::(_debug) Task fec0356d-de1f-4d59-9a67-29528bfd60fd: ref 0 aborting False
>Thread-15913048::INFO::2013-11-20 02:52:39,356::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913049::DEBUG::2013-11-20 02:52:39,359::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913049::INFO::2013-11-20 02:52:39,360::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913049::DEBUG::2013-11-20 02:52:39,361::task::495::TaskManager.Task::(_debug) Task 1894e705-ee8a-4c04-92b7-ab2f3a369857: moving from state init -> state preparing
>Thread-15913049::DEBUG::2013-11-20 02:52:39,362::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2b7cf11b-3dbf-4dee-b526-bb1f5fc920e8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913049::DEBUG::2013-11-20 02:52:39,363::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913049::DEBUG::2013-11-20 02:52:39,363::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913049::DEBUG::2013-11-20 02:52:39,364::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2b7cf11b-3dbf-4dee-b526-bb1f5fc920e8`::Granted request
>Thread-15913049::DEBUG::2013-11-20 02:52:39,365::task::495::TaskManager.Task::(_debug) Task 1894e705-ee8a-4c04-92b7-ab2f3a369857: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913049::DEBUG::2013-11-20 02:52:39,366::task::495::TaskManager.Task::(_debug) Task 1894e705-ee8a-4c04-92b7-ab2f3a369857: ref 1 aborting False
>Thread-15913049::WARNING::2013-11-20 02:52:39,368::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913049::DEBUG::2013-11-20 02:52:39,368::task::495::TaskManager.Task::(_debug) Task 1894e705-ee8a-4c04-92b7-ab2f3a369857: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913049::DEBUG::2013-11-20 02:52:39,369::task::495::TaskManager.Task::(_debug) Task 1894e705-ee8a-4c04-92b7-ab2f3a369857: moving from state preparing -> state finished
>Thread-15913049::DEBUG::2013-11-20 02:52:39,370::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913049::DEBUG::2013-11-20 02:52:39,370::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913049::DEBUG::2013-11-20 02:52:39,371::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913049::DEBUG::2013-11-20 02:52:39,372::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913049::DEBUG::2013-11-20 02:52:39,372::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913049::DEBUG::2013-11-20 02:52:39,373::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913049::DEBUG::2013-11-20 02:52:39,373::task::495::TaskManager.Task::(_debug) Task 1894e705-ee8a-4c04-92b7-ab2f3a369857: ref 0 aborting False
>Thread-15913049::INFO::2013-11-20 02:52:39,374::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:52:41,161::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:52:41,690::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0829689 s, 12.3 MB/s\n'; <rc> = 0
>Thread-15913050::INFO::2013-11-20 02:52:41,902::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913050::DEBUG::2013-11-20 02:52:41,903::task::495::TaskManager.Task::(_debug) Task dc60d0e2-29e8-49d0-a919-8887da294767: moving from state init -> state preparing
>Thread-15913050::DEBUG::2013-11-20 02:52:41,904::task::495::TaskManager.Task::(_debug) Task dc60d0e2-29e8-49d0-a919-8887da294767: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000300168991089', 'lastCheck': 1384915958.324738, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00156497955322', 'lastCheck': 1384915956.805903, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00141620635986', 'lastCheck': 1384915957.8051829, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00049090385437', 'lastCheck': 1384915955.7982471, 'valid': True, 'code': 0}}
>Thread-15913050::DEBUG::2013-11-20 02:52:41,905::task::495::TaskManager.Task::(_debug) Task dc60d0e2-29e8-49d0-a919-8887da294767: moving from state preparing -> state finished
>Thread-15913050::DEBUG::2013-11-20 02:52:41,906::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913050::DEBUG::2013-11-20 02:52:41,906::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913050::DEBUG::2013-11-20 02:52:41,907::task::495::TaskManager.Task::(_debug) Task dc60d0e2-29e8-49d0-a919-8887da294767: ref 0 aborting False
>Thread-15913050::INFO::2013-11-20 02:52:41,907::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000300168991089', 'lastCheck': 1384915958.324738, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00156497955322', 'lastCheck': 1384915956.805903, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00141620635986', 'lastCheck': 1384915957.8051829, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00049090385437', 'lastCheck': 1384915955.7982471, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:52:43,706::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:52:44,216::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0681689 s, 15.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:52:46,231::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:52:46,775::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0960525 s, 10.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:52:48,791::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:52:49,266::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0479717 s, 21.3 MB/s\n'; <rc> = 0
>Thread-15913054::DEBUG::2013-11-20 02:52:49,581::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913054::INFO::2013-11-20 02:52:49,583::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913054::DEBUG::2013-11-20 02:52:49,584::task::495::TaskManager.Task::(_debug) Task 3b869c51-1bc1-40a6-b8e2-3633e0e9fb31: moving from state init -> state preparing
>Thread-15913054::DEBUG::2013-11-20 02:52:49,586::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913054::DEBUG::2013-11-20 02:52:49,587::task::495::TaskManager.Task::(_debug) Task 3b869c51-1bc1-40a6-b8e2-3633e0e9fb31: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913054::DEBUG::2013-11-20 02:52:49,588::task::495::TaskManager.Task::(_debug) Task 3b869c51-1bc1-40a6-b8e2-3633e0e9fb31: moving from state preparing -> state finished
>Thread-15913054::DEBUG::2013-11-20 02:52:49,589::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913054::DEBUG::2013-11-20 02:52:49,589::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913054::DEBUG::2013-11-20 02:52:49,590::task::495::TaskManager.Task::(_debug) Task 3b869c51-1bc1-40a6-b8e2-3633e0e9fb31: ref 0 aborting False
>Thread-15913054::INFO::2013-11-20 02:52:49,591::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913055::DEBUG::2013-11-20 02:52:49,595::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913055::INFO::2013-11-20 02:52:49,596::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913055::DEBUG::2013-11-20 02:52:49,597::task::495::TaskManager.Task::(_debug) Task f9f0e4f0-eae7-44d3-acb6-957e9760d0ba: moving from state init -> state preparing
>Thread-15913055::DEBUG::2013-11-20 02:52:49,598::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bb3a305c-44eb-47d2-88c8-06920a3713e7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913055::DEBUG::2013-11-20 02:52:49,599::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913055::DEBUG::2013-11-20 02:52:49,600::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913055::DEBUG::2013-11-20 02:52:49,600::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bb3a305c-44eb-47d2-88c8-06920a3713e7`::Granted request
>Thread-15913055::DEBUG::2013-11-20 02:52:49,601::task::495::TaskManager.Task::(_debug) Task f9f0e4f0-eae7-44d3-acb6-957e9760d0ba: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913055::DEBUG::2013-11-20 02:52:49,602::task::495::TaskManager.Task::(_debug) Task f9f0e4f0-eae7-44d3-acb6-957e9760d0ba: ref 1 aborting False
>Thread-15913055::WARNING::2013-11-20 02:52:49,604::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913055::DEBUG::2013-11-20 02:52:49,605::task::495::TaskManager.Task::(_debug) Task f9f0e4f0-eae7-44d3-acb6-957e9760d0ba: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913055::DEBUG::2013-11-20 02:52:49,606::task::495::TaskManager.Task::(_debug) Task f9f0e4f0-eae7-44d3-acb6-957e9760d0ba: moving from state preparing -> state finished
>Thread-15913055::DEBUG::2013-11-20 02:52:49,607::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913055::DEBUG::2013-11-20 02:52:49,607::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913055::DEBUG::2013-11-20 02:52:49,608::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913055::DEBUG::2013-11-20 02:52:49,608::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913055::DEBUG::2013-11-20 02:52:49,609::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913055::DEBUG::2013-11-20 02:52:49,609::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913055::DEBUG::2013-11-20 02:52:49,610::task::495::TaskManager.Task::(_debug) Task f9f0e4f0-eae7-44d3-acb6-957e9760d0ba: ref 0 aborting False
>Thread-15913055::INFO::2013-11-20 02:52:49,611::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:52:51,282::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:52:51,833::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0744025 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:52:53,848::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:52:54,352::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0695366 s, 14.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:52:56,367::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:52:56,889::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0729321 s, 14.0 MB/s\n'; <rc> = 0
>Thread-15913058::INFO::2013-11-20 02:52:57,126::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913058::DEBUG::2013-11-20 02:52:57,127::task::495::TaskManager.Task::(_debug) Task 1225be40-3b44-4da8-a220-4592420bae22: moving from state init -> state preparing
>Thread-15913058::DEBUG::2013-11-20 02:52:57,128::task::495::TaskManager.Task::(_debug) Task 1225be40-3b44-4da8-a220-4592420bae22: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000334024429321', 'lastCheck': 1384915968.435797, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00156998634338', 'lastCheck': 1384915977.0586939, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00158905982971', 'lastCheck': 1384915967.945555, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000483989715576', 'lastCheck': 1384915976.052932, 'valid': True, 'code': 0}}
>Thread-15913058::DEBUG::2013-11-20 02:52:57,129::task::495::TaskManager.Task::(_debug) Task 1225be40-3b44-4da8-a220-4592420bae22: moving from state preparing -> state finished
>Thread-15913058::DEBUG::2013-11-20 02:52:57,129::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913058::DEBUG::2013-11-20 02:52:57,130::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913058::DEBUG::2013-11-20 02:52:57,130::task::495::TaskManager.Task::(_debug) Task 1225be40-3b44-4da8-a220-4592420bae22: ref 0 aborting False
>Thread-15913058::INFO::2013-11-20 02:52:57,131::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000334024429321', 'lastCheck': 1384915968.435797, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00156998634338', 'lastCheck': 1384915977.0586939, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00158905982971', 'lastCheck': 1384915967.945555, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000483989715576', 'lastCheck': 1384915976.052932, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:52:58,904::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:52:59,446::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.073278 s, 14.0 MB/s\n'; <rc> = 0
>Thread-15913060::DEBUG::2013-11-20 02:52:59,753::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913060::INFO::2013-11-20 02:52:59,754::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913060::DEBUG::2013-11-20 02:52:59,755::task::495::TaskManager.Task::(_debug) Task 0fcf85e3-ddf3-4481-9619-385dbec7cadc: moving from state init -> state preparing
>Thread-15913060::DEBUG::2013-11-20 02:52:59,756::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913060::DEBUG::2013-11-20 02:52:59,756::task::495::TaskManager.Task::(_debug) Task 0fcf85e3-ddf3-4481-9619-385dbec7cadc: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913060::DEBUG::2013-11-20 02:52:59,757::task::495::TaskManager.Task::(_debug) Task 0fcf85e3-ddf3-4481-9619-385dbec7cadc: moving from state preparing -> state finished
>Thread-15913060::DEBUG::2013-11-20 02:52:59,758::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913060::DEBUG::2013-11-20 02:52:59,758::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913060::DEBUG::2013-11-20 02:52:59,759::task::495::TaskManager.Task::(_debug) Task 0fcf85e3-ddf3-4481-9619-385dbec7cadc: ref 0 aborting False
>Thread-15913060::INFO::2013-11-20 02:52:59,759::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913061::DEBUG::2013-11-20 02:52:59,764::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913061::INFO::2013-11-20 02:52:59,765::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913061::DEBUG::2013-11-20 02:52:59,765::task::495::TaskManager.Task::(_debug) Task e515f78e-de64-4bac-8acd-7cbb5aa373f1: moving from state init -> state preparing
>Thread-15913061::DEBUG::2013-11-20 02:52:59,767::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e8606c6e-5860-46f4-8c08-e870ffdafd1e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913061::DEBUG::2013-11-20 02:52:59,767::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913061::DEBUG::2013-11-20 02:52:59,768::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913061::DEBUG::2013-11-20 02:52:59,768::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e8606c6e-5860-46f4-8c08-e870ffdafd1e`::Granted request
>Thread-15913061::DEBUG::2013-11-20 02:52:59,770::task::495::TaskManager.Task::(_debug) Task e515f78e-de64-4bac-8acd-7cbb5aa373f1: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913061::DEBUG::2013-11-20 02:52:59,770::task::495::TaskManager.Task::(_debug) Task e515f78e-de64-4bac-8acd-7cbb5aa373f1: ref 1 aborting False
>Thread-15913061::WARNING::2013-11-20 02:52:59,772::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913061::DEBUG::2013-11-20 02:52:59,773::task::495::TaskManager.Task::(_debug) Task e515f78e-de64-4bac-8acd-7cbb5aa373f1: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913061::DEBUG::2013-11-20 02:52:59,774::task::495::TaskManager.Task::(_debug) Task e515f78e-de64-4bac-8acd-7cbb5aa373f1: moving from state preparing -> state finished
>Thread-15913061::DEBUG::2013-11-20 02:52:59,774::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913061::DEBUG::2013-11-20 02:52:59,775::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913061::DEBUG::2013-11-20 02:52:59,776::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913061::DEBUG::2013-11-20 02:52:59,776::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913061::DEBUG::2013-11-20 02:52:59,777::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913061::DEBUG::2013-11-20 02:52:59,777::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913061::DEBUG::2013-11-20 02:52:59,778::task::495::TaskManager.Task::(_debug) Task e515f78e-de64-4bac-8acd-7cbb5aa373f1: ref 0 aborting False
>Thread-15913061::INFO::2013-11-20 02:52:59,779::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:53:01,461::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:53:01,989::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0731514 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:53:04,004::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:53:04,558::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0723847 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:53:06,573::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:53:07,103::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0515021 s, 19.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:53:09,114::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 02:53:09,360::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 02:53:09,362::task::495::TaskManager.Task::(_debug) Task e187b3f5-545a-460c-a987-00dddf5859f7: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 02:53:09,380::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`111b306a-ecd3-4df6-b1b6-0616e736e104`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 02:53:09,381::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 02:53:09,382::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 02:53:09,382::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`111b306a-ecd3-4df6-b1b6-0616e736e104`::Granted request
>Thread-15912392::DEBUG::2013-11-20 02:53:09,383::task::495::TaskManager.Task::(_debug) Task e187b3f5-545a-460c-a987-00dddf5859f7: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 02:53:09,384::task::495::TaskManager.Task::(_debug) Task e187b3f5-545a-460c-a987-00dddf5859f7: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 02:53:09,385::task::495::TaskManager.Task::(_debug) Task e187b3f5-545a-460c-a987-00dddf5859f7: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 02:53:09,385::task::495::TaskManager.Task::(_debug) Task e187b3f5-545a-460c-a987-00dddf5859f7: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 02:53:09,386::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 02:53:09,386::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 02:53:09,387::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 02:53:09,388::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 02:53:09,388::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 02:53:09,389::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 02:53:09,389::task::495::TaskManager.Task::(_debug) Task e187b3f5-545a-460c-a987-00dddf5859f7: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 02:53:09,390::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 02:53:09,662::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0585003 s, 17.5 MB/s\n'; <rc> = 0
>Thread-15913066::DEBUG::2013-11-20 02:53:10,352::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913066::INFO::2013-11-20 02:53:10,353::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913066::DEBUG::2013-11-20 02:53:10,354::task::495::TaskManager.Task::(_debug) Task 6fad9d52-2fe4-4600-b1d0-868720b24fd3: moving from state init -> state preparing
>Thread-15913066::DEBUG::2013-11-20 02:53:10,355::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913066::DEBUG::2013-11-20 02:53:10,355::task::495::TaskManager.Task::(_debug) Task 6fad9d52-2fe4-4600-b1d0-868720b24fd3: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913066::DEBUG::2013-11-20 02:53:10,355::task::495::TaskManager.Task::(_debug) Task 6fad9d52-2fe4-4600-b1d0-868720b24fd3: moving from state preparing -> state finished
>Thread-15913066::DEBUG::2013-11-20 02:53:10,356::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913066::DEBUG::2013-11-20 02:53:10,356::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913066::DEBUG::2013-11-20 02:53:10,356::task::495::TaskManager.Task::(_debug) Task 6fad9d52-2fe4-4600-b1d0-868720b24fd3: ref 0 aborting False
>Thread-15913066::INFO::2013-11-20 02:53:10,357::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913067::DEBUG::2013-11-20 02:53:10,360::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913067::INFO::2013-11-20 02:53:10,360::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913067::DEBUG::2013-11-20 02:53:10,361::task::495::TaskManager.Task::(_debug) Task f7361d0d-dadc-417b-b307-b0d155f7da5a: moving from state init -> state preparing
>Thread-15913067::DEBUG::2013-11-20 02:53:10,362::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4759f247-e48d-4ff1-b26b-4f34988d8904`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913067::DEBUG::2013-11-20 02:53:10,362::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913067::DEBUG::2013-11-20 02:53:10,362::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913067::DEBUG::2013-11-20 02:53:10,363::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4759f247-e48d-4ff1-b26b-4f34988d8904`::Granted request
>Thread-15913067::DEBUG::2013-11-20 02:53:10,363::task::495::TaskManager.Task::(_debug) Task f7361d0d-dadc-417b-b307-b0d155f7da5a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913067::DEBUG::2013-11-20 02:53:10,364::task::495::TaskManager.Task::(_debug) Task f7361d0d-dadc-417b-b307-b0d155f7da5a: ref 1 aborting False
>Thread-15913067::WARNING::2013-11-20 02:53:10,365::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913067::DEBUG::2013-11-20 02:53:10,365::task::495::TaskManager.Task::(_debug) Task f7361d0d-dadc-417b-b307-b0d155f7da5a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913067::DEBUG::2013-11-20 02:53:10,366::task::495::TaskManager.Task::(_debug) Task f7361d0d-dadc-417b-b307-b0d155f7da5a: moving from state preparing -> state finished
>Thread-15913067::DEBUG::2013-11-20 02:53:10,366::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913067::DEBUG::2013-11-20 02:53:10,366::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913067::DEBUG::2013-11-20 02:53:10,367::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913067::DEBUG::2013-11-20 02:53:10,367::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913067::DEBUG::2013-11-20 02:53:10,367::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913067::DEBUG::2013-11-20 02:53:10,368::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913067::DEBUG::2013-11-20 02:53:10,368::task::495::TaskManager.Task::(_debug) Task f7361d0d-dadc-417b-b307-b0d155f7da5a: ref 0 aborting False
>Thread-15913067::INFO::2013-11-20 02:53:10,369::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:53:11,672::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:53:12,156::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0576131 s, 17.8 MB/s\n'; <rc> = 0
>Thread-15913068::INFO::2013-11-20 02:53:12,855::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913068::DEBUG::2013-11-20 02:53:12,856::task::495::TaskManager.Task::(_debug) Task f6268304-c25e-4ae4-bcf7-6faf16e5e4ed: moving from state init -> state preparing
>Thread-15913068::DEBUG::2013-11-20 02:53:12,857::task::495::TaskManager.Task::(_debug) Task f6268304-c25e-4ae4-bcf7-6faf16e5e4ed: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000325918197632', 'lastCheck': 1384915988.8211141, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.01038813591', 'lastCheck': 1384915987.2520299, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0269019603729', 'lastCheck': 1384915988.273618, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000505208969116', 'lastCheck': 1384915986.240344, 'valid': True, 'code': 0}}
>Thread-15913068::DEBUG::2013-11-20 02:53:12,858::task::495::TaskManager.Task::(_debug) Task f6268304-c25e-4ae4-bcf7-6faf16e5e4ed: moving from state preparing -> state finished
>Thread-15913068::DEBUG::2013-11-20 02:53:12,858::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913068::DEBUG::2013-11-20 02:53:12,859::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913068::DEBUG::2013-11-20 02:53:12,859::task::495::TaskManager.Task::(_debug) Task f6268304-c25e-4ae4-bcf7-6faf16e5e4ed: ref 0 aborting False
>Thread-15913068::INFO::2013-11-20 02:53:12,860::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000325918197632', 'lastCheck': 1384915988.8211141, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.01038813591', 'lastCheck': 1384915987.2520299, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0269019603729', 'lastCheck': 1384915988.273618, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000505208969116', 'lastCheck': 1384915986.240344, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:53:14,169::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:53:14,728::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0722109 s, 14.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:53:16,743::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:53:17,285::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0934037 s, 11.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:53:19,301::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:53:19,824::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0524574 s, 19.5 MB/s\n'; <rc> = 0
>Thread-15913072::DEBUG::2013-11-20 02:53:20,878::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913072::INFO::2013-11-20 02:53:20,879::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913072::DEBUG::2013-11-20 02:53:20,880::task::495::TaskManager.Task::(_debug) Task 8ba0637f-174b-4443-9078-48bced3a414f: moving from state init -> state preparing
>Thread-15913072::DEBUG::2013-11-20 02:53:20,881::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913072::DEBUG::2013-11-20 02:53:20,881::task::495::TaskManager.Task::(_debug) Task 8ba0637f-174b-4443-9078-48bced3a414f: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913072::DEBUG::2013-11-20 02:53:20,882::task::495::TaskManager.Task::(_debug) Task 8ba0637f-174b-4443-9078-48bced3a414f: moving from state preparing -> state finished
>Thread-15913072::DEBUG::2013-11-20 02:53:20,883::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913072::DEBUG::2013-11-20 02:53:20,883::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913072::DEBUG::2013-11-20 02:53:20,884::task::495::TaskManager.Task::(_debug) Task 8ba0637f-174b-4443-9078-48bced3a414f: ref 0 aborting False
>Thread-15913072::INFO::2013-11-20 02:53:20,884::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913073::DEBUG::2013-11-20 02:53:20,888::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913073::INFO::2013-11-20 02:53:20,889::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913073::DEBUG::2013-11-20 02:53:20,890::task::495::TaskManager.Task::(_debug) Task 8c424a84-eff1-423e-bc2b-8b0071ae3171: moving from state init -> state preparing
>Thread-15913073::DEBUG::2013-11-20 02:53:20,891::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9be51bfa-70dd-4b58-9490-021ae7d594c0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913073::DEBUG::2013-11-20 02:53:20,892::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913073::DEBUG::2013-11-20 02:53:20,893::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913073::DEBUG::2013-11-20 02:53:20,893::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9be51bfa-70dd-4b58-9490-021ae7d594c0`::Granted request
>Thread-15913073::DEBUG::2013-11-20 02:53:20,895::task::495::TaskManager.Task::(_debug) Task 8c424a84-eff1-423e-bc2b-8b0071ae3171: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913073::DEBUG::2013-11-20 02:53:20,895::task::495::TaskManager.Task::(_debug) Task 8c424a84-eff1-423e-bc2b-8b0071ae3171: ref 1 aborting False
>Thread-15913073::WARNING::2013-11-20 02:53:20,897::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913073::DEBUG::2013-11-20 02:53:20,898::task::495::TaskManager.Task::(_debug) Task 8c424a84-eff1-423e-bc2b-8b0071ae3171: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913073::DEBUG::2013-11-20 02:53:20,899::task::495::TaskManager.Task::(_debug) Task 8c424a84-eff1-423e-bc2b-8b0071ae3171: moving from state preparing -> state finished
>Thread-15913073::DEBUG::2013-11-20 02:53:20,899::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913073::DEBUG::2013-11-20 02:53:20,900::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913073::DEBUG::2013-11-20 02:53:20,901::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913073::DEBUG::2013-11-20 02:53:20,901::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913073::DEBUG::2013-11-20 02:53:20,902::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913073::DEBUG::2013-11-20 02:53:20,902::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913073::DEBUG::2013-11-20 02:53:20,903::task::495::TaskManager.Task::(_debug) Task 8c424a84-eff1-423e-bc2b-8b0071ae3171: ref 0 aborting False
>Thread-15913073::INFO::2013-11-20 02:53:20,904::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:53:21,836::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:53:22,368::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0733921 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:53:24,383::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:53:24,877::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0556808 s, 18.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:53:26,891::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:53:27,410::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0583996 s, 17.5 MB/s\n'; <rc> = 0
>Thread-15913076::INFO::2013-11-20 02:53:28,570::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913076::DEBUG::2013-11-20 02:53:28,570::task::495::TaskManager.Task::(_debug) Task a1eccaa2-394e-47dc-8657-495e892084ad: moving from state init -> state preparing
>Thread-15913076::DEBUG::2013-11-20 02:53:28,571::task::495::TaskManager.Task::(_debug) Task a1eccaa2-394e-47dc-8657-495e892084ad: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000354051589966', 'lastCheck': 1384915998.969451, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00585985183716', 'lastCheck': 1384916007.624815, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00170588493347', 'lastCheck': 1384915998.427392, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000580072402954', 'lastCheck': 1384916006.6167281, 'valid': True, 'code': 0}}
>Thread-15913076::DEBUG::2013-11-20 02:53:28,571::task::495::TaskManager.Task::(_debug) Task a1eccaa2-394e-47dc-8657-495e892084ad: moving from state preparing -> state finished
>Thread-15913076::DEBUG::2013-11-20 02:53:28,572::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913076::DEBUG::2013-11-20 02:53:28,572::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913076::DEBUG::2013-11-20 02:53:28,572::task::495::TaskManager.Task::(_debug) Task a1eccaa2-394e-47dc-8657-495e892084ad: ref 0 aborting False
>Thread-15913076::INFO::2013-11-20 02:53:28,573::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000354051589966', 'lastCheck': 1384915998.969451, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00585985183716', 'lastCheck': 1384916007.624815, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00170588493347', 'lastCheck': 1384915998.427392, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000580072402954', 'lastCheck': 1384916006.6167281, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:53:29,421::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:53:29,904::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0587088 s, 17.4 MB/s\n'; <rc> = 0
>Thread-15913078::DEBUG::2013-11-20 02:53:31,207::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913078::INFO::2013-11-20 02:53:31,209::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913078::DEBUG::2013-11-20 02:53:31,209::task::495::TaskManager.Task::(_debug) Task e2643eb5-deff-4d09-af53-560734ffe268: moving from state init -> state preparing
>Thread-15913078::DEBUG::2013-11-20 02:53:31,210::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913078::DEBUG::2013-11-20 02:53:31,211::task::495::TaskManager.Task::(_debug) Task e2643eb5-deff-4d09-af53-560734ffe268: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913078::DEBUG::2013-11-20 02:53:31,211::task::495::TaskManager.Task::(_debug) Task e2643eb5-deff-4d09-af53-560734ffe268: moving from state preparing -> state finished
>Thread-15913078::DEBUG::2013-11-20 02:53:31,212::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913078::DEBUG::2013-11-20 02:53:31,212::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913078::DEBUG::2013-11-20 02:53:31,213::task::495::TaskManager.Task::(_debug) Task e2643eb5-deff-4d09-af53-560734ffe268: ref 0 aborting False
>Thread-15913078::INFO::2013-11-20 02:53:31,213::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913079::DEBUG::2013-11-20 02:53:31,218::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913079::INFO::2013-11-20 02:53:31,219::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913079::DEBUG::2013-11-20 02:53:31,219::task::495::TaskManager.Task::(_debug) Task c4a7bbbd-6203-4cb5-b44c-e34e3576809d: moving from state init -> state preparing
>Thread-15913079::DEBUG::2013-11-20 02:53:31,221::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d69ddc3e-598b-491c-ad0e-8863d825b8e7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913079::DEBUG::2013-11-20 02:53:31,221::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913079::DEBUG::2013-11-20 02:53:31,222::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913079::DEBUG::2013-11-20 02:53:31,223::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d69ddc3e-598b-491c-ad0e-8863d825b8e7`::Granted request
>Thread-15913079::DEBUG::2013-11-20 02:53:31,224::task::495::TaskManager.Task::(_debug) Task c4a7bbbd-6203-4cb5-b44c-e34e3576809d: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913079::DEBUG::2013-11-20 02:53:31,224::task::495::TaskManager.Task::(_debug) Task c4a7bbbd-6203-4cb5-b44c-e34e3576809d: ref 1 aborting False
>Thread-15913079::WARNING::2013-11-20 02:53:31,226::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913079::DEBUG::2013-11-20 02:53:31,227::task::495::TaskManager.Task::(_debug) Task c4a7bbbd-6203-4cb5-b44c-e34e3576809d: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913079::DEBUG::2013-11-20 02:53:31,228::task::495::TaskManager.Task::(_debug) Task c4a7bbbd-6203-4cb5-b44c-e34e3576809d: moving from state preparing -> state finished
>Thread-15913079::DEBUG::2013-11-20 02:53:31,229::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913079::DEBUG::2013-11-20 02:53:31,230::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913079::DEBUG::2013-11-20 02:53:31,231::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913079::DEBUG::2013-11-20 02:53:31,231::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913079::DEBUG::2013-11-20 02:53:31,232::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913079::DEBUG::2013-11-20 02:53:31,232::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913079::DEBUG::2013-11-20 02:53:31,233::task::495::TaskManager.Task::(_debug) Task c4a7bbbd-6203-4cb5-b44c-e34e3576809d: ref 0 aborting False
>Thread-15913079::INFO::2013-11-20 02:53:31,234::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:53:31,919::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:53:32,471::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.071518 s, 14.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:53:34,484::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:53:35,006::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0730648 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:53:37,021::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:53:37,554::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0729374 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:53:39,570::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:53:40,119::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0771091 s, 13.3 MB/s\n'; <rc> = 0
>Thread-15913084::DEBUG::2013-11-20 02:53:41,396::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913084::INFO::2013-11-20 02:53:41,397::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913084::DEBUG::2013-11-20 02:53:41,398::task::495::TaskManager.Task::(_debug) Task 4ca15dea-fea0-4517-9d78-c8c3588979e1: moving from state init -> state preparing
>Thread-15913084::DEBUG::2013-11-20 02:53:41,399::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913084::DEBUG::2013-11-20 02:53:41,400::task::495::TaskManager.Task::(_debug) Task 4ca15dea-fea0-4517-9d78-c8c3588979e1: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913084::DEBUG::2013-11-20 02:53:41,400::task::495::TaskManager.Task::(_debug) Task 4ca15dea-fea0-4517-9d78-c8c3588979e1: moving from state preparing -> state finished
>Thread-15913084::DEBUG::2013-11-20 02:53:41,401::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913084::DEBUG::2013-11-20 02:53:41,401::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913084::DEBUG::2013-11-20 02:53:41,402::task::495::TaskManager.Task::(_debug) Task 4ca15dea-fea0-4517-9d78-c8c3588979e1: ref 0 aborting False
>Thread-15913084::INFO::2013-11-20 02:53:41,402::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913085::DEBUG::2013-11-20 02:53:41,406::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913085::INFO::2013-11-20 02:53:41,407::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913085::DEBUG::2013-11-20 02:53:41,407::task::495::TaskManager.Task::(_debug) Task b75eeb42-9075-403d-9f0b-823d03b74c89: moving from state init -> state preparing
>Thread-15913085::DEBUG::2013-11-20 02:53:41,409::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0697a4de-0790-42d1-a864-b4c673b37eb1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913085::DEBUG::2013-11-20 02:53:41,409::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913085::DEBUG::2013-11-20 02:53:41,410::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913085::DEBUG::2013-11-20 02:53:41,411::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0697a4de-0790-42d1-a864-b4c673b37eb1`::Granted request
>Thread-15913085::DEBUG::2013-11-20 02:53:41,412::task::495::TaskManager.Task::(_debug) Task b75eeb42-9075-403d-9f0b-823d03b74c89: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913085::DEBUG::2013-11-20 02:53:41,413::task::495::TaskManager.Task::(_debug) Task b75eeb42-9075-403d-9f0b-823d03b74c89: ref 1 aborting False
>Thread-15913085::WARNING::2013-11-20 02:53:41,415::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913085::DEBUG::2013-11-20 02:53:41,416::task::495::TaskManager.Task::(_debug) Task b75eeb42-9075-403d-9f0b-823d03b74c89: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913085::DEBUG::2013-11-20 02:53:41,416::task::495::TaskManager.Task::(_debug) Task b75eeb42-9075-403d-9f0b-823d03b74c89: moving from state preparing -> state finished
>Thread-15913085::DEBUG::2013-11-20 02:53:41,417::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913085::DEBUG::2013-11-20 02:53:41,418::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913085::DEBUG::2013-11-20 02:53:41,418::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913085::DEBUG::2013-11-20 02:53:41,419::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913085::DEBUG::2013-11-20 02:53:41,420::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913085::DEBUG::2013-11-20 02:53:41,421::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913085::DEBUG::2013-11-20 02:53:41,422::task::495::TaskManager.Task::(_debug) Task b75eeb42-9075-403d-9f0b-823d03b74c89: ref 0 aborting False
>Thread-15913085::INFO::2013-11-20 02:53:41,422::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:53:42,135::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:53:42,658::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0556004 s, 18.4 MB/s\n'; <rc> = 0
>Thread-15913086::INFO::2013-11-20 02:53:44,205::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913086::DEBUG::2013-11-20 02:53:44,206::task::495::TaskManager.Task::(_debug) Task 4321a752-233a-41a9-88be-6f3da5790ec2: moving from state init -> state preparing
>Thread-15913086::DEBUG::2013-11-20 02:53:44,207::task::495::TaskManager.Task::(_debug) Task 4321a752-233a-41a9-88be-6f3da5790ec2: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000328063964844', 'lastCheck': 1384916019.2441721, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00145816802979', 'lastCheck': 1384916017.716085, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00133109092712', 'lastCheck': 1384916018.7147679, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000559091567993', 'lastCheck': 1384916016.7025559, 'valid': True, 'code': 0}}
>Thread-15913086::DEBUG::2013-11-20 02:53:44,207::task::495::TaskManager.Task::(_debug) Task 4321a752-233a-41a9-88be-6f3da5790ec2: moving from state preparing -> state finished
>Thread-15913086::DEBUG::2013-11-20 02:53:44,208::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913086::DEBUG::2013-11-20 02:53:44,209::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913086::DEBUG::2013-11-20 02:53:44,209::task::495::TaskManager.Task::(_debug) Task 4321a752-233a-41a9-88be-6f3da5790ec2: ref 0 aborting False
>Thread-15913086::INFO::2013-11-20 02:53:44,210::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000328063964844', 'lastCheck': 1384916019.2441721, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00145816802979', 'lastCheck': 1384916017.716085, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00133109092712', 'lastCheck': 1384916018.7147679, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000559091567993', 'lastCheck': 1384916016.7025559, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:53:44,668::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:53:45,138::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0703268 s, 14.6 MB/s\n'; <rc> = 0
>Thread-42840::DEBUG::2013-11-20 02:53:46,851::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:53:47,154::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-42840::DEBUG::2013-11-20 02:53:47,650::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:53:47,718::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0874692 s, 11.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:53:49,733::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:53:50,311::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0744929 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15913090::DEBUG::2013-11-20 02:53:51,587::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913090::INFO::2013-11-20 02:53:51,589::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913090::DEBUG::2013-11-20 02:53:51,589::task::495::TaskManager.Task::(_debug) Task da0f7c5f-1e7e-4c6e-a547-c34c13c7733e: moving from state init -> state preparing
>Thread-15913090::DEBUG::2013-11-20 02:53:51,590::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913090::DEBUG::2013-11-20 02:53:51,590::task::495::TaskManager.Task::(_debug) Task da0f7c5f-1e7e-4c6e-a547-c34c13c7733e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913090::DEBUG::2013-11-20 02:53:51,591::task::495::TaskManager.Task::(_debug) Task da0f7c5f-1e7e-4c6e-a547-c34c13c7733e: moving from state preparing -> state finished
>Thread-15913090::DEBUG::2013-11-20 02:53:51,591::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913090::DEBUG::2013-11-20 02:53:51,592::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913090::DEBUG::2013-11-20 02:53:51,592::task::495::TaskManager.Task::(_debug) Task da0f7c5f-1e7e-4c6e-a547-c34c13c7733e: ref 0 aborting False
>Thread-15913090::INFO::2013-11-20 02:53:51,593::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913091::DEBUG::2013-11-20 02:53:51,597::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913091::INFO::2013-11-20 02:53:51,598::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913091::DEBUG::2013-11-20 02:53:51,598::task::495::TaskManager.Task::(_debug) Task d7f221e2-c6d6-4379-a447-acbf2eaf4184: moving from state init -> state preparing
>Thread-15913091::DEBUG::2013-11-20 02:53:51,599::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`03e29636-013f-4af1-a52f-be320f089f46`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913091::DEBUG::2013-11-20 02:53:51,600::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913091::DEBUG::2013-11-20 02:53:51,601::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913091::DEBUG::2013-11-20 02:53:51,601::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`03e29636-013f-4af1-a52f-be320f089f46`::Granted request
>Thread-15913091::DEBUG::2013-11-20 02:53:51,602::task::495::TaskManager.Task::(_debug) Task d7f221e2-c6d6-4379-a447-acbf2eaf4184: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913091::DEBUG::2013-11-20 02:53:51,603::task::495::TaskManager.Task::(_debug) Task d7f221e2-c6d6-4379-a447-acbf2eaf4184: ref 1 aborting False
>Thread-15913091::WARNING::2013-11-20 02:53:51,604::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913091::DEBUG::2013-11-20 02:53:51,605::task::495::TaskManager.Task::(_debug) Task d7f221e2-c6d6-4379-a447-acbf2eaf4184: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913091::DEBUG::2013-11-20 02:53:51,606::task::495::TaskManager.Task::(_debug) Task d7f221e2-c6d6-4379-a447-acbf2eaf4184: moving from state preparing -> state finished
>Thread-15913091::DEBUG::2013-11-20 02:53:51,607::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913091::DEBUG::2013-11-20 02:53:51,607::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913091::DEBUG::2013-11-20 02:53:51,608::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913091::DEBUG::2013-11-20 02:53:51,608::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913091::DEBUG::2013-11-20 02:53:51,609::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913091::DEBUG::2013-11-20 02:53:51,609::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913091::DEBUG::2013-11-20 02:53:51,610::task::495::TaskManager.Task::(_debug) Task d7f221e2-c6d6-4379-a447-acbf2eaf4184: ref 0 aborting False
>Thread-15913091::INFO::2013-11-20 02:53:51,610::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:53:52,327::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:53:52,857::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0739461 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:53:54,872::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:53:55,396::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0737237 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:53:57,411::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:53:57,945::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0732893 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:53:59,959::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913094::INFO::2013-11-20 02:54:00,183::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913094::DEBUG::2013-11-20 02:54:00,185::task::495::TaskManager.Task::(_debug) Task 9341d916-0b3b-4bb3-8611-7e8062e93b2e: moving from state init -> state preparing
>Thread-15913094::DEBUG::2013-11-20 02:54:00,185::task::495::TaskManager.Task::(_debug) Task 9341d916-0b3b-4bb3-8611-7e8062e93b2e: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000306844711304', 'lastCheck': 1384916039.6294391, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00153589248657', 'lastCheck': 1384916038.101275, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00163912773132', 'lastCheck': 1384916039.1042919, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000545978546143', 'lastCheck': 1384916037.6657751, 'valid': True, 'code': 0}}
>Thread-15913094::DEBUG::2013-11-20 02:54:00,186::task::495::TaskManager.Task::(_debug) Task 9341d916-0b3b-4bb3-8611-7e8062e93b2e: moving from state preparing -> state finished
>Thread-15913094::DEBUG::2013-11-20 02:54:00,187::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913094::DEBUG::2013-11-20 02:54:00,188::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913094::DEBUG::2013-11-20 02:54:00,188::task::495::TaskManager.Task::(_debug) Task 9341d916-0b3b-4bb3-8611-7e8062e93b2e: ref 0 aborting False
>Thread-15913094::INFO::2013-11-20 02:54:00,189::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000306844711304', 'lastCheck': 1384916039.6294391, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00153589248657', 'lastCheck': 1384916038.101275, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00163912773132', 'lastCheck': 1384916039.1042919, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000545978546143', 'lastCheck': 1384916037.6657751, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:54:00,511::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0877834 s, 11.7 MB/s\n'; <rc> = 0
>Thread-15913096::DEBUG::2013-11-20 02:54:01,887::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913096::INFO::2013-11-20 02:54:01,888::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913096::DEBUG::2013-11-20 02:54:01,888::task::495::TaskManager.Task::(_debug) Task 27dc70e6-03bc-4388-90d3-1230ee4d15ca: moving from state init -> state preparing
>Thread-15913096::DEBUG::2013-11-20 02:54:01,889::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913096::DEBUG::2013-11-20 02:54:01,890::task::495::TaskManager.Task::(_debug) Task 27dc70e6-03bc-4388-90d3-1230ee4d15ca: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913096::DEBUG::2013-11-20 02:54:01,890::task::495::TaskManager.Task::(_debug) Task 27dc70e6-03bc-4388-90d3-1230ee4d15ca: moving from state preparing -> state finished
>Thread-15913096::DEBUG::2013-11-20 02:54:01,891::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913096::DEBUG::2013-11-20 02:54:01,892::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913096::DEBUG::2013-11-20 02:54:01,892::task::495::TaskManager.Task::(_debug) Task 27dc70e6-03bc-4388-90d3-1230ee4d15ca: ref 0 aborting False
>Thread-15913096::INFO::2013-11-20 02:54:01,893::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913097::DEBUG::2013-11-20 02:54:01,897::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913097::INFO::2013-11-20 02:54:01,898::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913097::DEBUG::2013-11-20 02:54:01,899::task::495::TaskManager.Task::(_debug) Task d62de80e-50b3-4667-aaff-fd7bd143abfb: moving from state init -> state preparing
>Thread-15913097::DEBUG::2013-11-20 02:54:01,900::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c511a962-5404-4701-81d5-fe8e42974e17`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913097::DEBUG::2013-11-20 02:54:01,900::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913097::DEBUG::2013-11-20 02:54:01,901::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913097::DEBUG::2013-11-20 02:54:01,902::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c511a962-5404-4701-81d5-fe8e42974e17`::Granted request
>Thread-15913097::DEBUG::2013-11-20 02:54:01,903::task::495::TaskManager.Task::(_debug) Task d62de80e-50b3-4667-aaff-fd7bd143abfb: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913097::DEBUG::2013-11-20 02:54:01,903::task::495::TaskManager.Task::(_debug) Task d62de80e-50b3-4667-aaff-fd7bd143abfb: ref 1 aborting False
>Thread-15913097::WARNING::2013-11-20 02:54:01,905::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913097::DEBUG::2013-11-20 02:54:01,906::task::495::TaskManager.Task::(_debug) Task d62de80e-50b3-4667-aaff-fd7bd143abfb: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913097::DEBUG::2013-11-20 02:54:01,907::task::495::TaskManager.Task::(_debug) Task d62de80e-50b3-4667-aaff-fd7bd143abfb: moving from state preparing -> state finished
>Thread-15913097::DEBUG::2013-11-20 02:54:01,908::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913097::DEBUG::2013-11-20 02:54:01,908::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913097::DEBUG::2013-11-20 02:54:01,909::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913097::DEBUG::2013-11-20 02:54:01,909::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913097::DEBUG::2013-11-20 02:54:01,910::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913097::DEBUG::2013-11-20 02:54:01,910::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913097::DEBUG::2013-11-20 02:54:01,911::task::495::TaskManager.Task::(_debug) Task d62de80e-50b3-4667-aaff-fd7bd143abfb: ref 0 aborting False
>Thread-15913097::INFO::2013-11-20 02:54:01,912::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:54:02,527::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:54:03,113::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0727365 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:54:05,131::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:54:05,696::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0705204 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:54:07,712::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:54:08,242::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0730522 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:54:10,257::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 02:54:10,487::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 02:54:10,488::task::495::TaskManager.Task::(_debug) Task 240553fa-4a4e-43ed-a12a-ab6b54aa91e3: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 02:54:10,491::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`08d1d030-304e-4fc3-8690-337e31be9708`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 02:54:10,492::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 02:54:10,493::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 02:54:10,493::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`08d1d030-304e-4fc3-8690-337e31be9708`::Granted request
>Thread-15912392::DEBUG::2013-11-20 02:54:10,494::task::495::TaskManager.Task::(_debug) Task 240553fa-4a4e-43ed-a12a-ab6b54aa91e3: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 02:54:10,495::task::495::TaskManager.Task::(_debug) Task 240553fa-4a4e-43ed-a12a-ab6b54aa91e3: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 02:54:10,496::task::495::TaskManager.Task::(_debug) Task 240553fa-4a4e-43ed-a12a-ab6b54aa91e3: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 02:54:10,497::task::495::TaskManager.Task::(_debug) Task 240553fa-4a4e-43ed-a12a-ab6b54aa91e3: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 02:54:10,497::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 02:54:10,498::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 02:54:10,498::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 02:54:10,499::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 02:54:10,499::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 02:54:10,500::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 02:54:10,501::task::495::TaskManager.Task::(_debug) Task 240553fa-4a4e-43ed-a12a-ab6b54aa91e3: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 02:54:10,501::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 02:54:10,820::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0834513 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:54:12,835::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:54:13,373::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0794512 s, 12.9 MB/s\n'; <rc> = 0
>Thread-15913099::DEBUG::2013-11-20 02:54:13,784::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913099::INFO::2013-11-20 02:54:13,786::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913099::DEBUG::2013-11-20 02:54:13,786::task::495::TaskManager.Task::(_debug) Task 5d993587-2b58-435d-acfe-fa0d1207850e: moving from state init -> state preparing
>Thread-15913099::DEBUG::2013-11-20 02:54:13,787::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913099::DEBUG::2013-11-20 02:54:13,788::task::495::TaskManager.Task::(_debug) Task 5d993587-2b58-435d-acfe-fa0d1207850e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913099::DEBUG::2013-11-20 02:54:13,788::task::495::TaskManager.Task::(_debug) Task 5d993587-2b58-435d-acfe-fa0d1207850e: moving from state preparing -> state finished
>Thread-15913099::DEBUG::2013-11-20 02:54:13,789::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913099::DEBUG::2013-11-20 02:54:13,790::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913099::DEBUG::2013-11-20 02:54:13,790::task::495::TaskManager.Task::(_debug) Task 5d993587-2b58-435d-acfe-fa0d1207850e: ref 0 aborting False
>Thread-15913099::INFO::2013-11-20 02:54:13,791::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913100::DEBUG::2013-11-20 02:54:13,796::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913100::INFO::2013-11-20 02:54:13,797::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913100::DEBUG::2013-11-20 02:54:13,798::task::495::TaskManager.Task::(_debug) Task 639ebe47-bbc9-4f8b-af0e-2cd4eef2a8f9: moving from state init -> state preparing
>Thread-15913100::DEBUG::2013-11-20 02:54:13,799::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`fbc926ac-64a7-4470-9745-358dd35875dc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913100::DEBUG::2013-11-20 02:54:13,799::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913100::DEBUG::2013-11-20 02:54:13,800::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913100::DEBUG::2013-11-20 02:54:13,801::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`fbc926ac-64a7-4470-9745-358dd35875dc`::Granted request
>Thread-15913100::DEBUG::2013-11-20 02:54:13,812::task::495::TaskManager.Task::(_debug) Task 639ebe47-bbc9-4f8b-af0e-2cd4eef2a8f9: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913100::DEBUG::2013-11-20 02:54:13,813::task::495::TaskManager.Task::(_debug) Task 639ebe47-bbc9-4f8b-af0e-2cd4eef2a8f9: ref 1 aborting False
>Thread-15913100::WARNING::2013-11-20 02:54:13,815::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913100::DEBUG::2013-11-20 02:54:13,816::task::495::TaskManager.Task::(_debug) Task 639ebe47-bbc9-4f8b-af0e-2cd4eef2a8f9: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913100::DEBUG::2013-11-20 02:54:13,817::task::495::TaskManager.Task::(_debug) Task 639ebe47-bbc9-4f8b-af0e-2cd4eef2a8f9: moving from state preparing -> state finished
>Thread-15913100::DEBUG::2013-11-20 02:54:13,817::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913100::DEBUG::2013-11-20 02:54:13,818::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913100::DEBUG::2013-11-20 02:54:13,819::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913100::DEBUG::2013-11-20 02:54:13,819::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913100::DEBUG::2013-11-20 02:54:13,820::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913100::DEBUG::2013-11-20 02:54:13,820::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913100::DEBUG::2013-11-20 02:54:13,821::task::495::TaskManager.Task::(_debug) Task 639ebe47-bbc9-4f8b-af0e-2cd4eef2a8f9: ref 0 aborting False
>Thread-15913100::INFO::2013-11-20 02:54:13,822::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:54:15,390::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:54:15,943::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0823547 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:54:17,959::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:54:18,492::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0782887 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:54:20,507::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:54:21,051::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0711056 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:54:23,066::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:54:23,589::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0709498 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:54:25,604::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913104::INFO::2013-11-20 02:54:26,011::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913104::DEBUG::2013-11-20 02:54:26,012::task::495::TaskManager.Task::(_debug) Task 9e9f655b-6852-48f4-b721-ba01dfc48817: moving from state init -> state preparing
>Thread-15913104::DEBUG::2013-11-20 02:54:26,013::task::495::TaskManager.Task::(_debug) Task 9e9f655b-6852-48f4-b721-ba01dfc48817: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00033712387085', 'lastCheck': 1384916060.1721649, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00171709060669', 'lastCheck': 1384916058.616981, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00169396400452', 'lastCheck': 1384916059.6241319, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00197196006775', 'lastCheck': 1384916058.170717, 'valid': True, 'code': 0}}
>Thread-15913104::DEBUG::2013-11-20 02:54:26,014::task::495::TaskManager.Task::(_debug) Task 9e9f655b-6852-48f4-b721-ba01dfc48817: moving from state preparing -> state finished
>Thread-15913104::DEBUG::2013-11-20 02:54:26,015::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913104::DEBUG::2013-11-20 02:54:26,016::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913104::DEBUG::2013-11-20 02:54:26,017::task::495::TaskManager.Task::(_debug) Task 9e9f655b-6852-48f4-b721-ba01dfc48817: ref 0 aborting False
>Thread-15913104::INFO::2013-11-20 02:54:26,018::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00033712387085', 'lastCheck': 1384916060.1721649, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00171709060669', 'lastCheck': 1384916058.616981, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00169396400452', 'lastCheck': 1384916059.6241319, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00197196006775', 'lastCheck': 1384916058.170717, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:54:26,093::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0876849 s, 11.7 MB/s\n'; <rc> = 0
>Thread-15913106::DEBUG::2013-11-20 02:54:27,187::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913106::INFO::2013-11-20 02:54:27,188::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913106::DEBUG::2013-11-20 02:54:27,188::task::495::TaskManager.Task::(_debug) Task 5853a9fa-8b15-42c9-8fda-8586ccc4617f: moving from state init -> state preparing
>Thread-15913106::DEBUG::2013-11-20 02:54:27,189::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913106::DEBUG::2013-11-20 02:54:27,190::task::495::TaskManager.Task::(_debug) Task 5853a9fa-8b15-42c9-8fda-8586ccc4617f: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913106::DEBUG::2013-11-20 02:54:27,190::task::495::TaskManager.Task::(_debug) Task 5853a9fa-8b15-42c9-8fda-8586ccc4617f: moving from state preparing -> state finished
>Thread-15913106::DEBUG::2013-11-20 02:54:27,191::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913106::DEBUG::2013-11-20 02:54:27,191::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913106::DEBUG::2013-11-20 02:54:27,192::task::495::TaskManager.Task::(_debug) Task 5853a9fa-8b15-42c9-8fda-8586ccc4617f: ref 0 aborting False
>Thread-15913106::INFO::2013-11-20 02:54:27,193::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913107::DEBUG::2013-11-20 02:54:27,197::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913107::INFO::2013-11-20 02:54:27,198::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913107::DEBUG::2013-11-20 02:54:27,198::task::495::TaskManager.Task::(_debug) Task acc88e56-fc7c-4e92-800b-f1f191f5bd32: moving from state init -> state preparing
>Thread-15913107::DEBUG::2013-11-20 02:54:27,200::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f19c0447-4105-4a0d-9496-bef3c83e71a0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913107::DEBUG::2013-11-20 02:54:27,200::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913107::DEBUG::2013-11-20 02:54:27,201::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913107::DEBUG::2013-11-20 02:54:27,201::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f19c0447-4105-4a0d-9496-bef3c83e71a0`::Granted request
>Thread-15913107::DEBUG::2013-11-20 02:54:27,202::task::495::TaskManager.Task::(_debug) Task acc88e56-fc7c-4e92-800b-f1f191f5bd32: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913107::DEBUG::2013-11-20 02:54:27,203::task::495::TaskManager.Task::(_debug) Task acc88e56-fc7c-4e92-800b-f1f191f5bd32: ref 1 aborting False
>Thread-15913107::WARNING::2013-11-20 02:54:27,205::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913107::DEBUG::2013-11-20 02:54:27,206::task::495::TaskManager.Task::(_debug) Task acc88e56-fc7c-4e92-800b-f1f191f5bd32: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913107::DEBUG::2013-11-20 02:54:27,207::task::495::TaskManager.Task::(_debug) Task acc88e56-fc7c-4e92-800b-f1f191f5bd32: moving from state preparing -> state finished
>Thread-15913107::DEBUG::2013-11-20 02:54:27,207::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913107::DEBUG::2013-11-20 02:54:27,208::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913107::DEBUG::2013-11-20 02:54:27,208::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913107::DEBUG::2013-11-20 02:54:27,209::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913107::DEBUG::2013-11-20 02:54:27,210::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913107::DEBUG::2013-11-20 02:54:27,210::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913107::DEBUG::2013-11-20 02:54:27,211::task::495::TaskManager.Task::(_debug) Task acc88e56-fc7c-4e92-800b-f1f191f5bd32: ref 0 aborting False
>Thread-15913107::INFO::2013-11-20 02:54:27,211::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:54:28,104::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:54:28,635::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.077718 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:54:30,651::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:54:31,206::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0763985 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:54:33,222::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:54:33,741::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.072845 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:54:35,752::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:54:36,252::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0761895 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:54:38,267::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:54:38,782::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0690676 s, 14.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:54:40,793::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:54:41,274::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0611572 s, 16.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:54:43,288::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:54:43,840::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0743571 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:54:45,855::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:54:46,289::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0459725 s, 22.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:54:48,299::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:54:48,838::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0733099 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:54:50,854::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:54:51,381::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0734805 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:54:53,396::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:54:53,948::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0732869 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:54:55,964::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:54:56,497::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0761405 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:54:58,512::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:54:59,096::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0799854 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:55:01,110::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:55:01,666::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0737786 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15913111::DEBUG::2013-11-20 02:55:01,685::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913111::INFO::2013-11-20 02:55:01,687::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913111::DEBUG::2013-11-20 02:55:01,687::task::495::TaskManager.Task::(_debug) Task c9ae9fb4-fa25-4b16-97ab-f04d81f1b2b8: moving from state init -> state preparing
>Thread-15913111::DEBUG::2013-11-20 02:55:01,689::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913111::DEBUG::2013-11-20 02:55:01,689::task::495::TaskManager.Task::(_debug) Task c9ae9fb4-fa25-4b16-97ab-f04d81f1b2b8: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913111::DEBUG::2013-11-20 02:55:01,690::task::495::TaskManager.Task::(_debug) Task c9ae9fb4-fa25-4b16-97ab-f04d81f1b2b8: moving from state preparing -> state finished
>Thread-15913111::DEBUG::2013-11-20 02:55:01,691::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913111::DEBUG::2013-11-20 02:55:01,692::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913111::DEBUG::2013-11-20 02:55:01,693::task::495::TaskManager.Task::(_debug) Task c9ae9fb4-fa25-4b16-97ab-f04d81f1b2b8: ref 0 aborting False
>Thread-15913111::INFO::2013-11-20 02:55:01,693::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913112::DEBUG::2013-11-20 02:55:01,699::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913112::INFO::2013-11-20 02:55:01,700::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913112::DEBUG::2013-11-20 02:55:01,700::task::495::TaskManager.Task::(_debug) Task a303fabe-cf25-49a2-9100-4da3942a7a5a: moving from state init -> state preparing
>Thread-15913112::DEBUG::2013-11-20 02:55:01,702::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`731ae53a-c03e-4744-aae1-ffe21dac385d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913112::DEBUG::2013-11-20 02:55:01,702::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913112::DEBUG::2013-11-20 02:55:01,703::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913112::DEBUG::2013-11-20 02:55:01,704::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`731ae53a-c03e-4744-aae1-ffe21dac385d`::Granted request
>Thread-15913112::DEBUG::2013-11-20 02:55:01,705::task::495::TaskManager.Task::(_debug) Task a303fabe-cf25-49a2-9100-4da3942a7a5a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913112::DEBUG::2013-11-20 02:55:01,705::task::495::TaskManager.Task::(_debug) Task a303fabe-cf25-49a2-9100-4da3942a7a5a: ref 1 aborting False
>Thread-15913112::WARNING::2013-11-20 02:55:01,707::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913112::DEBUG::2013-11-20 02:55:01,708::task::495::TaskManager.Task::(_debug) Task a303fabe-cf25-49a2-9100-4da3942a7a5a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913112::DEBUG::2013-11-20 02:55:01,709::task::495::TaskManager.Task::(_debug) Task a303fabe-cf25-49a2-9100-4da3942a7a5a: moving from state preparing -> state finished
>Thread-15913112::DEBUG::2013-11-20 02:55:01,709::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913112::DEBUG::2013-11-20 02:55:01,710::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913112::DEBUG::2013-11-20 02:55:01,711::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913112::DEBUG::2013-11-20 02:55:01,711::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913112::DEBUG::2013-11-20 02:55:01,712::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913112::DEBUG::2013-11-20 02:55:01,712::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913112::DEBUG::2013-11-20 02:55:01,713::task::495::TaskManager.Task::(_debug) Task a303fabe-cf25-49a2-9100-4da3942a7a5a: ref 0 aborting False
>Thread-15913112::INFO::2013-11-20 02:55:01,714::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:55:03,686::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:55:04,183::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0660613 s, 15.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:55:06,195::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:55:06,748::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0895006 s, 11.4 MB/s\n'; <rc> = 0
>Thread-15913114::INFO::2013-11-20 02:55:07,058::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913114::DEBUG::2013-11-20 02:55:07,059::task::495::TaskManager.Task::(_debug) Task da6c53fb-2c4a-48aa-b796-2f2964ac600b: moving from state init -> state preparing
>Thread-15913114::DEBUG::2013-11-20 02:55:07,059::task::495::TaskManager.Task::(_debug) Task da6c53fb-2c4a-48aa-b796-2f2964ac600b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000434160232544', 'lastCheck': 1384916100.7624979, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00123691558838', 'lastCheck': 1384916099.1927941, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00143790245056', 'lastCheck': 1384916100.206526, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000623941421509', 'lastCheck': 1384916098.761503, 'valid': True, 'code': 0}}
>Thread-15913114::DEBUG::2013-11-20 02:55:07,060::task::495::TaskManager.Task::(_debug) Task da6c53fb-2c4a-48aa-b796-2f2964ac600b: moving from state preparing -> state finished
>Thread-15913114::DEBUG::2013-11-20 02:55:07,061::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913114::DEBUG::2013-11-20 02:55:07,061::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913114::DEBUG::2013-11-20 02:55:07,062::task::495::TaskManager.Task::(_debug) Task da6c53fb-2c4a-48aa-b796-2f2964ac600b: ref 0 aborting False
>Thread-15913114::INFO::2013-11-20 02:55:07,063::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000434160232544', 'lastCheck': 1384916100.7624979, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00123691558838', 'lastCheck': 1384916099.1927941, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00143790245056', 'lastCheck': 1384916100.206526, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000623941421509', 'lastCheck': 1384916098.761503, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:55:08,764::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:55:09,329::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0829475 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:55:11,344::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 02:55:11,560::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 02:55:11,562::task::495::TaskManager.Task::(_debug) Task 3c1bf5c1-3ca0-44ef-82a9-98fbf68eeca3: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 02:55:11,564::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b0e72355-a6fb-401c-9096-77703b38fc70`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 02:55:11,565::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 02:55:11,566::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 02:55:11,567::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b0e72355-a6fb-401c-9096-77703b38fc70`::Granted request
>Thread-15912392::DEBUG::2013-11-20 02:55:11,568::task::495::TaskManager.Task::(_debug) Task 3c1bf5c1-3ca0-44ef-82a9-98fbf68eeca3: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 02:55:11,568::task::495::TaskManager.Task::(_debug) Task 3c1bf5c1-3ca0-44ef-82a9-98fbf68eeca3: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 02:55:11,569::task::495::TaskManager.Task::(_debug) Task 3c1bf5c1-3ca0-44ef-82a9-98fbf68eeca3: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 02:55:11,570::task::495::TaskManager.Task::(_debug) Task 3c1bf5c1-3ca0-44ef-82a9-98fbf68eeca3: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 02:55:11,570::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 02:55:11,571::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 02:55:11,572::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 02:55:11,572::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 02:55:11,573::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 02:55:11,573::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 02:55:11,574::task::495::TaskManager.Task::(_debug) Task 3c1bf5c1-3ca0-44ef-82a9-98fbf68eeca3: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 02:55:11,574::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 02:55:11,887::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0732148 s, 14.0 MB/s\n'; <rc> = 0
>Thread-15913116::DEBUG::2013-11-20 02:55:13,188::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913116::INFO::2013-11-20 02:55:13,189::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913116::DEBUG::2013-11-20 02:55:13,190::task::495::TaskManager.Task::(_debug) Task 1badf274-ae01-459c-b60c-820364e4197b: moving from state init -> state preparing
>Thread-15913116::DEBUG::2013-11-20 02:55:13,190::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913116::DEBUG::2013-11-20 02:55:13,191::task::495::TaskManager.Task::(_debug) Task 1badf274-ae01-459c-b60c-820364e4197b: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913116::DEBUG::2013-11-20 02:55:13,192::task::495::TaskManager.Task::(_debug) Task 1badf274-ae01-459c-b60c-820364e4197b: moving from state preparing -> state finished
>Thread-15913116::DEBUG::2013-11-20 02:55:13,192::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913116::DEBUG::2013-11-20 02:55:13,193::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913116::DEBUG::2013-11-20 02:55:13,193::task::495::TaskManager.Task::(_debug) Task 1badf274-ae01-459c-b60c-820364e4197b: ref 0 aborting False
>Thread-15913116::INFO::2013-11-20 02:55:13,194::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913117::DEBUG::2013-11-20 02:55:13,198::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913117::INFO::2013-11-20 02:55:13,199::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913117::DEBUG::2013-11-20 02:55:13,200::task::495::TaskManager.Task::(_debug) Task 56bd4268-8bb0-400a-9b17-0d85c09689c0: moving from state init -> state preparing
>Thread-15913117::DEBUG::2013-11-20 02:55:13,201::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`67ebe263-3220-4e46-b871-c0e91ab5ba42`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913117::DEBUG::2013-11-20 02:55:13,202::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913117::DEBUG::2013-11-20 02:55:13,202::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913117::DEBUG::2013-11-20 02:55:13,203::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`67ebe263-3220-4e46-b871-c0e91ab5ba42`::Granted request
>Thread-15913117::DEBUG::2013-11-20 02:55:13,204::task::495::TaskManager.Task::(_debug) Task 56bd4268-8bb0-400a-9b17-0d85c09689c0: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913117::DEBUG::2013-11-20 02:55:13,205::task::495::TaskManager.Task::(_debug) Task 56bd4268-8bb0-400a-9b17-0d85c09689c0: ref 1 aborting False
>Thread-15913117::WARNING::2013-11-20 02:55:13,206::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913117::DEBUG::2013-11-20 02:55:13,207::task::495::TaskManager.Task::(_debug) Task 56bd4268-8bb0-400a-9b17-0d85c09689c0: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913117::DEBUG::2013-11-20 02:55:13,208::task::495::TaskManager.Task::(_debug) Task 56bd4268-8bb0-400a-9b17-0d85c09689c0: moving from state preparing -> state finished
>Thread-15913117::DEBUG::2013-11-20 02:55:13,209::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913117::DEBUG::2013-11-20 02:55:13,209::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913117::DEBUG::2013-11-20 02:55:13,210::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913117::DEBUG::2013-11-20 02:55:13,211::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913117::DEBUG::2013-11-20 02:55:13,211::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913117::DEBUG::2013-11-20 02:55:13,212::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913117::DEBUG::2013-11-20 02:55:13,212::task::495::TaskManager.Task::(_debug) Task 56bd4268-8bb0-400a-9b17-0d85c09689c0: ref 0 aborting False
>Thread-15913117::INFO::2013-11-20 02:55:13,213::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:55:13,902::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:55:14,476::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0765125 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:55:16,488::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:55:17,053::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0739175 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:55:19,069::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:55:19,521::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0491067 s, 20.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:55:21,531::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:55:22,079::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0764697 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:55:24,094::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:55:24,635::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0784428 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:55:26,650::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:55:27,212::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0706435 s, 14.5 MB/s\n'; <rc> = 0
>Thread-15913122::DEBUG::2013-11-20 02:55:27,379::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913122::INFO::2013-11-20 02:55:27,381::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913122::DEBUG::2013-11-20 02:55:27,381::task::495::TaskManager.Task::(_debug) Task cc314da7-d604-4ab5-9036-2bf9ba719642: moving from state init -> state preparing
>Thread-15913122::DEBUG::2013-11-20 02:55:27,382::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913122::DEBUG::2013-11-20 02:55:27,383::task::495::TaskManager.Task::(_debug) Task cc314da7-d604-4ab5-9036-2bf9ba719642: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913122::DEBUG::2013-11-20 02:55:27,383::task::495::TaskManager.Task::(_debug) Task cc314da7-d604-4ab5-9036-2bf9ba719642: moving from state preparing -> state finished
>Thread-15913122::DEBUG::2013-11-20 02:55:27,384::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913122::DEBUG::2013-11-20 02:55:27,384::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913122::DEBUG::2013-11-20 02:55:27,385::task::495::TaskManager.Task::(_debug) Task cc314da7-d604-4ab5-9036-2bf9ba719642: ref 0 aborting False
>Thread-15913122::INFO::2013-11-20 02:55:27,386::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913123::DEBUG::2013-11-20 02:55:27,390::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913123::INFO::2013-11-20 02:55:27,391::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913123::DEBUG::2013-11-20 02:55:27,392::task::495::TaskManager.Task::(_debug) Task 2439dd85-4e99-4dbe-a93e-58283bcd1ba6: moving from state init -> state preparing
>Thread-15913123::DEBUG::2013-11-20 02:55:27,393::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`88b22d1b-08e3-47c6-b68c-79cd75895429`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913123::DEBUG::2013-11-20 02:55:27,394::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913123::DEBUG::2013-11-20 02:55:27,394::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913123::DEBUG::2013-11-20 02:55:27,395::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`88b22d1b-08e3-47c6-b68c-79cd75895429`::Granted request
>Thread-15913123::DEBUG::2013-11-20 02:55:27,396::task::495::TaskManager.Task::(_debug) Task 2439dd85-4e99-4dbe-a93e-58283bcd1ba6: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913123::DEBUG::2013-11-20 02:55:27,397::task::495::TaskManager.Task::(_debug) Task 2439dd85-4e99-4dbe-a93e-58283bcd1ba6: ref 1 aborting False
>Thread-15913123::WARNING::2013-11-20 02:55:27,399::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913123::DEBUG::2013-11-20 02:55:27,399::task::495::TaskManager.Task::(_debug) Task 2439dd85-4e99-4dbe-a93e-58283bcd1ba6: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913123::DEBUG::2013-11-20 02:55:27,400::task::495::TaskManager.Task::(_debug) Task 2439dd85-4e99-4dbe-a93e-58283bcd1ba6: moving from state preparing -> state finished
>Thread-15913123::DEBUG::2013-11-20 02:55:27,401::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913123::DEBUG::2013-11-20 02:55:27,401::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913123::DEBUG::2013-11-20 02:55:27,402::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913123::DEBUG::2013-11-20 02:55:27,403::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913123::DEBUG::2013-11-20 02:55:27,403::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913123::DEBUG::2013-11-20 02:55:27,404::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913123::DEBUG::2013-11-20 02:55:27,404::task::495::TaskManager.Task::(_debug) Task 2439dd85-4e99-4dbe-a93e-58283bcd1ba6: ref 0 aborting False
>Thread-15913123::INFO::2013-11-20 02:55:27,405::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913124::INFO::2013-11-20 02:55:28,927::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913124::DEBUG::2013-11-20 02:55:28,928::task::495::TaskManager.Task::(_debug) Task aeed87e5-de9c-4ae5-b5db-93636a525071: moving from state init -> state preparing
>Thread-15913124::DEBUG::2013-11-20 02:55:28,929::task::495::TaskManager.Task::(_debug) Task aeed87e5-de9c-4ae5-b5db-93636a525071: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000278949737549', 'lastCheck': 1384916121.289798, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00231599807739', 'lastCheck': 1384916119.743093, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00173902511597', 'lastCheck': 1384916120.7429709, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00714993476868', 'lastCheck': 1384916119.289396, 'valid': True, 'code': 0}}
>Thread-15913124::DEBUG::2013-11-20 02:55:28,930::task::495::TaskManager.Task::(_debug) Task aeed87e5-de9c-4ae5-b5db-93636a525071: moving from state preparing -> state finished
>Thread-15913124::DEBUG::2013-11-20 02:55:28,930::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913124::DEBUG::2013-11-20 02:55:28,931::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913124::DEBUG::2013-11-20 02:55:28,931::task::495::TaskManager.Task::(_debug) Task aeed87e5-de9c-4ae5-b5db-93636a525071: ref 0 aborting False
>Thread-15913124::INFO::2013-11-20 02:55:28,932::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000278949737549', 'lastCheck': 1384916121.289798, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00231599807739', 'lastCheck': 1384916119.743093, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00173902511597', 'lastCheck': 1384916120.7429709, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00714993476868', 'lastCheck': 1384916119.289396, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:55:29,227::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:55:29,774::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0950964 s, 10.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:55:31,788::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:55:32,340::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0893957 s, 11.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:55:34,359::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:55:34,880::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.074563 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:55:36,895::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:55:37,391::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0727639 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:55:39,405::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:55:39,994::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.075617 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:55:42,008::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:55:42,566::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.072423 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:55:44,581::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:55:45,135::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0731716 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:55:47,150::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:55:47,659::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0592019 s, 17.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:55:49,668::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:55:50,224::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0751314 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:55:52,236::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:55:52,771::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0732972 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:55:54,786::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913129::DEBUG::2013-11-20 02:55:55,007::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913129::INFO::2013-11-20 02:55:55,010::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913129::DEBUG::2013-11-20 02:55:55,011::task::495::TaskManager.Task::(_debug) Task 4656ca2f-c939-43d4-b919-1f7453e869aa: moving from state init -> state preparing
>Thread-15913129::DEBUG::2013-11-20 02:55:55,013::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913129::DEBUG::2013-11-20 02:55:55,014::task::495::TaskManager.Task::(_debug) Task 4656ca2f-c939-43d4-b919-1f7453e869aa: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913129::DEBUG::2013-11-20 02:55:55,015::task::495::TaskManager.Task::(_debug) Task 4656ca2f-c939-43d4-b919-1f7453e869aa: moving from state preparing -> state finished
>Thread-15913129::DEBUG::2013-11-20 02:55:55,015::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913129::DEBUG::2013-11-20 02:55:55,016::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913129::DEBUG::2013-11-20 02:55:55,017::task::495::TaskManager.Task::(_debug) Task 4656ca2f-c939-43d4-b919-1f7453e869aa: ref 0 aborting False
>Thread-15913129::INFO::2013-11-20 02:55:55,018::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913130::DEBUG::2013-11-20 02:55:55,024::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913130::INFO::2013-11-20 02:55:55,025::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913130::DEBUG::2013-11-20 02:55:55,026::task::495::TaskManager.Task::(_debug) Task 0e2b15bf-dcf1-40ca-9ea9-1e02515a2c8c: moving from state init -> state preparing
>Thread-15913130::DEBUG::2013-11-20 02:55:55,027::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2200cb36-8adf-410b-8ed6-93e5d8dee1d2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913130::DEBUG::2013-11-20 02:55:55,028::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913130::DEBUG::2013-11-20 02:55:55,029::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913130::DEBUG::2013-11-20 02:55:55,029::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2200cb36-8adf-410b-8ed6-93e5d8dee1d2`::Granted request
>Thread-15913130::DEBUG::2013-11-20 02:55:55,031::task::495::TaskManager.Task::(_debug) Task 0e2b15bf-dcf1-40ca-9ea9-1e02515a2c8c: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913130::DEBUG::2013-11-20 02:55:55,031::task::495::TaskManager.Task::(_debug) Task 0e2b15bf-dcf1-40ca-9ea9-1e02515a2c8c: ref 1 aborting False
>Thread-15913130::WARNING::2013-11-20 02:55:55,034::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913130::DEBUG::2013-11-20 02:55:55,035::task::495::TaskManager.Task::(_debug) Task 0e2b15bf-dcf1-40ca-9ea9-1e02515a2c8c: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913130::DEBUG::2013-11-20 02:55:55,036::task::495::TaskManager.Task::(_debug) Task 0e2b15bf-dcf1-40ca-9ea9-1e02515a2c8c: moving from state preparing -> state finished
>Thread-15913130::DEBUG::2013-11-20 02:55:55,036::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913130::DEBUG::2013-11-20 02:55:55,037::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913130::DEBUG::2013-11-20 02:55:55,037::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913130::DEBUG::2013-11-20 02:55:55,038::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913130::DEBUG::2013-11-20 02:55:55,039::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913130::DEBUG::2013-11-20 02:55:55,039::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913130::DEBUG::2013-11-20 02:55:55,040::task::495::TaskManager.Task::(_debug) Task 0e2b15bf-dcf1-40ca-9ea9-1e02515a2c8c: ref 0 aborting False
>Thread-15913130::INFO::2013-11-20 02:55:55,041::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:55:55,318::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0736744 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:55:57,333::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:55:57,863::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0855231 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:55:59,879::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913132::INFO::2013-11-20 02:56:00,120::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913132::DEBUG::2013-11-20 02:56:00,121::task::495::TaskManager.Task::(_debug) Task 45b581e5-c96a-43ff-80ba-d318bf37eba0: moving from state init -> state preparing
>Thread-15913132::DEBUG::2013-11-20 02:56:00,122::task::495::TaskManager.Task::(_debug) Task 45b581e5-c96a-43ff-80ba-d318bf37eba0: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000478982925415', 'lastCheck': 1384916151.9082601, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00574803352356', 'lastCheck': 1384916150.4076769, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00153398513794', 'lastCheck': 1384916151.391309, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00165390968323', 'lastCheck': 1384916160.1167221, 'valid': True, 'code': 0}}
>Thread-15913132::DEBUG::2013-11-20 02:56:00,123::task::495::TaskManager.Task::(_debug) Task 45b581e5-c96a-43ff-80ba-d318bf37eba0: moving from state preparing -> state finished
>Thread-15913132::DEBUG::2013-11-20 02:56:00,123::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913132::DEBUG::2013-11-20 02:56:00,123::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913132::DEBUG::2013-11-20 02:56:00,124::task::495::TaskManager.Task::(_debug) Task 45b581e5-c96a-43ff-80ba-d318bf37eba0: ref 0 aborting False
>Thread-15913132::INFO::2013-11-20 02:56:00,124::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000478982925415', 'lastCheck': 1384916151.9082601, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00574803352356', 'lastCheck': 1384916150.4076769, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00153398513794', 'lastCheck': 1384916151.391309, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00165390968323', 'lastCheck': 1384916160.1167221, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:56:00,442::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0856653 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:56:02,457::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:56:03,008::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0871502 s, 11.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:56:05,024::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:56:05,564::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0737444 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:56:07,579::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:56:08,104::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0731949 s, 14.0 MB/s\n'; <rc> = 0
>Thread-15913135::DEBUG::2013-11-20 02:56:08,762::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913135::INFO::2013-11-20 02:56:08,763::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913135::DEBUG::2013-11-20 02:56:08,764::task::495::TaskManager.Task::(_debug) Task 29b0cbb3-4e6b-427c-bd19-1575a289e77e: moving from state init -> state preparing
>Thread-15913135::DEBUG::2013-11-20 02:56:08,765::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913135::DEBUG::2013-11-20 02:56:08,765::task::495::TaskManager.Task::(_debug) Task 29b0cbb3-4e6b-427c-bd19-1575a289e77e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913135::DEBUG::2013-11-20 02:56:08,766::task::495::TaskManager.Task::(_debug) Task 29b0cbb3-4e6b-427c-bd19-1575a289e77e: moving from state preparing -> state finished
>Thread-15913135::DEBUG::2013-11-20 02:56:08,766::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913135::DEBUG::2013-11-20 02:56:08,767::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913135::DEBUG::2013-11-20 02:56:08,767::task::495::TaskManager.Task::(_debug) Task 29b0cbb3-4e6b-427c-bd19-1575a289e77e: ref 0 aborting False
>Thread-15913135::INFO::2013-11-20 02:56:08,768::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913136::DEBUG::2013-11-20 02:56:08,772::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913136::INFO::2013-11-20 02:56:08,773::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913136::DEBUG::2013-11-20 02:56:08,774::task::495::TaskManager.Task::(_debug) Task 11b8bc7b-87dd-4dee-83e1-12bee367bccf: moving from state init -> state preparing
>Thread-15913136::DEBUG::2013-11-20 02:56:08,775::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`affcb56f-f055-4099-8c9e-5071ffcb5a72`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913136::DEBUG::2013-11-20 02:56:08,776::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913136::DEBUG::2013-11-20 02:56:08,777::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913136::DEBUG::2013-11-20 02:56:08,777::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`affcb56f-f055-4099-8c9e-5071ffcb5a72`::Granted request
>Thread-15913136::DEBUG::2013-11-20 02:56:08,778::task::495::TaskManager.Task::(_debug) Task 11b8bc7b-87dd-4dee-83e1-12bee367bccf: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913136::DEBUG::2013-11-20 02:56:08,779::task::495::TaskManager.Task::(_debug) Task 11b8bc7b-87dd-4dee-83e1-12bee367bccf: ref 1 aborting False
>Thread-15913136::WARNING::2013-11-20 02:56:08,781::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913136::DEBUG::2013-11-20 02:56:08,782::task::495::TaskManager.Task::(_debug) Task 11b8bc7b-87dd-4dee-83e1-12bee367bccf: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913136::DEBUG::2013-11-20 02:56:08,782::task::495::TaskManager.Task::(_debug) Task 11b8bc7b-87dd-4dee-83e1-12bee367bccf: moving from state preparing -> state finished
>Thread-15913136::DEBUG::2013-11-20 02:56:08,783::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913136::DEBUG::2013-11-20 02:56:08,783::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913136::DEBUG::2013-11-20 02:56:08,784::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913136::DEBUG::2013-11-20 02:56:08,785::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913136::DEBUG::2013-11-20 02:56:08,785::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913136::DEBUG::2013-11-20 02:56:08,786::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913136::DEBUG::2013-11-20 02:56:08,786::task::495::TaskManager.Task::(_debug) Task 11b8bc7b-87dd-4dee-83e1-12bee367bccf: ref 0 aborting False
>Thread-15913136::INFO::2013-11-20 02:56:08,787::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:56:10,120::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:56:10,640::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0733682 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:56:12,655::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 02:56:12,883::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 02:56:12,894::task::495::TaskManager.Task::(_debug) Task 4c04dd5d-1a13-44d7-8e1f-26ddf73fab43: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 02:56:12,897::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4b1e662b-bc9e-4349-b0b4-48742726bf11`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 02:56:12,897::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 02:56:12,898::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 02:56:12,899::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4b1e662b-bc9e-4349-b0b4-48742726bf11`::Granted request
>Thread-15912392::DEBUG::2013-11-20 02:56:12,900::task::495::TaskManager.Task::(_debug) Task 4c04dd5d-1a13-44d7-8e1f-26ddf73fab43: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 02:56:12,900::task::495::TaskManager.Task::(_debug) Task 4c04dd5d-1a13-44d7-8e1f-26ddf73fab43: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 02:56:12,901::task::495::TaskManager.Task::(_debug) Task 4c04dd5d-1a13-44d7-8e1f-26ddf73fab43: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 02:56:12,902::task::495::TaskManager.Task::(_debug) Task 4c04dd5d-1a13-44d7-8e1f-26ddf73fab43: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 02:56:12,902::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 02:56:12,903::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 02:56:12,904::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 02:56:12,904::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 02:56:12,905::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 02:56:12,905::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 02:56:12,906::task::495::TaskManager.Task::(_debug) Task 4c04dd5d-1a13-44d7-8e1f-26ddf73fab43: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 02:56:12,906::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 02:56:13,188::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0735713 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:56:15,203::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:56:15,721::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0756686 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:56:17,734::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:56:18,253::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0751019 s, 13.6 MB/s\n'; <rc> = 0
>Thread-15913140::INFO::2013-11-20 02:56:19,985::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913140::DEBUG::2013-11-20 02:56:19,986::task::495::TaskManager.Task::(_debug) Task d9a411e1-d5d4-40aa-b228-363688588418: moving from state init -> state preparing
>Thread-15913140::DEBUG::2013-11-20 02:56:19,987::task::495::TaskManager.Task::(_debug) Task d9a411e1-d5d4-40aa-b228-363688588418: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.0002760887146', 'lastCheck': 1384916172.341984, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00152707099915', 'lastCheck': 1384916170.7997651, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0015709400177', 'lastCheck': 1384916171.7980311, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000684976577759', 'lastCheck': 1384916170.3384769, 'valid': True, 'code': 0}}
>Thread-15913140::DEBUG::2013-11-20 02:56:19,988::task::495::TaskManager.Task::(_debug) Task d9a411e1-d5d4-40aa-b228-363688588418: moving from state preparing -> state finished
>Thread-15913140::DEBUG::2013-11-20 02:56:19,988::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913140::DEBUG::2013-11-20 02:56:19,989::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913140::DEBUG::2013-11-20 02:56:19,989::task::495::TaskManager.Task::(_debug) Task d9a411e1-d5d4-40aa-b228-363688588418: ref 0 aborting False
>Thread-15913140::INFO::2013-11-20 02:56:19,990::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.0002760887146', 'lastCheck': 1384916172.341984, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00152707099915', 'lastCheck': 1384916170.7997651, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0015709400177', 'lastCheck': 1384916171.7980311, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000684976577759', 'lastCheck': 1384916170.3384769, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:56:20,269::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:56:20,825::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.077143 s, 13.3 MB/s\n'; <rc> = 0
>Thread-42838::DEBUG::2013-11-20 02:56:22,514::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:56:22,841::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-42838::DEBUG::2013-11-20 02:56:23,269::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:56:23,304::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0458807 s, 22.3 MB/s\n'; <rc> = 0
>Thread-15913142::DEBUG::2013-11-20 02:56:23,356::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913142::INFO::2013-11-20 02:56:23,358::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913142::DEBUG::2013-11-20 02:56:23,358::task::495::TaskManager.Task::(_debug) Task 7a4e2153-29d1-4960-90c6-63aaafaeefc1: moving from state init -> state preparing
>Thread-15913142::DEBUG::2013-11-20 02:56:23,359::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913142::DEBUG::2013-11-20 02:56:23,360::task::495::TaskManager.Task::(_debug) Task 7a4e2153-29d1-4960-90c6-63aaafaeefc1: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913142::DEBUG::2013-11-20 02:56:23,360::task::495::TaskManager.Task::(_debug) Task 7a4e2153-29d1-4960-90c6-63aaafaeefc1: moving from state preparing -> state finished
>Thread-15913142::DEBUG::2013-11-20 02:56:23,361::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913142::DEBUG::2013-11-20 02:56:23,361::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913142::DEBUG::2013-11-20 02:56:23,362::task::495::TaskManager.Task::(_debug) Task 7a4e2153-29d1-4960-90c6-63aaafaeefc1: ref 0 aborting False
>Thread-15913142::INFO::2013-11-20 02:56:23,362::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913143::DEBUG::2013-11-20 02:56:23,367::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913143::INFO::2013-11-20 02:56:23,368::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913143::DEBUG::2013-11-20 02:56:23,369::task::495::TaskManager.Task::(_debug) Task 1cacb3ed-56d3-4588-af5e-5884a07d4808: moving from state init -> state preparing
>Thread-15913143::DEBUG::2013-11-20 02:56:23,370::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c934213b-b82d-425f-9417-17364385f610`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913143::DEBUG::2013-11-20 02:56:23,371::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913143::DEBUG::2013-11-20 02:56:23,371::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913143::DEBUG::2013-11-20 02:56:23,372::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c934213b-b82d-425f-9417-17364385f610`::Granted request
>Thread-15913143::DEBUG::2013-11-20 02:56:23,373::task::495::TaskManager.Task::(_debug) Task 1cacb3ed-56d3-4588-af5e-5884a07d4808: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913143::DEBUG::2013-11-20 02:56:23,373::task::495::TaskManager.Task::(_debug) Task 1cacb3ed-56d3-4588-af5e-5884a07d4808: ref 1 aborting False
>Thread-15913143::WARNING::2013-11-20 02:56:23,375::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913143::DEBUG::2013-11-20 02:56:23,376::task::495::TaskManager.Task::(_debug) Task 1cacb3ed-56d3-4588-af5e-5884a07d4808: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913143::DEBUG::2013-11-20 02:56:23,377::task::495::TaskManager.Task::(_debug) Task 1cacb3ed-56d3-4588-af5e-5884a07d4808: moving from state preparing -> state finished
>Thread-15913143::DEBUG::2013-11-20 02:56:23,378::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913143::DEBUG::2013-11-20 02:56:23,378::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913143::DEBUG::2013-11-20 02:56:23,379::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913143::DEBUG::2013-11-20 02:56:23,380::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913143::DEBUG::2013-11-20 02:56:23,380::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913143::DEBUG::2013-11-20 02:56:23,380::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913143::DEBUG::2013-11-20 02:56:23,381::task::495::TaskManager.Task::(_debug) Task 1cacb3ed-56d3-4588-af5e-5884a07d4808: ref 0 aborting False
>Thread-15913143::INFO::2013-11-20 02:56:23,382::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:56:25,314::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:56:25,842::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0731701 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:56:27,856::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:56:28,376::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0754717 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:56:30,389::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:56:30,901::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0752473 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:56:32,919::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:56:33,476::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0743747 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:56:35,490::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:56:36,048::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0768599 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:56:38,063::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:56:38,498::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0468766 s, 21.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:56:40,508::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:56:41,085::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0737383 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:56:43,101::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:56:43,626::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0737032 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15913147::DEBUG::2013-11-20 02:56:44,618::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913147::INFO::2013-11-20 02:56:44,619::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913147::DEBUG::2013-11-20 02:56:44,620::task::495::TaskManager.Task::(_debug) Task 4ae5b15b-dfe6-4d5d-98ed-fd9f9d75b8a3: moving from state init -> state preparing
>Thread-15913147::DEBUG::2013-11-20 02:56:44,621::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913147::DEBUG::2013-11-20 02:56:44,621::task::495::TaskManager.Task::(_debug) Task 4ae5b15b-dfe6-4d5d-98ed-fd9f9d75b8a3: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913147::DEBUG::2013-11-20 02:56:44,622::task::495::TaskManager.Task::(_debug) Task 4ae5b15b-dfe6-4d5d-98ed-fd9f9d75b8a3: moving from state preparing -> state finished
>Thread-15913147::DEBUG::2013-11-20 02:56:44,622::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913147::DEBUG::2013-11-20 02:56:44,623::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913147::DEBUG::2013-11-20 02:56:44,624::task::495::TaskManager.Task::(_debug) Task 4ae5b15b-dfe6-4d5d-98ed-fd9f9d75b8a3: ref 0 aborting False
>Thread-15913147::INFO::2013-11-20 02:56:44,624::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913148::DEBUG::2013-11-20 02:56:44,629::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913148::INFO::2013-11-20 02:56:44,630::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913148::DEBUG::2013-11-20 02:56:44,630::task::495::TaskManager.Task::(_debug) Task 2c1364b8-2949-49cc-b0ba-84c8b5e2099a: moving from state init -> state preparing
>Thread-15913148::DEBUG::2013-11-20 02:56:44,631::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1bef695b-0bc1-4cbb-a361-98a67bf11028`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913148::DEBUG::2013-11-20 02:56:44,632::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913148::DEBUG::2013-11-20 02:56:44,633::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913148::DEBUG::2013-11-20 02:56:44,633::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1bef695b-0bc1-4cbb-a361-98a67bf11028`::Granted request
>Thread-15913148::DEBUG::2013-11-20 02:56:44,634::task::495::TaskManager.Task::(_debug) Task 2c1364b8-2949-49cc-b0ba-84c8b5e2099a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913148::DEBUG::2013-11-20 02:56:44,635::task::495::TaskManager.Task::(_debug) Task 2c1364b8-2949-49cc-b0ba-84c8b5e2099a: ref 1 aborting False
>Thread-15913148::WARNING::2013-11-20 02:56:44,637::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913148::DEBUG::2013-11-20 02:56:44,639::task::495::TaskManager.Task::(_debug) Task 2c1364b8-2949-49cc-b0ba-84c8b5e2099a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913148::DEBUG::2013-11-20 02:56:44,640::task::495::TaskManager.Task::(_debug) Task 2c1364b8-2949-49cc-b0ba-84c8b5e2099a: moving from state preparing -> state finished
>Thread-15913148::DEBUG::2013-11-20 02:56:44,641::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913148::DEBUG::2013-11-20 02:56:44,642::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913148::DEBUG::2013-11-20 02:56:44,643::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913148::DEBUG::2013-11-20 02:56:44,644::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913148::DEBUG::2013-11-20 02:56:44,644::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913148::DEBUG::2013-11-20 02:56:44,645::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913148::DEBUG::2013-11-20 02:56:44,646::task::495::TaskManager.Task::(_debug) Task 2c1364b8-2949-49cc-b0ba-84c8b5e2099a: ref 0 aborting False
>Thread-15913148::INFO::2013-11-20 02:56:44,647::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:56:45,642::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:56:46,199::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0777266 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:56:48,215::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:56:48,738::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0703409 s, 14.6 MB/s\n'; <rc> = 0
>Thread-15913150::INFO::2013-11-20 02:56:50,368::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913150::DEBUG::2013-11-20 02:56:50,369::task::495::TaskManager.Task::(_debug) Task 2b0b9c95-cb2b-425e-a74c-c5c84bac52bf: moving from state init -> state preparing
>Thread-15913150::DEBUG::2013-11-20 02:56:50,370::task::495::TaskManager.Task::(_debug) Task 2b0b9c95-cb2b-425e-a74c-c5c84bac52bf: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00043511390686', 'lastCheck': 1384916203.3164229, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00145983695984', 'lastCheck': 1384916201.2962019, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00132203102112', 'lastCheck': 1384916202.2966571, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00254797935486', 'lastCheck': 1384916200.7797329, 'valid': True, 'code': 0}}
>Thread-15913150::DEBUG::2013-11-20 02:56:50,371::task::495::TaskManager.Task::(_debug) Task 2b0b9c95-cb2b-425e-a74c-c5c84bac52bf: moving from state preparing -> state finished
>Thread-15913150::DEBUG::2013-11-20 02:56:50,371::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913150::DEBUG::2013-11-20 02:56:50,372::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913150::DEBUG::2013-11-20 02:56:50,372::task::495::TaskManager.Task::(_debug) Task 2b0b9c95-cb2b-425e-a74c-c5c84bac52bf: ref 0 aborting False
>Thread-15913150::INFO::2013-11-20 02:56:50,373::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00043511390686', 'lastCheck': 1384916203.3164229, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00145983695984', 'lastCheck': 1384916201.2962019, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00132203102112', 'lastCheck': 1384916202.2966571, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00254797935486', 'lastCheck': 1384916200.7797329, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:56:50,753::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:56:51,321::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0817211 s, 12.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:56:53,337::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:56:53,875::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0942589 s, 10.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:56:55,891::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:56:56,440::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0734101 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15913153::DEBUG::2013-11-20 02:56:58,318::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913153::INFO::2013-11-20 02:56:58,320::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913153::DEBUG::2013-11-20 02:56:58,321::task::495::TaskManager.Task::(_debug) Task 2c9b5420-09bc-46d5-b07c-d81237414ab3: moving from state init -> state preparing
>Thread-15913153::DEBUG::2013-11-20 02:56:58,321::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913153::DEBUG::2013-11-20 02:56:58,322::task::495::TaskManager.Task::(_debug) Task 2c9b5420-09bc-46d5-b07c-d81237414ab3: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913153::DEBUG::2013-11-20 02:56:58,323::task::495::TaskManager.Task::(_debug) Task 2c9b5420-09bc-46d5-b07c-d81237414ab3: moving from state preparing -> state finished
>Thread-15913153::DEBUG::2013-11-20 02:56:58,323::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913153::DEBUG::2013-11-20 02:56:58,324::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913153::DEBUG::2013-11-20 02:56:58,324::task::495::TaskManager.Task::(_debug) Task 2c9b5420-09bc-46d5-b07c-d81237414ab3: ref 0 aborting False
>Thread-15913153::INFO::2013-11-20 02:56:58,325::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913154::DEBUG::2013-11-20 02:56:58,329::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913154::INFO::2013-11-20 02:56:58,330::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913154::DEBUG::2013-11-20 02:56:58,331::task::495::TaskManager.Task::(_debug) Task 9ae24c19-ebb9-407e-a0a5-d49835e436fb: moving from state init -> state preparing
>Thread-15913154::DEBUG::2013-11-20 02:56:58,332::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f7739e9c-34ab-4a1b-9ceb-741422d3ac85`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913154::DEBUG::2013-11-20 02:56:58,333::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913154::DEBUG::2013-11-20 02:56:58,333::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913154::DEBUG::2013-11-20 02:56:58,334::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f7739e9c-34ab-4a1b-9ceb-741422d3ac85`::Granted request
>Thread-15913154::DEBUG::2013-11-20 02:56:58,335::task::495::TaskManager.Task::(_debug) Task 9ae24c19-ebb9-407e-a0a5-d49835e436fb: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913154::DEBUG::2013-11-20 02:56:58,335::task::495::TaskManager.Task::(_debug) Task 9ae24c19-ebb9-407e-a0a5-d49835e436fb: ref 1 aborting False
>Thread-15913154::WARNING::2013-11-20 02:56:58,337::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913154::DEBUG::2013-11-20 02:56:58,338::task::495::TaskManager.Task::(_debug) Task 9ae24c19-ebb9-407e-a0a5-d49835e436fb: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913154::DEBUG::2013-11-20 02:56:58,339::task::495::TaskManager.Task::(_debug) Task 9ae24c19-ebb9-407e-a0a5-d49835e436fb: moving from state preparing -> state finished
>Thread-15913154::DEBUG::2013-11-20 02:56:58,340::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913154::DEBUG::2013-11-20 02:56:58,340::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913154::DEBUG::2013-11-20 02:56:58,341::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913154::DEBUG::2013-11-20 02:56:58,342::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913154::DEBUG::2013-11-20 02:56:58,342::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913154::DEBUG::2013-11-20 02:56:58,343::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913154::DEBUG::2013-11-20 02:56:58,343::task::495::TaskManager.Task::(_debug) Task 9ae24c19-ebb9-407e-a0a5-d49835e436fb: ref 0 aborting False
>Thread-15913154::INFO::2013-11-20 02:56:58,344::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:56:58,455::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:56:59,000::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0737532 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:57:01,015::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:57:01,574::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0715127 s, 14.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:57:03,590::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:57:04,118::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0726809 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:57:06,133::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:57:06,681::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0713049 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:57:08,696::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:57:09,229::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0729577 s, 14.0 MB/s\n'; <rc> = 0
>Thread-15913158::INFO::2013-11-20 02:57:09,864::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913158::DEBUG::2013-11-20 02:57:09,865::task::495::TaskManager.Task::(_debug) Task 37b73f5b-a1b5-44c6-bf41-fcf2e1ffa561: moving from state init -> state preparing
>Thread-15913158::DEBUG::2013-11-20 02:57:09,866::task::495::TaskManager.Task::(_debug) Task 37b73f5b-a1b5-44c6-bf41-fcf2e1ffa561: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000982999801636', 'lastCheck': 1384916223.8282421, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00153708457947', 'lastCheck': 1384916221.6845579, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00139093399048', 'lastCheck': 1384916222.685411, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00415992736816', 'lastCheck': 1384916221.2657909, 'valid': True, 'code': 0}}
>Thread-15913158::DEBUG::2013-11-20 02:57:09,866::task::495::TaskManager.Task::(_debug) Task 37b73f5b-a1b5-44c6-bf41-fcf2e1ffa561: moving from state preparing -> state finished
>Thread-15913158::DEBUG::2013-11-20 02:57:09,867::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913158::DEBUG::2013-11-20 02:57:09,868::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913158::DEBUG::2013-11-20 02:57:09,868::task::495::TaskManager.Task::(_debug) Task 37b73f5b-a1b5-44c6-bf41-fcf2e1ffa561: ref 0 aborting False
>Thread-15913158::INFO::2013-11-20 02:57:09,869::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000982999801636', 'lastCheck': 1384916223.8282421, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00153708457947', 'lastCheck': 1384916221.6845579, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00139093399048', 'lastCheck': 1384916222.685411, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00415992736816', 'lastCheck': 1384916221.2657909, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:57:11,243::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:57:11,806::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0805771 s, 12.7 MB/s\n'; <rc> = 0
>Thread-15913160::DEBUG::2013-11-20 02:57:11,863::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913160::INFO::2013-11-20 02:57:11,864::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913160::DEBUG::2013-11-20 02:57:11,865::task::495::TaskManager.Task::(_debug) Task 86129046-31dd-4dc3-9040-17105fc5a09c: moving from state init -> state preparing
>Thread-15913160::DEBUG::2013-11-20 02:57:11,866::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913160::DEBUG::2013-11-20 02:57:11,866::task::495::TaskManager.Task::(_debug) Task 86129046-31dd-4dc3-9040-17105fc5a09c: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913160::DEBUG::2013-11-20 02:57:11,867::task::495::TaskManager.Task::(_debug) Task 86129046-31dd-4dc3-9040-17105fc5a09c: moving from state preparing -> state finished
>Thread-15913160::DEBUG::2013-11-20 02:57:11,867::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913160::DEBUG::2013-11-20 02:57:11,868::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913160::DEBUG::2013-11-20 02:57:11,868::task::495::TaskManager.Task::(_debug) Task 86129046-31dd-4dc3-9040-17105fc5a09c: ref 0 aborting False
>Thread-15913160::INFO::2013-11-20 02:57:11,869::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913161::DEBUG::2013-11-20 02:57:11,874::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913161::INFO::2013-11-20 02:57:11,875::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913161::DEBUG::2013-11-20 02:57:11,875::task::495::TaskManager.Task::(_debug) Task 2676d28b-33ce-41ab-bf42-50f1c9ed7549: moving from state init -> state preparing
>Thread-15913161::DEBUG::2013-11-20 02:57:11,877::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ce68fa29-a188-4175-9344-be9f50a692f4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913161::DEBUG::2013-11-20 02:57:11,877::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913161::DEBUG::2013-11-20 02:57:11,878::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913161::DEBUG::2013-11-20 02:57:11,879::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ce68fa29-a188-4175-9344-be9f50a692f4`::Granted request
>Thread-15913161::DEBUG::2013-11-20 02:57:11,880::task::495::TaskManager.Task::(_debug) Task 2676d28b-33ce-41ab-bf42-50f1c9ed7549: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913161::DEBUG::2013-11-20 02:57:11,880::task::495::TaskManager.Task::(_debug) Task 2676d28b-33ce-41ab-bf42-50f1c9ed7549: ref 1 aborting False
>Thread-15913161::WARNING::2013-11-20 02:57:11,882::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913161::DEBUG::2013-11-20 02:57:11,883::task::495::TaskManager.Task::(_debug) Task 2676d28b-33ce-41ab-bf42-50f1c9ed7549: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913161::DEBUG::2013-11-20 02:57:11,884::task::495::TaskManager.Task::(_debug) Task 2676d28b-33ce-41ab-bf42-50f1c9ed7549: moving from state preparing -> state finished
>Thread-15913161::DEBUG::2013-11-20 02:57:11,884::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913161::DEBUG::2013-11-20 02:57:11,885::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913161::DEBUG::2013-11-20 02:57:11,886::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913161::DEBUG::2013-11-20 02:57:11,886::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913161::DEBUG::2013-11-20 02:57:11,887::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913161::DEBUG::2013-11-20 02:57:11,887::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913161::DEBUG::2013-11-20 02:57:11,888::task::495::TaskManager.Task::(_debug) Task 2676d28b-33ce-41ab-bf42-50f1c9ed7549: ref 0 aborting False
>Thread-15913161::INFO::2013-11-20 02:57:11,889::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:57:13,822::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 02:57:13,998::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 02:57:14,001::task::495::TaskManager.Task::(_debug) Task 72d7bdb5-d00b-42fa-9cb8-56470da65d41: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 02:57:14,006::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`088dc98c-5fea-4593-80c8-0df5e8c52fff`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 02:57:14,007::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 02:57:14,009::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 02:57:14,010::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`088dc98c-5fea-4593-80c8-0df5e8c52fff`::Granted request
>Thread-15912392::DEBUG::2013-11-20 02:57:14,011::task::495::TaskManager.Task::(_debug) Task 72d7bdb5-d00b-42fa-9cb8-56470da65d41: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 02:57:14,011::task::495::TaskManager.Task::(_debug) Task 72d7bdb5-d00b-42fa-9cb8-56470da65d41: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 02:57:14,012::task::495::TaskManager.Task::(_debug) Task 72d7bdb5-d00b-42fa-9cb8-56470da65d41: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 02:57:14,013::task::495::TaskManager.Task::(_debug) Task 72d7bdb5-d00b-42fa-9cb8-56470da65d41: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 02:57:14,013::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 02:57:14,014::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 02:57:14,015::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 02:57:14,015::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 02:57:14,016::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 02:57:14,016::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 02:57:14,017::task::495::TaskManager.Task::(_debug) Task 72d7bdb5-d00b-42fa-9cb8-56470da65d41: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 02:57:14,017::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 02:57:14,323::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0732776 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:57:16,338::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:57:16,895::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0868101 s, 11.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:57:18,911::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:57:19,463::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0740697 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:57:21,478::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:57:22,007::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0732648 s, 14.0 MB/s\n'; <rc> = 0
>Thread-15913166::DEBUG::2013-11-20 02:57:23,755::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913166::INFO::2013-11-20 02:57:23,756::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913166::DEBUG::2013-11-20 02:57:23,757::task::495::TaskManager.Task::(_debug) Task 2c63fa7e-28d4-40f7-93fe-7fec7e832cf3: moving from state init -> state preparing
>Thread-15913166::DEBUG::2013-11-20 02:57:23,757::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913166::DEBUG::2013-11-20 02:57:23,758::task::495::TaskManager.Task::(_debug) Task 2c63fa7e-28d4-40f7-93fe-7fec7e832cf3: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913166::DEBUG::2013-11-20 02:57:23,759::task::495::TaskManager.Task::(_debug) Task 2c63fa7e-28d4-40f7-93fe-7fec7e832cf3: moving from state preparing -> state finished
>Thread-15913166::DEBUG::2013-11-20 02:57:23,759::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913166::DEBUG::2013-11-20 02:57:23,760::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913166::DEBUG::2013-11-20 02:57:23,760::task::495::TaskManager.Task::(_debug) Task 2c63fa7e-28d4-40f7-93fe-7fec7e832cf3: ref 0 aborting False
>Thread-15913166::INFO::2013-11-20 02:57:23,761::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913167::DEBUG::2013-11-20 02:57:23,766::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913167::INFO::2013-11-20 02:57:23,767::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913167::DEBUG::2013-11-20 02:57:23,767::task::495::TaskManager.Task::(_debug) Task 851da2e2-9180-4328-929b-f28236923ac3: moving from state init -> state preparing
>Thread-15913167::DEBUG::2013-11-20 02:57:23,768::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7c36a5e3-8ba5-4b49-9147-4689b5b7cd36`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913167::DEBUG::2013-11-20 02:57:23,769::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913167::DEBUG::2013-11-20 02:57:23,770::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913167::DEBUG::2013-11-20 02:57:23,770::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7c36a5e3-8ba5-4b49-9147-4689b5b7cd36`::Granted request
>Thread-15913167::DEBUG::2013-11-20 02:57:23,771::task::495::TaskManager.Task::(_debug) Task 851da2e2-9180-4328-929b-f28236923ac3: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913167::DEBUG::2013-11-20 02:57:23,772::task::495::TaskManager.Task::(_debug) Task 851da2e2-9180-4328-929b-f28236923ac3: ref 1 aborting False
>Thread-15913167::WARNING::2013-11-20 02:57:23,774::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913167::DEBUG::2013-11-20 02:57:23,775::task::495::TaskManager.Task::(_debug) Task 851da2e2-9180-4328-929b-f28236923ac3: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913167::DEBUG::2013-11-20 02:57:23,776::task::495::TaskManager.Task::(_debug) Task 851da2e2-9180-4328-929b-f28236923ac3: moving from state preparing -> state finished
>Thread-15913167::DEBUG::2013-11-20 02:57:23,776::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913167::DEBUG::2013-11-20 02:57:23,777::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913167::DEBUG::2013-11-20 02:57:23,777::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913167::DEBUG::2013-11-20 02:57:23,778::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913167::DEBUG::2013-11-20 02:57:23,778::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913167::DEBUG::2013-11-20 02:57:23,779::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913167::DEBUG::2013-11-20 02:57:23,780::task::495::TaskManager.Task::(_debug) Task 851da2e2-9180-4328-929b-f28236923ac3: ref 0 aborting False
>Thread-15913167::INFO::2013-11-20 02:57:23,780::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:57:24,023::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:57:24,564::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0736123 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15913168::INFO::2013-11-20 02:57:25,616::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913168::DEBUG::2013-11-20 02:57:25,616::task::495::TaskManager.Task::(_debug) Task eb117fc1-7161-486b-9ee1-de7b7f522018: moving from state init -> state preparing
>Thread-15913168::DEBUG::2013-11-20 02:57:25,617::task::495::TaskManager.Task::(_debug) Task eb117fc1-7161-486b-9ee1-de7b7f522018: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00039005279541', 'lastCheck': 1384916244.250705, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00199794769287', 'lastCheck': 1384916242.147239, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0015971660614', 'lastCheck': 1384916243.1467209, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000602960586548', 'lastCheck': 1384916241.6837521, 'valid': True, 'code': 0}}
>Thread-15913168::DEBUG::2013-11-20 02:57:25,618::task::495::TaskManager.Task::(_debug) Task eb117fc1-7161-486b-9ee1-de7b7f522018: moving from state preparing -> state finished
>Thread-15913168::DEBUG::2013-11-20 02:57:25,619::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913168::DEBUG::2013-11-20 02:57:25,619::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913168::DEBUG::2013-11-20 02:57:25,620::task::495::TaskManager.Task::(_debug) Task eb117fc1-7161-486b-9ee1-de7b7f522018: ref 0 aborting False
>Thread-15913168::INFO::2013-11-20 02:57:25,620::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00039005279541', 'lastCheck': 1384916244.250705, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00199794769287', 'lastCheck': 1384916242.147239, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0015971660614', 'lastCheck': 1384916243.1467209, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000602960586548', 'lastCheck': 1384916241.6837521, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:57:26,579::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:57:27,151::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.102043 s, 10.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:57:29,167::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:57:29,694::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0734433 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:57:31,710::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:57:32,165::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0551042 s, 18.6 MB/s\n'; <rc> = 0
>Thread-15913172::DEBUG::2013-11-20 02:57:33,976::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913172::INFO::2013-11-20 02:57:33,978::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913172::DEBUG::2013-11-20 02:57:33,978::task::495::TaskManager.Task::(_debug) Task a3e496d9-d61f-4c65-bedf-8374edb5eae6: moving from state init -> state preparing
>Thread-15913172::DEBUG::2013-11-20 02:57:33,979::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913172::DEBUG::2013-11-20 02:57:33,979::task::495::TaskManager.Task::(_debug) Task a3e496d9-d61f-4c65-bedf-8374edb5eae6: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913172::DEBUG::2013-11-20 02:57:33,979::task::495::TaskManager.Task::(_debug) Task a3e496d9-d61f-4c65-bedf-8374edb5eae6: moving from state preparing -> state finished
>Thread-15913172::DEBUG::2013-11-20 02:57:33,980::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913172::DEBUG::2013-11-20 02:57:33,980::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913172::DEBUG::2013-11-20 02:57:33,980::task::495::TaskManager.Task::(_debug) Task a3e496d9-d61f-4c65-bedf-8374edb5eae6: ref 0 aborting False
>Thread-15913172::INFO::2013-11-20 02:57:33,981::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913173::DEBUG::2013-11-20 02:57:33,984::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913173::INFO::2013-11-20 02:57:33,985::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913173::DEBUG::2013-11-20 02:57:33,985::task::495::TaskManager.Task::(_debug) Task 4e49640b-9dc4-4809-ac2c-f24decd5b1f3: moving from state init -> state preparing
>Thread-15913173::DEBUG::2013-11-20 02:57:33,986::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`44c3e1d8-da1f-4ded-997b-00f49bdabd25`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913173::DEBUG::2013-11-20 02:57:33,986::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913173::DEBUG::2013-11-20 02:57:33,987::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913173::DEBUG::2013-11-20 02:57:33,987::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`44c3e1d8-da1f-4ded-997b-00f49bdabd25`::Granted request
>Thread-15913173::DEBUG::2013-11-20 02:57:33,988::task::495::TaskManager.Task::(_debug) Task 4e49640b-9dc4-4809-ac2c-f24decd5b1f3: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913173::DEBUG::2013-11-20 02:57:33,988::task::495::TaskManager.Task::(_debug) Task 4e49640b-9dc4-4809-ac2c-f24decd5b1f3: ref 1 aborting False
>Thread-15913173::WARNING::2013-11-20 02:57:33,989::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913173::DEBUG::2013-11-20 02:57:33,990::task::495::TaskManager.Task::(_debug) Task 4e49640b-9dc4-4809-ac2c-f24decd5b1f3: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913173::DEBUG::2013-11-20 02:57:33,990::task::495::TaskManager.Task::(_debug) Task 4e49640b-9dc4-4809-ac2c-f24decd5b1f3: moving from state preparing -> state finished
>Thread-15913173::DEBUG::2013-11-20 02:57:33,991::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913173::DEBUG::2013-11-20 02:57:33,991::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913173::DEBUG::2013-11-20 02:57:33,992::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913173::DEBUG::2013-11-20 02:57:33,992::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913173::DEBUG::2013-11-20 02:57:33,992::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913173::DEBUG::2013-11-20 02:57:33,992::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913173::DEBUG::2013-11-20 02:57:33,993::task::495::TaskManager.Task::(_debug) Task 4e49640b-9dc4-4809-ac2c-f24decd5b1f3: ref 0 aborting False
>Thread-15913173::INFO::2013-11-20 02:57:33,993::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:57:34,177::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:57:34,710::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0747212 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:57:36,727::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:57:37,256::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0734809 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:57:39,271::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:57:39,826::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0706066 s, 14.5 MB/s\n'; <rc> = 0
>Thread-15913176::INFO::2013-11-20 02:57:41,103::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913176::DEBUG::2013-11-20 02:57:41,104::task::495::TaskManager.Task::(_debug) Task 008ff3e4-d7f5-4b3a-a079-45169d22ca7d: moving from state init -> state preparing
>Thread-15913176::DEBUG::2013-11-20 02:57:41,105::task::495::TaskManager.Task::(_debug) Task 008ff3e4-d7f5-4b3a-a079-45169d22ca7d: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000314950942993', 'lastCheck': 1384916254.409488, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00159692764282', 'lastCheck': 1384916252.3902881, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00330018997192', 'lastCheck': 1384916253.393724, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00049901008606', 'lastCheck': 1384916251.9233911, 'valid': True, 'code': 0}}
>Thread-15913176::DEBUG::2013-11-20 02:57:41,106::task::495::TaskManager.Task::(_debug) Task 008ff3e4-d7f5-4b3a-a079-45169d22ca7d: moving from state preparing -> state finished
>Thread-15913176::DEBUG::2013-11-20 02:57:41,106::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913176::DEBUG::2013-11-20 02:57:41,107::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913176::DEBUG::2013-11-20 02:57:41,107::task::495::TaskManager.Task::(_debug) Task 008ff3e4-d7f5-4b3a-a079-45169d22ca7d: ref 0 aborting False
>Thread-15913176::INFO::2013-11-20 02:57:41,108::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000314950942993', 'lastCheck': 1384916254.409488, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00159692764282', 'lastCheck': 1384916252.3902881, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00330018997192', 'lastCheck': 1384916253.393724, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00049901008606', 'lastCheck': 1384916251.9233911, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:57:41,842::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:57:42,333::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0707183 s, 14.5 MB/s\n'; <rc> = 0
>Thread-15913178::DEBUG::2013-11-20 02:57:44,109::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913178::INFO::2013-11-20 02:57:44,110::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913178::DEBUG::2013-11-20 02:57:44,111::task::495::TaskManager.Task::(_debug) Task a433fefe-c014-432f-ac2b-516c20498255: moving from state init -> state preparing
>Thread-15913178::DEBUG::2013-11-20 02:57:44,112::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913178::DEBUG::2013-11-20 02:57:44,112::task::495::TaskManager.Task::(_debug) Task a433fefe-c014-432f-ac2b-516c20498255: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913178::DEBUG::2013-11-20 02:57:44,113::task::495::TaskManager.Task::(_debug) Task a433fefe-c014-432f-ac2b-516c20498255: moving from state preparing -> state finished
>Thread-15913178::DEBUG::2013-11-20 02:57:44,113::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913178::DEBUG::2013-11-20 02:57:44,114::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913178::DEBUG::2013-11-20 02:57:44,115::task::495::TaskManager.Task::(_debug) Task a433fefe-c014-432f-ac2b-516c20498255: ref 0 aborting False
>Thread-15913178::INFO::2013-11-20 02:57:44,115::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913179::DEBUG::2013-11-20 02:57:44,120::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913179::INFO::2013-11-20 02:57:44,121::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913179::DEBUG::2013-11-20 02:57:44,121::task::495::TaskManager.Task::(_debug) Task 6b4fd3a4-925b-4c3c-ae64-5becdaf04aa0: moving from state init -> state preparing
>Thread-15913179::DEBUG::2013-11-20 02:57:44,123::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7c986911-dad0-4529-b9e8-d544afd74732`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913179::DEBUG::2013-11-20 02:57:44,123::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913179::DEBUG::2013-11-20 02:57:44,124::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913179::DEBUG::2013-11-20 02:57:44,125::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7c986911-dad0-4529-b9e8-d544afd74732`::Granted request
>Thread-15913179::DEBUG::2013-11-20 02:57:44,126::task::495::TaskManager.Task::(_debug) Task 6b4fd3a4-925b-4c3c-ae64-5becdaf04aa0: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913179::DEBUG::2013-11-20 02:57:44,126::task::495::TaskManager.Task::(_debug) Task 6b4fd3a4-925b-4c3c-ae64-5becdaf04aa0: ref 1 aborting False
>Thread-15913179::WARNING::2013-11-20 02:57:44,128::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913179::DEBUG::2013-11-20 02:57:44,129::task::495::TaskManager.Task::(_debug) Task 6b4fd3a4-925b-4c3c-ae64-5becdaf04aa0: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913179::DEBUG::2013-11-20 02:57:44,130::task::495::TaskManager.Task::(_debug) Task 6b4fd3a4-925b-4c3c-ae64-5becdaf04aa0: moving from state preparing -> state finished
>Thread-15913179::DEBUG::2013-11-20 02:57:44,130::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913179::DEBUG::2013-11-20 02:57:44,131::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913179::DEBUG::2013-11-20 02:57:44,132::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913179::DEBUG::2013-11-20 02:57:44,132::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913179::DEBUG::2013-11-20 02:57:44,133::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913179::DEBUG::2013-11-20 02:57:44,133::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913179::DEBUG::2013-11-20 02:57:44,134::task::495::TaskManager.Task::(_debug) Task 6b4fd3a4-925b-4c3c-ae64-5becdaf04aa0: ref 0 aborting False
>Thread-15913179::INFO::2013-11-20 02:57:44,135::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:57:44,348::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:57:44,882::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0726808 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:57:46,898::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:57:47,441::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0737351 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:57:49,457::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:57:49,978::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0724681 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:57:51,993::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:57:52,531::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0815752 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15913184::DEBUG::2013-11-20 02:57:54,282::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913184::INFO::2013-11-20 02:57:54,284::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913184::DEBUG::2013-11-20 02:57:54,284::task::495::TaskManager.Task::(_debug) Task e7d6a26d-9b41-4178-86f0-edaca0c77189: moving from state init -> state preparing
>Thread-15913184::DEBUG::2013-11-20 02:57:54,285::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913184::DEBUG::2013-11-20 02:57:54,286::task::495::TaskManager.Task::(_debug) Task e7d6a26d-9b41-4178-86f0-edaca0c77189: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913184::DEBUG::2013-11-20 02:57:54,286::task::495::TaskManager.Task::(_debug) Task e7d6a26d-9b41-4178-86f0-edaca0c77189: moving from state preparing -> state finished
>Thread-15913184::DEBUG::2013-11-20 02:57:54,286::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913184::DEBUG::2013-11-20 02:57:54,287::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913184::DEBUG::2013-11-20 02:57:54,287::task::495::TaskManager.Task::(_debug) Task e7d6a26d-9b41-4178-86f0-edaca0c77189: ref 0 aborting False
>Thread-15913184::INFO::2013-11-20 02:57:54,287::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913185::DEBUG::2013-11-20 02:57:54,291::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913185::INFO::2013-11-20 02:57:54,292::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913185::DEBUG::2013-11-20 02:57:54,292::task::495::TaskManager.Task::(_debug) Task 74852903-51bc-462c-b7cb-e5caf4eaeb5c: moving from state init -> state preparing
>Thread-15913185::DEBUG::2013-11-20 02:57:54,293::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`66f6711e-9011-4ff0-b344-6edafa1f5b6a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913185::DEBUG::2013-11-20 02:57:54,293::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913185::DEBUG::2013-11-20 02:57:54,293::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913185::DEBUG::2013-11-20 02:57:54,294::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`66f6711e-9011-4ff0-b344-6edafa1f5b6a`::Granted request
>Thread-15913185::DEBUG::2013-11-20 02:57:54,299::task::495::TaskManager.Task::(_debug) Task 74852903-51bc-462c-b7cb-e5caf4eaeb5c: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913185::DEBUG::2013-11-20 02:57:54,299::task::495::TaskManager.Task::(_debug) Task 74852903-51bc-462c-b7cb-e5caf4eaeb5c: ref 1 aborting False
>Thread-15913185::WARNING::2013-11-20 02:57:54,300::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913185::DEBUG::2013-11-20 02:57:54,301::task::495::TaskManager.Task::(_debug) Task 74852903-51bc-462c-b7cb-e5caf4eaeb5c: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913185::DEBUG::2013-11-20 02:57:54,301::task::495::TaskManager.Task::(_debug) Task 74852903-51bc-462c-b7cb-e5caf4eaeb5c: moving from state preparing -> state finished
>Thread-15913185::DEBUG::2013-11-20 02:57:54,301::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913185::DEBUG::2013-11-20 02:57:54,302::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913185::DEBUG::2013-11-20 02:57:54,302::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913185::DEBUG::2013-11-20 02:57:54,303::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913185::DEBUG::2013-11-20 02:57:54,303::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913185::DEBUG::2013-11-20 02:57:54,303::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913185::DEBUG::2013-11-20 02:57:54,304::task::495::TaskManager.Task::(_debug) Task 74852903-51bc-462c-b7cb-e5caf4eaeb5c: ref 0 aborting False
>Thread-15913185::INFO::2013-11-20 02:57:54,304::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:57:54,547::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:57:55,086::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0751298 s, 13.6 MB/s\n'; <rc> = 0
>Thread-15913186::INFO::2013-11-20 02:57:57,007::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913186::DEBUG::2013-11-20 02:57:57,008::task::495::TaskManager.Task::(_debug) Task c51d0568-1dfd-4762-84f0-2243b4084675: moving from state init -> state preparing
>Thread-15913186::DEBUG::2013-11-20 02:57:57,009::task::495::TaskManager.Task::(_debug) Task c51d0568-1dfd-4762-84f0-2243b4084675: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000292778015137', 'lastCheck': 1384916274.7786219, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014009475708', 'lastCheck': 1384916272.670645, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00188088417053', 'lastCheck': 1384916273.672101, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000557899475098', 'lastCheck': 1384916272.216763, 'valid': True, 'code': 0}}
>Thread-15913186::DEBUG::2013-11-20 02:57:57,010::task::495::TaskManager.Task::(_debug) Task c51d0568-1dfd-4762-84f0-2243b4084675: moving from state preparing -> state finished
>Thread-15913186::DEBUG::2013-11-20 02:57:57,010::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913186::DEBUG::2013-11-20 02:57:57,011::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913186::DEBUG::2013-11-20 02:57:57,011::task::495::TaskManager.Task::(_debug) Task c51d0568-1dfd-4762-84f0-2243b4084675: ref 0 aborting False
>Thread-15913186::INFO::2013-11-20 02:57:57,012::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000292778015137', 'lastCheck': 1384916274.7786219, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014009475708', 'lastCheck': 1384916272.670645, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00188088417053', 'lastCheck': 1384916273.672101, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000557899475098', 'lastCheck': 1384916272.216763, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:57:57,102::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:57:57,638::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0972072 s, 10.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:57:59,654::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:58:00,216::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0887499 s, 11.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:58:02,231::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:58:02,730::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0545971 s, 18.8 MB/s\n'; <rc> = 0
>Thread-15913190::DEBUG::2013-11-20 02:58:04,448::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913190::INFO::2013-11-20 02:58:04,450::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913190::DEBUG::2013-11-20 02:58:04,450::task::495::TaskManager.Task::(_debug) Task 4237b9fa-025d-430d-8149-018b3b39f555: moving from state init -> state preparing
>Thread-15913190::DEBUG::2013-11-20 02:58:04,451::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913190::DEBUG::2013-11-20 02:58:04,452::task::495::TaskManager.Task::(_debug) Task 4237b9fa-025d-430d-8149-018b3b39f555: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913190::DEBUG::2013-11-20 02:58:04,452::task::495::TaskManager.Task::(_debug) Task 4237b9fa-025d-430d-8149-018b3b39f555: moving from state preparing -> state finished
>Thread-15913190::DEBUG::2013-11-20 02:58:04,453::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913190::DEBUG::2013-11-20 02:58:04,453::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913190::DEBUG::2013-11-20 02:58:04,454::task::495::TaskManager.Task::(_debug) Task 4237b9fa-025d-430d-8149-018b3b39f555: ref 0 aborting False
>Thread-15913190::INFO::2013-11-20 02:58:04,455::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913191::DEBUG::2013-11-20 02:58:04,459::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913191::INFO::2013-11-20 02:58:04,460::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913191::DEBUG::2013-11-20 02:58:04,461::task::495::TaskManager.Task::(_debug) Task 1ae799d6-976f-4dbf-bb21-9e26799c843c: moving from state init -> state preparing
>Thread-15913191::DEBUG::2013-11-20 02:58:04,462::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`774237ee-fe85-4db5-bf90-f938e0a87f67`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913191::DEBUG::2013-11-20 02:58:04,463::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913191::DEBUG::2013-11-20 02:58:04,464::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913191::DEBUG::2013-11-20 02:58:04,465::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`774237ee-fe85-4db5-bf90-f938e0a87f67`::Granted request
>Thread-15913191::DEBUG::2013-11-20 02:58:04,466::task::495::TaskManager.Task::(_debug) Task 1ae799d6-976f-4dbf-bb21-9e26799c843c: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913191::DEBUG::2013-11-20 02:58:04,466::task::495::TaskManager.Task::(_debug) Task 1ae799d6-976f-4dbf-bb21-9e26799c843c: ref 1 aborting False
>Thread-15913191::WARNING::2013-11-20 02:58:04,468::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913191::DEBUG::2013-11-20 02:58:04,469::task::495::TaskManager.Task::(_debug) Task 1ae799d6-976f-4dbf-bb21-9e26799c843c: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913191::DEBUG::2013-11-20 02:58:04,470::task::495::TaskManager.Task::(_debug) Task 1ae799d6-976f-4dbf-bb21-9e26799c843c: moving from state preparing -> state finished
>Thread-15913191::DEBUG::2013-11-20 02:58:04,471::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913191::DEBUG::2013-11-20 02:58:04,471::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913191::DEBUG::2013-11-20 02:58:04,472::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913191::DEBUG::2013-11-20 02:58:04,473::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913191::DEBUG::2013-11-20 02:58:04,473::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913191::DEBUG::2013-11-20 02:58:04,474::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913191::DEBUG::2013-11-20 02:58:04,475::task::495::TaskManager.Task::(_debug) Task 1ae799d6-976f-4dbf-bb21-9e26799c843c: ref 0 aborting False
>Thread-15913191::INFO::2013-11-20 02:58:04,475::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:58:04,745::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:58:05,235::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0580124 s, 17.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:58:07,245::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:58:07,767::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0729963 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:58:09,783::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:58:10,356::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0767875 s, 13.3 MB/s\n'; <rc> = 0
>Thread-15913194::INFO::2013-11-20 02:58:12,260::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913194::DEBUG::2013-11-20 02:58:12,261::task::495::TaskManager.Task::(_debug) Task dce65abc-491d-439e-b924-c9e108f17823: moving from state init -> state preparing
>Thread-15913194::DEBUG::2013-11-20 02:58:12,262::task::495::TaskManager.Task::(_debug) Task dce65abc-491d-439e-b924-c9e108f17823: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00039005279541', 'lastCheck': 1384916284.990808, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00161695480347', 'lastCheck': 1384916282.8854771, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00132703781128', 'lastCheck': 1384916283.8859019, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000602960586548', 'lastCheck': 1384916282.4275939, 'valid': True, 'code': 0}}
>Thread-15913194::DEBUG::2013-11-20 02:58:12,263::task::495::TaskManager.Task::(_debug) Task dce65abc-491d-439e-b924-c9e108f17823: moving from state preparing -> state finished
>Thread-15913194::DEBUG::2013-11-20 02:58:12,264::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913194::DEBUG::2013-11-20 02:58:12,264::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913194::DEBUG::2013-11-20 02:58:12,265::task::495::TaskManager.Task::(_debug) Task dce65abc-491d-439e-b924-c9e108f17823: ref 0 aborting False
>Thread-15913194::INFO::2013-11-20 02:58:12,265::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00039005279541', 'lastCheck': 1384916284.990808, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00161695480347', 'lastCheck': 1384916282.8854771, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00132703781128', 'lastCheck': 1384916283.8859019, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000602960586548', 'lastCheck': 1384916282.4275939, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:58:12,370::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:58:12,987::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.116145 s, 8.8 MB/s\n'; <rc> = 0
>Thread-15913196::DEBUG::2013-11-20 02:58:14,606::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913196::INFO::2013-11-20 02:58:14,608::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913196::DEBUG::2013-11-20 02:58:14,608::task::495::TaskManager.Task::(_debug) Task ba113087-968a-499c-9c23-4396b804c164: moving from state init -> state preparing
>Thread-15913196::DEBUG::2013-11-20 02:58:14,609::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913196::DEBUG::2013-11-20 02:58:14,610::task::495::TaskManager.Task::(_debug) Task ba113087-968a-499c-9c23-4396b804c164: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913196::DEBUG::2013-11-20 02:58:14,610::task::495::TaskManager.Task::(_debug) Task ba113087-968a-499c-9c23-4396b804c164: moving from state preparing -> state finished
>Thread-15913196::DEBUG::2013-11-20 02:58:14,611::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913196::DEBUG::2013-11-20 02:58:14,611::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913196::DEBUG::2013-11-20 02:58:14,612::task::495::TaskManager.Task::(_debug) Task ba113087-968a-499c-9c23-4396b804c164: ref 0 aborting False
>Thread-15913196::INFO::2013-11-20 02:58:14,612::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913197::DEBUG::2013-11-20 02:58:14,617::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913197::INFO::2013-11-20 02:58:14,618::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913197::DEBUG::2013-11-20 02:58:14,618::task::495::TaskManager.Task::(_debug) Task 6a46442a-0c39-45b9-bfdc-a3fcf105dac8: moving from state init -> state preparing
>Thread-15913197::DEBUG::2013-11-20 02:58:14,619::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2e6f4010-d964-4e45-adf2-e5e4abc0d89e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913197::DEBUG::2013-11-20 02:58:14,620::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913197::DEBUG::2013-11-20 02:58:14,621::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913197::DEBUG::2013-11-20 02:58:14,621::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2e6f4010-d964-4e45-adf2-e5e4abc0d89e`::Granted request
>Thread-15913197::DEBUG::2013-11-20 02:58:14,622::task::495::TaskManager.Task::(_debug) Task 6a46442a-0c39-45b9-bfdc-a3fcf105dac8: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913197::DEBUG::2013-11-20 02:58:14,623::task::495::TaskManager.Task::(_debug) Task 6a46442a-0c39-45b9-bfdc-a3fcf105dac8: ref 1 aborting False
>Thread-15913197::WARNING::2013-11-20 02:58:14,625::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913197::DEBUG::2013-11-20 02:58:14,626::task::495::TaskManager.Task::(_debug) Task 6a46442a-0c39-45b9-bfdc-a3fcf105dac8: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913197::DEBUG::2013-11-20 02:58:14,626::task::495::TaskManager.Task::(_debug) Task 6a46442a-0c39-45b9-bfdc-a3fcf105dac8: moving from state preparing -> state finished
>Thread-15913197::DEBUG::2013-11-20 02:58:14,627::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913197::DEBUG::2013-11-20 02:58:14,628::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913197::DEBUG::2013-11-20 02:58:14,628::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913197::DEBUG::2013-11-20 02:58:14,629::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913197::DEBUG::2013-11-20 02:58:14,629::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913197::DEBUG::2013-11-20 02:58:14,630::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913197::DEBUG::2013-11-20 02:58:14,630::task::495::TaskManager.Task::(_debug) Task 6a46442a-0c39-45b9-bfdc-a3fcf105dac8: ref 0 aborting False
>Thread-15913197::INFO::2013-11-20 02:58:14,631::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:58:14,997::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 02:58:15,265::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 02:58:15,271::task::495::TaskManager.Task::(_debug) Task 1af01982-80b7-4513-8655-7f049a6c2e38: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 02:58:15,273::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4714b54d-b82e-4c01-9fa7-a61773f90d54`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 02:58:15,274::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 02:58:15,274::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 02:58:15,275::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4714b54d-b82e-4c01-9fa7-a61773f90d54`::Granted request
>Thread-15912392::DEBUG::2013-11-20 02:58:15,275::task::495::TaskManager.Task::(_debug) Task 1af01982-80b7-4513-8655-7f049a6c2e38: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 02:58:15,276::task::495::TaskManager.Task::(_debug) Task 1af01982-80b7-4513-8655-7f049a6c2e38: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 02:58:15,276::task::495::TaskManager.Task::(_debug) Task 1af01982-80b7-4513-8655-7f049a6c2e38: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 02:58:15,277::task::495::TaskManager.Task::(_debug) Task 1af01982-80b7-4513-8655-7f049a6c2e38: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 02:58:15,277::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 02:58:15,277::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 02:58:15,278::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 02:58:15,278::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 02:58:15,278::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 02:58:15,279::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 02:58:15,279::task::495::TaskManager.Task::(_debug) Task 1af01982-80b7-4513-8655-7f049a6c2e38: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 02:58:15,279::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 02:58:15,510::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0761971 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:58:17,534::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:58:17,997::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0455901 s, 22.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:58:20,008::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:58:20,546::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0759617 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:58:22,563::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:58:23,091::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0766346 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15913202::DEBUG::2013-11-20 02:58:24,763::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913202::INFO::2013-11-20 02:58:24,764::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913202::DEBUG::2013-11-20 02:58:24,764::task::495::TaskManager.Task::(_debug) Task c46e6aa9-10f1-4e76-881b-7629934eb270: moving from state init -> state preparing
>Thread-15913202::DEBUG::2013-11-20 02:58:24,765::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913202::DEBUG::2013-11-20 02:58:24,766::task::495::TaskManager.Task::(_debug) Task c46e6aa9-10f1-4e76-881b-7629934eb270: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913202::DEBUG::2013-11-20 02:58:24,766::task::495::TaskManager.Task::(_debug) Task c46e6aa9-10f1-4e76-881b-7629934eb270: moving from state preparing -> state finished
>Thread-15913202::DEBUG::2013-11-20 02:58:24,767::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913202::DEBUG::2013-11-20 02:58:24,768::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913202::DEBUG::2013-11-20 02:58:24,768::task::495::TaskManager.Task::(_debug) Task c46e6aa9-10f1-4e76-881b-7629934eb270: ref 0 aborting False
>Thread-15913202::INFO::2013-11-20 02:58:24,769::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913203::DEBUG::2013-11-20 02:58:24,773::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913203::INFO::2013-11-20 02:58:24,774::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913203::DEBUG::2013-11-20 02:58:24,775::task::495::TaskManager.Task::(_debug) Task f9693cf3-570a-4511-9fef-aa4708fc3fd0: moving from state init -> state preparing
>Thread-15913203::DEBUG::2013-11-20 02:58:24,776::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2092780a-5b76-4e85-9146-e12df08eaa35`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913203::DEBUG::2013-11-20 02:58:24,777::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913203::DEBUG::2013-11-20 02:58:24,778::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913203::DEBUG::2013-11-20 02:58:24,778::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2092780a-5b76-4e85-9146-e12df08eaa35`::Granted request
>Thread-15913203::DEBUG::2013-11-20 02:58:24,779::task::495::TaskManager.Task::(_debug) Task f9693cf3-570a-4511-9fef-aa4708fc3fd0: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913203::DEBUG::2013-11-20 02:58:24,780::task::495::TaskManager.Task::(_debug) Task f9693cf3-570a-4511-9fef-aa4708fc3fd0: ref 1 aborting False
>Thread-15913203::WARNING::2013-11-20 02:58:24,782::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913203::DEBUG::2013-11-20 02:58:24,783::task::495::TaskManager.Task::(_debug) Task f9693cf3-570a-4511-9fef-aa4708fc3fd0: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913203::DEBUG::2013-11-20 02:58:24,784::task::495::TaskManager.Task::(_debug) Task f9693cf3-570a-4511-9fef-aa4708fc3fd0: moving from state preparing -> state finished
>Thread-15913203::DEBUG::2013-11-20 02:58:24,784::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913203::DEBUG::2013-11-20 02:58:24,785::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913203::DEBUG::2013-11-20 02:58:24,786::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913203::DEBUG::2013-11-20 02:58:24,786::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913203::DEBUG::2013-11-20 02:58:24,787::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913203::DEBUG::2013-11-20 02:58:24,787::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913203::DEBUG::2013-11-20 02:58:24,788::task::495::TaskManager.Task::(_debug) Task f9693cf3-570a-4511-9fef-aa4708fc3fd0: ref 0 aborting False
>Thread-15913203::INFO::2013-11-20 02:58:24,789::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:58:25,106::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:58:25,693::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.103655 s, 9.9 MB/s\n'; <rc> = 0
>Thread-15913204::INFO::2013-11-20 02:58:27,513::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913204::DEBUG::2013-11-20 02:58:27,514::task::495::TaskManager.Task::(_debug) Task f18d68fa-7604-4e00-90d7-1dba6f6c07e1: moving from state init -> state preparing
>Thread-15913204::DEBUG::2013-11-20 02:58:27,515::task::495::TaskManager.Task::(_debug) Task f18d68fa-7604-4e00-90d7-1dba6f6c07e1: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000434875488281', 'lastCheck': 1384916305.3080609, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00228691101074', 'lastCheck': 1384916303.2722261, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00173807144165', 'lastCheck': 1384916304.2608531, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000536918640137', 'lastCheck': 1384916302.7748311, 'valid': True, 'code': 0}}
>Thread-15913204::DEBUG::2013-11-20 02:58:27,515::task::495::TaskManager.Task::(_debug) Task f18d68fa-7604-4e00-90d7-1dba6f6c07e1: moving from state preparing -> state finished
>Thread-15913204::DEBUG::2013-11-20 02:58:27,516::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913204::DEBUG::2013-11-20 02:58:27,517::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913204::DEBUG::2013-11-20 02:58:27,517::task::495::TaskManager.Task::(_debug) Task f18d68fa-7604-4e00-90d7-1dba6f6c07e1: ref 0 aborting False
>Thread-15913204::INFO::2013-11-20 02:58:27,518::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000434875488281', 'lastCheck': 1384916305.3080609, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00228691101074', 'lastCheck': 1384916303.2722261, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00173807144165', 'lastCheck': 1384916304.2608531, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000536918640137', 'lastCheck': 1384916302.7748311, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:58:27,717::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:58:28,257::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0744243 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:58:30,271::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:58:30,817::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0731876 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:58:32,832::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:58:33,296::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0523005 s, 19.6 MB/s\n'; <rc> = 0
>Thread-15913208::DEBUG::2013-11-20 02:58:34,894::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913208::INFO::2013-11-20 02:58:34,895::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913208::DEBUG::2013-11-20 02:58:34,896::task::495::TaskManager.Task::(_debug) Task ea7be426-66b0-4a6c-b838-9d0d971db179: moving from state init -> state preparing
>Thread-15913208::DEBUG::2013-11-20 02:58:34,897::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913208::DEBUG::2013-11-20 02:58:34,897::task::495::TaskManager.Task::(_debug) Task ea7be426-66b0-4a6c-b838-9d0d971db179: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913208::DEBUG::2013-11-20 02:58:34,898::task::495::TaskManager.Task::(_debug) Task ea7be426-66b0-4a6c-b838-9d0d971db179: moving from state preparing -> state finished
>Thread-15913208::DEBUG::2013-11-20 02:58:34,899::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913208::DEBUG::2013-11-20 02:58:34,899::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913208::DEBUG::2013-11-20 02:58:34,900::task::495::TaskManager.Task::(_debug) Task ea7be426-66b0-4a6c-b838-9d0d971db179: ref 0 aborting False
>Thread-15913208::INFO::2013-11-20 02:58:34,900::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913209::DEBUG::2013-11-20 02:58:34,905::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913209::INFO::2013-11-20 02:58:34,906::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913209::DEBUG::2013-11-20 02:58:34,906::task::495::TaskManager.Task::(_debug) Task 64b64f5d-848f-41a4-9a35-4b592df5a7a7: moving from state init -> state preparing
>Thread-15913209::DEBUG::2013-11-20 02:58:34,907::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`390c4272-abfc-420e-9727-2e52a27256cb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913209::DEBUG::2013-11-20 02:58:34,908::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913209::DEBUG::2013-11-20 02:58:34,909::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913209::DEBUG::2013-11-20 02:58:34,909::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`390c4272-abfc-420e-9727-2e52a27256cb`::Granted request
>Thread-15913209::DEBUG::2013-11-20 02:58:34,910::task::495::TaskManager.Task::(_debug) Task 64b64f5d-848f-41a4-9a35-4b592df5a7a7: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913209::DEBUG::2013-11-20 02:58:34,911::task::495::TaskManager.Task::(_debug) Task 64b64f5d-848f-41a4-9a35-4b592df5a7a7: ref 1 aborting False
>Thread-15913209::WARNING::2013-11-20 02:58:34,913::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913209::DEBUG::2013-11-20 02:58:34,914::task::495::TaskManager.Task::(_debug) Task 64b64f5d-848f-41a4-9a35-4b592df5a7a7: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913209::DEBUG::2013-11-20 02:58:34,914::task::495::TaskManager.Task::(_debug) Task 64b64f5d-848f-41a4-9a35-4b592df5a7a7: moving from state preparing -> state finished
>Thread-15913209::DEBUG::2013-11-20 02:58:34,915::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913209::DEBUG::2013-11-20 02:58:34,916::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913209::DEBUG::2013-11-20 02:58:34,916::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913209::DEBUG::2013-11-20 02:58:34,917::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913209::DEBUG::2013-11-20 02:58:34,917::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913209::DEBUG::2013-11-20 02:58:34,918::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913209::DEBUG::2013-11-20 02:58:34,918::task::495::TaskManager.Task::(_debug) Task 64b64f5d-848f-41a4-9a35-4b592df5a7a7: ref 0 aborting False
>Thread-15913209::INFO::2013-11-20 02:58:34,919::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:58:35,307::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:58:35,830::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0776648 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:58:37,845::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:58:38,407::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0733467 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:58:40,423::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:58:40,960::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0751801 s, 13.6 MB/s\n'; <rc> = 0
>Thread-15913212::INFO::2013-11-20 02:58:42,825::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913212::DEBUG::2013-11-20 02:58:42,826::task::495::TaskManager.Task::(_debug) Task 8580fde6-849b-48ef-a005-2c5ff1db4215: moving from state init -> state preparing
>Thread-15913212::DEBUG::2013-11-20 02:58:42,826::task::495::TaskManager.Task::(_debug) Task 8580fde6-849b-48ef-a005-2c5ff1db4215: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000668048858643', 'lastCheck': 1384916315.518204, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0011100769043', 'lastCheck': 1384916313.507431, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00123286247253', 'lastCheck': 1384916314.5126541, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000712871551514', 'lastCheck': 1384916313.0202761, 'valid': True, 'code': 0}}
>Thread-15913212::DEBUG::2013-11-20 02:58:42,827::task::495::TaskManager.Task::(_debug) Task 8580fde6-849b-48ef-a005-2c5ff1db4215: moving from state preparing -> state finished
>Thread-15913212::DEBUG::2013-11-20 02:58:42,828::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913212::DEBUG::2013-11-20 02:58:42,828::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913212::DEBUG::2013-11-20 02:58:42,829::task::495::TaskManager.Task::(_debug) Task 8580fde6-849b-48ef-a005-2c5ff1db4215: ref 0 aborting False
>Thread-15913212::INFO::2013-11-20 02:58:42,830::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000668048858643', 'lastCheck': 1384916315.518204, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0011100769043', 'lastCheck': 1384916313.507431, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00123286247253', 'lastCheck': 1384916314.5126541, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000712871551514', 'lastCheck': 1384916313.0202761, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:58:42,975::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:58:43,491::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0835166 s, 12.3 MB/s\n'; <rc> = 0
>Thread-15913214::DEBUG::2013-11-20 02:58:45,132::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913214::INFO::2013-11-20 02:58:45,133::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913214::DEBUG::2013-11-20 02:58:45,134::task::495::TaskManager.Task::(_debug) Task 2485399d-7f95-41fd-9a44-710261c5a268: moving from state init -> state preparing
>Thread-15913214::DEBUG::2013-11-20 02:58:45,135::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913214::DEBUG::2013-11-20 02:58:45,135::task::495::TaskManager.Task::(_debug) Task 2485399d-7f95-41fd-9a44-710261c5a268: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913214::DEBUG::2013-11-20 02:58:45,136::task::495::TaskManager.Task::(_debug) Task 2485399d-7f95-41fd-9a44-710261c5a268: moving from state preparing -> state finished
>Thread-15913214::DEBUG::2013-11-20 02:58:45,136::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913214::DEBUG::2013-11-20 02:58:45,137::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913214::DEBUG::2013-11-20 02:58:45,137::task::495::TaskManager.Task::(_debug) Task 2485399d-7f95-41fd-9a44-710261c5a268: ref 0 aborting False
>Thread-15913214::INFO::2013-11-20 02:58:45,138::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913215::DEBUG::2013-11-20 02:58:45,279::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913215::INFO::2013-11-20 02:58:45,280::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913215::DEBUG::2013-11-20 02:58:45,281::task::495::TaskManager.Task::(_debug) Task 145bd003-6dfe-4f88-a861-01a959815248: moving from state init -> state preparing
>Thread-15913215::DEBUG::2013-11-20 02:58:45,282::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`32a00f17-bf7d-4167-9f10-45d8b1fbc689`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913215::DEBUG::2013-11-20 02:58:45,283::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913215::DEBUG::2013-11-20 02:58:45,284::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913215::DEBUG::2013-11-20 02:58:45,284::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`32a00f17-bf7d-4167-9f10-45d8b1fbc689`::Granted request
>Thread-15913215::DEBUG::2013-11-20 02:58:45,285::task::495::TaskManager.Task::(_debug) Task 145bd003-6dfe-4f88-a861-01a959815248: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913215::DEBUG::2013-11-20 02:58:45,286::task::495::TaskManager.Task::(_debug) Task 145bd003-6dfe-4f88-a861-01a959815248: ref 1 aborting False
>Thread-15913215::WARNING::2013-11-20 02:58:45,287::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913215::DEBUG::2013-11-20 02:58:45,288::task::495::TaskManager.Task::(_debug) Task 145bd003-6dfe-4f88-a861-01a959815248: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913215::DEBUG::2013-11-20 02:58:45,289::task::495::TaskManager.Task::(_debug) Task 145bd003-6dfe-4f88-a861-01a959815248: moving from state preparing -> state finished
>Thread-15913215::DEBUG::2013-11-20 02:58:45,290::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913215::DEBUG::2013-11-20 02:58:45,290::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913215::DEBUG::2013-11-20 02:58:45,291::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913215::DEBUG::2013-11-20 02:58:45,291::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913215::DEBUG::2013-11-20 02:58:45,292::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913215::DEBUG::2013-11-20 02:58:45,292::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913215::DEBUG::2013-11-20 02:58:45,293::task::495::TaskManager.Task::(_debug) Task 145bd003-6dfe-4f88-a861-01a959815248: ref 0 aborting False
>Thread-15913215::INFO::2013-11-20 02:58:45,294::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:58:45,506::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:58:46,072::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.073462 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:58:48,087::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:58:48,627::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0742011 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:58:50,642::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:58:51,198::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0749834 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:58:53,213::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-42840::DEBUG::2013-11-20 02:58:53,426::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:58:53,861::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.127751 s, 8.0 MB/s\n'; <rc> = 0
>Thread-42840::DEBUG::2013-11-20 02:58:54,205::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913220::DEBUG::2013-11-20 02:58:55,473::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913220::INFO::2013-11-20 02:58:55,475::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913220::DEBUG::2013-11-20 02:58:55,475::task::495::TaskManager.Task::(_debug) Task 6943cb9d-0d9a-4fd8-a2ed-c0b99cc2a526: moving from state init -> state preparing
>Thread-15913220::DEBUG::2013-11-20 02:58:55,476::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913220::DEBUG::2013-11-20 02:58:55,477::task::495::TaskManager.Task::(_debug) Task 6943cb9d-0d9a-4fd8-a2ed-c0b99cc2a526: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913220::DEBUG::2013-11-20 02:58:55,477::task::495::TaskManager.Task::(_debug) Task 6943cb9d-0d9a-4fd8-a2ed-c0b99cc2a526: moving from state preparing -> state finished
>Thread-15913220::DEBUG::2013-11-20 02:58:55,477::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913220::DEBUG::2013-11-20 02:58:55,478::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913220::DEBUG::2013-11-20 02:58:55,478::task::495::TaskManager.Task::(_debug) Task 6943cb9d-0d9a-4fd8-a2ed-c0b99cc2a526: ref 0 aborting False
>Thread-15913220::INFO::2013-11-20 02:58:55,479::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913221::DEBUG::2013-11-20 02:58:55,482::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913221::INFO::2013-11-20 02:58:55,482::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913221::DEBUG::2013-11-20 02:58:55,483::task::495::TaskManager.Task::(_debug) Task 3e58ceff-a3be-4827-ae6d-8a8b880a84cc: moving from state init -> state preparing
>Thread-15913221::DEBUG::2013-11-20 02:58:55,483::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c559c498-407a-44d1-af56-9e7492f6df99`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913221::DEBUG::2013-11-20 02:58:55,484::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913221::DEBUG::2013-11-20 02:58:55,484::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913221::DEBUG::2013-11-20 02:58:55,485::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c559c498-407a-44d1-af56-9e7492f6df99`::Granted request
>Thread-15913221::DEBUG::2013-11-20 02:58:55,486::task::495::TaskManager.Task::(_debug) Task 3e58ceff-a3be-4827-ae6d-8a8b880a84cc: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913221::DEBUG::2013-11-20 02:58:55,487::task::495::TaskManager.Task::(_debug) Task 3e58ceff-a3be-4827-ae6d-8a8b880a84cc: ref 1 aborting False
>Thread-15913221::WARNING::2013-11-20 02:58:55,488::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913221::DEBUG::2013-11-20 02:58:55,489::task::495::TaskManager.Task::(_debug) Task 3e58ceff-a3be-4827-ae6d-8a8b880a84cc: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913221::DEBUG::2013-11-20 02:58:55,490::task::495::TaskManager.Task::(_debug) Task 3e58ceff-a3be-4827-ae6d-8a8b880a84cc: moving from state preparing -> state finished
>Thread-15913221::DEBUG::2013-11-20 02:58:55,491::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913221::DEBUG::2013-11-20 02:58:55,491::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913221::DEBUG::2013-11-20 02:58:55,492::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913221::DEBUG::2013-11-20 02:58:55,493::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913221::DEBUG::2013-11-20 02:58:55,493::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913221::DEBUG::2013-11-20 02:58:55,493::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913221::DEBUG::2013-11-20 02:58:55,494::task::495::TaskManager.Task::(_debug) Task 3e58ceff-a3be-4827-ae6d-8a8b880a84cc: ref 0 aborting False
>Thread-15913221::INFO::2013-11-20 02:58:55,495::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:58:55,872::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:58:56,407::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0793054 s, 12.9 MB/s\n'; <rc> = 0
>Thread-15913222::INFO::2013-11-20 02:58:58,312::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913222::DEBUG::2013-11-20 02:58:58,312::task::495::TaskManager.Task::(_debug) Task bc4d6559-b598-4bc2-80ca-fabd7254a2a4: moving from state init -> state preparing
>Thread-15913222::DEBUG::2013-11-20 02:58:58,313::task::495::TaskManager.Task::(_debug) Task bc4d6559-b598-4bc2-80ca-fabd7254a2a4: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000589847564697', 'lastCheck': 1384916336.1002679, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00150799751282', 'lastCheck': 1384916333.889853, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00127482414246', 'lastCheck': 1384916334.8867331, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000324010848999', 'lastCheck': 1384916334.2075441, 'valid': True, 'code': 0}}
>Thread-15913222::DEBUG::2013-11-20 02:58:58,314::task::495::TaskManager.Task::(_debug) Task bc4d6559-b598-4bc2-80ca-fabd7254a2a4: moving from state preparing -> state finished
>Thread-15913222::DEBUG::2013-11-20 02:58:58,315::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913222::DEBUG::2013-11-20 02:58:58,315::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913222::DEBUG::2013-11-20 02:58:58,316::task::495::TaskManager.Task::(_debug) Task bc4d6559-b598-4bc2-80ca-fabd7254a2a4: ref 0 aborting False
>Thread-15913222::INFO::2013-11-20 02:58:58,316::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000589847564697', 'lastCheck': 1384916336.1002679, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00150799751282', 'lastCheck': 1384916333.889853, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00127482414246', 'lastCheck': 1384916334.8867331, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000324010848999', 'lastCheck': 1384916334.2075441, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:58:58,422::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:58:59,012::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0878927 s, 11.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:59:01,027::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:59:01,614::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0737087 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:59:03,629::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:59:04,181::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0736066 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15913226::DEBUG::2013-11-20 02:59:05,634::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913226::INFO::2013-11-20 02:59:05,635::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913226::DEBUG::2013-11-20 02:59:05,636::task::495::TaskManager.Task::(_debug) Task 891d3a21-0836-4c93-883c-ec690ae3d460: moving from state init -> state preparing
>Thread-15913226::DEBUG::2013-11-20 02:59:05,637::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913226::DEBUG::2013-11-20 02:59:05,637::task::495::TaskManager.Task::(_debug) Task 891d3a21-0836-4c93-883c-ec690ae3d460: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913226::DEBUG::2013-11-20 02:59:05,638::task::495::TaskManager.Task::(_debug) Task 891d3a21-0836-4c93-883c-ec690ae3d460: moving from state preparing -> state finished
>Thread-15913226::DEBUG::2013-11-20 02:59:05,638::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913226::DEBUG::2013-11-20 02:59:05,639::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913226::DEBUG::2013-11-20 02:59:05,639::task::495::TaskManager.Task::(_debug) Task 891d3a21-0836-4c93-883c-ec690ae3d460: ref 0 aborting False
>Thread-15913226::INFO::2013-11-20 02:59:05,640::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913227::DEBUG::2013-11-20 02:59:05,644::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913227::INFO::2013-11-20 02:59:05,645::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913227::DEBUG::2013-11-20 02:59:05,646::task::495::TaskManager.Task::(_debug) Task 7d4f6125-c199-4646-adc5-8909ddfec23d: moving from state init -> state preparing
>Thread-15913227::DEBUG::2013-11-20 02:59:05,647::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`10ec247b-253c-44c9-accd-56083dcc9072`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913227::DEBUG::2013-11-20 02:59:05,648::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913227::DEBUG::2013-11-20 02:59:05,649::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913227::DEBUG::2013-11-20 02:59:05,649::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`10ec247b-253c-44c9-accd-56083dcc9072`::Granted request
>Thread-15913227::DEBUG::2013-11-20 02:59:05,650::task::495::TaskManager.Task::(_debug) Task 7d4f6125-c199-4646-adc5-8909ddfec23d: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913227::DEBUG::2013-11-20 02:59:05,651::task::495::TaskManager.Task::(_debug) Task 7d4f6125-c199-4646-adc5-8909ddfec23d: ref 1 aborting False
>Thread-15913227::WARNING::2013-11-20 02:59:05,653::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913227::DEBUG::2013-11-20 02:59:05,653::task::495::TaskManager.Task::(_debug) Task 7d4f6125-c199-4646-adc5-8909ddfec23d: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913227::DEBUG::2013-11-20 02:59:05,654::task::495::TaskManager.Task::(_debug) Task 7d4f6125-c199-4646-adc5-8909ddfec23d: moving from state preparing -> state finished
>Thread-15913227::DEBUG::2013-11-20 02:59:05,655::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913227::DEBUG::2013-11-20 02:59:05,655::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913227::DEBUG::2013-11-20 02:59:05,656::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913227::DEBUG::2013-11-20 02:59:05,657::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913227::DEBUG::2013-11-20 02:59:05,657::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913227::DEBUG::2013-11-20 02:59:05,658::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913227::DEBUG::2013-11-20 02:59:05,658::task::495::TaskManager.Task::(_debug) Task 7d4f6125-c199-4646-adc5-8909ddfec23d: ref 0 aborting False
>Thread-15913227::INFO::2013-11-20 02:59:05,659::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:59:06,196::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:59:06,734::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.110716 s, 9.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:59:08,749::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:59:09,196::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0443501 s, 23.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:59:11,211::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:59:11,735::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0729556 s, 14.0 MB/s\n'; <rc> = 0
>Thread-15913230::INFO::2013-11-20 02:59:13,654::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913230::DEBUG::2013-11-20 02:59:13,655::task::495::TaskManager.Task::(_debug) Task bce888a1-9c6a-484b-a727-f90b80d42be9: moving from state init -> state preparing
>Thread-15913230::DEBUG::2013-11-20 02:59:13,656::task::495::TaskManager.Task::(_debug) Task bce888a1-9c6a-484b-a727-f90b80d42be9: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000385046005249', 'lastCheck': 1384916346.4443099, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00134205818176', 'lastCheck': 1384916344.3229661, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00131607055664', 'lastCheck': 1384916345.296495, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00235104560852', 'lastCheck': 1384916344.314662, 'valid': True, 'code': 0}}
>Thread-15913230::DEBUG::2013-11-20 02:59:13,657::task::495::TaskManager.Task::(_debug) Task bce888a1-9c6a-484b-a727-f90b80d42be9: moving from state preparing -> state finished
>Thread-15913230::DEBUG::2013-11-20 02:59:13,657::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913230::DEBUG::2013-11-20 02:59:13,658::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913230::DEBUG::2013-11-20 02:59:13,659::task::495::TaskManager.Task::(_debug) Task bce888a1-9c6a-484b-a727-f90b80d42be9: ref 0 aborting False
>Thread-15913230::INFO::2013-11-20 02:59:13,661::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000385046005249', 'lastCheck': 1384916346.4443099, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00134205818176', 'lastCheck': 1384916344.3229661, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00131607055664', 'lastCheck': 1384916345.296495, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00235104560852', 'lastCheck': 1384916344.314662, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:59:13,750::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:59:14,287::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0735717 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15913232::DEBUG::2013-11-20 02:59:15,764::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913232::INFO::2013-11-20 02:59:15,766::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913232::DEBUG::2013-11-20 02:59:15,766::task::495::TaskManager.Task::(_debug) Task 5df1fd71-96fb-44b4-a171-5057a550f6a4: moving from state init -> state preparing
>Thread-15913232::DEBUG::2013-11-20 02:59:15,767::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913232::DEBUG::2013-11-20 02:59:15,768::task::495::TaskManager.Task::(_debug) Task 5df1fd71-96fb-44b4-a171-5057a550f6a4: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913232::DEBUG::2013-11-20 02:59:15,768::task::495::TaskManager.Task::(_debug) Task 5df1fd71-96fb-44b4-a171-5057a550f6a4: moving from state preparing -> state finished
>Thread-15913232::DEBUG::2013-11-20 02:59:15,769::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913232::DEBUG::2013-11-20 02:59:15,769::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913232::DEBUG::2013-11-20 02:59:15,770::task::495::TaskManager.Task::(_debug) Task 5df1fd71-96fb-44b4-a171-5057a550f6a4: ref 0 aborting False
>Thread-15913232::INFO::2013-11-20 02:59:15,771::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913233::DEBUG::2013-11-20 02:59:15,775::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913233::INFO::2013-11-20 02:59:15,776::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913233::DEBUG::2013-11-20 02:59:15,777::task::495::TaskManager.Task::(_debug) Task b669f193-017a-403e-bea5-6c32bd107ab1: moving from state init -> state preparing
>Thread-15913233::DEBUG::2013-11-20 02:59:15,778::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`691efe9c-713f-49c8-8ff3-0ed75491b8e5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913233::DEBUG::2013-11-20 02:59:15,779::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913233::DEBUG::2013-11-20 02:59:15,780::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913233::DEBUG::2013-11-20 02:59:15,780::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`691efe9c-713f-49c8-8ff3-0ed75491b8e5`::Granted request
>Thread-15913233::DEBUG::2013-11-20 02:59:15,781::task::495::TaskManager.Task::(_debug) Task b669f193-017a-403e-bea5-6c32bd107ab1: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913233::DEBUG::2013-11-20 02:59:15,782::task::495::TaskManager.Task::(_debug) Task b669f193-017a-403e-bea5-6c32bd107ab1: ref 1 aborting False
>Thread-15913233::WARNING::2013-11-20 02:59:15,784::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913233::DEBUG::2013-11-20 02:59:15,785::task::495::TaskManager.Task::(_debug) Task b669f193-017a-403e-bea5-6c32bd107ab1: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913233::DEBUG::2013-11-20 02:59:15,786::task::495::TaskManager.Task::(_debug) Task b669f193-017a-403e-bea5-6c32bd107ab1: moving from state preparing -> state finished
>Thread-15913233::DEBUG::2013-11-20 02:59:15,787::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913233::DEBUG::2013-11-20 02:59:15,787::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913233::DEBUG::2013-11-20 02:59:15,788::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913233::DEBUG::2013-11-20 02:59:15,789::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913233::DEBUG::2013-11-20 02:59:15,789::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913233::DEBUG::2013-11-20 02:59:15,790::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913233::DEBUG::2013-11-20 02:59:15,791::task::495::TaskManager.Task::(_debug) Task b669f193-017a-403e-bea5-6c32bd107ab1: ref 0 aborting False
>Thread-15913233::INFO::2013-11-20 02:59:15,791::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:59:16,302::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 02:59:16,557::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 02:59:16,559::task::495::TaskManager.Task::(_debug) Task f2ec60f1-f520-47c5-98c2-3522d756cd88: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 02:59:16,562::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6d1b3bf5-2d10-4042-bcfb-ac815ae8b21a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 02:59:16,562::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 02:59:16,563::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 02:59:16,564::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6d1b3bf5-2d10-4042-bcfb-ac815ae8b21a`::Granted request
>Thread-15912392::DEBUG::2013-11-20 02:59:16,565::task::495::TaskManager.Task::(_debug) Task f2ec60f1-f520-47c5-98c2-3522d756cd88: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 02:59:16,566::task::495::TaskManager.Task::(_debug) Task f2ec60f1-f520-47c5-98c2-3522d756cd88: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 02:59:16,566::task::495::TaskManager.Task::(_debug) Task f2ec60f1-f520-47c5-98c2-3522d756cd88: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 02:59:16,567::task::495::TaskManager.Task::(_debug) Task f2ec60f1-f520-47c5-98c2-3522d756cd88: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 02:59:16,567::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 02:59:16,568::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 02:59:16,569::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 02:59:16,569::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 02:59:16,570::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 02:59:16,570::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 02:59:16,571::task::495::TaskManager.Task::(_debug) Task f2ec60f1-f520-47c5-98c2-3522d756cd88: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 02:59:16,571::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 02:59:16,867::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0738402 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:59:18,881::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:59:19,428::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0815262 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:59:21,443::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:59:21,976::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0821618 s, 12.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:59:23,991::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:59:24,538::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0969849 s, 10.6 MB/s\n'; <rc> = 0
>Thread-15913238::DEBUG::2013-11-20 02:59:25,971::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913238::INFO::2013-11-20 02:59:25,973::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913238::DEBUG::2013-11-20 02:59:25,973::task::495::TaskManager.Task::(_debug) Task fb7bdce7-45f9-4b30-a473-a5fdfa8bbc6e: moving from state init -> state preparing
>Thread-15913238::DEBUG::2013-11-20 02:59:25,974::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913238::DEBUG::2013-11-20 02:59:25,975::task::495::TaskManager.Task::(_debug) Task fb7bdce7-45f9-4b30-a473-a5fdfa8bbc6e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913238::DEBUG::2013-11-20 02:59:25,975::task::495::TaskManager.Task::(_debug) Task fb7bdce7-45f9-4b30-a473-a5fdfa8bbc6e: moving from state preparing -> state finished
>Thread-15913238::DEBUG::2013-11-20 02:59:25,976::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913238::DEBUG::2013-11-20 02:59:25,977::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913238::DEBUG::2013-11-20 02:59:25,977::task::495::TaskManager.Task::(_debug) Task fb7bdce7-45f9-4b30-a473-a5fdfa8bbc6e: ref 0 aborting False
>Thread-15913238::INFO::2013-11-20 02:59:25,978::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913239::DEBUG::2013-11-20 02:59:25,983::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913239::INFO::2013-11-20 02:59:25,984::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913239::DEBUG::2013-11-20 02:59:25,984::task::495::TaskManager.Task::(_debug) Task bec4b0e8-f4a1-4e2d-9827-ea1c8239eb13: moving from state init -> state preparing
>Thread-15913239::DEBUG::2013-11-20 02:59:25,986::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`259379b0-fcbd-4b39-a60f-0f4fdfa39414`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913239::DEBUG::2013-11-20 02:59:25,986::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913239::DEBUG::2013-11-20 02:59:25,987::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913239::DEBUG::2013-11-20 02:59:25,988::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`259379b0-fcbd-4b39-a60f-0f4fdfa39414`::Granted request
>Thread-15913239::DEBUG::2013-11-20 02:59:25,989::task::495::TaskManager.Task::(_debug) Task bec4b0e8-f4a1-4e2d-9827-ea1c8239eb13: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913239::DEBUG::2013-11-20 02:59:25,989::task::495::TaskManager.Task::(_debug) Task bec4b0e8-f4a1-4e2d-9827-ea1c8239eb13: ref 1 aborting False
>Thread-15913239::WARNING::2013-11-20 02:59:25,991::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913239::DEBUG::2013-11-20 02:59:25,992::task::495::TaskManager.Task::(_debug) Task bec4b0e8-f4a1-4e2d-9827-ea1c8239eb13: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913239::DEBUG::2013-11-20 02:59:25,993::task::495::TaskManager.Task::(_debug) Task bec4b0e8-f4a1-4e2d-9827-ea1c8239eb13: moving from state preparing -> state finished
>Thread-15913239::DEBUG::2013-11-20 02:59:25,993::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913239::DEBUG::2013-11-20 02:59:25,994::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913239::DEBUG::2013-11-20 02:59:25,995::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913239::DEBUG::2013-11-20 02:59:25,995::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913239::DEBUG::2013-11-20 02:59:25,996::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913239::DEBUG::2013-11-20 02:59:25,996::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913239::DEBUG::2013-11-20 02:59:25,997::task::495::TaskManager.Task::(_debug) Task bec4b0e8-f4a1-4e2d-9827-ea1c8239eb13: ref 0 aborting False
>Thread-15913239::INFO::2013-11-20 02:59:25,998::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:59:26,553::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:59:27,103::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0706231 s, 14.5 MB/s\n'; <rc> = 0
>Thread-15913240::INFO::2013-11-20 02:59:28,899::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913240::DEBUG::2013-11-20 02:59:28,900::task::495::TaskManager.Task::(_debug) Task 20906ae3-e1cb-4569-b25b-7fd5432feb7b: moving from state init -> state preparing
>Thread-15913240::DEBUG::2013-11-20 02:59:28,901::task::495::TaskManager.Task::(_debug) Task 20906ae3-e1cb-4569-b25b-7fd5432feb7b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000354051589966', 'lastCheck': 1384916366.8071809, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00211191177368', 'lastCheck': 1384916364.6658731, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00181293487549', 'lastCheck': 1384916365.6644361, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.002366065979', 'lastCheck': 1384916364.6600189, 'valid': True, 'code': 0}}
>Thread-15913240::DEBUG::2013-11-20 02:59:28,901::task::495::TaskManager.Task::(_debug) Task 20906ae3-e1cb-4569-b25b-7fd5432feb7b: moving from state preparing -> state finished
>Thread-15913240::DEBUG::2013-11-20 02:59:28,902::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913240::DEBUG::2013-11-20 02:59:28,903::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913240::DEBUG::2013-11-20 02:59:28,903::task::495::TaskManager.Task::(_debug) Task 20906ae3-e1cb-4569-b25b-7fd5432feb7b: ref 0 aborting False
>Thread-15913240::INFO::2013-11-20 02:59:28,904::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000354051589966', 'lastCheck': 1384916366.8071809, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00211191177368', 'lastCheck': 1384916364.6658731, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00181293487549', 'lastCheck': 1384916365.6644361, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.002366065979', 'lastCheck': 1384916364.6600189, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:59:29,116::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:59:29,583::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0482029 s, 21.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:59:31,598::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:59:32,075::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0459884 s, 22.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:59:34,084::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:59:34,639::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0848739 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15913244::DEBUG::2013-11-20 02:59:36,179::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913244::INFO::2013-11-20 02:59:36,180::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913244::DEBUG::2013-11-20 02:59:36,181::task::495::TaskManager.Task::(_debug) Task 2061d586-df8c-45bb-8652-766491829589: moving from state init -> state preparing
>Thread-15913244::DEBUG::2013-11-20 02:59:36,182::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913244::DEBUG::2013-11-20 02:59:36,182::task::495::TaskManager.Task::(_debug) Task 2061d586-df8c-45bb-8652-766491829589: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913244::DEBUG::2013-11-20 02:59:36,183::task::495::TaskManager.Task::(_debug) Task 2061d586-df8c-45bb-8652-766491829589: moving from state preparing -> state finished
>Thread-15913244::DEBUG::2013-11-20 02:59:36,183::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913244::DEBUG::2013-11-20 02:59:36,184::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913244::DEBUG::2013-11-20 02:59:36,185::task::495::TaskManager.Task::(_debug) Task 2061d586-df8c-45bb-8652-766491829589: ref 0 aborting False
>Thread-15913244::INFO::2013-11-20 02:59:36,185::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913245::DEBUG::2013-11-20 02:59:36,190::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913245::INFO::2013-11-20 02:59:36,191::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913245::DEBUG::2013-11-20 02:59:36,191::task::495::TaskManager.Task::(_debug) Task 8bf4bcb9-a8b1-42ae-8f53-071e31c6fdd1: moving from state init -> state preparing
>Thread-15913245::DEBUG::2013-11-20 02:59:36,192::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`017a95bc-dffe-40f8-9329-fd38296e7850`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913245::DEBUG::2013-11-20 02:59:36,193::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913245::DEBUG::2013-11-20 02:59:36,194::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913245::DEBUG::2013-11-20 02:59:36,194::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`017a95bc-dffe-40f8-9329-fd38296e7850`::Granted request
>Thread-15913245::DEBUG::2013-11-20 02:59:36,195::task::495::TaskManager.Task::(_debug) Task 8bf4bcb9-a8b1-42ae-8f53-071e31c6fdd1: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913245::DEBUG::2013-11-20 02:59:36,196::task::495::TaskManager.Task::(_debug) Task 8bf4bcb9-a8b1-42ae-8f53-071e31c6fdd1: ref 1 aborting False
>Thread-15913245::WARNING::2013-11-20 02:59:36,198::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913245::DEBUG::2013-11-20 02:59:36,199::task::495::TaskManager.Task::(_debug) Task 8bf4bcb9-a8b1-42ae-8f53-071e31c6fdd1: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913245::DEBUG::2013-11-20 02:59:36,200::task::495::TaskManager.Task::(_debug) Task 8bf4bcb9-a8b1-42ae-8f53-071e31c6fdd1: moving from state preparing -> state finished
>Thread-15913245::DEBUG::2013-11-20 02:59:36,200::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913245::DEBUG::2013-11-20 02:59:36,201::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913245::DEBUG::2013-11-20 02:59:36,201::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913245::DEBUG::2013-11-20 02:59:36,202::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913245::DEBUG::2013-11-20 02:59:36,202::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913245::DEBUG::2013-11-20 02:59:36,203::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913245::DEBUG::2013-11-20 02:59:36,204::task::495::TaskManager.Task::(_debug) Task 8bf4bcb9-a8b1-42ae-8f53-071e31c6fdd1: ref 0 aborting False
>Thread-15913245::INFO::2013-11-20 02:59:36,204::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:59:36,654::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:59:37,210::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0751123 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:59:39,225::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:59:39,763::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0741205 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:59:41,778::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:59:42,328::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0780058 s, 13.1 MB/s\n'; <rc> = 0
>Thread-15913248::INFO::2013-11-20 02:59:44,185::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913248::DEBUG::2013-11-20 02:59:44,186::task::495::TaskManager.Task::(_debug) Task 63999f2f-70c3-476a-bf26-609273dd3e8f: moving from state init -> state preparing
>Thread-15913248::DEBUG::2013-11-20 02:59:44,187::task::495::TaskManager.Task::(_debug) Task 63999f2f-70c3-476a-bf26-609273dd3e8f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000416994094849', 'lastCheck': 1384916376.906837, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00488805770874', 'lastCheck': 1384916374.8139031, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0013689994812', 'lastCheck': 1384916375.8095939, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00142002105713', 'lastCheck': 1384916374.8098741, 'valid': True, 'code': 0}}
>Thread-15913248::DEBUG::2013-11-20 02:59:44,188::task::495::TaskManager.Task::(_debug) Task 63999f2f-70c3-476a-bf26-609273dd3e8f: moving from state preparing -> state finished
>Thread-15913248::DEBUG::2013-11-20 02:59:44,189::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913248::DEBUG::2013-11-20 02:59:44,189::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913248::DEBUG::2013-11-20 02:59:44,190::task::495::TaskManager.Task::(_debug) Task 63999f2f-70c3-476a-bf26-609273dd3e8f: ref 0 aborting False
>Thread-15913248::INFO::2013-11-20 02:59:44,190::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000416994094849', 'lastCheck': 1384916376.906837, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00488805770874', 'lastCheck': 1384916374.8139031, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0013689994812', 'lastCheck': 1384916375.8095939, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00142002105713', 'lastCheck': 1384916374.8098741, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:59:44,344::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:59:44,874::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0731782 s, 14.0 MB/s\n'; <rc> = 0
>Thread-15913250::DEBUG::2013-11-20 02:59:46,393::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913250::INFO::2013-11-20 02:59:46,395::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913250::DEBUG::2013-11-20 02:59:46,395::task::495::TaskManager.Task::(_debug) Task fbb40641-8783-4aef-8f40-01d20149916d: moving from state init -> state preparing
>Thread-15913250::DEBUG::2013-11-20 02:59:46,396::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913250::DEBUG::2013-11-20 02:59:46,397::task::495::TaskManager.Task::(_debug) Task fbb40641-8783-4aef-8f40-01d20149916d: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913250::DEBUG::2013-11-20 02:59:46,397::task::495::TaskManager.Task::(_debug) Task fbb40641-8783-4aef-8f40-01d20149916d: moving from state preparing -> state finished
>Thread-15913250::DEBUG::2013-11-20 02:59:46,398::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913250::DEBUG::2013-11-20 02:59:46,399::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913250::DEBUG::2013-11-20 02:59:46,399::task::495::TaskManager.Task::(_debug) Task fbb40641-8783-4aef-8f40-01d20149916d: ref 0 aborting False
>Thread-15913250::INFO::2013-11-20 02:59:46,400::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913251::DEBUG::2013-11-20 02:59:46,404::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913251::INFO::2013-11-20 02:59:46,405::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913251::DEBUG::2013-11-20 02:59:46,406::task::495::TaskManager.Task::(_debug) Task 1694a27a-b0df-430c-b512-261898851425: moving from state init -> state preparing
>Thread-15913251::DEBUG::2013-11-20 02:59:46,407::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bdfda5a9-6f59-4b96-aa18-e0a415e8174d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913251::DEBUG::2013-11-20 02:59:46,408::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913251::DEBUG::2013-11-20 02:59:46,409::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913251::DEBUG::2013-11-20 02:59:46,409::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bdfda5a9-6f59-4b96-aa18-e0a415e8174d`::Granted request
>Thread-15913251::DEBUG::2013-11-20 02:59:46,410::task::495::TaskManager.Task::(_debug) Task 1694a27a-b0df-430c-b512-261898851425: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913251::DEBUG::2013-11-20 02:59:46,411::task::495::TaskManager.Task::(_debug) Task 1694a27a-b0df-430c-b512-261898851425: ref 1 aborting False
>Thread-15913251::WARNING::2013-11-20 02:59:46,413::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913251::DEBUG::2013-11-20 02:59:46,414::task::495::TaskManager.Task::(_debug) Task 1694a27a-b0df-430c-b512-261898851425: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913251::DEBUG::2013-11-20 02:59:46,415::task::495::TaskManager.Task::(_debug) Task 1694a27a-b0df-430c-b512-261898851425: moving from state preparing -> state finished
>Thread-15913251::DEBUG::2013-11-20 02:59:46,415::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913251::DEBUG::2013-11-20 02:59:46,416::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913251::DEBUG::2013-11-20 02:59:46,416::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913251::DEBUG::2013-11-20 02:59:46,417::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913251::DEBUG::2013-11-20 02:59:46,418::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913251::DEBUG::2013-11-20 02:59:46,418::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913251::DEBUG::2013-11-20 02:59:46,419::task::495::TaskManager.Task::(_debug) Task 1694a27a-b0df-430c-b512-261898851425: ref 0 aborting False
>Thread-15913251::INFO::2013-11-20 02:59:46,419::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:59:46,889::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:59:47,434::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0741585 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:59:49,449::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:59:49,943::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0827072 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:59:51,959::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:59:52,492::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0762666 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 02:59:54,508::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:59:55,023::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0787463 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15913256::DEBUG::2013-11-20 02:59:56,667::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913256::INFO::2013-11-20 02:59:56,668::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913256::DEBUG::2013-11-20 02:59:56,669::task::495::TaskManager.Task::(_debug) Task 6da7c30e-bf0a-490b-8e5d-cecd4aedef25: moving from state init -> state preparing
>Thread-15913256::DEBUG::2013-11-20 02:59:56,670::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913256::DEBUG::2013-11-20 02:59:56,670::task::495::TaskManager.Task::(_debug) Task 6da7c30e-bf0a-490b-8e5d-cecd4aedef25: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913256::DEBUG::2013-11-20 02:59:56,671::task::495::TaskManager.Task::(_debug) Task 6da7c30e-bf0a-490b-8e5d-cecd4aedef25: moving from state preparing -> state finished
>Thread-15913256::DEBUG::2013-11-20 02:59:56,671::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913256::DEBUG::2013-11-20 02:59:56,672::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913256::DEBUG::2013-11-20 02:59:56,673::task::495::TaskManager.Task::(_debug) Task 6da7c30e-bf0a-490b-8e5d-cecd4aedef25: ref 0 aborting False
>Thread-15913256::INFO::2013-11-20 02:59:56,673::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913257::DEBUG::2013-11-20 02:59:56,678::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913257::INFO::2013-11-20 02:59:56,679::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913257::DEBUG::2013-11-20 02:59:56,680::task::495::TaskManager.Task::(_debug) Task 3b700c0e-8bf6-4179-9a0c-e1ae656a8036: moving from state init -> state preparing
>Thread-15913257::DEBUG::2013-11-20 02:59:56,681::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d8a0cf47-d54c-44d6-b1ad-255125c00ecb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913257::DEBUG::2013-11-20 02:59:56,682::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913257::DEBUG::2013-11-20 02:59:56,682::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913257::DEBUG::2013-11-20 02:59:56,683::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d8a0cf47-d54c-44d6-b1ad-255125c00ecb`::Granted request
>Thread-15913257::DEBUG::2013-11-20 02:59:56,684::task::495::TaskManager.Task::(_debug) Task 3b700c0e-8bf6-4179-9a0c-e1ae656a8036: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913257::DEBUG::2013-11-20 02:59:56,685::task::495::TaskManager.Task::(_debug) Task 3b700c0e-8bf6-4179-9a0c-e1ae656a8036: ref 1 aborting False
>Thread-15913257::WARNING::2013-11-20 02:59:56,686::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913257::DEBUG::2013-11-20 02:59:56,687::task::495::TaskManager.Task::(_debug) Task 3b700c0e-8bf6-4179-9a0c-e1ae656a8036: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913257::DEBUG::2013-11-20 02:59:56,688::task::495::TaskManager.Task::(_debug) Task 3b700c0e-8bf6-4179-9a0c-e1ae656a8036: moving from state preparing -> state finished
>Thread-15913257::DEBUG::2013-11-20 02:59:56,689::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913257::DEBUG::2013-11-20 02:59:56,689::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913257::DEBUG::2013-11-20 02:59:56,690::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913257::DEBUG::2013-11-20 02:59:56,691::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913257::DEBUG::2013-11-20 02:59:56,691::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913257::DEBUG::2013-11-20 02:59:56,692::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913257::DEBUG::2013-11-20 02:59:56,692::task::495::TaskManager.Task::(_debug) Task 3b700c0e-8bf6-4179-9a0c-e1ae656a8036: ref 0 aborting False
>Thread-15913257::INFO::2013-11-20 02:59:56,693::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 02:59:57,037::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 02:59:57,595::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0873732 s, 11.7 MB/s\n'; <rc> = 0
>Thread-15913258::INFO::2013-11-20 02:59:59,471::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913258::DEBUG::2013-11-20 02:59:59,472::task::495::TaskManager.Task::(_debug) Task c92ce316-2954-4c00-86af-64eacdcd0137: moving from state init -> state preparing
>Thread-15913258::DEBUG::2013-11-20 02:59:59,473::task::495::TaskManager.Task::(_debug) Task c92ce316-2954-4c00-86af-64eacdcd0137: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000452995300293', 'lastCheck': 1384916397.2781799, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148510932922', 'lastCheck': 1384916395.1808209, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00183415412903', 'lastCheck': 1384916396.1822181, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000585079193115', 'lastCheck': 1384916395.1753011, 'valid': True, 'code': 0}}
>Thread-15913258::DEBUG::2013-11-20 02:59:59,474::task::495::TaskManager.Task::(_debug) Task c92ce316-2954-4c00-86af-64eacdcd0137: moving from state preparing -> state finished
>Thread-15913258::DEBUG::2013-11-20 02:59:59,474::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913258::DEBUG::2013-11-20 02:59:59,475::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913258::DEBUG::2013-11-20 02:59:59,475::task::495::TaskManager.Task::(_debug) Task c92ce316-2954-4c00-86af-64eacdcd0137: ref 0 aborting False
>Thread-15913258::INFO::2013-11-20 02:59:59,476::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000452995300293', 'lastCheck': 1384916397.2781799, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148510932922', 'lastCheck': 1384916395.1808209, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00183415412903', 'lastCheck': 1384916396.1822181, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000585079193115', 'lastCheck': 1384916395.1753011, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 02:59:59,608::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:00:00,050::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0480607 s, 21.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:00:02,060::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:00:02,614::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0845828 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:00:04,629::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:00:05,191::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0739048 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15913262::DEBUG::2013-11-20 03:00:06,841::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913262::INFO::2013-11-20 03:00:06,843::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913262::DEBUG::2013-11-20 03:00:06,860::task::495::TaskManager.Task::(_debug) Task 6336c8bd-b413-45b0-84d1-dc6ae961e67d: moving from state init -> state preparing
>Thread-15913262::DEBUG::2013-11-20 03:00:06,861::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913262::DEBUG::2013-11-20 03:00:06,861::task::495::TaskManager.Task::(_debug) Task 6336c8bd-b413-45b0-84d1-dc6ae961e67d: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913262::DEBUG::2013-11-20 03:00:06,862::task::495::TaskManager.Task::(_debug) Task 6336c8bd-b413-45b0-84d1-dc6ae961e67d: moving from state preparing -> state finished
>Thread-15913262::DEBUG::2013-11-20 03:00:06,862::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913262::DEBUG::2013-11-20 03:00:06,863::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913262::DEBUG::2013-11-20 03:00:06,863::task::495::TaskManager.Task::(_debug) Task 6336c8bd-b413-45b0-84d1-dc6ae961e67d: ref 0 aborting False
>Thread-15913262::INFO::2013-11-20 03:00:06,864::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913263::DEBUG::2013-11-20 03:00:06,869::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913263::INFO::2013-11-20 03:00:06,870::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913263::DEBUG::2013-11-20 03:00:06,871::task::495::TaskManager.Task::(_debug) Task cc1fc145-b192-43d5-b569-eeac350662b4: moving from state init -> state preparing
>Thread-15913263::DEBUG::2013-11-20 03:00:06,872::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`68379c8a-cb7a-4799-8986-bd1ffae56290`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913263::DEBUG::2013-11-20 03:00:06,873::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913263::DEBUG::2013-11-20 03:00:06,874::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913263::DEBUG::2013-11-20 03:00:06,874::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`68379c8a-cb7a-4799-8986-bd1ffae56290`::Granted request
>Thread-15913263::DEBUG::2013-11-20 03:00:06,875::task::495::TaskManager.Task::(_debug) Task cc1fc145-b192-43d5-b569-eeac350662b4: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913263::DEBUG::2013-11-20 03:00:06,876::task::495::TaskManager.Task::(_debug) Task cc1fc145-b192-43d5-b569-eeac350662b4: ref 1 aborting False
>Thread-15913263::WARNING::2013-11-20 03:00:06,880::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913263::DEBUG::2013-11-20 03:00:06,881::task::495::TaskManager.Task::(_debug) Task cc1fc145-b192-43d5-b569-eeac350662b4: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913263::DEBUG::2013-11-20 03:00:06,883::task::495::TaskManager.Task::(_debug) Task cc1fc145-b192-43d5-b569-eeac350662b4: moving from state preparing -> state finished
>Thread-15913263::DEBUG::2013-11-20 03:00:06,884::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913263::DEBUG::2013-11-20 03:00:06,885::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913263::DEBUG::2013-11-20 03:00:06,887::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913263::DEBUG::2013-11-20 03:00:06,888::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913263::DEBUG::2013-11-20 03:00:06,889::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913263::DEBUG::2013-11-20 03:00:06,890::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913263::DEBUG::2013-11-20 03:00:06,892::task::495::TaskManager.Task::(_debug) Task cc1fc145-b192-43d5-b569-eeac350662b4: ref 0 aborting False
>Thread-15913263::INFO::2013-11-20 03:00:06,893::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:00:07,207::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:00:07,727::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.069524 s, 14.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:00:09,740::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:00:10,263::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0768087 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:00:12,279::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:00:12,815::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0739975 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15913266::INFO::2013-11-20 03:00:14,735::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913266::DEBUG::2013-11-20 03:00:14,736::task::495::TaskManager.Task::(_debug) Task 7cd4672f-2b05-45d2-9662-af26fb635778: moving from state init -> state preparing
>Thread-15913266::DEBUG::2013-11-20 03:00:14,736::task::495::TaskManager.Task::(_debug) Task 7cd4672f-2b05-45d2-9662-af26fb635778: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000590085983276', 'lastCheck': 1384916407.4461031, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014660358429', 'lastCheck': 1384916405.283742, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00134992599487', 'lastCheck': 1384916406.28438, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000519037246704', 'lastCheck': 1384916405.2775559, 'valid': True, 'code': 0}}
>Thread-15913266::DEBUG::2013-11-20 03:00:14,737::task::495::TaskManager.Task::(_debug) Task 7cd4672f-2b05-45d2-9662-af26fb635778: moving from state preparing -> state finished
>Thread-15913266::DEBUG::2013-11-20 03:00:14,737::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913266::DEBUG::2013-11-20 03:00:14,738::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913266::DEBUG::2013-11-20 03:00:14,738::task::495::TaskManager.Task::(_debug) Task 7cd4672f-2b05-45d2-9662-af26fb635778: ref 0 aborting False
>Thread-15913266::INFO::2013-11-20 03:00:14,739::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000590085983276', 'lastCheck': 1384916407.4461031, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014660358429', 'lastCheck': 1384916405.283742, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00134992599487', 'lastCheck': 1384916406.28438, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000519037246704', 'lastCheck': 1384916405.2775559, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:00:14,831::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:00:15,363::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0719022 s, 14.2 MB/s\n'; <rc> = 0
>Thread-15913268::DEBUG::2013-11-20 03:00:17,034::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913268::INFO::2013-11-20 03:00:17,035::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913268::DEBUG::2013-11-20 03:00:17,035::task::495::TaskManager.Task::(_debug) Task 72d38d60-f6d4-426b-98af-d6b84800fa60: moving from state init -> state preparing
>Thread-15913268::DEBUG::2013-11-20 03:00:17,036::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913268::DEBUG::2013-11-20 03:00:17,036::task::495::TaskManager.Task::(_debug) Task 72d38d60-f6d4-426b-98af-d6b84800fa60: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913268::DEBUG::2013-11-20 03:00:17,036::task::495::TaskManager.Task::(_debug) Task 72d38d60-f6d4-426b-98af-d6b84800fa60: moving from state preparing -> state finished
>Thread-15913268::DEBUG::2013-11-20 03:00:17,037::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913268::DEBUG::2013-11-20 03:00:17,037::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913268::DEBUG::2013-11-20 03:00:17,037::task::495::TaskManager.Task::(_debug) Task 72d38d60-f6d4-426b-98af-d6b84800fa60: ref 0 aborting False
>Thread-15913268::INFO::2013-11-20 03:00:17,038::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913269::DEBUG::2013-11-20 03:00:17,041::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913269::INFO::2013-11-20 03:00:17,042::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913269::DEBUG::2013-11-20 03:00:17,042::task::495::TaskManager.Task::(_debug) Task 46df6ce6-7bf1-4179-b952-bd1b8c9b5b1e: moving from state init -> state preparing
>Thread-15913269::DEBUG::2013-11-20 03:00:17,043::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`27a9d8e4-0fc4-4b41-bdf6-56d82375fbf9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913269::DEBUG::2013-11-20 03:00:17,043::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913269::DEBUG::2013-11-20 03:00:17,044::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913269::DEBUG::2013-11-20 03:00:17,044::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`27a9d8e4-0fc4-4b41-bdf6-56d82375fbf9`::Granted request
>Thread-15913269::DEBUG::2013-11-20 03:00:17,045::task::495::TaskManager.Task::(_debug) Task 46df6ce6-7bf1-4179-b952-bd1b8c9b5b1e: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913269::DEBUG::2013-11-20 03:00:17,045::task::495::TaskManager.Task::(_debug) Task 46df6ce6-7bf1-4179-b952-bd1b8c9b5b1e: ref 1 aborting False
>Thread-15913269::WARNING::2013-11-20 03:00:17,046::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913269::DEBUG::2013-11-20 03:00:17,047::task::495::TaskManager.Task::(_debug) Task 46df6ce6-7bf1-4179-b952-bd1b8c9b5b1e: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913269::DEBUG::2013-11-20 03:00:17,047::task::495::TaskManager.Task::(_debug) Task 46df6ce6-7bf1-4179-b952-bd1b8c9b5b1e: moving from state preparing -> state finished
>Thread-15913269::DEBUG::2013-11-20 03:00:17,048::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913269::DEBUG::2013-11-20 03:00:17,048::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913269::DEBUG::2013-11-20 03:00:17,048::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913269::DEBUG::2013-11-20 03:00:17,049::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913269::DEBUG::2013-11-20 03:00:17,049::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913269::DEBUG::2013-11-20 03:00:17,049::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913269::DEBUG::2013-11-20 03:00:17,050::task::495::TaskManager.Task::(_debug) Task 46df6ce6-7bf1-4179-b952-bd1b8c9b5b1e: ref 0 aborting False
>Thread-15913269::INFO::2013-11-20 03:00:17,050::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:00:17,378::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 03:00:17,619::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:00:17,637::task::495::TaskManager.Task::(_debug) Task 9fd40913-556a-40f6-85ff-eae4d164c491: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:00:17,640::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d8ba3c6b-8050-40b4-864d-6997fd53a8be`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:00:17,640::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:00:17,641::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:00:17,642::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d8ba3c6b-8050-40b4-864d-6997fd53a8be`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:00:17,643::task::495::TaskManager.Task::(_debug) Task 9fd40913-556a-40f6-85ff-eae4d164c491: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:00:17,644::task::495::TaskManager.Task::(_debug) Task 9fd40913-556a-40f6-85ff-eae4d164c491: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:00:17,644::task::495::TaskManager.Task::(_debug) Task 9fd40913-556a-40f6-85ff-eae4d164c491: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:00:17,645::task::495::TaskManager.Task::(_debug) Task 9fd40913-556a-40f6-85ff-eae4d164c491: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:00:17,645::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:00:17,646::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:00:17,647::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:00:17,647::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:00:17,648::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:00:17,648::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:00:17,649::task::495::TaskManager.Task::(_debug) Task 9fd40913-556a-40f6-85ff-eae4d164c491: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:00:17,649::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:00:17,958::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0757912 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:00:19,974::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:00:20,513::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0767598 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:00:22,529::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:00:23,044::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0717671 s, 14.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:00:25,058::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:00:25,593::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0477387 s, 21.5 MB/s\n'; <rc> = 0
>Thread-15913274::DEBUG::2013-11-20 03:00:27,243::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913274::INFO::2013-11-20 03:00:27,244::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913274::DEBUG::2013-11-20 03:00:27,245::task::495::TaskManager.Task::(_debug) Task 866bf968-95bf-468f-8c2f-8661faec8306: moving from state init -> state preparing
>Thread-15913274::DEBUG::2013-11-20 03:00:27,246::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913274::DEBUG::2013-11-20 03:00:27,246::task::495::TaskManager.Task::(_debug) Task 866bf968-95bf-468f-8c2f-8661faec8306: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913274::DEBUG::2013-11-20 03:00:27,247::task::495::TaskManager.Task::(_debug) Task 866bf968-95bf-468f-8c2f-8661faec8306: moving from state preparing -> state finished
>Thread-15913274::DEBUG::2013-11-20 03:00:27,248::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913274::DEBUG::2013-11-20 03:00:27,248::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913274::DEBUG::2013-11-20 03:00:27,249::task::495::TaskManager.Task::(_debug) Task 866bf968-95bf-468f-8c2f-8661faec8306: ref 0 aborting False
>Thread-15913274::INFO::2013-11-20 03:00:27,249::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913275::DEBUG::2013-11-20 03:00:27,254::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913275::INFO::2013-11-20 03:00:27,255::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913275::DEBUG::2013-11-20 03:00:27,255::task::495::TaskManager.Task::(_debug) Task 87125b1b-4468-4e16-8283-8eb40014c195: moving from state init -> state preparing
>Thread-15913275::DEBUG::2013-11-20 03:00:27,263::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4c94058b-0d6f-4120-8118-2c20463d790e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913275::DEBUG::2013-11-20 03:00:27,263::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913275::DEBUG::2013-11-20 03:00:27,264::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913275::DEBUG::2013-11-20 03:00:27,265::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4c94058b-0d6f-4120-8118-2c20463d790e`::Granted request
>Thread-15913275::DEBUG::2013-11-20 03:00:27,266::task::495::TaskManager.Task::(_debug) Task 87125b1b-4468-4e16-8283-8eb40014c195: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913275::DEBUG::2013-11-20 03:00:27,266::task::495::TaskManager.Task::(_debug) Task 87125b1b-4468-4e16-8283-8eb40014c195: ref 1 aborting False
>Thread-15913275::WARNING::2013-11-20 03:00:27,268::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913275::DEBUG::2013-11-20 03:00:27,269::task::495::TaskManager.Task::(_debug) Task 87125b1b-4468-4e16-8283-8eb40014c195: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913275::DEBUG::2013-11-20 03:00:27,270::task::495::TaskManager.Task::(_debug) Task 87125b1b-4468-4e16-8283-8eb40014c195: moving from state preparing -> state finished
>Thread-15913275::DEBUG::2013-11-20 03:00:27,270::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913275::DEBUG::2013-11-20 03:00:27,271::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913275::DEBUG::2013-11-20 03:00:27,272::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913275::DEBUG::2013-11-20 03:00:27,272::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913275::DEBUG::2013-11-20 03:00:27,273::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913275::DEBUG::2013-11-20 03:00:27,273::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913275::DEBUG::2013-11-20 03:00:27,274::task::495::TaskManager.Task::(_debug) Task 87125b1b-4468-4e16-8283-8eb40014c195: ref 0 aborting False
>Thread-15913275::INFO::2013-11-20 03:00:27,275::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:00:27,613::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:00:28,146::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0749663 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15913276::INFO::2013-11-20 03:00:30,019::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913276::DEBUG::2013-11-20 03:00:30,020::task::495::TaskManager.Task::(_debug) Task c4e94885-fb2b-4482-8922-4e676d7ed6e0: moving from state init -> state preparing
>Thread-15913276::DEBUG::2013-11-20 03:00:30,021::task::495::TaskManager.Task::(_debug) Task c4e94885-fb2b-4482-8922-4e676d7ed6e0: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000576972961426', 'lastCheck': 1384916427.8101799, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0118510723114', 'lastCheck': 1384916425.7849591, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0216488838196', 'lastCheck': 1384916426.796051, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000890970230103', 'lastCheck': 1384916425.7697649, 'valid': True, 'code': 0}}
>Thread-15913276::DEBUG::2013-11-20 03:00:30,022::task::495::TaskManager.Task::(_debug) Task c4e94885-fb2b-4482-8922-4e676d7ed6e0: moving from state preparing -> state finished
>Thread-15913276::DEBUG::2013-11-20 03:00:30,022::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913276::DEBUG::2013-11-20 03:00:30,023::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913276::DEBUG::2013-11-20 03:00:30,023::task::495::TaskManager.Task::(_debug) Task c4e94885-fb2b-4482-8922-4e676d7ed6e0: ref 0 aborting False
>Thread-15913276::INFO::2013-11-20 03:00:30,024::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000576972961426', 'lastCheck': 1384916427.8101799, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0118510723114', 'lastCheck': 1384916425.7849591, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0216488838196', 'lastCheck': 1384916426.796051, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000890970230103', 'lastCheck': 1384916425.7697649, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:00:30,161::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:00:30,637::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0797865 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:00:32,651::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:00:33,198::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0713212 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:00:35,214::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:00:35,738::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0820803 s, 12.5 MB/s\n'; <rc> = 0
>Thread-15913280::DEBUG::2013-11-20 03:00:37,443::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913280::INFO::2013-11-20 03:00:37,447::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913280::DEBUG::2013-11-20 03:00:37,450::task::495::TaskManager.Task::(_debug) Task 63a06972-1d59-4224-ad14-ddb04d68165b: moving from state init -> state preparing
>Thread-15913280::DEBUG::2013-11-20 03:00:37,453::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913280::DEBUG::2013-11-20 03:00:37,455::task::495::TaskManager.Task::(_debug) Task 63a06972-1d59-4224-ad14-ddb04d68165b: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913280::DEBUG::2013-11-20 03:00:37,457::task::495::TaskManager.Task::(_debug) Task 63a06972-1d59-4224-ad14-ddb04d68165b: moving from state preparing -> state finished
>Thread-15913280::DEBUG::2013-11-20 03:00:37,458::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913280::DEBUG::2013-11-20 03:00:37,459::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913280::DEBUG::2013-11-20 03:00:37,459::task::495::TaskManager.Task::(_debug) Task 63a06972-1d59-4224-ad14-ddb04d68165b: ref 0 aborting False
>Thread-15913280::INFO::2013-11-20 03:00:37,460::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913281::DEBUG::2013-11-20 03:00:37,464::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913281::INFO::2013-11-20 03:00:37,465::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913281::DEBUG::2013-11-20 03:00:37,466::task::495::TaskManager.Task::(_debug) Task 4b6eb0e7-ad5f-41ea-af15-d2bc8e7a528d: moving from state init -> state preparing
>Thread-15913281::DEBUG::2013-11-20 03:00:37,467::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`471f2c6c-6da5-4239-b68c-14ec07173be8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913281::DEBUG::2013-11-20 03:00:37,467::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913281::DEBUG::2013-11-20 03:00:37,468::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913281::DEBUG::2013-11-20 03:00:37,469::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`471f2c6c-6da5-4239-b68c-14ec07173be8`::Granted request
>Thread-15913281::DEBUG::2013-11-20 03:00:37,470::task::495::TaskManager.Task::(_debug) Task 4b6eb0e7-ad5f-41ea-af15-d2bc8e7a528d: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913281::DEBUG::2013-11-20 03:00:37,470::task::495::TaskManager.Task::(_debug) Task 4b6eb0e7-ad5f-41ea-af15-d2bc8e7a528d: ref 1 aborting False
>Thread-15913281::WARNING::2013-11-20 03:00:37,472::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913281::DEBUG::2013-11-20 03:00:37,473::task::495::TaskManager.Task::(_debug) Task 4b6eb0e7-ad5f-41ea-af15-d2bc8e7a528d: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913281::DEBUG::2013-11-20 03:00:37,474::task::495::TaskManager.Task::(_debug) Task 4b6eb0e7-ad5f-41ea-af15-d2bc8e7a528d: moving from state preparing -> state finished
>Thread-15913281::DEBUG::2013-11-20 03:00:37,474::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913281::DEBUG::2013-11-20 03:00:37,475::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913281::DEBUG::2013-11-20 03:00:37,476::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913281::DEBUG::2013-11-20 03:00:37,477::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913281::DEBUG::2013-11-20 03:00:37,477::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913281::DEBUG::2013-11-20 03:00:37,478::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913281::DEBUG::2013-11-20 03:00:37,478::task::495::TaskManager.Task::(_debug) Task 4b6eb0e7-ad5f-41ea-af15-d2bc8e7a528d: ref 0 aborting False
>Thread-15913281::INFO::2013-11-20 03:00:37,479::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:00:37,753::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:00:38,306::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0791031 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:00:40,322::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:00:40,847::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0761559 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:00:42,859::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:00:43,412::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0745475 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15913284::INFO::2013-11-20 03:00:45,377::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913284::DEBUG::2013-11-20 03:00:45,378::task::495::TaskManager.Task::(_debug) Task 3798064d-2353-4346-a5ce-cb8bf3dcaed3: moving from state init -> state preparing
>Thread-15913284::DEBUG::2013-11-20 03:00:45,387::task::495::TaskManager.Task::(_debug) Task 3798064d-2353-4346-a5ce-cb8bf3dcaed3: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000326156616211', 'lastCheck': 1384916438.023623, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00154089927673', 'lastCheck': 1384916435.8853381, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00140810012817', 'lastCheck': 1384916436.884948, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0024631023407', 'lastCheck': 1384916435.882122, 'valid': True, 'code': 0}}
>Thread-15913284::DEBUG::2013-11-20 03:00:45,388::task::495::TaskManager.Task::(_debug) Task 3798064d-2353-4346-a5ce-cb8bf3dcaed3: moving from state preparing -> state finished
>Thread-15913284::DEBUG::2013-11-20 03:00:45,389::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913284::DEBUG::2013-11-20 03:00:45,389::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913284::DEBUG::2013-11-20 03:00:45,390::task::495::TaskManager.Task::(_debug) Task 3798064d-2353-4346-a5ce-cb8bf3dcaed3: ref 0 aborting False
>Thread-15913284::INFO::2013-11-20 03:00:45,391::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000326156616211', 'lastCheck': 1384916438.023623, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00154089927673', 'lastCheck': 1384916435.8853381, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00140810012817', 'lastCheck': 1384916436.884948, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0024631023407', 'lastCheck': 1384916435.882122, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:00:45,426::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:00:45,954::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0706525 s, 14.5 MB/s\n'; <rc> = 0
>Thread-15913286::DEBUG::2013-11-20 03:00:47,641::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913286::INFO::2013-11-20 03:00:47,643::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913286::DEBUG::2013-11-20 03:00:47,643::task::495::TaskManager.Task::(_debug) Task d0b336bd-0cf5-40d6-8ab3-90d557580abf: moving from state init -> state preparing
>Thread-15913286::DEBUG::2013-11-20 03:00:47,644::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913286::DEBUG::2013-11-20 03:00:47,645::task::495::TaskManager.Task::(_debug) Task d0b336bd-0cf5-40d6-8ab3-90d557580abf: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913286::DEBUG::2013-11-20 03:00:47,645::task::495::TaskManager.Task::(_debug) Task d0b336bd-0cf5-40d6-8ab3-90d557580abf: moving from state preparing -> state finished
>Thread-15913286::DEBUG::2013-11-20 03:00:47,646::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913286::DEBUG::2013-11-20 03:00:47,646::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913286::DEBUG::2013-11-20 03:00:47,647::task::495::TaskManager.Task::(_debug) Task d0b336bd-0cf5-40d6-8ab3-90d557580abf: ref 0 aborting False
>Thread-15913286::INFO::2013-11-20 03:00:47,648::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913287::DEBUG::2013-11-20 03:00:47,652::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913287::INFO::2013-11-20 03:00:47,653::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913287::DEBUG::2013-11-20 03:00:47,654::task::495::TaskManager.Task::(_debug) Task 548df7e3-16b7-4cfe-bafa-8fc9deb662df: moving from state init -> state preparing
>Thread-15913287::DEBUG::2013-11-20 03:00:47,655::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1ee2889c-f199-4c82-9fed-111da68a3cd4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913287::DEBUG::2013-11-20 03:00:47,655::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913287::DEBUG::2013-11-20 03:00:47,656::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913287::DEBUG::2013-11-20 03:00:47,657::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1ee2889c-f199-4c82-9fed-111da68a3cd4`::Granted request
>Thread-15913287::DEBUG::2013-11-20 03:00:47,658::task::495::TaskManager.Task::(_debug) Task 548df7e3-16b7-4cfe-bafa-8fc9deb662df: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913287::DEBUG::2013-11-20 03:00:47,659::task::495::TaskManager.Task::(_debug) Task 548df7e3-16b7-4cfe-bafa-8fc9deb662df: ref 1 aborting False
>Thread-15913287::WARNING::2013-11-20 03:00:47,661::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913287::DEBUG::2013-11-20 03:00:47,662::task::495::TaskManager.Task::(_debug) Task 548df7e3-16b7-4cfe-bafa-8fc9deb662df: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913287::DEBUG::2013-11-20 03:00:47,662::task::495::TaskManager.Task::(_debug) Task 548df7e3-16b7-4cfe-bafa-8fc9deb662df: moving from state preparing -> state finished
>Thread-15913287::DEBUG::2013-11-20 03:00:47,663::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913287::DEBUG::2013-11-20 03:00:47,664::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913287::DEBUG::2013-11-20 03:00:47,665::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913287::DEBUG::2013-11-20 03:00:47,666::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913287::DEBUG::2013-11-20 03:00:47,667::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913287::DEBUG::2013-11-20 03:00:47,668::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913287::DEBUG::2013-11-20 03:00:47,669::task::495::TaskManager.Task::(_debug) Task 548df7e3-16b7-4cfe-bafa-8fc9deb662df: ref 0 aborting False
>Thread-15913287::INFO::2013-11-20 03:00:47,670::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:00:47,969::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:00:48,479::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0723366 s, 14.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:00:50,495::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:00:51,003::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.065653 s, 15.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:00:53,019::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:00:53,571::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0736796 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:00:55,586::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:00:56,116::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0867857 s, 11.8 MB/s\n'; <rc> = 0
>Thread-15913291::DEBUG::2013-11-20 03:00:57,834::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913291::INFO::2013-11-20 03:00:57,836::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913291::DEBUG::2013-11-20 03:00:57,837::task::495::TaskManager.Task::(_debug) Task a0a510ba-2eca-4dee-a5a1-3fac9938c37f: moving from state init -> state preparing
>Thread-15913291::DEBUG::2013-11-20 03:00:57,837::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913291::DEBUG::2013-11-20 03:00:57,838::task::495::TaskManager.Task::(_debug) Task a0a510ba-2eca-4dee-a5a1-3fac9938c37f: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913291::DEBUG::2013-11-20 03:00:57,839::task::495::TaskManager.Task::(_debug) Task a0a510ba-2eca-4dee-a5a1-3fac9938c37f: moving from state preparing -> state finished
>Thread-15913291::DEBUG::2013-11-20 03:00:57,839::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913291::DEBUG::2013-11-20 03:00:57,840::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913291::DEBUG::2013-11-20 03:00:57,840::task::495::TaskManager.Task::(_debug) Task a0a510ba-2eca-4dee-a5a1-3fac9938c37f: ref 0 aborting False
>Thread-15913291::INFO::2013-11-20 03:00:57,841::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913292::DEBUG::2013-11-20 03:00:57,846::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913292::INFO::2013-11-20 03:00:57,847::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913292::DEBUG::2013-11-20 03:00:57,848::task::495::TaskManager.Task::(_debug) Task 62974978-27bc-4f63-a96e-e2f93759c206: moving from state init -> state preparing
>Thread-15913292::DEBUG::2013-11-20 03:00:57,850::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`41a2d9db-8cd1-4c80-bb10-9fcdaa4897d4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913292::DEBUG::2013-11-20 03:00:57,851::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913292::DEBUG::2013-11-20 03:00:57,852::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913292::DEBUG::2013-11-20 03:00:57,852::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`41a2d9db-8cd1-4c80-bb10-9fcdaa4897d4`::Granted request
>Thread-15913292::DEBUG::2013-11-20 03:00:57,853::task::495::TaskManager.Task::(_debug) Task 62974978-27bc-4f63-a96e-e2f93759c206: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913292::DEBUG::2013-11-20 03:00:57,854::task::495::TaskManager.Task::(_debug) Task 62974978-27bc-4f63-a96e-e2f93759c206: ref 1 aborting False
>Thread-15913292::WARNING::2013-11-20 03:00:57,856::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913292::DEBUG::2013-11-20 03:00:57,857::task::495::TaskManager.Task::(_debug) Task 62974978-27bc-4f63-a96e-e2f93759c206: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913292::DEBUG::2013-11-20 03:00:57,857::task::495::TaskManager.Task::(_debug) Task 62974978-27bc-4f63-a96e-e2f93759c206: moving from state preparing -> state finished
>Thread-15913292::DEBUG::2013-11-20 03:00:57,858::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913292::DEBUG::2013-11-20 03:00:57,859::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913292::DEBUG::2013-11-20 03:00:57,859::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913292::DEBUG::2013-11-20 03:00:57,859::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913292::DEBUG::2013-11-20 03:00:57,860::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913292::DEBUG::2013-11-20 03:00:57,860::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913292::DEBUG::2013-11-20 03:00:57,860::task::495::TaskManager.Task::(_debug) Task 62974978-27bc-4f63-a96e-e2f93759c206: ref 0 aborting False
>Thread-15913292::INFO::2013-11-20 03:00:57,861::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:00:58,131::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:00:58,682::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0808616 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:01:00,697::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913294::INFO::2013-11-20 03:01:00,916::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913294::DEBUG::2013-11-20 03:01:00,917::task::495::TaskManager.Task::(_debug) Task f3580c5c-90ec-418e-9456-c735ef8b9de8: moving from state init -> state preparing
>Thread-15913294::DEBUG::2013-11-20 03:01:00,918::task::495::TaskManager.Task::(_debug) Task f3580c5c-90ec-418e-9456-c735ef8b9de8: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00044584274292', 'lastCheck': 1384916458.365896, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00327491760254', 'lastCheck': 1384916456.270303, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00177979469299', 'lastCheck': 1384916457.269845, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00108098983765', 'lastCheck': 1384916456.268971, 'valid': True, 'code': 0}}
>Thread-15913294::DEBUG::2013-11-20 03:01:00,919::task::495::TaskManager.Task::(_debug) Task f3580c5c-90ec-418e-9456-c735ef8b9de8: moving from state preparing -> state finished
>Thread-15913294::DEBUG::2013-11-20 03:01:00,919::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913294::DEBUG::2013-11-20 03:01:00,920::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913294::DEBUG::2013-11-20 03:01:00,921::task::495::TaskManager.Task::(_debug) Task f3580c5c-90ec-418e-9456-c735ef8b9de8: ref 0 aborting False
>Thread-15913294::INFO::2013-11-20 03:01:00,921::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00044584274292', 'lastCheck': 1384916458.365896, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00327491760254', 'lastCheck': 1384916456.270303, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00177979469299', 'lastCheck': 1384916457.269845, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00108098983765', 'lastCheck': 1384916456.268971, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:01:01,230::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0858177 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:01:03,244::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:01:03,773::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.090755 s, 11.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:01:05,788::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:01:06,334::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0742484 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15913298::DEBUG::2013-11-20 03:01:08,037::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913298::INFO::2013-11-20 03:01:08,039::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913298::DEBUG::2013-11-20 03:01:08,039::task::495::TaskManager.Task::(_debug) Task 51575fa4-9d42-4d4c-bbfb-0bc96c9dd66f: moving from state init -> state preparing
>Thread-15913298::DEBUG::2013-11-20 03:01:08,040::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913298::DEBUG::2013-11-20 03:01:08,041::task::495::TaskManager.Task::(_debug) Task 51575fa4-9d42-4d4c-bbfb-0bc96c9dd66f: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913298::DEBUG::2013-11-20 03:01:08,041::task::495::TaskManager.Task::(_debug) Task 51575fa4-9d42-4d4c-bbfb-0bc96c9dd66f: moving from state preparing -> state finished
>Thread-15913298::DEBUG::2013-11-20 03:01:08,042::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913298::DEBUG::2013-11-20 03:01:08,042::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913298::DEBUG::2013-11-20 03:01:08,043::task::495::TaskManager.Task::(_debug) Task 51575fa4-9d42-4d4c-bbfb-0bc96c9dd66f: ref 0 aborting False
>Thread-15913298::INFO::2013-11-20 03:01:08,044::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913299::DEBUG::2013-11-20 03:01:08,048::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913299::INFO::2013-11-20 03:01:08,049::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913299::DEBUG::2013-11-20 03:01:08,050::task::495::TaskManager.Task::(_debug) Task c1f61cd8-70e1-4401-b993-a2e90624ff9d: moving from state init -> state preparing
>Thread-15913299::DEBUG::2013-11-20 03:01:08,051::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`034a566a-76da-41e4-8028-cbe2474db773`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913299::DEBUG::2013-11-20 03:01:08,051::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913299::DEBUG::2013-11-20 03:01:08,052::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913299::DEBUG::2013-11-20 03:01:08,053::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`034a566a-76da-41e4-8028-cbe2474db773`::Granted request
>Thread-15913299::DEBUG::2013-11-20 03:01:08,054::task::495::TaskManager.Task::(_debug) Task c1f61cd8-70e1-4401-b993-a2e90624ff9d: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913299::DEBUG::2013-11-20 03:01:08,054::task::495::TaskManager.Task::(_debug) Task c1f61cd8-70e1-4401-b993-a2e90624ff9d: ref 1 aborting False
>Thread-15913299::WARNING::2013-11-20 03:01:08,056::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913299::DEBUG::2013-11-20 03:01:08,057::task::495::TaskManager.Task::(_debug) Task c1f61cd8-70e1-4401-b993-a2e90624ff9d: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913299::DEBUG::2013-11-20 03:01:08,058::task::495::TaskManager.Task::(_debug) Task c1f61cd8-70e1-4401-b993-a2e90624ff9d: moving from state preparing -> state finished
>Thread-15913299::DEBUG::2013-11-20 03:01:08,059::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913299::DEBUG::2013-11-20 03:01:08,059::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913299::DEBUG::2013-11-20 03:01:08,060::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913299::DEBUG::2013-11-20 03:01:08,060::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913299::DEBUG::2013-11-20 03:01:08,061::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913299::DEBUG::2013-11-20 03:01:08,061::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913299::DEBUG::2013-11-20 03:01:08,062::task::495::TaskManager.Task::(_debug) Task c1f61cd8-70e1-4401-b993-a2e90624ff9d: ref 0 aborting False
>Thread-15913299::INFO::2013-11-20 03:01:08,063::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:01:08,349::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:01:08,859::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.07359 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:01:10,874::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:01:11,422::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0741545 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:01:13,437::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:01:13,963::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0743286 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:01:15,978::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913302::INFO::2013-11-20 03:01:16,233::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913302::DEBUG::2013-11-20 03:01:16,234::task::495::TaskManager.Task::(_debug) Task 34b05ea9-08cc-429b-ba7d-012603a7435a: moving from state init -> state preparing
>Thread-15913302::DEBUG::2013-11-20 03:01:16,235::task::495::TaskManager.Task::(_debug) Task 34b05ea9-08cc-429b-ba7d-012603a7435a: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000546932220459', 'lastCheck': 1384916468.555757, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142288208008', 'lastCheck': 1384916466.468246, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0014660358429', 'lastCheck': 1384916467.468725, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000311136245728', 'lastCheck': 1384916466.463021, 'valid': True, 'code': 0}}
>Thread-15913302::DEBUG::2013-11-20 03:01:16,236::task::495::TaskManager.Task::(_debug) Task 34b05ea9-08cc-429b-ba7d-012603a7435a: moving from state preparing -> state finished
>Thread-15913302::DEBUG::2013-11-20 03:01:16,236::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913302::DEBUG::2013-11-20 03:01:16,236::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913302::DEBUG::2013-11-20 03:01:16,237::task::495::TaskManager.Task::(_debug) Task 34b05ea9-08cc-429b-ba7d-012603a7435a: ref 0 aborting False
>Thread-15913302::INFO::2013-11-20 03:01:16,237::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000546932220459', 'lastCheck': 1384916468.555757, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142288208008', 'lastCheck': 1384916466.468246, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0014660358429', 'lastCheck': 1384916467.468725, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000311136245728', 'lastCheck': 1384916466.463021, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:01:16,477::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0724543 s, 14.1 MB/s\n'; <rc> = 0
>Thread-15913304::DEBUG::2013-11-20 03:01:18,239::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913304::INFO::2013-11-20 03:01:18,240::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913304::DEBUG::2013-11-20 03:01:18,240::task::495::TaskManager.Task::(_debug) Task f20796bc-530e-4795-ad85-fe5bccee819c: moving from state init -> state preparing
>Thread-15913304::DEBUG::2013-11-20 03:01:18,241::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913304::DEBUG::2013-11-20 03:01:18,242::task::495::TaskManager.Task::(_debug) Task f20796bc-530e-4795-ad85-fe5bccee819c: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913304::DEBUG::2013-11-20 03:01:18,242::task::495::TaskManager.Task::(_debug) Task f20796bc-530e-4795-ad85-fe5bccee819c: moving from state preparing -> state finished
>Thread-15913304::DEBUG::2013-11-20 03:01:18,243::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913304::DEBUG::2013-11-20 03:01:18,243::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913304::DEBUG::2013-11-20 03:01:18,244::task::495::TaskManager.Task::(_debug) Task f20796bc-530e-4795-ad85-fe5bccee819c: ref 0 aborting False
>Thread-15913304::INFO::2013-11-20 03:01:18,245::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913305::DEBUG::2013-11-20 03:01:18,249::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913305::INFO::2013-11-20 03:01:18,250::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913305::DEBUG::2013-11-20 03:01:18,250::task::495::TaskManager.Task::(_debug) Task 86d62929-c9ce-4472-9e77-ad90ab530cb9: moving from state init -> state preparing
>Thread-15913305::DEBUG::2013-11-20 03:01:18,251::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`203cb164-b15e-4a42-8720-a47729b46785`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913305::DEBUG::2013-11-20 03:01:18,252::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913305::DEBUG::2013-11-20 03:01:18,253::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913305::DEBUG::2013-11-20 03:01:18,253::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`203cb164-b15e-4a42-8720-a47729b46785`::Granted request
>Thread-15913305::DEBUG::2013-11-20 03:01:18,254::task::495::TaskManager.Task::(_debug) Task 86d62929-c9ce-4472-9e77-ad90ab530cb9: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913305::DEBUG::2013-11-20 03:01:18,255::task::495::TaskManager.Task::(_debug) Task 86d62929-c9ce-4472-9e77-ad90ab530cb9: ref 1 aborting False
>Thread-15913305::WARNING::2013-11-20 03:01:18,257::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913305::DEBUG::2013-11-20 03:01:18,258::task::495::TaskManager.Task::(_debug) Task 86d62929-c9ce-4472-9e77-ad90ab530cb9: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913305::DEBUG::2013-11-20 03:01:18,259::task::495::TaskManager.Task::(_debug) Task 86d62929-c9ce-4472-9e77-ad90ab530cb9: moving from state preparing -> state finished
>Thread-15913305::DEBUG::2013-11-20 03:01:18,259::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913305::DEBUG::2013-11-20 03:01:18,260::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913305::DEBUG::2013-11-20 03:01:18,261::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913305::DEBUG::2013-11-20 03:01:18,261::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913305::DEBUG::2013-11-20 03:01:18,262::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913305::DEBUG::2013-11-20 03:01:18,262::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913305::DEBUG::2013-11-20 03:01:18,263::task::495::TaskManager.Task::(_debug) Task 86d62929-c9ce-4472-9e77-ad90ab530cb9: ref 0 aborting False
>Thread-15913305::INFO::2013-11-20 03:01:18,264::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:01:18,491::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 03:01:18,748::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:01:18,750::task::495::TaskManager.Task::(_debug) Task 370b38d4-6c5b-4214-8610-787f878ebd43: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:01:18,752::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`29ccad81-cbd0-418e-a77b-4fc4fff830f6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:01:18,753::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:01:18,753::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:01:18,754::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`29ccad81-cbd0-418e-a77b-4fc4fff830f6`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:01:18,755::task::495::TaskManager.Task::(_debug) Task 370b38d4-6c5b-4214-8610-787f878ebd43: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:01:18,756::task::495::TaskManager.Task::(_debug) Task 370b38d4-6c5b-4214-8610-787f878ebd43: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:01:18,756::task::495::TaskManager.Task::(_debug) Task 370b38d4-6c5b-4214-8610-787f878ebd43: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:01:18,757::task::495::TaskManager.Task::(_debug) Task 370b38d4-6c5b-4214-8610-787f878ebd43: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:01:18,758::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:01:18,758::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:01:18,759::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:01:18,759::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:01:18,760::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:01:18,760::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:01:18,761::task::495::TaskManager.Task::(_debug) Task 370b38d4-6c5b-4214-8610-787f878ebd43: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:01:18,762::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:01:19,015::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0739352 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:01:21,030::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:01:21,484::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0458684 s, 22.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:01:23,495::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:01:24,047::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0743449 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:01:26,062::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:01:26,586::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.073581 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15913309::DEBUG::2013-11-20 03:01:28,412::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913309::INFO::2013-11-20 03:01:28,413::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913309::DEBUG::2013-11-20 03:01:28,413::task::495::TaskManager.Task::(_debug) Task 988bb947-a53a-4536-ab31-ccff7b049521: moving from state init -> state preparing
>Thread-15913309::DEBUG::2013-11-20 03:01:28,414::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913309::DEBUG::2013-11-20 03:01:28,414::task::495::TaskManager.Task::(_debug) Task 988bb947-a53a-4536-ab31-ccff7b049521: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913309::DEBUG::2013-11-20 03:01:28,414::task::495::TaskManager.Task::(_debug) Task 988bb947-a53a-4536-ab31-ccff7b049521: moving from state preparing -> state finished
>Thread-15913309::DEBUG::2013-11-20 03:01:28,415::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913309::DEBUG::2013-11-20 03:01:28,415::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913309::DEBUG::2013-11-20 03:01:28,415::task::495::TaskManager.Task::(_debug) Task 988bb947-a53a-4536-ab31-ccff7b049521: ref 0 aborting False
>Thread-15913309::INFO::2013-11-20 03:01:28,416::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913310::DEBUG::2013-11-20 03:01:28,420::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913310::INFO::2013-11-20 03:01:28,420::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913310::DEBUG::2013-11-20 03:01:28,421::task::495::TaskManager.Task::(_debug) Task 2a32a0ba-90a9-4654-9b57-5038c2dfc074: moving from state init -> state preparing
>Thread-15913310::DEBUG::2013-11-20 03:01:28,422::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bc61e3b1-af72-422e-a273-46520a0924c2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913310::DEBUG::2013-11-20 03:01:28,422::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913310::DEBUG::2013-11-20 03:01:28,422::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913310::DEBUG::2013-11-20 03:01:28,423::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bc61e3b1-af72-422e-a273-46520a0924c2`::Granted request
>Thread-15913310::DEBUG::2013-11-20 03:01:28,424::task::495::TaskManager.Task::(_debug) Task 2a32a0ba-90a9-4654-9b57-5038c2dfc074: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913310::DEBUG::2013-11-20 03:01:28,424::task::495::TaskManager.Task::(_debug) Task 2a32a0ba-90a9-4654-9b57-5038c2dfc074: ref 1 aborting False
>Thread-15913310::WARNING::2013-11-20 03:01:28,425::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913310::DEBUG::2013-11-20 03:01:28,426::task::495::TaskManager.Task::(_debug) Task 2a32a0ba-90a9-4654-9b57-5038c2dfc074: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913310::DEBUG::2013-11-20 03:01:28,426::task::495::TaskManager.Task::(_debug) Task 2a32a0ba-90a9-4654-9b57-5038c2dfc074: moving from state preparing -> state finished
>Thread-15913310::DEBUG::2013-11-20 03:01:28,427::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913310::DEBUG::2013-11-20 03:01:28,427::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913310::DEBUG::2013-11-20 03:01:28,427::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913310::DEBUG::2013-11-20 03:01:28,428::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913310::DEBUG::2013-11-20 03:01:28,428::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913310::DEBUG::2013-11-20 03:01:28,429::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913310::DEBUG::2013-11-20 03:01:28,429::task::495::TaskManager.Task::(_debug) Task 2a32a0ba-90a9-4654-9b57-5038c2dfc074: ref 0 aborting False
>Thread-15913310::INFO::2013-11-20 03:01:28,429::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:01:28,601::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-42838::DEBUG::2013-11-20 03:01:28,813::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:01:29,221::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.130417 s, 7.9 MB/s\n'; <rc> = 0
>Thread-42838::DEBUG::2013-11-20 03:01:29,669::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:01:31,236::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913312::INFO::2013-11-20 03:01:31,567::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913312::DEBUG::2013-11-20 03:01:31,568::task::495::TaskManager.Task::(_debug) Task c263956b-ccee-4036-8050-0796f32433ec: moving from state init -> state preparing
>Thread-15913312::DEBUG::2013-11-20 03:01:31,569::task::495::TaskManager.Task::(_debug) Task c263956b-ccee-4036-8050-0796f32433ec: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000324010848999', 'lastCheck': 1384916489.6738529, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0138931274414', 'lastCheck': 1384916486.7516749, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0014271736145', 'lastCheck': 1384916487.7431469, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00140190124512', 'lastCheck': 1384916486.741118, 'valid': True, 'code': 0}}
>Thread-15913312::DEBUG::2013-11-20 03:01:31,570::task::495::TaskManager.Task::(_debug) Task c263956b-ccee-4036-8050-0796f32433ec: moving from state preparing -> state finished
>Thread-15913312::DEBUG::2013-11-20 03:01:31,571::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913312::DEBUG::2013-11-20 03:01:31,571::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913312::DEBUG::2013-11-20 03:01:31,572::task::495::TaskManager.Task::(_debug) Task c263956b-ccee-4036-8050-0796f32433ec: ref 0 aborting False
>Thread-15913312::INFO::2013-11-20 03:01:31,573::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000324010848999', 'lastCheck': 1384916489.6738529, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0138931274414', 'lastCheck': 1384916486.7516749, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0014271736145', 'lastCheck': 1384916487.7431469, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00140190124512', 'lastCheck': 1384916486.741118, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:01:31,778::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0778191 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:01:33,792::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:01:34,328::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0828243 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:01:36,350::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:01:36,896::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0765621 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15913316::DEBUG::2013-11-20 03:01:38,599::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913316::INFO::2013-11-20 03:01:38,601::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913316::DEBUG::2013-11-20 03:01:38,602::task::495::TaskManager.Task::(_debug) Task e5665f26-f7ad-4f5c-bf6b-b25fa43a78c1: moving from state init -> state preparing
>Thread-15913316::DEBUG::2013-11-20 03:01:38,603::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913316::DEBUG::2013-11-20 03:01:38,603::task::495::TaskManager.Task::(_debug) Task e5665f26-f7ad-4f5c-bf6b-b25fa43a78c1: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913316::DEBUG::2013-11-20 03:01:38,604::task::495::TaskManager.Task::(_debug) Task e5665f26-f7ad-4f5c-bf6b-b25fa43a78c1: moving from state preparing -> state finished
>Thread-15913316::DEBUG::2013-11-20 03:01:38,604::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913316::DEBUG::2013-11-20 03:01:38,605::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913316::DEBUG::2013-11-20 03:01:38,606::task::495::TaskManager.Task::(_debug) Task e5665f26-f7ad-4f5c-bf6b-b25fa43a78c1: ref 0 aborting False
>Thread-15913316::INFO::2013-11-20 03:01:38,606::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913317::DEBUG::2013-11-20 03:01:38,611::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913317::INFO::2013-11-20 03:01:38,612::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913317::DEBUG::2013-11-20 03:01:38,613::task::495::TaskManager.Task::(_debug) Task 5cf2116b-bf15-4a13-a3c5-e271ed39bdbb: moving from state init -> state preparing
>Thread-15913317::DEBUG::2013-11-20 03:01:38,615::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bf9c529c-2835-423e-a96e-7b8f9317d12a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913317::DEBUG::2013-11-20 03:01:38,615::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913317::DEBUG::2013-11-20 03:01:38,616::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913317::DEBUG::2013-11-20 03:01:38,617::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bf9c529c-2835-423e-a96e-7b8f9317d12a`::Granted request
>Thread-15913317::DEBUG::2013-11-20 03:01:38,618::task::495::TaskManager.Task::(_debug) Task 5cf2116b-bf15-4a13-a3c5-e271ed39bdbb: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913317::DEBUG::2013-11-20 03:01:38,618::task::495::TaskManager.Task::(_debug) Task 5cf2116b-bf15-4a13-a3c5-e271ed39bdbb: ref 1 aborting False
>Thread-15913317::WARNING::2013-11-20 03:01:38,620::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913317::DEBUG::2013-11-20 03:01:38,621::task::495::TaskManager.Task::(_debug) Task 5cf2116b-bf15-4a13-a3c5-e271ed39bdbb: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913317::DEBUG::2013-11-20 03:01:38,622::task::495::TaskManager.Task::(_debug) Task 5cf2116b-bf15-4a13-a3c5-e271ed39bdbb: moving from state preparing -> state finished
>Thread-15913317::DEBUG::2013-11-20 03:01:38,623::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913317::DEBUG::2013-11-20 03:01:38,623::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913317::DEBUG::2013-11-20 03:01:38,624::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913317::DEBUG::2013-11-20 03:01:38,625::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913317::DEBUG::2013-11-20 03:01:38,625::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913317::DEBUG::2013-11-20 03:01:38,625::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913317::DEBUG::2013-11-20 03:01:38,626::task::495::TaskManager.Task::(_debug) Task 5cf2116b-bf15-4a13-a3c5-e271ed39bdbb: ref 0 aborting False
>Thread-15913317::INFO::2013-11-20 03:01:38,627::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:01:38,913::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:01:39,413::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0521656 s, 19.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:01:41,424::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:01:41,983::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0786517 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:01:43,997::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:01:44,550::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0708651 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:01:46,566::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:01:47,103::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0734173 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15913320::INFO::2013-11-20 03:01:47,254::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913320::DEBUG::2013-11-20 03:01:47,254::task::495::TaskManager.Task::(_debug) Task 88fa7de7-6bc2-4817-911c-389dd459150d: moving from state init -> state preparing
>Thread-15913320::DEBUG::2013-11-20 03:01:47,255::task::495::TaskManager.Task::(_debug) Task 88fa7de7-6bc2-4817-911c-389dd459150d: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000402927398682', 'lastCheck': 1384916499.687809, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00165915489197', 'lastCheck': 1384916507.2403829, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00153803825378', 'lastCheck': 1384916497.997853, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00156402587891', 'lastCheck': 1384916507.236151, 'valid': True, 'code': 0}}
>Thread-15913320::DEBUG::2013-11-20 03:01:47,256::task::495::TaskManager.Task::(_debug) Task 88fa7de7-6bc2-4817-911c-389dd459150d: moving from state preparing -> state finished
>Thread-15913320::DEBUG::2013-11-20 03:01:47,257::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913320::DEBUG::2013-11-20 03:01:47,257::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913320::DEBUG::2013-11-20 03:01:47,258::task::495::TaskManager.Task::(_debug) Task 88fa7de7-6bc2-4817-911c-389dd459150d: ref 0 aborting False
>Thread-15913320::INFO::2013-11-20 03:01:47,258::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000402927398682', 'lastCheck': 1384916499.687809, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00165915489197', 'lastCheck': 1384916507.2403829, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00153803825378', 'lastCheck': 1384916497.997853, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00156402587891', 'lastCheck': 1384916507.236151, 'valid': True, 'code': 0}}
>Thread-15913322::DEBUG::2013-11-20 03:01:48,802::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913322::INFO::2013-11-20 03:01:48,803::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913322::DEBUG::2013-11-20 03:01:48,804::task::495::TaskManager.Task::(_debug) Task 8bae6e04-23e2-4a4e-9125-093111ed40fd: moving from state init -> state preparing
>Thread-15913322::DEBUG::2013-11-20 03:01:48,805::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913322::DEBUG::2013-11-20 03:01:48,805::task::495::TaskManager.Task::(_debug) Task 8bae6e04-23e2-4a4e-9125-093111ed40fd: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913322::DEBUG::2013-11-20 03:01:48,806::task::495::TaskManager.Task::(_debug) Task 8bae6e04-23e2-4a4e-9125-093111ed40fd: moving from state preparing -> state finished
>Thread-15913322::DEBUG::2013-11-20 03:01:48,806::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913322::DEBUG::2013-11-20 03:01:48,807::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913322::DEBUG::2013-11-20 03:01:48,807::task::495::TaskManager.Task::(_debug) Task 8bae6e04-23e2-4a4e-9125-093111ed40fd: ref 0 aborting False
>Thread-15913322::INFO::2013-11-20 03:01:48,808::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913323::DEBUG::2013-11-20 03:01:48,855::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913323::INFO::2013-11-20 03:01:48,855::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913323::DEBUG::2013-11-20 03:01:48,856::task::495::TaskManager.Task::(_debug) Task d0ac0b8d-438d-4a66-aee2-37d23a9d21eb: moving from state init -> state preparing
>Thread-15913323::DEBUG::2013-11-20 03:01:48,857::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f537be83-0193-4564-9d01-5ead0b3ad94a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913323::DEBUG::2013-11-20 03:01:48,858::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913323::DEBUG::2013-11-20 03:01:48,859::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913323::DEBUG::2013-11-20 03:01:48,859::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f537be83-0193-4564-9d01-5ead0b3ad94a`::Granted request
>Thread-15913323::DEBUG::2013-11-20 03:01:48,860::task::495::TaskManager.Task::(_debug) Task d0ac0b8d-438d-4a66-aee2-37d23a9d21eb: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913323::DEBUG::2013-11-20 03:01:48,861::task::495::TaskManager.Task::(_debug) Task d0ac0b8d-438d-4a66-aee2-37d23a9d21eb: ref 1 aborting False
>Thread-15913323::WARNING::2013-11-20 03:01:48,862::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913323::DEBUG::2013-11-20 03:01:48,863::task::495::TaskManager.Task::(_debug) Task d0ac0b8d-438d-4a66-aee2-37d23a9d21eb: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913323::DEBUG::2013-11-20 03:01:48,864::task::495::TaskManager.Task::(_debug) Task d0ac0b8d-438d-4a66-aee2-37d23a9d21eb: moving from state preparing -> state finished
>Thread-15913323::DEBUG::2013-11-20 03:01:48,865::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913323::DEBUG::2013-11-20 03:01:48,865::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913323::DEBUG::2013-11-20 03:01:48,866::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913323::DEBUG::2013-11-20 03:01:48,867::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913323::DEBUG::2013-11-20 03:01:48,867::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913323::DEBUG::2013-11-20 03:01:48,868::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913323::DEBUG::2013-11-20 03:01:48,868::task::495::TaskManager.Task::(_debug) Task d0ac0b8d-438d-4a66-aee2-37d23a9d21eb: ref 0 aborting False
>Thread-15913323::INFO::2013-11-20 03:01:48,869::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:01:49,118::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:01:49,642::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0741295 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:01:51,659::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:01:52,096::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0468533 s, 21.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:01:54,106::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:01:54,665::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.116353 s, 8.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:01:56,681::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:01:57,239::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0773553 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15913327::DEBUG::2013-11-20 03:01:59,222::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913327::INFO::2013-11-20 03:01:59,224::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913327::DEBUG::2013-11-20 03:01:59,224::task::495::TaskManager.Task::(_debug) Task 68428e70-c2c9-423b-9a07-6d4d7c11b410: moving from state init -> state preparing
>Thread-15913327::DEBUG::2013-11-20 03:01:59,226::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913327::DEBUG::2013-11-20 03:01:59,226::task::495::TaskManager.Task::(_debug) Task 68428e70-c2c9-423b-9a07-6d4d7c11b410: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913327::DEBUG::2013-11-20 03:01:59,227::task::495::TaskManager.Task::(_debug) Task 68428e70-c2c9-423b-9a07-6d4d7c11b410: moving from state preparing -> state finished
>Thread-15913327::DEBUG::2013-11-20 03:01:59,227::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913327::DEBUG::2013-11-20 03:01:59,228::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913327::DEBUG::2013-11-20 03:01:59,228::task::495::TaskManager.Task::(_debug) Task 68428e70-c2c9-423b-9a07-6d4d7c11b410: ref 0 aborting False
>Thread-15913327::INFO::2013-11-20 03:01:59,229::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913328::DEBUG::2013-11-20 03:01:59,234::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913328::INFO::2013-11-20 03:01:59,235::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913328::DEBUG::2013-11-20 03:01:59,235::task::495::TaskManager.Task::(_debug) Task d3a9e0c5-3929-4187-b246-c80e3b503db5: moving from state init -> state preparing
>Thread-15913328::DEBUG::2013-11-20 03:01:59,237::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`26e31e22-4ff2-4785-a499-ff75b3422507`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913328::DEBUG::2013-11-20 03:01:59,237::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913328::DEBUG::2013-11-20 03:01:59,238::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913328::DEBUG::2013-11-20 03:01:59,238::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`26e31e22-4ff2-4785-a499-ff75b3422507`::Granted request
>Thread-15913328::DEBUG::2013-11-20 03:01:59,239::task::495::TaskManager.Task::(_debug) Task d3a9e0c5-3929-4187-b246-c80e3b503db5: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913328::DEBUG::2013-11-20 03:01:59,240::task::495::TaskManager.Task::(_debug) Task d3a9e0c5-3929-4187-b246-c80e3b503db5: ref 1 aborting False
>Thread-15913328::WARNING::2013-11-20 03:01:59,242::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913328::DEBUG::2013-11-20 03:01:59,243::task::495::TaskManager.Task::(_debug) Task d3a9e0c5-3929-4187-b246-c80e3b503db5: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913328::DEBUG::2013-11-20 03:01:59,244::task::495::TaskManager.Task::(_debug) Task d3a9e0c5-3929-4187-b246-c80e3b503db5: moving from state preparing -> state finished
>Thread-15913328::DEBUG::2013-11-20 03:01:59,244::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913328::DEBUG::2013-11-20 03:01:59,245::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913328::DEBUG::2013-11-20 03:01:59,245::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913328::DEBUG::2013-11-20 03:01:59,246::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913328::DEBUG::2013-11-20 03:01:59,247::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913328::DEBUG::2013-11-20 03:01:59,247::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913328::DEBUG::2013-11-20 03:01:59,248::task::495::TaskManager.Task::(_debug) Task d3a9e0c5-3929-4187-b246-c80e3b503db5: ref 0 aborting False
>Thread-15913328::INFO::2013-11-20 03:01:59,248::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:01:59,254::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:01:59,811::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.073665 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:02:01,826::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:02:02,350::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0732466 s, 14.0 MB/s\n'; <rc> = 0
>Thread-15913330::INFO::2013-11-20 03:02:02,954::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913330::DEBUG::2013-11-20 03:02:02,955::task::495::TaskManager.Task::(_debug) Task f10ba8a0-dd57-4897-98f2-f70833f031c3: moving from state init -> state preparing
>Thread-15913330::DEBUG::2013-11-20 03:02:02,956::task::495::TaskManager.Task::(_debug) Task f10ba8a0-dd57-4897-98f2-f70833f031c3: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00102996826172', 'lastCheck': 1384916519.9193771, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00149202346802', 'lastCheck': 1384916517.3620491, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00170803070068', 'lastCheck': 1384916518.3754909, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000462055206299', 'lastCheck': 1384916517.3563211, 'valid': True, 'code': 0}}
>Thread-15913330::DEBUG::2013-11-20 03:02:02,957::task::495::TaskManager.Task::(_debug) Task f10ba8a0-dd57-4897-98f2-f70833f031c3: moving from state preparing -> state finished
>Thread-15913330::DEBUG::2013-11-20 03:02:02,957::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913330::DEBUG::2013-11-20 03:02:02,958::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913330::DEBUG::2013-11-20 03:02:02,958::task::495::TaskManager.Task::(_debug) Task f10ba8a0-dd57-4897-98f2-f70833f031c3: ref 0 aborting False
>Thread-15913330::INFO::2013-11-20 03:02:02,959::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00102996826172', 'lastCheck': 1384916519.9193771, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00149202346802', 'lastCheck': 1384916517.3620491, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00170803070068', 'lastCheck': 1384916518.3754909, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000462055206299', 'lastCheck': 1384916517.3563211, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:02:04,365::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:02:04,902::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0872997 s, 11.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:02:06,917::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:02:07,471::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0875833 s, 11.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:02:09,486::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913334::DEBUG::2013-11-20 03:02:09,734::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913334::INFO::2013-11-20 03:02:09,745::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913334::DEBUG::2013-11-20 03:02:09,746::task::495::TaskManager.Task::(_debug) Task 7ac88b51-87e0-47e6-a756-e3f541ee5149: moving from state init -> state preparing
>Thread-15913334::DEBUG::2013-11-20 03:02:09,748::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913334::DEBUG::2013-11-20 03:02:09,748::task::495::TaskManager.Task::(_debug) Task 7ac88b51-87e0-47e6-a756-e3f541ee5149: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913334::DEBUG::2013-11-20 03:02:09,749::task::495::TaskManager.Task::(_debug) Task 7ac88b51-87e0-47e6-a756-e3f541ee5149: moving from state preparing -> state finished
>Thread-15913334::DEBUG::2013-11-20 03:02:09,750::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913334::DEBUG::2013-11-20 03:02:09,751::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913334::DEBUG::2013-11-20 03:02:09,752::task::495::TaskManager.Task::(_debug) Task 7ac88b51-87e0-47e6-a756-e3f541ee5149: ref 0 aborting False
>Thread-15913334::INFO::2013-11-20 03:02:09,752::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913335::DEBUG::2013-11-20 03:02:09,758::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913335::INFO::2013-11-20 03:02:09,760::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913335::DEBUG::2013-11-20 03:02:09,760::task::495::TaskManager.Task::(_debug) Task 3e4d6625-cdaa-4ac9-b3f7-37155235a7fa: moving from state init -> state preparing
>Thread-15913335::DEBUG::2013-11-20 03:02:09,762::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e7a25d4f-05f0-4b39-8f5d-b6c20582b5a7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913335::DEBUG::2013-11-20 03:02:09,763::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913335::DEBUG::2013-11-20 03:02:09,764::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913335::DEBUG::2013-11-20 03:02:09,764::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e7a25d4f-05f0-4b39-8f5d-b6c20582b5a7`::Granted request
>Thread-15913335::DEBUG::2013-11-20 03:02:09,765::task::495::TaskManager.Task::(_debug) Task 3e4d6625-cdaa-4ac9-b3f7-37155235a7fa: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913335::DEBUG::2013-11-20 03:02:09,766::task::495::TaskManager.Task::(_debug) Task 3e4d6625-cdaa-4ac9-b3f7-37155235a7fa: ref 1 aborting False
>Thread-15913335::WARNING::2013-11-20 03:02:09,768::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913335::DEBUG::2013-11-20 03:02:09,769::task::495::TaskManager.Task::(_debug) Task 3e4d6625-cdaa-4ac9-b3f7-37155235a7fa: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913335::DEBUG::2013-11-20 03:02:09,770::task::495::TaskManager.Task::(_debug) Task 3e4d6625-cdaa-4ac9-b3f7-37155235a7fa: moving from state preparing -> state finished
>Thread-15913335::DEBUG::2013-11-20 03:02:09,771::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913335::DEBUG::2013-11-20 03:02:09,772::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913335::DEBUG::2013-11-20 03:02:09,772::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913335::DEBUG::2013-11-20 03:02:09,773::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913335::DEBUG::2013-11-20 03:02:09,774::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913335::DEBUG::2013-11-20 03:02:09,774::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913335::DEBUG::2013-11-20 03:02:09,775::task::495::TaskManager.Task::(_debug) Task 3e4d6625-cdaa-4ac9-b3f7-37155235a7fa: ref 0 aborting False
>Thread-15913335::INFO::2013-11-20 03:02:09,776::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:02:10,058::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0767573 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:02:12,074::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:02:12,577::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0624024 s, 16.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:02:14,593::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:02:15,122::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.06921 s, 14.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:02:17,138::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:02:17,665::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0709185 s, 14.4 MB/s\n'; <rc> = 0
>Thread-15913338::INFO::2013-11-20 03:02:18,335::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913338::DEBUG::2013-11-20 03:02:18,336::task::495::TaskManager.Task::(_debug) Task 588a38d2-8b8b-4542-a007-2754c4717812: moving from state init -> state preparing
>Thread-15913338::DEBUG::2013-11-20 03:02:18,337::task::495::TaskManager.Task::(_debug) Task 588a38d2-8b8b-4542-a007-2754c4717812: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000300168991089', 'lastCheck': 1384916530.1445191, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00152683258057', 'lastCheck': 1384916537.8342071, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00139188766479', 'lastCheck': 1384916528.594934, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00247287750244', 'lastCheck': 1384916537.8298481, 'valid': True, 'code': 0}}
>Thread-15913338::DEBUG::2013-11-20 03:02:18,338::task::495::TaskManager.Task::(_debug) Task 588a38d2-8b8b-4542-a007-2754c4717812: moving from state preparing -> state finished
>Thread-15913338::DEBUG::2013-11-20 03:02:18,338::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913338::DEBUG::2013-11-20 03:02:18,339::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913338::DEBUG::2013-11-20 03:02:18,340::task::495::TaskManager.Task::(_debug) Task 588a38d2-8b8b-4542-a007-2754c4717812: ref 0 aborting False
>Thread-15913338::INFO::2013-11-20 03:02:18,340::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000300168991089', 'lastCheck': 1384916530.1445191, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00152683258057', 'lastCheck': 1384916537.8342071, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00139188766479', 'lastCheck': 1384916528.594934, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00247287750244', 'lastCheck': 1384916537.8298481, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:02:19,680::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 03:02:19,914::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15913340::DEBUG::2013-11-20 03:02:19,917::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912392::DEBUG::2013-11-20 03:02:19,918::task::495::TaskManager.Task::(_debug) Task 1e3154f0-876e-4480-892d-8ea958176425: moving from state init -> state preparing
>Thread-15913340::INFO::2013-11-20 03:02:19,925::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912392::DEBUG::2013-11-20 03:02:19,927::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`894e2ff8-356e-4309-9851-bf876d2521e7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913340::DEBUG::2013-11-20 03:02:19,928::task::495::TaskManager.Task::(_debug) Task 39b7bdc7-2092-47e1-8ece-a99300138c42: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:02:19,928::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913340::DEBUG::2013-11-20 03:02:19,929::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913340::DEBUG::2013-11-20 03:02:19,930::task::495::TaskManager.Task::(_debug) Task 39b7bdc7-2092-47e1-8ece-a99300138c42: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912392::DEBUG::2013-11-20 03:02:19,931::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913340::DEBUG::2013-11-20 03:02:19,931::task::495::TaskManager.Task::(_debug) Task 39b7bdc7-2092-47e1-8ece-a99300138c42: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:02:19,932::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`894e2ff8-356e-4309-9851-bf876d2521e7`::Granted request
>Thread-15913340::DEBUG::2013-11-20 03:02:19,932::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912392::DEBUG::2013-11-20 03:02:19,933::task::495::TaskManager.Task::(_debug) Task 1e3154f0-876e-4480-892d-8ea958176425: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913340::DEBUG::2013-11-20 03:02:19,933::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:02:19,934::task::495::TaskManager.Task::(_debug) Task 1e3154f0-876e-4480-892d-8ea958176425: ref 1 aborting False
>Thread-15913340::DEBUG::2013-11-20 03:02:19,934::task::495::TaskManager.Task::(_debug) Task 39b7bdc7-2092-47e1-8ece-a99300138c42: ref 0 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:02:19,935::task::495::TaskManager.Task::(_debug) Task 1e3154f0-876e-4480-892d-8ea958176425: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913340::INFO::2013-11-20 03:02:19,936::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912392::DEBUG::2013-11-20 03:02:19,936::task::495::TaskManager.Task::(_debug) Task 1e3154f0-876e-4480-892d-8ea958176425: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:02:19,937::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:02:19,938::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:02:19,939::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:02:19,939::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:02:19,940::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:02:19,940::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913341::DEBUG::2013-11-20 03:02:19,941::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912392::DEBUG::2013-11-20 03:02:19,942::task::495::TaskManager.Task::(_debug) Task 1e3154f0-876e-4480-892d-8ea958176425: ref 0 aborting False
>Thread-15913341::INFO::2013-11-20 03:02:19,942::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15912392::INFO::2013-11-20 03:02:19,943::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913341::DEBUG::2013-11-20 03:02:19,943::task::495::TaskManager.Task::(_debug) Task c7020af0-9c5e-42d8-8138-d147408e70c4: moving from state init -> state preparing
>Thread-15913341::DEBUG::2013-11-20 03:02:19,945::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8e57cb1a-db32-4bfc-abd4-4e8e2d67a70e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913341::DEBUG::2013-11-20 03:02:19,945::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913341::DEBUG::2013-11-20 03:02:19,946::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913341::DEBUG::2013-11-20 03:02:19,947::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8e57cb1a-db32-4bfc-abd4-4e8e2d67a70e`::Granted request
>Thread-15913341::DEBUG::2013-11-20 03:02:19,949::task::495::TaskManager.Task::(_debug) Task c7020af0-9c5e-42d8-8138-d147408e70c4: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913341::DEBUG::2013-11-20 03:02:19,949::task::495::TaskManager.Task::(_debug) Task c7020af0-9c5e-42d8-8138-d147408e70c4: ref 1 aborting False
>Thread-15913341::WARNING::2013-11-20 03:02:19,951::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913341::DEBUG::2013-11-20 03:02:19,952::task::495::TaskManager.Task::(_debug) Task c7020af0-9c5e-42d8-8138-d147408e70c4: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913341::DEBUG::2013-11-20 03:02:19,953::task::495::TaskManager.Task::(_debug) Task c7020af0-9c5e-42d8-8138-d147408e70c4: moving from state preparing -> state finished
>Thread-15913341::DEBUG::2013-11-20 03:02:19,954::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913341::DEBUG::2013-11-20 03:02:19,954::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913341::DEBUG::2013-11-20 03:02:19,955::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913341::DEBUG::2013-11-20 03:02:19,956::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913341::DEBUG::2013-11-20 03:02:19,957::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913341::DEBUG::2013-11-20 03:02:19,957::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913341::DEBUG::2013-11-20 03:02:19,958::task::495::TaskManager.Task::(_debug) Task c7020af0-9c5e-42d8-8138-d147408e70c4: ref 0 aborting False
>Thread-15913341::INFO::2013-11-20 03:02:19,959::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:02:20,200::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0718925 s, 14.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:02:22,215::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:02:22,706::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.070126 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:02:24,722::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:02:25,265::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0766892 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:02:27,281::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:02:27,846::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0743702 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:02:29,861::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913345::DEBUG::2013-11-20 03:02:30,108::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913345::INFO::2013-11-20 03:02:30,110::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913345::DEBUG::2013-11-20 03:02:30,111::task::495::TaskManager.Task::(_debug) Task 9d264c69-d678-400d-a9a0-ae65cead9815: moving from state init -> state preparing
>Thread-15913345::DEBUG::2013-11-20 03:02:30,112::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913345::DEBUG::2013-11-20 03:02:30,113::task::495::TaskManager.Task::(_debug) Task 9d264c69-d678-400d-a9a0-ae65cead9815: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913345::DEBUG::2013-11-20 03:02:30,113::task::495::TaskManager.Task::(_debug) Task 9d264c69-d678-400d-a9a0-ae65cead9815: moving from state preparing -> state finished
>Thread-15913345::DEBUG::2013-11-20 03:02:30,114::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913345::DEBUG::2013-11-20 03:02:30,114::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913345::DEBUG::2013-11-20 03:02:30,115::task::495::TaskManager.Task::(_debug) Task 9d264c69-d678-400d-a9a0-ae65cead9815: ref 0 aborting False
>Thread-15913345::INFO::2013-11-20 03:02:30,116::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913346::DEBUG::2013-11-20 03:02:30,121::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913346::INFO::2013-11-20 03:02:30,122::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913346::DEBUG::2013-11-20 03:02:30,122::task::495::TaskManager.Task::(_debug) Task dd3a3077-e55a-4e99-b9f6-19dd36445482: moving from state init -> state preparing
>Thread-15913346::DEBUG::2013-11-20 03:02:30,124::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`10a46d7b-2c18-4dc2-bca8-5234d21e8db8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913346::DEBUG::2013-11-20 03:02:30,124::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913346::DEBUG::2013-11-20 03:02:30,125::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913346::DEBUG::2013-11-20 03:02:30,126::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`10a46d7b-2c18-4dc2-bca8-5234d21e8db8`::Granted request
>Thread-15913346::DEBUG::2013-11-20 03:02:30,127::task::495::TaskManager.Task::(_debug) Task dd3a3077-e55a-4e99-b9f6-19dd36445482: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913346::DEBUG::2013-11-20 03:02:30,128::task::495::TaskManager.Task::(_debug) Task dd3a3077-e55a-4e99-b9f6-19dd36445482: ref 1 aborting False
>Thread-15913346::WARNING::2013-11-20 03:02:30,130::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913346::DEBUG::2013-11-20 03:02:30,131::task::495::TaskManager.Task::(_debug) Task dd3a3077-e55a-4e99-b9f6-19dd36445482: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913346::DEBUG::2013-11-20 03:02:30,132::task::495::TaskManager.Task::(_debug) Task dd3a3077-e55a-4e99-b9f6-19dd36445482: moving from state preparing -> state finished
>Thread-15913346::DEBUG::2013-11-20 03:02:30,133::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913346::DEBUG::2013-11-20 03:02:30,133::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913346::DEBUG::2013-11-20 03:02:30,134::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913346::DEBUG::2013-11-20 03:02:30,135::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913346::DEBUG::2013-11-20 03:02:30,135::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913346::DEBUG::2013-11-20 03:02:30,136::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913346::DEBUG::2013-11-20 03:02:30,136::task::495::TaskManager.Task::(_debug) Task dd3a3077-e55a-4e99-b9f6-19dd36445482: ref 0 aborting False
>Thread-15913346::INFO::2013-11-20 03:02:30,137::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:02:30,395::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0749153 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:02:32,410::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:02:32,955::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0737981 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15913348::INFO::2013-11-20 03:02:33,640::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913348::DEBUG::2013-11-20 03:02:33,641::task::495::TaskManager.Task::(_debug) Task 4d57c18a-de50-427c-b959-102cf5f53583: moving from state init -> state preparing
>Thread-15913348::DEBUG::2013-11-20 03:02:33,642::task::495::TaskManager.Task::(_debug) Task 4d57c18a-de50-427c-b959-102cf5f53583: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000504970550537', 'lastCheck': 1384916550.5359321, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00235915184021', 'lastCheck': 1384916547.978622, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00185799598694', 'lastCheck': 1384916548.966913, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000635862350464', 'lastCheck': 1384916547.9574921, 'valid': True, 'code': 0}}
>Thread-15913348::DEBUG::2013-11-20 03:02:33,642::task::495::TaskManager.Task::(_debug) Task 4d57c18a-de50-427c-b959-102cf5f53583: moving from state preparing -> state finished
>Thread-15913348::DEBUG::2013-11-20 03:02:33,643::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913348::DEBUG::2013-11-20 03:02:33,643::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913348::DEBUG::2013-11-20 03:02:33,644::task::495::TaskManager.Task::(_debug) Task 4d57c18a-de50-427c-b959-102cf5f53583: ref 0 aborting False
>Thread-15913348::INFO::2013-11-20 03:02:33,645::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000504970550537', 'lastCheck': 1384916550.5359321, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00235915184021', 'lastCheck': 1384916547.978622, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00185799598694', 'lastCheck': 1384916548.966913, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000635862350464', 'lastCheck': 1384916547.9574921, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:02:34,970::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:02:35,495::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.079021 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:02:37,509::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:02:38,071::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0739297 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:02:40,087::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913352::DEBUG::2013-11-20 03:02:40,416::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913352::INFO::2013-11-20 03:02:40,418::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913352::DEBUG::2013-11-20 03:02:40,419::task::495::TaskManager.Task::(_debug) Task fb3eb998-0281-49d6-80fd-fa08693e565f: moving from state init -> state preparing
>Thread-15913352::DEBUG::2013-11-20 03:02:40,420::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913352::DEBUG::2013-11-20 03:02:40,420::task::495::TaskManager.Task::(_debug) Task fb3eb998-0281-49d6-80fd-fa08693e565f: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913352::DEBUG::2013-11-20 03:02:40,421::task::495::TaskManager.Task::(_debug) Task fb3eb998-0281-49d6-80fd-fa08693e565f: moving from state preparing -> state finished
>Thread-15913352::DEBUG::2013-11-20 03:02:40,422::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913352::DEBUG::2013-11-20 03:02:40,422::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913352::DEBUG::2013-11-20 03:02:40,423::task::495::TaskManager.Task::(_debug) Task fb3eb998-0281-49d6-80fd-fa08693e565f: ref 0 aborting False
>Thread-15913352::INFO::2013-11-20 03:02:40,425::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913353::DEBUG::2013-11-20 03:02:40,430::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913353::INFO::2013-11-20 03:02:40,431::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913353::DEBUG::2013-11-20 03:02:40,433::task::495::TaskManager.Task::(_debug) Task 0fc5d766-75f4-4b33-9738-7089ca82430a: moving from state init -> state preparing
>Thread-15913353::DEBUG::2013-11-20 03:02:40,436::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7c0170fe-7812-4d5a-859c-ceb53044af91`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913353::DEBUG::2013-11-20 03:02:40,438::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913353::DEBUG::2013-11-20 03:02:40,440::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913353::DEBUG::2013-11-20 03:02:40,446::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7c0170fe-7812-4d5a-859c-ceb53044af91`::Granted request
>Thread-15913353::DEBUG::2013-11-20 03:02:40,450::task::495::TaskManager.Task::(_debug) Task 0fc5d766-75f4-4b33-9738-7089ca82430a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913353::DEBUG::2013-11-20 03:02:40,451::task::495::TaskManager.Task::(_debug) Task 0fc5d766-75f4-4b33-9738-7089ca82430a: ref 1 aborting False
>Thread-15913353::WARNING::2013-11-20 03:02:40,454::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913353::DEBUG::2013-11-20 03:02:40,455::task::495::TaskManager.Task::(_debug) Task 0fc5d766-75f4-4b33-9738-7089ca82430a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913353::DEBUG::2013-11-20 03:02:40,456::task::495::TaskManager.Task::(_debug) Task 0fc5d766-75f4-4b33-9738-7089ca82430a: moving from state preparing -> state finished
>Thread-15913353::DEBUG::2013-11-20 03:02:40,456::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913353::DEBUG::2013-11-20 03:02:40,457::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913353::DEBUG::2013-11-20 03:02:40,457::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913353::DEBUG::2013-11-20 03:02:40,458::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913353::DEBUG::2013-11-20 03:02:40,459::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913353::DEBUG::2013-11-20 03:02:40,459::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913353::DEBUG::2013-11-20 03:02:40,460::task::495::TaskManager.Task::(_debug) Task 0fc5d766-75f4-4b33-9738-7089ca82430a: ref 0 aborting False
>Thread-15913353::INFO::2013-11-20 03:02:40,460::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:02:40,630::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0731539 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:02:42,645::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:02:43,269::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.080722 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:02:45,282::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:02:45,834::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0718047 s, 14.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:02:47,849::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:02:48,356::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0624285 s, 16.4 MB/s\n'; <rc> = 0
>Thread-15913356::INFO::2013-11-20 03:02:49,394::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913356::DEBUG::2013-11-20 03:02:49,395::task::495::TaskManager.Task::(_debug) Task 643c0aad-cbf0-4235-9b78-32c2e56cf1c4: moving from state init -> state preparing
>Thread-15913356::DEBUG::2013-11-20 03:02:49,396::task::495::TaskManager.Task::(_debug) Task 643c0aad-cbf0-4235-9b78-32c2e56cf1c4: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000452041625977', 'lastCheck': 1384916560.7507241, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00139403343201', 'lastCheck': 1384916568.5079219, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00128793716431', 'lastCheck': 1384916559.195143, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000478982925415', 'lastCheck': 1384916568.5010929, 'valid': True, 'code': 0}}
>Thread-15913356::DEBUG::2013-11-20 03:02:49,397::task::495::TaskManager.Task::(_debug) Task 643c0aad-cbf0-4235-9b78-32c2e56cf1c4: moving from state preparing -> state finished
>Thread-15913356::DEBUG::2013-11-20 03:02:49,397::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913356::DEBUG::2013-11-20 03:02:49,398::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913356::DEBUG::2013-11-20 03:02:49,398::task::495::TaskManager.Task::(_debug) Task 643c0aad-cbf0-4235-9b78-32c2e56cf1c4: ref 0 aborting False
>Thread-15913356::INFO::2013-11-20 03:02:49,399::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000452041625977', 'lastCheck': 1384916560.7507241, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00139403343201', 'lastCheck': 1384916568.5079219, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00128793716431', 'lastCheck': 1384916559.195143, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000478982925415', 'lastCheck': 1384916568.5010929, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:02:50,372::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913358::DEBUG::2013-11-20 03:02:50,635::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913358::INFO::2013-11-20 03:02:50,637::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913358::DEBUG::2013-11-20 03:02:50,638::task::495::TaskManager.Task::(_debug) Task c2ca3987-b25c-41b1-b7eb-db3f59d8c9be: moving from state init -> state preparing
>Thread-15913358::DEBUG::2013-11-20 03:02:50,639::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913358::DEBUG::2013-11-20 03:02:50,640::task::495::TaskManager.Task::(_debug) Task c2ca3987-b25c-41b1-b7eb-db3f59d8c9be: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913358::DEBUG::2013-11-20 03:02:50,641::task::495::TaskManager.Task::(_debug) Task c2ca3987-b25c-41b1-b7eb-db3f59d8c9be: moving from state preparing -> state finished
>Thread-15913358::DEBUG::2013-11-20 03:02:50,642::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913358::DEBUG::2013-11-20 03:02:50,643::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913358::DEBUG::2013-11-20 03:02:50,643::task::495::TaskManager.Task::(_debug) Task c2ca3987-b25c-41b1-b7eb-db3f59d8c9be: ref 0 aborting False
>Thread-15913358::INFO::2013-11-20 03:02:50,644::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913359::DEBUG::2013-11-20 03:02:50,652::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913359::INFO::2013-11-20 03:02:50,660::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913359::DEBUG::2013-11-20 03:02:50,666::task::495::TaskManager.Task::(_debug) Task 11caa5da-42ea-4244-bfcf-afab23af8d30: moving from state init -> state preparing
>Thread-15913359::DEBUG::2013-11-20 03:02:50,675::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d9b0411e-85d6-4ba9-8a86-814673e824ff`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913359::DEBUG::2013-11-20 03:02:50,675::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913359::DEBUG::2013-11-20 03:02:50,676::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913359::DEBUG::2013-11-20 03:02:50,677::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d9b0411e-85d6-4ba9-8a86-814673e824ff`::Granted request
>Thread-15913359::DEBUG::2013-11-20 03:02:50,678::task::495::TaskManager.Task::(_debug) Task 11caa5da-42ea-4244-bfcf-afab23af8d30: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913359::DEBUG::2013-11-20 03:02:50,679::task::495::TaskManager.Task::(_debug) Task 11caa5da-42ea-4244-bfcf-afab23af8d30: ref 1 aborting False
>Thread-15913359::WARNING::2013-11-20 03:02:50,681::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913359::DEBUG::2013-11-20 03:02:50,682::task::495::TaskManager.Task::(_debug) Task 11caa5da-42ea-4244-bfcf-afab23af8d30: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913359::DEBUG::2013-11-20 03:02:50,683::task::495::TaskManager.Task::(_debug) Task 11caa5da-42ea-4244-bfcf-afab23af8d30: moving from state preparing -> state finished
>Thread-15913359::DEBUG::2013-11-20 03:02:50,684::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913359::DEBUG::2013-11-20 03:02:50,684::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913359::DEBUG::2013-11-20 03:02:50,685::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913359::DEBUG::2013-11-20 03:02:50,686::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913359::DEBUG::2013-11-20 03:02:50,686::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913359::DEBUG::2013-11-20 03:02:50,687::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913359::DEBUG::2013-11-20 03:02:50,688::task::495::TaskManager.Task::(_debug) Task 11caa5da-42ea-4244-bfcf-afab23af8d30: ref 0 aborting False
>Thread-15913359::INFO::2013-11-20 03:02:50,688::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:02:50,908::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0742906 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:02:52,923::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:02:53,479::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0800357 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:02:55,493::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:02:56,062::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0734466 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:02:58,077::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:02:58,596::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0661102 s, 15.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:03:00,609::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:03:01,158::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0759352 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15913363::DEBUG::2013-11-20 03:03:01,327::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913363::INFO::2013-11-20 03:03:01,328::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913363::DEBUG::2013-11-20 03:03:01,329::task::495::TaskManager.Task::(_debug) Task 442c7635-5a5b-4f32-a72b-afe54febb92c: moving from state init -> state preparing
>Thread-15913363::DEBUG::2013-11-20 03:03:01,330::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913363::DEBUG::2013-11-20 03:03:01,330::task::495::TaskManager.Task::(_debug) Task 442c7635-5a5b-4f32-a72b-afe54febb92c: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913363::DEBUG::2013-11-20 03:03:01,331::task::495::TaskManager.Task::(_debug) Task 442c7635-5a5b-4f32-a72b-afe54febb92c: moving from state preparing -> state finished
>Thread-15913363::DEBUG::2013-11-20 03:03:01,332::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913363::DEBUG::2013-11-20 03:03:01,332::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913363::DEBUG::2013-11-20 03:03:01,333::task::495::TaskManager.Task::(_debug) Task 442c7635-5a5b-4f32-a72b-afe54febb92c: ref 0 aborting False
>Thread-15913363::INFO::2013-11-20 03:03:01,334::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913364::DEBUG::2013-11-20 03:03:01,339::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913364::INFO::2013-11-20 03:03:01,340::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913364::DEBUG::2013-11-20 03:03:01,340::task::495::TaskManager.Task::(_debug) Task 376ab2a2-8c89-4427-bb67-66301b5e1749: moving from state init -> state preparing
>Thread-15913364::DEBUG::2013-11-20 03:03:01,342::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7845797a-d2d0-4a69-8f46-18db55562257`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913364::DEBUG::2013-11-20 03:03:01,342::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913364::DEBUG::2013-11-20 03:03:01,343::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913364::DEBUG::2013-11-20 03:03:01,344::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7845797a-d2d0-4a69-8f46-18db55562257`::Granted request
>Thread-15913364::DEBUG::2013-11-20 03:03:01,345::task::495::TaskManager.Task::(_debug) Task 376ab2a2-8c89-4427-bb67-66301b5e1749: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913364::DEBUG::2013-11-20 03:03:01,345::task::495::TaskManager.Task::(_debug) Task 376ab2a2-8c89-4427-bb67-66301b5e1749: ref 1 aborting False
>Thread-15913364::WARNING::2013-11-20 03:03:01,347::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913364::DEBUG::2013-11-20 03:03:01,348::task::495::TaskManager.Task::(_debug) Task 376ab2a2-8c89-4427-bb67-66301b5e1749: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913364::DEBUG::2013-11-20 03:03:01,349::task::495::TaskManager.Task::(_debug) Task 376ab2a2-8c89-4427-bb67-66301b5e1749: moving from state preparing -> state finished
>Thread-15913364::DEBUG::2013-11-20 03:03:01,350::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913364::DEBUG::2013-11-20 03:03:01,350::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913364::DEBUG::2013-11-20 03:03:01,351::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913364::DEBUG::2013-11-20 03:03:01,351::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913364::DEBUG::2013-11-20 03:03:01,351::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913364::DEBUG::2013-11-20 03:03:01,352::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913364::DEBUG::2013-11-20 03:03:01,352::task::495::TaskManager.Task::(_debug) Task 376ab2a2-8c89-4427-bb67-66301b5e1749: ref 0 aborting False
>Thread-15913364::INFO::2013-11-20 03:03:01,352::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:03:03,174::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:03:03,753::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0741728 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15913366::INFO::2013-11-20 03:03:05,511::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913366::DEBUG::2013-11-20 03:03:05,512::task::495::TaskManager.Task::(_debug) Task a9c8710d-3e2d-4b8c-8bd7-3aa184979a35: moving from state init -> state preparing
>Thread-15913366::DEBUG::2013-11-20 03:03:05,513::task::495::TaskManager.Task::(_debug) Task a9c8710d-3e2d-4b8c-8bd7-3aa184979a35: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000447988510132', 'lastCheck': 1384916581.2859671, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00149416923523', 'lastCheck': 1384916578.755456, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00164604187012', 'lastCheck': 1384916579.755542, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000443935394287', 'lastCheck': 1384916578.7491939, 'valid': True, 'code': 0}}
>Thread-15913366::DEBUG::2013-11-20 03:03:05,513::task::495::TaskManager.Task::(_debug) Task a9c8710d-3e2d-4b8c-8bd7-3aa184979a35: moving from state preparing -> state finished
>Thread-15913366::DEBUG::2013-11-20 03:03:05,514::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913366::DEBUG::2013-11-20 03:03:05,514::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913366::DEBUG::2013-11-20 03:03:05,515::task::495::TaskManager.Task::(_debug) Task a9c8710d-3e2d-4b8c-8bd7-3aa184979a35: ref 0 aborting False
>Thread-15913366::INFO::2013-11-20 03:03:05,516::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000447988510132', 'lastCheck': 1384916581.2859671, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00149416923523', 'lastCheck': 1384916578.755456, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00164604187012', 'lastCheck': 1384916579.755542, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000443935394287', 'lastCheck': 1384916578.7491939, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:03:05,765::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:03:06,335::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0754488 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:03:08,351::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:03:08,864::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0730617 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:03:10,880::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:03:11,435::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0721546 s, 14.2 MB/s\n'; <rc> = 0
>Thread-15913369::DEBUG::2013-11-20 03:03:11,696::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913369::INFO::2013-11-20 03:03:11,698::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913369::DEBUG::2013-11-20 03:03:11,698::task::495::TaskManager.Task::(_debug) Task 38cd50f8-d71a-41a8-b78b-23868a799274: moving from state init -> state preparing
>Thread-15913369::DEBUG::2013-11-20 03:03:11,699::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913369::DEBUG::2013-11-20 03:03:11,700::task::495::TaskManager.Task::(_debug) Task 38cd50f8-d71a-41a8-b78b-23868a799274: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913369::DEBUG::2013-11-20 03:03:11,700::task::495::TaskManager.Task::(_debug) Task 38cd50f8-d71a-41a8-b78b-23868a799274: moving from state preparing -> state finished
>Thread-15913369::DEBUG::2013-11-20 03:03:11,701::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913369::DEBUG::2013-11-20 03:03:11,701::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913369::DEBUG::2013-11-20 03:03:11,702::task::495::TaskManager.Task::(_debug) Task 38cd50f8-d71a-41a8-b78b-23868a799274: ref 0 aborting False
>Thread-15913369::INFO::2013-11-20 03:03:11,702::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913370::DEBUG::2013-11-20 03:03:11,707::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913370::INFO::2013-11-20 03:03:11,708::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913370::DEBUG::2013-11-20 03:03:11,708::task::495::TaskManager.Task::(_debug) Task 32a757c5-7bd5-4bbc-809a-91b3ac554dda: moving from state init -> state preparing
>Thread-15913370::DEBUG::2013-11-20 03:03:11,710::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5d6caa6a-6f04-4adf-bbe7-6a72dedf8ddb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913370::DEBUG::2013-11-20 03:03:11,710::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913370::DEBUG::2013-11-20 03:03:11,711::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913370::DEBUG::2013-11-20 03:03:11,712::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5d6caa6a-6f04-4adf-bbe7-6a72dedf8ddb`::Granted request
>Thread-15913370::DEBUG::2013-11-20 03:03:11,719::task::495::TaskManager.Task::(_debug) Task 32a757c5-7bd5-4bbc-809a-91b3ac554dda: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913370::DEBUG::2013-11-20 03:03:11,720::task::495::TaskManager.Task::(_debug) Task 32a757c5-7bd5-4bbc-809a-91b3ac554dda: ref 1 aborting False
>Thread-15913370::WARNING::2013-11-20 03:03:11,722::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913370::DEBUG::2013-11-20 03:03:11,722::task::495::TaskManager.Task::(_debug) Task 32a757c5-7bd5-4bbc-809a-91b3ac554dda: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913370::DEBUG::2013-11-20 03:03:11,723::task::495::TaskManager.Task::(_debug) Task 32a757c5-7bd5-4bbc-809a-91b3ac554dda: moving from state preparing -> state finished
>Thread-15913370::DEBUG::2013-11-20 03:03:11,724::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913370::DEBUG::2013-11-20 03:03:11,725::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913370::DEBUG::2013-11-20 03:03:11,725::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913370::DEBUG::2013-11-20 03:03:11,726::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913370::DEBUG::2013-11-20 03:03:11,727::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913370::DEBUG::2013-11-20 03:03:11,727::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913370::DEBUG::2013-11-20 03:03:11,728::task::495::TaskManager.Task::(_debug) Task 32a757c5-7bd5-4bbc-809a-91b3ac554dda: ref 0 aborting False
>Thread-15913370::INFO::2013-11-20 03:03:11,728::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:03:13,451::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:03:13,982::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0728576 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:03:15,997::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:03:16,577::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0951199 s, 10.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:03:18,592::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:03:19,097::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0970488 s, 10.6 MB/s\n'; <rc> = 0
>Thread-15913374::INFO::2013-11-20 03:03:20,984::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913374::DEBUG::2013-11-20 03:03:20,985::task::495::TaskManager.Task::(_debug) Task 8784b357-707d-4530-99da-018d27cc94b1: moving from state init -> state preparing
>Thread-15913374::DEBUG::2013-11-20 03:03:20,986::task::495::TaskManager.Task::(_debug) Task 8784b357-707d-4530-99da-018d27cc94b1: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00052809715271', 'lastCheck': 1384916591.5548351, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00161504745483', 'lastCheck': 1384916599.236433, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00185298919678', 'lastCheck': 1384916600.2357991, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000537872314453', 'lastCheck': 1384916599.2286971, 'valid': True, 'code': 0}}
>Thread-15913374::DEBUG::2013-11-20 03:03:20,987::task::495::TaskManager.Task::(_debug) Task 8784b357-707d-4530-99da-018d27cc94b1: moving from state preparing -> state finished
>Thread-15913374::DEBUG::2013-11-20 03:03:20,987::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913374::DEBUG::2013-11-20 03:03:20,988::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913374::DEBUG::2013-11-20 03:03:20,989::task::495::TaskManager.Task::(_debug) Task 8784b357-707d-4530-99da-018d27cc94b1: ref 0 aborting False
>Thread-15913374::INFO::2013-11-20 03:03:20,989::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00052809715271', 'lastCheck': 1384916591.5548351, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00161504745483', 'lastCheck': 1384916599.236433, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00185298919678', 'lastCheck': 1384916600.2357991, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000537872314453', 'lastCheck': 1384916599.2286971, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:03:21,113::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 03:03:21,347::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:03:21,349::task::495::TaskManager.Task::(_debug) Task d2329a02-00dd-4670-8649-52245e3e8d6b: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:03:21,352::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7dc61396-46d5-42cf-a5f2-ddd2bb2a7b76`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:03:21,353::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:03:21,354::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:03:21,355::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7dc61396-46d5-42cf-a5f2-ddd2bb2a7b76`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:03:21,356::task::495::TaskManager.Task::(_debug) Task d2329a02-00dd-4670-8649-52245e3e8d6b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:03:21,356::task::495::TaskManager.Task::(_debug) Task d2329a02-00dd-4670-8649-52245e3e8d6b: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:03:21,357::task::495::TaskManager.Task::(_debug) Task d2329a02-00dd-4670-8649-52245e3e8d6b: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:03:21,358::task::495::TaskManager.Task::(_debug) Task d2329a02-00dd-4670-8649-52245e3e8d6b: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:03:21,358::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:03:21,359::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:03:21,360::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:03:21,360::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:03:21,361::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:03:21,361::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:03:21,362::task::495::TaskManager.Task::(_debug) Task d2329a02-00dd-4670-8649-52245e3e8d6b: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:03:21,363::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:03:21,670::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0741755 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15913376::DEBUG::2013-11-20 03:03:22,052::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913376::INFO::2013-11-20 03:03:22,053::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913376::DEBUG::2013-11-20 03:03:22,053::task::495::TaskManager.Task::(_debug) Task a656acbf-25fb-4d70-a093-45e762d74c16: moving from state init -> state preparing
>Thread-15913376::DEBUG::2013-11-20 03:03:22,054::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913376::DEBUG::2013-11-20 03:03:22,055::task::495::TaskManager.Task::(_debug) Task a656acbf-25fb-4d70-a093-45e762d74c16: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913376::DEBUG::2013-11-20 03:03:22,056::task::495::TaskManager.Task::(_debug) Task a656acbf-25fb-4d70-a093-45e762d74c16: moving from state preparing -> state finished
>Thread-15913376::DEBUG::2013-11-20 03:03:22,057::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913376::DEBUG::2013-11-20 03:03:22,057::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913376::DEBUG::2013-11-20 03:03:22,058::task::495::TaskManager.Task::(_debug) Task a656acbf-25fb-4d70-a093-45e762d74c16: ref 0 aborting False
>Thread-15913376::INFO::2013-11-20 03:03:22,058::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913377::DEBUG::2013-11-20 03:03:22,063::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913377::INFO::2013-11-20 03:03:22,064::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913377::DEBUG::2013-11-20 03:03:22,065::task::495::TaskManager.Task::(_debug) Task 6bc315dc-cb7f-4fe7-8ea2-1fa8041b3adf: moving from state init -> state preparing
>Thread-15913377::DEBUG::2013-11-20 03:03:22,066::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`173faeaa-2fc8-4d98-bc39-d0402d950825`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913377::DEBUG::2013-11-20 03:03:22,067::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913377::DEBUG::2013-11-20 03:03:22,067::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913377::DEBUG::2013-11-20 03:03:22,068::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`173faeaa-2fc8-4d98-bc39-d0402d950825`::Granted request
>Thread-15913377::DEBUG::2013-11-20 03:03:22,069::task::495::TaskManager.Task::(_debug) Task 6bc315dc-cb7f-4fe7-8ea2-1fa8041b3adf: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913377::DEBUG::2013-11-20 03:03:22,070::task::495::TaskManager.Task::(_debug) Task 6bc315dc-cb7f-4fe7-8ea2-1fa8041b3adf: ref 1 aborting False
>Thread-15913377::WARNING::2013-11-20 03:03:22,071::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913377::DEBUG::2013-11-20 03:03:22,072::task::495::TaskManager.Task::(_debug) Task 6bc315dc-cb7f-4fe7-8ea2-1fa8041b3adf: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913377::DEBUG::2013-11-20 03:03:22,073::task::495::TaskManager.Task::(_debug) Task 6bc315dc-cb7f-4fe7-8ea2-1fa8041b3adf: moving from state preparing -> state finished
>Thread-15913377::DEBUG::2013-11-20 03:03:22,074::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913377::DEBUG::2013-11-20 03:03:22,074::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913377::DEBUG::2013-11-20 03:03:22,075::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913377::DEBUG::2013-11-20 03:03:22,076::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913377::DEBUG::2013-11-20 03:03:22,076::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913377::DEBUG::2013-11-20 03:03:22,077::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913377::DEBUG::2013-11-20 03:03:22,077::task::495::TaskManager.Task::(_debug) Task 6bc315dc-cb7f-4fe7-8ea2-1fa8041b3adf: ref 0 aborting False
>Thread-15913377::INFO::2013-11-20 03:03:22,078::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:03:23,685::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:03:24,233::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0798883 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:03:26,247::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:03:26,771::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0679499 s, 15.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:03:28,781::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:03:29,274::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0797188 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:03:31,289::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:03:31,832::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0775865 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15913381::DEBUG::2013-11-20 03:03:32,250::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913381::INFO::2013-11-20 03:03:32,251::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913381::DEBUG::2013-11-20 03:03:32,252::task::495::TaskManager.Task::(_debug) Task da01bf5c-3263-4211-b4e3-be30dd12f5ab: moving from state init -> state preparing
>Thread-15913381::DEBUG::2013-11-20 03:03:32,253::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913381::DEBUG::2013-11-20 03:03:32,253::task::495::TaskManager.Task::(_debug) Task da01bf5c-3263-4211-b4e3-be30dd12f5ab: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913381::DEBUG::2013-11-20 03:03:32,254::task::495::TaskManager.Task::(_debug) Task da01bf5c-3263-4211-b4e3-be30dd12f5ab: moving from state preparing -> state finished
>Thread-15913381::DEBUG::2013-11-20 03:03:32,254::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913381::DEBUG::2013-11-20 03:03:32,255::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913381::DEBUG::2013-11-20 03:03:32,255::task::495::TaskManager.Task::(_debug) Task da01bf5c-3263-4211-b4e3-be30dd12f5ab: ref 0 aborting False
>Thread-15913381::INFO::2013-11-20 03:03:32,256::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913382::DEBUG::2013-11-20 03:03:32,260::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913382::INFO::2013-11-20 03:03:32,261::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913382::DEBUG::2013-11-20 03:03:32,262::task::495::TaskManager.Task::(_debug) Task 83f3105b-b343-40be-807e-199f22f1a036: moving from state init -> state preparing
>Thread-15913382::DEBUG::2013-11-20 03:03:32,263::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6f2f2903-74d5-45d7-8bc2-c32f6da09260`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913382::DEBUG::2013-11-20 03:03:32,263::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913382::DEBUG::2013-11-20 03:03:32,264::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913382::DEBUG::2013-11-20 03:03:32,265::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6f2f2903-74d5-45d7-8bc2-c32f6da09260`::Granted request
>Thread-15913382::DEBUG::2013-11-20 03:03:32,266::task::495::TaskManager.Task::(_debug) Task 83f3105b-b343-40be-807e-199f22f1a036: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913382::DEBUG::2013-11-20 03:03:32,266::task::495::TaskManager.Task::(_debug) Task 83f3105b-b343-40be-807e-199f22f1a036: ref 1 aborting False
>Thread-15913382::WARNING::2013-11-20 03:03:32,268::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913382::DEBUG::2013-11-20 03:03:32,269::task::495::TaskManager.Task::(_debug) Task 83f3105b-b343-40be-807e-199f22f1a036: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913382::DEBUG::2013-11-20 03:03:32,270::task::495::TaskManager.Task::(_debug) Task 83f3105b-b343-40be-807e-199f22f1a036: moving from state preparing -> state finished
>Thread-15913382::DEBUG::2013-11-20 03:03:32,271::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913382::DEBUG::2013-11-20 03:03:32,271::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913382::DEBUG::2013-11-20 03:03:32,272::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913382::DEBUG::2013-11-20 03:03:32,272::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913382::DEBUG::2013-11-20 03:03:32,273::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913382::DEBUG::2013-11-20 03:03:32,273::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913382::DEBUG::2013-11-20 03:03:32,274::task::495::TaskManager.Task::(_debug) Task 83f3105b-b343-40be-807e-199f22f1a036: ref 0 aborting False
>Thread-15913382::INFO::2013-11-20 03:03:32,275::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:03:33,847::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:03:34,376::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0749086 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15913384::INFO::2013-11-20 03:03:36,299::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913384::DEBUG::2013-11-20 03:03:36,299::task::495::TaskManager.Task::(_debug) Task b5bdfd84-9909-4214-98a1-2b7c52e4fe42: moving from state init -> state preparing
>Thread-15913384::DEBUG::2013-11-20 03:03:36,312::task::495::TaskManager.Task::(_debug) Task b5bdfd84-9909-4214-98a1-2b7c52e4fe42: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000366926193237', 'lastCheck': 1384916612.0060439, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0022919178009', 'lastCheck': 1384916609.486022, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00154995918274', 'lastCheck': 1384916610.4844, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0022599697113', 'lastCheck': 1384916609.4818709, 'valid': True, 'code': 0}}
>Thread-15913384::DEBUG::2013-11-20 03:03:36,313::task::495::TaskManager.Task::(_debug) Task b5bdfd84-9909-4214-98a1-2b7c52e4fe42: moving from state preparing -> state finished
>Thread-15913384::DEBUG::2013-11-20 03:03:36,314::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913384::DEBUG::2013-11-20 03:03:36,314::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913384::DEBUG::2013-11-20 03:03:36,315::task::495::TaskManager.Task::(_debug) Task b5bdfd84-9909-4214-98a1-2b7c52e4fe42: ref 0 aborting False
>Thread-15913384::INFO::2013-11-20 03:03:36,316::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000366926193237', 'lastCheck': 1384916612.0060439, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0022919178009', 'lastCheck': 1384916609.486022, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00154995918274', 'lastCheck': 1384916610.4844, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0022599697113', 'lastCheck': 1384916609.4818709, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:03:36,392::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:03:36,932::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0774215 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:03:38,948::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:03:39,487::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0739553 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:03:41,504::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:03:42,108::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0754468 s, 13.6 MB/s\n'; <rc> = 0
>Thread-15913387::DEBUG::2013-11-20 03:03:42,430::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913387::INFO::2013-11-20 03:03:42,431::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913387::DEBUG::2013-11-20 03:03:42,432::task::495::TaskManager.Task::(_debug) Task c0cbe6b9-a3f1-4175-aab3-3fb10e366ea3: moving from state init -> state preparing
>Thread-15913387::DEBUG::2013-11-20 03:03:42,433::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913387::DEBUG::2013-11-20 03:03:42,433::task::495::TaskManager.Task::(_debug) Task c0cbe6b9-a3f1-4175-aab3-3fb10e366ea3: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913387::DEBUG::2013-11-20 03:03:42,434::task::495::TaskManager.Task::(_debug) Task c0cbe6b9-a3f1-4175-aab3-3fb10e366ea3: moving from state preparing -> state finished
>Thread-15913387::DEBUG::2013-11-20 03:03:42,434::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913387::DEBUG::2013-11-20 03:03:42,435::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913387::DEBUG::2013-11-20 03:03:42,435::task::495::TaskManager.Task::(_debug) Task c0cbe6b9-a3f1-4175-aab3-3fb10e366ea3: ref 0 aborting False
>Thread-15913387::INFO::2013-11-20 03:03:42,436::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913388::DEBUG::2013-11-20 03:03:42,441::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913388::INFO::2013-11-20 03:03:42,442::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913388::DEBUG::2013-11-20 03:03:42,442::task::495::TaskManager.Task::(_debug) Task ce5b82f9-59cd-4821-bf85-de211d31482c: moving from state init -> state preparing
>Thread-15913388::DEBUG::2013-11-20 03:03:42,444::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3c29621a-c04b-4c22-b200-50d3c05e0e4c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913388::DEBUG::2013-11-20 03:03:42,444::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913388::DEBUG::2013-11-20 03:03:42,445::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913388::DEBUG::2013-11-20 03:03:42,446::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3c29621a-c04b-4c22-b200-50d3c05e0e4c`::Granted request
>Thread-15913388::DEBUG::2013-11-20 03:03:42,447::task::495::TaskManager.Task::(_debug) Task ce5b82f9-59cd-4821-bf85-de211d31482c: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913388::DEBUG::2013-11-20 03:03:42,447::task::495::TaskManager.Task::(_debug) Task ce5b82f9-59cd-4821-bf85-de211d31482c: ref 1 aborting False
>Thread-15913388::WARNING::2013-11-20 03:03:42,449::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913388::DEBUG::2013-11-20 03:03:42,450::task::495::TaskManager.Task::(_debug) Task ce5b82f9-59cd-4821-bf85-de211d31482c: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913388::DEBUG::2013-11-20 03:03:42,451::task::495::TaskManager.Task::(_debug) Task ce5b82f9-59cd-4821-bf85-de211d31482c: moving from state preparing -> state finished
>Thread-15913388::DEBUG::2013-11-20 03:03:42,451::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913388::DEBUG::2013-11-20 03:03:42,452::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913388::DEBUG::2013-11-20 03:03:42,453::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913388::DEBUG::2013-11-20 03:03:42,453::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913388::DEBUG::2013-11-20 03:03:42,454::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913388::DEBUG::2013-11-20 03:03:42,454::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913388::DEBUG::2013-11-20 03:03:42,455::task::495::TaskManager.Task::(_debug) Task ce5b82f9-59cd-4821-bf85-de211d31482c: ref 0 aborting False
>Thread-15913388::INFO::2013-11-20 03:03:42,456::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:03:44,123::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:03:44,710::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0738376 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:03:46,725::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:03:47,251::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0741322 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:03:49,265::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:03:49,851::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0709328 s, 14.4 MB/s\n'; <rc> = 0
>Thread-15913392::INFO::2013-11-20 03:03:51,613::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913392::DEBUG::2013-11-20 03:03:51,614::task::495::TaskManager.Task::(_debug) Task c6342e1c-f81f-4e15-9c13-9149f93f595d: moving from state init -> state preparing
>Thread-15913392::DEBUG::2013-11-20 03:03:51,615::task::495::TaskManager.Task::(_debug) Task c6342e1c-f81f-4e15-9c13-9149f93f595d: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00189280509949', 'lastCheck': 1384916622.1717269, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00123906135559', 'lastCheck': 1384916629.9370401, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0157740116119', 'lastCheck': 1384916630.950314, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0022611618042', 'lastCheck': 1384916629.9343081, 'valid': True, 'code': 0}}
>Thread-15913392::DEBUG::2013-11-20 03:03:51,616::task::495::TaskManager.Task::(_debug) Task c6342e1c-f81f-4e15-9c13-9149f93f595d: moving from state preparing -> state finished
>Thread-15913392::DEBUG::2013-11-20 03:03:51,617::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913392::DEBUG::2013-11-20 03:03:51,618::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913392::DEBUG::2013-11-20 03:03:51,619::task::495::TaskManager.Task::(_debug) Task c6342e1c-f81f-4e15-9c13-9149f93f595d: ref 0 aborting False
>Thread-15913392::INFO::2013-11-20 03:03:51,620::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00189280509949', 'lastCheck': 1384916622.1717269, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00123906135559', 'lastCheck': 1384916629.9370401, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0157740116119', 'lastCheck': 1384916630.950314, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0022611618042', 'lastCheck': 1384916629.9343081, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:03:51,865::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:03:52,429::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0797454 s, 12.8 MB/s\n'; <rc> = 0
>Thread-15913394::DEBUG::2013-11-20 03:03:52,989::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913394::INFO::2013-11-20 03:03:52,990::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913394::DEBUG::2013-11-20 03:03:52,991::task::495::TaskManager.Task::(_debug) Task 79ba4b79-8d05-4c15-8e68-a82e0436c2ba: moving from state init -> state preparing
>Thread-15913394::DEBUG::2013-11-20 03:03:52,992::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913394::DEBUG::2013-11-20 03:03:52,992::task::495::TaskManager.Task::(_debug) Task 79ba4b79-8d05-4c15-8e68-a82e0436c2ba: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913394::DEBUG::2013-11-20 03:03:52,993::task::495::TaskManager.Task::(_debug) Task 79ba4b79-8d05-4c15-8e68-a82e0436c2ba: moving from state preparing -> state finished
>Thread-15913394::DEBUG::2013-11-20 03:03:52,994::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913394::DEBUG::2013-11-20 03:03:52,994::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913394::DEBUG::2013-11-20 03:03:52,995::task::495::TaskManager.Task::(_debug) Task 79ba4b79-8d05-4c15-8e68-a82e0436c2ba: ref 0 aborting False
>Thread-15913394::INFO::2013-11-20 03:03:52,995::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913395::DEBUG::2013-11-20 03:03:53,000::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913395::INFO::2013-11-20 03:03:53,001::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913395::DEBUG::2013-11-20 03:03:53,001::task::495::TaskManager.Task::(_debug) Task ee59354d-f0b7-486c-9164-3aa36d84afd7: moving from state init -> state preparing
>Thread-15913395::DEBUG::2013-11-20 03:03:53,003::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`13819e59-3283-486b-89ee-325e0af42be3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913395::DEBUG::2013-11-20 03:03:53,003::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913395::DEBUG::2013-11-20 03:03:53,004::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913395::DEBUG::2013-11-20 03:03:53,005::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`13819e59-3283-486b-89ee-325e0af42be3`::Granted request
>Thread-15913395::DEBUG::2013-11-20 03:03:53,006::task::495::TaskManager.Task::(_debug) Task ee59354d-f0b7-486c-9164-3aa36d84afd7: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913395::DEBUG::2013-11-20 03:03:53,006::task::495::TaskManager.Task::(_debug) Task ee59354d-f0b7-486c-9164-3aa36d84afd7: ref 1 aborting False
>Thread-15913395::WARNING::2013-11-20 03:03:53,008::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913395::DEBUG::2013-11-20 03:03:53,009::task::495::TaskManager.Task::(_debug) Task ee59354d-f0b7-486c-9164-3aa36d84afd7: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913395::DEBUG::2013-11-20 03:03:53,010::task::495::TaskManager.Task::(_debug) Task ee59354d-f0b7-486c-9164-3aa36d84afd7: moving from state preparing -> state finished
>Thread-15913395::DEBUG::2013-11-20 03:03:53,011::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913395::DEBUG::2013-11-20 03:03:53,011::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913395::DEBUG::2013-11-20 03:03:53,012::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913395::DEBUG::2013-11-20 03:03:53,012::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913395::DEBUG::2013-11-20 03:03:53,013::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913395::DEBUG::2013-11-20 03:03:53,013::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913395::DEBUG::2013-11-20 03:03:53,014::task::495::TaskManager.Task::(_debug) Task ee59354d-f0b7-486c-9164-3aa36d84afd7: ref 0 aborting False
>Thread-15913395::INFO::2013-11-20 03:03:53,015::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:03:54,445::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:03:54,942::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0465415 s, 22.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:03:56,952::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:03:57,488::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0753649 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:03:59,503::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:04:00,085::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.073482 s, 13.9 MB/s\n'; <rc> = 0
>Thread-42840::DEBUG::2013-11-20 03:04:00,179::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 03:04:01,004::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:04:02,100::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:04:02,619::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0746035 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15913399::DEBUG::2013-11-20 03:04:03,327::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913399::INFO::2013-11-20 03:04:03,328::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913399::DEBUG::2013-11-20 03:04:03,329::task::495::TaskManager.Task::(_debug) Task 6e7dd196-d84b-4858-afd3-001755ae5005: moving from state init -> state preparing
>Thread-15913399::DEBUG::2013-11-20 03:04:03,329::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913399::DEBUG::2013-11-20 03:04:03,330::task::495::TaskManager.Task::(_debug) Task 6e7dd196-d84b-4858-afd3-001755ae5005: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913399::DEBUG::2013-11-20 03:04:03,331::task::495::TaskManager.Task::(_debug) Task 6e7dd196-d84b-4858-afd3-001755ae5005: moving from state preparing -> state finished
>Thread-15913399::DEBUG::2013-11-20 03:04:03,331::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913399::DEBUG::2013-11-20 03:04:03,332::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913399::DEBUG::2013-11-20 03:04:03,332::task::495::TaskManager.Task::(_debug) Task 6e7dd196-d84b-4858-afd3-001755ae5005: ref 0 aborting False
>Thread-15913399::INFO::2013-11-20 03:04:03,333::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913400::DEBUG::2013-11-20 03:04:03,338::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913400::INFO::2013-11-20 03:04:03,338::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913400::DEBUG::2013-11-20 03:04:03,339::task::495::TaskManager.Task::(_debug) Task fefd0323-d77a-4534-b156-0a1541a3700e: moving from state init -> state preparing
>Thread-15913400::DEBUG::2013-11-20 03:04:03,340::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e208a94c-911a-4e96-860c-234fe7d57b7d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913400::DEBUG::2013-11-20 03:04:03,341::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913400::DEBUG::2013-11-20 03:04:03,342::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913400::DEBUG::2013-11-20 03:04:03,342::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e208a94c-911a-4e96-860c-234fe7d57b7d`::Granted request
>Thread-15913400::DEBUG::2013-11-20 03:04:03,343::task::495::TaskManager.Task::(_debug) Task fefd0323-d77a-4534-b156-0a1541a3700e: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913400::DEBUG::2013-11-20 03:04:03,344::task::495::TaskManager.Task::(_debug) Task fefd0323-d77a-4534-b156-0a1541a3700e: ref 1 aborting False
>Thread-15913400::WARNING::2013-11-20 03:04:03,346::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913400::DEBUG::2013-11-20 03:04:03,346::task::495::TaskManager.Task::(_debug) Task fefd0323-d77a-4534-b156-0a1541a3700e: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913400::DEBUG::2013-11-20 03:04:03,347::task::495::TaskManager.Task::(_debug) Task fefd0323-d77a-4534-b156-0a1541a3700e: moving from state preparing -> state finished
>Thread-15913400::DEBUG::2013-11-20 03:04:03,348::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913400::DEBUG::2013-11-20 03:04:03,348::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913400::DEBUG::2013-11-20 03:04:03,349::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913400::DEBUG::2013-11-20 03:04:03,350::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913400::DEBUG::2013-11-20 03:04:03,351::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913400::DEBUG::2013-11-20 03:04:03,351::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913400::DEBUG::2013-11-20 03:04:03,352::task::495::TaskManager.Task::(_debug) Task fefd0323-d77a-4534-b156-0a1541a3700e: ref 0 aborting False
>Thread-15913400::INFO::2013-11-20 03:04:03,352::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:04:04,635::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:04:05,183::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0744093 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:04:07,198::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913402::INFO::2013-11-20 03:04:07,419::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913402::DEBUG::2013-11-20 03:04:07,420::task::495::TaskManager.Task::(_debug) Task 9ff4dd5a-8d63-46d3-8038-7b334a53ba35: moving from state init -> state preparing
>Thread-15913402::DEBUG::2013-11-20 03:04:07,421::task::495::TaskManager.Task::(_debug) Task 9ff4dd5a-8d63-46d3-8038-7b334a53ba35: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00126481056213', 'lastCheck': 1384916642.7752211, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00344204902649', 'lastCheck': 1384916640.1808009, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00153112411499', 'lastCheck': 1384916641.1822331, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00454092025757', 'lastCheck': 1384916641.011745, 'valid': True, 'code': 0}}
>Thread-15913402::DEBUG::2013-11-20 03:04:07,422::task::495::TaskManager.Task::(_debug) Task 9ff4dd5a-8d63-46d3-8038-7b334a53ba35: moving from state preparing -> state finished
>Thread-15913402::DEBUG::2013-11-20 03:04:07,423::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913402::DEBUG::2013-11-20 03:04:07,424::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913402::DEBUG::2013-11-20 03:04:07,424::task::495::TaskManager.Task::(_debug) Task 9ff4dd5a-8d63-46d3-8038-7b334a53ba35: ref 0 aborting False
>Thread-15913402::INFO::2013-11-20 03:04:07,425::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00126481056213', 'lastCheck': 1384916642.7752211, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00344204902649', 'lastCheck': 1384916640.1808009, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00153112411499', 'lastCheck': 1384916641.1822331, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00454092025757', 'lastCheck': 1384916641.011745, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:04:07,771::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0947713 s, 10.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:04:09,786::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:04:10,275::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0611202 s, 16.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:04:12,290::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:04:12,825::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0657174 s, 15.6 MB/s\n'; <rc> = 0
>Thread-15913405::DEBUG::2013-11-20 03:04:13,700::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913405::INFO::2013-11-20 03:04:13,701::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913405::DEBUG::2013-11-20 03:04:13,702::task::495::TaskManager.Task::(_debug) Task 20a7be5b-00b4-4f08-9f80-16cb399f9480: moving from state init -> state preparing
>Thread-15913405::DEBUG::2013-11-20 03:04:13,702::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913405::DEBUG::2013-11-20 03:04:13,703::task::495::TaskManager.Task::(_debug) Task 20a7be5b-00b4-4f08-9f80-16cb399f9480: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913405::DEBUG::2013-11-20 03:04:13,704::task::495::TaskManager.Task::(_debug) Task 20a7be5b-00b4-4f08-9f80-16cb399f9480: moving from state preparing -> state finished
>Thread-15913405::DEBUG::2013-11-20 03:04:13,704::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913405::DEBUG::2013-11-20 03:04:13,705::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913405::DEBUG::2013-11-20 03:04:13,705::task::495::TaskManager.Task::(_debug) Task 20a7be5b-00b4-4f08-9f80-16cb399f9480: ref 0 aborting False
>Thread-15913405::INFO::2013-11-20 03:04:13,706::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913406::DEBUG::2013-11-20 03:04:13,710::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913406::INFO::2013-11-20 03:04:13,711::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913406::DEBUG::2013-11-20 03:04:13,712::task::495::TaskManager.Task::(_debug) Task 631e5b47-e5b5-476e-892b-147f04afc351: moving from state init -> state preparing
>Thread-15913406::DEBUG::2013-11-20 03:04:13,713::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`61d4cb4b-6335-43c2-8f84-3f232cc753aa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913406::DEBUG::2013-11-20 03:04:13,713::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913406::DEBUG::2013-11-20 03:04:13,714::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913406::DEBUG::2013-11-20 03:04:13,715::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`61d4cb4b-6335-43c2-8f84-3f232cc753aa`::Granted request
>Thread-15913406::DEBUG::2013-11-20 03:04:13,716::task::495::TaskManager.Task::(_debug) Task 631e5b47-e5b5-476e-892b-147f04afc351: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913406::DEBUG::2013-11-20 03:04:13,716::task::495::TaskManager.Task::(_debug) Task 631e5b47-e5b5-476e-892b-147f04afc351: ref 1 aborting False
>Thread-15913406::WARNING::2013-11-20 03:04:13,718::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913406::DEBUG::2013-11-20 03:04:13,719::task::495::TaskManager.Task::(_debug) Task 631e5b47-e5b5-476e-892b-147f04afc351: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913406::DEBUG::2013-11-20 03:04:13,720::task::495::TaskManager.Task::(_debug) Task 631e5b47-e5b5-476e-892b-147f04afc351: moving from state preparing -> state finished
>Thread-15913406::DEBUG::2013-11-20 03:04:13,721::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913406::DEBUG::2013-11-20 03:04:13,721::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913406::DEBUG::2013-11-20 03:04:13,722::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913406::DEBUG::2013-11-20 03:04:13,722::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913406::DEBUG::2013-11-20 03:04:13,723::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913406::DEBUG::2013-11-20 03:04:13,723::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913406::DEBUG::2013-11-20 03:04:13,724::task::495::TaskManager.Task::(_debug) Task 631e5b47-e5b5-476e-892b-147f04afc351: ref 0 aborting False
>Thread-15913406::INFO::2013-11-20 03:04:13,725::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:04:14,840::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:04:15,360::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0729735 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:04:17,376::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:04:17,932::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0737291 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:04:19,947::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:04:20,482::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0825499 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:04:22,498::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 03:04:22,734::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:04:22,744::task::495::TaskManager.Task::(_debug) Task 7ce8ddc9-ce64-4a96-b894-500477c7456f: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:04:22,748::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e6072185-471a-4b0b-b442-9aeeed619848`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:04:22,749::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:04:22,751::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:04:22,751::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e6072185-471a-4b0b-b442-9aeeed619848`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:04:22,753::task::495::TaskManager.Task::(_debug) Task 7ce8ddc9-ce64-4a96-b894-500477c7456f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:04:22,754::task::495::TaskManager.Task::(_debug) Task 7ce8ddc9-ce64-4a96-b894-500477c7456f: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:04:22,755::task::495::TaskManager.Task::(_debug) Task 7ce8ddc9-ce64-4a96-b894-500477c7456f: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:04:22,755::task::495::TaskManager.Task::(_debug) Task 7ce8ddc9-ce64-4a96-b894-500477c7456f: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:04:22,756::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:04:22,757::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:04:22,758::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:04:22,759::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:04:22,759::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:04:22,760::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:04:22,761::task::495::TaskManager.Task::(_debug) Task 7ce8ddc9-ce64-4a96-b894-500477c7456f: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:04:22,762::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:04:23,031::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0648384 s, 15.8 MB/s\n'; <rc> = 0
>Thread-15913410::INFO::2013-11-20 03:04:23,180::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913410::DEBUG::2013-11-20 03:04:23,180::task::495::TaskManager.Task::(_debug) Task b8cf6f43-3ef1-4d15-a81a-53d116878a92: moving from state init -> state preparing
>Thread-15913410::DEBUG::2013-11-20 03:04:23,181::task::495::TaskManager.Task::(_debug) Task b8cf6f43-3ef1-4d15-a81a-53d116878a92: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000411987304688', 'lastCheck': 1384916663.1632409, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00121688842773', 'lastCheck': 1384916660.6189511, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00131988525391', 'lastCheck': 1384916661.6298881, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000414848327637', 'lastCheck': 1384916661.157161, 'valid': True, 'code': 0}}
>Thread-15913410::DEBUG::2013-11-20 03:04:23,182::task::495::TaskManager.Task::(_debug) Task b8cf6f43-3ef1-4d15-a81a-53d116878a92: moving from state preparing -> state finished
>Thread-15913410::DEBUG::2013-11-20 03:04:23,183::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913410::DEBUG::2013-11-20 03:04:23,183::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913410::DEBUG::2013-11-20 03:04:23,184::task::495::TaskManager.Task::(_debug) Task b8cf6f43-3ef1-4d15-a81a-53d116878a92: ref 0 aborting False
>Thread-15913410::INFO::2013-11-20 03:04:23,184::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000411987304688', 'lastCheck': 1384916663.1632409, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00121688842773', 'lastCheck': 1384916660.6189511, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00131988525391', 'lastCheck': 1384916661.6298881, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000414848327637', 'lastCheck': 1384916661.157161, 'valid': True, 'code': 0}}
>Thread-15913412::DEBUG::2013-11-20 03:04:24,081::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913412::INFO::2013-11-20 03:04:24,082::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913412::DEBUG::2013-11-20 03:04:24,083::task::495::TaskManager.Task::(_debug) Task a789e2c7-9512-45d9-b178-14fec1a2d7a4: moving from state init -> state preparing
>Thread-15913412::DEBUG::2013-11-20 03:04:24,084::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913412::DEBUG::2013-11-20 03:04:24,084::task::495::TaskManager.Task::(_debug) Task a789e2c7-9512-45d9-b178-14fec1a2d7a4: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913412::DEBUG::2013-11-20 03:04:24,085::task::495::TaskManager.Task::(_debug) Task a789e2c7-9512-45d9-b178-14fec1a2d7a4: moving from state preparing -> state finished
>Thread-15913412::DEBUG::2013-11-20 03:04:24,085::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913412::DEBUG::2013-11-20 03:04:24,086::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913412::DEBUG::2013-11-20 03:04:24,086::task::495::TaskManager.Task::(_debug) Task a789e2c7-9512-45d9-b178-14fec1a2d7a4: ref 0 aborting False
>Thread-15913412::INFO::2013-11-20 03:04:24,087::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913413::DEBUG::2013-11-20 03:04:24,091::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913413::INFO::2013-11-20 03:04:24,092::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913413::DEBUG::2013-11-20 03:04:24,093::task::495::TaskManager.Task::(_debug) Task 40db714c-d015-4e5e-ade0-6ea908894062: moving from state init -> state preparing
>Thread-15913413::DEBUG::2013-11-20 03:04:24,094::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e18194cc-c6b9-403d-a670-d42807c13b43`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913413::DEBUG::2013-11-20 03:04:24,094::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913413::DEBUG::2013-11-20 03:04:24,095::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913413::DEBUG::2013-11-20 03:04:24,096::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e18194cc-c6b9-403d-a670-d42807c13b43`::Granted request
>Thread-15913413::DEBUG::2013-11-20 03:04:24,096::task::495::TaskManager.Task::(_debug) Task 40db714c-d015-4e5e-ade0-6ea908894062: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913413::DEBUG::2013-11-20 03:04:24,097::task::495::TaskManager.Task::(_debug) Task 40db714c-d015-4e5e-ade0-6ea908894062: ref 1 aborting False
>Thread-15913413::WARNING::2013-11-20 03:04:24,099::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913413::DEBUG::2013-11-20 03:04:24,100::task::495::TaskManager.Task::(_debug) Task 40db714c-d015-4e5e-ade0-6ea908894062: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913413::DEBUG::2013-11-20 03:04:24,101::task::495::TaskManager.Task::(_debug) Task 40db714c-d015-4e5e-ade0-6ea908894062: moving from state preparing -> state finished
>Thread-15913413::DEBUG::2013-11-20 03:04:24,101::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913413::DEBUG::2013-11-20 03:04:24,102::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913413::DEBUG::2013-11-20 03:04:24,103::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913413::DEBUG::2013-11-20 03:04:24,103::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913413::DEBUG::2013-11-20 03:04:24,104::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913413::DEBUG::2013-11-20 03:04:24,104::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913413::DEBUG::2013-11-20 03:04:24,105::task::495::TaskManager.Task::(_debug) Task 40db714c-d015-4e5e-ade0-6ea908894062: ref 0 aborting False
>Thread-15913413::INFO::2013-11-20 03:04:24,106::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:04:25,045::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:04:25,590::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0734838 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:04:27,604::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:04:28,145::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.074059 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:04:30,160::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:04:30,712::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0735521 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:04:32,727::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:04:33,251::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0747814 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15913417::DEBUG::2013-11-20 03:04:34,240::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913417::INFO::2013-11-20 03:04:34,242::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913417::DEBUG::2013-11-20 03:04:34,243::task::495::TaskManager.Task::(_debug) Task 60ba0124-a81d-44d6-8535-5f900a106447: moving from state init -> state preparing
>Thread-15913417::DEBUG::2013-11-20 03:04:34,243::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913417::DEBUG::2013-11-20 03:04:34,244::task::495::TaskManager.Task::(_debug) Task 60ba0124-a81d-44d6-8535-5f900a106447: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913417::DEBUG::2013-11-20 03:04:34,245::task::495::TaskManager.Task::(_debug) Task 60ba0124-a81d-44d6-8535-5f900a106447: moving from state preparing -> state finished
>Thread-15913417::DEBUG::2013-11-20 03:04:34,245::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913417::DEBUG::2013-11-20 03:04:34,246::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913417::DEBUG::2013-11-20 03:04:34,246::task::495::TaskManager.Task::(_debug) Task 60ba0124-a81d-44d6-8535-5f900a106447: ref 0 aborting False
>Thread-15913417::INFO::2013-11-20 03:04:34,247::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913418::DEBUG::2013-11-20 03:04:34,252::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913418::INFO::2013-11-20 03:04:34,252::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913418::DEBUG::2013-11-20 03:04:34,253::task::495::TaskManager.Task::(_debug) Task 1b741d93-7fc4-4a22-b836-21f1e58ac960: moving from state init -> state preparing
>Thread-15913418::DEBUG::2013-11-20 03:04:34,254::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`507fe5d5-4a97-48fb-a8bb-843280ccc3bd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913418::DEBUG::2013-11-20 03:04:34,255::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913418::DEBUG::2013-11-20 03:04:34,256::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913418::DEBUG::2013-11-20 03:04:34,256::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`507fe5d5-4a97-48fb-a8bb-843280ccc3bd`::Granted request
>Thread-15913418::DEBUG::2013-11-20 03:04:34,257::task::495::TaskManager.Task::(_debug) Task 1b741d93-7fc4-4a22-b836-21f1e58ac960: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913418::DEBUG::2013-11-20 03:04:34,258::task::495::TaskManager.Task::(_debug) Task 1b741d93-7fc4-4a22-b836-21f1e58ac960: ref 1 aborting False
>Thread-15913418::WARNING::2013-11-20 03:04:34,260::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913418::DEBUG::2013-11-20 03:04:34,261::task::495::TaskManager.Task::(_debug) Task 1b741d93-7fc4-4a22-b836-21f1e58ac960: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913418::DEBUG::2013-11-20 03:04:34,261::task::495::TaskManager.Task::(_debug) Task 1b741d93-7fc4-4a22-b836-21f1e58ac960: moving from state preparing -> state finished
>Thread-15913418::DEBUG::2013-11-20 03:04:34,262::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913418::DEBUG::2013-11-20 03:04:34,263::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913418::DEBUG::2013-11-20 03:04:34,263::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913418::DEBUG::2013-11-20 03:04:34,264::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913418::DEBUG::2013-11-20 03:04:34,264::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913418::DEBUG::2013-11-20 03:04:34,265::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913418::DEBUG::2013-11-20 03:04:34,265::task::495::TaskManager.Task::(_debug) Task 1b741d93-7fc4-4a22-b836-21f1e58ac960: ref 0 aborting False
>Thread-15913418::INFO::2013-11-20 03:04:34,266::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:04:35,264::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:04:35,817::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.073512 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:04:37,832::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:04:38,342::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0769311 s, 13.3 MB/s\n'; <rc> = 0
>Thread-15913420::INFO::2013-11-20 03:04:38,522::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913420::DEBUG::2013-11-20 03:04:38,523::task::495::TaskManager.Task::(_debug) Task 42d41144-bca7-4fd8-8e80-af6952184c77: moving from state init -> state preparing
>Thread-15913420::DEBUG::2013-11-20 03:04:38,524::task::495::TaskManager.Task::(_debug) Task 42d41144-bca7-4fd8-8e80-af6952184c77: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000554084777832', 'lastCheck': 1384916673.4057069, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00281405448914', 'lastCheck': 1384916670.8374691, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00169396400452', 'lastCheck': 1384916671.8234911, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000450849533081', 'lastCheck': 1384916671.397594, 'valid': True, 'code': 0}}
>Thread-15913420::DEBUG::2013-11-20 03:04:38,525::task::495::TaskManager.Task::(_debug) Task 42d41144-bca7-4fd8-8e80-af6952184c77: moving from state preparing -> state finished
>Thread-15913420::DEBUG::2013-11-20 03:04:38,525::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913420::DEBUG::2013-11-20 03:04:38,526::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913420::DEBUG::2013-11-20 03:04:38,527::task::495::TaskManager.Task::(_debug) Task 42d41144-bca7-4fd8-8e80-af6952184c77: ref 0 aborting False
>Thread-15913420::INFO::2013-11-20 03:04:38,527::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000554084777832', 'lastCheck': 1384916673.4057069, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00281405448914', 'lastCheck': 1384916670.8374691, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00169396400452', 'lastCheck': 1384916671.8234911, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000450849533081', 'lastCheck': 1384916671.397594, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:04:40,357::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:04:40,899::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0701291 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:04:42,915::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:04:43,413::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0528653 s, 19.4 MB/s\n'; <rc> = 0
>Thread-15913423::DEBUG::2013-11-20 03:04:44,406::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913423::INFO::2013-11-20 03:04:44,408::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913423::DEBUG::2013-11-20 03:04:44,409::task::495::TaskManager.Task::(_debug) Task 8ad3370f-ad8d-443e-b013-93e496298393: moving from state init -> state preparing
>Thread-15913423::DEBUG::2013-11-20 03:04:44,409::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913423::DEBUG::2013-11-20 03:04:44,410::task::495::TaskManager.Task::(_debug) Task 8ad3370f-ad8d-443e-b013-93e496298393: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913423::DEBUG::2013-11-20 03:04:44,411::task::495::TaskManager.Task::(_debug) Task 8ad3370f-ad8d-443e-b013-93e496298393: moving from state preparing -> state finished
>Thread-15913423::DEBUG::2013-11-20 03:04:44,411::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913423::DEBUG::2013-11-20 03:04:44,412::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913423::DEBUG::2013-11-20 03:04:44,412::task::495::TaskManager.Task::(_debug) Task 8ad3370f-ad8d-443e-b013-93e496298393: ref 0 aborting False
>Thread-15913423::INFO::2013-11-20 03:04:44,413::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913424::DEBUG::2013-11-20 03:04:44,417::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913424::INFO::2013-11-20 03:04:44,418::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913424::DEBUG::2013-11-20 03:04:44,419::task::495::TaskManager.Task::(_debug) Task c9830739-0e42-4874-9439-2d920f5ea0ee: moving from state init -> state preparing
>Thread-15913424::DEBUG::2013-11-20 03:04:44,420::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7a046fed-ec53-4b4d-92f2-551f723c4c3b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913424::DEBUG::2013-11-20 03:04:44,421::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913424::DEBUG::2013-11-20 03:04:44,422::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913424::DEBUG::2013-11-20 03:04:44,422::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7a046fed-ec53-4b4d-92f2-551f723c4c3b`::Granted request
>Thread-15913424::DEBUG::2013-11-20 03:04:44,423::task::495::TaskManager.Task::(_debug) Task c9830739-0e42-4874-9439-2d920f5ea0ee: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913424::DEBUG::2013-11-20 03:04:44,424::task::495::TaskManager.Task::(_debug) Task c9830739-0e42-4874-9439-2d920f5ea0ee: ref 1 aborting False
>Thread-15913424::WARNING::2013-11-20 03:04:44,426::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913424::DEBUG::2013-11-20 03:04:44,427::task::495::TaskManager.Task::(_debug) Task c9830739-0e42-4874-9439-2d920f5ea0ee: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913424::DEBUG::2013-11-20 03:04:44,427::task::495::TaskManager.Task::(_debug) Task c9830739-0e42-4874-9439-2d920f5ea0ee: moving from state preparing -> state finished
>Thread-15913424::DEBUG::2013-11-20 03:04:44,428::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913424::DEBUG::2013-11-20 03:04:44,428::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913424::DEBUG::2013-11-20 03:04:44,429::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913424::DEBUG::2013-11-20 03:04:44,430::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913424::DEBUG::2013-11-20 03:04:44,430::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913424::DEBUG::2013-11-20 03:04:44,431::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913424::DEBUG::2013-11-20 03:04:44,431::task::495::TaskManager.Task::(_debug) Task c9830739-0e42-4874-9439-2d920f5ea0ee: ref 0 aborting False
>Thread-15913424::INFO::2013-11-20 03:04:44,432::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:04:45,424::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:04:45,951::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0864471 s, 11.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:04:47,966::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:04:48,481::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0802927 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:04:50,496::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:04:51,057::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0871984 s, 11.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:04:53,072::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:04:53,547::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.071315 s, 14.4 MB/s\n'; <rc> = 0
>Thread-15913428::INFO::2013-11-20 03:04:53,778::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913428::DEBUG::2013-11-20 03:04:53,779::task::495::TaskManager.Task::(_debug) Task e7db8e07-07e1-4cc2-a2fd-3e826426053b: moving from state init -> state preparing
>Thread-15913428::DEBUG::2013-11-20 03:04:53,780::task::495::TaskManager.Task::(_debug) Task e7db8e07-07e1-4cc2-a2fd-3e826426053b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000581026077271', 'lastCheck': 1384916693.7411571, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142788887024', 'lastCheck': 1384916691.179208, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0096869468689', 'lastCheck': 1384916692.1922121, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000582933425903', 'lastCheck': 1384916691.733341, 'valid': True, 'code': 0}}
>Thread-15913428::DEBUG::2013-11-20 03:04:53,781::task::495::TaskManager.Task::(_debug) Task e7db8e07-07e1-4cc2-a2fd-3e826426053b: moving from state preparing -> state finished
>Thread-15913428::DEBUG::2013-11-20 03:04:53,781::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913428::DEBUG::2013-11-20 03:04:53,782::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913428::DEBUG::2013-11-20 03:04:53,782::task::495::TaskManager.Task::(_debug) Task e7db8e07-07e1-4cc2-a2fd-3e826426053b: ref 0 aborting False
>Thread-15913428::INFO::2013-11-20 03:04:53,783::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000581026077271', 'lastCheck': 1384916693.7411571, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142788887024', 'lastCheck': 1384916691.179208, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0096869468689', 'lastCheck': 1384916692.1922121, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000582933425903', 'lastCheck': 1384916691.733341, 'valid': True, 'code': 0}}
>Thread-15913430::DEBUG::2013-11-20 03:04:54,789::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913430::INFO::2013-11-20 03:04:54,790::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913430::DEBUG::2013-11-20 03:04:54,790::task::495::TaskManager.Task::(_debug) Task 1199d935-15d6-49e8-af4a-8756b7fab69a: moving from state init -> state preparing
>Thread-15913430::DEBUG::2013-11-20 03:04:54,791::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913430::DEBUG::2013-11-20 03:04:54,792::task::495::TaskManager.Task::(_debug) Task 1199d935-15d6-49e8-af4a-8756b7fab69a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913430::DEBUG::2013-11-20 03:04:54,792::task::495::TaskManager.Task::(_debug) Task 1199d935-15d6-49e8-af4a-8756b7fab69a: moving from state preparing -> state finished
>Thread-15913430::DEBUG::2013-11-20 03:04:54,793::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913430::DEBUG::2013-11-20 03:04:54,793::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913430::DEBUG::2013-11-20 03:04:54,794::task::495::TaskManager.Task::(_debug) Task 1199d935-15d6-49e8-af4a-8756b7fab69a: ref 0 aborting False
>Thread-15913430::INFO::2013-11-20 03:04:54,795::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913431::DEBUG::2013-11-20 03:04:54,799::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913431::INFO::2013-11-20 03:04:54,800::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913431::DEBUG::2013-11-20 03:04:54,800::task::495::TaskManager.Task::(_debug) Task eb48afc5-5a9e-4c9c-b26f-c32e744d18ae: moving from state init -> state preparing
>Thread-15913431::DEBUG::2013-11-20 03:04:54,802::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`66078b24-cbd6-4c10-8245-ee02ce1a4483`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913431::DEBUG::2013-11-20 03:04:54,803::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913431::DEBUG::2013-11-20 03:04:54,804::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913431::DEBUG::2013-11-20 03:04:54,805::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`66078b24-cbd6-4c10-8245-ee02ce1a4483`::Granted request
>Thread-15913431::DEBUG::2013-11-20 03:04:54,807::task::495::TaskManager.Task::(_debug) Task eb48afc5-5a9e-4c9c-b26f-c32e744d18ae: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913431::DEBUG::2013-11-20 03:04:54,808::task::495::TaskManager.Task::(_debug) Task eb48afc5-5a9e-4c9c-b26f-c32e744d18ae: ref 1 aborting False
>Thread-15913431::WARNING::2013-11-20 03:04:54,811::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913431::DEBUG::2013-11-20 03:04:54,813::task::495::TaskManager.Task::(_debug) Task eb48afc5-5a9e-4c9c-b26f-c32e744d18ae: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913431::DEBUG::2013-11-20 03:04:54,814::task::495::TaskManager.Task::(_debug) Task eb48afc5-5a9e-4c9c-b26f-c32e744d18ae: moving from state preparing -> state finished
>Thread-15913431::DEBUG::2013-11-20 03:04:54,816::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913431::DEBUG::2013-11-20 03:04:54,817::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913431::DEBUG::2013-11-20 03:04:54,819::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913431::DEBUG::2013-11-20 03:04:54,820::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913431::DEBUG::2013-11-20 03:04:54,821::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913431::DEBUG::2013-11-20 03:04:54,822::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913431::DEBUG::2013-11-20 03:04:54,824::task::495::TaskManager.Task::(_debug) Task eb48afc5-5a9e-4c9c-b26f-c32e744d18ae: ref 0 aborting False
>Thread-15913431::INFO::2013-11-20 03:04:54,825::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:04:55,563::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:04:56,091::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0704603 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:04:58,105::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:04:58,613::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0683611 s, 15.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:05:00,623::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:05:01,168::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0728484 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:05:03,183::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:05:03,714::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0733808 s, 14.0 MB/s\n'; <rc> = 0
>Thread-15913435::DEBUG::2013-11-20 03:05:05,261::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913435::INFO::2013-11-20 03:05:05,263::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913435::DEBUG::2013-11-20 03:05:05,264::task::495::TaskManager.Task::(_debug) Task 25742598-5de9-4f5e-9273-fa0e7f429bb4: moving from state init -> state preparing
>Thread-15913435::DEBUG::2013-11-20 03:05:05,264::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913435::DEBUG::2013-11-20 03:05:05,265::task::495::TaskManager.Task::(_debug) Task 25742598-5de9-4f5e-9273-fa0e7f429bb4: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913435::DEBUG::2013-11-20 03:05:05,266::task::495::TaskManager.Task::(_debug) Task 25742598-5de9-4f5e-9273-fa0e7f429bb4: moving from state preparing -> state finished
>Thread-15913435::DEBUG::2013-11-20 03:05:05,266::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913435::DEBUG::2013-11-20 03:05:05,267::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913435::DEBUG::2013-11-20 03:05:05,267::task::495::TaskManager.Task::(_debug) Task 25742598-5de9-4f5e-9273-fa0e7f429bb4: ref 0 aborting False
>Thread-15913435::INFO::2013-11-20 03:05:05,268::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913436::DEBUG::2013-11-20 03:05:05,276::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913436::INFO::2013-11-20 03:05:05,277::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913436::DEBUG::2013-11-20 03:05:05,278::task::495::TaskManager.Task::(_debug) Task 30f65d7d-351d-4d95-95f0-5b63eea0573f: moving from state init -> state preparing
>Thread-15913436::DEBUG::2013-11-20 03:05:05,279::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`41f62e80-9468-4969-a90f-dc35195e4110`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913436::DEBUG::2013-11-20 03:05:05,280::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913436::DEBUG::2013-11-20 03:05:05,281::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913436::DEBUG::2013-11-20 03:05:05,281::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`41f62e80-9468-4969-a90f-dc35195e4110`::Granted request
>Thread-15913436::DEBUG::2013-11-20 03:05:05,282::task::495::TaskManager.Task::(_debug) Task 30f65d7d-351d-4d95-95f0-5b63eea0573f: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913436::DEBUG::2013-11-20 03:05:05,283::task::495::TaskManager.Task::(_debug) Task 30f65d7d-351d-4d95-95f0-5b63eea0573f: ref 1 aborting False
>Thread-15913436::WARNING::2013-11-20 03:05:05,285::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913436::DEBUG::2013-11-20 03:05:05,285::task::495::TaskManager.Task::(_debug) Task 30f65d7d-351d-4d95-95f0-5b63eea0573f: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913436::DEBUG::2013-11-20 03:05:05,286::task::495::TaskManager.Task::(_debug) Task 30f65d7d-351d-4d95-95f0-5b63eea0573f: moving from state preparing -> state finished
>Thread-15913436::DEBUG::2013-11-20 03:05:05,287::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913436::DEBUG::2013-11-20 03:05:05,287::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913436::DEBUG::2013-11-20 03:05:05,288::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913436::DEBUG::2013-11-20 03:05:05,289::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913436::DEBUG::2013-11-20 03:05:05,289::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913436::DEBUG::2013-11-20 03:05:05,290::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913436::DEBUG::2013-11-20 03:05:05,290::task::495::TaskManager.Task::(_debug) Task 30f65d7d-351d-4d95-95f0-5b63eea0573f: ref 0 aborting False
>Thread-15913436::INFO::2013-11-20 03:05:05,291::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:05:05,728::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:05:06,253::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0768855 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:05:08,268::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:05:08,803::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0751446 s, 13.6 MB/s\n'; <rc> = 0
>Thread-15913438::INFO::2013-11-20 03:05:09,454::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913438::DEBUG::2013-11-20 03:05:09,455::task::495::TaskManager.Task::(_debug) Task bd0bc79d-f044-4498-a5a5-b3c2a7bcfd03: moving from state init -> state preparing
>Thread-15913438::DEBUG::2013-11-20 03:05:09,455::task::495::TaskManager.Task::(_debug) Task bd0bc79d-f044-4498-a5a5-b3c2a7bcfd03: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000415802001953', 'lastCheck': 1384916703.865972, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00167083740234', 'lastCheck': 1384916701.332516, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00159215927124', 'lastCheck': 1384916702.3446119, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000380992889404', 'lastCheck': 1384916701.8539801, 'valid': True, 'code': 0}}
>Thread-15913438::DEBUG::2013-11-20 03:05:09,456::task::495::TaskManager.Task::(_debug) Task bd0bc79d-f044-4498-a5a5-b3c2a7bcfd03: moving from state preparing -> state finished
>Thread-15913438::DEBUG::2013-11-20 03:05:09,457::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913438::DEBUG::2013-11-20 03:05:09,457::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913438::DEBUG::2013-11-20 03:05:09,458::task::495::TaskManager.Task::(_debug) Task bd0bc79d-f044-4498-a5a5-b3c2a7bcfd03: ref 0 aborting False
>Thread-15913438::INFO::2013-11-20 03:05:09,458::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000415802001953', 'lastCheck': 1384916703.865972, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00167083740234', 'lastCheck': 1384916701.332516, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00159215927124', 'lastCheck': 1384916702.3446119, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000380992889404', 'lastCheck': 1384916701.8539801, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:05:10,819::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:05:11,353::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0694019 s, 14.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:05:13,368::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:05:13,916::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0862804 s, 11.9 MB/s\n'; <rc> = 0
>Thread-15913441::DEBUG::2013-11-20 03:05:15,764::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913441::INFO::2013-11-20 03:05:15,765::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913441::DEBUG::2013-11-20 03:05:15,766::task::495::TaskManager.Task::(_debug) Task eff70ab2-31eb-4eda-9a54-653827fdc925: moving from state init -> state preparing
>Thread-15913441::DEBUG::2013-11-20 03:05:15,766::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913441::DEBUG::2013-11-20 03:05:15,767::task::495::TaskManager.Task::(_debug) Task eff70ab2-31eb-4eda-9a54-653827fdc925: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913441::DEBUG::2013-11-20 03:05:15,767::task::495::TaskManager.Task::(_debug) Task eff70ab2-31eb-4eda-9a54-653827fdc925: moving from state preparing -> state finished
>Thread-15913441::DEBUG::2013-11-20 03:05:15,768::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913441::DEBUG::2013-11-20 03:05:15,769::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913441::DEBUG::2013-11-20 03:05:15,769::task::495::TaskManager.Task::(_debug) Task eff70ab2-31eb-4eda-9a54-653827fdc925: ref 0 aborting False
>Thread-15913441::INFO::2013-11-20 03:05:15,770::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913442::DEBUG::2013-11-20 03:05:15,774::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913442::INFO::2013-11-20 03:05:15,775::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913442::DEBUG::2013-11-20 03:05:15,776::task::495::TaskManager.Task::(_debug) Task b1c09bdf-c14d-408d-aa4f-94c869640ede: moving from state init -> state preparing
>Thread-15913442::DEBUG::2013-11-20 03:05:15,777::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a1cfadab-b702-4377-96ff-d12a0d7b1bdf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913442::DEBUG::2013-11-20 03:05:15,778::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913442::DEBUG::2013-11-20 03:05:15,778::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913442::DEBUG::2013-11-20 03:05:15,779::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a1cfadab-b702-4377-96ff-d12a0d7b1bdf`::Granted request
>Thread-15913442::DEBUG::2013-11-20 03:05:15,780::task::495::TaskManager.Task::(_debug) Task b1c09bdf-c14d-408d-aa4f-94c869640ede: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913442::DEBUG::2013-11-20 03:05:15,780::task::495::TaskManager.Task::(_debug) Task b1c09bdf-c14d-408d-aa4f-94c869640ede: ref 1 aborting False
>Thread-15913442::WARNING::2013-11-20 03:05:15,782::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913442::DEBUG::2013-11-20 03:05:15,783::task::495::TaskManager.Task::(_debug) Task b1c09bdf-c14d-408d-aa4f-94c869640ede: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913442::DEBUG::2013-11-20 03:05:15,784::task::495::TaskManager.Task::(_debug) Task b1c09bdf-c14d-408d-aa4f-94c869640ede: moving from state preparing -> state finished
>Thread-15913442::DEBUG::2013-11-20 03:05:15,785::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913442::DEBUG::2013-11-20 03:05:15,785::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913442::DEBUG::2013-11-20 03:05:15,786::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913442::DEBUG::2013-11-20 03:05:15,786::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913442::DEBUG::2013-11-20 03:05:15,787::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913442::DEBUG::2013-11-20 03:05:15,787::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913442::DEBUG::2013-11-20 03:05:15,788::task::495::TaskManager.Task::(_debug) Task b1c09bdf-c14d-408d-aa4f-94c869640ede: ref 0 aborting False
>Thread-15913442::INFO::2013-11-20 03:05:15,789::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:05:15,932::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:05:16,417::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0710237 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:05:18,427::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:05:18,964::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0826974 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:05:20,980::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:05:21,500::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.083333 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:05:23,516::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 03:05:23,753::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:05:23,755::task::495::TaskManager.Task::(_debug) Task 609b8572-61c6-47f7-ae6b-8bf533c4bf65: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:05:23,764::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6e821646-d1b4-4659-aca6-62361878de78`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:05:23,765::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:05:23,766::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:05:23,766::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6e821646-d1b4-4659-aca6-62361878de78`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:05:23,768::task::495::TaskManager.Task::(_debug) Task 609b8572-61c6-47f7-ae6b-8bf533c4bf65: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:05:23,768::task::495::TaskManager.Task::(_debug) Task 609b8572-61c6-47f7-ae6b-8bf533c4bf65: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:05:23,769::task::495::TaskManager.Task::(_debug) Task 609b8572-61c6-47f7-ae6b-8bf533c4bf65: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:05:23,769::task::495::TaskManager.Task::(_debug) Task 609b8572-61c6-47f7-ae6b-8bf533c4bf65: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:05:23,770::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:05:23,771::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:05:23,771::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:05:23,772::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:05:23,772::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:05:23,773::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:05:23,774::task::495::TaskManager.Task::(_debug) Task 609b8572-61c6-47f7-ae6b-8bf533c4bf65: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:05:23,774::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:05:24,097::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.100523 s, 10.2 MB/s\n'; <rc> = 0
>Thread-15913446::INFO::2013-11-20 03:05:25,183::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913446::DEBUG::2013-11-20 03:05:25,184::task::495::TaskManager.Task::(_debug) Task 27ead4e8-90b4-464d-ba50-b841c594e44d: moving from state init -> state preparing
>Thread-15913446::DEBUG::2013-11-20 03:05:25,184::task::495::TaskManager.Task::(_debug) Task 27ead4e8-90b4-464d-ba50-b841c594e44d: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000411033630371', 'lastCheck': 1384916724.1854701, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00159096717834', 'lastCheck': 1384916721.6453769, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00164198875427', 'lastCheck': 1384916722.6457801, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000447034835815', 'lastCheck': 1384916722.178664, 'valid': True, 'code': 0}}
>Thread-15913446::DEBUG::2013-11-20 03:05:25,185::task::495::TaskManager.Task::(_debug) Task 27ead4e8-90b4-464d-ba50-b841c594e44d: moving from state preparing -> state finished
>Thread-15913446::DEBUG::2013-11-20 03:05:25,186::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913446::DEBUG::2013-11-20 03:05:25,186::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913446::DEBUG::2013-11-20 03:05:25,187::task::495::TaskManager.Task::(_debug) Task 27ead4e8-90b4-464d-ba50-b841c594e44d: ref 0 aborting False
>Thread-15913446::INFO::2013-11-20 03:05:25,188::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000411033630371', 'lastCheck': 1384916724.1854701, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00159096717834', 'lastCheck': 1384916721.6453769, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00164198875427', 'lastCheck': 1384916722.6457801, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000447034835815', 'lastCheck': 1384916722.178664, 'valid': True, 'code': 0}}
>Thread-15913448::DEBUG::2013-11-20 03:05:25,872::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913448::INFO::2013-11-20 03:05:25,874::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913448::DEBUG::2013-11-20 03:05:25,874::task::495::TaskManager.Task::(_debug) Task 1d4e2abf-7682-47a3-b404-7286efdafda4: moving from state init -> state preparing
>Thread-15913448::DEBUG::2013-11-20 03:05:25,875::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913448::DEBUG::2013-11-20 03:05:25,875::task::495::TaskManager.Task::(_debug) Task 1d4e2abf-7682-47a3-b404-7286efdafda4: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913448::DEBUG::2013-11-20 03:05:25,876::task::495::TaskManager.Task::(_debug) Task 1d4e2abf-7682-47a3-b404-7286efdafda4: moving from state preparing -> state finished
>Thread-15913448::DEBUG::2013-11-20 03:05:25,877::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913448::DEBUG::2013-11-20 03:05:25,877::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913448::DEBUG::2013-11-20 03:05:25,878::task::495::TaskManager.Task::(_debug) Task 1d4e2abf-7682-47a3-b404-7286efdafda4: ref 0 aborting False
>Thread-15913448::INFO::2013-11-20 03:05:25,878::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913449::DEBUG::2013-11-20 03:05:25,883::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913449::INFO::2013-11-20 03:05:25,884::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913449::DEBUG::2013-11-20 03:05:25,884::task::495::TaskManager.Task::(_debug) Task fbaaf9ec-a99e-4a90-8648-ce5addac640d: moving from state init -> state preparing
>Thread-15913449::DEBUG::2013-11-20 03:05:25,885::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5c9d9f1b-ab04-4230-b2e1-1e709d5937d8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913449::DEBUG::2013-11-20 03:05:25,886::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913449::DEBUG::2013-11-20 03:05:25,887::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913449::DEBUG::2013-11-20 03:05:25,887::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5c9d9f1b-ab04-4230-b2e1-1e709d5937d8`::Granted request
>Thread-15913449::DEBUG::2013-11-20 03:05:25,888::task::495::TaskManager.Task::(_debug) Task fbaaf9ec-a99e-4a90-8648-ce5addac640d: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913449::DEBUG::2013-11-20 03:05:25,889::task::495::TaskManager.Task::(_debug) Task fbaaf9ec-a99e-4a90-8648-ce5addac640d: ref 1 aborting False
>Thread-15913449::WARNING::2013-11-20 03:05:25,891::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913449::DEBUG::2013-11-20 03:05:25,891::task::495::TaskManager.Task::(_debug) Task fbaaf9ec-a99e-4a90-8648-ce5addac640d: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913449::DEBUG::2013-11-20 03:05:25,892::task::495::TaskManager.Task::(_debug) Task fbaaf9ec-a99e-4a90-8648-ce5addac640d: moving from state preparing -> state finished
>Thread-15913449::DEBUG::2013-11-20 03:05:25,893::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913449::DEBUG::2013-11-20 03:05:25,893::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913449::DEBUG::2013-11-20 03:05:25,894::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913449::DEBUG::2013-11-20 03:05:25,895::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913449::DEBUG::2013-11-20 03:05:25,895::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913449::DEBUG::2013-11-20 03:05:25,896::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913449::DEBUG::2013-11-20 03:05:25,896::task::495::TaskManager.Task::(_debug) Task fbaaf9ec-a99e-4a90-8648-ce5addac640d: ref 0 aborting False
>Thread-15913449::INFO::2013-11-20 03:05:25,897::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:05:26,108::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:05:26,629::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0707103 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:05:28,644::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:05:29,160::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0546376 s, 18.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:05:31,171::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:05:31,717::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0749569 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:05:33,732::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:05:34,231::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0587828 s, 17.4 MB/s\n'; <rc> = 0
>Thread-15913453::DEBUG::2013-11-20 03:05:36,015::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913453::INFO::2013-11-20 03:05:36,017::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913453::DEBUG::2013-11-20 03:05:36,018::task::495::TaskManager.Task::(_debug) Task dba732b3-a64d-4464-8548-1c56c8d6b86a: moving from state init -> state preparing
>Thread-15913453::DEBUG::2013-11-20 03:05:36,019::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913453::DEBUG::2013-11-20 03:05:36,019::task::495::TaskManager.Task::(_debug) Task dba732b3-a64d-4464-8548-1c56c8d6b86a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913453::DEBUG::2013-11-20 03:05:36,020::task::495::TaskManager.Task::(_debug) Task dba732b3-a64d-4464-8548-1c56c8d6b86a: moving from state preparing -> state finished
>Thread-15913453::DEBUG::2013-11-20 03:05:36,021::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913453::DEBUG::2013-11-20 03:05:36,021::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913453::DEBUG::2013-11-20 03:05:36,022::task::495::TaskManager.Task::(_debug) Task dba732b3-a64d-4464-8548-1c56c8d6b86a: ref 0 aborting False
>Thread-15913453::INFO::2013-11-20 03:05:36,022::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913454::DEBUG::2013-11-20 03:05:36,027::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913454::INFO::2013-11-20 03:05:36,028::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913454::DEBUG::2013-11-20 03:05:36,028::task::495::TaskManager.Task::(_debug) Task 1f330e1d-18a3-41d5-afd5-b4e91f663076: moving from state init -> state preparing
>Thread-15913454::DEBUG::2013-11-20 03:05:36,030::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b62667df-4ba9-4d10-a1de-08095b117c52`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913454::DEBUG::2013-11-20 03:05:36,030::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913454::DEBUG::2013-11-20 03:05:36,031::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913454::DEBUG::2013-11-20 03:05:36,032::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b62667df-4ba9-4d10-a1de-08095b117c52`::Granted request
>Thread-15913454::DEBUG::2013-11-20 03:05:36,033::task::495::TaskManager.Task::(_debug) Task 1f330e1d-18a3-41d5-afd5-b4e91f663076: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913454::DEBUG::2013-11-20 03:05:36,033::task::495::TaskManager.Task::(_debug) Task 1f330e1d-18a3-41d5-afd5-b4e91f663076: ref 1 aborting False
>Thread-15913454::WARNING::2013-11-20 03:05:36,035::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913454::DEBUG::2013-11-20 03:05:36,036::task::495::TaskManager.Task::(_debug) Task 1f330e1d-18a3-41d5-afd5-b4e91f663076: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913454::DEBUG::2013-11-20 03:05:36,037::task::495::TaskManager.Task::(_debug) Task 1f330e1d-18a3-41d5-afd5-b4e91f663076: moving from state preparing -> state finished
>Thread-15913454::DEBUG::2013-11-20 03:05:36,037::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913454::DEBUG::2013-11-20 03:05:36,038::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913454::DEBUG::2013-11-20 03:05:36,039::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913454::DEBUG::2013-11-20 03:05:36,039::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913454::DEBUG::2013-11-20 03:05:36,040::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913454::DEBUG::2013-11-20 03:05:36,040::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913454::DEBUG::2013-11-20 03:05:36,041::task::495::TaskManager.Task::(_debug) Task 1f330e1d-18a3-41d5-afd5-b4e91f663076: ref 0 aborting False
>Thread-15913454::INFO::2013-11-20 03:05:36,041::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:05:36,246::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:05:36,818::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0825946 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:05:38,833::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:05:39,382::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0769691 s, 13.3 MB/s\n'; <rc> = 0
>Thread-15913456::INFO::2013-11-20 03:05:40,485::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913456::DEBUG::2013-11-20 03:05:40,486::task::495::TaskManager.Task::(_debug) Task 26575f63-fa9d-475a-8094-a799ebae4d65: moving from state init -> state preparing
>Thread-15913456::DEBUG::2013-11-20 03:05:40,487::task::495::TaskManager.Task::(_debug) Task 26575f63-fa9d-475a-8094-a799ebae4d65: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000312805175781', 'lastCheck': 1384916734.4314971, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00180983543396', 'lastCheck': 1384916731.899389, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00146317481995', 'lastCheck': 1384916732.8943069, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000438213348389', 'lastCheck': 1384916732.404979, 'valid': True, 'code': 0}}
>Thread-15913456::DEBUG::2013-11-20 03:05:40,487::task::495::TaskManager.Task::(_debug) Task 26575f63-fa9d-475a-8094-a799ebae4d65: moving from state preparing -> state finished
>Thread-15913456::DEBUG::2013-11-20 03:05:40,488::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913456::DEBUG::2013-11-20 03:05:40,489::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913456::DEBUG::2013-11-20 03:05:40,489::task::495::TaskManager.Task::(_debug) Task 26575f63-fa9d-475a-8094-a799ebae4d65: ref 0 aborting False
>Thread-15913456::INFO::2013-11-20 03:05:40,490::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000312805175781', 'lastCheck': 1384916734.4314971, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00180983543396', 'lastCheck': 1384916731.899389, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00146317481995', 'lastCheck': 1384916732.8943069, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000438213348389', 'lastCheck': 1384916732.404979, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:05:41,397::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:05:41,968::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0792528 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:05:43,980::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:05:44,597::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0915021 s, 11.2 MB/s\n'; <rc> = 0
>Thread-15913459::DEBUG::2013-11-20 03:05:46,225::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913459::INFO::2013-11-20 03:05:46,226::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913459::DEBUG::2013-11-20 03:05:46,227::task::495::TaskManager.Task::(_debug) Task 7c168868-a407-4a5b-88d5-52196e2cf631: moving from state init -> state preparing
>Thread-15913459::DEBUG::2013-11-20 03:05:46,228::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913459::DEBUG::2013-11-20 03:05:46,228::task::495::TaskManager.Task::(_debug) Task 7c168868-a407-4a5b-88d5-52196e2cf631: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913459::DEBUG::2013-11-20 03:05:46,229::task::495::TaskManager.Task::(_debug) Task 7c168868-a407-4a5b-88d5-52196e2cf631: moving from state preparing -> state finished
>Thread-15913459::DEBUG::2013-11-20 03:05:46,229::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913459::DEBUG::2013-11-20 03:05:46,230::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913459::DEBUG::2013-11-20 03:05:46,230::task::495::TaskManager.Task::(_debug) Task 7c168868-a407-4a5b-88d5-52196e2cf631: ref 0 aborting False
>Thread-15913459::INFO::2013-11-20 03:05:46,231::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913460::DEBUG::2013-11-20 03:05:46,236::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913460::INFO::2013-11-20 03:05:46,236::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913460::DEBUG::2013-11-20 03:05:46,237::task::495::TaskManager.Task::(_debug) Task 2128abb8-1c96-4e5a-84be-8e39cf093eaa: moving from state init -> state preparing
>Thread-15913460::DEBUG::2013-11-20 03:05:46,238::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`83f77196-2a55-46c9-9b31-eed1ce997888`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913460::DEBUG::2013-11-20 03:05:46,239::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913460::DEBUG::2013-11-20 03:05:46,240::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913460::DEBUG::2013-11-20 03:05:46,240::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`83f77196-2a55-46c9-9b31-eed1ce997888`::Granted request
>Thread-15913460::DEBUG::2013-11-20 03:05:46,241::task::495::TaskManager.Task::(_debug) Task 2128abb8-1c96-4e5a-84be-8e39cf093eaa: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913460::DEBUG::2013-11-20 03:05:46,242::task::495::TaskManager.Task::(_debug) Task 2128abb8-1c96-4e5a-84be-8e39cf093eaa: ref 1 aborting False
>Thread-15913460::WARNING::2013-11-20 03:05:46,244::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913460::DEBUG::2013-11-20 03:05:46,244::task::495::TaskManager.Task::(_debug) Task 2128abb8-1c96-4e5a-84be-8e39cf093eaa: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913460::DEBUG::2013-11-20 03:05:46,245::task::495::TaskManager.Task::(_debug) Task 2128abb8-1c96-4e5a-84be-8e39cf093eaa: moving from state preparing -> state finished
>Thread-15913460::DEBUG::2013-11-20 03:05:46,246::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913460::DEBUG::2013-11-20 03:05:46,246::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913460::DEBUG::2013-11-20 03:05:46,247::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913460::DEBUG::2013-11-20 03:05:46,248::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913460::DEBUG::2013-11-20 03:05:46,248::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913460::DEBUG::2013-11-20 03:05:46,249::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913460::DEBUG::2013-11-20 03:05:46,249::task::495::TaskManager.Task::(_debug) Task 2128abb8-1c96-4e5a-84be-8e39cf093eaa: ref 0 aborting False
>Thread-15913460::INFO::2013-11-20 03:05:46,250::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:05:46,613::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:05:47,094::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.057228 s, 17.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:05:49,108::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:05:49,622::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0775906 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:05:51,637::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:05:52,171::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0706431 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:05:54,186::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:05:54,718::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0757148 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15913464::INFO::2013-11-20 03:05:55,916::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913464::DEBUG::2013-11-20 03:05:55,917::task::495::TaskManager.Task::(_debug) Task 2113437d-98d1-42a7-9bd1-bc3377c93ab4: moving from state init -> state preparing
>Thread-15913464::DEBUG::2013-11-20 03:05:55,918::task::495::TaskManager.Task::(_debug) Task 2113437d-98d1-42a7-9bd1-bc3377c93ab4: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000388860702515', 'lastCheck': 1384916754.8622799, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00149893760681', 'lastCheck': 1384916752.3143871, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0122559070587', 'lastCheck': 1384916753.325098, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000472068786621', 'lastCheck': 1384916752.8574569, 'valid': True, 'code': 0}}
>Thread-15913464::DEBUG::2013-11-20 03:05:55,919::task::495::TaskManager.Task::(_debug) Task 2113437d-98d1-42a7-9bd1-bc3377c93ab4: moving from state preparing -> state finished
>Thread-15913464::DEBUG::2013-11-20 03:05:55,919::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913464::DEBUG::2013-11-20 03:05:55,920::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913464::DEBUG::2013-11-20 03:05:55,921::task::495::TaskManager.Task::(_debug) Task 2113437d-98d1-42a7-9bd1-bc3377c93ab4: ref 0 aborting False
>Thread-15913464::INFO::2013-11-20 03:05:55,922::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000388860702515', 'lastCheck': 1384916754.8622799, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00149893760681', 'lastCheck': 1384916752.3143871, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0122559070587', 'lastCheck': 1384916753.325098, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000472068786621', 'lastCheck': 1384916752.8574569, 'valid': True, 'code': 0}}
>Thread-15913466::DEBUG::2013-11-20 03:05:56,699::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913466::INFO::2013-11-20 03:05:56,700::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913466::DEBUG::2013-11-20 03:05:56,709::task::495::TaskManager.Task::(_debug) Task 6d17db36-b18e-47ca-8e08-e2aba1c8d71e: moving from state init -> state preparing
>Thread-15913466::DEBUG::2013-11-20 03:05:56,710::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913466::DEBUG::2013-11-20 03:05:56,711::task::495::TaskManager.Task::(_debug) Task 6d17db36-b18e-47ca-8e08-e2aba1c8d71e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913466::DEBUG::2013-11-20 03:05:56,712::task::495::TaskManager.Task::(_debug) Task 6d17db36-b18e-47ca-8e08-e2aba1c8d71e: moving from state preparing -> state finished
>Thread-15913466::DEBUG::2013-11-20 03:05:56,713::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913466::DEBUG::2013-11-20 03:05:56,714::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913466::DEBUG::2013-11-20 03:05:56,715::task::495::TaskManager.Task::(_debug) Task 6d17db36-b18e-47ca-8e08-e2aba1c8d71e: ref 0 aborting False
>Thread-15913466::INFO::2013-11-20 03:05:56,716::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913467::DEBUG::2013-11-20 03:05:56,722::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913467::INFO::2013-11-20 03:05:56,724::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913467::DEBUG::2013-11-20 03:05:56,724::task::495::TaskManager.Task::(_debug) Task 1e4e6b5a-6898-493f-870b-c17d14ce3b57: moving from state init -> state preparing
>Thread-15913467::DEBUG::2013-11-20 03:05:56,726::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`dc258d50-fa5c-40a3-8ef4-f86edd11b622`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913467::DEBUG::2013-11-20 03:05:56,727::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913467::DEBUG::2013-11-20 03:05:56,729::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913467::DEBUG::2013-11-20 03:05:56,729::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`dc258d50-fa5c-40a3-8ef4-f86edd11b622`::Granted request
>Dummy-15912651::DEBUG::2013-11-20 03:05:56,738::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913467::DEBUG::2013-11-20 03:05:56,972::task::495::TaskManager.Task::(_debug) Task 1e4e6b5a-6898-493f-870b-c17d14ce3b57: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913467::DEBUG::2013-11-20 03:05:56,974::task::495::TaskManager.Task::(_debug) Task 1e4e6b5a-6898-493f-870b-c17d14ce3b57: ref 1 aborting False
>Thread-15913467::WARNING::2013-11-20 03:05:56,976::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913467::DEBUG::2013-11-20 03:05:56,977::task::495::TaskManager.Task::(_debug) Task 1e4e6b5a-6898-493f-870b-c17d14ce3b57: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913467::DEBUG::2013-11-20 03:05:56,977::task::495::TaskManager.Task::(_debug) Task 1e4e6b5a-6898-493f-870b-c17d14ce3b57: moving from state preparing -> state finished
>Thread-15913467::DEBUG::2013-11-20 03:05:56,978::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913467::DEBUG::2013-11-20 03:05:56,978::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913467::DEBUG::2013-11-20 03:05:56,979::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913467::DEBUG::2013-11-20 03:05:56,979::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913467::DEBUG::2013-11-20 03:05:56,979::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913467::DEBUG::2013-11-20 03:05:56,980::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913467::DEBUG::2013-11-20 03:05:56,980::task::495::TaskManager.Task::(_debug) Task 1e4e6b5a-6898-493f-870b-c17d14ce3b57: ref 0 aborting False
>Thread-15913467::INFO::2013-11-20 03:05:56,981::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:05:57,267::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0735358 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:05:59,282::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:05:59,823::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.074823 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:06:01,838::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:06:02,375::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0777546 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:06:04,390::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:06:05,017::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0836346 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:06:07,037::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:06:07,583::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0777265 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15913471::DEBUG::2013-11-20 03:06:07,766::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913471::INFO::2013-11-20 03:06:07,768::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913471::DEBUG::2013-11-20 03:06:07,769::task::495::TaskManager.Task::(_debug) Task dce53b82-ab73-4cc9-9e84-b5ba207e957b: moving from state init -> state preparing
>Thread-15913471::DEBUG::2013-11-20 03:06:07,770::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913471::DEBUG::2013-11-20 03:06:07,770::task::495::TaskManager.Task::(_debug) Task dce53b82-ab73-4cc9-9e84-b5ba207e957b: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913471::DEBUG::2013-11-20 03:06:07,771::task::495::TaskManager.Task::(_debug) Task dce53b82-ab73-4cc9-9e84-b5ba207e957b: moving from state preparing -> state finished
>Thread-15913471::DEBUG::2013-11-20 03:06:07,771::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913471::DEBUG::2013-11-20 03:06:07,772::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913471::DEBUG::2013-11-20 03:06:07,772::task::495::TaskManager.Task::(_debug) Task dce53b82-ab73-4cc9-9e84-b5ba207e957b: ref 0 aborting False
>Thread-15913471::INFO::2013-11-20 03:06:07,773::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913472::DEBUG::2013-11-20 03:06:07,777::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913472::INFO::2013-11-20 03:06:07,778::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913472::DEBUG::2013-11-20 03:06:07,779::task::495::TaskManager.Task::(_debug) Task b9c79cd2-a4d0-4a2f-9111-aa9179e550d0: moving from state init -> state preparing
>Thread-15913472::DEBUG::2013-11-20 03:06:07,780::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`dfb8855a-5c03-45f6-885f-36c335a1ea6e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913472::DEBUG::2013-11-20 03:06:07,781::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913472::DEBUG::2013-11-20 03:06:07,782::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913472::DEBUG::2013-11-20 03:06:07,782::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`dfb8855a-5c03-45f6-885f-36c335a1ea6e`::Granted request
>Thread-15913472::DEBUG::2013-11-20 03:06:07,783::task::495::TaskManager.Task::(_debug) Task b9c79cd2-a4d0-4a2f-9111-aa9179e550d0: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913472::DEBUG::2013-11-20 03:06:07,784::task::495::TaskManager.Task::(_debug) Task b9c79cd2-a4d0-4a2f-9111-aa9179e550d0: ref 1 aborting False
>Thread-15913472::WARNING::2013-11-20 03:06:07,786::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913472::DEBUG::2013-11-20 03:06:07,786::task::495::TaskManager.Task::(_debug) Task b9c79cd2-a4d0-4a2f-9111-aa9179e550d0: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913472::DEBUG::2013-11-20 03:06:07,787::task::495::TaskManager.Task::(_debug) Task b9c79cd2-a4d0-4a2f-9111-aa9179e550d0: moving from state preparing -> state finished
>Thread-15913472::DEBUG::2013-11-20 03:06:07,788::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913472::DEBUG::2013-11-20 03:06:07,788::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913472::DEBUG::2013-11-20 03:06:07,789::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913472::DEBUG::2013-11-20 03:06:07,790::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913472::DEBUG::2013-11-20 03:06:07,790::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913472::DEBUG::2013-11-20 03:06:07,791::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913472::DEBUG::2013-11-20 03:06:07,791::task::495::TaskManager.Task::(_debug) Task b9c79cd2-a4d0-4a2f-9111-aa9179e550d0: ref 0 aborting False
>Thread-15913472::INFO::2013-11-20 03:06:07,792::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:06:09,599::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:06:10,090::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0757982 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15913474::INFO::2013-11-20 03:06:11,332::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913474::DEBUG::2013-11-20 03:06:11,333::task::495::TaskManager.Task::(_debug) Task 64a28809-dd82-4689-98f9-9a51da88fe19: moving from state init -> state preparing
>Thread-15913474::DEBUG::2013-11-20 03:06:11,334::task::495::TaskManager.Task::(_debug) Task 64a28809-dd82-4689-98f9-9a51da88fe19: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000549077987671', 'lastCheck': 1384916765.0667751, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00141906738281', 'lastCheck': 1384916762.5094309, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00165200233459', 'lastCheck': 1384916763.5119669, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000478982925415', 'lastCheck': 1384916763.0655949, 'valid': True, 'code': 0}}
>Thread-15913474::DEBUG::2013-11-20 03:06:11,335::task::495::TaskManager.Task::(_debug) Task 64a28809-dd82-4689-98f9-9a51da88fe19: moving from state preparing -> state finished
>Thread-15913474::DEBUG::2013-11-20 03:06:11,335::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913474::DEBUG::2013-11-20 03:06:11,336::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913474::DEBUG::2013-11-20 03:06:11,337::task::495::TaskManager.Task::(_debug) Task 64a28809-dd82-4689-98f9-9a51da88fe19: ref 0 aborting False
>Thread-15913474::INFO::2013-11-20 03:06:11,337::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000549077987671', 'lastCheck': 1384916765.0667751, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00141906738281', 'lastCheck': 1384916762.5094309, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00165200233459', 'lastCheck': 1384916763.5119669, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000478982925415', 'lastCheck': 1384916763.0655949, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:06:12,103::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:06:12,633::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0702525 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:06:14,649::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:06:15,214::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0991997 s, 10.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:06:17,229::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:06:17,742::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0732949 s, 14.0 MB/s\n'; <rc> = 0
>Thread-15913478::DEBUG::2013-11-20 03:06:18,004::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913478::INFO::2013-11-20 03:06:18,006::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913478::DEBUG::2013-11-20 03:06:18,007::task::495::TaskManager.Task::(_debug) Task a18f6b27-2be2-4200-bac8-36792255652b: moving from state init -> state preparing
>Thread-15913478::DEBUG::2013-11-20 03:06:18,008::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913478::DEBUG::2013-11-20 03:06:18,008::task::495::TaskManager.Task::(_debug) Task a18f6b27-2be2-4200-bac8-36792255652b: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913478::DEBUG::2013-11-20 03:06:18,009::task::495::TaskManager.Task::(_debug) Task a18f6b27-2be2-4200-bac8-36792255652b: moving from state preparing -> state finished
>Thread-15913478::DEBUG::2013-11-20 03:06:18,010::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913478::DEBUG::2013-11-20 03:06:18,010::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913478::DEBUG::2013-11-20 03:06:18,011::task::495::TaskManager.Task::(_debug) Task a18f6b27-2be2-4200-bac8-36792255652b: ref 0 aborting False
>Thread-15913478::INFO::2013-11-20 03:06:18,011::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913479::DEBUG::2013-11-20 03:06:18,016::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913479::INFO::2013-11-20 03:06:18,017::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913479::DEBUG::2013-11-20 03:06:18,017::task::495::TaskManager.Task::(_debug) Task 2793af67-ee53-460a-b38e-9d75ed2a1c77: moving from state init -> state preparing
>Thread-15913479::DEBUG::2013-11-20 03:06:18,018::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1147906c-7da3-487f-b300-762cc336579c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913479::DEBUG::2013-11-20 03:06:18,019::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913479::DEBUG::2013-11-20 03:06:18,020::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913479::DEBUG::2013-11-20 03:06:18,021::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1147906c-7da3-487f-b300-762cc336579c`::Granted request
>Thread-15913479::DEBUG::2013-11-20 03:06:18,022::task::495::TaskManager.Task::(_debug) Task 2793af67-ee53-460a-b38e-9d75ed2a1c77: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913479::DEBUG::2013-11-20 03:06:18,022::task::495::TaskManager.Task::(_debug) Task 2793af67-ee53-460a-b38e-9d75ed2a1c77: ref 1 aborting False
>Thread-15913479::WARNING::2013-11-20 03:06:18,024::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913479::DEBUG::2013-11-20 03:06:18,025::task::495::TaskManager.Task::(_debug) Task 2793af67-ee53-460a-b38e-9d75ed2a1c77: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913479::DEBUG::2013-11-20 03:06:18,026::task::495::TaskManager.Task::(_debug) Task 2793af67-ee53-460a-b38e-9d75ed2a1c77: moving from state preparing -> state finished
>Thread-15913479::DEBUG::2013-11-20 03:06:18,027::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913479::DEBUG::2013-11-20 03:06:18,027::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913479::DEBUG::2013-11-20 03:06:18,028::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913479::DEBUG::2013-11-20 03:06:18,028::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913479::DEBUG::2013-11-20 03:06:18,029::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913479::DEBUG::2013-11-20 03:06:18,029::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913479::DEBUG::2013-11-20 03:06:18,030::task::495::TaskManager.Task::(_debug) Task 2793af67-ee53-460a-b38e-9d75ed2a1c77: ref 0 aborting False
>Thread-15913479::INFO::2013-11-20 03:06:18,031::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:06:19,757::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:06:20,304::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0760459 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:06:22,319::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:06:22,833::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0575861 s, 17.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:06:24,849::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 03:06:25,068::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:06:25,074::task::495::TaskManager.Task::(_debug) Task 583eefa9-089e-4113-b37f-3c2ada74e4e8: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:06:25,079::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`763b29cc-e72e-40bf-b96e-7c8c7c84e112`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:06:25,080::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:06:25,081::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:06:25,082::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`763b29cc-e72e-40bf-b96e-7c8c7c84e112`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:06:25,084::task::495::TaskManager.Task::(_debug) Task 583eefa9-089e-4113-b37f-3c2ada74e4e8: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:06:25,085::task::495::TaskManager.Task::(_debug) Task 583eefa9-089e-4113-b37f-3c2ada74e4e8: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:06:25,086::task::495::TaskManager.Task::(_debug) Task 583eefa9-089e-4113-b37f-3c2ada74e4e8: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:06:25,087::task::495::TaskManager.Task::(_debug) Task 583eefa9-089e-4113-b37f-3c2ada74e4e8: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:06:25,088::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:06:25,089::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:06:25,090::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:06:25,091::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:06:25,092::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:06:25,093::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:06:25,094::task::495::TaskManager.Task::(_debug) Task 583eefa9-089e-4113-b37f-3c2ada74e4e8: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:06:25,095::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:06:25,371::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0740639 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15913482::INFO::2013-11-20 03:06:26,675::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913482::DEBUG::2013-11-20 03:06:26,676::task::495::TaskManager.Task::(_debug) Task e584abd1-924a-463b-a05f-75a388b5e657: moving from state init -> state preparing
>Thread-15913482::DEBUG::2013-11-20 03:06:26,677::task::495::TaskManager.Task::(_debug) Task e584abd1-924a-463b-a05f-75a388b5e657: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000367879867554', 'lastCheck': 1384916785.5345809, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0115990638733', 'lastCheck': 1384916783.0018809, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00162100791931', 'lastCheck': 1384916783.992455, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000425100326538', 'lastCheck': 1384916783.5292339, 'valid': True, 'code': 0}}
>Thread-15913482::DEBUG::2013-11-20 03:06:26,678::task::495::TaskManager.Task::(_debug) Task e584abd1-924a-463b-a05f-75a388b5e657: moving from state preparing -> state finished
>Thread-15913482::DEBUG::2013-11-20 03:06:26,678::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913482::DEBUG::2013-11-20 03:06:26,679::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913482::DEBUG::2013-11-20 03:06:26,679::task::495::TaskManager.Task::(_debug) Task e584abd1-924a-463b-a05f-75a388b5e657: ref 0 aborting False
>Thread-15913482::INFO::2013-11-20 03:06:26,680::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000367879867554', 'lastCheck': 1384916785.5345809, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0115990638733', 'lastCheck': 1384916783.0018809, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00162100791931', 'lastCheck': 1384916783.992455, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000425100326538', 'lastCheck': 1384916783.5292339, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:06:27,386::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:06:27,948::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0788499 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15913484::DEBUG::2013-11-20 03:06:28,404::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913484::INFO::2013-11-20 03:06:28,405::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913484::DEBUG::2013-11-20 03:06:28,406::task::495::TaskManager.Task::(_debug) Task c1d6c3f8-9b6b-4305-af90-75284367f21b: moving from state init -> state preparing
>Thread-15913484::DEBUG::2013-11-20 03:06:28,407::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913484::DEBUG::2013-11-20 03:06:28,407::task::495::TaskManager.Task::(_debug) Task c1d6c3f8-9b6b-4305-af90-75284367f21b: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913484::DEBUG::2013-11-20 03:06:28,408::task::495::TaskManager.Task::(_debug) Task c1d6c3f8-9b6b-4305-af90-75284367f21b: moving from state preparing -> state finished
>Thread-15913484::DEBUG::2013-11-20 03:06:28,409::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913484::DEBUG::2013-11-20 03:06:28,409::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913484::DEBUG::2013-11-20 03:06:28,410::task::495::TaskManager.Task::(_debug) Task c1d6c3f8-9b6b-4305-af90-75284367f21b: ref 0 aborting False
>Thread-15913484::INFO::2013-11-20 03:06:28,410::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913485::DEBUG::2013-11-20 03:06:28,415::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913485::INFO::2013-11-20 03:06:28,416::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913485::DEBUG::2013-11-20 03:06:28,416::task::495::TaskManager.Task::(_debug) Task f521ebd0-230d-43c2-9f58-d590e2806b21: moving from state init -> state preparing
>Thread-15913485::DEBUG::2013-11-20 03:06:28,418::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`80f19f4c-aecc-43db-a8c4-3da07b43743c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913485::DEBUG::2013-11-20 03:06:28,418::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913485::DEBUG::2013-11-20 03:06:28,419::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913485::DEBUG::2013-11-20 03:06:28,420::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`80f19f4c-aecc-43db-a8c4-3da07b43743c`::Granted request
>Thread-15913485::DEBUG::2013-11-20 03:06:28,421::task::495::TaskManager.Task::(_debug) Task f521ebd0-230d-43c2-9f58-d590e2806b21: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913485::DEBUG::2013-11-20 03:06:28,421::task::495::TaskManager.Task::(_debug) Task f521ebd0-230d-43c2-9f58-d590e2806b21: ref 1 aborting False
>Thread-15913485::WARNING::2013-11-20 03:06:28,423::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913485::DEBUG::2013-11-20 03:06:28,424::task::495::TaskManager.Task::(_debug) Task f521ebd0-230d-43c2-9f58-d590e2806b21: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913485::DEBUG::2013-11-20 03:06:28,425::task::495::TaskManager.Task::(_debug) Task f521ebd0-230d-43c2-9f58-d590e2806b21: moving from state preparing -> state finished
>Thread-15913485::DEBUG::2013-11-20 03:06:28,425::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913485::DEBUG::2013-11-20 03:06:28,426::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913485::DEBUG::2013-11-20 03:06:28,427::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913485::DEBUG::2013-11-20 03:06:28,427::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913485::DEBUG::2013-11-20 03:06:28,428::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913485::DEBUG::2013-11-20 03:06:28,428::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913485::DEBUG::2013-11-20 03:06:28,429::task::495::TaskManager.Task::(_debug) Task f521ebd0-230d-43c2-9f58-d590e2806b21: ref 0 aborting False
>Thread-15913485::INFO::2013-11-20 03:06:28,430::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:06:29,964::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:06:30,516::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0727677 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:06:32,532::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:06:33,016::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0501847 s, 20.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:06:35,031::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:06:35,607::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0729221 s, 14.0 MB/s\n'; <rc> = 0
>Thread-42838::DEBUG::2013-11-20 03:06:35,753::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 03:06:36,488::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:06:37,622::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:06:38,233::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0749718 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15913489::DEBUG::2013-11-20 03:06:38,658::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913489::INFO::2013-11-20 03:06:38,659::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913489::DEBUG::2013-11-20 03:06:38,660::task::495::TaskManager.Task::(_debug) Task 82117e64-9f65-4b79-a3a1-b3aaba459cd4: moving from state init -> state preparing
>Thread-15913489::DEBUG::2013-11-20 03:06:38,661::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913489::DEBUG::2013-11-20 03:06:38,661::task::495::TaskManager.Task::(_debug) Task 82117e64-9f65-4b79-a3a1-b3aaba459cd4: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913489::DEBUG::2013-11-20 03:06:38,662::task::495::TaskManager.Task::(_debug) Task 82117e64-9f65-4b79-a3a1-b3aaba459cd4: moving from state preparing -> state finished
>Thread-15913489::DEBUG::2013-11-20 03:06:38,662::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913489::DEBUG::2013-11-20 03:06:38,663::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913489::DEBUG::2013-11-20 03:06:38,664::task::495::TaskManager.Task::(_debug) Task 82117e64-9f65-4b79-a3a1-b3aaba459cd4: ref 0 aborting False
>Thread-15913489::INFO::2013-11-20 03:06:38,664::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913490::DEBUG::2013-11-20 03:06:38,669::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913490::INFO::2013-11-20 03:06:38,670::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913490::DEBUG::2013-11-20 03:06:38,670::task::495::TaskManager.Task::(_debug) Task 6ca435c9-3a52-4eb9-b204-7cd44cfc8919: moving from state init -> state preparing
>Thread-15913490::DEBUG::2013-11-20 03:06:38,671::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d3ddac3c-9145-4b84-b727-c5f3279a108b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913490::DEBUG::2013-11-20 03:06:38,672::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913490::DEBUG::2013-11-20 03:06:38,673::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913490::DEBUG::2013-11-20 03:06:38,673::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d3ddac3c-9145-4b84-b727-c5f3279a108b`::Granted request
>Thread-15913490::DEBUG::2013-11-20 03:06:38,674::task::495::TaskManager.Task::(_debug) Task 6ca435c9-3a52-4eb9-b204-7cd44cfc8919: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913490::DEBUG::2013-11-20 03:06:38,675::task::495::TaskManager.Task::(_debug) Task 6ca435c9-3a52-4eb9-b204-7cd44cfc8919: ref 1 aborting False
>Thread-15913490::WARNING::2013-11-20 03:06:38,677::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913490::DEBUG::2013-11-20 03:06:38,678::task::495::TaskManager.Task::(_debug) Task 6ca435c9-3a52-4eb9-b204-7cd44cfc8919: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913490::DEBUG::2013-11-20 03:06:38,679::task::495::TaskManager.Task::(_debug) Task 6ca435c9-3a52-4eb9-b204-7cd44cfc8919: moving from state preparing -> state finished
>Thread-15913490::DEBUG::2013-11-20 03:06:38,679::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913490::DEBUG::2013-11-20 03:06:38,680::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913490::DEBUG::2013-11-20 03:06:38,680::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913490::DEBUG::2013-11-20 03:06:38,681::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913490::DEBUG::2013-11-20 03:06:38,682::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913490::DEBUG::2013-11-20 03:06:38,682::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913490::DEBUG::2013-11-20 03:06:38,683::task::495::TaskManager.Task::(_debug) Task 6ca435c9-3a52-4eb9-b204-7cd44cfc8919: ref 0 aborting False
>Thread-15913490::INFO::2013-11-20 03:06:38,683::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:06:40,249::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:06:40,836::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0747142 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15913492::INFO::2013-11-20 03:06:42,325::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913492::DEBUG::2013-11-20 03:06:42,326::task::495::TaskManager.Task::(_debug) Task 3e6b0eb2-f5a9-4c17-bc33-b94206b73538: moving from state init -> state preparing
>Thread-15913492::DEBUG::2013-11-20 03:06:42,327::task::495::TaskManager.Task::(_debug) Task 3e6b0eb2-f5a9-4c17-bc33-b94206b73538: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000316143035889', 'lastCheck': 1384916796.493331, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00192093849182', 'lastCheck': 1384916793.219619, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00172996520996', 'lastCheck': 1384916794.232214, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000494003295898', 'lastCheck': 1384916793.7564139, 'valid': True, 'code': 0}}
>Thread-15913492::DEBUG::2013-11-20 03:06:42,328::task::495::TaskManager.Task::(_debug) Task 3e6b0eb2-f5a9-4c17-bc33-b94206b73538: moving from state preparing -> state finished
>Thread-15913492::DEBUG::2013-11-20 03:06:42,328::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913492::DEBUG::2013-11-20 03:06:42,329::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913492::DEBUG::2013-11-20 03:06:42,330::task::495::TaskManager.Task::(_debug) Task 3e6b0eb2-f5a9-4c17-bc33-b94206b73538: ref 0 aborting False
>Thread-15913492::INFO::2013-11-20 03:06:42,330::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000316143035889', 'lastCheck': 1384916796.493331, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00192093849182', 'lastCheck': 1384916793.219619, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00172996520996', 'lastCheck': 1384916794.232214, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000494003295898', 'lastCheck': 1384916793.7564139, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:06:42,851::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:06:43,379::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0723255 s, 14.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:06:45,394::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:06:45,926::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.070443 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:06:47,941::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:06:48,478::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.075858 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15913496::DEBUG::2013-11-20 03:06:49,326::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913496::INFO::2013-11-20 03:06:49,328::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913496::DEBUG::2013-11-20 03:06:49,328::task::495::TaskManager.Task::(_debug) Task 8e5f84c5-76ac-46e2-ab79-21c6fe96b547: moving from state init -> state preparing
>Thread-15913496::DEBUG::2013-11-20 03:06:49,329::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913496::DEBUG::2013-11-20 03:06:49,330::task::495::TaskManager.Task::(_debug) Task 8e5f84c5-76ac-46e2-ab79-21c6fe96b547: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913496::DEBUG::2013-11-20 03:06:49,330::task::495::TaskManager.Task::(_debug) Task 8e5f84c5-76ac-46e2-ab79-21c6fe96b547: moving from state preparing -> state finished
>Thread-15913496::DEBUG::2013-11-20 03:06:49,331::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913496::DEBUG::2013-11-20 03:06:49,331::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913496::DEBUG::2013-11-20 03:06:49,332::task::495::TaskManager.Task::(_debug) Task 8e5f84c5-76ac-46e2-ab79-21c6fe96b547: ref 0 aborting False
>Thread-15913496::INFO::2013-11-20 03:06:49,333::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913497::DEBUG::2013-11-20 03:06:49,337::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913497::INFO::2013-11-20 03:06:49,338::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913497::DEBUG::2013-11-20 03:06:49,339::task::495::TaskManager.Task::(_debug) Task be423a1a-68d8-4323-b00b-5ed4ddc8adcb: moving from state init -> state preparing
>Thread-15913497::DEBUG::2013-11-20 03:06:49,340::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6d30360c-3f6a-44bb-809e-ac266a8bc432`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913497::DEBUG::2013-11-20 03:06:49,340::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913497::DEBUG::2013-11-20 03:06:49,341::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913497::DEBUG::2013-11-20 03:06:49,342::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6d30360c-3f6a-44bb-809e-ac266a8bc432`::Granted request
>Thread-15913497::DEBUG::2013-11-20 03:06:49,343::task::495::TaskManager.Task::(_debug) Task be423a1a-68d8-4323-b00b-5ed4ddc8adcb: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913497::DEBUG::2013-11-20 03:06:49,343::task::495::TaskManager.Task::(_debug) Task be423a1a-68d8-4323-b00b-5ed4ddc8adcb: ref 1 aborting False
>Thread-15913497::WARNING::2013-11-20 03:06:49,346::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913497::DEBUG::2013-11-20 03:06:49,347::task::495::TaskManager.Task::(_debug) Task be423a1a-68d8-4323-b00b-5ed4ddc8adcb: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913497::DEBUG::2013-11-20 03:06:49,348::task::495::TaskManager.Task::(_debug) Task be423a1a-68d8-4323-b00b-5ed4ddc8adcb: moving from state preparing -> state finished
>Thread-15913497::DEBUG::2013-11-20 03:06:49,349::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913497::DEBUG::2013-11-20 03:06:49,350::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913497::DEBUG::2013-11-20 03:06:49,351::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913497::DEBUG::2013-11-20 03:06:49,352::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913497::DEBUG::2013-11-20 03:06:49,352::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913497::DEBUG::2013-11-20 03:06:49,353::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913497::DEBUG::2013-11-20 03:06:49,354::task::495::TaskManager.Task::(_debug) Task be423a1a-68d8-4323-b00b-5ed4ddc8adcb: ref 0 aborting False
>Thread-15913497::INFO::2013-11-20 03:06:49,355::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:06:50,492::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:06:51,030::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0739068 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:06:53,043::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:06:53,608::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0733065 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:06:55,623::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:06:56,201::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0803471 s, 12.7 MB/s\n'; <rc> = 0
>Thread-15913500::DEBUG::2013-11-20 03:06:57,024::clientIF::54::vds::(wrapper) [192.168.100.135]::call create with ({'bridge': 'rhevm,rhevm', 'acpiEnable': 'true', 'emulatedMachine': 'rhel6.2.0', 'vmId': '4b8ed9c8-dfb1-4de4-935d-6ba4009567b9', 'transparentHugePages': 'true', 'spiceSslCipherSuite': 'DEFAULT', 'cpuType': 'Westmere', 'smp': '2', 'macAddr': '00:1a:4a:a8:64:05,00:1a:4a:a8:64:0b', 'boot': 'c', 'custom': {}, 'vmType': 'kvm', 'memSize': 2048, 'smpCoresPerSocket': '1', 'vmName': 'KLEE-Fil01', 'spiceMonitors': '1', 'nice': '0', 'drives': [{'index': '0', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'format': 'cow', 'boot': 'true', 'volumeID': '0ecd1db7-48b0-4663-a038-e40e891b9424', 'imageID': 'b0980185-7de1-4e3b-bd76-4495169ab08b', 'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'propagateErrors': 'off', 'if': 'virtio'}, {'index': '1', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'format': 'cow', 'boot': 'false', 'volumeID': '6d82661d-9502-4cb3-a0a0-91cd82dd2a80', 'imageID': '0b89347c-4654-42bb-a8d7-1a627130b7bd', 'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'propagateErrors': 'off', 'if': 'virtio'}], 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO', 'smartcardEnable': 'false', 'nicModel': 'pv,pv', 'keyboardLayout': 'en-us', 'kvmEnable': 'true', 'timeOffset': '3601', 'spiceSecureChannels': 'smain,sinputs', 'display': 'qxl'},) {}
>Thread-15913500::INFO::2013-11-20 03:06:57,415::clientIF::523::vds::(_getNetworkIp) network None: using 0
>Thread-15913500::INFO::2013-11-20 03:06:57,417::clientIF::656::vds::(create) vmContainerLock acquired by vm 4b8ed9c8-dfb1-4de4-935d-6ba4009567b9
>Thread-15913501::DEBUG::2013-11-20 03:06:57,422::vm::342::vm.Vm::(_startUnderlyingVm) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Start
>Thread-15913500::DEBUG::2013-11-20 03:06:57,423::clientIF::669::vds::(create) Total desktops after creation of 4b8ed9c8-dfb1-4de4-935d-6ba4009567b9 is 2
>Thread-15913501::DEBUG::2013-11-20 03:06:57,424::vm::346::vm.Vm::(_startUnderlyingVm) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::_ongoingCreations acquired
>Thread-15913500::DEBUG::2013-11-20 03:06:57,424::clientIF::59::vds::(wrapper) return create with {'status': {'message': 'Done', 'code': 0}, 'vmList': {'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO', 'bridge': 'rhevm,rhevm', 'acpiEnable': 'true', 'emulatedMachine': 'rhel6.2.0', 'vmId': '4b8ed9c8-dfb1-4de4-935d-6ba4009567b9', 'pid': '0', 'drives': [{'index': '0', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'format': 'cow', 'boot': 'true', 'volumeID': '0ecd1db7-48b0-4663-a038-e40e891b9424', 'imageID': 'b0980185-7de1-4e3b-bd76-4495169ab08b', 'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'propagateErrors': 'off', 'if': 'virtio'}, {'index': '1', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'format': 'cow', 'boot': 'false', 'volumeID': '6d82661d-9502-4cb3-a0a0-91cd82dd2a80', 'imageID': '0b89347c-4654-42bb-a8d7-1a627130b7bd', 'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'propagateErrors': 'off', 'if': 'virtio'}], 'transparentHugePages': 'true', 'keyboardLayout': 'en-us', 'displaySecurePort': '-1', 'displayPort': '-1', 'cpuType': 'Westmere', 'custom': {}, 'clientIp': '', 'nicModel': 'pv,pv', 'macAddr': '00:1a:4a:a8:64:05,00:1a:4a:a8:64:0b', 'kvmEnable': 'true', 'smartcardEnable': 'false', 'boot': 'c', 'smp': '2', 'vmType': 'kvm', 'spiceSslCipherSuite': 'DEFAULT', 'status': 'WaitForLaunch', 'timeOffset': '3601', 'memSize': 2048, 'displayIp': '0', 'spiceSecureChannels': 'smain,sinputs', 'smpCoresPerSocket': '1', 'vmName': 'KLEE-Fil01', 'display': 'qxl', 'spiceMonitors': '1', 'nice': '0'}}
>Thread-15913501::INFO::2013-11-20 03:06:57,425::libvirtvm::1061::vm.Vm::(_run) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::VM wrapper has started
>Thread-15913501::INFO::2013-11-20 03:06:57,429::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913501::DEBUG::2013-11-20 03:06:57,430::task::495::TaskManager.Task::(_debug) Task ce5b44eb-1521-42df-a14a-c21ed0c9f96f: moving from state init -> state preparing
>Thread-15913501::DEBUG::2013-11-20 03:06:57,432::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`930891b2-62fa-45ab-8d70-9807178ae8a9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1690' at 'public_prepareVolume'
>Thread-15913501::DEBUG::2013-11-20 03:06:57,432::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913501::DEBUG::2013-11-20 03:06:57,433::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913501::DEBUG::2013-11-20 03:06:57,434::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`930891b2-62fa-45ab-8d70-9807178ae8a9`::Granted request
>Thread-15913501::DEBUG::2013-11-20 03:06:57,435::task::495::TaskManager.Task::(_debug) Task ce5b44eb-1521-42df-a14a-c21ed0c9f96f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913501::DEBUG::2013-11-20 03:06:57,435::task::495::TaskManager.Task::(_debug) Task ce5b44eb-1521-42df-a14a-c21ed0c9f96f: ref 1 aborting False
>Thread-15913501::DEBUG::2013-11-20 03:06:57,436::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.b0980185-7de1-4e3b-bd76-4495169ab08b`ReqID=`eebdfdf4-0a99-443a-a1a9-d91fa3368f94`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1695' at 'public_prepareVolume'
>Thread-15913501::DEBUG::2013-11-20 03:06:57,437::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.b0980185-7de1-4e3b-bd76-4495169ab08b' for lock type 'exclusive'
>Thread-15913501::DEBUG::2013-11-20 03:06:57,438::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913501::DEBUG::2013-11-20 03:06:57,439::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15913503::INFO::2013-11-20 03:06:58,018::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913503::DEBUG::2013-11-20 03:06:58,029::task::495::TaskManager.Task::(_debug) Task 8d429878-aa7a-404b-bc02-99ebfe0564ef: moving from state init -> state preparing
>Thread-15913503::DEBUG::2013-11-20 03:06:58,030::task::495::TaskManager.Task::(_debug) Task 8d429878-aa7a-404b-bc02-99ebfe0564ef: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000319004058838', 'lastCheck': 1384916816.8568971, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014979839325', 'lastCheck': 1384916813.745677, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0211429595947', 'lastCheck': 1384916814.766695, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00043797492981', 'lastCheck': 1384916814.286289, 'valid': True, 'code': 0}}
>Thread-15913503::DEBUG::2013-11-20 03:06:58,031::task::495::TaskManager.Task::(_debug) Task 8d429878-aa7a-404b-bc02-99ebfe0564ef: moving from state preparing -> state finished
>Thread-15913503::DEBUG::2013-11-20 03:06:58,031::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913503::DEBUG::2013-11-20 03:06:58,032::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913503::DEBUG::2013-11-20 03:06:58,032::task::495::TaskManager.Task::(_debug) Task 8d429878-aa7a-404b-bc02-99ebfe0564ef: ref 0 aborting False
>Thread-15913503::INFO::2013-11-20 03:06:58,033::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000319004058838', 'lastCheck': 1384916816.8568971, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014979839325', 'lastCheck': 1384916813.745677, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0211429595947', 'lastCheck': 1384916814.766695, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00043797492981', 'lastCheck': 1384916814.286289, 'valid': True, 'code': 0}}
>Thread-15913501::DEBUG::2013-11-20 03:06:58,140::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:06:58,263::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913501::DEBUG::2013-11-20 03:06:58,618::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913501::DEBUG::2013-11-20 03:06:58,628::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=21 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:06:59,096::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.111779 s, 9.2 MB/s\n'; <rc> = 0
>Thread-15913501::DEBUG::2013-11-20 03:06:59,130::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.00379108 s, 135 kB/s\n'; <rc> = 0
>Thread-15913501::DEBUG::2013-11-20 03:06:59,131::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.00379108 s, 135 kB/s'], size: 512
>Thread-15913501::DEBUG::2013-11-20 03:06:59,132::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=21 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15913501::DEBUG::2013-11-20 03:06:59,560::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000199929 s, 2.6 MB/s\n'; <rc> = 0
>Thread-15913501::DEBUG::2013-11-20 03:06:59,561::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000199929 s, 2.6 MB/s'], size: 512
>Thread-15913501::INFO::2013-11-20 03:06:59,562::image::340::Storage.Image::(getChain) sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b chain=[<storage.blockVolume.BlockVolume instance at 0x7f48cc193ab8>] 
>Thread-15913501::DEBUG::2013-11-20 03:06:59,564::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --autobackup n --available y 6b892067-ac7e-4039-a7be-5b7387525b16/0ecd1db7-48b0-4663-a038-e40e891b9424' (cwd None)
>Thread-15913505::DEBUG::2013-11-20 03:06:59,786::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913505::INFO::2013-11-20 03:06:59,788::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913505::DEBUG::2013-11-20 03:06:59,789::task::495::TaskManager.Task::(_debug) Task 5b2d598e-fad3-4fef-b958-b1fa69421123: moving from state init -> state preparing
>Thread-15913505::DEBUG::2013-11-20 03:06:59,791::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913505::DEBUG::2013-11-20 03:06:59,791::task::495::TaskManager.Task::(_debug) Task 5b2d598e-fad3-4fef-b958-b1fa69421123: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913505::DEBUG::2013-11-20 03:06:59,792::task::495::TaskManager.Task::(_debug) Task 5b2d598e-fad3-4fef-b958-b1fa69421123: moving from state preparing -> state finished
>Thread-15913505::DEBUG::2013-11-20 03:06:59,792::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913505::DEBUG::2013-11-20 03:06:59,793::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913505::DEBUG::2013-11-20 03:06:59,794::task::495::TaskManager.Task::(_debug) Task 5b2d598e-fad3-4fef-b958-b1fa69421123: ref 0 aborting False
>Thread-15913505::INFO::2013-11-20 03:06:59,794::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913506::DEBUG::2013-11-20 03:06:59,799::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913506::INFO::2013-11-20 03:06:59,800::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913506::DEBUG::2013-11-20 03:06:59,801::task::495::TaskManager.Task::(_debug) Task 70ea544c-9075-4b73-a958-2ab0dd5e8360: moving from state init -> state preparing
>Thread-15913506::DEBUG::2013-11-20 03:06:59,802::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4d5c20cb-fbc6-45e7-b8f3-ca4a1310e6f7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913506::DEBUG::2013-11-20 03:06:59,803::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913506::DEBUG::2013-11-20 03:06:59,804::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913506::DEBUG::2013-11-20 03:06:59,804::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4d5c20cb-fbc6-45e7-b8f3-ca4a1310e6f7`::Granted request
>Thread-15913506::DEBUG::2013-11-20 03:06:59,805::task::495::TaskManager.Task::(_debug) Task 70ea544c-9075-4b73-a958-2ab0dd5e8360: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913506::DEBUG::2013-11-20 03:06:59,806::task::495::TaskManager.Task::(_debug) Task 70ea544c-9075-4b73-a958-2ab0dd5e8360: ref 1 aborting False
>Thread-15913506::WARNING::2013-11-20 03:06:59,808::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913506::DEBUG::2013-11-20 03:06:59,809::task::495::TaskManager.Task::(_debug) Task 70ea544c-9075-4b73-a958-2ab0dd5e8360: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913506::DEBUG::2013-11-20 03:06:59,810::task::495::TaskManager.Task::(_debug) Task 70ea544c-9075-4b73-a958-2ab0dd5e8360: moving from state preparing -> state finished
>Thread-15913506::DEBUG::2013-11-20 03:06:59,811::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913506::DEBUG::2013-11-20 03:06:59,811::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913506::DEBUG::2013-11-20 03:06:59,812::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913506::DEBUG::2013-11-20 03:06:59,813::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913506::DEBUG::2013-11-20 03:06:59,813::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913506::DEBUG::2013-11-20 03:06:59,814::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913506::DEBUG::2013-11-20 03:06:59,814::task::495::TaskManager.Task::(_debug) Task 70ea544c-9075-4b73-a958-2ab0dd5e8360: ref 0 aborting False
>Thread-15913506::INFO::2013-11-20 03:06:59,815::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913501::DEBUG::2013-11-20 03:07:00,455::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913501::DEBUG::2013-11-20 03:07:00,456::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913501::DEBUG::2013-11-20 03:07:00,457::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913501::DEBUG::2013-11-20 03:07:00,458::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.0ecd1db7-48b0-4663-a038-e40e891b9424`ReqID=`aeb39440-87db-4845-97b7-5160e955b048`::Request was made in '/usr/share/vdsm/storage/resourceFactories.py' line '149' at '__getResourceCandidatesList'
>Thread-15913501::DEBUG::2013-11-20 03:07:00,459::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.0ecd1db7-48b0-4663-a038-e40e891b9424' for lock type 'exclusive'
>Thread-15913501::DEBUG::2013-11-20 03:07:00,460::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.0ecd1db7-48b0-4663-a038-e40e891b9424' is free. Now locking as 'exclusive' (1 active user)
>Thread-15913501::DEBUG::2013-11-20 03:07:00,460::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.0ecd1db7-48b0-4663-a038-e40e891b9424`ReqID=`aeb39440-87db-4845-97b7-5160e955b048`::Granted request
>Thread-15913501::DEBUG::2013-11-20 03:07:00,461::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.b0980185-7de1-4e3b-bd76-4495169ab08b' is free. Now locking as 'exclusive' (1 active user)
>Thread-15913501::DEBUG::2013-11-20 03:07:00,462::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.b0980185-7de1-4e3b-bd76-4495169ab08b`ReqID=`eebdfdf4-0a99-443a-a1a9-d91fa3368f94`::Granted request
>Thread-15913501::DEBUG::2013-11-20 03:07:00,463::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913501::DEBUG::2013-11-20 03:07:00,464::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:07:01,112::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913501::DEBUG::2013-11-20 03:07:01,383::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913501::DEBUG::2013-11-20 03:07:01,668::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913501::DEBUG::2013-11-20 03:07:01,673::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --refresh 6b892067-ac7e-4039-a7be-5b7387525b16/0ecd1db7-48b0-4663-a038-e40e891b9424' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:07:02,011::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.367596 s, 2.8 MB/s\n'; <rc> = 0
>Thread-15913501::DEBUG::2013-11-20 03:07:02,317::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913501::DEBUG::2013-11-20 03:07:02,319::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913501::DEBUG::2013-11-20 03:07:02,320::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913501::INFO::2013-11-20 03:07:02,320::volume::517::Storage.Volume::(prepare) Volume: preparing volume 6b892067-ac7e-4039-a7be-5b7387525b16/0ecd1db7-48b0-4663-a038-e40e891b9424
>Thread-15913501::DEBUG::2013-11-20 03:07:02,321::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913501::DEBUG::2013-11-20 03:07:02,322::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15913501::DEBUG::2013-11-20 03:07:03,012::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913501::DEBUG::2013-11-20 03:07:03,253::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913501::DEBUG::2013-11-20 03:07:03,253::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=21 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15913501::DEBUG::2013-11-20 03:07:03,628::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000289655 s, 1.8 MB/s\n'; <rc> = 0
>Thread-15913501::DEBUG::2013-11-20 03:07:03,629::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000289655 s, 1.8 MB/s'], size: 512
>Thread-15913501::DEBUG::2013-11-20 03:07:03,630::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=21 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:07:04,027::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913501::DEBUG::2013-11-20 03:07:04,276::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000333103 s, 1.5 MB/s\n'; <rc> = 0
>Thread-15913501::DEBUG::2013-11-20 03:07:04,278::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000333103 s, 1.5 MB/s'], size: 512
>Thread-15913501::DEBUG::2013-11-20 03:07:04,280::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.0ecd1db7-48b0-4663-a038-e40e891b9424`ReqID=`f1512aae-3e29-496a-94f6-7da7f963de20`::Request was made in '/usr/share/vdsm/storage/volume.py' line '533' at 'prepare'
>Thread-15913501::DEBUG::2013-11-20 03:07:04,281::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.0ecd1db7-48b0-4663-a038-e40e891b9424' for lock type 'exclusive'
>Thread-15913501::DEBUG::2013-11-20 03:07:04,282::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.0ecd1db7-48b0-4663-a038-e40e891b9424' is free. Now locking as 'exclusive' (1 active user)
>Thread-15913501::DEBUG::2013-11-20 03:07:04,282::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.0ecd1db7-48b0-4663-a038-e40e891b9424`ReqID=`f1512aae-3e29-496a-94f6-7da7f963de20`::Granted request
>Thread-15913501::DEBUG::2013-11-20 03:07:04,284::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=21 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:07:04,764::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.146654 s, 7.0 MB/s\n'; <rc> = 0
>Thread-15913501::DEBUG::2013-11-20 03:07:04,821::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000270587 s, 1.9 MB/s\n'; <rc> = 0
>Thread-15913501::DEBUG::2013-11-20 03:07:04,822::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000270587 s, 1.9 MB/s'], size: 512
>Thread-15913501::DEBUG::2013-11-20 03:07:04,824::task::495::TaskManager.Task::(_debug) Task ce5b44eb-1521-42df-a14a-c21ed0c9f96f: finished: None
>Thread-15913501::DEBUG::2013-11-20 03:07:04,824::task::495::TaskManager.Task::(_debug) Task ce5b44eb-1521-42df-a14a-c21ed0c9f96f: moving from state preparing -> state finished
>Thread-15913501::DEBUG::2013-11-20 03:07:04,825::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913501::DEBUG::2013-11-20 03:07:04,826::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913501::DEBUG::2013-11-20 03:07:04,826::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913501::DEBUG::2013-11-20 03:07:04,827::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913501::DEBUG::2013-11-20 03:07:04,828::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913501::DEBUG::2013-11-20 03:07:04,828::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913501::DEBUG::2013-11-20 03:07:04,829::task::495::TaskManager.Task::(_debug) Task ce5b44eb-1521-42df-a14a-c21ed0c9f96f: ref 0 aborting False
>Thread-15913501::INFO::2013-11-20 03:07:04,829::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, Return response: {'status': {'message': 'OK', 'code': 0}}
>Thread-15913501::INFO::2013-11-20 03:07:04,831::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913501::DEBUG::2013-11-20 03:07:04,831::task::495::TaskManager.Task::(_debug) Task b1af75d5-11a7-42dd-9e91-347fddeb2d7b: moving from state init -> state preparing
>Thread-15913501::DEBUG::2013-11-20 03:07:04,832::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9bdfdb9d-2642-4548-a522-2bacdc647654`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1666' at 'public_getVolumePath'
>Thread-15913501::DEBUG::2013-11-20 03:07:04,833::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913501::DEBUG::2013-11-20 03:07:04,834::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913501::DEBUG::2013-11-20 03:07:04,834::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9bdfdb9d-2642-4548-a522-2bacdc647654`::Granted request
>Thread-15913501::DEBUG::2013-11-20 03:07:04,835::task::495::TaskManager.Task::(_debug) Task b1af75d5-11a7-42dd-9e91-347fddeb2d7b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913501::DEBUG::2013-11-20 03:07:04,836::task::495::TaskManager.Task::(_debug) Task b1af75d5-11a7-42dd-9e91-347fddeb2d7b: ref 1 aborting False
>Thread-15913501::DEBUG::2013-11-20 03:07:04,837::task::495::TaskManager.Task::(_debug) Task b1af75d5-11a7-42dd-9e91-347fddeb2d7b: finished: {'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/b0980185-7de1-4e3b-bd76-4495169ab08b/0ecd1db7-48b0-4663-a038-e40e891b9424'}
>Thread-15913501::DEBUG::2013-11-20 03:07:04,837::task::495::TaskManager.Task::(_debug) Task b1af75d5-11a7-42dd-9e91-347fddeb2d7b: moving from state preparing -> state finished
>Thread-15913501::DEBUG::2013-11-20 03:07:04,838::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913501::DEBUG::2013-11-20 03:07:04,838::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913501::DEBUG::2013-11-20 03:07:04,839::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913501::DEBUG::2013-11-20 03:07:04,839::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913501::DEBUG::2013-11-20 03:07:04,840::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913501::DEBUG::2013-11-20 03:07:04,840::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913501::DEBUG::2013-11-20 03:07:04,841::task::495::TaskManager.Task::(_debug) Task b1af75d5-11a7-42dd-9e91-347fddeb2d7b: ref 0 aborting False
>Thread-15913501::INFO::2013-11-20 03:07:04,842::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, Return response: {'status': {'message': 'OK', 'code': 0}, 'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/b0980185-7de1-4e3b-bd76-4495169ab08b/0ecd1db7-48b0-4663-a038-e40e891b9424'}
>Thread-15913501::INFO::2013-11-20 03:07:04,842::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913501::DEBUG::2013-11-20 03:07:04,843::task::495::TaskManager.Task::(_debug) Task 92047c46-90e8-469c-b10b-19c106dfd47c: moving from state init -> state preparing
>Thread-15913501::DEBUG::2013-11-20 03:07:04,844::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b0c07c72-55b3-4781-b87e-b1e5f390f4f4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1690' at 'public_prepareVolume'
>Thread-15913501::DEBUG::2013-11-20 03:07:04,844::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913501::DEBUG::2013-11-20 03:07:04,845::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913501::DEBUG::2013-11-20 03:07:04,846::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b0c07c72-55b3-4781-b87e-b1e5f390f4f4`::Granted request
>Thread-15913501::DEBUG::2013-11-20 03:07:04,846::task::495::TaskManager.Task::(_debug) Task 92047c46-90e8-469c-b10b-19c106dfd47c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913501::DEBUG::2013-11-20 03:07:04,847::task::495::TaskManager.Task::(_debug) Task 92047c46-90e8-469c-b10b-19c106dfd47c: ref 1 aborting False
>Thread-15913501::DEBUG::2013-11-20 03:07:04,848::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.0b89347c-4654-42bb-a8d7-1a627130b7bd`ReqID=`7dd2c752-5acf-4b7c-b2df-bb8711c7f4f9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1695' at 'public_prepareVolume'
>Thread-15913501::DEBUG::2013-11-20 03:07:04,849::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.0b89347c-4654-42bb-a8d7-1a627130b7bd' for lock type 'exclusive'
>Thread-15913501::DEBUG::2013-11-20 03:07:04,850::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913501::DEBUG::2013-11-20 03:07:04,851::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15913501::DEBUG::2013-11-20 03:07:05,602::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913501::DEBUG::2013-11-20 03:07:05,830::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913501::DEBUG::2013-11-20 03:07:05,832::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=22 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15913501::DEBUG::2013-11-20 03:07:06,228::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.00533553 s, 96.0 kB/s\n'; <rc> = 0
>Thread-15913501::DEBUG::2013-11-20 03:07:06,229::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.00533553 s, 96.0 kB/s'], size: 512
>Thread-15913501::DEBUG::2013-11-20 03:07:06,230::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=22 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15913501::DEBUG::2013-11-20 03:07:06,637::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000215463 s, 2.4 MB/s\n'; <rc> = 0
>Thread-15913501::DEBUG::2013-11-20 03:07:06,638::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000215463 s, 2.4 MB/s'], size: 512
>Thread-15913501::INFO::2013-11-20 03:07:06,639::image::340::Storage.Image::(getChain) sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd chain=[<storage.blockVolume.BlockVolume instance at 0x7f487c135bd8>] 
>Thread-15913501::DEBUG::2013-11-20 03:07:06,641::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --autobackup n --available y 6b892067-ac7e-4039-a7be-5b7387525b16/6d82661d-9502-4cb3-a0a0-91cd82dd2a80' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:07:06,865::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913509::DEBUG::2013-11-20 03:07:07,369::clientIF::54::vds::(wrapper) [192.168.100.135]::call create with ({'bridge': 'rhevm', 'acpiEnable': 'true', 'emulatedMachine': 'rhel6.2.0', 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'transparentHugePages': 'true', 'spiceSslCipherSuite': 'DEFAULT', 'cpuType': 'Westmere', 'smp': '8', 'macAddr': '00:1a:4a:a8:64:08', 'boot': 'cd', 'custom': {}, 'vmType': 'kvm', 'memSize': 20480, 'smpCoresPerSocket': '4', 'vmName': 'KLEE-AXDB01', 'spiceMonitors': '1', 'nice': '0', 'drives': [{'index': '0', 'domainID': '772472cb-2429-466d-96cd-00b3301ef78a', 'format': 'raw', 'boot': 'true', 'volumeID': 'e6984064-e09e-4c0c-8b51-a8a1711994cf', 'imageID': 'c4460480-32d3-469f-b0f4-c03bb6d74e41', 'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'propagateErrors': 'off', 'if': 'virtio'}, {'index': '1', 'domainID': '772472cb-2429-466d-96cd-00b3301ef78a', 'format': 'raw', 'boot': 'false', 'volumeID': '2e903e43-1016-4339-a7f3-c5723a995725', 'imageID': 'df8b790c-c762-4feb-9378-d64a321ce43c', 'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'propagateErrors': 'off', 'if': 'virtio'}, {'index': '2', 'domainID': '772472cb-2429-466d-96cd-00b3301ef78a', 'format': 'raw', 'boot': 'false', 'volumeID': '088640f7-316c-4e2b-bfdc-0f64cf2913c8', 'imageID': '13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', 'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'propagateErrors': 'off', 'if': 'virtio'}, {'index': '3', 'domainID': '772472cb-2429-466d-96cd-00b3301ef78a', 'format': 'raw', 'boot': 'false', 'volumeID': 'fdc91117-476f-4b82-8fc4-d682fa283ecc', 'imageID': 'd368307e-425e-423a-85b7-266c1e4a99ec', 'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'propagateErrors': 'off', 'if': 'virtio'}], 'smartcardEnable': 'false', 'nicModel': 'pv', 'keyboardLayout': 'en-us', 'kvmEnable': 'true', 'timeOffset': '-1', 'spiceSecureChannels': 'smain,sinputs', 'display': 'qxl'},) {}
>Thread-15913501::DEBUG::2013-11-20 03:07:07,604::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913501::DEBUG::2013-11-20 03:07:07,607::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913501::DEBUG::2013-11-20 03:07:07,608::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913501::DEBUG::2013-11-20 03:07:07,611::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.6d82661d-9502-4cb3-a0a0-91cd82dd2a80`ReqID=`ea58bda5-a1e2-42dc-b7ab-d2e7e13825ae`::Request was made in '/usr/share/vdsm/storage/resourceFactories.py' line '149' at '__getResourceCandidatesList'
>Thread-15913501::DEBUG::2013-11-20 03:07:07,612::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.6d82661d-9502-4cb3-a0a0-91cd82dd2a80' for lock type 'exclusive'
>Thread-15913501::DEBUG::2013-11-20 03:07:07,613::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.6d82661d-9502-4cb3-a0a0-91cd82dd2a80' is free. Now locking as 'exclusive' (1 active user)
>Thread-15913501::DEBUG::2013-11-20 03:07:07,614::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.6d82661d-9502-4cb3-a0a0-91cd82dd2a80`ReqID=`ea58bda5-a1e2-42dc-b7ab-d2e7e13825ae`::Granted request
>Thread-15913501::DEBUG::2013-11-20 03:07:07,616::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.0b89347c-4654-42bb-a8d7-1a627130b7bd' is free. Now locking as 'exclusive' (1 active user)
>Thread-15913501::DEBUG::2013-11-20 03:07:07,616::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.0b89347c-4654-42bb-a8d7-1a627130b7bd`ReqID=`7dd2c752-5acf-4b7c-b2df-bb8711c7f4f9`::Granted request
>Thread-15913501::DEBUG::2013-11-20 03:07:07,620::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913501::DEBUG::2013-11-20 03:07:07,631::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15913509::INFO::2013-11-20 03:07:07,972::clientIF::523::vds::(_getNetworkIp) network None: using 0
>Thread-15913509::INFO::2013-11-20 03:07:07,975::clientIF::656::vds::(create) vmContainerLock acquired by vm 6574d88c-00ac-46f4-99ce-a2726932bfda
>Thread-15913509::DEBUG::2013-11-20 03:07:08,005::clientIF::669::vds::(create) Total desktops after creation of 6574d88c-00ac-46f4-99ce-a2726932bfda is 3
>Thread-15913510::DEBUG::2013-11-20 03:07:08,006::vm::342::vm.Vm::(_startUnderlyingVm) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Start
>Thread-15913509::DEBUG::2013-11-20 03:07:08,007::clientIF::59::vds::(wrapper) return create with {'status': {'message': 'Done', 'code': 0}, 'vmList': {'status': 'WaitForLaunch', 'bridge': 'rhevm', 'acpiEnable': 'true', 'emulatedMachine': 'rhel6.2.0', 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'pid': '0', 'drives': [{'index': '0', 'domainID': '772472cb-2429-466d-96cd-00b3301ef78a', 'format': 'raw', 'boot': 'true', 'volumeID': 'e6984064-e09e-4c0c-8b51-a8a1711994cf', 'imageID': 'c4460480-32d3-469f-b0f4-c03bb6d74e41', 'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'propagateErrors': 'off', 'if': 'virtio'}, {'index': '1', 'domainID': '772472cb-2429-466d-96cd-00b3301ef78a', 'format': 'raw', 'boot': 'false', 'volumeID': '2e903e43-1016-4339-a7f3-c5723a995725', 'imageID': 'df8b790c-c762-4feb-9378-d64a321ce43c', 'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'propagateErrors': 'off', 'if': 'virtio'}, {'index': '2', 'domainID': '772472cb-2429-466d-96cd-00b3301ef78a', 'format': 'raw', 'boot': 'false', 'volumeID': '088640f7-316c-4e2b-bfdc-0f64cf2913c8', 'imageID': '13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', 'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'propagateErrors': 'off', 'if': 'virtio'}, {'index': '3', 'domainID': '772472cb-2429-466d-96cd-00b3301ef78a', 'format': 'raw', 'boot': 'false', 'volumeID': 'fdc91117-476f-4b82-8fc4-d682fa283ecc', 'imageID': 'd368307e-425e-423a-85b7-266c1e4a99ec', 'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'propagateErrors': 'off', 'if': 'virtio'}], 'transparentHugePages': 'true', 'keyboardLayout': 'en-us', 'displayPort': '-1', 'displaySecurePort': '-1', 'spiceSslCipherSuite': 'DEFAULT', 'cpuType': 'Westmere', 'custom': {}, 'clientIp': '', 'nicModel': 'pv', 'macAddr': '00:1a:4a:a8:64:08', 'kvmEnable': 'true', 'boot': 'cd', 'smp': '8', 'vmType': 'kvm', 'smartcardEnable': 'false', 'timeOffset': '-1', 'memSize': 20480, 'displayIp': '0', 'spiceSecureChannels': 'smain,sinputs', 'smpCoresPerSocket': '4', 'vmName': 'KLEE-AXDB01', 'display': 'qxl', 'spiceMonitors': '1', 'nice': '0'}}
>Dummy-15912651::DEBUG::2013-11-20 03:07:08,095::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.690654 s, 1.5 MB/s\n'; <rc> = 0
>Thread-15913501::DEBUG::2013-11-20 03:07:08,510::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913501::DEBUG::2013-11-20 03:07:08,789::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913501::DEBUG::2013-11-20 03:07:08,791::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --refresh 6b892067-ac7e-4039-a7be-5b7387525b16/6d82661d-9502-4cb3-a0a0-91cd82dd2a80' (cwd None)
>Thread-15913501::DEBUG::2013-11-20 03:07:09,411::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913501::DEBUG::2013-11-20 03:07:09,412::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913501::DEBUG::2013-11-20 03:07:09,413::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913501::INFO::2013-11-20 03:07:09,414::volume::517::Storage.Volume::(prepare) Volume: preparing volume 6b892067-ac7e-4039-a7be-5b7387525b16/6d82661d-9502-4cb3-a0a0-91cd82dd2a80
>Thread-15913501::DEBUG::2013-11-20 03:07:09,414::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913501::DEBUG::2013-11-20 03:07:09,416::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:07:10,109::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913501::DEBUG::2013-11-20 03:07:10,387::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913513::DEBUG::2013-11-20 03:07:10,389::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913513::INFO::2013-11-20 03:07:10,466::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913513::DEBUG::2013-11-20 03:07:10,605::task::495::TaskManager.Task::(_debug) Task a5970242-47d5-42bb-ab77-056a166cbd37: moving from state init -> state preparing
>Thread-15913513::DEBUG::2013-11-20 03:07:10,714::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913501::DEBUG::2013-11-20 03:07:10,719::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913513::DEBUG::2013-11-20 03:07:10,720::task::495::TaskManager.Task::(_debug) Task a5970242-47d5-42bb-ab77-056a166cbd37: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913513::DEBUG::2013-11-20 03:07:10,721::task::495::TaskManager.Task::(_debug) Task a5970242-47d5-42bb-ab77-056a166cbd37: moving from state preparing -> state finished
>Thread-15913501::DEBUG::2013-11-20 03:07:10,722::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=22 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15913513::DEBUG::2013-11-20 03:07:10,722::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913513::DEBUG::2013-11-20 03:07:10,724::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913513::DEBUG::2013-11-20 03:07:10,924::task::495::TaskManager.Task::(_debug) Task a5970242-47d5-42bb-ab77-056a166cbd37: ref 0 aborting False
>Thread-15913513::INFO::2013-11-20 03:07:10,926::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913514::DEBUG::2013-11-20 03:07:10,935::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913514::INFO::2013-11-20 03:07:10,938::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913514::DEBUG::2013-11-20 03:07:10,939::task::495::TaskManager.Task::(_debug) Task 1783eae2-e767-400d-8851-9f1cf510ac98: moving from state init -> state preparing
>Thread-15913514::DEBUG::2013-11-20 03:07:10,941::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`fe25814a-5d1d-4aeb-b5e1-6f1ae398a8af`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913514::DEBUG::2013-11-20 03:07:10,941::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913514::DEBUG::2013-11-20 03:07:10,943::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913514::DEBUG::2013-11-20 03:07:10,944::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`fe25814a-5d1d-4aeb-b5e1-6f1ae398a8af`::Granted request
>Thread-15913514::DEBUG::2013-11-20 03:07:10,946::task::495::TaskManager.Task::(_debug) Task 1783eae2-e767-400d-8851-9f1cf510ac98: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913514::DEBUG::2013-11-20 03:07:10,946::task::495::TaskManager.Task::(_debug) Task 1783eae2-e767-400d-8851-9f1cf510ac98: ref 1 aborting False
>Thread-15913514::WARNING::2013-11-20 03:07:10,950::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913514::DEBUG::2013-11-20 03:07:10,951::task::495::TaskManager.Task::(_debug) Task 1783eae2-e767-400d-8851-9f1cf510ac98: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913514::DEBUG::2013-11-20 03:07:10,952::task::495::TaskManager.Task::(_debug) Task 1783eae2-e767-400d-8851-9f1cf510ac98: moving from state preparing -> state finished
>Thread-15913514::DEBUG::2013-11-20 03:07:10,953::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913514::DEBUG::2013-11-20 03:07:10,954::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913514::DEBUG::2013-11-20 03:07:10,955::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913514::DEBUG::2013-11-20 03:07:10,956::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913514::DEBUG::2013-11-20 03:07:10,957::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913514::DEBUG::2013-11-20 03:07:10,958::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913514::DEBUG::2013-11-20 03:07:10,959::task::495::TaskManager.Task::(_debug) Task 1783eae2-e767-400d-8851-9f1cf510ac98: ref 0 aborting False
>Thread-15913514::INFO::2013-11-20 03:07:10,959::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:07:11,112::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.488368 s, 2.1 MB/s\n'; <rc> = 0
>Thread-15913501::DEBUG::2013-11-20 03:07:11,131::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000188178 s, 2.7 MB/s\n'; <rc> = 0
>Thread-15913501::DEBUG::2013-11-20 03:07:11,132::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000188178 s, 2.7 MB/s'], size: 512
>Thread-15913501::DEBUG::2013-11-20 03:07:11,133::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=22 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15913501::DEBUG::2013-11-20 03:07:11,571::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000292993 s, 1.7 MB/s\n'; <rc> = 0
>Thread-15913501::DEBUG::2013-11-20 03:07:11,574::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000292993 s, 1.7 MB/s'], size: 512
>Thread-15913501::DEBUG::2013-11-20 03:07:11,575::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.6d82661d-9502-4cb3-a0a0-91cd82dd2a80`ReqID=`4c7e9522-ee21-4c23-89df-9a87e04d5509`::Request was made in '/usr/share/vdsm/storage/volume.py' line '533' at 'prepare'
>Thread-15913501::DEBUG::2013-11-20 03:07:11,576::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.6d82661d-9502-4cb3-a0a0-91cd82dd2a80' for lock type 'exclusive'
>Thread-15913501::DEBUG::2013-11-20 03:07:11,577::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.6d82661d-9502-4cb3-a0a0-91cd82dd2a80' is free. Now locking as 'exclusive' (1 active user)
>Thread-15913501::DEBUG::2013-11-20 03:07:11,578::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.6d82661d-9502-4cb3-a0a0-91cd82dd2a80`ReqID=`4c7e9522-ee21-4c23-89df-9a87e04d5509`::Granted request
>Thread-15913501::DEBUG::2013-11-20 03:07:11,581::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=22 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15913501::DEBUG::2013-11-20 03:07:11,988::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000344947 s, 1.5 MB/s\n'; <rc> = 0
>Thread-15913501::DEBUG::2013-11-20 03:07:11,989::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000344947 s, 1.5 MB/s'], size: 512
>Thread-15913501::DEBUG::2013-11-20 03:07:11,991::task::495::TaskManager.Task::(_debug) Task 92047c46-90e8-469c-b10b-19c106dfd47c: finished: None
>Thread-15913501::DEBUG::2013-11-20 03:07:11,992::task::495::TaskManager.Task::(_debug) Task 92047c46-90e8-469c-b10b-19c106dfd47c: moving from state preparing -> state finished
>Thread-15913501::DEBUG::2013-11-20 03:07:11,992::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913501::DEBUG::2013-11-20 03:07:11,993::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913501::DEBUG::2013-11-20 03:07:11,994::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913501::DEBUG::2013-11-20 03:07:11,995::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913501::DEBUG::2013-11-20 03:07:11,995::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913501::DEBUG::2013-11-20 03:07:11,996::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913501::DEBUG::2013-11-20 03:07:11,996::task::495::TaskManager.Task::(_debug) Task 92047c46-90e8-469c-b10b-19c106dfd47c: ref 0 aborting False
>Thread-15913501::INFO::2013-11-20 03:07:11,997::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, Return response: {'status': {'message': 'OK', 'code': 0}}
>Thread-15913501::INFO::2013-11-20 03:07:11,998::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913501::DEBUG::2013-11-20 03:07:11,999::task::495::TaskManager.Task::(_debug) Task 22aceda3-c582-4f79-8e84-c202526533a1: moving from state init -> state preparing
>Thread-15913501::DEBUG::2013-11-20 03:07:12,000::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`dfc2d478-11df-4cc7-a5b6-d4a3b5c51cbb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1666' at 'public_getVolumePath'
>Thread-15913501::DEBUG::2013-11-20 03:07:12,001::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913501::DEBUG::2013-11-20 03:07:12,001::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913501::DEBUG::2013-11-20 03:07:12,002::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`dfc2d478-11df-4cc7-a5b6-d4a3b5c51cbb`::Granted request
>Thread-15913501::DEBUG::2013-11-20 03:07:12,003::task::495::TaskManager.Task::(_debug) Task 22aceda3-c582-4f79-8e84-c202526533a1: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913501::DEBUG::2013-11-20 03:07:12,003::task::495::TaskManager.Task::(_debug) Task 22aceda3-c582-4f79-8e84-c202526533a1: ref 1 aborting False
>Thread-15913501::DEBUG::2013-11-20 03:07:12,004::task::495::TaskManager.Task::(_debug) Task 22aceda3-c582-4f79-8e84-c202526533a1: finished: {'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/0b89347c-4654-42bb-a8d7-1a627130b7bd/6d82661d-9502-4cb3-a0a0-91cd82dd2a80'}
>Thread-15913501::DEBUG::2013-11-20 03:07:12,005::task::495::TaskManager.Task::(_debug) Task 22aceda3-c582-4f79-8e84-c202526533a1: moving from state preparing -> state finished
>Thread-15913501::DEBUG::2013-11-20 03:07:12,005::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913501::DEBUG::2013-11-20 03:07:12,006::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913501::DEBUG::2013-11-20 03:07:12,007::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913501::DEBUG::2013-11-20 03:07:12,007::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913501::DEBUG::2013-11-20 03:07:12,008::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913501::DEBUG::2013-11-20 03:07:12,008::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913501::DEBUG::2013-11-20 03:07:12,009::task::495::TaskManager.Task::(_debug) Task 22aceda3-c582-4f79-8e84-c202526533a1: ref 0 aborting False
>Thread-15913501::INFO::2013-11-20 03:07:12,009::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, Return response: {'status': {'message': 'OK', 'code': 0}, 'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/0b89347c-4654-42bb-a8d7-1a627130b7bd/6d82661d-9502-4cb3-a0a0-91cd82dd2a80'}
>Thread-15913501::INFO::2013-11-20 03:07:12,011::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913501::DEBUG::2013-11-20 03:07:12,012::task::495::TaskManager.Task::(_debug) Task a922f760-a39f-4807-98ba-08aa31f59c65: moving from state init -> state preparing
>Thread-15913501::DEBUG::2013-11-20 03:07:12,013::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`68b3efb9-86fa-4428-898c-9886dcb5b933`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913501::DEBUG::2013-11-20 03:07:12,014::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913501::DEBUG::2013-11-20 03:07:12,014::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913501::DEBUG::2013-11-20 03:07:12,015::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`68b3efb9-86fa-4428-898c-9886dcb5b933`::Granted request
>Thread-15913501::DEBUG::2013-11-20 03:07:12,016::task::495::TaskManager.Task::(_debug) Task a922f760-a39f-4807-98ba-08aa31f59c65: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913501::DEBUG::2013-11-20 03:07:12,016::task::495::TaskManager.Task::(_debug) Task a922f760-a39f-4807-98ba-08aa31f59c65: ref 1 aborting False
>Thread-15913501::DEBUG::2013-11-20 03:07:12,017::task::495::TaskManager.Task::(_debug) Task a922f760-a39f-4807-98ba-08aa31f59c65: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913501::DEBUG::2013-11-20 03:07:12,018::task::495::TaskManager.Task::(_debug) Task a922f760-a39f-4807-98ba-08aa31f59c65: moving from state preparing -> state finished
>Thread-15913501::DEBUG::2013-11-20 03:07:12,018::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913501::DEBUG::2013-11-20 03:07:12,019::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913501::DEBUG::2013-11-20 03:07:12,019::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913501::DEBUG::2013-11-20 03:07:12,020::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913501::DEBUG::2013-11-20 03:07:12,020::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913501::DEBUG::2013-11-20 03:07:12,021::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913501::DEBUG::2013-11-20 03:07:12,021::task::495::TaskManager.Task::(_debug) Task a922f760-a39f-4807-98ba-08aa31f59c65: ref 0 aborting False
>Thread-15913501::INFO::2013-11-20 03:07:12,022::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913501::INFO::2013-11-20 03:07:12,023::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16)
>Thread-15913501::DEBUG::2013-11-20 03:07:12,023::task::495::TaskManager.Task::(_debug) Task f9338935-cfdf-4b83-9010-d9ef6a3869d7: moving from state init -> state preparing
>Thread-15913501::INFO::2013-11-20 03:07:12,024::blockSD::674::Storage.StorageDomain::(validate) sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16
>Thread-15913501::DEBUG::2013-11-20 03:07:12,024::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913501::DEBUG::2013-11-20 03:07:12,025::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913501::DEBUG::2013-11-20 03:07:12,026::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913501::DEBUG::2013-11-20 03:07:12,027::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913501::DEBUG::2013-11-20 03:07:12,028::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913501::DEBUG::2013-11-20 03:07:12,029::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15913501::DEBUG::2013-11-20 03:07:12,696::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913501::DEBUG::2013-11-20 03:07:12,700::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913501::DEBUG::2013-11-20 03:07:12,701::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'_SHA_CKSUM=716ea7492d9134a4b13de27ac61a336ad474727c', u'POOL_SPM_ID=2', u'POOL_SPM_LVER=287']
>Thread-15913501::DEBUG::2013-11-20 03:07:12,703::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c43e7ea6-d79f-408a-b684-937fdc62587f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1297' at 'public_getStorageDomainInfo'
>Thread-15913501::DEBUG::2013-11-20 03:07:12,704::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913501::DEBUG::2013-11-20 03:07:12,705::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913501::DEBUG::2013-11-20 03:07:12,705::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c43e7ea6-d79f-408a-b684-937fdc62587f`::Granted request
>Thread-15913501::DEBUG::2013-11-20 03:07:12,706::task::495::TaskManager.Task::(_debug) Task f9338935-cfdf-4b83-9010-d9ef6a3869d7: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913501::DEBUG::2013-11-20 03:07:12,707::task::495::TaskManager.Task::(_debug) Task f9338935-cfdf-4b83-9010-d9ef6a3869d7: ref 1 aborting False
>Thread-15913501::WARNING::2013-11-20 03:07:12,709::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913501::DEBUG::2013-11-20 03:07:12,710::task::495::TaskManager.Task::(_debug) Task f9338935-cfdf-4b83-9010-d9ef6a3869d7: finished: {'info': {'uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'vguuid': 'aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', 'lver': 287, 'state': 'OK', 'version': '2', 'role': 'Master', 'master_ver': 1, 'spm_id': 2, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'OS'}}
>Thread-15913501::DEBUG::2013-11-20 03:07:12,711::task::495::TaskManager.Task::(_debug) Task f9338935-cfdf-4b83-9010-d9ef6a3869d7: moving from state preparing -> state finished
>Thread-15913501::DEBUG::2013-11-20 03:07:12,711::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913501::DEBUG::2013-11-20 03:07:12,712::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913501::DEBUG::2013-11-20 03:07:12,713::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913501::DEBUG::2013-11-20 03:07:12,713::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913501::DEBUG::2013-11-20 03:07:12,714::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913501::DEBUG::2013-11-20 03:07:12,714::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913501::DEBUG::2013-11-20 03:07:12,715::task::495::TaskManager.Task::(_debug) Task f9338935-cfdf-4b83-9010-d9ef6a3869d7: ref 0 aborting False
>Thread-15913501::INFO::2013-11-20 03:07:12,715::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'vguuid': 'aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', 'lver': 287, 'state': 'OK', 'version': '2', 'role': 'Master', 'master_ver': 1, 'spm_id': 2, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'OS'}}
>Thread-15913501::INFO::2013-11-20 03:07:12,716::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913501::DEBUG::2013-11-20 03:07:12,717::task::495::TaskManager.Task::(_debug) Task d79850ca-fd0b-4a3b-bb60-cb4c1e74d8c9: moving from state init -> state preparing
>Thread-15913501::DEBUG::2013-11-20 03:07:12,718::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`301f15d2-38f4-451f-83db-785c8e8d89da`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913501::DEBUG::2013-11-20 03:07:12,718::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913501::DEBUG::2013-11-20 03:07:12,719::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913501::DEBUG::2013-11-20 03:07:12,720::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`301f15d2-38f4-451f-83db-785c8e8d89da`::Granted request
>Thread-15913501::DEBUG::2013-11-20 03:07:12,720::task::495::TaskManager.Task::(_debug) Task d79850ca-fd0b-4a3b-bb60-cb4c1e74d8c9: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913501::DEBUG::2013-11-20 03:07:12,721::task::495::TaskManager.Task::(_debug) Task d79850ca-fd0b-4a3b-bb60-cb4c1e74d8c9: ref 1 aborting False
>Thread-15913501::DEBUG::2013-11-20 03:07:12,722::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913501::DEBUG::2013-11-20 03:07:12,723::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:07:13,131::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913501::DEBUG::2013-11-20 03:07:13,399::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913501::DEBUG::2013-11-20 03:07:13,704::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913501::DEBUG::2013-11-20 03:07:13,706::task::495::TaskManager.Task::(_debug) Task d79850ca-fd0b-4a3b-bb60-cb4c1e74d8c9: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913501::DEBUG::2013-11-20 03:07:13,707::task::495::TaskManager.Task::(_debug) Task d79850ca-fd0b-4a3b-bb60-cb4c1e74d8c9: moving from state preparing -> state finished
>Thread-15913501::DEBUG::2013-11-20 03:07:13,708::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913501::DEBUG::2013-11-20 03:07:13,708::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913501::DEBUG::2013-11-20 03:07:13,710::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913501::DEBUG::2013-11-20 03:07:13,711::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913501::DEBUG::2013-11-20 03:07:13,712::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913501::DEBUG::2013-11-20 03:07:13,713::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913501::DEBUG::2013-11-20 03:07:13,714::task::495::TaskManager.Task::(_debug) Task d79850ca-fd0b-4a3b-bb60-cb4c1e74d8c9: ref 0 aborting False
>Thread-15913501::INFO::2013-11-20 03:07:13,715::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913501::INFO::2013-11-20 03:07:13,716::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16)
>Thread-15913501::DEBUG::2013-11-20 03:07:13,717::task::495::TaskManager.Task::(_debug) Task a5f67c37-ca6d-452f-a3fe-56de12b9eec7: moving from state init -> state preparing
>Thread-15913501::INFO::2013-11-20 03:07:13,718::blockSD::674::Storage.StorageDomain::(validate) sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16
>Thread-15913501::DEBUG::2013-11-20 03:07:13,719::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913501::DEBUG::2013-11-20 03:07:13,720::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913501::DEBUG::2013-11-20 03:07:13,721::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913501::DEBUG::2013-11-20 03:07:13,722::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913501::DEBUG::2013-11-20 03:07:13,724::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913501::DEBUG::2013-11-20 03:07:13,725::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:07:14,068::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.447456 s, 2.3 MB/s\n'; <rc> = 0
>Thread-15913501::DEBUG::2013-11-20 03:07:14,392::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913501::DEBUG::2013-11-20 03:07:14,396::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913501::DEBUG::2013-11-20 03:07:14,397::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'_SHA_CKSUM=716ea7492d9134a4b13de27ac61a336ad474727c', u'POOL_SPM_ID=2', u'POOL_SPM_LVER=287']
>Thread-15913501::DEBUG::2013-11-20 03:07:14,399::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`02bc3e30-fefa-4b7d-985f-b5263a4be490`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1297' at 'public_getStorageDomainInfo'
>Thread-15913501::DEBUG::2013-11-20 03:07:14,400::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913501::DEBUG::2013-11-20 03:07:14,401::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913501::DEBUG::2013-11-20 03:07:14,401::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`02bc3e30-fefa-4b7d-985f-b5263a4be490`::Granted request
>Thread-15913501::DEBUG::2013-11-20 03:07:14,403::task::495::TaskManager.Task::(_debug) Task a5f67c37-ca6d-452f-a3fe-56de12b9eec7: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913501::DEBUG::2013-11-20 03:07:14,403::task::495::TaskManager.Task::(_debug) Task a5f67c37-ca6d-452f-a3fe-56de12b9eec7: ref 1 aborting False
>Thread-15913501::WARNING::2013-11-20 03:07:14,407::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913501::DEBUG::2013-11-20 03:07:14,408::task::495::TaskManager.Task::(_debug) Task a5f67c37-ca6d-452f-a3fe-56de12b9eec7: finished: {'info': {'uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'vguuid': 'aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', 'lver': 287, 'state': 'OK', 'version': '2', 'role': 'Master', 'master_ver': 1, 'spm_id': 2, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'OS'}}
>Thread-15913501::DEBUG::2013-11-20 03:07:14,409::task::495::TaskManager.Task::(_debug) Task a5f67c37-ca6d-452f-a3fe-56de12b9eec7: moving from state preparing -> state finished
>Thread-15913501::DEBUG::2013-11-20 03:07:14,410::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913516::INFO::2013-11-20 03:07:14,412::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913501::DEBUG::2013-11-20 03:07:14,413::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913516::DEBUG::2013-11-20 03:07:14,414::task::495::TaskManager.Task::(_debug) Task 29f582fd-374c-4835-b39b-3a904d9601f7: moving from state init -> state preparing
>Thread-15913501::DEBUG::2013-11-20 03:07:14,415::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913516::DEBUG::2013-11-20 03:07:14,416::task::495::TaskManager.Task::(_debug) Task 29f582fd-374c-4835-b39b-3a904d9601f7: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000458002090454', 'lastCheck': 1384916827.1933119, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0236279964447', 'lastCheck': 1384916834.4015639, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00136518478394', 'lastCheck': 1384916825.078372, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00197911262512', 'lastCheck': 1384916824.616087, 'valid': True, 'code': 0}}
>Thread-15913501::DEBUG::2013-11-20 03:07:14,417::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913516::DEBUG::2013-11-20 03:07:14,418::task::495::TaskManager.Task::(_debug) Task 29f582fd-374c-4835-b39b-3a904d9601f7: moving from state preparing -> state finished
>Thread-15913501::DEBUG::2013-11-20 03:07:14,419::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913516::DEBUG::2013-11-20 03:07:14,419::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913501::DEBUG::2013-11-20 03:07:14,420::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913516::DEBUG::2013-11-20 03:07:14,421::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913501::DEBUG::2013-11-20 03:07:14,421::task::495::TaskManager.Task::(_debug) Task a5f67c37-ca6d-452f-a3fe-56de12b9eec7: ref 0 aborting False
>Thread-15913516::DEBUG::2013-11-20 03:07:14,422::task::495::TaskManager.Task::(_debug) Task 29f582fd-374c-4835-b39b-3a904d9601f7: ref 0 aborting False
>Thread-15913501::INFO::2013-11-20 03:07:14,423::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'vguuid': 'aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', 'lver': 287, 'state': 'OK', 'version': '2', 'role': 'Master', 'master_ver': 1, 'spm_id': 2, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'OS'}}
>Thread-15913516::INFO::2013-11-20 03:07:14,424::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000458002090454', 'lastCheck': 1384916827.1933119, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0236279964447', 'lastCheck': 1384916834.4015639, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00136518478394', 'lastCheck': 1384916825.078372, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00197911262512', 'lastCheck': 1384916824.616087, 'valid': True, 'code': 0}}
>Thread-15913501::DEBUG::2013-11-20 03:07:14,442::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/libexec/vdsm/hooks/before_vm_start/50_vhostmd' (cwd None)
>Thread-15913501::DEBUG::2013-11-20 03:07:14,860::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913501::INFO::2013-11-20 03:07:14,861::hooks::51::root::(_runHooksDir) 
>Thread-15913501::DEBUG::2013-11-20 03:07:14,862::libvirtvm::1077::vm.Vm::(_run) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::<?xml version="1.0" encoding="utf-8"?>
><domain type="kvm">
>	<name>KLEE-Fil01</name>
>	<uuid>4b8ed9c8-dfb1-4de4-935d-6ba4009567b9</uuid>
>	<memory>2097152</memory>
>	<currentMemory>2097152</currentMemory>
>	<vcpu>2</vcpu>
>	<devices>
>		<disk device="disk" type="block">
>			<source dev="/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/b0980185-7de1-4e3b-bd76-4495169ab08b/0ecd1db7-48b0-4663-a038-e40e891b9424"/>
>			<target bus="virtio" dev="vda"/>
>			<serial>3b-bd76-4495169ab08b</serial>
>			<driver cache="none" error_policy="stop" io="native" name="qemu" type="qcow2"/>
>		</disk>
>		<disk device="disk" type="block">
>			<source dev="/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/0b89347c-4654-42bb-a8d7-1a627130b7bd/6d82661d-9502-4cb3-a0a0-91cd82dd2a80"/>
>			<target bus="virtio" dev="vdb"/>
>			<serial>bb-a8d7-1a627130b7bd</serial>
>			<driver cache="none" error_policy="stop" io="native" name="qemu" type="qcow2"/>
>		</disk>
>		<disk device="cdrom" type="file">
>			<source file="/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO"/>
>			<target bus="ide" dev="hdc"/>
>		</disk>
>		<controller index="0" ports="16" type="virtio-serial"/>
>		<channel type="unix">
>			<target name="com.redhat.rhevm.vdsm" type="virtio"/>
>			<source mode="bind" path="/var/lib/libvirt/qemu/channels/KLEE-Fil01.com.redhat.rhevm.vdsm"/>
>		</channel>
>		<memballoon model="none"/>
>		<interface type="bridge">
>			<mac address="00:1a:4a:a8:64:05"/>
>			<model type="virtio"/>
>			<source bridge="rhevm"/>
>		</interface>
>		<interface type="bridge">
>			<mac address="00:1a:4a:a8:64:0b"/>
>			<model type="virtio"/>
>			<source bridge="rhevm"/>
>		</interface>
>		<input bus="ps2" type="mouse"/>
>		<video>
>			<model heads="1" type="qxl" vram="65536"/>
>		</video>
>		<channel type="spicevmc">
>			<target name="com.redhat.spice.0" type="virtio"/>
>		</channel>
>		<graphics autoport="yes" keymap="en-us" listen="0" passwd="*****" passwdValidTo="1970-01-01T00:00:01" port="-1" tlsPort="-1" type="spice">
>			<channel mode="secure" name="main"/>
>			<channel mode="secure" name="inputs"/>
>		</graphics>
>	</devices>
>	<os>
>		<type arch="x86_64" machine="rhel6.2.0">hvm</type>
>		<boot dev="hd"/>
>		<smbios mode="sysinfo"/>
>	</os>
>	<sysinfo type="smbios">
>		<system>
>			<entry name="manufacturer">Red Hat</entry>
>			<entry name="product">RHEV Hypervisor</entry>
>			<entry name="version">6.3-20121012.0.el6_3</entry>
>			<entry name="serial">30343536-3138-5A43-4A32-333630373442_AC:16:2D:76:EB:F0</entry>
>			<entry name="uuid">4b8ed9c8-dfb1-4de4-935d-6ba4009567b9</entry>
>		</system>
>	</sysinfo>
>	<clock adjustment="3601" offset="variable">
>		<timer name="rtc" tickpolicy="catchup"/>
>	</clock>
>	<features>
>		<acpi/>
>	</features>
>	<cpu match="exact">
>		<model>Westmere</model>
>		<topology cores="1" sockets="2" threads="1"/>
>	</cpu>
></domain>
>
>libvirtEventLoop::DEBUG::2013-11-20 03:07:15,825::libvirtvm::1401::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::event Started detail 0 opaque None
>Thread-15913501::DEBUG::2013-11-20 03:07:15,893::utils::307::vm.Vm::(start) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Start statistics collection
>Thread-15913518::DEBUG::2013-11-20 03:07:15,894::utils::336::vm.Vm::(run) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Stats thread started
>Thread-15913519::DEBUG::2013-11-20 03:07:15,904::guestIF::47::root::(_connect) Attempting connection to /var/lib/libvirt/qemu/channels/KLEE-Fil01.com.redhat.rhevm.vdsm
>Thread-15913501::WARNING::2013-11-20 03:07:15,907::libvirtvm::1124::vm.Vm::(_readPauseCode) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::_readPauseCode unsupported by libvirt vm
>Thread-15913519::DEBUG::2013-11-20 03:07:15,908::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /usr/share/vdsm/prepare-vmchannel /var/lib/libvirt/qemu/channels/KLEE-Fil01.com.redhat.rhevm.vdsm' (cwd None)
>Thread-15913501::DEBUG::2013-11-20 03:07:15,911::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /usr/share/vdsm/get-vm-pid KLEE-Fil01' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:07:16,391::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913519::DEBUG::2013-11-20 03:07:16,619::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913501::DEBUG::2013-11-20 03:07:16,621::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913519::DEBUG::2013-11-20 03:07:16,622::guestIF::53::root::(_connect) connected to /var/lib/libvirt/qemu/channels/KLEE-Fil01.com.redhat.rhevm.vdsm
>Thread-15913501::DEBUG::2013-11-20 03:07:16,633::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /sbin/service ksmtuned retune' (cwd None)
>Thread-15913518::INFO::2013-11-20 03:07:17,092::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:07:17,108::task::495::TaskManager.Task::(_debug) Task 14d61759-be24-4e42-be69-39ef5444c500: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:07:17,120::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5709af13-0d8a-4b7a-ae59-43acd2daa53b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:07:17,121::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:07:17,135::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:07:17,148::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5709af13-0d8a-4b7a-ae59-43acd2daa53b`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:07:17,164::task::495::TaskManager.Task::(_debug) Task 14d61759-be24-4e42-be69-39ef5444c500: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:07:17,176::task::495::TaskManager.Task::(_debug) Task 14d61759-be24-4e42-be69-39ef5444c500: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:07:17,182::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913518::DEBUG::2013-11-20 03:07:17,204::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15913501::DEBUG::2013-11-20 03:07:17,215::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913501::DEBUG::2013-11-20 03:07:17,427::vm::358::vm.Vm::(_startUnderlyingVm) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::_ongoingCreations released
>Thread-15913510::DEBUG::2013-11-20 03:07:17,429::vm::346::vm.Vm::(_startUnderlyingVm) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::_ongoingCreations acquired
>Thread-15913510::INFO::2013-11-20 03:07:17,436::libvirtvm::1061::vm.Vm::(_run) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::VM wrapper has started
>Thread-15913510::INFO::2013-11-20 03:07:17,439::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913510::DEBUG::2013-11-20 03:07:17,441::task::495::TaskManager.Task::(_debug) Task a11f83b3-356c-4c77-9f46-1b6ba987df6b: moving from state init -> state preparing
>Thread-15913510::DEBUG::2013-11-20 03:07:17,446::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`5520ddc5-423c-43ba-8021-db33274695dc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1690' at 'public_prepareVolume'
>Thread-15913510::DEBUG::2013-11-20 03:07:17,447::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913510::DEBUG::2013-11-20 03:07:17,450::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913510::DEBUG::2013-11-20 03:07:17,453::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`5520ddc5-423c-43ba-8021-db33274695dc`::Granted request
>Dummy-15912651::DEBUG::2013-11-20 03:07:17,454::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.105188 s, 9.7 MB/s\n'; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:17,457::task::495::TaskManager.Task::(_debug) Task a11f83b3-356c-4c77-9f46-1b6ba987df6b: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913510::DEBUG::2013-11-20 03:07:17,458::task::495::TaskManager.Task::(_debug) Task a11f83b3-356c-4c77-9f46-1b6ba987df6b: ref 1 aborting False
>Thread-15913510::DEBUG::2013-11-20 03:07:17,476::resourceManager::155::ResourceManager.Request::(__init__) ResName=`772472cb-2429-466d-96cd-00b3301ef78a_imageNS.c4460480-32d3-469f-b0f4-c03bb6d74e41`ReqID=`ce99774b-8c3e-476a-84bf-0c9b90d6c626`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1695' at 'public_prepareVolume'
>Thread-15913510::DEBUG::2013-11-20 03:07:17,477::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '772472cb-2429-466d-96cd-00b3301ef78a_imageNS.c4460480-32d3-469f-b0f4-c03bb6d74e41' for lock type 'exclusive'
>Thread-15913510::DEBUG::2013-11-20 03:07:17,480::lvm::480::OperationMutex::(_reloadlvs) Got the operational mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:17,483::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-15913518::DEBUG::2013-11-20 03:07:18,209::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913521::DEBUG::2013-11-20 03:07:18,873::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('4b8ed9c8-dfb1-4de4-935d-6ba4009567b9',) {}
>Thread-15913510::DEBUG::2013-11-20 03:07:18,912::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913521::DEBUG::2013-11-20 03:07:18,917::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::CPU stats not available
>Thread-15913521::DEBUG::2013-11-20 03:07:19,002::libvirtvm::202::vm.Vm::(_getNetworkStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Network stats not available
>Thread-15913521::DEBUG::2013-11-20 03:07:19,028::libvirtvm::202::vm.Vm::(_getNetworkStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Network stats not available
>Thread-15913510::DEBUG::2013-11-20 03:07:19,070::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=7 bs=512 if=/dev/772472cb-2429-466d-96cd-00b3301ef78a/metadata count=1' (cwd None)
>Thread-15913521::DEBUG::2013-11-20 03:07:19,073::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk stats not available
>Thread-15913521::DEBUG::2013-11-20 03:07:19,128::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk stats not available
>Thread-15913518::DEBUG::2013-11-20 03:07:19,463::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913521::DEBUG::2013-11-20 03:07:19,469::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk latency not available
>Thread-15913518::DEBUG::2013-11-20 03:07:19,471::task::495::TaskManager.Task::(_debug) Task 14d61759-be24-4e42-be69-39ef5444c500: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913521::DEBUG::2013-11-20 03:07:19,474::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk latency not available
>Thread-15913518::DEBUG::2013-11-20 03:07:19,475::task::495::TaskManager.Task::(_debug) Task 14d61759-be24-4e42-be69-39ef5444c500: moving from state preparing -> state finished
>Thread-15913521::DEBUG::2013-11-20 03:07:19,476::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '46387', 'displayIp': '0', 'displayPort': u'5902', 'session': 'Unknown', 'displaySecurePort': u'5903', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet1': {'state': 'unknown', 'macAddr': u'00:1a:4a:a8:64:05', 'speed': '1000', 'name': u'vnet1'}, u'vnet2': {'state': 'unknown', 'macAddr': u'00:1a:4a:a8:64:0b', 'speed': '1000', 'name': u'vnet2'}}, 'vmId': '4b8ed9c8-dfb1-4de4-935d-6ba4009567b9', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '30064771072', 'writeLatency': '0', 'imageID': 'b0980185-7de1-4e3b-bd76-4495169ab08b', 'flushLatency': '0', 'truesize': '30064771072'}, u'vdb': {'readLatency': '0', 'apparentsize': '826781204480', 'writeLatency': '0', 'imageID': '0b89347c-4654-42bb-a8d7-1a627130b7bd', 'flushLatency': '0', 'truesize': '826781204480'}}, 'boot': 'c', 'statsAge': '3.01', 'cpuIdle': '100.00', 'elapsedTime': '21', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '3601', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15913518::DEBUG::2013-11-20 03:07:19,478::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Dummy-15912651::DEBUG::2013-11-20 03:07:19,479::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913518::DEBUG::2013-11-20 03:07:19,484::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:07:19,737::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:07:19,742::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:07:19,744::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913510::DEBUG::2013-11-20 03:07:19,745::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000353597 s, 1.4 MB/s\n'; <rc> = 0
>Thread-15913518::DEBUG::2013-11-20 03:07:19,746::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913510::DEBUG::2013-11-20 03:07:19,747::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000353597 s, 1.4 MB/s'], size: 512
>Thread-15913510::DEBUG::2013-11-20 03:07:19,749::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=7 bs=512 if=/dev/772472cb-2429-466d-96cd-00b3301ef78a/metadata count=1' (cwd None)
>Thread-15913518::DEBUG::2013-11-20 03:07:19,750::task::495::TaskManager.Task::(_debug) Task 14d61759-be24-4e42-be69-39ef5444c500: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:07:19,752::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:07:20,129::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:07:20,131::task::495::TaskManager.Task::(_debug) Task 8a79cdf3-6016-44fb-90e9-69c998dbbd64: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:07:20,134::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`355c8b57-248c-43d6-a127-5c697996e646`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:07:20,136::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:07:20,138::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:07:20,139::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`355c8b57-248c-43d6-a127-5c697996e646`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:07:20,145::task::495::TaskManager.Task::(_debug) Task 8a79cdf3-6016-44fb-90e9-69c998dbbd64: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:07:20,146::task::495::TaskManager.Task::(_debug) Task 8a79cdf3-6016-44fb-90e9-69c998dbbd64: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:07:20,152::task::495::TaskManager.Task::(_debug) Task 8a79cdf3-6016-44fb-90e9-69c998dbbd64: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:07:20,153::task::495::TaskManager.Task::(_debug) Task 8a79cdf3-6016-44fb-90e9-69c998dbbd64: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:07:20,154::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:07:20,155::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:07:20,156::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:07:20,157::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:07:20,158::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:07:20,158::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:07:20,160::task::495::TaskManager.Task::(_debug) Task 8a79cdf3-6016-44fb-90e9-69c998dbbd64: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:07:20,161::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:07:20,286::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.187592 s, 5.5 MB/s\n'; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:20,311::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.00035512 s, 1.4 MB/s\n'; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:20,312::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.00035512 s, 1.4 MB/s'], size: 512
>Thread-15913510::INFO::2013-11-20 03:07:20,313::image::340::Storage.Image::(getChain) sdUUID=772472cb-2429-466d-96cd-00b3301ef78a imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 chain=[<storage.blockVolume.BlockVolume instance at 0x7f48841837a0>] 
>Thread-15913510::DEBUG::2013-11-20 03:07:20,314::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --autobackup n --available y 772472cb-2429-466d-96cd-00b3301ef78a/e6984064-e09e-4c0c-8b51-a8a1711994cf' (cwd None)
>Thread-15913510::DEBUG::2013-11-20 03:07:21,183::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:21,184::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:21,185::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:21,187::resourceManager::155::ResourceManager.Request::(__init__) ResName=`772472cb-2429-466d-96cd-00b3301ef78a_volumeNS.e6984064-e09e-4c0c-8b51-a8a1711994cf`ReqID=`bc967614-1770-4a2a-9cb1-1d44f4e518f0`::Request was made in '/usr/share/vdsm/storage/resourceFactories.py' line '149' at '__getResourceCandidatesList'
>Thread-15913510::DEBUG::2013-11-20 03:07:21,187::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '772472cb-2429-466d-96cd-00b3301ef78a_volumeNS.e6984064-e09e-4c0c-8b51-a8a1711994cf' for lock type 'exclusive'
>Thread-15913510::DEBUG::2013-11-20 03:07:21,188::resourceManager::505::ResourceManager::(registerResource) Resource '772472cb-2429-466d-96cd-00b3301ef78a_volumeNS.e6984064-e09e-4c0c-8b51-a8a1711994cf' is free. Now locking as 'exclusive' (1 active user)
>Thread-15913510::DEBUG::2013-11-20 03:07:21,189::resourceManager::192::ResourceManager.Request::(grant) ResName=`772472cb-2429-466d-96cd-00b3301ef78a_volumeNS.e6984064-e09e-4c0c-8b51-a8a1711994cf`ReqID=`bc967614-1770-4a2a-9cb1-1d44f4e518f0`::Granted request
>Thread-15913510::DEBUG::2013-11-20 03:07:21,190::resourceManager::505::ResourceManager::(registerResource) Resource '772472cb-2429-466d-96cd-00b3301ef78a_imageNS.c4460480-32d3-469f-b0f4-c03bb6d74e41' is free. Now locking as 'exclusive' (1 active user)
>Thread-15913510::DEBUG::2013-11-20 03:07:21,192::resourceManager::192::ResourceManager.Request::(grant) ResName=`772472cb-2429-466d-96cd-00b3301ef78a_imageNS.c4460480-32d3-469f-b0f4-c03bb6d74e41`ReqID=`ce99774b-8c3e-476a-84bf-0c9b90d6c626`::Granted request
>Thread-15913510::DEBUG::2013-11-20 03:07:21,194::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:21,195::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-15913522::DEBUG::2013-11-20 03:07:21,412::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913522::INFO::2013-11-20 03:07:21,414::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913522::DEBUG::2013-11-20 03:07:21,432::task::495::TaskManager.Task::(_debug) Task 577f2649-c035-4fca-bd1d-02fe70513ab6: moving from state init -> state preparing
>Thread-15913522::DEBUG::2013-11-20 03:07:21,433::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913522::DEBUG::2013-11-20 03:07:21,434::task::495::TaskManager.Task::(_debug) Task 577f2649-c035-4fca-bd1d-02fe70513ab6: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913522::DEBUG::2013-11-20 03:07:21,434::task::495::TaskManager.Task::(_debug) Task 577f2649-c035-4fca-bd1d-02fe70513ab6: moving from state preparing -> state finished
>Thread-15913522::DEBUG::2013-11-20 03:07:21,435::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913522::DEBUG::2013-11-20 03:07:21,436::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913522::DEBUG::2013-11-20 03:07:21,436::task::495::TaskManager.Task::(_debug) Task 577f2649-c035-4fca-bd1d-02fe70513ab6: ref 0 aborting False
>Thread-15913522::INFO::2013-11-20 03:07:21,437::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913523::DEBUG::2013-11-20 03:07:21,443::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913523::INFO::2013-11-20 03:07:21,444::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913523::DEBUG::2013-11-20 03:07:21,444::task::495::TaskManager.Task::(_debug) Task cc8432f1-915d-4a2c-bf80-ed18ff78c1fa: moving from state init -> state preparing
>Thread-15913523::DEBUG::2013-11-20 03:07:21,446::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`60197461-af0a-4e4c-8d1d-fafd7f0112d5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913523::DEBUG::2013-11-20 03:07:21,447::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913523::DEBUG::2013-11-20 03:07:21,448::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913523::DEBUG::2013-11-20 03:07:21,448::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`60197461-af0a-4e4c-8d1d-fafd7f0112d5`::Granted request
>Thread-15913523::DEBUG::2013-11-20 03:07:21,449::task::495::TaskManager.Task::(_debug) Task cc8432f1-915d-4a2c-bf80-ed18ff78c1fa: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913523::DEBUG::2013-11-20 03:07:21,450::task::495::TaskManager.Task::(_debug) Task cc8432f1-915d-4a2c-bf80-ed18ff78c1fa: ref 1 aborting False
>Thread-15913523::WARNING::2013-11-20 03:07:21,452::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913523::DEBUG::2013-11-20 03:07:21,453::task::495::TaskManager.Task::(_debug) Task cc8432f1-915d-4a2c-bf80-ed18ff78c1fa: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913523::DEBUG::2013-11-20 03:07:21,454::task::495::TaskManager.Task::(_debug) Task cc8432f1-915d-4a2c-bf80-ed18ff78c1fa: moving from state preparing -> state finished
>Thread-15913523::DEBUG::2013-11-20 03:07:21,455::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913523::DEBUG::2013-11-20 03:07:21,455::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913523::DEBUG::2013-11-20 03:07:21,456::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913523::DEBUG::2013-11-20 03:07:21,457::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913523::DEBUG::2013-11-20 03:07:21,457::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913523::DEBUG::2013-11-20 03:07:21,458::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913523::DEBUG::2013-11-20 03:07:21,458::task::495::TaskManager.Task::(_debug) Task cc8432f1-915d-4a2c-bf80-ed18ff78c1fa: ref 0 aborting False
>Thread-15913523::INFO::2013-11-20 03:07:21,459::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913510::DEBUG::2013-11-20 03:07:21,893::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:21,932::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:21,936::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --refresh 772472cb-2429-466d-96cd-00b3301ef78a/e6984064-e09e-4c0c-8b51-a8a1711994cf' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:07:22,300::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913510::DEBUG::2013-11-20 03:07:22,584::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:22,586::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:22,587::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913510::INFO::2013-11-20 03:07:22,587::volume::517::Storage.Volume::(prepare) Volume: preparing volume 772472cb-2429-466d-96cd-00b3301ef78a/e6984064-e09e-4c0c-8b51-a8a1711994cf
>Thread-15913510::DEBUG::2013-11-20 03:07:22,588::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:22,589::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-15913525::DEBUG::2013-11-20 03:07:22,820::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('4b8ed9c8-dfb1-4de4-935d-6ba4009567b9',) {}
>Thread-15913525::DEBUG::2013-11-20 03:07:22,822::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::CPU stats not available
>Thread-15913525::DEBUG::2013-11-20 03:07:22,823::libvirtvm::202::vm.Vm::(_getNetworkStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Network stats not available
>Thread-15913525::DEBUG::2013-11-20 03:07:22,824::libvirtvm::202::vm.Vm::(_getNetworkStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Network stats not available
>Thread-15913525::DEBUG::2013-11-20 03:07:22,824::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk stats not available
>Thread-15913525::DEBUG::2013-11-20 03:07:22,825::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk stats not available
>Thread-15913525::DEBUG::2013-11-20 03:07:22,826::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk latency not available
>Thread-15913525::DEBUG::2013-11-20 03:07:22,827::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk latency not available
>Thread-15913525::DEBUG::2013-11-20 03:07:22,828::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '46387', 'displayIp': '0', 'displayPort': u'5902', 'session': 'Unknown', 'displaySecurePort': u'5903', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet1': {'state': 'unknown', 'macAddr': u'00:1a:4a:a8:64:05', 'speed': '1000', 'name': u'vnet1'}, u'vnet2': {'state': 'unknown', 'macAddr': u'00:1a:4a:a8:64:0b', 'speed': '1000', 'name': u'vnet2'}}, 'vmId': '4b8ed9c8-dfb1-4de4-935d-6ba4009567b9', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '30064771072', 'writeLatency': '0', 'imageID': 'b0980185-7de1-4e3b-bd76-4495169ab08b', 'flushLatency': '0', 'truesize': '30064771072'}, u'vdb': {'readLatency': '0', 'apparentsize': '826781204480', 'writeLatency': '0', 'imageID': '0b89347c-4654-42bb-a8d7-1a627130b7bd', 'flushLatency': '0', 'truesize': '826781204480'}}, 'boot': 'c', 'statsAge': '0.62', 'cpuIdle': '100.00', 'elapsedTime': '25', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '3601', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:07:22,950::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.246242 s, 4.2 MB/s\n'; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:23,265::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:23,289::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:23,290::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=7 bs=512 if=/dev/772472cb-2429-466d-96cd-00b3301ef78a/metadata count=1' (cwd None)
>Thread-15913510::DEBUG::2013-11-20 03:07:23,688::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.0003183 s, 1.6 MB/s\n'; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:23,689::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.0003183 s, 1.6 MB/s'], size: 512
>Thread-15913510::DEBUG::2013-11-20 03:07:23,690::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=7 bs=512 if=/dev/772472cb-2429-466d-96cd-00b3301ef78a/metadata count=1' (cwd None)
>Thread-15913510::DEBUG::2013-11-20 03:07:24,082::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.00032937 s, 1.6 MB/s\n'; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:24,083::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.00032937 s, 1.6 MB/s'], size: 512
>Thread-15913510::DEBUG::2013-11-20 03:07:24,084::resourceManager::155::ResourceManager.Request::(__init__) ResName=`772472cb-2429-466d-96cd-00b3301ef78a_lvmActivationNS.e6984064-e09e-4c0c-8b51-a8a1711994cf`ReqID=`b9f74782-7b06-441d-a176-903d0ae7d74f`::Request was made in '/usr/share/vdsm/storage/volume.py' line '533' at 'prepare'
>Thread-15913510::DEBUG::2013-11-20 03:07:24,085::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '772472cb-2429-466d-96cd-00b3301ef78a_lvmActivationNS.e6984064-e09e-4c0c-8b51-a8a1711994cf' for lock type 'exclusive'
>Thread-15913510::DEBUG::2013-11-20 03:07:24,086::resourceManager::505::ResourceManager::(registerResource) Resource '772472cb-2429-466d-96cd-00b3301ef78a_lvmActivationNS.e6984064-e09e-4c0c-8b51-a8a1711994cf' is free. Now locking as 'exclusive' (1 active user)
>Thread-15913510::DEBUG::2013-11-20 03:07:24,087::resourceManager::192::ResourceManager.Request::(grant) ResName=`772472cb-2429-466d-96cd-00b3301ef78a_lvmActivationNS.e6984064-e09e-4c0c-8b51-a8a1711994cf`ReqID=`b9f74782-7b06-441d-a176-903d0ae7d74f`::Granted request
>Thread-15913510::DEBUG::2013-11-20 03:07:24,089::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=7 bs=512 if=/dev/772472cb-2429-466d-96cd-00b3301ef78a/metadata count=1' (cwd None)
>Thread-15913510::DEBUG::2013-11-20 03:07:24,486::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000221009 s, 2.3 MB/s\n'; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:24,488::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000221009 s, 2.3 MB/s'], size: 512
>Thread-15913510::DEBUG::2013-11-20 03:07:24,490::task::495::TaskManager.Task::(_debug) Task a11f83b3-356c-4c77-9f46-1b6ba987df6b: finished: None
>Thread-15913510::DEBUG::2013-11-20 03:07:24,490::task::495::TaskManager.Task::(_debug) Task a11f83b3-356c-4c77-9f46-1b6ba987df6b: moving from state preparing -> state finished
>Thread-15913510::DEBUG::2013-11-20 03:07:24,494::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913510::DEBUG::2013-11-20 03:07:24,495::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913510::DEBUG::2013-11-20 03:07:24,496::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913510::DEBUG::2013-11-20 03:07:24,497::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913510::DEBUG::2013-11-20 03:07:24,497::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913510::DEBUG::2013-11-20 03:07:24,498::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913510::DEBUG::2013-11-20 03:07:24,499::task::495::TaskManager.Task::(_debug) Task a11f83b3-356c-4c77-9f46-1b6ba987df6b: ref 0 aborting False
>Thread-15913510::INFO::2013-11-20 03:07:24,500::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, Return response: {'status': {'message': 'OK', 'code': 0}}
>Thread-15913510::INFO::2013-11-20 03:07:24,501::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913510::DEBUG::2013-11-20 03:07:24,501::task::495::TaskManager.Task::(_debug) Task ca93d5ff-b466-472b-86db-24ec0ad4708e: moving from state init -> state preparing
>Thread-15913510::DEBUG::2013-11-20 03:07:24,503::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`db7ae64f-ec43-4d1b-8a04-8ce7ac8ccdb1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1666' at 'public_getVolumePath'
>Thread-15913510::DEBUG::2013-11-20 03:07:24,504::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913510::DEBUG::2013-11-20 03:07:24,505::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913510::DEBUG::2013-11-20 03:07:24,505::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`db7ae64f-ec43-4d1b-8a04-8ce7ac8ccdb1`::Granted request
>Thread-15913510::DEBUG::2013-11-20 03:07:24,506::task::495::TaskManager.Task::(_debug) Task ca93d5ff-b466-472b-86db-24ec0ad4708e: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913510::DEBUG::2013-11-20 03:07:24,507::task::495::TaskManager.Task::(_debug) Task ca93d5ff-b466-472b-86db-24ec0ad4708e: ref 1 aborting False
>Thread-15913510::DEBUG::2013-11-20 03:07:24,508::task::495::TaskManager.Task::(_debug) Task ca93d5ff-b466-472b-86db-24ec0ad4708e: finished: {'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/772472cb-2429-466d-96cd-00b3301ef78a/images/c4460480-32d3-469f-b0f4-c03bb6d74e41/e6984064-e09e-4c0c-8b51-a8a1711994cf'}
>Thread-15913510::DEBUG::2013-11-20 03:07:24,509::task::495::TaskManager.Task::(_debug) Task ca93d5ff-b466-472b-86db-24ec0ad4708e: moving from state preparing -> state finished
>Thread-15913510::DEBUG::2013-11-20 03:07:24,510::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913510::DEBUG::2013-11-20 03:07:24,510::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913510::DEBUG::2013-11-20 03:07:24,511::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913510::DEBUG::2013-11-20 03:07:24,512::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913510::DEBUG::2013-11-20 03:07:24,513::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913510::DEBUG::2013-11-20 03:07:24,513::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913510::DEBUG::2013-11-20 03:07:24,514::task::495::TaskManager.Task::(_debug) Task ca93d5ff-b466-472b-86db-24ec0ad4708e: ref 0 aborting False
>Thread-15913510::INFO::2013-11-20 03:07:24,515::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, Return response: {'status': {'message': 'OK', 'code': 0}, 'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/772472cb-2429-466d-96cd-00b3301ef78a/images/c4460480-32d3-469f-b0f4-c03bb6d74e41/e6984064-e09e-4c0c-8b51-a8a1711994cf'}
>Thread-15913510::INFO::2013-11-20 03:07:24,515::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913510::DEBUG::2013-11-20 03:07:24,516::task::495::TaskManager.Task::(_debug) Task 24b989cb-d7f3-43d9-8f2f-e17729224734: moving from state init -> state preparing
>Thread-15913510::DEBUG::2013-11-20 03:07:24,517::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`627d5507-97d4-4c38-a267-7eccd932a56a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1690' at 'public_prepareVolume'
>Thread-15913510::DEBUG::2013-11-20 03:07:24,518::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913510::DEBUG::2013-11-20 03:07:24,519::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913510::DEBUG::2013-11-20 03:07:24,520::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`627d5507-97d4-4c38-a267-7eccd932a56a`::Granted request
>Thread-15913510::DEBUG::2013-11-20 03:07:24,530::task::495::TaskManager.Task::(_debug) Task 24b989cb-d7f3-43d9-8f2f-e17729224734: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913510::DEBUG::2013-11-20 03:07:24,531::task::495::TaskManager.Task::(_debug) Task 24b989cb-d7f3-43d9-8f2f-e17729224734: ref 1 aborting False
>Thread-15913510::DEBUG::2013-11-20 03:07:24,533::resourceManager::155::ResourceManager.Request::(__init__) ResName=`772472cb-2429-466d-96cd-00b3301ef78a_imageNS.df8b790c-c762-4feb-9378-d64a321ce43c`ReqID=`bc64df71-203d-46b4-bf3d-73c43284919a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1695' at 'public_prepareVolume'
>Thread-15913510::DEBUG::2013-11-20 03:07:24,535::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '772472cb-2429-466d-96cd-00b3301ef78a_imageNS.df8b790c-c762-4feb-9378-d64a321ce43c' for lock type 'exclusive'
>Thread-15913510::DEBUG::2013-11-20 03:07:24,536::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:24,538::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:07:24,965::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913510::DEBUG::2013-11-20 03:07:25,139::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:25,158::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:25,166::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=6 bs=512 if=/dev/772472cb-2429-466d-96cd-00b3301ef78a/metadata count=1' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:07:25,563::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.140498 s, 7.3 MB/s\n'; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:25,577::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.0001589 s, 3.2 MB/s\n'; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:25,577::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.0001589 s, 3.2 MB/s'], size: 512
>Thread-15913510::DEBUG::2013-11-20 03:07:25,578::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=6 bs=512 if=/dev/772472cb-2429-466d-96cd-00b3301ef78a/metadata count=1' (cwd None)
>Thread-15913527::DEBUG::2013-11-20 03:07:25,856::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('4b8ed9c8-dfb1-4de4-935d-6ba4009567b9',) {}
>Thread-15913527::DEBUG::2013-11-20 03:07:25,857::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::CPU stats not available
>Thread-15913527::DEBUG::2013-11-20 03:07:25,858::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk stats not available
>Thread-15913527::DEBUG::2013-11-20 03:07:25,859::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk stats not available
>Thread-15913527::DEBUG::2013-11-20 03:07:25,859::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk latency not available
>Thread-15913527::DEBUG::2013-11-20 03:07:25,860::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk latency not available
>Thread-15913527::DEBUG::2013-11-20 03:07:25,861::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '46387', 'displayIp': '0', 'displayPort': u'5902', 'session': 'Unknown', 'displaySecurePort': u'5903', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet1': {'macAddr': u'00:1a:4a:a8:64:05', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet1'}, u'vnet2': {'macAddr': u'00:1a:4a:a8:64:0b', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet2'}}, 'vmId': '4b8ed9c8-dfb1-4de4-935d-6ba4009567b9', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '30064771072', 'writeLatency': '0', 'imageID': 'b0980185-7de1-4e3b-bd76-4495169ab08b', 'flushLatency': '0', 'truesize': '30064771072'}, u'vdb': {'readLatency': '0', 'apparentsize': '826781204480', 'writeLatency': '0', 'imageID': '0b89347c-4654-42bb-a8d7-1a627130b7bd', 'flushLatency': '0', 'truesize': '826781204480'}}, 'boot': 'c', 'statsAge': '0.42', 'cpuIdle': '100.00', 'elapsedTime': '28', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '3601', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15913510::DEBUG::2013-11-20 03:07:25,917::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000271766 s, 1.9 MB/s\n'; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:25,917::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000271766 s, 1.9 MB/s'], size: 512
>Thread-15913510::INFO::2013-11-20 03:07:25,918::image::340::Storage.Image::(getChain) sdUUID=772472cb-2429-466d-96cd-00b3301ef78a imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c chain=[<storage.blockVolume.BlockVolume instance at 0x7f4884183998>] 
>Thread-15913510::DEBUG::2013-11-20 03:07:25,919::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --autobackup n --available y 772472cb-2429-466d-96cd-00b3301ef78a/2e903e43-1016-4339-a7f3-c5723a995725' (cwd None)
>Thread-15913510::DEBUG::2013-11-20 03:07:26,592::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:26,594::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:26,595::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:26,596::resourceManager::155::ResourceManager.Request::(__init__) ResName=`772472cb-2429-466d-96cd-00b3301ef78a_volumeNS.2e903e43-1016-4339-a7f3-c5723a995725`ReqID=`8c78613c-cf3a-4f65-83a7-5d7ea4cc7c16`::Request was made in '/usr/share/vdsm/storage/resourceFactories.py' line '149' at '__getResourceCandidatesList'
>Thread-15913510::DEBUG::2013-11-20 03:07:26,597::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '772472cb-2429-466d-96cd-00b3301ef78a_volumeNS.2e903e43-1016-4339-a7f3-c5723a995725' for lock type 'exclusive'
>Thread-15913510::DEBUG::2013-11-20 03:07:26,597::resourceManager::505::ResourceManager::(registerResource) Resource '772472cb-2429-466d-96cd-00b3301ef78a_volumeNS.2e903e43-1016-4339-a7f3-c5723a995725' is free. Now locking as 'exclusive' (1 active user)
>Thread-15913510::DEBUG::2013-11-20 03:07:26,598::resourceManager::192::ResourceManager.Request::(grant) ResName=`772472cb-2429-466d-96cd-00b3301ef78a_volumeNS.2e903e43-1016-4339-a7f3-c5723a995725`ReqID=`8c78613c-cf3a-4f65-83a7-5d7ea4cc7c16`::Granted request
>Thread-15913510::DEBUG::2013-11-20 03:07:26,600::resourceManager::505::ResourceManager::(registerResource) Resource '772472cb-2429-466d-96cd-00b3301ef78a_imageNS.df8b790c-c762-4feb-9378-d64a321ce43c' is free. Now locking as 'exclusive' (1 active user)
>Thread-15913510::DEBUG::2013-11-20 03:07:26,600::resourceManager::192::ResourceManager.Request::(grant) ResName=`772472cb-2429-466d-96cd-00b3301ef78a_imageNS.df8b790c-c762-4feb-9378-d64a321ce43c`ReqID=`bc64df71-203d-46b4-bf3d-73c43284919a`::Granted request
>Thread-15913510::DEBUG::2013-11-20 03:07:26,602::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:26,603::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-15912392::INFO::2013-11-20 03:07:26,802::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:07:26,803::task::495::TaskManager.Task::(_debug) Task 73feca8b-63dc-4d30-b69f-f4ecb94d1d58: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:07:26,805::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c260c3f2-b3a2-4f9f-a10a-2438286ac70f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:07:26,805::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:07:26,806::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:07:26,806::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c260c3f2-b3a2-4f9f-a10a-2438286ac70f`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:07:26,807::task::495::TaskManager.Task::(_debug) Task 73feca8b-63dc-4d30-b69f-f4ecb94d1d58: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:07:26,808::task::495::TaskManager.Task::(_debug) Task 73feca8b-63dc-4d30-b69f-f4ecb94d1d58: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:07:26,808::task::495::TaskManager.Task::(_debug) Task 73feca8b-63dc-4d30-b69f-f4ecb94d1d58: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:07:26,808::task::495::TaskManager.Task::(_debug) Task 73feca8b-63dc-4d30-b69f-f4ecb94d1d58: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:07:26,809::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:07:26,809::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:07:26,810::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:07:26,810::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:07:26,810::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:07:26,811::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:07:26,811::task::495::TaskManager.Task::(_debug) Task 73feca8b-63dc-4d30-b69f-f4ecb94d1d58: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:07:26,811::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913510::DEBUG::2013-11-20 03:07:27,279::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:27,307::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:27,312::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --refresh 772472cb-2429-466d-96cd-00b3301ef78a/2e903e43-1016-4339-a7f3-c5723a995725' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:07:27,578::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:07:28,090::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.048421 s, 21.1 MB/s\n'; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:28,117::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:28,118::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:28,119::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913510::INFO::2013-11-20 03:07:28,120::volume::517::Storage.Volume::(prepare) Volume: preparing volume 772472cb-2429-466d-96cd-00b3301ef78a/2e903e43-1016-4339-a7f3-c5723a995725
>Thread-15913510::DEBUG::2013-11-20 03:07:28,120::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:28,126::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-15913510::DEBUG::2013-11-20 03:07:28,773::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:28,787::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:28,788::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=6 bs=512 if=/dev/772472cb-2429-466d-96cd-00b3301ef78a/metadata count=1' (cwd None)
>Thread-15913529::DEBUG::2013-11-20 03:07:28,982::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('4b8ed9c8-dfb1-4de4-935d-6ba4009567b9',) {}
>Thread-15913529::DEBUG::2013-11-20 03:07:28,983::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::CPU stats not available
>Thread-15913529::DEBUG::2013-11-20 03:07:28,984::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk stats not available
>Thread-15913529::DEBUG::2013-11-20 03:07:28,985::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk stats not available
>Thread-15913529::DEBUG::2013-11-20 03:07:28,985::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk latency not available
>Thread-15913529::DEBUG::2013-11-20 03:07:28,986::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk latency not available
>Thread-15913529::DEBUG::2013-11-20 03:07:28,987::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '46387', 'displayIp': '0', 'displayPort': u'5902', 'session': 'Unknown', 'displaySecurePort': u'5903', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet1': {'macAddr': u'00:1a:4a:a8:64:05', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet1'}, u'vnet2': {'macAddr': u'00:1a:4a:a8:64:0b', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet2'}}, 'vmId': '4b8ed9c8-dfb1-4de4-935d-6ba4009567b9', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '30064771072', 'writeLatency': '0', 'imageID': 'b0980185-7de1-4e3b-bd76-4495169ab08b', 'flushLatency': '0', 'truesize': '30064771072'}, u'vdb': {'readLatency': '0', 'apparentsize': '826781204480', 'writeLatency': '0', 'imageID': '0b89347c-4654-42bb-a8d7-1a627130b7bd', 'flushLatency': '0', 'truesize': '826781204480'}}, 'boot': 'c', 'statsAge': '0.53', 'cpuIdle': '100.00', 'elapsedTime': '31', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '3601', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15913510::DEBUG::2013-11-20 03:07:29,116::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000246451 s, 2.1 MB/s\n'; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:29,117::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000246451 s, 2.1 MB/s'], size: 512
>Thread-15913510::DEBUG::2013-11-20 03:07:29,117::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=6 bs=512 if=/dev/772472cb-2429-466d-96cd-00b3301ef78a/metadata count=1' (cwd None)
>Thread-15913510::DEBUG::2013-11-20 03:07:29,499::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000279414 s, 1.8 MB/s\n'; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:29,501::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000279414 s, 1.8 MB/s'], size: 512
>Thread-15913510::DEBUG::2013-11-20 03:07:29,503::resourceManager::155::ResourceManager.Request::(__init__) ResName=`772472cb-2429-466d-96cd-00b3301ef78a_lvmActivationNS.2e903e43-1016-4339-a7f3-c5723a995725`ReqID=`37ad25ea-25d0-47de-ba4a-7668f32ea2ce`::Request was made in '/usr/share/vdsm/storage/volume.py' line '533' at 'prepare'
>Thread-15913510::DEBUG::2013-11-20 03:07:29,504::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '772472cb-2429-466d-96cd-00b3301ef78a_lvmActivationNS.2e903e43-1016-4339-a7f3-c5723a995725' for lock type 'exclusive'
>Thread-15913510::DEBUG::2013-11-20 03:07:29,506::resourceManager::505::ResourceManager::(registerResource) Resource '772472cb-2429-466d-96cd-00b3301ef78a_lvmActivationNS.2e903e43-1016-4339-a7f3-c5723a995725' is free. Now locking as 'exclusive' (1 active user)
>Thread-15913510::DEBUG::2013-11-20 03:07:29,507::resourceManager::192::ResourceManager.Request::(grant) ResName=`772472cb-2429-466d-96cd-00b3301ef78a_lvmActivationNS.2e903e43-1016-4339-a7f3-c5723a995725`ReqID=`37ad25ea-25d0-47de-ba4a-7668f32ea2ce`::Granted request
>Thread-15913510::DEBUG::2013-11-20 03:07:29,509::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=6 bs=512 if=/dev/772472cb-2429-466d-96cd-00b3301ef78a/metadata count=1' (cwd None)
>Thread-15913510::DEBUG::2013-11-20 03:07:29,932::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000196792 s, 2.6 MB/s\n'; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:29,933::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000196792 s, 2.6 MB/s'], size: 512
>Thread-15913510::DEBUG::2013-11-20 03:07:29,935::task::495::TaskManager.Task::(_debug) Task 24b989cb-d7f3-43d9-8f2f-e17729224734: finished: None
>Thread-15913510::DEBUG::2013-11-20 03:07:29,936::task::495::TaskManager.Task::(_debug) Task 24b989cb-d7f3-43d9-8f2f-e17729224734: moving from state preparing -> state finished
>Thread-15913510::DEBUG::2013-11-20 03:07:29,936::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913510::DEBUG::2013-11-20 03:07:29,937::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913510::DEBUG::2013-11-20 03:07:29,938::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913510::DEBUG::2013-11-20 03:07:29,939::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913510::DEBUG::2013-11-20 03:07:29,939::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913510::DEBUG::2013-11-20 03:07:29,940::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913510::DEBUG::2013-11-20 03:07:29,940::task::495::TaskManager.Task::(_debug) Task 24b989cb-d7f3-43d9-8f2f-e17729224734: ref 0 aborting False
>Thread-15913510::INFO::2013-11-20 03:07:29,941::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, Return response: {'status': {'message': 'OK', 'code': 0}}
>Thread-15913510::INFO::2013-11-20 03:07:29,942::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913510::DEBUG::2013-11-20 03:07:29,942::task::495::TaskManager.Task::(_debug) Task 2418428e-77e7-46ab-a63d-934a29257383: moving from state init -> state preparing
>Thread-15913510::DEBUG::2013-11-20 03:07:29,944::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`734a3000-5d6e-4f8c-9f8d-b2b71e544af2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1666' at 'public_getVolumePath'
>Thread-15913510::DEBUG::2013-11-20 03:07:29,944::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913510::DEBUG::2013-11-20 03:07:29,945::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913510::DEBUG::2013-11-20 03:07:29,945::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`734a3000-5d6e-4f8c-9f8d-b2b71e544af2`::Granted request
>Thread-15913510::DEBUG::2013-11-20 03:07:29,946::task::495::TaskManager.Task::(_debug) Task 2418428e-77e7-46ab-a63d-934a29257383: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913510::DEBUG::2013-11-20 03:07:29,947::task::495::TaskManager.Task::(_debug) Task 2418428e-77e7-46ab-a63d-934a29257383: ref 1 aborting False
>Thread-15913510::DEBUG::2013-11-20 03:07:29,948::task::495::TaskManager.Task::(_debug) Task 2418428e-77e7-46ab-a63d-934a29257383: finished: {'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/772472cb-2429-466d-96cd-00b3301ef78a/images/df8b790c-c762-4feb-9378-d64a321ce43c/2e903e43-1016-4339-a7f3-c5723a995725'}
>Thread-15913510::DEBUG::2013-11-20 03:07:29,949::task::495::TaskManager.Task::(_debug) Task 2418428e-77e7-46ab-a63d-934a29257383: moving from state preparing -> state finished
>Thread-15913510::DEBUG::2013-11-20 03:07:29,949::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913510::DEBUG::2013-11-20 03:07:29,950::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913510::DEBUG::2013-11-20 03:07:29,950::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913510::DEBUG::2013-11-20 03:07:29,951::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913510::DEBUG::2013-11-20 03:07:29,951::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913510::DEBUG::2013-11-20 03:07:29,952::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913510::DEBUG::2013-11-20 03:07:29,952::task::495::TaskManager.Task::(_debug) Task 2418428e-77e7-46ab-a63d-934a29257383: ref 0 aborting False
>Thread-15913510::INFO::2013-11-20 03:07:29,953::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, Return response: {'status': {'message': 'OK', 'code': 0}, 'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/772472cb-2429-466d-96cd-00b3301ef78a/images/df8b790c-c762-4feb-9378-d64a321ce43c/2e903e43-1016-4339-a7f3-c5723a995725'}
>Thread-15913510::INFO::2013-11-20 03:07:29,954::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913510::DEBUG::2013-11-20 03:07:29,954::task::495::TaskManager.Task::(_debug) Task 5b9741d7-a3ab-4621-ae82-1f3cdad89906: moving from state init -> state preparing
>Thread-15913510::DEBUG::2013-11-20 03:07:29,955::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6b146df4-bcc3-49c1-9576-d4df50c45155`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1690' at 'public_prepareVolume'
>Thread-15913510::DEBUG::2013-11-20 03:07:29,956::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913510::DEBUG::2013-11-20 03:07:29,956::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913510::DEBUG::2013-11-20 03:07:29,957::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6b146df4-bcc3-49c1-9576-d4df50c45155`::Granted request
>Thread-15913510::DEBUG::2013-11-20 03:07:29,958::task::495::TaskManager.Task::(_debug) Task 5b9741d7-a3ab-4621-ae82-1f3cdad89906: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913510::DEBUG::2013-11-20 03:07:29,958::task::495::TaskManager.Task::(_debug) Task 5b9741d7-a3ab-4621-ae82-1f3cdad89906: ref 1 aborting False
>Thread-15913510::DEBUG::2013-11-20 03:07:29,959::resourceManager::155::ResourceManager.Request::(__init__) ResName=`772472cb-2429-466d-96cd-00b3301ef78a_imageNS.13efb45b-f4d2-4c94-bbdb-91ce9b9b7321`ReqID=`271150b3-a987-44f3-9b21-c01ebef1b045`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1695' at 'public_prepareVolume'
>Thread-15913510::DEBUG::2013-11-20 03:07:29,960::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '772472cb-2429-466d-96cd-00b3301ef78a_imageNS.13efb45b-f4d2-4c94-bbdb-91ce9b9b7321' for lock type 'exclusive'
>Thread-15913510::DEBUG::2013-11-20 03:07:29,961::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:29,972::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:07:30,156::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913510::DEBUG::2013-11-20 03:07:30,710::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:30,741::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:30,749::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=5 bs=512 if=/dev/772472cb-2429-466d-96cd-00b3301ef78a/metadata count=1' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:07:31,123::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.384285 s, 2.7 MB/s\n'; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:31,142::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000296711 s, 1.7 MB/s\n'; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:31,143::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000296711 s, 1.7 MB/s'], size: 512
>Thread-15913510::DEBUG::2013-11-20 03:07:31,151::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=5 bs=512 if=/dev/772472cb-2429-466d-96cd-00b3301ef78a/metadata count=1' (cwd None)
>Thread-15913510::DEBUG::2013-11-20 03:07:31,596::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000702549 s, 729 kB/s\n'; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:31,598::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000702549 s, 729 kB/s'], size: 512
>Thread-15913510::INFO::2013-11-20 03:07:31,599::image::340::Storage.Image::(getChain) sdUUID=772472cb-2429-466d-96cd-00b3301ef78a imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 chain=[<storage.blockVolume.BlockVolume instance at 0x7f4884183680>] 
>Thread-15913510::DEBUG::2013-11-20 03:07:31,601::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --autobackup n --available y 772472cb-2429-466d-96cd-00b3301ef78a/088640f7-316c-4e2b-bfdc-0f64cf2913c8' (cwd None)
>Thread-15913530::INFO::2013-11-20 03:07:32,011::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913530::DEBUG::2013-11-20 03:07:32,012::task::495::TaskManager.Task::(_debug) Task cfeb5266-6674-4310-9f4c-fd37f859485b: moving from state init -> state preparing
>Thread-15913530::DEBUG::2013-11-20 03:07:32,013::task::495::TaskManager.Task::(_debug) Task cfeb5266-6674-4310-9f4c-fd37f859485b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000334978103638', 'lastCheck': 1384916847.912523, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.225172042847', 'lastCheck': 1384916844.7642519, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.240998983383', 'lastCheck': 1384916846.1448989, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00772905349731', 'lastCheck': 1384916845.4402161, 'valid': True, 'code': 0}}
>Thread-15913530::DEBUG::2013-11-20 03:07:32,014::task::495::TaskManager.Task::(_debug) Task cfeb5266-6674-4310-9f4c-fd37f859485b: moving from state preparing -> state finished
>Thread-15913530::DEBUG::2013-11-20 03:07:32,014::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913530::DEBUG::2013-11-20 03:07:32,015::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913530::DEBUG::2013-11-20 03:07:32,015::task::495::TaskManager.Task::(_debug) Task cfeb5266-6674-4310-9f4c-fd37f859485b: ref 0 aborting False
>Thread-15913530::INFO::2013-11-20 03:07:32,016::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000334978103638', 'lastCheck': 1384916847.912523, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.225172042847', 'lastCheck': 1384916844.7642519, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.240998983383', 'lastCheck': 1384916846.1448989, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00772905349731', 'lastCheck': 1384916845.4402161, 'valid': True, 'code': 0}}
>Thread-15913531::DEBUG::2013-11-20 03:07:32,030::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::CPU stats not available
>Thread-15913531::DEBUG::2013-11-20 03:07:32,030::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk stats not available
>Thread-15913531::DEBUG::2013-11-20 03:07:32,031::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk stats not available
>Thread-15913531::DEBUG::2013-11-20 03:07:32,031::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk latency not available
>Thread-15913531::DEBUG::2013-11-20 03:07:32,033::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk latency not available
>Thread-15913532::DEBUG::2013-11-20 03:07:32,053::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913532::INFO::2013-11-20 03:07:32,054::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913532::DEBUG::2013-11-20 03:07:32,055::task::495::TaskManager.Task::(_debug) Task 87a54070-bf4a-423c-8f26-d902002437d9: moving from state init -> state preparing
>Thread-15913532::DEBUG::2013-11-20 03:07:32,055::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913532::DEBUG::2013-11-20 03:07:32,056::task::495::TaskManager.Task::(_debug) Task 87a54070-bf4a-423c-8f26-d902002437d9: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913532::DEBUG::2013-11-20 03:07:32,056::task::495::TaskManager.Task::(_debug) Task 87a54070-bf4a-423c-8f26-d902002437d9: moving from state preparing -> state finished
>Thread-15913532::DEBUG::2013-11-20 03:07:32,057::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913532::DEBUG::2013-11-20 03:07:32,057::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913532::DEBUG::2013-11-20 03:07:32,058::task::495::TaskManager.Task::(_debug) Task 87a54070-bf4a-423c-8f26-d902002437d9: ref 0 aborting False
>Thread-15913532::INFO::2013-11-20 03:07:32,059::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913533::DEBUG::2013-11-20 03:07:32,063::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913533::INFO::2013-11-20 03:07:32,064::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913533::DEBUG::2013-11-20 03:07:32,064::task::495::TaskManager.Task::(_debug) Task aff89841-7c4a-4bc0-a0ca-6c32d4205b70: moving from state init -> state preparing
>Thread-15913533::DEBUG::2013-11-20 03:07:32,065::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9928261b-6e46-4b47-812f-31187b818477`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913533::DEBUG::2013-11-20 03:07:32,066::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913533::DEBUG::2013-11-20 03:07:32,067::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913533::DEBUG::2013-11-20 03:07:32,067::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9928261b-6e46-4b47-812f-31187b818477`::Granted request
>Thread-15913533::DEBUG::2013-11-20 03:07:32,068::task::495::TaskManager.Task::(_debug) Task aff89841-7c4a-4bc0-a0ca-6c32d4205b70: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913533::DEBUG::2013-11-20 03:07:32,069::task::495::TaskManager.Task::(_debug) Task aff89841-7c4a-4bc0-a0ca-6c32d4205b70: ref 1 aborting False
>Thread-15913533::WARNING::2013-11-20 03:07:32,071::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913533::DEBUG::2013-11-20 03:07:32,072::task::495::TaskManager.Task::(_debug) Task aff89841-7c4a-4bc0-a0ca-6c32d4205b70: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913533::DEBUG::2013-11-20 03:07:32,072::task::495::TaskManager.Task::(_debug) Task aff89841-7c4a-4bc0-a0ca-6c32d4205b70: moving from state preparing -> state finished
>Thread-15913533::DEBUG::2013-11-20 03:07:32,073::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913533::DEBUG::2013-11-20 03:07:32,073::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913533::DEBUG::2013-11-20 03:07:32,074::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913533::DEBUG::2013-11-20 03:07:32,075::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913533::DEBUG::2013-11-20 03:07:32,075::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913533::DEBUG::2013-11-20 03:07:32,076::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913533::DEBUG::2013-11-20 03:07:32,076::task::495::TaskManager.Task::(_debug) Task aff89841-7c4a-4bc0-a0ca-6c32d4205b70: ref 0 aborting False
>Thread-15913533::INFO::2013-11-20 03:07:32,077::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913510::DEBUG::2013-11-20 03:07:32,383::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:32,384::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:32,385::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:32,386::resourceManager::155::ResourceManager.Request::(__init__) ResName=`772472cb-2429-466d-96cd-00b3301ef78a_volumeNS.088640f7-316c-4e2b-bfdc-0f64cf2913c8`ReqID=`2e2ab8cd-2af8-45cb-9048-a361977d94d2`::Request was made in '/usr/share/vdsm/storage/resourceFactories.py' line '149' at '__getResourceCandidatesList'
>Thread-15913510::DEBUG::2013-11-20 03:07:32,387::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '772472cb-2429-466d-96cd-00b3301ef78a_volumeNS.088640f7-316c-4e2b-bfdc-0f64cf2913c8' for lock type 'exclusive'
>Thread-15913510::DEBUG::2013-11-20 03:07:32,388::resourceManager::505::ResourceManager::(registerResource) Resource '772472cb-2429-466d-96cd-00b3301ef78a_volumeNS.088640f7-316c-4e2b-bfdc-0f64cf2913c8' is free. Now locking as 'exclusive' (1 active user)
>Thread-15913510::DEBUG::2013-11-20 03:07:32,389::resourceManager::192::ResourceManager.Request::(grant) ResName=`772472cb-2429-466d-96cd-00b3301ef78a_volumeNS.088640f7-316c-4e2b-bfdc-0f64cf2913c8`ReqID=`2e2ab8cd-2af8-45cb-9048-a361977d94d2`::Granted request
>Thread-15913510::DEBUG::2013-11-20 03:07:32,390::resourceManager::505::ResourceManager::(registerResource) Resource '772472cb-2429-466d-96cd-00b3301ef78a_imageNS.13efb45b-f4d2-4c94-bbdb-91ce9b9b7321' is free. Now locking as 'exclusive' (1 active user)
>Thread-15913510::DEBUG::2013-11-20 03:07:32,391::resourceManager::192::ResourceManager.Request::(grant) ResName=`772472cb-2429-466d-96cd-00b3301ef78a_imageNS.13efb45b-f4d2-4c94-bbdb-91ce9b9b7321`ReqID=`271150b3-a987-44f3-9b21-c01ebef1b045`::Granted request
>Thread-15913510::DEBUG::2013-11-20 03:07:32,392::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:32,393::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-15913510::DEBUG::2013-11-20 03:07:33,067::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:33,091::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:33,092::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --refresh 772472cb-2429-466d-96cd-00b3301ef78a/088640f7-316c-4e2b-bfdc-0f64cf2913c8' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:07:33,289::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913510::DEBUG::2013-11-20 03:07:33,829::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:33,831::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:33,833::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913510::INFO::2013-11-20 03:07:33,833::volume::517::Storage.Volume::(prepare) Volume: preparing volume 772472cb-2429-466d-96cd-00b3301ef78a/088640f7-316c-4e2b-bfdc-0f64cf2913c8
>Thread-15913510::DEBUG::2013-11-20 03:07:33,835::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:33,837::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:07:34,188::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.337543 s, 3.0 MB/s\n'; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:34,534::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:34,559::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:34,561::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=5 bs=512 if=/dev/772472cb-2429-466d-96cd-00b3301ef78a/metadata count=1' (cwd None)
>Thread-15913510::DEBUG::2013-11-20 03:07:34,962::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000196947 s, 2.6 MB/s\n'; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:34,963::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000196947 s, 2.6 MB/s'], size: 512
>Thread-15913510::DEBUG::2013-11-20 03:07:34,964::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=5 bs=512 if=/dev/772472cb-2429-466d-96cd-00b3301ef78a/metadata count=1' (cwd None)
>Thread-15913510::DEBUG::2013-11-20 03:07:35,352::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000221347 s, 2.3 MB/s\n'; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:35,353::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000221347 s, 2.3 MB/s'], size: 512
>Thread-15913510::DEBUG::2013-11-20 03:07:35,354::resourceManager::155::ResourceManager.Request::(__init__) ResName=`772472cb-2429-466d-96cd-00b3301ef78a_lvmActivationNS.088640f7-316c-4e2b-bfdc-0f64cf2913c8`ReqID=`1b8c91d9-a270-4fdc-9cd5-b2e9cd5892ee`::Request was made in '/usr/share/vdsm/storage/volume.py' line '533' at 'prepare'
>Thread-15913510::DEBUG::2013-11-20 03:07:35,355::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '772472cb-2429-466d-96cd-00b3301ef78a_lvmActivationNS.088640f7-316c-4e2b-bfdc-0f64cf2913c8' for lock type 'exclusive'
>Thread-15913510::DEBUG::2013-11-20 03:07:35,355::resourceManager::505::ResourceManager::(registerResource) Resource '772472cb-2429-466d-96cd-00b3301ef78a_lvmActivationNS.088640f7-316c-4e2b-bfdc-0f64cf2913c8' is free. Now locking as 'exclusive' (1 active user)
>Thread-15913510::DEBUG::2013-11-20 03:07:35,356::resourceManager::192::ResourceManager.Request::(grant) ResName=`772472cb-2429-466d-96cd-00b3301ef78a_lvmActivationNS.088640f7-316c-4e2b-bfdc-0f64cf2913c8`ReqID=`1b8c91d9-a270-4fdc-9cd5-b2e9cd5892ee`::Granted request
>Thread-15913510::DEBUG::2013-11-20 03:07:35,357::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=5 bs=512 if=/dev/772472cb-2429-466d-96cd-00b3301ef78a/metadata count=1' (cwd None)
>Thread-15913535::DEBUG::2013-11-20 03:07:35,727::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('4b8ed9c8-dfb1-4de4-935d-6ba4009567b9',) {}
>Thread-15913535::DEBUG::2013-11-20 03:07:35,728::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::CPU stats not available
>Thread-15913535::DEBUG::2013-11-20 03:07:35,729::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk stats not available
>Thread-15913535::DEBUG::2013-11-20 03:07:35,729::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk stats not available
>Thread-15913535::DEBUG::2013-11-20 03:07:35,730::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk latency not available
>Thread-15913535::DEBUG::2013-11-20 03:07:35,730::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk latency not available
>Thread-15913535::DEBUG::2013-11-20 03:07:35,731::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '46387', 'displayIp': '0', 'displayPort': u'5902', 'session': 'Unknown', 'displaySecurePort': u'5903', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet1': {'macAddr': u'00:1a:4a:a8:64:05', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet1'}, u'vnet2': {'macAddr': u'00:1a:4a:a8:64:0b', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet2'}}, 'vmId': '4b8ed9c8-dfb1-4de4-935d-6ba4009567b9', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '30064771072', 'writeLatency': '0', 'imageID': 'b0980185-7de1-4e3b-bd76-4495169ab08b', 'flushLatency': '0', 'truesize': '30064771072'}, u'vdb': {'readLatency': '0', 'apparentsize': '826781204480', 'writeLatency': '0', 'imageID': '0b89347c-4654-42bb-a8d7-1a627130b7bd', 'flushLatency': '0', 'truesize': '826781204480'}}, 'boot': 'c', 'statsAge': '0.93', 'cpuIdle': '100.00', 'elapsedTime': '38', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '3601', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15913510::DEBUG::2013-11-20 03:07:35,736::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000146744 s, 3.5 MB/s\n'; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:35,737::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000146744 s, 3.5 MB/s'], size: 512
>Thread-15913510::DEBUG::2013-11-20 03:07:35,741::task::495::TaskManager.Task::(_debug) Task 5b9741d7-a3ab-4621-ae82-1f3cdad89906: finished: None
>Thread-15913510::DEBUG::2013-11-20 03:07:35,742::task::495::TaskManager.Task::(_debug) Task 5b9741d7-a3ab-4621-ae82-1f3cdad89906: moving from state preparing -> state finished
>Thread-15913510::DEBUG::2013-11-20 03:07:35,742::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913510::DEBUG::2013-11-20 03:07:35,742::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913510::DEBUG::2013-11-20 03:07:35,743::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913510::DEBUG::2013-11-20 03:07:35,744::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913510::DEBUG::2013-11-20 03:07:35,744::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913510::DEBUG::2013-11-20 03:07:35,744::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913510::DEBUG::2013-11-20 03:07:35,745::task::495::TaskManager.Task::(_debug) Task 5b9741d7-a3ab-4621-ae82-1f3cdad89906: ref 0 aborting False
>Thread-15913510::INFO::2013-11-20 03:07:35,745::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, Return response: {'status': {'message': 'OK', 'code': 0}}
>Thread-15913510::INFO::2013-11-20 03:07:35,746::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913510::DEBUG::2013-11-20 03:07:35,746::task::495::TaskManager.Task::(_debug) Task 2eca9d61-eaf3-4aef-b986-76c0fc3ab582: moving from state init -> state preparing
>Thread-15913510::DEBUG::2013-11-20 03:07:35,747::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b794f281-91b9-474c-991f-1c80d3808153`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1666' at 'public_getVolumePath'
>Thread-15913510::DEBUG::2013-11-20 03:07:35,748::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913510::DEBUG::2013-11-20 03:07:35,748::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913510::DEBUG::2013-11-20 03:07:35,748::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b794f281-91b9-474c-991f-1c80d3808153`::Granted request
>Thread-15913510::DEBUG::2013-11-20 03:07:35,749::task::495::TaskManager.Task::(_debug) Task 2eca9d61-eaf3-4aef-b986-76c0fc3ab582: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913510::DEBUG::2013-11-20 03:07:35,749::task::495::TaskManager.Task::(_debug) Task 2eca9d61-eaf3-4aef-b986-76c0fc3ab582: ref 1 aborting False
>Thread-15913510::DEBUG::2013-11-20 03:07:35,750::task::495::TaskManager.Task::(_debug) Task 2eca9d61-eaf3-4aef-b986-76c0fc3ab582: finished: {'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/772472cb-2429-466d-96cd-00b3301ef78a/images/13efb45b-f4d2-4c94-bbdb-91ce9b9b7321/088640f7-316c-4e2b-bfdc-0f64cf2913c8'}
>Thread-15913510::DEBUG::2013-11-20 03:07:35,750::task::495::TaskManager.Task::(_debug) Task 2eca9d61-eaf3-4aef-b986-76c0fc3ab582: moving from state preparing -> state finished
>Thread-15913510::DEBUG::2013-11-20 03:07:35,751::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913510::DEBUG::2013-11-20 03:07:35,751::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913510::DEBUG::2013-11-20 03:07:35,752::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913510::DEBUG::2013-11-20 03:07:35,752::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913510::DEBUG::2013-11-20 03:07:35,752::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913510::DEBUG::2013-11-20 03:07:35,753::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913510::DEBUG::2013-11-20 03:07:35,753::task::495::TaskManager.Task::(_debug) Task 2eca9d61-eaf3-4aef-b986-76c0fc3ab582: ref 0 aborting False
>Thread-15913510::INFO::2013-11-20 03:07:35,753::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, Return response: {'status': {'message': 'OK', 'code': 0}, 'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/772472cb-2429-466d-96cd-00b3301ef78a/images/13efb45b-f4d2-4c94-bbdb-91ce9b9b7321/088640f7-316c-4e2b-bfdc-0f64cf2913c8'}
>Thread-15913510::INFO::2013-11-20 03:07:35,754::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913510::DEBUG::2013-11-20 03:07:35,754::task::495::TaskManager.Task::(_debug) Task d069d465-769c-4c8d-a31f-064353f94623: moving from state init -> state preparing
>Thread-15913510::DEBUG::2013-11-20 03:07:35,755::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c44e0094-d1ea-4a5d-a886-8cdd338df392`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1690' at 'public_prepareVolume'
>Thread-15913510::DEBUG::2013-11-20 03:07:35,755::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913510::DEBUG::2013-11-20 03:07:35,756::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913510::DEBUG::2013-11-20 03:07:35,756::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c44e0094-d1ea-4a5d-a886-8cdd338df392`::Granted request
>Thread-15913510::DEBUG::2013-11-20 03:07:35,762::task::495::TaskManager.Task::(_debug) Task d069d465-769c-4c8d-a31f-064353f94623: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913510::DEBUG::2013-11-20 03:07:35,762::task::495::TaskManager.Task::(_debug) Task d069d465-769c-4c8d-a31f-064353f94623: ref 1 aborting False
>Thread-15913510::DEBUG::2013-11-20 03:07:35,763::resourceManager::155::ResourceManager.Request::(__init__) ResName=`772472cb-2429-466d-96cd-00b3301ef78a_imageNS.d368307e-425e-423a-85b7-266c1e4a99ec`ReqID=`ac5a0032-f66a-4183-830c-50efd74affcc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1695' at 'public_prepareVolume'
>Thread-15913510::DEBUG::2013-11-20 03:07:35,763::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '772472cb-2429-466d-96cd-00b3301ef78a_imageNS.d368307e-425e-423a-85b7-266c1e4a99ec' for lock type 'exclusive'
>Thread-15913510::DEBUG::2013-11-20 03:07:35,764::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:35,771::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:07:36,203::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913510::DEBUG::2013-11-20 03:07:36,442::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:36,460::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:36,462::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=4 bs=512 if=/dev/772472cb-2429-466d-96cd-00b3301ef78a/metadata count=1' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:07:36,912::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.170107 s, 6.0 MB/s\n'; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:36,988::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000253222 s, 2.0 MB/s\n'; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:36,990::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000253222 s, 2.0 MB/s'], size: 512
>Thread-15913510::DEBUG::2013-11-20 03:07:37,001::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=4 bs=512 if=/dev/772472cb-2429-466d-96cd-00b3301ef78a/metadata count=1' (cwd None)
>Thread-15913510::DEBUG::2013-11-20 03:07:37,444::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000338903 s, 1.5 MB/s\n'; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:37,445::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000338903 s, 1.5 MB/s'], size: 512
>Thread-15913510::INFO::2013-11-20 03:07:37,446::image::340::Storage.Image::(getChain) sdUUID=772472cb-2429-466d-96cd-00b3301ef78a imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec chain=[<storage.blockVolume.BlockVolume instance at 0x7f487c03bd40>] 
>Thread-15913510::DEBUG::2013-11-20 03:07:37,448::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --autobackup n --available y 772472cb-2429-466d-96cd-00b3301ef78a/fdc91117-476f-4b82-8fc4-d682fa283ecc' (cwd None)
>Thread-15913510::DEBUG::2013-11-20 03:07:38,110::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:38,112::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:38,113::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:38,115::resourceManager::155::ResourceManager.Request::(__init__) ResName=`772472cb-2429-466d-96cd-00b3301ef78a_volumeNS.fdc91117-476f-4b82-8fc4-d682fa283ecc`ReqID=`d0ad7fbf-de53-4daa-9081-0220ccc55a9c`::Request was made in '/usr/share/vdsm/storage/resourceFactories.py' line '149' at '__getResourceCandidatesList'
>Thread-15913510::DEBUG::2013-11-20 03:07:38,116::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '772472cb-2429-466d-96cd-00b3301ef78a_volumeNS.fdc91117-476f-4b82-8fc4-d682fa283ecc' for lock type 'exclusive'
>Thread-15913510::DEBUG::2013-11-20 03:07:38,117::resourceManager::505::ResourceManager::(registerResource) Resource '772472cb-2429-466d-96cd-00b3301ef78a_volumeNS.fdc91117-476f-4b82-8fc4-d682fa283ecc' is free. Now locking as 'exclusive' (1 active user)
>Thread-15913510::DEBUG::2013-11-20 03:07:38,118::resourceManager::192::ResourceManager.Request::(grant) ResName=`772472cb-2429-466d-96cd-00b3301ef78a_volumeNS.fdc91117-476f-4b82-8fc4-d682fa283ecc`ReqID=`d0ad7fbf-de53-4daa-9081-0220ccc55a9c`::Granted request
>Thread-15913510::DEBUG::2013-11-20 03:07:38,119::resourceManager::505::ResourceManager::(registerResource) Resource '772472cb-2429-466d-96cd-00b3301ef78a_imageNS.d368307e-425e-423a-85b7-266c1e4a99ec' is free. Now locking as 'exclusive' (1 active user)
>Thread-15913510::DEBUG::2013-11-20 03:07:38,120::resourceManager::192::ResourceManager.Request::(grant) ResName=`772472cb-2429-466d-96cd-00b3301ef78a_imageNS.d368307e-425e-423a-85b7-266c1e4a99ec`ReqID=`ac5a0032-f66a-4183-830c-50efd74affcc`::Granted request
>Thread-15913510::DEBUG::2013-11-20 03:07:38,123::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:38,125::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-15913537::DEBUG::2013-11-20 03:07:38,757::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('4b8ed9c8-dfb1-4de4-935d-6ba4009567b9',) {}
>Thread-15913537::DEBUG::2013-11-20 03:07:38,759::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk stats not available
>Thread-15913537::DEBUG::2013-11-20 03:07:38,759::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk stats not available
>Thread-15913537::DEBUG::2013-11-20 03:07:38,760::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk latency not available
>Thread-15913537::DEBUG::2013-11-20 03:07:38,760::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk latency not available
>Thread-15913537::DEBUG::2013-11-20 03:07:38,761::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '46387', 'displayIp': '0', 'displayPort': u'5902', 'session': 'Unknown', 'displaySecurePort': u'5903', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet1': {'macAddr': u'00:1a:4a:a8:64:05', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet1'}, u'vnet2': {'macAddr': u'00:1a:4a:a8:64:0b', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet2'}}, 'vmId': '4b8ed9c8-dfb1-4de4-935d-6ba4009567b9', 'monitorResponse': '0', 'cpuUser': '75.79', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '30064771072', 'writeLatency': '0', 'imageID': 'b0980185-7de1-4e3b-bd76-4495169ab08b', 'flushLatency': '0', 'truesize': '30064771072'}, u'vdb': {'readLatency': '0', 'apparentsize': '826781204480', 'writeLatency': '0', 'imageID': '0b89347c-4654-42bb-a8d7-1a627130b7bd', 'flushLatency': '0', 'truesize': '826781204480'}}, 'boot': 'c', 'statsAge': '1.76', 'cpuIdle': '24.21', 'elapsedTime': '41', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '3601', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15913510::DEBUG::2013-11-20 03:07:38,822::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:38,847::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:38,855::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --refresh 772472cb-2429-466d-96cd-00b3301ef78a/fdc91117-476f-4b82-8fc4-d682fa283ecc' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:07:39,110::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913510::DEBUG::2013-11-20 03:07:39,696::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:39,697::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:39,698::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913510::INFO::2013-11-20 03:07:39,699::volume::517::Storage.Volume::(prepare) Volume: preparing volume 772472cb-2429-466d-96cd-00b3301ef78a/fdc91117-476f-4b82-8fc4-d682fa283ecc
>Thread-15913510::DEBUG::2013-11-20 03:07:39,699::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:39,706::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:07:39,970::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.278717 s, 3.7 MB/s\n'; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:40,346::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:40,368::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:40,369::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=4 bs=512 if=/dev/772472cb-2429-466d-96cd-00b3301ef78a/metadata count=1' (cwd None)
>Thread-15913510::DEBUG::2013-11-20 03:07:40,756::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000255151 s, 2.0 MB/s\n'; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:40,757::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000255151 s, 2.0 MB/s'], size: 512
>Thread-15913510::DEBUG::2013-11-20 03:07:40,759::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=4 bs=512 if=/dev/772472cb-2429-466d-96cd-00b3301ef78a/metadata count=1' (cwd None)
>Thread-15913510::DEBUG::2013-11-20 03:07:41,229::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000292759 s, 1.7 MB/s\n'; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:41,231::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000292759 s, 1.7 MB/s'], size: 512
>Thread-15913510::DEBUG::2013-11-20 03:07:41,233::resourceManager::155::ResourceManager.Request::(__init__) ResName=`772472cb-2429-466d-96cd-00b3301ef78a_lvmActivationNS.fdc91117-476f-4b82-8fc4-d682fa283ecc`ReqID=`6264dc1b-7b1f-4852-a5da-e468ec66ada5`::Request was made in '/usr/share/vdsm/storage/volume.py' line '533' at 'prepare'
>Thread-15913510::DEBUG::2013-11-20 03:07:41,234::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '772472cb-2429-466d-96cd-00b3301ef78a_lvmActivationNS.fdc91117-476f-4b82-8fc4-d682fa283ecc' for lock type 'exclusive'
>Thread-15913510::DEBUG::2013-11-20 03:07:41,236::resourceManager::505::ResourceManager::(registerResource) Resource '772472cb-2429-466d-96cd-00b3301ef78a_lvmActivationNS.fdc91117-476f-4b82-8fc4-d682fa283ecc' is free. Now locking as 'exclusive' (1 active user)
>Thread-15913510::DEBUG::2013-11-20 03:07:41,236::resourceManager::192::ResourceManager.Request::(grant) ResName=`772472cb-2429-466d-96cd-00b3301ef78a_lvmActivationNS.fdc91117-476f-4b82-8fc4-d682fa283ecc`ReqID=`6264dc1b-7b1f-4852-a5da-e468ec66ada5`::Granted request
>Thread-15913510::DEBUG::2013-11-20 03:07:41,239::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=4 bs=512 if=/dev/772472cb-2429-466d-96cd-00b3301ef78a/metadata count=1' (cwd None)
>Thread-15913510::DEBUG::2013-11-20 03:07:41,694::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000200104 s, 2.6 MB/s\n'; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:41,696::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000200104 s, 2.6 MB/s'], size: 512
>Thread-15913510::DEBUG::2013-11-20 03:07:41,698::task::495::TaskManager.Task::(_debug) Task d069d465-769c-4c8d-a31f-064353f94623: finished: None
>Thread-15913510::DEBUG::2013-11-20 03:07:41,699::task::495::TaskManager.Task::(_debug) Task d069d465-769c-4c8d-a31f-064353f94623: moving from state preparing -> state finished
>Thread-15913510::DEBUG::2013-11-20 03:07:41,700::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913510::DEBUG::2013-11-20 03:07:41,701::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913510::DEBUG::2013-11-20 03:07:41,702::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913510::DEBUG::2013-11-20 03:07:41,703::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913510::DEBUG::2013-11-20 03:07:41,704::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913510::DEBUG::2013-11-20 03:07:41,704::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913510::DEBUG::2013-11-20 03:07:41,706::task::495::TaskManager.Task::(_debug) Task d069d465-769c-4c8d-a31f-064353f94623: ref 0 aborting False
>Thread-15913510::INFO::2013-11-20 03:07:41,707::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, Return response: {'status': {'message': 'OK', 'code': 0}}
>Thread-15913510::INFO::2013-11-20 03:07:41,708::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913510::DEBUG::2013-11-20 03:07:41,709::task::495::TaskManager.Task::(_debug) Task 94b7ec3e-f38a-4f67-969d-311086419f64: moving from state init -> state preparing
>Thread-15913510::DEBUG::2013-11-20 03:07:41,711::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`77ff3a75-11fe-47e0-96a1-7e2d3aec52de`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1666' at 'public_getVolumePath'
>Thread-15913510::DEBUG::2013-11-20 03:07:41,711::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913510::DEBUG::2013-11-20 03:07:41,712::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913510::DEBUG::2013-11-20 03:07:41,713::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`77ff3a75-11fe-47e0-96a1-7e2d3aec52de`::Granted request
>Thread-15913510::DEBUG::2013-11-20 03:07:41,715::task::495::TaskManager.Task::(_debug) Task 94b7ec3e-f38a-4f67-969d-311086419f64: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913510::DEBUG::2013-11-20 03:07:41,716::task::495::TaskManager.Task::(_debug) Task 94b7ec3e-f38a-4f67-969d-311086419f64: ref 1 aborting False
>Thread-15913510::DEBUG::2013-11-20 03:07:41,717::task::495::TaskManager.Task::(_debug) Task 94b7ec3e-f38a-4f67-969d-311086419f64: finished: {'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/772472cb-2429-466d-96cd-00b3301ef78a/images/d368307e-425e-423a-85b7-266c1e4a99ec/fdc91117-476f-4b82-8fc4-d682fa283ecc'}
>Thread-15913510::DEBUG::2013-11-20 03:07:41,718::task::495::TaskManager.Task::(_debug) Task 94b7ec3e-f38a-4f67-969d-311086419f64: moving from state preparing -> state finished
>Thread-15913510::DEBUG::2013-11-20 03:07:41,719::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913510::DEBUG::2013-11-20 03:07:41,720::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913510::DEBUG::2013-11-20 03:07:41,721::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913510::DEBUG::2013-11-20 03:07:41,722::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913510::DEBUG::2013-11-20 03:07:41,723::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913510::DEBUG::2013-11-20 03:07:41,724::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913510::DEBUG::2013-11-20 03:07:41,725::task::495::TaskManager.Task::(_debug) Task 94b7ec3e-f38a-4f67-969d-311086419f64: ref 0 aborting False
>Thread-15913510::INFO::2013-11-20 03:07:41,726::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, Return response: {'status': {'message': 'OK', 'code': 0}, 'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/772472cb-2429-466d-96cd-00b3301ef78a/images/d368307e-425e-423a-85b7-266c1e4a99ec/fdc91117-476f-4b82-8fc4-d682fa283ecc'}
>Thread-15913510::INFO::2013-11-20 03:07:41,727::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913510::DEBUG::2013-11-20 03:07:41,728::task::495::TaskManager.Task::(_debug) Task 840ddc4a-4251-493c-a994-e80f101887ad: moving from state init -> state preparing
>Thread-15913510::DEBUG::2013-11-20 03:07:41,729::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`69aa8778-9d4b-4788-9fba-c7555ee2e8e2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913510::DEBUG::2013-11-20 03:07:41,730::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913510::DEBUG::2013-11-20 03:07:41,731::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913510::DEBUG::2013-11-20 03:07:41,732::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`69aa8778-9d4b-4788-9fba-c7555ee2e8e2`::Granted request
>Thread-15913510::DEBUG::2013-11-20 03:07:41,733::task::495::TaskManager.Task::(_debug) Task 840ddc4a-4251-493c-a994-e80f101887ad: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913510::DEBUG::2013-11-20 03:07:41,734::task::495::TaskManager.Task::(_debug) Task 840ddc4a-4251-493c-a994-e80f101887ad: ref 1 aborting False
>Thread-15913510::DEBUG::2013-11-20 03:07:41,735::task::495::TaskManager.Task::(_debug) Task 840ddc4a-4251-493c-a994-e80f101887ad: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913510::DEBUG::2013-11-20 03:07:41,736::task::495::TaskManager.Task::(_debug) Task 840ddc4a-4251-493c-a994-e80f101887ad: moving from state preparing -> state finished
>Thread-15913510::DEBUG::2013-11-20 03:07:41,736::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913510::DEBUG::2013-11-20 03:07:41,737::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913510::DEBUG::2013-11-20 03:07:41,738::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913510::DEBUG::2013-11-20 03:07:41,738::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913510::DEBUG::2013-11-20 03:07:41,739::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913510::DEBUG::2013-11-20 03:07:41,740::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913510::DEBUG::2013-11-20 03:07:41,741::task::495::TaskManager.Task::(_debug) Task 840ddc4a-4251-493c-a994-e80f101887ad: ref 0 aborting False
>Thread-15913510::INFO::2013-11-20 03:07:41,741::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913510::INFO::2013-11-20 03:07:41,743::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a)
>Thread-15913510::DEBUG::2013-11-20 03:07:41,743::task::495::TaskManager.Task::(_debug) Task 1882563a-1e45-446c-92a3-20552b325bea: moving from state init -> state preparing
>Thread-15913510::INFO::2013-11-20 03:07:41,744::blockSD::674::Storage.StorageDomain::(validate) sdUUID=772472cb-2429-466d-96cd-00b3301ef78a
>Thread-15913510::DEBUG::2013-11-20 03:07:41,746::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:41,747::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:41,748::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:41,749::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:41,750::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:41,752::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-15913539::DEBUG::2013-11-20 03:07:41,974::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('4b8ed9c8-dfb1-4de4-935d-6ba4009567b9',) {}
>Thread-15913539::DEBUG::2013-11-20 03:07:41,976::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk stats not available
>Thread-15913539::DEBUG::2013-11-20 03:07:41,977::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk stats not available
>Thread-15913539::DEBUG::2013-11-20 03:07:41,977::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk latency not available
>Thread-15913539::DEBUG::2013-11-20 03:07:41,978::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk latency not available
>Thread-15913539::DEBUG::2013-11-20 03:07:41,979::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '46387', 'displayIp': '0', 'displayPort': u'5902', 'session': 'Unknown', 'displaySecurePort': u'5903', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet1': {'macAddr': u'00:1a:4a:a8:64:05', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet1'}, u'vnet2': {'macAddr': u'00:1a:4a:a8:64:0b', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet2'}}, 'vmId': '4b8ed9c8-dfb1-4de4-935d-6ba4009567b9', 'monitorResponse': '0', 'cpuUser': '75.79', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '30064771072', 'writeLatency': '0', 'imageID': 'b0980185-7de1-4e3b-bd76-4495169ab08b', 'flushLatency': '0', 'truesize': '30064771072'}, u'vdb': {'readLatency': '0', 'apparentsize': '826781204480', 'writeLatency': '0', 'imageID': '0b89347c-4654-42bb-a8d7-1a627130b7bd', 'flushLatency': '0', 'truesize': '826781204480'}}, 'boot': 'c', 'statsAge': '0.46', 'cpuIdle': '24.21', 'elapsedTime': '44', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '3601', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:07:41,987::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913510::DEBUG::2013-11-20 03:07:42,441::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:42,446::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:42,452::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'SDUUID=772472cb-2429-466d-96cd-00b3301ef78a', u'VERSION=2', u'LOCKPOLICY=', u'VGUUID=XTb294-NYv6-dVpO-bB1n-4f9P-d0GL-dhd2DI', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'LEASETIMESEC=5', u'LOCKRENEWALINTERVALSEC=5', u'ROLE=Regular', u'PHYBLKSIZE=512', u'DESCRIPTION=Database', u'IOOPTIMEOUTSEC=1', u'CLASS=Data', u'TYPE=FCP', u'PV0=pv:3600c0ff0001520c7414f605001000000,uuid:NujM33-vNiD-sBHY-mrPo-EQ2f-1qsz-HXfn0q,pestart:0,pecount:8929,mapoffset:0', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'MASTER_VERSION=0', u'_SHA_CKSUM=ef8840d9fdef24516a56fd970bed1960d2b1ae92']
>Thread-15913510::DEBUG::2013-11-20 03:07:42,457::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6c2d98ef-fe6f-4855-abc4-f64f0a8c44d2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1297' at 'public_getStorageDomainInfo'
>Thread-15913510::DEBUG::2013-11-20 03:07:42,458::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913510::DEBUG::2013-11-20 03:07:42,459::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913510::DEBUG::2013-11-20 03:07:42,459::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6c2d98ef-fe6f-4855-abc4-f64f0a8c44d2`::Granted request
>Thread-15913510::DEBUG::2013-11-20 03:07:42,460::task::495::TaskManager.Task::(_debug) Task 1882563a-1e45-446c-92a3-20552b325bea: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913510::DEBUG::2013-11-20 03:07:42,461::task::495::TaskManager.Task::(_debug) Task 1882563a-1e45-446c-92a3-20552b325bea: ref 1 aborting False
>Thread-15913510::DEBUG::2013-11-20 03:07:42,461::task::495::TaskManager.Task::(_debug) Task 1882563a-1e45-446c-92a3-20552b325bea: finished: {'info': {'uuid': '772472cb-2429-466d-96cd-00b3301ef78a', 'vguuid': 'XTb294-NYv6-dVpO-bB1n-4f9P-d0GL-dhd2DI', 'lver': -1, 'state': 'OK', 'version': '2', 'role': 'Regular', 'master_ver': 0, 'spm_id': -1, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'Database'}}
>Thread-15913510::DEBUG::2013-11-20 03:07:42,466::task::495::TaskManager.Task::(_debug) Task 1882563a-1e45-446c-92a3-20552b325bea: moving from state preparing -> state finished
>Thread-15913510::DEBUG::2013-11-20 03:07:42,467::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913510::DEBUG::2013-11-20 03:07:42,467::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913510::DEBUG::2013-11-20 03:07:42,467::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913510::DEBUG::2013-11-20 03:07:42,468::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913510::DEBUG::2013-11-20 03:07:42,468::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913510::DEBUG::2013-11-20 03:07:42,469::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913510::DEBUG::2013-11-20 03:07:42,469::task::495::TaskManager.Task::(_debug) Task 1882563a-1e45-446c-92a3-20552b325bea: ref 0 aborting False
>Thread-15913510::INFO::2013-11-20 03:07:42,470::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'uuid': '772472cb-2429-466d-96cd-00b3301ef78a', 'vguuid': 'XTb294-NYv6-dVpO-bB1n-4f9P-d0GL-dhd2DI', 'lver': -1, 'state': 'OK', 'version': '2', 'role': 'Regular', 'master_ver': 0, 'spm_id': -1, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'Database'}}
>Thread-15913510::INFO::2013-11-20 03:07:42,470::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913510::DEBUG::2013-11-20 03:07:42,471::task::495::TaskManager.Task::(_debug) Task cce91dc9-d6ba-446e-9205-c68d79494c1e: moving from state init -> state preparing
>Thread-15913510::DEBUG::2013-11-20 03:07:42,472::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`efc26c6b-dcda-4d2a-a328-8057420c099e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913510::DEBUG::2013-11-20 03:07:42,472::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913510::DEBUG::2013-11-20 03:07:42,473::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913510::DEBUG::2013-11-20 03:07:42,473::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`efc26c6b-dcda-4d2a-a328-8057420c099e`::Granted request
>Thread-15913510::DEBUG::2013-11-20 03:07:42,474::task::495::TaskManager.Task::(_debug) Task cce91dc9-d6ba-446e-9205-c68d79494c1e: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913510::DEBUG::2013-11-20 03:07:42,475::task::495::TaskManager.Task::(_debug) Task cce91dc9-d6ba-446e-9205-c68d79494c1e: ref 1 aborting False
>Thread-15913510::DEBUG::2013-11-20 03:07:42,476::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:42,481::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-15913540::DEBUG::2013-11-20 03:07:42,734::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913540::INFO::2013-11-20 03:07:42,737::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913540::DEBUG::2013-11-20 03:07:42,738::task::495::TaskManager.Task::(_debug) Task 83d5cf8e-9cc1-4491-9266-2d7d62bfc847: moving from state init -> state preparing
>Thread-15913540::DEBUG::2013-11-20 03:07:42,739::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913540::DEBUG::2013-11-20 03:07:42,740::task::495::TaskManager.Task::(_debug) Task 83d5cf8e-9cc1-4491-9266-2d7d62bfc847: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913540::DEBUG::2013-11-20 03:07:42,742::task::495::TaskManager.Task::(_debug) Task 83d5cf8e-9cc1-4491-9266-2d7d62bfc847: moving from state preparing -> state finished
>Thread-15913540::DEBUG::2013-11-20 03:07:42,743::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913540::DEBUG::2013-11-20 03:07:42,743::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913540::DEBUG::2013-11-20 03:07:42,745::task::495::TaskManager.Task::(_debug) Task 83d5cf8e-9cc1-4491-9266-2d7d62bfc847: ref 0 aborting False
>Thread-15913540::INFO::2013-11-20 03:07:42,746::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913541::DEBUG::2013-11-20 03:07:42,752::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913541::INFO::2013-11-20 03:07:42,753::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913541::DEBUG::2013-11-20 03:07:42,754::task::495::TaskManager.Task::(_debug) Task 68b6c3f2-3f98-4eb1-8702-d021d0a3f69b: moving from state init -> state preparing
>Thread-15913541::DEBUG::2013-11-20 03:07:42,755::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`dd75f309-cefc-4cc1-a1b9-69662c021eb8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913541::DEBUG::2013-11-20 03:07:42,756::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913541::DEBUG::2013-11-20 03:07:42,758::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913541::DEBUG::2013-11-20 03:07:42,759::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`dd75f309-cefc-4cc1-a1b9-69662c021eb8`::Granted request
>Thread-15913541::DEBUG::2013-11-20 03:07:42,760::task::495::TaskManager.Task::(_debug) Task 68b6c3f2-3f98-4eb1-8702-d021d0a3f69b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913541::DEBUG::2013-11-20 03:07:42,761::task::495::TaskManager.Task::(_debug) Task 68b6c3f2-3f98-4eb1-8702-d021d0a3f69b: ref 1 aborting False
>Thread-15913541::WARNING::2013-11-20 03:07:42,764::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913541::DEBUG::2013-11-20 03:07:42,766::task::495::TaskManager.Task::(_debug) Task 68b6c3f2-3f98-4eb1-8702-d021d0a3f69b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913541::DEBUG::2013-11-20 03:07:42,767::task::495::TaskManager.Task::(_debug) Task 68b6c3f2-3f98-4eb1-8702-d021d0a3f69b: moving from state preparing -> state finished
>Thread-15913541::DEBUG::2013-11-20 03:07:42,768::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913541::DEBUG::2013-11-20 03:07:42,768::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913541::DEBUG::2013-11-20 03:07:42,769::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913541::DEBUG::2013-11-20 03:07:42,770::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913541::DEBUG::2013-11-20 03:07:42,771::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913541::DEBUG::2013-11-20 03:07:42,772::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913541::DEBUG::2013-11-20 03:07:42,773::task::495::TaskManager.Task::(_debug) Task 68b6c3f2-3f98-4eb1-8702-d021d0a3f69b: ref 0 aborting False
>Thread-15913541::INFO::2013-11-20 03:07:42,774::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:07:42,894::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.350143 s, 2.9 MB/s\n'; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:43,195::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:43,233::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:43,234::task::495::TaskManager.Task::(_debug) Task cce91dc9-d6ba-446e-9205-c68d79494c1e: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913510::DEBUG::2013-11-20 03:07:43,235::task::495::TaskManager.Task::(_debug) Task cce91dc9-d6ba-446e-9205-c68d79494c1e: moving from state preparing -> state finished
>Thread-15913510::DEBUG::2013-11-20 03:07:43,236::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913510::DEBUG::2013-11-20 03:07:43,236::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913510::DEBUG::2013-11-20 03:07:43,237::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913510::DEBUG::2013-11-20 03:07:43,238::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913510::DEBUG::2013-11-20 03:07:43,238::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913510::DEBUG::2013-11-20 03:07:43,239::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913510::DEBUG::2013-11-20 03:07:43,239::task::495::TaskManager.Task::(_debug) Task cce91dc9-d6ba-446e-9205-c68d79494c1e: ref 0 aborting False
>Thread-15913510::INFO::2013-11-20 03:07:43,240::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913510::INFO::2013-11-20 03:07:43,241::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a)
>Thread-15913510::DEBUG::2013-11-20 03:07:43,242::task::495::TaskManager.Task::(_debug) Task d98f2249-6323-4563-8fad-95a0411e138c: moving from state init -> state preparing
>Thread-15913510::INFO::2013-11-20 03:07:43,242::blockSD::674::Storage.StorageDomain::(validate) sdUUID=772472cb-2429-466d-96cd-00b3301ef78a
>Thread-15913510::DEBUG::2013-11-20 03:07:43,243::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:43,244::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:43,245::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:43,246::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:43,247::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:43,248::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-15913510::DEBUG::2013-11-20 03:07:43,964::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:43,973::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:43,974::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'SDUUID=772472cb-2429-466d-96cd-00b3301ef78a', u'VERSION=2', u'LOCKPOLICY=', u'VGUUID=XTb294-NYv6-dVpO-bB1n-4f9P-d0GL-dhd2DI', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'LEASETIMESEC=5', u'LOCKRENEWALINTERVALSEC=5', u'ROLE=Regular', u'PHYBLKSIZE=512', u'DESCRIPTION=Database', u'IOOPTIMEOUTSEC=1', u'CLASS=Data', u'TYPE=FCP', u'PV0=pv:3600c0ff0001520c7414f605001000000,uuid:NujM33-vNiD-sBHY-mrPo-EQ2f-1qsz-HXfn0q,pestart:0,pecount:8929,mapoffset:0', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'MASTER_VERSION=0', u'_SHA_CKSUM=ef8840d9fdef24516a56fd970bed1960d2b1ae92']
>Thread-15913510::DEBUG::2013-11-20 03:07:43,977::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`3375f726-e955-46e9-bf67-ddb11989f83e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1297' at 'public_getStorageDomainInfo'
>Thread-15913510::DEBUG::2013-11-20 03:07:43,978::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913510::DEBUG::2013-11-20 03:07:43,978::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913510::DEBUG::2013-11-20 03:07:43,979::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`3375f726-e955-46e9-bf67-ddb11989f83e`::Granted request
>Thread-15913510::DEBUG::2013-11-20 03:07:43,980::task::495::TaskManager.Task::(_debug) Task d98f2249-6323-4563-8fad-95a0411e138c: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913510::DEBUG::2013-11-20 03:07:43,981::task::495::TaskManager.Task::(_debug) Task d98f2249-6323-4563-8fad-95a0411e138c: ref 1 aborting False
>Thread-15913510::DEBUG::2013-11-20 03:07:43,982::task::495::TaskManager.Task::(_debug) Task d98f2249-6323-4563-8fad-95a0411e138c: finished: {'info': {'uuid': '772472cb-2429-466d-96cd-00b3301ef78a', 'vguuid': 'XTb294-NYv6-dVpO-bB1n-4f9P-d0GL-dhd2DI', 'lver': -1, 'state': 'OK', 'version': '2', 'role': 'Regular', 'master_ver': 0, 'spm_id': -1, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'Database'}}
>Thread-15913510::DEBUG::2013-11-20 03:07:43,987::task::495::TaskManager.Task::(_debug) Task d98f2249-6323-4563-8fad-95a0411e138c: moving from state preparing -> state finished
>Thread-15913510::DEBUG::2013-11-20 03:07:43,987::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913510::DEBUG::2013-11-20 03:07:43,988::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913510::DEBUG::2013-11-20 03:07:43,989::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913510::DEBUG::2013-11-20 03:07:43,990::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913510::DEBUG::2013-11-20 03:07:43,990::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913510::DEBUG::2013-11-20 03:07:43,991::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913510::DEBUG::2013-11-20 03:07:43,991::task::495::TaskManager.Task::(_debug) Task d98f2249-6323-4563-8fad-95a0411e138c: ref 0 aborting False
>Thread-15913510::INFO::2013-11-20 03:07:43,992::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'uuid': '772472cb-2429-466d-96cd-00b3301ef78a', 'vguuid': 'XTb294-NYv6-dVpO-bB1n-4f9P-d0GL-dhd2DI', 'lver': -1, 'state': 'OK', 'version': '2', 'role': 'Regular', 'master_ver': 0, 'spm_id': -1, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'Database'}}
>Thread-15913510::INFO::2013-11-20 03:07:43,997::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913510::DEBUG::2013-11-20 03:07:43,997::task::495::TaskManager.Task::(_debug) Task 8f965bcb-bfa1-46a2-89e5-e66a493cb4cf: moving from state init -> state preparing
>Thread-15913510::DEBUG::2013-11-20 03:07:43,999::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`5f1fec54-a948-4881-a89f-d7fa65d254bd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913510::DEBUG::2013-11-20 03:07:43,999::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913510::DEBUG::2013-11-20 03:07:44,000::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913510::DEBUG::2013-11-20 03:07:44,001::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`5f1fec54-a948-4881-a89f-d7fa65d254bd`::Granted request
>Thread-15913510::DEBUG::2013-11-20 03:07:44,002::task::495::TaskManager.Task::(_debug) Task 8f965bcb-bfa1-46a2-89e5-e66a493cb4cf: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913510::DEBUG::2013-11-20 03:07:44,002::task::495::TaskManager.Task::(_debug) Task 8f965bcb-bfa1-46a2-89e5-e66a493cb4cf: ref 1 aborting False
>Thread-15913510::DEBUG::2013-11-20 03:07:44,003::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:44,009::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-15913510::DEBUG::2013-11-20 03:07:44,653::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:44,675::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:44,676::task::495::TaskManager.Task::(_debug) Task 8f965bcb-bfa1-46a2-89e5-e66a493cb4cf: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913510::DEBUG::2013-11-20 03:07:44,677::task::495::TaskManager.Task::(_debug) Task 8f965bcb-bfa1-46a2-89e5-e66a493cb4cf: moving from state preparing -> state finished
>Thread-15913510::DEBUG::2013-11-20 03:07:44,677::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913510::DEBUG::2013-11-20 03:07:44,678::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913510::DEBUG::2013-11-20 03:07:44,679::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913510::DEBUG::2013-11-20 03:07:44,680::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913510::DEBUG::2013-11-20 03:07:44,680::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913510::DEBUG::2013-11-20 03:07:44,681::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913510::DEBUG::2013-11-20 03:07:44,682::task::495::TaskManager.Task::(_debug) Task 8f965bcb-bfa1-46a2-89e5-e66a493cb4cf: ref 0 aborting False
>Thread-15913510::INFO::2013-11-20 03:07:44,682::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913510::INFO::2013-11-20 03:07:44,683::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a)
>Thread-15913510::DEBUG::2013-11-20 03:07:44,684::task::495::TaskManager.Task::(_debug) Task 759f54f4-d673-4bf8-8f1e-6b766bcd843a: moving from state init -> state preparing
>Thread-15913510::INFO::2013-11-20 03:07:44,685::blockSD::674::Storage.StorageDomain::(validate) sdUUID=772472cb-2429-466d-96cd-00b3301ef78a
>Thread-15913510::DEBUG::2013-11-20 03:07:44,685::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:44,686::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:44,687::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:44,688::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:44,689::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:44,690::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:07:44,911::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913543::DEBUG::2013-11-20 03:07:45,278::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('4b8ed9c8-dfb1-4de4-935d-6ba4009567b9',) {}
>Thread-15913543::DEBUG::2013-11-20 03:07:45,280::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk stats not available
>Thread-15913543::DEBUG::2013-11-20 03:07:45,280::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk stats not available
>Thread-15913543::DEBUG::2013-11-20 03:07:45,281::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk latency not available
>Thread-15913543::DEBUG::2013-11-20 03:07:45,282::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk latency not available
>Thread-15913543::DEBUG::2013-11-20 03:07:45,282::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '46387', 'displayIp': '0', 'displayPort': u'5902', 'session': 'Unknown', 'displaySecurePort': u'5903', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet1': {'macAddr': u'00:1a:4a:a8:64:05', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet1'}, u'vnet2': {'macAddr': u'00:1a:4a:a8:64:0b', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet2'}}, 'vmId': '4b8ed9c8-dfb1-4de4-935d-6ba4009567b9', 'monitorResponse': '0', 'cpuUser': '75.79', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '30064771072', 'writeLatency': '0', 'imageID': 'b0980185-7de1-4e3b-bd76-4495169ab08b', 'flushLatency': '0', 'truesize': '30064771072'}, u'vdb': {'readLatency': '0', 'apparentsize': '826781204480', 'writeLatency': '0', 'imageID': '0b89347c-4654-42bb-a8d7-1a627130b7bd', 'flushLatency': '0', 'truesize': '826781204480'}}, 'boot': 'c', 'statsAge': '1.74', 'cpuIdle': '24.21', 'elapsedTime': '47', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '3601', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15913510::DEBUG::2013-11-20 03:07:45,542::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:45,546::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:45,549::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'SDUUID=772472cb-2429-466d-96cd-00b3301ef78a', u'VERSION=2', u'LOCKPOLICY=', u'VGUUID=XTb294-NYv6-dVpO-bB1n-4f9P-d0GL-dhd2DI', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'LEASETIMESEC=5', u'LOCKRENEWALINTERVALSEC=5', u'ROLE=Regular', u'PHYBLKSIZE=512', u'DESCRIPTION=Database', u'IOOPTIMEOUTSEC=1', u'CLASS=Data', u'TYPE=FCP', u'PV0=pv:3600c0ff0001520c7414f605001000000,uuid:NujM33-vNiD-sBHY-mrPo-EQ2f-1qsz-HXfn0q,pestart:0,pecount:8929,mapoffset:0', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'MASTER_VERSION=0', u'_SHA_CKSUM=ef8840d9fdef24516a56fd970bed1960d2b1ae92']
>Thread-15913510::DEBUG::2013-11-20 03:07:45,558::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c20851bd-5995-41a0-83bc-25f139573c74`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1297' at 'public_getStorageDomainInfo'
>Thread-15913510::DEBUG::2013-11-20 03:07:45,559::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913510::DEBUG::2013-11-20 03:07:45,560::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913510::DEBUG::2013-11-20 03:07:45,561::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c20851bd-5995-41a0-83bc-25f139573c74`::Granted request
>Thread-15913510::DEBUG::2013-11-20 03:07:45,564::task::495::TaskManager.Task::(_debug) Task 759f54f4-d673-4bf8-8f1e-6b766bcd843a: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913510::DEBUG::2013-11-20 03:07:45,565::task::495::TaskManager.Task::(_debug) Task 759f54f4-d673-4bf8-8f1e-6b766bcd843a: ref 1 aborting False
>Thread-15913510::DEBUG::2013-11-20 03:07:45,566::task::495::TaskManager.Task::(_debug) Task 759f54f4-d673-4bf8-8f1e-6b766bcd843a: finished: {'info': {'uuid': '772472cb-2429-466d-96cd-00b3301ef78a', 'vguuid': 'XTb294-NYv6-dVpO-bB1n-4f9P-d0GL-dhd2DI', 'lver': -1, 'state': 'OK', 'version': '2', 'role': 'Regular', 'master_ver': 0, 'spm_id': -1, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'Database'}}
>Thread-15913510::DEBUG::2013-11-20 03:07:45,568::task::495::TaskManager.Task::(_debug) Task 759f54f4-d673-4bf8-8f1e-6b766bcd843a: moving from state preparing -> state finished
>Thread-15913510::DEBUG::2013-11-20 03:07:45,569::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913510::DEBUG::2013-11-20 03:07:45,570::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913510::DEBUG::2013-11-20 03:07:45,571::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913510::DEBUG::2013-11-20 03:07:45,572::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913510::DEBUG::2013-11-20 03:07:45,573::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913510::DEBUG::2013-11-20 03:07:45,574::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913510::DEBUG::2013-11-20 03:07:45,575::task::495::TaskManager.Task::(_debug) Task 759f54f4-d673-4bf8-8f1e-6b766bcd843a: ref 0 aborting False
>Thread-15913510::INFO::2013-11-20 03:07:45,576::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'uuid': '772472cb-2429-466d-96cd-00b3301ef78a', 'vguuid': 'XTb294-NYv6-dVpO-bB1n-4f9P-d0GL-dhd2DI', 'lver': -1, 'state': 'OK', 'version': '2', 'role': 'Regular', 'master_ver': 0, 'spm_id': -1, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'Database'}}
>Thread-15913510::INFO::2013-11-20 03:07:45,578::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913510::DEBUG::2013-11-20 03:07:45,579::task::495::TaskManager.Task::(_debug) Task aff57eed-5bec-44d5-9a86-1e3591bcc895: moving from state init -> state preparing
>Thread-15913510::DEBUG::2013-11-20 03:07:45,581::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`af8c0c41-36cc-4ad8-aad5-c18d6c7baf9d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913510::DEBUG::2013-11-20 03:07:45,582::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913510::DEBUG::2013-11-20 03:07:45,583::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913510::DEBUG::2013-11-20 03:07:45,584::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`af8c0c41-36cc-4ad8-aad5-c18d6c7baf9d`::Granted request
>Thread-15913510::DEBUG::2013-11-20 03:07:45,586::task::495::TaskManager.Task::(_debug) Task aff57eed-5bec-44d5-9a86-1e3591bcc895: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913510::DEBUG::2013-11-20 03:07:45,587::task::495::TaskManager.Task::(_debug) Task aff57eed-5bec-44d5-9a86-1e3591bcc895: ref 1 aborting False
>Thread-15913510::DEBUG::2013-11-20 03:07:45,588::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:45,589::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:07:45,920::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.426244 s, 2.4 MB/s\n'; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:46,280::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:46,303::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:46,304::task::495::TaskManager.Task::(_debug) Task aff57eed-5bec-44d5-9a86-1e3591bcc895: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913510::DEBUG::2013-11-20 03:07:46,305::task::495::TaskManager.Task::(_debug) Task aff57eed-5bec-44d5-9a86-1e3591bcc895: moving from state preparing -> state finished
>Thread-15913510::DEBUG::2013-11-20 03:07:46,305::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913510::DEBUG::2013-11-20 03:07:46,306::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913510::DEBUG::2013-11-20 03:07:46,307::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913510::DEBUG::2013-11-20 03:07:46,308::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913510::DEBUG::2013-11-20 03:07:46,308::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913510::DEBUG::2013-11-20 03:07:46,309::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913510::DEBUG::2013-11-20 03:07:46,309::task::495::TaskManager.Task::(_debug) Task aff57eed-5bec-44d5-9a86-1e3591bcc895: ref 0 aborting False
>Thread-15913510::INFO::2013-11-20 03:07:46,310::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913510::INFO::2013-11-20 03:07:46,311::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a)
>Thread-15913510::DEBUG::2013-11-20 03:07:46,312::task::495::TaskManager.Task::(_debug) Task 38ffd94f-9d66-4d2a-b7ff-030d5e2eb5f7: moving from state init -> state preparing
>Thread-15913510::INFO::2013-11-20 03:07:46,312::blockSD::674::Storage.StorageDomain::(validate) sdUUID=772472cb-2429-466d-96cd-00b3301ef78a
>Thread-15913510::DEBUG::2013-11-20 03:07:46,313::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:46,314::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:46,315::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:46,316::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:46,316::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:46,318::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 03:07:46,911::lvm::439::OperationMutex::(_reloadvgs) Got the operational mutex
>Thread-42840::DEBUG::2013-11-20 03:07:46,914::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-15913510::DEBUG::2013-11-20 03:07:47,129::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:47,136::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'SDUUID=772472cb-2429-466d-96cd-00b3301ef78a', u'VERSION=2', u'LOCKPOLICY=', u'VGUUID=XTb294-NYv6-dVpO-bB1n-4f9P-d0GL-dhd2DI', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'LEASETIMESEC=5', u'LOCKRENEWALINTERVALSEC=5', u'ROLE=Regular', u'PHYBLKSIZE=512', u'DESCRIPTION=Database', u'IOOPTIMEOUTSEC=1', u'CLASS=Data', u'TYPE=FCP', u'PV0=pv:3600c0ff0001520c7414f605001000000,uuid:NujM33-vNiD-sBHY-mrPo-EQ2f-1qsz-HXfn0q,pestart:0,pecount:8929,mapoffset:0', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'MASTER_VERSION=0', u'_SHA_CKSUM=ef8840d9fdef24516a56fd970bed1960d2b1ae92']
>Thread-15913510::DEBUG::2013-11-20 03:07:47,139::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f7377826-33bb-48c1-b52b-0995890ad4e1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1297' at 'public_getStorageDomainInfo'
>Thread-15913510::DEBUG::2013-11-20 03:07:47,140::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913510::DEBUG::2013-11-20 03:07:47,141::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913510::DEBUG::2013-11-20 03:07:47,142::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f7377826-33bb-48c1-b52b-0995890ad4e1`::Granted request
>Thread-15913510::DEBUG::2013-11-20 03:07:47,143::task::495::TaskManager.Task::(_debug) Task 38ffd94f-9d66-4d2a-b7ff-030d5e2eb5f7: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913510::DEBUG::2013-11-20 03:07:47,144::task::495::TaskManager.Task::(_debug) Task 38ffd94f-9d66-4d2a-b7ff-030d5e2eb5f7: ref 1 aborting False
>Thread-15913510::DEBUG::2013-11-20 03:07:47,145::task::495::TaskManager.Task::(_debug) Task 38ffd94f-9d66-4d2a-b7ff-030d5e2eb5f7: finished: {'info': {'uuid': '772472cb-2429-466d-96cd-00b3301ef78a', 'vguuid': 'XTb294-NYv6-dVpO-bB1n-4f9P-d0GL-dhd2DI', 'lver': -1, 'state': 'OK', 'version': '2', 'role': 'Regular', 'master_ver': 0, 'spm_id': -1, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'Database'}}
>Thread-15913510::DEBUG::2013-11-20 03:07:47,146::task::495::TaskManager.Task::(_debug) Task 38ffd94f-9d66-4d2a-b7ff-030d5e2eb5f7: moving from state preparing -> state finished
>Thread-15913510::DEBUG::2013-11-20 03:07:47,147::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913510::DEBUG::2013-11-20 03:07:47,147::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913510::DEBUG::2013-11-20 03:07:47,148::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913510::DEBUG::2013-11-20 03:07:47,149::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913510::DEBUG::2013-11-20 03:07:47,149::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913510::DEBUG::2013-11-20 03:07:47,150::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913510::DEBUG::2013-11-20 03:07:47,151::task::495::TaskManager.Task::(_debug) Task 38ffd94f-9d66-4d2a-b7ff-030d5e2eb5f7: ref 0 aborting False
>Thread-15913510::INFO::2013-11-20 03:07:47,151::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'uuid': '772472cb-2429-466d-96cd-00b3301ef78a', 'vguuid': 'XTb294-NYv6-dVpO-bB1n-4f9P-d0GL-dhd2DI', 'lver': -1, 'state': 'OK', 'version': '2', 'role': 'Regular', 'master_ver': 0, 'spm_id': -1, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'Database'}}
>Thread-15913510::DEBUG::2013-11-20 03:07:47,165::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/libexec/vdsm/hooks/before_vm_start/50_vhostmd' (cwd None)
>Thread-42840::DEBUG::2013-11-20 03:07:47,668::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-42840::DEBUG::2013-11-20 03:07:47,672::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913510::DEBUG::2013-11-20 03:07:47,755::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913510::INFO::2013-11-20 03:07:47,756::hooks::51::root::(_runHooksDir) 
>Thread-15913510::DEBUG::2013-11-20 03:07:47,757::libvirtvm::1077::vm.Vm::(_run) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::<?xml version="1.0" encoding="utf-8"?>
><domain type="kvm">
>	<name>KLEE-AXDB01</name>
>	<uuid>6574d88c-00ac-46f4-99ce-a2726932bfda</uuid>
>	<memory>20971520</memory>
>	<currentMemory>20971520</currentMemory>
>	<vcpu>8</vcpu>
>	<devices>
>		<disk device="disk" type="block">
>			<source dev="/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/772472cb-2429-466d-96cd-00b3301ef78a/images/c4460480-32d3-469f-b0f4-c03bb6d74e41/e6984064-e09e-4c0c-8b51-a8a1711994cf"/>
>			<target bus="virtio" dev="vda"/>
>			<serial>9f-b0f4-c03bb6d74e41</serial>
>			<driver cache="none" error_policy="stop" io="native" name="qemu" type="raw"/>
>		</disk>
>		<disk device="disk" type="block">
>			<source dev="/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/772472cb-2429-466d-96cd-00b3301ef78a/images/df8b790c-c762-4feb-9378-d64a321ce43c/2e903e43-1016-4339-a7f3-c5723a995725"/>
>			<target bus="virtio" dev="vdb"/>
>			<serial>eb-9378-d64a321ce43c</serial>
>			<driver cache="none" error_policy="stop" io="native" name="qemu" type="raw"/>
>		</disk>
>		<disk device="disk" type="block">
>			<source dev="/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/772472cb-2429-466d-96cd-00b3301ef78a/images/13efb45b-f4d2-4c94-bbdb-91ce9b9b7321/088640f7-316c-4e2b-bfdc-0f64cf2913c8"/>
>			<target bus="virtio" dev="vdc"/>
>			<serial>94-bbdb-91ce9b9b7321</serial>
>			<driver cache="none" error_policy="stop" io="native" name="qemu" type="raw"/>
>		</disk>
>		<disk device="disk" type="block">
>			<source dev="/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/772472cb-2429-466d-96cd-00b3301ef78a/images/d368307e-425e-423a-85b7-266c1e4a99ec/fdc91117-476f-4b82-8fc4-d682fa283ecc"/>
>			<target bus="virtio" dev="vdd"/>
>			<serial>3a-85b7-266c1e4a99ec</serial>
>			<driver cache="none" error_policy="stop" io="native" name="qemu" type="raw"/>
>		</disk>
>		<disk device="cdrom" type="file">
>			<target bus="ide" dev="hdc"/>
>		</disk>
>		<controller index="0" ports="16" type="virtio-serial"/>
>		<channel type="unix">
>			<target name="com.redhat.rhevm.vdsm" type="virtio"/>
>			<source mode="bind" path="/var/lib/libvirt/qemu/channels/KLEE-AXDB01.com.redhat.rhevm.vdsm"/>
>		</channel>
>		<memballoon model="none"/>
>		<interface type="bridge">
>			<mac address="00:1a:4a:a8:64:08"/>
>			<model type="virtio"/>
>			<source bridge="rhevm"/>
>		</interface>
>		<input bus="ps2" type="mouse"/>
>		<video>
>			<model heads="1" type="qxl" vram="65536"/>
>		</video>
>		<channel type="spicevmc">
>			<target name="com.redhat.spice.0" type="virtio"/>
>		</channel>
>		<graphics autoport="yes" keymap="en-us" listen="0" passwd="*****" passwdValidTo="1970-01-01T00:00:01" port="-1" tlsPort="-1" type="spice">
>			<channel mode="secure" name="main"/>
>			<channel mode="secure" name="inputs"/>
>		</graphics>
>	</devices>
>	<os>
>		<type arch="x86_64" machine="rhel6.2.0">hvm</type>
>		<boot dev="hd"/>
>		<boot dev="cdrom"/>
>		<smbios mode="sysinfo"/>
>	</os>
>	<sysinfo type="smbios">
>		<system>
>			<entry name="manufacturer">Red Hat</entry>
>			<entry name="product">RHEV Hypervisor</entry>
>			<entry name="version">6.3-20121012.0.el6_3</entry>
>			<entry name="serial">30343536-3138-5A43-4A32-333630373442_AC:16:2D:76:EB:F0</entry>
>			<entry name="uuid">6574d88c-00ac-46f4-99ce-a2726932bfda</entry>
>		</system>
>	</sysinfo>
>	<clock adjustment="-1" offset="variable">
>		<timer name="rtc" tickpolicy="catchup"/>
>	</clock>
>	<features>
>		<acpi/>
>	</features>
>	<cpu match="exact">
>		<model>Westmere</model>
>		<topology cores="4" sockets="2" threads="1"/>
>	</cpu>
></domain>
>
>Dummy-15912651::DEBUG::2013-11-20 03:07:47,937::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913544::INFO::2013-11-20 03:07:48,309::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913544::DEBUG::2013-11-20 03:07:48,315::task::495::TaskManager.Task::(_debug) Task a6faac54-ebf4-4221-bc12-dd8be7d4381a: moving from state init -> state preparing
>Thread-15913544::DEBUG::2013-11-20 03:07:48,318::task::495::TaskManager.Task::(_debug) Task a6faac54-ebf4-4221-bc12-dd8be7d4381a: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000380992889404', 'lastCheck': 1384916858.3912649, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0188131332397', 'lastCheck': 1384916865.935183, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00117206573486', 'lastCheck': 1384916867.5679879, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000330924987793', 'lastCheck': 1384916867.673826, 'valid': True, 'code': 0}}
>Thread-15913544::DEBUG::2013-11-20 03:07:48,321::task::495::TaskManager.Task::(_debug) Task a6faac54-ebf4-4221-bc12-dd8be7d4381a: moving from state preparing -> state finished
>Thread-15913544::DEBUG::2013-11-20 03:07:48,329::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913544::DEBUG::2013-11-20 03:07:48,330::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913544::DEBUG::2013-11-20 03:07:48,331::task::495::TaskManager.Task::(_debug) Task a6faac54-ebf4-4221-bc12-dd8be7d4381a: ref 0 aborting False
>Thread-15913544::INFO::2013-11-20 03:07:48,332::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000380992889404', 'lastCheck': 1384916858.3912649, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0188131332397', 'lastCheck': 1384916865.935183, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00117206573486', 'lastCheck': 1384916867.5679879, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000330924987793', 'lastCheck': 1384916867.673826, 'valid': True, 'code': 0}}
>Thread-15913545::DEBUG::2013-11-20 03:07:48,349::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk stats not available
>Thread-15913545::DEBUG::2013-11-20 03:07:48,350::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk stats not available
>Thread-15913545::DEBUG::2013-11-20 03:07:48,350::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk latency not available
>Thread-15913545::DEBUG::2013-11-20 03:07:48,351::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk latency not available
>Dummy-15912651::DEBUG::2013-11-20 03:07:48,489::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.078454 s, 13.1 MB/s\n'; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-20 03:07:49,390::libvirtvm::1401::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::event Started detail 0 opaque None
>Thread-15913510::DEBUG::2013-11-20 03:07:49,467::utils::307::vm.Vm::(start) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Start statistics collection
>Thread-15913546::DEBUG::2013-11-20 03:07:49,468::utils::336::vm.Vm::(run) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Stats thread started
>Thread-15913546::INFO::2013-11-20 03:07:49,471::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913547::DEBUG::2013-11-20 03:07:49,471::guestIF::47::root::(_connect) Attempting connection to /var/lib/libvirt/qemu/channels/KLEE-AXDB01.com.redhat.rhevm.vdsm
>Thread-15913546::DEBUG::2013-11-20 03:07:49,473::task::495::TaskManager.Task::(_debug) Task f444ca60-691d-4316-aef0-97cc13b088af: moving from state init -> state preparing
>Thread-15913547::DEBUG::2013-11-20 03:07:49,474::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /usr/share/vdsm/prepare-vmchannel /var/lib/libvirt/qemu/channels/KLEE-AXDB01.com.redhat.rhevm.vdsm' (cwd None)
>Thread-15913546::DEBUG::2013-11-20 03:07:49,476::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`819f1538-83b7-44ff-8be5-83ec25c2b021`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913510::WARNING::2013-11-20 03:07:49,477::libvirtvm::1124::vm.Vm::(_readPauseCode) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::_readPauseCode unsupported by libvirt vm
>Thread-15913546::DEBUG::2013-11-20 03:07:49,477::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913510::DEBUG::2013-11-20 03:07:49,774::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /usr/share/vdsm/get-vm-pid KLEE-AXDB01' (cwd None)
>Thread-15913546::DEBUG::2013-11-20 03:07:49,785::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:07:50,069::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`819f1538-83b7-44ff-8be5-83ec25c2b021`::Granted request
>Thread-15913547::DEBUG::2013-11-20 03:07:50,074::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913546::DEBUG::2013-11-20 03:07:50,076::task::495::TaskManager.Task::(_debug) Task f444ca60-691d-4316-aef0-97cc13b088af: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:07:50,077::task::495::TaskManager.Task::(_debug) Task f444ca60-691d-4316-aef0-97cc13b088af: ref 1 aborting False
>Thread-15913547::DEBUG::2013-11-20 03:07:50,078::guestIF::53::root::(_connect) connected to /var/lib/libvirt/qemu/channels/KLEE-AXDB01.com.redhat.rhevm.vdsm
>Thread-15913546::DEBUG::2013-11-20 03:07:50,079::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913546::DEBUG::2013-11-20 03:07:50,082::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-15913510::DEBUG::2013-11-20 03:07:50,445::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:50,459::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /sbin/service ksmtuned retune' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:07:50,836::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::DEBUG::2013-11-20 03:07:51,118::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913510::DEBUG::2013-11-20 03:07:51,120::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913546::DEBUG::2013-11-20 03:07:51,139::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913546::DEBUG::2013-11-20 03:07:51,141::task::495::TaskManager.Task::(_debug) Task f444ca60-691d-4316-aef0-97cc13b088af: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913510::DEBUG::2013-11-20 03:07:51,141::vm::358::vm.Vm::(_startUnderlyingVm) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::_ongoingCreations released
>Thread-15913546::DEBUG::2013-11-20 03:07:51,142::task::495::TaskManager.Task::(_debug) Task f444ca60-691d-4316-aef0-97cc13b088af: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:07:51,145::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:07:51,146::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:07:51,148::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:07:51,151::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:07:51,152::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:07:51,153::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:07:51,155::task::495::TaskManager.Task::(_debug) Task f444ca60-691d-4316-aef0-97cc13b088af: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:07:51,156::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:07:51,157::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:07:51,164::task::495::TaskManager.Task::(_debug) Task c2ca1c41-de9e-4b49-a9cc-f6f6b529040b: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:07:51,166::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`1732c5c8-a7c5-4155-a9d3-ff9292112bd2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:07:51,167::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:07:51,169::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:07:51,170::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`1732c5c8-a7c5-4155-a9d3-ff9292112bd2`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:07:51,172::task::495::TaskManager.Task::(_debug) Task c2ca1c41-de9e-4b49-a9cc-f6f6b529040b: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:07:51,173::task::495::TaskManager.Task::(_debug) Task c2ca1c41-de9e-4b49-a9cc-f6f6b529040b: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:07:51,174::task::495::TaskManager.Task::(_debug) Task c2ca1c41-de9e-4b49-a9cc-f6f6b529040b: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:07:51,175::task::495::TaskManager.Task::(_debug) Task c2ca1c41-de9e-4b49-a9cc-f6f6b529040b: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:07:51,175::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:07:51,176::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:07:51,176::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:07:51,177::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:07:51,178::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:07:51,179::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:07:51,179::task::495::TaskManager.Task::(_debug) Task c2ca1c41-de9e-4b49-a9cc-f6f6b529040b: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:07:51,180::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:07:51,181::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:07:51,181::task::495::TaskManager.Task::(_debug) Task de2f09d7-026f-47ad-86a2-222ced8d3911: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:07:51,182::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c02de5cf-49ff-4210-aded-ec70819fdbab`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:07:51,183::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:07:51,183::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:07:51,184::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c02de5cf-49ff-4210-aded-ec70819fdbab`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:07:51,184::task::495::TaskManager.Task::(_debug) Task de2f09d7-026f-47ad-86a2-222ced8d3911: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:07:51,185::task::495::TaskManager.Task::(_debug) Task de2f09d7-026f-47ad-86a2-222ced8d3911: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:07:51,185::task::495::TaskManager.Task::(_debug) Task de2f09d7-026f-47ad-86a2-222ced8d3911: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:07:51,186::task::495::TaskManager.Task::(_debug) Task de2f09d7-026f-47ad-86a2-222ced8d3911: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:07:51,186::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:07:51,187::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:07:51,187::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:07:51,188::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:07:51,188::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:07:51,189::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:07:51,189::task::495::TaskManager.Task::(_debug) Task de2f09d7-026f-47ad-86a2-222ced8d3911: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:07:51,190::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:07:51,190::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:07:51,191::task::495::TaskManager.Task::(_debug) Task a67a77ff-f969-47db-be08-054dc847e00c: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:07:51,192::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f2384e86-1bcc-4d7f-b4ab-68026dc8dc6b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:07:51,192::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:07:51,193::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:07:51,193::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f2384e86-1bcc-4d7f-b4ab-68026dc8dc6b`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:07:51,194::task::495::TaskManager.Task::(_debug) Task a67a77ff-f969-47db-be08-054dc847e00c: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:07:51,194::task::495::TaskManager.Task::(_debug) Task a67a77ff-f969-47db-be08-054dc847e00c: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:07:51,195::task::495::TaskManager.Task::(_debug) Task a67a77ff-f969-47db-be08-054dc847e00c: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:07:51,195::task::495::TaskManager.Task::(_debug) Task a67a77ff-f969-47db-be08-054dc847e00c: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:07:51,196::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:07:51,196::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:07:51,197::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:07:51,197::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:07:51,198::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:07:51,198::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:07:51,199::task::495::TaskManager.Task::(_debug) Task a67a77ff-f969-47db-be08-054dc847e00c: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:07:51,199::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:07:51,462::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0957757 s, 10.7 MB/s\n'; <rc> = 0
>Thread-15913549::DEBUG::2013-11-20 03:07:51,650::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('6574d88c-00ac-46f4-99ce-a2726932bfda',) {}
>Thread-15913549::DEBUG::2013-11-20 03:07:51,651::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::CPU stats not available
>Thread-15913549::DEBUG::2013-11-20 03:07:51,651::libvirtvm::202::vm.Vm::(_getNetworkStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Network stats not available
>Thread-15913549::DEBUG::2013-11-20 03:07:51,652::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913549::DEBUG::2013-11-20 03:07:51,653::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913549::DEBUG::2013-11-20 03:07:51,653::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913549::DEBUG::2013-11-20 03:07:51,654::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913549::DEBUG::2013-11-20 03:07:51,654::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913549::DEBUG::2013-11-20 03:07:51,655::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913549::DEBUG::2013-11-20 03:07:51,655::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913549::DEBUG::2013-11-20 03:07:51,656::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913549::DEBUG::2013-11-20 03:07:51,657::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '47120', 'displayIp': '0', 'displayPort': u'5904', 'session': 'Unknown', 'displaySecurePort': u'5905', 'timeOffset': '-1', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet3': {'state': 'unknown', 'macAddr': u'00:1a:4a:a8:64:08', 'speed': '1000', 'name': u'vnet3'}}, 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '64424509440', 'writeLatency': '0', 'imageID': 'c4460480-32d3-469f-b0f4-c03bb6d74e41', 'flushLatency': '0', 'truesize': '64424509440'}, u'vdc': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', 'flushLatency': '0', 'truesize': '107374182400'}, u'vdb': {'readLatency': '0', 'apparentsize': '150323855360', 'writeLatency': '0', 'imageID': 'df8b790c-c762-4feb-9378-d64a321ce43c', 'flushLatency': '0', 'truesize': '150323855360'}, u'vdd': {'readLatency': '0', 'apparentsize': '429496729600', 'writeLatency': '0', 'imageID': 'd368307e-425e-423a-85b7-266c1e4a99ec', 'flushLatency': '0', 'truesize': '429496729600'}}, 'boot': 'cd', 'statsAge': '2.18', 'cpuIdle': '100.00', 'elapsedTime': '43', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15913550::DEBUG::2013-11-20 03:07:51,663::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('4b8ed9c8-dfb1-4de4-935d-6ba4009567b9',) {}
>Thread-15913550::DEBUG::2013-11-20 03:07:51,664::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk stats not available
>Thread-15913550::DEBUG::2013-11-20 03:07:51,664::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk stats not available
>Thread-15913550::DEBUG::2013-11-20 03:07:51,665::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk latency not available
>Thread-15913550::DEBUG::2013-11-20 03:07:51,666::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk latency not available
>Thread-15913550::DEBUG::2013-11-20 03:07:51,666::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '46387', 'displayIp': '0', 'displayPort': u'5902', 'session': 'Unknown', 'displaySecurePort': u'5903', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet1': {'macAddr': u'00:1a:4a:a8:64:05', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet1'}, u'vnet2': {'macAddr': u'00:1a:4a:a8:64:0b', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet2'}}, 'vmId': '4b8ed9c8-dfb1-4de4-935d-6ba4009567b9', 'monitorResponse': '0', 'cpuUser': '75.79', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '30064771072', 'writeLatency': '0', 'imageID': 'b0980185-7de1-4e3b-bd76-4495169ab08b', 'flushLatency': '0', 'truesize': '30064771072'}, u'vdb': {'readLatency': '0', 'apparentsize': '826781204480', 'writeLatency': '0', 'imageID': '0b89347c-4654-42bb-a8d7-1a627130b7bd', 'flushLatency': '0', 'truesize': '826781204480'}}, 'boot': 'c', 'statsAge': '1.92', 'cpuIdle': '24.21', 'elapsedTime': '54', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '3601', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15913551::DEBUG::2013-11-20 03:07:53,026::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913551::INFO::2013-11-20 03:07:53,027::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913551::DEBUG::2013-11-20 03:07:53,028::task::495::TaskManager.Task::(_debug) Task 7a7ac2b1-3ca1-4bbe-97d3-409fd7cf0aa7: moving from state init -> state preparing
>Thread-15913551::DEBUG::2013-11-20 03:07:53,028::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913551::DEBUG::2013-11-20 03:07:53,029::task::495::TaskManager.Task::(_debug) Task 7a7ac2b1-3ca1-4bbe-97d3-409fd7cf0aa7: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913551::DEBUG::2013-11-20 03:07:53,030::task::495::TaskManager.Task::(_debug) Task 7a7ac2b1-3ca1-4bbe-97d3-409fd7cf0aa7: moving from state preparing -> state finished
>Thread-15913551::DEBUG::2013-11-20 03:07:53,030::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913551::DEBUG::2013-11-20 03:07:53,031::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913551::DEBUG::2013-11-20 03:07:53,031::task::495::TaskManager.Task::(_debug) Task 7a7ac2b1-3ca1-4bbe-97d3-409fd7cf0aa7: ref 0 aborting False
>Thread-15913551::INFO::2013-11-20 03:07:53,032::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913552::DEBUG::2013-11-20 03:07:53,043::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913552::INFO::2013-11-20 03:07:53,044::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913552::DEBUG::2013-11-20 03:07:53,045::task::495::TaskManager.Task::(_debug) Task 21a0d987-41e7-4f88-93c8-80c60fe40ed3: moving from state init -> state preparing
>Thread-15913552::DEBUG::2013-11-20 03:07:53,046::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0dfdfd46-42a1-497f-98db-484f871c9067`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913552::DEBUG::2013-11-20 03:07:53,046::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913552::DEBUG::2013-11-20 03:07:53,047::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913552::DEBUG::2013-11-20 03:07:53,048::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0dfdfd46-42a1-497f-98db-484f871c9067`::Granted request
>Thread-15913552::DEBUG::2013-11-20 03:07:53,049::task::495::TaskManager.Task::(_debug) Task 21a0d987-41e7-4f88-93c8-80c60fe40ed3: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913552::DEBUG::2013-11-20 03:07:53,049::task::495::TaskManager.Task::(_debug) Task 21a0d987-41e7-4f88-93c8-80c60fe40ed3: ref 1 aborting False
>Thread-15913552::WARNING::2013-11-20 03:07:53,051::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913552::DEBUG::2013-11-20 03:07:53,052::task::495::TaskManager.Task::(_debug) Task 21a0d987-41e7-4f88-93c8-80c60fe40ed3: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913552::DEBUG::2013-11-20 03:07:53,053::task::495::TaskManager.Task::(_debug) Task 21a0d987-41e7-4f88-93c8-80c60fe40ed3: moving from state preparing -> state finished
>Thread-15913552::DEBUG::2013-11-20 03:07:53,054::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913552::DEBUG::2013-11-20 03:07:53,054::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913552::DEBUG::2013-11-20 03:07:53,055::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913552::DEBUG::2013-11-20 03:07:53,055::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913552::DEBUG::2013-11-20 03:07:53,056::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913552::DEBUG::2013-11-20 03:07:53,057::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913552::DEBUG::2013-11-20 03:07:53,057::task::495::TaskManager.Task::(_debug) Task 21a0d987-41e7-4f88-93c8-80c60fe40ed3: ref 0 aborting False
>Thread-15913552::INFO::2013-11-20 03:07:53,058::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:07:53,476::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:07:53,992::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0706125 s, 14.5 MB/s\n'; <rc> = 0
>Thread-15913554::DEBUG::2013-11-20 03:07:54,802::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('6574d88c-00ac-46f4-99ce-a2726932bfda',) {}
>Thread-15913554::DEBUG::2013-11-20 03:07:54,803::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::CPU stats not available
>Thread-15913554::DEBUG::2013-11-20 03:07:54,821::libvirtvm::202::vm.Vm::(_getNetworkStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Network stats not available
>Thread-15913554::DEBUG::2013-11-20 03:07:54,821::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913554::DEBUG::2013-11-20 03:07:54,822::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913554::DEBUG::2013-11-20 03:07:54,822::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913554::DEBUG::2013-11-20 03:07:54,822::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913554::DEBUG::2013-11-20 03:07:54,822::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913554::DEBUG::2013-11-20 03:07:54,823::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913554::DEBUG::2013-11-20 03:07:54,823::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913554::DEBUG::2013-11-20 03:07:54,823::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913554::DEBUG::2013-11-20 03:07:54,824::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '47120', 'displayIp': '0', 'displayPort': u'5904', 'session': 'Unknown', 'displaySecurePort': u'5905', 'timeOffset': '-1', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet3': {'state': 'unknown', 'macAddr': u'00:1a:4a:a8:64:08', 'speed': '1000', 'name': u'vnet3'}}, 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '64424509440', 'writeLatency': '0', 'imageID': 'c4460480-32d3-469f-b0f4-c03bb6d74e41', 'flushLatency': '0', 'truesize': '64424509440'}, u'vdc': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', 'flushLatency': '0', 'truesize': '107374182400'}, u'vdb': {'readLatency': '0', 'apparentsize': '150323855360', 'writeLatency': '0', 'imageID': 'df8b790c-c762-4feb-9378-d64a321ce43c', 'flushLatency': '0', 'truesize': '150323855360'}, u'vdd': {'readLatency': '0', 'apparentsize': '429496729600', 'writeLatency': '0', 'imageID': 'd368307e-425e-423a-85b7-266c1e4a99ec', 'flushLatency': '0', 'truesize': '429496729600'}}, 'boot': 'cd', 'statsAge': '1.53', 'cpuIdle': '100.00', 'elapsedTime': '46', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15913555::DEBUG::2013-11-20 03:07:54,837::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('4b8ed9c8-dfb1-4de4-935d-6ba4009567b9',) {}
>Thread-15913555::DEBUG::2013-11-20 03:07:54,838::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk stats not available
>Thread-15913555::DEBUG::2013-11-20 03:07:54,838::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk stats not available
>Thread-15913555::DEBUG::2013-11-20 03:07:54,838::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk latency not available
>Thread-15913555::DEBUG::2013-11-20 03:07:54,839::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk latency not available
>Thread-15913555::DEBUG::2013-11-20 03:07:54,839::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '46387', 'displayIp': '0', 'displayPort': u'5902', 'session': 'Unknown', 'displaySecurePort': u'5903', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet1': {'macAddr': u'00:1a:4a:a8:64:05', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet1'}, u'vnet2': {'macAddr': u'00:1a:4a:a8:64:0b', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet2'}}, 'vmId': '4b8ed9c8-dfb1-4de4-935d-6ba4009567b9', 'monitorResponse': '0', 'cpuUser': '74.52', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '30064771072', 'writeLatency': '0', 'imageID': 'b0980185-7de1-4e3b-bd76-4495169ab08b', 'flushLatency': '0', 'truesize': '30064771072'}, u'vdb': {'readLatency': '0', 'apparentsize': '826781204480', 'writeLatency': '0', 'imageID': '0b89347c-4654-42bb-a8d7-1a627130b7bd', 'flushLatency': '0', 'truesize': '826781204480'}}, 'boot': 'c', 'statsAge': '0.73', 'cpuIdle': '25.48', 'elapsedTime': '57', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '3601', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:07:56,007::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:07:56,341::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0365219 s, 28.0 MB/s\n'; <rc> = 0
>Thread-15913557::DEBUG::2013-11-20 03:07:57,862::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('6574d88c-00ac-46f4-99ce-a2726932bfda',) {}
>Thread-15913557::DEBUG::2013-11-20 03:07:57,862::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::CPU stats not available
>Thread-15913557::DEBUG::2013-11-20 03:07:57,863::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913557::DEBUG::2013-11-20 03:07:57,863::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913557::DEBUG::2013-11-20 03:07:57,864::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913557::DEBUG::2013-11-20 03:07:57,864::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913557::DEBUG::2013-11-20 03:07:57,864::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913557::DEBUG::2013-11-20 03:07:57,864::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913557::DEBUG::2013-11-20 03:07:57,865::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913557::DEBUG::2013-11-20 03:07:57,865::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913557::DEBUG::2013-11-20 03:07:57,866::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '47120', 'displayIp': '0', 'displayPort': u'5904', 'session': 'Unknown', 'displaySecurePort': u'5905', 'timeOffset': '-1', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet3': {'macAddr': u'00:1a:4a:a8:64:08', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet3'}}, 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '64424509440', 'writeLatency': '0', 'imageID': 'c4460480-32d3-469f-b0f4-c03bb6d74e41', 'flushLatency': '0', 'truesize': '64424509440'}, u'vdc': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', 'flushLatency': '0', 'truesize': '107374182400'}, u'vdb': {'readLatency': '0', 'apparentsize': '150323855360', 'writeLatency': '0', 'imageID': 'df8b790c-c762-4feb-9378-d64a321ce43c', 'flushLatency': '0', 'truesize': '150323855360'}, u'vdd': {'readLatency': '0', 'apparentsize': '429496729600', 'writeLatency': '0', 'imageID': 'd368307e-425e-423a-85b7-266c1e4a99ec', 'flushLatency': '0', 'truesize': '429496729600'}}, 'boot': 'cd', 'statsAge': '0.59', 'cpuIdle': '100.00', 'elapsedTime': '49', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15913558::DEBUG::2013-11-20 03:07:57,870::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('4b8ed9c8-dfb1-4de4-935d-6ba4009567b9',) {}
>Thread-15913558::DEBUG::2013-11-20 03:07:57,871::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk stats not available
>Thread-15913558::DEBUG::2013-11-20 03:07:57,871::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk stats not available
>Thread-15913558::DEBUG::2013-11-20 03:07:57,872::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk latency not available
>Thread-15913558::DEBUG::2013-11-20 03:07:57,872::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk latency not available
>Thread-15913558::DEBUG::2013-11-20 03:07:57,872::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '46387', 'displayIp': '0', 'displayPort': u'5902', 'session': 'Unknown', 'displaySecurePort': u'5903', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet1': {'macAddr': u'00:1a:4a:a8:64:05', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet1'}, u'vnet2': {'macAddr': u'00:1a:4a:a8:64:0b', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet2'}}, 'vmId': '4b8ed9c8-dfb1-4de4-935d-6ba4009567b9', 'monitorResponse': '0', 'cpuUser': '74.52', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '30064771072', 'writeLatency': '0', 'imageID': 'b0980185-7de1-4e3b-bd76-4495169ab08b', 'flushLatency': '0', 'truesize': '30064771072'}, u'vdb': {'readLatency': '0', 'apparentsize': '826781204480', 'writeLatency': '0', 'imageID': '0b89347c-4654-42bb-a8d7-1a627130b7bd', 'flushLatency': '0', 'truesize': '826781204480'}}, 'boot': 'c', 'statsAge': '0.70', 'cpuIdle': '25.48', 'elapsedTime': '60', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '3601', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:07:58,350::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:07:58,736::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0465669 s, 22.0 MB/s\n'; <rc> = 0
>Thread-15913519::DEBUG::2013-11-20 03:07:59,666::guestIF::97::vm.Vm::(_handleMessage) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::u'None'
>Dummy-15912651::DEBUG::2013-11-20 03:08:00,750::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913560::DEBUG::2013-11-20 03:08:01,114::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('6574d88c-00ac-46f4-99ce-a2726932bfda',) {}
>Thread-15913560::DEBUG::2013-11-20 03:08:01,115::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::CPU stats not available
>Thread-15913560::DEBUG::2013-11-20 03:08:01,116::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913560::DEBUG::2013-11-20 03:08:01,116::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913560::DEBUG::2013-11-20 03:08:01,117::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913560::DEBUG::2013-11-20 03:08:01,117::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913560::DEBUG::2013-11-20 03:08:01,117::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913560::DEBUG::2013-11-20 03:08:01,118::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913560::DEBUG::2013-11-20 03:08:01,118::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913560::DEBUG::2013-11-20 03:08:01,118::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913560::DEBUG::2013-11-20 03:08:01,119::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '47120', 'displayIp': '0', 'displayPort': u'5904', 'session': 'Unknown', 'displaySecurePort': u'5905', 'timeOffset': '-1', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet3': {'macAddr': u'00:1a:4a:a8:64:08', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet3'}}, 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '64424509440', 'writeLatency': '0', 'imageID': 'c4460480-32d3-469f-b0f4-c03bb6d74e41', 'flushLatency': '0', 'truesize': '64424509440'}, u'vdc': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', 'flushLatency': '0', 'truesize': '107374182400'}, u'vdb': {'readLatency': '0', 'apparentsize': '150323855360', 'writeLatency': '0', 'imageID': 'df8b790c-c762-4feb-9378-d64a321ce43c', 'flushLatency': '0', 'truesize': '150323855360'}, u'vdd': {'readLatency': '0', 'apparentsize': '429496729600', 'writeLatency': '0', 'imageID': 'd368307e-425e-423a-85b7-266c1e4a99ec', 'flushLatency': '0', 'truesize': '429496729600'}}, 'boot': 'cd', 'statsAge': '1.84', 'cpuIdle': '100.00', 'elapsedTime': '53', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:08:01,329::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0896364 s, 11.4 MB/s\n'; <rc> = 0
>Thread-15913561::DEBUG::2013-11-20 03:08:03,287::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913561::INFO::2013-11-20 03:08:03,288::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913561::DEBUG::2013-11-20 03:08:03,288::task::495::TaskManager.Task::(_debug) Task 98e46aec-58ac-425d-8c64-1a7d11292d5d: moving from state init -> state preparing
>Thread-15913561::DEBUG::2013-11-20 03:08:03,289::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913561::DEBUG::2013-11-20 03:08:03,289::task::495::TaskManager.Task::(_debug) Task 98e46aec-58ac-425d-8c64-1a7d11292d5d: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913561::DEBUG::2013-11-20 03:08:03,289::task::495::TaskManager.Task::(_debug) Task 98e46aec-58ac-425d-8c64-1a7d11292d5d: moving from state preparing -> state finished
>Thread-15913561::DEBUG::2013-11-20 03:08:03,290::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913561::DEBUG::2013-11-20 03:08:03,290::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913561::DEBUG::2013-11-20 03:08:03,290::task::495::TaskManager.Task::(_debug) Task 98e46aec-58ac-425d-8c64-1a7d11292d5d: ref 0 aborting False
>Thread-15913561::INFO::2013-11-20 03:08:03,291::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913562::DEBUG::2013-11-20 03:08:03,295::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913562::INFO::2013-11-20 03:08:03,295::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913562::DEBUG::2013-11-20 03:08:03,296::task::495::TaskManager.Task::(_debug) Task 590c291b-e647-4407-822b-fb1b2234547f: moving from state init -> state preparing
>Thread-15913562::DEBUG::2013-11-20 03:08:03,296::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`dfce8ae3-d824-4608-846d-e2a0060d0c67`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913562::DEBUG::2013-11-20 03:08:03,297::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913562::DEBUG::2013-11-20 03:08:03,297::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913562::DEBUG::2013-11-20 03:08:03,298::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`dfce8ae3-d824-4608-846d-e2a0060d0c67`::Granted request
>Thread-15913562::DEBUG::2013-11-20 03:08:03,298::task::495::TaskManager.Task::(_debug) Task 590c291b-e647-4407-822b-fb1b2234547f: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913562::DEBUG::2013-11-20 03:08:03,299::task::495::TaskManager.Task::(_debug) Task 590c291b-e647-4407-822b-fb1b2234547f: ref 1 aborting False
>Thread-15913562::WARNING::2013-11-20 03:08:03,301::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913562::DEBUG::2013-11-20 03:08:03,310::task::495::TaskManager.Task::(_debug) Task 590c291b-e647-4407-822b-fb1b2234547f: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913562::DEBUG::2013-11-20 03:08:03,312::task::495::TaskManager.Task::(_debug) Task 590c291b-e647-4407-822b-fb1b2234547f: moving from state preparing -> state finished
>Thread-15913562::DEBUG::2013-11-20 03:08:03,313::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913562::DEBUG::2013-11-20 03:08:03,314::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913562::DEBUG::2013-11-20 03:08:03,314::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913562::DEBUG::2013-11-20 03:08:03,315::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913562::DEBUG::2013-11-20 03:08:03,315::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913562::DEBUG::2013-11-20 03:08:03,315::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913562::DEBUG::2013-11-20 03:08:03,316::task::495::TaskManager.Task::(_debug) Task 590c291b-e647-4407-822b-fb1b2234547f: ref 0 aborting False
>Thread-15913562::INFO::2013-11-20 03:08:03,316::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:08:03,347::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913563::INFO::2013-11-20 03:08:04,141::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913563::DEBUG::2013-11-20 03:08:04,142::task::495::TaskManager.Task::(_debug) Task e961d9cf-e7bc-40e8-8fbd-4e42b5f664d5: moving from state init -> state preparing
>Thread-15913563::DEBUG::2013-11-20 03:08:04,143::task::495::TaskManager.Task::(_debug) Task e961d9cf-e7bc-40e8-8fbd-4e42b5f664d5: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000277996063232', 'lastCheck': 1384916879.1610751, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00135111808777', 'lastCheck': 1384916876.4529259, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000972986221313', 'lastCheck': 1384916877.8451419, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00042200088501', 'lastCheck': 1384916877.833935, 'valid': True, 'code': 0}}
>Thread-15913563::DEBUG::2013-11-20 03:08:04,143::task::495::TaskManager.Task::(_debug) Task e961d9cf-e7bc-40e8-8fbd-4e42b5f664d5: moving from state preparing -> state finished
>Thread-15913563::DEBUG::2013-11-20 03:08:04,144::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913563::DEBUG::2013-11-20 03:08:04,144::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913563::DEBUG::2013-11-20 03:08:04,144::task::495::TaskManager.Task::(_debug) Task e961d9cf-e7bc-40e8-8fbd-4e42b5f664d5: ref 0 aborting False
>Thread-15913563::INFO::2013-11-20 03:08:04,145::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000277996063232', 'lastCheck': 1384916879.1610751, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00135111808777', 'lastCheck': 1384916876.4529259, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000972986221313', 'lastCheck': 1384916877.8451419, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00042200088501', 'lastCheck': 1384916877.833935, 'valid': True, 'code': 0}}
>Thread-15913564::DEBUG::2013-11-20 03:08:04,193::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::CPU stats not available
>Thread-15913564::DEBUG::2013-11-20 03:08:04,194::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913564::DEBUG::2013-11-20 03:08:04,194::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913564::DEBUG::2013-11-20 03:08:04,194::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913564::DEBUG::2013-11-20 03:08:04,195::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913564::DEBUG::2013-11-20 03:08:04,195::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913564::DEBUG::2013-11-20 03:08:04,195::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913564::DEBUG::2013-11-20 03:08:04,196::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913564::DEBUG::2013-11-20 03:08:04,196::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913564::DEBUG::2013-11-20 03:08:04,197::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk stats not available
>Thread-15913564::DEBUG::2013-11-20 03:08:04,198::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk stats not available
>Thread-15913564::DEBUG::2013-11-20 03:08:04,198::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk latency not available
>Thread-15913564::DEBUG::2013-11-20 03:08:04,199::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk latency not available
>Dummy-15912651::DEBUG::2013-11-20 03:08:04,349::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0787409 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:08:06,363::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:08:07,312::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0512357 s, 20.0 MB/s\n'; <rc> = 0
>Thread-15913566::DEBUG::2013-11-20 03:08:07,484::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('6574d88c-00ac-46f4-99ce-a2726932bfda',) {}
>Thread-15913566::DEBUG::2013-11-20 03:08:07,485::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913566::DEBUG::2013-11-20 03:08:07,485::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913566::DEBUG::2013-11-20 03:08:07,486::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913566::DEBUG::2013-11-20 03:08:07,486::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913566::DEBUG::2013-11-20 03:08:07,486::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913566::DEBUG::2013-11-20 03:08:07,487::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913566::DEBUG::2013-11-20 03:08:07,487::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913566::DEBUG::2013-11-20 03:08:07,487::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913566::DEBUG::2013-11-20 03:08:07,488::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '47120', 'displayIp': '0', 'displayPort': u'5904', 'session': 'Unknown', 'displaySecurePort': u'5905', 'timeOffset': '-1', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet3': {'macAddr': u'00:1a:4a:a8:64:08', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet3'}}, 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'monitorResponse': '0', 'cpuUser': '205.51', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '64424509440', 'writeLatency': '0', 'imageID': 'c4460480-32d3-469f-b0f4-c03bb6d74e41', 'flushLatency': '0', 'truesize': '64424509440'}, u'vdc': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', 'flushLatency': '0', 'truesize': '107374182400'}, u'vdb': {'readLatency': '0', 'apparentsize': '150323855360', 'writeLatency': '0', 'imageID': 'df8b790c-c762-4feb-9378-d64a321ce43c', 'flushLatency': '0', 'truesize': '150323855360'}, u'vdd': {'readLatency': '0', 'apparentsize': '429496729600', 'writeLatency': '0', 'imageID': 'd368307e-425e-423a-85b7-266c1e4a99ec', 'flushLatency': '0', 'truesize': '429496729600'}}, 'boot': 'cd', 'statsAge': '0.19', 'cpuIdle': '0.00', 'elapsedTime': '59', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:08:09,321::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:08:10,126::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0773255 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15913568::DEBUG::2013-11-20 03:08:10,517::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('6574d88c-00ac-46f4-99ce-a2726932bfda',) {}
>Thread-15913568::DEBUG::2013-11-20 03:08:10,517::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913568::DEBUG::2013-11-20 03:08:10,518::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913568::DEBUG::2013-11-20 03:08:10,518::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913568::DEBUG::2013-11-20 03:08:10,518::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913568::DEBUG::2013-11-20 03:08:10,519::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913568::DEBUG::2013-11-20 03:08:10,519::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913568::DEBUG::2013-11-20 03:08:10,519::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913568::DEBUG::2013-11-20 03:08:10,520::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913568::DEBUG::2013-11-20 03:08:10,520::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '47120', 'displayIp': '0', 'displayPort': u'5904', 'session': 'Unknown', 'displaySecurePort': u'5905', 'timeOffset': '-1', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet3': {'macAddr': u'00:1a:4a:a8:64:08', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet3'}}, 'vmId': '6574d88c-00ac-46f4-99ce-a2726932bfda', 'monitorResponse': '0', 'cpuUser': '205.51', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '64424509440', 'writeLatency': '0', 'imageID': 'c4460480-32d3-469f-b0f4-c03bb6d74e41', 'flushLatency': '0', 'truesize': '64424509440'}, u'vdc': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '13efb45b-f4d2-4c94-bbdb-91ce9b9b7321', 'flushLatency': '0', 'truesize': '107374182400'}, u'vdb': {'readLatency': '0', 'apparentsize': '150323855360', 'writeLatency': '0', 'imageID': 'df8b790c-c762-4feb-9378-d64a321ce43c', 'flushLatency': '0', 'truesize': '150323855360'}, u'vdd': {'readLatency': '0', 'apparentsize': '429496729600', 'writeLatency': '0', 'imageID': 'd368307e-425e-423a-85b7-266c1e4a99ec', 'flushLatency': '0', 'truesize': '429496729600'}}, 'boot': 'cd', 'statsAge': '1.22', 'cpuIdle': '0.00', 'elapsedTime': '62', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:08:12,140::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:08:12,685::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0680285 s, 15.1 MB/s\n'; <rc> = 0
>Thread-15913569::DEBUG::2013-11-20 03:08:13,560::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913569::INFO::2013-11-20 03:08:13,561::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913569::DEBUG::2013-11-20 03:08:13,561::task::495::TaskManager.Task::(_debug) Task c3a25da3-2b87-443c-be84-597a27e73f8d: moving from state init -> state preparing
>Thread-15913569::DEBUG::2013-11-20 03:08:13,562::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913569::DEBUG::2013-11-20 03:08:13,562::task::495::TaskManager.Task::(_debug) Task c3a25da3-2b87-443c-be84-597a27e73f8d: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913569::DEBUG::2013-11-20 03:08:13,563::task::495::TaskManager.Task::(_debug) Task c3a25da3-2b87-443c-be84-597a27e73f8d: moving from state preparing -> state finished
>Thread-15913569::DEBUG::2013-11-20 03:08:13,563::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913569::DEBUG::2013-11-20 03:08:13,563::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913569::DEBUG::2013-11-20 03:08:13,564::task::495::TaskManager.Task::(_debug) Task c3a25da3-2b87-443c-be84-597a27e73f8d: ref 0 aborting False
>Thread-15913569::INFO::2013-11-20 03:08:13,564::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913570::DEBUG::2013-11-20 03:08:13,568::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913570::INFO::2013-11-20 03:08:13,569::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913570::DEBUG::2013-11-20 03:08:13,569::task::495::TaskManager.Task::(_debug) Task 0cffa33d-0754-449f-85b7-8293e102767e: moving from state init -> state preparing
>Thread-15913570::DEBUG::2013-11-20 03:08:13,570::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c455b58a-a9c1-4e40-a53d-e427416135fe`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913570::DEBUG::2013-11-20 03:08:13,570::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913570::DEBUG::2013-11-20 03:08:13,571::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913570::DEBUG::2013-11-20 03:08:13,571::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c455b58a-a9c1-4e40-a53d-e427416135fe`::Granted request
>Thread-15913570::DEBUG::2013-11-20 03:08:13,572::task::495::TaskManager.Task::(_debug) Task 0cffa33d-0754-449f-85b7-8293e102767e: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913570::DEBUG::2013-11-20 03:08:13,572::task::495::TaskManager.Task::(_debug) Task 0cffa33d-0754-449f-85b7-8293e102767e: ref 1 aborting False
>Thread-15913570::WARNING::2013-11-20 03:08:13,574::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913570::DEBUG::2013-11-20 03:08:13,575::task::495::TaskManager.Task::(_debug) Task 0cffa33d-0754-449f-85b7-8293e102767e: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913570::DEBUG::2013-11-20 03:08:13,576::task::495::TaskManager.Task::(_debug) Task 0cffa33d-0754-449f-85b7-8293e102767e: moving from state preparing -> state finished
>Thread-15913570::DEBUG::2013-11-20 03:08:13,576::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913570::DEBUG::2013-11-20 03:08:13,577::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913570::DEBUG::2013-11-20 03:08:13,577::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913570::DEBUG::2013-11-20 03:08:13,578::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913570::DEBUG::2013-11-20 03:08:13,578::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913570::DEBUG::2013-11-20 03:08:13,579::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913570::DEBUG::2013-11-20 03:08:13,579::task::495::TaskManager.Task::(_debug) Task 0cffa33d-0754-449f-85b7-8293e102767e: ref 0 aborting False
>Thread-15913570::INFO::2013-11-20 03:08:13,580::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:08:14,696::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:08:15,448::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0537866 s, 19.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:08:17,458::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:08:19,029::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0496902 s, 20.6 MB/s\n'; <rc> = 0
>Thread-15913573::INFO::2013-11-20 03:08:19,917::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913573::DEBUG::2013-11-20 03:08:19,918::task::495::TaskManager.Task::(_debug) Task b80b7b55-7679-49e4-8c9c-8b54d6fb8742: moving from state init -> state preparing
>Thread-15913573::DEBUG::2013-11-20 03:08:19,919::task::495::TaskManager.Task::(_debug) Task b80b7b55-7679-49e4-8c9c-8b54d6fb8742: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000360012054443', 'lastCheck': 1384916899.3836329, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00088906288147', 'lastCheck': 1384916896.6111939, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0183851718903', 'lastCheck': 1384916897.970094, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00042200088501', 'lastCheck': 1384916897.9469571, 'valid': True, 'code': 0}}
>Thread-15913573::DEBUG::2013-11-20 03:08:19,919::task::495::TaskManager.Task::(_debug) Task b80b7b55-7679-49e4-8c9c-8b54d6fb8742: moving from state preparing -> state finished
>Thread-15913573::DEBUG::2013-11-20 03:08:19,920::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913573::DEBUG::2013-11-20 03:08:19,920::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913573::DEBUG::2013-11-20 03:08:19,920::task::495::TaskManager.Task::(_debug) Task b80b7b55-7679-49e4-8c9c-8b54d6fb8742: ref 0 aborting False
>Thread-15913573::INFO::2013-11-20 03:08:19,921::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000360012054443', 'lastCheck': 1384916899.3836329, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00088906288147', 'lastCheck': 1384916896.6111939, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0183851718903', 'lastCheck': 1384916897.970094, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00042200088501', 'lastCheck': 1384916897.9469571, 'valid': True, 'code': 0}}
>Thread-15913574::DEBUG::2013-11-20 03:08:19,933::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913574::DEBUG::2013-11-20 03:08:19,933::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913574::DEBUG::2013-11-20 03:08:19,934::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913574::DEBUG::2013-11-20 03:08:19,934::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913574::DEBUG::2013-11-20 03:08:19,934::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913574::DEBUG::2013-11-20 03:08:19,935::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913574::DEBUG::2013-11-20 03:08:19,935::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913574::DEBUG::2013-11-20 03:08:19,935::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913574::DEBUG::2013-11-20 03:08:19,936::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk stats not available
>Thread-15913574::DEBUG::2013-11-20 03:08:19,936::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk stats not available
>Thread-15913574::DEBUG::2013-11-20 03:08:19,937::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk latency not available
>Thread-15913574::DEBUG::2013-11-20 03:08:19,937::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::Disk latency not available
>Dummy-15912651::DEBUG::2013-11-20 03:08:21,044::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913575::DEBUG::2013-11-20 03:08:21,232::clientIF::54::vds::(wrapper) [192.168.100.135]::call setVmTicket with ('4b8ed9c8-dfb1-4de4-935d-6ba4009567b9', 'FNUITVoWrGEk', '120') {}
>Thread-15913575::DEBUG::2013-11-20 03:08:21,254::clientIF::59::vds::(wrapper) return setVmTicket with {'status': {'message': 'Done', 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:08:21,720::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0788343 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 03:08:22,460::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:08:22,461::task::495::TaskManager.Task::(_debug) Task 831b2274-9ef3-4ec4-8060-f5380ea4c3a5: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:08:22,462::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e8b9c4b4-04b6-4a68-b2ef-ac414198ebaa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:08:22,462::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:08:22,463::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:08:22,463::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e8b9c4b4-04b6-4a68-b2ef-ac414198ebaa`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:08:22,464::task::495::TaskManager.Task::(_debug) Task 831b2274-9ef3-4ec4-8060-f5380ea4c3a5: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:08:22,464::task::495::TaskManager.Task::(_debug) Task 831b2274-9ef3-4ec4-8060-f5380ea4c3a5: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:08:22,464::task::495::TaskManager.Task::(_debug) Task 831b2274-9ef3-4ec4-8060-f5380ea4c3a5: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:08:22,465::task::495::TaskManager.Task::(_debug) Task 831b2274-9ef3-4ec4-8060-f5380ea4c3a5: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:08:22,465::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:08:22,466::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:08:22,466::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:08:22,466::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:08:22,467::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:08:22,467::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:08:22,467::task::495::TaskManager.Task::(_debug) Task 831b2274-9ef3-4ec4-8060-f5380ea4c3a5: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:08:22,468::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:08:22,468::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:08:22,468::task::495::TaskManager.Task::(_debug) Task 0d59b8ac-ef9b-4b2e-9d2f-3b8f8705aa60: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:08:22,469::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`21ccf41b-aa1e-442b-92ea-c55edeb477a7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:08:22,469::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:08:22,470::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:08:22,470::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`21ccf41b-aa1e-442b-92ea-c55edeb477a7`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:08:22,471::task::495::TaskManager.Task::(_debug) Task 0d59b8ac-ef9b-4b2e-9d2f-3b8f8705aa60: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:08:22,471::task::495::TaskManager.Task::(_debug) Task 0d59b8ac-ef9b-4b2e-9d2f-3b8f8705aa60: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:08:22,471::task::495::TaskManager.Task::(_debug) Task 0d59b8ac-ef9b-4b2e-9d2f-3b8f8705aa60: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:08:22,472::task::495::TaskManager.Task::(_debug) Task 0d59b8ac-ef9b-4b2e-9d2f-3b8f8705aa60: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:08:22,472::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:08:22,473::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:08:22,473::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:08:22,473::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:08:22,474::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:08:22,474::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:08:22,474::task::495::TaskManager.Task::(_debug) Task 0d59b8ac-ef9b-4b2e-9d2f-3b8f8705aa60: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:08:22,475::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:08:23,730::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913577::DEBUG::2013-11-20 03:08:24,194::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913577::INFO::2013-11-20 03:08:24,196::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913577::DEBUG::2013-11-20 03:08:24,197::task::495::TaskManager.Task::(_debug) Task 89ffead0-6eee-4e2a-88bd-afd6d0400536: moving from state init -> state preparing
>Thread-15913577::DEBUG::2013-11-20 03:08:24,198::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913577::DEBUG::2013-11-20 03:08:24,198::task::495::TaskManager.Task::(_debug) Task 89ffead0-6eee-4e2a-88bd-afd6d0400536: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913577::DEBUG::2013-11-20 03:08:24,198::task::495::TaskManager.Task::(_debug) Task 89ffead0-6eee-4e2a-88bd-afd6d0400536: moving from state preparing -> state finished
>Thread-15913577::DEBUG::2013-11-20 03:08:24,199::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913577::DEBUG::2013-11-20 03:08:24,199::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913577::DEBUG::2013-11-20 03:08:24,200::task::495::TaskManager.Task::(_debug) Task 89ffead0-6eee-4e2a-88bd-afd6d0400536: ref 0 aborting False
>Thread-15913577::INFO::2013-11-20 03:08:24,200::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913578::DEBUG::2013-11-20 03:08:24,205::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913578::INFO::2013-11-20 03:08:24,206::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913578::DEBUG::2013-11-20 03:08:24,207::task::495::TaskManager.Task::(_debug) Task 0652dd3a-4a38-4b10-b2bd-45e4410c588e: moving from state init -> state preparing
>Thread-15913578::DEBUG::2013-11-20 03:08:24,208::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`fc41f4c8-78dc-45de-9b68-b648fc31fc9f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913578::DEBUG::2013-11-20 03:08:24,209::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913578::DEBUG::2013-11-20 03:08:24,210::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913578::DEBUG::2013-11-20 03:08:24,210::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`fc41f4c8-78dc-45de-9b68-b648fc31fc9f`::Granted request
>Thread-15913578::DEBUG::2013-11-20 03:08:24,212::task::495::TaskManager.Task::(_debug) Task 0652dd3a-4a38-4b10-b2bd-45e4410c588e: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913578::DEBUG::2013-11-20 03:08:24,212::task::495::TaskManager.Task::(_debug) Task 0652dd3a-4a38-4b10-b2bd-45e4410c588e: ref 1 aborting False
>Thread-15913578::WARNING::2013-11-20 03:08:24,214::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913578::DEBUG::2013-11-20 03:08:24,220::task::495::TaskManager.Task::(_debug) Task 0652dd3a-4a38-4b10-b2bd-45e4410c588e: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913578::DEBUG::2013-11-20 03:08:24,221::task::495::TaskManager.Task::(_debug) Task 0652dd3a-4a38-4b10-b2bd-45e4410c588e: moving from state preparing -> state finished
>Thread-15913578::DEBUG::2013-11-20 03:08:24,221::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913578::DEBUG::2013-11-20 03:08:24,222::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913578::DEBUG::2013-11-20 03:08:24,223::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913578::DEBUG::2013-11-20 03:08:24,223::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913578::DEBUG::2013-11-20 03:08:24,224::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913578::DEBUG::2013-11-20 03:08:24,224::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913578::DEBUG::2013-11-20 03:08:24,224::task::495::TaskManager.Task::(_debug) Task 0652dd3a-4a38-4b10-b2bd-45e4410c588e: ref 0 aborting False
>Thread-15913578::INFO::2013-11-20 03:08:24,225::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:08:24,660::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0608697 s, 16.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:08:26,670::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:08:27,098::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0526674 s, 19.4 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 03:08:28,949::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:08:28,949::task::495::TaskManager.Task::(_debug) Task 965bfc21-b78b-4178-a469-d15f901b10c3: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:08:28,950::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ad1162e6-c60d-4254-b247-28f7475f3e32`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:08:28,951::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:08:28,951::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:08:28,951::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ad1162e6-c60d-4254-b247-28f7475f3e32`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:08:28,952::task::495::TaskManager.Task::(_debug) Task 965bfc21-b78b-4178-a469-d15f901b10c3: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:08:28,952::task::495::TaskManager.Task::(_debug) Task 965bfc21-b78b-4178-a469-d15f901b10c3: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:08:28,953::task::495::TaskManager.Task::(_debug) Task 965bfc21-b78b-4178-a469-d15f901b10c3: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:08:28,953::task::495::TaskManager.Task::(_debug) Task 965bfc21-b78b-4178-a469-d15f901b10c3: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:08:28,953::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:08:28,954::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:08:28,954::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:08:28,955::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:08:28,955::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:08:28,955::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:08:28,956::task::495::TaskManager.Task::(_debug) Task 965bfc21-b78b-4178-a469-d15f901b10c3: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:08:28,956::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:08:29,108::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:08:30,225::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0476025 s, 21.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:08:32,237::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:08:32,795::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0429168 s, 23.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:08:34,804::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913582::DEBUG::2013-11-20 03:08:35,483::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913582::INFO::2013-11-20 03:08:35,484::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913582::DEBUG::2013-11-20 03:08:35,484::task::495::TaskManager.Task::(_debug) Task 6cda29ce-bb2c-4a7a-8682-6e161d3a96de: moving from state init -> state preparing
>Thread-15913582::DEBUG::2013-11-20 03:08:35,485::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913582::DEBUG::2013-11-20 03:08:35,485::task::495::TaskManager.Task::(_debug) Task 6cda29ce-bb2c-4a7a-8682-6e161d3a96de: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913582::DEBUG::2013-11-20 03:08:35,486::task::495::TaskManager.Task::(_debug) Task 6cda29ce-bb2c-4a7a-8682-6e161d3a96de: moving from state preparing -> state finished
>Thread-15913582::DEBUG::2013-11-20 03:08:35,486::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913582::DEBUG::2013-11-20 03:08:35,487::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913582::DEBUG::2013-11-20 03:08:35,487::task::495::TaskManager.Task::(_debug) Task 6cda29ce-bb2c-4a7a-8682-6e161d3a96de: ref 0 aborting False
>Thread-15913582::INFO::2013-11-20 03:08:35,487::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913583::DEBUG::2013-11-20 03:08:35,499::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913583::INFO::2013-11-20 03:08:35,500::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913583::DEBUG::2013-11-20 03:08:35,501::task::495::TaskManager.Task::(_debug) Task 9279fd0b-39bb-41f9-8750-b4a6d270853a: moving from state init -> state preparing
>Thread-15913583::DEBUG::2013-11-20 03:08:35,502::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3b6d5210-03c7-4eb5-a43d-532ba217d18c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913583::DEBUG::2013-11-20 03:08:35,503::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913583::DEBUG::2013-11-20 03:08:35,504::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913583::DEBUG::2013-11-20 03:08:35,504::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3b6d5210-03c7-4eb5-a43d-532ba217d18c`::Granted request
>Thread-15913583::DEBUG::2013-11-20 03:08:35,506::task::495::TaskManager.Task::(_debug) Task 9279fd0b-39bb-41f9-8750-b4a6d270853a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913583::DEBUG::2013-11-20 03:08:35,506::task::495::TaskManager.Task::(_debug) Task 9279fd0b-39bb-41f9-8750-b4a6d270853a: ref 1 aborting False
>Thread-15913583::WARNING::2013-11-20 03:08:35,508::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913583::DEBUG::2013-11-20 03:08:35,509::task::495::TaskManager.Task::(_debug) Task 9279fd0b-39bb-41f9-8750-b4a6d270853a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913583::DEBUG::2013-11-20 03:08:35,510::task::495::TaskManager.Task::(_debug) Task 9279fd0b-39bb-41f9-8750-b4a6d270853a: moving from state preparing -> state finished
>Thread-15913583::DEBUG::2013-11-20 03:08:35,511::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913583::DEBUG::2013-11-20 03:08:35,511::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913583::DEBUG::2013-11-20 03:08:35,512::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913583::DEBUG::2013-11-20 03:08:35,513::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913583::DEBUG::2013-11-20 03:08:35,513::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913583::DEBUG::2013-11-20 03:08:35,514::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913583::DEBUG::2013-11-20 03:08:35,515::task::495::TaskManager.Task::(_debug) Task 9279fd0b-39bb-41f9-8750-b4a6d270853a: ref 0 aborting False
>Thread-15913583::INFO::2013-11-20 03:08:35,515::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:08:35,582::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0594919 s, 17.2 MB/s\n'; <rc> = 0
>Thread-15913584::INFO::2013-11-20 03:08:35,606::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913584::DEBUG::2013-11-20 03:08:35,606::task::495::TaskManager.Task::(_debug) Task a2d5174c-8702-4910-aebb-8b7c7748bea2: moving from state init -> state preparing
>Thread-15913584::DEBUG::2013-11-20 03:08:35,607::task::495::TaskManager.Task::(_debug) Task a2d5174c-8702-4910-aebb-8b7c7748bea2: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000328779220581', 'lastCheck': 1384916909.396462, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.000893115997314', 'lastCheck': 1384916906.6511099, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000971794128418', 'lastCheck': 1384916907.982836, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000352144241333', 'lastCheck': 1384916907.957339, 'valid': True, 'code': 0}}
>Thread-15913584::DEBUG::2013-11-20 03:08:35,607::task::495::TaskManager.Task::(_debug) Task a2d5174c-8702-4910-aebb-8b7c7748bea2: moving from state preparing -> state finished
>Thread-15913584::DEBUG::2013-11-20 03:08:35,608::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913584::DEBUG::2013-11-20 03:08:35,608::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913584::DEBUG::2013-11-20 03:08:35,608::task::495::TaskManager.Task::(_debug) Task a2d5174c-8702-4910-aebb-8b7c7748bea2: ref 0 aborting False
>Thread-15913584::INFO::2013-11-20 03:08:35,615::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000328779220581', 'lastCheck': 1384916909.396462, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.000893115997314', 'lastCheck': 1384916906.6511099, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000971794128418', 'lastCheck': 1384916907.982836, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000352144241333', 'lastCheck': 1384916907.957339, 'valid': True, 'code': 0}}
>Thread-15913585::DEBUG::2013-11-20 03:08:35,653::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913585::DEBUG::2013-11-20 03:08:35,653::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913585::DEBUG::2013-11-20 03:08:35,654::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913585::DEBUG::2013-11-20 03:08:35,654::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913585::DEBUG::2013-11-20 03:08:35,654::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913585::DEBUG::2013-11-20 03:08:35,655::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913585::DEBUG::2013-11-20 03:08:35,655::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913585::DEBUG::2013-11-20 03:08:35,655::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Dummy-15912651::DEBUG::2013-11-20 03:08:37,592::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:08:38,020::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0500032 s, 20.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:08:40,030::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:08:41,355::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0471549 s, 21.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:08:43,364::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:08:44,038::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0463608 s, 22.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:08:46,047::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913519::DEBUG::2013-11-20 03:08:46,265::guestIF::97::vm.Vm::(_handleMessage) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::u'BRD-KLEE\\administrator'
>Thread-15913589::DEBUG::2013-11-20 03:08:46,274::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913589::INFO::2013-11-20 03:08:46,277::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913589::DEBUG::2013-11-20 03:08:46,277::task::495::TaskManager.Task::(_debug) Task 9f461470-9635-48c8-89c0-554690e44d98: moving from state init -> state preparing
>Thread-15913589::DEBUG::2013-11-20 03:08:46,279::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913589::DEBUG::2013-11-20 03:08:46,279::task::495::TaskManager.Task::(_debug) Task 9f461470-9635-48c8-89c0-554690e44d98: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913589::DEBUG::2013-11-20 03:08:46,280::task::495::TaskManager.Task::(_debug) Task 9f461470-9635-48c8-89c0-554690e44d98: moving from state preparing -> state finished
>Thread-15913589::DEBUG::2013-11-20 03:08:46,281::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913589::DEBUG::2013-11-20 03:08:46,281::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913589::DEBUG::2013-11-20 03:08:46,282::task::495::TaskManager.Task::(_debug) Task 9f461470-9635-48c8-89c0-554690e44d98: ref 0 aborting False
>Thread-15913589::INFO::2013-11-20 03:08:46,283::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913590::DEBUG::2013-11-20 03:08:46,288::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913590::INFO::2013-11-20 03:08:46,289::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913590::DEBUG::2013-11-20 03:08:46,289::task::495::TaskManager.Task::(_debug) Task 48974032-9511-4474-b75f-c733a94b1e2c: moving from state init -> state preparing
>Thread-15913590::DEBUG::2013-11-20 03:08:46,291::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a9b7608a-c4fa-4706-bd39-6e7521bbf976`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913590::DEBUG::2013-11-20 03:08:46,291::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913590::DEBUG::2013-11-20 03:08:46,292::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913590::DEBUG::2013-11-20 03:08:46,293::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a9b7608a-c4fa-4706-bd39-6e7521bbf976`::Granted request
>Thread-15913590::DEBUG::2013-11-20 03:08:46,294::task::495::TaskManager.Task::(_debug) Task 48974032-9511-4474-b75f-c733a94b1e2c: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913590::DEBUG::2013-11-20 03:08:46,294::task::495::TaskManager.Task::(_debug) Task 48974032-9511-4474-b75f-c733a94b1e2c: ref 1 aborting False
>Thread-15913590::WARNING::2013-11-20 03:08:46,297::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913590::DEBUG::2013-11-20 03:08:46,298::task::495::TaskManager.Task::(_debug) Task 48974032-9511-4474-b75f-c733a94b1e2c: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913590::DEBUG::2013-11-20 03:08:46,299::task::495::TaskManager.Task::(_debug) Task 48974032-9511-4474-b75f-c733a94b1e2c: moving from state preparing -> state finished
>Thread-15913590::DEBUG::2013-11-20 03:08:46,299::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913590::DEBUG::2013-11-20 03:08:46,300::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913590::DEBUG::2013-11-20 03:08:46,300::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913590::DEBUG::2013-11-20 03:08:46,301::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913590::DEBUG::2013-11-20 03:08:46,301::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913590::DEBUG::2013-11-20 03:08:46,302::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913590::DEBUG::2013-11-20 03:08:46,303::task::495::TaskManager.Task::(_debug) Task 48974032-9511-4474-b75f-c733a94b1e2c: ref 0 aborting False
>Thread-15913590::INFO::2013-11-20 03:08:46,304::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:08:46,614::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0721798 s, 14.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:08:48,629::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:08:49,186::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0725625 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:08:51,201::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 03:08:51,421::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913592::INFO::2013-11-20 03:08:51,426::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913546::DEBUG::2013-11-20 03:08:51,429::task::495::TaskManager.Task::(_debug) Task 0fbcc2bf-1335-47f2-afe4-ea2406307ab9: moving from state init -> state preparing
>Thread-15913592::DEBUG::2013-11-20 03:08:51,429::task::495::TaskManager.Task::(_debug) Task 9393ceba-1623-4ea3-8d00-fa0e4c3078c7: moving from state init -> state preparing
>Thread-15913592::DEBUG::2013-11-20 03:08:51,437::task::495::TaskManager.Task::(_debug) Task 9393ceba-1623-4ea3-8d00-fa0e4c3078c7: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000375986099243', 'lastCheck': 1384916929.5376949, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00121903419495', 'lastCheck': 1384916926.772265, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0011191368103', 'lastCheck': 1384916928.0153501, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000286102294922', 'lastCheck': 1384916927.977339, 'valid': True, 'code': 0}}
>Thread-15913546::DEBUG::2013-11-20 03:08:51,439::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fde9c862-8b36-4495-bb6a-18cebc008fce`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913592::DEBUG::2013-11-20 03:08:51,440::task::495::TaskManager.Task::(_debug) Task 9393ceba-1623-4ea3-8d00-fa0e4c3078c7: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:08:51,440::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913592::DEBUG::2013-11-20 03:08:51,441::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913592::DEBUG::2013-11-20 03:08:51,442::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:08:51,442::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913592::DEBUG::2013-11-20 03:08:51,443::task::495::TaskManager.Task::(_debug) Task 9393ceba-1623-4ea3-8d00-fa0e4c3078c7: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:08:51,443::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fde9c862-8b36-4495-bb6a-18cebc008fce`::Granted request
>Thread-15913592::INFO::2013-11-20 03:08:51,444::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000375986099243', 'lastCheck': 1384916929.5376949, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00121903419495', 'lastCheck': 1384916926.772265, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0011191368103', 'lastCheck': 1384916928.0153501, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000286102294922', 'lastCheck': 1384916927.977339, 'valid': True, 'code': 0}}
>Thread-15913546::DEBUG::2013-11-20 03:08:51,445::task::495::TaskManager.Task::(_debug) Task 0fbcc2bf-1335-47f2-afe4-ea2406307ab9: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:08:51,446::task::495::TaskManager.Task::(_debug) Task 0fbcc2bf-1335-47f2-afe4-ea2406307ab9: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:08:51,447::task::495::TaskManager.Task::(_debug) Task 0fbcc2bf-1335-47f2-afe4-ea2406307ab9: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:08:51,448::task::495::TaskManager.Task::(_debug) Task 0fbcc2bf-1335-47f2-afe4-ea2406307ab9: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:08:51,449::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:08:51,459::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:08:51,460::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:08:51,461::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:08:51,462::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:08:51,462::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:08:51,463::task::495::TaskManager.Task::(_debug) Task 0fbcc2bf-1335-47f2-afe4-ea2406307ab9: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:08:51,464::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:08:51,465::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:08:51,465::task::495::TaskManager.Task::(_debug) Task 39dc4fad-91d8-4ffa-88b5-1df65ffd1b47: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:08:51,466::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fab9ee33-20fe-40c7-8728-5f4cec5e81f4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:08:51,467::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:08:51,468::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:08:51,468::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fab9ee33-20fe-40c7-8728-5f4cec5e81f4`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:08:51,469::task::495::TaskManager.Task::(_debug) Task 39dc4fad-91d8-4ffa-88b5-1df65ffd1b47: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:08:51,470::task::495::TaskManager.Task::(_debug) Task 39dc4fad-91d8-4ffa-88b5-1df65ffd1b47: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:08:51,470::task::495::TaskManager.Task::(_debug) Task 39dc4fad-91d8-4ffa-88b5-1df65ffd1b47: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:08:51,471::task::495::TaskManager.Task::(_debug) Task 39dc4fad-91d8-4ffa-88b5-1df65ffd1b47: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:08:51,471::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:08:51,472::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:08:51,473::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:08:51,474::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:08:51,474::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:08:51,475::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:08:51,475::task::495::TaskManager.Task::(_debug) Task 39dc4fad-91d8-4ffa-88b5-1df65ffd1b47: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:08:51,476::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:08:51,477::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:08:51,477::task::495::TaskManager.Task::(_debug) Task bc65a2ff-8ba7-4227-addc-cba4b01489c4: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:08:51,478::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6c889ec0-1135-4a78-8f89-cbd0c8590a53`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:08:51,479::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:08:51,480::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:08:51,480::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6c889ec0-1135-4a78-8f89-cbd0c8590a53`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:08:51,481::task::495::TaskManager.Task::(_debug) Task bc65a2ff-8ba7-4227-addc-cba4b01489c4: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:08:51,482::task::495::TaskManager.Task::(_debug) Task bc65a2ff-8ba7-4227-addc-cba4b01489c4: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:08:51,482::task::495::TaskManager.Task::(_debug) Task bc65a2ff-8ba7-4227-addc-cba4b01489c4: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:08:51,483::task::495::TaskManager.Task::(_debug) Task bc65a2ff-8ba7-4227-addc-cba4b01489c4: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:08:51,483::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:08:51,484::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:08:51,485::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:08:51,485::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:08:51,486::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:08:51,487::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:08:51,488::task::495::TaskManager.Task::(_debug) Task bc65a2ff-8ba7-4227-addc-cba4b01489c4: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:08:51,489::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913593::DEBUG::2013-11-20 03:08:51,490::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913546::INFO::2013-11-20 03:08:51,491::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913593::DEBUG::2013-11-20 03:08:51,492::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913546::DEBUG::2013-11-20 03:08:51,492::task::495::TaskManager.Task::(_debug) Task 0807146f-7d87-434f-bdea-3238790e5377: moving from state init -> state preparing
>Thread-15913593::DEBUG::2013-11-20 03:08:51,493::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913546::DEBUG::2013-11-20 03:08:51,494::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`25989cbf-a80b-44eb-bf45-e2f24b8c4f8a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913593::DEBUG::2013-11-20 03:08:51,494::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk stats not available
>Thread-15913546::DEBUG::2013-11-20 03:08:51,495::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913593::DEBUG::2013-11-20 03:08:51,495::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913546::DEBUG::2013-11-20 03:08:51,496::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913593::DEBUG::2013-11-20 03:08:51,497::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913546::DEBUG::2013-11-20 03:08:51,497::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`25989cbf-a80b-44eb-bf45-e2f24b8c4f8a`::Granted request
>Thread-15913593::DEBUG::2013-11-20 03:08:51,498::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913546::DEBUG::2013-11-20 03:08:51,499::task::495::TaskManager.Task::(_debug) Task 0807146f-7d87-434f-bdea-3238790e5377: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913593::DEBUG::2013-11-20 03:08:51,499::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::Disk latency not available
>Thread-15913546::DEBUG::2013-11-20 03:08:51,500::task::495::TaskManager.Task::(_debug) Task 0807146f-7d87-434f-bdea-3238790e5377: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:08:51,512::task::495::TaskManager.Task::(_debug) Task 0807146f-7d87-434f-bdea-3238790e5377: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:08:51,513::task::495::TaskManager.Task::(_debug) Task 0807146f-7d87-434f-bdea-3238790e5377: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:08:51,513::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:08:51,514::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:08:51,515::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:08:51,515::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:08:51,516::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:08:51,516::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:08:51,517::task::495::TaskManager.Task::(_debug) Task 0807146f-7d87-434f-bdea-3238790e5377: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:08:51,517::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:08:51,737::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.072096 s, 14.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:08:53,752::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:08:54,328::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0767623 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:08:56,351::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:08:56,898::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0844411 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15913595::DEBUG::2013-11-20 03:08:57,368::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913595::INFO::2013-11-20 03:08:57,369::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913595::DEBUG::2013-11-20 03:08:57,370::task::495::TaskManager.Task::(_debug) Task 9b155268-4849-4fa2-bc41-c9ca2accbcb0: moving from state init -> state preparing
>Thread-15913595::DEBUG::2013-11-20 03:08:57,371::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913595::DEBUG::2013-11-20 03:08:57,371::task::495::TaskManager.Task::(_debug) Task 9b155268-4849-4fa2-bc41-c9ca2accbcb0: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913595::DEBUG::2013-11-20 03:08:57,372::task::495::TaskManager.Task::(_debug) Task 9b155268-4849-4fa2-bc41-c9ca2accbcb0: moving from state preparing -> state finished
>Thread-15913595::DEBUG::2013-11-20 03:08:57,373::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913595::DEBUG::2013-11-20 03:08:57,373::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913595::DEBUG::2013-11-20 03:08:57,374::task::495::TaskManager.Task::(_debug) Task 9b155268-4849-4fa2-bc41-c9ca2accbcb0: ref 0 aborting False
>Thread-15913595::INFO::2013-11-20 03:08:57,374::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913596::DEBUG::2013-11-20 03:08:57,383::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913596::INFO::2013-11-20 03:08:57,384::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913596::DEBUG::2013-11-20 03:08:57,384::task::495::TaskManager.Task::(_debug) Task 247454d8-2929-4626-bf2f-b67d38bc2023: moving from state init -> state preparing
>Thread-15913596::DEBUG::2013-11-20 03:08:57,386::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`76292a06-b5cd-4c35-8316-f7854aaab0ec`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913596::DEBUG::2013-11-20 03:08:57,386::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913596::DEBUG::2013-11-20 03:08:57,387::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913596::DEBUG::2013-11-20 03:08:57,388::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`76292a06-b5cd-4c35-8316-f7854aaab0ec`::Granted request
>Thread-15913596::DEBUG::2013-11-20 03:08:57,389::task::495::TaskManager.Task::(_debug) Task 247454d8-2929-4626-bf2f-b67d38bc2023: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913596::DEBUG::2013-11-20 03:08:57,389::task::495::TaskManager.Task::(_debug) Task 247454d8-2929-4626-bf2f-b67d38bc2023: ref 1 aborting False
>Thread-15913596::WARNING::2013-11-20 03:08:57,392::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913596::DEBUG::2013-11-20 03:08:57,393::task::495::TaskManager.Task::(_debug) Task 247454d8-2929-4626-bf2f-b67d38bc2023: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913596::DEBUG::2013-11-20 03:08:57,394::task::495::TaskManager.Task::(_debug) Task 247454d8-2929-4626-bf2f-b67d38bc2023: moving from state preparing -> state finished
>Thread-15913596::DEBUG::2013-11-20 03:08:57,395::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913596::DEBUG::2013-11-20 03:08:57,395::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913596::DEBUG::2013-11-20 03:08:57,396::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913596::DEBUG::2013-11-20 03:08:57,396::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913596::DEBUG::2013-11-20 03:08:57,397::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913596::DEBUG::2013-11-20 03:08:57,397::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913596::DEBUG::2013-11-20 03:08:57,398::task::495::TaskManager.Task::(_debug) Task 247454d8-2929-4626-bf2f-b67d38bc2023: ref 0 aborting False
>Thread-15913596::INFO::2013-11-20 03:08:57,399::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:08:58,919::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:08:59,442::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0749616 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:09:01,458::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:09:02,013::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0854365 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:09:04,029::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:09:04,564::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.075179 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:09:06,579::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:09:07,076::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0806673 s, 12.7 MB/s\n'; <rc> = 0
>Thread-15913600::INFO::2013-11-20 03:09:07,294::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913600::DEBUG::2013-11-20 03:09:07,295::task::495::TaskManager.Task::(_debug) Task 355e4cc7-0860-4089-b0a6-b390a78c561d: moving from state init -> state preparing
>Thread-15913600::DEBUG::2013-11-20 03:09:07,296::task::495::TaskManager.Task::(_debug) Task 355e4cc7-0860-4089-b0a6-b390a78c561d: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000380039215088', 'lastCheck': 1384916939.5629361, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148892402649', 'lastCheck': 1384916947.2746351, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.027379989624', 'lastCheck': 1384916938.0571539, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000699996948242', 'lastCheck': 1384916937.9867001, 'valid': True, 'code': 0}}
>Thread-15913600::DEBUG::2013-11-20 03:09:07,296::task::495::TaskManager.Task::(_debug) Task 355e4cc7-0860-4089-b0a6-b390a78c561d: moving from state preparing -> state finished
>Thread-15913600::DEBUG::2013-11-20 03:09:07,297::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913600::DEBUG::2013-11-20 03:09:07,298::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913600::DEBUG::2013-11-20 03:09:07,298::task::495::TaskManager.Task::(_debug) Task 355e4cc7-0860-4089-b0a6-b390a78c561d: ref 0 aborting False
>Thread-15913600::INFO::2013-11-20 03:09:07,299::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000380039215088', 'lastCheck': 1384916939.5629361, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148892402649', 'lastCheck': 1384916947.2746351, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.027379989624', 'lastCheck': 1384916938.0571539, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000699996948242', 'lastCheck': 1384916937.9867001, 'valid': True, 'code': 0}}
>Thread-15913602::DEBUG::2013-11-20 03:09:08,188::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913602::INFO::2013-11-20 03:09:08,189::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913602::DEBUG::2013-11-20 03:09:08,198::task::495::TaskManager.Task::(_debug) Task 6e30ef06-b303-4293-8c4f-a2b164c2dbf4: moving from state init -> state preparing
>Thread-15913602::DEBUG::2013-11-20 03:09:08,199::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913602::DEBUG::2013-11-20 03:09:08,199::task::495::TaskManager.Task::(_debug) Task 6e30ef06-b303-4293-8c4f-a2b164c2dbf4: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913602::DEBUG::2013-11-20 03:09:08,200::task::495::TaskManager.Task::(_debug) Task 6e30ef06-b303-4293-8c4f-a2b164c2dbf4: moving from state preparing -> state finished
>Thread-15913602::DEBUG::2013-11-20 03:09:08,201::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913602::DEBUG::2013-11-20 03:09:08,201::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913602::DEBUG::2013-11-20 03:09:08,202::task::495::TaskManager.Task::(_debug) Task 6e30ef06-b303-4293-8c4f-a2b164c2dbf4: ref 0 aborting False
>Thread-15913602::INFO::2013-11-20 03:09:08,203::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913603::DEBUG::2013-11-20 03:09:08,213::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913603::INFO::2013-11-20 03:09:08,214::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913603::DEBUG::2013-11-20 03:09:08,215::task::495::TaskManager.Task::(_debug) Task eb200327-f6d0-4b2e-8c7f-6657a79c7870: moving from state init -> state preparing
>Thread-15913603::DEBUG::2013-11-20 03:09:08,216::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d0ade912-c5a0-4867-b301-c0a1a15d9521`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913603::DEBUG::2013-11-20 03:09:08,217::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913603::DEBUG::2013-11-20 03:09:08,218::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913603::DEBUG::2013-11-20 03:09:08,219::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d0ade912-c5a0-4867-b301-c0a1a15d9521`::Granted request
>Thread-15913603::DEBUG::2013-11-20 03:09:08,220::task::495::TaskManager.Task::(_debug) Task eb200327-f6d0-4b2e-8c7f-6657a79c7870: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913603::DEBUG::2013-11-20 03:09:08,221::task::495::TaskManager.Task::(_debug) Task eb200327-f6d0-4b2e-8c7f-6657a79c7870: ref 1 aborting False
>Thread-15913603::WARNING::2013-11-20 03:09:08,223::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913603::DEBUG::2013-11-20 03:09:08,224::task::495::TaskManager.Task::(_debug) Task eb200327-f6d0-4b2e-8c7f-6657a79c7870: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913603::DEBUG::2013-11-20 03:09:08,226::task::495::TaskManager.Task::(_debug) Task eb200327-f6d0-4b2e-8c7f-6657a79c7870: moving from state preparing -> state finished
>Thread-15913603::DEBUG::2013-11-20 03:09:08,226::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913603::DEBUG::2013-11-20 03:09:08,227::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913603::DEBUG::2013-11-20 03:09:08,228::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913603::DEBUG::2013-11-20 03:09:08,229::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913603::DEBUG::2013-11-20 03:09:08,229::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913603::DEBUG::2013-11-20 03:09:08,230::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913603::DEBUG::2013-11-20 03:09:08,231::task::495::TaskManager.Task::(_debug) Task eb200327-f6d0-4b2e-8c7f-6657a79c7870: ref 0 aborting False
>Thread-15913603::INFO::2013-11-20 03:09:08,232::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-42840::DEBUG::2013-11-20 03:09:08,268::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-15913547::DEBUG::2013-11-20 03:09:08,752::guestIF::97::vm.Vm::(_handleMessage) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::u'None'
>Dummy-15912651::DEBUG::2013-11-20 03:09:09,093::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-42840::DEBUG::2013-11-20 03:09:09,359::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:09:09,712::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0744745 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:09:11,726::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:09:12,287::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0739559 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:09:14,302::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:09:14,865::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0766823 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:09:16,882::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:09:17,476::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0833164 s, 12.3 MB/s\n'; <rc> = 0
>Thread-15913607::DEBUG::2013-11-20 03:09:18,611::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913607::INFO::2013-11-20 03:09:18,612::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913607::DEBUG::2013-11-20 03:09:18,613::task::495::TaskManager.Task::(_debug) Task df625182-bc9c-4ad1-9dbc-59bf5b6d3de8: moving from state init -> state preparing
>Thread-15913607::DEBUG::2013-11-20 03:09:18,614::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913607::DEBUG::2013-11-20 03:09:18,614::task::495::TaskManager.Task::(_debug) Task df625182-bc9c-4ad1-9dbc-59bf5b6d3de8: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913607::DEBUG::2013-11-20 03:09:18,614::task::495::TaskManager.Task::(_debug) Task df625182-bc9c-4ad1-9dbc-59bf5b6d3de8: moving from state preparing -> state finished
>Thread-15913607::DEBUG::2013-11-20 03:09:18,615::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913607::DEBUG::2013-11-20 03:09:18,615::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913607::DEBUG::2013-11-20 03:09:18,615::task::495::TaskManager.Task::(_debug) Task df625182-bc9c-4ad1-9dbc-59bf5b6d3de8: ref 0 aborting False
>Thread-15913607::INFO::2013-11-20 03:09:18,616::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913608::DEBUG::2013-11-20 03:09:18,668::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913608::INFO::2013-11-20 03:09:18,669::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913608::DEBUG::2013-11-20 03:09:18,670::task::495::TaskManager.Task::(_debug) Task 4a4faef5-2a5e-45f5-bafd-768940d4667a: moving from state init -> state preparing
>Thread-15913608::DEBUG::2013-11-20 03:09:18,671::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`99c75046-c00d-49d8-842c-a61af16be77e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913608::DEBUG::2013-11-20 03:09:18,672::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913608::DEBUG::2013-11-20 03:09:18,672::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913608::DEBUG::2013-11-20 03:09:18,673::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`99c75046-c00d-49d8-842c-a61af16be77e`::Granted request
>Thread-15913608::DEBUG::2013-11-20 03:09:18,674::task::495::TaskManager.Task::(_debug) Task 4a4faef5-2a5e-45f5-bafd-768940d4667a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913608::DEBUG::2013-11-20 03:09:18,674::task::495::TaskManager.Task::(_debug) Task 4a4faef5-2a5e-45f5-bafd-768940d4667a: ref 1 aborting False
>Thread-15913608::WARNING::2013-11-20 03:09:18,676::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913608::DEBUG::2013-11-20 03:09:18,677::task::495::TaskManager.Task::(_debug) Task 4a4faef5-2a5e-45f5-bafd-768940d4667a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913608::DEBUG::2013-11-20 03:09:18,678::task::495::TaskManager.Task::(_debug) Task 4a4faef5-2a5e-45f5-bafd-768940d4667a: moving from state preparing -> state finished
>Thread-15913608::DEBUG::2013-11-20 03:09:18,679::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913608::DEBUG::2013-11-20 03:09:18,679::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913608::DEBUG::2013-11-20 03:09:18,680::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913608::DEBUG::2013-11-20 03:09:18,680::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913608::DEBUG::2013-11-20 03:09:18,681::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913608::DEBUG::2013-11-20 03:09:18,681::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913608::DEBUG::2013-11-20 03:09:18,682::task::495::TaskManager.Task::(_debug) Task 4a4faef5-2a5e-45f5-bafd-768940d4667a: ref 0 aborting False
>Thread-15913608::INFO::2013-11-20 03:09:18,682::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:09:19,491::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:09:20,001::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0813159 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:09:22,016::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:09:22,585::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0725957 s, 14.1 MB/s\n'; <rc> = 0
>Thread-15913610::INFO::2013-11-20 03:09:23,283::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913610::DEBUG::2013-11-20 03:09:23,284::task::495::TaskManager.Task::(_debug) Task c92eb1e8-e342-42bb-a515-87b0fb0b8335: moving from state init -> state preparing
>Thread-15913610::DEBUG::2013-11-20 03:09:23,285::task::495::TaskManager.Task::(_debug) Task c92eb1e8-e342-42bb-a515-87b0fb0b8335: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000266075134277', 'lastCheck': 1384916960.1370821, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0264580249786', 'lastCheck': 1384916957.5682299, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.001384973526', 'lastCheck': 1384916958.5433421, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00109219551086', 'lastCheck': 1384916959.6700101, 'valid': True, 'code': 0}}
>Thread-15913610::DEBUG::2013-11-20 03:09:23,292::task::495::TaskManager.Task::(_debug) Task c92eb1e8-e342-42bb-a515-87b0fb0b8335: moving from state preparing -> state finished
>Thread-15913610::DEBUG::2013-11-20 03:09:23,292::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913610::DEBUG::2013-11-20 03:09:23,293::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913610::DEBUG::2013-11-20 03:09:23,294::task::495::TaskManager.Task::(_debug) Task c92eb1e8-e342-42bb-a515-87b0fb0b8335: ref 0 aborting False
>Thread-15913610::INFO::2013-11-20 03:09:23,295::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000266075134277', 'lastCheck': 1384916960.1370821, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0264580249786', 'lastCheck': 1384916957.5682299, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.001384973526', 'lastCheck': 1384916958.5433421, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00109219551086', 'lastCheck': 1384916959.6700101, 'valid': True, 'code': 0}}
>Thread-15913518::INFO::2013-11-20 03:09:23,718::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:09:23,719::task::495::TaskManager.Task::(_debug) Task 4e02bd12-41cb-41fa-b3dd-34d409daca6a: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:09:23,721::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`593cff0f-f8bf-4dff-8b09-2db196dd2f2d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:09:23,725::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:09:23,726::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:09:23,727::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`593cff0f-f8bf-4dff-8b09-2db196dd2f2d`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:09:23,728::task::495::TaskManager.Task::(_debug) Task 4e02bd12-41cb-41fa-b3dd-34d409daca6a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:09:23,728::task::495::TaskManager.Task::(_debug) Task 4e02bd12-41cb-41fa-b3dd-34d409daca6a: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:09:23,729::task::495::TaskManager.Task::(_debug) Task 4e02bd12-41cb-41fa-b3dd-34d409daca6a: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:09:23,730::task::495::TaskManager.Task::(_debug) Task 4e02bd12-41cb-41fa-b3dd-34d409daca6a: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:09:23,730::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:09:23,731::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:09:23,731::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:09:23,732::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:09:23,733::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:09:23,733::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:09:23,734::task::495::TaskManager.Task::(_debug) Task 4e02bd12-41cb-41fa-b3dd-34d409daca6a: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:09:23,734::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:09:23,735::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:09:23,736::task::495::TaskManager.Task::(_debug) Task c5b964fc-3716-4bee-bbf4-d184ab058d23: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:09:23,737::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`99c4b2ed-20c2-47a7-81a8-fe195c36361d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:09:23,742::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:09:23,743::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:09:23,743::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`99c4b2ed-20c2-47a7-81a8-fe195c36361d`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:09:23,744::task::495::TaskManager.Task::(_debug) Task c5b964fc-3716-4bee-bbf4-d184ab058d23: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:09:23,745::task::495::TaskManager.Task::(_debug) Task c5b964fc-3716-4bee-bbf4-d184ab058d23: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:09:23,745::task::495::TaskManager.Task::(_debug) Task c5b964fc-3716-4bee-bbf4-d184ab058d23: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:09:23,746::task::495::TaskManager.Task::(_debug) Task c5b964fc-3716-4bee-bbf4-d184ab058d23: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:09:23,746::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:09:23,747::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:09:23,748::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:09:23,748::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:09:23,749::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:09:23,749::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:09:23,750::task::495::TaskManager.Task::(_debug) Task c5b964fc-3716-4bee-bbf4-d184ab058d23: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:09:23,750::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:09:24,600::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:09:25,068::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.050352 s, 20.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:09:27,082::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:09:27,641::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.081448 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15913613::DEBUG::2013-11-20 03:09:29,093::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913613::INFO::2013-11-20 03:09:29,095::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913613::DEBUG::2013-11-20 03:09:29,096::task::495::TaskManager.Task::(_debug) Task 24337010-50aa-41d2-9344-69afc23f94d9: moving from state init -> state preparing
>Thread-15913613::DEBUG::2013-11-20 03:09:29,097::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913613::DEBUG::2013-11-20 03:09:29,097::task::495::TaskManager.Task::(_debug) Task 24337010-50aa-41d2-9344-69afc23f94d9: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913613::DEBUG::2013-11-20 03:09:29,098::task::495::TaskManager.Task::(_debug) Task 24337010-50aa-41d2-9344-69afc23f94d9: moving from state preparing -> state finished
>Thread-15913613::DEBUG::2013-11-20 03:09:29,098::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913613::DEBUG::2013-11-20 03:09:29,098::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913613::DEBUG::2013-11-20 03:09:29,099::task::495::TaskManager.Task::(_debug) Task 24337010-50aa-41d2-9344-69afc23f94d9: ref 0 aborting False
>Thread-15913613::INFO::2013-11-20 03:09:29,099::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913614::DEBUG::2013-11-20 03:09:29,103::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913614::INFO::2013-11-20 03:09:29,103::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913614::DEBUG::2013-11-20 03:09:29,104::task::495::TaskManager.Task::(_debug) Task 2dfc18ca-3873-4682-97c4-6a8d03daf74e: moving from state init -> state preparing
>Thread-15913614::DEBUG::2013-11-20 03:09:29,105::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`18047469-5c3f-4ea9-b61b-4fa4b9dc72e5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913614::DEBUG::2013-11-20 03:09:29,105::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913614::DEBUG::2013-11-20 03:09:29,106::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913614::DEBUG::2013-11-20 03:09:29,106::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`18047469-5c3f-4ea9-b61b-4fa4b9dc72e5`::Granted request
>Thread-15913614::DEBUG::2013-11-20 03:09:29,111::task::495::TaskManager.Task::(_debug) Task 2dfc18ca-3873-4682-97c4-6a8d03daf74e: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913614::DEBUG::2013-11-20 03:09:29,111::task::495::TaskManager.Task::(_debug) Task 2dfc18ca-3873-4682-97c4-6a8d03daf74e: ref 1 aborting False
>Thread-15913614::WARNING::2013-11-20 03:09:29,113::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913614::DEBUG::2013-11-20 03:09:29,113::task::495::TaskManager.Task::(_debug) Task 2dfc18ca-3873-4682-97c4-6a8d03daf74e: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913614::DEBUG::2013-11-20 03:09:29,114::task::495::TaskManager.Task::(_debug) Task 2dfc18ca-3873-4682-97c4-6a8d03daf74e: moving from state preparing -> state finished
>Thread-15913614::DEBUG::2013-11-20 03:09:29,114::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913614::DEBUG::2013-11-20 03:09:29,115::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913614::DEBUG::2013-11-20 03:09:29,115::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913614::DEBUG::2013-11-20 03:09:29,115::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913614::DEBUG::2013-11-20 03:09:29,116::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913614::DEBUG::2013-11-20 03:09:29,116::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913614::DEBUG::2013-11-20 03:09:29,117::task::495::TaskManager.Task::(_debug) Task 2dfc18ca-3873-4682-97c4-6a8d03daf74e: ref 0 aborting False
>Thread-15913614::INFO::2013-11-20 03:09:29,117::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:09:29,655::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 03:09:29,884::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>libvirtEventLoop::DEBUG::2013-11-20 03:09:29,886::vm::525::vm.Vm::(_rtcUpdate) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::new rtc offset 1
>Thread-15912392::DEBUG::2013-11-20 03:09:29,887::task::495::TaskManager.Task::(_debug) Task a3759285-1977-4426-a836-00ba6767410e: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:09:29,890::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`783af11b-6537-48b0-abc7-9458fc2c6acc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>libvirtEventLoop::DEBUG::2013-11-20 03:09:29,891::vm::525::vm.Vm::(_rtcUpdate) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::new rtc offset 1
>Thread-15912392::DEBUG::2013-11-20 03:09:29,891::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>libvirtEventLoop::DEBUG::2013-11-20 03:09:29,892::vm::525::vm.Vm::(_rtcUpdate) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::new rtc offset 3601
>Thread-15912392::DEBUG::2013-11-20 03:09:29,894::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>libvirtEventLoop::DEBUG::2013-11-20 03:09:29,895::vm::525::vm.Vm::(_rtcUpdate) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::new rtc offset 3601
>Thread-15912392::DEBUG::2013-11-20 03:09:29,895::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`783af11b-6537-48b0-abc7-9458fc2c6acc`::Granted request
>libvirtEventLoop::DEBUG::2013-11-20 03:09:29,896::vm::525::vm.Vm::(_rtcUpdate) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::new rtc offset 3601
>Thread-15912392::DEBUG::2013-11-20 03:09:29,898::task::495::TaskManager.Task::(_debug) Task a3759285-1977-4426-a836-00ba6767410e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:09:29,899::task::495::TaskManager.Task::(_debug) Task a3759285-1977-4426-a836-00ba6767410e: ref 1 aborting False
>libvirtEventLoop::DEBUG::2013-11-20 03:09:29,899::vm::525::vm.Vm::(_rtcUpdate) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::new rtc offset 3601
>Thread-15912392::DEBUG::2013-11-20 03:09:29,900::task::495::TaskManager.Task::(_debug) Task a3759285-1977-4426-a836-00ba6767410e: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:09:29,901::task::495::TaskManager.Task::(_debug) Task a3759285-1977-4426-a836-00ba6767410e: moving from state preparing -> state finished
>libvirtEventLoop::DEBUG::2013-11-20 03:09:29,901::vm::525::vm.Vm::(_rtcUpdate) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::new rtc offset 3601
>Thread-15912392::DEBUG::2013-11-20 03:09:29,902::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>libvirtEventLoop::DEBUG::2013-11-20 03:09:29,903::vm::525::vm.Vm::(_rtcUpdate) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::new rtc offset 3601
>Thread-15912392::DEBUG::2013-11-20 03:09:29,904::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>libvirtEventLoop::DEBUG::2013-11-20 03:09:29,905::vm::525::vm.Vm::(_rtcUpdate) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::new rtc offset 3601
>Thread-15912392::DEBUG::2013-11-20 03:09:29,906::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>libvirtEventLoop::DEBUG::2013-11-20 03:09:29,906::vm::525::vm.Vm::(_rtcUpdate) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::new rtc offset 3601
>Thread-15912392::DEBUG::2013-11-20 03:09:29,907::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:09:29,908::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>libvirtEventLoop::DEBUG::2013-11-20 03:09:29,908::vm::525::vm.Vm::(_rtcUpdate) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::new rtc offset 3601
>Thread-15912392::DEBUG::2013-11-20 03:09:29,909::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>libvirtEventLoop::DEBUG::2013-11-20 03:09:29,910::vm::525::vm.Vm::(_rtcUpdate) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::new rtc offset 3601
>Thread-15912392::DEBUG::2013-11-20 03:09:29,910::task::495::TaskManager.Task::(_debug) Task a3759285-1977-4426-a836-00ba6767410e: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:09:29,911::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>libvirtEventLoop::DEBUG::2013-11-20 03:09:29,912::vm::525::vm.Vm::(_rtcUpdate) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::new rtc offset 3601
>libvirtEventLoop::DEBUG::2013-11-20 03:09:29,913::vm::525::vm.Vm::(_rtcUpdate) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::new rtc offset 3601
>Dummy-15912651::DEBUG::2013-11-20 03:09:30,216::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.072386 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:09:32,231::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:09:32,765::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0723186 s, 14.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:09:34,781::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913547::DEBUG::2013-11-20 03:09:35,188::guestIF::97::vm.Vm::(_handleMessage) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::u'BRD-KLEE\\administrator'
>Dummy-15912651::DEBUG::2013-11-20 03:09:35,324::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0708343 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:09:37,339::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:09:37,865::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0944137 s, 10.8 MB/s\n'; <rc> = 0
>Thread-15913618::INFO::2013-11-20 03:09:39,627::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913618::DEBUG::2013-11-20 03:09:39,628::task::495::TaskManager.Task::(_debug) Task 53aeb014-4def-4611-9b2e-c8f10720dd2d: moving from state init -> state preparing
>Thread-15913618::DEBUG::2013-11-20 03:09:39,629::task::495::TaskManager.Task::(_debug) Task 53aeb014-4def-4611-9b2e-c8f10720dd2d: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000306844711304', 'lastCheck': 1384916970.3152349, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00189018249512', 'lastCheck': 1384916978.0192449, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00112295150757', 'lastCheck': 1384916979.0197389, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00469899177551', 'lastCheck': 1384916969.885715, 'valid': True, 'code': 0}}
>Thread-15913618::DEBUG::2013-11-20 03:09:39,630::task::495::TaskManager.Task::(_debug) Task 53aeb014-4def-4611-9b2e-c8f10720dd2d: moving from state preparing -> state finished
>Thread-15913618::DEBUG::2013-11-20 03:09:39,637::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913618::DEBUG::2013-11-20 03:09:39,638::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913618::DEBUG::2013-11-20 03:09:39,639::task::495::TaskManager.Task::(_debug) Task 53aeb014-4def-4611-9b2e-c8f10720dd2d: ref 0 aborting False
>Thread-15913618::INFO::2013-11-20 03:09:39,640::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000306844711304', 'lastCheck': 1384916970.3152349, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00189018249512', 'lastCheck': 1384916978.0192449, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00112295150757', 'lastCheck': 1384916979.0197389, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00469899177551', 'lastCheck': 1384916969.885715, 'valid': True, 'code': 0}}
>Thread-15913619::DEBUG::2013-11-20 03:09:39,648::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913619::INFO::2013-11-20 03:09:39,650::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913619::DEBUG::2013-11-20 03:09:39,655::task::495::TaskManager.Task::(_debug) Task a1ab11c0-4c3f-4903-89c1-56cd429d70a7: moving from state init -> state preparing
>Thread-15913619::DEBUG::2013-11-20 03:09:39,656::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913619::DEBUG::2013-11-20 03:09:39,657::task::495::TaskManager.Task::(_debug) Task a1ab11c0-4c3f-4903-89c1-56cd429d70a7: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913619::DEBUG::2013-11-20 03:09:39,658::task::495::TaskManager.Task::(_debug) Task a1ab11c0-4c3f-4903-89c1-56cd429d70a7: moving from state preparing -> state finished
>Thread-15913619::DEBUG::2013-11-20 03:09:39,663::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913619::DEBUG::2013-11-20 03:09:39,663::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913619::DEBUG::2013-11-20 03:09:39,664::task::495::TaskManager.Task::(_debug) Task a1ab11c0-4c3f-4903-89c1-56cd429d70a7: ref 0 aborting False
>Thread-15913619::INFO::2013-11-20 03:09:39,665::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913620::DEBUG::2013-11-20 03:09:39,671::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913620::INFO::2013-11-20 03:09:39,672::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913620::DEBUG::2013-11-20 03:09:39,673::task::495::TaskManager.Task::(_debug) Task d095dd83-fc2a-4f24-8656-dd81ffdaa709: moving from state init -> state preparing
>Thread-15913620::DEBUG::2013-11-20 03:09:39,676::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`33ffd04e-d14c-4b27-b67e-15d05dbf2128`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913620::DEBUG::2013-11-20 03:09:39,690::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913620::DEBUG::2013-11-20 03:09:39,691::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913620::DEBUG::2013-11-20 03:09:39,692::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`33ffd04e-d14c-4b27-b67e-15d05dbf2128`::Granted request
>Thread-15913620::DEBUG::2013-11-20 03:09:39,694::task::495::TaskManager.Task::(_debug) Task d095dd83-fc2a-4f24-8656-dd81ffdaa709: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913620::DEBUG::2013-11-20 03:09:39,694::task::495::TaskManager.Task::(_debug) Task d095dd83-fc2a-4f24-8656-dd81ffdaa709: ref 1 aborting False
>Thread-15913620::WARNING::2013-11-20 03:09:39,696::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913620::DEBUG::2013-11-20 03:09:39,697::task::495::TaskManager.Task::(_debug) Task d095dd83-fc2a-4f24-8656-dd81ffdaa709: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913620::DEBUG::2013-11-20 03:09:39,698::task::495::TaskManager.Task::(_debug) Task d095dd83-fc2a-4f24-8656-dd81ffdaa709: moving from state preparing -> state finished
>Thread-15913620::DEBUG::2013-11-20 03:09:39,699::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913620::DEBUG::2013-11-20 03:09:39,699::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913620::DEBUG::2013-11-20 03:09:39,700::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913620::DEBUG::2013-11-20 03:09:39,701::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913620::DEBUG::2013-11-20 03:09:39,701::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913620::DEBUG::2013-11-20 03:09:39,702::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913620::DEBUG::2013-11-20 03:09:39,703::task::495::TaskManager.Task::(_debug) Task d095dd83-fc2a-4f24-8656-dd81ffdaa709: ref 0 aborting False
>Thread-15913620::INFO::2013-11-20 03:09:39,703::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:09:39,879::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:09:40,463::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0765378 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:09:42,479::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:09:43,132::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0939918 s, 10.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:09:45,146::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:09:45,764::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0875845 s, 11.7 MB/s\n'; <rc> = 0
>Thread-15913624::DEBUG::2013-11-20 03:09:47,006::clientIF::54::vds::(wrapper) [192.168.100.135]::call create with ({'bridge': 'rhevm', 'acpiEnable': 'true', 'emulatedMachine': 'rhel6.2.0', 'vmId': '4760db3e-d3a8-4045-9153-12c36ec16ba2', 'transparentHugePages': 'true', 'spiceSslCipherSuite': 'DEFAULT', 'cpuType': 'Westmere', 'smp': '4', 'macAddr': '00:1a:4a:a8:64:04', 'boot': 'cd', 'custom': {}, 'vmType': 'kvm', 'memSize': 4096, 'smpCoresPerSocket': '2', 'vmName': 'KLEE-AOS01', 'spiceMonitors': '1', 'nice': '0', 'drives': [{'index': '0', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'format': 'raw', 'boot': 'true', 'volumeID': '5eaa8dde-10d8-45b5-9e10-5417c297c4be', 'imageID': '6f97dfa5-ee35-4f37-8495-1e6b3136b92c', 'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'propagateErrors': 'off', 'if': 'virtio'}], 'smartcardEnable': 'false', 'nicModel': 'pv', 'keyboardLayout': 'en-us', 'kvmEnable': 'true', 'timeOffset': '-1', 'spiceSecureChannels': 'smain,sinputs', 'display': 'qxl'},) {}
>Thread-15913624::INFO::2013-11-20 03:09:47,413::clientIF::523::vds::(_getNetworkIp) network None: using 0
>Thread-15913624::INFO::2013-11-20 03:09:47,415::clientIF::656::vds::(create) vmContainerLock acquired by vm 4760db3e-d3a8-4045-9153-12c36ec16ba2
>Thread-15913625::DEBUG::2013-11-20 03:09:47,420::vm::342::vm.Vm::(_startUnderlyingVm) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Start
>Thread-15913624::DEBUG::2013-11-20 03:09:47,421::clientIF::669::vds::(create) Total desktops after creation of 4760db3e-d3a8-4045-9153-12c36ec16ba2 is 4
>Thread-15913625::DEBUG::2013-11-20 03:09:47,422::vm::346::vm.Vm::(_startUnderlyingVm) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::_ongoingCreations acquired
>Thread-15913624::DEBUG::2013-11-20 03:09:47,423::clientIF::59::vds::(wrapper) return create with {'status': {'message': 'Done', 'code': 0}, 'vmList': {'status': 'WaitForLaunch', 'bridge': 'rhevm', 'acpiEnable': 'true', 'emulatedMachine': 'rhel6.2.0', 'vmId': '4760db3e-d3a8-4045-9153-12c36ec16ba2', 'pid': '0', 'drives': [{'index': '0', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'format': 'raw', 'boot': 'true', 'volumeID': '5eaa8dde-10d8-45b5-9e10-5417c297c4be', 'imageID': '6f97dfa5-ee35-4f37-8495-1e6b3136b92c', 'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'propagateErrors': 'off', 'if': 'virtio'}], 'transparentHugePages': 'true', 'keyboardLayout': 'en-us', 'displayPort': '-1', 'displaySecurePort': '-1', 'spiceSslCipherSuite': 'DEFAULT', 'cpuType': 'Westmere', 'custom': {}, 'clientIp': '', 'nicModel': 'pv', 'macAddr': '00:1a:4a:a8:64:04', 'kvmEnable': 'true', 'boot': 'cd', 'smp': '4', 'vmType': 'kvm', 'smartcardEnable': 'false', 'timeOffset': '-1', 'memSize': 4096, 'displayIp': '0', 'spiceSecureChannels': 'smain,sinputs', 'smpCoresPerSocket': '2', 'vmName': 'KLEE-AOS01', 'display': 'qxl', 'spiceMonitors': '1', 'nice': '0'}}
>Thread-15913625::INFO::2013-11-20 03:09:47,423::libvirtvm::1061::vm.Vm::(_run) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::VM wrapper has started
>Thread-15913625::INFO::2013-11-20 03:09:47,428::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913625::DEBUG::2013-11-20 03:09:47,429::task::495::TaskManager.Task::(_debug) Task 7482bed7-f7a2-4f94-bfe4-19f216683321: moving from state init -> state preparing
>Thread-15913625::DEBUG::2013-11-20 03:09:47,431::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c28c64be-0174-4eb4-998f-d7dced5d5899`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1690' at 'public_prepareVolume'
>Thread-15913625::DEBUG::2013-11-20 03:09:47,432::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913625::DEBUG::2013-11-20 03:09:47,433::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913625::DEBUG::2013-11-20 03:09:47,433::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c28c64be-0174-4eb4-998f-d7dced5d5899`::Granted request
>Thread-15913625::DEBUG::2013-11-20 03:09:47,434::task::495::TaskManager.Task::(_debug) Task 7482bed7-f7a2-4f94-bfe4-19f216683321: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913625::DEBUG::2013-11-20 03:09:47,435::task::495::TaskManager.Task::(_debug) Task 7482bed7-f7a2-4f94-bfe4-19f216683321: ref 1 aborting False
>Thread-15913625::DEBUG::2013-11-20 03:09:47,436::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.6f97dfa5-ee35-4f37-8495-1e6b3136b92c`ReqID=`ead00eb7-045d-4261-80e9-e67860305216`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1695' at 'public_prepareVolume'
>Thread-15913625::DEBUG::2013-11-20 03:09:47,437::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.6f97dfa5-ee35-4f37-8495-1e6b3136b92c' for lock type 'exclusive'
>Thread-15913625::DEBUG::2013-11-20 03:09:47,439::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913625::DEBUG::2013-11-20 03:09:47,440::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:09:47,780::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913547::DEBUG::2013-11-20 03:09:48,034::guestIF::97::vm.Vm::(_handleMessage) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::u'None'
>Thread-15913625::DEBUG::2013-11-20 03:09:48,423::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:09:48,521::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0763715 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15913625::DEBUG::2013-11-20 03:09:48,758::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913625::DEBUG::2013-11-20 03:09:48,766::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=9 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15913625::DEBUG::2013-11-20 03:09:49,198::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000230876 s, 2.2 MB/s\n'; <rc> = 0
>Thread-15913625::DEBUG::2013-11-20 03:09:49,200::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000230876 s, 2.2 MB/s'], size: 512
>Thread-15913625::DEBUG::2013-11-20 03:09:49,216::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=9 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15913625::DEBUG::2013-11-20 03:09:49,632::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000243174 s, 2.1 MB/s\n'; <rc> = 0
>Thread-15913625::DEBUG::2013-11-20 03:09:49,633::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000243174 s, 2.1 MB/s'], size: 512
>Thread-15913625::INFO::2013-11-20 03:09:49,634::image::340::Storage.Image::(getChain) sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c chain=[<storage.blockVolume.BlockVolume instance at 0x7f487c1037e8>] 
>Thread-15913625::DEBUG::2013-11-20 03:09:49,635::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --autobackup n --available y 6b892067-ac7e-4039-a7be-5b7387525b16/5eaa8dde-10d8-45b5-9e10-5417c297c4be' (cwd None)
>Thread-15913628::DEBUG::2013-11-20 03:09:50,028::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913628::INFO::2013-11-20 03:09:50,030::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913628::DEBUG::2013-11-20 03:09:50,030::task::495::TaskManager.Task::(_debug) Task 86700ddf-a639-47de-9df5-47adaa38ecb7: moving from state init -> state preparing
>Thread-15913628::DEBUG::2013-11-20 03:09:50,031::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913628::DEBUG::2013-11-20 03:09:50,031::task::495::TaskManager.Task::(_debug) Task 86700ddf-a639-47de-9df5-47adaa38ecb7: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913628::DEBUG::2013-11-20 03:09:50,032::task::495::TaskManager.Task::(_debug) Task 86700ddf-a639-47de-9df5-47adaa38ecb7: moving from state preparing -> state finished
>Thread-15913628::DEBUG::2013-11-20 03:09:50,032::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913628::DEBUG::2013-11-20 03:09:50,032::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913628::DEBUG::2013-11-20 03:09:50,038::task::495::TaskManager.Task::(_debug) Task 86700ddf-a639-47de-9df5-47adaa38ecb7: ref 0 aborting False
>Thread-15913628::INFO::2013-11-20 03:09:50,039::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913629::DEBUG::2013-11-20 03:09:50,042::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913629::INFO::2013-11-20 03:09:50,043::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913629::DEBUG::2013-11-20 03:09:50,043::task::495::TaskManager.Task::(_debug) Task 30c4a7d4-7a3b-4431-b497-4def5c1ee50e: moving from state init -> state preparing
>Thread-15913629::DEBUG::2013-11-20 03:09:50,044::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`cb168538-a85b-4ed0-9940-4e6201e2fa50`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913629::DEBUG::2013-11-20 03:09:50,045::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913629::DEBUG::2013-11-20 03:09:50,045::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913629::DEBUG::2013-11-20 03:09:50,046::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`cb168538-a85b-4ed0-9940-4e6201e2fa50`::Granted request
>Thread-15913629::DEBUG::2013-11-20 03:09:50,046::task::495::TaskManager.Task::(_debug) Task 30c4a7d4-7a3b-4431-b497-4def5c1ee50e: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913629::DEBUG::2013-11-20 03:09:50,047::task::495::TaskManager.Task::(_debug) Task 30c4a7d4-7a3b-4431-b497-4def5c1ee50e: ref 1 aborting False
>Thread-15913629::WARNING::2013-11-20 03:09:50,048::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913629::DEBUG::2013-11-20 03:09:50,049::task::495::TaskManager.Task::(_debug) Task 30c4a7d4-7a3b-4431-b497-4def5c1ee50e: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913629::DEBUG::2013-11-20 03:09:50,049::task::495::TaskManager.Task::(_debug) Task 30c4a7d4-7a3b-4431-b497-4def5c1ee50e: moving from state preparing -> state finished
>Thread-15913629::DEBUG::2013-11-20 03:09:50,050::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913629::DEBUG::2013-11-20 03:09:50,050::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913629::DEBUG::2013-11-20 03:09:50,051::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913629::DEBUG::2013-11-20 03:09:50,051::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913629::DEBUG::2013-11-20 03:09:50,052::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913629::DEBUG::2013-11-20 03:09:50,052::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913629::DEBUG::2013-11-20 03:09:50,053::task::495::TaskManager.Task::(_debug) Task 30c4a7d4-7a3b-4431-b497-4def5c1ee50e: ref 0 aborting False
>Thread-15913629::INFO::2013-11-20 03:09:50,054::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913519::DEBUG::2013-11-20 03:09:50,497::guestIF::97::vm.Vm::(_handleMessage) vmId=`4b8ed9c8-dfb1-4de4-935d-6ba4009567b9`::u'None'
>Thread-15913625::DEBUG::2013-11-20 03:09:50,591::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913625::DEBUG::2013-11-20 03:09:50,592::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913625::DEBUG::2013-11-20 03:09:50,593::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913625::DEBUG::2013-11-20 03:09:50,594::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.5eaa8dde-10d8-45b5-9e10-5417c297c4be`ReqID=`50af7a47-c340-4ae0-a99a-a7d1fb2af5cf`::Request was made in '/usr/share/vdsm/storage/resourceFactories.py' line '149' at '__getResourceCandidatesList'
>Thread-15913625::DEBUG::2013-11-20 03:09:50,595::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.5eaa8dde-10d8-45b5-9e10-5417c297c4be' for lock type 'exclusive'
>Thread-15913625::DEBUG::2013-11-20 03:09:50,595::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.5eaa8dde-10d8-45b5-9e10-5417c297c4be' is free. Now locking as 'exclusive' (1 active user)
>Thread-15913625::DEBUG::2013-11-20 03:09:50,596::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.5eaa8dde-10d8-45b5-9e10-5417c297c4be`ReqID=`50af7a47-c340-4ae0-a99a-a7d1fb2af5cf`::Granted request
>Thread-15913625::DEBUG::2013-11-20 03:09:50,603::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.6f97dfa5-ee35-4f37-8495-1e6b3136b92c' is free. Now locking as 'exclusive' (1 active user)
>Thread-15913625::DEBUG::2013-11-20 03:09:50,603::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.6f97dfa5-ee35-4f37-8495-1e6b3136b92c`ReqID=`ead00eb7-045d-4261-80e9-e67860305216`::Granted request
>Thread-15913625::DEBUG::2013-11-20 03:09:50,604::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913625::DEBUG::2013-11-20 03:09:50,605::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:09:50,767::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913630::DEBUG::2013-11-20 03:09:51,140::clientIF::54::vds::(wrapper) [192.168.100.135]::call create with ({'bridge': 'rhevm', 'acpiEnable': 'true', 'emulatedMachine': 'rhel6.2.0', 'vmId': '0dc7e997-e884-4493-b701-790fb37b3d80', 'transparentHugePages': 'true', 'spiceSslCipherSuite': 'DEFAULT', 'cpuType': 'Westmere', 'smp': '4', 'macAddr': '00:1a:4a:a8:64:03', 'boot': 'cd', 'custom': {}, 'vmType': 'kvm', 'memSize': 4096, 'smpCoresPerSocket': '2', 'vmName': 'KLEE-AOS02', 'spiceMonitors': '1', 'nice': '0', 'drives': [{'index': '0', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'format': 'raw', 'boot': 'true', 'volumeID': '8506c15b-61ae-4830-84ed-c830955c25a3', 'imageID': 'b74dd33b-748d-48df-83f2-ebcc3523e117', 'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'propagateErrors': 'off', 'if': 'virtio'}], 'smartcardEnable': 'false', 'nicModel': 'pv', 'keyboardLayout': 'en-us', 'kvmEnable': 'true', 'timeOffset': '3600', 'spiceSecureChannels': 'smain,sinputs', 'display': 'qxl'},) {}
>Thread-15913625::DEBUG::2013-11-20 03:09:51,586::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913625::DEBUG::2013-11-20 03:09:51,902::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913630::INFO::2013-11-20 03:09:51,904::clientIF::523::vds::(_getNetworkIp) network None: using 0
>Thread-15913630::INFO::2013-11-20 03:09:51,906::clientIF::656::vds::(create) vmContainerLock acquired by vm 0dc7e997-e884-4493-b701-790fb37b3d80
>Thread-15913625::DEBUG::2013-11-20 03:09:51,907::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --refresh 6b892067-ac7e-4039-a7be-5b7387525b16/5eaa8dde-10d8-45b5-9e10-5417c297c4be' (cwd None)
>Thread-15913630::DEBUG::2013-11-20 03:09:52,150::clientIF::669::vds::(create) Total desktops after creation of 0dc7e997-e884-4493-b701-790fb37b3d80 is 5
>Thread-15913631::DEBUG::2013-11-20 03:09:52,151::vm::342::vm.Vm::(_startUnderlyingVm) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::Start
>Thread-15913630::DEBUG::2013-11-20 03:09:52,155::clientIF::59::vds::(wrapper) return create with {'status': {'message': 'Done', 'code': 0}, 'vmList': {'status': 'WaitForLaunch', 'bridge': 'rhevm', 'acpiEnable': 'true', 'emulatedMachine': 'rhel6.2.0', 'vmId': '0dc7e997-e884-4493-b701-790fb37b3d80', 'pid': '0', 'drives': [{'index': '0', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'format': 'raw', 'boot': 'true', 'volumeID': '8506c15b-61ae-4830-84ed-c830955c25a3', 'imageID': 'b74dd33b-748d-48df-83f2-ebcc3523e117', 'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'propagateErrors': 'off', 'if': 'virtio'}], 'transparentHugePages': 'true', 'keyboardLayout': 'en-us', 'displayPort': '-1', 'displaySecurePort': '-1', 'spiceSslCipherSuite': 'DEFAULT', 'cpuType': 'Westmere', 'custom': {}, 'clientIp': '', 'nicModel': 'pv', 'macAddr': '00:1a:4a:a8:64:03', 'kvmEnable': 'true', 'boot': 'cd', 'smp': '4', 'vmType': 'kvm', 'smartcardEnable': 'false', 'timeOffset': '3600', 'memSize': 4096, 'displayIp': '0', 'spiceSecureChannels': 'smain,sinputs', 'smpCoresPerSocket': '2', 'vmName': 'KLEE-AOS02', 'display': 'qxl', 'spiceMonitors': '1', 'nice': '0'}}
>Dummy-15912651::DEBUG::2013-11-20 03:09:52,308::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.77864 s, 1.3 MB/s\n'; <rc> = 0
>Thread-15913625::DEBUG::2013-11-20 03:09:52,697::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913625::DEBUG::2013-11-20 03:09:52,699::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913625::DEBUG::2013-11-20 03:09:52,699::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913625::INFO::2013-11-20 03:09:52,700::volume::517::Storage.Volume::(prepare) Volume: preparing volume 6b892067-ac7e-4039-a7be-5b7387525b16/5eaa8dde-10d8-45b5-9e10-5417c297c4be
>Thread-15913625::DEBUG::2013-11-20 03:09:52,701::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913625::DEBUG::2013-11-20 03:09:52,702::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15913546::INFO::2013-11-20 03:09:52,952::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:09:52,954::task::495::TaskManager.Task::(_debug) Task 0274b38a-476b-4ef3-aa65-43536912c356: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:09:52,956::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`331ec034-5368-4093-8506-21ba2ffbd5a7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:09:52,957::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:09:52,958::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:09:52,958::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`331ec034-5368-4093-8506-21ba2ffbd5a7`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:09:52,959::task::495::TaskManager.Task::(_debug) Task 0274b38a-476b-4ef3-aa65-43536912c356: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:09:52,960::task::495::TaskManager.Task::(_debug) Task 0274b38a-476b-4ef3-aa65-43536912c356: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:09:52,961::task::495::TaskManager.Task::(_debug) Task 0274b38a-476b-4ef3-aa65-43536912c356: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:09:52,962::task::495::TaskManager.Task::(_debug) Task 0274b38a-476b-4ef3-aa65-43536912c356: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:09:52,962::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:09:52,963::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:09:52,963::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:09:52,964::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:09:52,965::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:09:52,965::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:09:52,966::task::495::TaskManager.Task::(_debug) Task 0274b38a-476b-4ef3-aa65-43536912c356: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:09:52,967::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:09:52,967::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:09:52,968::task::495::TaskManager.Task::(_debug) Task 9d42237d-b171-4295-a663-42a68fbe1204: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:09:52,969::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fb41bbba-060d-43a9-9a77-45cf93a19d34`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:09:52,970::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:09:52,970::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:09:52,971::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fb41bbba-060d-43a9-9a77-45cf93a19d34`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:09:52,972::task::495::TaskManager.Task::(_debug) Task 9d42237d-b171-4295-a663-42a68fbe1204: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:09:52,973::task::495::TaskManager.Task::(_debug) Task 9d42237d-b171-4295-a663-42a68fbe1204: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:09:52,974::task::495::TaskManager.Task::(_debug) Task 9d42237d-b171-4295-a663-42a68fbe1204: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:09:52,974::task::495::TaskManager.Task::(_debug) Task 9d42237d-b171-4295-a663-42a68fbe1204: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:09:52,975::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:09:52,975::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:09:52,976::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:09:52,977::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:09:52,978::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:09:52,978::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:09:52,979::task::495::TaskManager.Task::(_debug) Task 9d42237d-b171-4295-a663-42a68fbe1204: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:09:52,979::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:09:52,980::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:09:52,981::task::495::TaskManager.Task::(_debug) Task 0223cbdf-f913-4341-a8ad-f5659e6f9189: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:09:52,982::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`289c1371-187e-485e-a939-d1d6ae56af8f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:09:52,983::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:09:52,984::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:09:52,984::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`289c1371-187e-485e-a939-d1d6ae56af8f`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:09:52,985::task::495::TaskManager.Task::(_debug) Task 0223cbdf-f913-4341-a8ad-f5659e6f9189: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:09:52,986::task::495::TaskManager.Task::(_debug) Task 0223cbdf-f913-4341-a8ad-f5659e6f9189: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:09:52,987::task::495::TaskManager.Task::(_debug) Task 0223cbdf-f913-4341-a8ad-f5659e6f9189: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:09:52,987::task::495::TaskManager.Task::(_debug) Task 0223cbdf-f913-4341-a8ad-f5659e6f9189: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:09:52,988::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:09:52,988::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:09:52,989::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:09:52,990::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:09:52,990::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:09:52,991::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:09:52,992::task::495::TaskManager.Task::(_debug) Task 0223cbdf-f913-4341-a8ad-f5659e6f9189: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:09:52,992::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:09:52,993::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:09:52,994::task::495::TaskManager.Task::(_debug) Task a74eef09-766d-472a-989c-09596a61cdb8: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:09:52,995::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`50299f1a-cd8a-40f0-8705-9d81c24f5a37`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:09:52,995::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:09:52,996::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:09:52,996::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`50299f1a-cd8a-40f0-8705-9d81c24f5a37`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:09:52,997::task::495::TaskManager.Task::(_debug) Task a74eef09-766d-472a-989c-09596a61cdb8: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:09:52,998::task::495::TaskManager.Task::(_debug) Task a74eef09-766d-472a-989c-09596a61cdb8: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:09:52,999::task::495::TaskManager.Task::(_debug) Task a74eef09-766d-472a-989c-09596a61cdb8: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:09:52,999::task::495::TaskManager.Task::(_debug) Task a74eef09-766d-472a-989c-09596a61cdb8: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:09:53,000::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:09:53,000::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:09:53,001::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:09:53,002::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:09:53,002::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:09:53,003::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:09:53,003::task::495::TaskManager.Task::(_debug) Task a74eef09-766d-472a-989c-09596a61cdb8: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:09:53,004::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913625::DEBUG::2013-11-20 03:09:53,617::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913625::DEBUG::2013-11-20 03:09:53,903::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913625::DEBUG::2013-11-20 03:09:53,905::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=9 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15913634::DEBUG::2013-11-20 03:09:54,090::clientIF::54::vds::(wrapper) [192.168.100.135]::call create with ({'bridge': 'rhevm', 'acpiEnable': 'true', 'emulatedMachine': 'rhel6.2.0', 'vmId': '11b5c641-e4f7-487e-9758-c42cb014e2d2', 'transparentHugePages': 'true', 'spiceSslCipherSuite': 'DEFAULT', 'cpuType': 'Westmere', 'smp': '2', 'macAddr': '00:1a:4a:a8:64:0d', 'boot': 'd', 'custom': {}, 'vmType': 'kvm', 'memSize': 2048, 'smpCoresPerSocket': '1', 'vmName': 'KLEE-Print01', 'spiceMonitors': '1', 'nice': '0', 'drives': [{'index': '0', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'format': 'cow', 'boot': 'true', 'volumeID': 'ff5e3845-0d0d-4037-a0de-a3f584a4aef5', 'imageID': 'cc863eeb-33db-4227-b4eb-9f4347299cf3', 'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'propagateErrors': 'off', 'if': 'virtio'}], 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO', 'smartcardEnable': 'false', 'nicModel': 'pv', 'keyboardLayout': 'en-us', 'kvmEnable': 'true', 'timeOffset': '-1', 'spiceSecureChannels': 'smain,sinputs', 'display': 'qxl'},) {}
>Dummy-15912651::DEBUG::2013-11-20 03:09:54,354::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913625::DEBUG::2013-11-20 03:09:54,670::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000185182 s, 2.8 MB/s\n'; <rc> = 0
>Thread-15913634::INFO::2013-11-20 03:09:54,673::clientIF::523::vds::(_getNetworkIp) network None: using 0
>Thread-15913634::INFO::2013-11-20 03:09:54,674::clientIF::656::vds::(create) vmContainerLock acquired by vm 11b5c641-e4f7-487e-9758-c42cb014e2d2
>Thread-15913625::DEBUG::2013-11-20 03:09:54,675::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000185182 s, 2.8 MB/s'], size: 512
>Thread-15913625::DEBUG::2013-11-20 03:09:54,679::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=9 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15913635::DEBUG::2013-11-20 03:09:54,931::vm::342::vm.Vm::(_startUnderlyingVm) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::Start
>Thread-15913634::DEBUG::2013-11-20 03:09:54,941::clientIF::669::vds::(create) Total desktops after creation of 11b5c641-e4f7-487e-9758-c42cb014e2d2 is 6
>Thread-15913634::DEBUG::2013-11-20 03:09:54,951::clientIF::59::vds::(wrapper) return create with {'status': {'message': 'Done', 'code': 0}, 'vmList': {'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO', 'bridge': 'rhevm', 'acpiEnable': 'true', 'emulatedMachine': 'rhel6.2.0', 'vmId': '11b5c641-e4f7-487e-9758-c42cb014e2d2', 'pid': '0', 'drives': [{'index': '0', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'format': 'cow', 'boot': 'true', 'volumeID': 'ff5e3845-0d0d-4037-a0de-a3f584a4aef5', 'imageID': 'cc863eeb-33db-4227-b4eb-9f4347299cf3', 'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'propagateErrors': 'off', 'if': 'virtio'}], 'transparentHugePages': 'true', 'keyboardLayout': 'en-us', 'displaySecurePort': '-1', 'displayPort': '-1', 'cpuType': 'Westmere', 'custom': {}, 'clientIp': '', 'nicModel': 'pv', 'macAddr': '00:1a:4a:a8:64:0d', 'kvmEnable': 'true', 'smartcardEnable': 'false', 'boot': 'd', 'smp': '2', 'vmType': 'kvm', 'spiceSslCipherSuite': 'DEFAULT', 'status': 'WaitForLaunch', 'timeOffset': '-1', 'memSize': 2048, 'displayIp': '0', 'spiceSecureChannels': 'smain,sinputs', 'smpCoresPerSocket': '1', 'vmName': 'KLEE-Print01', 'display': 'qxl', 'spiceMonitors': '1', 'nice': '0'}}
>Dummy-15912651::DEBUG::2013-11-20 03:09:55,068::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0982567 s, 10.4 MB/s\n'; <rc> = 0
>Thread-15913625::DEBUG::2013-11-20 03:09:55,103::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000181294 s, 2.8 MB/s\n'; <rc> = 0
>Thread-15913625::DEBUG::2013-11-20 03:09:55,105::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000181294 s, 2.8 MB/s'], size: 512
>Thread-15913625::DEBUG::2013-11-20 03:09:55,107::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.5eaa8dde-10d8-45b5-9e10-5417c297c4be`ReqID=`d6b9aa42-6636-4b0f-bdd2-5fcf16dbac6b`::Request was made in '/usr/share/vdsm/storage/volume.py' line '533' at 'prepare'
>Thread-15913625::DEBUG::2013-11-20 03:09:55,107::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.5eaa8dde-10d8-45b5-9e10-5417c297c4be' for lock type 'exclusive'
>Thread-15913625::DEBUG::2013-11-20 03:09:55,109::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.5eaa8dde-10d8-45b5-9e10-5417c297c4be' is free. Now locking as 'exclusive' (1 active user)
>Thread-15913625::DEBUG::2013-11-20 03:09:55,109::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.5eaa8dde-10d8-45b5-9e10-5417c297c4be`ReqID=`d6b9aa42-6636-4b0f-bdd2-5fcf16dbac6b`::Granted request
>Thread-15913625::DEBUG::2013-11-20 03:09:55,111::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=9 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15913637::INFO::2013-11-20 03:09:55,388::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913637::DEBUG::2013-11-20 03:09:55,390::task::495::TaskManager.Task::(_debug) Task 8e52d9a3-3fb5-48a9-bc3d-b06a3f533ae9: moving from state init -> state preparing
>Thread-15913637::DEBUG::2013-11-20 03:09:55,391::task::495::TaskManager.Task::(_debug) Task 8e52d9a3-3fb5-48a9-bc3d-b06a3f533ae9: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000442028045654', 'lastCheck': 1384916991.1371789, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00122809410095', 'lastCheck': 1384916988.2263241, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00136303901672', 'lastCheck': 1384916989.43997, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000420808792114', 'lastCheck': 1384916990.4352801, 'valid': True, 'code': 0}}
>Thread-15913637::DEBUG::2013-11-20 03:09:55,392::task::495::TaskManager.Task::(_debug) Task 8e52d9a3-3fb5-48a9-bc3d-b06a3f533ae9: moving from state preparing -> state finished
>Thread-15913637::DEBUG::2013-11-20 03:09:55,393::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913637::DEBUG::2013-11-20 03:09:55,393::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913637::DEBUG::2013-11-20 03:09:55,394::task::495::TaskManager.Task::(_debug) Task 8e52d9a3-3fb5-48a9-bc3d-b06a3f533ae9: ref 0 aborting False
>Thread-15913637::INFO::2013-11-20 03:09:55,394::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000442028045654', 'lastCheck': 1384916991.1371789, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00122809410095', 'lastCheck': 1384916988.2263241, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00136303901672', 'lastCheck': 1384916989.43997, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000420808792114', 'lastCheck': 1384916990.4352801, 'valid': True, 'code': 0}}
>Thread-15913625::DEBUG::2013-11-20 03:09:55,601::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.00018082 s, 2.8 MB/s\n'; <rc> = 0
>Thread-15913625::DEBUG::2013-11-20 03:09:55,602::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.00018082 s, 2.8 MB/s'], size: 512
>Thread-15913625::DEBUG::2013-11-20 03:09:55,604::task::495::TaskManager.Task::(_debug) Task 7482bed7-f7a2-4f94-bfe4-19f216683321: finished: None
>Thread-15913625::DEBUG::2013-11-20 03:09:55,604::task::495::TaskManager.Task::(_debug) Task 7482bed7-f7a2-4f94-bfe4-19f216683321: moving from state preparing -> state finished
>Thread-15913625::DEBUG::2013-11-20 03:09:55,605::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913625::DEBUG::2013-11-20 03:09:55,606::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913625::DEBUG::2013-11-20 03:09:55,606::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913625::DEBUG::2013-11-20 03:09:55,607::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913625::DEBUG::2013-11-20 03:09:55,608::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913625::DEBUG::2013-11-20 03:09:55,608::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913625::DEBUG::2013-11-20 03:09:55,609::task::495::TaskManager.Task::(_debug) Task 7482bed7-f7a2-4f94-bfe4-19f216683321: ref 0 aborting False
>Thread-15913625::INFO::2013-11-20 03:09:55,610::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, Return response: {'status': {'message': 'OK', 'code': 0}}
>Thread-15913625::INFO::2013-11-20 03:09:55,611::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913625::DEBUG::2013-11-20 03:09:55,611::task::495::TaskManager.Task::(_debug) Task c64c0f5e-fec3-48d9-a7dd-5efaf46e4f36: moving from state init -> state preparing
>Thread-15913625::DEBUG::2013-11-20 03:09:55,613::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f3baf50a-4ecc-413b-b377-0454eb1bc246`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1666' at 'public_getVolumePath'
>Thread-15913625::DEBUG::2013-11-20 03:09:55,613::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913625::DEBUG::2013-11-20 03:09:55,614::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913625::DEBUG::2013-11-20 03:09:55,615::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f3baf50a-4ecc-413b-b377-0454eb1bc246`::Granted request
>Thread-15913625::DEBUG::2013-11-20 03:09:55,615::task::495::TaskManager.Task::(_debug) Task c64c0f5e-fec3-48d9-a7dd-5efaf46e4f36: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913625::DEBUG::2013-11-20 03:09:55,616::task::495::TaskManager.Task::(_debug) Task c64c0f5e-fec3-48d9-a7dd-5efaf46e4f36: ref 1 aborting False
>Thread-15913625::DEBUG::2013-11-20 03:09:55,617::task::495::TaskManager.Task::(_debug) Task c64c0f5e-fec3-48d9-a7dd-5efaf46e4f36: finished: {'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/6f97dfa5-ee35-4f37-8495-1e6b3136b92c/5eaa8dde-10d8-45b5-9e10-5417c297c4be'}
>Thread-15913625::DEBUG::2013-11-20 03:09:55,618::task::495::TaskManager.Task::(_debug) Task c64c0f5e-fec3-48d9-a7dd-5efaf46e4f36: moving from state preparing -> state finished
>Thread-15913625::DEBUG::2013-11-20 03:09:55,618::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913625::DEBUG::2013-11-20 03:09:55,619::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913625::DEBUG::2013-11-20 03:09:55,620::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913625::DEBUG::2013-11-20 03:09:55,620::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913625::DEBUG::2013-11-20 03:09:55,621::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913625::DEBUG::2013-11-20 03:09:55,621::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913625::DEBUG::2013-11-20 03:09:55,622::task::495::TaskManager.Task::(_debug) Task c64c0f5e-fec3-48d9-a7dd-5efaf46e4f36: ref 0 aborting False
>Thread-15913625::INFO::2013-11-20 03:09:55,623::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, Return response: {'status': {'message': 'OK', 'code': 0}, 'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/6f97dfa5-ee35-4f37-8495-1e6b3136b92c/5eaa8dde-10d8-45b5-9e10-5417c297c4be'}
>Thread-15913625::INFO::2013-11-20 03:09:55,623::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913625::DEBUG::2013-11-20 03:09:55,624::task::495::TaskManager.Task::(_debug) Task 30e6a4a0-966d-4155-adfb-4bfef8aadde3: moving from state init -> state preparing
>Thread-15913625::DEBUG::2013-11-20 03:09:55,625::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`dcb1f46d-8c60-4a6e-b371-2e9c57c3efd1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913625::DEBUG::2013-11-20 03:09:55,626::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913625::DEBUG::2013-11-20 03:09:55,626::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913625::DEBUG::2013-11-20 03:09:55,627::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`dcb1f46d-8c60-4a6e-b371-2e9c57c3efd1`::Granted request
>Thread-15913625::DEBUG::2013-11-20 03:09:55,628::task::495::TaskManager.Task::(_debug) Task 30e6a4a0-966d-4155-adfb-4bfef8aadde3: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913625::DEBUG::2013-11-20 03:09:55,628::task::495::TaskManager.Task::(_debug) Task 30e6a4a0-966d-4155-adfb-4bfef8aadde3: ref 1 aborting False
>Thread-15913625::DEBUG::2013-11-20 03:09:55,629::task::495::TaskManager.Task::(_debug) Task 30e6a4a0-966d-4155-adfb-4bfef8aadde3: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913625::DEBUG::2013-11-20 03:09:55,630::task::495::TaskManager.Task::(_debug) Task 30e6a4a0-966d-4155-adfb-4bfef8aadde3: moving from state preparing -> state finished
>Thread-15913625::DEBUG::2013-11-20 03:09:55,630::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913625::DEBUG::2013-11-20 03:09:55,631::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913625::DEBUG::2013-11-20 03:09:55,632::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913625::DEBUG::2013-11-20 03:09:55,632::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913625::DEBUG::2013-11-20 03:09:55,633::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913625::DEBUG::2013-11-20 03:09:55,633::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913625::DEBUG::2013-11-20 03:09:55,634::task::495::TaskManager.Task::(_debug) Task 30e6a4a0-966d-4155-adfb-4bfef8aadde3: ref 0 aborting False
>Thread-15913625::INFO::2013-11-20 03:09:55,634::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913625::INFO::2013-11-20 03:09:55,635::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16)
>Thread-15913625::DEBUG::2013-11-20 03:09:55,636::task::495::TaskManager.Task::(_debug) Task c8bac55b-a828-488b-ac81-2452f484433f: moving from state init -> state preparing
>Thread-15913625::INFO::2013-11-20 03:09:55,636::blockSD::674::Storage.StorageDomain::(validate) sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16
>Thread-15913625::DEBUG::2013-11-20 03:09:55,637::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913625::DEBUG::2013-11-20 03:09:55,638::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913625::DEBUG::2013-11-20 03:09:55,639::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913625::DEBUG::2013-11-20 03:09:55,640::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913625::DEBUG::2013-11-20 03:09:55,641::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913625::DEBUG::2013-11-20 03:09:55,642::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15913625::DEBUG::2013-11-20 03:09:56,349::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913625::DEBUG::2013-11-20 03:09:56,353::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913625::DEBUG::2013-11-20 03:09:56,357::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'_SHA_CKSUM=716ea7492d9134a4b13de27ac61a336ad474727c', u'POOL_SPM_ID=2', u'POOL_SPM_LVER=287']
>Thread-15913625::DEBUG::2013-11-20 03:09:56,360::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`13713d99-cc7d-482a-97ac-58caad5f4ff3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1297' at 'public_getStorageDomainInfo'
>Thread-15913625::DEBUG::2013-11-20 03:09:56,361::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913625::DEBUG::2013-11-20 03:09:56,362::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913625::DEBUG::2013-11-20 03:09:56,362::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`13713d99-cc7d-482a-97ac-58caad5f4ff3`::Granted request
>Thread-15913625::DEBUG::2013-11-20 03:09:56,363::task::495::TaskManager.Task::(_debug) Task c8bac55b-a828-488b-ac81-2452f484433f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913625::DEBUG::2013-11-20 03:09:56,364::task::495::TaskManager.Task::(_debug) Task c8bac55b-a828-488b-ac81-2452f484433f: ref 1 aborting False
>Thread-15913625::WARNING::2013-11-20 03:09:56,366::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913625::DEBUG::2013-11-20 03:09:56,367::task::495::TaskManager.Task::(_debug) Task c8bac55b-a828-488b-ac81-2452f484433f: finished: {'info': {'uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'vguuid': 'aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', 'lver': 287, 'state': 'OK', 'version': '2', 'role': 'Master', 'master_ver': 1, 'spm_id': 2, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'OS'}}
>Thread-15913625::DEBUG::2013-11-20 03:09:56,372::task::495::TaskManager.Task::(_debug) Task c8bac55b-a828-488b-ac81-2452f484433f: moving from state preparing -> state finished
>Thread-15913625::DEBUG::2013-11-20 03:09:56,373::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913625::DEBUG::2013-11-20 03:09:56,374::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913625::DEBUG::2013-11-20 03:09:56,375::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913625::DEBUG::2013-11-20 03:09:56,375::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913625::DEBUG::2013-11-20 03:09:56,376::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913625::DEBUG::2013-11-20 03:09:56,377::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913625::DEBUG::2013-11-20 03:09:56,377::task::495::TaskManager.Task::(_debug) Task c8bac55b-a828-488b-ac81-2452f484433f: ref 0 aborting False
>Thread-15913625::INFO::2013-11-20 03:09:56,378::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'vguuid': 'aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', 'lver': 287, 'state': 'OK', 'version': '2', 'role': 'Master', 'master_ver': 1, 'spm_id': 2, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'OS'}}
>Thread-15913625::DEBUG::2013-11-20 03:09:56,398::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/libexec/vdsm/hooks/before_vm_start/50_vhostmd' (cwd None)
>Thread-15913625::DEBUG::2013-11-20 03:09:56,907::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913625::INFO::2013-11-20 03:09:56,908::hooks::51::root::(_runHooksDir) 
>Thread-15913625::DEBUG::2013-11-20 03:09:56,909::libvirtvm::1077::vm.Vm::(_run) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::<?xml version="1.0" encoding="utf-8"?>
><domain type="kvm">
>	<name>KLEE-AOS01</name>
>	<uuid>4760db3e-d3a8-4045-9153-12c36ec16ba2</uuid>
>	<memory>4194304</memory>
>	<currentMemory>4194304</currentMemory>
>	<vcpu>4</vcpu>
>	<devices>
>		<disk device="disk" type="block">
>			<source dev="/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/6f97dfa5-ee35-4f37-8495-1e6b3136b92c/5eaa8dde-10d8-45b5-9e10-5417c297c4be"/>
>			<target bus="virtio" dev="vda"/>
>			<serial>37-8495-1e6b3136b92c</serial>
>			<driver cache="none" error_policy="stop" io="native" name="qemu" type="raw"/>
>		</disk>
>		<disk device="cdrom" type="file">
>			<target bus="ide" dev="hdc"/>
>		</disk>
>		<controller index="0" ports="16" type="virtio-serial"/>
>		<channel type="unix">
>			<target name="com.redhat.rhevm.vdsm" type="virtio"/>
>			<source mode="bind" path="/var/lib/libvirt/qemu/channels/KLEE-AOS01.com.redhat.rhevm.vdsm"/>
>		</channel>
>		<memballoon model="none"/>
>		<interface type="bridge">
>			<mac address="00:1a:4a:a8:64:04"/>
>			<model type="virtio"/>
>			<source bridge="rhevm"/>
>		</interface>
>		<input bus="ps2" type="mouse"/>
>		<video>
>			<model heads="1" type="qxl" vram="65536"/>
>		</video>
>		<channel type="spicevmc">
>			<target name="com.redhat.spice.0" type="virtio"/>
>		</channel>
>		<graphics autoport="yes" keymap="en-us" listen="0" passwd="*****" passwdValidTo="1970-01-01T00:00:01" port="-1" tlsPort="-1" type="spice">
>			<channel mode="secure" name="main"/>
>			<channel mode="secure" name="inputs"/>
>		</graphics>
>	</devices>
>	<os>
>		<type arch="x86_64" machine="rhel6.2.0">hvm</type>
>		<boot dev="hd"/>
>		<boot dev="cdrom"/>
>		<smbios mode="sysinfo"/>
>	</os>
>	<sysinfo type="smbios">
>		<system>
>			<entry name="manufacturer">Red Hat</entry>
>			<entry name="product">RHEV Hypervisor</entry>
>			<entry name="version">6.3-20121012.0.el6_3</entry>
>			<entry name="serial">30343536-3138-5A43-4A32-333630373442_AC:16:2D:76:EB:F0</entry>
>			<entry name="uuid">4760db3e-d3a8-4045-9153-12c36ec16ba2</entry>
>		</system>
>	</sysinfo>
>	<clock adjustment="-1" offset="variable">
>		<timer name="rtc" tickpolicy="catchup"/>
>	</clock>
>	<features>
>		<acpi/>
>	</features>
>	<cpu match="exact">
>		<model>Westmere</model>
>		<topology cores="2" sockets="2" threads="1"/>
>	</cpu>
></domain>
>
>Dummy-15912651::DEBUG::2013-11-20 03:09:57,087::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:09:57,703::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0843804 s, 12.1 MB/s\n'; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-20 03:09:58,382::libvirtvm::1401::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::event Started detail 0 opaque None
>Thread-15913625::DEBUG::2013-11-20 03:09:58,435::utils::307::vm.Vm::(start) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Start statistics collection
>Thread-15913639::DEBUG::2013-11-20 03:09:58,436::utils::336::vm.Vm::(run) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Stats thread started
>Thread-15913640::DEBUG::2013-11-20 03:09:58,439::guestIF::47::root::(_connect) Attempting connection to /var/lib/libvirt/qemu/channels/KLEE-AOS01.com.redhat.rhevm.vdsm
>Thread-15913639::INFO::2013-11-20 03:09:58,441::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913640::DEBUG::2013-11-20 03:09:58,442::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /usr/share/vdsm/prepare-vmchannel /var/lib/libvirt/qemu/channels/KLEE-AOS01.com.redhat.rhevm.vdsm' (cwd None)
>Thread-15913639::DEBUG::2013-11-20 03:09:58,443::task::495::TaskManager.Task::(_debug) Task a1d0828f-420f-4416-aea1-531294dab989: moving from state init -> state preparing
>Thread-15913625::WARNING::2013-11-20 03:09:58,445::libvirtvm::1124::vm.Vm::(_readPauseCode) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::_readPauseCode unsupported by libvirt vm
>Thread-15913639::DEBUG::2013-11-20 03:09:58,677::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`93dd46a9-c5bb-4191-a11a-600c6dd980b2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913625::DEBUG::2013-11-20 03:09:58,679::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /usr/share/vdsm/get-vm-pid KLEE-AOS01' (cwd None)
>Thread-15913639::DEBUG::2013-11-20 03:09:58,680::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:09:59,104::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:09:59,109::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`93dd46a9-c5bb-4191-a11a-600c6dd980b2`::Granted request
>Thread-15913640::DEBUG::2013-11-20 03:09:59,112::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913639::DEBUG::2013-11-20 03:09:59,115::task::495::TaskManager.Task::(_debug) Task a1d0828f-420f-4416-aea1-531294dab989: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913640::DEBUG::2013-11-20 03:09:59,116::guestIF::53::root::(_connect) connected to /var/lib/libvirt/qemu/channels/KLEE-AOS01.com.redhat.rhevm.vdsm
>Thread-15913639::DEBUG::2013-11-20 03:09:59,116::task::495::TaskManager.Task::(_debug) Task a1d0828f-420f-4416-aea1-531294dab989: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:09:59,118::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913639::DEBUG::2013-11-20 03:09:59,120::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15913625::DEBUG::2013-11-20 03:09:59,497::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913625::DEBUG::2013-11-20 03:09:59,513::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /sbin/service ksmtuned retune' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:09:59,831::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::DEBUG::2013-11-20 03:10:00,233::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913625::DEBUG::2013-11-20 03:10:00,249::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913625::DEBUG::2013-11-20 03:10:00,251::vm::358::vm.Vm::(_startUnderlyingVm) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::_ongoingCreations released
>Thread-15913631::DEBUG::2013-11-20 03:10:00,251::vm::346::vm.Vm::(_startUnderlyingVm) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::_ongoingCreations acquired
>Thread-15913631::INFO::2013-11-20 03:10:00,316::libvirtvm::1061::vm.Vm::(_run) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::VM wrapper has started
>Thread-15913631::INFO::2013-11-20 03:10:00,389::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913631::DEBUG::2013-11-20 03:10:00,452::task::495::TaskManager.Task::(_debug) Task 6d5199b3-fd75-4ada-a50e-c85fdc68047f: moving from state init -> state preparing
>Thread-15913631::DEBUG::2013-11-20 03:10:00,559::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`47f0e53e-faa3-4b8c-87e7-0fdeaebe2b25`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1690' at 'public_prepareVolume'
>Thread-15913631::DEBUG::2013-11-20 03:10:00,570::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:10:00,618::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913631::DEBUG::2013-11-20 03:10:00,619::resourceManager::481::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' found in shared state and queue is empty, Joining current shared lock (2 active users)
>Thread-15913639::DEBUG::2013-11-20 03:10:00,622::task::495::TaskManager.Task::(_debug) Task a1d0828f-420f-4416-aea1-531294dab989: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913631::DEBUG::2013-11-20 03:10:00,623::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`47f0e53e-faa3-4b8c-87e7-0fdeaebe2b25`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:10:00,624::task::495::TaskManager.Task::(_debug) Task a1d0828f-420f-4416-aea1-531294dab989: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:10:00,626::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913631::DEBUG::2013-11-20 03:10:00,628::task::495::TaskManager.Task::(_debug) Task 6d5199b3-fd75-4ada-a50e-c85fdc68047f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:10:00,629::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913631::DEBUG::2013-11-20 03:10:00,630::task::495::TaskManager.Task::(_debug) Task 6d5199b3-fd75-4ada-a50e-c85fdc68047f: ref 1 aborting False
>Thread-15913631::DEBUG::2013-11-20 03:10:00,635::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.b74dd33b-748d-48df-83f2-ebcc3523e117`ReqID=`dc135158-444d-46f7-8e44-87c5210d8fb3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1695' at 'public_prepareVolume'
>Thread-15913639::DEBUG::2013-11-20 03:10:00,635::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913631::DEBUG::2013-11-20 03:10:00,636::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.b74dd33b-748d-48df-83f2-ebcc3523e117' for lock type 'exclusive'
>Thread-15913631::DEBUG::2013-11-20 03:10:00,639::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913639::DEBUG::2013-11-20 03:10:00,640::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (1 active users)
>Thread-15913631::DEBUG::2013-11-20 03:10:00,644::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15913639::DEBUG::2013-11-20 03:10:00,646::task::495::TaskManager.Task::(_debug) Task a1d0828f-420f-4416-aea1-531294dab989: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:10:00,647::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913642::DEBUG::2013-11-20 03:10:00,999::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913642::INFO::2013-11-20 03:10:01,001::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913642::DEBUG::2013-11-20 03:10:01,005::task::495::TaskManager.Task::(_debug) Task 7348e82f-4073-44fe-9662-9399756680ed: moving from state init -> state preparing
>Thread-15913642::DEBUG::2013-11-20 03:10:01,009::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913642::DEBUG::2013-11-20 03:10:01,010::task::495::TaskManager.Task::(_debug) Task 7348e82f-4073-44fe-9662-9399756680ed: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913642::DEBUG::2013-11-20 03:10:01,013::task::495::TaskManager.Task::(_debug) Task 7348e82f-4073-44fe-9662-9399756680ed: moving from state preparing -> state finished
>Thread-15913642::DEBUG::2013-11-20 03:10:01,016::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913642::DEBUG::2013-11-20 03:10:01,017::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913642::DEBUG::2013-11-20 03:10:01,019::task::495::TaskManager.Task::(_debug) Task 7348e82f-4073-44fe-9662-9399756680ed: ref 0 aborting False
>Thread-15913642::INFO::2013-11-20 03:10:01,020::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913643::DEBUG::2013-11-20 03:10:01,031::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913643::INFO::2013-11-20 03:10:01,033::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913643::DEBUG::2013-11-20 03:10:01,035::task::495::TaskManager.Task::(_debug) Task abe16ff2-2867-44bc-be19-2e7cb31b4d32: moving from state init -> state preparing
>Thread-15913643::DEBUG::2013-11-20 03:10:01,037::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b60010a2-628f-4c1c-8672-41b50f138692`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913643::DEBUG::2013-11-20 03:10:01,038::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913643::DEBUG::2013-11-20 03:10:01,043::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913643::DEBUG::2013-11-20 03:10:01,044::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b60010a2-628f-4c1c-8672-41b50f138692`::Granted request
>Thread-15913643::DEBUG::2013-11-20 03:10:01,047::task::495::TaskManager.Task::(_debug) Task abe16ff2-2867-44bc-be19-2e7cb31b4d32: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913643::DEBUG::2013-11-20 03:10:01,049::task::495::TaskManager.Task::(_debug) Task abe16ff2-2867-44bc-be19-2e7cb31b4d32: ref 1 aborting False
>Thread-15913643::WARNING::2013-11-20 03:10:01,054::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913643::DEBUG::2013-11-20 03:10:01,056::task::495::TaskManager.Task::(_debug) Task abe16ff2-2867-44bc-be19-2e7cb31b4d32: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913643::DEBUG::2013-11-20 03:10:01,058::task::495::TaskManager.Task::(_debug) Task abe16ff2-2867-44bc-be19-2e7cb31b4d32: moving from state preparing -> state finished
>Thread-15913643::DEBUG::2013-11-20 03:10:01,059::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913643::DEBUG::2013-11-20 03:10:01,060::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913643::DEBUG::2013-11-20 03:10:01,067::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913643::DEBUG::2013-11-20 03:10:01,068::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913643::DEBUG::2013-11-20 03:10:01,069::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913643::DEBUG::2013-11-20 03:10:01,071::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913643::DEBUG::2013-11-20 03:10:01,074::task::495::TaskManager.Task::(_debug) Task abe16ff2-2867-44bc-be19-2e7cb31b4d32: ref 0 aborting False
>Thread-15913643::INFO::2013-11-20 03:10:01,075::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:10:01,117::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.618191 s, 1.7 MB/s\n'; <rc> = 0
>Thread-15913631::DEBUG::2013-11-20 03:10:01,630::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913631::DEBUG::2013-11-20 03:10:01,884::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913631::DEBUG::2013-11-20 03:10:01,890::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=11 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15913645::DEBUG::2013-11-20 03:10:02,143::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('4760db3e-d3a8-4045-9153-12c36ec16ba2',) {}
>Thread-15913645::DEBUG::2013-11-20 03:10:02,144::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::CPU stats not available
>Thread-15913645::DEBUG::2013-11-20 03:10:02,145::libvirtvm::202::vm.Vm::(_getNetworkStats) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Network stats not available
>Thread-15913645::DEBUG::2013-11-20 03:10:02,146::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Disk stats not available
>Thread-15913645::DEBUG::2013-11-20 03:10:02,147::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Disk latency not available
>Thread-15913645::DEBUG::2013-11-20 03:10:02,148::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '49279', 'displayIp': '0', 'displayPort': u'5906', 'session': 'Unknown', 'displaySecurePort': u'5907', 'timeOffset': '-1', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet4': {'state': 'unknown', 'macAddr': u'00:1a:4a:a8:64:04', 'speed': '1000', 'name': u'vnet4'}}, 'vmId': '4760db3e-d3a8-4045-9153-12c36ec16ba2', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '85899345920', 'writeLatency': '0', 'imageID': '6f97dfa5-ee35-4f37-8495-1e6b3136b92c', 'flushLatency': '0', 'truesize': '85899345920'}}, 'boot': 'cd', 'statsAge': '3.70', 'cpuIdle': '100.00', 'elapsedTime': '14', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15913631::DEBUG::2013-11-20 03:10:02,340::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.00763903 s, 67.0 kB/s\n'; <rc> = 0
>Thread-15913631::DEBUG::2013-11-20 03:10:02,341::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.00763903 s, 67.0 kB/s'], size: 512
>Thread-15913631::DEBUG::2013-11-20 03:10:02,343::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=11 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15913631::DEBUG::2013-11-20 03:10:02,752::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000338672 s, 1.5 MB/s\n'; <rc> = 0
>Thread-15913631::DEBUG::2013-11-20 03:10:02,754::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000338672 s, 1.5 MB/s'], size: 512
>Thread-15913631::INFO::2013-11-20 03:10:02,755::image::340::Storage.Image::(getChain) sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 chain=[<storage.blockVolume.BlockVolume instance at 0x7f47180c4128>] 
>Thread-15913631::DEBUG::2013-11-20 03:10:02,756::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --autobackup n --available y 6b892067-ac7e-4039-a7be-5b7387525b16/8506c15b-61ae-4830-84ed-c830955c25a3' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:10:03,138::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:10:03,605::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0640672 s, 16.0 MB/s\n'; <rc> = 0
>Thread-15913631::DEBUG::2013-11-20 03:10:03,654::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913631::DEBUG::2013-11-20 03:10:03,656::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913631::DEBUG::2013-11-20 03:10:03,656::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913631::DEBUG::2013-11-20 03:10:03,658::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.8506c15b-61ae-4830-84ed-c830955c25a3`ReqID=`cefafd7f-8cd8-4c76-a571-146d0882160a`::Request was made in '/usr/share/vdsm/storage/resourceFactories.py' line '149' at '__getResourceCandidatesList'
>Thread-15913631::DEBUG::2013-11-20 03:10:03,659::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.8506c15b-61ae-4830-84ed-c830955c25a3' for lock type 'exclusive'
>Thread-15913631::DEBUG::2013-11-20 03:10:03,660::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.8506c15b-61ae-4830-84ed-c830955c25a3' is free. Now locking as 'exclusive' (1 active user)
>Thread-15913631::DEBUG::2013-11-20 03:10:03,660::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.8506c15b-61ae-4830-84ed-c830955c25a3`ReqID=`cefafd7f-8cd8-4c76-a571-146d0882160a`::Granted request
>Thread-15913631::DEBUG::2013-11-20 03:10:03,661::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.b74dd33b-748d-48df-83f2-ebcc3523e117' is free. Now locking as 'exclusive' (1 active user)
>Thread-15913631::DEBUG::2013-11-20 03:10:03,662::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.b74dd33b-748d-48df-83f2-ebcc3523e117`ReqID=`dc135158-444d-46f7-8e44-87c5210d8fb3`::Granted request
>Thread-15913631::DEBUG::2013-11-20 03:10:03,664::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913631::DEBUG::2013-11-20 03:10:03,666::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15913631::DEBUG::2013-11-20 03:10:04,504::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913631::DEBUG::2013-11-20 03:10:04,712::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913631::DEBUG::2013-11-20 03:10:04,713::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --refresh 6b892067-ac7e-4039-a7be-5b7387525b16/8506c15b-61ae-4830-84ed-c830955c25a3' (cwd None)
>Thread-15913647::DEBUG::2013-11-20 03:10:05,239::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('4760db3e-d3a8-4045-9153-12c36ec16ba2',) {}
>Thread-15913647::DEBUG::2013-11-20 03:10:05,240::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::CPU stats not available
>Thread-15913647::DEBUG::2013-11-20 03:10:05,241::libvirtvm::202::vm.Vm::(_getNetworkStats) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Network stats not available
>Thread-15913647::DEBUG::2013-11-20 03:10:05,241::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Disk stats not available
>Thread-15913647::DEBUG::2013-11-20 03:10:05,242::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Disk latency not available
>Thread-15913647::DEBUG::2013-11-20 03:10:05,243::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '49279', 'displayIp': '0', 'displayPort': u'5906', 'session': 'Unknown', 'displaySecurePort': u'5907', 'timeOffset': '-1', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet4': {'state': 'unknown', 'macAddr': u'00:1a:4a:a8:64:04', 'speed': '1000', 'name': u'vnet4'}}, 'vmId': '4760db3e-d3a8-4045-9153-12c36ec16ba2', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '85899345920', 'writeLatency': '0', 'imageID': '6f97dfa5-ee35-4f37-8495-1e6b3136b92c', 'flushLatency': '0', 'truesize': '85899345920'}}, 'boot': 'cd', 'statsAge': '0.25', 'cpuIdle': '100.00', 'elapsedTime': '17', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:10:05,615::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913631::DEBUG::2013-11-20 03:10:06,000::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913631::DEBUG::2013-11-20 03:10:06,001::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913631::DEBUG::2013-11-20 03:10:06,002::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913631::INFO::2013-11-20 03:10:06,003::volume::517::Storage.Volume::(prepare) Volume: preparing volume 6b892067-ac7e-4039-a7be-5b7387525b16/8506c15b-61ae-4830-84ed-c830955c25a3
>Thread-15913631::DEBUG::2013-11-20 03:10:06,004::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913631::DEBUG::2013-11-20 03:10:06,005::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:10:06,251::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.252287 s, 4.1 MB/s\n'; <rc> = 0
>Thread-15913631::DEBUG::2013-11-20 03:10:06,971::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913631::DEBUG::2013-11-20 03:10:07,296::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913631::DEBUG::2013-11-20 03:10:07,298::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=11 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15913631::DEBUG::2013-11-20 03:10:07,877::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000240376 s, 2.1 MB/s\n'; <rc> = 0
>Thread-15913631::DEBUG::2013-11-20 03:10:07,879::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000240376 s, 2.1 MB/s'], size: 512
>Thread-15913631::DEBUG::2013-11-20 03:10:07,880::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=11 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:10:08,263::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913649::DEBUG::2013-11-20 03:10:08,502::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('4760db3e-d3a8-4045-9153-12c36ec16ba2',) {}
>Thread-15913649::DEBUG::2013-11-20 03:10:08,503::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::CPU stats not available
>Thread-15913631::DEBUG::2013-11-20 03:10:08,505::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000230126 s, 2.2 MB/s\n'; <rc> = 0
>Thread-15913649::DEBUG::2013-11-20 03:10:08,505::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Disk stats not available
>Thread-15913631::DEBUG::2013-11-20 03:10:08,506::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000230126 s, 2.2 MB/s'], size: 512
>Thread-15913649::DEBUG::2013-11-20 03:10:08,506::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Disk latency not available
>Thread-15913631::DEBUG::2013-11-20 03:10:08,509::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.8506c15b-61ae-4830-84ed-c830955c25a3`ReqID=`8221faf1-4e78-4a4c-8772-cedc7fd01c58`::Request was made in '/usr/share/vdsm/storage/volume.py' line '533' at 'prepare'
>Thread-15913649::DEBUG::2013-11-20 03:10:08,509::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '49279', 'displayIp': '0', 'displayPort': u'5906', 'session': 'Unknown', 'displaySecurePort': u'5907', 'timeOffset': '-1', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet4': {'macAddr': u'00:1a:4a:a8:64:04', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet4'}}, 'vmId': '4760db3e-d3a8-4045-9153-12c36ec16ba2', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '85899345920', 'writeLatency': '0', 'imageID': '6f97dfa5-ee35-4f37-8495-1e6b3136b92c', 'flushLatency': '0', 'truesize': '85899345920'}}, 'boot': 'cd', 'statsAge': '1.48', 'cpuIdle': '100.00', 'elapsedTime': '21', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15913631::DEBUG::2013-11-20 03:10:08,510::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.8506c15b-61ae-4830-84ed-c830955c25a3' for lock type 'exclusive'
>Thread-15913631::DEBUG::2013-11-20 03:10:08,512::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.8506c15b-61ae-4830-84ed-c830955c25a3' is free. Now locking as 'exclusive' (1 active user)
>Thread-15913631::DEBUG::2013-11-20 03:10:08,513::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.8506c15b-61ae-4830-84ed-c830955c25a3`ReqID=`8221faf1-4e78-4a4c-8772-cedc7fd01c58`::Granted request
>Thread-15913631::DEBUG::2013-11-20 03:10:08,514::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=11 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:10:09,057::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.095956 s, 10.7 MB/s\n'; <rc> = 0
>Thread-15913631::DEBUG::2013-11-20 03:10:09,106::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000153089 s, 3.3 MB/s\n'; <rc> = 0
>Thread-15913631::DEBUG::2013-11-20 03:10:09,107::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000153089 s, 3.3 MB/s'], size: 512
>Thread-15913631::DEBUG::2013-11-20 03:10:09,108::task::495::TaskManager.Task::(_debug) Task 6d5199b3-fd75-4ada-a50e-c85fdc68047f: finished: None
>Thread-15913631::DEBUG::2013-11-20 03:10:09,109::task::495::TaskManager.Task::(_debug) Task 6d5199b3-fd75-4ada-a50e-c85fdc68047f: moving from state preparing -> state finished
>Thread-15913631::DEBUG::2013-11-20 03:10:09,109::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913631::DEBUG::2013-11-20 03:10:09,110::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913631::DEBUG::2013-11-20 03:10:09,111::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913631::DEBUG::2013-11-20 03:10:09,111::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913631::DEBUG::2013-11-20 03:10:09,112::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913631::DEBUG::2013-11-20 03:10:09,112::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913631::DEBUG::2013-11-20 03:10:09,112::task::495::TaskManager.Task::(_debug) Task 6d5199b3-fd75-4ada-a50e-c85fdc68047f: ref 0 aborting False
>Thread-15913631::INFO::2013-11-20 03:10:09,113::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, Return response: {'status': {'message': 'OK', 'code': 0}}
>Thread-15913631::INFO::2013-11-20 03:10:09,114::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913631::DEBUG::2013-11-20 03:10:09,114::task::495::TaskManager.Task::(_debug) Task ffe2360f-0e81-4786-b017-07d00f15862c: moving from state init -> state preparing
>Thread-15913631::DEBUG::2013-11-20 03:10:09,115::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1811646a-5859-4197-8ed9-009a1e9d0929`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1666' at 'public_getVolumePath'
>Thread-15913631::DEBUG::2013-11-20 03:10:09,115::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913631::DEBUG::2013-11-20 03:10:09,116::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913631::DEBUG::2013-11-20 03:10:09,116::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1811646a-5859-4197-8ed9-009a1e9d0929`::Granted request
>Thread-15913631::DEBUG::2013-11-20 03:10:09,117::task::495::TaskManager.Task::(_debug) Task ffe2360f-0e81-4786-b017-07d00f15862c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913631::DEBUG::2013-11-20 03:10:09,118::task::495::TaskManager.Task::(_debug) Task ffe2360f-0e81-4786-b017-07d00f15862c: ref 1 aborting False
>Thread-15913631::DEBUG::2013-11-20 03:10:09,118::task::495::TaskManager.Task::(_debug) Task ffe2360f-0e81-4786-b017-07d00f15862c: finished: {'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/b74dd33b-748d-48df-83f2-ebcc3523e117/8506c15b-61ae-4830-84ed-c830955c25a3'}
>Thread-15913631::DEBUG::2013-11-20 03:10:09,119::task::495::TaskManager.Task::(_debug) Task ffe2360f-0e81-4786-b017-07d00f15862c: moving from state preparing -> state finished
>Thread-15913631::DEBUG::2013-11-20 03:10:09,119::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913631::DEBUG::2013-11-20 03:10:09,120::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913631::DEBUG::2013-11-20 03:10:09,120::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913631::DEBUG::2013-11-20 03:10:09,120::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913631::DEBUG::2013-11-20 03:10:09,121::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913631::DEBUG::2013-11-20 03:10:09,121::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913631::DEBUG::2013-11-20 03:10:09,122::task::495::TaskManager.Task::(_debug) Task ffe2360f-0e81-4786-b017-07d00f15862c: ref 0 aborting False
>Thread-15913631::INFO::2013-11-20 03:10:09,122::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, Return response: {'status': {'message': 'OK', 'code': 0}, 'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/b74dd33b-748d-48df-83f2-ebcc3523e117/8506c15b-61ae-4830-84ed-c830955c25a3'}
>Thread-15913631::INFO::2013-11-20 03:10:09,122::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913631::DEBUG::2013-11-20 03:10:09,123::task::495::TaskManager.Task::(_debug) Task 3e95a198-743d-447e-9e03-401d6f7c901f: moving from state init -> state preparing
>Thread-15913631::DEBUG::2013-11-20 03:10:09,124::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`621c0327-8cc9-4d69-8b4d-b70099e1d312`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913631::DEBUG::2013-11-20 03:10:09,124::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913631::DEBUG::2013-11-20 03:10:09,125::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913631::DEBUG::2013-11-20 03:10:09,125::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`621c0327-8cc9-4d69-8b4d-b70099e1d312`::Granted request
>Thread-15913631::DEBUG::2013-11-20 03:10:09,125::task::495::TaskManager.Task::(_debug) Task 3e95a198-743d-447e-9e03-401d6f7c901f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913631::DEBUG::2013-11-20 03:10:09,126::task::495::TaskManager.Task::(_debug) Task 3e95a198-743d-447e-9e03-401d6f7c901f: ref 1 aborting False
>Thread-15913631::DEBUG::2013-11-20 03:10:09,126::task::495::TaskManager.Task::(_debug) Task 3e95a198-743d-447e-9e03-401d6f7c901f: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913631::DEBUG::2013-11-20 03:10:09,126::task::495::TaskManager.Task::(_debug) Task 3e95a198-743d-447e-9e03-401d6f7c901f: moving from state preparing -> state finished
>Thread-15913631::DEBUG::2013-11-20 03:10:09,127::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913631::DEBUG::2013-11-20 03:10:09,127::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913631::DEBUG::2013-11-20 03:10:09,128::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913631::DEBUG::2013-11-20 03:10:09,128::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913631::DEBUG::2013-11-20 03:10:09,128::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913631::DEBUG::2013-11-20 03:10:09,129::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913631::DEBUG::2013-11-20 03:10:09,129::task::495::TaskManager.Task::(_debug) Task 3e95a198-743d-447e-9e03-401d6f7c901f: ref 0 aborting False
>Thread-15913631::INFO::2013-11-20 03:10:09,130::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913631::INFO::2013-11-20 03:10:09,130::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16)
>Thread-15913631::DEBUG::2013-11-20 03:10:09,130::task::495::TaskManager.Task::(_debug) Task 567eb2d2-3196-4994-94b1-1f4075e91607: moving from state init -> state preparing
>Thread-15913631::INFO::2013-11-20 03:10:09,131::blockSD::674::Storage.StorageDomain::(validate) sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16
>Thread-15913631::DEBUG::2013-11-20 03:10:09,131::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913631::DEBUG::2013-11-20 03:10:09,132::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913631::DEBUG::2013-11-20 03:10:09,132::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913631::DEBUG::2013-11-20 03:10:09,133::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913631::DEBUG::2013-11-20 03:10:09,134::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913631::DEBUG::2013-11-20 03:10:09,134::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15913631::DEBUG::2013-11-20 03:10:10,540::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913631::DEBUG::2013-11-20 03:10:10,542::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913631::DEBUG::2013-11-20 03:10:10,543::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'_SHA_CKSUM=716ea7492d9134a4b13de27ac61a336ad474727c', u'POOL_SPM_ID=2', u'POOL_SPM_LVER=287']
>Thread-15913631::DEBUG::2013-11-20 03:10:10,544::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`25f25ed8-8051-48ba-ae5f-5d9cab618c17`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1297' at 'public_getStorageDomainInfo'
>Thread-15913631::DEBUG::2013-11-20 03:10:10,545::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913631::DEBUG::2013-11-20 03:10:10,545::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913631::DEBUG::2013-11-20 03:10:10,546::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`25f25ed8-8051-48ba-ae5f-5d9cab618c17`::Granted request
>Thread-15913631::DEBUG::2013-11-20 03:10:10,546::task::495::TaskManager.Task::(_debug) Task 567eb2d2-3196-4994-94b1-1f4075e91607: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913631::DEBUG::2013-11-20 03:10:10,547::task::495::TaskManager.Task::(_debug) Task 567eb2d2-3196-4994-94b1-1f4075e91607: ref 1 aborting False
>Thread-15913631::WARNING::2013-11-20 03:10:10,548::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913631::DEBUG::2013-11-20 03:10:10,549::task::495::TaskManager.Task::(_debug) Task 567eb2d2-3196-4994-94b1-1f4075e91607: finished: {'info': {'uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'vguuid': 'aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', 'lver': 287, 'state': 'OK', 'version': '2', 'role': 'Master', 'master_ver': 1, 'spm_id': 2, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'OS'}}
>Thread-15913631::DEBUG::2013-11-20 03:10:10,549::task::495::TaskManager.Task::(_debug) Task 567eb2d2-3196-4994-94b1-1f4075e91607: moving from state preparing -> state finished
>Thread-15913631::DEBUG::2013-11-20 03:10:10,549::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913631::DEBUG::2013-11-20 03:10:10,550::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913631::DEBUG::2013-11-20 03:10:10,550::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913631::DEBUG::2013-11-20 03:10:10,551::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913631::DEBUG::2013-11-20 03:10:10,551::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913631::DEBUG::2013-11-20 03:10:10,551::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913631::DEBUG::2013-11-20 03:10:10,552::task::495::TaskManager.Task::(_debug) Task 567eb2d2-3196-4994-94b1-1f4075e91607: ref 0 aborting False
>Thread-15913631::INFO::2013-11-20 03:10:10,552::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'vguuid': 'aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', 'lver': 287, 'state': 'OK', 'version': '2', 'role': 'Master', 'master_ver': 1, 'spm_id': 2, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'OS'}}
>Thread-15913631::DEBUG::2013-11-20 03:10:10,577::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/libexec/vdsm/hooks/before_vm_start/50_vhostmd' (cwd None)
>Thread-15913631::DEBUG::2013-11-20 03:10:10,989::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913631::INFO::2013-11-20 03:10:10,990::hooks::51::root::(_runHooksDir) 
>Thread-15913631::DEBUG::2013-11-20 03:10:10,991::libvirtvm::1077::vm.Vm::(_run) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::<?xml version="1.0" encoding="utf-8"?>
><domain type="kvm">
>	<name>KLEE-AOS02</name>
>	<uuid>0dc7e997-e884-4493-b701-790fb37b3d80</uuid>
>	<memory>4194304</memory>
>	<currentMemory>4194304</currentMemory>
>	<vcpu>4</vcpu>
>	<devices>
>		<disk device="disk" type="block">
>			<source dev="/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/b74dd33b-748d-48df-83f2-ebcc3523e117/8506c15b-61ae-4830-84ed-c830955c25a3"/>
>			<target bus="virtio" dev="vda"/>
>			<serial>df-83f2-ebcc3523e117</serial>
>			<driver cache="none" error_policy="stop" io="native" name="qemu" type="raw"/>
>		</disk>
>		<disk device="cdrom" type="file">
>			<target bus="ide" dev="hdc"/>
>		</disk>
>		<controller index="0" ports="16" type="virtio-serial"/>
>		<channel type="unix">
>			<target name="com.redhat.rhevm.vdsm" type="virtio"/>
>			<source mode="bind" path="/var/lib/libvirt/qemu/channels/KLEE-AOS02.com.redhat.rhevm.vdsm"/>
>		</channel>
>		<memballoon model="none"/>
>		<interface type="bridge">
>			<mac address="00:1a:4a:a8:64:03"/>
>			<model type="virtio"/>
>			<source bridge="rhevm"/>
>		</interface>
>		<input bus="ps2" type="mouse"/>
>		<video>
>			<model heads="1" type="qxl" vram="65536"/>
>		</video>
>		<channel type="spicevmc">
>			<target name="com.redhat.spice.0" type="virtio"/>
>		</channel>
>		<graphics autoport="yes" keymap="en-us" listen="0" passwd="*****" passwdValidTo="1970-01-01T00:00:01" port="-1" tlsPort="-1" type="spice">
>			<channel mode="secure" name="main"/>
>			<channel mode="secure" name="inputs"/>
>		</graphics>
>	</devices>
>	<os>
>		<type arch="x86_64" machine="rhel6.2.0">hvm</type>
>		<boot dev="hd"/>
>		<boot dev="cdrom"/>
>		<smbios mode="sysinfo"/>
>	</os>
>	<sysinfo type="smbios">
>		<system>
>			<entry name="manufacturer">Red Hat</entry>
>			<entry name="product">RHEV Hypervisor</entry>
>			<entry name="version">6.3-20121012.0.el6_3</entry>
>			<entry name="serial">30343536-3138-5A43-4A32-333630373442_AC:16:2D:76:EB:F0</entry>
>			<entry name="uuid">0dc7e997-e884-4493-b701-790fb37b3d80</entry>
>		</system>
>	</sysinfo>
>	<clock adjustment="3600" offset="variable">
>		<timer name="rtc" tickpolicy="catchup"/>
>	</clock>
>	<features>
>		<acpi/>
>	</features>
>	<cpu match="exact">
>		<model>Westmere</model>
>		<topology cores="2" sockets="2" threads="1"/>
>	</cpu>
></domain>
>
>Dummy-15912651::DEBUG::2013-11-20 03:10:11,068::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913650::DEBUG::2013-11-20 03:10:11,275::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913650::INFO::2013-11-20 03:10:11,276::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913650::DEBUG::2013-11-20 03:10:11,277::task::495::TaskManager.Task::(_debug) Task fab6245e-6ae9-4f48-8eae-ecfdab4234d6: moving from state init -> state preparing
>Thread-15913650::DEBUG::2013-11-20 03:10:11,278::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913650::DEBUG::2013-11-20 03:10:11,278::task::495::TaskManager.Task::(_debug) Task fab6245e-6ae9-4f48-8eae-ecfdab4234d6: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913650::DEBUG::2013-11-20 03:10:11,279::task::495::TaskManager.Task::(_debug) Task fab6245e-6ae9-4f48-8eae-ecfdab4234d6: moving from state preparing -> state finished
>Thread-15913650::DEBUG::2013-11-20 03:10:11,279::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913650::DEBUG::2013-11-20 03:10:11,279::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913650::DEBUG::2013-11-20 03:10:11,280::task::495::TaskManager.Task::(_debug) Task fab6245e-6ae9-4f48-8eae-ecfdab4234d6: ref 0 aborting False
>Thread-15913650::INFO::2013-11-20 03:10:11,280::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913651::DEBUG::2013-11-20 03:10:11,284::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913651::INFO::2013-11-20 03:10:11,285::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913651::DEBUG::2013-11-20 03:10:11,285::task::495::TaskManager.Task::(_debug) Task eb0fa976-5ca8-4463-a20f-ceaeae71d2c7: moving from state init -> state preparing
>Thread-15913651::DEBUG::2013-11-20 03:10:11,286::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d318ff2f-1cb7-47ea-963c-d07a3c56ec5c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913651::DEBUG::2013-11-20 03:10:11,287::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913651::DEBUG::2013-11-20 03:10:11,287::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913651::DEBUG::2013-11-20 03:10:11,288::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d318ff2f-1cb7-47ea-963c-d07a3c56ec5c`::Granted request
>Thread-15913651::DEBUG::2013-11-20 03:10:11,288::task::495::TaskManager.Task::(_debug) Task eb0fa976-5ca8-4463-a20f-ceaeae71d2c7: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913651::DEBUG::2013-11-20 03:10:11,289::task::495::TaskManager.Task::(_debug) Task eb0fa976-5ca8-4463-a20f-ceaeae71d2c7: ref 1 aborting False
>Thread-15913651::WARNING::2013-11-20 03:10:11,290::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913651::DEBUG::2013-11-20 03:10:11,291::task::495::TaskManager.Task::(_debug) Task eb0fa976-5ca8-4463-a20f-ceaeae71d2c7: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913651::DEBUG::2013-11-20 03:10:11,292::task::495::TaskManager.Task::(_debug) Task eb0fa976-5ca8-4463-a20f-ceaeae71d2c7: moving from state preparing -> state finished
>Thread-15913651::DEBUG::2013-11-20 03:10:11,292::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913651::DEBUG::2013-11-20 03:10:11,292::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913651::DEBUG::2013-11-20 03:10:11,293::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913651::DEBUG::2013-11-20 03:10:11,293::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913651::DEBUG::2013-11-20 03:10:11,293::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913651::DEBUG::2013-11-20 03:10:11,294::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913651::DEBUG::2013-11-20 03:10:11,294::task::495::TaskManager.Task::(_debug) Task eb0fa976-5ca8-4463-a20f-ceaeae71d2c7: ref 0 aborting False
>Thread-15913651::INFO::2013-11-20 03:10:11,295::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913652::INFO::2013-11-20 03:10:11,530::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913652::DEBUG::2013-11-20 03:10:11,530::task::495::TaskManager.Task::(_debug) Task e4b7b303-4a9f-46ae-b03d-8089670c629b: moving from state init -> state preparing
>Thread-15913652::DEBUG::2013-11-20 03:10:11,531::task::495::TaskManager.Task::(_debug) Task e4b7b303-4a9f-46ae-b03d-8089670c629b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000564098358154', 'lastCheck': 1384917001.5123861, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00155115127563', 'lastCheck': 1384917009.94769, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00176382064819', 'lastCheck': 1384917010.944814, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000507831573486', 'lastCheck': 1384917001.212626, 'valid': True, 'code': 0}}
>Thread-15913652::DEBUG::2013-11-20 03:10:11,532::task::495::TaskManager.Task::(_debug) Task e4b7b303-4a9f-46ae-b03d-8089670c629b: moving from state preparing -> state finished
>Thread-15913652::DEBUG::2013-11-20 03:10:11,532::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913652::DEBUG::2013-11-20 03:10:11,533::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913652::DEBUG::2013-11-20 03:10:11,533::task::495::TaskManager.Task::(_debug) Task e4b7b303-4a9f-46ae-b03d-8089670c629b: ref 0 aborting False
>Thread-15913652::INFO::2013-11-20 03:10:11,534::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000564098358154', 'lastCheck': 1384917001.5123861, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00155115127563', 'lastCheck': 1384917009.94769, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00176382064819', 'lastCheck': 1384917010.944814, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000507831573486', 'lastCheck': 1384917001.212626, 'valid': True, 'code': 0}}
>Thread-15913653::DEBUG::2013-11-20 03:10:11,548::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::CPU stats not available
>Thread-15913653::DEBUG::2013-11-20 03:10:11,550::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Disk stats not available
>Thread-15913653::DEBUG::2013-11-20 03:10:11,551::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Disk latency not available
>Dummy-15912651::DEBUG::2013-11-20 03:10:11,591::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0585906 s, 17.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:10:13,602::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:10:14,360::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0504733 s, 20.3 MB/s\n'; <rc> = 0
>Thread-15913655::DEBUG::2013-11-20 03:10:14,723::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('4760db3e-d3a8-4045-9153-12c36ec16ba2',) {}
>Thread-15913655::DEBUG::2013-11-20 03:10:14,724::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::CPU stats not available
>Thread-15913655::DEBUG::2013-11-20 03:10:14,724::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Disk stats not available
>Thread-15913655::DEBUG::2013-11-20 03:10:14,725::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Disk latency not available
>Thread-15913655::DEBUG::2013-11-20 03:10:14,725::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '49279', 'displayIp': '0', 'displayPort': u'5906', 'session': 'Unknown', 'displaySecurePort': u'5907', 'timeOffset': '-1', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet4': {'macAddr': u'00:1a:4a:a8:64:04', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet4'}}, 'vmId': '4760db3e-d3a8-4045-9153-12c36ec16ba2', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '85899345920', 'writeLatency': '0', 'imageID': '6f97dfa5-ee35-4f37-8495-1e6b3136b92c', 'flushLatency': '0', 'truesize': '85899345920'}}, 'boot': 'cd', 'statsAge': '3.70', 'cpuIdle': '100.00', 'elapsedTime': '27', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:10:16,370::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:10:17,309::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0671163 s, 15.3 MB/s\n'; <rc> = 0
>Thread-15913657::DEBUG::2013-11-20 03:10:17,749::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('4760db3e-d3a8-4045-9153-12c36ec16ba2',) {}
>Thread-15913657::DEBUG::2013-11-20 03:10:17,750::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::CPU stats not available
>Thread-15913657::DEBUG::2013-11-20 03:10:17,750::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Disk stats not available
>Thread-15913657::DEBUG::2013-11-20 03:10:17,751::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Disk latency not available
>Thread-15913657::DEBUG::2013-11-20 03:10:17,751::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '49279', 'displayIp': '0', 'displayPort': u'5906', 'session': 'Unknown', 'displaySecurePort': u'5907', 'timeOffset': '-1', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet4': {'macAddr': u'00:1a:4a:a8:64:04', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet4'}}, 'vmId': '4760db3e-d3a8-4045-9153-12c36ec16ba2', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '85899345920', 'writeLatency': '0', 'imageID': '6f97dfa5-ee35-4f37-8495-1e6b3136b92c', 'flushLatency': '0', 'truesize': '85899345920'}}, 'boot': 'cd', 'statsAge': '6.73', 'cpuIdle': '100.00', 'elapsedTime': '30', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:10:19,319::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:10:19,749::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0465877 s, 22.0 MB/s\n'; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-20 03:10:20,250::libvirtvm::1401::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::event Started detail 0 opaque None
>Thread-15913631::DEBUG::2013-11-20 03:10:20,299::utils::307::vm.Vm::(start) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::Start statistics collection
>Thread-15913658::DEBUG::2013-11-20 03:10:20,300::utils::336::vm.Vm::(run) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::Stats thread started
>Thread-15913659::DEBUG::2013-11-20 03:10:20,302::guestIF::47::root::(_connect) Attempting connection to /var/lib/libvirt/qemu/channels/KLEE-AOS02.com.redhat.rhevm.vdsm
>Thread-15913658::INFO::2013-11-20 03:10:20,302::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913659::DEBUG::2013-11-20 03:10:20,303::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /usr/share/vdsm/prepare-vmchannel /var/lib/libvirt/qemu/channels/KLEE-AOS02.com.redhat.rhevm.vdsm' (cwd None)
>Thread-15913658::DEBUG::2013-11-20 03:10:20,304::task::495::TaskManager.Task::(_debug) Task 41bf2d30-79d8-4dee-93e3-f07eddda1b5b: moving from state init -> state preparing
>Thread-15913631::WARNING::2013-11-20 03:10:20,305::libvirtvm::1124::vm.Vm::(_readPauseCode) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::_readPauseCode unsupported by libvirt vm
>Thread-15913658::DEBUG::2013-11-20 03:10:20,306::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`734c584a-fc62-4f29-94f0-e32c91056105`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913631::DEBUG::2013-11-20 03:10:20,464::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /usr/share/vdsm/get-vm-pid KLEE-AOS02' (cwd None)
>Thread-15913658::DEBUG::2013-11-20 03:10:20,465::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:10:20,627::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:10:20,628::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`734c584a-fc62-4f29-94f0-e32c91056105`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:10:20,651::task::495::TaskManager.Task::(_debug) Task 41bf2d30-79d8-4dee-93e3-f07eddda1b5b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:10:20,651::task::495::TaskManager.Task::(_debug) Task 41bf2d30-79d8-4dee-93e3-f07eddda1b5b: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:10:20,652::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913658::DEBUG::2013-11-20 03:10:20,654::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15913659::DEBUG::2013-11-20 03:10:21,073::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913631::DEBUG::2013-11-20 03:10:21,079::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913659::DEBUG::2013-11-20 03:10:21,081::guestIF::53::root::(_connect) connected to /var/lib/libvirt/qemu/channels/KLEE-AOS02.com.redhat.rhevm.vdsm
>Thread-15913661::DEBUG::2013-11-20 03:10:21,083::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('4760db3e-d3a8-4045-9153-12c36ec16ba2',) {}
>Thread-15913661::DEBUG::2013-11-20 03:10:21,085::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::CPU stats not available
>Thread-15913661::DEBUG::2013-11-20 03:10:21,085::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Disk stats not available
>Thread-15913661::DEBUG::2013-11-20 03:10:21,086::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Disk latency not available
>Thread-15913661::DEBUG::2013-11-20 03:10:21,087::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '49279', 'displayIp': '0', 'displayPort': u'5906', 'session': 'Unknown', 'displaySecurePort': u'5907', 'timeOffset': '-1', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet4': {'macAddr': u'00:1a:4a:a8:64:04', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet4'}}, 'vmId': '4760db3e-d3a8-4045-9153-12c36ec16ba2', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '85899345920', 'writeLatency': '0', 'imageID': '6f97dfa5-ee35-4f37-8495-1e6b3136b92c', 'flushLatency': '0', 'truesize': '85899345920'}}, 'boot': 'cd', 'statsAge': '1.87', 'cpuIdle': '100.00', 'elapsedTime': '33', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15913631::DEBUG::2013-11-20 03:10:21,094::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /sbin/service ksmtuned retune' (cwd None)
>Thread-15913662::DEBUG::2013-11-20 03:10:21,536::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913662::INFO::2013-11-20 03:10:21,538::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913662::DEBUG::2013-11-20 03:10:21,539::task::495::TaskManager.Task::(_debug) Task 5542c1e1-8a07-4431-9834-7a5edd02314b: moving from state init -> state preparing
>Thread-15913662::DEBUG::2013-11-20 03:10:21,540::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913662::DEBUG::2013-11-20 03:10:21,540::task::495::TaskManager.Task::(_debug) Task 5542c1e1-8a07-4431-9834-7a5edd02314b: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913662::DEBUG::2013-11-20 03:10:21,540::task::495::TaskManager.Task::(_debug) Task 5542c1e1-8a07-4431-9834-7a5edd02314b: moving from state preparing -> state finished
>Thread-15913662::DEBUG::2013-11-20 03:10:21,541::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913662::DEBUG::2013-11-20 03:10:21,541::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913662::DEBUG::2013-11-20 03:10:21,542::task::495::TaskManager.Task::(_debug) Task 5542c1e1-8a07-4431-9834-7a5edd02314b: ref 0 aborting False
>Thread-15913662::INFO::2013-11-20 03:10:21,542::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913663::DEBUG::2013-11-20 03:10:21,546::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913663::INFO::2013-11-20 03:10:21,547::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913663::DEBUG::2013-11-20 03:10:21,547::task::495::TaskManager.Task::(_debug) Task 6b5726fe-4d38-441c-9e92-c7fba7e554f5: moving from state init -> state preparing
>Thread-15913663::DEBUG::2013-11-20 03:10:21,549::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d8026d2e-2274-4a6b-9c37-8c5f7d1794d8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913663::DEBUG::2013-11-20 03:10:21,549::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913663::DEBUG::2013-11-20 03:10:21,550::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913663::DEBUG::2013-11-20 03:10:21,550::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d8026d2e-2274-4a6b-9c37-8c5f7d1794d8`::Granted request
>Thread-15913663::DEBUG::2013-11-20 03:10:21,551::task::495::TaskManager.Task::(_debug) Task 6b5726fe-4d38-441c-9e92-c7fba7e554f5: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913663::DEBUG::2013-11-20 03:10:21,552::task::495::TaskManager.Task::(_debug) Task 6b5726fe-4d38-441c-9e92-c7fba7e554f5: ref 1 aborting False
>Thread-15913663::WARNING::2013-11-20 03:10:21,554::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913663::DEBUG::2013-11-20 03:10:21,554::task::495::TaskManager.Task::(_debug) Task 6b5726fe-4d38-441c-9e92-c7fba7e554f5: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913663::DEBUG::2013-11-20 03:10:21,555::task::495::TaskManager.Task::(_debug) Task 6b5726fe-4d38-441c-9e92-c7fba7e554f5: moving from state preparing -> state finished
>Thread-15913663::DEBUG::2013-11-20 03:10:21,556::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913663::DEBUG::2013-11-20 03:10:21,556::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913663::DEBUG::2013-11-20 03:10:21,556::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913663::DEBUG::2013-11-20 03:10:21,557::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913663::DEBUG::2013-11-20 03:10:21,557::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913663::DEBUG::2013-11-20 03:10:21,557::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913663::DEBUG::2013-11-20 03:10:21,558::task::495::TaskManager.Task::(_debug) Task 6b5726fe-4d38-441c-9e92-c7fba7e554f5: ref 0 aborting False
>Thread-15913663::INFO::2013-11-20 03:10:21,558::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:10:21,760::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::DEBUG::2013-11-20 03:10:22,077::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913658::DEBUG::2013-11-20 03:10:22,245::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913631::DEBUG::2013-11-20 03:10:22,247::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913658::DEBUG::2013-11-20 03:10:22,248::task::495::TaskManager.Task::(_debug) Task 41bf2d30-79d8-4dee-93e3-f07eddda1b5b: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:10:22,249::task::495::TaskManager.Task::(_debug) Task 41bf2d30-79d8-4dee-93e3-f07eddda1b5b: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:10:22,250::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913631::DEBUG::2013-11-20 03:10:22,250::vm::358::vm.Vm::(_startUnderlyingVm) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::_ongoingCreations released
>Thread-15913635::DEBUG::2013-11-20 03:10:22,261::vm::346::vm.Vm::(_startUnderlyingVm) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::_ongoingCreations acquired
>Thread-15913658::DEBUG::2013-11-20 03:10:22,276::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913635::INFO::2013-11-20 03:10:22,282::libvirtvm::1061::vm.Vm::(_run) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::VM wrapper has started
>Thread-15913635::INFO::2013-11-20 03:10:22,315::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913658::DEBUG::2013-11-20 03:10:22,318::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913635::DEBUG::2013-11-20 03:10:22,320::task::495::TaskManager.Task::(_debug) Task 548931ca-5754-4a34-a2c3-0c27b288a873: moving from state init -> state preparing
>Thread-15913635::DEBUG::2013-11-20 03:10:22,344::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a94a39a5-d1dc-44fa-9e90-2b09aabb964f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1690' at 'public_prepareVolume'
>Thread-15913658::DEBUG::2013-11-20 03:10:22,349::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913635::DEBUG::2013-11-20 03:10:22,349::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:10:22,353::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:10:22,361::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913635::DEBUG::2013-11-20 03:10:22,361::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:10:22,362::task::495::TaskManager.Task::(_debug) Task 41bf2d30-79d8-4dee-93e3-f07eddda1b5b: ref 0 aborting False
>Thread-15913635::DEBUG::2013-11-20 03:10:22,363::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a94a39a5-d1dc-44fa-9e90-2b09aabb964f`::Granted request
>Thread-15913658::INFO::2013-11-20 03:10:22,363::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913635::DEBUG::2013-11-20 03:10:22,369::task::495::TaskManager.Task::(_debug) Task 548931ca-5754-4a34-a2c3-0c27b288a873: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913635::DEBUG::2013-11-20 03:10:22,377::task::495::TaskManager.Task::(_debug) Task 548931ca-5754-4a34-a2c3-0c27b288a873: ref 1 aborting False
>Thread-15913635::DEBUG::2013-11-20 03:10:22,411::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.cc863eeb-33db-4227-b4eb-9f4347299cf3`ReqID=`7225744f-a9b1-48fe-81ea-26e74febb937`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1695' at 'public_prepareVolume'
>Thread-15913635::DEBUG::2013-11-20 03:10:22,413::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.cc863eeb-33db-4227-b4eb-9f4347299cf3' for lock type 'exclusive'
>Thread-15913635::DEBUG::2013-11-20 03:10:22,421::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913635::DEBUG::2013-11-20 03:10:22,455::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:10:22,714::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.429326 s, 2.4 MB/s\n'; <rc> = 0
>Thread-15913635::DEBUG::2013-11-20 03:10:23,426::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913635::DEBUG::2013-11-20 03:10:24,600::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913635::DEBUG::2013-11-20 03:10:24,602::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=23 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15913665::DEBUG::2013-11-20 03:10:24,603::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('4760db3e-d3a8-4045-9153-12c36ec16ba2',) {}
>Thread-15913665::DEBUG::2013-11-20 03:10:24,604::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Disk stats not available
>Dummy-15912651::DEBUG::2013-11-20 03:10:24,814::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913665::DEBUG::2013-11-20 03:10:24,816::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Disk latency not available
>Thread-15913665::DEBUG::2013-11-20 03:10:24,818::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '49279', 'displayIp': '0', 'displayPort': u'5906', 'session': 'Unknown', 'displaySecurePort': u'5907', 'timeOffset': '-1', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet4': {'macAddr': u'00:1a:4a:a8:64:04', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet4'}}, 'vmId': '4760db3e-d3a8-4045-9153-12c36ec16ba2', 'monitorResponse': '0', 'cpuUser': '292.93', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '85899345920', 'writeLatency': '0', 'imageID': '6f97dfa5-ee35-4f37-8495-1e6b3136b92c', 'flushLatency': '0', 'truesize': '85899345920'}}, 'boot': 'cd', 'statsAge': '1.17', 'cpuIdle': '0.00', 'elapsedTime': '37', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15913666::DEBUG::2013-11-20 03:10:24,994::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('0dc7e997-e884-4493-b701-790fb37b3d80',) {}
>Thread-15913666::DEBUG::2013-11-20 03:10:24,996::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::CPU stats not available
>Thread-15913666::DEBUG::2013-11-20 03:10:24,997::libvirtvm::202::vm.Vm::(_getNetworkStats) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::Network stats not available
>Thread-15913666::DEBUG::2013-11-20 03:10:24,997::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::Disk stats not available
>Thread-15913666::DEBUG::2013-11-20 03:10:24,998::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::Disk latency not available
>Thread-15913666::DEBUG::2013-11-20 03:10:24,999::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '49674', 'displayIp': '0', 'displayPort': u'5908', 'session': 'Unknown', 'displaySecurePort': u'5909', 'timeOffset': '3600', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet5': {'state': 'unknown', 'macAddr': u'00:1a:4a:a8:64:03', 'speed': '1000', 'name': u'vnet5'}}, 'vmId': '0dc7e997-e884-4493-b701-790fb37b3d80', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '85899345920', 'writeLatency': '0', 'imageID': 'b74dd33b-748d-48df-83f2-ebcc3523e117', 'flushLatency': '0', 'truesize': '85899345920'}}, 'boot': 'cd', 'statsAge': '0.56', 'cpuIdle': '100.00', 'elapsedTime': '33', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15913635::DEBUG::2013-11-20 03:10:26,432::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.0116596 s, 43.9 kB/s\n'; <rc> = 0
>Thread-15913635::DEBUG::2013-11-20 03:10:26,433::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.0116596 s, 43.9 kB/s'], size: 512
>Thread-15913635::DEBUG::2013-11-20 03:10:26,435::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=23 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15913635::DEBUG::2013-11-20 03:10:27,025::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000144369 s, 3.5 MB/s\n'; <rc> = 0
>Thread-15913635::DEBUG::2013-11-20 03:10:27,026::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000144369 s, 3.5 MB/s'], size: 512
>Thread-15913635::INFO::2013-11-20 03:10:27,027::image::340::Storage.Image::(getChain) sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 chain=[<storage.blockVolume.BlockVolume instance at 0x7f488c120a28>] 
>Thread-15913635::DEBUG::2013-11-20 03:10:27,029::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --autobackup n --available y 6b892067-ac7e-4039-a7be-5b7387525b16/ff5e3845-0d0d-4037-a0de-a3f584a4aef5' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:10:27,539::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.112241 s, 9.1 MB/s\n'; <rc> = 0
>Thread-15913667::INFO::2013-11-20 03:10:28,200::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913667::DEBUG::2013-11-20 03:10:28,200::task::495::TaskManager.Task::(_debug) Task 46fc00d6-dddb-46fb-9e6e-de1f19c82cd5: moving from state init -> state preparing
>Thread-15913667::DEBUG::2013-11-20 03:10:28,201::task::495::TaskManager.Task::(_debug) Task 46fc00d6-dddb-46fb-9e6e-de1f19c82cd5: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000663995742798', 'lastCheck': 1384917022.0669079, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00814414024353', 'lastCheck': 1384917019.9834399, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0175859928131', 'lastCheck': 1384917021.095197, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00330805778503', 'lastCheck': 1384917022.0647261, 'valid': True, 'code': 0}}
>Thread-15913667::DEBUG::2013-11-20 03:10:28,202::task::495::TaskManager.Task::(_debug) Task 46fc00d6-dddb-46fb-9e6e-de1f19c82cd5: moving from state preparing -> state finished
>Thread-15913667::DEBUG::2013-11-20 03:10:28,202::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913667::DEBUG::2013-11-20 03:10:28,203::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913667::DEBUG::2013-11-20 03:10:28,204::task::495::TaskManager.Task::(_debug) Task 46fc00d6-dddb-46fb-9e6e-de1f19c82cd5: ref 0 aborting False
>Thread-15913667::INFO::2013-11-20 03:10:28,204::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000663995742798', 'lastCheck': 1384917022.0669079, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00814414024353', 'lastCheck': 1384917019.9834399, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0175859928131', 'lastCheck': 1384917021.095197, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00330805778503', 'lastCheck': 1384917022.0647261, 'valid': True, 'code': 0}}
>Thread-15913668::DEBUG::2013-11-20 03:10:28,231::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::CPU stats not available
>Thread-15913668::DEBUG::2013-11-20 03:10:28,231::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::Disk stats not available
>Thread-15913668::DEBUG::2013-11-20 03:10:28,232::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::Disk latency not available
>Thread-15913668::DEBUG::2013-11-20 03:10:28,232::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Disk stats not available
>Thread-15913668::DEBUG::2013-11-20 03:10:28,233::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Disk latency not available
>Thread-15913635::DEBUG::2013-11-20 03:10:29,112::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913635::DEBUG::2013-11-20 03:10:29,113::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913635::DEBUG::2013-11-20 03:10:29,114::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913635::DEBUG::2013-11-20 03:10:29,114::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.ff5e3845-0d0d-4037-a0de-a3f584a4aef5`ReqID=`1fa2644d-3391-4ad2-a3b1-c2ae45b4c5bc`::Request was made in '/usr/share/vdsm/storage/resourceFactories.py' line '149' at '__getResourceCandidatesList'
>Thread-15913635::DEBUG::2013-11-20 03:10:29,115::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.ff5e3845-0d0d-4037-a0de-a3f584a4aef5' for lock type 'exclusive'
>Thread-15913635::DEBUG::2013-11-20 03:10:29,115::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.ff5e3845-0d0d-4037-a0de-a3f584a4aef5' is free. Now locking as 'exclusive' (1 active user)
>Thread-15913635::DEBUG::2013-11-20 03:10:29,116::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.ff5e3845-0d0d-4037-a0de-a3f584a4aef5`ReqID=`1fa2644d-3391-4ad2-a3b1-c2ae45b4c5bc`::Granted request
>Thread-15913635::DEBUG::2013-11-20 03:10:29,116::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.cc863eeb-33db-4227-b4eb-9f4347299cf3' is free. Now locking as 'exclusive' (1 active user)
>Thread-15913635::DEBUG::2013-11-20 03:10:29,117::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.cc863eeb-33db-4227-b4eb-9f4347299cf3`ReqID=`7225744f-a9b1-48fe-81ea-26e74febb937`::Granted request
>Thread-15913635::DEBUG::2013-11-20 03:10:29,118::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913635::DEBUG::2013-11-20 03:10:29,119::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:10:29,561::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:10:29,992::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0515127 s, 19.9 MB/s\n'; <rc> = 0
>Thread-15913635::DEBUG::2013-11-20 03:10:30,424::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913635::DEBUG::2013-11-20 03:10:30,688::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913635::DEBUG::2013-11-20 03:10:30,690::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --refresh 6b892067-ac7e-4039-a7be-5b7387525b16/ff5e3845-0d0d-4037-a0de-a3f584a4aef5' (cwd None)
>Thread-15913635::DEBUG::2013-11-20 03:10:31,352::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913635::DEBUG::2013-11-20 03:10:31,353::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913635::DEBUG::2013-11-20 03:10:31,354::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913635::INFO::2013-11-20 03:10:31,354::volume::517::Storage.Volume::(prepare) Volume: preparing volume 6b892067-ac7e-4039-a7be-5b7387525b16/ff5e3845-0d0d-4037-a0de-a3f584a4aef5
>Thread-15913635::DEBUG::2013-11-20 03:10:31,354::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913635::DEBUG::2013-11-20 03:10:31,355::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15913670::DEBUG::2013-11-20 03:10:31,584::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('4760db3e-d3a8-4045-9153-12c36ec16ba2',) {}
>Thread-15913670::DEBUG::2013-11-20 03:10:31,585::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Disk stats not available
>Thread-15913670::DEBUG::2013-11-20 03:10:31,585::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Disk latency not available
>Thread-15913670::DEBUG::2013-11-20 03:10:31,586::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '49279', 'displayIp': '0', 'displayPort': u'5906', 'session': 'Unknown', 'displaySecurePort': u'5907', 'timeOffset': '-1', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet4': {'macAddr': u'00:1a:4a:a8:64:04', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet4'}}, 'vmId': '4760db3e-d3a8-4045-9153-12c36ec16ba2', 'monitorResponse': '0', 'cpuUser': '292.93', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '85899345920', 'writeLatency': '0', 'imageID': '6f97dfa5-ee35-4f37-8495-1e6b3136b92c', 'flushLatency': '0', 'truesize': '85899345920'}}, 'boot': 'cd', 'statsAge': '0.06', 'cpuIdle': '0.00', 'elapsedTime': '44', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15913671::DEBUG::2013-11-20 03:10:31,591::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('0dc7e997-e884-4493-b701-790fb37b3d80',) {}
>Thread-15913671::DEBUG::2013-11-20 03:10:31,591::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::CPU stats not available
>Thread-15913671::DEBUG::2013-11-20 03:10:31,592::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::Disk stats not available
>Thread-15913671::DEBUG::2013-11-20 03:10:31,592::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::Disk latency not available
>Thread-15913671::DEBUG::2013-11-20 03:10:31,593::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '49674', 'displayIp': '0', 'displayPort': u'5908', 'session': 'Unknown', 'displaySecurePort': u'5909', 'timeOffset': '3600', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet5': {'macAddr': u'00:1a:4a:a8:64:03', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet5'}}, 'vmId': '0dc7e997-e884-4493-b701-790fb37b3d80', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '85899345920', 'writeLatency': '0', 'imageID': 'b74dd33b-748d-48df-83f2-ebcc3523e117', 'flushLatency': '0', 'truesize': '85899345920'}}, 'boot': 'cd', 'statsAge': '0.74', 'cpuIdle': '100.00', 'elapsedTime': '39', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15913672::DEBUG::2013-11-20 03:10:31,746::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913672::INFO::2013-11-20 03:10:31,747::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913672::DEBUG::2013-11-20 03:10:31,747::task::495::TaskManager.Task::(_debug) Task e3952903-b5c1-41c0-b8b8-747b3be9432e: moving from state init -> state preparing
>Thread-15913672::DEBUG::2013-11-20 03:10:31,748::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913672::DEBUG::2013-11-20 03:10:31,748::task::495::TaskManager.Task::(_debug) Task e3952903-b5c1-41c0-b8b8-747b3be9432e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913672::DEBUG::2013-11-20 03:10:31,748::task::495::TaskManager.Task::(_debug) Task e3952903-b5c1-41c0-b8b8-747b3be9432e: moving from state preparing -> state finished
>Thread-15913672::DEBUG::2013-11-20 03:10:31,749::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913672::DEBUG::2013-11-20 03:10:31,749::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913672::DEBUG::2013-11-20 03:10:31,749::task::495::TaskManager.Task::(_debug) Task e3952903-b5c1-41c0-b8b8-747b3be9432e: ref 0 aborting False
>Thread-15913672::INFO::2013-11-20 03:10:31,750::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913673::DEBUG::2013-11-20 03:10:31,754::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913673::INFO::2013-11-20 03:10:31,754::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913673::DEBUG::2013-11-20 03:10:31,755::task::495::TaskManager.Task::(_debug) Task b147d4d9-00f1-48f6-b5d8-c66f3fec224d: moving from state init -> state preparing
>Thread-15913673::DEBUG::2013-11-20 03:10:31,756::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`28f9d381-8bce-42be-a68e-06b38b3d4812`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913673::DEBUG::2013-11-20 03:10:31,756::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913673::DEBUG::2013-11-20 03:10:31,757::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913673::DEBUG::2013-11-20 03:10:31,757::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`28f9d381-8bce-42be-a68e-06b38b3d4812`::Granted request
>Thread-15913673::DEBUG::2013-11-20 03:10:31,758::task::495::TaskManager.Task::(_debug) Task b147d4d9-00f1-48f6-b5d8-c66f3fec224d: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913673::DEBUG::2013-11-20 03:10:31,758::task::495::TaskManager.Task::(_debug) Task b147d4d9-00f1-48f6-b5d8-c66f3fec224d: ref 1 aborting False
>Thread-15913673::WARNING::2013-11-20 03:10:31,760::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913673::DEBUG::2013-11-20 03:10:31,761::task::495::TaskManager.Task::(_debug) Task b147d4d9-00f1-48f6-b5d8-c66f3fec224d: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913673::DEBUG::2013-11-20 03:10:31,761::task::495::TaskManager.Task::(_debug) Task b147d4d9-00f1-48f6-b5d8-c66f3fec224d: moving from state preparing -> state finished
>Thread-15913673::DEBUG::2013-11-20 03:10:31,762::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913673::DEBUG::2013-11-20 03:10:31,762::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913673::DEBUG::2013-11-20 03:10:31,763::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913673::DEBUG::2013-11-20 03:10:31,763::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913673::DEBUG::2013-11-20 03:10:31,763::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913673::DEBUG::2013-11-20 03:10:31,764::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913673::DEBUG::2013-11-20 03:10:31,765::task::495::TaskManager.Task::(_debug) Task b147d4d9-00f1-48f6-b5d8-c66f3fec224d: ref 0 aborting False
>Thread-15913673::INFO::2013-11-20 03:10:31,766::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913635::DEBUG::2013-11-20 03:10:31,899::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:10:32,004::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913635::DEBUG::2013-11-20 03:10:32,126::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913635::DEBUG::2013-11-20 03:10:32,309::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=23 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15912392::INFO::2013-11-20 03:10:32,320::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:10:32,490::task::495::TaskManager.Task::(_debug) Task 68abad21-304e-44e1-a883-e695f695a361: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:10:32,494::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`beca5588-0ce9-48c5-b9e8-b2a480e0a792`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:10:32,494::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:10:32,495::resourceManager::481::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' found in shared state and queue is empty, Joining current shared lock (2 active users)
>Thread-15912392::DEBUG::2013-11-20 03:10:32,496::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`beca5588-0ce9-48c5-b9e8-b2a480e0a792`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:10:32,497::task::495::TaskManager.Task::(_debug) Task 68abad21-304e-44e1-a883-e695f695a361: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:10:32,497::task::495::TaskManager.Task::(_debug) Task 68abad21-304e-44e1-a883-e695f695a361: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:10:32,498::task::495::TaskManager.Task::(_debug) Task 68abad21-304e-44e1-a883-e695f695a361: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:10:32,499::task::495::TaskManager.Task::(_debug) Task 68abad21-304e-44e1-a883-e695f695a361: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:10:32,499::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:10:32,500::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:10:32,501::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:10:32,501::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (1 active users)
>Thread-15912392::DEBUG::2013-11-20 03:10:32,502::task::495::TaskManager.Task::(_debug) Task 68abad21-304e-44e1-a883-e695f695a361: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:10:32,503::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913635::DEBUG::2013-11-20 03:10:32,664::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000191607 s, 2.7 MB/s\n'; <rc> = 0
>Thread-15913635::DEBUG::2013-11-20 03:10:32,664::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000191607 s, 2.7 MB/s'], size: 512
>Thread-15913635::DEBUG::2013-11-20 03:10:32,665::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=23 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15913518::INFO::2013-11-20 03:10:32,894::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:10:32,895::task::495::TaskManager.Task::(_debug) Task a887684d-d0f5-477d-9c9b-4aaec383c496: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:10:32,897::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b368d73f-0dbe-4f29-b2c3-8eedda2aac68`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:10:32,898::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:10:32,898::resourceManager::481::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' found in shared state and queue is empty, Joining current shared lock (2 active users)
>Thread-15913518::DEBUG::2013-11-20 03:10:32,899::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b368d73f-0dbe-4f29-b2c3-8eedda2aac68`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:10:32,900::task::495::TaskManager.Task::(_debug) Task a887684d-d0f5-477d-9c9b-4aaec383c496: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:10:32,900::task::495::TaskManager.Task::(_debug) Task a887684d-d0f5-477d-9c9b-4aaec383c496: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:10:32,901::task::495::TaskManager.Task::(_debug) Task a887684d-d0f5-477d-9c9b-4aaec383c496: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:10:32,901::task::495::TaskManager.Task::(_debug) Task a887684d-d0f5-477d-9c9b-4aaec383c496: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:10:32,901::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:10:32,902::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:10:32,902::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:10:32,903::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (1 active users)
>Thread-15913518::DEBUG::2013-11-20 03:10:32,903::task::495::TaskManager.Task::(_debug) Task a887684d-d0f5-477d-9c9b-4aaec383c496: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:10:32,904::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:10:32,904::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:10:32,905::task::495::TaskManager.Task::(_debug) Task 68e3ab14-cf4e-4148-a793-af3e8181b17a: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:10:32,905::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f86b241d-bf72-4873-84c5-b9e0038a2fb4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:10:32,906::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:10:32,906::resourceManager::481::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' found in shared state and queue is empty, Joining current shared lock (2 active users)
>Thread-15913518::DEBUG::2013-11-20 03:10:32,906::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f86b241d-bf72-4873-84c5-b9e0038a2fb4`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:10:32,907::task::495::TaskManager.Task::(_debug) Task 68e3ab14-cf4e-4148-a793-af3e8181b17a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:10:32,907::task::495::TaskManager.Task::(_debug) Task 68e3ab14-cf4e-4148-a793-af3e8181b17a: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:10:32,908::task::495::TaskManager.Task::(_debug) Task 68e3ab14-cf4e-4148-a793-af3e8181b17a: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:10:32,908::task::495::TaskManager.Task::(_debug) Task 68e3ab14-cf4e-4148-a793-af3e8181b17a: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:10:32,909::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:10:32,909::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:10:32,909::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:10:32,910::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (1 active users)
>Thread-15913518::DEBUG::2013-11-20 03:10:32,910::task::495::TaskManager.Task::(_debug) Task 68e3ab14-cf4e-4148-a793-af3e8181b17a: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:10:32,911::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:10:33,043::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0633458 s, 16.2 MB/s\n'; <rc> = 0
>Thread-15913635::DEBUG::2013-11-20 03:10:33,185::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000138831 s, 3.7 MB/s\n'; <rc> = 0
>Thread-15913635::DEBUG::2013-11-20 03:10:33,186::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000138831 s, 3.7 MB/s'], size: 512
>Thread-15913635::DEBUG::2013-11-20 03:10:33,187::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.ff5e3845-0d0d-4037-a0de-a3f584a4aef5`ReqID=`9b75a5f6-83da-4282-95c0-855f1457aa30`::Request was made in '/usr/share/vdsm/storage/volume.py' line '533' at 'prepare'
>Thread-15913635::DEBUG::2013-11-20 03:10:33,187::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.ff5e3845-0d0d-4037-a0de-a3f584a4aef5' for lock type 'exclusive'
>Thread-15913635::DEBUG::2013-11-20 03:10:33,188::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.ff5e3845-0d0d-4037-a0de-a3f584a4aef5' is free. Now locking as 'exclusive' (1 active user)
>Thread-15913635::DEBUG::2013-11-20 03:10:33,188::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.ff5e3845-0d0d-4037-a0de-a3f584a4aef5`ReqID=`9b75a5f6-83da-4282-95c0-855f1457aa30`::Granted request
>Thread-15913635::DEBUG::2013-11-20 03:10:33,189::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=23 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15913635::DEBUG::2013-11-20 03:10:34,456::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000151006 s, 3.4 MB/s\n'; <rc> = 0
>Thread-15913635::DEBUG::2013-11-20 03:10:34,457::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000151006 s, 3.4 MB/s'], size: 512
>Thread-15913635::DEBUG::2013-11-20 03:10:34,458::task::495::TaskManager.Task::(_debug) Task 548931ca-5754-4a34-a2c3-0c27b288a873: finished: None
>Thread-15913635::DEBUG::2013-11-20 03:10:34,458::task::495::TaskManager.Task::(_debug) Task 548931ca-5754-4a34-a2c3-0c27b288a873: moving from state preparing -> state finished
>Thread-15913635::DEBUG::2013-11-20 03:10:34,459::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913635::DEBUG::2013-11-20 03:10:34,459::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913635::DEBUG::2013-11-20 03:10:34,460::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913635::DEBUG::2013-11-20 03:10:34,460::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913635::DEBUG::2013-11-20 03:10:34,460::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913635::DEBUG::2013-11-20 03:10:34,461::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913635::DEBUG::2013-11-20 03:10:34,461::task::495::TaskManager.Task::(_debug) Task 548931ca-5754-4a34-a2c3-0c27b288a873: ref 0 aborting False
>Thread-15913635::INFO::2013-11-20 03:10:34,462::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, Return response: {'status': {'message': 'OK', 'code': 0}}
>Thread-15913635::INFO::2013-11-20 03:10:34,462::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913635::DEBUG::2013-11-20 03:10:34,463::task::495::TaskManager.Task::(_debug) Task c061563d-0e3d-4316-adbd-b8ba55de8659: moving from state init -> state preparing
>Thread-15913635::DEBUG::2013-11-20 03:10:34,463::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`77a8483d-1a24-40e0-956a-7257bb62e175`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1666' at 'public_getVolumePath'
>Thread-15913635::DEBUG::2013-11-20 03:10:34,464::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913635::DEBUG::2013-11-20 03:10:34,464::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913635::DEBUG::2013-11-20 03:10:34,465::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`77a8483d-1a24-40e0-956a-7257bb62e175`::Granted request
>Thread-15913635::DEBUG::2013-11-20 03:10:34,465::task::495::TaskManager.Task::(_debug) Task c061563d-0e3d-4316-adbd-b8ba55de8659: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913635::DEBUG::2013-11-20 03:10:34,466::task::495::TaskManager.Task::(_debug) Task c061563d-0e3d-4316-adbd-b8ba55de8659: ref 1 aborting False
>Thread-15913635::DEBUG::2013-11-20 03:10:34,466::task::495::TaskManager.Task::(_debug) Task c061563d-0e3d-4316-adbd-b8ba55de8659: finished: {'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/cc863eeb-33db-4227-b4eb-9f4347299cf3/ff5e3845-0d0d-4037-a0de-a3f584a4aef5'}
>Thread-15913635::DEBUG::2013-11-20 03:10:34,467::task::495::TaskManager.Task::(_debug) Task c061563d-0e3d-4316-adbd-b8ba55de8659: moving from state preparing -> state finished
>Thread-15913635::DEBUG::2013-11-20 03:10:34,467::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913635::DEBUG::2013-11-20 03:10:34,467::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913635::DEBUG::2013-11-20 03:10:34,468::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913635::DEBUG::2013-11-20 03:10:34,468::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913635::DEBUG::2013-11-20 03:10:34,469::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913635::DEBUG::2013-11-20 03:10:34,469::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913635::DEBUG::2013-11-20 03:10:34,469::task::495::TaskManager.Task::(_debug) Task c061563d-0e3d-4316-adbd-b8ba55de8659: ref 0 aborting False
>Thread-15913635::INFO::2013-11-20 03:10:34,470::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, Return response: {'status': {'message': 'OK', 'code': 0}, 'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/cc863eeb-33db-4227-b4eb-9f4347299cf3/ff5e3845-0d0d-4037-a0de-a3f584a4aef5'}
>Thread-15913635::INFO::2013-11-20 03:10:34,471::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913635::DEBUG::2013-11-20 03:10:34,472::task::495::TaskManager.Task::(_debug) Task a7e9eb23-6840-474d-a4a4-6e3a71a5044a: moving from state init -> state preparing
>Thread-15913635::DEBUG::2013-11-20 03:10:34,473::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`70e974a9-90c7-4235-986c-4053c3f10c23`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913635::DEBUG::2013-11-20 03:10:34,473::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913635::DEBUG::2013-11-20 03:10:34,474::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913635::DEBUG::2013-11-20 03:10:34,474::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`70e974a9-90c7-4235-986c-4053c3f10c23`::Granted request
>Thread-15913635::DEBUG::2013-11-20 03:10:34,475::task::495::TaskManager.Task::(_debug) Task a7e9eb23-6840-474d-a4a4-6e3a71a5044a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913635::DEBUG::2013-11-20 03:10:34,475::task::495::TaskManager.Task::(_debug) Task a7e9eb23-6840-474d-a4a4-6e3a71a5044a: ref 1 aborting False
>Thread-15913635::DEBUG::2013-11-20 03:10:34,475::task::495::TaskManager.Task::(_debug) Task a7e9eb23-6840-474d-a4a4-6e3a71a5044a: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913635::DEBUG::2013-11-20 03:10:34,476::task::495::TaskManager.Task::(_debug) Task a7e9eb23-6840-474d-a4a4-6e3a71a5044a: moving from state preparing -> state finished
>Thread-15913635::DEBUG::2013-11-20 03:10:34,476::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913635::DEBUG::2013-11-20 03:10:34,476::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913635::DEBUG::2013-11-20 03:10:34,477::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913635::DEBUG::2013-11-20 03:10:34,477::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913635::DEBUG::2013-11-20 03:10:34,477::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913635::DEBUG::2013-11-20 03:10:34,478::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913635::DEBUG::2013-11-20 03:10:34,478::task::495::TaskManager.Task::(_debug) Task a7e9eb23-6840-474d-a4a4-6e3a71a5044a: ref 0 aborting False
>Thread-15913635::INFO::2013-11-20 03:10:34,478::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913635::INFO::2013-11-20 03:10:34,479::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16)
>Thread-15913635::DEBUG::2013-11-20 03:10:34,479::task::495::TaskManager.Task::(_debug) Task cd6890f4-5509-4ba1-b908-c0d148b6db36: moving from state init -> state preparing
>Thread-15913635::INFO::2013-11-20 03:10:34,480::blockSD::674::Storage.StorageDomain::(validate) sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16
>Thread-15913635::DEBUG::2013-11-20 03:10:34,480::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913635::DEBUG::2013-11-20 03:10:34,481::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913635::DEBUG::2013-11-20 03:10:34,481::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15913635::DEBUG::2013-11-20 03:10:34,482::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15913635::DEBUG::2013-11-20 03:10:34,482::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913635::DEBUG::2013-11-20 03:10:34,483::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:10:35,963::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913675::DEBUG::2013-11-20 03:10:35,993::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('4760db3e-d3a8-4045-9153-12c36ec16ba2',) {}
>Thread-15913675::DEBUG::2013-11-20 03:10:35,999::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Disk stats not available
>Thread-15913675::DEBUG::2013-11-20 03:10:36,194::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Disk latency not available
>Thread-15913675::DEBUG::2013-11-20 03:10:36,196::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '49279', 'displayIp': '0', 'displayPort': u'5906', 'session': 'Unknown', 'displaySecurePort': u'5907', 'timeOffset': '-1', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet4': {'macAddr': u'00:1a:4a:a8:64:04', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet4'}}, 'vmId': '4760db3e-d3a8-4045-9153-12c36ec16ba2', 'monitorResponse': '0', 'cpuUser': '292.93', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '85899345920', 'writeLatency': '0', 'imageID': '6f97dfa5-ee35-4f37-8495-1e6b3136b92c', 'flushLatency': '0', 'truesize': '85899345920'}}, 'boot': 'cd', 'statsAge': '0.04', 'cpuIdle': '0.00', 'elapsedTime': '48', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15913676::DEBUG::2013-11-20 03:10:36,203::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('0dc7e997-e884-4493-b701-790fb37b3d80',) {}
>Thread-15913676::DEBUG::2013-11-20 03:10:36,204::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::CPU stats not available
>Thread-15913676::DEBUG::2013-11-20 03:10:36,204::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::Disk stats not available
>Thread-15913676::DEBUG::2013-11-20 03:10:36,205::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::Disk latency not available
>Thread-15913676::DEBUG::2013-11-20 03:10:36,205::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '49674', 'displayIp': '0', 'displayPort': u'5908', 'session': 'Unknown', 'displaySecurePort': u'5909', 'timeOffset': '3600', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet5': {'macAddr': u'00:1a:4a:a8:64:03', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet5'}}, 'vmId': '0dc7e997-e884-4493-b701-790fb37b3d80', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '85899345920', 'writeLatency': '0', 'imageID': 'b74dd33b-748d-48df-83f2-ebcc3523e117', 'flushLatency': '0', 'truesize': '85899345920'}}, 'boot': 'cd', 'statsAge': '0.26', 'cpuIdle': '100.00', 'elapsedTime': '44', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:10:37,648::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0438783 s, 23.3 MB/s\n'; <rc> = 0
>Thread-15913635::DEBUG::2013-11-20 03:10:37,810::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913635::DEBUG::2013-11-20 03:10:37,812::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913635::DEBUG::2013-11-20 03:10:37,813::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'_SHA_CKSUM=716ea7492d9134a4b13de27ac61a336ad474727c', u'POOL_SPM_ID=2', u'POOL_SPM_LVER=287']
>Thread-15913635::DEBUG::2013-11-20 03:10:37,815::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`16ed6b6d-1659-4db5-a15d-cae793c774d0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1297' at 'public_getStorageDomainInfo'
>Thread-15913635::DEBUG::2013-11-20 03:10:37,815::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913635::DEBUG::2013-11-20 03:10:37,816::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913635::DEBUG::2013-11-20 03:10:37,816::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`16ed6b6d-1659-4db5-a15d-cae793c774d0`::Granted request
>Thread-15913635::DEBUG::2013-11-20 03:10:37,817::task::495::TaskManager.Task::(_debug) Task cd6890f4-5509-4ba1-b908-c0d148b6db36: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913635::DEBUG::2013-11-20 03:10:37,817::task::495::TaskManager.Task::(_debug) Task cd6890f4-5509-4ba1-b908-c0d148b6db36: ref 1 aborting False
>Thread-15913635::WARNING::2013-11-20 03:10:37,819::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913635::DEBUG::2013-11-20 03:10:37,820::task::495::TaskManager.Task::(_debug) Task cd6890f4-5509-4ba1-b908-c0d148b6db36: finished: {'info': {'uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'vguuid': 'aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', 'lver': 287, 'state': 'OK', 'version': '2', 'role': 'Master', 'master_ver': 1, 'spm_id': 2, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'OS'}}
>Thread-15913635::DEBUG::2013-11-20 03:10:37,820::task::495::TaskManager.Task::(_debug) Task cd6890f4-5509-4ba1-b908-c0d148b6db36: moving from state preparing -> state finished
>Thread-15913635::DEBUG::2013-11-20 03:10:37,821::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913635::DEBUG::2013-11-20 03:10:37,821::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913635::DEBUG::2013-11-20 03:10:37,821::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913635::DEBUG::2013-11-20 03:10:37,822::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913635::DEBUG::2013-11-20 03:10:37,822::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913635::DEBUG::2013-11-20 03:10:37,823::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913635::DEBUG::2013-11-20 03:10:37,823::task::495::TaskManager.Task::(_debug) Task cd6890f4-5509-4ba1-b908-c0d148b6db36: ref 0 aborting False
>Thread-15913635::INFO::2013-11-20 03:10:37,824::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'vguuid': 'aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', 'lver': 287, 'state': 'OK', 'version': '2', 'role': 'Master', 'master_ver': 1, 'spm_id': 2, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'OS'}}
>Thread-15913635::DEBUG::2013-11-20 03:10:37,835::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/libexec/vdsm/hooks/before_vm_start/50_vhostmd' (cwd None)
>Thread-15913635::DEBUG::2013-11-20 03:10:38,329::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913635::INFO::2013-11-20 03:10:38,330::hooks::51::root::(_runHooksDir) 
>Thread-15913635::DEBUG::2013-11-20 03:10:38,337::libvirtvm::1077::vm.Vm::(_run) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::<?xml version="1.0" encoding="utf-8"?>
><domain type="kvm">
>	<name>KLEE-Print01</name>
>	<uuid>11b5c641-e4f7-487e-9758-c42cb014e2d2</uuid>
>	<memory>2097152</memory>
>	<currentMemory>2097152</currentMemory>
>	<vcpu>2</vcpu>
>	<devices>
>		<disk device="disk" type="block">
>			<source dev="/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/cc863eeb-33db-4227-b4eb-9f4347299cf3/ff5e3845-0d0d-4037-a0de-a3f584a4aef5"/>
>			<target bus="virtio" dev="vda"/>
>			<serial>27-b4eb-9f4347299cf3</serial>
>			<driver cache="none" error_policy="stop" io="native" name="qemu" type="qcow2"/>
>		</disk>
>		<disk device="cdrom" type="file">
>			<source file="/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO"/>
>			<target bus="ide" dev="hdc"/>
>		</disk>
>		<controller index="0" ports="16" type="virtio-serial"/>
>		<channel type="unix">
>			<target name="com.redhat.rhevm.vdsm" type="virtio"/>
>			<source mode="bind" path="/var/lib/libvirt/qemu/channels/KLEE-Print01.com.redhat.rhevm.vdsm"/>
>		</channel>
>		<memballoon model="none"/>
>		<interface type="bridge">
>			<mac address="00:1a:4a:a8:64:0d"/>
>			<model type="virtio"/>
>			<source bridge="rhevm"/>
>		</interface>
>		<input bus="ps2" type="mouse"/>
>		<video>
>			<model heads="1" type="qxl" vram="65536"/>
>		</video>
>		<channel type="spicevmc">
>			<target name="com.redhat.spice.0" type="virtio"/>
>		</channel>
>		<graphics autoport="yes" keymap="en-us" listen="0" passwd="*****" passwdValidTo="1970-01-01T00:00:01" port="-1" tlsPort="-1" type="spice">
>			<channel mode="secure" name="main"/>
>			<channel mode="secure" name="inputs"/>
>		</graphics>
>	</devices>
>	<os>
>		<type arch="x86_64" machine="rhel6.2.0">hvm</type>
>		<boot dev="cdrom"/>
>		<smbios mode="sysinfo"/>
>	</os>
>	<sysinfo type="smbios">
>		<system>
>			<entry name="manufacturer">Red Hat</entry>
>			<entry name="product">RHEV Hypervisor</entry>
>			<entry name="version">6.3-20121012.0.el6_3</entry>
>			<entry name="serial">30343536-3138-5A43-4A32-333630373442_AC:16:2D:76:EB:F0</entry>
>			<entry name="uuid">11b5c641-e4f7-487e-9758-c42cb014e2d2</entry>
>		</system>
>	</sysinfo>
>	<clock adjustment="-1" offset="variable">
>		<timer name="rtc" tickpolicy="catchup"/>
>	</clock>
>	<features>
>		<acpi/>
>	</features>
>	<cpu match="exact">
>		<model>Westmere</model>
>		<topology cores="1" sockets="2" threads="1"/>
>	</cpu>
></domain>
>
>Thread-15913678::DEBUG::2013-11-20 03:10:39,228::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('4760db3e-d3a8-4045-9153-12c36ec16ba2',) {}
>Thread-15913678::DEBUG::2013-11-20 03:10:39,229::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Disk stats not available
>Thread-15913678::DEBUG::2013-11-20 03:10:39,229::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Disk latency not available
>Thread-15913678::DEBUG::2013-11-20 03:10:39,230::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '49279', 'displayIp': '0', 'displayPort': u'5906', 'session': 'Unknown', 'displaySecurePort': u'5907', 'timeOffset': '-1', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet4': {'macAddr': u'00:1a:4a:a8:64:04', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet4'}}, 'vmId': '4760db3e-d3a8-4045-9153-12c36ec16ba2', 'monitorResponse': '0', 'cpuUser': '403.48', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '85899345920', 'writeLatency': '0', 'imageID': '6f97dfa5-ee35-4f37-8495-1e6b3136b92c', 'flushLatency': '0', 'truesize': '85899345920'}}, 'boot': 'cd', 'statsAge': '1.20', 'cpuIdle': '0.00', 'elapsedTime': '51', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15913679::DEBUG::2013-11-20 03:10:39,234::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('0dc7e997-e884-4493-b701-790fb37b3d80',) {}
>Thread-15913679::DEBUG::2013-11-20 03:10:39,235::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::CPU stats not available
>Thread-15913679::DEBUG::2013-11-20 03:10:39,235::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::Disk stats not available
>Thread-15913679::DEBUG::2013-11-20 03:10:39,235::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::Disk latency not available
>Thread-15913679::DEBUG::2013-11-20 03:10:39,236::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '49674', 'displayIp': '0', 'displayPort': u'5908', 'session': 'Unknown', 'displaySecurePort': u'5909', 'timeOffset': '3600', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet5': {'macAddr': u'00:1a:4a:a8:64:03', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet5'}}, 'vmId': '0dc7e997-e884-4493-b701-790fb37b3d80', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '85899345920', 'writeLatency': '0', 'imageID': 'b74dd33b-748d-48df-83f2-ebcc3523e117', 'flushLatency': '0', 'truesize': '85899345920'}}, 'boot': 'cd', 'statsAge': '0.21', 'cpuIdle': '100.00', 'elapsedTime': '47', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:10:39,657::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:10:40,114::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0474614 s, 21.6 MB/s\n'; <rc> = 0
>Thread-15913680::DEBUG::2013-11-20 03:10:42,060::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913680::INFO::2013-11-20 03:10:42,061::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913680::DEBUG::2013-11-20 03:10:42,062::task::495::TaskManager.Task::(_debug) Task 17ccea81-c0df-4296-9841-dc4c98705ce2: moving from state init -> state preparing
>Thread-15913680::DEBUG::2013-11-20 03:10:42,062::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913680::DEBUG::2013-11-20 03:10:42,063::task::495::TaskManager.Task::(_debug) Task 17ccea81-c0df-4296-9841-dc4c98705ce2: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913680::DEBUG::2013-11-20 03:10:42,063::task::495::TaskManager.Task::(_debug) Task 17ccea81-c0df-4296-9841-dc4c98705ce2: moving from state preparing -> state finished
>Thread-15913680::DEBUG::2013-11-20 03:10:42,063::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913680::DEBUG::2013-11-20 03:10:42,064::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913680::DEBUG::2013-11-20 03:10:42,064::task::495::TaskManager.Task::(_debug) Task 17ccea81-c0df-4296-9841-dc4c98705ce2: ref 0 aborting False
>Thread-15913680::INFO::2013-11-20 03:10:42,065::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913681::DEBUG::2013-11-20 03:10:42,068::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913681::INFO::2013-11-20 03:10:42,069::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913681::DEBUG::2013-11-20 03:10:42,069::task::495::TaskManager.Task::(_debug) Task 496be271-d34a-4515-9d3e-09649851cbc0: moving from state init -> state preparing
>Thread-15913681::DEBUG::2013-11-20 03:10:42,070::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a669af6a-45da-4402-970b-4165a35f7bf1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913681::DEBUG::2013-11-20 03:10:42,071::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913681::DEBUG::2013-11-20 03:10:42,071::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913681::DEBUG::2013-11-20 03:10:42,072::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a669af6a-45da-4402-970b-4165a35f7bf1`::Granted request
>Thread-15913681::DEBUG::2013-11-20 03:10:42,072::task::495::TaskManager.Task::(_debug) Task 496be271-d34a-4515-9d3e-09649851cbc0: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913681::DEBUG::2013-11-20 03:10:42,073::task::495::TaskManager.Task::(_debug) Task 496be271-d34a-4515-9d3e-09649851cbc0: ref 1 aborting False
>Thread-15913681::WARNING::2013-11-20 03:10:42,074::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913681::DEBUG::2013-11-20 03:10:42,075::task::495::TaskManager.Task::(_debug) Task 496be271-d34a-4515-9d3e-09649851cbc0: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913681::DEBUG::2013-11-20 03:10:42,076::task::495::TaskManager.Task::(_debug) Task 496be271-d34a-4515-9d3e-09649851cbc0: moving from state preparing -> state finished
>Thread-15913681::DEBUG::2013-11-20 03:10:42,076::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913681::DEBUG::2013-11-20 03:10:42,076::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913681::DEBUG::2013-11-20 03:10:42,077::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913681::DEBUG::2013-11-20 03:10:42,077::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913681::DEBUG::2013-11-20 03:10:42,078::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913681::DEBUG::2013-11-20 03:10:42,078::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913681::DEBUG::2013-11-20 03:10:42,079::task::495::TaskManager.Task::(_debug) Task 496be271-d34a-4515-9d3e-09649851cbc0: ref 0 aborting False
>Thread-15913681::INFO::2013-11-20 03:10:42,079::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:10:42,124::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913683::DEBUG::2013-11-20 03:10:42,313::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('4760db3e-d3a8-4045-9153-12c36ec16ba2',) {}
>Thread-15913683::DEBUG::2013-11-20 03:10:42,315::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Disk stats not available
>Thread-15913683::DEBUG::2013-11-20 03:10:42,316::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Disk latency not available
>Thread-15913683::DEBUG::2013-11-20 03:10:42,317::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '49279', 'displayIp': '0', 'displayPort': u'5906', 'session': 'Unknown', 'displaySecurePort': u'5907', 'timeOffset': '-1', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet4': {'macAddr': u'00:1a:4a:a8:64:04', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet4'}}, 'vmId': '4760db3e-d3a8-4045-9153-12c36ec16ba2', 'monitorResponse': '0', 'cpuUser': '403.48', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '85899345920', 'writeLatency': '0', 'imageID': '6f97dfa5-ee35-4f37-8495-1e6b3136b92c', 'flushLatency': '0', 'truesize': '85899345920'}}, 'boot': 'cd', 'statsAge': '0.29', 'cpuIdle': '0.00', 'elapsedTime': '54', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15913684::DEBUG::2013-11-20 03:10:42,322::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('0dc7e997-e884-4493-b701-790fb37b3d80',) {}
>Thread-15913684::DEBUG::2013-11-20 03:10:42,323::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::Disk stats not available
>Thread-15913684::DEBUG::2013-11-20 03:10:42,323::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::Disk latency not available
>Thread-15913684::DEBUG::2013-11-20 03:10:42,324::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '49674', 'displayIp': '0', 'displayPort': u'5908', 'session': 'Unknown', 'displaySecurePort': u'5909', 'timeOffset': '3600', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet5': {'macAddr': u'00:1a:4a:a8:64:03', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet5'}}, 'vmId': '0dc7e997-e884-4493-b701-790fb37b3d80', 'monitorResponse': '0', 'cpuUser': '142.45', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '85899345920', 'writeLatency': '0', 'imageID': 'b74dd33b-748d-48df-83f2-ebcc3523e117', 'flushLatency': '0', 'truesize': '85899345920'}}, 'boot': 'cd', 'statsAge': '3.30', 'cpuIdle': '0.00', 'elapsedTime': '50', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:10:42,806::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0491502 s, 20.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:10:44,817::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913685::INFO::2013-11-20 03:10:45,343::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913685::DEBUG::2013-11-20 03:10:45,344::task::495::TaskManager.Task::(_debug) Task 288edaed-2936-4af9-8d53-3672328b9302: moving from state init -> state preparing
>Thread-15913685::DEBUG::2013-11-20 03:10:45,345::task::495::TaskManager.Task::(_debug) Task 288edaed-2936-4af9-8d53-3672328b9302: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000231981277466', 'lastCheck': 1384917044.0317581, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00153398513794', 'lastCheck': 1384917042.032937, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00118613243103', 'lastCheck': 1384917043.0324719, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000571012496948', 'lastCheck': 1384917044.0299499, 'valid': True, 'code': 0}}
>Thread-15913685::DEBUG::2013-11-20 03:10:45,345::task::495::TaskManager.Task::(_debug) Task 288edaed-2936-4af9-8d53-3672328b9302: moving from state preparing -> state finished
>Thread-15913685::DEBUG::2013-11-20 03:10:45,346::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913685::DEBUG::2013-11-20 03:10:45,347::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913685::DEBUG::2013-11-20 03:10:45,347::task::495::TaskManager.Task::(_debug) Task 288edaed-2936-4af9-8d53-3672328b9302: ref 0 aborting False
>Thread-15913685::INFO::2013-11-20 03:10:45,348::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000231981277466', 'lastCheck': 1384917044.0317581, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00153398513794', 'lastCheck': 1384917042.032937, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00118613243103', 'lastCheck': 1384917043.0324719, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000571012496948', 'lastCheck': 1384917044.0299499, 'valid': True, 'code': 0}}
>Thread-15913686::DEBUG::2013-11-20 03:10:45,362::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::Disk stats not available
>Thread-15913686::DEBUG::2013-11-20 03:10:45,363::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::Disk latency not available
>Thread-15913686::DEBUG::2013-11-20 03:10:45,363::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Disk stats not available
>Thread-15913686::DEBUG::2013-11-20 03:10:45,364::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Disk latency not available
>Dummy-15912651::DEBUG::2013-11-20 03:10:45,423::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.074653 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:10:47,434::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:10:47,842::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.058134 s, 17.6 MB/s\n'; <rc> = 0
>Thread-15913688::DEBUG::2013-11-20 03:10:48,887::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('4760db3e-d3a8-4045-9153-12c36ec16ba2',) {}
>Thread-15913688::DEBUG::2013-11-20 03:10:48,888::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Disk stats not available
>Thread-15913688::DEBUG::2013-11-20 03:10:48,888::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Disk latency not available
>Thread-15913688::DEBUG::2013-11-20 03:10:48,889::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '49279', 'displayIp': '0', 'displayPort': u'5906', 'session': 'Unknown', 'displaySecurePort': u'5907', 'timeOffset': '-1', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet4': {'macAddr': u'00:1a:4a:a8:64:04', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet4'}}, 'vmId': '4760db3e-d3a8-4045-9153-12c36ec16ba2', 'monitorResponse': '0', 'cpuUser': '403.48', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '85899345920', 'writeLatency': '0', 'imageID': '6f97dfa5-ee35-4f37-8495-1e6b3136b92c', 'flushLatency': '0', 'truesize': '85899345920'}}, 'boot': 'cd', 'statsAge': '1.99', 'cpuIdle': '0.00', 'elapsedTime': '61', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15913689::DEBUG::2013-11-20 03:10:48,893::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('0dc7e997-e884-4493-b701-790fb37b3d80',) {}
>Thread-15913689::DEBUG::2013-11-20 03:10:48,893::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::Disk stats not available
>Thread-15913689::DEBUG::2013-11-20 03:10:48,894::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::Disk latency not available
>Thread-15913689::DEBUG::2013-11-20 03:10:48,895::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '49674', 'displayIp': '0', 'displayPort': u'5908', 'session': 'Unknown', 'displaySecurePort': u'5909', 'timeOffset': '3600', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet5': {'macAddr': u'00:1a:4a:a8:64:03', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet5'}}, 'vmId': '0dc7e997-e884-4493-b701-790fb37b3d80', 'monitorResponse': '0', 'cpuUser': '142.45', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '85899345920', 'writeLatency': '0', 'imageID': 'b74dd33b-748d-48df-83f2-ebcc3523e117', 'flushLatency': '0', 'truesize': '85899345920'}}, 'boot': 'cd', 'statsAge': '2.00', 'cpuIdle': '0.00', 'elapsedTime': '56', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:10:49,851::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:10:50,289::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0457747 s, 22.4 MB/s\n'; <rc> = 0
>Thread-15913691::DEBUG::2013-11-20 03:10:52,236::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('0dc7e997-e884-4493-b701-790fb37b3d80',) {}
>Thread-15913691::DEBUG::2013-11-20 03:10:52,237::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::Disk stats not available
>Thread-15913691::DEBUG::2013-11-20 03:10:52,237::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::Disk latency not available
>Thread-15913691::DEBUG::2013-11-20 03:10:52,238::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '49674', 'displayIp': '0', 'displayPort': u'5908', 'session': 'Unknown', 'displaySecurePort': u'5909', 'timeOffset': '3600', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet5': {'macAddr': u'00:1a:4a:a8:64:03', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet5'}}, 'vmId': '0dc7e997-e884-4493-b701-790fb37b3d80', 'monitorResponse': '0', 'cpuUser': '142.45', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '85899345920', 'writeLatency': '0', 'imageID': 'b74dd33b-748d-48df-83f2-ebcc3523e117', 'flushLatency': '0', 'truesize': '85899345920'}}, 'boot': 'cd', 'statsAge': '1.34', 'cpuIdle': '0.00', 'elapsedTime': '60', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:10:52,299::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913692::DEBUG::2013-11-20 03:10:52,828::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913692::INFO::2013-11-20 03:10:56,296::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>libvirtEventLoop::DEBUG::2013-11-20 03:10:56,321::libvirtvm::1401::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::event Started detail 0 opaque None
>Thread-15913692::DEBUG::2013-11-20 03:10:56,339::task::495::TaskManager.Task::(_debug) Task 5f3bc5b9-8751-451d-bc6a-fdbff25653b8: moving from state init -> state preparing
>Thread-15913692::DEBUG::2013-11-20 03:10:56,486::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913692::DEBUG::2013-11-20 03:10:56,492::task::495::TaskManager.Task::(_debug) Task 5f3bc5b9-8751-451d-bc6a-fdbff25653b8: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913692::DEBUG::2013-11-20 03:10:56,494::task::495::TaskManager.Task::(_debug) Task 5f3bc5b9-8751-451d-bc6a-fdbff25653b8: moving from state preparing -> state finished
>Thread-15913692::DEBUG::2013-11-20 03:10:56,494::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913692::DEBUG::2013-11-20 03:10:56,495::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913692::DEBUG::2013-11-20 03:10:56,496::task::495::TaskManager.Task::(_debug) Task 5f3bc5b9-8751-451d-bc6a-fdbff25653b8: ref 0 aborting False
>Thread-15913692::INFO::2013-11-20 03:10:56,496::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913694::DEBUG::2013-11-20 03:10:56,501::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913694::INFO::2013-11-20 03:10:56,503::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913694::DEBUG::2013-11-20 03:10:56,503::task::495::TaskManager.Task::(_debug) Task 037f9f2a-6d1e-453b-9634-b9e7d672e9c0: moving from state init -> state preparing
>Thread-15913694::DEBUG::2013-11-20 03:10:56,504::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6aa110e3-d866-4843-9eea-07ed58e5ad10`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913694::DEBUG::2013-11-20 03:10:56,505::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913694::DEBUG::2013-11-20 03:10:56,506::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913694::DEBUG::2013-11-20 03:10:56,507::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6aa110e3-d866-4843-9eea-07ed58e5ad10`::Granted request
>Thread-15913694::DEBUG::2013-11-20 03:10:56,509::task::495::TaskManager.Task::(_debug) Task 037f9f2a-6d1e-453b-9634-b9e7d672e9c0: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913694::DEBUG::2013-11-20 03:10:56,509::task::495::TaskManager.Task::(_debug) Task 037f9f2a-6d1e-453b-9634-b9e7d672e9c0: ref 1 aborting False
>Thread-15913694::WARNING::2013-11-20 03:10:56,511::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913694::DEBUG::2013-11-20 03:10:56,512::task::495::TaskManager.Task::(_debug) Task 037f9f2a-6d1e-453b-9634-b9e7d672e9c0: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913694::DEBUG::2013-11-20 03:10:56,513::task::495::TaskManager.Task::(_debug) Task 037f9f2a-6d1e-453b-9634-b9e7d672e9c0: moving from state preparing -> state finished
>Thread-15913694::DEBUG::2013-11-20 03:10:56,514::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913694::DEBUG::2013-11-20 03:10:56,514::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913694::DEBUG::2013-11-20 03:10:56,515::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913694::DEBUG::2013-11-20 03:10:56,516::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913694::DEBUG::2013-11-20 03:10:56,517::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913694::DEBUG::2013-11-20 03:10:56,517::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913694::DEBUG::2013-11-20 03:10:56,518::task::495::TaskManager.Task::(_debug) Task 037f9f2a-6d1e-453b-9634-b9e7d672e9c0: ref 0 aborting False
>Thread-15913694::INFO::2013-11-20 03:10:56,518::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913635::DEBUG::2013-11-20 03:10:56,617::utils::307::vm.Vm::(start) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::Start statistics collection
>Thread-15913695::DEBUG::2013-11-20 03:10:56,620::utils::336::vm.Vm::(run) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::Stats thread started
>Thread-15913696::DEBUG::2013-11-20 03:10:56,622::guestIF::47::root::(_connect) Attempting connection to /var/lib/libvirt/qemu/channels/KLEE-Print01.com.redhat.rhevm.vdsm
>Thread-15913696::DEBUG::2013-11-20 03:10:56,624::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /usr/share/vdsm/prepare-vmchannel /var/lib/libvirt/qemu/channels/KLEE-Print01.com.redhat.rhevm.vdsm' (cwd None)
>Thread-15913635::WARNING::2013-11-20 03:10:56,821::libvirtvm::1124::vm.Vm::(_readPauseCode) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::_readPauseCode unsupported by libvirt vm
>Thread-15913635::DEBUG::2013-11-20 03:10:56,824::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /usr/share/vdsm/get-vm-pid KLEE-Print01' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:10:57,052::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.205391 s, 5.0 MB/s\n'; <rc> = 0
>Thread-15913696::DEBUG::2013-11-20 03:10:57,055::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913696::DEBUG::2013-11-20 03:10:57,059::guestIF::53::root::(_connect) connected to /var/lib/libvirt/qemu/channels/KLEE-Print01.com.redhat.rhevm.vdsm
>Thread-15913635::DEBUG::2013-11-20 03:10:57,214::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913635::DEBUG::2013-11-20 03:10:57,221::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /sbin/service ksmtuned retune' (cwd None)
>Thread-15913695::INFO::2013-11-20 03:10:57,396::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:10:57,398::task::495::TaskManager.Task::(_debug) Task 51582c95-42b7-4322-9e4c-750f6a713988: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:10:57,400::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0f4b4cd8-2966-43a5-915e-ca3a4ed21f9a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:10:57,401::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:10:57,402::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:10:57,402::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0f4b4cd8-2966-43a5-915e-ca3a4ed21f9a`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:10:57,403::task::495::TaskManager.Task::(_debug) Task 51582c95-42b7-4322-9e4c-750f6a713988: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:10:57,404::task::495::TaskManager.Task::(_debug) Task 51582c95-42b7-4322-9e4c-750f6a713988: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:10:57,405::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913695::DEBUG::2013-11-20 03:10:57,406::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15913635::DEBUG::2013-11-20 03:10:57,761::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913635::DEBUG::2013-11-20 03:10:57,763::vm::358::vm.Vm::(_startUnderlyingVm) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::_ongoingCreations released
>Thread-15913695::DEBUG::2013-11-20 03:10:58,319::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913695::DEBUG::2013-11-20 03:10:58,490::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913695::DEBUG::2013-11-20 03:10:58,491::task::495::TaskManager.Task::(_debug) Task 51582c95-42b7-4322-9e4c-750f6a713988: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:10:58,492::task::495::TaskManager.Task::(_debug) Task 51582c95-42b7-4322-9e4c-750f6a713988: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:10:58,492::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:10:58,492::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:10:58,493::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:10:58,493::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:10:58,494::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:10:58,494::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:10:58,494::task::495::TaskManager.Task::(_debug) Task 51582c95-42b7-4322-9e4c-750f6a713988: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:10:58,495::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:10:59,073::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913698::DEBUG::2013-11-20 03:10:59,373::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('11b5c641-e4f7-487e-9758-c42cb014e2d2',) {}
>Thread-15913698::DEBUG::2013-11-20 03:10:59,374::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::CPU stats not available
>Thread-15913698::DEBUG::2013-11-20 03:10:59,375::libvirtvm::202::vm.Vm::(_getNetworkStats) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::Network stats not available
>Thread-15913698::DEBUG::2013-11-20 03:10:59,375::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::Disk stats not available
>Thread-15913698::DEBUG::2013-11-20 03:10:59,375::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::Disk latency not available
>Thread-15913698::DEBUG::2013-11-20 03:10:59,376::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '50239', 'displayIp': '0', 'displayPort': u'5910', 'session': 'Unknown', 'displaySecurePort': u'5911', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet6': {'state': 'unknown', 'macAddr': u'00:1a:4a:a8:64:0d', 'speed': '1000', 'name': u'vnet6'}}, 'vmId': '11b5c641-e4f7-487e-9758-c42cb014e2d2', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '20401094656', 'writeLatency': '0', 'imageID': 'cc863eeb-33db-4227-b4eb-9f4347299cf3', 'flushLatency': '0', 'truesize': '20401094656'}}, 'boot': 'd', 'statsAge': '2.75', 'cpuIdle': '100.00', 'elapsedTime': '64', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '-1', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:10:59,491::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0488093 s, 21.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:11:01,505::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:11:01,957::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0559277 s, 18.3 MB/s\n'; <rc> = 0
>Thread-15913699::INFO::2013-11-20 03:11:02,496::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913699::DEBUG::2013-11-20 03:11:02,496::task::495::TaskManager.Task::(_debug) Task 8826498a-6dcd-44f8-b291-857dd03a9156: moving from state init -> state preparing
>Thread-15913699::DEBUG::2013-11-20 03:11:02,497::task::495::TaskManager.Task::(_debug) Task 8826498a-6dcd-44f8-b291-857dd03a9156: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00118899345398', 'lastCheck': 1384917057.4003839, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.000864028930664', 'lastCheck': 1384917052.053179, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00623917579651', 'lastCheck': 1384917056.493603, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00217604637146', 'lastCheck': 1384917057.3941691, 'valid': True, 'code': 0}}
>Thread-15913699::DEBUG::2013-11-20 03:11:02,498::task::495::TaskManager.Task::(_debug) Task 8826498a-6dcd-44f8-b291-857dd03a9156: moving from state preparing -> state finished
>Thread-15913699::DEBUG::2013-11-20 03:11:02,498::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913699::DEBUG::2013-11-20 03:11:02,499::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913699::DEBUG::2013-11-20 03:11:02,499::task::495::TaskManager.Task::(_debug) Task 8826498a-6dcd-44f8-b291-857dd03a9156: ref 0 aborting False
>Thread-15913699::INFO::2013-11-20 03:11:02,500::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00118899345398', 'lastCheck': 1384917057.4003839, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.000864028930664', 'lastCheck': 1384917052.053179, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00623917579651', 'lastCheck': 1384917056.493603, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00217604637146', 'lastCheck': 1384917057.3941691, 'valid': True, 'code': 0}}
>Thread-15913700::DEBUG::2013-11-20 03:11:02,513::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::CPU stats not available
>Thread-15913700::DEBUG::2013-11-20 03:11:02,514::libvirtvm::202::vm.Vm::(_getNetworkStats) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::Network stats not available
>Thread-15913700::DEBUG::2013-11-20 03:11:02,514::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::Disk stats not available
>Thread-15913700::DEBUG::2013-11-20 03:11:02,515::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::Disk latency not available
>Thread-15913700::DEBUG::2013-11-20 03:11:02,516::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::Disk stats not available
>Thread-15913700::DEBUG::2013-11-20 03:11:02,517::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::Disk latency not available
>Thread-15913700::DEBUG::2013-11-20 03:11:02,518::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Disk stats not available
>Thread-15913700::DEBUG::2013-11-20 03:11:02,518::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::Disk latency not available
>Dummy-15912651::DEBUG::2013-11-20 03:11:03,968::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 03:11:04,349::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:11:04,350::task::495::TaskManager.Task::(_debug) Task f1d94931-68f2-48af-b09d-e6c8a09de24c: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:11:04,351::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`214fbc03-2b6e-481b-80a8-117110ac59cd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:11:04,352::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:11:04,352::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:11:04,353::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`214fbc03-2b6e-481b-80a8-117110ac59cd`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:11:04,353::task::495::TaskManager.Task::(_debug) Task f1d94931-68f2-48af-b09d-e6c8a09de24c: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:11:04,354::task::495::TaskManager.Task::(_debug) Task f1d94931-68f2-48af-b09d-e6c8a09de24c: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:11:04,354::task::495::TaskManager.Task::(_debug) Task f1d94931-68f2-48af-b09d-e6c8a09de24c: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:11:04,355::task::495::TaskManager.Task::(_debug) Task f1d94931-68f2-48af-b09d-e6c8a09de24c: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:11:04,355::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:11:04,355::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:11:04,356::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:11:04,356::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:11:04,357::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:11:04,357::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:11:04,357::task::495::TaskManager.Task::(_debug) Task f1d94931-68f2-48af-b09d-e6c8a09de24c: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:11:04,358::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:11:04,358::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:11:04,359::task::495::TaskManager.Task::(_debug) Task d8916e6e-690b-4078-9fbc-d09f23a03048: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:11:04,360::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6964572d-5ef5-4899-a909-96cc99098017`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:11:04,360::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:11:04,361::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:11:04,362::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6964572d-5ef5-4899-a909-96cc99098017`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:11:04,363::task::495::TaskManager.Task::(_debug) Task d8916e6e-690b-4078-9fbc-d09f23a03048: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:11:04,363::task::495::TaskManager.Task::(_debug) Task d8916e6e-690b-4078-9fbc-d09f23a03048: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:11:04,364::task::495::TaskManager.Task::(_debug) Task d8916e6e-690b-4078-9fbc-d09f23a03048: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:11:04,365::task::495::TaskManager.Task::(_debug) Task d8916e6e-690b-4078-9fbc-d09f23a03048: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:11:04,366::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:11:04,366::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:11:04,366::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:11:04,367::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:11:04,367::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:11:04,368::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:11:04,368::task::495::TaskManager.Task::(_debug) Task d8916e6e-690b-4078-9fbc-d09f23a03048: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:11:04,368::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:11:04,369::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:11:04,369::task::495::TaskManager.Task::(_debug) Task 8186739e-33ef-4064-a2e8-3dbd4e195987: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:11:04,370::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a383ab22-31af-479d-9bb2-a82d512b91f1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:11:04,370::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:11:04,371::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:11:04,371::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a383ab22-31af-479d-9bb2-a82d512b91f1`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:11:04,372::task::495::TaskManager.Task::(_debug) Task 8186739e-33ef-4064-a2e8-3dbd4e195987: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:11:04,372::task::495::TaskManager.Task::(_debug) Task 8186739e-33ef-4064-a2e8-3dbd4e195987: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:11:04,372::task::495::TaskManager.Task::(_debug) Task 8186739e-33ef-4064-a2e8-3dbd4e195987: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:11:04,373::task::495::TaskManager.Task::(_debug) Task 8186739e-33ef-4064-a2e8-3dbd4e195987: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:11:04,373::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:11:04,373::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:11:04,374::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:11:04,374::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:11:04,374::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:11:04,375::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:11:04,375::task::495::TaskManager.Task::(_debug) Task 8186739e-33ef-4064-a2e8-3dbd4e195987: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:11:04,376::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:11:04,376::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:11:04,376::task::495::TaskManager.Task::(_debug) Task 9a3c9a41-4853-43dc-ac52-53d8feba494d: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:11:04,377::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`99f7cc3f-c998-4f3a-976d-ff1bd694bc8f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:11:04,377::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:11:04,378::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:11:04,378::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`99f7cc3f-c998-4f3a-976d-ff1bd694bc8f`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:11:04,379::task::495::TaskManager.Task::(_debug) Task 9a3c9a41-4853-43dc-ac52-53d8feba494d: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:11:04,379::task::495::TaskManager.Task::(_debug) Task 9a3c9a41-4853-43dc-ac52-53d8feba494d: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:11:04,379::task::495::TaskManager.Task::(_debug) Task 9a3c9a41-4853-43dc-ac52-53d8feba494d: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:11:04,380::task::495::TaskManager.Task::(_debug) Task 9a3c9a41-4853-43dc-ac52-53d8feba494d: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:11:04,380::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:11:04,380::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:11:04,381::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:11:04,381::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:11:04,382::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:11:04,382::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:11:04,382::task::495::TaskManager.Task::(_debug) Task 9a3c9a41-4853-43dc-ac52-53d8feba494d: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:11:04,383::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:11:04,503::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0630825 s, 16.2 MB/s\n'; <rc> = 0
>Thread-15913701::DEBUG::2013-11-20 03:11:05,805::clientIF::54::vds::(wrapper) [192.168.100.135]::call setVmTicket with ('4760db3e-d3a8-4045-9153-12c36ec16ba2', 'xTPe7syms//D', '120') {}
>Thread-15913701::DEBUG::2013-11-20 03:11:05,826::clientIF::59::vds::(wrapper) return setVmTicket with {'status': {'message': 'Done', 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:11:06,512::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913703::DEBUG::2013-11-20 03:11:06,773::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913703::INFO::2013-11-20 03:11:06,791::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913703::DEBUG::2013-11-20 03:11:06,792::task::495::TaskManager.Task::(_debug) Task a97487c8-a13a-4d7e-a2b0-f368b1ae1207: moving from state init -> state preparing
>Thread-15913703::DEBUG::2013-11-20 03:11:06,792::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913703::DEBUG::2013-11-20 03:11:06,793::task::495::TaskManager.Task::(_debug) Task a97487c8-a13a-4d7e-a2b0-f368b1ae1207: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913703::DEBUG::2013-11-20 03:11:06,793::task::495::TaskManager.Task::(_debug) Task a97487c8-a13a-4d7e-a2b0-f368b1ae1207: moving from state preparing -> state finished
>Thread-15913703::DEBUG::2013-11-20 03:11:06,794::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913703::DEBUG::2013-11-20 03:11:06,794::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913703::DEBUG::2013-11-20 03:11:06,794::task::495::TaskManager.Task::(_debug) Task a97487c8-a13a-4d7e-a2b0-f368b1ae1207: ref 0 aborting False
>Thread-15913703::INFO::2013-11-20 03:11:06,795::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913704::DEBUG::2013-11-20 03:11:06,798::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913704::INFO::2013-11-20 03:11:06,799::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913704::DEBUG::2013-11-20 03:11:06,799::task::495::TaskManager.Task::(_debug) Task 399289b3-46fe-414b-8dfe-b29a2f61c53a: moving from state init -> state preparing
>Thread-15913704::DEBUG::2013-11-20 03:11:06,800::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7e2e1a83-dbe4-4941-8240-465c980e6731`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913704::DEBUG::2013-11-20 03:11:06,801::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913704::DEBUG::2013-11-20 03:11:06,801::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913704::DEBUG::2013-11-20 03:11:06,802::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7e2e1a83-dbe4-4941-8240-465c980e6731`::Granted request
>Thread-15913704::DEBUG::2013-11-20 03:11:06,802::task::495::TaskManager.Task::(_debug) Task 399289b3-46fe-414b-8dfe-b29a2f61c53a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913704::DEBUG::2013-11-20 03:11:06,803::task::495::TaskManager.Task::(_debug) Task 399289b3-46fe-414b-8dfe-b29a2f61c53a: ref 1 aborting False
>Thread-15913704::WARNING::2013-11-20 03:11:06,804::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913704::DEBUG::2013-11-20 03:11:06,805::task::495::TaskManager.Task::(_debug) Task 399289b3-46fe-414b-8dfe-b29a2f61c53a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913704::DEBUG::2013-11-20 03:11:06,806::task::495::TaskManager.Task::(_debug) Task 399289b3-46fe-414b-8dfe-b29a2f61c53a: moving from state preparing -> state finished
>Thread-15913704::DEBUG::2013-11-20 03:11:06,806::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913704::DEBUG::2013-11-20 03:11:06,807::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913704::DEBUG::2013-11-20 03:11:06,808::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913704::DEBUG::2013-11-20 03:11:06,808::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913704::DEBUG::2013-11-20 03:11:06,809::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913704::DEBUG::2013-11-20 03:11:06,809::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913704::DEBUG::2013-11-20 03:11:06,810::task::495::TaskManager.Task::(_debug) Task 399289b3-46fe-414b-8dfe-b29a2f61c53a: ref 0 aborting False
>Thread-15913704::INFO::2013-11-20 03:11:06,810::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:11:07,042::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0594308 s, 17.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:11:09,052::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:11:09,459::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.05087 s, 20.1 MB/s\n'; <rc> = 0
>Thread-15913706::DEBUG::2013-11-20 03:11:10,375::clientIF::54::vds::(wrapper) [192.168.100.135]::call setVmTicket with ('4760db3e-d3a8-4045-9153-12c36ec16ba2', 'MGHgWv5aMM1B', '120') {}
>Thread-15913706::DEBUG::2013-11-20 03:11:10,394::clientIF::59::vds::(wrapper) return setVmTicket with {'status': {'message': 'Done', 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:11:11,469::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:11:11,879::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0507813 s, 20.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:11:13,889::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 03:11:14,373::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:11:14,376::task::495::TaskManager.Task::(_debug) Task 4e120776-0c2a-4686-af90-ce8725f10d72: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:11:14,378::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a8d3a554-e24a-45ed-b20e-07abaa693204`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:11:14,378::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:11:14,380::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:11:14,380::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a8d3a554-e24a-45ed-b20e-07abaa693204`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:11:14,385::task::495::TaskManager.Task::(_debug) Task 4e120776-0c2a-4686-af90-ce8725f10d72: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:11:14,385::task::495::TaskManager.Task::(_debug) Task 4e120776-0c2a-4686-af90-ce8725f10d72: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:11:14,386::task::495::TaskManager.Task::(_debug) Task 4e120776-0c2a-4686-af90-ce8725f10d72: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:11:14,387::task::495::TaskManager.Task::(_debug) Task 4e120776-0c2a-4686-af90-ce8725f10d72: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:11:14,387::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:11:14,388::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:11:14,389::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:11:14,390::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:11:14,390::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:11:14,391::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:11:14,392::task::495::TaskManager.Task::(_debug) Task 4e120776-0c2a-4686-af90-ce8725f10d72: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:11:14,393::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:11:14,468::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.134324 s, 7.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:11:16,483::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:11:16,955::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0616967 s, 16.6 MB/s\n'; <rc> = 0
>Thread-15913709::DEBUG::2013-11-20 03:11:17,363::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913709::INFO::2013-11-20 03:11:17,364::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913709::DEBUG::2013-11-20 03:11:17,366::task::495::TaskManager.Task::(_debug) Task bcab8380-ef37-4e39-b843-462a4afb828a: moving from state init -> state preparing
>Thread-15913709::DEBUG::2013-11-20 03:11:17,371::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913709::DEBUG::2013-11-20 03:11:17,373::task::495::TaskManager.Task::(_debug) Task bcab8380-ef37-4e39-b843-462a4afb828a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913709::DEBUG::2013-11-20 03:11:17,374::task::495::TaskManager.Task::(_debug) Task bcab8380-ef37-4e39-b843-462a4afb828a: moving from state preparing -> state finished
>Thread-15913709::DEBUG::2013-11-20 03:11:17,374::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913709::DEBUG::2013-11-20 03:11:17,376::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913709::DEBUG::2013-11-20 03:11:17,377::task::495::TaskManager.Task::(_debug) Task bcab8380-ef37-4e39-b843-462a4afb828a: ref 0 aborting False
>Thread-15913709::INFO::2013-11-20 03:11:17,377::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913710::DEBUG::2013-11-20 03:11:17,382::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913710::INFO::2013-11-20 03:11:17,383::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913710::DEBUG::2013-11-20 03:11:17,383::task::495::TaskManager.Task::(_debug) Task fbf44ba7-0bbc-45db-aaa9-e7004aaa784b: moving from state init -> state preparing
>Thread-15913710::DEBUG::2013-11-20 03:11:17,384::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`35aa8c09-fbba-4086-af51-4726039ff196`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913710::DEBUG::2013-11-20 03:11:17,385::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913710::DEBUG::2013-11-20 03:11:17,385::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913710::DEBUG::2013-11-20 03:11:17,386::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`35aa8c09-fbba-4086-af51-4726039ff196`::Granted request
>Thread-15913710::DEBUG::2013-11-20 03:11:17,387::task::495::TaskManager.Task::(_debug) Task fbf44ba7-0bbc-45db-aaa9-e7004aaa784b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913710::DEBUG::2013-11-20 03:11:17,387::task::495::TaskManager.Task::(_debug) Task fbf44ba7-0bbc-45db-aaa9-e7004aaa784b: ref 1 aborting False
>Thread-15913710::WARNING::2013-11-20 03:11:17,389::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913710::DEBUG::2013-11-20 03:11:17,390::task::495::TaskManager.Task::(_debug) Task fbf44ba7-0bbc-45db-aaa9-e7004aaa784b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913710::DEBUG::2013-11-20 03:11:17,390::task::495::TaskManager.Task::(_debug) Task fbf44ba7-0bbc-45db-aaa9-e7004aaa784b: moving from state preparing -> state finished
>Thread-15913710::DEBUG::2013-11-20 03:11:17,391::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913710::DEBUG::2013-11-20 03:11:17,391::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913710::DEBUG::2013-11-20 03:11:17,392::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913710::DEBUG::2013-11-20 03:11:17,392::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913710::DEBUG::2013-11-20 03:11:17,393::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913710::DEBUG::2013-11-20 03:11:17,393::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913710::DEBUG::2013-11-20 03:11:17,394::task::495::TaskManager.Task::(_debug) Task fbf44ba7-0bbc-45db-aaa9-e7004aaa784b: ref 0 aborting False
>Thread-15913710::INFO::2013-11-20 03:11:17,394::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913711::INFO::2013-11-20 03:11:17,943::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913711::DEBUG::2013-11-20 03:11:17,943::task::495::TaskManager.Task::(_debug) Task de306af0-5571-4e65-9e3f-af06b3d6a528: moving from state init -> state preparing
>Thread-15913711::DEBUG::2013-11-20 03:11:17,955::task::495::TaskManager.Task::(_debug) Task de306af0-5571-4e65-9e3f-af06b3d6a528: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000243902206421', 'lastCheck': 1384917077.441107, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00100493431091', 'lastCheck': 1384917075.4347601, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00127005577087', 'lastCheck': 1384917076.7083399, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000327110290527', 'lastCheck': 1384917077.420923, 'valid': True, 'code': 0}}
>Thread-15913711::DEBUG::2013-11-20 03:11:17,956::task::495::TaskManager.Task::(_debug) Task de306af0-5571-4e65-9e3f-af06b3d6a528: moving from state preparing -> state finished
>Thread-15913711::DEBUG::2013-11-20 03:11:17,956::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913711::DEBUG::2013-11-20 03:11:17,957::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913711::DEBUG::2013-11-20 03:11:17,957::task::495::TaskManager.Task::(_debug) Task de306af0-5571-4e65-9e3f-af06b3d6a528: ref 0 aborting False
>Thread-15913711::INFO::2013-11-20 03:11:17,958::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000243902206421', 'lastCheck': 1384917077.441107, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00100493431091', 'lastCheck': 1384917075.4347601, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00127005577087', 'lastCheck': 1384917076.7083399, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000327110290527', 'lastCheck': 1384917077.420923, 'valid': True, 'code': 0}}
>Thread-15913712::DEBUG::2013-11-20 03:11:17,970::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::Disk stats not available
>Thread-15913712::DEBUG::2013-11-20 03:11:17,970::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::Disk latency not available
>Thread-15913712::DEBUG::2013-11-20 03:11:17,972::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::Disk stats not available
>Thread-15913712::DEBUG::2013-11-20 03:11:17,972::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::Disk latency not available
>Dummy-15912651::DEBUG::2013-11-20 03:11:18,966::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:11:20,149::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0517913 s, 19.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:11:22,160::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:11:22,607::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0488065 s, 21.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:11:24,618::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:11:25,193::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0553224 s, 18.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:11:27,206::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:11:27,610::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.04684 s, 21.9 MB/s\n'; <rc> = 0
>Thread-15913716::DEBUG::2013-11-20 03:11:27,729::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913716::INFO::2013-11-20 03:11:27,730::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913716::DEBUG::2013-11-20 03:11:27,731::task::495::TaskManager.Task::(_debug) Task b59c2a19-0f5a-46d3-85b4-3fd6f47dbba0: moving from state init -> state preparing
>Thread-15913716::DEBUG::2013-11-20 03:11:27,731::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913716::DEBUG::2013-11-20 03:11:27,732::task::495::TaskManager.Task::(_debug) Task b59c2a19-0f5a-46d3-85b4-3fd6f47dbba0: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913716::DEBUG::2013-11-20 03:11:27,732::task::495::TaskManager.Task::(_debug) Task b59c2a19-0f5a-46d3-85b4-3fd6f47dbba0: moving from state preparing -> state finished
>Thread-15913716::DEBUG::2013-11-20 03:11:27,733::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913716::DEBUG::2013-11-20 03:11:27,733::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913716::DEBUG::2013-11-20 03:11:27,733::task::495::TaskManager.Task::(_debug) Task b59c2a19-0f5a-46d3-85b4-3fd6f47dbba0: ref 0 aborting False
>Thread-15913716::INFO::2013-11-20 03:11:27,734::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913717::DEBUG::2013-11-20 03:11:27,737::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913717::INFO::2013-11-20 03:11:27,738::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913717::DEBUG::2013-11-20 03:11:27,738::task::495::TaskManager.Task::(_debug) Task 6a5eed91-6863-407c-b4ce-d075b4fbd54f: moving from state init -> state preparing
>Thread-15913717::DEBUG::2013-11-20 03:11:27,739::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a1c529cf-8c3b-4359-a7ab-5e89350364cc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913717::DEBUG::2013-11-20 03:11:27,739::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913717::DEBUG::2013-11-20 03:11:27,740::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913717::DEBUG::2013-11-20 03:11:27,740::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a1c529cf-8c3b-4359-a7ab-5e89350364cc`::Granted request
>Thread-15913717::DEBUG::2013-11-20 03:11:27,741::task::495::TaskManager.Task::(_debug) Task 6a5eed91-6863-407c-b4ce-d075b4fbd54f: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913717::DEBUG::2013-11-20 03:11:27,741::task::495::TaskManager.Task::(_debug) Task 6a5eed91-6863-407c-b4ce-d075b4fbd54f: ref 1 aborting False
>Thread-15913717::WARNING::2013-11-20 03:11:27,743::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913717::DEBUG::2013-11-20 03:11:27,743::task::495::TaskManager.Task::(_debug) Task 6a5eed91-6863-407c-b4ce-d075b4fbd54f: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913717::DEBUG::2013-11-20 03:11:27,744::task::495::TaskManager.Task::(_debug) Task 6a5eed91-6863-407c-b4ce-d075b4fbd54f: moving from state preparing -> state finished
>Thread-15913717::DEBUG::2013-11-20 03:11:27,744::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913717::DEBUG::2013-11-20 03:11:27,745::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913717::DEBUG::2013-11-20 03:11:27,745::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913717::DEBUG::2013-11-20 03:11:27,746::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913717::DEBUG::2013-11-20 03:11:27,746::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913717::DEBUG::2013-11-20 03:11:27,747::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913717::DEBUG::2013-11-20 03:11:27,747::task::495::TaskManager.Task::(_debug) Task 6a5eed91-6863-407c-b4ce-d075b4fbd54f: ref 0 aborting False
>Thread-15913717::INFO::2013-11-20 03:11:27,748::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:11:29,620::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:11:30,807::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0495314 s, 20.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:11:32,820::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:11:33,293::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0534308 s, 19.2 MB/s\n'; <rc> = 0
>Thread-15913719::INFO::2013-11-20 03:11:33,540::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913719::DEBUG::2013-11-20 03:11:33,540::task::495::TaskManager.Task::(_debug) Task c877edfa-5789-4153-9420-efdf027fd953: moving from state init -> state preparing
>Thread-15913719::DEBUG::2013-11-20 03:11:33,541::task::495::TaskManager.Task::(_debug) Task c877edfa-5789-4153-9420-efdf027fd953: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000301122665405', 'lastCheck': 1384917087.4535971, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0221340656281', 'lastCheck': 1384917085.4699149, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0141389369965', 'lastCheck': 1384917086.8147681, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000437021255493', 'lastCheck': 1384917087.4309959, 'valid': True, 'code': 0}}
>Thread-15913719::DEBUG::2013-11-20 03:11:33,541::task::495::TaskManager.Task::(_debug) Task c877edfa-5789-4153-9420-efdf027fd953: moving from state preparing -> state finished
>Thread-15913719::DEBUG::2013-11-20 03:11:33,542::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913719::DEBUG::2013-11-20 03:11:33,542::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913719::DEBUG::2013-11-20 03:11:33,543::task::495::TaskManager.Task::(_debug) Task c877edfa-5789-4153-9420-efdf027fd953: ref 0 aborting False
>Thread-15913719::INFO::2013-11-20 03:11:33,543::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000301122665405', 'lastCheck': 1384917087.4535971, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0221340656281', 'lastCheck': 1384917085.4699149, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0141389369965', 'lastCheck': 1384917086.8147681, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000437021255493', 'lastCheck': 1384917087.4309959, 'valid': True, 'code': 0}}
>Thread-15913720::DEBUG::2013-11-20 03:11:33,574::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::Disk stats not available
>Thread-15913720::DEBUG::2013-11-20 03:11:33,575::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::Disk latency not available
>Thread-15913720::DEBUG::2013-11-20 03:11:33,576::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::Disk stats not available
>Thread-15913720::DEBUG::2013-11-20 03:11:33,576::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::Disk latency not available
>Thread-15913658::INFO::2013-11-20 03:11:34,386::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:11:34,387::task::495::TaskManager.Task::(_debug) Task 62c9006a-495c-4ecf-b921-00928a82cfca: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:11:34,388::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1289aa57-26c6-490e-b2cd-c843bcf48dd5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:11:34,388::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:11:34,389::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:11:34,389::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1289aa57-26c6-490e-b2cd-c843bcf48dd5`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:11:34,390::task::495::TaskManager.Task::(_debug) Task 62c9006a-495c-4ecf-b921-00928a82cfca: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:11:34,391::task::495::TaskManager.Task::(_debug) Task 62c9006a-495c-4ecf-b921-00928a82cfca: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:11:34,391::task::495::TaskManager.Task::(_debug) Task 62c9006a-495c-4ecf-b921-00928a82cfca: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:11:34,392::task::495::TaskManager.Task::(_debug) Task 62c9006a-495c-4ecf-b921-00928a82cfca: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:11:34,392::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:11:34,393::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:11:34,393::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:11:34,394::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:11:34,394::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:11:34,394::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:11:34,395::task::495::TaskManager.Task::(_debug) Task 62c9006a-495c-4ecf-b921-00928a82cfca: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:11:34,395::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:11:35,304::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:11:37,021::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0552596 s, 18.5 MB/s\n'; <rc> = 0
>Thread-42838::DEBUG::2013-11-20 03:11:37,496::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 03:11:38,091::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913722::DEBUG::2013-11-20 03:11:38,460::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913722::INFO::2013-11-20 03:11:38,461::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913722::DEBUG::2013-11-20 03:11:38,462::task::495::TaskManager.Task::(_debug) Task 4d243623-ee1d-4a8f-a547-2422b1e9df0e: moving from state init -> state preparing
>Thread-15913722::DEBUG::2013-11-20 03:11:38,462::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913722::DEBUG::2013-11-20 03:11:38,463::task::495::TaskManager.Task::(_debug) Task 4d243623-ee1d-4a8f-a547-2422b1e9df0e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913722::DEBUG::2013-11-20 03:11:38,463::task::495::TaskManager.Task::(_debug) Task 4d243623-ee1d-4a8f-a547-2422b1e9df0e: moving from state preparing -> state finished
>Thread-15913722::DEBUG::2013-11-20 03:11:38,463::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913722::DEBUG::2013-11-20 03:11:38,464::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913722::DEBUG::2013-11-20 03:11:38,464::task::495::TaskManager.Task::(_debug) Task 4d243623-ee1d-4a8f-a547-2422b1e9df0e: ref 0 aborting False
>Thread-15913722::INFO::2013-11-20 03:11:38,465::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913723::DEBUG::2013-11-20 03:11:38,468::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913723::INFO::2013-11-20 03:11:38,469::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913723::DEBUG::2013-11-20 03:11:38,469::task::495::TaskManager.Task::(_debug) Task ced88d08-e93f-4e74-baf6-baa4d9fdf646: moving from state init -> state preparing
>Thread-15913723::DEBUG::2013-11-20 03:11:38,470::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`74b5f494-7a2f-4d09-9430-0cd14cf3c2af`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913723::DEBUG::2013-11-20 03:11:38,471::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913723::DEBUG::2013-11-20 03:11:38,471::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913723::DEBUG::2013-11-20 03:11:38,472::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`74b5f494-7a2f-4d09-9430-0cd14cf3c2af`::Granted request
>Thread-15913723::DEBUG::2013-11-20 03:11:38,472::task::495::TaskManager.Task::(_debug) Task ced88d08-e93f-4e74-baf6-baa4d9fdf646: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913723::DEBUG::2013-11-20 03:11:38,473::task::495::TaskManager.Task::(_debug) Task ced88d08-e93f-4e74-baf6-baa4d9fdf646: ref 1 aborting False
>Thread-15913723::WARNING::2013-11-20 03:11:38,474::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913723::DEBUG::2013-11-20 03:11:38,475::task::495::TaskManager.Task::(_debug) Task ced88d08-e93f-4e74-baf6-baa4d9fdf646: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913723::DEBUG::2013-11-20 03:11:38,475::task::495::TaskManager.Task::(_debug) Task ced88d08-e93f-4e74-baf6-baa4d9fdf646: moving from state preparing -> state finished
>Thread-15913723::DEBUG::2013-11-20 03:11:38,476::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913723::DEBUG::2013-11-20 03:11:38,476::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913723::DEBUG::2013-11-20 03:11:38,476::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913723::DEBUG::2013-11-20 03:11:38,477::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913723::DEBUG::2013-11-20 03:11:38,477::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913723::DEBUG::2013-11-20 03:11:38,477::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913723::DEBUG::2013-11-20 03:11:38,478::task::495::TaskManager.Task::(_debug) Task ced88d08-e93f-4e74-baf6-baa4d9fdf646: ref 0 aborting False
>Thread-15913723::INFO::2013-11-20 03:11:38,478::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:11:39,035::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:11:39,478::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0515771 s, 19.9 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 03:11:40,933::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:11:40,934::task::495::TaskManager.Task::(_debug) Task dc3b2580-e178-4471-a816-08568b7383f2: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:11:40,935::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4f78811a-4c15-436e-84b1-b214f22aa468`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:11:40,936::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:11:40,936::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:11:40,937::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4f78811a-4c15-436e-84b1-b214f22aa468`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:11:40,938::task::495::TaskManager.Task::(_debug) Task dc3b2580-e178-4471-a816-08568b7383f2: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:11:40,938::task::495::TaskManager.Task::(_debug) Task dc3b2580-e178-4471-a816-08568b7383f2: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:11:40,939::task::495::TaskManager.Task::(_debug) Task dc3b2580-e178-4471-a816-08568b7383f2: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:11:40,939::task::495::TaskManager.Task::(_debug) Task dc3b2580-e178-4471-a816-08568b7383f2: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:11:40,940::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:11:40,940::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:11:40,941::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:11:40,941::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:11:40,942::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:11:40,942::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:11:40,943::task::495::TaskManager.Task::(_debug) Task dc3b2580-e178-4471-a816-08568b7383f2: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:11:40,943::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:11:41,489::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:11:42,060::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.104569 s, 9.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:11:44,071::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:11:44,505::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0586969 s, 17.4 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 03:11:45,276::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:11:45,277::task::495::TaskManager.Task::(_debug) Task de646adf-7e54-4a28-905a-6acd12e14fa2: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:11:45,278::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0c311331-81cb-4228-9385-a8f3c433e293`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:11:45,278::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:11:45,279::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:11:45,279::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0c311331-81cb-4228-9385-a8f3c433e293`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:11:45,280::task::495::TaskManager.Task::(_debug) Task de646adf-7e54-4a28-905a-6acd12e14fa2: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:11:45,280::task::495::TaskManager.Task::(_debug) Task de646adf-7e54-4a28-905a-6acd12e14fa2: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:11:45,281::task::495::TaskManager.Task::(_debug) Task de646adf-7e54-4a28-905a-6acd12e14fa2: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:11:45,281::task::495::TaskManager.Task::(_debug) Task de646adf-7e54-4a28-905a-6acd12e14fa2: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:11:45,281::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:11:45,282::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:11:45,282::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:11:45,282::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:11:45,283::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:11:45,283::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:11:45,283::task::495::TaskManager.Task::(_debug) Task de646adf-7e54-4a28-905a-6acd12e14fa2: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:11:45,284::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:11:45,284::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:11:45,285::task::495::TaskManager.Task::(_debug) Task 4a0c9659-5d78-4927-8eff-47da250d59f8: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:11:45,285::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c9a5f04c-76d2-4055-9d7d-bc69ba2fc549`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:11:45,286::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:11:45,286::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:11:45,286::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c9a5f04c-76d2-4055-9d7d-bc69ba2fc549`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:11:45,287::task::495::TaskManager.Task::(_debug) Task 4a0c9659-5d78-4927-8eff-47da250d59f8: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:11:45,287::task::495::TaskManager.Task::(_debug) Task 4a0c9659-5d78-4927-8eff-47da250d59f8: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:11:45,287::task::495::TaskManager.Task::(_debug) Task 4a0c9659-5d78-4927-8eff-47da250d59f8: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:11:45,288::task::495::TaskManager.Task::(_debug) Task 4a0c9659-5d78-4927-8eff-47da250d59f8: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:11:45,288::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:11:45,288::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:11:45,289::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:11:45,289::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:11:45,290::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:11:45,290::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:11:45,290::task::495::TaskManager.Task::(_debug) Task 4a0c9659-5d78-4927-8eff-47da250d59f8: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:11:45,291::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:11:46,518::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:11:46,941::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0507749 s, 20.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:11:48,951::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913727::DEBUG::2013-11-20 03:11:49,181::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913727::INFO::2013-11-20 03:11:49,183::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913727::DEBUG::2013-11-20 03:11:49,184::task::495::TaskManager.Task::(_debug) Task e6bddd01-f817-4763-8583-e716e9eff46f: moving from state init -> state preparing
>Thread-15913727::DEBUG::2013-11-20 03:11:49,185::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913727::DEBUG::2013-11-20 03:11:49,185::task::495::TaskManager.Task::(_debug) Task e6bddd01-f817-4763-8583-e716e9eff46f: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913727::DEBUG::2013-11-20 03:11:49,185::task::495::TaskManager.Task::(_debug) Task e6bddd01-f817-4763-8583-e716e9eff46f: moving from state preparing -> state finished
>Thread-15913727::DEBUG::2013-11-20 03:11:49,186::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913727::DEBUG::2013-11-20 03:11:49,186::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913727::DEBUG::2013-11-20 03:11:49,187::task::495::TaskManager.Task::(_debug) Task e6bddd01-f817-4763-8583-e716e9eff46f: ref 0 aborting False
>Thread-15913727::INFO::2013-11-20 03:11:49,187::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913728::DEBUG::2013-11-20 03:11:49,191::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913728::INFO::2013-11-20 03:11:49,192::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913728::DEBUG::2013-11-20 03:11:49,192::task::495::TaskManager.Task::(_debug) Task b23d1a7d-d77d-4a09-a4f1-35a4b9bdf2f9: moving from state init -> state preparing
>Thread-15913728::DEBUG::2013-11-20 03:11:49,193::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`309305a6-76c9-4c77-abdc-011d44e4b989`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913728::DEBUG::2013-11-20 03:11:49,193::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913728::DEBUG::2013-11-20 03:11:49,194::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913728::DEBUG::2013-11-20 03:11:49,194::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`309305a6-76c9-4c77-abdc-011d44e4b989`::Granted request
>Thread-15913728::DEBUG::2013-11-20 03:11:49,195::task::495::TaskManager.Task::(_debug) Task b23d1a7d-d77d-4a09-a4f1-35a4b9bdf2f9: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913728::DEBUG::2013-11-20 03:11:49,195::task::495::TaskManager.Task::(_debug) Task b23d1a7d-d77d-4a09-a4f1-35a4b9bdf2f9: ref 1 aborting False
>Thread-15913728::WARNING::2013-11-20 03:11:49,197::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913728::DEBUG::2013-11-20 03:11:49,197::task::495::TaskManager.Task::(_debug) Task b23d1a7d-d77d-4a09-a4f1-35a4b9bdf2f9: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913728::DEBUG::2013-11-20 03:11:49,198::task::495::TaskManager.Task::(_debug) Task b23d1a7d-d77d-4a09-a4f1-35a4b9bdf2f9: moving from state preparing -> state finished
>Thread-15913728::DEBUG::2013-11-20 03:11:49,198::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913728::DEBUG::2013-11-20 03:11:49,199::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913728::DEBUG::2013-11-20 03:11:49,199::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913728::DEBUG::2013-11-20 03:11:49,200::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913728::DEBUG::2013-11-20 03:11:49,200::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913728::DEBUG::2013-11-20 03:11:49,200::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913728::DEBUG::2013-11-20 03:11:49,201::task::495::TaskManager.Task::(_debug) Task b23d1a7d-d77d-4a09-a4f1-35a4b9bdf2f9: ref 0 aborting False
>Thread-15913728::INFO::2013-11-20 03:11:49,201::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:11:49,398::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0508222 s, 20.1 MB/s\n'; <rc> = 0
>Thread-15913729::INFO::2013-11-20 03:11:49,481::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913729::DEBUG::2013-11-20 03:11:49,481::task::495::TaskManager.Task::(_debug) Task de48a4fb-89cf-42a8-8906-5529fd2b8f93: moving from state init -> state preparing
>Thread-15913729::DEBUG::2013-11-20 03:11:49,482::task::495::TaskManager.Task::(_debug) Task de48a4fb-89cf-42a8-8906-5529fd2b8f93: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000226974487305', 'lastCheck': 1384917108.2509661, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0129749774933', 'lastCheck': 1384917105.9833441, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0141789913177', 'lastCheck': 1384917107.0371189, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000257015228271', 'lastCheck': 1384917107.693994, 'valid': True, 'code': 0}}
>Thread-15913729::DEBUG::2013-11-20 03:11:49,483::task::495::TaskManager.Task::(_debug) Task de48a4fb-89cf-42a8-8906-5529fd2b8f93: moving from state preparing -> state finished
>Thread-15913729::DEBUG::2013-11-20 03:11:49,483::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913729::DEBUG::2013-11-20 03:11:49,483::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913729::DEBUG::2013-11-20 03:11:49,484::task::495::TaskManager.Task::(_debug) Task de48a4fb-89cf-42a8-8906-5529fd2b8f93: ref 0 aborting False
>Thread-15913729::INFO::2013-11-20 03:11:49,484::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000226974487305', 'lastCheck': 1384917108.2509661, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0129749774933', 'lastCheck': 1384917105.9833441, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0141789913177', 'lastCheck': 1384917107.0371189, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000257015228271', 'lastCheck': 1384917107.693994, 'valid': True, 'code': 0}}
>Thread-15913730::DEBUG::2013-11-20 03:11:49,542::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::Disk stats not available
>Thread-15913730::DEBUG::2013-11-20 03:11:49,542::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::Disk latency not available
>Dummy-15912651::DEBUG::2013-11-20 03:11:51,407::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:11:51,852::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0571939 s, 17.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:11:53,862::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:11:56,439::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0591032 s, 17.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:11:58,450::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:11:58,875::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0496416 s, 20.6 MB/s\n'; <rc> = 0
>Thread-15913734::DEBUG::2013-11-20 03:11:59,428::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913734::INFO::2013-11-20 03:11:59,435::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913734::DEBUG::2013-11-20 03:11:59,436::task::495::TaskManager.Task::(_debug) Task 3fe5a340-ae1c-455e-9951-fb74af25c357: moving from state init -> state preparing
>Thread-15913734::DEBUG::2013-11-20 03:11:59,437::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913734::DEBUG::2013-11-20 03:11:59,438::task::495::TaskManager.Task::(_debug) Task 3fe5a340-ae1c-455e-9951-fb74af25c357: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913734::DEBUG::2013-11-20 03:11:59,439::task::495::TaskManager.Task::(_debug) Task 3fe5a340-ae1c-455e-9951-fb74af25c357: moving from state preparing -> state finished
>Thread-15913734::DEBUG::2013-11-20 03:11:59,439::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913734::DEBUG::2013-11-20 03:11:59,440::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913734::DEBUG::2013-11-20 03:11:59,440::task::495::TaskManager.Task::(_debug) Task 3fe5a340-ae1c-455e-9951-fb74af25c357: ref 0 aborting False
>Thread-15913734::INFO::2013-11-20 03:11:59,441::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913735::DEBUG::2013-11-20 03:11:59,449::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913735::INFO::2013-11-20 03:11:59,450::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913735::DEBUG::2013-11-20 03:11:59,450::task::495::TaskManager.Task::(_debug) Task a19ea7c0-5812-42e2-b952-e3b921222099: moving from state init -> state preparing
>Thread-15913735::DEBUG::2013-11-20 03:11:59,451::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b09eacb6-1f08-4200-8f29-d7f123648d80`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913735::DEBUG::2013-11-20 03:11:59,451::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913735::DEBUG::2013-11-20 03:11:59,452::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913735::DEBUG::2013-11-20 03:11:59,453::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b09eacb6-1f08-4200-8f29-d7f123648d80`::Granted request
>Thread-15913735::DEBUG::2013-11-20 03:11:59,453::task::495::TaskManager.Task::(_debug) Task a19ea7c0-5812-42e2-b952-e3b921222099: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913735::DEBUG::2013-11-20 03:11:59,454::task::495::TaskManager.Task::(_debug) Task a19ea7c0-5812-42e2-b952-e3b921222099: ref 1 aborting False
>Thread-15913735::WARNING::2013-11-20 03:11:59,455::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913735::DEBUG::2013-11-20 03:11:59,456::task::495::TaskManager.Task::(_debug) Task a19ea7c0-5812-42e2-b952-e3b921222099: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913735::DEBUG::2013-11-20 03:11:59,457::task::495::TaskManager.Task::(_debug) Task a19ea7c0-5812-42e2-b952-e3b921222099: moving from state preparing -> state finished
>Thread-15913735::DEBUG::2013-11-20 03:11:59,457::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913735::DEBUG::2013-11-20 03:11:59,458::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913735::DEBUG::2013-11-20 03:11:59,458::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913735::DEBUG::2013-11-20 03:11:59,459::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913735::DEBUG::2013-11-20 03:11:59,459::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913735::DEBUG::2013-11-20 03:11:59,459::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913735::DEBUG::2013-11-20 03:11:59,460::task::495::TaskManager.Task::(_debug) Task a19ea7c0-5812-42e2-b952-e3b921222099: ref 0 aborting False
>Thread-15913735::INFO::2013-11-20 03:11:59,460::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:12:00,885::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:12:01,375::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0514781 s, 19.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:12:03,454::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:12:04,097::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0520837 s, 19.7 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 03:12:04,624::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:12:04,625::task::495::TaskManager.Task::(_debug) Task 5e7d9a65-1f1a-4d27-8d22-dc05ff85e01b: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:12:04,626::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f0874edc-f9ac-4426-86e3-536a622ed873`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:12:04,626::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:12:04,627::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:12:04,627::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f0874edc-f9ac-4426-86e3-536a622ed873`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:12:04,628::task::495::TaskManager.Task::(_debug) Task 5e7d9a65-1f1a-4d27-8d22-dc05ff85e01b: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:12:04,628::task::495::TaskManager.Task::(_debug) Task 5e7d9a65-1f1a-4d27-8d22-dc05ff85e01b: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:12:04,629::task::495::TaskManager.Task::(_debug) Task 5e7d9a65-1f1a-4d27-8d22-dc05ff85e01b: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:12:04,629::task::495::TaskManager.Task::(_debug) Task 5e7d9a65-1f1a-4d27-8d22-dc05ff85e01b: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:12:04,629::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:12:04,630::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:12:04,630::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:12:04,630::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:12:04,631::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:12:04,631::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:12:04,631::task::495::TaskManager.Task::(_debug) Task 5e7d9a65-1f1a-4d27-8d22-dc05ff85e01b: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:12:04,632::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:12:04,632::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:12:04,633::task::495::TaskManager.Task::(_debug) Task d01603eb-b077-42a0-98cf-a9982ef60397: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:12:04,633::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`30c5f505-9c6d-4d24-8f02-58c498ac9350`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:12:04,634::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:12:04,634::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:12:04,635::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`30c5f505-9c6d-4d24-8f02-58c498ac9350`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:12:04,636::task::495::TaskManager.Task::(_debug) Task d01603eb-b077-42a0-98cf-a9982ef60397: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:12:04,636::task::495::TaskManager.Task::(_debug) Task d01603eb-b077-42a0-98cf-a9982ef60397: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:12:04,636::task::495::TaskManager.Task::(_debug) Task d01603eb-b077-42a0-98cf-a9982ef60397: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:12:04,637::task::495::TaskManager.Task::(_debug) Task d01603eb-b077-42a0-98cf-a9982ef60397: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:12:04,637::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:12:04,637::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:12:04,638::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:12:04,638::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:12:04,638::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:12:04,639::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:12:04,639::task::495::TaskManager.Task::(_debug) Task d01603eb-b077-42a0-98cf-a9982ef60397: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:12:04,639::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:12:04,640::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:12:04,640::task::495::TaskManager.Task::(_debug) Task 2cf7f061-325d-47aa-be5b-fcc59f300052: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:12:04,641::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`bd97cb59-ac74-4f34-9397-01e240ffdbab`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:12:04,641::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:12:04,642::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:12:04,642::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`bd97cb59-ac74-4f34-9397-01e240ffdbab`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:12:04,643::task::495::TaskManager.Task::(_debug) Task 2cf7f061-325d-47aa-be5b-fcc59f300052: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:12:04,643::task::495::TaskManager.Task::(_debug) Task 2cf7f061-325d-47aa-be5b-fcc59f300052: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:12:04,643::task::495::TaskManager.Task::(_debug) Task 2cf7f061-325d-47aa-be5b-fcc59f300052: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:12:04,644::task::495::TaskManager.Task::(_debug) Task 2cf7f061-325d-47aa-be5b-fcc59f300052: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:12:04,644::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:12:04,644::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:12:04,645::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:12:04,645::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:12:04,645::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:12:04,646::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:12:04,646::task::495::TaskManager.Task::(_debug) Task 2cf7f061-325d-47aa-be5b-fcc59f300052: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:12:04,646::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:12:04,647::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:12:04,647::task::495::TaskManager.Task::(_debug) Task d5f15f2a-7469-43d8-98b9-12516c16220a: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:12:04,648::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b298d285-764a-4364-9f71-1d869fa0ca03`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:12:04,648::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:12:04,648::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:12:04,649::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b298d285-764a-4364-9f71-1d869fa0ca03`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:12:04,649::task::495::TaskManager.Task::(_debug) Task d5f15f2a-7469-43d8-98b9-12516c16220a: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:12:04,650::task::495::TaskManager.Task::(_debug) Task d5f15f2a-7469-43d8-98b9-12516c16220a: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:12:04,650::task::495::TaskManager.Task::(_debug) Task d5f15f2a-7469-43d8-98b9-12516c16220a: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:12:04,650::task::495::TaskManager.Task::(_debug) Task d5f15f2a-7469-43d8-98b9-12516c16220a: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:12:04,651::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:12:04,651::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:12:04,651::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:12:04,652::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:12:04,652::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:12:04,652::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:12:04,653::task::495::TaskManager.Task::(_debug) Task d5f15f2a-7469-43d8-98b9-12516c16220a: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:12:04,653::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913737::INFO::2013-11-20 03:12:05,403::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913737::DEBUG::2013-11-20 03:12:05,404::task::495::TaskManager.Task::(_debug) Task 54282b64-480d-487d-af1e-58d85d2dbe79: moving from state init -> state preparing
>Thread-15913737::DEBUG::2013-11-20 03:12:05,404::task::495::TaskManager.Task::(_debug) Task 54282b64-480d-487d-af1e-58d85d2dbe79: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000262022018433', 'lastCheck': 1384917119.1702211, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00093412399292', 'lastCheck': 1384917116.187726, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000868797302246', 'lastCheck': 1384917117.187288, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000303030014038', 'lastCheck': 1384917118.1651831, 'valid': True, 'code': 0}}
>Thread-15913737::DEBUG::2013-11-20 03:12:05,405::task::495::TaskManager.Task::(_debug) Task 54282b64-480d-487d-af1e-58d85d2dbe79: moving from state preparing -> state finished
>Thread-15913737::DEBUG::2013-11-20 03:12:05,405::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913737::DEBUG::2013-11-20 03:12:05,406::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913737::DEBUG::2013-11-20 03:12:05,406::task::495::TaskManager.Task::(_debug) Task 54282b64-480d-487d-af1e-58d85d2dbe79: ref 0 aborting False
>Thread-15913737::INFO::2013-11-20 03:12:05,406::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000262022018433', 'lastCheck': 1384917119.1702211, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00093412399292', 'lastCheck': 1384917116.187726, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000868797302246', 'lastCheck': 1384917117.187288, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000303030014038', 'lastCheck': 1384917118.1651831, 'valid': True, 'code': 0}}
>Thread-15913738::DEBUG::2013-11-20 03:12:05,439::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::Disk stats not available
>Thread-15913738::DEBUG::2013-11-20 03:12:05,439::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::Disk latency not available
>Dummy-15912651::DEBUG::2013-11-20 03:12:06,107::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:12:06,554::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0495487 s, 20.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:12:08,564::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913740::DEBUG::2013-11-20 03:12:09,767::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913740::INFO::2013-11-20 03:12:09,768::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913740::DEBUG::2013-11-20 03:12:09,769::task::495::TaskManager.Task::(_debug) Task 897d8a7d-da9e-4a34-88c7-b71823e3afdc: moving from state init -> state preparing
>Thread-15913740::DEBUG::2013-11-20 03:12:09,770::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913740::DEBUG::2013-11-20 03:12:09,770::task::495::TaskManager.Task::(_debug) Task 897d8a7d-da9e-4a34-88c7-b71823e3afdc: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913740::DEBUG::2013-11-20 03:12:09,771::task::495::TaskManager.Task::(_debug) Task 897d8a7d-da9e-4a34-88c7-b71823e3afdc: moving from state preparing -> state finished
>Thread-15913740::DEBUG::2013-11-20 03:12:09,771::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913740::DEBUG::2013-11-20 03:12:09,771::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913740::DEBUG::2013-11-20 03:12:09,772::task::495::TaskManager.Task::(_debug) Task 897d8a7d-da9e-4a34-88c7-b71823e3afdc: ref 0 aborting False
>Thread-15913740::INFO::2013-11-20 03:12:09,772::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913741::DEBUG::2013-11-20 03:12:09,777::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913741::INFO::2013-11-20 03:12:09,778::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913741::DEBUG::2013-11-20 03:12:09,779::task::495::TaskManager.Task::(_debug) Task bf123faf-fc5f-4e40-b6b7-f4152f51da98: moving from state init -> state preparing
>Thread-15913741::DEBUG::2013-11-20 03:12:09,780::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a188a763-cbb5-4dec-b10e-3af5d6c918ff`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913741::DEBUG::2013-11-20 03:12:09,781::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913741::DEBUG::2013-11-20 03:12:09,782::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913741::DEBUG::2013-11-20 03:12:09,782::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a188a763-cbb5-4dec-b10e-3af5d6c918ff`::Granted request
>Thread-15913741::DEBUG::2013-11-20 03:12:09,783::task::495::TaskManager.Task::(_debug) Task bf123faf-fc5f-4e40-b6b7-f4152f51da98: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913741::DEBUG::2013-11-20 03:12:09,784::task::495::TaskManager.Task::(_debug) Task bf123faf-fc5f-4e40-b6b7-f4152f51da98: ref 1 aborting False
>Thread-15913741::WARNING::2013-11-20 03:12:09,786::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913741::DEBUG::2013-11-20 03:12:09,786::task::495::TaskManager.Task::(_debug) Task bf123faf-fc5f-4e40-b6b7-f4152f51da98: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913741::DEBUG::2013-11-20 03:12:09,787::task::495::TaskManager.Task::(_debug) Task bf123faf-fc5f-4e40-b6b7-f4152f51da98: moving from state preparing -> state finished
>Thread-15913741::DEBUG::2013-11-20 03:12:09,788::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913741::DEBUG::2013-11-20 03:12:09,788::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913741::DEBUG::2013-11-20 03:12:09,789::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913741::DEBUG::2013-11-20 03:12:09,789::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913741::DEBUG::2013-11-20 03:12:09,790::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913741::DEBUG::2013-11-20 03:12:09,790::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913741::DEBUG::2013-11-20 03:12:09,791::task::495::TaskManager.Task::(_debug) Task bf123faf-fc5f-4e40-b6b7-f4152f51da98: ref 0 aborting False
>Thread-15913741::INFO::2013-11-20 03:12:09,791::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:12:10,009::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0475086 s, 21.6 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 03:12:11,465::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:12:11,466::task::495::TaskManager.Task::(_debug) Task a700e5ad-5866-49e7-ab22-b69b50d91e77: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:12:11,467::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a394817a-d712-4ca0-b9a0-74b7dc587491`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:12:11,468::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:12:11,468::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:12:11,469::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a394817a-d712-4ca0-b9a0-74b7dc587491`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:12:11,469::task::495::TaskManager.Task::(_debug) Task a700e5ad-5866-49e7-ab22-b69b50d91e77: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:12:11,470::task::495::TaskManager.Task::(_debug) Task a700e5ad-5866-49e7-ab22-b69b50d91e77: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:12:11,471::task::495::TaskManager.Task::(_debug) Task a700e5ad-5866-49e7-ab22-b69b50d91e77: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:12:11,471::task::495::TaskManager.Task::(_debug) Task a700e5ad-5866-49e7-ab22-b69b50d91e77: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:12:11,472::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:12:11,472::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:12:11,473::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:12:11,473::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:12:11,474::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:12:11,474::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:12:11,475::task::495::TaskManager.Task::(_debug) Task a700e5ad-5866-49e7-ab22-b69b50d91e77: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:12:11,475::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:12:12,018::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:12:13,292::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0523678 s, 19.6 MB/s\n'; <rc> = 0
>Thread-15913639::INFO::2013-11-20 03:12:14,464::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:12:14,469::task::495::TaskManager.Task::(_debug) Task 89f4e729-5f9f-4dfa-95f1-0ae4cf113497: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:12:14,471::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`292551e2-ea4d-4ac8-aa7a-750e7ed0d344`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:12:14,472::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:12:14,473::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:12:14,473::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`292551e2-ea4d-4ac8-aa7a-750e7ed0d344`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:12:14,474::task::495::TaskManager.Task::(_debug) Task 89f4e729-5f9f-4dfa-95f1-0ae4cf113497: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:12:14,475::task::495::TaskManager.Task::(_debug) Task 89f4e729-5f9f-4dfa-95f1-0ae4cf113497: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:12:14,475::task::495::TaskManager.Task::(_debug) Task 89f4e729-5f9f-4dfa-95f1-0ae4cf113497: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:12:14,476::task::495::TaskManager.Task::(_debug) Task 89f4e729-5f9f-4dfa-95f1-0ae4cf113497: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:12:14,477::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:12:14,477::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:12:14,478::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:12:14,479::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:12:14,479::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:12:14,480::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:12:14,480::task::495::TaskManager.Task::(_debug) Task 89f4e729-5f9f-4dfa-95f1-0ae4cf113497: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:12:14,481::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:12:15,303::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:12:15,709::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0468457 s, 21.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:12:17,719::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:12:18,930::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.050402 s, 20.3 MB/s\n'; <rc> = 0
>Thread-15913745::DEBUG::2013-11-20 03:12:20,222::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913745::INFO::2013-11-20 03:12:20,223::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913745::DEBUG::2013-11-20 03:12:20,223::task::495::TaskManager.Task::(_debug) Task 4670e633-10c5-4781-bac4-65ebbc0be1d9: moving from state init -> state preparing
>Thread-15913745::DEBUG::2013-11-20 03:12:20,224::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913745::DEBUG::2013-11-20 03:12:20,224::task::495::TaskManager.Task::(_debug) Task 4670e633-10c5-4781-bac4-65ebbc0be1d9: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913745::DEBUG::2013-11-20 03:12:20,225::task::495::TaskManager.Task::(_debug) Task 4670e633-10c5-4781-bac4-65ebbc0be1d9: moving from state preparing -> state finished
>Thread-15913745::DEBUG::2013-11-20 03:12:20,225::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913745::DEBUG::2013-11-20 03:12:20,226::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913745::DEBUG::2013-11-20 03:12:20,226::task::495::TaskManager.Task::(_debug) Task 4670e633-10c5-4781-bac4-65ebbc0be1d9: ref 0 aborting False
>Thread-15913745::INFO::2013-11-20 03:12:20,227::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913746::DEBUG::2013-11-20 03:12:20,231::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913746::INFO::2013-11-20 03:12:20,231::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913746::DEBUG::2013-11-20 03:12:20,232::task::495::TaskManager.Task::(_debug) Task 8905db49-258d-41e7-a400-b95a54a9a7da: moving from state init -> state preparing
>Thread-15913746::DEBUG::2013-11-20 03:12:20,233::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9ca352ff-670d-4f79-974b-7d1d0c044fda`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913746::DEBUG::2013-11-20 03:12:20,233::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913746::DEBUG::2013-11-20 03:12:20,234::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913746::DEBUG::2013-11-20 03:12:20,234::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9ca352ff-670d-4f79-974b-7d1d0c044fda`::Granted request
>Thread-15913746::DEBUG::2013-11-20 03:12:20,235::task::495::TaskManager.Task::(_debug) Task 8905db49-258d-41e7-a400-b95a54a9a7da: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913746::DEBUG::2013-11-20 03:12:20,236::task::495::TaskManager.Task::(_debug) Task 8905db49-258d-41e7-a400-b95a54a9a7da: ref 1 aborting False
>Thread-15913746::WARNING::2013-11-20 03:12:20,237::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913746::DEBUG::2013-11-20 03:12:20,238::task::495::TaskManager.Task::(_debug) Task 8905db49-258d-41e7-a400-b95a54a9a7da: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913746::DEBUG::2013-11-20 03:12:20,239::task::495::TaskManager.Task::(_debug) Task 8905db49-258d-41e7-a400-b95a54a9a7da: moving from state preparing -> state finished
>Thread-15913746::DEBUG::2013-11-20 03:12:20,239::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913746::DEBUG::2013-11-20 03:12:20,239::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913746::DEBUG::2013-11-20 03:12:20,240::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913746::DEBUG::2013-11-20 03:12:20,240::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913746::DEBUG::2013-11-20 03:12:20,240::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913746::DEBUG::2013-11-20 03:12:20,241::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913746::DEBUG::2013-11-20 03:12:20,241::task::495::TaskManager.Task::(_debug) Task 8905db49-258d-41e7-a400-b95a54a9a7da: ref 0 aborting False
>Thread-15913746::INFO::2013-11-20 03:12:20,241::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913747::INFO::2013-11-20 03:12:20,916::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913747::DEBUG::2013-11-20 03:12:20,917::task::495::TaskManager.Task::(_debug) Task 721af1fb-84d5-453b-b6c1-0ff53d4d863c: moving from state init -> state preparing
>Thread-15913747::DEBUG::2013-11-20 03:12:20,917::task::495::TaskManager.Task::(_debug) Task 721af1fb-84d5-453b-b6c1-0ff53d4d863c: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000274896621704', 'lastCheck': 1384917139.469301, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0367619991302', 'lastCheck': 1384917136.5124249, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0399279594421', 'lastCheck': 1384917137.5164299, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000701904296875', 'lastCheck': 1384917138.4712701, 'valid': True, 'code': 0}}
>Thread-15913747::DEBUG::2013-11-20 03:12:20,918::task::495::TaskManager.Task::(_debug) Task 721af1fb-84d5-453b-b6c1-0ff53d4d863c: moving from state preparing -> state finished
>Thread-15913747::DEBUG::2013-11-20 03:12:20,918::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913747::DEBUG::2013-11-20 03:12:20,918::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913747::DEBUG::2013-11-20 03:12:20,919::task::495::TaskManager.Task::(_debug) Task 721af1fb-84d5-453b-b6c1-0ff53d4d863c: ref 0 aborting False
>Thread-15913747::INFO::2013-11-20 03:12:20,919::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000274896621704', 'lastCheck': 1384917139.469301, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0367619991302', 'lastCheck': 1384917136.5124249, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0399279594421', 'lastCheck': 1384917137.5164299, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000701904296875', 'lastCheck': 1384917138.4712701, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:12:20,939::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:12:21,389::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0468736 s, 21.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:12:23,399::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:12:24,417::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0547368 s, 18.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:12:26,428::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:12:26,966::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0467099 s, 21.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:12:28,976::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:12:29,384::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0499612 s, 20.5 MB/s\n'; <rc> = 0
>Thread-15913752::DEBUG::2013-11-20 03:12:30,793::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913752::INFO::2013-11-20 03:12:30,794::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913752::DEBUG::2013-11-20 03:12:30,795::task::495::TaskManager.Task::(_debug) Task 2555e5ca-d16d-4fe7-a036-bea06b403f30: moving from state init -> state preparing
>Thread-15913752::DEBUG::2013-11-20 03:12:30,795::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913752::DEBUG::2013-11-20 03:12:30,796::task::495::TaskManager.Task::(_debug) Task 2555e5ca-d16d-4fe7-a036-bea06b403f30: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913752::DEBUG::2013-11-20 03:12:30,796::task::495::TaskManager.Task::(_debug) Task 2555e5ca-d16d-4fe7-a036-bea06b403f30: moving from state preparing -> state finished
>Thread-15913752::DEBUG::2013-11-20 03:12:30,797::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913752::DEBUG::2013-11-20 03:12:30,797::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913752::DEBUG::2013-11-20 03:12:30,797::task::495::TaskManager.Task::(_debug) Task 2555e5ca-d16d-4fe7-a036-bea06b403f30: ref 0 aborting False
>Thread-15913752::INFO::2013-11-20 03:12:30,798::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913753::DEBUG::2013-11-20 03:12:30,801::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913753::INFO::2013-11-20 03:12:30,802::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913753::DEBUG::2013-11-20 03:12:30,802::task::495::TaskManager.Task::(_debug) Task 9a81aeb6-a106-454d-954a-ffcce1f42894: moving from state init -> state preparing
>Thread-15913753::DEBUG::2013-11-20 03:12:30,803::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`041b2a06-a692-4c95-bb3f-d495d9f1a966`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913753::DEBUG::2013-11-20 03:12:30,803::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913753::DEBUG::2013-11-20 03:12:30,804::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913753::DEBUG::2013-11-20 03:12:30,804::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`041b2a06-a692-4c95-bb3f-d495d9f1a966`::Granted request
>Thread-15913753::DEBUG::2013-11-20 03:12:30,805::task::495::TaskManager.Task::(_debug) Task 9a81aeb6-a106-454d-954a-ffcce1f42894: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913753::DEBUG::2013-11-20 03:12:30,805::task::495::TaskManager.Task::(_debug) Task 9a81aeb6-a106-454d-954a-ffcce1f42894: ref 1 aborting False
>Thread-15913753::WARNING::2013-11-20 03:12:30,807::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913753::DEBUG::2013-11-20 03:12:30,807::task::495::TaskManager.Task::(_debug) Task 9a81aeb6-a106-454d-954a-ffcce1f42894: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913753::DEBUG::2013-11-20 03:12:30,808::task::495::TaskManager.Task::(_debug) Task 9a81aeb6-a106-454d-954a-ffcce1f42894: moving from state preparing -> state finished
>Thread-15913753::DEBUG::2013-11-20 03:12:30,808::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913753::DEBUG::2013-11-20 03:12:30,808::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913753::DEBUG::2013-11-20 03:12:30,809::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913753::DEBUG::2013-11-20 03:12:30,809::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913753::DEBUG::2013-11-20 03:12:30,810::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913753::DEBUG::2013-11-20 03:12:30,810::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913753::DEBUG::2013-11-20 03:12:30,810::task::495::TaskManager.Task::(_debug) Task 9a81aeb6-a106-454d-954a-ffcce1f42894: ref 0 aborting False
>Thread-15913753::INFO::2013-11-20 03:12:30,811::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:12:31,394::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:12:32,388::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0495414 s, 20.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:12:34,398::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 03:12:34,616::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:12:34,617::task::495::TaskManager.Task::(_debug) Task 070031ab-9dd8-45bf-9879-c40088fb7350: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:12:34,620::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`baed3696-781b-45ea-9b4d-c183ce176a29`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:12:34,621::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:12:34,622::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:12:34,622::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`baed3696-781b-45ea-9b4d-c183ce176a29`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:12:34,624::task::495::TaskManager.Task::(_debug) Task 070031ab-9dd8-45bf-9879-c40088fb7350: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:12:34,624::task::495::TaskManager.Task::(_debug) Task 070031ab-9dd8-45bf-9879-c40088fb7350: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:12:34,625::task::495::TaskManager.Task::(_debug) Task 070031ab-9dd8-45bf-9879-c40088fb7350: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:12:34,626::task::495::TaskManager.Task::(_debug) Task 070031ab-9dd8-45bf-9879-c40088fb7350: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:12:34,626::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:12:34,627::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:12:34,627::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:12:34,627::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:12:34,628::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:12:34,628::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:12:34,628::task::495::TaskManager.Task::(_debug) Task 070031ab-9dd8-45bf-9879-c40088fb7350: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:12:34,629::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:12:34,818::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0556724 s, 18.4 MB/s\n'; <rc> = 0
>Thread-15913755::INFO::2013-11-20 03:12:36,645::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913755::DEBUG::2013-11-20 03:12:36,646::task::495::TaskManager.Task::(_debug) Task 9bf4140f-2c3a-415a-8c84-d41d31cbb530: moving from state init -> state preparing
>Thread-15913755::DEBUG::2013-11-20 03:12:36,646::task::495::TaskManager.Task::(_debug) Task 9bf4140f-2c3a-415a-8c84-d41d31cbb530: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000273942947388', 'lastCheck': 1384917149.617115, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.000874042510986', 'lastCheck': 1384917156.6318271, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00113201141357', 'lastCheck': 1384917147.6141601, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000324010848999', 'lastCheck': 1384917148.6131041, 'valid': True, 'code': 0}}
>Thread-15913755::DEBUG::2013-11-20 03:12:36,647::task::495::TaskManager.Task::(_debug) Task 9bf4140f-2c3a-415a-8c84-d41d31cbb530: moving from state preparing -> state finished
>Thread-15913755::DEBUG::2013-11-20 03:12:36,647::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913755::DEBUG::2013-11-20 03:12:36,647::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913755::DEBUG::2013-11-20 03:12:36,648::task::495::TaskManager.Task::(_debug) Task 9bf4140f-2c3a-415a-8c84-d41d31cbb530: ref 0 aborting False
>Thread-15913755::INFO::2013-11-20 03:12:36,648::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000273942947388', 'lastCheck': 1384917149.617115, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.000874042510986', 'lastCheck': 1384917156.6318271, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00113201141357', 'lastCheck': 1384917147.6141601, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000324010848999', 'lastCheck': 1384917148.6131041, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:12:36,832::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:12:37,272::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0486889 s, 21.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:12:39,282::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:12:39,736::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0488889 s, 20.9 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 03:12:41,624::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:12:41,624::task::495::TaskManager.Task::(_debug) Task 62d91d3e-9d03-4b28-b836-806068189dfb: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:12:41,633::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8e162c8d-cd48-46e5-a2e0-503828d432ea`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:12:41,634::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:12:41,634::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:12:41,635::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8e162c8d-cd48-46e5-a2e0-503828d432ea`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:12:41,636::task::495::TaskManager.Task::(_debug) Task 62d91d3e-9d03-4b28-b836-806068189dfb: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:12:41,636::task::495::TaskManager.Task::(_debug) Task 62d91d3e-9d03-4b28-b836-806068189dfb: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:12:41,637::task::495::TaskManager.Task::(_debug) Task 62d91d3e-9d03-4b28-b836-806068189dfb: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:12:41,637::task::495::TaskManager.Task::(_debug) Task 62d91d3e-9d03-4b28-b836-806068189dfb: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:12:41,637::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:12:41,638::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:12:41,638::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:12:41,639::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:12:41,639::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:12:41,639::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:12:41,640::task::495::TaskManager.Task::(_debug) Task 62d91d3e-9d03-4b28-b836-806068189dfb: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:12:41,640::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:12:41,746::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913758::DEBUG::2013-11-20 03:12:42,208::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913758::INFO::2013-11-20 03:12:42,210::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913758::DEBUG::2013-11-20 03:12:42,210::task::495::TaskManager.Task::(_debug) Task bc363295-d772-467d-929a-300e5f0babe7: moving from state init -> state preparing
>Thread-15913758::DEBUG::2013-11-20 03:12:42,211::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913758::DEBUG::2013-11-20 03:12:42,212::task::495::TaskManager.Task::(_debug) Task bc363295-d772-467d-929a-300e5f0babe7: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913758::DEBUG::2013-11-20 03:12:42,212::task::495::TaskManager.Task::(_debug) Task bc363295-d772-467d-929a-300e5f0babe7: moving from state preparing -> state finished
>Thread-15913758::DEBUG::2013-11-20 03:12:42,213::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913758::DEBUG::2013-11-20 03:12:42,213::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913758::DEBUG::2013-11-20 03:12:42,214::task::495::TaskManager.Task::(_debug) Task bc363295-d772-467d-929a-300e5f0babe7: ref 0 aborting False
>Thread-15913758::INFO::2013-11-20 03:12:42,214::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913759::DEBUG::2013-11-20 03:12:42,219::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913759::INFO::2013-11-20 03:12:42,220::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913759::DEBUG::2013-11-20 03:12:42,220::task::495::TaskManager.Task::(_debug) Task dd0f2ce1-8f46-4014-942c-6df8202780a2: moving from state init -> state preparing
>Thread-15913759::DEBUG::2013-11-20 03:12:42,222::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7e6ed0e1-3837-4172-b8d8-b47d56520ce3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913759::DEBUG::2013-11-20 03:12:42,222::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913759::DEBUG::2013-11-20 03:12:42,223::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913759::DEBUG::2013-11-20 03:12:42,223::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7e6ed0e1-3837-4172-b8d8-b47d56520ce3`::Granted request
>Thread-15913759::DEBUG::2013-11-20 03:12:42,224::task::495::TaskManager.Task::(_debug) Task dd0f2ce1-8f46-4014-942c-6df8202780a2: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913759::DEBUG::2013-11-20 03:12:42,225::task::495::TaskManager.Task::(_debug) Task dd0f2ce1-8f46-4014-942c-6df8202780a2: ref 1 aborting False
>Thread-15913759::WARNING::2013-11-20 03:12:42,227::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913759::DEBUG::2013-11-20 03:12:42,228::task::495::TaskManager.Task::(_debug) Task dd0f2ce1-8f46-4014-942c-6df8202780a2: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913759::DEBUG::2013-11-20 03:12:42,228::task::495::TaskManager.Task::(_debug) Task dd0f2ce1-8f46-4014-942c-6df8202780a2: moving from state preparing -> state finished
>Thread-15913759::DEBUG::2013-11-20 03:12:42,229::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913759::DEBUG::2013-11-20 03:12:42,229::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913759::DEBUG::2013-11-20 03:12:42,230::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913759::DEBUG::2013-11-20 03:12:42,230::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913759::DEBUG::2013-11-20 03:12:42,231::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913759::DEBUG::2013-11-20 03:12:42,231::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913759::DEBUG::2013-11-20 03:12:42,231::task::495::TaskManager.Task::(_debug) Task dd0f2ce1-8f46-4014-942c-6df8202780a2: ref 0 aborting False
>Thread-15913759::INFO::2013-11-20 03:12:42,232::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:12:42,698::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0525957 s, 19.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:12:44,708::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:12:45,104::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0465989 s, 22.0 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 03:12:46,950::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:12:46,951::task::495::TaskManager.Task::(_debug) Task a2b937d7-3a70-4dad-a1a7-ed6f011db46b: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:12:46,953::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9fb59804-8580-4dc6-b206-08d36a0f1852`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:12:46,953::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:12:46,954::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:12:46,954::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9fb59804-8580-4dc6-b206-08d36a0f1852`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:12:46,955::task::495::TaskManager.Task::(_debug) Task a2b937d7-3a70-4dad-a1a7-ed6f011db46b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:12:46,955::task::495::TaskManager.Task::(_debug) Task a2b937d7-3a70-4dad-a1a7-ed6f011db46b: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:12:46,956::task::495::TaskManager.Task::(_debug) Task a2b937d7-3a70-4dad-a1a7-ed6f011db46b: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:12:46,956::task::495::TaskManager.Task::(_debug) Task a2b937d7-3a70-4dad-a1a7-ed6f011db46b: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:12:46,957::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:12:46,957::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:12:46,957::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:12:46,958::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:12:46,958::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:12:46,958::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:12:46,959::task::495::TaskManager.Task::(_debug) Task a2b937d7-3a70-4dad-a1a7-ed6f011db46b: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:12:46,959::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:12:46,960::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:12:46,960::task::495::TaskManager.Task::(_debug) Task ddb7315f-8ff5-4479-88f3-26e73d97e260: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:12:46,961::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bddc9ba8-93ed-4095-9ba1-d6ab5e50f8da`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:12:46,961::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:12:46,961::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:12:46,962::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bddc9ba8-93ed-4095-9ba1-d6ab5e50f8da`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:12:46,962::task::495::TaskManager.Task::(_debug) Task ddb7315f-8ff5-4479-88f3-26e73d97e260: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:12:46,962::task::495::TaskManager.Task::(_debug) Task ddb7315f-8ff5-4479-88f3-26e73d97e260: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:12:46,963::task::495::TaskManager.Task::(_debug) Task ddb7315f-8ff5-4479-88f3-26e73d97e260: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:12:46,963::task::495::TaskManager.Task::(_debug) Task ddb7315f-8ff5-4479-88f3-26e73d97e260: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:12:46,963::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:12:46,964::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:12:46,964::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:12:46,965::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:12:46,965::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:12:46,965::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:12:46,965::task::495::TaskManager.Task::(_debug) Task ddb7315f-8ff5-4479-88f3-26e73d97e260: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:12:46,966::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:12:47,114::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:12:47,540::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0461336 s, 22.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:12:49,550::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:12:50,575::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0494876 s, 20.7 MB/s\n'; <rc> = 0
>Thread-15913763::INFO::2013-11-20 03:12:52,219::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913763::DEBUG::2013-11-20 03:12:52,220::task::495::TaskManager.Task::(_debug) Task 46a1c140-41fa-4775-a69d-bcc0bd1dcce5: moving from state init -> state preparing
>Thread-15913763::DEBUG::2013-11-20 03:12:52,221::task::495::TaskManager.Task::(_debug) Task 46a1c140-41fa-4775-a69d-bcc0bd1dcce5: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000322818756104', 'lastCheck': 1384917169.7453351, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0147819519043', 'lastCheck': 1384917166.6587031, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0144040584564', 'lastCheck': 1384917167.6542349, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000259876251221', 'lastCheck': 1384917168.633007, 'valid': True, 'code': 0}}
>Thread-15913763::DEBUG::2013-11-20 03:12:52,222::task::495::TaskManager.Task::(_debug) Task 46a1c140-41fa-4775-a69d-bcc0bd1dcce5: moving from state preparing -> state finished
>Thread-15913763::DEBUG::2013-11-20 03:12:52,222::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913763::DEBUG::2013-11-20 03:12:52,223::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913763::DEBUG::2013-11-20 03:12:52,223::task::495::TaskManager.Task::(_debug) Task 46a1c140-41fa-4775-a69d-bcc0bd1dcce5: ref 0 aborting False
>Thread-15913763::INFO::2013-11-20 03:12:52,224::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000322818756104', 'lastCheck': 1384917169.7453351, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0147819519043', 'lastCheck': 1384917166.6587031, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0144040584564', 'lastCheck': 1384917167.6542349, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000259876251221', 'lastCheck': 1384917168.633007, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:12:52,585::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913765::DEBUG::2013-11-20 03:12:52,759::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913765::INFO::2013-11-20 03:12:52,761::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913765::DEBUG::2013-11-20 03:12:52,761::task::495::TaskManager.Task::(_debug) Task 920db819-04ec-4741-ac72-fea44ad77228: moving from state init -> state preparing
>Thread-15913765::DEBUG::2013-11-20 03:12:52,762::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913765::DEBUG::2013-11-20 03:12:52,763::task::495::TaskManager.Task::(_debug) Task 920db819-04ec-4741-ac72-fea44ad77228: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913765::DEBUG::2013-11-20 03:12:52,763::task::495::TaskManager.Task::(_debug) Task 920db819-04ec-4741-ac72-fea44ad77228: moving from state preparing -> state finished
>Thread-15913765::DEBUG::2013-11-20 03:12:52,764::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913765::DEBUG::2013-11-20 03:12:52,764::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913765::DEBUG::2013-11-20 03:12:52,765::task::495::TaskManager.Task::(_debug) Task 920db819-04ec-4741-ac72-fea44ad77228: ref 0 aborting False
>Thread-15913765::INFO::2013-11-20 03:12:52,766::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913766::DEBUG::2013-11-20 03:12:52,770::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913766::INFO::2013-11-20 03:12:52,771::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913766::DEBUG::2013-11-20 03:12:52,771::task::495::TaskManager.Task::(_debug) Task 615ed387-8db8-4a3c-8159-19e6cdd38c94: moving from state init -> state preparing
>Thread-15913766::DEBUG::2013-11-20 03:12:52,772::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2ee3df7f-acab-436e-b633-ecb24821083c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913766::DEBUG::2013-11-20 03:12:52,773::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913766::DEBUG::2013-11-20 03:12:52,774::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913766::DEBUG::2013-11-20 03:12:52,774::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2ee3df7f-acab-436e-b633-ecb24821083c`::Granted request
>Thread-15913766::DEBUG::2013-11-20 03:12:52,775::task::495::TaskManager.Task::(_debug) Task 615ed387-8db8-4a3c-8159-19e6cdd38c94: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913766::DEBUG::2013-11-20 03:12:52,775::task::495::TaskManager.Task::(_debug) Task 615ed387-8db8-4a3c-8159-19e6cdd38c94: ref 1 aborting False
>Thread-15913766::WARNING::2013-11-20 03:12:52,777::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913766::DEBUG::2013-11-20 03:12:52,778::task::495::TaskManager.Task::(_debug) Task 615ed387-8db8-4a3c-8159-19e6cdd38c94: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913766::DEBUG::2013-11-20 03:12:52,778::task::495::TaskManager.Task::(_debug) Task 615ed387-8db8-4a3c-8159-19e6cdd38c94: moving from state preparing -> state finished
>Thread-15913766::DEBUG::2013-11-20 03:12:52,778::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913766::DEBUG::2013-11-20 03:12:52,779::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913766::DEBUG::2013-11-20 03:12:52,779::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913766::DEBUG::2013-11-20 03:12:52,780::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913766::DEBUG::2013-11-20 03:12:52,780::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913766::DEBUG::2013-11-20 03:12:52,780::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913766::DEBUG::2013-11-20 03:12:52,781::task::495::TaskManager.Task::(_debug) Task 615ed387-8db8-4a3c-8159-19e6cdd38c94: ref 0 aborting False
>Thread-15913766::INFO::2013-11-20 03:12:52,781::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:12:57,328::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0851201 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:12:59,338::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:12:59,923::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0531298 s, 19.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:13:01,933::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:13:02,348::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0478763 s, 21.4 MB/s\n'; <rc> = 0
>Thread-15913770::DEBUG::2013-11-20 03:13:02,925::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913770::INFO::2013-11-20 03:13:02,926::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913770::DEBUG::2013-11-20 03:13:02,927::task::495::TaskManager.Task::(_debug) Task e397ad10-ea60-4450-9256-bb7fc888f585: moving from state init -> state preparing
>Thread-15913770::DEBUG::2013-11-20 03:13:02,928::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913770::DEBUG::2013-11-20 03:13:02,928::task::495::TaskManager.Task::(_debug) Task e397ad10-ea60-4450-9256-bb7fc888f585: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913770::DEBUG::2013-11-20 03:13:02,928::task::495::TaskManager.Task::(_debug) Task e397ad10-ea60-4450-9256-bb7fc888f585: moving from state preparing -> state finished
>Thread-15913770::DEBUG::2013-11-20 03:13:02,929::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913770::DEBUG::2013-11-20 03:13:02,929::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913770::DEBUG::2013-11-20 03:13:02,929::task::495::TaskManager.Task::(_debug) Task e397ad10-ea60-4450-9256-bb7fc888f585: ref 0 aborting False
>Thread-15913770::INFO::2013-11-20 03:13:02,930::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913771::DEBUG::2013-11-20 03:13:02,933::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913771::INFO::2013-11-20 03:13:02,934::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913771::DEBUG::2013-11-20 03:13:02,934::task::495::TaskManager.Task::(_debug) Task 0f721744-c90d-462d-9d03-00d1646a45a4: moving from state init -> state preparing
>Thread-15913771::DEBUG::2013-11-20 03:13:02,935::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`918637d0-bc65-4c6d-84db-60ca3e4c3131`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913771::DEBUG::2013-11-20 03:13:02,935::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913771::DEBUG::2013-11-20 03:13:02,936::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913771::DEBUG::2013-11-20 03:13:02,936::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`918637d0-bc65-4c6d-84db-60ca3e4c3131`::Granted request
>Thread-15913771::DEBUG::2013-11-20 03:13:02,937::task::495::TaskManager.Task::(_debug) Task 0f721744-c90d-462d-9d03-00d1646a45a4: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913771::DEBUG::2013-11-20 03:13:02,937::task::495::TaskManager.Task::(_debug) Task 0f721744-c90d-462d-9d03-00d1646a45a4: ref 1 aborting False
>Thread-15913771::WARNING::2013-11-20 03:13:02,939::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913771::DEBUG::2013-11-20 03:13:02,939::task::495::TaskManager.Task::(_debug) Task 0f721744-c90d-462d-9d03-00d1646a45a4: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913771::DEBUG::2013-11-20 03:13:02,940::task::495::TaskManager.Task::(_debug) Task 0f721744-c90d-462d-9d03-00d1646a45a4: moving from state preparing -> state finished
>Thread-15913771::DEBUG::2013-11-20 03:13:02,940::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913771::DEBUG::2013-11-20 03:13:02,941::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913771::DEBUG::2013-11-20 03:13:02,941::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913771::DEBUG::2013-11-20 03:13:02,942::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913771::DEBUG::2013-11-20 03:13:02,942::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913771::DEBUG::2013-11-20 03:13:02,942::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913771::DEBUG::2013-11-20 03:13:02,943::task::495::TaskManager.Task::(_debug) Task 0f721744-c90d-462d-9d03-00d1646a45a4: ref 0 aborting False
>Thread-15913771::INFO::2013-11-20 03:13:02,943::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:13:04,362::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:13:04,839::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0517386 s, 19.8 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 03:13:04,895::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:13:04,896::task::495::TaskManager.Task::(_debug) Task d30ecc33-eeaa-4cbf-9dfc-e36f21df7dc0: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:13:04,897::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`0c87e439-5549-4041-ba33-0cde170ff139`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:13:04,897::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:13:04,898::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:13:04,898::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`0c87e439-5549-4041-ba33-0cde170ff139`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:13:04,899::task::495::TaskManager.Task::(_debug) Task d30ecc33-eeaa-4cbf-9dfc-e36f21df7dc0: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:13:04,900::task::495::TaskManager.Task::(_debug) Task d30ecc33-eeaa-4cbf-9dfc-e36f21df7dc0: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:13:04,900::task::495::TaskManager.Task::(_debug) Task d30ecc33-eeaa-4cbf-9dfc-e36f21df7dc0: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:13:04,901::task::495::TaskManager.Task::(_debug) Task d30ecc33-eeaa-4cbf-9dfc-e36f21df7dc0: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:13:04,901::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:13:04,901::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:13:04,902::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:13:04,902::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:13:04,903::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:13:04,903::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:13:04,904::task::495::TaskManager.Task::(_debug) Task d30ecc33-eeaa-4cbf-9dfc-e36f21df7dc0: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:13:04,904::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:13:04,905::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:13:04,905::task::495::TaskManager.Task::(_debug) Task f34de3d5-752a-4d6d-a7eb-d8bcf1428028: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:13:04,906::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`761d8b19-055e-4e30-9ed2-921ce051b71b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:13:04,906::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:13:04,907::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:13:04,907::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`761d8b19-055e-4e30-9ed2-921ce051b71b`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:13:04,908::task::495::TaskManager.Task::(_debug) Task f34de3d5-752a-4d6d-a7eb-d8bcf1428028: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:13:04,908::task::495::TaskManager.Task::(_debug) Task f34de3d5-752a-4d6d-a7eb-d8bcf1428028: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:13:04,909::task::495::TaskManager.Task::(_debug) Task f34de3d5-752a-4d6d-a7eb-d8bcf1428028: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:13:04,909::task::495::TaskManager.Task::(_debug) Task f34de3d5-752a-4d6d-a7eb-d8bcf1428028: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:13:04,909::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:13:04,910::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:13:04,910::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:13:04,911::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:13:04,911::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:13:04,911::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:13:04,912::task::495::TaskManager.Task::(_debug) Task f34de3d5-752a-4d6d-a7eb-d8bcf1428028: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:13:04,912::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:13:04,913::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:13:04,913::task::495::TaskManager.Task::(_debug) Task 38f6a643-52af-488d-ae7f-68d03a92cfe9: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:13:04,914::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`9f18de12-8710-4f52-8df2-509f546b63c2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:13:04,914::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:13:04,915::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:13:04,915::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`9f18de12-8710-4f52-8df2-509f546b63c2`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:13:04,916::task::495::TaskManager.Task::(_debug) Task 38f6a643-52af-488d-ae7f-68d03a92cfe9: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:13:04,916::task::495::TaskManager.Task::(_debug) Task 38f6a643-52af-488d-ae7f-68d03a92cfe9: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:13:04,917::task::495::TaskManager.Task::(_debug) Task 38f6a643-52af-488d-ae7f-68d03a92cfe9: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:13:04,917::task::495::TaskManager.Task::(_debug) Task 38f6a643-52af-488d-ae7f-68d03a92cfe9: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:13:04,917::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:13:04,918::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:13:04,918::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:13:04,919::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:13:04,919::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:13:04,920::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:13:04,920::task::495::TaskManager.Task::(_debug) Task 38f6a643-52af-488d-ae7f-68d03a92cfe9: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:13:04,920::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:13:04,921::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:13:04,921::task::495::TaskManager.Task::(_debug) Task c6526e42-864d-48f0-a2a3-ddee4feddb93: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:13:04,922::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`51927a99-ee24-4dae-be2d-fc1aa39525db`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:13:04,922::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:13:04,923::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:13:04,923::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`51927a99-ee24-4dae-be2d-fc1aa39525db`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:13:04,924::task::495::TaskManager.Task::(_debug) Task c6526e42-864d-48f0-a2a3-ddee4feddb93: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:13:04,924::task::495::TaskManager.Task::(_debug) Task c6526e42-864d-48f0-a2a3-ddee4feddb93: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:13:04,925::task::495::TaskManager.Task::(_debug) Task c6526e42-864d-48f0-a2a3-ddee4feddb93: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:13:04,925::task::495::TaskManager.Task::(_debug) Task c6526e42-864d-48f0-a2a3-ddee4feddb93: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:13:04,926::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:13:04,926::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:13:04,926::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:13:04,927::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:13:04,927::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:13:04,928::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:13:04,928::task::495::TaskManager.Task::(_debug) Task c6526e42-864d-48f0-a2a3-ddee4feddb93: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:13:04,928::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:13:06,850::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:13:07,294::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.053438 s, 19.2 MB/s\n'; <rc> = 0
>Thread-15913773::INFO::2013-11-20 03:13:07,601::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913773::DEBUG::2013-11-20 03:13:07,602::task::495::TaskManager.Task::(_debug) Task 2a529376-580e-40d6-8a90-d192dcb30820: moving from state init -> state preparing
>Thread-15913773::DEBUG::2013-11-20 03:13:07,602::task::495::TaskManager.Task::(_debug) Task 2a529376-580e-40d6-8a90-d192dcb30820: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000294923782349', 'lastCheck': 1384917179.7657461, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00983905792236', 'lastCheck': 1384917186.797807, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00885200500488', 'lastCheck': 1384917177.769592, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000303030014038', 'lastCheck': 1384917178.7576039, 'valid': True, 'code': 0}}
>Thread-15913773::DEBUG::2013-11-20 03:13:07,603::task::495::TaskManager.Task::(_debug) Task 2a529376-580e-40d6-8a90-d192dcb30820: moving from state preparing -> state finished
>Thread-15913773::DEBUG::2013-11-20 03:13:07,603::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913773::DEBUG::2013-11-20 03:13:07,604::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913773::DEBUG::2013-11-20 03:13:07,604::task::495::TaskManager.Task::(_debug) Task 2a529376-580e-40d6-8a90-d192dcb30820: ref 0 aborting False
>Thread-15913773::INFO::2013-11-20 03:13:07,604::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000294923782349', 'lastCheck': 1384917179.7657461, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00983905792236', 'lastCheck': 1384917186.797807, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00885200500488', 'lastCheck': 1384917177.769592, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000303030014038', 'lastCheck': 1384917178.7576039, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:13:09,307::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:13:10,366::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0483934 s, 21.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:13:12,376::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:13:12,843::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0519392 s, 19.7 MB/s\n'; <rc> = 0
>Thread-15913776::DEBUG::2013-11-20 03:13:13,214::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913776::INFO::2013-11-20 03:13:13,215::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913776::DEBUG::2013-11-20 03:13:13,216::task::495::TaskManager.Task::(_debug) Task 69366f69-cb68-4c83-bc8a-9ba6453cd81a: moving from state init -> state preparing
>Thread-15913776::DEBUG::2013-11-20 03:13:13,217::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913776::DEBUG::2013-11-20 03:13:13,217::task::495::TaskManager.Task::(_debug) Task 69366f69-cb68-4c83-bc8a-9ba6453cd81a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913776::DEBUG::2013-11-20 03:13:13,217::task::495::TaskManager.Task::(_debug) Task 69366f69-cb68-4c83-bc8a-9ba6453cd81a: moving from state preparing -> state finished
>Thread-15913776::DEBUG::2013-11-20 03:13:13,218::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913776::DEBUG::2013-11-20 03:13:13,218::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913776::DEBUG::2013-11-20 03:13:13,218::task::495::TaskManager.Task::(_debug) Task 69366f69-cb68-4c83-bc8a-9ba6453cd81a: ref 0 aborting False
>Thread-15913776::INFO::2013-11-20 03:13:13,219::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913777::DEBUG::2013-11-20 03:13:13,223::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913777::INFO::2013-11-20 03:13:13,224::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913777::DEBUG::2013-11-20 03:13:13,225::task::495::TaskManager.Task::(_debug) Task 2772eca0-b92f-40c8-973d-ca47e0f997b2: moving from state init -> state preparing
>Thread-15913777::DEBUG::2013-11-20 03:13:13,226::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c4760386-b97e-4464-bffc-1c6649e3820b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913777::DEBUG::2013-11-20 03:13:13,226::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913777::DEBUG::2013-11-20 03:13:13,227::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913777::DEBUG::2013-11-20 03:13:13,227::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c4760386-b97e-4464-bffc-1c6649e3820b`::Granted request
>Thread-15913777::DEBUG::2013-11-20 03:13:13,228::task::495::TaskManager.Task::(_debug) Task 2772eca0-b92f-40c8-973d-ca47e0f997b2: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913777::DEBUG::2013-11-20 03:13:13,229::task::495::TaskManager.Task::(_debug) Task 2772eca0-b92f-40c8-973d-ca47e0f997b2: ref 1 aborting False
>Thread-15913777::WARNING::2013-11-20 03:13:13,230::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913777::DEBUG::2013-11-20 03:13:13,231::task::495::TaskManager.Task::(_debug) Task 2772eca0-b92f-40c8-973d-ca47e0f997b2: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913777::DEBUG::2013-11-20 03:13:13,232::task::495::TaskManager.Task::(_debug) Task 2772eca0-b92f-40c8-973d-ca47e0f997b2: moving from state preparing -> state finished
>Thread-15913777::DEBUG::2013-11-20 03:13:13,232::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913777::DEBUG::2013-11-20 03:13:13,233::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913777::DEBUG::2013-11-20 03:13:13,233::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913777::DEBUG::2013-11-20 03:13:13,234::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913777::DEBUG::2013-11-20 03:13:13,234::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913777::DEBUG::2013-11-20 03:13:13,235::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913777::DEBUG::2013-11-20 03:13:13,235::task::495::TaskManager.Task::(_debug) Task 2772eca0-b92f-40c8-973d-ca47e0f997b2: ref 0 aborting False
>Thread-15913777::INFO::2013-11-20 03:13:13,236::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913639::INFO::2013-11-20 03:13:14,774::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:13:14,775::task::495::TaskManager.Task::(_debug) Task e4d6b235-b3de-401c-8130-fad9a0021226: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:13:14,780::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c5401be6-11aa-4591-83d0-9215d6a7a377`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:13:14,780::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:13:14,781::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:13:14,781::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c5401be6-11aa-4591-83d0-9215d6a7a377`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:13:14,782::task::495::TaskManager.Task::(_debug) Task e4d6b235-b3de-401c-8130-fad9a0021226: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:13:14,782::task::495::TaskManager.Task::(_debug) Task e4d6b235-b3de-401c-8130-fad9a0021226: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:13:14,783::task::495::TaskManager.Task::(_debug) Task e4d6b235-b3de-401c-8130-fad9a0021226: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:13:14,783::task::495::TaskManager.Task::(_debug) Task e4d6b235-b3de-401c-8130-fad9a0021226: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:13:14,783::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:13:14,784::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:13:14,784::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:13:14,785::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:13:14,785::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:13:14,785::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:13:14,786::task::495::TaskManager.Task::(_debug) Task e4d6b235-b3de-401c-8130-fad9a0021226: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:13:14,786::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:13:14,854::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:13:19,841::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0576206 s, 17.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:13:21,852::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:13:22,544::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0526362 s, 19.5 MB/s\n'; <rc> = 0
>Thread-15913781::INFO::2013-11-20 03:13:23,015::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913781::DEBUG::2013-11-20 03:13:23,016::task::495::TaskManager.Task::(_debug) Task d9fbf123-377b-4ae2-97c5-3ead329f4440: moving from state init -> state preparing
>Thread-15913781::DEBUG::2013-11-20 03:13:23,016::task::495::TaskManager.Task::(_debug) Task d9fbf123-377b-4ae2-97c5-3ead329f4440: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000207185745239', 'lastCheck': 1384917200.5524969, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0117881298065', 'lastCheck': 1384917196.8216479, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0257561206818', 'lastCheck': 1384917199.6156311, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0146560668945', 'lastCheck': 1384917199.5775321, 'valid': True, 'code': 0}}
>Thread-15913781::DEBUG::2013-11-20 03:13:23,017::task::495::TaskManager.Task::(_debug) Task d9fbf123-377b-4ae2-97c5-3ead329f4440: moving from state preparing -> state finished
>Thread-15913781::DEBUG::2013-11-20 03:13:23,017::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913781::DEBUG::2013-11-20 03:13:23,018::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913781::DEBUG::2013-11-20 03:13:23,018::task::495::TaskManager.Task::(_debug) Task d9fbf123-377b-4ae2-97c5-3ead329f4440: ref 0 aborting False
>Thread-15913781::INFO::2013-11-20 03:13:23,019::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000207185745239', 'lastCheck': 1384917200.5524969, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0117881298065', 'lastCheck': 1384917196.8216479, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0257561206818', 'lastCheck': 1384917199.6156311, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0146560668945', 'lastCheck': 1384917199.5775321, 'valid': True, 'code': 0}}
>Thread-15913783::DEBUG::2013-11-20 03:13:23,388::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913783::INFO::2013-11-20 03:13:23,389::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913783::DEBUG::2013-11-20 03:13:23,389::task::495::TaskManager.Task::(_debug) Task 7a2fb530-1f02-4794-89f2-435c6ae9d74b: moving from state init -> state preparing
>Thread-15913783::DEBUG::2013-11-20 03:13:23,389::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913783::DEBUG::2013-11-20 03:13:23,390::task::495::TaskManager.Task::(_debug) Task 7a2fb530-1f02-4794-89f2-435c6ae9d74b: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913783::DEBUG::2013-11-20 03:13:23,390::task::495::TaskManager.Task::(_debug) Task 7a2fb530-1f02-4794-89f2-435c6ae9d74b: moving from state preparing -> state finished
>Thread-15913783::DEBUG::2013-11-20 03:13:23,391::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913783::DEBUG::2013-11-20 03:13:23,391::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913783::DEBUG::2013-11-20 03:13:23,391::task::495::TaskManager.Task::(_debug) Task 7a2fb530-1f02-4794-89f2-435c6ae9d74b: ref 0 aborting False
>Thread-15913783::INFO::2013-11-20 03:13:23,392::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913784::DEBUG::2013-11-20 03:13:23,407::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913784::INFO::2013-11-20 03:13:23,407::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913784::DEBUG::2013-11-20 03:13:23,408::task::495::TaskManager.Task::(_debug) Task 5aa4f701-53dd-4862-b85d-3477def682c5: moving from state init -> state preparing
>Thread-15913784::DEBUG::2013-11-20 03:13:23,409::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f2aa0492-fb8b-4b04-9b7b-9c116a509b2b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913784::DEBUG::2013-11-20 03:13:23,409::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913784::DEBUG::2013-11-20 03:13:23,409::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913784::DEBUG::2013-11-20 03:13:23,410::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f2aa0492-fb8b-4b04-9b7b-9c116a509b2b`::Granted request
>Thread-15913784::DEBUG::2013-11-20 03:13:23,410::task::495::TaskManager.Task::(_debug) Task 5aa4f701-53dd-4862-b85d-3477def682c5: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913784::DEBUG::2013-11-20 03:13:23,411::task::495::TaskManager.Task::(_debug) Task 5aa4f701-53dd-4862-b85d-3477def682c5: ref 1 aborting False
>Thread-15913784::WARNING::2013-11-20 03:13:23,412::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913784::DEBUG::2013-11-20 03:13:23,413::task::495::TaskManager.Task::(_debug) Task 5aa4f701-53dd-4862-b85d-3477def682c5: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913784::DEBUG::2013-11-20 03:13:23,413::task::495::TaskManager.Task::(_debug) Task 5aa4f701-53dd-4862-b85d-3477def682c5: moving from state preparing -> state finished
>Thread-15913784::DEBUG::2013-11-20 03:13:23,414::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913784::DEBUG::2013-11-20 03:13:23,414::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913784::DEBUG::2013-11-20 03:13:23,414::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913784::DEBUG::2013-11-20 03:13:23,415::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913784::DEBUG::2013-11-20 03:13:23,415::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913784::DEBUG::2013-11-20 03:13:23,415::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913784::DEBUG::2013-11-20 03:13:23,416::task::495::TaskManager.Task::(_debug) Task 5aa4f701-53dd-4862-b85d-3477def682c5: ref 0 aborting False
>Thread-15913784::INFO::2013-11-20 03:13:23,416::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:13:24,563::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:13:24,998::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.046167 s, 22.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:13:27,008::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:13:29,080::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0486069 s, 21.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:13:31,090::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:13:31,475::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0473744 s, 21.6 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 03:13:32,134::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:13:32,135::task::495::TaskManager.Task::(_debug) Task 52d43cd9-580d-47e8-8c6e-cb80a4a8b799: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:13:32,136::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`217afc28-e62c-433c-9a67-b16f4518c5ea`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:13:32,137::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:13:32,137::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:13:32,138::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`217afc28-e62c-433c-9a67-b16f4518c5ea`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:13:32,139::task::495::TaskManager.Task::(_debug) Task 52d43cd9-580d-47e8-8c6e-cb80a4a8b799: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:13:32,139::task::495::TaskManager.Task::(_debug) Task 52d43cd9-580d-47e8-8c6e-cb80a4a8b799: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:13:32,140::task::495::TaskManager.Task::(_debug) Task 52d43cd9-580d-47e8-8c6e-cb80a4a8b799: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:13:32,140::task::495::TaskManager.Task::(_debug) Task 52d43cd9-580d-47e8-8c6e-cb80a4a8b799: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:13:32,141::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:13:32,141::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:13:32,142::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:13:32,142::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:13:32,143::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:13:32,143::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:13:32,144::task::495::TaskManager.Task::(_debug) Task 52d43cd9-580d-47e8-8c6e-cb80a4a8b799: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:13:32,145::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:13:33,486::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913788::DEBUG::2013-11-20 03:13:33,666::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913788::INFO::2013-11-20 03:13:33,667::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913788::DEBUG::2013-11-20 03:13:33,668::task::495::TaskManager.Task::(_debug) Task e14101b3-3906-4bcf-9e21-fc206242a7df: moving from state init -> state preparing
>Thread-15913788::DEBUG::2013-11-20 03:13:33,669::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913788::DEBUG::2013-11-20 03:13:33,670::task::495::TaskManager.Task::(_debug) Task e14101b3-3906-4bcf-9e21-fc206242a7df: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913788::DEBUG::2013-11-20 03:13:33,671::task::495::TaskManager.Task::(_debug) Task e14101b3-3906-4bcf-9e21-fc206242a7df: moving from state preparing -> state finished
>Thread-15913788::DEBUG::2013-11-20 03:13:33,672::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913788::DEBUG::2013-11-20 03:13:33,672::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913788::DEBUG::2013-11-20 03:13:33,673::task::495::TaskManager.Task::(_debug) Task e14101b3-3906-4bcf-9e21-fc206242a7df: ref 0 aborting False
>Thread-15913788::INFO::2013-11-20 03:13:33,674::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913789::DEBUG::2013-11-20 03:13:33,679::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913789::INFO::2013-11-20 03:13:33,679::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913789::DEBUG::2013-11-20 03:13:33,680::task::495::TaskManager.Task::(_debug) Task 1285469f-99a9-4441-9e8b-8b4764cf1ace: moving from state init -> state preparing
>Thread-15913789::DEBUG::2013-11-20 03:13:33,681::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`184e79a9-b4e1-4934-824e-34d50c0f9c5b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913789::DEBUG::2013-11-20 03:13:33,682::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913789::DEBUG::2013-11-20 03:13:33,683::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913789::DEBUG::2013-11-20 03:13:33,683::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`184e79a9-b4e1-4934-824e-34d50c0f9c5b`::Granted request
>Thread-15913789::DEBUG::2013-11-20 03:13:33,684::task::495::TaskManager.Task::(_debug) Task 1285469f-99a9-4441-9e8b-8b4764cf1ace: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913789::DEBUG::2013-11-20 03:13:33,684::task::495::TaskManager.Task::(_debug) Task 1285469f-99a9-4441-9e8b-8b4764cf1ace: ref 1 aborting False
>Thread-15913789::WARNING::2013-11-20 03:13:33,686::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913789::DEBUG::2013-11-20 03:13:33,687::task::495::TaskManager.Task::(_debug) Task 1285469f-99a9-4441-9e8b-8b4764cf1ace: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913789::DEBUG::2013-11-20 03:13:33,687::task::495::TaskManager.Task::(_debug) Task 1285469f-99a9-4441-9e8b-8b4764cf1ace: moving from state preparing -> state finished
>Thread-15913789::DEBUG::2013-11-20 03:13:33,688::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913789::DEBUG::2013-11-20 03:13:33,688::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913789::DEBUG::2013-11-20 03:13:33,689::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913789::DEBUG::2013-11-20 03:13:33,689::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913789::DEBUG::2013-11-20 03:13:33,689::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913789::DEBUG::2013-11-20 03:13:33,690::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913789::DEBUG::2013-11-20 03:13:33,690::task::495::TaskManager.Task::(_debug) Task 1285469f-99a9-4441-9e8b-8b4764cf1ace: ref 0 aborting False
>Thread-15913789::INFO::2013-11-20 03:13:33,691::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:13:34,777::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0570995 s, 17.9 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 03:13:35,663::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:13:35,664::task::495::TaskManager.Task::(_debug) Task eddcf48c-281b-484e-934f-32e235ba8adf: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:13:35,665::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a4c77abb-60fe-4a6a-89b7-78eeff010b18`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:13:35,666::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:13:35,667::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:13:35,667::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a4c77abb-60fe-4a6a-89b7-78eeff010b18`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:13:35,668::task::495::TaskManager.Task::(_debug) Task eddcf48c-281b-484e-934f-32e235ba8adf: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:13:35,669::task::495::TaskManager.Task::(_debug) Task eddcf48c-281b-484e-934f-32e235ba8adf: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:13:35,669::task::495::TaskManager.Task::(_debug) Task eddcf48c-281b-484e-934f-32e235ba8adf: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:13:35,670::task::495::TaskManager.Task::(_debug) Task eddcf48c-281b-484e-934f-32e235ba8adf: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:13:35,670::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:13:35,670::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:13:35,671::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:13:35,671::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:13:35,672::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:13:35,672::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:13:35,673::task::495::TaskManager.Task::(_debug) Task eddcf48c-281b-484e-934f-32e235ba8adf: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:13:35,673::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:13:36,790::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:13:37,492::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0485095 s, 21.1 MB/s\n'; <rc> = 0
>Thread-15913791::INFO::2013-11-20 03:13:38,385::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913791::DEBUG::2013-11-20 03:13:38,386::task::495::TaskManager.Task::(_debug) Task 3ab41eaa-f699-4b3b-b56f-8eaa7fe1f5e0: moving from state init -> state preparing
>Thread-15913791::DEBUG::2013-11-20 03:13:38,386::task::495::TaskManager.Task::(_debug) Task 3ab41eaa-f699-4b3b-b56f-8eaa7fe1f5e0: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000235080718994', 'lastCheck': 1384917210.5646851, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0111620426178', 'lastCheck': 1384917217.6777799, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00116491317749', 'lastCheck': 1384917209.770822, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000340938568115', 'lastCheck': 1384917209.7632101, 'valid': True, 'code': 0}}
>Thread-15913791::DEBUG::2013-11-20 03:13:38,387::task::495::TaskManager.Task::(_debug) Task 3ab41eaa-f699-4b3b-b56f-8eaa7fe1f5e0: moving from state preparing -> state finished
>Thread-15913791::DEBUG::2013-11-20 03:13:38,388::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913791::DEBUG::2013-11-20 03:13:38,388::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913791::DEBUG::2013-11-20 03:13:38,389::task::495::TaskManager.Task::(_debug) Task 3ab41eaa-f699-4b3b-b56f-8eaa7fe1f5e0: ref 0 aborting False
>Thread-15913791::INFO::2013-11-20 03:13:38,389::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000235080718994', 'lastCheck': 1384917210.5646851, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0111620426178', 'lastCheck': 1384917217.6777799, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00116491317749', 'lastCheck': 1384917209.770822, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000340938568115', 'lastCheck': 1384917209.7632101, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:13:39,502::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:13:40,465::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0474033 s, 21.6 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 03:13:41,771::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:13:41,771::task::495::TaskManager.Task::(_debug) Task 155be91b-80c7-4cb6-b7f8-6172c712872e: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:13:41,773::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`49375163-b214-4183-953c-fc8cbfac1924`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:13:41,773::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:13:41,774::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:13:41,774::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`49375163-b214-4183-953c-fc8cbfac1924`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:13:41,775::task::495::TaskManager.Task::(_debug) Task 155be91b-80c7-4cb6-b7f8-6172c712872e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:13:41,775::task::495::TaskManager.Task::(_debug) Task 155be91b-80c7-4cb6-b7f8-6172c712872e: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:13:41,776::task::495::TaskManager.Task::(_debug) Task 155be91b-80c7-4cb6-b7f8-6172c712872e: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:13:41,776::task::495::TaskManager.Task::(_debug) Task 155be91b-80c7-4cb6-b7f8-6172c712872e: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:13:41,776::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:13:41,777::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:13:41,777::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:13:41,777::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:13:41,778::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:13:41,778::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:13:41,778::task::495::TaskManager.Task::(_debug) Task 155be91b-80c7-4cb6-b7f8-6172c712872e: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:13:41,779::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:13:42,475::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:13:43,031::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0562993 s, 18.2 MB/s\n'; <rc> = 0
>Thread-15913794::DEBUG::2013-11-20 03:13:44,535::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913794::INFO::2013-11-20 03:13:44,536::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913794::DEBUG::2013-11-20 03:13:44,537::task::495::TaskManager.Task::(_debug) Task d8bc8eb0-0e09-4be6-b0b3-8064101134e3: moving from state init -> state preparing
>Thread-15913794::DEBUG::2013-11-20 03:13:44,537::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913794::DEBUG::2013-11-20 03:13:44,538::task::495::TaskManager.Task::(_debug) Task d8bc8eb0-0e09-4be6-b0b3-8064101134e3: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913794::DEBUG::2013-11-20 03:13:44,538::task::495::TaskManager.Task::(_debug) Task d8bc8eb0-0e09-4be6-b0b3-8064101134e3: moving from state preparing -> state finished
>Thread-15913794::DEBUG::2013-11-20 03:13:44,538::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913794::DEBUG::2013-11-20 03:13:44,539::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913794::DEBUG::2013-11-20 03:13:44,539::task::495::TaskManager.Task::(_debug) Task d8bc8eb0-0e09-4be6-b0b3-8064101134e3: ref 0 aborting False
>Thread-15913794::INFO::2013-11-20 03:13:44,540::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913795::DEBUG::2013-11-20 03:13:44,543::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913795::INFO::2013-11-20 03:13:44,544::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913795::DEBUG::2013-11-20 03:13:44,545::task::495::TaskManager.Task::(_debug) Task 024aade7-19cb-47dc-aba1-0311dabeedc4: moving from state init -> state preparing
>Thread-15913795::DEBUG::2013-11-20 03:13:44,546::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`413a1151-ff8a-4300-a669-68038e23bcb8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913795::DEBUG::2013-11-20 03:13:44,546::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913795::DEBUG::2013-11-20 03:13:44,547::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913795::DEBUG::2013-11-20 03:13:44,547::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`413a1151-ff8a-4300-a669-68038e23bcb8`::Granted request
>Thread-15913795::DEBUG::2013-11-20 03:13:44,548::task::495::TaskManager.Task::(_debug) Task 024aade7-19cb-47dc-aba1-0311dabeedc4: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913795::DEBUG::2013-11-20 03:13:44,549::task::495::TaskManager.Task::(_debug) Task 024aade7-19cb-47dc-aba1-0311dabeedc4: ref 1 aborting False
>Thread-15913795::WARNING::2013-11-20 03:13:44,550::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913795::DEBUG::2013-11-20 03:13:44,551::task::495::TaskManager.Task::(_debug) Task 024aade7-19cb-47dc-aba1-0311dabeedc4: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913795::DEBUG::2013-11-20 03:13:44,552::task::495::TaskManager.Task::(_debug) Task 024aade7-19cb-47dc-aba1-0311dabeedc4: moving from state preparing -> state finished
>Thread-15913795::DEBUG::2013-11-20 03:13:44,552::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913795::DEBUG::2013-11-20 03:13:44,553::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913795::DEBUG::2013-11-20 03:13:44,553::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913795::DEBUG::2013-11-20 03:13:44,554::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913795::DEBUG::2013-11-20 03:13:44,554::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913795::DEBUG::2013-11-20 03:13:44,555::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913795::DEBUG::2013-11-20 03:13:44,555::task::495::TaskManager.Task::(_debug) Task 024aade7-19cb-47dc-aba1-0311dabeedc4: ref 0 aborting False
>Thread-15913795::INFO::2013-11-20 03:13:44,556::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:13:45,041::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:13:45,955::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0486552 s, 21.0 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 03:13:47,881::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:13:47,881::task::495::TaskManager.Task::(_debug) Task 8c77bea9-a52c-44de-a067-de1e58bf0e6d: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:13:47,888::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7122595f-94fe-45ef-8f6b-7a520c53ae32`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:13:47,888::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:13:47,888::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:13:47,889::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7122595f-94fe-45ef-8f6b-7a520c53ae32`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:13:47,889::task::495::TaskManager.Task::(_debug) Task 8c77bea9-a52c-44de-a067-de1e58bf0e6d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:13:47,890::task::495::TaskManager.Task::(_debug) Task 8c77bea9-a52c-44de-a067-de1e58bf0e6d: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:13:47,890::task::495::TaskManager.Task::(_debug) Task 8c77bea9-a52c-44de-a067-de1e58bf0e6d: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:13:47,891::task::495::TaskManager.Task::(_debug) Task 8c77bea9-a52c-44de-a067-de1e58bf0e6d: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:13:47,891::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:13:47,891::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:13:47,892::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:13:47,892::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:13:47,893::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:13:47,893::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:13:47,893::task::495::TaskManager.Task::(_debug) Task 8c77bea9-a52c-44de-a067-de1e58bf0e6d: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:13:47,894::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:13:47,894::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:13:47,895::task::495::TaskManager.Task::(_debug) Task e61e5a8e-6917-4903-a820-4b67fc9d024d: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:13:47,895::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4979ac63-cc04-4905-b4fa-74f15499566f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:13:47,896::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:13:47,896::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:13:47,896::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4979ac63-cc04-4905-b4fa-74f15499566f`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:13:47,897::task::495::TaskManager.Task::(_debug) Task e61e5a8e-6917-4903-a820-4b67fc9d024d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:13:47,897::task::495::TaskManager.Task::(_debug) Task e61e5a8e-6917-4903-a820-4b67fc9d024d: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:13:47,898::task::495::TaskManager.Task::(_debug) Task e61e5a8e-6917-4903-a820-4b67fc9d024d: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:13:47,898::task::495::TaskManager.Task::(_debug) Task e61e5a8e-6917-4903-a820-4b67fc9d024d: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:13:47,899::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:13:47,899::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:13:47,900::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:13:47,901::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:13:47,901::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:13:47,902::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:13:47,902::task::495::TaskManager.Task::(_debug) Task e61e5a8e-6917-4903-a820-4b67fc9d024d: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:13:47,903::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:13:47,965::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:13:48,377::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0531688 s, 19.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:13:50,387::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:13:50,824::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0530377 s, 19.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:13:52,902::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:13:53,370::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0715108 s, 14.3 MB/s\n'; <rc> = 0
>Thread-15913799::INFO::2013-11-20 03:13:54,465::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913799::DEBUG::2013-11-20 03:13:54,466::task::495::TaskManager.Task::(_debug) Task a465fb92-26f1-4bad-890f-62d1cd053ee5: moving from state init -> state preparing
>Thread-15913799::DEBUG::2013-11-20 03:13:54,467::task::495::TaskManager.Task::(_debug) Task a465fb92-26f1-4bad-890f-62d1cd053ee5: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000393867492676', 'lastCheck': 1384917230.6884651, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00132703781128', 'lastCheck': 1384917227.6896369, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00111508369446', 'lastCheck': 1384917229.8180089, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000381946563721', 'lastCheck': 1384917229.780268, 'valid': True, 'code': 0}}
>Thread-15913799::DEBUG::2013-11-20 03:13:54,467::task::495::TaskManager.Task::(_debug) Task a465fb92-26f1-4bad-890f-62d1cd053ee5: moving from state preparing -> state finished
>Thread-15913799::DEBUG::2013-11-20 03:13:54,468::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913799::DEBUG::2013-11-20 03:13:54,468::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913799::DEBUG::2013-11-20 03:13:54,469::task::495::TaskManager.Task::(_debug) Task a465fb92-26f1-4bad-890f-62d1cd053ee5: ref 0 aborting False
>Thread-15913799::INFO::2013-11-20 03:13:54,469::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000393867492676', 'lastCheck': 1384917230.6884651, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00132703781128', 'lastCheck': 1384917227.6896369, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00111508369446', 'lastCheck': 1384917229.8180089, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000381946563721', 'lastCheck': 1384917229.780268, 'valid': True, 'code': 0}}
>Thread-15913801::DEBUG::2013-11-20 03:13:54,808::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913801::INFO::2013-11-20 03:13:54,809::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913801::DEBUG::2013-11-20 03:13:54,809::task::495::TaskManager.Task::(_debug) Task bab0768d-a485-4945-aa5d-363368fb102b: moving from state init -> state preparing
>Thread-15913801::DEBUG::2013-11-20 03:13:54,810::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913801::DEBUG::2013-11-20 03:13:54,810::task::495::TaskManager.Task::(_debug) Task bab0768d-a485-4945-aa5d-363368fb102b: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913801::DEBUG::2013-11-20 03:13:54,811::task::495::TaskManager.Task::(_debug) Task bab0768d-a485-4945-aa5d-363368fb102b: moving from state preparing -> state finished
>Thread-15913801::DEBUG::2013-11-20 03:13:54,811::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913801::DEBUG::2013-11-20 03:13:54,812::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913801::DEBUG::2013-11-20 03:13:54,812::task::495::TaskManager.Task::(_debug) Task bab0768d-a485-4945-aa5d-363368fb102b: ref 0 aborting False
>Thread-15913801::INFO::2013-11-20 03:13:54,812::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913802::DEBUG::2013-11-20 03:13:54,816::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913802::INFO::2013-11-20 03:13:54,816::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913802::DEBUG::2013-11-20 03:13:54,816::task::495::TaskManager.Task::(_debug) Task 72964ec5-dc52-4a7e-93cc-0741e4746979: moving from state init -> state preparing
>Thread-15913802::DEBUG::2013-11-20 03:13:54,817::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`654af5d4-e971-4744-825d-a2ed6585c2f5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913802::DEBUG::2013-11-20 03:13:54,818::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913802::DEBUG::2013-11-20 03:13:54,818::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913802::DEBUG::2013-11-20 03:13:54,818::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`654af5d4-e971-4744-825d-a2ed6585c2f5`::Granted request
>Thread-15913802::DEBUG::2013-11-20 03:13:54,819::task::495::TaskManager.Task::(_debug) Task 72964ec5-dc52-4a7e-93cc-0741e4746979: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913802::DEBUG::2013-11-20 03:13:54,819::task::495::TaskManager.Task::(_debug) Task 72964ec5-dc52-4a7e-93cc-0741e4746979: ref 1 aborting False
>Thread-15913802::WARNING::2013-11-20 03:13:54,820::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913802::DEBUG::2013-11-20 03:13:54,821::task::495::TaskManager.Task::(_debug) Task 72964ec5-dc52-4a7e-93cc-0741e4746979: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913802::DEBUG::2013-11-20 03:13:54,822::task::495::TaskManager.Task::(_debug) Task 72964ec5-dc52-4a7e-93cc-0741e4746979: moving from state preparing -> state finished
>Thread-15913802::DEBUG::2013-11-20 03:13:54,822::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913802::DEBUG::2013-11-20 03:13:54,822::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913802::DEBUG::2013-11-20 03:13:54,823::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913802::DEBUG::2013-11-20 03:13:54,823::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913802::DEBUG::2013-11-20 03:13:54,823::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913802::DEBUG::2013-11-20 03:13:54,824::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913802::DEBUG::2013-11-20 03:13:54,824::task::495::TaskManager.Task::(_debug) Task 72964ec5-dc52-4a7e-93cc-0741e4746979: ref 0 aborting False
>Thread-15913802::INFO::2013-11-20 03:13:54,825::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:13:55,381::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:13:55,816::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0556619 s, 18.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:13:57,829::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:13:58,462::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0522219 s, 19.6 MB/s\n'; <rc> = 0
>Thread-15913696::DEBUG::2013-11-20 03:13:59,908::guestIF::97::vm.Vm::(_handleMessage) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::u'None'
>Dummy-15912651::DEBUG::2013-11-20 03:14:00,473::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:14:00,935::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.055671 s, 18.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:14:02,946::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:14:04,828::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0516887 s, 19.8 MB/s\n'; <rc> = 0
>Thread-15913806::DEBUG::2013-11-20 03:14:05,133::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913806::INFO::2013-11-20 03:14:05,134::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913806::DEBUG::2013-11-20 03:14:05,134::task::495::TaskManager.Task::(_debug) Task d93ea4e1-9b52-4648-9778-c42aa47314bc: moving from state init -> state preparing
>Thread-15913806::DEBUG::2013-11-20 03:14:05,135::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913806::DEBUG::2013-11-20 03:14:05,136::task::495::TaskManager.Task::(_debug) Task d93ea4e1-9b52-4648-9778-c42aa47314bc: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913806::DEBUG::2013-11-20 03:14:05,136::task::495::TaskManager.Task::(_debug) Task d93ea4e1-9b52-4648-9778-c42aa47314bc: moving from state preparing -> state finished
>Thread-15913806::DEBUG::2013-11-20 03:14:05,136::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913806::DEBUG::2013-11-20 03:14:05,137::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913806::DEBUG::2013-11-20 03:14:05,137::task::495::TaskManager.Task::(_debug) Task d93ea4e1-9b52-4648-9778-c42aa47314bc: ref 0 aborting False
>Thread-15913806::INFO::2013-11-20 03:14:05,137::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913807::DEBUG::2013-11-20 03:14:05,141::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913807::INFO::2013-11-20 03:14:05,142::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913807::DEBUG::2013-11-20 03:14:05,142::task::495::TaskManager.Task::(_debug) Task f50dbecd-e8d7-423f-a5e5-00a716422d8e: moving from state init -> state preparing
>Thread-15913807::DEBUG::2013-11-20 03:14:05,143::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ba661cfe-948e-453e-b814-e30202c5614f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913807::DEBUG::2013-11-20 03:14:05,143::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913807::DEBUG::2013-11-20 03:14:05,144::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913807::DEBUG::2013-11-20 03:14:05,144::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ba661cfe-948e-453e-b814-e30202c5614f`::Granted request
>Thread-15913807::DEBUG::2013-11-20 03:14:05,145::task::495::TaskManager.Task::(_debug) Task f50dbecd-e8d7-423f-a5e5-00a716422d8e: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913807::DEBUG::2013-11-20 03:14:05,145::task::495::TaskManager.Task::(_debug) Task f50dbecd-e8d7-423f-a5e5-00a716422d8e: ref 1 aborting False
>Thread-15913807::WARNING::2013-11-20 03:14:05,146::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913807::DEBUG::2013-11-20 03:14:05,147::task::495::TaskManager.Task::(_debug) Task f50dbecd-e8d7-423f-a5e5-00a716422d8e: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913807::DEBUG::2013-11-20 03:14:05,148::task::495::TaskManager.Task::(_debug) Task f50dbecd-e8d7-423f-a5e5-00a716422d8e: moving from state preparing -> state finished
>Thread-15913807::DEBUG::2013-11-20 03:14:05,148::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913807::DEBUG::2013-11-20 03:14:05,148::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913807::DEBUG::2013-11-20 03:14:05,149::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913807::DEBUG::2013-11-20 03:14:05,149::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913807::DEBUG::2013-11-20 03:14:05,150::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913807::DEBUG::2013-11-20 03:14:05,150::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913807::DEBUG::2013-11-20 03:14:05,151::task::495::TaskManager.Task::(_debug) Task f50dbecd-e8d7-423f-a5e5-00a716422d8e: ref 0 aborting False
>Thread-15913807::INFO::2013-11-20 03:14:05,151::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913546::INFO::2013-11-20 03:14:05,691::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:14:05,691::task::495::TaskManager.Task::(_debug) Task 4b3580b1-43aa-4f75-8536-14b61a3afa0f: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:14:05,692::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b085fd9e-dabb-4866-aa89-f5f6f74cb881`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:14:05,693::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:14:05,693::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:14:05,694::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b085fd9e-dabb-4866-aa89-f5f6f74cb881`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:14:05,695::task::495::TaskManager.Task::(_debug) Task 4b3580b1-43aa-4f75-8536-14b61a3afa0f: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:14:05,695::task::495::TaskManager.Task::(_debug) Task 4b3580b1-43aa-4f75-8536-14b61a3afa0f: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:14:05,696::task::495::TaskManager.Task::(_debug) Task 4b3580b1-43aa-4f75-8536-14b61a3afa0f: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:14:05,696::task::495::TaskManager.Task::(_debug) Task 4b3580b1-43aa-4f75-8536-14b61a3afa0f: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:14:05,697::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:14:05,697::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:14:05,698::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:14:05,699::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:14:05,699::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:14:05,700::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:14:05,700::task::495::TaskManager.Task::(_debug) Task 4b3580b1-43aa-4f75-8536-14b61a3afa0f: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:14:05,701::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:14:05,701::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:14:05,702::task::495::TaskManager.Task::(_debug) Task 1597042c-363b-4af6-9f51-cf53e82da1e3: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:14:05,703::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fb1bedfc-3893-4c29-b17d-b80650dab9aa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:14:05,703::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:14:05,704::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:14:05,704::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fb1bedfc-3893-4c29-b17d-b80650dab9aa`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:14:05,706::task::495::TaskManager.Task::(_debug) Task 1597042c-363b-4af6-9f51-cf53e82da1e3: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:14:05,706::task::495::TaskManager.Task::(_debug) Task 1597042c-363b-4af6-9f51-cf53e82da1e3: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:14:05,706::task::495::TaskManager.Task::(_debug) Task 1597042c-363b-4af6-9f51-cf53e82da1e3: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:14:05,707::task::495::TaskManager.Task::(_debug) Task 1597042c-363b-4af6-9f51-cf53e82da1e3: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:14:05,707::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:14:05,708::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:14:05,708::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:14:05,708::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:14:05,709::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:14:05,709::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:14:05,709::task::495::TaskManager.Task::(_debug) Task 1597042c-363b-4af6-9f51-cf53e82da1e3: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:14:05,710::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:14:05,710::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:14:05,710::task::495::TaskManager.Task::(_debug) Task 8989dcf1-06d3-40fb-8b74-8442db72b253: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:14:05,711::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b51a726c-7544-46f7-b36b-b592b9cc7c13`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:14:05,711::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:14:05,712::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:14:05,712::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b51a726c-7544-46f7-b36b-b592b9cc7c13`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:14:05,713::task::495::TaskManager.Task::(_debug) Task 8989dcf1-06d3-40fb-8b74-8442db72b253: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:14:05,713::task::495::TaskManager.Task::(_debug) Task 8989dcf1-06d3-40fb-8b74-8442db72b253: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:14:05,713::task::495::TaskManager.Task::(_debug) Task 8989dcf1-06d3-40fb-8b74-8442db72b253: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:14:05,714::task::495::TaskManager.Task::(_debug) Task 8989dcf1-06d3-40fb-8b74-8442db72b253: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:14:05,714::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:14:05,714::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:14:05,715::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:14:05,715::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:14:05,716::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:14:05,716::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:14:05,717::task::495::TaskManager.Task::(_debug) Task 8989dcf1-06d3-40fb-8b74-8442db72b253: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:14:05,717::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:14:05,718::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:14:05,718::task::495::TaskManager.Task::(_debug) Task 0846367e-31ed-4c9c-bef2-5e7c4f8333fe: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:14:05,719::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`ee622640-d9e4-488f-9df0-0f48e9a0e0e4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:14:05,719::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:14:05,719::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:14:05,720::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`ee622640-d9e4-488f-9df0-0f48e9a0e0e4`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:14:05,721::task::495::TaskManager.Task::(_debug) Task 0846367e-31ed-4c9c-bef2-5e7c4f8333fe: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:14:05,721::task::495::TaskManager.Task::(_debug) Task 0846367e-31ed-4c9c-bef2-5e7c4f8333fe: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:14:05,721::task::495::TaskManager.Task::(_debug) Task 0846367e-31ed-4c9c-bef2-5e7c4f8333fe: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:14:05,722::task::495::TaskManager.Task::(_debug) Task 0846367e-31ed-4c9c-bef2-5e7c4f8333fe: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:14:05,722::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:14:05,722::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:14:05,723::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:14:05,723::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:14:05,724::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:14:05,724::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:14:05,724::task::495::TaskManager.Task::(_debug) Task 0846367e-31ed-4c9c-bef2-5e7c4f8333fe: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:14:05,725::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:14:06,838::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:14:07,232::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0490772 s, 20.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:14:09,240::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-42840::DEBUG::2013-11-20 03:14:09,800::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:14:09,998::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.219772 s, 4.7 MB/s\n'; <rc> = 0
>Thread-15913809::INFO::2013-11-20 03:14:10,086::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913809::DEBUG::2013-11-20 03:14:10,087::task::495::TaskManager.Task::(_debug) Task e3264ad1-e5be-42a5-a109-b74918579b81: moving from state init -> state preparing
>Thread-15913809::DEBUG::2013-11-20 03:14:10,087::task::495::TaskManager.Task::(_debug) Task e3264ad1-e5be-42a5-a109-b74918579b81: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000308990478516', 'lastCheck': 1384917240.706538, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0117788314819', 'lastCheck': 1384917247.739619, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00097393989563', 'lastCheck': 1384917250.0140569, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000396013259888', 'lastCheck': 1384917239.790266, 'valid': True, 'code': 0}}
>Thread-15913809::DEBUG::2013-11-20 03:14:10,088::task::495::TaskManager.Task::(_debug) Task e3264ad1-e5be-42a5-a109-b74918579b81: moving from state preparing -> state finished
>Thread-15913809::DEBUG::2013-11-20 03:14:10,089::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913809::DEBUG::2013-11-20 03:14:10,089::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913809::DEBUG::2013-11-20 03:14:10,089::task::495::TaskManager.Task::(_debug) Task e3264ad1-e5be-42a5-a109-b74918579b81: ref 0 aborting False
>Thread-15913809::INFO::2013-11-20 03:14:10,090::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000308990478516', 'lastCheck': 1384917240.706538, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0117788314819', 'lastCheck': 1384917247.739619, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00097393989563', 'lastCheck': 1384917250.0140569, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000396013259888', 'lastCheck': 1384917239.790266, 'valid': True, 'code': 0}}
>Thread-42840::DEBUG::2013-11-20 03:14:10,504::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:14:12,010::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:14:12,426::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0501577 s, 20.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:14:14,435::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:14:14,818::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0486028 s, 21.1 MB/s\n'; <rc> = 0
>Thread-15913812::DEBUG::2013-11-20 03:14:15,355::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913812::INFO::2013-11-20 03:14:15,356::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913812::DEBUG::2013-11-20 03:14:15,356::task::495::TaskManager.Task::(_debug) Task cc417ad1-50a6-4d7c-af33-d446b47efec6: moving from state init -> state preparing
>Thread-15913812::DEBUG::2013-11-20 03:14:15,357::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913812::DEBUG::2013-11-20 03:14:15,357::task::495::TaskManager.Task::(_debug) Task cc417ad1-50a6-4d7c-af33-d446b47efec6: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913812::DEBUG::2013-11-20 03:14:15,358::task::495::TaskManager.Task::(_debug) Task cc417ad1-50a6-4d7c-af33-d446b47efec6: moving from state preparing -> state finished
>Thread-15913812::DEBUG::2013-11-20 03:14:15,358::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913812::DEBUG::2013-11-20 03:14:15,358::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913812::DEBUG::2013-11-20 03:14:15,359::task::495::TaskManager.Task::(_debug) Task cc417ad1-50a6-4d7c-af33-d446b47efec6: ref 0 aborting False
>Thread-15913812::INFO::2013-11-20 03:14:15,359::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913813::DEBUG::2013-11-20 03:14:15,362::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913813::INFO::2013-11-20 03:14:15,363::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913813::DEBUG::2013-11-20 03:14:15,363::task::495::TaskManager.Task::(_debug) Task 6de57773-62b0-4ac8-9fc5-09dcffe81dbd: moving from state init -> state preparing
>Thread-15913813::DEBUG::2013-11-20 03:14:15,364::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0863bcbc-5291-4b87-9fdf-00e55184f014`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913813::DEBUG::2013-11-20 03:14:15,364::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913813::DEBUG::2013-11-20 03:14:15,364::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913813::DEBUG::2013-11-20 03:14:15,365::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0863bcbc-5291-4b87-9fdf-00e55184f014`::Granted request
>Thread-15913813::DEBUG::2013-11-20 03:14:15,365::task::495::TaskManager.Task::(_debug) Task 6de57773-62b0-4ac8-9fc5-09dcffe81dbd: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913813::DEBUG::2013-11-20 03:14:15,366::task::495::TaskManager.Task::(_debug) Task 6de57773-62b0-4ac8-9fc5-09dcffe81dbd: ref 1 aborting False
>Thread-15913813::WARNING::2013-11-20 03:14:15,367::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913813::DEBUG::2013-11-20 03:14:15,368::task::495::TaskManager.Task::(_debug) Task 6de57773-62b0-4ac8-9fc5-09dcffe81dbd: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913813::DEBUG::2013-11-20 03:14:15,368::task::495::TaskManager.Task::(_debug) Task 6de57773-62b0-4ac8-9fc5-09dcffe81dbd: moving from state preparing -> state finished
>Thread-15913813::DEBUG::2013-11-20 03:14:15,368::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913813::DEBUG::2013-11-20 03:14:15,369::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913813::DEBUG::2013-11-20 03:14:15,369::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913813::DEBUG::2013-11-20 03:14:15,370::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913813::DEBUG::2013-11-20 03:14:15,370::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913813::DEBUG::2013-11-20 03:14:15,370::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913813::DEBUG::2013-11-20 03:14:15,370::task::495::TaskManager.Task::(_debug) Task 6de57773-62b0-4ac8-9fc5-09dcffe81dbd: ref 0 aborting False
>Thread-15913813::INFO::2013-11-20 03:14:15,371::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913639::INFO::2013-11-20 03:14:15,705::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:14:15,705::task::495::TaskManager.Task::(_debug) Task fcccf6bc-2717-41ba-aec6-37f81e7d8334: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:14:15,706::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9e1cbe14-5883-42e2-8cc7-549356370fe0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:14:15,707::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:14:15,707::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:14:15,708::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9e1cbe14-5883-42e2-8cc7-549356370fe0`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:14:15,708::task::495::TaskManager.Task::(_debug) Task fcccf6bc-2717-41ba-aec6-37f81e7d8334: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:14:15,709::task::495::TaskManager.Task::(_debug) Task fcccf6bc-2717-41ba-aec6-37f81e7d8334: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:14:15,709::task::495::TaskManager.Task::(_debug) Task fcccf6bc-2717-41ba-aec6-37f81e7d8334: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:14:15,710::task::495::TaskManager.Task::(_debug) Task fcccf6bc-2717-41ba-aec6-37f81e7d8334: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:14:15,710::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:14:15,711::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:14:15,711::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:14:15,712::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:14:15,712::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:14:15,713::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:14:15,713::task::495::TaskManager.Task::(_debug) Task fcccf6bc-2717-41ba-aec6-37f81e7d8334: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:14:15,713::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:14:16,828::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:14:17,241::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0545015 s, 18.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:14:19,251::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:14:20,123::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0663359 s, 15.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:14:22,135::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:14:22,541::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0461147 s, 22.2 MB/s\n'; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-20 03:14:23,427::vm::525::vm.Vm::(_rtcUpdate) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::new rtc offset 2
>libvirtEventLoop::DEBUG::2013-11-20 03:14:23,428::vm::525::vm.Vm::(_rtcUpdate) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::new rtc offset 2
>libvirtEventLoop::DEBUG::2013-11-20 03:14:23,428::vm::525::vm.Vm::(_rtcUpdate) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::new rtc offset 3602
>libvirtEventLoop::DEBUG::2013-11-20 03:14:23,429::vm::525::vm.Vm::(_rtcUpdate) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::new rtc offset 3602
>libvirtEventLoop::DEBUG::2013-11-20 03:14:23,430::vm::525::vm.Vm::(_rtcUpdate) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::new rtc offset 3602
>libvirtEventLoop::DEBUG::2013-11-20 03:14:23,430::vm::525::vm.Vm::(_rtcUpdate) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::new rtc offset 3602
>libvirtEventLoop::DEBUG::2013-11-20 03:14:23,431::vm::525::vm.Vm::(_rtcUpdate) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::new rtc offset 3602
>libvirtEventLoop::DEBUG::2013-11-20 03:14:23,432::vm::525::vm.Vm::(_rtcUpdate) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::new rtc offset 3602
>libvirtEventLoop::DEBUG::2013-11-20 03:14:23,432::vm::525::vm.Vm::(_rtcUpdate) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::new rtc offset 3602
>libvirtEventLoop::DEBUG::2013-11-20 03:14:23,433::vm::525::vm.Vm::(_rtcUpdate) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::new rtc offset 3602
>libvirtEventLoop::DEBUG::2013-11-20 03:14:23,434::vm::525::vm.Vm::(_rtcUpdate) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::new rtc offset 3602
>libvirtEventLoop::DEBUG::2013-11-20 03:14:23,434::vm::525::vm.Vm::(_rtcUpdate) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::new rtc offset 3602
>libvirtEventLoop::DEBUG::2013-11-20 03:14:23,435::vm::525::vm.Vm::(_rtcUpdate) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::new rtc offset 3602
>libvirtEventLoop::DEBUG::2013-11-20 03:14:23,436::vm::525::vm.Vm::(_rtcUpdate) vmId=`11b5c641-e4f7-487e-9758-c42cb014e2d2`::new rtc offset 3602
>Dummy-15912651::DEBUG::2013-11-20 03:14:24,551::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:14:24,979::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0453107 s, 22.6 MB/s\n'; <rc> = 0
>Thread-15913817::INFO::2013-11-20 03:14:25,618::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913817::DEBUG::2013-11-20 03:14:25,621::task::495::TaskManager.Task::(_debug) Task 7d8c0a12-e4bf-47cd-b087-93c5ef5846ab: moving from state init -> state preparing
>Thread-15913817::DEBUG::2013-11-20 03:14:25,624::task::495::TaskManager.Task::(_debug) Task 7d8c0a12-e4bf-47cd-b087-93c5ef5846ab: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000268936157227', 'lastCheck': 1384917260.732682, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0204908847809', 'lastCheck': 1384917257.771805, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0150320529938', 'lastCheck': 1384917260.1991401, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000453948974609', 'lastCheck': 1384917260.599427, 'valid': True, 'code': 0}}
>Thread-15913817::DEBUG::2013-11-20 03:14:25,627::task::495::TaskManager.Task::(_debug) Task 7d8c0a12-e4bf-47cd-b087-93c5ef5846ab: moving from state preparing -> state finished
>Thread-15913817::DEBUG::2013-11-20 03:14:25,628::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913817::DEBUG::2013-11-20 03:14:25,629::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913817::DEBUG::2013-11-20 03:14:25,629::task::495::TaskManager.Task::(_debug) Task 7d8c0a12-e4bf-47cd-b087-93c5ef5846ab: ref 0 aborting False
>Thread-15913817::INFO::2013-11-20 03:14:25,630::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000268936157227', 'lastCheck': 1384917260.732682, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0204908847809', 'lastCheck': 1384917257.771805, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0150320529938', 'lastCheck': 1384917260.1991401, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000453948974609', 'lastCheck': 1384917260.599427, 'valid': True, 'code': 0}}
>Thread-15913819::DEBUG::2013-11-20 03:14:25,664::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913819::INFO::2013-11-20 03:14:25,665::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913819::DEBUG::2013-11-20 03:14:25,666::task::495::TaskManager.Task::(_debug) Task 57490b90-2a6a-4cfb-a693-105611c54b71: moving from state init -> state preparing
>Thread-15913819::DEBUG::2013-11-20 03:14:25,666::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913819::DEBUG::2013-11-20 03:14:25,666::task::495::TaskManager.Task::(_debug) Task 57490b90-2a6a-4cfb-a693-105611c54b71: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913819::DEBUG::2013-11-20 03:14:25,667::task::495::TaskManager.Task::(_debug) Task 57490b90-2a6a-4cfb-a693-105611c54b71: moving from state preparing -> state finished
>Thread-15913819::DEBUG::2013-11-20 03:14:25,667::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913819::DEBUG::2013-11-20 03:14:25,672::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913819::DEBUG::2013-11-20 03:14:25,673::task::495::TaskManager.Task::(_debug) Task 57490b90-2a6a-4cfb-a693-105611c54b71: ref 0 aborting False
>Thread-15913819::INFO::2013-11-20 03:14:25,673::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913820::DEBUG::2013-11-20 03:14:25,680::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913820::INFO::2013-11-20 03:14:25,680::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913820::DEBUG::2013-11-20 03:14:25,686::task::495::TaskManager.Task::(_debug) Task eaf89485-21d7-41d8-b398-a71bb2a36204: moving from state init -> state preparing
>Thread-15913820::DEBUG::2013-11-20 03:14:25,692::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8efec2b2-e5ed-49d2-98d4-1e9744f3f81b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913820::DEBUG::2013-11-20 03:14:25,693::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913820::DEBUG::2013-11-20 03:14:25,693::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913820::DEBUG::2013-11-20 03:14:25,694::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8efec2b2-e5ed-49d2-98d4-1e9744f3f81b`::Granted request
>Thread-15913820::DEBUG::2013-11-20 03:14:25,694::task::495::TaskManager.Task::(_debug) Task eaf89485-21d7-41d8-b398-a71bb2a36204: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913820::DEBUG::2013-11-20 03:14:25,695::task::495::TaskManager.Task::(_debug) Task eaf89485-21d7-41d8-b398-a71bb2a36204: ref 1 aborting False
>Thread-15913820::WARNING::2013-11-20 03:14:25,696::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913820::DEBUG::2013-11-20 03:14:25,705::task::495::TaskManager.Task::(_debug) Task eaf89485-21d7-41d8-b398-a71bb2a36204: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913820::DEBUG::2013-11-20 03:14:25,716::task::495::TaskManager.Task::(_debug) Task eaf89485-21d7-41d8-b398-a71bb2a36204: moving from state preparing -> state finished
>Thread-15913820::DEBUG::2013-11-20 03:14:25,717::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913820::DEBUG::2013-11-20 03:14:25,717::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913820::DEBUG::2013-11-20 03:14:25,718::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913820::DEBUG::2013-11-20 03:14:25,719::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913820::DEBUG::2013-11-20 03:14:25,719::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913820::DEBUG::2013-11-20 03:14:25,720::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913820::DEBUG::2013-11-20 03:14:25,720::task::495::TaskManager.Task::(_debug) Task eaf89485-21d7-41d8-b398-a71bb2a36204: ref 0 aborting False
>Thread-15913820::INFO::2013-11-20 03:14:25,721::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:14:26,988::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:14:27,370::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0481656 s, 21.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:14:29,380::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:14:29,771::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.048415 s, 21.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:14:31,781::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:14:32,352::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0497303 s, 20.6 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 03:14:33,570::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:14:33,571::task::495::TaskManager.Task::(_debug) Task f3a081c1-9011-44d3-8bce-6a7fe31e094b: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:14:33,572::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`52730c5d-6d62-402a-94c8-716692a34ae1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:14:33,572::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:14:33,573::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:14:33,573::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`52730c5d-6d62-402a-94c8-716692a34ae1`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:14:33,574::task::495::TaskManager.Task::(_debug) Task f3a081c1-9011-44d3-8bce-6a7fe31e094b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:14:33,574::task::495::TaskManager.Task::(_debug) Task f3a081c1-9011-44d3-8bce-6a7fe31e094b: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:14:33,575::task::495::TaskManager.Task::(_debug) Task f3a081c1-9011-44d3-8bce-6a7fe31e094b: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:14:33,575::task::495::TaskManager.Task::(_debug) Task f3a081c1-9011-44d3-8bce-6a7fe31e094b: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:14:33,576::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:14:33,576::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:14:33,576::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:14:33,577::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:14:33,577::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:14:33,577::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:14:33,578::task::495::TaskManager.Task::(_debug) Task f3a081c1-9011-44d3-8bce-6a7fe31e094b: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:14:33,578::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:14:34,363::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:14:34,961::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.050368 s, 20.3 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 03:14:35,723::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:14:35,723::task::495::TaskManager.Task::(_debug) Task 14d850f7-4923-4ddd-a3ae-b018c7640949: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:14:35,724::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`84789643-678c-4109-9400-70de1573c493`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:14:35,725::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:14:35,725::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:14:35,726::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`84789643-678c-4109-9400-70de1573c493`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:14:35,731::task::495::TaskManager.Task::(_debug) Task 14d850f7-4923-4ddd-a3ae-b018c7640949: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:14:35,732::task::495::TaskManager.Task::(_debug) Task 14d850f7-4923-4ddd-a3ae-b018c7640949: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:14:35,732::task::495::TaskManager.Task::(_debug) Task 14d850f7-4923-4ddd-a3ae-b018c7640949: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:14:35,733::task::495::TaskManager.Task::(_debug) Task 14d850f7-4923-4ddd-a3ae-b018c7640949: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:14:35,733::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:14:35,733::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:14:35,734::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:14:35,734::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:14:35,734::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:14:35,735::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:14:35,735::task::495::TaskManager.Task::(_debug) Task 14d850f7-4923-4ddd-a3ae-b018c7640949: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:14:35,735::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913824::DEBUG::2013-11-20 03:14:36,278::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913824::INFO::2013-11-20 03:14:36,285::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913824::DEBUG::2013-11-20 03:14:36,285::task::495::TaskManager.Task::(_debug) Task 3d53d7ec-79f5-44c5-8687-6610326f0202: moving from state init -> state preparing
>Thread-15913824::DEBUG::2013-11-20 03:14:36,286::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913824::DEBUG::2013-11-20 03:14:36,287::task::495::TaskManager.Task::(_debug) Task 3d53d7ec-79f5-44c5-8687-6610326f0202: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913824::DEBUG::2013-11-20 03:14:36,287::task::495::TaskManager.Task::(_debug) Task 3d53d7ec-79f5-44c5-8687-6610326f0202: moving from state preparing -> state finished
>Thread-15913824::DEBUG::2013-11-20 03:14:36,288::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913824::DEBUG::2013-11-20 03:14:36,289::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913824::DEBUG::2013-11-20 03:14:36,289::task::495::TaskManager.Task::(_debug) Task 3d53d7ec-79f5-44c5-8687-6610326f0202: ref 0 aborting False
>Thread-15913824::INFO::2013-11-20 03:14:36,290::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913825::DEBUG::2013-11-20 03:14:36,300::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913825::INFO::2013-11-20 03:14:36,301::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913825::DEBUG::2013-11-20 03:14:36,301::task::495::TaskManager.Task::(_debug) Task 7d42a2fa-70ec-4df4-8f6f-abed957a4bb4: moving from state init -> state preparing
>Thread-15913825::DEBUG::2013-11-20 03:14:36,307::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f57c9ead-b869-4961-97c3-a21c5f7a090e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913825::DEBUG::2013-11-20 03:14:36,312::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913825::DEBUG::2013-11-20 03:14:36,313::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913825::DEBUG::2013-11-20 03:14:36,313::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f57c9ead-b869-4961-97c3-a21c5f7a090e`::Granted request
>Thread-15913825::DEBUG::2013-11-20 03:14:36,314::task::495::TaskManager.Task::(_debug) Task 7d42a2fa-70ec-4df4-8f6f-abed957a4bb4: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913825::DEBUG::2013-11-20 03:14:36,314::task::495::TaskManager.Task::(_debug) Task 7d42a2fa-70ec-4df4-8f6f-abed957a4bb4: ref 1 aborting False
>Thread-15913825::WARNING::2013-11-20 03:14:36,316::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913825::DEBUG::2013-11-20 03:14:36,321::task::495::TaskManager.Task::(_debug) Task 7d42a2fa-70ec-4df4-8f6f-abed957a4bb4: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913825::DEBUG::2013-11-20 03:14:36,329::task::495::TaskManager.Task::(_debug) Task 7d42a2fa-70ec-4df4-8f6f-abed957a4bb4: moving from state preparing -> state finished
>Thread-15913825::DEBUG::2013-11-20 03:14:36,329::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913825::DEBUG::2013-11-20 03:14:36,330::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913825::DEBUG::2013-11-20 03:14:36,332::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913825::DEBUG::2013-11-20 03:14:36,333::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913825::DEBUG::2013-11-20 03:14:36,333::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913825::DEBUG::2013-11-20 03:14:36,333::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913825::DEBUG::2013-11-20 03:14:36,334::task::495::TaskManager.Task::(_debug) Task 7d42a2fa-70ec-4df4-8f6f-abed957a4bb4: ref 0 aborting False
>Thread-15913825::INFO::2013-11-20 03:14:36,335::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:14:36,971::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:14:38,325::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0507918 s, 20.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:14:40,335::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:14:40,785::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0535432 s, 19.1 MB/s\n'; <rc> = 0
>Thread-15913827::INFO::2013-11-20 03:14:41,351::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913827::DEBUG::2013-11-20 03:14:41,352::task::495::TaskManager.Task::(_debug) Task d6cbc2a8-d250-464c-aa23-e29948c27796: moving from state init -> state preparing
>Thread-15913827::DEBUG::2013-11-20 03:14:41,353::task::495::TaskManager.Task::(_debug) Task d6cbc2a8-d250-464c-aa23-e29948c27796: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000553131103516', 'lastCheck': 1384917280.7583539, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00115084648132', 'lastCheck': 1384917278.0801699, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00112104415894', 'lastCheck': 1384917280.4999521, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000334978103638', 'lastCheck': 1384917280.732224, 'valid': True, 'code': 0}}
>Thread-15913827::DEBUG::2013-11-20 03:14:41,364::task::495::TaskManager.Task::(_debug) Task d6cbc2a8-d250-464c-aa23-e29948c27796: moving from state preparing -> state finished
>Thread-15913827::DEBUG::2013-11-20 03:14:41,364::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913827::DEBUG::2013-11-20 03:14:41,365::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913827::DEBUG::2013-11-20 03:14:41,365::task::495::TaskManager.Task::(_debug) Task d6cbc2a8-d250-464c-aa23-e29948c27796: ref 0 aborting False
>Thread-15913827::INFO::2013-11-20 03:14:41,366::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000553131103516', 'lastCheck': 1384917280.7583539, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00115084648132', 'lastCheck': 1384917278.0801699, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00112104415894', 'lastCheck': 1384917280.4999521, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000334978103638', 'lastCheck': 1384917280.732224, 'valid': True, 'code': 0}}
>Thread-15912392::INFO::2013-11-20 03:14:42,163::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:14:42,163::task::495::TaskManager.Task::(_debug) Task 93492eac-b502-4840-8fbb-8c96b70a34af: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:14:42,165::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ebbcede3-bcce-4a79-a978-e555f851d035`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:14:42,165::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:14:42,166::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:14:42,166::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ebbcede3-bcce-4a79-a978-e555f851d035`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:14:42,167::task::495::TaskManager.Task::(_debug) Task 93492eac-b502-4840-8fbb-8c96b70a34af: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:14:42,167::task::495::TaskManager.Task::(_debug) Task 93492eac-b502-4840-8fbb-8c96b70a34af: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:14:42,168::task::495::TaskManager.Task::(_debug) Task 93492eac-b502-4840-8fbb-8c96b70a34af: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:14:42,168::task::495::TaskManager.Task::(_debug) Task 93492eac-b502-4840-8fbb-8c96b70a34af: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:14:42,169::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:14:42,169::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:14:42,170::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:14:42,170::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:14:42,171::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:14:42,171::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:14:42,172::task::495::TaskManager.Task::(_debug) Task 93492eac-b502-4840-8fbb-8c96b70a34af: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:14:42,172::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:14:42,796::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:14:43,219::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.065694 s, 15.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:14:45,303::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:14:45,743::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0561768 s, 18.2 MB/s\n'; <rc> = 0
>Thread-15913640::DEBUG::2013-11-20 03:14:46,172::guestIF::97::vm.Vm::(_handleMessage) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::u'None'
>Thread-15913830::DEBUG::2013-11-20 03:14:46,762::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913830::INFO::2013-11-20 03:14:46,763::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913830::DEBUG::2013-11-20 03:14:46,764::task::495::TaskManager.Task::(_debug) Task ebf145aa-169b-4515-9083-feaf54cd5130: moving from state init -> state preparing
>Thread-15913830::DEBUG::2013-11-20 03:14:46,765::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913830::DEBUG::2013-11-20 03:14:46,765::task::495::TaskManager.Task::(_debug) Task ebf145aa-169b-4515-9083-feaf54cd5130: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913830::DEBUG::2013-11-20 03:14:46,766::task::495::TaskManager.Task::(_debug) Task ebf145aa-169b-4515-9083-feaf54cd5130: moving from state preparing -> state finished
>Thread-15913830::DEBUG::2013-11-20 03:14:46,766::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913830::DEBUG::2013-11-20 03:14:46,767::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913830::DEBUG::2013-11-20 03:14:46,767::task::495::TaskManager.Task::(_debug) Task ebf145aa-169b-4515-9083-feaf54cd5130: ref 0 aborting False
>Thread-15913830::INFO::2013-11-20 03:14:46,767::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913831::DEBUG::2013-11-20 03:14:46,788::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913831::INFO::2013-11-20 03:14:46,789::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913831::DEBUG::2013-11-20 03:14:46,812::task::495::TaskManager.Task::(_debug) Task dc3fb59b-4ab9-43d8-80f9-8acf46d10bc7: moving from state init -> state preparing
>Thread-15913831::DEBUG::2013-11-20 03:14:46,837::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`94f8706b-e624-41ca-b137-a1f95dc02242`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913831::DEBUG::2013-11-20 03:14:46,845::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913831::DEBUG::2013-11-20 03:14:46,846::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913831::DEBUG::2013-11-20 03:14:46,846::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`94f8706b-e624-41ca-b137-a1f95dc02242`::Granted request
>Thread-15913831::DEBUG::2013-11-20 03:14:46,852::task::495::TaskManager.Task::(_debug) Task dc3fb59b-4ab9-43d8-80f9-8acf46d10bc7: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913831::DEBUG::2013-11-20 03:14:46,853::task::495::TaskManager.Task::(_debug) Task dc3fb59b-4ab9-43d8-80f9-8acf46d10bc7: ref 1 aborting False
>Thread-15913831::WARNING::2013-11-20 03:14:46,854::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913831::DEBUG::2013-11-20 03:14:46,862::task::495::TaskManager.Task::(_debug) Task dc3fb59b-4ab9-43d8-80f9-8acf46d10bc7: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913831::DEBUG::2013-11-20 03:14:46,869::task::495::TaskManager.Task::(_debug) Task dc3fb59b-4ab9-43d8-80f9-8acf46d10bc7: moving from state preparing -> state finished
>Thread-15913831::DEBUG::2013-11-20 03:14:46,869::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913831::DEBUG::2013-11-20 03:14:46,870::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913831::DEBUG::2013-11-20 03:14:46,870::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913831::DEBUG::2013-11-20 03:14:46,875::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913831::DEBUG::2013-11-20 03:14:46,876::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913831::DEBUG::2013-11-20 03:14:46,876::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913831::DEBUG::2013-11-20 03:14:46,876::task::495::TaskManager.Task::(_debug) Task dc3fb59b-4ab9-43d8-80f9-8acf46d10bc7: ref 0 aborting False
>Thread-15913831::INFO::2013-11-20 03:14:46,877::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:14:47,754::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913518::INFO::2013-11-20 03:14:48,677::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:14:48,678::task::495::TaskManager.Task::(_debug) Task 654e3280-9a1f-4bce-ab21-c416b05e244a: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:14:48,680::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ce19f176-a270-4824-a6ec-7020c2201f88`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:14:48,680::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:14:48,681::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:14:48,682::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ce19f176-a270-4824-a6ec-7020c2201f88`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:14:48,683::task::495::TaskManager.Task::(_debug) Task 654e3280-9a1f-4bce-ab21-c416b05e244a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:14:48,683::task::495::TaskManager.Task::(_debug) Task 654e3280-9a1f-4bce-ab21-c416b05e244a: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:14:48,684::task::495::TaskManager.Task::(_debug) Task 654e3280-9a1f-4bce-ab21-c416b05e244a: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:14:48,685::task::495::TaskManager.Task::(_debug) Task 654e3280-9a1f-4bce-ab21-c416b05e244a: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:14:48,685::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:14:48,686::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:14:48,686::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:14:48,687::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:14:48,687::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:14:48,692::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:14:48,693::task::495::TaskManager.Task::(_debug) Task 654e3280-9a1f-4bce-ab21-c416b05e244a: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:14:48,693::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:14:48,694::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:14:48,694::task::495::TaskManager.Task::(_debug) Task 979351d0-7dfb-4ab4-9619-7d68822dbd4f: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:14:48,695::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`af42a3de-db8c-438b-a4c0-45c253af6233`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:14:48,696::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:14:48,696::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:14:48,697::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`af42a3de-db8c-438b-a4c0-45c253af6233`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:14:48,698::task::495::TaskManager.Task::(_debug) Task 979351d0-7dfb-4ab4-9619-7d68822dbd4f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:14:48,698::task::495::TaskManager.Task::(_debug) Task 979351d0-7dfb-4ab4-9619-7d68822dbd4f: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:14:48,699::task::495::TaskManager.Task::(_debug) Task 979351d0-7dfb-4ab4-9619-7d68822dbd4f: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:14:48,699::task::495::TaskManager.Task::(_debug) Task 979351d0-7dfb-4ab4-9619-7d68822dbd4f: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:14:48,700::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:14:48,703::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:14:48,704::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:14:48,704::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:14:48,705::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:14:48,705::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:14:48,705::task::495::TaskManager.Task::(_debug) Task 979351d0-7dfb-4ab4-9619-7d68822dbd4f: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:14:48,706::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:14:49,430::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0497402 s, 20.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:14:51,440::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:14:53,490::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0519181 s, 19.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:14:55,500::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:14:55,931::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0459256 s, 22.3 MB/s\n'; <rc> = 0
>Thread-15913835::DEBUG::2013-11-20 03:14:56,876::clientIF::54::vds::(wrapper) [192.168.100.135]::call setVmTicket with ('4760db3e-d3a8-4045-9153-12c36ec16ba2', 'ntakNoXQbUqB', '120') {}
>Thread-15913835::DEBUG::2013-11-20 03:14:56,921::clientIF::59::vds::(wrapper) return setVmTicket with {'status': {'message': 'Done', 'code': 0}}
>Thread-15913836::INFO::2013-11-20 03:14:57,049::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913836::DEBUG::2013-11-20 03:14:57,050::task::495::TaskManager.Task::(_debug) Task 51d0408a-bf84-4c1d-9987-f2a1a64d3120: moving from state init -> state preparing
>Thread-15913836::DEBUG::2013-11-20 03:14:57,051::task::495::TaskManager.Task::(_debug) Task 51d0408a-bf84-4c1d-9987-f2a1a64d3120: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00022292137146', 'lastCheck': 1384917290.946553, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.032075881958', 'lastCheck': 1384917288.127284, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00100708007812', 'lastCheck': 1384917290.513449, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000344038009644', 'lastCheck': 1384917290.7412441, 'valid': True, 'code': 0}}
>Thread-15913836::DEBUG::2013-11-20 03:14:57,063::task::495::TaskManager.Task::(_debug) Task 51d0408a-bf84-4c1d-9987-f2a1a64d3120: moving from state preparing -> state finished
>Thread-15913836::DEBUG::2013-11-20 03:14:57,064::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913836::DEBUG::2013-11-20 03:14:57,065::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913836::DEBUG::2013-11-20 03:14:57,066::task::495::TaskManager.Task::(_debug) Task 51d0408a-bf84-4c1d-9987-f2a1a64d3120: ref 0 aborting False
>Thread-15913836::INFO::2013-11-20 03:14:57,067::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00022292137146', 'lastCheck': 1384917290.946553, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.032075881958', 'lastCheck': 1384917288.127284, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00100708007812', 'lastCheck': 1384917290.513449, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000344038009644', 'lastCheck': 1384917290.7412441, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:14:57,941::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913838::DEBUG::2013-11-20 03:14:58,186::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913838::INFO::2013-11-20 03:14:58,188::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913838::DEBUG::2013-11-20 03:14:58,189::task::495::TaskManager.Task::(_debug) Task ffe79e12-cad8-4852-b5ae-30776134e0e6: moving from state init -> state preparing
>Thread-15913838::DEBUG::2013-11-20 03:14:58,190::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913838::DEBUG::2013-11-20 03:14:58,191::task::495::TaskManager.Task::(_debug) Task ffe79e12-cad8-4852-b5ae-30776134e0e6: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913838::DEBUG::2013-11-20 03:14:58,192::task::495::TaskManager.Task::(_debug) Task ffe79e12-cad8-4852-b5ae-30776134e0e6: moving from state preparing -> state finished
>Thread-15913838::DEBUG::2013-11-20 03:14:58,192::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913838::DEBUG::2013-11-20 03:14:58,193::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913838::DEBUG::2013-11-20 03:14:58,194::task::495::TaskManager.Task::(_debug) Task ffe79e12-cad8-4852-b5ae-30776134e0e6: ref 0 aborting False
>Thread-15913838::INFO::2013-11-20 03:14:58,194::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913839::DEBUG::2013-11-20 03:14:58,214::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913839::INFO::2013-11-20 03:14:58,215::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913839::DEBUG::2013-11-20 03:14:58,216::task::495::TaskManager.Task::(_debug) Task 099456f1-72e0-4d0e-b99f-499ce1a73db3: moving from state init -> state preparing
>Thread-15913839::DEBUG::2013-11-20 03:14:58,219::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5ad1422d-e598-4d75-a96f-f9d2317b51c1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913839::DEBUG::2013-11-20 03:14:58,220::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913839::DEBUG::2013-11-20 03:14:58,221::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913839::DEBUG::2013-11-20 03:14:58,222::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5ad1422d-e598-4d75-a96f-f9d2317b51c1`::Granted request
>Thread-15913839::DEBUG::2013-11-20 03:14:58,223::task::495::TaskManager.Task::(_debug) Task 099456f1-72e0-4d0e-b99f-499ce1a73db3: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913839::DEBUG::2013-11-20 03:14:58,224::task::495::TaskManager.Task::(_debug) Task 099456f1-72e0-4d0e-b99f-499ce1a73db3: ref 1 aborting False
>Thread-15913839::WARNING::2013-11-20 03:14:58,226::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913839::DEBUG::2013-11-20 03:14:58,234::task::495::TaskManager.Task::(_debug) Task 099456f1-72e0-4d0e-b99f-499ce1a73db3: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913839::DEBUG::2013-11-20 03:14:58,240::task::495::TaskManager.Task::(_debug) Task 099456f1-72e0-4d0e-b99f-499ce1a73db3: moving from state preparing -> state finished
>Thread-15913839::DEBUG::2013-11-20 03:14:58,241::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913839::DEBUG::2013-11-20 03:14:58,241::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913839::DEBUG::2013-11-20 03:14:58,242::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913839::DEBUG::2013-11-20 03:14:58,243::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913839::DEBUG::2013-11-20 03:14:58,244::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913839::DEBUG::2013-11-20 03:14:58,244::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913839::DEBUG::2013-11-20 03:14:58,245::task::495::TaskManager.Task::(_debug) Task 099456f1-72e0-4d0e-b99f-499ce1a73db3: ref 0 aborting False
>Thread-15913839::INFO::2013-11-20 03:14:58,251::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:14:58,440::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0796807 s, 12.9 MB/s\n'; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-20 03:14:58,585::vm::525::vm.Vm::(_rtcUpdate) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::new rtc offset 1
>libvirtEventLoop::DEBUG::2013-11-20 03:14:58,587::vm::525::vm.Vm::(_rtcUpdate) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::new rtc offset 1
>libvirtEventLoop::DEBUG::2013-11-20 03:14:58,588::vm::525::vm.Vm::(_rtcUpdate) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::new rtc offset 3601
>libvirtEventLoop::DEBUG::2013-11-20 03:14:58,588::vm::525::vm.Vm::(_rtcUpdate) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::new rtc offset 3601
>libvirtEventLoop::DEBUG::2013-11-20 03:14:58,589::vm::525::vm.Vm::(_rtcUpdate) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::new rtc offset 3601
>libvirtEventLoop::DEBUG::2013-11-20 03:14:58,591::vm::525::vm.Vm::(_rtcUpdate) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::new rtc offset 3601
>libvirtEventLoop::DEBUG::2013-11-20 03:14:58,591::vm::525::vm.Vm::(_rtcUpdate) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::new rtc offset 3601
>libvirtEventLoop::DEBUG::2013-11-20 03:14:58,617::vm::525::vm.Vm::(_rtcUpdate) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::new rtc offset 3601
>libvirtEventLoop::DEBUG::2013-11-20 03:14:58,618::vm::525::vm.Vm::(_rtcUpdate) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::new rtc offset 3601
>libvirtEventLoop::DEBUG::2013-11-20 03:14:58,619::vm::525::vm.Vm::(_rtcUpdate) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::new rtc offset 3601
>libvirtEventLoop::DEBUG::2013-11-20 03:14:58,619::vm::525::vm.Vm::(_rtcUpdate) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::new rtc offset 3601
>libvirtEventLoop::DEBUG::2013-11-20 03:14:58,620::vm::525::vm.Vm::(_rtcUpdate) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::new rtc offset 3601
>libvirtEventLoop::DEBUG::2013-11-20 03:14:58,620::vm::525::vm.Vm::(_rtcUpdate) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::new rtc offset 3601
>libvirtEventLoop::DEBUG::2013-11-20 03:14:58,621::vm::525::vm.Vm::(_rtcUpdate) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::new rtc offset 3601
>Dummy-15912651::DEBUG::2013-11-20 03:15:00,454::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>libvirtEventLoop::DEBUG::2013-11-20 03:15:00,700::libvirtconnection::43::vm.Vm::(__eventCallback) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::graphics event phase 0 localAddr {'node': '192.168.100.133', 'service': '', 'family': 0} remoteAddr {'node': '192.168.50.51', 'service': '', 'family': 0}authScheme spice subject []
>libvirtEventLoop::DEBUG::2013-11-20 03:15:00,702::libvirtconnection::43::vm.Vm::(__eventCallback) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::graphics event phase 1 localAddr {'node': '192.168.100.133', 'service': '', 'family': 0} remoteAddr {'node': '192.168.50.51', 'service': '', 'family': 0}authScheme spice subject []
>libvirtEventLoop::DEBUG::2013-11-20 03:15:00,702::libvirtconnection::43::vm.Vm::(__eventCallback) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::graphics event phase 0 localAddr {'node': '192.168.100.133', 'service': '', 'family': 0} remoteAddr {'node': '192.168.50.51', 'service': '', 'family': 0}authScheme spice subject []
>libvirtEventLoop::DEBUG::2013-11-20 03:15:00,704::libvirtconnection::43::vm.Vm::(__eventCallback) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::graphics event phase 1 localAddr {'node': '192.168.100.133', 'service': '', 'family': 0} remoteAddr {'node': '192.168.50.51', 'service': '', 'family': 0}authScheme spice subject []
>libvirtEventLoop::DEBUG::2013-11-20 03:15:00,773::libvirtconnection::43::vm.Vm::(__eventCallback) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::graphics event phase 0 localAddr {'node': '192.168.100.133', 'service': '', 'family': 0} remoteAddr {'node': '192.168.50.51', 'service': '', 'family': 0}authScheme spice subject []
>libvirtEventLoop::DEBUG::2013-11-20 03:15:00,774::libvirtconnection::43::vm.Vm::(__eventCallback) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::graphics event phase 1 localAddr {'node': '192.168.100.133', 'service': '', 'family': 0} remoteAddr {'node': '192.168.50.51', 'service': '', 'family': 0}authScheme spice subject []
>libvirtEventLoop::DEBUG::2013-11-20 03:15:00,818::libvirtconnection::43::vm.Vm::(__eventCallback) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::graphics event phase 0 localAddr {'node': '192.168.100.133', 'service': '', 'family': 0} remoteAddr {'node': '192.168.50.51', 'service': '', 'family': 0}authScheme spice subject []
>libvirtEventLoop::DEBUG::2013-11-20 03:15:00,819::libvirtconnection::43::vm.Vm::(__eventCallback) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::graphics event phase 1 localAddr {'node': '192.168.100.133', 'service': '', 'family': 0} remoteAddr {'node': '192.168.50.51', 'service': '', 'family': 0}authScheme spice subject []
>libvirtEventLoop::DEBUG::2013-11-20 03:15:00,837::libvirtconnection::43::vm.Vm::(__eventCallback) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::graphics event phase 0 localAddr {'node': '192.168.100.133', 'service': '', 'family': 0} remoteAddr {'node': '192.168.50.51', 'service': '', 'family': 0}authScheme spice subject []
>libvirtEventLoop::DEBUG::2013-11-20 03:15:00,839::libvirtconnection::43::vm.Vm::(__eventCallback) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::graphics event phase 1 localAddr {'node': '192.168.100.133', 'service': '', 'family': 0} remoteAddr {'node': '192.168.50.51', 'service': '', 'family': 0}authScheme spice subject []
>Dummy-15912651::DEBUG::2013-11-20 03:15:00,900::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0620971 s, 16.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:15:02,910::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:15:03,453::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0441118 s, 23.2 MB/s\n'; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-20 03:15:05,210::libvirtconnection::43::vm.Vm::(__eventCallback) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::graphics event phase 2 localAddr {'node': '192.168.100.133', 'service': '', 'family': 0} remoteAddr {'node': '192.168.50.51', 'service': '', 'family': 0}authScheme spice subject []
>libvirtEventLoop::DEBUG::2013-11-20 03:15:05,211::guestIF::154::vm.Vm::(desktopLock) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::desktopLock called
>Dummy-15912651::DEBUG::2013-11-20 03:15:05,464::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:15:06,011::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0727686 s, 14.1 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 03:15:06,155::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:15:06,156::task::495::TaskManager.Task::(_debug) Task 31abe6fb-024c-4b03-9915-bbbff0bf6c75: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:15:06,159::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6d85e528-0e5f-4574-a0dc-6893b088c7ae`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:15:06,161::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:15:06,163::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:15:06,163::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6d85e528-0e5f-4574-a0dc-6893b088c7ae`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:15:06,164::task::495::TaskManager.Task::(_debug) Task 31abe6fb-024c-4b03-9915-bbbff0bf6c75: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:15:06,164::task::495::TaskManager.Task::(_debug) Task 31abe6fb-024c-4b03-9915-bbbff0bf6c75: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:15:06,165::task::495::TaskManager.Task::(_debug) Task 31abe6fb-024c-4b03-9915-bbbff0bf6c75: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:15:06,165::task::495::TaskManager.Task::(_debug) Task 31abe6fb-024c-4b03-9915-bbbff0bf6c75: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:15:06,166::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:15:06,166::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:15:06,166::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:15:06,167::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:15:06,167::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:15:06,167::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:15:06,168::task::495::TaskManager.Task::(_debug) Task 31abe6fb-024c-4b03-9915-bbbff0bf6c75: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:15:06,168::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:15:06,169::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:15:06,169::task::495::TaskManager.Task::(_debug) Task 89a529dc-5346-4668-a920-aea771942ef3: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:15:06,170::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6029dfc1-ea2a-4ffe-820c-1ffb76623bbe`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:15:06,170::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:15:06,171::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:15:06,171::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6029dfc1-ea2a-4ffe-820c-1ffb76623bbe`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:15:06,171::task::495::TaskManager.Task::(_debug) Task 89a529dc-5346-4668-a920-aea771942ef3: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:15:06,172::task::495::TaskManager.Task::(_debug) Task 89a529dc-5346-4668-a920-aea771942ef3: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:15:06,172::task::495::TaskManager.Task::(_debug) Task 89a529dc-5346-4668-a920-aea771942ef3: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:15:06,172::task::495::TaskManager.Task::(_debug) Task 89a529dc-5346-4668-a920-aea771942ef3: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:15:06,173::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:15:06,173::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:15:06,173::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:15:06,174::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:15:06,174::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:15:06,174::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:15:06,175::task::495::TaskManager.Task::(_debug) Task 89a529dc-5346-4668-a920-aea771942ef3: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:15:06,175::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:15:06,176::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:15:06,176::task::495::TaskManager.Task::(_debug) Task e1231f7a-5fef-4648-ba2e-ba084afecb44: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:15:06,176::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`0fd9d3cc-a8a3-4524-8ab3-9db10bda95cc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:15:06,177::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:15:06,177::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:15:06,177::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`0fd9d3cc-a8a3-4524-8ab3-9db10bda95cc`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:15:06,178::task::495::TaskManager.Task::(_debug) Task e1231f7a-5fef-4648-ba2e-ba084afecb44: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:15:06,178::task::495::TaskManager.Task::(_debug) Task e1231f7a-5fef-4648-ba2e-ba084afecb44: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:15:06,179::task::495::TaskManager.Task::(_debug) Task e1231f7a-5fef-4648-ba2e-ba084afecb44: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:15:06,179::task::495::TaskManager.Task::(_debug) Task e1231f7a-5fef-4648-ba2e-ba084afecb44: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:15:06,180::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:15:06,180::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:15:06,181::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:15:06,181::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:15:06,182::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:15:06,182::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:15:06,183::task::495::TaskManager.Task::(_debug) Task e1231f7a-5fef-4648-ba2e-ba084afecb44: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:15:06,184::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:15:06,185::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:15:06,185::task::495::TaskManager.Task::(_debug) Task 5996c661-df24-4516-8aeb-f5289d7f5ac5: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:15:06,187::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e9cb2e59-1b0a-42d5-9612-1f92a5e9980e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:15:06,187::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:15:06,188::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:15:06,189::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e9cb2e59-1b0a-42d5-9612-1f92a5e9980e`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:15:06,189::task::495::TaskManager.Task::(_debug) Task 5996c661-df24-4516-8aeb-f5289d7f5ac5: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:15:06,190::task::495::TaskManager.Task::(_debug) Task 5996c661-df24-4516-8aeb-f5289d7f5ac5: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:15:06,190::task::495::TaskManager.Task::(_debug) Task 5996c661-df24-4516-8aeb-f5289d7f5ac5: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:15:06,191::task::495::TaskManager.Task::(_debug) Task 5996c661-df24-4516-8aeb-f5289d7f5ac5: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:15:06,191::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:15:06,191::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:15:06,192::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:15:06,193::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:15:06,193::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:15:06,194::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:15:06,194::task::495::TaskManager.Task::(_debug) Task 5996c661-df24-4516-8aeb-f5289d7f5ac5: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:15:06,194::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:15:08,026::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913840::DEBUG::2013-11-20 03:15:08,457::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913840::INFO::2013-11-20 03:15:08,459::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913840::DEBUG::2013-11-20 03:15:08,460::task::495::TaskManager.Task::(_debug) Task 68fb5e28-c9fe-47c8-9267-2d9cbff09a9d: moving from state init -> state preparing
>Thread-15913840::DEBUG::2013-11-20 03:15:08,462::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913840::DEBUG::2013-11-20 03:15:08,463::task::495::TaskManager.Task::(_debug) Task 68fb5e28-c9fe-47c8-9267-2d9cbff09a9d: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913840::DEBUG::2013-11-20 03:15:08,464::task::495::TaskManager.Task::(_debug) Task 68fb5e28-c9fe-47c8-9267-2d9cbff09a9d: moving from state preparing -> state finished
>Thread-15913840::DEBUG::2013-11-20 03:15:08,466::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913840::DEBUG::2013-11-20 03:15:08,467::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913840::DEBUG::2013-11-20 03:15:08,468::task::495::TaskManager.Task::(_debug) Task 68fb5e28-c9fe-47c8-9267-2d9cbff09a9d: ref 0 aborting False
>Thread-15913840::INFO::2013-11-20 03:15:08,470::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913841::DEBUG::2013-11-20 03:15:08,485::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913841::INFO::2013-11-20 03:15:08,486::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913841::DEBUG::2013-11-20 03:15:08,486::task::495::TaskManager.Task::(_debug) Task 367f0a68-07cd-4211-9e4d-c69eacafd03b: moving from state init -> state preparing
>Thread-15913841::DEBUG::2013-11-20 03:15:08,488::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b3a1ad0e-25e9-4e60-acae-13897727a434`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913841::DEBUG::2013-11-20 03:15:08,488::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913841::DEBUG::2013-11-20 03:15:08,489::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913841::DEBUG::2013-11-20 03:15:08,490::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b3a1ad0e-25e9-4e60-acae-13897727a434`::Granted request
>Thread-15913841::DEBUG::2013-11-20 03:15:08,499::task::495::TaskManager.Task::(_debug) Task 367f0a68-07cd-4211-9e4d-c69eacafd03b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913841::DEBUG::2013-11-20 03:15:08,500::task::495::TaskManager.Task::(_debug) Task 367f0a68-07cd-4211-9e4d-c69eacafd03b: ref 1 aborting False
>Thread-15913841::WARNING::2013-11-20 03:15:08,503::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913841::DEBUG::2013-11-20 03:15:08,508::task::495::TaskManager.Task::(_debug) Task 367f0a68-07cd-4211-9e4d-c69eacafd03b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913841::DEBUG::2013-11-20 03:15:08,515::task::495::TaskManager.Task::(_debug) Task 367f0a68-07cd-4211-9e4d-c69eacafd03b: moving from state preparing -> state finished
>Thread-15913841::DEBUG::2013-11-20 03:15:08,516::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913841::DEBUG::2013-11-20 03:15:08,517::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913841::DEBUG::2013-11-20 03:15:08,521::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913841::DEBUG::2013-11-20 03:15:08,522::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913841::DEBUG::2013-11-20 03:15:08,522::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913841::DEBUG::2013-11-20 03:15:08,523::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913841::DEBUG::2013-11-20 03:15:08,524::task::495::TaskManager.Task::(_debug) Task 367f0a68-07cd-4211-9e4d-c69eacafd03b: ref 0 aborting False
>Thread-15913841::INFO::2013-11-20 03:15:08,524::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:15:08,637::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.131657 s, 7.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:15:10,652::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:15:11,199::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0779735 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:15:13,214::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:15:13,709::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.073577 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:15:15,723::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 03:15:16,251::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:15:16,253::task::495::TaskManager.Task::(_debug) Task 3a8c377d-402b-411a-adfb-5658f6d7602f: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:15:16,258::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`84f8f04c-df4f-400e-bf89-7b9eb1ffcaf2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:15:16,259::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:15:16,263::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:15:16,264::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`84f8f04c-df4f-400e-bf89-7b9eb1ffcaf2`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:15:16,269::task::495::TaskManager.Task::(_debug) Task 3a8c377d-402b-411a-adfb-5658f6d7602f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:15:16,270::task::495::TaskManager.Task::(_debug) Task 3a8c377d-402b-411a-adfb-5658f6d7602f: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:15:16,271::task::495::TaskManager.Task::(_debug) Task 3a8c377d-402b-411a-adfb-5658f6d7602f: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:15:16,272::task::495::TaskManager.Task::(_debug) Task 3a8c377d-402b-411a-adfb-5658f6d7602f: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:15:16,274::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:15:16,281::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:15:16,284::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:15:16,285::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:15:16,287::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:15:16,288::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:15:16,289::task::495::TaskManager.Task::(_debug) Task 3a8c377d-402b-411a-adfb-5658f6d7602f: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:15:16,290::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:15:16,295::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0723194 s, 14.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:15:18,312::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913845::DEBUG::2013-11-20 03:15:18,818::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913845::INFO::2013-11-20 03:15:18,820::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913845::DEBUG::2013-11-20 03:15:18,821::task::495::TaskManager.Task::(_debug) Task 9504abf1-cbfd-472e-a612-b7cd4a03e62d: moving from state init -> state preparing
>Thread-15913845::DEBUG::2013-11-20 03:15:18,821::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913845::DEBUG::2013-11-20 03:15:18,822::task::495::TaskManager.Task::(_debug) Task 9504abf1-cbfd-472e-a612-b7cd4a03e62d: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913845::DEBUG::2013-11-20 03:15:18,823::task::495::TaskManager.Task::(_debug) Task 9504abf1-cbfd-472e-a612-b7cd4a03e62d: moving from state preparing -> state finished
>Thread-15913845::DEBUG::2013-11-20 03:15:18,823::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913845::DEBUG::2013-11-20 03:15:18,824::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913845::DEBUG::2013-11-20 03:15:18,824::task::495::TaskManager.Task::(_debug) Task 9504abf1-cbfd-472e-a612-b7cd4a03e62d: ref 0 aborting False
>Thread-15913845::INFO::2013-11-20 03:15:18,825::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913846::DEBUG::2013-11-20 03:15:18,836::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913846::INFO::2013-11-20 03:15:18,837::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913846::DEBUG::2013-11-20 03:15:18,838::task::495::TaskManager.Task::(_debug) Task 86e736cd-cc2d-4461-a90e-fe65509a393c: moving from state init -> state preparing
>Thread-15913846::DEBUG::2013-11-20 03:15:18,839::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e646df18-f3cc-4c48-86d1-b4b3a8bd091f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913846::DEBUG::2013-11-20 03:15:18,840::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913846::DEBUG::2013-11-20 03:15:18,841::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913846::DEBUG::2013-11-20 03:15:18,841::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e646df18-f3cc-4c48-86d1-b4b3a8bd091f`::Granted request
>Thread-15913846::DEBUG::2013-11-20 03:15:18,846::task::495::TaskManager.Task::(_debug) Task 86e736cd-cc2d-4461-a90e-fe65509a393c: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913846::DEBUG::2013-11-20 03:15:18,848::task::495::TaskManager.Task::(_debug) Task 86e736cd-cc2d-4461-a90e-fe65509a393c: ref 1 aborting False
>Thread-15913846::WARNING::2013-11-20 03:15:18,850::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913846::DEBUG::2013-11-20 03:15:18,854::task::495::TaskManager.Task::(_debug) Task 86e736cd-cc2d-4461-a90e-fe65509a393c: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913846::DEBUG::2013-11-20 03:15:18,855::task::495::TaskManager.Task::(_debug) Task 86e736cd-cc2d-4461-a90e-fe65509a393c: moving from state preparing -> state finished
>Thread-15913846::DEBUG::2013-11-20 03:15:18,855::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913846::DEBUG::2013-11-20 03:15:18,856::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913846::DEBUG::2013-11-20 03:15:18,859::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913846::DEBUG::2013-11-20 03:15:18,861::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913846::DEBUG::2013-11-20 03:15:18,862::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913846::DEBUG::2013-11-20 03:15:18,863::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913846::DEBUG::2013-11-20 03:15:18,865::task::495::TaskManager.Task::(_debug) Task 86e736cd-cc2d-4461-a90e-fe65509a393c: ref 0 aborting False
>Thread-15913846::INFO::2013-11-20 03:15:18,866::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:15:18,916::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.111414 s, 9.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:15:20,946::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:15:21,457::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0518454 s, 19.8 MB/s\n'; <rc> = 0
>Thread-15913848::INFO::2013-11-20 03:15:23,129::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913848::DEBUG::2013-11-20 03:15:23,130::task::495::TaskManager.Task::(_debug) Task a43b1a99-c660-4bc4-8bfb-8e8c5eb053a7: moving from state init -> state preparing
>Thread-15913848::DEBUG::2013-11-20 03:15:23,131::task::495::TaskManager.Task::(_debug) Task a43b1a99-c660-4bc4-8bfb-8e8c5eb053a7: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000326871871948', 'lastCheck': 1384917321.5482509, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00178408622742', 'lastCheck': 1384917318.5490601, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00157403945923', 'lastCheck': 1384917321.1813419, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00189995765686', 'lastCheck': 1384917321.1732731, 'valid': True, 'code': 0}}
>Thread-15913848::DEBUG::2013-11-20 03:15:23,138::task::495::TaskManager.Task::(_debug) Task a43b1a99-c660-4bc4-8bfb-8e8c5eb053a7: moving from state preparing -> state finished
>Thread-15913848::DEBUG::2013-11-20 03:15:23,147::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913848::DEBUG::2013-11-20 03:15:23,148::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913848::DEBUG::2013-11-20 03:15:23,148::task::495::TaskManager.Task::(_debug) Task a43b1a99-c660-4bc4-8bfb-8e8c5eb053a7: ref 0 aborting False
>Thread-15913848::INFO::2013-11-20 03:15:23,149::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000326871871948', 'lastCheck': 1384917321.5482509, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00178408622742', 'lastCheck': 1384917318.5490601, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00157403945923', 'lastCheck': 1384917321.1813419, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00189995765686', 'lastCheck': 1384917321.1732731, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:15:23,472::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:15:24,017::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.096318 s, 10.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:15:26,031::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:15:26,516::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0543675 s, 18.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:15:28,531::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:15:29,120::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0792786 s, 12.9 MB/s\n'; <rc> = 0
>Thread-15913851::DEBUG::2013-11-20 03:15:29,270::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913851::INFO::2013-11-20 03:15:29,272::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913851::DEBUG::2013-11-20 03:15:29,273::task::495::TaskManager.Task::(_debug) Task 16ffa110-8397-4723-87ab-1aa98091ed33: moving from state init -> state preparing
>Thread-15913851::DEBUG::2013-11-20 03:15:29,275::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913851::DEBUG::2013-11-20 03:15:29,275::task::495::TaskManager.Task::(_debug) Task 16ffa110-8397-4723-87ab-1aa98091ed33: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913851::DEBUG::2013-11-20 03:15:29,276::task::495::TaskManager.Task::(_debug) Task 16ffa110-8397-4723-87ab-1aa98091ed33: moving from state preparing -> state finished
>Thread-15913851::DEBUG::2013-11-20 03:15:29,277::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913851::DEBUG::2013-11-20 03:15:29,278::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913851::DEBUG::2013-11-20 03:15:29,278::task::495::TaskManager.Task::(_debug) Task 16ffa110-8397-4723-87ab-1aa98091ed33: ref 0 aborting False
>Thread-15913851::INFO::2013-11-20 03:15:29,279::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913852::DEBUG::2013-11-20 03:15:29,285::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913852::INFO::2013-11-20 03:15:29,286::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913852::DEBUG::2013-11-20 03:15:29,287::task::495::TaskManager.Task::(_debug) Task f12af0f5-f87e-4140-a6b9-1b43c7d95f84: moving from state init -> state preparing
>Thread-15913852::DEBUG::2013-11-20 03:15:29,288::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`32dcd40c-bfb7-4002-a120-eceea4af3514`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913852::DEBUG::2013-11-20 03:15:29,289::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913852::DEBUG::2013-11-20 03:15:29,290::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913852::DEBUG::2013-11-20 03:15:29,291::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`32dcd40c-bfb7-4002-a120-eceea4af3514`::Granted request
>Thread-15913852::DEBUG::2013-11-20 03:15:29,300::task::495::TaskManager.Task::(_debug) Task f12af0f5-f87e-4140-a6b9-1b43c7d95f84: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913852::DEBUG::2013-11-20 03:15:29,301::task::495::TaskManager.Task::(_debug) Task f12af0f5-f87e-4140-a6b9-1b43c7d95f84: ref 1 aborting False
>Thread-15913852::WARNING::2013-11-20 03:15:29,304::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913852::DEBUG::2013-11-20 03:15:29,305::task::495::TaskManager.Task::(_debug) Task f12af0f5-f87e-4140-a6b9-1b43c7d95f84: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913852::DEBUG::2013-11-20 03:15:29,306::task::495::TaskManager.Task::(_debug) Task f12af0f5-f87e-4140-a6b9-1b43c7d95f84: moving from state preparing -> state finished
>Thread-15913852::DEBUG::2013-11-20 03:15:29,307::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913852::DEBUG::2013-11-20 03:15:29,308::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913852::DEBUG::2013-11-20 03:15:29,309::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913852::DEBUG::2013-11-20 03:15:29,309::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913852::DEBUG::2013-11-20 03:15:29,310::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913852::DEBUG::2013-11-20 03:15:29,311::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913852::DEBUG::2013-11-20 03:15:29,311::task::495::TaskManager.Task::(_debug) Task f12af0f5-f87e-4140-a6b9-1b43c7d95f84: ref 0 aborting False
>Thread-15913852::INFO::2013-11-20 03:15:29,312::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:15:31,135::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:15:31,657::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0536244 s, 19.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:15:33,672::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 03:15:34,032::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:15:34,033::task::495::TaskManager.Task::(_debug) Task 0b41141d-8c6d-4bad-b0d1-77780ec3647f: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:15:34,034::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bfe08a4d-464a-4c48-b217-87711be8e4b2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:15:34,035::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:15:34,036::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:15:34,036::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bfe08a4d-464a-4c48-b217-87711be8e4b2`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:15:34,037::task::495::TaskManager.Task::(_debug) Task 0b41141d-8c6d-4bad-b0d1-77780ec3647f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:15:34,038::task::495::TaskManager.Task::(_debug) Task 0b41141d-8c6d-4bad-b0d1-77780ec3647f: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:15:34,038::task::495::TaskManager.Task::(_debug) Task 0b41141d-8c6d-4bad-b0d1-77780ec3647f: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:15:34,039::task::495::TaskManager.Task::(_debug) Task 0b41141d-8c6d-4bad-b0d1-77780ec3647f: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:15:34,039::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:15:34,040::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:15:34,040::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:15:34,041::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:15:34,041::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:15:34,042::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:15:34,042::task::495::TaskManager.Task::(_debug) Task 0b41141d-8c6d-4bad-b0d1-77780ec3647f: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:15:34,043::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:15:34,240::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0741407 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15913659::DEBUG::2013-11-20 03:15:34,963::guestIF::97::vm.Vm::(_handleMessage) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::u'None'
>Dummy-15912651::DEBUG::2013-11-20 03:15:36,256::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 03:15:36,488::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:15:36,491::task::495::TaskManager.Task::(_debug) Task b5c0537a-f6ba-4f43-a757-434a7ea3d151: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:15:36,495::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`86f0b649-205b-4504-a929-b5d67596edf5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:15:36,495::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:15:36,497::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:15:36,498::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`86f0b649-205b-4504-a929-b5d67596edf5`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:15:36,499::task::495::TaskManager.Task::(_debug) Task b5c0537a-f6ba-4f43-a757-434a7ea3d151: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:15:36,499::task::495::TaskManager.Task::(_debug) Task b5c0537a-f6ba-4f43-a757-434a7ea3d151: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:15:36,500::task::495::TaskManager.Task::(_debug) Task b5c0537a-f6ba-4f43-a757-434a7ea3d151: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:15:36,501::task::495::TaskManager.Task::(_debug) Task b5c0537a-f6ba-4f43-a757-434a7ea3d151: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:15:36,501::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:15:36,502::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:15:36,503::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:15:36,503::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:15:36,504::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:15:36,504::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:15:36,505::task::495::TaskManager.Task::(_debug) Task b5c0537a-f6ba-4f43-a757-434a7ea3d151: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:15:36,505::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:15:36,845::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0782756 s, 13.1 MB/s\n'; <rc> = 0
>Thread-15913856::INFO::2013-11-20 03:15:38,810::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913856::DEBUG::2013-11-20 03:15:38,811::task::495::TaskManager.Task::(_debug) Task a47b3566-05f5-491b-b772-4b6792476895: moving from state init -> state preparing
>Thread-15913856::DEBUG::2013-11-20 03:15:38,812::task::495::TaskManager.Task::(_debug) Task a47b3566-05f5-491b-b772-4b6792476895: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000524997711182', 'lastCheck': 1384917331.771553, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00150299072266', 'lastCheck': 1384917328.775408, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0134840011597', 'lastCheck': 1384917331.3896489, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00231218338013', 'lastCheck': 1384917331.3697641, 'valid': True, 'code': 0}}
>Thread-15913856::DEBUG::2013-11-20 03:15:38,813::task::495::TaskManager.Task::(_debug) Task a47b3566-05f5-491b-b772-4b6792476895: moving from state preparing -> state finished
>Thread-15913856::DEBUG::2013-11-20 03:15:38,814::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913856::DEBUG::2013-11-20 03:15:38,814::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913856::DEBUG::2013-11-20 03:15:38,815::task::495::TaskManager.Task::(_debug) Task a47b3566-05f5-491b-b772-4b6792476895: ref 0 aborting False
>Thread-15913856::INFO::2013-11-20 03:15:38,816::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000524997711182', 'lastCheck': 1384917331.771553, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00150299072266', 'lastCheck': 1384917328.775408, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0134840011597', 'lastCheck': 1384917331.3896489, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00231218338013', 'lastCheck': 1384917331.3697641, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:15:38,856::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:15:39,437::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0832579 s, 12.3 MB/s\n'; <rc> = 0
>Thread-15913858::DEBUG::2013-11-20 03:15:39,698::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913858::INFO::2013-11-20 03:15:39,700::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913858::DEBUG::2013-11-20 03:15:39,700::task::495::TaskManager.Task::(_debug) Task 6a7c176d-55fa-45ca-a23c-57b3e6bc8299: moving from state init -> state preparing
>Thread-15913858::DEBUG::2013-11-20 03:15:39,701::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913858::DEBUG::2013-11-20 03:15:39,702::task::495::TaskManager.Task::(_debug) Task 6a7c176d-55fa-45ca-a23c-57b3e6bc8299: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913858::DEBUG::2013-11-20 03:15:39,703::task::495::TaskManager.Task::(_debug) Task 6a7c176d-55fa-45ca-a23c-57b3e6bc8299: moving from state preparing -> state finished
>Thread-15913858::DEBUG::2013-11-20 03:15:39,704::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913858::DEBUG::2013-11-20 03:15:39,704::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913858::DEBUG::2013-11-20 03:15:39,705::task::495::TaskManager.Task::(_debug) Task 6a7c176d-55fa-45ca-a23c-57b3e6bc8299: ref 0 aborting False
>Thread-15913858::INFO::2013-11-20 03:15:39,705::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913859::DEBUG::2013-11-20 03:15:39,717::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913859::INFO::2013-11-20 03:15:39,718::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913859::DEBUG::2013-11-20 03:15:39,719::task::495::TaskManager.Task::(_debug) Task 3ed1efee-2ef0-4feb-a332-554fde45dd25: moving from state init -> state preparing
>Thread-15913859::DEBUG::2013-11-20 03:15:39,721::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0eba2c31-5043-4a9c-a47e-05845c82f590`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913859::DEBUG::2013-11-20 03:15:39,722::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913859::DEBUG::2013-11-20 03:15:39,723::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913859::DEBUG::2013-11-20 03:15:39,724::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0eba2c31-5043-4a9c-a47e-05845c82f590`::Granted request
>Thread-15913859::DEBUG::2013-11-20 03:15:39,726::task::495::TaskManager.Task::(_debug) Task 3ed1efee-2ef0-4feb-a332-554fde45dd25: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913859::DEBUG::2013-11-20 03:15:39,726::task::495::TaskManager.Task::(_debug) Task 3ed1efee-2ef0-4feb-a332-554fde45dd25: ref 1 aborting False
>Thread-15913859::WARNING::2013-11-20 03:15:39,728::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913859::DEBUG::2013-11-20 03:15:39,729::task::495::TaskManager.Task::(_debug) Task 3ed1efee-2ef0-4feb-a332-554fde45dd25: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913859::DEBUG::2013-11-20 03:15:39,730::task::495::TaskManager.Task::(_debug) Task 3ed1efee-2ef0-4feb-a332-554fde45dd25: moving from state preparing -> state finished
>Thread-15913859::DEBUG::2013-11-20 03:15:39,731::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913859::DEBUG::2013-11-20 03:15:39,731::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913859::DEBUG::2013-11-20 03:15:39,732::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913859::DEBUG::2013-11-20 03:15:39,732::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913859::DEBUG::2013-11-20 03:15:39,733::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913859::DEBUG::2013-11-20 03:15:39,733::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913859::DEBUG::2013-11-20 03:15:39,734::task::495::TaskManager.Task::(_debug) Task 3ed1efee-2ef0-4feb-a332-554fde45dd25: ref 0 aborting False
>Thread-15913859::INFO::2013-11-20 03:15:39,735::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:15:41,452::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:15:42,027::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.077363 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 03:15:42,490::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:15:42,491::task::495::TaskManager.Task::(_debug) Task 7df535df-a13d-450a-add3-cc69a9d3b02a: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:15:42,492::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d83c17b4-61ff-4353-a8db-e3a7abaea5d8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:15:42,492::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:15:42,493::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:15:42,493::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d83c17b4-61ff-4353-a8db-e3a7abaea5d8`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:15:42,494::task::495::TaskManager.Task::(_debug) Task 7df535df-a13d-450a-add3-cc69a9d3b02a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:15:42,494::task::495::TaskManager.Task::(_debug) Task 7df535df-a13d-450a-add3-cc69a9d3b02a: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:15:42,495::task::495::TaskManager.Task::(_debug) Task 7df535df-a13d-450a-add3-cc69a9d3b02a: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:15:42,495::task::495::TaskManager.Task::(_debug) Task 7df535df-a13d-450a-add3-cc69a9d3b02a: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:15:42,495::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:15:42,496::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:15:42,496::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:15:42,496::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:15:42,497::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:15:42,497::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:15:42,498::task::495::TaskManager.Task::(_debug) Task 7df535df-a13d-450a-add3-cc69a9d3b02a: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:15:42,498::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:15:44,041::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:15:44,499::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.059759 s, 17.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:15:46,512::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:15:47,029::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0836849 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:15:49,040::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:15:49,563::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0564909 s, 18.1 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 03:15:49,770::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:15:49,771::task::495::TaskManager.Task::(_debug) Task 5e4089d4-c208-4ed4-b683-d853f0e3fd89: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:15:49,773::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e2755a76-7349-4465-8e8d-338d6a1b504a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:15:49,774::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:15:49,775::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:15:49,776::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e2755a76-7349-4465-8e8d-338d6a1b504a`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:15:49,777::task::495::TaskManager.Task::(_debug) Task 5e4089d4-c208-4ed4-b683-d853f0e3fd89: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:15:49,778::task::495::TaskManager.Task::(_debug) Task 5e4089d4-c208-4ed4-b683-d853f0e3fd89: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:15:49,779::task::495::TaskManager.Task::(_debug) Task 5e4089d4-c208-4ed4-b683-d853f0e3fd89: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:15:49,780::task::495::TaskManager.Task::(_debug) Task 5e4089d4-c208-4ed4-b683-d853f0e3fd89: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:15:49,781::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:15:49,781::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:15:49,782::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:15:49,783::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:15:49,783::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:15:49,784::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:15:49,785::task::495::TaskManager.Task::(_debug) Task 5e4089d4-c208-4ed4-b683-d853f0e3fd89: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:15:49,785::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:15:49,786::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:15:49,787::task::495::TaskManager.Task::(_debug) Task eca4fdad-d323-4fbe-a202-54cd3c181600: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:15:49,788::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b038db89-8825-40fe-8e30-509739ca9b6e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:15:49,789::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:15:49,789::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:15:49,790::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b038db89-8825-40fe-8e30-509739ca9b6e`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:15:49,791::task::495::TaskManager.Task::(_debug) Task eca4fdad-d323-4fbe-a202-54cd3c181600: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:15:49,792::task::495::TaskManager.Task::(_debug) Task eca4fdad-d323-4fbe-a202-54cd3c181600: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:15:49,792::task::495::TaskManager.Task::(_debug) Task eca4fdad-d323-4fbe-a202-54cd3c181600: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:15:49,793::task::495::TaskManager.Task::(_debug) Task eca4fdad-d323-4fbe-a202-54cd3c181600: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:15:49,794::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:15:49,794::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:15:49,795::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:15:49,796::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:15:49,796::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:15:49,797::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:15:49,798::task::495::TaskManager.Task::(_debug) Task eca4fdad-d323-4fbe-a202-54cd3c181600: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:15:49,798::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913863::DEBUG::2013-11-20 03:15:49,985::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913863::INFO::2013-11-20 03:15:49,994::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913863::DEBUG::2013-11-20 03:15:49,999::task::495::TaskManager.Task::(_debug) Task 9642c63d-ae0d-4083-980f-2d471600f076: moving from state init -> state preparing
>Thread-15913863::DEBUG::2013-11-20 03:15:50,001::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913863::DEBUG::2013-11-20 03:15:50,002::task::495::TaskManager.Task::(_debug) Task 9642c63d-ae0d-4083-980f-2d471600f076: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913863::DEBUG::2013-11-20 03:15:50,003::task::495::TaskManager.Task::(_debug) Task 9642c63d-ae0d-4083-980f-2d471600f076: moving from state preparing -> state finished
>Thread-15913863::DEBUG::2013-11-20 03:15:50,004::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913863::DEBUG::2013-11-20 03:15:50,004::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913863::DEBUG::2013-11-20 03:15:50,005::task::495::TaskManager.Task::(_debug) Task 9642c63d-ae0d-4083-980f-2d471600f076: ref 0 aborting False
>Thread-15913863::INFO::2013-11-20 03:15:50,005::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913864::DEBUG::2013-11-20 03:15:50,010::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913864::INFO::2013-11-20 03:15:50,011::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913864::DEBUG::2013-11-20 03:15:50,011::task::495::TaskManager.Task::(_debug) Task bc984a27-c13c-4d8e-bb14-1c042818a49d: moving from state init -> state preparing
>Thread-15913864::DEBUG::2013-11-20 03:15:50,012::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a9b58190-1d20-40a2-b7e1-212f579644b8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913864::DEBUG::2013-11-20 03:15:50,013::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913864::DEBUG::2013-11-20 03:15:50,014::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913864::DEBUG::2013-11-20 03:15:50,014::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a9b58190-1d20-40a2-b7e1-212f579644b8`::Granted request
>Thread-15913864::DEBUG::2013-11-20 03:15:50,015::task::495::TaskManager.Task::(_debug) Task bc984a27-c13c-4d8e-bb14-1c042818a49d: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913864::DEBUG::2013-11-20 03:15:50,016::task::495::TaskManager.Task::(_debug) Task bc984a27-c13c-4d8e-bb14-1c042818a49d: ref 1 aborting False
>Thread-15913864::WARNING::2013-11-20 03:15:50,018::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913864::DEBUG::2013-11-20 03:15:50,020::task::495::TaskManager.Task::(_debug) Task bc984a27-c13c-4d8e-bb14-1c042818a49d: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913864::DEBUG::2013-11-20 03:15:50,021::task::495::TaskManager.Task::(_debug) Task bc984a27-c13c-4d8e-bb14-1c042818a49d: moving from state preparing -> state finished
>Thread-15913864::DEBUG::2013-11-20 03:15:50,021::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913864::DEBUG::2013-11-20 03:15:50,022::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913864::DEBUG::2013-11-20 03:15:50,022::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913864::DEBUG::2013-11-20 03:15:50,023::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913864::DEBUG::2013-11-20 03:15:50,024::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913864::DEBUG::2013-11-20 03:15:50,024::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913864::DEBUG::2013-11-20 03:15:50,025::task::495::TaskManager.Task::(_debug) Task bc984a27-c13c-4d8e-bb14-1c042818a49d: ref 0 aborting False
>Thread-15913864::INFO::2013-11-20 03:15:50,025::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:15:51,578::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:15:52,131::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0691709 s, 14.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:15:54,149::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:15:54,651::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0610361 s, 16.8 MB/s\n'; <rc> = 0
>Thread-15913866::INFO::2013-11-20 03:15:54,941::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913866::DEBUG::2013-11-20 03:15:54,942::task::495::TaskManager.Task::(_debug) Task c188476a-c608-4ad9-a9b8-432e760e55a0: moving from state init -> state preparing
>Thread-15913866::DEBUG::2013-11-20 03:15:54,943::task::495::TaskManager.Task::(_debug) Task c188476a-c608-4ad9-a9b8-432e760e55a0: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000339984893799', 'lastCheck': 1384917352.2572141, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00194907188416', 'lastCheck': 1384917349.262284, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0119290351868', 'lastCheck': 1384917351.8535111, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00456094741821', 'lastCheck': 1384917351.8383951, 'valid': True, 'code': 0}}
>Thread-15913866::DEBUG::2013-11-20 03:15:54,945::task::495::TaskManager.Task::(_debug) Task c188476a-c608-4ad9-a9b8-432e760e55a0: moving from state preparing -> state finished
>Thread-15913866::DEBUG::2013-11-20 03:15:54,946::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913866::DEBUG::2013-11-20 03:15:54,947::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913866::DEBUG::2013-11-20 03:15:54,948::task::495::TaskManager.Task::(_debug) Task c188476a-c608-4ad9-a9b8-432e760e55a0: ref 0 aborting False
>Thread-15913866::INFO::2013-11-20 03:15:54,949::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000339984893799', 'lastCheck': 1384917352.2572141, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00194907188416', 'lastCheck': 1384917349.262284, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0119290351868', 'lastCheck': 1384917351.8535111, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00456094741821', 'lastCheck': 1384917351.8383951, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:15:56,661::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:15:57,158::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.073785 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:15:59,175::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:15:59,675::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0761059 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15913869::DEBUG::2013-11-20 03:16:00,320::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913869::INFO::2013-11-20 03:16:00,321::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913869::DEBUG::2013-11-20 03:16:00,322::task::495::TaskManager.Task::(_debug) Task 9ea3a740-ebfb-47c0-a42d-13d29df3c2bd: moving from state init -> state preparing
>Thread-15913869::DEBUG::2013-11-20 03:16:00,323::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913869::DEBUG::2013-11-20 03:16:00,324::task::495::TaskManager.Task::(_debug) Task 9ea3a740-ebfb-47c0-a42d-13d29df3c2bd: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913869::DEBUG::2013-11-20 03:16:00,324::task::495::TaskManager.Task::(_debug) Task 9ea3a740-ebfb-47c0-a42d-13d29df3c2bd: moving from state preparing -> state finished
>Thread-15913869::DEBUG::2013-11-20 03:16:00,325::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913869::DEBUG::2013-11-20 03:16:00,326::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913869::DEBUG::2013-11-20 03:16:00,326::task::495::TaskManager.Task::(_debug) Task 9ea3a740-ebfb-47c0-a42d-13d29df3c2bd: ref 0 aborting False
>Thread-15913869::INFO::2013-11-20 03:16:00,327::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913870::DEBUG::2013-11-20 03:16:00,332::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913870::INFO::2013-11-20 03:16:00,334::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913870::DEBUG::2013-11-20 03:16:00,335::task::495::TaskManager.Task::(_debug) Task f4a2f341-2913-4b95-a05b-e9463f2c5dc9: moving from state init -> state preparing
>Thread-15913870::DEBUG::2013-11-20 03:16:00,337::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`716c5780-a27d-40fc-bb11-5719bde85fd8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913870::DEBUG::2013-11-20 03:16:00,337::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913870::DEBUG::2013-11-20 03:16:00,339::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913870::DEBUG::2013-11-20 03:16:00,340::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`716c5780-a27d-40fc-bb11-5719bde85fd8`::Granted request
>Thread-15913870::DEBUG::2013-11-20 03:16:00,341::task::495::TaskManager.Task::(_debug) Task f4a2f341-2913-4b95-a05b-e9463f2c5dc9: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913870::DEBUG::2013-11-20 03:16:00,342::task::495::TaskManager.Task::(_debug) Task f4a2f341-2913-4b95-a05b-e9463f2c5dc9: ref 1 aborting False
>Thread-15913870::WARNING::2013-11-20 03:16:00,345::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913870::DEBUG::2013-11-20 03:16:00,346::task::495::TaskManager.Task::(_debug) Task f4a2f341-2913-4b95-a05b-e9463f2c5dc9: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913870::DEBUG::2013-11-20 03:16:00,347::task::495::TaskManager.Task::(_debug) Task f4a2f341-2913-4b95-a05b-e9463f2c5dc9: moving from state preparing -> state finished
>Thread-15913870::DEBUG::2013-11-20 03:16:00,347::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913870::DEBUG::2013-11-20 03:16:00,348::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913870::DEBUG::2013-11-20 03:16:00,348::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913870::DEBUG::2013-11-20 03:16:00,349::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913870::DEBUG::2013-11-20 03:16:00,349::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913870::DEBUG::2013-11-20 03:16:00,350::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913870::DEBUG::2013-11-20 03:16:00,350::task::495::TaskManager.Task::(_debug) Task f4a2f341-2913-4b95-a05b-e9463f2c5dc9: ref 0 aborting False
>Thread-15913870::INFO::2013-11-20 03:16:00,351::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:16:01,692::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:16:02,246::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0909309 s, 11.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:16:04,262::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:16:04,752::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0763729 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:16:06,766::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 03:16:07,019::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:16:07,022::task::495::TaskManager.Task::(_debug) Task e4fb38c5-7bed-4a6f-8b4c-b21c9a90ffd7: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:16:07,033::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`edb3a3cc-bafa-46af-aa3b-a812438cfcfb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:16:07,034::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:16:07,035::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:16:07,036::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`edb3a3cc-bafa-46af-aa3b-a812438cfcfb`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:16:07,037::task::495::TaskManager.Task::(_debug) Task e4fb38c5-7bed-4a6f-8b4c-b21c9a90ffd7: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:16:07,038::task::495::TaskManager.Task::(_debug) Task e4fb38c5-7bed-4a6f-8b4c-b21c9a90ffd7: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:16:07,039::task::495::TaskManager.Task::(_debug) Task e4fb38c5-7bed-4a6f-8b4c-b21c9a90ffd7: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:16:07,039::task::495::TaskManager.Task::(_debug) Task e4fb38c5-7bed-4a6f-8b4c-b21c9a90ffd7: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:16:07,040::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:16:07,041::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:16:07,041::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:16:07,042::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:16:07,043::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:16:07,043::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:16:07,044::task::495::TaskManager.Task::(_debug) Task e4fb38c5-7bed-4a6f-8b4c-b21c9a90ffd7: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:16:07,044::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:16:07,045::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:16:07,046::task::495::TaskManager.Task::(_debug) Task 2aa22ee1-42b6-4393-8c00-9ddb235819f0: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:16:07,047::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`3992ff58-7c3d-498c-bbe8-b6419b9d5c1b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:16:07,047::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:16:07,048::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:16:07,049::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`3992ff58-7c3d-498c-bbe8-b6419b9d5c1b`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:16:07,050::task::495::TaskManager.Task::(_debug) Task 2aa22ee1-42b6-4393-8c00-9ddb235819f0: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:16:07,050::task::495::TaskManager.Task::(_debug) Task 2aa22ee1-42b6-4393-8c00-9ddb235819f0: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:16:07,051::task::495::TaskManager.Task::(_debug) Task 2aa22ee1-42b6-4393-8c00-9ddb235819f0: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:16:07,052::task::495::TaskManager.Task::(_debug) Task 2aa22ee1-42b6-4393-8c00-9ddb235819f0: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:16:07,052::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:16:07,053::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:16:07,053::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:16:07,054::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:16:07,055::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:16:07,055::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:16:07,056::task::495::TaskManager.Task::(_debug) Task 2aa22ee1-42b6-4393-8c00-9ddb235819f0: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:16:07,056::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:16:07,057::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:16:07,058::task::495::TaskManager.Task::(_debug) Task 33e27059-3986-4e7e-b6fd-6806cddc73ae: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:16:07,059::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d6a2dee8-6028-414c-841a-9751bb2cc1d6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:16:07,059::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:16:07,060::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:16:07,061::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d6a2dee8-6028-414c-841a-9751bb2cc1d6`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:16:07,062::task::495::TaskManager.Task::(_debug) Task 33e27059-3986-4e7e-b6fd-6806cddc73ae: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:16:07,062::task::495::TaskManager.Task::(_debug) Task 33e27059-3986-4e7e-b6fd-6806cddc73ae: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:16:07,063::task::495::TaskManager.Task::(_debug) Task 33e27059-3986-4e7e-b6fd-6806cddc73ae: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:16:07,063::task::495::TaskManager.Task::(_debug) Task 33e27059-3986-4e7e-b6fd-6806cddc73ae: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:16:07,064::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:16:07,065::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:16:07,065::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:16:07,066::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:16:07,066::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:16:07,067::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:16:07,067::task::495::TaskManager.Task::(_debug) Task 33e27059-3986-4e7e-b6fd-6806cddc73ae: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:16:07,068::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:16:07,069::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:16:07,069::task::495::TaskManager.Task::(_debug) Task bc0c406d-e9c5-49cc-80d0-c883e784f85c: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:16:07,071::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`20087400-d9d0-413e-b41d-cc33fa442329`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:16:07,071::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:16:07,072::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:16:07,072::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`20087400-d9d0-413e-b41d-cc33fa442329`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:16:07,073::task::495::TaskManager.Task::(_debug) Task bc0c406d-e9c5-49cc-80d0-c883e784f85c: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:16:07,074::task::495::TaskManager.Task::(_debug) Task bc0c406d-e9c5-49cc-80d0-c883e784f85c: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:16:07,075::task::495::TaskManager.Task::(_debug) Task bc0c406d-e9c5-49cc-80d0-c883e784f85c: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:16:07,075::task::495::TaskManager.Task::(_debug) Task bc0c406d-e9c5-49cc-80d0-c883e784f85c: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:16:07,076::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:16:07,076::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:16:07,077::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:16:07,077::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:16:07,078::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:16:07,078::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:16:07,079::task::495::TaskManager.Task::(_debug) Task bc0c406d-e9c5-49cc-80d0-c883e784f85c: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:16:07,080::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:16:07,286::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0716916 s, 14.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:16:09,300::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:16:09,783::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0519543 s, 19.7 MB/s\n'; <rc> = 0
>Thread-15913874::INFO::2013-11-20 03:16:10,656::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913874::DEBUG::2013-11-20 03:16:10,657::task::495::TaskManager.Task::(_debug) Task bd47bb11-f156-4a30-b224-33c5eb1dbc4f: moving from state init -> state preparing
>Thread-15913874::DEBUG::2013-11-20 03:16:10,658::task::495::TaskManager.Task::(_debug) Task bd47bb11-f156-4a30-b224-33c5eb1dbc4f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00037693977356', 'lastCheck': 1384917362.369127, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00123000144958', 'lastCheck': 1384917369.505178, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00136208534241', 'lastCheck': 1384917361.90255, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00277614593506', 'lastCheck': 1384917361.8993981, 'valid': True, 'code': 0}}
>Thread-15913874::DEBUG::2013-11-20 03:16:10,659::task::495::TaskManager.Task::(_debug) Task bd47bb11-f156-4a30-b224-33c5eb1dbc4f: moving from state preparing -> state finished
>Thread-15913874::DEBUG::2013-11-20 03:16:10,659::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913874::DEBUG::2013-11-20 03:16:10,660::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913874::DEBUG::2013-11-20 03:16:10,660::task::495::TaskManager.Task::(_debug) Task bd47bb11-f156-4a30-b224-33c5eb1dbc4f: ref 0 aborting False
>Thread-15913874::INFO::2013-11-20 03:16:10,661::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00037693977356', 'lastCheck': 1384917362.369127, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00123000144958', 'lastCheck': 1384917369.505178, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00136208534241', 'lastCheck': 1384917361.90255, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00277614593506', 'lastCheck': 1384917361.8993981, 'valid': True, 'code': 0}}
>Thread-15913876::DEBUG::2013-11-20 03:16:10,694::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913876::INFO::2013-11-20 03:16:10,695::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913876::DEBUG::2013-11-20 03:16:10,696::task::495::TaskManager.Task::(_debug) Task 30b5b5be-da61-4b75-92db-c64bdd7c1c89: moving from state init -> state preparing
>Thread-15913876::DEBUG::2013-11-20 03:16:10,696::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913876::DEBUG::2013-11-20 03:16:10,697::task::495::TaskManager.Task::(_debug) Task 30b5b5be-da61-4b75-92db-c64bdd7c1c89: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913876::DEBUG::2013-11-20 03:16:10,697::task::495::TaskManager.Task::(_debug) Task 30b5b5be-da61-4b75-92db-c64bdd7c1c89: moving from state preparing -> state finished
>Thread-15913876::DEBUG::2013-11-20 03:16:10,698::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913876::DEBUG::2013-11-20 03:16:10,698::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913876::DEBUG::2013-11-20 03:16:10,699::task::495::TaskManager.Task::(_debug) Task 30b5b5be-da61-4b75-92db-c64bdd7c1c89: ref 0 aborting False
>Thread-15913876::INFO::2013-11-20 03:16:10,699::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913877::DEBUG::2013-11-20 03:16:10,704::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913877::INFO::2013-11-20 03:16:10,704::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913877::DEBUG::2013-11-20 03:16:10,705::task::495::TaskManager.Task::(_debug) Task 3d8ad319-9ebf-4d29-9048-b30900192f25: moving from state init -> state preparing
>Thread-15913877::DEBUG::2013-11-20 03:16:10,706::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`22764006-5fa9-4e11-8c60-8b90a90bff06`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913877::DEBUG::2013-11-20 03:16:10,707::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913877::DEBUG::2013-11-20 03:16:10,708::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913877::DEBUG::2013-11-20 03:16:10,708::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`22764006-5fa9-4e11-8c60-8b90a90bff06`::Granted request
>Thread-15913877::DEBUG::2013-11-20 03:16:10,717::task::495::TaskManager.Task::(_debug) Task 3d8ad319-9ebf-4d29-9048-b30900192f25: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913877::DEBUG::2013-11-20 03:16:10,718::task::495::TaskManager.Task::(_debug) Task 3d8ad319-9ebf-4d29-9048-b30900192f25: ref 1 aborting False
>Thread-15913877::WARNING::2013-11-20 03:16:10,720::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913877::DEBUG::2013-11-20 03:16:10,721::task::495::TaskManager.Task::(_debug) Task 3d8ad319-9ebf-4d29-9048-b30900192f25: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913877::DEBUG::2013-11-20 03:16:10,721::task::495::TaskManager.Task::(_debug) Task 3d8ad319-9ebf-4d29-9048-b30900192f25: moving from state preparing -> state finished
>Thread-15913877::DEBUG::2013-11-20 03:16:10,721::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913877::DEBUG::2013-11-20 03:16:10,722::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913877::DEBUG::2013-11-20 03:16:10,722::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913877::DEBUG::2013-11-20 03:16:10,723::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913877::DEBUG::2013-11-20 03:16:10,723::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913877::DEBUG::2013-11-20 03:16:10,723::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913877::DEBUG::2013-11-20 03:16:10,724::task::495::TaskManager.Task::(_debug) Task 3d8ad319-9ebf-4d29-9048-b30900192f25: ref 0 aborting False
>Thread-15913877::INFO::2013-11-20 03:16:10,724::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:16:11,792::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:16:12,283::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0590973 s, 17.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:16:14,292::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:16:14,845::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0754751 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:16:16,861::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 03:16:17,152::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:16:17,165::task::495::TaskManager.Task::(_debug) Task 14118175-771f-48bf-a2a2-b65a9c928280: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:16:17,171::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`556683fd-43f1-4cb9-874a-e8acedd5c062`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:16:17,172::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:16:17,174::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:16:17,174::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`556683fd-43f1-4cb9-874a-e8acedd5c062`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:16:17,176::task::495::TaskManager.Task::(_debug) Task 14118175-771f-48bf-a2a2-b65a9c928280: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:16:17,177::task::495::TaskManager.Task::(_debug) Task 14118175-771f-48bf-a2a2-b65a9c928280: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:16:17,178::task::495::TaskManager.Task::(_debug) Task 14118175-771f-48bf-a2a2-b65a9c928280: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:16:17,179::task::495::TaskManager.Task::(_debug) Task 14118175-771f-48bf-a2a2-b65a9c928280: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:16:17,180::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:16:17,181::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:16:17,182::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:16:17,183::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:16:17,184::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:16:17,184::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:16:17,185::task::495::TaskManager.Task::(_debug) Task 14118175-771f-48bf-a2a2-b65a9c928280: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:16:17,187::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:16:17,366::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0616544 s, 16.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:16:19,382::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:16:19,892::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0802159 s, 12.8 MB/s\n'; <rc> = 0
>Thread-15913881::DEBUG::2013-11-20 03:16:21,735::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913881::INFO::2013-11-20 03:16:21,736::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913881::DEBUG::2013-11-20 03:16:21,736::task::495::TaskManager.Task::(_debug) Task e6596153-2f46-4d27-bce7-5d4225730977: moving from state init -> state preparing
>Thread-15913881::DEBUG::2013-11-20 03:16:21,737::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913881::DEBUG::2013-11-20 03:16:21,737::task::495::TaskManager.Task::(_debug) Task e6596153-2f46-4d27-bce7-5d4225730977: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913881::DEBUG::2013-11-20 03:16:21,738::task::495::TaskManager.Task::(_debug) Task e6596153-2f46-4d27-bce7-5d4225730977: moving from state preparing -> state finished
>Thread-15913881::DEBUG::2013-11-20 03:16:21,738::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913881::DEBUG::2013-11-20 03:16:21,738::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913881::DEBUG::2013-11-20 03:16:21,739::task::495::TaskManager.Task::(_debug) Task e6596153-2f46-4d27-bce7-5d4225730977: ref 0 aborting False
>Thread-15913881::INFO::2013-11-20 03:16:21,739::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913882::DEBUG::2013-11-20 03:16:21,742::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913882::INFO::2013-11-20 03:16:21,743::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913882::DEBUG::2013-11-20 03:16:21,743::task::495::TaskManager.Task::(_debug) Task 9d4f4856-535f-4248-a932-6ed14701e3d1: moving from state init -> state preparing
>Thread-15913882::DEBUG::2013-11-20 03:16:21,744::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`64d445cb-9002-4981-9fe4-51dd6b545f67`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913882::DEBUG::2013-11-20 03:16:21,744::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913882::DEBUG::2013-11-20 03:16:21,744::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913882::DEBUG::2013-11-20 03:16:21,745::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`64d445cb-9002-4981-9fe4-51dd6b545f67`::Granted request
>Thread-15913882::DEBUG::2013-11-20 03:16:21,745::task::495::TaskManager.Task::(_debug) Task 9d4f4856-535f-4248-a932-6ed14701e3d1: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913882::DEBUG::2013-11-20 03:16:21,746::task::495::TaskManager.Task::(_debug) Task 9d4f4856-535f-4248-a932-6ed14701e3d1: ref 1 aborting False
>Thread-15913882::WARNING::2013-11-20 03:16:21,747::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913882::DEBUG::2013-11-20 03:16:21,747::task::495::TaskManager.Task::(_debug) Task 9d4f4856-535f-4248-a932-6ed14701e3d1: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913882::DEBUG::2013-11-20 03:16:21,748::task::495::TaskManager.Task::(_debug) Task 9d4f4856-535f-4248-a932-6ed14701e3d1: moving from state preparing -> state finished
>Thread-15913882::DEBUG::2013-11-20 03:16:21,748::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913882::DEBUG::2013-11-20 03:16:21,749::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913882::DEBUG::2013-11-20 03:16:21,749::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913882::DEBUG::2013-11-20 03:16:21,749::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913882::DEBUG::2013-11-20 03:16:21,750::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913882::DEBUG::2013-11-20 03:16:21,750::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913882::DEBUG::2013-11-20 03:16:21,750::task::495::TaskManager.Task::(_debug) Task 9d4f4856-535f-4248-a932-6ed14701e3d1: ref 0 aborting False
>Thread-15913882::INFO::2013-11-20 03:16:21,751::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:16:21,908::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:16:22,358::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0521314 s, 19.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:16:24,370::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:16:24,831::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0859019 s, 11.9 MB/s\n'; <rc> = 0
>Thread-15913884::INFO::2013-11-20 03:16:26,694::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913884::DEBUG::2013-11-20 03:16:26,695::task::495::TaskManager.Task::(_debug) Task d5ddeb8e-be38-479d-b0fb-045a7d7fd315: moving from state init -> state preparing
>Thread-15913884::DEBUG::2013-11-20 03:16:26,696::task::495::TaskManager.Task::(_debug) Task d5ddeb8e-be38-479d-b0fb-045a7d7fd315: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00020694732666', 'lastCheck': 1384917382.563817, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0135819911957', 'lastCheck': 1384917379.5805919, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00110793113708', 'lastCheck': 1384917382.1207089, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00570297241211', 'lastCheck': 1384917382.1182389, 'valid': True, 'code': 0}}
>Thread-15913884::DEBUG::2013-11-20 03:16:26,697::task::495::TaskManager.Task::(_debug) Task d5ddeb8e-be38-479d-b0fb-045a7d7fd315: moving from state preparing -> state finished
>Thread-15913884::DEBUG::2013-11-20 03:16:26,697::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913884::DEBUG::2013-11-20 03:16:26,698::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913884::DEBUG::2013-11-20 03:16:26,698::task::495::TaskManager.Task::(_debug) Task d5ddeb8e-be38-479d-b0fb-045a7d7fd315: ref 0 aborting False
>Thread-15913884::INFO::2013-11-20 03:16:26,699::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00020694732666', 'lastCheck': 1384917382.563817, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0135819911957', 'lastCheck': 1384917379.5805919, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00110793113708', 'lastCheck': 1384917382.1207089, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00570297241211', 'lastCheck': 1384917382.1182389, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:16:26,843::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:16:27,348::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0779604 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:16:29,361::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:16:29,827::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0483567 s, 21.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:16:31,838::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913887::DEBUG::2013-11-20 03:16:32,042::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913887::INFO::2013-11-20 03:16:32,045::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913887::DEBUG::2013-11-20 03:16:32,045::task::495::TaskManager.Task::(_debug) Task f3582340-8488-4755-8da9-43d8ed7ef0cc: moving from state init -> state preparing
>Thread-15913887::DEBUG::2013-11-20 03:16:32,047::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913887::DEBUG::2013-11-20 03:16:32,047::task::495::TaskManager.Task::(_debug) Task f3582340-8488-4755-8da9-43d8ed7ef0cc: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913887::DEBUG::2013-11-20 03:16:32,048::task::495::TaskManager.Task::(_debug) Task f3582340-8488-4755-8da9-43d8ed7ef0cc: moving from state preparing -> state finished
>Thread-15913887::DEBUG::2013-11-20 03:16:32,049::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913887::DEBUG::2013-11-20 03:16:32,049::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913887::DEBUG::2013-11-20 03:16:32,050::task::495::TaskManager.Task::(_debug) Task f3582340-8488-4755-8da9-43d8ed7ef0cc: ref 0 aborting False
>Thread-15913887::INFO::2013-11-20 03:16:32,050::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913888::DEBUG::2013-11-20 03:16:32,056::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913888::INFO::2013-11-20 03:16:32,057::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913888::DEBUG::2013-11-20 03:16:32,058::task::495::TaskManager.Task::(_debug) Task 353a193d-fb32-4a06-ad6a-92ddc332a4de: moving from state init -> state preparing
>Thread-15913888::DEBUG::2013-11-20 03:16:32,059::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`03be33e9-2a32-4e32-940d-8d1e6bd899b9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913888::DEBUG::2013-11-20 03:16:32,060::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913888::DEBUG::2013-11-20 03:16:32,061::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913888::DEBUG::2013-11-20 03:16:32,062::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`03be33e9-2a32-4e32-940d-8d1e6bd899b9`::Granted request
>Thread-15913888::DEBUG::2013-11-20 03:16:32,063::task::495::TaskManager.Task::(_debug) Task 353a193d-fb32-4a06-ad6a-92ddc332a4de: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913888::DEBUG::2013-11-20 03:16:32,064::task::495::TaskManager.Task::(_debug) Task 353a193d-fb32-4a06-ad6a-92ddc332a4de: ref 1 aborting False
>Thread-15913888::WARNING::2013-11-20 03:16:32,067::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913888::DEBUG::2013-11-20 03:16:32,068::task::495::TaskManager.Task::(_debug) Task 353a193d-fb32-4a06-ad6a-92ddc332a4de: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913888::DEBUG::2013-11-20 03:16:32,069::task::495::TaskManager.Task::(_debug) Task 353a193d-fb32-4a06-ad6a-92ddc332a4de: moving from state preparing -> state finished
>Thread-15913888::DEBUG::2013-11-20 03:16:32,070::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913888::DEBUG::2013-11-20 03:16:32,070::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913888::DEBUG::2013-11-20 03:16:32,071::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913888::DEBUG::2013-11-20 03:16:32,072::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913888::DEBUG::2013-11-20 03:16:32,073::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913888::DEBUG::2013-11-20 03:16:32,073::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913888::DEBUG::2013-11-20 03:16:32,074::task::495::TaskManager.Task::(_debug) Task 353a193d-fb32-4a06-ad6a-92ddc332a4de: ref 0 aborting False
>Thread-15913888::INFO::2013-11-20 03:16:32,075::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:16:32,285::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0475622 s, 21.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:16:34,299::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 03:16:34,634::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:16:34,635::task::495::TaskManager.Task::(_debug) Task 4a21c1e4-68ee-4c1c-b13a-dc6504e2d09e: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:16:34,636::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ddc18742-c0c5-4cdc-a343-5b1866e8378b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:16:34,637::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:16:34,638::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:16:34,639::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ddc18742-c0c5-4cdc-a343-5b1866e8378b`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:16:34,640::task::495::TaskManager.Task::(_debug) Task 4a21c1e4-68ee-4c1c-b13a-dc6504e2d09e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:16:34,641::task::495::TaskManager.Task::(_debug) Task 4a21c1e4-68ee-4c1c-b13a-dc6504e2d09e: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:16:34,641::task::495::TaskManager.Task::(_debug) Task 4a21c1e4-68ee-4c1c-b13a-dc6504e2d09e: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:16:34,642::task::495::TaskManager.Task::(_debug) Task 4a21c1e4-68ee-4c1c-b13a-dc6504e2d09e: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:16:34,642::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:16:34,643::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:16:34,643::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:16:34,644::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:16:34,644::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:16:34,645::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:16:34,645::task::495::TaskManager.Task::(_debug) Task 4a21c1e4-68ee-4c1c-b13a-dc6504e2d09e: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:16:34,646::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:16:34,728::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0578738 s, 17.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:16:36,737::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:16:37,158::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.057075 s, 17.9 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 03:16:37,167::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:16:37,168::task::495::TaskManager.Task::(_debug) Task 22181d1b-7404-441e-acda-040b9eaf2272: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:16:37,169::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`639884ad-767c-4a94-949d-188ab78845d0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:16:37,169::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:16:37,170::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:16:37,170::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`639884ad-767c-4a94-949d-188ab78845d0`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:16:37,172::task::495::TaskManager.Task::(_debug) Task 22181d1b-7404-441e-acda-040b9eaf2272: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:16:37,172::task::495::TaskManager.Task::(_debug) Task 22181d1b-7404-441e-acda-040b9eaf2272: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:16:37,173::task::495::TaskManager.Task::(_debug) Task 22181d1b-7404-441e-acda-040b9eaf2272: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:16:37,174::task::495::TaskManager.Task::(_debug) Task 22181d1b-7404-441e-acda-040b9eaf2272: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:16:37,174::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:16:37,175::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:16:37,175::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:16:37,176::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:16:37,176::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:16:37,176::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:16:37,177::task::495::TaskManager.Task::(_debug) Task 22181d1b-7404-441e-acda-040b9eaf2272: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:16:37,177::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:16:39,169::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:16:39,719::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0861105 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:16:41,730::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:16:42,333::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0566787 s, 18.1 MB/s\n'; <rc> = 0
>Thread-15913892::INFO::2013-11-20 03:16:42,409::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913892::DEBUG::2013-11-20 03:16:42,409::task::495::TaskManager.Task::(_debug) Task c61beebd-928c-404f-9fe4-890e4b63b57f: moving from state init -> state preparing
>Thread-15913892::DEBUG::2013-11-20 03:16:42,410::task::495::TaskManager.Task::(_debug) Task c61beebd-928c-404f-9fe4-890e4b63b57f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000488996505737', 'lastCheck': 1384917392.5773411, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00140595436096', 'lastCheck': 1384917399.6175461, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0110759735107', 'lastCheck': 1384917402.180006, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000577211380005', 'lastCheck': 1384917402.142386, 'valid': True, 'code': 0}}
>Thread-15913892::DEBUG::2013-11-20 03:16:42,410::task::495::TaskManager.Task::(_debug) Task c61beebd-928c-404f-9fe4-890e4b63b57f: moving from state preparing -> state finished
>Thread-15913892::DEBUG::2013-11-20 03:16:42,411::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913892::DEBUG::2013-11-20 03:16:42,411::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913892::DEBUG::2013-11-20 03:16:42,411::task::495::TaskManager.Task::(_debug) Task c61beebd-928c-404f-9fe4-890e4b63b57f: ref 0 aborting False
>Thread-15913892::INFO::2013-11-20 03:16:42,412::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000488996505737', 'lastCheck': 1384917392.5773411, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00140595436096', 'lastCheck': 1384917399.6175461, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0110759735107', 'lastCheck': 1384917402.180006, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000577211380005', 'lastCheck': 1384917402.142386, 'valid': True, 'code': 0}}
>Thread-15913894::DEBUG::2013-11-20 03:16:42,532::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913894::INFO::2013-11-20 03:16:42,533::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913894::DEBUG::2013-11-20 03:16:42,534::task::495::TaskManager.Task::(_debug) Task b0163baa-d37e-46d7-8b59-7599145abd50: moving from state init -> state preparing
>Thread-15913894::DEBUG::2013-11-20 03:16:42,535::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913894::DEBUG::2013-11-20 03:16:42,535::task::495::TaskManager.Task::(_debug) Task b0163baa-d37e-46d7-8b59-7599145abd50: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913894::DEBUG::2013-11-20 03:16:42,536::task::495::TaskManager.Task::(_debug) Task b0163baa-d37e-46d7-8b59-7599145abd50: moving from state preparing -> state finished
>Thread-15913894::DEBUG::2013-11-20 03:16:42,537::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913894::DEBUG::2013-11-20 03:16:42,537::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913894::DEBUG::2013-11-20 03:16:42,538::task::495::TaskManager.Task::(_debug) Task b0163baa-d37e-46d7-8b59-7599145abd50: ref 0 aborting False
>Thread-15913894::INFO::2013-11-20 03:16:42,538::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913895::DEBUG::2013-11-20 03:16:42,542::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913895::INFO::2013-11-20 03:16:42,543::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913895::DEBUG::2013-11-20 03:16:42,544::task::495::TaskManager.Task::(_debug) Task eb5629dc-6913-4b3c-901f-95c40d8d7ee7: moving from state init -> state preparing
>Thread-15913895::DEBUG::2013-11-20 03:16:42,545::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`cf70a26a-df6f-457e-966b-172f06f1f463`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913895::DEBUG::2013-11-20 03:16:42,546::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913895::DEBUG::2013-11-20 03:16:42,547::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913895::DEBUG::2013-11-20 03:16:42,547::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`cf70a26a-df6f-457e-966b-172f06f1f463`::Granted request
>Thread-15913895::DEBUG::2013-11-20 03:16:42,548::task::495::TaskManager.Task::(_debug) Task eb5629dc-6913-4b3c-901f-95c40d8d7ee7: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913895::DEBUG::2013-11-20 03:16:42,549::task::495::TaskManager.Task::(_debug) Task eb5629dc-6913-4b3c-901f-95c40d8d7ee7: ref 1 aborting False
>Thread-15913895::WARNING::2013-11-20 03:16:42,552::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913895::DEBUG::2013-11-20 03:16:42,553::task::495::TaskManager.Task::(_debug) Task eb5629dc-6913-4b3c-901f-95c40d8d7ee7: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913895::DEBUG::2013-11-20 03:16:42,554::task::495::TaskManager.Task::(_debug) Task eb5629dc-6913-4b3c-901f-95c40d8d7ee7: moving from state preparing -> state finished
>Thread-15913895::DEBUG::2013-11-20 03:16:42,554::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913895::DEBUG::2013-11-20 03:16:42,555::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913895::DEBUG::2013-11-20 03:16:42,556::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913895::DEBUG::2013-11-20 03:16:42,557::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15912392::INFO::2013-11-20 03:16:42,557::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15913895::DEBUG::2013-11-20 03:16:42,558::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:16:42,558::task::495::TaskManager.Task::(_debug) Task f80f0453-c70e-4381-881d-5cfc83d40e28: moving from state init -> state preparing
>Thread-15913895::DEBUG::2013-11-20 03:16:42,559::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:16:42,560::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8b94d1f0-6abf-47d0-91fa-867f41b5f854`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913895::DEBUG::2013-11-20 03:16:42,561::task::495::TaskManager.Task::(_debug) Task eb5629dc-6913-4b3c-901f-95c40d8d7ee7: ref 0 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:16:42,561::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913895::INFO::2013-11-20 03:16:42,562::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912392::DEBUG::2013-11-20 03:16:42,563::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:16:42,564::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8b94d1f0-6abf-47d0-91fa-867f41b5f854`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:16:42,565::task::495::TaskManager.Task::(_debug) Task f80f0453-c70e-4381-881d-5cfc83d40e28: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:16:42,565::task::495::TaskManager.Task::(_debug) Task f80f0453-c70e-4381-881d-5cfc83d40e28: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:16:42,566::task::495::TaskManager.Task::(_debug) Task f80f0453-c70e-4381-881d-5cfc83d40e28: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:16:42,566::task::495::TaskManager.Task::(_debug) Task f80f0453-c70e-4381-881d-5cfc83d40e28: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:16:42,566::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:16:42,567::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:16:42,567::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:16:42,567::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:16:42,568::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:16:42,569::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:16:42,569::task::495::TaskManager.Task::(_debug) Task f80f0453-c70e-4381-881d-5cfc83d40e28: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:16:42,569::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-42838::DEBUG::2013-11-20 03:16:42,586::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 03:16:43,220::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:16:44,348::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:16:44,783::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0493433 s, 20.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:16:46,793::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:16:47,337::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.136662 s, 7.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:16:49,350::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:16:49,871::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0857151 s, 11.9 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 03:16:50,651::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:16:50,652::task::495::TaskManager.Task::(_debug) Task 33fa4453-2466-498e-8e72-07d7777b6bf8: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:16:50,653::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b9020216-4837-4fb6-83a9-34049e47ab3b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:16:50,653::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:16:50,654::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:16:50,654::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b9020216-4837-4fb6-83a9-34049e47ab3b`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:16:50,655::task::495::TaskManager.Task::(_debug) Task 33fa4453-2466-498e-8e72-07d7777b6bf8: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:16:50,655::task::495::TaskManager.Task::(_debug) Task 33fa4453-2466-498e-8e72-07d7777b6bf8: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:16:50,656::task::495::TaskManager.Task::(_debug) Task 33fa4453-2466-498e-8e72-07d7777b6bf8: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:16:50,665::task::495::TaskManager.Task::(_debug) Task 33fa4453-2466-498e-8e72-07d7777b6bf8: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:16:50,665::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:16:50,666::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:16:50,666::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:16:50,667::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:16:50,667::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:16:50,667::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:16:50,668::task::495::TaskManager.Task::(_debug) Task 33fa4453-2466-498e-8e72-07d7777b6bf8: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:16:50,668::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:16:50,669::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:16:50,669::task::495::TaskManager.Task::(_debug) Task 4232c3a0-47ba-4dd8-8b2b-8cf877fb148a: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:16:50,670::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`17a2a034-bbc9-413e-a290-98beb574c1f4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:16:50,671::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:16:50,671::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:16:50,671::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`17a2a034-bbc9-413e-a290-98beb574c1f4`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:16:50,672::task::495::TaskManager.Task::(_debug) Task 4232c3a0-47ba-4dd8-8b2b-8cf877fb148a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:16:50,673::task::495::TaskManager.Task::(_debug) Task 4232c3a0-47ba-4dd8-8b2b-8cf877fb148a: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:16:50,673::task::495::TaskManager.Task::(_debug) Task 4232c3a0-47ba-4dd8-8b2b-8cf877fb148a: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:16:50,673::task::495::TaskManager.Task::(_debug) Task 4232c3a0-47ba-4dd8-8b2b-8cf877fb148a: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:16:50,674::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:16:50,674::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:16:50,675::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:16:50,675::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:16:50,675::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:16:50,676::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:16:50,676::task::495::TaskManager.Task::(_debug) Task 4232c3a0-47ba-4dd8-8b2b-8cf877fb148a: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:16:50,677::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:16:51,885::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:16:52,385::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0735257 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15913899::DEBUG::2013-11-20 03:16:53,345::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913899::INFO::2013-11-20 03:16:53,346::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913899::DEBUG::2013-11-20 03:16:53,346::task::495::TaskManager.Task::(_debug) Task 98cfbd4c-46fa-469c-948d-ebad4b514f8b: moving from state init -> state preparing
>Thread-15913899::DEBUG::2013-11-20 03:16:53,346::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913899::DEBUG::2013-11-20 03:16:53,347::task::495::TaskManager.Task::(_debug) Task 98cfbd4c-46fa-469c-948d-ebad4b514f8b: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913899::DEBUG::2013-11-20 03:16:53,347::task::495::TaskManager.Task::(_debug) Task 98cfbd4c-46fa-469c-948d-ebad4b514f8b: moving from state preparing -> state finished
>Thread-15913899::DEBUG::2013-11-20 03:16:53,347::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913899::DEBUG::2013-11-20 03:16:53,348::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913899::DEBUG::2013-11-20 03:16:53,348::task::495::TaskManager.Task::(_debug) Task 98cfbd4c-46fa-469c-948d-ebad4b514f8b: ref 0 aborting False
>Thread-15913899::INFO::2013-11-20 03:16:53,348::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913900::DEBUG::2013-11-20 03:16:53,374::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913900::INFO::2013-11-20 03:16:53,374::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913900::DEBUG::2013-11-20 03:16:53,375::task::495::TaskManager.Task::(_debug) Task e6aa34c2-b52a-46e3-825d-88c5ca4be965: moving from state init -> state preparing
>Thread-15913900::DEBUG::2013-11-20 03:16:53,376::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0e975c8b-0fa0-426e-9169-5f93d1ea49f3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913900::DEBUG::2013-11-20 03:16:53,376::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913900::DEBUG::2013-11-20 03:16:53,376::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913900::DEBUG::2013-11-20 03:16:53,377::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0e975c8b-0fa0-426e-9169-5f93d1ea49f3`::Granted request
>Thread-15913900::DEBUG::2013-11-20 03:16:53,377::task::495::TaskManager.Task::(_debug) Task e6aa34c2-b52a-46e3-825d-88c5ca4be965: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913900::DEBUG::2013-11-20 03:16:53,378::task::495::TaskManager.Task::(_debug) Task e6aa34c2-b52a-46e3-825d-88c5ca4be965: ref 1 aborting False
>Thread-15913900::WARNING::2013-11-20 03:16:53,379::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913900::DEBUG::2013-11-20 03:16:53,379::task::495::TaskManager.Task::(_debug) Task e6aa34c2-b52a-46e3-825d-88c5ca4be965: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913900::DEBUG::2013-11-20 03:16:53,380::task::495::TaskManager.Task::(_debug) Task e6aa34c2-b52a-46e3-825d-88c5ca4be965: moving from state preparing -> state finished
>Thread-15913900::DEBUG::2013-11-20 03:16:53,380::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913900::DEBUG::2013-11-20 03:16:53,381::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913900::DEBUG::2013-11-20 03:16:53,381::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913900::DEBUG::2013-11-20 03:16:53,382::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913900::DEBUG::2013-11-20 03:16:53,382::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913900::DEBUG::2013-11-20 03:16:53,382::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913900::DEBUG::2013-11-20 03:16:53,383::task::495::TaskManager.Task::(_debug) Task e6aa34c2-b52a-46e3-825d-88c5ca4be965: ref 0 aborting False
>Thread-15913900::INFO::2013-11-20 03:16:53,383::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:16:54,400::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:16:54,894::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0837026 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:16:56,909::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:16:57,375::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0720696 s, 14.2 MB/s\n'; <rc> = 0
>Thread-15913902::INFO::2013-11-20 03:16:58,311::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913902::DEBUG::2013-11-20 03:16:58,312::task::495::TaskManager.Task::(_debug) Task bac06feb-9145-4c33-959e-890ad2a0cf0d: moving from state init -> state preparing
>Thread-15913902::DEBUG::2013-11-20 03:16:58,312::task::495::TaskManager.Task::(_debug) Task bac06feb-9145-4c33-959e-890ad2a0cf0d: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000300884246826', 'lastCheck': 1384917413.2335579, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00216507911682', 'lastCheck': 1384917409.7565651, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00174403190613', 'lastCheck': 1384917412.197372, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000508069992065', 'lastCheck': 1384917412.1540329, 'valid': True, 'code': 0}}
>Thread-15913902::DEBUG::2013-11-20 03:16:58,313::task::495::TaskManager.Task::(_debug) Task bac06feb-9145-4c33-959e-890ad2a0cf0d: moving from state preparing -> state finished
>Thread-15913902::DEBUG::2013-11-20 03:16:58,313::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913902::DEBUG::2013-11-20 03:16:58,314::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913902::DEBUG::2013-11-20 03:16:58,314::task::495::TaskManager.Task::(_debug) Task bac06feb-9145-4c33-959e-890ad2a0cf0d: ref 0 aborting False
>Thread-15913902::INFO::2013-11-20 03:16:58,314::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000300884246826', 'lastCheck': 1384917413.2335579, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00216507911682', 'lastCheck': 1384917409.7565651, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00174403190613', 'lastCheck': 1384917412.197372, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000508069992065', 'lastCheck': 1384917412.1540329, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:16:59,390::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:16:59,929::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0787665 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:17:01,944::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:17:02,335::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0452756 s, 22.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:17:04,344::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913905::DEBUG::2013-11-20 03:17:04,500::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913905::INFO::2013-11-20 03:17:04,502::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913905::DEBUG::2013-11-20 03:17:04,503::task::495::TaskManager.Task::(_debug) Task 9a385d3b-1895-48dc-9075-1f680e21c556: moving from state init -> state preparing
>Thread-15913905::DEBUG::2013-11-20 03:17:04,504::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913905::DEBUG::2013-11-20 03:17:04,504::task::495::TaskManager.Task::(_debug) Task 9a385d3b-1895-48dc-9075-1f680e21c556: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913905::DEBUG::2013-11-20 03:17:04,505::task::495::TaskManager.Task::(_debug) Task 9a385d3b-1895-48dc-9075-1f680e21c556: moving from state preparing -> state finished
>Thread-15913905::DEBUG::2013-11-20 03:17:04,505::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913905::DEBUG::2013-11-20 03:17:04,506::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913905::DEBUG::2013-11-20 03:17:04,506::task::495::TaskManager.Task::(_debug) Task 9a385d3b-1895-48dc-9075-1f680e21c556: ref 0 aborting False
>Thread-15913905::INFO::2013-11-20 03:17:04,507::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913906::DEBUG::2013-11-20 03:17:04,512::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913906::INFO::2013-11-20 03:17:04,513::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913906::DEBUG::2013-11-20 03:17:04,513::task::495::TaskManager.Task::(_debug) Task b59342f5-a065-4a0f-9a41-a4f5e9bcae81: moving from state init -> state preparing
>Thread-15913906::DEBUG::2013-11-20 03:17:04,514::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9a2eaad9-4869-4533-97b3-075975989e53`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913906::DEBUG::2013-11-20 03:17:04,515::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913906::DEBUG::2013-11-20 03:17:04,515::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913906::DEBUG::2013-11-20 03:17:04,516::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9a2eaad9-4869-4533-97b3-075975989e53`::Granted request
>Thread-15913906::DEBUG::2013-11-20 03:17:04,517::task::495::TaskManager.Task::(_debug) Task b59342f5-a065-4a0f-9a41-a4f5e9bcae81: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913906::DEBUG::2013-11-20 03:17:04,517::task::495::TaskManager.Task::(_debug) Task b59342f5-a065-4a0f-9a41-a4f5e9bcae81: ref 1 aborting False
>Thread-15913906::WARNING::2013-11-20 03:17:04,519::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913906::DEBUG::2013-11-20 03:17:04,519::task::495::TaskManager.Task::(_debug) Task b59342f5-a065-4a0f-9a41-a4f5e9bcae81: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913906::DEBUG::2013-11-20 03:17:04,520::task::495::TaskManager.Task::(_debug) Task b59342f5-a065-4a0f-9a41-a4f5e9bcae81: moving from state preparing -> state finished
>Thread-15913906::DEBUG::2013-11-20 03:17:04,520::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913906::DEBUG::2013-11-20 03:17:04,521::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913906::DEBUG::2013-11-20 03:17:04,521::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913906::DEBUG::2013-11-20 03:17:04,521::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913906::DEBUG::2013-11-20 03:17:04,522::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913906::DEBUG::2013-11-20 03:17:04,522::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913906::DEBUG::2013-11-20 03:17:04,523::task::495::TaskManager.Task::(_debug) Task b59342f5-a065-4a0f-9a41-a4f5e9bcae81: ref 0 aborting False
>Thread-15913906::INFO::2013-11-20 03:17:04,523::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:17:04,824::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0679863 s, 15.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:17:06,844::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:17:07,241::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0455438 s, 22.5 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 03:17:07,621::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:17:07,621::task::495::TaskManager.Task::(_debug) Task 07b2d93d-3295-43d1-ba45-cb810bb0308a: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:17:07,637::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a48b6f5d-6bce-4942-ac83-c0f39e45bd93`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:17:07,638::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:17:07,639::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:17:07,639::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a48b6f5d-6bce-4942-ac83-c0f39e45bd93`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:17:07,640::task::495::TaskManager.Task::(_debug) Task 07b2d93d-3295-43d1-ba45-cb810bb0308a: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:17:07,641::task::495::TaskManager.Task::(_debug) Task 07b2d93d-3295-43d1-ba45-cb810bb0308a: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:17:07,641::task::495::TaskManager.Task::(_debug) Task 07b2d93d-3295-43d1-ba45-cb810bb0308a: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:17:07,642::task::495::TaskManager.Task::(_debug) Task 07b2d93d-3295-43d1-ba45-cb810bb0308a: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:17:07,642::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:17:07,642::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:17:07,643::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:17:07,643::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:17:07,644::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:17:07,644::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:17:07,644::task::495::TaskManager.Task::(_debug) Task 07b2d93d-3295-43d1-ba45-cb810bb0308a: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:17:07,645::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:17:07,645::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:17:07,645::task::495::TaskManager.Task::(_debug) Task dd9b16bb-0f13-481b-8ed4-0d73b1055510: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:17:07,646::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`ef209a62-fda6-4c61-8449-e29e95bfa6d8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:17:07,646::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:17:07,647::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:17:07,647::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`ef209a62-fda6-4c61-8449-e29e95bfa6d8`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:17:07,648::task::495::TaskManager.Task::(_debug) Task dd9b16bb-0f13-481b-8ed4-0d73b1055510: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:17:07,648::task::495::TaskManager.Task::(_debug) Task dd9b16bb-0f13-481b-8ed4-0d73b1055510: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:17:07,649::task::495::TaskManager.Task::(_debug) Task dd9b16bb-0f13-481b-8ed4-0d73b1055510: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:17:07,649::task::495::TaskManager.Task::(_debug) Task dd9b16bb-0f13-481b-8ed4-0d73b1055510: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:17:07,649::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:17:07,650::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:17:07,650::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:17:07,650::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:17:07,651::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:17:07,651::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:17:07,651::task::495::TaskManager.Task::(_debug) Task dd9b16bb-0f13-481b-8ed4-0d73b1055510: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:17:07,652::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:17:07,652::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:17:07,653::task::495::TaskManager.Task::(_debug) Task b7306902-2dc1-4fb4-b42d-cb28fe856132: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:17:07,653::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`06b0a5a5-c7a1-4cb5-b4cf-6261537b8313`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:17:07,654::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:17:07,654::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:17:07,654::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`06b0a5a5-c7a1-4cb5-b4cf-6261537b8313`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:17:07,655::task::495::TaskManager.Task::(_debug) Task b7306902-2dc1-4fb4-b42d-cb28fe856132: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:17:07,655::task::495::TaskManager.Task::(_debug) Task b7306902-2dc1-4fb4-b42d-cb28fe856132: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:17:07,655::task::495::TaskManager.Task::(_debug) Task b7306902-2dc1-4fb4-b42d-cb28fe856132: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:17:07,656::task::495::TaskManager.Task::(_debug) Task b7306902-2dc1-4fb4-b42d-cb28fe856132: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:17:07,656::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:17:07,657::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:17:07,657::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:17:07,657::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:17:07,658::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:17:07,658::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:17:07,658::task::495::TaskManager.Task::(_debug) Task b7306902-2dc1-4fb4-b42d-cb28fe856132: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:17:07,659::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:17:07,659::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:17:07,659::task::495::TaskManager.Task::(_debug) Task 455f9286-30d3-4212-896a-8d85e3613f65: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:17:07,660::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e75de393-321b-49e4-9922-fab82b2963c1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:17:07,660::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:17:07,661::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:17:07,661::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e75de393-321b-49e4-9922-fab82b2963c1`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:17:07,662::task::495::TaskManager.Task::(_debug) Task 455f9286-30d3-4212-896a-8d85e3613f65: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:17:07,662::task::495::TaskManager.Task::(_debug) Task 455f9286-30d3-4212-896a-8d85e3613f65: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:17:07,662::task::495::TaskManager.Task::(_debug) Task 455f9286-30d3-4212-896a-8d85e3613f65: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:17:07,663::task::495::TaskManager.Task::(_debug) Task 455f9286-30d3-4212-896a-8d85e3613f65: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:17:07,663::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:17:07,663::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:17:07,664::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:17:07,664::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:17:07,664::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:17:07,665::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:17:07,665::task::495::TaskManager.Task::(_debug) Task 455f9286-30d3-4212-896a-8d85e3613f65: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:17:07,665::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:17:09,256::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:17:09,781::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0790301 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:17:11,804::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:17:12,231::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0461483 s, 22.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:17:14,240::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:17:14,751::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0835102 s, 12.3 MB/s\n'; <rc> = 0
>Thread-15913910::INFO::2013-11-20 03:17:15,036::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913910::DEBUG::2013-11-20 03:17:15,038::task::495::TaskManager.Task::(_debug) Task 73940958-d6a1-4eeb-8c92-03dd1ac77809: moving from state init -> state preparing
>Thread-15913910::DEBUG::2013-11-20 03:17:15,039::task::495::TaskManager.Task::(_debug) Task 73940958-d6a1-4eeb-8c92-03dd1ac77809: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000298023223877', 'lastCheck': 1384917433.2552011, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00583696365356', 'lastCheck': 1384917429.803071, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00137591362', 'lastCheck': 1384917432.246809, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000648021697998', 'lastCheck': 1384917432.1774869, 'valid': True, 'code': 0}}
>Thread-15913910::DEBUG::2013-11-20 03:17:15,041::task::495::TaskManager.Task::(_debug) Task 73940958-d6a1-4eeb-8c92-03dd1ac77809: moving from state preparing -> state finished
>Thread-15913910::DEBUG::2013-11-20 03:17:15,041::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913910::DEBUG::2013-11-20 03:17:15,042::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913910::DEBUG::2013-11-20 03:17:15,043::task::495::TaskManager.Task::(_debug) Task 73940958-d6a1-4eeb-8c92-03dd1ac77809: ref 0 aborting False
>Thread-15913910::INFO::2013-11-20 03:17:15,044::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000298023223877', 'lastCheck': 1384917433.2552011, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00583696365356', 'lastCheck': 1384917429.803071, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00137591362', 'lastCheck': 1384917432.246809, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000648021697998', 'lastCheck': 1384917432.1774869, 'valid': True, 'code': 0}}
>Thread-15913912::DEBUG::2013-11-20 03:17:15,213::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913912::INFO::2013-11-20 03:17:15,214::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913912::DEBUG::2013-11-20 03:17:15,215::task::495::TaskManager.Task::(_debug) Task a292b767-e240-4e06-8b4a-7380916feac6: moving from state init -> state preparing
>Thread-15913912::DEBUG::2013-11-20 03:17:15,216::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913912::DEBUG::2013-11-20 03:17:15,216::task::495::TaskManager.Task::(_debug) Task a292b767-e240-4e06-8b4a-7380916feac6: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913912::DEBUG::2013-11-20 03:17:15,217::task::495::TaskManager.Task::(_debug) Task a292b767-e240-4e06-8b4a-7380916feac6: moving from state preparing -> state finished
>Thread-15913912::DEBUG::2013-11-20 03:17:15,218::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913912::DEBUG::2013-11-20 03:17:15,219::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913912::DEBUG::2013-11-20 03:17:15,219::task::495::TaskManager.Task::(_debug) Task a292b767-e240-4e06-8b4a-7380916feac6: ref 0 aborting False
>Thread-15913912::INFO::2013-11-20 03:17:15,220::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913913::DEBUG::2013-11-20 03:17:15,224::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913913::INFO::2013-11-20 03:17:15,225::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913913::DEBUG::2013-11-20 03:17:15,226::task::495::TaskManager.Task::(_debug) Task 235d4292-374d-46b1-88ea-1359b5503d73: moving from state init -> state preparing
>Thread-15913913::DEBUG::2013-11-20 03:17:15,227::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1168bf05-b7be-4394-b844-cbc476d0983a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913913::DEBUG::2013-11-20 03:17:15,228::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913913::DEBUG::2013-11-20 03:17:15,229::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913913::DEBUG::2013-11-20 03:17:15,229::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1168bf05-b7be-4394-b844-cbc476d0983a`::Granted request
>Thread-15913913::DEBUG::2013-11-20 03:17:15,231::task::495::TaskManager.Task::(_debug) Task 235d4292-374d-46b1-88ea-1359b5503d73: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913913::DEBUG::2013-11-20 03:17:15,231::task::495::TaskManager.Task::(_debug) Task 235d4292-374d-46b1-88ea-1359b5503d73: ref 1 aborting False
>Thread-15913913::WARNING::2013-11-20 03:17:15,233::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913913::DEBUG::2013-11-20 03:17:15,234::task::495::TaskManager.Task::(_debug) Task 235d4292-374d-46b1-88ea-1359b5503d73: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913913::DEBUG::2013-11-20 03:17:15,235::task::495::TaskManager.Task::(_debug) Task 235d4292-374d-46b1-88ea-1359b5503d73: moving from state preparing -> state finished
>Thread-15913913::DEBUG::2013-11-20 03:17:15,236::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913913::DEBUG::2013-11-20 03:17:15,236::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913913::DEBUG::2013-11-20 03:17:15,237::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913913::DEBUG::2013-11-20 03:17:15,238::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913913::DEBUG::2013-11-20 03:17:15,238::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913913::DEBUG::2013-11-20 03:17:15,239::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913913::DEBUG::2013-11-20 03:17:15,239::task::495::TaskManager.Task::(_debug) Task 235d4292-374d-46b1-88ea-1359b5503d73: ref 0 aborting False
>Thread-15913913::INFO::2013-11-20 03:17:15,240::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:17:16,764::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:17:17,296::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0956234 s, 10.7 MB/s\n'; <rc> = 0
>Thread-15913639::INFO::2013-11-20 03:17:17,658::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:17:17,660::task::495::TaskManager.Task::(_debug) Task b47e925c-6d94-4228-8204-dc34fba051c7: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:17:17,669::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`660a27db-bd59-47d0-a5e6-b18ae2545b92`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:17:17,670::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:17:17,671::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:17:17,672::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`660a27db-bd59-47d0-a5e6-b18ae2545b92`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:17:17,674::task::495::TaskManager.Task::(_debug) Task b47e925c-6d94-4228-8204-dc34fba051c7: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:17:17,674::task::495::TaskManager.Task::(_debug) Task b47e925c-6d94-4228-8204-dc34fba051c7: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:17:17,676::task::495::TaskManager.Task::(_debug) Task b47e925c-6d94-4228-8204-dc34fba051c7: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:17:17,676::task::495::TaskManager.Task::(_debug) Task b47e925c-6d94-4228-8204-dc34fba051c7: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:17:17,677::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:17:17,678::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:17:17,678::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:17:17,679::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:17:17,680::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:17:17,680::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:17:17,681::task::495::TaskManager.Task::(_debug) Task b47e925c-6d94-4228-8204-dc34fba051c7: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:17:17,681::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:17:19,305::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:17:19,846::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0783429 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:17:21,859::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:17:22,367::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0555871 s, 18.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:17:24,381::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:17:24,962::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0755565 s, 13.6 MB/s\n'; <rc> = 0
>Thread-15913917::DEBUG::2013-11-20 03:17:25,679::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913917::INFO::2013-11-20 03:17:25,681::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913917::DEBUG::2013-11-20 03:17:25,682::task::495::TaskManager.Task::(_debug) Task b4a6040d-1196-4e1a-9939-6d9a563e0280: moving from state init -> state preparing
>Thread-15913917::DEBUG::2013-11-20 03:17:25,683::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913917::DEBUG::2013-11-20 03:17:25,684::task::495::TaskManager.Task::(_debug) Task b4a6040d-1196-4e1a-9939-6d9a563e0280: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913917::DEBUG::2013-11-20 03:17:25,685::task::495::TaskManager.Task::(_debug) Task b4a6040d-1196-4e1a-9939-6d9a563e0280: moving from state preparing -> state finished
>Thread-15913917::DEBUG::2013-11-20 03:17:25,686::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913917::DEBUG::2013-11-20 03:17:25,687::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913917::DEBUG::2013-11-20 03:17:25,688::task::495::TaskManager.Task::(_debug) Task b4a6040d-1196-4e1a-9939-6d9a563e0280: ref 0 aborting False
>Thread-15913917::INFO::2013-11-20 03:17:25,688::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913918::DEBUG::2013-11-20 03:17:25,699::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913918::INFO::2013-11-20 03:17:25,701::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913918::DEBUG::2013-11-20 03:17:25,701::task::495::TaskManager.Task::(_debug) Task 9af1b9db-99f1-4847-b8de-fc1e64abea00: moving from state init -> state preparing
>Thread-15913918::DEBUG::2013-11-20 03:17:25,703::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`fc291d07-4779-4879-a0e5-bc866d98d2c1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913918::DEBUG::2013-11-20 03:17:25,703::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913918::DEBUG::2013-11-20 03:17:25,704::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913918::DEBUG::2013-11-20 03:17:25,705::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`fc291d07-4779-4879-a0e5-bc866d98d2c1`::Granted request
>Thread-15913918::DEBUG::2013-11-20 03:17:25,706::task::495::TaskManager.Task::(_debug) Task 9af1b9db-99f1-4847-b8de-fc1e64abea00: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913918::DEBUG::2013-11-20 03:17:25,707::task::495::TaskManager.Task::(_debug) Task 9af1b9db-99f1-4847-b8de-fc1e64abea00: ref 1 aborting False
>Thread-15913918::WARNING::2013-11-20 03:17:25,709::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913918::DEBUG::2013-11-20 03:17:25,710::task::495::TaskManager.Task::(_debug) Task 9af1b9db-99f1-4847-b8de-fc1e64abea00: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913918::DEBUG::2013-11-20 03:17:25,711::task::495::TaskManager.Task::(_debug) Task 9af1b9db-99f1-4847-b8de-fc1e64abea00: moving from state preparing -> state finished
>Thread-15913918::DEBUG::2013-11-20 03:17:25,711::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913918::DEBUG::2013-11-20 03:17:25,712::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913918::DEBUG::2013-11-20 03:17:25,712::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913918::DEBUG::2013-11-20 03:17:25,713::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913918::DEBUG::2013-11-20 03:17:25,713::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913918::DEBUG::2013-11-20 03:17:25,714::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913918::DEBUG::2013-11-20 03:17:25,715::task::495::TaskManager.Task::(_debug) Task 9af1b9db-99f1-4847-b8de-fc1e64abea00: ref 0 aborting False
>Thread-15913918::INFO::2013-11-20 03:17:25,715::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:17:26,975::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:17:27,483::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0814004 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:17:29,499::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:17:30,065::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0784806 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15913920::INFO::2013-11-20 03:17:30,857::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913920::DEBUG::2013-11-20 03:17:30,857::task::495::TaskManager.Task::(_debug) Task eac4ceb4-db08-4a7c-aa86-56c956e26358: moving from state init -> state preparing
>Thread-15913920::DEBUG::2013-11-20 03:17:30,858::task::495::TaskManager.Task::(_debug) Task eac4ceb4-db08-4a7c-aa86-56c956e26358: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000340938568115', 'lastCheck': 1384917443.511121, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00126314163208', 'lastCheck': 1384917450.1647019, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0016782283783', 'lastCheck': 1384917442.5143099, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000478029251099', 'lastCheck': 1384917442.190304, 'valid': True, 'code': 0}}
>Thread-15913920::DEBUG::2013-11-20 03:17:30,858::task::495::TaskManager.Task::(_debug) Task eac4ceb4-db08-4a7c-aa86-56c956e26358: moving from state preparing -> state finished
>Thread-15913920::DEBUG::2013-11-20 03:17:30,859::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913920::DEBUG::2013-11-20 03:17:30,859::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913920::DEBUG::2013-11-20 03:17:30,860::task::495::TaskManager.Task::(_debug) Task eac4ceb4-db08-4a7c-aa86-56c956e26358: ref 0 aborting False
>Thread-15913920::INFO::2013-11-20 03:17:30,860::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000340938568115', 'lastCheck': 1384917443.511121, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00126314163208', 'lastCheck': 1384917450.1647019, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0016782283783', 'lastCheck': 1384917442.5143099, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000478029251099', 'lastCheck': 1384917442.190304, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:17:32,081::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:17:32,631::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.081181 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:17:34,647::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:17:35,145::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0750451 s, 13.6 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 03:17:35,202::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:17:35,203::task::495::TaskManager.Task::(_debug) Task a26ddb7e-7e7d-4a7a-affd-ac2317f45474: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:17:35,205::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d9be0dd8-d847-4a3c-8071-1c78cd6230dc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:17:35,205::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:17:35,206::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:17:35,207::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d9be0dd8-d847-4a3c-8071-1c78cd6230dc`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:17:35,208::task::495::TaskManager.Task::(_debug) Task a26ddb7e-7e7d-4a7a-affd-ac2317f45474: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:17:35,208::task::495::TaskManager.Task::(_debug) Task a26ddb7e-7e7d-4a7a-affd-ac2317f45474: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:17:35,209::task::495::TaskManager.Task::(_debug) Task a26ddb7e-7e7d-4a7a-affd-ac2317f45474: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:17:35,209::task::495::TaskManager.Task::(_debug) Task a26ddb7e-7e7d-4a7a-affd-ac2317f45474: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:17:35,210::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:17:35,210::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:17:35,211::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:17:35,211::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:17:35,212::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:17:35,212::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:17:35,213::task::495::TaskManager.Task::(_debug) Task a26ddb7e-7e7d-4a7a-affd-ac2317f45474: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:17:35,213::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913923::DEBUG::2013-11-20 03:17:35,956::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913923::INFO::2013-11-20 03:17:35,958::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913923::DEBUG::2013-11-20 03:17:35,959::task::495::TaskManager.Task::(_debug) Task b17a5d63-4b78-43ef-8d48-c73a976bbf88: moving from state init -> state preparing
>Thread-15913923::DEBUG::2013-11-20 03:17:35,960::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913923::DEBUG::2013-11-20 03:17:35,960::task::495::TaskManager.Task::(_debug) Task b17a5d63-4b78-43ef-8d48-c73a976bbf88: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913923::DEBUG::2013-11-20 03:17:35,961::task::495::TaskManager.Task::(_debug) Task b17a5d63-4b78-43ef-8d48-c73a976bbf88: moving from state preparing -> state finished
>Thread-15913923::DEBUG::2013-11-20 03:17:35,962::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913923::DEBUG::2013-11-20 03:17:35,963::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913923::DEBUG::2013-11-20 03:17:35,963::task::495::TaskManager.Task::(_debug) Task b17a5d63-4b78-43ef-8d48-c73a976bbf88: ref 0 aborting False
>Thread-15913923::INFO::2013-11-20 03:17:35,964::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913924::DEBUG::2013-11-20 03:17:35,969::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913924::INFO::2013-11-20 03:17:35,970::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913924::DEBUG::2013-11-20 03:17:35,971::task::495::TaskManager.Task::(_debug) Task 8dc1dea2-fb90-4fac-ab43-4a89a4207bf3: moving from state init -> state preparing
>Thread-15913924::DEBUG::2013-11-20 03:17:35,972::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5e7a3b1b-8610-441c-8b93-990403b585c1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913924::DEBUG::2013-11-20 03:17:35,973::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913924::DEBUG::2013-11-20 03:17:35,973::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913924::DEBUG::2013-11-20 03:17:35,974::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5e7a3b1b-8610-441c-8b93-990403b585c1`::Granted request
>Thread-15913924::DEBUG::2013-11-20 03:17:35,975::task::495::TaskManager.Task::(_debug) Task 8dc1dea2-fb90-4fac-ab43-4a89a4207bf3: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913924::DEBUG::2013-11-20 03:17:35,975::task::495::TaskManager.Task::(_debug) Task 8dc1dea2-fb90-4fac-ab43-4a89a4207bf3: ref 1 aborting False
>Thread-15913924::WARNING::2013-11-20 03:17:35,977::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913924::DEBUG::2013-11-20 03:17:35,978::task::495::TaskManager.Task::(_debug) Task 8dc1dea2-fb90-4fac-ab43-4a89a4207bf3: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913924::DEBUG::2013-11-20 03:17:35,979::task::495::TaskManager.Task::(_debug) Task 8dc1dea2-fb90-4fac-ab43-4a89a4207bf3: moving from state preparing -> state finished
>Thread-15913924::DEBUG::2013-11-20 03:17:35,980::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913924::DEBUG::2013-11-20 03:17:35,980::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913924::DEBUG::2013-11-20 03:17:35,981::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913924::DEBUG::2013-11-20 03:17:35,981::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913924::DEBUG::2013-11-20 03:17:35,982::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913924::DEBUG::2013-11-20 03:17:35,982::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913924::DEBUG::2013-11-20 03:17:35,983::task::495::TaskManager.Task::(_debug) Task 8dc1dea2-fb90-4fac-ab43-4a89a4207bf3: ref 0 aborting False
>Thread-15913924::INFO::2013-11-20 03:17:35,984::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:17:37,162::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:17:37,685::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0749641 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 03:17:37,741::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:17:37,741::task::495::TaskManager.Task::(_debug) Task 2decd3dd-cd0b-4e91-95a2-8c68dd2e5b38: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:17:37,743::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`918832c0-7e5a-4db0-9ee3-75047ec0ba06`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:17:37,744::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:17:37,744::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:17:37,745::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`918832c0-7e5a-4db0-9ee3-75047ec0ba06`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:17:37,746::task::495::TaskManager.Task::(_debug) Task 2decd3dd-cd0b-4e91-95a2-8c68dd2e5b38: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:17:37,746::task::495::TaskManager.Task::(_debug) Task 2decd3dd-cd0b-4e91-95a2-8c68dd2e5b38: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:17:37,747::task::495::TaskManager.Task::(_debug) Task 2decd3dd-cd0b-4e91-95a2-8c68dd2e5b38: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:17:37,748::task::495::TaskManager.Task::(_debug) Task 2decd3dd-cd0b-4e91-95a2-8c68dd2e5b38: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:17:37,748::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:17:37,748::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:17:37,749::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:17:37,750::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:17:37,750::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:17:37,751::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:17:37,752::task::495::TaskManager.Task::(_debug) Task 2decd3dd-cd0b-4e91-95a2-8c68dd2e5b38: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:17:37,752::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:17:39,700::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:17:40,239::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.091199 s, 11.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:17:42,256::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:17:42,777::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0775109 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 03:17:42,990::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:17:42,991::task::495::TaskManager.Task::(_debug) Task 987bf7e3-5e4a-4387-b04b-737161dad3ac: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:17:42,993::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d439fdd6-c6a1-4ef8-8c86-901cbdafb3fd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:17:42,993::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:17:42,994::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:17:42,995::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d439fdd6-c6a1-4ef8-8c86-901cbdafb3fd`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:17:42,996::task::495::TaskManager.Task::(_debug) Task 987bf7e3-5e4a-4387-b04b-737161dad3ac: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:17:42,997::task::495::TaskManager.Task::(_debug) Task 987bf7e3-5e4a-4387-b04b-737161dad3ac: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:17:42,997::task::495::TaskManager.Task::(_debug) Task 987bf7e3-5e4a-4387-b04b-737161dad3ac: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:17:42,998::task::495::TaskManager.Task::(_debug) Task 987bf7e3-5e4a-4387-b04b-737161dad3ac: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:17:42,998::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:17:42,999::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:17:43,000::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:17:43,001::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:17:43,001::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:17:43,002::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:17:43,002::task::495::TaskManager.Task::(_debug) Task 987bf7e3-5e4a-4387-b04b-737161dad3ac: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:17:43,003::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:17:44,793::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:17:45,291::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0609346 s, 16.8 MB/s\n'; <rc> = 0
>Thread-15913928::DEBUG::2013-11-20 03:17:46,366::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913928::INFO::2013-11-20 03:17:46,368::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913928::DEBUG::2013-11-20 03:17:46,368::task::495::TaskManager.Task::(_debug) Task 9bcb81f7-aba3-4b8d-9d16-c4977a91569f: moving from state init -> state preparing
>Thread-15913928::DEBUG::2013-11-20 03:17:46,369::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913928::DEBUG::2013-11-20 03:17:46,370::task::495::TaskManager.Task::(_debug) Task 9bcb81f7-aba3-4b8d-9d16-c4977a91569f: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913928::DEBUG::2013-11-20 03:17:46,371::task::495::TaskManager.Task::(_debug) Task 9bcb81f7-aba3-4b8d-9d16-c4977a91569f: moving from state preparing -> state finished
>Thread-15913928::DEBUG::2013-11-20 03:17:46,371::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913928::DEBUG::2013-11-20 03:17:46,372::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913928::DEBUG::2013-11-20 03:17:46,373::task::495::TaskManager.Task::(_debug) Task 9bcb81f7-aba3-4b8d-9d16-c4977a91569f: ref 0 aborting False
>Thread-15913928::INFO::2013-11-20 03:17:46,373::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913929::DEBUG::2013-11-20 03:17:46,378::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913929::INFO::2013-11-20 03:17:46,379::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913929::DEBUG::2013-11-20 03:17:46,380::task::495::TaskManager.Task::(_debug) Task 09972135-a189-43cd-b7c9-1ea7672fe889: moving from state init -> state preparing
>Thread-15913929::DEBUG::2013-11-20 03:17:46,381::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6e9c9ee2-7056-4d35-ba2c-54c4b47e337a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913929::DEBUG::2013-11-20 03:17:46,381::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913929::DEBUG::2013-11-20 03:17:46,381::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913929::DEBUG::2013-11-20 03:17:46,382::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6e9c9ee2-7056-4d35-ba2c-54c4b47e337a`::Granted request
>Thread-15913929::DEBUG::2013-11-20 03:17:46,382::task::495::TaskManager.Task::(_debug) Task 09972135-a189-43cd-b7c9-1ea7672fe889: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913929::DEBUG::2013-11-20 03:17:46,383::task::495::TaskManager.Task::(_debug) Task 09972135-a189-43cd-b7c9-1ea7672fe889: ref 1 aborting False
>Thread-15913929::WARNING::2013-11-20 03:17:46,384::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913929::DEBUG::2013-11-20 03:17:46,385::task::495::TaskManager.Task::(_debug) Task 09972135-a189-43cd-b7c9-1ea7672fe889: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913929::DEBUG::2013-11-20 03:17:46,385::task::495::TaskManager.Task::(_debug) Task 09972135-a189-43cd-b7c9-1ea7672fe889: moving from state preparing -> state finished
>Thread-15913929::DEBUG::2013-11-20 03:17:46,386::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913929::DEBUG::2013-11-20 03:17:46,386::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913929::DEBUG::2013-11-20 03:17:46,387::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913929::DEBUG::2013-11-20 03:17:46,387::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913929::DEBUG::2013-11-20 03:17:46,387::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913929::DEBUG::2013-11-20 03:17:46,388::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913929::DEBUG::2013-11-20 03:17:46,388::task::495::TaskManager.Task::(_debug) Task 09972135-a189-43cd-b7c9-1ea7672fe889: ref 0 aborting False
>Thread-15913929::INFO::2013-11-20 03:17:46,388::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913930::INFO::2013-11-20 03:17:46,737::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913930::DEBUG::2013-11-20 03:17:46,738::task::495::TaskManager.Task::(_debug) Task 7683faca-a357-4ae1-83a4-bddadf9392f6: moving from state init -> state preparing
>Thread-15913930::DEBUG::2013-11-20 03:17:46,739::task::495::TaskManager.Task::(_debug) Task 7683faca-a357-4ae1-83a4-bddadf9392f6: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000300884246826', 'lastCheck': 1384917463.9221289, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00141906738281', 'lastCheck': 1384917460.3949161, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0400328636169', 'lastCheck': 1384917462.9638219, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000539064407349', 'lastCheck': 1384917462.4630251, 'valid': True, 'code': 0}}
>Thread-15913930::DEBUG::2013-11-20 03:17:46,740::task::495::TaskManager.Task::(_debug) Task 7683faca-a357-4ae1-83a4-bddadf9392f6: moving from state preparing -> state finished
>Thread-15913930::DEBUG::2013-11-20 03:17:46,740::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913930::DEBUG::2013-11-20 03:17:46,741::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913930::DEBUG::2013-11-20 03:17:46,741::task::495::TaskManager.Task::(_debug) Task 7683faca-a357-4ae1-83a4-bddadf9392f6: ref 0 aborting False
>Thread-15913930::INFO::2013-11-20 03:17:46,742::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000300884246826', 'lastCheck': 1384917463.9221289, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00141906738281', 'lastCheck': 1384917460.3949161, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0400328636169', 'lastCheck': 1384917462.9638219, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000539064407349', 'lastCheck': 1384917462.4630251, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:17:47,307::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:17:48,231::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.45613 s, 2.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:17:50,245::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:17:50,834::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0763936 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 03:17:51,653::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:17:51,654::task::495::TaskManager.Task::(_debug) Task 45700514-a39d-4371-b6cd-eccfaab8c2bb: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:17:51,656::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c2fea0da-da87-4762-a8ef-b8f9d9ba5889`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:17:51,657::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:17:51,658::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:17:51,659::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c2fea0da-da87-4762-a8ef-b8f9d9ba5889`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:17:51,660::task::495::TaskManager.Task::(_debug) Task 45700514-a39d-4371-b6cd-eccfaab8c2bb: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:17:51,660::task::495::TaskManager.Task::(_debug) Task 45700514-a39d-4371-b6cd-eccfaab8c2bb: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:17:51,661::task::495::TaskManager.Task::(_debug) Task 45700514-a39d-4371-b6cd-eccfaab8c2bb: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:17:51,662::task::495::TaskManager.Task::(_debug) Task 45700514-a39d-4371-b6cd-eccfaab8c2bb: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:17:51,662::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:17:51,663::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:17:51,664::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:17:51,664::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:17:51,665::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:17:51,666::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:17:51,666::task::495::TaskManager.Task::(_debug) Task 45700514-a39d-4371-b6cd-eccfaab8c2bb: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:17:51,667::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:17:51,668::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:17:51,668::task::495::TaskManager.Task::(_debug) Task cd0008ad-dd6f-42da-865e-b65fb5fa107d: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:17:51,670::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`405dca0b-8edd-410c-99f6-dffb2f0b6bbc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:17:51,670::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:17:51,671::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:17:51,671::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`405dca0b-8edd-410c-99f6-dffb2f0b6bbc`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:17:51,672::task::495::TaskManager.Task::(_debug) Task cd0008ad-dd6f-42da-865e-b65fb5fa107d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:17:51,673::task::495::TaskManager.Task::(_debug) Task cd0008ad-dd6f-42da-865e-b65fb5fa107d: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:17:51,674::task::495::TaskManager.Task::(_debug) Task cd0008ad-dd6f-42da-865e-b65fb5fa107d: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:17:51,674::task::495::TaskManager.Task::(_debug) Task cd0008ad-dd6f-42da-865e-b65fb5fa107d: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:17:51,675::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:17:51,675::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:17:51,676::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:17:51,677::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:17:51,677::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:17:51,678::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:17:51,679::task::495::TaskManager.Task::(_debug) Task cd0008ad-dd6f-42da-865e-b65fb5fa107d: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:17:51,679::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:17:52,847::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:17:53,380::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0873444 s, 11.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:17:55,396::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:17:55,932::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0518697 s, 19.7 MB/s\n'; <rc> = 0
>Thread-15913935::DEBUG::2013-11-20 03:17:56,608::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913935::INFO::2013-11-20 03:17:56,611::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913935::DEBUG::2013-11-20 03:17:56,613::task::495::TaskManager.Task::(_debug) Task 7a0093b4-f1ad-4299-8308-b7156afb5193: moving from state init -> state preparing
>Thread-15913935::DEBUG::2013-11-20 03:17:56,616::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913935::DEBUG::2013-11-20 03:17:56,618::task::495::TaskManager.Task::(_debug) Task 7a0093b4-f1ad-4299-8308-b7156afb5193: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913935::DEBUG::2013-11-20 03:17:56,619::task::495::TaskManager.Task::(_debug) Task 7a0093b4-f1ad-4299-8308-b7156afb5193: moving from state preparing -> state finished
>Thread-15913935::DEBUG::2013-11-20 03:17:56,621::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913935::DEBUG::2013-11-20 03:17:56,622::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913935::DEBUG::2013-11-20 03:17:56,625::task::495::TaskManager.Task::(_debug) Task 7a0093b4-f1ad-4299-8308-b7156afb5193: ref 0 aborting False
>Thread-15913935::INFO::2013-11-20 03:17:56,628::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913936::DEBUG::2013-11-20 03:17:56,644::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913936::INFO::2013-11-20 03:17:56,655::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913936::DEBUG::2013-11-20 03:17:56,656::task::495::TaskManager.Task::(_debug) Task 23428aa1-0ace-4bfe-bc90-9c74356ceb1d: moving from state init -> state preparing
>Thread-15913936::DEBUG::2013-11-20 03:17:56,658::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`fcd869e2-233b-4a22-ad09-54665136cab0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913936::DEBUG::2013-11-20 03:17:56,658::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913936::DEBUG::2013-11-20 03:17:56,659::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913936::DEBUG::2013-11-20 03:17:56,660::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`fcd869e2-233b-4a22-ad09-54665136cab0`::Granted request
>Thread-15913936::DEBUG::2013-11-20 03:17:56,661::task::495::TaskManager.Task::(_debug) Task 23428aa1-0ace-4bfe-bc90-9c74356ceb1d: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913936::DEBUG::2013-11-20 03:17:56,662::task::495::TaskManager.Task::(_debug) Task 23428aa1-0ace-4bfe-bc90-9c74356ceb1d: ref 1 aborting False
>Thread-15913936::WARNING::2013-11-20 03:17:56,664::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913936::DEBUG::2013-11-20 03:17:56,666::task::495::TaskManager.Task::(_debug) Task 23428aa1-0ace-4bfe-bc90-9c74356ceb1d: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913936::DEBUG::2013-11-20 03:17:56,667::task::495::TaskManager.Task::(_debug) Task 23428aa1-0ace-4bfe-bc90-9c74356ceb1d: moving from state preparing -> state finished
>Thread-15913936::DEBUG::2013-11-20 03:17:56,667::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913936::DEBUG::2013-11-20 03:17:56,668::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913936::DEBUG::2013-11-20 03:17:56,669::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913936::DEBUG::2013-11-20 03:17:56,669::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913936::DEBUG::2013-11-20 03:17:56,670::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913936::DEBUG::2013-11-20 03:17:56,671::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913936::DEBUG::2013-11-20 03:17:56,671::task::495::TaskManager.Task::(_debug) Task 23428aa1-0ace-4bfe-bc90-9c74356ceb1d: ref 0 aborting False
>Thread-15913936::INFO::2013-11-20 03:17:56,672::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:17:57,946::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:17:58,448::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.061842 s, 16.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:18:00,459::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:18:01,021::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0777301 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15913938::INFO::2013-11-20 03:18:02,201::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913938::DEBUG::2013-11-20 03:18:02,202::task::495::TaskManager.Task::(_debug) Task 85d9e1a0-deeb-499a-abba-4a8b9712100e: moving from state init -> state preparing
>Thread-15913938::DEBUG::2013-11-20 03:18:02,203::task::495::TaskManager.Task::(_debug) Task 85d9e1a0-deeb-499a-abba-4a8b9712100e: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000453948974609', 'lastCheck': 1384917474.0994971, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0089910030365', 'lastCheck': 1384917480.6765549, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00166487693787', 'lastCheck': 1384917473.104147, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000359058380127', 'lastCheck': 1384917472.5292311, 'valid': True, 'code': 0}}
>Thread-15913938::DEBUG::2013-11-20 03:18:02,204::task::495::TaskManager.Task::(_debug) Task 85d9e1a0-deeb-499a-abba-4a8b9712100e: moving from state preparing -> state finished
>Thread-15913938::DEBUG::2013-11-20 03:18:02,205::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913938::DEBUG::2013-11-20 03:18:02,206::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913938::DEBUG::2013-11-20 03:18:02,207::task::495::TaskManager.Task::(_debug) Task 85d9e1a0-deeb-499a-abba-4a8b9712100e: ref 0 aborting False
>Thread-15913938::INFO::2013-11-20 03:18:02,208::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000453948974609', 'lastCheck': 1384917474.0994971, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0089910030365', 'lastCheck': 1384917480.6765549, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00166487693787', 'lastCheck': 1384917473.104147, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000359058380127', 'lastCheck': 1384917472.5292311, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:18:03,037::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:18:03,591::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0752684 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:18:05,602::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:18:06,117::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0753832 s, 13.6 MB/s\n'; <rc> = 0
>Thread-15913941::DEBUG::2013-11-20 03:18:06,928::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913941::INFO::2013-11-20 03:18:06,929::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913941::DEBUG::2013-11-20 03:18:06,930::task::495::TaskManager.Task::(_debug) Task 207eac8b-9f65-4d2a-b9f5-9e601cddfa89: moving from state init -> state preparing
>Thread-15913941::DEBUG::2013-11-20 03:18:06,931::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913941::DEBUG::2013-11-20 03:18:06,931::task::495::TaskManager.Task::(_debug) Task 207eac8b-9f65-4d2a-b9f5-9e601cddfa89: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913941::DEBUG::2013-11-20 03:18:06,932::task::495::TaskManager.Task::(_debug) Task 207eac8b-9f65-4d2a-b9f5-9e601cddfa89: moving from state preparing -> state finished
>Thread-15913941::DEBUG::2013-11-20 03:18:06,933::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913941::DEBUG::2013-11-20 03:18:06,933::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913941::DEBUG::2013-11-20 03:18:06,934::task::495::TaskManager.Task::(_debug) Task 207eac8b-9f65-4d2a-b9f5-9e601cddfa89: ref 0 aborting False
>Thread-15913941::INFO::2013-11-20 03:18:06,934::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913942::DEBUG::2013-11-20 03:18:06,939::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913942::INFO::2013-11-20 03:18:06,940::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913942::DEBUG::2013-11-20 03:18:06,941::task::495::TaskManager.Task::(_debug) Task b9818109-1a3c-4d22-9c70-125be7fbcbdd: moving from state init -> state preparing
>Thread-15913942::DEBUG::2013-11-20 03:18:06,943::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`39b7f09c-2453-47f3-83f0-6a7e692c5f05`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913942::DEBUG::2013-11-20 03:18:06,944::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913942::DEBUG::2013-11-20 03:18:06,945::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913942::DEBUG::2013-11-20 03:18:06,946::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`39b7f09c-2453-47f3-83f0-6a7e692c5f05`::Granted request
>Thread-15913942::DEBUG::2013-11-20 03:18:06,948::task::495::TaskManager.Task::(_debug) Task b9818109-1a3c-4d22-9c70-125be7fbcbdd: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913942::DEBUG::2013-11-20 03:18:06,948::task::495::TaskManager.Task::(_debug) Task b9818109-1a3c-4d22-9c70-125be7fbcbdd: ref 1 aborting False
>Thread-15913942::WARNING::2013-11-20 03:18:06,952::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913942::DEBUG::2013-11-20 03:18:06,953::task::495::TaskManager.Task::(_debug) Task b9818109-1a3c-4d22-9c70-125be7fbcbdd: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913942::DEBUG::2013-11-20 03:18:06,954::task::495::TaskManager.Task::(_debug) Task b9818109-1a3c-4d22-9c70-125be7fbcbdd: moving from state preparing -> state finished
>Thread-15913942::DEBUG::2013-11-20 03:18:06,955::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913942::DEBUG::2013-11-20 03:18:06,955::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913942::DEBUG::2013-11-20 03:18:06,956::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913942::DEBUG::2013-11-20 03:18:06,956::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913942::DEBUG::2013-11-20 03:18:06,956::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913942::DEBUG::2013-11-20 03:18:06,957::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913942::DEBUG::2013-11-20 03:18:06,958::task::495::TaskManager.Task::(_debug) Task b9818109-1a3c-4d22-9c70-125be7fbcbdd: ref 0 aborting False
>Thread-15913942::INFO::2013-11-20 03:18:06,958::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:18:08,133::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 03:18:08,328::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:18:08,329::task::495::TaskManager.Task::(_debug) Task 00831382-e8db-4e73-ac90-781a272248ff: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:18:08,333::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`31fe8bc7-03c0-49fd-8a78-3622cdd1a9ac`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:18:08,334::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:18:08,336::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:18:08,337::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`31fe8bc7-03c0-49fd-8a78-3622cdd1a9ac`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:18:08,338::task::495::TaskManager.Task::(_debug) Task 00831382-e8db-4e73-ac90-781a272248ff: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:18:08,339::task::495::TaskManager.Task::(_debug) Task 00831382-e8db-4e73-ac90-781a272248ff: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:18:08,340::task::495::TaskManager.Task::(_debug) Task 00831382-e8db-4e73-ac90-781a272248ff: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:18:08,340::task::495::TaskManager.Task::(_debug) Task 00831382-e8db-4e73-ac90-781a272248ff: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:18:08,341::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:18:08,341::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:18:08,342::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:18:08,343::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:18:08,344::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:18:08,344::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:18:08,345::task::495::TaskManager.Task::(_debug) Task 00831382-e8db-4e73-ac90-781a272248ff: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:18:08,345::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:18:08,346::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:18:08,347::task::495::TaskManager.Task::(_debug) Task 7080fb05-e3b2-4185-bc01-130302102f41: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:18:08,349::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`40e9d801-b197-46e2-9a16-0e10d9d1be7f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:18:08,350::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:18:08,351::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:18:08,352::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`40e9d801-b197-46e2-9a16-0e10d9d1be7f`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:18:08,353::task::495::TaskManager.Task::(_debug) Task 7080fb05-e3b2-4185-bc01-130302102f41: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:18:08,354::task::495::TaskManager.Task::(_debug) Task 7080fb05-e3b2-4185-bc01-130302102f41: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:18:08,355::task::495::TaskManager.Task::(_debug) Task 7080fb05-e3b2-4185-bc01-130302102f41: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:18:08,355::task::495::TaskManager.Task::(_debug) Task 7080fb05-e3b2-4185-bc01-130302102f41: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:18:08,356::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:18:08,356::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:18:08,357::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:18:08,357::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:18:08,358::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:18:08,358::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:18:08,359::task::495::TaskManager.Task::(_debug) Task 7080fb05-e3b2-4185-bc01-130302102f41: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:18:08,359::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:18:08,360::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:18:08,360::task::495::TaskManager.Task::(_debug) Task 0d53c51e-e666-477c-a992-ded599dc7c6a: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:18:08,361::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fb1642aa-52ac-4d89-9399-085e43b0ea51`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:18:08,362::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:18:08,362::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:18:08,363::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fb1642aa-52ac-4d89-9399-085e43b0ea51`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:18:08,363::task::495::TaskManager.Task::(_debug) Task 0d53c51e-e666-477c-a992-ded599dc7c6a: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:18:08,364::task::495::TaskManager.Task::(_debug) Task 0d53c51e-e666-477c-a992-ded599dc7c6a: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:18:08,364::task::495::TaskManager.Task::(_debug) Task 0d53c51e-e666-477c-a992-ded599dc7c6a: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:18:08,365::task::495::TaskManager.Task::(_debug) Task 0d53c51e-e666-477c-a992-ded599dc7c6a: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:18:08,365::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:18:08,366::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:18:08,366::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:18:08,367::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:18:08,367::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:18:08,368::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:18:08,368::task::495::TaskManager.Task::(_debug) Task 0d53c51e-e666-477c-a992-ded599dc7c6a: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:18:08,369::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:18:08,369::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:18:08,370::task::495::TaskManager.Task::(_debug) Task 4209f755-bd4a-486d-a569-c3cb963e63a4: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:18:08,371::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`bf6d7259-5137-476f-95da-25927047e6df`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:18:08,371::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:18:08,371::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:18:08,372::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`bf6d7259-5137-476f-95da-25927047e6df`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:18:08,373::task::495::TaskManager.Task::(_debug) Task 4209f755-bd4a-486d-a569-c3cb963e63a4: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:18:08,373::task::495::TaskManager.Task::(_debug) Task 4209f755-bd4a-486d-a569-c3cb963e63a4: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:18:08,374::task::495::TaskManager.Task::(_debug) Task 4209f755-bd4a-486d-a569-c3cb963e63a4: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:18:08,374::task::495::TaskManager.Task::(_debug) Task 4209f755-bd4a-486d-a569-c3cb963e63a4: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:18:08,374::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:18:08,375::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:18:08,375::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:18:08,376::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:18:08,376::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:18:08,377::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:18:08,377::task::495::TaskManager.Task::(_debug) Task 4209f755-bd4a-486d-a569-c3cb963e63a4: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:18:08,378::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:18:08,690::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.119183 s, 8.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:18:10,706::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:18:11,229::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0751744 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:18:13,243::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:18:13,818::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0970897 s, 10.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:18:15,835::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:18:16,388::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.083207 s, 12.3 MB/s\n'; <rc> = 0
>Thread-15913946::DEBUG::2013-11-20 03:18:17,169::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913946::INFO::2013-11-20 03:18:17,170::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913946::DEBUG::2013-11-20 03:18:17,171::task::495::TaskManager.Task::(_debug) Task 6cdc11ce-a576-4dbe-b98c-1f2da384e22c: moving from state init -> state preparing
>Thread-15913946::DEBUG::2013-11-20 03:18:17,172::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913946::DEBUG::2013-11-20 03:18:17,173::task::495::TaskManager.Task::(_debug) Task 6cdc11ce-a576-4dbe-b98c-1f2da384e22c: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913946::DEBUG::2013-11-20 03:18:17,174::task::495::TaskManager.Task::(_debug) Task 6cdc11ce-a576-4dbe-b98c-1f2da384e22c: moving from state preparing -> state finished
>Thread-15913946::DEBUG::2013-11-20 03:18:17,174::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913946::DEBUG::2013-11-20 03:18:17,175::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913946::DEBUG::2013-11-20 03:18:17,175::task::495::TaskManager.Task::(_debug) Task 6cdc11ce-a576-4dbe-b98c-1f2da384e22c: ref 0 aborting False
>Thread-15913946::INFO::2013-11-20 03:18:17,176::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913947::DEBUG::2013-11-20 03:18:17,180::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913947::INFO::2013-11-20 03:18:17,181::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913947::DEBUG::2013-11-20 03:18:17,182::task::495::TaskManager.Task::(_debug) Task 919bce8b-8b7a-4aeb-a74f-80b77aaad493: moving from state init -> state preparing
>Thread-15913947::DEBUG::2013-11-20 03:18:17,191::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9c5b9410-52d1-42bd-b508-da97201260f8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913947::DEBUG::2013-11-20 03:18:17,192::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913947::DEBUG::2013-11-20 03:18:17,193::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913947::DEBUG::2013-11-20 03:18:17,193::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9c5b9410-52d1-42bd-b508-da97201260f8`::Granted request
>Thread-15913947::DEBUG::2013-11-20 03:18:17,194::task::495::TaskManager.Task::(_debug) Task 919bce8b-8b7a-4aeb-a74f-80b77aaad493: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913947::DEBUG::2013-11-20 03:18:17,195::task::495::TaskManager.Task::(_debug) Task 919bce8b-8b7a-4aeb-a74f-80b77aaad493: ref 1 aborting False
>Thread-15913947::WARNING::2013-11-20 03:18:17,197::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913947::DEBUG::2013-11-20 03:18:17,198::task::495::TaskManager.Task::(_debug) Task 919bce8b-8b7a-4aeb-a74f-80b77aaad493: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913947::DEBUG::2013-11-20 03:18:17,199::task::495::TaskManager.Task::(_debug) Task 919bce8b-8b7a-4aeb-a74f-80b77aaad493: moving from state preparing -> state finished
>Thread-15913947::DEBUG::2013-11-20 03:18:17,200::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913947::DEBUG::2013-11-20 03:18:17,200::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913947::DEBUG::2013-11-20 03:18:17,201::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913947::DEBUG::2013-11-20 03:18:17,202::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913947::DEBUG::2013-11-20 03:18:17,202::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913947::DEBUG::2013-11-20 03:18:17,203::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913947::DEBUG::2013-11-20 03:18:17,203::task::495::TaskManager.Task::(_debug) Task 919bce8b-8b7a-4aeb-a74f-80b77aaad493: ref 0 aborting False
>Thread-15913947::INFO::2013-11-20 03:18:17,204::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913948::INFO::2013-11-20 03:18:17,927::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913948::DEBUG::2013-11-20 03:18:17,928::task::495::TaskManager.Task::(_debug) Task 133f1d8e-ebf4-4863-8c45-865870f7973d: moving from state init -> state preparing
>Thread-15913948::DEBUG::2013-11-20 03:18:17,929::task::495::TaskManager.Task::(_debug) Task 133f1d8e-ebf4-4863-8c45-865870f7973d: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000332832336426', 'lastCheck': 1384917494.4630909, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142002105713', 'lastCheck': 1384917490.899847, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0138359069824', 'lastCheck': 1384917493.4850581, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000556945800781', 'lastCheck': 1384917492.8946681, 'valid': True, 'code': 0}}
>Thread-15913948::DEBUG::2013-11-20 03:18:17,930::task::495::TaskManager.Task::(_debug) Task 133f1d8e-ebf4-4863-8c45-865870f7973d: moving from state preparing -> state finished
>Thread-15913948::DEBUG::2013-11-20 03:18:17,930::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913948::DEBUG::2013-11-20 03:18:17,931::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913948::DEBUG::2013-11-20 03:18:17,932::task::495::TaskManager.Task::(_debug) Task 133f1d8e-ebf4-4863-8c45-865870f7973d: ref 0 aborting False
>Thread-15913948::INFO::2013-11-20 03:18:17,932::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000332832336426', 'lastCheck': 1384917494.4630909, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142002105713', 'lastCheck': 1384917490.899847, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0138359069824', 'lastCheck': 1384917493.4850581, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000556945800781', 'lastCheck': 1384917492.8946681, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:18:18,407::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 03:18:18,633::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:18:18,636::task::495::TaskManager.Task::(_debug) Task 5fd1e375-f0b0-406c-8937-4b84cded6e87: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:18:18,639::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e3715110-47f4-42b3-89ab-97163ae8f8b0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:18:18,639::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:18:18,641::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:18:18,641::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e3715110-47f4-42b3-89ab-97163ae8f8b0`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:18:18,642::task::495::TaskManager.Task::(_debug) Task 5fd1e375-f0b0-406c-8937-4b84cded6e87: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:18:18,643::task::495::TaskManager.Task::(_debug) Task 5fd1e375-f0b0-406c-8937-4b84cded6e87: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:18:18,644::task::495::TaskManager.Task::(_debug) Task 5fd1e375-f0b0-406c-8937-4b84cded6e87: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:18:18,644::task::495::TaskManager.Task::(_debug) Task 5fd1e375-f0b0-406c-8937-4b84cded6e87: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:18:18,645::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:18:18,645::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:18:18,646::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:18:18,646::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:18:18,647::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:18:18,647::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:18:18,648::task::495::TaskManager.Task::(_debug) Task 5fd1e375-f0b0-406c-8937-4b84cded6e87: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:18:18,649::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:18:18,960::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0758006 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:18:20,973::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:18:21,537::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0912544 s, 11.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:18:23,553::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:18:24,102::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0554114 s, 18.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:18:26,117::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:18:26,717::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0764485 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15913952::DEBUG::2013-11-20 03:18:27,358::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913952::INFO::2013-11-20 03:18:27,360::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913952::DEBUG::2013-11-20 03:18:27,360::task::495::TaskManager.Task::(_debug) Task e25b978b-3beb-4b13-a6c9-11977f811988: moving from state init -> state preparing
>Thread-15913952::DEBUG::2013-11-20 03:18:27,361::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913952::DEBUG::2013-11-20 03:18:27,362::task::495::TaskManager.Task::(_debug) Task e25b978b-3beb-4b13-a6c9-11977f811988: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913952::DEBUG::2013-11-20 03:18:27,362::task::495::TaskManager.Task::(_debug) Task e25b978b-3beb-4b13-a6c9-11977f811988: moving from state preparing -> state finished
>Thread-15913952::DEBUG::2013-11-20 03:18:27,363::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913952::DEBUG::2013-11-20 03:18:27,363::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913952::DEBUG::2013-11-20 03:18:27,364::task::495::TaskManager.Task::(_debug) Task e25b978b-3beb-4b13-a6c9-11977f811988: ref 0 aborting False
>Thread-15913952::INFO::2013-11-20 03:18:27,365::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913953::DEBUG::2013-11-20 03:18:27,370::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913953::INFO::2013-11-20 03:18:27,371::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913953::DEBUG::2013-11-20 03:18:27,372::task::495::TaskManager.Task::(_debug) Task 1d70bfd7-7a64-410a-83ad-25b6bbab134b: moving from state init -> state preparing
>Thread-15913953::DEBUG::2013-11-20 03:18:27,374::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e832fd39-d8cb-4657-9ffe-123dc5c4fce2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913953::DEBUG::2013-11-20 03:18:27,374::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913953::DEBUG::2013-11-20 03:18:27,375::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913953::DEBUG::2013-11-20 03:18:27,376::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e832fd39-d8cb-4657-9ffe-123dc5c4fce2`::Granted request
>Thread-15913953::DEBUG::2013-11-20 03:18:27,377::task::495::TaskManager.Task::(_debug) Task 1d70bfd7-7a64-410a-83ad-25b6bbab134b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913953::DEBUG::2013-11-20 03:18:27,378::task::495::TaskManager.Task::(_debug) Task 1d70bfd7-7a64-410a-83ad-25b6bbab134b: ref 1 aborting False
>Thread-15913953::WARNING::2013-11-20 03:18:27,381::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913953::DEBUG::2013-11-20 03:18:27,383::task::495::TaskManager.Task::(_debug) Task 1d70bfd7-7a64-410a-83ad-25b6bbab134b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913953::DEBUG::2013-11-20 03:18:27,384::task::495::TaskManager.Task::(_debug) Task 1d70bfd7-7a64-410a-83ad-25b6bbab134b: moving from state preparing -> state finished
>Thread-15913953::DEBUG::2013-11-20 03:18:27,385::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913953::DEBUG::2013-11-20 03:18:27,386::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913953::DEBUG::2013-11-20 03:18:27,387::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913953::DEBUG::2013-11-20 03:18:27,388::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913953::DEBUG::2013-11-20 03:18:27,388::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913953::DEBUG::2013-11-20 03:18:27,389::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913953::DEBUG::2013-11-20 03:18:27,390::task::495::TaskManager.Task::(_debug) Task 1d70bfd7-7a64-410a-83ad-25b6bbab134b: ref 0 aborting False
>Thread-15913953::INFO::2013-11-20 03:18:27,391::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:18:28,728::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:18:29,228::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0916001 s, 11.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:18:31,243::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:18:31,768::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0527554 s, 19.4 MB/s\n'; <rc> = 0
>Thread-15913956::INFO::2013-11-20 03:18:33,471::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913956::DEBUG::2013-11-20 03:18:33,472::task::495::TaskManager.Task::(_debug) Task e663208b-22f3-495f-90c7-a08d879b114f: moving from state init -> state preparing
>Thread-15913956::DEBUG::2013-11-20 03:18:33,473::task::495::TaskManager.Task::(_debug) Task e663208b-22f3-495f-90c7-a08d879b114f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000425100326538', 'lastCheck': 1384917504.795975, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00154590606689', 'lastCheck': 1384917511.49985, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.017856836319', 'lastCheck': 1384917503.8996749, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000473976135254', 'lastCheck': 1384917503.1694419, 'valid': True, 'code': 0}}
>Thread-15913956::DEBUG::2013-11-20 03:18:33,474::task::495::TaskManager.Task::(_debug) Task e663208b-22f3-495f-90c7-a08d879b114f: moving from state preparing -> state finished
>Thread-15913956::DEBUG::2013-11-20 03:18:33,475::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913956::DEBUG::2013-11-20 03:18:33,475::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913956::DEBUG::2013-11-20 03:18:33,476::task::495::TaskManager.Task::(_debug) Task e663208b-22f3-495f-90c7-a08d879b114f: ref 0 aborting False
>Thread-15913956::INFO::2013-11-20 03:18:33,477::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000425100326538', 'lastCheck': 1384917504.795975, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00154590606689', 'lastCheck': 1384917511.49985, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.017856836319', 'lastCheck': 1384917503.8996749, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000473976135254', 'lastCheck': 1384917503.1694419, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:18:33,781::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:18:34,319::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.101118 s, 10.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:18:36,334::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 03:18:36,538::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:18:36,540::task::495::TaskManager.Task::(_debug) Task a752e3e6-9741-449c-ace9-d787b0b3c58f: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:18:36,543::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8a2e3a97-3f21-4080-ba16-d7353d50d65f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:18:36,544::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:18:36,545::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:18:36,546::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8a2e3a97-3f21-4080-ba16-d7353d50d65f`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:18:36,548::task::495::TaskManager.Task::(_debug) Task a752e3e6-9741-449c-ace9-d787b0b3c58f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:18:36,549::task::495::TaskManager.Task::(_debug) Task a752e3e6-9741-449c-ace9-d787b0b3c58f: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:18:36,550::task::495::TaskManager.Task::(_debug) Task a752e3e6-9741-449c-ace9-d787b0b3c58f: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:18:36,551::task::495::TaskManager.Task::(_debug) Task a752e3e6-9741-449c-ace9-d787b0b3c58f: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:18:36,552::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:18:36,553::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:18:36,554::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:18:36,555::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:18:36,556::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:18:36,557::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:18:36,558::task::495::TaskManager.Task::(_debug) Task a752e3e6-9741-449c-ace9-d787b0b3c58f: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:18:36,558::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:18:36,865::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.081769 s, 12.5 MB/s\n'; <rc> = 0
>Thread-15913959::DEBUG::2013-11-20 03:18:38,230::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913959::INFO::2013-11-20 03:18:38,231::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913959::DEBUG::2013-11-20 03:18:38,231::task::495::TaskManager.Task::(_debug) Task f71ae104-3ed0-43a0-834a-e06e6e549dc4: moving from state init -> state preparing
>Thread-15913959::DEBUG::2013-11-20 03:18:38,232::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913959::DEBUG::2013-11-20 03:18:38,233::task::495::TaskManager.Task::(_debug) Task f71ae104-3ed0-43a0-834a-e06e6e549dc4: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913959::DEBUG::2013-11-20 03:18:38,233::task::495::TaskManager.Task::(_debug) Task f71ae104-3ed0-43a0-834a-e06e6e549dc4: moving from state preparing -> state finished
>Thread-15913959::DEBUG::2013-11-20 03:18:38,234::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913959::DEBUG::2013-11-20 03:18:38,234::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913959::DEBUG::2013-11-20 03:18:38,235::task::495::TaskManager.Task::(_debug) Task f71ae104-3ed0-43a0-834a-e06e6e549dc4: ref 0 aborting False
>Thread-15913959::INFO::2013-11-20 03:18:38,235::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913960::DEBUG::2013-11-20 03:18:38,240::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913960::INFO::2013-11-20 03:18:38,241::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913960::DEBUG::2013-11-20 03:18:38,241::task::495::TaskManager.Task::(_debug) Task fd5c2052-c7c4-439f-8075-991008a2d0e7: moving from state init -> state preparing
>Thread-15913960::DEBUG::2013-11-20 03:18:38,242::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ddd272ae-edb6-434d-85ef-7109a792a534`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913960::DEBUG::2013-11-20 03:18:38,243::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913960::DEBUG::2013-11-20 03:18:38,244::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913960::DEBUG::2013-11-20 03:18:38,244::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ddd272ae-edb6-434d-85ef-7109a792a534`::Granted request
>Thread-15913960::DEBUG::2013-11-20 03:18:38,245::task::495::TaskManager.Task::(_debug) Task fd5c2052-c7c4-439f-8075-991008a2d0e7: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913960::DEBUG::2013-11-20 03:18:38,246::task::495::TaskManager.Task::(_debug) Task fd5c2052-c7c4-439f-8075-991008a2d0e7: ref 1 aborting False
>Thread-15913960::WARNING::2013-11-20 03:18:38,248::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913960::DEBUG::2013-11-20 03:18:38,248::task::495::TaskManager.Task::(_debug) Task fd5c2052-c7c4-439f-8075-991008a2d0e7: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913960::DEBUG::2013-11-20 03:18:38,249::task::495::TaskManager.Task::(_debug) Task fd5c2052-c7c4-439f-8075-991008a2d0e7: moving from state preparing -> state finished
>Thread-15913960::DEBUG::2013-11-20 03:18:38,250::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913960::DEBUG::2013-11-20 03:18:38,251::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913960::DEBUG::2013-11-20 03:18:38,251::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913960::DEBUG::2013-11-20 03:18:38,252::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913960::DEBUG::2013-11-20 03:18:38,253::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913960::DEBUG::2013-11-20 03:18:38,253::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913960::DEBUG::2013-11-20 03:18:38,254::task::495::TaskManager.Task::(_debug) Task fd5c2052-c7c4-439f-8075-991008a2d0e7: ref 0 aborting False
>Thread-15913960::INFO::2013-11-20 03:18:38,255::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:18:38,880::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 03:18:39,134::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:18:39,137::task::495::TaskManager.Task::(_debug) Task e20f53ee-2903-4830-b672-6af2e246a87a: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:18:39,142::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e1c3a1bf-a20a-4acd-985a-df52b273cd3b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:18:39,143::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:18:39,144::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:18:39,145::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e1c3a1bf-a20a-4acd-985a-df52b273cd3b`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:18:39,146::task::495::TaskManager.Task::(_debug) Task e20f53ee-2903-4830-b672-6af2e246a87a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:18:39,146::task::495::TaskManager.Task::(_debug) Task e20f53ee-2903-4830-b672-6af2e246a87a: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:18:39,147::task::495::TaskManager.Task::(_debug) Task e20f53ee-2903-4830-b672-6af2e246a87a: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:18:39,148::task::495::TaskManager.Task::(_debug) Task e20f53ee-2903-4830-b672-6af2e246a87a: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:18:39,148::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:18:39,149::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:18:39,150::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:18:39,150::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:18:39,151::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:18:39,151::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:18:39,152::task::495::TaskManager.Task::(_debug) Task e20f53ee-2903-4830-b672-6af2e246a87a: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:18:39,152::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:18:39,465::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0833504 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:18:41,477::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:18:42,007::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0744888 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:18:44,022::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 03:18:44,245::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:18:44,249::task::495::TaskManager.Task::(_debug) Task 7b633167-aec8-4a83-bfc9-2aa4d78a8648: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:18:44,256::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`735fc08f-3878-42f6-b4e5-39c8dec66cbf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:18:44,256::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:18:44,257::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:18:44,258::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`735fc08f-3878-42f6-b4e5-39c8dec66cbf`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:18:44,259::task::495::TaskManager.Task::(_debug) Task 7b633167-aec8-4a83-bfc9-2aa4d78a8648: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:18:44,260::task::495::TaskManager.Task::(_debug) Task 7b633167-aec8-4a83-bfc9-2aa4d78a8648: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:18:44,261::task::495::TaskManager.Task::(_debug) Task 7b633167-aec8-4a83-bfc9-2aa4d78a8648: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:18:44,261::task::495::TaskManager.Task::(_debug) Task 7b633167-aec8-4a83-bfc9-2aa4d78a8648: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:18:44,262::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:18:44,262::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:18:44,263::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:18:44,264::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:18:44,264::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:18:44,264::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:18:44,265::task::495::TaskManager.Task::(_debug) Task 7b633167-aec8-4a83-bfc9-2aa4d78a8648: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:18:44,266::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:18:44,596::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0717831 s, 14.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:18:46,609::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:18:47,209::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0872519 s, 11.7 MB/s\n'; <rc> = 0
>Thread-15913964::DEBUG::2013-11-20 03:18:48,452::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913964::INFO::2013-11-20 03:18:48,453::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913964::DEBUG::2013-11-20 03:18:48,454::task::495::TaskManager.Task::(_debug) Task 4cd402bb-bfc4-4cab-afef-75f08f1def6f: moving from state init -> state preparing
>Thread-15913964::DEBUG::2013-11-20 03:18:48,455::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913964::DEBUG::2013-11-20 03:18:48,456::task::495::TaskManager.Task::(_debug) Task 4cd402bb-bfc4-4cab-afef-75f08f1def6f: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913964::DEBUG::2013-11-20 03:18:48,457::task::495::TaskManager.Task::(_debug) Task 4cd402bb-bfc4-4cab-afef-75f08f1def6f: moving from state preparing -> state finished
>Thread-15913964::DEBUG::2013-11-20 03:18:48,458::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913964::DEBUG::2013-11-20 03:18:48,458::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913964::DEBUG::2013-11-20 03:18:48,459::task::495::TaskManager.Task::(_debug) Task 4cd402bb-bfc4-4cab-afef-75f08f1def6f: ref 0 aborting False
>Thread-15913964::INFO::2013-11-20 03:18:48,459::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913965::DEBUG::2013-11-20 03:18:48,465::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913965::INFO::2013-11-20 03:18:48,466::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913965::DEBUG::2013-11-20 03:18:48,467::task::495::TaskManager.Task::(_debug) Task be864e9d-173a-4fe1-aa5a-cdf6456d136a: moving from state init -> state preparing
>Thread-15913965::DEBUG::2013-11-20 03:18:48,468::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`85b5e3bc-9dad-457a-b9ac-04fead6daa6c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913965::DEBUG::2013-11-20 03:18:48,469::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913965::DEBUG::2013-11-20 03:18:48,470::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913965::DEBUG::2013-11-20 03:18:48,470::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`85b5e3bc-9dad-457a-b9ac-04fead6daa6c`::Granted request
>Thread-15913965::DEBUG::2013-11-20 03:18:48,471::task::495::TaskManager.Task::(_debug) Task be864e9d-173a-4fe1-aa5a-cdf6456d136a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913965::DEBUG::2013-11-20 03:18:48,472::task::495::TaskManager.Task::(_debug) Task be864e9d-173a-4fe1-aa5a-cdf6456d136a: ref 1 aborting False
>Thread-15913965::WARNING::2013-11-20 03:18:48,474::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913965::DEBUG::2013-11-20 03:18:48,475::task::495::TaskManager.Task::(_debug) Task be864e9d-173a-4fe1-aa5a-cdf6456d136a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913965::DEBUG::2013-11-20 03:18:48,476::task::495::TaskManager.Task::(_debug) Task be864e9d-173a-4fe1-aa5a-cdf6456d136a: moving from state preparing -> state finished
>Thread-15913965::DEBUG::2013-11-20 03:18:48,476::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913965::DEBUG::2013-11-20 03:18:48,477::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913965::DEBUG::2013-11-20 03:18:48,477::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913965::DEBUG::2013-11-20 03:18:48,478::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913965::DEBUG::2013-11-20 03:18:48,478::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913965::DEBUG::2013-11-20 03:18:48,479::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913965::DEBUG::2013-11-20 03:18:48,480::task::495::TaskManager.Task::(_debug) Task be864e9d-173a-4fe1-aa5a-cdf6456d136a: ref 0 aborting False
>Thread-15913965::INFO::2013-11-20 03:18:48,480::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913966::INFO::2013-11-20 03:18:49,007::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913966::DEBUG::2013-11-20 03:18:49,008::task::495::TaskManager.Task::(_debug) Task 16129398-459b-4070-8a56-33d7e96bf264: moving from state init -> state preparing
>Thread-15913966::DEBUG::2013-11-20 03:18:49,009::task::495::TaskManager.Task::(_debug) Task 16129398-459b-4070-8a56-33d7e96bf264: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000310897827148', 'lastCheck': 1384917525.245492, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00132393836975', 'lastCheck': 1384917521.69717, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0250089168549', 'lastCheck': 1384917524.2804661, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00034499168396', 'lastCheck': 1384917523.691849, 'valid': True, 'code': 0}}
>Thread-15913966::DEBUG::2013-11-20 03:18:49,010::task::495::TaskManager.Task::(_debug) Task 16129398-459b-4070-8a56-33d7e96bf264: moving from state preparing -> state finished
>Thread-15913966::DEBUG::2013-11-20 03:18:49,010::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913966::DEBUG::2013-11-20 03:18:49,011::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913966::DEBUG::2013-11-20 03:18:49,012::task::495::TaskManager.Task::(_debug) Task 16129398-459b-4070-8a56-33d7e96bf264: ref 0 aborting False
>Thread-15913966::INFO::2013-11-20 03:18:49,012::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000310897827148', 'lastCheck': 1384917525.245492, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00132393836975', 'lastCheck': 1384917521.69717, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0250089168549', 'lastCheck': 1384917524.2804661, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00034499168396', 'lastCheck': 1384917523.691849, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:18:49,225::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:18:49,753::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.072798 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:18:51,768::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:18:52,359::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0820192 s, 12.5 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 03:18:52,955::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:18:52,956::task::495::TaskManager.Task::(_debug) Task eace0933-9966-4d5a-a1bb-154432f9adaf: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:18:52,958::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1e35d681-8734-40fb-a032-619293dfde5e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:18:52,959::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:18:52,960::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:18:52,960::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1e35d681-8734-40fb-a032-619293dfde5e`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:18:52,961::task::495::TaskManager.Task::(_debug) Task eace0933-9966-4d5a-a1bb-154432f9adaf: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:18:52,962::task::495::TaskManager.Task::(_debug) Task eace0933-9966-4d5a-a1bb-154432f9adaf: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:18:52,963::task::495::TaskManager.Task::(_debug) Task eace0933-9966-4d5a-a1bb-154432f9adaf: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:18:52,963::task::495::TaskManager.Task::(_debug) Task eace0933-9966-4d5a-a1bb-154432f9adaf: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:18:52,964::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:18:52,965::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:18:52,966::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:18:52,967::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:18:52,967::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:18:52,968::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:18:52,969::task::495::TaskManager.Task::(_debug) Task eace0933-9966-4d5a-a1bb-154432f9adaf: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:18:52,970::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:18:52,971::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:18:52,971::task::495::TaskManager.Task::(_debug) Task 29a91c77-d89c-4c7d-bc17-e571e5c53b62: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:18:52,973::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`25e9e545-a034-4012-b2c5-02eaac2a0c39`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:18:52,973::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:18:52,974::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:18:52,974::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`25e9e545-a034-4012-b2c5-02eaac2a0c39`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:18:52,975::task::495::TaskManager.Task::(_debug) Task 29a91c77-d89c-4c7d-bc17-e571e5c53b62: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:18:52,976::task::495::TaskManager.Task::(_debug) Task 29a91c77-d89c-4c7d-bc17-e571e5c53b62: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:18:52,977::task::495::TaskManager.Task::(_debug) Task 29a91c77-d89c-4c7d-bc17-e571e5c53b62: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:18:52,977::task::495::TaskManager.Task::(_debug) Task 29a91c77-d89c-4c7d-bc17-e571e5c53b62: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:18:52,978::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:18:52,978::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:18:52,979::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:18:52,980::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:18:52,980::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:18:52,980::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:18:52,981::task::495::TaskManager.Task::(_debug) Task 29a91c77-d89c-4c7d-bc17-e571e5c53b62: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:18:52,982::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:18:54,373::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:18:54,938::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.080938 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:18:56,952::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:18:57,486::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0804723 s, 12.7 MB/s\n'; <rc> = 0
>Thread-15913971::DEBUG::2013-11-20 03:18:58,672::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913971::INFO::2013-11-20 03:18:58,674::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913971::DEBUG::2013-11-20 03:18:58,675::task::495::TaskManager.Task::(_debug) Task 8912b6e5-209c-42d2-aa9c-a0d845bcced7: moving from state init -> state preparing
>Thread-15913971::DEBUG::2013-11-20 03:18:58,676::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913971::DEBUG::2013-11-20 03:18:58,676::task::495::TaskManager.Task::(_debug) Task 8912b6e5-209c-42d2-aa9c-a0d845bcced7: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913971::DEBUG::2013-11-20 03:18:58,677::task::495::TaskManager.Task::(_debug) Task 8912b6e5-209c-42d2-aa9c-a0d845bcced7: moving from state preparing -> state finished
>Thread-15913971::DEBUG::2013-11-20 03:18:58,678::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913971::DEBUG::2013-11-20 03:18:58,678::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913971::DEBUG::2013-11-20 03:18:58,679::task::495::TaskManager.Task::(_debug) Task 8912b6e5-209c-42d2-aa9c-a0d845bcced7: ref 0 aborting False
>Thread-15913971::INFO::2013-11-20 03:18:58,679::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913972::DEBUG::2013-11-20 03:18:58,684::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913972::INFO::2013-11-20 03:18:58,685::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913972::DEBUG::2013-11-20 03:18:58,685::task::495::TaskManager.Task::(_debug) Task 619067f8-c460-4e88-a513-8c0b53f48ec7: moving from state init -> state preparing
>Thread-15913972::DEBUG::2013-11-20 03:18:58,686::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`106b6514-1b89-4fc3-952b-7b3f458b9e23`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913972::DEBUG::2013-11-20 03:18:58,687::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913972::DEBUG::2013-11-20 03:18:58,688::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913972::DEBUG::2013-11-20 03:18:58,689::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`106b6514-1b89-4fc3-952b-7b3f458b9e23`::Granted request
>Thread-15913972::DEBUG::2013-11-20 03:18:58,690::task::495::TaskManager.Task::(_debug) Task 619067f8-c460-4e88-a513-8c0b53f48ec7: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913972::DEBUG::2013-11-20 03:18:58,690::task::495::TaskManager.Task::(_debug) Task 619067f8-c460-4e88-a513-8c0b53f48ec7: ref 1 aborting False
>Thread-15913972::WARNING::2013-11-20 03:18:58,693::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913972::DEBUG::2013-11-20 03:18:58,694::task::495::TaskManager.Task::(_debug) Task 619067f8-c460-4e88-a513-8c0b53f48ec7: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913972::DEBUG::2013-11-20 03:18:58,695::task::495::TaskManager.Task::(_debug) Task 619067f8-c460-4e88-a513-8c0b53f48ec7: moving from state preparing -> state finished
>Thread-15913972::DEBUG::2013-11-20 03:18:58,695::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913972::DEBUG::2013-11-20 03:18:58,696::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913972::DEBUG::2013-11-20 03:18:58,697::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913972::DEBUG::2013-11-20 03:18:58,698::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913972::DEBUG::2013-11-20 03:18:58,699::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913972::DEBUG::2013-11-20 03:18:58,700::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913972::DEBUG::2013-11-20 03:18:58,701::task::495::TaskManager.Task::(_debug) Task 619067f8-c460-4e88-a513-8c0b53f48ec7: ref 0 aborting False
>Thread-15913972::INFO::2013-11-20 03:18:58,702::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:18:59,500::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:19:00,080::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0879906 s, 11.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:19:02,097::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:19:02,636::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0816534 s, 12.5 MB/s\n'; <rc> = 0
>Thread-15913974::INFO::2013-11-20 03:19:04,505::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913974::DEBUG::2013-11-20 03:19:04,505::task::495::TaskManager.Task::(_debug) Task 5fac1344-1c25-44de-a209-fec662f8e858: moving from state init -> state preparing
>Thread-15913974::DEBUG::2013-11-20 03:19:04,506::task::495::TaskManager.Task::(_debug) Task 5fac1344-1c25-44de-a209-fec662f8e858: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00033712387085', 'lastCheck': 1384917535.599185, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00123691558838', 'lastCheck': 1384917542.297734, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00817799568176', 'lastCheck': 1384917534.6123581, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000500917434692', 'lastCheck': 1384917544.2906871, 'valid': True, 'code': 0}}
>Thread-15913974::DEBUG::2013-11-20 03:19:04,507::task::495::TaskManager.Task::(_debug) Task 5fac1344-1c25-44de-a209-fec662f8e858: moving from state preparing -> state finished
>Thread-15913974::DEBUG::2013-11-20 03:19:04,507::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913974::DEBUG::2013-11-20 03:19:04,508::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913974::DEBUG::2013-11-20 03:19:04,508::task::495::TaskManager.Task::(_debug) Task 5fac1344-1c25-44de-a209-fec662f8e858: ref 0 aborting False
>Thread-15913974::INFO::2013-11-20 03:19:04,509::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00033712387085', 'lastCheck': 1384917535.599185, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00123691558838', 'lastCheck': 1384917542.297734, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00817799568176', 'lastCheck': 1384917534.6123581, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000500917434692', 'lastCheck': 1384917544.2906871, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:19:04,651::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:19:05,159::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0581533 s, 17.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:19:07,174::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:19:07,700::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0817327 s, 12.5 MB/s\n'; <rc> = 0
>Thread-15913977::DEBUG::2013-11-20 03:19:08,983::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913977::INFO::2013-11-20 03:19:08,985::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913977::DEBUG::2013-11-20 03:19:08,986::task::495::TaskManager.Task::(_debug) Task 3160eda0-c2d9-47cd-8301-e3d8645f47ff: moving from state init -> state preparing
>Thread-15913977::DEBUG::2013-11-20 03:19:08,987::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913977::DEBUG::2013-11-20 03:19:08,988::task::495::TaskManager.Task::(_debug) Task 3160eda0-c2d9-47cd-8301-e3d8645f47ff: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913977::DEBUG::2013-11-20 03:19:08,989::task::495::TaskManager.Task::(_debug) Task 3160eda0-c2d9-47cd-8301-e3d8645f47ff: moving from state preparing -> state finished
>Thread-15913977::DEBUG::2013-11-20 03:19:08,990::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913977::DEBUG::2013-11-20 03:19:08,991::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913977::DEBUG::2013-11-20 03:19:08,991::task::495::TaskManager.Task::(_debug) Task 3160eda0-c2d9-47cd-8301-e3d8645f47ff: ref 0 aborting False
>Thread-15913977::INFO::2013-11-20 03:19:08,992::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913978::DEBUG::2013-11-20 03:19:08,998::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913978::INFO::2013-11-20 03:19:08,999::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913978::DEBUG::2013-11-20 03:19:08,999::task::495::TaskManager.Task::(_debug) Task 2bf2c554-d6f5-4a9f-9408-28157a334cc6: moving from state init -> state preparing
>Thread-15913978::DEBUG::2013-11-20 03:19:09,001::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bb12d00e-c782-4e7f-a534-ded5822c92cc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913978::DEBUG::2013-11-20 03:19:09,001::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913978::DEBUG::2013-11-20 03:19:09,002::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913978::DEBUG::2013-11-20 03:19:09,003::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bb12d00e-c782-4e7f-a534-ded5822c92cc`::Granted request
>Thread-15913978::DEBUG::2013-11-20 03:19:09,003::task::495::TaskManager.Task::(_debug) Task 2bf2c554-d6f5-4a9f-9408-28157a334cc6: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913978::DEBUG::2013-11-20 03:19:09,004::task::495::TaskManager.Task::(_debug) Task 2bf2c554-d6f5-4a9f-9408-28157a334cc6: ref 1 aborting False
>Thread-15913978::WARNING::2013-11-20 03:19:09,006::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913978::DEBUG::2013-11-20 03:19:09,007::task::495::TaskManager.Task::(_debug) Task 2bf2c554-d6f5-4a9f-9408-28157a334cc6: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913978::DEBUG::2013-11-20 03:19:09,008::task::495::TaskManager.Task::(_debug) Task 2bf2c554-d6f5-4a9f-9408-28157a334cc6: moving from state preparing -> state finished
>Thread-15913978::DEBUG::2013-11-20 03:19:09,008::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913978::DEBUG::2013-11-20 03:19:09,009::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913978::DEBUG::2013-11-20 03:19:09,009::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913978::DEBUG::2013-11-20 03:19:09,010::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913978::DEBUG::2013-11-20 03:19:09,011::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913978::DEBUG::2013-11-20 03:19:09,011::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913978::DEBUG::2013-11-20 03:19:09,012::task::495::TaskManager.Task::(_debug) Task 2bf2c554-d6f5-4a9f-9408-28157a334cc6: ref 0 aborting False
>Thread-15913978::INFO::2013-11-20 03:19:09,012::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:19:09,714::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 03:19:09,907::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:19:09,910::task::495::TaskManager.Task::(_debug) Task 05e8d057-4438-4b0c-b4e4-5706f7bedf28: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:19:09,914::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`7111abe8-1119-465f-b39f-fea89631c796`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:19:09,915::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:19:09,916::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:19:09,917::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`7111abe8-1119-465f-b39f-fea89631c796`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:19:09,918::task::495::TaskManager.Task::(_debug) Task 05e8d057-4438-4b0c-b4e4-5706f7bedf28: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:19:09,919::task::495::TaskManager.Task::(_debug) Task 05e8d057-4438-4b0c-b4e4-5706f7bedf28: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:19:09,919::task::495::TaskManager.Task::(_debug) Task 05e8d057-4438-4b0c-b4e4-5706f7bedf28: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:19:09,920::task::495::TaskManager.Task::(_debug) Task 05e8d057-4438-4b0c-b4e4-5706f7bedf28: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:19:09,921::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:19:09,921::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:19:09,922::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:19:09,923::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:19:09,923::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:19:09,924::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:19:09,924::task::495::TaskManager.Task::(_debug) Task 05e8d057-4438-4b0c-b4e4-5706f7bedf28: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:19:09,925::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:19:09,926::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:19:09,926::task::495::TaskManager.Task::(_debug) Task 169c92b2-da45-412c-b328-8ee5a474eea2: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:19:09,927::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`da1dcff9-b6bd-4765-a6e5-cdf9f828a1ad`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:19:09,928::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:19:09,929::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:19:09,929::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`da1dcff9-b6bd-4765-a6e5-cdf9f828a1ad`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:19:09,930::task::495::TaskManager.Task::(_debug) Task 169c92b2-da45-412c-b328-8ee5a474eea2: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:19:09,931::task::495::TaskManager.Task::(_debug) Task 169c92b2-da45-412c-b328-8ee5a474eea2: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:19:09,931::task::495::TaskManager.Task::(_debug) Task 169c92b2-da45-412c-b328-8ee5a474eea2: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:19:09,932::task::495::TaskManager.Task::(_debug) Task 169c92b2-da45-412c-b328-8ee5a474eea2: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:19:09,933::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:19:09,933::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:19:09,934::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:19:09,934::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:19:09,935::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:19:09,935::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:19:09,936::task::495::TaskManager.Task::(_debug) Task 169c92b2-da45-412c-b328-8ee5a474eea2: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:19:09,937::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:19:09,937::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:19:09,938::task::495::TaskManager.Task::(_debug) Task e9fc08af-a381-445e-8296-32380c77bb86: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:19:09,939::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d48aeb09-88f0-4f76-800b-589301ac4dae`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:19:09,939::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:19:09,940::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:19:09,940::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d48aeb09-88f0-4f76-800b-589301ac4dae`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:19:09,941::task::495::TaskManager.Task::(_debug) Task e9fc08af-a381-445e-8296-32380c77bb86: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:19:09,942::task::495::TaskManager.Task::(_debug) Task e9fc08af-a381-445e-8296-32380c77bb86: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:19:09,943::task::495::TaskManager.Task::(_debug) Task e9fc08af-a381-445e-8296-32380c77bb86: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:19:09,943::task::495::TaskManager.Task::(_debug) Task e9fc08af-a381-445e-8296-32380c77bb86: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:19:09,944::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:19:09,944::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:19:09,945::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:19:09,945::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:19:09,946::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:19:09,946::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:19:09,947::task::495::TaskManager.Task::(_debug) Task e9fc08af-a381-445e-8296-32380c77bb86: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:19:09,948::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:19:09,948::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:19:09,949::task::495::TaskManager.Task::(_debug) Task 51b612b9-0634-47f5-a037-3657c1d11467: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:19:09,950::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f95a21b6-e2e7-4aad-a246-6727969f9e18`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:19:09,950::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:19:09,951::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:19:09,952::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f95a21b6-e2e7-4aad-a246-6727969f9e18`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:19:09,952::task::495::TaskManager.Task::(_debug) Task 51b612b9-0634-47f5-a037-3657c1d11467: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:19:09,953::task::495::TaskManager.Task::(_debug) Task 51b612b9-0634-47f5-a037-3657c1d11467: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:19:09,953::task::495::TaskManager.Task::(_debug) Task 51b612b9-0634-47f5-a037-3657c1d11467: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:19:09,954::task::495::TaskManager.Task::(_debug) Task 51b612b9-0634-47f5-a037-3657c1d11467: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:19:09,954::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:19:09,955::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:19:09,955::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:19:09,956::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:19:09,957::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:19:09,957::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:19:09,958::task::495::TaskManager.Task::(_debug) Task 51b612b9-0634-47f5-a037-3657c1d11467: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:19:09,958::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:19:10,230::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0851448 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:19:12,243::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:19:12,780::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0818085 s, 12.5 MB/s\n'; <rc> = 0
>Thread-42840::DEBUG::2013-11-20 03:19:14,468::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:19:14,796::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-42840::DEBUG::2013-11-20 03:19:15,240::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:19:15,359::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0807916 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:19:17,374::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:19:17,941::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0994658 s, 10.3 MB/s\n'; <rc> = 0
>Thread-15913982::DEBUG::2013-11-20 03:19:19,217::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913982::INFO::2013-11-20 03:19:19,220::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913982::DEBUG::2013-11-20 03:19:19,221::task::495::TaskManager.Task::(_debug) Task ef92d118-fbe5-47e4-a531-729f96f2a200: moving from state init -> state preparing
>Thread-15913982::DEBUG::2013-11-20 03:19:19,222::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913982::DEBUG::2013-11-20 03:19:19,222::task::495::TaskManager.Task::(_debug) Task ef92d118-fbe5-47e4-a531-729f96f2a200: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913982::DEBUG::2013-11-20 03:19:19,223::task::495::TaskManager.Task::(_debug) Task ef92d118-fbe5-47e4-a531-729f96f2a200: moving from state preparing -> state finished
>Thread-15913982::DEBUG::2013-11-20 03:19:19,223::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913982::DEBUG::2013-11-20 03:19:19,224::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913982::DEBUG::2013-11-20 03:19:19,225::task::495::TaskManager.Task::(_debug) Task ef92d118-fbe5-47e4-a531-729f96f2a200: ref 0 aborting False
>Thread-15913982::INFO::2013-11-20 03:19:19,225::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913983::DEBUG::2013-11-20 03:19:19,235::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913983::INFO::2013-11-20 03:19:19,236::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913983::DEBUG::2013-11-20 03:19:19,236::task::495::TaskManager.Task::(_debug) Task 75d04285-1595-4fb1-8ca9-aae17dc03ec5: moving from state init -> state preparing
>Thread-15913983::DEBUG::2013-11-20 03:19:19,238::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3157e01e-2a63-4308-9b50-1115a7418a17`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913983::DEBUG::2013-11-20 03:19:19,238::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913983::DEBUG::2013-11-20 03:19:19,239::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913983::DEBUG::2013-11-20 03:19:19,240::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3157e01e-2a63-4308-9b50-1115a7418a17`::Granted request
>Thread-15913983::DEBUG::2013-11-20 03:19:19,241::task::495::TaskManager.Task::(_debug) Task 75d04285-1595-4fb1-8ca9-aae17dc03ec5: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913983::DEBUG::2013-11-20 03:19:19,242::task::495::TaskManager.Task::(_debug) Task 75d04285-1595-4fb1-8ca9-aae17dc03ec5: ref 1 aborting False
>Thread-15913983::WARNING::2013-11-20 03:19:19,244::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913983::DEBUG::2013-11-20 03:19:19,245::task::495::TaskManager.Task::(_debug) Task 75d04285-1595-4fb1-8ca9-aae17dc03ec5: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913983::DEBUG::2013-11-20 03:19:19,246::task::495::TaskManager.Task::(_debug) Task 75d04285-1595-4fb1-8ca9-aae17dc03ec5: moving from state preparing -> state finished
>Thread-15913983::DEBUG::2013-11-20 03:19:19,246::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913983::DEBUG::2013-11-20 03:19:19,247::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913983::DEBUG::2013-11-20 03:19:19,248::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913983::DEBUG::2013-11-20 03:19:19,249::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913983::DEBUG::2013-11-20 03:19:19,249::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913983::DEBUG::2013-11-20 03:19:19,250::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913983::DEBUG::2013-11-20 03:19:19,251::task::495::TaskManager.Task::(_debug) Task 75d04285-1595-4fb1-8ca9-aae17dc03ec5: ref 0 aborting False
>Thread-15913983::INFO::2013-11-20 03:19:19,252::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:19:19,956::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 03:19:20,150::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:19:20,153::task::495::TaskManager.Task::(_debug) Task 1be2afb1-9daf-41cd-ad31-5655101e75a5: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:19:20,160::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0cdc2ce0-815f-4061-9903-f2fdf2e6ee9f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913984::INFO::2013-11-20 03:19:20,160::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913639::DEBUG::2013-11-20 03:19:20,161::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913984::DEBUG::2013-11-20 03:19:20,162::task::495::TaskManager.Task::(_debug) Task 5460c3b3-307d-4650-bb78-e3a3a770978e: moving from state init -> state preparing
>Thread-15913984::DEBUG::2013-11-20 03:19:20,164::task::495::TaskManager.Task::(_debug) Task 5460c3b3-307d-4650-bb78-e3a3a770978e: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000264883041382', 'lastCheck': 1384917556.033457, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0114569664001', 'lastCheck': 1384917552.4826281, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0103697776794', 'lastCheck': 1384917555.1196179, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000441074371338', 'lastCheck': 1384917555.2432389, 'valid': True, 'code': 0}}
>Thread-15913639::DEBUG::2013-11-20 03:19:20,164::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913984::DEBUG::2013-11-20 03:19:20,166::task::495::TaskManager.Task::(_debug) Task 5460c3b3-307d-4650-bb78-e3a3a770978e: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:19:20,166::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0cdc2ce0-815f-4061-9903-f2fdf2e6ee9f`::Granted request
>Thread-15913984::DEBUG::2013-11-20 03:19:20,167::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913984::DEBUG::2013-11-20 03:19:20,169::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913984::DEBUG::2013-11-20 03:19:20,169::task::495::TaskManager.Task::(_debug) Task 5460c3b3-307d-4650-bb78-e3a3a770978e: ref 0 aborting False
>Thread-15913984::INFO::2013-11-20 03:19:20,170::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000264883041382', 'lastCheck': 1384917556.033457, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0114569664001', 'lastCheck': 1384917552.4826281, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0103697776794', 'lastCheck': 1384917555.1196179, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000441074371338', 'lastCheck': 1384917555.2432389, 'valid': True, 'code': 0}}
>Thread-15913639::DEBUG::2013-11-20 03:19:20,180::task::495::TaskManager.Task::(_debug) Task 1be2afb1-9daf-41cd-ad31-5655101e75a5: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:19:20,181::task::495::TaskManager.Task::(_debug) Task 1be2afb1-9daf-41cd-ad31-5655101e75a5: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:19:20,182::task::495::TaskManager.Task::(_debug) Task 1be2afb1-9daf-41cd-ad31-5655101e75a5: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:19:20,183::task::495::TaskManager.Task::(_debug) Task 1be2afb1-9daf-41cd-ad31-5655101e75a5: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:19:20,184::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:19:20,184::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:19:20,185::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:19:20,186::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:19:20,187::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:19:20,187::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:19:20,188::task::495::TaskManager.Task::(_debug) Task 1be2afb1-9daf-41cd-ad31-5655101e75a5: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:19:20,189::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:19:20,437::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0660423 s, 15.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:19:22,450::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:19:22,990::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0803598 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:19:25,007::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:19:25,525::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0688275 s, 14.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:19:27,541::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:19:28,095::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0808873 s, 12.7 MB/s\n'; <rc> = 0
>Thread-15913988::DEBUG::2013-11-20 03:19:29,422::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913988::INFO::2013-11-20 03:19:29,424::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913988::DEBUG::2013-11-20 03:19:29,425::task::495::TaskManager.Task::(_debug) Task 57c2c696-cb51-4f94-b9e0-6671fa2a5a4a: moving from state init -> state preparing
>Thread-15913988::DEBUG::2013-11-20 03:19:29,426::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913988::DEBUG::2013-11-20 03:19:29,426::task::495::TaskManager.Task::(_debug) Task 57c2c696-cb51-4f94-b9e0-6671fa2a5a4a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913988::DEBUG::2013-11-20 03:19:29,427::task::495::TaskManager.Task::(_debug) Task 57c2c696-cb51-4f94-b9e0-6671fa2a5a4a: moving from state preparing -> state finished
>Thread-15913988::DEBUG::2013-11-20 03:19:29,427::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913988::DEBUG::2013-11-20 03:19:29,428::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913988::DEBUG::2013-11-20 03:19:29,428::task::495::TaskManager.Task::(_debug) Task 57c2c696-cb51-4f94-b9e0-6671fa2a5a4a: ref 0 aborting False
>Thread-15913988::INFO::2013-11-20 03:19:29,429::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913989::DEBUG::2013-11-20 03:19:29,433::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913989::INFO::2013-11-20 03:19:29,434::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913989::DEBUG::2013-11-20 03:19:29,435::task::495::TaskManager.Task::(_debug) Task 064b9dff-eab5-442d-9176-3c7140b28aef: moving from state init -> state preparing
>Thread-15913989::DEBUG::2013-11-20 03:19:29,436::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`83ed822b-4e1f-4d42-93ba-598cd04d4a24`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913989::DEBUG::2013-11-20 03:19:29,437::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913989::DEBUG::2013-11-20 03:19:29,437::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913989::DEBUG::2013-11-20 03:19:29,438::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`83ed822b-4e1f-4d42-93ba-598cd04d4a24`::Granted request
>Thread-15913989::DEBUG::2013-11-20 03:19:29,439::task::495::TaskManager.Task::(_debug) Task 064b9dff-eab5-442d-9176-3c7140b28aef: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913989::DEBUG::2013-11-20 03:19:29,440::task::495::TaskManager.Task::(_debug) Task 064b9dff-eab5-442d-9176-3c7140b28aef: ref 1 aborting False
>Thread-15913989::WARNING::2013-11-20 03:19:29,442::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913989::DEBUG::2013-11-20 03:19:29,443::task::495::TaskManager.Task::(_debug) Task 064b9dff-eab5-442d-9176-3c7140b28aef: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913989::DEBUG::2013-11-20 03:19:29,443::task::495::TaskManager.Task::(_debug) Task 064b9dff-eab5-442d-9176-3c7140b28aef: moving from state preparing -> state finished
>Thread-15913989::DEBUG::2013-11-20 03:19:29,444::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913989::DEBUG::2013-11-20 03:19:29,445::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913989::DEBUG::2013-11-20 03:19:29,446::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913989::DEBUG::2013-11-20 03:19:29,447::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913989::DEBUG::2013-11-20 03:19:29,447::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913989::DEBUG::2013-11-20 03:19:29,448::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913989::DEBUG::2013-11-20 03:19:29,448::task::495::TaskManager.Task::(_debug) Task 064b9dff-eab5-442d-9176-3c7140b28aef: ref 0 aborting False
>Thread-15913989::INFO::2013-11-20 03:19:29,449::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:19:30,105::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:19:30,665::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0854322 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:19:32,681::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:19:33,156::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0542529 s, 18.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:19:35,171::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:19:35,719::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0780855 s, 13.1 MB/s\n'; <rc> = 0
>Thread-15913992::INFO::2013-11-20 03:19:37,676::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913992::DEBUG::2013-11-20 03:19:37,677::task::495::TaskManager.Task::(_debug) Task 8444cf40-1757-4ed9-a820-3f02a6b9b078: moving from state init -> state preparing
>Thread-15913992::DEBUG::2013-11-20 03:19:37,678::task::495::TaskManager.Task::(_debug) Task 8444cf40-1757-4ed9-a820-3f02a6b9b078: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00294208526611', 'lastCheck': 1384917576.3917291, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00105714797974', 'lastCheck': 1384917572.879241, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00302386283875', 'lastCheck': 1384917575.3908601, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00115394592285', 'lastCheck': 1384917575.3830309, 'valid': True, 'code': 0}}
>Thread-15913992::DEBUG::2013-11-20 03:19:37,679::task::495::TaskManager.Task::(_debug) Task 8444cf40-1757-4ed9-a820-3f02a6b9b078: moving from state preparing -> state finished
>Thread-15913992::DEBUG::2013-11-20 03:19:37,679::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913992::DEBUG::2013-11-20 03:19:37,680::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913992::DEBUG::2013-11-20 03:19:37,681::task::495::TaskManager.Task::(_debug) Task 8444cf40-1757-4ed9-a820-3f02a6b9b078: ref 0 aborting False
>Thread-15913992::INFO::2013-11-20 03:19:37,681::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00294208526611', 'lastCheck': 1384917576.3917291, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00105714797974', 'lastCheck': 1384917572.879241, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00302386283875', 'lastCheck': 1384917575.3908601, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00115394592285', 'lastCheck': 1384917575.3830309, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:19:37,734::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 03:19:37,992::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:19:37,994::task::495::TaskManager.Task::(_debug) Task 8718f080-ba78-46be-a756-8ace0b136472: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:19:37,996::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`feed16f0-2b30-46c0-b362-f7a854514276`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:19:37,997::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:19:37,998::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:19:37,998::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`feed16f0-2b30-46c0-b362-f7a854514276`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:19:38,000::task::495::TaskManager.Task::(_debug) Task 8718f080-ba78-46be-a756-8ace0b136472: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:19:38,000::task::495::TaskManager.Task::(_debug) Task 8718f080-ba78-46be-a756-8ace0b136472: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:19:38,002::task::495::TaskManager.Task::(_debug) Task 8718f080-ba78-46be-a756-8ace0b136472: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:19:38,003::task::495::TaskManager.Task::(_debug) Task 8718f080-ba78-46be-a756-8ace0b136472: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:19:38,004::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:19:38,005::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:19:38,006::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:19:38,006::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:19:38,007::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:19:38,007::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:19:38,008::task::495::TaskManager.Task::(_debug) Task 8718f080-ba78-46be-a756-8ace0b136472: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:19:38,008::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:19:38,307::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0846897 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15913994::DEBUG::2013-11-20 03:19:40,148::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913994::INFO::2013-11-20 03:19:40,149::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913994::DEBUG::2013-11-20 03:19:40,150::task::495::TaskManager.Task::(_debug) Task 4df9ac09-aa24-4d1b-b628-4f2d169b5c74: moving from state init -> state preparing
>Thread-15913994::DEBUG::2013-11-20 03:19:40,150::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913994::DEBUG::2013-11-20 03:19:40,151::task::495::TaskManager.Task::(_debug) Task 4df9ac09-aa24-4d1b-b628-4f2d169b5c74: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913994::DEBUG::2013-11-20 03:19:40,151::task::495::TaskManager.Task::(_debug) Task 4df9ac09-aa24-4d1b-b628-4f2d169b5c74: moving from state preparing -> state finished
>Thread-15913994::DEBUG::2013-11-20 03:19:40,151::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913994::DEBUG::2013-11-20 03:19:40,152::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913994::DEBUG::2013-11-20 03:19:40,152::task::495::TaskManager.Task::(_debug) Task 4df9ac09-aa24-4d1b-b628-4f2d169b5c74: ref 0 aborting False
>Thread-15913994::INFO::2013-11-20 03:19:40,152::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913995::DEBUG::2013-11-20 03:19:40,155::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913995::INFO::2013-11-20 03:19:40,156::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913995::DEBUG::2013-11-20 03:19:40,156::task::495::TaskManager.Task::(_debug) Task a7379d56-325c-443b-ba1f-7ae4ced6f98f: moving from state init -> state preparing
>Thread-15913995::DEBUG::2013-11-20 03:19:40,157::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3e1b9906-4f8a-43c4-89fd-8443fb3c4b2e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913995::DEBUG::2013-11-20 03:19:40,157::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913995::DEBUG::2013-11-20 03:19:40,158::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913995::DEBUG::2013-11-20 03:19:40,158::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3e1b9906-4f8a-43c4-89fd-8443fb3c4b2e`::Granted request
>Thread-15913995::DEBUG::2013-11-20 03:19:40,159::task::495::TaskManager.Task::(_debug) Task a7379d56-325c-443b-ba1f-7ae4ced6f98f: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913995::DEBUG::2013-11-20 03:19:40,159::task::495::TaskManager.Task::(_debug) Task a7379d56-325c-443b-ba1f-7ae4ced6f98f: ref 1 aborting False
>Thread-15913995::WARNING::2013-11-20 03:19:40,160::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913995::DEBUG::2013-11-20 03:19:40,161::task::495::TaskManager.Task::(_debug) Task a7379d56-325c-443b-ba1f-7ae4ced6f98f: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913995::DEBUG::2013-11-20 03:19:40,161::task::495::TaskManager.Task::(_debug) Task a7379d56-325c-443b-ba1f-7ae4ced6f98f: moving from state preparing -> state finished
>Thread-15913995::DEBUG::2013-11-20 03:19:40,162::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913995::DEBUG::2013-11-20 03:19:40,162::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913995::DEBUG::2013-11-20 03:19:40,163::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913995::DEBUG::2013-11-20 03:19:40,163::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913995::DEBUG::2013-11-20 03:19:40,164::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913995::DEBUG::2013-11-20 03:19:40,164::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913995::DEBUG::2013-11-20 03:19:40,164::task::495::TaskManager.Task::(_debug) Task a7379d56-325c-443b-ba1f-7ae4ced6f98f: ref 0 aborting False
>Thread-15913995::INFO::2013-11-20 03:19:40,165::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:19:40,324::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 03:19:40,539::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:19:40,545::task::495::TaskManager.Task::(_debug) Task b20fced8-ddf5-40a7-94d1-6a564d52b205: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:19:40,548::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`195a2405-25bd-49fd-b874-84f727cda47a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:19:40,549::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:19:40,550::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:19:40,551::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`195a2405-25bd-49fd-b874-84f727cda47a`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:19:40,552::task::495::TaskManager.Task::(_debug) Task b20fced8-ddf5-40a7-94d1-6a564d52b205: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:19:40,552::task::495::TaskManager.Task::(_debug) Task b20fced8-ddf5-40a7-94d1-6a564d52b205: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:19:40,553::task::495::TaskManager.Task::(_debug) Task b20fced8-ddf5-40a7-94d1-6a564d52b205: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:19:40,554::task::495::TaskManager.Task::(_debug) Task b20fced8-ddf5-40a7-94d1-6a564d52b205: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:19:40,555::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:19:40,555::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:19:40,556::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:19:40,557::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:19:40,557::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:19:40,558::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:19:40,558::task::495::TaskManager.Task::(_debug) Task b20fced8-ddf5-40a7-94d1-6a564d52b205: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:19:40,559::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:19:40,868::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0870199 s, 11.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:19:42,882::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:19:43,335::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0687333 s, 14.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:19:45,346::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 03:19:45,561::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:19:45,595::task::495::TaskManager.Task::(_debug) Task 30e4e431-18eb-4d24-811c-a2bf66db40e9: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:19:45,604::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c77db48b-441f-4d3b-98f3-61cd31195ad6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:19:45,606::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:19:45,608::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:19:45,609::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c77db48b-441f-4d3b-98f3-61cd31195ad6`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:19:45,611::task::495::TaskManager.Task::(_debug) Task 30e4e431-18eb-4d24-811c-a2bf66db40e9: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:19:45,612::task::495::TaskManager.Task::(_debug) Task 30e4e431-18eb-4d24-811c-a2bf66db40e9: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:19:45,613::task::495::TaskManager.Task::(_debug) Task 30e4e431-18eb-4d24-811c-a2bf66db40e9: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:19:45,614::task::495::TaskManager.Task::(_debug) Task 30e4e431-18eb-4d24-811c-a2bf66db40e9: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:19:45,615::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:19:45,616::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:19:45,617::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:19:45,618::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:19:45,619::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:19:45,620::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:19:45,621::task::495::TaskManager.Task::(_debug) Task 30e4e431-18eb-4d24-811c-a2bf66db40e9: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:19:45,622::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:19:45,895::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0846956 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:19:47,912::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:19:48,464::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0860339 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:19:50,476::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913999::DEBUG::2013-11-20 03:19:50,730::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913999::INFO::2013-11-20 03:19:50,732::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913999::DEBUG::2013-11-20 03:19:50,733::task::495::TaskManager.Task::(_debug) Task ed3bf178-0569-4265-b32a-46a027c4fb79: moving from state init -> state preparing
>Thread-15913999::DEBUG::2013-11-20 03:19:50,734::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913999::DEBUG::2013-11-20 03:19:50,734::task::495::TaskManager.Task::(_debug) Task ed3bf178-0569-4265-b32a-46a027c4fb79: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913999::DEBUG::2013-11-20 03:19:50,735::task::495::TaskManager.Task::(_debug) Task ed3bf178-0569-4265-b32a-46a027c4fb79: moving from state preparing -> state finished
>Thread-15913999::DEBUG::2013-11-20 03:19:50,736::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913999::DEBUG::2013-11-20 03:19:50,736::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913999::DEBUG::2013-11-20 03:19:50,737::task::495::TaskManager.Task::(_debug) Task ed3bf178-0569-4265-b32a-46a027c4fb79: ref 0 aborting False
>Thread-15913999::INFO::2013-11-20 03:19:50,738::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914000::DEBUG::2013-11-20 03:19:50,743::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914000::INFO::2013-11-20 03:19:50,744::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914000::DEBUG::2013-11-20 03:19:50,745::task::495::TaskManager.Task::(_debug) Task 1b9db192-69b5-4cc7-b8b0-e19353604fae: moving from state init -> state preparing
>Thread-15914000::DEBUG::2013-11-20 03:19:50,746::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9eb2d4a5-200f-45f9-a881-852030b29b68`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914000::DEBUG::2013-11-20 03:19:50,747::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914000::DEBUG::2013-11-20 03:19:50,748::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914000::DEBUG::2013-11-20 03:19:50,748::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9eb2d4a5-200f-45f9-a881-852030b29b68`::Granted request
>Thread-15914000::DEBUG::2013-11-20 03:19:50,750::task::495::TaskManager.Task::(_debug) Task 1b9db192-69b5-4cc7-b8b0-e19353604fae: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914000::DEBUG::2013-11-20 03:19:50,750::task::495::TaskManager.Task::(_debug) Task 1b9db192-69b5-4cc7-b8b0-e19353604fae: ref 1 aborting False
>Thread-15914000::WARNING::2013-11-20 03:19:50,753::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914000::DEBUG::2013-11-20 03:19:50,754::task::495::TaskManager.Task::(_debug) Task 1b9db192-69b5-4cc7-b8b0-e19353604fae: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914000::DEBUG::2013-11-20 03:19:50,755::task::495::TaskManager.Task::(_debug) Task 1b9db192-69b5-4cc7-b8b0-e19353604fae: moving from state preparing -> state finished
>Thread-15914000::DEBUG::2013-11-20 03:19:50,755::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914000::DEBUG::2013-11-20 03:19:50,756::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914000::DEBUG::2013-11-20 03:19:50,757::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914000::DEBUG::2013-11-20 03:19:50,758::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914000::DEBUG::2013-11-20 03:19:50,758::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914000::DEBUG::2013-11-20 03:19:50,759::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914000::DEBUG::2013-11-20 03:19:50,759::task::495::TaskManager.Task::(_debug) Task 1b9db192-69b5-4cc7-b8b0-e19353604fae: ref 0 aborting False
>Thread-15914000::INFO::2013-11-20 03:19:50,760::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:19:51,012::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0769805 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:19:53,029::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:19:53,594::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0843319 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15914002::INFO::2013-11-20 03:19:53,815::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914002::DEBUG::2013-11-20 03:19:53,816::task::495::TaskManager.Task::(_debug) Task 5069d3fc-21a9-4cf8-b5bf-d7dda9af0d3c: moving from state init -> state preparing
>Thread-15914002::DEBUG::2013-11-20 03:19:53,817::task::495::TaskManager.Task::(_debug) Task 5069d3fc-21a9-4cf8-b5bf-d7dda9af0d3c: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000272035598755', 'lastCheck': 1384917586.5507591, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00218605995178', 'lastCheck': 1384917593.2640851, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00219297409058', 'lastCheck': 1384917585.6078019, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0113520622253', 'lastCheck': 1384917585.586195, 'valid': True, 'code': 0}}
>Thread-15914002::DEBUG::2013-11-20 03:19:53,818::task::495::TaskManager.Task::(_debug) Task 5069d3fc-21a9-4cf8-b5bf-d7dda9af0d3c: moving from state preparing -> state finished
>Thread-15914002::DEBUG::2013-11-20 03:19:53,818::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914002::DEBUG::2013-11-20 03:19:53,819::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914002::DEBUG::2013-11-20 03:19:53,819::task::495::TaskManager.Task::(_debug) Task 5069d3fc-21a9-4cf8-b5bf-d7dda9af0d3c: ref 0 aborting False
>Thread-15914002::INFO::2013-11-20 03:19:53,820::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000272035598755', 'lastCheck': 1384917586.5507591, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00218605995178', 'lastCheck': 1384917593.2640851, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00219297409058', 'lastCheck': 1384917585.6078019, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0113520622253', 'lastCheck': 1384917585.586195, 'valid': True, 'code': 0}}
>Thread-15913518::INFO::2013-11-20 03:19:54,237::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:19:54,238::task::495::TaskManager.Task::(_debug) Task 1e795189-534a-469b-9c94-7fd0360fbdf4: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:19:54,239::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`dcad802c-9353-4478-8758-f22240c34ad0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:19:54,240::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:19:54,240::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:19:54,241::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`dcad802c-9353-4478-8758-f22240c34ad0`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:19:54,241::task::495::TaskManager.Task::(_debug) Task 1e795189-534a-469b-9c94-7fd0360fbdf4: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:19:54,242::task::495::TaskManager.Task::(_debug) Task 1e795189-534a-469b-9c94-7fd0360fbdf4: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:19:54,242::task::495::TaskManager.Task::(_debug) Task 1e795189-534a-469b-9c94-7fd0360fbdf4: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:19:54,243::task::495::TaskManager.Task::(_debug) Task 1e795189-534a-469b-9c94-7fd0360fbdf4: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:19:54,243::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:19:54,244::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:19:54,244::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:19:54,245::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:19:54,245::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:19:54,245::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:19:54,246::task::495::TaskManager.Task::(_debug) Task 1e795189-534a-469b-9c94-7fd0360fbdf4: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:19:54,246::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:19:54,247::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:19:54,247::task::495::TaskManager.Task::(_debug) Task e96fb13e-457b-49cb-abe7-a249773757ef: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:19:54,248::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a996dafc-4b14-4716-a894-ec9a4fdbb7e5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:19:54,248::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:19:54,249::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:19:54,249::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a996dafc-4b14-4716-a894-ec9a4fdbb7e5`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:19:54,250::task::495::TaskManager.Task::(_debug) Task e96fb13e-457b-49cb-abe7-a249773757ef: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:19:54,250::task::495::TaskManager.Task::(_debug) Task e96fb13e-457b-49cb-abe7-a249773757ef: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:19:54,251::task::495::TaskManager.Task::(_debug) Task e96fb13e-457b-49cb-abe7-a249773757ef: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:19:54,251::task::495::TaskManager.Task::(_debug) Task e96fb13e-457b-49cb-abe7-a249773757ef: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:19:54,252::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:19:54,252::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:19:54,253::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:19:54,253::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:19:54,253::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:19:54,254::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:19:54,254::task::495::TaskManager.Task::(_debug) Task e96fb13e-457b-49cb-abe7-a249773757ef: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:19:54,255::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:19:55,604::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:19:56,117::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0804437 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:19:58,133::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:19:58,659::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0759298 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:20:00,675::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:20:01,151::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0591772 s, 17.3 MB/s\n'; <rc> = 0
>Thread-15914006::DEBUG::2013-11-20 03:20:01,239::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914006::INFO::2013-11-20 03:20:01,240::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914006::DEBUG::2013-11-20 03:20:01,240::task::495::TaskManager.Task::(_debug) Task 0a2679e9-0dd0-466a-b56a-34b1159fb3aa: moving from state init -> state preparing
>Thread-15914006::DEBUG::2013-11-20 03:20:01,241::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914006::DEBUG::2013-11-20 03:20:01,242::task::495::TaskManager.Task::(_debug) Task 0a2679e9-0dd0-466a-b56a-34b1159fb3aa: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914006::DEBUG::2013-11-20 03:20:01,242::task::495::TaskManager.Task::(_debug) Task 0a2679e9-0dd0-466a-b56a-34b1159fb3aa: moving from state preparing -> state finished
>Thread-15914006::DEBUG::2013-11-20 03:20:01,243::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914006::DEBUG::2013-11-20 03:20:01,243::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914006::DEBUG::2013-11-20 03:20:01,244::task::495::TaskManager.Task::(_debug) Task 0a2679e9-0dd0-466a-b56a-34b1159fb3aa: ref 0 aborting False
>Thread-15914006::INFO::2013-11-20 03:20:01,245::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914007::DEBUG::2013-11-20 03:20:01,252::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914007::INFO::2013-11-20 03:20:01,253::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914007::DEBUG::2013-11-20 03:20:01,254::task::495::TaskManager.Task::(_debug) Task 9367ec8c-119c-4eb3-ae55-0f7aa5f05588: moving from state init -> state preparing
>Thread-15914007::DEBUG::2013-11-20 03:20:01,255::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d8ddcb42-da01-4670-8d89-7c3d3c850f27`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914007::DEBUG::2013-11-20 03:20:01,255::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914007::DEBUG::2013-11-20 03:20:01,256::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914007::DEBUG::2013-11-20 03:20:01,256::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d8ddcb42-da01-4670-8d89-7c3d3c850f27`::Granted request
>Thread-15914007::DEBUG::2013-11-20 03:20:01,257::task::495::TaskManager.Task::(_debug) Task 9367ec8c-119c-4eb3-ae55-0f7aa5f05588: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914007::DEBUG::2013-11-20 03:20:01,258::task::495::TaskManager.Task::(_debug) Task 9367ec8c-119c-4eb3-ae55-0f7aa5f05588: ref 1 aborting False
>Thread-15914007::WARNING::2013-11-20 03:20:01,259::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914007::DEBUG::2013-11-20 03:20:01,260::task::495::TaskManager.Task::(_debug) Task 9367ec8c-119c-4eb3-ae55-0f7aa5f05588: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914007::DEBUG::2013-11-20 03:20:01,261::task::495::TaskManager.Task::(_debug) Task 9367ec8c-119c-4eb3-ae55-0f7aa5f05588: moving from state preparing -> state finished
>Thread-15914007::DEBUG::2013-11-20 03:20:01,261::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914007::DEBUG::2013-11-20 03:20:01,262::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914007::DEBUG::2013-11-20 03:20:01,263::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914007::DEBUG::2013-11-20 03:20:01,263::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914007::DEBUG::2013-11-20 03:20:01,264::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914007::DEBUG::2013-11-20 03:20:01,264::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914007::DEBUG::2013-11-20 03:20:01,265::task::495::TaskManager.Task::(_debug) Task 9367ec8c-119c-4eb3-ae55-0f7aa5f05588: ref 0 aborting False
>Thread-15914007::INFO::2013-11-20 03:20:01,266::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:20:03,165::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:20:03,706::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.100827 s, 10.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:20:05,722::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:20:06,272::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.119678 s, 8.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:20:08,287::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:20:08,827::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0842232 s, 12.2 MB/s\n'; <rc> = 0
>Thread-15914010::INFO::2013-11-20 03:20:09,463::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914010::DEBUG::2013-11-20 03:20:09,463::task::495::TaskManager.Task::(_debug) Task 58272913-b99a-4d45-b8ef-c803d4390fee: moving from state init -> state preparing
>Thread-15914010::DEBUG::2013-11-20 03:20:09,464::task::495::TaskManager.Task::(_debug) Task 58272913-b99a-4d45-b8ef-c803d4390fee: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000327110290527', 'lastCheck': 1384917606.922416, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00133490562439', 'lastCheck': 1384917603.3825209, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00126600265503', 'lastCheck': 1384917605.927319, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00577187538147', 'lastCheck': 1384917605.924561, 'valid': True, 'code': 0}}
>Thread-15914010::DEBUG::2013-11-20 03:20:09,465::task::495::TaskManager.Task::(_debug) Task 58272913-b99a-4d45-b8ef-c803d4390fee: moving from state preparing -> state finished
>Thread-15914010::DEBUG::2013-11-20 03:20:09,465::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914010::DEBUG::2013-11-20 03:20:09,465::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914010::DEBUG::2013-11-20 03:20:09,466::task::495::TaskManager.Task::(_debug) Task 58272913-b99a-4d45-b8ef-c803d4390fee: ref 0 aborting False
>Thread-15914010::INFO::2013-11-20 03:20:09,466::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000327110290527', 'lastCheck': 1384917606.922416, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00133490562439', 'lastCheck': 1384917603.3825209, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00126600265503', 'lastCheck': 1384917605.927319, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00577187538147', 'lastCheck': 1384917605.924561, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:20:10,844::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 03:20:11,040::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:20:11,042::task::495::TaskManager.Task::(_debug) Task 89f0a3fc-5a6f-4381-b87e-d34651919cd1: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:20:11,046::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`34b168f4-396c-4b7e-99f8-7f3aa11ef09b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:20:11,047::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:20:11,049::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:20:11,050::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`34b168f4-396c-4b7e-99f8-7f3aa11ef09b`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:20:11,051::task::495::TaskManager.Task::(_debug) Task 89f0a3fc-5a6f-4381-b87e-d34651919cd1: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:20:11,052::task::495::TaskManager.Task::(_debug) Task 89f0a3fc-5a6f-4381-b87e-d34651919cd1: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:20:11,053::task::495::TaskManager.Task::(_debug) Task 89f0a3fc-5a6f-4381-b87e-d34651919cd1: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:20:11,053::task::495::TaskManager.Task::(_debug) Task 89f0a3fc-5a6f-4381-b87e-d34651919cd1: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:20:11,054::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:20:11,055::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:20:11,055::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:20:11,056::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:20:11,057::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:20:11,057::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:20:11,058::task::495::TaskManager.Task::(_debug) Task 89f0a3fc-5a6f-4381-b87e-d34651919cd1: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:20:11,058::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:20:11,059::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:20:11,060::task::495::TaskManager.Task::(_debug) Task 7051634f-196e-458e-a293-b3659bf5cd03: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:20:11,061::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`1889dfc6-824c-4da3-93c9-95d2dbc15cbc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:20:11,062::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:20:11,062::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:20:11,063::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`1889dfc6-824c-4da3-93c9-95d2dbc15cbc`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:20:11,064::task::495::TaskManager.Task::(_debug) Task 7051634f-196e-458e-a293-b3659bf5cd03: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:20:11,065::task::495::TaskManager.Task::(_debug) Task 7051634f-196e-458e-a293-b3659bf5cd03: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:20:11,065::task::495::TaskManager.Task::(_debug) Task 7051634f-196e-458e-a293-b3659bf5cd03: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:20:11,066::task::495::TaskManager.Task::(_debug) Task 7051634f-196e-458e-a293-b3659bf5cd03: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:20:11,066::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:20:11,067::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:20:11,068::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:20:11,068::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:20:11,069::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:20:11,069::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:20:11,070::task::495::TaskManager.Task::(_debug) Task 7051634f-196e-458e-a293-b3659bf5cd03: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:20:11,070::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:20:11,071::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:20:11,072::task::495::TaskManager.Task::(_debug) Task 866b2280-451e-47f0-be03-6cdeabb56d50: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:20:11,073::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`40a87e90-4836-4016-bbd9-ea80e01bbba9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:20:11,074::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:20:11,074::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:20:11,075::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`40a87e90-4836-4016-bbd9-ea80e01bbba9`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:20:11,076::task::495::TaskManager.Task::(_debug) Task 866b2280-451e-47f0-be03-6cdeabb56d50: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:20:11,076::task::495::TaskManager.Task::(_debug) Task 866b2280-451e-47f0-be03-6cdeabb56d50: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:20:11,077::task::495::TaskManager.Task::(_debug) Task 866b2280-451e-47f0-be03-6cdeabb56d50: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:20:11,078::task::495::TaskManager.Task::(_debug) Task 866b2280-451e-47f0-be03-6cdeabb56d50: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:20:11,078::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:20:11,079::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:20:11,079::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:20:11,080::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:20:11,081::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:20:11,081::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:20:11,082::task::495::TaskManager.Task::(_debug) Task 866b2280-451e-47f0-be03-6cdeabb56d50: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:20:11,082::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:20:11,083::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:20:11,084::task::495::TaskManager.Task::(_debug) Task 9949b5a8-b99b-4256-8e77-800e022d8d7b: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:20:11,085::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`130522a1-fb78-4c73-a7e7-da20ad0ccc05`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:20:11,085::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:20:11,086::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:20:11,087::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`130522a1-fb78-4c73-a7e7-da20ad0ccc05`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:20:11,087::task::495::TaskManager.Task::(_debug) Task 9949b5a8-b99b-4256-8e77-800e022d8d7b: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:20:11,088::task::495::TaskManager.Task::(_debug) Task 9949b5a8-b99b-4256-8e77-800e022d8d7b: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:20:11,089::task::495::TaskManager.Task::(_debug) Task 9949b5a8-b99b-4256-8e77-800e022d8d7b: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:20:11,089::task::495::TaskManager.Task::(_debug) Task 9949b5a8-b99b-4256-8e77-800e022d8d7b: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:20:11,090::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:20:11,090::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:20:11,091::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:20:11,092::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:20:11,093::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:20:11,093::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:20:11,094::task::495::TaskManager.Task::(_debug) Task 9949b5a8-b99b-4256-8e77-800e022d8d7b: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:20:11,094::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:20:11,290::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0486868 s, 21.0 MB/s\n'; <rc> = 0
>Thread-15914012::DEBUG::2013-11-20 03:20:11,480::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914012::INFO::2013-11-20 03:20:11,481::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914012::DEBUG::2013-11-20 03:20:11,495::task::495::TaskManager.Task::(_debug) Task f54c7d0c-d121-469d-bf1e-6d4ace275e48: moving from state init -> state preparing
>Thread-15914012::DEBUG::2013-11-20 03:20:11,496::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914012::DEBUG::2013-11-20 03:20:11,496::task::495::TaskManager.Task::(_debug) Task f54c7d0c-d121-469d-bf1e-6d4ace275e48: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914012::DEBUG::2013-11-20 03:20:11,497::task::495::TaskManager.Task::(_debug) Task f54c7d0c-d121-469d-bf1e-6d4ace275e48: moving from state preparing -> state finished
>Thread-15914012::DEBUG::2013-11-20 03:20:11,497::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914012::DEBUG::2013-11-20 03:20:11,498::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914012::DEBUG::2013-11-20 03:20:11,498::task::495::TaskManager.Task::(_debug) Task f54c7d0c-d121-469d-bf1e-6d4ace275e48: ref 0 aborting False
>Thread-15914012::INFO::2013-11-20 03:20:11,499::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914013::DEBUG::2013-11-20 03:20:11,504::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914013::INFO::2013-11-20 03:20:11,505::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914013::DEBUG::2013-11-20 03:20:11,506::task::495::TaskManager.Task::(_debug) Task b7a40350-33f9-4509-8cad-f84bc9f579b2: moving from state init -> state preparing
>Thread-15914013::DEBUG::2013-11-20 03:20:11,516::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9e065e7e-7c50-4e04-b373-ed3daa7bd061`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914013::DEBUG::2013-11-20 03:20:11,517::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914013::DEBUG::2013-11-20 03:20:11,517::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914013::DEBUG::2013-11-20 03:20:11,518::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9e065e7e-7c50-4e04-b373-ed3daa7bd061`::Granted request
>Thread-15914013::DEBUG::2013-11-20 03:20:11,519::task::495::TaskManager.Task::(_debug) Task b7a40350-33f9-4509-8cad-f84bc9f579b2: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914013::DEBUG::2013-11-20 03:20:11,519::task::495::TaskManager.Task::(_debug) Task b7a40350-33f9-4509-8cad-f84bc9f579b2: ref 1 aborting False
>Thread-15914013::WARNING::2013-11-20 03:20:11,521::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914013::DEBUG::2013-11-20 03:20:11,522::task::495::TaskManager.Task::(_debug) Task b7a40350-33f9-4509-8cad-f84bc9f579b2: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914013::DEBUG::2013-11-20 03:20:11,523::task::495::TaskManager.Task::(_debug) Task b7a40350-33f9-4509-8cad-f84bc9f579b2: moving from state preparing -> state finished
>Thread-15914013::DEBUG::2013-11-20 03:20:11,523::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914013::DEBUG::2013-11-20 03:20:11,524::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914013::DEBUG::2013-11-20 03:20:11,524::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914013::DEBUG::2013-11-20 03:20:11,525::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914013::DEBUG::2013-11-20 03:20:11,525::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914013::DEBUG::2013-11-20 03:20:11,526::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914013::DEBUG::2013-11-20 03:20:11,526::task::495::TaskManager.Task::(_debug) Task b7a40350-33f9-4509-8cad-f84bc9f579b2: ref 0 aborting False
>Thread-15914013::INFO::2013-11-20 03:20:11,527::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:20:13,307::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:20:13,830::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0497964 s, 20.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:20:15,839::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:20:16,369::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0788496 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:20:18,380::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:20:18,888::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0751605 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:20:20,903::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 03:20:21,136::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:20:21,141::task::495::TaskManager.Task::(_debug) Task 1408587d-e27a-4abf-b2eb-eeb2efcccde5: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:20:21,148::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`22d8ad39-906e-4327-a973-0045b1c6af7f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:20:21,158::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:20:21,175::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:20:21,176::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`22d8ad39-906e-4327-a973-0045b1c6af7f`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:20:21,177::task::495::TaskManager.Task::(_debug) Task 1408587d-e27a-4abf-b2eb-eeb2efcccde5: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:20:21,178::task::495::TaskManager.Task::(_debug) Task 1408587d-e27a-4abf-b2eb-eeb2efcccde5: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:20:21,178::task::495::TaskManager.Task::(_debug) Task 1408587d-e27a-4abf-b2eb-eeb2efcccde5: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:20:21,179::task::495::TaskManager.Task::(_debug) Task 1408587d-e27a-4abf-b2eb-eeb2efcccde5: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:20:21,179::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:20:21,180::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:20:21,180::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:20:21,181::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:20:21,181::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:20:21,181::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:20:21,182::task::495::TaskManager.Task::(_debug) Task 1408587d-e27a-4abf-b2eb-eeb2efcccde5: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:20:21,182::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:20:21,459::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0831684 s, 12.3 MB/s\n'; <rc> = 0
>Thread-15914018::DEBUG::2013-11-20 03:20:21,961::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914018::INFO::2013-11-20 03:20:21,963::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914018::DEBUG::2013-11-20 03:20:21,973::task::495::TaskManager.Task::(_debug) Task 1207e0a7-41cc-4580-86ec-aed1c10cd2e4: moving from state init -> state preparing
>Thread-15914018::DEBUG::2013-11-20 03:20:21,974::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914018::DEBUG::2013-11-20 03:20:21,974::task::495::TaskManager.Task::(_debug) Task 1207e0a7-41cc-4580-86ec-aed1c10cd2e4: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914018::DEBUG::2013-11-20 03:20:21,975::task::495::TaskManager.Task::(_debug) Task 1207e0a7-41cc-4580-86ec-aed1c10cd2e4: moving from state preparing -> state finished
>Thread-15914018::DEBUG::2013-11-20 03:20:21,975::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914018::DEBUG::2013-11-20 03:20:21,976::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914018::DEBUG::2013-11-20 03:20:21,976::task::495::TaskManager.Task::(_debug) Task 1207e0a7-41cc-4580-86ec-aed1c10cd2e4: ref 0 aborting False
>Thread-15914018::INFO::2013-11-20 03:20:21,977::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914019::DEBUG::2013-11-20 03:20:21,981::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914019::INFO::2013-11-20 03:20:21,982::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914019::DEBUG::2013-11-20 03:20:21,983::task::495::TaskManager.Task::(_debug) Task 21392bca-a196-4f2c-bc36-eb9d28193d0c: moving from state init -> state preparing
>Thread-15914019::DEBUG::2013-11-20 03:20:21,984::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d97b70a1-cbae-492b-9af4-9895b138e9df`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914019::DEBUG::2013-11-20 03:20:21,984::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914019::DEBUG::2013-11-20 03:20:21,985::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914019::DEBUG::2013-11-20 03:20:21,985::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d97b70a1-cbae-492b-9af4-9895b138e9df`::Granted request
>Thread-15914019::DEBUG::2013-11-20 03:20:21,986::task::495::TaskManager.Task::(_debug) Task 21392bca-a196-4f2c-bc36-eb9d28193d0c: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914019::DEBUG::2013-11-20 03:20:21,987::task::495::TaskManager.Task::(_debug) Task 21392bca-a196-4f2c-bc36-eb9d28193d0c: ref 1 aborting False
>Thread-15914019::WARNING::2013-11-20 03:20:21,989::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914019::DEBUG::2013-11-20 03:20:21,989::task::495::TaskManager.Task::(_debug) Task 21392bca-a196-4f2c-bc36-eb9d28193d0c: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914019::DEBUG::2013-11-20 03:20:21,990::task::495::TaskManager.Task::(_debug) Task 21392bca-a196-4f2c-bc36-eb9d28193d0c: moving from state preparing -> state finished
>Thread-15914019::DEBUG::2013-11-20 03:20:21,990::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914019::DEBUG::2013-11-20 03:20:21,991::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914019::DEBUG::2013-11-20 03:20:21,991::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914019::DEBUG::2013-11-20 03:20:21,992::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914019::DEBUG::2013-11-20 03:20:21,993::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914019::DEBUG::2013-11-20 03:20:21,993::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914019::DEBUG::2013-11-20 03:20:21,994::task::495::TaskManager.Task::(_debug) Task 21392bca-a196-4f2c-bc36-eb9d28193d0c: ref 0 aborting False
>Thread-15914019::INFO::2013-11-20 03:20:21,994::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:20:23,473::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:20:23,994::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0824173 s, 12.4 MB/s\n'; <rc> = 0
>Thread-15914020::INFO::2013-11-20 03:20:24,883::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914020::DEBUG::2013-11-20 03:20:24,884::task::495::TaskManager.Task::(_debug) Task d44d6209-35b5-470a-8b88-4703d026c1c5: moving from state init -> state preparing
>Thread-15914020::DEBUG::2013-11-20 03:20:24,885::task::495::TaskManager.Task::(_debug) Task d44d6209-35b5-470a-8b88-4703d026c1c5: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000421047210693', 'lastCheck': 1384917617.0443921, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00104188919067', 'lastCheck': 1384917623.6470971, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00281882286072', 'lastCheck': 1384917616.0503371, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000764846801758', 'lastCheck': 1384917616.0410509, 'valid': True, 'code': 0}}
>Thread-15914020::DEBUG::2013-11-20 03:20:24,886::task::495::TaskManager.Task::(_debug) Task d44d6209-35b5-470a-8b88-4703d026c1c5: moving from state preparing -> state finished
>Thread-15914020::DEBUG::2013-11-20 03:20:24,886::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914020::DEBUG::2013-11-20 03:20:24,887::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914020::DEBUG::2013-11-20 03:20:24,887::task::495::TaskManager.Task::(_debug) Task d44d6209-35b5-470a-8b88-4703d026c1c5: ref 0 aborting False
>Thread-15914020::INFO::2013-11-20 03:20:24,888::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000421047210693', 'lastCheck': 1384917617.0443921, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00104188919067', 'lastCheck': 1384917623.6470971, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00281882286072', 'lastCheck': 1384917616.0503371, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000764846801758', 'lastCheck': 1384917616.0410509, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:20:26,010::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:20:26,537::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0675244 s, 15.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:20:28,551::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:20:29,119::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.075188 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:20:31,134::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:20:31,693::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.102537 s, 10.0 MB/s\n'; <rc> = 0
>Thread-15914024::DEBUG::2013-11-20 03:20:32,280::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914024::INFO::2013-11-20 03:20:32,282::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914024::DEBUG::2013-11-20 03:20:32,283::task::495::TaskManager.Task::(_debug) Task 534a55bd-4fd2-4833-ace8-9e28427c180e: moving from state init -> state preparing
>Thread-15914024::DEBUG::2013-11-20 03:20:32,285::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914024::DEBUG::2013-11-20 03:20:32,286::task::495::TaskManager.Task::(_debug) Task 534a55bd-4fd2-4833-ace8-9e28427c180e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914024::DEBUG::2013-11-20 03:20:32,287::task::495::TaskManager.Task::(_debug) Task 534a55bd-4fd2-4833-ace8-9e28427c180e: moving from state preparing -> state finished
>Thread-15914024::DEBUG::2013-11-20 03:20:32,288::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914024::DEBUG::2013-11-20 03:20:32,288::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914024::DEBUG::2013-11-20 03:20:32,289::task::495::TaskManager.Task::(_debug) Task 534a55bd-4fd2-4833-ace8-9e28427c180e: ref 0 aborting False
>Thread-15914024::INFO::2013-11-20 03:20:32,289::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914025::DEBUG::2013-11-20 03:20:32,294::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914025::INFO::2013-11-20 03:20:32,295::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914025::DEBUG::2013-11-20 03:20:32,296::task::495::TaskManager.Task::(_debug) Task 89280b1a-afc7-45d0-a726-b625bfeaa3cf: moving from state init -> state preparing
>Thread-15914025::DEBUG::2013-11-20 03:20:32,297::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3149a381-fd1d-4f1a-b128-39700ae73241`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914025::DEBUG::2013-11-20 03:20:32,298::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914025::DEBUG::2013-11-20 03:20:32,299::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914025::DEBUG::2013-11-20 03:20:32,300::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3149a381-fd1d-4f1a-b128-39700ae73241`::Granted request
>Thread-15914025::DEBUG::2013-11-20 03:20:32,301::task::495::TaskManager.Task::(_debug) Task 89280b1a-afc7-45d0-a726-b625bfeaa3cf: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914025::DEBUG::2013-11-20 03:20:32,302::task::495::TaskManager.Task::(_debug) Task 89280b1a-afc7-45d0-a726-b625bfeaa3cf: ref 1 aborting False
>Thread-15914025::WARNING::2013-11-20 03:20:32,304::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914025::DEBUG::2013-11-20 03:20:32,304::task::495::TaskManager.Task::(_debug) Task 89280b1a-afc7-45d0-a726-b625bfeaa3cf: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914025::DEBUG::2013-11-20 03:20:32,305::task::495::TaskManager.Task::(_debug) Task 89280b1a-afc7-45d0-a726-b625bfeaa3cf: moving from state preparing -> state finished
>Thread-15914025::DEBUG::2013-11-20 03:20:32,306::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914025::DEBUG::2013-11-20 03:20:32,306::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914025::DEBUG::2013-11-20 03:20:32,307::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914025::DEBUG::2013-11-20 03:20:32,308::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914025::DEBUG::2013-11-20 03:20:32,308::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914025::DEBUG::2013-11-20 03:20:32,309::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914025::DEBUG::2013-11-20 03:20:32,309::task::495::TaskManager.Task::(_debug) Task 89280b1a-afc7-45d0-a726-b625bfeaa3cf: ref 0 aborting False
>Thread-15914025::INFO::2013-11-20 03:20:32,310::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:20:33,709::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:20:34,142::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0450451 s, 22.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:20:36,153::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:20:36,749::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.109806 s, 9.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:20:38,762::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 03:20:38,977::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:20:38,980::task::495::TaskManager.Task::(_debug) Task 961399e5-a6f4-4183-b95e-6da64303ed56: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:20:38,982::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d1177cdb-e682-495a-aa28-2af89840378f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:20:38,983::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:20:38,984::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:20:38,985::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d1177cdb-e682-495a-aa28-2af89840378f`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:20:38,986::task::495::TaskManager.Task::(_debug) Task 961399e5-a6f4-4183-b95e-6da64303ed56: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:20:38,987::task::495::TaskManager.Task::(_debug) Task 961399e5-a6f4-4183-b95e-6da64303ed56: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:20:38,987::task::495::TaskManager.Task::(_debug) Task 961399e5-a6f4-4183-b95e-6da64303ed56: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:20:38,988::task::495::TaskManager.Task::(_debug) Task 961399e5-a6f4-4183-b95e-6da64303ed56: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:20:38,989::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:20:38,990::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:20:38,990::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:20:38,991::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:20:38,992::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:20:38,992::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:20:38,993::task::495::TaskManager.Task::(_debug) Task 961399e5-a6f4-4183-b95e-6da64303ed56: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:20:38,994::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:20:39,294::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0933445 s, 11.0 MB/s\n'; <rc> = 0
>Thread-15914028::INFO::2013-11-20 03:20:40,490::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914028::DEBUG::2013-11-20 03:20:40,491::task::495::TaskManager.Task::(_debug) Task e26e8354-38c6-459f-801b-f9d2ca05410b: moving from state init -> state preparing
>Thread-15914028::DEBUG::2013-11-20 03:20:40,492::task::495::TaskManager.Task::(_debug) Task e26e8354-38c6-459f-801b-f9d2ca05410b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00049901008606', 'lastCheck': 1384917637.4107251, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.010794878006', 'lastCheck': 1384917633.93749, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0103280544281', 'lastCheck': 1384917636.439688, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0166292190552', 'lastCheck': 1384917636.426199, 'valid': True, 'code': 0}}
>Thread-15914028::DEBUG::2013-11-20 03:20:40,493::task::495::TaskManager.Task::(_debug) Task e26e8354-38c6-459f-801b-f9d2ca05410b: moving from state preparing -> state finished
>Thread-15914028::DEBUG::2013-11-20 03:20:40,493::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914028::DEBUG::2013-11-20 03:20:40,494::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914028::DEBUG::2013-11-20 03:20:40,495::task::495::TaskManager.Task::(_debug) Task e26e8354-38c6-459f-801b-f9d2ca05410b: ref 0 aborting False
>Thread-15914028::INFO::2013-11-20 03:20:40,495::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00049901008606', 'lastCheck': 1384917637.4107251, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.010794878006', 'lastCheck': 1384917633.93749, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0103280544281', 'lastCheck': 1384917636.439688, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0166292190552', 'lastCheck': 1384917636.426199, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:20:41,310::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 03:20:41,522::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:20:41,525::task::495::TaskManager.Task::(_debug) Task e60ad848-516b-4d67-8ffc-e96c3a7c7c67: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:20:41,528::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8f1d8626-e6f3-47f0-badd-0cce2b77ca39`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:20:41,530::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:20:41,531::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:20:41,532::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8f1d8626-e6f3-47f0-badd-0cce2b77ca39`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:20:41,533::task::495::TaskManager.Task::(_debug) Task e60ad848-516b-4d67-8ffc-e96c3a7c7c67: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:20:41,533::task::495::TaskManager.Task::(_debug) Task e60ad848-516b-4d67-8ffc-e96c3a7c7c67: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:20:41,534::task::495::TaskManager.Task::(_debug) Task e60ad848-516b-4d67-8ffc-e96c3a7c7c67: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:20:41,534::task::495::TaskManager.Task::(_debug) Task e60ad848-516b-4d67-8ffc-e96c3a7c7c67: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:20:41,534::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:20:41,535::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:20:41,535::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:20:41,536::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:20:41,536::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:20:41,537::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:20:41,537::task::495::TaskManager.Task::(_debug) Task e60ad848-516b-4d67-8ffc-e96c3a7c7c67: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:20:41,538::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:20:41,832::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0726305 s, 14.1 MB/s\n'; <rc> = 0
>Thread-15914030::DEBUG::2013-11-20 03:20:42,528::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914030::INFO::2013-11-20 03:20:42,529::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914030::DEBUG::2013-11-20 03:20:42,530::task::495::TaskManager.Task::(_debug) Task 5f55291c-4d94-414f-a7b9-01861bdab898: moving from state init -> state preparing
>Thread-15914030::DEBUG::2013-11-20 03:20:42,531::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914030::DEBUG::2013-11-20 03:20:42,531::task::495::TaskManager.Task::(_debug) Task 5f55291c-4d94-414f-a7b9-01861bdab898: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914030::DEBUG::2013-11-20 03:20:42,532::task::495::TaskManager.Task::(_debug) Task 5f55291c-4d94-414f-a7b9-01861bdab898: moving from state preparing -> state finished
>Thread-15914030::DEBUG::2013-11-20 03:20:42,533::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914030::DEBUG::2013-11-20 03:20:42,533::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914030::DEBUG::2013-11-20 03:20:42,534::task::495::TaskManager.Task::(_debug) Task 5f55291c-4d94-414f-a7b9-01861bdab898: ref 0 aborting False
>Thread-15914030::INFO::2013-11-20 03:20:42,535::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914031::DEBUG::2013-11-20 03:20:42,540::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914031::INFO::2013-11-20 03:20:42,541::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914031::DEBUG::2013-11-20 03:20:42,542::task::495::TaskManager.Task::(_debug) Task 243b1d0c-256d-44fb-a60b-8dc50d176a5b: moving from state init -> state preparing
>Thread-15914031::DEBUG::2013-11-20 03:20:42,543::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6ddf80b0-6c16-4469-b847-2136dfc5d785`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914031::DEBUG::2013-11-20 03:20:42,544::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914031::DEBUG::2013-11-20 03:20:42,545::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914031::DEBUG::2013-11-20 03:20:42,545::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6ddf80b0-6c16-4469-b847-2136dfc5d785`::Granted request
>Thread-15914031::DEBUG::2013-11-20 03:20:42,547::task::495::TaskManager.Task::(_debug) Task 243b1d0c-256d-44fb-a60b-8dc50d176a5b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914031::DEBUG::2013-11-20 03:20:42,547::task::495::TaskManager.Task::(_debug) Task 243b1d0c-256d-44fb-a60b-8dc50d176a5b: ref 1 aborting False
>Thread-15914031::WARNING::2013-11-20 03:20:42,549::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914031::DEBUG::2013-11-20 03:20:42,550::task::495::TaskManager.Task::(_debug) Task 243b1d0c-256d-44fb-a60b-8dc50d176a5b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914031::DEBUG::2013-11-20 03:20:42,551::task::495::TaskManager.Task::(_debug) Task 243b1d0c-256d-44fb-a60b-8dc50d176a5b: moving from state preparing -> state finished
>Thread-15914031::DEBUG::2013-11-20 03:20:42,552::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914031::DEBUG::2013-11-20 03:20:42,552::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914031::DEBUG::2013-11-20 03:20:42,553::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914031::DEBUG::2013-11-20 03:20:42,554::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914031::DEBUG::2013-11-20 03:20:42,554::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914031::DEBUG::2013-11-20 03:20:42,555::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914031::DEBUG::2013-11-20 03:20:42,555::task::495::TaskManager.Task::(_debug) Task 243b1d0c-256d-44fb-a60b-8dc50d176a5b: ref 0 aborting False
>Thread-15914031::INFO::2013-11-20 03:20:42,556::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:20:43,847::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:20:44,399::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0719003 s, 14.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:20:46,413::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 03:20:46,649::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:20:46,656::task::495::TaskManager.Task::(_debug) Task 89e05ea0-2526-4913-b65d-93d120698324: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:20:46,665::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`80cde197-80b3-4034-b9a7-4d1a86ae480d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:20:46,668::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:20:46,672::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:20:46,674::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`80cde197-80b3-4034-b9a7-4d1a86ae480d`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:20:46,676::task::495::TaskManager.Task::(_debug) Task 89e05ea0-2526-4913-b65d-93d120698324: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:20:46,677::task::495::TaskManager.Task::(_debug) Task 89e05ea0-2526-4913-b65d-93d120698324: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:20:46,678::task::495::TaskManager.Task::(_debug) Task 89e05ea0-2526-4913-b65d-93d120698324: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:20:46,679::task::495::TaskManager.Task::(_debug) Task 89e05ea0-2526-4913-b65d-93d120698324: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:20:46,680::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:20:46,681::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:20:46,682::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:20:46,683::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:20:46,684::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:20:46,685::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:20:46,686::task::495::TaskManager.Task::(_debug) Task 89e05ea0-2526-4913-b65d-93d120698324: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:20:46,687::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:20:46,969::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0844453 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:20:48,984::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:20:49,531::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0822429 s, 12.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:20:51,547::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:20:52,098::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0767716 s, 13.3 MB/s\n'; <rc> = 0
>Thread-15914035::DEBUG::2013-11-20 03:20:52,749::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914035::INFO::2013-11-20 03:20:52,751::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914035::DEBUG::2013-11-20 03:20:52,751::task::495::TaskManager.Task::(_debug) Task ea5e4fa8-a88d-4fa8-ae6b-da8503ead8cf: moving from state init -> state preparing
>Thread-15914035::DEBUG::2013-11-20 03:20:52,752::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914035::DEBUG::2013-11-20 03:20:52,753::task::495::TaskManager.Task::(_debug) Task ea5e4fa8-a88d-4fa8-ae6b-da8503ead8cf: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914035::DEBUG::2013-11-20 03:20:52,754::task::495::TaskManager.Task::(_debug) Task ea5e4fa8-a88d-4fa8-ae6b-da8503ead8cf: moving from state preparing -> state finished
>Thread-15914035::DEBUG::2013-11-20 03:20:52,754::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914035::DEBUG::2013-11-20 03:20:52,755::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914035::DEBUG::2013-11-20 03:20:52,756::task::495::TaskManager.Task::(_debug) Task ea5e4fa8-a88d-4fa8-ae6b-da8503ead8cf: ref 0 aborting False
>Thread-15914035::INFO::2013-11-20 03:20:52,756::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914036::DEBUG::2013-11-20 03:20:52,761::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914036::INFO::2013-11-20 03:20:52,762::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914036::DEBUG::2013-11-20 03:20:52,763::task::495::TaskManager.Task::(_debug) Task 303df4f6-3f3a-4037-b7ac-af03a8c7b52f: moving from state init -> state preparing
>Thread-15914036::DEBUG::2013-11-20 03:20:52,764::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6e479bb4-be1a-42c6-9e81-4c4f60b58d90`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914036::DEBUG::2013-11-20 03:20:52,765::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914036::DEBUG::2013-11-20 03:20:52,766::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914036::DEBUG::2013-11-20 03:20:52,766::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6e479bb4-be1a-42c6-9e81-4c4f60b58d90`::Granted request
>Thread-15914036::DEBUG::2013-11-20 03:20:52,767::task::495::TaskManager.Task::(_debug) Task 303df4f6-3f3a-4037-b7ac-af03a8c7b52f: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914036::DEBUG::2013-11-20 03:20:52,768::task::495::TaskManager.Task::(_debug) Task 303df4f6-3f3a-4037-b7ac-af03a8c7b52f: ref 1 aborting False
>Thread-15914036::WARNING::2013-11-20 03:20:52,771::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914036::DEBUG::2013-11-20 03:20:52,772::task::495::TaskManager.Task::(_debug) Task 303df4f6-3f3a-4037-b7ac-af03a8c7b52f: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914036::DEBUG::2013-11-20 03:20:52,773::task::495::TaskManager.Task::(_debug) Task 303df4f6-3f3a-4037-b7ac-af03a8c7b52f: moving from state preparing -> state finished
>Thread-15914036::DEBUG::2013-11-20 03:20:52,774::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914036::DEBUG::2013-11-20 03:20:52,774::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914036::DEBUG::2013-11-20 03:20:52,775::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914036::DEBUG::2013-11-20 03:20:52,776::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914036::DEBUG::2013-11-20 03:20:52,777::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914036::DEBUG::2013-11-20 03:20:52,777::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914036::DEBUG::2013-11-20 03:20:52,778::task::495::TaskManager.Task::(_debug) Task 303df4f6-3f3a-4037-b7ac-af03a8c7b52f: ref 0 aborting False
>Thread-15914036::INFO::2013-11-20 03:20:52,779::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:20:54,112::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:20:54,651::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0615763 s, 16.6 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 03:20:55,287::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:20:55,288::task::495::TaskManager.Task::(_debug) Task 0b2c3537-1994-428c-be0f-8e8242efc19b: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:20:55,290::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4ef2b657-a98d-4b5e-88eb-4df61d6c4549`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:20:55,290::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:20:55,291::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:20:55,292::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4ef2b657-a98d-4b5e-88eb-4df61d6c4549`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:20:55,293::task::495::TaskManager.Task::(_debug) Task 0b2c3537-1994-428c-be0f-8e8242efc19b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:20:55,294::task::495::TaskManager.Task::(_debug) Task 0b2c3537-1994-428c-be0f-8e8242efc19b: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:20:55,295::task::495::TaskManager.Task::(_debug) Task 0b2c3537-1994-428c-be0f-8e8242efc19b: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:20:55,295::task::495::TaskManager.Task::(_debug) Task 0b2c3537-1994-428c-be0f-8e8242efc19b: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:20:55,296::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:20:55,297::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:20:55,297::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:20:55,298::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:20:55,298::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:20:55,299::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:20:55,300::task::495::TaskManager.Task::(_debug) Task 0b2c3537-1994-428c-be0f-8e8242efc19b: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:20:55,300::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:20:55,301::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:20:55,302::task::495::TaskManager.Task::(_debug) Task c0a826bb-e0f0-4fc5-860c-ccae9afbbb2b: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:20:55,303::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ff5ff32e-2a18-4c6b-8d4e-e3088e8b7796`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:20:55,304::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:20:55,304::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:20:55,305::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ff5ff32e-2a18-4c6b-8d4e-e3088e8b7796`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:20:55,306::task::495::TaskManager.Task::(_debug) Task c0a826bb-e0f0-4fc5-860c-ccae9afbbb2b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:20:55,307::task::495::TaskManager.Task::(_debug) Task c0a826bb-e0f0-4fc5-860c-ccae9afbbb2b: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:20:55,307::task::495::TaskManager.Task::(_debug) Task c0a826bb-e0f0-4fc5-860c-ccae9afbbb2b: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:20:55,308::task::495::TaskManager.Task::(_debug) Task c0a826bb-e0f0-4fc5-860c-ccae9afbbb2b: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:20:55,309::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:20:55,309::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:20:55,310::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:20:55,311::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:20:55,312::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:20:55,312::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:20:55,313::task::495::TaskManager.Task::(_debug) Task c0a826bb-e0f0-4fc5-860c-ccae9afbbb2b: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:20:55,314::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15914038::INFO::2013-11-20 03:20:55,950::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914038::DEBUG::2013-11-20 03:20:55,951::task::495::TaskManager.Task::(_debug) Task 0657b7cf-2320-4cb0-8e17-352623486176: moving from state init -> state preparing
>Thread-15914038::DEBUG::2013-11-20 03:20:55,952::task::495::TaskManager.Task::(_debug) Task 0657b7cf-2320-4cb0-8e17-352623486176: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000513076782227', 'lastCheck': 1384917647.6363089, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00146794319153', 'lastCheck': 1384917654.3721011, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00249600410461', 'lastCheck': 1384917646.6737411, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00871205329895', 'lastCheck': 1384917646.664993, 'valid': True, 'code': 0}}
>Thread-15914038::DEBUG::2013-11-20 03:20:55,953::task::495::TaskManager.Task::(_debug) Task 0657b7cf-2320-4cb0-8e17-352623486176: moving from state preparing -> state finished
>Thread-15914038::DEBUG::2013-11-20 03:20:55,954::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914038::DEBUG::2013-11-20 03:20:55,954::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914038::DEBUG::2013-11-20 03:20:55,955::task::495::TaskManager.Task::(_debug) Task 0657b7cf-2320-4cb0-8e17-352623486176: ref 0 aborting False
>Thread-15914038::INFO::2013-11-20 03:20:55,955::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000513076782227', 'lastCheck': 1384917647.6363089, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00146794319153', 'lastCheck': 1384917654.3721011, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00249600410461', 'lastCheck': 1384917646.6737411, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00871205329895', 'lastCheck': 1384917646.664993, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:20:56,661::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:20:57,226::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0790887 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:20:59,241::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:20:59,831::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0827599 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:21:01,845::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:21:02,376::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0746215 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15914042::DEBUG::2013-11-20 03:21:03,017::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914042::INFO::2013-11-20 03:21:03,019::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914042::DEBUG::2013-11-20 03:21:03,020::task::495::TaskManager.Task::(_debug) Task ba255771-2527-4ece-8d08-cc23c5faa6e6: moving from state init -> state preparing
>Thread-15914042::DEBUG::2013-11-20 03:21:03,021::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914042::DEBUG::2013-11-20 03:21:03,022::task::495::TaskManager.Task::(_debug) Task ba255771-2527-4ece-8d08-cc23c5faa6e6: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914042::DEBUG::2013-11-20 03:21:03,022::task::495::TaskManager.Task::(_debug) Task ba255771-2527-4ece-8d08-cc23c5faa6e6: moving from state preparing -> state finished
>Thread-15914042::DEBUG::2013-11-20 03:21:03,023::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914042::DEBUG::2013-11-20 03:21:03,024::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914042::DEBUG::2013-11-20 03:21:03,024::task::495::TaskManager.Task::(_debug) Task ba255771-2527-4ece-8d08-cc23c5faa6e6: ref 0 aborting False
>Thread-15914042::INFO::2013-11-20 03:21:03,025::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914043::DEBUG::2013-11-20 03:21:03,029::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914043::INFO::2013-11-20 03:21:03,030::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914043::DEBUG::2013-11-20 03:21:03,031::task::495::TaskManager.Task::(_debug) Task 5f3e2d8e-a154-4cb1-8282-258336de673f: moving from state init -> state preparing
>Thread-15914043::DEBUG::2013-11-20 03:21:03,032::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ea6c56f1-d533-4125-a295-838752d6b8da`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914043::DEBUG::2013-11-20 03:21:03,033::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914043::DEBUG::2013-11-20 03:21:03,033::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914043::DEBUG::2013-11-20 03:21:03,034::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ea6c56f1-d533-4125-a295-838752d6b8da`::Granted request
>Thread-15914043::DEBUG::2013-11-20 03:21:03,035::task::495::TaskManager.Task::(_debug) Task 5f3e2d8e-a154-4cb1-8282-258336de673f: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914043::DEBUG::2013-11-20 03:21:03,035::task::495::TaskManager.Task::(_debug) Task 5f3e2d8e-a154-4cb1-8282-258336de673f: ref 1 aborting False
>Thread-15914043::WARNING::2013-11-20 03:21:03,037::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914043::DEBUG::2013-11-20 03:21:03,038::task::495::TaskManager.Task::(_debug) Task 5f3e2d8e-a154-4cb1-8282-258336de673f: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914043::DEBUG::2013-11-20 03:21:03,039::task::495::TaskManager.Task::(_debug) Task 5f3e2d8e-a154-4cb1-8282-258336de673f: moving from state preparing -> state finished
>Thread-15914043::DEBUG::2013-11-20 03:21:03,040::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914043::DEBUG::2013-11-20 03:21:03,040::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914043::DEBUG::2013-11-20 03:21:03,041::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914043::DEBUG::2013-11-20 03:21:03,042::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914043::DEBUG::2013-11-20 03:21:03,042::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914043::DEBUG::2013-11-20 03:21:03,043::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914043::DEBUG::2013-11-20 03:21:03,043::task::495::TaskManager.Task::(_debug) Task 5f3e2d8e-a154-4cb1-8282-258336de673f: ref 0 aborting False
>Thread-15914043::INFO::2013-11-20 03:21:03,044::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:21:04,391::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913659::DEBUG::2013-11-20 03:21:04,816::guestIF::97::vm.Vm::(_handleMessage) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::u'AOS2Batch@brd-klee.dk'
>Dummy-15912651::DEBUG::2013-11-20 03:21:04,987::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0853172 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:21:07,006::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:21:07,564::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.077339 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:21:09,579::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:21:10,093::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0711724 s, 14.4 MB/s\n'; <rc> = 0
>Thread-15914046::INFO::2013-11-20 03:21:11,612::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914046::DEBUG::2013-11-20 03:21:11,613::task::495::TaskManager.Task::(_debug) Task 45e14a2e-29e1-4f82-8c9b-7ef0cb05adec: moving from state init -> state preparing
>Thread-15914046::DEBUG::2013-11-20 03:21:11,614::task::495::TaskManager.Task::(_debug) Task 45e14a2e-29e1-4f82-8c9b-7ef0cb05adec: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000303030014038', 'lastCheck': 1384917668.2381959, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00154900550842', 'lastCheck': 1384917664.6252999, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00170707702637', 'lastCheck': 1384917667.245239, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00131297111511', 'lastCheck': 1384917667.2373669, 'valid': True, 'code': 0}}
>Thread-15914046::DEBUG::2013-11-20 03:21:11,615::task::495::TaskManager.Task::(_debug) Task 45e14a2e-29e1-4f82-8c9b-7ef0cb05adec: moving from state preparing -> state finished
>Thread-15914046::DEBUG::2013-11-20 03:21:11,615::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914046::DEBUG::2013-11-20 03:21:11,616::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914046::DEBUG::2013-11-20 03:21:11,617::task::495::TaskManager.Task::(_debug) Task 45e14a2e-29e1-4f82-8c9b-7ef0cb05adec: ref 0 aborting False
>Thread-15914046::INFO::2013-11-20 03:21:11,617::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000303030014038', 'lastCheck': 1384917668.2381959, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00154900550842', 'lastCheck': 1384917664.6252999, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00170707702637', 'lastCheck': 1384917667.245239, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00131297111511', 'lastCheck': 1384917667.2373669, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:21:12,102::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 03:21:12,349::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:21:12,375::task::495::TaskManager.Task::(_debug) Task 688eee35-ee33-45a9-a835-13de89c77170: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:21:12,384::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8ec865ed-a1f0-4882-8e77-29b8f78a8a3e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:21:12,385::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:21:12,386::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:21:12,387::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8ec865ed-a1f0-4882-8e77-29b8f78a8a3e`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:21:12,388::task::495::TaskManager.Task::(_debug) Task 688eee35-ee33-45a9-a835-13de89c77170: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:21:12,389::task::495::TaskManager.Task::(_debug) Task 688eee35-ee33-45a9-a835-13de89c77170: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:21:12,390::task::495::TaskManager.Task::(_debug) Task 688eee35-ee33-45a9-a835-13de89c77170: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:21:12,391::task::495::TaskManager.Task::(_debug) Task 688eee35-ee33-45a9-a835-13de89c77170: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:21:12,391::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:21:12,392::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:21:12,393::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:21:12,393::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:21:12,394::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:21:12,395::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:21:12,396::task::495::TaskManager.Task::(_debug) Task 688eee35-ee33-45a9-a835-13de89c77170: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:21:12,396::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:21:12,398::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:21:12,399::task::495::TaskManager.Task::(_debug) Task d4de1189-f0f4-4419-b08b-bd5d5ef4cc8b: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:21:12,400::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8658c9da-08da-4274-a48b-d24f4ba4a3d7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:21:12,400::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:21:12,401::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:21:12,402::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8658c9da-08da-4274-a48b-d24f4ba4a3d7`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:21:12,403::task::495::TaskManager.Task::(_debug) Task d4de1189-f0f4-4419-b08b-bd5d5ef4cc8b: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:21:12,404::task::495::TaskManager.Task::(_debug) Task d4de1189-f0f4-4419-b08b-bd5d5ef4cc8b: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:21:12,405::task::495::TaskManager.Task::(_debug) Task d4de1189-f0f4-4419-b08b-bd5d5ef4cc8b: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:21:12,405::task::495::TaskManager.Task::(_debug) Task d4de1189-f0f4-4419-b08b-bd5d5ef4cc8b: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:21:12,406::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:21:12,407::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:21:12,408::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:21:12,409::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:21:12,410::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:21:12,411::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:21:12,412::task::495::TaskManager.Task::(_debug) Task d4de1189-f0f4-4419-b08b-bd5d5ef4cc8b: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:21:12,413::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:21:12,415::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:21:12,417::task::495::TaskManager.Task::(_debug) Task 59050bb4-ee60-4a36-8eca-df2a0893276d: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:21:12,418::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f422bdf9-a81c-4deb-8d9f-9b06a1665381`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:21:12,421::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:21:12,422::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:21:12,422::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f422bdf9-a81c-4deb-8d9f-9b06a1665381`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:21:12,425::task::495::TaskManager.Task::(_debug) Task 59050bb4-ee60-4a36-8eca-df2a0893276d: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:21:12,429::task::495::TaskManager.Task::(_debug) Task 59050bb4-ee60-4a36-8eca-df2a0893276d: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:21:12,432::task::495::TaskManager.Task::(_debug) Task 59050bb4-ee60-4a36-8eca-df2a0893276d: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:21:12,435::task::495::TaskManager.Task::(_debug) Task 59050bb4-ee60-4a36-8eca-df2a0893276d: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:21:12,437::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:21:12,437::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:21:12,438::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:21:12,439::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:21:12,439::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:21:12,440::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:21:12,440::task::495::TaskManager.Task::(_debug) Task 59050bb4-ee60-4a36-8eca-df2a0893276d: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:21:12,441::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:21:12,442::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:21:12,442::task::495::TaskManager.Task::(_debug) Task 5d23e1f7-c09d-4dc1-86ee-150a630dde2d: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:21:12,443::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`aaacb4e9-571d-40bf-ae95-9c92a6d300f8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:21:12,444::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:21:12,445::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:21:12,445::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`aaacb4e9-571d-40bf-ae95-9c92a6d300f8`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:21:12,446::task::495::TaskManager.Task::(_debug) Task 5d23e1f7-c09d-4dc1-86ee-150a630dde2d: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:21:12,447::task::495::TaskManager.Task::(_debug) Task 5d23e1f7-c09d-4dc1-86ee-150a630dde2d: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:21:12,447::task::495::TaskManager.Task::(_debug) Task 5d23e1f7-c09d-4dc1-86ee-150a630dde2d: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:21:12,448::task::495::TaskManager.Task::(_debug) Task 5d23e1f7-c09d-4dc1-86ee-150a630dde2d: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:21:12,449::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:21:12,450::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:21:12,450::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:21:12,451::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:21:12,452::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:21:12,452::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:21:12,453::task::495::TaskManager.Task::(_debug) Task 5d23e1f7-c09d-4dc1-86ee-150a630dde2d: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:21:12,454::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:21:12,674::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0814751 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15914048::DEBUG::2013-11-20 03:21:13,316::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914048::INFO::2013-11-20 03:21:13,318::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914048::DEBUG::2013-11-20 03:21:13,318::task::495::TaskManager.Task::(_debug) Task 492e6444-35a9-4262-b9a0-1bc30bb67268: moving from state init -> state preparing
>Thread-15914048::DEBUG::2013-11-20 03:21:13,319::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914048::DEBUG::2013-11-20 03:21:13,320::task::495::TaskManager.Task::(_debug) Task 492e6444-35a9-4262-b9a0-1bc30bb67268: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914048::DEBUG::2013-11-20 03:21:13,320::task::495::TaskManager.Task::(_debug) Task 492e6444-35a9-4262-b9a0-1bc30bb67268: moving from state preparing -> state finished
>Thread-15914048::DEBUG::2013-11-20 03:21:13,321::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914048::DEBUG::2013-11-20 03:21:13,321::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914048::DEBUG::2013-11-20 03:21:13,322::task::495::TaskManager.Task::(_debug) Task 492e6444-35a9-4262-b9a0-1bc30bb67268: ref 0 aborting False
>Thread-15914048::INFO::2013-11-20 03:21:13,322::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914049::DEBUG::2013-11-20 03:21:13,379::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914049::INFO::2013-11-20 03:21:13,380::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914049::DEBUG::2013-11-20 03:21:13,381::task::495::TaskManager.Task::(_debug) Task 45622d33-e363-4c28-81ae-5ffc722054b4: moving from state init -> state preparing
>Thread-15914049::DEBUG::2013-11-20 03:21:13,382::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9de4f802-a701-424f-a194-4ad66559ce04`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914049::DEBUG::2013-11-20 03:21:13,382::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914049::DEBUG::2013-11-20 03:21:13,383::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914049::DEBUG::2013-11-20 03:21:13,384::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9de4f802-a701-424f-a194-4ad66559ce04`::Granted request
>Thread-15914049::DEBUG::2013-11-20 03:21:13,384::task::495::TaskManager.Task::(_debug) Task 45622d33-e363-4c28-81ae-5ffc722054b4: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914049::DEBUG::2013-11-20 03:21:13,385::task::495::TaskManager.Task::(_debug) Task 45622d33-e363-4c28-81ae-5ffc722054b4: ref 1 aborting False
>Thread-15914049::WARNING::2013-11-20 03:21:13,387::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914049::DEBUG::2013-11-20 03:21:13,388::task::495::TaskManager.Task::(_debug) Task 45622d33-e363-4c28-81ae-5ffc722054b4: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914049::DEBUG::2013-11-20 03:21:13,389::task::495::TaskManager.Task::(_debug) Task 45622d33-e363-4c28-81ae-5ffc722054b4: moving from state preparing -> state finished
>Thread-15914049::DEBUG::2013-11-20 03:21:13,389::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914049::DEBUG::2013-11-20 03:21:13,390::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914049::DEBUG::2013-11-20 03:21:13,390::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914049::DEBUG::2013-11-20 03:21:13,391::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914049::DEBUG::2013-11-20 03:21:13,392::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914049::DEBUG::2013-11-20 03:21:13,392::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914049::DEBUG::2013-11-20 03:21:13,393::task::495::TaskManager.Task::(_debug) Task 45622d33-e363-4c28-81ae-5ffc722054b4: ref 0 aborting False
>Thread-15914049::INFO::2013-11-20 03:21:13,393::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:21:14,690::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:21:15,256::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0812706 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:21:17,272::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:21:17,843::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0833439 s, 12.3 MB/s\n'; <rc> = 0
>Thread-15913640::DEBUG::2013-11-20 03:21:19,532::guestIF::97::vm.Vm::(_handleMessage) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::u'AOS1Batch@brd-klee.dk'
>Dummy-15912651::DEBUG::2013-11-20 03:21:19,854::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:21:20,464::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0866953 s, 11.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:21:22,477::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 03:21:22,744::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:21:22,745::task::495::TaskManager.Task::(_debug) Task d116add1-0f62-4315-9e24-299eec1e95a1: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:21:22,748::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9c3a16c9-e0f9-4a04-975a-0e8cc200c521`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:21:22,749::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:21:22,750::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:21:22,751::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9c3a16c9-e0f9-4a04-975a-0e8cc200c521`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:21:22,752::task::495::TaskManager.Task::(_debug) Task d116add1-0f62-4315-9e24-299eec1e95a1: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:21:22,753::task::495::TaskManager.Task::(_debug) Task d116add1-0f62-4315-9e24-299eec1e95a1: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:21:22,754::task::495::TaskManager.Task::(_debug) Task d116add1-0f62-4315-9e24-299eec1e95a1: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:21:22,754::task::495::TaskManager.Task::(_debug) Task d116add1-0f62-4315-9e24-299eec1e95a1: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:21:22,755::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:21:22,756::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:21:22,757::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:21:22,757::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:21:22,758::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:21:22,759::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:21:22,759::task::495::TaskManager.Task::(_debug) Task d116add1-0f62-4315-9e24-299eec1e95a1: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:21:22,760::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:21:23,070::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0801527 s, 12.8 MB/s\n'; <rc> = 0
>Thread-15914053::DEBUG::2013-11-20 03:21:23,865::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914053::INFO::2013-11-20 03:21:23,867::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914053::DEBUG::2013-11-20 03:21:23,868::task::495::TaskManager.Task::(_debug) Task 922343eb-dcde-4247-98e5-f57e5cd451e3: moving from state init -> state preparing
>Thread-15914053::DEBUG::2013-11-20 03:21:23,869::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914053::DEBUG::2013-11-20 03:21:23,869::task::495::TaskManager.Task::(_debug) Task 922343eb-dcde-4247-98e5-f57e5cd451e3: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914053::DEBUG::2013-11-20 03:21:23,870::task::495::TaskManager.Task::(_debug) Task 922343eb-dcde-4247-98e5-f57e5cd451e3: moving from state preparing -> state finished
>Thread-15914053::DEBUG::2013-11-20 03:21:23,871::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914053::DEBUG::2013-11-20 03:21:23,871::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914053::DEBUG::2013-11-20 03:21:23,872::task::495::TaskManager.Task::(_debug) Task 922343eb-dcde-4247-98e5-f57e5cd451e3: ref 0 aborting False
>Thread-15914053::INFO::2013-11-20 03:21:23,873::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914054::DEBUG::2013-11-20 03:21:23,878::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914054::INFO::2013-11-20 03:21:23,880::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914054::DEBUG::2013-11-20 03:21:23,880::task::495::TaskManager.Task::(_debug) Task 6ca775b2-a3f6-493f-a094-4a0d20389158: moving from state init -> state preparing
>Thread-15914054::DEBUG::2013-11-20 03:21:23,881::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`66b4e6e5-9684-48b6-81ad-3f41a09c2966`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914054::DEBUG::2013-11-20 03:21:23,882::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914054::DEBUG::2013-11-20 03:21:23,883::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914054::DEBUG::2013-11-20 03:21:23,884::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`66b4e6e5-9684-48b6-81ad-3f41a09c2966`::Granted request
>Thread-15914054::DEBUG::2013-11-20 03:21:23,885::task::495::TaskManager.Task::(_debug) Task 6ca775b2-a3f6-493f-a094-4a0d20389158: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914054::DEBUG::2013-11-20 03:21:23,886::task::495::TaskManager.Task::(_debug) Task 6ca775b2-a3f6-493f-a094-4a0d20389158: ref 1 aborting False
>Thread-15914054::WARNING::2013-11-20 03:21:23,888::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914054::DEBUG::2013-11-20 03:21:23,888::task::495::TaskManager.Task::(_debug) Task 6ca775b2-a3f6-493f-a094-4a0d20389158: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914054::DEBUG::2013-11-20 03:21:23,889::task::495::TaskManager.Task::(_debug) Task 6ca775b2-a3f6-493f-a094-4a0d20389158: moving from state preparing -> state finished
>Thread-15914054::DEBUG::2013-11-20 03:21:23,890::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914054::DEBUG::2013-11-20 03:21:23,890::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914054::DEBUG::2013-11-20 03:21:23,891::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914054::DEBUG::2013-11-20 03:21:23,892::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914054::DEBUG::2013-11-20 03:21:23,892::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914054::DEBUG::2013-11-20 03:21:23,893::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914054::DEBUG::2013-11-20 03:21:23,894::task::495::TaskManager.Task::(_debug) Task 6ca775b2-a3f6-493f-a094-4a0d20389158: ref 0 aborting False
>Thread-15914054::INFO::2013-11-20 03:21:23,894::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:21:25,087::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:21:25,632::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0855935 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:21:27,648::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:21:28,250::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0843502 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15914056::INFO::2013-11-20 03:21:28,505::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914056::DEBUG::2013-11-20 03:21:28,506::task::495::TaskManager.Task::(_debug) Task 0c722992-a67d-481b-ab98-0bbf5ce32ddc: moving from state init -> state preparing
>Thread-15914056::DEBUG::2013-11-20 03:21:28,507::task::495::TaskManager.Task::(_debug) Task 0c722992-a67d-481b-ab98-0bbf5ce32ddc: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000383853912354', 'lastCheck': 1384917678.4852309, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00132513046265', 'lastCheck': 1384917685.309761, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00180387496948', 'lastCheck': 1384917687.949965, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0194098949432', 'lastCheck': 1384917687.933697, 'valid': True, 'code': 0}}
>Thread-15914056::DEBUG::2013-11-20 03:21:28,507::task::495::TaskManager.Task::(_debug) Task 0c722992-a67d-481b-ab98-0bbf5ce32ddc: moving from state preparing -> state finished
>Thread-15914056::DEBUG::2013-11-20 03:21:28,508::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914056::DEBUG::2013-11-20 03:21:28,508::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914056::DEBUG::2013-11-20 03:21:28,509::task::495::TaskManager.Task::(_debug) Task 0c722992-a67d-481b-ab98-0bbf5ce32ddc: ref 0 aborting False
>Thread-15914056::INFO::2013-11-20 03:21:28,510::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000383853912354', 'lastCheck': 1384917678.4852309, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00132513046265', 'lastCheck': 1384917685.309761, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00180387496948', 'lastCheck': 1384917687.949965, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0194098949432', 'lastCheck': 1384917687.933697, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:21:30,267::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:21:30,848::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0888232 s, 11.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:21:32,859::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:21:33,373::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.054935 s, 18.6 MB/s\n'; <rc> = 0
>Thread-15914059::DEBUG::2013-11-20 03:21:35,227::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914059::INFO::2013-11-20 03:21:35,228::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914059::DEBUG::2013-11-20 03:21:35,228::task::495::TaskManager.Task::(_debug) Task 154299de-71c9-48ca-860c-674db4bac90a: moving from state init -> state preparing
>Thread-15914059::DEBUG::2013-11-20 03:21:35,229::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914059::DEBUG::2013-11-20 03:21:35,229::task::495::TaskManager.Task::(_debug) Task 154299de-71c9-48ca-860c-674db4bac90a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914059::DEBUG::2013-11-20 03:21:35,230::task::495::TaskManager.Task::(_debug) Task 154299de-71c9-48ca-860c-674db4bac90a: moving from state preparing -> state finished
>Thread-15914059::DEBUG::2013-11-20 03:21:35,230::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914059::DEBUG::2013-11-20 03:21:35,230::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914059::DEBUG::2013-11-20 03:21:35,231::task::495::TaskManager.Task::(_debug) Task 154299de-71c9-48ca-860c-674db4bac90a: ref 0 aborting False
>Thread-15914059::INFO::2013-11-20 03:21:35,231::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914060::DEBUG::2013-11-20 03:21:35,235::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914060::INFO::2013-11-20 03:21:35,236::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914060::DEBUG::2013-11-20 03:21:35,237::task::495::TaskManager.Task::(_debug) Task b2ec2752-562b-425b-9d5b-383722f4ac8f: moving from state init -> state preparing
>Thread-15914060::DEBUG::2013-11-20 03:21:35,238::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`52b8ec82-7878-4735-ac38-e9f046c17fbb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914060::DEBUG::2013-11-20 03:21:35,239::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914060::DEBUG::2013-11-20 03:21:35,239::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914060::DEBUG::2013-11-20 03:21:35,240::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`52b8ec82-7878-4735-ac38-e9f046c17fbb`::Granted request
>Thread-15914060::DEBUG::2013-11-20 03:21:35,241::task::495::TaskManager.Task::(_debug) Task b2ec2752-562b-425b-9d5b-383722f4ac8f: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914060::DEBUG::2013-11-20 03:21:35,242::task::495::TaskManager.Task::(_debug) Task b2ec2752-562b-425b-9d5b-383722f4ac8f: ref 1 aborting False
>Thread-15914060::WARNING::2013-11-20 03:21:35,244::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914060::DEBUG::2013-11-20 03:21:35,245::task::495::TaskManager.Task::(_debug) Task b2ec2752-562b-425b-9d5b-383722f4ac8f: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914060::DEBUG::2013-11-20 03:21:35,246::task::495::TaskManager.Task::(_debug) Task b2ec2752-562b-425b-9d5b-383722f4ac8f: moving from state preparing -> state finished
>Thread-15914060::DEBUG::2013-11-20 03:21:35,246::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914060::DEBUG::2013-11-20 03:21:35,247::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914060::DEBUG::2013-11-20 03:21:35,247::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914060::DEBUG::2013-11-20 03:21:35,248::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914060::DEBUG::2013-11-20 03:21:35,249::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914060::DEBUG::2013-11-20 03:21:35,249::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914060::DEBUG::2013-11-20 03:21:35,250::task::495::TaskManager.Task::(_debug) Task b2ec2752-562b-425b-9d5b-383722f4ac8f: ref 0 aborting False
>Thread-15914060::INFO::2013-11-20 03:21:35,250::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:21:35,393::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:21:35,932::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0760732 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:21:37,948::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:21:38,459::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0815302 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:21:40,474::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 03:21:40,692::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:21:40,694::task::495::TaskManager.Task::(_debug) Task 2b9a65b3-34fb-4833-b975-9ae90b2a034f: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:21:40,697::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2b32bc57-341c-4890-aaa4-3da2ba55a4f6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:21:40,698::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:21:40,699::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:21:40,700::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2b32bc57-341c-4890-aaa4-3da2ba55a4f6`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:21:40,701::task::495::TaskManager.Task::(_debug) Task 2b9a65b3-34fb-4833-b975-9ae90b2a034f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:21:40,702::task::495::TaskManager.Task::(_debug) Task 2b9a65b3-34fb-4833-b975-9ae90b2a034f: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:21:40,703::task::495::TaskManager.Task::(_debug) Task 2b9a65b3-34fb-4833-b975-9ae90b2a034f: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:21:40,704::task::495::TaskManager.Task::(_debug) Task 2b9a65b3-34fb-4833-b975-9ae90b2a034f: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:21:40,704::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:21:40,705::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:21:40,706::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:21:40,707::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:21:40,707::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:21:40,708::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:21:40,709::task::495::TaskManager.Task::(_debug) Task 2b9a65b3-34fb-4833-b975-9ae90b2a034f: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:21:40,710::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:21:40,994::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0748524 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:21:43,007::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 03:21:43,224::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:21:43,226::task::495::TaskManager.Task::(_debug) Task 06aee392-df4b-4d97-94b0-f0b19f97b1e9: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:21:43,230::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`cc52529c-73e2-4845-86ec-6af72b8a9e04`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:21:43,231::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:21:43,233::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:21:43,233::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`cc52529c-73e2-4845-86ec-6af72b8a9e04`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:21:43,234::task::495::TaskManager.Task::(_debug) Task 06aee392-df4b-4d97-94b0-f0b19f97b1e9: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:21:43,235::task::495::TaskManager.Task::(_debug) Task 06aee392-df4b-4d97-94b0-f0b19f97b1e9: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:21:43,236::task::495::TaskManager.Task::(_debug) Task 06aee392-df4b-4d97-94b0-f0b19f97b1e9: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:21:43,236::task::495::TaskManager.Task::(_debug) Task 06aee392-df4b-4d97-94b0-f0b19f97b1e9: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:21:43,237::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:21:43,237::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:21:43,238::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:21:43,239::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:21:43,239::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:21:43,239::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:21:43,240::task::495::TaskManager.Task::(_debug) Task 06aee392-df4b-4d97-94b0-f0b19f97b1e9: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:21:43,241::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:21:43,437::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0450977 s, 22.7 MB/s\n'; <rc> = 0
>Thread-15914064::INFO::2013-11-20 03:21:45,438::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914064::DEBUG::2013-11-20 03:21:45,439::task::495::TaskManager.Task::(_debug) Task 2df59309-c719-4efa-8da8-60995fb21028: moving from state init -> state preparing
>Thread-15914064::DEBUG::2013-11-20 03:21:45,440::task::495::TaskManager.Task::(_debug) Task 2df59309-c719-4efa-8da8-60995fb21028: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000291109085083', 'lastCheck': 1384917699.1476021, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00137090682983', 'lastCheck': 1384917695.6106739, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00110101699829', 'lastCheck': 1384917698.150955, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00251984596252', 'lastCheck': 1384917698.147362, 'valid': True, 'code': 0}}
>Thread-15914064::DEBUG::2013-11-20 03:21:45,440::task::495::TaskManager.Task::(_debug) Task 2df59309-c719-4efa-8da8-60995fb21028: moving from state preparing -> state finished
>Thread-15914064::DEBUG::2013-11-20 03:21:45,441::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914064::DEBUG::2013-11-20 03:21:45,441::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914064::DEBUG::2013-11-20 03:21:45,441::task::495::TaskManager.Task::(_debug) Task 2df59309-c719-4efa-8da8-60995fb21028: ref 0 aborting False
>Thread-15914064::INFO::2013-11-20 03:21:45,442::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000291109085083', 'lastCheck': 1384917699.1476021, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00137090682983', 'lastCheck': 1384917695.6106739, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00110101699829', 'lastCheck': 1384917698.150955, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00251984596252', 'lastCheck': 1384917698.147362, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:21:45,448::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914066::DEBUG::2013-11-20 03:21:45,712::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914066::INFO::2013-11-20 03:21:45,715::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914066::DEBUG::2013-11-20 03:21:45,715::task::495::TaskManager.Task::(_debug) Task 5f338598-965d-4560-a0b2-b3ff14a72018: moving from state init -> state preparing
>Thread-15914066::DEBUG::2013-11-20 03:21:45,717::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914066::DEBUG::2013-11-20 03:21:45,718::task::495::TaskManager.Task::(_debug) Task 5f338598-965d-4560-a0b2-b3ff14a72018: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914066::DEBUG::2013-11-20 03:21:45,725::task::495::TaskManager.Task::(_debug) Task 5f338598-965d-4560-a0b2-b3ff14a72018: moving from state preparing -> state finished
>Thread-15914066::DEBUG::2013-11-20 03:21:45,726::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914066::DEBUG::2013-11-20 03:21:45,727::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914066::DEBUG::2013-11-20 03:21:45,728::task::495::TaskManager.Task::(_debug) Task 5f338598-965d-4560-a0b2-b3ff14a72018: ref 0 aborting False
>Thread-15914066::INFO::2013-11-20 03:21:45,728::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914067::DEBUG::2013-11-20 03:21:45,749::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914067::INFO::2013-11-20 03:21:45,750::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914067::DEBUG::2013-11-20 03:21:45,751::task::495::TaskManager.Task::(_debug) Task 20d4ce52-0d18-4f03-84d5-97f6690d268f: moving from state init -> state preparing
>Thread-15914067::DEBUG::2013-11-20 03:21:45,753::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`76ed6e25-0208-49f0-aa40-b8bb69700dc0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914067::DEBUG::2013-11-20 03:21:45,753::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914067::DEBUG::2013-11-20 03:21:45,754::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914067::DEBUG::2013-11-20 03:21:45,755::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`76ed6e25-0208-49f0-aa40-b8bb69700dc0`::Granted request
>Thread-15914067::DEBUG::2013-11-20 03:21:45,762::task::495::TaskManager.Task::(_debug) Task 20d4ce52-0d18-4f03-84d5-97f6690d268f: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914067::DEBUG::2013-11-20 03:21:45,762::task::495::TaskManager.Task::(_debug) Task 20d4ce52-0d18-4f03-84d5-97f6690d268f: ref 1 aborting False
>Thread-15914067::WARNING::2013-11-20 03:21:45,765::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914067::DEBUG::2013-11-20 03:21:45,772::task::495::TaskManager.Task::(_debug) Task 20d4ce52-0d18-4f03-84d5-97f6690d268f: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914067::DEBUG::2013-11-20 03:21:45,773::task::495::TaskManager.Task::(_debug) Task 20d4ce52-0d18-4f03-84d5-97f6690d268f: moving from state preparing -> state finished
>Thread-15914067::DEBUG::2013-11-20 03:21:45,774::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914067::DEBUG::2013-11-20 03:21:45,775::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914067::DEBUG::2013-11-20 03:21:45,776::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914067::DEBUG::2013-11-20 03:21:45,777::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914067::DEBUG::2013-11-20 03:21:45,777::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914067::DEBUG::2013-11-20 03:21:45,778::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914067::DEBUG::2013-11-20 03:21:45,779::task::495::TaskManager.Task::(_debug) Task 20d4ce52-0d18-4f03-84d5-97f6690d268f: ref 0 aborting False
>Thread-15914067::INFO::2013-11-20 03:21:45,779::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:21:46,022::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.113882 s, 9.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:21:48,037::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 03:21:48,240::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:21:48,246::task::495::TaskManager.Task::(_debug) Task 99d5ae1e-3228-4459-be49-3c32aed7d065: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:21:48,252::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`dd150457-5f61-4817-a198-6de97d51b9dc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:21:48,253::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:21:48,253::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:21:48,254::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`dd150457-5f61-4817-a198-6de97d51b9dc`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:21:48,254::task::495::TaskManager.Task::(_debug) Task 99d5ae1e-3228-4459-be49-3c32aed7d065: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:21:48,255::task::495::TaskManager.Task::(_debug) Task 99d5ae1e-3228-4459-be49-3c32aed7d065: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:21:48,255::task::495::TaskManager.Task::(_debug) Task 99d5ae1e-3228-4459-be49-3c32aed7d065: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:21:48,256::task::495::TaskManager.Task::(_debug) Task 99d5ae1e-3228-4459-be49-3c32aed7d065: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:21:48,257::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:21:48,257::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:21:48,257::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:21:48,258::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:21:48,258::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:21:48,259::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:21:48,259::task::495::TaskManager.Task::(_debug) Task 99d5ae1e-3228-4459-be49-3c32aed7d065: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:21:48,259::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:21:48,561::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0818734 s, 12.5 MB/s\n'; <rc> = 0
>Thread-42838::DEBUG::2013-11-20 03:21:49,244::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 03:21:49,857::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:21:50,578::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:21:51,047::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0870576 s, 11.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:21:53,062::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:21:53,586::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0844231 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:21:55,598::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914071::DEBUG::2013-11-20 03:21:56,076::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914071::INFO::2013-11-20 03:21:56,079::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914071::DEBUG::2013-11-20 03:21:56,080::task::495::TaskManager.Task::(_debug) Task 400c6aaf-f266-4682-a003-54fba8f84993: moving from state init -> state preparing
>Thread-15914071::DEBUG::2013-11-20 03:21:56,082::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914071::DEBUG::2013-11-20 03:21:56,083::task::495::TaskManager.Task::(_debug) Task 400c6aaf-f266-4682-a003-54fba8f84993: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914071::DEBUG::2013-11-20 03:21:56,084::task::495::TaskManager.Task::(_debug) Task 400c6aaf-f266-4682-a003-54fba8f84993: moving from state preparing -> state finished
>Thread-15914071::DEBUG::2013-11-20 03:21:56,085::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914071::DEBUG::2013-11-20 03:21:56,086::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914071::DEBUG::2013-11-20 03:21:56,087::task::495::TaskManager.Task::(_debug) Task 400c6aaf-f266-4682-a003-54fba8f84993: ref 0 aborting False
>Thread-15914071::INFO::2013-11-20 03:21:56,089::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914072::DEBUG::2013-11-20 03:21:56,096::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914072::INFO::2013-11-20 03:21:56,097::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914072::DEBUG::2013-11-20 03:21:56,099::task::495::TaskManager.Task::(_debug) Task ad5b3306-e7d4-4c72-82b4-d9a717212bab: moving from state init -> state preparing
>Thread-15914072::DEBUG::2013-11-20 03:21:56,101::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b37d9e1b-3341-41d5-b7fd-7fcc5cdb7fdb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914072::DEBUG::2013-11-20 03:21:56,103::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914072::DEBUG::2013-11-20 03:21:56,104::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914072::DEBUG::2013-11-20 03:21:56,105::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b37d9e1b-3341-41d5-b7fd-7fcc5cdb7fdb`::Granted request
>Thread-15914072::DEBUG::2013-11-20 03:21:56,107::task::495::TaskManager.Task::(_debug) Task ad5b3306-e7d4-4c72-82b4-d9a717212bab: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914072::DEBUG::2013-11-20 03:21:56,107::task::495::TaskManager.Task::(_debug) Task ad5b3306-e7d4-4c72-82b4-d9a717212bab: ref 1 aborting False
>Thread-15914072::WARNING::2013-11-20 03:21:56,110::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914072::DEBUG::2013-11-20 03:21:56,112::task::495::TaskManager.Task::(_debug) Task ad5b3306-e7d4-4c72-82b4-d9a717212bab: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914072::DEBUG::2013-11-20 03:21:56,113::task::495::TaskManager.Task::(_debug) Task ad5b3306-e7d4-4c72-82b4-d9a717212bab: moving from state preparing -> state finished
>Thread-15914072::DEBUG::2013-11-20 03:21:56,114::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914072::DEBUG::2013-11-20 03:21:56,115::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914072::DEBUG::2013-11-20 03:21:56,116::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914072::DEBUG::2013-11-20 03:21:56,117::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914072::DEBUG::2013-11-20 03:21:56,118::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914072::DEBUG::2013-11-20 03:21:56,119::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914072::DEBUG::2013-11-20 03:21:56,120::task::495::TaskManager.Task::(_debug) Task ad5b3306-e7d4-4c72-82b4-d9a717212bab: ref 0 aborting False
>Thread-15914072::INFO::2013-11-20 03:21:56,121::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:21:56,138::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0773564 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 03:21:56,502::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:21:56,503::task::495::TaskManager.Task::(_debug) Task ba714483-e062-4371-89c9-29aa95b16d74: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:21:56,504::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bb4e134c-72d2-4c1b-80bf-2a0dd01b42d2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:21:56,505::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:21:56,505::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:21:56,506::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bb4e134c-72d2-4c1b-80bf-2a0dd01b42d2`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:21:56,506::task::495::TaskManager.Task::(_debug) Task ba714483-e062-4371-89c9-29aa95b16d74: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:21:56,507::task::495::TaskManager.Task::(_debug) Task ba714483-e062-4371-89c9-29aa95b16d74: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:21:56,507::task::495::TaskManager.Task::(_debug) Task ba714483-e062-4371-89c9-29aa95b16d74: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:21:56,508::task::495::TaskManager.Task::(_debug) Task ba714483-e062-4371-89c9-29aa95b16d74: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:21:56,508::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:21:56,509::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:21:56,510::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:21:56,510::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:21:56,511::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:21:56,511::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:21:56,512::task::495::TaskManager.Task::(_debug) Task ba714483-e062-4371-89c9-29aa95b16d74: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:21:56,512::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:21:56,513::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:21:56,513::task::495::TaskManager.Task::(_debug) Task d07b1120-a050-457f-9083-76d5e015262e: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:21:56,514::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`173fc5be-2cbf-4513-ba7f-a0a885950d81`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:21:56,515::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:21:56,516::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:21:56,516::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`173fc5be-2cbf-4513-ba7f-a0a885950d81`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:21:56,517::task::495::TaskManager.Task::(_debug) Task d07b1120-a050-457f-9083-76d5e015262e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:21:56,517::task::495::TaskManager.Task::(_debug) Task d07b1120-a050-457f-9083-76d5e015262e: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:21:56,518::task::495::TaskManager.Task::(_debug) Task d07b1120-a050-457f-9083-76d5e015262e: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:21:56,518::task::495::TaskManager.Task::(_debug) Task d07b1120-a050-457f-9083-76d5e015262e: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:21:56,518::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:21:56,519::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:21:56,520::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:21:56,520::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:21:56,520::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:21:56,521::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:21:56,522::task::495::TaskManager.Task::(_debug) Task d07b1120-a050-457f-9083-76d5e015262e: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:21:56,522::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:21:58,151::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:21:58,553::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0526587 s, 19.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:22:00,572::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:22:01,105::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0766989 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15914074::INFO::2013-11-20 03:22:01,334::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914074::DEBUG::2013-11-20 03:22:01,335::task::495::TaskManager.Task::(_debug) Task 99a66120-90f8-48e3-9908-df15a8a92bb8: moving from state init -> state preparing
>Thread-15914074::DEBUG::2013-11-20 03:22:01,336::task::495::TaskManager.Task::(_debug) Task 99a66120-90f8-48e3-9908-df15a8a92bb8: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000314950942993', 'lastCheck': 1384917719.8721621, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00195813179016', 'lastCheck': 1384917715.8134961, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00116014480591', 'lastCheck': 1384917718.4123321, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00225400924683', 'lastCheck': 1384917718.409816, 'valid': True, 'code': 0}}
>Thread-15914074::DEBUG::2013-11-20 03:22:01,337::task::495::TaskManager.Task::(_debug) Task 99a66120-90f8-48e3-9908-df15a8a92bb8: moving from state preparing -> state finished
>Thread-15914074::DEBUG::2013-11-20 03:22:01,337::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914074::DEBUG::2013-11-20 03:22:01,338::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914074::DEBUG::2013-11-20 03:22:01,338::task::495::TaskManager.Task::(_debug) Task 99a66120-90f8-48e3-9908-df15a8a92bb8: ref 0 aborting False
>Thread-15914074::INFO::2013-11-20 03:22:01,339::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000314950942993', 'lastCheck': 1384917719.8721621, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00195813179016', 'lastCheck': 1384917715.8134961, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00116014480591', 'lastCheck': 1384917718.4123321, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00225400924683', 'lastCheck': 1384917718.409816, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:22:03,122::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:22:03,569::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0456477 s, 22.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:22:05,580::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:22:06,107::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0716224 s, 14.3 MB/s\n'; <rc> = 0
>Thread-15914077::DEBUG::2013-11-20 03:22:06,365::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914077::INFO::2013-11-20 03:22:06,368::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914077::DEBUG::2013-11-20 03:22:06,368::task::495::TaskManager.Task::(_debug) Task 0b9f633e-0482-43fe-955e-6b2a7d1102f8: moving from state init -> state preparing
>Thread-15914077::DEBUG::2013-11-20 03:22:06,370::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914077::DEBUG::2013-11-20 03:22:06,371::task::495::TaskManager.Task::(_debug) Task 0b9f633e-0482-43fe-955e-6b2a7d1102f8: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914077::DEBUG::2013-11-20 03:22:06,371::task::495::TaskManager.Task::(_debug) Task 0b9f633e-0482-43fe-955e-6b2a7d1102f8: moving from state preparing -> state finished
>Thread-15914077::DEBUG::2013-11-20 03:22:06,372::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914077::DEBUG::2013-11-20 03:22:06,372::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914077::DEBUG::2013-11-20 03:22:06,373::task::495::TaskManager.Task::(_debug) Task 0b9f633e-0482-43fe-955e-6b2a7d1102f8: ref 0 aborting False
>Thread-15914077::INFO::2013-11-20 03:22:06,374::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914078::DEBUG::2013-11-20 03:22:06,379::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914078::INFO::2013-11-20 03:22:06,380::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914078::DEBUG::2013-11-20 03:22:06,380::task::495::TaskManager.Task::(_debug) Task 03be9948-e697-40a1-8464-61d25593a927: moving from state init -> state preparing
>Thread-15914078::DEBUG::2013-11-20 03:22:06,381::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d38c3136-32a4-4632-b29a-e3f06affbf6c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914078::DEBUG::2013-11-20 03:22:06,382::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914078::DEBUG::2013-11-20 03:22:06,383::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914078::DEBUG::2013-11-20 03:22:06,384::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d38c3136-32a4-4632-b29a-e3f06affbf6c`::Granted request
>Thread-15914078::DEBUG::2013-11-20 03:22:06,385::task::495::TaskManager.Task::(_debug) Task 03be9948-e697-40a1-8464-61d25593a927: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914078::DEBUG::2013-11-20 03:22:06,386::task::495::TaskManager.Task::(_debug) Task 03be9948-e697-40a1-8464-61d25593a927: ref 1 aborting False
>Thread-15914078::WARNING::2013-11-20 03:22:06,389::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914078::DEBUG::2013-11-20 03:22:06,390::task::495::TaskManager.Task::(_debug) Task 03be9948-e697-40a1-8464-61d25593a927: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914078::DEBUG::2013-11-20 03:22:06,391::task::495::TaskManager.Task::(_debug) Task 03be9948-e697-40a1-8464-61d25593a927: moving from state preparing -> state finished
>Thread-15914078::DEBUG::2013-11-20 03:22:06,392::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914078::DEBUG::2013-11-20 03:22:06,393::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914078::DEBUG::2013-11-20 03:22:06,394::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914078::DEBUG::2013-11-20 03:22:06,394::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914078::DEBUG::2013-11-20 03:22:06,395::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914078::DEBUG::2013-11-20 03:22:06,396::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914078::DEBUG::2013-11-20 03:22:06,396::task::495::TaskManager.Task::(_debug) Task 03be9948-e697-40a1-8464-61d25593a927: ref 0 aborting False
>Thread-15914078::INFO::2013-11-20 03:22:06,397::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:22:08,123::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:22:08,678::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0797713 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:22:10,694::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:22:11,211::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0749017 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:22:13,226::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 03:22:13,434::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:22:13,436::task::495::TaskManager.Task::(_debug) Task 31c4a5b6-7200-487c-a578-30536d588221: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:22:13,438::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e91ee52f-7f07-40ba-8c8e-259014671ccf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:22:13,439::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:22:13,440::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:22:13,440::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e91ee52f-7f07-40ba-8c8e-259014671ccf`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:22:13,441::task::495::TaskManager.Task::(_debug) Task 31c4a5b6-7200-487c-a578-30536d588221: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:22:13,441::task::495::TaskManager.Task::(_debug) Task 31c4a5b6-7200-487c-a578-30536d588221: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:22:13,442::task::495::TaskManager.Task::(_debug) Task 31c4a5b6-7200-487c-a578-30536d588221: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:22:13,442::task::495::TaskManager.Task::(_debug) Task 31c4a5b6-7200-487c-a578-30536d588221: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:22:13,443::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:22:13,443::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:22:13,444::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:22:13,444::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:22:13,444::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:22:13,445::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:22:13,445::task::495::TaskManager.Task::(_debug) Task 31c4a5b6-7200-487c-a578-30536d588221: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:22:13,446::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:22:13,446::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:22:13,447::task::495::TaskManager.Task::(_debug) Task d26692f0-d6ec-4e83-aaaf-09a695619c69: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:22:13,447::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`059605e4-7b3c-4a4b-a404-4d83c2146ca2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:22:13,448::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:22:13,448::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:22:13,448::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`059605e4-7b3c-4a4b-a404-4d83c2146ca2`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:22:13,449::task::495::TaskManager.Task::(_debug) Task d26692f0-d6ec-4e83-aaaf-09a695619c69: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:22:13,449::task::495::TaskManager.Task::(_debug) Task d26692f0-d6ec-4e83-aaaf-09a695619c69: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:22:13,450::task::495::TaskManager.Task::(_debug) Task d26692f0-d6ec-4e83-aaaf-09a695619c69: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:22:13,450::task::495::TaskManager.Task::(_debug) Task d26692f0-d6ec-4e83-aaaf-09a695619c69: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:22:13,451::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:22:13,451::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:22:13,451::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:22:13,452::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:22:13,452::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:22:13,453::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:22:13,453::task::495::TaskManager.Task::(_debug) Task d26692f0-d6ec-4e83-aaaf-09a695619c69: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:22:13,454::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:22:13,454::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:22:13,454::task::495::TaskManager.Task::(_debug) Task 8383056b-c7ff-487e-9435-a8042b5d1503: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:22:13,455::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`9dc90df2-27b8-46b7-9ff1-2ca074cd7a60`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:22:13,455::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:22:13,456::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:22:13,456::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`9dc90df2-27b8-46b7-9ff1-2ca074cd7a60`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:22:13,457::task::495::TaskManager.Task::(_debug) Task 8383056b-c7ff-487e-9435-a8042b5d1503: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:22:13,457::task::495::TaskManager.Task::(_debug) Task 8383056b-c7ff-487e-9435-a8042b5d1503: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:22:13,458::task::495::TaskManager.Task::(_debug) Task 8383056b-c7ff-487e-9435-a8042b5d1503: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:22:13,458::task::495::TaskManager.Task::(_debug) Task 8383056b-c7ff-487e-9435-a8042b5d1503: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:22:13,458::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:22:13,459::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:22:13,459::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:22:13,460::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:22:13,460::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:22:13,460::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:22:13,461::task::495::TaskManager.Task::(_debug) Task 8383056b-c7ff-487e-9435-a8042b5d1503: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:22:13,461::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:22:13,462::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:22:13,462::task::495::TaskManager.Task::(_debug) Task 9c9ae5c3-0e7a-441a-a3c3-57edd866c041: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:22:13,463::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`95d5f89f-3884-4bf8-bb64-242325217073`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:22:13,463::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:22:13,464::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:22:13,464::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`95d5f89f-3884-4bf8-bb64-242325217073`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:22:13,465::task::495::TaskManager.Task::(_debug) Task 9c9ae5c3-0e7a-441a-a3c3-57edd866c041: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:22:13,466::task::495::TaskManager.Task::(_debug) Task 9c9ae5c3-0e7a-441a-a3c3-57edd866c041: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:22:13,467::task::495::TaskManager.Task::(_debug) Task 9c9ae5c3-0e7a-441a-a3c3-57edd866c041: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:22:13,467::task::495::TaskManager.Task::(_debug) Task 9c9ae5c3-0e7a-441a-a3c3-57edd866c041: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:22:13,468::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:22:13,468::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:22:13,469::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:22:13,470::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:22:13,470::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:22:13,471::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:22:13,471::task::495::TaskManager.Task::(_debug) Task 9c9ae5c3-0e7a-441a-a3c3-57edd866c041: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:22:13,472::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:22:13,757::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0833508 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:22:15,772::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:22:16,320::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0759192 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15914082::DEBUG::2013-11-20 03:22:16,951::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914082::INFO::2013-11-20 03:22:16,954::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914082::DEBUG::2013-11-20 03:22:16,954::task::495::TaskManager.Task::(_debug) Task daac5c45-a020-45f0-91f5-3782c4945f1d: moving from state init -> state preparing
>Thread-15914082::DEBUG::2013-11-20 03:22:16,955::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914082::DEBUG::2013-11-20 03:22:16,956::task::495::TaskManager.Task::(_debug) Task daac5c45-a020-45f0-91f5-3782c4945f1d: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914082::DEBUG::2013-11-20 03:22:16,957::task::495::TaskManager.Task::(_debug) Task daac5c45-a020-45f0-91f5-3782c4945f1d: moving from state preparing -> state finished
>Thread-15914082::DEBUG::2013-11-20 03:22:16,957::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914082::DEBUG::2013-11-20 03:22:16,958::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914082::DEBUG::2013-11-20 03:22:16,958::task::495::TaskManager.Task::(_debug) Task daac5c45-a020-45f0-91f5-3782c4945f1d: ref 0 aborting False
>Thread-15914082::INFO::2013-11-20 03:22:16,959::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914083::DEBUG::2013-11-20 03:22:16,963::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914083::INFO::2013-11-20 03:22:16,964::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914083::DEBUG::2013-11-20 03:22:16,965::task::495::TaskManager.Task::(_debug) Task a767ee14-a178-499e-89c6-57a2d115e00d: moving from state init -> state preparing
>Thread-15914083::DEBUG::2013-11-20 03:22:16,966::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8d24a512-1e08-4da2-9d3c-4ee8ab88091f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914083::DEBUG::2013-11-20 03:22:16,967::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914083::DEBUG::2013-11-20 03:22:16,968::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914083::DEBUG::2013-11-20 03:22:16,969::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8d24a512-1e08-4da2-9d3c-4ee8ab88091f`::Granted request
>Thread-15914083::DEBUG::2013-11-20 03:22:16,971::task::495::TaskManager.Task::(_debug) Task a767ee14-a178-499e-89c6-57a2d115e00d: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914083::DEBUG::2013-11-20 03:22:16,972::task::495::TaskManager.Task::(_debug) Task a767ee14-a178-499e-89c6-57a2d115e00d: ref 1 aborting False
>Thread-15914083::WARNING::2013-11-20 03:22:16,974::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914083::DEBUG::2013-11-20 03:22:16,975::task::495::TaskManager.Task::(_debug) Task a767ee14-a178-499e-89c6-57a2d115e00d: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914083::DEBUG::2013-11-20 03:22:16,976::task::495::TaskManager.Task::(_debug) Task a767ee14-a178-499e-89c6-57a2d115e00d: moving from state preparing -> state finished
>Thread-15914083::DEBUG::2013-11-20 03:22:16,977::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914083::DEBUG::2013-11-20 03:22:16,977::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914084::INFO::2013-11-20 03:22:16,980::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914084::DEBUG::2013-11-20 03:22:16,981::task::495::TaskManager.Task::(_debug) Task db48b610-cacf-4827-8309-49e1f1e7664c: moving from state init -> state preparing
>Thread-15914083::DEBUG::2013-11-20 03:22:16,982::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914084::DEBUG::2013-11-20 03:22:16,983::task::495::TaskManager.Task::(_debug) Task db48b610-cacf-4827-8309-49e1f1e7664c: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000312089920044', 'lastCheck': 1384917729.8859291, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00130796432495', 'lastCheck': 1384917736.076524, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0169382095337', 'lastCheck': 1384917728.443157, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000749111175537', 'lastCheck': 1384917728.4223771, 'valid': True, 'code': 0}}
>Thread-15914083::DEBUG::2013-11-20 03:22:16,984::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914084::DEBUG::2013-11-20 03:22:16,985::task::495::TaskManager.Task::(_debug) Task db48b610-cacf-4827-8309-49e1f1e7664c: moving from state preparing -> state finished
>Thread-15914083::DEBUG::2013-11-20 03:22:16,985::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914084::DEBUG::2013-11-20 03:22:16,986::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914083::DEBUG::2013-11-20 03:22:16,987::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914084::DEBUG::2013-11-20 03:22:16,987::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914083::DEBUG::2013-11-20 03:22:16,989::task::495::TaskManager.Task::(_debug) Task a767ee14-a178-499e-89c6-57a2d115e00d: ref 0 aborting False
>Thread-15914084::DEBUG::2013-11-20 03:22:16,989::task::495::TaskManager.Task::(_debug) Task db48b610-cacf-4827-8309-49e1f1e7664c: ref 0 aborting False
>Thread-15914083::INFO::2013-11-20 03:22:16,990::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914084::INFO::2013-11-20 03:22:16,991::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000312089920044', 'lastCheck': 1384917729.8859291, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00130796432495', 'lastCheck': 1384917736.076524, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0169382095337', 'lastCheck': 1384917728.443157, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000749111175537', 'lastCheck': 1384917728.4223771, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:22:18,334::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:22:18,867::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0797632 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:22:20,881::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:22:21,378::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0731363 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:22:23,394::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 03:22:23,611::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:22:23,614::task::495::TaskManager.Task::(_debug) Task 0935ea2a-fcc0-4136-8de5-d4288a1f0d92: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:22:23,616::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d0dd6d7b-2aba-4b5d-9545-5ebc16812fbf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:22:23,617::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:22:23,618::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:22:23,619::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d0dd6d7b-2aba-4b5d-9545-5ebc16812fbf`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:22:23,620::task::495::TaskManager.Task::(_debug) Task 0935ea2a-fcc0-4136-8de5-d4288a1f0d92: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:22:23,621::task::495::TaskManager.Task::(_debug) Task 0935ea2a-fcc0-4136-8de5-d4288a1f0d92: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:22:23,622::task::495::TaskManager.Task::(_debug) Task 0935ea2a-fcc0-4136-8de5-d4288a1f0d92: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:22:23,622::task::495::TaskManager.Task::(_debug) Task 0935ea2a-fcc0-4136-8de5-d4288a1f0d92: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:22:23,623::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:22:23,624::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:22:23,624::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:22:23,625::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:22:23,625::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:22:23,626::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:22:23,627::task::495::TaskManager.Task::(_debug) Task 0935ea2a-fcc0-4136-8de5-d4288a1f0d92: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:22:23,627::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:22:23,951::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.084958 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:22:25,963::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:22:26,499::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0854991 s, 12.0 MB/s\n'; <rc> = 0
>Thread-15914089::DEBUG::2013-11-20 03:22:27,649::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914089::INFO::2013-11-20 03:22:27,651::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914089::DEBUG::2013-11-20 03:22:27,652::task::495::TaskManager.Task::(_debug) Task e005abf1-8132-4733-b9e2-637264af1967: moving from state init -> state preparing
>Thread-15914089::DEBUG::2013-11-20 03:22:27,653::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914089::DEBUG::2013-11-20 03:22:27,653::task::495::TaskManager.Task::(_debug) Task e005abf1-8132-4733-b9e2-637264af1967: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914089::DEBUG::2013-11-20 03:22:27,654::task::495::TaskManager.Task::(_debug) Task e005abf1-8132-4733-b9e2-637264af1967: moving from state preparing -> state finished
>Thread-15914089::DEBUG::2013-11-20 03:22:27,655::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914089::DEBUG::2013-11-20 03:22:27,656::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914089::DEBUG::2013-11-20 03:22:27,656::task::495::TaskManager.Task::(_debug) Task e005abf1-8132-4733-b9e2-637264af1967: ref 0 aborting False
>Thread-15914089::INFO::2013-11-20 03:22:27,657::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914090::DEBUG::2013-11-20 03:22:27,662::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914090::INFO::2013-11-20 03:22:27,663::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914090::DEBUG::2013-11-20 03:22:27,664::task::495::TaskManager.Task::(_debug) Task 2bb10fab-9d72-4f51-b0a4-a2019f26ec12: moving from state init -> state preparing
>Thread-15914090::DEBUG::2013-11-20 03:22:27,665::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c5dd1723-3b93-4e32-b355-b900cbc49449`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914090::DEBUG::2013-11-20 03:22:27,665::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914090::DEBUG::2013-11-20 03:22:27,666::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914090::DEBUG::2013-11-20 03:22:27,667::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c5dd1723-3b93-4e32-b355-b900cbc49449`::Granted request
>Thread-15914090::DEBUG::2013-11-20 03:22:27,668::task::495::TaskManager.Task::(_debug) Task 2bb10fab-9d72-4f51-b0a4-a2019f26ec12: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914090::DEBUG::2013-11-20 03:22:27,669::task::495::TaskManager.Task::(_debug) Task 2bb10fab-9d72-4f51-b0a4-a2019f26ec12: ref 1 aborting False
>Thread-15914090::WARNING::2013-11-20 03:22:27,671::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914090::DEBUG::2013-11-20 03:22:27,672::task::495::TaskManager.Task::(_debug) Task 2bb10fab-9d72-4f51-b0a4-a2019f26ec12: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914090::DEBUG::2013-11-20 03:22:27,673::task::495::TaskManager.Task::(_debug) Task 2bb10fab-9d72-4f51-b0a4-a2019f26ec12: moving from state preparing -> state finished
>Thread-15914090::DEBUG::2013-11-20 03:22:27,674::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914090::DEBUG::2013-11-20 03:22:27,675::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914090::DEBUG::2013-11-20 03:22:27,675::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914090::DEBUG::2013-11-20 03:22:27,676::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914090::DEBUG::2013-11-20 03:22:27,676::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914090::DEBUG::2013-11-20 03:22:27,677::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914090::DEBUG::2013-11-20 03:22:27,678::task::495::TaskManager.Task::(_debug) Task 2bb10fab-9d72-4f51-b0a4-a2019f26ec12: ref 0 aborting False
>Thread-15914090::INFO::2013-11-20 03:22:27,678::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:22:28,510::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:22:29,045::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0805154 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:22:31,062::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:22:31,637::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0850937 s, 12.0 MB/s\n'; <rc> = 0
>Thread-15914092::INFO::2013-11-20 03:22:32,975::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914092::DEBUG::2013-11-20 03:22:32,976::task::495::TaskManager.Task::(_debug) Task 10477832-f20e-44f9-a6d9-ed08af94bb6c: moving from state init -> state preparing
>Thread-15914092::DEBUG::2013-11-20 03:22:32,977::task::495::TaskManager.Task::(_debug) Task 10477832-f20e-44f9-a6d9-ed08af94bb6c: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000281095504761', 'lastCheck': 1384917750.190949, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00139403343201', 'lastCheck': 1384917746.1893401, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0019690990448', 'lastCheck': 1384917748.734725, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00334692001343', 'lastCheck': 1384917748.727541, 'valid': True, 'code': 0}}
>Thread-15914092::DEBUG::2013-11-20 03:22:32,986::task::495::TaskManager.Task::(_debug) Task 10477832-f20e-44f9-a6d9-ed08af94bb6c: moving from state preparing -> state finished
>Thread-15914092::DEBUG::2013-11-20 03:22:32,987::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914092::DEBUG::2013-11-20 03:22:32,988::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914092::DEBUG::2013-11-20 03:22:32,989::task::495::TaskManager.Task::(_debug) Task 10477832-f20e-44f9-a6d9-ed08af94bb6c: ref 0 aborting False
>Thread-15914092::INFO::2013-11-20 03:22:32,989::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000281095504761', 'lastCheck': 1384917750.190949, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00139403343201', 'lastCheck': 1384917746.1893401, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0019690990448', 'lastCheck': 1384917748.734725, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00334692001343', 'lastCheck': 1384917748.727541, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:22:33,656::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:22:34,217::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.100393 s, 10.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:22:36,228::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:22:36,807::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0866096 s, 11.8 MB/s\n'; <rc> = 0
>Thread-15914095::DEBUG::2013-11-20 03:22:38,485::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914095::INFO::2013-11-20 03:22:38,487::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914095::DEBUG::2013-11-20 03:22:38,488::task::495::TaskManager.Task::(_debug) Task 80588929-b78a-4ecc-a4cb-2cc10fbb6b88: moving from state init -> state preparing
>Thread-15914095::DEBUG::2013-11-20 03:22:38,489::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914095::DEBUG::2013-11-20 03:22:38,489::task::495::TaskManager.Task::(_debug) Task 80588929-b78a-4ecc-a4cb-2cc10fbb6b88: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914095::DEBUG::2013-11-20 03:22:38,490::task::495::TaskManager.Task::(_debug) Task 80588929-b78a-4ecc-a4cb-2cc10fbb6b88: moving from state preparing -> state finished
>Thread-15914095::DEBUG::2013-11-20 03:22:38,491::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914095::DEBUG::2013-11-20 03:22:38,491::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914095::DEBUG::2013-11-20 03:22:38,492::task::495::TaskManager.Task::(_debug) Task 80588929-b78a-4ecc-a4cb-2cc10fbb6b88: ref 0 aborting False
>Thread-15914095::INFO::2013-11-20 03:22:38,493::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914096::DEBUG::2013-11-20 03:22:38,500::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914096::INFO::2013-11-20 03:22:38,501::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914096::DEBUG::2013-11-20 03:22:38,502::task::495::TaskManager.Task::(_debug) Task 26abd79e-266a-4017-b64e-e5c9728f0773: moving from state init -> state preparing
>Thread-15914096::DEBUG::2013-11-20 03:22:38,508::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e39ff4d1-67af-4a6f-b73a-ec8fa75b6800`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914096::DEBUG::2013-11-20 03:22:38,508::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914096::DEBUG::2013-11-20 03:22:38,509::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914096::DEBUG::2013-11-20 03:22:38,510::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e39ff4d1-67af-4a6f-b73a-ec8fa75b6800`::Granted request
>Thread-15914096::DEBUG::2013-11-20 03:22:38,511::task::495::TaskManager.Task::(_debug) Task 26abd79e-266a-4017-b64e-e5c9728f0773: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914096::DEBUG::2013-11-20 03:22:38,511::task::495::TaskManager.Task::(_debug) Task 26abd79e-266a-4017-b64e-e5c9728f0773: ref 1 aborting False
>Thread-15914096::WARNING::2013-11-20 03:22:38,513::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914096::DEBUG::2013-11-20 03:22:38,514::task::495::TaskManager.Task::(_debug) Task 26abd79e-266a-4017-b64e-e5c9728f0773: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914096::DEBUG::2013-11-20 03:22:38,515::task::495::TaskManager.Task::(_debug) Task 26abd79e-266a-4017-b64e-e5c9728f0773: moving from state preparing -> state finished
>Thread-15914096::DEBUG::2013-11-20 03:22:38,516::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914096::DEBUG::2013-11-20 03:22:38,517::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914096::DEBUG::2013-11-20 03:22:38,517::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914096::DEBUG::2013-11-20 03:22:38,518::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914096::DEBUG::2013-11-20 03:22:38,519::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914096::DEBUG::2013-11-20 03:22:38,519::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914096::DEBUG::2013-11-20 03:22:38,520::task::495::TaskManager.Task::(_debug) Task 26abd79e-266a-4017-b64e-e5c9728f0773: ref 0 aborting False
>Thread-15914096::INFO::2013-11-20 03:22:38,521::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:22:38,822::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:22:39,354::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0754735 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:22:41,370::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 03:22:41,598::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:22:41,599::task::495::TaskManager.Task::(_debug) Task b72a84a5-3dd5-4200-a7d1-c75054822d8d: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:22:41,603::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`79d2f120-6d45-441a-8e92-a6a5e06df341`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:22:41,604::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:22:41,605::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:22:41,606::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`79d2f120-6d45-441a-8e92-a6a5e06df341`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:22:41,608::task::495::TaskManager.Task::(_debug) Task b72a84a5-3dd5-4200-a7d1-c75054822d8d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:22:41,609::task::495::TaskManager.Task::(_debug) Task b72a84a5-3dd5-4200-a7d1-c75054822d8d: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:22:41,610::task::495::TaskManager.Task::(_debug) Task b72a84a5-3dd5-4200-a7d1-c75054822d8d: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:22:41,611::task::495::TaskManager.Task::(_debug) Task b72a84a5-3dd5-4200-a7d1-c75054822d8d: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:22:41,611::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:22:41,612::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:22:41,613::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:22:41,613::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:22:41,614::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:22:41,614::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:22:41,615::task::495::TaskManager.Task::(_debug) Task b72a84a5-3dd5-4200-a7d1-c75054822d8d: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:22:41,615::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:22:41,954::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0766965 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:22:43,970::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 03:22:44,157::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:22:44,160::task::495::TaskManager.Task::(_debug) Task a58bd9cd-70f3-40f5-b59c-054e43d28628: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:22:44,164::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`383a632d-ebc1-4d0e-90e6-96907c637e94`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:22:44,165::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:22:44,166::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:22:44,167::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`383a632d-ebc1-4d0e-90e6-96907c637e94`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:22:44,168::task::495::TaskManager.Task::(_debug) Task a58bd9cd-70f3-40f5-b59c-054e43d28628: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:22:44,169::task::495::TaskManager.Task::(_debug) Task a58bd9cd-70f3-40f5-b59c-054e43d28628: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:22:44,170::task::495::TaskManager.Task::(_debug) Task a58bd9cd-70f3-40f5-b59c-054e43d28628: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:22:44,170::task::495::TaskManager.Task::(_debug) Task a58bd9cd-70f3-40f5-b59c-054e43d28628: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:22:44,171::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:22:44,171::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:22:44,172::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:22:44,173::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:22:44,173::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:22:44,174::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:22:44,174::task::495::TaskManager.Task::(_debug) Task a58bd9cd-70f3-40f5-b59c-054e43d28628: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:22:44,175::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:22:44,498::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0856625 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:22:46,514::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:22:47,094::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0944997 s, 10.8 MB/s\n'; <rc> = 0
>Thread-15914100::INFO::2013-11-20 03:22:48,922::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914100::DEBUG::2013-11-20 03:22:48,923::task::495::TaskManager.Task::(_debug) Task 21d4d327-fb24-43b9-9714-7276d103e164: moving from state init -> state preparing
>Thread-15914100::DEBUG::2013-11-20 03:22:48,924::task::495::TaskManager.Task::(_debug) Task 21d4d327-fb24-43b9-9714-7276d103e164: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000268936157227', 'lastCheck': 1384917760.459975, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00128889083862', 'lastCheck': 1384917766.757694, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0487239360809', 'lastCheck': 1384917759.081192, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0020170211792', 'lastCheck': 1384917759.0255761, 'valid': True, 'code': 0}}
>Thread-15914100::DEBUG::2013-11-20 03:22:48,935::task::495::TaskManager.Task::(_debug) Task 21d4d327-fb24-43b9-9714-7276d103e164: moving from state preparing -> state finished
>Thread-15914100::DEBUG::2013-11-20 03:22:48,936::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914100::DEBUG::2013-11-20 03:22:48,936::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914100::DEBUG::2013-11-20 03:22:48,937::task::495::TaskManager.Task::(_debug) Task 21d4d327-fb24-43b9-9714-7276d103e164: ref 0 aborting False
>Thread-15914100::INFO::2013-11-20 03:22:48,938::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000268936157227', 'lastCheck': 1384917760.459975, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00128889083862', 'lastCheck': 1384917766.757694, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0487239360809', 'lastCheck': 1384917759.081192, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0020170211792', 'lastCheck': 1384917759.0255761, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:22:49,104::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 03:22:49,374::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:22:49,380::task::495::TaskManager.Task::(_debug) Task 937a09a4-d980-4159-ad4e-87f0e3338552: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:22:49,382::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bdcb0c0f-f824-4e8a-a118-b4d96bc62bab`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:22:49,383::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:22:49,385::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914102::DEBUG::2013-11-20 03:22:49,387::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15912392::DEBUG::2013-11-20 03:22:49,387::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bdcb0c0f-f824-4e8a-a118-b4d96bc62bab`::Granted request
>Thread-15914102::INFO::2013-11-20 03:22:49,389::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914102::DEBUG::2013-11-20 03:22:49,390::task::495::TaskManager.Task::(_debug) Task cdf0cc85-5b06-4247-95e8-7b39338761d2: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:22:49,391::task::495::TaskManager.Task::(_debug) Task 937a09a4-d980-4159-ad4e-87f0e3338552: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914102::DEBUG::2013-11-20 03:22:49,392::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15912392::DEBUG::2013-11-20 03:22:49,392::task::495::TaskManager.Task::(_debug) Task 937a09a4-d980-4159-ad4e-87f0e3338552: ref 1 aborting False
>Thread-15914102::DEBUG::2013-11-20 03:22:49,393::task::495::TaskManager.Task::(_debug) Task cdf0cc85-5b06-4247-95e8-7b39338761d2: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912392::DEBUG::2013-11-20 03:22:49,394::task::495::TaskManager.Task::(_debug) Task 937a09a4-d980-4159-ad4e-87f0e3338552: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914102::DEBUG::2013-11-20 03:22:49,395::task::495::TaskManager.Task::(_debug) Task cdf0cc85-5b06-4247-95e8-7b39338761d2: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:22:49,396::task::495::TaskManager.Task::(_debug) Task 937a09a4-d980-4159-ad4e-87f0e3338552: moving from state preparing -> state finished
>Thread-15914102::DEBUG::2013-11-20 03:22:49,396::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912392::DEBUG::2013-11-20 03:22:49,397::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914102::DEBUG::2013-11-20 03:22:49,398::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:22:49,398::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914102::DEBUG::2013-11-20 03:22:49,399::task::495::TaskManager.Task::(_debug) Task cdf0cc85-5b06-4247-95e8-7b39338761d2: ref 0 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:22:49,400::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914102::INFO::2013-11-20 03:22:49,400::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15912392::DEBUG::2013-11-20 03:22:49,401::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:22:49,402::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:22:49,403::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:22:49,405::task::495::TaskManager.Task::(_debug) Task 937a09a4-d980-4159-ad4e-87f0e3338552: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:22:49,405::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914103::DEBUG::2013-11-20 03:22:49,410::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914103::INFO::2013-11-20 03:22:49,411::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914103::DEBUG::2013-11-20 03:22:49,412::task::495::TaskManager.Task::(_debug) Task 3cfa4cb6-7118-47a2-98e8-a6e7fe76301a: moving from state init -> state preparing
>Thread-15914103::DEBUG::2013-11-20 03:22:49,415::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a9f4f14b-c2de-4075-b0a3-2a3c6a87d812`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914103::DEBUG::2013-11-20 03:22:49,416::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914103::DEBUG::2013-11-20 03:22:49,418::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914103::DEBUG::2013-11-20 03:22:49,418::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a9f4f14b-c2de-4075-b0a3-2a3c6a87d812`::Granted request
>Thread-15914103::DEBUG::2013-11-20 03:22:49,419::task::495::TaskManager.Task::(_debug) Task 3cfa4cb6-7118-47a2-98e8-a6e7fe76301a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914103::DEBUG::2013-11-20 03:22:49,420::task::495::TaskManager.Task::(_debug) Task 3cfa4cb6-7118-47a2-98e8-a6e7fe76301a: ref 1 aborting False
>Thread-15914103::WARNING::2013-11-20 03:22:49,422::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914103::DEBUG::2013-11-20 03:22:49,424::task::495::TaskManager.Task::(_debug) Task 3cfa4cb6-7118-47a2-98e8-a6e7fe76301a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914103::DEBUG::2013-11-20 03:22:49,429::task::495::TaskManager.Task::(_debug) Task 3cfa4cb6-7118-47a2-98e8-a6e7fe76301a: moving from state preparing -> state finished
>Thread-15914103::DEBUG::2013-11-20 03:22:49,430::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914103::DEBUG::2013-11-20 03:22:49,432::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914103::DEBUG::2013-11-20 03:22:49,433::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914103::DEBUG::2013-11-20 03:22:49,434::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914103::DEBUG::2013-11-20 03:22:49,435::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914103::DEBUG::2013-11-20 03:22:49,435::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914103::DEBUG::2013-11-20 03:22:49,436::task::495::TaskManager.Task::(_debug) Task 3cfa4cb6-7118-47a2-98e8-a6e7fe76301a: ref 0 aborting False
>Thread-15914103::INFO::2013-11-20 03:22:49,437::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:22:49,645::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0723585 s, 14.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:22:51,658::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:22:52,247::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0860252 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:22:54,264::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:22:54,829::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0848384 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:22:56,845::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:22:57,362::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0810217 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 03:22:58,001::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:22:58,002::task::495::TaskManager.Task::(_debug) Task 1470228f-3684-482c-8162-3d75b9df53d0: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:22:58,003::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d13b05e2-affd-4b4f-b383-8f8d5be1ce66`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:22:58,004::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:22:58,005::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:22:58,006::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d13b05e2-affd-4b4f-b383-8f8d5be1ce66`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:22:58,007::task::495::TaskManager.Task::(_debug) Task 1470228f-3684-482c-8162-3d75b9df53d0: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:22:58,008::task::495::TaskManager.Task::(_debug) Task 1470228f-3684-482c-8162-3d75b9df53d0: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:22:58,008::task::495::TaskManager.Task::(_debug) Task 1470228f-3684-482c-8162-3d75b9df53d0: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:22:58,009::task::495::TaskManager.Task::(_debug) Task 1470228f-3684-482c-8162-3d75b9df53d0: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:22:58,010::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:22:58,011::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:22:58,012::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:22:58,012::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:22:58,013::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:22:58,013::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:22:58,014::task::495::TaskManager.Task::(_debug) Task 1470228f-3684-482c-8162-3d75b9df53d0: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:22:58,015::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:22:58,016::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:22:58,016::task::495::TaskManager.Task::(_debug) Task 43335147-a59a-4d36-bfab-9b52dad991aa: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:22:58,017::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e17d7e12-c7a8-4609-97fb-ac5016a9b44f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:22:58,018::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:22:58,019::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:22:58,020::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e17d7e12-c7a8-4609-97fb-ac5016a9b44f`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:22:58,021::task::495::TaskManager.Task::(_debug) Task 43335147-a59a-4d36-bfab-9b52dad991aa: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:22:58,021::task::495::TaskManager.Task::(_debug) Task 43335147-a59a-4d36-bfab-9b52dad991aa: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:22:58,022::task::495::TaskManager.Task::(_debug) Task 43335147-a59a-4d36-bfab-9b52dad991aa: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:22:58,023::task::495::TaskManager.Task::(_debug) Task 43335147-a59a-4d36-bfab-9b52dad991aa: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:22:58,023::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:22:58,024::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:22:58,026::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:22:58,027::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:22:58,027::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:22:58,028::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:22:58,029::task::495::TaskManager.Task::(_debug) Task 43335147-a59a-4d36-bfab-9b52dad991aa: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:22:58,029::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:22:59,375::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:22:59,950::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0774626 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15914107::DEBUG::2013-11-20 03:23:00,301::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914107::INFO::2013-11-20 03:23:00,303::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914107::DEBUG::2013-11-20 03:23:00,303::task::495::TaskManager.Task::(_debug) Task aac98686-0d20-48b7-ade8-b1c680c5f0eb: moving from state init -> state preparing
>Thread-15914107::DEBUG::2013-11-20 03:23:00,304::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914107::DEBUG::2013-11-20 03:23:00,305::task::495::TaskManager.Task::(_debug) Task aac98686-0d20-48b7-ade8-b1c680c5f0eb: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914107::DEBUG::2013-11-20 03:23:00,306::task::495::TaskManager.Task::(_debug) Task aac98686-0d20-48b7-ade8-b1c680c5f0eb: moving from state preparing -> state finished
>Thread-15914107::DEBUG::2013-11-20 03:23:00,306::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914107::DEBUG::2013-11-20 03:23:00,307::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914107::DEBUG::2013-11-20 03:23:00,307::task::495::TaskManager.Task::(_debug) Task aac98686-0d20-48b7-ade8-b1c680c5f0eb: ref 0 aborting False
>Thread-15914107::INFO::2013-11-20 03:23:00,308::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914108::DEBUG::2013-11-20 03:23:00,320::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914108::INFO::2013-11-20 03:23:00,321::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914108::DEBUG::2013-11-20 03:23:00,322::task::495::TaskManager.Task::(_debug) Task 70281b5d-dd39-412e-b713-2da9b580e6b7: moving from state init -> state preparing
>Thread-15914108::DEBUG::2013-11-20 03:23:00,323::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bcf8cc59-043d-4e92-868e-10f87562fe4a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914108::DEBUG::2013-11-20 03:23:00,324::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914108::DEBUG::2013-11-20 03:23:00,325::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914108::DEBUG::2013-11-20 03:23:00,326::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bcf8cc59-043d-4e92-868e-10f87562fe4a`::Granted request
>Thread-15914108::DEBUG::2013-11-20 03:23:00,332::task::495::TaskManager.Task::(_debug) Task 70281b5d-dd39-412e-b713-2da9b580e6b7: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914108::DEBUG::2013-11-20 03:23:00,333::task::495::TaskManager.Task::(_debug) Task 70281b5d-dd39-412e-b713-2da9b580e6b7: ref 1 aborting False
>Thread-15914108::WARNING::2013-11-20 03:23:00,336::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914108::DEBUG::2013-11-20 03:23:00,337::task::495::TaskManager.Task::(_debug) Task 70281b5d-dd39-412e-b713-2da9b580e6b7: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914108::DEBUG::2013-11-20 03:23:00,338::task::495::TaskManager.Task::(_debug) Task 70281b5d-dd39-412e-b713-2da9b580e6b7: moving from state preparing -> state finished
>Thread-15914108::DEBUG::2013-11-20 03:23:00,339::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914108::DEBUG::2013-11-20 03:23:00,340::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914108::DEBUG::2013-11-20 03:23:00,340::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914108::DEBUG::2013-11-20 03:23:00,341::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914108::DEBUG::2013-11-20 03:23:00,342::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914108::DEBUG::2013-11-20 03:23:00,343::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914108::DEBUG::2013-11-20 03:23:00,343::task::495::TaskManager.Task::(_debug) Task 70281b5d-dd39-412e-b713-2da9b580e6b7: ref 0 aborting False
>Thread-15914108::INFO::2013-11-20 03:23:00,344::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:23:01,961::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:23:02,484::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0711486 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:23:04,500::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914110::INFO::2013-11-20 03:23:04,740::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914110::DEBUG::2013-11-20 03:23:04,742::task::495::TaskManager.Task::(_debug) Task 9d20c0f7-00ba-471c-9289-2051b5078dc6: moving from state init -> state preparing
>Thread-15914110::DEBUG::2013-11-20 03:23:04,743::task::495::TaskManager.Task::(_debug) Task 9d20c0f7-00ba-471c-9289-2051b5078dc6: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000328063964844', 'lastCheck': 1384917781.03455, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00263500213623', 'lastCheck': 1384917777.039649, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00180292129517', 'lastCheck': 1384917779.6180689, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00208306312561', 'lastCheck': 1384917779.607614, 'valid': True, 'code': 0}}
>Thread-15914110::DEBUG::2013-11-20 03:23:04,745::task::495::TaskManager.Task::(_debug) Task 9d20c0f7-00ba-471c-9289-2051b5078dc6: moving from state preparing -> state finished
>Thread-15914110::DEBUG::2013-11-20 03:23:04,758::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914110::DEBUG::2013-11-20 03:23:04,759::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914110::DEBUG::2013-11-20 03:23:04,760::task::495::TaskManager.Task::(_debug) Task 9d20c0f7-00ba-471c-9289-2051b5078dc6: ref 0 aborting False
>Thread-15914110::INFO::2013-11-20 03:23:04,761::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000328063964844', 'lastCheck': 1384917781.03455, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00263500213623', 'lastCheck': 1384917777.039649, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00180292129517', 'lastCheck': 1384917779.6180689, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00208306312561', 'lastCheck': 1384917779.607614, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:23:05,033::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0550246 s, 18.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:23:07,043::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:23:07,608::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0759564 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:23:09,628::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:23:10,117::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0630827 s, 16.2 MB/s\n'; <rc> = 0
>Thread-15914112::DEBUG::2013-11-20 03:23:10,573::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914112::INFO::2013-11-20 03:23:10,574::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914112::DEBUG::2013-11-20 03:23:10,575::task::495::TaskManager.Task::(_debug) Task 0838bb28-f1fb-42c5-a09c-680cc1651140: moving from state init -> state preparing
>Thread-15914112::DEBUG::2013-11-20 03:23:10,575::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914112::DEBUG::2013-11-20 03:23:10,576::task::495::TaskManager.Task::(_debug) Task 0838bb28-f1fb-42c5-a09c-680cc1651140: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914112::DEBUG::2013-11-20 03:23:10,576::task::495::TaskManager.Task::(_debug) Task 0838bb28-f1fb-42c5-a09c-680cc1651140: moving from state preparing -> state finished
>Thread-15914112::DEBUG::2013-11-20 03:23:10,576::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914112::DEBUG::2013-11-20 03:23:10,577::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914112::DEBUG::2013-11-20 03:23:10,577::task::495::TaskManager.Task::(_debug) Task 0838bb28-f1fb-42c5-a09c-680cc1651140: ref 0 aborting False
>Thread-15914112::INFO::2013-11-20 03:23:10,577::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914113::DEBUG::2013-11-20 03:23:10,585::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914113::INFO::2013-11-20 03:23:10,586::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914113::DEBUG::2013-11-20 03:23:10,586::task::495::TaskManager.Task::(_debug) Task dad24318-3776-4303-a4b3-ef520aed7b93: moving from state init -> state preparing
>Thread-15914113::DEBUG::2013-11-20 03:23:10,587::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`320566f4-0e84-4ae6-a52c-9683003f49ca`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914113::DEBUG::2013-11-20 03:23:10,588::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914113::DEBUG::2013-11-20 03:23:10,588::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914113::DEBUG::2013-11-20 03:23:10,588::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`320566f4-0e84-4ae6-a52c-9683003f49ca`::Granted request
>Thread-15914113::DEBUG::2013-11-20 03:23:10,589::task::495::TaskManager.Task::(_debug) Task dad24318-3776-4303-a4b3-ef520aed7b93: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914113::DEBUG::2013-11-20 03:23:10,590::task::495::TaskManager.Task::(_debug) Task dad24318-3776-4303-a4b3-ef520aed7b93: ref 1 aborting False
>Thread-15914113::WARNING::2013-11-20 03:23:10,591::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914113::DEBUG::2013-11-20 03:23:10,592::task::495::TaskManager.Task::(_debug) Task dad24318-3776-4303-a4b3-ef520aed7b93: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914113::DEBUG::2013-11-20 03:23:10,592::task::495::TaskManager.Task::(_debug) Task dad24318-3776-4303-a4b3-ef520aed7b93: moving from state preparing -> state finished
>Thread-15914113::DEBUG::2013-11-20 03:23:10,593::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914113::DEBUG::2013-11-20 03:23:10,593::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914113::DEBUG::2013-11-20 03:23:10,596::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914113::DEBUG::2013-11-20 03:23:10,597::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914113::DEBUG::2013-11-20 03:23:10,597::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914113::DEBUG::2013-11-20 03:23:10,597::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914113::DEBUG::2013-11-20 03:23:10,598::task::495::TaskManager.Task::(_debug) Task dad24318-3776-4303-a4b3-ef520aed7b93: ref 0 aborting False
>Thread-15914113::INFO::2013-11-20 03:23:10,598::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:23:12,128::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:23:12,639::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0683614 s, 15.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:23:14,649::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 03:23:14,875::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:23:14,879::task::495::TaskManager.Task::(_debug) Task 94b86a29-4c51-4d66-95d7-f183b760ac34: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:23:14,881::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2bf086c8-ba5d-4b09-8e2c-01fffb9986aa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:23:14,883::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:23:14,884::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:23:14,885::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2bf086c8-ba5d-4b09-8e2c-01fffb9986aa`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:23:14,886::task::495::TaskManager.Task::(_debug) Task 94b86a29-4c51-4d66-95d7-f183b760ac34: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:23:14,887::task::495::TaskManager.Task::(_debug) Task 94b86a29-4c51-4d66-95d7-f183b760ac34: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:23:14,888::task::495::TaskManager.Task::(_debug) Task 94b86a29-4c51-4d66-95d7-f183b760ac34: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:23:14,889::task::495::TaskManager.Task::(_debug) Task 94b86a29-4c51-4d66-95d7-f183b760ac34: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:23:14,890::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:23:14,890::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:23:14,891::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:23:14,892::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:23:14,893::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:23:14,893::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:23:14,894::task::495::TaskManager.Task::(_debug) Task 94b86a29-4c51-4d66-95d7-f183b760ac34: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:23:14,895::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:23:14,896::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:23:14,896::task::495::TaskManager.Task::(_debug) Task c5160395-314d-41a7-ac06-cbe61feaa8d5: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:23:14,897::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e08ab628-d2ac-4be8-aae1-d108292bcb08`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:23:14,898::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:23:14,899::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:23:14,899::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e08ab628-d2ac-4be8-aae1-d108292bcb08`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:23:14,900::task::495::TaskManager.Task::(_debug) Task c5160395-314d-41a7-ac06-cbe61feaa8d5: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:23:14,901::task::495::TaskManager.Task::(_debug) Task c5160395-314d-41a7-ac06-cbe61feaa8d5: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:23:14,902::task::495::TaskManager.Task::(_debug) Task c5160395-314d-41a7-ac06-cbe61feaa8d5: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:23:14,902::task::495::TaskManager.Task::(_debug) Task c5160395-314d-41a7-ac06-cbe61feaa8d5: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:23:14,903::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:23:14,904::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:23:14,904::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:23:14,905::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:23:14,905::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:23:14,906::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:23:14,907::task::495::TaskManager.Task::(_debug) Task c5160395-314d-41a7-ac06-cbe61feaa8d5: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:23:14,907::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:23:14,908::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:23:14,909::task::495::TaskManager.Task::(_debug) Task 8619e949-c4f5-4dda-8f44-11e61bb19962: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:23:14,910::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`64bcb84b-6a72-4f16-81fe-0d592cf5a5d5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:23:14,911::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:23:14,911::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:23:14,912::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`64bcb84b-6a72-4f16-81fe-0d592cf5a5d5`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:23:14,913::task::495::TaskManager.Task::(_debug) Task 8619e949-c4f5-4dda-8f44-11e61bb19962: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:23:14,914::task::495::TaskManager.Task::(_debug) Task 8619e949-c4f5-4dda-8f44-11e61bb19962: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:23:14,914::task::495::TaskManager.Task::(_debug) Task 8619e949-c4f5-4dda-8f44-11e61bb19962: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:23:14,915::task::495::TaskManager.Task::(_debug) Task 8619e949-c4f5-4dda-8f44-11e61bb19962: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:23:14,915::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:23:14,916::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:23:14,917::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:23:14,917::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:23:14,918::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:23:14,919::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:23:14,919::task::495::TaskManager.Task::(_debug) Task 8619e949-c4f5-4dda-8f44-11e61bb19962: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:23:14,920::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:23:14,921::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:23:14,921::task::495::TaskManager.Task::(_debug) Task c579a0c1-4edd-4e3c-9c77-1975edecb363: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:23:14,922::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a226ad1c-a3d8-4150-8030-0ac300c7ac3a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:23:14,923::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:23:14,924::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:23:14,924::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a226ad1c-a3d8-4150-8030-0ac300c7ac3a`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:23:14,925::task::495::TaskManager.Task::(_debug) Task c579a0c1-4edd-4e3c-9c77-1975edecb363: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:23:14,926::task::495::TaskManager.Task::(_debug) Task c579a0c1-4edd-4e3c-9c77-1975edecb363: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:23:14,927::task::495::TaskManager.Task::(_debug) Task c579a0c1-4edd-4e3c-9c77-1975edecb363: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:23:14,927::task::495::TaskManager.Task::(_debug) Task c579a0c1-4edd-4e3c-9c77-1975edecb363: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:23:14,928::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:23:14,928::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:23:14,929::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:23:14,930::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:23:14,930::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:23:14,931::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:23:14,932::task::495::TaskManager.Task::(_debug) Task c579a0c1-4edd-4e3c-9c77-1975edecb363: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:23:14,932::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:23:15,138::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0699892 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:23:17,150::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:23:17,545::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.049138 s, 20.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:23:19,556::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:23:20,081::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0759965 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15914116::DEBUG::2013-11-20 03:23:20,856::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914116::INFO::2013-11-20 03:23:20,857::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914116::DEBUG::2013-11-20 03:23:20,858::task::495::TaskManager.Task::(_debug) Task 574b867e-05bf-40b8-be78-a8f11d9cc9ce: moving from state init -> state preparing
>Thread-15914116::DEBUG::2013-11-20 03:23:20,858::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914116::DEBUG::2013-11-20 03:23:20,859::task::495::TaskManager.Task::(_debug) Task 574b867e-05bf-40b8-be78-a8f11d9cc9ce: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914116::DEBUG::2013-11-20 03:23:20,859::task::495::TaskManager.Task::(_debug) Task 574b867e-05bf-40b8-be78-a8f11d9cc9ce: moving from state preparing -> state finished
>Thread-15914116::DEBUG::2013-11-20 03:23:20,860::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914116::DEBUG::2013-11-20 03:23:20,860::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914116::DEBUG::2013-11-20 03:23:20,860::task::495::TaskManager.Task::(_debug) Task 574b867e-05bf-40b8-be78-a8f11d9cc9ce: ref 0 aborting False
>Thread-15914116::INFO::2013-11-20 03:23:20,861::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914117::DEBUG::2013-11-20 03:23:20,876::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914117::INFO::2013-11-20 03:23:20,877::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914117::DEBUG::2013-11-20 03:23:20,877::task::495::TaskManager.Task::(_debug) Task 6bf59448-ceb5-48a6-b850-3cfd38867e77: moving from state init -> state preparing
>Thread-15914117::DEBUG::2013-11-20 03:23:20,878::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`57f39d9c-02c4-49d7-a371-7104c7c4b46d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914117::DEBUG::2013-11-20 03:23:20,878::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914117::DEBUG::2013-11-20 03:23:20,879::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914117::DEBUG::2013-11-20 03:23:20,879::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`57f39d9c-02c4-49d7-a371-7104c7c4b46d`::Granted request
>Thread-15914117::DEBUG::2013-11-20 03:23:20,886::task::495::TaskManager.Task::(_debug) Task 6bf59448-ceb5-48a6-b850-3cfd38867e77: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914117::DEBUG::2013-11-20 03:23:20,886::task::495::TaskManager.Task::(_debug) Task 6bf59448-ceb5-48a6-b850-3cfd38867e77: ref 1 aborting False
>Thread-15914117::WARNING::2013-11-20 03:23:20,888::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914117::DEBUG::2013-11-20 03:23:20,889::task::495::TaskManager.Task::(_debug) Task 6bf59448-ceb5-48a6-b850-3cfd38867e77: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914117::DEBUG::2013-11-20 03:23:20,889::task::495::TaskManager.Task::(_debug) Task 6bf59448-ceb5-48a6-b850-3cfd38867e77: moving from state preparing -> state finished
>Thread-15914117::DEBUG::2013-11-20 03:23:20,890::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914117::DEBUG::2013-11-20 03:23:20,890::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914117::DEBUG::2013-11-20 03:23:20,890::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914117::DEBUG::2013-11-20 03:23:20,891::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914117::DEBUG::2013-11-20 03:23:20,891::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914117::DEBUG::2013-11-20 03:23:20,891::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914117::DEBUG::2013-11-20 03:23:20,892::task::495::TaskManager.Task::(_debug) Task 6bf59448-ceb5-48a6-b850-3cfd38867e77: ref 0 aborting False
>Thread-15914117::INFO::2013-11-20 03:23:20,892::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:23:22,096::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:23:22,547::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0644397 s, 15.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:23:24,562::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 03:23:24,892::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:23:24,894::task::495::TaskManager.Task::(_debug) Task b7227519-5aad-4d8a-ba62-a8af5274d98f: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:23:24,898::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8c881f39-b188-4876-b535-d29ec12e171e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:23:24,901::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:23:24,904::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:23:24,906::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8c881f39-b188-4876-b535-d29ec12e171e`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:23:24,910::task::495::TaskManager.Task::(_debug) Task b7227519-5aad-4d8a-ba62-a8af5274d98f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:23:24,911::task::495::TaskManager.Task::(_debug) Task b7227519-5aad-4d8a-ba62-a8af5274d98f: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:23:24,912::task::495::TaskManager.Task::(_debug) Task b7227519-5aad-4d8a-ba62-a8af5274d98f: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:23:24,913::task::495::TaskManager.Task::(_debug) Task b7227519-5aad-4d8a-ba62-a8af5274d98f: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:23:24,914::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:23:24,914::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:23:24,915::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:23:24,916::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:23:24,916::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:23:24,917::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:23:24,918::task::495::TaskManager.Task::(_debug) Task b7227519-5aad-4d8a-ba62-a8af5274d98f: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:23:24,918::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:23:25,069::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0846258 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:23:27,085::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914120::INFO::2013-11-20 03:23:27,250::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914120::DEBUG::2013-11-20 03:23:27,251::task::495::TaskManager.Task::(_debug) Task c324019f-b4e8-4d53-aec6-d06baafd5fc9: moving from state init -> state preparing
>Thread-15914120::DEBUG::2013-11-20 03:23:27,252::task::495::TaskManager.Task::(_debug) Task c324019f-b4e8-4d53-aec6-d06baafd5fc9: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000267028808594', 'lastCheck': 1384917801.3506579, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00118207931519', 'lastCheck': 1384917797.3470709, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00159692764282', 'lastCheck': 1384917799.905041, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000741958618164', 'lastCheck': 1384917799.883111, 'valid': True, 'code': 0}}
>Thread-15914120::DEBUG::2013-11-20 03:23:27,253::task::495::TaskManager.Task::(_debug) Task c324019f-b4e8-4d53-aec6-d06baafd5fc9: moving from state preparing -> state finished
>Thread-15914120::DEBUG::2013-11-20 03:23:27,253::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914120::DEBUG::2013-11-20 03:23:27,254::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914120::DEBUG::2013-11-20 03:23:27,254::task::495::TaskManager.Task::(_debug) Task c324019f-b4e8-4d53-aec6-d06baafd5fc9: ref 0 aborting False
>Thread-15914120::INFO::2013-11-20 03:23:27,254::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000267028808594', 'lastCheck': 1384917801.3506579, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00118207931519', 'lastCheck': 1384917797.3470709, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00159692764282', 'lastCheck': 1384917799.905041, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000741958618164', 'lastCheck': 1384917799.883111, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:23:27,534::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0791832 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:23:29,548::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:23:29,981::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0595611 s, 17.2 MB/s\n'; <rc> = 0
>Thread-15914123::DEBUG::2013-11-20 03:23:31,386::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914123::INFO::2013-11-20 03:23:31,387::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914123::DEBUG::2013-11-20 03:23:31,388::task::495::TaskManager.Task::(_debug) Task a455b583-832c-4acf-9114-bb2000ad2d96: moving from state init -> state preparing
>Thread-15914123::DEBUG::2013-11-20 03:23:31,389::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914123::DEBUG::2013-11-20 03:23:31,389::task::495::TaskManager.Task::(_debug) Task a455b583-832c-4acf-9114-bb2000ad2d96: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914123::DEBUG::2013-11-20 03:23:31,390::task::495::TaskManager.Task::(_debug) Task a455b583-832c-4acf-9114-bb2000ad2d96: moving from state preparing -> state finished
>Thread-15914123::DEBUG::2013-11-20 03:23:31,391::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914123::DEBUG::2013-11-20 03:23:31,391::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914123::DEBUG::2013-11-20 03:23:31,392::task::495::TaskManager.Task::(_debug) Task a455b583-832c-4acf-9114-bb2000ad2d96: ref 0 aborting False
>Thread-15914123::INFO::2013-11-20 03:23:31,392::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914124::DEBUG::2013-11-20 03:23:31,397::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914124::INFO::2013-11-20 03:23:31,398::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914124::DEBUG::2013-11-20 03:23:31,398::task::495::TaskManager.Task::(_debug) Task 59a06318-d81a-4c7c-b4f7-3b0d1dd3d429: moving from state init -> state preparing
>Thread-15914124::DEBUG::2013-11-20 03:23:31,400::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`69c29eae-1f96-4795-a16c-254f36b64117`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914124::DEBUG::2013-11-20 03:23:31,401::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914124::DEBUG::2013-11-20 03:23:31,401::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914124::DEBUG::2013-11-20 03:23:31,402::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`69c29eae-1f96-4795-a16c-254f36b64117`::Granted request
>Thread-15914124::DEBUG::2013-11-20 03:23:31,403::task::495::TaskManager.Task::(_debug) Task 59a06318-d81a-4c7c-b4f7-3b0d1dd3d429: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914124::DEBUG::2013-11-20 03:23:31,404::task::495::TaskManager.Task::(_debug) Task 59a06318-d81a-4c7c-b4f7-3b0d1dd3d429: ref 1 aborting False
>Thread-15914124::WARNING::2013-11-20 03:23:31,406::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914124::DEBUG::2013-11-20 03:23:31,410::task::495::TaskManager.Task::(_debug) Task 59a06318-d81a-4c7c-b4f7-3b0d1dd3d429: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914124::DEBUG::2013-11-20 03:23:31,411::task::495::TaskManager.Task::(_debug) Task 59a06318-d81a-4c7c-b4f7-3b0d1dd3d429: moving from state preparing -> state finished
>Thread-15914124::DEBUG::2013-11-20 03:23:31,412::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914124::DEBUG::2013-11-20 03:23:31,412::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914124::DEBUG::2013-11-20 03:23:31,413::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914124::DEBUG::2013-11-20 03:23:31,413::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914124::DEBUG::2013-11-20 03:23:31,414::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914124::DEBUG::2013-11-20 03:23:31,415::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914124::DEBUG::2013-11-20 03:23:31,415::task::495::TaskManager.Task::(_debug) Task 59a06318-d81a-4c7c-b4f7-3b0d1dd3d429: ref 0 aborting False
>Thread-15914124::INFO::2013-11-20 03:23:31,416::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:23:31,992::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:23:32,471::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0562038 s, 18.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:23:34,481::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:23:35,044::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0917103 s, 11.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:23:37,059::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:23:37,642::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.10011 s, 10.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:23:39,658::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:23:40,205::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0811716 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15914128::DEBUG::2013-11-20 03:23:42,013::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914128::INFO::2013-11-20 03:23:42,015::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914128::DEBUG::2013-11-20 03:23:42,015::task::495::TaskManager.Task::(_debug) Task b9c97805-ad51-4727-bce9-d93b47e31d77: moving from state init -> state preparing
>Thread-15914128::DEBUG::2013-11-20 03:23:42,016::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914128::DEBUG::2013-11-20 03:23:42,017::task::495::TaskManager.Task::(_debug) Task b9c97805-ad51-4727-bce9-d93b47e31d77: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914128::DEBUG::2013-11-20 03:23:42,017::task::495::TaskManager.Task::(_debug) Task b9c97805-ad51-4727-bce9-d93b47e31d77: moving from state preparing -> state finished
>Thread-15914128::DEBUG::2013-11-20 03:23:42,018::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914128::DEBUG::2013-11-20 03:23:42,019::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914128::DEBUG::2013-11-20 03:23:42,019::task::495::TaskManager.Task::(_debug) Task b9c97805-ad51-4727-bce9-d93b47e31d77: ref 0 aborting False
>Thread-15914128::INFO::2013-11-20 03:23:42,020::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914129::DEBUG::2013-11-20 03:23:42,027::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914129::INFO::2013-11-20 03:23:42,028::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914129::DEBUG::2013-11-20 03:23:42,029::task::495::TaskManager.Task::(_debug) Task fdafb356-ef9d-46d8-9818-901ccd46cdeb: moving from state init -> state preparing
>Thread-15914129::DEBUG::2013-11-20 03:23:42,030::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6c898de0-88c5-465c-95d6-a6ffa113ef43`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914129::DEBUG::2013-11-20 03:23:42,031::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914129::DEBUG::2013-11-20 03:23:42,032::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914129::DEBUG::2013-11-20 03:23:42,033::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6c898de0-88c5-465c-95d6-a6ffa113ef43`::Granted request
>Thread-15914129::DEBUG::2013-11-20 03:23:42,034::task::495::TaskManager.Task::(_debug) Task fdafb356-ef9d-46d8-9818-901ccd46cdeb: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914129::DEBUG::2013-11-20 03:23:42,035::task::495::TaskManager.Task::(_debug) Task fdafb356-ef9d-46d8-9818-901ccd46cdeb: ref 1 aborting False
>Thread-15914129::WARNING::2013-11-20 03:23:42,038::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914129::DEBUG::2013-11-20 03:23:42,039::task::495::TaskManager.Task::(_debug) Task fdafb356-ef9d-46d8-9818-901ccd46cdeb: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914129::DEBUG::2013-11-20 03:23:42,040::task::495::TaskManager.Task::(_debug) Task fdafb356-ef9d-46d8-9818-901ccd46cdeb: moving from state preparing -> state finished
>Thread-15914129::DEBUG::2013-11-20 03:23:42,040::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914129::DEBUG::2013-11-20 03:23:42,041::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914129::DEBUG::2013-11-20 03:23:42,042::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914129::DEBUG::2013-11-20 03:23:42,042::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914129::DEBUG::2013-11-20 03:23:42,043::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914129::DEBUG::2013-11-20 03:23:42,044::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914129::DEBUG::2013-11-20 03:23:42,044::task::495::TaskManager.Task::(_debug) Task fdafb356-ef9d-46d8-9818-901ccd46cdeb: ref 0 aborting False
>Thread-15914129::INFO::2013-11-20 03:23:42,045::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:23:42,220::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 03:23:42,463::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:23:42,464::task::495::TaskManager.Task::(_debug) Task bf5e7055-8096-482f-abde-95c4ef671803: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:23:42,465::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`39ea8d71-2bf5-4be3-9917-5ffe5e8158e6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:23:42,466::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:23:42,467::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:23:42,467::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`39ea8d71-2bf5-4be3-9917-5ffe5e8158e6`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:23:42,468::task::495::TaskManager.Task::(_debug) Task bf5e7055-8096-482f-abde-95c4ef671803: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:23:42,468::task::495::TaskManager.Task::(_debug) Task bf5e7055-8096-482f-abde-95c4ef671803: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:23:42,469::task::495::TaskManager.Task::(_debug) Task bf5e7055-8096-482f-abde-95c4ef671803: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:23:42,469::task::495::TaskManager.Task::(_debug) Task bf5e7055-8096-482f-abde-95c4ef671803: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:23:42,470::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:23:42,470::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:23:42,471::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:23:42,471::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:23:42,471::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:23:42,472::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:23:42,472::task::495::TaskManager.Task::(_debug) Task bf5e7055-8096-482f-abde-95c4ef671803: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:23:42,472::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:23:42,646::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0462369 s, 22.1 MB/s\n'; <rc> = 0
>Thread-15914130::INFO::2013-11-20 03:23:42,985::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914130::DEBUG::2013-11-20 03:23:42,986::task::495::TaskManager.Task::(_debug) Task 470cde72-2fba-4805-b411-4d61f2c96e30: moving from state init -> state preparing
>Thread-15914130::DEBUG::2013-11-20 03:23:42,987::task::495::TaskManager.Task::(_debug) Task 470cde72-2fba-4805-b411-4d61f2c96e30: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000257015228271', 'lastCheck': 1384917821.3769021, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00767993927002', 'lastCheck': 1384917817.379355, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000957012176514', 'lastCheck': 1384917819.934777, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00175595283508', 'lastCheck': 1384917819.908145, 'valid': True, 'code': 0}}
>Thread-15914130::DEBUG::2013-11-20 03:23:42,988::task::495::TaskManager.Task::(_debug) Task 470cde72-2fba-4805-b411-4d61f2c96e30: moving from state preparing -> state finished
>Thread-15914130::DEBUG::2013-11-20 03:23:42,988::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914130::DEBUG::2013-11-20 03:23:42,989::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914130::DEBUG::2013-11-20 03:23:42,989::task::495::TaskManager.Task::(_debug) Task 470cde72-2fba-4805-b411-4d61f2c96e30: ref 0 aborting False
>Thread-15914130::INFO::2013-11-20 03:23:42,990::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000257015228271', 'lastCheck': 1384917821.3769021, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00767993927002', 'lastCheck': 1384917817.379355, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000957012176514', 'lastCheck': 1384917819.934777, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00175595283508', 'lastCheck': 1384917819.908145, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:23:44,656::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 03:23:44,916::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:23:44,917::task::495::TaskManager.Task::(_debug) Task 099b39d5-86d6-49bc-ba86-7884c1b038f5: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:23:44,919::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0bd2e0d5-ffaf-4f9d-860f-3ae17d8d1a7a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:23:44,920::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:23:44,921::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:23:44,922::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0bd2e0d5-ffaf-4f9d-860f-3ae17d8d1a7a`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:23:44,923::task::495::TaskManager.Task::(_debug) Task 099b39d5-86d6-49bc-ba86-7884c1b038f5: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:23:44,924::task::495::TaskManager.Task::(_debug) Task 099b39d5-86d6-49bc-ba86-7884c1b038f5: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:23:44,924::task::495::TaskManager.Task::(_debug) Task 099b39d5-86d6-49bc-ba86-7884c1b038f5: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:23:44,925::task::495::TaskManager.Task::(_debug) Task 099b39d5-86d6-49bc-ba86-7884c1b038f5: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:23:44,926::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:23:44,926::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:23:44,927::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:23:44,928::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:23:44,928::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:23:44,929::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:23:44,929::task::495::TaskManager.Task::(_debug) Task 099b39d5-86d6-49bc-ba86-7884c1b038f5: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:23:44,930::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:23:45,149::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0771664 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:23:47,163::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:23:47,690::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.077777 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:23:49,704::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 03:23:49,924::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:23:49,928::task::495::TaskManager.Task::(_debug) Task efed7532-8aa6-4f78-986c-79d63d03db9f: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:23:49,932::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3d3c8c65-133b-4823-89e5-e3dae33d3bd2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:23:49,933::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:23:49,934::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:23:49,934::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3d3c8c65-133b-4823-89e5-e3dae33d3bd2`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:23:49,936::task::495::TaskManager.Task::(_debug) Task efed7532-8aa6-4f78-986c-79d63d03db9f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:23:49,936::task::495::TaskManager.Task::(_debug) Task efed7532-8aa6-4f78-986c-79d63d03db9f: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:23:49,937::task::495::TaskManager.Task::(_debug) Task efed7532-8aa6-4f78-986c-79d63d03db9f: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:23:49,937::task::495::TaskManager.Task::(_debug) Task efed7532-8aa6-4f78-986c-79d63d03db9f: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:23:49,938::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:23:49,939::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:23:49,939::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:23:49,940::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:23:49,940::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:23:49,941::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:23:49,943::task::495::TaskManager.Task::(_debug) Task efed7532-8aa6-4f78-986c-79d63d03db9f: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:23:49,945::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:23:50,226::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0830789 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:23:52,241::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914134::DEBUG::2013-11-20 03:23:52,491::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914134::INFO::2013-11-20 03:23:52,493::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914134::DEBUG::2013-11-20 03:23:52,493::task::495::TaskManager.Task::(_debug) Task 16ce2804-63c4-45e5-99ac-ab7bd72ea92d: moving from state init -> state preparing
>Thread-15914134::DEBUG::2013-11-20 03:23:52,494::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914134::DEBUG::2013-11-20 03:23:52,495::task::495::TaskManager.Task::(_debug) Task 16ce2804-63c4-45e5-99ac-ab7bd72ea92d: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914134::DEBUG::2013-11-20 03:23:52,495::task::495::TaskManager.Task::(_debug) Task 16ce2804-63c4-45e5-99ac-ab7bd72ea92d: moving from state preparing -> state finished
>Thread-15914134::DEBUG::2013-11-20 03:23:52,496::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914134::DEBUG::2013-11-20 03:23:52,496::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914134::DEBUG::2013-11-20 03:23:52,497::task::495::TaskManager.Task::(_debug) Task 16ce2804-63c4-45e5-99ac-ab7bd72ea92d: ref 0 aborting False
>Thread-15914134::INFO::2013-11-20 03:23:52,498::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914135::DEBUG::2013-11-20 03:23:52,502::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914135::INFO::2013-11-20 03:23:52,503::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914135::DEBUG::2013-11-20 03:23:52,504::task::495::TaskManager.Task::(_debug) Task 9a31086f-afee-4515-b2e8-760817bdfbb5: moving from state init -> state preparing
>Thread-15914135::DEBUG::2013-11-20 03:23:52,505::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9c7691b1-4cf9-47b7-87a8-64a70a4c7f3a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914135::DEBUG::2013-11-20 03:23:52,505::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914135::DEBUG::2013-11-20 03:23:52,506::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914135::DEBUG::2013-11-20 03:23:52,506::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9c7691b1-4cf9-47b7-87a8-64a70a4c7f3a`::Granted request
>Thread-15914135::DEBUG::2013-11-20 03:23:52,507::task::495::TaskManager.Task::(_debug) Task 9a31086f-afee-4515-b2e8-760817bdfbb5: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914135::DEBUG::2013-11-20 03:23:52,507::task::495::TaskManager.Task::(_debug) Task 9a31086f-afee-4515-b2e8-760817bdfbb5: ref 1 aborting False
>Thread-15914135::WARNING::2013-11-20 03:23:52,509::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914135::DEBUG::2013-11-20 03:23:52,509::task::495::TaskManager.Task::(_debug) Task 9a31086f-afee-4515-b2e8-760817bdfbb5: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914135::DEBUG::2013-11-20 03:23:52,510::task::495::TaskManager.Task::(_debug) Task 9a31086f-afee-4515-b2e8-760817bdfbb5: moving from state preparing -> state finished
>Thread-15914135::DEBUG::2013-11-20 03:23:52,510::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914135::DEBUG::2013-11-20 03:23:52,510::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914135::DEBUG::2013-11-20 03:23:52,511::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914135::DEBUG::2013-11-20 03:23:52,511::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914135::DEBUG::2013-11-20 03:23:52,512::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914135::DEBUG::2013-11-20 03:23:52,512::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914135::DEBUG::2013-11-20 03:23:52,512::task::495::TaskManager.Task::(_debug) Task 9a31086f-afee-4515-b2e8-760817bdfbb5: ref 0 aborting False
>Thread-15914135::INFO::2013-11-20 03:23:52,513::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:23:52,689::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0611704 s, 16.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:23:54,701::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:23:55,138::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.050262 s, 20.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:23:57,148::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:23:57,579::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0522138 s, 19.6 MB/s\n'; <rc> = 0
>Thread-15914138::INFO::2013-11-20 03:23:58,570::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914138::DEBUG::2013-11-20 03:23:58,571::task::495::TaskManager.Task::(_debug) Task eccb5990-3844-464e-b56e-f52905ad0f62: moving from state init -> state preparing
>Thread-15914138::DEBUG::2013-11-20 03:23:58,572::task::495::TaskManager.Task::(_debug) Task eccb5990-3844-464e-b56e-f52905ad0f62: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000241994857788', 'lastCheck': 1384917831.4261451, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00191593170166', 'lastCheck': 1384917837.4539461, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00144600868225', 'lastCheck': 1384917829.949064, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00685596466064', 'lastCheck': 1384917829.928618, 'valid': True, 'code': 0}}
>Thread-15914138::DEBUG::2013-11-20 03:23:58,572::task::495::TaskManager.Task::(_debug) Task eccb5990-3844-464e-b56e-f52905ad0f62: moving from state preparing -> state finished
>Thread-15914138::DEBUG::2013-11-20 03:23:58,573::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914138::DEBUG::2013-11-20 03:23:58,574::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914138::DEBUG::2013-11-20 03:23:58,574::task::495::TaskManager.Task::(_debug) Task eccb5990-3844-464e-b56e-f52905ad0f62: ref 0 aborting False
>Thread-15914138::INFO::2013-11-20 03:23:58,575::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000241994857788', 'lastCheck': 1384917831.4261451, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00191593170166', 'lastCheck': 1384917837.4539461, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00144600868225', 'lastCheck': 1384917829.949064, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00685596466064', 'lastCheck': 1384917829.928618, 'valid': True, 'code': 0}}
>Thread-15913518::INFO::2013-11-20 03:23:58,991::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:23:58,992::task::495::TaskManager.Task::(_debug) Task 0c58d459-8815-4e03-baef-3526d9041e52: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:23:58,993::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2110a916-4617-403a-b87b-1685900863f1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:23:58,994::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:23:58,994::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:23:58,994::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2110a916-4617-403a-b87b-1685900863f1`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:23:58,995::task::495::TaskManager.Task::(_debug) Task 0c58d459-8815-4e03-baef-3526d9041e52: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:23:58,996::task::495::TaskManager.Task::(_debug) Task 0c58d459-8815-4e03-baef-3526d9041e52: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:23:58,996::task::495::TaskManager.Task::(_debug) Task 0c58d459-8815-4e03-baef-3526d9041e52: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:23:58,996::task::495::TaskManager.Task::(_debug) Task 0c58d459-8815-4e03-baef-3526d9041e52: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:23:58,997::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:23:58,997::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:23:58,998::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:23:58,998::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:23:58,998::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:23:58,999::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:23:58,999::task::495::TaskManager.Task::(_debug) Task 0c58d459-8815-4e03-baef-3526d9041e52: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:23:58,999::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:23:59,000::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:23:59,000::task::495::TaskManager.Task::(_debug) Task 18ea06da-a53c-4ad7-be89-38d5b98993cc: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:23:59,001::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a7c6fb6a-d818-4c0c-8a9a-6f0970d79d6f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:23:59,001::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:23:59,002::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:23:59,002::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a7c6fb6a-d818-4c0c-8a9a-6f0970d79d6f`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:23:59,005::task::495::TaskManager.Task::(_debug) Task 18ea06da-a53c-4ad7-be89-38d5b98993cc: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:23:59,006::task::495::TaskManager.Task::(_debug) Task 18ea06da-a53c-4ad7-be89-38d5b98993cc: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:23:59,006::task::495::TaskManager.Task::(_debug) Task 18ea06da-a53c-4ad7-be89-38d5b98993cc: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:23:59,007::task::495::TaskManager.Task::(_debug) Task 18ea06da-a53c-4ad7-be89-38d5b98993cc: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:23:59,007::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:23:59,007::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:23:59,008::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:23:59,008::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:23:59,008::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:23:59,009::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:23:59,009::task::495::TaskManager.Task::(_debug) Task 18ea06da-a53c-4ad7-be89-38d5b98993cc: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:23:59,010::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913547::DEBUG::2013-11-20 03:23:59,579::guestIF::97::vm.Vm::(_handleMessage) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::u'BRD-KLEE\\administrator'
>Dummy-15912651::DEBUG::2013-11-20 03:23:59,588::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:24:00,141::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0858754 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:24:02,158::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:24:02,674::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0806518 s, 12.7 MB/s\n'; <rc> = 0
>Thread-15914141::DEBUG::2013-11-20 03:24:02,711::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914141::INFO::2013-11-20 03:24:02,713::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914141::DEBUG::2013-11-20 03:24:02,714::task::495::TaskManager.Task::(_debug) Task 8eb97f56-75e9-4a66-8920-5ce3f6a6d9f4: moving from state init -> state preparing
>Thread-15914141::DEBUG::2013-11-20 03:24:02,715::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914141::DEBUG::2013-11-20 03:24:02,716::task::495::TaskManager.Task::(_debug) Task 8eb97f56-75e9-4a66-8920-5ce3f6a6d9f4: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914141::DEBUG::2013-11-20 03:24:02,717::task::495::TaskManager.Task::(_debug) Task 8eb97f56-75e9-4a66-8920-5ce3f6a6d9f4: moving from state preparing -> state finished
>Thread-15914141::DEBUG::2013-11-20 03:24:02,717::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914141::DEBUG::2013-11-20 03:24:02,718::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914141::DEBUG::2013-11-20 03:24:02,718::task::495::TaskManager.Task::(_debug) Task 8eb97f56-75e9-4a66-8920-5ce3f6a6d9f4: ref 0 aborting False
>Thread-15914141::INFO::2013-11-20 03:24:02,719::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914142::DEBUG::2013-11-20 03:24:02,724::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914142::INFO::2013-11-20 03:24:02,725::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914142::DEBUG::2013-11-20 03:24:02,726::task::495::TaskManager.Task::(_debug) Task 17233ea1-ddda-4e49-9935-6c38d4657de0: moving from state init -> state preparing
>Thread-15914142::DEBUG::2013-11-20 03:24:02,727::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`54fe37ee-ee2b-4041-8733-f85c3a320a61`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914142::DEBUG::2013-11-20 03:24:02,728::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914142::DEBUG::2013-11-20 03:24:02,729::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914142::DEBUG::2013-11-20 03:24:02,729::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`54fe37ee-ee2b-4041-8733-f85c3a320a61`::Granted request
>Thread-15914142::DEBUG::2013-11-20 03:24:02,731::task::495::TaskManager.Task::(_debug) Task 17233ea1-ddda-4e49-9935-6c38d4657de0: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914142::DEBUG::2013-11-20 03:24:02,731::task::495::TaskManager.Task::(_debug) Task 17233ea1-ddda-4e49-9935-6c38d4657de0: ref 1 aborting False
>Thread-15914142::WARNING::2013-11-20 03:24:02,734::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914142::DEBUG::2013-11-20 03:24:02,735::task::495::TaskManager.Task::(_debug) Task 17233ea1-ddda-4e49-9935-6c38d4657de0: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914142::DEBUG::2013-11-20 03:24:02,736::task::495::TaskManager.Task::(_debug) Task 17233ea1-ddda-4e49-9935-6c38d4657de0: moving from state preparing -> state finished
>Thread-15914142::DEBUG::2013-11-20 03:24:02,736::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914142::DEBUG::2013-11-20 03:24:02,737::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914142::DEBUG::2013-11-20 03:24:02,737::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914142::DEBUG::2013-11-20 03:24:02,738::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914142::DEBUG::2013-11-20 03:24:02,739::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914142::DEBUG::2013-11-20 03:24:02,739::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914142::DEBUG::2013-11-20 03:24:02,740::task::495::TaskManager.Task::(_debug) Task 17233ea1-ddda-4e49-9935-6c38d4657de0: ref 0 aborting False
>Thread-15914142::INFO::2013-11-20 03:24:02,741::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:24:04,690::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:24:05,115::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0589821 s, 17.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:24:07,136::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:24:07,660::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0784508 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:24:09,676::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:24:10,192::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0790742 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:24:12,208::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:24:12,701::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0883056 s, 11.6 MB/s\n'; <rc> = 0
>Thread-15914146::DEBUG::2013-11-20 03:24:12,976::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914146::INFO::2013-11-20 03:24:12,978::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914146::DEBUG::2013-11-20 03:24:12,978::task::495::TaskManager.Task::(_debug) Task 5b0b0838-2208-485d-b9ae-507fd5c0929c: moving from state init -> state preparing
>Thread-15914146::DEBUG::2013-11-20 03:24:12,979::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914146::DEBUG::2013-11-20 03:24:12,979::task::495::TaskManager.Task::(_debug) Task 5b0b0838-2208-485d-b9ae-507fd5c0929c: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914146::DEBUG::2013-11-20 03:24:12,980::task::495::TaskManager.Task::(_debug) Task 5b0b0838-2208-485d-b9ae-507fd5c0929c: moving from state preparing -> state finished
>Thread-15914146::DEBUG::2013-11-20 03:24:12,980::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914146::DEBUG::2013-11-20 03:24:12,980::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914146::DEBUG::2013-11-20 03:24:12,981::task::495::TaskManager.Task::(_debug) Task 5b0b0838-2208-485d-b9ae-507fd5c0929c: ref 0 aborting False
>Thread-15914146::INFO::2013-11-20 03:24:12,981::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914147::DEBUG::2013-11-20 03:24:12,985::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914147::INFO::2013-11-20 03:24:12,985::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914147::DEBUG::2013-11-20 03:24:12,986::task::495::TaskManager.Task::(_debug) Task 8508f497-f2b2-4195-8573-f2294893c645: moving from state init -> state preparing
>Thread-15914147::DEBUG::2013-11-20 03:24:12,986::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2027148c-98e6-4cdb-92f9-32bfb9c8f9d0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914147::DEBUG::2013-11-20 03:24:12,987::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914147::DEBUG::2013-11-20 03:24:12,987::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914147::DEBUG::2013-11-20 03:24:12,988::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2027148c-98e6-4cdb-92f9-32bfb9c8f9d0`::Granted request
>Thread-15914147::DEBUG::2013-11-20 03:24:12,988::task::495::TaskManager.Task::(_debug) Task 8508f497-f2b2-4195-8573-f2294893c645: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914147::DEBUG::2013-11-20 03:24:12,989::task::495::TaskManager.Task::(_debug) Task 8508f497-f2b2-4195-8573-f2294893c645: ref 1 aborting False
>Thread-15914147::WARNING::2013-11-20 03:24:12,990::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914147::DEBUG::2013-11-20 03:24:12,990::task::495::TaskManager.Task::(_debug) Task 8508f497-f2b2-4195-8573-f2294893c645: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914147::DEBUG::2013-11-20 03:24:12,991::task::495::TaskManager.Task::(_debug) Task 8508f497-f2b2-4195-8573-f2294893c645: moving from state preparing -> state finished
>Thread-15914147::DEBUG::2013-11-20 03:24:12,991::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914147::DEBUG::2013-11-20 03:24:12,992::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914147::DEBUG::2013-11-20 03:24:12,992::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914147::DEBUG::2013-11-20 03:24:12,992::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914147::DEBUG::2013-11-20 03:24:12,993::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914147::DEBUG::2013-11-20 03:24:12,993::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914147::DEBUG::2013-11-20 03:24:12,993::task::495::TaskManager.Task::(_debug) Task 8508f497-f2b2-4195-8573-f2294893c645: ref 0 aborting False
>Thread-15914147::INFO::2013-11-20 03:24:12,994::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914148::INFO::2013-11-20 03:24:13,987::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914148::DEBUG::2013-11-20 03:24:13,987::task::495::TaskManager.Task::(_debug) Task 4a121574-5c90-4143-8a51-7aff0353aa47: moving from state init -> state preparing
>Thread-15914148::DEBUG::2013-11-20 03:24:13,988::task::495::TaskManager.Task::(_debug) Task 4a121574-5c90-4143-8a51-7aff0353aa47: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000250101089478', 'lastCheck': 1384917851.4937401, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0228769779205', 'lastCheck': 1384917847.521054, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00160479545593', 'lastCheck': 1384917850.0028369, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000396966934204', 'lastCheck': 1384917849.9499509, 'valid': True, 'code': 0}}
>Thread-15914148::DEBUG::2013-11-20 03:24:13,988::task::495::TaskManager.Task::(_debug) Task 4a121574-5c90-4143-8a51-7aff0353aa47: moving from state preparing -> state finished
>Thread-15914148::DEBUG::2013-11-20 03:24:13,988::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914148::DEBUG::2013-11-20 03:24:13,989::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914148::DEBUG::2013-11-20 03:24:13,989::task::495::TaskManager.Task::(_debug) Task 4a121574-5c90-4143-8a51-7aff0353aa47: ref 0 aborting False
>Thread-15914148::INFO::2013-11-20 03:24:13,990::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000250101089478', 'lastCheck': 1384917851.4937401, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0228769779205', 'lastCheck': 1384917847.521054, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00160479545593', 'lastCheck': 1384917850.0028369, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000396966934204', 'lastCheck': 1384917849.9499509, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:24:14,716::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 03:24:15,037::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:24:15,039::task::495::TaskManager.Task::(_debug) Task 03173789-5b61-4fd4-bb6c-6bee3f0ffb00: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:24:15,041::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8a87f5a8-3820-4c23-ba08-58a9bb0c7987`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:24:15,042::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:24:15,044::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:24:15,045::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8a87f5a8-3820-4c23-ba08-58a9bb0c7987`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:24:15,048::task::495::TaskManager.Task::(_debug) Task 03173789-5b61-4fd4-bb6c-6bee3f0ffb00: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:24:15,049::task::495::TaskManager.Task::(_debug) Task 03173789-5b61-4fd4-bb6c-6bee3f0ffb00: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:24:15,050::task::495::TaskManager.Task::(_debug) Task 03173789-5b61-4fd4-bb6c-6bee3f0ffb00: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:24:15,051::task::495::TaskManager.Task::(_debug) Task 03173789-5b61-4fd4-bb6c-6bee3f0ffb00: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:24:15,052::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:24:15,053::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:24:15,054::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:24:15,056::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:24:15,057::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:24:15,057::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:24:15,059::task::495::TaskManager.Task::(_debug) Task 03173789-5b61-4fd4-bb6c-6bee3f0ffb00: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:24:15,060::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:24:15,061::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:24:15,062::task::495::TaskManager.Task::(_debug) Task 1ae739bc-351f-4327-8f2a-6cec242ea64e: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:24:15,063::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`124fbf7b-0c6b-486d-9712-5e5137ea7e0f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:24:15,064::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:24:15,066::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:24:15,067::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`124fbf7b-0c6b-486d-9712-5e5137ea7e0f`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:24:15,069::task::495::TaskManager.Task::(_debug) Task 1ae739bc-351f-4327-8f2a-6cec242ea64e: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:24:15,070::task::495::TaskManager.Task::(_debug) Task 1ae739bc-351f-4327-8f2a-6cec242ea64e: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:24:15,071::task::495::TaskManager.Task::(_debug) Task 1ae739bc-351f-4327-8f2a-6cec242ea64e: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:24:15,072::task::495::TaskManager.Task::(_debug) Task 1ae739bc-351f-4327-8f2a-6cec242ea64e: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:24:15,073::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:24:15,074::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:24:15,075::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:24:15,076::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:24:15,077::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:24:15,078::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:24:15,079::task::495::TaskManager.Task::(_debug) Task 1ae739bc-351f-4327-8f2a-6cec242ea64e: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:24:15,080::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:24:15,081::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:24:15,081::task::495::TaskManager.Task::(_debug) Task 4c192c84-1898-4081-a378-302a012f10aa: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:24:15,083::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`349f1cdb-1200-4a79-9c7a-ecd333080015`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:24:15,084::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:24:15,085::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:24:15,085::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`349f1cdb-1200-4a79-9c7a-ecd333080015`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:24:15,087::task::495::TaskManager.Task::(_debug) Task 4c192c84-1898-4081-a378-302a012f10aa: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:24:15,088::task::495::TaskManager.Task::(_debug) Task 4c192c84-1898-4081-a378-302a012f10aa: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:24:15,089::task::495::TaskManager.Task::(_debug) Task 4c192c84-1898-4081-a378-302a012f10aa: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:24:15,089::task::495::TaskManager.Task::(_debug) Task 4c192c84-1898-4081-a378-302a012f10aa: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:24:15,090::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:24:15,091::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:24:15,092::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:24:15,093::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:24:15,094::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:24:15,095::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:24:15,096::task::495::TaskManager.Task::(_debug) Task 4c192c84-1898-4081-a378-302a012f10aa: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:24:15,096::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:24:15,097::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:24:15,098::task::495::TaskManager.Task::(_debug) Task 54743dc4-ceb4-4f86-89ba-84ee55e4171e: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:24:15,100::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`641e0c2c-7048-4f93-8720-3763b7835c49`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:24:15,100::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:24:15,101::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:24:15,102::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`641e0c2c-7048-4f93-8720-3763b7835c49`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:24:15,104::task::495::TaskManager.Task::(_debug) Task 54743dc4-ceb4-4f86-89ba-84ee55e4171e: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:24:15,104::task::495::TaskManager.Task::(_debug) Task 54743dc4-ceb4-4f86-89ba-84ee55e4171e: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:24:15,105::task::495::TaskManager.Task::(_debug) Task 54743dc4-ceb4-4f86-89ba-84ee55e4171e: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:24:15,106::task::495::TaskManager.Task::(_debug) Task 54743dc4-ceb4-4f86-89ba-84ee55e4171e: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:24:15,107::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:24:15,107::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:24:15,108::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:24:15,110::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:24:15,110::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:24:15,111::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:24:15,112::task::495::TaskManager.Task::(_debug) Task 54743dc4-ceb4-4f86-89ba-84ee55e4171e: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:24:15,113::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:24:15,230::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.128048 s, 8.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:24:17,245::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:24:17,690::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0749966 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:24:19,704::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-42840::DEBUG::2013-11-20 03:24:19,962::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:24:20,327::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.221923 s, 4.6 MB/s\n'; <rc> = 0
>Thread-42840::DEBUG::2013-11-20 03:24:20,719::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:24:22,344::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:24:22,753::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0470787 s, 21.8 MB/s\n'; <rc> = 0
>Thread-15914152::DEBUG::2013-11-20 03:24:23,185::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914152::INFO::2013-11-20 03:24:23,186::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914152::DEBUG::2013-11-20 03:24:23,186::task::495::TaskManager.Task::(_debug) Task 3a95c229-1a56-4f4e-9534-4ecd0c72c5b2: moving from state init -> state preparing
>Thread-15914152::DEBUG::2013-11-20 03:24:23,193::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914152::DEBUG::2013-11-20 03:24:23,193::task::495::TaskManager.Task::(_debug) Task 3a95c229-1a56-4f4e-9534-4ecd0c72c5b2: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914152::DEBUG::2013-11-20 03:24:23,194::task::495::TaskManager.Task::(_debug) Task 3a95c229-1a56-4f4e-9534-4ecd0c72c5b2: moving from state preparing -> state finished
>Thread-15914152::DEBUG::2013-11-20 03:24:23,194::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914152::DEBUG::2013-11-20 03:24:23,194::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914152::DEBUG::2013-11-20 03:24:23,195::task::495::TaskManager.Task::(_debug) Task 3a95c229-1a56-4f4e-9534-4ecd0c72c5b2: ref 0 aborting False
>Thread-15914152::INFO::2013-11-20 03:24:23,195::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914153::DEBUG::2013-11-20 03:24:23,198::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914153::INFO::2013-11-20 03:24:23,199::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914153::DEBUG::2013-11-20 03:24:23,199::task::495::TaskManager.Task::(_debug) Task ab3fe7dc-2bab-42d5-aa40-ee0605e032eb: moving from state init -> state preparing
>Thread-15914153::DEBUG::2013-11-20 03:24:23,200::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`092feff0-7e04-447e-b246-22e1a8815fd8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914153::DEBUG::2013-11-20 03:24:23,200::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914153::DEBUG::2013-11-20 03:24:23,201::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914153::DEBUG::2013-11-20 03:24:23,201::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`092feff0-7e04-447e-b246-22e1a8815fd8`::Granted request
>Thread-15914153::DEBUG::2013-11-20 03:24:23,202::task::495::TaskManager.Task::(_debug) Task ab3fe7dc-2bab-42d5-aa40-ee0605e032eb: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914153::DEBUG::2013-11-20 03:24:23,202::task::495::TaskManager.Task::(_debug) Task ab3fe7dc-2bab-42d5-aa40-ee0605e032eb: ref 1 aborting False
>Thread-15914153::WARNING::2013-11-20 03:24:23,203::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914153::DEBUG::2013-11-20 03:24:23,204::task::495::TaskManager.Task::(_debug) Task ab3fe7dc-2bab-42d5-aa40-ee0605e032eb: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914153::DEBUG::2013-11-20 03:24:23,204::task::495::TaskManager.Task::(_debug) Task ab3fe7dc-2bab-42d5-aa40-ee0605e032eb: moving from state preparing -> state finished
>Thread-15914153::DEBUG::2013-11-20 03:24:23,205::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914153::DEBUG::2013-11-20 03:24:23,205::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914153::DEBUG::2013-11-20 03:24:23,205::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914153::DEBUG::2013-11-20 03:24:23,206::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914153::DEBUG::2013-11-20 03:24:23,206::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914153::DEBUG::2013-11-20 03:24:23,206::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914153::DEBUG::2013-11-20 03:24:23,207::task::495::TaskManager.Task::(_debug) Task ab3fe7dc-2bab-42d5-aa40-ee0605e032eb: ref 0 aborting False
>Thread-15914153::INFO::2013-11-20 03:24:23,207::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:24:24,765::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 03:24:25,175::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:24:25,179::task::495::TaskManager.Task::(_debug) Task eca30c8e-dcc9-4db1-8325-54663b3210e1: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:24:25,182::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bd4414ef-d690-46a2-a335-5d392a1613fc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:24:25,183::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:24:25,185::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:24:25,186::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bd4414ef-d690-46a2-a335-5d392a1613fc`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:24:25,188::task::495::TaskManager.Task::(_debug) Task eca30c8e-dcc9-4db1-8325-54663b3210e1: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:24:25,189::task::495::TaskManager.Task::(_debug) Task eca30c8e-dcc9-4db1-8325-54663b3210e1: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:24:25,190::task::495::TaskManager.Task::(_debug) Task eca30c8e-dcc9-4db1-8325-54663b3210e1: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:24:25,192::task::495::TaskManager.Task::(_debug) Task eca30c8e-dcc9-4db1-8325-54663b3210e1: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:24:25,194::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:24:25,195::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:24:25,196::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:24:25,197::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:24:25,198::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:24:25,199::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:24:25,200::task::495::TaskManager.Task::(_debug) Task eca30c8e-dcc9-4db1-8325-54663b3210e1: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:24:25,201::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:24:25,278::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0905736 s, 11.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:24:27,296::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:24:27,766::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0525131 s, 19.5 MB/s\n'; <rc> = 0
>Thread-15914156::INFO::2013-11-20 03:24:29,516::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914156::DEBUG::2013-11-20 03:24:29,517::task::495::TaskManager.Task::(_debug) Task b4521f84-aa53-43f2-9f8c-168df7346179: moving from state init -> state preparing
>Thread-15914156::DEBUG::2013-11-20 03:24:29,525::task::495::TaskManager.Task::(_debug) Task b4521f84-aa53-43f2-9f8c-168df7346179: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000321865081787', 'lastCheck': 1384917861.5068901, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.000999927520752', 'lastCheck': 1384917867.5552089, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00120806694031', 'lastCheck': 1384917860.1748381, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000303983688354', 'lastCheck': 1384917860.7216909, 'valid': True, 'code': 0}}
>Thread-15914156::DEBUG::2013-11-20 03:24:29,526::task::495::TaskManager.Task::(_debug) Task b4521f84-aa53-43f2-9f8c-168df7346179: moving from state preparing -> state finished
>Thread-15914156::DEBUG::2013-11-20 03:24:29,527::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914156::DEBUG::2013-11-20 03:24:29,528::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914156::DEBUG::2013-11-20 03:24:29,528::task::495::TaskManager.Task::(_debug) Task b4521f84-aa53-43f2-9f8c-168df7346179: ref 0 aborting False
>Thread-15914156::INFO::2013-11-20 03:24:29,529::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000321865081787', 'lastCheck': 1384917861.5068901, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.000999927520752', 'lastCheck': 1384917867.5552089, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00120806694031', 'lastCheck': 1384917860.1748381, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000303983688354', 'lastCheck': 1384917860.7216909, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:24:29,776::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:24:30,280::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0911958 s, 11.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:24:32,295::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:24:32,695::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0465249 s, 22.0 MB/s\n'; <rc> = 0
>Thread-15914158::DEBUG::2013-11-20 03:24:33,392::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914158::INFO::2013-11-20 03:24:33,393::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914158::DEBUG::2013-11-20 03:24:33,394::task::495::TaskManager.Task::(_debug) Task 92998aa9-6428-4ea4-b498-550e7e68810b: moving from state init -> state preparing
>Thread-15914158::DEBUG::2013-11-20 03:24:33,394::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914158::DEBUG::2013-11-20 03:24:33,395::task::495::TaskManager.Task::(_debug) Task 92998aa9-6428-4ea4-b498-550e7e68810b: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914158::DEBUG::2013-11-20 03:24:33,395::task::495::TaskManager.Task::(_debug) Task 92998aa9-6428-4ea4-b498-550e7e68810b: moving from state preparing -> state finished
>Thread-15914158::DEBUG::2013-11-20 03:24:33,395::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914158::DEBUG::2013-11-20 03:24:33,396::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914158::DEBUG::2013-11-20 03:24:33,396::task::495::TaskManager.Task::(_debug) Task 92998aa9-6428-4ea4-b498-550e7e68810b: ref 0 aborting False
>Thread-15914158::INFO::2013-11-20 03:24:33,396::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914159::DEBUG::2013-11-20 03:24:33,400::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914159::INFO::2013-11-20 03:24:33,400::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914159::DEBUG::2013-11-20 03:24:33,401::task::495::TaskManager.Task::(_debug) Task 11117f9e-fd9a-49b8-835d-d73c08fd0f79: moving from state init -> state preparing
>Thread-15914159::DEBUG::2013-11-20 03:24:33,401::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`62bca08b-ec85-4e32-86ff-d19b8258c44d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914159::DEBUG::2013-11-20 03:24:33,402::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914159::DEBUG::2013-11-20 03:24:33,402::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914159::DEBUG::2013-11-20 03:24:33,403::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`62bca08b-ec85-4e32-86ff-d19b8258c44d`::Granted request
>Thread-15914159::DEBUG::2013-11-20 03:24:33,409::task::495::TaskManager.Task::(_debug) Task 11117f9e-fd9a-49b8-835d-d73c08fd0f79: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914159::DEBUG::2013-11-20 03:24:33,409::task::495::TaskManager.Task::(_debug) Task 11117f9e-fd9a-49b8-835d-d73c08fd0f79: ref 1 aborting False
>Thread-15914159::WARNING::2013-11-20 03:24:33,411::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914159::DEBUG::2013-11-20 03:24:33,411::task::495::TaskManager.Task::(_debug) Task 11117f9e-fd9a-49b8-835d-d73c08fd0f79: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914159::DEBUG::2013-11-20 03:24:33,412::task::495::TaskManager.Task::(_debug) Task 11117f9e-fd9a-49b8-835d-d73c08fd0f79: moving from state preparing -> state finished
>Thread-15914159::DEBUG::2013-11-20 03:24:33,412::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914159::DEBUG::2013-11-20 03:24:33,412::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914159::DEBUG::2013-11-20 03:24:33,413::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914159::DEBUG::2013-11-20 03:24:33,413::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914159::DEBUG::2013-11-20 03:24:33,413::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914159::DEBUG::2013-11-20 03:24:33,414::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914159::DEBUG::2013-11-20 03:24:33,414::task::495::TaskManager.Task::(_debug) Task 11117f9e-fd9a-49b8-835d-d73c08fd0f79: ref 0 aborting False
>Thread-15914159::INFO::2013-11-20 03:24:33,415::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:24:34,709::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:24:35,127::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0541877 s, 18.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:24:37,138::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:24:37,673::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.125659 s, 8.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:24:39,688::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:24:40,137::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0618561 s, 16.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:24:42,149::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:24:42,546::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0469905 s, 21.8 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 03:24:42,901::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:24:42,901::task::495::TaskManager.Task::(_debug) Task e297488f-5d86-4ee7-8cb2-8c5cc3eecf3f: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:24:42,902::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bb3e02e4-983d-4a7b-99ff-31f30cbecf77`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:24:42,903::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:24:42,903::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:24:42,904::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bb3e02e4-983d-4a7b-99ff-31f30cbecf77`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:24:42,904::task::495::TaskManager.Task::(_debug) Task e297488f-5d86-4ee7-8cb2-8c5cc3eecf3f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:24:42,905::task::495::TaskManager.Task::(_debug) Task e297488f-5d86-4ee7-8cb2-8c5cc3eecf3f: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:24:42,905::task::495::TaskManager.Task::(_debug) Task e297488f-5d86-4ee7-8cb2-8c5cc3eecf3f: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:24:42,906::task::495::TaskManager.Task::(_debug) Task e297488f-5d86-4ee7-8cb2-8c5cc3eecf3f: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:24:42,906::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:24:42,907::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:24:42,907::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:24:42,907::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:24:42,908::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:24:42,908::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:24:42,908::task::495::TaskManager.Task::(_debug) Task e297488f-5d86-4ee7-8cb2-8c5cc3eecf3f: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:24:42,909::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15914164::DEBUG::2013-11-20 03:24:44,071::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914164::INFO::2013-11-20 03:24:44,072::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914164::DEBUG::2013-11-20 03:24:44,073::task::495::TaskManager.Task::(_debug) Task 87558e6e-79db-494f-898a-fb1f80968e1d: moving from state init -> state preparing
>Thread-15914164::DEBUG::2013-11-20 03:24:44,074::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914164::DEBUG::2013-11-20 03:24:44,075::task::495::TaskManager.Task::(_debug) Task 87558e6e-79db-494f-898a-fb1f80968e1d: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914164::DEBUG::2013-11-20 03:24:44,075::task::495::TaskManager.Task::(_debug) Task 87558e6e-79db-494f-898a-fb1f80968e1d: moving from state preparing -> state finished
>Thread-15914164::DEBUG::2013-11-20 03:24:44,076::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914164::DEBUG::2013-11-20 03:24:44,077::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914164::DEBUG::2013-11-20 03:24:44,077::task::495::TaskManager.Task::(_debug) Task 87558e6e-79db-494f-898a-fb1f80968e1d: ref 0 aborting False
>Thread-15914164::INFO::2013-11-20 03:24:44,078::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914165::DEBUG::2013-11-20 03:24:44,083::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914165::INFO::2013-11-20 03:24:44,084::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914165::DEBUG::2013-11-20 03:24:44,084::task::495::TaskManager.Task::(_debug) Task 7d9f1892-c089-4400-931e-a59421baa245: moving from state init -> state preparing
>Thread-15914165::DEBUG::2013-11-20 03:24:44,086::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`09a776cc-88d7-4587-8129-b2bb8fc3c407`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914165::DEBUG::2013-11-20 03:24:44,086::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914165::DEBUG::2013-11-20 03:24:44,087::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914165::DEBUG::2013-11-20 03:24:44,088::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`09a776cc-88d7-4587-8129-b2bb8fc3c407`::Granted request
>Thread-15914165::DEBUG::2013-11-20 03:24:44,089::task::495::TaskManager.Task::(_debug) Task 7d9f1892-c089-4400-931e-a59421baa245: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914165::DEBUG::2013-11-20 03:24:44,089::task::495::TaskManager.Task::(_debug) Task 7d9f1892-c089-4400-931e-a59421baa245: ref 1 aborting False
>Thread-15914165::WARNING::2013-11-20 03:24:44,092::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914165::DEBUG::2013-11-20 03:24:44,093::task::495::TaskManager.Task::(_debug) Task 7d9f1892-c089-4400-931e-a59421baa245: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914165::DEBUG::2013-11-20 03:24:44,094::task::495::TaskManager.Task::(_debug) Task 7d9f1892-c089-4400-931e-a59421baa245: moving from state preparing -> state finished
>Thread-15914165::DEBUG::2013-11-20 03:24:44,094::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914165::DEBUG::2013-11-20 03:24:44,095::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914165::DEBUG::2013-11-20 03:24:44,096::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914165::DEBUG::2013-11-20 03:24:44,096::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914165::DEBUG::2013-11-20 03:24:44,097::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914165::DEBUG::2013-11-20 03:24:44,098::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914165::DEBUG::2013-11-20 03:24:44,098::task::495::TaskManager.Task::(_debug) Task 7d9f1892-c089-4400-931e-a59421baa245: ref 0 aborting False
>Thread-15914165::INFO::2013-11-20 03:24:44,099::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:24:44,559::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:24:44,991::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0511572 s, 20.0 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 03:24:45,320::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:24:45,321::task::495::TaskManager.Task::(_debug) Task d0cb5065-b888-4712-a5a2-75dc882b7cba: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:24:45,324::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`687b363d-e713-43e8-8e4b-d67c9391c4b5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:24:45,325::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:24:45,326::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:24:45,326::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`687b363d-e713-43e8-8e4b-d67c9391c4b5`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:24:45,327::task::495::TaskManager.Task::(_debug) Task d0cb5065-b888-4712-a5a2-75dc882b7cba: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:24:45,328::task::495::TaskManager.Task::(_debug) Task d0cb5065-b888-4712-a5a2-75dc882b7cba: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:24:45,329::task::495::TaskManager.Task::(_debug) Task d0cb5065-b888-4712-a5a2-75dc882b7cba: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:24:45,329::task::495::TaskManager.Task::(_debug) Task d0cb5065-b888-4712-a5a2-75dc882b7cba: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:24:45,330::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:24:45,331::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:24:45,331::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:24:45,332::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:24:45,333::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:24:45,333::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:24:45,334::task::495::TaskManager.Task::(_debug) Task d0cb5065-b888-4712-a5a2-75dc882b7cba: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:24:45,334::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15914166::INFO::2013-11-20 03:24:45,717::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914166::DEBUG::2013-11-20 03:24:45,718::task::495::TaskManager.Task::(_debug) Task 83afa418-4d3d-4e1d-b9c4-b034f89546d7: moving from state init -> state preparing
>Thread-15914166::DEBUG::2013-11-20 03:24:45,719::task::495::TaskManager.Task::(_debug) Task 83afa418-4d3d-4e1d-b9c4-b034f89546d7: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000272035598755', 'lastCheck': 1384917881.54861, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00165200233459', 'lastCheck': 1384917877.5721259, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00133800506592', 'lastCheck': 1384917880.3223679, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000862121582031', 'lastCheck': 1384917880.876127, 'valid': True, 'code': 0}}
>Thread-15914166::DEBUG::2013-11-20 03:24:45,720::task::495::TaskManager.Task::(_debug) Task 83afa418-4d3d-4e1d-b9c4-b034f89546d7: moving from state preparing -> state finished
>Thread-15914166::DEBUG::2013-11-20 03:24:45,720::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914166::DEBUG::2013-11-20 03:24:45,721::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914166::DEBUG::2013-11-20 03:24:45,721::task::495::TaskManager.Task::(_debug) Task 83afa418-4d3d-4e1d-b9c4-b034f89546d7: ref 0 aborting False
>Thread-15914166::INFO::2013-11-20 03:24:45,722::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000272035598755', 'lastCheck': 1384917881.54861, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00165200233459', 'lastCheck': 1384917877.5721259, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00133800506592', 'lastCheck': 1384917880.3223679, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000862121582031', 'lastCheck': 1384917880.876127, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:24:47,002::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:24:47,552::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0765975 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:24:49,567::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:24:50,098::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0742785 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 03:24:50,316::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:24:50,317::task::495::TaskManager.Task::(_debug) Task 45690f3b-9d12-4407-be73-b6033b4dff36: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:24:50,320::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2db53c16-fb83-4866-8ca9-3ffa9c9128cb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:24:50,320::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:24:50,321::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:24:50,322::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2db53c16-fb83-4866-8ca9-3ffa9c9128cb`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:24:50,324::task::495::TaskManager.Task::(_debug) Task 45690f3b-9d12-4407-be73-b6033b4dff36: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:24:50,325::task::495::TaskManager.Task::(_debug) Task 45690f3b-9d12-4407-be73-b6033b4dff36: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:24:50,326::task::495::TaskManager.Task::(_debug) Task 45690f3b-9d12-4407-be73-b6033b4dff36: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:24:50,327::task::495::TaskManager.Task::(_debug) Task 45690f3b-9d12-4407-be73-b6033b4dff36: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:24:50,328::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:24:50,329::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:24:50,331::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:24:50,331::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:24:50,332::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:24:50,334::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:24:50,335::task::495::TaskManager.Task::(_debug) Task 45690f3b-9d12-4407-be73-b6033b4dff36: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:24:50,336::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:24:52,112::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:24:52,619::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0631525 s, 16.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:24:54,634::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914170::DEBUG::2013-11-20 03:24:54,857::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914170::INFO::2013-11-20 03:24:54,860::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914170::DEBUG::2013-11-20 03:24:54,868::task::495::TaskManager.Task::(_debug) Task 2fb80eed-a863-4f0a-8e2d-1df19017e76d: moving from state init -> state preparing
>Thread-15914170::DEBUG::2013-11-20 03:24:54,870::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914170::DEBUG::2013-11-20 03:24:54,870::task::495::TaskManager.Task::(_debug) Task 2fb80eed-a863-4f0a-8e2d-1df19017e76d: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914170::DEBUG::2013-11-20 03:24:54,871::task::495::TaskManager.Task::(_debug) Task 2fb80eed-a863-4f0a-8e2d-1df19017e76d: moving from state preparing -> state finished
>Thread-15914170::DEBUG::2013-11-20 03:24:54,872::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914170::DEBUG::2013-11-20 03:24:54,872::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914170::DEBUG::2013-11-20 03:24:54,873::task::495::TaskManager.Task::(_debug) Task 2fb80eed-a863-4f0a-8e2d-1df19017e76d: ref 0 aborting False
>Thread-15914170::INFO::2013-11-20 03:24:54,873::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914171::DEBUG::2013-11-20 03:24:54,879::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914171::INFO::2013-11-20 03:24:54,880::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914171::DEBUG::2013-11-20 03:24:54,880::task::495::TaskManager.Task::(_debug) Task 96e79e1e-5a02-4f84-b606-851f12a7b524: moving from state init -> state preparing
>Thread-15914171::DEBUG::2013-11-20 03:24:54,882::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a3fbd7df-37f4-424a-a73f-442a77ce5fe0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914171::DEBUG::2013-11-20 03:24:54,882::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914171::DEBUG::2013-11-20 03:24:54,883::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914171::DEBUG::2013-11-20 03:24:54,883::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a3fbd7df-37f4-424a-a73f-442a77ce5fe0`::Granted request
>Thread-15914171::DEBUG::2013-11-20 03:24:54,884::task::495::TaskManager.Task::(_debug) Task 96e79e1e-5a02-4f84-b606-851f12a7b524: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914171::DEBUG::2013-11-20 03:24:54,885::task::495::TaskManager.Task::(_debug) Task 96e79e1e-5a02-4f84-b606-851f12a7b524: ref 1 aborting False
>Thread-15914171::WARNING::2013-11-20 03:24:54,887::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914171::DEBUG::2013-11-20 03:24:54,888::task::495::TaskManager.Task::(_debug) Task 96e79e1e-5a02-4f84-b606-851f12a7b524: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914171::DEBUG::2013-11-20 03:24:54,888::task::495::TaskManager.Task::(_debug) Task 96e79e1e-5a02-4f84-b606-851f12a7b524: moving from state preparing -> state finished
>Thread-15914171::DEBUG::2013-11-20 03:24:54,889::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914171::DEBUG::2013-11-20 03:24:54,889::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914171::DEBUG::2013-11-20 03:24:54,890::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914171::DEBUG::2013-11-20 03:24:54,891::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914171::DEBUG::2013-11-20 03:24:54,891::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914171::DEBUG::2013-11-20 03:24:54,891::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914171::DEBUG::2013-11-20 03:24:54,892::task::495::TaskManager.Task::(_debug) Task 96e79e1e-5a02-4f84-b606-851f12a7b524: ref 0 aborting False
>Thread-15914171::INFO::2013-11-20 03:24:54,892::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:24:55,163::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0730583 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:24:57,178::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:24:57,703::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0869889 s, 11.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:24:59,720::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913518::INFO::2013-11-20 03:24:59,961::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:24:59,962::task::495::TaskManager.Task::(_debug) Task 3fe6af1a-280f-4951-abae-174aef09b7a0: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:24:59,963::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`89bbb32b-5cee-4586-8309-4e1805713f4d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:24:59,964::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:24:59,964::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:24:59,965::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`89bbb32b-5cee-4586-8309-4e1805713f4d`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:24:59,966::task::495::TaskManager.Task::(_debug) Task 3fe6af1a-280f-4951-abae-174aef09b7a0: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:24:59,966::task::495::TaskManager.Task::(_debug) Task 3fe6af1a-280f-4951-abae-174aef09b7a0: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:24:59,967::task::495::TaskManager.Task::(_debug) Task 3fe6af1a-280f-4951-abae-174aef09b7a0: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:24:59,967::task::495::TaskManager.Task::(_debug) Task 3fe6af1a-280f-4951-abae-174aef09b7a0: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:24:59,968::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:24:59,968::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:24:59,968::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:24:59,969::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:24:59,969::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:24:59,970::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:24:59,970::task::495::TaskManager.Task::(_debug) Task 3fe6af1a-280f-4951-abae-174aef09b7a0: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:24:59,970::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:24:59,971::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:24:59,971::task::495::TaskManager.Task::(_debug) Task 4fe37c6f-5bcf-40b0-8493-e376052e9066: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:24:59,972::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`78f84e39-ebf5-4c57-a727-aafe4403c94e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:24:59,972::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:24:59,973::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:24:59,973::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`78f84e39-ebf5-4c57-a727-aafe4403c94e`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:24:59,974::task::495::TaskManager.Task::(_debug) Task 4fe37c6f-5bcf-40b0-8493-e376052e9066: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:24:59,974::task::495::TaskManager.Task::(_debug) Task 4fe37c6f-5bcf-40b0-8493-e376052e9066: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:24:59,975::task::495::TaskManager.Task::(_debug) Task 4fe37c6f-5bcf-40b0-8493-e376052e9066: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:24:59,975::task::495::TaskManager.Task::(_debug) Task 4fe37c6f-5bcf-40b0-8493-e376052e9066: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:24:59,975::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:24:59,976::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:24:59,976::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:24:59,976::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:24:59,977::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:24:59,977::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:24:59,977::task::495::TaskManager.Task::(_debug) Task 4fe37c6f-5bcf-40b0-8493-e376052e9066: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:24:59,978::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:25:00,245::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0799752 s, 12.8 MB/s\n'; <rc> = 0
>Thread-15914174::INFO::2013-11-20 03:25:02,034::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914174::DEBUG::2013-11-20 03:25:02,035::task::495::TaskManager.Task::(_debug) Task ba8cc424-71d5-4a3e-8dfc-ff922cbe6b28: moving from state init -> state preparing
>Thread-15914174::DEBUG::2013-11-20 03:25:02,035::task::495::TaskManager.Task::(_debug) Task ba8cc424-71d5-4a3e-8dfc-ff922cbe6b28: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000346183776855', 'lastCheck': 1384917901.581207, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0186901092529', 'lastCheck': 1384917897.8769, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00669407844543', 'lastCheck': 1384917900.3870571, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0002760887146', 'lastCheck': 1384917900.915432, 'valid': True, 'code': 0}}
>Thread-15914174::DEBUG::2013-11-20 03:25:02,036::task::495::TaskManager.Task::(_debug) Task ba8cc424-71d5-4a3e-8dfc-ff922cbe6b28: moving from state preparing -> state finished
>Thread-15914174::DEBUG::2013-11-20 03:25:02,036::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914174::DEBUG::2013-11-20 03:25:02,037::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914174::DEBUG::2013-11-20 03:25:02,037::task::495::TaskManager.Task::(_debug) Task ba8cc424-71d5-4a3e-8dfc-ff922cbe6b28: ref 0 aborting False
>Thread-15914174::INFO::2013-11-20 03:25:02,037::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000346183776855', 'lastCheck': 1384917901.581207, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0186901092529', 'lastCheck': 1384917897.8769, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00669407844543', 'lastCheck': 1384917900.3870571, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0002760887146', 'lastCheck': 1384917900.915432, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:25:02,258::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:25:02,680::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0728409 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:25:04,694::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:25:05,189::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0459419 s, 22.3 MB/s\n'; <rc> = 0
>Thread-15914176::DEBUG::2013-11-20 03:25:05,349::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914176::INFO::2013-11-20 03:25:05,350::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914176::DEBUG::2013-11-20 03:25:05,351::task::495::TaskManager.Task::(_debug) Task 0204fe53-6a8b-4e20-91e7-7f010e5d60c1: moving from state init -> state preparing
>Thread-15914176::DEBUG::2013-11-20 03:25:05,351::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914176::DEBUG::2013-11-20 03:25:05,351::task::495::TaskManager.Task::(_debug) Task 0204fe53-6a8b-4e20-91e7-7f010e5d60c1: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914176::DEBUG::2013-11-20 03:25:05,352::task::495::TaskManager.Task::(_debug) Task 0204fe53-6a8b-4e20-91e7-7f010e5d60c1: moving from state preparing -> state finished
>Thread-15914176::DEBUG::2013-11-20 03:25:05,352::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914176::DEBUG::2013-11-20 03:25:05,353::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914176::DEBUG::2013-11-20 03:25:05,353::task::495::TaskManager.Task::(_debug) Task 0204fe53-6a8b-4e20-91e7-7f010e5d60c1: ref 0 aborting False
>Thread-15914176::INFO::2013-11-20 03:25:05,353::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914177::DEBUG::2013-11-20 03:25:05,357::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914177::INFO::2013-11-20 03:25:05,358::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914177::DEBUG::2013-11-20 03:25:05,359::task::495::TaskManager.Task::(_debug) Task 26782f95-b7ec-4087-b026-32e8b79e023f: moving from state init -> state preparing
>Thread-15914177::DEBUG::2013-11-20 03:25:05,360::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b5b70b9f-4003-42ab-819a-c03649f4acc7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914177::DEBUG::2013-11-20 03:25:05,361::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914177::DEBUG::2013-11-20 03:25:05,362::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914177::DEBUG::2013-11-20 03:25:05,362::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b5b70b9f-4003-42ab-819a-c03649f4acc7`::Granted request
>Thread-15914177::DEBUG::2013-11-20 03:25:05,363::task::495::TaskManager.Task::(_debug) Task 26782f95-b7ec-4087-b026-32e8b79e023f: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914177::DEBUG::2013-11-20 03:25:05,364::task::495::TaskManager.Task::(_debug) Task 26782f95-b7ec-4087-b026-32e8b79e023f: ref 1 aborting False
>Thread-15914177::WARNING::2013-11-20 03:25:05,366::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914177::DEBUG::2013-11-20 03:25:05,367::task::495::TaskManager.Task::(_debug) Task 26782f95-b7ec-4087-b026-32e8b79e023f: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914177::DEBUG::2013-11-20 03:25:05,368::task::495::TaskManager.Task::(_debug) Task 26782f95-b7ec-4087-b026-32e8b79e023f: moving from state preparing -> state finished
>Thread-15914177::DEBUG::2013-11-20 03:25:05,368::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914177::DEBUG::2013-11-20 03:25:05,369::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914177::DEBUG::2013-11-20 03:25:05,369::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914177::DEBUG::2013-11-20 03:25:05,370::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914177::DEBUG::2013-11-20 03:25:05,370::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914177::DEBUG::2013-11-20 03:25:05,371::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914177::DEBUG::2013-11-20 03:25:05,372::task::495::TaskManager.Task::(_debug) Task 26782f95-b7ec-4087-b026-32e8b79e023f: ref 0 aborting False
>Thread-15914177::INFO::2013-11-20 03:25:05,372::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:25:07,201::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:25:07,676::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.050299 s, 20.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:25:09,684::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:25:10,155::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0599094 s, 17.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:25:12,169::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:25:12,632::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0852778 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:25:14,646::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:25:15,190::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.088593 s, 11.6 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 03:25:15,394::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:25:15,396::task::495::TaskManager.Task::(_debug) Task 72fa7c61-4f17-4fd0-8a63-0c12737af6f2: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:25:15,398::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fd31316e-cbf2-43cf-a4fd-a447e751137b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:25:15,399::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:25:15,400::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:25:15,401::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fd31316e-cbf2-43cf-a4fd-a447e751137b`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:25:15,402::task::495::TaskManager.Task::(_debug) Task 72fa7c61-4f17-4fd0-8a63-0c12737af6f2: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:25:15,402::task::495::TaskManager.Task::(_debug) Task 72fa7c61-4f17-4fd0-8a63-0c12737af6f2: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:25:15,403::task::495::TaskManager.Task::(_debug) Task 72fa7c61-4f17-4fd0-8a63-0c12737af6f2: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:25:15,404::task::495::TaskManager.Task::(_debug) Task 72fa7c61-4f17-4fd0-8a63-0c12737af6f2: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:25:15,404::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:25:15,405::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:25:15,405::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:25:15,406::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:25:15,407::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:25:15,407::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:25:15,408::task::495::TaskManager.Task::(_debug) Task 72fa7c61-4f17-4fd0-8a63-0c12737af6f2: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:25:15,408::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:25:15,409::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:25:15,410::task::495::TaskManager.Task::(_debug) Task 0bf396c1-8beb-4d76-9ed0-cce926a7b63c: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:25:15,411::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`756cfb85-5835-495b-8eab-7e00cde7d716`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:25:15,411::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:25:15,412::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:25:15,412::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`756cfb85-5835-495b-8eab-7e00cde7d716`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:25:15,413::task::495::TaskManager.Task::(_debug) Task 0bf396c1-8beb-4d76-9ed0-cce926a7b63c: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:25:15,414::task::495::TaskManager.Task::(_debug) Task 0bf396c1-8beb-4d76-9ed0-cce926a7b63c: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:25:15,414::task::495::TaskManager.Task::(_debug) Task 0bf396c1-8beb-4d76-9ed0-cce926a7b63c: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:25:15,415::task::495::TaskManager.Task::(_debug) Task 0bf396c1-8beb-4d76-9ed0-cce926a7b63c: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:25:15,415::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:25:15,416::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:25:15,417::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:25:15,417::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:25:15,418::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:25:15,418::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:25:15,419::task::495::TaskManager.Task::(_debug) Task 0bf396c1-8beb-4d76-9ed0-cce926a7b63c: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:25:15,419::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:25:15,420::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:25:15,421::task::495::TaskManager.Task::(_debug) Task e189f7a8-0ac9-4ea2-863a-fc1270340661: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:25:15,422::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`95bff181-5f02-4c1e-aaa1-57d24d272a65`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:25:15,422::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:25:15,423::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:25:15,423::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`95bff181-5f02-4c1e-aaa1-57d24d272a65`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:25:15,424::task::495::TaskManager.Task::(_debug) Task e189f7a8-0ac9-4ea2-863a-fc1270340661: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:25:15,425::task::495::TaskManager.Task::(_debug) Task e189f7a8-0ac9-4ea2-863a-fc1270340661: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:25:15,425::task::495::TaskManager.Task::(_debug) Task e189f7a8-0ac9-4ea2-863a-fc1270340661: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:25:15,426::task::495::TaskManager.Task::(_debug) Task e189f7a8-0ac9-4ea2-863a-fc1270340661: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:25:15,426::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:25:15,427::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:25:15,428::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:25:15,428::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:25:15,429::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:25:15,429::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:25:15,430::task::495::TaskManager.Task::(_debug) Task e189f7a8-0ac9-4ea2-863a-fc1270340661: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:25:15,431::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:25:15,432::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:25:15,432::task::495::TaskManager.Task::(_debug) Task 8ba31eac-9ae6-4a72-b290-b806b3666bc3: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:25:15,433::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`9d33dd12-ea3d-4293-9857-d29668ba2f45`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:25:15,434::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:25:15,435::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:25:15,435::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`9d33dd12-ea3d-4293-9857-d29668ba2f45`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:25:15,436::task::495::TaskManager.Task::(_debug) Task 8ba31eac-9ae6-4a72-b290-b806b3666bc3: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:25:15,436::task::495::TaskManager.Task::(_debug) Task 8ba31eac-9ae6-4a72-b290-b806b3666bc3: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:25:15,437::task::495::TaskManager.Task::(_debug) Task 8ba31eac-9ae6-4a72-b290-b806b3666bc3: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:25:15,438::task::495::TaskManager.Task::(_debug) Task 8ba31eac-9ae6-4a72-b290-b806b3666bc3: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:25:15,438::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:25:15,439::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:25:15,439::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:25:15,440::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:25:15,440::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:25:15,441::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:25:15,441::task::495::TaskManager.Task::(_debug) Task 8ba31eac-9ae6-4a72-b290-b806b3666bc3: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:25:15,442::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15914182::DEBUG::2013-11-20 03:25:16,100::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914182::INFO::2013-11-20 03:25:16,101::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914182::DEBUG::2013-11-20 03:25:16,102::task::495::TaskManager.Task::(_debug) Task f30dbafe-bf59-4032-bb72-ea9a45eac8cc: moving from state init -> state preparing
>Thread-15914182::DEBUG::2013-11-20 03:25:16,103::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914182::DEBUG::2013-11-20 03:25:16,103::task::495::TaskManager.Task::(_debug) Task f30dbafe-bf59-4032-bb72-ea9a45eac8cc: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914182::DEBUG::2013-11-20 03:25:16,104::task::495::TaskManager.Task::(_debug) Task f30dbafe-bf59-4032-bb72-ea9a45eac8cc: moving from state preparing -> state finished
>Thread-15914182::DEBUG::2013-11-20 03:25:16,105::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914182::DEBUG::2013-11-20 03:25:16,105::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914182::DEBUG::2013-11-20 03:25:16,106::task::495::TaskManager.Task::(_debug) Task f30dbafe-bf59-4032-bb72-ea9a45eac8cc: ref 0 aborting False
>Thread-15914182::INFO::2013-11-20 03:25:16,107::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914183::DEBUG::2013-11-20 03:25:16,111::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914183::INFO::2013-11-20 03:25:16,111::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914183::DEBUG::2013-11-20 03:25:16,112::task::495::TaskManager.Task::(_debug) Task 803d5f8e-2d47-4040-b02a-e0a14ecebc15: moving from state init -> state preparing
>Thread-15914183::DEBUG::2013-11-20 03:25:16,112::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`849d1137-1d75-459b-b248-ce459f74760d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914183::DEBUG::2013-11-20 03:25:16,113::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914183::DEBUG::2013-11-20 03:25:16,113::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914183::DEBUG::2013-11-20 03:25:16,114::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`849d1137-1d75-459b-b248-ce459f74760d`::Granted request
>Thread-15914183::DEBUG::2013-11-20 03:25:16,114::task::495::TaskManager.Task::(_debug) Task 803d5f8e-2d47-4040-b02a-e0a14ecebc15: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914183::DEBUG::2013-11-20 03:25:16,114::task::495::TaskManager.Task::(_debug) Task 803d5f8e-2d47-4040-b02a-e0a14ecebc15: ref 1 aborting False
>Thread-15914183::WARNING::2013-11-20 03:25:16,116::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914183::DEBUG::2013-11-20 03:25:16,116::task::495::TaskManager.Task::(_debug) Task 803d5f8e-2d47-4040-b02a-e0a14ecebc15: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914183::DEBUG::2013-11-20 03:25:16,117::task::495::TaskManager.Task::(_debug) Task 803d5f8e-2d47-4040-b02a-e0a14ecebc15: moving from state preparing -> state finished
>Thread-15914183::DEBUG::2013-11-20 03:25:16,117::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914183::DEBUG::2013-11-20 03:25:16,118::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914183::DEBUG::2013-11-20 03:25:16,118::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914183::DEBUG::2013-11-20 03:25:16,119::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914183::DEBUG::2013-11-20 03:25:16,119::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914183::DEBUG::2013-11-20 03:25:16,119::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914183::DEBUG::2013-11-20 03:25:16,120::task::495::TaskManager.Task::(_debug) Task 803d5f8e-2d47-4040-b02a-e0a14ecebc15: ref 0 aborting False
>Thread-15914183::INFO::2013-11-20 03:25:16,120::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:25:17,206::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:25:17,590::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0447946 s, 22.9 MB/s\n'; <rc> = 0
>Thread-15914184::INFO::2013-11-20 03:25:17,956::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914184::DEBUG::2013-11-20 03:25:17,956::task::495::TaskManager.Task::(_debug) Task f2d91abe-211e-477c-9682-a617508f80a5: moving from state init -> state preparing
>Thread-15914184::DEBUG::2013-11-20 03:25:17,957::task::495::TaskManager.Task::(_debug) Task f2d91abe-211e-477c-9682-a617508f80a5: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000247955322266', 'lastCheck': 1384917911.5935581, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00114488601685', 'lastCheck': 1384917917.9459331, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00110197067261', 'lastCheck': 1384917910.429177, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00131106376648', 'lastCheck': 1384917910.9349611, 'valid': True, 'code': 0}}
>Thread-15914184::DEBUG::2013-11-20 03:25:17,958::task::495::TaskManager.Task::(_debug) Task f2d91abe-211e-477c-9682-a617508f80a5: moving from state preparing -> state finished
>Thread-15914184::DEBUG::2013-11-20 03:25:17,958::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914184::DEBUG::2013-11-20 03:25:17,958::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914184::DEBUG::2013-11-20 03:25:17,959::task::495::TaskManager.Task::(_debug) Task f2d91abe-211e-477c-9682-a617508f80a5: ref 0 aborting False
>Thread-15914184::INFO::2013-11-20 03:25:17,959::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000247955322266', 'lastCheck': 1384917911.5935581, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00114488601685', 'lastCheck': 1384917917.9459331, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00110197067261', 'lastCheck': 1384917910.429177, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00131106376648', 'lastCheck': 1384917910.9349611, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:25:19,601::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:25:20,024::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.084817 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:25:22,034::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:25:22,420::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0488885 s, 20.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:25:24,431::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:25:24,925::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0874095 s, 11.7 MB/s\n'; <rc> = 0
>Thread-15913639::INFO::2013-11-20 03:25:25,405::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:25:25,406::task::495::TaskManager.Task::(_debug) Task c6eb6704-faf0-4ad5-9c1f-66b67c5bcb31: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:25:25,408::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6922a98f-58c2-4165-a149-450c7a9601d2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:25:25,409::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:25:25,410::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:25:25,411::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6922a98f-58c2-4165-a149-450c7a9601d2`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:25:25,412::task::495::TaskManager.Task::(_debug) Task c6eb6704-faf0-4ad5-9c1f-66b67c5bcb31: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:25:25,413::task::495::TaskManager.Task::(_debug) Task c6eb6704-faf0-4ad5-9c1f-66b67c5bcb31: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:25:25,414::task::495::TaskManager.Task::(_debug) Task c6eb6704-faf0-4ad5-9c1f-66b67c5bcb31: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:25:25,415::task::495::TaskManager.Task::(_debug) Task c6eb6704-faf0-4ad5-9c1f-66b67c5bcb31: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:25:25,415::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:25:25,416::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:25:25,417::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:25:25,418::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:25:25,418::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:25:25,419::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:25:25,420::task::495::TaskManager.Task::(_debug) Task c6eb6704-faf0-4ad5-9c1f-66b67c5bcb31: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:25:25,420::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15914188::DEBUG::2013-11-20 03:25:26,769::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914188::INFO::2013-11-20 03:25:26,770::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914188::DEBUG::2013-11-20 03:25:26,770::task::495::TaskManager.Task::(_debug) Task 87f2daf2-b02c-48bc-898f-39523ba97365: moving from state init -> state preparing
>Thread-15914188::DEBUG::2013-11-20 03:25:26,771::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914188::DEBUG::2013-11-20 03:25:26,771::task::495::TaskManager.Task::(_debug) Task 87f2daf2-b02c-48bc-898f-39523ba97365: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914188::DEBUG::2013-11-20 03:25:26,772::task::495::TaskManager.Task::(_debug) Task 87f2daf2-b02c-48bc-898f-39523ba97365: moving from state preparing -> state finished
>Thread-15914188::DEBUG::2013-11-20 03:25:26,772::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914188::DEBUG::2013-11-20 03:25:26,772::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914188::DEBUG::2013-11-20 03:25:26,773::task::495::TaskManager.Task::(_debug) Task 87f2daf2-b02c-48bc-898f-39523ba97365: ref 0 aborting False
>Thread-15914188::INFO::2013-11-20 03:25:26,773::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914189::DEBUG::2013-11-20 03:25:26,776::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914189::INFO::2013-11-20 03:25:26,777::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914189::DEBUG::2013-11-20 03:25:26,777::task::495::TaskManager.Task::(_debug) Task 7bf15b61-8265-4084-b2d6-44384af07a0c: moving from state init -> state preparing
>Thread-15914189::DEBUG::2013-11-20 03:25:26,778::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`17070512-edee-4218-ae83-d3f1f91e81c0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914189::DEBUG::2013-11-20 03:25:26,778::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914189::DEBUG::2013-11-20 03:25:26,778::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914189::DEBUG::2013-11-20 03:25:26,779::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`17070512-edee-4218-ae83-d3f1f91e81c0`::Granted request
>Thread-15914189::DEBUG::2013-11-20 03:25:26,779::task::495::TaskManager.Task::(_debug) Task 7bf15b61-8265-4084-b2d6-44384af07a0c: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914189::DEBUG::2013-11-20 03:25:26,780::task::495::TaskManager.Task::(_debug) Task 7bf15b61-8265-4084-b2d6-44384af07a0c: ref 1 aborting False
>Thread-15914189::WARNING::2013-11-20 03:25:26,781::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914189::DEBUG::2013-11-20 03:25:26,781::task::495::TaskManager.Task::(_debug) Task 7bf15b61-8265-4084-b2d6-44384af07a0c: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914189::DEBUG::2013-11-20 03:25:26,782::task::495::TaskManager.Task::(_debug) Task 7bf15b61-8265-4084-b2d6-44384af07a0c: moving from state preparing -> state finished
>Thread-15914189::DEBUG::2013-11-20 03:25:26,782::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914189::DEBUG::2013-11-20 03:25:26,782::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914189::DEBUG::2013-11-20 03:25:26,783::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914189::DEBUG::2013-11-20 03:25:26,783::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914189::DEBUG::2013-11-20 03:25:26,783::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914189::DEBUG::2013-11-20 03:25:26,784::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914189::DEBUG::2013-11-20 03:25:26,784::task::495::TaskManager.Task::(_debug) Task 7bf15b61-8265-4084-b2d6-44384af07a0c: ref 0 aborting False
>Thread-15914189::INFO::2013-11-20 03:25:26,785::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:25:26,941::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:25:27,462::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0792325 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:25:29,474::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:25:29,962::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0702964 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:25:31,976::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:25:32,420::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0713112 s, 14.4 MB/s\n'; <rc> = 0
>Thread-15914192::INFO::2013-11-20 03:25:33,881::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914192::DEBUG::2013-11-20 03:25:33,882::task::495::TaskManager.Task::(_debug) Task 8db60b99-0bc1-48f4-813a-539d01dcc3e8: moving from state init -> state preparing
>Thread-15914192::DEBUG::2013-11-20 03:25:33,883::task::495::TaskManager.Task::(_debug) Task 8db60b99-0bc1-48f4-813a-539d01dcc3e8: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000260829925537', 'lastCheck': 1384917931.6539071, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00118589401245', 'lastCheck': 1384917928.1423759, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.001051902771', 'lastCheck': 1384917930.6570599, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000253915786743', 'lastCheck': 1384917931.1412001, 'valid': True, 'code': 0}}
>Thread-15914192::DEBUG::2013-11-20 03:25:33,884::task::495::TaskManager.Task::(_debug) Task 8db60b99-0bc1-48f4-813a-539d01dcc3e8: moving from state preparing -> state finished
>Thread-15914192::DEBUG::2013-11-20 03:25:33,885::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914192::DEBUG::2013-11-20 03:25:33,885::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914192::DEBUG::2013-11-20 03:25:33,886::task::495::TaskManager.Task::(_debug) Task 8db60b99-0bc1-48f4-813a-539d01dcc3e8: ref 0 aborting False
>Thread-15914192::INFO::2013-11-20 03:25:33,886::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000260829925537', 'lastCheck': 1384917931.6539071, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00118589401245', 'lastCheck': 1384917928.1423759, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.001051902771', 'lastCheck': 1384917930.6570599, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000253915786743', 'lastCheck': 1384917931.1412001, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:25:34,435::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:25:34,877::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.045375 s, 22.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:25:36,888::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914194::DEBUG::2013-11-20 03:25:37,072::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914194::INFO::2013-11-20 03:25:37,074::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914194::DEBUG::2013-11-20 03:25:37,075::task::495::TaskManager.Task::(_debug) Task 534df733-84c4-43da-969a-623012dd8104: moving from state init -> state preparing
>Thread-15914194::DEBUG::2013-11-20 03:25:37,076::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914194::DEBUG::2013-11-20 03:25:37,076::task::495::TaskManager.Task::(_debug) Task 534df733-84c4-43da-969a-623012dd8104: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914194::DEBUG::2013-11-20 03:25:37,077::task::495::TaskManager.Task::(_debug) Task 534df733-84c4-43da-969a-623012dd8104: moving from state preparing -> state finished
>Thread-15914194::DEBUG::2013-11-20 03:25:37,078::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914194::DEBUG::2013-11-20 03:25:37,078::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914194::DEBUG::2013-11-20 03:25:37,079::task::495::TaskManager.Task::(_debug) Task 534df733-84c4-43da-969a-623012dd8104: ref 0 aborting False
>Thread-15914194::INFO::2013-11-20 03:25:37,079::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914195::DEBUG::2013-11-20 03:25:37,083::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914195::INFO::2013-11-20 03:25:37,084::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914195::DEBUG::2013-11-20 03:25:37,084::task::495::TaskManager.Task::(_debug) Task 4423a692-8f49-494b-83bb-bdd4742ade9c: moving from state init -> state preparing
>Thread-15914195::DEBUG::2013-11-20 03:25:37,085::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`68102de6-27a2-426d-83c9-477f80074f8b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914195::DEBUG::2013-11-20 03:25:37,085::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914195::DEBUG::2013-11-20 03:25:37,086::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914195::DEBUG::2013-11-20 03:25:37,086::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`68102de6-27a2-426d-83c9-477f80074f8b`::Granted request
>Thread-15914195::DEBUG::2013-11-20 03:25:37,087::task::495::TaskManager.Task::(_debug) Task 4423a692-8f49-494b-83bb-bdd4742ade9c: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914195::DEBUG::2013-11-20 03:25:37,088::task::495::TaskManager.Task::(_debug) Task 4423a692-8f49-494b-83bb-bdd4742ade9c: ref 1 aborting False
>Thread-15914195::WARNING::2013-11-20 03:25:37,089::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914195::DEBUG::2013-11-20 03:25:37,090::task::495::TaskManager.Task::(_debug) Task 4423a692-8f49-494b-83bb-bdd4742ade9c: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914195::DEBUG::2013-11-20 03:25:37,091::task::495::TaskManager.Task::(_debug) Task 4423a692-8f49-494b-83bb-bdd4742ade9c: moving from state preparing -> state finished
>Thread-15914195::DEBUG::2013-11-20 03:25:37,092::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914195::DEBUG::2013-11-20 03:25:37,092::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914195::DEBUG::2013-11-20 03:25:37,093::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914195::DEBUG::2013-11-20 03:25:37,094::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914195::DEBUG::2013-11-20 03:25:37,094::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914195::DEBUG::2013-11-20 03:25:37,095::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914195::DEBUG::2013-11-20 03:25:37,095::task::495::TaskManager.Task::(_debug) Task 4423a692-8f49-494b-83bb-bdd4742ade9c: ref 0 aborting False
>Thread-15914195::INFO::2013-11-20 03:25:37,096::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:25:37,365::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0657327 s, 15.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:25:39,377::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:25:39,942::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0887424 s, 11.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:25:41,956::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:25:42,500::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0598083 s, 17.1 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 03:25:43,211::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:25:43,211::task::495::TaskManager.Task::(_debug) Task ecea87f6-1929-493d-91c9-6a1b97c0d6ea: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:25:43,213::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0a260628-0b3d-4a9f-b435-3cfbe99e6e9d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:25:43,214::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:25:43,215::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:25:43,215::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0a260628-0b3d-4a9f-b435-3cfbe99e6e9d`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:25:43,216::task::495::TaskManager.Task::(_debug) Task ecea87f6-1929-493d-91c9-6a1b97c0d6ea: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:25:43,217::task::495::TaskManager.Task::(_debug) Task ecea87f6-1929-493d-91c9-6a1b97c0d6ea: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:25:43,218::task::495::TaskManager.Task::(_debug) Task ecea87f6-1929-493d-91c9-6a1b97c0d6ea: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:25:43,218::task::495::TaskManager.Task::(_debug) Task ecea87f6-1929-493d-91c9-6a1b97c0d6ea: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:25:43,219::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:25:43,220::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:25:43,220::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:25:43,221::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:25:43,222::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:25:43,222::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:25:43,223::task::495::TaskManager.Task::(_debug) Task ecea87f6-1929-493d-91c9-6a1b97c0d6ea: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:25:43,224::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:25:44,512::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:25:45,099::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0798212 s, 12.8 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 03:25:45,606::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:25:45,607::task::495::TaskManager.Task::(_debug) Task 0546d03e-f502-4262-96f6-3a5dcb5ef185: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:25:45,609::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fa2e8d4a-fbff-4fd8-abe0-0887b04ae618`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:25:45,610::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:25:45,611::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:25:45,612::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fa2e8d4a-fbff-4fd8-abe0-0887b04ae618`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:25:45,614::task::495::TaskManager.Task::(_debug) Task 0546d03e-f502-4262-96f6-3a5dcb5ef185: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:25:45,614::task::495::TaskManager.Task::(_debug) Task 0546d03e-f502-4262-96f6-3a5dcb5ef185: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:25:45,615::task::495::TaskManager.Task::(_debug) Task 0546d03e-f502-4262-96f6-3a5dcb5ef185: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:25:45,616::task::495::TaskManager.Task::(_debug) Task 0546d03e-f502-4262-96f6-3a5dcb5ef185: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:25:45,617::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:25:45,617::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:25:45,618::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:25:45,619::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:25:45,619::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:25:45,620::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:25:45,620::task::495::TaskManager.Task::(_debug) Task 0546d03e-f502-4262-96f6-3a5dcb5ef185: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:25:45,621::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:25:47,117::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:25:47,536::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0573256 s, 17.9 MB/s\n'; <rc> = 0
>Thread-15914200::DEBUG::2013-11-20 03:25:48,507::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914200::INFO::2013-11-20 03:25:48,508::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914200::DEBUG::2013-11-20 03:25:48,509::task::495::TaskManager.Task::(_debug) Task f8c3d8b2-9c58-47b6-924d-f99648f487a1: moving from state init -> state preparing
>Thread-15914200::DEBUG::2013-11-20 03:25:48,510::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914200::DEBUG::2013-11-20 03:25:48,510::task::495::TaskManager.Task::(_debug) Task f8c3d8b2-9c58-47b6-924d-f99648f487a1: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914200::DEBUG::2013-11-20 03:25:48,511::task::495::TaskManager.Task::(_debug) Task f8c3d8b2-9c58-47b6-924d-f99648f487a1: moving from state preparing -> state finished
>Thread-15914200::DEBUG::2013-11-20 03:25:48,511::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914200::DEBUG::2013-11-20 03:25:48,512::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914200::DEBUG::2013-11-20 03:25:48,513::task::495::TaskManager.Task::(_debug) Task f8c3d8b2-9c58-47b6-924d-f99648f487a1: ref 0 aborting False
>Thread-15914200::INFO::2013-11-20 03:25:48,513::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914201::DEBUG::2013-11-20 03:25:48,517::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914201::INFO::2013-11-20 03:25:48,518::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914201::DEBUG::2013-11-20 03:25:48,519::task::495::TaskManager.Task::(_debug) Task c112046d-0995-4876-ae39-57400cc29044: moving from state init -> state preparing
>Thread-15914201::DEBUG::2013-11-20 03:25:48,520::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`49518d10-e827-4bbc-b130-8ee1aba0189c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914201::DEBUG::2013-11-20 03:25:48,521::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914201::DEBUG::2013-11-20 03:25:48,522::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914201::DEBUG::2013-11-20 03:25:48,522::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`49518d10-e827-4bbc-b130-8ee1aba0189c`::Granted request
>Thread-15914201::DEBUG::2013-11-20 03:25:48,523::task::495::TaskManager.Task::(_debug) Task c112046d-0995-4876-ae39-57400cc29044: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914201::DEBUG::2013-11-20 03:25:48,524::task::495::TaskManager.Task::(_debug) Task c112046d-0995-4876-ae39-57400cc29044: ref 1 aborting False
>Thread-15914201::WARNING::2013-11-20 03:25:48,526::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914201::DEBUG::2013-11-20 03:25:48,527::task::495::TaskManager.Task::(_debug) Task c112046d-0995-4876-ae39-57400cc29044: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914201::DEBUG::2013-11-20 03:25:48,528::task::495::TaskManager.Task::(_debug) Task c112046d-0995-4876-ae39-57400cc29044: moving from state preparing -> state finished
>Thread-15914201::DEBUG::2013-11-20 03:25:48,529::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914201::DEBUG::2013-11-20 03:25:48,529::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914201::DEBUG::2013-11-20 03:25:48,530::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914201::DEBUG::2013-11-20 03:25:48,530::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914201::DEBUG::2013-11-20 03:25:48,531::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914201::DEBUG::2013-11-20 03:25:48,531::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914201::DEBUG::2013-11-20 03:25:48,532::task::495::TaskManager.Task::(_debug) Task c112046d-0995-4876-ae39-57400cc29044: ref 0 aborting False
>Thread-15914201::INFO::2013-11-20 03:25:48,532::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:25:49,547::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914202::INFO::2013-11-20 03:25:49,761::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914202::DEBUG::2013-11-20 03:25:49,762::task::495::TaskManager.Task::(_debug) Task ddf2d631-ad0d-4707-87aa-7db96f8b2bef: moving from state init -> state preparing
>Thread-15914202::DEBUG::2013-11-20 03:25:49,762::task::495::TaskManager.Task::(_debug) Task ddf2d631-ad0d-4707-87aa-7db96f8b2bef: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000285863876343', 'lastCheck': 1384917941.668704, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00100803375244', 'lastCheck': 1384917948.2951529, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00147485733032', 'lastCheck': 1384917940.6751621, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000445127487183', 'lastCheck': 1384917941.1554401, 'valid': True, 'code': 0}}
>Thread-15914202::DEBUG::2013-11-20 03:25:49,763::task::495::TaskManager.Task::(_debug) Task ddf2d631-ad0d-4707-87aa-7db96f8b2bef: moving from state preparing -> state finished
>Thread-15914202::DEBUG::2013-11-20 03:25:49,763::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914202::DEBUG::2013-11-20 03:25:49,764::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914202::DEBUG::2013-11-20 03:25:49,764::task::495::TaskManager.Task::(_debug) Task ddf2d631-ad0d-4707-87aa-7db96f8b2bef: ref 0 aborting False
>Thread-15914202::INFO::2013-11-20 03:25:49,765::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000285863876343', 'lastCheck': 1384917941.668704, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00100803375244', 'lastCheck': 1384917948.2951529, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00147485733032', 'lastCheck': 1384917940.6751621, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000445127487183', 'lastCheck': 1384917941.1554401, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:25:49,984::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.048866 s, 21.0 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 03:25:50,734::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:25:50,735::task::495::TaskManager.Task::(_debug) Task 3ae79593-f7d1-4054-9107-957e645984a2: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:25:50,736::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f5b33a9e-1418-4b65-88b8-ebd2db18169f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:25:50,736::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:25:50,736::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:25:50,737::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f5b33a9e-1418-4b65-88b8-ebd2db18169f`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:25:50,737::task::495::TaskManager.Task::(_debug) Task 3ae79593-f7d1-4054-9107-957e645984a2: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:25:50,738::task::495::TaskManager.Task::(_debug) Task 3ae79593-f7d1-4054-9107-957e645984a2: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:25:50,738::task::495::TaskManager.Task::(_debug) Task 3ae79593-f7d1-4054-9107-957e645984a2: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:25:50,738::task::495::TaskManager.Task::(_debug) Task 3ae79593-f7d1-4054-9107-957e645984a2: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:25:50,739::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:25:50,739::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:25:50,739::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:25:50,740::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:25:50,741::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:25:50,742::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:25:50,742::task::495::TaskManager.Task::(_debug) Task 3ae79593-f7d1-4054-9107-957e645984a2: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:25:50,743::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:25:51,994::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:25:52,591::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0859061 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:25:54,600::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:25:55,040::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0478088 s, 21.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:25:57,050::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:25:57,524::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0854156 s, 12.0 MB/s\n'; <rc> = 0
>Thread-15914206::DEBUG::2013-11-20 03:25:58,820::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914206::INFO::2013-11-20 03:25:58,821::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914206::DEBUG::2013-11-20 03:25:58,821::task::495::TaskManager.Task::(_debug) Task 693ec483-1cac-464f-a324-7f82e3ab7204: moving from state init -> state preparing
>Thread-15914206::DEBUG::2013-11-20 03:25:58,822::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914206::DEBUG::2013-11-20 03:25:58,822::task::495::TaskManager.Task::(_debug) Task 693ec483-1cac-464f-a324-7f82e3ab7204: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914206::DEBUG::2013-11-20 03:25:58,823::task::495::TaskManager.Task::(_debug) Task 693ec483-1cac-464f-a324-7f82e3ab7204: moving from state preparing -> state finished
>Thread-15914206::DEBUG::2013-11-20 03:25:58,823::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914206::DEBUG::2013-11-20 03:25:58,823::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914206::DEBUG::2013-11-20 03:25:58,824::task::495::TaskManager.Task::(_debug) Task 693ec483-1cac-464f-a324-7f82e3ab7204: ref 0 aborting False
>Thread-15914206::INFO::2013-11-20 03:25:58,824::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914207::DEBUG::2013-11-20 03:25:58,828::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914207::INFO::2013-11-20 03:25:58,828::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914207::DEBUG::2013-11-20 03:25:58,829::task::495::TaskManager.Task::(_debug) Task ea4bfaf8-c00d-4ed6-ad93-4f38d7e17f51: moving from state init -> state preparing
>Thread-15914207::DEBUG::2013-11-20 03:25:58,829::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`abf7bcc5-8d0c-4194-ba23-d930b972cb93`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914207::DEBUG::2013-11-20 03:25:58,830::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914207::DEBUG::2013-11-20 03:25:58,830::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914207::DEBUG::2013-11-20 03:25:58,831::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`abf7bcc5-8d0c-4194-ba23-d930b972cb93`::Granted request
>Thread-15914207::DEBUG::2013-11-20 03:25:58,831::task::495::TaskManager.Task::(_debug) Task ea4bfaf8-c00d-4ed6-ad93-4f38d7e17f51: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914207::DEBUG::2013-11-20 03:25:58,832::task::495::TaskManager.Task::(_debug) Task ea4bfaf8-c00d-4ed6-ad93-4f38d7e17f51: ref 1 aborting False
>Thread-15914207::WARNING::2013-11-20 03:25:58,833::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914207::DEBUG::2013-11-20 03:25:58,833::task::495::TaskManager.Task::(_debug) Task ea4bfaf8-c00d-4ed6-ad93-4f38d7e17f51: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914207::DEBUG::2013-11-20 03:25:58,834::task::495::TaskManager.Task::(_debug) Task ea4bfaf8-c00d-4ed6-ad93-4f38d7e17f51: moving from state preparing -> state finished
>Thread-15914207::DEBUG::2013-11-20 03:25:58,834::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914207::DEBUG::2013-11-20 03:25:58,834::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914207::DEBUG::2013-11-20 03:25:58,835::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914207::DEBUG::2013-11-20 03:25:58,835::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914207::DEBUG::2013-11-20 03:25:58,836::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914207::DEBUG::2013-11-20 03:25:58,836::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914207::DEBUG::2013-11-20 03:25:58,836::task::495::TaskManager.Task::(_debug) Task ea4bfaf8-c00d-4ed6-ad93-4f38d7e17f51: ref 0 aborting False
>Thread-15914207::INFO::2013-11-20 03:25:58,837::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:25:59,540::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:25:59,907::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0460046 s, 22.3 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 03:26:00,581::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:26:00,581::task::495::TaskManager.Task::(_debug) Task 6e0efeb8-d144-4d57-8446-100400e18fda: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:26:00,583::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0f269f2b-b10e-4fab-a051-6a345e3d7490`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:26:00,583::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:26:00,584::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:26:00,584::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0f269f2b-b10e-4fab-a051-6a345e3d7490`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:26:00,585::task::495::TaskManager.Task::(_debug) Task 6e0efeb8-d144-4d57-8446-100400e18fda: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:26:00,585::task::495::TaskManager.Task::(_debug) Task 6e0efeb8-d144-4d57-8446-100400e18fda: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:26:00,586::task::495::TaskManager.Task::(_debug) Task 6e0efeb8-d144-4d57-8446-100400e18fda: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:26:00,586::task::495::TaskManager.Task::(_debug) Task 6e0efeb8-d144-4d57-8446-100400e18fda: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:26:00,586::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:26:00,587::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:26:00,587::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:26:00,588::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:26:00,588::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:26:00,588::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:26:00,589::task::495::TaskManager.Task::(_debug) Task 6e0efeb8-d144-4d57-8446-100400e18fda: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:26:00,589::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:26:00,590::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:26:00,590::task::495::TaskManager.Task::(_debug) Task e35924e3-acbc-4530-985a-e4b3df1ce9d9: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:26:00,591::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0d8fbc5b-bd75-42db-b99e-a692ef625fe6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:26:00,592::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:26:00,592::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:26:00,593::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0d8fbc5b-bd75-42db-b99e-a692ef625fe6`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:26:00,593::task::495::TaskManager.Task::(_debug) Task e35924e3-acbc-4530-985a-e4b3df1ce9d9: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:26:00,594::task::495::TaskManager.Task::(_debug) Task e35924e3-acbc-4530-985a-e4b3df1ce9d9: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:26:00,594::task::495::TaskManager.Task::(_debug) Task e35924e3-acbc-4530-985a-e4b3df1ce9d9: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:26:00,595::task::495::TaskManager.Task::(_debug) Task e35924e3-acbc-4530-985a-e4b3df1ce9d9: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:26:00,595::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:26:00,595::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:26:00,596::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:26:00,596::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:26:00,597::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:26:00,597::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:26:00,598::task::495::TaskManager.Task::(_debug) Task e35924e3-acbc-4530-985a-e4b3df1ce9d9: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:26:00,598::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:26:01,917::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:26:02,323::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0521208 s, 19.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:26:04,337::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:26:04,727::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0549529 s, 18.6 MB/s\n'; <rc> = 0
>Thread-15914210::INFO::2013-11-20 03:26:05,200::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914210::DEBUG::2013-11-20 03:26:05,201::task::495::TaskManager.Task::(_debug) Task 096ae64f-e37f-42d2-be8a-3c1bb644fe91: moving from state init -> state preparing
>Thread-15914210::DEBUG::2013-11-20 03:26:05,201::task::495::TaskManager.Task::(_debug) Task 096ae64f-e37f-42d2-be8a-3c1bb644fe91: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000313997268677', 'lastCheck': 1384917961.813659, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0109629631042', 'lastCheck': 1384917958.319757, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00120091438293', 'lastCheck': 1384917960.8119359, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000341892242432', 'lastCheck': 1384917961.301254, 'valid': True, 'code': 0}}
>Thread-15914210::DEBUG::2013-11-20 03:26:05,202::task::495::TaskManager.Task::(_debug) Task 096ae64f-e37f-42d2-be8a-3c1bb644fe91: moving from state preparing -> state finished
>Thread-15914210::DEBUG::2013-11-20 03:26:05,202::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914210::DEBUG::2013-11-20 03:26:05,202::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914210::DEBUG::2013-11-20 03:26:05,203::task::495::TaskManager.Task::(_debug) Task 096ae64f-e37f-42d2-be8a-3c1bb644fe91: ref 0 aborting False
>Thread-15914210::INFO::2013-11-20 03:26:05,203::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000313997268677', 'lastCheck': 1384917961.813659, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0109629631042', 'lastCheck': 1384917958.319757, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00120091438293', 'lastCheck': 1384917960.8119359, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000341892242432', 'lastCheck': 1384917961.301254, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:26:06,743::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:26:07,235::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0646625 s, 15.8 MB/s\n'; <rc> = 0
>Thread-15914213::DEBUG::2013-11-20 03:26:09,084::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914213::INFO::2013-11-20 03:26:09,085::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914213::DEBUG::2013-11-20 03:26:09,085::task::495::TaskManager.Task::(_debug) Task b6abbe36-4eca-42f5-966b-9d9b6fa3ac89: moving from state init -> state preparing
>Thread-15914213::DEBUG::2013-11-20 03:26:09,086::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914213::DEBUG::2013-11-20 03:26:09,086::task::495::TaskManager.Task::(_debug) Task b6abbe36-4eca-42f5-966b-9d9b6fa3ac89: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914213::DEBUG::2013-11-20 03:26:09,087::task::495::TaskManager.Task::(_debug) Task b6abbe36-4eca-42f5-966b-9d9b6fa3ac89: moving from state preparing -> state finished
>Thread-15914213::DEBUG::2013-11-20 03:26:09,087::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914213::DEBUG::2013-11-20 03:26:09,087::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914213::DEBUG::2013-11-20 03:26:09,088::task::495::TaskManager.Task::(_debug) Task b6abbe36-4eca-42f5-966b-9d9b6fa3ac89: ref 0 aborting False
>Thread-15914213::INFO::2013-11-20 03:26:09,088::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914214::DEBUG::2013-11-20 03:26:09,092::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914214::INFO::2013-11-20 03:26:09,093::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914214::DEBUG::2013-11-20 03:26:09,094::task::495::TaskManager.Task::(_debug) Task 3c4fe439-58b0-4721-b549-1e3bac9f34dd: moving from state init -> state preparing
>Thread-15914214::DEBUG::2013-11-20 03:26:09,095::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`668e7355-ee53-48d9-a346-cea9a928ae0e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914214::DEBUG::2013-11-20 03:26:09,095::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914214::DEBUG::2013-11-20 03:26:09,096::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914214::DEBUG::2013-11-20 03:26:09,097::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`668e7355-ee53-48d9-a346-cea9a928ae0e`::Granted request
>Thread-15914214::DEBUG::2013-11-20 03:26:09,098::task::495::TaskManager.Task::(_debug) Task 3c4fe439-58b0-4721-b549-1e3bac9f34dd: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914214::DEBUG::2013-11-20 03:26:09,098::task::495::TaskManager.Task::(_debug) Task 3c4fe439-58b0-4721-b549-1e3bac9f34dd: ref 1 aborting False
>Thread-15914214::WARNING::2013-11-20 03:26:09,100::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914214::DEBUG::2013-11-20 03:26:09,101::task::495::TaskManager.Task::(_debug) Task 3c4fe439-58b0-4721-b549-1e3bac9f34dd: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914214::DEBUG::2013-11-20 03:26:09,102::task::495::TaskManager.Task::(_debug) Task 3c4fe439-58b0-4721-b549-1e3bac9f34dd: moving from state preparing -> state finished
>Thread-15914214::DEBUG::2013-11-20 03:26:09,102::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914214::DEBUG::2013-11-20 03:26:09,103::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914214::DEBUG::2013-11-20 03:26:09,103::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914214::DEBUG::2013-11-20 03:26:09,104::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914214::DEBUG::2013-11-20 03:26:09,105::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914214::DEBUG::2013-11-20 03:26:09,105::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914214::DEBUG::2013-11-20 03:26:09,106::task::495::TaskManager.Task::(_debug) Task 3c4fe439-58b0-4721-b549-1e3bac9f34dd: ref 0 aborting False
>Thread-15914214::INFO::2013-11-20 03:26:09,106::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:26:09,249::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:26:09,745::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0847992 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:26:11,758::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:26:12,266::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0764994 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:26:14,282::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:26:14,823::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0817977 s, 12.5 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 03:26:15,964::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:26:15,965::task::495::TaskManager.Task::(_debug) Task e362fa8a-298b-4186-a1de-ee3272445db5: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:26:15,968::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`9527fbd8-1363-4ef6-ac4f-0e1c2615690e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:26:15,968::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:26:15,969::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:26:15,970::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`9527fbd8-1363-4ef6-ac4f-0e1c2615690e`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:26:15,971::task::495::TaskManager.Task::(_debug) Task e362fa8a-298b-4186-a1de-ee3272445db5: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:26:15,972::task::495::TaskManager.Task::(_debug) Task e362fa8a-298b-4186-a1de-ee3272445db5: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:26:15,973::task::495::TaskManager.Task::(_debug) Task e362fa8a-298b-4186-a1de-ee3272445db5: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:26:15,973::task::495::TaskManager.Task::(_debug) Task e362fa8a-298b-4186-a1de-ee3272445db5: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:26:15,974::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:26:15,974::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:26:15,975::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:26:15,976::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:26:15,976::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:26:15,977::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:26:15,978::task::495::TaskManager.Task::(_debug) Task e362fa8a-298b-4186-a1de-ee3272445db5: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:26:15,978::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:26:15,979::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:26:15,980::task::495::TaskManager.Task::(_debug) Task 32cc8b85-8e09-4b56-8209-e10893b8e494: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:26:15,981::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`3289d277-10c6-409d-a879-29519daf4ec2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:26:15,981::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:26:15,982::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:26:15,983::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`3289d277-10c6-409d-a879-29519daf4ec2`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:26:15,984::task::495::TaskManager.Task::(_debug) Task 32cc8b85-8e09-4b56-8209-e10893b8e494: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:26:15,984::task::495::TaskManager.Task::(_debug) Task 32cc8b85-8e09-4b56-8209-e10893b8e494: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:26:15,985::task::495::TaskManager.Task::(_debug) Task 32cc8b85-8e09-4b56-8209-e10893b8e494: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:26:15,986::task::495::TaskManager.Task::(_debug) Task 32cc8b85-8e09-4b56-8209-e10893b8e494: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:26:15,986::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:26:15,987::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:26:15,988::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:26:15,988::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:26:15,989::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:26:15,989::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:26:15,990::task::495::TaskManager.Task::(_debug) Task 32cc8b85-8e09-4b56-8209-e10893b8e494: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:26:15,991::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:26:15,991::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:26:15,992::task::495::TaskManager.Task::(_debug) Task 1a8e739f-d9b7-4af3-8ccd-0eb351e61164: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:26:15,993::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`7b8abf4b-5a0f-4ce6-a2bc-d583adac623d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:26:15,994::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:26:15,994::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:26:15,995::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`7b8abf4b-5a0f-4ce6-a2bc-d583adac623d`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:26:15,996::task::495::TaskManager.Task::(_debug) Task 1a8e739f-d9b7-4af3-8ccd-0eb351e61164: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:26:15,997::task::495::TaskManager.Task::(_debug) Task 1a8e739f-d9b7-4af3-8ccd-0eb351e61164: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:26:15,997::task::495::TaskManager.Task::(_debug) Task 1a8e739f-d9b7-4af3-8ccd-0eb351e61164: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:26:15,998::task::495::TaskManager.Task::(_debug) Task 1a8e739f-d9b7-4af3-8ccd-0eb351e61164: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:26:15,998::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:26:15,999::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:26:16,000::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:26:16,000::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:26:16,001::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:26:16,001::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:26:16,002::task::495::TaskManager.Task::(_debug) Task 1a8e739f-d9b7-4af3-8ccd-0eb351e61164: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:26:16,002::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:26:16,003::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:26:16,004::task::495::TaskManager.Task::(_debug) Task db3181ee-d388-4103-a00c-2f2d19dfca76: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:26:16,005::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`0985cdb4-42f0-4fcc-84ac-29fb1e59f3bf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:26:16,006::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:26:16,006::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:26:16,007::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`0985cdb4-42f0-4fcc-84ac-29fb1e59f3bf`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:26:16,008::task::495::TaskManager.Task::(_debug) Task db3181ee-d388-4103-a00c-2f2d19dfca76: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:26:16,008::task::495::TaskManager.Task::(_debug) Task db3181ee-d388-4103-a00c-2f2d19dfca76: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:26:16,009::task::495::TaskManager.Task::(_debug) Task db3181ee-d388-4103-a00c-2f2d19dfca76: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:26:16,010::task::495::TaskManager.Task::(_debug) Task db3181ee-d388-4103-a00c-2f2d19dfca76: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:26:16,010::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:26:16,011::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:26:16,011::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:26:16,012::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:26:16,012::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:26:16,013::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:26:16,014::task::495::TaskManager.Task::(_debug) Task db3181ee-d388-4103-a00c-2f2d19dfca76: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:26:16,014::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:26:16,838::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:26:17,280::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0538265 s, 19.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:26:19,290::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914218::DEBUG::2013-11-20 03:26:19,514::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914218::INFO::2013-11-20 03:26:19,517::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914218::DEBUG::2013-11-20 03:26:19,518::task::495::TaskManager.Task::(_debug) Task 081fb30c-d435-4db0-9675-d2a6660e3b7c: moving from state init -> state preparing
>Thread-15914218::DEBUG::2013-11-20 03:26:19,519::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914218::DEBUG::2013-11-20 03:26:19,520::task::495::TaskManager.Task::(_debug) Task 081fb30c-d435-4db0-9675-d2a6660e3b7c: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914218::DEBUG::2013-11-20 03:26:19,521::task::495::TaskManager.Task::(_debug) Task 081fb30c-d435-4db0-9675-d2a6660e3b7c: moving from state preparing -> state finished
>Thread-15914218::DEBUG::2013-11-20 03:26:19,522::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914218::DEBUG::2013-11-20 03:26:19,522::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914218::DEBUG::2013-11-20 03:26:19,523::task::495::TaskManager.Task::(_debug) Task 081fb30c-d435-4db0-9675-d2a6660e3b7c: ref 0 aborting False
>Thread-15914218::INFO::2013-11-20 03:26:19,524::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914219::DEBUG::2013-11-20 03:26:19,529::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914219::INFO::2013-11-20 03:26:19,530::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914219::DEBUG::2013-11-20 03:26:19,530::task::495::TaskManager.Task::(_debug) Task 5acea737-22cd-4bf6-bf58-3d6c84e224ac: moving from state init -> state preparing
>Thread-15914219::DEBUG::2013-11-20 03:26:19,531::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0d38fa27-2126-4084-ab9c-12aed4f70e06`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914219::DEBUG::2013-11-20 03:26:19,532::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914219::DEBUG::2013-11-20 03:26:19,532::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914219::DEBUG::2013-11-20 03:26:19,533::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0d38fa27-2126-4084-ab9c-12aed4f70e06`::Granted request
>Thread-15914219::DEBUG::2013-11-20 03:26:19,534::task::495::TaskManager.Task::(_debug) Task 5acea737-22cd-4bf6-bf58-3d6c84e224ac: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914219::DEBUG::2013-11-20 03:26:19,534::task::495::TaskManager.Task::(_debug) Task 5acea737-22cd-4bf6-bf58-3d6c84e224ac: ref 1 aborting False
>Thread-15914219::WARNING::2013-11-20 03:26:19,536::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914219::DEBUG::2013-11-20 03:26:19,537::task::495::TaskManager.Task::(_debug) Task 5acea737-22cd-4bf6-bf58-3d6c84e224ac: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914219::DEBUG::2013-11-20 03:26:19,537::task::495::TaskManager.Task::(_debug) Task 5acea737-22cd-4bf6-bf58-3d6c84e224ac: moving from state preparing -> state finished
>Thread-15914219::DEBUG::2013-11-20 03:26:19,538::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914219::DEBUG::2013-11-20 03:26:19,538::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914219::DEBUG::2013-11-20 03:26:19,538::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914219::DEBUG::2013-11-20 03:26:19,539::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914219::DEBUG::2013-11-20 03:26:19,539::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914219::DEBUG::2013-11-20 03:26:19,539::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914219::DEBUG::2013-11-20 03:26:19,540::task::495::TaskManager.Task::(_debug) Task 5acea737-22cd-4bf6-bf58-3d6c84e224ac: ref 0 aborting False
>Thread-15914219::INFO::2013-11-20 03:26:19,540::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:26:19,771::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0504943 s, 20.3 MB/s\n'; <rc> = 0
>Thread-15914220::INFO::2013-11-20 03:26:20,646::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914220::DEBUG::2013-11-20 03:26:20,647::task::495::TaskManager.Task::(_debug) Task a9957d1d-28e8-44f1-a7c4-328930e58236: moving from state init -> state preparing
>Thread-15914220::DEBUG::2013-11-20 03:26:20,647::task::495::TaskManager.Task::(_debug) Task a9957d1d-28e8-44f1-a7c4-328930e58236: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000365018844604', 'lastCheck': 1384917971.9667611, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0106649398804', 'lastCheck': 1384917978.5083621, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0129110813141', 'lastCheck': 1384917970.9620841, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000373125076294', 'lastCheck': 1384917971.423285, 'valid': True, 'code': 0}}
>Thread-15914220::DEBUG::2013-11-20 03:26:20,648::task::495::TaskManager.Task::(_debug) Task a9957d1d-28e8-44f1-a7c4-328930e58236: moving from state preparing -> state finished
>Thread-15914220::DEBUG::2013-11-20 03:26:20,648::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914220::DEBUG::2013-11-20 03:26:20,648::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914220::DEBUG::2013-11-20 03:26:20,649::task::495::TaskManager.Task::(_debug) Task a9957d1d-28e8-44f1-a7c4-328930e58236: ref 0 aborting False
>Thread-15914220::INFO::2013-11-20 03:26:20,649::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000365018844604', 'lastCheck': 1384917971.9667611, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0106649398804', 'lastCheck': 1384917978.5083621, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0129110813141', 'lastCheck': 1384917970.9620841, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000373125076294', 'lastCheck': 1384917971.423285, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:26:21,789::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:26:22,246::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0664918 s, 15.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:26:24,259::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:26:24,770::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0725052 s, 14.1 MB/s\n'; <rc> = 0
>Thread-15913639::INFO::2013-11-20 03:26:25,999::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:26:26,000::task::495::TaskManager.Task::(_debug) Task 8336ef99-987f-4ceb-a645-9fa9940f5685: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:26:26,002::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fadbab66-666b-4f5a-a40a-d90923d1e231`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:26:26,002::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:26:26,003::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:26:26,003::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fadbab66-666b-4f5a-a40a-d90923d1e231`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:26:26,004::task::495::TaskManager.Task::(_debug) Task 8336ef99-987f-4ceb-a645-9fa9940f5685: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:26:26,005::task::495::TaskManager.Task::(_debug) Task 8336ef99-987f-4ceb-a645-9fa9940f5685: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:26:26,005::task::495::TaskManager.Task::(_debug) Task 8336ef99-987f-4ceb-a645-9fa9940f5685: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:26:26,006::task::495::TaskManager.Task::(_debug) Task 8336ef99-987f-4ceb-a645-9fa9940f5685: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:26:26,006::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:26:26,006::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:26:26,007::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:26:26,007::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:26:26,008::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:26:26,008::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:26:26,008::task::495::TaskManager.Task::(_debug) Task 8336ef99-987f-4ceb-a645-9fa9940f5685: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:26:26,009::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:26:26,784::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:26:27,333::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0802852 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:26:29,346::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914224::DEBUG::2013-11-20 03:26:29,748::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914224::INFO::2013-11-20 03:26:29,751::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914224::DEBUG::2013-11-20 03:26:29,751::task::495::TaskManager.Task::(_debug) Task 7fca5584-106f-4e96-88a2-40eb33c92871: moving from state init -> state preparing
>Thread-15914224::DEBUG::2013-11-20 03:26:29,753::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914224::DEBUG::2013-11-20 03:26:29,754::task::495::TaskManager.Task::(_debug) Task 7fca5584-106f-4e96-88a2-40eb33c92871: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914224::DEBUG::2013-11-20 03:26:29,754::task::495::TaskManager.Task::(_debug) Task 7fca5584-106f-4e96-88a2-40eb33c92871: moving from state preparing -> state finished
>Thread-15914224::DEBUG::2013-11-20 03:26:29,755::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914224::DEBUG::2013-11-20 03:26:29,756::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914224::DEBUG::2013-11-20 03:26:29,757::task::495::TaskManager.Task::(_debug) Task 7fca5584-106f-4e96-88a2-40eb33c92871: ref 0 aborting False
>Thread-15914224::INFO::2013-11-20 03:26:29,758::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914225::DEBUG::2013-11-20 03:26:29,769::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914225::INFO::2013-11-20 03:26:29,771::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914225::DEBUG::2013-11-20 03:26:29,771::task::495::TaskManager.Task::(_debug) Task f64894ca-4c8d-44ab-a26e-d92ff0559cbf: moving from state init -> state preparing
>Thread-15914225::DEBUG::2013-11-20 03:26:29,786::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`aa957722-7762-490b-9d24-eb270280794e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914225::DEBUG::2013-11-20 03:26:29,787::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914225::DEBUG::2013-11-20 03:26:29,789::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914225::DEBUG::2013-11-20 03:26:29,790::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`aa957722-7762-490b-9d24-eb270280794e`::Granted request
>Thread-15914225::DEBUG::2013-11-20 03:26:29,792::task::495::TaskManager.Task::(_debug) Task f64894ca-4c8d-44ab-a26e-d92ff0559cbf: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914225::DEBUG::2013-11-20 03:26:29,793::task::495::TaskManager.Task::(_debug) Task f64894ca-4c8d-44ab-a26e-d92ff0559cbf: ref 1 aborting False
>Thread-15914225::WARNING::2013-11-20 03:26:29,796::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914225::DEBUG::2013-11-20 03:26:29,797::task::495::TaskManager.Task::(_debug) Task f64894ca-4c8d-44ab-a26e-d92ff0559cbf: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914225::DEBUG::2013-11-20 03:26:29,798::task::495::TaskManager.Task::(_debug) Task f64894ca-4c8d-44ab-a26e-d92ff0559cbf: moving from state preparing -> state finished
>Thread-15914225::DEBUG::2013-11-20 03:26:29,799::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914225::DEBUG::2013-11-20 03:26:29,800::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914225::DEBUG::2013-11-20 03:26:29,801::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914225::DEBUG::2013-11-20 03:26:29,802::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914225::DEBUG::2013-11-20 03:26:29,803::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914225::DEBUG::2013-11-20 03:26:29,804::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914225::DEBUG::2013-11-20 03:26:29,805::task::495::TaskManager.Task::(_debug) Task f64894ca-4c8d-44ab-a26e-d92ff0559cbf: ref 0 aborting False
>Thread-15914225::INFO::2013-11-20 03:26:29,806::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:26:29,911::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.126573 s, 8.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:26:31,926::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:26:32,465::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0780003 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:26:34,476::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:26:35,009::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0721548 s, 14.2 MB/s\n'; <rc> = 0
>Thread-15914228::INFO::2013-11-20 03:26:36,148::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914228::DEBUG::2013-11-20 03:26:36,149::task::495::TaskManager.Task::(_debug) Task 957f2ca4-e131-49a6-b47c-fc578b734fdf: moving from state init -> state preparing
>Thread-15914228::DEBUG::2013-11-20 03:26:36,150::task::495::TaskManager.Task::(_debug) Task 957f2ca4-e131-49a6-b47c-fc578b734fdf: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000535011291504', 'lastCheck': 1384917992.1937079, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00150990486145', 'lastCheck': 1384917988.524219, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.010822057724', 'lastCheck': 1384917991.059525, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000527858734131', 'lastCheck': 1384917991.545583, 'valid': True, 'code': 0}}
>Thread-15914228::DEBUG::2013-11-20 03:26:36,150::task::495::TaskManager.Task::(_debug) Task 957f2ca4-e131-49a6-b47c-fc578b734fdf: moving from state preparing -> state finished
>Thread-15914228::DEBUG::2013-11-20 03:26:36,150::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914228::DEBUG::2013-11-20 03:26:36,151::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914228::DEBUG::2013-11-20 03:26:36,151::task::495::TaskManager.Task::(_debug) Task 957f2ca4-e131-49a6-b47c-fc578b734fdf: ref 0 aborting False
>Thread-15914228::INFO::2013-11-20 03:26:36,152::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000535011291504', 'lastCheck': 1384917992.1937079, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00150990486145', 'lastCheck': 1384917988.524219, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.010822057724', 'lastCheck': 1384917991.059525, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000527858734131', 'lastCheck': 1384917991.545583, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:26:37,025::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:26:37,565::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0675388 s, 15.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:26:39,582::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:26:40,120::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0913017 s, 11.2 MB/s\n'; <rc> = 0
>Thread-15914231::DEBUG::2013-11-20 03:26:41,164::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914231::INFO::2013-11-20 03:26:41,166::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914231::DEBUG::2013-11-20 03:26:41,166::task::495::TaskManager.Task::(_debug) Task e681d02a-4a4b-4eb0-8ff8-074c3bef4d87: moving from state init -> state preparing
>Thread-15914231::DEBUG::2013-11-20 03:26:41,169::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914231::DEBUG::2013-11-20 03:26:41,170::task::495::TaskManager.Task::(_debug) Task e681d02a-4a4b-4eb0-8ff8-074c3bef4d87: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914231::DEBUG::2013-11-20 03:26:41,174::task::495::TaskManager.Task::(_debug) Task e681d02a-4a4b-4eb0-8ff8-074c3bef4d87: moving from state preparing -> state finished
>Thread-15914231::DEBUG::2013-11-20 03:26:41,178::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914231::DEBUG::2013-11-20 03:26:41,181::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914231::DEBUG::2013-11-20 03:26:41,186::task::495::TaskManager.Task::(_debug) Task e681d02a-4a4b-4eb0-8ff8-074c3bef4d87: ref 0 aborting False
>Thread-15914231::INFO::2013-11-20 03:26:41,191::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914232::DEBUG::2013-11-20 03:26:41,211::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914232::INFO::2013-11-20 03:26:41,214::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914232::DEBUG::2013-11-20 03:26:41,216::task::495::TaskManager.Task::(_debug) Task 024d8dea-83da-4b5b-a500-bc5b5500d73b: moving from state init -> state preparing
>Thread-15914232::DEBUG::2013-11-20 03:26:41,217::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8a40209f-2430-469c-9d4b-bef172fb4f18`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914232::DEBUG::2013-11-20 03:26:41,218::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914232::DEBUG::2013-11-20 03:26:41,218::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914232::DEBUG::2013-11-20 03:26:41,219::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8a40209f-2430-469c-9d4b-bef172fb4f18`::Granted request
>Thread-15914232::DEBUG::2013-11-20 03:26:41,224::task::495::TaskManager.Task::(_debug) Task 024d8dea-83da-4b5b-a500-bc5b5500d73b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914232::DEBUG::2013-11-20 03:26:41,225::task::495::TaskManager.Task::(_debug) Task 024d8dea-83da-4b5b-a500-bc5b5500d73b: ref 1 aborting False
>Thread-15914232::WARNING::2013-11-20 03:26:41,227::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914232::DEBUG::2013-11-20 03:26:41,232::task::495::TaskManager.Task::(_debug) Task 024d8dea-83da-4b5b-a500-bc5b5500d73b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914232::DEBUG::2013-11-20 03:26:41,233::task::495::TaskManager.Task::(_debug) Task 024d8dea-83da-4b5b-a500-bc5b5500d73b: moving from state preparing -> state finished
>Thread-15914232::DEBUG::2013-11-20 03:26:41,234::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914232::DEBUG::2013-11-20 03:26:41,235::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914232::DEBUG::2013-11-20 03:26:41,235::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914232::DEBUG::2013-11-20 03:26:41,236::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914232::DEBUG::2013-11-20 03:26:41,237::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914232::DEBUG::2013-11-20 03:26:41,237::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914232::DEBUG::2013-11-20 03:26:41,238::task::495::TaskManager.Task::(_debug) Task 024d8dea-83da-4b5b-a500-bc5b5500d73b: ref 0 aborting False
>Thread-15914232::INFO::2013-11-20 03:26:41,239::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:26:42,135::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:26:42,653::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0908515 s, 11.3 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 03:26:43,808::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:26:43,809::task::495::TaskManager.Task::(_debug) Task 355f2756-a364-4b7e-b569-fda450e96f5d: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:26:43,811::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a1c73c2f-3309-4249-b719-297544b533ce`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:26:43,811::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:26:43,812::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:26:43,813::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a1c73c2f-3309-4249-b719-297544b533ce`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:26:43,814::task::495::TaskManager.Task::(_debug) Task 355f2756-a364-4b7e-b569-fda450e96f5d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:26:43,815::task::495::TaskManager.Task::(_debug) Task 355f2756-a364-4b7e-b569-fda450e96f5d: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:26:43,815::task::495::TaskManager.Task::(_debug) Task 355f2756-a364-4b7e-b569-fda450e96f5d: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:26:43,816::task::495::TaskManager.Task::(_debug) Task 355f2756-a364-4b7e-b569-fda450e96f5d: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:26:43,817::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:26:43,817::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:26:43,818::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:26:43,819::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:26:43,819::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:26:43,820::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:26:43,820::task::495::TaskManager.Task::(_debug) Task 355f2756-a364-4b7e-b569-fda450e96f5d: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:26:43,821::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:26:44,668::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:26:45,106::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0746779 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 03:26:46,327::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:26:46,328::task::495::TaskManager.Task::(_debug) Task 99602d19-04eb-49ee-b4e6-30612cd50599: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:26:46,330::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2b571e01-0007-42db-ad88-cd23b47d32ef`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:26:46,330::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:26:46,331::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:26:46,332::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2b571e01-0007-42db-ad88-cd23b47d32ef`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:26:46,333::task::495::TaskManager.Task::(_debug) Task 99602d19-04eb-49ee-b4e6-30612cd50599: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:26:46,333::task::495::TaskManager.Task::(_debug) Task 99602d19-04eb-49ee-b4e6-30612cd50599: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:26:46,334::task::495::TaskManager.Task::(_debug) Task 99602d19-04eb-49ee-b4e6-30612cd50599: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:26:46,335::task::495::TaskManager.Task::(_debug) Task 99602d19-04eb-49ee-b4e6-30612cd50599: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:26:46,335::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:26:46,336::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:26:46,336::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:26:46,337::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:26:46,338::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:26:46,338::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:26:46,339::task::495::TaskManager.Task::(_debug) Task 99602d19-04eb-49ee-b4e6-30612cd50599: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:26:46,339::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:26:47,122::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:26:47,616::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0856837 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:26:49,632::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:26:50,206::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0815278 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 03:26:51,302::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:26:51,303::task::495::TaskManager.Task::(_debug) Task 8ce32823-bfe2-48ba-a6de-29d01b811f96: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:26:51,305::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`918d91b4-e576-44bc-9750-fb9a50706dfa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:26:51,306::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:26:51,307::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:26:51,308::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`918d91b4-e576-44bc-9750-fb9a50706dfa`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:26:51,309::task::495::TaskManager.Task::(_debug) Task 8ce32823-bfe2-48ba-a6de-29d01b811f96: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:26:51,310::task::495::TaskManager.Task::(_debug) Task 8ce32823-bfe2-48ba-a6de-29d01b811f96: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:26:51,311::task::495::TaskManager.Task::(_debug) Task 8ce32823-bfe2-48ba-a6de-29d01b811f96: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:26:51,312::task::495::TaskManager.Task::(_debug) Task 8ce32823-bfe2-48ba-a6de-29d01b811f96: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:26:51,313::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:26:51,313::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:26:51,314::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:26:51,315::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:26:51,316::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:26:51,316::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:26:51,317::task::495::TaskManager.Task::(_debug) Task 8ce32823-bfe2-48ba-a6de-29d01b811f96: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:26:51,318::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914236::DEBUG::2013-11-20 03:26:51,439::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914236::INFO::2013-11-20 03:26:51,440::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914236::DEBUG::2013-11-20 03:26:51,441::task::495::TaskManager.Task::(_debug) Task 88cfff6f-fe18-4812-90a7-cdd4ba7dd695: moving from state init -> state preparing
>Thread-15914236::DEBUG::2013-11-20 03:26:51,442::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914236::DEBUG::2013-11-20 03:26:51,442::task::495::TaskManager.Task::(_debug) Task 88cfff6f-fe18-4812-90a7-cdd4ba7dd695: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914236::DEBUG::2013-11-20 03:26:51,443::task::495::TaskManager.Task::(_debug) Task 88cfff6f-fe18-4812-90a7-cdd4ba7dd695: moving from state preparing -> state finished
>Thread-15914236::DEBUG::2013-11-20 03:26:51,443::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914236::DEBUG::2013-11-20 03:26:51,444::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914236::DEBUG::2013-11-20 03:26:51,445::task::495::TaskManager.Task::(_debug) Task 88cfff6f-fe18-4812-90a7-cdd4ba7dd695: ref 0 aborting False
>Thread-15914236::INFO::2013-11-20 03:26:51,445::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914237::DEBUG::2013-11-20 03:26:51,456::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914237::INFO::2013-11-20 03:26:51,457::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914237::DEBUG::2013-11-20 03:26:51,458::task::495::TaskManager.Task::(_debug) Task 26fab461-7df8-438a-a9d0-19c5416cd5de: moving from state init -> state preparing
>Thread-15914237::DEBUG::2013-11-20 03:26:51,466::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`74b6d504-21ea-4c53-9dc7-ba49ad5dd3d2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914237::DEBUG::2013-11-20 03:26:51,467::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914237::DEBUG::2013-11-20 03:26:51,468::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914237::DEBUG::2013-11-20 03:26:51,468::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`74b6d504-21ea-4c53-9dc7-ba49ad5dd3d2`::Granted request
>Thread-15914237::DEBUG::2013-11-20 03:26:51,471::task::495::TaskManager.Task::(_debug) Task 26fab461-7df8-438a-a9d0-19c5416cd5de: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914237::DEBUG::2013-11-20 03:26:51,472::task::495::TaskManager.Task::(_debug) Task 26fab461-7df8-438a-a9d0-19c5416cd5de: ref 1 aborting False
>Thread-15914237::WARNING::2013-11-20 03:26:51,474::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914237::DEBUG::2013-11-20 03:26:51,475::task::495::TaskManager.Task::(_debug) Task 26fab461-7df8-438a-a9d0-19c5416cd5de: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914237::DEBUG::2013-11-20 03:26:51,476::task::495::TaskManager.Task::(_debug) Task 26fab461-7df8-438a-a9d0-19c5416cd5de: moving from state preparing -> state finished
>Thread-15914237::DEBUG::2013-11-20 03:26:51,476::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914237::DEBUG::2013-11-20 03:26:51,477::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914237::DEBUG::2013-11-20 03:26:51,477::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914237::DEBUG::2013-11-20 03:26:51,478::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914237::DEBUG::2013-11-20 03:26:51,478::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914237::DEBUG::2013-11-20 03:26:51,479::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914237::DEBUG::2013-11-20 03:26:51,480::task::495::TaskManager.Task::(_debug) Task 26fab461-7df8-438a-a9d0-19c5416cd5de: ref 0 aborting False
>Thread-15914237::INFO::2013-11-20 03:26:51,480::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914238::INFO::2013-11-20 03:26:51,615::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914238::DEBUG::2013-11-20 03:26:51,616::task::495::TaskManager.Task::(_debug) Task b82fc932-a9d2-43ba-9cf1-e1e5b6ef19a0: moving from state init -> state preparing
>Thread-15914238::DEBUG::2013-11-20 03:26:51,616::task::495::TaskManager.Task::(_debug) Task b82fc932-a9d2-43ba-9cf1-e1e5b6ef19a0: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000475883483887', 'lastCheck': 1384918002.3308311, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0128321647644', 'lastCheck': 1384918008.8547599, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00153589248657', 'lastCheck': 1384918011.3427219, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000518798828125', 'lastCheck': 1384918001.7824221, 'valid': True, 'code': 0}}
>Thread-15914238::DEBUG::2013-11-20 03:26:51,617::task::495::TaskManager.Task::(_debug) Task b82fc932-a9d2-43ba-9cf1-e1e5b6ef19a0: moving from state preparing -> state finished
>Thread-15914238::DEBUG::2013-11-20 03:26:51,617::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914238::DEBUG::2013-11-20 03:26:51,617::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914238::DEBUG::2013-11-20 03:26:51,618::task::495::TaskManager.Task::(_debug) Task b82fc932-a9d2-43ba-9cf1-e1e5b6ef19a0: ref 0 aborting False
>Thread-15914238::INFO::2013-11-20 03:26:51,618::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000475883483887', 'lastCheck': 1384918002.3308311, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0128321647644', 'lastCheck': 1384918008.8547599, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00153589248657', 'lastCheck': 1384918011.3427219, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000518798828125', 'lastCheck': 1384918001.7824221, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:26:52,220::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-42838::DEBUG::2013-11-20 03:26:52,409::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:26:52,690::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0513323 s, 19.9 MB/s\n'; <rc> = 0
>Thread-42838::DEBUG::2013-11-20 03:26:53,111::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:26:54,701::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:26:55,158::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0664979 s, 15.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:26:57,170::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:26:57,591::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0520295 s, 19.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:26:59,601::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:27:00,062::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0594011 s, 17.2 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 03:27:01,490::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:27:01,490::task::495::TaskManager.Task::(_debug) Task 38347dbb-cf0a-4d46-8d03-f3f7ef0a7cb6: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:27:01,492::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`25bcf7f0-5eeb-4e2c-860c-2d700a7dab53`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:27:01,493::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:27:01,494::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:27:01,494::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`25bcf7f0-5eeb-4e2c-860c-2d700a7dab53`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:27:01,495::task::495::TaskManager.Task::(_debug) Task 38347dbb-cf0a-4d46-8d03-f3f7ef0a7cb6: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:27:01,496::task::495::TaskManager.Task::(_debug) Task 38347dbb-cf0a-4d46-8d03-f3f7ef0a7cb6: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:27:01,496::task::495::TaskManager.Task::(_debug) Task 38347dbb-cf0a-4d46-8d03-f3f7ef0a7cb6: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:27:01,497::task::495::TaskManager.Task::(_debug) Task 38347dbb-cf0a-4d46-8d03-f3f7ef0a7cb6: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:27:01,498::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:27:01,498::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:27:01,499::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:27:01,499::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:27:01,500::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:27:01,500::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:27:01,501::task::495::TaskManager.Task::(_debug) Task 38347dbb-cf0a-4d46-8d03-f3f7ef0a7cb6: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:27:01,502::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:27:01,503::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:27:01,503::task::495::TaskManager.Task::(_debug) Task cdb4dc89-d5ed-460b-8fbe-10b1402606c3: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:27:01,504::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9e5f3493-77cf-40b3-9ab2-3df39bd295c1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:27:01,505::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:27:01,505::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:27:01,506::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9e5f3493-77cf-40b3-9ab2-3df39bd295c1`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:27:01,507::task::495::TaskManager.Task::(_debug) Task cdb4dc89-d5ed-460b-8fbe-10b1402606c3: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:27:01,508::task::495::TaskManager.Task::(_debug) Task cdb4dc89-d5ed-460b-8fbe-10b1402606c3: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:27:01,508::task::495::TaskManager.Task::(_debug) Task cdb4dc89-d5ed-460b-8fbe-10b1402606c3: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:27:01,509::task::495::TaskManager.Task::(_debug) Task cdb4dc89-d5ed-460b-8fbe-10b1402606c3: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:27:01,509::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:27:01,510::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:27:01,511::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:27:01,511::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:27:01,512::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:27:01,512::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:27:01,513::task::495::TaskManager.Task::(_debug) Task cdb4dc89-d5ed-460b-8fbe-10b1402606c3: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:27:01,514::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15914243::DEBUG::2013-11-20 03:27:01,976::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914243::INFO::2013-11-20 03:27:01,977::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914243::DEBUG::2013-11-20 03:27:01,978::task::495::TaskManager.Task::(_debug) Task e5bd68d3-d462-4ea4-b0e2-242a551def1c: moving from state init -> state preparing
>Thread-15914243::DEBUG::2013-11-20 03:27:01,979::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914243::DEBUG::2013-11-20 03:27:01,979::task::495::TaskManager.Task::(_debug) Task e5bd68d3-d462-4ea4-b0e2-242a551def1c: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914243::DEBUG::2013-11-20 03:27:01,980::task::495::TaskManager.Task::(_debug) Task e5bd68d3-d462-4ea4-b0e2-242a551def1c: moving from state preparing -> state finished
>Thread-15914243::DEBUG::2013-11-20 03:27:01,980::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914243::DEBUG::2013-11-20 03:27:01,981::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914243::DEBUG::2013-11-20 03:27:01,981::task::495::TaskManager.Task::(_debug) Task e5bd68d3-d462-4ea4-b0e2-242a551def1c: ref 0 aborting False
>Thread-15914243::INFO::2013-11-20 03:27:01,982::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914244::DEBUG::2013-11-20 03:27:01,986::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914244::INFO::2013-11-20 03:27:01,987::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914244::DEBUG::2013-11-20 03:27:01,987::task::495::TaskManager.Task::(_debug) Task ca8f4ae7-4e18-4e74-8111-77bfbfb16821: moving from state init -> state preparing
>Thread-15914244::DEBUG::2013-11-20 03:27:01,994::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3215e5f8-5a9e-4f52-8927-5422be031a6e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914244::DEBUG::2013-11-20 03:27:01,995::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914244::DEBUG::2013-11-20 03:27:02,001::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914244::DEBUG::2013-11-20 03:27:02,002::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3215e5f8-5a9e-4f52-8927-5422be031a6e`::Granted request
>Thread-15914244::DEBUG::2013-11-20 03:27:02,003::task::495::TaskManager.Task::(_debug) Task ca8f4ae7-4e18-4e74-8111-77bfbfb16821: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914244::DEBUG::2013-11-20 03:27:02,003::task::495::TaskManager.Task::(_debug) Task ca8f4ae7-4e18-4e74-8111-77bfbfb16821: ref 1 aborting False
>Thread-15914244::WARNING::2013-11-20 03:27:02,005::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914244::DEBUG::2013-11-20 03:27:02,006::task::495::TaskManager.Task::(_debug) Task ca8f4ae7-4e18-4e74-8111-77bfbfb16821: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914244::DEBUG::2013-11-20 03:27:02,007::task::495::TaskManager.Task::(_debug) Task ca8f4ae7-4e18-4e74-8111-77bfbfb16821: moving from state preparing -> state finished
>Thread-15914244::DEBUG::2013-11-20 03:27:02,007::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914244::DEBUG::2013-11-20 03:27:02,008::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914244::DEBUG::2013-11-20 03:27:02,008::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914244::DEBUG::2013-11-20 03:27:02,009::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914244::DEBUG::2013-11-20 03:27:02,009::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914244::DEBUG::2013-11-20 03:27:02,010::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914244::DEBUG::2013-11-20 03:27:02,011::task::495::TaskManager.Task::(_debug) Task ca8f4ae7-4e18-4e74-8111-77bfbfb16821: ref 0 aborting False
>Thread-15914244::INFO::2013-11-20 03:27:02,011::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:27:02,075::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:27:02,551::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0448105 s, 22.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:27:04,563::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:27:04,986::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0513742 s, 19.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:27:06,997::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914246::INFO::2013-11-20 03:27:07,364::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914246::DEBUG::2013-11-20 03:27:07,366::task::495::TaskManager.Task::(_debug) Task 77a76421-c0ae-4093-a54a-c33b39c257be: moving from state init -> state preparing
>Thread-15914246::DEBUG::2013-11-20 03:27:07,367::task::495::TaskManager.Task::(_debug) Task 77a76421-c0ae-4093-a54a-c33b39c257be: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000359058380127', 'lastCheck': 1384918023.2802761, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0606088638306', 'lastCheck': 1384918018.953573, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00896286964417', 'lastCheck': 1384918021.430521, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00142407417297', 'lastCheck': 1384918021.883749, 'valid': True, 'code': 0}}
>Thread-15914246::DEBUG::2013-11-20 03:27:07,376::task::495::TaskManager.Task::(_debug) Task 77a76421-c0ae-4093-a54a-c33b39c257be: moving from state preparing -> state finished
>Thread-15914246::DEBUG::2013-11-20 03:27:07,377::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914246::DEBUG::2013-11-20 03:27:07,377::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914246::DEBUG::2013-11-20 03:27:07,378::task::495::TaskManager.Task::(_debug) Task 77a76421-c0ae-4093-a54a-c33b39c257be: ref 0 aborting False
>Thread-15914246::INFO::2013-11-20 03:27:07,379::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000359058380127', 'lastCheck': 1384918023.2802761, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0606088638306', 'lastCheck': 1384918018.953573, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00896286964417', 'lastCheck': 1384918021.430521, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00142407417297', 'lastCheck': 1384918021.883749, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:27:07,545::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0971099 s, 10.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:27:09,560::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:27:10,028::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.056394 s, 18.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:27:12,042::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914249::DEBUG::2013-11-20 03:27:12,247::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914249::INFO::2013-11-20 03:27:12,250::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914249::DEBUG::2013-11-20 03:27:12,250::task::495::TaskManager.Task::(_debug) Task 3d322897-ea93-4f51-a12f-b245126a9470: moving from state init -> state preparing
>Thread-15914249::DEBUG::2013-11-20 03:27:12,251::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914249::DEBUG::2013-11-20 03:27:12,252::task::495::TaskManager.Task::(_debug) Task 3d322897-ea93-4f51-a12f-b245126a9470: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914249::DEBUG::2013-11-20 03:27:12,253::task::495::TaskManager.Task::(_debug) Task 3d322897-ea93-4f51-a12f-b245126a9470: moving from state preparing -> state finished
>Thread-15914249::DEBUG::2013-11-20 03:27:12,253::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914249::DEBUG::2013-11-20 03:27:12,254::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914249::DEBUG::2013-11-20 03:27:12,254::task::495::TaskManager.Task::(_debug) Task 3d322897-ea93-4f51-a12f-b245126a9470: ref 0 aborting False
>Thread-15914249::INFO::2013-11-20 03:27:12,255::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914250::DEBUG::2013-11-20 03:27:12,260::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914250::INFO::2013-11-20 03:27:12,261::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914250::DEBUG::2013-11-20 03:27:12,262::task::495::TaskManager.Task::(_debug) Task 43475944-106b-4498-a140-91c466473cdf: moving from state init -> state preparing
>Thread-15914250::DEBUG::2013-11-20 03:27:12,264::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`70c71b13-2992-4339-84c9-62cdf97ddfa7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914250::DEBUG::2013-11-20 03:27:12,265::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914250::DEBUG::2013-11-20 03:27:12,266::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914250::DEBUG::2013-11-20 03:27:12,266::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`70c71b13-2992-4339-84c9-62cdf97ddfa7`::Granted request
>Thread-15914250::DEBUG::2013-11-20 03:27:12,271::task::495::TaskManager.Task::(_debug) Task 43475944-106b-4498-a140-91c466473cdf: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914250::DEBUG::2013-11-20 03:27:12,272::task::495::TaskManager.Task::(_debug) Task 43475944-106b-4498-a140-91c466473cdf: ref 1 aborting False
>Thread-15914250::WARNING::2013-11-20 03:27:12,274::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914250::DEBUG::2013-11-20 03:27:12,280::task::495::TaskManager.Task::(_debug) Task 43475944-106b-4498-a140-91c466473cdf: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914250::DEBUG::2013-11-20 03:27:12,281::task::495::TaskManager.Task::(_debug) Task 43475944-106b-4498-a140-91c466473cdf: moving from state preparing -> state finished
>Thread-15914250::DEBUG::2013-11-20 03:27:12,281::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914250::DEBUG::2013-11-20 03:27:12,282::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914250::DEBUG::2013-11-20 03:27:12,283::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914250::DEBUG::2013-11-20 03:27:12,283::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914250::DEBUG::2013-11-20 03:27:12,284::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914250::DEBUG::2013-11-20 03:27:12,284::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914250::DEBUG::2013-11-20 03:27:12,285::task::495::TaskManager.Task::(_debug) Task 43475944-106b-4498-a140-91c466473cdf: ref 0 aborting False
>Thread-15914250::INFO::2013-11-20 03:27:12,285::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:27:12,496::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0626639 s, 16.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:27:14,507::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:27:14,988::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0745252 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 03:27:16,455::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:27:16,465::task::495::TaskManager.Task::(_debug) Task dea10702-dda1-4473-b405-9dd99bac5ddc: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:27:16,466::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fe8ee39e-6d07-41c5-bc17-4c1bbcdf51b4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:27:16,467::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:27:16,469::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:27:16,469::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fe8ee39e-6d07-41c5-bc17-4c1bbcdf51b4`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:27:16,470::task::495::TaskManager.Task::(_debug) Task dea10702-dda1-4473-b405-9dd99bac5ddc: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:27:16,471::task::495::TaskManager.Task::(_debug) Task dea10702-dda1-4473-b405-9dd99bac5ddc: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:27:16,471::task::495::TaskManager.Task::(_debug) Task dea10702-dda1-4473-b405-9dd99bac5ddc: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:27:16,471::task::495::TaskManager.Task::(_debug) Task dea10702-dda1-4473-b405-9dd99bac5ddc: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:27:16,472::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:27:16,472::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:27:16,473::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:27:16,473::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:27:16,474::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:27:16,474::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:27:16,474::task::495::TaskManager.Task::(_debug) Task dea10702-dda1-4473-b405-9dd99bac5ddc: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:27:16,475::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:27:16,475::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:27:16,476::task::495::TaskManager.Task::(_debug) Task 0d9128e3-f300-4ae1-a96e-af0aa4e4d74c: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:27:16,476::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2ece3f99-8ecd-4fee-a227-c607ef20d352`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:27:16,477::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:27:16,477::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:27:16,478::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2ece3f99-8ecd-4fee-a227-c607ef20d352`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:27:16,478::task::495::TaskManager.Task::(_debug) Task 0d9128e3-f300-4ae1-a96e-af0aa4e4d74c: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:27:16,479::task::495::TaskManager.Task::(_debug) Task 0d9128e3-f300-4ae1-a96e-af0aa4e4d74c: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:27:16,479::task::495::TaskManager.Task::(_debug) Task 0d9128e3-f300-4ae1-a96e-af0aa4e4d74c: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:27:16,479::task::495::TaskManager.Task::(_debug) Task 0d9128e3-f300-4ae1-a96e-af0aa4e4d74c: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:27:16,480::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:27:16,480::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:27:16,480::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:27:16,481::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:27:16,481::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:27:16,481::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:27:16,482::task::495::TaskManager.Task::(_debug) Task 0d9128e3-f300-4ae1-a96e-af0aa4e4d74c: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:27:16,482::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:27:16,483::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:27:16,483::task::495::TaskManager.Task::(_debug) Task d07e0ee8-4b51-4b9c-8bae-cffe3abd7f5a: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:27:16,484::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d4f85e67-c76b-4ef4-afd3-4755178f0878`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:27:16,484::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:27:16,485::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:27:16,485::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d4f85e67-c76b-4ef4-afd3-4755178f0878`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:27:16,486::task::495::TaskManager.Task::(_debug) Task d07e0ee8-4b51-4b9c-8bae-cffe3abd7f5a: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:27:16,487::task::495::TaskManager.Task::(_debug) Task d07e0ee8-4b51-4b9c-8bae-cffe3abd7f5a: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:27:16,488::task::495::TaskManager.Task::(_debug) Task d07e0ee8-4b51-4b9c-8bae-cffe3abd7f5a: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:27:16,488::task::495::TaskManager.Task::(_debug) Task d07e0ee8-4b51-4b9c-8bae-cffe3abd7f5a: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:27:16,489::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:27:16,489::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:27:16,490::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:27:16,491::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:27:16,491::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:27:16,492::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:27:16,492::task::495::TaskManager.Task::(_debug) Task d07e0ee8-4b51-4b9c-8bae-cffe3abd7f5a: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:27:16,493::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:27:16,494::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:27:16,494::task::495::TaskManager.Task::(_debug) Task 8b8f8245-7f5a-4490-8387-0711478cc6e1: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:27:16,496::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`009288a2-4dd3-46d8-aa85-ea5eaebac1c9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:27:16,496::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:27:16,497::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:27:16,497::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`009288a2-4dd3-46d8-aa85-ea5eaebac1c9`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:27:16,498::task::495::TaskManager.Task::(_debug) Task 8b8f8245-7f5a-4490-8387-0711478cc6e1: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:27:16,499::task::495::TaskManager.Task::(_debug) Task 8b8f8245-7f5a-4490-8387-0711478cc6e1: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:27:16,499::task::495::TaskManager.Task::(_debug) Task 8b8f8245-7f5a-4490-8387-0711478cc6e1: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:27:16,499::task::495::TaskManager.Task::(_debug) Task 8b8f8245-7f5a-4490-8387-0711478cc6e1: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:27:16,500::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:27:16,500::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:27:16,500::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:27:16,501::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:27:16,501::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:27:16,501::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:27:16,502::task::495::TaskManager.Task::(_debug) Task 8b8f8245-7f5a-4490-8387-0711478cc6e1: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:27:16,502::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:27:17,003::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:27:17,524::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0823539 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:27:19,540::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:27:19,999::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0463075 s, 22.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:27:22,011::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:27:22,417::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0510312 s, 20.1 MB/s\n'; <rc> = 0
>Thread-15914254::DEBUG::2013-11-20 03:27:22,744::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914254::INFO::2013-11-20 03:27:22,745::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914254::DEBUG::2013-11-20 03:27:22,746::task::495::TaskManager.Task::(_debug) Task 131fa2de-7a90-4ce4-916b-418b9a711b8e: moving from state init -> state preparing
>Thread-15914254::DEBUG::2013-11-20 03:27:22,747::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914254::DEBUG::2013-11-20 03:27:22,748::task::495::TaskManager.Task::(_debug) Task 131fa2de-7a90-4ce4-916b-418b9a711b8e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914254::DEBUG::2013-11-20 03:27:22,748::task::495::TaskManager.Task::(_debug) Task 131fa2de-7a90-4ce4-916b-418b9a711b8e: moving from state preparing -> state finished
>Thread-15914254::DEBUG::2013-11-20 03:27:22,749::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914254::DEBUG::2013-11-20 03:27:22,750::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914254::DEBUG::2013-11-20 03:27:22,750::task::495::TaskManager.Task::(_debug) Task 131fa2de-7a90-4ce4-916b-418b9a711b8e: ref 0 aborting False
>Thread-15914254::INFO::2013-11-20 03:27:22,751::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914255::DEBUG::2013-11-20 03:27:22,772::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914255::INFO::2013-11-20 03:27:22,773::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914255::DEBUG::2013-11-20 03:27:22,774::task::495::TaskManager.Task::(_debug) Task 03dcaf63-079e-4eb4-8dfc-53cd7a089905: moving from state init -> state preparing
>Thread-15914255::DEBUG::2013-11-20 03:27:22,785::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8527e6ff-772a-4c51-82c6-61fac0fb2fb1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914255::DEBUG::2013-11-20 03:27:22,785::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914255::DEBUG::2013-11-20 03:27:22,786::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914255::DEBUG::2013-11-20 03:27:22,786::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8527e6ff-772a-4c51-82c6-61fac0fb2fb1`::Granted request
>Thread-15914255::DEBUG::2013-11-20 03:27:22,787::task::495::TaskManager.Task::(_debug) Task 03dcaf63-079e-4eb4-8dfc-53cd7a089905: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914255::DEBUG::2013-11-20 03:27:22,788::task::495::TaskManager.Task::(_debug) Task 03dcaf63-079e-4eb4-8dfc-53cd7a089905: ref 1 aborting False
>Thread-15914255::WARNING::2013-11-20 03:27:22,789::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914255::DEBUG::2013-11-20 03:27:22,790::task::495::TaskManager.Task::(_debug) Task 03dcaf63-079e-4eb4-8dfc-53cd7a089905: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914255::DEBUG::2013-11-20 03:27:22,790::task::495::TaskManager.Task::(_debug) Task 03dcaf63-079e-4eb4-8dfc-53cd7a089905: moving from state preparing -> state finished
>Thread-15914255::DEBUG::2013-11-20 03:27:22,791::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914255::DEBUG::2013-11-20 03:27:22,791::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914255::DEBUG::2013-11-20 03:27:22,791::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914255::DEBUG::2013-11-20 03:27:22,792::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914255::DEBUG::2013-11-20 03:27:22,792::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914255::DEBUG::2013-11-20 03:27:22,793::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914255::DEBUG::2013-11-20 03:27:22,793::task::495::TaskManager.Task::(_debug) Task 03dcaf63-079e-4eb4-8dfc-53cd7a089905: ref 0 aborting False
>Thread-15914255::INFO::2013-11-20 03:27:22,793::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914256::INFO::2013-11-20 03:27:22,872::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914256::DEBUG::2013-11-20 03:27:22,873::task::495::TaskManager.Task::(_debug) Task ea9be4c0-8ee9-4420-8fa9-2e657482e03b: moving from state init -> state preparing
>Thread-15914256::DEBUG::2013-11-20 03:27:22,874::task::495::TaskManager.Task::(_debug) Task ea9be4c0-8ee9-4420-8fa9-2e657482e03b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00036096572876', 'lastCheck': 1384918033.2950549, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0151832103729', 'lastCheck': 1384918039.0007169, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00879907608032', 'lastCheck': 1384918041.4687941, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000380992889404', 'lastCheck': 1384918041.9038639, 'valid': True, 'code': 0}}
>Thread-15914256::DEBUG::2013-11-20 03:27:22,886::task::495::TaskManager.Task::(_debug) Task ea9be4c0-8ee9-4420-8fa9-2e657482e03b: moving from state preparing -> state finished
>Thread-15914256::DEBUG::2013-11-20 03:27:22,889::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914256::DEBUG::2013-11-20 03:27:22,892::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914256::DEBUG::2013-11-20 03:27:22,894::task::495::TaskManager.Task::(_debug) Task ea9be4c0-8ee9-4420-8fa9-2e657482e03b: ref 0 aborting False
>Thread-15914256::INFO::2013-11-20 03:27:22,897::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00036096572876', 'lastCheck': 1384918033.2950549, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0151832103729', 'lastCheck': 1384918039.0007169, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00879907608032', 'lastCheck': 1384918041.4687941, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000380992889404', 'lastCheck': 1384918041.9038639, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:27:24,438::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:27:24,986::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0826078 s, 12.4 MB/s\n'; <rc> = 0
>Thread-15913639::INFO::2013-11-20 03:27:26,658::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:27:26,663::task::495::TaskManager.Task::(_debug) Task a922e5e9-3ef7-406e-89ad-6e58fe94ddf9: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:27:26,665::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a14601e3-c7e1-4355-8f73-5462e75b9149`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:27:26,666::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:27:26,667::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:27:26,668::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a14601e3-c7e1-4355-8f73-5462e75b9149`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:27:26,668::task::495::TaskManager.Task::(_debug) Task a922e5e9-3ef7-406e-89ad-6e58fe94ddf9: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:27:26,669::task::495::TaskManager.Task::(_debug) Task a922e5e9-3ef7-406e-89ad-6e58fe94ddf9: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:27:26,670::task::495::TaskManager.Task::(_debug) Task a922e5e9-3ef7-406e-89ad-6e58fe94ddf9: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:27:26,670::task::495::TaskManager.Task::(_debug) Task a922e5e9-3ef7-406e-89ad-6e58fe94ddf9: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:27:26,671::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:27:26,671::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:27:26,672::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:27:26,673::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:27:26,673::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:27:26,673::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:27:26,674::task::495::TaskManager.Task::(_debug) Task a922e5e9-3ef7-406e-89ad-6e58fe94ddf9: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:27:26,675::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:27:27,000::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:27:27,450::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0496348 s, 20.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:27:29,460::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:27:29,889::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0529633 s, 19.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:27:31,898::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:27:32,291::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0477116 s, 21.5 MB/s\n'; <rc> = 0
>Thread-15914261::DEBUG::2013-11-20 03:27:33,503::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914261::INFO::2013-11-20 03:27:33,504::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914261::DEBUG::2013-11-20 03:27:33,505::task::495::TaskManager.Task::(_debug) Task 5799558c-3b9b-4f4f-bc3a-62b76600b2c9: moving from state init -> state preparing
>Thread-15914261::DEBUG::2013-11-20 03:27:33,505::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914261::DEBUG::2013-11-20 03:27:33,506::task::495::TaskManager.Task::(_debug) Task 5799558c-3b9b-4f4f-bc3a-62b76600b2c9: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914261::DEBUG::2013-11-20 03:27:33,506::task::495::TaskManager.Task::(_debug) Task 5799558c-3b9b-4f4f-bc3a-62b76600b2c9: moving from state preparing -> state finished
>Thread-15914261::DEBUG::2013-11-20 03:27:33,507::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914261::DEBUG::2013-11-20 03:27:33,507::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914261::DEBUG::2013-11-20 03:27:33,507::task::495::TaskManager.Task::(_debug) Task 5799558c-3b9b-4f4f-bc3a-62b76600b2c9: ref 0 aborting False
>Thread-15914261::INFO::2013-11-20 03:27:33,508::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914262::DEBUG::2013-11-20 03:27:33,516::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914262::INFO::2013-11-20 03:27:33,516::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914262::DEBUG::2013-11-20 03:27:33,517::task::495::TaskManager.Task::(_debug) Task 0f35ffeb-85ee-4df2-99f8-c7405ed802fc: moving from state init -> state preparing
>Thread-15914262::DEBUG::2013-11-20 03:27:33,528::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6884954d-828a-4f13-bf38-d9d19c554849`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914262::DEBUG::2013-11-20 03:27:33,529::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914262::DEBUG::2013-11-20 03:27:33,529::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914262::DEBUG::2013-11-20 03:27:33,530::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6884954d-828a-4f13-bf38-d9d19c554849`::Granted request
>Thread-15914262::DEBUG::2013-11-20 03:27:33,530::task::495::TaskManager.Task::(_debug) Task 0f35ffeb-85ee-4df2-99f8-c7405ed802fc: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914262::DEBUG::2013-11-20 03:27:33,531::task::495::TaskManager.Task::(_debug) Task 0f35ffeb-85ee-4df2-99f8-c7405ed802fc: ref 1 aborting False
>Thread-15914262::WARNING::2013-11-20 03:27:33,532::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914262::DEBUG::2013-11-20 03:27:33,538::task::495::TaskManager.Task::(_debug) Task 0f35ffeb-85ee-4df2-99f8-c7405ed802fc: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914262::DEBUG::2013-11-20 03:27:33,538::task::495::TaskManager.Task::(_debug) Task 0f35ffeb-85ee-4df2-99f8-c7405ed802fc: moving from state preparing -> state finished
>Thread-15914262::DEBUG::2013-11-20 03:27:33,539::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914262::DEBUG::2013-11-20 03:27:33,539::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914262::DEBUG::2013-11-20 03:27:33,540::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914262::DEBUG::2013-11-20 03:27:33,540::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914262::DEBUG::2013-11-20 03:27:33,541::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914262::DEBUG::2013-11-20 03:27:33,541::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914262::DEBUG::2013-11-20 03:27:33,541::task::495::TaskManager.Task::(_debug) Task 0f35ffeb-85ee-4df2-99f8-c7405ed802fc: ref 0 aborting False
>Thread-15914262::INFO::2013-11-20 03:27:33,542::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:27:34,306::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:27:34,778::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0811539 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:27:36,793::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:27:37,174::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0543953 s, 18.8 MB/s\n'; <rc> = 0
>Thread-15914264::INFO::2013-11-20 03:27:38,779::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914264::DEBUG::2013-11-20 03:27:38,779::task::495::TaskManager.Task::(_debug) Task a3b79a09-44c8-412f-b6b0-6dd05b59e383: moving from state init -> state preparing
>Thread-15914264::DEBUG::2013-11-20 03:27:38,780::task::495::TaskManager.Task::(_debug) Task a3b79a09-44c8-412f-b6b0-6dd05b59e383: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000218868255615', 'lastCheck': 1384918053.321713, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.000968933105469', 'lastCheck': 1384918049.203701, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00130891799927', 'lastCheck': 1384918051.6666851, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00103712081909', 'lastCheck': 1384918052.061547, 'valid': True, 'code': 0}}
>Thread-15914264::DEBUG::2013-11-20 03:27:38,791::task::495::TaskManager.Task::(_debug) Task a3b79a09-44c8-412f-b6b0-6dd05b59e383: moving from state preparing -> state finished
>Thread-15914264::DEBUG::2013-11-20 03:27:38,792::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914264::DEBUG::2013-11-20 03:27:38,792::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914264::DEBUG::2013-11-20 03:27:38,792::task::495::TaskManager.Task::(_debug) Task a3b79a09-44c8-412f-b6b0-6dd05b59e383: ref 0 aborting False
>Thread-15914264::INFO::2013-11-20 03:27:38,793::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000218868255615', 'lastCheck': 1384918053.321713, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.000968933105469', 'lastCheck': 1384918049.203701, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00130891799927', 'lastCheck': 1384918051.6666851, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00103712081909', 'lastCheck': 1384918052.061547, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:27:39,187::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:27:39,709::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0826891 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:27:41,722::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:27:42,241::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0848291 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15914267::DEBUG::2013-11-20 03:27:43,825::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914267::INFO::2013-11-20 03:27:43,827::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914267::DEBUG::2013-11-20 03:27:43,827::task::495::TaskManager.Task::(_debug) Task d313a45a-66a9-4b56-bfac-6b02fb5a579d: moving from state init -> state preparing
>Thread-15914267::DEBUG::2013-11-20 03:27:43,828::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914267::DEBUG::2013-11-20 03:27:43,829::task::495::TaskManager.Task::(_debug) Task d313a45a-66a9-4b56-bfac-6b02fb5a579d: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914267::DEBUG::2013-11-20 03:27:43,830::task::495::TaskManager.Task::(_debug) Task d313a45a-66a9-4b56-bfac-6b02fb5a579d: moving from state preparing -> state finished
>Thread-15914267::DEBUG::2013-11-20 03:27:43,830::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914267::DEBUG::2013-11-20 03:27:43,831::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914267::DEBUG::2013-11-20 03:27:43,831::task::495::TaskManager.Task::(_debug) Task d313a45a-66a9-4b56-bfac-6b02fb5a579d: ref 0 aborting False
>Thread-15914267::INFO::2013-11-20 03:27:43,832::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914268::DEBUG::2013-11-20 03:27:43,837::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914268::INFO::2013-11-20 03:27:43,838::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914268::DEBUG::2013-11-20 03:27:43,839::task::495::TaskManager.Task::(_debug) Task e6856ed0-dfd6-4c0c-9eac-1b5b60269736: moving from state init -> state preparing
>Thread-15914268::DEBUG::2013-11-20 03:27:43,843::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3c0cc879-868b-4ebf-b053-5c4def961b78`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914268::DEBUG::2013-11-20 03:27:43,843::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914268::DEBUG::2013-11-20 03:27:43,845::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914268::DEBUG::2013-11-20 03:27:43,845::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3c0cc879-868b-4ebf-b053-5c4def961b78`::Granted request
>Thread-15914268::DEBUG::2013-11-20 03:27:43,846::task::495::TaskManager.Task::(_debug) Task e6856ed0-dfd6-4c0c-9eac-1b5b60269736: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914268::DEBUG::2013-11-20 03:27:43,847::task::495::TaskManager.Task::(_debug) Task e6856ed0-dfd6-4c0c-9eac-1b5b60269736: ref 1 aborting False
>Thread-15914268::WARNING::2013-11-20 03:27:43,849::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914268::DEBUG::2013-11-20 03:27:43,850::task::495::TaskManager.Task::(_debug) Task e6856ed0-dfd6-4c0c-9eac-1b5b60269736: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914268::DEBUG::2013-11-20 03:27:43,851::task::495::TaskManager.Task::(_debug) Task e6856ed0-dfd6-4c0c-9eac-1b5b60269736: moving from state preparing -> state finished
>Thread-15914268::DEBUG::2013-11-20 03:27:43,852::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914268::DEBUG::2013-11-20 03:27:43,853::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914268::DEBUG::2013-11-20 03:27:43,854::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914268::DEBUG::2013-11-20 03:27:43,854::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914268::DEBUG::2013-11-20 03:27:43,855::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914268::DEBUG::2013-11-20 03:27:43,855::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914268::DEBUG::2013-11-20 03:27:43,856::task::495::TaskManager.Task::(_debug) Task e6856ed0-dfd6-4c0c-9eac-1b5b60269736: ref 0 aborting False
>Thread-15914268::INFO::2013-11-20 03:27:43,857::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:27:44,255::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 03:27:44,540::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:27:44,543::task::495::TaskManager.Task::(_debug) Task 67e7932c-8cd5-4362-91c8-9a13412b5d92: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:27:44,545::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b453272d-2bca-4e6e-8b98-9cd1587fe043`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:27:44,545::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:27:44,546::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:27:44,546::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b453272d-2bca-4e6e-8b98-9cd1587fe043`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:27:44,547::task::495::TaskManager.Task::(_debug) Task 67e7932c-8cd5-4362-91c8-9a13412b5d92: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:27:44,548::task::495::TaskManager.Task::(_debug) Task 67e7932c-8cd5-4362-91c8-9a13412b5d92: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:27:44,548::task::495::TaskManager.Task::(_debug) Task 67e7932c-8cd5-4362-91c8-9a13412b5d92: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:27:44,549::task::495::TaskManager.Task::(_debug) Task 67e7932c-8cd5-4362-91c8-9a13412b5d92: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:27:44,549::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:27:44,549::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:27:44,550::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:27:44,550::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:27:44,551::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:27:44,551::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:27:44,551::task::495::TaskManager.Task::(_debug) Task 67e7932c-8cd5-4362-91c8-9a13412b5d92: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:27:44,552::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:27:44,819::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0845021 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 03:27:46,697::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:27:46,698::task::495::TaskManager.Task::(_debug) Task 31f53d8b-5e27-4086-9853-0847883d59a6: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:27:46,706::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f71a0cb3-133e-483e-816e-de44f34ac8da`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:27:46,707::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:27:46,708::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:27:46,709::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f71a0cb3-133e-483e-816e-de44f34ac8da`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:27:46,710::task::495::TaskManager.Task::(_debug) Task 31f53d8b-5e27-4086-9853-0847883d59a6: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:27:46,711::task::495::TaskManager.Task::(_debug) Task 31f53d8b-5e27-4086-9853-0847883d59a6: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:27:46,712::task::495::TaskManager.Task::(_debug) Task 31f53d8b-5e27-4086-9853-0847883d59a6: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:27:46,712::task::495::TaskManager.Task::(_debug) Task 31f53d8b-5e27-4086-9853-0847883d59a6: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:27:46,713::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:27:46,713::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:27:46,714::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:27:46,715::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:27:46,715::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:27:46,715::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:27:46,715::task::495::TaskManager.Task::(_debug) Task 31f53d8b-5e27-4086-9853-0847883d59a6: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:27:46,716::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:27:46,834::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:27:47,230::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0499246 s, 20.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:27:49,241::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:27:49,753::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0775638 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 03:27:51,442::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:27:51,443::task::495::TaskManager.Task::(_debug) Task a49a3a13-d0cf-488e-a28b-0e8baaa6544d: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:27:51,445::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`08e3f4bd-3609-4279-9122-2404a084e552`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:27:51,445::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:27:51,446::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:27:51,446::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`08e3f4bd-3609-4279-9122-2404a084e552`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:27:51,447::task::495::TaskManager.Task::(_debug) Task a49a3a13-d0cf-488e-a28b-0e8baaa6544d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:27:51,448::task::495::TaskManager.Task::(_debug) Task a49a3a13-d0cf-488e-a28b-0e8baaa6544d: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:27:51,448::task::495::TaskManager.Task::(_debug) Task a49a3a13-d0cf-488e-a28b-0e8baaa6544d: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:27:51,449::task::495::TaskManager.Task::(_debug) Task a49a3a13-d0cf-488e-a28b-0e8baaa6544d: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:27:51,449::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:27:51,450::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:27:51,450::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:27:51,451::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:27:51,451::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:27:51,452::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:27:51,452::task::495::TaskManager.Task::(_debug) Task a49a3a13-d0cf-488e-a28b-0e8baaa6544d: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:27:51,453::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:27:51,769::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:27:52,188::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0576808 s, 17.8 MB/s\n'; <rc> = 0
>Thread-15914272::DEBUG::2013-11-20 03:27:54,195::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914272::INFO::2013-11-20 03:27:54,195::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914272::DEBUG::2013-11-20 03:27:54,196::task::495::TaskManager.Task::(_debug) Task e1eeac32-e072-4b96-9b67-f780259b3d8f: moving from state init -> state preparing
>Thread-15914272::DEBUG::2013-11-20 03:27:54,196::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914272::DEBUG::2013-11-20 03:27:54,197::task::495::TaskManager.Task::(_debug) Task e1eeac32-e072-4b96-9b67-f780259b3d8f: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914272::DEBUG::2013-11-20 03:27:54,197::task::495::TaskManager.Task::(_debug) Task e1eeac32-e072-4b96-9b67-f780259b3d8f: moving from state preparing -> state finished
>Thread-15914272::DEBUG::2013-11-20 03:27:54,197::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914272::DEBUG::2013-11-20 03:27:54,198::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914272::DEBUG::2013-11-20 03:27:54,198::task::495::TaskManager.Task::(_debug) Task e1eeac32-e072-4b96-9b67-f780259b3d8f: ref 0 aborting False
>Thread-15914272::INFO::2013-11-20 03:27:54,199::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Dummy-15912651::DEBUG::2013-11-20 03:27:54,203::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914274::INFO::2013-11-20 03:27:54,406::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914273::DEBUG::2013-11-20 03:27:54,408::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914274::DEBUG::2013-11-20 03:27:54,409::task::495::TaskManager.Task::(_debug) Task 9013f00e-e287-47d3-9689-2910f927bb1c: moving from state init -> state preparing
>Thread-15914273::INFO::2013-11-20 03:27:54,410::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914274::DEBUG::2013-11-20 03:27:54,412::task::495::TaskManager.Task::(_debug) Task 9013f00e-e287-47d3-9689-2910f927bb1c: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000416994094849', 'lastCheck': 1384918073.523025, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00126504898071', 'lastCheck': 1384918069.5144041, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00130677223206', 'lastCheck': 1384918071.695323, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000401973724365', 'lastCheck': 1384918072.0846519, 'valid': True, 'code': 0}}
>Thread-15914273::DEBUG::2013-11-20 03:27:54,413::task::495::TaskManager.Task::(_debug) Task d1a87c23-50dc-4f59-8024-274818e76465: moving from state init -> state preparing
>Thread-15914274::DEBUG::2013-11-20 03:27:54,413::task::495::TaskManager.Task::(_debug) Task 9013f00e-e287-47d3-9689-2910f927bb1c: moving from state preparing -> state finished
>Thread-15914273::DEBUG::2013-11-20 03:27:54,420::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`77c94afe-ded8-444b-8978-ce4985a7826a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914274::DEBUG::2013-11-20 03:27:54,421::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914273::DEBUG::2013-11-20 03:27:54,421::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914274::DEBUG::2013-11-20 03:27:54,422::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914274::DEBUG::2013-11-20 03:27:54,424::task::495::TaskManager.Task::(_debug) Task 9013f00e-e287-47d3-9689-2910f927bb1c: ref 0 aborting False
>Thread-15914273::DEBUG::2013-11-20 03:27:54,425::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914274::INFO::2013-11-20 03:27:54,425::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000416994094849', 'lastCheck': 1384918073.523025, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00126504898071', 'lastCheck': 1384918069.5144041, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00130677223206', 'lastCheck': 1384918071.695323, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000401973724365', 'lastCheck': 1384918072.0846519, 'valid': True, 'code': 0}}
>Thread-15914273::DEBUG::2013-11-20 03:27:54,426::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`77c94afe-ded8-444b-8978-ce4985a7826a`::Granted request
>Thread-15914273::DEBUG::2013-11-20 03:27:54,428::task::495::TaskManager.Task::(_debug) Task d1a87c23-50dc-4f59-8024-274818e76465: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914273::DEBUG::2013-11-20 03:27:54,429::task::495::TaskManager.Task::(_debug) Task d1a87c23-50dc-4f59-8024-274818e76465: ref 1 aborting False
>Thread-15914273::WARNING::2013-11-20 03:27:54,432::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914273::DEBUG::2013-11-20 03:27:54,433::task::495::TaskManager.Task::(_debug) Task d1a87c23-50dc-4f59-8024-274818e76465: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914273::DEBUG::2013-11-20 03:27:54,439::task::495::TaskManager.Task::(_debug) Task d1a87c23-50dc-4f59-8024-274818e76465: moving from state preparing -> state finished
>Thread-15914273::DEBUG::2013-11-20 03:27:54,440::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914273::DEBUG::2013-11-20 03:27:54,440::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914273::DEBUG::2013-11-20 03:27:54,443::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914273::DEBUG::2013-11-20 03:27:54,444::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914273::DEBUG::2013-11-20 03:27:54,444::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914273::DEBUG::2013-11-20 03:27:54,445::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914273::DEBUG::2013-11-20 03:27:54,446::task::495::TaskManager.Task::(_debug) Task d1a87c23-50dc-4f59-8024-274818e76465: ref 0 aborting False
>Thread-15914273::INFO::2013-11-20 03:27:54,447::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:27:54,689::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0443722 s, 23.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:27:56,700::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:27:57,216::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.074889 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:27:59,231::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:27:59,768::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0755837 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:28:01,784::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:28:02,238::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0504062 s, 20.3 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 03:28:02,482::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:28:02,483::task::495::TaskManager.Task::(_debug) Task e5c716bb-32d9-43b2-ab14-89f460aed84d: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:28:02,484::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2504678f-ec5a-4e73-9183-8f718dcdf0de`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:28:02,484::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:28:02,485::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:28:02,485::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2504678f-ec5a-4e73-9183-8f718dcdf0de`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:28:02,486::task::495::TaskManager.Task::(_debug) Task e5c716bb-32d9-43b2-ab14-89f460aed84d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:28:02,486::task::495::TaskManager.Task::(_debug) Task e5c716bb-32d9-43b2-ab14-89f460aed84d: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:28:02,487::task::495::TaskManager.Task::(_debug) Task e5c716bb-32d9-43b2-ab14-89f460aed84d: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:28:02,488::task::495::TaskManager.Task::(_debug) Task e5c716bb-32d9-43b2-ab14-89f460aed84d: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:28:02,488::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:28:02,489::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:28:02,490::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:28:02,490::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:28:02,491::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:28:02,491::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:28:02,492::task::495::TaskManager.Task::(_debug) Task e5c716bb-32d9-43b2-ab14-89f460aed84d: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:28:02,493::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:28:02,493::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:28:02,493::task::495::TaskManager.Task::(_debug) Task 19ed75ba-1d75-4e0e-a65f-985af1099396: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:28:02,495::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`dc780351-988a-4dd5-874f-0c3c2577beff`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:28:02,495::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:28:02,495::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:28:02,496::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`dc780351-988a-4dd5-874f-0c3c2577beff`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:28:02,496::task::495::TaskManager.Task::(_debug) Task 19ed75ba-1d75-4e0e-a65f-985af1099396: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:28:02,497::task::495::TaskManager.Task::(_debug) Task 19ed75ba-1d75-4e0e-a65f-985af1099396: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:28:02,497::task::495::TaskManager.Task::(_debug) Task 19ed75ba-1d75-4e0e-a65f-985af1099396: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:28:02,498::task::495::TaskManager.Task::(_debug) Task 19ed75ba-1d75-4e0e-a65f-985af1099396: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:28:02,498::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:28:02,499::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:28:02,500::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:28:02,501::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:28:02,501::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:28:02,501::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:28:02,502::task::495::TaskManager.Task::(_debug) Task 19ed75ba-1d75-4e0e-a65f-985af1099396: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:28:02,502::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:28:04,249::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:28:04,678::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0570404 s, 18.0 MB/s\n'; <rc> = 0
>Thread-15914279::DEBUG::2013-11-20 03:28:04,881::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914279::INFO::2013-11-20 03:28:04,882::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914279::DEBUG::2013-11-20 03:28:04,883::task::495::TaskManager.Task::(_debug) Task e270dff2-6f3a-4abf-9e45-2148dacdae99: moving from state init -> state preparing
>Thread-15914279::DEBUG::2013-11-20 03:28:04,883::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914279::DEBUG::2013-11-20 03:28:04,884::task::495::TaskManager.Task::(_debug) Task e270dff2-6f3a-4abf-9e45-2148dacdae99: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914279::DEBUG::2013-11-20 03:28:04,884::task::495::TaskManager.Task::(_debug) Task e270dff2-6f3a-4abf-9e45-2148dacdae99: moving from state preparing -> state finished
>Thread-15914279::DEBUG::2013-11-20 03:28:04,884::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914279::DEBUG::2013-11-20 03:28:04,885::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914279::DEBUG::2013-11-20 03:28:04,885::task::495::TaskManager.Task::(_debug) Task e270dff2-6f3a-4abf-9e45-2148dacdae99: ref 0 aborting False
>Thread-15914279::INFO::2013-11-20 03:28:04,885::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914280::DEBUG::2013-11-20 03:28:04,897::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914280::INFO::2013-11-20 03:28:04,898::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914280::DEBUG::2013-11-20 03:28:04,899::task::495::TaskManager.Task::(_debug) Task c59c0c0b-13ea-4fd7-a1ae-f32fe6776afc: moving from state init -> state preparing
>Thread-15914280::DEBUG::2013-11-20 03:28:04,900::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7e305418-a961-437b-b24d-dfb454e3221e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914280::DEBUG::2013-11-20 03:28:04,900::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914280::DEBUG::2013-11-20 03:28:04,901::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914280::DEBUG::2013-11-20 03:28:04,902::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7e305418-a961-437b-b24d-dfb454e3221e`::Granted request
>Thread-15914280::DEBUG::2013-11-20 03:28:04,903::task::495::TaskManager.Task::(_debug) Task c59c0c0b-13ea-4fd7-a1ae-f32fe6776afc: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914280::DEBUG::2013-11-20 03:28:04,903::task::495::TaskManager.Task::(_debug) Task c59c0c0b-13ea-4fd7-a1ae-f32fe6776afc: ref 1 aborting False
>Thread-15914280::WARNING::2013-11-20 03:28:04,905::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914280::DEBUG::2013-11-20 03:28:04,906::task::495::TaskManager.Task::(_debug) Task c59c0c0b-13ea-4fd7-a1ae-f32fe6776afc: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914280::DEBUG::2013-11-20 03:28:04,907::task::495::TaskManager.Task::(_debug) Task c59c0c0b-13ea-4fd7-a1ae-f32fe6776afc: moving from state preparing -> state finished
>Thread-15914280::DEBUG::2013-11-20 03:28:04,907::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914280::DEBUG::2013-11-20 03:28:04,908::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914280::DEBUG::2013-11-20 03:28:04,908::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914280::DEBUG::2013-11-20 03:28:04,909::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914280::DEBUG::2013-11-20 03:28:04,909::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914280::DEBUG::2013-11-20 03:28:04,910::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914280::DEBUG::2013-11-20 03:28:04,910::task::495::TaskManager.Task::(_debug) Task c59c0c0b-13ea-4fd7-a1ae-f32fe6776afc: ref 0 aborting False
>Thread-15914280::INFO::2013-11-20 03:28:04,911::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:28:06,694::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:28:07,215::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.073595 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:28:09,228::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:28:09,748::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0817558 s, 12.5 MB/s\n'; <rc> = 0
>Thread-15914282::INFO::2013-11-20 03:28:10,465::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914282::DEBUG::2013-11-20 03:28:10,466::task::495::TaskManager.Task::(_debug) Task bba06498-1820-48eb-be05-dbbb4f5f3813: moving from state init -> state preparing
>Thread-15914282::DEBUG::2013-11-20 03:28:10,466::task::495::TaskManager.Task::(_debug) Task bba06498-1820-48eb-be05-dbbb4f5f3813: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000304937362671', 'lastCheck': 1384918083.535651, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00163698196411', 'lastCheck': 1384918089.5788319, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00153398513794', 'lastCheck': 1384918081.7109261, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00044322013855', 'lastCheck': 1384918082.0956869, 'valid': True, 'code': 0}}
>Thread-15914282::DEBUG::2013-11-20 03:28:10,475::task::495::TaskManager.Task::(_debug) Task bba06498-1820-48eb-be05-dbbb4f5f3813: moving from state preparing -> state finished
>Thread-15914282::DEBUG::2013-11-20 03:28:10,484::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914282::DEBUG::2013-11-20 03:28:10,486::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914282::DEBUG::2013-11-20 03:28:10,486::task::495::TaskManager.Task::(_debug) Task bba06498-1820-48eb-be05-dbbb4f5f3813: ref 0 aborting False
>Thread-15914282::INFO::2013-11-20 03:28:10,488::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000304937362671', 'lastCheck': 1384918083.535651, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00163698196411', 'lastCheck': 1384918089.5788319, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00153398513794', 'lastCheck': 1384918081.7109261, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00044322013855', 'lastCheck': 1384918082.0956869, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:28:11,764::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:28:12,280::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0776105 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:28:14,295::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:28:14,726::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0481314 s, 21.3 MB/s\n'; <rc> = 0
>Thread-15914285::DEBUG::2013-11-20 03:28:15,049::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914285::INFO::2013-11-20 03:28:15,050::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914285::DEBUG::2013-11-20 03:28:15,051::task::495::TaskManager.Task::(_debug) Task e53dcd56-18e1-42bc-97ba-83ff4f928f09: moving from state init -> state preparing
>Thread-15914285::DEBUG::2013-11-20 03:28:15,052::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914285::DEBUG::2013-11-20 03:28:15,053::task::495::TaskManager.Task::(_debug) Task e53dcd56-18e1-42bc-97ba-83ff4f928f09: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914285::DEBUG::2013-11-20 03:28:15,053::task::495::TaskManager.Task::(_debug) Task e53dcd56-18e1-42bc-97ba-83ff4f928f09: moving from state preparing -> state finished
>Thread-15914285::DEBUG::2013-11-20 03:28:15,054::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914285::DEBUG::2013-11-20 03:28:15,055::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914285::DEBUG::2013-11-20 03:28:15,055::task::495::TaskManager.Task::(_debug) Task e53dcd56-18e1-42bc-97ba-83ff4f928f09: ref 0 aborting False
>Thread-15914285::INFO::2013-11-20 03:28:15,056::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914286::DEBUG::2013-11-20 03:28:15,064::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914286::INFO::2013-11-20 03:28:15,065::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914286::DEBUG::2013-11-20 03:28:15,066::task::495::TaskManager.Task::(_debug) Task 33cb7545-1572-48e7-b0c6-47e7bd55bcef: moving from state init -> state preparing
>Thread-15914286::DEBUG::2013-11-20 03:28:15,071::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`692c3f80-ada3-4fea-abdc-34074462be5d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914286::DEBUG::2013-11-20 03:28:15,071::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914286::DEBUG::2013-11-20 03:28:15,080::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914286::DEBUG::2013-11-20 03:28:15,081::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`692c3f80-ada3-4fea-abdc-34074462be5d`::Granted request
>Thread-15914286::DEBUG::2013-11-20 03:28:15,082::task::495::TaskManager.Task::(_debug) Task 33cb7545-1572-48e7-b0c6-47e7bd55bcef: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914286::DEBUG::2013-11-20 03:28:15,083::task::495::TaskManager.Task::(_debug) Task 33cb7545-1572-48e7-b0c6-47e7bd55bcef: ref 1 aborting False
>Thread-15914286::WARNING::2013-11-20 03:28:15,085::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914286::DEBUG::2013-11-20 03:28:15,093::task::495::TaskManager.Task::(_debug) Task 33cb7545-1572-48e7-b0c6-47e7bd55bcef: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914286::DEBUG::2013-11-20 03:28:15,093::task::495::TaskManager.Task::(_debug) Task 33cb7545-1572-48e7-b0c6-47e7bd55bcef: moving from state preparing -> state finished
>Thread-15914286::DEBUG::2013-11-20 03:28:15,094::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914286::DEBUG::2013-11-20 03:28:15,094::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914286::DEBUG::2013-11-20 03:28:15,094::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914286::DEBUG::2013-11-20 03:28:15,095::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914286::DEBUG::2013-11-20 03:28:15,095::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914286::DEBUG::2013-11-20 03:28:15,095::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914286::DEBUG::2013-11-20 03:28:15,096::task::495::TaskManager.Task::(_debug) Task 33cb7545-1572-48e7-b0c6-47e7bd55bcef: ref 0 aborting False
>Thread-15914286::INFO::2013-11-20 03:28:15,096::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:28:16,737::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 03:28:17,036::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:28:17,038::task::495::TaskManager.Task::(_debug) Task f47c7510-d862-45fb-8938-14ace3023757: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:28:17,040::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e5e54a71-1dae-4c0c-8d9e-63ed7c400265`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:28:17,041::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:28:17,041::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:28:17,042::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e5e54a71-1dae-4c0c-8d9e-63ed7c400265`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:28:17,042::task::495::TaskManager.Task::(_debug) Task f47c7510-d862-45fb-8938-14ace3023757: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:28:17,043::task::495::TaskManager.Task::(_debug) Task f47c7510-d862-45fb-8938-14ace3023757: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:28:17,043::task::495::TaskManager.Task::(_debug) Task f47c7510-d862-45fb-8938-14ace3023757: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:28:17,044::task::495::TaskManager.Task::(_debug) Task f47c7510-d862-45fb-8938-14ace3023757: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:28:17,044::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:28:17,045::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:28:17,045::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:28:17,046::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:28:17,046::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:28:17,046::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:28:17,047::task::495::TaskManager.Task::(_debug) Task f47c7510-d862-45fb-8938-14ace3023757: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:28:17,047::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:28:17,048::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:28:17,048::task::495::TaskManager.Task::(_debug) Task c40cdb3b-3f9e-4e31-9bbe-f7e6d0d26927: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:28:17,049::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`9ed91156-16af-477c-8b83-6bc3822d8fba`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:28:17,049::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:28:17,050::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:28:17,050::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`9ed91156-16af-477c-8b83-6bc3822d8fba`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:28:17,050::task::495::TaskManager.Task::(_debug) Task c40cdb3b-3f9e-4e31-9bbe-f7e6d0d26927: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:28:17,051::task::495::TaskManager.Task::(_debug) Task c40cdb3b-3f9e-4e31-9bbe-f7e6d0d26927: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:28:17,051::task::495::TaskManager.Task::(_debug) Task c40cdb3b-3f9e-4e31-9bbe-f7e6d0d26927: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:28:17,051::task::495::TaskManager.Task::(_debug) Task c40cdb3b-3f9e-4e31-9bbe-f7e6d0d26927: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:28:17,052::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:28:17,052::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:28:17,053::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:28:17,053::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:28:17,053::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:28:17,054::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:28:17,054::task::495::TaskManager.Task::(_debug) Task c40cdb3b-3f9e-4e31-9bbe-f7e6d0d26927: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:28:17,054::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:28:17,055::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:28:17,055::task::495::TaskManager.Task::(_debug) Task 7178c219-c10a-4faa-9a46-66dd3f988558: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:28:17,056::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f28fc690-9a2d-459b-8ae0-1ce2c1f20ca8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:28:17,056::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:28:17,057::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:28:17,057::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f28fc690-9a2d-459b-8ae0-1ce2c1f20ca8`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:28:17,058::task::495::TaskManager.Task::(_debug) Task 7178c219-c10a-4faa-9a46-66dd3f988558: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:28:17,058::task::495::TaskManager.Task::(_debug) Task 7178c219-c10a-4faa-9a46-66dd3f988558: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:28:17,058::task::495::TaskManager.Task::(_debug) Task 7178c219-c10a-4faa-9a46-66dd3f988558: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:28:17,059::task::495::TaskManager.Task::(_debug) Task 7178c219-c10a-4faa-9a46-66dd3f988558: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:28:17,059::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:28:17,060::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:28:17,060::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:28:17,060::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:28:17,061::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:28:17,061::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:28:17,061::task::495::TaskManager.Task::(_debug) Task 7178c219-c10a-4faa-9a46-66dd3f988558: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:28:17,062::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:28:17,062::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:28:17,062::task::495::TaskManager.Task::(_debug) Task 42024f08-6a09-4e14-8a9c-d2eb5308ebc5: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:28:17,063::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`5c0eba02-e66b-4108-ad7b-4428f06715a5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:28:17,063::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:28:17,064::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:28:17,064::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`5c0eba02-e66b-4108-ad7b-4428f06715a5`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:28:17,065::task::495::TaskManager.Task::(_debug) Task 42024f08-6a09-4e14-8a9c-d2eb5308ebc5: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:28:17,065::task::495::TaskManager.Task::(_debug) Task 42024f08-6a09-4e14-8a9c-d2eb5308ebc5: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:28:17,065::task::495::TaskManager.Task::(_debug) Task 42024f08-6a09-4e14-8a9c-d2eb5308ebc5: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:28:17,066::task::495::TaskManager.Task::(_debug) Task 42024f08-6a09-4e14-8a9c-d2eb5308ebc5: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:28:17,067::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:28:17,067::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:28:17,067::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:28:17,068::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:28:17,069::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:28:17,070::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:28:17,070::task::495::TaskManager.Task::(_debug) Task 42024f08-6a09-4e14-8a9c-d2eb5308ebc5: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:28:17,071::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:28:17,274::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0841064 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:28:19,288::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:28:19,792::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0680418 s, 15.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:28:21,805::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:28:22,311::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0737721 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:28:24,327::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:28:24,847::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0847635 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15914290::DEBUG::2013-11-20 03:28:25,212::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914290::INFO::2013-11-20 03:28:25,213::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914290::DEBUG::2013-11-20 03:28:25,214::task::495::TaskManager.Task::(_debug) Task 5ecedf84-a6d8-4fa2-8af2-c59736e253b1: moving from state init -> state preparing
>Thread-15914290::DEBUG::2013-11-20 03:28:25,215::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914290::DEBUG::2013-11-20 03:28:25,216::task::495::TaskManager.Task::(_debug) Task 5ecedf84-a6d8-4fa2-8af2-c59736e253b1: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914290::DEBUG::2013-11-20 03:28:25,216::task::495::TaskManager.Task::(_debug) Task 5ecedf84-a6d8-4fa2-8af2-c59736e253b1: moving from state preparing -> state finished
>Thread-15914290::DEBUG::2013-11-20 03:28:25,217::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914290::DEBUG::2013-11-20 03:28:25,217::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914290::DEBUG::2013-11-20 03:28:25,218::task::495::TaskManager.Task::(_debug) Task 5ecedf84-a6d8-4fa2-8af2-c59736e253b1: ref 0 aborting False
>Thread-15914290::INFO::2013-11-20 03:28:25,219::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914291::DEBUG::2013-11-20 03:28:25,222::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914291::INFO::2013-11-20 03:28:25,223::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914291::DEBUG::2013-11-20 03:28:25,223::task::495::TaskManager.Task::(_debug) Task 4bf4754d-35a8-421a-9689-9688cb565c2a: moving from state init -> state preparing
>Thread-15914291::DEBUG::2013-11-20 03:28:25,224::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`06a2a1f6-1150-49af-918f-f42a007ca52f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914291::DEBUG::2013-11-20 03:28:25,225::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914291::DEBUG::2013-11-20 03:28:25,225::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914291::DEBUG::2013-11-20 03:28:25,226::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`06a2a1f6-1150-49af-918f-f42a007ca52f`::Granted request
>Thread-15914291::DEBUG::2013-11-20 03:28:25,226::task::495::TaskManager.Task::(_debug) Task 4bf4754d-35a8-421a-9689-9688cb565c2a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914291::DEBUG::2013-11-20 03:28:25,227::task::495::TaskManager.Task::(_debug) Task 4bf4754d-35a8-421a-9689-9688cb565c2a: ref 1 aborting False
>Thread-15914291::WARNING::2013-11-20 03:28:25,228::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914291::DEBUG::2013-11-20 03:28:25,229::task::495::TaskManager.Task::(_debug) Task 4bf4754d-35a8-421a-9689-9688cb565c2a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914291::DEBUG::2013-11-20 03:28:25,229::task::495::TaskManager.Task::(_debug) Task 4bf4754d-35a8-421a-9689-9688cb565c2a: moving from state preparing -> state finished
>Thread-15914291::DEBUG::2013-11-20 03:28:25,230::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914291::DEBUG::2013-11-20 03:28:25,230::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914291::DEBUG::2013-11-20 03:28:25,231::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914291::DEBUG::2013-11-20 03:28:25,231::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914291::DEBUG::2013-11-20 03:28:25,231::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914291::DEBUG::2013-11-20 03:28:25,232::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914291::DEBUG::2013-11-20 03:28:25,232::task::495::TaskManager.Task::(_debug) Task 4bf4754d-35a8-421a-9689-9688cb565c2a: ref 0 aborting False
>Thread-15914291::INFO::2013-11-20 03:28:25,232::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914292::INFO::2013-11-20 03:28:26,012::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914292::DEBUG::2013-11-20 03:28:26,013::task::495::TaskManager.Task::(_debug) Task 15e70cd1-7e94-4646-8563-0942416e9fa5: moving from state init -> state preparing
>Thread-15914292::DEBUG::2013-11-20 03:28:26,013::task::495::TaskManager.Task::(_debug) Task 15e70cd1-7e94-4646-8563-0942416e9fa5: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000411033630371', 'lastCheck': 1384918103.567255, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00165200233459', 'lastCheck': 1384918099.595237, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00134110450745', 'lastCheck': 1384918102.0013189, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000554084777832', 'lastCheck': 1384918102.115217, 'valid': True, 'code': 0}}
>Thread-15914292::DEBUG::2013-11-20 03:28:26,014::task::495::TaskManager.Task::(_debug) Task 15e70cd1-7e94-4646-8563-0942416e9fa5: moving from state preparing -> state finished
>Thread-15914292::DEBUG::2013-11-20 03:28:26,015::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914292::DEBUG::2013-11-20 03:28:26,016::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914292::DEBUG::2013-11-20 03:28:26,017::task::495::TaskManager.Task::(_debug) Task 15e70cd1-7e94-4646-8563-0942416e9fa5: ref 0 aborting False
>Thread-15914292::INFO::2013-11-20 03:28:26,017::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000411033630371', 'lastCheck': 1384918103.567255, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00165200233459', 'lastCheck': 1384918099.595237, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00134110450745', 'lastCheck': 1384918102.0013189, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000554084777832', 'lastCheck': 1384918102.115217, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:28:26,862::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 03:28:27,068::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:28:27,076::task::495::TaskManager.Task::(_debug) Task aa4c8ab0-98d1-449e-be44-d198dcc8b010: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:28:27,079::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`aa373961-c850-4a91-96d4-9aa02a0abb89`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:28:27,079::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:28:27,080::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:28:27,081::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`aa373961-c850-4a91-96d4-9aa02a0abb89`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:28:27,082::task::495::TaskManager.Task::(_debug) Task aa4c8ab0-98d1-449e-be44-d198dcc8b010: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:28:27,083::task::495::TaskManager.Task::(_debug) Task aa4c8ab0-98d1-449e-be44-d198dcc8b010: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:28:27,084::task::495::TaskManager.Task::(_debug) Task aa4c8ab0-98d1-449e-be44-d198dcc8b010: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:28:27,085::task::495::TaskManager.Task::(_debug) Task aa4c8ab0-98d1-449e-be44-d198dcc8b010: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:28:27,086::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:28:27,086::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:28:27,087::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:28:27,088::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:28:27,088::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:28:27,089::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:28:27,090::task::495::TaskManager.Task::(_debug) Task aa4c8ab0-98d1-449e-be44-d198dcc8b010: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:28:27,091::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:28:27,377::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.067159 s, 15.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:28:29,391::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:28:29,887::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0762699 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:28:31,903::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:28:32,470::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0826379 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:28:34,483::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:28:35,008::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.077081 s, 13.3 MB/s\n'; <rc> = 0
>Thread-15914297::DEBUG::2013-11-20 03:28:35,368::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914297::INFO::2013-11-20 03:28:35,370::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914297::DEBUG::2013-11-20 03:28:35,370::task::495::TaskManager.Task::(_debug) Task 6b56c5a1-b94f-48de-bcea-3067afcdde68: moving from state init -> state preparing
>Thread-15914297::DEBUG::2013-11-20 03:28:35,371::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914297::DEBUG::2013-11-20 03:28:35,372::task::495::TaskManager.Task::(_debug) Task 6b56c5a1-b94f-48de-bcea-3067afcdde68: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914297::DEBUG::2013-11-20 03:28:35,372::task::495::TaskManager.Task::(_debug) Task 6b56c5a1-b94f-48de-bcea-3067afcdde68: moving from state preparing -> state finished
>Thread-15914297::DEBUG::2013-11-20 03:28:35,373::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914297::DEBUG::2013-11-20 03:28:35,373::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914297::DEBUG::2013-11-20 03:28:35,374::task::495::TaskManager.Task::(_debug) Task 6b56c5a1-b94f-48de-bcea-3067afcdde68: ref 0 aborting False
>Thread-15914297::INFO::2013-11-20 03:28:35,374::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914298::DEBUG::2013-11-20 03:28:35,379::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914298::INFO::2013-11-20 03:28:35,380::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914298::DEBUG::2013-11-20 03:28:35,381::task::495::TaskManager.Task::(_debug) Task 9d10d974-231b-4252-95b5-feb80f655afe: moving from state init -> state preparing
>Thread-15914298::DEBUG::2013-11-20 03:28:35,382::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`49f3a758-8bc0-4be2-aaa4-2a80a2a00dcf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914298::DEBUG::2013-11-20 03:28:35,383::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914298::DEBUG::2013-11-20 03:28:35,384::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914298::DEBUG::2013-11-20 03:28:35,384::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`49f3a758-8bc0-4be2-aaa4-2a80a2a00dcf`::Granted request
>Thread-15914298::DEBUG::2013-11-20 03:28:35,385::task::495::TaskManager.Task::(_debug) Task 9d10d974-231b-4252-95b5-feb80f655afe: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914298::DEBUG::2013-11-20 03:28:35,386::task::495::TaskManager.Task::(_debug) Task 9d10d974-231b-4252-95b5-feb80f655afe: ref 1 aborting False
>Thread-15914298::WARNING::2013-11-20 03:28:35,388::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914298::DEBUG::2013-11-20 03:28:35,389::task::495::TaskManager.Task::(_debug) Task 9d10d974-231b-4252-95b5-feb80f655afe: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914298::DEBUG::2013-11-20 03:28:35,390::task::495::TaskManager.Task::(_debug) Task 9d10d974-231b-4252-95b5-feb80f655afe: moving from state preparing -> state finished
>Thread-15914298::DEBUG::2013-11-20 03:28:35,391::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914298::DEBUG::2013-11-20 03:28:35,391::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914298::DEBUG::2013-11-20 03:28:35,392::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914298::DEBUG::2013-11-20 03:28:35,393::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914298::DEBUG::2013-11-20 03:28:35,393::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914298::DEBUG::2013-11-20 03:28:35,394::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914298::DEBUG::2013-11-20 03:28:35,394::task::495::TaskManager.Task::(_debug) Task 9d10d974-231b-4252-95b5-feb80f655afe: ref 0 aborting False
>Thread-15914298::INFO::2013-11-20 03:28:35,395::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115404177408', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:28:37,023::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:28:37,538::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0780063 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:28:39,553::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:28:40,001::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0584619 s, 17.5 MB/s\n'; <rc> = 0
>Thread-15914300::INFO::2013-11-20 03:28:41,412::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914300::DEBUG::2013-11-20 03:28:41,413::task::495::TaskManager.Task::(_debug) Task 86272dbf-cf7b-4282-898f-d48a42af6929: moving from state init -> state preparing
>Thread-15914300::DEBUG::2013-11-20 03:28:41,413::task::495::TaskManager.Task::(_debug) Task 86272dbf-cf7b-4282-898f-d48a42af6929: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000328063964844', 'lastCheck': 1384918113.583462, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.139843940735', 'lastCheck': 1384918119.8902819, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00206017494202', 'lastCheck': 1384918112.1712279, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0167138576508', 'lastCheck': 1384918112.156785, 'valid': True, 'code': 0}}
>Thread-15914300::DEBUG::2013-11-20 03:28:41,426::task::495::TaskManager.Task::(_debug) Task 86272dbf-cf7b-4282-898f-d48a42af6929: moving from state preparing -> state finished
>Thread-15914300::DEBUG::2013-11-20 03:28:41,427::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914300::DEBUG::2013-11-20 03:28:41,427::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914300::DEBUG::2013-11-20 03:28:41,428::task::495::TaskManager.Task::(_debug) Task 86272dbf-cf7b-4282-898f-d48a42af6929: ref 0 aborting False
>Thread-15914300::INFO::2013-11-20 03:28:41,429::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000328063964844', 'lastCheck': 1384918113.583462, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.139843940735', 'lastCheck': 1384918119.8902819, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00206017494202', 'lastCheck': 1384918112.1712279, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0167138576508', 'lastCheck': 1384918112.156785, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:28:42,011::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:28:42,463::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.072042 s, 14.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:28:44,478::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 03:28:44,825::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:28:44,826::task::495::TaskManager.Task::(_debug) Task 377782b3-8556-4f5d-9438-6171e2c0dd58: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:28:44,827::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`947396ca-7ec7-4cb0-991a-bc80b5bf4411`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:28:44,827::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:28:44,828::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:28:44,828::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`947396ca-7ec7-4cb0-991a-bc80b5bf4411`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:28:44,829::task::495::TaskManager.Task::(_debug) Task 377782b3-8556-4f5d-9438-6171e2c0dd58: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:28:44,829::task::495::TaskManager.Task::(_debug) Task 377782b3-8556-4f5d-9438-6171e2c0dd58: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:28:44,830::task::495::TaskManager.Task::(_debug) Task 377782b3-8556-4f5d-9438-6171e2c0dd58: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:28:44,830::task::495::TaskManager.Task::(_debug) Task 377782b3-8556-4f5d-9438-6171e2c0dd58: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:28:44,831::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:28:44,831::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:28:44,831::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:28:44,832::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:28:44,832::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:28:44,832::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:28:44,833::task::495::TaskManager.Task::(_debug) Task 377782b3-8556-4f5d-9438-6171e2c0dd58: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:28:44,833::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:28:44,996::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0811123 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15914303::DEBUG::2013-11-20 03:28:46,667::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914303::INFO::2013-11-20 03:28:46,668::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914303::DEBUG::2013-11-20 03:28:46,668::task::495::TaskManager.Task::(_debug) Task 5104de9f-427e-4cc1-a9d9-b3d4bded23b4: moving from state init -> state preparing
>Thread-15914303::DEBUG::2013-11-20 03:28:46,669::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914303::DEBUG::2013-11-20 03:28:46,669::task::495::TaskManager.Task::(_debug) Task 5104de9f-427e-4cc1-a9d9-b3d4bded23b4: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914303::DEBUG::2013-11-20 03:28:46,670::task::495::TaskManager.Task::(_debug) Task 5104de9f-427e-4cc1-a9d9-b3d4bded23b4: moving from state preparing -> state finished
>Thread-15914303::DEBUG::2013-11-20 03:28:46,671::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914303::DEBUG::2013-11-20 03:28:46,671::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914303::DEBUG::2013-11-20 03:28:46,672::task::495::TaskManager.Task::(_debug) Task 5104de9f-427e-4cc1-a9d9-b3d4bded23b4: ref 0 aborting False
>Thread-15914303::INFO::2013-11-20 03:28:46,673::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914304::DEBUG::2013-11-20 03:28:46,676::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914304::INFO::2013-11-20 03:28:46,677::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914304::DEBUG::2013-11-20 03:28:46,677::task::495::TaskManager.Task::(_debug) Task 92dbe56d-09af-41cf-90e2-dfa8728e0c58: moving from state init -> state preparing
>Thread-15914304::DEBUG::2013-11-20 03:28:46,678::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f55704db-070f-4c4f-bfb8-4ecdd4508e79`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914304::DEBUG::2013-11-20 03:28:46,679::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914304::DEBUG::2013-11-20 03:28:46,679::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914304::DEBUG::2013-11-20 03:28:46,680::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f55704db-070f-4c4f-bfb8-4ecdd4508e79`::Granted request
>Thread-15914304::DEBUG::2013-11-20 03:28:46,681::task::495::TaskManager.Task::(_debug) Task 92dbe56d-09af-41cf-90e2-dfa8728e0c58: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914304::DEBUG::2013-11-20 03:28:46,682::task::495::TaskManager.Task::(_debug) Task 92dbe56d-09af-41cf-90e2-dfa8728e0c58: ref 1 aborting False
>Thread-15914304::WARNING::2013-11-20 03:28:46,683::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914304::DEBUG::2013-11-20 03:28:46,691::task::495::TaskManager.Task::(_debug) Task 92dbe56d-09af-41cf-90e2-dfa8728e0c58: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115214385152', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115214385152', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914304::DEBUG::2013-11-20 03:28:46,696::task::495::TaskManager.Task::(_debug) Task 92dbe56d-09af-41cf-90e2-dfa8728e0c58: moving from state preparing -> state finished
>Thread-15914304::DEBUG::2013-11-20 03:28:46,697::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914304::DEBUG::2013-11-20 03:28:46,698::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914304::DEBUG::2013-11-20 03:28:46,699::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914304::DEBUG::2013-11-20 03:28:46,700::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914304::DEBUG::2013-11-20 03:28:46,701::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914304::DEBUG::2013-11-20 03:28:46,701::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914304::DEBUG::2013-11-20 03:28:46,702::task::495::TaskManager.Task::(_debug) Task 92dbe56d-09af-41cf-90e2-dfa8728e0c58: ref 0 aborting False
>Thread-15914304::INFO::2013-11-20 03:28:46,703::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115214385152', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115214385152', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:28:47,011::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 03:28:47,245::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:28:47,254::task::495::TaskManager.Task::(_debug) Task 436f729b-2c91-43e7-9b0e-555656dd9d7d: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:28:47,256::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9f41fc22-685a-4c04-b6e8-f82a60be91f6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:28:47,257::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:28:47,257::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:28:47,258::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9f41fc22-685a-4c04-b6e8-f82a60be91f6`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:28:47,258::task::495::TaskManager.Task::(_debug) Task 436f729b-2c91-43e7-9b0e-555656dd9d7d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:28:47,259::task::495::TaskManager.Task::(_debug) Task 436f729b-2c91-43e7-9b0e-555656dd9d7d: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:28:47,259::task::495::TaskManager.Task::(_debug) Task 436f729b-2c91-43e7-9b0e-555656dd9d7d: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:28:47,260::task::495::TaskManager.Task::(_debug) Task 436f729b-2c91-43e7-9b0e-555656dd9d7d: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:28:47,260::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:28:47,260::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:28:47,261::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:28:47,261::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:28:47,261::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:28:47,262::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:28:47,262::task::495::TaskManager.Task::(_debug) Task 436f729b-2c91-43e7-9b0e-555656dd9d7d: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:28:47,263::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:28:47,454::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0551589 s, 18.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:28:49,464::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:28:49,997::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0838671 s, 12.2 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 03:28:51,752::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:28:51,753::task::495::TaskManager.Task::(_debug) Task ac0cede7-ef04-4cc4-a6e1-ac76f2644265: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:28:51,755::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`53f835cc-9c4a-4dbf-b455-72a5a5981ca7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:28:51,756::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:28:51,757::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:28:51,757::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`53f835cc-9c4a-4dbf-b455-72a5a5981ca7`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:28:51,758::task::495::TaskManager.Task::(_debug) Task ac0cede7-ef04-4cc4-a6e1-ac76f2644265: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:28:51,759::task::495::TaskManager.Task::(_debug) Task ac0cede7-ef04-4cc4-a6e1-ac76f2644265: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:28:51,760::task::495::TaskManager.Task::(_debug) Task ac0cede7-ef04-4cc4-a6e1-ac76f2644265: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:28:51,760::task::495::TaskManager.Task::(_debug) Task ac0cede7-ef04-4cc4-a6e1-ac76f2644265: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:28:51,761::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:28:51,762::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:28:51,762::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:28:51,763::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:28:51,763::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:28:51,764::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:28:51,765::task::495::TaskManager.Task::(_debug) Task ac0cede7-ef04-4cc4-a6e1-ac76f2644265: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:28:51,765::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:28:52,012::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:28:52,568::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0752094 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:28:54,583::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:28:55,144::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0757156 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15914308::DEBUG::2013-11-20 03:28:57,077::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914308::INFO::2013-11-20 03:28:57,078::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914308::DEBUG::2013-11-20 03:28:57,078::task::495::TaskManager.Task::(_debug) Task 4f8b3d2f-0a97-4897-9d7a-1d2b300fa468: moving from state init -> state preparing
>Thread-15914308::DEBUG::2013-11-20 03:28:57,079::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914308::DEBUG::2013-11-20 03:28:57,079::task::495::TaskManager.Task::(_debug) Task 4f8b3d2f-0a97-4897-9d7a-1d2b300fa468: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914308::DEBUG::2013-11-20 03:28:57,080::task::495::TaskManager.Task::(_debug) Task 4f8b3d2f-0a97-4897-9d7a-1d2b300fa468: moving from state preparing -> state finished
>Thread-15914308::DEBUG::2013-11-20 03:28:57,080::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914308::DEBUG::2013-11-20 03:28:57,081::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914308::DEBUG::2013-11-20 03:28:57,082::task::495::TaskManager.Task::(_debug) Task 4f8b3d2f-0a97-4897-9d7a-1d2b300fa468: ref 0 aborting False
>Thread-15914308::INFO::2013-11-20 03:28:57,082::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914309::DEBUG::2013-11-20 03:28:57,095::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914309::INFO::2013-11-20 03:28:57,097::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914309::DEBUG::2013-11-20 03:28:57,097::task::495::TaskManager.Task::(_debug) Task 2c244ace-bd66-46b2-87ff-9a409a53d7b8: moving from state init -> state preparing
>Thread-15914309::DEBUG::2013-11-20 03:28:57,099::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d8974292-3246-459a-b5b5-8c73704ecb6f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914309::DEBUG::2013-11-20 03:28:57,099::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914309::DEBUG::2013-11-20 03:28:57,100::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914309::DEBUG::2013-11-20 03:28:57,101::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d8974292-3246-459a-b5b5-8c73704ecb6f`::Granted request
>Thread-15914309::DEBUG::2013-11-20 03:28:57,107::task::495::TaskManager.Task::(_debug) Task 2c244ace-bd66-46b2-87ff-9a409a53d7b8: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914309::DEBUG::2013-11-20 03:28:57,107::task::495::TaskManager.Task::(_debug) Task 2c244ace-bd66-46b2-87ff-9a409a53d7b8: ref 1 aborting False
>Thread-15914309::WARNING::2013-11-20 03:28:57,109::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914309::DEBUG::2013-11-20 03:28:57,110::task::495::TaskManager.Task::(_debug) Task 2c244ace-bd66-46b2-87ff-9a409a53d7b8: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115214385152', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115214385152', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914309::DEBUG::2013-11-20 03:28:57,111::task::495::TaskManager.Task::(_debug) Task 2c244ace-bd66-46b2-87ff-9a409a53d7b8: moving from state preparing -> state finished
>Thread-15914309::DEBUG::2013-11-20 03:28:57,112::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914309::DEBUG::2013-11-20 03:28:57,113::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914309::DEBUG::2013-11-20 03:28:57,119::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914309::DEBUG::2013-11-20 03:28:57,120::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914309::DEBUG::2013-11-20 03:28:57,120::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914309::DEBUG::2013-11-20 03:28:57,121::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914309::DEBUG::2013-11-20 03:28:57,121::task::495::TaskManager.Task::(_debug) Task 2c244ace-bd66-46b2-87ff-9a409a53d7b8: ref 0 aborting False
>Thread-15914309::INFO::2013-11-20 03:28:57,122::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115214385152', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115214385152', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914310::INFO::2013-11-20 03:28:57,152::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914310::DEBUG::2013-11-20 03:28:57,153::task::495::TaskManager.Task::(_debug) Task 4f928763-d719-46a4-b86a-9d3e433205e1: moving from state init -> state preparing
>Thread-15914310::DEBUG::2013-11-20 03:28:57,153::task::495::TaskManager.Task::(_debug) Task 4f928763-d719-46a4-b86a-9d3e433205e1: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000302791595459', 'lastCheck': 1384918133.767503, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00150799751282', 'lastCheck': 1384918129.9062841, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00165009498596', 'lastCheck': 1384918132.2686639, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000535011291504', 'lastCheck': 1384918132.2409589, 'valid': True, 'code': 0}}
>Thread-15914310::DEBUG::2013-11-20 03:28:57,157::task::495::TaskManager.Task::(_debug) Task 4f928763-d719-46a4-b86a-9d3e433205e1: moving from state preparing -> state finished
>Thread-15914310::DEBUG::2013-11-20 03:28:57,161::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Dummy-15912651::DEBUG::2013-11-20 03:28:57,161::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914310::DEBUG::2013-11-20 03:28:57,161::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914310::DEBUG::2013-11-20 03:28:57,162::task::495::TaskManager.Task::(_debug) Task 4f928763-d719-46a4-b86a-9d3e433205e1: ref 0 aborting False
>Thread-15914310::INFO::2013-11-20 03:28:57,345::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000302791595459', 'lastCheck': 1384918133.767503, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00150799751282', 'lastCheck': 1384918129.9062841, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00165009498596', 'lastCheck': 1384918132.2686639, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000535011291504', 'lastCheck': 1384918132.2409589, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:28:57,632::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0767575 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:28:59,647::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:29:00,225::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.076334 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:29:02,240::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:29:02,825::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0842621 s, 12.2 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 03:29:03,078::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:29:03,079::task::495::TaskManager.Task::(_debug) Task d05ebdf6-c6a6-44fd-8431-9b87e87f77c8: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:29:03,081::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6ebdc1ef-c234-4fcf-8086-9f07eae56d93`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:29:03,082::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:29:03,083::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:29:03,083::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6ebdc1ef-c234-4fcf-8086-9f07eae56d93`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:29:03,085::task::495::TaskManager.Task::(_debug) Task d05ebdf6-c6a6-44fd-8431-9b87e87f77c8: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:29:03,085::task::495::TaskManager.Task::(_debug) Task d05ebdf6-c6a6-44fd-8431-9b87e87f77c8: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:29:03,086::task::495::TaskManager.Task::(_debug) Task d05ebdf6-c6a6-44fd-8431-9b87e87f77c8: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:29:03,087::task::495::TaskManager.Task::(_debug) Task d05ebdf6-c6a6-44fd-8431-9b87e87f77c8: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:29:03,088::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:29:03,088::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:29:03,089::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:29:03,090::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:29:03,091::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:29:03,091::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:29:03,092::task::495::TaskManager.Task::(_debug) Task d05ebdf6-c6a6-44fd-8431-9b87e87f77c8: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:29:03,093::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:29:03,094::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:29:03,095::task::495::TaskManager.Task::(_debug) Task 00f023f3-0236-4df3-978f-c1f714afc2d1: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:29:03,096::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`143c4c0f-9750-4a00-8678-6b77ce0a0a82`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:29:03,097::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:29:03,098::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:29:03,098::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`143c4c0f-9750-4a00-8678-6b77ce0a0a82`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:29:03,099::task::495::TaskManager.Task::(_debug) Task 00f023f3-0236-4df3-978f-c1f714afc2d1: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:29:03,100::task::495::TaskManager.Task::(_debug) Task 00f023f3-0236-4df3-978f-c1f714afc2d1: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:29:03,101::task::495::TaskManager.Task::(_debug) Task 00f023f3-0236-4df3-978f-c1f714afc2d1: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:29:03,101::task::495::TaskManager.Task::(_debug) Task 00f023f3-0236-4df3-978f-c1f714afc2d1: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:29:03,102::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:29:03,103::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:29:03,104::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:29:03,104::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:29:03,105::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:29:03,105::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:29:03,106::task::495::TaskManager.Task::(_debug) Task 00f023f3-0236-4df3-978f-c1f714afc2d1: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:29:03,107::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:29:04,841::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:29:05,386::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0804396 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:29:07,400::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:29:07,939::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0754296 s, 13.6 MB/s\n'; <rc> = 0
>Thread-15914315::DEBUG::2013-11-20 03:29:08,075::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914315::INFO::2013-11-20 03:29:08,077::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914315::DEBUG::2013-11-20 03:29:08,077::task::495::TaskManager.Task::(_debug) Task 6989ad9f-fd18-4177-b5ea-b25b8fe6fd8f: moving from state init -> state preparing
>Thread-15914315::DEBUG::2013-11-20 03:29:08,078::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914315::DEBUG::2013-11-20 03:29:08,079::task::495::TaskManager.Task::(_debug) Task 6989ad9f-fd18-4177-b5ea-b25b8fe6fd8f: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914315::DEBUG::2013-11-20 03:29:08,080::task::495::TaskManager.Task::(_debug) Task 6989ad9f-fd18-4177-b5ea-b25b8fe6fd8f: moving from state preparing -> state finished
>Thread-15914315::DEBUG::2013-11-20 03:29:08,080::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914315::DEBUG::2013-11-20 03:29:08,081::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914315::DEBUG::2013-11-20 03:29:08,081::task::495::TaskManager.Task::(_debug) Task 6989ad9f-fd18-4177-b5ea-b25b8fe6fd8f: ref 0 aborting False
>Thread-15914315::INFO::2013-11-20 03:29:08,082::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914316::DEBUG::2013-11-20 03:29:08,092::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914316::INFO::2013-11-20 03:29:08,093::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914316::DEBUG::2013-11-20 03:29:08,093::task::495::TaskManager.Task::(_debug) Task 30e7c5a0-701e-4546-8b2a-cf6dc5e07b79: moving from state init -> state preparing
>Thread-15914316::DEBUG::2013-11-20 03:29:08,095::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ae6af5fc-6171-47bd-bfbf-86d5d10abe07`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914316::DEBUG::2013-11-20 03:29:08,095::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914316::DEBUG::2013-11-20 03:29:08,096::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914316::DEBUG::2013-11-20 03:29:08,097::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ae6af5fc-6171-47bd-bfbf-86d5d10abe07`::Granted request
>Thread-15914316::DEBUG::2013-11-20 03:29:08,098::task::495::TaskManager.Task::(_debug) Task 30e7c5a0-701e-4546-8b2a-cf6dc5e07b79: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914316::DEBUG::2013-11-20 03:29:08,099::task::495::TaskManager.Task::(_debug) Task 30e7c5a0-701e-4546-8b2a-cf6dc5e07b79: ref 1 aborting False
>Thread-15914316::WARNING::2013-11-20 03:29:08,101::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914316::DEBUG::2013-11-20 03:29:08,102::task::495::TaskManager.Task::(_debug) Task 30e7c5a0-701e-4546-8b2a-cf6dc5e07b79: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115214385152', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115214385152', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914316::DEBUG::2013-11-20 03:29:08,103::task::495::TaskManager.Task::(_debug) Task 30e7c5a0-701e-4546-8b2a-cf6dc5e07b79: moving from state preparing -> state finished
>Thread-15914316::DEBUG::2013-11-20 03:29:08,104::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914316::DEBUG::2013-11-20 03:29:08,104::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914316::DEBUG::2013-11-20 03:29:08,105::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914316::DEBUG::2013-11-20 03:29:08,106::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914316::DEBUG::2013-11-20 03:29:08,106::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914316::DEBUG::2013-11-20 03:29:08,107::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914316::DEBUG::2013-11-20 03:29:08,108::task::495::TaskManager.Task::(_debug) Task 30e7c5a0-701e-4546-8b2a-cf6dc5e07b79: ref 0 aborting False
>Thread-15914316::INFO::2013-11-20 03:29:08,108::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115214385152', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115214385152', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:29:09,954::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:29:10,490::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0887183 s, 11.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:29:12,506::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:29:13,085::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0863412 s, 11.9 MB/s\n'; <rc> = 0
>Thread-15914318::INFO::2013-11-20 03:29:13,200::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914318::DEBUG::2013-11-20 03:29:13,201::task::495::TaskManager.Task::(_debug) Task 807f4938-c7f8-44c3-9ce4-b58f2d861d93: moving from state init -> state preparing
>Thread-15914318::DEBUG::2013-11-20 03:29:13,202::task::495::TaskManager.Task::(_debug) Task 807f4938-c7f8-44c3-9ce4-b58f2d861d93: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000249862670898', 'lastCheck': 1384918143.8883941, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0373349189758', 'lastCheck': 1384918150.1951139, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00139307975769', 'lastCheck': 1384918152.735827, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00135612487793', 'lastCheck': 1384918152.7317009, 'valid': True, 'code': 0}}
>Thread-15914318::DEBUG::2013-11-20 03:29:13,203::task::495::TaskManager.Task::(_debug) Task 807f4938-c7f8-44c3-9ce4-b58f2d861d93: moving from state preparing -> state finished
>Thread-15914318::DEBUG::2013-11-20 03:29:13,203::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914318::DEBUG::2013-11-20 03:29:13,204::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914318::DEBUG::2013-11-20 03:29:13,204::task::495::TaskManager.Task::(_debug) Task 807f4938-c7f8-44c3-9ce4-b58f2d861d93: ref 0 aborting False
>Thread-15914318::INFO::2013-11-20 03:29:13,205::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000249862670898', 'lastCheck': 1384918143.8883941, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0373349189758', 'lastCheck': 1384918150.1951139, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00139307975769', 'lastCheck': 1384918152.735827, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00135612487793', 'lastCheck': 1384918152.7317009, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:29:15,098::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:29:15,673::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0734134 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:29:17,688::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 03:29:17,909::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:29:17,921::task::495::TaskManager.Task::(_debug) Task 0d1e3cb2-4ca1-48d3-87ab-b252de93e553: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:29:17,944::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`179cfacf-6192-448c-92b4-a45ddc510d2c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:29:17,949::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:29:17,954::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:29:17,955::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`179cfacf-6192-448c-92b4-a45ddc510d2c`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:29:17,957::task::495::TaskManager.Task::(_debug) Task 0d1e3cb2-4ca1-48d3-87ab-b252de93e553: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:29:17,958::task::495::TaskManager.Task::(_debug) Task 0d1e3cb2-4ca1-48d3-87ab-b252de93e553: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:29:17,959::task::495::TaskManager.Task::(_debug) Task 0d1e3cb2-4ca1-48d3-87ab-b252de93e553: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:29:17,959::task::495::TaskManager.Task::(_debug) Task 0d1e3cb2-4ca1-48d3-87ab-b252de93e553: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:29:17,960::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:29:17,960::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:29:17,961::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:29:17,962::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:29:17,962::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:29:17,963::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:29:17,964::task::495::TaskManager.Task::(_debug) Task 0d1e3cb2-4ca1-48d3-87ab-b252de93e553: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:29:17,965::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:29:17,965::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:29:17,966::task::495::TaskManager.Task::(_debug) Task c4669175-95e7-42cf-9499-3dcb0b2bcdf1: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:29:17,967::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a6f9970f-694e-4848-b0a4-bc0e45820642`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:29:17,968::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:29:17,968::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:29:17,969::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a6f9970f-694e-4848-b0a4-bc0e45820642`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:29:17,970::task::495::TaskManager.Task::(_debug) Task c4669175-95e7-42cf-9499-3dcb0b2bcdf1: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:29:17,970::task::495::TaskManager.Task::(_debug) Task c4669175-95e7-42cf-9499-3dcb0b2bcdf1: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:29:17,971::task::495::TaskManager.Task::(_debug) Task c4669175-95e7-42cf-9499-3dcb0b2bcdf1: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:29:17,971::task::495::TaskManager.Task::(_debug) Task c4669175-95e7-42cf-9499-3dcb0b2bcdf1: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:29:17,972::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:29:17,972::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:29:17,973::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:29:17,974::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:29:17,974::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:29:17,975::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:29:17,975::task::495::TaskManager.Task::(_debug) Task c4669175-95e7-42cf-9499-3dcb0b2bcdf1: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:29:17,976::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:29:17,977::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:29:17,977::task::495::TaskManager.Task::(_debug) Task 8d61623d-f2c3-478a-a92d-bb394a0206f0: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:29:17,978::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fe048a2f-7777-48e1-8b01-0f7566cf7146`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:29:17,979::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:29:17,979::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:29:17,980::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fe048a2f-7777-48e1-8b01-0f7566cf7146`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:29:17,981::task::495::TaskManager.Task::(_debug) Task 8d61623d-f2c3-478a-a92d-bb394a0206f0: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:29:17,981::task::495::TaskManager.Task::(_debug) Task 8d61623d-f2c3-478a-a92d-bb394a0206f0: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:29:17,982::task::495::TaskManager.Task::(_debug) Task 8d61623d-f2c3-478a-a92d-bb394a0206f0: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:29:17,982::task::495::TaskManager.Task::(_debug) Task 8d61623d-f2c3-478a-a92d-bb394a0206f0: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:29:17,983::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:29:17,984::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:29:17,984::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:29:17,985::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:29:17,985::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:29:17,986::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:29:17,986::task::495::TaskManager.Task::(_debug) Task 8d61623d-f2c3-478a-a92d-bb394a0206f0: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:29:17,987::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:29:17,988::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:29:17,988::task::495::TaskManager.Task::(_debug) Task 58674057-362b-4ef6-961a-bc396892fb6e: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:29:17,989::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`dcfedc2a-ef31-47f3-a3a6-a6e5254113e9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:29:17,990::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:29:17,990::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:29:17,991::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`dcfedc2a-ef31-47f3-a3a6-a6e5254113e9`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:29:17,992::task::495::TaskManager.Task::(_debug) Task 58674057-362b-4ef6-961a-bc396892fb6e: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:29:17,992::task::495::TaskManager.Task::(_debug) Task 58674057-362b-4ef6-961a-bc396892fb6e: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:29:17,993::task::495::TaskManager.Task::(_debug) Task 58674057-362b-4ef6-961a-bc396892fb6e: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:29:17,993::task::495::TaskManager.Task::(_debug) Task 58674057-362b-4ef6-961a-bc396892fb6e: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:29:17,994::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:29:17,994::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:29:17,995::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:29:17,996::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:29:17,996::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:29:17,997::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:29:17,997::task::495::TaskManager.Task::(_debug) Task 58674057-362b-4ef6-961a-bc396892fb6e: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:29:17,998::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:29:18,191::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0862049 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:29:20,206::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:29:20,690::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0716282 s, 14.3 MB/s\n'; <rc> = 0
>Thread-15914322::DEBUG::2013-11-20 03:29:21,201::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914322::INFO::2013-11-20 03:29:21,202::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914322::DEBUG::2013-11-20 03:29:21,203::task::495::TaskManager.Task::(_debug) Task 2aee0790-8f50-4b48-838f-2b5026c05f51: moving from state init -> state preparing
>Thread-15914322::DEBUG::2013-11-20 03:29:21,204::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914322::DEBUG::2013-11-20 03:29:21,204::task::495::TaskManager.Task::(_debug) Task 2aee0790-8f50-4b48-838f-2b5026c05f51: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914322::DEBUG::2013-11-20 03:29:21,205::task::495::TaskManager.Task::(_debug) Task 2aee0790-8f50-4b48-838f-2b5026c05f51: moving from state preparing -> state finished
>Thread-15914322::DEBUG::2013-11-20 03:29:21,205::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914322::DEBUG::2013-11-20 03:29:21,206::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914322::DEBUG::2013-11-20 03:29:21,206::task::495::TaskManager.Task::(_debug) Task 2aee0790-8f50-4b48-838f-2b5026c05f51: ref 0 aborting False
>Thread-15914322::INFO::2013-11-20 03:29:21,207::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914323::DEBUG::2013-11-20 03:29:21,211::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914323::INFO::2013-11-20 03:29:21,212::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914323::DEBUG::2013-11-20 03:29:21,212::task::495::TaskManager.Task::(_debug) Task 21136643-7a33-4eea-b99d-d2c0f60e799e: moving from state init -> state preparing
>Thread-15914323::DEBUG::2013-11-20 03:29:21,213::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f4f3ca66-43de-49d9-8f4a-de9e0cbfcaf5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914323::DEBUG::2013-11-20 03:29:21,213::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914323::DEBUG::2013-11-20 03:29:21,214::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914323::DEBUG::2013-11-20 03:29:21,215::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f4f3ca66-43de-49d9-8f4a-de9e0cbfcaf5`::Granted request
>Thread-15914323::DEBUG::2013-11-20 03:29:21,219::task::495::TaskManager.Task::(_debug) Task 21136643-7a33-4eea-b99d-d2c0f60e799e: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914323::DEBUG::2013-11-20 03:29:21,220::task::495::TaskManager.Task::(_debug) Task 21136643-7a33-4eea-b99d-d2c0f60e799e: ref 1 aborting False
>Thread-15914323::WARNING::2013-11-20 03:29:21,221::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914323::DEBUG::2013-11-20 03:29:21,222::task::495::TaskManager.Task::(_debug) Task 21136643-7a33-4eea-b99d-d2c0f60e799e: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914323::DEBUG::2013-11-20 03:29:21,223::task::495::TaskManager.Task::(_debug) Task 21136643-7a33-4eea-b99d-d2c0f60e799e: moving from state preparing -> state finished
>Thread-15914323::DEBUG::2013-11-20 03:29:21,223::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914323::DEBUG::2013-11-20 03:29:21,224::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914323::DEBUG::2013-11-20 03:29:21,224::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914323::DEBUG::2013-11-20 03:29:21,225::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914323::DEBUG::2013-11-20 03:29:21,225::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914323::DEBUG::2013-11-20 03:29:21,225::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914323::DEBUG::2013-11-20 03:29:21,226::task::495::TaskManager.Task::(_debug) Task 21136643-7a33-4eea-b99d-d2c0f60e799e: ref 0 aborting False
>Thread-15914323::INFO::2013-11-20 03:29:21,226::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:29:22,706::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-42840::DEBUG::2013-11-20 03:29:22,905::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:29:23,314::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0682476 s, 15.0 MB/s\n'; <rc> = 0
>Thread-42840::DEBUG::2013-11-20 03:29:23,775::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:29:25,324::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:29:25,845::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0759301 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:29:27,860::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 03:29:28,226::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:29:28,244::task::495::TaskManager.Task::(_debug) Task 40925be5-ab53-4a27-aa66-f9e9e110ad10: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:29:28,248::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b37c09b2-563a-4e63-9958-c8de8fa34f87`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:29:28,249::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:29:28,250::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:29:28,251::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b37c09b2-563a-4e63-9958-c8de8fa34f87`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:29:28,252::task::495::TaskManager.Task::(_debug) Task 40925be5-ab53-4a27-aa66-f9e9e110ad10: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:29:28,253::task::495::TaskManager.Task::(_debug) Task 40925be5-ab53-4a27-aa66-f9e9e110ad10: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:29:28,254::task::495::TaskManager.Task::(_debug) Task 40925be5-ab53-4a27-aa66-f9e9e110ad10: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:29:28,254::task::495::TaskManager.Task::(_debug) Task 40925be5-ab53-4a27-aa66-f9e9e110ad10: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:29:28,255::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:29:28,256::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:29:28,256::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:29:28,257::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:29:28,257::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:29:28,258::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:29:28,259::task::495::TaskManager.Task::(_debug) Task 40925be5-ab53-4a27-aa66-f9e9e110ad10: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:29:28,259::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:29:28,428::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0765887 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15914326::INFO::2013-11-20 03:29:28,615::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914326::DEBUG::2013-11-20 03:29:28,616::task::495::TaskManager.Task::(_debug) Task 19cc637e-f023-43da-a799-b2499934456d: moving from state init -> state preparing
>Thread-15914326::DEBUG::2013-11-20 03:29:28,616::task::495::TaskManager.Task::(_debug) Task 19cc637e-f023-43da-a799-b2499934456d: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000354051589966', 'lastCheck': 1384918164.4194181, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00136303901672', 'lastCheck': 1384918160.420423, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00369501113892', 'lastCheck': 1384918163.219424, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000316143035889', 'lastCheck': 1384918163.7770109, 'valid': True, 'code': 0}}
>Thread-15914326::DEBUG::2013-11-20 03:29:28,627::task::495::TaskManager.Task::(_debug) Task 19cc637e-f023-43da-a799-b2499934456d: moving from state preparing -> state finished
>Thread-15914326::DEBUG::2013-11-20 03:29:28,628::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914326::DEBUG::2013-11-20 03:29:28,628::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914326::DEBUG::2013-11-20 03:29:28,629::task::495::TaskManager.Task::(_debug) Task 19cc637e-f023-43da-a799-b2499934456d: ref 0 aborting False
>Thread-15914326::INFO::2013-11-20 03:29:28,630::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000354051589966', 'lastCheck': 1384918164.4194181, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00136303901672', 'lastCheck': 1384918160.420423, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00369501113892', 'lastCheck': 1384918163.219424, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000316143035889', 'lastCheck': 1384918163.7770109, 'valid': True, 'code': 0}}
>Thread-15914328::DEBUG::2013-11-20 03:29:30,405::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914328::INFO::2013-11-20 03:29:30,406::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: updateVM, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 vmList=[{'imglist': '5af747d8-95b1-4b7e-af53-cb77e4fcc896', 'ovf': u'<?xml version=\'1.0\' encoding=\'UTF-8\'?><ovf:Envelope xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/1/" xmlns:rasd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_ResourceAllocationSettingData" xmlns:vssd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_VirtualSystemSettingData" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" ovf:version="3.2.0.0"><References><File ovf:href="5af747d8-95b1-4b7e-af53-cb77e4fcc896/8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f" ovf:id="8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f" ovf:size="85899345920" ovf:description="Active VM snapshot"/><Nic ovf:id="fd38b98a-8c16-4dae-817c-794715ec6b17"/></References><Section xsi:type="ovf:NetworkSection_Type"><Info>List of networks</Info><Network ovf:name="Network 1"/></Section><Section xsi:type="ovf:DiskSection_Type"><Info>List of Virtual Disks</Info><Disk ovf:diskId="8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f" ovf:size="80" ovf:actual_size="80" ovf:vm_snapshot_id="a8605287-4e5e-4bcc-965b-707787b502ee" ovf:parentRef="" ovf:fileRef="5af747d8-95b1-4b7e-af53-cb77e4fcc896/8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f" ovf:format="http://www.vmware.com/specifications/vmdk.html#sparse" ovf:volume-format="RAW" ovf:volume-type="Preallocated" ovf:disk-interface="IDE" ovf:boot="true" ovf:disk-alias="sukhumi_Disk0" ovf:wipe-after-delete="false"/></Section><Content ovf:id="out" xsi:type="ovf:VirtualSystem_Type"><Description>Windows 2008 - App server</Description><Domain>brd-klee.dk</Domain><CreationDate>2013/01/15 01:20:27</CreationDate><ExportDate>2013/11/20 03:29:29</ExportDate><IsAutoSuspend>false</IsAutoSuspend><DeleteProtected>false</DeleteProtected><IsSmartcardEnabled>false</IsSmartcardEnabled><TimeZone></TimeZone><default_boot_sequence>0</default_boot_sequence><Generation>2</Generation><VmType>1</VmType><MinAllocatedMem>4500</MinAllocatedMem><Name>sukhumi</Name><TemplateId>00000000-0000-0000-0000-000000000000</TemplateId><TemplateName>Blank</TemplateName><IsInitilized>true</IsInitilized><IsStateless>false</IsStateless><Origin>0</Origin><app_list>Foxit Reader,Experian DK X-Point 6.6.26,Microsoft SQL Server 2005,TCSconnexion_3607,Zabbix Agent 2.0.8,PM,RHEV-Tools 3.0.39,RHEV-Network64 3.0.7,Java(TM) 6 Update 30,Microsoft SQL Server 2005 Tools Express Edition,Microsoft SQL Server 2005 Express Edition (SQLEXPRESS),RHEV-Block64 3.0.9,RHEV-Spice-Agent64 3.0.5,Java Auto Updater,Microsoft SQL Server Setup Support Files (English),Acronis Backup &amp; Recovery 11.5 Agent Core,Acronis Backup &amp; Recovery 11.5\xa0Agent for Windows,RHEV-Serial64 3.0.6,RHEV-Agent64 3.0.10,Consignor Server 9.1.0.6,Update for Microsoft .NET Framework 3.5 SP1 (KB963707)</app_list><quota_id>00000000-0000-0000-0000-000000000000</quota_id><DefaultDisplayType>1</DefaultDisplayType><Section ovf:id="e97d1c52-b688-4192-bdeb-2dfadcc79786" ovf:required="false" xsi:type="ovf:OperatingSystemSection_Type"><Info>Guest Operating System</Info><Description>Windows2008x64</Description></Section><Section xsi:type="ovf:VirtualHardwareSection_Type"><Info>4 CPU, 4500 Memeory</Info><System><vssd:VirtualSystemType>RHEVM 3.2.0.0</vssd:VirtualSystemType></System><Item><rasd:Caption>4 virtual cpu</rasd:Caption><rasd:Description>Number of virtual CPU</rasd:Description><rasd:InstanceId>1</rasd:InstanceId><rasd:ResourceType>3</rasd:ResourceType><rasd:num_of_sockets>4</rasd:num_of_sockets><rasd:cpu_per_socket>1</rasd:cpu_per_socket></Item><Item><rasd:Caption>4500 MB of memory</rasd:Caption><rasd:Description>Memory Size</rasd:Description><rasd:InstanceId>2</rasd:InstanceId><rasd:ResourceType>4</rasd:ResourceType><rasd:AllocationUnits>MegaBytes</rasd:AllocationUnits><rasd:VirtualQuantity>4500</rasd:VirtualQuantity></Item><Item><rasd:Caption>Drive 1</rasd:Caption><rasd:InstanceId>8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f</rasd:InstanceId><rasd:ResourceType>17</rasd:ResourceType><rasd:HostResource>5af747d8-95b1-4b7e-af53-cb77e4fcc896/8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f</rasd:HostResource><rasd:Parent>00000000-0000-0000-0000-000000000000</rasd:Parent><rasd:Template>00000000-0000-0000-0000-000000000000</rasd:Template><rasd:ApplicationList></rasd:ApplicationList><rasd:StorageId>6b892067-ac7e-4039-a7be-5b7387525b16</rasd:StorageId><rasd:StoragePoolId>c4e5889a-348d-11e2-83fe-9f72369eab30</rasd:StoragePoolId><rasd:CreationDate>2013/01/15 02:06:28</rasd:CreationDate><rasd:LastModified>2013/01/15 02:06:28</rasd:LastModified><rasd:last_modified_date>2013/11/20 03:29:28</rasd:last_modified_date></Item><Item><rasd:Caption>Ethernet adapter on rhevm</rasd:Caption><rasd:InstanceId>fd38b98a-8c16-4dae-817c-794715ec6b17</rasd:InstanceId><rasd:ResourceType>10</rasd:ResourceType><rasd:ResourceSubType>3</rasd:ResourceSubType><rasd:Connection>rhevm</rasd:Connection><rasd:Linked>true</rasd:Linked><rasd:Name>nic1</rasd:Name><rasd:MACAddress>00:1a:4a:a8:64:13</rasd:MACAddress><rasd:speed>1000</rasd:speed></Item><Item><rasd:Caption>USB Controller</rasd:Caption><rasd:InstanceId>3</rasd:InstanceId><rasd:ResourceType>23</rasd:ResourceType><rasd:UsbPolicy>Disabled</rasd:UsbPolicy></Item></Section></Content></ovf:Envelope>\n', 'vm': 'e97d1c52-b688-4192-bdeb-2dfadcc79786'}])
>Thread-15914328::DEBUG::2013-11-20 03:29:30,407::task::495::TaskManager.Task::(_debug) Task 2ca55320-b992-49d0-a570-ded1ea4ae0d5: moving from state init -> state preparing
>Thread-15914328::DEBUG::2013-11-20 03:29:30,408::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ce1bf18b-0578-427b-a79a-2623d910577a`::Request was made in '/usr/share/vdsm/storage/spm.py' line '1240' at 'public_updateVM'
>Thread-15914328::DEBUG::2013-11-20 03:29:30,409::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914328::DEBUG::2013-11-20 03:29:30,410::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914328::DEBUG::2013-11-20 03:29:30,410::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ce1bf18b-0578-427b-a79a-2623d910577a`::Granted request
>Thread-15914328::DEBUG::2013-11-20 03:29:30,411::task::495::TaskManager.Task::(_debug) Task 2ca55320-b992-49d0-a570-ded1ea4ae0d5: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914328::DEBUG::2013-11-20 03:29:30,412::task::495::TaskManager.Task::(_debug) Task 2ca55320-b992-49d0-a570-ded1ea4ae0d5: ref 1 aborting False
>Thread-15914328::INFO::2013-11-20 03:29:30,412::sp::927::Storage.StoragePool::(updateVM) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16
>Thread-15914328::INFO::2013-11-20 03:29:30,413::sp::938::Storage.StoragePool::(updateVM) vmUUID=e97d1c52-b688-4192-bdeb-2dfadcc79786 imgList=['5af747d8-95b1-4b7e-af53-cb77e4fcc896']
>Dummy-15912651::DEBUG::2013-11-20 03:29:30,443::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914328::DEBUG::2013-11-20 03:29:30,665::task::495::TaskManager.Task::(_debug) Task 2ca55320-b992-49d0-a570-ded1ea4ae0d5: finished: None
>Thread-15914328::DEBUG::2013-11-20 03:29:30,667::task::495::TaskManager.Task::(_debug) Task 2ca55320-b992-49d0-a570-ded1ea4ae0d5: moving from state preparing -> state finished
>Thread-15914328::DEBUG::2013-11-20 03:29:30,668::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914328::DEBUG::2013-11-20 03:29:30,669::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914328::DEBUG::2013-11-20 03:29:30,670::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914328::DEBUG::2013-11-20 03:29:30,671::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914328::DEBUG::2013-11-20 03:29:30,672::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914328::DEBUG::2013-11-20 03:29:30,673::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914328::DEBUG::2013-11-20 03:29:30,674::task::495::TaskManager.Task::(_debug) Task 2ca55320-b992-49d0-a570-ded1ea4ae0d5: ref 0 aborting False
>Thread-15914328::INFO::2013-11-20 03:29:30,674::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: updateVM, Return response: {'status': {'message': 'OK', 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:29:31,007::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0827341 s, 12.4 MB/s\n'; <rc> = 0
>Thread-15914329::DEBUG::2013-11-20 03:29:31,383::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914329::INFO::2013-11-20 03:29:31,384::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914329::DEBUG::2013-11-20 03:29:31,385::task::495::TaskManager.Task::(_debug) Task ecb7f167-2acd-4cd2-bfd3-27f59afe7093: moving from state init -> state preparing
>Thread-15914329::DEBUG::2013-11-20 03:29:31,386::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914329::DEBUG::2013-11-20 03:29:31,386::task::495::TaskManager.Task::(_debug) Task ecb7f167-2acd-4cd2-bfd3-27f59afe7093: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914329::DEBUG::2013-11-20 03:29:31,387::task::495::TaskManager.Task::(_debug) Task ecb7f167-2acd-4cd2-bfd3-27f59afe7093: moving from state preparing -> state finished
>Thread-15914329::DEBUG::2013-11-20 03:29:31,388::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914329::DEBUG::2013-11-20 03:29:31,388::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914329::DEBUG::2013-11-20 03:29:31,389::task::495::TaskManager.Task::(_debug) Task ecb7f167-2acd-4cd2-bfd3-27f59afe7093: ref 0 aborting False
>Thread-15914329::INFO::2013-11-20 03:29:31,390::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914330::DEBUG::2013-11-20 03:29:31,396::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914330::INFO::2013-11-20 03:29:31,396::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914330::DEBUG::2013-11-20 03:29:31,397::task::495::TaskManager.Task::(_debug) Task ad0b0f8b-f154-4962-89cf-ef6ff3e7e67d: moving from state init -> state preparing
>Thread-15914330::DEBUG::2013-11-20 03:29:31,398::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5a191715-3d0c-4425-98e4-d0ae675771e9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914330::DEBUG::2013-11-20 03:29:31,399::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914330::DEBUG::2013-11-20 03:29:31,400::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914330::DEBUG::2013-11-20 03:29:31,400::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5a191715-3d0c-4425-98e4-d0ae675771e9`::Granted request
>Thread-15914330::DEBUG::2013-11-20 03:29:31,401::task::495::TaskManager.Task::(_debug) Task ad0b0f8b-f154-4962-89cf-ef6ff3e7e67d: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914330::DEBUG::2013-11-20 03:29:31,402::task::495::TaskManager.Task::(_debug) Task ad0b0f8b-f154-4962-89cf-ef6ff3e7e67d: ref 1 aborting False
>Thread-15914330::WARNING::2013-11-20 03:29:31,404::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914330::DEBUG::2013-11-20 03:29:31,405::task::495::TaskManager.Task::(_debug) Task ad0b0f8b-f154-4962-89cf-ef6ff3e7e67d: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914330::DEBUG::2013-11-20 03:29:31,406::task::495::TaskManager.Task::(_debug) Task ad0b0f8b-f154-4962-89cf-ef6ff3e7e67d: moving from state preparing -> state finished
>Thread-15914330::DEBUG::2013-11-20 03:29:31,407::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914330::DEBUG::2013-11-20 03:29:31,408::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914330::DEBUG::2013-11-20 03:29:31,408::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914330::DEBUG::2013-11-20 03:29:31,409::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914330::DEBUG::2013-11-20 03:29:31,410::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914330::DEBUG::2013-11-20 03:29:31,410::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914330::DEBUG::2013-11-20 03:29:31,411::task::495::TaskManager.Task::(_debug) Task ad0b0f8b-f154-4962-89cf-ef6ff3e7e67d: ref 0 aborting False
>Thread-15914330::INFO::2013-11-20 03:29:31,412::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:29:33,021::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:29:33,557::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0774062 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:29:35,574::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:29:36,130::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0796798 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:29:38,146::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:29:38,721::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0771647 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:29:40,731::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:29:41,315::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.085683 s, 12.0 MB/s\n'; <rc> = 0
>Thread-15914335::DEBUG::2013-11-20 03:29:42,095::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914335::INFO::2013-11-20 03:29:42,097::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914335::DEBUG::2013-11-20 03:29:42,097::task::495::TaskManager.Task::(_debug) Task 69065b85-1014-4e47-826e-b7a7cac4d12e: moving from state init -> state preparing
>Thread-15914335::DEBUG::2013-11-20 03:29:42,098::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914335::DEBUG::2013-11-20 03:29:42,099::task::495::TaskManager.Task::(_debug) Task 69065b85-1014-4e47-826e-b7a7cac4d12e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914335::DEBUG::2013-11-20 03:29:42,100::task::495::TaskManager.Task::(_debug) Task 69065b85-1014-4e47-826e-b7a7cac4d12e: moving from state preparing -> state finished
>Thread-15914335::DEBUG::2013-11-20 03:29:42,100::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914335::DEBUG::2013-11-20 03:29:42,101::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914335::DEBUG::2013-11-20 03:29:42,102::task::495::TaskManager.Task::(_debug) Task 69065b85-1014-4e47-826e-b7a7cac4d12e: ref 0 aborting False
>Thread-15914335::INFO::2013-11-20 03:29:42,103::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914336::DEBUG::2013-11-20 03:29:42,108::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914336::INFO::2013-11-20 03:29:42,109::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914336::DEBUG::2013-11-20 03:29:42,110::task::495::TaskManager.Task::(_debug) Task ab980c16-6e03-45b4-b71b-79764ce0476a: moving from state init -> state preparing
>Thread-15914336::DEBUG::2013-11-20 03:29:42,111::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5f06e5f9-d067-4bd0-b447-840a6c28f147`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914336::DEBUG::2013-11-20 03:29:42,112::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914336::DEBUG::2013-11-20 03:29:42,113::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914336::DEBUG::2013-11-20 03:29:42,113::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5f06e5f9-d067-4bd0-b447-840a6c28f147`::Granted request
>Thread-15914336::DEBUG::2013-11-20 03:29:42,114::task::495::TaskManager.Task::(_debug) Task ab980c16-6e03-45b4-b71b-79764ce0476a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914336::DEBUG::2013-11-20 03:29:42,115::task::495::TaskManager.Task::(_debug) Task ab980c16-6e03-45b4-b71b-79764ce0476a: ref 1 aborting False
>Thread-15914336::WARNING::2013-11-20 03:29:42,117::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914336::DEBUG::2013-11-20 03:29:42,125::task::495::TaskManager.Task::(_debug) Task ab980c16-6e03-45b4-b71b-79764ce0476a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914336::DEBUG::2013-11-20 03:29:42,126::task::495::TaskManager.Task::(_debug) Task ab980c16-6e03-45b4-b71b-79764ce0476a: moving from state preparing -> state finished
>Thread-15914336::DEBUG::2013-11-20 03:29:42,127::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914336::DEBUG::2013-11-20 03:29:42,128::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914336::DEBUG::2013-11-20 03:29:42,129::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914336::DEBUG::2013-11-20 03:29:42,129::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914336::DEBUG::2013-11-20 03:29:42,130::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914336::DEBUG::2013-11-20 03:29:42,131::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914336::DEBUG::2013-11-20 03:29:42,132::task::495::TaskManager.Task::(_debug) Task ab980c16-6e03-45b4-b71b-79764ce0476a: ref 0 aborting False
>Thread-15914336::INFO::2013-11-20 03:29:42,132::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:29:43,331::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:29:43,871::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0791102 s, 12.9 MB/s\n'; <rc> = 0
>Thread-15914337::INFO::2013-11-20 03:29:44,582::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914337::DEBUG::2013-11-20 03:29:44,583::task::495::TaskManager.Task::(_debug) Task 5090d5cd-5c52-4ea4-8376-acb43f1f5aba: moving from state init -> state preparing
>Thread-15914337::DEBUG::2013-11-20 03:29:44,584::task::495::TaskManager.Task::(_debug) Task 5090d5cd-5c52-4ea4-8376-acb43f1f5aba: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000332117080688', 'lastCheck': 1384918174.6563959, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00163912773132', 'lastCheck': 1384918180.9701509, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00172901153564', 'lastCheck': 1384918183.5476329, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000399112701416', 'lastCheck': 1384918183.9586871, 'valid': True, 'code': 0}}
>Thread-15914337::DEBUG::2013-11-20 03:29:44,585::task::495::TaskManager.Task::(_debug) Task 5090d5cd-5c52-4ea4-8376-acb43f1f5aba: moving from state preparing -> state finished
>Thread-15914337::DEBUG::2013-11-20 03:29:44,585::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914337::DEBUG::2013-11-20 03:29:44,586::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914337::DEBUG::2013-11-20 03:29:44,587::task::495::TaskManager.Task::(_debug) Task 5090d5cd-5c52-4ea4-8376-acb43f1f5aba: ref 0 aborting False
>Thread-15914337::INFO::2013-11-20 03:29:44,587::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000332117080688', 'lastCheck': 1384918174.6563959, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00163912773132', 'lastCheck': 1384918180.9701509, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00172901153564', 'lastCheck': 1384918183.5476329, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000399112701416', 'lastCheck': 1384918183.9586871, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:29:45,889::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 03:29:46,126::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:29:46,128::task::495::TaskManager.Task::(_debug) Task 714bec9a-5a98-436b-9271-95f52da843bd: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:29:46,131::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d1fb841c-7243-400c-9abb-84da38a0812b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:29:46,131::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:29:46,133::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:29:46,134::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d1fb841c-7243-400c-9abb-84da38a0812b`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:29:46,135::task::495::TaskManager.Task::(_debug) Task 714bec9a-5a98-436b-9271-95f52da843bd: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:29:46,136::task::495::TaskManager.Task::(_debug) Task 714bec9a-5a98-436b-9271-95f52da843bd: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:29:46,138::task::495::TaskManager.Task::(_debug) Task 714bec9a-5a98-436b-9271-95f52da843bd: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:29:46,139::task::495::TaskManager.Task::(_debug) Task 714bec9a-5a98-436b-9271-95f52da843bd: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:29:46,139::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:29:46,140::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:29:46,141::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:29:46,141::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:29:46,142::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:29:46,142::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:29:46,143::task::495::TaskManager.Task::(_debug) Task 714bec9a-5a98-436b-9271-95f52da843bd: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:29:46,144::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:29:46,438::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0797128 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:29:48,454::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 03:29:48,651::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:29:48,661::task::495::TaskManager.Task::(_debug) Task 3ade764b-5c45-4ab6-ab09-b9045fea3dd4: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:29:48,666::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c1ddcb24-7d75-4399-bb58-dd73079f5414`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:29:48,668::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:29:48,670::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:29:48,671::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c1ddcb24-7d75-4399-bb58-dd73079f5414`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:29:48,673::task::495::TaskManager.Task::(_debug) Task 3ade764b-5c45-4ab6-ab09-b9045fea3dd4: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:29:48,673::task::495::TaskManager.Task::(_debug) Task 3ade764b-5c45-4ab6-ab09-b9045fea3dd4: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:29:48,674::task::495::TaskManager.Task::(_debug) Task 3ade764b-5c45-4ab6-ab09-b9045fea3dd4: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:29:48,675::task::495::TaskManager.Task::(_debug) Task 3ade764b-5c45-4ab6-ab09-b9045fea3dd4: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:29:48,676::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:29:48,676::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:29:48,677::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:29:48,677::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:29:48,678::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:29:48,678::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:29:48,679::task::495::TaskManager.Task::(_debug) Task 3ade764b-5c45-4ab6-ab09-b9045fea3dd4: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:29:48,680::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:29:48,986::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0833071 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:29:51,000::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:29:51,555::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0786856 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 03:29:52,118::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:29:52,119::task::495::TaskManager.Task::(_debug) Task 15d70b9c-ea64-4ca7-ae3d-d5c89c1ce0d1: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:29:52,121::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c3925e77-3117-4202-b362-1989915ab5d6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:29:52,121::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:29:52,122::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:29:52,123::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c3925e77-3117-4202-b362-1989915ab5d6`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:29:52,124::task::495::TaskManager.Task::(_debug) Task 15d70b9c-ea64-4ca7-ae3d-d5c89c1ce0d1: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:29:52,124::task::495::TaskManager.Task::(_debug) Task 15d70b9c-ea64-4ca7-ae3d-d5c89c1ce0d1: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:29:52,125::task::495::TaskManager.Task::(_debug) Task 15d70b9c-ea64-4ca7-ae3d-d5c89c1ce0d1: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:29:52,126::task::495::TaskManager.Task::(_debug) Task 15d70b9c-ea64-4ca7-ae3d-d5c89c1ce0d1: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:29:52,126::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:29:52,127::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:29:52,128::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:29:52,128::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:29:52,129::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:29:52,129::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:29:52,130::task::495::TaskManager.Task::(_debug) Task 15d70b9c-ea64-4ca7-ae3d-d5c89c1ce0d1: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:29:52,131::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914341::DEBUG::2013-11-20 03:29:52,285::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914341::INFO::2013-11-20 03:29:52,286::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914341::DEBUG::2013-11-20 03:29:52,287::task::495::TaskManager.Task::(_debug) Task ac4da876-4cb4-4e89-8eaa-ecb46767e156: moving from state init -> state preparing
>Thread-15914341::DEBUG::2013-11-20 03:29:52,288::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914341::DEBUG::2013-11-20 03:29:52,288::task::495::TaskManager.Task::(_debug) Task ac4da876-4cb4-4e89-8eaa-ecb46767e156: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914341::DEBUG::2013-11-20 03:29:52,289::task::495::TaskManager.Task::(_debug) Task ac4da876-4cb4-4e89-8eaa-ecb46767e156: moving from state preparing -> state finished
>Thread-15914341::DEBUG::2013-11-20 03:29:52,290::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914341::DEBUG::2013-11-20 03:29:52,290::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914341::DEBUG::2013-11-20 03:29:52,291::task::495::TaskManager.Task::(_debug) Task ac4da876-4cb4-4e89-8eaa-ecb46767e156: ref 0 aborting False
>Thread-15914341::INFO::2013-11-20 03:29:52,292::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914342::DEBUG::2013-11-20 03:29:52,296::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914342::INFO::2013-11-20 03:29:52,297::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914342::DEBUG::2013-11-20 03:29:52,298::task::495::TaskManager.Task::(_debug) Task 7a4d4826-8cdb-44a5-9823-cb455582aa6f: moving from state init -> state preparing
>Thread-15914342::DEBUG::2013-11-20 03:29:52,299::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`fa028e58-d974-43de-a968-5a676ea23a8a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914342::DEBUG::2013-11-20 03:29:52,299::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914342::DEBUG::2013-11-20 03:29:52,300::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914342::DEBUG::2013-11-20 03:29:52,301::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`fa028e58-d974-43de-a968-5a676ea23a8a`::Granted request
>Thread-15914342::DEBUG::2013-11-20 03:29:52,302::task::495::TaskManager.Task::(_debug) Task 7a4d4826-8cdb-44a5-9823-cb455582aa6f: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914342::DEBUG::2013-11-20 03:29:52,302::task::495::TaskManager.Task::(_debug) Task 7a4d4826-8cdb-44a5-9823-cb455582aa6f: ref 1 aborting False
>Thread-15914342::WARNING::2013-11-20 03:29:52,304::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914342::DEBUG::2013-11-20 03:29:52,305::task::495::TaskManager.Task::(_debug) Task 7a4d4826-8cdb-44a5-9823-cb455582aa6f: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914342::DEBUG::2013-11-20 03:29:52,306::task::495::TaskManager.Task::(_debug) Task 7a4d4826-8cdb-44a5-9823-cb455582aa6f: moving from state preparing -> state finished
>Thread-15914342::DEBUG::2013-11-20 03:29:52,307::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914342::DEBUG::2013-11-20 03:29:52,307::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914342::DEBUG::2013-11-20 03:29:52,308::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914342::DEBUG::2013-11-20 03:29:52,309::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914342::DEBUG::2013-11-20 03:29:52,309::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914342::DEBUG::2013-11-20 03:29:52,310::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914342::DEBUG::2013-11-20 03:29:52,310::task::495::TaskManager.Task::(_debug) Task 7a4d4826-8cdb-44a5-9823-cb455582aa6f: ref 0 aborting False
>Thread-15914342::INFO::2013-11-20 03:29:52,311::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:29:53,572::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:29:54,139::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0767546 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:29:56,156::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:29:56,701::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0756895 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:29:58,717::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:29:59,240::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0816407 s, 12.5 MB/s\n'; <rc> = 0
>Thread-15914345::INFO::2013-11-20 03:30:00,274::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914345::DEBUG::2013-11-20 03:30:00,274::task::495::TaskManager.Task::(_debug) Task 1f2523d6-e397-4b02-9b5f-864ce6c8a2e7: moving from state init -> state preparing
>Thread-15914345::DEBUG::2013-11-20 03:30:00,275::task::495::TaskManager.Task::(_debug) Task 1f2523d6-e397-4b02-9b5f-864ce6c8a2e7: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000378847122192', 'lastCheck': 1384918195.2213161, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00144815444946', 'lastCheck': 1384918191.2276659, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00191307067871', 'lastCheck': 1384918193.7932119, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000526189804077', 'lastCheck': 1384918194.2159481, 'valid': True, 'code': 0}}
>Thread-15914345::DEBUG::2013-11-20 03:30:00,280::task::495::TaskManager.Task::(_debug) Task 1f2523d6-e397-4b02-9b5f-864ce6c8a2e7: moving from state preparing -> state finished
>Thread-15914345::DEBUG::2013-11-20 03:30:00,280::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914345::DEBUG::2013-11-20 03:30:00,280::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914345::DEBUG::2013-11-20 03:30:00,281::task::495::TaskManager.Task::(_debug) Task 1f2523d6-e397-4b02-9b5f-864ce6c8a2e7: ref 0 aborting False
>Thread-15914345::INFO::2013-11-20 03:30:00,281::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000378847122192', 'lastCheck': 1384918195.2213161, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00144815444946', 'lastCheck': 1384918191.2276659, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00191307067871', 'lastCheck': 1384918193.7932119, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000526189804077', 'lastCheck': 1384918194.2159481, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:30:01,257::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:30:01,828::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0938741 s, 10.9 MB/s\n'; <rc> = 0
>Thread-15914347::DEBUG::2013-11-20 03:30:02,495::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914347::INFO::2013-11-20 03:30:02,499::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914347::DEBUG::2013-11-20 03:30:02,502::task::495::TaskManager.Task::(_debug) Task 0772e2ab-17fe-4c99-b9fe-0e7103a4e18e: moving from state init -> state preparing
>Thread-15914347::DEBUG::2013-11-20 03:30:02,504::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914347::DEBUG::2013-11-20 03:30:02,505::task::495::TaskManager.Task::(_debug) Task 0772e2ab-17fe-4c99-b9fe-0e7103a4e18e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914347::DEBUG::2013-11-20 03:30:02,506::task::495::TaskManager.Task::(_debug) Task 0772e2ab-17fe-4c99-b9fe-0e7103a4e18e: moving from state preparing -> state finished
>Thread-15914347::DEBUG::2013-11-20 03:30:02,506::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914347::DEBUG::2013-11-20 03:30:02,507::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914347::DEBUG::2013-11-20 03:30:02,507::task::495::TaskManager.Task::(_debug) Task 0772e2ab-17fe-4c99-b9fe-0e7103a4e18e: ref 0 aborting False
>Thread-15914347::INFO::2013-11-20 03:30:02,508::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914348::DEBUG::2013-11-20 03:30:02,512::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914348::INFO::2013-11-20 03:30:02,513::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914348::DEBUG::2013-11-20 03:30:02,514::task::495::TaskManager.Task::(_debug) Task d96ee0b8-3cdb-474b-9eb9-27dd24c52305: moving from state init -> state preparing
>Thread-15914348::DEBUG::2013-11-20 03:30:02,515::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b6e8de14-1a72-4b85-81b6-91dd672481cf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914348::DEBUG::2013-11-20 03:30:02,516::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914348::DEBUG::2013-11-20 03:30:02,517::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914348::DEBUG::2013-11-20 03:30:02,517::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b6e8de14-1a72-4b85-81b6-91dd672481cf`::Granted request
>Thread-15914348::DEBUG::2013-11-20 03:30:02,518::task::495::TaskManager.Task::(_debug) Task d96ee0b8-3cdb-474b-9eb9-27dd24c52305: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914348::DEBUG::2013-11-20 03:30:02,519::task::495::TaskManager.Task::(_debug) Task d96ee0b8-3cdb-474b-9eb9-27dd24c52305: ref 1 aborting False
>Thread-15914348::WARNING::2013-11-20 03:30:02,520::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914348::DEBUG::2013-11-20 03:30:02,526::task::495::TaskManager.Task::(_debug) Task d96ee0b8-3cdb-474b-9eb9-27dd24c52305: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914348::DEBUG::2013-11-20 03:30:02,527::task::495::TaskManager.Task::(_debug) Task d96ee0b8-3cdb-474b-9eb9-27dd24c52305: moving from state preparing -> state finished
>Thread-15914348::DEBUG::2013-11-20 03:30:02,528::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914348::DEBUG::2013-11-20 03:30:02,528::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914348::DEBUG::2013-11-20 03:30:02,529::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914348::DEBUG::2013-11-20 03:30:02,530::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914348::DEBUG::2013-11-20 03:30:02,530::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914348::DEBUG::2013-11-20 03:30:02,530::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914348::DEBUG::2013-11-20 03:30:02,531::task::495::TaskManager.Task::(_debug) Task d96ee0b8-3cdb-474b-9eb9-27dd24c52305: ref 0 aborting False
>Thread-15914348::INFO::2013-11-20 03:30:02,532::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:30:03,843::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:30:04,342::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0727515 s, 14.1 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 03:30:04,467::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:30:04,468::task::495::TaskManager.Task::(_debug) Task 5a79e085-3216-4ec3-8db8-9777c9ea33b2: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:30:04,470::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ad518400-3a8c-4193-bacc-cfb78a09589d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:30:04,470::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:30:04,471::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:30:04,472::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ad518400-3a8c-4193-bacc-cfb78a09589d`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:30:04,473::task::495::TaskManager.Task::(_debug) Task 5a79e085-3216-4ec3-8db8-9777c9ea33b2: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:30:04,473::task::495::TaskManager.Task::(_debug) Task 5a79e085-3216-4ec3-8db8-9777c9ea33b2: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:30:04,474::task::495::TaskManager.Task::(_debug) Task 5a79e085-3216-4ec3-8db8-9777c9ea33b2: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:30:04,475::task::495::TaskManager.Task::(_debug) Task 5a79e085-3216-4ec3-8db8-9777c9ea33b2: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:30:04,475::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:30:04,476::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:30:04,478::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:30:04,479::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:30:04,479::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:30:04,480::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:30:04,480::task::495::TaskManager.Task::(_debug) Task 5a79e085-3216-4ec3-8db8-9777c9ea33b2: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:30:04,481::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:30:04,482::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:30:04,482::task::495::TaskManager.Task::(_debug) Task 54542d2b-9b21-4e96-a5f6-803058f1798c: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:30:04,483::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3323da51-2fd9-49ba-8321-7105a3dba8bc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:30:04,484::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:30:04,485::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:30:04,485::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3323da51-2fd9-49ba-8321-7105a3dba8bc`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:30:04,486::task::495::TaskManager.Task::(_debug) Task 54542d2b-9b21-4e96-a5f6-803058f1798c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:30:04,487::task::495::TaskManager.Task::(_debug) Task 54542d2b-9b21-4e96-a5f6-803058f1798c: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:30:04,487::task::495::TaskManager.Task::(_debug) Task 54542d2b-9b21-4e96-a5f6-803058f1798c: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:30:04,488::task::495::TaskManager.Task::(_debug) Task 54542d2b-9b21-4e96-a5f6-803058f1798c: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:30:04,488::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:30:04,489::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:30:04,490::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:30:04,491::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:30:04,492::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:30:04,492::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:30:04,493::task::495::TaskManager.Task::(_debug) Task 54542d2b-9b21-4e96-a5f6-803058f1798c: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:30:04,494::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:30:06,357::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:30:06,901::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0773576 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:30:08,920::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:30:09,499::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0939553 s, 10.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:30:11,514::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:30:12,073::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0673833 s, 15.2 MB/s\n'; <rc> = 0
>Thread-15914353::DEBUG::2013-11-20 03:30:13,256::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914353::INFO::2013-11-20 03:30:13,257::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914353::DEBUG::2013-11-20 03:30:13,258::task::495::TaskManager.Task::(_debug) Task cb478993-87fe-4294-a3fe-52379323f2a9: moving from state init -> state preparing
>Thread-15914353::DEBUG::2013-11-20 03:30:13,259::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914353::DEBUG::2013-11-20 03:30:13,259::task::495::TaskManager.Task::(_debug) Task cb478993-87fe-4294-a3fe-52379323f2a9: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914353::DEBUG::2013-11-20 03:30:13,260::task::495::TaskManager.Task::(_debug) Task cb478993-87fe-4294-a3fe-52379323f2a9: moving from state preparing -> state finished
>Thread-15914353::DEBUG::2013-11-20 03:30:13,261::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914353::DEBUG::2013-11-20 03:30:13,261::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914353::DEBUG::2013-11-20 03:30:13,262::task::495::TaskManager.Task::(_debug) Task cb478993-87fe-4294-a3fe-52379323f2a9: ref 0 aborting False
>Thread-15914353::INFO::2013-11-20 03:30:13,262::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914354::DEBUG::2013-11-20 03:30:13,267::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914354::INFO::2013-11-20 03:30:13,268::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914354::DEBUG::2013-11-20 03:30:13,269::task::495::TaskManager.Task::(_debug) Task 6e8bb751-fdf9-4a6f-baa9-37e7318be74a: moving from state init -> state preparing
>Thread-15914354::DEBUG::2013-11-20 03:30:13,271::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a64548b6-5e57-437b-831e-d38b1f9682ee`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914354::DEBUG::2013-11-20 03:30:13,272::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914354::DEBUG::2013-11-20 03:30:13,274::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914354::DEBUG::2013-11-20 03:30:13,275::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a64548b6-5e57-437b-831e-d38b1f9682ee`::Granted request
>Thread-15914354::DEBUG::2013-11-20 03:30:13,276::task::495::TaskManager.Task::(_debug) Task 6e8bb751-fdf9-4a6f-baa9-37e7318be74a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914354::DEBUG::2013-11-20 03:30:13,277::task::495::TaskManager.Task::(_debug) Task 6e8bb751-fdf9-4a6f-baa9-37e7318be74a: ref 1 aborting False
>Thread-15914354::WARNING::2013-11-20 03:30:13,281::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914354::DEBUG::2013-11-20 03:30:13,282::task::495::TaskManager.Task::(_debug) Task 6e8bb751-fdf9-4a6f-baa9-37e7318be74a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914354::DEBUG::2013-11-20 03:30:13,284::task::495::TaskManager.Task::(_debug) Task 6e8bb751-fdf9-4a6f-baa9-37e7318be74a: moving from state preparing -> state finished
>Thread-15914354::DEBUG::2013-11-20 03:30:13,285::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914354::DEBUG::2013-11-20 03:30:13,286::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914354::DEBUG::2013-11-20 03:30:13,287::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914354::DEBUG::2013-11-20 03:30:13,288::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914354::DEBUG::2013-11-20 03:30:13,289::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914354::DEBUG::2013-11-20 03:30:13,289::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914354::DEBUG::2013-11-20 03:30:13,290::task::495::TaskManager.Task::(_debug) Task 6e8bb751-fdf9-4a6f-baa9-37e7318be74a: ref 0 aborting False
>Thread-15914354::INFO::2013-11-20 03:30:13,291::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:30:14,087::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:30:14,634::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0770708 s, 13.3 MB/s\n'; <rc> = 0
>Thread-15914355::INFO::2013-11-20 03:30:15,672::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914355::DEBUG::2013-11-20 03:30:15,674::task::495::TaskManager.Task::(_debug) Task 3e1708d3-5d5e-463a-a191-808268c70a61: moving from state init -> state preparing
>Thread-15914355::DEBUG::2013-11-20 03:30:15,674::task::495::TaskManager.Task::(_debug) Task 3e1708d3-5d5e-463a-a191-808268c70a61: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000300884246826', 'lastCheck': 1384918205.480418, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00107479095459', 'lastCheck': 1384918211.774291, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0018470287323', 'lastCheck': 1384918214.304702, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000363111495972', 'lastCheck': 1384918214.7703249, 'valid': True, 'code': 0}}
>Thread-15914355::DEBUG::2013-11-20 03:30:15,675::task::495::TaskManager.Task::(_debug) Task 3e1708d3-5d5e-463a-a191-808268c70a61: moving from state preparing -> state finished
>Thread-15914355::DEBUG::2013-11-20 03:30:15,676::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914355::DEBUG::2013-11-20 03:30:15,676::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914355::DEBUG::2013-11-20 03:30:15,677::task::495::TaskManager.Task::(_debug) Task 3e1708d3-5d5e-463a-a191-808268c70a61: ref 0 aborting False
>Thread-15914355::INFO::2013-11-20 03:30:15,678::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000300884246826', 'lastCheck': 1384918205.480418, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00107479095459', 'lastCheck': 1384918211.774291, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0018470287323', 'lastCheck': 1384918214.304702, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000363111495972', 'lastCheck': 1384918214.7703249, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:30:16,649::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:30:17,211::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0880386 s, 11.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:30:19,223::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 03:30:19,441::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:30:19,444::task::495::TaskManager.Task::(_debug) Task 51ba59e0-646a-4d11-902e-ba98379c3856: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:30:19,447::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`580075f0-db91-4ee9-877f-a153d6b223d2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:30:19,449::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:30:19,450::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:30:19,450::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`580075f0-db91-4ee9-877f-a153d6b223d2`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:30:19,452::task::495::TaskManager.Task::(_debug) Task 51ba59e0-646a-4d11-902e-ba98379c3856: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:30:19,452::task::495::TaskManager.Task::(_debug) Task 51ba59e0-646a-4d11-902e-ba98379c3856: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:30:19,453::task::495::TaskManager.Task::(_debug) Task 51ba59e0-646a-4d11-902e-ba98379c3856: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:30:19,454::task::495::TaskManager.Task::(_debug) Task 51ba59e0-646a-4d11-902e-ba98379c3856: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:30:19,454::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:30:19,455::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:30:19,456::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:30:19,456::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:30:19,457::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:30:19,457::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:30:19,458::task::495::TaskManager.Task::(_debug) Task 51ba59e0-646a-4d11-902e-ba98379c3856: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:30:19,459::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:30:19,460::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:30:19,460::task::495::TaskManager.Task::(_debug) Task db6a2c96-cf54-478b-82b6-99fceb578ed8: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:30:19,461::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c78aadae-e553-4662-a8d0-a0a0027c8e5f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:30:19,462::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:30:19,462::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:30:19,463::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c78aadae-e553-4662-a8d0-a0a0027c8e5f`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:30:19,464::task::495::TaskManager.Task::(_debug) Task db6a2c96-cf54-478b-82b6-99fceb578ed8: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:30:19,465::task::495::TaskManager.Task::(_debug) Task db6a2c96-cf54-478b-82b6-99fceb578ed8: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:30:19,466::task::495::TaskManager.Task::(_debug) Task db6a2c96-cf54-478b-82b6-99fceb578ed8: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:30:19,466::task::495::TaskManager.Task::(_debug) Task db6a2c96-cf54-478b-82b6-99fceb578ed8: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:30:19,467::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:30:19,467::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:30:19,468::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:30:19,468::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:30:19,469::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:30:19,470::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:30:19,470::task::495::TaskManager.Task::(_debug) Task db6a2c96-cf54-478b-82b6-99fceb578ed8: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:30:19,471::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:30:19,472::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:30:19,472::task::495::TaskManager.Task::(_debug) Task c4c4f2a7-c1e5-467e-bccb-f0976616f057: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:30:19,473::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`7c6ed090-cb77-40c0-a295-fd8a31e38571`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:30:19,474::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:30:19,474::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:30:19,475::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`7c6ed090-cb77-40c0-a295-fd8a31e38571`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:30:19,476::task::495::TaskManager.Task::(_debug) Task c4c4f2a7-c1e5-467e-bccb-f0976616f057: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:30:19,476::task::495::TaskManager.Task::(_debug) Task c4c4f2a7-c1e5-467e-bccb-f0976616f057: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:30:19,477::task::495::TaskManager.Task::(_debug) Task c4c4f2a7-c1e5-467e-bccb-f0976616f057: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:30:19,478::task::495::TaskManager.Task::(_debug) Task c4c4f2a7-c1e5-467e-bccb-f0976616f057: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:30:19,478::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:30:19,479::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:30:19,479::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:30:19,480::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:30:19,480::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:30:19,481::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:30:19,482::task::495::TaskManager.Task::(_debug) Task c4c4f2a7-c1e5-467e-bccb-f0976616f057: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:30:19,482::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:30:19,483::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:30:19,483::task::495::TaskManager.Task::(_debug) Task 054d8bd0-223f-4a35-b199-47682e966b36: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:30:19,485::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fe6fef9a-c27c-4bba-944e-34394fa5fe51`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:30:19,485::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:30:19,486::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:30:19,486::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fe6fef9a-c27c-4bba-944e-34394fa5fe51`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:30:19,487::task::495::TaskManager.Task::(_debug) Task 054d8bd0-223f-4a35-b199-47682e966b36: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:30:19,488::task::495::TaskManager.Task::(_debug) Task 054d8bd0-223f-4a35-b199-47682e966b36: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:30:19,488::task::495::TaskManager.Task::(_debug) Task 054d8bd0-223f-4a35-b199-47682e966b36: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:30:19,489::task::495::TaskManager.Task::(_debug) Task 054d8bd0-223f-4a35-b199-47682e966b36: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:30:19,489::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:30:19,490::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:30:19,490::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:30:19,491::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:30:19,492::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:30:19,492::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:30:19,493::task::495::TaskManager.Task::(_debug) Task 054d8bd0-223f-4a35-b199-47682e966b36: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:30:19,493::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:30:19,765::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0788895 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:30:21,781::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:30:22,349::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0802797 s, 12.8 MB/s\n'; <rc> = 0
>Thread-15914359::DEBUG::2013-11-20 03:30:23,389::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914359::INFO::2013-11-20 03:30:23,391::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914359::DEBUG::2013-11-20 03:30:23,391::task::495::TaskManager.Task::(_debug) Task d32ba378-fdce-46e1-b611-a0e8903127cb: moving from state init -> state preparing
>Thread-15914359::DEBUG::2013-11-20 03:30:23,392::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914359::DEBUG::2013-11-20 03:30:23,393::task::495::TaskManager.Task::(_debug) Task d32ba378-fdce-46e1-b611-a0e8903127cb: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914359::DEBUG::2013-11-20 03:30:23,394::task::495::TaskManager.Task::(_debug) Task d32ba378-fdce-46e1-b611-a0e8903127cb: moving from state preparing -> state finished
>Thread-15914359::DEBUG::2013-11-20 03:30:23,394::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914359::DEBUG::2013-11-20 03:30:23,395::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914359::DEBUG::2013-11-20 03:30:23,396::task::495::TaskManager.Task::(_debug) Task d32ba378-fdce-46e1-b611-a0e8903127cb: ref 0 aborting False
>Thread-15914359::INFO::2013-11-20 03:30:23,396::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914360::DEBUG::2013-11-20 03:30:23,408::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914360::INFO::2013-11-20 03:30:23,415::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914360::DEBUG::2013-11-20 03:30:23,419::task::495::TaskManager.Task::(_debug) Task 94245858-1af5-40fc-846e-dfda6f6e7675: moving from state init -> state preparing
>Thread-15914360::DEBUG::2013-11-20 03:30:23,425::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d5a8b221-9a3e-4006-88cc-adaab73aebc0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914360::DEBUG::2013-11-20 03:30:23,427::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914360::DEBUG::2013-11-20 03:30:23,428::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914360::DEBUG::2013-11-20 03:30:23,429::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d5a8b221-9a3e-4006-88cc-adaab73aebc0`::Granted request
>Thread-15914360::DEBUG::2013-11-20 03:30:23,430::task::495::TaskManager.Task::(_debug) Task 94245858-1af5-40fc-846e-dfda6f6e7675: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914360::DEBUG::2013-11-20 03:30:23,431::task::495::TaskManager.Task::(_debug) Task 94245858-1af5-40fc-846e-dfda6f6e7675: ref 1 aborting False
>Thread-15914360::WARNING::2013-11-20 03:30:23,433::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914360::DEBUG::2013-11-20 03:30:23,434::task::495::TaskManager.Task::(_debug) Task 94245858-1af5-40fc-846e-dfda6f6e7675: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914360::DEBUG::2013-11-20 03:30:23,435::task::495::TaskManager.Task::(_debug) Task 94245858-1af5-40fc-846e-dfda6f6e7675: moving from state preparing -> state finished
>Thread-15914360::DEBUG::2013-11-20 03:30:23,436::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914360::DEBUG::2013-11-20 03:30:23,436::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914360::DEBUG::2013-11-20 03:30:23,437::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914360::DEBUG::2013-11-20 03:30:23,438::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914360::DEBUG::2013-11-20 03:30:23,438::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914360::DEBUG::2013-11-20 03:30:23,439::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914360::DEBUG::2013-11-20 03:30:23,440::task::495::TaskManager.Task::(_debug) Task 94245858-1af5-40fc-846e-dfda6f6e7675: ref 0 aborting False
>Thread-15914360::INFO::2013-11-20 03:30:23,440::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:30:24,363::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:30:24,888::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.073269 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:30:26,903::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:30:27,436::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0738863 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:30:29,452::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 03:30:29,666::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:30:29,668::task::495::TaskManager.Task::(_debug) Task ed99edb0-b49f-4b0d-9ff3-a3f4fa19ed8d: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:30:29,671::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5fd76e02-fdb9-49b7-aeee-0bc549d12081`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:30:29,672::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:30:29,674::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:30:29,674::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5fd76e02-fdb9-49b7-aeee-0bc549d12081`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:30:29,676::task::495::TaskManager.Task::(_debug) Task ed99edb0-b49f-4b0d-9ff3-a3f4fa19ed8d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:30:29,677::task::495::TaskManager.Task::(_debug) Task ed99edb0-b49f-4b0d-9ff3-a3f4fa19ed8d: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:30:29,677::task::495::TaskManager.Task::(_debug) Task ed99edb0-b49f-4b0d-9ff3-a3f4fa19ed8d: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:30:29,678::task::495::TaskManager.Task::(_debug) Task ed99edb0-b49f-4b0d-9ff3-a3f4fa19ed8d: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:30:29,679::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:30:29,679::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:30:29,680::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:30:29,681::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:30:29,681::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:30:29,682::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:30:29,683::task::495::TaskManager.Task::(_debug) Task ed99edb0-b49f-4b0d-9ff3-a3f4fa19ed8d: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:30:29,683::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:30:29,972::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0819056 s, 12.5 MB/s\n'; <rc> = 0
>Thread-15914363::DEBUG::2013-11-20 03:30:30,106::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Thread-15914363::DEBUG::2013-11-20 03:30:30,787::clientIF::1219::vds::(fenceNode) rc 0 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = On
>Done
> err 
>Thread-15914364::DEBUG::2013-11-20 03:30:30,800::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=off,secure=,options=power_wait=4
>lanplus=1)
>Thread-15914365::INFO::2013-11-20 03:30:31,078::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914365::DEBUG::2013-11-20 03:30:31,080::task::495::TaskManager.Task::(_debug) Task f37398bb-8f94-45b6-a932-1ea7e23c6d07: moving from state init -> state preparing
>Thread-15914365::DEBUG::2013-11-20 03:30:31,081::task::495::TaskManager.Task::(_debug) Task f37398bb-8f94-45b6-a932-1ea7e23c6d07: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000282049179077', 'lastCheck': 1384918226.016243, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00170803070068', 'lastCheck': 1384918222.0241311, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00166201591492', 'lastCheck': 1384918224.5941899, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000417947769165', 'lastCheck': 1384918225.0157051, 'valid': True, 'code': 0}}
>Thread-15914365::DEBUG::2013-11-20 03:30:31,082::task::495::TaskManager.Task::(_debug) Task f37398bb-8f94-45b6-a932-1ea7e23c6d07: moving from state preparing -> state finished
>Thread-15914365::DEBUG::2013-11-20 03:30:31,082::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914365::DEBUG::2013-11-20 03:30:31,083::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914365::DEBUG::2013-11-20 03:30:31,084::task::495::TaskManager.Task::(_debug) Task f37398bb-8f94-45b6-a932-1ea7e23c6d07: ref 0 aborting False
>Thread-15914365::INFO::2013-11-20 03:30:31,084::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000282049179077', 'lastCheck': 1384918226.016243, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00170803070068', 'lastCheck': 1384918222.0241311, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00166201591492', 'lastCheck': 1384918224.5941899, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000417947769165', 'lastCheck': 1384918225.0157051, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:30:31,987::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:30:32,513::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0719724 s, 14.2 MB/s\n'; <rc> = 0
>Thread-15914368::DEBUG::2013-11-20 03:30:33,536::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914368::INFO::2013-11-20 03:30:33,537::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914368::DEBUG::2013-11-20 03:30:33,538::task::495::TaskManager.Task::(_debug) Task 7feeabad-c0e0-48f5-a6a5-5fd2cb887050: moving from state init -> state preparing
>Thread-15914368::DEBUG::2013-11-20 03:30:33,539::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914368::DEBUG::2013-11-20 03:30:33,540::task::495::TaskManager.Task::(_debug) Task 7feeabad-c0e0-48f5-a6a5-5fd2cb887050: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914368::DEBUG::2013-11-20 03:30:33,540::task::495::TaskManager.Task::(_debug) Task 7feeabad-c0e0-48f5-a6a5-5fd2cb887050: moving from state preparing -> state finished
>Thread-15914368::DEBUG::2013-11-20 03:30:33,541::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914368::DEBUG::2013-11-20 03:30:33,541::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914368::DEBUG::2013-11-20 03:30:33,542::task::495::TaskManager.Task::(_debug) Task 7feeabad-c0e0-48f5-a6a5-5fd2cb887050: ref 0 aborting False
>Thread-15914368::INFO::2013-11-20 03:30:33,543::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914369::DEBUG::2013-11-20 03:30:33,552::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914369::INFO::2013-11-20 03:30:33,553::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914369::DEBUG::2013-11-20 03:30:33,553::task::495::TaskManager.Task::(_debug) Task 3189833a-84ba-42f1-bf77-9b6712e22287: moving from state init -> state preparing
>Thread-15914369::DEBUG::2013-11-20 03:30:33,555::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`efb9e2cd-f963-43a1-af1a-f1ff7fabe6ac`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914369::DEBUG::2013-11-20 03:30:33,555::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914369::DEBUG::2013-11-20 03:30:33,556::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914369::DEBUG::2013-11-20 03:30:33,557::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`efb9e2cd-f963-43a1-af1a-f1ff7fabe6ac`::Granted request
>Thread-15914369::DEBUG::2013-11-20 03:30:33,558::task::495::TaskManager.Task::(_debug) Task 3189833a-84ba-42f1-bf77-9b6712e22287: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914369::DEBUG::2013-11-20 03:30:33,559::task::495::TaskManager.Task::(_debug) Task 3189833a-84ba-42f1-bf77-9b6712e22287: ref 1 aborting False
>Thread-15914369::WARNING::2013-11-20 03:30:33,561::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914369::DEBUG::2013-11-20 03:30:33,562::task::495::TaskManager.Task::(_debug) Task 3189833a-84ba-42f1-bf77-9b6712e22287: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914369::DEBUG::2013-11-20 03:30:33,563::task::495::TaskManager.Task::(_debug) Task 3189833a-84ba-42f1-bf77-9b6712e22287: moving from state preparing -> state finished
>Thread-15914369::DEBUG::2013-11-20 03:30:33,564::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914369::DEBUG::2013-11-20 03:30:33,564::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914369::DEBUG::2013-11-20 03:30:33,565::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914369::DEBUG::2013-11-20 03:30:33,566::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914369::DEBUG::2013-11-20 03:30:33,566::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914369::DEBUG::2013-11-20 03:30:33,567::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914369::DEBUG::2013-11-20 03:30:33,568::task::495::TaskManager.Task::(_debug) Task 3189833a-84ba-42f1-bf77-9b6712e22287: ref 0 aborting False
>Thread-15914369::INFO::2013-11-20 03:30:33,568::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:30:34,531::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:30:35,018::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0742731 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15914367::DEBUG::2013-11-20 03:30:36,225::clientIF::1179::vds::(waitForPid) rc 0 inp agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=off
>passwd=XXXX
>power_wait=4
>lanplus=1 out Powering off machine @ IPMI:192.168.100.119...Done
> err 
>Thread-15914371::DEBUG::2013-11-20 03:30:36,323::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Thread-15914371::DEBUG::2013-11-20 03:30:36,916::clientIF::1219::vds::(fenceNode) rc 2 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = Off
>Done
> err 
>Thread-15914372::DEBUG::2013-11-20 03:30:36,923::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Dummy-15912651::DEBUG::2013-11-20 03:30:37,128::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914372::DEBUG::2013-11-20 03:30:37,739::clientIF::1219::vds::(fenceNode) rc 2 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = Off
>Done
> err 
>Dummy-15912651::DEBUG::2013-11-20 03:30:37,872::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0837742 s, 12.2 MB/s\n'; <rc> = 0
>Thread-15914374::DEBUG::2013-11-20 03:30:39,602::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Dummy-15912651::DEBUG::2013-11-20 03:30:39,909::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914374::DEBUG::2013-11-20 03:30:40,326::clientIF::1219::vds::(fenceNode) rc 2 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = Off
>Done
> err 
>Thread-15914375::DEBUG::2013-11-20 03:30:40,334::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=on,secure=,options=power_wait=4
>lanplus=1)
>Dummy-15912651::DEBUG::2013-11-20 03:30:40,583::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.224642 s, 4.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:30:42,600::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914376::DEBUG::2013-11-20 03:30:42,837::clientIF::1179::vds::(waitForPid) rc 1 inp agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=on
>passwd=XXXX
>power_wait=4
>lanplus=1 out Powering on machine @ IPMI:192.168.100.119...Failed
> err 
>Dummy-15912651::DEBUG::2013-11-20 03:30:43,167::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0774949 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15914379::DEBUG::2013-11-20 03:30:44,111::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914379::INFO::2013-11-20 03:30:44,112::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914379::DEBUG::2013-11-20 03:30:44,113::task::495::TaskManager.Task::(_debug) Task 190125c1-231b-4cae-9f2b-0a976046fed2: moving from state init -> state preparing
>Thread-15914379::DEBUG::2013-11-20 03:30:44,113::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914379::DEBUG::2013-11-20 03:30:44,114::task::495::TaskManager.Task::(_debug) Task 190125c1-231b-4cae-9f2b-0a976046fed2: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914379::DEBUG::2013-11-20 03:30:44,114::task::495::TaskManager.Task::(_debug) Task 190125c1-231b-4cae-9f2b-0a976046fed2: moving from state preparing -> state finished
>Thread-15914379::DEBUG::2013-11-20 03:30:44,115::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914379::DEBUG::2013-11-20 03:30:44,115::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914379::DEBUG::2013-11-20 03:30:44,115::task::495::TaskManager.Task::(_debug) Task 190125c1-231b-4cae-9f2b-0a976046fed2: ref 0 aborting False
>Thread-15914379::INFO::2013-11-20 03:30:44,116::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914380::DEBUG::2013-11-20 03:30:44,119::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914380::INFO::2013-11-20 03:30:44,120::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914380::DEBUG::2013-11-20 03:30:44,120::task::495::TaskManager.Task::(_debug) Task 1c5215fa-1ee0-4568-8d22-81852b364a8e: moving from state init -> state preparing
>Thread-15914380::DEBUG::2013-11-20 03:30:44,121::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`444306b7-bb8a-427c-bd40-f77e9609c1d3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914380::DEBUG::2013-11-20 03:30:44,121::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914380::DEBUG::2013-11-20 03:30:44,122::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914380::DEBUG::2013-11-20 03:30:44,122::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`444306b7-bb8a-427c-bd40-f77e9609c1d3`::Granted request
>Thread-15914380::DEBUG::2013-11-20 03:30:44,123::task::495::TaskManager.Task::(_debug) Task 1c5215fa-1ee0-4568-8d22-81852b364a8e: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914380::DEBUG::2013-11-20 03:30:44,123::task::495::TaskManager.Task::(_debug) Task 1c5215fa-1ee0-4568-8d22-81852b364a8e: ref 1 aborting False
>Thread-15914380::WARNING::2013-11-20 03:30:44,124::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914380::DEBUG::2013-11-20 03:30:44,132::task::495::TaskManager.Task::(_debug) Task 1c5215fa-1ee0-4568-8d22-81852b364a8e: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914380::DEBUG::2013-11-20 03:30:44,133::task::495::TaskManager.Task::(_debug) Task 1c5215fa-1ee0-4568-8d22-81852b364a8e: moving from state preparing -> state finished
>Thread-15914380::DEBUG::2013-11-20 03:30:44,133::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914380::DEBUG::2013-11-20 03:30:44,133::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914380::DEBUG::2013-11-20 03:30:44,134::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914380::DEBUG::2013-11-20 03:30:44,134::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914380::DEBUG::2013-11-20 03:30:44,135::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914380::DEBUG::2013-11-20 03:30:44,135::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914380::DEBUG::2013-11-20 03:30:44,135::task::495::TaskManager.Task::(_debug) Task 1c5215fa-1ee0-4568-8d22-81852b364a8e: ref 0 aborting False
>Thread-15914380::INFO::2013-11-20 03:30:44,136::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:30:45,183::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914381::DEBUG::2013-11-20 03:30:45,769::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Dummy-15912651::DEBUG::2013-11-20 03:30:45,991::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0929709 s, 11.0 MB/s\n'; <rc> = 0
>Thread-15914382::INFO::2013-11-20 03:30:46,826::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914382::DEBUG::2013-11-20 03:30:46,827::task::495::TaskManager.Task::(_debug) Task f2f38615-a0fe-4984-9651-b08192d71a90: moving from state init -> state preparing
>Thread-15914382::DEBUG::2013-11-20 03:30:46,828::task::495::TaskManager.Task::(_debug) Task f2f38615-a0fe-4984-9651-b08192d71a90: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00034499168396', 'lastCheck': 1384918246.6027889, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142502784729', 'lastCheck': 1384918242.5514531, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00155091285706', 'lastCheck': 1384918244.8458891, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00145387649536', 'lastCheck': 1384918245.548317, 'valid': True, 'code': 0}}
>Thread-15914382::DEBUG::2013-11-20 03:30:46,829::task::495::TaskManager.Task::(_debug) Task f2f38615-a0fe-4984-9651-b08192d71a90: moving from state preparing -> state finished
>Thread-15914382::DEBUG::2013-11-20 03:30:46,829::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914382::DEBUG::2013-11-20 03:30:46,830::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914382::DEBUG::2013-11-20 03:30:46,830::task::495::TaskManager.Task::(_debug) Task f2f38615-a0fe-4984-9651-b08192d71a90: ref 0 aborting False
>Thread-15914382::INFO::2013-11-20 03:30:46,831::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00034499168396', 'lastCheck': 1384918246.6027889, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142502784729', 'lastCheck': 1384918242.5514531, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00155091285706', 'lastCheck': 1384918244.8458891, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00145387649536', 'lastCheck': 1384918245.548317, 'valid': True, 'code': 0}}
>Thread-15913695::INFO::2013-11-20 03:30:47,598::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:30:47,599::task::495::TaskManager.Task::(_debug) Task 9b4c94d9-6021-4d2b-9c29-4024d658b817: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:30:47,601::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`295729f3-d6cb-45d0-95e8-8ebaf0c65f54`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:30:47,602::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:30:47,602::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:30:47,603::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`295729f3-d6cb-45d0-95e8-8ebaf0c65f54`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:30:47,604::task::495::TaskManager.Task::(_debug) Task 9b4c94d9-6021-4d2b-9c29-4024d658b817: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:30:47,605::task::495::TaskManager.Task::(_debug) Task 9b4c94d9-6021-4d2b-9c29-4024d658b817: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:30:47,605::task::495::TaskManager.Task::(_debug) Task 9b4c94d9-6021-4d2b-9c29-4024d658b817: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:30:47,606::task::495::TaskManager.Task::(_debug) Task 9b4c94d9-6021-4d2b-9c29-4024d658b817: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:30:47,607::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:30:47,607::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:30:47,608::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:30:47,609::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:30:47,609::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:30:47,610::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:30:47,610::task::495::TaskManager.Task::(_debug) Task 9b4c94d9-6021-4d2b-9c29-4024d658b817: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:30:47,611::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:30:48,006::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914381::DEBUG::2013-11-20 03:30:48,330::clientIF::1219::vds::(fenceNode) rc 1 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = Unknown
>Failed
> err 
>Thread-15914384::DEBUG::2013-11-20 03:30:48,342::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Dummy-15912651::DEBUG::2013-11-20 03:30:48,748::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.318232 s, 3.2 MB/s\n'; <rc> = 0
>Thread-15914384::DEBUG::2013-11-20 03:30:48,933::clientIF::1219::vds::(fenceNode) rc 2 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = Off
>Done
> err 
>Thread-15913658::INFO::2013-11-20 03:30:49,982::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:30:49,989::task::495::TaskManager.Task::(_debug) Task d7aa541f-aca8-4317-9ac8-66bc9223dd02: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:30:49,991::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ad33fea4-c0ec-4d4b-b23e-94d483a45473`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:30:49,992::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:30:49,993::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:30:49,994::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ad33fea4-c0ec-4d4b-b23e-94d483a45473`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:30:49,995::task::495::TaskManager.Task::(_debug) Task d7aa541f-aca8-4317-9ac8-66bc9223dd02: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:30:49,996::task::495::TaskManager.Task::(_debug) Task d7aa541f-aca8-4317-9ac8-66bc9223dd02: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:30:49,997::task::495::TaskManager.Task::(_debug) Task d7aa541f-aca8-4317-9ac8-66bc9223dd02: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:30:49,998::task::495::TaskManager.Task::(_debug) Task d7aa541f-aca8-4317-9ac8-66bc9223dd02: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:30:49,998::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:30:49,999::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:30:50,000::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:30:50,000::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:30:50,001::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:30:50,002::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:30:50,002::task::495::TaskManager.Task::(_debug) Task d7aa541f-aca8-4317-9ac8-66bc9223dd02: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:30:50,003::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:30:50,764::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:30:51,331::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0813691 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 03:30:52,561::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:30:52,562::task::495::TaskManager.Task::(_debug) Task 2bb88f48-6f72-493c-b94c-d359b2614b39: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:30:52,566::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f38e380f-a5de-4a8a-a017-00b957796129`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:30:52,567::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:30:52,568::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:30:52,569::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f38e380f-a5de-4a8a-a017-00b957796129`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:30:52,570::task::495::TaskManager.Task::(_debug) Task 2bb88f48-6f72-493c-b94c-d359b2614b39: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:30:52,572::task::495::TaskManager.Task::(_debug) Task 2bb88f48-6f72-493c-b94c-d359b2614b39: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:30:52,572::task::495::TaskManager.Task::(_debug) Task 2bb88f48-6f72-493c-b94c-d359b2614b39: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:30:52,573::task::495::TaskManager.Task::(_debug) Task 2bb88f48-6f72-493c-b94c-d359b2614b39: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:30:52,574::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:30:52,574::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:30:52,575::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:30:52,576::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:30:52,576::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:30:52,577::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:30:52,577::task::495::TaskManager.Task::(_debug) Task 2bb88f48-6f72-493c-b94c-d359b2614b39: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:30:52,578::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:30:53,344::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:30:53,884::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0828897 s, 12.4 MB/s\n'; <rc> = 0
>Thread-15914387::DEBUG::2013-11-20 03:30:54,290::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914387::INFO::2013-11-20 03:30:54,292::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914387::DEBUG::2013-11-20 03:30:54,292::task::495::TaskManager.Task::(_debug) Task 0a8bd090-7dbb-4df9-ab7f-e131ebd28a8b: moving from state init -> state preparing
>Thread-15914387::DEBUG::2013-11-20 03:30:54,293::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914387::DEBUG::2013-11-20 03:30:54,294::task::495::TaskManager.Task::(_debug) Task 0a8bd090-7dbb-4df9-ab7f-e131ebd28a8b: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914387::DEBUG::2013-11-20 03:30:54,295::task::495::TaskManager.Task::(_debug) Task 0a8bd090-7dbb-4df9-ab7f-e131ebd28a8b: moving from state preparing -> state finished
>Thread-15914387::DEBUG::2013-11-20 03:30:54,295::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914387::DEBUG::2013-11-20 03:30:54,296::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914387::DEBUG::2013-11-20 03:30:54,296::task::495::TaskManager.Task::(_debug) Task 0a8bd090-7dbb-4df9-ab7f-e131ebd28a8b: ref 0 aborting False
>Thread-15914387::INFO::2013-11-20 03:30:54,297::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914388::DEBUG::2013-11-20 03:30:54,313::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914388::INFO::2013-11-20 03:30:54,314::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914388::DEBUG::2013-11-20 03:30:54,314::task::495::TaskManager.Task::(_debug) Task 7d8ea776-acee-492e-b59e-9e7cdf8339d1: moving from state init -> state preparing
>Thread-15914388::DEBUG::2013-11-20 03:30:54,316::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a3a7163e-f065-4f3e-9a67-3c6f8b6b0083`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914388::DEBUG::2013-11-20 03:30:54,317::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914388::DEBUG::2013-11-20 03:30:54,317::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914388::DEBUG::2013-11-20 03:30:54,318::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a3a7163e-f065-4f3e-9a67-3c6f8b6b0083`::Granted request
>Thread-15914388::DEBUG::2013-11-20 03:30:54,319::task::495::TaskManager.Task::(_debug) Task 7d8ea776-acee-492e-b59e-9e7cdf8339d1: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914388::DEBUG::2013-11-20 03:30:54,320::task::495::TaskManager.Task::(_debug) Task 7d8ea776-acee-492e-b59e-9e7cdf8339d1: ref 1 aborting False
>Thread-15914388::WARNING::2013-11-20 03:30:54,322::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914388::DEBUG::2013-11-20 03:30:54,323::task::495::TaskManager.Task::(_debug) Task 7d8ea776-acee-492e-b59e-9e7cdf8339d1: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914388::DEBUG::2013-11-20 03:30:54,324::task::495::TaskManager.Task::(_debug) Task 7d8ea776-acee-492e-b59e-9e7cdf8339d1: moving from state preparing -> state finished
>Thread-15914388::DEBUG::2013-11-20 03:30:54,325::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914388::DEBUG::2013-11-20 03:30:54,325::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914388::DEBUG::2013-11-20 03:30:54,326::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914388::DEBUG::2013-11-20 03:30:54,327::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914388::DEBUG::2013-11-20 03:30:54,327::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914388::DEBUG::2013-11-20 03:30:54,328::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914388::DEBUG::2013-11-20 03:30:54,329::task::495::TaskManager.Task::(_debug) Task 7d8ea776-acee-492e-b59e-9e7cdf8339d1: ref 0 aborting False
>Thread-15914388::INFO::2013-11-20 03:30:54,329::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:30:55,900::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:30:56,471::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0788241 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:30:58,485::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914390::DEBUG::2013-11-20 03:30:58,988::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Dummy-15912651::DEBUG::2013-11-20 03:30:59,315::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.330806 s, 3.1 MB/s\n'; <rc> = 0
>Thread-15914390::DEBUG::2013-11-20 03:30:59,565::clientIF::1219::vds::(fenceNode) rc 2 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = Off
>Done
> err 
>Thread-15914392::DEBUG::2013-11-20 03:30:59,571::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Thread-15914392::DEBUG::2013-11-20 03:31:00,159::clientIF::1219::vds::(fenceNode) rc 2 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = Off
>Done
> err 
>Dummy-15912651::DEBUG::2013-11-20 03:31:01,332::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:31:01,873::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0825349 s, 12.4 MB/s\n'; <rc> = 0
>Thread-15914393::INFO::2013-11-20 03:31:02,247::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914393::DEBUG::2013-11-20 03:31:02,248::task::495::TaskManager.Task::(_debug) Task a31c2986-60f9-46a7-93b1-9a05a29e52ec: moving from state init -> state preparing
>Thread-15914393::DEBUG::2013-11-20 03:31:02,249::task::495::TaskManager.Task::(_debug) Task a31c2986-60f9-46a7-93b1-9a05a29e52ec: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000333070755005', 'lastCheck': 1384918256.61498, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00151586532593', 'lastCheck': 1384918252.573221, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00178909301758', 'lastCheck': 1384918254.9975531, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000470876693726', 'lastCheck': 1384918255.563375, 'valid': True, 'code': 0}}
>Thread-15914393::DEBUG::2013-11-20 03:31:02,253::task::495::TaskManager.Task::(_debug) Task a31c2986-60f9-46a7-93b1-9a05a29e52ec: moving from state preparing -> state finished
>Thread-15914393::DEBUG::2013-11-20 03:31:02,254::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914393::DEBUG::2013-11-20 03:31:02,255::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914393::DEBUG::2013-11-20 03:31:02,255::task::495::TaskManager.Task::(_debug) Task a31c2986-60f9-46a7-93b1-9a05a29e52ec: ref 0 aborting False
>Thread-15914393::INFO::2013-11-20 03:31:02,256::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000333070755005', 'lastCheck': 1384918256.61498, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00151586532593', 'lastCheck': 1384918252.573221, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00178909301758', 'lastCheck': 1384918254.9975531, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000470876693726', 'lastCheck': 1384918255.563375, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:31:03,895::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:31:04,447::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0904328 s, 11.3 MB/s\n'; <rc> = 0
>Thread-15914395::DEBUG::2013-11-20 03:31:04,514::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914395::INFO::2013-11-20 03:31:04,515::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914395::DEBUG::2013-11-20 03:31:04,516::task::495::TaskManager.Task::(_debug) Task 76b43534-5ac5-4f0b-abcb-b56f9121e169: moving from state init -> state preparing
>Thread-15914395::DEBUG::2013-11-20 03:31:04,517::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914395::DEBUG::2013-11-20 03:31:04,517::task::495::TaskManager.Task::(_debug) Task 76b43534-5ac5-4f0b-abcb-b56f9121e169: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914395::DEBUG::2013-11-20 03:31:04,518::task::495::TaskManager.Task::(_debug) Task 76b43534-5ac5-4f0b-abcb-b56f9121e169: moving from state preparing -> state finished
>Thread-15914395::DEBUG::2013-11-20 03:31:04,519::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914395::DEBUG::2013-11-20 03:31:04,519::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914395::DEBUG::2013-11-20 03:31:04,520::task::495::TaskManager.Task::(_debug) Task 76b43534-5ac5-4f0b-abcb-b56f9121e169: ref 0 aborting False
>Thread-15914395::INFO::2013-11-20 03:31:04,520::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914396::DEBUG::2013-11-20 03:31:04,530::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914396::INFO::2013-11-20 03:31:04,531::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914396::DEBUG::2013-11-20 03:31:04,532::task::495::TaskManager.Task::(_debug) Task 8a0577ee-9c97-4aa9-ae2b-204a9feeb69c: moving from state init -> state preparing
>Thread-15914396::DEBUG::2013-11-20 03:31:04,533::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5783af83-e659-4682-9666-4d0c1d8a602b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914396::DEBUG::2013-11-20 03:31:04,534::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914396::DEBUG::2013-11-20 03:31:04,534::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914396::DEBUG::2013-11-20 03:31:04,535::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5783af83-e659-4682-9666-4d0c1d8a602b`::Granted request
>Thread-15914396::DEBUG::2013-11-20 03:31:04,536::task::495::TaskManager.Task::(_debug) Task 8a0577ee-9c97-4aa9-ae2b-204a9feeb69c: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914396::DEBUG::2013-11-20 03:31:04,537::task::495::TaskManager.Task::(_debug) Task 8a0577ee-9c97-4aa9-ae2b-204a9feeb69c: ref 1 aborting False
>Thread-15914396::WARNING::2013-11-20 03:31:04,539::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914396::DEBUG::2013-11-20 03:31:04,540::task::495::TaskManager.Task::(_debug) Task 8a0577ee-9c97-4aa9-ae2b-204a9feeb69c: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914396::DEBUG::2013-11-20 03:31:04,540::task::495::TaskManager.Task::(_debug) Task 8a0577ee-9c97-4aa9-ae2b-204a9feeb69c: moving from state preparing -> state finished
>Thread-15914396::DEBUG::2013-11-20 03:31:04,541::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914396::DEBUG::2013-11-20 03:31:04,542::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914396::DEBUG::2013-11-20 03:31:04,542::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914396::DEBUG::2013-11-20 03:31:04,543::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914396::DEBUG::2013-11-20 03:31:04,543::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914396::DEBUG::2013-11-20 03:31:04,544::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914396::DEBUG::2013-11-20 03:31:04,545::task::495::TaskManager.Task::(_debug) Task 8a0577ee-9c97-4aa9-ae2b-204a9feeb69c: ref 0 aborting False
>Thread-15914396::INFO::2013-11-20 03:31:04,545::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913518::INFO::2013-11-20 03:31:05,875::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:31:05,876::task::495::TaskManager.Task::(_debug) Task 36e4eed0-d706-4d80-82cc-5e071ddb6a33: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:31:05,878::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`82107db2-854b-4f8d-b0d4-30f8e360448a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:31:05,879::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:31:05,880::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:31:05,881::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`82107db2-854b-4f8d-b0d4-30f8e360448a`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:31:05,882::task::495::TaskManager.Task::(_debug) Task 36e4eed0-d706-4d80-82cc-5e071ddb6a33: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:31:05,883::task::495::TaskManager.Task::(_debug) Task 36e4eed0-d706-4d80-82cc-5e071ddb6a33: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:31:05,884::task::495::TaskManager.Task::(_debug) Task 36e4eed0-d706-4d80-82cc-5e071ddb6a33: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:31:05,885::task::495::TaskManager.Task::(_debug) Task 36e4eed0-d706-4d80-82cc-5e071ddb6a33: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:31:05,886::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:31:05,887::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:31:05,888::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:31:05,889::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:31:05,889::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:31:05,890::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:31:05,891::task::495::TaskManager.Task::(_debug) Task 36e4eed0-d706-4d80-82cc-5e071ddb6a33: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:31:05,892::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:31:05,893::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:31:05,894::task::495::TaskManager.Task::(_debug) Task 7ec9d58e-7625-4e53-ae30-90398a839b5d: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:31:05,895::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bff64e7c-799e-446f-922d-6b412b3bf1df`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:31:05,896::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:31:05,897::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:31:05,898::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bff64e7c-799e-446f-922d-6b412b3bf1df`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:31:05,899::task::495::TaskManager.Task::(_debug) Task 7ec9d58e-7625-4e53-ae30-90398a839b5d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:31:05,899::task::495::TaskManager.Task::(_debug) Task 7ec9d58e-7625-4e53-ae30-90398a839b5d: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:31:05,900::task::495::TaskManager.Task::(_debug) Task 7ec9d58e-7625-4e53-ae30-90398a839b5d: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:31:05,900::task::495::TaskManager.Task::(_debug) Task 7ec9d58e-7625-4e53-ae30-90398a839b5d: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:31:05,901::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:31:05,902::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:31:05,903::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:31:05,904::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:31:05,905::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:31:05,905::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:31:05,906::task::495::TaskManager.Task::(_debug) Task 7ec9d58e-7625-4e53-ae30-90398a839b5d: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:31:05,907::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:31:06,463::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:31:06,975::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0792617 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:31:08,988::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:31:09,509::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0779901 s, 13.1 MB/s\n'; <rc> = 0
>Thread-15914399::DEBUG::2013-11-20 03:31:10,212::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Thread-15914399::DEBUG::2013-11-20 03:31:10,828::clientIF::1219::vds::(fenceNode) rc 2 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = Off
>Done
> err 
>Thread-15914400::DEBUG::2013-11-20 03:31:10,835::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Thread-15914400::DEBUG::2013-11-20 03:31:11,448::clientIF::1219::vds::(fenceNode) rc 2 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = Off
>Done
> err 
>Dummy-15912651::DEBUG::2013-11-20 03:31:11,524::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:31:12,039::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0571554 s, 17.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:31:14,052::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:31:14,605::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0852773 s, 12.0 MB/s\n'; <rc> = 0
>Thread-15914402::DEBUG::2013-11-20 03:31:14,712::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914402::INFO::2013-11-20 03:31:14,714::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914402::DEBUG::2013-11-20 03:31:14,715::task::495::TaskManager.Task::(_debug) Task c88f22b0-eab2-4208-b526-77e1c3eac0d5: moving from state init -> state preparing
>Thread-15914402::DEBUG::2013-11-20 03:31:14,716::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914402::DEBUG::2013-11-20 03:31:14,717::task::495::TaskManager.Task::(_debug) Task c88f22b0-eab2-4208-b526-77e1c3eac0d5: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914402::DEBUG::2013-11-20 03:31:14,717::task::495::TaskManager.Task::(_debug) Task c88f22b0-eab2-4208-b526-77e1c3eac0d5: moving from state preparing -> state finished
>Thread-15914402::DEBUG::2013-11-20 03:31:14,718::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914402::DEBUG::2013-11-20 03:31:14,719::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914402::DEBUG::2013-11-20 03:31:14,719::task::495::TaskManager.Task::(_debug) Task c88f22b0-eab2-4208-b526-77e1c3eac0d5: ref 0 aborting False
>Thread-15914402::INFO::2013-11-20 03:31:14,720::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914403::DEBUG::2013-11-20 03:31:14,725::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914403::INFO::2013-11-20 03:31:14,726::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914403::DEBUG::2013-11-20 03:31:14,727::task::495::TaskManager.Task::(_debug) Task 07ca339b-94d5-48aa-8beb-ec8dac5ded0e: moving from state init -> state preparing
>Thread-15914403::DEBUG::2013-11-20 03:31:14,728::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b9d3b10f-f4e3-4eab-ac2b-90d61d94dadd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914403::DEBUG::2013-11-20 03:31:14,729::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914403::DEBUG::2013-11-20 03:31:14,729::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914403::DEBUG::2013-11-20 03:31:14,730::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b9d3b10f-f4e3-4eab-ac2b-90d61d94dadd`::Granted request
>Thread-15914403::DEBUG::2013-11-20 03:31:14,731::task::495::TaskManager.Task::(_debug) Task 07ca339b-94d5-48aa-8beb-ec8dac5ded0e: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914403::DEBUG::2013-11-20 03:31:14,732::task::495::TaskManager.Task::(_debug) Task 07ca339b-94d5-48aa-8beb-ec8dac5ded0e: ref 1 aborting False
>Thread-15914403::WARNING::2013-11-20 03:31:14,734::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914403::DEBUG::2013-11-20 03:31:14,741::task::495::TaskManager.Task::(_debug) Task 07ca339b-94d5-48aa-8beb-ec8dac5ded0e: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914403::DEBUG::2013-11-20 03:31:14,743::task::495::TaskManager.Task::(_debug) Task 07ca339b-94d5-48aa-8beb-ec8dac5ded0e: moving from state preparing -> state finished
>Thread-15914403::DEBUG::2013-11-20 03:31:14,744::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914403::DEBUG::2013-11-20 03:31:14,746::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914403::DEBUG::2013-11-20 03:31:14,746::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914403::DEBUG::2013-11-20 03:31:14,747::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914403::DEBUG::2013-11-20 03:31:14,748::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914403::DEBUG::2013-11-20 03:31:14,749::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914403::DEBUG::2013-11-20 03:31:14,751::task::495::TaskManager.Task::(_debug) Task 07ca339b-94d5-48aa-8beb-ec8dac5ded0e: ref 0 aborting False
>Thread-15914403::INFO::2013-11-20 03:31:14,754::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:31:16,620::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:31:17,194::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0844752 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15914405::INFO::2013-11-20 03:31:17,803::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914405::DEBUG::2013-11-20 03:31:17,804::task::495::TaskManager.Task::(_debug) Task 95d5a659-f938-4620-80f5-e6b5418c5054: moving from state init -> state preparing
>Thread-15914405::DEBUG::2013-11-20 03:31:17,805::task::495::TaskManager.Task::(_debug) Task 95d5a659-f938-4620-80f5-e6b5418c5054: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000521898269653', 'lastCheck': 1384918276.8574729, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0127351284027', 'lastCheck': 1384918272.8105929, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0097188949585', 'lastCheck': 1384918275.454607, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000421047210693', 'lastCheck': 1384918275.7921989, 'valid': True, 'code': 0}}
>Thread-15914405::DEBUG::2013-11-20 03:31:17,807::task::495::TaskManager.Task::(_debug) Task 95d5a659-f938-4620-80f5-e6b5418c5054: moving from state preparing -> state finished
>Thread-15914405::DEBUG::2013-11-20 03:31:17,815::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914405::DEBUG::2013-11-20 03:31:17,816::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914405::DEBUG::2013-11-20 03:31:17,817::task::495::TaskManager.Task::(_debug) Task 95d5a659-f938-4620-80f5-e6b5418c5054: ref 0 aborting False
>Thread-15914405::INFO::2013-11-20 03:31:17,818::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000521898269653', 'lastCheck': 1384918276.8574729, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0127351284027', 'lastCheck': 1384918272.8105929, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0097188949585', 'lastCheck': 1384918275.454607, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000421047210693', 'lastCheck': 1384918275.7921989, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:31:19,210::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:31:19,757::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0721018 s, 14.2 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 03:31:20,283::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:31:20,286::task::495::TaskManager.Task::(_debug) Task d37c81d3-a870-4c46-b416-444eead4d7ae: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:31:20,291::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`7a9facd1-543f-4b51-8b5c-14d95f2a2736`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:31:20,295::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:31:20,297::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:31:20,298::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`7a9facd1-543f-4b51-8b5c-14d95f2a2736`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:31:20,300::task::495::TaskManager.Task::(_debug) Task d37c81d3-a870-4c46-b416-444eead4d7ae: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:31:20,300::task::495::TaskManager.Task::(_debug) Task d37c81d3-a870-4c46-b416-444eead4d7ae: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:31:20,302::task::495::TaskManager.Task::(_debug) Task d37c81d3-a870-4c46-b416-444eead4d7ae: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:31:20,302::task::495::TaskManager.Task::(_debug) Task d37c81d3-a870-4c46-b416-444eead4d7ae: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:31:20,303::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:31:20,304::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:31:20,304::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:31:20,305::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:31:20,306::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:31:20,306::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:31:20,307::task::495::TaskManager.Task::(_debug) Task d37c81d3-a870-4c46-b416-444eead4d7ae: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:31:20,308::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:31:20,308::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:31:20,309::task::495::TaskManager.Task::(_debug) Task 0753ebfe-d71f-48b0-83d7-9033e66b6e20: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:31:20,310::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`3f2efdc6-c4e9-4172-acb4-f28c2ba23ee2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:31:20,311::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:31:20,311::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:31:20,312::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`3f2efdc6-c4e9-4172-acb4-f28c2ba23ee2`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:31:20,313::task::495::TaskManager.Task::(_debug) Task 0753ebfe-d71f-48b0-83d7-9033e66b6e20: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:31:20,314::task::495::TaskManager.Task::(_debug) Task 0753ebfe-d71f-48b0-83d7-9033e66b6e20: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:31:20,314::task::495::TaskManager.Task::(_debug) Task 0753ebfe-d71f-48b0-83d7-9033e66b6e20: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:31:20,315::task::495::TaskManager.Task::(_debug) Task 0753ebfe-d71f-48b0-83d7-9033e66b6e20: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:31:20,315::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:31:20,316::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:31:20,317::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:31:20,318::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:31:20,318::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:31:20,319::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:31:20,320::task::495::TaskManager.Task::(_debug) Task 0753ebfe-d71f-48b0-83d7-9033e66b6e20: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:31:20,320::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:31:20,321::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:31:20,321::task::495::TaskManager.Task::(_debug) Task f1fcc48c-9ff3-41d4-a844-16ac6b780775: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:31:20,323::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`71f003b4-6b57-4a6c-bb7a-f4f3291429c3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:31:20,323::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:31:20,324::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:31:20,325::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`71f003b4-6b57-4a6c-bb7a-f4f3291429c3`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:31:20,326::task::495::TaskManager.Task::(_debug) Task f1fcc48c-9ff3-41d4-a844-16ac6b780775: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:31:20,327::task::495::TaskManager.Task::(_debug) Task f1fcc48c-9ff3-41d4-a844-16ac6b780775: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:31:20,328::task::495::TaskManager.Task::(_debug) Task f1fcc48c-9ff3-41d4-a844-16ac6b780775: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:31:20,329::task::495::TaskManager.Task::(_debug) Task f1fcc48c-9ff3-41d4-a844-16ac6b780775: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:31:20,330::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:31:20,331::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:31:20,331::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:31:20,332::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:31:20,333::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:31:20,333::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:31:20,334::task::495::TaskManager.Task::(_debug) Task f1fcc48c-9ff3-41d4-a844-16ac6b780775: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:31:20,335::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:31:20,335::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:31:20,336::task::495::TaskManager.Task::(_debug) Task 7d2672b2-64a3-4dd6-b31c-f7a1f406be3a: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:31:20,337::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a9af02ab-a1a0-4ee1-9386-530ab174f8d7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:31:20,338::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:31:20,339::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:31:20,339::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a9af02ab-a1a0-4ee1-9386-530ab174f8d7`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:31:20,340::task::495::TaskManager.Task::(_debug) Task 7d2672b2-64a3-4dd6-b31c-f7a1f406be3a: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:31:20,340::task::495::TaskManager.Task::(_debug) Task 7d2672b2-64a3-4dd6-b31c-f7a1f406be3a: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:31:20,341::task::495::TaskManager.Task::(_debug) Task 7d2672b2-64a3-4dd6-b31c-f7a1f406be3a: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:31:20,342::task::495::TaskManager.Task::(_debug) Task 7d2672b2-64a3-4dd6-b31c-f7a1f406be3a: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:31:20,342::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:31:20,343::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:31:20,343::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:31:20,344::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:31:20,344::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:31:20,345::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:31:20,346::task::495::TaskManager.Task::(_debug) Task 7d2672b2-64a3-4dd6-b31c-f7a1f406be3a: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:31:20,346::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15914408::DEBUG::2013-11-20 03:31:21,523::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Dummy-15912651::DEBUG::2013-11-20 03:31:21,779::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914408::DEBUG::2013-11-20 03:31:22,275::clientIF::1219::vds::(fenceNode) rc 2 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = Off
>Done
> err 
>Thread-15914409::DEBUG::2013-11-20 03:31:22,282::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Dummy-15912651::DEBUG::2013-11-20 03:31:22,594::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.283409 s, 3.6 MB/s\n'; <rc> = 0
>Thread-15914409::DEBUG::2013-11-20 03:31:22,893::clientIF::1219::vds::(fenceNode) rc 2 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = Off
>Done
> err 
>Dummy-15912651::DEBUG::2013-11-20 03:31:24,612::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914411::DEBUG::2013-11-20 03:31:24,867::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914411::INFO::2013-11-20 03:31:24,869::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914411::DEBUG::2013-11-20 03:31:24,869::task::495::TaskManager.Task::(_debug) Task c3667b52-cf0b-40dd-be6f-a5fbf93ea9f5: moving from state init -> state preparing
>Thread-15914411::DEBUG::2013-11-20 03:31:24,870::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914411::DEBUG::2013-11-20 03:31:24,871::task::495::TaskManager.Task::(_debug) Task c3667b52-cf0b-40dd-be6f-a5fbf93ea9f5: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914411::DEBUG::2013-11-20 03:31:24,871::task::495::TaskManager.Task::(_debug) Task c3667b52-cf0b-40dd-be6f-a5fbf93ea9f5: moving from state preparing -> state finished
>Thread-15914411::DEBUG::2013-11-20 03:31:24,872::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914411::DEBUG::2013-11-20 03:31:24,872::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914411::DEBUG::2013-11-20 03:31:24,873::task::495::TaskManager.Task::(_debug) Task c3667b52-cf0b-40dd-be6f-a5fbf93ea9f5: ref 0 aborting False
>Thread-15914411::INFO::2013-11-20 03:31:24,874::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914412::DEBUG::2013-11-20 03:31:24,879::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914412::INFO::2013-11-20 03:31:24,880::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914412::DEBUG::2013-11-20 03:31:24,880::task::495::TaskManager.Task::(_debug) Task a697726f-7163-4ace-a82d-7f402f9607cb: moving from state init -> state preparing
>Thread-15914412::DEBUG::2013-11-20 03:31:24,882::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6bc2d0a8-ab0f-4a09-abc7-9754e3c5bd64`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914412::DEBUG::2013-11-20 03:31:24,883::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914412::DEBUG::2013-11-20 03:31:24,884::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914412::DEBUG::2013-11-20 03:31:24,884::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6bc2d0a8-ab0f-4a09-abc7-9754e3c5bd64`::Granted request
>Thread-15914412::DEBUG::2013-11-20 03:31:24,885::task::495::TaskManager.Task::(_debug) Task a697726f-7163-4ace-a82d-7f402f9607cb: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914412::DEBUG::2013-11-20 03:31:24,886::task::495::TaskManager.Task::(_debug) Task a697726f-7163-4ace-a82d-7f402f9607cb: ref 1 aborting False
>Thread-15914412::WARNING::2013-11-20 03:31:24,888::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914412::DEBUG::2013-11-20 03:31:24,889::task::495::TaskManager.Task::(_debug) Task a697726f-7163-4ace-a82d-7f402f9607cb: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914412::DEBUG::2013-11-20 03:31:24,890::task::495::TaskManager.Task::(_debug) Task a697726f-7163-4ace-a82d-7f402f9607cb: moving from state preparing -> state finished
>Thread-15914412::DEBUG::2013-11-20 03:31:24,891::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914412::DEBUG::2013-11-20 03:31:24,892::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914412::DEBUG::2013-11-20 03:31:24,892::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914412::DEBUG::2013-11-20 03:31:24,893::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914412::DEBUG::2013-11-20 03:31:24,894::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914412::DEBUG::2013-11-20 03:31:24,894::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914412::DEBUG::2013-11-20 03:31:24,895::task::495::TaskManager.Task::(_debug) Task a697726f-7163-4ace-a82d-7f402f9607cb: ref 0 aborting False
>Thread-15914412::INFO::2013-11-20 03:31:24,896::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:31:25,140::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0860026 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:31:27,157::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:31:27,651::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0806644 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:31:29,667::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:31:30,226::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0848797 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15913639::INFO::2013-11-20 03:31:30,479::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:31:30,490::task::495::TaskManager.Task::(_debug) Task 83c728ab-7e58-4c9f-9965-abe38c84cd94: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:31:30,493::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`da5d4a27-ed0d-4ab1-95a4-03435da54d65`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:31:30,494::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:31:30,495::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:31:30,496::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`da5d4a27-ed0d-4ab1-95a4-03435da54d65`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:31:30,497::task::495::TaskManager.Task::(_debug) Task 83c728ab-7e58-4c9f-9965-abe38c84cd94: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:31:30,497::task::495::TaskManager.Task::(_debug) Task 83c728ab-7e58-4c9f-9965-abe38c84cd94: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:31:30,498::task::495::TaskManager.Task::(_debug) Task 83c728ab-7e58-4c9f-9965-abe38c84cd94: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:31:30,499::task::495::TaskManager.Task::(_debug) Task 83c728ab-7e58-4c9f-9965-abe38c84cd94: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:31:30,500::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:31:30,500::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:31:30,501::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:31:30,502::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:31:30,502::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:31:30,503::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:31:30,503::task::495::TaskManager.Task::(_debug) Task 83c728ab-7e58-4c9f-9965-abe38c84cd94: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:31:30,504::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:31:32,240::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:31:32,700::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0622011 s, 16.5 MB/s\n'; <rc> = 0
>Thread-15914415::DEBUG::2013-11-20 03:31:32,928::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Thread-15914416::INFO::2013-11-20 03:31:33,134::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914416::DEBUG::2013-11-20 03:31:33,149::task::495::TaskManager.Task::(_debug) Task c8191942-d32d-4074-938d-c3c4c2f5db43: moving from state init -> state preparing
>Thread-15914416::DEBUG::2013-11-20 03:31:33,150::task::495::TaskManager.Task::(_debug) Task c8191942-d32d-4074-938d-c3c4c2f5db43: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000313997268677', 'lastCheck': 1384918287.065321, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00115394592285', 'lastCheck': 1384918293.1484239, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00141477584839', 'lastCheck': 1384918285.4756269, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000412940979004', 'lastCheck': 1384918286.056818, 'valid': True, 'code': 0}}
>Thread-15914416::DEBUG::2013-11-20 03:31:33,150::task::495::TaskManager.Task::(_debug) Task c8191942-d32d-4074-938d-c3c4c2f5db43: moving from state preparing -> state finished
>Thread-15914416::DEBUG::2013-11-20 03:31:33,151::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914416::DEBUG::2013-11-20 03:31:33,151::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914416::DEBUG::2013-11-20 03:31:33,151::task::495::TaskManager.Task::(_debug) Task c8191942-d32d-4074-938d-c3c4c2f5db43: ref 0 aborting False
>Thread-15914416::INFO::2013-11-20 03:31:33,152::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000313997268677', 'lastCheck': 1384918287.065321, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00115394592285', 'lastCheck': 1384918293.1484239, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00141477584839', 'lastCheck': 1384918285.4756269, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000412940979004', 'lastCheck': 1384918286.056818, 'valid': True, 'code': 0}}
>Thread-15914415::DEBUG::2013-11-20 03:31:33,479::clientIF::1219::vds::(fenceNode) rc 2 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = Off
>Done
> err 
>Thread-15914418::DEBUG::2013-11-20 03:31:33,485::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Thread-15914418::DEBUG::2013-11-20 03:31:34,088::clientIF::1219::vds::(fenceNode) rc 2 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = Off
>Done
> err 
>Dummy-15912651::DEBUG::2013-11-20 03:31:34,715::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914419::DEBUG::2013-11-20 03:31:35,033::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914419::INFO::2013-11-20 03:31:35,035::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914419::DEBUG::2013-11-20 03:31:35,037::task::495::TaskManager.Task::(_debug) Task 5dd916a2-66af-4603-9b68-3a3e8e6ebfad: moving from state init -> state preparing
>Thread-15914419::DEBUG::2013-11-20 03:31:35,039::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914419::DEBUG::2013-11-20 03:31:35,044::task::495::TaskManager.Task::(_debug) Task 5dd916a2-66af-4603-9b68-3a3e8e6ebfad: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914419::DEBUG::2013-11-20 03:31:35,048::task::495::TaskManager.Task::(_debug) Task 5dd916a2-66af-4603-9b68-3a3e8e6ebfad: moving from state preparing -> state finished
>Thread-15914419::DEBUG::2013-11-20 03:31:35,053::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914419::DEBUG::2013-11-20 03:31:35,055::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914419::DEBUG::2013-11-20 03:31:35,056::task::495::TaskManager.Task::(_debug) Task 5dd916a2-66af-4603-9b68-3a3e8e6ebfad: ref 0 aborting False
>Thread-15914419::INFO::2013-11-20 03:31:35,056::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914420::DEBUG::2013-11-20 03:31:35,062::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914420::INFO::2013-11-20 03:31:35,063::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914420::DEBUG::2013-11-20 03:31:35,063::task::495::TaskManager.Task::(_debug) Task 1eac4fe0-7fd8-44d3-9a52-5977605e4eda: moving from state init -> state preparing
>Thread-15914420::DEBUG::2013-11-20 03:31:35,065::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`22721561-ca8b-40ce-8abe-aab501dd37a6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914420::DEBUG::2013-11-20 03:31:35,065::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914420::DEBUG::2013-11-20 03:31:35,066::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914420::DEBUG::2013-11-20 03:31:35,067::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`22721561-ca8b-40ce-8abe-aab501dd37a6`::Granted request
>Thread-15914420::DEBUG::2013-11-20 03:31:35,068::task::495::TaskManager.Task::(_debug) Task 1eac4fe0-7fd8-44d3-9a52-5977605e4eda: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914420::DEBUG::2013-11-20 03:31:35,069::task::495::TaskManager.Task::(_debug) Task 1eac4fe0-7fd8-44d3-9a52-5977605e4eda: ref 1 aborting False
>Thread-15914420::WARNING::2013-11-20 03:31:35,071::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914420::DEBUG::2013-11-20 03:31:35,073::task::495::TaskManager.Task::(_debug) Task 1eac4fe0-7fd8-44d3-9a52-5977605e4eda: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914420::DEBUG::2013-11-20 03:31:35,074::task::495::TaskManager.Task::(_debug) Task 1eac4fe0-7fd8-44d3-9a52-5977605e4eda: moving from state preparing -> state finished
>Thread-15914420::DEBUG::2013-11-20 03:31:35,074::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914420::DEBUG::2013-11-20 03:31:35,075::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914420::DEBUG::2013-11-20 03:31:35,076::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914420::DEBUG::2013-11-20 03:31:35,076::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914420::DEBUG::2013-11-20 03:31:35,077::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914420::DEBUG::2013-11-20 03:31:35,078::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914420::DEBUG::2013-11-20 03:31:35,078::task::495::TaskManager.Task::(_debug) Task 1eac4fe0-7fd8-44d3-9a52-5977605e4eda: ref 0 aborting False
>Thread-15914420::INFO::2013-11-20 03:31:35,079::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115215433728', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:31:35,216::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0540513 s, 18.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:31:37,227::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:31:37,771::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0746346 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:31:39,787::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:31:40,331::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0814821 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15914423::DEBUG::2013-11-20 03:31:42,341::clientIF::54::vds::(wrapper) [192.168.100.135]::call create with ({'bridge': 'rhevm', 'acpiEnable': 'true', 'emulatedMachine': 'rhel6.2.0', 'vmId': '5f587d5c-d4b4-4242-b049-ede331c7f8b4', 'transparentHugePages': 'true', 'spiceSslCipherSuite': 'DEFAULT', 'cpuType': 'Westmere', 'smp': '2', 'macAddr': '00:1a:4a:a8:64:17', 'boot': 'd', 'custom': {}, 'vmType': 'kvm', 'memSize': 2048, 'smpCoresPerSocket': '1', 'vmName': 'Taipei', 'spiceMonitors': '1', 'nice': '0', 'drives': [{'index': '0', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'format': 'cow', 'boot': 'true', 'volumeID': 'd7028f7c-0551-4bbe-a907-6f804cc9bd54', 'imageID': '143cb26d-de08-4d30-89cf-aee10245b66a', 'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'propagateErrors': 'off', 'if': 'virtio'}], 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO', 'smartcardEnable': 'false', 'nicModel': 'pv', 'keyboardLayout': 'en-us', 'kvmEnable': 'true', 'timeOffset': '25201', 'spiceSecureChannels': 'smain,sinputs', 'display': 'qxl'},) {}
>Dummy-15912651::DEBUG::2013-11-20 03:31:42,538::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914423::INFO::2013-11-20 03:31:42,872::clientIF::523::vds::(_getNetworkIp) network None: using 0
>Thread-15914423::INFO::2013-11-20 03:31:42,874::clientIF::656::vds::(create) vmContainerLock acquired by vm 5f587d5c-d4b4-4242-b049-ede331c7f8b4
>Thread-15914424::DEBUG::2013-11-20 03:31:42,881::vm::342::vm.Vm::(_startUnderlyingVm) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Start
>Thread-15914423::DEBUG::2013-11-20 03:31:42,881::clientIF::669::vds::(create) Total desktops after creation of 5f587d5c-d4b4-4242-b049-ede331c7f8b4 is 7
>Thread-15914424::DEBUG::2013-11-20 03:31:42,882::vm::346::vm.Vm::(_startUnderlyingVm) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::_ongoingCreations acquired
>Thread-15914423::DEBUG::2013-11-20 03:31:42,883::clientIF::59::vds::(wrapper) return create with {'status': {'message': 'Done', 'code': 0}, 'vmList': {'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO', 'bridge': 'rhevm', 'acpiEnable': 'true', 'emulatedMachine': 'rhel6.2.0', 'vmId': '5f587d5c-d4b4-4242-b049-ede331c7f8b4', 'pid': '0', 'drives': [{'index': '0', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'format': 'cow', 'boot': 'true', 'volumeID': 'd7028f7c-0551-4bbe-a907-6f804cc9bd54', 'imageID': '143cb26d-de08-4d30-89cf-aee10245b66a', 'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'propagateErrors': 'off', 'if': 'virtio'}], 'transparentHugePages': 'true', 'keyboardLayout': 'en-us', 'displaySecurePort': '-1', 'displayPort': '-1', 'cpuType': 'Westmere', 'custom': {}, 'clientIp': '', 'nicModel': 'pv', 'macAddr': '00:1a:4a:a8:64:17', 'kvmEnable': 'true', 'smartcardEnable': 'false', 'boot': 'd', 'smp': '2', 'vmType': 'kvm', 'spiceSslCipherSuite': 'DEFAULT', 'status': 'WaitForLaunch', 'timeOffset': '25201', 'memSize': 2048, 'displayIp': '0', 'spiceSecureChannels': 'smain,sinputs', 'smpCoresPerSocket': '1', 'vmName': 'Taipei', 'display': 'qxl', 'spiceMonitors': '1', 'nice': '0'}}
>Thread-15914424::INFO::2013-11-20 03:31:42,884::libvirtvm::1061::vm.Vm::(_run) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::VM wrapper has started
>Thread-15914424::INFO::2013-11-20 03:31:42,887::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914424::DEBUG::2013-11-20 03:31:42,888::task::495::TaskManager.Task::(_debug) Task 5519e9ba-e7d4-4d63-9d74-6e4597e59b92: moving from state init -> state preparing
>Thread-15914424::DEBUG::2013-11-20 03:31:42,891::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d49132b5-dd02-4912-9add-49eba2780a32`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1690' at 'public_prepareVolume'
>Thread-15914424::DEBUG::2013-11-20 03:31:42,892::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914424::DEBUG::2013-11-20 03:31:42,895::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914424::DEBUG::2013-11-20 03:31:42,895::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d49132b5-dd02-4912-9add-49eba2780a32`::Granted request
>Thread-15914424::DEBUG::2013-11-20 03:31:42,896::task::495::TaskManager.Task::(_debug) Task 5519e9ba-e7d4-4d63-9d74-6e4597e59b92: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914424::DEBUG::2013-11-20 03:31:42,897::task::495::TaskManager.Task::(_debug) Task 5519e9ba-e7d4-4d63-9d74-6e4597e59b92: ref 1 aborting False
>Thread-15914424::DEBUG::2013-11-20 03:31:42,898::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.143cb26d-de08-4d30-89cf-aee10245b66a`ReqID=`5ecd9ec3-f997-422a-9d33-40c5ce7f3568`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1695' at 'public_prepareVolume'
>Thread-15914424::DEBUG::2013-11-20 03:31:42,899::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.143cb26d-de08-4d30-89cf-aee10245b66a' for lock type 'exclusive'
>Thread-15914424::DEBUG::2013-11-20 03:31:42,900::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15914424::DEBUG::2013-11-20 03:31:42,902::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:31:43,397::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.166475 s, 6.2 MB/s\n'; <rc> = 0
>Thread-15914424::DEBUG::2013-11-20 03:31:44,098::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15914427::DEBUG::2013-11-20 03:31:44,348::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Thread-15914424::DEBUG::2013-11-20 03:31:44,350::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15914424::DEBUG::2013-11-20 03:31:44,562::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=14 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15914424::DEBUG::2013-11-20 03:31:45,105::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000177243 s, 2.9 MB/s\n'; <rc> = 0
>Thread-15914424::DEBUG::2013-11-20 03:31:45,106::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000177243 s, 2.9 MB/s'], size: 512
>Thread-15914424::DEBUG::2013-11-20 03:31:45,107::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=14 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15914427::DEBUG::2013-11-20 03:31:45,324::clientIF::1219::vds::(fenceNode) rc 2 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = Off
>Done
> err 
>Thread-15914428::DEBUG::2013-11-20 03:31:45,332::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914428::INFO::2013-11-20 03:31:45,334::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914428::DEBUG::2013-11-20 03:31:45,335::task::495::TaskManager.Task::(_debug) Task 6d58f57e-09a8-4031-9e58-d9ad3e7a7c54: moving from state init -> state preparing
>Thread-15914428::DEBUG::2013-11-20 03:31:45,342::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914429::DEBUG::2013-11-20 03:31:45,357::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Thread-15914428::DEBUG::2013-11-20 03:31:45,358::task::495::TaskManager.Task::(_debug) Task 6d58f57e-09a8-4031-9e58-d9ad3e7a7c54: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914428::DEBUG::2013-11-20 03:31:45,360::task::495::TaskManager.Task::(_debug) Task 6d58f57e-09a8-4031-9e58-d9ad3e7a7c54: moving from state preparing -> state finished
>Dummy-15912651::DEBUG::2013-11-20 03:31:45,675::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914428::DEBUG::2013-11-20 03:31:45,693::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914428::DEBUG::2013-11-20 03:31:45,923::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914428::DEBUG::2013-11-20 03:31:45,931::task::495::TaskManager.Task::(_debug) Task 6d58f57e-09a8-4031-9e58-d9ad3e7a7c54: ref 0 aborting False
>Thread-15914428::INFO::2013-11-20 03:31:45,933::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914424::DEBUG::2013-11-20 03:31:45,935::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000194874 s, 2.6 MB/s\n'; <rc> = 0
>Thread-15914424::DEBUG::2013-11-20 03:31:45,941::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000194874 s, 2.6 MB/s'], size: 512
>Thread-15914424::INFO::2013-11-20 03:31:45,942::image::340::Storage.Image::(getChain) sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a chain=[<storage.blockVolume.BlockVolume instance at 0x7f48ac1a7638>] 
>Thread-15914424::DEBUG::2013-11-20 03:31:45,944::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --autobackup n --available y 6b892067-ac7e-4039-a7be-5b7387525b16/d7028f7c-0551-4bbe-a907-6f804cc9bd54' (cwd None)
>Thread-15914429::DEBUG::2013-11-20 03:31:46,167::clientIF::1219::vds::(fenceNode) rc 2 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = Off
>Done
> err 
>Thread-15914430::DEBUG::2013-11-20 03:31:46,181::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914430::INFO::2013-11-20 03:31:46,184::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914430::DEBUG::2013-11-20 03:31:46,185::task::495::TaskManager.Task::(_debug) Task 9341f498-0d8f-4ef2-a20c-5a80d1a31c4b: moving from state init -> state preparing
>Thread-15914430::DEBUG::2013-11-20 03:31:46,193::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a4145c34-75c9-47b4-a46a-0a1e74d2f39c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914430::DEBUG::2013-11-20 03:31:46,194::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914430::DEBUG::2013-11-20 03:31:46,196::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914430::DEBUG::2013-11-20 03:31:46,197::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a4145c34-75c9-47b4-a46a-0a1e74d2f39c`::Granted request
>Thread-15914430::DEBUG::2013-11-20 03:31:46,199::task::495::TaskManager.Task::(_debug) Task 9341f498-0d8f-4ef2-a20c-5a80d1a31c4b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914430::DEBUG::2013-11-20 03:31:46,200::task::495::TaskManager.Task::(_debug) Task 9341f498-0d8f-4ef2-a20c-5a80d1a31c4b: ref 1 aborting False
>Thread-15914430::WARNING::2013-11-20 03:31:46,204::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914430::DEBUG::2013-11-20 03:31:46,206::task::495::TaskManager.Task::(_debug) Task 9341f498-0d8f-4ef2-a20c-5a80d1a31c4b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914430::DEBUG::2013-11-20 03:31:46,207::task::495::TaskManager.Task::(_debug) Task 9341f498-0d8f-4ef2-a20c-5a80d1a31c4b: moving from state preparing -> state finished
>Thread-15914430::DEBUG::2013-11-20 03:31:46,208::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914430::DEBUG::2013-11-20 03:31:46,209::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914430::DEBUG::2013-11-20 03:31:46,210::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914430::DEBUG::2013-11-20 03:31:46,212::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914430::DEBUG::2013-11-20 03:31:46,212::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914430::DEBUG::2013-11-20 03:31:46,214::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914430::DEBUG::2013-11-20 03:31:46,215::task::495::TaskManager.Task::(_debug) Task 9341f498-0d8f-4ef2-a20c-5a80d1a31c4b: ref 0 aborting False
>Thread-15914430::INFO::2013-11-20 03:31:46,216::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:31:46,359::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.18917 s, 5.4 MB/s\n'; <rc> = 0
>Thread-15914424::DEBUG::2013-11-20 03:31:46,831::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15914424::DEBUG::2013-11-20 03:31:46,840::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15914424::DEBUG::2013-11-20 03:31:46,841::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15914424::DEBUG::2013-11-20 03:31:46,842::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.d7028f7c-0551-4bbe-a907-6f804cc9bd54`ReqID=`e45cc610-c6bf-4fd3-a194-bc5cb0578e3d`::Request was made in '/usr/share/vdsm/storage/resourceFactories.py' line '149' at '__getResourceCandidatesList'
>Thread-15914424::DEBUG::2013-11-20 03:31:46,842::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.d7028f7c-0551-4bbe-a907-6f804cc9bd54' for lock type 'exclusive'
>Thread-15914424::DEBUG::2013-11-20 03:31:46,843::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.d7028f7c-0551-4bbe-a907-6f804cc9bd54' is free. Now locking as 'exclusive' (1 active user)
>Thread-15914424::DEBUG::2013-11-20 03:31:46,844::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.d7028f7c-0551-4bbe-a907-6f804cc9bd54`ReqID=`e45cc610-c6bf-4fd3-a194-bc5cb0578e3d`::Granted request
>Thread-15914424::DEBUG::2013-11-20 03:31:46,845::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.143cb26d-de08-4d30-89cf-aee10245b66a' is free. Now locking as 'exclusive' (1 active user)
>Thread-15914424::DEBUG::2013-11-20 03:31:46,845::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.143cb26d-de08-4d30-89cf-aee10245b66a`ReqID=`5ecd9ec3-f997-422a-9d33-40c5ce7f3568`::Granted request
>Thread-15914424::DEBUG::2013-11-20 03:31:46,847::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15914424::DEBUG::2013-11-20 03:31:46,848::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15914424::DEBUG::2013-11-20 03:31:47,862::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15914424::DEBUG::2013-11-20 03:31:48,141::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15914424::DEBUG::2013-11-20 03:31:48,143::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --refresh 6b892067-ac7e-4039-a7be-5b7387525b16/d7028f7c-0551-4bbe-a907-6f804cc9bd54' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:31:48,372::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 03:31:48,724::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:31:48,727::task::495::TaskManager.Task::(_debug) Task 52c4adbb-7281-45db-993d-60ba4e323a68: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:31:48,729::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c13c804b-3df4-473d-84c4-80a252739090`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:31:48,730::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:31:48,732::resourceManager::481::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' found in shared state and queue is empty, Joining current shared lock (2 active users)
>Thread-15913695::DEBUG::2013-11-20 03:31:48,732::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c13c804b-3df4-473d-84c4-80a252739090`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:31:48,734::task::495::TaskManager.Task::(_debug) Task 52c4adbb-7281-45db-993d-60ba4e323a68: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:31:48,735::task::495::TaskManager.Task::(_debug) Task 52c4adbb-7281-45db-993d-60ba4e323a68: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:31:48,736::task::495::TaskManager.Task::(_debug) Task 52c4adbb-7281-45db-993d-60ba4e323a68: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:31:48,736::task::495::TaskManager.Task::(_debug) Task 52c4adbb-7281-45db-993d-60ba4e323a68: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:31:48,737::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:31:48,738::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:31:48,738::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:31:48,739::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (1 active users)
>Thread-15913695::DEBUG::2013-11-20 03:31:48,740::task::495::TaskManager.Task::(_debug) Task 52c4adbb-7281-45db-993d-60ba4e323a68: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:31:48,741::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:31:49,023::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0691001 s, 14.8 MB/s\n'; <rc> = 0
>Thread-15914424::DEBUG::2013-11-20 03:31:49,084::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15914424::DEBUG::2013-11-20 03:31:49,085::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15914424::DEBUG::2013-11-20 03:31:49,086::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15914424::INFO::2013-11-20 03:31:49,087::volume::517::Storage.Volume::(prepare) Volume: preparing volume 6b892067-ac7e-4039-a7be-5b7387525b16/d7028f7c-0551-4bbe-a907-6f804cc9bd54
>Thread-15914424::DEBUG::2013-11-20 03:31:49,087::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15914424::DEBUG::2013-11-20 03:31:49,089::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15914432::INFO::2013-11-20 03:31:49,355::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914432::DEBUG::2013-11-20 03:31:49,356::task::495::TaskManager.Task::(_debug) Task fee4631b-ee4c-46ec-9458-437a6bf8290a: moving from state init -> state preparing
>Thread-15914432::DEBUG::2013-11-20 03:31:49,357::task::495::TaskManager.Task::(_debug) Task fee4631b-ee4c-46ec-9458-437a6bf8290a: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000355005264282', 'lastCheck': 1384918307.326561, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00123500823975', 'lastCheck': 1384918303.2946169, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.232423067093', 'lastCheck': 1384918305.928488, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000983953475952', 'lastCheck': 1384918306.324789, 'valid': True, 'code': 0}}
>Thread-15914432::DEBUG::2013-11-20 03:31:49,358::task::495::TaskManager.Task::(_debug) Task fee4631b-ee4c-46ec-9458-437a6bf8290a: moving from state preparing -> state finished
>Thread-15914432::DEBUG::2013-11-20 03:31:49,359::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914432::DEBUG::2013-11-20 03:31:49,360::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914432::DEBUG::2013-11-20 03:31:49,360::task::495::TaskManager.Task::(_debug) Task fee4631b-ee4c-46ec-9458-437a6bf8290a: ref 0 aborting False
>Thread-15914432::INFO::2013-11-20 03:31:49,361::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000355005264282', 'lastCheck': 1384918307.326561, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00123500823975', 'lastCheck': 1384918303.2946169, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.232423067093', 'lastCheck': 1384918305.928488, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000983953475952', 'lastCheck': 1384918306.324789, 'valid': True, 'code': 0}}
>Thread-15914424::DEBUG::2013-11-20 03:31:50,045::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15914424::DEBUG::2013-11-20 03:31:50,318::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15914424::DEBUG::2013-11-20 03:31:50,319::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=14 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15914424::DEBUG::2013-11-20 03:31:50,687::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000160435 s, 3.2 MB/s\n'; <rc> = 0
>Thread-15914424::DEBUG::2013-11-20 03:31:50,689::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000160435 s, 3.2 MB/s'], size: 512
>Thread-15914424::DEBUG::2013-11-20 03:31:50,702::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=14 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:31:51,045::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 03:31:51,046::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:31:51,049::task::495::TaskManager.Task::(_debug) Task 1a25a783-755d-4037-b177-c6ff2f517a99: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:31:51,276::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4ec2bfd8-afc0-4d2a-b8a1-c000ce9793b4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:31:51,280::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:31:51,285::resourceManager::481::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' found in shared state and queue is empty, Joining current shared lock (2 active users)
>Thread-15914424::DEBUG::2013-11-20 03:31:51,285::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000242659 s, 2.1 MB/s\n'; <rc> = 0
>Thread-15913658::DEBUG::2013-11-20 03:31:51,286::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4ec2bfd8-afc0-4d2a-b8a1-c000ce9793b4`::Granted request
>Thread-15914424::DEBUG::2013-11-20 03:31:51,288::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000242659 s, 2.1 MB/s'], size: 512
>Thread-15914424::DEBUG::2013-11-20 03:31:51,291::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.d7028f7c-0551-4bbe-a907-6f804cc9bd54`ReqID=`238d00d7-7ec0-4531-9380-600f51363cd6`::Request was made in '/usr/share/vdsm/storage/volume.py' line '533' at 'prepare'
>Thread-15914424::DEBUG::2013-11-20 03:31:51,292::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.d7028f7c-0551-4bbe-a907-6f804cc9bd54' for lock type 'exclusive'
>Thread-15913658::DEBUG::2013-11-20 03:31:51,293::task::495::TaskManager.Task::(_debug) Task 1a25a783-755d-4037-b177-c6ff2f517a99: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:31:51,295::task::495::TaskManager.Task::(_debug) Task 1a25a783-755d-4037-b177-c6ff2f517a99: ref 1 aborting False
>Thread-15914424::DEBUG::2013-11-20 03:31:51,295::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.d7028f7c-0551-4bbe-a907-6f804cc9bd54' is free. Now locking as 'exclusive' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:31:51,296::task::495::TaskManager.Task::(_debug) Task 1a25a783-755d-4037-b177-c6ff2f517a99: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15914424::DEBUG::2013-11-20 03:31:51,297::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.d7028f7c-0551-4bbe-a907-6f804cc9bd54`ReqID=`238d00d7-7ec0-4531-9380-600f51363cd6`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:31:51,298::task::495::TaskManager.Task::(_debug) Task 1a25a783-755d-4037-b177-c6ff2f517a99: moving from state preparing -> state finished
>Thread-15914424::DEBUG::2013-11-20 03:31:51,300::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=14 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15913658::DEBUG::2013-11-20 03:31:51,301::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:31:51,302::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:31:51,628::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:31:51,632::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (1 active users)
>Thread-15913658::DEBUG::2013-11-20 03:31:51,634::task::495::TaskManager.Task::(_debug) Task 1a25a783-755d-4037-b177-c6ff2f517a99: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:31:51,635::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:31:51,795::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.165426 s, 6.2 MB/s\n'; <rc> = 0
>Thread-15914424::DEBUG::2013-11-20 03:31:51,835::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000196598 s, 2.6 MB/s\n'; <rc> = 0
>Thread-15914424::DEBUG::2013-11-20 03:31:51,837::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000196598 s, 2.6 MB/s'], size: 512
>Thread-15914424::DEBUG::2013-11-20 03:31:51,839::task::495::TaskManager.Task::(_debug) Task 5519e9ba-e7d4-4d63-9d74-6e4597e59b92: finished: None
>Thread-15914424::DEBUG::2013-11-20 03:31:51,839::task::495::TaskManager.Task::(_debug) Task 5519e9ba-e7d4-4d63-9d74-6e4597e59b92: moving from state preparing -> state finished
>Thread-15914424::DEBUG::2013-11-20 03:31:51,840::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914424::DEBUG::2013-11-20 03:31:51,841::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914424::DEBUG::2013-11-20 03:31:51,841::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914424::DEBUG::2013-11-20 03:31:51,842::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914424::DEBUG::2013-11-20 03:31:51,843::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914424::DEBUG::2013-11-20 03:31:51,843::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914424::DEBUG::2013-11-20 03:31:51,844::task::495::TaskManager.Task::(_debug) Task 5519e9ba-e7d4-4d63-9d74-6e4597e59b92: ref 0 aborting False
>Thread-15914424::INFO::2013-11-20 03:31:51,845::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, Return response: {'status': {'message': 'OK', 'code': 0}}
>Thread-15914424::INFO::2013-11-20 03:31:51,846::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914424::DEBUG::2013-11-20 03:31:51,846::task::495::TaskManager.Task::(_debug) Task ef558dc3-c146-4080-9a5a-c1b22ffcadba: moving from state init -> state preparing
>Thread-15914424::DEBUG::2013-11-20 03:31:51,848::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c39f7d48-4e4d-4383-be57-13f0011ac095`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1666' at 'public_getVolumePath'
>Thread-15914424::DEBUG::2013-11-20 03:31:51,848::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914424::DEBUG::2013-11-20 03:31:51,849::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914424::DEBUG::2013-11-20 03:31:51,850::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c39f7d48-4e4d-4383-be57-13f0011ac095`::Granted request
>Thread-15914424::DEBUG::2013-11-20 03:31:51,851::task::495::TaskManager.Task::(_debug) Task ef558dc3-c146-4080-9a5a-c1b22ffcadba: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914424::DEBUG::2013-11-20 03:31:51,851::task::495::TaskManager.Task::(_debug) Task ef558dc3-c146-4080-9a5a-c1b22ffcadba: ref 1 aborting False
>Thread-15914424::DEBUG::2013-11-20 03:31:51,853::task::495::TaskManager.Task::(_debug) Task ef558dc3-c146-4080-9a5a-c1b22ffcadba: finished: {'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/143cb26d-de08-4d30-89cf-aee10245b66a/d7028f7c-0551-4bbe-a907-6f804cc9bd54'}
>Thread-15914424::DEBUG::2013-11-20 03:31:51,853::task::495::TaskManager.Task::(_debug) Task ef558dc3-c146-4080-9a5a-c1b22ffcadba: moving from state preparing -> state finished
>Thread-15914424::DEBUG::2013-11-20 03:31:51,854::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914424::DEBUG::2013-11-20 03:31:51,854::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914424::DEBUG::2013-11-20 03:31:51,855::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914424::DEBUG::2013-11-20 03:31:51,856::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914424::DEBUG::2013-11-20 03:31:51,856::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914424::DEBUG::2013-11-20 03:31:51,857::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914424::DEBUG::2013-11-20 03:31:51,857::task::495::TaskManager.Task::(_debug) Task ef558dc3-c146-4080-9a5a-c1b22ffcadba: ref 0 aborting False
>Thread-15914424::INFO::2013-11-20 03:31:51,858::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, Return response: {'status': {'message': 'OK', 'code': 0}, 'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/143cb26d-de08-4d30-89cf-aee10245b66a/d7028f7c-0551-4bbe-a907-6f804cc9bd54'}
>Thread-15914424::INFO::2013-11-20 03:31:51,860::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914424::DEBUG::2013-11-20 03:31:51,860::task::495::TaskManager.Task::(_debug) Task 2193c045-1ee5-4c10-8464-10bdaa4e0b2a: moving from state init -> state preparing
>Thread-15914424::DEBUG::2013-11-20 03:31:51,862::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`826160ca-9ff5-4cb7-b4a0-ae0afa37c68e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914424::DEBUG::2013-11-20 03:31:51,862::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914424::DEBUG::2013-11-20 03:31:51,863::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914424::DEBUG::2013-11-20 03:31:51,864::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`826160ca-9ff5-4cb7-b4a0-ae0afa37c68e`::Granted request
>Thread-15914424::DEBUG::2013-11-20 03:31:51,864::task::495::TaskManager.Task::(_debug) Task 2193c045-1ee5-4c10-8464-10bdaa4e0b2a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914424::DEBUG::2013-11-20 03:31:51,865::task::495::TaskManager.Task::(_debug) Task 2193c045-1ee5-4c10-8464-10bdaa4e0b2a: ref 1 aborting False
>Thread-15914424::DEBUG::2013-11-20 03:31:51,866::task::495::TaskManager.Task::(_debug) Task 2193c045-1ee5-4c10-8464-10bdaa4e0b2a: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914424::DEBUG::2013-11-20 03:31:51,866::task::495::TaskManager.Task::(_debug) Task 2193c045-1ee5-4c10-8464-10bdaa4e0b2a: moving from state preparing -> state finished
>Thread-15914424::DEBUG::2013-11-20 03:31:51,867::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914424::DEBUG::2013-11-20 03:31:51,867::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914424::DEBUG::2013-11-20 03:31:51,868::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914424::DEBUG::2013-11-20 03:31:51,869::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914424::DEBUG::2013-11-20 03:31:51,869::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914424::DEBUG::2013-11-20 03:31:51,870::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914424::DEBUG::2013-11-20 03:31:51,871::task::495::TaskManager.Task::(_debug) Task 2193c045-1ee5-4c10-8464-10bdaa4e0b2a: ref 0 aborting False
>Thread-15914424::INFO::2013-11-20 03:31:51,871::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914424::INFO::2013-11-20 03:31:51,872::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16)
>Thread-15914424::DEBUG::2013-11-20 03:31:51,873::task::495::TaskManager.Task::(_debug) Task 657f9c47-bc40-467c-8d17-4d9e6212cbf6: moving from state init -> state preparing
>Thread-15914424::INFO::2013-11-20 03:31:51,873::blockSD::674::Storage.StorageDomain::(validate) sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16
>Thread-15914424::DEBUG::2013-11-20 03:31:51,874::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15914424::DEBUG::2013-11-20 03:31:51,875::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15914424::DEBUG::2013-11-20 03:31:51,875::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15914424::DEBUG::2013-11-20 03:31:51,877::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15914424::DEBUG::2013-11-20 03:31:51,877::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-15914424::DEBUG::2013-11-20 03:31:51,879::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15914424::DEBUG::2013-11-20 03:31:52,521::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15914424::DEBUG::2013-11-20 03:31:52,525::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-15914424::DEBUG::2013-11-20 03:31:52,526::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'_SHA_CKSUM=716ea7492d9134a4b13de27ac61a336ad474727c', u'POOL_SPM_ID=2', u'POOL_SPM_LVER=287']
>Thread-15914424::DEBUG::2013-11-20 03:31:52,529::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f3340c23-9b6e-4a35-b978-248c4d65d03e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1297' at 'public_getStorageDomainInfo'
>Thread-15914424::DEBUG::2013-11-20 03:31:52,530::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914424::DEBUG::2013-11-20 03:31:52,531::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914424::DEBUG::2013-11-20 03:31:52,531::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f3340c23-9b6e-4a35-b978-248c4d65d03e`::Granted request
>Thread-15914424::DEBUG::2013-11-20 03:31:52,532::task::495::TaskManager.Task::(_debug) Task 657f9c47-bc40-467c-8d17-4d9e6212cbf6: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914424::DEBUG::2013-11-20 03:31:52,533::task::495::TaskManager.Task::(_debug) Task 657f9c47-bc40-467c-8d17-4d9e6212cbf6: ref 1 aborting False
>Thread-15914424::WARNING::2013-11-20 03:31:52,536::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914424::DEBUG::2013-11-20 03:31:52,537::task::495::TaskManager.Task::(_debug) Task 657f9c47-bc40-467c-8d17-4d9e6212cbf6: finished: {'info': {'uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'vguuid': 'aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', 'lver': 287, 'state': 'OK', 'version': '2', 'role': 'Master', 'master_ver': 1, 'spm_id': 2, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'OS'}}
>Thread-15914424::DEBUG::2013-11-20 03:31:52,537::task::495::TaskManager.Task::(_debug) Task 657f9c47-bc40-467c-8d17-4d9e6212cbf6: moving from state preparing -> state finished
>Thread-15914424::DEBUG::2013-11-20 03:31:52,538::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914424::DEBUG::2013-11-20 03:31:52,539::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914424::DEBUG::2013-11-20 03:31:52,539::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914424::DEBUG::2013-11-20 03:31:52,540::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914424::DEBUG::2013-11-20 03:31:52,541::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914424::DEBUG::2013-11-20 03:31:52,541::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914424::DEBUG::2013-11-20 03:31:52,542::task::495::TaskManager.Task::(_debug) Task 657f9c47-bc40-467c-8d17-4d9e6212cbf6: ref 0 aborting False
>Thread-15914424::INFO::2013-11-20 03:31:52,543::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'vguuid': 'aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', 'lver': 287, 'state': 'OK', 'version': '2', 'role': 'Master', 'master_ver': 1, 'spm_id': 2, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'OS'}}
>Thread-15914424::DEBUG::2013-11-20 03:31:52,558::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/libexec/vdsm/hooks/before_vm_start/50_vhostmd' (cwd None)
>Thread-15914424::DEBUG::2013-11-20 03:31:53,032::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15914424::INFO::2013-11-20 03:31:53,034::hooks::51::root::(_runHooksDir) 
>Thread-15914424::DEBUG::2013-11-20 03:31:53,035::libvirtvm::1077::vm.Vm::(_run) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::<?xml version="1.0" encoding="utf-8"?>
><domain type="kvm">
>	<name>Taipei</name>
>	<uuid>5f587d5c-d4b4-4242-b049-ede331c7f8b4</uuid>
>	<memory>2097152</memory>
>	<currentMemory>2097152</currentMemory>
>	<vcpu>2</vcpu>
>	<devices>
>		<disk device="disk" type="block">
>			<source dev="/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/143cb26d-de08-4d30-89cf-aee10245b66a/d7028f7c-0551-4bbe-a907-6f804cc9bd54"/>
>			<target bus="virtio" dev="vda"/>
>			<serial>30-89cf-aee10245b66a</serial>
>			<driver cache="none" error_policy="stop" io="native" name="qemu" type="qcow2"/>
>		</disk>
>		<disk device="cdrom" type="file">
>			<source file="/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO"/>
>			<target bus="ide" dev="hdc"/>
>		</disk>
>		<controller index="0" ports="16" type="virtio-serial"/>
>		<channel type="unix">
>			<target name="com.redhat.rhevm.vdsm" type="virtio"/>
>			<source mode="bind" path="/var/lib/libvirt/qemu/channels/Taipei.com.redhat.rhevm.vdsm"/>
>		</channel>
>		<memballoon model="none"/>
>		<interface type="bridge">
>			<mac address="00:1a:4a:a8:64:17"/>
>			<model type="virtio"/>
>			<source bridge="rhevm"/>
>		</interface>
>		<input bus="ps2" type="mouse"/>
>		<video>
>			<model heads="1" type="qxl" vram="65536"/>
>		</video>
>		<channel type="spicevmc">
>			<target name="com.redhat.spice.0" type="virtio"/>
>		</channel>
>		<graphics autoport="yes" keymap="en-us" listen="0" passwd="*****" passwdValidTo="1970-01-01T00:00:01" port="-1" tlsPort="-1" type="spice">
>			<channel mode="secure" name="main"/>
>			<channel mode="secure" name="inputs"/>
>		</graphics>
>	</devices>
>	<os>
>		<type arch="x86_64" machine="rhel6.2.0">hvm</type>
>		<boot dev="cdrom"/>
>		<smbios mode="sysinfo"/>
>	</os>
>	<sysinfo type="smbios">
>		<system>
>			<entry name="manufacturer">Red Hat</entry>
>			<entry name="product">RHEV Hypervisor</entry>
>			<entry name="version">6.3-20121012.0.el6_3</entry>
>			<entry name="serial">30343536-3138-5A43-4A32-333630373442_AC:16:2D:76:EB:F0</entry>
>			<entry name="uuid">5f587d5c-d4b4-4242-b049-ede331c7f8b4</entry>
>		</system>
>	</sysinfo>
>	<clock adjustment="25201" offset="variable">
>		<timer name="rtc" tickpolicy="catchup"/>
>	</clock>
>	<features>
>		<acpi/>
>	</features>
>	<cpu match="exact">
>		<model>Westmere</model>
>		<topology cores="1" sockets="2" threads="1"/>
>	</cpu>
></domain>
>
>Thread-15912392::INFO::2013-11-20 03:31:53,626::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:31:53,627::task::495::TaskManager.Task::(_debug) Task 51c1f67f-7634-4cf3-b094-0b1e8ff3a18f: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:31:53,630::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1ca04b10-2ac7-455c-a078-78a75abd5794`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:31:53,631::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:31:53,632::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:31:53,633::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1ca04b10-2ac7-455c-a078-78a75abd5794`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:31:53,634::task::495::TaskManager.Task::(_debug) Task 51c1f67f-7634-4cf3-b094-0b1e8ff3a18f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:31:53,635::task::495::TaskManager.Task::(_debug) Task 51c1f67f-7634-4cf3-b094-0b1e8ff3a18f: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:31:53,636::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15912392::DEBUG::2013-11-20 03:31:53,638::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:31:53,867::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:31:54,553::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.081357 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15912392::DEBUG::2013-11-20 03:31:54,597::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-20 03:31:54,722::libvirtvm::1401::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::event Started detail 0 opaque None
>Thread-15912392::DEBUG::2013-11-20 03:31:54,893::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15912392::DEBUG::2013-11-20 03:31:54,894::task::495::TaskManager.Task::(_debug) Task 51c1f67f-7634-4cf3-b094-0b1e8ff3a18f: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:31:54,895::task::495::TaskManager.Task::(_debug) Task 51c1f67f-7634-4cf3-b094-0b1e8ff3a18f: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:31:54,905::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:31:54,906::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:31:54,907::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:31:54,917::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:31:54,918::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:31:54,919::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914424::DEBUG::2013-11-20 03:31:54,929::utils::307::vm.Vm::(start) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Start statistics collection
>Thread-15912392::DEBUG::2013-11-20 03:31:54,930::task::495::TaskManager.Task::(_debug) Task 51c1f67f-7634-4cf3-b094-0b1e8ff3a18f: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:31:54,931::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914435::DEBUG::2013-11-20 03:31:54,931::utils::336::vm.Vm::(run) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Stats thread started
>Thread-15914436::DEBUG::2013-11-20 03:31:54,933::guestIF::47::root::(_connect) Attempting connection to /var/lib/libvirt/qemu/channels/Taipei.com.redhat.rhevm.vdsm
>Thread-15914436::DEBUG::2013-11-20 03:31:54,934::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /usr/share/vdsm/prepare-vmchannel /var/lib/libvirt/qemu/channels/Taipei.com.redhat.rhevm.vdsm' (cwd None)
>Thread-15914424::WARNING::2013-11-20 03:31:55,117::libvirtvm::1124::vm.Vm::(_readPauseCode) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::_readPauseCode unsupported by libvirt vm
>Thread-15914435::INFO::2013-11-20 03:31:55,121::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914424::DEBUG::2013-11-20 03:31:55,124::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /usr/share/vdsm/get-vm-pid Taipei' (cwd None)
>Thread-15914435::DEBUG::2013-11-20 03:31:55,125::task::495::TaskManager.Task::(_debug) Task cd431d70-a263-458f-9ecc-ff807bdfb990: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 03:31:55,128::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`31156151-a9bd-4cbb-a954-e87d484abfb3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 03:31:55,406::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 03:31:55,420::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 03:31:55,422::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`31156151-a9bd-4cbb-a954-e87d484abfb3`::Granted request
>Thread-15914435::DEBUG::2013-11-20 03:31:55,424::task::495::TaskManager.Task::(_debug) Task cd431d70-a263-458f-9ecc-ff807bdfb990: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 03:31:55,425::task::495::TaskManager.Task::(_debug) Task cd431d70-a263-458f-9ecc-ff807bdfb990: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 03:31:55,426::task::495::TaskManager.Task::(_debug) Task cd431d70-a263-458f-9ecc-ff807bdfb990: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 03:31:55,427::task::495::TaskManager.Task::(_debug) Task cd431d70-a263-458f-9ecc-ff807bdfb990: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 03:31:55,428::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 03:31:55,428::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 03:31:55,429::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 03:31:55,430::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 03:31:55,430::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 03:31:55,431::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 03:31:55,432::task::495::TaskManager.Task::(_debug) Task cd431d70-a263-458f-9ecc-ff807bdfb990: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 03:31:55,432::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914436::DEBUG::2013-11-20 03:31:55,679::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15914436::DEBUG::2013-11-20 03:31:55,681::guestIF::53::root::(_connect) connected to /var/lib/libvirt/qemu/channels/Taipei.com.redhat.rhevm.vdsm
>Thread-15914424::DEBUG::2013-11-20 03:31:55,696::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15914424::DEBUG::2013-11-20 03:31:55,708::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /sbin/service ksmtuned retune' (cwd None)
>Thread-15914424::DEBUG::2013-11-20 03:31:56,157::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15914424::DEBUG::2013-11-20 03:31:56,158::vm::358::vm.Vm::(_startUnderlyingVm) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::_ongoingCreations released
>Thread-15914438::DEBUG::2013-11-20 03:31:56,221::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Thread-15914439::DEBUG::2013-11-20 03:31:56,412::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914439::INFO::2013-11-20 03:31:56,416::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914439::DEBUG::2013-11-20 03:31:56,417::task::495::TaskManager.Task::(_debug) Task 8c11591e-0b44-4ab4-abcd-edee7f4d545c: moving from state init -> state preparing
>Thread-15914439::DEBUG::2013-11-20 03:31:56,418::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914439::DEBUG::2013-11-20 03:31:56,419::task::495::TaskManager.Task::(_debug) Task 8c11591e-0b44-4ab4-abcd-edee7f4d545c: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914439::DEBUG::2013-11-20 03:31:56,420::task::495::TaskManager.Task::(_debug) Task 8c11591e-0b44-4ab4-abcd-edee7f4d545c: moving from state preparing -> state finished
>Thread-15914439::DEBUG::2013-11-20 03:31:56,421::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914439::DEBUG::2013-11-20 03:31:56,421::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914439::DEBUG::2013-11-20 03:31:56,422::task::495::TaskManager.Task::(_debug) Task 8c11591e-0b44-4ab4-abcd-edee7f4d545c: ref 0 aborting False
>Thread-15914439::INFO::2013-11-20 03:31:56,423::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914440::DEBUG::2013-11-20 03:31:56,428::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914440::INFO::2013-11-20 03:31:56,440::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914440::DEBUG::2013-11-20 03:31:56,441::task::495::TaskManager.Task::(_debug) Task 8a0d50b5-7818-4c1a-b262-0c6539d8d2d6: moving from state init -> state preparing
>Thread-15914440::DEBUG::2013-11-20 03:31:56,443::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`69df2f61-00cc-4ecd-9982-e8926bc08f79`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914440::DEBUG::2013-11-20 03:31:56,444::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914440::DEBUG::2013-11-20 03:31:56,445::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914440::DEBUG::2013-11-20 03:31:56,445::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`69df2f61-00cc-4ecd-9982-e8926bc08f79`::Granted request
>Thread-15914440::DEBUG::2013-11-20 03:31:56,446::task::495::TaskManager.Task::(_debug) Task 8a0d50b5-7818-4c1a-b262-0c6539d8d2d6: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914440::DEBUG::2013-11-20 03:31:56,447::task::495::TaskManager.Task::(_debug) Task 8a0d50b5-7818-4c1a-b262-0c6539d8d2d6: ref 1 aborting False
>Thread-15914440::WARNING::2013-11-20 03:31:56,449::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914440::DEBUG::2013-11-20 03:31:56,450::task::495::TaskManager.Task::(_debug) Task 8a0d50b5-7818-4c1a-b262-0c6539d8d2d6: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914440::DEBUG::2013-11-20 03:31:56,451::task::495::TaskManager.Task::(_debug) Task 8a0d50b5-7818-4c1a-b262-0c6539d8d2d6: moving from state preparing -> state finished
>Thread-15914440::DEBUG::2013-11-20 03:31:56,452::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914440::DEBUG::2013-11-20 03:31:56,452::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914440::DEBUG::2013-11-20 03:31:56,453::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914440::DEBUG::2013-11-20 03:31:56,454::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914440::DEBUG::2013-11-20 03:31:56,454::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914440::DEBUG::2013-11-20 03:31:56,455::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914440::DEBUG::2013-11-20 03:31:56,455::task::495::TaskManager.Task::(_debug) Task 8a0d50b5-7818-4c1a-b262-0c6539d8d2d6: ref 0 aborting False
>Thread-15914440::INFO::2013-11-20 03:31:56,456::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:31:56,569::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914438::DEBUG::2013-11-20 03:31:57,010::clientIF::1219::vds::(fenceNode) rc 2 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = Off
>Done
> err 
>Thread-15914441::DEBUG::2013-11-20 03:31:57,019::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Thread-15914442::DEBUG::2013-11-20 03:31:57,246::clientIF::54::vds::(wrapper) [192.168.100.135]::call create with ({'bridge': 'rhevm', 'acpiEnable': 'true', 'emulatedMachine': 'rhel6.2.0', 'vmId': '163a3426-5ec5-46a5-b8f1-af06fc5d7e7d', 'transparentHugePages': 'true', 'spiceSslCipherSuite': 'DEFAULT', 'cpuType': 'Westmere', 'smp': '4', 'macAddr': '00:1a:4a:a8:64:15', 'boot': 'c', 'custom': {}, 'vmType': 'kvm', 'memSize': 4096, 'smpCoresPerSocket': '1', 'vmName': 'Luanda', 'spiceMonitors': '1', 'nice': '0', 'drives': [{'index': '0', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'format': 'raw', 'volumeID': 'b9cffdbe-152e-41ce-ae5d-08db7c08454e', 'imageID': '5f930ef3-763f-4294-a62a-6d52df220f8c', 'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'propagateErrors': 'off', 'if': 'ide'}], 'smartcardEnable': 'false', 'nicModel': 'pv', 'keyboardLayout': 'en-us', 'kvmEnable': 'true', 'pitReinjection': 'false', 'timeOffset': '0', 'spiceSecureChannels': 'smain,sinputs', 'display': 'qxl'},) {}
>Dummy-15912651::DEBUG::2013-11-20 03:31:57,703::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.699069 s, 1.5 MB/s\n'; <rc> = 0
>Thread-15914442::INFO::2013-11-20 03:31:57,820::clientIF::523::vds::(_getNetworkIp) network None: using 0
>Thread-15914442::INFO::2013-11-20 03:31:57,821::clientIF::656::vds::(create) vmContainerLock acquired by vm 163a3426-5ec5-46a5-b8f1-af06fc5d7e7d
>Thread-15914443::DEBUG::2013-11-20 03:31:57,825::vm::342::vm.Vm::(_startUnderlyingVm) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::Start
>Thread-15914442::DEBUG::2013-11-20 03:31:57,826::clientIF::669::vds::(create) Total desktops after creation of 163a3426-5ec5-46a5-b8f1-af06fc5d7e7d is 8
>Thread-15914443::DEBUG::2013-11-20 03:31:57,827::vm::346::vm.Vm::(_startUnderlyingVm) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::_ongoingCreations acquired
>Thread-15914442::DEBUG::2013-11-20 03:31:57,828::clientIF::59::vds::(wrapper) return create with {'status': {'message': 'Done', 'code': 0}, 'vmList': {'status': 'WaitForLaunch', 'bridge': 'rhevm', 'acpiEnable': 'true', 'emulatedMachine': 'rhel6.2.0', 'vmId': '163a3426-5ec5-46a5-b8f1-af06fc5d7e7d', 'pid': '0', 'drives': [{'index': '0', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'format': 'raw', 'volumeID': 'b9cffdbe-152e-41ce-ae5d-08db7c08454e', 'imageID': '5f930ef3-763f-4294-a62a-6d52df220f8c', 'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'propagateErrors': 'off', 'if': 'ide'}], 'transparentHugePages': 'true', 'keyboardLayout': 'en-us', 'displayPort': '-1', 'displaySecurePort': '-1', 'spiceSslCipherSuite': 'DEFAULT', 'cpuType': 'Westmere', 'custom': {}, 'clientIp': '', 'nicModel': 'pv', 'macAddr': '00:1a:4a:a8:64:15', 'kvmEnable': 'true', 'pitReinjection': 'false', 'boot': 'c', 'smp': '4', 'vmType': 'kvm', 'smartcardEnable': 'false', 'timeOffset': '0', 'memSize': 4096, 'displayIp': '0', 'spiceSecureChannels': 'smain,sinputs', 'smpCoresPerSocket': '1', 'vmName': 'Luanda', 'display': 'qxl', 'spiceMonitors': '1', 'nice': '0'}}
>Thread-15914443::INFO::2013-11-20 03:31:57,829::libvirtvm::1061::vm.Vm::(_run) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::VM wrapper has started
>Thread-15914443::INFO::2013-11-20 03:31:57,833::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914443::DEBUG::2013-11-20 03:31:57,834::task::495::TaskManager.Task::(_debug) Task 5970dcff-dc75-4fcf-ab88-4fcf11a7ce28: moving from state init -> state preparing
>Thread-15914443::DEBUG::2013-11-20 03:31:57,836::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b1ad435a-87cb-4308-90fa-644c87281444`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1690' at 'public_prepareVolume'
>Thread-15914443::DEBUG::2013-11-20 03:31:57,836::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914443::DEBUG::2013-11-20 03:31:57,837::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914443::DEBUG::2013-11-20 03:31:57,838::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b1ad435a-87cb-4308-90fa-644c87281444`::Granted request
>Thread-15914443::DEBUG::2013-11-20 03:31:57,838::task::495::TaskManager.Task::(_debug) Task 5970dcff-dc75-4fcf-ab88-4fcf11a7ce28: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914443::DEBUG::2013-11-20 03:31:57,839::task::495::TaskManager.Task::(_debug) Task 5970dcff-dc75-4fcf-ab88-4fcf11a7ce28: ref 1 aborting False
>Thread-15914443::DEBUG::2013-11-20 03:31:57,839::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.5f930ef3-763f-4294-a62a-6d52df220f8c`ReqID=`56675b6f-aa9a-47ba-9c23-9942c3884454`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1695' at 'public_prepareVolume'
>Thread-15914443::DEBUG::2013-11-20 03:31:57,840::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.5f930ef3-763f-4294-a62a-6d52df220f8c' for lock type 'exclusive'
>Thread-15914443::DEBUG::2013-11-20 03:31:57,841::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15914443::DEBUG::2013-11-20 03:31:57,841::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 03:31:58,004::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15914441::DEBUG::2013-11-20 03:31:58,007::clientIF::1219::vds::(fenceNode) rc 2 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = Off
>Done
> err 
>Thread-15914443::DEBUG::2013-11-20 03:31:58,753::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-42838::DEBUG::2013-11-20 03:31:58,960::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15914443::DEBUG::2013-11-20 03:31:59,061::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15914446::DEBUG::2013-11-20 03:31:59,064::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('5f587d5c-d4b4-4242-b049-ede331c7f8b4',) {}
>Thread-15914443::DEBUG::2013-11-20 03:31:59,066::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=31 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15914446::DEBUG::2013-11-20 03:31:59,067::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::CPU stats not available
>Thread-15914446::DEBUG::2013-11-20 03:31:59,068::libvirtvm::202::vm.Vm::(_getNetworkStats) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Network stats not available
>Thread-15914446::DEBUG::2013-11-20 03:31:59,272::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Disk stats not available
>Thread-15914446::DEBUG::2013-11-20 03:31:59,274::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Disk latency not available
>Thread-15914446::DEBUG::2013-11-20 03:31:59,275::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '5523', 'displayIp': '0', 'displayPort': u'5912', 'session': 'Unknown', 'displaySecurePort': u'5913', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet7': {'state': 'unknown', 'macAddr': u'00:1a:4a:a8:64:17', 'speed': '1000', 'name': u'vnet7'}}, 'vmId': '5f587d5c-d4b4-4242-b049-ede331c7f8b4', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '56908316672', 'writeLatency': '0', 'imageID': '143cb26d-de08-4d30-89cf-aee10245b66a', 'flushLatency': '0', 'truesize': '56908316672'}}, 'boot': 'd', 'statsAge': '1.46', 'cpuIdle': '100.00', 'elapsedTime': '16', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '25201', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15914443::DEBUG::2013-11-20 03:31:59,463::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.00718333 s, 71.3 kB/s\n'; <rc> = 0
>Thread-15914443::DEBUG::2013-11-20 03:31:59,464::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.00718333 s, 71.3 kB/s'], size: 512
>Thread-15914443::DEBUG::2013-11-20 03:31:59,465::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=31 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:31:59,719::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914443::DEBUG::2013-11-20 03:31:59,952::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.0001905 s, 2.7 MB/s\n'; <rc> = 0
>Thread-15914443::DEBUG::2013-11-20 03:31:59,955::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.0001905 s, 2.7 MB/s'], size: 512
>Thread-15914443::INFO::2013-11-20 03:31:59,956::image::340::Storage.Image::(getChain) sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c chain=[<storage.blockVolume.BlockVolume instance at 0x7f48ac6029e0>] 
>Thread-15914443::DEBUG::2013-11-20 03:31:59,957::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --autobackup n --available y 6b892067-ac7e-4039-a7be-5b7387525b16/b9cffdbe-152e-41ce-ae5d-08db7c08454e' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:32:00,394::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0899939 s, 11.4 MB/s\n'; <rc> = 0
>Thread-15914443::DEBUG::2013-11-20 03:32:00,773::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15914443::DEBUG::2013-11-20 03:32:00,774::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15914443::DEBUG::2013-11-20 03:32:00,775::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15914443::DEBUG::2013-11-20 03:32:00,776::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.b9cffdbe-152e-41ce-ae5d-08db7c08454e`ReqID=`e416d3d3-2558-4cf8-a0e5-9e1d171d8fce`::Request was made in '/usr/share/vdsm/storage/resourceFactories.py' line '149' at '__getResourceCandidatesList'
>Thread-15914443::DEBUG::2013-11-20 03:32:00,776::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.b9cffdbe-152e-41ce-ae5d-08db7c08454e' for lock type 'exclusive'
>Thread-15914443::DEBUG::2013-11-20 03:32:00,777::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.b9cffdbe-152e-41ce-ae5d-08db7c08454e' is free. Now locking as 'exclusive' (1 active user)
>Thread-15914443::DEBUG::2013-11-20 03:32:00,777::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.b9cffdbe-152e-41ce-ae5d-08db7c08454e`ReqID=`e416d3d3-2558-4cf8-a0e5-9e1d171d8fce`::Granted request
>Thread-15914443::DEBUG::2013-11-20 03:32:00,778::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.5f930ef3-763f-4294-a62a-6d52df220f8c' is free. Now locking as 'exclusive' (1 active user)
>Thread-15914443::DEBUG::2013-11-20 03:32:00,778::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.5f930ef3-763f-4294-a62a-6d52df220f8c`ReqID=`56675b6f-aa9a-47ba-9c23-9942c3884454`::Granted request
>Thread-15914443::DEBUG::2013-11-20 03:32:00,779::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15914443::DEBUG::2013-11-20 03:32:00,780::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15914443::DEBUG::2013-11-20 03:32:01,501::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15914443::DEBUG::2013-11-20 03:32:01,739::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15914443::DEBUG::2013-11-20 03:32:01,742::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --refresh 6b892067-ac7e-4039-a7be-5b7387525b16/b9cffdbe-152e-41ce-ae5d-08db7c08454e' (cwd None)
>Thread-15914443::DEBUG::2013-11-20 03:32:02,340::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15914443::DEBUG::2013-11-20 03:32:02,350::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15914443::DEBUG::2013-11-20 03:32:02,357::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15914443::INFO::2013-11-20 03:32:02,358::volume::517::Storage.Volume::(prepare) Volume: preparing volume 6b892067-ac7e-4039-a7be-5b7387525b16/b9cffdbe-152e-41ce-ae5d-08db7c08454e
>Thread-15914443::DEBUG::2013-11-20 03:32:02,359::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15914443::DEBUG::2013-11-20 03:32:02,360::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:32:02,540::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914448::DEBUG::2013-11-20 03:32:02,804::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('5f587d5c-d4b4-4242-b049-ede331c7f8b4',) {}
>Thread-15914448::DEBUG::2013-11-20 03:32:02,805::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::CPU stats not available
>Thread-15914448::DEBUG::2013-11-20 03:32:02,806::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Disk stats not available
>Thread-15914448::DEBUG::2013-11-20 03:32:02,806::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Disk latency not available
>Thread-15914448::DEBUG::2013-11-20 03:32:02,807::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '5523', 'displayIp': '0', 'displayPort': u'5912', 'session': 'Unknown', 'displaySecurePort': u'5913', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet7': {'macAddr': u'00:1a:4a:a8:64:17', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet7'}}, 'vmId': '5f587d5c-d4b4-4242-b049-ede331c7f8b4', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '56908316672', 'writeLatency': '0', 'imageID': '143cb26d-de08-4d30-89cf-aee10245b66a', 'flushLatency': '0', 'truesize': '56908316672'}}, 'boot': 'd', 'statsAge': '1.13', 'cpuIdle': '100.00', 'elapsedTime': '19', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '25201', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:32:03,007::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0476825 s, 21.5 MB/s\n'; <rc> = 0
>Thread-15914443::DEBUG::2013-11-20 03:32:03,060::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15914443::DEBUG::2013-11-20 03:32:03,248::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15914443::DEBUG::2013-11-20 03:32:03,249::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=31 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15914443::DEBUG::2013-11-20 03:32:03,606::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000206212 s, 2.5 MB/s\n'; <rc> = 0
>Thread-15914443::DEBUG::2013-11-20 03:32:03,608::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000206212 s, 2.5 MB/s'], size: 512
>Thread-15914443::DEBUG::2013-11-20 03:32:03,609::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=31 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15914443::DEBUG::2013-11-20 03:32:04,025::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000221991 s, 2.3 MB/s\n'; <rc> = 0
>Thread-15914443::DEBUG::2013-11-20 03:32:04,026::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000221991 s, 2.3 MB/s'], size: 512
>Thread-15914443::DEBUG::2013-11-20 03:32:04,029::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.b9cffdbe-152e-41ce-ae5d-08db7c08454e`ReqID=`7e87b0f3-e582-4035-9590-420f72cc1c9e`::Request was made in '/usr/share/vdsm/storage/volume.py' line '533' at 'prepare'
>Thread-15914443::DEBUG::2013-11-20 03:32:04,030::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.b9cffdbe-152e-41ce-ae5d-08db7c08454e' for lock type 'exclusive'
>Thread-15914443::DEBUG::2013-11-20 03:32:04,031::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.b9cffdbe-152e-41ce-ae5d-08db7c08454e' is free. Now locking as 'exclusive' (1 active user)
>Thread-15914443::DEBUG::2013-11-20 03:32:04,032::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.b9cffdbe-152e-41ce-ae5d-08db7c08454e`ReqID=`7e87b0f3-e582-4035-9590-420f72cc1c9e`::Granted request
>Thread-15914443::DEBUG::2013-11-20 03:32:04,034::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=31 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15914443::DEBUG::2013-11-20 03:32:04,696::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000219516 s, 2.3 MB/s\n'; <rc> = 0
>Thread-15914443::DEBUG::2013-11-20 03:32:04,697::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000219516 s, 2.3 MB/s'], size: 512
>Thread-15914443::DEBUG::2013-11-20 03:32:04,699::task::495::TaskManager.Task::(_debug) Task 5970dcff-dc75-4fcf-ab88-4fcf11a7ce28: finished: None
>Thread-15914443::DEBUG::2013-11-20 03:32:04,699::task::495::TaskManager.Task::(_debug) Task 5970dcff-dc75-4fcf-ab88-4fcf11a7ce28: moving from state preparing -> state finished
>Thread-15914443::DEBUG::2013-11-20 03:32:04,700::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914443::DEBUG::2013-11-20 03:32:04,701::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914443::DEBUG::2013-11-20 03:32:04,702::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914443::DEBUG::2013-11-20 03:32:04,702::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914443::DEBUG::2013-11-20 03:32:04,703::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914443::DEBUG::2013-11-20 03:32:04,704::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914443::DEBUG::2013-11-20 03:32:04,705::task::495::TaskManager.Task::(_debug) Task 5970dcff-dc75-4fcf-ab88-4fcf11a7ce28: ref 0 aborting False
>Thread-15914443::INFO::2013-11-20 03:32:04,705::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, Return response: {'status': {'message': 'OK', 'code': 0}}
>Thread-15914443::INFO::2013-11-20 03:32:04,706::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914443::DEBUG::2013-11-20 03:32:04,707::task::495::TaskManager.Task::(_debug) Task a7c76b9a-b0ef-495c-a422-a99ea841f61f: moving from state init -> state preparing
>Thread-15914443::DEBUG::2013-11-20 03:32:04,708::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`576f44dc-0036-4c83-97ae-929801b51f5c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1666' at 'public_getVolumePath'
>Thread-15914443::DEBUG::2013-11-20 03:32:04,709::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914443::DEBUG::2013-11-20 03:32:04,710::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914443::DEBUG::2013-11-20 03:32:04,710::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`576f44dc-0036-4c83-97ae-929801b51f5c`::Granted request
>Thread-15914443::DEBUG::2013-11-20 03:32:04,711::task::495::TaskManager.Task::(_debug) Task a7c76b9a-b0ef-495c-a422-a99ea841f61f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914443::DEBUG::2013-11-20 03:32:04,712::task::495::TaskManager.Task::(_debug) Task a7c76b9a-b0ef-495c-a422-a99ea841f61f: ref 1 aborting False
>Thread-15914443::DEBUG::2013-11-20 03:32:04,713::task::495::TaskManager.Task::(_debug) Task a7c76b9a-b0ef-495c-a422-a99ea841f61f: finished: {'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/5f930ef3-763f-4294-a62a-6d52df220f8c/b9cffdbe-152e-41ce-ae5d-08db7c08454e'}
>Thread-15914443::DEBUG::2013-11-20 03:32:04,714::task::495::TaskManager.Task::(_debug) Task a7c76b9a-b0ef-495c-a422-a99ea841f61f: moving from state preparing -> state finished
>Thread-15914443::DEBUG::2013-11-20 03:32:04,715::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914443::DEBUG::2013-11-20 03:32:04,715::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914443::DEBUG::2013-11-20 03:32:04,716::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914443::DEBUG::2013-11-20 03:32:04,717::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914443::DEBUG::2013-11-20 03:32:04,717::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914443::DEBUG::2013-11-20 03:32:04,718::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914443::DEBUG::2013-11-20 03:32:04,718::task::495::TaskManager.Task::(_debug) Task a7c76b9a-b0ef-495c-a422-a99ea841f61f: ref 0 aborting False
>Thread-15914443::INFO::2013-11-20 03:32:04,719::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, Return response: {'status': {'message': 'OK', 'code': 0}, 'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/5f930ef3-763f-4294-a62a-6d52df220f8c/b9cffdbe-152e-41ce-ae5d-08db7c08454e'}
>Thread-15914443::INFO::2013-11-20 03:32:04,720::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914443::DEBUG::2013-11-20 03:32:04,721::task::495::TaskManager.Task::(_debug) Task 5c6b5979-8662-4799-a435-847e6c3b2e52: moving from state init -> state preparing
>Thread-15914443::DEBUG::2013-11-20 03:32:04,722::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a820f8d5-0db7-49ad-b1b0-4b1f81b47d17`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914443::DEBUG::2013-11-20 03:32:04,723::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914443::DEBUG::2013-11-20 03:32:04,723::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914443::DEBUG::2013-11-20 03:32:04,724::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a820f8d5-0db7-49ad-b1b0-4b1f81b47d17`::Granted request
>Thread-15914443::DEBUG::2013-11-20 03:32:04,725::task::495::TaskManager.Task::(_debug) Task 5c6b5979-8662-4799-a435-847e6c3b2e52: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914443::DEBUG::2013-11-20 03:32:04,726::task::495::TaskManager.Task::(_debug) Task 5c6b5979-8662-4799-a435-847e6c3b2e52: ref 1 aborting False
>Thread-15914443::DEBUG::2013-11-20 03:32:04,726::task::495::TaskManager.Task::(_debug) Task 5c6b5979-8662-4799-a435-847e6c3b2e52: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914443::DEBUG::2013-11-20 03:32:04,727::task::495::TaskManager.Task::(_debug) Task 5c6b5979-8662-4799-a435-847e6c3b2e52: moving from state preparing -> state finished
>Thread-15914443::DEBUG::2013-11-20 03:32:04,728::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914443::DEBUG::2013-11-20 03:32:04,728::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914443::DEBUG::2013-11-20 03:32:04,729::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914443::DEBUG::2013-11-20 03:32:04,729::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914443::DEBUG::2013-11-20 03:32:04,730::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914443::DEBUG::2013-11-20 03:32:04,731::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914443::DEBUG::2013-11-20 03:32:04,731::task::495::TaskManager.Task::(_debug) Task 5c6b5979-8662-4799-a435-847e6c3b2e52: ref 0 aborting False
>Thread-15914443::INFO::2013-11-20 03:32:04,732::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914443::INFO::2013-11-20 03:32:04,733::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16)
>Thread-15914443::DEBUG::2013-11-20 03:32:04,733::task::495::TaskManager.Task::(_debug) Task 11f6d703-c803-435c-9fce-deb9f74961eb: moving from state init -> state preparing
>Thread-15914443::INFO::2013-11-20 03:32:04,734::blockSD::674::Storage.StorageDomain::(validate) sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16
>Thread-15914443::DEBUG::2013-11-20 03:32:04,735::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15914443::DEBUG::2013-11-20 03:32:04,736::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15914443::DEBUG::2013-11-20 03:32:04,736::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15914443::DEBUG::2013-11-20 03:32:04,738::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15914443::DEBUG::2013-11-20 03:32:04,739::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-15914443::DEBUG::2013-11-20 03:32:04,740::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:32:05,018::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:32:05,816::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0762156 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15914449::INFO::2013-11-20 03:32:05,835::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914449::DEBUG::2013-11-20 03:32:05,836::task::495::TaskManager.Task::(_debug) Task 1236ef9e-63c7-4b59-8ba7-43a808ea1c6c: moving from state init -> state preparing
>Thread-15914449::DEBUG::2013-11-20 03:32:05,836::task::495::TaskManager.Task::(_debug) Task 1236ef9e-63c7-4b59-8ba7-43a808ea1c6c: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00133514404297', 'lastCheck': 1384918319.0651929, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00243592262268', 'lastCheck': 1384918324.952491, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00214195251465', 'lastCheck': 1384918316.4166529, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000575065612793', 'lastCheck': 1384918316.920346, 'valid': True, 'code': 0}}
>Thread-15914449::DEBUG::2013-11-20 03:32:05,838::task::495::TaskManager.Task::(_debug) Task 1236ef9e-63c7-4b59-8ba7-43a808ea1c6c: moving from state preparing -> state finished
>Thread-15914449::DEBUG::2013-11-20 03:32:05,838::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914449::DEBUG::2013-11-20 03:32:05,839::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914449::DEBUG::2013-11-20 03:32:05,840::task::495::TaskManager.Task::(_debug) Task 1236ef9e-63c7-4b59-8ba7-43a808ea1c6c: ref 0 aborting False
>Thread-15914449::INFO::2013-11-20 03:32:05,840::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00133514404297', 'lastCheck': 1384918319.0651929, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00243592262268', 'lastCheck': 1384918324.952491, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00214195251465', 'lastCheck': 1384918316.4166529, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000575065612793', 'lastCheck': 1384918316.920346, 'valid': True, 'code': 0}}
>Thread-15914443::DEBUG::2013-11-20 03:32:05,842::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15914443::DEBUG::2013-11-20 03:32:05,845::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-15914443::DEBUG::2013-11-20 03:32:05,846::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'_SHA_CKSUM=716ea7492d9134a4b13de27ac61a336ad474727c', u'POOL_SPM_ID=2', u'POOL_SPM_LVER=287']
>Thread-15914443::DEBUG::2013-11-20 03:32:05,849::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a1d0f8cd-d5a4-46d8-9356-f128ae225d9f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1297' at 'public_getStorageDomainInfo'
>Thread-15914443::DEBUG::2013-11-20 03:32:05,850::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914443::DEBUG::2013-11-20 03:32:05,851::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914443::DEBUG::2013-11-20 03:32:05,856::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a1d0f8cd-d5a4-46d8-9356-f128ae225d9f`::Granted request
>Thread-15914443::DEBUG::2013-11-20 03:32:05,857::task::495::TaskManager.Task::(_debug) Task 11f6d703-c803-435c-9fce-deb9f74961eb: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914443::DEBUG::2013-11-20 03:32:05,857::task::495::TaskManager.Task::(_debug) Task 11f6d703-c803-435c-9fce-deb9f74961eb: ref 1 aborting False
>Thread-15914443::WARNING::2013-11-20 03:32:05,859::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914443::DEBUG::2013-11-20 03:32:05,860::task::495::TaskManager.Task::(_debug) Task 11f6d703-c803-435c-9fce-deb9f74961eb: finished: {'info': {'uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'vguuid': 'aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', 'lver': 287, 'state': 'OK', 'version': '2', 'role': 'Master', 'master_ver': 1, 'spm_id': 2, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'OS'}}
>Thread-15914443::DEBUG::2013-11-20 03:32:05,860::task::495::TaskManager.Task::(_debug) Task 11f6d703-c803-435c-9fce-deb9f74961eb: moving from state preparing -> state finished
>Thread-15914443::DEBUG::2013-11-20 03:32:05,861::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914443::DEBUG::2013-11-20 03:32:05,861::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914443::DEBUG::2013-11-20 03:32:05,862::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914443::DEBUG::2013-11-20 03:32:05,863::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914443::DEBUG::2013-11-20 03:32:05,864::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914450::DEBUG::2013-11-20 03:32:05,866::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::CPU stats not available
>Thread-15914443::DEBUG::2013-11-20 03:32:05,867::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914450::DEBUG::2013-11-20 03:32:05,867::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Disk stats not available
>Thread-15914450::DEBUG::2013-11-20 03:32:05,868::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Disk latency not available
>Thread-15914443::DEBUG::2013-11-20 03:32:05,868::task::495::TaskManager.Task::(_debug) Task 11f6d703-c803-435c-9fce-deb9f74961eb: ref 0 aborting False
>Thread-15914443::INFO::2013-11-20 03:32:05,876::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'vguuid': 'aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', 'lver': 287, 'state': 'OK', 'version': '2', 'role': 'Master', 'master_ver': 1, 'spm_id': 2, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'OS'}}
>Thread-15914443::DEBUG::2013-11-20 03:32:05,888::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/libexec/vdsm/hooks/before_vm_start/50_vhostmd' (cwd None)
>Thread-15914443::DEBUG::2013-11-20 03:32:06,360::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15914443::INFO::2013-11-20 03:32:06,361::hooks::51::root::(_runHooksDir) 
>Thread-15914443::DEBUG::2013-11-20 03:32:06,362::libvirtvm::1077::vm.Vm::(_run) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::<?xml version="1.0" encoding="utf-8"?>
><domain type="kvm">
>	<name>Luanda</name>
>	<uuid>163a3426-5ec5-46a5-b8f1-af06fc5d7e7d</uuid>
>	<memory>4194304</memory>
>	<currentMemory>4194304</currentMemory>
>	<vcpu>4</vcpu>
>	<devices>
>		<disk device="disk" type="block">
>			<source dev="/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/5f930ef3-763f-4294-a62a-6d52df220f8c/b9cffdbe-152e-41ce-ae5d-08db7c08454e"/>
>			<target bus="ide" dev="hda"/>
>			<serial>94-a62a-6d52df220f8c</serial>
>			<driver cache="none" error_policy="stop" io="native" name="qemu" type="raw"/>
>		</disk>
>		<disk device="cdrom" type="file">
>			<target bus="ide" dev="hdc"/>
>		</disk>
>		<controller index="0" ports="16" type="virtio-serial"/>
>		<channel type="unix">
>			<target name="com.redhat.rhevm.vdsm" type="virtio"/>
>			<source mode="bind" path="/var/lib/libvirt/qemu/channels/Luanda.com.redhat.rhevm.vdsm"/>
>		</channel>
>		<memballoon model="none"/>
>		<interface type="bridge">
>			<mac address="00:1a:4a:a8:64:15"/>
>			<model type="virtio"/>
>			<source bridge="rhevm"/>
>		</interface>
>		<input bus="ps2" type="mouse"/>
>		<video>
>			<model heads="1" type="qxl" vram="65536"/>
>		</video>
>		<channel type="spicevmc">
>			<target name="com.redhat.spice.0" type="virtio"/>
>		</channel>
>		<graphics autoport="yes" keymap="en-us" listen="0" passwd="*****" passwdValidTo="1970-01-01T00:00:01" port="-1" tlsPort="-1" type="spice">
>			<channel mode="secure" name="main"/>
>			<channel mode="secure" name="inputs"/>
>		</graphics>
>	</devices>
>	<os>
>		<type arch="x86_64" machine="rhel6.2.0">hvm</type>
>		<boot dev="hd"/>
>		<smbios mode="sysinfo"/>
>	</os>
>	<sysinfo type="smbios">
>		<system>
>			<entry name="manufacturer">Red Hat</entry>
>			<entry name="product">RHEV Hypervisor</entry>
>			<entry name="version">6.3-20121012.0.el6_3</entry>
>			<entry name="serial">30343536-3138-5A43-4A32-333630373442_AC:16:2D:76:EB:F0</entry>
>			<entry name="uuid">163a3426-5ec5-46a5-b8f1-af06fc5d7e7d</entry>
>		</system>
>	</sysinfo>
>	<clock adjustment="0" offset="variable">
>		<timer name="rtc" tickpolicy="catchup"/>
>	</clock>
>	<features>
>		<acpi/>
>	</features>
>	<cpu match="exact">
>		<model>Westmere</model>
>		<topology cores="1" sockets="4" threads="1"/>
>	</cpu>
></domain>
>
>Thread-15914451::DEBUG::2013-11-20 03:32:06,627::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914451::INFO::2013-11-20 03:32:06,628::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914451::DEBUG::2013-11-20 03:32:06,628::task::495::TaskManager.Task::(_debug) Task 06e9c810-27e2-4c4c-9aa3-f8ff2592c356: moving from state init -> state preparing
>Thread-15914451::DEBUG::2013-11-20 03:32:06,629::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914451::DEBUG::2013-11-20 03:32:06,629::task::495::TaskManager.Task::(_debug) Task 06e9c810-27e2-4c4c-9aa3-f8ff2592c356: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914451::DEBUG::2013-11-20 03:32:06,630::task::495::TaskManager.Task::(_debug) Task 06e9c810-27e2-4c4c-9aa3-f8ff2592c356: moving from state preparing -> state finished
>Thread-15914451::DEBUG::2013-11-20 03:32:06,630::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914451::DEBUG::2013-11-20 03:32:06,630::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914451::DEBUG::2013-11-20 03:32:06,631::task::495::TaskManager.Task::(_debug) Task 06e9c810-27e2-4c4c-9aa3-f8ff2592c356: ref 0 aborting False
>Thread-15914451::INFO::2013-11-20 03:32:06,631::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914452::DEBUG::2013-11-20 03:32:06,635::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914452::INFO::2013-11-20 03:32:06,636::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914452::DEBUG::2013-11-20 03:32:06,636::task::495::TaskManager.Task::(_debug) Task 4369c542-a6a8-4434-880c-429150255f40: moving from state init -> state preparing
>Thread-15914452::DEBUG::2013-11-20 03:32:06,637::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b0d176f7-af44-4730-ad79-ba61b40ffd63`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914452::DEBUG::2013-11-20 03:32:06,637::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914452::DEBUG::2013-11-20 03:32:06,638::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914452::DEBUG::2013-11-20 03:32:06,638::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b0d176f7-af44-4730-ad79-ba61b40ffd63`::Granted request
>Thread-15914452::DEBUG::2013-11-20 03:32:06,639::task::495::TaskManager.Task::(_debug) Task 4369c542-a6a8-4434-880c-429150255f40: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914452::DEBUG::2013-11-20 03:32:06,639::task::495::TaskManager.Task::(_debug) Task 4369c542-a6a8-4434-880c-429150255f40: ref 1 aborting False
>Thread-15914452::WARNING::2013-11-20 03:32:06,641::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914452::DEBUG::2013-11-20 03:32:06,642::task::495::TaskManager.Task::(_debug) Task 4369c542-a6a8-4434-880c-429150255f40: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914452::DEBUG::2013-11-20 03:32:06,642::task::495::TaskManager.Task::(_debug) Task 4369c542-a6a8-4434-880c-429150255f40: moving from state preparing -> state finished
>Thread-15914452::DEBUG::2013-11-20 03:32:06,643::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914452::DEBUG::2013-11-20 03:32:06,643::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914452::DEBUG::2013-11-20 03:32:06,644::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914452::DEBUG::2013-11-20 03:32:06,644::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914452::DEBUG::2013-11-20 03:32:06,644::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914452::DEBUG::2013-11-20 03:32:06,645::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914452::DEBUG::2013-11-20 03:32:06,645::task::495::TaskManager.Task::(_debug) Task 4369c542-a6a8-4434-880c-429150255f40: ref 0 aborting False
>Thread-15914452::INFO::2013-11-20 03:32:06,646::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>libvirtEventLoop::DEBUG::2013-11-20 03:32:07,339::libvirtvm::1401::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::event Started detail 0 opaque None
>Thread-15914443::DEBUG::2013-11-20 03:32:07,376::utils::307::vm.Vm::(start) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::Start statistics collection
>Thread-15914453::DEBUG::2013-11-20 03:32:07,377::utils::336::vm.Vm::(run) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::Stats thread started
>Thread-15914454::DEBUG::2013-11-20 03:32:07,378::guestIF::47::root::(_connect) Attempting connection to /var/lib/libvirt/qemu/channels/Luanda.com.redhat.rhevm.vdsm
>Thread-15914453::INFO::2013-11-20 03:32:07,379::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914454::DEBUG::2013-11-20 03:32:07,380::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /usr/share/vdsm/prepare-vmchannel /var/lib/libvirt/qemu/channels/Luanda.com.redhat.rhevm.vdsm' (cwd None)
>Thread-15914453::DEBUG::2013-11-20 03:32:07,380::task::495::TaskManager.Task::(_debug) Task d1b40b70-69bc-450c-93a4-44c03326fbab: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 03:32:07,382::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e57d86e2-4734-4082-898f-c060122ada2f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914443::WARNING::2013-11-20 03:32:07,546::libvirtvm::1124::vm.Vm::(_readPauseCode) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::_readPauseCode unsupported by libvirt vm
>Thread-15914453::DEBUG::2013-11-20 03:32:07,575::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914443::DEBUG::2013-11-20 03:32:07,587::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /usr/share/vdsm/get-vm-pid Luanda' (cwd None)
>Thread-15914453::DEBUG::2013-11-20 03:32:07,595::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Dummy-15912651::DEBUG::2013-11-20 03:32:07,863::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::DEBUG::2013-11-20 03:32:07,866::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e57d86e2-4734-4082-898f-c060122ada2f`::Granted request
>Thread-15914454::DEBUG::2013-11-20 03:32:07,870::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15914453::DEBUG::2013-11-20 03:32:07,871::task::495::TaskManager.Task::(_debug) Task d1b40b70-69bc-450c-93a4-44c03326fbab: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914454::DEBUG::2013-11-20 03:32:08,148::guestIF::53::root::(_connect) connected to /var/lib/libvirt/qemu/channels/Luanda.com.redhat.rhevm.vdsm
>Thread-15914453::DEBUG::2013-11-20 03:32:08,149::task::495::TaskManager.Task::(_debug) Task d1b40b70-69bc-450c-93a4-44c03326fbab: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 03:32:08,152::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15914443::DEBUG::2013-11-20 03:32:08,155::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15914453::DEBUG::2013-11-20 03:32:08,156::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15913518::INFO::2013-11-20 03:32:08,463::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:32:08,470::task::495::TaskManager.Task::(_debug) Task 639b875a-f2d9-40bb-a186-31e5cf75d3cf: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:32:08,473::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`71fc05db-135a-4181-b992-23ea4527cd64`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:32:08,474::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:32:08,476::resourceManager::481::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' found in shared state and queue is empty, Joining current shared lock (2 active users)
>Thread-15913518::DEBUG::2013-11-20 03:32:08,476::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`71fc05db-135a-4181-b992-23ea4527cd64`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:32:08,479::task::495::TaskManager.Task::(_debug) Task 639b875a-f2d9-40bb-a186-31e5cf75d3cf: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:32:08,480::task::495::TaskManager.Task::(_debug) Task 639b875a-f2d9-40bb-a186-31e5cf75d3cf: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:32:08,481::lvm::480::OperationMutex::(_reloadlvs) Got the operational mutex
>Thread-15913518::DEBUG::2013-11-20 03:32:08,482::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15914443::DEBUG::2013-11-20 03:32:08,483::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /sbin/service ksmtuned retune' (cwd None)
>Thread-15914455::DEBUG::2013-11-20 03:32:09,063::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Thread-15914443::DEBUG::2013-11-20 03:32:09,382::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15914443::DEBUG::2013-11-20 03:32:09,401::vm::358::vm.Vm::(_startUnderlyingVm) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::_ongoingCreations released
>Thread-15914457::DEBUG::2013-11-20 03:32:09,431::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('5f587d5c-d4b4-4242-b049-ede331c7f8b4',) {}
>Thread-15914457::DEBUG::2013-11-20 03:32:09,436::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::CPU stats not available
>Thread-15914457::DEBUG::2013-11-20 03:32:09,437::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Disk stats not available
>Thread-15914457::DEBUG::2013-11-20 03:32:09,440::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Disk latency not available
>Thread-15914457::DEBUG::2013-11-20 03:32:09,442::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '5523', 'displayIp': '0', 'displayPort': u'5912', 'session': 'Unknown', 'displaySecurePort': u'5913', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet7': {'macAddr': u'00:1a:4a:a8:64:17', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet7'}}, 'vmId': '5f587d5c-d4b4-4242-b049-ede331c7f8b4', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '56908316672', 'writeLatency': '0', 'imageID': '143cb26d-de08-4d30-89cf-aee10245b66a', 'flushLatency': '0', 'truesize': '56908316672'}}, 'boot': 'd', 'statsAge': '1.57', 'cpuIdle': '100.00', 'elapsedTime': '26', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '25201', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15914453::DEBUG::2013-11-20 03:32:09,510::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913518::DEBUG::2013-11-20 03:32:09,652::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15914453::DEBUG::2013-11-20 03:32:09,848::task::495::TaskManager.Task::(_debug) Task d1b40b70-69bc-450c-93a4-44c03326fbab: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914455::DEBUG::2013-11-20 03:32:09,861::clientIF::1219::vds::(fenceNode) rc 2 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = Off
>Done
> err 
>Thread-15914453::DEBUG::2013-11-20 03:32:09,874::task::495::TaskManager.Task::(_debug) Task d1b40b70-69bc-450c-93a4-44c03326fbab: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 03:32:09,947::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914458::DEBUG::2013-11-20 03:32:09,961::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Thread-15914453::DEBUG::2013-11-20 03:32:10,010::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 03:32:10,332::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:32:10,336::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913518::DEBUG::2013-11-20 03:32:10,340::task::495::TaskManager.Task::(_debug) Task 639b875a-f2d9-40bb-a186-31e5cf75d3cf: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15914453::DEBUG::2013-11-20 03:32:10,340::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (1 active users)
>Thread-15913518::DEBUG::2013-11-20 03:32:10,341::task::495::TaskManager.Task::(_debug) Task 639b875a-f2d9-40bb-a186-31e5cf75d3cf: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:32:10,343::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 03:32:10,344::task::495::TaskManager.Task::(_debug) Task d1b40b70-69bc-450c-93a4-44c03326fbab: ref 0 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:32:10,345::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::INFO::2013-11-20 03:32:10,346::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15913518::DEBUG::2013-11-20 03:32:10,348::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:32:10,349::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:32:10,350::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:32:10,351::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:32:10,352::task::495::TaskManager.Task::(_debug) Task 639b875a-f2d9-40bb-a186-31e5cf75d3cf: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:32:10,353::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:32:10,355::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:32:10,356::task::495::TaskManager.Task::(_debug) Task f2b117d7-da7f-4e67-bd21-96ac61458849: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:32:10,359::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a691a3de-58e3-4e9c-9f68-02dd9a5af275`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:32:10,360::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:32:10,363::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:32:10,364::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a691a3de-58e3-4e9c-9f68-02dd9a5af275`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:32:10,366::task::495::TaskManager.Task::(_debug) Task f2b117d7-da7f-4e67-bd21-96ac61458849: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:32:10,367::task::495::TaskManager.Task::(_debug) Task f2b117d7-da7f-4e67-bd21-96ac61458849: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:32:10,368::task::495::TaskManager.Task::(_debug) Task f2b117d7-da7f-4e67-bd21-96ac61458849: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:32:10,368::task::495::TaskManager.Task::(_debug) Task f2b117d7-da7f-4e67-bd21-96ac61458849: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:32:10,369::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:32:10,370::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:32:10,371::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:32:10,372::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:32:10,373::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:32:10,374::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:32:10,375::task::495::TaskManager.Task::(_debug) Task f2b117d7-da7f-4e67-bd21-96ac61458849: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:32:10,376::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:32:10,451::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 1.9687 s, 520 kB/s\n'; <rc> = 0
>Thread-15914458::DEBUG::2013-11-20 03:32:11,079::clientIF::1219::vds::(fenceNode) rc 2 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = Off
>Done
> err 
>Dummy-15912651::DEBUG::2013-11-20 03:32:12,461::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914460::DEBUG::2013-11-20 03:32:12,463::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('5f587d5c-d4b4-4242-b049-ede331c7f8b4',) {}
>Thread-15914460::DEBUG::2013-11-20 03:32:12,642::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Disk stats not available
>Thread-15914460::DEBUG::2013-11-20 03:32:12,643::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Disk latency not available
>Thread-15914460::DEBUG::2013-11-20 03:32:12,644::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '5523', 'displayIp': '0', 'displayPort': u'5912', 'session': 'Unknown', 'displaySecurePort': u'5913', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet7': {'macAddr': u'00:1a:4a:a8:64:17', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet7'}}, 'vmId': '5f587d5c-d4b4-4242-b049-ede331c7f8b4', 'monitorResponse': '0', 'cpuUser': '81.89', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '56908316672', 'writeLatency': '0', 'imageID': '143cb26d-de08-4d30-89cf-aee10245b66a', 'flushLatency': '0', 'truesize': '56908316672'}}, 'boot': 'd', 'statsAge': '0.33', 'cpuIdle': '18.11', 'elapsedTime': '29', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '25201', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15914461::DEBUG::2013-11-20 03:32:12,651::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('163a3426-5ec5-46a5-b8f1-af06fc5d7e7d',) {}
>Thread-15914461::DEBUG::2013-11-20 03:32:12,652::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::CPU stats not available
>Thread-15914461::DEBUG::2013-11-20 03:32:12,653::libvirtvm::202::vm.Vm::(_getNetworkStats) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::Network stats not available
>Thread-15914461::DEBUG::2013-11-20 03:32:12,653::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::Disk stats not available
>Thread-15914461::DEBUG::2013-11-20 03:32:12,654::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::Disk latency not available
>Thread-15914461::DEBUG::2013-11-20 03:32:12,654::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '5851', 'displayIp': '0', 'displayPort': u'5914', 'session': 'Unknown', 'displaySecurePort': u'5915', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet8': {'state': 'unknown', 'macAddr': u'00:1a:4a:a8:64:15', 'speed': '1000', 'name': u'vnet8'}}, 'vmId': '163a3426-5ec5-46a5-b8f1-af06fc5d7e7d', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '53821308928', 'writeLatency': '0', 'imageID': '5f930ef3-763f-4294-a62a-6d52df220f8c', 'flushLatency': '0', 'truesize': '53821308928'}}, 'boot': 'c', 'statsAge': '0.29', 'cpuIdle': '100.00', 'elapsedTime': '14', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:32:12,955::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.05525 s, 18.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:32:14,972::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:32:15,882::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0789659 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15914463::DEBUG::2013-11-20 03:32:15,968::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('5f587d5c-d4b4-4242-b049-ede331c7f8b4',) {}
>Thread-15914463::DEBUG::2013-11-20 03:32:15,969::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Disk stats not available
>Thread-15914463::DEBUG::2013-11-20 03:32:15,969::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Disk latency not available
>Thread-15914463::DEBUG::2013-11-20 03:32:15,970::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '5523', 'displayIp': '0', 'displayPort': u'5912', 'session': 'Unknown', 'displaySecurePort': u'5913', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet7': {'macAddr': u'00:1a:4a:a8:64:17', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet7'}}, 'vmId': '5f587d5c-d4b4-4242-b049-ede331c7f8b4', 'monitorResponse': '0', 'cpuUser': '81.89', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '56908316672', 'writeLatency': '0', 'imageID': '143cb26d-de08-4d30-89cf-aee10245b66a', 'flushLatency': '0', 'truesize': '56908316672'}}, 'boot': 'd', 'statsAge': '1.61', 'cpuIdle': '18.11', 'elapsedTime': '33', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '25201', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15914464::DEBUG::2013-11-20 03:32:15,975::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('163a3426-5ec5-46a5-b8f1-af06fc5d7e7d',) {}
>Thread-15914464::DEBUG::2013-11-20 03:32:15,975::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::CPU stats not available
>Thread-15914464::DEBUG::2013-11-20 03:32:15,976::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::Disk stats not available
>Thread-15914464::DEBUG::2013-11-20 03:32:15,977::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::Disk latency not available
>Thread-15914464::DEBUG::2013-11-20 03:32:15,978::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '5851', 'displayIp': '0', 'displayPort': u'5914', 'session': 'Unknown', 'displaySecurePort': u'5915', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet8': {'macAddr': u'00:1a:4a:a8:64:15', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet8'}}, 'vmId': '163a3426-5ec5-46a5-b8f1-af06fc5d7e7d', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '53821308928', 'writeLatency': '0', 'imageID': '5f930ef3-763f-4294-a62a-6d52df220f8c', 'flushLatency': '0', 'truesize': '53821308928'}}, 'boot': 'c', 'statsAge': '0.61', 'cpuIdle': '100.00', 'elapsedTime': '18', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15914465::DEBUG::2013-11-20 03:32:16,861::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914465::INFO::2013-11-20 03:32:16,862::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914465::DEBUG::2013-11-20 03:32:16,863::task::495::TaskManager.Task::(_debug) Task 77d54c13-52d5-49f7-afe7-8ad1a1f4cb1a: moving from state init -> state preparing
>Thread-15914465::DEBUG::2013-11-20 03:32:16,863::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914465::DEBUG::2013-11-20 03:32:16,864::task::495::TaskManager.Task::(_debug) Task 77d54c13-52d5-49f7-afe7-8ad1a1f4cb1a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914465::DEBUG::2013-11-20 03:32:16,864::task::495::TaskManager.Task::(_debug) Task 77d54c13-52d5-49f7-afe7-8ad1a1f4cb1a: moving from state preparing -> state finished
>Thread-15914465::DEBUG::2013-11-20 03:32:16,865::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914465::DEBUG::2013-11-20 03:32:16,865::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914465::DEBUG::2013-11-20 03:32:16,865::task::495::TaskManager.Task::(_debug) Task 77d54c13-52d5-49f7-afe7-8ad1a1f4cb1a: ref 0 aborting False
>Thread-15914465::INFO::2013-11-20 03:32:16,866::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914466::DEBUG::2013-11-20 03:32:16,871::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914466::INFO::2013-11-20 03:32:16,871::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914466::DEBUG::2013-11-20 03:32:16,872::task::495::TaskManager.Task::(_debug) Task d023d428-aaee-4340-948e-60569e6441f8: moving from state init -> state preparing
>Thread-15914466::DEBUG::2013-11-20 03:32:16,873::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`aa5b793f-5224-403d-8276-14ee46515617`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914466::DEBUG::2013-11-20 03:32:16,873::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914466::DEBUG::2013-11-20 03:32:16,874::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914466::DEBUG::2013-11-20 03:32:16,875::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`aa5b793f-5224-403d-8276-14ee46515617`::Granted request
>Thread-15914466::DEBUG::2013-11-20 03:32:16,876::task::495::TaskManager.Task::(_debug) Task d023d428-aaee-4340-948e-60569e6441f8: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914466::DEBUG::2013-11-20 03:32:16,876::task::495::TaskManager.Task::(_debug) Task d023d428-aaee-4340-948e-60569e6441f8: ref 1 aborting False
>Thread-15914466::WARNING::2013-11-20 03:32:16,878::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914466::DEBUG::2013-11-20 03:32:16,879::task::495::TaskManager.Task::(_debug) Task d023d428-aaee-4340-948e-60569e6441f8: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914466::DEBUG::2013-11-20 03:32:16,879::task::495::TaskManager.Task::(_debug) Task d023d428-aaee-4340-948e-60569e6441f8: moving from state preparing -> state finished
>Thread-15914466::DEBUG::2013-11-20 03:32:16,879::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914466::DEBUG::2013-11-20 03:32:16,880::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914466::DEBUG::2013-11-20 03:32:16,880::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914466::DEBUG::2013-11-20 03:32:16,881::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914466::DEBUG::2013-11-20 03:32:16,881::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914466::DEBUG::2013-11-20 03:32:16,881::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914466::DEBUG::2013-11-20 03:32:16,882::task::495::TaskManager.Task::(_debug) Task d023d428-aaee-4340-948e-60569e6441f8: ref 0 aborting False
>Thread-15914466::INFO::2013-11-20 03:32:16,882::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:32:17,899::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:32:18,458::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0862256 s, 11.9 MB/s\n'; <rc> = 0
>Thread-15914468::DEBUG::2013-11-20 03:32:19,002::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('5f587d5c-d4b4-4242-b049-ede331c7f8b4',) {}
>Thread-15914468::DEBUG::2013-11-20 03:32:19,003::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Disk stats not available
>Thread-15914468::DEBUG::2013-11-20 03:32:19,004::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Disk latency not available
>Thread-15914468::DEBUG::2013-11-20 03:32:19,005::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '5523', 'displayIp': '0', 'displayPort': u'5912', 'session': 'Unknown', 'displaySecurePort': u'5913', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet7': {'macAddr': u'00:1a:4a:a8:64:17', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet7'}}, 'vmId': '5f587d5c-d4b4-4242-b049-ede331c7f8b4', 'monitorResponse': '0', 'cpuUser': '81.89', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '56908316672', 'writeLatency': '0', 'imageID': '143cb26d-de08-4d30-89cf-aee10245b66a', 'flushLatency': '0', 'truesize': '56908316672'}}, 'boot': 'd', 'statsAge': '2.59', 'cpuIdle': '18.11', 'elapsedTime': '36', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '25201', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15914469::DEBUG::2013-11-20 03:32:19,011::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('163a3426-5ec5-46a5-b8f1-af06fc5d7e7d',) {}
>Thread-15914469::DEBUG::2013-11-20 03:32:19,012::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::CPU stats not available
>Thread-15914469::DEBUG::2013-11-20 03:32:19,013::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::Disk stats not available
>Thread-15914469::DEBUG::2013-11-20 03:32:19,013::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::Disk latency not available
>Thread-15914469::DEBUG::2013-11-20 03:32:19,014::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '5851', 'displayIp': '0', 'displayPort': u'5914', 'session': 'Unknown', 'displaySecurePort': u'5915', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet8': {'macAddr': u'00:1a:4a:a8:64:15', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet8'}}, 'vmId': '163a3426-5ec5-46a5-b8f1-af06fc5d7e7d', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '53821308928', 'writeLatency': '0', 'imageID': '5f930ef3-763f-4294-a62a-6d52df220f8c', 'flushLatency': '0', 'truesize': '53821308928'}}, 'boot': 'c', 'statsAge': '0.64', 'cpuIdle': '100.00', 'elapsedTime': '21', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:32:20,475::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:32:21,012::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0492122 s, 20.8 MB/s\n'; <rc> = 0
>Thread-15914470::DEBUG::2013-11-20 03:32:21,132::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Thread-15913546::INFO::2013-11-20 03:32:21,137::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:32:21,139::task::495::TaskManager.Task::(_debug) Task 20d527f3-79bc-45f9-9939-76665de188ad: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:32:21,369::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f58ac0e2-c6ad-4c0b-9a3b-bc42b75ce6f5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:32:21,371::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:32:21,372::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:32:21,372::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f58ac0e2-c6ad-4c0b-9a3b-bc42b75ce6f5`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:32:21,373::task::495::TaskManager.Task::(_debug) Task 20d527f3-79bc-45f9-9939-76665de188ad: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:32:21,374::task::495::TaskManager.Task::(_debug) Task 20d527f3-79bc-45f9-9939-76665de188ad: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:32:21,375::task::495::TaskManager.Task::(_debug) Task 20d527f3-79bc-45f9-9939-76665de188ad: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:32:21,375::task::495::TaskManager.Task::(_debug) Task 20d527f3-79bc-45f9-9939-76665de188ad: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:32:21,376::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:32:21,376::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:32:21,377::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:32:21,377::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:32:21,377::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:32:21,378::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:32:21,378::task::495::TaskManager.Task::(_debug) Task 20d527f3-79bc-45f9-9939-76665de188ad: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:32:21,378::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:32:21,379::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:32:21,380::task::495::TaskManager.Task::(_debug) Task 51816d1a-badc-4a2b-9bf1-7bc68bb8d102: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:32:21,381::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`af74b6eb-9a79-4348-888c-3f625ba62182`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:32:21,381::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:32:21,382::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:32:21,382::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`af74b6eb-9a79-4348-888c-3f625ba62182`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:32:21,383::task::495::TaskManager.Task::(_debug) Task 51816d1a-badc-4a2b-9bf1-7bc68bb8d102: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:32:21,383::task::495::TaskManager.Task::(_debug) Task 51816d1a-badc-4a2b-9bf1-7bc68bb8d102: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:32:21,384::task::495::TaskManager.Task::(_debug) Task 51816d1a-badc-4a2b-9bf1-7bc68bb8d102: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:32:21,384::task::495::TaskManager.Task::(_debug) Task 51816d1a-badc-4a2b-9bf1-7bc68bb8d102: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:32:21,385::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:32:21,385::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:32:21,385::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:32:21,386::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:32:21,386::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:32:21,386::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:32:21,387::task::495::TaskManager.Task::(_debug) Task 51816d1a-badc-4a2b-9bf1-7bc68bb8d102: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:32:21,387::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:32:21,388::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:32:21,388::task::495::TaskManager.Task::(_debug) Task 61d1471f-3043-4ff5-b81d-24c8594ae2ac: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:32:21,389::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b6c1ecbc-3c98-46e4-977d-ed766400a9a1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:32:21,389::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:32:21,389::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:32:21,390::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b6c1ecbc-3c98-46e4-977d-ed766400a9a1`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:32:21,390::task::495::TaskManager.Task::(_debug) Task 61d1471f-3043-4ff5-b81d-24c8594ae2ac: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:32:21,391::task::495::TaskManager.Task::(_debug) Task 61d1471f-3043-4ff5-b81d-24c8594ae2ac: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:32:21,391::task::495::TaskManager.Task::(_debug) Task 61d1471f-3043-4ff5-b81d-24c8594ae2ac: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:32:21,391::task::495::TaskManager.Task::(_debug) Task 61d1471f-3043-4ff5-b81d-24c8594ae2ac: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:32:21,392::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:32:21,392::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:32:21,392::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:32:21,393::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:32:21,393::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:32:21,393::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:32:21,394::task::495::TaskManager.Task::(_debug) Task 61d1471f-3043-4ff5-b81d-24c8594ae2ac: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:32:21,394::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:32:21,395::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:32:21,395::task::495::TaskManager.Task::(_debug) Task ea18323e-d475-4f29-9a4c-8253aa1f6179: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:32:21,396::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c7b16ad3-8de1-4449-b6f2-5b06bdebaa52`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:32:21,396::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:32:21,396::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:32:21,397::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c7b16ad3-8de1-4449-b6f2-5b06bdebaa52`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:32:21,397::task::495::TaskManager.Task::(_debug) Task ea18323e-d475-4f29-9a4c-8253aa1f6179: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:32:21,398::task::495::TaskManager.Task::(_debug) Task ea18323e-d475-4f29-9a4c-8253aa1f6179: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:32:21,398::task::495::TaskManager.Task::(_debug) Task ea18323e-d475-4f29-9a4c-8253aa1f6179: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:32:21,398::task::495::TaskManager.Task::(_debug) Task ea18323e-d475-4f29-9a4c-8253aa1f6179: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:32:21,399::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:32:21,399::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:32:21,399::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:32:21,400::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:32:21,400::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:32:21,400::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:32:21,401::task::495::TaskManager.Task::(_debug) Task ea18323e-d475-4f29-9a4c-8253aa1f6179: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:32:21,401::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15914470::DEBUG::2013-11-20 03:32:21,818::clientIF::1219::vds::(fenceNode) rc 2 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = Off
>Done
> err 
>Thread-15914471::DEBUG::2013-11-20 03:32:21,824::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Thread-15914472::INFO::2013-11-20 03:32:22,104::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914472::DEBUG::2013-11-20 03:32:22,121::task::495::TaskManager.Task::(_debug) Task 21b1da7c-d2ef-4eb6-8bef-a2a14bd5cdb3: moving from state init -> state preparing
>Thread-15914472::DEBUG::2013-11-20 03:32:22,123::task::495::TaskManager.Task::(_debug) Task 21b1da7c-d2ef-4eb6-8bef-a2a14bd5cdb3: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000314950942993', 'lastCheck': 1384918340.0713339, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00159001350403', 'lastCheck': 1384918335.3766799, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00109195709229', 'lastCheck': 1384918337.3727779, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00300216674805', 'lastCheck': 1384918338.136579, 'valid': True, 'code': 0}}
>Thread-15914472::DEBUG::2013-11-20 03:32:22,154::task::495::TaskManager.Task::(_debug) Task 21b1da7c-d2ef-4eb6-8bef-a2a14bd5cdb3: moving from state preparing -> state finished
>Thread-15914472::DEBUG::2013-11-20 03:32:22,166::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914472::DEBUG::2013-11-20 03:32:22,172::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914472::DEBUG::2013-11-20 03:32:22,181::task::495::TaskManager.Task::(_debug) Task 21b1da7c-d2ef-4eb6-8bef-a2a14bd5cdb3: ref 0 aborting False
>Thread-15914472::INFO::2013-11-20 03:32:22,190::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000314950942993', 'lastCheck': 1384918340.0713339, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00159001350403', 'lastCheck': 1384918335.3766799, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00109195709229', 'lastCheck': 1384918337.3727779, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00300216674805', 'lastCheck': 1384918338.136579, 'valid': True, 'code': 0}}
>Thread-15914473::DEBUG::2013-11-20 03:32:22,336::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::CPU stats not available
>Thread-15914473::DEBUG::2013-11-20 03:32:22,348::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::Disk stats not available
>Thread-15914473::DEBUG::2013-11-20 03:32:22,355::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::Disk latency not available
>Thread-15914473::DEBUG::2013-11-20 03:32:22,379::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Disk stats not available
>Thread-15914473::DEBUG::2013-11-20 03:32:22,386::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Disk latency not available
>Thread-15914471::DEBUG::2013-11-20 03:32:22,457::clientIF::1219::vds::(fenceNode) rc 2 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = Off
>Done
> err 
>Dummy-15912651::DEBUG::2013-11-20 03:32:23,023::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:32:24,046::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0465804 s, 22.0 MB/s\n'; <rc> = 0
>Thread-15914475::DEBUG::2013-11-20 03:32:25,733::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('5f587d5c-d4b4-4242-b049-ede331c7f8b4',) {}
>Thread-15914475::DEBUG::2013-11-20 03:32:25,745::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Disk stats not available
>Thread-15914475::DEBUG::2013-11-20 03:32:25,762::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Disk latency not available
>Thread-15914475::DEBUG::2013-11-20 03:32:25,775::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '5523', 'displayIp': '0', 'displayPort': u'5912', 'session': 'Unknown', 'displaySecurePort': u'5913', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet7': {'macAddr': u'00:1a:4a:a8:64:17', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet7'}}, 'vmId': '5f587d5c-d4b4-4242-b049-ede331c7f8b4', 'monitorResponse': '0', 'cpuUser': '81.89', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '56908316672', 'writeLatency': '0', 'imageID': '143cb26d-de08-4d30-89cf-aee10245b66a', 'flushLatency': '0', 'truesize': '56908316672'}}, 'boot': 'd', 'statsAge': '2.36', 'cpuIdle': '18.11', 'elapsedTime': '42', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '25201', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15914476::DEBUG::2013-11-20 03:32:25,835::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('163a3426-5ec5-46a5-b8f1-af06fc5d7e7d',) {}
>Thread-15914476::DEBUG::2013-11-20 03:32:25,837::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::Disk stats not available
>Thread-15914476::DEBUG::2013-11-20 03:32:25,838::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::Disk latency not available
>Thread-15914476::DEBUG::2013-11-20 03:32:25,838::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '5851', 'displayIp': '0', 'displayPort': u'5914', 'session': 'Unknown', 'displaySecurePort': u'5915', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet8': {'macAddr': u'00:1a:4a:a8:64:15', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet8'}}, 'vmId': '163a3426-5ec5-46a5-b8f1-af06fc5d7e7d', 'monitorResponse': '0', 'cpuUser': '59.86', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '53821308928', 'writeLatency': '0', 'imageID': '5f930ef3-763f-4294-a62a-6d52df220f8c', 'flushLatency': '0', 'truesize': '53821308928'}}, 'boot': 'c', 'statsAge': '0.45', 'cpuIdle': '40.14', 'elapsedTime': '28', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:32:26,056::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:32:26,608::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0927764 s, 11.0 MB/s\n'; <rc> = 0
>Thread-15914477::DEBUG::2013-11-20 03:32:27,039::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914477::INFO::2013-11-20 03:32:27,040::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914477::DEBUG::2013-11-20 03:32:27,041::task::495::TaskManager.Task::(_debug) Task b25a48b1-5c97-4369-8139-02e5895f06f9: moving from state init -> state preparing
>Thread-15914477::DEBUG::2013-11-20 03:32:27,041::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914477::DEBUG::2013-11-20 03:32:27,042::task::495::TaskManager.Task::(_debug) Task b25a48b1-5c97-4369-8139-02e5895f06f9: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914477::DEBUG::2013-11-20 03:32:27,042::task::495::TaskManager.Task::(_debug) Task b25a48b1-5c97-4369-8139-02e5895f06f9: moving from state preparing -> state finished
>Thread-15914477::DEBUG::2013-11-20 03:32:27,043::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914477::DEBUG::2013-11-20 03:32:27,043::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914477::DEBUG::2013-11-20 03:32:27,044::task::495::TaskManager.Task::(_debug) Task b25a48b1-5c97-4369-8139-02e5895f06f9: ref 0 aborting False
>Thread-15914477::INFO::2013-11-20 03:32:27,044::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914478::DEBUG::2013-11-20 03:32:27,048::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914478::INFO::2013-11-20 03:32:27,049::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914478::DEBUG::2013-11-20 03:32:27,050::task::495::TaskManager.Task::(_debug) Task d87a59dd-021e-4d39-8006-81bfa6c29951: moving from state init -> state preparing
>Thread-15914478::DEBUG::2013-11-20 03:32:27,051::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b21d3a7e-a6a5-42b6-9d6c-65dd3b7cb4c6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914478::DEBUG::2013-11-20 03:32:27,052::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914478::DEBUG::2013-11-20 03:32:27,053::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914478::DEBUG::2013-11-20 03:32:27,053::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b21d3a7e-a6a5-42b6-9d6c-65dd3b7cb4c6`::Granted request
>Thread-15914478::DEBUG::2013-11-20 03:32:27,054::task::495::TaskManager.Task::(_debug) Task d87a59dd-021e-4d39-8006-81bfa6c29951: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914478::DEBUG::2013-11-20 03:32:27,055::task::495::TaskManager.Task::(_debug) Task d87a59dd-021e-4d39-8006-81bfa6c29951: ref 1 aborting False
>Thread-15914478::WARNING::2013-11-20 03:32:27,057::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914478::DEBUG::2013-11-20 03:32:27,058::task::495::TaskManager.Task::(_debug) Task d87a59dd-021e-4d39-8006-81bfa6c29951: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914478::DEBUG::2013-11-20 03:32:27,058::task::495::TaskManager.Task::(_debug) Task d87a59dd-021e-4d39-8006-81bfa6c29951: moving from state preparing -> state finished
>Thread-15914478::DEBUG::2013-11-20 03:32:27,059::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914478::DEBUG::2013-11-20 03:32:27,059::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914478::DEBUG::2013-11-20 03:32:27,060::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914478::DEBUG::2013-11-20 03:32:27,061::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914478::DEBUG::2013-11-20 03:32:27,061::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914478::DEBUG::2013-11-20 03:32:27,062::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914478::DEBUG::2013-11-20 03:32:27,063::task::495::TaskManager.Task::(_debug) Task d87a59dd-021e-4d39-8006-81bfa6c29951: ref 0 aborting False
>Thread-15914478::INFO::2013-11-20 03:32:27,063::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:32:28,621::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914480::DEBUG::2013-11-20 03:32:28,891::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('5f587d5c-d4b4-4242-b049-ede331c7f8b4',) {}
>Thread-15914480::DEBUG::2013-11-20 03:32:28,893::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Disk stats not available
>Thread-15914480::DEBUG::2013-11-20 03:32:28,894::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Disk latency not available
>Thread-15914480::DEBUG::2013-11-20 03:32:28,895::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '5523', 'displayIp': '0', 'displayPort': u'5912', 'session': 'Unknown', 'displaySecurePort': u'5913', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet7': {'macAddr': u'00:1a:4a:a8:64:17', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet7'}}, 'vmId': '5f587d5c-d4b4-4242-b049-ede331c7f8b4', 'monitorResponse': '0', 'cpuUser': '188.92', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '56908316672', 'writeLatency': '0', 'imageID': '143cb26d-de08-4d30-89cf-aee10245b66a', 'flushLatency': '0', 'truesize': '56908316672'}}, 'boot': 'd', 'statsAge': '0.86', 'cpuIdle': '0.00', 'elapsedTime': '46', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '25201', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15914481::DEBUG::2013-11-20 03:32:28,901::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('163a3426-5ec5-46a5-b8f1-af06fc5d7e7d',) {}
>Thread-15914481::DEBUG::2013-11-20 03:32:28,902::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::Disk stats not available
>Thread-15914481::DEBUG::2013-11-20 03:32:28,902::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::Disk latency not available
>Thread-15914481::DEBUG::2013-11-20 03:32:28,903::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '5851', 'displayIp': '0', 'displayPort': u'5914', 'session': 'Unknown', 'displaySecurePort': u'5915', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet8': {'macAddr': u'00:1a:4a:a8:64:15', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet8'}}, 'vmId': '163a3426-5ec5-46a5-b8f1-af06fc5d7e7d', 'monitorResponse': '0', 'cpuUser': '59.86', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '53821308928', 'writeLatency': '0', 'imageID': '5f930ef3-763f-4294-a62a-6d52df220f8c', 'flushLatency': '0', 'truesize': '53821308928'}}, 'boot': 'c', 'statsAge': '0.50', 'cpuIdle': '40.14', 'elapsedTime': '31', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:32:29,236::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0862576 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:32:31,251::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914483::DEBUG::2013-11-20 03:32:31,940::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('5f587d5c-d4b4-4242-b049-ede331c7f8b4',) {}
>Thread-15914483::DEBUG::2013-11-20 03:32:31,941::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Disk stats not available
>Thread-15914483::DEBUG::2013-11-20 03:32:31,941::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Disk latency not available
>Thread-15914483::DEBUG::2013-11-20 03:32:31,942::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '5523', 'displayIp': '0', 'displayPort': u'5912', 'session': 'Unknown', 'displaySecurePort': u'5913', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet7': {'macAddr': u'00:1a:4a:a8:64:17', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet7'}}, 'vmId': '5f587d5c-d4b4-4242-b049-ede331c7f8b4', 'monitorResponse': '0', 'cpuUser': '188.92', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '56908316672', 'writeLatency': '0', 'imageID': '143cb26d-de08-4d30-89cf-aee10245b66a', 'flushLatency': '0', 'truesize': '56908316672'}}, 'boot': 'd', 'statsAge': '1.69', 'cpuIdle': '0.00', 'elapsedTime': '49', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '25201', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15914484::DEBUG::2013-11-20 03:32:31,947::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('163a3426-5ec5-46a5-b8f1-af06fc5d7e7d',) {}
>Thread-15914484::DEBUG::2013-11-20 03:32:31,947::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::Disk stats not available
>Thread-15914484::DEBUG::2013-11-20 03:32:31,948::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::Disk latency not available
>Thread-15914484::DEBUG::2013-11-20 03:32:31,948::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '5851', 'displayIp': '0', 'displayPort': u'5914', 'session': 'Unknown', 'displaySecurePort': u'5915', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet8': {'macAddr': u'00:1a:4a:a8:64:15', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet8'}}, 'vmId': '163a3426-5ec5-46a5-b8f1-af06fc5d7e7d', 'monitorResponse': '0', 'cpuUser': '59.86', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '53821308928', 'writeLatency': '0', 'imageID': '5f930ef3-763f-4294-a62a-6d52df220f8c', 'flushLatency': '0', 'truesize': '53821308928'}}, 'boot': 'c', 'statsAge': '1.55', 'cpuIdle': '40.14', 'elapsedTime': '34', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15913639::INFO::2013-11-20 03:32:32,309::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:32:32,310::task::495::TaskManager.Task::(_debug) Task 76ebffb5-efa6-4a66-a655-adff85194c91: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:32:32,313::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8a43c8ca-be1b-4087-977c-2995c2ebbd25`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:32:32,314::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:32:32,315::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:32:32,316::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8a43c8ca-be1b-4087-977c-2995c2ebbd25`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:32:32,329::task::495::TaskManager.Task::(_debug) Task 76ebffb5-efa6-4a66-a655-adff85194c91: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:32:32,330::task::495::TaskManager.Task::(_debug) Task 76ebffb5-efa6-4a66-a655-adff85194c91: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:32:32,331::task::495::TaskManager.Task::(_debug) Task 76ebffb5-efa6-4a66-a655-adff85194c91: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:32:32,332::task::495::TaskManager.Task::(_debug) Task 76ebffb5-efa6-4a66-a655-adff85194c91: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:32:32,333::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:32:32,334::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:32:32,335::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:32:32,336::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:32:32,338::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:32:32,338::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:32:32,340::task::495::TaskManager.Task::(_debug) Task 76ebffb5-efa6-4a66-a655-adff85194c91: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:32:32,341::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:32:32,506::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.126347 s, 8.1 MB/s\n'; <rc> = 0
>Thread-15914485::DEBUG::2013-11-20 03:32:32,538::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Thread-15914485::DEBUG::2013-11-20 03:32:33,650::clientIF::1219::vds::(fenceNode) rc 2 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = Off
>Done
> err 
>Thread-15914486::DEBUG::2013-11-20 03:32:33,655::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Thread-15914486::DEBUG::2013-11-20 03:32:34,409::clientIF::1219::vds::(fenceNode) rc 2 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = Off
>Done
> err 
>Dummy-15912651::DEBUG::2013-11-20 03:32:34,521::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914488::DEBUG::2013-11-20 03:32:34,976::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('5f587d5c-d4b4-4242-b049-ede331c7f8b4',) {}
>Thread-15914488::DEBUG::2013-11-20 03:32:34,978::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Disk stats not available
>Thread-15914488::DEBUG::2013-11-20 03:32:34,978::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Disk latency not available
>Thread-15914488::DEBUG::2013-11-20 03:32:34,979::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '5523', 'displayIp': '0', 'displayPort': u'5912', 'session': 'Unknown', 'displaySecurePort': u'5913', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet7': {'macAddr': u'00:1a:4a:a8:64:17', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet7'}}, 'vmId': '5f587d5c-d4b4-4242-b049-ede331c7f8b4', 'monitorResponse': '0', 'cpuUser': '188.92', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '56908316672', 'writeLatency': '0', 'imageID': '143cb26d-de08-4d30-89cf-aee10245b66a', 'flushLatency': '0', 'truesize': '56908316672'}}, 'boot': 'd', 'statsAge': '0.11', 'cpuIdle': '0.00', 'elapsedTime': '52', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '25201', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15914489::DEBUG::2013-11-20 03:32:34,985::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('163a3426-5ec5-46a5-b8f1-af06fc5d7e7d',) {}
>Thread-15914489::DEBUG::2013-11-20 03:32:34,986::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::Disk stats not available
>Thread-15914489::DEBUG::2013-11-20 03:32:34,987::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::Disk latency not available
>Thread-15914489::DEBUG::2013-11-20 03:32:34,988::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '5851', 'displayIp': '0', 'displayPort': u'5914', 'session': 'Unknown', 'displaySecurePort': u'5915', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet8': {'macAddr': u'00:1a:4a:a8:64:15', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet8'}}, 'vmId': '163a3426-5ec5-46a5-b8f1-af06fc5d7e7d', 'monitorResponse': '0', 'cpuUser': '59.86', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '53821308928', 'writeLatency': '0', 'imageID': '5f930ef3-763f-4294-a62a-6d52df220f8c', 'flushLatency': '0', 'truesize': '53821308928'}}, 'boot': 'c', 'statsAge': '0.58', 'cpuIdle': '40.14', 'elapsedTime': '37', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:32:35,305::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0501967 s, 20.4 MB/s\n'; <rc> = 0
>Thread-15914490::DEBUG::2013-11-20 03:32:37,185::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914490::INFO::2013-11-20 03:32:37,187::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914490::DEBUG::2013-11-20 03:32:37,187::task::495::TaskManager.Task::(_debug) Task 631708b4-98d4-4832-86cc-3f5bb04bdb74: moving from state init -> state preparing
>Thread-15914490::DEBUG::2013-11-20 03:32:37,188::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914490::DEBUG::2013-11-20 03:32:37,189::task::495::TaskManager.Task::(_debug) Task 631708b4-98d4-4832-86cc-3f5bb04bdb74: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914490::DEBUG::2013-11-20 03:32:37,190::task::495::TaskManager.Task::(_debug) Task 631708b4-98d4-4832-86cc-3f5bb04bdb74: moving from state preparing -> state finished
>Thread-15914490::DEBUG::2013-11-20 03:32:37,190::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914490::DEBUG::2013-11-20 03:32:37,191::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914490::DEBUG::2013-11-20 03:32:37,192::task::495::TaskManager.Task::(_debug) Task 631708b4-98d4-4832-86cc-3f5bb04bdb74: ref 0 aborting False
>Thread-15914490::INFO::2013-11-20 03:32:37,192::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914491::DEBUG::2013-11-20 03:32:37,197::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914491::INFO::2013-11-20 03:32:37,197::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914491::DEBUG::2013-11-20 03:32:37,198::task::495::TaskManager.Task::(_debug) Task 7882378a-b52c-4c8c-9924-0afbc74d492e: moving from state init -> state preparing
>Thread-15914491::DEBUG::2013-11-20 03:32:37,199::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d494dd46-7ebe-4744-9d3a-0d3d43fcacc8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914491::DEBUG::2013-11-20 03:32:37,200::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914491::DEBUG::2013-11-20 03:32:37,201::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914491::DEBUG::2013-11-20 03:32:37,201::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d494dd46-7ebe-4744-9d3a-0d3d43fcacc8`::Granted request
>Thread-15914491::DEBUG::2013-11-20 03:32:37,203::task::495::TaskManager.Task::(_debug) Task 7882378a-b52c-4c8c-9924-0afbc74d492e: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914491::DEBUG::2013-11-20 03:32:37,203::task::495::TaskManager.Task::(_debug) Task 7882378a-b52c-4c8c-9924-0afbc74d492e: ref 1 aborting False
>Thread-15914491::WARNING::2013-11-20 03:32:37,205::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914491::DEBUG::2013-11-20 03:32:37,206::task::495::TaskManager.Task::(_debug) Task 7882378a-b52c-4c8c-9924-0afbc74d492e: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914491::DEBUG::2013-11-20 03:32:37,207::task::495::TaskManager.Task::(_debug) Task 7882378a-b52c-4c8c-9924-0afbc74d492e: moving from state preparing -> state finished
>Thread-15914491::DEBUG::2013-11-20 03:32:37,208::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914491::DEBUG::2013-11-20 03:32:37,208::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914491::DEBUG::2013-11-20 03:32:37,209::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914491::DEBUG::2013-11-20 03:32:37,210::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914491::DEBUG::2013-11-20 03:32:37,210::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914491::DEBUG::2013-11-20 03:32:37,211::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914491::DEBUG::2013-11-20 03:32:37,211::task::495::TaskManager.Task::(_debug) Task 7882378a-b52c-4c8c-9924-0afbc74d492e: ref 0 aborting False
>Thread-15914491::INFO::2013-11-20 03:32:37,212::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:32:37,314::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:32:37,833::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0516247 s, 19.8 MB/s\n'; <rc> = 0
>Thread-15914492::INFO::2013-11-20 03:32:38,008::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914492::DEBUG::2013-11-20 03:32:38,009::task::495::TaskManager.Task::(_debug) Task aa4d4e37-ebcb-4484-a3b6-a522f9b8f8b8: moving from state init -> state preparing
>Thread-15914492::DEBUG::2013-11-20 03:32:38,009::task::495::TaskManager.Task::(_debug) Task aa4d4e37-ebcb-4484-a3b6-a522f9b8f8b8: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000453948974609', 'lastCheck': 1384918350.2863319, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00128388404846', 'lastCheck': 1384918355.493988, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00121712684631', 'lastCheck': 1384918357.4968879, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000419139862061', 'lastCheck': 1384918348.2810299, 'valid': True, 'code': 0}}
>Thread-15914492::DEBUG::2013-11-20 03:32:38,010::task::495::TaskManager.Task::(_debug) Task aa4d4e37-ebcb-4484-a3b6-a522f9b8f8b8: moving from state preparing -> state finished
>Thread-15914492::DEBUG::2013-11-20 03:32:38,011::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914492::DEBUG::2013-11-20 03:32:38,011::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914492::DEBUG::2013-11-20 03:32:38,012::task::495::TaskManager.Task::(_debug) Task aa4d4e37-ebcb-4484-a3b6-a522f9b8f8b8: ref 0 aborting False
>Thread-15914492::INFO::2013-11-20 03:32:38,012::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000453948974609', 'lastCheck': 1384918350.2863319, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00128388404846', 'lastCheck': 1384918355.493988, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00121712684631', 'lastCheck': 1384918357.4968879, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000419139862061', 'lastCheck': 1384918348.2810299, 'valid': True, 'code': 0}}
>Thread-15914493::DEBUG::2013-11-20 03:32:38,058::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::Disk stats not available
>Thread-15914493::DEBUG::2013-11-20 03:32:38,058::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::Disk latency not available
>Thread-15914493::DEBUG::2013-11-20 03:32:38,060::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Disk stats not available
>Thread-15914493::DEBUG::2013-11-20 03:32:38,061::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Disk latency not available
>Dummy-15912651::DEBUG::2013-11-20 03:32:39,843::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:32:40,281::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0523558 s, 19.6 MB/s\n'; <rc> = 0
>Thread-15914495::DEBUG::2013-11-20 03:32:41,511::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('5f587d5c-d4b4-4242-b049-ede331c7f8b4',) {}
>Thread-15914495::DEBUG::2013-11-20 03:32:41,512::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Disk stats not available
>Thread-15914495::DEBUG::2013-11-20 03:32:41,512::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Disk latency not available
>Thread-15914495::DEBUG::2013-11-20 03:32:41,513::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '5523', 'displayIp': '0', 'displayPort': u'5912', 'session': 'Unknown', 'displaySecurePort': u'5913', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet7': {'macAddr': u'00:1a:4a:a8:64:17', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet7'}}, 'vmId': '5f587d5c-d4b4-4242-b049-ede331c7f8b4', 'monitorResponse': '0', 'cpuUser': '188.92', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '56908316672', 'writeLatency': '0', 'imageID': '143cb26d-de08-4d30-89cf-aee10245b66a', 'flushLatency': '0', 'truesize': '56908316672'}}, 'boot': 'd', 'statsAge': '0.21', 'cpuIdle': '0.00', 'elapsedTime': '58', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '25201', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15914496::DEBUG::2013-11-20 03:32:41,518::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('163a3426-5ec5-46a5-b8f1-af06fc5d7e7d',) {}
>Thread-15914496::DEBUG::2013-11-20 03:32:41,520::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::Disk stats not available
>Thread-15914496::DEBUG::2013-11-20 03:32:41,521::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::Disk latency not available
>Thread-15914496::DEBUG::2013-11-20 03:32:41,522::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '5851', 'displayIp': '0', 'displayPort': u'5914', 'session': 'Unknown', 'displaySecurePort': u'5915', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet8': {'macAddr': u'00:1a:4a:a8:64:15', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet8'}}, 'vmId': '163a3426-5ec5-46a5-b8f1-af06fc5d7e7d', 'monitorResponse': '0', 'cpuUser': '73.22', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '53821308928', 'writeLatency': '0', 'imageID': '5f930ef3-763f-4294-a62a-6d52df220f8c', 'flushLatency': '0', 'truesize': '53821308928'}}, 'boot': 'c', 'statsAge': '1.11', 'cpuIdle': '26.78', 'elapsedTime': '43', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15914497::DEBUG::2013-11-20 03:32:42,276::clientIF::54::vds::(wrapper) [192.168.100.135]::call create with ({'bridge': 'rhevm', 'acpiEnable': 'true', 'emulatedMachine': 'rhel6.2.0', 'vmId': 'e97d1c52-b688-4192-bdeb-2dfadcc79786', 'transparentHugePages': 'true', 'spiceSslCipherSuite': 'DEFAULT', 'cpuType': 'Westmere', 'smp': '4', 'macAddr': '00:1a:4a:a8:64:13', 'boot': 'c', 'custom': {}, 'vmType': 'kvm', 'memSize': 4500, 'smpCoresPerSocket': '1', 'vmName': 'sukhumi', 'spiceMonitors': '1', 'nice': '0', 'drives': [{'index': '0', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'format': 'raw', 'volumeID': '8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f', 'imageID': '5af747d8-95b1-4b7e-af53-cb77e4fcc896', 'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'propagateErrors': 'off', 'if': 'ide'}], 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/RHEV-toolsSetup_3.0_39.iso', 'smartcardEnable': 'false', 'nicModel': 'pv', 'keyboardLayout': 'en-us', 'kvmEnable': 'true', 'timeOffset': '0', 'spiceSecureChannels': 'smain,sinputs', 'display': 'qxl'},) {}
>Dummy-15912651::DEBUG::2013-11-20 03:32:42,452::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914497::INFO::2013-11-20 03:32:42,807::clientIF::523::vds::(_getNetworkIp) network None: using 0
>Thread-15914497::INFO::2013-11-20 03:32:42,809::clientIF::656::vds::(create) vmContainerLock acquired by vm e97d1c52-b688-4192-bdeb-2dfadcc79786
>Thread-15914497::DEBUG::2013-11-20 03:32:42,815::clientIF::669::vds::(create) Total desktops after creation of e97d1c52-b688-4192-bdeb-2dfadcc79786 is 9
>Thread-15914498::DEBUG::2013-11-20 03:32:42,815::vm::342::vm.Vm::(_startUnderlyingVm) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::Start
>Thread-15914497::DEBUG::2013-11-20 03:32:42,829::clientIF::59::vds::(wrapper) return create with {'status': {'message': 'Done', 'code': 0}, 'vmList': {'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/RHEV-toolsSetup_3.0_39.iso', 'bridge': 'rhevm', 'acpiEnable': 'true', 'emulatedMachine': 'rhel6.2.0', 'vmId': 'e97d1c52-b688-4192-bdeb-2dfadcc79786', 'pid': '0', 'drives': [{'index': '0', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'format': 'raw', 'volumeID': '8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f', 'imageID': '5af747d8-95b1-4b7e-af53-cb77e4fcc896', 'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'propagateErrors': 'off', 'if': 'ide'}], 'transparentHugePages': 'true', 'keyboardLayout': 'en-us', 'displaySecurePort': '-1', 'displayPort': '-1', 'cpuType': 'Westmere', 'custom': {}, 'clientIp': '', 'nicModel': 'pv', 'macAddr': '00:1a:4a:a8:64:13', 'kvmEnable': 'true', 'smartcardEnable': 'false', 'boot': 'c', 'smp': '4', 'vmType': 'kvm', 'spiceSslCipherSuite': 'DEFAULT', 'status': 'WaitForLaunch', 'timeOffset': '0', 'memSize': 4500, 'displayIp': '0', 'spiceSecureChannels': 'smain,sinputs', 'smpCoresPerSocket': '1', 'vmName': 'sukhumi', 'display': 'qxl', 'spiceMonitors': '1', 'nice': '0'}}
>Thread-15914498::DEBUG::2013-11-20 03:32:42,836::vm::346::vm.Vm::(_startUnderlyingVm) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::_ongoingCreations acquired
>Thread-15914498::INFO::2013-11-20 03:32:42,848::libvirtvm::1061::vm.Vm::(_run) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::VM wrapper has started
>Thread-15914498::INFO::2013-11-20 03:32:42,865::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914498::DEBUG::2013-11-20 03:32:42,866::task::495::TaskManager.Task::(_debug) Task f377e2e2-4ed8-4891-9481-7df918a98f8a: moving from state init -> state preparing
>Thread-15914498::DEBUG::2013-11-20 03:32:42,903::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ae180d1d-0e46-4f2f-aff1-5192bd351520`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1690' at 'public_prepareVolume'
>Thread-15914498::DEBUG::2013-11-20 03:32:42,915::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914498::DEBUG::2013-11-20 03:32:42,928::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914498::DEBUG::2013-11-20 03:32:42,942::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ae180d1d-0e46-4f2f-aff1-5192bd351520`::Granted request
>Thread-15914498::DEBUG::2013-11-20 03:32:42,961::task::495::TaskManager.Task::(_debug) Task f377e2e2-4ed8-4891-9481-7df918a98f8a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914498::DEBUG::2013-11-20 03:32:42,963::task::495::TaskManager.Task::(_debug) Task f377e2e2-4ed8-4891-9481-7df918a98f8a: ref 1 aborting False
>Thread-15914498::DEBUG::2013-11-20 03:32:42,974::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.5af747d8-95b1-4b7e-af53-cb77e4fcc896`ReqID=`b43d2d66-0a05-49bc-9593-246563eeebaf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1695' at 'public_prepareVolume'
>Thread-15914498::DEBUG::2013-11-20 03:32:42,974::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.5af747d8-95b1-4b7e-af53-cb77e4fcc896' for lock type 'exclusive'
>Thread-15914498::DEBUG::2013-11-20 03:32:42,992::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15914498::DEBUG::2013-11-20 03:32:43,017::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:32:43,402::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0781838 s, 13.1 MB/s\n'; <rc> = 0
>Thread-15914498::DEBUG::2013-11-20 03:32:44,382::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:32:45,954::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914500::DEBUG::2013-11-20 03:32:46,886::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Thread-15914502::DEBUG::2013-11-20 03:32:50,858::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914502::INFO::2013-11-20 03:32:54,116::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914502::DEBUG::2013-11-20 03:32:54,194::task::495::TaskManager.Task::(_debug) Task 286296e7-6ae2-4a8a-a4bb-dafbead492a5: moving from state init -> state preparing
>Thread-15914502::DEBUG::2013-11-20 03:32:54,391::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914502::DEBUG::2013-11-20 03:32:54,497::task::495::TaskManager.Task::(_debug) Task 286296e7-6ae2-4a8a-a4bb-dafbead492a5: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914498::DEBUG::2013-11-20 03:32:54,595::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15914502::DEBUG::2013-11-20 03:32:54,603::task::495::TaskManager.Task::(_debug) Task 286296e7-6ae2-4a8a-a4bb-dafbead492a5: moving from state preparing -> state finished
>Thread-15914502::DEBUG::2013-11-20 03:32:54,605::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914498::DEBUG::2013-11-20 03:32:54,610::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=29 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15914502::DEBUG::2013-11-20 03:32:54,611::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914502::DEBUG::2013-11-20 03:32:54,612::task::495::TaskManager.Task::(_debug) Task 286296e7-6ae2-4a8a-a4bb-dafbead492a5: ref 0 aborting False
>Thread-15914502::INFO::2013-11-20 03:32:54,779::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914503::DEBUG::2013-11-20 03:32:54,822::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('5f587d5c-d4b4-4242-b049-ede331c7f8b4',) {}
>Thread-15914503::DEBUG::2013-11-20 03:32:54,824::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Disk stats not available
>Thread-15914504::DEBUG::2013-11-20 03:32:54,829::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914503::DEBUG::2013-11-20 03:32:54,830::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Disk latency not available
>Thread-15914504::INFO::2013-11-20 03:32:54,836::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914503::DEBUG::2013-11-20 03:32:54,836::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '5523', 'displayIp': '0', 'displayPort': u'5912', 'session': 'Unknown', 'displaySecurePort': u'5913', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet7': {'macAddr': u'00:1a:4a:a8:64:17', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet7'}}, 'vmId': '5f587d5c-d4b4-4242-b049-ede331c7f8b4', 'monitorResponse': '0', 'cpuUser': '192.29', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '56908316672', 'writeLatency': '0', 'imageID': '143cb26d-de08-4d30-89cf-aee10245b66a', 'flushLatency': '0', 'truesize': '56908316672'}}, 'boot': 'd', 'statsAge': '5.58', 'cpuIdle': '0.00', 'elapsedTime': '71', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '25201', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15914504::DEBUG::2013-11-20 03:32:54,838::task::495::TaskManager.Task::(_debug) Task 533b0c7b-2bf4-4843-831a-5636a6546cd2: moving from state init -> state preparing
>Thread-15914504::DEBUG::2013-11-20 03:32:54,843::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3504ded7-d43f-4422-b6ef-b2fbaf52edfe`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914504::DEBUG::2013-11-20 03:32:54,843::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914504::DEBUG::2013-11-20 03:32:54,845::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914504::DEBUG::2013-11-20 03:32:54,846::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3504ded7-d43f-4422-b6ef-b2fbaf52edfe`::Granted request
>Thread-15914504::DEBUG::2013-11-20 03:32:54,852::task::495::TaskManager.Task::(_debug) Task 533b0c7b-2bf4-4843-831a-5636a6546cd2: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914504::DEBUG::2013-11-20 03:32:54,852::task::495::TaskManager.Task::(_debug) Task 533b0c7b-2bf4-4843-831a-5636a6546cd2: ref 1 aborting False
>Thread-15914504::WARNING::2013-11-20 03:32:54,863::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914505::DEBUG::2013-11-20 03:32:54,867::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('163a3426-5ec5-46a5-b8f1-af06fc5d7e7d',) {}
>Thread-15914504::DEBUG::2013-11-20 03:32:54,870::task::495::TaskManager.Task::(_debug) Task 533b0c7b-2bf4-4843-831a-5636a6546cd2: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914505::DEBUG::2013-11-20 03:32:54,878::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::Disk stats not available
>Thread-15914504::DEBUG::2013-11-20 03:32:54,883::task::495::TaskManager.Task::(_debug) Task 533b0c7b-2bf4-4843-831a-5636a6546cd2: moving from state preparing -> state finished
>Thread-15914505::DEBUG::2013-11-20 03:32:54,884::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::Disk latency not available
>Thread-15914504::DEBUG::2013-11-20 03:32:54,886::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914505::DEBUG::2013-11-20 03:32:54,891::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '5851', 'displayIp': '0', 'displayPort': u'5914', 'session': 'Unknown', 'displaySecurePort': u'5915', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet8': {'macAddr': u'00:1a:4a:a8:64:15', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet8'}}, 'vmId': '163a3426-5ec5-46a5-b8f1-af06fc5d7e7d', 'monitorResponse': '0', 'cpuUser': '73.22', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '53821308928', 'writeLatency': '0', 'imageID': '5f930ef3-763f-4294-a62a-6d52df220f8c', 'flushLatency': '0', 'truesize': '53821308928'}}, 'boot': 'c', 'statsAge': '1.49', 'cpuIdle': '26.78', 'elapsedTime': '57', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15914504::DEBUG::2013-11-20 03:32:54,892::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914504::DEBUG::2013-11-20 03:32:54,899::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914504::DEBUG::2013-11-20 03:32:54,901::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914504::DEBUG::2013-11-20 03:32:54,901::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914504::DEBUG::2013-11-20 03:32:54,902::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914504::DEBUG::2013-11-20 03:32:54,903::task::495::TaskManager.Task::(_debug) Task 533b0c7b-2bf4-4843-831a-5636a6546cd2: ref 0 aborting False
>Thread-15914504::INFO::2013-11-20 03:32:54,908::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:32:54,996::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 7.11684 s, 144 kB/s\n'; <rc> = 0
>Thread-15914498::DEBUG::2013-11-20 03:32:55,716::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.00870098 s, 58.8 kB/s\n'; <rc> = 0
>Thread-15914498::DEBUG::2013-11-20 03:32:55,716::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.00870098 s, 58.8 kB/s'], size: 512
>Thread-15914498::DEBUG::2013-11-20 03:32:55,718::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=29 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15914498::DEBUG::2013-11-20 03:32:56,042::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000139037 s, 3.7 MB/s\n'; <rc> = 0
>Thread-15914498::DEBUG::2013-11-20 03:32:56,042::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000139037 s, 3.7 MB/s'], size: 512
>Thread-15914498::INFO::2013-11-20 03:32:56,043::image::340::Storage.Image::(getChain) sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 chain=[<storage.blockVolume.BlockVolume instance at 0x7f487c0ab7e8>] 
>Thread-15914498::DEBUG::2013-11-20 03:32:56,047::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --autobackup n --available y 6b892067-ac7e-4039-a7be-5b7387525b16/8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f' (cwd None)
>Thread-15914498::DEBUG::2013-11-20 03:32:56,800::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15914498::DEBUG::2013-11-20 03:32:56,807::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15914498::DEBUG::2013-11-20 03:32:56,807::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15914498::DEBUG::2013-11-20 03:32:56,817::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f`ReqID=`de772223-8613-4c21-ba66-5a617e453a0a`::Request was made in '/usr/share/vdsm/storage/resourceFactories.py' line '149' at '__getResourceCandidatesList'
>Thread-15914498::DEBUG::2013-11-20 03:32:56,818::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f' for lock type 'exclusive'
>Thread-15914498::DEBUG::2013-11-20 03:32:56,822::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f' is free. Now locking as 'exclusive' (1 active user)
>Thread-15914498::DEBUG::2013-11-20 03:32:56,822::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f`ReqID=`de772223-8613-4c21-ba66-5a617e453a0a`::Granted request
>Thread-15914498::DEBUG::2013-11-20 03:32:56,823::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.5af747d8-95b1-4b7e-af53-cb77e4fcc896' is free. Now locking as 'exclusive' (1 active user)
>Thread-15914498::DEBUG::2013-11-20 03:32:56,824::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.5af747d8-95b1-4b7e-af53-cb77e4fcc896`ReqID=`b43d2d66-0a05-49bc-9593-246563eeebaf`::Granted request
>Thread-15914498::DEBUG::2013-11-20 03:32:56,830::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15914498::DEBUG::2013-11-20 03:32:56,845::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:32:57,006::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 03:32:57,396::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:32:57,397::task::495::TaskManager.Task::(_debug) Task 5fbcf04f-9d83-4ec3-ae83-6c1f75a118b5: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:32:57,399::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`53d40546-66a4-406d-a8cb-bf7205e16982`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:32:57,400::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:32:57,401::resourceManager::481::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' found in shared state and queue is empty, Joining current shared lock (2 active users)
>Thread-15913658::DEBUG::2013-11-20 03:32:57,401::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`53d40546-66a4-406d-a8cb-bf7205e16982`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:32:57,402::task::495::TaskManager.Task::(_debug) Task 5fbcf04f-9d83-4ec3-ae83-6c1f75a118b5: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:32:57,402::task::495::TaskManager.Task::(_debug) Task 5fbcf04f-9d83-4ec3-ae83-6c1f75a118b5: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:32:57,403::task::495::TaskManager.Task::(_debug) Task 5fbcf04f-9d83-4ec3-ae83-6c1f75a118b5: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:32:57,403::task::495::TaskManager.Task::(_debug) Task 5fbcf04f-9d83-4ec3-ae83-6c1f75a118b5: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:32:57,404::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:32:57,404::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:32:57,405::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:32:57,405::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (1 active users)
>Thread-15913658::DEBUG::2013-11-20 03:32:57,406::task::495::TaskManager.Task::(_debug) Task 5fbcf04f-9d83-4ec3-ae83-6c1f75a118b5: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:32:57,406::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15914498::DEBUG::2013-11-20 03:32:57,691::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913695::INFO::2013-11-20 03:33:00,219::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:33:00,443::task::495::TaskManager.Task::(_debug) Task 8472d58f-9926-4cbd-8277-15dbf3e0c8be: moving from state init -> state preparing
>Thread-15914500::DEBUG::2013-11-20 03:33:00,950::clientIF::1219::vds::(fenceNode) rc 0 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = On
>Done
> err 
>Thread-15913695::DEBUG::2013-11-20 03:33:01,320::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8fdc5ce3-219b-4861-a6ca-2e1f0a1fea1c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914507::DEBUG::2013-11-20 03:33:01,385::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('163a3426-5ec5-46a5-b8f1-af06fc5d7e7d',) {}
>Thread-15913695::DEBUG::2013-11-20 03:33:01,391::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914507::DEBUG::2013-11-20 03:33:01,669::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::Disk stats not available
>Thread-15914507::DEBUG::2013-11-20 03:33:01,853::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::Disk latency not available
>Thread-15914508::DEBUG::2013-11-20 03:33:01,893::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Thread-15913695::DEBUG::2013-11-20 03:33:01,967::resourceManager::481::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' found in shared state and queue is empty, Joining current shared lock (2 active users)
>Thread-15914507::DEBUG::2013-11-20 03:33:02,003::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '5851', 'displayIp': '0', 'displayPort': u'5914', 'session': 'Unknown', 'displaySecurePort': u'5915', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet8': {'macAddr': u'00:1a:4a:a8:64:15', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet8'}}, 'vmId': '163a3426-5ec5-46a5-b8f1-af06fc5d7e7d', 'monitorResponse': '0', 'cpuUser': '365.29', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '53821308928', 'writeLatency': '0', 'imageID': '5f930ef3-763f-4294-a62a-6d52df220f8c', 'flushLatency': '0', 'truesize': '53821308928'}}, 'boot': 'c', 'statsAge': '1.92', 'cpuIdle': '0.00', 'elapsedTime': '63', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15914498::DEBUG::2013-11-20 03:33:02,226::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913695::DEBUG::2013-11-20 03:33:02,226::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8fdc5ce3-219b-4861-a6ca-2e1f0a1fea1c`::Granted request
>Thread-15912392::INFO::2013-11-20 03:33:02,227::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:33:02,418::task::495::TaskManager.Task::(_debug) Task 883052a6-83f9-483c-9906-61f82073285e: moving from state init -> state preparing
>Thread-15914498::DEBUG::2013-11-20 03:33:02,421::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --refresh 6b892067-ac7e-4039-a7be-5b7387525b16/8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f' (cwd None)
>Thread-15913695::DEBUG::2013-11-20 03:33:02,424::task::495::TaskManager.Task::(_debug) Task 8472d58f-9926-4cbd-8277-15dbf3e0c8be: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Dummy-15912651::DEBUG::2013-11-20 03:33:02,530::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 3.5718 s, 287 kB/s\n'; <rc> = 0
>Thread-15912392::DEBUG::2013-11-20 03:33:02,542::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7565d92b-481e-4a0b-b3ee-7d67d494d39e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:33:02,543::task::495::TaskManager.Task::(_debug) Task 8472d58f-9926-4cbd-8277-15dbf3e0c8be: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:33:02,544::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:33:02,738::task::495::TaskManager.Task::(_debug) Task 8472d58f-9926-4cbd-8277-15dbf3e0c8be: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:33:02,740::task::495::TaskManager.Task::(_debug) Task 8472d58f-9926-4cbd-8277-15dbf3e0c8be: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:33:02,741::resourceManager::481::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' found in shared state and queue is empty, Joining current shared lock (3 active users)
>Thread-15913695::DEBUG::2013-11-20 03:33:02,741::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:33:02,742::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7565d92b-481e-4a0b-b3ee-7d67d494d39e`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:33:02,743::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:33:02,745::task::495::TaskManager.Task::(_debug) Task 883052a6-83f9-483c-9906-61f82073285e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:33:02,745::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:33:02,747::task::495::TaskManager.Task::(_debug) Task 883052a6-83f9-483c-9906-61f82073285e: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:33:02,749::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (2 active users)
>Thread-15912392::DEBUG::2013-11-20 03:33:02,749::task::495::TaskManager.Task::(_debug) Task 883052a6-83f9-483c-9906-61f82073285e: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:33:02,752::task::495::TaskManager.Task::(_debug) Task 883052a6-83f9-483c-9906-61f82073285e: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:33:02,753::task::495::TaskManager.Task::(_debug) Task 8472d58f-9926-4cbd-8277-15dbf3e0c8be: ref 0 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:33:02,754::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::INFO::2013-11-20 03:33:02,755::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15912392::DEBUG::2013-11-20 03:33:02,755::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:33:02,756::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:33:02,757::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (1 active users)
>Thread-15912392::DEBUG::2013-11-20 03:33:02,758::task::495::TaskManager.Task::(_debug) Task 883052a6-83f9-483c-9906-61f82073285e: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:33:02,758::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914508::DEBUG::2013-11-20 03:33:03,004::clientIF::1219::vds::(fenceNode) rc 0 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = On
>Done
> err 
>Thread-15914498::DEBUG::2013-11-20 03:33:04,150::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15914498::DEBUG::2013-11-20 03:33:04,154::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15914498::DEBUG::2013-11-20 03:33:04,154::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15914498::INFO::2013-11-20 03:33:04,155::volume::517::Storage.Volume::(prepare) Volume: preparing volume 6b892067-ac7e-4039-a7be-5b7387525b16/8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f
>Thread-15914498::DEBUG::2013-11-20 03:33:04,156::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15914498::DEBUG::2013-11-20 03:33:04,156::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:33:04,740::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914498::DEBUG::2013-11-20 03:33:04,955::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15914509::DEBUG::2013-11-20 03:33:05,927::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914509::INFO::2013-11-20 03:33:06,221::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914509::DEBUG::2013-11-20 03:33:06,363::task::495::TaskManager.Task::(_debug) Task 77ffc288-d967-43c1-acdd-0cf34a563e78: moving from state init -> state preparing
>Thread-15914509::DEBUG::2013-11-20 03:33:06,441::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914498::DEBUG::2013-11-20 03:33:06,618::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15914509::DEBUG::2013-11-20 03:33:06,620::task::495::TaskManager.Task::(_debug) Task 77ffc288-d967-43c1-acdd-0cf34a563e78: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914498::DEBUG::2013-11-20 03:33:06,625::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=29 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15914509::DEBUG::2013-11-20 03:33:06,625::task::495::TaskManager.Task::(_debug) Task 77ffc288-d967-43c1-acdd-0cf34a563e78: moving from state preparing -> state finished
>Thread-15914509::DEBUG::2013-11-20 03:33:06,791::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914509::DEBUG::2013-11-20 03:33:06,793::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914509::DEBUG::2013-11-20 03:33:06,794::task::495::TaskManager.Task::(_debug) Task 77ffc288-d967-43c1-acdd-0cf34a563e78: ref 0 aborting False
>Thread-15914509::INFO::2013-11-20 03:33:06,794::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914511::DEBUG::2013-11-20 03:33:06,821::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914511::INFO::2013-11-20 03:33:06,822::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914511::DEBUG::2013-11-20 03:33:06,823::task::495::TaskManager.Task::(_debug) Task 76919e73-3eaa-4e27-b2a8-2abd639c0020: moving from state init -> state preparing
>Thread-15914511::DEBUG::2013-11-20 03:33:06,829::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2c514c08-b11f-4b22-b59a-cbbd1e16704d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914511::DEBUG::2013-11-20 03:33:06,830::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914511::DEBUG::2013-11-20 03:33:06,831::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914511::DEBUG::2013-11-20 03:33:06,834::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2c514c08-b11f-4b22-b59a-cbbd1e16704d`::Granted request
>Thread-15914511::DEBUG::2013-11-20 03:33:06,844::task::495::TaskManager.Task::(_debug) Task 76919e73-3eaa-4e27-b2a8-2abd639c0020: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914511::DEBUG::2013-11-20 03:33:06,845::task::495::TaskManager.Task::(_debug) Task 76919e73-3eaa-4e27-b2a8-2abd639c0020: ref 1 aborting False
>Thread-15914511::WARNING::2013-11-20 03:33:06,851::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914511::DEBUG::2013-11-20 03:33:06,872::task::495::TaskManager.Task::(_debug) Task 76919e73-3eaa-4e27-b2a8-2abd639c0020: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914511::DEBUG::2013-11-20 03:33:06,896::task::495::TaskManager.Task::(_debug) Task 76919e73-3eaa-4e27-b2a8-2abd639c0020: moving from state preparing -> state finished
>Thread-15914511::DEBUG::2013-11-20 03:33:06,898::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914511::DEBUG::2013-11-20 03:33:06,899::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914511::DEBUG::2013-11-20 03:33:06,900::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914511::DEBUG::2013-11-20 03:33:06,901::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914511::DEBUG::2013-11-20 03:33:06,902::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914511::DEBUG::2013-11-20 03:33:06,903::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914511::DEBUG::2013-11-20 03:33:06,904::task::495::TaskManager.Task::(_debug) Task 76919e73-3eaa-4e27-b2a8-2abd639c0020: ref 0 aborting False
>Thread-15914511::INFO::2013-11-20 03:33:06,905::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914498::DEBUG::2013-11-20 03:33:06,960::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000382175 s, 1.3 MB/s\n'; <rc> = 0
>Thread-15914498::DEBUG::2013-11-20 03:33:06,961::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000382175 s, 1.3 MB/s'], size: 512
>Thread-15914498::DEBUG::2013-11-20 03:33:06,962::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=29 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:33:07,228::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 2.05167 s, 499 kB/s\n'; <rc> = 0
>Thread-15914498::DEBUG::2013-11-20 03:33:07,286::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000135513 s, 3.8 MB/s\n'; <rc> = 0
>Thread-15914498::DEBUG::2013-11-20 03:33:07,287::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000135513 s, 3.8 MB/s'], size: 512
>Thread-15914498::DEBUG::2013-11-20 03:33:07,295::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f`ReqID=`85afb3e7-c824-4ede-832b-a43c601a1c1f`::Request was made in '/usr/share/vdsm/storage/volume.py' line '533' at 'prepare'
>Thread-15914498::DEBUG::2013-11-20 03:33:07,296::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f' for lock type 'exclusive'
>Thread-15914498::DEBUG::2013-11-20 03:33:07,297::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f' is free. Now locking as 'exclusive' (1 active user)
>Thread-15914498::DEBUG::2013-11-20 03:33:07,298::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f`ReqID=`85afb3e7-c824-4ede-832b-a43c601a1c1f`::Granted request
>Thread-15914498::DEBUG::2013-11-20 03:33:07,302::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=29 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15914498::DEBUG::2013-11-20 03:33:07,594::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000207349 s, 2.5 MB/s\n'; <rc> = 0
>Thread-15914498::DEBUG::2013-11-20 03:33:07,595::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000207349 s, 2.5 MB/s'], size: 512
>Thread-15914498::DEBUG::2013-11-20 03:33:07,596::task::495::TaskManager.Task::(_debug) Task f377e2e2-4ed8-4891-9481-7df918a98f8a: finished: None
>Thread-15914498::DEBUG::2013-11-20 03:33:07,596::task::495::TaskManager.Task::(_debug) Task f377e2e2-4ed8-4891-9481-7df918a98f8a: moving from state preparing -> state finished
>Thread-15914498::DEBUG::2013-11-20 03:33:07,597::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914498::DEBUG::2013-11-20 03:33:07,597::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914498::DEBUG::2013-11-20 03:33:07,598::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914498::DEBUG::2013-11-20 03:33:07,598::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914498::DEBUG::2013-11-20 03:33:07,598::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914498::DEBUG::2013-11-20 03:33:07,599::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914498::DEBUG::2013-11-20 03:33:07,599::task::495::TaskManager.Task::(_debug) Task f377e2e2-4ed8-4891-9481-7df918a98f8a: ref 0 aborting False
>Thread-15914498::INFO::2013-11-20 03:33:07,600::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, Return response: {'status': {'message': 'OK', 'code': 0}}
>Thread-15914498::INFO::2013-11-20 03:33:07,600::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914498::DEBUG::2013-11-20 03:33:07,601::task::495::TaskManager.Task::(_debug) Task 9ce081f2-2124-4594-bceb-86efbe558583: moving from state init -> state preparing
>Thread-15914498::DEBUG::2013-11-20 03:33:07,602::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1fd6b1cb-d338-47e5-9d21-39f746de90bc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1666' at 'public_getVolumePath'
>Thread-15914498::DEBUG::2013-11-20 03:33:07,602::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914498::DEBUG::2013-11-20 03:33:07,602::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914498::DEBUG::2013-11-20 03:33:07,603::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1fd6b1cb-d338-47e5-9d21-39f746de90bc`::Granted request
>Thread-15914498::DEBUG::2013-11-20 03:33:07,603::task::495::TaskManager.Task::(_debug) Task 9ce081f2-2124-4594-bceb-86efbe558583: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914498::DEBUG::2013-11-20 03:33:07,604::task::495::TaskManager.Task::(_debug) Task 9ce081f2-2124-4594-bceb-86efbe558583: ref 1 aborting False
>Thread-15914498::DEBUG::2013-11-20 03:33:07,604::task::495::TaskManager.Task::(_debug) Task 9ce081f2-2124-4594-bceb-86efbe558583: finished: {'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/5af747d8-95b1-4b7e-af53-cb77e4fcc896/8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f'}
>Thread-15914498::DEBUG::2013-11-20 03:33:07,605::task::495::TaskManager.Task::(_debug) Task 9ce081f2-2124-4594-bceb-86efbe558583: moving from state preparing -> state finished
>Thread-15914498::DEBUG::2013-11-20 03:33:07,605::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914498::DEBUG::2013-11-20 03:33:07,605::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914498::DEBUG::2013-11-20 03:33:07,606::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914498::DEBUG::2013-11-20 03:33:07,606::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914498::DEBUG::2013-11-20 03:33:07,606::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914498::DEBUG::2013-11-20 03:33:07,607::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914498::DEBUG::2013-11-20 03:33:07,607::task::495::TaskManager.Task::(_debug) Task 9ce081f2-2124-4594-bceb-86efbe558583: ref 0 aborting False
>Thread-15914498::INFO::2013-11-20 03:33:07,607::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, Return response: {'status': {'message': 'OK', 'code': 0}, 'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/5af747d8-95b1-4b7e-af53-cb77e4fcc896/8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f'}
>Thread-15914498::INFO::2013-11-20 03:33:07,609::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914498::DEBUG::2013-11-20 03:33:07,609::task::495::TaskManager.Task::(_debug) Task 42a2c1f4-3574-427c-a622-6b02d6d8ea08: moving from state init -> state preparing
>Thread-15914498::DEBUG::2013-11-20 03:33:07,612::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`98a550c7-77b4-4fd2-9f7f-a0290471b1f8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914498::DEBUG::2013-11-20 03:33:07,612::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914498::DEBUG::2013-11-20 03:33:07,613::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914498::DEBUG::2013-11-20 03:33:07,613::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`98a550c7-77b4-4fd2-9f7f-a0290471b1f8`::Granted request
>Thread-15914498::DEBUG::2013-11-20 03:33:07,614::task::495::TaskManager.Task::(_debug) Task 42a2c1f4-3574-427c-a622-6b02d6d8ea08: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914498::DEBUG::2013-11-20 03:33:07,614::task::495::TaskManager.Task::(_debug) Task 42a2c1f4-3574-427c-a622-6b02d6d8ea08: ref 1 aborting False
>Thread-15914498::DEBUG::2013-11-20 03:33:07,614::task::495::TaskManager.Task::(_debug) Task 42a2c1f4-3574-427c-a622-6b02d6d8ea08: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914498::DEBUG::2013-11-20 03:33:07,615::task::495::TaskManager.Task::(_debug) Task 42a2c1f4-3574-427c-a622-6b02d6d8ea08: moving from state preparing -> state finished
>Thread-15914498::DEBUG::2013-11-20 03:33:07,615::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914498::DEBUG::2013-11-20 03:33:07,615::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914498::DEBUG::2013-11-20 03:33:07,618::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914498::DEBUG::2013-11-20 03:33:07,618::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914498::DEBUG::2013-11-20 03:33:07,619::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914498::DEBUG::2013-11-20 03:33:07,619::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914498::DEBUG::2013-11-20 03:33:07,619::task::495::TaskManager.Task::(_debug) Task 42a2c1f4-3574-427c-a622-6b02d6d8ea08: ref 0 aborting False
>Thread-15914498::INFO::2013-11-20 03:33:07,620::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914498::INFO::2013-11-20 03:33:07,620::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16)
>Thread-15914498::DEBUG::2013-11-20 03:33:07,620::task::495::TaskManager.Task::(_debug) Task 478f9436-6265-4f00-97b7-17e79ec7de22: moving from state init -> state preparing
>Thread-15914498::INFO::2013-11-20 03:33:07,621::blockSD::674::Storage.StorageDomain::(validate) sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16
>Thread-15914498::DEBUG::2013-11-20 03:33:07,621::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15914498::DEBUG::2013-11-20 03:33:07,622::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15914498::DEBUG::2013-11-20 03:33:07,622::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15914498::DEBUG::2013-11-20 03:33:07,623::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15914498::DEBUG::2013-11-20 03:33:07,623::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-15914498::DEBUG::2013-11-20 03:33:07,628::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15914498::DEBUG::2013-11-20 03:33:08,173::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15914498::DEBUG::2013-11-20 03:33:08,193::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-15914498::DEBUG::2013-11-20 03:33:08,194::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'_SHA_CKSUM=716ea7492d9134a4b13de27ac61a336ad474727c', u'POOL_SPM_ID=2', u'POOL_SPM_LVER=287']
>Thread-15914498::DEBUG::2013-11-20 03:33:08,203::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`33928e4c-e86a-418e-a4e6-866caa71f305`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1297' at 'public_getStorageDomainInfo'
>Thread-15914498::DEBUG::2013-11-20 03:33:08,204::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914498::DEBUG::2013-11-20 03:33:08,205::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914498::DEBUG::2013-11-20 03:33:08,205::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`33928e4c-e86a-418e-a4e6-866caa71f305`::Granted request
>Thread-15914498::DEBUG::2013-11-20 03:33:08,206::task::495::TaskManager.Task::(_debug) Task 478f9436-6265-4f00-97b7-17e79ec7de22: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914498::DEBUG::2013-11-20 03:33:08,207::task::495::TaskManager.Task::(_debug) Task 478f9436-6265-4f00-97b7-17e79ec7de22: ref 1 aborting False
>Thread-15914498::WARNING::2013-11-20 03:33:08,209::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914498::DEBUG::2013-11-20 03:33:08,209::task::495::TaskManager.Task::(_debug) Task 478f9436-6265-4f00-97b7-17e79ec7de22: finished: {'info': {'uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'vguuid': 'aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', 'lver': 287, 'state': 'OK', 'version': '2', 'role': 'Master', 'master_ver': 1, 'spm_id': 2, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'OS'}}
>Thread-15914498::DEBUG::2013-11-20 03:33:08,210::task::495::TaskManager.Task::(_debug) Task 478f9436-6265-4f00-97b7-17e79ec7de22: moving from state preparing -> state finished
>Thread-15914498::DEBUG::2013-11-20 03:33:08,210::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914498::DEBUG::2013-11-20 03:33:08,211::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914498::DEBUG::2013-11-20 03:33:08,211::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914498::DEBUG::2013-11-20 03:33:08,212::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914498::DEBUG::2013-11-20 03:33:08,212::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914498::DEBUG::2013-11-20 03:33:08,213::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914498::DEBUG::2013-11-20 03:33:08,213::task::495::TaskManager.Task::(_debug) Task 478f9436-6265-4f00-97b7-17e79ec7de22: ref 0 aborting False
>Thread-15914498::INFO::2013-11-20 03:33:08,213::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'vguuid': 'aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', 'lver': 287, 'state': 'OK', 'version': '2', 'role': 'Master', 'master_ver': 1, 'spm_id': 2, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'OS'}}
>Thread-15914498::DEBUG::2013-11-20 03:33:08,230::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/libexec/vdsm/hooks/before_vm_start/50_vhostmd' (cwd None)
>Thread-15914512::INFO::2013-11-20 03:33:08,949::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914512::DEBUG::2013-11-20 03:33:08,951::task::495::TaskManager.Task::(_debug) Task 446165ea-8faa-4592-974d-e9772d2ee250: moving from state init -> state preparing
>Thread-15914512::DEBUG::2013-11-20 03:33:08,952::task::495::TaskManager.Task::(_debug) Task 446165ea-8faa-4592-974d-e9772d2ee250: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00112295150757', 'lastCheck': 1384918386.62466, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0161530971527', 'lastCheck': 1384918382.4393289, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0522398948669', 'lastCheck': 1384918385.782506, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.209988832474', 'lastCheck': 1384918381.320569, 'valid': True, 'code': 0}}
>Thread-15914512::DEBUG::2013-11-20 03:33:08,953::task::495::TaskManager.Task::(_debug) Task 446165ea-8faa-4592-974d-e9772d2ee250: moving from state preparing -> state finished
>Thread-15914512::DEBUG::2013-11-20 03:33:08,953::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914512::DEBUG::2013-11-20 03:33:08,954::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914512::DEBUG::2013-11-20 03:33:08,954::task::495::TaskManager.Task::(_debug) Task 446165ea-8faa-4592-974d-e9772d2ee250: ref 0 aborting False
>Thread-15914512::INFO::2013-11-20 03:33:08,957::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00112295150757', 'lastCheck': 1384918386.62466, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0161530971527', 'lastCheck': 1384918382.4393289, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0522398948669', 'lastCheck': 1384918385.782506, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.209988832474', 'lastCheck': 1384918381.320569, 'valid': True, 'code': 0}}
>Thread-15914513::DEBUG::2013-11-20 03:33:09,150::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::Disk stats not available
>Thread-15914513::DEBUG::2013-11-20 03:33:09,151::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::Disk latency not available
>Thread-15914513::DEBUG::2013-11-20 03:33:09,156::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Disk stats not available
>Thread-15914513::DEBUG::2013-11-20 03:33:09,156::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::Disk latency not available
>Thread-15914498::DEBUG::2013-11-20 03:33:09,180::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15914498::INFO::2013-11-20 03:33:09,181::hooks::51::root::(_runHooksDir) 
>Thread-15914498::DEBUG::2013-11-20 03:33:09,181::libvirtvm::1077::vm.Vm::(_run) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::<?xml version="1.0" encoding="utf-8"?>
><domain type="kvm">
>	<name>sukhumi</name>
>	<uuid>e97d1c52-b688-4192-bdeb-2dfadcc79786</uuid>
>	<memory>4608000</memory>
>	<currentMemory>4608000</currentMemory>
>	<vcpu>4</vcpu>
>	<devices>
>		<disk device="disk" type="block">
>			<source dev="/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/5af747d8-95b1-4b7e-af53-cb77e4fcc896/8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f"/>
>			<target bus="ide" dev="hda"/>
>			<serial>7e-af53-cb77e4fcc896</serial>
>			<driver cache="none" error_policy="stop" io="native" name="qemu" type="raw"/>
>		</disk>
>		<disk device="cdrom" type="file">
>			<source file="/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/RHEV-toolsSetup_3.0_39.iso"/>
>			<target bus="ide" dev="hdc"/>
>		</disk>
>		<controller index="0" ports="16" type="virtio-serial"/>
>		<channel type="unix">
>			<target name="com.redhat.rhevm.vdsm" type="virtio"/>
>			<source mode="bind" path="/var/lib/libvirt/qemu/channels/sukhumi.com.redhat.rhevm.vdsm"/>
>		</channel>
>		<memballoon model="none"/>
>		<interface type="bridge">
>			<mac address="00:1a:4a:a8:64:13"/>
>			<model type="virtio"/>
>			<source bridge="rhevm"/>
>		</interface>
>		<input bus="ps2" type="mouse"/>
>		<video>
>			<model heads="1" type="qxl" vram="65536"/>
>		</video>
>		<channel type="spicevmc">
>			<target name="com.redhat.spice.0" type="virtio"/>
>		</channel>
>		<graphics autoport="yes" keymap="en-us" listen="0" passwd="*****" passwdValidTo="1970-01-01T00:00:01" port="-1" tlsPort="-1" type="spice">
>			<channel mode="secure" name="main"/>
>			<channel mode="secure" name="inputs"/>
>		</graphics>
>	</devices>
>	<os>
>		<type arch="x86_64" machine="rhel6.2.0">hvm</type>
>		<boot dev="hd"/>
>		<smbios mode="sysinfo"/>
>	</os>
>	<sysinfo type="smbios">
>		<system>
>			<entry name="manufacturer">Red Hat</entry>
>			<entry name="product">RHEV Hypervisor</entry>
>			<entry name="version">6.3-20121012.0.el6_3</entry>
>			<entry name="serial">30343536-3138-5A43-4A32-333630373442_AC:16:2D:76:EB:F0</entry>
>			<entry name="uuid">e97d1c52-b688-4192-bdeb-2dfadcc79786</entry>
>		</system>
>	</sysinfo>
>	<clock adjustment="0" offset="variable">
>		<timer name="rtc" tickpolicy="catchup"/>
>	</clock>
>	<features>
>		<acpi/>
>	</features>
>	<cpu match="exact">
>		<model>Westmere</model>
>		<topology cores="1" sockets="4" threads="1"/>
>	</cpu>
></domain>
>
>Dummy-15912651::DEBUG::2013-11-20 03:33:09,244::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:33:09,628::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0479395 s, 21.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:33:11,638::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:33:12,034::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0521188 s, 19.6 MB/s\n'; <rc> = 0
>Thread-15914435::INFO::2013-11-20 03:33:13,941::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 03:33:13,942::task::495::TaskManager.Task::(_debug) Task 8acc6d76-103e-4825-8877-5714bdbf16b1: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 03:33:13,944::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d26f61de-a3e7-4ae9-aeca-9e1bbf947e6e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 03:33:13,944::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 03:33:13,945::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 03:33:13,945::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d26f61de-a3e7-4ae9-aeca-9e1bbf947e6e`::Granted request
>Thread-15914435::DEBUG::2013-11-20 03:33:13,946::task::495::TaskManager.Task::(_debug) Task 8acc6d76-103e-4825-8877-5714bdbf16b1: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 03:33:13,946::task::495::TaskManager.Task::(_debug) Task 8acc6d76-103e-4825-8877-5714bdbf16b1: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 03:33:13,947::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15914435::DEBUG::2013-11-20 03:33:13,948::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:33:14,116::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:33:14,670::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0473682 s, 21.6 MB/s\n'; <rc> = 0
>Thread-15914435::DEBUG::2013-11-20 03:33:14,817::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15914435::DEBUG::2013-11-20 03:33:16,049::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15914435::DEBUG::2013-11-20 03:33:16,052::task::495::TaskManager.Task::(_debug) Task 8acc6d76-103e-4825-8877-5714bdbf16b1: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 03:33:16,059::task::495::TaskManager.Task::(_debug) Task 8acc6d76-103e-4825-8877-5714bdbf16b1: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 03:33:16,060::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 03:33:16,060::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 03:33:16,061::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 03:33:16,062::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 03:33:16,063::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 03:33:16,063::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 03:33:16,064::task::495::TaskManager.Task::(_debug) Task 8acc6d76-103e-4825-8877-5714bdbf16b1: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 03:33:16,065::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 03:33:16,679::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:33:17,071::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0511597 s, 20.0 MB/s\n'; <rc> = 0
>Thread-15914516::DEBUG::2013-11-20 03:33:17,357::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914516::INFO::2013-11-20 03:33:17,362::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914516::DEBUG::2013-11-20 03:33:17,363::task::495::TaskManager.Task::(_debug) Task bf0cb843-faa2-4c8b-af48-30dff3352034: moving from state init -> state preparing
>Thread-15914516::DEBUG::2013-11-20 03:33:17,364::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914516::DEBUG::2013-11-20 03:33:17,364::task::495::TaskManager.Task::(_debug) Task bf0cb843-faa2-4c8b-af48-30dff3352034: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914516::DEBUG::2013-11-20 03:33:17,367::task::495::TaskManager.Task::(_debug) Task bf0cb843-faa2-4c8b-af48-30dff3352034: moving from state preparing -> state finished
>Thread-15914516::DEBUG::2013-11-20 03:33:17,367::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914516::DEBUG::2013-11-20 03:33:17,375::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914516::DEBUG::2013-11-20 03:33:17,375::task::495::TaskManager.Task::(_debug) Task bf0cb843-faa2-4c8b-af48-30dff3352034: ref 0 aborting False
>Thread-15914516::INFO::2013-11-20 03:33:17,376::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914517::DEBUG::2013-11-20 03:33:17,384::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914517::INFO::2013-11-20 03:33:17,385::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914517::DEBUG::2013-11-20 03:33:17,385::task::495::TaskManager.Task::(_debug) Task c0549829-95f5-4965-8127-ac7b714316dd: moving from state init -> state preparing
>Thread-15914517::DEBUG::2013-11-20 03:33:17,389::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6f038319-6244-42d0-83d4-6f26ef110261`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914517::DEBUG::2013-11-20 03:33:17,389::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914517::DEBUG::2013-11-20 03:33:17,395::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914517::DEBUG::2013-11-20 03:33:17,397::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6f038319-6244-42d0-83d4-6f26ef110261`::Granted request
>Thread-15914517::DEBUG::2013-11-20 03:33:17,401::task::495::TaskManager.Task::(_debug) Task c0549829-95f5-4965-8127-ac7b714316dd: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914517::DEBUG::2013-11-20 03:33:17,404::task::495::TaskManager.Task::(_debug) Task c0549829-95f5-4965-8127-ac7b714316dd: ref 1 aborting False
>Thread-15914517::WARNING::2013-11-20 03:33:17,410::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914517::DEBUG::2013-11-20 03:33:17,421::task::495::TaskManager.Task::(_debug) Task c0549829-95f5-4965-8127-ac7b714316dd: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914517::DEBUG::2013-11-20 03:33:17,440::task::495::TaskManager.Task::(_debug) Task c0549829-95f5-4965-8127-ac7b714316dd: moving from state preparing -> state finished
>Thread-15914517::DEBUG::2013-11-20 03:33:17,440::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914517::DEBUG::2013-11-20 03:33:17,446::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914517::DEBUG::2013-11-20 03:33:17,448::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914517::DEBUG::2013-11-20 03:33:17,448::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914517::DEBUG::2013-11-20 03:33:17,449::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914517::DEBUG::2013-11-20 03:33:17,449::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914517::DEBUG::2013-11-20 03:33:17,451::task::495::TaskManager.Task::(_debug) Task c0549829-95f5-4965-8127-ac7b714316dd: ref 0 aborting False
>Thread-15914517::INFO::2013-11-20 03:33:17,453::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914453::INFO::2013-11-20 03:33:19,066::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 03:33:19,066::task::495::TaskManager.Task::(_debug) Task 29d4952f-c219-4890-8400-58e2ba3344f2: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 03:33:19,067::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`23290589-5826-49b4-8095-82a7aaeb6315`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 03:33:19,068::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 03:33:19,068::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 03:33:19,069::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`23290589-5826-49b4-8095-82a7aaeb6315`::Granted request
>Thread-15914453::DEBUG::2013-11-20 03:33:19,069::task::495::TaskManager.Task::(_debug) Task 29d4952f-c219-4890-8400-58e2ba3344f2: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 03:33:19,070::task::495::TaskManager.Task::(_debug) Task 29d4952f-c219-4890-8400-58e2ba3344f2: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 03:33:19,070::task::495::TaskManager.Task::(_debug) Task 29d4952f-c219-4890-8400-58e2ba3344f2: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 03:33:19,070::task::495::TaskManager.Task::(_debug) Task 29d4952f-c219-4890-8400-58e2ba3344f2: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 03:33:19,071::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 03:33:19,071::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 03:33:19,072::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 03:33:19,072::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 03:33:19,072::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 03:33:19,073::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 03:33:19,073::task::495::TaskManager.Task::(_debug) Task 29d4952f-c219-4890-8400-58e2ba3344f2: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 03:33:19,073::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 03:33:19,080::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>libvirtEventLoop::DEBUG::2013-11-20 03:33:19,489::libvirtvm::1401::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::event Started detail 0 opaque None
>Thread-15914498::DEBUG::2013-11-20 03:33:19,573::utils::307::vm.Vm::(start) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::Start statistics collection
>Thread-15914519::DEBUG::2013-11-20 03:33:19,575::utils::336::vm.Vm::(run) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::Stats thread started
>Thread-15914520::DEBUG::2013-11-20 03:33:19,586::guestIF::47::root::(_connect) Attempting connection to /var/lib/libvirt/qemu/channels/sukhumi.com.redhat.rhevm.vdsm
>Thread-15914519::INFO::2013-11-20 03:33:19,596::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914520::DEBUG::2013-11-20 03:33:19,598::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /usr/share/vdsm/prepare-vmchannel /var/lib/libvirt/qemu/channels/sukhumi.com.redhat.rhevm.vdsm' (cwd None)
>Thread-15914498::WARNING::2013-11-20 03:33:19,599::libvirtvm::1124::vm.Vm::(_readPauseCode) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::_readPauseCode unsupported by libvirt vm
>Thread-15914519::DEBUG::2013-11-20 03:33:19,600::task::495::TaskManager.Task::(_debug) Task a6782986-0ea5-4d5a-ba00-eaa306248363: moving from state init -> state preparing
>Thread-15914498::DEBUG::2013-11-20 03:33:19,755::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /usr/share/vdsm/get-vm-pid sukhumi' (cwd None)
>Thread-15914519::DEBUG::2013-11-20 03:33:19,770::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`13bcf4d6-b98d-4f35-a044-b6ccbbfa6537`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 03:33:19,772::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914520::DEBUG::2013-11-20 03:33:20,081::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15914519::DEBUG::2013-11-20 03:33:20,082::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 03:33:20,088::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`13bcf4d6-b98d-4f35-a044-b6ccbbfa6537`::Granted request
>Thread-15914520::DEBUG::2013-11-20 03:33:20,090::guestIF::53::root::(_connect) connected to /var/lib/libvirt/qemu/channels/sukhumi.com.redhat.rhevm.vdsm
>Thread-15914519::DEBUG::2013-11-20 03:33:20,101::task::495::TaskManager.Task::(_debug) Task a6782986-0ea5-4d5a-ba00-eaa306248363: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 03:33:20,104::task::495::TaskManager.Task::(_debug) Task a6782986-0ea5-4d5a-ba00-eaa306248363: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 03:33:20,108::task::495::TaskManager.Task::(_debug) Task a6782986-0ea5-4d5a-ba00-eaa306248363: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 03:33:20,114::task::495::TaskManager.Task::(_debug) Task a6782986-0ea5-4d5a-ba00-eaa306248363: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 03:33:20,116::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 03:33:20,129::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 03:33:20,137::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 03:33:20,141::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 03:33:20,142::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 03:33:20,148::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 03:33:20,151::task::495::TaskManager.Task::(_debug) Task a6782986-0ea5-4d5a-ba00-eaa306248363: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 03:33:20,152::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 03:33:20,168::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0991058 s, 10.3 MB/s\n'; <rc> = 0
>Thread-15914498::DEBUG::2013-11-20 03:33:21,506::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15914498::DEBUG::2013-11-20 03:33:21,558::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /sbin/service ksmtuned retune' (cwd None)
>Thread-15914498::DEBUG::2013-11-20 03:33:21,976::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15914498::DEBUG::2013-11-20 03:33:21,980::vm::358::vm.Vm::(_startUnderlyingVm) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::_ongoingCreations released
>Thread-15914522::DEBUG::2013-11-20 03:33:22,124::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('e97d1c52-b688-4192-bdeb-2dfadcc79786',) {}
>Thread-15914522::DEBUG::2013-11-20 03:33:22,125::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::CPU stats not available
>Thread-15914522::DEBUG::2013-11-20 03:33:22,125::libvirtvm::202::vm.Vm::(_getNetworkStats) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::Network stats not available
>Thread-15914522::DEBUG::2013-11-20 03:33:22,126::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::Disk stats not available
>Thread-15914522::DEBUG::2013-11-20 03:33:22,127::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::Disk latency not available
>Thread-15914522::DEBUG::2013-11-20 03:33:22,127::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '7039', 'displayIp': '0', 'displayPort': u'5916', 'session': 'Unknown', 'displaySecurePort': u'5917', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/RHEV-toolsSetup_3.0_39.iso', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet9': {'state': 'unknown', 'macAddr': u'00:1a:4a:a8:64:13', 'speed': '1000', 'name': u'vnet9'}}, 'vmId': 'e97d1c52-b688-4192-bdeb-2dfadcc79786', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '86033563648', 'writeLatency': '0', 'imageID': '5af747d8-95b1-4b7e-af53-cb77e4fcc896', 'flushLatency': '0', 'truesize': '86033563648'}}, 'boot': 'c', 'statsAge': '2.54', 'cpuIdle': '100.00', 'elapsedTime': '39', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '0', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:33:22,179::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:33:22,818::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.083188 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:33:24,834::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:33:25,207::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0468094 s, 21.9 MB/s\n'; <rc> = 0
>Thread-15914523::INFO::2013-11-20 03:33:25,472::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914523::DEBUG::2013-11-20 03:33:25,474::task::495::TaskManager.Task::(_debug) Task 8d6f5d3b-76fa-4ca0-a7e4-3cc7f9842ef3: moving from state init -> state preparing
>Thread-15914523::DEBUG::2013-11-20 03:33:25,477::task::495::TaskManager.Task::(_debug) Task 8d6f5d3b-76fa-4ca0-a7e4-3cc7f9842ef3: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000340938568115', 'lastCheck': 1384918396.843962, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0111448764801', 'lastCheck': 1384918403.0861681, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.013267993927', 'lastCheck': 1384918396.3462951, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00055193901062', 'lastCheck': 1384918401.493926, 'valid': True, 'code': 0}}
>Thread-15914523::DEBUG::2013-11-20 03:33:25,478::task::495::TaskManager.Task::(_debug) Task 8d6f5d3b-76fa-4ca0-a7e4-3cc7f9842ef3: moving from state preparing -> state finished
>Thread-15914523::DEBUG::2013-11-20 03:33:25,478::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914523::DEBUG::2013-11-20 03:33:25,479::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914523::DEBUG::2013-11-20 03:33:25,480::task::495::TaskManager.Task::(_debug) Task 8d6f5d3b-76fa-4ca0-a7e4-3cc7f9842ef3: ref 0 aborting False
>Thread-15914523::INFO::2013-11-20 03:33:25,481::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000340938568115', 'lastCheck': 1384918396.843962, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0111448764801', 'lastCheck': 1384918403.0861681, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.013267993927', 'lastCheck': 1384918396.3462951, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00055193901062', 'lastCheck': 1384918401.493926, 'valid': True, 'code': 0}}
>Thread-15914524::DEBUG::2013-11-20 03:33:25,782::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::CPU stats not available
>Thread-15914524::DEBUG::2013-11-20 03:33:25,783::libvirtvm::202::vm.Vm::(_getNetworkStats) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::Network stats not available
>Thread-15914524::DEBUG::2013-11-20 03:33:25,783::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::Disk stats not available
>Thread-15914524::DEBUG::2013-11-20 03:33:25,784::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::Disk latency not available
>Thread-15913518::INFO::2013-11-20 03:33:26,158::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:33:26,159::task::495::TaskManager.Task::(_debug) Task fb6082c5-f2b2-4629-9f02-6fa256ff4c73: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:33:26,160::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e94b13be-956a-4fbb-b8da-4385608a2d56`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:33:26,160::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:33:26,161::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:33:26,161::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e94b13be-956a-4fbb-b8da-4385608a2d56`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:33:26,162::task::495::TaskManager.Task::(_debug) Task fb6082c5-f2b2-4629-9f02-6fa256ff4c73: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:33:26,163::task::495::TaskManager.Task::(_debug) Task fb6082c5-f2b2-4629-9f02-6fa256ff4c73: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:33:26,163::task::495::TaskManager.Task::(_debug) Task fb6082c5-f2b2-4629-9f02-6fa256ff4c73: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:33:26,164::task::495::TaskManager.Task::(_debug) Task fb6082c5-f2b2-4629-9f02-6fa256ff4c73: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:33:26,164::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:33:26,164::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:33:26,165::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:33:26,166::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:33:26,166::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:33:26,167::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:33:26,167::task::495::TaskManager.Task::(_debug) Task fb6082c5-f2b2-4629-9f02-6fa256ff4c73: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:33:26,168::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:33:26,168::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:33:26,168::task::495::TaskManager.Task::(_debug) Task 1a41cbfd-56ed-4e82-adfe-39796d768e24: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:33:26,169::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4fae2134-1311-4c5c-9aaf-994a9c8c1055`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:33:26,170::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:33:26,170::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:33:26,170::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4fae2134-1311-4c5c-9aaf-994a9c8c1055`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:33:26,171::task::495::TaskManager.Task::(_debug) Task 1a41cbfd-56ed-4e82-adfe-39796d768e24: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:33:26,171::task::495::TaskManager.Task::(_debug) Task 1a41cbfd-56ed-4e82-adfe-39796d768e24: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:33:26,172::task::495::TaskManager.Task::(_debug) Task 1a41cbfd-56ed-4e82-adfe-39796d768e24: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:33:26,172::task::495::TaskManager.Task::(_debug) Task 1a41cbfd-56ed-4e82-adfe-39796d768e24: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:33:26,172::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:33:26,173::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:33:26,173::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:33:26,174::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:33:26,174::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:33:26,174::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:33:26,175::task::495::TaskManager.Task::(_debug) Task 1a41cbfd-56ed-4e82-adfe-39796d768e24: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:33:26,175::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:33:27,218::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:33:27,717::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0486837 s, 21.0 MB/s\n'; <rc> = 0
>Thread-15914525::DEBUG::2013-11-20 03:33:27,982::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914525::INFO::2013-11-20 03:33:27,983::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914525::DEBUG::2013-11-20 03:33:27,984::task::495::TaskManager.Task::(_debug) Task 1953f0e4-f417-45e8-8291-8280f8bdb6c3: moving from state init -> state preparing
>Thread-15914525::DEBUG::2013-11-20 03:33:27,985::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914525::DEBUG::2013-11-20 03:33:27,985::task::495::TaskManager.Task::(_debug) Task 1953f0e4-f417-45e8-8291-8280f8bdb6c3: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914525::DEBUG::2013-11-20 03:33:27,985::task::495::TaskManager.Task::(_debug) Task 1953f0e4-f417-45e8-8291-8280f8bdb6c3: moving from state preparing -> state finished
>Thread-15914525::DEBUG::2013-11-20 03:33:27,986::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914525::DEBUG::2013-11-20 03:33:27,986::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914525::DEBUG::2013-11-20 03:33:27,986::task::495::TaskManager.Task::(_debug) Task 1953f0e4-f417-45e8-8291-8280f8bdb6c3: ref 0 aborting False
>Thread-15914525::INFO::2013-11-20 03:33:27,987::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914526::DEBUG::2013-11-20 03:33:28,003::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914526::INFO::2013-11-20 03:33:28,003::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914526::DEBUG::2013-11-20 03:33:28,004::task::495::TaskManager.Task::(_debug) Task 6b20f676-b817-4f4d-8b08-37ce164fb093: moving from state init -> state preparing
>Thread-15914526::DEBUG::2013-11-20 03:33:28,006::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`09b35266-5eeb-4ac8-b012-6811b8a3b13a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914526::DEBUG::2013-11-20 03:33:28,008::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914526::DEBUG::2013-11-20 03:33:28,009::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914526::DEBUG::2013-11-20 03:33:28,009::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`09b35266-5eeb-4ac8-b012-6811b8a3b13a`::Granted request
>Thread-15914526::DEBUG::2013-11-20 03:33:28,130::task::495::TaskManager.Task::(_debug) Task 6b20f676-b817-4f4d-8b08-37ce164fb093: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914526::DEBUG::2013-11-20 03:33:28,130::task::495::TaskManager.Task::(_debug) Task 6b20f676-b817-4f4d-8b08-37ce164fb093: ref 1 aborting False
>Thread-15914526::WARNING::2013-11-20 03:33:28,132::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914526::DEBUG::2013-11-20 03:33:28,132::task::495::TaskManager.Task::(_debug) Task 6b20f676-b817-4f4d-8b08-37ce164fb093: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914526::DEBUG::2013-11-20 03:33:28,133::task::495::TaskManager.Task::(_debug) Task 6b20f676-b817-4f4d-8b08-37ce164fb093: moving from state preparing -> state finished
>Thread-15914526::DEBUG::2013-11-20 03:33:28,133::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914526::DEBUG::2013-11-20 03:33:28,133::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914526::DEBUG::2013-11-20 03:33:28,134::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914526::DEBUG::2013-11-20 03:33:28,134::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914526::DEBUG::2013-11-20 03:33:28,135::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914526::DEBUG::2013-11-20 03:33:28,135::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914526::DEBUG::2013-11-20 03:33:28,136::task::495::TaskManager.Task::(_debug) Task 6b20f676-b817-4f4d-8b08-37ce164fb093: ref 0 aborting False
>Thread-15914526::INFO::2013-11-20 03:33:28,136::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914528::DEBUG::2013-11-20 03:33:29,256::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('e97d1c52-b688-4192-bdeb-2dfadcc79786',) {}
>Thread-15914528::DEBUG::2013-11-20 03:33:29,257::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::CPU stats not available
>Thread-15914528::DEBUG::2013-11-20 03:33:29,257::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::Disk stats not available
>Thread-15914528::DEBUG::2013-11-20 03:33:29,257::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::Disk latency not available
>Thread-15914528::DEBUG::2013-11-20 03:33:29,258::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '7039', 'displayIp': '0', 'displayPort': u'5916', 'session': 'Unknown', 'displaySecurePort': u'5917', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/RHEV-toolsSetup_3.0_39.iso', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet9': {'macAddr': u'00:1a:4a:a8:64:13', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet9'}}, 'vmId': 'e97d1c52-b688-4192-bdeb-2dfadcc79786', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '86033563648', 'writeLatency': '0', 'imageID': '5af747d8-95b1-4b7e-af53-cb77e4fcc896', 'flushLatency': '0', 'truesize': '86033563648'}}, 'boot': 'c', 'statsAge': '0.84', 'cpuIdle': '100.00', 'elapsedTime': '46', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '0', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15913546::INFO::2013-11-20 03:33:29,485::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:33:29,486::task::495::TaskManager.Task::(_debug) Task 7cc01f7e-3d82-4af4-b1ea-0c0b6f70062b: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:33:29,487::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`cac5c5ab-71c0-485c-81d5-13f1920dc3f3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:33:29,488::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:33:29,488::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:33:29,489::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`cac5c5ab-71c0-485c-81d5-13f1920dc3f3`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:33:29,489::task::495::TaskManager.Task::(_debug) Task 7cc01f7e-3d82-4af4-b1ea-0c0b6f70062b: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:33:29,490::task::495::TaskManager.Task::(_debug) Task 7cc01f7e-3d82-4af4-b1ea-0c0b6f70062b: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:33:29,491::task::495::TaskManager.Task::(_debug) Task 7cc01f7e-3d82-4af4-b1ea-0c0b6f70062b: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:33:29,491::task::495::TaskManager.Task::(_debug) Task 7cc01f7e-3d82-4af4-b1ea-0c0b6f70062b: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:33:29,492::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:33:29,492::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:33:29,493::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:33:29,493::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:33:29,493::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:33:29,494::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:33:29,494::task::495::TaskManager.Task::(_debug) Task 7cc01f7e-3d82-4af4-b1ea-0c0b6f70062b: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:33:29,495::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:33:29,495::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:33:29,496::task::495::TaskManager.Task::(_debug) Task 7716432c-19ca-4116-990d-1c8bc53a3f06: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:33:29,496::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f8d3505d-a3dd-4a27-9a9f-6bc342537783`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:33:29,497::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:33:29,497::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:33:29,498::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f8d3505d-a3dd-4a27-9a9f-6bc342537783`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:33:29,498::task::495::TaskManager.Task::(_debug) Task 7716432c-19ca-4116-990d-1c8bc53a3f06: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:33:29,499::task::495::TaskManager.Task::(_debug) Task 7716432c-19ca-4116-990d-1c8bc53a3f06: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:33:29,500::task::495::TaskManager.Task::(_debug) Task 7716432c-19ca-4116-990d-1c8bc53a3f06: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:33:29,500::task::495::TaskManager.Task::(_debug) Task 7716432c-19ca-4116-990d-1c8bc53a3f06: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:33:29,501::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:33:29,501::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:33:29,502::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:33:29,502::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:33:29,502::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:33:29,503::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:33:29,503::task::495::TaskManager.Task::(_debug) Task 7716432c-19ca-4116-990d-1c8bc53a3f06: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:33:29,504::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:33:29,504::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:33:29,505::task::495::TaskManager.Task::(_debug) Task 888526e3-ba4e-4b60-bfe2-0f4c8f398d14: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:33:29,506::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2d01c298-ea4b-42c9-bee1-45b4907c3b35`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:33:29,506::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:33:29,507::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:33:29,507::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2d01c298-ea4b-42c9-bee1-45b4907c3b35`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:33:29,508::task::495::TaskManager.Task::(_debug) Task 888526e3-ba4e-4b60-bfe2-0f4c8f398d14: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:33:29,508::task::495::TaskManager.Task::(_debug) Task 888526e3-ba4e-4b60-bfe2-0f4c8f398d14: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:33:29,509::task::495::TaskManager.Task::(_debug) Task 888526e3-ba4e-4b60-bfe2-0f4c8f398d14: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:33:29,509::task::495::TaskManager.Task::(_debug) Task 888526e3-ba4e-4b60-bfe2-0f4c8f398d14: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:33:29,509::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:33:29,510::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:33:29,510::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:33:29,511::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:33:29,511::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:33:29,511::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:33:29,512::task::495::TaskManager.Task::(_debug) Task 888526e3-ba4e-4b60-bfe2-0f4c8f398d14: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:33:29,512::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:33:29,513::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:33:29,513::task::495::TaskManager.Task::(_debug) Task 7b99fbbd-c089-4f07-a008-7b110db04b51: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:33:29,514::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f3bd4daa-de51-42fe-9f30-c4ad1ff62c8b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:33:29,514::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:33:29,515::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:33:29,515::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f3bd4daa-de51-42fe-9f30-c4ad1ff62c8b`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:33:29,516::task::495::TaskManager.Task::(_debug) Task 7b99fbbd-c089-4f07-a008-7b110db04b51: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:33:29,516::task::495::TaskManager.Task::(_debug) Task 7b99fbbd-c089-4f07-a008-7b110db04b51: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:33:29,517::task::495::TaskManager.Task::(_debug) Task 7b99fbbd-c089-4f07-a008-7b110db04b51: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:33:29,517::task::495::TaskManager.Task::(_debug) Task 7b99fbbd-c089-4f07-a008-7b110db04b51: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:33:29,518::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:33:29,518::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:33:29,518::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:33:29,519::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:33:29,519::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:33:29,520::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:33:29,520::task::495::TaskManager.Task::(_debug) Task 7b99fbbd-c089-4f07-a008-7b110db04b51: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:33:29,521::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:33:29,727::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:33:30,388::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0490419 s, 20.9 MB/s\n'; <rc> = 0
>Thread-15914530::DEBUG::2013-11-20 03:33:32,330::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('e97d1c52-b688-4192-bdeb-2dfadcc79786',) {}
>Thread-15914530::DEBUG::2013-11-20 03:33:32,331::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::CPU stats not available
>Thread-15914530::DEBUG::2013-11-20 03:33:32,331::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::Disk stats not available
>Thread-15914530::DEBUG::2013-11-20 03:33:32,332::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::Disk latency not available
>Thread-15914530::DEBUG::2013-11-20 03:33:32,333::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '7039', 'displayIp': '0', 'displayPort': u'5916', 'session': 'Unknown', 'displaySecurePort': u'5917', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/RHEV-toolsSetup_3.0_39.iso', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet9': {'macAddr': u'00:1a:4a:a8:64:13', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet9'}}, 'vmId': 'e97d1c52-b688-4192-bdeb-2dfadcc79786', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '86033563648', 'writeLatency': '0', 'imageID': '5af747d8-95b1-4b7e-af53-cb77e4fcc896', 'flushLatency': '0', 'truesize': '86033563648'}}, 'boot': 'c', 'statsAge': '1.92', 'cpuIdle': '100.00', 'elapsedTime': '49', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '0', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:33:32,399::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:33:32,819::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0466718 s, 21.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:33:34,830::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:33:35,321::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0530039 s, 19.3 MB/s\n'; <rc> = 0
>Thread-15914532::DEBUG::2013-11-20 03:33:35,369::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('e97d1c52-b688-4192-bdeb-2dfadcc79786',) {}
>Thread-15914532::DEBUG::2013-11-20 03:33:35,370::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::CPU stats not available
>Thread-15914532::DEBUG::2013-11-20 03:33:35,371::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::Disk stats not available
>Thread-15914532::DEBUG::2013-11-20 03:33:35,371::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::Disk latency not available
>Thread-15914532::DEBUG::2013-11-20 03:33:35,372::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '7039', 'displayIp': '0', 'displayPort': u'5916', 'session': 'Unknown', 'displaySecurePort': u'5917', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/RHEV-toolsSetup_3.0_39.iso', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet9': {'macAddr': u'00:1a:4a:a8:64:13', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet9'}}, 'vmId': 'e97d1c52-b688-4192-bdeb-2dfadcc79786', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '86033563648', 'writeLatency': '0', 'imageID': '5af747d8-95b1-4b7e-af53-cb77e4fcc896', 'flushLatency': '0', 'truesize': '86033563648'}}, 'boot': 'c', 'statsAge': '0.81', 'cpuIdle': '100.00', 'elapsedTime': '52', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '0', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:33:37,331::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:33:37,767::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.047309 s, 21.6 MB/s\n'; <rc> = 0
>Thread-15914534::DEBUG::2013-11-20 03:33:38,396::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('e97d1c52-b688-4192-bdeb-2dfadcc79786',) {}
>Thread-15914534::DEBUG::2013-11-20 03:33:38,398::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::Disk stats not available
>Thread-15914534::DEBUG::2013-11-20 03:33:38,398::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::Disk latency not available
>Thread-15914534::DEBUG::2013-11-20 03:33:38,399::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '7039', 'displayIp': '0', 'displayPort': u'5916', 'session': 'Unknown', 'displaySecurePort': u'5917', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/RHEV-toolsSetup_3.0_39.iso', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet9': {'macAddr': u'00:1a:4a:a8:64:13', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet9'}}, 'vmId': 'e97d1c52-b688-4192-bdeb-2dfadcc79786', 'monitorResponse': '0', 'cpuUser': '51.69', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '86033563648', 'writeLatency': '0', 'imageID': '5af747d8-95b1-4b7e-af53-cb77e4fcc896', 'flushLatency': '0', 'truesize': '86033563648'}}, 'boot': 'c', 'statsAge': '0.84', 'cpuIdle': '48.31', 'elapsedTime': '55', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '0', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15914535::DEBUG::2013-11-20 03:33:38,407::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914535::INFO::2013-11-20 03:33:38,408::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914535::DEBUG::2013-11-20 03:33:38,409::task::495::TaskManager.Task::(_debug) Task 72db870f-1fd8-4993-8839-390dfd396dc6: moving from state init -> state preparing
>Thread-15914535::DEBUG::2013-11-20 03:33:38,410::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914535::DEBUG::2013-11-20 03:33:38,410::task::495::TaskManager.Task::(_debug) Task 72db870f-1fd8-4993-8839-390dfd396dc6: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914535::DEBUG::2013-11-20 03:33:38,411::task::495::TaskManager.Task::(_debug) Task 72db870f-1fd8-4993-8839-390dfd396dc6: moving from state preparing -> state finished
>Thread-15914535::DEBUG::2013-11-20 03:33:38,412::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914535::DEBUG::2013-11-20 03:33:38,412::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914535::DEBUG::2013-11-20 03:33:38,413::task::495::TaskManager.Task::(_debug) Task 72db870f-1fd8-4993-8839-390dfd396dc6: ref 0 aborting False
>Thread-15914535::INFO::2013-11-20 03:33:38,413::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914536::DEBUG::2013-11-20 03:33:38,427::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914536::INFO::2013-11-20 03:33:38,428::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914536::DEBUG::2013-11-20 03:33:38,429::task::495::TaskManager.Task::(_debug) Task a78d133a-0846-43db-91e1-bfac8cc33032: moving from state init -> state preparing
>Thread-15914536::DEBUG::2013-11-20 03:33:38,436::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ee9d4292-16ae-48fc-a3d0-c1a17f61bd1e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914536::DEBUG::2013-11-20 03:33:38,437::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914536::DEBUG::2013-11-20 03:33:38,438::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914536::DEBUG::2013-11-20 03:33:38,438::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ee9d4292-16ae-48fc-a3d0-c1a17f61bd1e`::Granted request
>Thread-15914536::DEBUG::2013-11-20 03:33:38,440::task::495::TaskManager.Task::(_debug) Task a78d133a-0846-43db-91e1-bfac8cc33032: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914536::DEBUG::2013-11-20 03:33:38,440::task::495::TaskManager.Task::(_debug) Task a78d133a-0846-43db-91e1-bfac8cc33032: ref 1 aborting False
>Thread-15914536::WARNING::2013-11-20 03:33:38,443::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914536::DEBUG::2013-11-20 03:33:38,444::task::495::TaskManager.Task::(_debug) Task a78d133a-0846-43db-91e1-bfac8cc33032: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914536::DEBUG::2013-11-20 03:33:38,445::task::495::TaskManager.Task::(_debug) Task a78d133a-0846-43db-91e1-bfac8cc33032: moving from state preparing -> state finished
>Thread-15914536::DEBUG::2013-11-20 03:33:38,445::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914536::DEBUG::2013-11-20 03:33:38,446::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914536::DEBUG::2013-11-20 03:33:38,447::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914536::DEBUG::2013-11-20 03:33:38,447::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914536::DEBUG::2013-11-20 03:33:38,448::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914536::DEBUG::2013-11-20 03:33:38,449::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914536::DEBUG::2013-11-20 03:33:38,449::task::495::TaskManager.Task::(_debug) Task a78d133a-0846-43db-91e1-bfac8cc33032: ref 0 aborting False
>Thread-15914536::INFO::2013-11-20 03:33:38,450::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:33:39,777::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:33:40,664::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0908262 s, 11.3 MB/s\n'; <rc> = 0
>Thread-15914537::INFO::2013-11-20 03:33:41,441::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914537::DEBUG::2013-11-20 03:33:41,442::task::495::TaskManager.Task::(_debug) Task ad593370-5876-4500-9e4c-2b7010cf1b90: moving from state init -> state preparing
>Thread-15914537::DEBUG::2013-11-20 03:33:41,443::task::495::TaskManager.Task::(_debug) Task ad593370-5876-4500-9e4c-2b7010cf1b90: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000235795974731', 'lastCheck': 1384918417.1302299, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.000959873199463', 'lastCheck': 1384918413.1315451, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000960111618042', 'lastCheck': 1384918416.5685389, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000353097915649', 'lastCheck': 1384918411.5051551, 'valid': True, 'code': 0}}
>Thread-15914537::DEBUG::2013-11-20 03:33:41,447::task::495::TaskManager.Task::(_debug) Task ad593370-5876-4500-9e4c-2b7010cf1b90: moving from state preparing -> state finished
>Thread-15914537::DEBUG::2013-11-20 03:33:41,448::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914537::DEBUG::2013-11-20 03:33:41,448::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914537::DEBUG::2013-11-20 03:33:41,449::task::495::TaskManager.Task::(_debug) Task ad593370-5876-4500-9e4c-2b7010cf1b90: ref 0 aborting False
>Thread-15914537::INFO::2013-11-20 03:33:41,449::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000235795974731', 'lastCheck': 1384918417.1302299, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.000959873199463', 'lastCheck': 1384918413.1315451, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000960111618042', 'lastCheck': 1384918416.5685389, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000353097915649', 'lastCheck': 1384918411.5051551, 'valid': True, 'code': 0}}
>Thread-15914538::DEBUG::2013-11-20 03:33:41,557::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::Disk stats not available
>Thread-15914538::DEBUG::2013-11-20 03:33:41,559::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::Disk latency not available
>Dummy-15912651::DEBUG::2013-11-20 03:33:42,677::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:33:43,713::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.117225 s, 8.7 MB/s\n'; <rc> = 0
>Thread-15913639::INFO::2013-11-20 03:33:44,569::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:33:44,588::task::495::TaskManager.Task::(_debug) Task 1eb91a55-8dfe-450b-87a5-c8ac33861ea6: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:33:44,589::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c4aa3ac7-4f19-48f4-83c3-6fd0d23a6fbd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:33:44,590::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:33:44,591::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:33:44,592::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c4aa3ac7-4f19-48f4-83c3-6fd0d23a6fbd`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:33:44,593::task::495::TaskManager.Task::(_debug) Task 1eb91a55-8dfe-450b-87a5-c8ac33861ea6: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:33:44,594::task::495::TaskManager.Task::(_debug) Task 1eb91a55-8dfe-450b-87a5-c8ac33861ea6: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:33:44,595::task::495::TaskManager.Task::(_debug) Task 1eb91a55-8dfe-450b-87a5-c8ac33861ea6: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:33:44,595::task::495::TaskManager.Task::(_debug) Task 1eb91a55-8dfe-450b-87a5-c8ac33861ea6: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:33:44,596::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:33:44,596::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:33:44,597::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:33:44,597::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:33:44,598::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:33:44,598::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:33:44,599::task::495::TaskManager.Task::(_debug) Task 1eb91a55-8dfe-450b-87a5-c8ac33861ea6: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:33:44,604::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15914540::DEBUG::2013-11-20 03:33:45,121::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('e97d1c52-b688-4192-bdeb-2dfadcc79786',) {}
>Thread-15914540::DEBUG::2013-11-20 03:33:45,122::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::Disk stats not available
>Thread-15914540::DEBUG::2013-11-20 03:33:45,122::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::Disk latency not available
>Thread-15914540::DEBUG::2013-11-20 03:33:45,123::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '7039', 'displayIp': '0', 'displayPort': u'5916', 'session': 'Unknown', 'displaySecurePort': u'5917', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/RHEV-toolsSetup_3.0_39.iso', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet9': {'macAddr': u'00:1a:4a:a8:64:13', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet9'}}, 'vmId': 'e97d1c52-b688-4192-bdeb-2dfadcc79786', 'monitorResponse': '0', 'cpuUser': '51.69', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '86033563648', 'writeLatency': '0', 'imageID': '5af747d8-95b1-4b7e-af53-cb77e4fcc896', 'flushLatency': '0', 'truesize': '86033563648'}}, 'boot': 'c', 'statsAge': '0.54', 'cpuIdle': '48.31', 'elapsedTime': '62', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '0', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:33:45,723::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:33:46,082::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0477413 s, 21.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:33:48,092::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:33:48,548::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0571942 s, 17.9 MB/s\n'; <rc> = 0
>Thread-15914542::DEBUG::2013-11-20 03:33:48,716::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914542::INFO::2013-11-20 03:33:48,717::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914542::DEBUG::2013-11-20 03:33:48,718::task::495::TaskManager.Task::(_debug) Task 9ca5293f-ff85-4844-8ac3-5e7d827d2a4b: moving from state init -> state preparing
>Thread-15914542::DEBUG::2013-11-20 03:33:48,719::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914542::DEBUG::2013-11-20 03:33:48,720::task::495::TaskManager.Task::(_debug) Task 9ca5293f-ff85-4844-8ac3-5e7d827d2a4b: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914542::DEBUG::2013-11-20 03:33:48,721::task::495::TaskManager.Task::(_debug) Task 9ca5293f-ff85-4844-8ac3-5e7d827d2a4b: moving from state preparing -> state finished
>Thread-15914542::DEBUG::2013-11-20 03:33:48,721::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914542::DEBUG::2013-11-20 03:33:48,722::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914542::DEBUG::2013-11-20 03:33:48,723::task::495::TaskManager.Task::(_debug) Task 9ca5293f-ff85-4844-8ac3-5e7d827d2a4b: ref 0 aborting False
>Thread-15914542::INFO::2013-11-20 03:33:48,723::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914543::DEBUG::2013-11-20 03:33:48,740::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914543::INFO::2013-11-20 03:33:48,741::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914543::DEBUG::2013-11-20 03:33:48,742::task::495::TaskManager.Task::(_debug) Task 1671377d-be56-4380-823b-519a3479e090: moving from state init -> state preparing
>Thread-15914543::DEBUG::2013-11-20 03:33:48,746::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5a134d03-f077-4f7e-9bc2-973c45c1e7f7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914543::DEBUG::2013-11-20 03:33:48,747::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914543::DEBUG::2013-11-20 03:33:48,748::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914543::DEBUG::2013-11-20 03:33:48,749::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5a134d03-f077-4f7e-9bc2-973c45c1e7f7`::Granted request
>Thread-15914543::DEBUG::2013-11-20 03:33:48,753::task::495::TaskManager.Task::(_debug) Task 1671377d-be56-4380-823b-519a3479e090: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914543::DEBUG::2013-11-20 03:33:48,754::task::495::TaskManager.Task::(_debug) Task 1671377d-be56-4380-823b-519a3479e090: ref 1 aborting False
>Thread-15914543::WARNING::2013-11-20 03:33:48,758::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914543::DEBUG::2013-11-20 03:33:48,762::task::495::TaskManager.Task::(_debug) Task 1671377d-be56-4380-823b-519a3479e090: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914543::DEBUG::2013-11-20 03:33:48,763::task::495::TaskManager.Task::(_debug) Task 1671377d-be56-4380-823b-519a3479e090: moving from state preparing -> state finished
>Thread-15914543::DEBUG::2013-11-20 03:33:48,763::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914543::DEBUG::2013-11-20 03:33:48,764::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914543::DEBUG::2013-11-20 03:33:48,765::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914543::DEBUG::2013-11-20 03:33:48,766::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914543::DEBUG::2013-11-20 03:33:48,767::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914543::DEBUG::2013-11-20 03:33:48,768::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914543::DEBUG::2013-11-20 03:33:48,769::task::495::TaskManager.Task::(_debug) Task 1671377d-be56-4380-823b-519a3479e090: ref 0 aborting False
>Thread-15914543::INFO::2013-11-20 03:33:48,770::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:33:50,559::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:33:50,962::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0515776 s, 19.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:33:52,972::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:33:53,416::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0490247 s, 20.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:33:55,431::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:33:55,865::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0472158 s, 21.7 MB/s\n'; <rc> = 0
>Thread-15914546::INFO::2013-11-20 03:33:57,452::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914546::DEBUG::2013-11-20 03:33:57,453::task::495::TaskManager.Task::(_debug) Task 4a71ddfb-cb93-4878-a24f-c7d85c117342: moving from state init -> state preparing
>Thread-15914546::DEBUG::2013-11-20 03:33:57,454::task::495::TaskManager.Task::(_debug) Task 4a71ddfb-cb93-4878-a24f-c7d85c117342: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000285148620605', 'lastCheck': 1384918437.2874401, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00111794471741', 'lastCheck': 1384918433.2869589, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000910997390747', 'lastCheck': 1384918436.7378831, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000389099121094', 'lastCheck': 1384918431.7309451, 'valid': True, 'code': 0}}
>Thread-15914546::DEBUG::2013-11-20 03:33:57,454::task::495::TaskManager.Task::(_debug) Task 4a71ddfb-cb93-4878-a24f-c7d85c117342: moving from state preparing -> state finished
>Thread-15914546::DEBUG::2013-11-20 03:33:57,455::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914546::DEBUG::2013-11-20 03:33:57,455::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914546::DEBUG::2013-11-20 03:33:57,456::task::495::TaskManager.Task::(_debug) Task 4a71ddfb-cb93-4878-a24f-c7d85c117342: ref 0 aborting False
>Thread-15914546::INFO::2013-11-20 03:33:57,456::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000285148620605', 'lastCheck': 1384918437.2874401, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00111794471741', 'lastCheck': 1384918433.2869589, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000910997390747', 'lastCheck': 1384918436.7378831, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000389099121094', 'lastCheck': 1384918431.7309451, 'valid': True, 'code': 0}}
>Thread-15914547::DEBUG::2013-11-20 03:33:57,522::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::Disk stats not available
>Thread-15914547::DEBUG::2013-11-20 03:33:57,523::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::Disk latency not available
>Dummy-15912651::DEBUG::2013-11-20 03:33:57,875::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:33:58,387::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.061715 s, 16.6 MB/s\n'; <rc> = 0
>Thread-15914548::DEBUG::2013-11-20 03:33:59,328::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914548::INFO::2013-11-20 03:33:59,329::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914548::DEBUG::2013-11-20 03:33:59,330::task::495::TaskManager.Task::(_debug) Task 88569765-0000-410d-b85c-73921519231e: moving from state init -> state preparing
>Thread-15914548::DEBUG::2013-11-20 03:33:59,331::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914548::DEBUG::2013-11-20 03:33:59,331::task::495::TaskManager.Task::(_debug) Task 88569765-0000-410d-b85c-73921519231e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914548::DEBUG::2013-11-20 03:33:59,332::task::495::TaskManager.Task::(_debug) Task 88569765-0000-410d-b85c-73921519231e: moving from state preparing -> state finished
>Thread-15914548::DEBUG::2013-11-20 03:33:59,332::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914548::DEBUG::2013-11-20 03:33:59,333::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914548::DEBUG::2013-11-20 03:33:59,333::task::495::TaskManager.Task::(_debug) Task 88569765-0000-410d-b85c-73921519231e: ref 0 aborting False
>Thread-15914548::INFO::2013-11-20 03:33:59,334::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914549::DEBUG::2013-11-20 03:33:59,340::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914549::INFO::2013-11-20 03:33:59,341::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914549::DEBUG::2013-11-20 03:33:59,341::task::495::TaskManager.Task::(_debug) Task f91a39f3-35f3-4b91-9089-5316d1f61ec6: moving from state init -> state preparing
>Thread-15914549::DEBUG::2013-11-20 03:33:59,342::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`19dbde40-010f-42ce-9e7d-a938da84e20b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914549::DEBUG::2013-11-20 03:33:59,342::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914549::DEBUG::2013-11-20 03:33:59,343::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914549::DEBUG::2013-11-20 03:33:59,343::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`19dbde40-010f-42ce-9e7d-a938da84e20b`::Granted request
>Thread-15914549::DEBUG::2013-11-20 03:33:59,344::task::495::TaskManager.Task::(_debug) Task f91a39f3-35f3-4b91-9089-5316d1f61ec6: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914549::DEBUG::2013-11-20 03:33:59,348::task::495::TaskManager.Task::(_debug) Task f91a39f3-35f3-4b91-9089-5316d1f61ec6: ref 1 aborting False
>Thread-15914549::WARNING::2013-11-20 03:33:59,350::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914549::DEBUG::2013-11-20 03:33:59,350::task::495::TaskManager.Task::(_debug) Task f91a39f3-35f3-4b91-9089-5316d1f61ec6: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914549::DEBUG::2013-11-20 03:33:59,351::task::495::TaskManager.Task::(_debug) Task f91a39f3-35f3-4b91-9089-5316d1f61ec6: moving from state preparing -> state finished
>Thread-15914549::DEBUG::2013-11-20 03:33:59,351::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914549::DEBUG::2013-11-20 03:33:59,352::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914549::DEBUG::2013-11-20 03:33:59,352::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914549::DEBUG::2013-11-20 03:33:59,352::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914549::DEBUG::2013-11-20 03:33:59,353::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914549::DEBUG::2013-11-20 03:33:59,353::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914549::DEBUG::2013-11-20 03:33:59,353::task::495::TaskManager.Task::(_debug) Task f91a39f3-35f3-4b91-9089-5316d1f61ec6: ref 0 aborting False
>Thread-15914549::INFO::2013-11-20 03:33:59,354::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913658::INFO::2013-11-20 03:33:59,608::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:33:59,609::task::495::TaskManager.Task::(_debug) Task 78ed25f3-f2eb-4b37-8627-31d657a7faa9: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:33:59,610::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`59f5f36c-c313-4863-be6c-f809883150a9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:33:59,610::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:33:59,611::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:33:59,611::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`59f5f36c-c313-4863-be6c-f809883150a9`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:33:59,612::task::495::TaskManager.Task::(_debug) Task 78ed25f3-f2eb-4b37-8627-31d657a7faa9: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:33:59,612::task::495::TaskManager.Task::(_debug) Task 78ed25f3-f2eb-4b37-8627-31d657a7faa9: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:33:59,613::task::495::TaskManager.Task::(_debug) Task 78ed25f3-f2eb-4b37-8627-31d657a7faa9: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:33:59,613::task::495::TaskManager.Task::(_debug) Task 78ed25f3-f2eb-4b37-8627-31d657a7faa9: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:33:59,614::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:33:59,614::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:33:59,615::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:33:59,615::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:33:59,616::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:33:59,616::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:33:59,617::task::495::TaskManager.Task::(_debug) Task 78ed25f3-f2eb-4b37-8627-31d657a7faa9: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:33:59,618::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:34:00,397::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:34:01,851::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0504666 s, 20.3 MB/s\n'; <rc> = 0
>Thread-15914436::DEBUG::2013-11-20 03:34:03,372::guestIF::97::vm.Vm::(_handleMessage) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::u'None'
>Dummy-15912651::DEBUG::2013-11-20 03:34:03,861::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:34:04,310::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0559272 s, 18.3 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 03:34:04,736::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:34:04,737::task::495::TaskManager.Task::(_debug) Task 8a13a4e5-8786-4905-9497-a655304146a8: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:34:04,739::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e71562e9-64ab-41bc-9634-2db524069139`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:34:04,740::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:34:04,740::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:34:04,741::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e71562e9-64ab-41bc-9634-2db524069139`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:34:04,742::task::495::TaskManager.Task::(_debug) Task 8a13a4e5-8786-4905-9497-a655304146a8: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:34:04,742::task::495::TaskManager.Task::(_debug) Task 8a13a4e5-8786-4905-9497-a655304146a8: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:34:04,743::task::495::TaskManager.Task::(_debug) Task 8a13a4e5-8786-4905-9497-a655304146a8: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:34:04,743::task::495::TaskManager.Task::(_debug) Task 8a13a4e5-8786-4905-9497-a655304146a8: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:34:04,743::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:34:04,744::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:34:04,744::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:34:04,745::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:34:04,745::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:34:04,745::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:34:04,746::task::495::TaskManager.Task::(_debug) Task 8a13a4e5-8786-4905-9497-a655304146a8: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:34:04,746::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:34:06,321::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 03:34:06,499::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:34:06,501::task::495::TaskManager.Task::(_debug) Task 71ad15f7-895e-406e-a20f-fe1257b2f516: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:34:06,503::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f35a4b0b-9931-494e-ac8a-e7ca5d4d6138`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:34:06,504::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:34:06,505::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:34:06,505::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f35a4b0b-9931-494e-ac8a-e7ca5d4d6138`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:34:06,506::task::495::TaskManager.Task::(_debug) Task 71ad15f7-895e-406e-a20f-fe1257b2f516: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:34:06,506::task::495::TaskManager.Task::(_debug) Task 71ad15f7-895e-406e-a20f-fe1257b2f516: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:34:06,507::task::495::TaskManager.Task::(_debug) Task 71ad15f7-895e-406e-a20f-fe1257b2f516: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:34:06,507::task::495::TaskManager.Task::(_debug) Task 71ad15f7-895e-406e-a20f-fe1257b2f516: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:34:06,507::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:34:06,508::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:34:06,508::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:34:06,509::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:34:06,509::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:34:06,509::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:34:06,510::task::495::TaskManager.Task::(_debug) Task 71ad15f7-895e-406e-a20f-fe1257b2f516: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:34:06,510::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:34:06,758::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0522821 s, 19.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:34:08,769::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:34:09,194::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0476029 s, 21.5 MB/s\n'; <rc> = 0
>Thread-15914553::DEBUG::2013-11-20 03:34:10,081::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914553::INFO::2013-11-20 03:34:10,082::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914553::DEBUG::2013-11-20 03:34:10,083::task::495::TaskManager.Task::(_debug) Task 4d471e2d-96e6-4c4e-b76f-c1089317b351: moving from state init -> state preparing
>Thread-15914553::DEBUG::2013-11-20 03:34:10,084::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914553::DEBUG::2013-11-20 03:34:10,084::task::495::TaskManager.Task::(_debug) Task 4d471e2d-96e6-4c4e-b76f-c1089317b351: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914553::DEBUG::2013-11-20 03:34:10,084::task::495::TaskManager.Task::(_debug) Task 4d471e2d-96e6-4c4e-b76f-c1089317b351: moving from state preparing -> state finished
>Thread-15914553::DEBUG::2013-11-20 03:34:10,085::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914553::DEBUG::2013-11-20 03:34:10,085::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914553::DEBUG::2013-11-20 03:34:10,086::task::495::TaskManager.Task::(_debug) Task 4d471e2d-96e6-4c4e-b76f-c1089317b351: ref 0 aborting False
>Thread-15914553::INFO::2013-11-20 03:34:10,086::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914554::DEBUG::2013-11-20 03:34:10,099::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914554::INFO::2013-11-20 03:34:10,099::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914554::DEBUG::2013-11-20 03:34:10,100::task::495::TaskManager.Task::(_debug) Task 1885de10-53b1-4f48-bcaf-dfd53df8de21: moving from state init -> state preparing
>Thread-15914554::DEBUG::2013-11-20 03:34:10,111::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`040ebcbe-c28f-4a11-8d50-628ddedd05da`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914554::DEBUG::2013-11-20 03:34:10,127::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914554::DEBUG::2013-11-20 03:34:10,135::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914554::DEBUG::2013-11-20 03:34:10,136::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`040ebcbe-c28f-4a11-8d50-628ddedd05da`::Granted request
>Thread-15914554::DEBUG::2013-11-20 03:34:10,136::task::495::TaskManager.Task::(_debug) Task 1885de10-53b1-4f48-bcaf-dfd53df8de21: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914554::DEBUG::2013-11-20 03:34:10,137::task::495::TaskManager.Task::(_debug) Task 1885de10-53b1-4f48-bcaf-dfd53df8de21: ref 1 aborting False
>Thread-15914554::WARNING::2013-11-20 03:34:10,138::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914554::DEBUG::2013-11-20 03:34:10,139::task::495::TaskManager.Task::(_debug) Task 1885de10-53b1-4f48-bcaf-dfd53df8de21: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914554::DEBUG::2013-11-20 03:34:10,139::task::495::TaskManager.Task::(_debug) Task 1885de10-53b1-4f48-bcaf-dfd53df8de21: moving from state preparing -> state finished
>Thread-15914554::DEBUG::2013-11-20 03:34:10,140::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914554::DEBUG::2013-11-20 03:34:10,140::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914554::DEBUG::2013-11-20 03:34:10,141::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914554::DEBUG::2013-11-20 03:34:10,141::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914554::DEBUG::2013-11-20 03:34:10,142::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914554::DEBUG::2013-11-20 03:34:10,142::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914554::DEBUG::2013-11-20 03:34:10,142::task::495::TaskManager.Task::(_debug) Task 1885de10-53b1-4f48-bcaf-dfd53df8de21: ref 0 aborting False
>Thread-15914554::INFO::2013-11-20 03:34:10,143::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:34:11,204::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:34:11,609::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0502048 s, 20.4 MB/s\n'; <rc> = 0
>Thread-15914556::INFO::2013-11-20 03:34:13,402::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914556::DEBUG::2013-11-20 03:34:13,402::task::495::TaskManager.Task::(_debug) Task 8a99443b-c742-4096-8794-0096a606b531: moving from state init -> state preparing
>Thread-15914556::DEBUG::2013-11-20 03:34:13,403::task::495::TaskManager.Task::(_debug) Task 8a99443b-c742-4096-8794-0096a606b531: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000276803970337', 'lastCheck': 1384918447.2985301, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00147891044617', 'lastCheck': 1384918453.3734889, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00186204910278', 'lastCheck': 1384918446.75231, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000308990478516', 'lastCheck': 1384918451.750294, 'valid': True, 'code': 0}}
>Thread-15914556::DEBUG::2013-11-20 03:34:13,403::task::495::TaskManager.Task::(_debug) Task 8a99443b-c742-4096-8794-0096a606b531: moving from state preparing -> state finished
>Thread-15914556::DEBUG::2013-11-20 03:34:13,404::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914556::DEBUG::2013-11-20 03:34:13,404::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914556::DEBUG::2013-11-20 03:34:13,404::task::495::TaskManager.Task::(_debug) Task 8a99443b-c742-4096-8794-0096a606b531: ref 0 aborting False
>Thread-15914556::INFO::2013-11-20 03:34:13,405::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000276803970337', 'lastCheck': 1384918447.2985301, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00147891044617', 'lastCheck': 1384918453.3734889, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00186204910278', 'lastCheck': 1384918446.75231, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000308990478516', 'lastCheck': 1384918451.750294, 'valid': True, 'code': 0}}
>Thread-15914557::DEBUG::2013-11-20 03:34:13,466::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::Disk stats not available
>Thread-15914557::DEBUG::2013-11-20 03:34:13,467::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::Disk latency not available
>Dummy-15912651::DEBUG::2013-11-20 03:34:13,620::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:34:14,015::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0493171 s, 20.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:34:16,026::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:34:16,470::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0550574 s, 18.6 MB/s\n'; <rc> = 0
>Thread-15914435::INFO::2013-11-20 03:34:16,538::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 03:34:16,539::task::495::TaskManager.Task::(_debug) Task 55d1a819-ec4c-470d-bb28-8cbfb31dbbd9: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 03:34:16,540::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f34cee40-71f1-43e0-9b20-2b1e9a2176bd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 03:34:16,540::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 03:34:16,541::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 03:34:16,541::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f34cee40-71f1-43e0-9b20-2b1e9a2176bd`::Granted request
>Thread-15914435::DEBUG::2013-11-20 03:34:16,542::task::495::TaskManager.Task::(_debug) Task 55d1a819-ec4c-470d-bb28-8cbfb31dbbd9: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 03:34:16,543::task::495::TaskManager.Task::(_debug) Task 55d1a819-ec4c-470d-bb28-8cbfb31dbbd9: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 03:34:16,543::task::495::TaskManager.Task::(_debug) Task 55d1a819-ec4c-470d-bb28-8cbfb31dbbd9: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 03:34:16,544::task::495::TaskManager.Task::(_debug) Task 55d1a819-ec4c-470d-bb28-8cbfb31dbbd9: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 03:34:16,544::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 03:34:16,544::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 03:34:16,545::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 03:34:16,545::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 03:34:16,546::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 03:34:16,546::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 03:34:16,547::task::495::TaskManager.Task::(_debug) Task 55d1a819-ec4c-470d-bb28-8cbfb31dbbd9: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 03:34:16,547::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 03:34:18,480::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:34:19,140::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.252569 s, 4.1 MB/s\n'; <rc> = 0
>Thread-15914453::INFO::2013-11-20 03:34:19,784::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 03:34:19,788::task::495::TaskManager.Task::(_debug) Task 80e453fb-2e41-425e-8543-f3155981aef7: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 03:34:19,791::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1cb4aa7b-f8e5-45a1-95e0-8f63e5ebc01e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 03:34:19,793::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 03:34:19,800::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 03:34:19,801::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1cb4aa7b-f8e5-45a1-95e0-8f63e5ebc01e`::Granted request
>Thread-15914453::DEBUG::2013-11-20 03:34:19,803::task::495::TaskManager.Task::(_debug) Task 80e453fb-2e41-425e-8543-f3155981aef7: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 03:34:19,803::task::495::TaskManager.Task::(_debug) Task 80e453fb-2e41-425e-8543-f3155981aef7: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 03:34:19,804::task::495::TaskManager.Task::(_debug) Task 80e453fb-2e41-425e-8543-f3155981aef7: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 03:34:19,804::task::495::TaskManager.Task::(_debug) Task 80e453fb-2e41-425e-8543-f3155981aef7: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 03:34:19,805::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 03:34:19,805::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 03:34:19,806::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 03:34:19,806::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 03:34:19,807::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 03:34:19,807::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 03:34:19,808::task::495::TaskManager.Task::(_debug) Task 80e453fb-2e41-425e-8543-f3155981aef7: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 03:34:19,808::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914560::DEBUG::2013-11-20 03:34:20,838::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914560::INFO::2013-11-20 03:34:20,839::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914560::DEBUG::2013-11-20 03:34:20,840::task::495::TaskManager.Task::(_debug) Task 5baae15d-2c85-4f25-a63f-c8e4ad296ed6: moving from state init -> state preparing
>Thread-15914560::DEBUG::2013-11-20 03:34:20,840::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914560::DEBUG::2013-11-20 03:34:20,841::task::495::TaskManager.Task::(_debug) Task 5baae15d-2c85-4f25-a63f-c8e4ad296ed6: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914560::DEBUG::2013-11-20 03:34:20,841::task::495::TaskManager.Task::(_debug) Task 5baae15d-2c85-4f25-a63f-c8e4ad296ed6: moving from state preparing -> state finished
>Thread-15914560::DEBUG::2013-11-20 03:34:20,842::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914560::DEBUG::2013-11-20 03:34:20,842::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914560::DEBUG::2013-11-20 03:34:20,842::task::495::TaskManager.Task::(_debug) Task 5baae15d-2c85-4f25-a63f-c8e4ad296ed6: ref 0 aborting False
>Thread-15914560::INFO::2013-11-20 03:34:20,843::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914561::DEBUG::2013-11-20 03:34:20,852::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914561::INFO::2013-11-20 03:34:20,853::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914561::DEBUG::2013-11-20 03:34:20,853::task::495::TaskManager.Task::(_debug) Task d4377859-f1b3-4946-b8e8-e91a96f06efd: moving from state init -> state preparing
>Thread-15914561::DEBUG::2013-11-20 03:34:20,859::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a8eb0bbe-7639-4a7c-bf99-fe1daff69525`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914561::DEBUG::2013-11-20 03:34:20,860::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914561::DEBUG::2013-11-20 03:34:20,861::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914561::DEBUG::2013-11-20 03:34:20,861::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a8eb0bbe-7639-4a7c-bf99-fe1daff69525`::Granted request
>Thread-15914561::DEBUG::2013-11-20 03:34:20,862::task::495::TaskManager.Task::(_debug) Task d4377859-f1b3-4946-b8e8-e91a96f06efd: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914561::DEBUG::2013-11-20 03:34:20,862::task::495::TaskManager.Task::(_debug) Task d4377859-f1b3-4946-b8e8-e91a96f06efd: ref 1 aborting False
>Thread-15914561::WARNING::2013-11-20 03:34:20,864::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914561::DEBUG::2013-11-20 03:34:20,865::task::495::TaskManager.Task::(_debug) Task d4377859-f1b3-4946-b8e8-e91a96f06efd: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914561::DEBUG::2013-11-20 03:34:20,866::task::495::TaskManager.Task::(_debug) Task d4377859-f1b3-4946-b8e8-e91a96f06efd: moving from state preparing -> state finished
>Thread-15914561::DEBUG::2013-11-20 03:34:20,866::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914561::DEBUG::2013-11-20 03:34:20,867::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914561::DEBUG::2013-11-20 03:34:20,867::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914561::DEBUG::2013-11-20 03:34:20,868::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914561::DEBUG::2013-11-20 03:34:20,868::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914561::DEBUG::2013-11-20 03:34:20,869::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914561::DEBUG::2013-11-20 03:34:20,869::task::495::TaskManager.Task::(_debug) Task d4377859-f1b3-4946-b8e8-e91a96f06efd: ref 0 aborting False
>Thread-15914561::INFO::2013-11-20 03:34:20,870::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:34:21,151::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913547::DEBUG::2013-11-20 03:34:21,324::guestIF::97::vm.Vm::(_handleMessage) vmId=`6574d88c-00ac-46f4-99ce-a2726932bfda`::u'None'
>Dummy-15912651::DEBUG::2013-11-20 03:34:21,576::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0481104 s, 21.3 MB/s\n'; <rc> = 0
>Thread-15914519::INFO::2013-11-20 03:34:23,032::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 03:34:23,043::task::495::TaskManager.Task::(_debug) Task 2c1a6768-0fd1-46be-92ff-d35a85e503a0: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 03:34:23,053::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`843ce11a-0ea2-4824-aa6a-9ed11765d9d5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 03:34:23,054::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 03:34:23,055::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 03:34:23,056::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`843ce11a-0ea2-4824-aa6a-9ed11765d9d5`::Granted request
>Thread-15914519::DEBUG::2013-11-20 03:34:23,066::task::495::TaskManager.Task::(_debug) Task 2c1a6768-0fd1-46be-92ff-d35a85e503a0: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 03:34:23,071::task::495::TaskManager.Task::(_debug) Task 2c1a6768-0fd1-46be-92ff-d35a85e503a0: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 03:34:23,071::task::495::TaskManager.Task::(_debug) Task 2c1a6768-0fd1-46be-92ff-d35a85e503a0: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 03:34:23,072::task::495::TaskManager.Task::(_debug) Task 2c1a6768-0fd1-46be-92ff-d35a85e503a0: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 03:34:23,073::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 03:34:23,074::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 03:34:23,075::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 03:34:23,075::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 03:34:23,076::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 03:34:23,077::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 03:34:23,077::task::495::TaskManager.Task::(_debug) Task 2c1a6768-0fd1-46be-92ff-d35a85e503a0: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 03:34:23,078::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 03:34:23,585::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:34:24,064::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0525048 s, 19.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:34:26,075::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:34:26,483::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0477096 s, 21.5 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 03:34:27,104::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:34:27,104::task::495::TaskManager.Task::(_debug) Task d04463b7-1a32-4145-ba3f-253f1be0afd1: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:34:27,110::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0f87fe20-0fba-4c56-ba45-13d1024377e6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:34:27,111::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:34:27,111::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:34:27,112::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0f87fe20-0fba-4c56-ba45-13d1024377e6`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:34:27,112::task::495::TaskManager.Task::(_debug) Task d04463b7-1a32-4145-ba3f-253f1be0afd1: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:34:27,113::task::495::TaskManager.Task::(_debug) Task d04463b7-1a32-4145-ba3f-253f1be0afd1: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:34:27,113::task::495::TaskManager.Task::(_debug) Task d04463b7-1a32-4145-ba3f-253f1be0afd1: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:34:27,113::task::495::TaskManager.Task::(_debug) Task d04463b7-1a32-4145-ba3f-253f1be0afd1: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:34:27,114::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:34:27,114::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:34:27,115::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:34:27,115::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:34:27,115::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:34:27,116::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:34:27,116::task::495::TaskManager.Task::(_debug) Task d04463b7-1a32-4145-ba3f-253f1be0afd1: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:34:27,116::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:34:27,117::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:34:27,117::task::495::TaskManager.Task::(_debug) Task 89e148b0-1ab5-4b5f-9e9c-b5e78ef1f658: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:34:27,118::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3d26ae06-df56-4cc1-8759-dc1ec3c2029e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:34:27,118::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:34:27,118::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:34:27,119::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3d26ae06-df56-4cc1-8759-dc1ec3c2029e`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:34:27,119::task::495::TaskManager.Task::(_debug) Task 89e148b0-1ab5-4b5f-9e9c-b5e78ef1f658: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:34:27,120::task::495::TaskManager.Task::(_debug) Task 89e148b0-1ab5-4b5f-9e9c-b5e78ef1f658: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:34:27,120::task::495::TaskManager.Task::(_debug) Task 89e148b0-1ab5-4b5f-9e9c-b5e78ef1f658: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:34:27,120::task::495::TaskManager.Task::(_debug) Task 89e148b0-1ab5-4b5f-9e9c-b5e78ef1f658: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:34:27,121::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:34:27,121::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:34:27,121::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:34:27,122::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:34:27,122::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:34:27,122::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:34:27,123::task::495::TaskManager.Task::(_debug) Task 89e148b0-1ab5-4b5f-9e9c-b5e78ef1f658: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:34:27,123::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:34:28,493::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914564::INFO::2013-11-20 03:34:29,310::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914564::DEBUG::2013-11-20 03:34:29,311::task::495::TaskManager.Task::(_debug) Task 16febbc3-d896-40fd-a952-c1e39325c811: moving from state init -> state preparing
>Thread-15914564::DEBUG::2013-11-20 03:34:29,312::task::495::TaskManager.Task::(_debug) Task 16febbc3-d896-40fd-a952-c1e39325c811: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000256061553955', 'lastCheck': 1384918467.3858111, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0106360912323', 'lastCheck': 1384918463.3950641, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000881910324097', 'lastCheck': 1384918467.0397849, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000660181045532', 'lastCheck': 1384918462.0317309, 'valid': True, 'code': 0}}
>Thread-15914564::DEBUG::2013-11-20 03:34:29,313::task::495::TaskManager.Task::(_debug) Task 16febbc3-d896-40fd-a952-c1e39325c811: moving from state preparing -> state finished
>Thread-15914564::DEBUG::2013-11-20 03:34:29,314::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914564::DEBUG::2013-11-20 03:34:29,314::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914564::DEBUG::2013-11-20 03:34:29,315::task::495::TaskManager.Task::(_debug) Task 16febbc3-d896-40fd-a952-c1e39325c811: ref 0 aborting False
>Thread-15914564::INFO::2013-11-20 03:34:29,315::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000256061553955', 'lastCheck': 1384918467.3858111, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0106360912323', 'lastCheck': 1384918463.3950641, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000881910324097', 'lastCheck': 1384918467.0397849, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000660181045532', 'lastCheck': 1384918462.0317309, 'valid': True, 'code': 0}}
>Thread-15913546::INFO::2013-11-20 03:34:29,804::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:34:29,805::task::495::TaskManager.Task::(_debug) Task 8b53dd1a-40e0-48b5-b8be-06eea0f58540: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:34:29,806::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fa215797-3550-496f-b0d3-691ff4216507`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:34:29,807::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:34:29,808::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:34:29,808::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fa215797-3550-496f-b0d3-691ff4216507`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:34:29,809::task::495::TaskManager.Task::(_debug) Task 8b53dd1a-40e0-48b5-b8be-06eea0f58540: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:34:29,809::task::495::TaskManager.Task::(_debug) Task 8b53dd1a-40e0-48b5-b8be-06eea0f58540: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:34:29,810::task::495::TaskManager.Task::(_debug) Task 8b53dd1a-40e0-48b5-b8be-06eea0f58540: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:34:29,810::task::495::TaskManager.Task::(_debug) Task 8b53dd1a-40e0-48b5-b8be-06eea0f58540: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:34:29,811::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:34:29,811::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:34:29,812::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:34:29,812::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:34:29,813::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:34:29,813::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:34:29,814::task::495::TaskManager.Task::(_debug) Task 8b53dd1a-40e0-48b5-b8be-06eea0f58540: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:34:29,814::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:34:29,815::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:34:29,815::task::495::TaskManager.Task::(_debug) Task 9913c84a-a44e-4640-b5c8-d5eb7c0ebe37: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:34:29,816::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`404895b7-ac02-43c3-b914-9675c58770f5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:34:29,816::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:34:29,817::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:34:29,817::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`404895b7-ac02-43c3-b914-9675c58770f5`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:34:29,818::task::495::TaskManager.Task::(_debug) Task 9913c84a-a44e-4640-b5c8-d5eb7c0ebe37: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:34:29,818::task::495::TaskManager.Task::(_debug) Task 9913c84a-a44e-4640-b5c8-d5eb7c0ebe37: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:34:29,819::task::495::TaskManager.Task::(_debug) Task 9913c84a-a44e-4640-b5c8-d5eb7c0ebe37: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:34:29,819::task::495::TaskManager.Task::(_debug) Task 9913c84a-a44e-4640-b5c8-d5eb7c0ebe37: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:34:29,820::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:34:29,820::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:34:29,821::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:34:29,821::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:34:29,821::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:34:29,822::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:34:29,822::task::495::TaskManager.Task::(_debug) Task 9913c84a-a44e-4640-b5c8-d5eb7c0ebe37: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:34:29,823::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:34:29,823::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:34:29,824::task::495::TaskManager.Task::(_debug) Task 7eaeca28-0767-47dd-a86c-a13f5b764d12: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:34:29,824::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d7cd440e-3539-4849-b534-fd8b348beab5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:34:29,825::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:34:29,825::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:34:29,826::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d7cd440e-3539-4849-b534-fd8b348beab5`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:34:29,826::task::495::TaskManager.Task::(_debug) Task 7eaeca28-0767-47dd-a86c-a13f5b764d12: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:34:29,827::task::495::TaskManager.Task::(_debug) Task 7eaeca28-0767-47dd-a86c-a13f5b764d12: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:34:29,827::task::495::TaskManager.Task::(_debug) Task 7eaeca28-0767-47dd-a86c-a13f5b764d12: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:34:29,843::task::495::TaskManager.Task::(_debug) Task 7eaeca28-0767-47dd-a86c-a13f5b764d12: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:34:29,844::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:34:29,844::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:34:29,845::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:34:29,845::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:34:29,846::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:34:29,846::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:34:29,847::task::495::TaskManager.Task::(_debug) Task 7eaeca28-0767-47dd-a86c-a13f5b764d12: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:34:29,847::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:34:29,848::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:34:29,848::task::495::TaskManager.Task::(_debug) Task 68b3aab4-b5d1-4d97-b125-124a6e2ebdfc: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:34:29,849::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`ccc4a9b8-bdf9-482b-b119-528608e6a1a3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:34:29,849::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:34:29,850::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:34:29,850::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`ccc4a9b8-bdf9-482b-b119-528608e6a1a3`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:34:29,851::task::495::TaskManager.Task::(_debug) Task 68b3aab4-b5d1-4d97-b125-124a6e2ebdfc: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:34:29,851::task::495::TaskManager.Task::(_debug) Task 68b3aab4-b5d1-4d97-b125-124a6e2ebdfc: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:34:29,852::task::495::TaskManager.Task::(_debug) Task 68b3aab4-b5d1-4d97-b125-124a6e2ebdfc: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:34:29,852::task::495::TaskManager.Task::(_debug) Task 68b3aab4-b5d1-4d97-b125-124a6e2ebdfc: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:34:29,853::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:34:29,853::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:34:29,853::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:34:29,854::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:34:29,854::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:34:29,855::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:34:29,855::task::495::TaskManager.Task::(_debug) Task 68b3aab4-b5d1-4d97-b125-124a6e2ebdfc: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:34:29,855::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15914566::DEBUG::2013-11-20 03:34:31,094::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914566::INFO::2013-11-20 03:34:31,095::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914566::DEBUG::2013-11-20 03:34:31,095::task::495::TaskManager.Task::(_debug) Task 80019b38-0d7e-490b-bba6-c16827362141: moving from state init -> state preparing
>Thread-15914566::DEBUG::2013-11-20 03:34:31,096::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914566::DEBUG::2013-11-20 03:34:31,096::task::495::TaskManager.Task::(_debug) Task 80019b38-0d7e-490b-bba6-c16827362141: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914566::DEBUG::2013-11-20 03:34:31,097::task::495::TaskManager.Task::(_debug) Task 80019b38-0d7e-490b-bba6-c16827362141: moving from state preparing -> state finished
>Thread-15914566::DEBUG::2013-11-20 03:34:31,097::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914566::DEBUG::2013-11-20 03:34:31,098::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914566::DEBUG::2013-11-20 03:34:31,098::task::495::TaskManager.Task::(_debug) Task 80019b38-0d7e-490b-bba6-c16827362141: ref 0 aborting False
>Thread-15914566::INFO::2013-11-20 03:34:31,099::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914567::DEBUG::2013-11-20 03:34:31,113::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914567::INFO::2013-11-20 03:34:31,114::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914567::DEBUG::2013-11-20 03:34:31,115::task::495::TaskManager.Task::(_debug) Task 7c7f5673-40da-47c1-aeca-011d0f611e1f: moving from state init -> state preparing
>Thread-15914567::DEBUG::2013-11-20 03:34:31,116::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`48272940-6dba-4bd5-915a-2d9c860b0c0a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914567::DEBUG::2013-11-20 03:34:31,116::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914567::DEBUG::2013-11-20 03:34:31,117::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914567::DEBUG::2013-11-20 03:34:31,117::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`48272940-6dba-4bd5-915a-2d9c860b0c0a`::Granted request
>Thread-15914567::DEBUG::2013-11-20 03:34:31,118::task::495::TaskManager.Task::(_debug) Task 7c7f5673-40da-47c1-aeca-011d0f611e1f: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914567::DEBUG::2013-11-20 03:34:31,119::task::495::TaskManager.Task::(_debug) Task 7c7f5673-40da-47c1-aeca-011d0f611e1f: ref 1 aborting False
>Thread-15914567::WARNING::2013-11-20 03:34:31,121::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914567::DEBUG::2013-11-20 03:34:31,122::task::495::TaskManager.Task::(_debug) Task 7c7f5673-40da-47c1-aeca-011d0f611e1f: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914567::DEBUG::2013-11-20 03:34:31,123::task::495::TaskManager.Task::(_debug) Task 7c7f5673-40da-47c1-aeca-011d0f611e1f: moving from state preparing -> state finished
>Thread-15914567::DEBUG::2013-11-20 03:34:31,124::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914567::DEBUG::2013-11-20 03:34:31,124::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914567::DEBUG::2013-11-20 03:34:31,125::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914567::DEBUG::2013-11-20 03:34:31,126::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914567::DEBUG::2013-11-20 03:34:31,126::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914567::DEBUG::2013-11-20 03:34:31,127::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914567::DEBUG::2013-11-20 03:34:31,127::task::495::TaskManager.Task::(_debug) Task 7c7f5673-40da-47c1-aeca-011d0f611e1f: ref 0 aborting False
>Thread-15914567::INFO::2013-11-20 03:34:31,128::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:34:31,314::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0568641 s, 18.0 MB/s\n'; <rc> = 0
>Thread-42840::DEBUG::2013-11-20 03:34:32,045::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 03:34:32,637::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:34:33,325::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:34:33,730::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.066942 s, 15.3 MB/s\n'; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-20 03:34:34,728::vm::525::vm.Vm::(_rtcUpdate) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::new rtc offset -1
>libvirtEventLoop::DEBUG::2013-11-20 03:34:34,729::vm::525::vm.Vm::(_rtcUpdate) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::new rtc offset -1
>libvirtEventLoop::DEBUG::2013-11-20 03:34:34,730::vm::525::vm.Vm::(_rtcUpdate) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::new rtc offset 3599
>libvirtEventLoop::DEBUG::2013-11-20 03:34:34,730::vm::525::vm.Vm::(_rtcUpdate) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::new rtc offset 3599
>libvirtEventLoop::DEBUG::2013-11-20 03:34:34,731::vm::525::vm.Vm::(_rtcUpdate) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::new rtc offset 3599
>libvirtEventLoop::DEBUG::2013-11-20 03:34:34,732::vm::525::vm.Vm::(_rtcUpdate) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::new rtc offset 3599
>libvirtEventLoop::DEBUG::2013-11-20 03:34:34,732::vm::525::vm.Vm::(_rtcUpdate) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::new rtc offset 3599
>libvirtEventLoop::DEBUG::2013-11-20 03:34:34,736::vm::525::vm.Vm::(_rtcUpdate) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::new rtc offset 3599
>libvirtEventLoop::DEBUG::2013-11-20 03:34:34,737::vm::525::vm.Vm::(_rtcUpdate) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::new rtc offset 3599
>libvirtEventLoop::DEBUG::2013-11-20 03:34:34,738::vm::525::vm.Vm::(_rtcUpdate) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::new rtc offset 3599
>libvirtEventLoop::DEBUG::2013-11-20 03:34:34,739::vm::525::vm.Vm::(_rtcUpdate) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::new rtc offset 3599
>libvirtEventLoop::DEBUG::2013-11-20 03:34:34,739::vm::525::vm.Vm::(_rtcUpdate) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::new rtc offset 3599
>libvirtEventLoop::DEBUG::2013-11-20 03:34:34,740::vm::525::vm.Vm::(_rtcUpdate) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::new rtc offset 3599
>libvirtEventLoop::DEBUG::2013-11-20 03:34:34,741::vm::525::vm.Vm::(_rtcUpdate) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::new rtc offset 3599
>Dummy-15912651::DEBUG::2013-11-20 03:34:35,740::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:34:36,234::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0436533 s, 23.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:34:38,243::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:34:38,625::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0509532 s, 20.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:34:40,635::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914571::DEBUG::2013-11-20 03:34:41,603::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914571::INFO::2013-11-20 03:34:41,605::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914571::DEBUG::2013-11-20 03:34:41,606::task::495::TaskManager.Task::(_debug) Task c0fdd9f3-a07d-454c-8ec0-7bf6b97e63ec: moving from state init -> state preparing
>Thread-15914571::DEBUG::2013-11-20 03:34:41,607::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914571::DEBUG::2013-11-20 03:34:41,607::task::495::TaskManager.Task::(_debug) Task c0fdd9f3-a07d-454c-8ec0-7bf6b97e63ec: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914571::DEBUG::2013-11-20 03:34:41,608::task::495::TaskManager.Task::(_debug) Task c0fdd9f3-a07d-454c-8ec0-7bf6b97e63ec: moving from state preparing -> state finished
>Thread-15914571::DEBUG::2013-11-20 03:34:41,608::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914571::DEBUG::2013-11-20 03:34:41,609::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914571::DEBUG::2013-11-20 03:34:41,609::task::495::TaskManager.Task::(_debug) Task c0fdd9f3-a07d-454c-8ec0-7bf6b97e63ec: ref 0 aborting False
>Thread-15914571::INFO::2013-11-20 03:34:41,609::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914572::DEBUG::2013-11-20 03:34:41,624::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914572::INFO::2013-11-20 03:34:41,625::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914572::DEBUG::2013-11-20 03:34:41,625::task::495::TaskManager.Task::(_debug) Task 92331752-9e28-4693-b91d-013829ff2ee7: moving from state init -> state preparing
>Thread-15914572::DEBUG::2013-11-20 03:34:41,627::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`071e312d-3c78-4136-8ad4-9f3a91246f71`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914572::DEBUG::2013-11-20 03:34:41,627::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914572::DEBUG::2013-11-20 03:34:41,628::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914572::DEBUG::2013-11-20 03:34:41,628::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`071e312d-3c78-4136-8ad4-9f3a91246f71`::Granted request
>Thread-15914572::DEBUG::2013-11-20 03:34:41,629::task::495::TaskManager.Task::(_debug) Task 92331752-9e28-4693-b91d-013829ff2ee7: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914572::DEBUG::2013-11-20 03:34:41,630::task::495::TaskManager.Task::(_debug) Task 92331752-9e28-4693-b91d-013829ff2ee7: ref 1 aborting False
>Thread-15914572::WARNING::2013-11-20 03:34:41,632::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914572::DEBUG::2013-11-20 03:34:41,633::task::495::TaskManager.Task::(_debug) Task 92331752-9e28-4693-b91d-013829ff2ee7: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914572::DEBUG::2013-11-20 03:34:41,634::task::495::TaskManager.Task::(_debug) Task 92331752-9e28-4693-b91d-013829ff2ee7: moving from state preparing -> state finished
>Thread-15914572::DEBUG::2013-11-20 03:34:41,634::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914572::DEBUG::2013-11-20 03:34:41,635::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914572::DEBUG::2013-11-20 03:34:41,638::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914572::DEBUG::2013-11-20 03:34:41,638::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914572::DEBUG::2013-11-20 03:34:41,639::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914572::DEBUG::2013-11-20 03:34:41,639::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914572::DEBUG::2013-11-20 03:34:41,640::task::495::TaskManager.Task::(_debug) Task 92331752-9e28-4693-b91d-013829ff2ee7: ref 0 aborting False
>Thread-15914572::INFO::2013-11-20 03:34:41,641::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:34:42,246::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0490081 s, 20.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:34:44,258::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 03:34:45,056::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:34:45,058::task::495::TaskManager.Task::(_debug) Task 186dca3c-9d07-49cc-8bff-66ee8da00d07: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:34:45,060::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`726773b1-a620-4c42-846b-9f03ea54a936`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:34:45,060::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:34:45,061::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:34:45,062::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`726773b1-a620-4c42-846b-9f03ea54a936`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:34:45,062::task::495::TaskManager.Task::(_debug) Task 186dca3c-9d07-49cc-8bff-66ee8da00d07: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:34:45,063::task::495::TaskManager.Task::(_debug) Task 186dca3c-9d07-49cc-8bff-66ee8da00d07: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:34:45,064::task::495::TaskManager.Task::(_debug) Task 186dca3c-9d07-49cc-8bff-66ee8da00d07: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:34:45,064::task::495::TaskManager.Task::(_debug) Task 186dca3c-9d07-49cc-8bff-66ee8da00d07: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:34:45,064::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:34:45,065::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:34:45,065::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:34:45,066::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:34:45,066::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:34:45,066::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:34:45,067::task::495::TaskManager.Task::(_debug) Task 186dca3c-9d07-49cc-8bff-66ee8da00d07: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:34:45,067::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15914574::INFO::2013-11-20 03:34:45,614::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914574::DEBUG::2013-11-20 03:34:45,615::task::495::TaskManager.Task::(_debug) Task ac88b49f-7dad-4a73-b432-d0b081fdf3a3: moving from state init -> state preparing
>Thread-15914574::DEBUG::2013-11-20 03:34:45,616::task::495::TaskManager.Task::(_debug) Task ac88b49f-7dad-4a73-b432-d0b081fdf3a3: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000227928161621', 'lastCheck': 1384918477.480813, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00110983848572', 'lastCheck': 1384918483.493912, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000996112823486', 'lastCheck': 1384918477.0526969, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00031590461731', 'lastCheck': 1384918482.8180659, 'valid': True, 'code': 0}}
>Thread-15914574::DEBUG::2013-11-20 03:34:45,617::task::495::TaskManager.Task::(_debug) Task ac88b49f-7dad-4a73-b432-d0b081fdf3a3: moving from state preparing -> state finished
>Thread-15914574::DEBUG::2013-11-20 03:34:45,618::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914574::DEBUG::2013-11-20 03:34:45,618::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914574::DEBUG::2013-11-20 03:34:45,619::task::495::TaskManager.Task::(_debug) Task ac88b49f-7dad-4a73-b432-d0b081fdf3a3: ref 0 aborting False
>Thread-15914574::INFO::2013-11-20 03:34:45,620::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000227928161621', 'lastCheck': 1384918477.480813, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00110983848572', 'lastCheck': 1384918483.493912, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000996112823486', 'lastCheck': 1384918477.0526969, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00031590461731', 'lastCheck': 1384918482.8180659, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:34:45,677::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0626231 s, 16.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:34:47,695::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:34:48,124::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0485093 s, 21.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:34:50,134::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:34:50,597::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0482564 s, 21.2 MB/s\n'; <rc> = 0
>Thread-15914578::DEBUG::2013-11-20 03:34:52,243::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914578::INFO::2013-11-20 03:34:52,245::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914578::DEBUG::2013-11-20 03:34:52,246::task::495::TaskManager.Task::(_debug) Task fbdff6fc-5526-4c10-b88f-33475eb021ae: moving from state init -> state preparing
>Thread-15914578::DEBUG::2013-11-20 03:34:52,248::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914578::DEBUG::2013-11-20 03:34:52,249::task::495::TaskManager.Task::(_debug) Task fbdff6fc-5526-4c10-b88f-33475eb021ae: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914578::DEBUG::2013-11-20 03:34:52,249::task::495::TaskManager.Task::(_debug) Task fbdff6fc-5526-4c10-b88f-33475eb021ae: moving from state preparing -> state finished
>Thread-15914578::DEBUG::2013-11-20 03:34:52,250::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914578::DEBUG::2013-11-20 03:34:52,250::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914578::DEBUG::2013-11-20 03:34:52,251::task::495::TaskManager.Task::(_debug) Task fbdff6fc-5526-4c10-b88f-33475eb021ae: ref 0 aborting False
>Thread-15914578::INFO::2013-11-20 03:34:52,251::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914579::DEBUG::2013-11-20 03:34:52,268::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914579::INFO::2013-11-20 03:34:52,269::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914579::DEBUG::2013-11-20 03:34:52,269::task::495::TaskManager.Task::(_debug) Task 335a05e6-b633-46d2-babc-2fc336b4ed3f: moving from state init -> state preparing
>Thread-15914579::DEBUG::2013-11-20 03:34:52,270::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e19a7ea0-1bc9-46e8-a915-edb7eb2f8c79`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914579::DEBUG::2013-11-20 03:34:52,271::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914579::DEBUG::2013-11-20 03:34:52,271::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914579::DEBUG::2013-11-20 03:34:52,277::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e19a7ea0-1bc9-46e8-a915-edb7eb2f8c79`::Granted request
>Thread-15914579::DEBUG::2013-11-20 03:34:52,279::task::495::TaskManager.Task::(_debug) Task 335a05e6-b633-46d2-babc-2fc336b4ed3f: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914579::DEBUG::2013-11-20 03:34:52,279::task::495::TaskManager.Task::(_debug) Task 335a05e6-b633-46d2-babc-2fc336b4ed3f: ref 1 aborting False
>Thread-15914579::WARNING::2013-11-20 03:34:52,281::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914579::DEBUG::2013-11-20 03:34:52,281::task::495::TaskManager.Task::(_debug) Task 335a05e6-b633-46d2-babc-2fc336b4ed3f: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914579::DEBUG::2013-11-20 03:34:52,282::task::495::TaskManager.Task::(_debug) Task 335a05e6-b633-46d2-babc-2fc336b4ed3f: moving from state preparing -> state finished
>Thread-15914579::DEBUG::2013-11-20 03:34:52,282::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914579::DEBUG::2013-11-20 03:34:52,283::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914579::DEBUG::2013-11-20 03:34:52,283::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914579::DEBUG::2013-11-20 03:34:52,284::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914579::DEBUG::2013-11-20 03:34:52,284::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914579::DEBUG::2013-11-20 03:34:52,284::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914579::DEBUG::2013-11-20 03:34:52,285::task::495::TaskManager.Task::(_debug) Task 335a05e6-b633-46d2-babc-2fc336b4ed3f: ref 0 aborting False
>Thread-15914579::INFO::2013-11-20 03:34:52,285::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:34:52,610::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:34:53,000::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0534901 s, 19.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:34:55,010::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:34:55,481::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0458152 s, 22.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:34:57,491::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:34:57,903::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0485026 s, 21.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:34:59,912::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 03:35:00,091::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:35:00,092::task::495::TaskManager.Task::(_debug) Task 67a94347-7254-4f4b-ba7b-96bebe64037e: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:35:00,094::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0ea9ae83-9571-4bc3-85e3-abf9ddc65e21`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:35:00,095::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:35:00,096::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:35:00,096::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0ea9ae83-9571-4bc3-85e3-abf9ddc65e21`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:35:00,097::task::495::TaskManager.Task::(_debug) Task 67a94347-7254-4f4b-ba7b-96bebe64037e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:35:00,098::task::495::TaskManager.Task::(_debug) Task 67a94347-7254-4f4b-ba7b-96bebe64037e: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:35:00,098::task::495::TaskManager.Task::(_debug) Task 67a94347-7254-4f4b-ba7b-96bebe64037e: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:35:00,099::task::495::TaskManager.Task::(_debug) Task 67a94347-7254-4f4b-ba7b-96bebe64037e: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:35:00,099::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:35:00,100::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:35:00,100::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:35:00,101::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:35:00,101::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:35:00,102::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:35:00,102::task::495::TaskManager.Task::(_debug) Task 67a94347-7254-4f4b-ba7b-96bebe64037e: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:35:00,103::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:35:00,964::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0535012 s, 19.1 MB/s\n'; <rc> = 0
>Thread-15914582::INFO::2013-11-20 03:35:01,360::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914582::DEBUG::2013-11-20 03:35:01,361::task::495::TaskManager.Task::(_debug) Task 8466d027-362b-4134-9e16-6f6f9224608d: moving from state init -> state preparing
>Thread-15914582::DEBUG::2013-11-20 03:35:01,362::task::495::TaskManager.Task::(_debug) Task 8466d027-362b-4134-9e16-6f6f9224608d: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000375986099243', 'lastCheck': 1384918497.6561351, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00091814994812', 'lastCheck': 1384918493.5062599, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0110697746277', 'lastCheck': 1384918497.2010829, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000396013259888', 'lastCheck': 1384918492.882699, 'valid': True, 'code': 0}}
>Thread-15914582::DEBUG::2013-11-20 03:35:01,363::task::495::TaskManager.Task::(_debug) Task 8466d027-362b-4134-9e16-6f6f9224608d: moving from state preparing -> state finished
>Thread-15914582::DEBUG::2013-11-20 03:35:01,363::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914582::DEBUG::2013-11-20 03:35:01,363::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914582::DEBUG::2013-11-20 03:35:01,364::task::495::TaskManager.Task::(_debug) Task 8466d027-362b-4134-9e16-6f6f9224608d: ref 0 aborting False
>Thread-15914582::INFO::2013-11-20 03:35:01,364::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000375986099243', 'lastCheck': 1384918497.6561351, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00091814994812', 'lastCheck': 1384918493.5062599, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0110697746277', 'lastCheck': 1384918497.2010829, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000396013259888', 'lastCheck': 1384918492.882699, 'valid': True, 'code': 0}}
>Thread-15914584::DEBUG::2013-11-20 03:35:02,571::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914584::INFO::2013-11-20 03:35:02,572::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914584::DEBUG::2013-11-20 03:35:02,573::task::495::TaskManager.Task::(_debug) Task 0f6f608b-304b-423b-8e09-c9384bd615ca: moving from state init -> state preparing
>Thread-15914584::DEBUG::2013-11-20 03:35:02,574::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914584::DEBUG::2013-11-20 03:35:02,574::task::495::TaskManager.Task::(_debug) Task 0f6f608b-304b-423b-8e09-c9384bd615ca: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914584::DEBUG::2013-11-20 03:35:02,574::task::495::TaskManager.Task::(_debug) Task 0f6f608b-304b-423b-8e09-c9384bd615ca: moving from state preparing -> state finished
>Thread-15914584::DEBUG::2013-11-20 03:35:02,575::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914584::DEBUG::2013-11-20 03:35:02,575::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914584::DEBUG::2013-11-20 03:35:02,576::task::495::TaskManager.Task::(_debug) Task 0f6f608b-304b-423b-8e09-c9384bd615ca: ref 0 aborting False
>Thread-15914584::INFO::2013-11-20 03:35:02,576::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914585::DEBUG::2013-11-20 03:35:02,587::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914585::INFO::2013-11-20 03:35:02,588::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914585::DEBUG::2013-11-20 03:35:02,588::task::495::TaskManager.Task::(_debug) Task 183b0390-7fdd-4510-bf6a-d232599ed96d: moving from state init -> state preparing
>Thread-15914585::DEBUG::2013-11-20 03:35:02,589::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b3f30a76-89de-469b-b244-9f98bff3892d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914585::DEBUG::2013-11-20 03:35:02,590::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914585::DEBUG::2013-11-20 03:35:02,590::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914585::DEBUG::2013-11-20 03:35:02,590::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b3f30a76-89de-469b-b244-9f98bff3892d`::Granted request
>Thread-15914585::DEBUG::2013-11-20 03:35:02,591::task::495::TaskManager.Task::(_debug) Task 183b0390-7fdd-4510-bf6a-d232599ed96d: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914585::DEBUG::2013-11-20 03:35:02,591::task::495::TaskManager.Task::(_debug) Task 183b0390-7fdd-4510-bf6a-d232599ed96d: ref 1 aborting False
>Thread-15914585::WARNING::2013-11-20 03:35:02,593::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914585::DEBUG::2013-11-20 03:35:02,593::task::495::TaskManager.Task::(_debug) Task 183b0390-7fdd-4510-bf6a-d232599ed96d: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914585::DEBUG::2013-11-20 03:35:02,594::task::495::TaskManager.Task::(_debug) Task 183b0390-7fdd-4510-bf6a-d232599ed96d: moving from state preparing -> state finished
>Thread-15914585::DEBUG::2013-11-20 03:35:02,594::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914585::DEBUG::2013-11-20 03:35:02,595::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914585::DEBUG::2013-11-20 03:35:02,595::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914585::DEBUG::2013-11-20 03:35:02,596::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914585::DEBUG::2013-11-20 03:35:02,596::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914585::DEBUG::2013-11-20 03:35:02,596::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914585::DEBUG::2013-11-20 03:35:02,597::task::495::TaskManager.Task::(_debug) Task 183b0390-7fdd-4510-bf6a-d232599ed96d: ref 0 aborting False
>Thread-15914585::INFO::2013-11-20 03:35:02,597::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:35:02,975::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:35:03,383::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0506343 s, 20.2 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 03:35:05,223::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:35:05,224::task::495::TaskManager.Task::(_debug) Task 82c01ac7-9e5c-4e10-b620-8a324b111ddb: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:35:05,225::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`99937e20-1c79-4ed2-b124-1a206faaebc0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:35:05,225::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:35:05,226::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:35:05,226::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`99937e20-1c79-4ed2-b124-1a206faaebc0`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:35:05,227::task::495::TaskManager.Task::(_debug) Task 82c01ac7-9e5c-4e10-b620-8a324b111ddb: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:35:05,227::task::495::TaskManager.Task::(_debug) Task 82c01ac7-9e5c-4e10-b620-8a324b111ddb: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:35:05,228::task::495::TaskManager.Task::(_debug) Task 82c01ac7-9e5c-4e10-b620-8a324b111ddb: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:35:05,229::task::495::TaskManager.Task::(_debug) Task 82c01ac7-9e5c-4e10-b620-8a324b111ddb: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:35:05,229::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:35:05,230::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:35:05,231::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:35:05,231::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:35:05,232::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:35:05,232::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:35:05,232::task::495::TaskManager.Task::(_debug) Task 82c01ac7-9e5c-4e10-b620-8a324b111ddb: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:35:05,233::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:35:05,393::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:35:05,821::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0549609 s, 18.6 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 03:35:07,340::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:35:07,340::task::495::TaskManager.Task::(_debug) Task c8b4202f-52ba-4dfe-a3ef-73f38075595d: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:35:07,341::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0b2e28c0-48b3-4d7a-a6ea-115120671525`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:35:07,342::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:35:07,342::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:35:07,343::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0b2e28c0-48b3-4d7a-a6ea-115120671525`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:35:07,343::task::495::TaskManager.Task::(_debug) Task c8b4202f-52ba-4dfe-a3ef-73f38075595d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:35:07,344::task::495::TaskManager.Task::(_debug) Task c8b4202f-52ba-4dfe-a3ef-73f38075595d: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:35:07,345::task::495::TaskManager.Task::(_debug) Task c8b4202f-52ba-4dfe-a3ef-73f38075595d: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:35:07,345::task::495::TaskManager.Task::(_debug) Task c8b4202f-52ba-4dfe-a3ef-73f38075595d: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:35:07,345::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:35:07,346::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:35:07,346::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:35:07,347::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:35:07,347::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:35:07,347::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:35:07,348::task::495::TaskManager.Task::(_debug) Task c8b4202f-52ba-4dfe-a3ef-73f38075595d: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:35:07,348::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:35:07,832::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:35:08,275::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0530487 s, 19.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:35:10,289::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:35:10,679::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0532089 s, 19.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:35:12,690::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914589::DEBUG::2013-11-20 03:35:13,119::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914589::INFO::2013-11-20 03:35:13,121::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914589::DEBUG::2013-11-20 03:35:13,121::task::495::TaskManager.Task::(_debug) Task 8277621f-e694-4035-8275-8824e4be9bb7: moving from state init -> state preparing
>Thread-15914589::DEBUG::2013-11-20 03:35:13,122::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914589::DEBUG::2013-11-20 03:35:13,123::task::495::TaskManager.Task::(_debug) Task 8277621f-e694-4035-8275-8824e4be9bb7: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914589::DEBUG::2013-11-20 03:35:13,123::task::495::TaskManager.Task::(_debug) Task 8277621f-e694-4035-8275-8824e4be9bb7: moving from state preparing -> state finished
>Thread-15914589::DEBUG::2013-11-20 03:35:13,124::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914589::DEBUG::2013-11-20 03:35:13,124::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914589::DEBUG::2013-11-20 03:35:13,125::task::495::TaskManager.Task::(_debug) Task 8277621f-e694-4035-8275-8824e4be9bb7: ref 0 aborting False
>Thread-15914589::INFO::2013-11-20 03:35:13,125::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914590::DEBUG::2013-11-20 03:35:13,134::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914590::INFO::2013-11-20 03:35:13,135::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914590::DEBUG::2013-11-20 03:35:13,135::task::495::TaskManager.Task::(_debug) Task 9c5b23b1-b83b-4ea6-b4ed-cd5a7bb180a6: moving from state init -> state preparing
>Thread-15914590::DEBUG::2013-11-20 03:35:13,136::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1e0276d1-ab40-4363-8a57-a433010e6497`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914590::DEBUG::2013-11-20 03:35:13,137::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914590::DEBUG::2013-11-20 03:35:13,137::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914590::DEBUG::2013-11-20 03:35:13,138::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1e0276d1-ab40-4363-8a57-a433010e6497`::Granted request
>Thread-15914590::DEBUG::2013-11-20 03:35:13,141::task::495::TaskManager.Task::(_debug) Task 9c5b23b1-b83b-4ea6-b4ed-cd5a7bb180a6: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914590::DEBUG::2013-11-20 03:35:13,147::task::495::TaskManager.Task::(_debug) Task 9c5b23b1-b83b-4ea6-b4ed-cd5a7bb180a6: ref 1 aborting False
>Thread-15914590::WARNING::2013-11-20 03:35:13,149::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914590::DEBUG::2013-11-20 03:35:13,153::task::495::TaskManager.Task::(_debug) Task 9c5b23b1-b83b-4ea6-b4ed-cd5a7bb180a6: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914590::DEBUG::2013-11-20 03:35:13,154::task::495::TaskManager.Task::(_debug) Task 9c5b23b1-b83b-4ea6-b4ed-cd5a7bb180a6: moving from state preparing -> state finished
>Thread-15914590::DEBUG::2013-11-20 03:35:13,155::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914590::DEBUG::2013-11-20 03:35:13,155::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914590::DEBUG::2013-11-20 03:35:13,155::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914590::DEBUG::2013-11-20 03:35:13,156::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914590::DEBUG::2013-11-20 03:35:13,156::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914590::DEBUG::2013-11-20 03:35:13,157::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914590::DEBUG::2013-11-20 03:35:13,157::task::495::TaskManager.Task::(_debug) Task 9c5b23b1-b83b-4ea6-b4ed-cd5a7bb180a6: ref 0 aborting False
>Thread-15914590::INFO::2013-11-20 03:35:13,158::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:35:13,585::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0549145 s, 18.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:35:15,596::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:35:15,984::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0480675 s, 21.3 MB/s\n'; <rc> = 0
>Thread-15914592::INFO::2013-11-20 03:35:17,099::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914592::DEBUG::2013-11-20 03:35:17,100::task::495::TaskManager.Task::(_debug) Task 7423b35d-3370-4cbc-a9b3-035ef0dd4f99: moving from state init -> state preparing
>Thread-15914592::DEBUG::2013-11-20 03:35:17,101::task::495::TaskManager.Task::(_debug) Task 7423b35d-3370-4cbc-a9b3-035ef0dd4f99: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000241041183472', 'lastCheck': 1384918507.6680329, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00144720077515', 'lastCheck': 1384918513.695401, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00119590759277', 'lastCheck': 1384918507.214628, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000439882278442', 'lastCheck': 1384918513.0087731, 'valid': True, 'code': 0}}
>Thread-15914592::DEBUG::2013-11-20 03:35:17,102::task::495::TaskManager.Task::(_debug) Task 7423b35d-3370-4cbc-a9b3-035ef0dd4f99: moving from state preparing -> state finished
>Thread-15914592::DEBUG::2013-11-20 03:35:17,102::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914592::DEBUG::2013-11-20 03:35:17,103::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914592::DEBUG::2013-11-20 03:35:17,103::task::495::TaskManager.Task::(_debug) Task 7423b35d-3370-4cbc-a9b3-035ef0dd4f99: ref 0 aborting False
>Thread-15914592::INFO::2013-11-20 03:35:17,104::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000241041183472', 'lastCheck': 1384918507.6680329, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00144720077515', 'lastCheck': 1384918513.695401, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00119590759277', 'lastCheck': 1384918507.214628, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000439882278442', 'lastCheck': 1384918513.0087731, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:35:17,994::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 03:35:18,216::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 03:35:18,217::task::495::TaskManager.Task::(_debug) Task ac155344-461c-4b36-ad5b-026a4d9c8a65: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 03:35:18,220::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a2bf5066-4211-4d9d-a624-6f3c2e855a27`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 03:35:18,221::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 03:35:18,222::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 03:35:18,222::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a2bf5066-4211-4d9d-a624-6f3c2e855a27`::Granted request
>Thread-15914435::DEBUG::2013-11-20 03:35:18,224::task::495::TaskManager.Task::(_debug) Task ac155344-461c-4b36-ad5b-026a4d9c8a65: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 03:35:18,224::task::495::TaskManager.Task::(_debug) Task ac155344-461c-4b36-ad5b-026a4d9c8a65: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 03:35:18,225::task::495::TaskManager.Task::(_debug) Task ac155344-461c-4b36-ad5b-026a4d9c8a65: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 03:35:18,226::task::495::TaskManager.Task::(_debug) Task ac155344-461c-4b36-ad5b-026a4d9c8a65: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 03:35:18,226::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 03:35:18,227::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 03:35:18,227::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 03:35:18,228::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 03:35:18,228::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 03:35:18,229::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 03:35:18,229::task::495::TaskManager.Task::(_debug) Task ac155344-461c-4b36-ad5b-026a4d9c8a65: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 03:35:18,230::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 03:35:18,407::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0522994 s, 19.6 MB/s\n'; <rc> = 0
>Thread-15914453::INFO::2013-11-20 03:35:20,163::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 03:35:20,165::task::495::TaskManager.Task::(_debug) Task bb653145-2c2d-4991-ad80-7f336ad46308: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 03:35:20,166::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5f6634a7-9dc6-444a-b080-7709279e876e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 03:35:20,167::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 03:35:20,167::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 03:35:20,168::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5f6634a7-9dc6-444a-b080-7709279e876e`::Granted request
>Thread-15914453::DEBUG::2013-11-20 03:35:20,168::task::495::TaskManager.Task::(_debug) Task bb653145-2c2d-4991-ad80-7f336ad46308: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 03:35:20,169::task::495::TaskManager.Task::(_debug) Task bb653145-2c2d-4991-ad80-7f336ad46308: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 03:35:20,169::task::495::TaskManager.Task::(_debug) Task bb653145-2c2d-4991-ad80-7f336ad46308: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 03:35:20,170::task::495::TaskManager.Task::(_debug) Task bb653145-2c2d-4991-ad80-7f336ad46308: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 03:35:20,170::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 03:35:20,171::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 03:35:20,171::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 03:35:20,172::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 03:35:20,172::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 03:35:20,172::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 03:35:20,173::task::495::TaskManager.Task::(_debug) Task bb653145-2c2d-4991-ad80-7f336ad46308: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 03:35:20,173::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 03:35:20,417::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:35:20,824::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0515338 s, 19.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:35:22,837::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 03:35:23,305::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 03:35:23,306::task::495::TaskManager.Task::(_debug) Task 73a62ad0-6b01-4642-ba00-0f278eadc612: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 03:35:23,315::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`88ad181f-5643-49ab-b1d8-1dbe11a72f20`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 03:35:23,316::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 03:35:23,317::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 03:35:23,318::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`88ad181f-5643-49ab-b1d8-1dbe11a72f20`::Granted request
>Thread-15914519::DEBUG::2013-11-20 03:35:23,320::task::495::TaskManager.Task::(_debug) Task 73a62ad0-6b01-4642-ba00-0f278eadc612: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 03:35:23,320::task::495::TaskManager.Task::(_debug) Task 73a62ad0-6b01-4642-ba00-0f278eadc612: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 03:35:23,321::task::495::TaskManager.Task::(_debug) Task 73a62ad0-6b01-4642-ba00-0f278eadc612: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 03:35:23,322::task::495::TaskManager.Task::(_debug) Task 73a62ad0-6b01-4642-ba00-0f278eadc612: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 03:35:23,322::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 03:35:23,323::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 03:35:23,324::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 03:35:23,325::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 03:35:23,325::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 03:35:23,326::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 03:35:23,327::task::495::TaskManager.Task::(_debug) Task 73a62ad0-6b01-4642-ba00-0f278eadc612: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 03:35:23,327::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914595::DEBUG::2013-11-20 03:35:23,482::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914595::INFO::2013-11-20 03:35:23,483::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914595::DEBUG::2013-11-20 03:35:23,483::task::495::TaskManager.Task::(_debug) Task c37c5bf0-1af0-43e1-a0fb-4e27a32124e2: moving from state init -> state preparing
>Thread-15914595::DEBUG::2013-11-20 03:35:23,484::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914595::DEBUG::2013-11-20 03:35:23,484::task::495::TaskManager.Task::(_debug) Task c37c5bf0-1af0-43e1-a0fb-4e27a32124e2: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914595::DEBUG::2013-11-20 03:35:23,485::task::495::TaskManager.Task::(_debug) Task c37c5bf0-1af0-43e1-a0fb-4e27a32124e2: moving from state preparing -> state finished
>Thread-15914595::DEBUG::2013-11-20 03:35:23,485::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914595::DEBUG::2013-11-20 03:35:23,486::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914595::DEBUG::2013-11-20 03:35:23,486::task::495::TaskManager.Task::(_debug) Task c37c5bf0-1af0-43e1-a0fb-4e27a32124e2: ref 0 aborting False
>Thread-15914595::INFO::2013-11-20 03:35:23,486::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914596::DEBUG::2013-11-20 03:35:23,500::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914596::INFO::2013-11-20 03:35:23,500::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914596::DEBUG::2013-11-20 03:35:23,501::task::495::TaskManager.Task::(_debug) Task 6c8b94b4-584e-47fe-b9dd-167e9deccec3: moving from state init -> state preparing
>Thread-15914596::DEBUG::2013-11-20 03:35:23,502::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`df8783f1-43c2-4542-ab49-cedcb652e6a7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914596::DEBUG::2013-11-20 03:35:23,502::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914596::DEBUG::2013-11-20 03:35:23,503::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914596::DEBUG::2013-11-20 03:35:23,503::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`df8783f1-43c2-4542-ab49-cedcb652e6a7`::Granted request
>Thread-15914596::DEBUG::2013-11-20 03:35:23,504::task::495::TaskManager.Task::(_debug) Task 6c8b94b4-584e-47fe-b9dd-167e9deccec3: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914596::DEBUG::2013-11-20 03:35:23,504::task::495::TaskManager.Task::(_debug) Task 6c8b94b4-584e-47fe-b9dd-167e9deccec3: ref 1 aborting False
>Thread-15914596::WARNING::2013-11-20 03:35:23,506::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914596::DEBUG::2013-11-20 03:35:23,507::task::495::TaskManager.Task::(_debug) Task 6c8b94b4-584e-47fe-b9dd-167e9deccec3: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914596::DEBUG::2013-11-20 03:35:23,508::task::495::TaskManager.Task::(_debug) Task 6c8b94b4-584e-47fe-b9dd-167e9deccec3: moving from state preparing -> state finished
>Thread-15914596::DEBUG::2013-11-20 03:35:23,508::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914596::DEBUG::2013-11-20 03:35:23,509::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914596::DEBUG::2013-11-20 03:35:23,509::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914596::DEBUG::2013-11-20 03:35:23,510::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914596::DEBUG::2013-11-20 03:35:23,510::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914596::DEBUG::2013-11-20 03:35:23,510::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914596::DEBUG::2013-11-20 03:35:23,511::task::495::TaskManager.Task::(_debug) Task 6c8b94b4-584e-47fe-b9dd-167e9deccec3: ref 0 aborting False
>Thread-15914596::INFO::2013-11-20 03:35:23,511::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:35:25,999::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0535362 s, 19.1 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 03:35:27,865::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:35:27,866::task::495::TaskManager.Task::(_debug) Task e49658d7-8359-4817-96d1-73ffe4aaef65: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:35:27,867::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`214b908b-4458-441a-bbe9-d2be097e4892`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:35:27,867::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:35:27,868::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:35:27,868::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`214b908b-4458-441a-bbe9-d2be097e4892`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:35:27,869::task::495::TaskManager.Task::(_debug) Task e49658d7-8359-4817-96d1-73ffe4aaef65: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:35:27,869::task::495::TaskManager.Task::(_debug) Task e49658d7-8359-4817-96d1-73ffe4aaef65: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:35:27,869::task::495::TaskManager.Task::(_debug) Task e49658d7-8359-4817-96d1-73ffe4aaef65: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:35:27,870::task::495::TaskManager.Task::(_debug) Task e49658d7-8359-4817-96d1-73ffe4aaef65: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:35:27,870::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:35:27,870::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:35:27,871::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:35:27,871::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:35:27,872::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:35:27,872::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:35:27,872::task::495::TaskManager.Task::(_debug) Task e49658d7-8359-4817-96d1-73ffe4aaef65: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:35:27,873::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:35:27,873::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:35:27,873::task::495::TaskManager.Task::(_debug) Task b11ae253-1f5c-44f5-bff3-6b540febafbe: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:35:27,874::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fa5bb5eb-9fad-4dd3-9801-d7b2833b21b2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:35:27,875::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:35:27,875::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:35:27,875::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fa5bb5eb-9fad-4dd3-9801-d7b2833b21b2`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:35:27,876::task::495::TaskManager.Task::(_debug) Task b11ae253-1f5c-44f5-bff3-6b540febafbe: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:35:27,876::task::495::TaskManager.Task::(_debug) Task b11ae253-1f5c-44f5-bff3-6b540febafbe: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:35:27,877::task::495::TaskManager.Task::(_debug) Task b11ae253-1f5c-44f5-bff3-6b540febafbe: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:35:27,877::task::495::TaskManager.Task::(_debug) Task b11ae253-1f5c-44f5-bff3-6b540febafbe: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:35:27,877::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:35:27,878::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:35:27,878::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:35:27,879::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:35:27,879::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:35:27,879::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:35:27,880::task::495::TaskManager.Task::(_debug) Task b11ae253-1f5c-44f5-bff3-6b540febafbe: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:35:27,880::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:35:28,018::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:35:28,520::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0564533 s, 18.1 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 03:35:30,194::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:35:30,205::task::495::TaskManager.Task::(_debug) Task 73cbbfe6-e056-4125-a248-0209d53277b7: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:35:30,207::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d80932e2-1ac5-435f-8ae0-d796112283f3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:35:30,208::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:35:30,209::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:35:30,209::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d80932e2-1ac5-435f-8ae0-d796112283f3`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:35:30,210::task::495::TaskManager.Task::(_debug) Task 73cbbfe6-e056-4125-a248-0209d53277b7: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:35:30,210::task::495::TaskManager.Task::(_debug) Task 73cbbfe6-e056-4125-a248-0209d53277b7: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:35:30,211::task::495::TaskManager.Task::(_debug) Task 73cbbfe6-e056-4125-a248-0209d53277b7: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:35:30,211::task::495::TaskManager.Task::(_debug) Task 73cbbfe6-e056-4125-a248-0209d53277b7: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:35:30,212::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:35:30,212::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:35:30,213::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:35:30,213::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:35:30,214::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:35:30,214::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:35:30,214::task::495::TaskManager.Task::(_debug) Task 73cbbfe6-e056-4125-a248-0209d53277b7: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:35:30,215::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:35:30,215::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:35:30,216::task::495::TaskManager.Task::(_debug) Task f3ed7ca4-8270-42a3-a06b-09119f29a87e: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:35:30,217::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`665c40a2-0025-49de-9453-4e2011b0ab37`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:35:30,217::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:35:30,218::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:35:30,218::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`665c40a2-0025-49de-9453-4e2011b0ab37`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:35:30,219::task::495::TaskManager.Task::(_debug) Task f3ed7ca4-8270-42a3-a06b-09119f29a87e: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:35:30,219::task::495::TaskManager.Task::(_debug) Task f3ed7ca4-8270-42a3-a06b-09119f29a87e: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:35:30,219::task::495::TaskManager.Task::(_debug) Task f3ed7ca4-8270-42a3-a06b-09119f29a87e: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:35:30,220::task::495::TaskManager.Task::(_debug) Task f3ed7ca4-8270-42a3-a06b-09119f29a87e: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:35:30,220::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:35:30,221::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:35:30,221::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:35:30,222::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:35:30,222::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:35:30,222::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:35:30,223::task::495::TaskManager.Task::(_debug) Task f3ed7ca4-8270-42a3-a06b-09119f29a87e: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:35:30,223::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:35:30,224::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:35:30,224::task::495::TaskManager.Task::(_debug) Task 6d7b8049-260a-4ce8-b170-bb5625988f75: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:35:30,225::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b6b6e807-8124-48c5-b88c-62fc067a2b29`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:35:30,225::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:35:30,226::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:35:30,226::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b6b6e807-8124-48c5-b88c-62fc067a2b29`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:35:30,227::task::495::TaskManager.Task::(_debug) Task 6d7b8049-260a-4ce8-b170-bb5625988f75: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:35:30,227::task::495::TaskManager.Task::(_debug) Task 6d7b8049-260a-4ce8-b170-bb5625988f75: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:35:30,228::task::495::TaskManager.Task::(_debug) Task 6d7b8049-260a-4ce8-b170-bb5625988f75: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:35:30,228::task::495::TaskManager.Task::(_debug) Task 6d7b8049-260a-4ce8-b170-bb5625988f75: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:35:30,228::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:35:30,229::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:35:30,229::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:35:30,230::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:35:30,230::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:35:30,230::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:35:30,231::task::495::TaskManager.Task::(_debug) Task 6d7b8049-260a-4ce8-b170-bb5625988f75: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:35:30,231::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:35:30,232::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:35:30,232::task::495::TaskManager.Task::(_debug) Task fafe2867-e71e-42b2-9e9f-ff37a18cd54f: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:35:30,233::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`663fd657-9b6b-4546-a9f5-cfaa37e43368`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:35:30,233::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:35:30,234::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:35:30,234::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`663fd657-9b6b-4546-a9f5-cfaa37e43368`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:35:30,235::task::495::TaskManager.Task::(_debug) Task fafe2867-e71e-42b2-9e9f-ff37a18cd54f: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:35:30,235::task::495::TaskManager.Task::(_debug) Task fafe2867-e71e-42b2-9e9f-ff37a18cd54f: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:35:30,236::task::495::TaskManager.Task::(_debug) Task fafe2867-e71e-42b2-9e9f-ff37a18cd54f: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:35:30,236::task::495::TaskManager.Task::(_debug) Task fafe2867-e71e-42b2-9e9f-ff37a18cd54f: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:35:30,237::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:35:30,237::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:35:30,238::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:35:30,238::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:35:30,238::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:35:30,239::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:35:30,239::task::495::TaskManager.Task::(_debug) Task fafe2867-e71e-42b2-9e9f-ff37a18cd54f: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:35:30,240::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:35:30,530::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:35:30,912::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0478249 s, 21.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:35:32,923::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914600::INFO::2013-11-20 03:35:32,926::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914600::DEBUG::2013-11-20 03:35:33,077::task::495::TaskManager.Task::(_debug) Task ce73e285-1146-480c-9a73-e5fee1a339d0: moving from state init -> state preparing
>Thread-15914600::DEBUG::2013-11-20 03:35:33,079::task::495::TaskManager.Task::(_debug) Task ce73e285-1146-480c-9a73-e5fee1a339d0: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000240087509155', 'lastCheck': 1384918527.773176, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0011260509491', 'lastCheck': 1384918523.7671471, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000925779342651', 'lastCheck': 1384918527.238373, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00043797492981', 'lastCheck': 1384918523.1650529, 'valid': True, 'code': 0}}
>Thread-15914600::DEBUG::2013-11-20 03:35:33,080::task::495::TaskManager.Task::(_debug) Task ce73e285-1146-480c-9a73-e5fee1a339d0: moving from state preparing -> state finished
>Thread-15914600::DEBUG::2013-11-20 03:35:33,080::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914600::DEBUG::2013-11-20 03:35:33,080::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914600::DEBUG::2013-11-20 03:35:33,081::task::495::TaskManager.Task::(_debug) Task ce73e285-1146-480c-9a73-e5fee1a339d0: ref 0 aborting False
>Thread-15914600::INFO::2013-11-20 03:35:33,082::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000240087509155', 'lastCheck': 1384918527.773176, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0011260509491', 'lastCheck': 1384918523.7671471, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000925779342651', 'lastCheck': 1384918527.238373, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00043797492981', 'lastCheck': 1384918523.1650529, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:35:33,325::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0530863 s, 19.3 MB/s\n'; <rc> = 0
>Thread-15914602::DEBUG::2013-11-20 03:35:33,929::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914602::INFO::2013-11-20 03:35:33,930::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914602::DEBUG::2013-11-20 03:35:33,930::task::495::TaskManager.Task::(_debug) Task edd2d863-9576-4f80-84d4-e7b5d37a8eef: moving from state init -> state preparing
>Thread-15914602::DEBUG::2013-11-20 03:35:33,931::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914602::DEBUG::2013-11-20 03:35:33,931::task::495::TaskManager.Task::(_debug) Task edd2d863-9576-4f80-84d4-e7b5d37a8eef: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914602::DEBUG::2013-11-20 03:35:33,932::task::495::TaskManager.Task::(_debug) Task edd2d863-9576-4f80-84d4-e7b5d37a8eef: moving from state preparing -> state finished
>Thread-15914602::DEBUG::2013-11-20 03:35:33,932::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914602::DEBUG::2013-11-20 03:35:33,933::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914602::DEBUG::2013-11-20 03:35:33,933::task::495::TaskManager.Task::(_debug) Task edd2d863-9576-4f80-84d4-e7b5d37a8eef: ref 0 aborting False
>Thread-15914602::INFO::2013-11-20 03:35:33,933::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914603::DEBUG::2013-11-20 03:35:33,938::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914603::INFO::2013-11-20 03:35:33,938::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914603::DEBUG::2013-11-20 03:35:33,939::task::495::TaskManager.Task::(_debug) Task d1c8380f-9fad-42dd-ab94-d8faf3c54d16: moving from state init -> state preparing
>Thread-15914603::DEBUG::2013-11-20 03:35:33,940::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2e5e0451-7e16-49aa-ac4a-d7964d8f1c38`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914603::DEBUG::2013-11-20 03:35:33,940::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914603::DEBUG::2013-11-20 03:35:33,941::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914603::DEBUG::2013-11-20 03:35:33,941::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2e5e0451-7e16-49aa-ac4a-d7964d8f1c38`::Granted request
>Thread-15914603::DEBUG::2013-11-20 03:35:33,944::task::495::TaskManager.Task::(_debug) Task d1c8380f-9fad-42dd-ab94-d8faf3c54d16: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914603::DEBUG::2013-11-20 03:35:33,950::task::495::TaskManager.Task::(_debug) Task d1c8380f-9fad-42dd-ab94-d8faf3c54d16: ref 1 aborting False
>Thread-15914603::WARNING::2013-11-20 03:35:33,951::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914603::DEBUG::2013-11-20 03:35:33,952::task::495::TaskManager.Task::(_debug) Task d1c8380f-9fad-42dd-ab94-d8faf3c54d16: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914603::DEBUG::2013-11-20 03:35:33,952::task::495::TaskManager.Task::(_debug) Task d1c8380f-9fad-42dd-ab94-d8faf3c54d16: moving from state preparing -> state finished
>Thread-15914603::DEBUG::2013-11-20 03:35:33,952::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914603::DEBUG::2013-11-20 03:35:33,953::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914603::DEBUG::2013-11-20 03:35:33,953::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914603::DEBUG::2013-11-20 03:35:33,954::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914603::DEBUG::2013-11-20 03:35:33,954::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914603::DEBUG::2013-11-20 03:35:33,954::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914603::DEBUG::2013-11-20 03:35:33,955::task::495::TaskManager.Task::(_debug) Task d1c8380f-9fad-42dd-ab94-d8faf3c54d16: ref 0 aborting False
>Thread-15914603::INFO::2013-11-20 03:35:33,955::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:35:35,335::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:35:36,803::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0572287 s, 17.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:35:38,813::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:35:39,211::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0546583 s, 18.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:35:41,221::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:35:42,098::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0442831 s, 23.1 MB/s\n'; <rc> = 0
>Thread-15914436::DEBUG::2013-11-20 03:35:43,388::guestIF::97::vm.Vm::(_handleMessage) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::u'BRD-KLEE\\administrator'
>Dummy-15912651::DEBUG::2013-11-20 03:35:44,108::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914607::DEBUG::2013-11-20 03:35:44,323::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914607::INFO::2013-11-20 03:35:44,326::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914607::DEBUG::2013-11-20 03:35:44,326::task::495::TaskManager.Task::(_debug) Task 43bfd8e3-e341-4cd8-9462-cb569b10840e: moving from state init -> state preparing
>Thread-15914607::DEBUG::2013-11-20 03:35:44,327::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914607::DEBUG::2013-11-20 03:35:44,327::task::495::TaskManager.Task::(_debug) Task 43bfd8e3-e341-4cd8-9462-cb569b10840e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914607::DEBUG::2013-11-20 03:35:44,328::task::495::TaskManager.Task::(_debug) Task 43bfd8e3-e341-4cd8-9462-cb569b10840e: moving from state preparing -> state finished
>Thread-15914607::DEBUG::2013-11-20 03:35:44,329::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914607::DEBUG::2013-11-20 03:35:44,329::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914607::DEBUG::2013-11-20 03:35:44,330::task::495::TaskManager.Task::(_debug) Task 43bfd8e3-e341-4cd8-9462-cb569b10840e: ref 0 aborting False
>Thread-15914607::INFO::2013-11-20 03:35:44,330::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914608::DEBUG::2013-11-20 03:35:44,343::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914608::INFO::2013-11-20 03:35:44,344::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914608::DEBUG::2013-11-20 03:35:44,345::task::495::TaskManager.Task::(_debug) Task d28e61bf-104f-4820-8826-1563803d48fc: moving from state init -> state preparing
>Thread-15914608::DEBUG::2013-11-20 03:35:44,351::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2be672fa-539c-485d-8913-6900833bb4c1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914608::DEBUG::2013-11-20 03:35:44,353::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914608::DEBUG::2013-11-20 03:35:44,353::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914608::DEBUG::2013-11-20 03:35:44,354::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2be672fa-539c-485d-8913-6900833bb4c1`::Granted request
>Thread-15914608::DEBUG::2013-11-20 03:35:44,355::task::495::TaskManager.Task::(_debug) Task d28e61bf-104f-4820-8826-1563803d48fc: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914608::DEBUG::2013-11-20 03:35:44,355::task::495::TaskManager.Task::(_debug) Task d28e61bf-104f-4820-8826-1563803d48fc: ref 1 aborting False
>Thread-15914608::WARNING::2013-11-20 03:35:44,357::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914608::DEBUG::2013-11-20 03:35:44,357::task::495::TaskManager.Task::(_debug) Task d28e61bf-104f-4820-8826-1563803d48fc: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914608::DEBUG::2013-11-20 03:35:44,358::task::495::TaskManager.Task::(_debug) Task d28e61bf-104f-4820-8826-1563803d48fc: moving from state preparing -> state finished
>Thread-15914608::DEBUG::2013-11-20 03:35:44,358::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914608::DEBUG::2013-11-20 03:35:44,359::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914608::DEBUG::2013-11-20 03:35:44,362::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914608::DEBUG::2013-11-20 03:35:44,363::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914608::DEBUG::2013-11-20 03:35:44,363::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914608::DEBUG::2013-11-20 03:35:44,364::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914608::DEBUG::2013-11-20 03:35:44,364::task::495::TaskManager.Task::(_debug) Task d28e61bf-104f-4820-8826-1563803d48fc: ref 0 aborting False
>Thread-15914608::INFO::2013-11-20 03:35:44,365::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:35:44,493::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0447846 s, 22.9 MB/s\n'; <rc> = 0
>Thread-15913639::INFO::2013-11-20 03:35:45,438::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:35:45,439::task::495::TaskManager.Task::(_debug) Task 24e628fd-3f42-4aa6-b751-515eed177c49: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:35:45,441::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0c30eb12-2242-467b-bb96-74d263beb10b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:35:45,441::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:35:45,442::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:35:45,442::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0c30eb12-2242-467b-bb96-74d263beb10b`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:35:45,443::task::495::TaskManager.Task::(_debug) Task 24e628fd-3f42-4aa6-b751-515eed177c49: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:35:45,444::task::495::TaskManager.Task::(_debug) Task 24e628fd-3f42-4aa6-b751-515eed177c49: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:35:45,444::task::495::TaskManager.Task::(_debug) Task 24e628fd-3f42-4aa6-b751-515eed177c49: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:35:45,445::task::495::TaskManager.Task::(_debug) Task 24e628fd-3f42-4aa6-b751-515eed177c49: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:35:45,445::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:35:45,446::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:35:45,446::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:35:45,447::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:35:45,447::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:35:45,448::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:35:45,448::task::495::TaskManager.Task::(_debug) Task 24e628fd-3f42-4aa6-b751-515eed177c49: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:35:45,449::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:35:46,503::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:35:46,920::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0784612 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:35:48,930::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914610::INFO::2013-11-20 03:35:49,097::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914610::DEBUG::2013-11-20 03:35:49,098::task::495::TaskManager.Task::(_debug) Task ed1b5161-b520-4947-afc8-9c8844862c42: moving from state init -> state preparing
>Thread-15914610::DEBUG::2013-11-20 03:35:49,099::task::495::TaskManager.Task::(_debug) Task ed1b5161-b520-4947-afc8-9c8844862c42: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000236988067627', 'lastCheck': 1384918547.7964621, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0397200584412', 'lastCheck': 1384918543.8350899, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000925064086914', 'lastCheck': 1384918547.4431219, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00066089630127', 'lastCheck': 1384918543.2095771, 'valid': True, 'code': 0}}
>Thread-15914610::DEBUG::2013-11-20 03:35:49,099::task::495::TaskManager.Task::(_debug) Task ed1b5161-b520-4947-afc8-9c8844862c42: moving from state preparing -> state finished
>Thread-15914610::DEBUG::2013-11-20 03:35:49,100::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914610::DEBUG::2013-11-20 03:35:49,100::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914610::DEBUG::2013-11-20 03:35:49,101::task::495::TaskManager.Task::(_debug) Task ed1b5161-b520-4947-afc8-9c8844862c42: ref 0 aborting False
>Thread-15914610::INFO::2013-11-20 03:35:49,101::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000236988067627', 'lastCheck': 1384918547.7964621, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0397200584412', 'lastCheck': 1384918543.8350899, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000925064086914', 'lastCheck': 1384918547.4431219, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00066089630127', 'lastCheck': 1384918543.2095771, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:35:49,339::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0576739 s, 17.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:35:51,349::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:35:51,780::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0495278 s, 20.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:35:53,792::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:35:54,210::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.056467 s, 18.1 MB/s\n'; <rc> = 0
>Thread-15914613::DEBUG::2013-11-20 03:35:54,770::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914613::INFO::2013-11-20 03:35:54,771::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914613::DEBUG::2013-11-20 03:35:54,772::task::495::TaskManager.Task::(_debug) Task 1621fd4e-72ae-40a4-9d9a-033135e76a51: moving from state init -> state preparing
>Thread-15914613::DEBUG::2013-11-20 03:35:54,772::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914613::DEBUG::2013-11-20 03:35:54,773::task::495::TaskManager.Task::(_debug) Task 1621fd4e-72ae-40a4-9d9a-033135e76a51: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914613::DEBUG::2013-11-20 03:35:54,773::task::495::TaskManager.Task::(_debug) Task 1621fd4e-72ae-40a4-9d9a-033135e76a51: moving from state preparing -> state finished
>Thread-15914613::DEBUG::2013-11-20 03:35:54,774::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914613::DEBUG::2013-11-20 03:35:54,774::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914613::DEBUG::2013-11-20 03:35:54,774::task::495::TaskManager.Task::(_debug) Task 1621fd4e-72ae-40a4-9d9a-033135e76a51: ref 0 aborting False
>Thread-15914613::INFO::2013-11-20 03:35:54,775::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914614::DEBUG::2013-11-20 03:35:54,782::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914614::INFO::2013-11-20 03:35:54,783::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914614::DEBUG::2013-11-20 03:35:54,783::task::495::TaskManager.Task::(_debug) Task fb3c4cdb-7795-41e4-b983-a26afda4a4af: moving from state init -> state preparing
>Thread-15914614::DEBUG::2013-11-20 03:35:54,796::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a2d9710b-de7c-47b4-a17f-4f3ce37f8a1d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914614::DEBUG::2013-11-20 03:35:54,797::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914614::DEBUG::2013-11-20 03:35:54,797::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914614::DEBUG::2013-11-20 03:35:54,798::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a2d9710b-de7c-47b4-a17f-4f3ce37f8a1d`::Granted request
>Thread-15914614::DEBUG::2013-11-20 03:35:54,799::task::495::TaskManager.Task::(_debug) Task fb3c4cdb-7795-41e4-b983-a26afda4a4af: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914614::DEBUG::2013-11-20 03:35:54,799::task::495::TaskManager.Task::(_debug) Task fb3c4cdb-7795-41e4-b983-a26afda4a4af: ref 1 aborting False
>Thread-15914614::WARNING::2013-11-20 03:35:54,801::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914614::DEBUG::2013-11-20 03:35:54,801::task::495::TaskManager.Task::(_debug) Task fb3c4cdb-7795-41e4-b983-a26afda4a4af: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914614::DEBUG::2013-11-20 03:35:54,802::task::495::TaskManager.Task::(_debug) Task fb3c4cdb-7795-41e4-b983-a26afda4a4af: moving from state preparing -> state finished
>Thread-15914614::DEBUG::2013-11-20 03:35:54,803::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914614::DEBUG::2013-11-20 03:35:54,803::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914614::DEBUG::2013-11-20 03:35:54,804::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914614::DEBUG::2013-11-20 03:35:54,804::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914614::DEBUG::2013-11-20 03:35:54,805::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914614::DEBUG::2013-11-20 03:35:54,805::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914614::DEBUG::2013-11-20 03:35:54,806::task::495::TaskManager.Task::(_debug) Task fb3c4cdb-7795-41e4-b983-a26afda4a4af: ref 0 aborting False
>Thread-15914614::INFO::2013-11-20 03:35:54,806::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:35:56,221::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:35:56,639::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0590469 s, 17.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:35:58,647::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:35:59,068::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0587303 s, 17.4 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 03:36:00,390::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:36:00,401::task::495::TaskManager.Task::(_debug) Task dc361f7f-540e-45b5-bbab-3be91d7bdec9: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:36:00,403::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f67d8db9-0b81-48a9-9c30-11f8fdbb9f02`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:36:00,404::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:36:00,404::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:36:00,405::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f67d8db9-0b81-48a9-9c30-11f8fdbb9f02`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:36:00,406::task::495::TaskManager.Task::(_debug) Task dc361f7f-540e-45b5-bbab-3be91d7bdec9: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:36:00,406::task::495::TaskManager.Task::(_debug) Task dc361f7f-540e-45b5-bbab-3be91d7bdec9: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:36:00,407::task::495::TaskManager.Task::(_debug) Task dc361f7f-540e-45b5-bbab-3be91d7bdec9: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:36:00,407::task::495::TaskManager.Task::(_debug) Task dc361f7f-540e-45b5-bbab-3be91d7bdec9: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:36:00,408::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:36:00,408::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:36:00,409::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:36:00,410::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:36:00,410::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:36:00,411::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:36:00,411::task::495::TaskManager.Task::(_debug) Task dc361f7f-540e-45b5-bbab-3be91d7bdec9: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:36:00,412::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:36:01,081::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:36:02,481::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0567035 s, 18.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:36:04,494::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:36:04,921::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0705668 s, 14.5 MB/s\n'; <rc> = 0
>Thread-15914618::INFO::2013-11-20 03:36:05,266::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914618::DEBUG::2013-11-20 03:36:05,267::task::495::TaskManager.Task::(_debug) Task f6db3d7a-350d-467f-9738-d3893cb51b25: moving from state init -> state preparing
>Thread-15914618::DEBUG::2013-11-20 03:36:05,267::task::495::TaskManager.Task::(_debug) Task f6db3d7a-350d-467f-9738-d3893cb51b25: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000257968902588', 'lastCheck': 1384918557.9730289, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.000876188278198', 'lastCheck': 1384918563.9844229, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000818014144897', 'lastCheck': 1384918557.533258, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000280141830444', 'lastCheck': 1384918563.3964851, 'valid': True, 'code': 0}}
>Thread-15914618::DEBUG::2013-11-20 03:36:05,268::task::495::TaskManager.Task::(_debug) Task f6db3d7a-350d-467f-9738-d3893cb51b25: moving from state preparing -> state finished
>Thread-15914618::DEBUG::2013-11-20 03:36:05,269::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914619::DEBUG::2013-11-20 03:36:05,282::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914618::DEBUG::2013-11-20 03:36:05,282::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914619::INFO::2013-11-20 03:36:05,283::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914618::DEBUG::2013-11-20 03:36:05,283::task::495::TaskManager.Task::(_debug) Task f6db3d7a-350d-467f-9738-d3893cb51b25: ref 0 aborting False
>Thread-15914619::DEBUG::2013-11-20 03:36:05,284::task::495::TaskManager.Task::(_debug) Task 411eae88-7129-4bda-ac47-3e94d4163cbf: moving from state init -> state preparing
>Thread-15914618::INFO::2013-11-20 03:36:05,284::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000257968902588', 'lastCheck': 1384918557.9730289, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.000876188278198', 'lastCheck': 1384918563.9844229, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000818014144897', 'lastCheck': 1384918557.533258, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000280141830444', 'lastCheck': 1384918563.3964851, 'valid': True, 'code': 0}}
>Thread-15914619::DEBUG::2013-11-20 03:36:05,285::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914619::DEBUG::2013-11-20 03:36:05,286::task::495::TaskManager.Task::(_debug) Task 411eae88-7129-4bda-ac47-3e94d4163cbf: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914619::DEBUG::2013-11-20 03:36:05,287::task::495::TaskManager.Task::(_debug) Task 411eae88-7129-4bda-ac47-3e94d4163cbf: moving from state preparing -> state finished
>Thread-15914619::DEBUG::2013-11-20 03:36:05,287::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914619::DEBUG::2013-11-20 03:36:05,288::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914619::DEBUG::2013-11-20 03:36:05,311::task::495::TaskManager.Task::(_debug) Task 411eae88-7129-4bda-ac47-3e94d4163cbf: ref 0 aborting False
>Thread-15914619::INFO::2013-11-20 03:36:05,312::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914620::DEBUG::2013-11-20 03:36:05,341::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914620::INFO::2013-11-20 03:36:05,342::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914620::DEBUG::2013-11-20 03:36:05,343::task::495::TaskManager.Task::(_debug) Task 56d2f663-6385-41cd-96d8-d05dbd45fcfc: moving from state init -> state preparing
>Thread-15914620::DEBUG::2013-11-20 03:36:05,346::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6d76d377-c499-4848-b4c4-514cd5b8c7ef`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914620::DEBUG::2013-11-20 03:36:05,347::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914620::DEBUG::2013-11-20 03:36:05,347::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914620::DEBUG::2013-11-20 03:36:05,348::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6d76d377-c499-4848-b4c4-514cd5b8c7ef`::Granted request
>Thread-15914620::DEBUG::2013-11-20 03:36:05,349::task::495::TaskManager.Task::(_debug) Task 56d2f663-6385-41cd-96d8-d05dbd45fcfc: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914620::DEBUG::2013-11-20 03:36:05,349::task::495::TaskManager.Task::(_debug) Task 56d2f663-6385-41cd-96d8-d05dbd45fcfc: ref 1 aborting False
>Thread-15914620::WARNING::2013-11-20 03:36:05,351::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914620::DEBUG::2013-11-20 03:36:05,352::task::495::TaskManager.Task::(_debug) Task 56d2f663-6385-41cd-96d8-d05dbd45fcfc: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914620::DEBUG::2013-11-20 03:36:05,353::task::495::TaskManager.Task::(_debug) Task 56d2f663-6385-41cd-96d8-d05dbd45fcfc: moving from state preparing -> state finished
>Thread-15914620::DEBUG::2013-11-20 03:36:05,354::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914620::DEBUG::2013-11-20 03:36:05,354::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914620::DEBUG::2013-11-20 03:36:05,355::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914620::DEBUG::2013-11-20 03:36:05,355::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914620::DEBUG::2013-11-20 03:36:05,355::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914620::DEBUG::2013-11-20 03:36:05,356::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914620::DEBUG::2013-11-20 03:36:05,356::task::495::TaskManager.Task::(_debug) Task 56d2f663-6385-41cd-96d8-d05dbd45fcfc: ref 0 aborting False
>Thread-15914620::INFO::2013-11-20 03:36:05,361::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912392::INFO::2013-11-20 03:36:05,533::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:36:05,534::task::495::TaskManager.Task::(_debug) Task 4691648a-e5cc-4e57-a6f8-10c0b84fba84: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:36:05,535::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d2593d85-0a0b-46dd-a1f4-d9b1ba3b2934`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:36:05,536::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:36:05,537::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:36:05,537::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d2593d85-0a0b-46dd-a1f4-d9b1ba3b2934`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:36:05,538::task::495::TaskManager.Task::(_debug) Task 4691648a-e5cc-4e57-a6f8-10c0b84fba84: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:36:05,538::task::495::TaskManager.Task::(_debug) Task 4691648a-e5cc-4e57-a6f8-10c0b84fba84: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:36:05,539::task::495::TaskManager.Task::(_debug) Task 4691648a-e5cc-4e57-a6f8-10c0b84fba84: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:36:05,539::task::495::TaskManager.Task::(_debug) Task 4691648a-e5cc-4e57-a6f8-10c0b84fba84: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:36:05,540::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:36:05,540::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:36:05,541::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:36:05,541::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:36:05,542::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:36:05,542::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:36:05,542::task::495::TaskManager.Task::(_debug) Task 4691648a-e5cc-4e57-a6f8-10c0b84fba84: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:36:05,543::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:36:06,932::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:36:07,356::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0766409 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 03:36:08,211::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:36:08,212::task::495::TaskManager.Task::(_debug) Task c203883f-869d-44ca-990b-ef08cbf41732: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:36:08,213::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e80bf180-8c10-4617-b31d-56dcaf7120b1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:36:08,214::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:36:08,214::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:36:08,214::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e80bf180-8c10-4617-b31d-56dcaf7120b1`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:36:08,215::task::495::TaskManager.Task::(_debug) Task c203883f-869d-44ca-990b-ef08cbf41732: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:36:08,216::task::495::TaskManager.Task::(_debug) Task c203883f-869d-44ca-990b-ef08cbf41732: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:36:08,216::task::495::TaskManager.Task::(_debug) Task c203883f-869d-44ca-990b-ef08cbf41732: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:36:08,216::task::495::TaskManager.Task::(_debug) Task c203883f-869d-44ca-990b-ef08cbf41732: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:36:08,217::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:36:08,217::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:36:08,218::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:36:08,218::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:36:08,218::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:36:08,219::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:36:08,219::task::495::TaskManager.Task::(_debug) Task c203883f-869d-44ca-990b-ef08cbf41732: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:36:08,220::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:36:09,367::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:36:09,807::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0529161 s, 19.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:36:11,819::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:36:12,259::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0541716 s, 18.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:36:14,269::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:36:14,689::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0492822 s, 20.8 MB/s\n'; <rc> = 0
>Thread-15914625::DEBUG::2013-11-20 03:36:15,910::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914625::INFO::2013-11-20 03:36:15,911::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914625::DEBUG::2013-11-20 03:36:15,912::task::495::TaskManager.Task::(_debug) Task 69b22a07-85ce-4220-8a75-0592fbf8dbd4: moving from state init -> state preparing
>Thread-15914625::DEBUG::2013-11-20 03:36:15,913::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914625::DEBUG::2013-11-20 03:36:15,913::task::495::TaskManager.Task::(_debug) Task 69b22a07-85ce-4220-8a75-0592fbf8dbd4: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914625::DEBUG::2013-11-20 03:36:15,914::task::495::TaskManager.Task::(_debug) Task 69b22a07-85ce-4220-8a75-0592fbf8dbd4: moving from state preparing -> state finished
>Thread-15914625::DEBUG::2013-11-20 03:36:15,914::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914625::DEBUG::2013-11-20 03:36:15,915::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914625::DEBUG::2013-11-20 03:36:15,915::task::495::TaskManager.Task::(_debug) Task 69b22a07-85ce-4220-8a75-0592fbf8dbd4: ref 0 aborting False
>Thread-15914625::INFO::2013-11-20 03:36:15,916::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914626::DEBUG::2013-11-20 03:36:15,925::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914626::INFO::2013-11-20 03:36:15,926::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914626::DEBUG::2013-11-20 03:36:15,926::task::495::TaskManager.Task::(_debug) Task 6bf08d54-6536-4e40-a2bf-47674d5d0522: moving from state init -> state preparing
>Thread-15914626::DEBUG::2013-11-20 03:36:15,927::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6e22e976-fa5e-4e23-a9d4-4953976433d4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914626::DEBUG::2013-11-20 03:36:15,928::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914626::DEBUG::2013-11-20 03:36:15,928::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914626::DEBUG::2013-11-20 03:36:15,929::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6e22e976-fa5e-4e23-a9d4-4953976433d4`::Granted request
>Thread-15914626::DEBUG::2013-11-20 03:36:15,929::task::495::TaskManager.Task::(_debug) Task 6bf08d54-6536-4e40-a2bf-47674d5d0522: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914626::DEBUG::2013-11-20 03:36:15,934::task::495::TaskManager.Task::(_debug) Task 6bf08d54-6536-4e40-a2bf-47674d5d0522: ref 1 aborting False
>Thread-15914626::WARNING::2013-11-20 03:36:15,935::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914626::DEBUG::2013-11-20 03:36:15,936::task::495::TaskManager.Task::(_debug) Task 6bf08d54-6536-4e40-a2bf-47674d5d0522: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914626::DEBUG::2013-11-20 03:36:15,937::task::495::TaskManager.Task::(_debug) Task 6bf08d54-6536-4e40-a2bf-47674d5d0522: moving from state preparing -> state finished
>Thread-15914626::DEBUG::2013-11-20 03:36:15,938::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914626::DEBUG::2013-11-20 03:36:15,938::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914626::DEBUG::2013-11-20 03:36:15,939::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914626::DEBUG::2013-11-20 03:36:15,939::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914626::DEBUG::2013-11-20 03:36:15,939::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914626::DEBUG::2013-11-20 03:36:15,940::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914626::DEBUG::2013-11-20 03:36:15,940::task::495::TaskManager.Task::(_debug) Task 6bf08d54-6536-4e40-a2bf-47674d5d0522: ref 0 aborting False
>Thread-15914626::INFO::2013-11-20 03:36:15,941::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:36:16,698::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:36:17,090::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0466652 s, 21.9 MB/s\n'; <rc> = 0
>Thread-15914435::INFO::2013-11-20 03:36:18,490::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 03:36:18,491::task::495::TaskManager.Task::(_debug) Task 4c5e6ba8-c896-4a4a-9ce0-ba35b30e2751: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 03:36:18,492::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`dd3deb08-02c6-4ee6-b00d-9be377e153ef`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 03:36:18,492::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 03:36:18,493::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 03:36:18,493::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`dd3deb08-02c6-4ee6-b00d-9be377e153ef`::Granted request
>Thread-15914435::DEBUG::2013-11-20 03:36:18,494::task::495::TaskManager.Task::(_debug) Task 4c5e6ba8-c896-4a4a-9ce0-ba35b30e2751: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 03:36:18,494::task::495::TaskManager.Task::(_debug) Task 4c5e6ba8-c896-4a4a-9ce0-ba35b30e2751: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 03:36:18,495::task::495::TaskManager.Task::(_debug) Task 4c5e6ba8-c896-4a4a-9ce0-ba35b30e2751: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 03:36:18,495::task::495::TaskManager.Task::(_debug) Task 4c5e6ba8-c896-4a4a-9ce0-ba35b30e2751: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 03:36:18,496::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 03:36:18,496::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 03:36:18,497::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 03:36:18,497::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 03:36:18,499::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 03:36:18,500::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 03:36:18,500::task::495::TaskManager.Task::(_debug) Task 4c5e6ba8-c896-4a4a-9ce0-ba35b30e2751: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 03:36:18,501::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 03:36:19,100::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:36:19,757::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0499419 s, 20.5 MB/s\n'; <rc> = 0
>Thread-15914453::INFO::2013-11-20 03:36:20,466::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 03:36:20,467::task::495::TaskManager.Task::(_debug) Task 395a7bbb-f416-4769-adaa-b82283bf56ce: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 03:36:20,484::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d465cb05-f80d-4ee5-8e07-457443bd41d9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 03:36:20,485::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 03:36:20,486::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 03:36:20,487::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d465cb05-f80d-4ee5-8e07-457443bd41d9`::Granted request
>Thread-15914453::DEBUG::2013-11-20 03:36:20,488::task::495::TaskManager.Task::(_debug) Task 395a7bbb-f416-4769-adaa-b82283bf56ce: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 03:36:20,488::task::495::TaskManager.Task::(_debug) Task 395a7bbb-f416-4769-adaa-b82283bf56ce: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 03:36:20,489::task::495::TaskManager.Task::(_debug) Task 395a7bbb-f416-4769-adaa-b82283bf56ce: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 03:36:20,489::task::495::TaskManager.Task::(_debug) Task 395a7bbb-f416-4769-adaa-b82283bf56ce: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 03:36:20,490::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 03:36:20,490::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 03:36:20,491::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 03:36:20,491::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 03:36:20,491::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 03:36:20,492::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 03:36:20,492::task::495::TaskManager.Task::(_debug) Task 395a7bbb-f416-4769-adaa-b82283bf56ce: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 03:36:20,493::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914628::INFO::2013-11-20 03:36:21,214::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914628::DEBUG::2013-11-20 03:36:21,214::task::495::TaskManager.Task::(_debug) Task 161a4fe8-b5e0-4d2b-a642-02c606ed7707: moving from state init -> state preparing
>Thread-15914628::DEBUG::2013-11-20 03:36:21,215::task::495::TaskManager.Task::(_debug) Task 161a4fe8-b5e0-4d2b-a642-02c606ed7707: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000280141830444', 'lastCheck': 1384918578.109441, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0189471244812', 'lastCheck': 1384918574.1225669, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00127506256104', 'lastCheck': 1384918577.6890521, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000320911407471', 'lastCheck': 1384918573.5659461, 'valid': True, 'code': 0}}
>Thread-15914628::DEBUG::2013-11-20 03:36:21,216::task::495::TaskManager.Task::(_debug) Task 161a4fe8-b5e0-4d2b-a642-02c606ed7707: moving from state preparing -> state finished
>Thread-15914628::DEBUG::2013-11-20 03:36:21,216::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914628::DEBUG::2013-11-20 03:36:21,217::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914628::DEBUG::2013-11-20 03:36:21,217::task::495::TaskManager.Task::(_debug) Task 161a4fe8-b5e0-4d2b-a642-02c606ed7707: ref 0 aborting False
>Thread-15914628::INFO::2013-11-20 03:36:21,218::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000280141830444', 'lastCheck': 1384918578.109441, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0189471244812', 'lastCheck': 1384918574.1225669, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00127506256104', 'lastCheck': 1384918577.6890521, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000320911407471', 'lastCheck': 1384918573.5659461, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:36:21,767::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:36:22,175::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0509196 s, 20.1 MB/s\n'; <rc> = 0
>Thread-15914519::INFO::2013-11-20 03:36:23,980::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 03:36:23,981::task::495::TaskManager.Task::(_debug) Task f31e806c-be39-4738-9aa8-790bcda6f703: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 03:36:23,993::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`89472347-8f57-4e81-8b06-928048ad5432`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 03:36:23,993::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 03:36:23,994::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 03:36:23,994::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`89472347-8f57-4e81-8b06-928048ad5432`::Granted request
>Thread-15914519::DEBUG::2013-11-20 03:36:24,003::task::495::TaskManager.Task::(_debug) Task f31e806c-be39-4738-9aa8-790bcda6f703: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 03:36:24,005::task::495::TaskManager.Task::(_debug) Task f31e806c-be39-4738-9aa8-790bcda6f703: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 03:36:24,006::task::495::TaskManager.Task::(_debug) Task f31e806c-be39-4738-9aa8-790bcda6f703: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 03:36:24,006::task::495::TaskManager.Task::(_debug) Task f31e806c-be39-4738-9aa8-790bcda6f703: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 03:36:24,007::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 03:36:24,007::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 03:36:24,008::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 03:36:24,008::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 03:36:24,009::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 03:36:24,009::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 03:36:24,010::task::495::TaskManager.Task::(_debug) Task f31e806c-be39-4738-9aa8-790bcda6f703: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 03:36:24,010::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 03:36:24,186::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:36:24,712::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0582696 s, 17.6 MB/s\n'; <rc> = 0
>Thread-15914631::DEBUG::2013-11-20 03:36:26,548::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914631::INFO::2013-11-20 03:36:26,549::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914631::DEBUG::2013-11-20 03:36:26,549::task::495::TaskManager.Task::(_debug) Task 778046d5-8be7-4544-98a4-5684a3b33e92: moving from state init -> state preparing
>Thread-15914631::DEBUG::2013-11-20 03:36:26,550::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914631::DEBUG::2013-11-20 03:36:26,550::task::495::TaskManager.Task::(_debug) Task 778046d5-8be7-4544-98a4-5684a3b33e92: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914631::DEBUG::2013-11-20 03:36:26,551::task::495::TaskManager.Task::(_debug) Task 778046d5-8be7-4544-98a4-5684a3b33e92: moving from state preparing -> state finished
>Thread-15914631::DEBUG::2013-11-20 03:36:26,551::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914631::DEBUG::2013-11-20 03:36:26,552::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914631::DEBUG::2013-11-20 03:36:26,552::task::495::TaskManager.Task::(_debug) Task 778046d5-8be7-4544-98a4-5684a3b33e92: ref 0 aborting False
>Thread-15914631::INFO::2013-11-20 03:36:26,556::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914632::DEBUG::2013-11-20 03:36:26,560::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914632::INFO::2013-11-20 03:36:26,561::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914632::DEBUG::2013-11-20 03:36:26,570::task::495::TaskManager.Task::(_debug) Task ade2c76e-f984-4e01-914c-d339f0a0fffc: moving from state init -> state preparing
>Thread-15914632::DEBUG::2013-11-20 03:36:26,571::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f14fea54-9719-4556-bd91-4aef88a50a21`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914632::DEBUG::2013-11-20 03:36:26,571::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914632::DEBUG::2013-11-20 03:36:26,575::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914632::DEBUG::2013-11-20 03:36:26,575::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f14fea54-9719-4556-bd91-4aef88a50a21`::Granted request
>Thread-15914632::DEBUG::2013-11-20 03:36:26,576::task::495::TaskManager.Task::(_debug) Task ade2c76e-f984-4e01-914c-d339f0a0fffc: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914632::DEBUG::2013-11-20 03:36:26,577::task::495::TaskManager.Task::(_debug) Task ade2c76e-f984-4e01-914c-d339f0a0fffc: ref 1 aborting False
>Thread-15914632::WARNING::2013-11-20 03:36:26,579::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914632::DEBUG::2013-11-20 03:36:26,579::task::495::TaskManager.Task::(_debug) Task ade2c76e-f984-4e01-914c-d339f0a0fffc: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914632::DEBUG::2013-11-20 03:36:26,580::task::495::TaskManager.Task::(_debug) Task ade2c76e-f984-4e01-914c-d339f0a0fffc: moving from state preparing -> state finished
>Thread-15914632::DEBUG::2013-11-20 03:36:26,581::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914632::DEBUG::2013-11-20 03:36:26,581::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914632::DEBUG::2013-11-20 03:36:26,582::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914632::DEBUG::2013-11-20 03:36:26,582::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914632::DEBUG::2013-11-20 03:36:26,583::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914632::DEBUG::2013-11-20 03:36:26,583::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914632::DEBUG::2013-11-20 03:36:26,584::task::495::TaskManager.Task::(_debug) Task ade2c76e-f984-4e01-914c-d339f0a0fffc: ref 0 aborting False
>Thread-15914632::INFO::2013-11-20 03:36:26,584::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:36:26,722::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:36:27,135::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.047749 s, 21.4 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 03:36:28,649::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:36:28,650::task::495::TaskManager.Task::(_debug) Task 6aa7eea4-7d93-4bbe-8bb0-a00f22adbdfd: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:36:28,651::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`358dc341-d058-4af6-b888-1ae76391fd55`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:36:28,652::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:36:28,658::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:36:28,659::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`358dc341-d058-4af6-b888-1ae76391fd55`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:36:28,659::task::495::TaskManager.Task::(_debug) Task 6aa7eea4-7d93-4bbe-8bb0-a00f22adbdfd: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:36:28,660::task::495::TaskManager.Task::(_debug) Task 6aa7eea4-7d93-4bbe-8bb0-a00f22adbdfd: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:36:28,660::task::495::TaskManager.Task::(_debug) Task 6aa7eea4-7d93-4bbe-8bb0-a00f22adbdfd: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:36:28,661::task::495::TaskManager.Task::(_debug) Task 6aa7eea4-7d93-4bbe-8bb0-a00f22adbdfd: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:36:28,661::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:36:28,662::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:36:28,662::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:36:28,662::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:36:28,663::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:36:28,663::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:36:28,663::task::495::TaskManager.Task::(_debug) Task 6aa7eea4-7d93-4bbe-8bb0-a00f22adbdfd: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:36:28,664::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:36:28,664::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:36:28,665::task::495::TaskManager.Task::(_debug) Task d1ba8b54-97da-4e4e-ac5e-7b8a6f2a9862: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:36:28,665::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`29f92141-feff-46a6-8c29-de77a97d3455`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:36:28,666::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:36:28,666::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:36:28,667::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`29f92141-feff-46a6-8c29-de77a97d3455`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:36:28,667::task::495::TaskManager.Task::(_debug) Task d1ba8b54-97da-4e4e-ac5e-7b8a6f2a9862: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:36:28,668::task::495::TaskManager.Task::(_debug) Task d1ba8b54-97da-4e4e-ac5e-7b8a6f2a9862: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:36:28,668::task::495::TaskManager.Task::(_debug) Task d1ba8b54-97da-4e4e-ac5e-7b8a6f2a9862: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:36:28,668::task::495::TaskManager.Task::(_debug) Task d1ba8b54-97da-4e4e-ac5e-7b8a6f2a9862: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:36:28,669::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:36:28,669::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:36:28,669::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:36:28,670::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:36:28,670::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:36:28,670::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:36:28,671::task::495::TaskManager.Task::(_debug) Task d1ba8b54-97da-4e4e-ac5e-7b8a6f2a9862: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:36:28,671::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:36:29,145::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:36:30,058::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0510143 s, 20.1 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 03:36:30,642::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:36:30,658::task::495::TaskManager.Task::(_debug) Task 7216eb00-0bbd-4806-9b39-e1c65ddc7ce7: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:36:30,660::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a9b9e01d-5c1e-4860-bca0-55af77fff4f3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:36:30,661::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:36:30,662::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:36:30,662::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a9b9e01d-5c1e-4860-bca0-55af77fff4f3`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:36:30,664::task::495::TaskManager.Task::(_debug) Task 7216eb00-0bbd-4806-9b39-e1c65ddc7ce7: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:36:30,664::task::495::TaskManager.Task::(_debug) Task 7216eb00-0bbd-4806-9b39-e1c65ddc7ce7: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:36:30,665::task::495::TaskManager.Task::(_debug) Task 7216eb00-0bbd-4806-9b39-e1c65ddc7ce7: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:36:30,666::task::495::TaskManager.Task::(_debug) Task 7216eb00-0bbd-4806-9b39-e1c65ddc7ce7: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:36:30,666::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:36:30,667::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:36:30,667::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:36:30,668::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:36:30,668::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:36:30,669::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:36:30,669::task::495::TaskManager.Task::(_debug) Task 7216eb00-0bbd-4806-9b39-e1c65ddc7ce7: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:36:30,670::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:36:30,670::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:36:30,671::task::495::TaskManager.Task::(_debug) Task cf13df7c-01a8-4f17-9161-b1831900cb59: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:36:30,672::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8a7269fe-46ae-4315-88a4-4be424fd0c44`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:36:30,672::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:36:30,673::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:36:30,673::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8a7269fe-46ae-4315-88a4-4be424fd0c44`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:36:30,674::task::495::TaskManager.Task::(_debug) Task cf13df7c-01a8-4f17-9161-b1831900cb59: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:36:30,674::task::495::TaskManager.Task::(_debug) Task cf13df7c-01a8-4f17-9161-b1831900cb59: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:36:30,675::task::495::TaskManager.Task::(_debug) Task cf13df7c-01a8-4f17-9161-b1831900cb59: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:36:30,675::task::495::TaskManager.Task::(_debug) Task cf13df7c-01a8-4f17-9161-b1831900cb59: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:36:30,675::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:36:30,676::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:36:30,676::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:36:30,677::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:36:30,677::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:36:30,677::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:36:30,678::task::495::TaskManager.Task::(_debug) Task cf13df7c-01a8-4f17-9161-b1831900cb59: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:36:30,678::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:36:30,679::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:36:30,679::task::495::TaskManager.Task::(_debug) Task e23562e2-e055-44f0-a14b-5ba3515d2e58: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:36:30,680::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`cf141f97-2a97-4f7a-aff8-17281c04f835`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:36:30,680::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:36:30,681::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:36:30,681::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`cf141f97-2a97-4f7a-aff8-17281c04f835`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:36:30,682::task::495::TaskManager.Task::(_debug) Task e23562e2-e055-44f0-a14b-5ba3515d2e58: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:36:30,682::task::495::TaskManager.Task::(_debug) Task e23562e2-e055-44f0-a14b-5ba3515d2e58: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:36:30,683::task::495::TaskManager.Task::(_debug) Task e23562e2-e055-44f0-a14b-5ba3515d2e58: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:36:30,683::task::495::TaskManager.Task::(_debug) Task e23562e2-e055-44f0-a14b-5ba3515d2e58: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:36:30,683::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:36:30,684::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:36:30,684::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:36:30,685::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:36:30,685::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:36:30,685::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:36:30,686::task::495::TaskManager.Task::(_debug) Task e23562e2-e055-44f0-a14b-5ba3515d2e58: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:36:30,686::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:36:30,687::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:36:30,687::task::495::TaskManager.Task::(_debug) Task 4645e177-568a-436a-bff8-7eb2c9d05e91: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:36:30,688::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`16d26ba2-662c-4ae4-8728-4c0a444a90f5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:36:30,688::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:36:30,689::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:36:30,689::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`16d26ba2-662c-4ae4-8728-4c0a444a90f5`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:36:30,690::task::495::TaskManager.Task::(_debug) Task 4645e177-568a-436a-bff8-7eb2c9d05e91: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:36:30,690::task::495::TaskManager.Task::(_debug) Task 4645e177-568a-436a-bff8-7eb2c9d05e91: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:36:30,691::task::495::TaskManager.Task::(_debug) Task 4645e177-568a-436a-bff8-7eb2c9d05e91: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:36:30,691::task::495::TaskManager.Task::(_debug) Task 4645e177-568a-436a-bff8-7eb2c9d05e91: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:36:30,692::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:36:30,692::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:36:30,692::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:36:30,693::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:36:30,693::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:36:30,694::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:36:30,694::task::495::TaskManager.Task::(_debug) Task 4645e177-568a-436a-bff8-7eb2c9d05e91: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:36:30,694::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:36:32,068::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:36:34,723::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0615824 s, 16.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:36:36,738::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914636::DEBUG::2013-11-20 03:36:36,924::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914637::INFO::2013-11-20 03:36:36,928::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914636::INFO::2013-11-20 03:36:36,929::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914637::DEBUG::2013-11-20 03:36:36,930::task::495::TaskManager.Task::(_debug) Task 83e97da7-d2b7-496e-8b06-a1d0abf8ba5e: moving from state init -> state preparing
>Thread-15914636::DEBUG::2013-11-20 03:36:36,930::task::495::TaskManager.Task::(_debug) Task 8bc76c5c-7804-454e-9fce-87fc8ecb7d53: moving from state init -> state preparing
>Thread-15914637::DEBUG::2013-11-20 03:36:36,931::task::495::TaskManager.Task::(_debug) Task 83e97da7-d2b7-496e-8b06-a1d0abf8ba5e: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000470161437988', 'lastCheck': 1384918588.2784729, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00102710723877', 'lastCheck': 1384918594.3160679, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000917196273804', 'lastCheck': 1384918587.701437, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00043511390686', 'lastCheck': 1384918593.585772, 'valid': True, 'code': 0}}
>Thread-15914636::DEBUG::2013-11-20 03:36:36,932::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914637::DEBUG::2013-11-20 03:36:36,933::task::495::TaskManager.Task::(_debug) Task 83e97da7-d2b7-496e-8b06-a1d0abf8ba5e: moving from state preparing -> state finished
>Thread-15914636::DEBUG::2013-11-20 03:36:36,934::task::495::TaskManager.Task::(_debug) Task 8bc76c5c-7804-454e-9fce-87fc8ecb7d53: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914637::DEBUG::2013-11-20 03:36:36,934::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914636::DEBUG::2013-11-20 03:36:36,935::task::495::TaskManager.Task::(_debug) Task 8bc76c5c-7804-454e-9fce-87fc8ecb7d53: moving from state preparing -> state finished
>Thread-15914637::DEBUG::2013-11-20 03:36:36,936::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914636::DEBUG::2013-11-20 03:36:36,936::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914637::DEBUG::2013-11-20 03:36:36,937::task::495::TaskManager.Task::(_debug) Task 83e97da7-d2b7-496e-8b06-a1d0abf8ba5e: ref 0 aborting False
>Thread-15914636::DEBUG::2013-11-20 03:36:36,938::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914637::INFO::2013-11-20 03:36:36,938::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000470161437988', 'lastCheck': 1384918588.2784729, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00102710723877', 'lastCheck': 1384918594.3160679, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000917196273804', 'lastCheck': 1384918587.701437, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00043511390686', 'lastCheck': 1384918593.585772, 'valid': True, 'code': 0}}
>Thread-15914636::DEBUG::2013-11-20 03:36:36,939::task::495::TaskManager.Task::(_debug) Task 8bc76c5c-7804-454e-9fce-87fc8ecb7d53: ref 0 aborting False
>Thread-15914636::INFO::2013-11-20 03:36:36,940::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914638::DEBUG::2013-11-20 03:36:36,952::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914638::INFO::2013-11-20 03:36:36,953::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914638::DEBUG::2013-11-20 03:36:36,953::task::495::TaskManager.Task::(_debug) Task 983ea87a-845c-4b89-90c8-226d1ff1dda9: moving from state init -> state preparing
>Thread-15914638::DEBUG::2013-11-20 03:36:36,955::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1cd9d01f-2944-4a1d-ae07-eea804e78834`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914638::DEBUG::2013-11-20 03:36:36,956::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914638::DEBUG::2013-11-20 03:36:36,956::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914638::DEBUG::2013-11-20 03:36:36,957::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1cd9d01f-2944-4a1d-ae07-eea804e78834`::Granted request
>Thread-15914638::DEBUG::2013-11-20 03:36:36,957::task::495::TaskManager.Task::(_debug) Task 983ea87a-845c-4b89-90c8-226d1ff1dda9: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914638::DEBUG::2013-11-20 03:36:36,958::task::495::TaskManager.Task::(_debug) Task 983ea87a-845c-4b89-90c8-226d1ff1dda9: ref 1 aborting False
>Thread-15914638::WARNING::2013-11-20 03:36:36,959::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914638::DEBUG::2013-11-20 03:36:36,960::task::495::TaskManager.Task::(_debug) Task 983ea87a-845c-4b89-90c8-226d1ff1dda9: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914638::DEBUG::2013-11-20 03:36:36,961::task::495::TaskManager.Task::(_debug) Task 983ea87a-845c-4b89-90c8-226d1ff1dda9: moving from state preparing -> state finished
>Thread-15914638::DEBUG::2013-11-20 03:36:36,961::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914638::DEBUG::2013-11-20 03:36:36,962::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914638::DEBUG::2013-11-20 03:36:36,962::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914638::DEBUG::2013-11-20 03:36:36,963::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914638::DEBUG::2013-11-20 03:36:36,963::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914638::DEBUG::2013-11-20 03:36:36,963::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914638::DEBUG::2013-11-20 03:36:36,964::task::495::TaskManager.Task::(_debug) Task 983ea87a-845c-4b89-90c8-226d1ff1dda9: ref 0 aborting False
>Thread-15914638::INFO::2013-11-20 03:36:36,964::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:36:37,165::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0597052 s, 17.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:36:39,177::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:36:39,601::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0601218 s, 17.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:36:41,612::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:36:42,446::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0492968 s, 20.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:36:44,456::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:36:44,863::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0625085 s, 16.4 MB/s\n'; <rc> = 0
>Thread-15913639::INFO::2013-11-20 03:36:45,766::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:36:45,766::task::495::TaskManager.Task::(_debug) Task cd10ae1b-429f-4f91-be60-51d6266bb7e1: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:36:45,768::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9b4c1794-eb20-44f7-b05f-723e872fa38c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:36:45,769::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:36:45,770::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:36:45,770::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9b4c1794-eb20-44f7-b05f-723e872fa38c`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:36:45,771::task::495::TaskManager.Task::(_debug) Task cd10ae1b-429f-4f91-be60-51d6266bb7e1: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:36:45,772::task::495::TaskManager.Task::(_debug) Task cd10ae1b-429f-4f91-be60-51d6266bb7e1: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:36:45,772::task::495::TaskManager.Task::(_debug) Task cd10ae1b-429f-4f91-be60-51d6266bb7e1: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:36:45,773::task::495::TaskManager.Task::(_debug) Task cd10ae1b-429f-4f91-be60-51d6266bb7e1: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:36:45,773::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:36:45,774::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:36:45,774::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:36:45,775::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:36:45,775::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:36:45,776::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:36:45,776::task::495::TaskManager.Task::(_debug) Task cd10ae1b-429f-4f91-be60-51d6266bb7e1: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:36:45,777::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:36:46,873::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914643::DEBUG::2013-11-20 03:36:48,724::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914643::INFO::2013-11-20 03:36:48,728::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914643::DEBUG::2013-11-20 03:36:48,728::task::495::TaskManager.Task::(_debug) Task 80ddf458-9ce7-4294-822d-0ed94993b3d4: moving from state init -> state preparing
>Thread-15914643::DEBUG::2013-11-20 03:36:48,735::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914643::DEBUG::2013-11-20 03:36:48,736::task::495::TaskManager.Task::(_debug) Task 80ddf458-9ce7-4294-822d-0ed94993b3d4: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914643::DEBUG::2013-11-20 03:36:48,736::task::495::TaskManager.Task::(_debug) Task 80ddf458-9ce7-4294-822d-0ed94993b3d4: moving from state preparing -> state finished
>Thread-15914643::DEBUG::2013-11-20 03:36:48,737::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914643::DEBUG::2013-11-20 03:36:48,738::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914643::DEBUG::2013-11-20 03:36:48,738::task::495::TaskManager.Task::(_debug) Task 80ddf458-9ce7-4294-822d-0ed94993b3d4: ref 0 aborting False
>Thread-15914643::INFO::2013-11-20 03:36:48,739::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914644::DEBUG::2013-11-20 03:36:48,745::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914644::INFO::2013-11-20 03:36:48,746::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914644::DEBUG::2013-11-20 03:36:48,746::task::495::TaskManager.Task::(_debug) Task 4fb3236b-d43c-4e9f-887e-de513f72277d: moving from state init -> state preparing
>Thread-15914644::DEBUG::2013-11-20 03:36:48,748::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`75bdb7fb-4c1a-41f7-8b8f-a0de00de8b67`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914644::DEBUG::2013-11-20 03:36:48,748::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914644::DEBUG::2013-11-20 03:36:48,749::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914644::DEBUG::2013-11-20 03:36:48,750::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`75bdb7fb-4c1a-41f7-8b8f-a0de00de8b67`::Granted request
>Thread-15914644::DEBUG::2013-11-20 03:36:48,751::task::495::TaskManager.Task::(_debug) Task 4fb3236b-d43c-4e9f-887e-de513f72277d: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914644::DEBUG::2013-11-20 03:36:48,752::task::495::TaskManager.Task::(_debug) Task 4fb3236b-d43c-4e9f-887e-de513f72277d: ref 1 aborting False
>Thread-15914644::WARNING::2013-11-20 03:36:48,754::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914644::DEBUG::2013-11-20 03:36:48,755::task::495::TaskManager.Task::(_debug) Task 4fb3236b-d43c-4e9f-887e-de513f72277d: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914644::DEBUG::2013-11-20 03:36:48,757::task::495::TaskManager.Task::(_debug) Task 4fb3236b-d43c-4e9f-887e-de513f72277d: moving from state preparing -> state finished
>Thread-15914644::DEBUG::2013-11-20 03:36:48,757::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914644::DEBUG::2013-11-20 03:36:48,758::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914644::DEBUG::2013-11-20 03:36:48,759::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914644::DEBUG::2013-11-20 03:36:48,759::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914644::DEBUG::2013-11-20 03:36:48,760::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914644::DEBUG::2013-11-20 03:36:48,760::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914644::DEBUG::2013-11-20 03:36:48,761::task::495::TaskManager.Task::(_debug) Task 4fb3236b-d43c-4e9f-887e-de513f72277d: ref 0 aborting False
>Thread-15914644::INFO::2013-11-20 03:36:48,761::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:36:48,943::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0453946 s, 22.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:36:50,952::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914646::INFO::2013-11-20 03:36:53,173::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914646::DEBUG::2013-11-20 03:36:53,175::task::495::TaskManager.Task::(_debug) Task 9e7a09b8-59a5-4f48-9cc6-3cc6cff3ee13: moving from state init -> state preparing
>Thread-15914646::DEBUG::2013-11-20 03:36:53,175::task::495::TaskManager.Task::(_debug) Task 9e7a09b8-59a5-4f48-9cc6-3cc6cff3ee13: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000593185424805', 'lastCheck': 1384918608.726352, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0191969871521', 'lastCheck': 1384918604.3605211, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00306797027588', 'lastCheck': 1384918608.7259099, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000400066375732', 'lastCheck': 1384918603.5963161, 'valid': True, 'code': 0}}
>Thread-15914646::DEBUG::2013-11-20 03:36:53,185::task::495::TaskManager.Task::(_debug) Task 9e7a09b8-59a5-4f48-9cc6-3cc6cff3ee13: moving from state preparing -> state finished
>Thread-15914646::DEBUG::2013-11-20 03:36:53,186::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914646::DEBUG::2013-11-20 03:36:53,187::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914646::DEBUG::2013-11-20 03:36:53,187::task::495::TaskManager.Task::(_debug) Task 9e7a09b8-59a5-4f48-9cc6-3cc6cff3ee13: ref 0 aborting False
>Thread-15914646::INFO::2013-11-20 03:36:53,188::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000593185424805', 'lastCheck': 1384918608.726352, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0191969871521', 'lastCheck': 1384918604.3605211, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00306797027588', 'lastCheck': 1384918608.7259099, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000400066375732', 'lastCheck': 1384918603.5963161, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:36:53,873::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.049969 s, 20.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:36:55,884::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:36:56,395::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.054387 s, 18.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:36:58,405::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-42838::DEBUG::2013-11-20 03:36:58,736::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15914649::DEBUG::2013-11-20 03:36:59,066::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914649::INFO::2013-11-20 03:36:59,075::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914649::DEBUG::2013-11-20 03:36:59,076::task::495::TaskManager.Task::(_debug) Task 7085851e-105d-4b34-92d5-af29e9cd4213: moving from state init -> state preparing
>Thread-15914649::DEBUG::2013-11-20 03:36:59,077::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914649::DEBUG::2013-11-20 03:36:59,077::task::495::TaskManager.Task::(_debug) Task 7085851e-105d-4b34-92d5-af29e9cd4213: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914649::DEBUG::2013-11-20 03:36:59,078::task::495::TaskManager.Task::(_debug) Task 7085851e-105d-4b34-92d5-af29e9cd4213: moving from state preparing -> state finished
>Thread-15914649::DEBUG::2013-11-20 03:36:59,078::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914649::DEBUG::2013-11-20 03:36:59,079::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914649::DEBUG::2013-11-20 03:36:59,079::task::495::TaskManager.Task::(_debug) Task 7085851e-105d-4b34-92d5-af29e9cd4213: ref 0 aborting False
>Thread-15914649::INFO::2013-11-20 03:36:59,080::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914650::DEBUG::2013-11-20 03:36:59,084::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914650::INFO::2013-11-20 03:36:59,085::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914650::DEBUG::2013-11-20 03:36:59,086::task::495::TaskManager.Task::(_debug) Task aeaf0456-9009-4ea2-9083-d6044914a871: moving from state init -> state preparing
>Thread-15914650::DEBUG::2013-11-20 03:36:59,087::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7befe63a-499f-4876-9a7f-30d57a1620b3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914650::DEBUG::2013-11-20 03:36:59,088::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914650::DEBUG::2013-11-20 03:36:59,090::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914650::DEBUG::2013-11-20 03:36:59,090::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7befe63a-499f-4876-9a7f-30d57a1620b3`::Granted request
>Thread-15914650::DEBUG::2013-11-20 03:36:59,092::task::495::TaskManager.Task::(_debug) Task aeaf0456-9009-4ea2-9083-d6044914a871: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914650::DEBUG::2013-11-20 03:36:59,092::task::495::TaskManager.Task::(_debug) Task aeaf0456-9009-4ea2-9083-d6044914a871: ref 1 aborting False
>Thread-15914650::WARNING::2013-11-20 03:36:59,103::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914650::DEBUG::2013-11-20 03:36:59,107::task::495::TaskManager.Task::(_debug) Task aeaf0456-9009-4ea2-9083-d6044914a871: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914650::DEBUG::2013-11-20 03:36:59,110::task::495::TaskManager.Task::(_debug) Task aeaf0456-9009-4ea2-9083-d6044914a871: moving from state preparing -> state finished
>Thread-15914650::DEBUG::2013-11-20 03:36:59,111::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914650::DEBUG::2013-11-20 03:36:59,111::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914650::DEBUG::2013-11-20 03:36:59,112::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914650::DEBUG::2013-11-20 03:36:59,112::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914650::DEBUG::2013-11-20 03:36:59,113::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914650::DEBUG::2013-11-20 03:36:59,113::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914650::DEBUG::2013-11-20 03:36:59,114::task::495::TaskManager.Task::(_debug) Task aeaf0456-9009-4ea2-9083-d6044914a871: ref 0 aborting False
>Thread-15914650::INFO::2013-11-20 03:36:59,114::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-42838::DEBUG::2013-11-20 03:36:59,458::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:37:00,404::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.052462 s, 19.5 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 03:37:00,903::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:37:00,912::task::495::TaskManager.Task::(_debug) Task abcab17a-b786-483e-b67b-d42283e9b3fd: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:37:00,913::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`65b9040a-dc83-4be1-8178-d4d7ba58b477`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:37:00,914::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:37:00,915::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:37:00,915::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`65b9040a-dc83-4be1-8178-d4d7ba58b477`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:37:00,916::task::495::TaskManager.Task::(_debug) Task abcab17a-b786-483e-b67b-d42283e9b3fd: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:37:00,917::task::495::TaskManager.Task::(_debug) Task abcab17a-b786-483e-b67b-d42283e9b3fd: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:37:00,918::task::495::TaskManager.Task::(_debug) Task abcab17a-b786-483e-b67b-d42283e9b3fd: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:37:00,919::task::495::TaskManager.Task::(_debug) Task abcab17a-b786-483e-b67b-d42283e9b3fd: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:37:00,919::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:37:00,920::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:37:00,920::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:37:00,921::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:37:00,922::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:37:00,922::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:37:00,923::task::495::TaskManager.Task::(_debug) Task abcab17a-b786-483e-b67b-d42283e9b3fd: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:37:00,923::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:37:02,414::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:37:02,891::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.076222 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:37:04,907::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:37:05,381::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0572546 s, 17.9 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 03:37:06,728::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:37:06,729::task::495::TaskManager.Task::(_debug) Task f98d9a90-f142-4ca6-acb6-273b45e00a5a: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:37:06,730::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5c948f74-7a77-4d31-b34f-e3601c862641`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:37:06,731::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:37:06,731::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:37:06,732::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5c948f74-7a77-4d31-b34f-e3601c862641`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:37:06,733::task::495::TaskManager.Task::(_debug) Task f98d9a90-f142-4ca6-acb6-273b45e00a5a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:37:06,733::task::495::TaskManager.Task::(_debug) Task f98d9a90-f142-4ca6-acb6-273b45e00a5a: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:37:06,734::task::495::TaskManager.Task::(_debug) Task f98d9a90-f142-4ca6-acb6-273b45e00a5a: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:37:06,734::task::495::TaskManager.Task::(_debug) Task f98d9a90-f142-4ca6-acb6-273b45e00a5a: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:37:06,735::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:37:06,735::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:37:06,736::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:37:06,736::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:37:06,736::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:37:06,737::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:37:06,737::task::495::TaskManager.Task::(_debug) Task f98d9a90-f142-4ca6-acb6-273b45e00a5a: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:37:06,738::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:37:07,393::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:37:07,883::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0538173 s, 19.0 MB/s\n'; <rc> = 0
>Thread-15914654::DEBUG::2013-11-20 03:37:08,089::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Thread-15914655::INFO::2013-11-20 03:37:08,795::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914655::DEBUG::2013-11-20 03:37:08,797::task::495::TaskManager.Task::(_debug) Task 152c3875-80c7-4f66-964e-70d836bab38f: moving from state init -> state preparing
>Thread-15914655::DEBUG::2013-11-20 03:37:08,798::task::495::TaskManager.Task::(_debug) Task 152c3875-80c7-4f66-964e-70d836bab38f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000318050384521', 'lastCheck': 1384918619.462513, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00131320953369', 'lastCheck': 1384918624.7372329, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.165114164352', 'lastCheck': 1384918618.9037509, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000554084777832', 'lastCheck': 1384918623.7183621, 'valid': True, 'code': 0}}
>Thread-15914655::DEBUG::2013-11-20 03:37:08,800::task::495::TaskManager.Task::(_debug) Task 152c3875-80c7-4f66-964e-70d836bab38f: moving from state preparing -> state finished
>Thread-15914655::DEBUG::2013-11-20 03:37:08,801::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914655::DEBUG::2013-11-20 03:37:08,802::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914655::DEBUG::2013-11-20 03:37:08,803::task::495::TaskManager.Task::(_debug) Task 152c3875-80c7-4f66-964e-70d836bab38f: ref 0 aborting False
>Thread-15914655::INFO::2013-11-20 03:37:08,803::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000318050384521', 'lastCheck': 1384918619.462513, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00131320953369', 'lastCheck': 1384918624.7372329, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.165114164352', 'lastCheck': 1384918618.9037509, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000554084777832', 'lastCheck': 1384918623.7183621, 'valid': True, 'code': 0}}
>Thread-15913695::INFO::2013-11-20 03:37:09,173::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:37:09,174::task::495::TaskManager.Task::(_debug) Task 5ba21f86-0c90-4457-b743-6db305fde2c0: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:37:09,176::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b08ce65b-7ed9-44e8-81f4-b413e1b9d923`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:37:09,176::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:37:09,178::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:37:09,179::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b08ce65b-7ed9-44e8-81f4-b413e1b9d923`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:37:09,180::task::495::TaskManager.Task::(_debug) Task 5ba21f86-0c90-4457-b743-6db305fde2c0: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:37:09,181::task::495::TaskManager.Task::(_debug) Task 5ba21f86-0c90-4457-b743-6db305fde2c0: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:37:09,182::task::495::TaskManager.Task::(_debug) Task 5ba21f86-0c90-4457-b743-6db305fde2c0: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:37:09,183::task::495::TaskManager.Task::(_debug) Task 5ba21f86-0c90-4457-b743-6db305fde2c0: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:37:09,183::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:37:09,184::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:37:09,185::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:37:09,185::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:37:09,186::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:37:09,186::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:37:09,187::task::495::TaskManager.Task::(_debug) Task 5ba21f86-0c90-4457-b743-6db305fde2c0: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:37:09,188::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15914654::DEBUG::2013-11-20 03:37:09,300::clientIF::1219::vds::(fenceNode) rc 0 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = On
>Done
> err 
>Thread-15914657::DEBUG::2013-11-20 03:37:09,320::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=off,secure=,options=power_wait=4
>lanplus=1)
>Thread-15914659::DEBUG::2013-11-20 03:37:09,702::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914659::INFO::2013-11-20 03:37:09,704::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914659::DEBUG::2013-11-20 03:37:09,705::task::495::TaskManager.Task::(_debug) Task 0b7aa4fe-2649-4f18-9dd8-23f628d1cb4b: moving from state init -> state preparing
>Thread-15914659::DEBUG::2013-11-20 03:37:09,705::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914659::DEBUG::2013-11-20 03:37:09,706::task::495::TaskManager.Task::(_debug) Task 0b7aa4fe-2649-4f18-9dd8-23f628d1cb4b: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914659::DEBUG::2013-11-20 03:37:09,706::task::495::TaskManager.Task::(_debug) Task 0b7aa4fe-2649-4f18-9dd8-23f628d1cb4b: moving from state preparing -> state finished
>Thread-15914659::DEBUG::2013-11-20 03:37:09,707::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914659::DEBUG::2013-11-20 03:37:09,707::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914659::DEBUG::2013-11-20 03:37:09,708::task::495::TaskManager.Task::(_debug) Task 0b7aa4fe-2649-4f18-9dd8-23f628d1cb4b: ref 0 aborting False
>Thread-15914659::INFO::2013-11-20 03:37:09,708::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914660::DEBUG::2013-11-20 03:37:09,716::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914660::INFO::2013-11-20 03:37:09,718::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914660::DEBUG::2013-11-20 03:37:09,719::task::495::TaskManager.Task::(_debug) Task 20dc5815-4eac-4b57-ad67-a73debe75dc7: moving from state init -> state preparing
>Thread-15914660::DEBUG::2013-11-20 03:37:09,720::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`81b4194a-4ad0-4ba4-bbb4-d60bd3b26338`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914660::DEBUG::2013-11-20 03:37:09,721::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914660::DEBUG::2013-11-20 03:37:09,721::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914660::DEBUG::2013-11-20 03:37:09,727::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`81b4194a-4ad0-4ba4-bbb4-d60bd3b26338`::Granted request
>Thread-15914660::DEBUG::2013-11-20 03:37:09,727::task::495::TaskManager.Task::(_debug) Task 20dc5815-4eac-4b57-ad67-a73debe75dc7: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914660::DEBUG::2013-11-20 03:37:09,728::task::495::TaskManager.Task::(_debug) Task 20dc5815-4eac-4b57-ad67-a73debe75dc7: ref 1 aborting False
>Thread-15914660::WARNING::2013-11-20 03:37:09,729::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914660::DEBUG::2013-11-20 03:37:09,730::task::495::TaskManager.Task::(_debug) Task 20dc5815-4eac-4b57-ad67-a73debe75dc7: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914660::DEBUG::2013-11-20 03:37:09,730::task::495::TaskManager.Task::(_debug) Task 20dc5815-4eac-4b57-ad67-a73debe75dc7: moving from state preparing -> state finished
>Thread-15914660::DEBUG::2013-11-20 03:37:09,736::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914660::DEBUG::2013-11-20 03:37:09,742::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914660::DEBUG::2013-11-20 03:37:09,747::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914660::DEBUG::2013-11-20 03:37:09,748::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914660::DEBUG::2013-11-20 03:37:09,749::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914660::DEBUG::2013-11-20 03:37:09,749::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914660::DEBUG::2013-11-20 03:37:09,750::task::495::TaskManager.Task::(_debug) Task 20dc5815-4eac-4b57-ad67-a73debe75dc7: ref 0 aborting False
>Thread-15914660::INFO::2013-11-20 03:37:09,751::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:37:09,895::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:37:10,990::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0508171 s, 20.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:37:13,001::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:37:13,402::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0497847 s, 20.6 MB/s\n'; <rc> = 0
>Thread-15914658::DEBUG::2013-11-20 03:37:14,627::clientIF::1179::vds::(waitForPid) rc 0 inp agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=off
>passwd=XXXX
>power_wait=4
>lanplus=1 out Powering off machine @ IPMI:192.168.100.119...Done
> err 
>Thread-15914662::DEBUG::2013-11-20 03:37:14,883::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Dummy-15912651::DEBUG::2013-11-20 03:37:15,413::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914662::DEBUG::2013-11-20 03:37:15,616::clientIF::1219::vds::(fenceNode) rc 2 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = Off
>Done
> err 
>Thread-15914664::DEBUG::2013-11-20 03:37:15,623::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Dummy-15912651::DEBUG::2013-11-20 03:37:16,442::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.424345 s, 2.4 MB/s\n'; <rc> = 0
>Thread-15914664::DEBUG::2013-11-20 03:37:16,575::clientIF::1219::vds::(fenceNode) rc 2 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = Off
>Done
> err 
>Dummy-15912651::DEBUG::2013-11-20 03:37:18,453::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 03:37:19,239::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 03:37:19,240::task::495::TaskManager.Task::(_debug) Task 3c6450c9-f938-49af-a7a9-e1bf150559e5: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 03:37:19,247::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4313e9fe-ff8a-41ea-ae09-9a867515d4b6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 03:37:19,248::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 03:37:19,249::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 03:37:19,249::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4313e9fe-ff8a-41ea-ae09-9a867515d4b6`::Granted request
>Thread-15914435::DEBUG::2013-11-20 03:37:19,250::task::495::TaskManager.Task::(_debug) Task 3c6450c9-f938-49af-a7a9-e1bf150559e5: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 03:37:19,251::task::495::TaskManager.Task::(_debug) Task 3c6450c9-f938-49af-a7a9-e1bf150559e5: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 03:37:19,252::task::495::TaskManager.Task::(_debug) Task 3c6450c9-f938-49af-a7a9-e1bf150559e5: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 03:37:19,253::task::495::TaskManager.Task::(_debug) Task 3c6450c9-f938-49af-a7a9-e1bf150559e5: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 03:37:19,253::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 03:37:19,254::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 03:37:19,254::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 03:37:19,254::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 03:37:19,255::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 03:37:19,255::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 03:37:19,256::task::495::TaskManager.Task::(_debug) Task 3c6450c9-f938-49af-a7a9-e1bf150559e5: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 03:37:19,256::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914666::DEBUG::2013-11-20 03:37:20,642::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914666::INFO::2013-11-20 03:37:20,643::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914666::DEBUG::2013-11-20 03:37:20,643::task::495::TaskManager.Task::(_debug) Task f92ea81e-f4d5-4142-a973-909ff81a1188: moving from state init -> state preparing
>Thread-15914666::DEBUG::2013-11-20 03:37:20,644::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914666::DEBUG::2013-11-20 03:37:20,644::task::495::TaskManager.Task::(_debug) Task f92ea81e-f4d5-4142-a973-909ff81a1188: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914666::DEBUG::2013-11-20 03:37:20,644::task::495::TaskManager.Task::(_debug) Task f92ea81e-f4d5-4142-a973-909ff81a1188: moving from state preparing -> state finished
>Thread-15914666::DEBUG::2013-11-20 03:37:20,645::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914666::DEBUG::2013-11-20 03:37:20,645::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914666::DEBUG::2013-11-20 03:37:20,646::task::495::TaskManager.Task::(_debug) Task f92ea81e-f4d5-4142-a973-909ff81a1188: ref 0 aborting False
>Thread-15914666::INFO::2013-11-20 03:37:20,646::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914667::DEBUG::2013-11-20 03:37:20,650::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914667::INFO::2013-11-20 03:37:20,651::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914667::DEBUG::2013-11-20 03:37:20,652::task::495::TaskManager.Task::(_debug) Task 2b6a1bab-f3b6-405c-8d00-efd0c062e2c2: moving from state init -> state preparing
>Thread-15914667::DEBUG::2013-11-20 03:37:20,655::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`486502a0-dc9f-470c-9404-7ade9acde5e3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914667::DEBUG::2013-11-20 03:37:20,656::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914667::DEBUG::2013-11-20 03:37:20,657::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914667::DEBUG::2013-11-20 03:37:20,657::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`486502a0-dc9f-470c-9404-7ade9acde5e3`::Granted request
>Thread-15914667::DEBUG::2013-11-20 03:37:20,658::task::495::TaskManager.Task::(_debug) Task 2b6a1bab-f3b6-405c-8d00-efd0c062e2c2: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914667::DEBUG::2013-11-20 03:37:20,658::task::495::TaskManager.Task::(_debug) Task 2b6a1bab-f3b6-405c-8d00-efd0c062e2c2: ref 1 aborting False
>Thread-15914667::WARNING::2013-11-20 03:37:20,660::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914667::DEBUG::2013-11-20 03:37:20,661::task::495::TaskManager.Task::(_debug) Task 2b6a1bab-f3b6-405c-8d00-efd0c062e2c2: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914667::DEBUG::2013-11-20 03:37:20,662::task::495::TaskManager.Task::(_debug) Task 2b6a1bab-f3b6-405c-8d00-efd0c062e2c2: moving from state preparing -> state finished
>Thread-15914667::DEBUG::2013-11-20 03:37:20,663::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914667::DEBUG::2013-11-20 03:37:20,663::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914667::DEBUG::2013-11-20 03:37:20,664::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914667::DEBUG::2013-11-20 03:37:20,664::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914667::DEBUG::2013-11-20 03:37:20,665::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914667::DEBUG::2013-11-20 03:37:20,665::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914667::DEBUG::2013-11-20 03:37:20,666::task::495::TaskManager.Task::(_debug) Task 2b6a1bab-f3b6-405c-8d00-efd0c062e2c2: ref 0 aborting False
>Thread-15914667::INFO::2013-11-20 03:37:20,666::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:37:21,183::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0486728 s, 21.0 MB/s\n'; <rc> = 0
>Thread-15914453::INFO::2013-11-20 03:37:22,392::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 03:37:22,393::task::495::TaskManager.Task::(_debug) Task 6722cfd3-d347-4538-bc88-363051a9b2f5: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 03:37:22,394::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6c7fc913-1c59-4ade-a44c-487a4fcb3b1e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 03:37:22,395::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 03:37:22,395::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 03:37:22,396::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6c7fc913-1c59-4ade-a44c-487a4fcb3b1e`::Granted request
>Thread-15914453::DEBUG::2013-11-20 03:37:22,396::task::495::TaskManager.Task::(_debug) Task 6722cfd3-d347-4538-bc88-363051a9b2f5: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 03:37:22,397::task::495::TaskManager.Task::(_debug) Task 6722cfd3-d347-4538-bc88-363051a9b2f5: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 03:37:22,397::task::495::TaskManager.Task::(_debug) Task 6722cfd3-d347-4538-bc88-363051a9b2f5: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 03:37:22,397::task::495::TaskManager.Task::(_debug) Task 6722cfd3-d347-4538-bc88-363051a9b2f5: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 03:37:22,398::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 03:37:22,398::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 03:37:22,399::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 03:37:22,399::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 03:37:22,399::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 03:37:22,400::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 03:37:22,400::task::495::TaskManager.Task::(_debug) Task 6722cfd3-d347-4538-bc88-363051a9b2f5: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 03:37:22,400::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 03:37:23,194::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:37:24,161::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.052298 s, 19.6 MB/s\n'; <rc> = 0
>Thread-15914669::INFO::2013-11-20 03:37:24,340::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914669::DEBUG::2013-11-20 03:37:24,341::task::495::TaskManager.Task::(_debug) Task 09830d34-887d-402c-aaa0-c7e4b62be6c6: moving from state init -> state preparing
>Thread-15914669::DEBUG::2013-11-20 03:37:24,342::task::495::TaskManager.Task::(_debug) Task 09830d34-887d-402c-aaa0-c7e4b62be6c6: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00108313560486', 'lastCheck': 1384918640.4566481, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0227658748627', 'lastCheck': 1384918634.7731681, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00116491317749', 'lastCheck': 1384918640.4568701, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000399827957153', 'lastCheck': 1384918633.726419, 'valid': True, 'code': 0}}
>Thread-15914669::DEBUG::2013-11-20 03:37:24,342::task::495::TaskManager.Task::(_debug) Task 09830d34-887d-402c-aaa0-c7e4b62be6c6: moving from state preparing -> state finished
>Thread-15914669::DEBUG::2013-11-20 03:37:24,343::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914669::DEBUG::2013-11-20 03:37:24,343::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914669::DEBUG::2013-11-20 03:37:24,344::task::495::TaskManager.Task::(_debug) Task 09830d34-887d-402c-aaa0-c7e4b62be6c6: ref 0 aborting False
>Thread-15914669::INFO::2013-11-20 03:37:24,344::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00108313560486', 'lastCheck': 1384918640.4566481, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0227658748627', 'lastCheck': 1384918634.7731681, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00116491317749', 'lastCheck': 1384918640.4568701, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000399827957153', 'lastCheck': 1384918633.726419, 'valid': True, 'code': 0}}
>Thread-15914519::INFO::2013-11-20 03:37:25,450::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 03:37:25,451::task::495::TaskManager.Task::(_debug) Task 1f76def7-e4f1-4024-9cb0-f7ce8ec3028d: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 03:37:25,454::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3b94cb75-0024-4790-9165-540e78890c29`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 03:37:25,455::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 03:37:25,456::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 03:37:25,457::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3b94cb75-0024-4790-9165-540e78890c29`::Granted request
>Thread-15914519::DEBUG::2013-11-20 03:37:25,467::task::495::TaskManager.Task::(_debug) Task 1f76def7-e4f1-4024-9cb0-f7ce8ec3028d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 03:37:25,468::task::495::TaskManager.Task::(_debug) Task 1f76def7-e4f1-4024-9cb0-f7ce8ec3028d: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 03:37:25,469::task::495::TaskManager.Task::(_debug) Task 1f76def7-e4f1-4024-9cb0-f7ce8ec3028d: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 03:37:25,470::task::495::TaskManager.Task::(_debug) Task 1f76def7-e4f1-4024-9cb0-f7ce8ec3028d: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 03:37:25,471::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 03:37:25,471::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 03:37:25,472::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 03:37:25,473::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 03:37:25,479::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 03:37:25,480::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 03:37:25,481::task::495::TaskManager.Task::(_debug) Task 1f76def7-e4f1-4024-9cb0-f7ce8ec3028d: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 03:37:25,483::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 03:37:26,172::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:37:26,611::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0855587 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:37:28,635::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:37:29,554::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0880683 s, 11.6 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 03:37:30,569::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:37:30,570::task::495::TaskManager.Task::(_debug) Task 3ac35c31-8cd3-4928-a236-18acf3646bd6: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:37:30,571::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6a33f83b-9887-4a77-b872-0a50e235e29f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:37:30,571::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:37:30,572::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:37:30,572::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6a33f83b-9887-4a77-b872-0a50e235e29f`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:37:30,573::task::495::TaskManager.Task::(_debug) Task 3ac35c31-8cd3-4928-a236-18acf3646bd6: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:37:30,573::task::495::TaskManager.Task::(_debug) Task 3ac35c31-8cd3-4928-a236-18acf3646bd6: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:37:30,574::task::495::TaskManager.Task::(_debug) Task 3ac35c31-8cd3-4928-a236-18acf3646bd6: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:37:30,574::task::495::TaskManager.Task::(_debug) Task 3ac35c31-8cd3-4928-a236-18acf3646bd6: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:37:30,575::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:37:30,575::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:37:30,576::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:37:30,576::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:37:30,576::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:37:30,577::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:37:30,577::task::495::TaskManager.Task::(_debug) Task 3ac35c31-8cd3-4928-a236-18acf3646bd6: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:37:30,578::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:37:30,578::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:37:30,579::task::495::TaskManager.Task::(_debug) Task adee1d79-45af-48b6-8385-608906be0e60: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:37:30,580::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c0b92e6d-b6be-4d6e-b746-d4cbf83afb8f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:37:30,580::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:37:30,581::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:37:30,581::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c0b92e6d-b6be-4d6e-b746-d4cbf83afb8f`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:37:30,582::task::495::TaskManager.Task::(_debug) Task adee1d79-45af-48b6-8385-608906be0e60: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:37:30,582::task::495::TaskManager.Task::(_debug) Task adee1d79-45af-48b6-8385-608906be0e60: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:37:30,582::task::495::TaskManager.Task::(_debug) Task adee1d79-45af-48b6-8385-608906be0e60: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:37:30,583::task::495::TaskManager.Task::(_debug) Task adee1d79-45af-48b6-8385-608906be0e60: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:37:30,583::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:37:30,583::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:37:30,584::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:37:30,584::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:37:30,584::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:37:30,585::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:37:30,585::task::495::TaskManager.Task::(_debug) Task adee1d79-45af-48b6-8385-608906be0e60: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:37:30,586::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15914672::DEBUG::2013-11-20 03:37:30,935::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914672::INFO::2013-11-20 03:37:30,936::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914672::DEBUG::2013-11-20 03:37:30,937::task::495::TaskManager.Task::(_debug) Task 6800d690-8ce3-4000-9ca7-227f8e682a9e: moving from state init -> state preparing
>Thread-15914672::DEBUG::2013-11-20 03:37:30,938::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914672::DEBUG::2013-11-20 03:37:30,938::task::495::TaskManager.Task::(_debug) Task 6800d690-8ce3-4000-9ca7-227f8e682a9e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914672::DEBUG::2013-11-20 03:37:30,939::task::495::TaskManager.Task::(_debug) Task 6800d690-8ce3-4000-9ca7-227f8e682a9e: moving from state preparing -> state finished
>Thread-15914672::DEBUG::2013-11-20 03:37:30,941::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914672::DEBUG::2013-11-20 03:37:30,941::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914672::DEBUG::2013-11-20 03:37:30,942::task::495::TaskManager.Task::(_debug) Task 6800d690-8ce3-4000-9ca7-227f8e682a9e: ref 0 aborting False
>Thread-15914672::INFO::2013-11-20 03:37:30,942::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914674::DEBUG::2013-11-20 03:37:30,946::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914674::INFO::2013-11-20 03:37:30,946::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914674::DEBUG::2013-11-20 03:37:30,947::task::495::TaskManager.Task::(_debug) Task e7fb3ea1-acd2-4918-9042-59f699c7e69b: moving from state init -> state preparing
>Thread-15914674::DEBUG::2013-11-20 03:37:30,950::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a01d1ee2-2d06-4ac1-a191-a3b698be2ad6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914674::DEBUG::2013-11-20 03:37:30,950::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914674::DEBUG::2013-11-20 03:37:30,951::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914674::DEBUG::2013-11-20 03:37:30,951::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a01d1ee2-2d06-4ac1-a191-a3b698be2ad6`::Granted request
>Thread-15914674::DEBUG::2013-11-20 03:37:30,951::task::495::TaskManager.Task::(_debug) Task e7fb3ea1-acd2-4918-9042-59f699c7e69b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914674::DEBUG::2013-11-20 03:37:30,952::task::495::TaskManager.Task::(_debug) Task e7fb3ea1-acd2-4918-9042-59f699c7e69b: ref 1 aborting False
>Thread-15914674::WARNING::2013-11-20 03:37:30,953::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914674::DEBUG::2013-11-20 03:37:30,954::task::495::TaskManager.Task::(_debug) Task e7fb3ea1-acd2-4918-9042-59f699c7e69b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914674::DEBUG::2013-11-20 03:37:30,954::task::495::TaskManager.Task::(_debug) Task e7fb3ea1-acd2-4918-9042-59f699c7e69b: moving from state preparing -> state finished
>Thread-15914674::DEBUG::2013-11-20 03:37:30,955::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914674::DEBUG::2013-11-20 03:37:30,955::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914674::DEBUG::2013-11-20 03:37:30,955::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914674::DEBUG::2013-11-20 03:37:30,956::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914674::DEBUG::2013-11-20 03:37:30,956::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914674::DEBUG::2013-11-20 03:37:30,956::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914674::DEBUG::2013-11-20 03:37:30,957::task::495::TaskManager.Task::(_debug) Task e7fb3ea1-acd2-4918-9042-59f699c7e69b: ref 0 aborting False
>Thread-15914674::INFO::2013-11-20 03:37:30,957::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:37:31,564::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:37:31,987::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0525519 s, 19.5 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 03:37:32,421::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:37:32,423::task::495::TaskManager.Task::(_debug) Task 8c14a2a8-ec58-4dd7-ad2c-f7bf5f2cdaf7: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:37:32,426::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`25aabcbc-e0aa-44f0-9b38-11237885da59`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:37:32,427::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:37:32,429::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:37:32,429::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`25aabcbc-e0aa-44f0-9b38-11237885da59`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:37:32,430::task::495::TaskManager.Task::(_debug) Task 8c14a2a8-ec58-4dd7-ad2c-f7bf5f2cdaf7: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:37:32,431::task::495::TaskManager.Task::(_debug) Task 8c14a2a8-ec58-4dd7-ad2c-f7bf5f2cdaf7: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:37:32,431::task::495::TaskManager.Task::(_debug) Task 8c14a2a8-ec58-4dd7-ad2c-f7bf5f2cdaf7: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:37:32,432::task::495::TaskManager.Task::(_debug) Task 8c14a2a8-ec58-4dd7-ad2c-f7bf5f2cdaf7: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:37:32,433::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:37:32,434::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:37:32,436::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:37:32,437::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:37:32,438::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:37:32,439::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:37:32,440::task::495::TaskManager.Task::(_debug) Task 8c14a2a8-ec58-4dd7-ad2c-f7bf5f2cdaf7: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:37:32,441::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:37:32,443::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:37:32,444::task::495::TaskManager.Task::(_debug) Task b3d8503f-7f7f-4191-ae99-9f65669affd8: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:37:32,445::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`4b602ec5-640e-45e2-9316-8d944d676b95`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:37:32,446::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:37:32,447::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:37:32,447::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`4b602ec5-640e-45e2-9316-8d944d676b95`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:37:32,448::task::495::TaskManager.Task::(_debug) Task b3d8503f-7f7f-4191-ae99-9f65669affd8: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:37:32,449::task::495::TaskManager.Task::(_debug) Task b3d8503f-7f7f-4191-ae99-9f65669affd8: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:37:32,449::task::495::TaskManager.Task::(_debug) Task b3d8503f-7f7f-4191-ae99-9f65669affd8: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:37:32,450::task::495::TaskManager.Task::(_debug) Task b3d8503f-7f7f-4191-ae99-9f65669affd8: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:37:32,451::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:37:32,452::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:37:32,452::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:37:32,453::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:37:32,453::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:37:32,454::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:37:32,454::task::495::TaskManager.Task::(_debug) Task b3d8503f-7f7f-4191-ae99-9f65669affd8: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:37:32,455::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:37:32,456::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:37:32,456::task::495::TaskManager.Task::(_debug) Task 1381adb3-0b93-42ef-8d72-f2c0255c6f5f: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:37:32,457::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`5962d5ab-4214-4504-b2b4-ba8ddfb47350`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:37:32,457::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:37:32,458::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:37:32,458::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`5962d5ab-4214-4504-b2b4-ba8ddfb47350`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:37:32,459::task::495::TaskManager.Task::(_debug) Task 1381adb3-0b93-42ef-8d72-f2c0255c6f5f: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:37:32,460::task::495::TaskManager.Task::(_debug) Task 1381adb3-0b93-42ef-8d72-f2c0255c6f5f: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:37:32,461::task::495::TaskManager.Task::(_debug) Task 1381adb3-0b93-42ef-8d72-f2c0255c6f5f: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:37:32,461::task::495::TaskManager.Task::(_debug) Task 1381adb3-0b93-42ef-8d72-f2c0255c6f5f: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:37:32,462::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:37:32,462::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:37:32,462::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:37:32,463::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:37:32,464::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:37:32,464::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:37:32,465::task::495::TaskManager.Task::(_debug) Task 1381adb3-0b93-42ef-8d72-f2c0255c6f5f: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:37:32,465::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:37:32,466::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:37:32,467::task::495::TaskManager.Task::(_debug) Task e729767f-9d61-4fd8-98c3-13cd0a44bafd: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:37:32,468::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`7308e1da-dd1d-4130-a479-5c7486f9dce0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:37:32,469::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:37:32,470::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:37:32,471::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`7308e1da-dd1d-4130-a479-5c7486f9dce0`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:37:32,474::task::495::TaskManager.Task::(_debug) Task e729767f-9d61-4fd8-98c3-13cd0a44bafd: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:37:32,475::task::495::TaskManager.Task::(_debug) Task e729767f-9d61-4fd8-98c3-13cd0a44bafd: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:37:32,476::task::495::TaskManager.Task::(_debug) Task e729767f-9d61-4fd8-98c3-13cd0a44bafd: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:37:32,477::task::495::TaskManager.Task::(_debug) Task e729767f-9d61-4fd8-98c3-13cd0a44bafd: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:37:32,478::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:37:32,479::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:37:32,480::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:37:32,482::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:37:32,483::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:37:32,485::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:37:32,487::task::495::TaskManager.Task::(_debug) Task e729767f-9d61-4fd8-98c3-13cd0a44bafd: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:37:32,487::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:37:33,997::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:37:34,396::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0629641 s, 16.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:37:36,406::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:37:39,378::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0467262 s, 21.9 MB/s\n'; <rc> = 0
>Thread-15914677::INFO::2013-11-20 03:37:40,126::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914677::DEBUG::2013-11-20 03:37:40,127::task::495::TaskManager.Task::(_debug) Task 97856968-fcde-41f3-b7ae-69970386484b: moving from state init -> state preparing
>Thread-15914677::DEBUG::2013-11-20 03:37:40,127::task::495::TaskManager.Task::(_debug) Task 97856968-fcde-41f3-b7ae-69970386484b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000234127044678', 'lastCheck': 1384918650.4769681, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.000960111618042', 'lastCheck': 1384918655.4306259, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000908136367798', 'lastCheck': 1384918650.477262, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000332117080688', 'lastCheck': 1384918654.448143, 'valid': True, 'code': 0}}
>Thread-15914677::DEBUG::2013-11-20 03:37:40,183::task::495::TaskManager.Task::(_debug) Task 97856968-fcde-41f3-b7ae-69970386484b: moving from state preparing -> state finished
>Thread-15914677::DEBUG::2013-11-20 03:37:40,184::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914677::DEBUG::2013-11-20 03:37:40,184::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914677::DEBUG::2013-11-20 03:37:40,185::task::495::TaskManager.Task::(_debug) Task 97856968-fcde-41f3-b7ae-69970386484b: ref 0 aborting False
>Thread-15914677::INFO::2013-11-20 03:37:40,185::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000234127044678', 'lastCheck': 1384918650.4769681, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.000960111618042', 'lastCheck': 1384918655.4306259, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000908136367798', 'lastCheck': 1384918650.477262, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000332117080688', 'lastCheck': 1384918654.448143, 'valid': True, 'code': 0}}
>Thread-15914679::DEBUG::2013-11-20 03:37:41,254::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914679::INFO::2013-11-20 03:37:41,255::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914679::DEBUG::2013-11-20 03:37:41,256::task::495::TaskManager.Task::(_debug) Task aee4e481-04b9-41ea-97fc-5ecd4d7bae4e: moving from state init -> state preparing
>Thread-15914679::DEBUG::2013-11-20 03:37:41,256::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914679::DEBUG::2013-11-20 03:37:41,257::task::495::TaskManager.Task::(_debug) Task aee4e481-04b9-41ea-97fc-5ecd4d7bae4e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914679::DEBUG::2013-11-20 03:37:41,258::task::495::TaskManager.Task::(_debug) Task aee4e481-04b9-41ea-97fc-5ecd4d7bae4e: moving from state preparing -> state finished
>Thread-15914679::DEBUG::2013-11-20 03:37:41,258::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914679::DEBUG::2013-11-20 03:37:41,259::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914679::DEBUG::2013-11-20 03:37:41,259::task::495::TaskManager.Task::(_debug) Task aee4e481-04b9-41ea-97fc-5ecd4d7bae4e: ref 0 aborting False
>Thread-15914679::INFO::2013-11-20 03:37:41,260::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914680::DEBUG::2013-11-20 03:37:41,268::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914680::INFO::2013-11-20 03:37:41,269::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914680::DEBUG::2013-11-20 03:37:41,270::task::495::TaskManager.Task::(_debug) Task a1e6a3b9-0478-4d07-ad24-b30f72ae5e1b: moving from state init -> state preparing
>Thread-15914680::DEBUG::2013-11-20 03:37:41,276::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`21262ab0-2d5a-46d2-a618-37f288b1f950`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914680::DEBUG::2013-11-20 03:37:41,277::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914680::DEBUG::2013-11-20 03:37:41,278::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914680::DEBUG::2013-11-20 03:37:41,278::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`21262ab0-2d5a-46d2-a618-37f288b1f950`::Granted request
>Thread-15914680::DEBUG::2013-11-20 03:37:41,279::task::495::TaskManager.Task::(_debug) Task a1e6a3b9-0478-4d07-ad24-b30f72ae5e1b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914680::DEBUG::2013-11-20 03:37:41,279::task::495::TaskManager.Task::(_debug) Task a1e6a3b9-0478-4d07-ad24-b30f72ae5e1b: ref 1 aborting False
>Thread-15914680::WARNING::2013-11-20 03:37:41,281::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914680::DEBUG::2013-11-20 03:37:41,287::task::495::TaskManager.Task::(_debug) Task a1e6a3b9-0478-4d07-ad24-b30f72ae5e1b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914680::DEBUG::2013-11-20 03:37:41,288::task::495::TaskManager.Task::(_debug) Task a1e6a3b9-0478-4d07-ad24-b30f72ae5e1b: moving from state preparing -> state finished
>Thread-15914680::DEBUG::2013-11-20 03:37:41,289::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914680::DEBUG::2013-11-20 03:37:41,289::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914680::DEBUG::2013-11-20 03:37:41,290::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914680::DEBUG::2013-11-20 03:37:41,290::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914680::DEBUG::2013-11-20 03:37:41,291::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914680::DEBUG::2013-11-20 03:37:41,291::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914680::DEBUG::2013-11-20 03:37:41,292::task::495::TaskManager.Task::(_debug) Task a1e6a3b9-0478-4d07-ad24-b30f72ae5e1b: ref 0 aborting False
>Thread-15914680::INFO::2013-11-20 03:37:41,292::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:37:41,388::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:37:43,507::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.048719 s, 21.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:37:45,517::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 03:37:46,832::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:37:49,517::task::495::TaskManager.Task::(_debug) Task 12c492b5-0277-47f1-b735-fe824ca55e12: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:37:49,569::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`edf08044-816b-444b-8162-dd003adbd497`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:37:49,586::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:37:49,589::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:37:49,589::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`edf08044-816b-444b-8162-dd003adbd497`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:37:49,591::task::495::TaskManager.Task::(_debug) Task 12c492b5-0277-47f1-b735-fe824ca55e12: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:37:49,591::task::495::TaskManager.Task::(_debug) Task 12c492b5-0277-47f1-b735-fe824ca55e12: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:37:49,592::task::495::TaskManager.Task::(_debug) Task 12c492b5-0277-47f1-b735-fe824ca55e12: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:37:49,593::task::495::TaskManager.Task::(_debug) Task 12c492b5-0277-47f1-b735-fe824ca55e12: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:37:49,593::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:37:49,594::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:37:49,595::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:37:49,596::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:37:49,597::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:37:49,597::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:37:49,598::task::495::TaskManager.Task::(_debug) Task 12c492b5-0277-47f1-b735-fe824ca55e12: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:37:49,598::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:37:49,763::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0584681 s, 17.5 MB/s\n'; <rc> = 0
>Thread-15914683::DEBUG::2013-11-20 03:37:51,712::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914683::INFO::2013-11-20 03:37:51,712::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914683::DEBUG::2013-11-20 03:37:51,713::task::495::TaskManager.Task::(_debug) Task 0ab6d987-1007-41ed-aac9-d0573aee3fab: moving from state init -> state preparing
>Thread-15914683::DEBUG::2013-11-20 03:37:51,713::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914683::DEBUG::2013-11-20 03:37:51,714::task::495::TaskManager.Task::(_debug) Task 0ab6d987-1007-41ed-aac9-d0573aee3fab: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914683::DEBUG::2013-11-20 03:37:51,714::task::495::TaskManager.Task::(_debug) Task 0ab6d987-1007-41ed-aac9-d0573aee3fab: moving from state preparing -> state finished
>Thread-15914683::DEBUG::2013-11-20 03:37:51,714::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914683::DEBUG::2013-11-20 03:37:51,715::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914683::DEBUG::2013-11-20 03:37:51,715::task::495::TaskManager.Task::(_debug) Task 0ab6d987-1007-41ed-aac9-d0573aee3fab: ref 0 aborting False
>Thread-15914683::INFO::2013-11-20 03:37:51,715::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914684::DEBUG::2013-11-20 03:37:51,722::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914684::INFO::2013-11-20 03:37:51,723::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914684::DEBUG::2013-11-20 03:37:51,723::task::495::TaskManager.Task::(_debug) Task e5d8a42b-543e-40fc-9739-32136decf7c0: moving from state init -> state preparing
>Thread-15914684::DEBUG::2013-11-20 03:37:51,724::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`fc6b2417-787c-4309-8b88-d09897c322a6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914684::DEBUG::2013-11-20 03:37:51,725::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914684::DEBUG::2013-11-20 03:37:51,725::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914684::DEBUG::2013-11-20 03:37:51,726::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`fc6b2417-787c-4309-8b88-d09897c322a6`::Granted request
>Thread-15914684::DEBUG::2013-11-20 03:37:51,730::task::495::TaskManager.Task::(_debug) Task e5d8a42b-543e-40fc-9739-32136decf7c0: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914684::DEBUG::2013-11-20 03:37:51,733::task::495::TaskManager.Task::(_debug) Task e5d8a42b-543e-40fc-9739-32136decf7c0: ref 1 aborting False
>Thread-15914684::WARNING::2013-11-20 03:37:51,735::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914684::DEBUG::2013-11-20 03:37:51,736::task::495::TaskManager.Task::(_debug) Task e5d8a42b-543e-40fc-9739-32136decf7c0: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914684::DEBUG::2013-11-20 03:37:51,736::task::495::TaskManager.Task::(_debug) Task e5d8a42b-543e-40fc-9739-32136decf7c0: moving from state preparing -> state finished
>Thread-15914684::DEBUG::2013-11-20 03:37:51,737::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914684::DEBUG::2013-11-20 03:37:51,737::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914684::DEBUG::2013-11-20 03:37:51,738::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914684::DEBUG::2013-11-20 03:37:51,738::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914684::DEBUG::2013-11-20 03:37:51,738::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914684::DEBUG::2013-11-20 03:37:51,739::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914684::DEBUG::2013-11-20 03:37:51,739::task::495::TaskManager.Task::(_debug) Task e5d8a42b-543e-40fc-9739-32136decf7c0: ref 0 aborting False
>Thread-15914684::INFO::2013-11-20 03:37:51,740::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:37:51,773::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914685::DEBUG::2013-11-20 03:37:52,053::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Thread-15914685::DEBUG::2013-11-20 03:37:54,245::clientIF::1219::vds::(fenceNode) rc 0 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = On
>Done
> err 
>Dummy-15912651::DEBUG::2013-11-20 03:37:54,481::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0544322 s, 18.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:37:56,492::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:37:56,908::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0519747 s, 19.7 MB/s\n'; <rc> = 0
>Thread-15914688::INFO::2013-11-20 03:37:58,689::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914688::DEBUG::2013-11-20 03:37:58,690::task::495::TaskManager.Task::(_debug) Task 0e035cda-4ebe-4bbd-805c-44f6edf2345e: moving from state init -> state preparing
>Thread-15914688::DEBUG::2013-11-20 03:37:58,703::task::495::TaskManager.Task::(_debug) Task 0e035cda-4ebe-4bbd-805c-44f6edf2345e: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00072193145752', 'lastCheck': 1384918672.5010259, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00947713851929', 'lastCheck': 1384918677.6737339, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00101804733276', 'lastCheck': 1384918672.5023389, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00279402732849', 'lastCheck': 1384918676.6652191, 'valid': True, 'code': 0}}
>Thread-15914688::DEBUG::2013-11-20 03:37:58,704::task::495::TaskManager.Task::(_debug) Task 0e035cda-4ebe-4bbd-805c-44f6edf2345e: moving from state preparing -> state finished
>Thread-15914688::DEBUG::2013-11-20 03:37:58,704::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914688::DEBUG::2013-11-20 03:37:58,704::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914688::DEBUG::2013-11-20 03:37:58,705::task::495::TaskManager.Task::(_debug) Task 0e035cda-4ebe-4bbd-805c-44f6edf2345e: ref 0 aborting False
>Thread-15914688::INFO::2013-11-20 03:37:58,705::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00072193145752', 'lastCheck': 1384918672.5010259, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00947713851929', 'lastCheck': 1384918677.6737339, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00101804733276', 'lastCheck': 1384918672.5023389, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00279402732849', 'lastCheck': 1384918676.6652191, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:37:58,959::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914690::DEBUG::2013-11-20 03:37:59,732::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Dummy-15912651::DEBUG::2013-11-20 03:37:59,965::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.563991 s, 1.8 MB/s\n'; <rc> = 0
>Thread-15914690::DEBUG::2013-11-20 03:38:00,481::clientIF::1219::vds::(fenceNode) rc 0 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = On
>Done
> err 
>Thread-15914691::DEBUG::2013-11-20 03:38:00,489::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Thread-15914691::DEBUG::2013-11-20 03:38:00,973::clientIF::1219::vds::(fenceNode) rc 0 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = On
>Done
> err 
>Dummy-15912651::DEBUG::2013-11-20 03:38:01,975::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914693::DEBUG::2013-11-20 03:38:02,251::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914693::INFO::2013-11-20 03:38:02,254::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914693::DEBUG::2013-11-20 03:38:02,254::task::495::TaskManager.Task::(_debug) Task 02a7aea7-1adc-42bd-b821-3fde9e145ef5: moving from state init -> state preparing
>Thread-15914693::DEBUG::2013-11-20 03:38:02,255::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914693::DEBUG::2013-11-20 03:38:02,256::task::495::TaskManager.Task::(_debug) Task 02a7aea7-1adc-42bd-b821-3fde9e145ef5: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914693::DEBUG::2013-11-20 03:38:02,256::task::495::TaskManager.Task::(_debug) Task 02a7aea7-1adc-42bd-b821-3fde9e145ef5: moving from state preparing -> state finished
>Thread-15914693::DEBUG::2013-11-20 03:38:02,257::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914693::DEBUG::2013-11-20 03:38:02,257::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914693::DEBUG::2013-11-20 03:38:02,258::task::495::TaskManager.Task::(_debug) Task 02a7aea7-1adc-42bd-b821-3fde9e145ef5: ref 0 aborting False
>Thread-15914693::INFO::2013-11-20 03:38:02,258::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914694::DEBUG::2013-11-20 03:38:02,273::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914694::INFO::2013-11-20 03:38:02,274::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914694::DEBUG::2013-11-20 03:38:02,274::task::495::TaskManager.Task::(_debug) Task eee8f5e0-ae26-466c-925d-a7549eb7029c: moving from state init -> state preparing
>Thread-15914694::DEBUG::2013-11-20 03:38:02,276::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`22499dd0-9abb-4fb7-8889-a7f95c4eae84`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914694::DEBUG::2013-11-20 03:38:02,276::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914694::DEBUG::2013-11-20 03:38:02,277::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914694::DEBUG::2013-11-20 03:38:02,277::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`22499dd0-9abb-4fb7-8889-a7f95c4eae84`::Granted request
>Thread-15914694::DEBUG::2013-11-20 03:38:02,278::task::495::TaskManager.Task::(_debug) Task eee8f5e0-ae26-466c-925d-a7549eb7029c: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914694::DEBUG::2013-11-20 03:38:02,283::task::495::TaskManager.Task::(_debug) Task eee8f5e0-ae26-466c-925d-a7549eb7029c: ref 1 aborting False
>Thread-15914694::WARNING::2013-11-20 03:38:02,284::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914694::DEBUG::2013-11-20 03:38:02,285::task::495::TaskManager.Task::(_debug) Task eee8f5e0-ae26-466c-925d-a7549eb7029c: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914694::DEBUG::2013-11-20 03:38:02,286::task::495::TaskManager.Task::(_debug) Task eee8f5e0-ae26-466c-925d-a7549eb7029c: moving from state preparing -> state finished
>Thread-15914694::DEBUG::2013-11-20 03:38:02,286::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914694::DEBUG::2013-11-20 03:38:02,287::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914694::DEBUG::2013-11-20 03:38:02,287::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914694::DEBUG::2013-11-20 03:38:02,288::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914694::DEBUG::2013-11-20 03:38:02,288::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914694::DEBUG::2013-11-20 03:38:02,288::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914694::DEBUG::2013-11-20 03:38:02,289::task::495::TaskManager.Task::(_debug) Task eee8f5e0-ae26-466c-925d-a7549eb7029c: ref 0 aborting False
>Thread-15914694::INFO::2013-11-20 03:38:02,289::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:38:05,209::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 2.26363 s, 452 kB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 03:38:07,062::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:38:07,063::task::495::TaskManager.Task::(_debug) Task d9548475-7c36-4439-9b1e-4b84ef37e3a6: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:38:07,064::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ce556582-d87c-43ed-bfc4-c458fc0bf8ec`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:38:07,064::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:38:07,065::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:38:07,066::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ce556582-d87c-43ed-bfc4-c458fc0bf8ec`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:38:07,066::task::495::TaskManager.Task::(_debug) Task d9548475-7c36-4439-9b1e-4b84ef37e3a6: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:38:07,067::task::495::TaskManager.Task::(_debug) Task d9548475-7c36-4439-9b1e-4b84ef37e3a6: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:38:07,067::task::495::TaskManager.Task::(_debug) Task d9548475-7c36-4439-9b1e-4b84ef37e3a6: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:38:07,068::task::495::TaskManager.Task::(_debug) Task d9548475-7c36-4439-9b1e-4b84ef37e3a6: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:38:07,068::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:38:07,068::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:38:07,069::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:38:07,069::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:38:07,070::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:38:07,070::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:38:07,071::task::495::TaskManager.Task::(_debug) Task d9548475-7c36-4439-9b1e-4b84ef37e3a6: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:38:07,071::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:38:07,219::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:38:07,634::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0474537 s, 21.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:38:09,644::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:38:10,565::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0498078 s, 20.6 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 03:38:11,028::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:38:11,029::task::495::TaskManager.Task::(_debug) Task 97aa7a64-2a88-4680-b7c8-a220e8912a93: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:38:11,030::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e9c6f406-75e7-400c-89f8-5477845e3f37`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:38:11,030::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:38:11,031::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:38:11,032::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e9c6f406-75e7-400c-89f8-5477845e3f37`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:38:11,032::task::495::TaskManager.Task::(_debug) Task 97aa7a64-2a88-4680-b7c8-a220e8912a93: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:38:11,033::task::495::TaskManager.Task::(_debug) Task 97aa7a64-2a88-4680-b7c8-a220e8912a93: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:38:11,034::task::495::TaskManager.Task::(_debug) Task 97aa7a64-2a88-4680-b7c8-a220e8912a93: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:38:11,034::task::495::TaskManager.Task::(_debug) Task 97aa7a64-2a88-4680-b7c8-a220e8912a93: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:38:11,034::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:38:11,035::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:38:11,035::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:38:11,036::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:38:11,036::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:38:11,037::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:38:11,037::task::495::TaskManager.Task::(_debug) Task 97aa7a64-2a88-4680-b7c8-a220e8912a93: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:38:11,038::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:38:12,575::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914698::DEBUG::2013-11-20 03:38:12,794::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914698::INFO::2013-11-20 03:38:12,796::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914698::DEBUG::2013-11-20 03:38:12,797::task::495::TaskManager.Task::(_debug) Task da789efb-02d0-4125-9953-79ac1de9705c: moving from state init -> state preparing
>Thread-15914698::DEBUG::2013-11-20 03:38:12,798::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914698::DEBUG::2013-11-20 03:38:12,798::task::495::TaskManager.Task::(_debug) Task da789efb-02d0-4125-9953-79ac1de9705c: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914698::DEBUG::2013-11-20 03:38:12,799::task::495::TaskManager.Task::(_debug) Task da789efb-02d0-4125-9953-79ac1de9705c: moving from state preparing -> state finished
>Thread-15914698::DEBUG::2013-11-20 03:38:12,799::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914698::DEBUG::2013-11-20 03:38:12,800::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914698::DEBUG::2013-11-20 03:38:12,800::task::495::TaskManager.Task::(_debug) Task da789efb-02d0-4125-9953-79ac1de9705c: ref 0 aborting False
>Thread-15914698::INFO::2013-11-20 03:38:12,801::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914699::DEBUG::2013-11-20 03:38:12,807::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914699::INFO::2013-11-20 03:38:12,808::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914699::DEBUG::2013-11-20 03:38:12,808::task::495::TaskManager.Task::(_debug) Task dfb99a07-6d7f-4bfe-8977-62a2c675cd74: moving from state init -> state preparing
>Thread-15914699::DEBUG::2013-11-20 03:38:12,810::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`13c44c6c-6f6d-4025-89a6-eef2b1538f76`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914699::DEBUG::2013-11-20 03:38:12,810::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914699::DEBUG::2013-11-20 03:38:12,811::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914699::DEBUG::2013-11-20 03:38:12,812::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`13c44c6c-6f6d-4025-89a6-eef2b1538f76`::Granted request
>Thread-15914699::DEBUG::2013-11-20 03:38:12,813::task::495::TaskManager.Task::(_debug) Task dfb99a07-6d7f-4bfe-8977-62a2c675cd74: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914699::DEBUG::2013-11-20 03:38:12,813::task::495::TaskManager.Task::(_debug) Task dfb99a07-6d7f-4bfe-8977-62a2c675cd74: ref 1 aborting False
>Thread-15914699::WARNING::2013-11-20 03:38:12,815::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914699::DEBUG::2013-11-20 03:38:12,816::task::495::TaskManager.Task::(_debug) Task dfb99a07-6d7f-4bfe-8977-62a2c675cd74: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914699::DEBUG::2013-11-20 03:38:12,816::task::495::TaskManager.Task::(_debug) Task dfb99a07-6d7f-4bfe-8977-62a2c675cd74: moving from state preparing -> state finished
>Thread-15914699::DEBUG::2013-11-20 03:38:12,817::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914699::DEBUG::2013-11-20 03:38:12,817::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914699::DEBUG::2013-11-20 03:38:12,818::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914699::DEBUG::2013-11-20 03:38:12,818::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914699::DEBUG::2013-11-20 03:38:12,819::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914699::DEBUG::2013-11-20 03:38:12,819::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914699::DEBUG::2013-11-20 03:38:12,819::task::495::TaskManager.Task::(_debug) Task dfb99a07-6d7f-4bfe-8977-62a2c675cd74: ref 0 aborting False
>Thread-15914699::INFO::2013-11-20 03:38:12,820::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:38:13,036::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0485796 s, 21.1 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 03:38:13,147::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:38:13,148::task::495::TaskManager.Task::(_debug) Task 126e657f-156b-4b85-9897-28cf2e09e43c: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:38:13,149::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9b17a1e2-a8e9-45f9-8e40-9fdc47f37fe0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:38:13,149::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:38:13,150::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:38:13,150::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9b17a1e2-a8e9-45f9-8e40-9fdc47f37fe0`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:38:13,151::task::495::TaskManager.Task::(_debug) Task 126e657f-156b-4b85-9897-28cf2e09e43c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:38:13,151::task::495::TaskManager.Task::(_debug) Task 126e657f-156b-4b85-9897-28cf2e09e43c: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:38:13,151::task::495::TaskManager.Task::(_debug) Task 126e657f-156b-4b85-9897-28cf2e09e43c: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:38:13,152::task::495::TaskManager.Task::(_debug) Task 126e657f-156b-4b85-9897-28cf2e09e43c: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:38:13,152::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:38:13,152::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:38:13,153::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:38:13,153::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:38:13,154::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:38:13,154::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:38:13,154::task::495::TaskManager.Task::(_debug) Task 126e657f-156b-4b85-9897-28cf2e09e43c: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:38:13,155::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15914700::INFO::2013-11-20 03:38:14,287::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914700::DEBUG::2013-11-20 03:38:14,287::task::495::TaskManager.Task::(_debug) Task 41002617-db92-4684-9272-4f16f9396d3d: moving from state init -> state preparing
>Thread-15914700::DEBUG::2013-11-20 03:38:14,288::task::495::TaskManager.Task::(_debug) Task 41002617-db92-4684-9272-4f16f9396d3d: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000248908996582', 'lastCheck': 1384918694.0153489, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.000996112823486', 'lastCheck': 1384918689.0189869, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00234699249268', 'lastCheck': 1384918685.1095481, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000396966934204', 'lastCheck': 1384918688.007751, 'valid': True, 'code': 0}}
>Thread-15914700::DEBUG::2013-11-20 03:38:14,289::task::495::TaskManager.Task::(_debug) Task 41002617-db92-4684-9272-4f16f9396d3d: moving from state preparing -> state finished
>Thread-15914700::DEBUG::2013-11-20 03:38:14,289::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914700::DEBUG::2013-11-20 03:38:14,289::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914700::DEBUG::2013-11-20 03:38:14,290::task::495::TaskManager.Task::(_debug) Task 41002617-db92-4684-9272-4f16f9396d3d: ref 0 aborting False
>Thread-15914700::INFO::2013-11-20 03:38:14,290::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000248908996582', 'lastCheck': 1384918694.0153489, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.000996112823486', 'lastCheck': 1384918689.0189869, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00234699249268', 'lastCheck': 1384918685.1095481, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000396966934204', 'lastCheck': 1384918688.007751, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:38:15,047::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:38:15,560::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0727395 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:38:17,573::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:38:18,775::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0482866 s, 21.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:38:20,786::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:38:22,561::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0631438 s, 16.2 MB/s\n'; <rc> = 0
>Thread-15914435::INFO::2013-11-20 03:38:23,297::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 03:38:23,298::task::495::TaskManager.Task::(_debug) Task bc80af7a-ee68-4d42-9058-db6c05b3da83: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 03:38:23,300::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f473ba6e-59eb-4bf8-a23c-b6834065bcf1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 03:38:23,301::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 03:38:23,302::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 03:38:23,303::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f473ba6e-59eb-4bf8-a23c-b6834065bcf1`::Granted request
>Thread-15914435::DEBUG::2013-11-20 03:38:23,304::task::495::TaskManager.Task::(_debug) Task bc80af7a-ee68-4d42-9058-db6c05b3da83: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 03:38:23,305::task::495::TaskManager.Task::(_debug) Task bc80af7a-ee68-4d42-9058-db6c05b3da83: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 03:38:23,307::task::495::TaskManager.Task::(_debug) Task bc80af7a-ee68-4d42-9058-db6c05b3da83: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914704::DEBUG::2013-11-20 03:38:23,308::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914435::DEBUG::2013-11-20 03:38:23,309::task::495::TaskManager.Task::(_debug) Task bc80af7a-ee68-4d42-9058-db6c05b3da83: moving from state preparing -> state finished
>Thread-15914704::INFO::2013-11-20 03:38:23,310::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914435::DEBUG::2013-11-20 03:38:23,310::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914704::DEBUG::2013-11-20 03:38:23,311::task::495::TaskManager.Task::(_debug) Task e280666f-8b11-4129-aed5-20edc6dcc35a: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 03:38:23,312::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914704::DEBUG::2013-11-20 03:38:23,312::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914435::DEBUG::2013-11-20 03:38:23,314::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914704::DEBUG::2013-11-20 03:38:23,314::task::495::TaskManager.Task::(_debug) Task e280666f-8b11-4129-aed5-20edc6dcc35a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914704::DEBUG::2013-11-20 03:38:23,315::task::495::TaskManager.Task::(_debug) Task e280666f-8b11-4129-aed5-20edc6dcc35a: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 03:38:23,316::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914704::DEBUG::2013-11-20 03:38:23,316::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914435::DEBUG::2013-11-20 03:38:23,317::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914704::DEBUG::2013-11-20 03:38:23,317::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 03:38:23,318::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914704::DEBUG::2013-11-20 03:38:23,319::task::495::TaskManager.Task::(_debug) Task e280666f-8b11-4129-aed5-20edc6dcc35a: ref 0 aborting False
>Thread-15914435::DEBUG::2013-11-20 03:38:23,320::task::495::TaskManager.Task::(_debug) Task bc80af7a-ee68-4d42-9058-db6c05b3da83: ref 0 aborting False
>Thread-15914704::INFO::2013-11-20 03:38:23,320::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914435::INFO::2013-11-20 03:38:23,321::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914705::DEBUG::2013-11-20 03:38:23,329::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914705::INFO::2013-11-20 03:38:23,330::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914705::DEBUG::2013-11-20 03:38:23,330::task::495::TaskManager.Task::(_debug) Task c6078c9a-9a22-408f-b1f9-1a1c3f23636a: moving from state init -> state preparing
>Thread-15914705::DEBUG::2013-11-20 03:38:23,331::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2af81856-186e-4dee-bd46-d6dca27c12a6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914705::DEBUG::2013-11-20 03:38:23,332::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914705::DEBUG::2013-11-20 03:38:23,332::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914705::DEBUG::2013-11-20 03:38:23,332::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2af81856-186e-4dee-bd46-d6dca27c12a6`::Granted request
>Thread-15914705::DEBUG::2013-11-20 03:38:23,333::task::495::TaskManager.Task::(_debug) Task c6078c9a-9a22-408f-b1f9-1a1c3f23636a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914705::DEBUG::2013-11-20 03:38:23,333::task::495::TaskManager.Task::(_debug) Task c6078c9a-9a22-408f-b1f9-1a1c3f23636a: ref 1 aborting False
>Thread-15914705::WARNING::2013-11-20 03:38:23,335::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914705::DEBUG::2013-11-20 03:38:23,335::task::495::TaskManager.Task::(_debug) Task c6078c9a-9a22-408f-b1f9-1a1c3f23636a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914705::DEBUG::2013-11-20 03:38:23,336::task::495::TaskManager.Task::(_debug) Task c6078c9a-9a22-408f-b1f9-1a1c3f23636a: moving from state preparing -> state finished
>Thread-15914705::DEBUG::2013-11-20 03:38:23,336::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914705::DEBUG::2013-11-20 03:38:23,337::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914705::DEBUG::2013-11-20 03:38:23,337::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914705::DEBUG::2013-11-20 03:38:23,338::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914705::DEBUG::2013-11-20 03:38:23,338::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914705::DEBUG::2013-11-20 03:38:23,338::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914705::DEBUG::2013-11-20 03:38:23,339::task::495::TaskManager.Task::(_debug) Task c6078c9a-9a22-408f-b1f9-1a1c3f23636a: ref 0 aborting False
>Thread-15914705::INFO::2013-11-20 03:38:23,339::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:38:24,570::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:38:24,976::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0468105 s, 21.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:38:26,984::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 03:38:27,143::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 03:38:27,145::task::495::TaskManager.Task::(_debug) Task 108e17bb-a003-478a-b1da-ebcba5176179: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 03:38:27,147::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1ef6eb4a-f5f0-4e4d-a6c1-e04ddaf4438a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 03:38:27,147::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 03:38:27,148::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 03:38:27,148::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1ef6eb4a-f5f0-4e4d-a6c1-e04ddaf4438a`::Granted request
>Thread-15914453::DEBUG::2013-11-20 03:38:27,149::task::495::TaskManager.Task::(_debug) Task 108e17bb-a003-478a-b1da-ebcba5176179: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 03:38:27,150::task::495::TaskManager.Task::(_debug) Task 108e17bb-a003-478a-b1da-ebcba5176179: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 03:38:27,150::task::495::TaskManager.Task::(_debug) Task 108e17bb-a003-478a-b1da-ebcba5176179: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 03:38:27,151::task::495::TaskManager.Task::(_debug) Task 108e17bb-a003-478a-b1da-ebcba5176179: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 03:38:27,151::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 03:38:27,152::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 03:38:27,152::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 03:38:27,152::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 03:38:27,153::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 03:38:27,153::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 03:38:27,154::task::495::TaskManager.Task::(_debug) Task 108e17bb-a003-478a-b1da-ebcba5176179: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 03:38:27,154::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 03:38:27,427::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0710322 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:38:29,448::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:38:29,998::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0867894 s, 11.8 MB/s\n'; <rc> = 0
>Thread-15914708::INFO::2013-11-20 03:38:30,243::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914708::DEBUG::2013-11-20 03:38:30,244::task::495::TaskManager.Task::(_debug) Task 721bc26a-517c-4bda-9fde-4f2ba425f5e3: moving from state init -> state preparing
>Thread-15914708::DEBUG::2013-11-20 03:38:30,245::task::495::TaskManager.Task::(_debug) Task 721bc26a-517c-4bda-9fde-4f2ba425f5e3: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000270128250122', 'lastCheck': 1384918704.0276289, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00129294395447', 'lastCheck': 1384918709.153007, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00109100341797', 'lastCheck': 1384918705.285018, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000608205795288', 'lastCheck': 1384918708.142421, 'valid': True, 'code': 0}}
>Thread-15914708::DEBUG::2013-11-20 03:38:30,246::task::495::TaskManager.Task::(_debug) Task 721bc26a-517c-4bda-9fde-4f2ba425f5e3: moving from state preparing -> state finished
>Thread-15914708::DEBUG::2013-11-20 03:38:30,247::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914708::DEBUG::2013-11-20 03:38:30,247::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914708::DEBUG::2013-11-20 03:38:30,248::task::495::TaskManager.Task::(_debug) Task 721bc26a-517c-4bda-9fde-4f2ba425f5e3: ref 0 aborting False
>Thread-15914708::INFO::2013-11-20 03:38:30,249::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000270128250122', 'lastCheck': 1384918704.0276289, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00129294395447', 'lastCheck': 1384918709.153007, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00109100341797', 'lastCheck': 1384918705.285018, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000608205795288', 'lastCheck': 1384918708.142421, 'valid': True, 'code': 0}}
>Thread-15914519::INFO::2013-11-20 03:38:30,281::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 03:38:30,283::task::495::TaskManager.Task::(_debug) Task 7643b6ab-c4fb-41f6-baab-b993f1ed5da7: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 03:38:30,290::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fbe5869d-12b4-4c6d-8dd8-54f68c56a4d3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 03:38:30,291::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 03:38:30,292::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 03:38:30,292::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fbe5869d-12b4-4c6d-8dd8-54f68c56a4d3`::Granted request
>Thread-15914519::DEBUG::2013-11-20 03:38:30,294::task::495::TaskManager.Task::(_debug) Task 7643b6ab-c4fb-41f6-baab-b993f1ed5da7: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 03:38:30,294::task::495::TaskManager.Task::(_debug) Task 7643b6ab-c4fb-41f6-baab-b993f1ed5da7: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 03:38:30,295::task::495::TaskManager.Task::(_debug) Task 7643b6ab-c4fb-41f6-baab-b993f1ed5da7: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 03:38:30,296::task::495::TaskManager.Task::(_debug) Task 7643b6ab-c4fb-41f6-baab-b993f1ed5da7: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 03:38:30,297::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 03:38:30,298::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 03:38:30,298::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 03:38:30,299::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 03:38:30,300::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 03:38:30,301::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 03:38:30,301::task::495::TaskManager.Task::(_debug) Task 7643b6ab-c4fb-41f6-baab-b993f1ed5da7: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 03:38:30,302::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 03:38:32,013::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:38:32,593::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0868053 s, 11.8 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 03:38:33,461::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:38:33,462::task::495::TaskManager.Task::(_debug) Task b2be4377-b186-4fa1-92d7-4661f07e444e: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:38:33,464::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`07c826c4-40e6-47f9-ad47-b8572f43a1c7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:38:33,465::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:38:33,466::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:38:33,466::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`07c826c4-40e6-47f9-ad47-b8572f43a1c7`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:38:33,468::task::495::TaskManager.Task::(_debug) Task b2be4377-b186-4fa1-92d7-4661f07e444e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:38:33,469::task::495::TaskManager.Task::(_debug) Task b2be4377-b186-4fa1-92d7-4661f07e444e: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:38:33,470::task::495::TaskManager.Task::(_debug) Task b2be4377-b186-4fa1-92d7-4661f07e444e: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:38:33,470::task::495::TaskManager.Task::(_debug) Task b2be4377-b186-4fa1-92d7-4661f07e444e: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:38:33,471::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:38:33,472::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:38:33,473::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:38:33,474::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:38:33,474::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:38:33,475::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:38:33,475::task::495::TaskManager.Task::(_debug) Task b2be4377-b186-4fa1-92d7-4661f07e444e: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:38:33,476::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:38:33,477::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:38:33,477::task::495::TaskManager.Task::(_debug) Task bf1e648f-039d-4220-a93e-3264fb943f0f: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:38:33,478::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`45a0219b-dd21-4565-8b84-dbaa5d84caf3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:38:33,479::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:38:33,480::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:38:33,480::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`45a0219b-dd21-4565-8b84-dbaa5d84caf3`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:38:33,481::task::495::TaskManager.Task::(_debug) Task bf1e648f-039d-4220-a93e-3264fb943f0f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:38:33,482::task::495::TaskManager.Task::(_debug) Task bf1e648f-039d-4220-a93e-3264fb943f0f: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:38:33,482::task::495::TaskManager.Task::(_debug) Task bf1e648f-039d-4220-a93e-3264fb943f0f: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:38:33,483::task::495::TaskManager.Task::(_debug) Task bf1e648f-039d-4220-a93e-3264fb943f0f: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:38:33,483::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:38:33,484::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:38:33,485::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:38:33,486::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:38:33,486::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:38:33,487::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:38:33,487::task::495::TaskManager.Task::(_debug) Task bf1e648f-039d-4220-a93e-3264fb943f0f: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:38:33,488::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15914710::DEBUG::2013-11-20 03:38:33,760::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914710::INFO::2013-11-20 03:38:33,761::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914710::DEBUG::2013-11-20 03:38:33,762::task::495::TaskManager.Task::(_debug) Task d71e556d-e47e-460d-be8c-32b5dfd9362b: moving from state init -> state preparing
>Thread-15914710::DEBUG::2013-11-20 03:38:33,763::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914710::DEBUG::2013-11-20 03:38:33,764::task::495::TaskManager.Task::(_debug) Task d71e556d-e47e-460d-be8c-32b5dfd9362b: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914710::DEBUG::2013-11-20 03:38:33,764::task::495::TaskManager.Task::(_debug) Task d71e556d-e47e-460d-be8c-32b5dfd9362b: moving from state preparing -> state finished
>Thread-15914710::DEBUG::2013-11-20 03:38:33,765::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914710::DEBUG::2013-11-20 03:38:33,766::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914710::DEBUG::2013-11-20 03:38:33,767::task::495::TaskManager.Task::(_debug) Task d71e556d-e47e-460d-be8c-32b5dfd9362b: ref 0 aborting False
>Thread-15914710::INFO::2013-11-20 03:38:33,767::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914711::DEBUG::2013-11-20 03:38:33,782::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914711::INFO::2013-11-20 03:38:33,783::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914711::DEBUG::2013-11-20 03:38:33,784::task::495::TaskManager.Task::(_debug) Task a0e12ebd-6f4b-46c2-a27b-6f0d244454a4: moving from state init -> state preparing
>Thread-15914711::DEBUG::2013-11-20 03:38:33,785::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`558cbd46-a7cc-4f8f-b046-84331b37d342`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914711::DEBUG::2013-11-20 03:38:33,786::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914711::DEBUG::2013-11-20 03:38:33,786::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914711::DEBUG::2013-11-20 03:38:33,787::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`558cbd46-a7cc-4f8f-b046-84331b37d342`::Granted request
>Thread-15914711::DEBUG::2013-11-20 03:38:33,788::task::495::TaskManager.Task::(_debug) Task a0e12ebd-6f4b-46c2-a27b-6f0d244454a4: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914711::DEBUG::2013-11-20 03:38:33,789::task::495::TaskManager.Task::(_debug) Task a0e12ebd-6f4b-46c2-a27b-6f0d244454a4: ref 1 aborting False
>Thread-15914711::WARNING::2013-11-20 03:38:33,791::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914711::DEBUG::2013-11-20 03:38:33,792::task::495::TaskManager.Task::(_debug) Task a0e12ebd-6f4b-46c2-a27b-6f0d244454a4: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914711::DEBUG::2013-11-20 03:38:33,793::task::495::TaskManager.Task::(_debug) Task a0e12ebd-6f4b-46c2-a27b-6f0d244454a4: moving from state preparing -> state finished
>Thread-15914711::DEBUG::2013-11-20 03:38:33,793::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914711::DEBUG::2013-11-20 03:38:33,794::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914711::DEBUG::2013-11-20 03:38:33,794::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914711::DEBUG::2013-11-20 03:38:33,795::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914711::DEBUG::2013-11-20 03:38:33,796::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914711::DEBUG::2013-11-20 03:38:33,796::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914711::DEBUG::2013-11-20 03:38:33,797::task::495::TaskManager.Task::(_debug) Task a0e12ebd-6f4b-46c2-a27b-6f0d244454a4: ref 0 aborting False
>Thread-15914711::INFO::2013-11-20 03:38:33,798::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:38:34,604::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:38:35,469::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0951538 s, 10.8 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 03:38:37,280::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:38:37,281::task::495::TaskManager.Task::(_debug) Task fa7784a6-9b95-4c44-9dce-7b7399b64f08: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:38:37,282::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d1dbcd93-2770-49f0-a59a-2a9803470ec5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:38:37,283::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:38:37,284::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:38:37,285::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d1dbcd93-2770-49f0-a59a-2a9803470ec5`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:38:37,286::task::495::TaskManager.Task::(_debug) Task fa7784a6-9b95-4c44-9dce-7b7399b64f08: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:38:37,287::task::495::TaskManager.Task::(_debug) Task fa7784a6-9b95-4c44-9dce-7b7399b64f08: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:38:37,288::task::495::TaskManager.Task::(_debug) Task fa7784a6-9b95-4c44-9dce-7b7399b64f08: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:38:37,289::task::495::TaskManager.Task::(_debug) Task fa7784a6-9b95-4c44-9dce-7b7399b64f08: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:38:37,290::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:38:37,290::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:38:37,292::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:38:37,293::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:38:37,293::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:38:37,294::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:38:37,295::task::495::TaskManager.Task::(_debug) Task fa7784a6-9b95-4c44-9dce-7b7399b64f08: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:38:37,296::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:38:37,297::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:38:37,298::task::495::TaskManager.Task::(_debug) Task 61cb8d8a-795d-4732-9ec7-7f2a414c4ff8: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:38:37,299::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2b3dc324-fea7-4db9-8855-a6143db04384`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:38:37,300::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:38:37,301::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:38:37,301::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2b3dc324-fea7-4db9-8855-a6143db04384`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:38:37,302::task::495::TaskManager.Task::(_debug) Task 61cb8d8a-795d-4732-9ec7-7f2a414c4ff8: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:38:37,303::task::495::TaskManager.Task::(_debug) Task 61cb8d8a-795d-4732-9ec7-7f2a414c4ff8: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:38:37,304::task::495::TaskManager.Task::(_debug) Task 61cb8d8a-795d-4732-9ec7-7f2a414c4ff8: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:38:37,304::task::495::TaskManager.Task::(_debug) Task 61cb8d8a-795d-4732-9ec7-7f2a414c4ff8: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:38:37,305::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:38:37,305::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:38:37,306::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:38:37,307::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:38:37,307::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:38:37,308::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:38:37,309::task::495::TaskManager.Task::(_debug) Task 61cb8d8a-795d-4732-9ec7-7f2a414c4ff8: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:38:37,309::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:38:37,310::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:38:37,311::task::495::TaskManager.Task::(_debug) Task f1dfd8c4-6fa6-4633-ad05-776620a45f3c: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:38:37,312::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`541a4af3-90fb-4f75-b946-ed760cb10d58`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:38:37,312::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:38:37,313::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:38:37,314::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`541a4af3-90fb-4f75-b946-ed760cb10d58`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:38:37,315::task::495::TaskManager.Task::(_debug) Task f1dfd8c4-6fa6-4633-ad05-776620a45f3c: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:38:37,315::task::495::TaskManager.Task::(_debug) Task f1dfd8c4-6fa6-4633-ad05-776620a45f3c: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:38:37,316::task::495::TaskManager.Task::(_debug) Task f1dfd8c4-6fa6-4633-ad05-776620a45f3c: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:38:37,316::task::495::TaskManager.Task::(_debug) Task f1dfd8c4-6fa6-4633-ad05-776620a45f3c: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:38:37,317::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:38:37,318::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:38:37,318::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:38:37,319::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:38:37,319::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:38:37,320::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:38:37,320::task::495::TaskManager.Task::(_debug) Task f1dfd8c4-6fa6-4633-ad05-776620a45f3c: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:38:37,321::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:38:37,322::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:38:37,322::task::495::TaskManager.Task::(_debug) Task f5798a30-6e69-4911-ac4d-72e83af0e37b: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:38:37,323::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`af0cec48-3c28-4140-a131-dbe7a3d41dde`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:38:37,324::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:38:37,324::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:38:37,325::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`af0cec48-3c28-4140-a131-dbe7a3d41dde`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:38:37,326::task::495::TaskManager.Task::(_debug) Task f5798a30-6e69-4911-ac4d-72e83af0e37b: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:38:37,326::task::495::TaskManager.Task::(_debug) Task f5798a30-6e69-4911-ac4d-72e83af0e37b: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:38:37,327::task::495::TaskManager.Task::(_debug) Task f5798a30-6e69-4911-ac4d-72e83af0e37b: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:38:37,328::task::495::TaskManager.Task::(_debug) Task f5798a30-6e69-4911-ac4d-72e83af0e37b: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:38:37,328::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:38:37,329::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:38:37,330::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:38:37,330::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:38:37,331::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:38:37,331::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:38:37,332::task::495::TaskManager.Task::(_debug) Task f5798a30-6e69-4911-ac4d-72e83af0e37b: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:38:37,332::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:38:37,486::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:38:38,058::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0881193 s, 11.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:38:40,078::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:38:40,618::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0736225 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:38:42,631::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:38:43,216::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.098209 s, 10.4 MB/s\n'; <rc> = 0
>Thread-15914716::DEBUG::2013-11-20 03:38:44,335::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914716::INFO::2013-11-20 03:38:44,342::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914716::DEBUG::2013-11-20 03:38:44,344::task::495::TaskManager.Task::(_debug) Task d2a6611e-6a08-40b9-8eeb-fe5213854e05: moving from state init -> state preparing
>Thread-15914716::DEBUG::2013-11-20 03:38:44,346::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914716::DEBUG::2013-11-20 03:38:44,347::task::495::TaskManager.Task::(_debug) Task d2a6611e-6a08-40b9-8eeb-fe5213854e05: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914716::DEBUG::2013-11-20 03:38:44,350::task::495::TaskManager.Task::(_debug) Task d2a6611e-6a08-40b9-8eeb-fe5213854e05: moving from state preparing -> state finished
>Thread-15914716::DEBUG::2013-11-20 03:38:44,352::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914716::DEBUG::2013-11-20 03:38:44,355::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914716::DEBUG::2013-11-20 03:38:44,358::task::495::TaskManager.Task::(_debug) Task d2a6611e-6a08-40b9-8eeb-fe5213854e05: ref 0 aborting False
>Thread-15914716::INFO::2013-11-20 03:38:44,359::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914717::DEBUG::2013-11-20 03:38:44,368::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914717::INFO::2013-11-20 03:38:44,370::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914717::DEBUG::2013-11-20 03:38:44,372::task::495::TaskManager.Task::(_debug) Task 9680edfd-8e21-41f6-beca-d68ef4dd433b: moving from state init -> state preparing
>Thread-15914717::DEBUG::2013-11-20 03:38:44,375::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d2bcbbae-4363-49de-a23b-be71fa143bac`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914717::DEBUG::2013-11-20 03:38:44,378::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914717::DEBUG::2013-11-20 03:38:44,383::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914717::DEBUG::2013-11-20 03:38:44,384::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d2bcbbae-4363-49de-a23b-be71fa143bac`::Granted request
>Thread-15914717::DEBUG::2013-11-20 03:38:44,388::task::495::TaskManager.Task::(_debug) Task 9680edfd-8e21-41f6-beca-d68ef4dd433b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914717::DEBUG::2013-11-20 03:38:44,391::task::495::TaskManager.Task::(_debug) Task 9680edfd-8e21-41f6-beca-d68ef4dd433b: ref 1 aborting False
>Thread-15914717::WARNING::2013-11-20 03:38:44,399::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914717::DEBUG::2013-11-20 03:38:44,413::task::495::TaskManager.Task::(_debug) Task 9680edfd-8e21-41f6-beca-d68ef4dd433b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914717::DEBUG::2013-11-20 03:38:44,417::task::495::TaskManager.Task::(_debug) Task 9680edfd-8e21-41f6-beca-d68ef4dd433b: moving from state preparing -> state finished
>Thread-15914717::DEBUG::2013-11-20 03:38:44,420::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914717::DEBUG::2013-11-20 03:38:44,427::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914717::DEBUG::2013-11-20 03:38:44,428::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914717::DEBUG::2013-11-20 03:38:44,429::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914717::DEBUG::2013-11-20 03:38:44,430::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914717::DEBUG::2013-11-20 03:38:44,430::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914717::DEBUG::2013-11-20 03:38:44,431::task::495::TaskManager.Task::(_debug) Task 9680edfd-8e21-41f6-beca-d68ef4dd433b: ref 0 aborting False
>Thread-15914717::INFO::2013-11-20 03:38:44,432::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:38:45,232::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:38:45,795::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0845643 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15914718::INFO::2013-11-20 03:38:46,124::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914718::DEBUG::2013-11-20 03:38:46,124::task::495::TaskManager.Task::(_debug) Task 563e9e8d-fe7b-45b8-a1bd-1f6fdc334862: moving from state init -> state preparing
>Thread-15914718::DEBUG::2013-11-20 03:38:46,125::task::495::TaskManager.Task::(_debug) Task 563e9e8d-fe7b-45b8-a1bd-1f6fdc334862: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000436782836914', 'lastCheck': 1384918724.288604, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0145769119263', 'lastCheck': 1384918719.2674069, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0231680870056', 'lastCheck': 1384918725.481946, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000478982925415', 'lastCheck': 1384918718.2482181, 'valid': True, 'code': 0}}
>Thread-15914718::DEBUG::2013-11-20 03:38:46,131::task::495::TaskManager.Task::(_debug) Task 563e9e8d-fe7b-45b8-a1bd-1f6fdc334862: moving from state preparing -> state finished
>Thread-15914718::DEBUG::2013-11-20 03:38:46,132::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914718::DEBUG::2013-11-20 03:38:46,132::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914718::DEBUG::2013-11-20 03:38:46,133::task::495::TaskManager.Task::(_debug) Task 563e9e8d-fe7b-45b8-a1bd-1f6fdc334862: ref 0 aborting False
>Thread-15914718::INFO::2013-11-20 03:38:46,134::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000436782836914', 'lastCheck': 1384918724.288604, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0145769119263', 'lastCheck': 1384918719.2674069, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0231680870056', 'lastCheck': 1384918725.481946, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000478982925415', 'lastCheck': 1384918718.2482181, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:38:47,812::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:38:48,345::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0777258 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:38:50,359::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:38:50,980::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0895854 s, 11.4 MB/s\n'; <rc> = 0
>Thread-15913639::INFO::2013-11-20 03:38:51,456::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:38:51,458::task::495::TaskManager.Task::(_debug) Task b1bb2ec0-f1e0-4550-ad1c-1f19e396e658: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:38:51,462::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`cfb89a0e-84e1-408b-98f7-c7f22c02970c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:38:51,463::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:38:51,464::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:38:51,465::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`cfb89a0e-84e1-408b-98f7-c7f22c02970c`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:38:51,467::task::495::TaskManager.Task::(_debug) Task b1bb2ec0-f1e0-4550-ad1c-1f19e396e658: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:38:51,468::task::495::TaskManager.Task::(_debug) Task b1bb2ec0-f1e0-4550-ad1c-1f19e396e658: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:38:51,469::task::495::TaskManager.Task::(_debug) Task b1bb2ec0-f1e0-4550-ad1c-1f19e396e658: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:38:51,470::task::495::TaskManager.Task::(_debug) Task b1bb2ec0-f1e0-4550-ad1c-1f19e396e658: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:38:51,471::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:38:51,472::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:38:51,473::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:38:51,474::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:38:51,475::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:38:51,476::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:38:51,477::task::495::TaskManager.Task::(_debug) Task b1bb2ec0-f1e0-4550-ad1c-1f19e396e658: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:38:51,478::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:38:52,994::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:38:53,570::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0948608 s, 10.8 MB/s\n'; <rc> = 0
>Thread-15914722::DEBUG::2013-11-20 03:38:54,645::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914722::INFO::2013-11-20 03:38:54,647::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914722::DEBUG::2013-11-20 03:38:54,648::task::495::TaskManager.Task::(_debug) Task e0556f28-9b37-463e-9f18-41d6533cb0a2: moving from state init -> state preparing
>Thread-15914722::DEBUG::2013-11-20 03:38:54,649::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914722::DEBUG::2013-11-20 03:38:54,650::task::495::TaskManager.Task::(_debug) Task e0556f28-9b37-463e-9f18-41d6533cb0a2: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914722::DEBUG::2013-11-20 03:38:54,651::task::495::TaskManager.Task::(_debug) Task e0556f28-9b37-463e-9f18-41d6533cb0a2: moving from state preparing -> state finished
>Thread-15914722::DEBUG::2013-11-20 03:38:54,651::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914722::DEBUG::2013-11-20 03:38:54,652::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914722::DEBUG::2013-11-20 03:38:54,653::task::495::TaskManager.Task::(_debug) Task e0556f28-9b37-463e-9f18-41d6533cb0a2: ref 0 aborting False
>Thread-15914722::INFO::2013-11-20 03:38:54,653::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914723::DEBUG::2013-11-20 03:38:54,662::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914723::INFO::2013-11-20 03:38:54,663::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914723::DEBUG::2013-11-20 03:38:54,664::task::495::TaskManager.Task::(_debug) Task b9b9b6c8-c115-4d4d-94db-d90f687926e2: moving from state init -> state preparing
>Thread-15914723::DEBUG::2013-11-20 03:38:54,665::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`023c6c31-4e02-4cbf-8040-997778f2353f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914723::DEBUG::2013-11-20 03:38:54,666::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914723::DEBUG::2013-11-20 03:38:54,667::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914723::DEBUG::2013-11-20 03:38:54,667::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`023c6c31-4e02-4cbf-8040-997778f2353f`::Granted request
>Thread-15914723::DEBUG::2013-11-20 03:38:54,668::task::495::TaskManager.Task::(_debug) Task b9b9b6c8-c115-4d4d-94db-d90f687926e2: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914723::DEBUG::2013-11-20 03:38:54,669::task::495::TaskManager.Task::(_debug) Task b9b9b6c8-c115-4d4d-94db-d90f687926e2: ref 1 aborting False
>Thread-15914723::WARNING::2013-11-20 03:38:54,671::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914723::DEBUG::2013-11-20 03:38:54,672::task::495::TaskManager.Task::(_debug) Task b9b9b6c8-c115-4d4d-94db-d90f687926e2: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914723::DEBUG::2013-11-20 03:38:54,673::task::495::TaskManager.Task::(_debug) Task b9b9b6c8-c115-4d4d-94db-d90f687926e2: moving from state preparing -> state finished
>Thread-15914723::DEBUG::2013-11-20 03:38:54,674::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914723::DEBUG::2013-11-20 03:38:54,674::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914723::DEBUG::2013-11-20 03:38:54,675::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914723::DEBUG::2013-11-20 03:38:54,676::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914723::DEBUG::2013-11-20 03:38:54,676::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914723::DEBUG::2013-11-20 03:38:54,677::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914723::DEBUG::2013-11-20 03:38:54,678::task::495::TaskManager.Task::(_debug) Task b9b9b6c8-c115-4d4d-94db-d90f687926e2: ref 0 aborting False
>Thread-15914723::INFO::2013-11-20 03:38:54,678::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:38:55,586::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:38:56,102::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0815019 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:38:58,115::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:38:58,655::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.088716 s, 11.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:39:00,672::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:39:01,234::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0801699 s, 12.8 MB/s\n'; <rc> = 0
>Thread-15914726::INFO::2013-11-20 03:39:03,037::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914726::DEBUG::2013-11-20 03:39:03,039::task::495::TaskManager.Task::(_debug) Task d9f801b2-affd-42c6-8cba-d118505a12c4: moving from state init -> state preparing
>Thread-15914726::DEBUG::2013-11-20 03:39:03,039::task::495::TaskManager.Task::(_debug) Task d9f801b2-affd-42c6-8cba-d118505a12c4: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000540018081665', 'lastCheck': 1384918734.6618481, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0172529220581', 'lastCheck': 1384918739.798147, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00188302993774', 'lastCheck': 1384918735.7821119, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000523805618286', 'lastCheck': 1384918738.774688, 'valid': True, 'code': 0}}
>Thread-15914726::DEBUG::2013-11-20 03:39:03,040::task::495::TaskManager.Task::(_debug) Task d9f801b2-affd-42c6-8cba-d118505a12c4: moving from state preparing -> state finished
>Thread-15914726::DEBUG::2013-11-20 03:39:03,041::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914726::DEBUG::2013-11-20 03:39:03,042::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914726::DEBUG::2013-11-20 03:39:03,042::task::495::TaskManager.Task::(_debug) Task d9f801b2-affd-42c6-8cba-d118505a12c4: ref 0 aborting False
>Thread-15914726::INFO::2013-11-20 03:39:03,043::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000540018081665', 'lastCheck': 1384918734.6618481, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0172529220581', 'lastCheck': 1384918739.798147, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00188302993774', 'lastCheck': 1384918735.7821119, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000523805618286', 'lastCheck': 1384918738.774688, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:39:03,250::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:39:03,838::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0956534 s, 10.7 MB/s\n'; <rc> = 0
>Thread-15914728::DEBUG::2013-11-20 03:39:04,901::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914728::INFO::2013-11-20 03:39:04,902::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914728::DEBUG::2013-11-20 03:39:04,903::task::495::TaskManager.Task::(_debug) Task c4c37a7c-1323-4f83-8176-cc7f0f139205: moving from state init -> state preparing
>Thread-15914728::DEBUG::2013-11-20 03:39:04,904::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914728::DEBUG::2013-11-20 03:39:04,904::task::495::TaskManager.Task::(_debug) Task c4c37a7c-1323-4f83-8176-cc7f0f139205: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914728::DEBUG::2013-11-20 03:39:04,905::task::495::TaskManager.Task::(_debug) Task c4c37a7c-1323-4f83-8176-cc7f0f139205: moving from state preparing -> state finished
>Thread-15914728::DEBUG::2013-11-20 03:39:04,906::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914728::DEBUG::2013-11-20 03:39:04,906::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914728::DEBUG::2013-11-20 03:39:04,907::task::495::TaskManager.Task::(_debug) Task c4c37a7c-1323-4f83-8176-cc7f0f139205: ref 0 aborting False
>Thread-15914728::INFO::2013-11-20 03:39:04,908::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914729::DEBUG::2013-11-20 03:39:04,912::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914729::INFO::2013-11-20 03:39:04,913::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914729::DEBUG::2013-11-20 03:39:04,914::task::495::TaskManager.Task::(_debug) Task 69580d16-79df-41d4-9e98-1d731380ab9f: moving from state init -> state preparing
>Thread-15914729::DEBUG::2013-11-20 03:39:04,916::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4016dc2b-f74b-4049-8006-e5c958444f20`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914729::DEBUG::2013-11-20 03:39:04,916::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914729::DEBUG::2013-11-20 03:39:04,917::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914729::DEBUG::2013-11-20 03:39:04,918::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4016dc2b-f74b-4049-8006-e5c958444f20`::Granted request
>Thread-15914729::DEBUG::2013-11-20 03:39:04,919::task::495::TaskManager.Task::(_debug) Task 69580d16-79df-41d4-9e98-1d731380ab9f: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914729::DEBUG::2013-11-20 03:39:04,919::task::495::TaskManager.Task::(_debug) Task 69580d16-79df-41d4-9e98-1d731380ab9f: ref 1 aborting False
>Thread-15914729::WARNING::2013-11-20 03:39:04,921::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914729::DEBUG::2013-11-20 03:39:04,922::task::495::TaskManager.Task::(_debug) Task 69580d16-79df-41d4-9e98-1d731380ab9f: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914729::DEBUG::2013-11-20 03:39:04,923::task::495::TaskManager.Task::(_debug) Task 69580d16-79df-41d4-9e98-1d731380ab9f: moving from state preparing -> state finished
>Thread-15914729::DEBUG::2013-11-20 03:39:04,924::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914729::DEBUG::2013-11-20 03:39:04,924::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914729::DEBUG::2013-11-20 03:39:04,925::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914729::DEBUG::2013-11-20 03:39:04,926::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914729::DEBUG::2013-11-20 03:39:04,926::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914729::DEBUG::2013-11-20 03:39:04,927::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914729::DEBUG::2013-11-20 03:39:04,927::task::495::TaskManager.Task::(_debug) Task 69580d16-79df-41d4-9e98-1d731380ab9f: ref 0 aborting False
>Thread-15914729::INFO::2013-11-20 03:39:04,928::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:39:05,847::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:39:06,404::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0818027 s, 12.5 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 03:39:07,495::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:39:07,511::task::495::TaskManager.Task::(_debug) Task bb16d6a6-ba3e-4931-a14b-76bd4831b424: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:39:07,513::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`61bf2eaa-764a-4d1e-adc1-3a64bf290138`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:39:07,514::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:39:07,515::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:39:07,516::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`61bf2eaa-764a-4d1e-adc1-3a64bf290138`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:39:07,518::task::495::TaskManager.Task::(_debug) Task bb16d6a6-ba3e-4931-a14b-76bd4831b424: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:39:07,518::task::495::TaskManager.Task::(_debug) Task bb16d6a6-ba3e-4931-a14b-76bd4831b424: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:39:07,519::task::495::TaskManager.Task::(_debug) Task bb16d6a6-ba3e-4931-a14b-76bd4831b424: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:39:07,520::task::495::TaskManager.Task::(_debug) Task bb16d6a6-ba3e-4931-a14b-76bd4831b424: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:39:07,520::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:39:07,521::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:39:07,521::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:39:07,522::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:39:07,523::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:39:07,523::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:39:07,524::task::495::TaskManager.Task::(_debug) Task bb16d6a6-ba3e-4931-a14b-76bd4831b424: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:39:07,525::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:39:08,420::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:39:08,974::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0813242 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15914454::DEBUG::2013-11-20 03:39:09,803::guestIF::97::vm.Vm::(_handleMessage) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::u'None'
>Dummy-15912651::DEBUG::2013-11-20 03:39:10,989::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 03:39:11,482::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:39:11,483::task::495::TaskManager.Task::(_debug) Task ae73a64a-4221-4beb-9f6f-deb7c294507a: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:39:11,486::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5b0e84b5-ba95-4e34-8476-a5c4533addef`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:39:11,486::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:39:11,488::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:39:11,489::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5b0e84b5-ba95-4e34-8476-a5c4533addef`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:39:11,490::task::495::TaskManager.Task::(_debug) Task ae73a64a-4221-4beb-9f6f-deb7c294507a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:39:11,491::task::495::TaskManager.Task::(_debug) Task ae73a64a-4221-4beb-9f6f-deb7c294507a: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:39:11,492::task::495::TaskManager.Task::(_debug) Task ae73a64a-4221-4beb-9f6f-deb7c294507a: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:39:11,493::task::495::TaskManager.Task::(_debug) Task ae73a64a-4221-4beb-9f6f-deb7c294507a: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:39:11,493::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:39:11,494::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:39:11,495::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:39:11,496::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:39:11,496::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:39:11,497::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:39:11,499::task::495::TaskManager.Task::(_debug) Task ae73a64a-4221-4beb-9f6f-deb7c294507a: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:39:11,500::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:39:11,595::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0975456 s, 10.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:39:13,610::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 03:39:13,853::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:39:13,855::task::495::TaskManager.Task::(_debug) Task 2bd3f3ae-c929-460c-8786-9f3bcfd015fa: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:39:13,858::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3fadd4b8-85f1-4eef-a815-aecd55f871ed`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:39:13,859::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:39:13,860::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:39:13,861::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3fadd4b8-85f1-4eef-a815-aecd55f871ed`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:39:13,862::task::495::TaskManager.Task::(_debug) Task 2bd3f3ae-c929-460c-8786-9f3bcfd015fa: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:39:13,863::task::495::TaskManager.Task::(_debug) Task 2bd3f3ae-c929-460c-8786-9f3bcfd015fa: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:39:13,863::task::495::TaskManager.Task::(_debug) Task 2bd3f3ae-c929-460c-8786-9f3bcfd015fa: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:39:13,864::task::495::TaskManager.Task::(_debug) Task 2bd3f3ae-c929-460c-8786-9f3bcfd015fa: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:39:13,865::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:39:13,865::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:39:13,866::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:39:13,867::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:39:13,867::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:39:13,868::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:39:13,868::task::495::TaskManager.Task::(_debug) Task 2bd3f3ae-c929-460c-8786-9f3bcfd015fa: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:39:13,869::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:39:14,183::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0814146 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15914733::DEBUG::2013-11-20 03:39:15,127::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914733::INFO::2013-11-20 03:39:15,129::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914733::DEBUG::2013-11-20 03:39:15,129::task::495::TaskManager.Task::(_debug) Task 9337a869-ac6b-4627-b529-c2ef73b10a7f: moving from state init -> state preparing
>Thread-15914733::DEBUG::2013-11-20 03:39:15,130::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914733::DEBUG::2013-11-20 03:39:15,131::task::495::TaskManager.Task::(_debug) Task 9337a869-ac6b-4627-b529-c2ef73b10a7f: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914733::DEBUG::2013-11-20 03:39:15,132::task::495::TaskManager.Task::(_debug) Task 9337a869-ac6b-4627-b529-c2ef73b10a7f: moving from state preparing -> state finished
>Thread-15914733::DEBUG::2013-11-20 03:39:15,133::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914733::DEBUG::2013-11-20 03:39:15,133::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914733::DEBUG::2013-11-20 03:39:15,134::task::495::TaskManager.Task::(_debug) Task 9337a869-ac6b-4627-b529-c2ef73b10a7f: ref 0 aborting False
>Thread-15914733::INFO::2013-11-20 03:39:15,134::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914734::DEBUG::2013-11-20 03:39:15,153::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914734::INFO::2013-11-20 03:39:15,153::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914734::DEBUG::2013-11-20 03:39:15,154::task::495::TaskManager.Task::(_debug) Task 5af25de2-ae7f-4840-85be-2bd9101a903b: moving from state init -> state preparing
>Thread-15914734::DEBUG::2013-11-20 03:39:15,155::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9a5f4ac8-9a40-483f-8e2e-0d9596fb5fe1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914734::DEBUG::2013-11-20 03:39:15,156::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914734::DEBUG::2013-11-20 03:39:15,157::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914734::DEBUG::2013-11-20 03:39:15,158::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9a5f4ac8-9a40-483f-8e2e-0d9596fb5fe1`::Granted request
>Thread-15914734::DEBUG::2013-11-20 03:39:15,164::task::495::TaskManager.Task::(_debug) Task 5af25de2-ae7f-4840-85be-2bd9101a903b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914734::DEBUG::2013-11-20 03:39:15,165::task::495::TaskManager.Task::(_debug) Task 5af25de2-ae7f-4840-85be-2bd9101a903b: ref 1 aborting False
>Thread-15914734::WARNING::2013-11-20 03:39:15,167::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914734::DEBUG::2013-11-20 03:39:15,168::task::495::TaskManager.Task::(_debug) Task 5af25de2-ae7f-4840-85be-2bd9101a903b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914734::DEBUG::2013-11-20 03:39:15,169::task::495::TaskManager.Task::(_debug) Task 5af25de2-ae7f-4840-85be-2bd9101a903b: moving from state preparing -> state finished
>Thread-15914734::DEBUG::2013-11-20 03:39:15,169::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914734::DEBUG::2013-11-20 03:39:15,170::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914734::DEBUG::2013-11-20 03:39:15,171::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914734::DEBUG::2013-11-20 03:39:15,171::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914734::DEBUG::2013-11-20 03:39:15,172::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914734::DEBUG::2013-11-20 03:39:15,172::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914734::DEBUG::2013-11-20 03:39:15,173::task::495::TaskManager.Task::(_debug) Task 5af25de2-ae7f-4840-85be-2bd9101a903b: ref 0 aborting False
>Thread-15914734::INFO::2013-11-20 03:39:15,174::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:39:16,198::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:39:16,738::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0893539 s, 11.5 MB/s\n'; <rc> = 0
>Thread-15914736::INFO::2013-11-20 03:39:18,469::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914736::DEBUG::2013-11-20 03:39:18,470::task::495::TaskManager.Task::(_debug) Task bb0f0d95-fd62-4cb3-8196-ca8276fb4e9e: moving from state init -> state preparing
>Thread-15914736::DEBUG::2013-11-20 03:39:18,471::task::495::TaskManager.Task::(_debug) Task bb0f0d95-fd62-4cb3-8196-ca8276fb4e9e: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000332117080688', 'lastCheck': 1384918755.218853, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00137996673584', 'lastCheck': 1384918750.0782471, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00185894966125', 'lastCheck': 1384918756.3992879, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000499963760376', 'lastCheck': 1384918749.0746341, 'valid': True, 'code': 0}}
>Thread-15914736::DEBUG::2013-11-20 03:39:18,472::task::495::TaskManager.Task::(_debug) Task bb0f0d95-fd62-4cb3-8196-ca8276fb4e9e: moving from state preparing -> state finished
>Thread-15914736::DEBUG::2013-11-20 03:39:18,473::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914736::DEBUG::2013-11-20 03:39:18,473::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914736::DEBUG::2013-11-20 03:39:18,474::task::495::TaskManager.Task::(_debug) Task bb0f0d95-fd62-4cb3-8196-ca8276fb4e9e: ref 0 aborting False
>Thread-15914736::INFO::2013-11-20 03:39:18,475::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000332117080688', 'lastCheck': 1384918755.218853, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00137996673584', 'lastCheck': 1384918750.0782471, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00185894966125', 'lastCheck': 1384918756.3992879, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000499963760376', 'lastCheck': 1384918749.0746341, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:39:18,754::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:39:19,275::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.072695 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:39:21,284::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:39:21,852::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0890431 s, 11.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:39:23,867::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 03:39:24,105::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 03:39:24,107::task::495::TaskManager.Task::(_debug) Task 78873771-16ca-48f1-a33d-9adeb2cd92dd: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 03:39:24,109::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c00a7ad8-7e73-47ed-b132-7d8f0333649e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 03:39:24,110::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 03:39:24,111::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 03:39:24,111::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c00a7ad8-7e73-47ed-b132-7d8f0333649e`::Granted request
>Thread-15914435::DEBUG::2013-11-20 03:39:24,112::task::495::TaskManager.Task::(_debug) Task 78873771-16ca-48f1-a33d-9adeb2cd92dd: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 03:39:24,113::task::495::TaskManager.Task::(_debug) Task 78873771-16ca-48f1-a33d-9adeb2cd92dd: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 03:39:24,114::task::495::TaskManager.Task::(_debug) Task 78873771-16ca-48f1-a33d-9adeb2cd92dd: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 03:39:24,114::task::495::TaskManager.Task::(_debug) Task 78873771-16ca-48f1-a33d-9adeb2cd92dd: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 03:39:24,115::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 03:39:24,116::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 03:39:24,117::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 03:39:24,117::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 03:39:24,118::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 03:39:24,118::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 03:39:24,119::task::495::TaskManager.Task::(_debug) Task 78873771-16ca-48f1-a33d-9adeb2cd92dd: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 03:39:24,120::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 03:39:24,465::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0878632 s, 11.7 MB/s\n'; <rc> = 0
>Thread-15914740::DEBUG::2013-11-20 03:39:25,329::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914740::INFO::2013-11-20 03:39:25,331::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914740::DEBUG::2013-11-20 03:39:25,331::task::495::TaskManager.Task::(_debug) Task 3e97f851-ba80-42a6-86ff-ff06b9b41a77: moving from state init -> state preparing
>Thread-15914740::DEBUG::2013-11-20 03:39:25,332::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914740::DEBUG::2013-11-20 03:39:25,333::task::495::TaskManager.Task::(_debug) Task 3e97f851-ba80-42a6-86ff-ff06b9b41a77: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914740::DEBUG::2013-11-20 03:39:25,333::task::495::TaskManager.Task::(_debug) Task 3e97f851-ba80-42a6-86ff-ff06b9b41a77: moving from state preparing -> state finished
>Thread-15914740::DEBUG::2013-11-20 03:39:25,334::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914740::DEBUG::2013-11-20 03:39:25,335::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914740::DEBUG::2013-11-20 03:39:25,336::task::495::TaskManager.Task::(_debug) Task 3e97f851-ba80-42a6-86ff-ff06b9b41a77: ref 0 aborting False
>Thread-15914740::INFO::2013-11-20 03:39:25,336::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914741::DEBUG::2013-11-20 03:39:25,347::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914741::INFO::2013-11-20 03:39:25,349::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914741::DEBUG::2013-11-20 03:39:25,350::task::495::TaskManager.Task::(_debug) Task efa79dc7-fa19-4763-baac-8c9229257d08: moving from state init -> state preparing
>Thread-15914741::DEBUG::2013-11-20 03:39:25,351::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6ef5b4e6-f42c-4376-8bed-6b4cf2306922`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914741::DEBUG::2013-11-20 03:39:25,352::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914741::DEBUG::2013-11-20 03:39:25,352::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914741::DEBUG::2013-11-20 03:39:25,353::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6ef5b4e6-f42c-4376-8bed-6b4cf2306922`::Granted request
>Thread-15914741::DEBUG::2013-11-20 03:39:25,354::task::495::TaskManager.Task::(_debug) Task efa79dc7-fa19-4763-baac-8c9229257d08: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914741::DEBUG::2013-11-20 03:39:25,355::task::495::TaskManager.Task::(_debug) Task efa79dc7-fa19-4763-baac-8c9229257d08: ref 1 aborting False
>Thread-15914741::WARNING::2013-11-20 03:39:25,357::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914741::DEBUG::2013-11-20 03:39:25,358::task::495::TaskManager.Task::(_debug) Task efa79dc7-fa19-4763-baac-8c9229257d08: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914741::DEBUG::2013-11-20 03:39:25,359::task::495::TaskManager.Task::(_debug) Task efa79dc7-fa19-4763-baac-8c9229257d08: moving from state preparing -> state finished
>Thread-15914741::DEBUG::2013-11-20 03:39:25,359::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914741::DEBUG::2013-11-20 03:39:25,360::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914741::DEBUG::2013-11-20 03:39:25,361::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914741::DEBUG::2013-11-20 03:39:25,361::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914741::DEBUG::2013-11-20 03:39:25,362::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914741::DEBUG::2013-11-20 03:39:25,363::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914741::DEBUG::2013-11-20 03:39:25,364::task::495::TaskManager.Task::(_debug) Task efa79dc7-fa19-4763-baac-8c9229257d08: ref 0 aborting False
>Thread-15914741::INFO::2013-11-20 03:39:25,365::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:39:26,478::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:39:27,063::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0911705 s, 11.2 MB/s\n'; <rc> = 0
>Thread-15914453::INFO::2013-11-20 03:39:27,549::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 03:39:27,550::task::495::TaskManager.Task::(_debug) Task 21abfc1f-548f-436d-a759-86dd7d29f872: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 03:39:27,559::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6508f390-a6d7-405f-a8ee-d57072f4267e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 03:39:27,560::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 03:39:27,561::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 03:39:27,562::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6508f390-a6d7-405f-a8ee-d57072f4267e`::Granted request
>Thread-15914453::DEBUG::2013-11-20 03:39:27,564::task::495::TaskManager.Task::(_debug) Task 21abfc1f-548f-436d-a759-86dd7d29f872: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 03:39:27,564::task::495::TaskManager.Task::(_debug) Task 21abfc1f-548f-436d-a759-86dd7d29f872: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 03:39:27,565::task::495::TaskManager.Task::(_debug) Task 21abfc1f-548f-436d-a759-86dd7d29f872: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 03:39:27,566::task::495::TaskManager.Task::(_debug) Task 21abfc1f-548f-436d-a759-86dd7d29f872: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 03:39:27,566::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 03:39:27,567::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 03:39:27,567::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 03:39:27,568::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 03:39:27,569::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 03:39:27,569::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 03:39:27,570::task::495::TaskManager.Task::(_debug) Task 21abfc1f-548f-436d-a759-86dd7d29f872: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 03:39:27,570::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 03:39:29,083::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:39:29,618::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0765005 s, 13.4 MB/s\n'; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-20 03:39:30,346::vm::525::vm.Vm::(_rtcUpdate) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::new rtc offset 0
>libvirtEventLoop::DEBUG::2013-11-20 03:39:30,348::vm::525::vm.Vm::(_rtcUpdate) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::new rtc offset 0
>libvirtEventLoop::DEBUG::2013-11-20 03:39:30,349::vm::525::vm.Vm::(_rtcUpdate) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::new rtc offset 3600
>libvirtEventLoop::DEBUG::2013-11-20 03:39:30,350::vm::525::vm.Vm::(_rtcUpdate) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::new rtc offset 3600
>libvirtEventLoop::DEBUG::2013-11-20 03:39:30,351::vm::525::vm.Vm::(_rtcUpdate) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::new rtc offset 3600
>libvirtEventLoop::DEBUG::2013-11-20 03:39:30,352::vm::525::vm.Vm::(_rtcUpdate) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::new rtc offset 3600
>libvirtEventLoop::DEBUG::2013-11-20 03:39:30,353::vm::525::vm.Vm::(_rtcUpdate) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::new rtc offset 3600
>libvirtEventLoop::DEBUG::2013-11-20 03:39:30,357::vm::525::vm.Vm::(_rtcUpdate) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::new rtc offset 3600
>libvirtEventLoop::DEBUG::2013-11-20 03:39:30,358::vm::525::vm.Vm::(_rtcUpdate) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::new rtc offset 3600
>libvirtEventLoop::DEBUG::2013-11-20 03:39:30,359::vm::525::vm.Vm::(_rtcUpdate) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::new rtc offset 3600
>libvirtEventLoop::DEBUG::2013-11-20 03:39:30,361::vm::525::vm.Vm::(_rtcUpdate) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::new rtc offset 3600
>libvirtEventLoop::DEBUG::2013-11-20 03:39:30,362::vm::525::vm.Vm::(_rtcUpdate) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::new rtc offset 3600
>libvirtEventLoop::DEBUG::2013-11-20 03:39:30,363::vm::525::vm.Vm::(_rtcUpdate) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::new rtc offset 3600
>libvirtEventLoop::DEBUG::2013-11-20 03:39:30,364::vm::525::vm.Vm::(_rtcUpdate) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::new rtc offset 3600
>Dummy-15912651::DEBUG::2013-11-20 03:39:31,633::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 03:39:31,844::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 03:39:31,847::task::495::TaskManager.Task::(_debug) Task 09bad1ba-50f8-4f14-824d-bbbf3e5f02dd: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 03:39:31,850::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e3170c01-31b7-4a1d-8b23-9b643a3b6f8e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 03:39:31,851::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 03:39:31,852::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 03:39:31,853::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e3170c01-31b7-4a1d-8b23-9b643a3b6f8e`::Granted request
>Thread-15914519::DEBUG::2013-11-20 03:39:31,859::task::495::TaskManager.Task::(_debug) Task 09bad1ba-50f8-4f14-824d-bbbf3e5f02dd: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 03:39:31,869::task::495::TaskManager.Task::(_debug) Task 09bad1ba-50f8-4f14-824d-bbbf3e5f02dd: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 03:39:31,870::task::495::TaskManager.Task::(_debug) Task 09bad1ba-50f8-4f14-824d-bbbf3e5f02dd: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 03:39:31,871::task::495::TaskManager.Task::(_debug) Task 09bad1ba-50f8-4f14-824d-bbbf3e5f02dd: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 03:39:31,872::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 03:39:31,872::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 03:39:31,873::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 03:39:31,875::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 03:39:31,875::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 03:39:31,882::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 03:39:31,883::task::495::TaskManager.Task::(_debug) Task 09bad1ba-50f8-4f14-824d-bbbf3e5f02dd: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 03:39:31,883::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 03:39:32,191::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.089468 s, 11.4 MB/s\n'; <rc> = 0
>Thread-15914744::INFO::2013-11-20 03:39:33,917::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914744::DEBUG::2013-11-20 03:39:33,918::task::495::TaskManager.Task::(_debug) Task 8537a8f8-dce7-4955-8ce0-a900b40905fe: moving from state init -> state preparing
>Thread-15914744::DEBUG::2013-11-20 03:39:33,918::task::495::TaskManager.Task::(_debug) Task 8537a8f8-dce7-4955-8ce0-a900b40905fe: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000316143035889', 'lastCheck': 1384918765.5023, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00108981132507', 'lastCheck': 1384918770.735548, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0147969722748', 'lastCheck': 1384918766.809154, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000377178192139', 'lastCheck': 1384918769.732513, 'valid': True, 'code': 0}}
>Thread-15914744::DEBUG::2013-11-20 03:39:33,919::task::495::TaskManager.Task::(_debug) Task 8537a8f8-dce7-4955-8ce0-a900b40905fe: moving from state preparing -> state finished
>Thread-15914744::DEBUG::2013-11-20 03:39:33,920::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914744::DEBUG::2013-11-20 03:39:33,921::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914744::DEBUG::2013-11-20 03:39:33,921::task::495::TaskManager.Task::(_debug) Task 8537a8f8-dce7-4955-8ce0-a900b40905fe: ref 0 aborting False
>Thread-15914744::INFO::2013-11-20 03:39:33,922::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000316143035889', 'lastCheck': 1384918765.5023, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00108981132507', 'lastCheck': 1384918770.735548, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0147969722748', 'lastCheck': 1384918766.809154, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000377178192139', 'lastCheck': 1384918769.732513, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:39:34,206::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:39:34,744::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0744805 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 03:39:35,356::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:39:35,362::task::495::TaskManager.Task::(_debug) Task 7ff9f858-d475-4eaf-8a8e-b982425ae511: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:39:35,366::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7a78997f-e334-4bc0-8bc8-3d2510fe8fd4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:39:35,367::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:39:35,368::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:39:35,368::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7a78997f-e334-4bc0-8bc8-3d2510fe8fd4`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:39:35,369::task::495::TaskManager.Task::(_debug) Task 7ff9f858-d475-4eaf-8a8e-b982425ae511: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:39:35,370::task::495::TaskManager.Task::(_debug) Task 7ff9f858-d475-4eaf-8a8e-b982425ae511: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:39:35,371::task::495::TaskManager.Task::(_debug) Task 7ff9f858-d475-4eaf-8a8e-b982425ae511: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:39:35,372::task::495::TaskManager.Task::(_debug) Task 7ff9f858-d475-4eaf-8a8e-b982425ae511: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:39:35,372::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:39:35,373::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:39:35,374::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:39:35,375::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:39:35,375::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:39:35,376::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:39:35,377::task::495::TaskManager.Task::(_debug) Task 7ff9f858-d475-4eaf-8a8e-b982425ae511: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:39:35,378::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:39:35,380::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:39:35,380::task::495::TaskManager.Task::(_debug) Task f534da80-d2ba-4279-8fc0-df34e98c8e24: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:39:35,382::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`cba0b6ec-726b-49c3-b8ec-5b5a3d5e292b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:39:35,383::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:39:35,384::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:39:35,385::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`cba0b6ec-726b-49c3-b8ec-5b5a3d5e292b`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:39:35,386::task::495::TaskManager.Task::(_debug) Task f534da80-d2ba-4279-8fc0-df34e98c8e24: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:39:35,387::task::495::TaskManager.Task::(_debug) Task f534da80-d2ba-4279-8fc0-df34e98c8e24: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:39:35,388::task::495::TaskManager.Task::(_debug) Task f534da80-d2ba-4279-8fc0-df34e98c8e24: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:39:35,389::task::495::TaskManager.Task::(_debug) Task f534da80-d2ba-4279-8fc0-df34e98c8e24: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:39:35,390::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:39:35,391::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:39:35,392::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:39:35,393::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:39:35,394::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:39:35,394::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:39:35,396::task::495::TaskManager.Task::(_debug) Task f534da80-d2ba-4279-8fc0-df34e98c8e24: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:39:35,397::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15914746::DEBUG::2013-11-20 03:39:35,524::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914746::INFO::2013-11-20 03:39:35,525::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914746::DEBUG::2013-11-20 03:39:35,526::task::495::TaskManager.Task::(_debug) Task 17dcfe77-58f1-470d-a4f8-b8e5c21f3731: moving from state init -> state preparing
>Thread-15914746::DEBUG::2013-11-20 03:39:35,527::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914746::DEBUG::2013-11-20 03:39:35,527::task::495::TaskManager.Task::(_debug) Task 17dcfe77-58f1-470d-a4f8-b8e5c21f3731: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914746::DEBUG::2013-11-20 03:39:35,528::task::495::TaskManager.Task::(_debug) Task 17dcfe77-58f1-470d-a4f8-b8e5c21f3731: moving from state preparing -> state finished
>Thread-15914746::DEBUG::2013-11-20 03:39:35,529::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914746::DEBUG::2013-11-20 03:39:35,529::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914746::DEBUG::2013-11-20 03:39:35,530::task::495::TaskManager.Task::(_debug) Task 17dcfe77-58f1-470d-a4f8-b8e5c21f3731: ref 0 aborting False
>Thread-15914746::INFO::2013-11-20 03:39:35,531::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914747::DEBUG::2013-11-20 03:39:35,536::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914747::INFO::2013-11-20 03:39:35,537::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914747::DEBUG::2013-11-20 03:39:35,538::task::495::TaskManager.Task::(_debug) Task 057d152c-5eee-4c85-b15b-7f555acd73a5: moving from state init -> state preparing
>Thread-15914747::DEBUG::2013-11-20 03:39:35,540::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1917551d-e330-471a-a1a8-30bc00cca71c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914747::DEBUG::2013-11-20 03:39:35,540::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914747::DEBUG::2013-11-20 03:39:35,541::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914747::DEBUG::2013-11-20 03:39:35,542::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1917551d-e330-471a-a1a8-30bc00cca71c`::Granted request
>Thread-15914747::DEBUG::2013-11-20 03:39:35,557::task::495::TaskManager.Task::(_debug) Task 057d152c-5eee-4c85-b15b-7f555acd73a5: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914747::DEBUG::2013-11-20 03:39:35,558::task::495::TaskManager.Task::(_debug) Task 057d152c-5eee-4c85-b15b-7f555acd73a5: ref 1 aborting False
>Thread-15914747::WARNING::2013-11-20 03:39:35,560::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914747::DEBUG::2013-11-20 03:39:35,561::task::495::TaskManager.Task::(_debug) Task 057d152c-5eee-4c85-b15b-7f555acd73a5: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914747::DEBUG::2013-11-20 03:39:35,562::task::495::TaskManager.Task::(_debug) Task 057d152c-5eee-4c85-b15b-7f555acd73a5: moving from state preparing -> state finished
>Thread-15914747::DEBUG::2013-11-20 03:39:35,563::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914747::DEBUG::2013-11-20 03:39:35,563::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914747::DEBUG::2013-11-20 03:39:35,564::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914747::DEBUG::2013-11-20 03:39:35,565::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914747::DEBUG::2013-11-20 03:39:35,566::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914747::DEBUG::2013-11-20 03:39:35,567::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914747::DEBUG::2013-11-20 03:39:35,568::task::495::TaskManager.Task::(_debug) Task 057d152c-5eee-4c85-b15b-7f555acd73a5: ref 0 aborting False
>Thread-15914747::INFO::2013-11-20 03:39:35,568::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:39:36,763::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:39:37,336::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0866712 s, 11.8 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 03:39:37,573::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:39:37,587::task::495::TaskManager.Task::(_debug) Task dc526968-8c16-492a-9003-3301c0652116: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:39:37,589::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`1099f89e-a94f-4ae0-925d-13e51614c1f9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:39:37,590::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:39:37,592::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:39:37,592::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`1099f89e-a94f-4ae0-925d-13e51614c1f9`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:39:37,593::task::495::TaskManager.Task::(_debug) Task dc526968-8c16-492a-9003-3301c0652116: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:39:37,594::task::495::TaskManager.Task::(_debug) Task dc526968-8c16-492a-9003-3301c0652116: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:39:37,595::task::495::TaskManager.Task::(_debug) Task dc526968-8c16-492a-9003-3301c0652116: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:39:37,595::task::495::TaskManager.Task::(_debug) Task dc526968-8c16-492a-9003-3301c0652116: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:39:37,596::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:39:37,597::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:39:37,598::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:39:37,599::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:39:37,599::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:39:37,600::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:39:37,601::task::495::TaskManager.Task::(_debug) Task dc526968-8c16-492a-9003-3301c0652116: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:39:37,601::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:39:37,602::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:39:37,603::task::495::TaskManager.Task::(_debug) Task f5c6b359-8c3a-47d8-a032-0e27029c45d4: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:39:37,604::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8e2fc5ab-2b33-4a65-a2e8-3f3ce91244db`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:39:37,605::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:39:37,606::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:39:37,606::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8e2fc5ab-2b33-4a65-a2e8-3f3ce91244db`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:39:37,607::task::495::TaskManager.Task::(_debug) Task f5c6b359-8c3a-47d8-a032-0e27029c45d4: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:39:37,608::task::495::TaskManager.Task::(_debug) Task f5c6b359-8c3a-47d8-a032-0e27029c45d4: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:39:37,608::task::495::TaskManager.Task::(_debug) Task f5c6b359-8c3a-47d8-a032-0e27029c45d4: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:39:37,609::task::495::TaskManager.Task::(_debug) Task f5c6b359-8c3a-47d8-a032-0e27029c45d4: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:39:37,610::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:39:37,610::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:39:37,611::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:39:37,612::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:39:37,612::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:39:37,613::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:39:37,613::task::495::TaskManager.Task::(_debug) Task f5c6b359-8c3a-47d8-a032-0e27029c45d4: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:39:37,614::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:39:37,615::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:39:37,615::task::495::TaskManager.Task::(_debug) Task 0cd5df65-4457-4cb6-8483-34ba6cedad60: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:39:37,616::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`596bf55e-d7d8-4217-9c05-194be58dd7b4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:39:37,617::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:39:37,618::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:39:37,618::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`596bf55e-d7d8-4217-9c05-194be58dd7b4`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:39:37,619::task::495::TaskManager.Task::(_debug) Task 0cd5df65-4457-4cb6-8483-34ba6cedad60: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:39:37,620::task::495::TaskManager.Task::(_debug) Task 0cd5df65-4457-4cb6-8483-34ba6cedad60: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:39:37,621::task::495::TaskManager.Task::(_debug) Task 0cd5df65-4457-4cb6-8483-34ba6cedad60: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:39:37,621::task::495::TaskManager.Task::(_debug) Task 0cd5df65-4457-4cb6-8483-34ba6cedad60: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:39:37,622::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:39:37,622::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:39:37,623::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:39:37,624::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:39:37,624::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:39:37,625::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:39:37,626::task::495::TaskManager.Task::(_debug) Task 0cd5df65-4457-4cb6-8483-34ba6cedad60: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:39:37,626::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:39:37,627::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:39:37,628::task::495::TaskManager.Task::(_debug) Task 7c3cf769-d803-4379-b20d-f174557c77f0: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:39:37,629::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e2590b17-87de-4abf-9e1c-d64d91c82d77`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:39:37,629::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:39:37,630::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:39:37,631::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e2590b17-87de-4abf-9e1c-d64d91c82d77`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:39:37,632::task::495::TaskManager.Task::(_debug) Task 7c3cf769-d803-4379-b20d-f174557c77f0: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:39:37,632::task::495::TaskManager.Task::(_debug) Task 7c3cf769-d803-4379-b20d-f174557c77f0: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:39:37,633::task::495::TaskManager.Task::(_debug) Task 7c3cf769-d803-4379-b20d-f174557c77f0: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:39:37,633::task::495::TaskManager.Task::(_debug) Task 7c3cf769-d803-4379-b20d-f174557c77f0: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:39:37,634::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:39:37,634::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:39:37,635::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:39:37,636::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:39:37,636::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:39:37,637::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:39:37,637::task::495::TaskManager.Task::(_debug) Task 7c3cf769-d803-4379-b20d-f174557c77f0: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:39:37,638::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:39:39,353::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:39:39,941::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0910764 s, 11.2 MB/s\n'; <rc> = 0
>Thread-42840::DEBUG::2013-11-20 03:39:39,997::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 03:39:40,892::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:39:41,959::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:39:42,541::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0939748 s, 10.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:39:44,557::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:39:45,116::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0794634 s, 12.9 MB/s\n'; <rc> = 0
>Thread-15914751::DEBUG::2013-11-20 03:39:45,932::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914751::INFO::2013-11-20 03:39:45,935::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914751::DEBUG::2013-11-20 03:39:45,935::task::495::TaskManager.Task::(_debug) Task 3d77ebd7-e166-474e-837b-39fb465adb53: moving from state init -> state preparing
>Thread-15914751::DEBUG::2013-11-20 03:39:45,937::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914751::DEBUG::2013-11-20 03:39:45,938::task::495::TaskManager.Task::(_debug) Task 3d77ebd7-e166-474e-837b-39fb465adb53: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914751::DEBUG::2013-11-20 03:39:45,939::task::495::TaskManager.Task::(_debug) Task 3d77ebd7-e166-474e-837b-39fb465adb53: moving from state preparing -> state finished
>Thread-15914751::DEBUG::2013-11-20 03:39:45,940::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914751::DEBUG::2013-11-20 03:39:45,941::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914751::DEBUG::2013-11-20 03:39:45,942::task::495::TaskManager.Task::(_debug) Task 3d77ebd7-e166-474e-837b-39fb465adb53: ref 0 aborting False
>Thread-15914751::INFO::2013-11-20 03:39:45,943::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914752::DEBUG::2013-11-20 03:39:45,948::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914752::INFO::2013-11-20 03:39:45,948::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914752::DEBUG::2013-11-20 03:39:45,949::task::495::TaskManager.Task::(_debug) Task ef61a6b0-6b1a-4ae8-912e-7fadf082205c: moving from state init -> state preparing
>Thread-15914752::DEBUG::2013-11-20 03:39:45,950::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f2568d8f-47d0-4543-8ac1-ffd1383c2fd5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914752::DEBUG::2013-11-20 03:39:45,951::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914752::DEBUG::2013-11-20 03:39:45,952::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914752::DEBUG::2013-11-20 03:39:45,952::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f2568d8f-47d0-4543-8ac1-ffd1383c2fd5`::Granted request
>Thread-15914752::DEBUG::2013-11-20 03:39:45,953::task::495::TaskManager.Task::(_debug) Task ef61a6b0-6b1a-4ae8-912e-7fadf082205c: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914752::DEBUG::2013-11-20 03:39:45,954::task::495::TaskManager.Task::(_debug) Task ef61a6b0-6b1a-4ae8-912e-7fadf082205c: ref 1 aborting False
>Thread-15914752::WARNING::2013-11-20 03:39:45,956::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914752::DEBUG::2013-11-20 03:39:45,957::task::495::TaskManager.Task::(_debug) Task ef61a6b0-6b1a-4ae8-912e-7fadf082205c: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914752::DEBUG::2013-11-20 03:39:45,958::task::495::TaskManager.Task::(_debug) Task ef61a6b0-6b1a-4ae8-912e-7fadf082205c: moving from state preparing -> state finished
>Thread-15914752::DEBUG::2013-11-20 03:39:45,959::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914752::DEBUG::2013-11-20 03:39:45,960::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914752::DEBUG::2013-11-20 03:39:45,961::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914752::DEBUG::2013-11-20 03:39:45,961::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914752::DEBUG::2013-11-20 03:39:45,962::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914752::DEBUG::2013-11-20 03:39:45,962::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914752::DEBUG::2013-11-20 03:39:45,963::task::495::TaskManager.Task::(_debug) Task ef61a6b0-6b1a-4ae8-912e-7fadf082205c: ref 0 aborting False
>Thread-15914752::INFO::2013-11-20 03:39:45,964::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:39:47,132::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:39:47,666::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0856793 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:39:49,682::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:39:50,248::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0909686 s, 11.3 MB/s\n'; <rc> = 0
>Thread-15914754::INFO::2013-11-20 03:39:50,434::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914754::DEBUG::2013-11-20 03:39:50,435::task::495::TaskManager.Task::(_debug) Task 0318517e-0eda-495b-b3bd-db8763ecd35d: moving from state init -> state preparing
>Thread-15914754::DEBUG::2013-11-20 03:39:50,436::task::495::TaskManager.Task::(_debug) Task 0318517e-0eda-495b-b3bd-db8763ecd35d: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000432014465332', 'lastCheck': 1384918786.172827, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0122680664062', 'lastCheck': 1384918781.0086751, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0014500617981', 'lastCheck': 1384918787.410187, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000354051589966', 'lastCheck': 1384918780.8949449, 'valid': True, 'code': 0}}
>Thread-15914754::DEBUG::2013-11-20 03:39:50,437::task::495::TaskManager.Task::(_debug) Task 0318517e-0eda-495b-b3bd-db8763ecd35d: moving from state preparing -> state finished
>Thread-15914754::DEBUG::2013-11-20 03:39:50,438::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914754::DEBUG::2013-11-20 03:39:50,438::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914754::DEBUG::2013-11-20 03:39:50,439::task::495::TaskManager.Task::(_debug) Task 0318517e-0eda-495b-b3bd-db8763ecd35d: ref 0 aborting False
>Thread-15914754::INFO::2013-11-20 03:39:50,440::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000432014465332', 'lastCheck': 1384918786.172827, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0122680664062', 'lastCheck': 1384918781.0086751, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0014500617981', 'lastCheck': 1384918787.410187, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000354051589966', 'lastCheck': 1384918780.8949449, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:39:52,269::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 03:39:52,493::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:39:52,495::task::495::TaskManager.Task::(_debug) Task 8fc02f16-a80a-4aec-a88e-48b7c64d0114: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:39:52,500::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c4c47d67-c65e-4ff2-914b-0d08466bb807`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:39:52,502::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:39:52,504::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:39:52,505::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c4c47d67-c65e-4ff2-914b-0d08466bb807`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:39:52,507::task::495::TaskManager.Task::(_debug) Task 8fc02f16-a80a-4aec-a88e-48b7c64d0114: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:39:52,508::task::495::TaskManager.Task::(_debug) Task 8fc02f16-a80a-4aec-a88e-48b7c64d0114: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:39:52,509::task::495::TaskManager.Task::(_debug) Task 8fc02f16-a80a-4aec-a88e-48b7c64d0114: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:39:52,510::task::495::TaskManager.Task::(_debug) Task 8fc02f16-a80a-4aec-a88e-48b7c64d0114: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:39:52,511::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:39:52,511::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:39:52,512::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:39:52,516::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:39:52,517::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:39:52,518::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:39:52,519::task::495::TaskManager.Task::(_debug) Task 8fc02f16-a80a-4aec-a88e-48b7c64d0114: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:39:52,520::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:39:52,842::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0959972 s, 10.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:39:54,853::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:39:55,410::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0750098 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15914757::DEBUG::2013-11-20 03:39:56,418::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914757::INFO::2013-11-20 03:39:56,420::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914757::DEBUG::2013-11-20 03:39:56,421::task::495::TaskManager.Task::(_debug) Task 2300ed59-acbf-4af5-a5e7-b17721f11c53: moving from state init -> state preparing
>Thread-15914757::DEBUG::2013-11-20 03:39:56,422::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914757::DEBUG::2013-11-20 03:39:56,422::task::495::TaskManager.Task::(_debug) Task 2300ed59-acbf-4af5-a5e7-b17721f11c53: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914757::DEBUG::2013-11-20 03:39:56,423::task::495::TaskManager.Task::(_debug) Task 2300ed59-acbf-4af5-a5e7-b17721f11c53: moving from state preparing -> state finished
>Thread-15914757::DEBUG::2013-11-20 03:39:56,424::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914757::DEBUG::2013-11-20 03:39:56,425::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914757::DEBUG::2013-11-20 03:39:56,426::task::495::TaskManager.Task::(_debug) Task 2300ed59-acbf-4af5-a5e7-b17721f11c53: ref 0 aborting False
>Thread-15914757::INFO::2013-11-20 03:39:56,427::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914758::DEBUG::2013-11-20 03:39:56,441::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914758::INFO::2013-11-20 03:39:56,442::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914758::DEBUG::2013-11-20 03:39:56,442::task::495::TaskManager.Task::(_debug) Task 3ad10717-ed36-40c8-aff1-978947e962ea: moving from state init -> state preparing
>Thread-15914758::DEBUG::2013-11-20 03:39:56,444::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`02edf4b4-69ad-4caa-82ca-f100d205ce6b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914758::DEBUG::2013-11-20 03:39:56,445::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914758::DEBUG::2013-11-20 03:39:56,445::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914758::DEBUG::2013-11-20 03:39:56,446::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`02edf4b4-69ad-4caa-82ca-f100d205ce6b`::Granted request
>Thread-15914758::DEBUG::2013-11-20 03:39:56,447::task::495::TaskManager.Task::(_debug) Task 3ad10717-ed36-40c8-aff1-978947e962ea: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914758::DEBUG::2013-11-20 03:39:56,448::task::495::TaskManager.Task::(_debug) Task 3ad10717-ed36-40c8-aff1-978947e962ea: ref 1 aborting False
>Thread-15914758::WARNING::2013-11-20 03:39:56,450::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914758::DEBUG::2013-11-20 03:39:56,451::task::495::TaskManager.Task::(_debug) Task 3ad10717-ed36-40c8-aff1-978947e962ea: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914758::DEBUG::2013-11-20 03:39:56,452::task::495::TaskManager.Task::(_debug) Task 3ad10717-ed36-40c8-aff1-978947e962ea: moving from state preparing -> state finished
>Thread-15914758::DEBUG::2013-11-20 03:39:56,453::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914758::DEBUG::2013-11-20 03:39:56,454::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914758::DEBUG::2013-11-20 03:39:56,455::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914758::DEBUG::2013-11-20 03:39:56,455::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914758::DEBUG::2013-11-20 03:39:56,456::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914758::DEBUG::2013-11-20 03:39:56,457::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914758::DEBUG::2013-11-20 03:39:56,457::task::495::TaskManager.Task::(_debug) Task 3ad10717-ed36-40c8-aff1-978947e962ea: ref 0 aborting False
>Thread-15914758::INFO::2013-11-20 03:39:56,458::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:39:57,425::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:39:57,967::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0758782 s, 13.5 MB/s\n'; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-20 03:39:59,478::vm::525::vm.Vm::(_rtcUpdate) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::new rtc offset 1
>libvirtEventLoop::DEBUG::2013-11-20 03:39:59,479::vm::525::vm.Vm::(_rtcUpdate) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::new rtc offset 1
>libvirtEventLoop::DEBUG::2013-11-20 03:39:59,480::vm::525::vm.Vm::(_rtcUpdate) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::new rtc offset 3601
>libvirtEventLoop::DEBUG::2013-11-20 03:39:59,481::vm::525::vm.Vm::(_rtcUpdate) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::new rtc offset 3601
>libvirtEventLoop::DEBUG::2013-11-20 03:39:59,482::vm::525::vm.Vm::(_rtcUpdate) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::new rtc offset 3601
>libvirtEventLoop::DEBUG::2013-11-20 03:39:59,483::vm::525::vm.Vm::(_rtcUpdate) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::new rtc offset 3601
>libvirtEventLoop::DEBUG::2013-11-20 03:39:59,484::vm::525::vm.Vm::(_rtcUpdate) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::new rtc offset 3601
>libvirtEventLoop::DEBUG::2013-11-20 03:39:59,592::vm::525::vm.Vm::(_rtcUpdate) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::new rtc offset 3601
>libvirtEventLoop::DEBUG::2013-11-20 03:39:59,593::vm::525::vm.Vm::(_rtcUpdate) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::new rtc offset 3601
>libvirtEventLoop::DEBUG::2013-11-20 03:39:59,594::vm::525::vm.Vm::(_rtcUpdate) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::new rtc offset 3601
>libvirtEventLoop::DEBUG::2013-11-20 03:39:59,596::vm::525::vm.Vm::(_rtcUpdate) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::new rtc offset 3601
>libvirtEventLoop::DEBUG::2013-11-20 03:39:59,597::vm::525::vm.Vm::(_rtcUpdate) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::new rtc offset 3601
>libvirtEventLoop::DEBUG::2013-11-20 03:39:59,598::vm::525::vm.Vm::(_rtcUpdate) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::new rtc offset 3601
>libvirtEventLoop::DEBUG::2013-11-20 03:39:59,599::vm::525::vm.Vm::(_rtcUpdate) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::new rtc offset 3601
>Dummy-15912651::DEBUG::2013-11-20 03:39:59,981::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:40:00,500::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0543004 s, 18.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:40:02,513::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:40:03,116::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.078797 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:40:05,131::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:40:05,683::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0813414 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15914762::INFO::2013-11-20 03:40:06,291::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914762::DEBUG::2013-11-20 03:40:06,292::task::495::TaskManager.Task::(_debug) Task 920a4c9f-2c0e-4576-b269-eee7a6082118: moving from state init -> state preparing
>Thread-15914762::DEBUG::2013-11-20 03:40:06,301::task::495::TaskManager.Task::(_debug) Task 920a4c9f-2c0e-4576-b269-eee7a6082118: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.0080931186676', 'lastCheck': 1384918796.508862, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.013237953186', 'lastCheck': 1384918801.6469009, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00836992263794', 'lastCheck': 1384918797.6476221, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000415086746216', 'lastCheck': 1384918801.2030921, 'valid': True, 'code': 0}}
>Thread-15914762::DEBUG::2013-11-20 03:40:06,303::task::495::TaskManager.Task::(_debug) Task 920a4c9f-2c0e-4576-b269-eee7a6082118: moving from state preparing -> state finished
>Thread-15914762::DEBUG::2013-11-20 03:40:06,304::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914762::DEBUG::2013-11-20 03:40:06,305::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914762::DEBUG::2013-11-20 03:40:06,306::task::495::TaskManager.Task::(_debug) Task 920a4c9f-2c0e-4576-b269-eee7a6082118: ref 0 aborting False
>Thread-15914762::INFO::2013-11-20 03:40:06,307::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.0080931186676', 'lastCheck': 1384918796.508862, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.013237953186', 'lastCheck': 1384918801.6469009, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00836992263794', 'lastCheck': 1384918797.6476221, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000415086746216', 'lastCheck': 1384918801.2030921, 'valid': True, 'code': 0}}
>Thread-15914764::DEBUG::2013-11-20 03:40:06,794::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914764::INFO::2013-11-20 03:40:06,795::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914764::DEBUG::2013-11-20 03:40:06,796::task::495::TaskManager.Task::(_debug) Task 554130f0-e5ec-4862-bb5d-a486b523676c: moving from state init -> state preparing
>Thread-15914764::DEBUG::2013-11-20 03:40:06,797::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914764::DEBUG::2013-11-20 03:40:06,798::task::495::TaskManager.Task::(_debug) Task 554130f0-e5ec-4862-bb5d-a486b523676c: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914764::DEBUG::2013-11-20 03:40:06,798::task::495::TaskManager.Task::(_debug) Task 554130f0-e5ec-4862-bb5d-a486b523676c: moving from state preparing -> state finished
>Thread-15914764::DEBUG::2013-11-20 03:40:06,799::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914764::DEBUG::2013-11-20 03:40:06,799::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914764::DEBUG::2013-11-20 03:40:06,800::task::495::TaskManager.Task::(_debug) Task 554130f0-e5ec-4862-bb5d-a486b523676c: ref 0 aborting False
>Thread-15914764::INFO::2013-11-20 03:40:06,801::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914765::DEBUG::2013-11-20 03:40:06,806::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914765::INFO::2013-11-20 03:40:06,807::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914765::DEBUG::2013-11-20 03:40:06,808::task::495::TaskManager.Task::(_debug) Task 9ec87ef7-e711-4bc2-98b3-7f9c91a1896e: moving from state init -> state preparing
>Thread-15914765::DEBUG::2013-11-20 03:40:06,809::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`261ed122-cc95-4739-9186-2393cd75e2c3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914765::DEBUG::2013-11-20 03:40:06,810::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914765::DEBUG::2013-11-20 03:40:06,810::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914765::DEBUG::2013-11-20 03:40:06,811::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`261ed122-cc95-4739-9186-2393cd75e2c3`::Granted request
>Thread-15914765::DEBUG::2013-11-20 03:40:06,812::task::495::TaskManager.Task::(_debug) Task 9ec87ef7-e711-4bc2-98b3-7f9c91a1896e: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914765::DEBUG::2013-11-20 03:40:06,813::task::495::TaskManager.Task::(_debug) Task 9ec87ef7-e711-4bc2-98b3-7f9c91a1896e: ref 1 aborting False
>Thread-15914765::WARNING::2013-11-20 03:40:06,815::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914765::DEBUG::2013-11-20 03:40:06,816::task::495::TaskManager.Task::(_debug) Task 9ec87ef7-e711-4bc2-98b3-7f9c91a1896e: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914765::DEBUG::2013-11-20 03:40:06,817::task::495::TaskManager.Task::(_debug) Task 9ec87ef7-e711-4bc2-98b3-7f9c91a1896e: moving from state preparing -> state finished
>Thread-15914765::DEBUG::2013-11-20 03:40:06,818::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914765::DEBUG::2013-11-20 03:40:06,819::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914765::DEBUG::2013-11-20 03:40:06,819::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914765::DEBUG::2013-11-20 03:40:06,820::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914765::DEBUG::2013-11-20 03:40:06,821::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914765::DEBUG::2013-11-20 03:40:06,821::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914765::DEBUG::2013-11-20 03:40:06,822::task::495::TaskManager.Task::(_debug) Task 9ec87ef7-e711-4bc2-98b3-7f9c91a1896e: ref 0 aborting False
>Thread-15914765::INFO::2013-11-20 03:40:06,823::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:40:07,700::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 03:40:07,918::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:40:07,923::task::495::TaskManager.Task::(_debug) Task f3ed8dde-20c6-46a3-a770-e1cb107dab5e: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:40:07,928::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4c74074a-b10a-43be-b4f9-f62ac1b7abeb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:40:07,930::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:40:07,932::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:40:07,933::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4c74074a-b10a-43be-b4f9-f62ac1b7abeb`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:40:07,934::task::495::TaskManager.Task::(_debug) Task f3ed8dde-20c6-46a3-a770-e1cb107dab5e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:40:07,935::task::495::TaskManager.Task::(_debug) Task f3ed8dde-20c6-46a3-a770-e1cb107dab5e: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:40:07,936::task::495::TaskManager.Task::(_debug) Task f3ed8dde-20c6-46a3-a770-e1cb107dab5e: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:40:07,936::task::495::TaskManager.Task::(_debug) Task f3ed8dde-20c6-46a3-a770-e1cb107dab5e: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:40:07,937::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:40:07,938::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:40:07,938::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:40:07,939::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:40:07,940::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:40:07,940::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:40:07,941::task::495::TaskManager.Task::(_debug) Task f3ed8dde-20c6-46a3-a770-e1cb107dab5e: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:40:07,942::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15914520::DEBUG::2013-11-20 03:40:08,041::guestIF::97::vm.Vm::(_handleMessage) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::u'None'
>Dummy-15912651::DEBUG::2013-11-20 03:40:08,249::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0791791 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:40:10,265::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:40:10,849::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0781208 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:40:12,859::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 03:40:13,068::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:40:13,071::task::495::TaskManager.Task::(_debug) Task ef7889b1-9d5e-4465-9159-a84e97e4e1f3: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:40:13,075::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`295ab1f9-05e7-4431-ad01-2525b84ac2cb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:40:13,077::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:40:13,079::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:40:13,080::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`295ab1f9-05e7-4431-ad01-2525b84ac2cb`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:40:13,082::task::495::TaskManager.Task::(_debug) Task ef7889b1-9d5e-4465-9159-a84e97e4e1f3: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:40:13,082::task::495::TaskManager.Task::(_debug) Task ef7889b1-9d5e-4465-9159-a84e97e4e1f3: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:40:13,083::task::495::TaskManager.Task::(_debug) Task ef7889b1-9d5e-4465-9159-a84e97e4e1f3: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:40:13,084::task::495::TaskManager.Task::(_debug) Task ef7889b1-9d5e-4465-9159-a84e97e4e1f3: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:40:13,085::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:40:13,086::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:40:13,087::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:40:13,087::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:40:13,088::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:40:13,088::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:40:13,089::task::495::TaskManager.Task::(_debug) Task ef7889b1-9d5e-4465-9159-a84e97e4e1f3: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:40:13,090::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:40:13,379::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0737994 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:40:15,394::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 03:40:15,628::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:40:15,631::task::495::TaskManager.Task::(_debug) Task a45031f3-a52e-4e9f-b0b4-9f8660d1fc63: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:40:15,634::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`11db9cc2-141c-432a-8699-aaa4e4551241`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:40:15,634::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:40:15,636::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:40:15,636::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`11db9cc2-141c-432a-8699-aaa4e4551241`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:40:15,638::task::495::TaskManager.Task::(_debug) Task a45031f3-a52e-4e9f-b0b4-9f8660d1fc63: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:40:15,639::task::495::TaskManager.Task::(_debug) Task a45031f3-a52e-4e9f-b0b4-9f8660d1fc63: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:40:15,640::task::495::TaskManager.Task::(_debug) Task a45031f3-a52e-4e9f-b0b4-9f8660d1fc63: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:40:15,640::task::495::TaskManager.Task::(_debug) Task a45031f3-a52e-4e9f-b0b4-9f8660d1fc63: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:40:15,641::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:40:15,642::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:40:15,643::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:40:15,644::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:40:15,644::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:40:15,645::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:40:15,646::task::495::TaskManager.Task::(_debug) Task a45031f3-a52e-4e9f-b0b4-9f8660d1fc63: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:40:15,646::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:40:15,967::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0770696 s, 13.3 MB/s\n'; <rc> = 0
>Thread-15914769::DEBUG::2013-11-20 03:40:17,139::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914769::INFO::2013-11-20 03:40:17,140::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914769::DEBUG::2013-11-20 03:40:17,141::task::495::TaskManager.Task::(_debug) Task 1385482c-d46f-4746-b58c-f9bd7a88e75e: moving from state init -> state preparing
>Thread-15914769::DEBUG::2013-11-20 03:40:17,142::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914769::DEBUG::2013-11-20 03:40:17,142::task::495::TaskManager.Task::(_debug) Task 1385482c-d46f-4746-b58c-f9bd7a88e75e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914769::DEBUG::2013-11-20 03:40:17,143::task::495::TaskManager.Task::(_debug) Task 1385482c-d46f-4746-b58c-f9bd7a88e75e: moving from state preparing -> state finished
>Thread-15914769::DEBUG::2013-11-20 03:40:17,144::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914769::DEBUG::2013-11-20 03:40:17,144::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914769::DEBUG::2013-11-20 03:40:17,145::task::495::TaskManager.Task::(_debug) Task 1385482c-d46f-4746-b58c-f9bd7a88e75e: ref 0 aborting False
>Thread-15914769::INFO::2013-11-20 03:40:17,145::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914770::DEBUG::2013-11-20 03:40:17,155::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914770::INFO::2013-11-20 03:40:17,156::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914770::DEBUG::2013-11-20 03:40:17,156::task::495::TaskManager.Task::(_debug) Task fc28c667-4eba-481f-973a-d7165b06eddf: moving from state init -> state preparing
>Thread-15914770::DEBUG::2013-11-20 03:40:17,158::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c5ef425a-a193-48c0-b0ed-ff5209cd3f0d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914770::DEBUG::2013-11-20 03:40:17,159::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914770::DEBUG::2013-11-20 03:40:17,159::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914770::DEBUG::2013-11-20 03:40:17,160::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c5ef425a-a193-48c0-b0ed-ff5209cd3f0d`::Granted request
>Thread-15914770::DEBUG::2013-11-20 03:40:17,161::task::495::TaskManager.Task::(_debug) Task fc28c667-4eba-481f-973a-d7165b06eddf: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914770::DEBUG::2013-11-20 03:40:17,162::task::495::TaskManager.Task::(_debug) Task fc28c667-4eba-481f-973a-d7165b06eddf: ref 1 aborting False
>Thread-15914770::WARNING::2013-11-20 03:40:17,163::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914770::DEBUG::2013-11-20 03:40:17,164::task::495::TaskManager.Task::(_debug) Task fc28c667-4eba-481f-973a-d7165b06eddf: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914770::DEBUG::2013-11-20 03:40:17,165::task::495::TaskManager.Task::(_debug) Task fc28c667-4eba-481f-973a-d7165b06eddf: moving from state preparing -> state finished
>Thread-15914770::DEBUG::2013-11-20 03:40:17,166::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914770::DEBUG::2013-11-20 03:40:17,166::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914770::DEBUG::2013-11-20 03:40:17,167::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914770::DEBUG::2013-11-20 03:40:17,168::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914770::DEBUG::2013-11-20 03:40:17,168::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914770::DEBUG::2013-11-20 03:40:17,169::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914770::DEBUG::2013-11-20 03:40:17,170::task::495::TaskManager.Task::(_debug) Task fc28c667-4eba-481f-973a-d7165b06eddf: ref 0 aborting False
>Thread-15914770::INFO::2013-11-20 03:40:17,170::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:40:17,981::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:40:18,505::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0772749 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:40:20,521::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:40:21,088::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0905284 s, 11.3 MB/s\n'; <rc> = 0
>Thread-15914772::INFO::2013-11-20 03:40:21,865::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914772::DEBUG::2013-11-20 03:40:21,866::task::495::TaskManager.Task::(_debug) Task dade7a31-5190-4242-aa72-849d681cd749: moving from state init -> state preparing
>Thread-15914772::DEBUG::2013-11-20 03:40:21,867::task::495::TaskManager.Task::(_debug) Task dade7a31-5190-4242-aa72-849d681cd749: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000437021255493', 'lastCheck': 1384918817.0693901, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00783109664917', 'lastCheck': 1384918811.9287269, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0135209560394', 'lastCheck': 1384918818.218282, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000478982925415', 'lastCheck': 1384918821.7371399, 'valid': True, 'code': 0}}
>Thread-15914772::DEBUG::2013-11-20 03:40:21,868::task::495::TaskManager.Task::(_debug) Task dade7a31-5190-4242-aa72-849d681cd749: moving from state preparing -> state finished
>Thread-15914772::DEBUG::2013-11-20 03:40:21,869::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914772::DEBUG::2013-11-20 03:40:21,869::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914772::DEBUG::2013-11-20 03:40:21,870::task::495::TaskManager.Task::(_debug) Task dade7a31-5190-4242-aa72-849d681cd749: ref 0 aborting False
>Thread-15914772::INFO::2013-11-20 03:40:21,871::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000437021255493', 'lastCheck': 1384918817.0693901, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00783109664917', 'lastCheck': 1384918811.9287269, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0135209560394', 'lastCheck': 1384918818.218282, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000478982925415', 'lastCheck': 1384918821.7371399, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:40:23,098::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:40:23,639::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0786303 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:40:25,654::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 03:40:25,871::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 03:40:25,873::task::495::TaskManager.Task::(_debug) Task cafe54e5-e1e8-46a7-a2fb-a582c046831c: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 03:40:25,876::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1bbdc8a2-7fed-4be4-a454-abef4b90678b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 03:40:25,877::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 03:40:25,878::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 03:40:25,878::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1bbdc8a2-7fed-4be4-a454-abef4b90678b`::Granted request
>Thread-15914435::DEBUG::2013-11-20 03:40:25,880::task::495::TaskManager.Task::(_debug) Task cafe54e5-e1e8-46a7-a2fb-a582c046831c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 03:40:25,881::task::495::TaskManager.Task::(_debug) Task cafe54e5-e1e8-46a7-a2fb-a582c046831c: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 03:40:25,882::task::495::TaskManager.Task::(_debug) Task cafe54e5-e1e8-46a7-a2fb-a582c046831c: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 03:40:25,882::task::495::TaskManager.Task::(_debug) Task cafe54e5-e1e8-46a7-a2fb-a582c046831c: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 03:40:25,883::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 03:40:25,884::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 03:40:25,885::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 03:40:25,886::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 03:40:25,886::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 03:40:25,887::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 03:40:25,888::task::495::TaskManager.Task::(_debug) Task cafe54e5-e1e8-46a7-a2fb-a582c046831c: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 03:40:25,888::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 03:40:26,218::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0793789 s, 12.9 MB/s\n'; <rc> = 0
>Thread-15914775::DEBUG::2013-11-20 03:40:27,999::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914775::INFO::2013-11-20 03:40:28,000::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914775::DEBUG::2013-11-20 03:40:28,001::task::495::TaskManager.Task::(_debug) Task 0214c0d3-ba9a-4719-933f-2998a9599e96: moving from state init -> state preparing
>Thread-15914775::DEBUG::2013-11-20 03:40:28,002::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914775::DEBUG::2013-11-20 03:40:28,003::task::495::TaskManager.Task::(_debug) Task 0214c0d3-ba9a-4719-933f-2998a9599e96: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914775::DEBUG::2013-11-20 03:40:28,004::task::495::TaskManager.Task::(_debug) Task 0214c0d3-ba9a-4719-933f-2998a9599e96: moving from state preparing -> state finished
>Thread-15914775::DEBUG::2013-11-20 03:40:28,005::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914775::DEBUG::2013-11-20 03:40:28,006::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914775::DEBUG::2013-11-20 03:40:28,007::task::495::TaskManager.Task::(_debug) Task 0214c0d3-ba9a-4719-933f-2998a9599e96: ref 0 aborting False
>Thread-15914775::INFO::2013-11-20 03:40:28,008::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914776::DEBUG::2013-11-20 03:40:28,020::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914776::INFO::2013-11-20 03:40:28,021::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914776::DEBUG::2013-11-20 03:40:28,021::task::495::TaskManager.Task::(_debug) Task 77c04b52-0969-4a03-b0de-463b5bf2809b: moving from state init -> state preparing
>Thread-15914776::DEBUG::2013-11-20 03:40:28,023::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ce406999-8b4d-4711-a290-bddd605d1a94`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914776::DEBUG::2013-11-20 03:40:28,023::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914776::DEBUG::2013-11-20 03:40:28,024::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914776::DEBUG::2013-11-20 03:40:28,025::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ce406999-8b4d-4711-a290-bddd605d1a94`::Granted request
>Thread-15914776::DEBUG::2013-11-20 03:40:28,026::task::495::TaskManager.Task::(_debug) Task 77c04b52-0969-4a03-b0de-463b5bf2809b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914776::DEBUG::2013-11-20 03:40:28,026::task::495::TaskManager.Task::(_debug) Task 77c04b52-0969-4a03-b0de-463b5bf2809b: ref 1 aborting False
>Thread-15914776::WARNING::2013-11-20 03:40:28,028::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914776::DEBUG::2013-11-20 03:40:28,029::task::495::TaskManager.Task::(_debug) Task 77c04b52-0969-4a03-b0de-463b5bf2809b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914776::DEBUG::2013-11-20 03:40:28,030::task::495::TaskManager.Task::(_debug) Task 77c04b52-0969-4a03-b0de-463b5bf2809b: moving from state preparing -> state finished
>Thread-15914776::DEBUG::2013-11-20 03:40:28,031::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914776::DEBUG::2013-11-20 03:40:28,031::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914776::DEBUG::2013-11-20 03:40:28,032::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914776::DEBUG::2013-11-20 03:40:28,033::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914776::DEBUG::2013-11-20 03:40:28,033::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914776::DEBUG::2013-11-20 03:40:28,034::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914776::DEBUG::2013-11-20 03:40:28,035::task::495::TaskManager.Task::(_debug) Task 77c04b52-0969-4a03-b0de-463b5bf2809b: ref 0 aborting False
>Thread-15914776::INFO::2013-11-20 03:40:28,035::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:40:28,232::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 03:40:28,444::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 03:40:28,450::task::495::TaskManager.Task::(_debug) Task 9fe72cf2-cf29-44af-9801-265d40c55455: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 03:40:28,456::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`44e9a9b5-fdc0-4f3f-9b22-0a5f08691a6c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 03:40:28,461::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 03:40:28,464::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 03:40:28,465::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`44e9a9b5-fdc0-4f3f-9b22-0a5f08691a6c`::Granted request
>Thread-15914453::DEBUG::2013-11-20 03:40:28,466::task::495::TaskManager.Task::(_debug) Task 9fe72cf2-cf29-44af-9801-265d40c55455: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 03:40:28,467::task::495::TaskManager.Task::(_debug) Task 9fe72cf2-cf29-44af-9801-265d40c55455: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 03:40:28,468::task::495::TaskManager.Task::(_debug) Task 9fe72cf2-cf29-44af-9801-265d40c55455: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 03:40:28,469::task::495::TaskManager.Task::(_debug) Task 9fe72cf2-cf29-44af-9801-265d40c55455: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 03:40:28,470::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 03:40:28,471::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 03:40:28,472::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 03:40:28,473::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 03:40:28,473::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 03:40:28,474::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 03:40:28,475::task::495::TaskManager.Task::(_debug) Task 9fe72cf2-cf29-44af-9801-265d40c55455: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 03:40:28,476::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 03:40:28,771::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0827744 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:40:30,787::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:40:31,319::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0792759 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:40:33,335::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 03:40:33,529::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 03:40:33,532::task::495::TaskManager.Task::(_debug) Task 8420cafb-6810-48a0-9fdd-ba6c848a7f73: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 03:40:33,543::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9756df82-dbf3-4b93-af11-693937633851`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 03:40:33,544::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 03:40:33,545::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 03:40:33,546::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9756df82-dbf3-4b93-af11-693937633851`::Granted request
>Thread-15914519::DEBUG::2013-11-20 03:40:33,547::task::495::TaskManager.Task::(_debug) Task 8420cafb-6810-48a0-9fdd-ba6c848a7f73: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 03:40:33,548::task::495::TaskManager.Task::(_debug) Task 8420cafb-6810-48a0-9fdd-ba6c848a7f73: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 03:40:33,549::task::495::TaskManager.Task::(_debug) Task 8420cafb-6810-48a0-9fdd-ba6c848a7f73: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 03:40:33,550::task::495::TaskManager.Task::(_debug) Task 8420cafb-6810-48a0-9fdd-ba6c848a7f73: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 03:40:33,551::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 03:40:33,557::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 03:40:33,557::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 03:40:33,558::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 03:40:33,559::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 03:40:33,559::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 03:40:33,560::task::495::TaskManager.Task::(_debug) Task 8420cafb-6810-48a0-9fdd-ba6c848a7f73: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 03:40:33,561::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 03:40:33,871::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0843173 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:40:35,887::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:40:36,436::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0739761 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 03:40:37,071::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:40:37,072::task::495::TaskManager.Task::(_debug) Task 702560b2-d3db-493e-8510-3d2fe3e1bf94: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:40:37,074::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f88f0176-4b97-411a-8f0b-d5df9bf87518`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:40:37,075::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:40:37,076::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:40:37,076::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f88f0176-4b97-411a-8f0b-d5df9bf87518`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:40:37,090::task::495::TaskManager.Task::(_debug) Task 702560b2-d3db-493e-8510-3d2fe3e1bf94: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:40:37,091::task::495::TaskManager.Task::(_debug) Task 702560b2-d3db-493e-8510-3d2fe3e1bf94: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:40:37,092::task::495::TaskManager.Task::(_debug) Task 702560b2-d3db-493e-8510-3d2fe3e1bf94: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:40:37,093::task::495::TaskManager.Task::(_debug) Task 702560b2-d3db-493e-8510-3d2fe3e1bf94: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:40:37,094::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:40:37,095::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:40:37,096::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:40:37,096::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:40:37,097::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:40:37,098::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:40:37,099::task::495::TaskManager.Task::(_debug) Task 702560b2-d3db-493e-8510-3d2fe3e1bf94: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:40:37,099::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:40:37,101::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:40:37,101::task::495::TaskManager.Task::(_debug) Task e9824903-883a-49b6-a893-f673fd160f95: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:40:37,102::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d46dee8f-e506-4df0-a8ec-027ceb4adb8a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:40:37,103::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:40:37,103::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:40:37,104::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d46dee8f-e506-4df0-a8ec-027ceb4adb8a`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:40:37,105::task::495::TaskManager.Task::(_debug) Task e9824903-883a-49b6-a893-f673fd160f95: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:40:37,106::task::495::TaskManager.Task::(_debug) Task e9824903-883a-49b6-a893-f673fd160f95: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:40:37,107::task::495::TaskManager.Task::(_debug) Task e9824903-883a-49b6-a893-f673fd160f95: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:40:37,107::task::495::TaskManager.Task::(_debug) Task e9824903-883a-49b6-a893-f673fd160f95: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:40:37,108::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:40:37,108::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:40:37,109::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:40:37,110::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:40:37,110::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:40:37,111::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:40:37,112::task::495::TaskManager.Task::(_debug) Task e9824903-883a-49b6-a893-f673fd160f95: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:40:37,113::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15914780::INFO::2013-11-20 03:40:37,530::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914780::DEBUG::2013-11-20 03:40:37,531::task::495::TaskManager.Task::(_debug) Task 5afc6a19-4529-48f6-9c4e-78ef92baa4bb: moving from state init -> state preparing
>Thread-15914780::DEBUG::2013-11-20 03:40:37,532::task::495::TaskManager.Task::(_debug) Task 5afc6a19-4529-48f6-9c4e-78ef92baa4bb: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00038480758667', 'lastCheck': 1384918827.304878, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0174870491028', 'lastCheck': 1384918832.4591911, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0123951435089', 'lastCheck': 1384918828.4757431, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000436067581177', 'lastCheck': 1384918831.9872079, 'valid': True, 'code': 0}}
>Thread-15914780::DEBUG::2013-11-20 03:40:37,533::task::495::TaskManager.Task::(_debug) Task 5afc6a19-4529-48f6-9c4e-78ef92baa4bb: moving from state preparing -> state finished
>Thread-15914780::DEBUG::2013-11-20 03:40:37,534::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914780::DEBUG::2013-11-20 03:40:37,535::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914780::DEBUG::2013-11-20 03:40:37,535::task::495::TaskManager.Task::(_debug) Task 5afc6a19-4529-48f6-9c4e-78ef92baa4bb: ref 0 aborting False
>Thread-15914780::INFO::2013-11-20 03:40:37,536::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00038480758667', 'lastCheck': 1384918827.304878, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0174870491028', 'lastCheck': 1384918832.4591911, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0123951435089', 'lastCheck': 1384918828.4757431, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000436067581177', 'lastCheck': 1384918831.9872079, 'valid': True, 'code': 0}}
>Thread-15914782::DEBUG::2013-11-20 03:40:38,410::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914782::INFO::2013-11-20 03:40:38,413::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914782::DEBUG::2013-11-20 03:40:38,414::task::495::TaskManager.Task::(_debug) Task 9ac97937-0ff3-4199-a4bc-ce7c058119de: moving from state init -> state preparing
>Thread-15914782::DEBUG::2013-11-20 03:40:38,416::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914782::DEBUG::2013-11-20 03:40:38,417::task::495::TaskManager.Task::(_debug) Task 9ac97937-0ff3-4199-a4bc-ce7c058119de: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914782::DEBUG::2013-11-20 03:40:38,419::task::495::TaskManager.Task::(_debug) Task 9ac97937-0ff3-4199-a4bc-ce7c058119de: moving from state preparing -> state finished
>Thread-15914782::DEBUG::2013-11-20 03:40:38,419::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914782::DEBUG::2013-11-20 03:40:38,420::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914782::DEBUG::2013-11-20 03:40:38,420::task::495::TaskManager.Task::(_debug) Task 9ac97937-0ff3-4199-a4bc-ce7c058119de: ref 0 aborting False
>Thread-15914782::INFO::2013-11-20 03:40:38,421::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914783::DEBUG::2013-11-20 03:40:38,435::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914783::INFO::2013-11-20 03:40:38,436::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914783::DEBUG::2013-11-20 03:40:38,437::task::495::TaskManager.Task::(_debug) Task d622e7ed-a780-445d-b9ab-f9121783478b: moving from state init -> state preparing
>Thread-15914783::DEBUG::2013-11-20 03:40:38,438::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7a2f15d6-f846-468a-9b13-5449d722986f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914783::DEBUG::2013-11-20 03:40:38,439::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914783::DEBUG::2013-11-20 03:40:38,439::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914783::DEBUG::2013-11-20 03:40:38,440::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7a2f15d6-f846-468a-9b13-5449d722986f`::Granted request
>Thread-15914783::DEBUG::2013-11-20 03:40:38,441::task::495::TaskManager.Task::(_debug) Task d622e7ed-a780-445d-b9ab-f9121783478b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914783::DEBUG::2013-11-20 03:40:38,442::task::495::TaskManager.Task::(_debug) Task d622e7ed-a780-445d-b9ab-f9121783478b: ref 1 aborting False
>Thread-15914783::WARNING::2013-11-20 03:40:38,444::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914783::DEBUG::2013-11-20 03:40:38,445::task::495::TaskManager.Task::(_debug) Task d622e7ed-a780-445d-b9ab-f9121783478b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914783::DEBUG::2013-11-20 03:40:38,445::task::495::TaskManager.Task::(_debug) Task d622e7ed-a780-445d-b9ab-f9121783478b: moving from state preparing -> state finished
>Thread-15914783::DEBUG::2013-11-20 03:40:38,446::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914783::DEBUG::2013-11-20 03:40:38,447::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914783::DEBUG::2013-11-20 03:40:38,447::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914783::DEBUG::2013-11-20 03:40:38,448::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914783::DEBUG::2013-11-20 03:40:38,449::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914783::DEBUG::2013-11-20 03:40:38,449::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914783::DEBUG::2013-11-20 03:40:38,450::task::495::TaskManager.Task::(_debug) Task d622e7ed-a780-445d-b9ab-f9121783478b: ref 0 aborting False
>Thread-15914783::INFO::2013-11-20 03:40:38,451::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:40:38,457::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 03:40:38,652::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:40:38,656::task::495::TaskManager.Task::(_debug) Task 188c52bf-cc1d-424e-9571-0e144cff8562: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:40:38,659::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8fd753f4-43ed-4e67-b992-ac7f1672d4d6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:40:38,660::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:40:38,662::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:40:38,663::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8fd753f4-43ed-4e67-b992-ac7f1672d4d6`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:40:38,664::task::495::TaskManager.Task::(_debug) Task 188c52bf-cc1d-424e-9571-0e144cff8562: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:40:38,664::task::495::TaskManager.Task::(_debug) Task 188c52bf-cc1d-424e-9571-0e144cff8562: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:40:38,665::task::495::TaskManager.Task::(_debug) Task 188c52bf-cc1d-424e-9571-0e144cff8562: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:40:38,666::task::495::TaskManager.Task::(_debug) Task 188c52bf-cc1d-424e-9571-0e144cff8562: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:40:38,667::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:40:38,667::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:40:38,668::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:40:38,669::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:40:38,669::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:40:38,670::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:40:38,670::task::495::TaskManager.Task::(_debug) Task 188c52bf-cc1d-424e-9571-0e144cff8562: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:40:38,671::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:40:38,672::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:40:38,672::task::495::TaskManager.Task::(_debug) Task 3b5f7aab-de1c-4efc-b396-72e747445455: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:40:38,674::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`150f16e1-f112-47ab-ac66-b4b31af22abf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:40:38,674::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:40:38,675::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:40:38,675::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`150f16e1-f112-47ab-ac66-b4b31af22abf`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:40:38,676::task::495::TaskManager.Task::(_debug) Task 3b5f7aab-de1c-4efc-b396-72e747445455: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:40:38,677::task::495::TaskManager.Task::(_debug) Task 3b5f7aab-de1c-4efc-b396-72e747445455: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:40:38,678::task::495::TaskManager.Task::(_debug) Task 3b5f7aab-de1c-4efc-b396-72e747445455: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:40:38,678::task::495::TaskManager.Task::(_debug) Task 3b5f7aab-de1c-4efc-b396-72e747445455: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:40:38,679::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:40:38,679::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:40:38,680::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:40:38,681::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:40:38,681::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:40:38,682::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:40:38,682::task::495::TaskManager.Task::(_debug) Task 3b5f7aab-de1c-4efc-b396-72e747445455: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:40:38,683::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:40:38,684::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:40:38,684::task::495::TaskManager.Task::(_debug) Task 393dc692-daa4-4f9d-84d9-5e8628a7b844: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:40:38,685::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c19f7f10-7c02-43c0-9dfc-618bf3b7d432`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:40:38,686::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:40:38,687::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:40:38,687::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c19f7f10-7c02-43c0-9dfc-618bf3b7d432`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:40:38,688::task::495::TaskManager.Task::(_debug) Task 393dc692-daa4-4f9d-84d9-5e8628a7b844: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:40:38,689::task::495::TaskManager.Task::(_debug) Task 393dc692-daa4-4f9d-84d9-5e8628a7b844: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:40:38,689::task::495::TaskManager.Task::(_debug) Task 393dc692-daa4-4f9d-84d9-5e8628a7b844: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:40:38,690::task::495::TaskManager.Task::(_debug) Task 393dc692-daa4-4f9d-84d9-5e8628a7b844: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:40:38,691::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:40:38,691::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:40:38,692::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:40:38,692::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:40:38,693::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:40:38,693::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:40:38,694::task::495::TaskManager.Task::(_debug) Task 393dc692-daa4-4f9d-84d9-5e8628a7b844: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:40:38,695::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:40:38,695::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:40:38,696::task::495::TaskManager.Task::(_debug) Task 6414f5ed-122e-453b-b742-15a14f556a4d: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:40:38,697::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`1669781b-53bb-42c3-966b-d52178fcf43f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:40:38,698::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:40:38,698::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:40:38,699::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`1669781b-53bb-42c3-966b-d52178fcf43f`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:40:38,700::task::495::TaskManager.Task::(_debug) Task 6414f5ed-122e-453b-b742-15a14f556a4d: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:40:38,700::task::495::TaskManager.Task::(_debug) Task 6414f5ed-122e-453b-b742-15a14f556a4d: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:40:38,701::task::495::TaskManager.Task::(_debug) Task 6414f5ed-122e-453b-b742-15a14f556a4d: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:40:38,702::task::495::TaskManager.Task::(_debug) Task 6414f5ed-122e-453b-b742-15a14f556a4d: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:40:38,702::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:40:38,703::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:40:38,703::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:40:38,704::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:40:38,705::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:40:38,705::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:40:38,706::task::495::TaskManager.Task::(_debug) Task 6414f5ed-122e-453b-b742-15a14f556a4d: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:40:38,706::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:40:38,983::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0879304 s, 11.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:40:40,999::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:40:41,561::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0880167 s, 11.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:40:43,577::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:40:44,104::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0751755 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:40:46,119::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:40:46,660::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0861763 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:40:48,676::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914787::DEBUG::2013-11-20 03:40:48,904::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914787::INFO::2013-11-20 03:40:48,908::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914787::DEBUG::2013-11-20 03:40:48,909::task::495::TaskManager.Task::(_debug) Task 75404294-d877-4418-93e7-88fa42594de1: moving from state init -> state preparing
>Thread-15914787::DEBUG::2013-11-20 03:40:48,910::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914787::DEBUG::2013-11-20 03:40:48,911::task::495::TaskManager.Task::(_debug) Task 75404294-d877-4418-93e7-88fa42594de1: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914787::DEBUG::2013-11-20 03:40:48,911::task::495::TaskManager.Task::(_debug) Task 75404294-d877-4418-93e7-88fa42594de1: moving from state preparing -> state finished
>Thread-15914787::DEBUG::2013-11-20 03:40:48,912::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914787::DEBUG::2013-11-20 03:40:48,912::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914787::DEBUG::2013-11-20 03:40:48,913::task::495::TaskManager.Task::(_debug) Task 75404294-d877-4418-93e7-88fa42594de1: ref 0 aborting False
>Thread-15914787::INFO::2013-11-20 03:40:48,913::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914788::DEBUG::2013-11-20 03:40:48,918::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914788::INFO::2013-11-20 03:40:48,920::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914788::DEBUG::2013-11-20 03:40:48,920::task::495::TaskManager.Task::(_debug) Task 5b9cd077-5994-4498-b4ff-a09eb8363321: moving from state init -> state preparing
>Thread-15914788::DEBUG::2013-11-20 03:40:48,922::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b0b5d24f-ce3d-4a24-b302-3f17bc523ec9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914788::DEBUG::2013-11-20 03:40:48,923::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914788::DEBUG::2013-11-20 03:40:48,925::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914788::DEBUG::2013-11-20 03:40:48,926::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b0b5d24f-ce3d-4a24-b302-3f17bc523ec9`::Granted request
>Thread-15914788::DEBUG::2013-11-20 03:40:48,927::task::495::TaskManager.Task::(_debug) Task 5b9cd077-5994-4498-b4ff-a09eb8363321: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914788::DEBUG::2013-11-20 03:40:48,927::task::495::TaskManager.Task::(_debug) Task 5b9cd077-5994-4498-b4ff-a09eb8363321: ref 1 aborting False
>Thread-15914788::WARNING::2013-11-20 03:40:48,930::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914788::DEBUG::2013-11-20 03:40:48,931::task::495::TaskManager.Task::(_debug) Task 5b9cd077-5994-4498-b4ff-a09eb8363321: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914788::DEBUG::2013-11-20 03:40:48,932::task::495::TaskManager.Task::(_debug) Task 5b9cd077-5994-4498-b4ff-a09eb8363321: moving from state preparing -> state finished
>Thread-15914788::DEBUG::2013-11-20 03:40:48,933::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914788::DEBUG::2013-11-20 03:40:48,934::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914788::DEBUG::2013-11-20 03:40:48,935::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914788::DEBUG::2013-11-20 03:40:48,935::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914788::DEBUG::2013-11-20 03:40:48,936::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914788::DEBUG::2013-11-20 03:40:48,937::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914788::DEBUG::2013-11-20 03:40:48,937::task::495::TaskManager.Task::(_debug) Task 5b9cd077-5994-4498-b4ff-a09eb8363321: ref 0 aborting False
>Thread-15914788::INFO::2013-11-20 03:40:48,938::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:40:49,219::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.074813 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:40:51,239::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:40:51,790::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.077913 s, 13.1 MB/s\n'; <rc> = 0
>Thread-15914790::INFO::2013-11-20 03:40:53,355::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914790::DEBUG::2013-11-20 03:40:53,358::task::495::TaskManager.Task::(_debug) Task 4bce7c82-0fec-45e9-957c-aec5cbc01649: moving from state init -> state preparing
>Thread-15914790::DEBUG::2013-11-20 03:40:53,369::task::495::TaskManager.Task::(_debug) Task 4bce7c82-0fec-45e9-957c-aec5cbc01649: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000405788421631', 'lastCheck': 1384918847.7788, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148606300354', 'lastCheck': 1384918852.8972321, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0170228481293', 'lastCheck': 1384918848.9237039, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000508069992065', 'lastCheck': 1384918852.462908, 'valid': True, 'code': 0}}
>Thread-15914790::DEBUG::2013-11-20 03:40:53,371::task::495::TaskManager.Task::(_debug) Task 4bce7c82-0fec-45e9-957c-aec5cbc01649: moving from state preparing -> state finished
>Thread-15914790::DEBUG::2013-11-20 03:40:53,372::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914790::DEBUG::2013-11-20 03:40:53,374::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914790::DEBUG::2013-11-20 03:40:53,377::task::495::TaskManager.Task::(_debug) Task 4bce7c82-0fec-45e9-957c-aec5cbc01649: ref 0 aborting False
>Thread-15914790::INFO::2013-11-20 03:40:53,381::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000405788421631', 'lastCheck': 1384918847.7788, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148606300354', 'lastCheck': 1384918852.8972321, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0170228481293', 'lastCheck': 1384918848.9237039, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000508069992065', 'lastCheck': 1384918852.462908, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:40:53,805::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 03:40:54,060::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:40:54,076::task::495::TaskManager.Task::(_debug) Task 83da3fe3-e9b8-4630-a0bb-3eec84d41687: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:40:54,079::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`299eea5f-1359-4e3e-b47b-c82387fe5f6b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:40:54,081::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:40:54,083::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:40:54,083::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`299eea5f-1359-4e3e-b47b-c82387fe5f6b`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:40:54,085::task::495::TaskManager.Task::(_debug) Task 83da3fe3-e9b8-4630-a0bb-3eec84d41687: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:40:54,086::task::495::TaskManager.Task::(_debug) Task 83da3fe3-e9b8-4630-a0bb-3eec84d41687: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:40:54,086::task::495::TaskManager.Task::(_debug) Task 83da3fe3-e9b8-4630-a0bb-3eec84d41687: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:40:54,087::task::495::TaskManager.Task::(_debug) Task 83da3fe3-e9b8-4630-a0bb-3eec84d41687: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:40:54,088::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:40:54,088::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:40:54,089::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:40:54,090::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:40:54,090::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:40:54,091::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:40:54,092::task::495::TaskManager.Task::(_debug) Task 83da3fe3-e9b8-4630-a0bb-3eec84d41687: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:40:54,092::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:40:54,354::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0725574 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:40:56,369::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:40:56,943::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0746765 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:40:58,959::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914793::DEBUG::2013-11-20 03:40:59,187::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914793::INFO::2013-11-20 03:40:59,188::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914793::DEBUG::2013-11-20 03:40:59,189::task::495::TaskManager.Task::(_debug) Task 0d12acbe-53bd-4954-84ab-c967cce7b373: moving from state init -> state preparing
>Thread-15914793::DEBUG::2013-11-20 03:40:59,189::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914793::DEBUG::2013-11-20 03:40:59,190::task::495::TaskManager.Task::(_debug) Task 0d12acbe-53bd-4954-84ab-c967cce7b373: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914793::DEBUG::2013-11-20 03:40:59,190::task::495::TaskManager.Task::(_debug) Task 0d12acbe-53bd-4954-84ab-c967cce7b373: moving from state preparing -> state finished
>Thread-15914793::DEBUG::2013-11-20 03:40:59,191::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914793::DEBUG::2013-11-20 03:40:59,192::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914793::DEBUG::2013-11-20 03:40:59,193::task::495::TaskManager.Task::(_debug) Task 0d12acbe-53bd-4954-84ab-c967cce7b373: ref 0 aborting False
>Thread-15914793::INFO::2013-11-20 03:40:59,194::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914794::DEBUG::2013-11-20 03:40:59,205::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914794::INFO::2013-11-20 03:40:59,206::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914794::DEBUG::2013-11-20 03:40:59,206::task::495::TaskManager.Task::(_debug) Task c839a29b-e456-497a-9fab-993f45e0c33d: moving from state init -> state preparing
>Thread-15914794::DEBUG::2013-11-20 03:40:59,208::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ef67be1b-c38a-4d46-a271-6c32b1eabf59`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914794::DEBUG::2013-11-20 03:40:59,209::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914794::DEBUG::2013-11-20 03:40:59,210::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914794::DEBUG::2013-11-20 03:40:59,211::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ef67be1b-c38a-4d46-a271-6c32b1eabf59`::Granted request
>Thread-15914794::DEBUG::2013-11-20 03:40:59,212::task::495::TaskManager.Task::(_debug) Task c839a29b-e456-497a-9fab-993f45e0c33d: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914794::DEBUG::2013-11-20 03:40:59,213::task::495::TaskManager.Task::(_debug) Task c839a29b-e456-497a-9fab-993f45e0c33d: ref 1 aborting False
>Thread-15914794::WARNING::2013-11-20 03:40:59,216::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914794::DEBUG::2013-11-20 03:40:59,217::task::495::TaskManager.Task::(_debug) Task c839a29b-e456-497a-9fab-993f45e0c33d: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914794::DEBUG::2013-11-20 03:40:59,218::task::495::TaskManager.Task::(_debug) Task c839a29b-e456-497a-9fab-993f45e0c33d: moving from state preparing -> state finished
>Thread-15914794::DEBUG::2013-11-20 03:40:59,219::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914794::DEBUG::2013-11-20 03:40:59,219::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914794::DEBUG::2013-11-20 03:40:59,220::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914794::DEBUG::2013-11-20 03:40:59,221::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914794::DEBUG::2013-11-20 03:40:59,222::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914794::DEBUG::2013-11-20 03:40:59,222::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914794::DEBUG::2013-11-20 03:40:59,223::task::495::TaskManager.Task::(_debug) Task c839a29b-e456-497a-9fab-993f45e0c33d: ref 0 aborting False
>Thread-15914794::INFO::2013-11-20 03:40:59,224::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:40:59,508::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0866634 s, 11.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:41:01,527::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:41:02,091::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0876911 s, 11.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:41:04,101::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:41:04,659::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0791111 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:41:06,679::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:41:07,258::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0868171 s, 11.8 MB/s\n'; <rc> = 0
>Thread-15914798::INFO::2013-11-20 03:41:09,076::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914798::DEBUG::2013-11-20 03:41:09,081::task::495::TaskManager.Task::(_debug) Task acf4ed13-7a24-4921-a4e0-43e65c26cc4e: moving from state init -> state preparing
>Thread-15914798::DEBUG::2013-11-20 03:41:09,086::task::495::TaskManager.Task::(_debug) Task acf4ed13-7a24-4921-a4e0-43e65c26cc4e: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000391006469727', 'lastCheck': 1384918868.3336909, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0112571716309', 'lastCheck': 1384918863.1716111, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0020010471344', 'lastCheck': 1384918859.1727719, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000478029251099', 'lastCheck': 1384918862.7409871, 'valid': True, 'code': 0}}
>Thread-15914798::DEBUG::2013-11-20 03:41:09,088::task::495::TaskManager.Task::(_debug) Task acf4ed13-7a24-4921-a4e0-43e65c26cc4e: moving from state preparing -> state finished
>Thread-15914798::DEBUG::2013-11-20 03:41:09,089::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914798::DEBUG::2013-11-20 03:41:09,090::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914798::DEBUG::2013-11-20 03:41:09,090::task::495::TaskManager.Task::(_debug) Task acf4ed13-7a24-4921-a4e0-43e65c26cc4e: ref 0 aborting False
>Thread-15914798::INFO::2013-11-20 03:41:09,091::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000391006469727', 'lastCheck': 1384918868.3336909, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0112571716309', 'lastCheck': 1384918863.1716111, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0020010471344', 'lastCheck': 1384918859.1727719, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000478029251099', 'lastCheck': 1384918862.7409871, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:41:09,273::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 03:41:09,498::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:41:09,504::task::495::TaskManager.Task::(_debug) Task c80f8916-e01c-4ec9-9645-03ed399f274e: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:41:09,508::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e5687289-b9e8-4c14-ba16-01d2f1ddabed`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:41:09,510::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:41:09,512::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:41:09,512::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e5687289-b9e8-4c14-ba16-01d2f1ddabed`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:41:09,514::task::495::TaskManager.Task::(_debug) Task c80f8916-e01c-4ec9-9645-03ed399f274e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:41:09,514::task::495::TaskManager.Task::(_debug) Task c80f8916-e01c-4ec9-9645-03ed399f274e: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:41:09,515::task::495::TaskManager.Task::(_debug) Task c80f8916-e01c-4ec9-9645-03ed399f274e: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:41:09,516::task::495::TaskManager.Task::(_debug) Task c80f8916-e01c-4ec9-9645-03ed399f274e: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:41:09,517::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:41:09,517::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:41:09,518::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:41:09,519::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:41:09,519::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:41:09,520::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:41:09,521::task::495::TaskManager.Task::(_debug) Task c80f8916-e01c-4ec9-9645-03ed399f274e: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:41:09,521::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15914800::DEBUG::2013-11-20 03:41:09,579::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914800::INFO::2013-11-20 03:41:09,580::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914800::DEBUG::2013-11-20 03:41:09,581::task::495::TaskManager.Task::(_debug) Task dcc84584-fd45-4906-a253-fbf58e316621: moving from state init -> state preparing
>Thread-15914800::DEBUG::2013-11-20 03:41:09,582::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914800::DEBUG::2013-11-20 03:41:09,582::task::495::TaskManager.Task::(_debug) Task dcc84584-fd45-4906-a253-fbf58e316621: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914800::DEBUG::2013-11-20 03:41:09,583::task::495::TaskManager.Task::(_debug) Task dcc84584-fd45-4906-a253-fbf58e316621: moving from state preparing -> state finished
>Thread-15914800::DEBUG::2013-11-20 03:41:09,584::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914800::DEBUG::2013-11-20 03:41:09,585::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914800::DEBUG::2013-11-20 03:41:09,585::task::495::TaskManager.Task::(_debug) Task dcc84584-fd45-4906-a253-fbf58e316621: ref 0 aborting False
>Thread-15914800::INFO::2013-11-20 03:41:09,586::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914801::DEBUG::2013-11-20 03:41:09,593::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914801::INFO::2013-11-20 03:41:09,594::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914801::DEBUG::2013-11-20 03:41:09,594::task::495::TaskManager.Task::(_debug) Task 90a6e22a-4d3b-4745-9328-70808c8a04e0: moving from state init -> state preparing
>Thread-15914801::DEBUG::2013-11-20 03:41:09,596::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f6d286ba-8031-4118-bc1d-05ec2edf0c0e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914801::DEBUG::2013-11-20 03:41:09,596::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914801::DEBUG::2013-11-20 03:41:09,597::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914801::DEBUG::2013-11-20 03:41:09,598::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f6d286ba-8031-4118-bc1d-05ec2edf0c0e`::Granted request
>Thread-15914801::DEBUG::2013-11-20 03:41:09,599::task::495::TaskManager.Task::(_debug) Task 90a6e22a-4d3b-4745-9328-70808c8a04e0: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914801::DEBUG::2013-11-20 03:41:09,599::task::495::TaskManager.Task::(_debug) Task 90a6e22a-4d3b-4745-9328-70808c8a04e0: ref 1 aborting False
>Thread-15914801::WARNING::2013-11-20 03:41:09,602::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914801::DEBUG::2013-11-20 03:41:09,604::task::495::TaskManager.Task::(_debug) Task 90a6e22a-4d3b-4745-9328-70808c8a04e0: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914801::DEBUG::2013-11-20 03:41:09,605::task::495::TaskManager.Task::(_debug) Task 90a6e22a-4d3b-4745-9328-70808c8a04e0: moving from state preparing -> state finished
>Thread-15914801::DEBUG::2013-11-20 03:41:09,605::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914801::DEBUG::2013-11-20 03:41:09,606::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914801::DEBUG::2013-11-20 03:41:09,607::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914801::DEBUG::2013-11-20 03:41:09,608::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914801::DEBUG::2013-11-20 03:41:09,608::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914801::DEBUG::2013-11-20 03:41:09,609::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914801::DEBUG::2013-11-20 03:41:09,610::task::495::TaskManager.Task::(_debug) Task 90a6e22a-4d3b-4745-9328-70808c8a04e0: ref 0 aborting False
>Thread-15914801::INFO::2013-11-20 03:41:09,611::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:41:09,852::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0903065 s, 11.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:41:11,870::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:41:12,454::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0798228 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:41:14,470::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 03:41:14,733::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:41:14,739::task::495::TaskManager.Task::(_debug) Task 1452d6a8-a22e-4e02-8433-c26df3ac9fe4: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:41:14,744::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`50b481e8-6f8c-49d6-8613-bb7b0c231bd3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:41:14,745::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:41:14,746::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:41:14,746::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`50b481e8-6f8c-49d6-8613-bb7b0c231bd3`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:41:14,748::task::495::TaskManager.Task::(_debug) Task 1452d6a8-a22e-4e02-8433-c26df3ac9fe4: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:41:14,748::task::495::TaskManager.Task::(_debug) Task 1452d6a8-a22e-4e02-8433-c26df3ac9fe4: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:41:14,749::task::495::TaskManager.Task::(_debug) Task 1452d6a8-a22e-4e02-8433-c26df3ac9fe4: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:41:14,750::task::495::TaskManager.Task::(_debug) Task 1452d6a8-a22e-4e02-8433-c26df3ac9fe4: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:41:14,751::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:41:14,752::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:41:14,752::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:41:14,753::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:41:14,754::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:41:14,754::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:41:14,755::task::495::TaskManager.Task::(_debug) Task 1452d6a8-a22e-4e02-8433-c26df3ac9fe4: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:41:14,756::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:41:15,017::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0790754 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:41:17,031::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 03:41:17,244::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:41:17,245::task::495::TaskManager.Task::(_debug) Task 87c5c6f9-2baa-427b-a8b6-66da72e4d5bc: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:41:17,248::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ba1b6b31-7466-4f59-a87f-cd8c554ec49f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:41:17,249::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:41:17,250::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:41:17,251::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ba1b6b31-7466-4f59-a87f-cd8c554ec49f`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:41:17,252::task::495::TaskManager.Task::(_debug) Task 87c5c6f9-2baa-427b-a8b6-66da72e4d5bc: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:41:17,253::task::495::TaskManager.Task::(_debug) Task 87c5c6f9-2baa-427b-a8b6-66da72e4d5bc: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:41:17,254::task::495::TaskManager.Task::(_debug) Task 87c5c6f9-2baa-427b-a8b6-66da72e4d5bc: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:41:17,255::task::495::TaskManager.Task::(_debug) Task 87c5c6f9-2baa-427b-a8b6-66da72e4d5bc: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:41:17,256::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:41:17,256::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:41:17,257::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:41:17,258::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:41:17,259::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:41:17,260::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:41:17,261::task::495::TaskManager.Task::(_debug) Task 87c5c6f9-2baa-427b-a8b6-66da72e4d5bc: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:41:17,262::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:41:17,564::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0833699 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:41:19,578::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:41:20,110::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0743475 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15914805::DEBUG::2013-11-20 03:41:20,204::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914805::INFO::2013-11-20 03:41:20,205::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914805::DEBUG::2013-11-20 03:41:20,206::task::495::TaskManager.Task::(_debug) Task 1564052c-a141-44b2-b54f-047967e5041a: moving from state init -> state preparing
>Thread-15914805::DEBUG::2013-11-20 03:41:20,206::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914805::DEBUG::2013-11-20 03:41:20,207::task::495::TaskManager.Task::(_debug) Task 1564052c-a141-44b2-b54f-047967e5041a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914805::DEBUG::2013-11-20 03:41:20,208::task::495::TaskManager.Task::(_debug) Task 1564052c-a141-44b2-b54f-047967e5041a: moving from state preparing -> state finished
>Thread-15914805::DEBUG::2013-11-20 03:41:20,208::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914805::DEBUG::2013-11-20 03:41:20,209::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914805::DEBUG::2013-11-20 03:41:20,210::task::495::TaskManager.Task::(_debug) Task 1564052c-a141-44b2-b54f-047967e5041a: ref 0 aborting False
>Thread-15914805::INFO::2013-11-20 03:41:20,211::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914806::DEBUG::2013-11-20 03:41:20,216::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914806::INFO::2013-11-20 03:41:20,217::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914806::DEBUG::2013-11-20 03:41:20,218::task::495::TaskManager.Task::(_debug) Task d4783517-de0b-44aa-9e8a-f1ceb1423f15: moving from state init -> state preparing
>Thread-15914806::DEBUG::2013-11-20 03:41:20,219::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`fa00c77c-f060-4afc-ac83-c1644ae2f267`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914806::DEBUG::2013-11-20 03:41:20,220::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914806::DEBUG::2013-11-20 03:41:20,221::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914806::DEBUG::2013-11-20 03:41:20,221::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`fa00c77c-f060-4afc-ac83-c1644ae2f267`::Granted request
>Thread-15914806::DEBUG::2013-11-20 03:41:20,222::task::495::TaskManager.Task::(_debug) Task d4783517-de0b-44aa-9e8a-f1ceb1423f15: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914806::DEBUG::2013-11-20 03:41:20,223::task::495::TaskManager.Task::(_debug) Task d4783517-de0b-44aa-9e8a-f1ceb1423f15: ref 1 aborting False
>Thread-15914806::WARNING::2013-11-20 03:41:20,225::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914806::DEBUG::2013-11-20 03:41:20,226::task::495::TaskManager.Task::(_debug) Task d4783517-de0b-44aa-9e8a-f1ceb1423f15: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914806::DEBUG::2013-11-20 03:41:20,227::task::495::TaskManager.Task::(_debug) Task d4783517-de0b-44aa-9e8a-f1ceb1423f15: moving from state preparing -> state finished
>Thread-15914806::DEBUG::2013-11-20 03:41:20,228::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914806::DEBUG::2013-11-20 03:41:20,229::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914806::DEBUG::2013-11-20 03:41:20,229::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914806::DEBUG::2013-11-20 03:41:20,230::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914806::DEBUG::2013-11-20 03:41:20,231::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914806::DEBUG::2013-11-20 03:41:20,231::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914806::DEBUG::2013-11-20 03:41:20,232::task::495::TaskManager.Task::(_debug) Task d4783517-de0b-44aa-9e8a-f1ceb1423f15: ref 0 aborting False
>Thread-15914806::INFO::2013-11-20 03:41:20,233::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:41:22,124::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:41:22,701::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0875029 s, 11.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:41:24,719::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914808::INFO::2013-11-20 03:41:24,966::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914808::DEBUG::2013-11-20 03:41:24,968::task::495::TaskManager.Task::(_debug) Task 7c9dd240-2626-4848-8410-26097a281af3: moving from state init -> state preparing
>Thread-15914808::DEBUG::2013-11-20 03:41:24,970::task::495::TaskManager.Task::(_debug) Task 7c9dd240-2626-4848-8410-26097a281af3: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000357151031494', 'lastCheck': 1384918878.7101481, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0128710269928', 'lastCheck': 1384918883.7938311, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00178194046021', 'lastCheck': 1384918879.7812841, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000530958175659', 'lastCheck': 1384918883.338341, 'valid': True, 'code': 0}}
>Thread-15914808::DEBUG::2013-11-20 03:41:24,972::task::495::TaskManager.Task::(_debug) Task 7c9dd240-2626-4848-8410-26097a281af3: moving from state preparing -> state finished
>Thread-15914808::DEBUG::2013-11-20 03:41:24,974::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914808::DEBUG::2013-11-20 03:41:24,975::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914808::DEBUG::2013-11-20 03:41:24,977::task::495::TaskManager.Task::(_debug) Task 7c9dd240-2626-4848-8410-26097a281af3: ref 0 aborting False
>Thread-15914808::INFO::2013-11-20 03:41:24,977::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000357151031494', 'lastCheck': 1384918878.7101481, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0128710269928', 'lastCheck': 1384918883.7938311, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00178194046021', 'lastCheck': 1384918879.7812841, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000530958175659', 'lastCheck': 1384918883.338341, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:41:25,275::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.081237 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:41:27,292::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 03:41:27,499::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 03:41:27,501::task::495::TaskManager.Task::(_debug) Task 1460d88c-e8c4-492c-8876-4b97c00a469a: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 03:41:27,503::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c90f1535-3584-47d3-9abc-a755c5aaa09d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 03:41:27,504::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 03:41:27,505::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 03:41:27,506::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c90f1535-3584-47d3-9abc-a755c5aaa09d`::Granted request
>Thread-15914435::DEBUG::2013-11-20 03:41:27,507::task::495::TaskManager.Task::(_debug) Task 1460d88c-e8c4-492c-8876-4b97c00a469a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 03:41:27,508::task::495::TaskManager.Task::(_debug) Task 1460d88c-e8c4-492c-8876-4b97c00a469a: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 03:41:27,509::task::495::TaskManager.Task::(_debug) Task 1460d88c-e8c4-492c-8876-4b97c00a469a: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 03:41:27,510::task::495::TaskManager.Task::(_debug) Task 1460d88c-e8c4-492c-8876-4b97c00a469a: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 03:41:27,510::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 03:41:27,511::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 03:41:27,512::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 03:41:27,512::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 03:41:27,513::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 03:41:27,514::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 03:41:27,514::task::495::TaskManager.Task::(_debug) Task 1460d88c-e8c4-492c-8876-4b97c00a469a: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 03:41:27,515::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 03:41:27,851::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0808023 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:41:29,873::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 03:41:30,152::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 03:41:30,166::task::495::TaskManager.Task::(_debug) Task 09f9058b-74a4-40c6-a27a-c194e3ee625a: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 03:41:30,184::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d1b1cb66-5b77-48a5-a031-6663f95b3d9a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 03:41:30,191::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 03:41:30,194::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 03:41:30,195::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d1b1cb66-5b77-48a5-a031-6663f95b3d9a`::Granted request
>Thread-15914453::DEBUG::2013-11-20 03:41:30,197::task::495::TaskManager.Task::(_debug) Task 09f9058b-74a4-40c6-a27a-c194e3ee625a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 03:41:30,198::task::495::TaskManager.Task::(_debug) Task 09f9058b-74a4-40c6-a27a-c194e3ee625a: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 03:41:30,199::task::495::TaskManager.Task::(_debug) Task 09f9058b-74a4-40c6-a27a-c194e3ee625a: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 03:41:30,200::task::495::TaskManager.Task::(_debug) Task 09f9058b-74a4-40c6-a27a-c194e3ee625a: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 03:41:30,201::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 03:41:30,202::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 03:41:30,203::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 03:41:30,204::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 03:41:30,205::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 03:41:30,206::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 03:41:30,207::task::495::TaskManager.Task::(_debug) Task 09f9058b-74a4-40c6-a27a-c194e3ee625a: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 03:41:30,208::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 03:41:30,402::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0522276 s, 19.6 MB/s\n'; <rc> = 0
>Thread-15914811::DEBUG::2013-11-20 03:41:30,520::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914811::INFO::2013-11-20 03:41:30,521::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914811::DEBUG::2013-11-20 03:41:30,521::task::495::TaskManager.Task::(_debug) Task d0201e81-9ef3-4670-8943-3fa8ed7c950b: moving from state init -> state preparing
>Thread-15914811::DEBUG::2013-11-20 03:41:30,522::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914811::DEBUG::2013-11-20 03:41:30,523::task::495::TaskManager.Task::(_debug) Task d0201e81-9ef3-4670-8943-3fa8ed7c950b: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914811::DEBUG::2013-11-20 03:41:30,523::task::495::TaskManager.Task::(_debug) Task d0201e81-9ef3-4670-8943-3fa8ed7c950b: moving from state preparing -> state finished
>Thread-15914811::DEBUG::2013-11-20 03:41:30,524::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914811::DEBUG::2013-11-20 03:41:30,525::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914811::DEBUG::2013-11-20 03:41:30,526::task::495::TaskManager.Task::(_debug) Task d0201e81-9ef3-4670-8943-3fa8ed7c950b: ref 0 aborting False
>Thread-15914811::INFO::2013-11-20 03:41:30,526::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914812::DEBUG::2013-11-20 03:41:30,531::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914812::INFO::2013-11-20 03:41:30,532::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914812::DEBUG::2013-11-20 03:41:30,532::task::495::TaskManager.Task::(_debug) Task 1500f378-defb-4463-bf50-670ceeb24edb: moving from state init -> state preparing
>Thread-15914812::DEBUG::2013-11-20 03:41:30,533::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e274e891-e954-4a9c-b0b2-3068681028ff`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914812::DEBUG::2013-11-20 03:41:30,534::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914812::DEBUG::2013-11-20 03:41:30,534::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914812::DEBUG::2013-11-20 03:41:30,535::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e274e891-e954-4a9c-b0b2-3068681028ff`::Granted request
>Thread-15914812::DEBUG::2013-11-20 03:41:30,542::task::495::TaskManager.Task::(_debug) Task 1500f378-defb-4463-bf50-670ceeb24edb: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914812::DEBUG::2013-11-20 03:41:30,542::task::495::TaskManager.Task::(_debug) Task 1500f378-defb-4463-bf50-670ceeb24edb: ref 1 aborting False
>Thread-15914812::WARNING::2013-11-20 03:41:30,544::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914812::DEBUG::2013-11-20 03:41:30,545::task::495::TaskManager.Task::(_debug) Task 1500f378-defb-4463-bf50-670ceeb24edb: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914812::DEBUG::2013-11-20 03:41:30,545::task::495::TaskManager.Task::(_debug) Task 1500f378-defb-4463-bf50-670ceeb24edb: moving from state preparing -> state finished
>Thread-15914812::DEBUG::2013-11-20 03:41:30,546::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914812::DEBUG::2013-11-20 03:41:30,546::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914812::DEBUG::2013-11-20 03:41:30,547::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914812::DEBUG::2013-11-20 03:41:30,548::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914812::DEBUG::2013-11-20 03:41:30,548::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914812::DEBUG::2013-11-20 03:41:30,549::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914812::DEBUG::2013-11-20 03:41:30,549::task::495::TaskManager.Task::(_debug) Task 1500f378-defb-4463-bf50-670ceeb24edb: ref 0 aborting False
>Thread-15914812::INFO::2013-11-20 03:41:30,550::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:41:32,414::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:41:32,972::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0774547 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:41:34,988::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 03:41:35,187::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 03:41:35,194::task::495::TaskManager.Task::(_debug) Task 727b3428-108e-4f51-8f0d-9c59bd8e3cd0: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 03:41:35,209::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bba742dd-108b-427e-88c6-4aea3d55c188`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 03:41:35,211::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 03:41:35,213::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 03:41:35,214::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bba742dd-108b-427e-88c6-4aea3d55c188`::Granted request
>Thread-15914519::DEBUG::2013-11-20 03:41:35,216::task::495::TaskManager.Task::(_debug) Task 727b3428-108e-4f51-8f0d-9c59bd8e3cd0: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 03:41:35,216::task::495::TaskManager.Task::(_debug) Task 727b3428-108e-4f51-8f0d-9c59bd8e3cd0: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 03:41:35,218::task::495::TaskManager.Task::(_debug) Task 727b3428-108e-4f51-8f0d-9c59bd8e3cd0: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 03:41:35,219::task::495::TaskManager.Task::(_debug) Task 727b3428-108e-4f51-8f0d-9c59bd8e3cd0: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 03:41:35,220::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 03:41:35,220::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 03:41:35,222::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 03:41:35,223::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 03:41:35,223::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 03:41:35,224::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 03:41:35,225::task::495::TaskManager.Task::(_debug) Task 727b3428-108e-4f51-8f0d-9c59bd8e3cd0: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 03:41:35,229::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 03:41:35,551::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.088044 s, 11.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:41:37,567::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:41:38,108::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0743966 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 03:41:38,725::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:41:38,728::task::495::TaskManager.Task::(_debug) Task 2b14aa22-184b-4ffb-a80f-3ac78534859e: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:41:38,731::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`49d2e416-9eba-4b7a-a56d-1d60f9088bd2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:41:38,734::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:41:38,736::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:41:38,737::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`49d2e416-9eba-4b7a-a56d-1d60f9088bd2`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:41:38,740::task::495::TaskManager.Task::(_debug) Task 2b14aa22-184b-4ffb-a80f-3ac78534859e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:41:38,742::task::495::TaskManager.Task::(_debug) Task 2b14aa22-184b-4ffb-a80f-3ac78534859e: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:41:38,745::task::495::TaskManager.Task::(_debug) Task 2b14aa22-184b-4ffb-a80f-3ac78534859e: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:41:38,749::task::495::TaskManager.Task::(_debug) Task 2b14aa22-184b-4ffb-a80f-3ac78534859e: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:41:38,752::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:41:38,753::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:41:38,754::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:41:38,755::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:41:38,755::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:41:38,756::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:41:38,757::task::495::TaskManager.Task::(_debug) Task 2b14aa22-184b-4ffb-a80f-3ac78534859e: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:41:38,757::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:41:38,758::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:41:38,758::task::495::TaskManager.Task::(_debug) Task c2bf1d3d-c17e-46c3-b7fc-f63ea0df163e: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:41:38,760::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d36b12c8-6394-4896-aaef-b0de68636332`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:41:38,760::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:41:38,761::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:41:38,761::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d36b12c8-6394-4896-aaef-b0de68636332`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:41:38,762::task::495::TaskManager.Task::(_debug) Task c2bf1d3d-c17e-46c3-b7fc-f63ea0df163e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:41:38,763::task::495::TaskManager.Task::(_debug) Task c2bf1d3d-c17e-46c3-b7fc-f63ea0df163e: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:41:38,763::task::495::TaskManager.Task::(_debug) Task c2bf1d3d-c17e-46c3-b7fc-f63ea0df163e: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:41:38,764::task::495::TaskManager.Task::(_debug) Task c2bf1d3d-c17e-46c3-b7fc-f63ea0df163e: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:41:38,764::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:41:38,765::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:41:38,766::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:41:38,766::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:41:38,767::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:41:38,767::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:41:38,768::task::495::TaskManager.Task::(_debug) Task c2bf1d3d-c17e-46c3-b7fc-f63ea0df163e: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:41:38,769::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:41:40,124::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 03:41:40,345::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:41:40,353::task::495::TaskManager.Task::(_debug) Task 0cfa50f5-b4aa-4739-b3c9-979b5accba8c: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:41:40,358::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`9d2c7d5a-34d8-4bf0-8181-cf018f034c51`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:41:40,360::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:41:40,362::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:41:40,363::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`9d2c7d5a-34d8-4bf0-8181-cf018f034c51`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:41:40,366::task::495::TaskManager.Task::(_debug) Task 0cfa50f5-b4aa-4739-b3c9-979b5accba8c: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:41:40,367::task::495::TaskManager.Task::(_debug) Task 0cfa50f5-b4aa-4739-b3c9-979b5accba8c: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:41:40,368::task::495::TaskManager.Task::(_debug) Task 0cfa50f5-b4aa-4739-b3c9-979b5accba8c: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:41:40,369::task::495::TaskManager.Task::(_debug) Task 0cfa50f5-b4aa-4739-b3c9-979b5accba8c: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:41:40,370::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:41:40,371::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:41:40,373::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:41:40,374::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:41:40,375::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:41:40,376::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:41:40,377::task::495::TaskManager.Task::(_debug) Task 0cfa50f5-b4aa-4739-b3c9-979b5accba8c: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:41:40,378::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:41:40,379::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:41:40,380::task::495::TaskManager.Task::(_debug) Task a6f757d9-adfe-499d-8af2-ea70eecbf774: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:41:40,382::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c5ae584d-04e8-4f33-a0e2-4f2895793e57`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:41:40,383::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:41:40,384::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:41:40,384::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c5ae584d-04e8-4f33-a0e2-4f2895793e57`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:41:40,385::task::495::TaskManager.Task::(_debug) Task a6f757d9-adfe-499d-8af2-ea70eecbf774: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:41:40,386::task::495::TaskManager.Task::(_debug) Task a6f757d9-adfe-499d-8af2-ea70eecbf774: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:41:40,387::task::495::TaskManager.Task::(_debug) Task a6f757d9-adfe-499d-8af2-ea70eecbf774: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:41:40,387::task::495::TaskManager.Task::(_debug) Task a6f757d9-adfe-499d-8af2-ea70eecbf774: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:41:40,388::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:41:40,389::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:41:40,389::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:41:40,390::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:41:40,391::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:41:40,391::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:41:40,392::task::495::TaskManager.Task::(_debug) Task a6f757d9-adfe-499d-8af2-ea70eecbf774: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:41:40,393::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:41:40,393::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:41:40,394::task::495::TaskManager.Task::(_debug) Task 12c389bf-8498-4b86-95ce-2291d222d80b: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:41:40,395::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b26b78e2-2f7d-4ea0-bad8-9a6473fc2844`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:41:40,396::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:41:40,397::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:41:40,397::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b26b78e2-2f7d-4ea0-bad8-9a6473fc2844`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:41:40,398::task::495::TaskManager.Task::(_debug) Task 12c389bf-8498-4b86-95ce-2291d222d80b: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:41:40,399::task::495::TaskManager.Task::(_debug) Task 12c389bf-8498-4b86-95ce-2291d222d80b: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:41:40,399::task::495::TaskManager.Task::(_debug) Task 12c389bf-8498-4b86-95ce-2291d222d80b: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:41:40,400::task::495::TaskManager.Task::(_debug) Task 12c389bf-8498-4b86-95ce-2291d222d80b: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:41:40,401::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:41:40,401::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:41:40,402::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:41:40,403::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:41:40,403::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:41:40,404::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:41:40,404::task::495::TaskManager.Task::(_debug) Task 12c389bf-8498-4b86-95ce-2291d222d80b: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:41:40,405::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:41:40,406::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:41:40,406::task::495::TaskManager.Task::(_debug) Task 47be7f53-10ee-4381-b7f5-bf6cf6fa47ee: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:41:40,407::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`4010ec08-c728-46f7-925a-655f081d9546`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:41:40,408::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:41:40,409::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:41:40,409::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`4010ec08-c728-46f7-925a-655f081d9546`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:41:40,410::task::495::TaskManager.Task::(_debug) Task 47be7f53-10ee-4381-b7f5-bf6cf6fa47ee: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:41:40,411::task::495::TaskManager.Task::(_debug) Task 47be7f53-10ee-4381-b7f5-bf6cf6fa47ee: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:41:40,412::task::495::TaskManager.Task::(_debug) Task 47be7f53-10ee-4381-b7f5-bf6cf6fa47ee: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:41:40,412::task::495::TaskManager.Task::(_debug) Task 47be7f53-10ee-4381-b7f5-bf6cf6fa47ee: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:41:40,413::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:41:40,413::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:41:40,414::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:41:40,415::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:41:40,415::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:41:40,416::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:41:40,416::task::495::TaskManager.Task::(_debug) Task 47be7f53-10ee-4381-b7f5-bf6cf6fa47ee: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:41:40,417::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:41:40,695::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0812489 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15914816::INFO::2013-11-20 03:41:40,801::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914816::DEBUG::2013-11-20 03:41:40,802::task::495::TaskManager.Task::(_debug) Task 664d7c19-3efb-485d-8f54-f7d44eef9803: moving from state init -> state preparing
>Thread-15914816::DEBUG::2013-11-20 03:41:40,803::task::495::TaskManager.Task::(_debug) Task 664d7c19-3efb-485d-8f54-f7d44eef9803: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000624895095825', 'lastCheck': 1384918899.186826, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014009475708', 'lastCheck': 1384918894.145828, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00224685668945', 'lastCheck': 1384918900.355577, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000494003295898', 'lastCheck': 1384918893.6320159, 'valid': True, 'code': 0}}
>Thread-15914816::DEBUG::2013-11-20 03:41:40,804::task::495::TaskManager.Task::(_debug) Task 664d7c19-3efb-485d-8f54-f7d44eef9803: moving from state preparing -> state finished
>Thread-15914816::DEBUG::2013-11-20 03:41:40,805::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914816::DEBUG::2013-11-20 03:41:40,805::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914816::DEBUG::2013-11-20 03:41:40,806::task::495::TaskManager.Task::(_debug) Task 664d7c19-3efb-485d-8f54-f7d44eef9803: ref 0 aborting False
>Thread-15914816::INFO::2013-11-20 03:41:40,807::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000624895095825', 'lastCheck': 1384918899.186826, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014009475708', 'lastCheck': 1384918894.145828, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00224685668945', 'lastCheck': 1384918900.355577, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000494003295898', 'lastCheck': 1384918893.6320159, 'valid': True, 'code': 0}}
>Thread-15914817::DEBUG::2013-11-20 03:41:40,811::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914817::INFO::2013-11-20 03:41:40,812::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914817::DEBUG::2013-11-20 03:41:40,828::task::495::TaskManager.Task::(_debug) Task 41c098cb-25d4-458a-a166-2abab3636771: moving from state init -> state preparing
>Thread-15914817::DEBUG::2013-11-20 03:41:40,829::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914817::DEBUG::2013-11-20 03:41:40,830::task::495::TaskManager.Task::(_debug) Task 41c098cb-25d4-458a-a166-2abab3636771: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914817::DEBUG::2013-11-20 03:41:40,830::task::495::TaskManager.Task::(_debug) Task 41c098cb-25d4-458a-a166-2abab3636771: moving from state preparing -> state finished
>Thread-15914817::DEBUG::2013-11-20 03:41:40,831::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914817::DEBUG::2013-11-20 03:41:40,832::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914817::DEBUG::2013-11-20 03:41:40,832::task::495::TaskManager.Task::(_debug) Task 41c098cb-25d4-458a-a166-2abab3636771: ref 0 aborting False
>Thread-15914817::INFO::2013-11-20 03:41:40,833::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914818::DEBUG::2013-11-20 03:41:40,839::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914818::INFO::2013-11-20 03:41:40,840::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914818::DEBUG::2013-11-20 03:41:40,841::task::495::TaskManager.Task::(_debug) Task fc00e94b-a7bf-45c6-bd86-fd004d698a86: moving from state init -> state preparing
>Thread-15914818::DEBUG::2013-11-20 03:41:40,842::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d78ac86a-4682-41bb-9e9b-860b8ad59217`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914818::DEBUG::2013-11-20 03:41:40,843::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914818::DEBUG::2013-11-20 03:41:40,844::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914818::DEBUG::2013-11-20 03:41:40,845::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d78ac86a-4682-41bb-9e9b-860b8ad59217`::Granted request
>Thread-15914818::DEBUG::2013-11-20 03:41:40,846::task::495::TaskManager.Task::(_debug) Task fc00e94b-a7bf-45c6-bd86-fd004d698a86: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914818::DEBUG::2013-11-20 03:41:40,846::task::495::TaskManager.Task::(_debug) Task fc00e94b-a7bf-45c6-bd86-fd004d698a86: ref 1 aborting False
>Thread-15914818::WARNING::2013-11-20 03:41:40,849::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914818::DEBUG::2013-11-20 03:41:40,850::task::495::TaskManager.Task::(_debug) Task fc00e94b-a7bf-45c6-bd86-fd004d698a86: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914818::DEBUG::2013-11-20 03:41:40,854::task::495::TaskManager.Task::(_debug) Task fc00e94b-a7bf-45c6-bd86-fd004d698a86: moving from state preparing -> state finished
>Thread-15914818::DEBUG::2013-11-20 03:41:40,854::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914818::DEBUG::2013-11-20 03:41:40,855::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914818::DEBUG::2013-11-20 03:41:40,856::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914818::DEBUG::2013-11-20 03:41:40,857::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914818::DEBUG::2013-11-20 03:41:40,857::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914818::DEBUG::2013-11-20 03:41:40,858::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914818::DEBUG::2013-11-20 03:41:40,859::task::495::TaskManager.Task::(_debug) Task fc00e94b-a7bf-45c6-bd86-fd004d698a86: ref 0 aborting False
>Thread-15914818::INFO::2013-11-20 03:41:40,865::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:41:42,709::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:41:43,243::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0748993 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:41:45,259::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:41:45,827::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0830543 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:41:47,843::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:41:48,410::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0844311 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:41:50,431::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:41:51,004::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0798218 s, 12.8 MB/s\n'; <rc> = 0
>Thread-15914823::DEBUG::2013-11-20 03:41:51,293::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914823::INFO::2013-11-20 03:41:51,295::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914823::DEBUG::2013-11-20 03:41:51,295::task::495::TaskManager.Task::(_debug) Task 58b402fb-346a-4464-876c-d2a49a9a80ee: moving from state init -> state preparing
>Thread-15914823::DEBUG::2013-11-20 03:41:51,296::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914823::DEBUG::2013-11-20 03:41:51,297::task::495::TaskManager.Task::(_debug) Task 58b402fb-346a-4464-876c-d2a49a9a80ee: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914823::DEBUG::2013-11-20 03:41:51,302::task::495::TaskManager.Task::(_debug) Task 58b402fb-346a-4464-876c-d2a49a9a80ee: moving from state preparing -> state finished
>Thread-15914823::DEBUG::2013-11-20 03:41:51,302::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914823::DEBUG::2013-11-20 03:41:51,305::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914823::DEBUG::2013-11-20 03:41:51,306::task::495::TaskManager.Task::(_debug) Task 58b402fb-346a-4464-876c-d2a49a9a80ee: ref 0 aborting False
>Thread-15914823::INFO::2013-11-20 03:41:51,307::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914824::DEBUG::2013-11-20 03:41:51,311::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914824::INFO::2013-11-20 03:41:51,312::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914824::DEBUG::2013-11-20 03:41:51,313::task::495::TaskManager.Task::(_debug) Task b3dac94e-5ba0-402e-abf2-5a9c98d7502a: moving from state init -> state preparing
>Thread-15914824::DEBUG::2013-11-20 03:41:51,314::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`18df5142-e537-4579-871a-3f983cdb28d5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914824::DEBUG::2013-11-20 03:41:51,315::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914824::DEBUG::2013-11-20 03:41:51,316::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914824::DEBUG::2013-11-20 03:41:51,316::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`18df5142-e537-4579-871a-3f983cdb28d5`::Granted request
>Thread-15914824::DEBUG::2013-11-20 03:41:51,317::task::495::TaskManager.Task::(_debug) Task b3dac94e-5ba0-402e-abf2-5a9c98d7502a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914824::DEBUG::2013-11-20 03:41:51,318::task::495::TaskManager.Task::(_debug) Task b3dac94e-5ba0-402e-abf2-5a9c98d7502a: ref 1 aborting False
>Thread-15914824::WARNING::2013-11-20 03:41:51,321::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914824::DEBUG::2013-11-20 03:41:51,322::task::495::TaskManager.Task::(_debug) Task b3dac94e-5ba0-402e-abf2-5a9c98d7502a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914824::DEBUG::2013-11-20 03:41:51,323::task::495::TaskManager.Task::(_debug) Task b3dac94e-5ba0-402e-abf2-5a9c98d7502a: moving from state preparing -> state finished
>Thread-15914824::DEBUG::2013-11-20 03:41:51,324::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914824::DEBUG::2013-11-20 03:41:51,324::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914824::DEBUG::2013-11-20 03:41:51,325::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914824::DEBUG::2013-11-20 03:41:51,326::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914824::DEBUG::2013-11-20 03:41:51,326::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914824::DEBUG::2013-11-20 03:41:51,327::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914824::DEBUG::2013-11-20 03:41:51,328::task::495::TaskManager.Task::(_debug) Task b3dac94e-5ba0-402e-abf2-5a9c98d7502a: ref 0 aborting False
>Thread-15914824::INFO::2013-11-20 03:41:51,328::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:41:53,024::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:41:53,580::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0891952 s, 11.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:41:55,593::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 03:41:55,816::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:41:55,820::task::495::TaskManager.Task::(_debug) Task f9acdf32-b015-40e7-8eed-cc06ff3c1335: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:41:55,823::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`48d864aa-a041-420d-99b4-3079456357eb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:41:55,824::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:41:55,832::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:41:55,833::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`48d864aa-a041-420d-99b4-3079456357eb`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:41:55,835::task::495::TaskManager.Task::(_debug) Task f9acdf32-b015-40e7-8eed-cc06ff3c1335: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:41:55,836::task::495::TaskManager.Task::(_debug) Task f9acdf32-b015-40e7-8eed-cc06ff3c1335: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:41:55,837::task::495::TaskManager.Task::(_debug) Task f9acdf32-b015-40e7-8eed-cc06ff3c1335: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:41:55,838::task::495::TaskManager.Task::(_debug) Task f9acdf32-b015-40e7-8eed-cc06ff3c1335: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:41:55,839::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:41:55,839::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:41:55,840::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:41:55,841::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:41:55,842::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:41:55,842::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:41:55,843::task::495::TaskManager.Task::(_debug) Task f9acdf32-b015-40e7-8eed-cc06ff3c1335: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:41:55,844::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:41:56,326::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0897639 s, 11.4 MB/s\n'; <rc> = 0
>Thread-15914826::INFO::2013-11-20 03:41:56,796::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914826::DEBUG::2013-11-20 03:41:56,799::task::495::TaskManager.Task::(_debug) Task f71a7e43-5f43-45cd-b8b7-7ee221d939ed: moving from state init -> state preparing
>Thread-15914826::DEBUG::2013-11-20 03:41:56,800::task::495::TaskManager.Task::(_debug) Task f71a7e43-5f43-45cd-b8b7-7ee221d939ed: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000750064849854', 'lastCheck': 1384918909.494782, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.001296043396', 'lastCheck': 1384918914.652004, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0138900279999', 'lastCheck': 1384918910.672935, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000420093536377', 'lastCheck': 1384918914.2256589, 'valid': True, 'code': 0}}
>Thread-15914826::DEBUG::2013-11-20 03:41:56,802::task::495::TaskManager.Task::(_debug) Task f71a7e43-5f43-45cd-b8b7-7ee221d939ed: moving from state preparing -> state finished
>Thread-15914826::DEBUG::2013-11-20 03:41:56,808::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914826::DEBUG::2013-11-20 03:41:56,808::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914826::DEBUG::2013-11-20 03:41:56,809::task::495::TaskManager.Task::(_debug) Task f71a7e43-5f43-45cd-b8b7-7ee221d939ed: ref 0 aborting False
>Thread-15914826::INFO::2013-11-20 03:41:56,810::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000750064849854', 'lastCheck': 1384918909.494782, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.001296043396', 'lastCheck': 1384918914.652004, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0138900279999', 'lastCheck': 1384918910.672935, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000420093536377', 'lastCheck': 1384918914.2256589, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:41:58,339::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:41:58,891::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0874581 s, 11.7 MB/s\n'; <rc> = 0
>Thread-42838::DEBUG::2013-11-20 03:41:59,813::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 03:42:00,579::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:42:00,910::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:42:01,484::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0890414 s, 11.5 MB/s\n'; <rc> = 0
>Thread-15914829::DEBUG::2013-11-20 03:42:01,634::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914829::INFO::2013-11-20 03:42:01,645::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914829::DEBUG::2013-11-20 03:42:01,646::task::495::TaskManager.Task::(_debug) Task 28b853cc-eb33-4ecf-9fa9-8e64bbc4bcf7: moving from state init -> state preparing
>Thread-15914829::DEBUG::2013-11-20 03:42:01,647::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914829::DEBUG::2013-11-20 03:42:01,647::task::495::TaskManager.Task::(_debug) Task 28b853cc-eb33-4ecf-9fa9-8e64bbc4bcf7: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914829::DEBUG::2013-11-20 03:42:01,647::task::495::TaskManager.Task::(_debug) Task 28b853cc-eb33-4ecf-9fa9-8e64bbc4bcf7: moving from state preparing -> state finished
>Thread-15914829::DEBUG::2013-11-20 03:42:01,648::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914829::DEBUG::2013-11-20 03:42:01,648::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914829::DEBUG::2013-11-20 03:42:01,648::task::495::TaskManager.Task::(_debug) Task 28b853cc-eb33-4ecf-9fa9-8e64bbc4bcf7: ref 0 aborting False
>Thread-15914829::INFO::2013-11-20 03:42:01,649::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914830::DEBUG::2013-11-20 03:42:01,653::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914830::INFO::2013-11-20 03:42:01,654::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914830::DEBUG::2013-11-20 03:42:01,654::task::495::TaskManager.Task::(_debug) Task 46917569-b5b3-4c30-ac9c-28a40b7d4cd6: moving from state init -> state preparing
>Thread-15914830::DEBUG::2013-11-20 03:42:01,655::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`03e98ca0-cf5a-4ffa-92b1-46d5bf3fffa8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914830::DEBUG::2013-11-20 03:42:01,655::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914830::DEBUG::2013-11-20 03:42:01,656::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914830::DEBUG::2013-11-20 03:42:01,657::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`03e98ca0-cf5a-4ffa-92b1-46d5bf3fffa8`::Granted request
>Thread-15914830::DEBUG::2013-11-20 03:42:01,657::task::495::TaskManager.Task::(_debug) Task 46917569-b5b3-4c30-ac9c-28a40b7d4cd6: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914830::DEBUG::2013-11-20 03:42:01,658::task::495::TaskManager.Task::(_debug) Task 46917569-b5b3-4c30-ac9c-28a40b7d4cd6: ref 1 aborting False
>Thread-15914830::WARNING::2013-11-20 03:42:01,659::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914830::DEBUG::2013-11-20 03:42:01,660::task::495::TaskManager.Task::(_debug) Task 46917569-b5b3-4c30-ac9c-28a40b7d4cd6: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914830::DEBUG::2013-11-20 03:42:01,661::task::495::TaskManager.Task::(_debug) Task 46917569-b5b3-4c30-ac9c-28a40b7d4cd6: moving from state preparing -> state finished
>Thread-15914830::DEBUG::2013-11-20 03:42:01,662::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914830::DEBUG::2013-11-20 03:42:01,662::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914830::DEBUG::2013-11-20 03:42:01,663::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914830::DEBUG::2013-11-20 03:42:01,664::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914830::DEBUG::2013-11-20 03:42:01,665::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914830::DEBUG::2013-11-20 03:42:01,665::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914830::DEBUG::2013-11-20 03:42:01,666::task::495::TaskManager.Task::(_debug) Task 46917569-b5b3-4c30-ac9c-28a40b7d4cd6: ref 0 aborting False
>Thread-15914830::INFO::2013-11-20 03:42:01,667::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:42:03,500::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:42:04,073::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0794474 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:42:06,087::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:42:06,609::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0664136 s, 15.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:42:08,625::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:42:09,202::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0774818 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 03:42:10,889::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:42:10,890::task::495::TaskManager.Task::(_debug) Task 727f75b4-5168-4f30-8a52-ea84f931769a: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:42:10,893::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2295863b-d96f-4c94-ba5c-366c445c769e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:42:10,894::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:42:10,894::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:42:10,895::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2295863b-d96f-4c94-ba5c-366c445c769e`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:42:10,896::task::495::TaskManager.Task::(_debug) Task 727f75b4-5168-4f30-8a52-ea84f931769a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:42:10,897::task::495::TaskManager.Task::(_debug) Task 727f75b4-5168-4f30-8a52-ea84f931769a: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:42:10,897::task::495::TaskManager.Task::(_debug) Task 727f75b4-5168-4f30-8a52-ea84f931769a: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:42:10,898::task::495::TaskManager.Task::(_debug) Task 727f75b4-5168-4f30-8a52-ea84f931769a: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:42:10,899::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:42:10,899::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:42:10,900::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:42:10,901::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:42:10,901::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:42:10,902::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:42:10,903::task::495::TaskManager.Task::(_debug) Task 727f75b4-5168-4f30-8a52-ea84f931769a: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:42:10,903::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:42:11,215::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:42:11,749::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0829868 s, 12.3 MB/s\n'; <rc> = 0
>Thread-15914834::DEBUG::2013-11-20 03:42:11,970::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914834::INFO::2013-11-20 03:42:11,971::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914834::DEBUG::2013-11-20 03:42:11,975::task::495::TaskManager.Task::(_debug) Task a17c6b4b-0001-4b14-a4fc-ebc0175f4ed9: moving from state init -> state preparing
>Thread-15914834::DEBUG::2013-11-20 03:42:11,976::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914834::DEBUG::2013-11-20 03:42:11,976::task::495::TaskManager.Task::(_debug) Task a17c6b4b-0001-4b14-a4fc-ebc0175f4ed9: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914834::DEBUG::2013-11-20 03:42:11,977::task::495::TaskManager.Task::(_debug) Task a17c6b4b-0001-4b14-a4fc-ebc0175f4ed9: moving from state preparing -> state finished
>Thread-15914834::DEBUG::2013-11-20 03:42:11,978::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914834::DEBUG::2013-11-20 03:42:11,978::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914834::DEBUG::2013-11-20 03:42:11,979::task::495::TaskManager.Task::(_debug) Task a17c6b4b-0001-4b14-a4fc-ebc0175f4ed9: ref 0 aborting False
>Thread-15914834::INFO::2013-11-20 03:42:11,979::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914835::DEBUG::2013-11-20 03:42:11,985::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914835::INFO::2013-11-20 03:42:11,986::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914835::DEBUG::2013-11-20 03:42:11,987::task::495::TaskManager.Task::(_debug) Task 7060f5a4-ddec-4535-a619-576c24a21d57: moving from state init -> state preparing
>Thread-15914835::DEBUG::2013-11-20 03:42:11,989::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a4e1081d-8278-43f4-b7d6-018c6ad413ee`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914835::DEBUG::2013-11-20 03:42:11,990::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914835::DEBUG::2013-11-20 03:42:11,991::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914835::DEBUG::2013-11-20 03:42:11,992::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a4e1081d-8278-43f4-b7d6-018c6ad413ee`::Granted request
>Thread-15914835::DEBUG::2013-11-20 03:42:11,993::task::495::TaskManager.Task::(_debug) Task 7060f5a4-ddec-4535-a619-576c24a21d57: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914835::DEBUG::2013-11-20 03:42:11,993::task::495::TaskManager.Task::(_debug) Task 7060f5a4-ddec-4535-a619-576c24a21d57: ref 1 aborting False
>Thread-15914835::WARNING::2013-11-20 03:42:11,996::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914835::DEBUG::2013-11-20 03:42:12,000::task::495::TaskManager.Task::(_debug) Task 7060f5a4-ddec-4535-a619-576c24a21d57: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914835::DEBUG::2013-11-20 03:42:12,001::task::495::TaskManager.Task::(_debug) Task 7060f5a4-ddec-4535-a619-576c24a21d57: moving from state preparing -> state finished
>Thread-15914835::DEBUG::2013-11-20 03:42:12,002::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914835::DEBUG::2013-11-20 03:42:12,002::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914835::DEBUG::2013-11-20 03:42:12,003::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914835::DEBUG::2013-11-20 03:42:12,004::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914835::DEBUG::2013-11-20 03:42:12,004::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914835::DEBUG::2013-11-20 03:42:12,005::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914835::DEBUG::2013-11-20 03:42:12,006::task::495::TaskManager.Task::(_debug) Task 7060f5a4-ddec-4535-a619-576c24a21d57: ref 0 aborting False
>Thread-15914835::INFO::2013-11-20 03:42:12,007::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914836::INFO::2013-11-20 03:42:12,537::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914836::DEBUG::2013-11-20 03:42:12,538::task::495::TaskManager.Task::(_debug) Task 74560b01-7025-4d27-a5ff-51e5d61b1fc3: moving from state init -> state preparing
>Thread-15914836::DEBUG::2013-11-20 03:42:12,539::task::495::TaskManager.Task::(_debug) Task 74560b01-7025-4d27-a5ff-51e5d61b1fc3: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000441789627075', 'lastCheck': 1384918930.8845, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00118803977966', 'lastCheck': 1384918924.8225491, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00145578384399', 'lastCheck': 1384918930.872896, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000493049621582', 'lastCheck': 1384918924.236609, 'valid': True, 'code': 0}}
>Thread-15914836::DEBUG::2013-11-20 03:42:12,547::task::495::TaskManager.Task::(_debug) Task 74560b01-7025-4d27-a5ff-51e5d61b1fc3: moving from state preparing -> state finished
>Thread-15914836::DEBUG::2013-11-20 03:42:12,547::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914836::DEBUG::2013-11-20 03:42:12,548::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914836::DEBUG::2013-11-20 03:42:12,549::task::495::TaskManager.Task::(_debug) Task 74560b01-7025-4d27-a5ff-51e5d61b1fc3: ref 0 aborting False
>Thread-15914836::INFO::2013-11-20 03:42:12,549::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000441789627075', 'lastCheck': 1384918930.8845, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00118803977966', 'lastCheck': 1384918924.8225491, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00145578384399', 'lastCheck': 1384918930.872896, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000493049621582', 'lastCheck': 1384918924.236609, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:42:13,766::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:42:14,376::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0731179 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:42:16,393::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 03:42:16,619::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:42:16,622::task::495::TaskManager.Task::(_debug) Task 42853e36-3797-4528-a741-7c0ff0546abb: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:42:16,625::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`27a04bec-73bc-4429-8689-3ce6216e0ae8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:42:16,626::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:42:16,627::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:42:16,628::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`27a04bec-73bc-4429-8689-3ce6216e0ae8`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:42:16,630::task::495::TaskManager.Task::(_debug) Task 42853e36-3797-4528-a741-7c0ff0546abb: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:42:16,630::task::495::TaskManager.Task::(_debug) Task 42853e36-3797-4528-a741-7c0ff0546abb: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:42:16,631::task::495::TaskManager.Task::(_debug) Task 42853e36-3797-4528-a741-7c0ff0546abb: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:42:16,632::task::495::TaskManager.Task::(_debug) Task 42853e36-3797-4528-a741-7c0ff0546abb: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:42:16,633::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:42:16,633::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:42:16,634::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:42:16,635::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:42:16,635::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:42:16,636::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:42:16,637::task::495::TaskManager.Task::(_debug) Task 42853e36-3797-4528-a741-7c0ff0546abb: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:42:16,638::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:42:16,974::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0799976 s, 12.8 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 03:42:18,654::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:42:18,655::task::495::TaskManager.Task::(_debug) Task 86041a8e-2a07-4436-a371-4ac3a9787135: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:42:18,657::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`70cad9e4-27cc-479f-8ec8-97f5766c15b5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:42:18,658::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:42:18,659::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:42:18,660::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`70cad9e4-27cc-479f-8ec8-97f5766c15b5`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:42:18,662::task::495::TaskManager.Task::(_debug) Task 86041a8e-2a07-4436-a371-4ac3a9787135: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:42:18,663::task::495::TaskManager.Task::(_debug) Task 86041a8e-2a07-4436-a371-4ac3a9787135: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:42:18,663::task::495::TaskManager.Task::(_debug) Task 86041a8e-2a07-4436-a371-4ac3a9787135: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:42:18,664::task::495::TaskManager.Task::(_debug) Task 86041a8e-2a07-4436-a371-4ac3a9787135: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:42:18,665::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:42:18,665::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:42:18,666::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:42:18,667::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:42:18,667::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:42:18,668::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:42:18,669::task::495::TaskManager.Task::(_debug) Task 86041a8e-2a07-4436-a371-4ac3a9787135: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:42:18,669::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:42:18,989::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:42:19,575::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0984243 s, 10.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:42:21,594::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:42:22,175::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0931936 s, 11.0 MB/s\n'; <rc> = 0
>Thread-15914841::DEBUG::2013-11-20 03:42:22,227::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914841::INFO::2013-11-20 03:42:22,229::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914841::DEBUG::2013-11-20 03:42:22,230::task::495::TaskManager.Task::(_debug) Task 2e5fa721-9b53-45ea-a1ee-5998ee153be6: moving from state init -> state preparing
>Thread-15914841::DEBUG::2013-11-20 03:42:22,231::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914841::DEBUG::2013-11-20 03:42:22,232::task::495::TaskManager.Task::(_debug) Task 2e5fa721-9b53-45ea-a1ee-5998ee153be6: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914841::DEBUG::2013-11-20 03:42:22,233::task::495::TaskManager.Task::(_debug) Task 2e5fa721-9b53-45ea-a1ee-5998ee153be6: moving from state preparing -> state finished
>Thread-15914841::DEBUG::2013-11-20 03:42:22,234::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914841::DEBUG::2013-11-20 03:42:22,235::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914841::DEBUG::2013-11-20 03:42:22,236::task::495::TaskManager.Task::(_debug) Task 2e5fa721-9b53-45ea-a1ee-5998ee153be6: ref 0 aborting False
>Thread-15914841::INFO::2013-11-20 03:42:22,237::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914842::DEBUG::2013-11-20 03:42:22,248::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914842::INFO::2013-11-20 03:42:22,249::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914842::DEBUG::2013-11-20 03:42:22,249::task::495::TaskManager.Task::(_debug) Task aff35b05-32ec-46f1-9ad4-458b9271438c: moving from state init -> state preparing
>Thread-15914842::DEBUG::2013-11-20 03:42:22,251::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`487cf23f-eefe-4471-98f4-1923ac7228b1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914842::DEBUG::2013-11-20 03:42:22,252::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914842::DEBUG::2013-11-20 03:42:22,253::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914842::DEBUG::2013-11-20 03:42:22,253::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`487cf23f-eefe-4471-98f4-1923ac7228b1`::Granted request
>Thread-15914842::DEBUG::2013-11-20 03:42:22,255::task::495::TaskManager.Task::(_debug) Task aff35b05-32ec-46f1-9ad4-458b9271438c: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914842::DEBUG::2013-11-20 03:42:22,255::task::495::TaskManager.Task::(_debug) Task aff35b05-32ec-46f1-9ad4-458b9271438c: ref 1 aborting False
>Thread-15914842::WARNING::2013-11-20 03:42:22,258::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914842::DEBUG::2013-11-20 03:42:22,259::task::495::TaskManager.Task::(_debug) Task aff35b05-32ec-46f1-9ad4-458b9271438c: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914842::DEBUG::2013-11-20 03:42:22,260::task::495::TaskManager.Task::(_debug) Task aff35b05-32ec-46f1-9ad4-458b9271438c: moving from state preparing -> state finished
>Thread-15914842::DEBUG::2013-11-20 03:42:22,261::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914842::DEBUG::2013-11-20 03:42:22,262::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914842::DEBUG::2013-11-20 03:42:22,263::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914842::DEBUG::2013-11-20 03:42:22,264::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914842::DEBUG::2013-11-20 03:42:22,264::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914842::DEBUG::2013-11-20 03:42:22,265::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914842::DEBUG::2013-11-20 03:42:22,266::task::495::TaskManager.Task::(_debug) Task aff35b05-32ec-46f1-9ad4-458b9271438c: ref 0 aborting False
>Thread-15914842::INFO::2013-11-20 03:42:22,266::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:42:24,190::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:42:24,743::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0791171 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:42:26,759::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:42:27,319::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0779239 s, 13.1 MB/s\n'; <rc> = 0
>Thread-15914844::INFO::2013-11-20 03:42:27,969::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914844::DEBUG::2013-11-20 03:42:27,973::task::495::TaskManager.Task::(_debug) Task 7f3ada08-78d5-4b0c-a1e2-8f18f22ffd3f: moving from state init -> state preparing
>Thread-15914844::DEBUG::2013-11-20 03:42:27,974::task::495::TaskManager.Task::(_debug) Task 7f3ada08-78d5-4b0c-a1e2-8f18f22ffd3f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000416994094849', 'lastCheck': 1384918941.231364, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.001953125', 'lastCheck': 1384918945.4239819, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00152802467346', 'lastCheck': 1384918941.2318881, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000484943389893', 'lastCheck': 1384918944.8317449, 'valid': True, 'code': 0}}
>Thread-15914844::DEBUG::2013-11-20 03:42:27,975::task::495::TaskManager.Task::(_debug) Task 7f3ada08-78d5-4b0c-a1e2-8f18f22ffd3f: moving from state preparing -> state finished
>Thread-15914844::DEBUG::2013-11-20 03:42:27,975::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914844::DEBUG::2013-11-20 03:42:27,976::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914844::DEBUG::2013-11-20 03:42:27,976::task::495::TaskManager.Task::(_debug) Task 7f3ada08-78d5-4b0c-a1e2-8f18f22ffd3f: ref 0 aborting False
>Thread-15914844::INFO::2013-11-20 03:42:27,977::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000416994094849', 'lastCheck': 1384918941.231364, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.001953125', 'lastCheck': 1384918945.4239819, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00152802467346', 'lastCheck': 1384918941.2318881, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000484943389893', 'lastCheck': 1384918944.8317449, 'valid': True, 'code': 0}}
>Thread-15914435::INFO::2013-11-20 03:42:28,704::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 03:42:28,705::task::495::TaskManager.Task::(_debug) Task 56671927-b0f0-4a9e-b8f5-b7527c26c493: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 03:42:28,707::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d24cee3d-1976-40b2-8230-7dc3e23940b4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 03:42:28,708::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 03:42:28,709::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 03:42:28,710::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d24cee3d-1976-40b2-8230-7dc3e23940b4`::Granted request
>Thread-15914435::DEBUG::2013-11-20 03:42:28,711::task::495::TaskManager.Task::(_debug) Task 56671927-b0f0-4a9e-b8f5-b7527c26c493: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 03:42:28,712::task::495::TaskManager.Task::(_debug) Task 56671927-b0f0-4a9e-b8f5-b7527c26c493: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 03:42:28,713::task::495::TaskManager.Task::(_debug) Task 56671927-b0f0-4a9e-b8f5-b7527c26c493: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 03:42:28,714::task::495::TaskManager.Task::(_debug) Task 56671927-b0f0-4a9e-b8f5-b7527c26c493: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 03:42:28,714::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 03:42:28,715::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 03:42:28,716::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 03:42:28,717::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 03:42:28,717::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 03:42:28,718::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 03:42:28,719::task::495::TaskManager.Task::(_debug) Task 56671927-b0f0-4a9e-b8f5-b7527c26c493: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 03:42:28,720::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 03:42:29,334::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:42:29,900::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0944168 s, 10.8 MB/s\n'; <rc> = 0
>Thread-15914453::INFO::2013-11-20 03:42:30,994::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 03:42:30,995::task::495::TaskManager.Task::(_debug) Task 70caf324-ce2e-4aee-af74-9fb298ae5c78: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 03:42:31,002::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`708b4c7f-d6c0-42b9-a1d6-0fcca41579fd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 03:42:31,003::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 03:42:31,004::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 03:42:31,005::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`708b4c7f-d6c0-42b9-a1d6-0fcca41579fd`::Granted request
>Thread-15914453::DEBUG::2013-11-20 03:42:31,006::task::495::TaskManager.Task::(_debug) Task 70caf324-ce2e-4aee-af74-9fb298ae5c78: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 03:42:31,006::task::495::TaskManager.Task::(_debug) Task 70caf324-ce2e-4aee-af74-9fb298ae5c78: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 03:42:31,007::task::495::TaskManager.Task::(_debug) Task 70caf324-ce2e-4aee-af74-9fb298ae5c78: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 03:42:31,008::task::495::TaskManager.Task::(_debug) Task 70caf324-ce2e-4aee-af74-9fb298ae5c78: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 03:42:31,009::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 03:42:31,010::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 03:42:31,012::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 03:42:31,013::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 03:42:31,014::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 03:42:31,015::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 03:42:31,016::task::495::TaskManager.Task::(_debug) Task 70caf324-ce2e-4aee-af74-9fb298ae5c78: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 03:42:31,017::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 03:42:31,914::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:42:32,515::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0886883 s, 11.5 MB/s\n'; <rc> = 0
>Thread-15914846::DEBUG::2013-11-20 03:42:32,635::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914846::INFO::2013-11-20 03:42:32,636::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914846::DEBUG::2013-11-20 03:42:32,637::task::495::TaskManager.Task::(_debug) Task 08e3c01b-f98e-4f1a-9585-9772e3ddadde: moving from state init -> state preparing
>Thread-15914846::DEBUG::2013-11-20 03:42:32,638::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914846::DEBUG::2013-11-20 03:42:32,639::task::495::TaskManager.Task::(_debug) Task 08e3c01b-f98e-4f1a-9585-9772e3ddadde: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914846::DEBUG::2013-11-20 03:42:32,639::task::495::TaskManager.Task::(_debug) Task 08e3c01b-f98e-4f1a-9585-9772e3ddadde: moving from state preparing -> state finished
>Thread-15914846::DEBUG::2013-11-20 03:42:32,640::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914846::DEBUG::2013-11-20 03:42:32,640::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914846::DEBUG::2013-11-20 03:42:32,641::task::495::TaskManager.Task::(_debug) Task 08e3c01b-f98e-4f1a-9585-9772e3ddadde: ref 0 aborting False
>Thread-15914846::INFO::2013-11-20 03:42:32,642::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914847::DEBUG::2013-11-20 03:42:32,665::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914847::INFO::2013-11-20 03:42:32,667::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914847::DEBUG::2013-11-20 03:42:32,668::task::495::TaskManager.Task::(_debug) Task 816ccfb4-09a8-4cd8-b94b-f8b52269ace4: moving from state init -> state preparing
>Thread-15914847::DEBUG::2013-11-20 03:42:32,670::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b1d14303-7dfd-4b97-bd82-abac68deacc1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914847::DEBUG::2013-11-20 03:42:32,671::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914847::DEBUG::2013-11-20 03:42:32,672::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914847::DEBUG::2013-11-20 03:42:32,672::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b1d14303-7dfd-4b97-bd82-abac68deacc1`::Granted request
>Thread-15914847::DEBUG::2013-11-20 03:42:32,673::task::495::TaskManager.Task::(_debug) Task 816ccfb4-09a8-4cd8-b94b-f8b52269ace4: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914847::DEBUG::2013-11-20 03:42:32,674::task::495::TaskManager.Task::(_debug) Task 816ccfb4-09a8-4cd8-b94b-f8b52269ace4: ref 1 aborting False
>Thread-15914847::WARNING::2013-11-20 03:42:32,676::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914847::DEBUG::2013-11-20 03:42:32,677::task::495::TaskManager.Task::(_debug) Task 816ccfb4-09a8-4cd8-b94b-f8b52269ace4: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914847::DEBUG::2013-11-20 03:42:32,678::task::495::TaskManager.Task::(_debug) Task 816ccfb4-09a8-4cd8-b94b-f8b52269ace4: moving from state preparing -> state finished
>Thread-15914847::DEBUG::2013-11-20 03:42:32,679::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914847::DEBUG::2013-11-20 03:42:32,679::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914847::DEBUG::2013-11-20 03:42:32,680::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914847::DEBUG::2013-11-20 03:42:32,681::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914847::DEBUG::2013-11-20 03:42:32,681::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914847::DEBUG::2013-11-20 03:42:32,682::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914847::DEBUG::2013-11-20 03:42:32,683::task::495::TaskManager.Task::(_debug) Task 816ccfb4-09a8-4cd8-b94b-f8b52269ace4: ref 0 aborting False
>Thread-15914847::INFO::2013-11-20 03:42:32,683::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:42:34,532::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:42:35,109::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0809183 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:42:37,125::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 03:42:37,345::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 03:42:37,348::task::495::TaskManager.Task::(_debug) Task 859faaf3-5eb1-4e8c-9a01-b13ca6a80e30: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 03:42:37,356::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1a0b992e-5e25-4358-b6ff-ebfb9d4467ba`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 03:42:37,358::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 03:42:37,359::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 03:42:37,360::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1a0b992e-5e25-4358-b6ff-ebfb9d4467ba`::Granted request
>Thread-15914519::DEBUG::2013-11-20 03:42:37,361::task::495::TaskManager.Task::(_debug) Task 859faaf3-5eb1-4e8c-9a01-b13ca6a80e30: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 03:42:37,362::task::495::TaskManager.Task::(_debug) Task 859faaf3-5eb1-4e8c-9a01-b13ca6a80e30: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 03:42:37,363::task::495::TaskManager.Task::(_debug) Task 859faaf3-5eb1-4e8c-9a01-b13ca6a80e30: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 03:42:37,364::task::495::TaskManager.Task::(_debug) Task 859faaf3-5eb1-4e8c-9a01-b13ca6a80e30: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 03:42:37,364::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 03:42:37,365::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 03:42:37,368::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 03:42:37,368::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 03:42:37,369::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 03:42:37,370::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 03:42:37,370::task::495::TaskManager.Task::(_debug) Task 859faaf3-5eb1-4e8c-9a01-b13ca6a80e30: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 03:42:37,371::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 03:42:37,681::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0763001 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:42:39,697::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913518::INFO::2013-11-20 03:42:39,913::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:42:39,915::task::495::TaskManager.Task::(_debug) Task dd18e1c6-a562-47de-b547-6cd093d36d60: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:42:39,918::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9407524b-5cfa-468b-a2d4-4616bdb4e7a6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:42:39,919::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:42:39,920::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:42:39,921::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9407524b-5cfa-468b-a2d4-4616bdb4e7a6`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:42:39,922::task::495::TaskManager.Task::(_debug) Task dd18e1c6-a562-47de-b547-6cd093d36d60: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:42:39,923::task::495::TaskManager.Task::(_debug) Task dd18e1c6-a562-47de-b547-6cd093d36d60: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:42:39,924::task::495::TaskManager.Task::(_debug) Task dd18e1c6-a562-47de-b547-6cd093d36d60: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:42:39,924::task::495::TaskManager.Task::(_debug) Task dd18e1c6-a562-47de-b547-6cd093d36d60: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:42:39,925::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:42:39,926::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:42:39,927::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:42:39,928::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:42:39,929::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:42:39,930::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:42:39,930::task::495::TaskManager.Task::(_debug) Task dd18e1c6-a562-47de-b547-6cd093d36d60: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:42:39,931::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:42:39,932::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:42:39,933::task::495::TaskManager.Task::(_debug) Task fb2995b6-99fa-4135-ac81-2b74efb44c77: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:42:39,934::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c91b20e8-f460-4717-ad62-b0dc875fecf7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:42:39,934::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:42:39,935::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:42:39,936::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c91b20e8-f460-4717-ad62-b0dc875fecf7`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:42:39,937::task::495::TaskManager.Task::(_debug) Task fb2995b6-99fa-4135-ac81-2b74efb44c77: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:42:39,937::task::495::TaskManager.Task::(_debug) Task fb2995b6-99fa-4135-ac81-2b74efb44c77: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:42:39,938::task::495::TaskManager.Task::(_debug) Task fb2995b6-99fa-4135-ac81-2b74efb44c77: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:42:39,939::task::495::TaskManager.Task::(_debug) Task fb2995b6-99fa-4135-ac81-2b74efb44c77: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:42:39,939::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:42:39,940::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:42:39,941::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:42:39,941::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:42:39,942::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:42:39,942::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:42:39,943::task::495::TaskManager.Task::(_debug) Task fb2995b6-99fa-4135-ac81-2b74efb44c77: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:42:39,944::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:42:40,247::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0869432 s, 11.8 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 03:42:41,030::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:42:41,039::task::495::TaskManager.Task::(_debug) Task 4da8fd9e-7ac9-4151-a0fe-6bccc4af584e: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:42:41,040::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f526e124-743c-4c99-857c-a6b725e4a601`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:42:41,041::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:42:41,042::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:42:41,043::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f526e124-743c-4c99-857c-a6b725e4a601`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:42:41,044::task::495::TaskManager.Task::(_debug) Task 4da8fd9e-7ac9-4151-a0fe-6bccc4af584e: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:42:41,044::task::495::TaskManager.Task::(_debug) Task 4da8fd9e-7ac9-4151-a0fe-6bccc4af584e: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:42:41,045::task::495::TaskManager.Task::(_debug) Task 4da8fd9e-7ac9-4151-a0fe-6bccc4af584e: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:42:41,046::task::495::TaskManager.Task::(_debug) Task 4da8fd9e-7ac9-4151-a0fe-6bccc4af584e: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:42:41,046::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:42:41,047::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:42:41,048::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:42:41,048::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:42:41,049::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:42:41,049::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:42:41,050::task::495::TaskManager.Task::(_debug) Task 4da8fd9e-7ac9-4151-a0fe-6bccc4af584e: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:42:41,050::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:42:41,051::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:42:41,052::task::495::TaskManager.Task::(_debug) Task c21e3072-c248-4b53-95da-e055fa251bcf: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:42:41,054::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`075cdc07-7ca1-46d9-acc3-f73c322f18af`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:42:41,054::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:42:41,055::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:42:41,056::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`075cdc07-7ca1-46d9-acc3-f73c322f18af`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:42:41,057::task::495::TaskManager.Task::(_debug) Task c21e3072-c248-4b53-95da-e055fa251bcf: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:42:41,058::task::495::TaskManager.Task::(_debug) Task c21e3072-c248-4b53-95da-e055fa251bcf: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:42:41,059::task::495::TaskManager.Task::(_debug) Task c21e3072-c248-4b53-95da-e055fa251bcf: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:42:41,059::task::495::TaskManager.Task::(_debug) Task c21e3072-c248-4b53-95da-e055fa251bcf: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:42:41,060::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:42:41,060::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:42:41,061::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:42:41,061::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:42:41,062::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:42:41,063::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:42:41,064::task::495::TaskManager.Task::(_debug) Task c21e3072-c248-4b53-95da-e055fa251bcf: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:42:41,064::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:42:41,065::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:42:41,066::task::495::TaskManager.Task::(_debug) Task 40030f9c-bb65-437e-937c-68cdbdca8c96: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:42:41,067::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6cf88195-a643-473d-b2ab-64a9f10f45e0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:42:41,068::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:42:41,068::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:42:41,069::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6cf88195-a643-473d-b2ab-64a9f10f45e0`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:42:41,070::task::495::TaskManager.Task::(_debug) Task 40030f9c-bb65-437e-937c-68cdbdca8c96: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:42:41,070::task::495::TaskManager.Task::(_debug) Task 40030f9c-bb65-437e-937c-68cdbdca8c96: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:42:41,071::task::495::TaskManager.Task::(_debug) Task 40030f9c-bb65-437e-937c-68cdbdca8c96: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:42:41,072::task::495::TaskManager.Task::(_debug) Task 40030f9c-bb65-437e-937c-68cdbdca8c96: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:42:41,072::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:42:41,073::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:42:41,073::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:42:41,074::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:42:41,074::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:42:41,075::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:42:41,076::task::495::TaskManager.Task::(_debug) Task 40030f9c-bb65-437e-937c-68cdbdca8c96: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:42:41,076::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:42:41,077::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:42:41,077::task::495::TaskManager.Task::(_debug) Task de2ca9ab-7c8d-4622-be90-26a89d240b51: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:42:41,079::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`05468d4a-3e25-47ef-b241-2f6d9f19987e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:42:41,079::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:42:41,080::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:42:41,081::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`05468d4a-3e25-47ef-b241-2f6d9f19987e`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:42:41,081::task::495::TaskManager.Task::(_debug) Task de2ca9ab-7c8d-4622-be90-26a89d240b51: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:42:41,082::task::495::TaskManager.Task::(_debug) Task de2ca9ab-7c8d-4622-be90-26a89d240b51: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:42:41,083::task::495::TaskManager.Task::(_debug) Task de2ca9ab-7c8d-4622-be90-26a89d240b51: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:42:41,083::task::495::TaskManager.Task::(_debug) Task de2ca9ab-7c8d-4622-be90-26a89d240b51: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:42:41,084::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:42:41,084::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:42:41,084::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:42:41,085::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:42:41,086::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:42:41,086::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:42:41,087::task::495::TaskManager.Task::(_debug) Task de2ca9ab-7c8d-4622-be90-26a89d240b51: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:42:41,087::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:42:42,262::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:42:42,848::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.074635 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15914852::DEBUG::2013-11-20 03:42:43,674::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914852::INFO::2013-11-20 03:42:43,675::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914852::DEBUG::2013-11-20 03:42:43,676::task::495::TaskManager.Task::(_debug) Task 47fdc66b-cf7f-4cf8-83a8-b12d800cbad5: moving from state init -> state preparing
>Thread-15914852::DEBUG::2013-11-20 03:42:43,677::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914852::DEBUG::2013-11-20 03:42:43,678::task::495::TaskManager.Task::(_debug) Task 47fdc66b-cf7f-4cf8-83a8-b12d800cbad5: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914852::DEBUG::2013-11-20 03:42:43,678::task::495::TaskManager.Task::(_debug) Task 47fdc66b-cf7f-4cf8-83a8-b12d800cbad5: moving from state preparing -> state finished
>Thread-15914852::DEBUG::2013-11-20 03:42:43,679::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914852::DEBUG::2013-11-20 03:42:43,680::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914852::DEBUG::2013-11-20 03:42:43,680::task::495::TaskManager.Task::(_debug) Task 47fdc66b-cf7f-4cf8-83a8-b12d800cbad5: ref 0 aborting False
>Thread-15914852::INFO::2013-11-20 03:42:43,681::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914853::DEBUG::2013-11-20 03:42:43,690::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914853::INFO::2013-11-20 03:42:43,691::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914853::DEBUG::2013-11-20 03:42:43,691::task::495::TaskManager.Task::(_debug) Task e07e916b-9965-4ba8-877e-d1f1fec723b0: moving from state init -> state preparing
>Thread-15914853::DEBUG::2013-11-20 03:42:43,693::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`be95bf3d-4a6f-4d78-aa6a-8b63935a5726`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914853::DEBUG::2013-11-20 03:42:43,694::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914853::DEBUG::2013-11-20 03:42:43,695::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914853::DEBUG::2013-11-20 03:42:43,695::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`be95bf3d-4a6f-4d78-aa6a-8b63935a5726`::Granted request
>Thread-15914853::DEBUG::2013-11-20 03:42:43,696::task::495::TaskManager.Task::(_debug) Task e07e916b-9965-4ba8-877e-d1f1fec723b0: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914853::DEBUG::2013-11-20 03:42:43,697::task::495::TaskManager.Task::(_debug) Task e07e916b-9965-4ba8-877e-d1f1fec723b0: ref 1 aborting False
>Thread-15914853::WARNING::2013-11-20 03:42:43,699::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914853::DEBUG::2013-11-20 03:42:43,700::task::495::TaskManager.Task::(_debug) Task e07e916b-9965-4ba8-877e-d1f1fec723b0: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914853::DEBUG::2013-11-20 03:42:43,702::task::495::TaskManager.Task::(_debug) Task e07e916b-9965-4ba8-877e-d1f1fec723b0: moving from state preparing -> state finished
>Thread-15914853::DEBUG::2013-11-20 03:42:43,702::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914853::DEBUG::2013-11-20 03:42:43,703::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914853::DEBUG::2013-11-20 03:42:43,704::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914853::DEBUG::2013-11-20 03:42:43,704::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914853::DEBUG::2013-11-20 03:42:43,705::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914853::DEBUG::2013-11-20 03:42:43,706::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914853::DEBUG::2013-11-20 03:42:43,706::task::495::TaskManager.Task::(_debug) Task e07e916b-9965-4ba8-877e-d1f1fec723b0: ref 0 aborting False
>Thread-15914853::INFO::2013-11-20 03:42:43,707::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:42:44,864::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914854::INFO::2013-11-20 03:42:45,165::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914854::DEBUG::2013-11-20 03:42:45,169::task::495::TaskManager.Task::(_debug) Task 48858104-6594-4765-b3af-f7eea0ccef46: moving from state init -> state preparing
>Thread-15914854::DEBUG::2013-11-20 03:42:45,170::task::495::TaskManager.Task::(_debug) Task 48858104-6594-4765-b3af-f7eea0ccef46: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000431060791016', 'lastCheck': 1384918961.9010761, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00171518325806', 'lastCheck': 1384918955.7428911, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00151300430298', 'lastCheck': 1384918961.901762, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000457048416138', 'lastCheck': 1384918955.1424949, 'valid': True, 'code': 0}}
>Thread-15914854::DEBUG::2013-11-20 03:42:45,172::task::495::TaskManager.Task::(_debug) Task 48858104-6594-4765-b3af-f7eea0ccef46: moving from state preparing -> state finished
>Thread-15914854::DEBUG::2013-11-20 03:42:45,173::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914854::DEBUG::2013-11-20 03:42:45,174::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914854::DEBUG::2013-11-20 03:42:45,175::task::495::TaskManager.Task::(_debug) Task 48858104-6594-4765-b3af-f7eea0ccef46: ref 0 aborting False
>Thread-15914854::INFO::2013-11-20 03:42:45,176::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000431060791016', 'lastCheck': 1384918961.9010761, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00171518325806', 'lastCheck': 1384918955.7428911, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00151300430298', 'lastCheck': 1384918961.901762, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000457048416138', 'lastCheck': 1384918955.1424949, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:42:45,433::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0733718 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:42:47,448::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:42:47,964::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0765569 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:42:49,980::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:42:50,528::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0782253 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:42:52,543::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:42:53,009::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0525546 s, 19.5 MB/s\n'; <rc> = 0
>Thread-15914858::DEBUG::2013-11-20 03:42:54,080::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914858::INFO::2013-11-20 03:42:54,082::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914858::DEBUG::2013-11-20 03:42:54,082::task::495::TaskManager.Task::(_debug) Task 666cf10e-c98e-46a5-a5c1-f4e8307234ac: moving from state init -> state preparing
>Thread-15914858::DEBUG::2013-11-20 03:42:54,084::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914858::DEBUG::2013-11-20 03:42:54,084::task::495::TaskManager.Task::(_debug) Task 666cf10e-c98e-46a5-a5c1-f4e8307234ac: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914858::DEBUG::2013-11-20 03:42:54,085::task::495::TaskManager.Task::(_debug) Task 666cf10e-c98e-46a5-a5c1-f4e8307234ac: moving from state preparing -> state finished
>Thread-15914858::DEBUG::2013-11-20 03:42:54,086::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914858::DEBUG::2013-11-20 03:42:54,086::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914858::DEBUG::2013-11-20 03:42:54,087::task::495::TaskManager.Task::(_debug) Task 666cf10e-c98e-46a5-a5c1-f4e8307234ac: ref 0 aborting False
>Thread-15914858::INFO::2013-11-20 03:42:54,088::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914859::DEBUG::2013-11-20 03:42:54,100::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914859::INFO::2013-11-20 03:42:54,101::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914859::DEBUG::2013-11-20 03:42:54,102::task::495::TaskManager.Task::(_debug) Task 78145966-78bf-4e54-8e76-19058f194010: moving from state init -> state preparing
>Thread-15914859::DEBUG::2013-11-20 03:42:54,104::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`78c72064-672e-4af6-8b4f-778446c46d8e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914859::DEBUG::2013-11-20 03:42:54,105::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914859::DEBUG::2013-11-20 03:42:54,107::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914859::DEBUG::2013-11-20 03:42:54,107::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`78c72064-672e-4af6-8b4f-778446c46d8e`::Granted request
>Thread-15914859::DEBUG::2013-11-20 03:42:54,118::task::495::TaskManager.Task::(_debug) Task 78145966-78bf-4e54-8e76-19058f194010: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914859::DEBUG::2013-11-20 03:42:54,119::task::495::TaskManager.Task::(_debug) Task 78145966-78bf-4e54-8e76-19058f194010: ref 1 aborting False
>Thread-15914859::WARNING::2013-11-20 03:42:54,122::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914859::DEBUG::2013-11-20 03:42:54,124::task::495::TaskManager.Task::(_debug) Task 78145966-78bf-4e54-8e76-19058f194010: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914859::DEBUG::2013-11-20 03:42:54,126::task::495::TaskManager.Task::(_debug) Task 78145966-78bf-4e54-8e76-19058f194010: moving from state preparing -> state finished
>Thread-15914859::DEBUG::2013-11-20 03:42:54,127::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914859::DEBUG::2013-11-20 03:42:54,128::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914859::DEBUG::2013-11-20 03:42:54,129::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914859::DEBUG::2013-11-20 03:42:54,131::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914859::DEBUG::2013-11-20 03:42:54,133::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914859::DEBUG::2013-11-20 03:42:54,134::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914859::DEBUG::2013-11-20 03:42:54,135::task::495::TaskManager.Task::(_debug) Task 78145966-78bf-4e54-8e76-19058f194010: ref 0 aborting False
>Thread-15914859::INFO::2013-11-20 03:42:54,138::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:42:55,020::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:42:55,590::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0733614 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:42:57,604::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 03:42:57,830::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:42:57,833::task::495::TaskManager.Task::(_debug) Task 00e4c532-d910-416d-9dba-ff859afb9451: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:42:57,836::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`778dba75-5c5c-4415-96c7-220ecd86a18a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:42:57,837::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:42:57,838::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:42:57,839::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`778dba75-5c5c-4415-96c7-220ecd86a18a`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:42:57,840::task::495::TaskManager.Task::(_debug) Task 00e4c532-d910-416d-9dba-ff859afb9451: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:42:57,841::task::495::TaskManager.Task::(_debug) Task 00e4c532-d910-416d-9dba-ff859afb9451: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:42:57,842::task::495::TaskManager.Task::(_debug) Task 00e4c532-d910-416d-9dba-ff859afb9451: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:42:57,843::task::495::TaskManager.Task::(_debug) Task 00e4c532-d910-416d-9dba-ff859afb9451: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:42:57,844::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:42:57,845::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:42:57,846::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:42:57,847::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:42:57,848::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:42:57,849::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:42:57,850::task::495::TaskManager.Task::(_debug) Task 00e4c532-d910-416d-9dba-ff859afb9451: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:42:57,850::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:42:58,152::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0885794 s, 11.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:43:00,167::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:43:00,735::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0812775 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15914862::INFO::2013-11-20 03:43:00,978::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914862::DEBUG::2013-11-20 03:43:00,979::task::495::TaskManager.Task::(_debug) Task 42fe94e7-fedf-4d39-92b8-f810ce49328d: moving from state init -> state preparing
>Thread-15914862::DEBUG::2013-11-20 03:43:00,980::task::495::TaskManager.Task::(_debug) Task 42fe94e7-fedf-4d39-92b8-f810ce49328d: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00100588798523', 'lastCheck': 1384918972.196964, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0454580783844', 'lastCheck': 1384918976.305496, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0013279914856', 'lastCheck': 1384918972.19889, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000439882278442', 'lastCheck': 1384918975.757828, 'valid': True, 'code': 0}}
>Thread-15914862::DEBUG::2013-11-20 03:43:00,988::task::495::TaskManager.Task::(_debug) Task 42fe94e7-fedf-4d39-92b8-f810ce49328d: moving from state preparing -> state finished
>Thread-15914862::DEBUG::2013-11-20 03:43:00,989::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914862::DEBUG::2013-11-20 03:43:00,990::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914862::DEBUG::2013-11-20 03:43:00,991::task::495::TaskManager.Task::(_debug) Task 42fe94e7-fedf-4d39-92b8-f810ce49328d: ref 0 aborting False
>Thread-15914862::INFO::2013-11-20 03:43:00,995::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00100588798523', 'lastCheck': 1384918972.196964, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0454580783844', 'lastCheck': 1384918976.305496, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0013279914856', 'lastCheck': 1384918972.19889, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000439882278442', 'lastCheck': 1384918975.757828, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:43:02,751::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:43:03,318::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0766278 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15914864::DEBUG::2013-11-20 03:43:04,641::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914864::INFO::2013-11-20 03:43:04,643::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914864::DEBUG::2013-11-20 03:43:04,643::task::495::TaskManager.Task::(_debug) Task c435aafc-b318-4b9a-ba8a-23a73cb84ce2: moving from state init -> state preparing
>Thread-15914864::DEBUG::2013-11-20 03:43:04,644::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914864::DEBUG::2013-11-20 03:43:04,645::task::495::TaskManager.Task::(_debug) Task c435aafc-b318-4b9a-ba8a-23a73cb84ce2: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914864::DEBUG::2013-11-20 03:43:04,646::task::495::TaskManager.Task::(_debug) Task c435aafc-b318-4b9a-ba8a-23a73cb84ce2: moving from state preparing -> state finished
>Thread-15914864::DEBUG::2013-11-20 03:43:04,646::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914864::DEBUG::2013-11-20 03:43:04,647::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914864::DEBUG::2013-11-20 03:43:04,648::task::495::TaskManager.Task::(_debug) Task c435aafc-b318-4b9a-ba8a-23a73cb84ce2: ref 0 aborting False
>Thread-15914864::INFO::2013-11-20 03:43:04,648::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914865::DEBUG::2013-11-20 03:43:04,664::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914865::INFO::2013-11-20 03:43:04,665::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914865::DEBUG::2013-11-20 03:43:04,666::task::495::TaskManager.Task::(_debug) Task e1c1b0ea-7ae2-49a3-9f5c-8e7f607d2e64: moving from state init -> state preparing
>Thread-15914865::DEBUG::2013-11-20 03:43:04,667::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b421c6fd-b700-4f91-9e28-6f57682b8450`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914865::DEBUG::2013-11-20 03:43:04,668::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914865::DEBUG::2013-11-20 03:43:04,669::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914865::DEBUG::2013-11-20 03:43:04,670::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b421c6fd-b700-4f91-9e28-6f57682b8450`::Granted request
>Thread-15914865::DEBUG::2013-11-20 03:43:04,671::task::495::TaskManager.Task::(_debug) Task e1c1b0ea-7ae2-49a3-9f5c-8e7f607d2e64: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914865::DEBUG::2013-11-20 03:43:04,671::task::495::TaskManager.Task::(_debug) Task e1c1b0ea-7ae2-49a3-9f5c-8e7f607d2e64: ref 1 aborting False
>Thread-15914865::WARNING::2013-11-20 03:43:04,674::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914865::DEBUG::2013-11-20 03:43:04,678::task::495::TaskManager.Task::(_debug) Task e1c1b0ea-7ae2-49a3-9f5c-8e7f607d2e64: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914865::DEBUG::2013-11-20 03:43:04,680::task::495::TaskManager.Task::(_debug) Task e1c1b0ea-7ae2-49a3-9f5c-8e7f607d2e64: moving from state preparing -> state finished
>Thread-15914865::DEBUG::2013-11-20 03:43:04,680::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914865::DEBUG::2013-11-20 03:43:04,681::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914865::DEBUG::2013-11-20 03:43:04,682::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914865::DEBUG::2013-11-20 03:43:04,682::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914865::DEBUG::2013-11-20 03:43:04,683::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914865::DEBUG::2013-11-20 03:43:04,683::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914865::DEBUG::2013-11-20 03:43:04,684::task::495::TaskManager.Task::(_debug) Task e1c1b0ea-7ae2-49a3-9f5c-8e7f607d2e64: ref 0 aborting False
>Thread-15914865::INFO::2013-11-20 03:43:04,689::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:43:05,332::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:43:05,845::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0770279 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:43:07,858::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:43:08,430::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0827979 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:43:10,446::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:43:11,002::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0843817 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 03:43:11,286::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:43:11,297::task::495::TaskManager.Task::(_debug) Task 889004c5-fef9-4ce2-ab38-ebb1246c249c: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:43:11,299::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`81477354-c514-47c7-a2d2-c89d7c51af4b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:43:11,301::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:43:11,302::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:43:11,303::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`81477354-c514-47c7-a2d2-c89d7c51af4b`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:43:11,304::task::495::TaskManager.Task::(_debug) Task 889004c5-fef9-4ce2-ab38-ebb1246c249c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:43:11,304::task::495::TaskManager.Task::(_debug) Task 889004c5-fef9-4ce2-ab38-ebb1246c249c: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:43:11,305::task::495::TaskManager.Task::(_debug) Task 889004c5-fef9-4ce2-ab38-ebb1246c249c: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:43:11,306::task::495::TaskManager.Task::(_debug) Task 889004c5-fef9-4ce2-ab38-ebb1246c249c: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:43:11,306::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:43:11,307::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:43:11,307::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:43:11,308::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:43:11,308::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:43:11,309::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:43:11,310::task::495::TaskManager.Task::(_debug) Task 889004c5-fef9-4ce2-ab38-ebb1246c249c: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:43:11,310::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:43:13,019::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:43:13,595::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0709613 s, 14.4 MB/s\n'; <rc> = 0
>Thread-15914870::DEBUG::2013-11-20 03:43:15,055::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914870::INFO::2013-11-20 03:43:15,056::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914870::DEBUG::2013-11-20 03:43:15,057::task::495::TaskManager.Task::(_debug) Task 8b7c10ca-a810-4b2d-801a-d76cc0323a42: moving from state init -> state preparing
>Thread-15914870::DEBUG::2013-11-20 03:43:15,058::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914870::DEBUG::2013-11-20 03:43:15,059::task::495::TaskManager.Task::(_debug) Task 8b7c10ca-a810-4b2d-801a-d76cc0323a42: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914870::DEBUG::2013-11-20 03:43:15,060::task::495::TaskManager.Task::(_debug) Task 8b7c10ca-a810-4b2d-801a-d76cc0323a42: moving from state preparing -> state finished
>Thread-15914870::DEBUG::2013-11-20 03:43:15,060::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914870::DEBUG::2013-11-20 03:43:15,061::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914870::DEBUG::2013-11-20 03:43:15,061::task::495::TaskManager.Task::(_debug) Task 8b7c10ca-a810-4b2d-801a-d76cc0323a42: ref 0 aborting False
>Thread-15914870::INFO::2013-11-20 03:43:15,062::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914871::DEBUG::2013-11-20 03:43:15,078::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914871::INFO::2013-11-20 03:43:15,079::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914871::DEBUG::2013-11-20 03:43:15,084::task::495::TaskManager.Task::(_debug) Task c056cf28-b93a-4c01-a9e5-0d33cec6e079: moving from state init -> state preparing
>Thread-15914871::DEBUG::2013-11-20 03:43:15,086::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`40c0a3b4-b7d4-4fb5-9e3b-eea3b7eba207`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914871::DEBUG::2013-11-20 03:43:15,086::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914871::DEBUG::2013-11-20 03:43:15,087::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914871::DEBUG::2013-11-20 03:43:15,088::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`40c0a3b4-b7d4-4fb5-9e3b-eea3b7eba207`::Granted request
>Thread-15914871::DEBUG::2013-11-20 03:43:15,090::task::495::TaskManager.Task::(_debug) Task c056cf28-b93a-4c01-a9e5-0d33cec6e079: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914871::DEBUG::2013-11-20 03:43:15,091::task::495::TaskManager.Task::(_debug) Task c056cf28-b93a-4c01-a9e5-0d33cec6e079: ref 1 aborting False
>Thread-15914871::WARNING::2013-11-20 03:43:15,093::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914871::DEBUG::2013-11-20 03:43:15,094::task::495::TaskManager.Task::(_debug) Task c056cf28-b93a-4c01-a9e5-0d33cec6e079: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914871::DEBUG::2013-11-20 03:43:15,095::task::495::TaskManager.Task::(_debug) Task c056cf28-b93a-4c01-a9e5-0d33cec6e079: moving from state preparing -> state finished
>Thread-15914871::DEBUG::2013-11-20 03:43:15,096::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914871::DEBUG::2013-11-20 03:43:15,097::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914871::DEBUG::2013-11-20 03:43:15,097::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914871::DEBUG::2013-11-20 03:43:15,098::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914871::DEBUG::2013-11-20 03:43:15,099::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914871::DEBUG::2013-11-20 03:43:15,099::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914871::DEBUG::2013-11-20 03:43:15,100::task::495::TaskManager.Task::(_debug) Task c056cf28-b93a-4c01-a9e5-0d33cec6e079: ref 0 aborting False
>Thread-15914871::INFO::2013-11-20 03:43:15,101::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:43:15,614::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:43:16,164::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.084332 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15914872::INFO::2013-11-20 03:43:17,220::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914872::DEBUG::2013-11-20 03:43:17,221::task::495::TaskManager.Task::(_debug) Task 906927ba-3120-4b6a-b24a-ad9f4fde1948: moving from state init -> state preparing
>Thread-15914872::DEBUG::2013-11-20 03:43:17,222::task::495::TaskManager.Task::(_debug) Task 906927ba-3120-4b6a-b24a-ad9f4fde1948: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000354051589966', 'lastCheck': 1384918992.654346, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00160598754883', 'lastCheck': 1384918996.8374791, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00800085067749', 'lastCheck': 1384918992.661474, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000545978546143', 'lastCheck': 1384918996.271744, 'valid': True, 'code': 0}}
>Thread-15914872::DEBUG::2013-11-20 03:43:17,223::task::495::TaskManager.Task::(_debug) Task 906927ba-3120-4b6a-b24a-ad9f4fde1948: moving from state preparing -> state finished
>Thread-15914872::DEBUG::2013-11-20 03:43:17,223::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914872::DEBUG::2013-11-20 03:43:17,224::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914872::DEBUG::2013-11-20 03:43:17,225::task::495::TaskManager.Task::(_debug) Task 906927ba-3120-4b6a-b24a-ad9f4fde1948: ref 0 aborting False
>Thread-15914872::INFO::2013-11-20 03:43:17,225::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000354051589966', 'lastCheck': 1384918992.654346, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00160598754883', 'lastCheck': 1384918996.8374791, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00800085067749', 'lastCheck': 1384918992.661474, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000545978546143', 'lastCheck': 1384918996.271744, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:43:18,181::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 03:43:18,400::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:43:18,404::task::495::TaskManager.Task::(_debug) Task 6e47677d-6b99-4f69-b5d8-ff5478886ccd: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:43:18,407::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`cb16cd1c-1e2e-4b5f-91b7-d7c8241f4baa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:43:18,409::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:43:18,410::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:43:18,410::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`cb16cd1c-1e2e-4b5f-91b7-d7c8241f4baa`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:43:18,411::task::495::TaskManager.Task::(_debug) Task 6e47677d-6b99-4f69-b5d8-ff5478886ccd: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:43:18,412::task::495::TaskManager.Task::(_debug) Task 6e47677d-6b99-4f69-b5d8-ff5478886ccd: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:43:18,413::task::495::TaskManager.Task::(_debug) Task 6e47677d-6b99-4f69-b5d8-ff5478886ccd: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:43:18,414::task::495::TaskManager.Task::(_debug) Task 6e47677d-6b99-4f69-b5d8-ff5478886ccd: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:43:18,415::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:43:18,415::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:43:18,416::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:43:18,417::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:43:18,417::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:43:18,418::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:43:18,419::task::495::TaskManager.Task::(_debug) Task 6e47677d-6b99-4f69-b5d8-ff5478886ccd: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:43:18,419::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:43:18,730::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.072033 s, 14.2 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 03:43:19,390::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:43:19,391::task::495::TaskManager.Task::(_debug) Task 9781a6f3-b40f-4e6a-adf9-621703f2b067: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:43:19,392::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ca0d044b-d2a1-46c9-9709-85f707813ad1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:43:19,393::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:43:19,394::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:43:19,395::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ca0d044b-d2a1-46c9-9709-85f707813ad1`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:43:19,396::task::495::TaskManager.Task::(_debug) Task 9781a6f3-b40f-4e6a-adf9-621703f2b067: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:43:19,397::task::495::TaskManager.Task::(_debug) Task 9781a6f3-b40f-4e6a-adf9-621703f2b067: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:43:19,398::task::495::TaskManager.Task::(_debug) Task 9781a6f3-b40f-4e6a-adf9-621703f2b067: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:43:19,398::task::495::TaskManager.Task::(_debug) Task 9781a6f3-b40f-4e6a-adf9-621703f2b067: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:43:19,399::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:43:19,400::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:43:19,400::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:43:19,401::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:43:19,402::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:43:19,402::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:43:19,403::task::495::TaskManager.Task::(_debug) Task 9781a6f3-b40f-4e6a-adf9-621703f2b067: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:43:19,403::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:43:20,746::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:43:21,300::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.111797 s, 9.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:43:23,317::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:43:23,863::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0761448 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15914876::DEBUG::2013-11-20 03:43:25,745::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914876::INFO::2013-11-20 03:43:25,747::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914876::DEBUG::2013-11-20 03:43:25,747::task::495::TaskManager.Task::(_debug) Task bc824250-8f9a-48d8-b910-6802804fdc9b: moving from state init -> state preparing
>Thread-15914876::DEBUG::2013-11-20 03:43:25,748::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914876::DEBUG::2013-11-20 03:43:25,749::task::495::TaskManager.Task::(_debug) Task bc824250-8f9a-48d8-b910-6802804fdc9b: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914876::DEBUG::2013-11-20 03:43:25,750::task::495::TaskManager.Task::(_debug) Task bc824250-8f9a-48d8-b910-6802804fdc9b: moving from state preparing -> state finished
>Thread-15914876::DEBUG::2013-11-20 03:43:25,751::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914876::DEBUG::2013-11-20 03:43:25,751::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914876::DEBUG::2013-11-20 03:43:25,752::task::495::TaskManager.Task::(_debug) Task bc824250-8f9a-48d8-b910-6802804fdc9b: ref 0 aborting False
>Thread-15914876::INFO::2013-11-20 03:43:25,752::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914877::DEBUG::2013-11-20 03:43:25,757::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914877::INFO::2013-11-20 03:43:25,758::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914877::DEBUG::2013-11-20 03:43:25,759::task::495::TaskManager.Task::(_debug) Task 24f2bdad-cd94-42f4-89a8-579533dc38b0: moving from state init -> state preparing
>Thread-15914877::DEBUG::2013-11-20 03:43:25,760::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ef0c1414-90ad-40b1-bb73-5a0edd14baab`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914877::DEBUG::2013-11-20 03:43:25,761::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914877::DEBUG::2013-11-20 03:43:25,762::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914877::DEBUG::2013-11-20 03:43:25,762::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ef0c1414-90ad-40b1-bb73-5a0edd14baab`::Granted request
>Thread-15914877::DEBUG::2013-11-20 03:43:25,763::task::495::TaskManager.Task::(_debug) Task 24f2bdad-cd94-42f4-89a8-579533dc38b0: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914877::DEBUG::2013-11-20 03:43:25,764::task::495::TaskManager.Task::(_debug) Task 24f2bdad-cd94-42f4-89a8-579533dc38b0: ref 1 aborting False
>Thread-15914877::WARNING::2013-11-20 03:43:25,772::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914877::DEBUG::2013-11-20 03:43:25,773::task::495::TaskManager.Task::(_debug) Task 24f2bdad-cd94-42f4-89a8-579533dc38b0: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914877::DEBUG::2013-11-20 03:43:25,774::task::495::TaskManager.Task::(_debug) Task 24f2bdad-cd94-42f4-89a8-579533dc38b0: moving from state preparing -> state finished
>Thread-15914877::DEBUG::2013-11-20 03:43:25,775::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914877::DEBUG::2013-11-20 03:43:25,776::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914877::DEBUG::2013-11-20 03:43:25,777::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914877::DEBUG::2013-11-20 03:43:25,777::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914877::DEBUG::2013-11-20 03:43:25,782::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914877::DEBUG::2013-11-20 03:43:25,782::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914877::DEBUG::2013-11-20 03:43:25,783::task::495::TaskManager.Task::(_debug) Task 24f2bdad-cd94-42f4-89a8-579533dc38b0: ref 0 aborting False
>Thread-15914877::INFO::2013-11-20 03:43:25,784::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:43:25,882::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:43:26,450::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0802675 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:43:28,465::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:43:28,997::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0848239 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15914435::INFO::2013-11-20 03:43:29,576::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 03:43:29,577::task::495::TaskManager.Task::(_debug) Task 8af02c75-2636-47d5-9036-e3190b519955: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 03:43:29,580::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bde21245-9408-40e1-a266-077e76d87a02`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 03:43:29,581::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 03:43:29,583::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 03:43:29,584::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bde21245-9408-40e1-a266-077e76d87a02`::Granted request
>Thread-15914435::DEBUG::2013-11-20 03:43:29,586::task::495::TaskManager.Task::(_debug) Task 8af02c75-2636-47d5-9036-e3190b519955: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 03:43:29,587::task::495::TaskManager.Task::(_debug) Task 8af02c75-2636-47d5-9036-e3190b519955: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 03:43:29,587::task::495::TaskManager.Task::(_debug) Task 8af02c75-2636-47d5-9036-e3190b519955: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 03:43:29,588::task::495::TaskManager.Task::(_debug) Task 8af02c75-2636-47d5-9036-e3190b519955: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 03:43:29,589::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 03:43:29,590::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 03:43:29,591::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 03:43:29,592::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 03:43:29,592::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 03:43:29,594::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 03:43:29,594::task::495::TaskManager.Task::(_debug) Task 8af02c75-2636-47d5-9036-e3190b519955: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 03:43:29,595::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 03:43:31,012::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 03:43:31,534::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 03:43:31,542::task::495::TaskManager.Task::(_debug) Task ac5334cc-d34f-4ba4-a402-8c59ded6faad: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 03:43:31,562::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b42f6bbe-e927-4fcf-af29-38033f2ff658`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 03:43:31,563::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 03:43:31,565::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 03:43:31,566::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b42f6bbe-e927-4fcf-af29-38033f2ff658`::Granted request
>Thread-15914453::DEBUG::2013-11-20 03:43:31,569::task::495::TaskManager.Task::(_debug) Task ac5334cc-d34f-4ba4-a402-8c59ded6faad: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 03:43:31,570::task::495::TaskManager.Task::(_debug) Task ac5334cc-d34f-4ba4-a402-8c59ded6faad: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 03:43:31,571::task::495::TaskManager.Task::(_debug) Task ac5334cc-d34f-4ba4-a402-8c59ded6faad: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 03:43:31,572::task::495::TaskManager.Task::(_debug) Task ac5334cc-d34f-4ba4-a402-8c59ded6faad: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 03:43:31,573::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 03:43:31,575::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 03:43:31,576::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 03:43:31,577::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 03:43:31,578::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 03:43:31,579::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 03:43:31,581::task::495::TaskManager.Task::(_debug) Task ac5334cc-d34f-4ba4-a402-8c59ded6faad: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 03:43:31,583::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 03:43:31,683::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.13965 s, 7.3 MB/s\n'; <rc> = 0
>Thread-15914880::INFO::2013-11-20 03:43:33,036::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914880::DEBUG::2013-11-20 03:43:33,037::task::495::TaskManager.Task::(_debug) Task badd42b3-0a76-4174-afc7-9188f3e357ac: moving from state init -> state preparing
>Thread-15914880::DEBUG::2013-11-20 03:43:33,038::task::495::TaskManager.Task::(_debug) Task badd42b3-0a76-4174-afc7-9188f3e357ac: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00214195251465', 'lastCheck': 1384919002.9472449, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0146331787109', 'lastCheck': 1384919007.1259949, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00171709060669', 'lastCheck': 1384919002.9503579, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000575065612793', 'lastCheck': 1384919006.526047, 'valid': True, 'code': 0}}
>Thread-15914880::DEBUG::2013-11-20 03:43:33,039::task::495::TaskManager.Task::(_debug) Task badd42b3-0a76-4174-afc7-9188f3e357ac: moving from state preparing -> state finished
>Thread-15914880::DEBUG::2013-11-20 03:43:33,040::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914880::DEBUG::2013-11-20 03:43:33,040::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914880::DEBUG::2013-11-20 03:43:33,041::task::495::TaskManager.Task::(_debug) Task badd42b3-0a76-4174-afc7-9188f3e357ac: ref 0 aborting False
>Thread-15914880::INFO::2013-11-20 03:43:33,049::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00214195251465', 'lastCheck': 1384919002.9472449, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0146331787109', 'lastCheck': 1384919007.1259949, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00171709060669', 'lastCheck': 1384919002.9503579, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000575065612793', 'lastCheck': 1384919006.526047, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:43:33,698::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:43:34,243::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0814546 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:43:36,260::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914882::DEBUG::2013-11-20 03:43:36,556::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914882::INFO::2013-11-20 03:43:36,579::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914882::DEBUG::2013-11-20 03:43:36,583::task::495::TaskManager.Task::(_debug) Task f709124a-da12-4ff0-9951-a98e3cd9cc2e: moving from state init -> state preparing
>Thread-15914882::DEBUG::2013-11-20 03:43:36,587::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914882::DEBUG::2013-11-20 03:43:36,588::task::495::TaskManager.Task::(_debug) Task f709124a-da12-4ff0-9951-a98e3cd9cc2e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914882::DEBUG::2013-11-20 03:43:36,592::task::495::TaskManager.Task::(_debug) Task f709124a-da12-4ff0-9951-a98e3cd9cc2e: moving from state preparing -> state finished
>Thread-15914882::DEBUG::2013-11-20 03:43:36,593::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914882::DEBUG::2013-11-20 03:43:36,595::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914882::DEBUG::2013-11-20 03:43:36,595::task::495::TaskManager.Task::(_debug) Task f709124a-da12-4ff0-9951-a98e3cd9cc2e: ref 0 aborting False
>Thread-15914882::INFO::2013-11-20 03:43:36,596::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914884::DEBUG::2013-11-20 03:43:36,608::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914884::INFO::2013-11-20 03:43:36,609::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914884::DEBUG::2013-11-20 03:43:36,610::task::495::TaskManager.Task::(_debug) Task ba39664d-8f25-4a52-8839-c5d1f6b00b83: moving from state init -> state preparing
>Thread-15914884::DEBUG::2013-11-20 03:43:36,612::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f6cbfd03-9094-4dde-a05b-e01122862d19`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914884::DEBUG::2013-11-20 03:43:36,614::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914884::DEBUG::2013-11-20 03:43:36,615::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914884::DEBUG::2013-11-20 03:43:36,616::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f6cbfd03-9094-4dde-a05b-e01122862d19`::Granted request
>Thread-15914884::DEBUG::2013-11-20 03:43:36,617::task::495::TaskManager.Task::(_debug) Task ba39664d-8f25-4a52-8839-c5d1f6b00b83: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914884::DEBUG::2013-11-20 03:43:36,618::task::495::TaskManager.Task::(_debug) Task ba39664d-8f25-4a52-8839-c5d1f6b00b83: ref 1 aborting False
>Thread-15914884::WARNING::2013-11-20 03:43:36,621::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914884::DEBUG::2013-11-20 03:43:36,622::task::495::TaskManager.Task::(_debug) Task ba39664d-8f25-4a52-8839-c5d1f6b00b83: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914884::DEBUG::2013-11-20 03:43:36,626::task::495::TaskManager.Task::(_debug) Task ba39664d-8f25-4a52-8839-c5d1f6b00b83: moving from state preparing -> state finished
>Thread-15914884::DEBUG::2013-11-20 03:43:36,627::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914884::DEBUG::2013-11-20 03:43:36,628::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914884::DEBUG::2013-11-20 03:43:36,628::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914884::DEBUG::2013-11-20 03:43:36,629::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914884::DEBUG::2013-11-20 03:43:36,630::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914884::DEBUG::2013-11-20 03:43:36,630::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914884::DEBUG::2013-11-20 03:43:36,631::task::495::TaskManager.Task::(_debug) Task ba39664d-8f25-4a52-8839-c5d1f6b00b83: ref 0 aborting False
>Thread-15914884::INFO::2013-11-20 03:43:36,632::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:43:36,853::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0828563 s, 12.4 MB/s\n'; <rc> = 0
>Thread-15914519::INFO::2013-11-20 03:43:38,665::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 03:43:38,673::task::495::TaskManager.Task::(_debug) Task cb6caaa4-3ad9-4bb0-945d-a52bd73c056b: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 03:43:38,681::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`77b3035d-c6d4-465b-9e75-62f5e0633f54`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 03:43:38,683::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 03:43:38,684::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 03:43:38,685::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`77b3035d-c6d4-465b-9e75-62f5e0633f54`::Granted request
>Thread-15914519::DEBUG::2013-11-20 03:43:38,686::task::495::TaskManager.Task::(_debug) Task cb6caaa4-3ad9-4bb0-945d-a52bd73c056b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 03:43:38,687::task::495::TaskManager.Task::(_debug) Task cb6caaa4-3ad9-4bb0-945d-a52bd73c056b: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 03:43:38,688::task::495::TaskManager.Task::(_debug) Task cb6caaa4-3ad9-4bb0-945d-a52bd73c056b: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 03:43:38,688::task::495::TaskManager.Task::(_debug) Task cb6caaa4-3ad9-4bb0-945d-a52bd73c056b: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 03:43:38,689::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 03:43:38,690::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 03:43:38,691::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 03:43:38,692::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 03:43:38,692::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 03:43:38,693::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 03:43:38,694::task::495::TaskManager.Task::(_debug) Task cb6caaa4-3ad9-4bb0-945d-a52bd73c056b: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 03:43:38,694::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 03:43:38,867::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:43:39,413::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.074605 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:43:41,428::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 03:43:41,638::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:43:41,643::task::495::TaskManager.Task::(_debug) Task ef702d3b-0506-4402-b3c2-3db26c829686: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:43:41,650::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`09a8143a-99b0-46ae-88ce-4c1e477125a1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:43:41,655::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:43:41,670::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:43:41,671::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`09a8143a-99b0-46ae-88ce-4c1e477125a1`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:43:41,673::task::495::TaskManager.Task::(_debug) Task ef702d3b-0506-4402-b3c2-3db26c829686: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:43:41,675::task::495::TaskManager.Task::(_debug) Task ef702d3b-0506-4402-b3c2-3db26c829686: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:43:41,677::task::495::TaskManager.Task::(_debug) Task ef702d3b-0506-4402-b3c2-3db26c829686: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:43:41,678::task::495::TaskManager.Task::(_debug) Task ef702d3b-0506-4402-b3c2-3db26c829686: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:43:41,678::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:43:41,680::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:43:41,681::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:43:41,682::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:43:41,682::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913518::INFO::2013-11-20 03:43:41,684::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913546::DEBUG::2013-11-20 03:43:41,685::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:43:41,686::task::495::TaskManager.Task::(_debug) Task 9b39db18-192f-4943-b89e-1e18ead775f2: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:43:41,687::task::495::TaskManager.Task::(_debug) Task ef702d3b-0506-4402-b3c2-3db26c829686: ref 0 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:43:41,688::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d00e0a49-e680-4ca2-a406-9860685087a4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::INFO::2013-11-20 03:43:41,689::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913518::DEBUG::2013-11-20 03:43:41,690::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913546::INFO::2013-11-20 03:43:41,691::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:43:41,692::task::495::TaskManager.Task::(_debug) Task 4bae2377-0e84-4f6d-b804-2eb763903e85: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:43:41,693::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:43:41,694::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`1828f3f5-1a0a-4df2-b69b-e4c63810f834`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:43:41,695::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d00e0a49-e680-4ca2-a406-9860685087a4`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:43:41,695::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:43:41,697::task::495::TaskManager.Task::(_debug) Task 9b39db18-192f-4943-b89e-1e18ead775f2: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913546::DEBUG::2013-11-20 03:43:41,698::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:43:41,698::task::495::TaskManager.Task::(_debug) Task 9b39db18-192f-4943-b89e-1e18ead775f2: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:43:41,699::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`1828f3f5-1a0a-4df2-b69b-e4c63810f834`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:43:41,700::task::495::TaskManager.Task::(_debug) Task 9b39db18-192f-4943-b89e-1e18ead775f2: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:43:41,701::task::495::TaskManager.Task::(_debug) Task 9b39db18-192f-4943-b89e-1e18ead775f2: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:43:41,702::task::495::TaskManager.Task::(_debug) Task 4bae2377-0e84-4f6d-b804-2eb763903e85: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913518::DEBUG::2013-11-20 03:43:41,703::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:43:41,703::task::495::TaskManager.Task::(_debug) Task 4bae2377-0e84-4f6d-b804-2eb763903e85: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:43:41,704::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:43:41,706::task::495::TaskManager.Task::(_debug) Task 4bae2377-0e84-4f6d-b804-2eb763903e85: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913518::DEBUG::2013-11-20 03:43:41,706::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913546::DEBUG::2013-11-20 03:43:41,707::task::495::TaskManager.Task::(_debug) Task 4bae2377-0e84-4f6d-b804-2eb763903e85: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:43:41,707::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:43:41,708::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:43:41,709::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:43:41,710::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:43:41,710::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:43:41,716::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913518::DEBUG::2013-11-20 03:43:41,717::task::495::TaskManager.Task::(_debug) Task 9b39db18-192f-4943-b89e-1e18ead775f2: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:43:41,717::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913518::INFO::2013-11-20 03:43:41,718::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913546::DEBUG::2013-11-20 03:43:41,719::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913518::INFO::2013-11-20 03:43:41,720::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913546::DEBUG::2013-11-20 03:43:41,720::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:43:41,721::task::495::TaskManager.Task::(_debug) Task e9838804-d699-47b3-b9c6-29a1de0a7960: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:43:41,722::task::495::TaskManager.Task::(_debug) Task 4bae2377-0e84-4f6d-b804-2eb763903e85: ref 0 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:43:41,724::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0c5560ac-e9e1-4ab2-8411-4a6908eae302`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::INFO::2013-11-20 03:43:41,724::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913518::DEBUG::2013-11-20 03:43:41,725::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913546::INFO::2013-11-20 03:43:41,726::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913518::DEBUG::2013-11-20 03:43:41,727::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:43:41,727::task::495::TaskManager.Task::(_debug) Task 460a8d7c-3eb4-4dc9-9238-11bdde5925a7: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:43:41,728::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0c5560ac-e9e1-4ab2-8411-4a6908eae302`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:43:41,730::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c99c9e76-ae34-40a3-8064-8b7cbe75090f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:43:41,731::task::495::TaskManager.Task::(_debug) Task e9838804-d699-47b3-b9c6-29a1de0a7960: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913546::DEBUG::2013-11-20 03:43:41,732::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:43:41,732::task::495::TaskManager.Task::(_debug) Task e9838804-d699-47b3-b9c6-29a1de0a7960: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:43:41,734::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:43:41,734::task::495::TaskManager.Task::(_debug) Task e9838804-d699-47b3-b9c6-29a1de0a7960: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913546::DEBUG::2013-11-20 03:43:41,735::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c99c9e76-ae34-40a3-8064-8b7cbe75090f`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:43:41,736::task::495::TaskManager.Task::(_debug) Task e9838804-d699-47b3-b9c6-29a1de0a7960: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:43:41,737::task::495::TaskManager.Task::(_debug) Task 460a8d7c-3eb4-4dc9-9238-11bdde5925a7: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913518::DEBUG::2013-11-20 03:43:41,737::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:43:41,738::task::495::TaskManager.Task::(_debug) Task 460a8d7c-3eb4-4dc9-9238-11bdde5925a7: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:43:41,739::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:43:41,739::task::495::TaskManager.Task::(_debug) Task 460a8d7c-3eb4-4dc9-9238-11bdde5925a7: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913518::DEBUG::2013-11-20 03:43:41,741::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913546::DEBUG::2013-11-20 03:43:41,741::task::495::TaskManager.Task::(_debug) Task 460a8d7c-3eb4-4dc9-9238-11bdde5925a7: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:43:41,742::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:43:41,743::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:43:41,743::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:43:41,744::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:43:41,745::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:43:41,746::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913518::DEBUG::2013-11-20 03:43:41,747::task::495::TaskManager.Task::(_debug) Task e9838804-d699-47b3-b9c6-29a1de0a7960: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:43:41,747::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913518::INFO::2013-11-20 03:43:41,748::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913546::DEBUG::2013-11-20 03:43:41,749::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:43:41,750::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:43:41,751::task::495::TaskManager.Task::(_debug) Task 460a8d7c-3eb4-4dc9-9238-11bdde5925a7: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:43:41,751::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:43:41,753::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:43:41,754::task::495::TaskManager.Task::(_debug) Task be2e582f-a3a0-4d85-bd9c-9b3aba24903b: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:43:41,755::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`bb18345b-3f29-4ff6-b024-549bcf5cb77f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:43:41,755::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:43:41,756::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:43:41,757::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`bb18345b-3f29-4ff6-b024-549bcf5cb77f`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:43:41,757::task::495::TaskManager.Task::(_debug) Task be2e582f-a3a0-4d85-bd9c-9b3aba24903b: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:43:41,758::task::495::TaskManager.Task::(_debug) Task be2e582f-a3a0-4d85-bd9c-9b3aba24903b: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:43:41,759::task::495::TaskManager.Task::(_debug) Task be2e582f-a3a0-4d85-bd9c-9b3aba24903b: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:43:41,759::task::495::TaskManager.Task::(_debug) Task be2e582f-a3a0-4d85-bd9c-9b3aba24903b: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:43:41,760::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:43:41,760::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:43:41,761::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:43:41,761::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:43:41,762::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:43:41,762::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:43:41,763::task::495::TaskManager.Task::(_debug) Task be2e582f-a3a0-4d85-bd9c-9b3aba24903b: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:43:41,763::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:43:41,957::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0530192 s, 19.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:43:43,969::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:43:44,593::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0870696 s, 11.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:43:46,607::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:43:47,159::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0884575 s, 11.6 MB/s\n'; <rc> = 0
>Thread-15914888::DEBUG::2013-11-20 03:43:47,241::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914888::INFO::2013-11-20 03:43:47,243::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914888::DEBUG::2013-11-20 03:43:47,243::task::495::TaskManager.Task::(_debug) Task 0d08ede4-9605-4ec8-806e-49efd8340bec: moving from state init -> state preparing
>Thread-15914888::DEBUG::2013-11-20 03:43:47,245::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914888::DEBUG::2013-11-20 03:43:47,245::task::495::TaskManager.Task::(_debug) Task 0d08ede4-9605-4ec8-806e-49efd8340bec: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914888::DEBUG::2013-11-20 03:43:47,246::task::495::TaskManager.Task::(_debug) Task 0d08ede4-9605-4ec8-806e-49efd8340bec: moving from state preparing -> state finished
>Thread-15914888::DEBUG::2013-11-20 03:43:47,247::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914888::DEBUG::2013-11-20 03:43:47,248::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914888::DEBUG::2013-11-20 03:43:47,248::task::495::TaskManager.Task::(_debug) Task 0d08ede4-9605-4ec8-806e-49efd8340bec: ref 0 aborting False
>Thread-15914888::INFO::2013-11-20 03:43:47,249::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914889::DEBUG::2013-11-20 03:43:47,255::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914889::INFO::2013-11-20 03:43:47,256::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914889::DEBUG::2013-11-20 03:43:47,257::task::495::TaskManager.Task::(_debug) Task e0531df6-d0d3-4b7f-8b5e-45791e2d615b: moving from state init -> state preparing
>Thread-15914889::DEBUG::2013-11-20 03:43:47,258::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`94bd3fa3-dd16-445f-9dad-a32c58febe21`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914889::DEBUG::2013-11-20 03:43:47,259::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914889::DEBUG::2013-11-20 03:43:47,260::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914889::DEBUG::2013-11-20 03:43:47,261::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`94bd3fa3-dd16-445f-9dad-a32c58febe21`::Granted request
>Thread-15914889::DEBUG::2013-11-20 03:43:47,262::task::495::TaskManager.Task::(_debug) Task e0531df6-d0d3-4b7f-8b5e-45791e2d615b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914889::DEBUG::2013-11-20 03:43:47,263::task::495::TaskManager.Task::(_debug) Task e0531df6-d0d3-4b7f-8b5e-45791e2d615b: ref 1 aborting False
>Thread-15914889::WARNING::2013-11-20 03:43:47,266::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914889::DEBUG::2013-11-20 03:43:47,267::task::495::TaskManager.Task::(_debug) Task e0531df6-d0d3-4b7f-8b5e-45791e2d615b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914889::DEBUG::2013-11-20 03:43:47,268::task::495::TaskManager.Task::(_debug) Task e0531df6-d0d3-4b7f-8b5e-45791e2d615b: moving from state preparing -> state finished
>Thread-15914889::DEBUG::2013-11-20 03:43:47,268::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914889::DEBUG::2013-11-20 03:43:47,269::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914889::DEBUG::2013-11-20 03:43:47,270::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914889::DEBUG::2013-11-20 03:43:47,271::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914889::DEBUG::2013-11-20 03:43:47,271::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914889::DEBUG::2013-11-20 03:43:47,272::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914889::DEBUG::2013-11-20 03:43:47,273::task::495::TaskManager.Task::(_debug) Task e0531df6-d0d3-4b7f-8b5e-45791e2d615b: ref 0 aborting False
>Thread-15914889::INFO::2013-11-20 03:43:47,274::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914890::INFO::2013-11-20 03:43:48,676::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914890::DEBUG::2013-11-20 03:43:48,677::task::495::TaskManager.Task::(_debug) Task 946b9d1f-024f-4ba8-9923-17c4e0f5228e: moving from state init -> state preparing
>Thread-15914890::DEBUG::2013-11-20 03:43:48,678::task::495::TaskManager.Task::(_debug) Task 946b9d1f-024f-4ba8-9923-17c4e0f5228e: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000349044799805', 'lastCheck': 1384919023.6408851, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.13670706749', 'lastCheck': 1384919027.964659, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00162506103516', 'lastCheck': 1384919023.641083, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000593900680542', 'lastCheck': 1384919026.8310499, 'valid': True, 'code': 0}}
>Thread-15914890::DEBUG::2013-11-20 03:43:48,692::task::495::TaskManager.Task::(_debug) Task 946b9d1f-024f-4ba8-9923-17c4e0f5228e: moving from state preparing -> state finished
>Thread-15914890::DEBUG::2013-11-20 03:43:48,693::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914890::DEBUG::2013-11-20 03:43:48,694::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914890::DEBUG::2013-11-20 03:43:48,695::task::495::TaskManager.Task::(_debug) Task 946b9d1f-024f-4ba8-9923-17c4e0f5228e: ref 0 aborting False
>Thread-15914890::INFO::2013-11-20 03:43:48,696::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000349044799805', 'lastCheck': 1384919023.6408851, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.13670706749', 'lastCheck': 1384919027.964659, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00162506103516', 'lastCheck': 1384919023.641083, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000593900680542', 'lastCheck': 1384919026.8310499, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:43:49,178::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:43:49,697::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0830651 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:43:51,713::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:43:52,193::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0519049 s, 19.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:43:54,209::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:43:54,770::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0813336 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:43:56,786::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:43:57,328::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0739712 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15914894::DEBUG::2013-11-20 03:43:58,228::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914894::INFO::2013-11-20 03:43:58,229::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914894::DEBUG::2013-11-20 03:43:58,230::task::495::TaskManager.Task::(_debug) Task 2c5248be-5973-40d1-80a1-53bc757d8cbf: moving from state init -> state preparing
>Thread-15914894::DEBUG::2013-11-20 03:43:58,231::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914894::DEBUG::2013-11-20 03:43:58,232::task::495::TaskManager.Task::(_debug) Task 2c5248be-5973-40d1-80a1-53bc757d8cbf: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914894::DEBUG::2013-11-20 03:43:58,232::task::495::TaskManager.Task::(_debug) Task 2c5248be-5973-40d1-80a1-53bc757d8cbf: moving from state preparing -> state finished
>Thread-15914894::DEBUG::2013-11-20 03:43:58,233::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914894::DEBUG::2013-11-20 03:43:58,234::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914894::DEBUG::2013-11-20 03:43:58,234::task::495::TaskManager.Task::(_debug) Task 2c5248be-5973-40d1-80a1-53bc757d8cbf: ref 0 aborting False
>Thread-15914894::INFO::2013-11-20 03:43:58,235::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914895::DEBUG::2013-11-20 03:43:58,249::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914895::INFO::2013-11-20 03:43:58,250::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914895::DEBUG::2013-11-20 03:43:58,251::task::495::TaskManager.Task::(_debug) Task 5f6619a5-4de6-4448-8c37-feba32ea9ba2: moving from state init -> state preparing
>Thread-15914895::DEBUG::2013-11-20 03:43:58,252::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5feaa647-c0d9-49bf-87cd-b7f63fe7074e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914895::DEBUG::2013-11-20 03:43:58,253::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914895::DEBUG::2013-11-20 03:43:58,254::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914895::DEBUG::2013-11-20 03:43:58,254::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5feaa647-c0d9-49bf-87cd-b7f63fe7074e`::Granted request
>Thread-15914895::DEBUG::2013-11-20 03:43:58,256::task::495::TaskManager.Task::(_debug) Task 5f6619a5-4de6-4448-8c37-feba32ea9ba2: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914895::DEBUG::2013-11-20 03:43:58,256::task::495::TaskManager.Task::(_debug) Task 5f6619a5-4de6-4448-8c37-feba32ea9ba2: ref 1 aborting False
>Thread-15914895::WARNING::2013-11-20 03:43:58,259::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914895::DEBUG::2013-11-20 03:43:58,260::task::495::TaskManager.Task::(_debug) Task 5f6619a5-4de6-4448-8c37-feba32ea9ba2: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914895::DEBUG::2013-11-20 03:43:58,261::task::495::TaskManager.Task::(_debug) Task 5f6619a5-4de6-4448-8c37-feba32ea9ba2: moving from state preparing -> state finished
>Thread-15914895::DEBUG::2013-11-20 03:43:58,262::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914895::DEBUG::2013-11-20 03:43:58,262::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914895::DEBUG::2013-11-20 03:43:58,263::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914895::DEBUG::2013-11-20 03:43:58,264::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914895::DEBUG::2013-11-20 03:43:58,264::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914895::DEBUG::2013-11-20 03:43:58,265::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914895::DEBUG::2013-11-20 03:43:58,266::task::495::TaskManager.Task::(_debug) Task 5f6619a5-4de6-4448-8c37-feba32ea9ba2: ref 0 aborting False
>Thread-15914895::INFO::2013-11-20 03:43:58,267::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913639::INFO::2013-11-20 03:43:58,997::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:43:58,999::task::495::TaskManager.Task::(_debug) Task c3498451-7fd1-4e41-82df-e7bbb77bb993: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:43:59,001::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1287692c-066d-4712-9399-c3b3111305df`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:43:59,003::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:43:59,003::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:43:59,004::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1287692c-066d-4712-9399-c3b3111305df`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:43:59,005::task::495::TaskManager.Task::(_debug) Task c3498451-7fd1-4e41-82df-e7bbb77bb993: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:43:59,005::task::495::TaskManager.Task::(_debug) Task c3498451-7fd1-4e41-82df-e7bbb77bb993: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:43:59,006::task::495::TaskManager.Task::(_debug) Task c3498451-7fd1-4e41-82df-e7bbb77bb993: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:43:59,007::task::495::TaskManager.Task::(_debug) Task c3498451-7fd1-4e41-82df-e7bbb77bb993: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:43:59,007::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:43:59,008::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:43:59,009::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:43:59,009::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:43:59,010::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:43:59,011::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:43:59,011::task::495::TaskManager.Task::(_debug) Task c3498451-7fd1-4e41-82df-e7bbb77bb993: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:43:59,012::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:43:59,342::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:43:59,868::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0739803 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:44:01,883::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:44:02,324::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0490259 s, 20.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:44:04,335::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:44:04,868::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0919322 s, 11.1 MB/s\n'; <rc> = 0
>Thread-15914898::INFO::2013-11-20 03:44:05,111::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914898::DEBUG::2013-11-20 03:44:05,112::task::495::TaskManager.Task::(_debug) Task 07bd4069-8565-4e26-8c6f-0e616173baad: moving from state init -> state preparing
>Thread-15914898::DEBUG::2013-11-20 03:44:05,114::task::495::TaskManager.Task::(_debug) Task 07bd4069-8565-4e26-8c6f-0e616173baad: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000292062759399', 'lastCheck': 1384919044.086386, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00143599510193', 'lastCheck': 1384919037.9989049, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0012149810791', 'lastCheck': 1384919044.0877011, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00270104408264', 'lastCheck': 1384919036.9978149, 'valid': True, 'code': 0}}
>Thread-15914898::DEBUG::2013-11-20 03:44:05,129::task::495::TaskManager.Task::(_debug) Task 07bd4069-8565-4e26-8c6f-0e616173baad: moving from state preparing -> state finished
>Thread-15914898::DEBUG::2013-11-20 03:44:05,130::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914898::DEBUG::2013-11-20 03:44:05,131::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914898::DEBUG::2013-11-20 03:44:05,132::task::495::TaskManager.Task::(_debug) Task 07bd4069-8565-4e26-8c6f-0e616173baad: ref 0 aborting False
>Thread-15914898::INFO::2013-11-20 03:44:05,132::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000292062759399', 'lastCheck': 1384919044.086386, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00143599510193', 'lastCheck': 1384919037.9989049, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0012149810791', 'lastCheck': 1384919044.0877011, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00270104408264', 'lastCheck': 1384919036.9978149, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:44:06,883::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:44:07,456::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0844608 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15914900::DEBUG::2013-11-20 03:44:08,758::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914900::INFO::2013-11-20 03:44:08,760::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914900::DEBUG::2013-11-20 03:44:08,761::task::495::TaskManager.Task::(_debug) Task 289a915e-5568-46c0-98b2-4edabc060e54: moving from state init -> state preparing
>Thread-15914900::DEBUG::2013-11-20 03:44:08,762::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914900::DEBUG::2013-11-20 03:44:08,762::task::495::TaskManager.Task::(_debug) Task 289a915e-5568-46c0-98b2-4edabc060e54: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914900::DEBUG::2013-11-20 03:44:08,763::task::495::TaskManager.Task::(_debug) Task 289a915e-5568-46c0-98b2-4edabc060e54: moving from state preparing -> state finished
>Thread-15914900::DEBUG::2013-11-20 03:44:08,764::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914900::DEBUG::2013-11-20 03:44:08,764::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914900::DEBUG::2013-11-20 03:44:08,765::task::495::TaskManager.Task::(_debug) Task 289a915e-5568-46c0-98b2-4edabc060e54: ref 0 aborting False
>Thread-15914900::INFO::2013-11-20 03:44:08,766::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914901::DEBUG::2013-11-20 03:44:08,796::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914901::INFO::2013-11-20 03:44:08,797::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914901::DEBUG::2013-11-20 03:44:08,798::task::495::TaskManager.Task::(_debug) Task 27dcee9e-247d-4555-ad37-cb8993e0c3a0: moving from state init -> state preparing
>Thread-15914901::DEBUG::2013-11-20 03:44:08,800::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`74e794a4-d003-481b-b285-23167e615bb0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914901::DEBUG::2013-11-20 03:44:08,801::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914901::DEBUG::2013-11-20 03:44:08,802::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914901::DEBUG::2013-11-20 03:44:08,802::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`74e794a4-d003-481b-b285-23167e615bb0`::Granted request
>Thread-15914901::DEBUG::2013-11-20 03:44:08,804::task::495::TaskManager.Task::(_debug) Task 27dcee9e-247d-4555-ad37-cb8993e0c3a0: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914901::DEBUG::2013-11-20 03:44:08,804::task::495::TaskManager.Task::(_debug) Task 27dcee9e-247d-4555-ad37-cb8993e0c3a0: ref 1 aborting False
>Thread-15914901::WARNING::2013-11-20 03:44:08,806::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914901::DEBUG::2013-11-20 03:44:08,808::task::495::TaskManager.Task::(_debug) Task 27dcee9e-247d-4555-ad37-cb8993e0c3a0: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914901::DEBUG::2013-11-20 03:44:08,809::task::495::TaskManager.Task::(_debug) Task 27dcee9e-247d-4555-ad37-cb8993e0c3a0: moving from state preparing -> state finished
>Thread-15914901::DEBUG::2013-11-20 03:44:08,809::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914901::DEBUG::2013-11-20 03:44:08,810::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914901::DEBUG::2013-11-20 03:44:08,811::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914901::DEBUG::2013-11-20 03:44:08,812::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914901::DEBUG::2013-11-20 03:44:08,812::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914901::DEBUG::2013-11-20 03:44:08,813::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914901::DEBUG::2013-11-20 03:44:08,814::task::495::TaskManager.Task::(_debug) Task 27dcee9e-247d-4555-ad37-cb8993e0c3a0: ref 0 aborting False
>Thread-15914901::INFO::2013-11-20 03:44:08,815::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:44:09,467::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:44:09,957::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0482044 s, 21.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:44:11,967::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 03:44:12,255::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:44:12,263::task::495::TaskManager.Task::(_debug) Task 907c8e97-7237-41e7-8955-06ea9a5ef485: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:44:12,267::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`eea6b25b-c3ae-4366-a458-fc3b0c09ed74`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:44:12,268::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:44:12,270::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:44:12,271::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`eea6b25b-c3ae-4366-a458-fc3b0c09ed74`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:44:12,273::task::495::TaskManager.Task::(_debug) Task 907c8e97-7237-41e7-8955-06ea9a5ef485: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:44:12,273::task::495::TaskManager.Task::(_debug) Task 907c8e97-7237-41e7-8955-06ea9a5ef485: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:44:12,274::task::495::TaskManager.Task::(_debug) Task 907c8e97-7237-41e7-8955-06ea9a5ef485: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:44:12,276::task::495::TaskManager.Task::(_debug) Task 907c8e97-7237-41e7-8955-06ea9a5ef485: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:44:12,276::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:44:12,278::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:44:12,279::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:44:12,280::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:44:12,280::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:44:12,281::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:44:12,282::task::495::TaskManager.Task::(_debug) Task 907c8e97-7237-41e7-8955-06ea9a5ef485: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:44:12,284::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:44:12,533::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0859743 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:44:14,548::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:44:15,085::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0884623 s, 11.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:44:17,095::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:44:17,637::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0769573 s, 13.3 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 03:44:19,332::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:44:19,333::task::495::TaskManager.Task::(_debug) Task e7f3ad00-8546-496c-adc8-e71d86d0b4be: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:44:19,335::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b6b9e09a-eeb9-4888-b7c6-8f17cce1c6f5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:44:19,335::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:44:19,336::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:44:19,337::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b6b9e09a-eeb9-4888-b7c6-8f17cce1c6f5`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:44:19,338::task::495::TaskManager.Task::(_debug) Task e7f3ad00-8546-496c-adc8-e71d86d0b4be: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:44:19,339::task::495::TaskManager.Task::(_debug) Task e7f3ad00-8546-496c-adc8-e71d86d0b4be: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:44:19,339::task::495::TaskManager.Task::(_debug) Task e7f3ad00-8546-496c-adc8-e71d86d0b4be: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:44:19,340::task::495::TaskManager.Task::(_debug) Task e7f3ad00-8546-496c-adc8-e71d86d0b4be: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:44:19,340::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:44:19,341::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:44:19,341::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:44:19,342::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:44:19,343::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:44:19,343::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:44:19,344::task::495::TaskManager.Task::(_debug) Task e7f3ad00-8546-496c-adc8-e71d86d0b4be: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:44:19,344::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:44:19,652::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914906::DEBUG::2013-11-20 03:44:19,653::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914906::INFO::2013-11-20 03:44:19,655::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914906::DEBUG::2013-11-20 03:44:19,873::task::495::TaskManager.Task::(_debug) Task 5d688867-75cb-4201-b74f-6b2e99cbdb3b: moving from state init -> state preparing
>Thread-15914906::DEBUG::2013-11-20 03:44:19,874::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914906::DEBUG::2013-11-20 03:44:19,875::task::495::TaskManager.Task::(_debug) Task 5d688867-75cb-4201-b74f-6b2e99cbdb3b: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914906::DEBUG::2013-11-20 03:44:19,875::task::495::TaskManager.Task::(_debug) Task 5d688867-75cb-4201-b74f-6b2e99cbdb3b: moving from state preparing -> state finished
>Thread-15914906::DEBUG::2013-11-20 03:44:19,876::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914906::DEBUG::2013-11-20 03:44:19,876::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914906::DEBUG::2013-11-20 03:44:19,876::task::495::TaskManager.Task::(_debug) Task 5d688867-75cb-4201-b74f-6b2e99cbdb3b: ref 0 aborting False
>Thread-15914906::INFO::2013-11-20 03:44:19,877::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914907::DEBUG::2013-11-20 03:44:19,889::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914907::INFO::2013-11-20 03:44:19,891::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914907::DEBUG::2013-11-20 03:44:19,892::task::495::TaskManager.Task::(_debug) Task d60e9e83-0588-43e5-9f87-c0f381615c27: moving from state init -> state preparing
>Thread-15914907::DEBUG::2013-11-20 03:44:19,894::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`cc139e01-8ccc-470f-aeb3-e55d1dc36979`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914907::DEBUG::2013-11-20 03:44:19,895::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914907::DEBUG::2013-11-20 03:44:19,896::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914907::DEBUG::2013-11-20 03:44:19,896::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`cc139e01-8ccc-470f-aeb3-e55d1dc36979`::Granted request
>Thread-15914907::DEBUG::2013-11-20 03:44:19,898::task::495::TaskManager.Task::(_debug) Task d60e9e83-0588-43e5-9f87-c0f381615c27: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914907::DEBUG::2013-11-20 03:44:19,898::task::495::TaskManager.Task::(_debug) Task d60e9e83-0588-43e5-9f87-c0f381615c27: ref 1 aborting False
>Thread-15914907::WARNING::2013-11-20 03:44:19,901::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914907::DEBUG::2013-11-20 03:44:19,902::task::495::TaskManager.Task::(_debug) Task d60e9e83-0588-43e5-9f87-c0f381615c27: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914907::DEBUG::2013-11-20 03:44:19,908::task::495::TaskManager.Task::(_debug) Task d60e9e83-0588-43e5-9f87-c0f381615c27: moving from state preparing -> state finished
>Thread-15914907::DEBUG::2013-11-20 03:44:19,909::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914907::DEBUG::2013-11-20 03:44:19,909::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914907::DEBUG::2013-11-20 03:44:19,911::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914907::DEBUG::2013-11-20 03:44:19,911::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914907::DEBUG::2013-11-20 03:44:19,912::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914907::DEBUG::2013-11-20 03:44:19,913::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914907::DEBUG::2013-11-20 03:44:19,913::task::495::TaskManager.Task::(_debug) Task d60e9e83-0588-43e5-9f87-c0f381615c27: ref 0 aborting False
>Thread-15914907::INFO::2013-11-20 03:44:19,914::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:44:20,227::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0826567 s, 12.4 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 03:44:20,306::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:44:20,307::task::495::TaskManager.Task::(_debug) Task fc47380e-f8c7-408e-b96c-d9c01aa2ad42: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:44:20,310::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b34d5ea3-46b5-4ca0-b001-a3a1d5e62d90`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:44:20,311::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:44:20,313::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:44:20,314::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b34d5ea3-46b5-4ca0-b001-a3a1d5e62d90`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:44:20,316::task::495::TaskManager.Task::(_debug) Task fc47380e-f8c7-408e-b96c-d9c01aa2ad42: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:44:20,317::task::495::TaskManager.Task::(_debug) Task fc47380e-f8c7-408e-b96c-d9c01aa2ad42: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:44:20,318::task::495::TaskManager.Task::(_debug) Task fc47380e-f8c7-408e-b96c-d9c01aa2ad42: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:44:20,319::task::495::TaskManager.Task::(_debug) Task fc47380e-f8c7-408e-b96c-d9c01aa2ad42: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:44:20,320::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:44:20,321::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:44:20,322::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:44:20,323::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:44:20,324::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:44:20,325::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:44:20,326::task::495::TaskManager.Task::(_debug) Task fc47380e-f8c7-408e-b96c-d9c01aa2ad42: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:44:20,327::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15914454::DEBUG::2013-11-20 03:44:21,004::guestIF::97::vm.Vm::(_handleMessage) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::u'BRD-KLEE\\administrator'
>Thread-15914908::INFO::2013-11-20 03:44:21,890::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914908::DEBUG::2013-11-20 03:44:21,891::task::495::TaskManager.Task::(_debug) Task 6b702ffc-4364-4d17-b04d-9a5105f72e5b: moving from state init -> state preparing
>Thread-15914908::DEBUG::2013-11-20 03:44:21,892::task::495::TaskManager.Task::(_debug) Task 6b702ffc-4364-4d17-b04d-9a5105f72e5b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000310897827148', 'lastCheck': 1384919054.228816, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0254390239716', 'lastCheck': 1384919058.3604889, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0273048877716', 'lastCheck': 1384919054.242502, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00247502326965', 'lastCheck': 1384919057.3324289, 'valid': True, 'code': 0}}
>Thread-15914908::DEBUG::2013-11-20 03:44:21,893::task::495::TaskManager.Task::(_debug) Task 6b702ffc-4364-4d17-b04d-9a5105f72e5b: moving from state preparing -> state finished
>Thread-15914908::DEBUG::2013-11-20 03:44:21,894::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914908::DEBUG::2013-11-20 03:44:21,894::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914908::DEBUG::2013-11-20 03:44:21,895::task::495::TaskManager.Task::(_debug) Task 6b702ffc-4364-4d17-b04d-9a5105f72e5b: ref 0 aborting False
>Thread-15914908::INFO::2013-11-20 03:44:21,896::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000310897827148', 'lastCheck': 1384919054.228816, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0254390239716', 'lastCheck': 1384919058.3604889, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0273048877716', 'lastCheck': 1384919054.242502, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00247502326965', 'lastCheck': 1384919057.3324289, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:44:22,242::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:44:22,768::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0792072 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:44:24,786::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:44:25,337::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0910739 s, 11.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:44:27,347::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:44:27,903::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0898773 s, 11.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:44:29,924::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:44:30,391::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0581014 s, 17.6 MB/s\n'; <rc> = 0
>Thread-15914435::INFO::2013-11-20 03:44:30,501::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 03:44:30,502::task::495::TaskManager.Task::(_debug) Task fb4096fc-3339-4eaa-89f1-8434f1eeff82: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 03:44:30,504::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c1894fdd-e4b4-418c-84a9-19de2fd6b1f5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 03:44:30,505::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 03:44:30,505::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 03:44:30,506::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c1894fdd-e4b4-418c-84a9-19de2fd6b1f5`::Granted request
>Thread-15914435::DEBUG::2013-11-20 03:44:30,507::task::495::TaskManager.Task::(_debug) Task fb4096fc-3339-4eaa-89f1-8434f1eeff82: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 03:44:30,508::task::495::TaskManager.Task::(_debug) Task fb4096fc-3339-4eaa-89f1-8434f1eeff82: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 03:44:30,509::task::495::TaskManager.Task::(_debug) Task fb4096fc-3339-4eaa-89f1-8434f1eeff82: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 03:44:30,510::task::495::TaskManager.Task::(_debug) Task fb4096fc-3339-4eaa-89f1-8434f1eeff82: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 03:44:30,510::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 03:44:30,511::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 03:44:30,512::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 03:44:30,512::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 03:44:30,514::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 03:44:30,515::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 03:44:30,520::task::495::TaskManager.Task::(_debug) Task fb4096fc-3339-4eaa-89f1-8434f1eeff82: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 03:44:30,521::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914912::DEBUG::2013-11-20 03:44:30,591::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914912::INFO::2013-11-20 03:44:30,592::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914912::DEBUG::2013-11-20 03:44:30,593::task::495::TaskManager.Task::(_debug) Task 44177486-a10a-44cc-8c64-bc3cbd5671e3: moving from state init -> state preparing
>Thread-15914912::DEBUG::2013-11-20 03:44:30,594::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914912::DEBUG::2013-11-20 03:44:30,595::task::495::TaskManager.Task::(_debug) Task 44177486-a10a-44cc-8c64-bc3cbd5671e3: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914912::DEBUG::2013-11-20 03:44:30,595::task::495::TaskManager.Task::(_debug) Task 44177486-a10a-44cc-8c64-bc3cbd5671e3: moving from state preparing -> state finished
>Thread-15914912::DEBUG::2013-11-20 03:44:30,596::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914912::DEBUG::2013-11-20 03:44:30,597::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914912::DEBUG::2013-11-20 03:44:30,597::task::495::TaskManager.Task::(_debug) Task 44177486-a10a-44cc-8c64-bc3cbd5671e3: ref 0 aborting False
>Thread-15914912::INFO::2013-11-20 03:44:30,598::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914913::DEBUG::2013-11-20 03:44:30,612::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914913::INFO::2013-11-20 03:44:30,613::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914913::DEBUG::2013-11-20 03:44:30,614::task::495::TaskManager.Task::(_debug) Task c6aebbc1-843a-4154-9aa7-0a23f610a81f: moving from state init -> state preparing
>Thread-15914913::DEBUG::2013-11-20 03:44:30,615::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7abd7c9a-76b2-4ce8-a3fc-b304ddf19fd5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914913::DEBUG::2013-11-20 03:44:30,616::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914913::DEBUG::2013-11-20 03:44:30,617::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914913::DEBUG::2013-11-20 03:44:30,617::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7abd7c9a-76b2-4ce8-a3fc-b304ddf19fd5`::Granted request
>Thread-15914913::DEBUG::2013-11-20 03:44:30,618::task::495::TaskManager.Task::(_debug) Task c6aebbc1-843a-4154-9aa7-0a23f610a81f: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914913::DEBUG::2013-11-20 03:44:30,619::task::495::TaskManager.Task::(_debug) Task c6aebbc1-843a-4154-9aa7-0a23f610a81f: ref 1 aborting False
>Thread-15914913::WARNING::2013-11-20 03:44:30,621::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914913::DEBUG::2013-11-20 03:44:30,622::task::495::TaskManager.Task::(_debug) Task c6aebbc1-843a-4154-9aa7-0a23f610a81f: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914913::DEBUG::2013-11-20 03:44:30,623::task::495::TaskManager.Task::(_debug) Task c6aebbc1-843a-4154-9aa7-0a23f610a81f: moving from state preparing -> state finished
>Thread-15914913::DEBUG::2013-11-20 03:44:30,623::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914913::DEBUG::2013-11-20 03:44:30,624::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914913::DEBUG::2013-11-20 03:44:30,626::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914913::DEBUG::2013-11-20 03:44:30,626::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914913::DEBUG::2013-11-20 03:44:30,627::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914913::DEBUG::2013-11-20 03:44:30,628::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914913::DEBUG::2013-11-20 03:44:30,629::task::495::TaskManager.Task::(_debug) Task c6aebbc1-843a-4154-9aa7-0a23f610a81f: ref 0 aborting False
>Thread-15914913::INFO::2013-11-20 03:44:30,630::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914520::DEBUG::2013-11-20 03:44:31,701::guestIF::97::vm.Vm::(_handleMessage) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::u'BRD-KLEE\\administrator'
>Dummy-15912651::DEBUG::2013-11-20 03:44:32,405::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 03:44:32,614::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 03:44:32,617::task::495::TaskManager.Task::(_debug) Task 9d6bc462-4bcc-4d19-961e-4815c21ecbcf: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 03:44:32,621::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2b73e97e-88f9-42da-965b-c1ff0915f525`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 03:44:32,623::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 03:44:32,625::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 03:44:32,626::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2b73e97e-88f9-42da-965b-c1ff0915f525`::Granted request
>Thread-15914453::DEBUG::2013-11-20 03:44:32,630::task::495::TaskManager.Task::(_debug) Task 9d6bc462-4bcc-4d19-961e-4815c21ecbcf: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 03:44:32,631::task::495::TaskManager.Task::(_debug) Task 9d6bc462-4bcc-4d19-961e-4815c21ecbcf: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 03:44:32,632::task::495::TaskManager.Task::(_debug) Task 9d6bc462-4bcc-4d19-961e-4815c21ecbcf: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 03:44:32,633::task::495::TaskManager.Task::(_debug) Task 9d6bc462-4bcc-4d19-961e-4815c21ecbcf: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 03:44:32,634::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 03:44:32,635::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 03:44:32,637::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 03:44:32,638::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 03:44:32,639::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 03:44:32,640::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 03:44:32,641::task::495::TaskManager.Task::(_debug) Task 9d6bc462-4bcc-4d19-961e-4815c21ecbcf: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 03:44:32,642::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 03:44:32,950::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0782985 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:44:34,963::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:44:35,488::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0788137 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:44:37,504::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914916::INFO::2013-11-20 03:44:37,885::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914916::DEBUG::2013-11-20 03:44:37,886::task::495::TaskManager.Task::(_debug) Task 3fdfae1e-505c-4f11-bcc7-b56d507c6555: moving from state init -> state preparing
>Thread-15914916::DEBUG::2013-11-20 03:44:37,887::task::495::TaskManager.Task::(_debug) Task 3fdfae1e-505c-4f11-bcc7-b56d507c6555: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000395059585571', 'lastCheck': 1384919074.616246, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0019519329071', 'lastCheck': 1384919068.572329, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00960111618042', 'lastCheck': 1384919074.624815, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00112104415894', 'lastCheck': 1384919077.7420759, 'valid': True, 'code': 0}}
>Thread-15914916::DEBUG::2013-11-20 03:44:37,889::task::495::TaskManager.Task::(_debug) Task 3fdfae1e-505c-4f11-bcc7-b56d507c6555: moving from state preparing -> state finished
>Thread-15914916::DEBUG::2013-11-20 03:44:37,890::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914916::DEBUG::2013-11-20 03:44:37,891::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914916::DEBUG::2013-11-20 03:44:37,891::task::495::TaskManager.Task::(_debug) Task 3fdfae1e-505c-4f11-bcc7-b56d507c6555: ref 0 aborting False
>Thread-15914916::INFO::2013-11-20 03:44:37,892::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000395059585571', 'lastCheck': 1384919074.616246, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0019519329071', 'lastCheck': 1384919068.572329, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00960111618042', 'lastCheck': 1384919074.624815, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00112104415894', 'lastCheck': 1384919077.7420759, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:44:38,120::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.126435 s, 8.1 MB/s\n'; <rc> = 0
>Thread-15914519::INFO::2013-11-20 03:44:39,352::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 03:44:39,364::task::495::TaskManager.Task::(_debug) Task ce595c0c-e423-454c-952a-88bcd52846f9: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 03:44:39,370::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d8392e74-ac9d-4cf0-b8c0-efeb90d05270`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 03:44:39,371::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 03:44:39,372::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 03:44:39,373::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d8392e74-ac9d-4cf0-b8c0-efeb90d05270`::Granted request
>Thread-15914519::DEBUG::2013-11-20 03:44:39,374::task::495::TaskManager.Task::(_debug) Task ce595c0c-e423-454c-952a-88bcd52846f9: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 03:44:39,375::task::495::TaskManager.Task::(_debug) Task ce595c0c-e423-454c-952a-88bcd52846f9: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 03:44:39,376::task::495::TaskManager.Task::(_debug) Task ce595c0c-e423-454c-952a-88bcd52846f9: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 03:44:39,377::task::495::TaskManager.Task::(_debug) Task ce595c0c-e423-454c-952a-88bcd52846f9: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 03:44:39,378::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 03:44:39,378::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 03:44:39,383::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 03:44:39,384::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 03:44:39,385::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 03:44:39,386::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 03:44:39,386::task::495::TaskManager.Task::(_debug) Task ce595c0c-e423-454c-952a-88bcd52846f9: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 03:44:39,387::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 03:44:40,137::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:44:40,716::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0770988 s, 13.3 MB/s\n'; <rc> = 0
>Thread-15914918::DEBUG::2013-11-20 03:44:40,997::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914918::INFO::2013-11-20 03:44:40,999::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914918::DEBUG::2013-11-20 03:44:40,999::task::495::TaskManager.Task::(_debug) Task 80ab8dae-2d01-40ff-a137-0733ce06b579: moving from state init -> state preparing
>Thread-15914918::DEBUG::2013-11-20 03:44:41,001::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914918::DEBUG::2013-11-20 03:44:41,001::task::495::TaskManager.Task::(_debug) Task 80ab8dae-2d01-40ff-a137-0733ce06b579: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914918::DEBUG::2013-11-20 03:44:41,002::task::495::TaskManager.Task::(_debug) Task 80ab8dae-2d01-40ff-a137-0733ce06b579: moving from state preparing -> state finished
>Thread-15914918::DEBUG::2013-11-20 03:44:41,003::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914918::DEBUG::2013-11-20 03:44:41,003::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914918::DEBUG::2013-11-20 03:44:41,004::task::495::TaskManager.Task::(_debug) Task 80ab8dae-2d01-40ff-a137-0733ce06b579: ref 0 aborting False
>Thread-15914918::INFO::2013-11-20 03:44:41,005::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914919::DEBUG::2013-11-20 03:44:41,013::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914919::INFO::2013-11-20 03:44:41,014::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914919::DEBUG::2013-11-20 03:44:41,015::task::495::TaskManager.Task::(_debug) Task c11d2af2-9b5b-4330-9626-f45c21efadd2: moving from state init -> state preparing
>Thread-15914919::DEBUG::2013-11-20 03:44:41,017::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`47b45d45-22db-42bd-98ae-9df9ab864c7a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914919::DEBUG::2013-11-20 03:44:41,018::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914919::DEBUG::2013-11-20 03:44:41,019::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914919::DEBUG::2013-11-20 03:44:41,031::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`47b45d45-22db-42bd-98ae-9df9ab864c7a`::Granted request
>Thread-15914919::DEBUG::2013-11-20 03:44:41,032::task::495::TaskManager.Task::(_debug) Task c11d2af2-9b5b-4330-9626-f45c21efadd2: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914919::DEBUG::2013-11-20 03:44:41,033::task::495::TaskManager.Task::(_debug) Task c11d2af2-9b5b-4330-9626-f45c21efadd2: ref 1 aborting False
>Thread-15914919::WARNING::2013-11-20 03:44:41,035::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914919::DEBUG::2013-11-20 03:44:41,036::task::495::TaskManager.Task::(_debug) Task c11d2af2-9b5b-4330-9626-f45c21efadd2: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914919::DEBUG::2013-11-20 03:44:41,038::task::495::TaskManager.Task::(_debug) Task c11d2af2-9b5b-4330-9626-f45c21efadd2: moving from state preparing -> state finished
>Thread-15914919::DEBUG::2013-11-20 03:44:41,038::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914919::DEBUG::2013-11-20 03:44:41,039::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914919::DEBUG::2013-11-20 03:44:41,040::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914919::DEBUG::2013-11-20 03:44:41,047::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914919::DEBUG::2013-11-20 03:44:41,048::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914919::DEBUG::2013-11-20 03:44:41,048::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914919::DEBUG::2013-11-20 03:44:41,049::task::495::TaskManager.Task::(_debug) Task c11d2af2-9b5b-4330-9626-f45c21efadd2: ref 0 aborting False
>Thread-15914919::INFO::2013-11-20 03:44:41,050::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:44:42,730::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 03:44:42,990::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:44:42,997::task::495::TaskManager.Task::(_debug) Task f50f6b23-1e2d-476b-8642-ba181d520246: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:44:43,002::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`5248da86-e7a7-444c-96b7-a501075ba611`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:44:43,004::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:44:43,005::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:44:43,006::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`5248da86-e7a7-444c-96b7-a501075ba611`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:44:43,007::task::495::TaskManager.Task::(_debug) Task f50f6b23-1e2d-476b-8642-ba181d520246: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:44:43,008::task::495::TaskManager.Task::(_debug) Task f50f6b23-1e2d-476b-8642-ba181d520246: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:44:43,009::task::495::TaskManager.Task::(_debug) Task f50f6b23-1e2d-476b-8642-ba181d520246: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:44:43,009::task::495::TaskManager.Task::(_debug) Task f50f6b23-1e2d-476b-8642-ba181d520246: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:44:43,010::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:44:43,011::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:44:43,012::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:44:43,012::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:44:43,013::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913518::INFO::2013-11-20 03:44:43,015::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913546::DEBUG::2013-11-20 03:44:43,015::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:44:43,016::task::495::TaskManager.Task::(_debug) Task c3a20c39-be15-4c46-826c-7de2e69d8144: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:44:43,017::task::495::TaskManager.Task::(_debug) Task f50f6b23-1e2d-476b-8642-ba181d520246: ref 0 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:44:43,019::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`38f2abdd-3ea4-47fe-8b14-f7e761182c27`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::INFO::2013-11-20 03:44:43,020::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913518::DEBUG::2013-11-20 03:44:43,021::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913546::INFO::2013-11-20 03:44:43,021::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913518::DEBUG::2013-11-20 03:44:43,022::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:44:43,023::task::495::TaskManager.Task::(_debug) Task 8ab1a34e-e0eb-4a55-99f7-b70f9807da3d: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:44:43,024::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`38f2abdd-3ea4-47fe-8b14-f7e761182c27`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:44:43,026::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`913438ef-ffc1-4701-b36c-fb1323fe76a1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:44:43,027::task::495::TaskManager.Task::(_debug) Task c3a20c39-be15-4c46-826c-7de2e69d8144: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913546::DEBUG::2013-11-20 03:44:43,028::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:44:43,029::task::495::TaskManager.Task::(_debug) Task c3a20c39-be15-4c46-826c-7de2e69d8144: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:44:43,030::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:44:43,030::task::495::TaskManager.Task::(_debug) Task c3a20c39-be15-4c46-826c-7de2e69d8144: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913546::DEBUG::2013-11-20 03:44:43,031::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`913438ef-ffc1-4701-b36c-fb1323fe76a1`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:44:43,032::task::495::TaskManager.Task::(_debug) Task c3a20c39-be15-4c46-826c-7de2e69d8144: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:44:43,034::task::495::TaskManager.Task::(_debug) Task 8ab1a34e-e0eb-4a55-99f7-b70f9807da3d: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913518::DEBUG::2013-11-20 03:44:43,035::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:44:43,035::task::495::TaskManager.Task::(_debug) Task 8ab1a34e-e0eb-4a55-99f7-b70f9807da3d: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:44:43,036::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:44:43,037::task::495::TaskManager.Task::(_debug) Task 8ab1a34e-e0eb-4a55-99f7-b70f9807da3d: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913518::DEBUG::2013-11-20 03:44:43,038::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913546::DEBUG::2013-11-20 03:44:43,039::task::495::TaskManager.Task::(_debug) Task 8ab1a34e-e0eb-4a55-99f7-b70f9807da3d: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:44:43,039::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:44:43,040::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:44:43,041::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:44:43,041::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:44:43,042::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:44:43,043::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913518::DEBUG::2013-11-20 03:44:43,044::task::495::TaskManager.Task::(_debug) Task c3a20c39-be15-4c46-826c-7de2e69d8144: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:44:43,045::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913518::INFO::2013-11-20 03:44:43,045::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913546::DEBUG::2013-11-20 03:44:43,046::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913518::INFO::2013-11-20 03:44:43,047::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913546::DEBUG::2013-11-20 03:44:43,048::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:44:43,048::task::495::TaskManager.Task::(_debug) Task f327d1ce-5247-4c0c-8a96-d1c328b7ea8a: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:44:43,049::task::495::TaskManager.Task::(_debug) Task 8ab1a34e-e0eb-4a55-99f7-b70f9807da3d: ref 0 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:44:43,051::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6b84edde-515a-43d7-9ef2-8fa57307942f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::INFO::2013-11-20 03:44:43,052::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913518::DEBUG::2013-11-20 03:44:43,052::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913546::INFO::2013-11-20 03:44:43,053::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913518::DEBUG::2013-11-20 03:44:43,054::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:44:43,055::task::495::TaskManager.Task::(_debug) Task 606ca852-2592-40ca-bc69-6232916040d6: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:44:43,056::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6b84edde-515a-43d7-9ef2-8fa57307942f`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:44:43,057::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8b38a340-6d0f-412e-99eb-0aec8e5ffa4e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:44:43,058::task::495::TaskManager.Task::(_debug) Task f327d1ce-5247-4c0c-8a96-d1c328b7ea8a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913546::DEBUG::2013-11-20 03:44:43,059::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:44:43,060::task::495::TaskManager.Task::(_debug) Task f327d1ce-5247-4c0c-8a96-d1c328b7ea8a: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:44:43,061::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:44:43,062::task::495::TaskManager.Task::(_debug) Task f327d1ce-5247-4c0c-8a96-d1c328b7ea8a: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913546::DEBUG::2013-11-20 03:44:43,063::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8b38a340-6d0f-412e-99eb-0aec8e5ffa4e`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:44:43,064::task::495::TaskManager.Task::(_debug) Task f327d1ce-5247-4c0c-8a96-d1c328b7ea8a: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:44:43,066::task::495::TaskManager.Task::(_debug) Task 606ca852-2592-40ca-bc69-6232916040d6: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913518::DEBUG::2013-11-20 03:44:43,066::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:44:43,067::task::495::TaskManager.Task::(_debug) Task 606ca852-2592-40ca-bc69-6232916040d6: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:44:43,068::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:44:43,069::task::495::TaskManager.Task::(_debug) Task 606ca852-2592-40ca-bc69-6232916040d6: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:44:43,071::task::495::TaskManager.Task::(_debug) Task 606ca852-2592-40ca-bc69-6232916040d6: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:44:43,071::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913546::DEBUG::2013-11-20 03:44:43,072::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:44:43,073::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:44:43,074::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:44:43,075::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:44:43,076::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913518::DEBUG::2013-11-20 03:44:43,076::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:44:43,078::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:44:43,078::task::495::TaskManager.Task::(_debug) Task f327d1ce-5247-4c0c-8a96-d1c328b7ea8a: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:44:43,079::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913518::INFO::2013-11-20 03:44:43,080::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913546::DEBUG::2013-11-20 03:44:43,081::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:44:43,082::task::495::TaskManager.Task::(_debug) Task 606ca852-2592-40ca-bc69-6232916040d6: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:44:43,083::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:44:43,084::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:44:43,085::task::495::TaskManager.Task::(_debug) Task 31d13e50-7a4e-42b3-b6f4-a947d64b9557: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:44:43,086::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`0bfad1d7-8fb0-428a-b327-1418fa2fb445`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:44:43,087::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:44:43,088::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:44:43,088::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`0bfad1d7-8fb0-428a-b327-1418fa2fb445`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:44:43,089::task::495::TaskManager.Task::(_debug) Task 31d13e50-7a4e-42b3-b6f4-a947d64b9557: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:44:43,090::task::495::TaskManager.Task::(_debug) Task 31d13e50-7a4e-42b3-b6f4-a947d64b9557: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:44:43,090::task::495::TaskManager.Task::(_debug) Task 31d13e50-7a4e-42b3-b6f4-a947d64b9557: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:44:43,091::task::495::TaskManager.Task::(_debug) Task 31d13e50-7a4e-42b3-b6f4-a947d64b9557: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:44:43,092::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:44:43,093::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:44:43,094::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:44:43,095::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:44:43,096::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:44:43,096::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:44:43,097::task::495::TaskManager.Task::(_debug) Task 31d13e50-7a4e-42b3-b6f4-a947d64b9557: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:44:43,098::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:44:43,278::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0890398 s, 11.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:44:45,295::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:44:45,857::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.105932 s, 9.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:44:47,877::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-42840::DEBUG::2013-11-20 03:44:48,105::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:44:48,761::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.16786 s, 6.1 MB/s\n'; <rc> = 0
>Thread-42840::DEBUG::2013-11-20 03:44:49,205::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:44:50,776::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:44:51,347::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0883161 s, 11.6 MB/s\n'; <rc> = 0
>Thread-15914923::DEBUG::2013-11-20 03:44:51,458::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914923::INFO::2013-11-20 03:44:51,459::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914923::DEBUG::2013-11-20 03:44:51,460::task::495::TaskManager.Task::(_debug) Task 626250a8-0f19-4141-a8d3-f05992b95fae: moving from state init -> state preparing
>Thread-15914923::DEBUG::2013-11-20 03:44:51,461::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914923::DEBUG::2013-11-20 03:44:51,462::task::495::TaskManager.Task::(_debug) Task 626250a8-0f19-4141-a8d3-f05992b95fae: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914923::DEBUG::2013-11-20 03:44:51,463::task::495::TaskManager.Task::(_debug) Task 626250a8-0f19-4141-a8d3-f05992b95fae: moving from state preparing -> state finished
>Thread-15914923::DEBUG::2013-11-20 03:44:51,464::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914923::DEBUG::2013-11-20 03:44:51,464::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914923::DEBUG::2013-11-20 03:44:51,465::task::495::TaskManager.Task::(_debug) Task 626250a8-0f19-4141-a8d3-f05992b95fae: ref 0 aborting False
>Thread-15914923::INFO::2013-11-20 03:44:51,466::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914924::DEBUG::2013-11-20 03:44:51,472::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914924::INFO::2013-11-20 03:44:51,473::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914924::DEBUG::2013-11-20 03:44:51,474::task::495::TaskManager.Task::(_debug) Task 1bcac420-d1f4-46ca-806d-87c71f4d9a5f: moving from state init -> state preparing
>Thread-15914924::DEBUG::2013-11-20 03:44:51,475::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3acc1a61-0d83-47c8-a1cc-f345c1a5747a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914924::DEBUG::2013-11-20 03:44:51,476::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914924::DEBUG::2013-11-20 03:44:51,477::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914924::DEBUG::2013-11-20 03:44:51,478::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3acc1a61-0d83-47c8-a1cc-f345c1a5747a`::Granted request
>Thread-15914924::DEBUG::2013-11-20 03:44:51,479::task::495::TaskManager.Task::(_debug) Task 1bcac420-d1f4-46ca-806d-87c71f4d9a5f: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914924::DEBUG::2013-11-20 03:44:51,485::task::495::TaskManager.Task::(_debug) Task 1bcac420-d1f4-46ca-806d-87c71f4d9a5f: ref 1 aborting False
>Thread-15914924::WARNING::2013-11-20 03:44:51,488::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914924::DEBUG::2013-11-20 03:44:51,489::task::495::TaskManager.Task::(_debug) Task 1bcac420-d1f4-46ca-806d-87c71f4d9a5f: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914924::DEBUG::2013-11-20 03:44:51,490::task::495::TaskManager.Task::(_debug) Task 1bcac420-d1f4-46ca-806d-87c71f4d9a5f: moving from state preparing -> state finished
>Thread-15914924::DEBUG::2013-11-20 03:44:51,491::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914924::DEBUG::2013-11-20 03:44:51,492::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914924::DEBUG::2013-11-20 03:44:51,493::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914924::DEBUG::2013-11-20 03:44:51,494::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914924::DEBUG::2013-11-20 03:44:51,494::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914924::DEBUG::2013-11-20 03:44:51,495::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914924::DEBUG::2013-11-20 03:44:51,497::task::495::TaskManager.Task::(_debug) Task 1bcac420-d1f4-46ca-806d-87c71f4d9a5f: ref 0 aborting False
>Thread-15914924::INFO::2013-11-20 03:44:51,498::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:44:53,363::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:44:53,915::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0824887 s, 12.4 MB/s\n'; <rc> = 0
>Thread-15914926::INFO::2013-11-20 03:44:54,688::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914926::DEBUG::2013-11-20 03:44:54,689::task::495::TaskManager.Task::(_debug) Task 3d916504-f173-4b1d-9085-80d84c6462c9: moving from state init -> state preparing
>Thread-15914926::DEBUG::2013-11-20 03:44:54,689::task::495::TaskManager.Task::(_debug) Task 3d916504-f173-4b1d-9085-80d84c6462c9: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000415086746216', 'lastCheck': 1384919084.9607019, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00172591209412', 'lastCheck': 1384919089.1205289, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00466990470886', 'lastCheck': 1384919084.9609411, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000704050064087', 'lastCheck': 1384919089.2082651, 'valid': True, 'code': 0}}
>Thread-15914926::DEBUG::2013-11-20 03:44:54,695::task::495::TaskManager.Task::(_debug) Task 3d916504-f173-4b1d-9085-80d84c6462c9: moving from state preparing -> state finished
>Thread-15914926::DEBUG::2013-11-20 03:44:54,696::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914926::DEBUG::2013-11-20 03:44:54,697::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914926::DEBUG::2013-11-20 03:44:54,698::task::495::TaskManager.Task::(_debug) Task 3d916504-f173-4b1d-9085-80d84c6462c9: ref 0 aborting False
>Thread-15914926::INFO::2013-11-20 03:44:54,700::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000415086746216', 'lastCheck': 1384919084.9607019, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00172591209412', 'lastCheck': 1384919089.1205289, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00466990470886', 'lastCheck': 1384919084.9609411, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000704050064087', 'lastCheck': 1384919089.2082651, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:44:55,929::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:44:56,503::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0946371 s, 10.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:44:58,519::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:44:59,118::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0825559 s, 12.4 MB/s\n'; <rc> = 0
>Thread-15913639::INFO::2013-11-20 03:44:59,590::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:44:59,591::task::495::TaskManager.Task::(_debug) Task 1ccb9d6b-4726-4ddd-b449-d28ecee61f28: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:44:59,593::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e9108366-93f1-4863-92de-4c7f8346bbce`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:44:59,593::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:44:59,595::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:44:59,595::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e9108366-93f1-4863-92de-4c7f8346bbce`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:44:59,598::task::495::TaskManager.Task::(_debug) Task 1ccb9d6b-4726-4ddd-b449-d28ecee61f28: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:44:59,599::task::495::TaskManager.Task::(_debug) Task 1ccb9d6b-4726-4ddd-b449-d28ecee61f28: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:44:59,600::task::495::TaskManager.Task::(_debug) Task 1ccb9d6b-4726-4ddd-b449-d28ecee61f28: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:44:59,601::task::495::TaskManager.Task::(_debug) Task 1ccb9d6b-4726-4ddd-b449-d28ecee61f28: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:44:59,602::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:44:59,603::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:44:59,603::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:44:59,604::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:44:59,605::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:44:59,605::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:44:59,606::task::495::TaskManager.Task::(_debug) Task 1ccb9d6b-4726-4ddd-b449-d28ecee61f28: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:44:59,607::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:45:01,133::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:45:01,727::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0794391 s, 12.9 MB/s\n'; <rc> = 0
>Thread-15914930::DEBUG::2013-11-20 03:45:01,913::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914930::INFO::2013-11-20 03:45:01,914::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914930::DEBUG::2013-11-20 03:45:01,915::task::495::TaskManager.Task::(_debug) Task 047fa5a6-f012-4d93-b0a8-c475c89ac706: moving from state init -> state preparing
>Thread-15914930::DEBUG::2013-11-20 03:45:01,917::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914930::DEBUG::2013-11-20 03:45:01,917::task::495::TaskManager.Task::(_debug) Task 047fa5a6-f012-4d93-b0a8-c475c89ac706: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914930::DEBUG::2013-11-20 03:45:01,918::task::495::TaskManager.Task::(_debug) Task 047fa5a6-f012-4d93-b0a8-c475c89ac706: moving from state preparing -> state finished
>Thread-15914930::DEBUG::2013-11-20 03:45:01,919::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914930::DEBUG::2013-11-20 03:45:01,919::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914930::DEBUG::2013-11-20 03:45:01,920::task::495::TaskManager.Task::(_debug) Task 047fa5a6-f012-4d93-b0a8-c475c89ac706: ref 0 aborting False
>Thread-15914930::INFO::2013-11-20 03:45:01,921::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914931::DEBUG::2013-11-20 03:45:01,932::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914931::INFO::2013-11-20 03:45:01,933::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914931::DEBUG::2013-11-20 03:45:01,934::task::495::TaskManager.Task::(_debug) Task 25624cd5-f5a8-4869-a279-c33e0144ec4a: moving from state init -> state preparing
>Thread-15914931::DEBUG::2013-11-20 03:45:01,935::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`73860e7d-e57b-40a2-a196-0f75efaec61d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914931::DEBUG::2013-11-20 03:45:01,936::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914931::DEBUG::2013-11-20 03:45:01,937::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914931::DEBUG::2013-11-20 03:45:01,938::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`73860e7d-e57b-40a2-a196-0f75efaec61d`::Granted request
>Thread-15914931::DEBUG::2013-11-20 03:45:01,939::task::495::TaskManager.Task::(_debug) Task 25624cd5-f5a8-4869-a279-c33e0144ec4a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914931::DEBUG::2013-11-20 03:45:01,939::task::495::TaskManager.Task::(_debug) Task 25624cd5-f5a8-4869-a279-c33e0144ec4a: ref 1 aborting False
>Thread-15914931::WARNING::2013-11-20 03:45:01,941::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914931::DEBUG::2013-11-20 03:45:01,948::task::495::TaskManager.Task::(_debug) Task 25624cd5-f5a8-4869-a279-c33e0144ec4a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914931::DEBUG::2013-11-20 03:45:01,949::task::495::TaskManager.Task::(_debug) Task 25624cd5-f5a8-4869-a279-c33e0144ec4a: moving from state preparing -> state finished
>Thread-15914931::DEBUG::2013-11-20 03:45:01,950::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914931::DEBUG::2013-11-20 03:45:01,951::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914931::DEBUG::2013-11-20 03:45:01,952::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914931::DEBUG::2013-11-20 03:45:01,953::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914931::DEBUG::2013-11-20 03:45:01,954::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914931::DEBUG::2013-11-20 03:45:01,954::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914931::DEBUG::2013-11-20 03:45:01,960::task::495::TaskManager.Task::(_debug) Task 25624cd5-f5a8-4869-a279-c33e0144ec4a: ref 0 aborting False
>Thread-15914931::INFO::2013-11-20 03:45:01,961::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:45:03,741::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:45:04,334::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0661012 s, 15.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:45:06,346::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:45:06,915::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0926753 s, 11.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:45:08,930::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:45:09,468::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0666259 s, 15.4 MB/s\n'; <rc> = 0
>Thread-15914934::INFO::2013-11-20 03:45:10,992::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914934::DEBUG::2013-11-20 03:45:10,993::task::495::TaskManager.Task::(_debug) Task 6309cd5a-7ffe-45d4-8a60-8bfce9e4fa86: moving from state init -> state preparing
>Thread-15914934::DEBUG::2013-11-20 03:45:10,994::task::495::TaskManager.Task::(_debug) Task 6309cd5a-7ffe-45d4-8a60-8bfce9e4fa86: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000442028045654', 'lastCheck': 1384919105.3725281, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00137591362', 'lastCheck': 1384919109.5875731, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00167202949524', 'lastCheck': 1384919105.3746951, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000420093536377', 'lastCheck': 1384919109.577297, 'valid': True, 'code': 0}}
>Thread-15914934::DEBUG::2013-11-20 03:45:10,995::task::495::TaskManager.Task::(_debug) Task 6309cd5a-7ffe-45d4-8a60-8bfce9e4fa86: moving from state preparing -> state finished
>Thread-15914934::DEBUG::2013-11-20 03:45:10,996::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914934::DEBUG::2013-11-20 03:45:10,997::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914934::DEBUG::2013-11-20 03:45:10,997::task::495::TaskManager.Task::(_debug) Task 6309cd5a-7ffe-45d4-8a60-8bfce9e4fa86: ref 0 aborting False
>Thread-15914934::INFO::2013-11-20 03:45:11,004::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000442028045654', 'lastCheck': 1384919105.3725281, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00137591362', 'lastCheck': 1384919109.5875731, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00167202949524', 'lastCheck': 1384919105.3746951, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000420093536377', 'lastCheck': 1384919109.577297, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:45:11,482::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:45:12,114::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0875194 s, 11.7 MB/s\n'; <rc> = 0
>Thread-15914936::DEBUG::2013-11-20 03:45:12,752::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914936::INFO::2013-11-20 03:45:12,753::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914936::DEBUG::2013-11-20 03:45:12,754::task::495::TaskManager.Task::(_debug) Task f768e0c7-9cca-4110-869e-693994a28c9f: moving from state init -> state preparing
>Thread-15914936::DEBUG::2013-11-20 03:45:12,754::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914936::DEBUG::2013-11-20 03:45:12,755::task::495::TaskManager.Task::(_debug) Task f768e0c7-9cca-4110-869e-693994a28c9f: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914936::DEBUG::2013-11-20 03:45:12,755::task::495::TaskManager.Task::(_debug) Task f768e0c7-9cca-4110-869e-693994a28c9f: moving from state preparing -> state finished
>Thread-15914936::DEBUG::2013-11-20 03:45:12,756::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914936::DEBUG::2013-11-20 03:45:12,756::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914936::DEBUG::2013-11-20 03:45:12,757::task::495::TaskManager.Task::(_debug) Task f768e0c7-9cca-4110-869e-693994a28c9f: ref 0 aborting False
>Thread-15914936::INFO::2013-11-20 03:45:12,757::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914937::DEBUG::2013-11-20 03:45:12,768::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914937::INFO::2013-11-20 03:45:12,772::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914937::DEBUG::2013-11-20 03:45:12,773::task::495::TaskManager.Task::(_debug) Task ab4113fb-53e3-418d-ab89-d90ce8d25506: moving from state init -> state preparing
>Thread-15914937::DEBUG::2013-11-20 03:45:12,774::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1f5088dd-65bc-4d82-90a2-eca3e9b66ab5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914937::DEBUG::2013-11-20 03:45:12,774::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914937::DEBUG::2013-11-20 03:45:12,775::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914937::DEBUG::2013-11-20 03:45:12,775::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1f5088dd-65bc-4d82-90a2-eca3e9b66ab5`::Granted request
>Thread-15914937::DEBUG::2013-11-20 03:45:12,776::task::495::TaskManager.Task::(_debug) Task ab4113fb-53e3-418d-ab89-d90ce8d25506: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914937::DEBUG::2013-11-20 03:45:12,777::task::495::TaskManager.Task::(_debug) Task ab4113fb-53e3-418d-ab89-d90ce8d25506: ref 1 aborting False
>Thread-15914937::WARNING::2013-11-20 03:45:12,778::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914937::DEBUG::2013-11-20 03:45:12,779::task::495::TaskManager.Task::(_debug) Task ab4113fb-53e3-418d-ab89-d90ce8d25506: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914937::DEBUG::2013-11-20 03:45:12,780::task::495::TaskManager.Task::(_debug) Task ab4113fb-53e3-418d-ab89-d90ce8d25506: moving from state preparing -> state finished
>Thread-15914937::DEBUG::2013-11-20 03:45:12,780::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914937::DEBUG::2013-11-20 03:45:12,781::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914937::DEBUG::2013-11-20 03:45:12,781::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914937::DEBUG::2013-11-20 03:45:12,782::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914937::DEBUG::2013-11-20 03:45:12,782::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914937::DEBUG::2013-11-20 03:45:12,783::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914937::DEBUG::2013-11-20 03:45:12,783::task::495::TaskManager.Task::(_debug) Task ab4113fb-53e3-418d-ab89-d90ce8d25506: ref 0 aborting False
>Thread-15914937::INFO::2013-11-20 03:45:12,784::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913658::INFO::2013-11-20 03:45:13,140::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:45:13,141::task::495::TaskManager.Task::(_debug) Task 693291c2-60fe-44db-8536-ae204fd97efe: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:45:13,143::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`573ebd6f-3f29-4233-8674-6bd40ba81b69`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:45:13,144::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:45:13,145::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:45:13,146::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`573ebd6f-3f29-4233-8674-6bd40ba81b69`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:45:13,147::task::495::TaskManager.Task::(_debug) Task 693291c2-60fe-44db-8536-ae204fd97efe: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:45:13,148::task::495::TaskManager.Task::(_debug) Task 693291c2-60fe-44db-8536-ae204fd97efe: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:45:13,149::task::495::TaskManager.Task::(_debug) Task 693291c2-60fe-44db-8536-ae204fd97efe: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:45:13,150::task::495::TaskManager.Task::(_debug) Task 693291c2-60fe-44db-8536-ae204fd97efe: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:45:13,151::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:45:13,152::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:45:13,153::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:45:13,153::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:45:13,154::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:45:13,155::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:45:13,156::task::495::TaskManager.Task::(_debug) Task 693291c2-60fe-44db-8536-ae204fd97efe: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:45:13,157::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:45:14,131::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:45:14,710::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0903212 s, 11.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:45:16,726::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:45:17,390::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0887813 s, 11.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:45:19,406::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:45:19,958::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0778786 s, 13.1 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 03:45:20,029::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:45:20,030::task::495::TaskManager.Task::(_debug) Task d77b770f-1805-4b62-b012-49b58436cce8: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:45:20,032::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`63d47d72-931f-4935-9357-aa592e6409a0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:45:20,033::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:45:20,034::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:45:20,034::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`63d47d72-931f-4935-9357-aa592e6409a0`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:45:20,036::task::495::TaskManager.Task::(_debug) Task d77b770f-1805-4b62-b012-49b58436cce8: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:45:20,036::task::495::TaskManager.Task::(_debug) Task d77b770f-1805-4b62-b012-49b58436cce8: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:45:20,037::task::495::TaskManager.Task::(_debug) Task d77b770f-1805-4b62-b012-49b58436cce8: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:45:20,038::task::495::TaskManager.Task::(_debug) Task d77b770f-1805-4b62-b012-49b58436cce8: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:45:20,038::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:45:20,039::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:45:20,040::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:45:20,040::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:45:20,041::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:45:20,041::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:45:20,042::task::495::TaskManager.Task::(_debug) Task d77b770f-1805-4b62-b012-49b58436cce8: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:45:20,043::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913695::INFO::2013-11-20 03:45:21,017::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:45:21,019::task::495::TaskManager.Task::(_debug) Task ea8c0208-f9ae-4fcc-847e-734575768611: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:45:21,020::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`00aebfe0-d589-47df-b071-4ce6d5879740`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:45:21,022::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:45:21,022::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:45:21,023::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`00aebfe0-d589-47df-b071-4ce6d5879740`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:45:21,024::task::495::TaskManager.Task::(_debug) Task ea8c0208-f9ae-4fcc-847e-734575768611: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:45:21,025::task::495::TaskManager.Task::(_debug) Task ea8c0208-f9ae-4fcc-847e-734575768611: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:45:21,026::task::495::TaskManager.Task::(_debug) Task ea8c0208-f9ae-4fcc-847e-734575768611: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:45:21,026::task::495::TaskManager.Task::(_debug) Task ea8c0208-f9ae-4fcc-847e-734575768611: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:45:21,027::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:45:21,028::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:45:21,028::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:45:21,029::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:45:21,030::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:45:21,030::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:45:21,031::task::495::TaskManager.Task::(_debug) Task ea8c0208-f9ae-4fcc-847e-734575768611: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:45:21,032::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:45:21,973::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:45:22,506::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.075497 s, 13.6 MB/s\n'; <rc> = 0
>Thread-15914941::DEBUG::2013-11-20 03:45:23,435::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914941::INFO::2013-11-20 03:45:23,436::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914941::DEBUG::2013-11-20 03:45:23,437::task::495::TaskManager.Task::(_debug) Task dae56966-4656-412f-9775-a317a4f1dc31: moving from state init -> state preparing
>Thread-15914941::DEBUG::2013-11-20 03:45:23,438::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914941::DEBUG::2013-11-20 03:45:23,439::task::495::TaskManager.Task::(_debug) Task dae56966-4656-412f-9775-a317a4f1dc31: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914941::DEBUG::2013-11-20 03:45:23,440::task::495::TaskManager.Task::(_debug) Task dae56966-4656-412f-9775-a317a4f1dc31: moving from state preparing -> state finished
>Thread-15914941::DEBUG::2013-11-20 03:45:23,441::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914941::DEBUG::2013-11-20 03:45:23,441::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914941::DEBUG::2013-11-20 03:45:23,442::task::495::TaskManager.Task::(_debug) Task dae56966-4656-412f-9775-a317a4f1dc31: ref 0 aborting False
>Thread-15914941::INFO::2013-11-20 03:45:23,443::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914942::DEBUG::2013-11-20 03:45:23,448::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914942::INFO::2013-11-20 03:45:23,450::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914942::DEBUG::2013-11-20 03:45:23,450::task::495::TaskManager.Task::(_debug) Task 5e0622b3-d196-4b2a-aac1-5a8e71b1ab5b: moving from state init -> state preparing
>Thread-15914942::DEBUG::2013-11-20 03:45:23,452::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`30c1d814-9e53-46bc-8315-4e979c736f6b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914942::DEBUG::2013-11-20 03:45:23,453::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914942::DEBUG::2013-11-20 03:45:23,455::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914942::DEBUG::2013-11-20 03:45:23,456::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`30c1d814-9e53-46bc-8315-4e979c736f6b`::Granted request
>Thread-15914942::DEBUG::2013-11-20 03:45:23,458::task::495::TaskManager.Task::(_debug) Task 5e0622b3-d196-4b2a-aac1-5a8e71b1ab5b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914942::DEBUG::2013-11-20 03:45:23,458::task::495::TaskManager.Task::(_debug) Task 5e0622b3-d196-4b2a-aac1-5a8e71b1ab5b: ref 1 aborting False
>Thread-15914942::WARNING::2013-11-20 03:45:23,460::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914942::DEBUG::2013-11-20 03:45:23,461::task::495::TaskManager.Task::(_debug) Task 5e0622b3-d196-4b2a-aac1-5a8e71b1ab5b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914942::DEBUG::2013-11-20 03:45:23,462::task::495::TaskManager.Task::(_debug) Task 5e0622b3-d196-4b2a-aac1-5a8e71b1ab5b: moving from state preparing -> state finished
>Thread-15914942::DEBUG::2013-11-20 03:45:23,463::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914942::DEBUG::2013-11-20 03:45:23,464::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914942::DEBUG::2013-11-20 03:45:23,464::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914942::DEBUG::2013-11-20 03:45:23,465::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914942::DEBUG::2013-11-20 03:45:23,466::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914942::DEBUG::2013-11-20 03:45:23,466::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914942::DEBUG::2013-11-20 03:45:23,467::task::495::TaskManager.Task::(_debug) Task 5e0622b3-d196-4b2a-aac1-5a8e71b1ab5b: ref 0 aborting False
>Thread-15914942::INFO::2013-11-20 03:45:23,467::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:45:24,523::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:45:25,070::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0697507 s, 14.7 MB/s\n'; <rc> = 0
>Thread-15914944::INFO::2013-11-20 03:45:26,981::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914944::DEBUG::2013-11-20 03:45:26,982::task::495::TaskManager.Task::(_debug) Task 4ff69088-d7a6-4a15-92a3-d7759cd01d57: moving from state init -> state preparing
>Thread-15914944::DEBUG::2013-11-20 03:45:26,983::task::495::TaskManager.Task::(_debug) Task 4ff69088-d7a6-4a15-92a3-d7759cd01d57: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000382900238037', 'lastCheck': 1384919126.1989319, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00136303901672', 'lastCheck': 1384919119.995054, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0147619247437', 'lastCheck': 1384919126.2237229, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000497817993164', 'lastCheck': 1384919119.98894, 'valid': True, 'code': 0}}
>Thread-15914944::DEBUG::2013-11-20 03:45:26,984::task::495::TaskManager.Task::(_debug) Task 4ff69088-d7a6-4a15-92a3-d7759cd01d57: moving from state preparing -> state finished
>Thread-15914944::DEBUG::2013-11-20 03:45:26,985::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914944::DEBUG::2013-11-20 03:45:26,986::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914944::DEBUG::2013-11-20 03:45:26,986::task::495::TaskManager.Task::(_debug) Task 4ff69088-d7a6-4a15-92a3-d7759cd01d57: ref 0 aborting False
>Thread-15914944::INFO::2013-11-20 03:45:26,987::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000382900238037', 'lastCheck': 1384919126.1989319, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00136303901672', 'lastCheck': 1384919119.995054, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0147619247437', 'lastCheck': 1384919126.2237229, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000497817993164', 'lastCheck': 1384919119.98894, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:45:27,089::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:45:27,663::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0942184 s, 10.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:45:29,681::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:45:30,459::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0748137 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15914435::INFO::2013-11-20 03:45:31,088::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 03:45:31,089::task::495::TaskManager.Task::(_debug) Task 77b7a9f7-7fae-4d38-8522-519fcd9763b4: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 03:45:31,091::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f348000d-3698-4a8f-bd44-4fb0b6b7e10d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 03:45:31,091::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 03:45:31,092::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 03:45:31,093::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f348000d-3698-4a8f-bd44-4fb0b6b7e10d`::Granted request
>Thread-15914435::DEBUG::2013-11-20 03:45:31,094::task::495::TaskManager.Task::(_debug) Task 77b7a9f7-7fae-4d38-8522-519fcd9763b4: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 03:45:31,095::task::495::TaskManager.Task::(_debug) Task 77b7a9f7-7fae-4d38-8522-519fcd9763b4: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 03:45:31,096::task::495::TaskManager.Task::(_debug) Task 77b7a9f7-7fae-4d38-8522-519fcd9763b4: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 03:45:31,097::task::495::TaskManager.Task::(_debug) Task 77b7a9f7-7fae-4d38-8522-519fcd9763b4: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 03:45:31,098::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 03:45:31,098::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 03:45:31,099::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 03:45:31,100::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 03:45:31,100::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 03:45:31,101::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 03:45:31,102::task::495::TaskManager.Task::(_debug) Task 77b7a9f7-7fae-4d38-8522-519fcd9763b4: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 03:45:31,103::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 03:45:32,474::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:45:33,033::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0908433 s, 11.3 MB/s\n'; <rc> = 0
>Thread-15914948::DEBUG::2013-11-20 03:45:33,683::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914948::INFO::2013-11-20 03:45:33,685::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914948::DEBUG::2013-11-20 03:45:33,685::task::495::TaskManager.Task::(_debug) Task 83d11186-42e4-4533-9eb2-2b11b51abcf1: moving from state init -> state preparing
>Thread-15914948::DEBUG::2013-11-20 03:45:33,686::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914948::DEBUG::2013-11-20 03:45:33,687::task::495::TaskManager.Task::(_debug) Task 83d11186-42e4-4533-9eb2-2b11b51abcf1: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914948::DEBUG::2013-11-20 03:45:33,688::task::495::TaskManager.Task::(_debug) Task 83d11186-42e4-4533-9eb2-2b11b51abcf1: moving from state preparing -> state finished
>Thread-15914948::DEBUG::2013-11-20 03:45:33,689::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914948::DEBUG::2013-11-20 03:45:33,689::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914948::DEBUG::2013-11-20 03:45:33,690::task::495::TaskManager.Task::(_debug) Task 83d11186-42e4-4533-9eb2-2b11b51abcf1: ref 0 aborting False
>Thread-15914948::INFO::2013-11-20 03:45:33,691::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914949::DEBUG::2013-11-20 03:45:33,700::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914949::INFO::2013-11-20 03:45:33,701::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914949::DEBUG::2013-11-20 03:45:33,702::task::495::TaskManager.Task::(_debug) Task 7a303f86-dbb4-430e-a8cb-ea65c1386aec: moving from state init -> state preparing
>Thread-15914949::DEBUG::2013-11-20 03:45:33,703::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4be4f6ee-6f77-4ba0-b589-9b2350396044`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914949::DEBUG::2013-11-20 03:45:33,704::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914949::DEBUG::2013-11-20 03:45:33,705::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914949::DEBUG::2013-11-20 03:45:33,706::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4be4f6ee-6f77-4ba0-b589-9b2350396044`::Granted request
>Thread-15914949::DEBUG::2013-11-20 03:45:33,717::task::495::TaskManager.Task::(_debug) Task 7a303f86-dbb4-430e-a8cb-ea65c1386aec: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914949::DEBUG::2013-11-20 03:45:33,718::task::495::TaskManager.Task::(_debug) Task 7a303f86-dbb4-430e-a8cb-ea65c1386aec: ref 1 aborting False
>Thread-15914949::WARNING::2013-11-20 03:45:33,722::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914949::DEBUG::2013-11-20 03:45:33,723::task::495::TaskManager.Task::(_debug) Task 7a303f86-dbb4-430e-a8cb-ea65c1386aec: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914949::DEBUG::2013-11-20 03:45:33,724::task::495::TaskManager.Task::(_debug) Task 7a303f86-dbb4-430e-a8cb-ea65c1386aec: moving from state preparing -> state finished
>Thread-15914949::DEBUG::2013-11-20 03:45:33,725::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914949::DEBUG::2013-11-20 03:45:33,726::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914949::DEBUG::2013-11-20 03:45:33,727::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914949::DEBUG::2013-11-20 03:45:33,728::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914949::DEBUG::2013-11-20 03:45:33,729::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914949::DEBUG::2013-11-20 03:45:33,730::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914949::DEBUG::2013-11-20 03:45:33,731::task::495::TaskManager.Task::(_debug) Task 7a303f86-dbb4-430e-a8cb-ea65c1386aec: ref 0 aborting False
>Thread-15914949::INFO::2013-11-20 03:45:33,732::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914453::INFO::2013-11-20 03:45:34,397::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 03:45:34,398::task::495::TaskManager.Task::(_debug) Task bc239481-2c3c-429a-b5b4-82a73e12188d: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 03:45:34,400::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`498a620b-7078-48da-ac34-0ee2e13f2528`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 03:45:34,401::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 03:45:34,401::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 03:45:34,402::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`498a620b-7078-48da-ac34-0ee2e13f2528`::Granted request
>Thread-15914453::DEBUG::2013-11-20 03:45:34,403::task::495::TaskManager.Task::(_debug) Task bc239481-2c3c-429a-b5b4-82a73e12188d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 03:45:34,404::task::495::TaskManager.Task::(_debug) Task bc239481-2c3c-429a-b5b4-82a73e12188d: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 03:45:34,405::task::495::TaskManager.Task::(_debug) Task bc239481-2c3c-429a-b5b4-82a73e12188d: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 03:45:34,405::task::495::TaskManager.Task::(_debug) Task bc239481-2c3c-429a-b5b4-82a73e12188d: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 03:45:34,406::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 03:45:34,407::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 03:45:34,408::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 03:45:34,408::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 03:45:34,409::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 03:45:34,409::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 03:45:34,410::task::495::TaskManager.Task::(_debug) Task bc239481-2c3c-429a-b5b4-82a73e12188d: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 03:45:34,411::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 03:45:35,050::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:45:35,644::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0855843 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:45:37,661::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:45:38,222::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0844869 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15914519::INFO::2013-11-20 03:45:39,923::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 03:45:39,932::task::495::TaskManager.Task::(_debug) Task 19effa57-134e-4480-87cf-62b5540006a8: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 03:45:39,935::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5b575ba0-d768-42ef-9827-3769e01c49c6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 03:45:39,937::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 03:45:39,938::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 03:45:39,939::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5b575ba0-d768-42ef-9827-3769e01c49c6`::Granted request
>Thread-15914519::DEBUG::2013-11-20 03:45:39,940::task::495::TaskManager.Task::(_debug) Task 19effa57-134e-4480-87cf-62b5540006a8: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 03:45:39,943::task::495::TaskManager.Task::(_debug) Task 19effa57-134e-4480-87cf-62b5540006a8: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 03:45:39,944::task::495::TaskManager.Task::(_debug) Task 19effa57-134e-4480-87cf-62b5540006a8: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 03:45:39,945::task::495::TaskManager.Task::(_debug) Task 19effa57-134e-4480-87cf-62b5540006a8: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 03:45:39,946::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 03:45:39,947::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 03:45:39,948::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 03:45:39,949::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 03:45:39,949::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 03:45:39,950::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 03:45:39,951::task::495::TaskManager.Task::(_debug) Task 19effa57-134e-4480-87cf-62b5540006a8: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 03:45:39,952::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 03:45:40,238::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:45:40,766::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0763289 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15914952::INFO::2013-11-20 03:45:42,654::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914952::DEBUG::2013-11-20 03:45:42,655::task::495::TaskManager.Task::(_debug) Task a90d3f45-1a9b-43f5-8efc-5756af2dc561: moving from state init -> state preparing
>Thread-15914952::DEBUG::2013-11-20 03:45:42,657::task::495::TaskManager.Task::(_debug) Task a90d3f45-1a9b-43f5-8efc-5756af2dc561: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00024676322937', 'lastCheck': 1384919136.2920129, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00119209289551', 'lastCheck': 1384919140.456955, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00166583061218', 'lastCheck': 1384919136.292532, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00424003601074', 'lastCheck': 1384919140.450222, 'valid': True, 'code': 0}}
>Thread-15914952::DEBUG::2013-11-20 03:45:42,658::task::495::TaskManager.Task::(_debug) Task a90d3f45-1a9b-43f5-8efc-5756af2dc561: moving from state preparing -> state finished
>Thread-15914952::DEBUG::2013-11-20 03:45:42,659::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914952::DEBUG::2013-11-20 03:45:42,659::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914952::DEBUG::2013-11-20 03:45:42,660::task::495::TaskManager.Task::(_debug) Task a90d3f45-1a9b-43f5-8efc-5756af2dc561: ref 0 aborting False
>Thread-15914952::INFO::2013-11-20 03:45:42,661::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00024676322937', 'lastCheck': 1384919136.2920129, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00119209289551', 'lastCheck': 1384919140.456955, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00166583061218', 'lastCheck': 1384919136.292532, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00424003601074', 'lastCheck': 1384919140.450222, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:45:42,782::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:45:43,379::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.103205 s, 9.9 MB/s\n'; <rc> = 0
>Thread-15914954::DEBUG::2013-11-20 03:45:44,325::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914954::INFO::2013-11-20 03:45:44,327::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914954::DEBUG::2013-11-20 03:45:44,328::task::495::TaskManager.Task::(_debug) Task 62f8b4d9-e4dc-4485-8cc1-b27fa74c48a8: moving from state init -> state preparing
>Thread-15914954::DEBUG::2013-11-20 03:45:44,329::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914954::DEBUG::2013-11-20 03:45:44,330::task::495::TaskManager.Task::(_debug) Task 62f8b4d9-e4dc-4485-8cc1-b27fa74c48a8: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914954::DEBUG::2013-11-20 03:45:44,330::task::495::TaskManager.Task::(_debug) Task 62f8b4d9-e4dc-4485-8cc1-b27fa74c48a8: moving from state preparing -> state finished
>Thread-15914954::DEBUG::2013-11-20 03:45:44,331::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914954::DEBUG::2013-11-20 03:45:44,332::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914954::DEBUG::2013-11-20 03:45:44,333::task::495::TaskManager.Task::(_debug) Task 62f8b4d9-e4dc-4485-8cc1-b27fa74c48a8: ref 0 aborting False
>Thread-15914954::INFO::2013-11-20 03:45:44,334::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914955::DEBUG::2013-11-20 03:45:44,344::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914955::INFO::2013-11-20 03:45:44,345::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914955::DEBUG::2013-11-20 03:45:44,345::task::495::TaskManager.Task::(_debug) Task db10e58f-ae20-41c4-aff4-06049f03ced1: moving from state init -> state preparing
>Thread-15914955::DEBUG::2013-11-20 03:45:44,347::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`da3c6818-3ce0-498f-8403-0b5e3bc0299d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914955::DEBUG::2013-11-20 03:45:44,347::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914955::DEBUG::2013-11-20 03:45:44,348::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914955::DEBUG::2013-11-20 03:45:44,349::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`da3c6818-3ce0-498f-8403-0b5e3bc0299d`::Granted request
>Thread-15914955::DEBUG::2013-11-20 03:45:44,350::task::495::TaskManager.Task::(_debug) Task db10e58f-ae20-41c4-aff4-06049f03ced1: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914955::DEBUG::2013-11-20 03:45:44,351::task::495::TaskManager.Task::(_debug) Task db10e58f-ae20-41c4-aff4-06049f03ced1: ref 1 aborting False
>Thread-15914955::WARNING::2013-11-20 03:45:44,353::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914955::DEBUG::2013-11-20 03:45:44,354::task::495::TaskManager.Task::(_debug) Task db10e58f-ae20-41c4-aff4-06049f03ced1: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914955::DEBUG::2013-11-20 03:45:44,355::task::495::TaskManager.Task::(_debug) Task db10e58f-ae20-41c4-aff4-06049f03ced1: moving from state preparing -> state finished
>Thread-15914955::DEBUG::2013-11-20 03:45:44,356::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914955::DEBUG::2013-11-20 03:45:44,360::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914955::DEBUG::2013-11-20 03:45:44,361::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914955::DEBUG::2013-11-20 03:45:44,362::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914955::DEBUG::2013-11-20 03:45:44,363::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914955::DEBUG::2013-11-20 03:45:44,364::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914955::DEBUG::2013-11-20 03:45:44,365::task::495::TaskManager.Task::(_debug) Task db10e58f-ae20-41c4-aff4-06049f03ced1: ref 0 aborting False
>Thread-15914955::INFO::2013-11-20 03:45:44,366::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913546::INFO::2013-11-20 03:45:44,450::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:45:44,452::task::495::TaskManager.Task::(_debug) Task 66e53887-3bbd-4055-95d4-074e9f37a668: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:45:44,453::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b373a24b-0d59-44f1-8388-3276132384b5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:45:44,454::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:45:44,455::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:45:44,456::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b373a24b-0d59-44f1-8388-3276132384b5`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:45:44,457::task::495::TaskManager.Task::(_debug) Task 66e53887-3bbd-4055-95d4-074e9f37a668: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:45:44,458::task::495::TaskManager.Task::(_debug) Task 66e53887-3bbd-4055-95d4-074e9f37a668: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:45:44,458::task::495::TaskManager.Task::(_debug) Task 66e53887-3bbd-4055-95d4-074e9f37a668: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:45:44,459::task::495::TaskManager.Task::(_debug) Task 66e53887-3bbd-4055-95d4-074e9f37a668: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:45:44,459::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:45:44,460::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:45:44,461::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:45:44,461::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:45:44,462::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:45:44,462::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:45:44,463::task::495::TaskManager.Task::(_debug) Task 66e53887-3bbd-4055-95d4-074e9f37a668: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:45:44,463::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:45:44,464::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:45:44,465::task::495::TaskManager.Task::(_debug) Task 0c14015d-83dc-484b-822e-40a86ba0cbff: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:45:44,466::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`00d8d686-d395-443c-bca2-ee26fd948433`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:45:44,466::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:45:44,467::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:45:44,467::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`00d8d686-d395-443c-bca2-ee26fd948433`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:45:44,468::task::495::TaskManager.Task::(_debug) Task 0c14015d-83dc-484b-822e-40a86ba0cbff: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:45:44,469::task::495::TaskManager.Task::(_debug) Task 0c14015d-83dc-484b-822e-40a86ba0cbff: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:45:44,469::task::495::TaskManager.Task::(_debug) Task 0c14015d-83dc-484b-822e-40a86ba0cbff: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:45:44,470::task::495::TaskManager.Task::(_debug) Task 0c14015d-83dc-484b-822e-40a86ba0cbff: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:45:44,471::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:45:44,471::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:45:44,472::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:45:44,472::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:45:44,473::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:45:44,473::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:45:44,474::task::495::TaskManager.Task::(_debug) Task 0c14015d-83dc-484b-822e-40a86ba0cbff: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:45:44,475::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:45:44,475::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:45:44,476::task::495::TaskManager.Task::(_debug) Task 97006790-94f7-4029-b532-ea34be0009b7: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:45:44,477::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f880d6b4-7780-4dac-8322-3e9774179f34`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:45:44,477::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:45:44,478::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:45:44,479::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f880d6b4-7780-4dac-8322-3e9774179f34`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:45:44,480::task::495::TaskManager.Task::(_debug) Task 97006790-94f7-4029-b532-ea34be0009b7: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:45:44,480::task::495::TaskManager.Task::(_debug) Task 97006790-94f7-4029-b532-ea34be0009b7: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:45:44,481::task::495::TaskManager.Task::(_debug) Task 97006790-94f7-4029-b532-ea34be0009b7: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:45:44,481::task::495::TaskManager.Task::(_debug) Task 97006790-94f7-4029-b532-ea34be0009b7: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:45:44,482::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:45:44,482::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:45:44,483::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:45:44,484::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:45:44,484::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:45:44,485::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:45:44,485::task::495::TaskManager.Task::(_debug) Task 97006790-94f7-4029-b532-ea34be0009b7: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:45:44,486::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:45:44,487::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:45:44,487::task::495::TaskManager.Task::(_debug) Task 3fb75443-6312-4c8d-8517-98e108937c12: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:45:44,488::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c4d0b75c-131a-4110-9d45-b59d4b4be8b0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:45:44,489::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:45:44,490::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:45:44,490::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c4d0b75c-131a-4110-9d45-b59d4b4be8b0`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:45:44,491::task::495::TaskManager.Task::(_debug) Task 3fb75443-6312-4c8d-8517-98e108937c12: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:45:44,492::task::495::TaskManager.Task::(_debug) Task 3fb75443-6312-4c8d-8517-98e108937c12: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:45:44,492::task::495::TaskManager.Task::(_debug) Task 3fb75443-6312-4c8d-8517-98e108937c12: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:45:44,493::task::495::TaskManager.Task::(_debug) Task 3fb75443-6312-4c8d-8517-98e108937c12: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:45:44,493::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:45:44,494::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:45:44,495::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:45:44,495::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:45:44,496::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:45:44,496::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:45:44,497::task::495::TaskManager.Task::(_debug) Task 3fb75443-6312-4c8d-8517-98e108937c12: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:45:44,498::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913518::INFO::2013-11-20 03:45:45,100::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:45:45,100::task::495::TaskManager.Task::(_debug) Task 0f09a5f5-163f-4fde-9e35-cbc60e52fe3e: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:45:45,102::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`44d3c453-09da-4ae5-8873-88058aa4cdbd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:45:45,102::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:45:45,103::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:45:45,104::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`44d3c453-09da-4ae5-8873-88058aa4cdbd`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:45:45,105::task::495::TaskManager.Task::(_debug) Task 0f09a5f5-163f-4fde-9e35-cbc60e52fe3e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:45:45,105::task::495::TaskManager.Task::(_debug) Task 0f09a5f5-163f-4fde-9e35-cbc60e52fe3e: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:45:45,106::task::495::TaskManager.Task::(_debug) Task 0f09a5f5-163f-4fde-9e35-cbc60e52fe3e: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:45:45,107::task::495::TaskManager.Task::(_debug) Task 0f09a5f5-163f-4fde-9e35-cbc60e52fe3e: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:45:45,107::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:45:45,108::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:45:45,108::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:45:45,109::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:45:45,110::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:45:45,110::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:45:45,111::task::495::TaskManager.Task::(_debug) Task 0f09a5f5-163f-4fde-9e35-cbc60e52fe3e: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:45:45,112::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:45:45,112::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:45:45,113::task::495::TaskManager.Task::(_debug) Task a2623a3d-3c7e-439a-a1a6-e66e157576c2: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:45:45,114::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5373377c-f74b-422f-bb4a-4f389472bdfd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:45:45,115::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:45:45,116::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:45:45,117::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5373377c-f74b-422f-bb4a-4f389472bdfd`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:45:45,118::task::495::TaskManager.Task::(_debug) Task a2623a3d-3c7e-439a-a1a6-e66e157576c2: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:45:45,118::task::495::TaskManager.Task::(_debug) Task a2623a3d-3c7e-439a-a1a6-e66e157576c2: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:45:45,119::task::495::TaskManager.Task::(_debug) Task a2623a3d-3c7e-439a-a1a6-e66e157576c2: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:45:45,119::task::495::TaskManager.Task::(_debug) Task a2623a3d-3c7e-439a-a1a6-e66e157576c2: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:45:45,120::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:45:45,120::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:45:45,121::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:45:45,122::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:45:45,122::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:45:45,123::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:45:45,123::task::495::TaskManager.Task::(_debug) Task a2623a3d-3c7e-439a-a1a6-e66e157576c2: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:45:45,124::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:45:45,396::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:45:45,938::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.04898 s, 20.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:45:47,951::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:45:48,476::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.102993 s, 9.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:45:50,491::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:45:51,035::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0895291 s, 11.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:45:53,049::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:45:53,573::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0640349 s, 16.0 MB/s\n'; <rc> = 0
>Thread-15914959::DEBUG::2013-11-20 03:45:55,327::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914959::INFO::2013-11-20 03:45:55,329::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914959::DEBUG::2013-11-20 03:45:55,329::task::495::TaskManager.Task::(_debug) Task 75cb25cc-4c15-45d4-8aa3-d1534332525a: moving from state init -> state preparing
>Thread-15914959::DEBUG::2013-11-20 03:45:55,331::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914959::DEBUG::2013-11-20 03:45:55,331::task::495::TaskManager.Task::(_debug) Task 75cb25cc-4c15-45d4-8aa3-d1534332525a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914959::DEBUG::2013-11-20 03:45:55,332::task::495::TaskManager.Task::(_debug) Task 75cb25cc-4c15-45d4-8aa3-d1534332525a: moving from state preparing -> state finished
>Thread-15914959::DEBUG::2013-11-20 03:45:55,333::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914959::DEBUG::2013-11-20 03:45:55,334::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914959::DEBUG::2013-11-20 03:45:55,335::task::495::TaskManager.Task::(_debug) Task 75cb25cc-4c15-45d4-8aa3-d1534332525a: ref 0 aborting False
>Thread-15914959::INFO::2013-11-20 03:45:55,335::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914960::DEBUG::2013-11-20 03:45:55,350::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914960::INFO::2013-11-20 03:45:55,351::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914960::DEBUG::2013-11-20 03:45:55,352::task::495::TaskManager.Task::(_debug) Task 74c6566c-d4df-493f-9805-d0ee0bbd51ac: moving from state init -> state preparing
>Thread-15914960::DEBUG::2013-11-20 03:45:55,354::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7637aefb-2bc5-4327-ae3d-9a51b8334d92`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914960::DEBUG::2013-11-20 03:45:55,355::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914960::DEBUG::2013-11-20 03:45:55,357::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914960::DEBUG::2013-11-20 03:45:55,358::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7637aefb-2bc5-4327-ae3d-9a51b8334d92`::Granted request
>Thread-15914960::DEBUG::2013-11-20 03:45:55,359::task::495::TaskManager.Task::(_debug) Task 74c6566c-d4df-493f-9805-d0ee0bbd51ac: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914960::DEBUG::2013-11-20 03:45:55,360::task::495::TaskManager.Task::(_debug) Task 74c6566c-d4df-493f-9805-d0ee0bbd51ac: ref 1 aborting False
>Thread-15914960::WARNING::2013-11-20 03:45:55,363::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914960::DEBUG::2013-11-20 03:45:55,364::task::495::TaskManager.Task::(_debug) Task 74c6566c-d4df-493f-9805-d0ee0bbd51ac: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914960::DEBUG::2013-11-20 03:45:55,365::task::495::TaskManager.Task::(_debug) Task 74c6566c-d4df-493f-9805-d0ee0bbd51ac: moving from state preparing -> state finished
>Thread-15914960::DEBUG::2013-11-20 03:45:55,366::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914960::DEBUG::2013-11-20 03:45:55,366::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914960::DEBUG::2013-11-20 03:45:55,367::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914960::DEBUG::2013-11-20 03:45:55,368::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914960::DEBUG::2013-11-20 03:45:55,368::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914960::DEBUG::2013-11-20 03:45:55,372::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914960::DEBUG::2013-11-20 03:45:55,374::task::495::TaskManager.Task::(_debug) Task 74c6566c-d4df-493f-9805-d0ee0bbd51ac: ref 0 aborting False
>Thread-15914960::INFO::2013-11-20 03:45:55,375::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:45:55,590::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:45:56,124::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0765295 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15914520::DEBUG::2013-11-20 03:45:57,445::guestIF::97::vm.Vm::(_handleMessage) vmId=`e97d1c52-b688-4192-bdeb-2dfadcc79786`::u'None'
>Dummy-15912651::DEBUG::2013-11-20 03:45:58,138::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:45:58,726::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.103594 s, 9.9 MB/s\n'; <rc> = 0
>Thread-15914962::INFO::2013-11-20 03:45:58,853::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914962::DEBUG::2013-11-20 03:45:58,854::task::495::TaskManager.Task::(_debug) Task 5ecef50f-56bc-4eea-9b28-83a3f75c19a7: moving from state init -> state preparing
>Thread-15914962::DEBUG::2013-11-20 03:45:58,855::task::495::TaskManager.Task::(_debug) Task 5ecef50f-56bc-4eea-9b28-83a3f75c19a7: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000396013259888', 'lastCheck': 1384919156.8203001, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00156021118164', 'lastCheck': 1384919150.7007849, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0103521347046', 'lastCheck': 1384919156.82953, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000940084457397', 'lastCheck': 1384919150.6948459, 'valid': True, 'code': 0}}
>Thread-15914962::DEBUG::2013-11-20 03:45:58,856::task::495::TaskManager.Task::(_debug) Task 5ecef50f-56bc-4eea-9b28-83a3f75c19a7: moving from state preparing -> state finished
>Thread-15914962::DEBUG::2013-11-20 03:45:58,857::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914962::DEBUG::2013-11-20 03:45:58,858::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914962::DEBUG::2013-11-20 03:45:58,859::task::495::TaskManager.Task::(_debug) Task 5ecef50f-56bc-4eea-9b28-83a3f75c19a7: ref 0 aborting False
>Thread-15914962::INFO::2013-11-20 03:45:58,860::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000396013259888', 'lastCheck': 1384919156.8203001, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00156021118164', 'lastCheck': 1384919150.7007849, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0103521347046', 'lastCheck': 1384919156.82953, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000940084457397', 'lastCheck': 1384919150.6948459, 'valid': True, 'code': 0}}
>Thread-15913639::INFO::2013-11-20 03:45:59,959::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:45:59,960::task::495::TaskManager.Task::(_debug) Task 0e1585f4-c501-473a-b093-fc61b434799b: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:45:59,962::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5a1d90f2-7233-41e2-ba64-423fa0350d88`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:45:59,962::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:45:59,963::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:45:59,964::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5a1d90f2-7233-41e2-ba64-423fa0350d88`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:45:59,965::task::495::TaskManager.Task::(_debug) Task 0e1585f4-c501-473a-b093-fc61b434799b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:45:59,966::task::495::TaskManager.Task::(_debug) Task 0e1585f4-c501-473a-b093-fc61b434799b: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:45:59,967::task::495::TaskManager.Task::(_debug) Task 0e1585f4-c501-473a-b093-fc61b434799b: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:45:59,967::task::495::TaskManager.Task::(_debug) Task 0e1585f4-c501-473a-b093-fc61b434799b: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:45:59,968::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:45:59,969::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:45:59,969::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:45:59,970::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:45:59,971::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:45:59,971::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:45:59,972::task::495::TaskManager.Task::(_debug) Task 0e1585f4-c501-473a-b093-fc61b434799b: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:45:59,972::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:46:00,737::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:46:01,355::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0956751 s, 10.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:46:03,395::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:46:03,956::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0769584 s, 13.3 MB/s\n'; <rc> = 0
>Thread-15914965::DEBUG::2013-11-20 03:46:05,805::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914965::INFO::2013-11-20 03:46:05,807::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914965::DEBUG::2013-11-20 03:46:05,808::task::495::TaskManager.Task::(_debug) Task 0dd89555-8d86-497d-a714-ccbae2a75a4c: moving from state init -> state preparing
>Thread-15914965::DEBUG::2013-11-20 03:46:05,810::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914965::DEBUG::2013-11-20 03:46:05,811::task::495::TaskManager.Task::(_debug) Task 0dd89555-8d86-497d-a714-ccbae2a75a4c: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914965::DEBUG::2013-11-20 03:46:05,812::task::495::TaskManager.Task::(_debug) Task 0dd89555-8d86-497d-a714-ccbae2a75a4c: moving from state preparing -> state finished
>Thread-15914965::DEBUG::2013-11-20 03:46:05,814::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914965::DEBUG::2013-11-20 03:46:05,819::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914965::DEBUG::2013-11-20 03:46:05,820::task::495::TaskManager.Task::(_debug) Task 0dd89555-8d86-497d-a714-ccbae2a75a4c: ref 0 aborting False
>Thread-15914965::INFO::2013-11-20 03:46:05,821::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914967::DEBUG::2013-11-20 03:46:05,841::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914967::INFO::2013-11-20 03:46:05,842::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914967::DEBUG::2013-11-20 03:46:05,843::task::495::TaskManager.Task::(_debug) Task 9ea3e029-9b0a-4a18-80e2-2b6584b3a449: moving from state init -> state preparing
>Thread-15914967::DEBUG::2013-11-20 03:46:05,844::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`16421dc5-47ed-4545-8b72-c01a7c662826`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914967::DEBUG::2013-11-20 03:46:05,845::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914967::DEBUG::2013-11-20 03:46:05,846::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914967::DEBUG::2013-11-20 03:46:05,846::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`16421dc5-47ed-4545-8b72-c01a7c662826`::Granted request
>Thread-15914967::DEBUG::2013-11-20 03:46:05,848::task::495::TaskManager.Task::(_debug) Task 9ea3e029-9b0a-4a18-80e2-2b6584b3a449: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914967::DEBUG::2013-11-20 03:46:05,848::task::495::TaskManager.Task::(_debug) Task 9ea3e029-9b0a-4a18-80e2-2b6584b3a449: ref 1 aborting False
>Thread-15914967::WARNING::2013-11-20 03:46:05,851::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914967::DEBUG::2013-11-20 03:46:05,852::task::495::TaskManager.Task::(_debug) Task 9ea3e029-9b0a-4a18-80e2-2b6584b3a449: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914967::DEBUG::2013-11-20 03:46:05,853::task::495::TaskManager.Task::(_debug) Task 9ea3e029-9b0a-4a18-80e2-2b6584b3a449: moving from state preparing -> state finished
>Thread-15914967::DEBUG::2013-11-20 03:46:05,853::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914967::DEBUG::2013-11-20 03:46:05,854::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914967::DEBUG::2013-11-20 03:46:05,855::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914967::DEBUG::2013-11-20 03:46:05,856::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914967::DEBUG::2013-11-20 03:46:05,856::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914967::DEBUG::2013-11-20 03:46:05,857::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914967::DEBUG::2013-11-20 03:46:05,857::task::495::TaskManager.Task::(_debug) Task 9ea3e029-9b0a-4a18-80e2-2b6584b3a449: ref 0 aborting False
>Thread-15914967::INFO::2013-11-20 03:46:05,858::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:46:05,967::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:46:06,559::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0807401 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:46:08,574::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:46:09,132::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0815125 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:46:11,149::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:46:11,738::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.10307 s, 9.9 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 03:46:13,396::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:46:13,399::task::495::TaskManager.Task::(_debug) Task a2879605-7864-4906-988b-ea591aedbc12: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:46:13,402::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5ef7e432-b2e9-4790-bffb-4fd8851bae19`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:46:13,403::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:46:13,404::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:46:13,405::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5ef7e432-b2e9-4790-bffb-4fd8851bae19`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:46:13,406::task::495::TaskManager.Task::(_debug) Task a2879605-7864-4906-988b-ea591aedbc12: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:46:13,407::task::495::TaskManager.Task::(_debug) Task a2879605-7864-4906-988b-ea591aedbc12: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:46:13,408::task::495::TaskManager.Task::(_debug) Task a2879605-7864-4906-988b-ea591aedbc12: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:46:13,409::task::495::TaskManager.Task::(_debug) Task a2879605-7864-4906-988b-ea591aedbc12: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:46:13,410::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:46:13,411::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:46:13,412::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:46:13,413::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:46:13,414::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:46:13,414::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:46:13,416::task::495::TaskManager.Task::(_debug) Task a2879605-7864-4906-988b-ea591aedbc12: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:46:13,416::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:46:13,758::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:46:14,334::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0856352 s, 12.0 MB/s\n'; <rc> = 0
>Thread-15914970::INFO::2013-11-20 03:46:14,973::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914970::DEBUG::2013-11-20 03:46:14,974::task::495::TaskManager.Task::(_debug) Task 9aa2dcae-da9a-4bf7-a379-494a6b3ff184: moving from state init -> state preparing
>Thread-15914970::DEBUG::2013-11-20 03:46:14,975::task::495::TaskManager.Task::(_debug) Task 9aa2dcae-da9a-4bf7-a379-494a6b3ff184: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000489950180054', 'lastCheck': 1384919167.2255521, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00135397911072', 'lastCheck': 1384919171.3799801, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00357294082642', 'lastCheck': 1384919167.230746, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000894069671631', 'lastCheck': 1384919171.3718231, 'valid': True, 'code': 0}}
>Thread-15914970::DEBUG::2013-11-20 03:46:14,976::task::495::TaskManager.Task::(_debug) Task 9aa2dcae-da9a-4bf7-a379-494a6b3ff184: moving from state preparing -> state finished
>Thread-15914970::DEBUG::2013-11-20 03:46:14,977::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914970::DEBUG::2013-11-20 03:46:14,978::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914970::DEBUG::2013-11-20 03:46:14,978::task::495::TaskManager.Task::(_debug) Task 9aa2dcae-da9a-4bf7-a379-494a6b3ff184: ref 0 aborting False
>Thread-15914970::INFO::2013-11-20 03:46:14,979::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000489950180054', 'lastCheck': 1384919167.2255521, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00135397911072', 'lastCheck': 1384919171.3799801, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00357294082642', 'lastCheck': 1384919167.230746, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000894069671631', 'lastCheck': 1384919171.3718231, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:46:16,344::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914972::DEBUG::2013-11-20 03:46:16,755::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914972::INFO::2013-11-20 03:46:16,757::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914972::DEBUG::2013-11-20 03:46:16,758::task::495::TaskManager.Task::(_debug) Task 8a40bb52-4be8-48bb-b6e3-fb6a2321195a: moving from state init -> state preparing
>Thread-15914972::DEBUG::2013-11-20 03:46:16,759::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914972::DEBUG::2013-11-20 03:46:16,759::task::495::TaskManager.Task::(_debug) Task 8a40bb52-4be8-48bb-b6e3-fb6a2321195a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914972::DEBUG::2013-11-20 03:46:16,760::task::495::TaskManager.Task::(_debug) Task 8a40bb52-4be8-48bb-b6e3-fb6a2321195a: moving from state preparing -> state finished
>Thread-15914972::DEBUG::2013-11-20 03:46:16,761::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914972::DEBUG::2013-11-20 03:46:16,761::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914972::DEBUG::2013-11-20 03:46:16,762::task::495::TaskManager.Task::(_debug) Task 8a40bb52-4be8-48bb-b6e3-fb6a2321195a: ref 0 aborting False
>Thread-15914972::INFO::2013-11-20 03:46:16,762::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914973::DEBUG::2013-11-20 03:46:16,773::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914973::INFO::2013-11-20 03:46:16,774::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914973::DEBUG::2013-11-20 03:46:16,775::task::495::TaskManager.Task::(_debug) Task c01478a3-101e-4651-892f-97d7018243f5: moving from state init -> state preparing
>Thread-15914973::DEBUG::2013-11-20 03:46:16,777::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9c2d544e-36e4-470c-ab16-be8711168879`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914973::DEBUG::2013-11-20 03:46:16,778::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914973::DEBUG::2013-11-20 03:46:16,780::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914973::DEBUG::2013-11-20 03:46:16,781::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9c2d544e-36e4-470c-ab16-be8711168879`::Granted request
>Thread-15914973::DEBUG::2013-11-20 03:46:16,794::task::495::TaskManager.Task::(_debug) Task c01478a3-101e-4651-892f-97d7018243f5: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914973::DEBUG::2013-11-20 03:46:16,795::task::495::TaskManager.Task::(_debug) Task c01478a3-101e-4651-892f-97d7018243f5: ref 1 aborting False
>Thread-15914973::WARNING::2013-11-20 03:46:16,799::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914973::DEBUG::2013-11-20 03:46:16,801::task::495::TaskManager.Task::(_debug) Task c01478a3-101e-4651-892f-97d7018243f5: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914973::DEBUG::2013-11-20 03:46:16,802::task::495::TaskManager.Task::(_debug) Task c01478a3-101e-4651-892f-97d7018243f5: moving from state preparing -> state finished
>Thread-15914973::DEBUG::2013-11-20 03:46:16,803::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914973::DEBUG::2013-11-20 03:46:16,804::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914973::DEBUG::2013-11-20 03:46:16,805::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914973::DEBUG::2013-11-20 03:46:16,807::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914973::DEBUG::2013-11-20 03:46:16,808::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914973::DEBUG::2013-11-20 03:46:16,809::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914973::DEBUG::2013-11-20 03:46:16,810::task::495::TaskManager.Task::(_debug) Task c01478a3-101e-4651-892f-97d7018243f5: ref 0 aborting False
>Thread-15914973::INFO::2013-11-20 03:46:16,811::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:46:16,930::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.11111 s, 9.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:46:18,945::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:46:19,470::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.088701 s, 11.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:46:21,486::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 03:46:21,746::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:46:21,750::task::495::TaskManager.Task::(_debug) Task c57a4c49-69e1-488a-bcad-ab647c4e881e: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:46:21,754::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b573be01-fe46-4af9-a4b0-e21390e22b22`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:46:21,756::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:46:21,757::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:46:21,759::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b573be01-fe46-4af9-a4b0-e21390e22b22`::Granted request
>Thread-15913695::INFO::2013-11-20 03:46:21,762::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15912392::DEBUG::2013-11-20 03:46:21,762::task::495::TaskManager.Task::(_debug) Task c57a4c49-69e1-488a-bcad-ab647c4e881e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:46:21,763::task::495::TaskManager.Task::(_debug) Task 89be7958-46bd-4a0c-9872-87c748617ac4: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:46:21,764::task::495::TaskManager.Task::(_debug) Task c57a4c49-69e1-488a-bcad-ab647c4e881e: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:46:21,766::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9a5d0f87-e19e-41bd-b6c6-cbcf87a7d011`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:46:21,767::task::495::TaskManager.Task::(_debug) Task c57a4c49-69e1-488a-bcad-ab647c4e881e: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913695::DEBUG::2013-11-20 03:46:21,768::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:46:21,769::task::495::TaskManager.Task::(_debug) Task c57a4c49-69e1-488a-bcad-ab647c4e881e: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:46:21,770::resourceManager::481::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' found in shared state and queue is empty, Joining current shared lock (2 active users)
>Thread-15912392::DEBUG::2013-11-20 03:46:21,771::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:46:21,772::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9a5d0f87-e19e-41bd-b6c6-cbcf87a7d011`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:46:21,773::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:46:21,775::task::495::TaskManager.Task::(_debug) Task 89be7958-46bd-4a0c-9872-87c748617ac4: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:46:21,776::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:46:21,777::task::495::TaskManager.Task::(_debug) Task 89be7958-46bd-4a0c-9872-87c748617ac4: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:46:21,778::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (1 active users)
>Thread-15913695::DEBUG::2013-11-20 03:46:21,779::task::495::TaskManager.Task::(_debug) Task 89be7958-46bd-4a0c-9872-87c748617ac4: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:46:21,780::task::495::TaskManager.Task::(_debug) Task 89be7958-46bd-4a0c-9872-87c748617ac4: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:46:21,780::task::495::TaskManager.Task::(_debug) Task c57a4c49-69e1-488a-bcad-ab647c4e881e: ref 0 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:46:21,782::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::INFO::2013-11-20 03:46:21,782::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913695::DEBUG::2013-11-20 03:46:21,783::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:46:21,785::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:46:21,787::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:46:21,788::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:46:21,791::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:46:21,792::task::495::TaskManager.Task::(_debug) Task 89be7958-46bd-4a0c-9872-87c748617ac4: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:46:21,793::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:46:22,063::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0845121 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:46:24,078::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:46:24,656::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.083771 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:46:26,676::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914977::DEBUG::2013-11-20 03:46:27,177::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914977::INFO::2013-11-20 03:46:27,180::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914977::DEBUG::2013-11-20 03:46:27,181::task::495::TaskManager.Task::(_debug) Task 203d3aca-f5dd-4c14-af6d-5b35ac6d5803: moving from state init -> state preparing
>Thread-15914977::DEBUG::2013-11-20 03:46:27,182::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914977::DEBUG::2013-11-20 03:46:27,183::task::495::TaskManager.Task::(_debug) Task 203d3aca-f5dd-4c14-af6d-5b35ac6d5803: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914977::DEBUG::2013-11-20 03:46:27,185::task::495::TaskManager.Task::(_debug) Task 203d3aca-f5dd-4c14-af6d-5b35ac6d5803: moving from state preparing -> state finished
>Thread-15914977::DEBUG::2013-11-20 03:46:27,186::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914977::DEBUG::2013-11-20 03:46:27,187::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914977::DEBUG::2013-11-20 03:46:27,189::task::495::TaskManager.Task::(_debug) Task 203d3aca-f5dd-4c14-af6d-5b35ac6d5803: ref 0 aborting False
>Thread-15914977::INFO::2013-11-20 03:46:27,195::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914978::DEBUG::2013-11-20 03:46:27,215::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914978::INFO::2013-11-20 03:46:27,218::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914978::DEBUG::2013-11-20 03:46:27,220::task::495::TaskManager.Task::(_debug) Task ba5e037e-3812-4e94-ac52-15827c2426db: moving from state init -> state preparing
>Thread-15914978::DEBUG::2013-11-20 03:46:27,224::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`458f72f3-01b4-4455-97d8-9a585dfa14cd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914978::DEBUG::2013-11-20 03:46:27,225::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914978::DEBUG::2013-11-20 03:46:27,230::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914978::DEBUG::2013-11-20 03:46:27,231::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`458f72f3-01b4-4455-97d8-9a585dfa14cd`::Granted request
>Thread-15914978::DEBUG::2013-11-20 03:46:27,236::task::495::TaskManager.Task::(_debug) Task ba5e037e-3812-4e94-ac52-15827c2426db: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914978::DEBUG::2013-11-20 03:46:27,239::task::495::TaskManager.Task::(_debug) Task ba5e037e-3812-4e94-ac52-15827c2426db: ref 1 aborting False
>Thread-15914978::WARNING::2013-11-20 03:46:27,245::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914978::DEBUG::2013-11-20 03:46:27,253::task::495::TaskManager.Task::(_debug) Task ba5e037e-3812-4e94-ac52-15827c2426db: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914978::DEBUG::2013-11-20 03:46:27,256::task::495::TaskManager.Task::(_debug) Task ba5e037e-3812-4e94-ac52-15827c2426db: moving from state preparing -> state finished
>Thread-15914978::DEBUG::2013-11-20 03:46:27,257::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914978::DEBUG::2013-11-20 03:46:27,260::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914978::DEBUG::2013-11-20 03:46:27,262::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914978::DEBUG::2013-11-20 03:46:27,264::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914978::DEBUG::2013-11-20 03:46:27,265::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914978::DEBUG::2013-11-20 03:46:27,267::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914978::DEBUG::2013-11-20 03:46:27,268::task::495::TaskManager.Task::(_debug) Task ba5e037e-3812-4e94-ac52-15827c2426db: ref 0 aborting False
>Thread-15914978::INFO::2013-11-20 03:46:27,271::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:46:27,302::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0825284 s, 12.4 MB/s\n'; <rc> = 0
>Thread-15914454::DEBUG::2013-11-20 03:46:29,051::guestIF::97::vm.Vm::(_handleMessage) vmId=`163a3426-5ec5-46a5-b8f1-af06fc5d7e7d`::u'None'
>Dummy-15912651::DEBUG::2013-11-20 03:46:29,318::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:46:29,875::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0763658 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15914980::INFO::2013-11-20 03:46:31,380::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914980::DEBUG::2013-11-20 03:46:31,381::task::495::TaskManager.Task::(_debug) Task 061e2fd8-e1b5-4e86-bb5e-23b88894c3a9: moving from state init -> state preparing
>Thread-15914980::DEBUG::2013-11-20 03:46:31,382::task::495::TaskManager.Task::(_debug) Task 061e2fd8-e1b5-4e86-bb5e-23b88894c3a9: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000370025634766', 'lastCheck': 1384919187.886755, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00169801712036', 'lastCheck': 1384919181.751282, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00138592720032', 'lastCheck': 1384919187.8893909, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00620818138123', 'lastCheck': 1384919181.7457111, 'valid': True, 'code': 0}}
>Thread-15914980::DEBUG::2013-11-20 03:46:31,383::task::495::TaskManager.Task::(_debug) Task 061e2fd8-e1b5-4e86-bb5e-23b88894c3a9: moving from state preparing -> state finished
>Thread-15914980::DEBUG::2013-11-20 03:46:31,383::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914980::DEBUG::2013-11-20 03:46:31,384::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914980::DEBUG::2013-11-20 03:46:31,385::task::495::TaskManager.Task::(_debug) Task 061e2fd8-e1b5-4e86-bb5e-23b88894c3a9: ref 0 aborting False
>Thread-15914980::INFO::2013-11-20 03:46:31,386::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000370025634766', 'lastCheck': 1384919187.886755, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00169801712036', 'lastCheck': 1384919181.751282, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00138592720032', 'lastCheck': 1384919187.8893909, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00620818138123', 'lastCheck': 1384919181.7457111, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:46:31,896::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 03:46:32,105::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 03:46:32,106::task::495::TaskManager.Task::(_debug) Task 12532908-5d43-479b-ab92-31b3fa294b34: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 03:46:32,108::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`14323898-c9ac-408c-ac9d-9750b96a3ca9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 03:46:32,109::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 03:46:32,109::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 03:46:32,110::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`14323898-c9ac-408c-ac9d-9750b96a3ca9`::Granted request
>Thread-15914435::DEBUG::2013-11-20 03:46:32,111::task::495::TaskManager.Task::(_debug) Task 12532908-5d43-479b-ab92-31b3fa294b34: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 03:46:32,111::task::495::TaskManager.Task::(_debug) Task 12532908-5d43-479b-ab92-31b3fa294b34: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 03:46:32,112::task::495::TaskManager.Task::(_debug) Task 12532908-5d43-479b-ab92-31b3fa294b34: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 03:46:32,112::task::495::TaskManager.Task::(_debug) Task 12532908-5d43-479b-ab92-31b3fa294b34: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 03:46:32,113::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 03:46:32,113::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 03:46:32,113::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 03:46:32,114::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 03:46:32,114::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 03:46:32,114::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 03:46:32,115::task::495::TaskManager.Task::(_debug) Task 12532908-5d43-479b-ab92-31b3fa294b34: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 03:46:32,115::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 03:46:32,450::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0850098 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:46:34,466::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 03:46:34,743::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 03:46:34,751::task::495::TaskManager.Task::(_debug) Task 49584492-90d5-4ba5-9890-f58c242aaca3: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 03:46:34,755::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c3b5a1a5-7d28-4448-9493-31d39071babe`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 03:46:34,756::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 03:46:34,757::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 03:46:34,758::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c3b5a1a5-7d28-4448-9493-31d39071babe`::Granted request
>Thread-15914453::DEBUG::2013-11-20 03:46:34,759::task::495::TaskManager.Task::(_debug) Task 49584492-90d5-4ba5-9890-f58c242aaca3: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 03:46:34,760::task::495::TaskManager.Task::(_debug) Task 49584492-90d5-4ba5-9890-f58c242aaca3: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 03:46:34,760::task::495::TaskManager.Task::(_debug) Task 49584492-90d5-4ba5-9890-f58c242aaca3: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 03:46:34,761::task::495::TaskManager.Task::(_debug) Task 49584492-90d5-4ba5-9890-f58c242aaca3: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 03:46:34,762::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 03:46:34,762::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 03:46:34,763::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 03:46:34,764::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 03:46:34,764::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 03:46:34,765::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 03:46:34,766::task::495::TaskManager.Task::(_debug) Task 49584492-90d5-4ba5-9890-f58c242aaca3: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 03:46:34,767::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 03:46:35,063::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.088754 s, 11.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:46:37,077::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:46:37,627::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0819657 s, 12.5 MB/s\n'; <rc> = 0
>Thread-15914983::DEBUG::2013-11-20 03:46:37,738::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914983::INFO::2013-11-20 03:46:37,749::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914983::DEBUG::2013-11-20 03:46:37,752::task::495::TaskManager.Task::(_debug) Task dd10a925-cbe7-4702-bf1f-047d792c0a28: moving from state init -> state preparing
>Thread-15914983::DEBUG::2013-11-20 03:46:37,762::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914983::DEBUG::2013-11-20 03:46:37,763::task::495::TaskManager.Task::(_debug) Task dd10a925-cbe7-4702-bf1f-047d792c0a28: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914983::DEBUG::2013-11-20 03:46:37,764::task::495::TaskManager.Task::(_debug) Task dd10a925-cbe7-4702-bf1f-047d792c0a28: moving from state preparing -> state finished
>Thread-15914983::DEBUG::2013-11-20 03:46:37,765::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914983::DEBUG::2013-11-20 03:46:37,766::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914983::DEBUG::2013-11-20 03:46:37,767::task::495::TaskManager.Task::(_debug) Task dd10a925-cbe7-4702-bf1f-047d792c0a28: ref 0 aborting False
>Thread-15914983::INFO::2013-11-20 03:46:37,767::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914984::DEBUG::2013-11-20 03:46:37,785::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914984::INFO::2013-11-20 03:46:37,786::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914984::DEBUG::2013-11-20 03:46:37,787::task::495::TaskManager.Task::(_debug) Task d4e5c225-fd2f-4620-b3e3-1934ec2d1ba1: moving from state init -> state preparing
>Thread-15914984::DEBUG::2013-11-20 03:46:37,788::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bb8be66f-88f5-4493-b845-060647d7e81e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914984::DEBUG::2013-11-20 03:46:37,789::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914984::DEBUG::2013-11-20 03:46:37,790::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914984::DEBUG::2013-11-20 03:46:37,790::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bb8be66f-88f5-4493-b845-060647d7e81e`::Granted request
>Thread-15914984::DEBUG::2013-11-20 03:46:37,791::task::495::TaskManager.Task::(_debug) Task d4e5c225-fd2f-4620-b3e3-1934ec2d1ba1: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914984::DEBUG::2013-11-20 03:46:37,792::task::495::TaskManager.Task::(_debug) Task d4e5c225-fd2f-4620-b3e3-1934ec2d1ba1: ref 1 aborting False
>Thread-15914984::WARNING::2013-11-20 03:46:37,794::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914984::DEBUG::2013-11-20 03:46:37,795::task::495::TaskManager.Task::(_debug) Task d4e5c225-fd2f-4620-b3e3-1934ec2d1ba1: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914984::DEBUG::2013-11-20 03:46:37,796::task::495::TaskManager.Task::(_debug) Task d4e5c225-fd2f-4620-b3e3-1934ec2d1ba1: moving from state preparing -> state finished
>Thread-15914984::DEBUG::2013-11-20 03:46:37,797::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914984::DEBUG::2013-11-20 03:46:37,803::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914984::DEBUG::2013-11-20 03:46:37,803::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914984::DEBUG::2013-11-20 03:46:37,804::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914984::DEBUG::2013-11-20 03:46:37,805::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914984::DEBUG::2013-11-20 03:46:37,806::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914984::DEBUG::2013-11-20 03:46:37,807::task::495::TaskManager.Task::(_debug) Task d4e5c225-fd2f-4620-b3e3-1934ec2d1ba1: ref 0 aborting False
>Thread-15914984::INFO::2013-11-20 03:46:37,808::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:46:39,643::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:46:40,175::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0881151 s, 11.6 MB/s\n'; <rc> = 0
>Thread-15914519::INFO::2013-11-20 03:46:40,732::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 03:46:40,734::task::495::TaskManager.Task::(_debug) Task c0908852-a8d3-49dc-bca4-f65e0e3f852f: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 03:46:40,746::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`eaaa3539-3717-449c-97ec-ab115ac3ce7b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 03:46:40,754::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 03:46:40,763::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 03:46:40,767::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`eaaa3539-3717-449c-97ec-ab115ac3ce7b`::Granted request
>Thread-15914519::DEBUG::2013-11-20 03:46:40,769::task::495::TaskManager.Task::(_debug) Task c0908852-a8d3-49dc-bca4-f65e0e3f852f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 03:46:40,770::task::495::TaskManager.Task::(_debug) Task c0908852-a8d3-49dc-bca4-f65e0e3f852f: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 03:46:40,771::task::495::TaskManager.Task::(_debug) Task c0908852-a8d3-49dc-bca4-f65e0e3f852f: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 03:46:40,772::task::495::TaskManager.Task::(_debug) Task c0908852-a8d3-49dc-bca4-f65e0e3f852f: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 03:46:40,773::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 03:46:40,773::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 03:46:40,774::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 03:46:40,775::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 03:46:40,776::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 03:46:40,776::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 03:46:40,777::task::495::TaskManager.Task::(_debug) Task c0908852-a8d3-49dc-bca4-f65e0e3f852f: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 03:46:40,778::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 03:46:42,191::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:46:42,715::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0777866 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:46:44,729::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 03:46:44,979::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:46:44,982::task::495::TaskManager.Task::(_debug) Task a608205a-949d-4d7b-bafc-04078a2106b8: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:46:44,985::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`9eb09c09-5bec-4d69-92a9-9cd8f1455685`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:46:44,986::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:46:44,987::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:46:44,988::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`9eb09c09-5bec-4d69-92a9-9cd8f1455685`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:46:44,990::task::495::TaskManager.Task::(_debug) Task a608205a-949d-4d7b-bafc-04078a2106b8: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:46:44,990::task::495::TaskManager.Task::(_debug) Task a608205a-949d-4d7b-bafc-04078a2106b8: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:46:44,992::task::495::TaskManager.Task::(_debug) Task a608205a-949d-4d7b-bafc-04078a2106b8: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:46:44,992::task::495::TaskManager.Task::(_debug) Task a608205a-949d-4d7b-bafc-04078a2106b8: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:46:44,993::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:46:44,994::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:46:44,995::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:46:44,996::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:46:44,997::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:46:44,997::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:46:44,998::task::495::TaskManager.Task::(_debug) Task a608205a-949d-4d7b-bafc-04078a2106b8: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:46:44,999::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:46:45,000::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:46:45,001::task::495::TaskManager.Task::(_debug) Task b8d2d91f-3679-43f1-b565-34f14da1a734: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:46:45,002::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e99a98d5-6ee9-4190-924a-0a41290a82d6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:46:45,003::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:46:45,004::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:46:45,004::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e99a98d5-6ee9-4190-924a-0a41290a82d6`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:46:45,005::task::495::TaskManager.Task::(_debug) Task b8d2d91f-3679-43f1-b565-34f14da1a734: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:46:45,006::task::495::TaskManager.Task::(_debug) Task b8d2d91f-3679-43f1-b565-34f14da1a734: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:46:45,006::task::495::TaskManager.Task::(_debug) Task b8d2d91f-3679-43f1-b565-34f14da1a734: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:46:45,007::task::495::TaskManager.Task::(_debug) Task b8d2d91f-3679-43f1-b565-34f14da1a734: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:46:45,008::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:46:45,009::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:46:45,010::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:46:45,010::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:46:45,011::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:46:45,012::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:46:45,013::task::495::TaskManager.Task::(_debug) Task b8d2d91f-3679-43f1-b565-34f14da1a734: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:46:45,013::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:46:45,014::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:46:45,015::task::495::TaskManager.Task::(_debug) Task 1afb152a-cb9d-4f2a-b8a0-d12c898ccd9c: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:46:45,016::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`dc7ea801-5de9-4dc9-8ea5-e49db123df8a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:46:45,017::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:46:45,018::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:46:45,018::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`dc7ea801-5de9-4dc9-8ea5-e49db123df8a`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:46:45,019::task::495::TaskManager.Task::(_debug) Task 1afb152a-cb9d-4f2a-b8a0-d12c898ccd9c: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:46:45,020::task::495::TaskManager.Task::(_debug) Task 1afb152a-cb9d-4f2a-b8a0-d12c898ccd9c: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:46:45,020::task::495::TaskManager.Task::(_debug) Task 1afb152a-cb9d-4f2a-b8a0-d12c898ccd9c: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:46:45,021::task::495::TaskManager.Task::(_debug) Task 1afb152a-cb9d-4f2a-b8a0-d12c898ccd9c: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:46:45,022::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:46:45,022::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:46:45,023::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:46:45,023::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:46:45,024::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:46:45,024::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:46:45,025::task::495::TaskManager.Task::(_debug) Task 1afb152a-cb9d-4f2a-b8a0-d12c898ccd9c: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:46:45,025::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:46:45,026::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:46:45,027::task::495::TaskManager.Task::(_debug) Task 89ce529e-650e-47a5-bc97-66af36f641ce: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:46:45,028::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`0b3e0284-ffaa-42a9-ac83-32974c24b7ac`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:46:45,028::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:46:45,029::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:46:45,030::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`0b3e0284-ffaa-42a9-ac83-32974c24b7ac`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:46:45,030::task::495::TaskManager.Task::(_debug) Task 89ce529e-650e-47a5-bc97-66af36f641ce: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:46:45,031::task::495::TaskManager.Task::(_debug) Task 89ce529e-650e-47a5-bc97-66af36f641ce: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:46:45,032::task::495::TaskManager.Task::(_debug) Task 89ce529e-650e-47a5-bc97-66af36f641ce: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:46:45,032::task::495::TaskManager.Task::(_debug) Task 89ce529e-650e-47a5-bc97-66af36f641ce: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:46:45,033::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:46:45,033::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:46:45,034::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:46:45,034::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:46:45,035::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:46:45,035::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:46:45,036::task::495::TaskManager.Task::(_debug) Task 89ce529e-650e-47a5-bc97-66af36f641ce: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:46:45,036::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:46:45,327::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0873727 s, 11.7 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 03:46:46,548::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:46:46,549::task::495::TaskManager.Task::(_debug) Task 36076cdc-4382-4911-9713-4deaf0b2486a: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:46:46,551::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e92df6e2-4c26-4ebd-8c67-8a425139bf26`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:46:46,552::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:46:46,553::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:46:46,554::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e92df6e2-4c26-4ebd-8c67-8a425139bf26`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:46:46,555::task::495::TaskManager.Task::(_debug) Task 36076cdc-4382-4911-9713-4deaf0b2486a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:46:46,556::task::495::TaskManager.Task::(_debug) Task 36076cdc-4382-4911-9713-4deaf0b2486a: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:46:46,557::task::495::TaskManager.Task::(_debug) Task 36076cdc-4382-4911-9713-4deaf0b2486a: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:46:46,558::task::495::TaskManager.Task::(_debug) Task 36076cdc-4382-4911-9713-4deaf0b2486a: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:46:46,559::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:46:46,559::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:46:46,561::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:46:46,562::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:46:46,562::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:46:46,563::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:46:46,564::task::495::TaskManager.Task::(_debug) Task 36076cdc-4382-4911-9713-4deaf0b2486a: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:46:46,565::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:46:46,567::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:46:46,567::task::495::TaskManager.Task::(_debug) Task 0c95df08-127c-4592-a0b4-7071c0d7dc1f: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:46:46,568::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8fd184e4-8987-4aed-9187-f0865b6fc743`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:46:46,569::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:46:46,570::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:46:46,570::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8fd184e4-8987-4aed-9187-f0865b6fc743`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:46:46,571::task::495::TaskManager.Task::(_debug) Task 0c95df08-127c-4592-a0b4-7071c0d7dc1f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:46:46,572::task::495::TaskManager.Task::(_debug) Task 0c95df08-127c-4592-a0b4-7071c0d7dc1f: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:46:46,572::task::495::TaskManager.Task::(_debug) Task 0c95df08-127c-4592-a0b4-7071c0d7dc1f: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:46:46,573::task::495::TaskManager.Task::(_debug) Task 0c95df08-127c-4592-a0b4-7071c0d7dc1f: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:46:46,573::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:46:46,574::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:46:46,575::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:46:46,575::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:46:46,576::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:46:46,576::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:46:46,577::task::495::TaskManager.Task::(_debug) Task 0c95df08-127c-4592-a0b4-7071c0d7dc1f: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:46:46,577::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:46:47,340::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914988::INFO::2013-11-20 03:46:47,537::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914988::DEBUG::2013-11-20 03:46:47,548::task::495::TaskManager.Task::(_debug) Task 5e1738a4-3299-407a-963c-a284fd8f9b59: moving from state init -> state preparing
>Thread-15914988::DEBUG::2013-11-20 03:46:47,550::task::495::TaskManager.Task::(_debug) Task 5e1738a4-3299-407a-963c-a284fd8f9b59: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000514984130859', 'lastCheck': 1384919197.8995709, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00149202346802', 'lastCheck': 1384919201.904825, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0267097949982', 'lastCheck': 1384919197.9302609, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00144600868225', 'lastCheck': 1384919201.902971, 'valid': True, 'code': 0}}
>Thread-15914988::DEBUG::2013-11-20 03:46:47,552::task::495::TaskManager.Task::(_debug) Task 5e1738a4-3299-407a-963c-a284fd8f9b59: moving from state preparing -> state finished
>Thread-15914988::DEBUG::2013-11-20 03:46:47,554::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914988::DEBUG::2013-11-20 03:46:47,555::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914988::DEBUG::2013-11-20 03:46:47,556::task::495::TaskManager.Task::(_debug) Task 5e1738a4-3299-407a-963c-a284fd8f9b59: ref 0 aborting False
>Thread-15914988::INFO::2013-11-20 03:46:47,557::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000514984130859', 'lastCheck': 1384919197.8995709, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00149202346802', 'lastCheck': 1384919201.904825, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0267097949982', 'lastCheck': 1384919197.9302609, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00144600868225', 'lastCheck': 1384919201.902971, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:46:47,846::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0776612 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15914990::DEBUG::2013-11-20 03:46:48,208::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914990::INFO::2013-11-20 03:46:48,209::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914990::DEBUG::2013-11-20 03:46:48,210::task::495::TaskManager.Task::(_debug) Task 976e9713-b36f-4497-9716-0558e8db8e6c: moving from state init -> state preparing
>Thread-15914990::DEBUG::2013-11-20 03:46:48,211::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914990::DEBUG::2013-11-20 03:46:48,211::task::495::TaskManager.Task::(_debug) Task 976e9713-b36f-4497-9716-0558e8db8e6c: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914990::DEBUG::2013-11-20 03:46:48,212::task::495::TaskManager.Task::(_debug) Task 976e9713-b36f-4497-9716-0558e8db8e6c: moving from state preparing -> state finished
>Thread-15914990::DEBUG::2013-11-20 03:46:48,213::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914990::DEBUG::2013-11-20 03:46:48,214::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914990::DEBUG::2013-11-20 03:46:48,214::task::495::TaskManager.Task::(_debug) Task 976e9713-b36f-4497-9716-0558e8db8e6c: ref 0 aborting False
>Thread-15914990::INFO::2013-11-20 03:46:48,215::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914991::DEBUG::2013-11-20 03:46:48,220::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914991::INFO::2013-11-20 03:46:48,221::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914991::DEBUG::2013-11-20 03:46:48,222::task::495::TaskManager.Task::(_debug) Task 10511320-0d94-4536-a048-96e682c17f43: moving from state init -> state preparing
>Thread-15914991::DEBUG::2013-11-20 03:46:48,223::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6db5032e-02f3-4430-874e-742b60bd9043`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914991::DEBUG::2013-11-20 03:46:48,224::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914991::DEBUG::2013-11-20 03:46:48,225::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914991::DEBUG::2013-11-20 03:46:48,226::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6db5032e-02f3-4430-874e-742b60bd9043`::Granted request
>Thread-15914991::DEBUG::2013-11-20 03:46:48,227::task::495::TaskManager.Task::(_debug) Task 10511320-0d94-4536-a048-96e682c17f43: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914991::DEBUG::2013-11-20 03:46:48,227::task::495::TaskManager.Task::(_debug) Task 10511320-0d94-4536-a048-96e682c17f43: ref 1 aborting False
>Thread-15914991::WARNING::2013-11-20 03:46:48,230::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914991::DEBUG::2013-11-20 03:46:48,231::task::495::TaskManager.Task::(_debug) Task 10511320-0d94-4536-a048-96e682c17f43: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914991::DEBUG::2013-11-20 03:46:48,232::task::495::TaskManager.Task::(_debug) Task 10511320-0d94-4536-a048-96e682c17f43: moving from state preparing -> state finished
>Thread-15914991::DEBUG::2013-11-20 03:46:48,233::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914991::DEBUG::2013-11-20 03:46:48,233::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914991::DEBUG::2013-11-20 03:46:48,234::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914991::DEBUG::2013-11-20 03:46:48,235::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914991::DEBUG::2013-11-20 03:46:48,236::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914991::DEBUG::2013-11-20 03:46:48,236::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914991::DEBUG::2013-11-20 03:46:48,243::task::495::TaskManager.Task::(_debug) Task 10511320-0d94-4536-a048-96e682c17f43: ref 0 aborting False
>Thread-15914991::INFO::2013-11-20 03:46:48,244::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:46:49,860::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:46:50,384::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0866059 s, 11.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:46:52,402::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:46:52,970::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0758458 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:46:54,985::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:46:55,516::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.104899 s, 9.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:46:57,530::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:46:58,086::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0915995 s, 11.2 MB/s\n'; <rc> = 0
>Thread-15914995::DEBUG::2013-11-20 03:46:59,190::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914995::INFO::2013-11-20 03:46:59,191::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914995::DEBUG::2013-11-20 03:46:59,192::task::495::TaskManager.Task::(_debug) Task b5aad7d3-5f00-471b-ac02-e62ff660bb8d: moving from state init -> state preparing
>Thread-15914995::DEBUG::2013-11-20 03:46:59,193::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15914995::DEBUG::2013-11-20 03:46:59,194::task::495::TaskManager.Task::(_debug) Task b5aad7d3-5f00-471b-ac02-e62ff660bb8d: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914995::DEBUG::2013-11-20 03:46:59,195::task::495::TaskManager.Task::(_debug) Task b5aad7d3-5f00-471b-ac02-e62ff660bb8d: moving from state preparing -> state finished
>Thread-15914995::DEBUG::2013-11-20 03:46:59,196::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914995::DEBUG::2013-11-20 03:46:59,196::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914995::DEBUG::2013-11-20 03:46:59,197::task::495::TaskManager.Task::(_debug) Task b5aad7d3-5f00-471b-ac02-e62ff660bb8d: ref 0 aborting False
>Thread-15914995::INFO::2013-11-20 03:46:59,198::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914996::DEBUG::2013-11-20 03:46:59,206::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914996::INFO::2013-11-20 03:46:59,208::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914996::DEBUG::2013-11-20 03:46:59,208::task::495::TaskManager.Task::(_debug) Task 981ba0ec-7c3d-42f6-bb5a-0f12998b4a45: moving from state init -> state preparing
>Thread-15914996::DEBUG::2013-11-20 03:46:59,210::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`10f09e48-9af4-4786-bd96-fb0e7c3f6684`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914996::DEBUG::2013-11-20 03:46:59,210::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914996::DEBUG::2013-11-20 03:46:59,211::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914996::DEBUG::2013-11-20 03:46:59,212::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`10f09e48-9af4-4786-bd96-fb0e7c3f6684`::Granted request
>Thread-15914996::DEBUG::2013-11-20 03:46:59,213::task::495::TaskManager.Task::(_debug) Task 981ba0ec-7c3d-42f6-bb5a-0f12998b4a45: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914996::DEBUG::2013-11-20 03:46:59,214::task::495::TaskManager.Task::(_debug) Task 981ba0ec-7c3d-42f6-bb5a-0f12998b4a45: ref 1 aborting False
>Thread-15914996::WARNING::2013-11-20 03:46:59,216::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914996::DEBUG::2013-11-20 03:46:59,217::task::495::TaskManager.Task::(_debug) Task 981ba0ec-7c3d-42f6-bb5a-0f12998b4a45: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914996::DEBUG::2013-11-20 03:46:59,218::task::495::TaskManager.Task::(_debug) Task 981ba0ec-7c3d-42f6-bb5a-0f12998b4a45: moving from state preparing -> state finished
>Thread-15914996::DEBUG::2013-11-20 03:46:59,219::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15914996::DEBUG::2013-11-20 03:46:59,219::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914996::DEBUG::2013-11-20 03:46:59,220::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914996::DEBUG::2013-11-20 03:46:59,221::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914996::DEBUG::2013-11-20 03:46:59,221::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914996::DEBUG::2013-11-20 03:46:59,222::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914996::DEBUG::2013-11-20 03:46:59,223::task::495::TaskManager.Task::(_debug) Task 981ba0ec-7c3d-42f6-bb5a-0f12998b4a45: ref 0 aborting False
>Thread-15914996::INFO::2013-11-20 03:46:59,223::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:47:00,097::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 03:47:00,411::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:47:00,413::task::495::TaskManager.Task::(_debug) Task b756b7a5-b60e-49dd-aa83-e03c2f2198b9: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:47:00,416::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9ea8ebbf-be7b-48dd-a301-447d4e3697c6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:47:00,418::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:47:00,420::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:47:00,424::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9ea8ebbf-be7b-48dd-a301-447d4e3697c6`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:47:00,426::task::495::TaskManager.Task::(_debug) Task b756b7a5-b60e-49dd-aa83-e03c2f2198b9: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:47:00,427::task::495::TaskManager.Task::(_debug) Task b756b7a5-b60e-49dd-aa83-e03c2f2198b9: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:47:00,428::task::495::TaskManager.Task::(_debug) Task b756b7a5-b60e-49dd-aa83-e03c2f2198b9: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:47:00,428::task::495::TaskManager.Task::(_debug) Task b756b7a5-b60e-49dd-aa83-e03c2f2198b9: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:47:00,429::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:47:00,430::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:47:00,430::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:47:00,431::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:47:00,432::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:47:00,432::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:47:00,433::task::495::TaskManager.Task::(_debug) Task b756b7a5-b60e-49dd-aa83-e03c2f2198b9: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:47:00,434::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:47:00,637::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0850583 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:47:02,654::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:47:03,211::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0771289 s, 13.3 MB/s\n'; <rc> = 0
>Thread-15914998::INFO::2013-11-20 03:47:03,770::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15914998::DEBUG::2013-11-20 03:47:03,772::task::495::TaskManager.Task::(_debug) Task 5b92a4c3-a41c-4109-ab6c-c5e3d973f608: moving from state init -> state preparing
>Thread-15914998::DEBUG::2013-11-20 03:47:03,773::task::495::TaskManager.Task::(_debug) Task 5b92a4c3-a41c-4109-ab6c-c5e3d973f608: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000535011291504', 'lastCheck': 1384919218.1597109, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00128293037415', 'lastCheck': 1384919222.2868631, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00156092643738', 'lastCheck': 1384919218.1601679, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000463962554932', 'lastCheck': 1384919222.280576, 'valid': True, 'code': 0}}
>Thread-15914998::DEBUG::2013-11-20 03:47:03,774::task::495::TaskManager.Task::(_debug) Task 5b92a4c3-a41c-4109-ab6c-c5e3d973f608: moving from state preparing -> state finished
>Thread-15914998::DEBUG::2013-11-20 03:47:03,774::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914998::DEBUG::2013-11-20 03:47:03,775::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914998::DEBUG::2013-11-20 03:47:03,775::task::495::TaskManager.Task::(_debug) Task 5b92a4c3-a41c-4109-ab6c-c5e3d973f608: ref 0 aborting False
>Thread-15914998::INFO::2013-11-20 03:47:03,776::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000535011291504', 'lastCheck': 1384919218.1597109, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00128293037415', 'lastCheck': 1384919222.2868631, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00156092643738', 'lastCheck': 1384919218.1601679, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000463962554932', 'lastCheck': 1384919222.280576, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:47:05,225::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:47:05,790::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0910078 s, 11.3 MB/s\n'; <rc> = 0
>Thread-15913659::DEBUG::2013-11-20 03:47:07,663::guestIF::97::vm.Vm::(_handleMessage) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::u'None'
>Dummy-15912651::DEBUG::2013-11-20 03:47:07,807::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:47:08,372::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.083556 s, 12.3 MB/s\n'; <rc> = 0
>Thread-42838::DEBUG::2013-11-20 03:47:08,449::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 03:47:09,205::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15915001::DEBUG::2013-11-20 03:47:09,728::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915001::INFO::2013-11-20 03:47:09,730::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915001::DEBUG::2013-11-20 03:47:09,731::task::495::TaskManager.Task::(_debug) Task c4997ec9-3e28-4b9f-9114-c49c6c390a4b: moving from state init -> state preparing
>Thread-15915001::DEBUG::2013-11-20 03:47:09,731::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915001::DEBUG::2013-11-20 03:47:09,732::task::495::TaskManager.Task::(_debug) Task c4997ec9-3e28-4b9f-9114-c49c6c390a4b: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915001::DEBUG::2013-11-20 03:47:09,733::task::495::TaskManager.Task::(_debug) Task c4997ec9-3e28-4b9f-9114-c49c6c390a4b: moving from state preparing -> state finished
>Thread-15915001::DEBUG::2013-11-20 03:47:09,733::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915001::DEBUG::2013-11-20 03:47:09,734::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915001::DEBUG::2013-11-20 03:47:09,735::task::495::TaskManager.Task::(_debug) Task c4997ec9-3e28-4b9f-9114-c49c6c390a4b: ref 0 aborting False
>Thread-15915001::INFO::2013-11-20 03:47:09,735::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915002::DEBUG::2013-11-20 03:47:09,743::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915002::INFO::2013-11-20 03:47:09,744::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915002::DEBUG::2013-11-20 03:47:09,744::task::495::TaskManager.Task::(_debug) Task 9d3d7cb9-1de3-403a-a48b-15fe21ce3612: moving from state init -> state preparing
>Thread-15915002::DEBUG::2013-11-20 03:47:09,746::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e8eb749c-c900-4e83-86c3-859a0d3d44f0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915002::DEBUG::2013-11-20 03:47:09,746::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915002::DEBUG::2013-11-20 03:47:09,747::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915002::DEBUG::2013-11-20 03:47:09,748::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e8eb749c-c900-4e83-86c3-859a0d3d44f0`::Granted request
>Thread-15915002::DEBUG::2013-11-20 03:47:09,749::task::495::TaskManager.Task::(_debug) Task 9d3d7cb9-1de3-403a-a48b-15fe21ce3612: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915002::DEBUG::2013-11-20 03:47:09,750::task::495::TaskManager.Task::(_debug) Task 9d3d7cb9-1de3-403a-a48b-15fe21ce3612: ref 1 aborting False
>Thread-15915002::WARNING::2013-11-20 03:47:09,752::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915002::DEBUG::2013-11-20 03:47:09,753::task::495::TaskManager.Task::(_debug) Task 9d3d7cb9-1de3-403a-a48b-15fe21ce3612: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915002::DEBUG::2013-11-20 03:47:09,754::task::495::TaskManager.Task::(_debug) Task 9d3d7cb9-1de3-403a-a48b-15fe21ce3612: moving from state preparing -> state finished
>Thread-15915002::DEBUG::2013-11-20 03:47:09,754::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915002::DEBUG::2013-11-20 03:47:09,755::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915002::DEBUG::2013-11-20 03:47:09,756::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915002::DEBUG::2013-11-20 03:47:09,756::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915002::DEBUG::2013-11-20 03:47:09,757::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915002::DEBUG::2013-11-20 03:47:09,757::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915002::DEBUG::2013-11-20 03:47:09,758::task::495::TaskManager.Task::(_debug) Task 9d3d7cb9-1de3-403a-a48b-15fe21ce3612: ref 0 aborting False
>Thread-15915002::INFO::2013-11-20 03:47:09,759::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:47:10,387::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:47:10,975::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0814039 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:47:12,996::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:47:13,553::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0788872 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 03:47:13,770::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:47:13,774::task::495::TaskManager.Task::(_debug) Task f5f1b67f-5642-4c35-8c95-d49017638520: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:47:13,779::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`58f5c88f-052c-4f30-bdec-65d0b10a66d3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:47:13,780::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:47:13,781::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:47:13,782::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`58f5c88f-052c-4f30-bdec-65d0b10a66d3`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:47:13,783::task::495::TaskManager.Task::(_debug) Task f5f1b67f-5642-4c35-8c95-d49017638520: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:47:13,784::task::495::TaskManager.Task::(_debug) Task f5f1b67f-5642-4c35-8c95-d49017638520: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:47:13,784::task::495::TaskManager.Task::(_debug) Task f5f1b67f-5642-4c35-8c95-d49017638520: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:47:13,785::task::495::TaskManager.Task::(_debug) Task f5f1b67f-5642-4c35-8c95-d49017638520: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:47:13,785::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:47:13,786::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:47:13,787::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:47:13,787::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:47:13,788::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:47:13,788::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:47:13,789::task::495::TaskManager.Task::(_debug) Task f5f1b67f-5642-4c35-8c95-d49017638520: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:47:13,790::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:47:15,568::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:47:16,054::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0597926 s, 17.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:47:18,072::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:47:18,570::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0731063 s, 14.0 MB/s\n'; <rc> = 0
>Thread-15915006::INFO::2013-11-20 03:47:19,691::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915006::DEBUG::2013-11-20 03:47:19,692::task::495::TaskManager.Task::(_debug) Task 36124ade-ba7c-4a52-b709-82fcb620e56f: moving from state init -> state preparing
>Thread-15915006::DEBUG::2013-11-20 03:47:19,692::task::495::TaskManager.Task::(_debug) Task 36124ade-ba7c-4a52-b709-82fcb620e56f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000363826751709', 'lastCheck': 1384919239.2587509, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0164239406586', 'lastCheck': 1384919232.6360371, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0345158576965', 'lastCheck': 1384919238.8181729, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000612020492554', 'lastCheck': 1384919232.6150219, 'valid': True, 'code': 0}}
>Thread-15915006::DEBUG::2013-11-20 03:47:19,693::task::495::TaskManager.Task::(_debug) Task 36124ade-ba7c-4a52-b709-82fcb620e56f: moving from state preparing -> state finished
>Thread-15915006::DEBUG::2013-11-20 03:47:19,699::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915006::DEBUG::2013-11-20 03:47:19,700::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915006::DEBUG::2013-11-20 03:47:19,701::task::495::TaskManager.Task::(_debug) Task 36124ade-ba7c-4a52-b709-82fcb620e56f: ref 0 aborting False
>Thread-15915006::INFO::2013-11-20 03:47:19,701::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000363826751709', 'lastCheck': 1384919239.2587509, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0164239406586', 'lastCheck': 1384919232.6360371, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0345158576965', 'lastCheck': 1384919238.8181729, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000612020492554', 'lastCheck': 1384919232.6150219, 'valid': True, 'code': 0}}
>Thread-15915008::DEBUG::2013-11-20 03:47:20,232::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915008::INFO::2013-11-20 03:47:20,233::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915008::DEBUG::2013-11-20 03:47:20,234::task::495::TaskManager.Task::(_debug) Task 0c2971e3-54de-4a5e-9f53-25abfbef76a1: moving from state init -> state preparing
>Thread-15915008::DEBUG::2013-11-20 03:47:20,235::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915008::DEBUG::2013-11-20 03:47:20,236::task::495::TaskManager.Task::(_debug) Task 0c2971e3-54de-4a5e-9f53-25abfbef76a1: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915008::DEBUG::2013-11-20 03:47:20,236::task::495::TaskManager.Task::(_debug) Task 0c2971e3-54de-4a5e-9f53-25abfbef76a1: moving from state preparing -> state finished
>Thread-15915008::DEBUG::2013-11-20 03:47:20,237::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915008::DEBUG::2013-11-20 03:47:20,238::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915008::DEBUG::2013-11-20 03:47:20,238::task::495::TaskManager.Task::(_debug) Task 0c2971e3-54de-4a5e-9f53-25abfbef76a1: ref 0 aborting False
>Thread-15915008::INFO::2013-11-20 03:47:20,239::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915009::DEBUG::2013-11-20 03:47:20,243::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915009::INFO::2013-11-20 03:47:20,244::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915009::DEBUG::2013-11-20 03:47:20,245::task::495::TaskManager.Task::(_debug) Task b6a3ecf5-115d-47c3-9be1-4b0e4c7cb6fe: moving from state init -> state preparing
>Thread-15915009::DEBUG::2013-11-20 03:47:20,246::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`cc266f55-66df-415e-90d7-9ff62c97da68`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915009::DEBUG::2013-11-20 03:47:20,247::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915009::DEBUG::2013-11-20 03:47:20,248::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915009::DEBUG::2013-11-20 03:47:20,248::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`cc266f55-66df-415e-90d7-9ff62c97da68`::Granted request
>Thread-15915009::DEBUG::2013-11-20 03:47:20,249::task::495::TaskManager.Task::(_debug) Task b6a3ecf5-115d-47c3-9be1-4b0e4c7cb6fe: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915009::DEBUG::2013-11-20 03:47:20,250::task::495::TaskManager.Task::(_debug) Task b6a3ecf5-115d-47c3-9be1-4b0e4c7cb6fe: ref 1 aborting False
>Thread-15915009::WARNING::2013-11-20 03:47:20,252::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915009::DEBUG::2013-11-20 03:47:20,252::task::495::TaskManager.Task::(_debug) Task b6a3ecf5-115d-47c3-9be1-4b0e4c7cb6fe: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915009::DEBUG::2013-11-20 03:47:20,253::task::495::TaskManager.Task::(_debug) Task b6a3ecf5-115d-47c3-9be1-4b0e4c7cb6fe: moving from state preparing -> state finished
>Thread-15915009::DEBUG::2013-11-20 03:47:20,254::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915009::DEBUG::2013-11-20 03:47:20,254::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915009::DEBUG::2013-11-20 03:47:20,255::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915009::DEBUG::2013-11-20 03:47:20,256::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915009::DEBUG::2013-11-20 03:47:20,256::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915009::DEBUG::2013-11-20 03:47:20,257::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915009::DEBUG::2013-11-20 03:47:20,258::task::495::TaskManager.Task::(_debug) Task b6a3ecf5-115d-47c3-9be1-4b0e4c7cb6fe: ref 0 aborting False
>Thread-15915009::INFO::2013-11-20 03:47:20,258::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:47:20,585::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:47:21,141::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0801043 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:47:23,156::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 03:47:23,373::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:47:23,384::task::495::TaskManager.Task::(_debug) Task edb57727-b479-4fea-9ea3-a655142aa3a3: moving from state init -> state preparing
>Thread-15913695::INFO::2013-11-20 03:47:23,388::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:47:23,389::task::495::TaskManager.Task::(_debug) Task 9f276531-10b0-495e-9ac3-febabaa6f852: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:47:23,396::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8b3dd423-7bc9-4d0b-be93-610dc584c652`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:47:23,397::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fd00fe05-9cea-45d6-9071-912ceffba820`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:47:23,397::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:47:23,398::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:47:23,400::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:47:23,401::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fd00fe05-9cea-45d6-9071-912ceffba820`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:47:23,402::resourceManager::481::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' found in shared state and queue is empty, Joining current shared lock (2 active users)
>Thread-15913695::DEBUG::2013-11-20 03:47:23,403::task::495::TaskManager.Task::(_debug) Task 9f276531-10b0-495e-9ac3-febabaa6f852: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:47:23,404::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8b3dd423-7bc9-4d0b-be93-610dc584c652`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:47:23,404::task::495::TaskManager.Task::(_debug) Task 9f276531-10b0-495e-9ac3-febabaa6f852: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:47:23,406::task::495::TaskManager.Task::(_debug) Task edb57727-b479-4fea-9ea3-a655142aa3a3: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:47:23,407::task::495::TaskManager.Task::(_debug) Task 9f276531-10b0-495e-9ac3-febabaa6f852: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15912392::DEBUG::2013-11-20 03:47:23,409::task::495::TaskManager.Task::(_debug) Task edb57727-b479-4fea-9ea3-a655142aa3a3: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:47:23,409::task::495::TaskManager.Task::(_debug) Task 9f276531-10b0-495e-9ac3-febabaa6f852: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:47:23,410::task::495::TaskManager.Task::(_debug) Task edb57727-b479-4fea-9ea3-a655142aa3a3: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913695::DEBUG::2013-11-20 03:47:23,411::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:47:23,412::task::495::TaskManager.Task::(_debug) Task edb57727-b479-4fea-9ea3-a655142aa3a3: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:47:23,412::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:47:23,413::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:47:23,414::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:47:23,415::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:47:23,416::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (1 active users)
>Thread-15912392::DEBUG::2013-11-20 03:47:23,417::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:47:23,417::task::495::TaskManager.Task::(_debug) Task 9f276531-10b0-495e-9ac3-febabaa6f852: ref 0 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:47:23,418::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::INFO::2013-11-20 03:47:23,419::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15912392::DEBUG::2013-11-20 03:47:23,420::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:47:23,421::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:47:23,421::task::495::TaskManager.Task::(_debug) Task edb57727-b479-4fea-9ea3-a655142aa3a3: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:47:23,422::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:47:23,715::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0818804 s, 12.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:47:25,730::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:47:26,300::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.086848 s, 11.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:47:28,316::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:47:28,898::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0870597 s, 11.8 MB/s\n'; <rc> = 0
>Thread-15915013::DEBUG::2013-11-20 03:47:30,636::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915013::INFO::2013-11-20 03:47:30,638::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915013::DEBUG::2013-11-20 03:47:30,639::task::495::TaskManager.Task::(_debug) Task 6a918e06-ab17-430a-a138-721034a30cf9: moving from state init -> state preparing
>Thread-15915013::DEBUG::2013-11-20 03:47:30,640::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915013::DEBUG::2013-11-20 03:47:30,640::task::495::TaskManager.Task::(_debug) Task 6a918e06-ab17-430a-a138-721034a30cf9: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915013::DEBUG::2013-11-20 03:47:30,641::task::495::TaskManager.Task::(_debug) Task 6a918e06-ab17-430a-a138-721034a30cf9: moving from state preparing -> state finished
>Thread-15915013::DEBUG::2013-11-20 03:47:30,642::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915013::DEBUG::2013-11-20 03:47:30,643::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915013::DEBUG::2013-11-20 03:47:30,644::task::495::TaskManager.Task::(_debug) Task 6a918e06-ab17-430a-a138-721034a30cf9: ref 0 aborting False
>Thread-15915013::INFO::2013-11-20 03:47:30,644::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915014::DEBUG::2013-11-20 03:47:30,658::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915014::INFO::2013-11-20 03:47:30,659::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915014::DEBUG::2013-11-20 03:47:30,659::task::495::TaskManager.Task::(_debug) Task 6840836d-897a-440c-9c7f-f11a1208a783: moving from state init -> state preparing
>Thread-15915014::DEBUG::2013-11-20 03:47:30,661::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7473926c-2b27-4bef-bbfd-f2cdad0e2e18`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915014::DEBUG::2013-11-20 03:47:30,661::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915014::DEBUG::2013-11-20 03:47:30,662::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915014::DEBUG::2013-11-20 03:47:30,663::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7473926c-2b27-4bef-bbfd-f2cdad0e2e18`::Granted request
>Thread-15915014::DEBUG::2013-11-20 03:47:30,664::task::495::TaskManager.Task::(_debug) Task 6840836d-897a-440c-9c7f-f11a1208a783: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915014::DEBUG::2013-11-20 03:47:30,664::task::495::TaskManager.Task::(_debug) Task 6840836d-897a-440c-9c7f-f11a1208a783: ref 1 aborting False
>Thread-15915014::WARNING::2013-11-20 03:47:30,666::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915014::DEBUG::2013-11-20 03:47:30,667::task::495::TaskManager.Task::(_debug) Task 6840836d-897a-440c-9c7f-f11a1208a783: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915014::DEBUG::2013-11-20 03:47:30,668::task::495::TaskManager.Task::(_debug) Task 6840836d-897a-440c-9c7f-f11a1208a783: moving from state preparing -> state finished
>Thread-15915014::DEBUG::2013-11-20 03:47:30,669::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915014::DEBUG::2013-11-20 03:47:30,669::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915014::DEBUG::2013-11-20 03:47:30,670::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915014::DEBUG::2013-11-20 03:47:30,670::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915014::DEBUG::2013-11-20 03:47:30,671::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915014::DEBUG::2013-11-20 03:47:30,672::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915014::DEBUG::2013-11-20 03:47:30,672::task::495::TaskManager.Task::(_debug) Task 6840836d-897a-440c-9c7f-f11a1208a783: ref 0 aborting False
>Thread-15915014::INFO::2013-11-20 03:47:30,673::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:47:30,915::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:47:31,489::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0773664 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:47:33,504::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 03:47:33,692::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 03:47:33,693::task::495::TaskManager.Task::(_debug) Task c6eac1f4-6037-4deb-8206-9c91b84c9d24: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 03:47:33,695::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`63355be9-8772-408d-83d6-0f4db6329b25`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 03:47:33,695::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 03:47:33,696::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 03:47:33,696::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`63355be9-8772-408d-83d6-0f4db6329b25`::Granted request
>Thread-15914435::DEBUG::2013-11-20 03:47:33,697::task::495::TaskManager.Task::(_debug) Task c6eac1f4-6037-4deb-8206-9c91b84c9d24: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 03:47:33,698::task::495::TaskManager.Task::(_debug) Task c6eac1f4-6037-4deb-8206-9c91b84c9d24: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 03:47:33,698::task::495::TaskManager.Task::(_debug) Task c6eac1f4-6037-4deb-8206-9c91b84c9d24: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 03:47:33,699::task::495::TaskManager.Task::(_debug) Task c6eac1f4-6037-4deb-8206-9c91b84c9d24: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 03:47:33,699::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 03:47:33,699::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 03:47:33,700::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 03:47:33,700::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 03:47:33,701::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 03:47:33,701::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 03:47:33,701::task::495::TaskManager.Task::(_debug) Task c6eac1f4-6037-4deb-8206-9c91b84c9d24: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 03:47:33,702::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 03:47:33,998::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0843429 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15915016::INFO::2013-11-20 03:47:35,712::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915016::DEBUG::2013-11-20 03:47:35,713::task::495::TaskManager.Task::(_debug) Task 4dabed6e-2fd1-4fcb-a346-678528f51c1a: moving from state init -> state preparing
>Thread-15915016::DEBUG::2013-11-20 03:47:35,714::task::495::TaskManager.Task::(_debug) Task 4dabed6e-2fd1-4fcb-a346-678528f51c1a: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000383138656616', 'lastCheck': 1384919249.55246, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00189399719238', 'lastCheck': 1384919253.1431329, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00180697441101', 'lastCheck': 1384919248.9730239, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000555038452148', 'lastCheck': 1384919253.1374991, 'valid': True, 'code': 0}}
>Thread-15915016::DEBUG::2013-11-20 03:47:35,715::task::495::TaskManager.Task::(_debug) Task 4dabed6e-2fd1-4fcb-a346-678528f51c1a: moving from state preparing -> state finished
>Thread-15915016::DEBUG::2013-11-20 03:47:35,716::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915016::DEBUG::2013-11-20 03:47:35,717::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915016::DEBUG::2013-11-20 03:47:35,718::task::495::TaskManager.Task::(_debug) Task 4dabed6e-2fd1-4fcb-a346-678528f51c1a: ref 0 aborting False
>Thread-15915016::INFO::2013-11-20 03:47:35,718::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000383138656616', 'lastCheck': 1384919249.55246, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00189399719238', 'lastCheck': 1384919253.1431329, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00180697441101', 'lastCheck': 1384919248.9730239, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000555038452148', 'lastCheck': 1384919253.1374991, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:47:36,015::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 03:47:36,243::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 03:47:36,245::task::495::TaskManager.Task::(_debug) Task ba126a4f-4923-464a-9863-947bb7f08307: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 03:47:36,248::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b16fecfb-30b8-4858-b33d-94a3558a1c3e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 03:47:36,248::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 03:47:36,250::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 03:47:36,251::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b16fecfb-30b8-4858-b33d-94a3558a1c3e`::Granted request
>Thread-15914453::DEBUG::2013-11-20 03:47:36,252::task::495::TaskManager.Task::(_debug) Task ba126a4f-4923-464a-9863-947bb7f08307: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 03:47:36,253::task::495::TaskManager.Task::(_debug) Task ba126a4f-4923-464a-9863-947bb7f08307: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 03:47:36,254::task::495::TaskManager.Task::(_debug) Task ba126a4f-4923-464a-9863-947bb7f08307: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 03:47:36,255::task::495::TaskManager.Task::(_debug) Task ba126a4f-4923-464a-9863-947bb7f08307: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 03:47:36,256::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 03:47:36,256::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 03:47:36,257::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 03:47:36,258::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 03:47:36,259::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 03:47:36,259::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 03:47:36,260::task::495::TaskManager.Task::(_debug) Task ba126a4f-4923-464a-9863-947bb7f08307: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 03:47:36,261::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 03:47:36,560::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0753226 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:47:38,576::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:47:39,125::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0925404 s, 11.1 MB/s\n'; <rc> = 0
>Thread-15915019::DEBUG::2013-11-20 03:47:41,056::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915019::INFO::2013-11-20 03:47:41,058::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915019::DEBUG::2013-11-20 03:47:41,058::task::495::TaskManager.Task::(_debug) Task 13103505-c114-415b-b719-1f7444d10da7: moving from state init -> state preparing
>Thread-15915019::DEBUG::2013-11-20 03:47:41,059::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915019::DEBUG::2013-11-20 03:47:41,060::task::495::TaskManager.Task::(_debug) Task 13103505-c114-415b-b719-1f7444d10da7: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915019::DEBUG::2013-11-20 03:47:41,060::task::495::TaskManager.Task::(_debug) Task 13103505-c114-415b-b719-1f7444d10da7: moving from state preparing -> state finished
>Thread-15915019::DEBUG::2013-11-20 03:47:41,061::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915019::DEBUG::2013-11-20 03:47:41,061::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915019::DEBUG::2013-11-20 03:47:41,062::task::495::TaskManager.Task::(_debug) Task 13103505-c114-415b-b719-1f7444d10da7: ref 0 aborting False
>Thread-15915019::INFO::2013-11-20 03:47:41,063::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915020::DEBUG::2013-11-20 03:47:41,075::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915020::INFO::2013-11-20 03:47:41,076::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915020::DEBUG::2013-11-20 03:47:41,076::task::495::TaskManager.Task::(_debug) Task 49a2f10a-aa1d-406b-8ea1-20d5064e0fbc: moving from state init -> state preparing
>Thread-15915020::DEBUG::2013-11-20 03:47:41,078::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6c7b5044-556a-442a-9efb-909366eb962b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915020::DEBUG::2013-11-20 03:47:41,078::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915020::DEBUG::2013-11-20 03:47:41,080::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915020::DEBUG::2013-11-20 03:47:41,080::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6c7b5044-556a-442a-9efb-909366eb962b`::Granted request
>Thread-15915020::DEBUG::2013-11-20 03:47:41,082::task::495::TaskManager.Task::(_debug) Task 49a2f10a-aa1d-406b-8ea1-20d5064e0fbc: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915020::DEBUG::2013-11-20 03:47:41,083::task::495::TaskManager.Task::(_debug) Task 49a2f10a-aa1d-406b-8ea1-20d5064e0fbc: ref 1 aborting False
>Thread-15915020::WARNING::2013-11-20 03:47:41,085::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915020::DEBUG::2013-11-20 03:47:41,086::task::495::TaskManager.Task::(_debug) Task 49a2f10a-aa1d-406b-8ea1-20d5064e0fbc: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915020::DEBUG::2013-11-20 03:47:41,086::task::495::TaskManager.Task::(_debug) Task 49a2f10a-aa1d-406b-8ea1-20d5064e0fbc: moving from state preparing -> state finished
>Thread-15915020::DEBUG::2013-11-20 03:47:41,087::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915020::DEBUG::2013-11-20 03:47:41,088::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915020::DEBUG::2013-11-20 03:47:41,088::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915020::DEBUG::2013-11-20 03:47:41,089::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915020::DEBUG::2013-11-20 03:47:41,090::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915020::DEBUG::2013-11-20 03:47:41,090::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915020::DEBUG::2013-11-20 03:47:41,091::task::495::TaskManager.Task::(_debug) Task 49a2f10a-aa1d-406b-8ea1-20d5064e0fbc: ref 0 aborting False
>Thread-15915020::INFO::2013-11-20 03:47:41,092::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:47:41,140::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 03:47:41,375::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 03:47:41,377::task::495::TaskManager.Task::(_debug) Task ac8a7d40-2160-4bd7-a9f3-af7e725bead8: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 03:47:41,385::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bb251d17-34d7-4f65-a4c6-ca1bc32e6c8c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 03:47:41,386::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 03:47:41,387::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 03:47:41,388::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bb251d17-34d7-4f65-a4c6-ca1bc32e6c8c`::Granted request
>Thread-15914519::DEBUG::2013-11-20 03:47:41,390::task::495::TaskManager.Task::(_debug) Task ac8a7d40-2160-4bd7-a9f3-af7e725bead8: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 03:47:41,398::task::495::TaskManager.Task::(_debug) Task ac8a7d40-2160-4bd7-a9f3-af7e725bead8: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 03:47:41,400::task::495::TaskManager.Task::(_debug) Task ac8a7d40-2160-4bd7-a9f3-af7e725bead8: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 03:47:41,401::task::495::TaskManager.Task::(_debug) Task ac8a7d40-2160-4bd7-a9f3-af7e725bead8: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 03:47:41,402::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 03:47:41,402::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 03:47:41,407::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 03:47:41,408::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 03:47:41,409::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 03:47:41,409::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 03:47:41,411::task::495::TaskManager.Task::(_debug) Task ac8a7d40-2160-4bd7-a9f3-af7e725bead8: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 03:47:41,411::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 03:47:41,677::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0678372 s, 15.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:47:43,694::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:47:44,232::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0883394 s, 11.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:47:46,247::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 03:47:46,473::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:47:46,475::task::495::TaskManager.Task::(_debug) Task d5ea50b8-4036-45da-810a-427279db01cc: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:47:46,478::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`1b94d176-da54-4e9f-aefe-300f03a75335`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:47:46,478::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:47:46,479::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:47:46,480::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`1b94d176-da54-4e9f-aefe-300f03a75335`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:47:46,481::task::495::TaskManager.Task::(_debug) Task d5ea50b8-4036-45da-810a-427279db01cc: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:47:46,481::task::495::TaskManager.Task::(_debug) Task d5ea50b8-4036-45da-810a-427279db01cc: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:47:46,482::task::495::TaskManager.Task::(_debug) Task d5ea50b8-4036-45da-810a-427279db01cc: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:47:46,482::task::495::TaskManager.Task::(_debug) Task d5ea50b8-4036-45da-810a-427279db01cc: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:47:46,483::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:47:46,483::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:47:46,483::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:47:46,484::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:47:46,484::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:47:46,485::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:47:46,485::task::495::TaskManager.Task::(_debug) Task d5ea50b8-4036-45da-810a-427279db01cc: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:47:46,485::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:47:46,486::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:47:46,486::task::495::TaskManager.Task::(_debug) Task b2068dec-8249-4189-bfd9-2df7f9721086: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:47:46,487::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d25407be-4e16-40f2-9487-d571cfd84c14`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:47:46,487::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:47:46,488::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:47:46,488::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d25407be-4e16-40f2-9487-d571cfd84c14`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:47:46,489::task::495::TaskManager.Task::(_debug) Task b2068dec-8249-4189-bfd9-2df7f9721086: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:47:46,489::task::495::TaskManager.Task::(_debug) Task b2068dec-8249-4189-bfd9-2df7f9721086: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:47:46,490::task::495::TaskManager.Task::(_debug) Task b2068dec-8249-4189-bfd9-2df7f9721086: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:47:46,490::task::495::TaskManager.Task::(_debug) Task b2068dec-8249-4189-bfd9-2df7f9721086: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:47:46,490::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:47:46,491::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:47:46,491::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:47:46,491::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:47:46,492::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:47:46,492::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:47:46,493::task::495::TaskManager.Task::(_debug) Task b2068dec-8249-4189-bfd9-2df7f9721086: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:47:46,493::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:47:46,493::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:47:46,494::task::495::TaskManager.Task::(_debug) Task 9e6426f7-d4f8-4b84-b3c5-55d2e073ca01: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:47:46,494::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e909ffdc-1db5-4a62-866f-539ceaea6429`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:47:46,495::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:47:46,495::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:47:46,496::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e909ffdc-1db5-4a62-866f-539ceaea6429`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:47:46,496::task::495::TaskManager.Task::(_debug) Task 9e6426f7-d4f8-4b84-b3c5-55d2e073ca01: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:47:46,496::task::495::TaskManager.Task::(_debug) Task 9e6426f7-d4f8-4b84-b3c5-55d2e073ca01: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:47:46,497::task::495::TaskManager.Task::(_debug) Task 9e6426f7-d4f8-4b84-b3c5-55d2e073ca01: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:47:46,497::task::495::TaskManager.Task::(_debug) Task 9e6426f7-d4f8-4b84-b3c5-55d2e073ca01: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:47:46,498::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:47:46,498::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:47:46,498::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:47:46,499::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:47:46,499::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:47:46,499::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:47:46,500::task::495::TaskManager.Task::(_debug) Task 9e6426f7-d4f8-4b84-b3c5-55d2e073ca01: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:47:46,500::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:47:46,501::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:47:46,501::task::495::TaskManager.Task::(_debug) Task a7965931-5b09-4199-b797-298230a8cfe6: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:47:46,502::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`5ee063d8-9b78-4219-958c-628eaca47fc0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:47:46,502::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:47:46,502::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:47:46,503::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`5ee063d8-9b78-4219-958c-628eaca47fc0`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:47:46,503::task::495::TaskManager.Task::(_debug) Task a7965931-5b09-4199-b797-298230a8cfe6: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:47:46,504::task::495::TaskManager.Task::(_debug) Task a7965931-5b09-4199-b797-298230a8cfe6: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:47:46,504::task::495::TaskManager.Task::(_debug) Task a7965931-5b09-4199-b797-298230a8cfe6: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:47:46,504::task::495::TaskManager.Task::(_debug) Task a7965931-5b09-4199-b797-298230a8cfe6: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:47:46,505::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:47:46,505::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:47:46,505::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:47:46,506::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:47:46,506::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:47:46,506::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:47:46,507::task::495::TaskManager.Task::(_debug) Task a7965931-5b09-4199-b797-298230a8cfe6: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:47:46,507::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:47:46,741::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0839657 s, 12.2 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 03:47:47,946::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:47:47,947::task::495::TaskManager.Task::(_debug) Task b56ff1ea-7f9f-40d3-a2fd-872059194429: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:47:47,948::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b208736c-e67e-4d5d-bf47-c791b5f238c7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:47:47,949::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:47:47,950::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:47:47,950::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b208736c-e67e-4d5d-bf47-c791b5f238c7`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:47:47,951::task::495::TaskManager.Task::(_debug) Task b56ff1ea-7f9f-40d3-a2fd-872059194429: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:47:47,952::task::495::TaskManager.Task::(_debug) Task b56ff1ea-7f9f-40d3-a2fd-872059194429: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:47:47,953::task::495::TaskManager.Task::(_debug) Task b56ff1ea-7f9f-40d3-a2fd-872059194429: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:47:47,954::task::495::TaskManager.Task::(_debug) Task b56ff1ea-7f9f-40d3-a2fd-872059194429: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:47:47,954::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:47:47,955::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:47:47,956::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:47:47,957::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:47:47,958::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:47:47,958::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:47:47,959::task::495::TaskManager.Task::(_debug) Task b56ff1ea-7f9f-40d3-a2fd-872059194429: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:47:47,960::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:47:47,960::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:47:47,961::task::495::TaskManager.Task::(_debug) Task d8b96338-58fb-4ab8-a0f8-2c17566dbded: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:47:47,965::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ccc508ad-2509-409c-958b-3f8b6ee53681`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:47:47,966::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:47:47,967::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:47:47,967::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ccc508ad-2509-409c-958b-3f8b6ee53681`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:47:47,968::task::495::TaskManager.Task::(_debug) Task d8b96338-58fb-4ab8-a0f8-2c17566dbded: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:47:47,969::task::495::TaskManager.Task::(_debug) Task d8b96338-58fb-4ab8-a0f8-2c17566dbded: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:47:47,971::task::495::TaskManager.Task::(_debug) Task d8b96338-58fb-4ab8-a0f8-2c17566dbded: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:47:47,972::task::495::TaskManager.Task::(_debug) Task d8b96338-58fb-4ab8-a0f8-2c17566dbded: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:47:47,973::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:47:47,975::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:47:47,977::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:47:47,978::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:47:47,979::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:47:47,981::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:47:47,984::task::495::TaskManager.Task::(_debug) Task d8b96338-58fb-4ab8-a0f8-2c17566dbded: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:47:47,985::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:47:48,752::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:47:49,346::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0830222 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:47:51,360::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915024::DEBUG::2013-11-20 03:47:51,599::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915024::INFO::2013-11-20 03:47:51,602::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915024::DEBUG::2013-11-20 03:47:51,604::task::495::TaskManager.Task::(_debug) Task c25afda5-5eaf-4aed-a758-366fd2e03057: moving from state init -> state preparing
>Thread-15915024::DEBUG::2013-11-20 03:47:51,606::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915024::DEBUG::2013-11-20 03:47:51,607::task::495::TaskManager.Task::(_debug) Task c25afda5-5eaf-4aed-a758-366fd2e03057: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915024::DEBUG::2013-11-20 03:47:51,608::task::495::TaskManager.Task::(_debug) Task c25afda5-5eaf-4aed-a758-366fd2e03057: moving from state preparing -> state finished
>Thread-15915024::DEBUG::2013-11-20 03:47:51,609::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915024::DEBUG::2013-11-20 03:47:51,609::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915024::DEBUG::2013-11-20 03:47:51,610::task::495::TaskManager.Task::(_debug) Task c25afda5-5eaf-4aed-a758-366fd2e03057: ref 0 aborting False
>Thread-15915024::INFO::2013-11-20 03:47:51,611::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915025::DEBUG::2013-11-20 03:47:51,617::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915025::INFO::2013-11-20 03:47:51,618::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915025::DEBUG::2013-11-20 03:47:51,619::task::495::TaskManager.Task::(_debug) Task a8de1d45-f00e-4056-bb86-4698e7158f8b: moving from state init -> state preparing
>Thread-15915025::DEBUG::2013-11-20 03:47:51,621::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`024e2694-e51c-400c-804e-135f49d5069f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915025::DEBUG::2013-11-20 03:47:51,621::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915025::DEBUG::2013-11-20 03:47:51,623::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915025::DEBUG::2013-11-20 03:47:51,623::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`024e2694-e51c-400c-804e-135f49d5069f`::Granted request
>Thread-15915025::DEBUG::2013-11-20 03:47:51,625::task::495::TaskManager.Task::(_debug) Task a8de1d45-f00e-4056-bb86-4698e7158f8b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915025::DEBUG::2013-11-20 03:47:51,625::task::495::TaskManager.Task::(_debug) Task a8de1d45-f00e-4056-bb86-4698e7158f8b: ref 1 aborting False
>Thread-15915025::WARNING::2013-11-20 03:47:51,628::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915025::DEBUG::2013-11-20 03:47:51,629::task::495::TaskManager.Task::(_debug) Task a8de1d45-f00e-4056-bb86-4698e7158f8b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915025::DEBUG::2013-11-20 03:47:51,631::task::495::TaskManager.Task::(_debug) Task a8de1d45-f00e-4056-bb86-4698e7158f8b: moving from state preparing -> state finished
>Thread-15915025::DEBUG::2013-11-20 03:47:51,631::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915025::DEBUG::2013-11-20 03:47:51,632::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915025::DEBUG::2013-11-20 03:47:51,633::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915025::DEBUG::2013-11-20 03:47:51,634::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915025::DEBUG::2013-11-20 03:47:51,635::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915025::DEBUG::2013-11-20 03:47:51,635::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915025::DEBUG::2013-11-20 03:47:51,636::task::495::TaskManager.Task::(_debug) Task a8de1d45-f00e-4056-bb86-4698e7158f8b: ref 0 aborting False
>Thread-15915025::INFO::2013-11-20 03:47:51,637::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915026::INFO::2013-11-20 03:47:51,810::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915026::DEBUG::2013-11-20 03:47:51,811::task::495::TaskManager.Task::(_debug) Task 786f0711-9b2d-4d47-b330-adb73fb1ddcb: moving from state init -> state preparing
>Thread-15915026::DEBUG::2013-11-20 03:47:51,813::task::495::TaskManager.Task::(_debug) Task 786f0711-9b2d-4d47-b330-adb73fb1ddcb: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000483989715576', 'lastCheck': 1384919269.978915, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.000942945480347', 'lastCheck': 1384919263.3746381, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00145602226257', 'lastCheck': 1384919269.477864, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00138401985168', 'lastCheck': 1384919263.372719, 'valid': True, 'code': 0}}
>Thread-15915026::DEBUG::2013-11-20 03:47:51,814::task::495::TaskManager.Task::(_debug) Task 786f0711-9b2d-4d47-b330-adb73fb1ddcb: moving from state preparing -> state finished
>Thread-15915026::DEBUG::2013-11-20 03:47:51,815::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915026::DEBUG::2013-11-20 03:47:51,816::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915026::DEBUG::2013-11-20 03:47:51,817::task::495::TaskManager.Task::(_debug) Task 786f0711-9b2d-4d47-b330-adb73fb1ddcb: ref 0 aborting False
>Thread-15915026::INFO::2013-11-20 03:47:51,818::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000483989715576', 'lastCheck': 1384919269.978915, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.000942945480347', 'lastCheck': 1384919263.3746381, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00145602226257', 'lastCheck': 1384919269.477864, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00138401985168', 'lastCheck': 1384919263.372719, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:47:52,010::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.107018 s, 9.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:47:54,026::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:47:54,654::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0888398 s, 11.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:47:56,669::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:47:57,225::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0740345 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15913659::DEBUG::2013-11-20 03:47:57,748::guestIF::97::vm.Vm::(_handleMessage) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::u'AOS2Batch@brd-klee.dk'
>Thread-15913640::DEBUG::2013-11-20 03:47:58,993::guestIF::97::vm.Vm::(_handleMessage) vmId=`4760db3e-d3a8-4045-9153-12c36ec16ba2`::u'None'
>Dummy-15912651::DEBUG::2013-11-20 03:47:59,239::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:47:59,774::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0865142 s, 11.8 MB/s\n'; <rc> = 0
>Thread-15913639::INFO::2013-11-20 03:48:01,608::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:48:01,611::task::495::TaskManager.Task::(_debug) Task 5378c410-b21a-4bd3-8fb0-881c504a2c51: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:48:01,615::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6d17d975-156e-4863-b320-a0360c56b82c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:48:01,617::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:48:01,623::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:48:01,626::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6d17d975-156e-4863-b320-a0360c56b82c`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:48:01,637::task::495::TaskManager.Task::(_debug) Task 5378c410-b21a-4bd3-8fb0-881c504a2c51: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:48:01,639::task::495::TaskManager.Task::(_debug) Task 5378c410-b21a-4bd3-8fb0-881c504a2c51: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:48:01,639::task::495::TaskManager.Task::(_debug) Task 5378c410-b21a-4bd3-8fb0-881c504a2c51: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:48:01,640::task::495::TaskManager.Task::(_debug) Task 5378c410-b21a-4bd3-8fb0-881c504a2c51: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:48:01,641::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:48:01,641::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:48:01,642::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:48:01,643::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:48:01,643::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:48:01,644::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:48:01,645::task::495::TaskManager.Task::(_debug) Task 5378c410-b21a-4bd3-8fb0-881c504a2c51: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:48:01,645::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:48:01,789::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915031::DEBUG::2013-11-20 03:48:02,243::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915031::INFO::2013-11-20 03:48:02,244::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915031::DEBUG::2013-11-20 03:48:02,245::task::495::TaskManager.Task::(_debug) Task d174d820-4168-496d-9f42-40914e1327c2: moving from state init -> state preparing
>Thread-15915031::DEBUG::2013-11-20 03:48:02,246::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915031::DEBUG::2013-11-20 03:48:02,247::task::495::TaskManager.Task::(_debug) Task d174d820-4168-496d-9f42-40914e1327c2: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915031::DEBUG::2013-11-20 03:48:02,248::task::495::TaskManager.Task::(_debug) Task d174d820-4168-496d-9f42-40914e1327c2: moving from state preparing -> state finished
>Thread-15915031::DEBUG::2013-11-20 03:48:02,249::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915031::DEBUG::2013-11-20 03:48:02,250::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915031::DEBUG::2013-11-20 03:48:02,251::task::495::TaskManager.Task::(_debug) Task d174d820-4168-496d-9f42-40914e1327c2: ref 0 aborting False
>Thread-15915031::INFO::2013-11-20 03:48:02,252::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915032::DEBUG::2013-11-20 03:48:02,260::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915032::INFO::2013-11-20 03:48:02,262::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915032::DEBUG::2013-11-20 03:48:02,263::task::495::TaskManager.Task::(_debug) Task facb170e-c546-4484-98d6-fa81242d6bfc: moving from state init -> state preparing
>Thread-15915032::DEBUG::2013-11-20 03:48:02,265::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ed55757f-5b5b-4cf1-846c-70e1a52890ba`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915032::DEBUG::2013-11-20 03:48:02,266::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915032::DEBUG::2013-11-20 03:48:02,267::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915032::DEBUG::2013-11-20 03:48:02,268::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ed55757f-5b5b-4cf1-846c-70e1a52890ba`::Granted request
>Thread-15915032::DEBUG::2013-11-20 03:48:02,270::task::495::TaskManager.Task::(_debug) Task facb170e-c546-4484-98d6-fa81242d6bfc: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915032::DEBUG::2013-11-20 03:48:02,271::task::495::TaskManager.Task::(_debug) Task facb170e-c546-4484-98d6-fa81242d6bfc: ref 1 aborting False
>Thread-15915032::WARNING::2013-11-20 03:48:02,274::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915032::DEBUG::2013-11-20 03:48:02,275::task::495::TaskManager.Task::(_debug) Task facb170e-c546-4484-98d6-fa81242d6bfc: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915032::DEBUG::2013-11-20 03:48:02,276::task::495::TaskManager.Task::(_debug) Task facb170e-c546-4484-98d6-fa81242d6bfc: moving from state preparing -> state finished
>Thread-15915032::DEBUG::2013-11-20 03:48:02,277::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915032::DEBUG::2013-11-20 03:48:02,278::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915032::DEBUG::2013-11-20 03:48:02,279::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915032::DEBUG::2013-11-20 03:48:02,280::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915032::DEBUG::2013-11-20 03:48:02,281::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915032::DEBUG::2013-11-20 03:48:02,282::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915032::DEBUG::2013-11-20 03:48:02,283::task::495::TaskManager.Task::(_debug) Task facb170e-c546-4484-98d6-fa81242d6bfc: ref 0 aborting False
>Thread-15915032::INFO::2013-11-20 03:48:02,284::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:48:02,377::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.108228 s, 9.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:48:04,391::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:48:04,924::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0569394 s, 18.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:48:06,934::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:48:07,476::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0831336 s, 12.3 MB/s\n'; <rc> = 0
>Thread-15915034::INFO::2013-11-20 03:48:08,077::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915034::DEBUG::2013-11-20 03:48:08,078::task::495::TaskManager.Task::(_debug) Task 6a5a3235-e471-4fac-a1b4-125a7ac17bcf: moving from state init -> state preparing
>Thread-15915034::DEBUG::2013-11-20 03:48:08,079::task::495::TaskManager.Task::(_debug) Task 6a5a3235-e471-4fac-a1b4-125a7ac17bcf: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00050687789917', 'lastCheck': 1384919279.9945531, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.000903844833374', 'lastCheck': 1384919283.6069131, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00108098983765', 'lastCheck': 1384919279.599659, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000473022460938', 'lastCheck': 1384919283.597295, 'valid': True, 'code': 0}}
>Thread-15915034::DEBUG::2013-11-20 03:48:08,080::task::495::TaskManager.Task::(_debug) Task 6a5a3235-e471-4fac-a1b4-125a7ac17bcf: moving from state preparing -> state finished
>Thread-15915034::DEBUG::2013-11-20 03:48:08,080::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915034::DEBUG::2013-11-20 03:48:08,080::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915034::DEBUG::2013-11-20 03:48:08,081::task::495::TaskManager.Task::(_debug) Task 6a5a3235-e471-4fac-a1b4-125a7ac17bcf: ref 0 aborting False
>Thread-15915034::INFO::2013-11-20 03:48:08,082::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00050687789917', 'lastCheck': 1384919279.9945531, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.000903844833374', 'lastCheck': 1384919283.6069131, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00108098983765', 'lastCheck': 1384919279.599659, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000473022460938', 'lastCheck': 1384919283.597295, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:48:09,499::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:48:09,936::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0488552 s, 21.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:48:11,948::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:48:12,387::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0491136 s, 20.8 MB/s\n'; <rc> = 0
>Thread-15915037::DEBUG::2013-11-20 03:48:12,595::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915037::INFO::2013-11-20 03:48:12,596::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915037::DEBUG::2013-11-20 03:48:12,597::task::495::TaskManager.Task::(_debug) Task 45b91694-ffcd-4c56-a66e-69697e2bd94f: moving from state init -> state preparing
>Thread-15915037::DEBUG::2013-11-20 03:48:12,598::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915037::DEBUG::2013-11-20 03:48:12,599::task::495::TaskManager.Task::(_debug) Task 45b91694-ffcd-4c56-a66e-69697e2bd94f: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915037::DEBUG::2013-11-20 03:48:12,599::task::495::TaskManager.Task::(_debug) Task 45b91694-ffcd-4c56-a66e-69697e2bd94f: moving from state preparing -> state finished
>Thread-15915037::DEBUG::2013-11-20 03:48:12,600::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915037::DEBUG::2013-11-20 03:48:12,601::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915037::DEBUG::2013-11-20 03:48:12,601::task::495::TaskManager.Task::(_debug) Task 45b91694-ffcd-4c56-a66e-69697e2bd94f: ref 0 aborting False
>Thread-15915037::INFO::2013-11-20 03:48:12,602::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915038::DEBUG::2013-11-20 03:48:12,611::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915038::INFO::2013-11-20 03:48:12,612::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915038::DEBUG::2013-11-20 03:48:12,613::task::495::TaskManager.Task::(_debug) Task aad2c3cc-e0ba-41ed-878a-1779c98fd125: moving from state init -> state preparing
>Thread-15915038::DEBUG::2013-11-20 03:48:12,615::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0fb877ad-dc3b-4d86-abdd-5697417ba72c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915038::DEBUG::2013-11-20 03:48:12,615::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915038::DEBUG::2013-11-20 03:48:12,617::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915038::DEBUG::2013-11-20 03:48:12,617::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0fb877ad-dc3b-4d86-abdd-5697417ba72c`::Granted request
>Thread-15915038::DEBUG::2013-11-20 03:48:12,619::task::495::TaskManager.Task::(_debug) Task aad2c3cc-e0ba-41ed-878a-1779c98fd125: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915038::DEBUG::2013-11-20 03:48:12,620::task::495::TaskManager.Task::(_debug) Task aad2c3cc-e0ba-41ed-878a-1779c98fd125: ref 1 aborting False
>Thread-15915038::WARNING::2013-11-20 03:48:12,622::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915038::DEBUG::2013-11-20 03:48:12,623::task::495::TaskManager.Task::(_debug) Task aad2c3cc-e0ba-41ed-878a-1779c98fd125: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915038::DEBUG::2013-11-20 03:48:12,624::task::495::TaskManager.Task::(_debug) Task aad2c3cc-e0ba-41ed-878a-1779c98fd125: moving from state preparing -> state finished
>Thread-15915038::DEBUG::2013-11-20 03:48:12,624::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915038::DEBUG::2013-11-20 03:48:12,625::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915038::DEBUG::2013-11-20 03:48:12,626::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915038::DEBUG::2013-11-20 03:48:12,627::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915038::DEBUG::2013-11-20 03:48:12,627::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915038::DEBUG::2013-11-20 03:48:12,628::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915038::DEBUG::2013-11-20 03:48:12,629::task::495::TaskManager.Task::(_debug) Task aad2c3cc-e0ba-41ed-878a-1779c98fd125: ref 0 aborting False
>Thread-15915038::INFO::2013-11-20 03:48:12,629::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913658::INFO::2013-11-20 03:48:14,138::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:48:14,140::task::495::TaskManager.Task::(_debug) Task 9f62998c-5f17-4132-9661-f47c3db0ad26: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:48:14,141::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`16397c7f-b0dd-4da8-9b62-933c76ca3dbc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:48:14,142::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:48:14,143::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:48:14,143::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`16397c7f-b0dd-4da8-9b62-933c76ca3dbc`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:48:14,144::task::495::TaskManager.Task::(_debug) Task 9f62998c-5f17-4132-9661-f47c3db0ad26: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:48:14,145::task::495::TaskManager.Task::(_debug) Task 9f62998c-5f17-4132-9661-f47c3db0ad26: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:48:14,145::task::495::TaskManager.Task::(_debug) Task 9f62998c-5f17-4132-9661-f47c3db0ad26: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:48:14,146::task::495::TaskManager.Task::(_debug) Task 9f62998c-5f17-4132-9661-f47c3db0ad26: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:48:14,146::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:48:14,147::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:48:14,148::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:48:14,148::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:48:14,150::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:48:14,151::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:48:14,152::task::495::TaskManager.Task::(_debug) Task 9f62998c-5f17-4132-9661-f47c3db0ad26: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:48:14,153::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:48:14,403::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:48:14,980::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0790976 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:48:16,994::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:48:17,502::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0744818 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15913659::DEBUG::2013-11-20 03:48:17,774::guestIF::97::vm.Vm::(_handleMessage) vmId=`0dc7e997-e884-4493-b701-790fb37b3d80`::u'None'
>Dummy-15912651::DEBUG::2013-11-20 03:48:19,516::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:48:20,084::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.089777 s, 11.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:48:22,094::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:48:22,663::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0855151 s, 12.0 MB/s\n'; <rc> = 0
>Thread-15915042::DEBUG::2013-11-20 03:48:22,877::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915042::INFO::2013-11-20 03:48:22,878::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915042::DEBUG::2013-11-20 03:48:22,879::task::495::TaskManager.Task::(_debug) Task d68e992c-63c9-4337-9680-75f8f1763871: moving from state init -> state preparing
>Thread-15915042::DEBUG::2013-11-20 03:48:22,880::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915042::DEBUG::2013-11-20 03:48:22,881::task::495::TaskManager.Task::(_debug) Task d68e992c-63c9-4337-9680-75f8f1763871: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915042::DEBUG::2013-11-20 03:48:22,881::task::495::TaskManager.Task::(_debug) Task d68e992c-63c9-4337-9680-75f8f1763871: moving from state preparing -> state finished
>Thread-15915042::DEBUG::2013-11-20 03:48:22,882::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915042::DEBUG::2013-11-20 03:48:22,882::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915042::DEBUG::2013-11-20 03:48:22,883::task::495::TaskManager.Task::(_debug) Task d68e992c-63c9-4337-9680-75f8f1763871: ref 0 aborting False
>Thread-15915042::INFO::2013-11-20 03:48:22,884::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915043::DEBUG::2013-11-20 03:48:22,889::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915043::INFO::2013-11-20 03:48:22,890::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915043::DEBUG::2013-11-20 03:48:22,891::task::495::TaskManager.Task::(_debug) Task d790e2a8-cd1e-442d-9797-1554b4077711: moving from state init -> state preparing
>Thread-15915043::DEBUG::2013-11-20 03:48:22,893::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`304eb981-2b51-44b2-8f1a-d250ac84ed33`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915043::DEBUG::2013-11-20 03:48:22,893::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915043::DEBUG::2013-11-20 03:48:22,895::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915043::DEBUG::2013-11-20 03:48:22,895::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`304eb981-2b51-44b2-8f1a-d250ac84ed33`::Granted request
>Thread-15915043::DEBUG::2013-11-20 03:48:22,897::task::495::TaskManager.Task::(_debug) Task d790e2a8-cd1e-442d-9797-1554b4077711: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915043::DEBUG::2013-11-20 03:48:22,897::task::495::TaskManager.Task::(_debug) Task d790e2a8-cd1e-442d-9797-1554b4077711: ref 1 aborting False
>Thread-15915043::WARNING::2013-11-20 03:48:22,900::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915043::DEBUG::2013-11-20 03:48:22,901::task::495::TaskManager.Task::(_debug) Task d790e2a8-cd1e-442d-9797-1554b4077711: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915043::DEBUG::2013-11-20 03:48:22,902::task::495::TaskManager.Task::(_debug) Task d790e2a8-cd1e-442d-9797-1554b4077711: moving from state preparing -> state finished
>Thread-15915043::DEBUG::2013-11-20 03:48:22,902::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915043::DEBUG::2013-11-20 03:48:22,903::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915043::DEBUG::2013-11-20 03:48:22,904::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915043::DEBUG::2013-11-20 03:48:22,904::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915043::DEBUG::2013-11-20 03:48:22,905::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915043::DEBUG::2013-11-20 03:48:22,905::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915043::DEBUG::2013-11-20 03:48:22,906::task::495::TaskManager.Task::(_debug) Task d790e2a8-cd1e-442d-9797-1554b4077711: ref 0 aborting False
>Thread-15915043::INFO::2013-11-20 03:48:22,907::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915044::INFO::2013-11-20 03:48:23,708::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915044::DEBUG::2013-11-20 03:48:23,709::task::495::TaskManager.Task::(_debug) Task a6adf413-1844-4628-9dee-53e053885566: moving from state init -> state preparing
>Thread-15915044::DEBUG::2013-11-20 03:48:23,710::task::495::TaskManager.Task::(_debug) Task a6adf413-1844-4628-9dee-53e053885566: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00034499168396', 'lastCheck': 1384919300.2023139, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00571393966675', 'lastCheck': 1384919293.693161, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00131487846375', 'lastCheck': 1384919299.773123, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00259208679199', 'lastCheck': 1384919293.6807921, 'valid': True, 'code': 0}}
>Thread-15915044::DEBUG::2013-11-20 03:48:23,711::task::495::TaskManager.Task::(_debug) Task a6adf413-1844-4628-9dee-53e053885566: moving from state preparing -> state finished
>Thread-15915044::DEBUG::2013-11-20 03:48:23,713::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915044::DEBUG::2013-11-20 03:48:23,714::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915044::DEBUG::2013-11-20 03:48:23,715::task::495::TaskManager.Task::(_debug) Task a6adf413-1844-4628-9dee-53e053885566: ref 0 aborting False
>Thread-15915044::INFO::2013-11-20 03:48:23,716::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00034499168396', 'lastCheck': 1384919300.2023139, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00571393966675', 'lastCheck': 1384919293.693161, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00131487846375', 'lastCheck': 1384919299.773123, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00259208679199', 'lastCheck': 1384919293.6807921, 'valid': True, 'code': 0}}
>Thread-15912392::INFO::2013-11-20 03:48:24,320::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:48:24,321::task::495::TaskManager.Task::(_debug) Task 6ab8fc33-26ce-4f37-b145-e3fe2dbf4042: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:48:24,337::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b2b5d540-ba13-4b52-9e33-dbdb0f78e326`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:48:24,338::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:48:24,339::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:48:24,340::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b2b5d540-ba13-4b52-9e33-dbdb0f78e326`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:48:24,341::task::495::TaskManager.Task::(_debug) Task 6ab8fc33-26ce-4f37-b145-e3fe2dbf4042: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:48:24,341::task::495::TaskManager.Task::(_debug) Task 6ab8fc33-26ce-4f37-b145-e3fe2dbf4042: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:48:24,342::task::495::TaskManager.Task::(_debug) Task 6ab8fc33-26ce-4f37-b145-e3fe2dbf4042: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:48:24,342::task::495::TaskManager.Task::(_debug) Task 6ab8fc33-26ce-4f37-b145-e3fe2dbf4042: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:48:24,343::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:48:24,344::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:48:24,345::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::INFO::2013-11-20 03:48:24,346::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:48:24,348::task::495::TaskManager.Task::(_debug) Task 21da9fe0-8651-4151-9d5b-e3551dadbc72: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:48:24,348::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:48:24,350::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2f9e08e2-8d3b-4ef0-a079-46dc85029844`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:48:24,351::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:48:24,352::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:48:24,353::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:48:24,354::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:48:24,354::task::495::TaskManager.Task::(_debug) Task 6ab8fc33-26ce-4f37-b145-e3fe2dbf4042: ref 0 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:48:24,355::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2f9e08e2-8d3b-4ef0-a079-46dc85029844`::Granted request
>Thread-15912392::INFO::2013-11-20 03:48:24,356::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913695::DEBUG::2013-11-20 03:48:24,357::task::495::TaskManager.Task::(_debug) Task 21da9fe0-8651-4151-9d5b-e3551dadbc72: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:48:24,358::task::495::TaskManager.Task::(_debug) Task 21da9fe0-8651-4151-9d5b-e3551dadbc72: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:48:24,359::task::495::TaskManager.Task::(_debug) Task 21da9fe0-8651-4151-9d5b-e3551dadbc72: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:48:24,360::task::495::TaskManager.Task::(_debug) Task 21da9fe0-8651-4151-9d5b-e3551dadbc72: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:48:24,361::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:48:24,362::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:48:24,363::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:48:24,363::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:48:24,364::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:48:24,364::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:48:24,365::task::495::TaskManager.Task::(_debug) Task 21da9fe0-8651-4151-9d5b-e3551dadbc72: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:48:24,366::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:48:24,680::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:48:25,226::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0822396 s, 12.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:48:27,241::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:48:27,840::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0929132 s, 11.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:48:29,855::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:48:30,390::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0823914 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:48:32,406::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:48:32,877::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0691593 s, 14.8 MB/s\n'; <rc> = 0
>Thread-15915048::DEBUG::2013-11-20 03:48:33,136::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915048::INFO::2013-11-20 03:48:33,143::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915048::DEBUG::2013-11-20 03:48:33,146::task::495::TaskManager.Task::(_debug) Task ff225a2f-00ec-478b-8b77-f95593be2292: moving from state init -> state preparing
>Thread-15915048::DEBUG::2013-11-20 03:48:33,149::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915048::DEBUG::2013-11-20 03:48:33,150::task::495::TaskManager.Task::(_debug) Task ff225a2f-00ec-478b-8b77-f95593be2292: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915048::DEBUG::2013-11-20 03:48:33,151::task::495::TaskManager.Task::(_debug) Task ff225a2f-00ec-478b-8b77-f95593be2292: moving from state preparing -> state finished
>Thread-15915048::DEBUG::2013-11-20 03:48:33,152::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915048::DEBUG::2013-11-20 03:48:33,153::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915048::DEBUG::2013-11-20 03:48:33,153::task::495::TaskManager.Task::(_debug) Task ff225a2f-00ec-478b-8b77-f95593be2292: ref 0 aborting False
>Thread-15915048::INFO::2013-11-20 03:48:33,154::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915049::DEBUG::2013-11-20 03:48:33,160::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915049::INFO::2013-11-20 03:48:33,161::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915049::DEBUG::2013-11-20 03:48:33,161::task::495::TaskManager.Task::(_debug) Task 9718744f-6877-4a83-ab59-b952ccf13de7: moving from state init -> state preparing
>Thread-15915049::DEBUG::2013-11-20 03:48:33,163::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`886bcacb-ba27-41c9-9dc2-c3a373893426`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915049::DEBUG::2013-11-20 03:48:33,163::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915049::DEBUG::2013-11-20 03:48:33,164::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915049::DEBUG::2013-11-20 03:48:33,165::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`886bcacb-ba27-41c9-9dc2-c3a373893426`::Granted request
>Thread-15915049::DEBUG::2013-11-20 03:48:33,166::task::495::TaskManager.Task::(_debug) Task 9718744f-6877-4a83-ab59-b952ccf13de7: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915049::DEBUG::2013-11-20 03:48:33,166::task::495::TaskManager.Task::(_debug) Task 9718744f-6877-4a83-ab59-b952ccf13de7: ref 1 aborting False
>Thread-15915049::WARNING::2013-11-20 03:48:33,168::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915049::DEBUG::2013-11-20 03:48:33,169::task::495::TaskManager.Task::(_debug) Task 9718744f-6877-4a83-ab59-b952ccf13de7: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915049::DEBUG::2013-11-20 03:48:33,170::task::495::TaskManager.Task::(_debug) Task 9718744f-6877-4a83-ab59-b952ccf13de7: moving from state preparing -> state finished
>Thread-15915049::DEBUG::2013-11-20 03:48:33,171::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915049::DEBUG::2013-11-20 03:48:33,172::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915049::DEBUG::2013-11-20 03:48:33,173::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915049::DEBUG::2013-11-20 03:48:33,173::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915049::DEBUG::2013-11-20 03:48:33,174::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915049::DEBUG::2013-11-20 03:48:33,175::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915049::DEBUG::2013-11-20 03:48:33,176::task::495::TaskManager.Task::(_debug) Task 9718744f-6877-4a83-ab59-b952ccf13de7: ref 0 aborting False
>Thread-15915049::INFO::2013-11-20 03:48:33,176::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914435::INFO::2013-11-20 03:48:34,404::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 03:48:34,405::task::495::TaskManager.Task::(_debug) Task 900c1ad3-42c1-4cd3-a5dc-f7b3a486dad8: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 03:48:34,407::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`168ccc5a-d1d5-442b-88b0-a8686170580c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 03:48:34,408::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 03:48:34,408::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 03:48:34,409::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`168ccc5a-d1d5-442b-88b0-a8686170580c`::Granted request
>Thread-15914435::DEBUG::2013-11-20 03:48:34,410::task::495::TaskManager.Task::(_debug) Task 900c1ad3-42c1-4cd3-a5dc-f7b3a486dad8: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 03:48:34,411::task::495::TaskManager.Task::(_debug) Task 900c1ad3-42c1-4cd3-a5dc-f7b3a486dad8: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 03:48:34,412::task::495::TaskManager.Task::(_debug) Task 900c1ad3-42c1-4cd3-a5dc-f7b3a486dad8: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 03:48:34,412::task::495::TaskManager.Task::(_debug) Task 900c1ad3-42c1-4cd3-a5dc-f7b3a486dad8: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 03:48:34,413::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 03:48:34,413::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 03:48:34,414::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 03:48:34,415::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 03:48:34,415::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 03:48:34,416::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 03:48:34,417::task::495::TaskManager.Task::(_debug) Task 900c1ad3-42c1-4cd3-a5dc-f7b3a486dad8: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 03:48:34,417::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 03:48:34,890::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:48:35,453::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.070133 s, 14.6 MB/s\n'; <rc> = 0
>Thread-15914453::INFO::2013-11-20 03:48:36,674::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 03:48:36,675::task::495::TaskManager.Task::(_debug) Task 9b129faa-46b8-47fc-9182-2b64089a5970: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 03:48:36,677::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`52d497be-2a41-4adc-90a1-7d1332f72c56`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 03:48:36,677::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 03:48:36,678::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 03:48:36,679::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`52d497be-2a41-4adc-90a1-7d1332f72c56`::Granted request
>Thread-15914453::DEBUG::2013-11-20 03:48:36,681::task::495::TaskManager.Task::(_debug) Task 9b129faa-46b8-47fc-9182-2b64089a5970: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 03:48:36,681::task::495::TaskManager.Task::(_debug) Task 9b129faa-46b8-47fc-9182-2b64089a5970: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 03:48:36,682::task::495::TaskManager.Task::(_debug) Task 9b129faa-46b8-47fc-9182-2b64089a5970: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 03:48:36,683::task::495::TaskManager.Task::(_debug) Task 9b129faa-46b8-47fc-9182-2b64089a5970: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 03:48:36,683::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 03:48:36,684::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 03:48:36,685::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 03:48:36,685::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 03:48:36,686::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 03:48:36,687::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 03:48:36,687::task::495::TaskManager.Task::(_debug) Task 9b129faa-46b8-47fc-9182-2b64089a5970: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 03:48:36,688::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 03:48:37,466::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:48:38,081::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0774569 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15915052::INFO::2013-11-20 03:48:39,622::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915052::DEBUG::2013-11-20 03:48:39,623::task::495::TaskManager.Task::(_debug) Task 3bf8144b-4574-444f-a777-b282153cb0e5: moving from state init -> state preparing
>Thread-15915052::DEBUG::2013-11-20 03:48:39,630::task::495::TaskManager.Task::(_debug) Task 3bf8144b-4574-444f-a777-b282153cb0e5: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000458955764771', 'lastCheck': 1384919310.48175, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00206804275513', 'lastCheck': 1384919314.0488949, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00119590759277', 'lastCheck': 1384919310.0451679, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000777006149292', 'lastCheck': 1384919314.0428979, 'valid': True, 'code': 0}}
>Thread-15915052::DEBUG::2013-11-20 03:48:39,632::task::495::TaskManager.Task::(_debug) Task 3bf8144b-4574-444f-a777-b282153cb0e5: moving from state preparing -> state finished
>Thread-15915052::DEBUG::2013-11-20 03:48:39,633::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915052::DEBUG::2013-11-20 03:48:39,633::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915052::DEBUG::2013-11-20 03:48:39,634::task::495::TaskManager.Task::(_debug) Task 3bf8144b-4574-444f-a777-b282153cb0e5: ref 0 aborting False
>Thread-15915052::INFO::2013-11-20 03:48:39,635::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000458955764771', 'lastCheck': 1384919310.48175, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00206804275513', 'lastCheck': 1384919314.0488949, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00119590759277', 'lastCheck': 1384919310.0451679, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000777006149292', 'lastCheck': 1384919314.0428979, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:48:40,090::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:48:40,656::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0830342 s, 12.3 MB/s\n'; <rc> = 0
>Thread-15914519::INFO::2013-11-20 03:48:41,736::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 03:48:41,739::task::495::TaskManager.Task::(_debug) Task 3c45fbed-dc9e-4662-990e-a634667fa5df: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 03:48:41,744::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ea0ea7e4-dea6-4359-89b1-1c2cc498e713`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 03:48:41,745::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 03:48:41,747::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 03:48:41,747::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ea0ea7e4-dea6-4359-89b1-1c2cc498e713`::Granted request
>Thread-15914519::DEBUG::2013-11-20 03:48:41,749::task::495::TaskManager.Task::(_debug) Task 3c45fbed-dc9e-4662-990e-a634667fa5df: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 03:48:41,750::task::495::TaskManager.Task::(_debug) Task 3c45fbed-dc9e-4662-990e-a634667fa5df: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 03:48:41,751::task::495::TaskManager.Task::(_debug) Task 3c45fbed-dc9e-4662-990e-a634667fa5df: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 03:48:41,751::task::495::TaskManager.Task::(_debug) Task 3c45fbed-dc9e-4662-990e-a634667fa5df: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 03:48:41,752::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 03:48:41,753::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 03:48:41,753::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 03:48:41,754::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 03:48:41,754::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 03:48:41,755::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 03:48:41,756::task::495::TaskManager.Task::(_debug) Task 3c45fbed-dc9e-4662-990e-a634667fa5df: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 03:48:41,756::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 03:48:42,674::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:48:43,250::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.083801 s, 12.2 MB/s\n'; <rc> = 0
>Thread-15915055::DEBUG::2013-11-20 03:48:43,522::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915055::INFO::2013-11-20 03:48:43,523::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915055::DEBUG::2013-11-20 03:48:43,523::task::495::TaskManager.Task::(_debug) Task 17a2c314-35f7-487a-b911-af37e7555ae1: moving from state init -> state preparing
>Thread-15915055::DEBUG::2013-11-20 03:48:43,524::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915055::DEBUG::2013-11-20 03:48:43,524::task::495::TaskManager.Task::(_debug) Task 17a2c314-35f7-487a-b911-af37e7555ae1: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915055::DEBUG::2013-11-20 03:48:43,524::task::495::TaskManager.Task::(_debug) Task 17a2c314-35f7-487a-b911-af37e7555ae1: moving from state preparing -> state finished
>Thread-15915055::DEBUG::2013-11-20 03:48:43,525::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915055::DEBUG::2013-11-20 03:48:43,526::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915055::DEBUG::2013-11-20 03:48:43,526::task::495::TaskManager.Task::(_debug) Task 17a2c314-35f7-487a-b911-af37e7555ae1: ref 0 aborting False
>Thread-15915055::INFO::2013-11-20 03:48:43,526::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915056::DEBUG::2013-11-20 03:48:43,531::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915056::INFO::2013-11-20 03:48:43,533::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915056::DEBUG::2013-11-20 03:48:43,534::task::495::TaskManager.Task::(_debug) Task db5ae438-c38b-4eae-8478-bfe416a600ab: moving from state init -> state preparing
>Thread-15915056::DEBUG::2013-11-20 03:48:43,536::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`dc1a489e-b09a-4ee6-acdf-3b5dc6eed5c8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915056::DEBUG::2013-11-20 03:48:43,537::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915056::DEBUG::2013-11-20 03:48:43,538::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915056::DEBUG::2013-11-20 03:48:43,538::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`dc1a489e-b09a-4ee6-acdf-3b5dc6eed5c8`::Granted request
>Thread-15915056::DEBUG::2013-11-20 03:48:43,540::task::495::TaskManager.Task::(_debug) Task db5ae438-c38b-4eae-8478-bfe416a600ab: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915056::DEBUG::2013-11-20 03:48:43,540::task::495::TaskManager.Task::(_debug) Task db5ae438-c38b-4eae-8478-bfe416a600ab: ref 1 aborting False
>Thread-15915056::WARNING::2013-11-20 03:48:43,543::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915056::DEBUG::2013-11-20 03:48:43,544::task::495::TaskManager.Task::(_debug) Task db5ae438-c38b-4eae-8478-bfe416a600ab: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915056::DEBUG::2013-11-20 03:48:43,545::task::495::TaskManager.Task::(_debug) Task db5ae438-c38b-4eae-8478-bfe416a600ab: moving from state preparing -> state finished
>Thread-15915056::DEBUG::2013-11-20 03:48:43,545::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915056::DEBUG::2013-11-20 03:48:43,546::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915056::DEBUG::2013-11-20 03:48:43,547::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915056::DEBUG::2013-11-20 03:48:43,548::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915056::DEBUG::2013-11-20 03:48:43,548::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915056::DEBUG::2013-11-20 03:48:43,549::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915056::DEBUG::2013-11-20 03:48:43,550::task::495::TaskManager.Task::(_debug) Task db5ae438-c38b-4eae-8478-bfe416a600ab: ref 0 aborting False
>Thread-15915056::INFO::2013-11-20 03:48:43,551::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:48:45,270::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:48:45,859::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0809529 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 03:48:46,672::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:48:46,673::task::495::TaskManager.Task::(_debug) Task aaf42434-eda5-4918-940c-33fbba730303: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:48:46,676::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2c6b6347-308d-43d9-be55-3e56d7da2807`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:48:46,677::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:48:46,678::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:48:46,678::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2c6b6347-308d-43d9-be55-3e56d7da2807`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:48:46,679::task::495::TaskManager.Task::(_debug) Task aaf42434-eda5-4918-940c-33fbba730303: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:48:46,680::task::495::TaskManager.Task::(_debug) Task aaf42434-eda5-4918-940c-33fbba730303: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:48:46,681::task::495::TaskManager.Task::(_debug) Task aaf42434-eda5-4918-940c-33fbba730303: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:48:46,682::task::495::TaskManager.Task::(_debug) Task aaf42434-eda5-4918-940c-33fbba730303: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:48:46,682::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:48:46,683::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:48:46,684::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:48:46,685::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:48:46,685::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:48:46,686::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:48:46,687::task::495::TaskManager.Task::(_debug) Task aaf42434-eda5-4918-940c-33fbba730303: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:48:46,687::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:48:46,688::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:48:46,689::task::495::TaskManager.Task::(_debug) Task ac49ff05-2e2e-493d-a269-b5d5ab9552f9: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:48:46,691::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6e39abe1-81a9-4bb1-a6b7-c5c54e40fc69`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:48:46,691::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:48:46,692::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:48:46,693::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6e39abe1-81a9-4bb1-a6b7-c5c54e40fc69`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:48:46,694::task::495::TaskManager.Task::(_debug) Task ac49ff05-2e2e-493d-a269-b5d5ab9552f9: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:48:46,695::task::495::TaskManager.Task::(_debug) Task ac49ff05-2e2e-493d-a269-b5d5ab9552f9: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:48:46,696::task::495::TaskManager.Task::(_debug) Task ac49ff05-2e2e-493d-a269-b5d5ab9552f9: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:48:46,696::task::495::TaskManager.Task::(_debug) Task ac49ff05-2e2e-493d-a269-b5d5ab9552f9: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:48:46,697::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:48:46,697::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:48:46,698::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:48:46,699::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:48:46,699::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:48:46,699::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:48:46,700::task::495::TaskManager.Task::(_debug) Task ac49ff05-2e2e-493d-a269-b5d5ab9552f9: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:48:46,701::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:48:46,702::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:48:46,702::task::495::TaskManager.Task::(_debug) Task 33142863-a9a0-4889-ac76-5874eb04003a: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:48:46,704::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6ebab557-1282-48c7-97b2-3cfdedec0e7e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:48:46,704::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:48:46,705::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:48:46,705::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6ebab557-1282-48c7-97b2-3cfdedec0e7e`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:48:46,707::task::495::TaskManager.Task::(_debug) Task 33142863-a9a0-4889-ac76-5874eb04003a: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:48:46,707::task::495::TaskManager.Task::(_debug) Task 33142863-a9a0-4889-ac76-5874eb04003a: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:48:46,708::task::495::TaskManager.Task::(_debug) Task 33142863-a9a0-4889-ac76-5874eb04003a: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:48:46,709::task::495::TaskManager.Task::(_debug) Task 33142863-a9a0-4889-ac76-5874eb04003a: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:48:46,709::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:48:46,710::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:48:46,710::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:48:46,711::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:48:46,712::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:48:46,712::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:48:46,713::task::495::TaskManager.Task::(_debug) Task 33142863-a9a0-4889-ac76-5874eb04003a: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:48:46,713::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:48:46,714::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:48:46,715::task::495::TaskManager.Task::(_debug) Task 7e49e10c-46a8-4cad-ad18-83119ea1e0a6: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:48:46,716::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`85833571-cff1-4e83-8654-5f727dc54402`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:48:46,716::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:48:46,717::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:48:46,718::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`85833571-cff1-4e83-8654-5f727dc54402`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:48:46,719::task::495::TaskManager.Task::(_debug) Task 7e49e10c-46a8-4cad-ad18-83119ea1e0a6: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:48:46,719::task::495::TaskManager.Task::(_debug) Task 7e49e10c-46a8-4cad-ad18-83119ea1e0a6: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:48:46,720::task::495::TaskManager.Task::(_debug) Task 7e49e10c-46a8-4cad-ad18-83119ea1e0a6: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:48:46,721::task::495::TaskManager.Task::(_debug) Task 7e49e10c-46a8-4cad-ad18-83119ea1e0a6: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:48:46,721::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:48:46,722::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:48:46,722::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:48:46,723::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:48:46,724::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:48:46,724::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:48:46,725::task::495::TaskManager.Task::(_debug) Task 7e49e10c-46a8-4cad-ad18-83119ea1e0a6: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:48:46,725::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:48:47,878::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:48:48,464::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0744087 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 03:48:49,323::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:48:49,324::task::495::TaskManager.Task::(_debug) Task 2362eef4-ad1c-413c-a24d-990a9d541b68: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:48:49,326::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4ef04817-6c4f-4864-8cc3-da46a53ef7fa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:48:49,326::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:48:49,327::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:48:49,328::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4ef04817-6c4f-4864-8cc3-da46a53ef7fa`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:48:49,329::task::495::TaskManager.Task::(_debug) Task 2362eef4-ad1c-413c-a24d-990a9d541b68: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:48:49,329::task::495::TaskManager.Task::(_debug) Task 2362eef4-ad1c-413c-a24d-990a9d541b68: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:48:49,330::task::495::TaskManager.Task::(_debug) Task 2362eef4-ad1c-413c-a24d-990a9d541b68: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:48:49,331::task::495::TaskManager.Task::(_debug) Task 2362eef4-ad1c-413c-a24d-990a9d541b68: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:48:49,332::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:48:49,332::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:48:49,333::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:48:49,334::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:48:49,334::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:48:49,335::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:48:49,335::task::495::TaskManager.Task::(_debug) Task 2362eef4-ad1c-413c-a24d-990a9d541b68: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:48:49,336::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:48:49,337::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:48:49,338::task::495::TaskManager.Task::(_debug) Task 2f5b2e31-064a-43f4-99a7-9c87975d639d: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:48:49,339::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d0a3997f-707d-4a01-b80f-d99368171ac0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:48:49,340::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:48:49,340::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:48:49,341::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d0a3997f-707d-4a01-b80f-d99368171ac0`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:48:49,342::task::495::TaskManager.Task::(_debug) Task 2f5b2e31-064a-43f4-99a7-9c87975d639d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:48:49,343::task::495::TaskManager.Task::(_debug) Task 2f5b2e31-064a-43f4-99a7-9c87975d639d: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:48:49,343::task::495::TaskManager.Task::(_debug) Task 2f5b2e31-064a-43f4-99a7-9c87975d639d: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:48:49,344::task::495::TaskManager.Task::(_debug) Task 2f5b2e31-064a-43f4-99a7-9c87975d639d: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:48:49,345::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:48:49,345::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:48:49,346::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:48:49,347::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:48:49,347::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:48:49,348::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:48:49,348::task::495::TaskManager.Task::(_debug) Task 2f5b2e31-064a-43f4-99a7-9c87975d639d: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:48:49,349::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:48:50,478::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:48:51,018::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0635557 s, 16.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:48:53,032::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:48:53,626::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0924506 s, 11.1 MB/s\n'; <rc> = 0
>Thread-15915060::DEBUG::2013-11-20 03:48:54,049::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915060::INFO::2013-11-20 03:48:54,051::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915060::DEBUG::2013-11-20 03:48:54,052::task::495::TaskManager.Task::(_debug) Task 89fd2762-48f5-46da-9e99-a9a1ecb72a48: moving from state init -> state preparing
>Thread-15915060::DEBUG::2013-11-20 03:48:54,053::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915060::DEBUG::2013-11-20 03:48:54,053::task::495::TaskManager.Task::(_debug) Task 89fd2762-48f5-46da-9e99-a9a1ecb72a48: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915060::DEBUG::2013-11-20 03:48:54,054::task::495::TaskManager.Task::(_debug) Task 89fd2762-48f5-46da-9e99-a9a1ecb72a48: moving from state preparing -> state finished
>Thread-15915060::DEBUG::2013-11-20 03:48:54,055::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915060::DEBUG::2013-11-20 03:48:54,055::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915060::DEBUG::2013-11-20 03:48:54,056::task::495::TaskManager.Task::(_debug) Task 89fd2762-48f5-46da-9e99-a9a1ecb72a48: ref 0 aborting False
>Thread-15915060::INFO::2013-11-20 03:48:54,057::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915061::DEBUG::2013-11-20 03:48:54,066::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915061::INFO::2013-11-20 03:48:54,068::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915061::DEBUG::2013-11-20 03:48:54,069::task::495::TaskManager.Task::(_debug) Task 50820d34-7557-430f-b69e-b0f10d5aab61: moving from state init -> state preparing
>Thread-15915061::DEBUG::2013-11-20 03:48:54,070::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`82bd3963-07fb-4196-8604-ae0d8299d057`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915061::DEBUG::2013-11-20 03:48:54,071::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915061::DEBUG::2013-11-20 03:48:54,072::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915061::DEBUG::2013-11-20 03:48:54,072::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`82bd3963-07fb-4196-8604-ae0d8299d057`::Granted request
>Thread-15915061::DEBUG::2013-11-20 03:48:54,073::task::495::TaskManager.Task::(_debug) Task 50820d34-7557-430f-b69e-b0f10d5aab61: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915061::DEBUG::2013-11-20 03:48:54,074::task::495::TaskManager.Task::(_debug) Task 50820d34-7557-430f-b69e-b0f10d5aab61: ref 1 aborting False
>Thread-15915061::WARNING::2013-11-20 03:48:54,076::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915061::DEBUG::2013-11-20 03:48:54,077::task::495::TaskManager.Task::(_debug) Task 50820d34-7557-430f-b69e-b0f10d5aab61: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915061::DEBUG::2013-11-20 03:48:54,078::task::495::TaskManager.Task::(_debug) Task 50820d34-7557-430f-b69e-b0f10d5aab61: moving from state preparing -> state finished
>Thread-15915061::DEBUG::2013-11-20 03:48:54,078::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915061::DEBUG::2013-11-20 03:48:54,079::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915061::DEBUG::2013-11-20 03:48:54,080::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915061::DEBUG::2013-11-20 03:48:54,080::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915061::DEBUG::2013-11-20 03:48:54,081::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915061::DEBUG::2013-11-20 03:48:54,082::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915061::DEBUG::2013-11-20 03:48:54,082::task::495::TaskManager.Task::(_debug) Task 50820d34-7557-430f-b69e-b0f10d5aab61: ref 0 aborting False
>Thread-15915061::INFO::2013-11-20 03:48:54,083::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915062::INFO::2013-11-20 03:48:55,360::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915062::DEBUG::2013-11-20 03:48:55,361::task::495::TaskManager.Task::(_debug) Task 7fbecf32-0051-4801-a14a-4e9010715dae: moving from state init -> state preparing
>Thread-15915062::DEBUG::2013-11-20 03:48:55,362::task::495::TaskManager.Task::(_debug) Task 7fbecf32-0051-4801-a14a-4e9010715dae: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000355958938599', 'lastCheck': 1384919331.107744, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00139904022217', 'lastCheck': 1384919334.695749, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00131988525391', 'lastCheck': 1384919330.6958549, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000615835189819', 'lastCheck': 1384919334.6929531, 'valid': True, 'code': 0}}
>Thread-15915062::DEBUG::2013-11-20 03:48:55,363::task::495::TaskManager.Task::(_debug) Task 7fbecf32-0051-4801-a14a-4e9010715dae: moving from state preparing -> state finished
>Thread-15915062::DEBUG::2013-11-20 03:48:55,363::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915062::DEBUG::2013-11-20 03:48:55,364::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915062::DEBUG::2013-11-20 03:48:55,365::task::495::TaskManager.Task::(_debug) Task 7fbecf32-0051-4801-a14a-4e9010715dae: ref 0 aborting False
>Thread-15915062::INFO::2013-11-20 03:48:55,365::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000355958938599', 'lastCheck': 1384919331.107744, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00139904022217', 'lastCheck': 1384919334.695749, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00131988525391', 'lastCheck': 1384919330.6958549, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000615835189819', 'lastCheck': 1384919334.6929531, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:48:55,640::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:48:56,236::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0869206 s, 11.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:48:58,252::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:48:58,672::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0628627 s, 16.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:49:00,686::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:49:01,223::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0768277 s, 13.3 MB/s\n'; <rc> = 0
>Thread-15913639::INFO::2013-11-20 03:49:01,880::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:49:01,881::task::495::TaskManager.Task::(_debug) Task 77d69360-9469-4c94-b3f5-f46f84bee6d5: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:49:01,883::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7ee4f400-b374-470c-9aaa-70b9454593b6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:49:01,884::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:49:01,885::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:49:01,886::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7ee4f400-b374-470c-9aaa-70b9454593b6`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:49:01,887::task::495::TaskManager.Task::(_debug) Task 77d69360-9469-4c94-b3f5-f46f84bee6d5: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:49:01,888::task::495::TaskManager.Task::(_debug) Task 77d69360-9469-4c94-b3f5-f46f84bee6d5: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:49:01,889::task::495::TaskManager.Task::(_debug) Task 77d69360-9469-4c94-b3f5-f46f84bee6d5: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:49:01,891::task::495::TaskManager.Task::(_debug) Task 77d69360-9469-4c94-b3f5-f46f84bee6d5: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:49:01,893::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:49:01,893::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:49:01,895::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:49:01,908::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:49:01,915::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:49:01,925::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:49:01,931::task::495::TaskManager.Task::(_debug) Task 77d69360-9469-4c94-b3f5-f46f84bee6d5: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:49:01,933::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:49:03,236::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:49:03,766::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0790036 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15915066::DEBUG::2013-11-20 03:49:04,917::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915066::INFO::2013-11-20 03:49:04,924::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915066::DEBUG::2013-11-20 03:49:04,933::task::495::TaskManager.Task::(_debug) Task 3e110e52-7c54-40a7-bae7-2c748edea9ba: moving from state init -> state preparing
>Thread-15915066::DEBUG::2013-11-20 03:49:04,936::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915066::DEBUG::2013-11-20 03:49:04,939::task::495::TaskManager.Task::(_debug) Task 3e110e52-7c54-40a7-bae7-2c748edea9ba: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915066::DEBUG::2013-11-20 03:49:04,941::task::495::TaskManager.Task::(_debug) Task 3e110e52-7c54-40a7-bae7-2c748edea9ba: moving from state preparing -> state finished
>Thread-15915066::DEBUG::2013-11-20 03:49:04,947::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915066::DEBUG::2013-11-20 03:49:04,953::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915066::DEBUG::2013-11-20 03:49:04,955::task::495::TaskManager.Task::(_debug) Task 3e110e52-7c54-40a7-bae7-2c748edea9ba: ref 0 aborting False
>Thread-15915066::INFO::2013-11-20 03:49:04,957::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915067::DEBUG::2013-11-20 03:49:05,353::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915067::INFO::2013-11-20 03:49:05,354::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915067::DEBUG::2013-11-20 03:49:05,355::task::495::TaskManager.Task::(_debug) Task a99d43b9-f8d1-4350-bda4-620c66270ea8: moving from state init -> state preparing
>Thread-15915067::DEBUG::2013-11-20 03:49:05,356::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3f13fc67-cfb1-47a3-8383-641ba5d911a5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915067::DEBUG::2013-11-20 03:49:05,357::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915067::DEBUG::2013-11-20 03:49:05,358::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915067::DEBUG::2013-11-20 03:49:05,359::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3f13fc67-cfb1-47a3-8383-641ba5d911a5`::Granted request
>Thread-15915067::DEBUG::2013-11-20 03:49:05,360::task::495::TaskManager.Task::(_debug) Task a99d43b9-f8d1-4350-bda4-620c66270ea8: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915067::DEBUG::2013-11-20 03:49:05,361::task::495::TaskManager.Task::(_debug) Task a99d43b9-f8d1-4350-bda4-620c66270ea8: ref 1 aborting False
>Thread-15915067::WARNING::2013-11-20 03:49:05,363::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915067::DEBUG::2013-11-20 03:49:05,364::task::495::TaskManager.Task::(_debug) Task a99d43b9-f8d1-4350-bda4-620c66270ea8: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915067::DEBUG::2013-11-20 03:49:05,366::task::495::TaskManager.Task::(_debug) Task a99d43b9-f8d1-4350-bda4-620c66270ea8: moving from state preparing -> state finished
>Thread-15915067::DEBUG::2013-11-20 03:49:05,367::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915067::DEBUG::2013-11-20 03:49:05,368::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915067::DEBUG::2013-11-20 03:49:05,370::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915067::DEBUG::2013-11-20 03:49:05,372::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915067::DEBUG::2013-11-20 03:49:05,374::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915067::DEBUG::2013-11-20 03:49:05,374::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915067::DEBUG::2013-11-20 03:49:05,375::task::495::TaskManager.Task::(_debug) Task a99d43b9-f8d1-4350-bda4-620c66270ea8: ref 0 aborting False
>Thread-15915067::INFO::2013-11-20 03:49:05,376::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:49:05,780::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:49:06,357::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0828994 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:49:08,373::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:49:08,969::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0847843 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:49:10,987::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:49:11,533::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0768175 s, 13.3 MB/s\n'; <rc> = 0
>Thread-15915070::INFO::2013-11-20 03:49:11,639::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915070::DEBUG::2013-11-20 03:49:11,641::task::495::TaskManager.Task::(_debug) Task b51e2156-c590-4ef2-8b88-f48f3925f1e9: moving from state init -> state preparing
>Thread-15915070::DEBUG::2013-11-20 03:49:11,641::task::495::TaskManager.Task::(_debug) Task b51e2156-c590-4ef2-8b88-f48f3925f1e9: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000694036483765', 'lastCheck': 1384919351.6097369, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0449078083038', 'lastCheck': 1384919344.9546759, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.207326889038', 'lastCheck': 1384919351.1941149, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00080680847168', 'lastCheck': 1384919344.9019179, 'valid': True, 'code': 0}}
>Thread-15915070::DEBUG::2013-11-20 03:49:11,642::task::495::TaskManager.Task::(_debug) Task b51e2156-c590-4ef2-8b88-f48f3925f1e9: moving from state preparing -> state finished
>Thread-15915070::DEBUG::2013-11-20 03:49:11,643::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915070::DEBUG::2013-11-20 03:49:11,644::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915070::DEBUG::2013-11-20 03:49:11,644::task::495::TaskManager.Task::(_debug) Task b51e2156-c590-4ef2-8b88-f48f3925f1e9: ref 0 aborting False
>Thread-15915070::INFO::2013-11-20 03:49:11,645::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000694036483765', 'lastCheck': 1384919351.6097369, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0449078083038', 'lastCheck': 1384919344.9546759, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.207326889038', 'lastCheck': 1384919351.1941149, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00080680847168', 'lastCheck': 1384919344.9019179, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:49:13,549::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:49:14,053::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.1036 s, 9.9 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 03:49:14,607::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:49:14,609::task::495::TaskManager.Task::(_debug) Task c7b6ff0c-70fa-4bfa-a0bc-521b8330e39e: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:49:14,611::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`274d92b8-2bb0-42c5-9dc6-61db888d3946`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:49:14,612::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:49:14,613::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:49:14,613::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`274d92b8-2bb0-42c5-9dc6-61db888d3946`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:49:14,614::task::495::TaskManager.Task::(_debug) Task c7b6ff0c-70fa-4bfa-a0bc-521b8330e39e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:49:14,615::task::495::TaskManager.Task::(_debug) Task c7b6ff0c-70fa-4bfa-a0bc-521b8330e39e: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:49:14,616::task::495::TaskManager.Task::(_debug) Task c7b6ff0c-70fa-4bfa-a0bc-521b8330e39e: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:49:14,616::task::495::TaskManager.Task::(_debug) Task c7b6ff0c-70fa-4bfa-a0bc-521b8330e39e: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:49:14,617::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:49:14,617::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:49:14,618::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:49:14,619::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:49:14,619::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:49:14,620::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:49:14,621::task::495::TaskManager.Task::(_debug) Task c7b6ff0c-70fa-4bfa-a0bc-521b8330e39e: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:49:14,621::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15915073::DEBUG::2013-11-20 03:49:15,781::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915073::INFO::2013-11-20 03:49:15,783::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915073::DEBUG::2013-11-20 03:49:15,784::task::495::TaskManager.Task::(_debug) Task c619911e-2d2b-45f4-9589-2ebd8eb0461a: moving from state init -> state preparing
>Thread-15915073::DEBUG::2013-11-20 03:49:15,785::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915073::DEBUG::2013-11-20 03:49:15,785::task::495::TaskManager.Task::(_debug) Task c619911e-2d2b-45f4-9589-2ebd8eb0461a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915073::DEBUG::2013-11-20 03:49:15,786::task::495::TaskManager.Task::(_debug) Task c619911e-2d2b-45f4-9589-2ebd8eb0461a: moving from state preparing -> state finished
>Thread-15915073::DEBUG::2013-11-20 03:49:15,786::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915073::DEBUG::2013-11-20 03:49:15,787::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915073::DEBUG::2013-11-20 03:49:15,788::task::495::TaskManager.Task::(_debug) Task c619911e-2d2b-45f4-9589-2ebd8eb0461a: ref 0 aborting False
>Thread-15915073::INFO::2013-11-20 03:49:15,788::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915074::DEBUG::2013-11-20 03:49:15,793::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915074::INFO::2013-11-20 03:49:15,794::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915074::DEBUG::2013-11-20 03:49:15,794::task::495::TaskManager.Task::(_debug) Task 61e1a9c2-0d7e-4fe2-81b8-8c2ceb80898a: moving from state init -> state preparing
>Thread-15915074::DEBUG::2013-11-20 03:49:15,796::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f83f7b0f-914a-4c68-ba9f-c8be7b2d00b3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915074::DEBUG::2013-11-20 03:49:15,796::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915074::DEBUG::2013-11-20 03:49:15,797::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915074::DEBUG::2013-11-20 03:49:15,798::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f83f7b0f-914a-4c68-ba9f-c8be7b2d00b3`::Granted request
>Thread-15915074::DEBUG::2013-11-20 03:49:15,804::task::495::TaskManager.Task::(_debug) Task 61e1a9c2-0d7e-4fe2-81b8-8c2ceb80898a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915074::DEBUG::2013-11-20 03:49:15,804::task::495::TaskManager.Task::(_debug) Task 61e1a9c2-0d7e-4fe2-81b8-8c2ceb80898a: ref 1 aborting False
>Thread-15915074::WARNING::2013-11-20 03:49:15,806::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915074::DEBUG::2013-11-20 03:49:15,807::task::495::TaskManager.Task::(_debug) Task 61e1a9c2-0d7e-4fe2-81b8-8c2ceb80898a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915074::DEBUG::2013-11-20 03:49:15,808::task::495::TaskManager.Task::(_debug) Task 61e1a9c2-0d7e-4fe2-81b8-8c2ceb80898a: moving from state preparing -> state finished
>Thread-15915074::DEBUG::2013-11-20 03:49:15,808::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915074::DEBUG::2013-11-20 03:49:15,809::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915074::DEBUG::2013-11-20 03:49:15,810::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915074::DEBUG::2013-11-20 03:49:15,810::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915074::DEBUG::2013-11-20 03:49:15,811::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915074::DEBUG::2013-11-20 03:49:15,811::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915074::DEBUG::2013-11-20 03:49:15,812::task::495::TaskManager.Task::(_debug) Task 61e1a9c2-0d7e-4fe2-81b8-8c2ceb80898a: ref 0 aborting False
>Thread-15915074::INFO::2013-11-20 03:49:15,813::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:49:16,064::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:49:16,623::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0816728 s, 12.5 MB/s\n'; <rc> = 0
>Thread-15915075::DEBUG::2013-11-20 03:49:17,626::clientIF::54::vds::(wrapper) [192.168.100.135]::call create with ({'bridge': 'DMZ', 'acpiEnable': 'true', 'emulatedMachine': 'rhel6.2.0', 'vmId': 'dab1628d-5a54-49ba-9685-47e4c156842c', 'transparentHugePages': 'true', 'spiceSslCipherSuite': 'DEFAULT', 'cpuType': 'Westmere', 'smp': '4', 'macAddr': '00:1a:4a:a8:64:1a', 'boot': 'cd', 'custom': {}, 'vmType': 'kvm', 'memSize': 6144, 'smpCoresPerSocket': '2', 'vmName': 'Dili', 'spiceMonitors': '1', 'nice': '0', 'drives': [{'index': '0', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'format': 'raw', 'boot': 'true', 'volumeID': '66b599fc-b7b5-4934-a5d5-ef7283460029', 'imageID': '089b08fc-6544-4235-a061-fb55b09d2715', 'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'propagateErrors': 'off', 'if': 'virtio'}], 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/ubuntu-12.04.1-server-amd64.iso', 'smartcardEnable': 'false', 'nicModel': 'pv', 'keyboardLayout': 'en-us', 'kvmEnable': 'true', 'pitReinjection': 'false', 'timeOffset': '0', 'spiceSecureChannels': 'smain,sinputs', 'display': 'qxl'},) {}
>Thread-15915075::INFO::2013-11-20 03:49:18,102::clientIF::523::vds::(_getNetworkIp) network None: using 0
>Thread-15915075::INFO::2013-11-20 03:49:18,104::clientIF::656::vds::(create) vmContainerLock acquired by vm dab1628d-5a54-49ba-9685-47e4c156842c
>Thread-15915076::DEBUG::2013-11-20 03:49:18,112::vm::342::vm.Vm::(_startUnderlyingVm) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Start
>Thread-15915075::DEBUG::2013-11-20 03:49:18,112::clientIF::669::vds::(create) Total desktops after creation of dab1628d-5a54-49ba-9685-47e4c156842c is 10
>Thread-15915076::DEBUG::2013-11-20 03:49:18,113::vm::346::vm.Vm::(_startUnderlyingVm) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::_ongoingCreations acquired
>Thread-15915075::DEBUG::2013-11-20 03:49:18,114::clientIF::59::vds::(wrapper) return create with {'status': {'message': 'Done', 'code': 0}, 'vmList': {'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/ubuntu-12.04.1-server-amd64.iso', 'bridge': 'DMZ', 'acpiEnable': 'true', 'emulatedMachine': 'rhel6.2.0', 'vmId': 'dab1628d-5a54-49ba-9685-47e4c156842c', 'pid': '0', 'drives': [{'index': '0', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'format': 'raw', 'boot': 'true', 'volumeID': '66b599fc-b7b5-4934-a5d5-ef7283460029', 'imageID': '089b08fc-6544-4235-a061-fb55b09d2715', 'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'propagateErrors': 'off', 'if': 'virtio'}], 'transparentHugePages': 'true', 'keyboardLayout': 'en-us', 'displaySecurePort': '-1', 'displayPort': '-1', 'cpuType': 'Westmere', 'custom': {}, 'clientIp': '', 'nicModel': 'pv', 'macAddr': '00:1a:4a:a8:64:1a', 'kvmEnable': 'true', 'pitReinjection': 'false', 'smartcardEnable': 'false', 'boot': 'cd', 'smp': '4', 'vmType': 'kvm', 'spiceSslCipherSuite': 'DEFAULT', 'status': 'WaitForLaunch', 'timeOffset': '0', 'memSize': 6144, 'displayIp': '0', 'spiceSecureChannels': 'smain,sinputs', 'smpCoresPerSocket': '2', 'vmName': 'Dili', 'display': 'qxl', 'spiceMonitors': '1', 'nice': '0'}}
>Thread-15915076::INFO::2013-11-20 03:49:18,115::libvirtvm::1061::vm.Vm::(_run) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::VM wrapper has started
>Thread-15915076::INFO::2013-11-20 03:49:18,131::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=089b08fc-6544-4235-a061-fb55b09d2715 volUUID=66b599fc-b7b5-4934-a5d5-ef7283460029)
>Thread-15915076::DEBUG::2013-11-20 03:49:18,133::task::495::TaskManager.Task::(_debug) Task 367c82e8-d196-4b84-a169-6fe9b8774cd9: moving from state init -> state preparing
>Thread-15915076::DEBUG::2013-11-20 03:49:18,138::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`27551a05-6788-4b01-87a2-de07a243698b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1690' at 'public_prepareVolume'
>Thread-15915076::DEBUG::2013-11-20 03:49:18,142::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915076::DEBUG::2013-11-20 03:49:18,147::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915076::DEBUG::2013-11-20 03:49:18,149::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`27551a05-6788-4b01-87a2-de07a243698b`::Granted request
>Thread-15915076::DEBUG::2013-11-20 03:49:18,150::task::495::TaskManager.Task::(_debug) Task 367c82e8-d196-4b84-a169-6fe9b8774cd9: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915076::DEBUG::2013-11-20 03:49:18,151::task::495::TaskManager.Task::(_debug) Task 367c82e8-d196-4b84-a169-6fe9b8774cd9: ref 1 aborting False
>Thread-15915076::DEBUG::2013-11-20 03:49:18,155::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.089b08fc-6544-4235-a061-fb55b09d2715`ReqID=`ff729c1f-b9cd-4291-8b60-bd1815551455`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1695' at 'public_prepareVolume'
>Thread-15915076::DEBUG::2013-11-20 03:49:18,157::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.089b08fc-6544-4235-a061-fb55b09d2715' for lock type 'exclusive'
>Thread-15915076::DEBUG::2013-11-20 03:49:18,160::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15915076::DEBUG::2013-11-20 03:49:18,162::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:49:18,640::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915076::DEBUG::2013-11-20 03:49:19,135::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15915076::DEBUG::2013-11-20 03:49:19,548::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15915076::DEBUG::2013-11-20 03:49:19,552::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=4 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:49:19,821::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.707088 s, 1.4 MB/s\n'; <rc> = 0
>Thread-15915076::DEBUG::2013-11-20 03:49:19,960::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000202216 s, 2.5 MB/s\n'; <rc> = 0
>Thread-15915076::DEBUG::2013-11-20 03:49:19,962::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000202216 s, 2.5 MB/s'], size: 512
>Thread-15915076::DEBUG::2013-11-20 03:49:19,966::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=4 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15915076::DEBUG::2013-11-20 03:49:20,425::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000200985 s, 2.5 MB/s\n'; <rc> = 0
>Thread-15915076::DEBUG::2013-11-20 03:49:20,427::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000200985 s, 2.5 MB/s'], size: 512
>Thread-15915076::INFO::2013-11-20 03:49:20,429::image::340::Storage.Image::(getChain) sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 imgUUID=089b08fc-6544-4235-a061-fb55b09d2715 chain=[<storage.blockVolume.BlockVolume instance at 0x7f48ac3848c0>] 
>Thread-15915076::DEBUG::2013-11-20 03:49:20,431::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --autobackup n --available y 6b892067-ac7e-4039-a7be-5b7387525b16/66b599fc-b7b5-4934-a5d5-ef7283460029' (cwd None)
>Thread-15915076::DEBUG::2013-11-20 03:49:21,401::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15915076::DEBUG::2013-11-20 03:49:21,403::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15915076::DEBUG::2013-11-20 03:49:21,404::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15915076::DEBUG::2013-11-20 03:49:21,407::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.66b599fc-b7b5-4934-a5d5-ef7283460029`ReqID=`f00cdbec-3990-4b04-8a6b-f57bc022444a`::Request was made in '/usr/share/vdsm/storage/resourceFactories.py' line '149' at '__getResourceCandidatesList'
>Thread-15915076::DEBUG::2013-11-20 03:49:21,409::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.66b599fc-b7b5-4934-a5d5-ef7283460029' for lock type 'exclusive'
>Thread-15915076::DEBUG::2013-11-20 03:49:21,411::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.66b599fc-b7b5-4934-a5d5-ef7283460029' is free. Now locking as 'exclusive' (1 active user)
>Thread-15915076::DEBUG::2013-11-20 03:49:21,412::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.66b599fc-b7b5-4934-a5d5-ef7283460029`ReqID=`f00cdbec-3990-4b04-8a6b-f57bc022444a`::Granted request
>Thread-15915076::DEBUG::2013-11-20 03:49:21,414::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.089b08fc-6544-4235-a061-fb55b09d2715' is free. Now locking as 'exclusive' (1 active user)
>Thread-15915076::DEBUG::2013-11-20 03:49:21,415::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.089b08fc-6544-4235-a061-fb55b09d2715`ReqID=`ff729c1f-b9cd-4291-8b60-bd1815551455`::Granted request
>Thread-15915076::DEBUG::2013-11-20 03:49:21,417::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15915076::DEBUG::2013-11-20 03:49:21,418::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:49:21,832::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915076::DEBUG::2013-11-20 03:49:22,368::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15915076::DEBUG::2013-11-20 03:49:22,719::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15915076::DEBUG::2013-11-20 03:49:22,728::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --refresh 6b892067-ac7e-4039-a7be-5b7387525b16/66b599fc-b7b5-4934-a5d5-ef7283460029' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:49:23,111::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.810669 s, 1.3 MB/s\n'; <rc> = 0
>Thread-15915076::DEBUG::2013-11-20 03:49:23,836::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15915076::DEBUG::2013-11-20 03:49:23,846::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15915076::DEBUG::2013-11-20 03:49:23,847::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15915076::INFO::2013-11-20 03:49:23,848::volume::517::Storage.Volume::(prepare) Volume: preparing volume 6b892067-ac7e-4039-a7be-5b7387525b16/66b599fc-b7b5-4934-a5d5-ef7283460029
>Thread-15915076::DEBUG::2013-11-20 03:49:23,848::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15915076::DEBUG::2013-11-20 03:49:23,850::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15915076::DEBUG::2013-11-20 03:49:24,769::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912392::INFO::2013-11-20 03:49:25,048::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Dummy-15912651::DEBUG::2013-11-20 03:49:25,140::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::DEBUG::2013-11-20 03:49:25,141::task::495::TaskManager.Task::(_debug) Task bcdc0730-673f-4ce6-88b9-728bca537176: moving from state init -> state preparing
>Thread-15915076::DEBUG::2013-11-20 03:49:25,377::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15912392::DEBUG::2013-11-20 03:49:25,384::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bc39f761-1b61-4988-ad42-d6fc917ffc0d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915076::DEBUG::2013-11-20 03:49:25,400::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=4 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15912392::DEBUG::2013-11-20 03:49:25,402::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:49:25,816::resourceManager::481::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' found in shared state and queue is empty, Joining current shared lock (2 active users)
>Thread-15913695::INFO::2013-11-20 03:49:25,823::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15912392::DEBUG::2013-11-20 03:49:25,826::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bc39f761-1b61-4988-ad42-d6fc917ffc0d`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:49:25,826::task::495::TaskManager.Task::(_debug) Task 51b51177-d0a2-4101-acc5-6d091a00243f: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:49:25,831::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`71083bb8-ce90-454f-95e2-3ddf7b067fa9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:49:25,832::task::495::TaskManager.Task::(_debug) Task bcdc0730-673f-4ce6-88b9-728bca537176: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:49:25,833::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:49:25,834::task::495::TaskManager.Task::(_debug) Task bcdc0730-673f-4ce6-88b9-728bca537176: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:49:25,836::task::495::TaskManager.Task::(_debug) Task bcdc0730-673f-4ce6-88b9-728bca537176: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913695::DEBUG::2013-11-20 03:49:25,836::resourceManager::481::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' found in shared state and queue is empty, Joining current shared lock (3 active users)
>Thread-15912392::DEBUG::2013-11-20 03:49:25,837::task::495::TaskManager.Task::(_debug) Task bcdc0730-673f-4ce6-88b9-728bca537176: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:49:25,837::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`71083bb8-ce90-454f-95e2-3ddf7b067fa9`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:49:25,839::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:49:25,841::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:49:25,842::task::495::TaskManager.Task::(_debug) Task 51b51177-d0a2-4101-acc5-6d091a00243f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:49:25,843::task::495::TaskManager.Task::(_debug) Task 51b51177-d0a2-4101-acc5-6d091a00243f: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:49:25,844::task::495::TaskManager.Task::(_debug) Task 51b51177-d0a2-4101-acc5-6d091a00243f: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15912392::DEBUG::2013-11-20 03:49:25,845::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:49:25,846::task::495::TaskManager.Task::(_debug) Task 51b51177-d0a2-4101-acc5-6d091a00243f: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:49:25,847::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (2 active users)
>Thread-15913695::DEBUG::2013-11-20 03:49:25,847::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:49:25,849::task::495::TaskManager.Task::(_debug) Task bcdc0730-673f-4ce6-88b9-728bca537176: ref 0 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:49:25,849::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:49:25,851::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:49:25,852::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (1 active users)
>Thread-15913695::DEBUG::2013-11-20 03:49:25,853::task::495::TaskManager.Task::(_debug) Task 51b51177-d0a2-4101-acc5-6d091a00243f: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:49:25,854::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15912392::INFO::2013-11-20 03:49:25,850::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:49:25,995::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.138505 s, 7.4 MB/s\n'; <rc> = 0
>Thread-15915076::DEBUG::2013-11-20 03:49:26,068::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000187115 s, 2.7 MB/s\n'; <rc> = 0
>Thread-15915076::DEBUG::2013-11-20 03:49:26,069::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000187115 s, 2.7 MB/s'], size: 512
>Thread-15915076::DEBUG::2013-11-20 03:49:26,070::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=4 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15915076::DEBUG::2013-11-20 03:49:26,485::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000203125 s, 2.5 MB/s\n'; <rc> = 0
>Thread-15915076::DEBUG::2013-11-20 03:49:26,486::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000203125 s, 2.5 MB/s'], size: 512
>Thread-15915076::DEBUG::2013-11-20 03:49:26,488::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.66b599fc-b7b5-4934-a5d5-ef7283460029`ReqID=`0a733e64-ac17-4ca9-921b-f2b2f6960762`::Request was made in '/usr/share/vdsm/storage/volume.py' line '533' at 'prepare'
>Thread-15915076::DEBUG::2013-11-20 03:49:26,489::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.66b599fc-b7b5-4934-a5d5-ef7283460029' for lock type 'exclusive'
>Thread-15915076::DEBUG::2013-11-20 03:49:26,490::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.66b599fc-b7b5-4934-a5d5-ef7283460029' is free. Now locking as 'exclusive' (1 active user)
>Thread-15915076::DEBUG::2013-11-20 03:49:26,490::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.66b599fc-b7b5-4934-a5d5-ef7283460029`ReqID=`0a733e64-ac17-4ca9-921b-f2b2f6960762`::Granted request
>Thread-15915076::DEBUG::2013-11-20 03:49:26,492::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=4 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15915076::DEBUG::2013-11-20 03:49:26,897::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000238389 s, 2.1 MB/s\n'; <rc> = 0
>Thread-15915076::DEBUG::2013-11-20 03:49:26,898::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000238389 s, 2.1 MB/s'], size: 512
>Thread-15915076::DEBUG::2013-11-20 03:49:26,900::task::495::TaskManager.Task::(_debug) Task 367c82e8-d196-4b84-a169-6fe9b8774cd9: finished: None
>Thread-15915076::DEBUG::2013-11-20 03:49:26,901::task::495::TaskManager.Task::(_debug) Task 367c82e8-d196-4b84-a169-6fe9b8774cd9: moving from state preparing -> state finished
>Thread-15915076::DEBUG::2013-11-20 03:49:26,902::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915076::DEBUG::2013-11-20 03:49:26,902::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915076::DEBUG::2013-11-20 03:49:26,903::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915076::DEBUG::2013-11-20 03:49:26,904::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915076::DEBUG::2013-11-20 03:49:26,905::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915076::DEBUG::2013-11-20 03:49:26,906::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915076::DEBUG::2013-11-20 03:49:26,907::task::495::TaskManager.Task::(_debug) Task 367c82e8-d196-4b84-a169-6fe9b8774cd9: ref 0 aborting False
>Thread-15915076::INFO::2013-11-20 03:49:26,908::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, Return response: {'status': {'message': 'OK', 'code': 0}}
>Thread-15915076::INFO::2013-11-20 03:49:26,909::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=089b08fc-6544-4235-a061-fb55b09d2715 volUUID=66b599fc-b7b5-4934-a5d5-ef7283460029)
>Thread-15915076::DEBUG::2013-11-20 03:49:26,910::task::495::TaskManager.Task::(_debug) Task 7c221f26-4da4-4972-a442-4e035eaaf67b: moving from state init -> state preparing
>Thread-15915076::DEBUG::2013-11-20 03:49:26,912::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ed140b7d-5643-4960-a1d0-33fbed8f514c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1666' at 'public_getVolumePath'
>Thread-15915076::DEBUG::2013-11-20 03:49:26,912::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915076::DEBUG::2013-11-20 03:49:26,913::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915076::DEBUG::2013-11-20 03:49:26,914::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ed140b7d-5643-4960-a1d0-33fbed8f514c`::Granted request
>Thread-15915076::DEBUG::2013-11-20 03:49:26,921::task::495::TaskManager.Task::(_debug) Task 7c221f26-4da4-4972-a442-4e035eaaf67b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915076::DEBUG::2013-11-20 03:49:26,922::task::495::TaskManager.Task::(_debug) Task 7c221f26-4da4-4972-a442-4e035eaaf67b: ref 1 aborting False
>Thread-15915076::DEBUG::2013-11-20 03:49:26,923::task::495::TaskManager.Task::(_debug) Task 7c221f26-4da4-4972-a442-4e035eaaf67b: finished: {'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/089b08fc-6544-4235-a061-fb55b09d2715/66b599fc-b7b5-4934-a5d5-ef7283460029'}
>Thread-15915076::DEBUG::2013-11-20 03:49:26,924::task::495::TaskManager.Task::(_debug) Task 7c221f26-4da4-4972-a442-4e035eaaf67b: moving from state preparing -> state finished
>Thread-15915076::DEBUG::2013-11-20 03:49:26,925::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915076::DEBUG::2013-11-20 03:49:26,926::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915076::DEBUG::2013-11-20 03:49:26,927::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915076::DEBUG::2013-11-20 03:49:26,928::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915076::DEBUG::2013-11-20 03:49:26,929::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915076::DEBUG::2013-11-20 03:49:26,929::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915076::DEBUG::2013-11-20 03:49:26,930::task::495::TaskManager.Task::(_debug) Task 7c221f26-4da4-4972-a442-4e035eaaf67b: ref 0 aborting False
>Thread-15915076::INFO::2013-11-20 03:49:26,931::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, Return response: {'status': {'message': 'OK', 'code': 0}, 'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/089b08fc-6544-4235-a061-fb55b09d2715/66b599fc-b7b5-4934-a5d5-ef7283460029'}
>Thread-15915076::INFO::2013-11-20 03:49:26,942::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=089b08fc-6544-4235-a061-fb55b09d2715 volUUID=66b599fc-b7b5-4934-a5d5-ef7283460029)
>Thread-15915076::DEBUG::2013-11-20 03:49:26,943::task::495::TaskManager.Task::(_debug) Task f68d7aba-f3f2-4e06-a27e-a697e6044ed0: moving from state init -> state preparing
>Thread-15915076::DEBUG::2013-11-20 03:49:26,944::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`078ed24e-9023-4f74-a54c-59a000493234`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915076::DEBUG::2013-11-20 03:49:26,945::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915076::DEBUG::2013-11-20 03:49:26,946::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915076::DEBUG::2013-11-20 03:49:26,947::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`078ed24e-9023-4f74-a54c-59a000493234`::Granted request
>Thread-15915076::DEBUG::2013-11-20 03:49:26,948::task::495::TaskManager.Task::(_debug) Task f68d7aba-f3f2-4e06-a27e-a697e6044ed0: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915076::DEBUG::2013-11-20 03:49:26,948::task::495::TaskManager.Task::(_debug) Task f68d7aba-f3f2-4e06-a27e-a697e6044ed0: ref 1 aborting False
>Thread-15915076::DEBUG::2013-11-20 03:49:26,949::task::495::TaskManager.Task::(_debug) Task f68d7aba-f3f2-4e06-a27e-a697e6044ed0: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15915076::DEBUG::2013-11-20 03:49:26,950::task::495::TaskManager.Task::(_debug) Task f68d7aba-f3f2-4e06-a27e-a697e6044ed0: moving from state preparing -> state finished
>Thread-15915076::DEBUG::2013-11-20 03:49:26,951::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915076::DEBUG::2013-11-20 03:49:26,952::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915076::DEBUG::2013-11-20 03:49:26,952::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915076::DEBUG::2013-11-20 03:49:26,953::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915076::DEBUG::2013-11-20 03:49:26,954::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915076::DEBUG::2013-11-20 03:49:26,955::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915076::DEBUG::2013-11-20 03:49:26,956::task::495::TaskManager.Task::(_debug) Task f68d7aba-f3f2-4e06-a27e-a697e6044ed0: ref 0 aborting False
>Thread-15915076::INFO::2013-11-20 03:49:26,957::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15915076::INFO::2013-11-20 03:49:26,958::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16)
>Thread-15915076::DEBUG::2013-11-20 03:49:26,959::task::495::TaskManager.Task::(_debug) Task b82402db-28bd-4713-b00c-0e5a664aacc8: moving from state init -> state preparing
>Thread-15915076::INFO::2013-11-20 03:49:26,960::blockSD::674::Storage.StorageDomain::(validate) sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16
>Thread-15915076::DEBUG::2013-11-20 03:49:26,961::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15915076::DEBUG::2013-11-20 03:49:26,962::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15915076::DEBUG::2013-11-20 03:49:26,962::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15915076::DEBUG::2013-11-20 03:49:26,964::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15915076::DEBUG::2013-11-20 03:49:26,965::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-15915076::DEBUG::2013-11-20 03:49:26,966::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15915081::DEBUG::2013-11-20 03:49:27,553::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915081::INFO::2013-11-20 03:49:27,554::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915081::DEBUG::2013-11-20 03:49:27,555::task::495::TaskManager.Task::(_debug) Task 62e2dcb3-ab83-4b82-a386-e362369bc994: moving from state init -> state preparing
>Thread-15915076::DEBUG::2013-11-20 03:49:27,672::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15915076::DEBUG::2013-11-20 03:49:27,676::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-15915076::DEBUG::2013-11-20 03:49:27,677::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'_SHA_CKSUM=716ea7492d9134a4b13de27ac61a336ad474727c', u'POOL_SPM_ID=2', u'POOL_SPM_LVER=287']
>Thread-15915081::DEBUG::2013-11-20 03:49:27,680::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915076::DEBUG::2013-11-20 03:49:27,681::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8bdb579a-aa2a-4bcf-9de6-8ea34eb40467`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1297' at 'public_getStorageDomainInfo'
>Thread-15915081::DEBUG::2013-11-20 03:49:27,682::task::495::TaskManager.Task::(_debug) Task 62e2dcb3-ab83-4b82-a386-e362369bc994: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915076::DEBUG::2013-11-20 03:49:27,683::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915081::DEBUG::2013-11-20 03:49:27,684::task::495::TaskManager.Task::(_debug) Task 62e2dcb3-ab83-4b82-a386-e362369bc994: moving from state preparing -> state finished
>Thread-15915081::DEBUG::2013-11-20 03:49:27,686::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915076::DEBUG::2013-11-20 03:49:27,687::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915081::DEBUG::2013-11-20 03:49:27,687::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915076::DEBUG::2013-11-20 03:49:27,688::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8bdb579a-aa2a-4bcf-9de6-8ea34eb40467`::Granted request
>Thread-15915081::DEBUG::2013-11-20 03:49:27,689::task::495::TaskManager.Task::(_debug) Task 62e2dcb3-ab83-4b82-a386-e362369bc994: ref 0 aborting False
>Thread-15915076::DEBUG::2013-11-20 03:49:27,691::task::495::TaskManager.Task::(_debug) Task b82402db-28bd-4713-b00c-0e5a664aacc8: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915081::INFO::2013-11-20 03:49:27,692::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915076::DEBUG::2013-11-20 03:49:27,693::task::495::TaskManager.Task::(_debug) Task b82402db-28bd-4713-b00c-0e5a664aacc8: ref 1 aborting False
>Thread-15915076::WARNING::2013-11-20 03:49:27,701::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915076::DEBUG::2013-11-20 03:49:27,703::task::495::TaskManager.Task::(_debug) Task b82402db-28bd-4713-b00c-0e5a664aacc8: finished: {'info': {'uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'vguuid': 'aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', 'lver': 287, 'state': 'OK', 'version': '2', 'role': 'Master', 'master_ver': 1, 'spm_id': 2, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'OS'}}
>Thread-15915082::DEBUG::2013-11-20 03:49:27,705::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915076::DEBUG::2013-11-20 03:49:27,706::task::495::TaskManager.Task::(_debug) Task b82402db-28bd-4713-b00c-0e5a664aacc8: moving from state preparing -> state finished
>Thread-15915082::INFO::2013-11-20 03:49:27,707::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915076::DEBUG::2013-11-20 03:49:27,707::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915082::DEBUG::2013-11-20 03:49:27,708::task::495::TaskManager.Task::(_debug) Task cafd7439-50c8-488f-85cf-3e3a8e2eed3c: moving from state init -> state preparing
>Thread-15915076::DEBUG::2013-11-20 03:49:27,709::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915082::DEBUG::2013-11-20 03:49:27,710::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6becb1b8-ab3a-4aef-b284-31feaf259387`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915082::DEBUG::2013-11-20 03:49:27,711::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915076::DEBUG::2013-11-20 03:49:27,711::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915082::DEBUG::2013-11-20 03:49:27,712::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915082::DEBUG::2013-11-20 03:49:27,713::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6becb1b8-ab3a-4aef-b284-31feaf259387`::Granted request
>Thread-15915076::DEBUG::2013-11-20 03:49:27,713::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915082::DEBUG::2013-11-20 03:49:27,714::task::495::TaskManager.Task::(_debug) Task cafd7439-50c8-488f-85cf-3e3a8e2eed3c: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915076::DEBUG::2013-11-20 03:49:27,715::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915082::DEBUG::2013-11-20 03:49:27,715::task::495::TaskManager.Task::(_debug) Task cafd7439-50c8-488f-85cf-3e3a8e2eed3c: ref 1 aborting False
>Thread-15915076::DEBUG::2013-11-20 03:49:27,716::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915082::WARNING::2013-11-20 03:49:27,717::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915076::DEBUG::2013-11-20 03:49:27,718::task::495::TaskManager.Task::(_debug) Task b82402db-28bd-4713-b00c-0e5a664aacc8: ref 0 aborting False
>Thread-15915082::DEBUG::2013-11-20 03:49:27,719::task::495::TaskManager.Task::(_debug) Task cafd7439-50c8-488f-85cf-3e3a8e2eed3c: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915076::INFO::2013-11-20 03:49:27,720::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'vguuid': 'aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', 'lver': 287, 'state': 'OK', 'version': '2', 'role': 'Master', 'master_ver': 1, 'spm_id': 2, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'OS'}}
>Thread-15915082::DEBUG::2013-11-20 03:49:27,721::task::495::TaskManager.Task::(_debug) Task cafd7439-50c8-488f-85cf-3e3a8e2eed3c: moving from state preparing -> state finished
>Thread-15915082::DEBUG::2013-11-20 03:49:27,739::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915082::DEBUG::2013-11-20 03:49:27,749::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915082::DEBUG::2013-11-20 03:49:27,750::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915076::DEBUG::2013-11-20 03:49:27,751::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/libexec/vdsm/hooks/before_vm_start/50_vhostmd' (cwd None)
>Thread-15915082::DEBUG::2013-11-20 03:49:27,751::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915082::DEBUG::2013-11-20 03:49:27,752::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915082::DEBUG::2013-11-20 03:49:27,972::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915082::DEBUG::2013-11-20 03:49:27,974::task::495::TaskManager.Task::(_debug) Task cafd7439-50c8-488f-85cf-3e3a8e2eed3c: ref 0 aborting False
>Thread-15915082::INFO::2013-11-20 03:49:27,976::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:49:28,014::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915076::DEBUG::2013-11-20 03:49:28,334::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15915076::INFO::2013-11-20 03:49:28,335::hooks::51::root::(_runHooksDir) 
>Thread-15915076::DEBUG::2013-11-20 03:49:28,337::libvirtvm::1077::vm.Vm::(_run) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::<?xml version="1.0" encoding="utf-8"?>
><domain type="kvm">
>	<name>Dili</name>
>	<uuid>dab1628d-5a54-49ba-9685-47e4c156842c</uuid>
>	<memory>6291456</memory>
>	<currentMemory>6291456</currentMemory>
>	<vcpu>4</vcpu>
>	<devices>
>		<disk device="disk" type="block">
>			<source dev="/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/089b08fc-6544-4235-a061-fb55b09d2715/66b599fc-b7b5-4934-a5d5-ef7283460029"/>
>			<target bus="virtio" dev="vda"/>
>			<serial>35-a061-fb55b09d2715</serial>
>			<driver cache="none" error_policy="stop" io="native" name="qemu" type="raw"/>
>		</disk>
>		<disk device="cdrom" type="file">
>			<source file="/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/ubuntu-12.04.1-server-amd64.iso"/>
>			<target bus="ide" dev="hdc"/>
>		</disk>
>		<controller index="0" ports="16" type="virtio-serial"/>
>		<channel type="unix">
>			<target name="com.redhat.rhevm.vdsm" type="virtio"/>
>			<source mode="bind" path="/var/lib/libvirt/qemu/channels/Dili.com.redhat.rhevm.vdsm"/>
>		</channel>
>		<memballoon model="none"/>
>		<interface type="bridge">
>			<mac address="00:1a:4a:a8:64:1a"/>
>			<model type="virtio"/>
>			<source bridge="DMZ"/>
>		</interface>
>		<input bus="ps2" type="mouse"/>
>		<video>
>			<model heads="1" type="qxl" vram="65536"/>
>		</video>
>		<channel type="spicevmc">
>			<target name="com.redhat.spice.0" type="virtio"/>
>		</channel>
>		<graphics autoport="yes" keymap="en-us" listen="0" passwd="*****" passwdValidTo="1970-01-01T00:00:01" port="-1" tlsPort="-1" type="spice">
>			<channel mode="secure" name="main"/>
>			<channel mode="secure" name="inputs"/>
>		</graphics>
>	</devices>
>	<os>
>		<type arch="x86_64" machine="rhel6.2.0">hvm</type>
>		<boot dev="hd"/>
>		<boot dev="cdrom"/>
>		<smbios mode="sysinfo"/>
>	</os>
>	<sysinfo type="smbios">
>		<system>
>			<entry name="manufacturer">Red Hat</entry>
>			<entry name="product">RHEV Hypervisor</entry>
>			<entry name="version">6.3-20121012.0.el6_3</entry>
>			<entry name="serial">30343536-3138-5A43-4A32-333630373442_AC:16:2D:76:EB:F0</entry>
>			<entry name="uuid">dab1628d-5a54-49ba-9685-47e4c156842c</entry>
>		</system>
>	</sysinfo>
>	<clock adjustment="0" offset="variable">
>		<timer name="rtc" tickpolicy="catchup"/>
>	</clock>
>	<features>
>		<acpi/>
>	</features>
>	<cpu match="exact">
>		<model>Westmere</model>
>		<topology cores="2" sockets="2" threads="1"/>
>	</cpu>
></domain>
>
>Thread-15915083::INFO::2013-11-20 03:49:28,464::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915083::DEBUG::2013-11-20 03:49:28,466::task::495::TaskManager.Task::(_debug) Task 9c823f0b-a752-4e75-9db5-34746a64edad: moving from state init -> state preparing
>Thread-15915083::DEBUG::2013-11-20 03:49:28,467::task::495::TaskManager.Task::(_debug) Task 9c823f0b-a752-4e75-9db5-34746a64edad: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000877857208252', 'lastCheck': 1384919362.03878, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00756597518921', 'lastCheck': 1384919365.8265829, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00198101997375', 'lastCheck': 1384919361.414196, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00476002693176', 'lastCheck': 1384919365.3935709, 'valid': True, 'code': 0}}
>Thread-15915083::DEBUG::2013-11-20 03:49:28,468::task::495::TaskManager.Task::(_debug) Task 9c823f0b-a752-4e75-9db5-34746a64edad: moving from state preparing -> state finished
>Thread-15915083::DEBUG::2013-11-20 03:49:28,469::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915083::DEBUG::2013-11-20 03:49:28,469::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915083::DEBUG::2013-11-20 03:49:28,470::task::495::TaskManager.Task::(_debug) Task 9c823f0b-a752-4e75-9db5-34746a64edad: ref 0 aborting False
>Thread-15915083::INFO::2013-11-20 03:49:28,471::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000877857208252', 'lastCheck': 1384919362.03878, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00756597518921', 'lastCheck': 1384919365.8265829, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00198101997375', 'lastCheck': 1384919361.414196, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00476002693176', 'lastCheck': 1384919365.3935709, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:49:28,643::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.087907 s, 11.6 MB/s\n'; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-20 03:49:29,755::libvirtvm::1401::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::event Started detail 0 opaque None
>Thread-15915076::DEBUG::2013-11-20 03:49:29,804::utils::307::vm.Vm::(start) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Start statistics collection
>Thread-15915085::DEBUG::2013-11-20 03:49:29,805::utils::336::vm.Vm::(run) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Stats thread started
>Thread-15915086::DEBUG::2013-11-20 03:49:29,806::guestIF::47::root::(_connect) Attempting connection to /var/lib/libvirt/qemu/channels/Dili.com.redhat.rhevm.vdsm
>Thread-15915085::INFO::2013-11-20 03:49:29,809::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=089b08fc-6544-4235-a061-fb55b09d2715 volUUID=66b599fc-b7b5-4934-a5d5-ef7283460029)
>Thread-15915086::DEBUG::2013-11-20 03:49:29,816::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /usr/share/vdsm/prepare-vmchannel /var/lib/libvirt/qemu/channels/Dili.com.redhat.rhevm.vdsm' (cwd None)
>Thread-15915085::DEBUG::2013-11-20 03:49:29,817::task::495::TaskManager.Task::(_debug) Task 75395982-905e-49be-ba61-812e3123a11c: moving from state init -> state preparing
>Thread-15915076::WARNING::2013-11-20 03:49:29,818::libvirtvm::1124::vm.Vm::(_readPauseCode) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::_readPauseCode unsupported by libvirt vm
>Thread-15915085::DEBUG::2013-11-20 03:49:30,033::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ec5d1dfc-a5a6-4ea3-8102-b7ec4ee9786e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915076::DEBUG::2013-11-20 03:49:30,035::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /usr/share/vdsm/get-vm-pid Dili' (cwd None)
>Thread-15915085::DEBUG::2013-11-20 03:49:30,036::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915085::DEBUG::2013-11-20 03:49:30,492::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915086::DEBUG::2013-11-20 03:49:30,499::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15915085::DEBUG::2013-11-20 03:49:30,500::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ec5d1dfc-a5a6-4ea3-8102-b7ec4ee9786e`::Granted request
>Thread-15915086::DEBUG::2013-11-20 03:49:30,504::guestIF::53::root::(_connect) connected to /var/lib/libvirt/qemu/channels/Dili.com.redhat.rhevm.vdsm
>Thread-15915085::DEBUG::2013-11-20 03:49:30,505::task::495::TaskManager.Task::(_debug) Task 75395982-905e-49be-ba61-812e3123a11c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915085::DEBUG::2013-11-20 03:49:30,506::task::495::TaskManager.Task::(_debug) Task 75395982-905e-49be-ba61-812e3123a11c: ref 1 aborting False
>Thread-15915085::DEBUG::2013-11-20 03:49:30,508::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15915085::DEBUG::2013-11-20 03:49:30,511::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:49:30,751::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915076::DEBUG::2013-11-20 03:49:31,140::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15915076::DEBUG::2013-11-20 03:49:31,168::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /sbin/service ksmtuned retune' (cwd None)
>Thread-15915085::DEBUG::2013-11-20 03:49:31,643::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15915076::DEBUG::2013-11-20 03:49:31,857::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15915085::DEBUG::2013-11-20 03:49:31,897::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15915076::DEBUG::2013-11-20 03:49:31,899::vm::358::vm.Vm::(_startUnderlyingVm) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::_ongoingCreations released
>Thread-15915085::DEBUG::2013-11-20 03:49:31,907::task::495::TaskManager.Task::(_debug) Task 75395982-905e-49be-ba61-812e3123a11c: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15915085::DEBUG::2013-11-20 03:49:31,909::task::495::TaskManager.Task::(_debug) Task 75395982-905e-49be-ba61-812e3123a11c: moving from state preparing -> state finished
>Thread-15915085::DEBUG::2013-11-20 03:49:31,912::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915085::DEBUG::2013-11-20 03:49:31,913::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915085::DEBUG::2013-11-20 03:49:31,915::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915085::DEBUG::2013-11-20 03:49:31,917::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915085::DEBUG::2013-11-20 03:49:31,918::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915085::DEBUG::2013-11-20 03:49:31,920::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915085::DEBUG::2013-11-20 03:49:31,923::task::495::TaskManager.Task::(_debug) Task 75395982-905e-49be-ba61-812e3123a11c: ref 0 aborting False
>Thread-15915085::INFO::2013-11-20 03:49:31,924::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Dummy-15912651::DEBUG::2013-11-20 03:49:32,037::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.420354 s, 2.4 MB/s\n'; <rc> = 0
>Thread-15915088::DEBUG::2013-11-20 03:49:32,370::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('dab1628d-5a54-49ba-9685-47e4c156842c',) {}
>Thread-15915088::DEBUG::2013-11-20 03:49:32,371::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::CPU stats not available
>Thread-15915088::DEBUG::2013-11-20 03:49:32,372::libvirtvm::202::vm.Vm::(_getNetworkStats) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Network stats not available
>Thread-15915088::DEBUG::2013-11-20 03:49:32,372::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Disk stats not available
>Thread-15915088::DEBUG::2013-11-20 03:49:32,373::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Disk latency not available
>Thread-15915088::DEBUG::2013-11-20 03:49:32,374::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '22921', 'displayIp': '0', 'displayPort': u'5918', 'session': 'Unknown', 'displaySecurePort': u'5919', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/ubuntu-12.04.1-server-amd64.iso', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet10': {'state': 'unknown', 'macAddr': u'00:1a:4a:a8:64:1a', 'speed': '1000', 'name': u'vnet10'}}, 'vmId': 'dab1628d-5a54-49ba-9685-47e4c156842c', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '089b08fc-6544-4235-a061-fb55b09d2715', 'flushLatency': '0', 'truesize': '107374182400'}}, 'boot': 'cd', 'statsAge': '2.56', 'cpuIdle': '100.00', 'elapsedTime': '14', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '0', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:49:34,088::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:49:34,616::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0780084 s, 13.1 MB/s\n'; <rc> = 0
>Thread-15915090::DEBUG::2013-11-20 03:49:35,532::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('dab1628d-5a54-49ba-9685-47e4c156842c',) {}
>Thread-15915090::DEBUG::2013-11-20 03:49:35,533::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::CPU stats not available
>Thread-15915090::DEBUG::2013-11-20 03:49:35,533::libvirtvm::202::vm.Vm::(_getNetworkStats) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Network stats not available
>Thread-15915090::DEBUG::2013-11-20 03:49:35,534::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Disk stats not available
>Thread-15915090::DEBUG::2013-11-20 03:49:35,535::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Disk latency not available
>Thread-15915090::DEBUG::2013-11-20 03:49:35,535::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '22921', 'displayIp': '0', 'displayPort': u'5918', 'session': 'Unknown', 'displaySecurePort': u'5919', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/ubuntu-12.04.1-server-amd64.iso', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet10': {'state': 'unknown', 'macAddr': u'00:1a:4a:a8:64:1a', 'speed': '1000', 'name': u'vnet10'}}, 'vmId': 'dab1628d-5a54-49ba-9685-47e4c156842c', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '089b08fc-6544-4235-a061-fb55b09d2715', 'flushLatency': '0', 'truesize': '107374182400'}}, 'boot': 'cd', 'statsAge': '1.57', 'cpuIdle': '100.00', 'elapsedTime': '17', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '0', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15914435::INFO::2013-11-20 03:49:36,584::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 03:49:36,584::task::495::TaskManager.Task::(_debug) Task d620993f-25b2-4578-8450-0bbdb72b28e1: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 03:49:36,586::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`cee603ee-2fae-422c-bd00-871427156e7a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 03:49:36,586::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 03:49:36,587::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 03:49:36,587::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`cee603ee-2fae-422c-bd00-871427156e7a`::Granted request
>Thread-15914435::DEBUG::2013-11-20 03:49:36,588::task::495::TaskManager.Task::(_debug) Task d620993f-25b2-4578-8450-0bbdb72b28e1: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 03:49:36,588::task::495::TaskManager.Task::(_debug) Task d620993f-25b2-4578-8450-0bbdb72b28e1: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 03:49:36,589::task::495::TaskManager.Task::(_debug) Task d620993f-25b2-4578-8450-0bbdb72b28e1: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 03:49:36,589::task::495::TaskManager.Task::(_debug) Task d620993f-25b2-4578-8450-0bbdb72b28e1: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 03:49:36,589::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 03:49:36,590::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 03:49:36,590::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 03:49:36,591::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 03:49:36,591::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 03:49:36,591::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 03:49:36,592::task::495::TaskManager.Task::(_debug) Task d620993f-25b2-4578-8450-0bbdb72b28e1: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 03:49:36,592::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 03:49:36,632::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:49:37,098::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.047633 s, 21.5 MB/s\n'; <rc> = 0
>Thread-15914453::INFO::2013-11-20 03:49:37,551::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 03:49:37,552::task::495::TaskManager.Task::(_debug) Task 54978503-1d1f-4e69-9886-9d1b3095b530: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 03:49:37,553::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5165fe11-b7ea-47b9-848d-fa2bff9ede27`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 03:49:37,554::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 03:49:37,555::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 03:49:37,555::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5165fe11-b7ea-47b9-848d-fa2bff9ede27`::Granted request
>Thread-15914453::DEBUG::2013-11-20 03:49:37,557::task::495::TaskManager.Task::(_debug) Task 54978503-1d1f-4e69-9886-9d1b3095b530: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 03:49:37,557::task::495::TaskManager.Task::(_debug) Task 54978503-1d1f-4e69-9886-9d1b3095b530: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 03:49:37,557::task::495::TaskManager.Task::(_debug) Task 54978503-1d1f-4e69-9886-9d1b3095b530: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 03:49:37,558::task::495::TaskManager.Task::(_debug) Task 54978503-1d1f-4e69-9886-9d1b3095b530: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 03:49:37,558::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 03:49:37,559::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 03:49:37,559::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 03:49:37,560::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 03:49:37,560::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 03:49:37,560::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 03:49:37,561::task::495::TaskManager.Task::(_debug) Task 54978503-1d1f-4e69-9886-9d1b3095b530: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 03:49:37,561::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15915091::DEBUG::2013-11-20 03:49:38,381::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915091::INFO::2013-11-20 03:49:38,382::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915091::DEBUG::2013-11-20 03:49:38,388::task::495::TaskManager.Task::(_debug) Task bcd9072a-f3c4-49cd-a152-839c0c91433d: moving from state init -> state preparing
>Thread-15915091::DEBUG::2013-11-20 03:49:38,389::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915091::DEBUG::2013-11-20 03:49:38,389::task::495::TaskManager.Task::(_debug) Task bcd9072a-f3c4-49cd-a152-839c0c91433d: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915091::DEBUG::2013-11-20 03:49:38,390::task::495::TaskManager.Task::(_debug) Task bcd9072a-f3c4-49cd-a152-839c0c91433d: moving from state preparing -> state finished
>Thread-15915091::DEBUG::2013-11-20 03:49:38,390::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915091::DEBUG::2013-11-20 03:49:38,390::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915091::DEBUG::2013-11-20 03:49:38,391::task::495::TaskManager.Task::(_debug) Task bcd9072a-f3c4-49cd-a152-839c0c91433d: ref 0 aborting False
>Thread-15915091::INFO::2013-11-20 03:49:38,391::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915092::DEBUG::2013-11-20 03:49:38,395::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915092::INFO::2013-11-20 03:49:38,396::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915092::DEBUG::2013-11-20 03:49:38,396::task::495::TaskManager.Task::(_debug) Task e9ef1555-5940-4198-a04c-47b3eafada4c: moving from state init -> state preparing
>Thread-15915092::DEBUG::2013-11-20 03:49:38,397::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f3197bfb-ccde-438b-99df-21dd676d1f3b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915092::DEBUG::2013-11-20 03:49:38,397::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915092::DEBUG::2013-11-20 03:49:38,398::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915092::DEBUG::2013-11-20 03:49:38,398::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f3197bfb-ccde-438b-99df-21dd676d1f3b`::Granted request
>Thread-15915092::DEBUG::2013-11-20 03:49:38,399::task::495::TaskManager.Task::(_debug) Task e9ef1555-5940-4198-a04c-47b3eafada4c: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915092::DEBUG::2013-11-20 03:49:38,400::task::495::TaskManager.Task::(_debug) Task e9ef1555-5940-4198-a04c-47b3eafada4c: ref 1 aborting False
>Thread-15915092::WARNING::2013-11-20 03:49:38,401::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915092::DEBUG::2013-11-20 03:49:38,402::task::495::TaskManager.Task::(_debug) Task e9ef1555-5940-4198-a04c-47b3eafada4c: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915092::DEBUG::2013-11-20 03:49:38,402::task::495::TaskManager.Task::(_debug) Task e9ef1555-5940-4198-a04c-47b3eafada4c: moving from state preparing -> state finished
>Thread-15915092::DEBUG::2013-11-20 03:49:38,403::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915092::DEBUG::2013-11-20 03:49:38,403::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915092::DEBUG::2013-11-20 03:49:38,404::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915092::DEBUG::2013-11-20 03:49:38,404::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915092::DEBUG::2013-11-20 03:49:38,404::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915092::DEBUG::2013-11-20 03:49:38,405::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915092::DEBUG::2013-11-20 03:49:38,405::task::495::TaskManager.Task::(_debug) Task e9ef1555-5940-4198-a04c-47b3eafada4c: ref 0 aborting False
>Thread-15915092::INFO::2013-11-20 03:49:38,406::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915094::DEBUG::2013-11-20 03:49:38,558::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('dab1628d-5a54-49ba-9685-47e4c156842c',) {}
>Thread-15915094::DEBUG::2013-11-20 03:49:38,559::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::CPU stats not available
>Thread-15915094::DEBUG::2013-11-20 03:49:38,559::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Disk stats not available
>Thread-15915094::DEBUG::2013-11-20 03:49:38,560::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Disk latency not available
>Thread-15915094::DEBUG::2013-11-20 03:49:38,560::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '22921', 'displayIp': '0', 'displayPort': u'5918', 'session': 'Unknown', 'displaySecurePort': u'5919', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/ubuntu-12.04.1-server-amd64.iso', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet10': {'macAddr': u'00:1a:4a:a8:64:1a', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet10'}}, 'vmId': 'dab1628d-5a54-49ba-9685-47e4c156842c', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '089b08fc-6544-4235-a061-fb55b09d2715', 'flushLatency': '0', 'truesize': '107374182400'}}, 'boot': 'cd', 'statsAge': '0.59', 'cpuIdle': '100.00', 'elapsedTime': '20', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '0', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:49:39,113::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:49:39,666::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0902724 s, 11.3 MB/s\n'; <rc> = 0
>Thread-15915096::DEBUG::2013-11-20 03:49:41,586::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('dab1628d-5a54-49ba-9685-47e4c156842c',) {}
>Thread-15915096::DEBUG::2013-11-20 03:49:41,590::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::CPU stats not available
>Thread-15915096::DEBUG::2013-11-20 03:49:41,591::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Disk stats not available
>Thread-15915096::DEBUG::2013-11-20 03:49:41,591::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Disk latency not available
>Thread-15915096::DEBUG::2013-11-20 03:49:41,592::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '22921', 'displayIp': '0', 'displayPort': u'5918', 'session': 'Unknown', 'displaySecurePort': u'5919', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/ubuntu-12.04.1-server-amd64.iso', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet10': {'macAddr': u'00:1a:4a:a8:64:1a', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet10'}}, 'vmId': 'dab1628d-5a54-49ba-9685-47e4c156842c', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '089b08fc-6544-4235-a061-fb55b09d2715', 'flushLatency': '0', 'truesize': '107374182400'}}, 'boot': 'cd', 'statsAge': '1.62', 'cpuIdle': '100.00', 'elapsedTime': '23', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '0', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:49:41,680::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:49:42,262::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0757471 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15914519::INFO::2013-11-20 03:49:42,495::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 03:49:42,496::task::495::TaskManager.Task::(_debug) Task d2a11c29-2e9b-4f8a-8cb4-d8fd0d974c1e: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 03:49:42,506::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`24af6cc1-1158-4f38-a783-64bb2edf5747`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 03:49:42,507::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 03:49:42,508::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 03:49:42,509::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`24af6cc1-1158-4f38-a783-64bb2edf5747`::Granted request
>Thread-15914519::DEBUG::2013-11-20 03:49:42,510::task::495::TaskManager.Task::(_debug) Task d2a11c29-2e9b-4f8a-8cb4-d8fd0d974c1e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 03:49:42,510::task::495::TaskManager.Task::(_debug) Task d2a11c29-2e9b-4f8a-8cb4-d8fd0d974c1e: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 03:49:42,511::task::495::TaskManager.Task::(_debug) Task d2a11c29-2e9b-4f8a-8cb4-d8fd0d974c1e: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 03:49:42,511::task::495::TaskManager.Task::(_debug) Task d2a11c29-2e9b-4f8a-8cb4-d8fd0d974c1e: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 03:49:42,511::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 03:49:42,512::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 03:49:42,512::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 03:49:42,513::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 03:49:42,513::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 03:49:42,513::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 03:49:42,514::task::495::TaskManager.Task::(_debug) Task d2a11c29-2e9b-4f8a-8cb4-d8fd0d974c1e: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 03:49:42,514::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 03:49:44,276::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915097::INFO::2013-11-20 03:49:44,617::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915097::DEBUG::2013-11-20 03:49:44,619::task::495::TaskManager.Task::(_debug) Task 5c7fd0c2-607d-46c8-b996-72e31f15b4e6: moving from state init -> state preparing
>Thread-15915097::DEBUG::2013-11-20 03:49:44,620::task::495::TaskManager.Task::(_debug) Task 5c7fd0c2-607d-46c8-b996-72e31f15b4e6: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000322103500366', 'lastCheck': 1384919382.5757279, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00210404396057', 'lastCheck': 1384919376.2749281, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00144219398499', 'lastCheck': 1384919382.3271019, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000333070755005', 'lastCheck': 1384919375.548059, 'valid': True, 'code': 0}}
>Thread-15915097::DEBUG::2013-11-20 03:49:44,621::task::495::TaskManager.Task::(_debug) Task 5c7fd0c2-607d-46c8-b996-72e31f15b4e6: moving from state preparing -> state finished
>Thread-15915097::DEBUG::2013-11-20 03:49:44,622::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915097::DEBUG::2013-11-20 03:49:44,622::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915097::DEBUG::2013-11-20 03:49:44,623::task::495::TaskManager.Task::(_debug) Task 5c7fd0c2-607d-46c8-b996-72e31f15b4e6: ref 0 aborting False
>Thread-15915097::INFO::2013-11-20 03:49:44,623::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000322103500366', 'lastCheck': 1384919382.5757279, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00210404396057', 'lastCheck': 1384919376.2749281, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00144219398499', 'lastCheck': 1384919382.3271019, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000333070755005', 'lastCheck': 1384919375.548059, 'valid': True, 'code': 0}}
>Thread-15915098::DEBUG::2013-11-20 03:49:44,664::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::CPU stats not available
>Thread-15915098::DEBUG::2013-11-20 03:49:44,666::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Disk stats not available
>Thread-15915098::DEBUG::2013-11-20 03:49:44,666::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Disk latency not available
>Dummy-15912651::DEBUG::2013-11-20 03:49:44,792::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.097312 s, 10.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:49:46,807::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 03:49:47,561::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:49:47,562::task::495::TaskManager.Task::(_debug) Task 5b83b6ff-1023-4f6c-896b-558db22ed5b7: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:49:47,564::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c5e98814-a907-4c9c-8170-e742d6745383`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:49:47,565::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:49:47,565::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:49:47,566::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c5e98814-a907-4c9c-8170-e742d6745383`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:49:47,567::task::495::TaskManager.Task::(_debug) Task 5b83b6ff-1023-4f6c-896b-558db22ed5b7: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:49:47,567::task::495::TaskManager.Task::(_debug) Task 5b83b6ff-1023-4f6c-896b-558db22ed5b7: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:49:47,568::task::495::TaskManager.Task::(_debug) Task 5b83b6ff-1023-4f6c-896b-558db22ed5b7: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:49:47,569::task::495::TaskManager.Task::(_debug) Task 5b83b6ff-1023-4f6c-896b-558db22ed5b7: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:49:47,569::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:49:47,570::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:49:47,570::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:49:47,571::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:49:47,571::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:49:47,572::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:49:47,573::task::495::TaskManager.Task::(_debug) Task 5b83b6ff-1023-4f6c-896b-558db22ed5b7: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:49:47,573::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:49:47,574::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:49:47,574::task::495::TaskManager.Task::(_debug) Task a048286c-b515-4001-b48b-320a39f73646: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:49:47,575::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6e56d339-e04b-4e29-b814-ce3fe247a72e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:49:47,576::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:49:47,576::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:49:47,577::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6e56d339-e04b-4e29-b814-ce3fe247a72e`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:49:47,577::task::495::TaskManager.Task::(_debug) Task a048286c-b515-4001-b48b-320a39f73646: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:49:47,578::task::495::TaskManager.Task::(_debug) Task a048286c-b515-4001-b48b-320a39f73646: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:49:47,579::task::495::TaskManager.Task::(_debug) Task a048286c-b515-4001-b48b-320a39f73646: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:49:47,579::task::495::TaskManager.Task::(_debug) Task a048286c-b515-4001-b48b-320a39f73646: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:49:47,580::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:49:47,581::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:49:47,581::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:49:47,582::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:49:47,582::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:49:47,582::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:49:47,583::task::495::TaskManager.Task::(_debug) Task a048286c-b515-4001-b48b-320a39f73646: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:49:47,583::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:49:47,584::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:49:47,584::task::495::TaskManager.Task::(_debug) Task cba1ad5c-9031-40ec-9e03-441b4e234319: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:49:47,585::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`96bd93b7-5c70-4256-9699-939e85ad905f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:49:47,585::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:49:47,585::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:49:47,586::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`96bd93b7-5c70-4256-9699-939e85ad905f`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:49:47,586::task::495::TaskManager.Task::(_debug) Task cba1ad5c-9031-40ec-9e03-441b4e234319: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:49:47,587::task::495::TaskManager.Task::(_debug) Task cba1ad5c-9031-40ec-9e03-441b4e234319: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:49:47,587::task::495::TaskManager.Task::(_debug) Task cba1ad5c-9031-40ec-9e03-441b4e234319: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:49:47,588::task::495::TaskManager.Task::(_debug) Task cba1ad5c-9031-40ec-9e03-441b4e234319: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:49:47,588::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:49:47,589::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:49:47,589::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:49:47,589::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:49:47,590::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:49:47,590::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:49:47,591::task::495::TaskManager.Task::(_debug) Task cba1ad5c-9031-40ec-9e03-441b4e234319: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:49:47,591::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:49:47,592::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:49:47,592::task::495::TaskManager.Task::(_debug) Task 596d762a-8740-4905-9515-0e43fb321904: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:49:47,593::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`133dbe01-0a4c-4eac-aa79-6fc511fb2e1f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:49:47,594::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:49:47,594::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:49:47,595::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`133dbe01-0a4c-4eac-aa79-6fc511fb2e1f`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:49:47,596::task::495::TaskManager.Task::(_debug) Task 596d762a-8740-4905-9515-0e43fb321904: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:49:47,597::task::495::TaskManager.Task::(_debug) Task 596d762a-8740-4905-9515-0e43fb321904: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:49:47,597::task::495::TaskManager.Task::(_debug) Task 596d762a-8740-4905-9515-0e43fb321904: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:49:47,597::task::495::TaskManager.Task::(_debug) Task 596d762a-8740-4905-9515-0e43fb321904: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:49:47,598::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:49:47,598::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:49:47,599::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:49:47,599::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:49:47,599::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:49:47,600::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:49:47,600::task::495::TaskManager.Task::(_debug) Task 596d762a-8740-4905-9515-0e43fb321904: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:49:47,601::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15915100::DEBUG::2013-11-20 03:49:48,174::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('dab1628d-5a54-49ba-9685-47e4c156842c',) {}
>Thread-15915100::DEBUG::2013-11-20 03:49:48,175::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Disk stats not available
>Thread-15915100::DEBUG::2013-11-20 03:49:48,175::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Disk latency not available
>Thread-15915100::DEBUG::2013-11-20 03:49:48,176::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '22921', 'displayIp': '0', 'displayPort': u'5918', 'session': 'Unknown', 'displaySecurePort': u'5919', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/ubuntu-12.04.1-server-amd64.iso', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet10': {'macAddr': u'00:1a:4a:a8:64:1a', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet10'}}, 'vmId': 'dab1628d-5a54-49ba-9685-47e4c156842c', 'monitorResponse': '0', 'cpuUser': '132.92', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '089b08fc-6544-4235-a061-fb55b09d2715', 'flushLatency': '0', 'truesize': '107374182400'}}, 'boot': 'cd', 'statsAge': '0.20', 'cpuIdle': '0.00', 'elapsedTime': '30', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '0', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:49:48,639::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0911352 s, 11.2 MB/s\n'; <rc> = 0
>Thread-15915101::DEBUG::2013-11-20 03:49:48,931::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915101::INFO::2013-11-20 03:49:48,932::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915101::DEBUG::2013-11-20 03:49:48,933::task::495::TaskManager.Task::(_debug) Task 388f98c3-3b5e-4dd7-af86-140b707d652c: moving from state init -> state preparing
>Thread-15915101::DEBUG::2013-11-20 03:49:48,934::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915101::DEBUG::2013-11-20 03:49:48,934::task::495::TaskManager.Task::(_debug) Task 388f98c3-3b5e-4dd7-af86-140b707d652c: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915101::DEBUG::2013-11-20 03:49:48,935::task::495::TaskManager.Task::(_debug) Task 388f98c3-3b5e-4dd7-af86-140b707d652c: moving from state preparing -> state finished
>Thread-15915101::DEBUG::2013-11-20 03:49:48,936::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915101::DEBUG::2013-11-20 03:49:48,936::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915101::DEBUG::2013-11-20 03:49:48,937::task::495::TaskManager.Task::(_debug) Task 388f98c3-3b5e-4dd7-af86-140b707d652c: ref 0 aborting False
>Thread-15915101::INFO::2013-11-20 03:49:48,938::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915102::DEBUG::2013-11-20 03:49:48,943::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915102::INFO::2013-11-20 03:49:48,944::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915102::DEBUG::2013-11-20 03:49:48,945::task::495::TaskManager.Task::(_debug) Task bebab185-2e10-4e34-86dd-bb681d2232ac: moving from state init -> state preparing
>Thread-15915102::DEBUG::2013-11-20 03:49:48,946::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`428c579f-c4d0-4af4-b31a-3380d5666b98`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915102::DEBUG::2013-11-20 03:49:48,947::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915102::DEBUG::2013-11-20 03:49:48,948::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915102::DEBUG::2013-11-20 03:49:48,948::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`428c579f-c4d0-4af4-b31a-3380d5666b98`::Granted request
>Thread-15915102::DEBUG::2013-11-20 03:49:48,949::task::495::TaskManager.Task::(_debug) Task bebab185-2e10-4e34-86dd-bb681d2232ac: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915102::DEBUG::2013-11-20 03:49:48,950::task::495::TaskManager.Task::(_debug) Task bebab185-2e10-4e34-86dd-bb681d2232ac: ref 1 aborting False
>Thread-15915102::WARNING::2013-11-20 03:49:48,953::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915102::DEBUG::2013-11-20 03:49:48,954::task::495::TaskManager.Task::(_debug) Task bebab185-2e10-4e34-86dd-bb681d2232ac: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915102::DEBUG::2013-11-20 03:49:48,956::task::495::TaskManager.Task::(_debug) Task bebab185-2e10-4e34-86dd-bb681d2232ac: moving from state preparing -> state finished
>Thread-15915102::DEBUG::2013-11-20 03:49:48,956::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915102::DEBUG::2013-11-20 03:49:48,957::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915102::DEBUG::2013-11-20 03:49:48,958::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915102::DEBUG::2013-11-20 03:49:48,959::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915102::DEBUG::2013-11-20 03:49:48,959::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915102::DEBUG::2013-11-20 03:49:48,960::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915102::DEBUG::2013-11-20 03:49:48,961::task::495::TaskManager.Task::(_debug) Task bebab185-2e10-4e34-86dd-bb681d2232ac: ref 0 aborting False
>Thread-15915102::INFO::2013-11-20 03:49:48,961::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:49:50,655::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915104::DEBUG::2013-11-20 03:49:51,207::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('dab1628d-5a54-49ba-9685-47e4c156842c',) {}
>Thread-15915104::DEBUG::2013-11-20 03:49:51,209::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Disk stats not available
>Thread-15915104::DEBUG::2013-11-20 03:49:51,210::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Disk latency not available
>Thread-15915104::DEBUG::2013-11-20 03:49:51,212::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '22921', 'displayIp': '0', 'displayPort': u'5918', 'session': 'Unknown', 'displaySecurePort': u'5919', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/ubuntu-12.04.1-server-amd64.iso', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet10': {'macAddr': u'00:1a:4a:a8:64:1a', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet10'}}, 'vmId': 'dab1628d-5a54-49ba-9685-47e4c156842c', 'monitorResponse': '0', 'cpuUser': '132.92', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '089b08fc-6544-4235-a061-fb55b09d2715', 'flushLatency': '0', 'truesize': '107374182400'}}, 'boot': 'cd', 'statsAge': '1.23', 'cpuIdle': '0.00', 'elapsedTime': '33', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '0', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:49:51,324::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0869511 s, 11.8 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 03:49:51,777::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:49:51,778::task::495::TaskManager.Task::(_debug) Task ddb62152-dc8d-4cd6-a068-e3ee5b3bcbc5: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:49:51,780::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`cc9eb634-76c2-40f9-bc84-6346cfcfec4c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:49:51,780::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:49:51,781::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:49:51,782::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`cc9eb634-76c2-40f9-bc84-6346cfcfec4c`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:49:51,783::task::495::TaskManager.Task::(_debug) Task ddb62152-dc8d-4cd6-a068-e3ee5b3bcbc5: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:49:51,784::task::495::TaskManager.Task::(_debug) Task ddb62152-dc8d-4cd6-a068-e3ee5b3bcbc5: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:49:51,785::task::495::TaskManager.Task::(_debug) Task ddb62152-dc8d-4cd6-a068-e3ee5b3bcbc5: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:49:51,786::task::495::TaskManager.Task::(_debug) Task ddb62152-dc8d-4cd6-a068-e3ee5b3bcbc5: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:49:51,787::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:49:51,787::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:49:51,788::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:49:51,789::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:49:51,790::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:49:51,790::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:49:51,791::task::495::TaskManager.Task::(_debug) Task ddb62152-dc8d-4cd6-a068-e3ee5b3bcbc5: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:49:51,792::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:49:51,793::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:49:51,794::task::495::TaskManager.Task::(_debug) Task ac2bb3c5-50c2-4fcf-87bd-f172150838ec: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:49:51,795::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8935aaef-a673-4766-a108-47c61e52dddd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:49:51,796::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:49:51,797::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:49:51,797::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8935aaef-a673-4766-a108-47c61e52dddd`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:49:51,798::task::495::TaskManager.Task::(_debug) Task ac2bb3c5-50c2-4fcf-87bd-f172150838ec: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:49:51,799::task::495::TaskManager.Task::(_debug) Task ac2bb3c5-50c2-4fcf-87bd-f172150838ec: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:49:51,800::task::495::TaskManager.Task::(_debug) Task ac2bb3c5-50c2-4fcf-87bd-f172150838ec: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:49:51,800::task::495::TaskManager.Task::(_debug) Task ac2bb3c5-50c2-4fcf-87bd-f172150838ec: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:49:51,801::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:49:51,802::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:49:51,802::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:49:51,803::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:49:51,804::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:49:51,804::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:49:51,805::task::495::TaskManager.Task::(_debug) Task ac2bb3c5-50c2-4fcf-87bd-f172150838ec: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:49:51,806::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:49:53,337::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:49:53,882::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0717362 s, 14.3 MB/s\n'; <rc> = 0
>Thread-15915106::DEBUG::2013-11-20 03:49:54,251::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('dab1628d-5a54-49ba-9685-47e4c156842c',) {}
>Thread-15915106::DEBUG::2013-11-20 03:49:54,253::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Disk stats not available
>Thread-15915106::DEBUG::2013-11-20 03:49:54,254::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Disk latency not available
>Thread-15915106::DEBUG::2013-11-20 03:49:54,255::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '22921', 'displayIp': '0', 'displayPort': u'5918', 'session': 'Unknown', 'displaySecurePort': u'5919', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/ubuntu-12.04.1-server-amd64.iso', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet10': {'macAddr': u'00:1a:4a:a8:64:1a', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet10'}}, 'vmId': 'dab1628d-5a54-49ba-9685-47e4c156842c', 'monitorResponse': '0', 'cpuUser': '132.92', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '089b08fc-6544-4235-a061-fb55b09d2715', 'flushLatency': '0', 'truesize': '107374182400'}}, 'boot': 'cd', 'statsAge': '0.27', 'cpuIdle': '0.00', 'elapsedTime': '36', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '0', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-42840::DEBUG::2013-11-20 03:49:55,574::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:49:55,900::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:49:56,410::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0749466 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15915108::DEBUG::2013-11-20 03:49:57,332::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('dab1628d-5a54-49ba-9685-47e4c156842c',) {}
>Thread-15915108::DEBUG::2013-11-20 03:49:57,333::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Disk stats not available
>Thread-15915108::DEBUG::2013-11-20 03:49:57,334::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Disk latency not available
>Thread-15915108::DEBUG::2013-11-20 03:49:57,334::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '22921', 'displayIp': '0', 'displayPort': u'5918', 'session': 'Unknown', 'displaySecurePort': u'5919', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/ubuntu-12.04.1-server-amd64.iso', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet10': {'macAddr': u'00:1a:4a:a8:64:1a', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet10'}}, 'vmId': 'dab1628d-5a54-49ba-9685-47e4c156842c', 'monitorResponse': '0', 'cpuUser': '132.92', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '089b08fc-6544-4235-a061-fb55b09d2715', 'flushLatency': '0', 'truesize': '107374182400'}}, 'boot': 'cd', 'statsAge': '0.25', 'cpuIdle': '0.00', 'elapsedTime': '39', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '0', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:49:59,490::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-42840::DEBUG::2013-11-20 03:49:59,508::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15915109::DEBUG::2013-11-20 03:49:59,665::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915109::INFO::2013-11-20 03:49:59,713::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915109::DEBUG::2013-11-20 03:49:59,714::task::495::TaskManager.Task::(_debug) Task b64c7e8f-6f39-4973-b184-422c92fabd23: moving from state init -> state preparing
>Thread-15915109::DEBUG::2013-11-20 03:49:59,715::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915109::DEBUG::2013-11-20 03:49:59,715::task::495::TaskManager.Task::(_debug) Task b64c7e8f-6f39-4973-b184-422c92fabd23: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915109::DEBUG::2013-11-20 03:49:59,716::task::495::TaskManager.Task::(_debug) Task b64c7e8f-6f39-4973-b184-422c92fabd23: moving from state preparing -> state finished
>Thread-15915109::DEBUG::2013-11-20 03:49:59,717::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915109::DEBUG::2013-11-20 03:49:59,717::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915109::DEBUG::2013-11-20 03:49:59,717::task::495::TaskManager.Task::(_debug) Task b64c7e8f-6f39-4973-b184-422c92fabd23: ref 0 aborting False
>Thread-15915109::INFO::2013-11-20 03:49:59,718::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915110::DEBUG::2013-11-20 03:49:59,731::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915110::INFO::2013-11-20 03:49:59,731::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915110::DEBUG::2013-11-20 03:49:59,732::task::495::TaskManager.Task::(_debug) Task 9248d877-4187-4cd1-afc1-74606a9e551b: moving from state init -> state preparing
>Thread-15915110::DEBUG::2013-11-20 03:49:59,733::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f049e69b-df33-4ece-aa61-7a9cd137dfc8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915110::DEBUG::2013-11-20 03:49:59,733::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915110::DEBUG::2013-11-20 03:49:59,734::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915110::DEBUG::2013-11-20 03:49:59,735::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f049e69b-df33-4ece-aa61-7a9cd137dfc8`::Granted request
>Thread-15915110::DEBUG::2013-11-20 03:49:59,736::task::495::TaskManager.Task::(_debug) Task 9248d877-4187-4cd1-afc1-74606a9e551b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915110::DEBUG::2013-11-20 03:49:59,736::task::495::TaskManager.Task::(_debug) Task 9248d877-4187-4cd1-afc1-74606a9e551b: ref 1 aborting False
>Thread-15915110::WARNING::2013-11-20 03:49:59,737::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915110::DEBUG::2013-11-20 03:49:59,738::task::495::TaskManager.Task::(_debug) Task 9248d877-4187-4cd1-afc1-74606a9e551b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915110::DEBUG::2013-11-20 03:49:59,739::task::495::TaskManager.Task::(_debug) Task 9248d877-4187-4cd1-afc1-74606a9e551b: moving from state preparing -> state finished
>Thread-15915110::DEBUG::2013-11-20 03:49:59,739::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915110::DEBUG::2013-11-20 03:49:59,740::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915110::DEBUG::2013-11-20 03:49:59,741::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915110::DEBUG::2013-11-20 03:49:59,741::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915110::DEBUG::2013-11-20 03:49:59,741::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915110::DEBUG::2013-11-20 03:49:59,742::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915110::DEBUG::2013-11-20 03:49:59,742::task::495::TaskManager.Task::(_debug) Task 9248d877-4187-4cd1-afc1-74606a9e551b: ref 0 aborting False
>Thread-15915110::INFO::2013-11-20 03:49:59,743::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:49:59,932::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0566813 s, 18.1 MB/s\n'; <rc> = 0
>Thread-15915111::INFO::2013-11-20 03:50:00,360::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915111::DEBUG::2013-11-20 03:50:00,361::task::495::TaskManager.Task::(_debug) Task eae76325-4f16-4408-bfbd-89ccbadad0fa: moving from state init -> state preparing
>Thread-15915111::DEBUG::2013-11-20 03:50:00,362::task::495::TaskManager.Task::(_debug) Task eae76325-4f16-4408-bfbd-89ccbadad0fa: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000302076339722', 'lastCheck': 1384919392.587033, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0318050384521', 'lastCheck': 1384919396.6123559, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0080668926239', 'lastCheck': 1384919392.4547279, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000920057296753', 'lastCheck': 1384919399.6936049, 'valid': True, 'code': 0}}
>Thread-15915111::DEBUG::2013-11-20 03:50:00,363::task::495::TaskManager.Task::(_debug) Task eae76325-4f16-4408-bfbd-89ccbadad0fa: moving from state preparing -> state finished
>Thread-15915111::DEBUG::2013-11-20 03:50:00,363::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915111::DEBUG::2013-11-20 03:50:00,364::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915111::DEBUG::2013-11-20 03:50:00,364::task::495::TaskManager.Task::(_debug) Task eae76325-4f16-4408-bfbd-89ccbadad0fa: ref 0 aborting False
>Thread-15915111::INFO::2013-11-20 03:50:00,365::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000302076339722', 'lastCheck': 1384919392.587033, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0318050384521', 'lastCheck': 1384919396.6123559, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0080668926239', 'lastCheck': 1384919392.4547279, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000920057296753', 'lastCheck': 1384919399.6936049, 'valid': True, 'code': 0}}
>Thread-15915112::DEBUG::2013-11-20 03:50:00,417::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Disk stats not available
>Thread-15915112::DEBUG::2013-11-20 03:50:00,418::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Disk latency not available
>Dummy-15912651::DEBUG::2013-11-20 03:50:01,944::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:50:02,551::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0933292 s, 11.0 MB/s\n'; <rc> = 0
>Thread-15913639::INFO::2013-11-20 03:50:03,490::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:50:03,494::task::495::TaskManager.Task::(_debug) Task 9f53fae9-ef3e-4010-935d-0f02df6f2911: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:50:03,498::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`82941088-c6b6-4928-94e9-3c6f804015f7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:50:03,501::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:50:03,502::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:50:03,503::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`82941088-c6b6-4928-94e9-3c6f804015f7`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:50:03,504::task::495::TaskManager.Task::(_debug) Task 9f53fae9-ef3e-4010-935d-0f02df6f2911: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:50:03,504::task::495::TaskManager.Task::(_debug) Task 9f53fae9-ef3e-4010-935d-0f02df6f2911: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:50:03,505::task::495::TaskManager.Task::(_debug) Task 9f53fae9-ef3e-4010-935d-0f02df6f2911: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:50:03,506::task::495::TaskManager.Task::(_debug) Task 9f53fae9-ef3e-4010-935d-0f02df6f2911: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:50:03,506::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:50:03,507::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:50:03,508::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:50:03,509::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:50:03,509::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:50:03,510::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:50:03,510::task::495::TaskManager.Task::(_debug) Task 9f53fae9-ef3e-4010-935d-0f02df6f2911: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:50:03,511::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15915114::DEBUG::2013-11-20 03:50:03,932::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('dab1628d-5a54-49ba-9685-47e4c156842c',) {}
>Thread-15915114::DEBUG::2013-11-20 03:50:03,933::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Disk stats not available
>Thread-15915114::DEBUG::2013-11-20 03:50:03,934::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Disk latency not available
>Thread-15915114::DEBUG::2013-11-20 03:50:03,935::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '22921', 'displayIp': '0', 'displayPort': u'5918', 'session': 'Unknown', 'displaySecurePort': u'5919', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/ubuntu-12.04.1-server-amd64.iso', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet10': {'macAddr': u'00:1a:4a:a8:64:1a', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet10'}}, 'vmId': 'dab1628d-5a54-49ba-9685-47e4c156842c', 'monitorResponse': '0', 'cpuUser': '181.86', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '089b08fc-6544-4235-a061-fb55b09d2715', 'flushLatency': '0', 'truesize': '107374182400'}}, 'boot': 'cd', 'statsAge': '0.46', 'cpuIdle': '0.00', 'elapsedTime': '45', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '0', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:50:04,568::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:50:05,090::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0736082 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15915116::DEBUG::2013-11-20 03:50:06,962::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('dab1628d-5a54-49ba-9685-47e4c156842c',) {}
>Thread-15915116::DEBUG::2013-11-20 03:50:06,963::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Disk stats not available
>Thread-15915116::DEBUG::2013-11-20 03:50:06,964::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Disk latency not available
>Thread-15915116::DEBUG::2013-11-20 03:50:06,965::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '22921', 'displayIp': '0', 'displayPort': u'5918', 'session': 'Unknown', 'displaySecurePort': u'5919', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/ubuntu-12.04.1-server-amd64.iso', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet10': {'macAddr': u'00:1a:4a:a8:64:1a', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet10'}}, 'vmId': 'dab1628d-5a54-49ba-9685-47e4c156842c', 'monitorResponse': '0', 'cpuUser': '181.86', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '089b08fc-6544-4235-a061-fb55b09d2715', 'flushLatency': '0', 'truesize': '107374182400'}}, 'boot': 'cd', 'statsAge': '1.49', 'cpuIdle': '0.00', 'elapsedTime': '48', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '0', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:50:07,104::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:50:07,678::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.101358 s, 10.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:50:09,696::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915118::DEBUG::2013-11-20 03:50:10,019::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('dab1628d-5a54-49ba-9685-47e4c156842c',) {}
>Thread-15915118::DEBUG::2013-11-20 03:50:10,021::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Disk stats not available
>Thread-15915118::DEBUG::2013-11-20 03:50:10,021::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Disk latency not available
>Thread-15915118::DEBUG::2013-11-20 03:50:10,022::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '22921', 'displayIp': '0', 'displayPort': u'5918', 'session': 'Unknown', 'displaySecurePort': u'5919', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/ubuntu-12.04.1-server-amd64.iso', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet10': {'macAddr': u'00:1a:4a:a8:64:1a', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet10'}}, 'vmId': 'dab1628d-5a54-49ba-9685-47e4c156842c', 'monitorResponse': '0', 'cpuUser': '181.86', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '089b08fc-6544-4235-a061-fb55b09d2715', 'flushLatency': '0', 'truesize': '107374182400'}}, 'boot': 'cd', 'statsAge': '0.54', 'cpuIdle': '0.00', 'elapsedTime': '51', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '0', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15915119::DEBUG::2013-11-20 03:50:10,090::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915119::INFO::2013-11-20 03:50:10,091::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915119::DEBUG::2013-11-20 03:50:10,092::task::495::TaskManager.Task::(_debug) Task b2928a91-f2ad-4380-a5c4-e806414c73f6: moving from state init -> state preparing
>Thread-15915119::DEBUG::2013-11-20 03:50:10,093::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915119::DEBUG::2013-11-20 03:50:10,094::task::495::TaskManager.Task::(_debug) Task b2928a91-f2ad-4380-a5c4-e806414c73f6: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915119::DEBUG::2013-11-20 03:50:10,095::task::495::TaskManager.Task::(_debug) Task b2928a91-f2ad-4380-a5c4-e806414c73f6: moving from state preparing -> state finished
>Thread-15915119::DEBUG::2013-11-20 03:50:10,095::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915119::DEBUG::2013-11-20 03:50:10,096::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915119::DEBUG::2013-11-20 03:50:10,097::task::495::TaskManager.Task::(_debug) Task b2928a91-f2ad-4380-a5c4-e806414c73f6: ref 0 aborting False
>Thread-15915119::INFO::2013-11-20 03:50:10,097::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915120::DEBUG::2013-11-20 03:50:10,109::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915120::INFO::2013-11-20 03:50:10,110::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915120::DEBUG::2013-11-20 03:50:10,111::task::495::TaskManager.Task::(_debug) Task afcbdda4-e896-4b33-9864-b10dabce12c1: moving from state init -> state preparing
>Thread-15915120::DEBUG::2013-11-20 03:50:10,113::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c04d1a5a-f844-42b0-a9e9-f7a66b9e28bd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915120::DEBUG::2013-11-20 03:50:10,114::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915120::DEBUG::2013-11-20 03:50:10,116::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915120::DEBUG::2013-11-20 03:50:10,116::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c04d1a5a-f844-42b0-a9e9-f7a66b9e28bd`::Granted request
>Thread-15915120::DEBUG::2013-11-20 03:50:10,118::task::495::TaskManager.Task::(_debug) Task afcbdda4-e896-4b33-9864-b10dabce12c1: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915120::DEBUG::2013-11-20 03:50:10,119::task::495::TaskManager.Task::(_debug) Task afcbdda4-e896-4b33-9864-b10dabce12c1: ref 1 aborting False
>Thread-15915120::WARNING::2013-11-20 03:50:10,122::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915120::DEBUG::2013-11-20 03:50:10,124::task::495::TaskManager.Task::(_debug) Task afcbdda4-e896-4b33-9864-b10dabce12c1: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915120::DEBUG::2013-11-20 03:50:10,125::task::495::TaskManager.Task::(_debug) Task afcbdda4-e896-4b33-9864-b10dabce12c1: moving from state preparing -> state finished
>Thread-15915120::DEBUG::2013-11-20 03:50:10,126::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915120::DEBUG::2013-11-20 03:50:10,127::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915120::DEBUG::2013-11-20 03:50:10,128::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915120::DEBUG::2013-11-20 03:50:10,129::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915120::DEBUG::2013-11-20 03:50:10,130::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915120::DEBUG::2013-11-20 03:50:10,131::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915120::DEBUG::2013-11-20 03:50:10,132::task::495::TaskManager.Task::(_debug) Task afcbdda4-e896-4b33-9864-b10dabce12c1: ref 0 aborting False
>Thread-15915120::INFO::2013-11-20 03:50:10,133::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:50:10,245::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.108319 s, 9.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:50:12,261::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:50:12,918::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0918054 s, 11.2 MB/s\n'; <rc> = 0
>Thread-15915122::DEBUG::2013-11-20 03:50:13,121::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('dab1628d-5a54-49ba-9685-47e4c156842c',) {}
>Thread-15915122::DEBUG::2013-11-20 03:50:13,122::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Disk stats not available
>Thread-15915122::DEBUG::2013-11-20 03:50:13,123::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Disk latency not available
>Thread-15915122::DEBUG::2013-11-20 03:50:13,124::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '22921', 'displayIp': '0', 'displayPort': u'5918', 'session': 'Unknown', 'displaySecurePort': u'5919', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/ubuntu-12.04.1-server-amd64.iso', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet10': {'macAddr': u'00:1a:4a:a8:64:1a', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet10'}}, 'vmId': 'dab1628d-5a54-49ba-9685-47e4c156842c', 'monitorResponse': '0', 'cpuUser': '181.86', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '089b08fc-6544-4235-a061-fb55b09d2715', 'flushLatency': '0', 'truesize': '107374182400'}}, 'boot': 'cd', 'statsAge': '1.64', 'cpuIdle': '0.00', 'elapsedTime': '55', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '0', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:50:14,934::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:50:15,442::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0517772 s, 19.8 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 03:50:15,694::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:50:15,695::task::495::TaskManager.Task::(_debug) Task 7cc4858e-dca8-4601-9999-e75b6232dba6: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:50:15,697::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9a801957-fd58-4379-8d5d-7279e3175a77`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:50:15,698::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:50:15,699::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:50:15,699::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9a801957-fd58-4379-8d5d-7279e3175a77`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:50:15,700::task::495::TaskManager.Task::(_debug) Task 7cc4858e-dca8-4601-9999-e75b6232dba6: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:50:15,701::task::495::TaskManager.Task::(_debug) Task 7cc4858e-dca8-4601-9999-e75b6232dba6: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:50:15,702::task::495::TaskManager.Task::(_debug) Task 7cc4858e-dca8-4601-9999-e75b6232dba6: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:50:15,703::task::495::TaskManager.Task::(_debug) Task 7cc4858e-dca8-4601-9999-e75b6232dba6: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:50:15,704::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:50:15,704::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:50:15,705::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:50:15,706::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:50:15,707::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:50:15,707::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:50:15,708::task::495::TaskManager.Task::(_debug) Task 7cc4858e-dca8-4601-9999-e75b6232dba6: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:50:15,709::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15915123::INFO::2013-11-20 03:50:16,146::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915123::DEBUG::2013-11-20 03:50:16,147::task::495::TaskManager.Task::(_debug) Task eba89e8b-3374-489d-b753-19bc7181dc9b: moving from state init -> state preparing
>Thread-15915123::DEBUG::2013-11-20 03:50:16,148::task::495::TaskManager.Task::(_debug) Task eba89e8b-3374-489d-b753-19bc7181dc9b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00038480758667', 'lastCheck': 1384919413.5057571, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148391723633', 'lastCheck': 1384919407.486299, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00172519683838', 'lastCheck': 1384919413.5133171, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00198984146118', 'lastCheck': 1384919409.873642, 'valid': True, 'code': 0}}
>Thread-15915123::DEBUG::2013-11-20 03:50:16,161::task::495::TaskManager.Task::(_debug) Task eba89e8b-3374-489d-b753-19bc7181dc9b: moving from state preparing -> state finished
>Thread-15915123::DEBUG::2013-11-20 03:50:16,162::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915123::DEBUG::2013-11-20 03:50:16,163::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915123::DEBUG::2013-11-20 03:50:16,164::task::495::TaskManager.Task::(_debug) Task eba89e8b-3374-489d-b753-19bc7181dc9b: ref 0 aborting False
>Thread-15915123::INFO::2013-11-20 03:50:16,165::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00038480758667', 'lastCheck': 1384919413.5057571, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148391723633', 'lastCheck': 1384919407.486299, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00172519683838', 'lastCheck': 1384919413.5133171, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00198984146118', 'lastCheck': 1384919409.873642, 'valid': True, 'code': 0}}
>Thread-15915124::DEBUG::2013-11-20 03:50:16,279::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Disk stats not available
>Thread-15915124::DEBUG::2013-11-20 03:50:16,280::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Disk latency not available
>Dummy-15912651::DEBUG::2013-11-20 03:50:17,454::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:50:18,043::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.121068 s, 8.5 MB/s\n'; <rc> = 0
>Thread-15915126::DEBUG::2013-11-20 03:50:19,978::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('dab1628d-5a54-49ba-9685-47e4c156842c',) {}
>Thread-15915126::DEBUG::2013-11-20 03:50:19,980::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Disk stats not available
>Thread-15915126::DEBUG::2013-11-20 03:50:19,980::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Disk latency not available
>Thread-15915126::DEBUG::2013-11-20 03:50:19,981::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '22921', 'displayIp': '0', 'displayPort': u'5918', 'session': 'Unknown', 'displaySecurePort': u'5919', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/ubuntu-12.04.1-server-amd64.iso', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet10': {'macAddr': u'00:1a:4a:a8:64:1a', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet10'}}, 'vmId': 'dab1628d-5a54-49ba-9685-47e4c156842c', 'monitorResponse': '0', 'cpuUser': '52.62', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '107374182400', 'writeLatency': '0', 'imageID': '089b08fc-6544-4235-a061-fb55b09d2715', 'flushLatency': '0', 'truesize': '107374182400'}}, 'boot': 'cd', 'statsAge': '0.30', 'cpuIdle': '47.38', 'elapsedTime': '61', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '0', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 03:50:20,059::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915127::DEBUG::2013-11-20 03:50:20,342::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915127::INFO::2013-11-20 03:50:20,345::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915127::DEBUG::2013-11-20 03:50:20,346::task::495::TaskManager.Task::(_debug) Task d805ee6c-4084-40b1-8e63-b88a4e119b51: moving from state init -> state preparing
>Thread-15915127::DEBUG::2013-11-20 03:50:20,348::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915127::DEBUG::2013-11-20 03:50:20,349::task::495::TaskManager.Task::(_debug) Task d805ee6c-4084-40b1-8e63-b88a4e119b51: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915127::DEBUG::2013-11-20 03:50:20,349::task::495::TaskManager.Task::(_debug) Task d805ee6c-4084-40b1-8e63-b88a4e119b51: moving from state preparing -> state finished
>Thread-15915127::DEBUG::2013-11-20 03:50:20,350::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915127::DEBUG::2013-11-20 03:50:20,351::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915127::DEBUG::2013-11-20 03:50:20,352::task::495::TaskManager.Task::(_debug) Task d805ee6c-4084-40b1-8e63-b88a4e119b51: ref 0 aborting False
>Thread-15915127::INFO::2013-11-20 03:50:20,352::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915128::DEBUG::2013-11-20 03:50:20,364::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915128::INFO::2013-11-20 03:50:20,365::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915128::DEBUG::2013-11-20 03:50:20,366::task::495::TaskManager.Task::(_debug) Task 2ce629f7-0793-4fb7-9520-a72db4df5bc2: moving from state init -> state preparing
>Thread-15915128::DEBUG::2013-11-20 03:50:20,368::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e93553d6-4f1c-4b5a-9e2d-242d414ac602`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915128::DEBUG::2013-11-20 03:50:20,368::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915128::DEBUG::2013-11-20 03:50:20,370::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915128::DEBUG::2013-11-20 03:50:20,370::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e93553d6-4f1c-4b5a-9e2d-242d414ac602`::Granted request
>Thread-15915128::DEBUG::2013-11-20 03:50:20,372::task::495::TaskManager.Task::(_debug) Task 2ce629f7-0793-4fb7-9520-a72db4df5bc2: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915128::DEBUG::2013-11-20 03:50:20,372::task::495::TaskManager.Task::(_debug) Task 2ce629f7-0793-4fb7-9520-a72db4df5bc2: ref 1 aborting False
>Thread-15915128::WARNING::2013-11-20 03:50:20,385::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915128::DEBUG::2013-11-20 03:50:20,387::task::495::TaskManager.Task::(_debug) Task 2ce629f7-0793-4fb7-9520-a72db4df5bc2: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915128::DEBUG::2013-11-20 03:50:20,388::task::495::TaskManager.Task::(_debug) Task 2ce629f7-0793-4fb7-9520-a72db4df5bc2: moving from state preparing -> state finished
>Thread-15915128::DEBUG::2013-11-20 03:50:20,389::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915128::DEBUG::2013-11-20 03:50:20,390::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915128::DEBUG::2013-11-20 03:50:20,391::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915128::DEBUG::2013-11-20 03:50:20,392::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915128::DEBUG::2013-11-20 03:50:20,393::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915128::DEBUG::2013-11-20 03:50:20,394::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915128::DEBUG::2013-11-20 03:50:20,395::task::495::TaskManager.Task::(_debug) Task 2ce629f7-0793-4fb7-9520-a72db4df5bc2: ref 0 aborting False
>Thread-15915128::INFO::2013-11-20 03:50:20,396::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:50:20,639::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0729456 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:50:22,655::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:50:23,353::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0764907 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:50:25,367::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:50:25,936::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0777092 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 03:50:27,893::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:50:27,894::task::495::TaskManager.Task::(_debug) Task 2b4946f1-411c-4735-afa3-baa939b5aefc: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:50:27,903::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`07f11dcb-bcf5-4913-8697-6269be2552d2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:50:27,904::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:50:27,905::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:50:27,906::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`07f11dcb-bcf5-4913-8697-6269be2552d2`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:50:27,908::task::495::TaskManager.Task::(_debug) Task 2b4946f1-411c-4735-afa3-baa939b5aefc: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:50:27,908::task::495::TaskManager.Task::(_debug) Task 2b4946f1-411c-4735-afa3-baa939b5aefc: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:50:27,909::task::495::TaskManager.Task::(_debug) Task 2b4946f1-411c-4735-afa3-baa939b5aefc: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:50:27,910::task::495::TaskManager.Task::(_debug) Task 2b4946f1-411c-4735-afa3-baa939b5aefc: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:50:27,911::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:50:27,912::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:50:27,913::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:50:27,914::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:50:27,914::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:50:27,915::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:50:27,916::task::495::TaskManager.Task::(_debug) Task 2b4946f1-411c-4735-afa3-baa939b5aefc: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:50:27,917::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913695::INFO::2013-11-20 03:50:27,936::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:50:27,938::task::495::TaskManager.Task::(_debug) Task 4697fbff-b9ef-4702-8410-37d44a322d35: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:50:27,941::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`514d3643-c0c5-4480-a2a1-79e2bdfce45f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:50:27,942::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:50:27,943::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:50:27,944::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`514d3643-c0c5-4480-a2a1-79e2bdfce45f`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:50:27,945::task::495::TaskManager.Task::(_debug) Task 4697fbff-b9ef-4702-8410-37d44a322d35: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:50:27,946::task::495::TaskManager.Task::(_debug) Task 4697fbff-b9ef-4702-8410-37d44a322d35: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:50:27,947::task::495::TaskManager.Task::(_debug) Task 4697fbff-b9ef-4702-8410-37d44a322d35: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:50:27,947::task::495::TaskManager.Task::(_debug) Task 4697fbff-b9ef-4702-8410-37d44a322d35: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:50:27,948::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:50:27,952::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:50:27,952::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:50:27,953::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Dummy-15912651::DEBUG::2013-11-20 03:50:27,954::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::DEBUG::2013-11-20 03:50:27,955::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:50:27,956::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:50:28,167::task::495::TaskManager.Task::(_debug) Task 4697fbff-b9ef-4702-8410-37d44a322d35: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:50:28,168::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:50:28,508::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0809575 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:50:30,524::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:50:31,066::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0773893 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15915132::DEBUG::2013-11-20 03:50:31,167::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915132::INFO::2013-11-20 03:50:31,169::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915132::DEBUG::2013-11-20 03:50:31,169::task::495::TaskManager.Task::(_debug) Task 462bf4c6-2b12-4984-b788-5696a15e4a24: moving from state init -> state preparing
>Thread-15915132::DEBUG::2013-11-20 03:50:31,170::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915132::DEBUG::2013-11-20 03:50:31,171::task::495::TaskManager.Task::(_debug) Task 462bf4c6-2b12-4984-b788-5696a15e4a24: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915132::DEBUG::2013-11-20 03:50:31,172::task::495::TaskManager.Task::(_debug) Task 462bf4c6-2b12-4984-b788-5696a15e4a24: moving from state preparing -> state finished
>Thread-15915132::DEBUG::2013-11-20 03:50:31,173::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915132::DEBUG::2013-11-20 03:50:31,173::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915132::DEBUG::2013-11-20 03:50:31,174::task::495::TaskManager.Task::(_debug) Task 462bf4c6-2b12-4984-b788-5696a15e4a24: ref 0 aborting False
>Thread-15915132::INFO::2013-11-20 03:50:31,175::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915133::DEBUG::2013-11-20 03:50:31,180::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915133::INFO::2013-11-20 03:50:31,182::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915133::DEBUG::2013-11-20 03:50:31,182::task::495::TaskManager.Task::(_debug) Task 84eff016-b07c-41bf-98dd-94d6e1e21e5a: moving from state init -> state preparing
>Thread-15915133::DEBUG::2013-11-20 03:50:31,184::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`06db6500-1711-4f4f-8e40-c450797d600b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915133::DEBUG::2013-11-20 03:50:31,185::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915133::DEBUG::2013-11-20 03:50:31,186::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915133::DEBUG::2013-11-20 03:50:31,186::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`06db6500-1711-4f4f-8e40-c450797d600b`::Granted request
>Thread-15915133::DEBUG::2013-11-20 03:50:31,188::task::495::TaskManager.Task::(_debug) Task 84eff016-b07c-41bf-98dd-94d6e1e21e5a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915133::DEBUG::2013-11-20 03:50:31,188::task::495::TaskManager.Task::(_debug) Task 84eff016-b07c-41bf-98dd-94d6e1e21e5a: ref 1 aborting False
>Thread-15915133::WARNING::2013-11-20 03:50:31,191::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915133::DEBUG::2013-11-20 03:50:31,192::task::495::TaskManager.Task::(_debug) Task 84eff016-b07c-41bf-98dd-94d6e1e21e5a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915133::DEBUG::2013-11-20 03:50:31,193::task::495::TaskManager.Task::(_debug) Task 84eff016-b07c-41bf-98dd-94d6e1e21e5a: moving from state preparing -> state finished
>Thread-15915133::DEBUG::2013-11-20 03:50:31,194::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915133::DEBUG::2013-11-20 03:50:31,194::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915133::DEBUG::2013-11-20 03:50:31,195::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915133::DEBUG::2013-11-20 03:50:31,196::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915133::DEBUG::2013-11-20 03:50:31,197::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915133::DEBUG::2013-11-20 03:50:31,198::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915133::DEBUG::2013-11-20 03:50:31,198::task::495::TaskManager.Task::(_debug) Task 84eff016-b07c-41bf-98dd-94d6e1e21e5a: ref 0 aborting False
>Thread-15915133::INFO::2013-11-20 03:50:31,199::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915134::INFO::2013-11-20 03:50:32,286::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915134::DEBUG::2013-11-20 03:50:32,287::task::495::TaskManager.Task::(_debug) Task 6bfbcda0-6469-4264-ab67-247b9d69d459: moving from state init -> state preparing
>Thread-15915134::DEBUG::2013-11-20 03:50:32,294::task::495::TaskManager.Task::(_debug) Task 6bfbcda0-6469-4264-ab67-247b9d69d459: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00414204597473', 'lastCheck': 1384919423.8755419, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00153803825378', 'lastCheck': 1384919427.872494, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00172185897827', 'lastCheck': 1384919423.8715739, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000428199768066', 'lastCheck': 1384919429.902817, 'valid': True, 'code': 0}}
>Thread-15915134::DEBUG::2013-11-20 03:50:32,300::task::495::TaskManager.Task::(_debug) Task 6bfbcda0-6469-4264-ab67-247b9d69d459: moving from state preparing -> state finished
>Thread-15915134::DEBUG::2013-11-20 03:50:32,300::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915134::DEBUG::2013-11-20 03:50:32,301::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915134::DEBUG::2013-11-20 03:50:32,301::task::495::TaskManager.Task::(_debug) Task 6bfbcda0-6469-4264-ab67-247b9d69d459: ref 0 aborting False
>Thread-15915134::INFO::2013-11-20 03:50:32,302::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00414204597473', 'lastCheck': 1384919423.8755419, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00153803825378', 'lastCheck': 1384919427.872494, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00172185897827', 'lastCheck': 1384919423.8715739, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000428199768066', 'lastCheck': 1384919429.902817, 'valid': True, 'code': 0}}
>Thread-15915135::DEBUG::2013-11-20 03:50:32,350::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Disk stats not available
>Thread-15915135::DEBUG::2013-11-20 03:50:32,351::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Disk latency not available
>Dummy-15912651::DEBUG::2013-11-20 03:50:33,082::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:50:33,604::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0586713 s, 17.5 MB/s\n'; <rc> = 0
>Thread-15915085::INFO::2013-11-20 03:50:33,689::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=089b08fc-6544-4235-a061-fb55b09d2715 volUUID=66b599fc-b7b5-4934-a5d5-ef7283460029)
>Thread-15915085::DEBUG::2013-11-20 03:50:33,690::task::495::TaskManager.Task::(_debug) Task 765c2f6e-171b-4700-8d5d-31a70512adfe: moving from state init -> state preparing
>Thread-15915085::DEBUG::2013-11-20 03:50:33,692::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`768c8c38-be02-43f9-8717-159c9c2424e2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915085::DEBUG::2013-11-20 03:50:33,693::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915085::DEBUG::2013-11-20 03:50:33,694::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915085::DEBUG::2013-11-20 03:50:33,695::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`768c8c38-be02-43f9-8717-159c9c2424e2`::Granted request
>Thread-15915085::DEBUG::2013-11-20 03:50:33,697::task::495::TaskManager.Task::(_debug) Task 765c2f6e-171b-4700-8d5d-31a70512adfe: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915085::DEBUG::2013-11-20 03:50:33,698::task::495::TaskManager.Task::(_debug) Task 765c2f6e-171b-4700-8d5d-31a70512adfe: ref 1 aborting False
>Thread-15915085::DEBUG::2013-11-20 03:50:33,700::task::495::TaskManager.Task::(_debug) Task 765c2f6e-171b-4700-8d5d-31a70512adfe: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15915085::DEBUG::2013-11-20 03:50:33,702::task::495::TaskManager.Task::(_debug) Task 765c2f6e-171b-4700-8d5d-31a70512adfe: moving from state preparing -> state finished
>Thread-15915085::DEBUG::2013-11-20 03:50:33,703::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915085::DEBUG::2013-11-20 03:50:33,703::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915085::DEBUG::2013-11-20 03:50:33,705::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915085::DEBUG::2013-11-20 03:50:33,706::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915085::DEBUG::2013-11-20 03:50:33,707::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915085::DEBUG::2013-11-20 03:50:33,707::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915085::DEBUG::2013-11-20 03:50:33,708::task::495::TaskManager.Task::(_debug) Task 765c2f6e-171b-4700-8d5d-31a70512adfe: ref 0 aborting False
>Thread-15915085::INFO::2013-11-20 03:50:33,708::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Dummy-15912651::DEBUG::2013-11-20 03:50:35,615::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:50:36,263::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0783929 s, 13.1 MB/s\n'; <rc> = 0
>Thread-15914453::INFO::2013-11-20 03:50:37,944::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 03:50:37,945::task::495::TaskManager.Task::(_debug) Task 79a97e24-c4ed-4129-8a13-fd86172e1239: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 03:50:37,958::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`693f7e80-312f-4eac-aeac-3ca1c1f4334b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 03:50:37,958::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 03:50:37,960::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 03:50:37,960::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`693f7e80-312f-4eac-aeac-3ca1c1f4334b`::Granted request
>Thread-15914453::DEBUG::2013-11-20 03:50:37,961::task::495::TaskManager.Task::(_debug) Task 79a97e24-c4ed-4129-8a13-fd86172e1239: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 03:50:37,962::task::495::TaskManager.Task::(_debug) Task 79a97e24-c4ed-4129-8a13-fd86172e1239: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 03:50:37,963::task::495::TaskManager.Task::(_debug) Task 79a97e24-c4ed-4129-8a13-fd86172e1239: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 03:50:37,963::task::495::TaskManager.Task::(_debug) Task 79a97e24-c4ed-4129-8a13-fd86172e1239: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 03:50:37,964::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 03:50:37,964::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 03:50:37,965::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 03:50:37,966::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 03:50:37,966::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 03:50:37,967::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 03:50:37,968::task::495::TaskManager.Task::(_debug) Task 79a97e24-c4ed-4129-8a13-fd86172e1239: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 03:50:37,968::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914435::INFO::2013-11-20 03:50:38,004::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 03:50:38,005::task::495::TaskManager.Task::(_debug) Task 73df003c-84df-4cb1-81a1-b36f2d67c4a6: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 03:50:38,007::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9efc486e-1bc0-41b1-9bde-8b524059fde7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 03:50:38,008::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 03:50:38,010::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 03:50:38,010::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9efc486e-1bc0-41b1-9bde-8b524059fde7`::Granted request
>Thread-15914435::DEBUG::2013-11-20 03:50:38,011::task::495::TaskManager.Task::(_debug) Task 73df003c-84df-4cb1-81a1-b36f2d67c4a6: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 03:50:38,012::task::495::TaskManager.Task::(_debug) Task 73df003c-84df-4cb1-81a1-b36f2d67c4a6: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 03:50:38,012::task::495::TaskManager.Task::(_debug) Task 73df003c-84df-4cb1-81a1-b36f2d67c4a6: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 03:50:38,013::task::495::TaskManager.Task::(_debug) Task 73df003c-84df-4cb1-81a1-b36f2d67c4a6: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 03:50:38,014::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 03:50:38,014::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 03:50:38,015::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 03:50:38,016::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 03:50:38,016::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 03:50:38,016::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 03:50:38,017::task::495::TaskManager.Task::(_debug) Task 73df003c-84df-4cb1-81a1-b36f2d67c4a6: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 03:50:38,018::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 03:50:38,282::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:50:38,808::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0828551 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:50:40,823::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:50:41,386::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0795663 s, 12.9 MB/s\n'; <rc> = 0
>Thread-15915138::DEBUG::2013-11-20 03:50:41,847::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915138::INFO::2013-11-20 03:50:41,848::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915138::DEBUG::2013-11-20 03:50:41,849::task::495::TaskManager.Task::(_debug) Task c377fea1-0576-42f0-9db5-6f88fc5e9f22: moving from state init -> state preparing
>Thread-15915138::DEBUG::2013-11-20 03:50:41,850::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915138::DEBUG::2013-11-20 03:50:41,851::task::495::TaskManager.Task::(_debug) Task c377fea1-0576-42f0-9db5-6f88fc5e9f22: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915138::DEBUG::2013-11-20 03:50:41,851::task::495::TaskManager.Task::(_debug) Task c377fea1-0576-42f0-9db5-6f88fc5e9f22: moving from state preparing -> state finished
>Thread-15915138::DEBUG::2013-11-20 03:50:41,852::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915138::DEBUG::2013-11-20 03:50:41,852::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915138::DEBUG::2013-11-20 03:50:41,853::task::495::TaskManager.Task::(_debug) Task c377fea1-0576-42f0-9db5-6f88fc5e9f22: ref 0 aborting False
>Thread-15915138::INFO::2013-11-20 03:50:41,854::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915139::DEBUG::2013-11-20 03:50:41,858::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915139::INFO::2013-11-20 03:50:41,860::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915139::DEBUG::2013-11-20 03:50:41,860::task::495::TaskManager.Task::(_debug) Task 96b35f98-928c-4b78-852a-237d7ea1b385: moving from state init -> state preparing
>Thread-15915139::DEBUG::2013-11-20 03:50:41,863::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b6f1b1de-c080-439e-a0e4-c36652a57c2f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915139::DEBUG::2013-11-20 03:50:41,864::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915139::DEBUG::2013-11-20 03:50:41,865::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915139::DEBUG::2013-11-20 03:50:41,866::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b6f1b1de-c080-439e-a0e4-c36652a57c2f`::Granted request
>Thread-15915139::DEBUG::2013-11-20 03:50:41,868::task::495::TaskManager.Task::(_debug) Task 96b35f98-928c-4b78-852a-237d7ea1b385: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915139::DEBUG::2013-11-20 03:50:41,869::task::495::TaskManager.Task::(_debug) Task 96b35f98-928c-4b78-852a-237d7ea1b385: ref 1 aborting False
>Thread-15915139::WARNING::2013-11-20 03:50:41,872::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915139::DEBUG::2013-11-20 03:50:41,874::task::495::TaskManager.Task::(_debug) Task 96b35f98-928c-4b78-852a-237d7ea1b385: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915139::DEBUG::2013-11-20 03:50:41,876::task::495::TaskManager.Task::(_debug) Task 96b35f98-928c-4b78-852a-237d7ea1b385: moving from state preparing -> state finished
>Thread-15915139::DEBUG::2013-11-20 03:50:41,877::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915139::DEBUG::2013-11-20 03:50:41,877::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915139::DEBUG::2013-11-20 03:50:41,879::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915139::DEBUG::2013-11-20 03:50:41,880::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915139::DEBUG::2013-11-20 03:50:41,881::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915139::DEBUG::2013-11-20 03:50:41,882::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915139::DEBUG::2013-11-20 03:50:41,883::task::495::TaskManager.Task::(_debug) Task 96b35f98-928c-4b78-852a-237d7ea1b385: ref 0 aborting False
>Thread-15915139::INFO::2013-11-20 03:50:41,884::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:50:43,401::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 03:50:43,889::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 03:50:43,890::task::495::TaskManager.Task::(_debug) Task 3a9fa6e0-e328-4bf8-8a48-13c1c36f40da: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 03:50:43,900::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`410dadf0-0ba6-4fbc-87bb-4673c164762c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 03:50:43,902::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 03:50:43,905::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 03:50:43,906::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`410dadf0-0ba6-4fbc-87bb-4673c164762c`::Granted request
>Thread-15914519::DEBUG::2013-11-20 03:50:43,908::task::495::TaskManager.Task::(_debug) Task 3a9fa6e0-e328-4bf8-8a48-13c1c36f40da: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 03:50:43,909::task::495::TaskManager.Task::(_debug) Task 3a9fa6e0-e328-4bf8-8a48-13c1c36f40da: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 03:50:43,910::task::495::TaskManager.Task::(_debug) Task 3a9fa6e0-e328-4bf8-8a48-13c1c36f40da: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 03:50:43,911::task::495::TaskManager.Task::(_debug) Task 3a9fa6e0-e328-4bf8-8a48-13c1c36f40da: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 03:50:43,912::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 03:50:43,913::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 03:50:43,915::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 03:50:43,916::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 03:50:43,917::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 03:50:43,918::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 03:50:43,920::task::495::TaskManager.Task::(_debug) Task 3a9fa6e0-e328-4bf8-8a48-13c1c36f40da: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 03:50:43,921::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 03:50:44,021::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.112849 s, 9.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:50:46,036::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:50:46,671::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.100972 s, 10.1 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 03:50:47,975::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:50:47,979::task::495::TaskManager.Task::(_debug) Task 48b273ab-b5ef-457d-aed1-74683b95feca: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:50:47,989::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`793e5331-8d89-4861-ad0a-f1cb0bb94a7b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:50:47,994::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:50:48,010::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:50:48,013::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`793e5331-8d89-4861-ad0a-f1cb0bb94a7b`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:50:48,016::task::495::TaskManager.Task::(_debug) Task 48b273ab-b5ef-457d-aed1-74683b95feca: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:50:48,018::task::495::TaskManager.Task::(_debug) Task 48b273ab-b5ef-457d-aed1-74683b95feca: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:50:48,019::task::495::TaskManager.Task::(_debug) Task 48b273ab-b5ef-457d-aed1-74683b95feca: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:50:48,019::task::495::TaskManager.Task::(_debug) Task 48b273ab-b5ef-457d-aed1-74683b95feca: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:50:48,020::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:50:48,020::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:50:48,021::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:50:48,022::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:50:48,022::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:50:48,023::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:50:48,023::task::495::TaskManager.Task::(_debug) Task 48b273ab-b5ef-457d-aed1-74683b95feca: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:50:48,024::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:50:48,024::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:50:48,025::task::495::TaskManager.Task::(_debug) Task c0e4115b-c1d3-4c9f-ab2b-d4eeb062d70a: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:50:48,026::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`ed5fd424-c0a7-4d4c-8a16-63c7da6e17f4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:50:48,027::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:50:48,027::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:50:48,028::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`ed5fd424-c0a7-4d4c-8a16-63c7da6e17f4`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:50:48,028::task::495::TaskManager.Task::(_debug) Task c0e4115b-c1d3-4c9f-ab2b-d4eeb062d70a: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:50:48,029::task::495::TaskManager.Task::(_debug) Task c0e4115b-c1d3-4c9f-ab2b-d4eeb062d70a: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:50:48,030::task::495::TaskManager.Task::(_debug) Task c0e4115b-c1d3-4c9f-ab2b-d4eeb062d70a: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:50:48,030::task::495::TaskManager.Task::(_debug) Task c0e4115b-c1d3-4c9f-ab2b-d4eeb062d70a: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:50:48,031::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:50:48,031::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:50:48,032::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:50:48,032::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:50:48,033::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:50:48,033::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:50:48,034::task::495::TaskManager.Task::(_debug) Task c0e4115b-c1d3-4c9f-ab2b-d4eeb062d70a: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:50:48,034::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:50:48,035::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:50:48,035::task::495::TaskManager.Task::(_debug) Task 9d9cd5aa-39ab-48da-a6e2-ccdaa54d2ac4: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:50:48,036::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`bf13720c-1d97-4490-a090-87d7d24aa607`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:50:48,037::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:50:48,037::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:50:48,038::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`bf13720c-1d97-4490-a090-87d7d24aa607`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:50:48,039::task::495::TaskManager.Task::(_debug) Task 9d9cd5aa-39ab-48da-a6e2-ccdaa54d2ac4: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:50:48,040::task::495::TaskManager.Task::(_debug) Task 9d9cd5aa-39ab-48da-a6e2-ccdaa54d2ac4: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:50:48,041::task::495::TaskManager.Task::(_debug) Task 9d9cd5aa-39ab-48da-a6e2-ccdaa54d2ac4: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:50:48,043::task::495::TaskManager.Task::(_debug) Task 9d9cd5aa-39ab-48da-a6e2-ccdaa54d2ac4: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:50:48,044::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:50:48,045::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:50:48,046::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:50:48,047::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:50:48,047::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:50:48,048::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:50:48,049::task::495::TaskManager.Task::(_debug) Task 9d9cd5aa-39ab-48da-a6e2-ccdaa54d2ac4: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:50:48,049::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:50:48,050::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:50:48,051::task::495::TaskManager.Task::(_debug) Task 057f79a8-d5b8-47bb-bcb2-11a9304d7aa7: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:50:48,052::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`962e2a8f-1753-4b1b-a128-89fe068dbe53`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:50:48,053::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:50:48,053::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:50:48,054::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`962e2a8f-1753-4b1b-a128-89fe068dbe53`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:50:48,055::task::495::TaskManager.Task::(_debug) Task 057f79a8-d5b8-47bb-bcb2-11a9304d7aa7: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:50:48,055::task::495::TaskManager.Task::(_debug) Task 057f79a8-d5b8-47bb-bcb2-11a9304d7aa7: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:50:48,056::task::495::TaskManager.Task::(_debug) Task 057f79a8-d5b8-47bb-bcb2-11a9304d7aa7: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:50:48,056::task::495::TaskManager.Task::(_debug) Task 057f79a8-d5b8-47bb-bcb2-11a9304d7aa7: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:50:48,057::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:50:48,057::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:50:48,058::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:50:48,058::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:50:48,059::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:50:48,059::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:50:48,060::task::495::TaskManager.Task::(_debug) Task 057f79a8-d5b8-47bb-bcb2-11a9304d7aa7: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:50:48,060::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15915142::INFO::2013-11-20 03:50:48,177::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915142::DEBUG::2013-11-20 03:50:48,177::task::495::TaskManager.Task::(_debug) Task 0ce601be-4c72-449f-87bc-420a4122784e: moving from state init -> state preparing
>Thread-15915142::DEBUG::2013-11-20 03:50:48,178::task::495::TaskManager.Task::(_debug) Task 0ce601be-4c72-449f-87bc-420a4122784e: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000410079956055', 'lastCheck': 1384919444.0430901, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00177001953125', 'lastCheck': 1384919448.0474751, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00187516212463', 'lastCheck': 1384919444.0440259, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000968933105469', 'lastCheck': 1384919439.9128001, 'valid': True, 'code': 0}}
>Thread-15915142::DEBUG::2013-11-20 03:50:48,179::task::495::TaskManager.Task::(_debug) Task 0ce601be-4c72-449f-87bc-420a4122784e: moving from state preparing -> state finished
>Thread-15915142::DEBUG::2013-11-20 03:50:48,180::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915142::DEBUG::2013-11-20 03:50:48,181::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915142::DEBUG::2013-11-20 03:50:48,181::task::495::TaskManager.Task::(_debug) Task 0ce601be-4c72-449f-87bc-420a4122784e: ref 0 aborting False
>Thread-15915142::INFO::2013-11-20 03:50:48,182::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000410079956055', 'lastCheck': 1384919444.0430901, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00177001953125', 'lastCheck': 1384919448.0474751, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00187516212463', 'lastCheck': 1384919444.0440259, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000968933105469', 'lastCheck': 1384919439.9128001, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:50:48,689::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:50:49,216::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0754477 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:50:51,232::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:50:51,818::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0549608 s, 18.6 MB/s\n'; <rc> = 0
>Thread-15915145::DEBUG::2013-11-20 03:50:52,457::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915145::INFO::2013-11-20 03:50:52,458::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915145::DEBUG::2013-11-20 03:50:52,458::task::495::TaskManager.Task::(_debug) Task f997e3b3-e7e6-4b6b-a2ab-792dfd48185e: moving from state init -> state preparing
>Thread-15915145::DEBUG::2013-11-20 03:50:52,459::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915145::DEBUG::2013-11-20 03:50:52,459::task::495::TaskManager.Task::(_debug) Task f997e3b3-e7e6-4b6b-a2ab-792dfd48185e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915145::DEBUG::2013-11-20 03:50:52,460::task::495::TaskManager.Task::(_debug) Task f997e3b3-e7e6-4b6b-a2ab-792dfd48185e: moving from state preparing -> state finished
>Thread-15915145::DEBUG::2013-11-20 03:50:52,460::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915145::DEBUG::2013-11-20 03:50:52,460::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915145::DEBUG::2013-11-20 03:50:52,461::task::495::TaskManager.Task::(_debug) Task f997e3b3-e7e6-4b6b-a2ab-792dfd48185e: ref 0 aborting False
>Thread-15915145::INFO::2013-11-20 03:50:52,461::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915146::DEBUG::2013-11-20 03:50:52,468::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915146::INFO::2013-11-20 03:50:52,469::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915146::DEBUG::2013-11-20 03:50:52,469::task::495::TaskManager.Task::(_debug) Task 5164cf34-2e25-4c65-932c-b208148fb1a4: moving from state init -> state preparing
>Thread-15915146::DEBUG::2013-11-20 03:50:52,470::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a6cd243a-037c-4a7c-a81e-4c3c2c13eff9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915146::DEBUG::2013-11-20 03:50:52,470::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915146::DEBUG::2013-11-20 03:50:52,478::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915146::DEBUG::2013-11-20 03:50:52,478::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a6cd243a-037c-4a7c-a81e-4c3c2c13eff9`::Granted request
>Thread-15915146::DEBUG::2013-11-20 03:50:52,479::task::495::TaskManager.Task::(_debug) Task 5164cf34-2e25-4c65-932c-b208148fb1a4: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915146::DEBUG::2013-11-20 03:50:52,480::task::495::TaskManager.Task::(_debug) Task 5164cf34-2e25-4c65-932c-b208148fb1a4: ref 1 aborting False
>Thread-15915146::WARNING::2013-11-20 03:50:52,481::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915146::DEBUG::2013-11-20 03:50:52,482::task::495::TaskManager.Task::(_debug) Task 5164cf34-2e25-4c65-932c-b208148fb1a4: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915146::DEBUG::2013-11-20 03:50:52,483::task::495::TaskManager.Task::(_debug) Task 5164cf34-2e25-4c65-932c-b208148fb1a4: moving from state preparing -> state finished
>Thread-15915146::DEBUG::2013-11-20 03:50:52,484::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915146::DEBUG::2013-11-20 03:50:52,485::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915146::DEBUG::2013-11-20 03:50:52,486::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915146::DEBUG::2013-11-20 03:50:52,486::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915146::DEBUG::2013-11-20 03:50:52,487::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915146::DEBUG::2013-11-20 03:50:52,488::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915146::DEBUG::2013-11-20 03:50:52,489::task::495::TaskManager.Task::(_debug) Task 5164cf34-2e25-4c65-932c-b208148fb1a4: ref 0 aborting False
>Thread-15915146::INFO::2013-11-20 03:50:52,490::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:50:53,831::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:50:54,413::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0972429 s, 10.5 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 03:50:54,724::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:50:54,725::task::495::TaskManager.Task::(_debug) Task 82bc50f4-75a2-4a7f-bed7-76617e03b72e: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:50:54,727::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b1fd8962-0f32-464a-810b-4709a318499d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:50:54,728::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:50:54,729::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:50:54,730::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b1fd8962-0f32-464a-810b-4709a318499d`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:50:54,731::task::495::TaskManager.Task::(_debug) Task 82bc50f4-75a2-4a7f-bed7-76617e03b72e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:50:54,732::task::495::TaskManager.Task::(_debug) Task 82bc50f4-75a2-4a7f-bed7-76617e03b72e: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:50:54,733::task::495::TaskManager.Task::(_debug) Task 82bc50f4-75a2-4a7f-bed7-76617e03b72e: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:50:54,734::task::495::TaskManager.Task::(_debug) Task 82bc50f4-75a2-4a7f-bed7-76617e03b72e: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:50:54,734::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:50:54,735::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:50:54,736::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:50:54,737::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:50:54,737::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:50:54,738::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:50:54,739::task::495::TaskManager.Task::(_debug) Task 82bc50f4-75a2-4a7f-bed7-76617e03b72e: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:50:54,740::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:50:54,741::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:50:54,741::task::495::TaskManager.Task::(_debug) Task 70a4d766-6f13-4197-9572-09abba5707ed: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:50:54,743::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`31cdf0e7-2e13-4e9d-b063-7f9e4648650a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:50:54,743::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:50:54,744::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:50:54,745::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`31cdf0e7-2e13-4e9d-b063-7f9e4648650a`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:50:54,746::task::495::TaskManager.Task::(_debug) Task 70a4d766-6f13-4197-9572-09abba5707ed: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:50:54,747::task::495::TaskManager.Task::(_debug) Task 70a4d766-6f13-4197-9572-09abba5707ed: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:50:54,748::task::495::TaskManager.Task::(_debug) Task 70a4d766-6f13-4197-9572-09abba5707ed: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:50:54,749::task::495::TaskManager.Task::(_debug) Task 70a4d766-6f13-4197-9572-09abba5707ed: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:50:54,749::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:50:54,750::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:50:54,751::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:50:54,752::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:50:54,753::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:50:54,754::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:50:54,755::task::495::TaskManager.Task::(_debug) Task 70a4d766-6f13-4197-9572-09abba5707ed: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:50:54,756::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:50:56,429::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:50:57,244::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0866323 s, 11.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:50:59,255::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:50:59,723::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0618087 s, 16.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:51:01,735::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:51:02,288::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0607894 s, 16.8 MB/s\n'; <rc> = 0
>Thread-15915150::DEBUG::2013-11-20 03:51:02,885::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915150::INFO::2013-11-20 03:51:02,887::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915150::DEBUG::2013-11-20 03:51:02,888::task::495::TaskManager.Task::(_debug) Task 633e2abd-927f-47e8-b109-cce41e52e95e: moving from state init -> state preparing
>Thread-15915150::DEBUG::2013-11-20 03:51:02,889::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915150::DEBUG::2013-11-20 03:51:02,890::task::495::TaskManager.Task::(_debug) Task 633e2abd-927f-47e8-b109-cce41e52e95e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915150::DEBUG::2013-11-20 03:51:02,891::task::495::TaskManager.Task::(_debug) Task 633e2abd-927f-47e8-b109-cce41e52e95e: moving from state preparing -> state finished
>Thread-15915150::DEBUG::2013-11-20 03:51:02,892::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915150::DEBUG::2013-11-20 03:51:02,892::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915150::DEBUG::2013-11-20 03:51:02,893::task::495::TaskManager.Task::(_debug) Task 633e2abd-927f-47e8-b109-cce41e52e95e: ref 0 aborting False
>Thread-15915150::INFO::2013-11-20 03:51:02,894::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915151::DEBUG::2013-11-20 03:51:02,899::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915151::INFO::2013-11-20 03:51:02,901::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915151::DEBUG::2013-11-20 03:51:02,901::task::495::TaskManager.Task::(_debug) Task ebd017b2-b51e-4a33-8d68-c4dac8580c7c: moving from state init -> state preparing
>Thread-15915151::DEBUG::2013-11-20 03:51:02,903::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3467a584-eea4-465c-82a7-c448909aebf3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915151::DEBUG::2013-11-20 03:51:02,904::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915151::DEBUG::2013-11-20 03:51:02,905::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915151::DEBUG::2013-11-20 03:51:02,905::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3467a584-eea4-465c-82a7-c448909aebf3`::Granted request
>Thread-15915151::DEBUG::2013-11-20 03:51:02,907::task::495::TaskManager.Task::(_debug) Task ebd017b2-b51e-4a33-8d68-c4dac8580c7c: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915151::DEBUG::2013-11-20 03:51:02,907::task::495::TaskManager.Task::(_debug) Task ebd017b2-b51e-4a33-8d68-c4dac8580c7c: ref 1 aborting False
>Thread-15915151::WARNING::2013-11-20 03:51:02,909::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915151::DEBUG::2013-11-20 03:51:02,911::task::495::TaskManager.Task::(_debug) Task ebd017b2-b51e-4a33-8d68-c4dac8580c7c: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915151::DEBUG::2013-11-20 03:51:02,912::task::495::TaskManager.Task::(_debug) Task ebd017b2-b51e-4a33-8d68-c4dac8580c7c: moving from state preparing -> state finished
>Thread-15915151::DEBUG::2013-11-20 03:51:02,912::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915151::DEBUG::2013-11-20 03:51:02,913::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915151::DEBUG::2013-11-20 03:51:02,914::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915151::DEBUG::2013-11-20 03:51:02,914::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915151::DEBUG::2013-11-20 03:51:02,915::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915151::DEBUG::2013-11-20 03:51:02,915::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915151::DEBUG::2013-11-20 03:51:02,916::task::495::TaskManager.Task::(_debug) Task ebd017b2-b51e-4a33-8d68-c4dac8580c7c: ref 0 aborting False
>Thread-15915151::INFO::2013-11-20 03:51:02,917::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915152::INFO::2013-11-20 03:51:03,973::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915152::DEBUG::2013-11-20 03:51:03,974::task::495::TaskManager.Task::(_debug) Task 261eb453-37a4-4d37-b29c-8859ff7abac6: moving from state init -> state preparing
>Thread-15915152::DEBUG::2013-11-20 03:51:03,975::task::495::TaskManager.Task::(_debug) Task 261eb453-37a4-4d37-b29c-8859ff7abac6: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000413179397583', 'lastCheck': 1384919454.493659, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.000938892364502', 'lastCheck': 1384919458.0953009, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0014009475708', 'lastCheck': 1384919454.4960959, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000733852386475', 'lastCheck': 1384919460.0635791, 'valid': True, 'code': 0}}
>Thread-15915152::DEBUG::2013-11-20 03:51:03,977::task::495::TaskManager.Task::(_debug) Task 261eb453-37a4-4d37-b29c-8859ff7abac6: moving from state preparing -> state finished
>Thread-15915152::DEBUG::2013-11-20 03:51:03,978::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915152::DEBUG::2013-11-20 03:51:03,979::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915152::DEBUG::2013-11-20 03:51:03,980::task::495::TaskManager.Task::(_debug) Task 261eb453-37a4-4d37-b29c-8859ff7abac6: ref 0 aborting False
>Thread-15915152::INFO::2013-11-20 03:51:03,981::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000413179397583', 'lastCheck': 1384919454.493659, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.000938892364502', 'lastCheck': 1384919458.0953009, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0014009475708', 'lastCheck': 1384919454.4960959, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000733852386475', 'lastCheck': 1384919460.0635791, 'valid': True, 'code': 0}}
>Thread-15913639::INFO::2013-11-20 03:51:04,149::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:51:04,150::task::495::TaskManager.Task::(_debug) Task aac05409-6e09-45b1-868c-eddc230da6b1: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:51:04,152::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`88b751a4-5b2b-45df-8194-b19092a25de2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:51:04,153::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:51:04,154::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:51:04,154::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`88b751a4-5b2b-45df-8194-b19092a25de2`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:51:04,155::task::495::TaskManager.Task::(_debug) Task aac05409-6e09-45b1-868c-eddc230da6b1: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:51:04,156::task::495::TaskManager.Task::(_debug) Task aac05409-6e09-45b1-868c-eddc230da6b1: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:51:04,157::task::495::TaskManager.Task::(_debug) Task aac05409-6e09-45b1-868c-eddc230da6b1: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:51:04,158::task::495::TaskManager.Task::(_debug) Task aac05409-6e09-45b1-868c-eddc230da6b1: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:51:04,158::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:51:04,159::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:51:04,159::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:51:04,160::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:51:04,161::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:51:04,161::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:51:04,162::task::495::TaskManager.Task::(_debug) Task aac05409-6e09-45b1-868c-eddc230da6b1: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:51:04,162::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:51:04,300::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:51:04,856::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0941423 s, 10.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:51:06,866::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:51:07,501::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.092037 s, 11.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:51:09,517::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:51:10,070::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0526967 s, 19.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:51:12,087::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:51:12,651::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0978218 s, 10.5 MB/s\n'; <rc> = 0
>Thread-15915156::DEBUG::2013-11-20 03:51:13,233::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915156::INFO::2013-11-20 03:51:13,235::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915156::DEBUG::2013-11-20 03:51:13,236::task::495::TaskManager.Task::(_debug) Task 0c6f0090-804a-44e8-8f73-ad2b2e547531: moving from state init -> state preparing
>Thread-15915156::DEBUG::2013-11-20 03:51:13,237::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915156::DEBUG::2013-11-20 03:51:13,237::task::495::TaskManager.Task::(_debug) Task 0c6f0090-804a-44e8-8f73-ad2b2e547531: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915156::DEBUG::2013-11-20 03:51:13,238::task::495::TaskManager.Task::(_debug) Task 0c6f0090-804a-44e8-8f73-ad2b2e547531: moving from state preparing -> state finished
>Thread-15915156::DEBUG::2013-11-20 03:51:13,239::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915156::DEBUG::2013-11-20 03:51:13,240::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915156::DEBUG::2013-11-20 03:51:13,241::task::495::TaskManager.Task::(_debug) Task 0c6f0090-804a-44e8-8f73-ad2b2e547531: ref 0 aborting False
>Thread-15915156::INFO::2013-11-20 03:51:13,241::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915157::DEBUG::2013-11-20 03:51:13,247::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915157::INFO::2013-11-20 03:51:13,248::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915157::DEBUG::2013-11-20 03:51:13,248::task::495::TaskManager.Task::(_debug) Task fe1f17b3-4ad5-4a1b-a912-c3be0132b275: moving from state init -> state preparing
>Thread-15915157::DEBUG::2013-11-20 03:51:13,250::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9a050dad-aff7-49f6-8a6c-d9513bd630e9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915157::DEBUG::2013-11-20 03:51:13,251::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915157::DEBUG::2013-11-20 03:51:13,252::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915157::DEBUG::2013-11-20 03:51:13,253::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9a050dad-aff7-49f6-8a6c-d9513bd630e9`::Granted request
>Thread-15915157::DEBUG::2013-11-20 03:51:13,254::task::495::TaskManager.Task::(_debug) Task fe1f17b3-4ad5-4a1b-a912-c3be0132b275: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915157::DEBUG::2013-11-20 03:51:13,255::task::495::TaskManager.Task::(_debug) Task fe1f17b3-4ad5-4a1b-a912-c3be0132b275: ref 1 aborting False
>Thread-15915157::WARNING::2013-11-20 03:51:13,257::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915157::DEBUG::2013-11-20 03:51:13,258::task::495::TaskManager.Task::(_debug) Task fe1f17b3-4ad5-4a1b-a912-c3be0132b275: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915157::DEBUG::2013-11-20 03:51:13,259::task::495::TaskManager.Task::(_debug) Task fe1f17b3-4ad5-4a1b-a912-c3be0132b275: moving from state preparing -> state finished
>Thread-15915157::DEBUG::2013-11-20 03:51:13,260::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915157::DEBUG::2013-11-20 03:51:13,260::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915157::DEBUG::2013-11-20 03:51:13,261::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915157::DEBUG::2013-11-20 03:51:13,262::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915157::DEBUG::2013-11-20 03:51:13,263::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915157::DEBUG::2013-11-20 03:51:13,263::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915157::DEBUG::2013-11-20 03:51:13,264::task::495::TaskManager.Task::(_debug) Task fe1f17b3-4ad5-4a1b-a912-c3be0132b275: ref 0 aborting False
>Thread-15915157::INFO::2013-11-20 03:51:13,265::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:51:14,666::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:51:15,192::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0792539 s, 12.9 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 03:51:16,640::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:51:16,642::task::495::TaskManager.Task::(_debug) Task eeffbe9f-331b-4b02-b077-3274d5bfb65d: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:51:16,643::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e5ab69b5-03b2-4fce-adf3-3e72256d5167`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:51:16,644::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:51:16,645::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:51:16,646::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e5ab69b5-03b2-4fce-adf3-3e72256d5167`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:51:16,647::task::495::TaskManager.Task::(_debug) Task eeffbe9f-331b-4b02-b077-3274d5bfb65d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:51:16,648::task::495::TaskManager.Task::(_debug) Task eeffbe9f-331b-4b02-b077-3274d5bfb65d: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:51:16,648::task::495::TaskManager.Task::(_debug) Task eeffbe9f-331b-4b02-b077-3274d5bfb65d: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:51:16,649::task::495::TaskManager.Task::(_debug) Task eeffbe9f-331b-4b02-b077-3274d5bfb65d: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:51:16,650::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:51:16,650::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:51:16,651::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:51:16,652::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:51:16,652::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:51:16,653::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:51:16,654::task::495::TaskManager.Task::(_debug) Task eeffbe9f-331b-4b02-b077-3274d5bfb65d: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:51:16,654::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:51:17,206::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:51:17,722::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0797827 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:51:19,736::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915160::INFO::2013-11-20 03:51:20,419::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915160::DEBUG::2013-11-20 03:51:20,420::task::495::TaskManager.Task::(_debug) Task b81b802d-89a8-4348-86a1-b228ab48353f: moving from state init -> state preparing
>Thread-15915160::DEBUG::2013-11-20 03:51:20,421::task::495::TaskManager.Task::(_debug) Task b81b802d-89a8-4348-86a1-b228ab48353f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00106501579285', 'lastCheck': 1384919474.8707571, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0086350440979', 'lastCheck': 1384919478.4159391, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.028589963913', 'lastCheck': 1384919474.8988321, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000350952148438', 'lastCheck': 1384919480.3921969, 'valid': True, 'code': 0}}
>Thread-15915160::DEBUG::2013-11-20 03:51:20,422::task::495::TaskManager.Task::(_debug) Task b81b802d-89a8-4348-86a1-b228ab48353f: moving from state preparing -> state finished
>Thread-15915160::DEBUG::2013-11-20 03:51:20,422::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915160::DEBUG::2013-11-20 03:51:20,423::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915160::DEBUG::2013-11-20 03:51:20,423::task::495::TaskManager.Task::(_debug) Task b81b802d-89a8-4348-86a1-b228ab48353f: ref 0 aborting False
>Thread-15915160::INFO::2013-11-20 03:51:20,424::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00106501579285', 'lastCheck': 1384919474.8707571, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0086350440979', 'lastCheck': 1384919478.4159391, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.028589963913', 'lastCheck': 1384919474.8988321, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000350952148438', 'lastCheck': 1384919480.3921969, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:51:20,823::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0846142 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:51:22,840::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:51:23,361::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0622826 s, 16.4 MB/s\n'; <rc> = 0
>Thread-15915162::DEBUG::2013-11-20 03:51:23,924::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915162::INFO::2013-11-20 03:51:23,925::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915162::DEBUG::2013-11-20 03:51:23,936::task::495::TaskManager.Task::(_debug) Task 698f45da-392d-4f87-8190-8df0d30f7ed8: moving from state init -> state preparing
>Thread-15915162::DEBUG::2013-11-20 03:51:23,937::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915162::DEBUG::2013-11-20 03:51:23,938::task::495::TaskManager.Task::(_debug) Task 698f45da-392d-4f87-8190-8df0d30f7ed8: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915162::DEBUG::2013-11-20 03:51:23,938::task::495::TaskManager.Task::(_debug) Task 698f45da-392d-4f87-8190-8df0d30f7ed8: moving from state preparing -> state finished
>Thread-15915162::DEBUG::2013-11-20 03:51:23,939::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915162::DEBUG::2013-11-20 03:51:23,939::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915162::DEBUG::2013-11-20 03:51:23,940::task::495::TaskManager.Task::(_debug) Task 698f45da-392d-4f87-8190-8df0d30f7ed8: ref 0 aborting False
>Thread-15915162::INFO::2013-11-20 03:51:23,940::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915164::DEBUG::2013-11-20 03:51:24,476::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915164::INFO::2013-11-20 03:51:24,477::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915164::DEBUG::2013-11-20 03:51:24,478::task::495::TaskManager.Task::(_debug) Task c50ac839-bd2d-4b16-ab61-a7e0b27bc6dd: moving from state init -> state preparing
>Thread-15915164::DEBUG::2013-11-20 03:51:24,479::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`384bedbb-6b9e-474b-b3a3-7a3708b4c884`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915164::DEBUG::2013-11-20 03:51:24,480::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915164::DEBUG::2013-11-20 03:51:24,480::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915164::DEBUG::2013-11-20 03:51:24,481::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`384bedbb-6b9e-474b-b3a3-7a3708b4c884`::Granted request
>Thread-15915164::DEBUG::2013-11-20 03:51:24,482::task::495::TaskManager.Task::(_debug) Task c50ac839-bd2d-4b16-ab61-a7e0b27bc6dd: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915164::DEBUG::2013-11-20 03:51:24,483::task::495::TaskManager.Task::(_debug) Task c50ac839-bd2d-4b16-ab61-a7e0b27bc6dd: ref 1 aborting False
>Thread-15915164::WARNING::2013-11-20 03:51:24,485::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915164::DEBUG::2013-11-20 03:51:24,486::task::495::TaskManager.Task::(_debug) Task c50ac839-bd2d-4b16-ab61-a7e0b27bc6dd: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915164::DEBUG::2013-11-20 03:51:24,491::task::495::TaskManager.Task::(_debug) Task c50ac839-bd2d-4b16-ab61-a7e0b27bc6dd: moving from state preparing -> state finished
>Thread-15915164::DEBUG::2013-11-20 03:51:24,491::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915164::DEBUG::2013-11-20 03:51:24,492::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915164::DEBUG::2013-11-20 03:51:24,493::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915164::DEBUG::2013-11-20 03:51:24,493::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915164::DEBUG::2013-11-20 03:51:24,494::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915164::DEBUG::2013-11-20 03:51:24,494::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915164::DEBUG::2013-11-20 03:51:24,495::task::495::TaskManager.Task::(_debug) Task c50ac839-bd2d-4b16-ab61-a7e0b27bc6dd: ref 0 aborting False
>Thread-15915164::INFO::2013-11-20 03:51:24,496::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:51:25,377::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:51:27,134::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0895773 s, 11.4 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 03:51:28,386::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:51:28,388::task::495::TaskManager.Task::(_debug) Task 6aa74fd9-5e5a-44e0-b771-34ca047bbd9d: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:51:28,390::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2b22233b-c2b9-47d7-accb-399353dcb259`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:51:28,391::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:51:28,392::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:51:28,393::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2b22233b-c2b9-47d7-accb-399353dcb259`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:51:28,394::task::495::TaskManager.Task::(_debug) Task 6aa74fd9-5e5a-44e0-b771-34ca047bbd9d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:51:28,395::task::495::TaskManager.Task::(_debug) Task 6aa74fd9-5e5a-44e0-b771-34ca047bbd9d: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:51:28,396::task::495::TaskManager.Task::(_debug) Task 6aa74fd9-5e5a-44e0-b771-34ca047bbd9d: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:51:28,396::task::495::TaskManager.Task::(_debug) Task 6aa74fd9-5e5a-44e0-b771-34ca047bbd9d: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:51:28,397::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:51:28,397::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:51:28,398::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:51:28,399::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:51:28,399::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:51:28,400::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:51:28,401::task::495::TaskManager.Task::(_debug) Task 6aa74fd9-5e5a-44e0-b771-34ca047bbd9d: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:51:28,402::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:51:29,149::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 03:51:29,655::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:51:29,656::task::495::TaskManager.Task::(_debug) Task 396c6e4e-d6c8-4d7a-8ed0-0d5f60c47b0c: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:51:29,660::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`64ad1a50-6ffb-4f41-a07d-4fa3f533087e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:51:29,661::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:51:29,663::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:51:29,664::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`64ad1a50-6ffb-4f41-a07d-4fa3f533087e`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:51:29,679::task::495::TaskManager.Task::(_debug) Task 396c6e4e-d6c8-4d7a-8ed0-0d5f60c47b0c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:51:29,681::task::495::TaskManager.Task::(_debug) Task 396c6e4e-d6c8-4d7a-8ed0-0d5f60c47b0c: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:51:29,682::task::495::TaskManager.Task::(_debug) Task 396c6e4e-d6c8-4d7a-8ed0-0d5f60c47b0c: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:51:29,683::task::495::TaskManager.Task::(_debug) Task 396c6e4e-d6c8-4d7a-8ed0-0d5f60c47b0c: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:51:29,684::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:51:29,685::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:51:29,686::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:51:29,687::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:51:29,688::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:51:29,689::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:51:29,691::task::495::TaskManager.Task::(_debug) Task 396c6e4e-d6c8-4d7a-8ed0-0d5f60c47b0c: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:51:29,692::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:51:29,781::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.126645 s, 8.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:51:31,801::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:51:32,316::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0860076 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:51:34,332::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915085::INFO::2013-11-20 03:51:34,518::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=089b08fc-6544-4235-a061-fb55b09d2715 volUUID=66b599fc-b7b5-4934-a5d5-ef7283460029)
>Thread-15915085::DEBUG::2013-11-20 03:51:34,522::task::495::TaskManager.Task::(_debug) Task 4bcdd9ef-f8a2-418c-8d6c-df520cf13d4a: moving from state init -> state preparing
>Thread-15915085::DEBUG::2013-11-20 03:51:34,526::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5341f363-5702-4a42-856a-6820955cbf63`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915085::DEBUG::2013-11-20 03:51:34,527::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915085::DEBUG::2013-11-20 03:51:34,528::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915085::DEBUG::2013-11-20 03:51:34,529::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5341f363-5702-4a42-856a-6820955cbf63`::Granted request
>Thread-15915085::DEBUG::2013-11-20 03:51:34,531::task::495::TaskManager.Task::(_debug) Task 4bcdd9ef-f8a2-418c-8d6c-df520cf13d4a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915085::DEBUG::2013-11-20 03:51:34,532::task::495::TaskManager.Task::(_debug) Task 4bcdd9ef-f8a2-418c-8d6c-df520cf13d4a: ref 1 aborting False
>Thread-15915085::DEBUG::2013-11-20 03:51:34,534::task::495::TaskManager.Task::(_debug) Task 4bcdd9ef-f8a2-418c-8d6c-df520cf13d4a: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15915085::DEBUG::2013-11-20 03:51:34,535::task::495::TaskManager.Task::(_debug) Task 4bcdd9ef-f8a2-418c-8d6c-df520cf13d4a: moving from state preparing -> state finished
>Thread-15915085::DEBUG::2013-11-20 03:51:34,535::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915085::DEBUG::2013-11-20 03:51:34,536::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915085::DEBUG::2013-11-20 03:51:34,537::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915085::DEBUG::2013-11-20 03:51:34,538::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915085::DEBUG::2013-11-20 03:51:34,539::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915085::DEBUG::2013-11-20 03:51:34,539::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915085::DEBUG::2013-11-20 03:51:34,540::task::495::TaskManager.Task::(_debug) Task 4bcdd9ef-f8a2-418c-8d6c-df520cf13d4a: ref 0 aborting False
>Thread-15915085::INFO::2013-11-20 03:51:34,541::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15915168::DEBUG::2013-11-20 03:51:34,850::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915168::INFO::2013-11-20 03:51:34,852::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915168::DEBUG::2013-11-20 03:51:34,853::task::495::TaskManager.Task::(_debug) Task ec77ac65-2144-4c3f-9652-820c3393b114: moving from state init -> state preparing
>Thread-15915168::DEBUG::2013-11-20 03:51:34,854::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915168::DEBUG::2013-11-20 03:51:34,855::task::495::TaskManager.Task::(_debug) Task ec77ac65-2144-4c3f-9652-820c3393b114: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915168::DEBUG::2013-11-20 03:51:34,856::task::495::TaskManager.Task::(_debug) Task ec77ac65-2144-4c3f-9652-820c3393b114: moving from state preparing -> state finished
>Thread-15915168::DEBUG::2013-11-20 03:51:34,857::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915168::DEBUG::2013-11-20 03:51:34,857::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915168::DEBUG::2013-11-20 03:51:34,858::task::495::TaskManager.Task::(_debug) Task ec77ac65-2144-4c3f-9652-820c3393b114: ref 0 aborting False
>Thread-15915168::INFO::2013-11-20 03:51:34,859::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915169::DEBUG::2013-11-20 03:51:34,871::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915169::INFO::2013-11-20 03:51:34,872::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915169::DEBUG::2013-11-20 03:51:34,873::task::495::TaskManager.Task::(_debug) Task 8d330359-28e4-4ba4-9c7b-40f06d0ed9c9: moving from state init -> state preparing
>Thread-15915169::DEBUG::2013-11-20 03:51:34,874::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0590cc8f-1034-4364-bc92-b8c94b42383a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915169::DEBUG::2013-11-20 03:51:34,875::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915169::DEBUG::2013-11-20 03:51:34,876::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915169::DEBUG::2013-11-20 03:51:34,877::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0590cc8f-1034-4364-bc92-b8c94b42383a`::Granted request
>Thread-15915169::DEBUG::2013-11-20 03:51:34,878::task::495::TaskManager.Task::(_debug) Task 8d330359-28e4-4ba4-9c7b-40f06d0ed9c9: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915169::DEBUG::2013-11-20 03:51:34,878::task::495::TaskManager.Task::(_debug) Task 8d330359-28e4-4ba4-9c7b-40f06d0ed9c9: ref 1 aborting False
>Thread-15915169::WARNING::2013-11-20 03:51:34,881::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915169::DEBUG::2013-11-20 03:51:34,882::task::495::TaskManager.Task::(_debug) Task 8d330359-28e4-4ba4-9c7b-40f06d0ed9c9: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915169::DEBUG::2013-11-20 03:51:34,883::task::495::TaskManager.Task::(_debug) Task 8d330359-28e4-4ba4-9c7b-40f06d0ed9c9: moving from state preparing -> state finished
>Thread-15915169::DEBUG::2013-11-20 03:51:34,884::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915169::DEBUG::2013-11-20 03:51:34,884::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915169::DEBUG::2013-11-20 03:51:34,885::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915169::DEBUG::2013-11-20 03:51:34,886::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915169::DEBUG::2013-11-20 03:51:34,886::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915169::DEBUG::2013-11-20 03:51:34,887::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915169::DEBUG::2013-11-20 03:51:34,887::task::495::TaskManager.Task::(_debug) Task 8d330359-28e4-4ba4-9c7b-40f06d0ed9c9: ref 0 aborting False
>Thread-15915169::INFO::2013-11-20 03:51:34,888::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:51:35,129::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0814184 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15915170::INFO::2013-11-20 03:51:36,581::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915170::DEBUG::2013-11-20 03:51:36,581::task::495::TaskManager.Task::(_debug) Task a2516fa4-c265-48db-814d-16ddc4251a72: moving from state init -> state preparing
>Thread-15915170::DEBUG::2013-11-20 03:51:36,582::task::495::TaskManager.Task::(_debug) Task a2516fa4-c265-48db-814d-16ddc4251a72: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00026798248291', 'lastCheck': 1384919495.642874, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00135397911072', 'lastCheck': 1384919488.6300571, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000902891159058', 'lastCheck': 1384919495.6445601, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000370979309082', 'lastCheck': 1384919490.62731, 'valid': True, 'code': 0}}
>Thread-15915170::DEBUG::2013-11-20 03:51:36,582::task::495::TaskManager.Task::(_debug) Task a2516fa4-c265-48db-814d-16ddc4251a72: moving from state preparing -> state finished
>Thread-15915170::DEBUG::2013-11-20 03:51:36,583::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915170::DEBUG::2013-11-20 03:51:36,583::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915170::DEBUG::2013-11-20 03:51:36,584::task::495::TaskManager.Task::(_debug) Task a2516fa4-c265-48db-814d-16ddc4251a72: ref 0 aborting False
>Thread-15915170::INFO::2013-11-20 03:51:36,584::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00026798248291', 'lastCheck': 1384919495.642874, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00135397911072', 'lastCheck': 1384919488.6300571, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000902891159058', 'lastCheck': 1384919495.6445601, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000370979309082', 'lastCheck': 1384919490.62731, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:51:37,145::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:51:37,948::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0941878 s, 10.9 MB/s\n'; <rc> = 0
>Thread-15914453::INFO::2013-11-20 03:51:38,514::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 03:51:38,515::task::495::TaskManager.Task::(_debug) Task 9f1b0a58-9e6f-40c3-a8c0-bb1ef4ff2353: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 03:51:38,518::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`30111cd8-a5df-4de3-8a66-d3282c3c5bd7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 03:51:38,520::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 03:51:38,521::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 03:51:38,522::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`30111cd8-a5df-4de3-8a66-d3282c3c5bd7`::Granted request
>Thread-15914453::DEBUG::2013-11-20 03:51:38,525::task::495::TaskManager.Task::(_debug) Task 9f1b0a58-9e6f-40c3-a8c0-bb1ef4ff2353: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 03:51:38,526::task::495::TaskManager.Task::(_debug) Task 9f1b0a58-9e6f-40c3-a8c0-bb1ef4ff2353: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 03:51:38,527::task::495::TaskManager.Task::(_debug) Task 9f1b0a58-9e6f-40c3-a8c0-bb1ef4ff2353: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 03:51:38,528::task::495::TaskManager.Task::(_debug) Task 9f1b0a58-9e6f-40c3-a8c0-bb1ef4ff2353: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 03:51:38,529::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 03:51:38,529::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 03:51:38,530::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 03:51:38,531::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 03:51:38,532::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 03:51:38,532::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 03:51:38,533::task::495::TaskManager.Task::(_debug) Task 9f1b0a58-9e6f-40c3-a8c0-bb1ef4ff2353: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 03:51:38,534::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914435::INFO::2013-11-20 03:51:38,717::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 03:51:38,718::task::495::TaskManager.Task::(_debug) Task 5ef22dad-a7c3-465d-bb60-16b7a9990ccf: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 03:51:38,720::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8d684cb3-85f6-4384-90ba-f612c5d0ef51`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 03:51:38,720::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 03:51:38,721::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 03:51:38,722::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8d684cb3-85f6-4384-90ba-f612c5d0ef51`::Granted request
>Thread-15914435::DEBUG::2013-11-20 03:51:38,723::task::495::TaskManager.Task::(_debug) Task 5ef22dad-a7c3-465d-bb60-16b7a9990ccf: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 03:51:38,724::task::495::TaskManager.Task::(_debug) Task 5ef22dad-a7c3-465d-bb60-16b7a9990ccf: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 03:51:38,724::task::495::TaskManager.Task::(_debug) Task 5ef22dad-a7c3-465d-bb60-16b7a9990ccf: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 03:51:38,725::task::495::TaskManager.Task::(_debug) Task 5ef22dad-a7c3-465d-bb60-16b7a9990ccf: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 03:51:38,726::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 03:51:38,727::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 03:51:38,727::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 03:51:38,728::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 03:51:38,729::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 03:51:38,729::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 03:51:38,730::task::495::TaskManager.Task::(_debug) Task 5ef22dad-a7c3-465d-bb60-16b7a9990ccf: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 03:51:38,731::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 03:51:39,961::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:51:40,507::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0823951 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:51:42,523::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:51:43,515::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0510526 s, 20.1 MB/s\n'; <rc> = 0
>Thread-15914519::INFO::2013-11-20 03:51:44,747::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 03:51:44,757::task::495::TaskManager.Task::(_debug) Task a99e767d-b58a-4f21-a7c3-6fd6a1807b52: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 03:51:44,760::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bd95cce6-813c-4c78-b2bb-6121cf3b20cc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 03:51:44,761::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 03:51:44,763::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 03:51:44,764::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bd95cce6-813c-4c78-b2bb-6121cf3b20cc`::Granted request
>Thread-15914519::DEBUG::2013-11-20 03:51:44,771::task::495::TaskManager.Task::(_debug) Task a99e767d-b58a-4f21-a7c3-6fd6a1807b52: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 03:51:44,772::task::495::TaskManager.Task::(_debug) Task a99e767d-b58a-4f21-a7c3-6fd6a1807b52: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 03:51:44,773::task::495::TaskManager.Task::(_debug) Task a99e767d-b58a-4f21-a7c3-6fd6a1807b52: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 03:51:44,774::task::495::TaskManager.Task::(_debug) Task a99e767d-b58a-4f21-a7c3-6fd6a1807b52: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 03:51:44,775::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 03:51:44,776::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 03:51:44,777::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 03:51:44,778::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 03:51:44,779::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 03:51:44,780::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 03:51:44,781::task::495::TaskManager.Task::(_debug) Task a99e767d-b58a-4f21-a7c3-6fd6a1807b52: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 03:51:44,782::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15915174::DEBUG::2013-11-20 03:51:45,266::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915174::INFO::2013-11-20 03:51:45,268::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915174::DEBUG::2013-11-20 03:51:45,269::task::495::TaskManager.Task::(_debug) Task c218ac04-43b3-43c2-98bb-56aa03d97137: moving from state init -> state preparing
>Thread-15915174::DEBUG::2013-11-20 03:51:45,269::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915174::DEBUG::2013-11-20 03:51:45,270::task::495::TaskManager.Task::(_debug) Task c218ac04-43b3-43c2-98bb-56aa03d97137: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915174::DEBUG::2013-11-20 03:51:45,271::task::495::TaskManager.Task::(_debug) Task c218ac04-43b3-43c2-98bb-56aa03d97137: moving from state preparing -> state finished
>Thread-15915174::DEBUG::2013-11-20 03:51:45,272::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915174::DEBUG::2013-11-20 03:51:45,272::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915174::DEBUG::2013-11-20 03:51:45,273::task::495::TaskManager.Task::(_debug) Task c218ac04-43b3-43c2-98bb-56aa03d97137: ref 0 aborting False
>Thread-15915174::INFO::2013-11-20 03:51:45,274::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915175::DEBUG::2013-11-20 03:51:45,279::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915175::INFO::2013-11-20 03:51:45,280::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915175::DEBUG::2013-11-20 03:51:45,280::task::495::TaskManager.Task::(_debug) Task 8d9ab05e-dd40-42b4-a491-0cb7d0fefd05: moving from state init -> state preparing
>Thread-15915175::DEBUG::2013-11-20 03:51:45,281::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b41c4fb2-1eb3-427c-83d8-915ee2e2815a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915175::DEBUG::2013-11-20 03:51:45,282::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915175::DEBUG::2013-11-20 03:51:45,283::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915175::DEBUG::2013-11-20 03:51:45,284::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b41c4fb2-1eb3-427c-83d8-915ee2e2815a`::Granted request
>Thread-15915175::DEBUG::2013-11-20 03:51:45,285::task::495::TaskManager.Task::(_debug) Task 8d9ab05e-dd40-42b4-a491-0cb7d0fefd05: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915175::DEBUG::2013-11-20 03:51:45,285::task::495::TaskManager.Task::(_debug) Task 8d9ab05e-dd40-42b4-a491-0cb7d0fefd05: ref 1 aborting False
>Thread-15915175::WARNING::2013-11-20 03:51:45,288::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915175::DEBUG::2013-11-20 03:51:45,289::task::495::TaskManager.Task::(_debug) Task 8d9ab05e-dd40-42b4-a491-0cb7d0fefd05: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915175::DEBUG::2013-11-20 03:51:45,290::task::495::TaskManager.Task::(_debug) Task 8d9ab05e-dd40-42b4-a491-0cb7d0fefd05: moving from state preparing -> state finished
>Thread-15915175::DEBUG::2013-11-20 03:51:45,291::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915175::DEBUG::2013-11-20 03:51:45,291::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915175::DEBUG::2013-11-20 03:51:45,292::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915175::DEBUG::2013-11-20 03:51:45,293::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915175::DEBUG::2013-11-20 03:51:45,293::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915175::DEBUG::2013-11-20 03:51:45,294::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915175::DEBUG::2013-11-20 03:51:45,294::task::495::TaskManager.Task::(_debug) Task 8d9ab05e-dd40-42b4-a491-0cb7d0fefd05: ref 0 aborting False
>Thread-15915175::INFO::2013-11-20 03:51:45,295::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:51:45,528::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:51:46,082::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0781528 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:51:48,095::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 03:51:48,530::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:51:48,532::task::495::TaskManager.Task::(_debug) Task d1049c1f-04fd-4e4f-a9ce-49e74b7ec289: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:51:48,534::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`45b4324f-fb4f-4450-9303-0272689e5110`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:51:48,535::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:51:48,536::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:51:48,537::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`45b4324f-fb4f-4450-9303-0272689e5110`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:51:48,538::task::495::TaskManager.Task::(_debug) Task d1049c1f-04fd-4e4f-a9ce-49e74b7ec289: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:51:48,539::task::495::TaskManager.Task::(_debug) Task d1049c1f-04fd-4e4f-a9ce-49e74b7ec289: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:51:48,541::task::495::TaskManager.Task::(_debug) Task d1049c1f-04fd-4e4f-a9ce-49e74b7ec289: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:51:48,543::task::495::TaskManager.Task::(_debug) Task d1049c1f-04fd-4e4f-a9ce-49e74b7ec289: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:51:48,544::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:51:48,545::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:51:48,546::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:51:48,547::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:51:48,548::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:51:48,548::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:51:48,549::task::495::TaskManager.Task::(_debug) Task d1049c1f-04fd-4e4f-a9ce-49e74b7ec289: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:51:48,550::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:51:48,551::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:51:48,552::task::495::TaskManager.Task::(_debug) Task 7eefae22-25b8-47f6-9c1e-d0c466997be2: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:51:48,554::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e18a6644-8939-4825-bff4-bc67c73f4806`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:51:48,555::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:51:48,556::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:51:48,557::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e18a6644-8939-4825-bff4-bc67c73f4806`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:51:48,559::task::495::TaskManager.Task::(_debug) Task 7eefae22-25b8-47f6-9c1e-d0c466997be2: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:51:48,560::task::495::TaskManager.Task::(_debug) Task 7eefae22-25b8-47f6-9c1e-d0c466997be2: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:51:48,561::task::495::TaskManager.Task::(_debug) Task 7eefae22-25b8-47f6-9c1e-d0c466997be2: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:51:48,562::task::495::TaskManager.Task::(_debug) Task 7eefae22-25b8-47f6-9c1e-d0c466997be2: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:51:48,563::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:51:48,564::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:51:48,565::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:51:48,566::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:51:48,567::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:51:48,567::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:51:48,569::task::495::TaskManager.Task::(_debug) Task 7eefae22-25b8-47f6-9c1e-d0c466997be2: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:51:48,569::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:51:48,571::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:51:48,571::task::495::TaskManager.Task::(_debug) Task 907014b6-fe4e-40b5-a049-16f03d7a082b: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:51:48,573::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`98e912c8-cfef-4668-844f-fcfbb5af26ad`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:51:48,573::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:51:48,574::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:51:48,575::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`98e912c8-cfef-4668-844f-fcfbb5af26ad`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:51:48,577::task::495::TaskManager.Task::(_debug) Task 907014b6-fe4e-40b5-a049-16f03d7a082b: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:51:48,577::task::495::TaskManager.Task::(_debug) Task 907014b6-fe4e-40b5-a049-16f03d7a082b: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:51:48,578::task::495::TaskManager.Task::(_debug) Task 907014b6-fe4e-40b5-a049-16f03d7a082b: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:51:48,579::task::495::TaskManager.Task::(_debug) Task 907014b6-fe4e-40b5-a049-16f03d7a082b: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:51:48,580::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:51:48,581::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:51:48,582::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:51:48,584::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:51:48,584::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:51:48,585::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:51:48,586::task::495::TaskManager.Task::(_debug) Task 907014b6-fe4e-40b5-a049-16f03d7a082b: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:51:48,587::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:51:48,588::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:51:48,589::task::495::TaskManager.Task::(_debug) Task 8612c324-a700-49aa-8ce3-399e9ac3c8e2: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:51:48,590::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`63dc6e8a-2087-4ea9-b2c2-b844a029754b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:51:48,591::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:51:48,592::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:51:48,593::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`63dc6e8a-2087-4ea9-b2c2-b844a029754b`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:51:48,595::task::495::TaskManager.Task::(_debug) Task 8612c324-a700-49aa-8ce3-399e9ac3c8e2: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:51:48,596::task::495::TaskManager.Task::(_debug) Task 8612c324-a700-49aa-8ce3-399e9ac3c8e2: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:51:48,597::task::495::TaskManager.Task::(_debug) Task 8612c324-a700-49aa-8ce3-399e9ac3c8e2: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:51:48,598::task::495::TaskManager.Task::(_debug) Task 8612c324-a700-49aa-8ce3-399e9ac3c8e2: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:51:48,598::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:51:48,599::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:51:48,600::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:51:48,601::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:51:48,602::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:51:48,603::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:51:48,604::task::495::TaskManager.Task::(_debug) Task 8612c324-a700-49aa-8ce3-399e9ac3c8e2: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:51:48,605::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:51:48,764::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.130897 s, 7.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:51:50,782::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:51:51,663::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0814433 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15915178::INFO::2013-11-20 03:51:52,338::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915178::DEBUG::2013-11-20 03:51:52,339::task::495::TaskManager.Task::(_debug) Task 62f912b7-03eb-4557-9342-8ae57aee4902: moving from state init -> state preparing
>Thread-15915178::DEBUG::2013-11-20 03:51:52,340::task::495::TaskManager.Task::(_debug) Task 62f912b7-03eb-4557-9342-8ae57aee4902: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00127196311951', 'lastCheck': 1384919505.7562399, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00254011154175', 'lastCheck': 1384919508.7519801, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00169205665588', 'lastCheck': 1384919505.7574, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000377178192139', 'lastCheck': 1384919510.7531769, 'valid': True, 'code': 0}}
>Thread-15915178::DEBUG::2013-11-20 03:51:52,341::task::495::TaskManager.Task::(_debug) Task 62f912b7-03eb-4557-9342-8ae57aee4902: moving from state preparing -> state finished
>Thread-15915178::DEBUG::2013-11-20 03:51:52,341::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915178::DEBUG::2013-11-20 03:51:52,342::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915178::DEBUG::2013-11-20 03:51:52,342::task::495::TaskManager.Task::(_debug) Task 62f912b7-03eb-4557-9342-8ae57aee4902: ref 0 aborting False
>Thread-15915178::INFO::2013-11-20 03:51:52,343::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00127196311951', 'lastCheck': 1384919505.7562399, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00254011154175', 'lastCheck': 1384919508.7519801, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00169205665588', 'lastCheck': 1384919505.7574, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000377178192139', 'lastCheck': 1384919510.7531769, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:51:53,679::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:51:54,270::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0955863 s, 10.7 MB/s\n'; <rc> = 0
>Thread-15915180::DEBUG::2013-11-20 03:51:55,679::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915180::INFO::2013-11-20 03:51:55,681::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915180::DEBUG::2013-11-20 03:51:55,682::task::495::TaskManager.Task::(_debug) Task 1f9e7d75-6970-4f74-ac5d-088dae52013d: moving from state init -> state preparing
>Thread-15915180::DEBUG::2013-11-20 03:51:55,683::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915180::DEBUG::2013-11-20 03:51:55,684::task::495::TaskManager.Task::(_debug) Task 1f9e7d75-6970-4f74-ac5d-088dae52013d: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915180::DEBUG::2013-11-20 03:51:55,685::task::495::TaskManager.Task::(_debug) Task 1f9e7d75-6970-4f74-ac5d-088dae52013d: moving from state preparing -> state finished
>Thread-15915180::DEBUG::2013-11-20 03:51:55,686::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915180::DEBUG::2013-11-20 03:51:55,686::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915180::DEBUG::2013-11-20 03:51:55,687::task::495::TaskManager.Task::(_debug) Task 1f9e7d75-6970-4f74-ac5d-088dae52013d: ref 0 aborting False
>Thread-15915180::INFO::2013-11-20 03:51:55,688::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915181::DEBUG::2013-11-20 03:51:55,694::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915181::INFO::2013-11-20 03:51:55,695::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915181::DEBUG::2013-11-20 03:51:55,695::task::495::TaskManager.Task::(_debug) Task 33fa95bb-65b5-4a99-8d97-0cecd630e9b7: moving from state init -> state preparing
>Thread-15915181::DEBUG::2013-11-20 03:51:55,696::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c4fcc68e-00cc-42bb-ac33-e69cc71ac9b9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915181::DEBUG::2013-11-20 03:51:55,697::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915181::DEBUG::2013-11-20 03:51:55,697::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915181::DEBUG::2013-11-20 03:51:55,698::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c4fcc68e-00cc-42bb-ac33-e69cc71ac9b9`::Granted request
>Thread-15915181::DEBUG::2013-11-20 03:51:55,699::task::495::TaskManager.Task::(_debug) Task 33fa95bb-65b5-4a99-8d97-0cecd630e9b7: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915181::DEBUG::2013-11-20 03:51:55,699::task::495::TaskManager.Task::(_debug) Task 33fa95bb-65b5-4a99-8d97-0cecd630e9b7: ref 1 aborting False
>Thread-15915181::WARNING::2013-11-20 03:51:55,700::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915181::DEBUG::2013-11-20 03:51:55,701::task::495::TaskManager.Task::(_debug) Task 33fa95bb-65b5-4a99-8d97-0cecd630e9b7: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915181::DEBUG::2013-11-20 03:51:55,702::task::495::TaskManager.Task::(_debug) Task 33fa95bb-65b5-4a99-8d97-0cecd630e9b7: moving from state preparing -> state finished
>Thread-15915181::DEBUG::2013-11-20 03:51:55,702::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915181::DEBUG::2013-11-20 03:51:55,702::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915181::DEBUG::2013-11-20 03:51:55,703::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915181::DEBUG::2013-11-20 03:51:55,703::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915181::DEBUG::2013-11-20 03:51:55,704::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915181::DEBUG::2013-11-20 03:51:55,704::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915181::DEBUG::2013-11-20 03:51:55,705::task::495::TaskManager.Task::(_debug) Task 33fa95bb-65b5-4a99-8d97-0cecd630e9b7: ref 0 aborting False
>Thread-15915181::INFO::2013-11-20 03:51:55,706::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913518::INFO::2013-11-20 03:51:56,118::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:51:56,119::task::495::TaskManager.Task::(_debug) Task 9a32d5d4-b658-4c83-b4b9-1e186034813c: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:51:56,121::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`94f522a6-f661-4308-83ac-0582d9a1734d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:51:56,121::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:51:56,122::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:51:56,123::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`94f522a6-f661-4308-83ac-0582d9a1734d`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:51:56,124::task::495::TaskManager.Task::(_debug) Task 9a32d5d4-b658-4c83-b4b9-1e186034813c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:51:56,125::task::495::TaskManager.Task::(_debug) Task 9a32d5d4-b658-4c83-b4b9-1e186034813c: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:51:56,126::task::495::TaskManager.Task::(_debug) Task 9a32d5d4-b658-4c83-b4b9-1e186034813c: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:51:56,126::task::495::TaskManager.Task::(_debug) Task 9a32d5d4-b658-4c83-b4b9-1e186034813c: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:51:56,127::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:51:56,128::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:51:56,128::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:51:56,129::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:51:56,130::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:51:56,130::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:51:56,131::task::495::TaskManager.Task::(_debug) Task 9a32d5d4-b658-4c83-b4b9-1e186034813c: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:51:56,132::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:51:56,133::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:51:56,134::task::495::TaskManager.Task::(_debug) Task 8d7c71e9-1ca0-419a-ad29-5fb085ca0bc8: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:51:56,135::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d61a6b51-a285-4175-9a60-40156b6a2a83`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:51:56,135::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:51:56,136::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:51:56,137::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d61a6b51-a285-4175-9a60-40156b6a2a83`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:51:56,138::task::495::TaskManager.Task::(_debug) Task 8d7c71e9-1ca0-419a-ad29-5fb085ca0bc8: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:51:56,138::task::495::TaskManager.Task::(_debug) Task 8d7c71e9-1ca0-419a-ad29-5fb085ca0bc8: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:51:56,139::task::495::TaskManager.Task::(_debug) Task 8d7c71e9-1ca0-419a-ad29-5fb085ca0bc8: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:51:56,140::task::495::TaskManager.Task::(_debug) Task 8d7c71e9-1ca0-419a-ad29-5fb085ca0bc8: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:51:56,140::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:51:56,141::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:51:56,142::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:51:56,143::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:51:56,143::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:51:56,144::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:51:56,145::task::495::TaskManager.Task::(_debug) Task 8d7c71e9-1ca0-419a-ad29-5fb085ca0bc8: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:51:56,146::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:51:56,286::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:51:56,928::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0996702 s, 10.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:51:58,950::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:51:59,536::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0803584 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:52:01,552::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:52:02,174::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0857743 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:52:04,200::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:52:04,758::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0935215 s, 10.9 MB/s\n'; <rc> = 0
>Thread-15913639::INFO::2013-11-20 03:52:04,809::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:52:04,810::task::495::TaskManager.Task::(_debug) Task af187380-e9e6-4df9-bb3c-ef477c411f3a: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:52:04,822::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`64c57406-2a95-4711-b272-2a32740db819`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:52:04,823::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:52:04,824::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:52:04,825::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`64c57406-2a95-4711-b272-2a32740db819`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:52:04,828::task::495::TaskManager.Task::(_debug) Task af187380-e9e6-4df9-bb3c-ef477c411f3a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:52:04,828::task::495::TaskManager.Task::(_debug) Task af187380-e9e6-4df9-bb3c-ef477c411f3a: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:52:04,829::task::495::TaskManager.Task::(_debug) Task af187380-e9e6-4df9-bb3c-ef477c411f3a: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:52:04,830::task::495::TaskManager.Task::(_debug) Task af187380-e9e6-4df9-bb3c-ef477c411f3a: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:52:04,831::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:52:04,831::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:52:04,832::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:52:04,833::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:52:04,833::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:52:04,834::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:52:04,835::task::495::TaskManager.Task::(_debug) Task af187380-e9e6-4df9-bb3c-ef477c411f3a: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:52:04,835::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15915186::DEBUG::2013-11-20 03:52:06,262::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915186::INFO::2013-11-20 03:52:06,264::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915186::DEBUG::2013-11-20 03:52:06,265::task::495::TaskManager.Task::(_debug) Task fdad2636-a73e-403b-9d70-1af48385127a: moving from state init -> state preparing
>Thread-15915186::DEBUG::2013-11-20 03:52:06,266::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915186::DEBUG::2013-11-20 03:52:06,266::task::495::TaskManager.Task::(_debug) Task fdad2636-a73e-403b-9d70-1af48385127a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915186::DEBUG::2013-11-20 03:52:06,267::task::495::TaskManager.Task::(_debug) Task fdad2636-a73e-403b-9d70-1af48385127a: moving from state preparing -> state finished
>Thread-15915186::DEBUG::2013-11-20 03:52:06,268::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915186::DEBUG::2013-11-20 03:52:06,268::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915186::DEBUG::2013-11-20 03:52:06,269::task::495::TaskManager.Task::(_debug) Task fdad2636-a73e-403b-9d70-1af48385127a: ref 0 aborting False
>Thread-15915186::INFO::2013-11-20 03:52:06,270::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915187::DEBUG::2013-11-20 03:52:06,597::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915187::INFO::2013-11-20 03:52:06,599::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915187::DEBUG::2013-11-20 03:52:06,600::task::495::TaskManager.Task::(_debug) Task d30c0b72-b187-4a88-be57-e502eaa99a28: moving from state init -> state preparing
>Thread-15915187::DEBUG::2013-11-20 03:52:06,602::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3d4c504e-4933-42bf-a2e3-b12d80b8d891`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915187::DEBUG::2013-11-20 03:52:06,603::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915187::DEBUG::2013-11-20 03:52:06,604::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915187::DEBUG::2013-11-20 03:52:06,605::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3d4c504e-4933-42bf-a2e3-b12d80b8d891`::Granted request
>Thread-15915187::DEBUG::2013-11-20 03:52:06,606::task::495::TaskManager.Task::(_debug) Task d30c0b72-b187-4a88-be57-e502eaa99a28: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915187::DEBUG::2013-11-20 03:52:06,607::task::495::TaskManager.Task::(_debug) Task d30c0b72-b187-4a88-be57-e502eaa99a28: ref 1 aborting False
>Thread-15915187::WARNING::2013-11-20 03:52:06,609::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915187::DEBUG::2013-11-20 03:52:06,610::task::495::TaskManager.Task::(_debug) Task d30c0b72-b187-4a88-be57-e502eaa99a28: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915187::DEBUG::2013-11-20 03:52:06,610::task::495::TaskManager.Task::(_debug) Task d30c0b72-b187-4a88-be57-e502eaa99a28: moving from state preparing -> state finished
>Thread-15915187::DEBUG::2013-11-20 03:52:06,611::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915187::DEBUG::2013-11-20 03:52:06,611::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915187::DEBUG::2013-11-20 03:52:06,612::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915187::DEBUG::2013-11-20 03:52:06,613::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915187::DEBUG::2013-11-20 03:52:06,613::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915187::DEBUG::2013-11-20 03:52:06,614::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915187::DEBUG::2013-11-20 03:52:06,614::task::495::TaskManager.Task::(_debug) Task d30c0b72-b187-4a88-be57-e502eaa99a28: ref 0 aborting False
>Thread-15915187::INFO::2013-11-20 03:52:06,615::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:52:06,777::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:52:07,432::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.106771 s, 9.6 MB/s\n'; <rc> = 0
>Thread-15915188::INFO::2013-11-20 03:52:08,194::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915188::DEBUG::2013-11-20 03:52:08,195::task::495::TaskManager.Task::(_debug) Task edc6c340-a877-4f42-940f-92542ab51e1e: moving from state init -> state preparing
>Thread-15915188::DEBUG::2013-11-20 03:52:08,196::task::495::TaskManager.Task::(_debug) Task edc6c340-a877-4f42-940f-92542ab51e1e: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00230598449707', 'lastCheck': 1384919525.913806, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.000926971435547', 'lastCheck': 1384919518.8986571, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00942993164062', 'lastCheck': 1384919525.9250391, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000420093536377', 'lastCheck': 1384919520.895026, 'valid': True, 'code': 0}}
>Thread-15915188::DEBUG::2013-11-20 03:52:08,197::task::495::TaskManager.Task::(_debug) Task edc6c340-a877-4f42-940f-92542ab51e1e: moving from state preparing -> state finished
>Thread-15915188::DEBUG::2013-11-20 03:52:08,198::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915188::DEBUG::2013-11-20 03:52:08,199::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915188::DEBUG::2013-11-20 03:52:08,199::task::495::TaskManager.Task::(_debug) Task edc6c340-a877-4f42-940f-92542ab51e1e: ref 0 aborting False
>Thread-15915188::INFO::2013-11-20 03:52:08,200::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00230598449707', 'lastCheck': 1384919525.913806, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.000926971435547', 'lastCheck': 1384919518.8986571, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00942993164062', 'lastCheck': 1384919525.9250391, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000420093536377', 'lastCheck': 1384919520.895026, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:52:09,447::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:52:10,410::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0977252 s, 10.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:52:12,430::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:52:13,029::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0836119 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:52:15,046::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:52:15,538::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0598147 s, 17.1 MB/s\n'; <rc> = 0
>Thread-42838::DEBUG::2013-11-20 03:52:16,015::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 03:52:16,752::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913658::INFO::2013-11-20 03:52:17,015::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:52:17,016::task::495::TaskManager.Task::(_debug) Task a57a4cd4-a1d0-4132-b789-95fcd5f53d6f: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:52:17,018::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`be385206-fa2e-422f-ac9b-eb2dd3c20260`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:52:17,018::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:52:17,019::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:52:17,020::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`be385206-fa2e-422f-ac9b-eb2dd3c20260`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:52:17,021::task::495::TaskManager.Task::(_debug) Task a57a4cd4-a1d0-4132-b789-95fcd5f53d6f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:52:17,022::task::495::TaskManager.Task::(_debug) Task a57a4cd4-a1d0-4132-b789-95fcd5f53d6f: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:52:17,022::task::495::TaskManager.Task::(_debug) Task a57a4cd4-a1d0-4132-b789-95fcd5f53d6f: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:52:17,023::task::495::TaskManager.Task::(_debug) Task a57a4cd4-a1d0-4132-b789-95fcd5f53d6f: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:52:17,024::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:52:17,024::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:52:17,025::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:52:17,025::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:52:17,026::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:52:17,026::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:52:17,027::task::495::TaskManager.Task::(_debug) Task a57a4cd4-a1d0-4132-b789-95fcd5f53d6f: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:52:17,027::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15915192::DEBUG::2013-11-20 03:52:17,321::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915192::INFO::2013-11-20 03:52:17,322::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915192::DEBUG::2013-11-20 03:52:17,322::task::495::TaskManager.Task::(_debug) Task 09d8c8f4-30f7-4af4-9d56-9ca5c5daa8f8: moving from state init -> state preparing
>Thread-15915192::DEBUG::2013-11-20 03:52:17,323::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915192::DEBUG::2013-11-20 03:52:17,324::task::495::TaskManager.Task::(_debug) Task 09d8c8f4-30f7-4af4-9d56-9ca5c5daa8f8: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915192::DEBUG::2013-11-20 03:52:17,325::task::495::TaskManager.Task::(_debug) Task 09d8c8f4-30f7-4af4-9d56-9ca5c5daa8f8: moving from state preparing -> state finished
>Thread-15915192::DEBUG::2013-11-20 03:52:17,325::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915192::DEBUG::2013-11-20 03:52:17,326::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915192::DEBUG::2013-11-20 03:52:17,327::task::495::TaskManager.Task::(_debug) Task 09d8c8f4-30f7-4af4-9d56-9ca5c5daa8f8: ref 0 aborting False
>Thread-15915192::INFO::2013-11-20 03:52:17,328::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915193::DEBUG::2013-11-20 03:52:17,332::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915193::INFO::2013-11-20 03:52:17,334::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915193::DEBUG::2013-11-20 03:52:17,334::task::495::TaskManager.Task::(_debug) Task 045c31b5-3c01-4b58-be5f-debf92c2b128: moving from state init -> state preparing
>Thread-15915193::DEBUG::2013-11-20 03:52:17,336::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`47160b47-b1d1-4f8f-9e1c-552287d629ab`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915193::DEBUG::2013-11-20 03:52:17,336::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915193::DEBUG::2013-11-20 03:52:17,337::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915193::DEBUG::2013-11-20 03:52:17,338::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`47160b47-b1d1-4f8f-9e1c-552287d629ab`::Granted request
>Thread-15915193::DEBUG::2013-11-20 03:52:17,339::task::495::TaskManager.Task::(_debug) Task 045c31b5-3c01-4b58-be5f-debf92c2b128: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915193::DEBUG::2013-11-20 03:52:17,340::task::495::TaskManager.Task::(_debug) Task 045c31b5-3c01-4b58-be5f-debf92c2b128: ref 1 aborting False
>Thread-15915193::WARNING::2013-11-20 03:52:17,342::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915193::DEBUG::2013-11-20 03:52:17,344::task::495::TaskManager.Task::(_debug) Task 045c31b5-3c01-4b58-be5f-debf92c2b128: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915193::DEBUG::2013-11-20 03:52:17,345::task::495::TaskManager.Task::(_debug) Task 045c31b5-3c01-4b58-be5f-debf92c2b128: moving from state preparing -> state finished
>Thread-15915193::DEBUG::2013-11-20 03:52:17,345::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915193::DEBUG::2013-11-20 03:52:17,346::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915193::DEBUG::2013-11-20 03:52:17,347::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915193::DEBUG::2013-11-20 03:52:17,347::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915193::DEBUG::2013-11-20 03:52:17,348::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915193::DEBUG::2013-11-20 03:52:17,349::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915193::DEBUG::2013-11-20 03:52:17,349::task::495::TaskManager.Task::(_debug) Task 045c31b5-3c01-4b58-be5f-debf92c2b128: ref 0 aborting False
>Thread-15915193::INFO::2013-11-20 03:52:17,350::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:52:17,548::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:52:18,078::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0765447 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:52:20,091::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:52:20,636::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0760936 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:52:22,652::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:52:23,225::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0757318 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15915196::INFO::2013-11-20 03:52:24,333::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915196::DEBUG::2013-11-20 03:52:24,334::task::495::TaskManager.Task::(_debug) Task 85d904a1-7292-451e-bae3-ef56789da008: moving from state init -> state preparing
>Thread-15915196::DEBUG::2013-11-20 03:52:24,335::task::495::TaskManager.Task::(_debug) Task 85d904a1-7292-451e-bae3-ef56789da008: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000310897827148', 'lastCheck': 1384919536.75721, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00122022628784', 'lastCheck': 1384919539.1816549, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00173091888428', 'lastCheck': 1384919536.0162089, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000319957733154', 'lastCheck': 1384919541.0181489, 'valid': True, 'code': 0}}
>Thread-15915196::DEBUG::2013-11-20 03:52:24,336::task::495::TaskManager.Task::(_debug) Task 85d904a1-7292-451e-bae3-ef56789da008: moving from state preparing -> state finished
>Thread-15915196::DEBUG::2013-11-20 03:52:24,336::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915196::DEBUG::2013-11-20 03:52:24,337::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915196::DEBUG::2013-11-20 03:52:24,338::task::495::TaskManager.Task::(_debug) Task 85d904a1-7292-451e-bae3-ef56789da008: ref 0 aborting False
>Thread-15915196::INFO::2013-11-20 03:52:24,338::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000310897827148', 'lastCheck': 1384919536.75721, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00122022628784', 'lastCheck': 1384919539.1816549, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00173091888428', 'lastCheck': 1384919536.0162089, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000319957733154', 'lastCheck': 1384919541.0181489, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:52:25,241::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:52:25,851::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0885231 s, 11.6 MB/s\n'; <rc> = 0
>Thread-15915198::DEBUG::2013-11-20 03:52:27,537::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915198::INFO::2013-11-20 03:52:27,538::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915198::DEBUG::2013-11-20 03:52:27,539::task::495::TaskManager.Task::(_debug) Task 308837bf-6677-45e3-99ff-3d10c109f746: moving from state init -> state preparing
>Thread-15915198::DEBUG::2013-11-20 03:52:27,540::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915198::DEBUG::2013-11-20 03:52:27,541::task::495::TaskManager.Task::(_debug) Task 308837bf-6677-45e3-99ff-3d10c109f746: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915198::DEBUG::2013-11-20 03:52:27,541::task::495::TaskManager.Task::(_debug) Task 308837bf-6677-45e3-99ff-3d10c109f746: moving from state preparing -> state finished
>Thread-15915198::DEBUG::2013-11-20 03:52:27,542::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915198::DEBUG::2013-11-20 03:52:27,542::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915198::DEBUG::2013-11-20 03:52:27,543::task::495::TaskManager.Task::(_debug) Task 308837bf-6677-45e3-99ff-3d10c109f746: ref 0 aborting False
>Thread-15915198::INFO::2013-11-20 03:52:27,544::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915199::DEBUG::2013-11-20 03:52:27,549::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915199::INFO::2013-11-20 03:52:27,550::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915199::DEBUG::2013-11-20 03:52:27,551::task::495::TaskManager.Task::(_debug) Task a80bee96-5c6c-4b48-b742-879ddbd821ba: moving from state init -> state preparing
>Thread-15915199::DEBUG::2013-11-20 03:52:27,553::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`df264608-4f31-4617-92a2-17adfd9d01e2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915199::DEBUG::2013-11-20 03:52:27,553::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915199::DEBUG::2013-11-20 03:52:27,554::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915199::DEBUG::2013-11-20 03:52:27,555::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`df264608-4f31-4617-92a2-17adfd9d01e2`::Granted request
>Thread-15915199::DEBUG::2013-11-20 03:52:27,556::task::495::TaskManager.Task::(_debug) Task a80bee96-5c6c-4b48-b742-879ddbd821ba: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915199::DEBUG::2013-11-20 03:52:27,557::task::495::TaskManager.Task::(_debug) Task a80bee96-5c6c-4b48-b742-879ddbd821ba: ref 1 aborting False
>Thread-15915199::WARNING::2013-11-20 03:52:27,559::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915199::DEBUG::2013-11-20 03:52:27,560::task::495::TaskManager.Task::(_debug) Task a80bee96-5c6c-4b48-b742-879ddbd821ba: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915199::DEBUG::2013-11-20 03:52:27,561::task::495::TaskManager.Task::(_debug) Task a80bee96-5c6c-4b48-b742-879ddbd821ba: moving from state preparing -> state finished
>Thread-15915199::DEBUG::2013-11-20 03:52:27,561::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915199::DEBUG::2013-11-20 03:52:27,562::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915199::DEBUG::2013-11-20 03:52:27,563::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915199::DEBUG::2013-11-20 03:52:27,563::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915199::DEBUG::2013-11-20 03:52:27,564::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915199::DEBUG::2013-11-20 03:52:27,565::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915199::DEBUG::2013-11-20 03:52:27,566::task::495::TaskManager.Task::(_debug) Task a80bee96-5c6c-4b48-b742-879ddbd821ba: ref 0 aborting False
>Thread-15915199::INFO::2013-11-20 03:52:27,567::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:52:27,866::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:52:28,474::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.072815 s, 14.1 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 03:52:28,894::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:52:28,895::task::495::TaskManager.Task::(_debug) Task 1661480a-fdb7-454e-bfec-4a44c6a32888: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:52:28,897::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`52d4a9a0-f650-48a2-8243-fa5d5d45bd63`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:52:28,897::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:52:28,898::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:52:28,899::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`52d4a9a0-f650-48a2-8243-fa5d5d45bd63`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:52:28,900::task::495::TaskManager.Task::(_debug) Task 1661480a-fdb7-454e-bfec-4a44c6a32888: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:52:28,901::task::495::TaskManager.Task::(_debug) Task 1661480a-fdb7-454e-bfec-4a44c6a32888: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:52:28,901::task::495::TaskManager.Task::(_debug) Task 1661480a-fdb7-454e-bfec-4a44c6a32888: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:52:28,902::task::495::TaskManager.Task::(_debug) Task 1661480a-fdb7-454e-bfec-4a44c6a32888: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:52:28,903::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:52:28,903::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:52:28,904::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:52:28,905::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:52:28,906::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:52:28,906::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:52:28,907::task::495::TaskManager.Task::(_debug) Task 1661480a-fdb7-454e-bfec-4a44c6a32888: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:52:28,908::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913695::INFO::2013-11-20 03:52:30,493::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:52:30,494::task::495::TaskManager.Task::(_debug) Task 87d81dc1-7c02-4b5e-b912-dda1c03a9a20: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:52:30,496::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`84a72a14-19c0-4f19-ad57-03c126726bd8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:52:30,496::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Dummy-15912651::DEBUG::2013-11-20 03:52:30,497::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::DEBUG::2013-11-20 03:52:30,499::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:52:30,500::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`84a72a14-19c0-4f19-ad57-03c126726bd8`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:52:30,808::task::495::TaskManager.Task::(_debug) Task 87d81dc1-7c02-4b5e-b912-dda1c03a9a20: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:52:30,809::task::495::TaskManager.Task::(_debug) Task 87d81dc1-7c02-4b5e-b912-dda1c03a9a20: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:52:30,811::task::495::TaskManager.Task::(_debug) Task 87d81dc1-7c02-4b5e-b912-dda1c03a9a20: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:52:30,812::task::495::TaskManager.Task::(_debug) Task 87d81dc1-7c02-4b5e-b912-dda1c03a9a20: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:52:30,813::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:52:30,814::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:52:30,815::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:52:30,816::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:52:30,817::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:52:30,818::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:52:30,819::task::495::TaskManager.Task::(_debug) Task 87d81dc1-7c02-4b5e-b912-dda1c03a9a20: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:52:30,820::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:52:31,046::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0638588 s, 16.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:52:33,063::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:52:33,610::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.094537 s, 10.8 MB/s\n'; <rc> = 0
>Thread-15915085::INFO::2013-11-20 03:52:35,026::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=089b08fc-6544-4235-a061-fb55b09d2715 volUUID=66b599fc-b7b5-4934-a5d5-ef7283460029)
>Thread-15915085::DEBUG::2013-11-20 03:52:35,028::task::495::TaskManager.Task::(_debug) Task 2244c69a-dae4-4624-b52d-c9ac35e92d2d: moving from state init -> state preparing
>Thread-15915085::DEBUG::2013-11-20 03:52:35,030::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e33e7d4c-b7b5-4eeb-9df6-e2d88605bc7d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915085::DEBUG::2013-11-20 03:52:35,031::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915085::DEBUG::2013-11-20 03:52:35,032::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915085::DEBUG::2013-11-20 03:52:35,033::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e33e7d4c-b7b5-4eeb-9df6-e2d88605bc7d`::Granted request
>Thread-15915085::DEBUG::2013-11-20 03:52:35,043::task::495::TaskManager.Task::(_debug) Task 2244c69a-dae4-4624-b52d-c9ac35e92d2d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915085::DEBUG::2013-11-20 03:52:35,044::task::495::TaskManager.Task::(_debug) Task 2244c69a-dae4-4624-b52d-c9ac35e92d2d: ref 1 aborting False
>Thread-15915085::DEBUG::2013-11-20 03:52:35,045::task::495::TaskManager.Task::(_debug) Task 2244c69a-dae4-4624-b52d-c9ac35e92d2d: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15915085::DEBUG::2013-11-20 03:52:35,045::task::495::TaskManager.Task::(_debug) Task 2244c69a-dae4-4624-b52d-c9ac35e92d2d: moving from state preparing -> state finished
>Thread-15915085::DEBUG::2013-11-20 03:52:35,046::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915085::DEBUG::2013-11-20 03:52:35,047::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915085::DEBUG::2013-11-20 03:52:35,048::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915085::DEBUG::2013-11-20 03:52:35,049::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915085::DEBUG::2013-11-20 03:52:35,049::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915085::DEBUG::2013-11-20 03:52:35,050::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915085::DEBUG::2013-11-20 03:52:35,051::task::495::TaskManager.Task::(_debug) Task 2244c69a-dae4-4624-b52d-c9ac35e92d2d: ref 0 aborting False
>Thread-15915085::INFO::2013-11-20 03:52:35,052::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Dummy-15912651::DEBUG::2013-11-20 03:52:35,625::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:52:36,225::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0816023 s, 12.5 MB/s\n'; <rc> = 0
>Thread-15915204::DEBUG::2013-11-20 03:52:37,847::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915204::INFO::2013-11-20 03:52:37,849::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915204::DEBUG::2013-11-20 03:52:37,850::task::495::TaskManager.Task::(_debug) Task a7cd5144-83b7-4fe8-93f8-ef4ab3df1720: moving from state init -> state preparing
>Thread-15915204::DEBUG::2013-11-20 03:52:37,851::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915204::DEBUG::2013-11-20 03:52:37,852::task::495::TaskManager.Task::(_debug) Task a7cd5144-83b7-4fe8-93f8-ef4ab3df1720: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915204::DEBUG::2013-11-20 03:52:37,853::task::495::TaskManager.Task::(_debug) Task a7cd5144-83b7-4fe8-93f8-ef4ab3df1720: moving from state preparing -> state finished
>Thread-15915204::DEBUG::2013-11-20 03:52:37,854::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915204::DEBUG::2013-11-20 03:52:37,855::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915204::DEBUG::2013-11-20 03:52:37,855::task::495::TaskManager.Task::(_debug) Task a7cd5144-83b7-4fe8-93f8-ef4ab3df1720: ref 0 aborting False
>Thread-15915204::INFO::2013-11-20 03:52:37,856::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915205::DEBUG::2013-11-20 03:52:37,862::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915205::INFO::2013-11-20 03:52:37,863::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915205::DEBUG::2013-11-20 03:52:37,864::task::495::TaskManager.Task::(_debug) Task 8d7d8ed1-ade7-4d69-9a87-b49e863c2527: moving from state init -> state preparing
>Thread-15915205::DEBUG::2013-11-20 03:52:37,865::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5964bbe3-5306-4ed9-8fd9-fe3d4805c2c4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915205::DEBUG::2013-11-20 03:52:37,866::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915205::DEBUG::2013-11-20 03:52:37,867::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915205::DEBUG::2013-11-20 03:52:37,868::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5964bbe3-5306-4ed9-8fd9-fe3d4805c2c4`::Granted request
>Thread-15915205::DEBUG::2013-11-20 03:52:37,869::task::495::TaskManager.Task::(_debug) Task 8d7d8ed1-ade7-4d69-9a87-b49e863c2527: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915205::DEBUG::2013-11-20 03:52:37,869::task::495::TaskManager.Task::(_debug) Task 8d7d8ed1-ade7-4d69-9a87-b49e863c2527: ref 1 aborting False
>Thread-15915205::WARNING::2013-11-20 03:52:37,872::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915205::DEBUG::2013-11-20 03:52:37,873::task::495::TaskManager.Task::(_debug) Task 8d7d8ed1-ade7-4d69-9a87-b49e863c2527: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915205::DEBUG::2013-11-20 03:52:37,874::task::495::TaskManager.Task::(_debug) Task 8d7d8ed1-ade7-4d69-9a87-b49e863c2527: moving from state preparing -> state finished
>Thread-15915205::DEBUG::2013-11-20 03:52:37,874::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915205::DEBUG::2013-11-20 03:52:37,875::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915205::DEBUG::2013-11-20 03:52:37,876::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915205::DEBUG::2013-11-20 03:52:37,877::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915205::DEBUG::2013-11-20 03:52:37,878::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915205::DEBUG::2013-11-20 03:52:37,878::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915205::DEBUG::2013-11-20 03:52:37,879::task::495::TaskManager.Task::(_debug) Task 8d7d8ed1-ade7-4d69-9a87-b49e863c2527: ref 0 aborting False
>Thread-15915205::INFO::2013-11-20 03:52:37,880::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:52:38,239::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:52:38,760::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0838795 s, 12.2 MB/s\n'; <rc> = 0
>Thread-15914453::INFO::2013-11-20 03:52:39,043::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 03:52:39,045::task::495::TaskManager.Task::(_debug) Task 34ca6c46-c006-4abf-ab5c-f711a9f08b1e: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 03:52:39,048::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3b9f43ca-a9e5-4ea8-8031-7592c6be115c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 03:52:39,049::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 03:52:39,051::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 03:52:39,051::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3b9f43ca-a9e5-4ea8-8031-7592c6be115c`::Granted request
>Thread-15914453::DEBUG::2013-11-20 03:52:39,053::task::495::TaskManager.Task::(_debug) Task 34ca6c46-c006-4abf-ab5c-f711a9f08b1e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 03:52:39,054::task::495::TaskManager.Task::(_debug) Task 34ca6c46-c006-4abf-ab5c-f711a9f08b1e: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 03:52:39,055::task::495::TaskManager.Task::(_debug) Task 34ca6c46-c006-4abf-ab5c-f711a9f08b1e: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 03:52:39,056::task::495::TaskManager.Task::(_debug) Task 34ca6c46-c006-4abf-ab5c-f711a9f08b1e: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 03:52:39,057::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 03:52:39,058::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 03:52:39,059::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 03:52:39,060::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 03:52:39,062::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 03:52:39,062::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 03:52:39,064::task::495::TaskManager.Task::(_debug) Task 34ca6c46-c006-4abf-ab5c-f711a9f08b1e: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 03:52:39,065::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914435::INFO::2013-11-20 03:52:39,538::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 03:52:39,539::task::495::TaskManager.Task::(_debug) Task 22d4c162-ad6d-4b98-8483-9550d80cdf0f: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 03:52:39,541::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d64d7381-1661-4245-919d-75a73c9ad4e8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 03:52:39,541::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 03:52:39,542::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 03:52:39,543::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d64d7381-1661-4245-919d-75a73c9ad4e8`::Granted request
>Thread-15914435::DEBUG::2013-11-20 03:52:39,544::task::495::TaskManager.Task::(_debug) Task 22d4c162-ad6d-4b98-8483-9550d80cdf0f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 03:52:39,545::task::495::TaskManager.Task::(_debug) Task 22d4c162-ad6d-4b98-8483-9550d80cdf0f: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 03:52:39,545::task::495::TaskManager.Task::(_debug) Task 22d4c162-ad6d-4b98-8483-9550d80cdf0f: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 03:52:39,546::task::495::TaskManager.Task::(_debug) Task 22d4c162-ad6d-4b98-8483-9550d80cdf0f: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 03:52:39,547::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 03:52:39,548::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 03:52:39,548::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 03:52:39,549::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 03:52:39,550::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 03:52:39,550::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 03:52:39,551::task::495::TaskManager.Task::(_debug) Task 22d4c162-ad6d-4b98-8483-9550d80cdf0f: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 03:52:39,552::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15915206::INFO::2013-11-20 03:52:39,909::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915206::DEBUG::2013-11-20 03:52:39,909::task::495::TaskManager.Task::(_debug) Task 4aa5d3b1-b422-4676-acbd-f5a5a9635063: moving from state init -> state preparing
>Thread-15915206::DEBUG::2013-11-20 03:52:39,910::task::495::TaskManager.Task::(_debug) Task 4aa5d3b1-b422-4676-acbd-f5a5a9635063: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000380992889404', 'lastCheck': 1384919557.2733541, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00208783149719', 'lastCheck': 1384919559.477309, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00177717208862', 'lastCheck': 1384919556.2722399, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000372886657715', 'lastCheck': 1384919551.1164949, 'valid': True, 'code': 0}}
>Thread-15915206::DEBUG::2013-11-20 03:52:39,911::task::495::TaskManager.Task::(_debug) Task 4aa5d3b1-b422-4676-acbd-f5a5a9635063: moving from state preparing -> state finished
>Thread-15915206::DEBUG::2013-11-20 03:52:39,912::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915206::DEBUG::2013-11-20 03:52:39,913::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915206::DEBUG::2013-11-20 03:52:39,913::task::495::TaskManager.Task::(_debug) Task 4aa5d3b1-b422-4676-acbd-f5a5a9635063: ref 0 aborting False
>Thread-15915206::INFO::2013-11-20 03:52:39,914::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000380992889404', 'lastCheck': 1384919557.2733541, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00208783149719', 'lastCheck': 1384919559.477309, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00177717208862', 'lastCheck': 1384919556.2722399, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000372886657715', 'lastCheck': 1384919551.1164949, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:52:40,775::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:52:41,384::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.089675 s, 11.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:52:43,402::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:52:44,029::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0945641 s, 10.8 MB/s\n'; <rc> = 0
>Thread-15914519::INFO::2013-11-20 03:52:45,637::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 03:52:45,638::task::495::TaskManager.Task::(_debug) Task f8fbb9e7-02fe-4e45-bf80-ffa6510e4eff: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 03:52:45,647::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e6fb6c99-b84e-4de1-9543-0a48bd59a429`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 03:52:45,655::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 03:52:45,657::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 03:52:45,658::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e6fb6c99-b84e-4de1-9543-0a48bd59a429`::Granted request
>Thread-15914519::DEBUG::2013-11-20 03:52:45,659::task::495::TaskManager.Task::(_debug) Task f8fbb9e7-02fe-4e45-bf80-ffa6510e4eff: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 03:52:45,660::task::495::TaskManager.Task::(_debug) Task f8fbb9e7-02fe-4e45-bf80-ffa6510e4eff: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 03:52:45,661::task::495::TaskManager.Task::(_debug) Task f8fbb9e7-02fe-4e45-bf80-ffa6510e4eff: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 03:52:45,661::task::495::TaskManager.Task::(_debug) Task f8fbb9e7-02fe-4e45-bf80-ffa6510e4eff: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 03:52:45,662::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 03:52:45,662::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 03:52:45,663::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 03:52:45,663::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 03:52:45,664::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 03:52:45,664::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 03:52:45,665::task::495::TaskManager.Task::(_debug) Task f8fbb9e7-02fe-4e45-bf80-ffa6510e4eff: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 03:52:45,665::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 03:52:46,044::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:52:46,617::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0857364 s, 11.9 MB/s\n'; <rc> = 0
>Thread-15915210::DEBUG::2013-11-20 03:52:48,150::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915210::INFO::2013-11-20 03:52:48,152::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915210::DEBUG::2013-11-20 03:52:48,153::task::495::TaskManager.Task::(_debug) Task 7d79b40b-ab86-4acc-b78e-e4c346437fc2: moving from state init -> state preparing
>Thread-15915210::DEBUG::2013-11-20 03:52:48,154::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915210::DEBUG::2013-11-20 03:52:48,154::task::495::TaskManager.Task::(_debug) Task 7d79b40b-ab86-4acc-b78e-e4c346437fc2: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915210::DEBUG::2013-11-20 03:52:48,155::task::495::TaskManager.Task::(_debug) Task 7d79b40b-ab86-4acc-b78e-e4c346437fc2: moving from state preparing -> state finished
>Thread-15915210::DEBUG::2013-11-20 03:52:48,156::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915210::DEBUG::2013-11-20 03:52:48,157::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915210::DEBUG::2013-11-20 03:52:48,157::task::495::TaskManager.Task::(_debug) Task 7d79b40b-ab86-4acc-b78e-e4c346437fc2: ref 0 aborting False
>Thread-15915210::INFO::2013-11-20 03:52:48,158::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915211::DEBUG::2013-11-20 03:52:48,163::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915211::INFO::2013-11-20 03:52:48,164::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915211::DEBUG::2013-11-20 03:52:48,164::task::495::TaskManager.Task::(_debug) Task efdc934d-6cf0-434f-9d29-4e422e4d0495: moving from state init -> state preparing
>Thread-15915211::DEBUG::2013-11-20 03:52:48,166::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7dfaf15c-ec94-41a8-aea3-b1cf6250dc0f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915211::DEBUG::2013-11-20 03:52:48,166::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915211::DEBUG::2013-11-20 03:52:48,167::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915211::DEBUG::2013-11-20 03:52:48,168::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7dfaf15c-ec94-41a8-aea3-b1cf6250dc0f`::Granted request
>Thread-15915211::DEBUG::2013-11-20 03:52:48,169::task::495::TaskManager.Task::(_debug) Task efdc934d-6cf0-434f-9d29-4e422e4d0495: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915211::DEBUG::2013-11-20 03:52:48,170::task::495::TaskManager.Task::(_debug) Task efdc934d-6cf0-434f-9d29-4e422e4d0495: ref 1 aborting False
>Thread-15915211::WARNING::2013-11-20 03:52:48,172::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915211::DEBUG::2013-11-20 03:52:48,173::task::495::TaskManager.Task::(_debug) Task efdc934d-6cf0-434f-9d29-4e422e4d0495: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915211::DEBUG::2013-11-20 03:52:48,174::task::495::TaskManager.Task::(_debug) Task efdc934d-6cf0-434f-9d29-4e422e4d0495: moving from state preparing -> state finished
>Thread-15915211::DEBUG::2013-11-20 03:52:48,175::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915211::DEBUG::2013-11-20 03:52:48,175::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915211::DEBUG::2013-11-20 03:52:48,176::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915211::DEBUG::2013-11-20 03:52:48,177::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915211::DEBUG::2013-11-20 03:52:48,177::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915211::DEBUG::2013-11-20 03:52:48,178::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915211::DEBUG::2013-11-20 03:52:48,178::task::495::TaskManager.Task::(_debug) Task efdc934d-6cf0-434f-9d29-4e422e4d0495: ref 0 aborting False
>Thread-15915211::INFO::2013-11-20 03:52:48,179::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:52:48,632::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 03:52:49,055::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:52:49,056::task::495::TaskManager.Task::(_debug) Task 5aec8b5e-8509-430f-bfba-4e5b3582eac2: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:52:49,059::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`4f814490-935a-4d16-a670-0cd4be825bc5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:52:49,059::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:52:49,060::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:52:49,061::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`4f814490-935a-4d16-a670-0cd4be825bc5`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:52:49,063::task::495::TaskManager.Task::(_debug) Task 5aec8b5e-8509-430f-bfba-4e5b3582eac2: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:52:49,063::task::495::TaskManager.Task::(_debug) Task 5aec8b5e-8509-430f-bfba-4e5b3582eac2: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:52:49,064::task::495::TaskManager.Task::(_debug) Task 5aec8b5e-8509-430f-bfba-4e5b3582eac2: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:52:49,065::task::495::TaskManager.Task::(_debug) Task 5aec8b5e-8509-430f-bfba-4e5b3582eac2: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:52:49,066::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:52:49,066::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:52:49,067::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:52:49,068::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:52:49,069::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:52:49,069::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:52:49,070::task::495::TaskManager.Task::(_debug) Task 5aec8b5e-8509-430f-bfba-4e5b3582eac2: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:52:49,071::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:52:49,072::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:52:49,073::task::495::TaskManager.Task::(_debug) Task 661ae1a5-1df5-40bd-9678-59ce25afddce: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:52:49,074::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b7ce674d-5573-4f60-96ca-4c9b60a5da01`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:52:49,075::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:52:49,076::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:52:49,076::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b7ce674d-5573-4f60-96ca-4c9b60a5da01`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:52:49,077::task::495::TaskManager.Task::(_debug) Task 661ae1a5-1df5-40bd-9678-59ce25afddce: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:52:49,078::task::495::TaskManager.Task::(_debug) Task 661ae1a5-1df5-40bd-9678-59ce25afddce: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:52:49,079::task::495::TaskManager.Task::(_debug) Task 661ae1a5-1df5-40bd-9678-59ce25afddce: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:52:49,080::task::495::TaskManager.Task::(_debug) Task 661ae1a5-1df5-40bd-9678-59ce25afddce: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:52:49,080::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:52:49,080::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:52:49,081::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:52:49,081::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:52:49,082::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:52:49,082::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:52:49,083::task::495::TaskManager.Task::(_debug) Task 661ae1a5-1df5-40bd-9678-59ce25afddce: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:52:49,083::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:52:49,083::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:52:49,084::task::495::TaskManager.Task::(_debug) Task 5c6f14f5-9c8c-4430-9985-6edf22ec2ed1: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:52:49,084::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`5cfcb508-f46e-41e2-8a5e-603ed5fe5493`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:52:49,085::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:52:49,086::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:52:49,086::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`5cfcb508-f46e-41e2-8a5e-603ed5fe5493`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:52:49,087::task::495::TaskManager.Task::(_debug) Task 5c6f14f5-9c8c-4430-9985-6edf22ec2ed1: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:52:49,087::task::495::TaskManager.Task::(_debug) Task 5c6f14f5-9c8c-4430-9985-6edf22ec2ed1: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:52:49,088::task::495::TaskManager.Task::(_debug) Task 5c6f14f5-9c8c-4430-9985-6edf22ec2ed1: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:52:49,088::task::495::TaskManager.Task::(_debug) Task 5c6f14f5-9c8c-4430-9985-6edf22ec2ed1: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:52:49,088::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:52:49,089::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:52:49,089::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:52:49,089::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:52:49,090::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:52:49,090::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:52:49,091::task::495::TaskManager.Task::(_debug) Task 5c6f14f5-9c8c-4430-9985-6edf22ec2ed1: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:52:49,091::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:52:49,091::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:52:49,092::task::495::TaskManager.Task::(_debug) Task 2f410865-f333-4b3d-8630-a97a0c6e1030: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:52:49,092::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`4a714f00-60aa-4711-b1fe-e1c2b2fe282f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:52:49,093::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:52:49,093::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:52:49,093::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`4a714f00-60aa-4711-b1fe-e1c2b2fe282f`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:52:49,094::task::495::TaskManager.Task::(_debug) Task 2f410865-f333-4b3d-8630-a97a0c6e1030: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:52:49,094::task::495::TaskManager.Task::(_debug) Task 2f410865-f333-4b3d-8630-a97a0c6e1030: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:52:49,095::task::495::TaskManager.Task::(_debug) Task 2f410865-f333-4b3d-8630-a97a0c6e1030: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:52:49,095::task::495::TaskManager.Task::(_debug) Task 2f410865-f333-4b3d-8630-a97a0c6e1030: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:52:49,095::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:52:49,096::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:52:49,096::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:52:49,097::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:52:49,097::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:52:49,097::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:52:49,098::task::495::TaskManager.Task::(_debug) Task 2f410865-f333-4b3d-8630-a97a0c6e1030: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:52:49,098::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:52:49,271::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.102614 s, 10.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:52:51,288::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:52:51,883::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0845412 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:52:53,981::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:52:54,934::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0859847 s, 11.9 MB/s\n'; <rc> = 0
>Thread-15915214::INFO::2013-11-20 03:52:56,102::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915214::DEBUG::2013-11-20 03:52:56,103::task::495::TaskManager.Task::(_debug) Task d267b61e-0207-4b10-8c18-48e5d4608b73: moving from state init -> state preparing
>Thread-15915214::DEBUG::2013-11-20 03:52:56,104::task::495::TaskManager.Task::(_debug) Task d267b61e-0207-4b10-8c18-48e5d4608b73: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000313997268677', 'lastCheck': 1384919567.6424589, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0232179164886', 'lastCheck': 1384919569.8887739, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0366621017456', 'lastCheck': 1384919566.680052, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00132584571838', 'lastCheck': 1384919571.8654649, 'valid': True, 'code': 0}}
>Thread-15915214::DEBUG::2013-11-20 03:52:56,106::task::495::TaskManager.Task::(_debug) Task d267b61e-0207-4b10-8c18-48e5d4608b73: moving from state preparing -> state finished
>Thread-15915214::DEBUG::2013-11-20 03:52:56,106::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915214::DEBUG::2013-11-20 03:52:56,107::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915214::DEBUG::2013-11-20 03:52:56,108::task::495::TaskManager.Task::(_debug) Task d267b61e-0207-4b10-8c18-48e5d4608b73: ref 0 aborting False
>Thread-15915214::INFO::2013-11-20 03:52:56,108::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000313997268677', 'lastCheck': 1384919567.6424589, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0232179164886', 'lastCheck': 1384919569.8887739, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0366621017456', 'lastCheck': 1384919566.680052, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00132584571838', 'lastCheck': 1384919571.8654649, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:52:56,991::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913518::INFO::2013-11-20 03:52:57,371::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:52:57,372::task::495::TaskManager.Task::(_debug) Task b90110e0-950f-4802-b605-22a2b7462983: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:52:57,374::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8049da82-6e01-475d-987b-389cbb5b34d9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:52:57,375::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:52:57,376::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:52:57,377::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8049da82-6e01-475d-987b-389cbb5b34d9`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:52:57,378::task::495::TaskManager.Task::(_debug) Task b90110e0-950f-4802-b605-22a2b7462983: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:52:57,379::task::495::TaskManager.Task::(_debug) Task b90110e0-950f-4802-b605-22a2b7462983: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:52:57,380::task::495::TaskManager.Task::(_debug) Task b90110e0-950f-4802-b605-22a2b7462983: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:52:57,380::task::495::TaskManager.Task::(_debug) Task b90110e0-950f-4802-b605-22a2b7462983: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:52:57,381::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:52:57,382::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:52:57,383::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:52:57,383::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:52:57,384::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:52:57,384::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:52:57,385::task::495::TaskManager.Task::(_debug) Task b90110e0-950f-4802-b605-22a2b7462983: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:52:57,386::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:52:57,387::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:52:57,387::task::495::TaskManager.Task::(_debug) Task 51db1b4d-f41f-420a-8c46-9cbdfe67f48a: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:52:57,388::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`763631b9-ce3f-405a-a535-2e549910eb07`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:52:57,389::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:52:57,389::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:52:57,390::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`763631b9-ce3f-405a-a535-2e549910eb07`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:52:57,390::task::495::TaskManager.Task::(_debug) Task 51db1b4d-f41f-420a-8c46-9cbdfe67f48a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:52:57,390::task::495::TaskManager.Task::(_debug) Task 51db1b4d-f41f-420a-8c46-9cbdfe67f48a: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:52:57,391::task::495::TaskManager.Task::(_debug) Task 51db1b4d-f41f-420a-8c46-9cbdfe67f48a: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:52:57,391::task::495::TaskManager.Task::(_debug) Task 51db1b4d-f41f-420a-8c46-9cbdfe67f48a: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:52:57,392::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:52:57,392::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:52:57,392::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:52:57,393::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:52:57,393::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:52:57,394::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:52:57,394::task::495::TaskManager.Task::(_debug) Task 51db1b4d-f41f-420a-8c46-9cbdfe67f48a: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:52:57,394::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:52:57,711::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0869352 s, 11.8 MB/s\n'; <rc> = 0
>Thread-15915216::DEBUG::2013-11-20 03:52:58,524::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915216::INFO::2013-11-20 03:52:58,526::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915216::DEBUG::2013-11-20 03:52:58,526::task::495::TaskManager.Task::(_debug) Task ac5da79d-b336-4438-80f7-d06971158be4: moving from state init -> state preparing
>Thread-15915216::DEBUG::2013-11-20 03:52:58,527::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915216::DEBUG::2013-11-20 03:52:58,528::task::495::TaskManager.Task::(_debug) Task ac5da79d-b336-4438-80f7-d06971158be4: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915216::DEBUG::2013-11-20 03:52:58,529::task::495::TaskManager.Task::(_debug) Task ac5da79d-b336-4438-80f7-d06971158be4: moving from state preparing -> state finished
>Thread-15915216::DEBUG::2013-11-20 03:52:58,530::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915216::DEBUG::2013-11-20 03:52:58,530::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915216::DEBUG::2013-11-20 03:52:58,531::task::495::TaskManager.Task::(_debug) Task ac5da79d-b336-4438-80f7-d06971158be4: ref 0 aborting False
>Thread-15915216::INFO::2013-11-20 03:52:58,532::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915217::DEBUG::2013-11-20 03:52:58,538::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915217::INFO::2013-11-20 03:52:58,539::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915217::DEBUG::2013-11-20 03:52:58,540::task::495::TaskManager.Task::(_debug) Task d42d3345-9583-4e28-8fa3-0b8a308aee8b: moving from state init -> state preparing
>Thread-15915217::DEBUG::2013-11-20 03:52:58,541::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`63d08415-bc7f-4534-9ab8-1b094f96edc0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915217::DEBUG::2013-11-20 03:52:58,542::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915217::DEBUG::2013-11-20 03:52:58,543::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915217::DEBUG::2013-11-20 03:52:58,543::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`63d08415-bc7f-4534-9ab8-1b094f96edc0`::Granted request
>Thread-15915217::DEBUG::2013-11-20 03:52:58,545::task::495::TaskManager.Task::(_debug) Task d42d3345-9583-4e28-8fa3-0b8a308aee8b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915217::DEBUG::2013-11-20 03:52:58,546::task::495::TaskManager.Task::(_debug) Task d42d3345-9583-4e28-8fa3-0b8a308aee8b: ref 1 aborting False
>Thread-15915217::WARNING::2013-11-20 03:52:58,549::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915217::DEBUG::2013-11-20 03:52:58,550::task::495::TaskManager.Task::(_debug) Task d42d3345-9583-4e28-8fa3-0b8a308aee8b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915217::DEBUG::2013-11-20 03:52:58,551::task::495::TaskManager.Task::(_debug) Task d42d3345-9583-4e28-8fa3-0b8a308aee8b: moving from state preparing -> state finished
>Thread-15915217::DEBUG::2013-11-20 03:52:58,551::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915217::DEBUG::2013-11-20 03:52:58,552::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915217::DEBUG::2013-11-20 03:52:58,553::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915217::DEBUG::2013-11-20 03:52:58,554::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915217::DEBUG::2013-11-20 03:52:58,554::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915217::DEBUG::2013-11-20 03:52:58,555::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915217::DEBUG::2013-11-20 03:52:58,556::task::495::TaskManager.Task::(_debug) Task d42d3345-9583-4e28-8fa3-0b8a308aee8b: ref 0 aborting False
>Thread-15915217::INFO::2013-11-20 03:52:58,557::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:52:59,724::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:53:00,336::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0926895 s, 11.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:53:02,352::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:53:02,954::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0914108 s, 11.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:53:04,974::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 03:53:05,205::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:53:05,208::task::495::TaskManager.Task::(_debug) Task 4c7f691e-08f0-48e9-8086-6dfa8e83889c: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:53:05,211::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4edf578e-4b16-4590-8291-56917dea9fc9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:53:05,212::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:53:05,214::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:53:05,215::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4edf578e-4b16-4590-8291-56917dea9fc9`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:53:05,218::task::495::TaskManager.Task::(_debug) Task 4c7f691e-08f0-48e9-8086-6dfa8e83889c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:53:05,219::task::495::TaskManager.Task::(_debug) Task 4c7f691e-08f0-48e9-8086-6dfa8e83889c: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:53:05,220::task::495::TaskManager.Task::(_debug) Task 4c7f691e-08f0-48e9-8086-6dfa8e83889c: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:53:05,221::task::495::TaskManager.Task::(_debug) Task 4c7f691e-08f0-48e9-8086-6dfa8e83889c: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:53:05,222::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:53:05,223::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:53:05,223::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:53:05,224::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:53:05,225::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:53:05,225::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:53:05,226::task::495::TaskManager.Task::(_debug) Task 4c7f691e-08f0-48e9-8086-6dfa8e83889c: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:53:05,226::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:53:05,582::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.084168 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:53:07,592::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:53:08,202::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0857638 s, 11.9 MB/s\n'; <rc> = 0
>Thread-15915221::DEBUG::2013-11-20 03:53:08,803::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915221::INFO::2013-11-20 03:53:08,804::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915221::DEBUG::2013-11-20 03:53:08,805::task::495::TaskManager.Task::(_debug) Task 0b711137-748b-4ddd-9070-ac4b4b9a8237: moving from state init -> state preparing
>Thread-15915221::DEBUG::2013-11-20 03:53:08,806::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915221::DEBUG::2013-11-20 03:53:08,807::task::495::TaskManager.Task::(_debug) Task 0b711137-748b-4ddd-9070-ac4b4b9a8237: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915221::DEBUG::2013-11-20 03:53:08,807::task::495::TaskManager.Task::(_debug) Task 0b711137-748b-4ddd-9070-ac4b4b9a8237: moving from state preparing -> state finished
>Thread-15915221::DEBUG::2013-11-20 03:53:08,808::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915221::DEBUG::2013-11-20 03:53:08,809::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915221::DEBUG::2013-11-20 03:53:08,810::task::495::TaskManager.Task::(_debug) Task 0b711137-748b-4ddd-9070-ac4b4b9a8237: ref 0 aborting False
>Thread-15915221::INFO::2013-11-20 03:53:08,810::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915222::DEBUG::2013-11-20 03:53:08,815::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915222::INFO::2013-11-20 03:53:08,817::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915222::DEBUG::2013-11-20 03:53:08,817::task::495::TaskManager.Task::(_debug) Task c104ea49-6b2c-472c-8340-abe0d2e64095: moving from state init -> state preparing
>Thread-15915222::DEBUG::2013-11-20 03:53:08,819::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6d2696a9-c3b6-407c-88eb-59963a2e0506`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915222::DEBUG::2013-11-20 03:53:08,820::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915222::DEBUG::2013-11-20 03:53:08,822::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915222::DEBUG::2013-11-20 03:53:08,822::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6d2696a9-c3b6-407c-88eb-59963a2e0506`::Granted request
>Thread-15915222::DEBUG::2013-11-20 03:53:08,824::task::495::TaskManager.Task::(_debug) Task c104ea49-6b2c-472c-8340-abe0d2e64095: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915222::DEBUG::2013-11-20 03:53:08,824::task::495::TaskManager.Task::(_debug) Task c104ea49-6b2c-472c-8340-abe0d2e64095: ref 1 aborting False
>Thread-15915222::WARNING::2013-11-20 03:53:08,827::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915222::DEBUG::2013-11-20 03:53:08,828::task::495::TaskManager.Task::(_debug) Task c104ea49-6b2c-472c-8340-abe0d2e64095: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915222::DEBUG::2013-11-20 03:53:08,829::task::495::TaskManager.Task::(_debug) Task c104ea49-6b2c-472c-8340-abe0d2e64095: moving from state preparing -> state finished
>Thread-15915222::DEBUG::2013-11-20 03:53:08,830::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915222::DEBUG::2013-11-20 03:53:08,830::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915222::DEBUG::2013-11-20 03:53:08,831::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915222::DEBUG::2013-11-20 03:53:08,832::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915222::DEBUG::2013-11-20 03:53:08,832::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915222::DEBUG::2013-11-20 03:53:08,833::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915222::DEBUG::2013-11-20 03:53:08,834::task::495::TaskManager.Task::(_debug) Task c104ea49-6b2c-472c-8340-abe0d2e64095: ref 0 aborting False
>Thread-15915222::INFO::2013-11-20 03:53:08,834::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:53:10,213::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:53:11,105::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.079774 s, 12.8 MB/s\n'; <rc> = 0
>Thread-15915224::INFO::2013-11-20 03:53:12,036::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915224::DEBUG::2013-11-20 03:53:12,038::task::495::TaskManager.Task::(_debug) Task e3ffb680-664e-4c78-a591-c0bed163678e: moving from state init -> state preparing
>Thread-15915224::DEBUG::2013-11-20 03:53:12,039::task::495::TaskManager.Task::(_debug) Task e3ffb680-664e-4c78-a591-c0bed163678e: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000434875488281', 'lastCheck': 1384919587.937737, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0023238658905', 'lastCheck': 1384919589.954849, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00210189819336', 'lastCheck': 1384919586.940057, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000449180603027', 'lastCheck': 1384919591.938004, 'valid': True, 'code': 0}}
>Thread-15915224::DEBUG::2013-11-20 03:53:12,040::task::495::TaskManager.Task::(_debug) Task e3ffb680-664e-4c78-a591-c0bed163678e: moving from state preparing -> state finished
>Thread-15915224::DEBUG::2013-11-20 03:53:12,040::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915224::DEBUG::2013-11-20 03:53:12,041::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915224::DEBUG::2013-11-20 03:53:12,042::task::495::TaskManager.Task::(_debug) Task e3ffb680-664e-4c78-a591-c0bed163678e: ref 0 aborting False
>Thread-15915224::INFO::2013-11-20 03:53:12,043::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000434875488281', 'lastCheck': 1384919587.937737, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0023238658905', 'lastCheck': 1384919589.954849, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00210189819336', 'lastCheck': 1384919586.940057, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000449180603027', 'lastCheck': 1384919591.938004, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:53:13,121::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:53:13,688::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0533876 s, 19.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:53:15,703::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:53:16,259::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0758492 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 03:53:17,348::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:53:17,349::task::495::TaskManager.Task::(_debug) Task 4db7bb7d-5d99-436d-befe-99de7a88055c: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:53:17,350::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a0657405-7fb8-4441-a353-f5b0c2b71d1a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:53:17,351::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:53:17,352::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:53:17,352::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a0657405-7fb8-4441-a353-f5b0c2b71d1a`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:53:17,353::task::495::TaskManager.Task::(_debug) Task 4db7bb7d-5d99-436d-befe-99de7a88055c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:53:17,354::task::495::TaskManager.Task::(_debug) Task 4db7bb7d-5d99-436d-befe-99de7a88055c: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:53:17,354::task::495::TaskManager.Task::(_debug) Task 4db7bb7d-5d99-436d-befe-99de7a88055c: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:53:17,355::task::495::TaskManager.Task::(_debug) Task 4db7bb7d-5d99-436d-befe-99de7a88055c: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:53:17,356::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:53:17,356::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:53:17,357::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:53:17,357::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:53:17,358::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:53:17,358::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:53:17,359::task::495::TaskManager.Task::(_debug) Task 4db7bb7d-5d99-436d-befe-99de7a88055c: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:53:17,360::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:53:18,275::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:53:18,835::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0806791 s, 12.7 MB/s\n'; <rc> = 0
>Thread-15915228::DEBUG::2013-11-20 03:53:19,132::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915228::INFO::2013-11-20 03:53:19,134::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915228::DEBUG::2013-11-20 03:53:19,135::task::495::TaskManager.Task::(_debug) Task 10134483-6927-415e-a525-83b1ac50c351: moving from state init -> state preparing
>Thread-15915228::DEBUG::2013-11-20 03:53:19,136::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915228::DEBUG::2013-11-20 03:53:19,137::task::495::TaskManager.Task::(_debug) Task 10134483-6927-415e-a525-83b1ac50c351: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915228::DEBUG::2013-11-20 03:53:19,137::task::495::TaskManager.Task::(_debug) Task 10134483-6927-415e-a525-83b1ac50c351: moving from state preparing -> state finished
>Thread-15915228::DEBUG::2013-11-20 03:53:19,138::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915228::DEBUG::2013-11-20 03:53:19,138::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915228::DEBUG::2013-11-20 03:53:19,139::task::495::TaskManager.Task::(_debug) Task 10134483-6927-415e-a525-83b1ac50c351: ref 0 aborting False
>Thread-15915228::INFO::2013-11-20 03:53:19,140::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915229::DEBUG::2013-11-20 03:53:19,145::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915229::INFO::2013-11-20 03:53:19,146::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915229::DEBUG::2013-11-20 03:53:19,147::task::495::TaskManager.Task::(_debug) Task 91df1c6b-bfa3-4495-b749-946e4773bc28: moving from state init -> state preparing
>Thread-15915229::DEBUG::2013-11-20 03:53:19,148::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`58d442f4-5d49-449c-b933-afd7070530a9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915229::DEBUG::2013-11-20 03:53:19,149::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915229::DEBUG::2013-11-20 03:53:19,150::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915229::DEBUG::2013-11-20 03:53:19,150::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`58d442f4-5d49-449c-b933-afd7070530a9`::Granted request
>Thread-15915229::DEBUG::2013-11-20 03:53:19,151::task::495::TaskManager.Task::(_debug) Task 91df1c6b-bfa3-4495-b749-946e4773bc28: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915229::DEBUG::2013-11-20 03:53:19,152::task::495::TaskManager.Task::(_debug) Task 91df1c6b-bfa3-4495-b749-946e4773bc28: ref 1 aborting False
>Thread-15915229::WARNING::2013-11-20 03:53:19,154::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915229::DEBUG::2013-11-20 03:53:19,155::task::495::TaskManager.Task::(_debug) Task 91df1c6b-bfa3-4495-b749-946e4773bc28: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915229::DEBUG::2013-11-20 03:53:19,156::task::495::TaskManager.Task::(_debug) Task 91df1c6b-bfa3-4495-b749-946e4773bc28: moving from state preparing -> state finished
>Thread-15915229::DEBUG::2013-11-20 03:53:19,157::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915229::DEBUG::2013-11-20 03:53:19,158::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915229::DEBUG::2013-11-20 03:53:19,158::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915229::DEBUG::2013-11-20 03:53:19,159::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915229::DEBUG::2013-11-20 03:53:19,160::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915229::DEBUG::2013-11-20 03:53:19,161::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915229::DEBUG::2013-11-20 03:53:19,162::task::495::TaskManager.Task::(_debug) Task 91df1c6b-bfa3-4495-b749-946e4773bc28: ref 0 aborting False
>Thread-15915229::INFO::2013-11-20 03:53:19,162::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:53:20,844::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:53:21,427::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0860286 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:53:23,449::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:53:23,993::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0799443 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:53:26,001::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:53:26,584::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0928496 s, 11.0 MB/s\n'; <rc> = 0
>Thread-15915232::INFO::2013-11-20 03:53:27,660::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915232::DEBUG::2013-11-20 03:53:27,661::task::495::TaskManager.Task::(_debug) Task d1623317-41ef-4324-a56e-f7b342a3f241: moving from state init -> state preparing
>Thread-15915232::DEBUG::2013-11-20 03:53:27,662::task::495::TaskManager.Task::(_debug) Task d1623317-41ef-4324-a56e-f7b342a3f241: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000306129455566', 'lastCheck': 1384919597.9555161, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00151395797729', 'lastCheck': 1384919599.9783151, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00794816017151', 'lastCheck': 1384919607.230624, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000572919845581', 'lastCheck': 1384919602.0549381, 'valid': True, 'code': 0}}
>Thread-15915232::DEBUG::2013-11-20 03:53:27,663::task::495::TaskManager.Task::(_debug) Task d1623317-41ef-4324-a56e-f7b342a3f241: moving from state preparing -> state finished
>Thread-15915232::DEBUG::2013-11-20 03:53:27,664::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915232::DEBUG::2013-11-20 03:53:27,665::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915232::DEBUG::2013-11-20 03:53:27,665::task::495::TaskManager.Task::(_debug) Task d1623317-41ef-4324-a56e-f7b342a3f241: ref 0 aborting False
>Thread-15915232::INFO::2013-11-20 03:53:27,666::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000306129455566', 'lastCheck': 1384919597.9555161, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00151395797729', 'lastCheck': 1384919599.9783151, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00794816017151', 'lastCheck': 1384919607.230624, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000572919845581', 'lastCheck': 1384919602.0549381, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:53:28,601::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:53:29,129::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0745831 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 03:53:29,390::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:53:29,391::task::495::TaskManager.Task::(_debug) Task a8971f52-b433-46f9-a1a5-2caa890271bc: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:53:29,400::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`32b8d0da-8421-4096-9137-d5ff0ccf0cec`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:53:29,401::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:53:29,402::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:53:29,404::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`32b8d0da-8421-4096-9137-d5ff0ccf0cec`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:53:29,406::task::495::TaskManager.Task::(_debug) Task a8971f52-b433-46f9-a1a5-2caa890271bc: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:53:29,406::task::495::TaskManager.Task::(_debug) Task a8971f52-b433-46f9-a1a5-2caa890271bc: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:53:29,407::task::495::TaskManager.Task::(_debug) Task a8971f52-b433-46f9-a1a5-2caa890271bc: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:53:29,408::task::495::TaskManager.Task::(_debug) Task a8971f52-b433-46f9-a1a5-2caa890271bc: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:53:29,409::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:53:29,410::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:53:29,411::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:53:29,411::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:53:29,412::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:53:29,412::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:53:29,413::task::495::TaskManager.Task::(_debug) Task a8971f52-b433-46f9-a1a5-2caa890271bc: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:53:29,414::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15915234::DEBUG::2013-11-20 03:53:29,487::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915234::INFO::2013-11-20 03:53:29,489::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915234::DEBUG::2013-11-20 03:53:29,489::task::495::TaskManager.Task::(_debug) Task ba75529a-9a3d-489c-bd09-e28f2eb17696: moving from state init -> state preparing
>Thread-15915234::DEBUG::2013-11-20 03:53:29,490::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915234::DEBUG::2013-11-20 03:53:29,491::task::495::TaskManager.Task::(_debug) Task ba75529a-9a3d-489c-bd09-e28f2eb17696: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915234::DEBUG::2013-11-20 03:53:29,491::task::495::TaskManager.Task::(_debug) Task ba75529a-9a3d-489c-bd09-e28f2eb17696: moving from state preparing -> state finished
>Thread-15915234::DEBUG::2013-11-20 03:53:29,492::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915234::DEBUG::2013-11-20 03:53:29,493::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915234::DEBUG::2013-11-20 03:53:29,493::task::495::TaskManager.Task::(_debug) Task ba75529a-9a3d-489c-bd09-e28f2eb17696: ref 0 aborting False
>Thread-15915234::INFO::2013-11-20 03:53:29,494::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915235::DEBUG::2013-11-20 03:53:29,498::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915235::INFO::2013-11-20 03:53:29,500::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915235::DEBUG::2013-11-20 03:53:29,501::task::495::TaskManager.Task::(_debug) Task ae7fc2fb-c41f-41c2-a58a-6a40532a3a13: moving from state init -> state preparing
>Thread-15915235::DEBUG::2013-11-20 03:53:29,502::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`68690eb9-949b-4fee-bbd7-3426a3d6d755`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915235::DEBUG::2013-11-20 03:53:29,503::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915235::DEBUG::2013-11-20 03:53:29,503::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915235::DEBUG::2013-11-20 03:53:29,504::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`68690eb9-949b-4fee-bbd7-3426a3d6d755`::Granted request
>Thread-15915235::DEBUG::2013-11-20 03:53:29,505::task::495::TaskManager.Task::(_debug) Task ae7fc2fb-c41f-41c2-a58a-6a40532a3a13: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915235::DEBUG::2013-11-20 03:53:29,507::task::495::TaskManager.Task::(_debug) Task ae7fc2fb-c41f-41c2-a58a-6a40532a3a13: ref 1 aborting False
>Thread-15915235::WARNING::2013-11-20 03:53:29,509::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915235::DEBUG::2013-11-20 03:53:29,510::task::495::TaskManager.Task::(_debug) Task ae7fc2fb-c41f-41c2-a58a-6a40532a3a13: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915235::DEBUG::2013-11-20 03:53:29,511::task::495::TaskManager.Task::(_debug) Task ae7fc2fb-c41f-41c2-a58a-6a40532a3a13: moving from state preparing -> state finished
>Thread-15915235::DEBUG::2013-11-20 03:53:29,512::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915235::DEBUG::2013-11-20 03:53:29,513::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915235::DEBUG::2013-11-20 03:53:29,514::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915235::DEBUG::2013-11-20 03:53:29,515::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915235::DEBUG::2013-11-20 03:53:29,515::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915235::DEBUG::2013-11-20 03:53:29,516::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915235::DEBUG::2013-11-20 03:53:29,517::task::495::TaskManager.Task::(_debug) Task ae7fc2fb-c41f-41c2-a58a-6a40532a3a13: ref 0 aborting False
>Thread-15915235::INFO::2013-11-20 03:53:29,518::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:53:31,145::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 03:53:31,475::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:53:31,476::task::495::TaskManager.Task::(_debug) Task c2b9ec1f-69ba-49c5-89dc-ac64feec55cc: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:53:31,478::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`afe11dcf-3197-40cd-8131-9adfc76f31e8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:53:31,479::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:53:31,480::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:53:31,481::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`afe11dcf-3197-40cd-8131-9adfc76f31e8`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:53:31,482::task::495::TaskManager.Task::(_debug) Task c2b9ec1f-69ba-49c5-89dc-ac64feec55cc: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:53:31,483::task::495::TaskManager.Task::(_debug) Task c2b9ec1f-69ba-49c5-89dc-ac64feec55cc: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:53:31,484::task::495::TaskManager.Task::(_debug) Task c2b9ec1f-69ba-49c5-89dc-ac64feec55cc: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:53:31,484::task::495::TaskManager.Task::(_debug) Task c2b9ec1f-69ba-49c5-89dc-ac64feec55cc: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:53:31,485::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:53:31,486::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:53:31,487::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:53:31,487::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:53:31,488::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:53:31,489::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:53:31,489::task::495::TaskManager.Task::(_debug) Task c2b9ec1f-69ba-49c5-89dc-ac64feec55cc: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:53:31,490::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:53:31,706::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0913064 s, 11.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:53:33,717::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:53:34,293::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0827195 s, 12.4 MB/s\n'; <rc> = 0
>Thread-15915085::INFO::2013-11-20 03:53:35,441::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=089b08fc-6544-4235-a061-fb55b09d2715 volUUID=66b599fc-b7b5-4934-a5d5-ef7283460029)
>Thread-15915085::DEBUG::2013-11-20 03:53:35,442::task::495::TaskManager.Task::(_debug) Task b5ebf1c6-45d4-4104-a9df-302ebf697964: moving from state init -> state preparing
>Thread-15915085::DEBUG::2013-11-20 03:53:35,444::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7c70c7fd-aef8-4a18-a966-bbb66d779c41`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915085::DEBUG::2013-11-20 03:53:35,445::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915085::DEBUG::2013-11-20 03:53:35,446::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915085::DEBUG::2013-11-20 03:53:35,447::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7c70c7fd-aef8-4a18-a966-bbb66d779c41`::Granted request
>Thread-15915085::DEBUG::2013-11-20 03:53:35,449::task::495::TaskManager.Task::(_debug) Task b5ebf1c6-45d4-4104-a9df-302ebf697964: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915085::DEBUG::2013-11-20 03:53:35,450::task::495::TaskManager.Task::(_debug) Task b5ebf1c6-45d4-4104-a9df-302ebf697964: ref 1 aborting False
>Thread-15915085::DEBUG::2013-11-20 03:53:35,451::task::495::TaskManager.Task::(_debug) Task b5ebf1c6-45d4-4104-a9df-302ebf697964: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15915085::DEBUG::2013-11-20 03:53:35,452::task::495::TaskManager.Task::(_debug) Task b5ebf1c6-45d4-4104-a9df-302ebf697964: moving from state preparing -> state finished
>Thread-15915085::DEBUG::2013-11-20 03:53:35,452::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915085::DEBUG::2013-11-20 03:53:35,453::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915085::DEBUG::2013-11-20 03:53:35,454::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915085::DEBUG::2013-11-20 03:53:35,455::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915085::DEBUG::2013-11-20 03:53:35,455::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915085::DEBUG::2013-11-20 03:53:35,456::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915085::DEBUG::2013-11-20 03:53:35,456::task::495::TaskManager.Task::(_debug) Task b5ebf1c6-45d4-4104-a9df-302ebf697964: ref 0 aborting False
>Thread-15915085::INFO::2013-11-20 03:53:35,457::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Dummy-15912651::DEBUG::2013-11-20 03:53:36,308::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:53:36,884::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0850635 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:53:38,901::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 03:53:39,427::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 03:53:39,438::task::495::TaskManager.Task::(_debug) Task c7261765-2e04-4459-9927-754af05e1405: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 03:53:39,441::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c3fa1a3f-300a-4b0e-bc93-5d478a5c90b9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 03:53:39,442::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 03:53:39,444::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 03:53:39,445::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c3fa1a3f-300a-4b0e-bc93-5d478a5c90b9`::Granted request
>Thread-15914453::DEBUG::2013-11-20 03:53:39,465::task::495::TaskManager.Task::(_debug) Task c7261765-2e04-4459-9927-754af05e1405: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 03:53:39,466::task::495::TaskManager.Task::(_debug) Task c7261765-2e04-4459-9927-754af05e1405: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 03:53:39,468::task::495::TaskManager.Task::(_debug) Task c7261765-2e04-4459-9927-754af05e1405: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 03:53:39,469::task::495::TaskManager.Task::(_debug) Task c7261765-2e04-4459-9927-754af05e1405: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 03:53:39,471::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 03:53:39,473::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 03:53:39,474::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 03:53:39,475::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 03:53:39,477::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 03:53:39,478::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 03:53:39,480::task::495::TaskManager.Task::(_debug) Task c7261765-2e04-4459-9927-754af05e1405: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 03:53:39,481::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 03:53:39,514::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0793609 s, 12.9 MB/s\n'; <rc> = 0
>Thread-15915239::DEBUG::2013-11-20 03:53:40,037::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915239::INFO::2013-11-20 03:53:40,040::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915239::DEBUG::2013-11-20 03:53:40,041::task::495::TaskManager.Task::(_debug) Task bef62223-5821-46ed-a4c7-7da18fc150dc: moving from state init -> state preparing
>Thread-15915239::DEBUG::2013-11-20 03:53:40,042::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915239::DEBUG::2013-11-20 03:53:40,043::task::495::TaskManager.Task::(_debug) Task bef62223-5821-46ed-a4c7-7da18fc150dc: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915239::DEBUG::2013-11-20 03:53:40,043::task::495::TaskManager.Task::(_debug) Task bef62223-5821-46ed-a4c7-7da18fc150dc: moving from state preparing -> state finished
>Thread-15915239::DEBUG::2013-11-20 03:53:40,044::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915239::DEBUG::2013-11-20 03:53:40,045::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915239::DEBUG::2013-11-20 03:53:40,045::task::495::TaskManager.Task::(_debug) Task bef62223-5821-46ed-a4c7-7da18fc150dc: ref 0 aborting False
>Thread-15915239::INFO::2013-11-20 03:53:40,046::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915240::DEBUG::2013-11-20 03:53:40,051::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915240::INFO::2013-11-20 03:53:40,052::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915240::DEBUG::2013-11-20 03:53:40,053::task::495::TaskManager.Task::(_debug) Task 2431c84e-ec97-4ccd-9954-aa4332feee44: moving from state init -> state preparing
>Thread-15915240::DEBUG::2013-11-20 03:53:40,054::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`19f73f82-4dd1-4b25-bfd6-45cf2d82991c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915240::DEBUG::2013-11-20 03:53:40,055::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915240::DEBUG::2013-11-20 03:53:40,056::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915240::DEBUG::2013-11-20 03:53:40,056::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`19f73f82-4dd1-4b25-bfd6-45cf2d82991c`::Granted request
>Thread-15915240::DEBUG::2013-11-20 03:53:40,057::task::495::TaskManager.Task::(_debug) Task 2431c84e-ec97-4ccd-9954-aa4332feee44: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915240::DEBUG::2013-11-20 03:53:40,058::task::495::TaskManager.Task::(_debug) Task 2431c84e-ec97-4ccd-9954-aa4332feee44: ref 1 aborting False
>Thread-15915240::WARNING::2013-11-20 03:53:40,060::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915240::DEBUG::2013-11-20 03:53:40,061::task::495::TaskManager.Task::(_debug) Task 2431c84e-ec97-4ccd-9954-aa4332feee44: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915240::DEBUG::2013-11-20 03:53:40,062::task::495::TaskManager.Task::(_debug) Task 2431c84e-ec97-4ccd-9954-aa4332feee44: moving from state preparing -> state finished
>Thread-15915240::DEBUG::2013-11-20 03:53:40,063::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915240::DEBUG::2013-11-20 03:53:40,063::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915240::DEBUG::2013-11-20 03:53:40,064::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915240::DEBUG::2013-11-20 03:53:40,065::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915240::DEBUG::2013-11-20 03:53:40,066::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915240::DEBUG::2013-11-20 03:53:40,066::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915240::DEBUG::2013-11-20 03:53:40,067::task::495::TaskManager.Task::(_debug) Task 2431c84e-ec97-4ccd-9954-aa4332feee44: ref 0 aborting False
>Thread-15915240::INFO::2013-11-20 03:53:40,068::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914435::INFO::2013-11-20 03:53:40,313::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 03:53:40,314::task::495::TaskManager.Task::(_debug) Task d19a3e16-3345-4f3e-9767-ad1ed9eeaf42: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 03:53:40,316::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`731370a1-149a-42d3-84b6-2ce551952ce9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 03:53:40,316::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 03:53:40,317::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 03:53:40,318::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`731370a1-149a-42d3-84b6-2ce551952ce9`::Granted request
>Thread-15914435::DEBUG::2013-11-20 03:53:40,319::task::495::TaskManager.Task::(_debug) Task d19a3e16-3345-4f3e-9767-ad1ed9eeaf42: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 03:53:40,319::task::495::TaskManager.Task::(_debug) Task d19a3e16-3345-4f3e-9767-ad1ed9eeaf42: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 03:53:40,320::task::495::TaskManager.Task::(_debug) Task d19a3e16-3345-4f3e-9767-ad1ed9eeaf42: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 03:53:40,321::task::495::TaskManager.Task::(_debug) Task d19a3e16-3345-4f3e-9767-ad1ed9eeaf42: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 03:53:40,322::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 03:53:40,322::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 03:53:40,323::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 03:53:40,324::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 03:53:40,324::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 03:53:40,325::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 03:53:40,325::task::495::TaskManager.Task::(_debug) Task d19a3e16-3345-4f3e-9767-ad1ed9eeaf42: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 03:53:40,326::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 03:53:41,528::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:53:42,107::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.089746 s, 11.4 MB/s\n'; <rc> = 0
>Thread-15915242::INFO::2013-11-20 03:53:43,736::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915242::DEBUG::2013-11-20 03:53:43,737::task::495::TaskManager.Task::(_debug) Task 6e441fea-2ee8-40cf-9a9e-e616569ce5da: moving from state init -> state preparing
>Thread-15915242::DEBUG::2013-11-20 03:53:43,738::task::495::TaskManager.Task::(_debug) Task 6e441fea-2ee8-40cf-9a9e-e616569ce5da: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000313997268677', 'lastCheck': 1384919618.549165, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0020809173584', 'lastCheck': 1384919620.5501699, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00136685371399', 'lastCheck': 1384919617.545826, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000424146652222', 'lastCheck': 1384919622.7547081, 'valid': True, 'code': 0}}
>Thread-15915242::DEBUG::2013-11-20 03:53:43,739::task::495::TaskManager.Task::(_debug) Task 6e441fea-2ee8-40cf-9a9e-e616569ce5da: moving from state preparing -> state finished
>Thread-15915242::DEBUG::2013-11-20 03:53:43,740::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915242::DEBUG::2013-11-20 03:53:43,740::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915242::DEBUG::2013-11-20 03:53:43,741::task::495::TaskManager.Task::(_debug) Task 6e441fea-2ee8-40cf-9a9e-e616569ce5da: ref 0 aborting False
>Thread-15915242::INFO::2013-11-20 03:53:43,741::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000313997268677', 'lastCheck': 1384919618.549165, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0020809173584', 'lastCheck': 1384919620.5501699, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00136685371399', 'lastCheck': 1384919617.545826, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000424146652222', 'lastCheck': 1384919622.7547081, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:53:44,123::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:53:44,743::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0863105 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:53:46,759::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 03:53:46,761::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 03:53:46,762::task::495::TaskManager.Task::(_debug) Task 38448bf8-8dc1-43ac-8f17-09d753e9a3de: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 03:53:46,994::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8a96ff1d-9bab-4d08-a19b-c231f39e1b36`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 03:53:46,996::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 03:53:46,997::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 03:53:46,998::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8a96ff1d-9bab-4d08-a19b-c231f39e1b36`::Granted request
>Thread-15914519::DEBUG::2013-11-20 03:53:47,000::task::495::TaskManager.Task::(_debug) Task 38448bf8-8dc1-43ac-8f17-09d753e9a3de: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 03:53:47,001::task::495::TaskManager.Task::(_debug) Task 38448bf8-8dc1-43ac-8f17-09d753e9a3de: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 03:53:47,002::task::495::TaskManager.Task::(_debug) Task 38448bf8-8dc1-43ac-8f17-09d753e9a3de: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 03:53:47,002::task::495::TaskManager.Task::(_debug) Task 38448bf8-8dc1-43ac-8f17-09d753e9a3de: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 03:53:47,003::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 03:53:47,004::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 03:53:47,005::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 03:53:47,005::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 03:53:47,006::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 03:53:47,007::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 03:53:47,007::task::495::TaskManager.Task::(_debug) Task 38448bf8-8dc1-43ac-8f17-09d753e9a3de: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 03:53:47,008::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 03:53:47,330::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0599405 s, 17.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:53:49,344::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 03:53:49,560::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:53:49,567::task::495::TaskManager.Task::(_debug) Task 9938943d-ac56-4d80-8940-539b63cbcadc: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:53:49,570::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fec19c38-6e0a-4827-a6f8-f1cddc76913b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:53:49,571::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:53:49,574::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:53:49,574::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fec19c38-6e0a-4827-a6f8-f1cddc76913b`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:53:49,576::task::495::TaskManager.Task::(_debug) Task 9938943d-ac56-4d80-8940-539b63cbcadc: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:53:49,577::task::495::TaskManager.Task::(_debug) Task 9938943d-ac56-4d80-8940-539b63cbcadc: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:53:49,577::task::495::TaskManager.Task::(_debug) Task 9938943d-ac56-4d80-8940-539b63cbcadc: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:53:49,578::task::495::TaskManager.Task::(_debug) Task 9938943d-ac56-4d80-8940-539b63cbcadc: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:53:49,579::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:53:49,580::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:53:49,581::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:53:49,582::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:53:49,582::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:53:49,583::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:53:49,584::task::495::TaskManager.Task::(_debug) Task 9938943d-ac56-4d80-8940-539b63cbcadc: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:53:49,584::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:53:49,585::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:53:49,586::task::495::TaskManager.Task::(_debug) Task f117e50a-6803-4c82-a903-c60078eafae9: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:53:49,587::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f3531642-2e28-4cc2-8609-6cae8859faa1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:53:49,588::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:53:49,589::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:53:49,589::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f3531642-2e28-4cc2-8609-6cae8859faa1`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:53:49,591::task::495::TaskManager.Task::(_debug) Task f117e50a-6803-4c82-a903-c60078eafae9: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:53:49,592::task::495::TaskManager.Task::(_debug) Task f117e50a-6803-4c82-a903-c60078eafae9: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:53:49,592::task::495::TaskManager.Task::(_debug) Task f117e50a-6803-4c82-a903-c60078eafae9: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:53:49,593::task::495::TaskManager.Task::(_debug) Task f117e50a-6803-4c82-a903-c60078eafae9: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:53:49,593::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:53:49,594::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:53:49,595::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:53:49,595::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:53:49,596::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:53:49,596::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:53:49,597::task::495::TaskManager.Task::(_debug) Task f117e50a-6803-4c82-a903-c60078eafae9: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:53:49,598::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:53:49,599::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:53:49,599::task::495::TaskManager.Task::(_debug) Task 8754bca5-30d4-4fca-9915-209f92cda5b2: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:53:49,600::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f6049e5d-a317-49c2-a6e2-2ab4b3620e64`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:53:49,601::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:53:49,601::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:53:49,602::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f6049e5d-a317-49c2-a6e2-2ab4b3620e64`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:53:49,603::task::495::TaskManager.Task::(_debug) Task 8754bca5-30d4-4fca-9915-209f92cda5b2: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:53:49,604::task::495::TaskManager.Task::(_debug) Task 8754bca5-30d4-4fca-9915-209f92cda5b2: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:53:49,604::task::495::TaskManager.Task::(_debug) Task 8754bca5-30d4-4fca-9915-209f92cda5b2: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:53:49,605::task::495::TaskManager.Task::(_debug) Task 8754bca5-30d4-4fca-9915-209f92cda5b2: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:53:49,605::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:53:49,606::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:53:49,607::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:53:49,607::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:53:49,608::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:53:49,608::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:53:49,609::task::495::TaskManager.Task::(_debug) Task 8754bca5-30d4-4fca-9915-209f92cda5b2: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:53:49,610::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:53:49,611::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:53:49,611::task::495::TaskManager.Task::(_debug) Task b972ea92-e0d1-434f-9bdf-08f3f10c400e: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:53:49,612::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`ff0ab6ef-5534-4f3a-aa76-43b9bd481a79`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:53:49,613::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:53:49,614::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:53:49,614::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`ff0ab6ef-5534-4f3a-aa76-43b9bd481a79`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:53:49,615::task::495::TaskManager.Task::(_debug) Task b972ea92-e0d1-434f-9bdf-08f3f10c400e: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:53:49,616::task::495::TaskManager.Task::(_debug) Task b972ea92-e0d1-434f-9bdf-08f3f10c400e: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:53:49,617::task::495::TaskManager.Task::(_debug) Task b972ea92-e0d1-434f-9bdf-08f3f10c400e: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:53:49,617::task::495::TaskManager.Task::(_debug) Task b972ea92-e0d1-434f-9bdf-08f3f10c400e: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:53:49,618::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:53:49,618::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:53:49,619::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:53:49,620::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:53:49,620::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:53:49,621::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:53:49,622::task::495::TaskManager.Task::(_debug) Task b972ea92-e0d1-434f-9bdf-08f3f10c400e: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:53:49,622::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:53:49,899::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0920882 s, 11.1 MB/s\n'; <rc> = 0
>Thread-15915246::DEBUG::2013-11-20 03:53:50,541::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915246::INFO::2013-11-20 03:53:50,542::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915246::DEBUG::2013-11-20 03:53:50,542::task::495::TaskManager.Task::(_debug) Task f448bf4f-17d5-404b-88c6-6d390dcda6d2: moving from state init -> state preparing
>Thread-15915246::DEBUG::2013-11-20 03:53:50,543::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915246::DEBUG::2013-11-20 03:53:50,543::task::495::TaskManager.Task::(_debug) Task f448bf4f-17d5-404b-88c6-6d390dcda6d2: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915246::DEBUG::2013-11-20 03:53:50,543::task::495::TaskManager.Task::(_debug) Task f448bf4f-17d5-404b-88c6-6d390dcda6d2: moving from state preparing -> state finished
>Thread-15915246::DEBUG::2013-11-20 03:53:50,544::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915246::DEBUG::2013-11-20 03:53:50,544::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915246::DEBUG::2013-11-20 03:53:50,545::task::495::TaskManager.Task::(_debug) Task f448bf4f-17d5-404b-88c6-6d390dcda6d2: ref 0 aborting False
>Thread-15915246::INFO::2013-11-20 03:53:50,545::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915247::DEBUG::2013-11-20 03:53:50,550::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915247::INFO::2013-11-20 03:53:50,550::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915247::DEBUG::2013-11-20 03:53:50,551::task::495::TaskManager.Task::(_debug) Task b1065594-eeda-41c2-a2d5-d4a881647e4a: moving from state init -> state preparing
>Thread-15915247::DEBUG::2013-11-20 03:53:50,552::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a0279784-9122-4823-a1f3-b282d8448932`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915247::DEBUG::2013-11-20 03:53:50,553::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915247::DEBUG::2013-11-20 03:53:50,554::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915247::DEBUG::2013-11-20 03:53:50,555::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a0279784-9122-4823-a1f3-b282d8448932`::Granted request
>Thread-15915247::DEBUG::2013-11-20 03:53:50,557::task::495::TaskManager.Task::(_debug) Task b1065594-eeda-41c2-a2d5-d4a881647e4a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915247::DEBUG::2013-11-20 03:53:50,557::task::495::TaskManager.Task::(_debug) Task b1065594-eeda-41c2-a2d5-d4a881647e4a: ref 1 aborting False
>Thread-15915247::WARNING::2013-11-20 03:53:50,562::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915247::DEBUG::2013-11-20 03:53:50,563::task::495::TaskManager.Task::(_debug) Task b1065594-eeda-41c2-a2d5-d4a881647e4a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915247::DEBUG::2013-11-20 03:53:50,564::task::495::TaskManager.Task::(_debug) Task b1065594-eeda-41c2-a2d5-d4a881647e4a: moving from state preparing -> state finished
>Thread-15915247::DEBUG::2013-11-20 03:53:50,564::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915247::DEBUG::2013-11-20 03:53:50,565::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915247::DEBUG::2013-11-20 03:53:50,567::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915247::DEBUG::2013-11-20 03:53:50,568::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915247::DEBUG::2013-11-20 03:53:50,569::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915247::DEBUG::2013-11-20 03:53:50,569::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915247::DEBUG::2013-11-20 03:53:50,570::task::495::TaskManager.Task::(_debug) Task b1065594-eeda-41c2-a2d5-d4a881647e4a: ref 0 aborting False
>Thread-15915247::INFO::2013-11-20 03:53:50,571::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:53:51,917::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:53:52,374::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0673502 s, 15.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:53:54,389::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:53:54,959::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0858023 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:53:56,971::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:53:57,652::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0860348 s, 11.9 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 03:53:58,849::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:53:58,850::task::495::TaskManager.Task::(_debug) Task 579a5d6d-3335-4810-9e6b-c5f14e1f27c6: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:53:58,852::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a1b85984-40d0-427c-9fd1-a76729a6a817`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:53:58,853::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:53:58,854::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:53:58,854::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a1b85984-40d0-427c-9fd1-a76729a6a817`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:53:58,856::task::495::TaskManager.Task::(_debug) Task 579a5d6d-3335-4810-9e6b-c5f14e1f27c6: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:53:58,856::task::495::TaskManager.Task::(_debug) Task 579a5d6d-3335-4810-9e6b-c5f14e1f27c6: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:53:58,857::task::495::TaskManager.Task::(_debug) Task 579a5d6d-3335-4810-9e6b-c5f14e1f27c6: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:53:58,858::task::495::TaskManager.Task::(_debug) Task 579a5d6d-3335-4810-9e6b-c5f14e1f27c6: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:53:58,858::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:53:58,859::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:53:58,860::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:53:58,861::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:53:58,861::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:53:58,862::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:53:58,862::task::495::TaskManager.Task::(_debug) Task 579a5d6d-3335-4810-9e6b-c5f14e1f27c6: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:53:58,863::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:53:58,864::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:53:58,865::task::495::TaskManager.Task::(_debug) Task 50805936-5acc-4a4d-b462-f8126305ca90: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:53:58,866::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e3bbc97a-8e21-43a9-ad15-738f276a5eec`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:53:58,866::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:53:58,867::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:53:58,868::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e3bbc97a-8e21-43a9-ad15-738f276a5eec`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:53:58,869::task::495::TaskManager.Task::(_debug) Task 50805936-5acc-4a4d-b462-f8126305ca90: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:53:58,869::task::495::TaskManager.Task::(_debug) Task 50805936-5acc-4a4d-b462-f8126305ca90: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:53:58,870::task::495::TaskManager.Task::(_debug) Task 50805936-5acc-4a4d-b462-f8126305ca90: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:53:58,871::task::495::TaskManager.Task::(_debug) Task 50805936-5acc-4a4d-b462-f8126305ca90: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:53:58,871::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:53:58,872::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:53:58,873::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:53:58,873::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:53:58,874::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:53:58,874::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:53:58,875::task::495::TaskManager.Task::(_debug) Task 50805936-5acc-4a4d-b462-f8126305ca90: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:53:58,876::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15915250::INFO::2013-11-20 03:53:59,463::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915250::DEBUG::2013-11-20 03:53:59,464::task::495::TaskManager.Task::(_debug) Task e9320e0f-e919-4379-b333-be6ae146935f: moving from state init -> state preparing
>Thread-15915250::DEBUG::2013-11-20 03:53:59,465::task::495::TaskManager.Task::(_debug) Task e9320e0f-e919-4379-b333-be6ae146935f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000358104705811', 'lastCheck': 1384919638.7744551, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00152015686035', 'lastCheck': 1384919630.768626, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.001140832901', 'lastCheck': 1384919637.779753, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00492191314697', 'lastCheck': 1384919632.7701621, 'valid': True, 'code': 0}}
>Thread-15915250::DEBUG::2013-11-20 03:53:59,466::task::495::TaskManager.Task::(_debug) Task e9320e0f-e919-4379-b333-be6ae146935f: moving from state preparing -> state finished
>Thread-15915250::DEBUG::2013-11-20 03:53:59,467::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915250::DEBUG::2013-11-20 03:53:59,468::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915250::DEBUG::2013-11-20 03:53:59,468::task::495::TaskManager.Task::(_debug) Task e9320e0f-e919-4379-b333-be6ae146935f: ref 0 aborting False
>Thread-15915250::INFO::2013-11-20 03:53:59,469::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000358104705811', 'lastCheck': 1384919638.7744551, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00152015686035', 'lastCheck': 1384919630.768626, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.001140832901', 'lastCheck': 1384919637.779753, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00492191314697', 'lastCheck': 1384919632.7701621, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:53:59,669::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:54:00,279::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0789745 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15915252::DEBUG::2013-11-20 03:54:00,856::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915252::INFO::2013-11-20 03:54:00,857::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915252::DEBUG::2013-11-20 03:54:00,858::task::495::TaskManager.Task::(_debug) Task 6f00df1c-3e5d-4771-89f4-da092ff849e4: moving from state init -> state preparing
>Thread-15915252::DEBUG::2013-11-20 03:54:00,859::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915252::DEBUG::2013-11-20 03:54:00,860::task::495::TaskManager.Task::(_debug) Task 6f00df1c-3e5d-4771-89f4-da092ff849e4: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915252::DEBUG::2013-11-20 03:54:00,861::task::495::TaskManager.Task::(_debug) Task 6f00df1c-3e5d-4771-89f4-da092ff849e4: moving from state preparing -> state finished
>Thread-15915252::DEBUG::2013-11-20 03:54:00,862::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915252::DEBUG::2013-11-20 03:54:00,862::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915252::DEBUG::2013-11-20 03:54:00,863::task::495::TaskManager.Task::(_debug) Task 6f00df1c-3e5d-4771-89f4-da092ff849e4: ref 0 aborting False
>Thread-15915252::INFO::2013-11-20 03:54:00,864::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915253::DEBUG::2013-11-20 03:54:00,910::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915253::INFO::2013-11-20 03:54:00,911::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915253::DEBUG::2013-11-20 03:54:00,912::task::495::TaskManager.Task::(_debug) Task ba736283-a539-470f-9290-25913d474a9d: moving from state init -> state preparing
>Thread-15915253::DEBUG::2013-11-20 03:54:00,913::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`01c3eb42-1f8c-4176-acac-22a9d036bd18`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915253::DEBUG::2013-11-20 03:54:00,914::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915253::DEBUG::2013-11-20 03:54:00,915::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915253::DEBUG::2013-11-20 03:54:00,915::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`01c3eb42-1f8c-4176-acac-22a9d036bd18`::Granted request
>Thread-15915253::DEBUG::2013-11-20 03:54:00,916::task::495::TaskManager.Task::(_debug) Task ba736283-a539-470f-9290-25913d474a9d: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915253::DEBUG::2013-11-20 03:54:00,917::task::495::TaskManager.Task::(_debug) Task ba736283-a539-470f-9290-25913d474a9d: ref 1 aborting False
>Thread-15915253::WARNING::2013-11-20 03:54:00,918::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915253::DEBUG::2013-11-20 03:54:00,919::task::495::TaskManager.Task::(_debug) Task ba736283-a539-470f-9290-25913d474a9d: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915253::DEBUG::2013-11-20 03:54:00,920::task::495::TaskManager.Task::(_debug) Task ba736283-a539-470f-9290-25913d474a9d: moving from state preparing -> state finished
>Thread-15915253::DEBUG::2013-11-20 03:54:00,921::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915253::DEBUG::2013-11-20 03:54:00,921::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915253::DEBUG::2013-11-20 03:54:00,923::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915253::DEBUG::2013-11-20 03:54:00,923::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915253::DEBUG::2013-11-20 03:54:00,924::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915253::DEBUG::2013-11-20 03:54:00,925::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915253::DEBUG::2013-11-20 03:54:00,925::task::495::TaskManager.Task::(_debug) Task ba736283-a539-470f-9290-25913d474a9d: ref 0 aborting False
>Thread-15915253::INFO::2013-11-20 03:54:00,926::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:54:02,297::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:54:02,874::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0832569 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:54:04,896::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:54:05,455::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0760619 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15913639::INFO::2013-11-20 03:54:05,578::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:54:05,579::task::495::TaskManager.Task::(_debug) Task f742b039-ee44-4d86-b639-60459467a3a0: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:54:05,581::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9641dc3c-ed9c-4612-bdf0-a74bb4adf9d0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:54:05,582::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:54:05,583::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:54:05,584::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9641dc3c-ed9c-4612-bdf0-a74bb4adf9d0`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:54:05,585::task::495::TaskManager.Task::(_debug) Task f742b039-ee44-4d86-b639-60459467a3a0: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:54:05,586::task::495::TaskManager.Task::(_debug) Task f742b039-ee44-4d86-b639-60459467a3a0: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:54:05,587::task::495::TaskManager.Task::(_debug) Task f742b039-ee44-4d86-b639-60459467a3a0: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:54:05,587::task::495::TaskManager.Task::(_debug) Task f742b039-ee44-4d86-b639-60459467a3a0: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:54:05,588::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:54:05,589::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:54:05,589::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:54:05,590::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:54:05,591::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:54:05,591::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:54:05,592::task::495::TaskManager.Task::(_debug) Task f742b039-ee44-4d86-b639-60459467a3a0: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:54:05,592::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:54:07,475::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:54:08,082::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.084015 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:54:10,091::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:54:10,661::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0835174 s, 12.3 MB/s\n'; <rc> = 0
>Thread-15915257::DEBUG::2013-11-20 03:54:11,250::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915257::INFO::2013-11-20 03:54:11,252::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915257::DEBUG::2013-11-20 03:54:11,252::task::495::TaskManager.Task::(_debug) Task 48f928d3-de61-4b52-a2d9-fa35f673cde9: moving from state init -> state preparing
>Thread-15915257::DEBUG::2013-11-20 03:54:11,253::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915257::DEBUG::2013-11-20 03:54:11,254::task::495::TaskManager.Task::(_debug) Task 48f928d3-de61-4b52-a2d9-fa35f673cde9: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915257::DEBUG::2013-11-20 03:54:11,255::task::495::TaskManager.Task::(_debug) Task 48f928d3-de61-4b52-a2d9-fa35f673cde9: moving from state preparing -> state finished
>Thread-15915257::DEBUG::2013-11-20 03:54:11,255::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915257::DEBUG::2013-11-20 03:54:11,256::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915257::DEBUG::2013-11-20 03:54:11,256::task::495::TaskManager.Task::(_debug) Task 48f928d3-de61-4b52-a2d9-fa35f673cde9: ref 0 aborting False
>Thread-15915257::INFO::2013-11-20 03:54:11,257::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915258::DEBUG::2013-11-20 03:54:11,262::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915258::INFO::2013-11-20 03:54:11,263::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915258::DEBUG::2013-11-20 03:54:11,264::task::495::TaskManager.Task::(_debug) Task 5fbc12aa-401b-48c1-a032-365680a01a52: moving from state init -> state preparing
>Thread-15915258::DEBUG::2013-11-20 03:54:11,265::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`dd881645-7b4b-47bf-bb0e-609ac209bce0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915258::DEBUG::2013-11-20 03:54:11,266::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915258::DEBUG::2013-11-20 03:54:11,267::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915258::DEBUG::2013-11-20 03:54:11,268::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`dd881645-7b4b-47bf-bb0e-609ac209bce0`::Granted request
>Thread-15915258::DEBUG::2013-11-20 03:54:11,269::task::495::TaskManager.Task::(_debug) Task 5fbc12aa-401b-48c1-a032-365680a01a52: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915258::DEBUG::2013-11-20 03:54:11,269::task::495::TaskManager.Task::(_debug) Task 5fbc12aa-401b-48c1-a032-365680a01a52: ref 1 aborting False
>Thread-15915258::WARNING::2013-11-20 03:54:11,271::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915258::DEBUG::2013-11-20 03:54:11,272::task::495::TaskManager.Task::(_debug) Task 5fbc12aa-401b-48c1-a032-365680a01a52: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915258::DEBUG::2013-11-20 03:54:11,273::task::495::TaskManager.Task::(_debug) Task 5fbc12aa-401b-48c1-a032-365680a01a52: moving from state preparing -> state finished
>Thread-15915258::DEBUG::2013-11-20 03:54:11,274::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915258::DEBUG::2013-11-20 03:54:11,274::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915258::DEBUG::2013-11-20 03:54:11,275::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915258::DEBUG::2013-11-20 03:54:11,276::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915258::DEBUG::2013-11-20 03:54:11,276::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915258::DEBUG::2013-11-20 03:54:11,277::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915258::DEBUG::2013-11-20 03:54:11,277::task::495::TaskManager.Task::(_debug) Task 5fbc12aa-401b-48c1-a032-365680a01a52: ref 0 aborting False
>Thread-15915258::INFO::2013-11-20 03:54:11,278::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:54:12,677::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:54:13,242::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0774294 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15915260::INFO::2013-11-20 03:54:15,004::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915260::DEBUG::2013-11-20 03:54:15,005::task::495::TaskManager.Task::(_debug) Task 2a77a3e0-5c4e-45b2-8182-f2e7ff0ee48d: moving from state init -> state preparing
>Thread-15915260::DEBUG::2013-11-20 03:54:15,006::task::495::TaskManager.Task::(_debug) Task 2a77a3e0-5c4e-45b2-8182-f2e7ff0ee48d: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000379800796509', 'lastCheck': 1384919648.895082, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014910697937', 'lastCheck': 1384919650.9071951, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00144600868225', 'lastCheck': 1384919647.8983309, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000440120697021', 'lastCheck': 1384919652.905514, 'valid': True, 'code': 0}}
>Thread-15915260::DEBUG::2013-11-20 03:54:15,007::task::495::TaskManager.Task::(_debug) Task 2a77a3e0-5c4e-45b2-8182-f2e7ff0ee48d: moving from state preparing -> state finished
>Thread-15915260::DEBUG::2013-11-20 03:54:15,008::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915260::DEBUG::2013-11-20 03:54:15,009::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915260::DEBUG::2013-11-20 03:54:15,009::task::495::TaskManager.Task::(_debug) Task 2a77a3e0-5c4e-45b2-8182-f2e7ff0ee48d: ref 0 aborting False
>Thread-15915260::INFO::2013-11-20 03:54:15,010::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000379800796509', 'lastCheck': 1384919648.895082, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014910697937', 'lastCheck': 1384919650.9071951, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00144600868225', 'lastCheck': 1384919647.8983309, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000440120697021', 'lastCheck': 1384919652.905514, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:54:15,262::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:54:15,870::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.104265 s, 9.8 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 03:54:17,887::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:54:17,888::task::495::TaskManager.Task::(_debug) Task f94ef0b6-a948-403a-945b-1a0c76dbeb01: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:54:17,890::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c84334cd-b09f-406e-9137-50b1da3794a6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:54:17,891::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Dummy-15912651::DEBUG::2013-11-20 03:54:17,892::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::DEBUG::2013-11-20 03:54:17,893::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:54:17,894::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c84334cd-b09f-406e-9137-50b1da3794a6`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:54:18,132::task::495::TaskManager.Task::(_debug) Task f94ef0b6-a948-403a-945b-1a0c76dbeb01: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:54:18,133::task::495::TaskManager.Task::(_debug) Task f94ef0b6-a948-403a-945b-1a0c76dbeb01: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:54:18,135::task::495::TaskManager.Task::(_debug) Task f94ef0b6-a948-403a-945b-1a0c76dbeb01: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:54:18,135::task::495::TaskManager.Task::(_debug) Task f94ef0b6-a948-403a-945b-1a0c76dbeb01: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:54:18,136::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:54:18,136::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:54:18,136::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:54:18,137::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:54:18,137::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:54:18,138::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:54:18,138::task::495::TaskManager.Task::(_debug) Task f94ef0b6-a948-403a-945b-1a0c76dbeb01: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:54:18,139::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:54:18,445::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0769476 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:54:20,460::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:54:21,021::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.105428 s, 9.7 MB/s\n'; <rc> = 0
>Thread-15915264::DEBUG::2013-11-20 03:54:21,650::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915264::INFO::2013-11-20 03:54:21,652::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915264::DEBUG::2013-11-20 03:54:21,652::task::495::TaskManager.Task::(_debug) Task ebbff863-b9e6-486a-b512-1e0553ee97bc: moving from state init -> state preparing
>Thread-15915264::DEBUG::2013-11-20 03:54:21,653::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915264::DEBUG::2013-11-20 03:54:21,654::task::495::TaskManager.Task::(_debug) Task ebbff863-b9e6-486a-b512-1e0553ee97bc: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915264::DEBUG::2013-11-20 03:54:21,654::task::495::TaskManager.Task::(_debug) Task ebbff863-b9e6-486a-b512-1e0553ee97bc: moving from state preparing -> state finished
>Thread-15915264::DEBUG::2013-11-20 03:54:21,655::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915264::DEBUG::2013-11-20 03:54:21,655::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915264::DEBUG::2013-11-20 03:54:21,656::task::495::TaskManager.Task::(_debug) Task ebbff863-b9e6-486a-b512-1e0553ee97bc: ref 0 aborting False
>Thread-15915264::INFO::2013-11-20 03:54:21,656::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915265::DEBUG::2013-11-20 03:54:21,662::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915265::INFO::2013-11-20 03:54:21,663::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915265::DEBUG::2013-11-20 03:54:21,663::task::495::TaskManager.Task::(_debug) Task c1c8276f-3709-4c6f-8e31-4c1c58af79cc: moving from state init -> state preparing
>Thread-15915265::DEBUG::2013-11-20 03:54:21,665::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1dd4b9f5-7096-42f1-8c44-19d468c5ab52`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915265::DEBUG::2013-11-20 03:54:21,665::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915265::DEBUG::2013-11-20 03:54:21,666::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915265::DEBUG::2013-11-20 03:54:21,667::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1dd4b9f5-7096-42f1-8c44-19d468c5ab52`::Granted request
>Thread-15915265::DEBUG::2013-11-20 03:54:21,668::task::495::TaskManager.Task::(_debug) Task c1c8276f-3709-4c6f-8e31-4c1c58af79cc: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915265::DEBUG::2013-11-20 03:54:21,669::task::495::TaskManager.Task::(_debug) Task c1c8276f-3709-4c6f-8e31-4c1c58af79cc: ref 1 aborting False
>Thread-15915265::WARNING::2013-11-20 03:54:21,671::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915265::DEBUG::2013-11-20 03:54:21,672::task::495::TaskManager.Task::(_debug) Task c1c8276f-3709-4c6f-8e31-4c1c58af79cc: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915265::DEBUG::2013-11-20 03:54:21,673::task::495::TaskManager.Task::(_debug) Task c1c8276f-3709-4c6f-8e31-4c1c58af79cc: moving from state preparing -> state finished
>Thread-15915265::DEBUG::2013-11-20 03:54:21,674::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915265::DEBUG::2013-11-20 03:54:21,675::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915265::DEBUG::2013-11-20 03:54:21,676::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915265::DEBUG::2013-11-20 03:54:21,676::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915265::DEBUG::2013-11-20 03:54:21,677::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915265::DEBUG::2013-11-20 03:54:21,678::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915265::DEBUG::2013-11-20 03:54:21,678::task::495::TaskManager.Task::(_debug) Task c1c8276f-3709-4c6f-8e31-4c1c58af79cc: ref 0 aborting False
>Thread-15915265::INFO::2013-11-20 03:54:21,679::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:54:23,035::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:54:23,586::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0839857 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:54:25,596::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:54:26,233::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0791357 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:54:28,248::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:54:28,859::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0884497 s, 11.6 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 03:54:29,864::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:54:29,865::task::495::TaskManager.Task::(_debug) Task b1210023-1bff-4af0-90f9-393a3665cad8: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:54:29,873::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8af022ea-7386-4e61-8083-513b473a45b8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:54:29,874::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:54:29,875::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:54:29,875::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8af022ea-7386-4e61-8083-513b473a45b8`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:54:29,876::task::495::TaskManager.Task::(_debug) Task b1210023-1bff-4af0-90f9-393a3665cad8: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:54:29,877::task::495::TaskManager.Task::(_debug) Task b1210023-1bff-4af0-90f9-393a3665cad8: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:54:29,878::task::495::TaskManager.Task::(_debug) Task b1210023-1bff-4af0-90f9-393a3665cad8: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:54:29,878::task::495::TaskManager.Task::(_debug) Task b1210023-1bff-4af0-90f9-393a3665cad8: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:54:29,879::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:54:29,879::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:54:29,880::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:54:29,881::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:54:29,881::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:54:29,881::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:54:29,882::task::495::TaskManager.Task::(_debug) Task b1210023-1bff-4af0-90f9-393a3665cad8: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:54:29,883::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15915268::INFO::2013-11-20 03:54:30,720::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915268::DEBUG::2013-11-20 03:54:30,721::task::495::TaskManager.Task::(_debug) Task 3cd1b05e-4be3-456a-afbe-73da6db13ac6: moving from state init -> state preparing
>Thread-15915268::DEBUG::2013-11-20 03:54:30,722::task::495::TaskManager.Task::(_debug) Task 3cd1b05e-4be3-456a-afbe-73da6db13ac6: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000427961349487', 'lastCheck': 1384919669.250447, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0161330699921', 'lastCheck': 1384919661.1516981, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00154185295105', 'lastCheck': 1384919668.603678, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00163006782532', 'lastCheck': 1384919663.2465279, 'valid': True, 'code': 0}}
>Thread-15915268::DEBUG::2013-11-20 03:54:30,723::task::495::TaskManager.Task::(_debug) Task 3cd1b05e-4be3-456a-afbe-73da6db13ac6: moving from state preparing -> state finished
>Thread-15915268::DEBUG::2013-11-20 03:54:30,724::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915268::DEBUG::2013-11-20 03:54:30,725::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915268::DEBUG::2013-11-20 03:54:30,726::task::495::TaskManager.Task::(_debug) Task 3cd1b05e-4be3-456a-afbe-73da6db13ac6: ref 0 aborting False
>Thread-15915268::INFO::2013-11-20 03:54:30,727::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000427961349487', 'lastCheck': 1384919669.250447, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0161330699921', 'lastCheck': 1384919661.1516981, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00154185295105', 'lastCheck': 1384919668.603678, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00163006782532', 'lastCheck': 1384919663.2465279, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:54:30,873::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:54:31,445::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0756372 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15915270::DEBUG::2013-11-20 03:54:32,168::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915270::INFO::2013-11-20 03:54:32,169::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915270::DEBUG::2013-11-20 03:54:32,170::task::495::TaskManager.Task::(_debug) Task 72f359d2-d163-4692-a552-17a094d50aa9: moving from state init -> state preparing
>Thread-15915270::DEBUG::2013-11-20 03:54:32,171::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915270::DEBUG::2013-11-20 03:54:32,172::task::495::TaskManager.Task::(_debug) Task 72f359d2-d163-4692-a552-17a094d50aa9: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915270::DEBUG::2013-11-20 03:54:32,173::task::495::TaskManager.Task::(_debug) Task 72f359d2-d163-4692-a552-17a094d50aa9: moving from state preparing -> state finished
>Thread-15915270::DEBUG::2013-11-20 03:54:32,173::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915270::DEBUG::2013-11-20 03:54:32,174::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915270::DEBUG::2013-11-20 03:54:32,175::task::495::TaskManager.Task::(_debug) Task 72f359d2-d163-4692-a552-17a094d50aa9: ref 0 aborting False
>Thread-15915270::INFO::2013-11-20 03:54:32,175::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915271::DEBUG::2013-11-20 03:54:32,180::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915271::INFO::2013-11-20 03:54:32,181::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915271::DEBUG::2013-11-20 03:54:32,182::task::495::TaskManager.Task::(_debug) Task 350d8f11-5e80-4879-9e36-4fb975be84d2: moving from state init -> state preparing
>Thread-15915271::DEBUG::2013-11-20 03:54:32,183::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`706c9be4-d875-4a50-91ec-d1653fb48106`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915271::DEBUG::2013-11-20 03:54:32,184::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915271::DEBUG::2013-11-20 03:54:32,185::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915271::DEBUG::2013-11-20 03:54:32,186::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`706c9be4-d875-4a50-91ec-d1653fb48106`::Granted request
>Thread-15915271::DEBUG::2013-11-20 03:54:32,187::task::495::TaskManager.Task::(_debug) Task 350d8f11-5e80-4879-9e36-4fb975be84d2: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915271::DEBUG::2013-11-20 03:54:32,188::task::495::TaskManager.Task::(_debug) Task 350d8f11-5e80-4879-9e36-4fb975be84d2: ref 1 aborting False
>Thread-15915271::WARNING::2013-11-20 03:54:32,190::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915271::DEBUG::2013-11-20 03:54:32,191::task::495::TaskManager.Task::(_debug) Task 350d8f11-5e80-4879-9e36-4fb975be84d2: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915271::DEBUG::2013-11-20 03:54:32,192::task::495::TaskManager.Task::(_debug) Task 350d8f11-5e80-4879-9e36-4fb975be84d2: moving from state preparing -> state finished
>Thread-15915271::DEBUG::2013-11-20 03:54:32,193::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915271::DEBUG::2013-11-20 03:54:32,193::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915271::DEBUG::2013-11-20 03:54:32,194::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915271::DEBUG::2013-11-20 03:54:32,195::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915271::DEBUG::2013-11-20 03:54:32,195::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915271::DEBUG::2013-11-20 03:54:32,196::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915271::DEBUG::2013-11-20 03:54:32,197::task::495::TaskManager.Task::(_debug) Task 350d8f11-5e80-4879-9e36-4fb975be84d2: ref 0 aborting False
>Thread-15915271::INFO::2013-11-20 03:54:32,197::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913695::INFO::2013-11-20 03:54:32,759::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:54:32,761::task::495::TaskManager.Task::(_debug) Task 4380bb6c-9a32-4c20-8ae1-64af2462d430: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:54:32,762::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7cf3afd3-4c4f-4aa1-aa0c-4175593c43d0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:54:32,763::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:54:32,764::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:54:32,765::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7cf3afd3-4c4f-4aa1-aa0c-4175593c43d0`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:54:32,766::task::495::TaskManager.Task::(_debug) Task 4380bb6c-9a32-4c20-8ae1-64af2462d430: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:54:32,767::task::495::TaskManager.Task::(_debug) Task 4380bb6c-9a32-4c20-8ae1-64af2462d430: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:54:32,768::task::495::TaskManager.Task::(_debug) Task 4380bb6c-9a32-4c20-8ae1-64af2462d430: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:54:32,768::task::495::TaskManager.Task::(_debug) Task 4380bb6c-9a32-4c20-8ae1-64af2462d430: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:54:32,769::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:54:32,770::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:54:32,771::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:54:32,771::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:54:32,772::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:54:32,772::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:54:32,773::task::495::TaskManager.Task::(_debug) Task 4380bb6c-9a32-4c20-8ae1-64af2462d430: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:54:32,774::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:54:33,458::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:54:34,000::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0858593 s, 11.9 MB/s\n'; <rc> = 0
>Thread-15915085::INFO::2013-11-20 03:54:35,878::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=089b08fc-6544-4235-a061-fb55b09d2715 volUUID=66b599fc-b7b5-4934-a5d5-ef7283460029)
>Thread-15915085::DEBUG::2013-11-20 03:54:35,879::task::495::TaskManager.Task::(_debug) Task 1ce6d604-bb1a-4302-b9bb-9b56052626bb: moving from state init -> state preparing
>Thread-15915085::DEBUG::2013-11-20 03:54:35,881::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`58f9a1a1-0de4-4117-88ff-42472bba1cab`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915085::DEBUG::2013-11-20 03:54:35,882::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915085::DEBUG::2013-11-20 03:54:35,883::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915085::DEBUG::2013-11-20 03:54:35,884::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`58f9a1a1-0de4-4117-88ff-42472bba1cab`::Granted request
>Thread-15915085::DEBUG::2013-11-20 03:54:35,885::task::495::TaskManager.Task::(_debug) Task 1ce6d604-bb1a-4302-b9bb-9b56052626bb: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915085::DEBUG::2013-11-20 03:54:35,886::task::495::TaskManager.Task::(_debug) Task 1ce6d604-bb1a-4302-b9bb-9b56052626bb: ref 1 aborting False
>Thread-15915085::DEBUG::2013-11-20 03:54:35,887::task::495::TaskManager.Task::(_debug) Task 1ce6d604-bb1a-4302-b9bb-9b56052626bb: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15915085::DEBUG::2013-11-20 03:54:35,887::task::495::TaskManager.Task::(_debug) Task 1ce6d604-bb1a-4302-b9bb-9b56052626bb: moving from state preparing -> state finished
>Thread-15915085::DEBUG::2013-11-20 03:54:35,888::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915085::DEBUG::2013-11-20 03:54:35,888::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915085::DEBUG::2013-11-20 03:54:35,889::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915085::DEBUG::2013-11-20 03:54:35,890::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915085::DEBUG::2013-11-20 03:54:35,890::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915085::DEBUG::2013-11-20 03:54:35,891::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915085::DEBUG::2013-11-20 03:54:35,892::task::495::TaskManager.Task::(_debug) Task 1ce6d604-bb1a-4302-b9bb-9b56052626bb: ref 0 aborting False
>Thread-15915085::INFO::2013-11-20 03:54:35,892::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Dummy-15912651::DEBUG::2013-11-20 03:54:36,012::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:54:36,622::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0764623 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:54:38,638::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:54:39,211::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0993149 s, 10.3 MB/s\n'; <rc> = 0
>Thread-15914453::INFO::2013-11-20 03:54:39,904::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 03:54:39,905::task::495::TaskManager.Task::(_debug) Task 5e99da08-3144-4dbc-a7bd-16b636c4c56a: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 03:54:39,915::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b73fa6e9-5f1f-4d84-8ea5-8d91e0a1ef96`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 03:54:39,916::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 03:54:39,917::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 03:54:39,918::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b73fa6e9-5f1f-4d84-8ea5-8d91e0a1ef96`::Granted request
>Thread-15914453::DEBUG::2013-11-20 03:54:39,919::task::495::TaskManager.Task::(_debug) Task 5e99da08-3144-4dbc-a7bd-16b636c4c56a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 03:54:39,920::task::495::TaskManager.Task::(_debug) Task 5e99da08-3144-4dbc-a7bd-16b636c4c56a: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 03:54:39,921::task::495::TaskManager.Task::(_debug) Task 5e99da08-3144-4dbc-a7bd-16b636c4c56a: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 03:54:39,922::task::495::TaskManager.Task::(_debug) Task 5e99da08-3144-4dbc-a7bd-16b636c4c56a: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 03:54:39,922::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 03:54:39,923::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 03:54:39,924::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 03:54:39,925::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 03:54:39,925::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 03:54:39,926::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 03:54:39,927::task::495::TaskManager.Task::(_debug) Task 5e99da08-3144-4dbc-a7bd-16b636c4c56a: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 03:54:39,928::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 03:54:41,230::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 03:54:41,488::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 03:54:41,490::task::495::TaskManager.Task::(_debug) Task 7a994ccc-18c3-4031-9a24-2cfbbc6f9109: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 03:54:41,493::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`88cec876-81aa-4245-9895-a4e1e67d4b84`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 03:54:41,493::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 03:54:41,495::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 03:54:41,495::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`88cec876-81aa-4245-9895-a4e1e67d4b84`::Granted request
>Thread-15914435::DEBUG::2013-11-20 03:54:41,497::task::495::TaskManager.Task::(_debug) Task 7a994ccc-18c3-4031-9a24-2cfbbc6f9109: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 03:54:41,498::task::495::TaskManager.Task::(_debug) Task 7a994ccc-18c3-4031-9a24-2cfbbc6f9109: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 03:54:41,499::task::495::TaskManager.Task::(_debug) Task 7a994ccc-18c3-4031-9a24-2cfbbc6f9109: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 03:54:41,499::task::495::TaskManager.Task::(_debug) Task 7a994ccc-18c3-4031-9a24-2cfbbc6f9109: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 03:54:41,500::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 03:54:41,501::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 03:54:41,501::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 03:54:41,502::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 03:54:41,503::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 03:54:41,503::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 03:54:41,504::task::495::TaskManager.Task::(_debug) Task 7a994ccc-18c3-4031-9a24-2cfbbc6f9109: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 03:54:41,504::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 03:54:41,819::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0796912 s, 12.8 MB/s\n'; <rc> = 0
>Thread-15915275::DEBUG::2013-11-20 03:54:42,546::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915275::INFO::2013-11-20 03:54:42,548::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915275::DEBUG::2013-11-20 03:54:42,549::task::495::TaskManager.Task::(_debug) Task efb1faf8-1134-4b8b-9443-ebf860da19c6: moving from state init -> state preparing
>Thread-15915275::DEBUG::2013-11-20 03:54:42,549::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915275::DEBUG::2013-11-20 03:54:42,550::task::495::TaskManager.Task::(_debug) Task efb1faf8-1134-4b8b-9443-ebf860da19c6: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915275::DEBUG::2013-11-20 03:54:42,551::task::495::TaskManager.Task::(_debug) Task efb1faf8-1134-4b8b-9443-ebf860da19c6: moving from state preparing -> state finished
>Thread-15915275::DEBUG::2013-11-20 03:54:42,551::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915275::DEBUG::2013-11-20 03:54:42,552::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915275::DEBUG::2013-11-20 03:54:42,552::task::495::TaskManager.Task::(_debug) Task efb1faf8-1134-4b8b-9443-ebf860da19c6: ref 0 aborting False
>Thread-15915275::INFO::2013-11-20 03:54:42,553::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915276::DEBUG::2013-11-20 03:54:42,558::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915276::INFO::2013-11-20 03:54:42,559::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915276::DEBUG::2013-11-20 03:54:42,560::task::495::TaskManager.Task::(_debug) Task 8e1a2dd6-f202-4adf-a5fa-ce9b6e2e32e7: moving from state init -> state preparing
>Thread-15915276::DEBUG::2013-11-20 03:54:42,561::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5a95ff29-470a-4821-90e7-8b91d8646602`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915276::DEBUG::2013-11-20 03:54:42,562::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915276::DEBUG::2013-11-20 03:54:42,563::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915276::DEBUG::2013-11-20 03:54:42,563::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5a95ff29-470a-4821-90e7-8b91d8646602`::Granted request
>Thread-15915276::DEBUG::2013-11-20 03:54:42,565::task::495::TaskManager.Task::(_debug) Task 8e1a2dd6-f202-4adf-a5fa-ce9b6e2e32e7: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915276::DEBUG::2013-11-20 03:54:42,566::task::495::TaskManager.Task::(_debug) Task 8e1a2dd6-f202-4adf-a5fa-ce9b6e2e32e7: ref 1 aborting False
>Thread-15915276::WARNING::2013-11-20 03:54:42,568::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915276::DEBUG::2013-11-20 03:54:42,569::task::495::TaskManager.Task::(_debug) Task 8e1a2dd6-f202-4adf-a5fa-ce9b6e2e32e7: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915276::DEBUG::2013-11-20 03:54:42,570::task::495::TaskManager.Task::(_debug) Task 8e1a2dd6-f202-4adf-a5fa-ce9b6e2e32e7: moving from state preparing -> state finished
>Thread-15915276::DEBUG::2013-11-20 03:54:42,571::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915276::DEBUG::2013-11-20 03:54:42,571::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915276::DEBUG::2013-11-20 03:54:42,572::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915276::DEBUG::2013-11-20 03:54:42,573::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915276::DEBUG::2013-11-20 03:54:42,574::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915276::DEBUG::2013-11-20 03:54:42,574::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915276::DEBUG::2013-11-20 03:54:42,575::task::495::TaskManager.Task::(_debug) Task 8e1a2dd6-f202-4adf-a5fa-ce9b6e2e32e7: ref 0 aborting False
>Thread-15915276::INFO::2013-11-20 03:54:42,575::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:54:43,834::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:54:44,372::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0699609 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:54:46,387::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915278::INFO::2013-11-20 03:54:46,710::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915278::DEBUG::2013-11-20 03:54:46,712::task::495::TaskManager.Task::(_debug) Task 4b7ea3af-477f-40e2-a5d4-c93f33bcd687: moving from state init -> state preparing
>Thread-15915278::DEBUG::2013-11-20 03:54:46,713::task::495::TaskManager.Task::(_debug) Task 4b7ea3af-477f-40e2-a5d4-c93f33bcd687: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00046181678772', 'lastCheck': 1384919679.274982, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00190401077271', 'lastCheck': 1384919681.4838941, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00264120101929', 'lastCheck': 1384919678.865921, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00352811813354', 'lastCheck': 1384919684.084795, 'valid': True, 'code': 0}}
>Thread-15915278::DEBUG::2013-11-20 03:54:46,714::task::495::TaskManager.Task::(_debug) Task 4b7ea3af-477f-40e2-a5d4-c93f33bcd687: moving from state preparing -> state finished
>Thread-15915278::DEBUG::2013-11-20 03:54:46,714::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915278::DEBUG::2013-11-20 03:54:46,715::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915278::DEBUG::2013-11-20 03:54:46,716::task::495::TaskManager.Task::(_debug) Task 4b7ea3af-477f-40e2-a5d4-c93f33bcd687: ref 0 aborting False
>Thread-15915278::INFO::2013-11-20 03:54:46,716::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00046181678772', 'lastCheck': 1384919679.274982, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00190401077271', 'lastCheck': 1384919681.4838941, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00264120101929', 'lastCheck': 1384919678.865921, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00352811813354', 'lastCheck': 1384919684.084795, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:54:46,987::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.080648 s, 12.7 MB/s\n'; <rc> = 0
>Thread-15914519::INFO::2013-11-20 03:54:47,699::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 03:54:47,701::task::495::TaskManager.Task::(_debug) Task e3ac620b-eff5-4915-8638-826c918dde4a: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 03:54:47,704::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0ba0537c-844f-4beb-b10a-eb92023dedf3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 03:54:47,705::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 03:54:47,707::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 03:54:47,707::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0ba0537c-844f-4beb-b10a-eb92023dedf3`::Granted request
>Thread-15914519::DEBUG::2013-11-20 03:54:47,719::task::495::TaskManager.Task::(_debug) Task e3ac620b-eff5-4915-8638-826c918dde4a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 03:54:47,720::task::495::TaskManager.Task::(_debug) Task e3ac620b-eff5-4915-8638-826c918dde4a: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 03:54:47,721::task::495::TaskManager.Task::(_debug) Task e3ac620b-eff5-4915-8638-826c918dde4a: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 03:54:47,721::task::495::TaskManager.Task::(_debug) Task e3ac620b-eff5-4915-8638-826c918dde4a: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 03:54:47,722::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 03:54:47,723::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 03:54:47,723::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 03:54:47,724::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 03:54:47,725::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 03:54:47,725::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 03:54:47,726::task::495::TaskManager.Task::(_debug) Task e3ac620b-eff5-4915-8638-826c918dde4a: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 03:54:47,727::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 03:54:49,003::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:54:49,579::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0642914 s, 15.9 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 03:54:50,261::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:54:50,263::task::495::TaskManager.Task::(_debug) Task 3f8af248-f444-4572-9723-5993930c111f: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:54:50,265::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`5874ab43-53ff-46da-9471-bc53cb2df579`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:54:50,265::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:54:50,266::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:54:50,267::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`5874ab43-53ff-46da-9471-bc53cb2df579`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:54:50,268::task::495::TaskManager.Task::(_debug) Task 3f8af248-f444-4572-9723-5993930c111f: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:54:50,269::task::495::TaskManager.Task::(_debug) Task 3f8af248-f444-4572-9723-5993930c111f: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:54:50,270::task::495::TaskManager.Task::(_debug) Task 3f8af248-f444-4572-9723-5993930c111f: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:54:50,271::task::495::TaskManager.Task::(_debug) Task 3f8af248-f444-4572-9723-5993930c111f: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:54:50,272::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:54:50,273::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:54:50,273::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:54:50,274::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:54:50,275::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:54:50,276::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:54:50,277::task::495::TaskManager.Task::(_debug) Task 3f8af248-f444-4572-9723-5993930c111f: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:54:50,277::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:54:50,278::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:54:50,279::task::495::TaskManager.Task::(_debug) Task 53956edf-7771-426b-8aca-bcda2a474a4d: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:54:50,281::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`639103e6-c025-4fb4-9887-39487b055c95`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:54:50,281::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:54:50,282::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:54:50,283::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`639103e6-c025-4fb4-9887-39487b055c95`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:54:50,284::task::495::TaskManager.Task::(_debug) Task 53956edf-7771-426b-8aca-bcda2a474a4d: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:54:50,285::task::495::TaskManager.Task::(_debug) Task 53956edf-7771-426b-8aca-bcda2a474a4d: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:54:50,286::task::495::TaskManager.Task::(_debug) Task 53956edf-7771-426b-8aca-bcda2a474a4d: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:54:50,286::task::495::TaskManager.Task::(_debug) Task 53956edf-7771-426b-8aca-bcda2a474a4d: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:54:50,287::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:54:50,287::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:54:50,288::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:54:50,289::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:54:50,289::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:54:50,290::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:54:50,291::task::495::TaskManager.Task::(_debug) Task 53956edf-7771-426b-8aca-bcda2a474a4d: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:54:50,291::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:54:50,292::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:54:50,293::task::495::TaskManager.Task::(_debug) Task 1572e562-fa4b-46dc-a558-5e6b758b0c2b: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:54:50,294::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6fc47554-072d-4e03-9ed8-52a86872dfc7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:54:50,295::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:54:50,295::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:54:50,296::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6fc47554-072d-4e03-9ed8-52a86872dfc7`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:54:50,297::task::495::TaskManager.Task::(_debug) Task 1572e562-fa4b-46dc-a558-5e6b758b0c2b: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:54:50,298::task::495::TaskManager.Task::(_debug) Task 1572e562-fa4b-46dc-a558-5e6b758b0c2b: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:54:50,298::task::495::TaskManager.Task::(_debug) Task 1572e562-fa4b-46dc-a558-5e6b758b0c2b: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:54:50,299::task::495::TaskManager.Task::(_debug) Task 1572e562-fa4b-46dc-a558-5e6b758b0c2b: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:54:50,299::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:54:50,300::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:54:50,301::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:54:50,301::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:54:50,302::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:54:50,302::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:54:50,303::task::495::TaskManager.Task::(_debug) Task 1572e562-fa4b-46dc-a558-5e6b758b0c2b: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:54:50,304::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:54:50,305::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:54:50,305::task::495::TaskManager.Task::(_debug) Task fa3bcc0d-199f-4d20-9329-c527bcb30f79: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:54:50,307::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`4dcd5ae2-f948-4e7c-9e2c-3ef47a41e0ff`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:54:50,307::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:54:50,308::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:54:50,309::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`4dcd5ae2-f948-4e7c-9e2c-3ef47a41e0ff`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:54:50,316::task::495::TaskManager.Task::(_debug) Task fa3bcc0d-199f-4d20-9329-c527bcb30f79: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:54:50,316::task::495::TaskManager.Task::(_debug) Task fa3bcc0d-199f-4d20-9329-c527bcb30f79: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:54:50,317::task::495::TaskManager.Task::(_debug) Task fa3bcc0d-199f-4d20-9329-c527bcb30f79: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:54:50,318::task::495::TaskManager.Task::(_debug) Task fa3bcc0d-199f-4d20-9329-c527bcb30f79: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:54:50,318::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:54:50,319::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:54:50,320::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:54:50,320::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:54:50,321::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:54:50,321::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:54:50,322::task::495::TaskManager.Task::(_debug) Task fa3bcc0d-199f-4d20-9329-c527bcb30f79: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:54:50,322::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:54:51,592::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:54:52,167::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0816067 s, 12.5 MB/s\n'; <rc> = 0
>Thread-15915281::DEBUG::2013-11-20 03:54:52,934::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915281::INFO::2013-11-20 03:54:52,936::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915281::DEBUG::2013-11-20 03:54:52,937::task::495::TaskManager.Task::(_debug) Task afd2d5d2-8fe9-47a9-a61b-f27ee73789b7: moving from state init -> state preparing
>Thread-15915281::DEBUG::2013-11-20 03:54:52,938::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915281::DEBUG::2013-11-20 03:54:52,938::task::495::TaskManager.Task::(_debug) Task afd2d5d2-8fe9-47a9-a61b-f27ee73789b7: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915281::DEBUG::2013-11-20 03:54:52,939::task::495::TaskManager.Task::(_debug) Task afd2d5d2-8fe9-47a9-a61b-f27ee73789b7: moving from state preparing -> state finished
>Thread-15915281::DEBUG::2013-11-20 03:54:52,940::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915281::DEBUG::2013-11-20 03:54:52,940::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915281::DEBUG::2013-11-20 03:54:52,941::task::495::TaskManager.Task::(_debug) Task afd2d5d2-8fe9-47a9-a61b-f27ee73789b7: ref 0 aborting False
>Thread-15915281::INFO::2013-11-20 03:54:52,942::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915282::DEBUG::2013-11-20 03:54:52,947::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915282::INFO::2013-11-20 03:54:52,948::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915282::DEBUG::2013-11-20 03:54:52,949::task::495::TaskManager.Task::(_debug) Task a0b02df1-97e3-42f1-999a-6396f5d12c9b: moving from state init -> state preparing
>Thread-15915282::DEBUG::2013-11-20 03:54:52,950::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8ce1d27d-ec4a-43d8-94fb-d087ec4fe14e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915282::DEBUG::2013-11-20 03:54:52,951::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915282::DEBUG::2013-11-20 03:54:52,953::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915282::DEBUG::2013-11-20 03:54:52,954::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8ce1d27d-ec4a-43d8-94fb-d087ec4fe14e`::Granted request
>Thread-15915282::DEBUG::2013-11-20 03:54:52,956::task::495::TaskManager.Task::(_debug) Task a0b02df1-97e3-42f1-999a-6396f5d12c9b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915282::DEBUG::2013-11-20 03:54:52,956::task::495::TaskManager.Task::(_debug) Task a0b02df1-97e3-42f1-999a-6396f5d12c9b: ref 1 aborting False
>Thread-15915282::WARNING::2013-11-20 03:54:52,959::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915282::DEBUG::2013-11-20 03:54:52,960::task::495::TaskManager.Task::(_debug) Task a0b02df1-97e3-42f1-999a-6396f5d12c9b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915282::DEBUG::2013-11-20 03:54:52,961::task::495::TaskManager.Task::(_debug) Task a0b02df1-97e3-42f1-999a-6396f5d12c9b: moving from state preparing -> state finished
>Thread-15915282::DEBUG::2013-11-20 03:54:52,962::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915282::DEBUG::2013-11-20 03:54:52,962::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915282::DEBUG::2013-11-20 03:54:52,963::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915282::DEBUG::2013-11-20 03:54:52,964::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915282::DEBUG::2013-11-20 03:54:52,964::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915282::DEBUG::2013-11-20 03:54:52,965::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915282::DEBUG::2013-11-20 03:54:52,966::task::495::TaskManager.Task::(_debug) Task a0b02df1-97e3-42f1-999a-6396f5d12c9b: ref 0 aborting False
>Thread-15915282::INFO::2013-11-20 03:54:52,966::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:54:54,185::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:54:54,721::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.082941 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:54:56,734::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:54:57,347::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0775926 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:54:59,363::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:54:59,881::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0815238 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 03:55:00,498::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:55:00,499::task::495::TaskManager.Task::(_debug) Task 4fc832e8-a9be-49ec-80ee-d960609fb846: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:55:00,501::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fb4c82bc-6909-4515-8b04-fdd7fbf060c3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:55:00,501::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:55:00,502::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:55:00,503::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fb4c82bc-6909-4515-8b04-fdd7fbf060c3`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:55:00,504::task::495::TaskManager.Task::(_debug) Task 4fc832e8-a9be-49ec-80ee-d960609fb846: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:55:00,505::task::495::TaskManager.Task::(_debug) Task 4fc832e8-a9be-49ec-80ee-d960609fb846: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:55:00,506::task::495::TaskManager.Task::(_debug) Task 4fc832e8-a9be-49ec-80ee-d960609fb846: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:55:00,507::task::495::TaskManager.Task::(_debug) Task 4fc832e8-a9be-49ec-80ee-d960609fb846: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:55:00,508::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:55:00,508::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:55:00,509::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:55:00,510::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:55:00,510::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:55:00,511::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:55:00,512::task::495::TaskManager.Task::(_debug) Task 4fc832e8-a9be-49ec-80ee-d960609fb846: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:55:00,513::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:55:00,514::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:55:00,514::task::495::TaskManager.Task::(_debug) Task d704d3b7-1ed5-4184-82b8-68f9c8159b8e: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:55:00,515::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d6965b85-ad44-444e-9bb7-897f86f836a2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:55:00,516::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:55:00,517::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:55:00,517::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d6965b85-ad44-444e-9bb7-897f86f836a2`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:55:00,518::task::495::TaskManager.Task::(_debug) Task d704d3b7-1ed5-4184-82b8-68f9c8159b8e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:55:00,519::task::495::TaskManager.Task::(_debug) Task d704d3b7-1ed5-4184-82b8-68f9c8159b8e: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:55:00,520::task::495::TaskManager.Task::(_debug) Task d704d3b7-1ed5-4184-82b8-68f9c8159b8e: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:55:00,520::task::495::TaskManager.Task::(_debug) Task d704d3b7-1ed5-4184-82b8-68f9c8159b8e: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:55:00,521::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:55:00,522::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:55:00,523::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:55:00,523::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:55:00,524::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:55:00,524::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:55:00,525::task::495::TaskManager.Task::(_debug) Task d704d3b7-1ed5-4184-82b8-68f9c8159b8e: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:55:00,526::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:55:01,895::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:55:02,389::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0556428 s, 18.4 MB/s\n'; <rc> = 0
>Thread-15915286::INFO::2013-11-20 03:55:03,111::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915286::DEBUG::2013-11-20 03:55:03,112::task::495::TaskManager.Task::(_debug) Task cb231118-9017-4838-a5c5-7b45132c0658: moving from state init -> state preparing
>Thread-15915286::DEBUG::2013-11-20 03:55:03,113::task::495::TaskManager.Task::(_debug) Task cb231118-9017-4838-a5c5-7b45132c0658: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000233173370361', 'lastCheck': 1384919699.960973, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00139307975769', 'lastCheck': 1384919702.134346, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00185298919678', 'lastCheck': 1384919699.5843229, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00109696388245', 'lastCheck': 1384919694.4188681, 'valid': True, 'code': 0}}
>Thread-15915286::DEBUG::2013-11-20 03:55:03,114::task::495::TaskManager.Task::(_debug) Task cb231118-9017-4838-a5c5-7b45132c0658: moving from state preparing -> state finished
>Thread-15915286::DEBUG::2013-11-20 03:55:03,114::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915286::DEBUG::2013-11-20 03:55:03,115::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915286::DEBUG::2013-11-20 03:55:03,116::task::495::TaskManager.Task::(_debug) Task cb231118-9017-4838-a5c5-7b45132c0658: ref 0 aborting False
>Thread-15915286::INFO::2013-11-20 03:55:03,116::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000233173370361', 'lastCheck': 1384919699.960973, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00139307975769', 'lastCheck': 1384919702.134346, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00185298919678', 'lastCheck': 1384919699.5843229, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00109696388245', 'lastCheck': 1384919694.4188681, 'valid': True, 'code': 0}}
>Thread-15915288::DEBUG::2013-11-20 03:55:03,577::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915288::INFO::2013-11-20 03:55:03,579::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915288::DEBUG::2013-11-20 03:55:03,579::task::495::TaskManager.Task::(_debug) Task 8b538da9-9f3b-47d7-a069-cf7713b85295: moving from state init -> state preparing
>Thread-15915288::DEBUG::2013-11-20 03:55:03,580::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915288::DEBUG::2013-11-20 03:55:03,581::task::495::TaskManager.Task::(_debug) Task 8b538da9-9f3b-47d7-a069-cf7713b85295: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915288::DEBUG::2013-11-20 03:55:03,582::task::495::TaskManager.Task::(_debug) Task 8b538da9-9f3b-47d7-a069-cf7713b85295: moving from state preparing -> state finished
>Thread-15915288::DEBUG::2013-11-20 03:55:03,582::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915288::DEBUG::2013-11-20 03:55:03,583::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915288::DEBUG::2013-11-20 03:55:03,583::task::495::TaskManager.Task::(_debug) Task 8b538da9-9f3b-47d7-a069-cf7713b85295: ref 0 aborting False
>Thread-15915288::INFO::2013-11-20 03:55:03,584::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915289::DEBUG::2013-11-20 03:55:03,589::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915289::INFO::2013-11-20 03:55:03,590::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915289::DEBUG::2013-11-20 03:55:03,591::task::495::TaskManager.Task::(_debug) Task 16965ac0-cbbb-4a52-97cd-173668e77836: moving from state init -> state preparing
>Thread-15915289::DEBUG::2013-11-20 03:55:03,593::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1560e9d7-7f95-4575-beda-8e1773f69f6f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915289::DEBUG::2013-11-20 03:55:03,593::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915289::DEBUG::2013-11-20 03:55:03,594::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915289::DEBUG::2013-11-20 03:55:03,595::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1560e9d7-7f95-4575-beda-8e1773f69f6f`::Granted request
>Thread-15915289::DEBUG::2013-11-20 03:55:03,597::task::495::TaskManager.Task::(_debug) Task 16965ac0-cbbb-4a52-97cd-173668e77836: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915289::DEBUG::2013-11-20 03:55:03,597::task::495::TaskManager.Task::(_debug) Task 16965ac0-cbbb-4a52-97cd-173668e77836: ref 1 aborting False
>Thread-15915289::WARNING::2013-11-20 03:55:03,600::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915289::DEBUG::2013-11-20 03:55:03,601::task::495::TaskManager.Task::(_debug) Task 16965ac0-cbbb-4a52-97cd-173668e77836: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915289::DEBUG::2013-11-20 03:55:03,602::task::495::TaskManager.Task::(_debug) Task 16965ac0-cbbb-4a52-97cd-173668e77836: moving from state preparing -> state finished
>Thread-15915289::DEBUG::2013-11-20 03:55:03,603::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915289::DEBUG::2013-11-20 03:55:03,604::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915289::DEBUG::2013-11-20 03:55:03,604::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915289::DEBUG::2013-11-20 03:55:03,605::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915289::DEBUG::2013-11-20 03:55:03,606::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915289::DEBUG::2013-11-20 03:55:03,606::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915289::DEBUG::2013-11-20 03:55:03,607::task::495::TaskManager.Task::(_debug) Task 16965ac0-cbbb-4a52-97cd-173668e77836: ref 0 aborting False
>Thread-15915289::INFO::2013-11-20 03:55:03,608::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:55:04,400::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-42840::DEBUG::2013-11-20 03:55:04,622::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:55:05,119::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0946657 s, 10.8 MB/s\n'; <rc> = 0
>Thread-42840::DEBUG::2013-11-20 03:55:05,431::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913639::INFO::2013-11-20 03:55:06,128::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:55:06,129::task::495::TaskManager.Task::(_debug) Task 5e66c697-76cb-456a-83c8-240c2f3290f6: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:55:06,132::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`64fec767-becc-4db1-8951-48512e780cc7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:55:06,133::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:55:06,134::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:55:06,135::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`64fec767-becc-4db1-8951-48512e780cc7`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:55:06,136::task::495::TaskManager.Task::(_debug) Task 5e66c697-76cb-456a-83c8-240c2f3290f6: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:55:06,137::task::495::TaskManager.Task::(_debug) Task 5e66c697-76cb-456a-83c8-240c2f3290f6: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:55:06,138::task::495::TaskManager.Task::(_debug) Task 5e66c697-76cb-456a-83c8-240c2f3290f6: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:55:06,139::task::495::TaskManager.Task::(_debug) Task 5e66c697-76cb-456a-83c8-240c2f3290f6: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:55:06,139::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:55:06,140::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:55:06,141::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:55:06,142::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:55:06,142::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:55:06,143::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:55:06,143::task::495::TaskManager.Task::(_debug) Task 5e66c697-76cb-456a-83c8-240c2f3290f6: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:55:06,144::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:55:07,151::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:55:07,754::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0911508 s, 11.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:55:09,771::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:55:10,279::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0839017 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:55:12,294::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:55:12,830::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0772534 s, 13.3 MB/s\n'; <rc> = 0
>Thread-15915293::DEBUG::2013-11-20 03:55:14,071::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915293::INFO::2013-11-20 03:55:14,073::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915293::DEBUG::2013-11-20 03:55:14,073::task::495::TaskManager.Task::(_debug) Task ce5b0490-e7d5-482f-9d59-65b88b32bd9a: moving from state init -> state preparing
>Thread-15915293::DEBUG::2013-11-20 03:55:14,074::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915293::DEBUG::2013-11-20 03:55:14,075::task::495::TaskManager.Task::(_debug) Task ce5b0490-e7d5-482f-9d59-65b88b32bd9a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915293::DEBUG::2013-11-20 03:55:14,076::task::495::TaskManager.Task::(_debug) Task ce5b0490-e7d5-482f-9d59-65b88b32bd9a: moving from state preparing -> state finished
>Thread-15915293::DEBUG::2013-11-20 03:55:14,076::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915293::DEBUG::2013-11-20 03:55:14,077::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915293::DEBUG::2013-11-20 03:55:14,078::task::495::TaskManager.Task::(_debug) Task ce5b0490-e7d5-482f-9d59-65b88b32bd9a: ref 0 aborting False
>Thread-15915293::INFO::2013-11-20 03:55:14,079::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915294::DEBUG::2013-11-20 03:55:14,084::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915294::INFO::2013-11-20 03:55:14,085::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915294::DEBUG::2013-11-20 03:55:14,085::task::495::TaskManager.Task::(_debug) Task e85dca46-79de-42e8-9bd9-05e5824db74e: moving from state init -> state preparing
>Thread-15915294::DEBUG::2013-11-20 03:55:14,087::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`640d8f5e-ca58-4a1a-9414-733c4705211f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915294::DEBUG::2013-11-20 03:55:14,087::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915294::DEBUG::2013-11-20 03:55:14,088::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915294::DEBUG::2013-11-20 03:55:14,089::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`640d8f5e-ca58-4a1a-9414-733c4705211f`::Granted request
>Thread-15915294::DEBUG::2013-11-20 03:55:14,090::task::495::TaskManager.Task::(_debug) Task e85dca46-79de-42e8-9bd9-05e5824db74e: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915294::DEBUG::2013-11-20 03:55:14,091::task::495::TaskManager.Task::(_debug) Task e85dca46-79de-42e8-9bd9-05e5824db74e: ref 1 aborting False
>Thread-15915294::WARNING::2013-11-20 03:55:14,094::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915294::DEBUG::2013-11-20 03:55:14,095::task::495::TaskManager.Task::(_debug) Task e85dca46-79de-42e8-9bd9-05e5824db74e: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915294::DEBUG::2013-11-20 03:55:14,096::task::495::TaskManager.Task::(_debug) Task e85dca46-79de-42e8-9bd9-05e5824db74e: moving from state preparing -> state finished
>Thread-15915294::DEBUG::2013-11-20 03:55:14,097::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915294::DEBUG::2013-11-20 03:55:14,098::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915294::DEBUG::2013-11-20 03:55:14,099::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915294::DEBUG::2013-11-20 03:55:14,100::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915294::DEBUG::2013-11-20 03:55:14,101::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915294::DEBUG::2013-11-20 03:55:14,101::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915294::DEBUG::2013-11-20 03:55:14,102::task::495::TaskManager.Task::(_debug) Task e85dca46-79de-42e8-9bd9-05e5824db74e: ref 0 aborting False
>Thread-15915294::INFO::2013-11-20 03:55:14,103::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:55:14,844::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:55:15,385::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0720623 s, 14.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:55:17,411::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:55:17,985::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0844389 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15915296::INFO::2013-11-20 03:55:19,366::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915296::DEBUG::2013-11-20 03:55:19,367::task::495::TaskManager.Task::(_debug) Task e477e9e8-1ac9-4a0f-8c32-f57b0be780d4: moving from state init -> state preparing
>Thread-15915296::DEBUG::2013-11-20 03:55:19,368::task::495::TaskManager.Task::(_debug) Task e477e9e8-1ac9-4a0f-8c32-f57b0be780d4: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000308036804199', 'lastCheck': 1384919710.1377931, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0197360515594', 'lastCheck': 1384919712.1684289, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00127506256104', 'lastCheck': 1384919709.6377749, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000364065170288', 'lastCheck': 1384919715.486546, 'valid': True, 'code': 0}}
>Thread-15915296::DEBUG::2013-11-20 03:55:19,369::task::495::TaskManager.Task::(_debug) Task e477e9e8-1ac9-4a0f-8c32-f57b0be780d4: moving from state preparing -> state finished
>Thread-15915296::DEBUG::2013-11-20 03:55:19,369::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915296::DEBUG::2013-11-20 03:55:19,370::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915296::DEBUG::2013-11-20 03:55:19,371::task::495::TaskManager.Task::(_debug) Task e477e9e8-1ac9-4a0f-8c32-f57b0be780d4: ref 0 aborting False
>Thread-15915296::INFO::2013-11-20 03:55:19,372::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000308036804199', 'lastCheck': 1384919710.1377931, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0197360515594', 'lastCheck': 1384919712.1684289, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00127506256104', 'lastCheck': 1384919709.6377749, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000364065170288', 'lastCheck': 1384919715.486546, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:55:20,000::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 03:55:20,275::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:55:20,284::task::495::TaskManager.Task::(_debug) Task f807748a-933f-428f-be37-a9931e6aff15: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:55:20,289::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7e81eb6b-4e94-44cd-b8dd-fb9e6547d0e2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:55:20,290::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:55:20,291::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:55:20,291::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7e81eb6b-4e94-44cd-b8dd-fb9e6547d0e2`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:55:20,292::task::495::TaskManager.Task::(_debug) Task f807748a-933f-428f-be37-a9931e6aff15: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:55:20,293::task::495::TaskManager.Task::(_debug) Task f807748a-933f-428f-be37-a9931e6aff15: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:55:20,293::task::495::TaskManager.Task::(_debug) Task f807748a-933f-428f-be37-a9931e6aff15: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:55:20,294::task::495::TaskManager.Task::(_debug) Task f807748a-933f-428f-be37-a9931e6aff15: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:55:20,294::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:55:20,295::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:55:20,296::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:55:20,296::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:55:20,297::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:55:20,297::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:55:20,298::task::495::TaskManager.Task::(_debug) Task f807748a-933f-428f-be37-a9931e6aff15: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:55:20,298::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:55:20,598::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0577125 s, 17.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:55:22,614::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:55:23,219::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.103235 s, 9.9 MB/s\n'; <rc> = 0
>Thread-15915299::DEBUG::2013-11-20 03:55:24,736::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915299::INFO::2013-11-20 03:55:24,737::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915299::DEBUG::2013-11-20 03:55:24,738::task::495::TaskManager.Task::(_debug) Task 4a300a19-299e-4f08-ad7d-6048f41cf708: moving from state init -> state preparing
>Thread-15915299::DEBUG::2013-11-20 03:55:24,739::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915299::DEBUG::2013-11-20 03:55:24,739::task::495::TaskManager.Task::(_debug) Task 4a300a19-299e-4f08-ad7d-6048f41cf708: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915299::DEBUG::2013-11-20 03:55:24,740::task::495::TaskManager.Task::(_debug) Task 4a300a19-299e-4f08-ad7d-6048f41cf708: moving from state preparing -> state finished
>Thread-15915299::DEBUG::2013-11-20 03:55:24,740::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915299::DEBUG::2013-11-20 03:55:24,741::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915299::DEBUG::2013-11-20 03:55:24,741::task::495::TaskManager.Task::(_debug) Task 4a300a19-299e-4f08-ad7d-6048f41cf708: ref 0 aborting False
>Thread-15915299::INFO::2013-11-20 03:55:24,742::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915300::DEBUG::2013-11-20 03:55:24,746::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915300::INFO::2013-11-20 03:55:24,747::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915300::DEBUG::2013-11-20 03:55:24,748::task::495::TaskManager.Task::(_debug) Task b935dafa-923e-43b9-ab0d-09ea1e0ed7cb: moving from state init -> state preparing
>Thread-15915300::DEBUG::2013-11-20 03:55:24,749::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ad01b8d7-d39a-42cf-bbe2-3018785c4ee8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915300::DEBUG::2013-11-20 03:55:24,750::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915300::DEBUG::2013-11-20 03:55:24,750::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915300::DEBUG::2013-11-20 03:55:24,751::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ad01b8d7-d39a-42cf-bbe2-3018785c4ee8`::Granted request
>Thread-15915300::DEBUG::2013-11-20 03:55:24,752::task::495::TaskManager.Task::(_debug) Task b935dafa-923e-43b9-ab0d-09ea1e0ed7cb: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915300::DEBUG::2013-11-20 03:55:24,753::task::495::TaskManager.Task::(_debug) Task b935dafa-923e-43b9-ab0d-09ea1e0ed7cb: ref 1 aborting False
>Thread-15915300::WARNING::2013-11-20 03:55:24,755::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915300::DEBUG::2013-11-20 03:55:24,756::task::495::TaskManager.Task::(_debug) Task b935dafa-923e-43b9-ab0d-09ea1e0ed7cb: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915300::DEBUG::2013-11-20 03:55:24,756::task::495::TaskManager.Task::(_debug) Task b935dafa-923e-43b9-ab0d-09ea1e0ed7cb: moving from state preparing -> state finished
>Thread-15915300::DEBUG::2013-11-20 03:55:24,757::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915300::DEBUG::2013-11-20 03:55:24,757::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915300::DEBUG::2013-11-20 03:55:24,758::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915300::DEBUG::2013-11-20 03:55:24,759::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915300::DEBUG::2013-11-20 03:55:24,759::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915300::DEBUG::2013-11-20 03:55:24,760::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915300::DEBUG::2013-11-20 03:55:24,760::task::495::TaskManager.Task::(_debug) Task b935dafa-923e-43b9-ab0d-09ea1e0ed7cb: ref 0 aborting False
>Thread-15915300::INFO::2013-11-20 03:55:24,761::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:55:25,238::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:55:25,834::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0835264 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:55:27,854::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:55:28,465::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0944848 s, 10.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:55:30,475::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 03:55:30,723::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:55:30,725::task::495::TaskManager.Task::(_debug) Task f45cebe3-2a94-4898-b8ae-1a5c99cc9bb7: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:55:30,730::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`145c7a93-3380-4160-8e33-312554cde9d6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:55:30,731::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:55:30,732::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:55:30,732::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`145c7a93-3380-4160-8e33-312554cde9d6`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:55:30,733::task::495::TaskManager.Task::(_debug) Task f45cebe3-2a94-4898-b8ae-1a5c99cc9bb7: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:55:30,734::task::495::TaskManager.Task::(_debug) Task f45cebe3-2a94-4898-b8ae-1a5c99cc9bb7: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:55:30,735::task::495::TaskManager.Task::(_debug) Task f45cebe3-2a94-4898-b8ae-1a5c99cc9bb7: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:55:30,736::task::495::TaskManager.Task::(_debug) Task f45cebe3-2a94-4898-b8ae-1a5c99cc9bb7: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:55:30,736::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:55:30,737::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:55:30,737::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:55:30,738::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:55:30,739::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:55:30,739::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:55:30,740::task::495::TaskManager.Task::(_debug) Task f45cebe3-2a94-4898-b8ae-1a5c99cc9bb7: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:55:30,740::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:55:31,069::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0738386 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:55:33,088::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 03:55:33,300::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:55:33,302::task::495::TaskManager.Task::(_debug) Task 95fa4d3b-7b7f-4031-8444-88bb1168a97f: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:55:33,305::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f001135a-bc90-4b91-97a3-ae84e41ac2f7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:55:33,305::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:55:33,306::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:55:33,307::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f001135a-bc90-4b91-97a3-ae84e41ac2f7`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:55:33,308::task::495::TaskManager.Task::(_debug) Task 95fa4d3b-7b7f-4031-8444-88bb1168a97f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:55:33,309::task::495::TaskManager.Task::(_debug) Task 95fa4d3b-7b7f-4031-8444-88bb1168a97f: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:55:33,309::task::495::TaskManager.Task::(_debug) Task 95fa4d3b-7b7f-4031-8444-88bb1168a97f: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:55:33,310::task::495::TaskManager.Task::(_debug) Task 95fa4d3b-7b7f-4031-8444-88bb1168a97f: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:55:33,311::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:55:33,311::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:55:33,312::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:55:33,313::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:55:33,313::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:55:33,314::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:55:33,314::task::495::TaskManager.Task::(_debug) Task 95fa4d3b-7b7f-4031-8444-88bb1168a97f: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:55:33,315::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:55:33,625::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0708011 s, 14.5 MB/s\n'; <rc> = 0
>Thread-15915304::DEBUG::2013-11-20 03:55:35,097::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915304::INFO::2013-11-20 03:55:35,099::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915304::DEBUG::2013-11-20 03:55:35,099::task::495::TaskManager.Task::(_debug) Task 5c975950-9bac-443c-b094-cf5c145a287e: moving from state init -> state preparing
>Thread-15915304::DEBUG::2013-11-20 03:55:35,100::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915304::DEBUG::2013-11-20 03:55:35,101::task::495::TaskManager.Task::(_debug) Task 5c975950-9bac-443c-b094-cf5c145a287e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915304::DEBUG::2013-11-20 03:55:35,101::task::495::TaskManager.Task::(_debug) Task 5c975950-9bac-443c-b094-cf5c145a287e: moving from state preparing -> state finished
>Thread-15915304::DEBUG::2013-11-20 03:55:35,102::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915304::DEBUG::2013-11-20 03:55:35,102::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915304::DEBUG::2013-11-20 03:55:35,103::task::495::TaskManager.Task::(_debug) Task 5c975950-9bac-443c-b094-cf5c145a287e: ref 0 aborting False
>Thread-15915304::INFO::2013-11-20 03:55:35,104::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915305::DEBUG::2013-11-20 03:55:35,109::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915305::INFO::2013-11-20 03:55:35,110::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915305::DEBUG::2013-11-20 03:55:35,111::task::495::TaskManager.Task::(_debug) Task 7f901a34-d5c5-449f-b195-b97969bbaf35: moving from state init -> state preparing
>Thread-15915305::DEBUG::2013-11-20 03:55:35,112::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`872c7d73-a088-48eb-93ad-9a7ad740ec9e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915305::DEBUG::2013-11-20 03:55:35,113::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915305::DEBUG::2013-11-20 03:55:35,114::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915305::DEBUG::2013-11-20 03:55:35,114::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`872c7d73-a088-48eb-93ad-9a7ad740ec9e`::Granted request
>Thread-15915305::DEBUG::2013-11-20 03:55:35,115::task::495::TaskManager.Task::(_debug) Task 7f901a34-d5c5-449f-b195-b97969bbaf35: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915305::DEBUG::2013-11-20 03:55:35,116::task::495::TaskManager.Task::(_debug) Task 7f901a34-d5c5-449f-b195-b97969bbaf35: ref 1 aborting False
>Thread-15915305::WARNING::2013-11-20 03:55:35,118::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915305::DEBUG::2013-11-20 03:55:35,119::task::495::TaskManager.Task::(_debug) Task 7f901a34-d5c5-449f-b195-b97969bbaf35: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915305::DEBUG::2013-11-20 03:55:35,120::task::495::TaskManager.Task::(_debug) Task 7f901a34-d5c5-449f-b195-b97969bbaf35: moving from state preparing -> state finished
>Thread-15915305::DEBUG::2013-11-20 03:55:35,121::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915305::DEBUG::2013-11-20 03:55:35,122::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915305::DEBUG::2013-11-20 03:55:35,122::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915305::DEBUG::2013-11-20 03:55:35,123::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915305::DEBUG::2013-11-20 03:55:35,124::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915305::DEBUG::2013-11-20 03:55:35,124::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915305::DEBUG::2013-11-20 03:55:35,125::task::495::TaskManager.Task::(_debug) Task 7f901a34-d5c5-449f-b195-b97969bbaf35: ref 0 aborting False
>Thread-15915305::INFO::2013-11-20 03:55:35,126::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915306::INFO::2013-11-20 03:55:35,215::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915306::DEBUG::2013-11-20 03:55:35,216::task::495::TaskManager.Task::(_debug) Task 0e42e121-ac74-4162-b627-6b1773d2c32e: moving from state init -> state preparing
>Thread-15915306::DEBUG::2013-11-20 03:55:35,216::task::495::TaskManager.Task::(_debug) Task 0e42e121-ac74-4162-b627-6b1773d2c32e: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00203013420105', 'lastCheck': 1384919730.729876, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00119614601135', 'lastCheck': 1384919732.7245431, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00143313407898', 'lastCheck': 1384919729.845453, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000484943389893', 'lastCheck': 1384919725.612834, 'valid': True, 'code': 0}}
>Thread-15915306::DEBUG::2013-11-20 03:55:35,217::task::495::TaskManager.Task::(_debug) Task 0e42e121-ac74-4162-b627-6b1773d2c32e: moving from state preparing -> state finished
>Thread-15915306::DEBUG::2013-11-20 03:55:35,218::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915306::DEBUG::2013-11-20 03:55:35,218::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915306::DEBUG::2013-11-20 03:55:35,219::task::495::TaskManager.Task::(_debug) Task 0e42e121-ac74-4162-b627-6b1773d2c32e: ref 0 aborting False
>Thread-15915306::INFO::2013-11-20 03:55:35,219::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00203013420105', 'lastCheck': 1384919730.729876, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00119614601135', 'lastCheck': 1384919732.7245431, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00143313407898', 'lastCheck': 1384919729.845453, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000484943389893', 'lastCheck': 1384919725.612834, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:55:35,640::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:55:36,181::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.076901 s, 13.3 MB/s\n'; <rc> = 0
>Thread-15915085::INFO::2013-11-20 03:55:36,258::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=089b08fc-6544-4235-a061-fb55b09d2715 volUUID=66b599fc-b7b5-4934-a5d5-ef7283460029)
>Thread-15915085::DEBUG::2013-11-20 03:55:36,259::task::495::TaskManager.Task::(_debug) Task 54e12386-ac04-47b3-8bb7-4d4ffdb742d4: moving from state init -> state preparing
>Thread-15915085::DEBUG::2013-11-20 03:55:36,261::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0d15d90e-9703-47f1-bf34-e399a5601d01`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915085::DEBUG::2013-11-20 03:55:36,262::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915085::DEBUG::2013-11-20 03:55:36,263::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915085::DEBUG::2013-11-20 03:55:36,264::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0d15d90e-9703-47f1-bf34-e399a5601d01`::Granted request
>Thread-15915085::DEBUG::2013-11-20 03:55:36,265::task::495::TaskManager.Task::(_debug) Task 54e12386-ac04-47b3-8bb7-4d4ffdb742d4: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915085::DEBUG::2013-11-20 03:55:36,266::task::495::TaskManager.Task::(_debug) Task 54e12386-ac04-47b3-8bb7-4d4ffdb742d4: ref 1 aborting False
>Thread-15915085::DEBUG::2013-11-20 03:55:36,267::task::495::TaskManager.Task::(_debug) Task 54e12386-ac04-47b3-8bb7-4d4ffdb742d4: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15915085::DEBUG::2013-11-20 03:55:36,267::task::495::TaskManager.Task::(_debug) Task 54e12386-ac04-47b3-8bb7-4d4ffdb742d4: moving from state preparing -> state finished
>Thread-15915085::DEBUG::2013-11-20 03:55:36,268::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915085::DEBUG::2013-11-20 03:55:36,269::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915085::DEBUG::2013-11-20 03:55:36,270::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915085::DEBUG::2013-11-20 03:55:36,271::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915085::DEBUG::2013-11-20 03:55:36,272::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915085::DEBUG::2013-11-20 03:55:36,273::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915085::DEBUG::2013-11-20 03:55:36,273::task::495::TaskManager.Task::(_debug) Task 54e12386-ac04-47b3-8bb7-4d4ffdb742d4: ref 0 aborting False
>Thread-15915085::INFO::2013-11-20 03:55:36,275::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Dummy-15912651::DEBUG::2013-11-20 03:55:38,195::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:55:38,777::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.090821 s, 11.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:55:40,792::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 03:55:41,070::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 03:55:41,082::task::495::TaskManager.Task::(_debug) Task b343783d-294e-4df4-80a6-033484fe28b8: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 03:55:41,092::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bb3368d3-7d5d-4c52-bdbf-35521ece5b6b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 03:55:41,094::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 03:55:41,095::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 03:55:41,096::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bb3368d3-7d5d-4c52-bdbf-35521ece5b6b`::Granted request
>Thread-15914453::DEBUG::2013-11-20 03:55:41,097::task::495::TaskManager.Task::(_debug) Task b343783d-294e-4df4-80a6-033484fe28b8: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 03:55:41,098::task::495::TaskManager.Task::(_debug) Task b343783d-294e-4df4-80a6-033484fe28b8: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 03:55:41,099::task::495::TaskManager.Task::(_debug) Task b343783d-294e-4df4-80a6-033484fe28b8: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 03:55:41,099::task::495::TaskManager.Task::(_debug) Task b343783d-294e-4df4-80a6-033484fe28b8: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 03:55:41,100::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 03:55:41,101::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 03:55:41,102::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 03:55:41,103::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 03:55:41,103::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 03:55:41,104::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 03:55:41,105::task::495::TaskManager.Task::(_debug) Task b343783d-294e-4df4-80a6-033484fe28b8: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 03:55:41,105::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 03:55:41,392::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0773889 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:55:43,415::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 03:55:43,644::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 03:55:43,646::task::495::TaskManager.Task::(_debug) Task b1b90dbf-752c-4646-8f83-e391f3b14bb1: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 03:55:43,648::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`31a532e0-3ee9-4faa-ad94-be31e5943655`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 03:55:43,649::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 03:55:43,650::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 03:55:43,651::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`31a532e0-3ee9-4faa-ad94-be31e5943655`::Granted request
>Thread-15914435::DEBUG::2013-11-20 03:55:43,652::task::495::TaskManager.Task::(_debug) Task b1b90dbf-752c-4646-8f83-e391f3b14bb1: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 03:55:43,653::task::495::TaskManager.Task::(_debug) Task b1b90dbf-752c-4646-8f83-e391f3b14bb1: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 03:55:43,654::task::495::TaskManager.Task::(_debug) Task b1b90dbf-752c-4646-8f83-e391f3b14bb1: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 03:55:43,654::task::495::TaskManager.Task::(_debug) Task b1b90dbf-752c-4646-8f83-e391f3b14bb1: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 03:55:43,655::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 03:55:43,655::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 03:55:43,656::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 03:55:43,657::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 03:55:43,657::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 03:55:43,658::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 03:55:43,659::task::495::TaskManager.Task::(_debug) Task b1b90dbf-752c-4646-8f83-e391f3b14bb1: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 03:55:43,659::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 03:55:43,986::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0808885 s, 12.7 MB/s\n'; <rc> = 0
>Thread-15915311::DEBUG::2013-11-20 03:55:45,568::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915311::INFO::2013-11-20 03:55:45,570::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915311::DEBUG::2013-11-20 03:55:45,570::task::495::TaskManager.Task::(_debug) Task d048fca3-3228-4f6f-9ac4-f8eceb9aa5c4: moving from state init -> state preparing
>Thread-15915311::DEBUG::2013-11-20 03:55:45,571::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915311::DEBUG::2013-11-20 03:55:45,572::task::495::TaskManager.Task::(_debug) Task d048fca3-3228-4f6f-9ac4-f8eceb9aa5c4: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915311::DEBUG::2013-11-20 03:55:45,572::task::495::TaskManager.Task::(_debug) Task d048fca3-3228-4f6f-9ac4-f8eceb9aa5c4: moving from state preparing -> state finished
>Thread-15915311::DEBUG::2013-11-20 03:55:45,573::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915311::DEBUG::2013-11-20 03:55:45,574::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915311::DEBUG::2013-11-20 03:55:45,574::task::495::TaskManager.Task::(_debug) Task d048fca3-3228-4f6f-9ac4-f8eceb9aa5c4: ref 0 aborting False
>Thread-15915311::INFO::2013-11-20 03:55:45,575::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915312::DEBUG::2013-11-20 03:55:45,581::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915312::INFO::2013-11-20 03:55:45,582::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915312::DEBUG::2013-11-20 03:55:45,584::task::495::TaskManager.Task::(_debug) Task fadf7e5d-5806-4c95-a4cb-9f8078ce3ff7: moving from state init -> state preparing
>Thread-15915312::DEBUG::2013-11-20 03:55:45,585::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b2c5dab0-ab6c-482a-ade1-f3d07110cc7a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915312::DEBUG::2013-11-20 03:55:45,586::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915312::DEBUG::2013-11-20 03:55:45,588::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915312::DEBUG::2013-11-20 03:55:45,589::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b2c5dab0-ab6c-482a-ade1-f3d07110cc7a`::Granted request
>Thread-15915312::DEBUG::2013-11-20 03:55:45,590::task::495::TaskManager.Task::(_debug) Task fadf7e5d-5806-4c95-a4cb-9f8078ce3ff7: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915312::DEBUG::2013-11-20 03:55:45,591::task::495::TaskManager.Task::(_debug) Task fadf7e5d-5806-4c95-a4cb-9f8078ce3ff7: ref 1 aborting False
>Thread-15915312::WARNING::2013-11-20 03:55:45,593::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915312::DEBUG::2013-11-20 03:55:45,594::task::495::TaskManager.Task::(_debug) Task fadf7e5d-5806-4c95-a4cb-9f8078ce3ff7: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915312::DEBUG::2013-11-20 03:55:45,595::task::495::TaskManager.Task::(_debug) Task fadf7e5d-5806-4c95-a4cb-9f8078ce3ff7: moving from state preparing -> state finished
>Thread-15915312::DEBUG::2013-11-20 03:55:45,596::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915312::DEBUG::2013-11-20 03:55:45,596::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915312::DEBUG::2013-11-20 03:55:45,597::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915312::DEBUG::2013-11-20 03:55:45,598::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915312::DEBUG::2013-11-20 03:55:45,598::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915312::DEBUG::2013-11-20 03:55:45,599::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915312::DEBUG::2013-11-20 03:55:45,600::task::495::TaskManager.Task::(_debug) Task fadf7e5d-5806-4c95-a4cb-9f8078ce3ff7: ref 0 aborting False
>Thread-15915312::INFO::2013-11-20 03:55:45,600::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:55:46,000::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:55:46,565::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0983598 s, 10.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:55:48,584::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 03:55:48,821::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 03:55:48,824::task::495::TaskManager.Task::(_debug) Task c586339c-56d5-4a65-b81a-30e6b55ffbdb: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 03:55:48,833::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`271343c2-6b78-4ddf-82ca-29934baa9553`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 03:55:48,835::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 03:55:48,836::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 03:55:48,837::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`271343c2-6b78-4ddf-82ca-29934baa9553`::Granted request
>Thread-15914519::DEBUG::2013-11-20 03:55:48,843::task::495::TaskManager.Task::(_debug) Task c586339c-56d5-4a65-b81a-30e6b55ffbdb: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 03:55:48,844::task::495::TaskManager.Task::(_debug) Task c586339c-56d5-4a65-b81a-30e6b55ffbdb: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 03:55:48,845::task::495::TaskManager.Task::(_debug) Task c586339c-56d5-4a65-b81a-30e6b55ffbdb: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 03:55:48,846::task::495::TaskManager.Task::(_debug) Task c586339c-56d5-4a65-b81a-30e6b55ffbdb: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 03:55:48,847::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 03:55:48,848::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 03:55:48,849::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 03:55:48,850::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 03:55:48,851::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 03:55:48,852::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 03:55:48,853::task::495::TaskManager.Task::(_debug) Task c586339c-56d5-4a65-b81a-30e6b55ffbdb: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 03:55:48,854::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 03:55:49,133::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0793519 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:55:51,148::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 03:55:51,385::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:55:51,393::task::495::TaskManager.Task::(_debug) Task ea39fb91-d1ff-4642-bd56-eb29f3ddce1d: moving from state init -> state preparing
>Thread-15915314::INFO::2013-11-20 03:55:51,399::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15913546::DEBUG::2013-11-20 03:55:51,402::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`5aabe665-eb26-4a03-ade2-dab6479c8694`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915314::DEBUG::2013-11-20 03:55:51,403::task::495::TaskManager.Task::(_debug) Task 1bf0f4ed-1e84-4db6-9afd-1a3a2f5a4540: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:55:51,405::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15915314::DEBUG::2013-11-20 03:55:51,406::task::495::TaskManager.Task::(_debug) Task 1bf0f4ed-1e84-4db6-9afd-1a3a2f5a4540: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000570058822632', 'lastCheck': 1384919751.406168, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00232100486755', 'lastCheck': 1384919743.0793581, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00136709213257', 'lastCheck': 1384919750.2321479, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0239851474762', 'lastCheck': 1384919746.25614, 'valid': True, 'code': 0}}
>Thread-15915314::DEBUG::2013-11-20 03:55:51,408::task::495::TaskManager.Task::(_debug) Task 1bf0f4ed-1e84-4db6-9afd-1a3a2f5a4540: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:55:51,409::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15915314::DEBUG::2013-11-20 03:55:51,410::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913546::DEBUG::2013-11-20 03:55:51,411::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`5aabe665-eb26-4a03-ade2-dab6479c8694`::Granted request
>Thread-15915314::DEBUG::2013-11-20 03:55:51,411::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:55:51,414::task::495::TaskManager.Task::(_debug) Task ea39fb91-d1ff-4642-bd56-eb29f3ddce1d: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15915314::DEBUG::2013-11-20 03:55:51,414::task::495::TaskManager.Task::(_debug) Task 1bf0f4ed-1e84-4db6-9afd-1a3a2f5a4540: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:55:51,415::task::495::TaskManager.Task::(_debug) Task ea39fb91-d1ff-4642-bd56-eb29f3ddce1d: ref 1 aborting False
>Thread-15915314::INFO::2013-11-20 03:55:51,416::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000570058822632', 'lastCheck': 1384919751.406168, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00232100486755', 'lastCheck': 1384919743.0793581, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00136709213257', 'lastCheck': 1384919750.2321479, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0239851474762', 'lastCheck': 1384919746.25614, 'valid': True, 'code': 0}}
>Thread-15913546::DEBUG::2013-11-20 03:55:51,417::task::495::TaskManager.Task::(_debug) Task ea39fb91-d1ff-4642-bd56-eb29f3ddce1d: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:55:51,418::task::495::TaskManager.Task::(_debug) Task ea39fb91-d1ff-4642-bd56-eb29f3ddce1d: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:55:51,419::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:55:51,420::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:55:51,422::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:55:51,424::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:55:51,426::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:55:51,427::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:55:51,429::task::495::TaskManager.Task::(_debug) Task ea39fb91-d1ff-4642-bd56-eb29f3ddce1d: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:55:51,430::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:55:51,432::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:55:51,433::task::495::TaskManager.Task::(_debug) Task ccfda834-046e-4cad-85a6-8fbfc66b702d: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:55:51,439::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6e3cd6ef-0043-4c77-9dfe-459f080f7c71`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:55:51,440::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:55:51,441::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:55:51,442::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6e3cd6ef-0043-4c77-9dfe-459f080f7c71`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:55:51,445::task::495::TaskManager.Task::(_debug) Task ccfda834-046e-4cad-85a6-8fbfc66b702d: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:55:51,447::task::495::TaskManager.Task::(_debug) Task ccfda834-046e-4cad-85a6-8fbfc66b702d: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:55:51,448::task::495::TaskManager.Task::(_debug) Task ccfda834-046e-4cad-85a6-8fbfc66b702d: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:55:51,476::task::495::TaskManager.Task::(_debug) Task ccfda834-046e-4cad-85a6-8fbfc66b702d: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:55:51,477::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:55:51,478::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:55:51,479::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:55:51,480::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:55:51,481::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:55:51,482::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:55:51,483::task::495::TaskManager.Task::(_debug) Task ccfda834-046e-4cad-85a6-8fbfc66b702d: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:55:51,484::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:55:51,485::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:55:51,486::task::495::TaskManager.Task::(_debug) Task a2a1c6ea-9064-48f6-845f-8d897be1a872: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:55:51,487::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`853bcec3-0bbb-4e62-8ef9-b1f2b2fa4c54`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:55:51,488::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:55:51,488::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:55:51,489::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`853bcec3-0bbb-4e62-8ef9-b1f2b2fa4c54`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:55:51,490::task::495::TaskManager.Task::(_debug) Task a2a1c6ea-9064-48f6-845f-8d897be1a872: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:55:51,490::task::495::TaskManager.Task::(_debug) Task a2a1c6ea-9064-48f6-845f-8d897be1a872: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:55:51,491::task::495::TaskManager.Task::(_debug) Task a2a1c6ea-9064-48f6-845f-8d897be1a872: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:55:51,492::task::495::TaskManager.Task::(_debug) Task a2a1c6ea-9064-48f6-845f-8d897be1a872: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:55:51,492::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:55:51,493::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:55:51,494::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:55:51,494::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:55:51,495::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:55:51,495::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:55:51,496::task::495::TaskManager.Task::(_debug) Task a2a1c6ea-9064-48f6-845f-8d897be1a872: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:55:51,496::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:55:51,497::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:55:51,498::task::495::TaskManager.Task::(_debug) Task 2a09a0c5-1aa6-47a5-a7f9-de6df13165a5: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:55:51,499::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b84fb471-913d-4f67-b722-d99b0fbd96d0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:55:51,500::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:55:51,500::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:55:51,501::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b84fb471-913d-4f67-b722-d99b0fbd96d0`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:55:51,502::task::495::TaskManager.Task::(_debug) Task 2a09a0c5-1aa6-47a5-a7f9-de6df13165a5: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:55:51,503::task::495::TaskManager.Task::(_debug) Task 2a09a0c5-1aa6-47a5-a7f9-de6df13165a5: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:55:51,503::task::495::TaskManager.Task::(_debug) Task 2a09a0c5-1aa6-47a5-a7f9-de6df13165a5: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:55:51,504::task::495::TaskManager.Task::(_debug) Task 2a09a0c5-1aa6-47a5-a7f9-de6df13165a5: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:55:51,504::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:55:51,505::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:55:51,506::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:55:51,506::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:55:51,507::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:55:51,507::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:55:51,508::task::495::TaskManager.Task::(_debug) Task 2a09a0c5-1aa6-47a5-a7f9-de6df13165a5: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:55:51,509::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:55:51,753::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0912205 s, 11.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:55:53,770::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:55:54,318::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0516663 s, 19.8 MB/s\n'; <rc> = 0
>Thread-15915317::DEBUG::2013-11-20 03:55:55,901::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915317::INFO::2013-11-20 03:55:55,902::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915317::DEBUG::2013-11-20 03:55:55,903::task::495::TaskManager.Task::(_debug) Task 63fc7c21-5703-4261-9e89-eddf1448104a: moving from state init -> state preparing
>Thread-15915317::DEBUG::2013-11-20 03:55:55,903::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915317::DEBUG::2013-11-20 03:55:55,904::task::495::TaskManager.Task::(_debug) Task 63fc7c21-5703-4261-9e89-eddf1448104a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915317::DEBUG::2013-11-20 03:55:55,904::task::495::TaskManager.Task::(_debug) Task 63fc7c21-5703-4261-9e89-eddf1448104a: moving from state preparing -> state finished
>Thread-15915317::DEBUG::2013-11-20 03:55:55,905::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915317::DEBUG::2013-11-20 03:55:55,905::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915317::DEBUG::2013-11-20 03:55:55,906::task::495::TaskManager.Task::(_debug) Task 63fc7c21-5703-4261-9e89-eddf1448104a: ref 0 aborting False
>Thread-15915317::INFO::2013-11-20 03:55:55,906::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915318::DEBUG::2013-11-20 03:55:55,911::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915318::INFO::2013-11-20 03:55:55,912::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915318::DEBUG::2013-11-20 03:55:55,913::task::495::TaskManager.Task::(_debug) Task 572da371-5a4b-45e3-a3ce-be17ee5a72b4: moving from state init -> state preparing
>Thread-15915318::DEBUG::2013-11-20 03:55:55,914::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`762f5358-f9f1-4f15-88a3-b02efbb37450`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915318::DEBUG::2013-11-20 03:55:55,915::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915318::DEBUG::2013-11-20 03:55:55,916::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915318::DEBUG::2013-11-20 03:55:55,917::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`762f5358-f9f1-4f15-88a3-b02efbb37450`::Granted request
>Thread-15915318::DEBUG::2013-11-20 03:55:55,918::task::495::TaskManager.Task::(_debug) Task 572da371-5a4b-45e3-a3ce-be17ee5a72b4: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915318::DEBUG::2013-11-20 03:55:55,918::task::495::TaskManager.Task::(_debug) Task 572da371-5a4b-45e3-a3ce-be17ee5a72b4: ref 1 aborting False
>Thread-15915318::WARNING::2013-11-20 03:55:55,921::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915318::DEBUG::2013-11-20 03:55:55,922::task::495::TaskManager.Task::(_debug) Task 572da371-5a4b-45e3-a3ce-be17ee5a72b4: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915318::DEBUG::2013-11-20 03:55:55,923::task::495::TaskManager.Task::(_debug) Task 572da371-5a4b-45e3-a3ce-be17ee5a72b4: moving from state preparing -> state finished
>Thread-15915318::DEBUG::2013-11-20 03:55:55,924::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915318::DEBUG::2013-11-20 03:55:55,924::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915318::DEBUG::2013-11-20 03:55:55,925::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915318::DEBUG::2013-11-20 03:55:55,926::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915318::DEBUG::2013-11-20 03:55:55,927::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915318::DEBUG::2013-11-20 03:55:55,927::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915318::DEBUG::2013-11-20 03:55:55,928::task::495::TaskManager.Task::(_debug) Task 572da371-5a4b-45e3-a3ce-be17ee5a72b4: ref 0 aborting False
>Thread-15915318::INFO::2013-11-20 03:55:55,929::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:55:56,327::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:55:56,841::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0515667 s, 19.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:55:58,854::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:55:59,426::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0981186 s, 10.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:56:01,444::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:56:01,965::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0730944 s, 14.0 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 03:56:02,125::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:56:02,126::task::495::TaskManager.Task::(_debug) Task 84b9a5d7-5555-4c99-84d1-06f223e56071: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:56:02,128::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3baa6a14-8f95-49a4-aed7-c3ff12c31770`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:56:02,129::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:56:02,130::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:56:02,131::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3baa6a14-8f95-49a4-aed7-c3ff12c31770`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:56:02,132::task::495::TaskManager.Task::(_debug) Task 84b9a5d7-5555-4c99-84d1-06f223e56071: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:56:02,133::task::495::TaskManager.Task::(_debug) Task 84b9a5d7-5555-4c99-84d1-06f223e56071: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:56:02,134::task::495::TaskManager.Task::(_debug) Task 84b9a5d7-5555-4c99-84d1-06f223e56071: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:56:02,134::task::495::TaskManager.Task::(_debug) Task 84b9a5d7-5555-4c99-84d1-06f223e56071: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:56:02,135::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:56:02,136::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:56:02,137::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:56:02,137::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:56:02,138::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:56:02,138::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:56:02,139::task::495::TaskManager.Task::(_debug) Task 84b9a5d7-5555-4c99-84d1-06f223e56071: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:56:02,140::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:56:02,141::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:56:02,142::task::495::TaskManager.Task::(_debug) Task bac763d1-91c8-4f8e-a5da-0ef983fd160d: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:56:02,143::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1076c88b-6598-4b5c-b9e6-4c755692c80a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:56:02,144::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:56:02,144::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:56:02,145::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1076c88b-6598-4b5c-b9e6-4c755692c80a`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:56:02,146::task::495::TaskManager.Task::(_debug) Task bac763d1-91c8-4f8e-a5da-0ef983fd160d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:56:02,147::task::495::TaskManager.Task::(_debug) Task bac763d1-91c8-4f8e-a5da-0ef983fd160d: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:56:02,147::task::495::TaskManager.Task::(_debug) Task bac763d1-91c8-4f8e-a5da-0ef983fd160d: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:56:02,148::task::495::TaskManager.Task::(_debug) Task bac763d1-91c8-4f8e-a5da-0ef983fd160d: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:56:02,149::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:56:02,149::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:56:02,150::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:56:02,151::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:56:02,151::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:56:02,152::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:56:02,153::task::495::TaskManager.Task::(_debug) Task bac763d1-91c8-4f8e-a5da-0ef983fd160d: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:56:02,153::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:56:03,979::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:56:04,542::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0895019 s, 11.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:56:06,556::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 03:56:06,789::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:56:06,793::task::495::TaskManager.Task::(_debug) Task d98f24b8-8d4e-40a3-9035-cbd76cfc55f6: moving from state init -> state preparing
>Thread-15915322::DEBUG::2013-11-20 03:56:06,797::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913639::DEBUG::2013-11-20 03:56:06,801::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a7ac076e-b652-4ec3-ac3c-17da303adc02`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915322::INFO::2013-11-20 03:56:06,804::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913639::DEBUG::2013-11-20 03:56:06,805::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915322::DEBUG::2013-11-20 03:56:06,806::task::495::TaskManager.Task::(_debug) Task 64971121-5a77-4856-af7f-b0763089a5b0: moving from state init -> state preparing
>Thread-15915322::DEBUG::2013-11-20 03:56:06,809::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913639::DEBUG::2013-11-20 03:56:06,810::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915322::DEBUG::2013-11-20 03:56:06,811::task::495::TaskManager.Task::(_debug) Task 64971121-5a77-4856-af7f-b0763089a5b0: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913639::DEBUG::2013-11-20 03:56:06,812::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a7ac076e-b652-4ec3-ac3c-17da303adc02`::Granted request
>Thread-15915322::DEBUG::2013-11-20 03:56:06,813::task::495::TaskManager.Task::(_debug) Task 64971121-5a77-4856-af7f-b0763089a5b0: moving from state preparing -> state finished
>Thread-15915322::DEBUG::2013-11-20 03:56:06,816::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913639::DEBUG::2013-11-20 03:56:06,817::task::495::TaskManager.Task::(_debug) Task d98f24b8-8d4e-40a3-9035-cbd76cfc55f6: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915322::DEBUG::2013-11-20 03:56:06,818::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:56:06,819::task::495::TaskManager.Task::(_debug) Task d98f24b8-8d4e-40a3-9035-cbd76cfc55f6: ref 1 aborting False
>Thread-15915322::DEBUG::2013-11-20 03:56:06,820::task::495::TaskManager.Task::(_debug) Task 64971121-5a77-4856-af7f-b0763089a5b0: ref 0 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:56:06,821::task::495::TaskManager.Task::(_debug) Task d98f24b8-8d4e-40a3-9035-cbd76cfc55f6: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15915322::INFO::2013-11-20 03:56:06,822::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913639::DEBUG::2013-11-20 03:56:06,823::task::495::TaskManager.Task::(_debug) Task d98f24b8-8d4e-40a3-9035-cbd76cfc55f6: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:56:06,825::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:56:06,828::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:56:06,830::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:56:06,831::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915323::DEBUG::2013-11-20 03:56:06,833::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913639::DEBUG::2013-11-20 03:56:06,834::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915323::INFO::2013-11-20 03:56:06,835::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913639::DEBUG::2013-11-20 03:56:06,836::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915323::DEBUG::2013-11-20 03:56:06,837::task::495::TaskManager.Task::(_debug) Task b8b49973-729c-463d-999a-7ebd14a17fa3: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:56:06,838::task::495::TaskManager.Task::(_debug) Task d98f24b8-8d4e-40a3-9035-cbd76cfc55f6: ref 0 aborting False
>Thread-15915323::DEBUG::2013-11-20 03:56:06,840::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7540aca6-44f0-46ca-91a9-df0467aba597`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913639::INFO::2013-11-20 03:56:06,841::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15915323::DEBUG::2013-11-20 03:56:06,842::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915323::DEBUG::2013-11-20 03:56:06,843::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915323::DEBUG::2013-11-20 03:56:06,844::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7540aca6-44f0-46ca-91a9-df0467aba597`::Granted request
>Thread-15915323::DEBUG::2013-11-20 03:56:06,847::task::495::TaskManager.Task::(_debug) Task b8b49973-729c-463d-999a-7ebd14a17fa3: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915323::DEBUG::2013-11-20 03:56:06,848::task::495::TaskManager.Task::(_debug) Task b8b49973-729c-463d-999a-7ebd14a17fa3: ref 1 aborting False
>Thread-15915323::WARNING::2013-11-20 03:56:06,852::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915323::DEBUG::2013-11-20 03:56:06,854::task::495::TaskManager.Task::(_debug) Task b8b49973-729c-463d-999a-7ebd14a17fa3: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915323::DEBUG::2013-11-20 03:56:06,855::task::495::TaskManager.Task::(_debug) Task b8b49973-729c-463d-999a-7ebd14a17fa3: moving from state preparing -> state finished
>Thread-15915323::DEBUG::2013-11-20 03:56:06,856::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915323::DEBUG::2013-11-20 03:56:06,857::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915323::DEBUG::2013-11-20 03:56:06,866::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915323::DEBUG::2013-11-20 03:56:06,868::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915323::DEBUG::2013-11-20 03:56:06,869::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915323::DEBUG::2013-11-20 03:56:06,870::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915323::DEBUG::2013-11-20 03:56:06,873::task::495::TaskManager.Task::(_debug) Task b8b49973-729c-463d-999a-7ebd14a17fa3: ref 0 aborting False
>Thread-15915323::INFO::2013-11-20 03:56:06,874::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:56:07,154::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.102113 s, 10.0 MB/s\n'; <rc> = 0
>Thread-15915324::INFO::2013-11-20 03:56:07,270::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915324::DEBUG::2013-11-20 03:56:07,272::task::495::TaskManager.Task::(_debug) Task 28ba8ceb-1713-44f8-bb87-ff446212f8a2: moving from state init -> state preparing
>Thread-15915324::DEBUG::2013-11-20 03:56:07,274::task::495::TaskManager.Task::(_debug) Task 28ba8ceb-1713-44f8-bb87-ff446212f8a2: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00068998336792', 'lastCheck': 1384919761.6598411, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00230479240417', 'lastCheck': 1384919763.658814, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00147318840027', 'lastCheck': 1384919760.5748689, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00401401519775', 'lastCheck': 1384919766.7885561, 'valid': True, 'code': 0}}
>Thread-15915324::DEBUG::2013-11-20 03:56:07,275::task::495::TaskManager.Task::(_debug) Task 28ba8ceb-1713-44f8-bb87-ff446212f8a2: moving from state preparing -> state finished
>Thread-15915324::DEBUG::2013-11-20 03:56:07,276::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915324::DEBUG::2013-11-20 03:56:07,276::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915324::DEBUG::2013-11-20 03:56:07,277::task::495::TaskManager.Task::(_debug) Task 28ba8ceb-1713-44f8-bb87-ff446212f8a2: ref 0 aborting False
>Thread-15915324::INFO::2013-11-20 03:56:07,278::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00068998336792', 'lastCheck': 1384919761.6598411, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00230479240417', 'lastCheck': 1384919763.658814, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00147318840027', 'lastCheck': 1384919760.5748689, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00401401519775', 'lastCheck': 1384919766.7885561, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:56:09,173::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:56:09,760::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0857545 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:56:11,777::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:56:12,378::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0842111 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:56:14,398::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:56:14,964::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0948938 s, 10.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:56:16,974::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915328::DEBUG::2013-11-20 03:56:17,269::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915328::INFO::2013-11-20 03:56:17,272::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915328::DEBUG::2013-11-20 03:56:17,273::task::495::TaskManager.Task::(_debug) Task 86647f2a-602e-40d9-ba88-b35f24072196: moving from state init -> state preparing
>Thread-15915328::DEBUG::2013-11-20 03:56:17,274::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915328::DEBUG::2013-11-20 03:56:17,275::task::495::TaskManager.Task::(_debug) Task 86647f2a-602e-40d9-ba88-b35f24072196: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915328::DEBUG::2013-11-20 03:56:17,275::task::495::TaskManager.Task::(_debug) Task 86647f2a-602e-40d9-ba88-b35f24072196: moving from state preparing -> state finished
>Thread-15915328::DEBUG::2013-11-20 03:56:17,276::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915328::DEBUG::2013-11-20 03:56:17,277::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915328::DEBUG::2013-11-20 03:56:17,278::task::495::TaskManager.Task::(_debug) Task 86647f2a-602e-40d9-ba88-b35f24072196: ref 0 aborting False
>Thread-15915328::INFO::2013-11-20 03:56:17,279::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915329::DEBUG::2013-11-20 03:56:17,290::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915329::INFO::2013-11-20 03:56:17,291::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915329::DEBUG::2013-11-20 03:56:17,292::task::495::TaskManager.Task::(_debug) Task eb9963a3-42e4-405a-b88e-e0af629c31e9: moving from state init -> state preparing
>Thread-15915329::DEBUG::2013-11-20 03:56:17,293::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c8800f35-ded7-4db9-b1ae-bc4b8f3a70c6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915329::DEBUG::2013-11-20 03:56:17,294::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915329::DEBUG::2013-11-20 03:56:17,295::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915329::DEBUG::2013-11-20 03:56:17,296::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c8800f35-ded7-4db9-b1ae-bc4b8f3a70c6`::Granted request
>Thread-15915329::DEBUG::2013-11-20 03:56:17,297::task::495::TaskManager.Task::(_debug) Task eb9963a3-42e4-405a-b88e-e0af629c31e9: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915329::DEBUG::2013-11-20 03:56:17,298::task::495::TaskManager.Task::(_debug) Task eb9963a3-42e4-405a-b88e-e0af629c31e9: ref 1 aborting False
>Thread-15915329::WARNING::2013-11-20 03:56:17,305::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915329::DEBUG::2013-11-20 03:56:17,306::task::495::TaskManager.Task::(_debug) Task eb9963a3-42e4-405a-b88e-e0af629c31e9: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915329::DEBUG::2013-11-20 03:56:17,307::task::495::TaskManager.Task::(_debug) Task eb9963a3-42e4-405a-b88e-e0af629c31e9: moving from state preparing -> state finished
>Thread-15915329::DEBUG::2013-11-20 03:56:17,308::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915329::DEBUG::2013-11-20 03:56:17,308::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915329::DEBUG::2013-11-20 03:56:17,309::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915329::DEBUG::2013-11-20 03:56:17,310::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915329::DEBUG::2013-11-20 03:56:17,311::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915329::DEBUG::2013-11-20 03:56:17,311::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915329::DEBUG::2013-11-20 03:56:17,312::task::495::TaskManager.Task::(_debug) Task eb9963a3-42e4-405a-b88e-e0af629c31e9: ref 0 aborting False
>Thread-15915329::INFO::2013-11-20 03:56:17,313::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:56:17,525::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0663944 s, 15.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:56:19,539::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:56:20,067::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0964532 s, 10.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:56:22,085::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 03:56:22,320::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:56:22,323::task::495::TaskManager.Task::(_debug) Task 1bf19988-506c-4e47-ae98-5824ae81f4f2: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:56:22,328::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f8aad029-a19a-4378-a5c9-e3499773827e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:56:22,329::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:56:22,331::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:56:22,331::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f8aad029-a19a-4378-a5c9-e3499773827e`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:56:22,333::task::495::TaskManager.Task::(_debug) Task 1bf19988-506c-4e47-ae98-5824ae81f4f2: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:56:22,333::task::495::TaskManager.Task::(_debug) Task 1bf19988-506c-4e47-ae98-5824ae81f4f2: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:56:22,334::task::495::TaskManager.Task::(_debug) Task 1bf19988-506c-4e47-ae98-5824ae81f4f2: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:56:22,335::task::495::TaskManager.Task::(_debug) Task 1bf19988-506c-4e47-ae98-5824ae81f4f2: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:56:22,336::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:56:22,336::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:56:22,337::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:56:22,338::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:56:22,339::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:56:22,340::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:56:22,341::task::495::TaskManager.Task::(_debug) Task 1bf19988-506c-4e47-ae98-5824ae81f4f2: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:56:22,342::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:56:22,637::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.077413 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15915332::INFO::2013-11-20 03:56:23,381::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915332::DEBUG::2013-11-20 03:56:23,382::task::495::TaskManager.Task::(_debug) Task b2b2a6ca-0118-41a1-8828-b3142f9e1c8b: moving from state init -> state preparing
>Thread-15915332::DEBUG::2013-11-20 03:56:23,382::task::495::TaskManager.Task::(_debug) Task b2b2a6ca-0118-41a1-8828-b3142f9e1c8b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00248503684998', 'lastCheck': 1384919782.32692, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00133514404297', 'lastCheck': 1384919774.0444281, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00142002105713', 'lastCheck': 1384919781.210012, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0346610546112', 'lastCheck': 1384919777.2454901, 'valid': True, 'code': 0}}
>Thread-15915332::DEBUG::2013-11-20 03:56:23,383::task::495::TaskManager.Task::(_debug) Task b2b2a6ca-0118-41a1-8828-b3142f9e1c8b: moving from state preparing -> state finished
>Thread-15915332::DEBUG::2013-11-20 03:56:23,383::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915332::DEBUG::2013-11-20 03:56:23,384::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915332::DEBUG::2013-11-20 03:56:23,384::task::495::TaskManager.Task::(_debug) Task b2b2a6ca-0118-41a1-8828-b3142f9e1c8b: ref 0 aborting False
>Thread-15915332::INFO::2013-11-20 03:56:23,385::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00248503684998', 'lastCheck': 1384919782.32692, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00133514404297', 'lastCheck': 1384919774.0444281, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00142002105713', 'lastCheck': 1384919781.210012, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0346610546112', 'lastCheck': 1384919777.2454901, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:56:24,651::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:56:25,206::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0907082 s, 11.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:56:27,222::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915335::DEBUG::2013-11-20 03:56:27,630::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915335::INFO::2013-11-20 03:56:27,632::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915335::DEBUG::2013-11-20 03:56:27,633::task::495::TaskManager.Task::(_debug) Task d1b827a3-003c-493b-b1ef-a79e695ebca7: moving from state init -> state preparing
>Thread-15915335::DEBUG::2013-11-20 03:56:27,634::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915335::DEBUG::2013-11-20 03:56:27,635::task::495::TaskManager.Task::(_debug) Task d1b827a3-003c-493b-b1ef-a79e695ebca7: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915335::DEBUG::2013-11-20 03:56:27,635::task::495::TaskManager.Task::(_debug) Task d1b827a3-003c-493b-b1ef-a79e695ebca7: moving from state preparing -> state finished
>Thread-15915335::DEBUG::2013-11-20 03:56:27,636::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915335::DEBUG::2013-11-20 03:56:27,637::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915335::DEBUG::2013-11-20 03:56:27,637::task::495::TaskManager.Task::(_debug) Task d1b827a3-003c-493b-b1ef-a79e695ebca7: ref 0 aborting False
>Thread-15915335::INFO::2013-11-20 03:56:27,638::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915336::DEBUG::2013-11-20 03:56:27,649::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915336::INFO::2013-11-20 03:56:27,650::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915336::DEBUG::2013-11-20 03:56:27,651::task::495::TaskManager.Task::(_debug) Task 59ddcdd2-bf32-43f9-9929-0430ab2818a1: moving from state init -> state preparing
>Thread-15915336::DEBUG::2013-11-20 03:56:27,653::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3dbfba9f-3658-4736-9531-7d142f34c971`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915336::DEBUG::2013-11-20 03:56:27,653::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915336::DEBUG::2013-11-20 03:56:27,654::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915336::DEBUG::2013-11-20 03:56:27,655::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3dbfba9f-3658-4736-9531-7d142f34c971`::Granted request
>Thread-15915336::DEBUG::2013-11-20 03:56:27,656::task::495::TaskManager.Task::(_debug) Task 59ddcdd2-bf32-43f9-9929-0430ab2818a1: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915336::DEBUG::2013-11-20 03:56:27,657::task::495::TaskManager.Task::(_debug) Task 59ddcdd2-bf32-43f9-9929-0430ab2818a1: ref 1 aborting False
>Thread-15915336::WARNING::2013-11-20 03:56:27,659::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915336::DEBUG::2013-11-20 03:56:27,660::task::495::TaskManager.Task::(_debug) Task 59ddcdd2-bf32-43f9-9929-0430ab2818a1: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915336::DEBUG::2013-11-20 03:56:27,662::task::495::TaskManager.Task::(_debug) Task 59ddcdd2-bf32-43f9-9929-0430ab2818a1: moving from state preparing -> state finished
>Thread-15915336::DEBUG::2013-11-20 03:56:27,662::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915336::DEBUG::2013-11-20 03:56:27,663::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915336::DEBUG::2013-11-20 03:56:27,664::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915336::DEBUG::2013-11-20 03:56:27,665::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915336::DEBUG::2013-11-20 03:56:27,665::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915336::DEBUG::2013-11-20 03:56:27,666::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915336::DEBUG::2013-11-20 03:56:27,667::task::495::TaskManager.Task::(_debug) Task 59ddcdd2-bf32-43f9-9929-0430ab2818a1: ref 0 aborting False
>Thread-15915336::INFO::2013-11-20 03:56:27,667::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:56:27,820::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0818451 s, 12.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:56:29,835::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:56:30,442::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0912228 s, 11.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:56:32,458::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 03:56:32,730::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:56:32,752::task::495::TaskManager.Task::(_debug) Task 1f9d21ed-4314-4cfe-a5d0-08192744fa2a: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:56:32,755::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e96292c3-2ce0-4116-ad29-3e2ac479c001`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:56:32,756::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:56:32,757::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:56:32,758::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e96292c3-2ce0-4116-ad29-3e2ac479c001`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:56:32,758::task::495::TaskManager.Task::(_debug) Task 1f9d21ed-4314-4cfe-a5d0-08192744fa2a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:56:32,759::task::495::TaskManager.Task::(_debug) Task 1f9d21ed-4314-4cfe-a5d0-08192744fa2a: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:56:32,760::task::495::TaskManager.Task::(_debug) Task 1f9d21ed-4314-4cfe-a5d0-08192744fa2a: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:56:32,760::task::495::TaskManager.Task::(_debug) Task 1f9d21ed-4314-4cfe-a5d0-08192744fa2a: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:56:32,761::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:56:32,762::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:56:32,763::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:56:32,765::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:56:32,765::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:56:32,766::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:56:32,768::task::495::TaskManager.Task::(_debug) Task 1f9d21ed-4314-4cfe-a5d0-08192744fa2a: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:56:32,769::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:56:33,017::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.07236 s, 14.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:56:35,033::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 03:56:35,298::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:56:35,299::task::495::TaskManager.Task::(_debug) Task fcf219a5-1d54-4f48-912d-0b01bbcdc3d3: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:56:35,302::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9b212ada-16f3-455b-95de-b8db67836f91`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:56:35,303::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:56:35,304::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:56:35,304::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9b212ada-16f3-455b-95de-b8db67836f91`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:56:35,305::task::495::TaskManager.Task::(_debug) Task fcf219a5-1d54-4f48-912d-0b01bbcdc3d3: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:56:35,305::task::495::TaskManager.Task::(_debug) Task fcf219a5-1d54-4f48-912d-0b01bbcdc3d3: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:56:35,306::task::495::TaskManager.Task::(_debug) Task fcf219a5-1d54-4f48-912d-0b01bbcdc3d3: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:56:35,306::task::495::TaskManager.Task::(_debug) Task fcf219a5-1d54-4f48-912d-0b01bbcdc3d3: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:56:35,307::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:56:35,307::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:56:35,308::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:56:35,308::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:56:35,309::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:56:35,309::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:56:35,310::task::495::TaskManager.Task::(_debug) Task fcf219a5-1d54-4f48-912d-0b01bbcdc3d3: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:56:35,310::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:56:35,597::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0638587 s, 16.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:56:37,612::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915085::INFO::2013-11-20 03:56:37,840::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=089b08fc-6544-4235-a061-fb55b09d2715 volUUID=66b599fc-b7b5-4934-a5d5-ef7283460029)
>Thread-15915085::DEBUG::2013-11-20 03:56:37,849::task::495::TaskManager.Task::(_debug) Task 300515f9-b00c-46d8-b2cb-ffbbb2b16462: moving from state init -> state preparing
>Thread-15915085::DEBUG::2013-11-20 03:56:37,852::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4b564af4-b3ff-4fd3-bd42-53c17a8ea9e5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915085::DEBUG::2013-11-20 03:56:37,852::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915085::DEBUG::2013-11-20 03:56:37,855::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915085::DEBUG::2013-11-20 03:56:37,856::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4b564af4-b3ff-4fd3-bd42-53c17a8ea9e5`::Granted request
>Thread-15915085::DEBUG::2013-11-20 03:56:37,859::task::495::TaskManager.Task::(_debug) Task 300515f9-b00c-46d8-b2cb-ffbbb2b16462: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915085::DEBUG::2013-11-20 03:56:37,860::task::495::TaskManager.Task::(_debug) Task 300515f9-b00c-46d8-b2cb-ffbbb2b16462: ref 1 aborting False
>Thread-15915085::DEBUG::2013-11-20 03:56:37,861::task::495::TaskManager.Task::(_debug) Task 300515f9-b00c-46d8-b2cb-ffbbb2b16462: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15915085::DEBUG::2013-11-20 03:56:37,862::task::495::TaskManager.Task::(_debug) Task 300515f9-b00c-46d8-b2cb-ffbbb2b16462: moving from state preparing -> state finished
>Thread-15915085::DEBUG::2013-11-20 03:56:37,863::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915085::DEBUG::2013-11-20 03:56:37,863::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915085::DEBUG::2013-11-20 03:56:37,864::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915085::DEBUG::2013-11-20 03:56:37,865::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915085::DEBUG::2013-11-20 03:56:37,865::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915085::DEBUG::2013-11-20 03:56:37,866::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915085::DEBUG::2013-11-20 03:56:37,867::task::495::TaskManager.Task::(_debug) Task 300515f9-b00c-46d8-b2cb-ffbbb2b16462: ref 0 aborting False
>Thread-15915085::INFO::2013-11-20 03:56:37,867::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15915340::DEBUG::2013-11-20 03:56:37,958::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915340::INFO::2013-11-20 03:56:37,960::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915340::DEBUG::2013-11-20 03:56:37,961::task::495::TaskManager.Task::(_debug) Task fcda5c1e-1e4b-448d-96b1-6dc53537aa41: moving from state init -> state preparing
>Thread-15915340::DEBUG::2013-11-20 03:56:37,962::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915340::DEBUG::2013-11-20 03:56:37,963::task::495::TaskManager.Task::(_debug) Task fcda5c1e-1e4b-448d-96b1-6dc53537aa41: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915340::DEBUG::2013-11-20 03:56:37,964::task::495::TaskManager.Task::(_debug) Task fcda5c1e-1e4b-448d-96b1-6dc53537aa41: moving from state preparing -> state finished
>Thread-15915340::DEBUG::2013-11-20 03:56:37,965::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915340::DEBUG::2013-11-20 03:56:37,966::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915340::DEBUG::2013-11-20 03:56:37,966::task::495::TaskManager.Task::(_debug) Task fcda5c1e-1e4b-448d-96b1-6dc53537aa41: ref 0 aborting False
>Thread-15915340::INFO::2013-11-20 03:56:37,967::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915341::DEBUG::2013-11-20 03:56:37,980::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915341::INFO::2013-11-20 03:56:37,981::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915341::DEBUG::2013-11-20 03:56:37,982::task::495::TaskManager.Task::(_debug) Task ba0ae94e-d315-45d6-b778-04216b8445cf: moving from state init -> state preparing
>Thread-15915341::DEBUG::2013-11-20 03:56:37,988::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0398c275-b1a2-4729-a5c5-451edf9e3943`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915341::DEBUG::2013-11-20 03:56:37,989::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915341::DEBUG::2013-11-20 03:56:37,990::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915341::DEBUG::2013-11-20 03:56:37,990::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0398c275-b1a2-4729-a5c5-451edf9e3943`::Granted request
>Thread-15915341::DEBUG::2013-11-20 03:56:37,992::task::495::TaskManager.Task::(_debug) Task ba0ae94e-d315-45d6-b778-04216b8445cf: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915341::DEBUG::2013-11-20 03:56:37,992::task::495::TaskManager.Task::(_debug) Task ba0ae94e-d315-45d6-b778-04216b8445cf: ref 1 aborting False
>Thread-15915341::WARNING::2013-11-20 03:56:37,995::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915341::DEBUG::2013-11-20 03:56:37,996::task::495::TaskManager.Task::(_debug) Task ba0ae94e-d315-45d6-b778-04216b8445cf: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915341::DEBUG::2013-11-20 03:56:37,997::task::495::TaskManager.Task::(_debug) Task ba0ae94e-d315-45d6-b778-04216b8445cf: moving from state preparing -> state finished
>Thread-15915341::DEBUG::2013-11-20 03:56:37,998::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915341::DEBUG::2013-11-20 03:56:37,999::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915341::DEBUG::2013-11-20 03:56:37,999::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915341::DEBUG::2013-11-20 03:56:38,000::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915341::DEBUG::2013-11-20 03:56:38,001::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915341::DEBUG::2013-11-20 03:56:38,002::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915341::DEBUG::2013-11-20 03:56:38,003::task::495::TaskManager.Task::(_debug) Task ba0ae94e-d315-45d6-b778-04216b8445cf: ref 0 aborting False
>Thread-15915341::INFO::2013-11-20 03:56:38,005::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:56:38,194::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0991516 s, 10.3 MB/s\n'; <rc> = 0
>Thread-15915342::INFO::2013-11-20 03:56:38,891::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915342::DEBUG::2013-11-20 03:56:38,892::task::495::TaskManager.Task::(_debug) Task 527a65bc-22f5-4d32-ac02-2521cda1e415: moving from state init -> state preparing
>Thread-15915342::DEBUG::2013-11-20 03:56:38,893::task::495::TaskManager.Task::(_debug) Task 527a65bc-22f5-4d32-ac02-2521cda1e415: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00211691856384', 'lastCheck': 1384919792.756629, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0156118869781', 'lastCheck': 1384919794.69083, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00160694122314', 'lastCheck': 1384919791.469033, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00424194335938', 'lastCheck': 1384919797.8422699, 'valid': True, 'code': 0}}
>Thread-15915342::DEBUG::2013-11-20 03:56:38,894::task::495::TaskManager.Task::(_debug) Task 527a65bc-22f5-4d32-ac02-2521cda1e415: moving from state preparing -> state finished
>Thread-15915342::DEBUG::2013-11-20 03:56:38,895::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915342::DEBUG::2013-11-20 03:56:38,896::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915342::DEBUG::2013-11-20 03:56:38,897::task::495::TaskManager.Task::(_debug) Task 527a65bc-22f5-4d32-ac02-2521cda1e415: ref 0 aborting False
>Thread-15915342::INFO::2013-11-20 03:56:38,897::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00211691856384', 'lastCheck': 1384919792.756629, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0156118869781', 'lastCheck': 1384919794.69083, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00160694122314', 'lastCheck': 1384919791.469033, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00424194335938', 'lastCheck': 1384919797.8422699, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:56:40,208::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:56:40,750::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.079419 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:56:42,769::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 03:56:43,013::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 03:56:43,017::task::495::TaskManager.Task::(_debug) Task 7b4292aa-cdd2-48de-87e0-9a335941ed6e: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 03:56:43,022::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`491ec0ee-0fb7-4aa5-8b10-3232e7b0d9e7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 03:56:43,024::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 03:56:43,025::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 03:56:43,026::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`491ec0ee-0fb7-4aa5-8b10-3232e7b0d9e7`::Granted request
>Thread-15914453::DEBUG::2013-11-20 03:56:43,027::task::495::TaskManager.Task::(_debug) Task 7b4292aa-cdd2-48de-87e0-9a335941ed6e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 03:56:43,028::task::495::TaskManager.Task::(_debug) Task 7b4292aa-cdd2-48de-87e0-9a335941ed6e: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 03:56:43,029::task::495::TaskManager.Task::(_debug) Task 7b4292aa-cdd2-48de-87e0-9a335941ed6e: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 03:56:43,030::task::495::TaskManager.Task::(_debug) Task 7b4292aa-cdd2-48de-87e0-9a335941ed6e: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 03:56:43,030::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 03:56:43,031::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 03:56:43,033::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 03:56:43,034::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 03:56:43,035::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 03:56:43,036::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 03:56:43,037::task::495::TaskManager.Task::(_debug) Task 7b4292aa-cdd2-48de-87e0-9a335941ed6e: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 03:56:43,038::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 03:56:43,352::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0793962 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:56:45,368::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 03:56:45,631::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 03:56:45,634::task::495::TaskManager.Task::(_debug) Task 36da0fd6-343e-4751-a08f-3c078ac1b642: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 03:56:45,636::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8b043fc7-3ff4-4ef4-b2ec-6e869ee86502`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 03:56:45,637::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 03:56:45,638::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 03:56:45,638::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8b043fc7-3ff4-4ef4-b2ec-6e869ee86502`::Granted request
>Thread-15914435::DEBUG::2013-11-20 03:56:45,639::task::495::TaskManager.Task::(_debug) Task 36da0fd6-343e-4751-a08f-3c078ac1b642: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 03:56:45,640::task::495::TaskManager.Task::(_debug) Task 36da0fd6-343e-4751-a08f-3c078ac1b642: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 03:56:45,640::task::495::TaskManager.Task::(_debug) Task 36da0fd6-343e-4751-a08f-3c078ac1b642: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 03:56:45,641::task::495::TaskManager.Task::(_debug) Task 36da0fd6-343e-4751-a08f-3c078ac1b642: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 03:56:45,641::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 03:56:45,642::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 03:56:45,642::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 03:56:45,643::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 03:56:45,643::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 03:56:45,644::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 03:56:45,644::task::495::TaskManager.Task::(_debug) Task 36da0fd6-343e-4751-a08f-3c078ac1b642: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 03:56:45,644::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 03:56:45,905::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0638688 s, 16.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:56:47,919::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915346::DEBUG::2013-11-20 03:56:48,361::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915346::INFO::2013-11-20 03:56:48,363::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915346::DEBUG::2013-11-20 03:56:48,364::task::495::TaskManager.Task::(_debug) Task a159bbc1-ce14-42ed-bdda-84b19d6e6553: moving from state init -> state preparing
>Thread-15915346::DEBUG::2013-11-20 03:56:48,366::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915346::DEBUG::2013-11-20 03:56:48,366::task::495::TaskManager.Task::(_debug) Task a159bbc1-ce14-42ed-bdda-84b19d6e6553: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915346::DEBUG::2013-11-20 03:56:48,368::task::495::TaskManager.Task::(_debug) Task a159bbc1-ce14-42ed-bdda-84b19d6e6553: moving from state preparing -> state finished
>Thread-15915346::DEBUG::2013-11-20 03:56:48,369::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915346::DEBUG::2013-11-20 03:56:48,370::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915346::DEBUG::2013-11-20 03:56:48,371::task::495::TaskManager.Task::(_debug) Task a159bbc1-ce14-42ed-bdda-84b19d6e6553: ref 0 aborting False
>Thread-15915346::INFO::2013-11-20 03:56:48,372::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915347::DEBUG::2013-11-20 03:56:48,385::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915347::INFO::2013-11-20 03:56:48,387::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915347::DEBUG::2013-11-20 03:56:48,388::task::495::TaskManager.Task::(_debug) Task 49b82600-484c-46ac-959e-b4da264c9646: moving from state init -> state preparing
>Thread-15915347::DEBUG::2013-11-20 03:56:48,391::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e2e0588b-1d90-4a35-ae84-090090c40a82`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915347::DEBUG::2013-11-20 03:56:48,392::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915347::DEBUG::2013-11-20 03:56:48,394::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915347::DEBUG::2013-11-20 03:56:48,395::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e2e0588b-1d90-4a35-ae84-090090c40a82`::Granted request
>Thread-15915347::DEBUG::2013-11-20 03:56:48,397::task::495::TaskManager.Task::(_debug) Task 49b82600-484c-46ac-959e-b4da264c9646: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915347::DEBUG::2013-11-20 03:56:48,399::task::495::TaskManager.Task::(_debug) Task 49b82600-484c-46ac-959e-b4da264c9646: ref 1 aborting False
>Thread-15915347::WARNING::2013-11-20 03:56:48,403::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915347::DEBUG::2013-11-20 03:56:48,405::task::495::TaskManager.Task::(_debug) Task 49b82600-484c-46ac-959e-b4da264c9646: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915347::DEBUG::2013-11-20 03:56:48,406::task::495::TaskManager.Task::(_debug) Task 49b82600-484c-46ac-959e-b4da264c9646: moving from state preparing -> state finished
>Thread-15915347::DEBUG::2013-11-20 03:56:48,407::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915347::DEBUG::2013-11-20 03:56:48,408::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915347::DEBUG::2013-11-20 03:56:48,410::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915347::DEBUG::2013-11-20 03:56:48,411::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915347::DEBUG::2013-11-20 03:56:48,412::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915347::DEBUG::2013-11-20 03:56:48,413::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915347::DEBUG::2013-11-20 03:56:48,415::task::495::TaskManager.Task::(_debug) Task 49b82600-484c-46ac-959e-b4da264c9646: ref 0 aborting False
>Thread-15915347::INFO::2013-11-20 03:56:48,416::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:56:48,551::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.123113 s, 8.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:56:50,568::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 03:56:50,801::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 03:56:50,815::task::495::TaskManager.Task::(_debug) Task fc6889dd-4d37-4fca-9caa-4d4f1eac18cb: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 03:56:50,827::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`aee8af25-15a9-445f-8a9c-4fa0191b4886`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 03:56:50,828::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 03:56:50,830::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 03:56:50,830::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`aee8af25-15a9-445f-8a9c-4fa0191b4886`::Granted request
>Thread-15914519::DEBUG::2013-11-20 03:56:50,832::task::495::TaskManager.Task::(_debug) Task fc6889dd-4d37-4fca-9caa-4d4f1eac18cb: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 03:56:50,833::task::495::TaskManager.Task::(_debug) Task fc6889dd-4d37-4fca-9caa-4d4f1eac18cb: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 03:56:50,834::task::495::TaskManager.Task::(_debug) Task fc6889dd-4d37-4fca-9caa-4d4f1eac18cb: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 03:56:50,835::task::495::TaskManager.Task::(_debug) Task fc6889dd-4d37-4fca-9caa-4d4f1eac18cb: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 03:56:50,835::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 03:56:50,836::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 03:56:50,837::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 03:56:50,837::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 03:56:50,838::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 03:56:50,838::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 03:56:50,839::task::495::TaskManager.Task::(_debug) Task fc6889dd-4d37-4fca-9caa-4d4f1eac18cb: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 03:56:50,839::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 03:56:51,107::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0794067 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:56:53,125::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 03:56:53,395::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:56:53,400::task::495::TaskManager.Task::(_debug) Task 983934cf-23d2-4ad5-8324-bd5ed4b08ee6: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:56:53,404::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fab33dd9-938e-440e-8484-8d1b282c3580`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:56:53,405::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:56:53,406::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:56:53,407::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fab33dd9-938e-440e-8484-8d1b282c3580`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:56:53,408::task::495::TaskManager.Task::(_debug) Task 983934cf-23d2-4ad5-8324-bd5ed4b08ee6: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:56:53,409::task::495::TaskManager.Task::(_debug) Task 983934cf-23d2-4ad5-8324-bd5ed4b08ee6: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:56:53,410::task::495::TaskManager.Task::(_debug) Task 983934cf-23d2-4ad5-8324-bd5ed4b08ee6: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:56:53,412::task::495::TaskManager.Task::(_debug) Task 983934cf-23d2-4ad5-8324-bd5ed4b08ee6: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:56:53,413::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:56:53,414::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:56:53,414::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:56:53,415::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:56:53,416::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:56:53,416::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:56:53,417::task::495::TaskManager.Task::(_debug) Task 983934cf-23d2-4ad5-8324-bd5ed4b08ee6: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:56:53,417::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:56:53,418::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:56:53,419::task::495::TaskManager.Task::(_debug) Task 8d39a14c-d493-4e4c-af00-f5dc21b95640: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:56:53,420::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8115044a-9c0a-459f-90d5-e3480bc2cecc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:56:53,421::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:56:53,421::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:56:53,423::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8115044a-9c0a-459f-90d5-e3480bc2cecc`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:56:53,425::task::495::TaskManager.Task::(_debug) Task 8d39a14c-d493-4e4c-af00-f5dc21b95640: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:56:53,426::task::495::TaskManager.Task::(_debug) Task 8d39a14c-d493-4e4c-af00-f5dc21b95640: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:56:53,427::task::495::TaskManager.Task::(_debug) Task 8d39a14c-d493-4e4c-af00-f5dc21b95640: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:56:53,429::task::495::TaskManager.Task::(_debug) Task 8d39a14c-d493-4e4c-af00-f5dc21b95640: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:56:53,430::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:56:53,431::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:56:53,433::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:56:53,434::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:56:53,435::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:56:53,435::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:56:53,437::task::495::TaskManager.Task::(_debug) Task 8d39a14c-d493-4e4c-af00-f5dc21b95640: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:56:53,438::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:56:53,439::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:56:53,440::task::495::TaskManager.Task::(_debug) Task 42656332-da57-4132-aa52-e0fc32623206: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:56:53,441::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2006ad03-552e-437a-99d4-1ea3650ab14b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:56:53,442::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:56:53,443::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:56:53,443::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2006ad03-552e-437a-99d4-1ea3650ab14b`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:56:53,445::task::495::TaskManager.Task::(_debug) Task 42656332-da57-4132-aa52-e0fc32623206: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:56:53,446::task::495::TaskManager.Task::(_debug) Task 42656332-da57-4132-aa52-e0fc32623206: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:56:53,447::task::495::TaskManager.Task::(_debug) Task 42656332-da57-4132-aa52-e0fc32623206: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:56:53,448::task::495::TaskManager.Task::(_debug) Task 42656332-da57-4132-aa52-e0fc32623206: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:56:53,449::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:56:53,450::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:56:53,451::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:56:53,452::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:56:53,453::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:56:53,454::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:56:53,455::task::495::TaskManager.Task::(_debug) Task 42656332-da57-4132-aa52-e0fc32623206: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:56:53,455::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:56:53,457::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:56:53,457::task::495::TaskManager.Task::(_debug) Task 7049746e-60ff-4e87-9e65-898d8e894290: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:56:53,459::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`1dfef8e7-e78d-4fec-828a-6d4b28577b9e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:56:53,460::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:56:53,461::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:56:53,462::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`1dfef8e7-e78d-4fec-828a-6d4b28577b9e`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:56:53,462::task::495::TaskManager.Task::(_debug) Task 7049746e-60ff-4e87-9e65-898d8e894290: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:56:53,463::task::495::TaskManager.Task::(_debug) Task 7049746e-60ff-4e87-9e65-898d8e894290: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:56:53,464::task::495::TaskManager.Task::(_debug) Task 7049746e-60ff-4e87-9e65-898d8e894290: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:56:53,465::task::495::TaskManager.Task::(_debug) Task 7049746e-60ff-4e87-9e65-898d8e894290: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:56:53,466::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:56:53,466::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:56:53,467::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:56:53,468::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:56:53,469::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:56:53,470::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:56:53,470::task::495::TaskManager.Task::(_debug) Task 7049746e-60ff-4e87-9e65-898d8e894290: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:56:53,471::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:56:53,724::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0758088 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:56:55,738::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915350::INFO::2013-11-20 03:56:56,414::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915350::DEBUG::2013-11-20 03:56:56,417::task::495::TaskManager.Task::(_debug) Task 111d3acf-8a1d-430d-86ec-811f7c6d2994: moving from state init -> state preparing
>Thread-15915350::DEBUG::2013-11-20 03:56:56,418::task::495::TaskManager.Task::(_debug) Task 111d3acf-8a1d-430d-86ec-811f7c6d2994: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00162792205811', 'lastCheck': 1384919813.401747, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00418519973755', 'lastCheck': 1384919815.3534169, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00707101821899', 'lastCheck': 1384919812.162297, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00186800956726', 'lastCheck': 1384919808.148514, 'valid': True, 'code': 0}}
>Thread-15915350::DEBUG::2013-11-20 03:56:56,420::task::495::TaskManager.Task::(_debug) Task 111d3acf-8a1d-430d-86ec-811f7c6d2994: moving from state preparing -> state finished
>Thread-15915350::DEBUG::2013-11-20 03:56:56,422::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915350::DEBUG::2013-11-20 03:56:56,423::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915350::DEBUG::2013-11-20 03:56:56,425::task::495::TaskManager.Task::(_debug) Task 111d3acf-8a1d-430d-86ec-811f7c6d2994: ref 0 aborting False
>Thread-15915350::INFO::2013-11-20 03:56:56,427::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00162792205811', 'lastCheck': 1384919813.401747, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00418519973755', 'lastCheck': 1384919815.3534169, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00707101821899', 'lastCheck': 1384919812.162297, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00186800956726', 'lastCheck': 1384919808.148514, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:56:56,438::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.100899 s, 10.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:56:58,459::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915352::DEBUG::2013-11-20 03:56:58,782::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915352::INFO::2013-11-20 03:56:58,785::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915352::DEBUG::2013-11-20 03:56:58,786::task::495::TaskManager.Task::(_debug) Task 02825aa6-21be-45a4-9107-da32980f407d: moving from state init -> state preparing
>Thread-15915352::DEBUG::2013-11-20 03:56:58,787::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915352::DEBUG::2013-11-20 03:56:58,788::task::495::TaskManager.Task::(_debug) Task 02825aa6-21be-45a4-9107-da32980f407d: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915352::DEBUG::2013-11-20 03:56:58,788::task::495::TaskManager.Task::(_debug) Task 02825aa6-21be-45a4-9107-da32980f407d: moving from state preparing -> state finished
>Thread-15915352::DEBUG::2013-11-20 03:56:58,789::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915352::DEBUG::2013-11-20 03:56:58,790::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915352::DEBUG::2013-11-20 03:56:58,791::task::495::TaskManager.Task::(_debug) Task 02825aa6-21be-45a4-9107-da32980f407d: ref 0 aborting False
>Thread-15915352::INFO::2013-11-20 03:56:58,791::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915353::DEBUG::2013-11-20 03:56:58,797::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915353::INFO::2013-11-20 03:56:58,798::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915353::DEBUG::2013-11-20 03:56:58,799::task::495::TaskManager.Task::(_debug) Task a69e5b2f-bda9-4550-ace3-9dc57215bf0d: moving from state init -> state preparing
>Thread-15915353::DEBUG::2013-11-20 03:56:58,801::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bfe9128b-9ecc-45a4-90c7-822627ba46bd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915353::DEBUG::2013-11-20 03:56:58,802::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915353::DEBUG::2013-11-20 03:56:58,803::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915353::DEBUG::2013-11-20 03:56:58,804::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bfe9128b-9ecc-45a4-90c7-822627ba46bd`::Granted request
>Thread-15915353::DEBUG::2013-11-20 03:56:58,805::task::495::TaskManager.Task::(_debug) Task a69e5b2f-bda9-4550-ace3-9dc57215bf0d: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915353::DEBUG::2013-11-20 03:56:58,806::task::495::TaskManager.Task::(_debug) Task a69e5b2f-bda9-4550-ace3-9dc57215bf0d: ref 1 aborting False
>Thread-15915353::WARNING::2013-11-20 03:56:58,809::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915353::DEBUG::2013-11-20 03:56:58,811::task::495::TaskManager.Task::(_debug) Task a69e5b2f-bda9-4550-ace3-9dc57215bf0d: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915353::DEBUG::2013-11-20 03:56:58,812::task::495::TaskManager.Task::(_debug) Task a69e5b2f-bda9-4550-ace3-9dc57215bf0d: moving from state preparing -> state finished
>Thread-15915353::DEBUG::2013-11-20 03:56:58,813::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915353::DEBUG::2013-11-20 03:56:58,813::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915353::DEBUG::2013-11-20 03:56:58,814::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915353::DEBUG::2013-11-20 03:56:58,815::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915353::DEBUG::2013-11-20 03:56:58,816::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915353::DEBUG::2013-11-20 03:56:58,816::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915353::DEBUG::2013-11-20 03:56:58,817::task::495::TaskManager.Task::(_debug) Task a69e5b2f-bda9-4550-ace3-9dc57215bf0d: ref 0 aborting False
>Thread-15915353::INFO::2013-11-20 03:56:58,818::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:56:59,051::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0872316 s, 11.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:57:01,069::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:57:01,640::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0901373 s, 11.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:57:03,654::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913518::INFO::2013-11-20 03:57:03,894::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:57:03,896::task::495::TaskManager.Task::(_debug) Task dd540816-17c1-495b-bc2c-450a109cdebd: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:57:03,898::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`dc6f43db-277f-41e3-acfa-0f6305839d4c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:57:03,899::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:57:03,900::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:57:03,900::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`dc6f43db-277f-41e3-acfa-0f6305839d4c`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:57:03,902::task::495::TaskManager.Task::(_debug) Task dd540816-17c1-495b-bc2c-450a109cdebd: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:57:03,903::task::495::TaskManager.Task::(_debug) Task dd540816-17c1-495b-bc2c-450a109cdebd: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:57:03,903::task::495::TaskManager.Task::(_debug) Task dd540816-17c1-495b-bc2c-450a109cdebd: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:57:03,904::task::495::TaskManager.Task::(_debug) Task dd540816-17c1-495b-bc2c-450a109cdebd: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:57:03,905::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:57:03,905::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:57:03,906::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:57:03,907::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:57:03,908::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:57:03,908::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:57:03,909::task::495::TaskManager.Task::(_debug) Task dd540816-17c1-495b-bc2c-450a109cdebd: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:57:03,909::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:57:03,910::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:57:03,911::task::495::TaskManager.Task::(_debug) Task 06f917a5-de8f-43f5-8f37-6e8db9abf366: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:57:03,913::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a8ecde84-a4d8-4a03-a1a0-b24b0f04a4ce`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:57:03,913::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:57:03,914::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:57:03,914::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a8ecde84-a4d8-4a03-a1a0-b24b0f04a4ce`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:57:03,915::task::495::TaskManager.Task::(_debug) Task 06f917a5-de8f-43f5-8f37-6e8db9abf366: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:57:03,916::task::495::TaskManager.Task::(_debug) Task 06f917a5-de8f-43f5-8f37-6e8db9abf366: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:57:03,917::task::495::TaskManager.Task::(_debug) Task 06f917a5-de8f-43f5-8f37-6e8db9abf366: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:57:03,918::task::495::TaskManager.Task::(_debug) Task 06f917a5-de8f-43f5-8f37-6e8db9abf366: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:57:03,919::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:57:03,920::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:57:03,921::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:57:03,922::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:57:03,923::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:57:03,923::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:57:03,924::task::495::TaskManager.Task::(_debug) Task 06f917a5-de8f-43f5-8f37-6e8db9abf366: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:57:03,925::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:57:04,219::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0815182 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:57:06,235::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:57:06,819::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0886768 s, 11.5 MB/s\n'; <rc> = 0
>Thread-15913639::INFO::2013-11-20 03:57:08,456::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:57:08,458::task::495::TaskManager.Task::(_debug) Task 03a08573-507f-487b-ae23-71f565c6afa3: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:57:08,460::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8812c5c2-fe92-4154-85b7-2ac9827ee157`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:57:08,461::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:57:08,463::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:57:08,464::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8812c5c2-fe92-4154-85b7-2ac9827ee157`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:57:08,466::task::495::TaskManager.Task::(_debug) Task 03a08573-507f-487b-ae23-71f565c6afa3: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:57:08,466::task::495::TaskManager.Task::(_debug) Task 03a08573-507f-487b-ae23-71f565c6afa3: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:57:08,467::task::495::TaskManager.Task::(_debug) Task 03a08573-507f-487b-ae23-71f565c6afa3: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:57:08,468::task::495::TaskManager.Task::(_debug) Task 03a08573-507f-487b-ae23-71f565c6afa3: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:57:08,469::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:57:08,476::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:57:08,477::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:57:08,478::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:57:08,479::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:57:08,479::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:57:08,480::task::495::TaskManager.Task::(_debug) Task 03a08573-507f-487b-ae23-71f565c6afa3: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:57:08,481::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:57:08,833::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915357::DEBUG::2013-11-20 03:57:09,261::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915357::INFO::2013-11-20 03:57:09,263::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915357::DEBUG::2013-11-20 03:57:09,264::task::495::TaskManager.Task::(_debug) Task 8a03fe67-6513-4238-a0d6-b47a4ba1d168: moving from state init -> state preparing
>Thread-15915357::DEBUG::2013-11-20 03:57:09,266::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915357::DEBUG::2013-11-20 03:57:09,267::task::495::TaskManager.Task::(_debug) Task 8a03fe67-6513-4238-a0d6-b47a4ba1d168: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915357::DEBUG::2013-11-20 03:57:09,268::task::495::TaskManager.Task::(_debug) Task 8a03fe67-6513-4238-a0d6-b47a4ba1d168: moving from state preparing -> state finished
>Thread-15915357::DEBUG::2013-11-20 03:57:09,269::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915357::DEBUG::2013-11-20 03:57:09,270::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915357::DEBUG::2013-11-20 03:57:09,271::task::495::TaskManager.Task::(_debug) Task 8a03fe67-6513-4238-a0d6-b47a4ba1d168: ref 0 aborting False
>Thread-15915357::INFO::2013-11-20 03:57:09,272::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915358::DEBUG::2013-11-20 03:57:09,279::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915358::INFO::2013-11-20 03:57:09,280::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915358::DEBUG::2013-11-20 03:57:09,282::task::495::TaskManager.Task::(_debug) Task 36a3e2de-8f4c-4cdd-ba2a-5a494d1cb67a: moving from state init -> state preparing
>Thread-15915358::DEBUG::2013-11-20 03:57:09,299::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`345f6c05-3322-4670-a215-f1a5da0d04ae`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915358::DEBUG::2013-11-20 03:57:09,301::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915358::DEBUG::2013-11-20 03:57:09,310::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915358::DEBUG::2013-11-20 03:57:09,311::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`345f6c05-3322-4670-a215-f1a5da0d04ae`::Granted request
>Thread-15915358::DEBUG::2013-11-20 03:57:09,313::task::495::TaskManager.Task::(_debug) Task 36a3e2de-8f4c-4cdd-ba2a-5a494d1cb67a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915358::DEBUG::2013-11-20 03:57:09,314::task::495::TaskManager.Task::(_debug) Task 36a3e2de-8f4c-4cdd-ba2a-5a494d1cb67a: ref 1 aborting False
>Thread-15915358::WARNING::2013-11-20 03:57:09,317::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915358::DEBUG::2013-11-20 03:57:09,319::task::495::TaskManager.Task::(_debug) Task 36a3e2de-8f4c-4cdd-ba2a-5a494d1cb67a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915358::DEBUG::2013-11-20 03:57:09,320::task::495::TaskManager.Task::(_debug) Task 36a3e2de-8f4c-4cdd-ba2a-5a494d1cb67a: moving from state preparing -> state finished
>Thread-15915358::DEBUG::2013-11-20 03:57:09,321::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915358::DEBUG::2013-11-20 03:57:09,322::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915358::DEBUG::2013-11-20 03:57:09,323::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915358::DEBUG::2013-11-20 03:57:09,325::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915358::DEBUG::2013-11-20 03:57:09,325::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915358::DEBUG::2013-11-20 03:57:09,326::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915358::DEBUG::2013-11-20 03:57:09,329::task::495::TaskManager.Task::(_debug) Task 36a3e2de-8f4c-4cdd-ba2a-5a494d1cb67a: ref 0 aborting False
>Thread-15915358::INFO::2013-11-20 03:57:09,330::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:57:09,408::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.154669 s, 6.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:57:11,423::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:57:12,016::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0766762 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15915360::INFO::2013-11-20 03:57:12,508::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915360::DEBUG::2013-11-20 03:57:12,509::task::495::TaskManager.Task::(_debug) Task cbab4a44-0e4f-419e-a9eb-bb4cc7022d48: moving from state init -> state preparing
>Thread-15915360::DEBUG::2013-11-20 03:57:12,510::task::495::TaskManager.Task::(_debug) Task cbab4a44-0e4f-419e-a9eb-bb4cc7022d48: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000414848327637', 'lastCheck': 1384919823.421788, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014979839325', 'lastCheck': 1384919825.370796, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0013952255249', 'lastCheck': 1384919822.3568299, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00202608108521', 'lastCheck': 1384919828.457478, 'valid': True, 'code': 0}}
>Thread-15915360::DEBUG::2013-11-20 03:57:12,511::task::495::TaskManager.Task::(_debug) Task cbab4a44-0e4f-419e-a9eb-bb4cc7022d48: moving from state preparing -> state finished
>Thread-15915360::DEBUG::2013-11-20 03:57:12,511::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915360::DEBUG::2013-11-20 03:57:12,512::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915360::DEBUG::2013-11-20 03:57:12,512::task::495::TaskManager.Task::(_debug) Task cbab4a44-0e4f-419e-a9eb-bb4cc7022d48: ref 0 aborting False
>Thread-15915360::INFO::2013-11-20 03:57:12,513::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000414848327637', 'lastCheck': 1384919823.421788, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014979839325', 'lastCheck': 1384919825.370796, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0013952255249', 'lastCheck': 1384919822.3568299, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00202608108521', 'lastCheck': 1384919828.457478, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:57:14,031::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:57:14,634::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.085294 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:57:16,653::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:57:17,276::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0986527 s, 10.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:57:19,292::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915364::DEBUG::2013-11-20 03:57:19,569::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915364::INFO::2013-11-20 03:57:19,572::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915364::DEBUG::2013-11-20 03:57:19,572::task::495::TaskManager.Task::(_debug) Task bbda9365-6198-4c12-b58f-cc4ffc1df28b: moving from state init -> state preparing
>Thread-15915364::DEBUG::2013-11-20 03:57:19,574::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915364::DEBUG::2013-11-20 03:57:19,574::task::495::TaskManager.Task::(_debug) Task bbda9365-6198-4c12-b58f-cc4ffc1df28b: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915364::DEBUG::2013-11-20 03:57:19,584::task::495::TaskManager.Task::(_debug) Task bbda9365-6198-4c12-b58f-cc4ffc1df28b: moving from state preparing -> state finished
>Thread-15915364::DEBUG::2013-11-20 03:57:19,585::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915364::DEBUG::2013-11-20 03:57:19,585::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915364::DEBUG::2013-11-20 03:57:19,586::task::495::TaskManager.Task::(_debug) Task bbda9365-6198-4c12-b58f-cc4ffc1df28b: ref 0 aborting False
>Thread-15915364::INFO::2013-11-20 03:57:19,587::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915365::DEBUG::2013-11-20 03:57:19,593::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915365::INFO::2013-11-20 03:57:19,594::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915365::DEBUG::2013-11-20 03:57:19,595::task::495::TaskManager.Task::(_debug) Task c0d1907f-4a5b-4405-be25-d6d00d358dc3: moving from state init -> state preparing
>Thread-15915365::DEBUG::2013-11-20 03:57:19,597::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`33992b11-4c80-4d83-a636-e2acd61b4fa8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915365::DEBUG::2013-11-20 03:57:19,597::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915365::DEBUG::2013-11-20 03:57:19,599::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915365::DEBUG::2013-11-20 03:57:19,599::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`33992b11-4c80-4d83-a636-e2acd61b4fa8`::Granted request
>Thread-15915365::DEBUG::2013-11-20 03:57:19,601::task::495::TaskManager.Task::(_debug) Task c0d1907f-4a5b-4405-be25-d6d00d358dc3: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915365::DEBUG::2013-11-20 03:57:19,601::task::495::TaskManager.Task::(_debug) Task c0d1907f-4a5b-4405-be25-d6d00d358dc3: ref 1 aborting False
>Thread-15915365::WARNING::2013-11-20 03:57:19,604::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915365::DEBUG::2013-11-20 03:57:19,605::task::495::TaskManager.Task::(_debug) Task c0d1907f-4a5b-4405-be25-d6d00d358dc3: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915365::DEBUG::2013-11-20 03:57:19,610::task::495::TaskManager.Task::(_debug) Task c0d1907f-4a5b-4405-be25-d6d00d358dc3: moving from state preparing -> state finished
>Thread-15915365::DEBUG::2013-11-20 03:57:19,610::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915365::DEBUG::2013-11-20 03:57:19,611::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915365::DEBUG::2013-11-20 03:57:19,612::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915365::DEBUG::2013-11-20 03:57:19,613::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915365::DEBUG::2013-11-20 03:57:19,613::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915365::DEBUG::2013-11-20 03:57:19,614::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915365::DEBUG::2013-11-20 03:57:19,614::task::495::TaskManager.Task::(_debug) Task c0d1907f-4a5b-4405-be25-d6d00d358dc3: ref 0 aborting False
>Thread-15915365::INFO::2013-11-20 03:57:19,615::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:57:19,914::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0844001 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:57:21,933::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:57:22,586::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0904999 s, 11.3 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 03:57:23,659::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-42838::DEBUG::2013-11-20 03:57:23,662::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15913658::DEBUG::2013-11-20 03:57:23,666::task::495::TaskManager.Task::(_debug) Task dfba1170-98f2-4835-aa9a-7d7e3b5b02a1: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:57:23,669::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a2df1831-9e9a-423b-bb9a-61c58c1fcb40`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:57:23,934::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:57:23,943::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:57:23,944::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a2df1831-9e9a-423b-bb9a-61c58c1fcb40`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:57:23,945::task::495::TaskManager.Task::(_debug) Task dfba1170-98f2-4835-aa9a-7d7e3b5b02a1: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:57:23,946::task::495::TaskManager.Task::(_debug) Task dfba1170-98f2-4835-aa9a-7d7e3b5b02a1: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:57:23,947::task::495::TaskManager.Task::(_debug) Task dfba1170-98f2-4835-aa9a-7d7e3b5b02a1: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:57:23,948::task::495::TaskManager.Task::(_debug) Task dfba1170-98f2-4835-aa9a-7d7e3b5b02a1: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:57:23,949::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:57:23,950::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:57:23,950::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:57:23,951::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:57:23,951::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:57:23,952::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:57:23,953::task::495::TaskManager.Task::(_debug) Task dfba1170-98f2-4835-aa9a-7d7e3b5b02a1: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:57:23,954::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-42838::DEBUG::2013-11-20 03:57:24,471::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:57:24,601::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:57:25,200::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0878019 s, 11.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:57:27,211::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:57:27,742::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.054895 s, 18.7 MB/s\n'; <rc> = 0
>Thread-15915368::INFO::2013-11-20 03:57:28,310::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915368::DEBUG::2013-11-20 03:57:28,311::task::495::TaskManager.Task::(_debug) Task 1fc2e5ad-a6a5-40de-9ece-b427fc211a36: moving from state init -> state preparing
>Thread-15915368::DEBUG::2013-11-20 03:57:28,312::task::495::TaskManager.Task::(_debug) Task 1fc2e5ad-a6a5-40de-9ece-b427fc211a36: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000391960144043', 'lastCheck': 1384919844.4780569, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014979839325', 'lastCheck': 1384919845.9242151, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00267195701599', 'lastCheck': 1384919842.907922, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000743865966797', 'lastCheck': 1384919838.6534209, 'valid': True, 'code': 0}}
>Thread-15915368::DEBUG::2013-11-20 03:57:28,313::task::495::TaskManager.Task::(_debug) Task 1fc2e5ad-a6a5-40de-9ece-b427fc211a36: moving from state preparing -> state finished
>Thread-15915368::DEBUG::2013-11-20 03:57:28,314::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915368::DEBUG::2013-11-20 03:57:28,314::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915368::DEBUG::2013-11-20 03:57:28,315::task::495::TaskManager.Task::(_debug) Task 1fc2e5ad-a6a5-40de-9ece-b427fc211a36: ref 0 aborting False
>Thread-15915368::INFO::2013-11-20 03:57:28,316::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000391960144043', 'lastCheck': 1384919844.4780569, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014979839325', 'lastCheck': 1384919845.9242151, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00267195701599', 'lastCheck': 1384919842.907922, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000743865966797', 'lastCheck': 1384919838.6534209, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:57:29,758::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915370::DEBUG::2013-11-20 03:57:30,035::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915370::INFO::2013-11-20 03:57:30,045::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915370::DEBUG::2013-11-20 03:57:30,046::task::495::TaskManager.Task::(_debug) Task e1c8d70c-3e5b-446a-825b-d49ade6c4af2: moving from state init -> state preparing
>Thread-15915370::DEBUG::2013-11-20 03:57:30,047::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915370::DEBUG::2013-11-20 03:57:30,048::task::495::TaskManager.Task::(_debug) Task e1c8d70c-3e5b-446a-825b-d49ade6c4af2: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915370::DEBUG::2013-11-20 03:57:30,049::task::495::TaskManager.Task::(_debug) Task e1c8d70c-3e5b-446a-825b-d49ade6c4af2: moving from state preparing -> state finished
>Thread-15915370::DEBUG::2013-11-20 03:57:30,050::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915370::DEBUG::2013-11-20 03:57:30,051::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915370::DEBUG::2013-11-20 03:57:30,051::task::495::TaskManager.Task::(_debug) Task e1c8d70c-3e5b-446a-825b-d49ade6c4af2: ref 0 aborting False
>Thread-15915370::INFO::2013-11-20 03:57:30,052::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915371::DEBUG::2013-11-20 03:57:30,058::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915371::INFO::2013-11-20 03:57:30,060::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915371::DEBUG::2013-11-20 03:57:30,060::task::495::TaskManager.Task::(_debug) Task 59a8a5df-b292-44ac-804b-fd6573e5aa67: moving from state init -> state preparing
>Thread-15915371::DEBUG::2013-11-20 03:57:30,062::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8a4fa474-c5aa-4020-b921-651b9307b36d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915371::DEBUG::2013-11-20 03:57:30,063::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915371::DEBUG::2013-11-20 03:57:30,064::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915371::DEBUG::2013-11-20 03:57:30,065::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8a4fa474-c5aa-4020-b921-651b9307b36d`::Granted request
>Thread-15915371::DEBUG::2013-11-20 03:57:30,066::task::495::TaskManager.Task::(_debug) Task 59a8a5df-b292-44ac-804b-fd6573e5aa67: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915371::DEBUG::2013-11-20 03:57:30,066::task::495::TaskManager.Task::(_debug) Task 59a8a5df-b292-44ac-804b-fd6573e5aa67: ref 1 aborting False
>Thread-15915371::WARNING::2013-11-20 03:57:30,068::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915371::DEBUG::2013-11-20 03:57:30,068::task::495::TaskManager.Task::(_debug) Task 59a8a5df-b292-44ac-804b-fd6573e5aa67: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915371::DEBUG::2013-11-20 03:57:30,069::task::495::TaskManager.Task::(_debug) Task 59a8a5df-b292-44ac-804b-fd6573e5aa67: moving from state preparing -> state finished
>Thread-15915371::DEBUG::2013-11-20 03:57:30,069::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915371::DEBUG::2013-11-20 03:57:30,070::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915371::DEBUG::2013-11-20 03:57:30,070::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915371::DEBUG::2013-11-20 03:57:30,071::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915371::DEBUG::2013-11-20 03:57:30,071::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915371::DEBUG::2013-11-20 03:57:30,071::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915371::DEBUG::2013-11-20 03:57:30,072::task::495::TaskManager.Task::(_debug) Task 59a8a5df-b292-44ac-804b-fd6573e5aa67: ref 0 aborting False
>Thread-15915371::INFO::2013-11-20 03:57:30,072::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:57:30,344::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.078872 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:57:32,361::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:57:32,951::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0589551 s, 17.4 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 03:57:33,983::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:57:33,984::task::495::TaskManager.Task::(_debug) Task 9853d6eb-2c38-4516-86b9-0f089d5abed3: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:57:33,986::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`656b55bc-841c-4dbb-bcbd-8da6fa89be7e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:57:33,987::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:57:33,988::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:57:33,989::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`656b55bc-841c-4dbb-bcbd-8da6fa89be7e`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:57:33,990::task::495::TaskManager.Task::(_debug) Task 9853d6eb-2c38-4516-86b9-0f089d5abed3: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:57:33,991::task::495::TaskManager.Task::(_debug) Task 9853d6eb-2c38-4516-86b9-0f089d5abed3: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:57:33,992::task::495::TaskManager.Task::(_debug) Task 9853d6eb-2c38-4516-86b9-0f089d5abed3: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:57:33,992::task::495::TaskManager.Task::(_debug) Task 9853d6eb-2c38-4516-86b9-0f089d5abed3: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:57:33,993::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:57:33,993::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:57:33,994::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:57:33,995::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:57:33,995::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:57:33,996::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:57:33,996::task::495::TaskManager.Task::(_debug) Task 9853d6eb-2c38-4516-86b9-0f089d5abed3: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:57:33,997::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:57:34,968::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:57:35,443::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0706373 s, 14.5 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 03:57:36,687::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:57:36,689::task::495::TaskManager.Task::(_debug) Task 7592c776-33ab-4305-b7c3-27b7cc6168ec: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:57:36,692::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`dff738f8-33c8-4c22-afbd-67b5453a1044`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:57:36,693::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:57:36,694::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:57:36,695::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`dff738f8-33c8-4c22-afbd-67b5453a1044`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:57:36,696::task::495::TaskManager.Task::(_debug) Task 7592c776-33ab-4305-b7c3-27b7cc6168ec: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:57:36,697::task::495::TaskManager.Task::(_debug) Task 7592c776-33ab-4305-b7c3-27b7cc6168ec: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:57:36,698::task::495::TaskManager.Task::(_debug) Task 7592c776-33ab-4305-b7c3-27b7cc6168ec: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:57:36,698::task::495::TaskManager.Task::(_debug) Task 7592c776-33ab-4305-b7c3-27b7cc6168ec: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:57:36,699::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:57:36,699::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:57:36,700::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:57:36,701::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:57:36,701::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:57:36,702::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:57:36,703::task::495::TaskManager.Task::(_debug) Task 7592c776-33ab-4305-b7c3-27b7cc6168ec: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:57:36,704::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:57:37,457::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:57:37,981::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0562161 s, 18.2 MB/s\n'; <rc> = 0
>Thread-15915085::INFO::2013-11-20 03:57:38,656::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=089b08fc-6544-4235-a061-fb55b09d2715 volUUID=66b599fc-b7b5-4934-a5d5-ef7283460029)
>Thread-15915085::DEBUG::2013-11-20 03:57:38,662::task::495::TaskManager.Task::(_debug) Task 0910e67e-140a-490f-8ca8-81bd3f021a92: moving from state init -> state preparing
>Thread-15915085::DEBUG::2013-11-20 03:57:38,666::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`918fa62f-db59-49b1-bdf1-d5399076666f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915085::DEBUG::2013-11-20 03:57:38,667::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915085::DEBUG::2013-11-20 03:57:38,668::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915085::DEBUG::2013-11-20 03:57:38,669::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`918fa62f-db59-49b1-bdf1-d5399076666f`::Granted request
>Thread-15915085::DEBUG::2013-11-20 03:57:38,670::task::495::TaskManager.Task::(_debug) Task 0910e67e-140a-490f-8ca8-81bd3f021a92: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915085::DEBUG::2013-11-20 03:57:38,671::task::495::TaskManager.Task::(_debug) Task 0910e67e-140a-490f-8ca8-81bd3f021a92: ref 1 aborting False
>Thread-15915085::DEBUG::2013-11-20 03:57:38,672::task::495::TaskManager.Task::(_debug) Task 0910e67e-140a-490f-8ca8-81bd3f021a92: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15915085::DEBUG::2013-11-20 03:57:38,672::task::495::TaskManager.Task::(_debug) Task 0910e67e-140a-490f-8ca8-81bd3f021a92: moving from state preparing -> state finished
>Thread-15915085::DEBUG::2013-11-20 03:57:38,673::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915085::DEBUG::2013-11-20 03:57:38,674::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915085::DEBUG::2013-11-20 03:57:38,675::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915085::DEBUG::2013-11-20 03:57:38,676::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915085::DEBUG::2013-11-20 03:57:38,676::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915085::DEBUG::2013-11-20 03:57:38,677::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915085::DEBUG::2013-11-20 03:57:38,678::task::495::TaskManager.Task::(_debug) Task 0910e67e-140a-490f-8ca8-81bd3f021a92: ref 0 aborting False
>Thread-15915085::INFO::2013-11-20 03:57:38,679::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Dummy-15912651::DEBUG::2013-11-20 03:57:39,997::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:57:40,414::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.046484 s, 22.0 MB/s\n'; <rc> = 0
>Thread-15915375::DEBUG::2013-11-20 03:57:40,483::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915375::INFO::2013-11-20 03:57:40,485::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915375::DEBUG::2013-11-20 03:57:40,486::task::495::TaskManager.Task::(_debug) Task 162d94f4-ab29-4b3d-a45b-077c108e9092: moving from state init -> state preparing
>Thread-15915375::DEBUG::2013-11-20 03:57:40,487::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915375::DEBUG::2013-11-20 03:57:40,488::task::495::TaskManager.Task::(_debug) Task 162d94f4-ab29-4b3d-a45b-077c108e9092: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915375::DEBUG::2013-11-20 03:57:40,489::task::495::TaskManager.Task::(_debug) Task 162d94f4-ab29-4b3d-a45b-077c108e9092: moving from state preparing -> state finished
>Thread-15915375::DEBUG::2013-11-20 03:57:40,502::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915375::DEBUG::2013-11-20 03:57:40,503::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915375::DEBUG::2013-11-20 03:57:40,504::task::495::TaskManager.Task::(_debug) Task 162d94f4-ab29-4b3d-a45b-077c108e9092: ref 0 aborting False
>Thread-15915375::INFO::2013-11-20 03:57:40,505::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915376::DEBUG::2013-11-20 03:57:40,510::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915376::INFO::2013-11-20 03:57:40,512::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915376::DEBUG::2013-11-20 03:57:40,512::task::495::TaskManager.Task::(_debug) Task 543ba278-b39b-484d-94a4-b426d17336bf: moving from state init -> state preparing
>Thread-15915376::DEBUG::2013-11-20 03:57:40,514::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`48ea0b42-2b84-4c10-9804-b90c7a7371ee`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915376::DEBUG::2013-11-20 03:57:40,514::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915376::DEBUG::2013-11-20 03:57:40,515::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915376::DEBUG::2013-11-20 03:57:40,516::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`48ea0b42-2b84-4c10-9804-b90c7a7371ee`::Granted request
>Thread-15915376::DEBUG::2013-11-20 03:57:40,517::task::495::TaskManager.Task::(_debug) Task 543ba278-b39b-484d-94a4-b426d17336bf: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915376::DEBUG::2013-11-20 03:57:40,518::task::495::TaskManager.Task::(_debug) Task 543ba278-b39b-484d-94a4-b426d17336bf: ref 1 aborting False
>Thread-15915376::WARNING::2013-11-20 03:57:40,520::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915376::DEBUG::2013-11-20 03:57:40,521::task::495::TaskManager.Task::(_debug) Task 543ba278-b39b-484d-94a4-b426d17336bf: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915376::DEBUG::2013-11-20 03:57:40,522::task::495::TaskManager.Task::(_debug) Task 543ba278-b39b-484d-94a4-b426d17336bf: moving from state preparing -> state finished
>Thread-15915376::DEBUG::2013-11-20 03:57:40,522::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915376::DEBUG::2013-11-20 03:57:40,523::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915376::DEBUG::2013-11-20 03:57:40,523::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915376::DEBUG::2013-11-20 03:57:40,524::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915376::DEBUG::2013-11-20 03:57:40,524::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915376::DEBUG::2013-11-20 03:57:40,525::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915376::DEBUG::2013-11-20 03:57:40,525::task::495::TaskManager.Task::(_debug) Task 543ba278-b39b-484d-94a4-b426d17336bf: ref 0 aborting False
>Thread-15915376::INFO::2013-11-20 03:57:40,526::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:57:42,429::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:57:42,961::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0759907 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15914453::INFO::2013-11-20 03:57:43,704::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 03:57:43,707::task::495::TaskManager.Task::(_debug) Task 48aac2a1-a562-43c4-8557-b1992cbcf01f: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 03:57:43,710::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9e0f25af-922a-4212-a8ae-87a4505b0bb6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 03:57:43,712::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 03:57:43,713::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 03:57:43,714::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9e0f25af-922a-4212-a8ae-87a4505b0bb6`::Granted request
>Thread-15914453::DEBUG::2013-11-20 03:57:43,716::task::495::TaskManager.Task::(_debug) Task 48aac2a1-a562-43c4-8557-b1992cbcf01f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 03:57:43,716::task::495::TaskManager.Task::(_debug) Task 48aac2a1-a562-43c4-8557-b1992cbcf01f: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 03:57:43,717::task::495::TaskManager.Task::(_debug) Task 48aac2a1-a562-43c4-8557-b1992cbcf01f: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 03:57:43,718::task::495::TaskManager.Task::(_debug) Task 48aac2a1-a562-43c4-8557-b1992cbcf01f: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 03:57:43,719::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 03:57:43,720::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 03:57:43,721::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 03:57:43,721::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 03:57:43,722::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 03:57:43,723::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 03:57:43,724::task::495::TaskManager.Task::(_debug) Task 48aac2a1-a562-43c4-8557-b1992cbcf01f: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 03:57:43,725::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15915378::INFO::2013-11-20 03:57:43,991::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915378::DEBUG::2013-11-20 03:57:44,002::task::495::TaskManager.Task::(_debug) Task b6559c07-52da-4a0e-b763-b6360e09fe25: moving from state init -> state preparing
>Thread-15915378::DEBUG::2013-11-20 03:57:44,011::task::495::TaskManager.Task::(_debug) Task b6559c07-52da-4a0e-b763-b6360e09fe25: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000332832336426', 'lastCheck': 1384919854.638005, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00147795677185', 'lastCheck': 1384919856.168221, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0191450119019', 'lastCheck': 1384919863.194519, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000440835952759', 'lastCheck': 1384919859.164377, 'valid': True, 'code': 0}}
>Thread-15915378::DEBUG::2013-11-20 03:57:44,012::task::495::TaskManager.Task::(_debug) Task b6559c07-52da-4a0e-b763-b6360e09fe25: moving from state preparing -> state finished
>Thread-15915378::DEBUG::2013-11-20 03:57:44,012::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915378::DEBUG::2013-11-20 03:57:44,021::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915378::DEBUG::2013-11-20 03:57:44,022::task::495::TaskManager.Task::(_debug) Task b6559c07-52da-4a0e-b763-b6360e09fe25: ref 0 aborting False
>Thread-15915378::INFO::2013-11-20 03:57:44,023::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000332832336426', 'lastCheck': 1384919854.638005, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00147795677185', 'lastCheck': 1384919856.168221, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0191450119019', 'lastCheck': 1384919863.194519, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000440835952759', 'lastCheck': 1384919859.164377, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:57:44,975::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:57:45,540::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0801941 s, 12.8 MB/s\n'; <rc> = 0
>Thread-15914435::INFO::2013-11-20 03:57:46,759::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 03:57:46,760::task::495::TaskManager.Task::(_debug) Task 7dbd3d32-0dc6-401f-9972-133daf866c38: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 03:57:46,763::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f97ba37a-ff1f-48a8-b511-0fb8b015bb35`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 03:57:46,764::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 03:57:46,765::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 03:57:46,766::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f97ba37a-ff1f-48a8-b511-0fb8b015bb35`::Granted request
>Thread-15914435::DEBUG::2013-11-20 03:57:46,767::task::495::TaskManager.Task::(_debug) Task 7dbd3d32-0dc6-401f-9972-133daf866c38: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 03:57:46,768::task::495::TaskManager.Task::(_debug) Task 7dbd3d32-0dc6-401f-9972-133daf866c38: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 03:57:46,768::task::495::TaskManager.Task::(_debug) Task 7dbd3d32-0dc6-401f-9972-133daf866c38: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 03:57:46,769::task::495::TaskManager.Task::(_debug) Task 7dbd3d32-0dc6-401f-9972-133daf866c38: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 03:57:46,770::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 03:57:46,770::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 03:57:46,771::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 03:57:46,772::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 03:57:46,772::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 03:57:46,773::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 03:57:46,774::task::495::TaskManager.Task::(_debug) Task 7dbd3d32-0dc6-401f-9972-133daf866c38: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 03:57:46,774::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 03:57:47,554::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:57:48,093::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0641803 s, 16.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:57:50,108::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:57:50,707::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0872362 s, 11.7 MB/s\n'; <rc> = 0
>Thread-15915382::DEBUG::2013-11-20 03:57:50,841::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915382::INFO::2013-11-20 03:57:50,842::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915382::DEBUG::2013-11-20 03:57:50,843::task::495::TaskManager.Task::(_debug) Task 19a8b0cd-df2f-4b15-a1f2-707828c48d9f: moving from state init -> state preparing
>Thread-15915382::DEBUG::2013-11-20 03:57:50,844::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915382::DEBUG::2013-11-20 03:57:50,845::task::495::TaskManager.Task::(_debug) Task 19a8b0cd-df2f-4b15-a1f2-707828c48d9f: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915382::DEBUG::2013-11-20 03:57:50,846::task::495::TaskManager.Task::(_debug) Task 19a8b0cd-df2f-4b15-a1f2-707828c48d9f: moving from state preparing -> state finished
>Thread-15915382::DEBUG::2013-11-20 03:57:50,847::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915382::DEBUG::2013-11-20 03:57:50,847::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915382::DEBUG::2013-11-20 03:57:50,848::task::495::TaskManager.Task::(_debug) Task 19a8b0cd-df2f-4b15-a1f2-707828c48d9f: ref 0 aborting False
>Thread-15915382::INFO::2013-11-20 03:57:50,849::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915383::DEBUG::2013-11-20 03:57:50,855::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915383::INFO::2013-11-20 03:57:50,856::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915383::DEBUG::2013-11-20 03:57:50,856::task::495::TaskManager.Task::(_debug) Task 730f57f1-d9a5-4e8a-b8c3-b00f3f5d826a: moving from state init -> state preparing
>Thread-15915383::DEBUG::2013-11-20 03:57:50,858::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`08dc995e-1d2a-402e-adca-3a6a395bd53a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915383::DEBUG::2013-11-20 03:57:50,858::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915383::DEBUG::2013-11-20 03:57:50,859::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915383::DEBUG::2013-11-20 03:57:50,860::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`08dc995e-1d2a-402e-adca-3a6a395bd53a`::Granted request
>Thread-15915383::DEBUG::2013-11-20 03:57:50,861::task::495::TaskManager.Task::(_debug) Task 730f57f1-d9a5-4e8a-b8c3-b00f3f5d826a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915383::DEBUG::2013-11-20 03:57:50,861::task::495::TaskManager.Task::(_debug) Task 730f57f1-d9a5-4e8a-b8c3-b00f3f5d826a: ref 1 aborting False
>Thread-15915383::WARNING::2013-11-20 03:57:50,863::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915383::DEBUG::2013-11-20 03:57:50,864::task::495::TaskManager.Task::(_debug) Task 730f57f1-d9a5-4e8a-b8c3-b00f3f5d826a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915383::DEBUG::2013-11-20 03:57:50,865::task::495::TaskManager.Task::(_debug) Task 730f57f1-d9a5-4e8a-b8c3-b00f3f5d826a: moving from state preparing -> state finished
>Thread-15915383::DEBUG::2013-11-20 03:57:50,866::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915383::DEBUG::2013-11-20 03:57:50,867::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915383::DEBUG::2013-11-20 03:57:50,867::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915383::DEBUG::2013-11-20 03:57:50,868::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915383::DEBUG::2013-11-20 03:57:50,868::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915383::DEBUG::2013-11-20 03:57:50,869::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915383::DEBUG::2013-11-20 03:57:50,870::task::495::TaskManager.Task::(_debug) Task 730f57f1-d9a5-4e8a-b8c3-b00f3f5d826a: ref 0 aborting False
>Thread-15915383::INFO::2013-11-20 03:57:50,870::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914519::INFO::2013-11-20 03:57:51,249::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 03:57:51,253::task::495::TaskManager.Task::(_debug) Task d637913e-11b2-44bc-b8ad-c08486035af4: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 03:57:51,272::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`22eb6f65-cee4-4663-91a5-0489ddb77aa8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 03:57:51,287::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 03:57:51,289::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 03:57:51,290::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`22eb6f65-cee4-4663-91a5-0489ddb77aa8`::Granted request
>Thread-15914519::DEBUG::2013-11-20 03:57:51,295::task::495::TaskManager.Task::(_debug) Task d637913e-11b2-44bc-b8ad-c08486035af4: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 03:57:51,296::task::495::TaskManager.Task::(_debug) Task d637913e-11b2-44bc-b8ad-c08486035af4: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 03:57:51,298::task::495::TaskManager.Task::(_debug) Task d637913e-11b2-44bc-b8ad-c08486035af4: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 03:57:51,299::task::495::TaskManager.Task::(_debug) Task d637913e-11b2-44bc-b8ad-c08486035af4: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 03:57:51,300::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 03:57:51,301::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 03:57:51,302::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 03:57:51,303::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 03:57:51,304::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 03:57:51,305::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 03:57:51,306::task::495::TaskManager.Task::(_debug) Task d637913e-11b2-44bc-b8ad-c08486035af4: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 03:57:51,307::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 03:57:52,723::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:57:53,297::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0914534 s, 11.2 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 03:57:54,343::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:57:54,347::task::495::TaskManager.Task::(_debug) Task 1d3d45ad-4caa-4ab6-8dec-ae1e334499a3: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:57:54,350::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`0c214f40-059b-460f-8cb4-9a7da4251ca5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:57:54,351::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:57:54,352::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:57:54,352::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`0c214f40-059b-460f-8cb4-9a7da4251ca5`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:57:54,354::task::495::TaskManager.Task::(_debug) Task 1d3d45ad-4caa-4ab6-8dec-ae1e334499a3: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:57:54,355::task::495::TaskManager.Task::(_debug) Task 1d3d45ad-4caa-4ab6-8dec-ae1e334499a3: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:57:54,355::task::495::TaskManager.Task::(_debug) Task 1d3d45ad-4caa-4ab6-8dec-ae1e334499a3: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:57:54,356::task::495::TaskManager.Task::(_debug) Task 1d3d45ad-4caa-4ab6-8dec-ae1e334499a3: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:57:54,357::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:57:54,357::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:57:54,358::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:57:54,358::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:57:54,359::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:57:54,359::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:57:54,360::task::495::TaskManager.Task::(_debug) Task 1d3d45ad-4caa-4ab6-8dec-ae1e334499a3: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:57:54,360::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:57:54,361::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:57:54,361::task::495::TaskManager.Task::(_debug) Task 80fb5bc3-4f38-4a69-9280-d55e630f2eae: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:57:54,362::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`aed572e2-c398-4838-a8f0-db5e4b8e94df`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:57:54,363::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:57:54,363::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:57:54,364::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`aed572e2-c398-4838-a8f0-db5e4b8e94df`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:57:54,365::task::495::TaskManager.Task::(_debug) Task 80fb5bc3-4f38-4a69-9280-d55e630f2eae: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:57:54,365::task::495::TaskManager.Task::(_debug) Task 80fb5bc3-4f38-4a69-9280-d55e630f2eae: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:57:54,366::task::495::TaskManager.Task::(_debug) Task 80fb5bc3-4f38-4a69-9280-d55e630f2eae: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:57:54,367::task::495::TaskManager.Task::(_debug) Task 80fb5bc3-4f38-4a69-9280-d55e630f2eae: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:57:54,367::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:57:54,368::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:57:54,369::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:57:54,369::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:57:54,370::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:57:54,370::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:57:54,371::task::495::TaskManager.Task::(_debug) Task 80fb5bc3-4f38-4a69-9280-d55e630f2eae: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:57:54,371::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:57:54,372::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:57:54,373::task::495::TaskManager.Task::(_debug) Task f097a386-b7fb-443d-b89d-918191f0f292: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:57:54,374::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d3a258cc-7916-432b-8d16-6f7ef60f919e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:57:54,375::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:57:54,375::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:57:54,376::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d3a258cc-7916-432b-8d16-6f7ef60f919e`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:57:54,377::task::495::TaskManager.Task::(_debug) Task f097a386-b7fb-443d-b89d-918191f0f292: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:57:54,377::task::495::TaskManager.Task::(_debug) Task f097a386-b7fb-443d-b89d-918191f0f292: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:57:54,378::task::495::TaskManager.Task::(_debug) Task f097a386-b7fb-443d-b89d-918191f0f292: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:57:54,378::task::495::TaskManager.Task::(_debug) Task f097a386-b7fb-443d-b89d-918191f0f292: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:57:54,379::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:57:54,379::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:57:54,380::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:57:54,381::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:57:54,381::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:57:54,382::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:57:54,382::task::495::TaskManager.Task::(_debug) Task f097a386-b7fb-443d-b89d-918191f0f292: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:57:54,383::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:57:54,384::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:57:54,384::task::495::TaskManager.Task::(_debug) Task 0b1d6416-37f8-4e80-9831-c24ac6e7f865: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:57:54,385::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a20db333-117e-472f-ba89-486abbfb5446`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:57:54,386::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:57:54,387::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:57:54,387::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a20db333-117e-472f-ba89-486abbfb5446`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:57:54,393::task::495::TaskManager.Task::(_debug) Task 0b1d6416-37f8-4e80-9831-c24ac6e7f865: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:57:54,393::task::495::TaskManager.Task::(_debug) Task 0b1d6416-37f8-4e80-9831-c24ac6e7f865: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:57:54,394::task::495::TaskManager.Task::(_debug) Task 0b1d6416-37f8-4e80-9831-c24ac6e7f865: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:57:54,394::task::495::TaskManager.Task::(_debug) Task 0b1d6416-37f8-4e80-9831-c24ac6e7f865: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:57:54,395::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:57:54,395::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:57:54,395::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:57:54,396::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:57:54,396::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:57:54,396::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:57:54,397::task::495::TaskManager.Task::(_debug) Task 0b1d6416-37f8-4e80-9831-c24ac6e7f865: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:57:54,397::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:57:55,313::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:57:55,863::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0751055 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:57:57,877::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:57:58,398::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0759544 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15915386::INFO::2013-11-20 03:57:59,624::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915386::DEBUG::2013-11-20 03:57:59,625::task::495::TaskManager.Task::(_debug) Task 24639621-dd28-4750-810a-ec233ef23a78: moving from state init -> state preparing
>Thread-15915386::DEBUG::2013-11-20 03:57:59,626::task::495::TaskManager.Task::(_debug) Task 24639621-dd28-4750-810a-ec233ef23a78: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000411033630371', 'lastCheck': 1384919874.9399011, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00159788131714', 'lastCheck': 1384919876.549042, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00144290924072', 'lastCheck': 1384919873.348264, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000452995300293', 'lastCheck': 1384919879.546747, 'valid': True, 'code': 0}}
>Thread-15915386::DEBUG::2013-11-20 03:57:59,628::task::495::TaskManager.Task::(_debug) Task 24639621-dd28-4750-810a-ec233ef23a78: moving from state preparing -> state finished
>Thread-15915386::DEBUG::2013-11-20 03:57:59,629::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915386::DEBUG::2013-11-20 03:57:59,629::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915386::DEBUG::2013-11-20 03:57:59,630::task::495::TaskManager.Task::(_debug) Task 24639621-dd28-4750-810a-ec233ef23a78: ref 0 aborting False
>Thread-15915386::INFO::2013-11-20 03:57:59,631::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000411033630371', 'lastCheck': 1384919874.9399011, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00159788131714', 'lastCheck': 1384919876.549042, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00144290924072', 'lastCheck': 1384919873.348264, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000452995300293', 'lastCheck': 1384919879.546747, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:58:00,416::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:58:00,966::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0931422 s, 11.0 MB/s\n'; <rc> = 0
>Thread-15915388::DEBUG::2013-11-20 03:58:01,336::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915388::INFO::2013-11-20 03:58:01,338::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915388::DEBUG::2013-11-20 03:58:01,339::task::495::TaskManager.Task::(_debug) Task 0f41e715-21d6-47b8-8e99-3bbf1ef4d66e: moving from state init -> state preparing
>Thread-15915388::DEBUG::2013-11-20 03:58:01,340::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915388::DEBUG::2013-11-20 03:58:01,340::task::495::TaskManager.Task::(_debug) Task 0f41e715-21d6-47b8-8e99-3bbf1ef4d66e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915388::DEBUG::2013-11-20 03:58:01,341::task::495::TaskManager.Task::(_debug) Task 0f41e715-21d6-47b8-8e99-3bbf1ef4d66e: moving from state preparing -> state finished
>Thread-15915388::DEBUG::2013-11-20 03:58:01,342::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915388::DEBUG::2013-11-20 03:58:01,342::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915388::DEBUG::2013-11-20 03:58:01,343::task::495::TaskManager.Task::(_debug) Task 0f41e715-21d6-47b8-8e99-3bbf1ef4d66e: ref 0 aborting False
>Thread-15915388::INFO::2013-11-20 03:58:01,344::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915389::DEBUG::2013-11-20 03:58:01,349::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915389::INFO::2013-11-20 03:58:01,350::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915389::DEBUG::2013-11-20 03:58:01,350::task::495::TaskManager.Task::(_debug) Task c9fcdbfc-b8f8-4652-8da9-b701060a960a: moving from state init -> state preparing
>Thread-15915389::DEBUG::2013-11-20 03:58:01,352::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ce1eab1f-9ed3-4539-9ea8-fdf39470fb41`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915389::DEBUG::2013-11-20 03:58:01,352::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915389::DEBUG::2013-11-20 03:58:01,353::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915389::DEBUG::2013-11-20 03:58:01,354::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ce1eab1f-9ed3-4539-9ea8-fdf39470fb41`::Granted request
>Thread-15915389::DEBUG::2013-11-20 03:58:01,355::task::495::TaskManager.Task::(_debug) Task c9fcdbfc-b8f8-4652-8da9-b701060a960a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915389::DEBUG::2013-11-20 03:58:01,356::task::495::TaskManager.Task::(_debug) Task c9fcdbfc-b8f8-4652-8da9-b701060a960a: ref 1 aborting False
>Thread-15915389::WARNING::2013-11-20 03:58:01,359::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915389::DEBUG::2013-11-20 03:58:01,360::task::495::TaskManager.Task::(_debug) Task c9fcdbfc-b8f8-4652-8da9-b701060a960a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915389::DEBUG::2013-11-20 03:58:01,361::task::495::TaskManager.Task::(_debug) Task c9fcdbfc-b8f8-4652-8da9-b701060a960a: moving from state preparing -> state finished
>Thread-15915389::DEBUG::2013-11-20 03:58:01,362::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915389::DEBUG::2013-11-20 03:58:01,363::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915389::DEBUG::2013-11-20 03:58:01,364::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915389::DEBUG::2013-11-20 03:58:01,364::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915389::DEBUG::2013-11-20 03:58:01,365::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915389::DEBUG::2013-11-20 03:58:01,366::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915389::DEBUG::2013-11-20 03:58:01,367::task::495::TaskManager.Task::(_debug) Task c9fcdbfc-b8f8-4652-8da9-b701060a960a: ref 0 aborting False
>Thread-15915389::INFO::2013-11-20 03:58:01,368::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:58:02,982::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:58:03,576::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0780505 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:58:05,592::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913518::INFO::2013-11-20 03:58:05,880::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:58:05,882::task::495::TaskManager.Task::(_debug) Task 4c970d8c-ec54-4f34-9672-c8be34bcfb9a: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:58:05,885::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e785394c-f189-4154-867f-32cd08de6f41`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:58:05,886::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:58:05,887::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:58:05,888::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e785394c-f189-4154-867f-32cd08de6f41`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:58:05,889::task::495::TaskManager.Task::(_debug) Task 4c970d8c-ec54-4f34-9672-c8be34bcfb9a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:58:05,890::task::495::TaskManager.Task::(_debug) Task 4c970d8c-ec54-4f34-9672-c8be34bcfb9a: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:58:05,891::task::495::TaskManager.Task::(_debug) Task 4c970d8c-ec54-4f34-9672-c8be34bcfb9a: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:58:05,892::task::495::TaskManager.Task::(_debug) Task 4c970d8c-ec54-4f34-9672-c8be34bcfb9a: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:58:05,892::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:58:05,893::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:58:05,894::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:58:05,894::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:58:05,895::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:58:05,895::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:58:05,896::task::495::TaskManager.Task::(_debug) Task 4c970d8c-ec54-4f34-9672-c8be34bcfb9a: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:58:05,897::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:58:05,898::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:58:05,898::task::495::TaskManager.Task::(_debug) Task 3a188e8e-8e15-4f27-8239-c915f796ec28: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:58:05,899::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`dc56f9cc-537e-4130-b5d0-72136d9811b6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:58:05,900::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:58:05,901::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:58:05,901::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`dc56f9cc-537e-4130-b5d0-72136d9811b6`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:58:05,902::task::495::TaskManager.Task::(_debug) Task 3a188e8e-8e15-4f27-8239-c915f796ec28: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:58:05,903::task::495::TaskManager.Task::(_debug) Task 3a188e8e-8e15-4f27-8239-c915f796ec28: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:58:05,903::task::495::TaskManager.Task::(_debug) Task 3a188e8e-8e15-4f27-8239-c915f796ec28: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:58:05,904::task::495::TaskManager.Task::(_debug) Task 3a188e8e-8e15-4f27-8239-c915f796ec28: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:58:05,905::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:58:05,905::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:58:05,906::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:58:05,907::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:58:05,908::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:58:05,908::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:58:05,909::task::495::TaskManager.Task::(_debug) Task 3a188e8e-8e15-4f27-8239-c915f796ec28: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:58:05,909::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:58:06,121::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0713915 s, 14.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:58:08,140::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 03:58:08,654::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:58:08,656::task::495::TaskManager.Task::(_debug) Task 3706f9e0-eab9-4770-a672-cb8aa9d04ff1: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:58:08,659::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5a998ec6-9f5c-4129-aeb1-12f00d3608e6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:58:08,660::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:58:08,662::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:58:08,663::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5a998ec6-9f5c-4129-aeb1-12f00d3608e6`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:58:08,665::task::495::TaskManager.Task::(_debug) Task 3706f9e0-eab9-4770-a672-cb8aa9d04ff1: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:58:08,666::task::495::TaskManager.Task::(_debug) Task 3706f9e0-eab9-4770-a672-cb8aa9d04ff1: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:58:08,667::task::495::TaskManager.Task::(_debug) Task 3706f9e0-eab9-4770-a672-cb8aa9d04ff1: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:58:08,668::task::495::TaskManager.Task::(_debug) Task 3706f9e0-eab9-4770-a672-cb8aa9d04ff1: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:58:08,670::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:58:08,671::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:58:08,673::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:58:08,677::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:58:08,679::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:58:08,681::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:58:08,683::task::495::TaskManager.Task::(_debug) Task 3706f9e0-eab9-4770-a672-cb8aa9d04ff1: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:58:08,684::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:58:08,696::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0801867 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:58:10,714::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:58:11,263::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0773101 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15915393::DEBUG::2013-11-20 03:58:11,686::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915393::INFO::2013-11-20 03:58:11,687::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915393::DEBUG::2013-11-20 03:58:11,688::task::495::TaskManager.Task::(_debug) Task 23cee54a-c699-4138-a6a7-efeed4bbce90: moving from state init -> state preparing
>Thread-15915393::DEBUG::2013-11-20 03:58:11,690::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915393::DEBUG::2013-11-20 03:58:11,690::task::495::TaskManager.Task::(_debug) Task 23cee54a-c699-4138-a6a7-efeed4bbce90: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915393::DEBUG::2013-11-20 03:58:11,691::task::495::TaskManager.Task::(_debug) Task 23cee54a-c699-4138-a6a7-efeed4bbce90: moving from state preparing -> state finished
>Thread-15915393::DEBUG::2013-11-20 03:58:11,692::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915393::DEBUG::2013-11-20 03:58:11,699::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915393::DEBUG::2013-11-20 03:58:11,700::task::495::TaskManager.Task::(_debug) Task 23cee54a-c699-4138-a6a7-efeed4bbce90: ref 0 aborting False
>Thread-15915393::INFO::2013-11-20 03:58:11,701::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915394::DEBUG::2013-11-20 03:58:11,707::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915394::INFO::2013-11-20 03:58:11,709::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915394::DEBUG::2013-11-20 03:58:11,709::task::495::TaskManager.Task::(_debug) Task 7f5ea15d-735e-49e5-b5c4-5816772f887a: moving from state init -> state preparing
>Thread-15915394::DEBUG::2013-11-20 03:58:11,711::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2d89b56f-e794-4814-9684-5927a0655e5e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915394::DEBUG::2013-11-20 03:58:11,711::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915394::DEBUG::2013-11-20 03:58:11,712::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915394::DEBUG::2013-11-20 03:58:11,713::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2d89b56f-e794-4814-9684-5927a0655e5e`::Granted request
>Thread-15915394::DEBUG::2013-11-20 03:58:11,714::task::495::TaskManager.Task::(_debug) Task 7f5ea15d-735e-49e5-b5c4-5816772f887a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915394::DEBUG::2013-11-20 03:58:11,714::task::495::TaskManager.Task::(_debug) Task 7f5ea15d-735e-49e5-b5c4-5816772f887a: ref 1 aborting False
>Thread-15915394::WARNING::2013-11-20 03:58:11,717::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915394::DEBUG::2013-11-20 03:58:11,718::task::495::TaskManager.Task::(_debug) Task 7f5ea15d-735e-49e5-b5c4-5816772f887a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915394::DEBUG::2013-11-20 03:58:11,719::task::495::TaskManager.Task::(_debug) Task 7f5ea15d-735e-49e5-b5c4-5816772f887a: moving from state preparing -> state finished
>Thread-15915394::DEBUG::2013-11-20 03:58:11,719::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915394::DEBUG::2013-11-20 03:58:11,720::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915394::DEBUG::2013-11-20 03:58:11,721::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915394::DEBUG::2013-11-20 03:58:11,721::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915394::DEBUG::2013-11-20 03:58:11,722::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915394::DEBUG::2013-11-20 03:58:11,723::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915394::DEBUG::2013-11-20 03:58:11,723::task::495::TaskManager.Task::(_debug) Task 7f5ea15d-735e-49e5-b5c4-5816772f887a: ref 0 aborting False
>Thread-15915394::INFO::2013-11-20 03:58:11,724::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:58:13,277::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:58:13,892::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0781295 s, 13.1 MB/s\n'; <rc> = 0
>Thread-15915396::INFO::2013-11-20 03:58:15,364::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915396::DEBUG::2013-11-20 03:58:15,365::task::495::TaskManager.Task::(_debug) Task a6e7e3dd-ddc3-4989-833f-311d22068b3e: moving from state init -> state preparing
>Thread-15915396::DEBUG::2013-11-20 03:58:15,366::task::495::TaskManager.Task::(_debug) Task a6e7e3dd-ddc3-4989-833f-311d22068b3e: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000431060791016', 'lastCheck': 1384919885.2425981, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00945901870728', 'lastCheck': 1384919886.8281159, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00147604942322', 'lastCheck': 1384919893.9389949, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000449180603027', 'lastCheck': 1384919889.824347, 'valid': True, 'code': 0}}
>Thread-15915396::DEBUG::2013-11-20 03:58:15,367::task::495::TaskManager.Task::(_debug) Task a6e7e3dd-ddc3-4989-833f-311d22068b3e: moving from state preparing -> state finished
>Thread-15915396::DEBUG::2013-11-20 03:58:15,368::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915396::DEBUG::2013-11-20 03:58:15,369::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915396::DEBUG::2013-11-20 03:58:15,369::task::495::TaskManager.Task::(_debug) Task a6e7e3dd-ddc3-4989-833f-311d22068b3e: ref 0 aborting False
>Thread-15915396::INFO::2013-11-20 03:58:15,370::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000431060791016', 'lastCheck': 1384919885.2425981, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00945901870728', 'lastCheck': 1384919886.8281159, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00147604942322', 'lastCheck': 1384919893.9389949, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000449180603027', 'lastCheck': 1384919889.824347, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:58:15,914::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:58:16,462::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0777566 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:58:18,480::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:58:19,120::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0945317 s, 10.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:58:21,147::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:58:21,696::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0680647 s, 15.0 MB/s\n'; <rc> = 0
>Thread-15915400::DEBUG::2013-11-20 03:58:22,092::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915400::INFO::2013-11-20 03:58:22,094::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915400::DEBUG::2013-11-20 03:58:22,095::task::495::TaskManager.Task::(_debug) Task 8357ac70-5f5d-441b-90c3-3ee19af1f487: moving from state init -> state preparing
>Thread-15915400::DEBUG::2013-11-20 03:58:22,097::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915400::DEBUG::2013-11-20 03:58:22,097::task::495::TaskManager.Task::(_debug) Task 8357ac70-5f5d-441b-90c3-3ee19af1f487: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915400::DEBUG::2013-11-20 03:58:22,098::task::495::TaskManager.Task::(_debug) Task 8357ac70-5f5d-441b-90c3-3ee19af1f487: moving from state preparing -> state finished
>Thread-15915400::DEBUG::2013-11-20 03:58:22,099::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915400::DEBUG::2013-11-20 03:58:22,100::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915400::DEBUG::2013-11-20 03:58:22,101::task::495::TaskManager.Task::(_debug) Task 8357ac70-5f5d-441b-90c3-3ee19af1f487: ref 0 aborting False
>Thread-15915400::INFO::2013-11-20 03:58:22,102::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915401::DEBUG::2013-11-20 03:58:22,107::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915401::INFO::2013-11-20 03:58:22,111::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915401::DEBUG::2013-11-20 03:58:22,112::task::495::TaskManager.Task::(_debug) Task f02f7c93-d9ca-4605-aa2e-59064975852a: moving from state init -> state preparing
>Thread-15915401::DEBUG::2013-11-20 03:58:22,119::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c6f3c231-d522-4ee1-9d97-dd7c878e0743`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915401::DEBUG::2013-11-20 03:58:22,120::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915401::DEBUG::2013-11-20 03:58:22,121::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915401::DEBUG::2013-11-20 03:58:22,122::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c6f3c231-d522-4ee1-9d97-dd7c878e0743`::Granted request
>Thread-15915401::DEBUG::2013-11-20 03:58:22,123::task::495::TaskManager.Task::(_debug) Task f02f7c93-d9ca-4605-aa2e-59064975852a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915401::DEBUG::2013-11-20 03:58:22,124::task::495::TaskManager.Task::(_debug) Task f02f7c93-d9ca-4605-aa2e-59064975852a: ref 1 aborting False
>Thread-15915401::WARNING::2013-11-20 03:58:22,127::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915401::DEBUG::2013-11-20 03:58:22,128::task::495::TaskManager.Task::(_debug) Task f02f7c93-d9ca-4605-aa2e-59064975852a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915401::DEBUG::2013-11-20 03:58:22,130::task::495::TaskManager.Task::(_debug) Task f02f7c93-d9ca-4605-aa2e-59064975852a: moving from state preparing -> state finished
>Thread-15915401::DEBUG::2013-11-20 03:58:22,130::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915401::DEBUG::2013-11-20 03:58:22,131::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915401::DEBUG::2013-11-20 03:58:22,132::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915401::DEBUG::2013-11-20 03:58:22,133::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915401::DEBUG::2013-11-20 03:58:22,134::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915401::DEBUG::2013-11-20 03:58:22,135::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915401::DEBUG::2013-11-20 03:58:22,136::task::495::TaskManager.Task::(_debug) Task f02f7c93-d9ca-4605-aa2e-59064975852a: ref 0 aborting False
>Thread-15915401::INFO::2013-11-20 03:58:22,137::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:58:23,709::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:58:24,284::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0842564 s, 12.2 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 03:58:24,413::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:58:24,414::task::495::TaskManager.Task::(_debug) Task 46985141-e390-4e3b-8478-cab1cd655bf0: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:58:24,421::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3d181c7b-9b5b-4951-b87e-d93a13c46ddc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:58:24,422::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:58:24,424::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:58:24,425::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3d181c7b-9b5b-4951-b87e-d93a13c46ddc`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:58:24,430::task::495::TaskManager.Task::(_debug) Task 46985141-e390-4e3b-8478-cab1cd655bf0: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:58:24,433::task::495::TaskManager.Task::(_debug) Task 46985141-e390-4e3b-8478-cab1cd655bf0: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:58:24,436::task::495::TaskManager.Task::(_debug) Task 46985141-e390-4e3b-8478-cab1cd655bf0: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:58:24,440::task::495::TaskManager.Task::(_debug) Task 46985141-e390-4e3b-8478-cab1cd655bf0: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:58:24,444::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:58:24,448::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:58:24,472::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:58:24,474::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:58:24,475::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:58:24,475::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:58:24,476::task::495::TaskManager.Task::(_debug) Task 46985141-e390-4e3b-8478-cab1cd655bf0: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:58:24,477::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:58:26,300::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:58:26,840::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0745206 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:58:28,853::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:58:29,422::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.086562 s, 11.8 MB/s\n'; <rc> = 0
>Thread-15915404::INFO::2013-11-20 03:58:31,046::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915404::DEBUG::2013-11-20 03:58:31,047::task::495::TaskManager.Task::(_debug) Task fdaafee9-191b-45f6-a997-d80e5a656905: moving from state init -> state preparing
>Thread-15915404::DEBUG::2013-11-20 03:58:31,049::task::495::TaskManager.Task::(_debug) Task fdaafee9-191b-45f6-a997-d80e5a656905: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000316143035889', 'lastCheck': 1384919905.954869, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00114607810974', 'lastCheck': 1384919907.1328189, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00167608261108', 'lastCheck': 1384919904.1293719, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000484943389893', 'lastCheck': 1384919910.1260231, 'valid': True, 'code': 0}}
>Thread-15915404::DEBUG::2013-11-20 03:58:31,050::task::495::TaskManager.Task::(_debug) Task fdaafee9-191b-45f6-a997-d80e5a656905: moving from state preparing -> state finished
>Thread-15915404::DEBUG::2013-11-20 03:58:31,056::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915404::DEBUG::2013-11-20 03:58:31,057::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915404::DEBUG::2013-11-20 03:58:31,057::task::495::TaskManager.Task::(_debug) Task fdaafee9-191b-45f6-a997-d80e5a656905: ref 0 aborting False
>Thread-15915404::INFO::2013-11-20 03:58:31,058::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000316143035889', 'lastCheck': 1384919905.954869, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00114607810974', 'lastCheck': 1384919907.1328189, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00167608261108', 'lastCheck': 1384919904.1293719, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000484943389893', 'lastCheck': 1384919910.1260231, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:58:31,438::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:58:32,004::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0747049 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15915406::DEBUG::2013-11-20 03:58:32,474::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915406::INFO::2013-11-20 03:58:32,476::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915406::DEBUG::2013-11-20 03:58:32,477::task::495::TaskManager.Task::(_debug) Task 7e1fcec1-c880-4e3b-b69b-a1307ec13004: moving from state init -> state preparing
>Thread-15915406::DEBUG::2013-11-20 03:58:32,479::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915406::DEBUG::2013-11-20 03:58:32,479::task::495::TaskManager.Task::(_debug) Task 7e1fcec1-c880-4e3b-b69b-a1307ec13004: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915406::DEBUG::2013-11-20 03:58:32,480::task::495::TaskManager.Task::(_debug) Task 7e1fcec1-c880-4e3b-b69b-a1307ec13004: moving from state preparing -> state finished
>Thread-15915406::DEBUG::2013-11-20 03:58:32,481::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915406::DEBUG::2013-11-20 03:58:32,482::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915406::DEBUG::2013-11-20 03:58:32,483::task::495::TaskManager.Task::(_debug) Task 7e1fcec1-c880-4e3b-b69b-a1307ec13004: ref 0 aborting False
>Thread-15915406::INFO::2013-11-20 03:58:32,484::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915407::DEBUG::2013-11-20 03:58:32,492::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915407::INFO::2013-11-20 03:58:32,495::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915407::DEBUG::2013-11-20 03:58:32,496::task::495::TaskManager.Task::(_debug) Task 2c8381ee-b8c3-4470-9959-5e7036b0ee1d: moving from state init -> state preparing
>Thread-15915407::DEBUG::2013-11-20 03:58:32,497::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`71c8e4dc-d25d-4f5b-bc2b-2cef25750e51`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915407::DEBUG::2013-11-20 03:58:32,498::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915407::DEBUG::2013-11-20 03:58:32,499::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915407::DEBUG::2013-11-20 03:58:32,499::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`71c8e4dc-d25d-4f5b-bc2b-2cef25750e51`::Granted request
>Thread-15915407::DEBUG::2013-11-20 03:58:32,501::task::495::TaskManager.Task::(_debug) Task 2c8381ee-b8c3-4470-9959-5e7036b0ee1d: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915407::DEBUG::2013-11-20 03:58:32,501::task::495::TaskManager.Task::(_debug) Task 2c8381ee-b8c3-4470-9959-5e7036b0ee1d: ref 1 aborting False
>Thread-15915407::WARNING::2013-11-20 03:58:32,504::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915407::DEBUG::2013-11-20 03:58:32,505::task::495::TaskManager.Task::(_debug) Task 2c8381ee-b8c3-4470-9959-5e7036b0ee1d: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915407::DEBUG::2013-11-20 03:58:32,505::task::495::TaskManager.Task::(_debug) Task 2c8381ee-b8c3-4470-9959-5e7036b0ee1d: moving from state preparing -> state finished
>Thread-15915407::DEBUG::2013-11-20 03:58:32,506::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915407::DEBUG::2013-11-20 03:58:32,507::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915407::DEBUG::2013-11-20 03:58:32,508::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915407::DEBUG::2013-11-20 03:58:32,509::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915407::DEBUG::2013-11-20 03:58:32,510::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915407::DEBUG::2013-11-20 03:58:32,510::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915407::DEBUG::2013-11-20 03:58:32,512::task::495::TaskManager.Task::(_debug) Task 2c8381ee-b8c3-4470-9959-5e7036b0ee1d: ref 0 aborting False
>Thread-15915407::INFO::2013-11-20 03:58:32,513::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:58:34,020::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 03:58:34,497::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:58:34,499::task::495::TaskManager.Task::(_debug) Task ad8a20a9-7a23-4ef8-beef-172c0716262a: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:58:34,515::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b64dae45-8eff-4ead-ab98-de09da9edde4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:58:34,516::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:58:34,518::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:58:34,519::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b64dae45-8eff-4ead-ab98-de09da9edde4`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:58:34,521::task::495::TaskManager.Task::(_debug) Task ad8a20a9-7a23-4ef8-beef-172c0716262a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:58:34,522::task::495::TaskManager.Task::(_debug) Task ad8a20a9-7a23-4ef8-beef-172c0716262a: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:58:34,523::task::495::TaskManager.Task::(_debug) Task ad8a20a9-7a23-4ef8-beef-172c0716262a: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:58:34,524::task::495::TaskManager.Task::(_debug) Task ad8a20a9-7a23-4ef8-beef-172c0716262a: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:58:34,525::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:58:34,526::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:58:34,527::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:58:34,529::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:58:34,529::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:58:34,530::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:58:34,532::task::495::TaskManager.Task::(_debug) Task ad8a20a9-7a23-4ef8-beef-172c0716262a: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:58:34,533::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:58:34,643::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.113361 s, 9.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:58:36,660::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:58:37,230::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0734266 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 03:58:37,703::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:58:37,707::task::495::TaskManager.Task::(_debug) Task daac8a8a-7a11-495d-8561-325d7857ad89: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:58:37,714::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`709acb04-f260-4db1-b8a3-6cd3f726bb1f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:58:37,715::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:58:37,716::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:58:37,716::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`709acb04-f260-4db1-b8a3-6cd3f726bb1f`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:58:37,718::task::495::TaskManager.Task::(_debug) Task daac8a8a-7a11-495d-8561-325d7857ad89: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:58:37,718::task::495::TaskManager.Task::(_debug) Task daac8a8a-7a11-495d-8561-325d7857ad89: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:58:37,719::task::495::TaskManager.Task::(_debug) Task daac8a8a-7a11-495d-8561-325d7857ad89: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:58:37,720::task::495::TaskManager.Task::(_debug) Task daac8a8a-7a11-495d-8561-325d7857ad89: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:58:37,721::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:58:37,721::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:58:37,722::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:58:37,723::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:58:37,723::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:58:37,724::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:58:37,725::task::495::TaskManager.Task::(_debug) Task daac8a8a-7a11-495d-8561-325d7857ad89: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:58:37,725::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:58:39,244::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915085::INFO::2013-11-20 03:58:39,522::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=089b08fc-6544-4235-a061-fb55b09d2715 volUUID=66b599fc-b7b5-4934-a5d5-ef7283460029)
>Thread-15915085::DEBUG::2013-11-20 03:58:39,544::task::495::TaskManager.Task::(_debug) Task a8274516-0b21-40c4-8424-944ce27ab8fd: moving from state init -> state preparing
>Thread-15915085::DEBUG::2013-11-20 03:58:39,550::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0a2f3438-cc5c-421a-8d72-3911d1ca6f0f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915085::DEBUG::2013-11-20 03:58:39,552::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915085::DEBUG::2013-11-20 03:58:39,553::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915085::DEBUG::2013-11-20 03:58:39,553::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0a2f3438-cc5c-421a-8d72-3911d1ca6f0f`::Granted request
>Thread-15915085::DEBUG::2013-11-20 03:58:39,554::task::495::TaskManager.Task::(_debug) Task a8274516-0b21-40c4-8424-944ce27ab8fd: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915085::DEBUG::2013-11-20 03:58:39,555::task::495::TaskManager.Task::(_debug) Task a8274516-0b21-40c4-8424-944ce27ab8fd: ref 1 aborting False
>Thread-15915085::DEBUG::2013-11-20 03:58:39,556::task::495::TaskManager.Task::(_debug) Task a8274516-0b21-40c4-8424-944ce27ab8fd: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15915085::DEBUG::2013-11-20 03:58:39,557::task::495::TaskManager.Task::(_debug) Task a8274516-0b21-40c4-8424-944ce27ab8fd: moving from state preparing -> state finished
>Thread-15915085::DEBUG::2013-11-20 03:58:39,558::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915085::DEBUG::2013-11-20 03:58:39,559::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915085::DEBUG::2013-11-20 03:58:39,560::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915085::DEBUG::2013-11-20 03:58:39,560::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915085::DEBUG::2013-11-20 03:58:39,561::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915085::DEBUG::2013-11-20 03:58:39,561::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915085::DEBUG::2013-11-20 03:58:39,562::task::495::TaskManager.Task::(_debug) Task a8274516-0b21-40c4-8424-944ce27ab8fd: ref 0 aborting False
>Thread-15915085::INFO::2013-11-20 03:58:39,563::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Dummy-15912651::DEBUG::2013-11-20 03:58:39,830::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0861334 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:58:41,849::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:58:42,413::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.098806 s, 10.4 MB/s\n'; <rc> = 0
>Thread-15915411::DEBUG::2013-11-20 03:58:42,962::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915411::INFO::2013-11-20 03:58:42,963::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915411::DEBUG::2013-11-20 03:58:42,964::task::495::TaskManager.Task::(_debug) Task 7b7e575b-6d9d-4681-8099-8dc18b99bd92: moving from state init -> state preparing
>Thread-15915411::DEBUG::2013-11-20 03:58:42,965::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915411::DEBUG::2013-11-20 03:58:42,966::task::495::TaskManager.Task::(_debug) Task 7b7e575b-6d9d-4681-8099-8dc18b99bd92: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915411::DEBUG::2013-11-20 03:58:42,967::task::495::TaskManager.Task::(_debug) Task 7b7e575b-6d9d-4681-8099-8dc18b99bd92: moving from state preparing -> state finished
>Thread-15915411::DEBUG::2013-11-20 03:58:42,967::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915411::DEBUG::2013-11-20 03:58:42,968::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915411::DEBUG::2013-11-20 03:58:42,969::task::495::TaskManager.Task::(_debug) Task 7b7e575b-6d9d-4681-8099-8dc18b99bd92: ref 0 aborting False
>Thread-15915411::INFO::2013-11-20 03:58:42,970::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915412::DEBUG::2013-11-20 03:58:42,976::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915412::INFO::2013-11-20 03:58:42,977::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915412::DEBUG::2013-11-20 03:58:42,978::task::495::TaskManager.Task::(_debug) Task 182f1686-c465-47cf-87b1-949802d125b6: moving from state init -> state preparing
>Thread-15915412::DEBUG::2013-11-20 03:58:42,979::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7d067203-a744-46ef-89db-09ecba06284b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915412::DEBUG::2013-11-20 03:58:42,980::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915412::DEBUG::2013-11-20 03:58:42,981::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915412::DEBUG::2013-11-20 03:58:42,981::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7d067203-a744-46ef-89db-09ecba06284b`::Granted request
>Thread-15915412::DEBUG::2013-11-20 03:58:42,983::task::495::TaskManager.Task::(_debug) Task 182f1686-c465-47cf-87b1-949802d125b6: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915412::DEBUG::2013-11-20 03:58:42,984::task::495::TaskManager.Task::(_debug) Task 182f1686-c465-47cf-87b1-949802d125b6: ref 1 aborting False
>Thread-15915412::WARNING::2013-11-20 03:58:42,987::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915412::DEBUG::2013-11-20 03:58:42,988::task::495::TaskManager.Task::(_debug) Task 182f1686-c465-47cf-87b1-949802d125b6: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915412::DEBUG::2013-11-20 03:58:42,989::task::495::TaskManager.Task::(_debug) Task 182f1686-c465-47cf-87b1-949802d125b6: moving from state preparing -> state finished
>Thread-15915412::DEBUG::2013-11-20 03:58:42,990::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915412::DEBUG::2013-11-20 03:58:42,991::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915412::DEBUG::2013-11-20 03:58:42,991::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915412::DEBUG::2013-11-20 03:58:42,992::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915412::DEBUG::2013-11-20 03:58:42,993::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915412::DEBUG::2013-11-20 03:58:42,994::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915412::DEBUG::2013-11-20 03:58:42,995::task::495::TaskManager.Task::(_debug) Task 182f1686-c465-47cf-87b1-949802d125b6: ref 0 aborting False
>Thread-15915412::INFO::2013-11-20 03:58:42,996::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:58:44,429::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 03:58:44,656::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 03:58:44,661::task::495::TaskManager.Task::(_debug) Task 17a107ed-eca3-4fa4-b8cf-341e4b5a6c99: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 03:58:44,667::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6b0b0e80-2d2b-49d8-9c71-394477b85669`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 03:58:44,669::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 03:58:44,670::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 03:58:44,671::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6b0b0e80-2d2b-49d8-9c71-394477b85669`::Granted request
>Thread-15914453::DEBUG::2013-11-20 03:58:44,672::task::495::TaskManager.Task::(_debug) Task 17a107ed-eca3-4fa4-b8cf-341e4b5a6c99: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 03:58:44,672::task::495::TaskManager.Task::(_debug) Task 17a107ed-eca3-4fa4-b8cf-341e4b5a6c99: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 03:58:44,673::task::495::TaskManager.Task::(_debug) Task 17a107ed-eca3-4fa4-b8cf-341e4b5a6c99: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 03:58:44,674::task::495::TaskManager.Task::(_debug) Task 17a107ed-eca3-4fa4-b8cf-341e4b5a6c99: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 03:58:44,675::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 03:58:44,675::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 03:58:44,676::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 03:58:44,677::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 03:58:44,677::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 03:58:44,678::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 03:58:44,679::task::495::TaskManager.Task::(_debug) Task 17a107ed-eca3-4fa4-b8cf-341e4b5a6c99: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 03:58:44,679::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 03:58:44,950::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0603127 s, 17.0 MB/s\n'; <rc> = 0
>Thread-15915414::INFO::2013-11-20 03:58:46,812::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915414::DEBUG::2013-11-20 03:58:46,814::task::495::TaskManager.Task::(_debug) Task 3fede91f-1dc6-44ad-b933-ca01cca16e4f: moving from state init -> state preparing
>Thread-15915414::DEBUG::2013-11-20 03:58:46,815::task::495::TaskManager.Task::(_debug) Task 3fede91f-1dc6-44ad-b933-ca01cca16e4f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000387191772461', 'lastCheck': 1384919926.6518331, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148415565491', 'lastCheck': 1384919917.266367, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00272798538208', 'lastCheck': 1384919924.671314, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000347852706909', 'lastCheck': 1384919920.4982309, 'valid': True, 'code': 0}}
>Thread-15915414::DEBUG::2013-11-20 03:58:46,816::task::495::TaskManager.Task::(_debug) Task 3fede91f-1dc6-44ad-b933-ca01cca16e4f: moving from state preparing -> state finished
>Thread-15915414::DEBUG::2013-11-20 03:58:46,816::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915414::DEBUG::2013-11-20 03:58:46,817::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915414::DEBUG::2013-11-20 03:58:46,818::task::495::TaskManager.Task::(_debug) Task 3fede91f-1dc6-44ad-b933-ca01cca16e4f: ref 0 aborting False
>Thread-15915414::INFO::2013-11-20 03:58:46,819::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000387191772461', 'lastCheck': 1384919926.6518331, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148415565491', 'lastCheck': 1384919917.266367, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00272798538208', 'lastCheck': 1384919924.671314, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000347852706909', 'lastCheck': 1384919920.4982309, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:58:46,968::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:58:47,490::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0742907 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15914435::INFO::2013-11-20 03:58:47,763::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 03:58:47,764::task::495::TaskManager.Task::(_debug) Task 93142285-e866-4206-8c38-1367b7448db2: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 03:58:47,766::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`134aef55-ed7e-4035-9de3-b7d59ec211de`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 03:58:47,767::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 03:58:47,768::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 03:58:47,768::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`134aef55-ed7e-4035-9de3-b7d59ec211de`::Granted request
>Thread-15914435::DEBUG::2013-11-20 03:58:47,769::task::495::TaskManager.Task::(_debug) Task 93142285-e866-4206-8c38-1367b7448db2: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 03:58:47,770::task::495::TaskManager.Task::(_debug) Task 93142285-e866-4206-8c38-1367b7448db2: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 03:58:47,771::task::495::TaskManager.Task::(_debug) Task 93142285-e866-4206-8c38-1367b7448db2: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 03:58:47,771::task::495::TaskManager.Task::(_debug) Task 93142285-e866-4206-8c38-1367b7448db2: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 03:58:47,772::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 03:58:47,773::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 03:58:47,773::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 03:58:47,774::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 03:58:47,774::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 03:58:47,775::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 03:58:47,776::task::495::TaskManager.Task::(_debug) Task 93142285-e866-4206-8c38-1367b7448db2: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 03:58:47,776::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 03:58:49,507::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:58:50,083::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0793193 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:58:52,097::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 03:58:52,347::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 03:58:52,351::task::495::TaskManager.Task::(_debug) Task 290cc709-50c8-4694-ba88-c3e45fb8eedb: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 03:58:52,356::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2824e38d-011d-4f9d-b0aa-a38f7b985326`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 03:58:52,358::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 03:58:52,360::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 03:58:52,361::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2824e38d-011d-4f9d-b0aa-a38f7b985326`::Granted request
>Thread-15914519::DEBUG::2013-11-20 03:58:52,363::task::495::TaskManager.Task::(_debug) Task 290cc709-50c8-4694-ba88-c3e45fb8eedb: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 03:58:52,364::task::495::TaskManager.Task::(_debug) Task 290cc709-50c8-4694-ba88-c3e45fb8eedb: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 03:58:52,365::task::495::TaskManager.Task::(_debug) Task 290cc709-50c8-4694-ba88-c3e45fb8eedb: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 03:58:52,366::task::495::TaskManager.Task::(_debug) Task 290cc709-50c8-4694-ba88-c3e45fb8eedb: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 03:58:52,368::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 03:58:52,369::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 03:58:52,376::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 03:58:52,377::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 03:58:52,378::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 03:58:52,379::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 03:58:52,380::task::495::TaskManager.Task::(_debug) Task 290cc709-50c8-4694-ba88-c3e45fb8eedb: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 03:58:52,381::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 03:58:52,659::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0550985 s, 18.6 MB/s\n'; <rc> = 0
>Thread-15915417::DEBUG::2013-11-20 03:58:53,277::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915417::INFO::2013-11-20 03:58:53,279::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915417::DEBUG::2013-11-20 03:58:53,280::task::495::TaskManager.Task::(_debug) Task ab815611-665b-434a-9896-ad18c6b7d902: moving from state init -> state preparing
>Thread-15915417::DEBUG::2013-11-20 03:58:53,280::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915417::DEBUG::2013-11-20 03:58:53,281::task::495::TaskManager.Task::(_debug) Task ab815611-665b-434a-9896-ad18c6b7d902: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915417::DEBUG::2013-11-20 03:58:53,282::task::495::TaskManager.Task::(_debug) Task ab815611-665b-434a-9896-ad18c6b7d902: moving from state preparing -> state finished
>Thread-15915417::DEBUG::2013-11-20 03:58:53,283::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915417::DEBUG::2013-11-20 03:58:53,283::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915417::DEBUG::2013-11-20 03:58:53,284::task::495::TaskManager.Task::(_debug) Task ab815611-665b-434a-9896-ad18c6b7d902: ref 0 aborting False
>Thread-15915417::INFO::2013-11-20 03:58:53,284::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915418::DEBUG::2013-11-20 03:58:53,289::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915418::INFO::2013-11-20 03:58:53,290::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915418::DEBUG::2013-11-20 03:58:53,291::task::495::TaskManager.Task::(_debug) Task e8b184cd-1e24-4040-a292-23a10f892a5b: moving from state init -> state preparing
>Thread-15915418::DEBUG::2013-11-20 03:58:53,301::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ceece8e7-15ec-4ac6-920f-d85a3867446e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915418::DEBUG::2013-11-20 03:58:53,302::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915418::DEBUG::2013-11-20 03:58:53,303::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915418::DEBUG::2013-11-20 03:58:53,303::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ceece8e7-15ec-4ac6-920f-d85a3867446e`::Granted request
>Thread-15915418::DEBUG::2013-11-20 03:58:53,304::task::495::TaskManager.Task::(_debug) Task e8b184cd-1e24-4040-a292-23a10f892a5b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915418::DEBUG::2013-11-20 03:58:53,305::task::495::TaskManager.Task::(_debug) Task e8b184cd-1e24-4040-a292-23a10f892a5b: ref 1 aborting False
>Thread-15915418::WARNING::2013-11-20 03:58:53,307::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915418::DEBUG::2013-11-20 03:58:53,308::task::495::TaskManager.Task::(_debug) Task e8b184cd-1e24-4040-a292-23a10f892a5b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915418::DEBUG::2013-11-20 03:58:53,309::task::495::TaskManager.Task::(_debug) Task e8b184cd-1e24-4040-a292-23a10f892a5b: moving from state preparing -> state finished
>Thread-15915418::DEBUG::2013-11-20 03:58:53,310::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915418::DEBUG::2013-11-20 03:58:53,310::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915418::DEBUG::2013-11-20 03:58:53,311::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915418::DEBUG::2013-11-20 03:58:53,312::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915418::DEBUG::2013-11-20 03:58:53,312::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915418::DEBUG::2013-11-20 03:58:53,313::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915418::DEBUG::2013-11-20 03:58:53,314::task::495::TaskManager.Task::(_debug) Task e8b184cd-1e24-4040-a292-23a10f892a5b: ref 0 aborting False
>Thread-15915418::INFO::2013-11-20 03:58:53,314::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:58:54,671::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 03:58:54,914::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:58:54,917::task::495::TaskManager.Task::(_debug) Task cb126d51-2480-4b34-94a3-53255ab0b007: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:58:54,922::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`25709036-5b34-455e-8778-33f2700f12cf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:58:54,926::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:58:54,928::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:58:54,929::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`25709036-5b34-455e-8778-33f2700f12cf`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:58:54,931::task::495::TaskManager.Task::(_debug) Task cb126d51-2480-4b34-94a3-53255ab0b007: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:58:54,932::task::495::TaskManager.Task::(_debug) Task cb126d51-2480-4b34-94a3-53255ab0b007: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:58:54,933::task::495::TaskManager.Task::(_debug) Task cb126d51-2480-4b34-94a3-53255ab0b007: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 03:58:54,934::task::495::TaskManager.Task::(_debug) Task cb126d51-2480-4b34-94a3-53255ab0b007: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:58:54,935::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:58:54,937::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:58:54,938::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:58:54,939::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:58:54,940::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:58:54,941::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:58:54,942::task::495::TaskManager.Task::(_debug) Task cb126d51-2480-4b34-94a3-53255ab0b007: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:58:54,943::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 03:58:54,944::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 03:58:54,945::task::495::TaskManager.Task::(_debug) Task 8e5344f2-761f-4162-a13f-daab6c5703e6: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:58:54,947::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d8363415-523b-43ed-9722-27319fc54d2b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:58:54,947::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:58:54,949::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:58:54,949::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d8363415-523b-43ed-9722-27319fc54d2b`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:58:54,951::task::495::TaskManager.Task::(_debug) Task 8e5344f2-761f-4162-a13f-daab6c5703e6: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:58:54,952::task::495::TaskManager.Task::(_debug) Task 8e5344f2-761f-4162-a13f-daab6c5703e6: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:58:54,953::task::495::TaskManager.Task::(_debug) Task 8e5344f2-761f-4162-a13f-daab6c5703e6: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 03:58:54,954::task::495::TaskManager.Task::(_debug) Task 8e5344f2-761f-4162-a13f-daab6c5703e6: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:58:54,955::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:58:54,955::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:58:54,956::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:58:54,957::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:58:54,958::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:58:54,959::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:58:54,960::task::495::TaskManager.Task::(_debug) Task 8e5344f2-761f-4162-a13f-daab6c5703e6: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:58:54,961::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:58:54,962::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:58:54,963::task::495::TaskManager.Task::(_debug) Task 6172424a-5124-4c1b-aa3f-00cde63068f0: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:58:54,964::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`90ccf093-313c-4b6b-9fe0-6a0b728503ab`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:58:54,965::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:58:54,966::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:58:54,967::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`90ccf093-313c-4b6b-9fe0-6a0b728503ab`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:58:54,968::task::495::TaskManager.Task::(_debug) Task 6172424a-5124-4c1b-aa3f-00cde63068f0: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:58:54,969::task::495::TaskManager.Task::(_debug) Task 6172424a-5124-4c1b-aa3f-00cde63068f0: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:58:54,970::task::495::TaskManager.Task::(_debug) Task 6172424a-5124-4c1b-aa3f-00cde63068f0: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:58:54,971::task::495::TaskManager.Task::(_debug) Task 6172424a-5124-4c1b-aa3f-00cde63068f0: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:58:54,971::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:58:54,972::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:58:54,973::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:58:54,973::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:58:54,974::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:58:54,975::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:58:54,975::task::495::TaskManager.Task::(_debug) Task 6172424a-5124-4c1b-aa3f-00cde63068f0: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:58:54,976::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:58:54,977::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:58:54,978::task::495::TaskManager.Task::(_debug) Task 0be2cf6d-0890-443b-b48d-2b9272e347ee: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:58:54,979::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`ad915658-baab-48c6-9adf-3d04e208015d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:58:54,980::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:58:54,980::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:58:54,981::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`ad915658-baab-48c6-9adf-3d04e208015d`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:58:54,982::task::495::TaskManager.Task::(_debug) Task 0be2cf6d-0890-443b-b48d-2b9272e347ee: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:58:54,983::task::495::TaskManager.Task::(_debug) Task 0be2cf6d-0890-443b-b48d-2b9272e347ee: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:58:54,983::task::495::TaskManager.Task::(_debug) Task 0be2cf6d-0890-443b-b48d-2b9272e347ee: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:58:54,984::task::495::TaskManager.Task::(_debug) Task 0be2cf6d-0890-443b-b48d-2b9272e347ee: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:58:54,985::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:58:54,986::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:58:54,986::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:58:54,987::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:58:54,988::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:58:54,988::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:58:54,989::task::495::TaskManager.Task::(_debug) Task 0be2cf6d-0890-443b-b48d-2b9272e347ee: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:58:54,989::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:58:55,271::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0817271 s, 12.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:58:57,290::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:58:57,845::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0749623 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:58:59,861::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:59:00,479::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0802705 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:59:02,499::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915422::INFO::2013-11-20 03:59:02,807::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915422::DEBUG::2013-11-20 03:59:02,809::task::495::TaskManager.Task::(_debug) Task 9821b464-0eed-4c6b-a02e-6ed01066b67d: moving from state init -> state preparing
>Thread-15915422::DEBUG::2013-11-20 03:59:02,810::task::495::TaskManager.Task::(_debug) Task 9821b464-0eed-4c6b-a02e-6ed01066b67d: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000301122665405', 'lastCheck': 1384919936.907624, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0139470100403', 'lastCheck': 1384919937.927603, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00340580940247', 'lastCheck': 1384919934.929127, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000375032424927', 'lastCheck': 1384919941.1073911, 'valid': True, 'code': 0}}
>Thread-15915422::DEBUG::2013-11-20 03:59:02,811::task::495::TaskManager.Task::(_debug) Task 9821b464-0eed-4c6b-a02e-6ed01066b67d: moving from state preparing -> state finished
>Thread-15915422::DEBUG::2013-11-20 03:59:02,812::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915422::DEBUG::2013-11-20 03:59:02,813::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915422::DEBUG::2013-11-20 03:59:02,813::task::495::TaskManager.Task::(_debug) Task 9821b464-0eed-4c6b-a02e-6ed01066b67d: ref 0 aborting False
>Thread-15915422::INFO::2013-11-20 03:59:02,814::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000301122665405', 'lastCheck': 1384919936.907624, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0139470100403', 'lastCheck': 1384919937.927603, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00340580940247', 'lastCheck': 1384919934.929127, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000375032424927', 'lastCheck': 1384919941.1073911, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:59:03,104::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0795537 s, 12.9 MB/s\n'; <rc> = 0
>Thread-15915424::DEBUG::2013-11-20 03:59:03,891::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915424::INFO::2013-11-20 03:59:03,892::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915424::DEBUG::2013-11-20 03:59:03,893::task::495::TaskManager.Task::(_debug) Task 15c5c564-2f43-4e09-8b6e-7ab0045aef0e: moving from state init -> state preparing
>Thread-15915424::DEBUG::2013-11-20 03:59:03,894::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915424::DEBUG::2013-11-20 03:59:03,895::task::495::TaskManager.Task::(_debug) Task 15c5c564-2f43-4e09-8b6e-7ab0045aef0e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915424::DEBUG::2013-11-20 03:59:03,895::task::495::TaskManager.Task::(_debug) Task 15c5c564-2f43-4e09-8b6e-7ab0045aef0e: moving from state preparing -> state finished
>Thread-15915424::DEBUG::2013-11-20 03:59:03,896::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915424::DEBUG::2013-11-20 03:59:03,897::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915424::DEBUG::2013-11-20 03:59:03,897::task::495::TaskManager.Task::(_debug) Task 15c5c564-2f43-4e09-8b6e-7ab0045aef0e: ref 0 aborting False
>Thread-15915424::INFO::2013-11-20 03:59:03,898::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915425::DEBUG::2013-11-20 03:59:03,902::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915425::INFO::2013-11-20 03:59:03,903::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915425::DEBUG::2013-11-20 03:59:03,904::task::495::TaskManager.Task::(_debug) Task 6d6d7d79-787c-4df4-a1aa-0123a584c2a7: moving from state init -> state preparing
>Thread-15915425::DEBUG::2013-11-20 03:59:03,905::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`47f6161d-bcde-4b79-9908-eb180265baf3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915425::DEBUG::2013-11-20 03:59:03,906::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915425::DEBUG::2013-11-20 03:59:03,907::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915425::DEBUG::2013-11-20 03:59:03,908::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`47f6161d-bcde-4b79-9908-eb180265baf3`::Granted request
>Thread-15915425::DEBUG::2013-11-20 03:59:03,910::task::495::TaskManager.Task::(_debug) Task 6d6d7d79-787c-4df4-a1aa-0123a584c2a7: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915425::DEBUG::2013-11-20 03:59:03,911::task::495::TaskManager.Task::(_debug) Task 6d6d7d79-787c-4df4-a1aa-0123a584c2a7: ref 1 aborting False
>Thread-15915425::WARNING::2013-11-20 03:59:03,914::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915425::DEBUG::2013-11-20 03:59:03,915::task::495::TaskManager.Task::(_debug) Task 6d6d7d79-787c-4df4-a1aa-0123a584c2a7: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915425::DEBUG::2013-11-20 03:59:03,916::task::495::TaskManager.Task::(_debug) Task 6d6d7d79-787c-4df4-a1aa-0123a584c2a7: moving from state preparing -> state finished
>Thread-15915425::DEBUG::2013-11-20 03:59:03,917::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915425::DEBUG::2013-11-20 03:59:03,918::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915425::DEBUG::2013-11-20 03:59:03,919::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915425::DEBUG::2013-11-20 03:59:03,920::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915425::DEBUG::2013-11-20 03:59:03,921::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915425::DEBUG::2013-11-20 03:59:03,921::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915425::DEBUG::2013-11-20 03:59:03,922::task::495::TaskManager.Task::(_debug) Task 6d6d7d79-787c-4df4-a1aa-0123a584c2a7: ref 0 aborting False
>Thread-15915425::INFO::2013-11-20 03:59:03,923::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:59:05,125::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:59:05,708::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0769041 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:59:07,721::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913518::INFO::2013-11-20 03:59:08,002::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 03:59:08,003::task::495::TaskManager.Task::(_debug) Task 60d31116-dc09-4999-a47a-3f22bd017ee2: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:59:08,005::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`cb880813-dc72-4965-aa0c-42190572163a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:59:08,006::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:59:08,007::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:59:08,008::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`cb880813-dc72-4965-aa0c-42190572163a`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:59:08,009::task::495::TaskManager.Task::(_debug) Task 60d31116-dc09-4999-a47a-3f22bd017ee2: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:59:08,010::task::495::TaskManager.Task::(_debug) Task 60d31116-dc09-4999-a47a-3f22bd017ee2: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:59:08,011::task::495::TaskManager.Task::(_debug) Task 60d31116-dc09-4999-a47a-3f22bd017ee2: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 03:59:08,011::task::495::TaskManager.Task::(_debug) Task 60d31116-dc09-4999-a47a-3f22bd017ee2: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:59:08,012::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:59:08,012::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:59:08,013::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:59:08,014::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:59:08,014::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:59:08,015::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:59:08,016::task::495::TaskManager.Task::(_debug) Task 60d31116-dc09-4999-a47a-3f22bd017ee2: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:59:08,017::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 03:59:08,018::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 03:59:08,018::task::495::TaskManager.Task::(_debug) Task f8ce0480-bb6a-4d04-9d7c-a4be0bcf316c: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 03:59:08,020::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d90ae0ab-0277-42a8-90da-7eadca139896`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 03:59:08,020::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 03:59:08,021::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 03:59:08,021::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d90ae0ab-0277-42a8-90da-7eadca139896`::Granted request
>Thread-15913518::DEBUG::2013-11-20 03:59:08,023::task::495::TaskManager.Task::(_debug) Task f8ce0480-bb6a-4d04-9d7c-a4be0bcf316c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 03:59:08,024::task::495::TaskManager.Task::(_debug) Task f8ce0480-bb6a-4d04-9d7c-a4be0bcf316c: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 03:59:08,025::task::495::TaskManager.Task::(_debug) Task f8ce0480-bb6a-4d04-9d7c-a4be0bcf316c: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 03:59:08,026::task::495::TaskManager.Task::(_debug) Task f8ce0480-bb6a-4d04-9d7c-a4be0bcf316c: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 03:59:08,026::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 03:59:08,027::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 03:59:08,028::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 03:59:08,029::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 03:59:08,029::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 03:59:08,030::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 03:59:08,031::task::495::TaskManager.Task::(_debug) Task f8ce0480-bb6a-4d04-9d7c-a4be0bcf316c: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 03:59:08,031::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 03:59:08,307::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0993152 s, 10.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:59:10,326::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 03:59:10,525::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 03:59:10,530::task::495::TaskManager.Task::(_debug) Task fac99e0e-870b-4fc5-ad4e-6257943322f0: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 03:59:10,542::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5903ee72-b386-42eb-9943-2ef23137d51c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 03:59:10,545::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 03:59:10,549::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 03:59:10,551::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5903ee72-b386-42eb-9943-2ef23137d51c`::Granted request
>Thread-15913639::DEBUG::2013-11-20 03:59:10,554::task::495::TaskManager.Task::(_debug) Task fac99e0e-870b-4fc5-ad4e-6257943322f0: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 03:59:10,555::task::495::TaskManager.Task::(_debug) Task fac99e0e-870b-4fc5-ad4e-6257943322f0: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 03:59:10,556::task::495::TaskManager.Task::(_debug) Task fac99e0e-870b-4fc5-ad4e-6257943322f0: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 03:59:10,557::task::495::TaskManager.Task::(_debug) Task fac99e0e-870b-4fc5-ad4e-6257943322f0: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 03:59:10,558::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 03:59:10,559::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 03:59:10,560::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 03:59:10,560::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 03:59:10,561::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 03:59:10,562::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 03:59:10,563::task::495::TaskManager.Task::(_debug) Task fac99e0e-870b-4fc5-ad4e-6257943322f0: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 03:59:10,563::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:59:10,862::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.083037 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:59:12,879::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:59:13,448::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0846571 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15915429::DEBUG::2013-11-20 03:59:14,291::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915429::INFO::2013-11-20 03:59:14,292::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915429::DEBUG::2013-11-20 03:59:14,293::task::495::TaskManager.Task::(_debug) Task e99d6abb-55ba-422a-920b-80aac7248dc9: moving from state init -> state preparing
>Thread-15915429::DEBUG::2013-11-20 03:59:14,294::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915429::DEBUG::2013-11-20 03:59:14,295::task::495::TaskManager.Task::(_debug) Task e99d6abb-55ba-422a-920b-80aac7248dc9: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915429::DEBUG::2013-11-20 03:59:14,296::task::495::TaskManager.Task::(_debug) Task e99d6abb-55ba-422a-920b-80aac7248dc9: moving from state preparing -> state finished
>Thread-15915429::DEBUG::2013-11-20 03:59:14,296::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915429::DEBUG::2013-11-20 03:59:14,297::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915429::DEBUG::2013-11-20 03:59:14,298::task::495::TaskManager.Task::(_debug) Task e99d6abb-55ba-422a-920b-80aac7248dc9: ref 0 aborting False
>Thread-15915429::INFO::2013-11-20 03:59:14,298::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915430::DEBUG::2013-11-20 03:59:14,303::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915430::INFO::2013-11-20 03:59:14,305::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915430::DEBUG::2013-11-20 03:59:14,306::task::495::TaskManager.Task::(_debug) Task 5e61d193-21dd-4dc3-8e54-1d2d2f13556f: moving from state init -> state preparing
>Thread-15915430::DEBUG::2013-11-20 03:59:14,308::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9ef0cedc-1162-466e-a288-c0e40397eb39`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915430::DEBUG::2013-11-20 03:59:14,309::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915430::DEBUG::2013-11-20 03:59:14,310::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915430::DEBUG::2013-11-20 03:59:14,311::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9ef0cedc-1162-466e-a288-c0e40397eb39`::Granted request
>Thread-15915430::DEBUG::2013-11-20 03:59:14,312::task::495::TaskManager.Task::(_debug) Task 5e61d193-21dd-4dc3-8e54-1d2d2f13556f: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915430::DEBUG::2013-11-20 03:59:14,313::task::495::TaskManager.Task::(_debug) Task 5e61d193-21dd-4dc3-8e54-1d2d2f13556f: ref 1 aborting False
>Thread-15915430::WARNING::2013-11-20 03:59:14,315::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915430::DEBUG::2013-11-20 03:59:14,316::task::495::TaskManager.Task::(_debug) Task 5e61d193-21dd-4dc3-8e54-1d2d2f13556f: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915430::DEBUG::2013-11-20 03:59:14,317::task::495::TaskManager.Task::(_debug) Task 5e61d193-21dd-4dc3-8e54-1d2d2f13556f: moving from state preparing -> state finished
>Thread-15915430::DEBUG::2013-11-20 03:59:14,318::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915430::DEBUG::2013-11-20 03:59:14,319::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915430::DEBUG::2013-11-20 03:59:14,320::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915430::DEBUG::2013-11-20 03:59:14,321::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915430::DEBUG::2013-11-20 03:59:14,321::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915430::DEBUG::2013-11-20 03:59:14,322::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915430::DEBUG::2013-11-20 03:59:14,323::task::495::TaskManager.Task::(_debug) Task 5e61d193-21dd-4dc3-8e54-1d2d2f13556f: ref 0 aborting False
>Thread-15915430::INFO::2013-11-20 03:59:14,324::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 03:59:15,460::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:59:16,069::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0850414 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:59:18,084::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:59:18,671::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.099882 s, 10.3 MB/s\n'; <rc> = 0
>Thread-15915432::INFO::2013-11-20 03:59:18,809::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915432::DEBUG::2013-11-20 03:59:18,810::task::495::TaskManager.Task::(_debug) Task a7f6c62a-2239-4450-b5cb-91301171dc13: moving from state init -> state preparing
>Thread-15915432::DEBUG::2013-11-20 03:59:18,811::task::495::TaskManager.Task::(_debug) Task a7f6c62a-2239-4450-b5cb-91301171dc13: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000401973724365', 'lastCheck': 1384919957.7144301, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00190615653992', 'lastCheck': 1384919958.3285739, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0161399841309', 'lastCheck': 1384919955.800168, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000354051589966', 'lastCheck': 1384919951.1175721, 'valid': True, 'code': 0}}
>Thread-15915432::DEBUG::2013-11-20 03:59:18,812::task::495::TaskManager.Task::(_debug) Task a7f6c62a-2239-4450-b5cb-91301171dc13: moving from state preparing -> state finished
>Thread-15915432::DEBUG::2013-11-20 03:59:18,812::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915432::DEBUG::2013-11-20 03:59:18,813::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915432::DEBUG::2013-11-20 03:59:18,814::task::495::TaskManager.Task::(_debug) Task a7f6c62a-2239-4450-b5cb-91301171dc13: ref 0 aborting False
>Thread-15915432::INFO::2013-11-20 03:59:18,814::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000401973724365', 'lastCheck': 1384919957.7144301, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00190615653992', 'lastCheck': 1384919958.3285739, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0161399841309', 'lastCheck': 1384919955.800168, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000354051589966', 'lastCheck': 1384919951.1175721, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:59:20,686::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:59:21,347::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.099168 s, 10.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:59:23,369::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:59:23,973::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0830399 s, 12.3 MB/s\n'; <rc> = 0
>Thread-15915435::DEBUG::2013-11-20 03:59:24,833::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915435::INFO::2013-11-20 03:59:24,834::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915435::DEBUG::2013-11-20 03:59:24,835::task::495::TaskManager.Task::(_debug) Task bfacd13f-84b3-47f9-9474-a712289fd1a1: moving from state init -> state preparing
>Thread-15915435::DEBUG::2013-11-20 03:59:24,836::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915435::DEBUG::2013-11-20 03:59:24,837::task::495::TaskManager.Task::(_debug) Task bfacd13f-84b3-47f9-9474-a712289fd1a1: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915435::DEBUG::2013-11-20 03:59:24,837::task::495::TaskManager.Task::(_debug) Task bfacd13f-84b3-47f9-9474-a712289fd1a1: moving from state preparing -> state finished
>Thread-15915435::DEBUG::2013-11-20 03:59:24,838::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915435::DEBUG::2013-11-20 03:59:24,838::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915435::DEBUG::2013-11-20 03:59:24,839::task::495::TaskManager.Task::(_debug) Task bfacd13f-84b3-47f9-9474-a712289fd1a1: ref 0 aborting False
>Thread-15915435::INFO::2013-11-20 03:59:24,840::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915436::DEBUG::2013-11-20 03:59:24,845::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915436::INFO::2013-11-20 03:59:24,846::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915436::DEBUG::2013-11-20 03:59:24,846::task::495::TaskManager.Task::(_debug) Task e0d874fe-7ce8-480e-bc00-8705b49951d2: moving from state init -> state preparing
>Thread-15915436::DEBUG::2013-11-20 03:59:24,848::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e0719616-5796-46df-89c1-607da12b89b6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915436::DEBUG::2013-11-20 03:59:24,848::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915436::DEBUG::2013-11-20 03:59:24,849::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915436::DEBUG::2013-11-20 03:59:24,850::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e0719616-5796-46df-89c1-607da12b89b6`::Granted request
>Thread-15915436::DEBUG::2013-11-20 03:59:24,851::task::495::TaskManager.Task::(_debug) Task e0d874fe-7ce8-480e-bc00-8705b49951d2: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915436::DEBUG::2013-11-20 03:59:24,852::task::495::TaskManager.Task::(_debug) Task e0d874fe-7ce8-480e-bc00-8705b49951d2: ref 1 aborting False
>Thread-15915436::WARNING::2013-11-20 03:59:24,854::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915436::DEBUG::2013-11-20 03:59:24,855::task::495::TaskManager.Task::(_debug) Task e0d874fe-7ce8-480e-bc00-8705b49951d2: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915436::DEBUG::2013-11-20 03:59:24,856::task::495::TaskManager.Task::(_debug) Task e0d874fe-7ce8-480e-bc00-8705b49951d2: moving from state preparing -> state finished
>Thread-15915436::DEBUG::2013-11-20 03:59:24,856::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915436::DEBUG::2013-11-20 03:59:24,857::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915436::DEBUG::2013-11-20 03:59:24,858::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915436::DEBUG::2013-11-20 03:59:24,858::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915436::DEBUG::2013-11-20 03:59:24,859::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915436::DEBUG::2013-11-20 03:59:24,860::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915436::DEBUG::2013-11-20 03:59:24,860::task::495::TaskManager.Task::(_debug) Task e0d874fe-7ce8-480e-bc00-8705b49951d2: ref 0 aborting False
>Thread-15915436::INFO::2013-11-20 03:59:24,861::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913658::INFO::2013-11-20 03:59:25,929::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 03:59:25,933::task::495::TaskManager.Task::(_debug) Task 787c3d09-1694-44bc-be6d-fa5984469f96: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 03:59:25,936::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6ff572c8-e74c-4fe3-b012-c63f375000ed`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 03:59:25,938::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 03:59:25,940::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 03:59:25,941::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6ff572c8-e74c-4fe3-b012-c63f375000ed`::Granted request
>Thread-15913658::DEBUG::2013-11-20 03:59:25,943::task::495::TaskManager.Task::(_debug) Task 787c3d09-1694-44bc-be6d-fa5984469f96: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 03:59:25,944::task::495::TaskManager.Task::(_debug) Task 787c3d09-1694-44bc-be6d-fa5984469f96: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 03:59:25,945::task::495::TaskManager.Task::(_debug) Task 787c3d09-1694-44bc-be6d-fa5984469f96: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 03:59:25,946::task::495::TaskManager.Task::(_debug) Task 787c3d09-1694-44bc-be6d-fa5984469f96: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 03:59:25,947::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 03:59:25,948::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 03:59:25,948::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 03:59:25,949::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 03:59:25,950::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 03:59:25,951::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 03:59:25,952::task::495::TaskManager.Task::(_debug) Task 787c3d09-1694-44bc-be6d-fa5984469f96: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 03:59:25,954::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 03:59:25,988::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:59:26,593::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0963506 s, 10.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:59:28,608::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:59:29,149::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0815944 s, 12.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:59:31,164::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:59:31,713::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0792016 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:59:33,731::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:59:34,278::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0843604 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15915440::INFO::2013-11-20 03:59:34,704::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915440::DEBUG::2013-11-20 03:59:34,705::task::495::TaskManager.Task::(_debug) Task f5abdba6-e4e8-4f82-ad52-a6e448a5bec3: moving from state init -> state preparing
>Thread-15915440::DEBUG::2013-11-20 03:59:34,706::task::495::TaskManager.Task::(_debug) Task f5abdba6-e4e8-4f82-ad52-a6e448a5bec3: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000408172607422', 'lastCheck': 1384919967.921257, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0013120174408', 'lastCheck': 1384919968.3467059, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00197505950928', 'lastCheck': 1384919965.9218619, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000561952590942', 'lastCheck': 1384919971.368665, 'valid': True, 'code': 0}}
>Thread-15915440::DEBUG::2013-11-20 03:59:34,707::task::495::TaskManager.Task::(_debug) Task f5abdba6-e4e8-4f82-ad52-a6e448a5bec3: moving from state preparing -> state finished
>Thread-15915440::DEBUG::2013-11-20 03:59:34,708::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915440::DEBUG::2013-11-20 03:59:34,708::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915440::DEBUG::2013-11-20 03:59:34,709::task::495::TaskManager.Task::(_debug) Task f5abdba6-e4e8-4f82-ad52-a6e448a5bec3: ref 0 aborting False
>Thread-15915440::INFO::2013-11-20 03:59:34,710::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000408172607422', 'lastCheck': 1384919967.921257, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0013120174408', 'lastCheck': 1384919968.3467059, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00197505950928', 'lastCheck': 1384919965.9218619, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000561952590942', 'lastCheck': 1384919971.368665, 'valid': True, 'code': 0}}
>Thread-15915442::DEBUG::2013-11-20 03:59:35,292::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915442::INFO::2013-11-20 03:59:35,293::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915442::DEBUG::2013-11-20 03:59:35,294::task::495::TaskManager.Task::(_debug) Task 968ade05-7753-429a-a421-665e0fb41884: moving from state init -> state preparing
>Thread-15915442::DEBUG::2013-11-20 03:59:35,295::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915442::DEBUG::2013-11-20 03:59:35,296::task::495::TaskManager.Task::(_debug) Task 968ade05-7753-429a-a421-665e0fb41884: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915442::DEBUG::2013-11-20 03:59:35,296::task::495::TaskManager.Task::(_debug) Task 968ade05-7753-429a-a421-665e0fb41884: moving from state preparing -> state finished
>Thread-15915442::DEBUG::2013-11-20 03:59:35,297::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915442::DEBUG::2013-11-20 03:59:35,297::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915442::DEBUG::2013-11-20 03:59:35,298::task::495::TaskManager.Task::(_debug) Task 968ade05-7753-429a-a421-665e0fb41884: ref 0 aborting False
>Thread-15915442::INFO::2013-11-20 03:59:35,299::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915443::DEBUG::2013-11-20 03:59:35,303::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915443::INFO::2013-11-20 03:59:35,304::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915443::DEBUG::2013-11-20 03:59:35,305::task::495::TaskManager.Task::(_debug) Task 99f99ece-a32a-4755-85a0-e6d4f4147381: moving from state init -> state preparing
>Thread-15915443::DEBUG::2013-11-20 03:59:35,306::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`869d882e-2a3c-4587-ad1d-c4a184d1c92d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915443::DEBUG::2013-11-20 03:59:35,307::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915443::DEBUG::2013-11-20 03:59:35,308::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915443::DEBUG::2013-11-20 03:59:35,309::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`869d882e-2a3c-4587-ad1d-c4a184d1c92d`::Granted request
>Thread-15915443::DEBUG::2013-11-20 03:59:35,310::task::495::TaskManager.Task::(_debug) Task 99f99ece-a32a-4755-85a0-e6d4f4147381: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915443::DEBUG::2013-11-20 03:59:35,310::task::495::TaskManager.Task::(_debug) Task 99f99ece-a32a-4755-85a0-e6d4f4147381: ref 1 aborting False
>Thread-15915443::WARNING::2013-11-20 03:59:35,313::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915443::DEBUG::2013-11-20 03:59:35,314::task::495::TaskManager.Task::(_debug) Task 99f99ece-a32a-4755-85a0-e6d4f4147381: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915443::DEBUG::2013-11-20 03:59:35,314::task::495::TaskManager.Task::(_debug) Task 99f99ece-a32a-4755-85a0-e6d4f4147381: moving from state preparing -> state finished
>Thread-15915443::DEBUG::2013-11-20 03:59:35,315::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915443::DEBUG::2013-11-20 03:59:35,316::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915443::DEBUG::2013-11-20 03:59:35,316::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915443::DEBUG::2013-11-20 03:59:35,317::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915443::DEBUG::2013-11-20 03:59:35,318::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915443::DEBUG::2013-11-20 03:59:35,318::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915443::DEBUG::2013-11-20 03:59:35,319::task::495::TaskManager.Task::(_debug) Task 99f99ece-a32a-4755-85a0-e6d4f4147381: ref 0 aborting False
>Thread-15915443::INFO::2013-11-20 03:59:35,320::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912392::INFO::2013-11-20 03:59:35,973::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 03:59:35,974::task::495::TaskManager.Task::(_debug) Task 68c62526-35fd-4e71-bfca-235f4fc70319: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 03:59:35,976::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a6cd1b22-7962-4d17-8d67-d9d3da86b6cc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 03:59:35,977::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 03:59:35,978::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 03:59:35,978::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a6cd1b22-7962-4d17-8d67-d9d3da86b6cc`::Granted request
>Thread-15912392::DEBUG::2013-11-20 03:59:35,979::task::495::TaskManager.Task::(_debug) Task 68c62526-35fd-4e71-bfca-235f4fc70319: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 03:59:35,980::task::495::TaskManager.Task::(_debug) Task 68c62526-35fd-4e71-bfca-235f4fc70319: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 03:59:35,981::task::495::TaskManager.Task::(_debug) Task 68c62526-35fd-4e71-bfca-235f4fc70319: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 03:59:35,982::task::495::TaskManager.Task::(_debug) Task 68c62526-35fd-4e71-bfca-235f4fc70319: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 03:59:35,982::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 03:59:35,983::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 03:59:35,984::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 03:59:35,985::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 03:59:35,986::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 03:59:35,987::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 03:59:35,987::task::495::TaskManager.Task::(_debug) Task 68c62526-35fd-4e71-bfca-235f4fc70319: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 03:59:35,988::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 03:59:36,294::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:59:36,838::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0781075 s, 13.1 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 03:59:38,543::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 03:59:38,544::task::495::TaskManager.Task::(_debug) Task 48ad139a-ca97-475d-b4ae-8b59f2830b12: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 03:59:38,546::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b93dd759-40a3-47c0-9de9-9a370bc6f648`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 03:59:38,547::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 03:59:38,548::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 03:59:38,548::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b93dd759-40a3-47c0-9de9-9a370bc6f648`::Granted request
>Thread-15913695::DEBUG::2013-11-20 03:59:38,549::task::495::TaskManager.Task::(_debug) Task 48ad139a-ca97-475d-b4ae-8b59f2830b12: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 03:59:38,550::task::495::TaskManager.Task::(_debug) Task 48ad139a-ca97-475d-b4ae-8b59f2830b12: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 03:59:38,551::task::495::TaskManager.Task::(_debug) Task 48ad139a-ca97-475d-b4ae-8b59f2830b12: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 03:59:38,551::task::495::TaskManager.Task::(_debug) Task 48ad139a-ca97-475d-b4ae-8b59f2830b12: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 03:59:38,552::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 03:59:38,553::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 03:59:38,553::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 03:59:38,554::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 03:59:38,555::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 03:59:38,555::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 03:59:38,556::task::495::TaskManager.Task::(_debug) Task 48ad139a-ca97-475d-b4ae-8b59f2830b12: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 03:59:38,557::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 03:59:38,863::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:59:39,459::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0874276 s, 11.7 MB/s\n'; <rc> = 0
>Thread-15915085::INFO::2013-11-20 03:59:41,093::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=089b08fc-6544-4235-a061-fb55b09d2715 volUUID=66b599fc-b7b5-4934-a5d5-ef7283460029)
>Thread-15915085::DEBUG::2013-11-20 03:59:41,094::task::495::TaskManager.Task::(_debug) Task 51088063-a459-461b-a54b-e42e407aa879: moving from state init -> state preparing
>Thread-15915085::DEBUG::2013-11-20 03:59:41,097::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3be9d3a8-b4b3-4665-aa39-a639886accfc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915085::DEBUG::2013-11-20 03:59:41,099::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915085::DEBUG::2013-11-20 03:59:41,100::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915085::DEBUG::2013-11-20 03:59:41,101::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3be9d3a8-b4b3-4665-aa39-a639886accfc`::Granted request
>Thread-15915085::DEBUG::2013-11-20 03:59:41,104::task::495::TaskManager.Task::(_debug) Task 51088063-a459-461b-a54b-e42e407aa879: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915085::DEBUG::2013-11-20 03:59:41,105::task::495::TaskManager.Task::(_debug) Task 51088063-a459-461b-a54b-e42e407aa879: ref 1 aborting False
>Thread-15915085::DEBUG::2013-11-20 03:59:41,106::task::495::TaskManager.Task::(_debug) Task 51088063-a459-461b-a54b-e42e407aa879: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15915085::DEBUG::2013-11-20 03:59:41,108::task::495::TaskManager.Task::(_debug) Task 51088063-a459-461b-a54b-e42e407aa879: moving from state preparing -> state finished
>Thread-15915085::DEBUG::2013-11-20 03:59:41,109::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915085::DEBUG::2013-11-20 03:59:41,110::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915085::DEBUG::2013-11-20 03:59:41,111::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915085::DEBUG::2013-11-20 03:59:41,113::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915085::DEBUG::2013-11-20 03:59:41,114::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915085::DEBUG::2013-11-20 03:59:41,114::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915085::DEBUG::2013-11-20 03:59:41,116::task::495::TaskManager.Task::(_debug) Task 51088063-a459-461b-a54b-e42e407aa879: ref 0 aborting False
>Thread-15915085::INFO::2013-11-20 03:59:41,117::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Dummy-15912651::DEBUG::2013-11-20 03:59:41,471::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:59:42,035::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0722844 s, 14.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:59:44,050::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:59:44,623::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0823462 s, 12.4 MB/s\n'; <rc> = 0
>Thread-15915447::DEBUG::2013-11-20 03:59:45,657::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915447::INFO::2013-11-20 03:59:45,658::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915447::DEBUG::2013-11-20 03:59:45,659::task::495::TaskManager.Task::(_debug) Task ec1e8739-50e9-4523-b524-e9027948f9e4: moving from state init -> state preparing
>Thread-15915447::DEBUG::2013-11-20 03:59:45,660::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915447::DEBUG::2013-11-20 03:59:45,661::task::495::TaskManager.Task::(_debug) Task ec1e8739-50e9-4523-b524-e9027948f9e4: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915447::DEBUG::2013-11-20 03:59:45,661::task::495::TaskManager.Task::(_debug) Task ec1e8739-50e9-4523-b524-e9027948f9e4: moving from state preparing -> state finished
>Thread-15915447::DEBUG::2013-11-20 03:59:45,662::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915447::DEBUG::2013-11-20 03:59:45,663::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915447::DEBUG::2013-11-20 03:59:45,663::task::495::TaskManager.Task::(_debug) Task ec1e8739-50e9-4523-b524-e9027948f9e4: ref 0 aborting False
>Thread-15915447::INFO::2013-11-20 03:59:45,664::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915448::DEBUG::2013-11-20 03:59:45,669::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915448::INFO::2013-11-20 03:59:45,670::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915448::DEBUG::2013-11-20 03:59:45,670::task::495::TaskManager.Task::(_debug) Task 22f44982-c69c-4431-a01b-012ff1b97022: moving from state init -> state preparing
>Thread-15915448::DEBUG::2013-11-20 03:59:45,672::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`633d1117-b6dc-4caf-a7da-7e9a7b32edaf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915448::DEBUG::2013-11-20 03:59:45,672::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915448::DEBUG::2013-11-20 03:59:45,673::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915448::DEBUG::2013-11-20 03:59:45,674::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`633d1117-b6dc-4caf-a7da-7e9a7b32edaf`::Granted request
>Thread-15915448::DEBUG::2013-11-20 03:59:45,675::task::495::TaskManager.Task::(_debug) Task 22f44982-c69c-4431-a01b-012ff1b97022: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915448::DEBUG::2013-11-20 03:59:45,676::task::495::TaskManager.Task::(_debug) Task 22f44982-c69c-4431-a01b-012ff1b97022: ref 1 aborting False
>Thread-15915448::WARNING::2013-11-20 03:59:45,678::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915448::DEBUG::2013-11-20 03:59:45,679::task::495::TaskManager.Task::(_debug) Task 22f44982-c69c-4431-a01b-012ff1b97022: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915448::DEBUG::2013-11-20 03:59:45,680::task::495::TaskManager.Task::(_debug) Task 22f44982-c69c-4431-a01b-012ff1b97022: moving from state preparing -> state finished
>Thread-15915448::DEBUG::2013-11-20 03:59:45,681::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915448::DEBUG::2013-11-20 03:59:45,682::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915448::DEBUG::2013-11-20 03:59:45,683::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915448::DEBUG::2013-11-20 03:59:45,683::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915448::DEBUG::2013-11-20 03:59:45,684::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915448::DEBUG::2013-11-20 03:59:45,684::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915448::DEBUG::2013-11-20 03:59:45,685::task::495::TaskManager.Task::(_debug) Task 22f44982-c69c-4431-a01b-012ff1b97022: ref 0 aborting False
>Thread-15915448::INFO::2013-11-20 03:59:45,686::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914453::INFO::2013-11-20 03:59:45,993::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 03:59:45,995::task::495::TaskManager.Task::(_debug) Task 4c59cb60-e12f-42df-87fc-eccd133ce552: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 03:59:46,002::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`562e80bf-8042-4f0c-b11a-c375bd9b124e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 03:59:46,007::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 03:59:46,015::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 03:59:46,018::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`562e80bf-8042-4f0c-b11a-c375bd9b124e`::Granted request
>Thread-15914453::DEBUG::2013-11-20 03:59:46,028::task::495::TaskManager.Task::(_debug) Task 4c59cb60-e12f-42df-87fc-eccd133ce552: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 03:59:46,032::task::495::TaskManager.Task::(_debug) Task 4c59cb60-e12f-42df-87fc-eccd133ce552: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 03:59:46,034::task::495::TaskManager.Task::(_debug) Task 4c59cb60-e12f-42df-87fc-eccd133ce552: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 03:59:46,035::task::495::TaskManager.Task::(_debug) Task 4c59cb60-e12f-42df-87fc-eccd133ce552: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 03:59:46,036::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 03:59:46,037::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 03:59:46,037::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 03:59:46,038::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 03:59:46,039::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 03:59:46,039::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 03:59:46,040::task::495::TaskManager.Task::(_debug) Task 4c59cb60-e12f-42df-87fc-eccd133ce552: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 03:59:46,041::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 03:59:46,639::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:59:47,204::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0819637 s, 12.5 MB/s\n'; <rc> = 0
>Thread-15914435::INFO::2013-11-20 03:59:48,600::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 03:59:48,601::task::495::TaskManager.Task::(_debug) Task 65a9b10d-aba1-4a09-9a3c-d021910a763d: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 03:59:48,602::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1b517e04-9497-46b7-bb3d-4bef408f7d8a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 03:59:48,603::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 03:59:48,604::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 03:59:48,604::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1b517e04-9497-46b7-bb3d-4bef408f7d8a`::Granted request
>Thread-15914435::DEBUG::2013-11-20 03:59:48,605::task::495::TaskManager.Task::(_debug) Task 65a9b10d-aba1-4a09-9a3c-d021910a763d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 03:59:48,606::task::495::TaskManager.Task::(_debug) Task 65a9b10d-aba1-4a09-9a3c-d021910a763d: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 03:59:48,607::task::495::TaskManager.Task::(_debug) Task 65a9b10d-aba1-4a09-9a3c-d021910a763d: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 03:59:48,607::task::495::TaskManager.Task::(_debug) Task 65a9b10d-aba1-4a09-9a3c-d021910a763d: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 03:59:48,608::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 03:59:48,609::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 03:59:48,609::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 03:59:48,610::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 03:59:48,611::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 03:59:48,611::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 03:59:48,612::task::495::TaskManager.Task::(_debug) Task 65a9b10d-aba1-4a09-9a3c-d021910a763d: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 03:59:48,613::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 03:59:49,219::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:59:49,805::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0822594 s, 12.4 MB/s\n'; <rc> = 0
>Thread-15915450::INFO::2013-11-20 03:59:50,393::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915450::DEBUG::2013-11-20 03:59:50,395::task::495::TaskManager.Task::(_debug) Task f83c3af8-ce1b-40e8-8dd8-97754293e017: moving from state init -> state preparing
>Thread-15915450::DEBUG::2013-11-20 03:59:50,396::task::495::TaskManager.Task::(_debug) Task f83c3af8-ce1b-40e8-8dd8-97754293e017: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000424861907959', 'lastCheck': 1384919988.259485, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00165891647339', 'lastCheck': 1384919988.873455, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00168514251709', 'lastCheck': 1384919986.2607441, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000644207000732', 'lastCheck': 1384919981.7199199, 'valid': True, 'code': 0}}
>Thread-15915450::DEBUG::2013-11-20 03:59:50,397::task::495::TaskManager.Task::(_debug) Task f83c3af8-ce1b-40e8-8dd8-97754293e017: moving from state preparing -> state finished
>Thread-15915450::DEBUG::2013-11-20 03:59:50,398::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915450::DEBUG::2013-11-20 03:59:50,398::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915450::DEBUG::2013-11-20 03:59:50,399::task::495::TaskManager.Task::(_debug) Task f83c3af8-ce1b-40e8-8dd8-97754293e017: ref 0 aborting False
>Thread-15915450::INFO::2013-11-20 03:59:50,400::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000424861907959', 'lastCheck': 1384919988.259485, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00165891647339', 'lastCheck': 1384919988.873455, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00168514251709', 'lastCheck': 1384919986.2607441, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000644207000732', 'lastCheck': 1384919981.7199199, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 03:59:51,823::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914436::DEBUG::2013-11-20 03:59:52,074::guestIF::97::vm.Vm::(_handleMessage) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::u'None'
>Dummy-15912651::DEBUG::2013-11-20 03:59:52,376::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0802836 s, 12.8 MB/s\n'; <rc> = 0
>Thread-15914519::INFO::2013-11-20 03:59:53,457::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 03:59:53,463::task::495::TaskManager.Task::(_debug) Task 907889cc-0501-4f42-b5c9-469317673799: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 03:59:53,464::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c70da7c3-d5b6-46cc-bb59-2003c887e9ba`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 03:59:53,465::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 03:59:53,466::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 03:59:53,467::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c70da7c3-d5b6-46cc-bb59-2003c887e9ba`::Granted request
>Thread-15914519::DEBUG::2013-11-20 03:59:53,468::task::495::TaskManager.Task::(_debug) Task 907889cc-0501-4f42-b5c9-469317673799: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 03:59:53,469::task::495::TaskManager.Task::(_debug) Task 907889cc-0501-4f42-b5c9-469317673799: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 03:59:53,470::task::495::TaskManager.Task::(_debug) Task 907889cc-0501-4f42-b5c9-469317673799: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 03:59:53,471::task::495::TaskManager.Task::(_debug) Task 907889cc-0501-4f42-b5c9-469317673799: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 03:59:53,472::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 03:59:53,472::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 03:59:53,473::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 03:59:53,474::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 03:59:53,475::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 03:59:53,475::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 03:59:53,476::task::495::TaskManager.Task::(_debug) Task 907889cc-0501-4f42-b5c9-469317673799: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 03:59:53,477::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 03:59:54,391::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914436::DEBUG::2013-11-20 03:59:54,634::guestIF::97::vm.Vm::(_handleMessage) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::u'BRD-KLEE\\administrator'
>Dummy-15912651::DEBUG::2013-11-20 03:59:54,929::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0795689 s, 12.9 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 03:59:56,068::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 03:59:56,070::task::495::TaskManager.Task::(_debug) Task 43289ebf-2b4f-4479-8c1e-7706b7d1ce5b: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:59:56,072::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b207a7c9-7b84-4aa5-a6b9-4440adf826a9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:59:56,075::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:59:56,077::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15915453::DEBUG::2013-11-20 03:59:56,079::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913546::DEBUG::2013-11-20 03:59:56,080::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b207a7c9-7b84-4aa5-a6b9-4440adf826a9`::Granted request
>Thread-15915453::INFO::2013-11-20 03:59:56,083::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915453::DEBUG::2013-11-20 03:59:56,085::task::495::TaskManager.Task::(_debug) Task 23c07516-3ece-4d41-a2c6-91c3433074f9: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:59:56,085::task::495::TaskManager.Task::(_debug) Task 43289ebf-2b4f-4479-8c1e-7706b7d1ce5b: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15915453::DEBUG::2013-11-20 03:59:56,087::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913546::DEBUG::2013-11-20 03:59:56,088::task::495::TaskManager.Task::(_debug) Task 43289ebf-2b4f-4479-8c1e-7706b7d1ce5b: ref 1 aborting False
>Thread-15915453::DEBUG::2013-11-20 03:59:56,089::task::495::TaskManager.Task::(_debug) Task 23c07516-3ece-4d41-a2c6-91c3433074f9: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913546::DEBUG::2013-11-20 03:59:56,091::task::495::TaskManager.Task::(_debug) Task 43289ebf-2b4f-4479-8c1e-7706b7d1ce5b: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15915453::DEBUG::2013-11-20 03:59:56,092::task::495::TaskManager.Task::(_debug) Task 23c07516-3ece-4d41-a2c6-91c3433074f9: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:59:56,092::task::495::TaskManager.Task::(_debug) Task 43289ebf-2b4f-4479-8c1e-7706b7d1ce5b: moving from state preparing -> state finished
>Thread-15915453::DEBUG::2013-11-20 03:59:56,093::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913546::DEBUG::2013-11-20 03:59:56,094::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15915453::DEBUG::2013-11-20 03:59:56,095::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:59:56,096::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915453::DEBUG::2013-11-20 03:59:56,097::task::495::TaskManager.Task::(_debug) Task 23c07516-3ece-4d41-a2c6-91c3433074f9: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:59:56,098::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15915453::INFO::2013-11-20 03:59:56,099::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913546::DEBUG::2013-11-20 03:59:56,100::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:59:56,101::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:59:56,103::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:59:56,104::task::495::TaskManager.Task::(_debug) Task 43289ebf-2b4f-4479-8c1e-7706b7d1ce5b: ref 0 aborting False
>Thread-15915454::DEBUG::2013-11-20 03:59:56,106::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913546::INFO::2013-11-20 03:59:56,106::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15915454::INFO::2013-11-20 03:59:56,107::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913546::INFO::2013-11-20 03:59:56,108::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15915454::DEBUG::2013-11-20 03:59:56,109::task::495::TaskManager.Task::(_debug) Task ed13df98-aef3-405c-a35a-dea98d68b832: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:59:56,109::task::495::TaskManager.Task::(_debug) Task ab811e85-9cf1-4284-868b-b360d132496e: moving from state init -> state preparing
>Thread-15915454::DEBUG::2013-11-20 03:59:56,111::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a895846f-7926-449b-bb95-7074bed2ef61`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913546::DEBUG::2013-11-20 03:59:56,114::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a3f52572-3ea3-4953-8523-539ffa8d1c89`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915454::DEBUG::2013-11-20 03:59:56,114::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:59:56,115::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15915454::DEBUG::2013-11-20 03:59:56,116::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915454::DEBUG::2013-11-20 03:59:56,118::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a895846f-7926-449b-bb95-7074bed2ef61`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:59:56,119::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15915454::DEBUG::2013-11-20 03:59:56,119::task::495::TaskManager.Task::(_debug) Task ed13df98-aef3-405c-a35a-dea98d68b832: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913546::DEBUG::2013-11-20 03:59:56,121::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a3f52572-3ea3-4953-8523-539ffa8d1c89`::Granted request
>Thread-15915454::DEBUG::2013-11-20 03:59:56,121::task::495::TaskManager.Task::(_debug) Task ed13df98-aef3-405c-a35a-dea98d68b832: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:59:56,123::task::495::TaskManager.Task::(_debug) Task ab811e85-9cf1-4284-868b-b360d132496e: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15915454::WARNING::2013-11-20 03:59:56,126::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913546::DEBUG::2013-11-20 03:59:56,126::task::495::TaskManager.Task::(_debug) Task ab811e85-9cf1-4284-868b-b360d132496e: ref 1 aborting False
>Thread-15915454::DEBUG::2013-11-20 03:59:56,127::task::495::TaskManager.Task::(_debug) Task ed13df98-aef3-405c-a35a-dea98d68b832: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913546::DEBUG::2013-11-20 03:59:56,128::task::495::TaskManager.Task::(_debug) Task ab811e85-9cf1-4284-868b-b360d132496e: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15915454::DEBUG::2013-11-20 03:59:56,129::task::495::TaskManager.Task::(_debug) Task ed13df98-aef3-405c-a35a-dea98d68b832: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:59:56,130::task::495::TaskManager.Task::(_debug) Task ab811e85-9cf1-4284-868b-b360d132496e: moving from state preparing -> state finished
>Thread-15915454::DEBUG::2013-11-20 03:59:56,131::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:59:56,132::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15915454::DEBUG::2013-11-20 03:59:56,132::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:59:56,133::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915454::DEBUG::2013-11-20 03:59:56,134::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913546::DEBUG::2013-11-20 03:59:56,135::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15915454::DEBUG::2013-11-20 03:59:56,136::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915454::DEBUG::2013-11-20 03:59:56,137::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915454::DEBUG::2013-11-20 03:59:56,138::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:59:56,139::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15915454::DEBUG::2013-11-20 03:59:56,140::task::495::TaskManager.Task::(_debug) Task ed13df98-aef3-405c-a35a-dea98d68b832: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:59:56,141::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15915454::INFO::2013-11-20 03:59:56,142::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913546::DEBUG::2013-11-20 03:59:56,142::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:59:56,145::task::495::TaskManager.Task::(_debug) Task ab811e85-9cf1-4284-868b-b360d132496e: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:59:56,146::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 03:59:56,147::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 03:59:56,148::task::495::TaskManager.Task::(_debug) Task db918cdb-cbd2-4f5f-8b8b-19acdb011df7: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:59:56,149::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d6efbea1-90fc-40f0-a1dc-20161992f862`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:59:56,150::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:59:56,150::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:59:56,151::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d6efbea1-90fc-40f0-a1dc-20161992f862`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:59:56,152::task::495::TaskManager.Task::(_debug) Task db918cdb-cbd2-4f5f-8b8b-19acdb011df7: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:59:56,153::task::495::TaskManager.Task::(_debug) Task db918cdb-cbd2-4f5f-8b8b-19acdb011df7: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:59:56,153::task::495::TaskManager.Task::(_debug) Task db918cdb-cbd2-4f5f-8b8b-19acdb011df7: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 03:59:56,154::task::495::TaskManager.Task::(_debug) Task db918cdb-cbd2-4f5f-8b8b-19acdb011df7: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:59:56,155::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:59:56,155::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:59:56,156::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:59:56,158::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:59:56,158::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:59:56,160::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:59:56,161::task::495::TaskManager.Task::(_debug) Task db918cdb-cbd2-4f5f-8b8b-19acdb011df7: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:59:56,162::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 03:59:56,163::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 03:59:56,164::task::495::TaskManager.Task::(_debug) Task 1f5b200a-8276-4e02-bcea-a925825de7bc: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 03:59:56,165::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f52af06b-c68a-4067-8a0d-97a2b5e84ac2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 03:59:56,166::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 03:59:56,167::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 03:59:56,167::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f52af06b-c68a-4067-8a0d-97a2b5e84ac2`::Granted request
>Thread-15913546::DEBUG::2013-11-20 03:59:56,168::task::495::TaskManager.Task::(_debug) Task 1f5b200a-8276-4e02-bcea-a925825de7bc: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 03:59:56,169::task::495::TaskManager.Task::(_debug) Task 1f5b200a-8276-4e02-bcea-a925825de7bc: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 03:59:56,169::task::495::TaskManager.Task::(_debug) Task 1f5b200a-8276-4e02-bcea-a925825de7bc: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 03:59:56,170::task::495::TaskManager.Task::(_debug) Task 1f5b200a-8276-4e02-bcea-a925825de7bc: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 03:59:56,171::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 03:59:56,172::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 03:59:56,172::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 03:59:56,173::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 03:59:56,174::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 03:59:56,174::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 03:59:56,175::task::495::TaskManager.Task::(_debug) Task 1f5b200a-8276-4e02-bcea-a925825de7bc: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 03:59:56,176::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 03:59:56,945::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 03:59:57,525::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0834049 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 03:59:59,539::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:00:00,026::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0521567 s, 19.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:00:02,037::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:00:02,549::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.063526 s, 16.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:00:04,562::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:00:05,125::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0806955 s, 12.7 MB/s\n'; <rc> = 0
>Thread-15915458::DEBUG::2013-11-20 04:00:06,513::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915458::INFO::2013-11-20 04:00:06,514::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915458::DEBUG::2013-11-20 04:00:06,541::task::495::TaskManager.Task::(_debug) Task 6dcade5b-b6b7-48e0-8d2d-d5e6d4c6ca09: moving from state init -> state preparing
>Thread-15915458::DEBUG::2013-11-20 04:00:06,542::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915458::DEBUG::2013-11-20 04:00:06,542::task::495::TaskManager.Task::(_debug) Task 6dcade5b-b6b7-48e0-8d2d-d5e6d4c6ca09: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915458::DEBUG::2013-11-20 04:00:06,543::task::495::TaskManager.Task::(_debug) Task 6dcade5b-b6b7-48e0-8d2d-d5e6d4c6ca09: moving from state preparing -> state finished
>Thread-15915458::DEBUG::2013-11-20 04:00:06,544::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915458::DEBUG::2013-11-20 04:00:06,544::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915458::DEBUG::2013-11-20 04:00:06,545::task::495::TaskManager.Task::(_debug) Task 6dcade5b-b6b7-48e0-8d2d-d5e6d4c6ca09: ref 0 aborting False
>Thread-15915458::INFO::2013-11-20 04:00:06,545::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915459::DEBUG::2013-11-20 04:00:06,550::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915459::INFO::2013-11-20 04:00:06,551::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915459::DEBUG::2013-11-20 04:00:06,551::task::495::TaskManager.Task::(_debug) Task 4e29ab07-2dc1-4748-b50c-7b5924843bc6: moving from state init -> state preparing
>Thread-15915459::DEBUG::2013-11-20 04:00:06,553::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c32c981f-94ca-4c68-98b4-b730ccae6aa2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915459::DEBUG::2013-11-20 04:00:06,553::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915459::DEBUG::2013-11-20 04:00:06,554::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915459::DEBUG::2013-11-20 04:00:06,555::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c32c981f-94ca-4c68-98b4-b730ccae6aa2`::Granted request
>Thread-15915459::DEBUG::2013-11-20 04:00:06,556::task::495::TaskManager.Task::(_debug) Task 4e29ab07-2dc1-4748-b50c-7b5924843bc6: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915459::DEBUG::2013-11-20 04:00:06,556::task::495::TaskManager.Task::(_debug) Task 4e29ab07-2dc1-4748-b50c-7b5924843bc6: ref 1 aborting False
>Thread-15915459::WARNING::2013-11-20 04:00:06,559::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915459::DEBUG::2013-11-20 04:00:06,560::task::495::TaskManager.Task::(_debug) Task 4e29ab07-2dc1-4748-b50c-7b5924843bc6: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915459::DEBUG::2013-11-20 04:00:06,560::task::495::TaskManager.Task::(_debug) Task 4e29ab07-2dc1-4748-b50c-7b5924843bc6: moving from state preparing -> state finished
>Thread-15915459::DEBUG::2013-11-20 04:00:06,561::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915459::DEBUG::2013-11-20 04:00:06,561::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915459::DEBUG::2013-11-20 04:00:06,562::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915459::DEBUG::2013-11-20 04:00:06,563::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915459::DEBUG::2013-11-20 04:00:06,564::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915459::DEBUG::2013-11-20 04:00:06,564::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915459::DEBUG::2013-11-20 04:00:06,565::task::495::TaskManager.Task::(_debug) Task 4e29ab07-2dc1-4748-b50c-7b5924843bc6: ref 0 aborting False
>Thread-15915459::INFO::2013-11-20 04:00:06,565::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915460::INFO::2013-11-20 04:00:06,592::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915460::DEBUG::2013-11-20 04:00:06,593::task::495::TaskManager.Task::(_debug) Task 02157617-b110-41e8-9be9-dfc7ffb2eb82: moving from state init -> state preparing
>Thread-15915460::DEBUG::2013-11-20 04:00:06,593::task::495::TaskManager.Task::(_debug) Task 02157617-b110-41e8-9be9-dfc7ffb2eb82: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.0003981590271', 'lastCheck': 1384919998.6411159, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00173401832581', 'lastCheck': 1384919999.1698289, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00135517120361', 'lastCheck': 1384919996.6550429, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0011727809906', 'lastCheck': 1384920002.251786, 'valid': True, 'code': 0}}
>Thread-15915460::DEBUG::2013-11-20 04:00:06,594::task::495::TaskManager.Task::(_debug) Task 02157617-b110-41e8-9be9-dfc7ffb2eb82: moving from state preparing -> state finished
>Thread-15915460::DEBUG::2013-11-20 04:00:06,595::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915460::DEBUG::2013-11-20 04:00:06,595::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915460::DEBUG::2013-11-20 04:00:06,596::task::495::TaskManager.Task::(_debug) Task 02157617-b110-41e8-9be9-dfc7ffb2eb82: ref 0 aborting False
>Thread-15915460::INFO::2013-11-20 04:00:06,597::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.0003981590271', 'lastCheck': 1384919998.6411159, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00173401832581', 'lastCheck': 1384919999.1698289, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00135517120361', 'lastCheck': 1384919996.6550429, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0011727809906', 'lastCheck': 1384920002.251786, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:00:07,140::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:00:07,727::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0865707 s, 11.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:00:09,742::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913518::INFO::2013-11-20 04:00:09,978::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:00:09,979::task::495::TaskManager.Task::(_debug) Task fd7a9058-e611-494a-af5b-e87c316bb1d8: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:00:09,982::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`198949e3-cd28-4b97-bdd2-68f12df1d383`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:00:09,982::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:00:09,983::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:00:09,984::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`198949e3-cd28-4b97-bdd2-68f12df1d383`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:00:09,986::task::495::TaskManager.Task::(_debug) Task fd7a9058-e611-494a-af5b-e87c316bb1d8: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:00:09,986::task::495::TaskManager.Task::(_debug) Task fd7a9058-e611-494a-af5b-e87c316bb1d8: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:00:09,987::task::495::TaskManager.Task::(_debug) Task fd7a9058-e611-494a-af5b-e87c316bb1d8: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:00:09,988::task::495::TaskManager.Task::(_debug) Task fd7a9058-e611-494a-af5b-e87c316bb1d8: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:00:09,989::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:00:09,989::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:00:09,990::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:00:09,991::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:00:09,991::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:00:09,992::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:00:09,992::task::495::TaskManager.Task::(_debug) Task fd7a9058-e611-494a-af5b-e87c316bb1d8: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:00:09,993::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:00:09,994::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:00:09,994::task::495::TaskManager.Task::(_debug) Task 9b430191-a7ed-4e43-a037-7dd75b120cc6: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:00:09,996::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8627654f-ce71-4d80-b56e-b78a2b6b4f2d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:00:09,996::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:00:09,997::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:00:09,998::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8627654f-ce71-4d80-b56e-b78a2b6b4f2d`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:00:09,999::task::495::TaskManager.Task::(_debug) Task 9b430191-a7ed-4e43-a037-7dd75b120cc6: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:00:10,000::task::495::TaskManager.Task::(_debug) Task 9b430191-a7ed-4e43-a037-7dd75b120cc6: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:00:10,000::task::495::TaskManager.Task::(_debug) Task 9b430191-a7ed-4e43-a037-7dd75b120cc6: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:00:10,001::task::495::TaskManager.Task::(_debug) Task 9b430191-a7ed-4e43-a037-7dd75b120cc6: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:00:10,002::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:00:10,002::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:00:10,003::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:00:10,004::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:00:10,005::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:00:10,005::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:00:10,006::task::495::TaskManager.Task::(_debug) Task 9b430191-a7ed-4e43-a037-7dd75b120cc6: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:00:10,007::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:00:10,309::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0859592 s, 11.9 MB/s\n'; <rc> = 0
>Thread-15913639::INFO::2013-11-20 04:00:11,369::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:00:11,370::task::495::TaskManager.Task::(_debug) Task 1e053e6e-be4a-44de-9ac3-2fb253f91b83: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:00:11,372::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`44c43bc2-1789-4985-9268-b2f277ac563b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:00:11,373::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:00:11,374::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:00:11,374::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`44c43bc2-1789-4985-9268-b2f277ac563b`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:00:11,376::task::495::TaskManager.Task::(_debug) Task 1e053e6e-be4a-44de-9ac3-2fb253f91b83: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:00:11,376::task::495::TaskManager.Task::(_debug) Task 1e053e6e-be4a-44de-9ac3-2fb253f91b83: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:00:11,377::task::495::TaskManager.Task::(_debug) Task 1e053e6e-be4a-44de-9ac3-2fb253f91b83: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:00:11,378::task::495::TaskManager.Task::(_debug) Task 1e053e6e-be4a-44de-9ac3-2fb253f91b83: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:00:11,378::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:00:11,379::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:00:11,380::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:00:11,380::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:00:11,381::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:00:11,381::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:00:11,382::task::495::TaskManager.Task::(_debug) Task 1e053e6e-be4a-44de-9ac3-2fb253f91b83: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:00:11,383::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:00:12,324::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-42840::DEBUG::2013-11-20 04:00:12,514::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:00:13,124::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.105413 s, 9.7 MB/s\n'; <rc> = 0
>Thread-42840::DEBUG::2013-11-20 04:00:13,535::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:00:15,136::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:00:15,718::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0635101 s, 16.1 MB/s\n'; <rc> = 0
>Thread-15915465::DEBUG::2013-11-20 04:00:17,093::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915465::INFO::2013-11-20 04:00:17,094::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915465::DEBUG::2013-11-20 04:00:17,095::task::495::TaskManager.Task::(_debug) Task 08102f87-19fe-4e0b-8ed7-5409db2f2355: moving from state init -> state preparing
>Thread-15915465::DEBUG::2013-11-20 04:00:17,096::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915465::DEBUG::2013-11-20 04:00:17,096::task::495::TaskManager.Task::(_debug) Task 08102f87-19fe-4e0b-8ed7-5409db2f2355: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915465::DEBUG::2013-11-20 04:00:17,097::task::495::TaskManager.Task::(_debug) Task 08102f87-19fe-4e0b-8ed7-5409db2f2355: moving from state preparing -> state finished
>Thread-15915465::DEBUG::2013-11-20 04:00:17,097::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915465::DEBUG::2013-11-20 04:00:17,098::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915465::DEBUG::2013-11-20 04:00:17,098::task::495::TaskManager.Task::(_debug) Task 08102f87-19fe-4e0b-8ed7-5409db2f2355: ref 0 aborting False
>Thread-15915465::INFO::2013-11-20 04:00:17,099::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915466::DEBUG::2013-11-20 04:00:17,104::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915466::INFO::2013-11-20 04:00:17,106::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915466::DEBUG::2013-11-20 04:00:17,106::task::495::TaskManager.Task::(_debug) Task 9e3310c1-aec6-438e-b9f1-2f00bb1dc7c8: moving from state init -> state preparing
>Thread-15915466::DEBUG::2013-11-20 04:00:17,108::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a25d09e9-e4bc-476b-b746-e91c6d939c67`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915466::DEBUG::2013-11-20 04:00:17,108::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915466::DEBUG::2013-11-20 04:00:17,109::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915466::DEBUG::2013-11-20 04:00:17,110::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a25d09e9-e4bc-476b-b746-e91c6d939c67`::Granted request
>Thread-15915466::DEBUG::2013-11-20 04:00:17,116::task::495::TaskManager.Task::(_debug) Task 9e3310c1-aec6-438e-b9f1-2f00bb1dc7c8: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915466::DEBUG::2013-11-20 04:00:17,117::task::495::TaskManager.Task::(_debug) Task 9e3310c1-aec6-438e-b9f1-2f00bb1dc7c8: ref 1 aborting False
>Thread-15915466::WARNING::2013-11-20 04:00:17,119::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915466::DEBUG::2013-11-20 04:00:17,120::task::495::TaskManager.Task::(_debug) Task 9e3310c1-aec6-438e-b9f1-2f00bb1dc7c8: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915466::DEBUG::2013-11-20 04:00:17,121::task::495::TaskManager.Task::(_debug) Task 9e3310c1-aec6-438e-b9f1-2f00bb1dc7c8: moving from state preparing -> state finished
>Thread-15915466::DEBUG::2013-11-20 04:00:17,121::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915466::DEBUG::2013-11-20 04:00:17,122::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915466::DEBUG::2013-11-20 04:00:17,123::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915466::DEBUG::2013-11-20 04:00:17,123::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915466::DEBUG::2013-11-20 04:00:17,124::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915466::DEBUG::2013-11-20 04:00:17,124::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915466::DEBUG::2013-11-20 04:00:17,125::task::495::TaskManager.Task::(_debug) Task 9e3310c1-aec6-438e-b9f1-2f00bb1dc7c8: ref 0 aborting False
>Thread-15915466::INFO::2013-11-20 04:00:17,126::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:00:17,733::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:00:18,327::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0894044 s, 11.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:00:20,344::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:00:20,998::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.083474 s, 12.3 MB/s\n'; <rc> = 0
>Thread-15915468::INFO::2013-11-20 04:00:22,704::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915468::DEBUG::2013-11-20 04:00:22,705::task::495::TaskManager.Task::(_debug) Task 3a1580bf-e693-400a-ace9-8d7efa548ca4: moving from state init -> state preparing
>Thread-15915468::DEBUG::2013-11-20 04:00:22,705::task::495::TaskManager.Task::(_debug) Task 3a1580bf-e693-400a-ace9-8d7efa548ca4: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.0028018951416', 'lastCheck': 1384920018.9749501, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142693519592', 'lastCheck': 1384920019.531662, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00145483016968', 'lastCheck': 1384920016.970202, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000333070755005', 'lastCheck': 1384920013.5482061, 'valid': True, 'code': 0}}
>Thread-15915468::DEBUG::2013-11-20 04:00:22,706::task::495::TaskManager.Task::(_debug) Task 3a1580bf-e693-400a-ace9-8d7efa548ca4: moving from state preparing -> state finished
>Thread-15915468::DEBUG::2013-11-20 04:00:22,707::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915468::DEBUG::2013-11-20 04:00:22,707::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915468::DEBUG::2013-11-20 04:00:22,708::task::495::TaskManager.Task::(_debug) Task 3a1580bf-e693-400a-ace9-8d7efa548ca4: ref 0 aborting False
>Thread-15915468::INFO::2013-11-20 04:00:22,708::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.0028018951416', 'lastCheck': 1384920018.9749501, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142693519592', 'lastCheck': 1384920019.531662, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00145483016968', 'lastCheck': 1384920016.970202, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000333070755005', 'lastCheck': 1384920013.5482061, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:00:23,019::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:00:23,501::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0610396 s, 16.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:00:25,514::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:00:26,125::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0749987 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 04:00:26,278::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:00:26,280::task::495::TaskManager.Task::(_debug) Task ef742325-5012-4fe4-a6d2-ff3b48c005a4: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:00:26,283::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`dd0733ef-9236-4f80-b956-4c857e6b7bb8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:00:26,284::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:00:26,285::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:00:26,285::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`dd0733ef-9236-4f80-b956-4c857e6b7bb8`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:00:26,287::task::495::TaskManager.Task::(_debug) Task ef742325-5012-4fe4-a6d2-ff3b48c005a4: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:00:26,288::task::495::TaskManager.Task::(_debug) Task ef742325-5012-4fe4-a6d2-ff3b48c005a4: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:00:26,288::task::495::TaskManager.Task::(_debug) Task ef742325-5012-4fe4-a6d2-ff3b48c005a4: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:00:26,289::task::495::TaskManager.Task::(_debug) Task ef742325-5012-4fe4-a6d2-ff3b48c005a4: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:00:26,289::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:00:26,290::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:00:26,291::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:00:26,291::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:00:26,292::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:00:26,293::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:00:26,294::task::495::TaskManager.Task::(_debug) Task ef742325-5012-4fe4-a6d2-ff3b48c005a4: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:00:26,294::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15915471::DEBUG::2013-11-20 04:00:27,417::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915471::INFO::2013-11-20 04:00:27,418::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915471::DEBUG::2013-11-20 04:00:27,419::task::495::TaskManager.Task::(_debug) Task 11ef096f-6ea5-421a-90a0-978e4867efbe: moving from state init -> state preparing
>Thread-15915471::DEBUG::2013-11-20 04:00:27,419::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915471::DEBUG::2013-11-20 04:00:27,419::task::495::TaskManager.Task::(_debug) Task 11ef096f-6ea5-421a-90a0-978e4867efbe: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915471::DEBUG::2013-11-20 04:00:27,420::task::495::TaskManager.Task::(_debug) Task 11ef096f-6ea5-421a-90a0-978e4867efbe: moving from state preparing -> state finished
>Thread-15915471::DEBUG::2013-11-20 04:00:27,420::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915471::DEBUG::2013-11-20 04:00:27,421::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915471::DEBUG::2013-11-20 04:00:27,421::task::495::TaskManager.Task::(_debug) Task 11ef096f-6ea5-421a-90a0-978e4867efbe: ref 0 aborting False
>Thread-15915471::INFO::2013-11-20 04:00:27,421::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915472::DEBUG::2013-11-20 04:00:27,425::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915472::INFO::2013-11-20 04:00:27,427::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915472::DEBUG::2013-11-20 04:00:27,427::task::495::TaskManager.Task::(_debug) Task 75cba583-ddfb-42ed-9181-274abda0577b: moving from state init -> state preparing
>Thread-15915472::DEBUG::2013-11-20 04:00:27,429::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3f5c6103-11d5-4ea4-8078-2a45359a5b12`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915472::DEBUG::2013-11-20 04:00:27,430::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915472::DEBUG::2013-11-20 04:00:27,431::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915472::DEBUG::2013-11-20 04:00:27,432::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3f5c6103-11d5-4ea4-8078-2a45359a5b12`::Granted request
>Thread-15915472::DEBUG::2013-11-20 04:00:27,433::task::495::TaskManager.Task::(_debug) Task 75cba583-ddfb-42ed-9181-274abda0577b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915472::DEBUG::2013-11-20 04:00:27,433::task::495::TaskManager.Task::(_debug) Task 75cba583-ddfb-42ed-9181-274abda0577b: ref 1 aborting False
>Thread-15915472::WARNING::2013-11-20 04:00:27,435::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915472::DEBUG::2013-11-20 04:00:27,436::task::495::TaskManager.Task::(_debug) Task 75cba583-ddfb-42ed-9181-274abda0577b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915472::DEBUG::2013-11-20 04:00:27,438::task::495::TaskManager.Task::(_debug) Task 75cba583-ddfb-42ed-9181-274abda0577b: moving from state preparing -> state finished
>Thread-15915472::DEBUG::2013-11-20 04:00:27,438::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915472::DEBUG::2013-11-20 04:00:27,439::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915472::DEBUG::2013-11-20 04:00:27,440::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915472::DEBUG::2013-11-20 04:00:27,441::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915472::DEBUG::2013-11-20 04:00:27,441::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915472::DEBUG::2013-11-20 04:00:27,442::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915472::DEBUG::2013-11-20 04:00:27,442::task::495::TaskManager.Task::(_debug) Task 75cba583-ddfb-42ed-9181-274abda0577b: ref 0 aborting False
>Thread-15915472::INFO::2013-11-20 04:00:27,443::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:00:28,141::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:00:28,699::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0736092 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:00:30,712::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:00:31,246::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0798033 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:00:33,267::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:00:33,832::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0830726 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:00:35,847::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 04:00:36,368::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:00:36,373::task::495::TaskManager.Task::(_debug) Task cdf9272f-497c-49ee-9e46-902f36b84c93: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:00:36,376::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9e35f70a-11dd-4c0f-b8b6-daaf03232658`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:00:36,378::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:00:36,380::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:00:36,381::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9e35f70a-11dd-4c0f-b8b6-daaf03232658`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:00:36,385::task::495::TaskManager.Task::(_debug) Task cdf9272f-497c-49ee-9e46-902f36b84c93: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:00:36,386::task::495::TaskManager.Task::(_debug) Task cdf9272f-497c-49ee-9e46-902f36b84c93: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:00:36,388::task::495::TaskManager.Task::(_debug) Task cdf9272f-497c-49ee-9e46-902f36b84c93: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:00:36,389::task::495::TaskManager.Task::(_debug) Task cdf9272f-497c-49ee-9e46-902f36b84c93: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:00:36,390::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:00:36,391::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:00:36,393::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:00:36,395::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:00:36,396::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:00:36,397::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:00:36,399::task::495::TaskManager.Task::(_debug) Task cdf9272f-497c-49ee-9e46-902f36b84c93: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:00:36,401::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:00:36,423::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0829802 s, 12.3 MB/s\n'; <rc> = 0
>Thread-15915476::DEBUG::2013-11-20 04:00:37,886::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915476::INFO::2013-11-20 04:00:37,887::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915476::DEBUG::2013-11-20 04:00:37,887::task::495::TaskManager.Task::(_debug) Task e182bb29-40b9-4c15-8601-d2a74a484cf6: moving from state init -> state preparing
>Thread-15915476::DEBUG::2013-11-20 04:00:37,888::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915476::DEBUG::2013-11-20 04:00:37,889::task::495::TaskManager.Task::(_debug) Task e182bb29-40b9-4c15-8601-d2a74a484cf6: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915476::DEBUG::2013-11-20 04:00:37,889::task::495::TaskManager.Task::(_debug) Task e182bb29-40b9-4c15-8601-d2a74a484cf6: moving from state preparing -> state finished
>Thread-15915476::DEBUG::2013-11-20 04:00:37,890::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915476::DEBUG::2013-11-20 04:00:37,890::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915476::DEBUG::2013-11-20 04:00:37,891::task::495::TaskManager.Task::(_debug) Task e182bb29-40b9-4c15-8601-d2a74a484cf6: ref 0 aborting False
>Thread-15915476::INFO::2013-11-20 04:00:37,892::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915477::DEBUG::2013-11-20 04:00:37,896::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915477::INFO::2013-11-20 04:00:37,898::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915477::DEBUG::2013-11-20 04:00:37,898::task::495::TaskManager.Task::(_debug) Task 144f3601-4669-43b4-ab83-030e5fa5dcb3: moving from state init -> state preparing
>Thread-15915477::DEBUG::2013-11-20 04:00:37,900::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d4caf1c0-0b76-4a7e-948c-0fc382d29d49`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915477::DEBUG::2013-11-20 04:00:37,901::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915477::DEBUG::2013-11-20 04:00:37,902::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915477::DEBUG::2013-11-20 04:00:37,902::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d4caf1c0-0b76-4a7e-948c-0fc382d29d49`::Granted request
>Thread-15915477::DEBUG::2013-11-20 04:00:37,904::task::495::TaskManager.Task::(_debug) Task 144f3601-4669-43b4-ab83-030e5fa5dcb3: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915477::DEBUG::2013-11-20 04:00:37,905::task::495::TaskManager.Task::(_debug) Task 144f3601-4669-43b4-ab83-030e5fa5dcb3: ref 1 aborting False
>Thread-15915477::WARNING::2013-11-20 04:00:37,908::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915477::DEBUG::2013-11-20 04:00:37,909::task::495::TaskManager.Task::(_debug) Task 144f3601-4669-43b4-ab83-030e5fa5dcb3: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915477::DEBUG::2013-11-20 04:00:37,910::task::495::TaskManager.Task::(_debug) Task 144f3601-4669-43b4-ab83-030e5fa5dcb3: moving from state preparing -> state finished
>Thread-15915477::DEBUG::2013-11-20 04:00:37,911::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915477::DEBUG::2013-11-20 04:00:37,912::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915477::DEBUG::2013-11-20 04:00:37,913::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915477::DEBUG::2013-11-20 04:00:37,914::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915477::DEBUG::2013-11-20 04:00:37,915::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915477::DEBUG::2013-11-20 04:00:37,915::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915477::DEBUG::2013-11-20 04:00:37,916::task::495::TaskManager.Task::(_debug) Task 144f3601-4669-43b4-ab83-030e5fa5dcb3: ref 0 aborting False
>Thread-15915477::INFO::2013-11-20 04:00:37,917::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:00:38,444::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915478::INFO::2013-11-20 04:00:38,445::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915478::DEBUG::2013-11-20 04:00:38,649::task::495::TaskManager.Task::(_debug) Task 6da6a512-7eb5-4256-85f6-8f9931184da7: moving from state init -> state preparing
>Thread-15915478::DEBUG::2013-11-20 04:00:38,654::task::495::TaskManager.Task::(_debug) Task 6da6a512-7eb5-4256-85f6-8f9931184da7: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000304937362671', 'lastCheck': 1384920028.990139, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.120426177979', 'lastCheck': 1384920029.874845, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00135588645935', 'lastCheck': 1384920037.0660701, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000396013259888', 'lastCheck': 1384920033.9137959, 'valid': True, 'code': 0}}
>Thread-15915478::DEBUG::2013-11-20 04:00:38,655::task::495::TaskManager.Task::(_debug) Task 6da6a512-7eb5-4256-85f6-8f9931184da7: moving from state preparing -> state finished
>Thread-15915478::DEBUG::2013-11-20 04:00:38,656::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915478::DEBUG::2013-11-20 04:00:38,656::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915478::DEBUG::2013-11-20 04:00:38,657::task::495::TaskManager.Task::(_debug) Task 6da6a512-7eb5-4256-85f6-8f9931184da7: ref 0 aborting False
>Thread-15915478::INFO::2013-11-20 04:00:38,658::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000304937362671', 'lastCheck': 1384920028.990139, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.120426177979', 'lastCheck': 1384920029.874845, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00135588645935', 'lastCheck': 1384920037.0660701, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000396013259888', 'lastCheck': 1384920033.9137959, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:00:38,974::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0768334 s, 13.3 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 04:00:39,556::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:00:39,557::task::495::TaskManager.Task::(_debug) Task bfc48424-dd4d-4951-94b0-9446113ef570: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:00:39,559::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`514b3aed-fd0c-41cd-a13c-9fce8f625fc2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:00:39,560::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:00:39,561::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:00:39,562::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`514b3aed-fd0c-41cd-a13c-9fce8f625fc2`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:00:39,563::task::495::TaskManager.Task::(_debug) Task bfc48424-dd4d-4951-94b0-9446113ef570: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:00:39,564::task::495::TaskManager.Task::(_debug) Task bfc48424-dd4d-4951-94b0-9446113ef570: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:00:39,565::task::495::TaskManager.Task::(_debug) Task bfc48424-dd4d-4951-94b0-9446113ef570: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:00:39,566::task::495::TaskManager.Task::(_debug) Task bfc48424-dd4d-4951-94b0-9446113ef570: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:00:39,567::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:00:39,567::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:00:39,568::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:00:39,569::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:00:39,569::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:00:39,570::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:00:39,571::task::495::TaskManager.Task::(_debug) Task bfc48424-dd4d-4951-94b0-9446113ef570: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:00:39,572::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:00:40,990::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915085::INFO::2013-11-20 04:00:41,503::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=089b08fc-6544-4235-a061-fb55b09d2715 volUUID=66b599fc-b7b5-4934-a5d5-ef7283460029)
>Thread-15915085::DEBUG::2013-11-20 04:00:41,506::task::495::TaskManager.Task::(_debug) Task d5d71399-eaee-4b43-a7ff-592824d7431c: moving from state init -> state preparing
>Thread-15915085::DEBUG::2013-11-20 04:00:41,511::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8e776c51-0e5d-400e-b949-ecf8dc0a8d94`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915085::DEBUG::2013-11-20 04:00:41,512::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915085::DEBUG::2013-11-20 04:00:41,517::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915085::DEBUG::2013-11-20 04:00:41,519::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8e776c51-0e5d-400e-b949-ecf8dc0a8d94`::Granted request
>Thread-15915085::DEBUG::2013-11-20 04:00:41,521::task::495::TaskManager.Task::(_debug) Task d5d71399-eaee-4b43-a7ff-592824d7431c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915085::DEBUG::2013-11-20 04:00:41,522::task::495::TaskManager.Task::(_debug) Task d5d71399-eaee-4b43-a7ff-592824d7431c: ref 1 aborting False
>Thread-15915085::DEBUG::2013-11-20 04:00:41,524::task::495::TaskManager.Task::(_debug) Task d5d71399-eaee-4b43-a7ff-592824d7431c: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15915085::DEBUG::2013-11-20 04:00:41,525::task::495::TaskManager.Task::(_debug) Task d5d71399-eaee-4b43-a7ff-592824d7431c: moving from state preparing -> state finished
>Thread-15915085::DEBUG::2013-11-20 04:00:41,526::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915085::DEBUG::2013-11-20 04:00:41,527::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915085::DEBUG::2013-11-20 04:00:41,529::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915085::DEBUG::2013-11-20 04:00:41,530::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915085::DEBUG::2013-11-20 04:00:41,531::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915085::DEBUG::2013-11-20 04:00:41,533::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915085::DEBUG::2013-11-20 04:00:41,535::task::495::TaskManager.Task::(_debug) Task d5d71399-eaee-4b43-a7ff-592824d7431c: ref 0 aborting False
>Thread-15915085::INFO::2013-11-20 04:00:41,537::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Dummy-15912651::DEBUG::2013-11-20 04:00:41,564::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0692888 s, 14.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:00:43,585::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:00:44,065::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0627645 s, 16.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:00:46,076::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 04:00:46,381::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:00:46,384::task::495::TaskManager.Task::(_debug) Task 247658d9-75f0-4006-871a-757e0e6c8fda: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:00:46,387::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9617956a-a304-45e9-9cb6-89f80bf668e8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:00:46,388::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:00:46,389::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:00:46,390::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9617956a-a304-45e9-9cb6-89f80bf668e8`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:00:46,391::task::495::TaskManager.Task::(_debug) Task 247658d9-75f0-4006-871a-757e0e6c8fda: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:00:46,392::task::495::TaskManager.Task::(_debug) Task 247658d9-75f0-4006-871a-757e0e6c8fda: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:00:46,393::task::495::TaskManager.Task::(_debug) Task 247658d9-75f0-4006-871a-757e0e6c8fda: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:00:46,393::task::495::TaskManager.Task::(_debug) Task 247658d9-75f0-4006-871a-757e0e6c8fda: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:00:46,394::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:00:46,395::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:00:46,396::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:00:46,396::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:00:46,397::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:00:46,398::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:00:46,398::task::495::TaskManager.Task::(_debug) Task 247658d9-75f0-4006-871a-757e0e6c8fda: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:00:46,399::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:00:46,616::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0739411 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15915482::DEBUG::2013-11-20 04:00:48,252::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915482::INFO::2013-11-20 04:00:48,253::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915482::DEBUG::2013-11-20 04:00:48,254::task::495::TaskManager.Task::(_debug) Task d2942117-f142-448e-99f4-8bc5c543fb0e: moving from state init -> state preparing
>Thread-15915482::DEBUG::2013-11-20 04:00:48,255::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915482::DEBUG::2013-11-20 04:00:48,256::task::495::TaskManager.Task::(_debug) Task d2942117-f142-448e-99f4-8bc5c543fb0e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915482::DEBUG::2013-11-20 04:00:48,257::task::495::TaskManager.Task::(_debug) Task d2942117-f142-448e-99f4-8bc5c543fb0e: moving from state preparing -> state finished
>Thread-15915482::DEBUG::2013-11-20 04:00:48,258::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915482::DEBUG::2013-11-20 04:00:48,259::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915482::DEBUG::2013-11-20 04:00:48,259::task::495::TaskManager.Task::(_debug) Task d2942117-f142-448e-99f4-8bc5c543fb0e: ref 0 aborting False
>Thread-15915482::INFO::2013-11-20 04:00:48,260::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915483::DEBUG::2013-11-20 04:00:48,265::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915483::INFO::2013-11-20 04:00:48,267::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915483::DEBUG::2013-11-20 04:00:48,267::task::495::TaskManager.Task::(_debug) Task b106b30f-b50b-4368-88d3-39e27834fe3d: moving from state init -> state preparing
>Thread-15915483::DEBUG::2013-11-20 04:00:48,269::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f265fc8c-2d60-4c58-9187-17fb2aceb0c0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915483::DEBUG::2013-11-20 04:00:48,270::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915483::DEBUG::2013-11-20 04:00:48,271::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915483::DEBUG::2013-11-20 04:00:48,272::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f265fc8c-2d60-4c58-9187-17fb2aceb0c0`::Granted request
>Thread-15915483::DEBUG::2013-11-20 04:00:48,274::task::495::TaskManager.Task::(_debug) Task b106b30f-b50b-4368-88d3-39e27834fe3d: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915483::DEBUG::2013-11-20 04:00:48,275::task::495::TaskManager.Task::(_debug) Task b106b30f-b50b-4368-88d3-39e27834fe3d: ref 1 aborting False
>Thread-15915483::WARNING::2013-11-20 04:00:48,278::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915483::DEBUG::2013-11-20 04:00:48,279::task::495::TaskManager.Task::(_debug) Task b106b30f-b50b-4368-88d3-39e27834fe3d: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915483::DEBUG::2013-11-20 04:00:48,280::task::495::TaskManager.Task::(_debug) Task b106b30f-b50b-4368-88d3-39e27834fe3d: moving from state preparing -> state finished
>Thread-15915483::DEBUG::2013-11-20 04:00:48,281::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915483::DEBUG::2013-11-20 04:00:48,282::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915483::DEBUG::2013-11-20 04:00:48,283::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915483::DEBUG::2013-11-20 04:00:48,285::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915483::DEBUG::2013-11-20 04:00:48,285::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915483::DEBUG::2013-11-20 04:00:48,286::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915483::DEBUG::2013-11-20 04:00:48,287::task::495::TaskManager.Task::(_debug) Task b106b30f-b50b-4368-88d3-39e27834fe3d: ref 0 aborting False
>Thread-15915483::INFO::2013-11-20 04:00:48,288::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:00:48,632::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:00:49,173::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0857055 s, 11.9 MB/s\n'; <rc> = 0
>Thread-15914435::INFO::2013-11-20 04:00:49,599::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:00:49,600::task::495::TaskManager.Task::(_debug) Task 65bd74a4-e580-4544-8eef-8250a7593686: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:00:49,602::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`52ebe6af-e1ad-4040-8124-45c6c2c89839`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:00:49,602::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:00:49,603::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:00:49,604::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`52ebe6af-e1ad-4040-8124-45c6c2c89839`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:00:49,605::task::495::TaskManager.Task::(_debug) Task 65bd74a4-e580-4544-8eef-8250a7593686: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:00:49,606::task::495::TaskManager.Task::(_debug) Task 65bd74a4-e580-4544-8eef-8250a7593686: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:00:49,606::task::495::TaskManager.Task::(_debug) Task 65bd74a4-e580-4544-8eef-8250a7593686: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:00:49,607::task::495::TaskManager.Task::(_debug) Task 65bd74a4-e580-4544-8eef-8250a7593686: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:00:49,608::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:00:49,609::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:00:49,609::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:00:49,610::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:00:49,611::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:00:49,612::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:00:49,612::task::495::TaskManager.Task::(_debug) Task 65bd74a4-e580-4544-8eef-8250a7593686: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:00:49,613::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:00:51,187::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:00:51,744::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0824546 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:00:53,760::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 04:00:53,985::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:00:53,990::task::495::TaskManager.Task::(_debug) Task c8b980dc-f5d3-4052-ab5a-884eaa8db627: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:00:53,998::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c2cbadf4-bfac-4f19-b625-ee31ab74ef9e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:00:53,999::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:00:54,002::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:00:54,003::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c2cbadf4-bfac-4f19-b625-ee31ab74ef9e`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:00:54,005::task::495::TaskManager.Task::(_debug) Task c8b980dc-f5d3-4052-ab5a-884eaa8db627: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 04:00:54,006::task::495::TaskManager.Task::(_debug) Task c8b980dc-f5d3-4052-ab5a-884eaa8db627: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:00:54,007::task::495::TaskManager.Task::(_debug) Task c8b980dc-f5d3-4052-ab5a-884eaa8db627: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 04:00:54,008::task::495::TaskManager.Task::(_debug) Task c8b980dc-f5d3-4052-ab5a-884eaa8db627: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:00:54,009::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:00:54,010::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:00:54,011::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 04:00:54,012::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:00:54,013::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:00:54,014::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:00:54,015::task::495::TaskManager.Task::(_debug) Task c8b980dc-f5d3-4052-ab5a-884eaa8db627: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 04:00:54,016::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 04:00:54,354::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0954065 s, 10.7 MB/s\n'; <rc> = 0
>Thread-15915486::INFO::2013-11-20 04:00:54,486::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915486::DEBUG::2013-11-20 04:00:54,487::task::495::TaskManager.Task::(_debug) Task be0741da-9f03-43ea-a01c-dde3546ff547: moving from state init -> state preparing
>Thread-15915486::DEBUG::2013-11-20 04:00:54,487::task::495::TaskManager.Task::(_debug) Task be0741da-9f03-43ea-a01c-dde3546ff547: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000490188598633', 'lastCheck': 1384920049.2960329, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00163388252258', 'lastCheck': 1384920050.2925, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00150299072266', 'lastCheck': 1384920047.2881, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000580072402954', 'lastCheck': 1384920054.3908479, 'valid': True, 'code': 0}}
>Thread-15915486::DEBUG::2013-11-20 04:00:54,488::task::495::TaskManager.Task::(_debug) Task be0741da-9f03-43ea-a01c-dde3546ff547: moving from state preparing -> state finished
>Thread-15915486::DEBUG::2013-11-20 04:00:54,489::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915486::DEBUG::2013-11-20 04:00:54,489::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915486::DEBUG::2013-11-20 04:00:54,490::task::495::TaskManager.Task::(_debug) Task be0741da-9f03-43ea-a01c-dde3546ff547: ref 0 aborting False
>Thread-15915486::INFO::2013-11-20 04:00:54,490::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000490188598633', 'lastCheck': 1384920049.2960329, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00163388252258', 'lastCheck': 1384920050.2925, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00150299072266', 'lastCheck': 1384920047.2881, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000580072402954', 'lastCheck': 1384920054.3908479, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:00:56,372::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 04:00:56,600::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:00:56,602::task::495::TaskManager.Task::(_debug) Task 82236074-0eda-4e7b-a67d-836bb7062aec: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:00:56,606::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c22edd81-6c8a-4b0f-a93b-7aa948afcb4e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:00:56,607::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:00:56,609::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:00:56,610::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c22edd81-6c8a-4b0f-a93b-7aa948afcb4e`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:00:56,611::task::495::TaskManager.Task::(_debug) Task 82236074-0eda-4e7b-a67d-836bb7062aec: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:00:56,612::task::495::TaskManager.Task::(_debug) Task 82236074-0eda-4e7b-a67d-836bb7062aec: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:00:56,613::task::495::TaskManager.Task::(_debug) Task 82236074-0eda-4e7b-a67d-836bb7062aec: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 04:00:56,614::task::495::TaskManager.Task::(_debug) Task 82236074-0eda-4e7b-a67d-836bb7062aec: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:00:56,614::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:00:56,615::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:00:56,616::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:00:56,617::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:00:56,617::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:00:56,618::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:00:56,619::task::495::TaskManager.Task::(_debug) Task 82236074-0eda-4e7b-a67d-836bb7062aec: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:00:56,619::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:00:56,620::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:00:56,621::task::495::TaskManager.Task::(_debug) Task 2477947f-4679-4afc-bf5f-e84ca12e136c: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:00:56,622::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e54387e0-27e8-4032-bc18-1dcdb0686b40`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:00:56,623::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:00:56,623::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:00:56,624::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e54387e0-27e8-4032-bc18-1dcdb0686b40`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:00:56,625::task::495::TaskManager.Task::(_debug) Task 2477947f-4679-4afc-bf5f-e84ca12e136c: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:00:56,626::task::495::TaskManager.Task::(_debug) Task 2477947f-4679-4afc-bf5f-e84ca12e136c: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:00:56,627::task::495::TaskManager.Task::(_debug) Task 2477947f-4679-4afc-bf5f-e84ca12e136c: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:00:56,627::task::495::TaskManager.Task::(_debug) Task 2477947f-4679-4afc-bf5f-e84ca12e136c: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:00:56,628::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:00:56,629::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:00:56,629::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:00:56,630::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:00:56,631::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:00:56,631::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:00:56,632::task::495::TaskManager.Task::(_debug) Task 2477947f-4679-4afc-bf5f-e84ca12e136c: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:00:56,633::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:00:56,634::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:00:56,635::task::495::TaskManager.Task::(_debug) Task 7a08ec02-40c4-4a3c-a48f-0c717c5afa09: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:00:56,636::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f14b6525-40f0-4bb4-a734-e03765f7e271`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:00:56,637::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:00:56,637::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:00:56,638::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f14b6525-40f0-4bb4-a734-e03765f7e271`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:00:56,639::task::495::TaskManager.Task::(_debug) Task 7a08ec02-40c4-4a3c-a48f-0c717c5afa09: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:00:56,640::task::495::TaskManager.Task::(_debug) Task 7a08ec02-40c4-4a3c-a48f-0c717c5afa09: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:00:56,641::task::495::TaskManager.Task::(_debug) Task 7a08ec02-40c4-4a3c-a48f-0c717c5afa09: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:00:56,642::task::495::TaskManager.Task::(_debug) Task 7a08ec02-40c4-4a3c-a48f-0c717c5afa09: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:00:56,643::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:00:56,644::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:00:56,646::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:00:56,647::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:00:56,648::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:00:56,649::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:00:56,651::task::495::TaskManager.Task::(_debug) Task 7a08ec02-40c4-4a3c-a48f-0c717c5afa09: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:00:56,653::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:00:56,654::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:00:56,655::task::495::TaskManager.Task::(_debug) Task e13eca69-e9e8-4f2b-91e8-648f0c727ac7: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:00:56,659::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f3ea338b-a8b4-4ef2-9f4e-30e641c6cb75`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:00:56,659::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:00:56,660::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:00:56,661::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f3ea338b-a8b4-4ef2-9f4e-30e641c6cb75`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:00:56,662::task::495::TaskManager.Task::(_debug) Task e13eca69-e9e8-4f2b-91e8-648f0c727ac7: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:00:56,663::task::495::TaskManager.Task::(_debug) Task e13eca69-e9e8-4f2b-91e8-648f0c727ac7: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:00:56,663::task::495::TaskManager.Task::(_debug) Task e13eca69-e9e8-4f2b-91e8-648f0c727ac7: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:00:56,664::task::495::TaskManager.Task::(_debug) Task e13eca69-e9e8-4f2b-91e8-648f0c727ac7: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:00:56,665::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:00:56,665::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:00:56,666::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:00:56,667::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:00:56,668::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:00:56,668::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:00:56,669::task::495::TaskManager.Task::(_debug) Task e13eca69-e9e8-4f2b-91e8-648f0c727ac7: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:00:56,670::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:00:56,974::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0993491 s, 10.3 MB/s\n'; <rc> = 0
>Thread-15915489::DEBUG::2013-11-20 04:00:58,948::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915489::INFO::2013-11-20 04:00:58,950::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915489::DEBUG::2013-11-20 04:00:58,951::task::495::TaskManager.Task::(_debug) Task 17b0074d-7b99-4457-8391-54445a844688: moving from state init -> state preparing
>Thread-15915489::DEBUG::2013-11-20 04:00:58,952::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915489::DEBUG::2013-11-20 04:00:58,952::task::495::TaskManager.Task::(_debug) Task 17b0074d-7b99-4457-8391-54445a844688: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915489::DEBUG::2013-11-20 04:00:58,953::task::495::TaskManager.Task::(_debug) Task 17b0074d-7b99-4457-8391-54445a844688: moving from state preparing -> state finished
>Thread-15915489::DEBUG::2013-11-20 04:00:58,954::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915489::DEBUG::2013-11-20 04:00:58,955::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915489::DEBUG::2013-11-20 04:00:58,955::task::495::TaskManager.Task::(_debug) Task 17b0074d-7b99-4457-8391-54445a844688: ref 0 aborting False
>Thread-15915489::INFO::2013-11-20 04:00:58,956::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915490::DEBUG::2013-11-20 04:00:58,961::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915490::INFO::2013-11-20 04:00:58,962::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915490::DEBUG::2013-11-20 04:00:58,963::task::495::TaskManager.Task::(_debug) Task a6564db8-3a9e-4851-a6f1-4c1d99b9b174: moving from state init -> state preparing
>Thread-15915490::DEBUG::2013-11-20 04:00:58,966::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`dfc54735-7752-4869-8edd-49904a6e48c8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915490::DEBUG::2013-11-20 04:00:58,967::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915490::DEBUG::2013-11-20 04:00:58,968::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915490::DEBUG::2013-11-20 04:00:58,970::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`dfc54735-7752-4869-8edd-49904a6e48c8`::Granted request
>Thread-15915490::DEBUG::2013-11-20 04:00:58,974::task::495::TaskManager.Task::(_debug) Task a6564db8-3a9e-4851-a6f1-4c1d99b9b174: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915490::DEBUG::2013-11-20 04:00:58,975::task::495::TaskManager.Task::(_debug) Task a6564db8-3a9e-4851-a6f1-4c1d99b9b174: ref 1 aborting False
>Thread-15915490::WARNING::2013-11-20 04:00:58,981::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915490::DEBUG::2013-11-20 04:00:58,988::task::495::TaskManager.Task::(_debug) Task a6564db8-3a9e-4851-a6f1-4c1d99b9b174: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915490::DEBUG::2013-11-20 04:00:58,990::task::495::TaskManager.Task::(_debug) Task a6564db8-3a9e-4851-a6f1-4c1d99b9b174: moving from state preparing -> state finished
>Thread-15915490::DEBUG::2013-11-20 04:00:58,992::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Dummy-15912651::DEBUG::2013-11-20 04:00:58,992::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915490::DEBUG::2013-11-20 04:00:58,993::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915490::DEBUG::2013-11-20 04:00:59,240::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915490::DEBUG::2013-11-20 04:00:59,243::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915490::DEBUG::2013-11-20 04:00:59,244::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915490::DEBUG::2013-11-20 04:00:59,245::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915490::DEBUG::2013-11-20 04:00:59,246::task::495::TaskManager.Task::(_debug) Task a6564db8-3a9e-4851-a6f1-4c1d99b9b174: ref 0 aborting False
>Thread-15915490::INFO::2013-11-20 04:00:59,247::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:00:59,527::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0506024 s, 20.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:01:01,536::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:01:02,134::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0953441 s, 10.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:01:04,146::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:01:04,731::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0928012 s, 11.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:01:06,745::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:01:07,280::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.05316 s, 19.3 MB/s\n'; <rc> = 0
>Thread-15915493::DEBUG::2013-11-20 04:01:07,432::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ilo,user=Administrator,passwd=XXXX,action=status,secure=,options=ssl=no)
>Thread-15915493::DEBUG::2013-11-20 04:01:09,065::clientIF::1219::vds::(fenceNode) rc 1 in agent=fence_ilo
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>ssl=no out  err Unable to connect/login to fencing device
>
>Thread-15915495::DEBUG::2013-11-20 04:01:09,072::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ilo,user=Administrator,passwd=XXXX,action=status,secure=,options=ssl=no)
>Dummy-15912651::DEBUG::2013-11-20 04:01:09,292::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915496::DEBUG::2013-11-20 04:01:09,730::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915496::INFO::2013-11-20 04:01:09,732::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915496::DEBUG::2013-11-20 04:01:09,732::task::495::TaskManager.Task::(_debug) Task 6a7c59af-ef9e-4120-bb68-deb23db17a1a: moving from state init -> state preparing
>Thread-15915496::DEBUG::2013-11-20 04:01:09,733::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915496::DEBUG::2013-11-20 04:01:09,734::task::495::TaskManager.Task::(_debug) Task 6a7c59af-ef9e-4120-bb68-deb23db17a1a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915496::DEBUG::2013-11-20 04:01:09,735::task::495::TaskManager.Task::(_debug) Task 6a7c59af-ef9e-4120-bb68-deb23db17a1a: moving from state preparing -> state finished
>Thread-15915496::DEBUG::2013-11-20 04:01:09,735::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915496::DEBUG::2013-11-20 04:01:09,736::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915496::DEBUG::2013-11-20 04:01:09,736::task::495::TaskManager.Task::(_debug) Task 6a7c59af-ef9e-4120-bb68-deb23db17a1a: ref 0 aborting False
>Thread-15915496::INFO::2013-11-20 04:01:09,737::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915497::DEBUG::2013-11-20 04:01:09,742::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915497::INFO::2013-11-20 04:01:09,743::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915497::DEBUG::2013-11-20 04:01:09,744::task::495::TaskManager.Task::(_debug) Task 12a7c332-7e4d-45b3-9bbe-d0a1b916cac3: moving from state init -> state preparing
>Thread-15915497::DEBUG::2013-11-20 04:01:09,745::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8b56421a-b8b1-4dd2-8e9f-d4ee74f7221c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915497::DEBUG::2013-11-20 04:01:09,746::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915497::DEBUG::2013-11-20 04:01:09,747::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915497::DEBUG::2013-11-20 04:01:09,748::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8b56421a-b8b1-4dd2-8e9f-d4ee74f7221c`::Granted request
>Thread-15915497::DEBUG::2013-11-20 04:01:09,749::task::495::TaskManager.Task::(_debug) Task 12a7c332-7e4d-45b3-9bbe-d0a1b916cac3: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915497::DEBUG::2013-11-20 04:01:09,750::task::495::TaskManager.Task::(_debug) Task 12a7c332-7e4d-45b3-9bbe-d0a1b916cac3: ref 1 aborting False
>Thread-15915497::WARNING::2013-11-20 04:01:09,753::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915497::DEBUG::2013-11-20 04:01:09,755::task::495::TaskManager.Task::(_debug) Task 12a7c332-7e4d-45b3-9bbe-d0a1b916cac3: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915497::DEBUG::2013-11-20 04:01:09,756::task::495::TaskManager.Task::(_debug) Task 12a7c332-7e4d-45b3-9bbe-d0a1b916cac3: moving from state preparing -> state finished
>Thread-15915497::DEBUG::2013-11-20 04:01:09,757::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915497::DEBUG::2013-11-20 04:01:09,757::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915497::DEBUG::2013-11-20 04:01:09,758::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915497::DEBUG::2013-11-20 04:01:09,759::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915497::DEBUG::2013-11-20 04:01:09,759::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915497::DEBUG::2013-11-20 04:01:09,760::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915497::DEBUG::2013-11-20 04:01:09,761::task::495::TaskManager.Task::(_debug) Task 12a7c332-7e4d-45b3-9bbe-d0a1b916cac3: ref 0 aborting False
>Thread-15915497::INFO::2013-11-20 04:01:09,762::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:01:10,161::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.165679 s, 6.2 MB/s\n'; <rc> = 0
>Thread-15915498::INFO::2013-11-20 04:01:10,674::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915498::DEBUG::2013-11-20 04:01:10,675::task::495::TaskManager.Task::(_debug) Task 1e4db3dd-3ecd-4d3f-9542-f13be9dd44cc: moving from state init -> state preparing
>Thread-15915498::DEBUG::2013-11-20 04:01:10,676::task::495::TaskManager.Task::(_debug) Task 1e4db3dd-3ecd-4d3f-9542-f13be9dd44cc: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000695943832397', 'lastCheck': 1384920070.0424161, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00135612487793', 'lastCheck': 1384920060.6041739, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0288028717041', 'lastCheck': 1384920068.0238011, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000323057174683', 'lastCheck': 1384920064.7713959, 'valid': True, 'code': 0}}
>Thread-15915498::DEBUG::2013-11-20 04:01:10,678::task::495::TaskManager.Task::(_debug) Task 1e4db3dd-3ecd-4d3f-9542-f13be9dd44cc: moving from state preparing -> state finished
>Thread-15915498::DEBUG::2013-11-20 04:01:10,679::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915498::DEBUG::2013-11-20 04:01:10,680::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915498::DEBUG::2013-11-20 04:01:10,681::task::495::TaskManager.Task::(_debug) Task 1e4db3dd-3ecd-4d3f-9542-f13be9dd44cc: ref 0 aborting False
>Thread-15915498::INFO::2013-11-20 04:01:10,682::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000695943832397', 'lastCheck': 1384920070.0424161, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00135612487793', 'lastCheck': 1384920060.6041739, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0288028717041', 'lastCheck': 1384920068.0238011, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000323057174683', 'lastCheck': 1384920064.7713959, 'valid': True, 'code': 0}}
>Thread-15915495::DEBUG::2013-11-20 04:01:10,886::clientIF::1219::vds::(fenceNode) rc 1 in agent=fence_ilo
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>ssl=no out  err Unable to connect/login to fencing device
>
>Thread-15913518::INFO::2013-11-20 04:01:11,760::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:01:11,761::task::495::TaskManager.Task::(_debug) Task f68c0ed3-1496-41cf-a616-3bf40ddd1ff1: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:01:11,762::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8b5db46c-a1eb-4a86-a50b-b1aa7729a8b1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:01:11,763::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:01:11,764::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:01:11,764::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8b5db46c-a1eb-4a86-a50b-b1aa7729a8b1`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:01:11,765::task::495::TaskManager.Task::(_debug) Task f68c0ed3-1496-41cf-a616-3bf40ddd1ff1: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:01:11,766::task::495::TaskManager.Task::(_debug) Task f68c0ed3-1496-41cf-a616-3bf40ddd1ff1: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:01:11,767::task::495::TaskManager.Task::(_debug) Task f68c0ed3-1496-41cf-a616-3bf40ddd1ff1: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:01:11,767::task::495::TaskManager.Task::(_debug) Task f68c0ed3-1496-41cf-a616-3bf40ddd1ff1: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:01:11,768::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:01:11,769::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:01:11,769::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:01:11,770::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:01:11,771::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:01:11,771::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:01:11,772::task::495::TaskManager.Task::(_debug) Task f68c0ed3-1496-41cf-a616-3bf40ddd1ff1: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:01:11,773::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:01:11,773::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:01:11,774::task::495::TaskManager.Task::(_debug) Task fbe63b51-88fd-46a0-973f-eb26693ef635: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:01:11,775::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`264ab6e9-712f-406d-b0e2-28f22779b9ac`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:01:11,776::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:01:11,777::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:01:11,777::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`264ab6e9-712f-406d-b0e2-28f22779b9ac`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:01:11,778::task::495::TaskManager.Task::(_debug) Task fbe63b51-88fd-46a0-973f-eb26693ef635: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:01:11,779::task::495::TaskManager.Task::(_debug) Task fbe63b51-88fd-46a0-973f-eb26693ef635: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:01:11,780::task::495::TaskManager.Task::(_debug) Task fbe63b51-88fd-46a0-973f-eb26693ef635: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:01:11,780::task::495::TaskManager.Task::(_debug) Task fbe63b51-88fd-46a0-973f-eb26693ef635: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:01:11,781::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:01:11,782::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:01:11,783::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:01:11,784::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:01:11,784::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:01:11,785::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:01:11,786::task::495::TaskManager.Task::(_debug) Task fbe63b51-88fd-46a0-973f-eb26693ef635: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:01:11,786::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913639::INFO::2013-11-20 04:01:12,027::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:01:12,029::task::495::TaskManager.Task::(_debug) Task 645e0966-d9f1-4959-acf8-db9d79e8c7ae: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:01:12,032::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a07499df-1e7e-45dd-b2da-c03ef52347a3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:01:12,033::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:01:12,035::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:01:12,035::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a07499df-1e7e-45dd-b2da-c03ef52347a3`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:01:12,037::task::495::TaskManager.Task::(_debug) Task 645e0966-d9f1-4959-acf8-db9d79e8c7ae: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:01:12,038::task::495::TaskManager.Task::(_debug) Task 645e0966-d9f1-4959-acf8-db9d79e8c7ae: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:01:12,038::task::495::TaskManager.Task::(_debug) Task 645e0966-d9f1-4959-acf8-db9d79e8c7ae: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:01:12,039::task::495::TaskManager.Task::(_debug) Task 645e0966-d9f1-4959-acf8-db9d79e8c7ae: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:01:12,040::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:01:12,040::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:01:12,041::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:01:12,042::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:01:12,042::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:01:12,043::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:01:12,044::task::495::TaskManager.Task::(_debug) Task 645e0966-d9f1-4959-acf8-db9d79e8c7ae: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:01:12,044::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:01:12,177::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:01:12,757::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0749027 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:01:14,773::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:01:15,365::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0864902 s, 11.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:01:17,381::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:01:17,983::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0805208 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:01:20,003::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915502::DEBUG::2013-11-20 04:01:20,334::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915502::INFO::2013-11-20 04:01:20,337::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915502::DEBUG::2013-11-20 04:01:20,338::task::495::TaskManager.Task::(_debug) Task 86ef8078-f8a0-4952-82ef-bc53d965dd89: moving from state init -> state preparing
>Thread-15915502::DEBUG::2013-11-20 04:01:20,339::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915502::DEBUG::2013-11-20 04:01:20,339::task::495::TaskManager.Task::(_debug) Task 86ef8078-f8a0-4952-82ef-bc53d965dd89: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915502::DEBUG::2013-11-20 04:01:20,340::task::495::TaskManager.Task::(_debug) Task 86ef8078-f8a0-4952-82ef-bc53d965dd89: moving from state preparing -> state finished
>Thread-15915502::DEBUG::2013-11-20 04:01:20,341::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915502::DEBUG::2013-11-20 04:01:20,342::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915502::DEBUG::2013-11-20 04:01:20,343::task::495::TaskManager.Task::(_debug) Task 86ef8078-f8a0-4952-82ef-bc53d965dd89: ref 0 aborting False
>Thread-15915502::INFO::2013-11-20 04:01:20,343::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915503::DEBUG::2013-11-20 04:01:20,349::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915503::INFO::2013-11-20 04:01:20,351::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915503::DEBUG::2013-11-20 04:01:20,351::task::495::TaskManager.Task::(_debug) Task da985cea-050b-40a7-859c-9198d6fc876a: moving from state init -> state preparing
>Thread-15915503::DEBUG::2013-11-20 04:01:20,354::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2b4b11da-6135-44c1-acf9-0b45d4aa8d8a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915503::DEBUG::2013-11-20 04:01:20,354::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915503::DEBUG::2013-11-20 04:01:20,356::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915503::DEBUG::2013-11-20 04:01:20,356::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2b4b11da-6135-44c1-acf9-0b45d4aa8d8a`::Granted request
>Thread-15915503::DEBUG::2013-11-20 04:01:20,358::task::495::TaskManager.Task::(_debug) Task da985cea-050b-40a7-859c-9198d6fc876a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915503::DEBUG::2013-11-20 04:01:20,359::task::495::TaskManager.Task::(_debug) Task da985cea-050b-40a7-859c-9198d6fc876a: ref 1 aborting False
>Thread-15915503::WARNING::2013-11-20 04:01:20,362::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915503::DEBUG::2013-11-20 04:01:20,363::task::495::TaskManager.Task::(_debug) Task da985cea-050b-40a7-859c-9198d6fc876a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915503::DEBUG::2013-11-20 04:01:20,364::task::495::TaskManager.Task::(_debug) Task da985cea-050b-40a7-859c-9198d6fc876a: moving from state preparing -> state finished
>Thread-15915503::DEBUG::2013-11-20 04:01:20,365::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915503::DEBUG::2013-11-20 04:01:20,365::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915503::DEBUG::2013-11-20 04:01:20,366::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915503::DEBUG::2013-11-20 04:01:20,367::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915503::DEBUG::2013-11-20 04:01:20,367::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915503::DEBUG::2013-11-20 04:01:20,368::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915503::DEBUG::2013-11-20 04:01:20,369::task::495::TaskManager.Task::(_debug) Task da985cea-050b-40a7-859c-9198d6fc876a: ref 0 aborting False
>Thread-15915503::INFO::2013-11-20 04:01:20,369::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:01:20,579::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0769786 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:01:22,591::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:01:23,171::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0918015 s, 11.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:01:25,187::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:01:25,785::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.104374 s, 9.8 MB/s\n'; <rc> = 0
>Thread-15915506::INFO::2013-11-20 04:01:26,769::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915506::DEBUG::2013-11-20 04:01:26,771::task::495::TaskManager.Task::(_debug) Task 98d2fe30-7e6c-4e6e-958b-784408c8b687: moving from state init -> state preparing
>Thread-15915506::DEBUG::2013-11-20 04:01:26,771::task::495::TaskManager.Task::(_debug) Task 98d2fe30-7e6c-4e6e-958b-784408c8b687: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000576972961426', 'lastCheck': 1384920080.2269681, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00143384933472', 'lastCheck': 1384920081.002708, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00127100944519', 'lastCheck': 1384920078.0446579, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000393152236938', 'lastCheck': 1384920084.9935491, 'valid': True, 'code': 0}}
>Thread-15915506::DEBUG::2013-11-20 04:01:26,773::task::495::TaskManager.Task::(_debug) Task 98d2fe30-7e6c-4e6e-958b-784408c8b687: moving from state preparing -> state finished
>Thread-15915506::DEBUG::2013-11-20 04:01:26,773::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915506::DEBUG::2013-11-20 04:01:26,774::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915506::DEBUG::2013-11-20 04:01:26,774::task::495::TaskManager.Task::(_debug) Task 98d2fe30-7e6c-4e6e-958b-784408c8b687: ref 0 aborting False
>Thread-15915506::INFO::2013-11-20 04:01:26,775::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000576972961426', 'lastCheck': 1384920080.2269681, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00143384933472', 'lastCheck': 1384920081.002708, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00127100944519', 'lastCheck': 1384920078.0446579, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000393152236938', 'lastCheck': 1384920084.9935491, 'valid': True, 'code': 0}}
>Thread-15913658::INFO::2013-11-20 04:01:27,042::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:01:27,043::task::495::TaskManager.Task::(_debug) Task 4140f82f-9625-4704-aacc-08173d87cfa0: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:01:27,045::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6a870d6b-d532-40ea-a722-b1ff15214e39`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:01:27,047::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:01:27,048::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:01:27,048::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6a870d6b-d532-40ea-a722-b1ff15214e39`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:01:27,051::task::495::TaskManager.Task::(_debug) Task 4140f82f-9625-4704-aacc-08173d87cfa0: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:01:27,051::task::495::TaskManager.Task::(_debug) Task 4140f82f-9625-4704-aacc-08173d87cfa0: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:01:27,052::task::495::TaskManager.Task::(_debug) Task 4140f82f-9625-4704-aacc-08173d87cfa0: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:01:27,053::task::495::TaskManager.Task::(_debug) Task 4140f82f-9625-4704-aacc-08173d87cfa0: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:01:27,054::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:01:27,054::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:01:27,055::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:01:27,056::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:01:27,056::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:01:27,057::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:01:27,058::task::495::TaskManager.Task::(_debug) Task 4140f82f-9625-4704-aacc-08173d87cfa0: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:01:27,058::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:01:27,802::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:01:28,449::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0944615 s, 10.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:01:30,474::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:01:30,979::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0533548 s, 19.2 MB/s\n'; <rc> = 0
>Thread-15915509::DEBUG::2013-11-20 04:01:31,181::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915509::INFO::2013-11-20 04:01:31,183::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915509::DEBUG::2013-11-20 04:01:31,184::task::495::TaskManager.Task::(_debug) Task a9313d1c-9c70-42ed-b846-d639ae862523: moving from state init -> state preparing
>Thread-15915509::DEBUG::2013-11-20 04:01:31,185::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915509::DEBUG::2013-11-20 04:01:31,185::task::495::TaskManager.Task::(_debug) Task a9313d1c-9c70-42ed-b846-d639ae862523: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915509::DEBUG::2013-11-20 04:01:31,186::task::495::TaskManager.Task::(_debug) Task a9313d1c-9c70-42ed-b846-d639ae862523: moving from state preparing -> state finished
>Thread-15915509::DEBUG::2013-11-20 04:01:31,187::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915509::DEBUG::2013-11-20 04:01:31,187::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915509::DEBUG::2013-11-20 04:01:31,188::task::495::TaskManager.Task::(_debug) Task a9313d1c-9c70-42ed-b846-d639ae862523: ref 0 aborting False
>Thread-15915509::INFO::2013-11-20 04:01:31,189::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915510::DEBUG::2013-11-20 04:01:31,194::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915510::INFO::2013-11-20 04:01:31,196::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915510::DEBUG::2013-11-20 04:01:31,197::task::495::TaskManager.Task::(_debug) Task 9e0d4fec-2e3f-430f-9599-e14129a03492: moving from state init -> state preparing
>Thread-15915510::DEBUG::2013-11-20 04:01:31,198::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a60eab35-e925-43b6-8b65-d3c260865124`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915510::DEBUG::2013-11-20 04:01:31,199::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915510::DEBUG::2013-11-20 04:01:31,200::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915510::DEBUG::2013-11-20 04:01:31,201::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a60eab35-e925-43b6-8b65-d3c260865124`::Granted request
>Thread-15915510::DEBUG::2013-11-20 04:01:31,203::task::495::TaskManager.Task::(_debug) Task 9e0d4fec-2e3f-430f-9599-e14129a03492: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915510::DEBUG::2013-11-20 04:01:31,204::task::495::TaskManager.Task::(_debug) Task 9e0d4fec-2e3f-430f-9599-e14129a03492: ref 1 aborting False
>Thread-15915510::WARNING::2013-11-20 04:01:31,207::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915510::DEBUG::2013-11-20 04:01:31,208::task::495::TaskManager.Task::(_debug) Task 9e0d4fec-2e3f-430f-9599-e14129a03492: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915510::DEBUG::2013-11-20 04:01:31,209::task::495::TaskManager.Task::(_debug) Task 9e0d4fec-2e3f-430f-9599-e14129a03492: moving from state preparing -> state finished
>Thread-15915510::DEBUG::2013-11-20 04:01:31,210::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915510::DEBUG::2013-11-20 04:01:31,211::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915510::DEBUG::2013-11-20 04:01:31,212::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915510::DEBUG::2013-11-20 04:01:31,213::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915510::DEBUG::2013-11-20 04:01:31,214::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915510::DEBUG::2013-11-20 04:01:31,214::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915510::DEBUG::2013-11-20 04:01:31,215::task::495::TaskManager.Task::(_debug) Task 9e0d4fec-2e3f-430f-9599-e14129a03492: ref 0 aborting False
>Thread-15915510::INFO::2013-11-20 04:01:31,216::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:01:32,994::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:01:33,528::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0808026 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:01:35,548::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:01:36,127::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0803607 s, 12.7 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 04:01:37,214::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:01:37,216::task::495::TaskManager.Task::(_debug) Task d4fd4706-378f-4a2a-9e30-5f2023464e9a: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:01:37,219::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`67e32679-b116-44ec-abf4-3aa8bbd0206b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:01:37,221::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:01:37,221::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:01:37,223::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`67e32679-b116-44ec-abf4-3aa8bbd0206b`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:01:37,224::task::495::TaskManager.Task::(_debug) Task d4fd4706-378f-4a2a-9e30-5f2023464e9a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:01:37,224::task::495::TaskManager.Task::(_debug) Task d4fd4706-378f-4a2a-9e30-5f2023464e9a: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:01:37,225::task::495::TaskManager.Task::(_debug) Task d4fd4706-378f-4a2a-9e30-5f2023464e9a: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:01:37,226::task::495::TaskManager.Task::(_debug) Task d4fd4706-378f-4a2a-9e30-5f2023464e9a: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:01:37,227::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:01:37,227::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:01:37,228::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:01:37,229::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:01:37,229::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:01:37,230::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:01:37,231::task::495::TaskManager.Task::(_debug) Task d4fd4706-378f-4a2a-9e30-5f2023464e9a: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:01:37,231::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:01:38,143::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:01:38,692::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0794498 s, 12.9 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 04:01:40,480::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:01:40,481::task::495::TaskManager.Task::(_debug) Task bda27699-4300-4003-8dd0-f94e534389de: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:01:40,483::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c935730d-57ff-4472-a64d-b0445852c0f8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:01:40,484::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:01:40,485::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:01:40,485::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c935730d-57ff-4472-a64d-b0445852c0f8`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:01:40,486::task::495::TaskManager.Task::(_debug) Task bda27699-4300-4003-8dd0-f94e534389de: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:01:40,487::task::495::TaskManager.Task::(_debug) Task bda27699-4300-4003-8dd0-f94e534389de: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:01:40,488::task::495::TaskManager.Task::(_debug) Task bda27699-4300-4003-8dd0-f94e534389de: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:01:40,489::task::495::TaskManager.Task::(_debug) Task bda27699-4300-4003-8dd0-f94e534389de: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:01:40,490::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:01:40,490::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:01:40,491::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:01:40,492::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:01:40,493::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:01:40,493::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:01:40,494::task::495::TaskManager.Task::(_debug) Task bda27699-4300-4003-8dd0-f94e534389de: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:01:40,495::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:01:40,706::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:01:41,244::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0831297 s, 12.3 MB/s\n'; <rc> = 0
>Thread-15915514::DEBUG::2013-11-20 04:01:41,580::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915514::INFO::2013-11-20 04:01:41,581::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915514::DEBUG::2013-11-20 04:01:41,582::task::495::TaskManager.Task::(_debug) Task 53aeb43c-8921-4e1f-b758-7dea996d9962: moving from state init -> state preparing
>Thread-15915514::DEBUG::2013-11-20 04:01:41,583::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915514::DEBUG::2013-11-20 04:01:41,583::task::495::TaskManager.Task::(_debug) Task 53aeb43c-8921-4e1f-b758-7dea996d9962: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915514::DEBUG::2013-11-20 04:01:41,584::task::495::TaskManager.Task::(_debug) Task 53aeb43c-8921-4e1f-b758-7dea996d9962: moving from state preparing -> state finished
>Thread-15915514::DEBUG::2013-11-20 04:01:41,585::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915514::DEBUG::2013-11-20 04:01:41,585::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915514::DEBUG::2013-11-20 04:01:41,586::task::495::TaskManager.Task::(_debug) Task 53aeb43c-8921-4e1f-b758-7dea996d9962: ref 0 aborting False
>Thread-15915514::INFO::2013-11-20 04:01:41,586::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915515::DEBUG::2013-11-20 04:01:41,592::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915515::INFO::2013-11-20 04:01:41,594::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915515::DEBUG::2013-11-20 04:01:41,595::task::495::TaskManager.Task::(_debug) Task 26d5405e-db00-4258-abe9-e2de7e2b0e0f: moving from state init -> state preparing
>Thread-15915515::DEBUG::2013-11-20 04:01:41,597::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`881bc545-b04a-4936-93ae-b036eab28345`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915515::DEBUG::2013-11-20 04:01:41,598::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915515::DEBUG::2013-11-20 04:01:41,599::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915515::DEBUG::2013-11-20 04:01:41,600::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`881bc545-b04a-4936-93ae-b036eab28345`::Granted request
>Thread-15915515::DEBUG::2013-11-20 04:01:41,601::task::495::TaskManager.Task::(_debug) Task 26d5405e-db00-4258-abe9-e2de7e2b0e0f: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915515::DEBUG::2013-11-20 04:01:41,602::task::495::TaskManager.Task::(_debug) Task 26d5405e-db00-4258-abe9-e2de7e2b0e0f: ref 1 aborting False
>Thread-15915515::WARNING::2013-11-20 04:01:41,605::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915515::DEBUG::2013-11-20 04:01:41,606::task::495::TaskManager.Task::(_debug) Task 26d5405e-db00-4258-abe9-e2de7e2b0e0f: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915515::DEBUG::2013-11-20 04:01:41,608::task::495::TaskManager.Task::(_debug) Task 26d5405e-db00-4258-abe9-e2de7e2b0e0f: moving from state preparing -> state finished
>Thread-15915515::DEBUG::2013-11-20 04:01:41,608::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915515::DEBUG::2013-11-20 04:01:41,609::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915515::DEBUG::2013-11-20 04:01:41,610::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915515::DEBUG::2013-11-20 04:01:41,611::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915515::DEBUG::2013-11-20 04:01:41,612::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915515::DEBUG::2013-11-20 04:01:41,613::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915515::DEBUG::2013-11-20 04:01:41,614::task::495::TaskManager.Task::(_debug) Task 26d5405e-db00-4258-abe9-e2de7e2b0e0f: ref 0 aborting False
>Thread-15915515::INFO::2013-11-20 04:01:41,615::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915085::INFO::2013-11-20 04:01:42,356::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=089b08fc-6544-4235-a061-fb55b09d2715 volUUID=66b599fc-b7b5-4934-a5d5-ef7283460029)
>Thread-15915085::DEBUG::2013-11-20 04:01:42,357::task::495::TaskManager.Task::(_debug) Task 3486bc0a-1416-4f77-a3f0-e8057801ad0b: moving from state init -> state preparing
>Thread-15915085::DEBUG::2013-11-20 04:01:42,358::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f534770d-f91f-4f69-a6f7-a46894baa884`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915085::DEBUG::2013-11-20 04:01:42,358::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915085::DEBUG::2013-11-20 04:01:42,359::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915085::DEBUG::2013-11-20 04:01:42,359::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f534770d-f91f-4f69-a6f7-a46894baa884`::Granted request
>Thread-15915085::DEBUG::2013-11-20 04:01:42,360::task::495::TaskManager.Task::(_debug) Task 3486bc0a-1416-4f77-a3f0-e8057801ad0b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915085::DEBUG::2013-11-20 04:01:42,361::task::495::TaskManager.Task::(_debug) Task 3486bc0a-1416-4f77-a3f0-e8057801ad0b: ref 1 aborting False
>Thread-15915085::DEBUG::2013-11-20 04:01:42,361::task::495::TaskManager.Task::(_debug) Task 3486bc0a-1416-4f77-a3f0-e8057801ad0b: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15915085::DEBUG::2013-11-20 04:01:42,362::task::495::TaskManager.Task::(_debug) Task 3486bc0a-1416-4f77-a3f0-e8057801ad0b: moving from state preparing -> state finished
>Thread-15915085::DEBUG::2013-11-20 04:01:42,362::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915085::DEBUG::2013-11-20 04:01:42,363::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915085::DEBUG::2013-11-20 04:01:42,363::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915085::DEBUG::2013-11-20 04:01:42,364::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915085::DEBUG::2013-11-20 04:01:42,364::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915085::DEBUG::2013-11-20 04:01:42,365::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915085::DEBUG::2013-11-20 04:01:42,365::task::495::TaskManager.Task::(_debug) Task 3486bc0a-1416-4f77-a3f0-e8057801ad0b: ref 0 aborting False
>Thread-15915085::INFO::2013-11-20 04:01:42,366::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15915516::INFO::2013-11-20 04:01:42,803::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915516::DEBUG::2013-11-20 04:01:42,804::task::495::TaskManager.Task::(_debug) Task 939a7b53-ed8b-44f0-8acd-32f13e20f8ac: moving from state init -> state preparing
>Thread-15915516::DEBUG::2013-11-20 04:01:42,805::task::495::TaskManager.Task::(_debug) Task 939a7b53-ed8b-44f0-8acd-32f13e20f8ac: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000305891036987', 'lastCheck': 1384920100.4026201, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00130200386047', 'lastCheck': 1384920101.359679, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00137805938721', 'lastCheck': 1384920098.4127641, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0003981590271', 'lastCheck': 1384920095.216476, 'valid': True, 'code': 0}}
>Thread-15915516::DEBUG::2013-11-20 04:01:42,806::task::495::TaskManager.Task::(_debug) Task 939a7b53-ed8b-44f0-8acd-32f13e20f8ac: moving from state preparing -> state finished
>Thread-15915516::DEBUG::2013-11-20 04:01:42,807::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915516::DEBUG::2013-11-20 04:01:42,808::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915516::DEBUG::2013-11-20 04:01:42,809::task::495::TaskManager.Task::(_debug) Task 939a7b53-ed8b-44f0-8acd-32f13e20f8ac: ref 0 aborting False
>Thread-15915516::INFO::2013-11-20 04:01:42,809::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000305891036987', 'lastCheck': 1384920100.4026201, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00130200386047', 'lastCheck': 1384920101.359679, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00137805938721', 'lastCheck': 1384920098.4127641, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0003981590271', 'lastCheck': 1384920095.216476, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:01:43,258::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:01:43,817::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0908455 s, 11.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:01:45,831::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:01:46,380::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0819222 s, 12.5 MB/s\n'; <rc> = 0
>Thread-15914453::INFO::2013-11-20 04:01:47,235::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:01:47,239::task::495::TaskManager.Task::(_debug) Task 5d989984-a4ec-42ff-969f-70f3cdf3c103: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:01:47,242::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2838cf9a-638f-48ec-bc68-89d740fe4ba6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:01:47,243::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:01:47,244::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:01:47,245::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2838cf9a-638f-48ec-bc68-89d740fe4ba6`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:01:47,246::task::495::TaskManager.Task::(_debug) Task 5d989984-a4ec-42ff-969f-70f3cdf3c103: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:01:47,247::task::495::TaskManager.Task::(_debug) Task 5d989984-a4ec-42ff-969f-70f3cdf3c103: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:01:47,248::task::495::TaskManager.Task::(_debug) Task 5d989984-a4ec-42ff-969f-70f3cdf3c103: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:01:47,248::task::495::TaskManager.Task::(_debug) Task 5d989984-a4ec-42ff-969f-70f3cdf3c103: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:01:47,249::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:01:47,249::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:01:47,250::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:01:47,251::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:01:47,252::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:01:47,252::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:01:47,253::task::495::TaskManager.Task::(_debug) Task 5d989984-a4ec-42ff-969f-70f3cdf3c103: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:01:47,254::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:01:48,400::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:01:48,946::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0651732 s, 15.7 MB/s\n'; <rc> = 0
>Thread-15914435::INFO::2013-11-20 04:01:50,203::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:01:50,204::task::495::TaskManager.Task::(_debug) Task 7a01ff34-4fa8-48b7-8177-8444c5852965: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:01:50,206::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1f1b37fe-c4ec-4f86-b55a-390386e0f4dc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:01:50,207::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:01:50,208::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:01:50,208::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1f1b37fe-c4ec-4f86-b55a-390386e0f4dc`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:01:50,210::task::495::TaskManager.Task::(_debug) Task 7a01ff34-4fa8-48b7-8177-8444c5852965: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:01:50,210::task::495::TaskManager.Task::(_debug) Task 7a01ff34-4fa8-48b7-8177-8444c5852965: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:01:50,211::task::495::TaskManager.Task::(_debug) Task 7a01ff34-4fa8-48b7-8177-8444c5852965: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:01:50,212::task::495::TaskManager.Task::(_debug) Task 7a01ff34-4fa8-48b7-8177-8444c5852965: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:01:50,213::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:01:50,214::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:01:50,215::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:01:50,215::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:01:50,216::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:01:50,217::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:01:50,218::task::495::TaskManager.Task::(_debug) Task 7a01ff34-4fa8-48b7-8177-8444c5852965: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:01:50,219::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:01:50,960::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:01:51,473::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0815129 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15915520::DEBUG::2013-11-20 04:01:51,973::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915520::INFO::2013-11-20 04:01:51,974::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915520::DEBUG::2013-11-20 04:01:51,975::task::495::TaskManager.Task::(_debug) Task 1503edba-92f6-4668-80d6-37856d91bb63: moving from state init -> state preparing
>Thread-15915520::DEBUG::2013-11-20 04:01:51,975::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915520::DEBUG::2013-11-20 04:01:51,976::task::495::TaskManager.Task::(_debug) Task 1503edba-92f6-4668-80d6-37856d91bb63: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915520::DEBUG::2013-11-20 04:01:51,976::task::495::TaskManager.Task::(_debug) Task 1503edba-92f6-4668-80d6-37856d91bb63: moving from state preparing -> state finished
>Thread-15915520::DEBUG::2013-11-20 04:01:51,976::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915520::DEBUG::2013-11-20 04:01:51,977::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915520::DEBUG::2013-11-20 04:01:51,977::task::495::TaskManager.Task::(_debug) Task 1503edba-92f6-4668-80d6-37856d91bb63: ref 0 aborting False
>Thread-15915520::INFO::2013-11-20 04:01:51,978::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915521::DEBUG::2013-11-20 04:01:51,981::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915521::INFO::2013-11-20 04:01:51,982::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915521::DEBUG::2013-11-20 04:01:51,982::task::495::TaskManager.Task::(_debug) Task 3f3b9b93-a7a1-494d-bf0c-ac1c75fd8ebe: moving from state init -> state preparing
>Thread-15915521::DEBUG::2013-11-20 04:01:51,983::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`46002935-4a6a-40e3-9c09-445013fef788`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915521::DEBUG::2013-11-20 04:01:51,983::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915521::DEBUG::2013-11-20 04:01:51,984::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915521::DEBUG::2013-11-20 04:01:51,984::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`46002935-4a6a-40e3-9c09-445013fef788`::Granted request
>Thread-15915521::DEBUG::2013-11-20 04:01:51,985::task::495::TaskManager.Task::(_debug) Task 3f3b9b93-a7a1-494d-bf0c-ac1c75fd8ebe: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915521::DEBUG::2013-11-20 04:01:51,985::task::495::TaskManager.Task::(_debug) Task 3f3b9b93-a7a1-494d-bf0c-ac1c75fd8ebe: ref 1 aborting False
>Thread-15915521::WARNING::2013-11-20 04:01:51,986::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915521::DEBUG::2013-11-20 04:01:51,987::task::495::TaskManager.Task::(_debug) Task 3f3b9b93-a7a1-494d-bf0c-ac1c75fd8ebe: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915521::DEBUG::2013-11-20 04:01:51,988::task::495::TaskManager.Task::(_debug) Task 3f3b9b93-a7a1-494d-bf0c-ac1c75fd8ebe: moving from state preparing -> state finished
>Thread-15915521::DEBUG::2013-11-20 04:01:51,988::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915521::DEBUG::2013-11-20 04:01:51,989::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915521::DEBUG::2013-11-20 04:01:51,989::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915521::DEBUG::2013-11-20 04:01:51,989::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915521::DEBUG::2013-11-20 04:01:51,990::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915521::DEBUG::2013-11-20 04:01:51,990::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915521::DEBUG::2013-11-20 04:01:51,991::task::495::TaskManager.Task::(_debug) Task 3f3b9b93-a7a1-494d-bf0c-ac1c75fd8ebe: ref 0 aborting False
>Thread-15915521::INFO::2013-11-20 04:01:51,991::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915523::DEBUG::2013-11-20 04:01:53,354::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=ssl=no)
>Dummy-15912651::DEBUG::2013-11-20 04:01:53,621::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:01:54,220::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.077829 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15914519::INFO::2013-11-20 04:01:54,899::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:01:54,900::task::495::TaskManager.Task::(_debug) Task 81641ad4-befd-4205-9a1a-211d2380da92: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:01:54,906::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f0048d48-78cd-49dd-896b-ba9725198105`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:01:54,907::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:01:54,908::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:01:54,909::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f0048d48-78cd-49dd-896b-ba9725198105`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:01:54,911::task::495::TaskManager.Task::(_debug) Task 81641ad4-befd-4205-9a1a-211d2380da92: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 04:01:54,914::task::495::TaskManager.Task::(_debug) Task 81641ad4-befd-4205-9a1a-211d2380da92: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:01:54,915::task::495::TaskManager.Task::(_debug) Task 81641ad4-befd-4205-9a1a-211d2380da92: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 04:01:54,916::task::495::TaskManager.Task::(_debug) Task 81641ad4-befd-4205-9a1a-211d2380da92: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:01:54,917::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:01:54,918::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:01:54,923::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 04:01:54,924::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:01:54,924::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:01:54,925::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:01:54,926::task::495::TaskManager.Task::(_debug) Task 81641ad4-befd-4205-9a1a-211d2380da92: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 04:01:54,926::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15915523::DEBUG::2013-11-20 04:01:54,983::clientIF::1219::vds::(fenceNode) rc 1 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>ssl=no out Getting status of IPMI:192.168.100.119...Chassis power = Unknown
>Failed
> err 
>Thread-15915524::DEBUG::2013-11-20 04:01:54,989::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=ssl=no)
>Dummy-15912651::DEBUG::2013-11-20 04:01:56,236::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915524::DEBUG::2013-11-20 04:01:56,498::clientIF::1219::vds::(fenceNode) rc 1 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>ssl=no out Getting status of IPMI:192.168.100.119...Chassis power = Unknown
>Failed
> err 
>Dummy-15912651::DEBUG::2013-11-20 04:01:56,834::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0789196 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 04:01:57,259::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:01:57,260::task::495::TaskManager.Task::(_debug) Task cf632ad0-8e00-44ff-a457-e42cf62b271b: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:01:57,269::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`9fd54e11-b060-4d1a-8a9b-c3701f563ba1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:01:57,270::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:01:57,271::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:01:57,272::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`9fd54e11-b060-4d1a-8a9b-c3701f563ba1`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:01:57,274::task::495::TaskManager.Task::(_debug) Task cf632ad0-8e00-44ff-a457-e42cf62b271b: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:01:57,275::task::495::TaskManager.Task::(_debug) Task cf632ad0-8e00-44ff-a457-e42cf62b271b: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:01:57,276::task::495::TaskManager.Task::(_debug) Task cf632ad0-8e00-44ff-a457-e42cf62b271b: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 04:01:57,277::task::495::TaskManager.Task::(_debug) Task cf632ad0-8e00-44ff-a457-e42cf62b271b: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:01:57,277::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:01:57,278::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:01:57,279::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:01:57,280::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:01:57,280::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:01:57,281::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:01:57,282::task::495::TaskManager.Task::(_debug) Task cf632ad0-8e00-44ff-a457-e42cf62b271b: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:01:57,283::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:01:57,284::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:01:57,284::task::495::TaskManager.Task::(_debug) Task 226ab5c2-8dd0-4182-b0e7-079dc2f6cd83: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:01:57,285::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`4e2be2f3-78f1-44aa-bf61-108b0877bb77`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:01:57,286::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:01:57,287::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:01:57,287::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`4e2be2f3-78f1-44aa-bf61-108b0877bb77`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:01:57,288::task::495::TaskManager.Task::(_debug) Task 226ab5c2-8dd0-4182-b0e7-079dc2f6cd83: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:01:57,289::task::495::TaskManager.Task::(_debug) Task 226ab5c2-8dd0-4182-b0e7-079dc2f6cd83: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:01:57,290::task::495::TaskManager.Task::(_debug) Task 226ab5c2-8dd0-4182-b0e7-079dc2f6cd83: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:01:57,291::task::495::TaskManager.Task::(_debug) Task 226ab5c2-8dd0-4182-b0e7-079dc2f6cd83: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:01:57,292::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:01:57,293::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:01:57,293::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:01:57,294::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:01:57,295::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:01:57,295::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:01:57,296::task::495::TaskManager.Task::(_debug) Task 226ab5c2-8dd0-4182-b0e7-079dc2f6cd83: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:01:57,297::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:01:57,298::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:01:57,298::task::495::TaskManager.Task::(_debug) Task 2f8fa3df-71c6-47b0-b239-0f4bcc4611cb: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:01:57,299::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`bd5f3f55-fe72-4904-b850-ad0b9f207e39`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:01:57,300::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:01:57,301::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:01:57,301::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`bd5f3f55-fe72-4904-b850-ad0b9f207e39`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:01:57,302::task::495::TaskManager.Task::(_debug) Task 2f8fa3df-71c6-47b0-b239-0f4bcc4611cb: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:01:57,303::task::495::TaskManager.Task::(_debug) Task 2f8fa3df-71c6-47b0-b239-0f4bcc4611cb: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:01:57,303::task::495::TaskManager.Task::(_debug) Task 2f8fa3df-71c6-47b0-b239-0f4bcc4611cb: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:01:57,304::task::495::TaskManager.Task::(_debug) Task 2f8fa3df-71c6-47b0-b239-0f4bcc4611cb: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:01:57,304::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:01:57,305::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:01:57,306::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:01:57,306::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:01:57,307::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:01:57,307::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:01:57,308::task::495::TaskManager.Task::(_debug) Task 2f8fa3df-71c6-47b0-b239-0f4bcc4611cb: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:01:57,309::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:01:57,310::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:01:57,310::task::495::TaskManager.Task::(_debug) Task 95448f88-842c-4ace-92bb-51b88b8338c2: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:01:57,312::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fd3efa36-e43c-4bbe-8151-7d449a729677`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:01:57,312::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:01:57,313::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:01:57,314::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fd3efa36-e43c-4bbe-8151-7d449a729677`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:01:57,315::task::495::TaskManager.Task::(_debug) Task 95448f88-842c-4ace-92bb-51b88b8338c2: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:01:57,315::task::495::TaskManager.Task::(_debug) Task 95448f88-842c-4ace-92bb-51b88b8338c2: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:01:57,316::task::495::TaskManager.Task::(_debug) Task 95448f88-842c-4ace-92bb-51b88b8338c2: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:01:57,316::task::495::TaskManager.Task::(_debug) Task 95448f88-842c-4ace-92bb-51b88b8338c2: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:01:57,317::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:01:57,317::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:01:57,318::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:01:57,319::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:01:57,319::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:01:57,320::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:01:57,320::task::495::TaskManager.Task::(_debug) Task 95448f88-842c-4ace-92bb-51b88b8338c2: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:01:57,321::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15915526::INFO::2013-11-20 04:01:58,576::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915526::DEBUG::2013-11-20 04:01:58,577::task::495::TaskManager.Task::(_debug) Task dc404f9e-ec44-4d8b-b165-952f7035857c: moving from state init -> state preparing
>Thread-15915526::DEBUG::2013-11-20 04:01:58,578::task::495::TaskManager.Task::(_debug) Task dc404f9e-ec44-4d8b-b165-952f7035857c: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000301122665405', 'lastCheck': 1384920110.605983, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00200700759888', 'lastCheck': 1384920111.6100531, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00177192687988', 'lastCheck': 1384920108.6138501, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000452041625977', 'lastCheck': 1384920115.6225221, 'valid': True, 'code': 0}}
>Thread-15915526::DEBUG::2013-11-20 04:01:58,579::task::495::TaskManager.Task::(_debug) Task dc404f9e-ec44-4d8b-b165-952f7035857c: moving from state preparing -> state finished
>Thread-15915526::DEBUG::2013-11-20 04:01:58,580::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915526::DEBUG::2013-11-20 04:01:58,580::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915526::DEBUG::2013-11-20 04:01:58,581::task::495::TaskManager.Task::(_debug) Task dc404f9e-ec44-4d8b-b165-952f7035857c: ref 0 aborting False
>Thread-15915526::INFO::2013-11-20 04:01:58,582::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000301122665405', 'lastCheck': 1384920110.605983, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00200700759888', 'lastCheck': 1384920111.6100531, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00177192687988', 'lastCheck': 1384920108.6138501, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000452041625977', 'lastCheck': 1384920115.6225221, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:01:58,851::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:01:59,414::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0897355 s, 11.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:02:01,431::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:02:01,975::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0854809 s, 12.0 MB/s\n'; <rc> = 0
>Thread-15915528::DEBUG::2013-11-20 04:02:02,384::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915528::INFO::2013-11-20 04:02:02,388::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915528::DEBUG::2013-11-20 04:02:02,389::task::495::TaskManager.Task::(_debug) Task 80a93014-414c-47f2-a03f-e05260728f7a: moving from state init -> state preparing
>Thread-15915528::DEBUG::2013-11-20 04:02:02,397::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915528::DEBUG::2013-11-20 04:02:02,398::task::495::TaskManager.Task::(_debug) Task 80a93014-414c-47f2-a03f-e05260728f7a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915528::DEBUG::2013-11-20 04:02:02,398::task::495::TaskManager.Task::(_debug) Task 80a93014-414c-47f2-a03f-e05260728f7a: moving from state preparing -> state finished
>Thread-15915528::DEBUG::2013-11-20 04:02:02,399::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915528::DEBUG::2013-11-20 04:02:02,399::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915528::DEBUG::2013-11-20 04:02:02,400::task::495::TaskManager.Task::(_debug) Task 80a93014-414c-47f2-a03f-e05260728f7a: ref 0 aborting False
>Thread-15915528::INFO::2013-11-20 04:02:02,400::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915530::DEBUG::2013-11-20 04:02:02,406::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915530::INFO::2013-11-20 04:02:02,407::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915530::DEBUG::2013-11-20 04:02:02,408::task::495::TaskManager.Task::(_debug) Task e952ebad-e00b-4d69-9108-f67b0874a2db: moving from state init -> state preparing
>Thread-15915530::DEBUG::2013-11-20 04:02:02,410::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ed09b8af-372f-4b0c-a698-eff906abee6b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915530::DEBUG::2013-11-20 04:02:02,411::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915530::DEBUG::2013-11-20 04:02:02,412::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915530::DEBUG::2013-11-20 04:02:02,413::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ed09b8af-372f-4b0c-a698-eff906abee6b`::Granted request
>Thread-15915530::DEBUG::2013-11-20 04:02:02,415::task::495::TaskManager.Task::(_debug) Task e952ebad-e00b-4d69-9108-f67b0874a2db: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915530::DEBUG::2013-11-20 04:02:02,416::task::495::TaskManager.Task::(_debug) Task e952ebad-e00b-4d69-9108-f67b0874a2db: ref 1 aborting False
>Thread-15915530::WARNING::2013-11-20 04:02:02,418::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915530::DEBUG::2013-11-20 04:02:02,419::task::495::TaskManager.Task::(_debug) Task e952ebad-e00b-4d69-9108-f67b0874a2db: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915530::DEBUG::2013-11-20 04:02:02,420::task::495::TaskManager.Task::(_debug) Task e952ebad-e00b-4d69-9108-f67b0874a2db: moving from state preparing -> state finished
>Thread-15915530::DEBUG::2013-11-20 04:02:02,421::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915530::DEBUG::2013-11-20 04:02:02,422::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915530::DEBUG::2013-11-20 04:02:02,423::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915530::DEBUG::2013-11-20 04:02:02,425::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915530::DEBUG::2013-11-20 04:02:02,426::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915530::DEBUG::2013-11-20 04:02:02,427::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915530::DEBUG::2013-11-20 04:02:02,428::task::495::TaskManager.Task::(_debug) Task e952ebad-e00b-4d69-9108-f67b0874a2db: ref 0 aborting False
>Thread-15915530::INFO::2013-11-20 04:02:02,429::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:02:03,991::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:02:04,513::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0788863 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:02:06,534::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:02:07,107::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0778029 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15915532::DEBUG::2013-11-20 04:02:07,306::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=ssl=no)
>Thread-15915532::DEBUG::2013-11-20 04:02:08,821::clientIF::1219::vds::(fenceNode) rc 1 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>ssl=no out Getting status of IPMI:192.168.100.119...Chassis power = Unknown
>Failed
> err 
>Thread-15915534::DEBUG::2013-11-20 04:02:08,828::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=ssl=no)
>Dummy-15912651::DEBUG::2013-11-20 04:02:09,121::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:02:09,733::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0753547 s, 13.6 MB/s\n'; <rc> = 0
>Thread-15915534::DEBUG::2013-11-20 04:02:10,450::clientIF::1219::vds::(fenceNode) rc 1 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>ssl=no out Getting status of IPMI:192.168.100.119...Chassis power = Unknown
>Failed
> err 
>Dummy-15912651::DEBUG::2013-11-20 04:02:11,759::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:02:12,350::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0799903 s, 12.8 MB/s\n'; <rc> = 0
>Thread-15913639::INFO::2013-11-20 04:02:12,544::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:02:12,546::task::495::TaskManager.Task::(_debug) Task a4beb123-9438-4f2a-ba5f-f08ba037fa4a: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:02:12,549::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e8fcd509-15a8-4ecd-aa6c-868c6bd5b94e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:02:12,551::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:02:12,553::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:02:12,555::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e8fcd509-15a8-4ecd-aa6c-868c6bd5b94e`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:02:12,556::task::495::TaskManager.Task::(_debug) Task a4beb123-9438-4f2a-ba5f-f08ba037fa4a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:02:12,557::task::495::TaskManager.Task::(_debug) Task a4beb123-9438-4f2a-ba5f-f08ba037fa4a: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:02:12,558::task::495::TaskManager.Task::(_debug) Task a4beb123-9438-4f2a-ba5f-f08ba037fa4a: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:02:12,558::task::495::TaskManager.Task::(_debug) Task a4beb123-9438-4f2a-ba5f-f08ba037fa4a: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:02:12,559::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:02:12,560::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:02:12,560::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:02:12,561::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:02:12,561::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:02:12,562::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:02:12,563::task::495::TaskManager.Task::(_debug) Task a4beb123-9438-4f2a-ba5f-f08ba037fa4a: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:02:12,564::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15915536::DEBUG::2013-11-20 04:02:13,060::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915536::INFO::2013-11-20 04:02:13,062::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915536::DEBUG::2013-11-20 04:02:13,062::task::495::TaskManager.Task::(_debug) Task 211d67cd-c48b-41d4-97ff-ca430c8fba7c: moving from state init -> state preparing
>Thread-15915536::DEBUG::2013-11-20 04:02:13,063::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915536::DEBUG::2013-11-20 04:02:13,064::task::495::TaskManager.Task::(_debug) Task 211d67cd-c48b-41d4-97ff-ca430c8fba7c: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915536::DEBUG::2013-11-20 04:02:13,065::task::495::TaskManager.Task::(_debug) Task 211d67cd-c48b-41d4-97ff-ca430c8fba7c: moving from state preparing -> state finished
>Thread-15915536::DEBUG::2013-11-20 04:02:13,065::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915536::DEBUG::2013-11-20 04:02:13,066::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915536::DEBUG::2013-11-20 04:02:13,067::task::495::TaskManager.Task::(_debug) Task 211d67cd-c48b-41d4-97ff-ca430c8fba7c: ref 0 aborting False
>Thread-15915536::INFO::2013-11-20 04:02:13,067::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915537::DEBUG::2013-11-20 04:02:13,072::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915537::INFO::2013-11-20 04:02:13,073::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915537::DEBUG::2013-11-20 04:02:13,074::task::495::TaskManager.Task::(_debug) Task 8b989002-9b43-493b-a4e6-b8b3b54b83b6: moving from state init -> state preparing
>Thread-15915537::DEBUG::2013-11-20 04:02:13,076::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c83d4cbc-c182-4e9d-abe9-78cc7b6ff90d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915537::DEBUG::2013-11-20 04:02:13,077::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915537::DEBUG::2013-11-20 04:02:13,078::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915537::DEBUG::2013-11-20 04:02:13,079::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c83d4cbc-c182-4e9d-abe9-78cc7b6ff90d`::Granted request
>Thread-15915537::DEBUG::2013-11-20 04:02:13,094::task::495::TaskManager.Task::(_debug) Task 8b989002-9b43-493b-a4e6-b8b3b54b83b6: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915537::DEBUG::2013-11-20 04:02:13,095::task::495::TaskManager.Task::(_debug) Task 8b989002-9b43-493b-a4e6-b8b3b54b83b6: ref 1 aborting False
>Thread-15915537::WARNING::2013-11-20 04:02:13,097::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915537::DEBUG::2013-11-20 04:02:13,099::task::495::TaskManager.Task::(_debug) Task 8b989002-9b43-493b-a4e6-b8b3b54b83b6: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915537::DEBUG::2013-11-20 04:02:13,100::task::495::TaskManager.Task::(_debug) Task 8b989002-9b43-493b-a4e6-b8b3b54b83b6: moving from state preparing -> state finished
>Thread-15915537::DEBUG::2013-11-20 04:02:13,101::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915537::DEBUG::2013-11-20 04:02:13,102::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915537::DEBUG::2013-11-20 04:02:13,104::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915537::DEBUG::2013-11-20 04:02:13,104::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915537::DEBUG::2013-11-20 04:02:13,105::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915537::DEBUG::2013-11-20 04:02:13,106::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915537::DEBUG::2013-11-20 04:02:13,106::task::495::TaskManager.Task::(_debug) Task 8b989002-9b43-493b-a4e6-b8b3b54b83b6: ref 0 aborting False
>Thread-15915537::INFO::2013-11-20 04:02:13,107::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913518::INFO::2013-11-20 04:02:13,114::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:02:13,115::task::495::TaskManager.Task::(_debug) Task d343cc52-7ebd-4317-ae15-04e63784c523: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:02:13,117::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`315f33eb-c6a4-4214-8a95-75d6662b00a2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:02:13,117::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:02:13,118::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:02:13,119::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`315f33eb-c6a4-4214-8a95-75d6662b00a2`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:02:13,120::task::495::TaskManager.Task::(_debug) Task d343cc52-7ebd-4317-ae15-04e63784c523: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:02:13,121::task::495::TaskManager.Task::(_debug) Task d343cc52-7ebd-4317-ae15-04e63784c523: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:02:13,121::task::495::TaskManager.Task::(_debug) Task d343cc52-7ebd-4317-ae15-04e63784c523: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:02:13,122::task::495::TaskManager.Task::(_debug) Task d343cc52-7ebd-4317-ae15-04e63784c523: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:02:13,123::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:02:13,123::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:02:13,124::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:02:13,125::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:02:13,126::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:02:13,127::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:02:13,128::task::495::TaskManager.Task::(_debug) Task d343cc52-7ebd-4317-ae15-04e63784c523: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:02:13,128::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:02:13,129::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:02:13,130::task::495::TaskManager.Task::(_debug) Task e4e6b477-12db-442e-84ca-b9d035f229f2: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:02:13,131::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4950df02-73aa-4492-8dd1-7abf9d0a8e91`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:02:13,132::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:02:13,132::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:02:13,133::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4950df02-73aa-4492-8dd1-7abf9d0a8e91`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:02:13,133::task::495::TaskManager.Task::(_debug) Task e4e6b477-12db-442e-84ca-b9d035f229f2: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:02:13,134::task::495::TaskManager.Task::(_debug) Task e4e6b477-12db-442e-84ca-b9d035f229f2: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:02:13,134::task::495::TaskManager.Task::(_debug) Task e4e6b477-12db-442e-84ca-b9d035f229f2: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:02:13,135::task::495::TaskManager.Task::(_debug) Task e4e6b477-12db-442e-84ca-b9d035f229f2: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:02:13,135::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:02:13,136::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:02:13,137::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:02:13,137::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:02:13,138::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:02:13,138::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:02:13,139::task::495::TaskManager.Task::(_debug) Task e4e6b477-12db-442e-84ca-b9d035f229f2: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:02:13,140::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:02:14,367::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915538::INFO::2013-11-20 04:02:14,592::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915538::DEBUG::2013-11-20 04:02:14,595::task::495::TaskManager.Task::(_debug) Task 2e29531b-c83e-4f94-a09a-d2687294de9d: moving from state init -> state preparing
>Thread-15915538::DEBUG::2013-11-20 04:02:14,597::task::495::TaskManager.Task::(_debug) Task 2e29531b-c83e-4f94-a09a-d2687294de9d: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000351190567017', 'lastCheck': 1384920130.7533901, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.000971078872681', 'lastCheck': 1384920131.7519751, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00125789642334', 'lastCheck': 1384920128.745676, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000389814376831', 'lastCheck': 1384920125.6545889, 'valid': True, 'code': 0}}
>Thread-15915538::DEBUG::2013-11-20 04:02:14,598::task::495::TaskManager.Task::(_debug) Task 2e29531b-c83e-4f94-a09a-d2687294de9d: moving from state preparing -> state finished
>Thread-15915538::DEBUG::2013-11-20 04:02:14,599::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915538::DEBUG::2013-11-20 04:02:14,600::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915538::DEBUG::2013-11-20 04:02:14,601::task::495::TaskManager.Task::(_debug) Task 2e29531b-c83e-4f94-a09a-d2687294de9d: ref 0 aborting False
>Thread-15915538::INFO::2013-11-20 04:02:14,602::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000351190567017', 'lastCheck': 1384920130.7533901, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.000971078872681', 'lastCheck': 1384920131.7519751, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00125789642334', 'lastCheck': 1384920128.745676, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000389814376831', 'lastCheck': 1384920125.6545889, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:02:14,936::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.107344 s, 9.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:02:16,951::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:02:17,488::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.085571 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:02:19,505::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:02:20,119::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0826319 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:02:22,140::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:02:22,707::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0814415 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15915542::DEBUG::2013-11-20 04:02:23,548::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915542::INFO::2013-11-20 04:02:23,550::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915542::DEBUG::2013-11-20 04:02:23,551::task::495::TaskManager.Task::(_debug) Task b1e862c1-5c51-4524-a6e9-b7fc968d3266: moving from state init -> state preparing
>Thread-15915542::DEBUG::2013-11-20 04:02:23,552::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915542::DEBUG::2013-11-20 04:02:23,553::task::495::TaskManager.Task::(_debug) Task b1e862c1-5c51-4524-a6e9-b7fc968d3266: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915542::DEBUG::2013-11-20 04:02:23,554::task::495::TaskManager.Task::(_debug) Task b1e862c1-5c51-4524-a6e9-b7fc968d3266: moving from state preparing -> state finished
>Thread-15915542::DEBUG::2013-11-20 04:02:23,555::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915542::DEBUG::2013-11-20 04:02:23,556::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915542::DEBUG::2013-11-20 04:02:23,556::task::495::TaskManager.Task::(_debug) Task b1e862c1-5c51-4524-a6e9-b7fc968d3266: ref 0 aborting False
>Thread-15915542::INFO::2013-11-20 04:02:23,557::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915543::DEBUG::2013-11-20 04:02:23,562::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915543::INFO::2013-11-20 04:02:23,563::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915543::DEBUG::2013-11-20 04:02:23,563::task::495::TaskManager.Task::(_debug) Task 3c32e582-4969-41df-bf6f-04c3670da0b4: moving from state init -> state preparing
>Thread-15915543::DEBUG::2013-11-20 04:02:23,565::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5ff6ef82-ce7f-4888-8eec-1482ab0cd943`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915543::DEBUG::2013-11-20 04:02:23,565::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915543::DEBUG::2013-11-20 04:02:23,566::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915543::DEBUG::2013-11-20 04:02:23,567::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5ff6ef82-ce7f-4888-8eec-1482ab0cd943`::Granted request
>Thread-15915543::DEBUG::2013-11-20 04:02:23,568::task::495::TaskManager.Task::(_debug) Task 3c32e582-4969-41df-bf6f-04c3670da0b4: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915543::DEBUG::2013-11-20 04:02:23,569::task::495::TaskManager.Task::(_debug) Task 3c32e582-4969-41df-bf6f-04c3670da0b4: ref 1 aborting False
>Thread-15915543::WARNING::2013-11-20 04:02:23,572::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915543::DEBUG::2013-11-20 04:02:23,573::task::495::TaskManager.Task::(_debug) Task 3c32e582-4969-41df-bf6f-04c3670da0b4: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915543::DEBUG::2013-11-20 04:02:23,574::task::495::TaskManager.Task::(_debug) Task 3c32e582-4969-41df-bf6f-04c3670da0b4: moving from state preparing -> state finished
>Thread-15915543::DEBUG::2013-11-20 04:02:23,575::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915543::DEBUG::2013-11-20 04:02:23,575::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915543::DEBUG::2013-11-20 04:02:23,576::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915543::DEBUG::2013-11-20 04:02:23,577::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915543::DEBUG::2013-11-20 04:02:23,577::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915543::DEBUG::2013-11-20 04:02:23,578::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915543::DEBUG::2013-11-20 04:02:23,579::task::495::TaskManager.Task::(_debug) Task 3c32e582-4969-41df-bf6f-04c3670da0b4: ref 0 aborting False
>Thread-15915543::INFO::2013-11-20 04:02:23,579::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:02:24,721::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:02:25,318::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.088902 s, 11.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:02:27,333::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 04:02:27,530::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:02:27,536::task::495::TaskManager.Task::(_debug) Task b230e950-2744-4fa8-8f8b-38a358663425: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:02:27,540::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`108a5870-b0f1-416c-a79d-822c390878ac`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:02:27,540::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:02:27,541::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:02:27,543::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`108a5870-b0f1-416c-a79d-822c390878ac`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:02:27,544::task::495::TaskManager.Task::(_debug) Task b230e950-2744-4fa8-8f8b-38a358663425: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:02:27,545::task::495::TaskManager.Task::(_debug) Task b230e950-2744-4fa8-8f8b-38a358663425: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:02:27,546::task::495::TaskManager.Task::(_debug) Task b230e950-2744-4fa8-8f8b-38a358663425: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:02:27,547::task::495::TaskManager.Task::(_debug) Task b230e950-2744-4fa8-8f8b-38a358663425: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:02:27,547::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:02:27,548::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:02:27,549::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:02:27,550::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:02:27,550::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:02:27,551::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:02:27,552::task::495::TaskManager.Task::(_debug) Task b230e950-2744-4fa8-8f8b-38a358663425: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:02:27,553::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:02:27,887::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0948812 s, 10.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:02:29,903::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:02:30,436::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0787058 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15915546::INFO::2013-11-20 04:02:30,556::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915546::DEBUG::2013-11-20 04:02:30,557::task::495::TaskManager.Task::(_debug) Task 7eb76c25-d80b-41b4-97ec-51b4e0efb2b5: moving from state init -> state preparing
>Thread-15915546::DEBUG::2013-11-20 04:02:30,558::task::495::TaskManager.Task::(_debug) Task 7eb76c25-d80b-41b4-97ec-51b4e0efb2b5: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000348091125488', 'lastCheck': 1384920140.7771671, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0135390758514', 'lastCheck': 1384920141.784385, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00136804580688', 'lastCheck': 1384920148.9704621, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000403881072998', 'lastCheck': 1384920145.9737761, 'valid': True, 'code': 0}}
>Thread-15915546::DEBUG::2013-11-20 04:02:30,559::task::495::TaskManager.Task::(_debug) Task 7eb76c25-d80b-41b4-97ec-51b4e0efb2b5: moving from state preparing -> state finished
>Thread-15915546::DEBUG::2013-11-20 04:02:30,560::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915546::DEBUG::2013-11-20 04:02:30,560::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915546::DEBUG::2013-11-20 04:02:30,561::task::495::TaskManager.Task::(_debug) Task 7eb76c25-d80b-41b4-97ec-51b4e0efb2b5: ref 0 aborting False
>Thread-15915546::INFO::2013-11-20 04:02:30,561::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000348091125488', 'lastCheck': 1384920140.7771671, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0135390758514', 'lastCheck': 1384920141.784385, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00136804580688', 'lastCheck': 1384920148.9704621, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000403881072998', 'lastCheck': 1384920145.9737761, 'valid': True, 'code': 0}}
>Thread-42838::DEBUG::2013-11-20 04:02:31,119::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 04:02:32,002::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:02:32,453::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:02:33,064::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0749543 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15915548::DEBUG::2013-11-20 04:02:33,928::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915548::INFO::2013-11-20 04:02:33,929::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915548::DEBUG::2013-11-20 04:02:33,930::task::495::TaskManager.Task::(_debug) Task 79efa72e-9386-4caa-8b13-ed8919ac241d: moving from state init -> state preparing
>Thread-15915548::DEBUG::2013-11-20 04:02:33,931::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915548::DEBUG::2013-11-20 04:02:33,931::task::495::TaskManager.Task::(_debug) Task 79efa72e-9386-4caa-8b13-ed8919ac241d: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915548::DEBUG::2013-11-20 04:02:33,932::task::495::TaskManager.Task::(_debug) Task 79efa72e-9386-4caa-8b13-ed8919ac241d: moving from state preparing -> state finished
>Thread-15915548::DEBUG::2013-11-20 04:02:33,932::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915548::DEBUG::2013-11-20 04:02:33,933::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915548::DEBUG::2013-11-20 04:02:33,933::task::495::TaskManager.Task::(_debug) Task 79efa72e-9386-4caa-8b13-ed8919ac241d: ref 0 aborting False
>Thread-15915548::INFO::2013-11-20 04:02:33,934::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915549::DEBUG::2013-11-20 04:02:33,939::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915549::INFO::2013-11-20 04:02:33,940::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915549::DEBUG::2013-11-20 04:02:33,941::task::495::TaskManager.Task::(_debug) Task abb4aa91-56b2-499c-8813-e6f0391db89f: moving from state init -> state preparing
>Thread-15915549::DEBUG::2013-11-20 04:02:33,943::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`399e5b4c-90da-4bb9-8aee-b2ed893dd4d0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915549::DEBUG::2013-11-20 04:02:33,943::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915549::DEBUG::2013-11-20 04:02:33,944::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915549::DEBUG::2013-11-20 04:02:33,945::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`399e5b4c-90da-4bb9-8aee-b2ed893dd4d0`::Granted request
>Thread-15915549::DEBUG::2013-11-20 04:02:33,946::task::495::TaskManager.Task::(_debug) Task abb4aa91-56b2-499c-8813-e6f0391db89f: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915549::DEBUG::2013-11-20 04:02:33,947::task::495::TaskManager.Task::(_debug) Task abb4aa91-56b2-499c-8813-e6f0391db89f: ref 1 aborting False
>Thread-15915549::WARNING::2013-11-20 04:02:33,949::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915549::DEBUG::2013-11-20 04:02:33,951::task::495::TaskManager.Task::(_debug) Task abb4aa91-56b2-499c-8813-e6f0391db89f: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915549::DEBUG::2013-11-20 04:02:33,952::task::495::TaskManager.Task::(_debug) Task abb4aa91-56b2-499c-8813-e6f0391db89f: moving from state preparing -> state finished
>Thread-15915549::DEBUG::2013-11-20 04:02:33,952::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915549::DEBUG::2013-11-20 04:02:33,953::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915549::DEBUG::2013-11-20 04:02:33,954::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915549::DEBUG::2013-11-20 04:02:33,955::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915549::DEBUG::2013-11-20 04:02:33,956::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915549::DEBUG::2013-11-20 04:02:33,956::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915549::DEBUG::2013-11-20 04:02:33,957::task::495::TaskManager.Task::(_debug) Task abb4aa91-56b2-499c-8813-e6f0391db89f: ref 0 aborting False
>Thread-15915549::INFO::2013-11-20 04:02:33,958::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:02:35,083::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:02:35,519::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0521569 s, 19.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:02:37,531::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 04:02:37,814::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:02:37,819::task::495::TaskManager.Task::(_debug) Task ec67e0a5-aecf-499b-979c-c3438fd0acb4: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:02:37,824::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`04444186-27b3-4268-9e9e-cbe61c2bfc82`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:02:37,825::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:02:37,826::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:02:37,827::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`04444186-27b3-4268-9e9e-cbe61c2bfc82`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:02:37,829::task::495::TaskManager.Task::(_debug) Task ec67e0a5-aecf-499b-979c-c3438fd0acb4: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:02:37,830::task::495::TaskManager.Task::(_debug) Task ec67e0a5-aecf-499b-979c-c3438fd0acb4: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:02:37,831::task::495::TaskManager.Task::(_debug) Task ec67e0a5-aecf-499b-979c-c3438fd0acb4: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:02:37,832::task::495::TaskManager.Task::(_debug) Task ec67e0a5-aecf-499b-979c-c3438fd0acb4: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:02:37,832::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:02:37,833::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:02:37,833::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:02:37,834::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:02:37,835::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:02:37,835::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:02:37,836::task::495::TaskManager.Task::(_debug) Task ec67e0a5-aecf-499b-979c-c3438fd0acb4: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:02:37,837::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:02:38,116::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0936567 s, 10.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:02:40,135::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:02:40,704::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0841748 s, 12.2 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 04:02:41,093::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:02:41,094::task::495::TaskManager.Task::(_debug) Task 2f59f543-5e82-45b5-aa2b-34ef85523717: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:02:41,097::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`29e5135e-4a41-4941-9234-4a2d059cd255`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:02:41,097::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:02:41,098::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:02:41,099::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`29e5135e-4a41-4941-9234-4a2d059cd255`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:02:41,100::task::495::TaskManager.Task::(_debug) Task 2f59f543-5e82-45b5-aa2b-34ef85523717: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:02:41,101::task::495::TaskManager.Task::(_debug) Task 2f59f543-5e82-45b5-aa2b-34ef85523717: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:02:41,102::task::495::TaskManager.Task::(_debug) Task 2f59f543-5e82-45b5-aa2b-34ef85523717: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:02:41,102::task::495::TaskManager.Task::(_debug) Task 2f59f543-5e82-45b5-aa2b-34ef85523717: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:02:41,103::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:02:41,103::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:02:41,104::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:02:41,105::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:02:41,105::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:02:41,106::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:02:41,107::task::495::TaskManager.Task::(_debug) Task 2f59f543-5e82-45b5-aa2b-34ef85523717: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:02:41,107::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:02:42,714::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915085::INFO::2013-11-20 04:02:42,940::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=089b08fc-6544-4235-a061-fb55b09d2715 volUUID=66b599fc-b7b5-4934-a5d5-ef7283460029)
>Thread-15915085::DEBUG::2013-11-20 04:02:42,943::task::495::TaskManager.Task::(_debug) Task 1c8b666f-c4ae-4e95-9f04-c69bd66a249e: moving from state init -> state preparing
>Thread-15915085::DEBUG::2013-11-20 04:02:42,948::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`051efbcf-c899-4699-897d-9873f18f13ee`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915085::DEBUG::2013-11-20 04:02:42,951::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915085::DEBUG::2013-11-20 04:02:42,952::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915085::DEBUG::2013-11-20 04:02:42,953::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`051efbcf-c899-4699-897d-9873f18f13ee`::Granted request
>Thread-15915085::DEBUG::2013-11-20 04:02:42,956::task::495::TaskManager.Task::(_debug) Task 1c8b666f-c4ae-4e95-9f04-c69bd66a249e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915085::DEBUG::2013-11-20 04:02:42,956::task::495::TaskManager.Task::(_debug) Task 1c8b666f-c4ae-4e95-9f04-c69bd66a249e: ref 1 aborting False
>Thread-15915085::DEBUG::2013-11-20 04:02:42,957::task::495::TaskManager.Task::(_debug) Task 1c8b666f-c4ae-4e95-9f04-c69bd66a249e: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15915085::DEBUG::2013-11-20 04:02:42,958::task::495::TaskManager.Task::(_debug) Task 1c8b666f-c4ae-4e95-9f04-c69bd66a249e: moving from state preparing -> state finished
>Thread-15915085::DEBUG::2013-11-20 04:02:42,958::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915085::DEBUG::2013-11-20 04:02:42,959::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915085::DEBUG::2013-11-20 04:02:42,959::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915085::DEBUG::2013-11-20 04:02:42,960::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915085::DEBUG::2013-11-20 04:02:42,960::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915085::DEBUG::2013-11-20 04:02:42,960::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915085::DEBUG::2013-11-20 04:02:42,961::task::495::TaskManager.Task::(_debug) Task 1c8b666f-c4ae-4e95-9f04-c69bd66a249e: ref 0 aborting False
>Thread-15915085::INFO::2013-11-20 04:02:42,961::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Dummy-15912651::DEBUG::2013-11-20 04:02:43,228::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0757595 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15915554::DEBUG::2013-11-20 04:02:44,486::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915554::INFO::2013-11-20 04:02:44,488::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915554::DEBUG::2013-11-20 04:02:44,488::task::495::TaskManager.Task::(_debug) Task 8a0d4be9-8ba7-468c-937b-75ba861a27a4: moving from state init -> state preparing
>Thread-15915554::DEBUG::2013-11-20 04:02:44,489::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915554::DEBUG::2013-11-20 04:02:44,490::task::495::TaskManager.Task::(_debug) Task 8a0d4be9-8ba7-468c-937b-75ba861a27a4: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915554::DEBUG::2013-11-20 04:02:44,491::task::495::TaskManager.Task::(_debug) Task 8a0d4be9-8ba7-468c-937b-75ba861a27a4: moving from state preparing -> state finished
>Thread-15915554::DEBUG::2013-11-20 04:02:44,491::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915554::DEBUG::2013-11-20 04:02:44,492::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915554::DEBUG::2013-11-20 04:02:44,493::task::495::TaskManager.Task::(_debug) Task 8a0d4be9-8ba7-468c-937b-75ba861a27a4: ref 0 aborting False
>Thread-15915554::INFO::2013-11-20 04:02:44,493::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915555::DEBUG::2013-11-20 04:02:44,498::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915555::INFO::2013-11-20 04:02:44,499::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915555::DEBUG::2013-11-20 04:02:44,500::task::495::TaskManager.Task::(_debug) Task 6781dd3f-5c7b-4c89-945a-45a4221601ad: moving from state init -> state preparing
>Thread-15915555::DEBUG::2013-11-20 04:02:44,501::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a052c3f3-f4aa-4c10-a74f-a93591d83a1d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915555::DEBUG::2013-11-20 04:02:44,502::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915555::DEBUG::2013-11-20 04:02:44,503::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915555::DEBUG::2013-11-20 04:02:44,504::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a052c3f3-f4aa-4c10-a74f-a93591d83a1d`::Granted request
>Thread-15915555::DEBUG::2013-11-20 04:02:44,505::task::495::TaskManager.Task::(_debug) Task 6781dd3f-5c7b-4c89-945a-45a4221601ad: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915555::DEBUG::2013-11-20 04:02:44,506::task::495::TaskManager.Task::(_debug) Task 6781dd3f-5c7b-4c89-945a-45a4221601ad: ref 1 aborting False
>Thread-15915555::WARNING::2013-11-20 04:02:44,508::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915555::DEBUG::2013-11-20 04:02:44,509::task::495::TaskManager.Task::(_debug) Task 6781dd3f-5c7b-4c89-945a-45a4221601ad: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915555::DEBUG::2013-11-20 04:02:44,510::task::495::TaskManager.Task::(_debug) Task 6781dd3f-5c7b-4c89-945a-45a4221601ad: moving from state preparing -> state finished
>Thread-15915555::DEBUG::2013-11-20 04:02:44,511::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915555::DEBUG::2013-11-20 04:02:44,512::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915555::DEBUG::2013-11-20 04:02:44,513::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915555::DEBUG::2013-11-20 04:02:44,514::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915555::DEBUG::2013-11-20 04:02:44,514::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915555::DEBUG::2013-11-20 04:02:44,515::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915555::DEBUG::2013-11-20 04:02:44,516::task::495::TaskManager.Task::(_debug) Task 6781dd3f-5c7b-4c89-945a-45a4221601ad: ref 0 aborting False
>Thread-15915555::INFO::2013-11-20 04:02:44,516::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:02:45,243::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:02:45,754::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0709918 s, 14.4 MB/s\n'; <rc> = 0
>Thread-15915556::INFO::2013-11-20 04:02:46,423::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915556::DEBUG::2013-11-20 04:02:46,425::task::495::TaskManager.Task::(_debug) Task 9797e527-fe88-4908-8de7-641d5ada93aa: moving from state init -> state preparing
>Thread-15915556::DEBUG::2013-11-20 04:02:46,427::task::495::TaskManager.Task::(_debug) Task 9797e527-fe88-4908-8de7-641d5ada93aa: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000324010848999', 'lastCheck': 1384920162.0206029, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00818204879761', 'lastCheck': 1384920162.372035, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.001384973526', 'lastCheck': 1384920159.277797, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000399112701416', 'lastCheck': 1384920156.3285971, 'valid': True, 'code': 0}}
>Thread-15915556::DEBUG::2013-11-20 04:02:46,429::task::495::TaskManager.Task::(_debug) Task 9797e527-fe88-4908-8de7-641d5ada93aa: moving from state preparing -> state finished
>Thread-15915556::DEBUG::2013-11-20 04:02:46,431::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915556::DEBUG::2013-11-20 04:02:46,436::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915556::DEBUG::2013-11-20 04:02:46,438::task::495::TaskManager.Task::(_debug) Task 9797e527-fe88-4908-8de7-641d5ada93aa: ref 0 aborting False
>Thread-15915556::INFO::2013-11-20 04:02:46,438::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000324010848999', 'lastCheck': 1384920162.0206029, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00818204879761', 'lastCheck': 1384920162.372035, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.001384973526', 'lastCheck': 1384920159.277797, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000399112701416', 'lastCheck': 1384920156.3285971, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:02:47,768::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 04:02:47,988::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:02:47,996::task::495::TaskManager.Task::(_debug) Task b416035e-52ee-476d-999f-5b0b15293b03: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:02:48,000::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d339637e-645a-4fbc-b867-3bb0ece94d23`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:02:48,002::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:02:48,006::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:02:48,007::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d339637e-645a-4fbc-b867-3bb0ece94d23`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:02:48,009::task::495::TaskManager.Task::(_debug) Task b416035e-52ee-476d-999f-5b0b15293b03: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:02:48,010::task::495::TaskManager.Task::(_debug) Task b416035e-52ee-476d-999f-5b0b15293b03: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:02:48,010::task::495::TaskManager.Task::(_debug) Task b416035e-52ee-476d-999f-5b0b15293b03: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:02:48,011::task::495::TaskManager.Task::(_debug) Task b416035e-52ee-476d-999f-5b0b15293b03: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:02:48,013::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:02:48,013::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:02:48,014::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:02:48,015::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:02:48,015::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:02:48,016::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:02:48,017::task::495::TaskManager.Task::(_debug) Task b416035e-52ee-476d-999f-5b0b15293b03: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:02:48,019::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:02:48,327::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0856024 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:02:50,337::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:02:50,930::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0873159 s, 11.7 MB/s\n'; <rc> = 0
>Thread-15914435::INFO::2013-11-20 04:02:51,154::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:02:51,155::task::495::TaskManager.Task::(_debug) Task 1b5b4afc-6d1d-42a2-b2fa-e4df1035957a: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:02:51,157::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a97f437d-a0a5-4bb2-9d05-b163810fb9eb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:02:51,157::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:02:51,158::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:02:51,159::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a97f437d-a0a5-4bb2-9d05-b163810fb9eb`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:02:51,160::task::495::TaskManager.Task::(_debug) Task 1b5b4afc-6d1d-42a2-b2fa-e4df1035957a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:02:51,161::task::495::TaskManager.Task::(_debug) Task 1b5b4afc-6d1d-42a2-b2fa-e4df1035957a: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:02:51,162::task::495::TaskManager.Task::(_debug) Task 1b5b4afc-6d1d-42a2-b2fa-e4df1035957a: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:02:51,162::task::495::TaskManager.Task::(_debug) Task 1b5b4afc-6d1d-42a2-b2fa-e4df1035957a: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:02:51,163::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:02:51,164::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:02:51,164::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:02:51,165::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:02:51,166::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:02:51,166::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:02:51,167::task::495::TaskManager.Task::(_debug) Task 1b5b4afc-6d1d-42a2-b2fa-e4df1035957a: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:02:51,168::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:02:52,947::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:02:53,480::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0750542 s, 13.6 MB/s\n'; <rc> = 0
>Thread-15915560::DEBUG::2013-11-20 04:02:54,469::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Thread-15915561::DEBUG::2013-11-20 04:02:54,971::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915561::INFO::2013-11-20 04:02:54,974::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915561::DEBUG::2013-11-20 04:02:54,976::task::495::TaskManager.Task::(_debug) Task 097b64ee-b8e1-4683-8742-c1db9a0f6e1c: moving from state init -> state preparing
>Thread-15915561::DEBUG::2013-11-20 04:02:54,979::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915561::DEBUG::2013-11-20 04:02:54,980::task::495::TaskManager.Task::(_debug) Task 097b64ee-b8e1-4683-8742-c1db9a0f6e1c: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915561::DEBUG::2013-11-20 04:02:54,981::task::495::TaskManager.Task::(_debug) Task 097b64ee-b8e1-4683-8742-c1db9a0f6e1c: moving from state preparing -> state finished
>Thread-15915561::DEBUG::2013-11-20 04:02:54,982::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915561::DEBUG::2013-11-20 04:02:54,986::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915561::DEBUG::2013-11-20 04:02:54,987::task::495::TaskManager.Task::(_debug) Task 097b64ee-b8e1-4683-8742-c1db9a0f6e1c: ref 0 aborting False
>Thread-15915561::INFO::2013-11-20 04:02:54,988::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915562::DEBUG::2013-11-20 04:02:54,993::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915562::INFO::2013-11-20 04:02:54,995::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915562::DEBUG::2013-11-20 04:02:54,995::task::495::TaskManager.Task::(_debug) Task 999d11bd-9422-4a5b-b1b3-c3d6db8b49ab: moving from state init -> state preparing
>Thread-15915562::DEBUG::2013-11-20 04:02:54,997::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ff2f8add-064f-4610-9f0d-604bd8f43b83`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915562::DEBUG::2013-11-20 04:02:54,997::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915562::DEBUG::2013-11-20 04:02:54,998::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915562::DEBUG::2013-11-20 04:02:54,999::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ff2f8add-064f-4610-9f0d-604bd8f43b83`::Granted request
>Thread-15915562::DEBUG::2013-11-20 04:02:55,011::task::495::TaskManager.Task::(_debug) Task 999d11bd-9422-4a5b-b1b3-c3d6db8b49ab: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915562::DEBUG::2013-11-20 04:02:55,011::task::495::TaskManager.Task::(_debug) Task 999d11bd-9422-4a5b-b1b3-c3d6db8b49ab: ref 1 aborting False
>Thread-15915562::WARNING::2013-11-20 04:02:55,014::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915562::DEBUG::2013-11-20 04:02:55,015::task::495::TaskManager.Task::(_debug) Task 999d11bd-9422-4a5b-b1b3-c3d6db8b49ab: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915562::DEBUG::2013-11-20 04:02:55,016::task::495::TaskManager.Task::(_debug) Task 999d11bd-9422-4a5b-b1b3-c3d6db8b49ab: moving from state preparing -> state finished
>Thread-15915562::DEBUG::2013-11-20 04:02:55,017::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915562::DEBUG::2013-11-20 04:02:55,017::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915562::DEBUG::2013-11-20 04:02:55,018::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915562::DEBUG::2013-11-20 04:02:55,019::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915562::DEBUG::2013-11-20 04:02:55,020::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915562::DEBUG::2013-11-20 04:02:55,020::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915562::DEBUG::2013-11-20 04:02:55,021::task::495::TaskManager.Task::(_debug) Task 999d11bd-9422-4a5b-b1b3-c3d6db8b49ab: ref 0 aborting False
>Thread-15915562::INFO::2013-11-20 04:02:55,022::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915560::DEBUG::2013-11-20 04:02:55,146::clientIF::1219::vds::(fenceNode) rc 0 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = On
>Done
> err 
>Thread-15915563::DEBUG::2013-11-20 04:02:55,151::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Dummy-15912651::DEBUG::2013-11-20 04:02:55,496::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 04:02:55,709::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:02:55,710::task::495::TaskManager.Task::(_debug) Task eb5c865c-d2ab-4b53-9315-26c3cbbe78e4: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:02:55,718::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1632cccf-9571-4258-9fe5-0dc6d3898f18`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:02:55,719::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:02:55,721::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:02:55,721::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1632cccf-9571-4258-9fe5-0dc6d3898f18`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:02:55,727::task::495::TaskManager.Task::(_debug) Task eb5c865c-d2ab-4b53-9315-26c3cbbe78e4: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 04:02:55,728::task::495::TaskManager.Task::(_debug) Task eb5c865c-d2ab-4b53-9315-26c3cbbe78e4: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:02:55,729::task::495::TaskManager.Task::(_debug) Task eb5c865c-d2ab-4b53-9315-26c3cbbe78e4: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 04:02:55,730::task::495::TaskManager.Task::(_debug) Task eb5c865c-d2ab-4b53-9315-26c3cbbe78e4: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:02:55,730::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:02:55,731::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:02:55,732::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 04:02:55,732::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:02:55,733::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:02:55,733::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:02:55,734::task::495::TaskManager.Task::(_debug) Task eb5c865c-d2ab-4b53-9315-26c3cbbe78e4: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 04:02:55,735::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15915563::DEBUG::2013-11-20 04:02:55,914::clientIF::1219::vds::(fenceNode) rc 0 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = On
>Done
> err 
>Dummy-15912651::DEBUG::2013-11-20 04:02:56,060::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.084431 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:02:58,076::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 04:02:58,280::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:02:58,283::task::495::TaskManager.Task::(_debug) Task 6b973e96-8832-4cd0-a684-156c2abc7220: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:02:58,286::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`af53f0e3-0bfa-4120-8dad-b65ef3210528`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:02:58,287::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:02:58,288::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:02:58,289::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`af53f0e3-0bfa-4120-8dad-b65ef3210528`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:02:58,290::task::495::TaskManager.Task::(_debug) Task 6b973e96-8832-4cd0-a684-156c2abc7220: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:02:58,291::task::495::TaskManager.Task::(_debug) Task 6b973e96-8832-4cd0-a684-156c2abc7220: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:02:58,291::task::495::TaskManager.Task::(_debug) Task 6b973e96-8832-4cd0-a684-156c2abc7220: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 04:02:58,292::task::495::TaskManager.Task::(_debug) Task 6b973e96-8832-4cd0-a684-156c2abc7220: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:02:58,292::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:02:58,292::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:02:58,293::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:02:58,293::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:02:58,294::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:02:58,294::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:02:58,295::task::495::TaskManager.Task::(_debug) Task 6b973e96-8832-4cd0-a684-156c2abc7220: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:02:58,295::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:02:58,296::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:02:58,296::task::495::TaskManager.Task::(_debug) Task 6bacb231-4c00-4c93-a09e-b010ec67d161: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:02:58,297::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`29b5c47a-8f24-4ed2-b028-e8a840c2f0f8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:02:58,298::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:02:58,299::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:02:58,299::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`29b5c47a-8f24-4ed2-b028-e8a840c2f0f8`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:02:58,301::task::495::TaskManager.Task::(_debug) Task 6bacb231-4c00-4c93-a09e-b010ec67d161: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:02:58,301::task::495::TaskManager.Task::(_debug) Task 6bacb231-4c00-4c93-a09e-b010ec67d161: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:02:58,302::task::495::TaskManager.Task::(_debug) Task 6bacb231-4c00-4c93-a09e-b010ec67d161: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:02:58,302::task::495::TaskManager.Task::(_debug) Task 6bacb231-4c00-4c93-a09e-b010ec67d161: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:02:58,303::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:02:58,304::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:02:58,304::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:02:58,305::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:02:58,306::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:02:58,306::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:02:58,307::task::495::TaskManager.Task::(_debug) Task 6bacb231-4c00-4c93-a09e-b010ec67d161: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:02:58,307::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:02:58,308::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:02:58,308::task::495::TaskManager.Task::(_debug) Task 93f94807-f13e-493d-96f6-43ee288fae0c: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:02:58,310::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`0c5b1555-3bf4-48fd-9544-8ea582f468d8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:02:58,310::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:02:58,311::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:02:58,311::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`0c5b1555-3bf4-48fd-9544-8ea582f468d8`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:02:58,312::task::495::TaskManager.Task::(_debug) Task 93f94807-f13e-493d-96f6-43ee288fae0c: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:02:58,312::task::495::TaskManager.Task::(_debug) Task 93f94807-f13e-493d-96f6-43ee288fae0c: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:02:58,314::task::495::TaskManager.Task::(_debug) Task 93f94807-f13e-493d-96f6-43ee288fae0c: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:02:58,314::task::495::TaskManager.Task::(_debug) Task 93f94807-f13e-493d-96f6-43ee288fae0c: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:02:58,315::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:02:58,315::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:02:58,316::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:02:58,317::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:02:58,317::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:02:58,318::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:02:58,319::task::495::TaskManager.Task::(_debug) Task 93f94807-f13e-493d-96f6-43ee288fae0c: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:02:58,319::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:02:58,320::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:02:58,320::task::495::TaskManager.Task::(_debug) Task ba41ceb3-c8b8-42ff-b43a-5798078e0d5d: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:02:58,321::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`ec98b5cf-f87f-458c-b6da-3fd0d8d3c3a1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:02:58,322::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:02:58,323::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:02:58,323::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`ec98b5cf-f87f-458c-b6da-3fd0d8d3c3a1`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:02:58,324::task::495::TaskManager.Task::(_debug) Task ba41ceb3-c8b8-42ff-b43a-5798078e0d5d: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:02:58,325::task::495::TaskManager.Task::(_debug) Task ba41ceb3-c8b8-42ff-b43a-5798078e0d5d: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:02:58,326::task::495::TaskManager.Task::(_debug) Task ba41ceb3-c8b8-42ff-b43a-5798078e0d5d: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:02:58,327::task::495::TaskManager.Task::(_debug) Task ba41ceb3-c8b8-42ff-b43a-5798078e0d5d: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:02:58,327::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:02:58,328::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:02:58,328::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:02:58,329::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:02:58,329::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:02:58,330::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:02:58,331::task::495::TaskManager.Task::(_debug) Task ba41ceb3-c8b8-42ff-b43a-5798078e0d5d: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:02:58,331::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:02:58,639::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0794758 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:03:00,654::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:03:01,233::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.079475 s, 12.9 MB/s\n'; <rc> = 0
>Thread-15915566::INFO::2013-11-20 04:03:02,420::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915566::DEBUG::2013-11-20 04:03:02,421::task::495::TaskManager.Task::(_debug) Task f2ee8195-1ae6-4460-ac54-118c482eea59: moving from state init -> state preparing
>Thread-15915566::DEBUG::2013-11-20 04:03:02,422::task::495::TaskManager.Task::(_debug) Task f2ee8195-1ae6-4460-ac54-118c482eea59: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000603914260864', 'lastCheck': 1384920182.2845559, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00187802314758', 'lastCheck': 1384920172.5815439, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0400347709656', 'lastCheck': 1384920179.758744, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000403881072998', 'lastCheck': 1384920176.7114551, 'valid': True, 'code': 0}}
>Thread-15915566::DEBUG::2013-11-20 04:03:02,423::task::495::TaskManager.Task::(_debug) Task f2ee8195-1ae6-4460-ac54-118c482eea59: moving from state preparing -> state finished
>Thread-15915566::DEBUG::2013-11-20 04:03:02,424::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915566::DEBUG::2013-11-20 04:03:02,424::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915566::DEBUG::2013-11-20 04:03:02,425::task::495::TaskManager.Task::(_debug) Task f2ee8195-1ae6-4460-ac54-118c482eea59: ref 0 aborting False
>Thread-15915566::INFO::2013-11-20 04:03:02,426::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000603914260864', 'lastCheck': 1384920182.2845559, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00187802314758', 'lastCheck': 1384920172.5815439, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0400347709656', 'lastCheck': 1384920179.758744, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000403881072998', 'lastCheck': 1384920176.7114551, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:03:03,248::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:03:03,810::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0773308 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15915568::DEBUG::2013-11-20 04:03:05,483::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915568::INFO::2013-11-20 04:03:05,485::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915568::DEBUG::2013-11-20 04:03:05,485::task::495::TaskManager.Task::(_debug) Task 881ebb6e-3e89-42e4-98ea-470e78d73b13: moving from state init -> state preparing
>Thread-15915568::DEBUG::2013-11-20 04:03:05,487::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915568::DEBUG::2013-11-20 04:03:05,487::task::495::TaskManager.Task::(_debug) Task 881ebb6e-3e89-42e4-98ea-470e78d73b13: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915568::DEBUG::2013-11-20 04:03:05,488::task::495::TaskManager.Task::(_debug) Task 881ebb6e-3e89-42e4-98ea-470e78d73b13: moving from state preparing -> state finished
>Thread-15915568::DEBUG::2013-11-20 04:03:05,489::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915568::DEBUG::2013-11-20 04:03:05,490::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915568::DEBUG::2013-11-20 04:03:05,491::task::495::TaskManager.Task::(_debug) Task 881ebb6e-3e89-42e4-98ea-470e78d73b13: ref 0 aborting False
>Thread-15915568::INFO::2013-11-20 04:03:05,492::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915569::DEBUG::2013-11-20 04:03:05,498::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915569::INFO::2013-11-20 04:03:05,500::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915569::DEBUG::2013-11-20 04:03:05,501::task::495::TaskManager.Task::(_debug) Task 6625ea19-9aa2-4781-ba33-c0a50607956b: moving from state init -> state preparing
>Thread-15915569::DEBUG::2013-11-20 04:03:05,503::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6dc7ae04-32a4-4293-891f-0e005e72123d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915569::DEBUG::2013-11-20 04:03:05,504::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915569::DEBUG::2013-11-20 04:03:05,506::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915569::DEBUG::2013-11-20 04:03:05,507::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6dc7ae04-32a4-4293-891f-0e005e72123d`::Granted request
>Thread-15915569::DEBUG::2013-11-20 04:03:05,509::task::495::TaskManager.Task::(_debug) Task 6625ea19-9aa2-4781-ba33-c0a50607956b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915569::DEBUG::2013-11-20 04:03:05,509::task::495::TaskManager.Task::(_debug) Task 6625ea19-9aa2-4781-ba33-c0a50607956b: ref 1 aborting False
>Thread-15915569::WARNING::2013-11-20 04:03:05,512::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915569::DEBUG::2013-11-20 04:03:05,514::task::495::TaskManager.Task::(_debug) Task 6625ea19-9aa2-4781-ba33-c0a50607956b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915569::DEBUG::2013-11-20 04:03:05,515::task::495::TaskManager.Task::(_debug) Task 6625ea19-9aa2-4781-ba33-c0a50607956b: moving from state preparing -> state finished
>Thread-15915569::DEBUG::2013-11-20 04:03:05,516::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915569::DEBUG::2013-11-20 04:03:05,516::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915569::DEBUG::2013-11-20 04:03:05,517::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915569::DEBUG::2013-11-20 04:03:05,518::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915569::DEBUG::2013-11-20 04:03:05,519::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915569::DEBUG::2013-11-20 04:03:05,519::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915569::DEBUG::2013-11-20 04:03:05,520::task::495::TaskManager.Task::(_debug) Task 6625ea19-9aa2-4781-ba33-c0a50607956b: ref 0 aborting False
>Thread-15915569::INFO::2013-11-20 04:03:05,521::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:03:05,827::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:03:06,364::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0852381 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:03:08,387::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:03:08,938::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0980886 s, 10.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:03:10,955::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:03:11,523::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0822102 s, 12.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:03:13,549::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:03:13,754::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:03:13,759::task::495::TaskManager.Task::(_debug) Task 92c0f1da-5a62-4358-aeec-016b461ae7d5: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:03:13,763::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bc08de90-fda6-4fb9-9bc8-5893a0218c38`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:03:13,765::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:03:13,769::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:03:13,772::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bc08de90-fda6-4fb9-9bc8-5893a0218c38`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:03:13,776::task::495::TaskManager.Task::(_debug) Task 92c0f1da-5a62-4358-aeec-016b461ae7d5: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:03:13,777::task::495::TaskManager.Task::(_debug) Task 92c0f1da-5a62-4358-aeec-016b461ae7d5: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:03:13,779::task::495::TaskManager.Task::(_debug) Task 92c0f1da-5a62-4358-aeec-016b461ae7d5: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:03:13,780::task::495::TaskManager.Task::(_debug) Task 92c0f1da-5a62-4358-aeec-016b461ae7d5: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:03:13,781::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:03:13,783::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:03:13,787::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:03:13,794::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:03:13,795::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:03:13,796::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:03:13,797::task::495::TaskManager.Task::(_debug) Task 92c0f1da-5a62-4358-aeec-016b461ae7d5: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:03:13,797::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:03:14,105::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0814966 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 04:03:14,674::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:03:14,675::task::495::TaskManager.Task::(_debug) Task a5e5a057-4ef9-4ac0-9673-d214f18f7532: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:03:14,676::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5f39a8fe-3424-438a-8cb3-66a528f1a072`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:03:14,677::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:03:14,678::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:03:14,679::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5f39a8fe-3424-438a-8cb3-66a528f1a072`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:03:14,680::task::495::TaskManager.Task::(_debug) Task a5e5a057-4ef9-4ac0-9673-d214f18f7532: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:03:14,680::task::495::TaskManager.Task::(_debug) Task a5e5a057-4ef9-4ac0-9673-d214f18f7532: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:03:14,681::task::495::TaskManager.Task::(_debug) Task a5e5a057-4ef9-4ac0-9673-d214f18f7532: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:03:14,682::task::495::TaskManager.Task::(_debug) Task a5e5a057-4ef9-4ac0-9673-d214f18f7532: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:03:14,683::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:03:14,683::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:03:14,684::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:03:14,685::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:03:14,685::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:03:14,686::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:03:14,687::task::495::TaskManager.Task::(_debug) Task a5e5a057-4ef9-4ac0-9673-d214f18f7532: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:03:14,688::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:03:14,689::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:03:14,689::task::495::TaskManager.Task::(_debug) Task 4a4fee7b-a5b0-40da-809d-d1173f88d6c2: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:03:14,690::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9ecf7635-f0cc-4265-9b27-45618536f311`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:03:14,691::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:03:14,692::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:03:14,692::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9ecf7635-f0cc-4265-9b27-45618536f311`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:03:14,693::task::495::TaskManager.Task::(_debug) Task 4a4fee7b-a5b0-40da-809d-d1173f88d6c2: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:03:14,694::task::495::TaskManager.Task::(_debug) Task 4a4fee7b-a5b0-40da-809d-d1173f88d6c2: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:03:14,695::task::495::TaskManager.Task::(_debug) Task 4a4fee7b-a5b0-40da-809d-d1173f88d6c2: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:03:14,695::task::495::TaskManager.Task::(_debug) Task 4a4fee7b-a5b0-40da-809d-d1173f88d6c2: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:03:14,696::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:03:14,696::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:03:14,697::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:03:14,698::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:03:14,698::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:03:14,699::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:03:14,699::task::495::TaskManager.Task::(_debug) Task 4a4fee7b-a5b0-40da-809d-d1173f88d6c2: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:03:14,700::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15915574::DEBUG::2013-11-20 04:03:15,692::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915574::INFO::2013-11-20 04:03:15,694::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915574::DEBUG::2013-11-20 04:03:15,695::task::495::TaskManager.Task::(_debug) Task dda41f7b-5197-4f1e-b47c-17588f99eb01: moving from state init -> state preparing
>Thread-15915574::DEBUG::2013-11-20 04:03:15,695::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915574::DEBUG::2013-11-20 04:03:15,696::task::495::TaskManager.Task::(_debug) Task dda41f7b-5197-4f1e-b47c-17588f99eb01: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915574::DEBUG::2013-11-20 04:03:15,697::task::495::TaskManager.Task::(_debug) Task dda41f7b-5197-4f1e-b47c-17588f99eb01: moving from state preparing -> state finished
>Thread-15915574::DEBUG::2013-11-20 04:03:15,697::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915574::DEBUG::2013-11-20 04:03:15,698::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915574::DEBUG::2013-11-20 04:03:15,698::task::495::TaskManager.Task::(_debug) Task dda41f7b-5197-4f1e-b47c-17588f99eb01: ref 0 aborting False
>Thread-15915574::INFO::2013-11-20 04:03:15,699::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915575::DEBUG::2013-11-20 04:03:15,704::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915575::INFO::2013-11-20 04:03:15,705::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915575::DEBUG::2013-11-20 04:03:15,706::task::495::TaskManager.Task::(_debug) Task 8f05c2d1-ea40-490a-8c95-a1d403ef86d0: moving from state init -> state preparing
>Thread-15915575::DEBUG::2013-11-20 04:03:15,707::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e7fb6f67-8972-468f-b90f-dc3874ba3eaf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915575::DEBUG::2013-11-20 04:03:15,707::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915575::DEBUG::2013-11-20 04:03:15,708::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915575::DEBUG::2013-11-20 04:03:15,709::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e7fb6f67-8972-468f-b90f-dc3874ba3eaf`::Granted request
>Thread-15915575::DEBUG::2013-11-20 04:03:15,710::task::495::TaskManager.Task::(_debug) Task 8f05c2d1-ea40-490a-8c95-a1d403ef86d0: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915575::DEBUG::2013-11-20 04:03:15,711::task::495::TaskManager.Task::(_debug) Task 8f05c2d1-ea40-490a-8c95-a1d403ef86d0: ref 1 aborting False
>Thread-15915575::WARNING::2013-11-20 04:03:15,713::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915575::DEBUG::2013-11-20 04:03:15,713::task::495::TaskManager.Task::(_debug) Task 8f05c2d1-ea40-490a-8c95-a1d403ef86d0: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915575::DEBUG::2013-11-20 04:03:15,714::task::495::TaskManager.Task::(_debug) Task 8f05c2d1-ea40-490a-8c95-a1d403ef86d0: moving from state preparing -> state finished
>Thread-15915575::DEBUG::2013-11-20 04:03:15,715::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915575::DEBUG::2013-11-20 04:03:15,715::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915575::DEBUG::2013-11-20 04:03:15,716::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915575::DEBUG::2013-11-20 04:03:15,717::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915575::DEBUG::2013-11-20 04:03:15,717::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915575::DEBUG::2013-11-20 04:03:15,718::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915575::DEBUG::2013-11-20 04:03:15,718::task::495::TaskManager.Task::(_debug) Task 8f05c2d1-ea40-490a-8c95-a1d403ef86d0: ref 0 aborting False
>Thread-15915575::INFO::2013-11-20 04:03:15,719::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:03:16,122::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:03:16,714::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0936615 s, 10.9 MB/s\n'; <rc> = 0
>Thread-15915576::INFO::2013-11-20 04:03:18,094::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915576::DEBUG::2013-11-20 04:03:18,095::task::495::TaskManager.Task::(_debug) Task e1e87ad9-9f32-4229-a7a3-6cef580cb026: moving from state init -> state preparing
>Thread-15915576::DEBUG::2013-11-20 04:03:18,096::task::495::TaskManager.Task::(_debug) Task e1e87ad9-9f32-4229-a7a3-6cef580cb026: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000414848327637', 'lastCheck': 1384920192.5691831, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.015153169632', 'lastCheck': 1384920193.1669371, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00141000747681', 'lastCheck': 1384920190.016551, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000459909439087', 'lastCheck': 1384920197.3506529, 'valid': True, 'code': 0}}
>Thread-15915576::DEBUG::2013-11-20 04:03:18,097::task::495::TaskManager.Task::(_debug) Task e1e87ad9-9f32-4229-a7a3-6cef580cb026: moving from state preparing -> state finished
>Thread-15915576::DEBUG::2013-11-20 04:03:18,097::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915576::DEBUG::2013-11-20 04:03:18,098::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915576::DEBUG::2013-11-20 04:03:18,098::task::495::TaskManager.Task::(_debug) Task e1e87ad9-9f32-4229-a7a3-6cef580cb026: ref 0 aborting False
>Thread-15915576::INFO::2013-11-20 04:03:18,099::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000414848327637', 'lastCheck': 1384920192.5691831, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.015153169632', 'lastCheck': 1384920193.1669371, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00141000747681', 'lastCheck': 1384920190.016551, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000459909439087', 'lastCheck': 1384920197.3506529, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:03:18,726::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:03:19,318::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0755056 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:03:21,331::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:03:21,907::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0875072 s, 11.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:03:23,923::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:03:24,516::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0931391 s, 11.0 MB/s\n'; <rc> = 0
>Thread-15915580::DEBUG::2013-11-20 04:03:26,119::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915580::INFO::2013-11-20 04:03:26,120::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915580::DEBUG::2013-11-20 04:03:26,120::task::495::TaskManager.Task::(_debug) Task 8f30691e-bd7b-4a09-adfc-022fa82e6de7: moving from state init -> state preparing
>Thread-15915580::DEBUG::2013-11-20 04:03:26,121::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915580::DEBUG::2013-11-20 04:03:26,122::task::495::TaskManager.Task::(_debug) Task 8f30691e-bd7b-4a09-adfc-022fa82e6de7: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915580::DEBUG::2013-11-20 04:03:26,122::task::495::TaskManager.Task::(_debug) Task 8f30691e-bd7b-4a09-adfc-022fa82e6de7: moving from state preparing -> state finished
>Thread-15915580::DEBUG::2013-11-20 04:03:26,122::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915580::DEBUG::2013-11-20 04:03:26,123::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915580::DEBUG::2013-11-20 04:03:26,123::task::495::TaskManager.Task::(_debug) Task 8f30691e-bd7b-4a09-adfc-022fa82e6de7: ref 0 aborting False
>Thread-15915580::INFO::2013-11-20 04:03:26,124::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915581::DEBUG::2013-11-20 04:03:26,127::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915581::INFO::2013-11-20 04:03:26,128::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915581::DEBUG::2013-11-20 04:03:26,129::task::495::TaskManager.Task::(_debug) Task b730e123-7447-4924-a8ef-8765264f0690: moving from state init -> state preparing
>Thread-15915581::DEBUG::2013-11-20 04:03:26,130::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d1f70e63-f1fe-4c53-aa92-ffdd073a8db0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915581::DEBUG::2013-11-20 04:03:26,130::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915581::DEBUG::2013-11-20 04:03:26,131::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915581::DEBUG::2013-11-20 04:03:26,131::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d1f70e63-f1fe-4c53-aa92-ffdd073a8db0`::Granted request
>Thread-15915581::DEBUG::2013-11-20 04:03:26,132::task::495::TaskManager.Task::(_debug) Task b730e123-7447-4924-a8ef-8765264f0690: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915581::DEBUG::2013-11-20 04:03:26,132::task::495::TaskManager.Task::(_debug) Task b730e123-7447-4924-a8ef-8765264f0690: ref 1 aborting False
>Thread-15915581::WARNING::2013-11-20 04:03:26,134::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915581::DEBUG::2013-11-20 04:03:26,135::task::495::TaskManager.Task::(_debug) Task b730e123-7447-4924-a8ef-8765264f0690: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915581::DEBUG::2013-11-20 04:03:26,135::task::495::TaskManager.Task::(_debug) Task b730e123-7447-4924-a8ef-8765264f0690: moving from state preparing -> state finished
>Thread-15915581::DEBUG::2013-11-20 04:03:26,136::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915581::DEBUG::2013-11-20 04:03:26,136::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915581::DEBUG::2013-11-20 04:03:26,137::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915581::DEBUG::2013-11-20 04:03:26,137::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915581::DEBUG::2013-11-20 04:03:26,137::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915581::DEBUG::2013-11-20 04:03:26,138::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915581::DEBUG::2013-11-20 04:03:26,138::task::495::TaskManager.Task::(_debug) Task b730e123-7447-4924-a8ef-8765264f0690: ref 0 aborting False
>Thread-15915581::INFO::2013-11-20 04:03:26,138::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:03:26,535::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:03:27,060::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0519878 s, 19.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:03:29,071::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 04:03:29,328::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:03:29,336::task::495::TaskManager.Task::(_debug) Task 0c328dc6-4a49-40d4-a91f-ba313f0daa08: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:03:29,338::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`774e5121-93e9-41a6-8b5c-549e5059e257`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:03:29,339::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:03:29,340::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:03:29,340::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`774e5121-93e9-41a6-8b5c-549e5059e257`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:03:29,342::task::495::TaskManager.Task::(_debug) Task 0c328dc6-4a49-40d4-a91f-ba313f0daa08: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:03:29,342::task::495::TaskManager.Task::(_debug) Task 0c328dc6-4a49-40d4-a91f-ba313f0daa08: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:03:29,343::task::495::TaskManager.Task::(_debug) Task 0c328dc6-4a49-40d4-a91f-ba313f0daa08: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:03:29,344::task::495::TaskManager.Task::(_debug) Task 0c328dc6-4a49-40d4-a91f-ba313f0daa08: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:03:29,344::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:03:29,345::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:03:29,345::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:03:29,346::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:03:29,347::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:03:29,347::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:03:29,348::task::495::TaskManager.Task::(_debug) Task 0c328dc6-4a49-40d4-a91f-ba313f0daa08: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:03:29,349::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:03:29,705::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0981575 s, 10.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:03:31,720::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:03:32,316::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0711891 s, 14.4 MB/s\n'; <rc> = 0
>Thread-15915584::INFO::2013-11-20 04:03:33,690::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915584::DEBUG::2013-11-20 04:03:33,691::task::495::TaskManager.Task::(_debug) Task 29693b9f-fd49-4fef-8a17-7d21a455beca: moving from state init -> state preparing
>Thread-15915584::DEBUG::2013-11-20 04:03:33,692::task::495::TaskManager.Task::(_debug) Task 29693b9f-fd49-4fef-8a17-7d21a455beca: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000332117080688', 'lastCheck': 1384920213.30688, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00160503387451', 'lastCheck': 1384920203.5398519, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00161695480347', 'lastCheck': 1384920210.5463991, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000352144241333', 'lastCheck': 1384920207.5383379, 'valid': True, 'code': 0}}
>Thread-15915584::DEBUG::2013-11-20 04:03:33,693::task::495::TaskManager.Task::(_debug) Task 29693b9f-fd49-4fef-8a17-7d21a455beca: moving from state preparing -> state finished
>Thread-15915584::DEBUG::2013-11-20 04:03:33,693::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915584::DEBUG::2013-11-20 04:03:33,694::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915584::DEBUG::2013-11-20 04:03:33,695::task::495::TaskManager.Task::(_debug) Task 29693b9f-fd49-4fef-8a17-7d21a455beca: ref 0 aborting False
>Thread-15915584::INFO::2013-11-20 04:03:33,696::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000332117080688', 'lastCheck': 1384920213.30688, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00160503387451', 'lastCheck': 1384920203.5398519, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00161695480347', 'lastCheck': 1384920210.5463991, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000352144241333', 'lastCheck': 1384920207.5383379, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:03:34,348::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:03:34,901::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0563379 s, 18.2 MB/s\n'; <rc> = 0
>Thread-15915586::DEBUG::2013-11-20 04:03:36,877::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915586::INFO::2013-11-20 04:03:36,879::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915586::DEBUG::2013-11-20 04:03:36,880::task::495::TaskManager.Task::(_debug) Task da9ac612-21e1-4b5e-b84e-027ae597ae25: moving from state init -> state preparing
>Thread-15915586::DEBUG::2013-11-20 04:03:36,880::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915586::DEBUG::2013-11-20 04:03:36,881::task::495::TaskManager.Task::(_debug) Task da9ac612-21e1-4b5e-b84e-027ae597ae25: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915586::DEBUG::2013-11-20 04:03:36,881::task::495::TaskManager.Task::(_debug) Task da9ac612-21e1-4b5e-b84e-027ae597ae25: moving from state preparing -> state finished
>Thread-15915586::DEBUG::2013-11-20 04:03:36,882::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915586::DEBUG::2013-11-20 04:03:36,882::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915586::DEBUG::2013-11-20 04:03:36,883::task::495::TaskManager.Task::(_debug) Task da9ac612-21e1-4b5e-b84e-027ae597ae25: ref 0 aborting False
>Thread-15915586::INFO::2013-11-20 04:03:36,883::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915587::DEBUG::2013-11-20 04:03:36,889::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915587::INFO::2013-11-20 04:03:36,890::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915587::DEBUG::2013-11-20 04:03:36,891::task::495::TaskManager.Task::(_debug) Task 42f0d533-afd0-459d-8517-086ab55c66d3: moving from state init -> state preparing
>Thread-15915587::DEBUG::2013-11-20 04:03:36,893::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c775cc39-2645-4ce1-af07-0c479bd98332`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915587::DEBUG::2013-11-20 04:03:36,893::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915587::DEBUG::2013-11-20 04:03:36,894::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915587::DEBUG::2013-11-20 04:03:36,895::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c775cc39-2645-4ce1-af07-0c479bd98332`::Granted request
>Thread-15915587::DEBUG::2013-11-20 04:03:36,896::task::495::TaskManager.Task::(_debug) Task 42f0d533-afd0-459d-8517-086ab55c66d3: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915587::DEBUG::2013-11-20 04:03:36,897::task::495::TaskManager.Task::(_debug) Task 42f0d533-afd0-459d-8517-086ab55c66d3: ref 1 aborting False
>Thread-15915587::WARNING::2013-11-20 04:03:36,899::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915587::DEBUG::2013-11-20 04:03:36,900::task::495::TaskManager.Task::(_debug) Task 42f0d533-afd0-459d-8517-086ab55c66d3: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915587::DEBUG::2013-11-20 04:03:36,901::task::495::TaskManager.Task::(_debug) Task 42f0d533-afd0-459d-8517-086ab55c66d3: moving from state preparing -> state finished
>Thread-15915587::DEBUG::2013-11-20 04:03:36,902::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915587::DEBUG::2013-11-20 04:03:36,903::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915587::DEBUG::2013-11-20 04:03:36,904::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915587::DEBUG::2013-11-20 04:03:36,904::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915587::DEBUG::2013-11-20 04:03:36,905::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915587::DEBUG::2013-11-20 04:03:36,906::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915587::DEBUG::2013-11-20 04:03:36,906::task::495::TaskManager.Task::(_debug) Task 42f0d533-afd0-459d-8517-086ab55c66d3: ref 0 aborting False
>Thread-15915587::INFO::2013-11-20 04:03:36,907::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:03:36,911::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:03:37,465::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0744011 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 04:03:39,348::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:03:39,362::task::495::TaskManager.Task::(_debug) Task 57a1ece1-381c-4664-968d-a9c8ac275db6: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:03:39,365::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`34d5b6f8-ef12-40fe-91be-e282194d6a85`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:03:39,366::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:03:39,368::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:03:39,368::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`34d5b6f8-ef12-40fe-91be-e282194d6a85`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:03:39,369::task::495::TaskManager.Task::(_debug) Task 57a1ece1-381c-4664-968d-a9c8ac275db6: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:03:39,370::task::495::TaskManager.Task::(_debug) Task 57a1ece1-381c-4664-968d-a9c8ac275db6: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:03:39,370::task::495::TaskManager.Task::(_debug) Task 57a1ece1-381c-4664-968d-a9c8ac275db6: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:03:39,371::task::495::TaskManager.Task::(_debug) Task 57a1ece1-381c-4664-968d-a9c8ac275db6: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:03:39,372::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:03:39,372::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:03:39,373::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:03:39,373::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:03:39,374::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:03:39,374::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:03:39,375::task::495::TaskManager.Task::(_debug) Task 57a1ece1-381c-4664-968d-a9c8ac275db6: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:03:39,376::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:03:39,479::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:03:40,117::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0802265 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:03:42,143::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 04:03:42,336::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:03:42,339::task::495::TaskManager.Task::(_debug) Task c088b763-9137-4409-a102-111912c6c566: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:03:42,342::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7d0e94f5-b6c3-406c-8486-ca0eed9457df`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:03:42,342::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:03:42,344::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:03:42,344::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7d0e94f5-b6c3-406c-8486-ca0eed9457df`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:03:42,345::task::495::TaskManager.Task::(_debug) Task c088b763-9137-4409-a102-111912c6c566: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:03:42,346::task::495::TaskManager.Task::(_debug) Task c088b763-9137-4409-a102-111912c6c566: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:03:42,347::task::495::TaskManager.Task::(_debug) Task c088b763-9137-4409-a102-111912c6c566: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:03:42,348::task::495::TaskManager.Task::(_debug) Task c088b763-9137-4409-a102-111912c6c566: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:03:42,348::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:03:42,349::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:03:42,349::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:03:42,350::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:03:42,350::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:03:42,350::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:03:42,351::task::495::TaskManager.Task::(_debug) Task c088b763-9137-4409-a102-111912c6c566: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:03:42,351::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:03:42,654::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0802589 s, 12.8 MB/s\n'; <rc> = 0
>Thread-15915085::INFO::2013-11-20 04:03:44,369::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=089b08fc-6544-4235-a061-fb55b09d2715 volUUID=66b599fc-b7b5-4934-a5d5-ef7283460029)
>Thread-15915085::DEBUG::2013-11-20 04:03:44,371::task::495::TaskManager.Task::(_debug) Task bd8f7309-27cb-4f04-91e2-e1efc5dd6400: moving from state init -> state preparing
>Thread-15915085::DEBUG::2013-11-20 04:03:44,373::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d0009ffb-1018-49c6-9e88-64b293e48db1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915085::DEBUG::2013-11-20 04:03:44,374::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915085::DEBUG::2013-11-20 04:03:44,376::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915085::DEBUG::2013-11-20 04:03:44,377::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d0009ffb-1018-49c6-9e88-64b293e48db1`::Granted request
>Thread-15915085::DEBUG::2013-11-20 04:03:44,379::task::495::TaskManager.Task::(_debug) Task bd8f7309-27cb-4f04-91e2-e1efc5dd6400: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915085::DEBUG::2013-11-20 04:03:44,380::task::495::TaskManager.Task::(_debug) Task bd8f7309-27cb-4f04-91e2-e1efc5dd6400: ref 1 aborting False
>Thread-15915085::DEBUG::2013-11-20 04:03:44,381::task::495::TaskManager.Task::(_debug) Task bd8f7309-27cb-4f04-91e2-e1efc5dd6400: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15915085::DEBUG::2013-11-20 04:03:44,381::task::495::TaskManager.Task::(_debug) Task bd8f7309-27cb-4f04-91e2-e1efc5dd6400: moving from state preparing -> state finished
>Thread-15915085::DEBUG::2013-11-20 04:03:44,382::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915085::DEBUG::2013-11-20 04:03:44,382::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915085::DEBUG::2013-11-20 04:03:44,383::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915085::DEBUG::2013-11-20 04:03:44,384::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915085::DEBUG::2013-11-20 04:03:44,385::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915085::DEBUG::2013-11-20 04:03:44,385::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915085::DEBUG::2013-11-20 04:03:44,386::task::495::TaskManager.Task::(_debug) Task bd8f7309-27cb-4f04-91e2-e1efc5dd6400: ref 0 aborting False
>Thread-15915085::INFO::2013-11-20 04:03:44,386::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Dummy-15912651::DEBUG::2013-11-20 04:03:44,667::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:03:45,241::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0759647 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15915592::DEBUG::2013-11-20 04:03:47,217::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915592::INFO::2013-11-20 04:03:47,219::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915592::DEBUG::2013-11-20 04:03:47,220::task::495::TaskManager.Task::(_debug) Task 7567988a-885b-4fa0-b4a9-1f4a6296afd1: moving from state init -> state preparing
>Thread-15915592::DEBUG::2013-11-20 04:03:47,221::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915592::DEBUG::2013-11-20 04:03:47,222::task::495::TaskManager.Task::(_debug) Task 7567988a-885b-4fa0-b4a9-1f4a6296afd1: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915592::DEBUG::2013-11-20 04:03:47,223::task::495::TaskManager.Task::(_debug) Task 7567988a-885b-4fa0-b4a9-1f4a6296afd1: moving from state preparing -> state finished
>Thread-15915592::DEBUG::2013-11-20 04:03:47,224::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915592::DEBUG::2013-11-20 04:03:47,224::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915592::DEBUG::2013-11-20 04:03:47,225::task::495::TaskManager.Task::(_debug) Task 7567988a-885b-4fa0-b4a9-1f4a6296afd1: ref 0 aborting False
>Thread-15915592::INFO::2013-11-20 04:03:47,226::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915593::DEBUG::2013-11-20 04:03:47,231::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915593::INFO::2013-11-20 04:03:47,233::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915593::DEBUG::2013-11-20 04:03:47,233::task::495::TaskManager.Task::(_debug) Task ef7fe385-6837-4200-b231-6ef448b4d6d2: moving from state init -> state preparing
>Thread-15915593::DEBUG::2013-11-20 04:03:47,235::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9b86d95b-9ee3-4fab-bb3e-15fbc9a6e2c6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915593::DEBUG::2013-11-20 04:03:47,236::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915593::DEBUG::2013-11-20 04:03:47,237::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915593::DEBUG::2013-11-20 04:03:47,238::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9b86d95b-9ee3-4fab-bb3e-15fbc9a6e2c6`::Granted request
>Thread-15915593::DEBUG::2013-11-20 04:03:47,239::task::495::TaskManager.Task::(_debug) Task ef7fe385-6837-4200-b231-6ef448b4d6d2: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915593::DEBUG::2013-11-20 04:03:47,240::task::495::TaskManager.Task::(_debug) Task ef7fe385-6837-4200-b231-6ef448b4d6d2: ref 1 aborting False
>Thread-15915593::WARNING::2013-11-20 04:03:47,242::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915593::DEBUG::2013-11-20 04:03:47,243::task::495::TaskManager.Task::(_debug) Task ef7fe385-6837-4200-b231-6ef448b4d6d2: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915593::DEBUG::2013-11-20 04:03:47,244::task::495::TaskManager.Task::(_debug) Task ef7fe385-6837-4200-b231-6ef448b4d6d2: moving from state preparing -> state finished
>Thread-15915593::DEBUG::2013-11-20 04:03:47,245::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915593::DEBUG::2013-11-20 04:03:47,245::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915593::DEBUG::2013-11-20 04:03:47,246::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915593::DEBUG::2013-11-20 04:03:47,247::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915593::DEBUG::2013-11-20 04:03:47,247::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915593::DEBUG::2013-11-20 04:03:47,248::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915593::DEBUG::2013-11-20 04:03:47,248::task::495::TaskManager.Task::(_debug) Task ef7fe385-6837-4200-b231-6ef448b4d6d2: ref 0 aborting False
>Thread-15915593::INFO::2013-11-20 04:03:47,249::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:03:47,257::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:03:47,798::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0806488 s, 12.7 MB/s\n'; <rc> = 0
>Thread-15915594::INFO::2013-11-20 04:03:49,268::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915594::DEBUG::2013-11-20 04:03:49,269::task::495::TaskManager.Task::(_debug) Task 5588335e-5c3b-43f0-bd93-6d8a35ddca9c: moving from state init -> state preparing
>Thread-15915594::DEBUG::2013-11-20 04:03:49,270::task::495::TaskManager.Task::(_debug) Task 5588335e-5c3b-43f0-bd93-6d8a35ddca9c: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00035285949707', 'lastCheck': 1384920223.324182, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00173401832581', 'lastCheck': 1384920224.1525109, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.014319896698', 'lastCheck': 1384920220.7389131, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000375986099243', 'lastCheck': 1384920227.8823149, 'valid': True, 'code': 0}}
>Thread-15915594::DEBUG::2013-11-20 04:03:49,271::task::495::TaskManager.Task::(_debug) Task 5588335e-5c3b-43f0-bd93-6d8a35ddca9c: moving from state preparing -> state finished
>Thread-15915594::DEBUG::2013-11-20 04:03:49,272::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915594::DEBUG::2013-11-20 04:03:49,272::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915594::DEBUG::2013-11-20 04:03:49,273::task::495::TaskManager.Task::(_debug) Task 5588335e-5c3b-43f0-bd93-6d8a35ddca9c: ref 0 aborting False
>Thread-15915594::INFO::2013-11-20 04:03:49,273::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00035285949707', 'lastCheck': 1384920223.324182, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00173401832581', 'lastCheck': 1384920224.1525109, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.014319896698', 'lastCheck': 1384920220.7389131, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000375986099243', 'lastCheck': 1384920227.8823149, 'valid': True, 'code': 0}}
>Thread-15914453::INFO::2013-11-20 04:03:49,476::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:03:49,476::task::495::TaskManager.Task::(_debug) Task 621f787e-0219-4c82-8264-d93190584388: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:03:49,478::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5227a64d-8358-4576-9e3c-8a6e3dd14c6c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:03:49,479::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:03:49,480::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:03:49,480::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5227a64d-8358-4576-9e3c-8a6e3dd14c6c`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:03:49,481::task::495::TaskManager.Task::(_debug) Task 621f787e-0219-4c82-8264-d93190584388: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:03:49,482::task::495::TaskManager.Task::(_debug) Task 621f787e-0219-4c82-8264-d93190584388: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:03:49,483::task::495::TaskManager.Task::(_debug) Task 621f787e-0219-4c82-8264-d93190584388: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:03:49,484::task::495::TaskManager.Task::(_debug) Task 621f787e-0219-4c82-8264-d93190584388: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:03:49,485::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:03:49,486::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:03:49,487::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:03:49,488::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:03:49,489::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:03:49,489::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:03:49,490::task::495::TaskManager.Task::(_debug) Task 621f787e-0219-4c82-8264-d93190584388: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:03:49,491::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:03:49,819::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:03:50,366::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0800052 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:03:52,382::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 04:03:52,651::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:03:52,656::task::495::TaskManager.Task::(_debug) Task c4187891-e253-4381-89c2-ccf74ae012d3: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:03:52,662::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6aef84e1-d9b3-4db3-9d1b-89e01404601d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:03:52,663::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:03:52,664::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:03:52,665::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6aef84e1-d9b3-4db3-9d1b-89e01404601d`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:03:52,666::task::495::TaskManager.Task::(_debug) Task c4187891-e253-4381-89c2-ccf74ae012d3: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:03:52,667::task::495::TaskManager.Task::(_debug) Task c4187891-e253-4381-89c2-ccf74ae012d3: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:03:52,668::task::495::TaskManager.Task::(_debug) Task c4187891-e253-4381-89c2-ccf74ae012d3: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:03:52,668::task::495::TaskManager.Task::(_debug) Task c4187891-e253-4381-89c2-ccf74ae012d3: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:03:52,669::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:03:52,669::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:03:52,670::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:03:52,671::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:03:52,671::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:03:52,672::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:03:52,673::task::495::TaskManager.Task::(_debug) Task c4187891-e253-4381-89c2-ccf74ae012d3: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:03:52,673::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15915596::DEBUG::2013-11-20 04:03:52,689::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Dummy-15912651::DEBUG::2013-11-20 04:03:53,112::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.19565 s, 5.2 MB/s\n'; <rc> = 0
>Thread-15915596::DEBUG::2013-11-20 04:03:53,341::clientIF::1219::vds::(fenceNode) rc 0 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = On
>Done
> err 
>Thread-15915598::DEBUG::2013-11-20 04:03:53,348::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Thread-15915598::DEBUG::2013-11-20 04:03:54,024::clientIF::1219::vds::(fenceNode) rc 0 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = On
>Done
> err 
>Dummy-15912651::DEBUG::2013-11-20 04:03:55,129::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:03:55,772::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.12267 s, 8.3 MB/s\n'; <rc> = 0
>Thread-15914519::INFO::2013-11-20 04:03:57,652::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:03:57,653::task::495::TaskManager.Task::(_debug) Task 9354be1c-0345-44a2-a6a1-10e34750fe5a: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:03:57,660::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9a066b9a-90ff-4023-b094-48fcdf7be8cd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:03:57,661::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:03:57,662::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:03:57,663::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9a066b9a-90ff-4023-b094-48fcdf7be8cd`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:03:57,665::task::495::TaskManager.Task::(_debug) Task 9354be1c-0345-44a2-a6a1-10e34750fe5a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 04:03:57,665::task::495::TaskManager.Task::(_debug) Task 9354be1c-0345-44a2-a6a1-10e34750fe5a: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:03:57,666::task::495::TaskManager.Task::(_debug) Task 9354be1c-0345-44a2-a6a1-10e34750fe5a: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 04:03:57,676::task::495::TaskManager.Task::(_debug) Task 9354be1c-0345-44a2-a6a1-10e34750fe5a: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:03:57,677::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:03:57,678::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:03:57,682::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 04:03:57,683::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:03:57,683::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:03:57,684::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:03:57,685::task::495::TaskManager.Task::(_debug) Task 9354be1c-0345-44a2-a6a1-10e34750fe5a: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 04:03:57,686::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 04:03:57,788::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915600::DEBUG::2013-11-20 04:03:58,009::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915600::INFO::2013-11-20 04:03:58,013::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915600::DEBUG::2013-11-20 04:03:58,014::task::495::TaskManager.Task::(_debug) Task a5da2c17-c33b-4f84-8659-691dce4434b4: moving from state init -> state preparing
>Thread-15915600::DEBUG::2013-11-20 04:03:58,015::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915600::DEBUG::2013-11-20 04:03:58,016::task::495::TaskManager.Task::(_debug) Task a5da2c17-c33b-4f84-8659-691dce4434b4: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915600::DEBUG::2013-11-20 04:03:58,017::task::495::TaskManager.Task::(_debug) Task a5da2c17-c33b-4f84-8659-691dce4434b4: moving from state preparing -> state finished
>Thread-15915600::DEBUG::2013-11-20 04:03:58,017::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915600::DEBUG::2013-11-20 04:03:58,018::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915600::DEBUG::2013-11-20 04:03:58,018::task::495::TaskManager.Task::(_debug) Task a5da2c17-c33b-4f84-8659-691dce4434b4: ref 0 aborting False
>Thread-15915600::INFO::2013-11-20 04:03:58,019::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915601::DEBUG::2013-11-20 04:03:58,025::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915601::INFO::2013-11-20 04:03:58,026::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915601::DEBUG::2013-11-20 04:03:58,027::task::495::TaskManager.Task::(_debug) Task abba4228-ee91-470a-8f57-3bc9f0917d64: moving from state init -> state preparing
>Thread-15915601::DEBUG::2013-11-20 04:03:58,028::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`322ce247-f474-48f3-927c-8467dc678417`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915601::DEBUG::2013-11-20 04:03:58,029::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915601::DEBUG::2013-11-20 04:03:58,030::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915601::DEBUG::2013-11-20 04:03:58,030::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`322ce247-f474-48f3-927c-8467dc678417`::Granted request
>Thread-15915601::DEBUG::2013-11-20 04:03:58,031::task::495::TaskManager.Task::(_debug) Task abba4228-ee91-470a-8f57-3bc9f0917d64: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915601::DEBUG::2013-11-20 04:03:58,032::task::495::TaskManager.Task::(_debug) Task abba4228-ee91-470a-8f57-3bc9f0917d64: ref 1 aborting False
>Thread-15915601::WARNING::2013-11-20 04:03:58,035::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915601::DEBUG::2013-11-20 04:03:58,037::task::495::TaskManager.Task::(_debug) Task abba4228-ee91-470a-8f57-3bc9f0917d64: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915601::DEBUG::2013-11-20 04:03:58,038::task::495::TaskManager.Task::(_debug) Task abba4228-ee91-470a-8f57-3bc9f0917d64: moving from state preparing -> state finished
>Thread-15915601::DEBUG::2013-11-20 04:03:58,039::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915601::DEBUG::2013-11-20 04:03:58,040::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915601::DEBUG::2013-11-20 04:03:58,041::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915601::DEBUG::2013-11-20 04:03:58,042::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915601::DEBUG::2013-11-20 04:03:58,043::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915601::DEBUG::2013-11-20 04:03:58,044::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915601::DEBUG::2013-11-20 04:03:58,044::task::495::TaskManager.Task::(_debug) Task abba4228-ee91-470a-8f57-3bc9f0917d64: ref 0 aborting False
>Thread-15915601::INFO::2013-11-20 04:03:58,045::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:03:58,318::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0582691 s, 17.6 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 04:03:59,644::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:03:59,645::task::495::TaskManager.Task::(_debug) Task f53edf41-fb0f-4c78-ab15-f44f696589de: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:03:59,647::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e0871136-8fe7-4e0f-aa35-ef8c549ebe41`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:03:59,648::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:03:59,649::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:03:59,650::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e0871136-8fe7-4e0f-aa35-ef8c549ebe41`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:03:59,651::task::495::TaskManager.Task::(_debug) Task f53edf41-fb0f-4c78-ab15-f44f696589de: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:03:59,652::task::495::TaskManager.Task::(_debug) Task f53edf41-fb0f-4c78-ab15-f44f696589de: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:03:59,652::task::495::TaskManager.Task::(_debug) Task f53edf41-fb0f-4c78-ab15-f44f696589de: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 04:03:59,653::task::495::TaskManager.Task::(_debug) Task f53edf41-fb0f-4c78-ab15-f44f696589de: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:03:59,654::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:03:59,655::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:03:59,656::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:03:59,657::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:03:59,658::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:03:59,659::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:03:59,660::task::495::TaskManager.Task::(_debug) Task f53edf41-fb0f-4c78-ab15-f44f696589de: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:03:59,661::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:03:59,662::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:03:59,663::task::495::TaskManager.Task::(_debug) Task 839fd808-b224-48cd-b4f3-5ad74dd72ea2: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:03:59,665::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8ba2db7c-c194-42a0-afc8-c967fbb3b100`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:03:59,666::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:03:59,667::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:03:59,667::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8ba2db7c-c194-42a0-afc8-c967fbb3b100`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:03:59,674::task::495::TaskManager.Task::(_debug) Task 839fd808-b224-48cd-b4f3-5ad74dd72ea2: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:03:59,675::task::495::TaskManager.Task::(_debug) Task 839fd808-b224-48cd-b4f3-5ad74dd72ea2: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:03:59,675::task::495::TaskManager.Task::(_debug) Task 839fd808-b224-48cd-b4f3-5ad74dd72ea2: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:03:59,676::task::495::TaskManager.Task::(_debug) Task 839fd808-b224-48cd-b4f3-5ad74dd72ea2: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:03:59,677::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:03:59,677::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:03:59,678::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:03:59,678::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:03:59,679::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:03:59,679::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:03:59,680::task::495::TaskManager.Task::(_debug) Task 839fd808-b224-48cd-b4f3-5ad74dd72ea2: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:03:59,681::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:03:59,681::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:03:59,682::task::495::TaskManager.Task::(_debug) Task e1004294-b371-43fb-843f-245d73f0c1df: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:03:59,683::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c3d4150c-a704-4991-a82c-46077e7da19f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:03:59,683::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:03:59,684::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:03:59,685::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c3d4150c-a704-4991-a82c-46077e7da19f`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:03:59,685::task::495::TaskManager.Task::(_debug) Task e1004294-b371-43fb-843f-245d73f0c1df: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:03:59,686::task::495::TaskManager.Task::(_debug) Task e1004294-b371-43fb-843f-245d73f0c1df: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:03:59,687::task::495::TaskManager.Task::(_debug) Task e1004294-b371-43fb-843f-245d73f0c1df: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:03:59,687::task::495::TaskManager.Task::(_debug) Task e1004294-b371-43fb-843f-245d73f0c1df: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:03:59,688::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:03:59,688::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:03:59,689::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:03:59,689::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:03:59,690::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:03:59,690::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:03:59,691::task::495::TaskManager.Task::(_debug) Task e1004294-b371-43fb-843f-245d73f0c1df: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:03:59,692::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:03:59,692::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:03:59,693::task::495::TaskManager.Task::(_debug) Task e25e45b0-8543-43d3-8f3b-2cc1daae1d82: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:03:59,694::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`3a36da49-9d5a-47c0-9262-2b2ecff5745b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:03:59,694::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:03:59,695::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:03:59,695::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`3a36da49-9d5a-47c0-9262-2b2ecff5745b`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:03:59,696::task::495::TaskManager.Task::(_debug) Task e25e45b0-8543-43d3-8f3b-2cc1daae1d82: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:03:59,697::task::495::TaskManager.Task::(_debug) Task e25e45b0-8543-43d3-8f3b-2cc1daae1d82: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:03:59,697::task::495::TaskManager.Task::(_debug) Task e25e45b0-8543-43d3-8f3b-2cc1daae1d82: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:03:59,698::task::495::TaskManager.Task::(_debug) Task e25e45b0-8543-43d3-8f3b-2cc1daae1d82: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:03:59,698::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:03:59,699::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:03:59,700::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:03:59,700::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:03:59,701::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:03:59,701::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:03:59,702::task::495::TaskManager.Task::(_debug) Task e25e45b0-8543-43d3-8f3b-2cc1daae1d82: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:03:59,703::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:04:00,329::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:04:00,879::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.08866 s, 11.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:04:02,895::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:04:03,465::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0903884 s, 11.3 MB/s\n'; <rc> = 0
>Thread-15915604::INFO::2013-11-20 04:04:05,166::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915604::DEBUG::2013-11-20 04:04:05,167::task::495::TaskManager.Task::(_debug) Task 50d54e8f-0b73-4b14-b08b-ab5d02d66d2b: moving from state init -> state preparing
>Thread-15915604::DEBUG::2013-11-20 04:04:05,168::task::495::TaskManager.Task::(_debug) Task 50d54e8f-0b73-4b14-b08b-ab5d02d66d2b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000356912612915', 'lastCheck': 1384920243.659678, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014660358429', 'lastCheck': 1384920244.5220211, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00164103507996', 'lastCheck': 1384920241.033319, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000947952270508', 'lastCheck': 1384920238.022897, 'valid': True, 'code': 0}}
>Thread-15915604::DEBUG::2013-11-20 04:04:05,169::task::495::TaskManager.Task::(_debug) Task 50d54e8f-0b73-4b14-b08b-ab5d02d66d2b: moving from state preparing -> state finished
>Thread-15915604::DEBUG::2013-11-20 04:04:05,169::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915604::DEBUG::2013-11-20 04:04:05,170::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915604::DEBUG::2013-11-20 04:04:05,170::task::495::TaskManager.Task::(_debug) Task 50d54e8f-0b73-4b14-b08b-ab5d02d66d2b: ref 0 aborting False
>Thread-15915604::INFO::2013-11-20 04:04:05,171::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000356912612915', 'lastCheck': 1384920243.659678, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014660358429', 'lastCheck': 1384920244.5220211, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00164103507996', 'lastCheck': 1384920241.033319, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000947952270508', 'lastCheck': 1384920238.022897, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:04:05,482::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:04:06,084::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0790558 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:04:08,101::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915606::DEBUG::2013-11-20 04:04:08,442::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915606::INFO::2013-11-20 04:04:08,445::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915606::DEBUG::2013-11-20 04:04:08,446::task::495::TaskManager.Task::(_debug) Task c6d5f246-6710-4f9c-a178-4fd753fcc9e7: moving from state init -> state preparing
>Thread-15915606::DEBUG::2013-11-20 04:04:08,447::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915606::DEBUG::2013-11-20 04:04:08,448::task::495::TaskManager.Task::(_debug) Task c6d5f246-6710-4f9c-a178-4fd753fcc9e7: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915606::DEBUG::2013-11-20 04:04:08,449::task::495::TaskManager.Task::(_debug) Task c6d5f246-6710-4f9c-a178-4fd753fcc9e7: moving from state preparing -> state finished
>Thread-15915606::DEBUG::2013-11-20 04:04:08,450::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915606::DEBUG::2013-11-20 04:04:08,450::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915606::DEBUG::2013-11-20 04:04:08,451::task::495::TaskManager.Task::(_debug) Task c6d5f246-6710-4f9c-a178-4fd753fcc9e7: ref 0 aborting False
>Thread-15915606::INFO::2013-11-20 04:04:08,452::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915607::DEBUG::2013-11-20 04:04:08,457::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915607::INFO::2013-11-20 04:04:08,458::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915607::DEBUG::2013-11-20 04:04:08,459::task::495::TaskManager.Task::(_debug) Task 7c45fa68-d0da-45fc-8714-e7b61e22901d: moving from state init -> state preparing
>Thread-15915607::DEBUG::2013-11-20 04:04:08,461::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e1c10764-0be0-45dd-9d05-73364ce50885`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915607::DEBUG::2013-11-20 04:04:08,461::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915607::DEBUG::2013-11-20 04:04:08,463::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915607::DEBUG::2013-11-20 04:04:08,463::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e1c10764-0be0-45dd-9d05-73364ce50885`::Granted request
>Thread-15915607::DEBUG::2013-11-20 04:04:08,464::task::495::TaskManager.Task::(_debug) Task 7c45fa68-d0da-45fc-8714-e7b61e22901d: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915607::DEBUG::2013-11-20 04:04:08,465::task::495::TaskManager.Task::(_debug) Task 7c45fa68-d0da-45fc-8714-e7b61e22901d: ref 1 aborting False
>Thread-15915607::WARNING::2013-11-20 04:04:08,468::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915607::DEBUG::2013-11-20 04:04:08,469::task::495::TaskManager.Task::(_debug) Task 7c45fa68-d0da-45fc-8714-e7b61e22901d: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915607::DEBUG::2013-11-20 04:04:08,470::task::495::TaskManager.Task::(_debug) Task 7c45fa68-d0da-45fc-8714-e7b61e22901d: moving from state preparing -> state finished
>Thread-15915607::DEBUG::2013-11-20 04:04:08,471::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915607::DEBUG::2013-11-20 04:04:08,472::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915607::DEBUG::2013-11-20 04:04:08,473::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915607::DEBUG::2013-11-20 04:04:08,474::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915607::DEBUG::2013-11-20 04:04:08,474::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915607::DEBUG::2013-11-20 04:04:08,475::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915607::DEBUG::2013-11-20 04:04:08,476::task::495::TaskManager.Task::(_debug) Task 7c45fa68-d0da-45fc-8714-e7b61e22901d: ref 0 aborting False
>Thread-15915607::INFO::2013-11-20 04:04:08,477::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:04:08,688::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0871113 s, 11.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:04:10,704::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:04:11,256::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0739444 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:04:13,274::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:04:13,837::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0667251 s, 15.3 MB/s\n'; <rc> = 0
>Thread-15913639::INFO::2013-11-20 04:04:14,525::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:04:14,526::task::495::TaskManager.Task::(_debug) Task 8b83b797-1e87-4263-8f3b-23e89282375e: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:04:14,529::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`051ec62a-eea0-4e90-ab5a-c7aed0791906`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:04:14,530::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:04:14,532::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:04:14,532::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`051ec62a-eea0-4e90-ab5a-c7aed0791906`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:04:14,534::task::495::TaskManager.Task::(_debug) Task 8b83b797-1e87-4263-8f3b-23e89282375e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:04:14,535::task::495::TaskManager.Task::(_debug) Task 8b83b797-1e87-4263-8f3b-23e89282375e: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:04:14,536::task::495::TaskManager.Task::(_debug) Task 8b83b797-1e87-4263-8f3b-23e89282375e: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:04:14,537::task::495::TaskManager.Task::(_debug) Task 8b83b797-1e87-4263-8f3b-23e89282375e: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:04:14,538::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:04:14,539::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:04:14,540::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:04:14,541::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:04:14,549::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:04:14,550::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:04:14,551::task::495::TaskManager.Task::(_debug) Task 8b83b797-1e87-4263-8f3b-23e89282375e: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:04:14,552::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:04:15,851::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:04:16,431::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0950617 s, 10.8 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 04:04:16,441::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:04:16,449::task::495::TaskManager.Task::(_debug) Task 50afe460-1c8b-4428-9bc9-1d6ce1790dd0: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:04:16,455::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5216d71b-f8b5-4252-99b8-687199875178`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:04:16,456::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:04:16,458::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:04:16,458::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5216d71b-f8b5-4252-99b8-687199875178`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:04:16,460::task::495::TaskManager.Task::(_debug) Task 50afe460-1c8b-4428-9bc9-1d6ce1790dd0: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:04:16,461::task::495::TaskManager.Task::(_debug) Task 50afe460-1c8b-4428-9bc9-1d6ce1790dd0: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:04:16,463::task::495::TaskManager.Task::(_debug) Task 50afe460-1c8b-4428-9bc9-1d6ce1790dd0: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:04:16,463::task::495::TaskManager.Task::(_debug) Task 50afe460-1c8b-4428-9bc9-1d6ce1790dd0: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:04:16,464::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:04:16,465::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:04:16,467::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:04:16,468::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:04:16,469::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:04:16,469::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:04:16,471::task::495::TaskManager.Task::(_debug) Task 50afe460-1c8b-4428-9bc9-1d6ce1790dd0: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:04:16,471::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:04:16,473::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:04:16,473::task::495::TaskManager.Task::(_debug) Task f3fb2c56-8218-49ff-a6d0-6772e3a3526b: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:04:16,475::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a5111b48-3831-4e0e-8f94-5a205dc5fd2d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:04:16,475::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:04:16,476::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:04:16,477::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a5111b48-3831-4e0e-8f94-5a205dc5fd2d`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:04:16,478::task::495::TaskManager.Task::(_debug) Task f3fb2c56-8218-49ff-a6d0-6772e3a3526b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:04:16,479::task::495::TaskManager.Task::(_debug) Task f3fb2c56-8218-49ff-a6d0-6772e3a3526b: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:04:16,480::task::495::TaskManager.Task::(_debug) Task f3fb2c56-8218-49ff-a6d0-6772e3a3526b: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:04:16,481::task::495::TaskManager.Task::(_debug) Task f3fb2c56-8218-49ff-a6d0-6772e3a3526b: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:04:16,481::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:04:16,482::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:04:16,483::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:04:16,484::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:04:16,485::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:04:16,485::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:04:16,486::task::495::TaskManager.Task::(_debug) Task f3fb2c56-8218-49ff-a6d0-6772e3a3526b: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:04:16,487::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:04:18,455::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915612::DEBUG::2013-11-20 04:04:18,739::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915612::INFO::2013-11-20 04:04:18,741::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915612::DEBUG::2013-11-20 04:04:18,742::task::495::TaskManager.Task::(_debug) Task c9a5bdb4-0a23-4ead-870d-266cf833aed2: moving from state init -> state preparing
>Thread-15915612::DEBUG::2013-11-20 04:04:18,743::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915612::DEBUG::2013-11-20 04:04:18,744::task::495::TaskManager.Task::(_debug) Task c9a5bdb4-0a23-4ead-870d-266cf833aed2: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915612::DEBUG::2013-11-20 04:04:18,745::task::495::TaskManager.Task::(_debug) Task c9a5bdb4-0a23-4ead-870d-266cf833aed2: moving from state preparing -> state finished
>Thread-15915612::DEBUG::2013-11-20 04:04:18,745::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915612::DEBUG::2013-11-20 04:04:18,746::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915612::DEBUG::2013-11-20 04:04:18,747::task::495::TaskManager.Task::(_debug) Task c9a5bdb4-0a23-4ead-870d-266cf833aed2: ref 0 aborting False
>Thread-15915612::INFO::2013-11-20 04:04:18,747::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915613::DEBUG::2013-11-20 04:04:18,753::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915613::INFO::2013-11-20 04:04:18,754::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915613::DEBUG::2013-11-20 04:04:18,755::task::495::TaskManager.Task::(_debug) Task 299e8f5c-9671-48c4-bb97-faa8590f8e07: moving from state init -> state preparing
>Thread-15915613::DEBUG::2013-11-20 04:04:18,757::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`34eb6e72-ffed-48c4-81f8-c3b3d506c92b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915613::DEBUG::2013-11-20 04:04:18,758::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915613::DEBUG::2013-11-20 04:04:18,759::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915613::DEBUG::2013-11-20 04:04:18,760::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`34eb6e72-ffed-48c4-81f8-c3b3d506c92b`::Granted request
>Thread-15915613::DEBUG::2013-11-20 04:04:18,761::task::495::TaskManager.Task::(_debug) Task 299e8f5c-9671-48c4-bb97-faa8590f8e07: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915613::DEBUG::2013-11-20 04:04:18,762::task::495::TaskManager.Task::(_debug) Task 299e8f5c-9671-48c4-bb97-faa8590f8e07: ref 1 aborting False
>Thread-15915613::WARNING::2013-11-20 04:04:18,766::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915613::DEBUG::2013-11-20 04:04:18,767::task::495::TaskManager.Task::(_debug) Task 299e8f5c-9671-48c4-bb97-faa8590f8e07: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915613::DEBUG::2013-11-20 04:04:18,769::task::495::TaskManager.Task::(_debug) Task 299e8f5c-9671-48c4-bb97-faa8590f8e07: moving from state preparing -> state finished
>Thread-15915613::DEBUG::2013-11-20 04:04:18,769::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915613::DEBUG::2013-11-20 04:04:18,770::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915613::DEBUG::2013-11-20 04:04:18,771::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915613::DEBUG::2013-11-20 04:04:18,772::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915613::DEBUG::2013-11-20 04:04:18,772::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915613::DEBUG::2013-11-20 04:04:18,773::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915613::DEBUG::2013-11-20 04:04:18,773::task::495::TaskManager.Task::(_debug) Task 299e8f5c-9671-48c4-bb97-faa8590f8e07: ref 0 aborting False
>Thread-15915613::INFO::2013-11-20 04:04:18,774::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:04:19,025::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.101309 s, 10.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:04:21,046::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915614::INFO::2013-11-20 04:04:21,270::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915614::DEBUG::2013-11-20 04:04:21,272::task::495::TaskManager.Task::(_debug) Task 1691cc8f-4366-455b-b298-ca36bc13aa79: moving from state init -> state preparing
>Thread-15915614::DEBUG::2013-11-20 04:04:21,273::task::495::TaskManager.Task::(_debug) Task 1691cc8f-4366-455b-b298-ca36bc13aa79: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000324964523315', 'lastCheck': 1384920253.9152441, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0173609256744', 'lastCheck': 1384920254.9347899, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00160598754883', 'lastCheck': 1384920251.312727, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00191116333008', 'lastCheck': 1384920258.649617, 'valid': True, 'code': 0}}
>Thread-15915614::DEBUG::2013-11-20 04:04:21,274::task::495::TaskManager.Task::(_debug) Task 1691cc8f-4366-455b-b298-ca36bc13aa79: moving from state preparing -> state finished
>Thread-15915614::DEBUG::2013-11-20 04:04:21,274::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915614::DEBUG::2013-11-20 04:04:21,275::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915614::DEBUG::2013-11-20 04:04:21,276::task::495::TaskManager.Task::(_debug) Task 1691cc8f-4366-455b-b298-ca36bc13aa79: ref 0 aborting False
>Thread-15915614::INFO::2013-11-20 04:04:21,276::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000324964523315', 'lastCheck': 1384920253.9152441, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0173609256744', 'lastCheck': 1384920254.9347899, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00160598754883', 'lastCheck': 1384920251.312727, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00191116333008', 'lastCheck': 1384920258.649617, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:04:21,570::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.083894 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:04:23,584::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:04:24,141::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.076906 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:04:26,155::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:04:26,724::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0801947 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:04:28,741::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915618::DEBUG::2013-11-20 04:04:29,052::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915618::INFO::2013-11-20 04:04:29,060::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915618::DEBUG::2013-11-20 04:04:29,062::task::495::TaskManager.Task::(_debug) Task e65390fc-9d03-4d87-8f12-0633d6ef5b09: moving from state init -> state preparing
>Thread-15915618::DEBUG::2013-11-20 04:04:29,064::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915618::DEBUG::2013-11-20 04:04:29,067::task::495::TaskManager.Task::(_debug) Task e65390fc-9d03-4d87-8f12-0633d6ef5b09: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915618::DEBUG::2013-11-20 04:04:29,069::task::495::TaskManager.Task::(_debug) Task e65390fc-9d03-4d87-8f12-0633d6ef5b09: moving from state preparing -> state finished
>Thread-15915618::DEBUG::2013-11-20 04:04:29,071::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915618::DEBUG::2013-11-20 04:04:29,076::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915618::DEBUG::2013-11-20 04:04:29,077::task::495::TaskManager.Task::(_debug) Task e65390fc-9d03-4d87-8f12-0633d6ef5b09: ref 0 aborting False
>Thread-15915618::INFO::2013-11-20 04:04:29,077::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915619::DEBUG::2013-11-20 04:04:29,084::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915619::INFO::2013-11-20 04:04:29,085::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915619::DEBUG::2013-11-20 04:04:29,086::task::495::TaskManager.Task::(_debug) Task 08ab8c1b-87bf-4c3f-a716-2233fe30f243: moving from state init -> state preparing
>Thread-15915619::DEBUG::2013-11-20 04:04:29,088::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bdda31c0-d06b-4270-9e29-95ede22a61aa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915619::DEBUG::2013-11-20 04:04:29,088::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915619::DEBUG::2013-11-20 04:04:29,090::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915619::DEBUG::2013-11-20 04:04:29,090::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bdda31c0-d06b-4270-9e29-95ede22a61aa`::Granted request
>Thread-15915619::DEBUG::2013-11-20 04:04:29,092::task::495::TaskManager.Task::(_debug) Task 08ab8c1b-87bf-4c3f-a716-2233fe30f243: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915619::DEBUG::2013-11-20 04:04:29,092::task::495::TaskManager.Task::(_debug) Task 08ab8c1b-87bf-4c3f-a716-2233fe30f243: ref 1 aborting False
>Thread-15915619::WARNING::2013-11-20 04:04:29,095::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915619::DEBUG::2013-11-20 04:04:29,096::task::495::TaskManager.Task::(_debug) Task 08ab8c1b-87bf-4c3f-a716-2233fe30f243: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915619::DEBUG::2013-11-20 04:04:29,098::task::495::TaskManager.Task::(_debug) Task 08ab8c1b-87bf-4c3f-a716-2233fe30f243: moving from state preparing -> state finished
>Thread-15915619::DEBUG::2013-11-20 04:04:29,098::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915619::DEBUG::2013-11-20 04:04:29,099::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915619::DEBUG::2013-11-20 04:04:29,100::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915619::DEBUG::2013-11-20 04:04:29,101::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915619::DEBUG::2013-11-20 04:04:29,102::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915619::DEBUG::2013-11-20 04:04:29,102::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915619::DEBUG::2013-11-20 04:04:29,103::task::495::TaskManager.Task::(_debug) Task 08ab8c1b-87bf-4c3f-a716-2233fe30f243: ref 0 aborting False
>Thread-15915619::INFO::2013-11-20 04:04:29,104::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:04:29,296::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0829292 s, 12.3 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 04:04:30,087::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:04:30,088::task::495::TaskManager.Task::(_debug) Task eb364a43-95cb-486f-9b22-6d99f021724b: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:04:30,090::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`81c341fe-67b8-4341-92aa-ef7394988885`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:04:30,091::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:04:30,091::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:04:30,092::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`81c341fe-67b8-4341-92aa-ef7394988885`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:04:30,093::task::495::TaskManager.Task::(_debug) Task eb364a43-95cb-486f-9b22-6d99f021724b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:04:30,094::task::495::TaskManager.Task::(_debug) Task eb364a43-95cb-486f-9b22-6d99f021724b: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:04:30,094::task::495::TaskManager.Task::(_debug) Task eb364a43-95cb-486f-9b22-6d99f021724b: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:04:30,095::task::495::TaskManager.Task::(_debug) Task eb364a43-95cb-486f-9b22-6d99f021724b: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:04:30,096::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:04:30,096::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:04:30,097::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:04:30,098::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:04:30,098::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:04:30,099::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:04:30,100::task::495::TaskManager.Task::(_debug) Task eb364a43-95cb-486f-9b22-6d99f021724b: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:04:30,100::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:04:31,311::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:04:31,899::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0775304 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:04:33,915::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:04:34,565::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.102404 s, 10.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:04:36,581::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915622::INFO::2013-11-20 04:04:36,852::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915622::DEBUG::2013-11-20 04:04:36,853::task::495::TaskManager.Task::(_debug) Task 3ac545f3-2cd1-4921-9f93-5b8ae6f5193f: moving from state init -> state preparing
>Thread-15915622::DEBUG::2013-11-20 04:04:36,855::task::495::TaskManager.Task::(_debug) Task 3ac545f3-2cd1-4921-9f93-5b8ae6f5193f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000671148300171', 'lastCheck': 1384920274.554409, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00344800949097', 'lastCheck': 1384920275.5536461, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0290517807007', 'lastCheck': 1384920271.995646, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00154495239258', 'lastCheck': 1384920268.962899, 'valid': True, 'code': 0}}
>Thread-15915622::DEBUG::2013-11-20 04:04:36,857::task::495::TaskManager.Task::(_debug) Task 3ac545f3-2cd1-4921-9f93-5b8ae6f5193f: moving from state preparing -> state finished
>Thread-15915622::DEBUG::2013-11-20 04:04:36,857::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915622::DEBUG::2013-11-20 04:04:36,858::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915622::DEBUG::2013-11-20 04:04:36,859::task::495::TaskManager.Task::(_debug) Task 3ac545f3-2cd1-4921-9f93-5b8ae6f5193f: ref 0 aborting False
>Thread-15915622::INFO::2013-11-20 04:04:36,860::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000671148300171', 'lastCheck': 1384920274.554409, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00344800949097', 'lastCheck': 1384920275.5536461, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0290517807007', 'lastCheck': 1384920271.995646, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00154495239258', 'lastCheck': 1384920268.962899, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:04:37,175::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.111573 s, 9.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:04:39,192::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915624::DEBUG::2013-11-20 04:04:39,459::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915624::INFO::2013-11-20 04:04:39,461::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915624::DEBUG::2013-11-20 04:04:39,462::task::495::TaskManager.Task::(_debug) Task a56f2e42-65ba-433c-8f7c-373e83d23a61: moving from state init -> state preparing
>Thread-15915624::DEBUG::2013-11-20 04:04:39,463::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915624::DEBUG::2013-11-20 04:04:39,464::task::495::TaskManager.Task::(_debug) Task a56f2e42-65ba-433c-8f7c-373e83d23a61: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915624::DEBUG::2013-11-20 04:04:39,464::task::495::TaskManager.Task::(_debug) Task a56f2e42-65ba-433c-8f7c-373e83d23a61: moving from state preparing -> state finished
>Thread-15915624::DEBUG::2013-11-20 04:04:39,465::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915624::DEBUG::2013-11-20 04:04:39,465::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915624::DEBUG::2013-11-20 04:04:39,466::task::495::TaskManager.Task::(_debug) Task a56f2e42-65ba-433c-8f7c-373e83d23a61: ref 0 aborting False
>Thread-15915624::INFO::2013-11-20 04:04:39,467::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915625::DEBUG::2013-11-20 04:04:39,472::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915625::INFO::2013-11-20 04:04:39,473::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915625::DEBUG::2013-11-20 04:04:39,474::task::495::TaskManager.Task::(_debug) Task 4e65d38b-d9a5-4204-b8e3-5a9fb7969caf: moving from state init -> state preparing
>Thread-15915625::DEBUG::2013-11-20 04:04:39,476::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0803c9e6-671c-4f21-99d7-53afb31bf371`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915625::DEBUG::2013-11-20 04:04:39,476::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915625::DEBUG::2013-11-20 04:04:39,478::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915625::DEBUG::2013-11-20 04:04:39,478::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0803c9e6-671c-4f21-99d7-53afb31bf371`::Granted request
>Thread-15915625::DEBUG::2013-11-20 04:04:39,479::task::495::TaskManager.Task::(_debug) Task 4e65d38b-d9a5-4204-b8e3-5a9fb7969caf: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915625::DEBUG::2013-11-20 04:04:39,480::task::495::TaskManager.Task::(_debug) Task 4e65d38b-d9a5-4204-b8e3-5a9fb7969caf: ref 1 aborting False
>Thread-15915625::WARNING::2013-11-20 04:04:39,483::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915625::DEBUG::2013-11-20 04:04:39,484::task::495::TaskManager.Task::(_debug) Task 4e65d38b-d9a5-4204-b8e3-5a9fb7969caf: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915625::DEBUG::2013-11-20 04:04:39,485::task::495::TaskManager.Task::(_debug) Task 4e65d38b-d9a5-4204-b8e3-5a9fb7969caf: moving from state preparing -> state finished
>Thread-15915625::DEBUG::2013-11-20 04:04:39,486::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915625::DEBUG::2013-11-20 04:04:39,487::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915625::DEBUG::2013-11-20 04:04:39,488::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915625::DEBUG::2013-11-20 04:04:39,488::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915625::DEBUG::2013-11-20 04:04:39,489::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915625::DEBUG::2013-11-20 04:04:39,490::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915625::DEBUG::2013-11-20 04:04:39,491::task::495::TaskManager.Task::(_debug) Task 4e65d38b-d9a5-4204-b8e3-5a9fb7969caf: ref 0 aborting False
>Thread-15915625::INFO::2013-11-20 04:04:39,491::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:04:39,678::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0713747 s, 14.3 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 04:04:39,820::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:04:39,821::task::495::TaskManager.Task::(_debug) Task 57577251-b10a-46d1-88c6-7bfdab1cf104: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:04:39,823::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8e63f6a3-7885-43f2-905e-897629f43853`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:04:39,823::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:04:39,824::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:04:39,825::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8e63f6a3-7885-43f2-905e-897629f43853`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:04:39,826::task::495::TaskManager.Task::(_debug) Task 57577251-b10a-46d1-88c6-7bfdab1cf104: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:04:39,826::task::495::TaskManager.Task::(_debug) Task 57577251-b10a-46d1-88c6-7bfdab1cf104: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:04:39,827::task::495::TaskManager.Task::(_debug) Task 57577251-b10a-46d1-88c6-7bfdab1cf104: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:04:39,827::task::495::TaskManager.Task::(_debug) Task 57577251-b10a-46d1-88c6-7bfdab1cf104: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:04:39,828::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:04:39,829::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:04:39,830::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:04:39,830::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:04:39,831::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:04:39,832::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:04:39,832::task::495::TaskManager.Task::(_debug) Task 57577251-b10a-46d1-88c6-7bfdab1cf104: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:04:39,833::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:04:41,694::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:04:42,265::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0952366 s, 10.8 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 04:04:43,127::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:04:43,129::task::495::TaskManager.Task::(_debug) Task e5cc2e4e-db1f-44f0-9b9c-e049510d4995: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:04:43,131::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5018f1b2-31e3-4117-87ec-c59f30406c8d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:04:43,131::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:04:43,133::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:04:43,133::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5018f1b2-31e3-4117-87ec-c59f30406c8d`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:04:43,135::task::495::TaskManager.Task::(_debug) Task e5cc2e4e-db1f-44f0-9b9c-e049510d4995: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:04:43,135::task::495::TaskManager.Task::(_debug) Task e5cc2e4e-db1f-44f0-9b9c-e049510d4995: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:04:43,136::task::495::TaskManager.Task::(_debug) Task e5cc2e4e-db1f-44f0-9b9c-e049510d4995: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:04:43,137::task::495::TaskManager.Task::(_debug) Task e5cc2e4e-db1f-44f0-9b9c-e049510d4995: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:04:43,138::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:04:43,139::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:04:43,140::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:04:43,140::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:04:43,141::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:04:43,142::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:04:43,143::task::495::TaskManager.Task::(_debug) Task e5cc2e4e-db1f-44f0-9b9c-e049510d4995: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:04:43,144::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:04:44,279::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915085::INFO::2013-11-20 04:04:44,777::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=089b08fc-6544-4235-a061-fb55b09d2715 volUUID=66b599fc-b7b5-4934-a5d5-ef7283460029)
>Thread-15915085::DEBUG::2013-11-20 04:04:44,780::task::495::TaskManager.Task::(_debug) Task f4d51017-242d-4ee5-b656-b651e06baa3c: moving from state init -> state preparing
>Thread-15915085::DEBUG::2013-11-20 04:04:44,787::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`811c9fd2-a341-4730-93dd-fdab3254dc91`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915085::DEBUG::2013-11-20 04:04:44,789::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915085::DEBUG::2013-11-20 04:04:44,792::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915085::DEBUG::2013-11-20 04:04:44,793::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`811c9fd2-a341-4730-93dd-fdab3254dc91`::Granted request
>Thread-15915085::DEBUG::2013-11-20 04:04:44,796::task::495::TaskManager.Task::(_debug) Task f4d51017-242d-4ee5-b656-b651e06baa3c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915085::DEBUG::2013-11-20 04:04:44,797::task::495::TaskManager.Task::(_debug) Task f4d51017-242d-4ee5-b656-b651e06baa3c: ref 1 aborting False
>Thread-15915085::DEBUG::2013-11-20 04:04:44,799::task::495::TaskManager.Task::(_debug) Task f4d51017-242d-4ee5-b656-b651e06baa3c: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15915085::DEBUG::2013-11-20 04:04:44,800::task::495::TaskManager.Task::(_debug) Task f4d51017-242d-4ee5-b656-b651e06baa3c: moving from state preparing -> state finished
>Thread-15915085::DEBUG::2013-11-20 04:04:44,801::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915085::DEBUG::2013-11-20 04:04:44,802::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915085::DEBUG::2013-11-20 04:04:44,804::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915085::DEBUG::2013-11-20 04:04:44,806::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915085::DEBUG::2013-11-20 04:04:44,807::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915085::DEBUG::2013-11-20 04:04:44,808::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915085::DEBUG::2013-11-20 04:04:44,810::task::495::TaskManager.Task::(_debug) Task f4d51017-242d-4ee5-b656-b651e06baa3c: ref 0 aborting False
>Thread-15915085::INFO::2013-11-20 04:04:44,811::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Dummy-15912651::DEBUG::2013-11-20 04:04:44,923::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.11715 s, 8.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:04:46,939::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:04:47,456::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0643843 s, 15.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:04:49,468::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 04:04:49,909::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:04:49,910::task::495::TaskManager.Task::(_debug) Task 881e22e5-ab70-4aa0-9404-ac290f74a0ea: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:04:49,914::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`25d050f3-ec64-46a6-8298-25201e73ee9a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:04:49,915::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:04:49,918::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:04:49,919::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`25d050f3-ec64-46a6-8298-25201e73ee9a`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:04:49,922::task::495::TaskManager.Task::(_debug) Task 881e22e5-ab70-4aa0-9404-ac290f74a0ea: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:04:49,923::task::495::TaskManager.Task::(_debug) Task 881e22e5-ab70-4aa0-9404-ac290f74a0ea: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:04:49,924::task::495::TaskManager.Task::(_debug) Task 881e22e5-ab70-4aa0-9404-ac290f74a0ea: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:04:49,925::task::495::TaskManager.Task::(_debug) Task 881e22e5-ab70-4aa0-9404-ac290f74a0ea: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:04:49,927::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:04:49,928::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:04:49,929::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:04:49,930::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:04:49,931::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:04:49,932::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:04:49,933::task::495::TaskManager.Task::(_debug) Task 881e22e5-ab70-4aa0-9404-ac290f74a0ea: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:04:49,934::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:04:50,027::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0941047 s, 10.9 MB/s\n'; <rc> = 0
>Thread-15915629::DEBUG::2013-11-20 04:04:50,156::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915629::INFO::2013-11-20 04:04:50,158::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915629::DEBUG::2013-11-20 04:04:50,159::task::495::TaskManager.Task::(_debug) Task 9daf2d5e-ef14-44bd-be0e-d72d22e0d227: moving from state init -> state preparing
>Thread-15915629::DEBUG::2013-11-20 04:04:50,160::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915629::DEBUG::2013-11-20 04:04:50,161::task::495::TaskManager.Task::(_debug) Task 9daf2d5e-ef14-44bd-be0e-d72d22e0d227: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915629::DEBUG::2013-11-20 04:04:50,162::task::495::TaskManager.Task::(_debug) Task 9daf2d5e-ef14-44bd-be0e-d72d22e0d227: moving from state preparing -> state finished
>Thread-15915629::DEBUG::2013-11-20 04:04:50,163::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915629::DEBUG::2013-11-20 04:04:50,163::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915629::DEBUG::2013-11-20 04:04:50,164::task::495::TaskManager.Task::(_debug) Task 9daf2d5e-ef14-44bd-be0e-d72d22e0d227: ref 0 aborting False
>Thread-15915629::INFO::2013-11-20 04:04:50,165::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915630::DEBUG::2013-11-20 04:04:50,221::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915630::INFO::2013-11-20 04:04:50,222::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915630::DEBUG::2013-11-20 04:04:50,223::task::495::TaskManager.Task::(_debug) Task 8477581a-bece-46a4-8c25-c9fc1a73f925: moving from state init -> state preparing
>Thread-15915630::DEBUG::2013-11-20 04:04:50,224::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c7449f7c-6ba3-4a24-9f0f-03fe80e078f2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915630::DEBUG::2013-11-20 04:04:50,225::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915630::DEBUG::2013-11-20 04:04:50,226::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915630::DEBUG::2013-11-20 04:04:50,227::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c7449f7c-6ba3-4a24-9f0f-03fe80e078f2`::Granted request
>Thread-15915630::DEBUG::2013-11-20 04:04:50,228::task::495::TaskManager.Task::(_debug) Task 8477581a-bece-46a4-8c25-c9fc1a73f925: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915630::DEBUG::2013-11-20 04:04:50,228::task::495::TaskManager.Task::(_debug) Task 8477581a-bece-46a4-8c25-c9fc1a73f925: ref 1 aborting False
>Thread-15915630::WARNING::2013-11-20 04:04:50,231::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915630::DEBUG::2013-11-20 04:04:50,232::task::495::TaskManager.Task::(_debug) Task 8477581a-bece-46a4-8c25-c9fc1a73f925: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915630::DEBUG::2013-11-20 04:04:50,233::task::495::TaskManager.Task::(_debug) Task 8477581a-bece-46a4-8c25-c9fc1a73f925: moving from state preparing -> state finished
>Thread-15915630::DEBUG::2013-11-20 04:04:50,234::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915630::DEBUG::2013-11-20 04:04:50,234::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915630::DEBUG::2013-11-20 04:04:50,235::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915630::DEBUG::2013-11-20 04:04:50,236::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915630::DEBUG::2013-11-20 04:04:50,237::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915630::DEBUG::2013-11-20 04:04:50,237::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915630::DEBUG::2013-11-20 04:04:50,238::task::495::TaskManager.Task::(_debug) Task 8477581a-bece-46a4-8c25-c9fc1a73f925: ref 0 aborting False
>Thread-15915630::INFO::2013-11-20 04:04:50,239::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:04:52,041::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:04:52,646::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.098535 s, 10.4 MB/s\n'; <rc> = 0
>Thread-15915632::INFO::2013-11-20 04:04:53,378::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915632::DEBUG::2013-11-20 04:04:53,379::task::495::TaskManager.Task::(_debug) Task 76c43e42-79a8-4b9a-8494-03277591a774: moving from state init -> state preparing
>Thread-15915632::DEBUG::2013-11-20 04:04:53,380::task::495::TaskManager.Task::(_debug) Task 76c43e42-79a8-4b9a-8494-03277591a774: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000478982925415', 'lastCheck': 1384920284.5691111, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00176501274109', 'lastCheck': 1384920285.5694411, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00169992446899', 'lastCheck': 1384920292.259917, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000308990478516', 'lastCheck': 1384920289.1808619, 'valid': True, 'code': 0}}
>Thread-15915632::DEBUG::2013-11-20 04:04:53,381::task::495::TaskManager.Task::(_debug) Task 76c43e42-79a8-4b9a-8494-03277591a774: moving from state preparing -> state finished
>Thread-15915632::DEBUG::2013-11-20 04:04:53,382::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915632::DEBUG::2013-11-20 04:04:53,383::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915632::DEBUG::2013-11-20 04:04:53,384::task::495::TaskManager.Task::(_debug) Task 76c43e42-79a8-4b9a-8494-03277591a774: ref 0 aborting False
>Thread-15915632::INFO::2013-11-20 04:04:53,384::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000478982925415', 'lastCheck': 1384920284.5691111, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00176501274109', 'lastCheck': 1384920285.5694411, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00169992446899', 'lastCheck': 1384920292.259917, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000308990478516', 'lastCheck': 1384920289.1808619, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:04:54,661::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 04:04:54,915::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:04:54,917::task::495::TaskManager.Task::(_debug) Task 11a7d873-d39c-44ad-b351-2e38894cc921: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:04:54,920::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`dafc28fa-0150-4c0b-b31a-95a4e7405c7b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:04:54,921::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:04:54,922::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:04:54,923::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`dafc28fa-0150-4c0b-b31a-95a4e7405c7b`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:04:54,924::task::495::TaskManager.Task::(_debug) Task 11a7d873-d39c-44ad-b351-2e38894cc921: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:04:54,926::task::495::TaskManager.Task::(_debug) Task 11a7d873-d39c-44ad-b351-2e38894cc921: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:04:54,927::task::495::TaskManager.Task::(_debug) Task 11a7d873-d39c-44ad-b351-2e38894cc921: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:04:54,927::task::495::TaskManager.Task::(_debug) Task 11a7d873-d39c-44ad-b351-2e38894cc921: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:04:54,928::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:04:54,929::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:04:54,930::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:04:54,931::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:04:54,932::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:04:54,932::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:04:54,933::task::495::TaskManager.Task::(_debug) Task 11a7d873-d39c-44ad-b351-2e38894cc921: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:04:54,934::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:04:55,152::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0521448 s, 19.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:04:57,164::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:04:57,709::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0808485 s, 12.7 MB/s\n'; <rc> = 0
>Thread-15914519::INFO::2013-11-20 04:04:57,926::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:04:57,941::task::495::TaskManager.Task::(_debug) Task aa9ba1cc-0635-4ff8-8a24-7f745d53841b: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:04:57,943::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`70117e5f-b025-41d4-a6d6-104d6e2549a3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:04:57,944::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:04:57,945::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:04:57,946::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`70117e5f-b025-41d4-a6d6-104d6e2549a3`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:04:57,947::task::495::TaskManager.Task::(_debug) Task aa9ba1cc-0635-4ff8-8a24-7f745d53841b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 04:04:57,948::task::495::TaskManager.Task::(_debug) Task aa9ba1cc-0635-4ff8-8a24-7f745d53841b: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:04:57,948::task::495::TaskManager.Task::(_debug) Task aa9ba1cc-0635-4ff8-8a24-7f745d53841b: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 04:04:57,949::task::495::TaskManager.Task::(_debug) Task aa9ba1cc-0635-4ff8-8a24-7f745d53841b: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:04:57,950::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:04:57,951::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:04:57,951::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 04:04:57,952::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:04:57,953::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:04:57,953::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:04:57,954::task::495::TaskManager.Task::(_debug) Task aa9ba1cc-0635-4ff8-8a24-7f745d53841b: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 04:04:57,954::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 04:04:59,725::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 04:04:59,981::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:04:59,987::task::495::TaskManager.Task::(_debug) Task 6695f661-b61a-4f74-9f3a-8f6f8e9019b3: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:04:59,992::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`52be5c52-77f6-4edb-a374-8464284e62f0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:04:59,994::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:04:59,995::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:04:59,996::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`52be5c52-77f6-4edb-a374-8464284e62f0`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:04:59,998::task::495::TaskManager.Task::(_debug) Task 6695f661-b61a-4f74-9f3a-8f6f8e9019b3: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:04:59,999::task::495::TaskManager.Task::(_debug) Task 6695f661-b61a-4f74-9f3a-8f6f8e9019b3: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:05:00,000::task::495::TaskManager.Task::(_debug) Task 6695f661-b61a-4f74-9f3a-8f6f8e9019b3: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 04:05:00,001::task::495::TaskManager.Task::(_debug) Task 6695f661-b61a-4f74-9f3a-8f6f8e9019b3: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:05:00,002::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:05:00,003::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:05:00,004::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:05:00,005::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:05:00,006::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:05:00,006::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:05:00,007::task::495::TaskManager.Task::(_debug) Task 6695f661-b61a-4f74-9f3a-8f6f8e9019b3: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:05:00,008::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:05:00,009::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:05:00,010::task::495::TaskManager.Task::(_debug) Task 681a71a8-94b0-4320-9450-8cf5af1111f2: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:05:00,012::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`36a3c7b7-1fb1-49f6-bdd8-244ce0c996d5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:05:00,013::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:05:00,014::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:05:00,015::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`36a3c7b7-1fb1-49f6-bdd8-244ce0c996d5`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:05:00,016::task::495::TaskManager.Task::(_debug) Task 681a71a8-94b0-4320-9450-8cf5af1111f2: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:05:00,017::task::495::TaskManager.Task::(_debug) Task 681a71a8-94b0-4320-9450-8cf5af1111f2: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:05:00,018::task::495::TaskManager.Task::(_debug) Task 681a71a8-94b0-4320-9450-8cf5af1111f2: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:05:00,019::task::495::TaskManager.Task::(_debug) Task 681a71a8-94b0-4320-9450-8cf5af1111f2: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:05:00,019::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:05:00,020::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:05:00,021::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:05:00,022::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:05:00,023::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:05:00,023::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:05:00,024::task::495::TaskManager.Task::(_debug) Task 681a71a8-94b0-4320-9450-8cf5af1111f2: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:05:00,025::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:05:00,026::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:05:00,027::task::495::TaskManager.Task::(_debug) Task 7f7fc7c7-7196-4578-9fc4-8fca7de4b8e2: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:05:00,029::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f7dfad80-8fd8-4c6c-aefa-554f74ea9864`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:05:00,030::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:05:00,031::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:05:00,031::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f7dfad80-8fd8-4c6c-aefa-554f74ea9864`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:05:00,033::task::495::TaskManager.Task::(_debug) Task 7f7fc7c7-7196-4578-9fc4-8fca7de4b8e2: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:05:00,034::task::495::TaskManager.Task::(_debug) Task 7f7fc7c7-7196-4578-9fc4-8fca7de4b8e2: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:05:00,034::task::495::TaskManager.Task::(_debug) Task 7f7fc7c7-7196-4578-9fc4-8fca7de4b8e2: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:05:00,035::task::495::TaskManager.Task::(_debug) Task 7f7fc7c7-7196-4578-9fc4-8fca7de4b8e2: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:05:00,036::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:05:00,037::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:05:00,038::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:05:00,039::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:05:00,039::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:05:00,040::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:05:00,041::task::495::TaskManager.Task::(_debug) Task 7f7fc7c7-7196-4578-9fc4-8fca7de4b8e2: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:05:00,041::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:05:00,043::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:05:00,043::task::495::TaskManager.Task::(_debug) Task 6f1709fe-681a-4504-b110-2a72702cfc9d: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:05:00,045::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`27e8479e-648d-415a-8ef2-5a4f65db1bf9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:05:00,045::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:05:00,046::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:05:00,047::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`27e8479e-648d-415a-8ef2-5a4f65db1bf9`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:05:00,048::task::495::TaskManager.Task::(_debug) Task 6f1709fe-681a-4504-b110-2a72702cfc9d: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:05:00,049::task::495::TaskManager.Task::(_debug) Task 6f1709fe-681a-4504-b110-2a72702cfc9d: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:05:00,049::task::495::TaskManager.Task::(_debug) Task 6f1709fe-681a-4504-b110-2a72702cfc9d: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:05:00,050::task::495::TaskManager.Task::(_debug) Task 6f1709fe-681a-4504-b110-2a72702cfc9d: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:05:00,051::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:05:00,051::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:05:00,052::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:05:00,053::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:05:00,054::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:05:00,054::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:05:00,055::task::495::TaskManager.Task::(_debug) Task 6f1709fe-681a-4504-b110-2a72702cfc9d: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:05:00,056::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:05:00,327::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0936757 s, 10.9 MB/s\n'; <rc> = 0
>Thread-15915636::DEBUG::2013-11-20 04:05:00,760::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915636::INFO::2013-11-20 04:05:00,762::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915636::DEBUG::2013-11-20 04:05:00,763::task::495::TaskManager.Task::(_debug) Task f832ced8-e670-4a38-9ca9-e044a3eea4a4: moving from state init -> state preparing
>Thread-15915636::DEBUG::2013-11-20 04:05:00,764::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915636::DEBUG::2013-11-20 04:05:00,764::task::495::TaskManager.Task::(_debug) Task f832ced8-e670-4a38-9ca9-e044a3eea4a4: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915636::DEBUG::2013-11-20 04:05:00,765::task::495::TaskManager.Task::(_debug) Task f832ced8-e670-4a38-9ca9-e044a3eea4a4: moving from state preparing -> state finished
>Thread-15915636::DEBUG::2013-11-20 04:05:00,766::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915636::DEBUG::2013-11-20 04:05:00,767::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915636::DEBUG::2013-11-20 04:05:00,768::task::495::TaskManager.Task::(_debug) Task f832ced8-e670-4a38-9ca9-e044a3eea4a4: ref 0 aborting False
>Thread-15915636::INFO::2013-11-20 04:05:00,768::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915637::DEBUG::2013-11-20 04:05:00,774::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915637::INFO::2013-11-20 04:05:00,775::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915637::DEBUG::2013-11-20 04:05:00,776::task::495::TaskManager.Task::(_debug) Task ff05e47b-633b-4688-97ac-33ed198a15d5: moving from state init -> state preparing
>Thread-15915637::DEBUG::2013-11-20 04:05:00,786::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`fb3d1092-db9d-4570-8b1c-267c580f2f1f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915637::DEBUG::2013-11-20 04:05:00,787::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915637::DEBUG::2013-11-20 04:05:00,788::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915637::DEBUG::2013-11-20 04:05:00,789::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`fb3d1092-db9d-4570-8b1c-267c580f2f1f`::Granted request
>Thread-15915637::DEBUG::2013-11-20 04:05:00,790::task::495::TaskManager.Task::(_debug) Task ff05e47b-633b-4688-97ac-33ed198a15d5: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915637::DEBUG::2013-11-20 04:05:00,790::task::495::TaskManager.Task::(_debug) Task ff05e47b-633b-4688-97ac-33ed198a15d5: ref 1 aborting False
>Thread-15915637::WARNING::2013-11-20 04:05:00,792::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915637::DEBUG::2013-11-20 04:05:00,793::task::495::TaskManager.Task::(_debug) Task ff05e47b-633b-4688-97ac-33ed198a15d5: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115247939584', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115247939584', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915637::DEBUG::2013-11-20 04:05:00,794::task::495::TaskManager.Task::(_debug) Task ff05e47b-633b-4688-97ac-33ed198a15d5: moving from state preparing -> state finished
>Thread-15915637::DEBUG::2013-11-20 04:05:00,795::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915637::DEBUG::2013-11-20 04:05:00,795::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915637::DEBUG::2013-11-20 04:05:00,796::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915637::DEBUG::2013-11-20 04:05:00,797::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915637::DEBUG::2013-11-20 04:05:00,798::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915637::DEBUG::2013-11-20 04:05:00,798::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915637::DEBUG::2013-11-20 04:05:00,799::task::495::TaskManager.Task::(_debug) Task ff05e47b-633b-4688-97ac-33ed198a15d5: ref 0 aborting False
>Thread-15915637::INFO::2013-11-20 04:05:00,800::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115247939584', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115247939584', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:05:02,342::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:05:02,923::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0920677 s, 11.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:05:04,941::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:05:05,477::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0623438 s, 16.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:05:07,491::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:05:08,044::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0740678 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15915640::INFO::2013-11-20 04:05:09,068::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915640::DEBUG::2013-11-20 04:05:09,069::task::495::TaskManager.Task::(_debug) Task 547e0d3f-efc2-431f-b175-521a64e865f7: moving from state init -> state preparing
>Thread-15915640::DEBUG::2013-11-20 04:05:09,070::task::495::TaskManager.Task::(_debug) Task 547e0d3f-efc2-431f-b175-521a64e865f7: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00369191169739', 'lastCheck': 1384920305.2232759, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0013861656189', 'lastCheck': 1384920306.181375, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00129294395447', 'lastCheck': 1384920302.6016321, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00046706199646', 'lastCheck': 1384920299.372822, 'valid': True, 'code': 0}}
>Thread-15915640::DEBUG::2013-11-20 04:05:09,072::task::495::TaskManager.Task::(_debug) Task 547e0d3f-efc2-431f-b175-521a64e865f7: moving from state preparing -> state finished
>Thread-15915640::DEBUG::2013-11-20 04:05:09,072::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915640::DEBUG::2013-11-20 04:05:09,073::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915640::DEBUG::2013-11-20 04:05:09,074::task::495::TaskManager.Task::(_debug) Task 547e0d3f-efc2-431f-b175-521a64e865f7: ref 0 aborting False
>Thread-15915640::INFO::2013-11-20 04:05:09,075::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00369191169739', 'lastCheck': 1384920305.2232759, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0013861656189', 'lastCheck': 1384920306.181375, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00129294395447', 'lastCheck': 1384920302.6016321, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00046706199646', 'lastCheck': 1384920299.372822, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:05:10,063::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:05:10,662::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0850816 s, 12.0 MB/s\n'; <rc> = 0
>Thread-15915642::DEBUG::2013-11-20 04:05:11,056::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915642::INFO::2013-11-20 04:05:11,058::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915642::DEBUG::2013-11-20 04:05:11,059::task::495::TaskManager.Task::(_debug) Task 93f814de-e9e2-4280-8867-31e1e47f1d6a: moving from state init -> state preparing
>Thread-15915642::DEBUG::2013-11-20 04:05:11,060::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915642::DEBUG::2013-11-20 04:05:11,061::task::495::TaskManager.Task::(_debug) Task 93f814de-e9e2-4280-8867-31e1e47f1d6a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915642::DEBUG::2013-11-20 04:05:11,061::task::495::TaskManager.Task::(_debug) Task 93f814de-e9e2-4280-8867-31e1e47f1d6a: moving from state preparing -> state finished
>Thread-15915642::DEBUG::2013-11-20 04:05:11,062::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915642::DEBUG::2013-11-20 04:05:11,062::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915642::DEBUG::2013-11-20 04:05:11,063::task::495::TaskManager.Task::(_debug) Task 93f814de-e9e2-4280-8867-31e1e47f1d6a: ref 0 aborting False
>Thread-15915642::INFO::2013-11-20 04:05:11,064::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915643::DEBUG::2013-11-20 04:05:11,068::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915643::INFO::2013-11-20 04:05:11,070::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915643::DEBUG::2013-11-20 04:05:11,070::task::495::TaskManager.Task::(_debug) Task de63d15e-4e55-4290-b757-c8c508e607b7: moving from state init -> state preparing
>Thread-15915643::DEBUG::2013-11-20 04:05:11,071::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f7cc7cc2-9469-4eee-8c84-866efbcff32c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915643::DEBUG::2013-11-20 04:05:11,072::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915643::DEBUG::2013-11-20 04:05:11,073::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915643::DEBUG::2013-11-20 04:05:11,073::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f7cc7cc2-9469-4eee-8c84-866efbcff32c`::Granted request
>Thread-15915643::DEBUG::2013-11-20 04:05:11,075::task::495::TaskManager.Task::(_debug) Task de63d15e-4e55-4290-b757-c8c508e607b7: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915643::DEBUG::2013-11-20 04:05:11,075::task::495::TaskManager.Task::(_debug) Task de63d15e-4e55-4290-b757-c8c508e607b7: ref 1 aborting False
>Thread-15915643::WARNING::2013-11-20 04:05:11,079::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915643::DEBUG::2013-11-20 04:05:11,080::task::495::TaskManager.Task::(_debug) Task de63d15e-4e55-4290-b757-c8c508e607b7: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115247939584', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115247939584', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915643::DEBUG::2013-11-20 04:05:11,081::task::495::TaskManager.Task::(_debug) Task de63d15e-4e55-4290-b757-c8c508e607b7: moving from state preparing -> state finished
>Thread-15915643::DEBUG::2013-11-20 04:05:11,081::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915643::DEBUG::2013-11-20 04:05:11,082::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915643::DEBUG::2013-11-20 04:05:11,082::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915643::DEBUG::2013-11-20 04:05:11,083::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915643::DEBUG::2013-11-20 04:05:11,084::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915643::DEBUG::2013-11-20 04:05:11,084::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915643::DEBUG::2013-11-20 04:05:11,085::task::495::TaskManager.Task::(_debug) Task de63d15e-4e55-4290-b757-c8c508e607b7: ref 0 aborting False
>Thread-15915643::INFO::2013-11-20 04:05:11,085::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115247939584', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115247939584', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:05:12,688::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:05:13,190::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0516991 s, 19.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:05:15,201::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:05:15,421::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:05:15,424::task::495::TaskManager.Task::(_debug) Task ef8e2291-3d3e-4f22-90b0-e82a9837268e: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:05:15,427::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7813d696-3a62-4903-ba5b-8ced6135501c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:05:15,428::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:05:15,431::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:05:15,432::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7813d696-3a62-4903-ba5b-8ced6135501c`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:05:15,433::task::495::TaskManager.Task::(_debug) Task ef8e2291-3d3e-4f22-90b0-e82a9837268e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:05:15,440::task::495::TaskManager.Task::(_debug) Task ef8e2291-3d3e-4f22-90b0-e82a9837268e: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:05:15,440::task::495::TaskManager.Task::(_debug) Task ef8e2291-3d3e-4f22-90b0-e82a9837268e: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:05:15,441::task::495::TaskManager.Task::(_debug) Task ef8e2291-3d3e-4f22-90b0-e82a9837268e: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:05:15,442::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:05:15,443::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:05:15,444::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:05:15,444::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:05:15,445::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:05:15,446::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:05:15,447::task::495::TaskManager.Task::(_debug) Task ef8e2291-3d3e-4f22-90b0-e82a9837268e: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:05:15,447::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:05:15,769::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0877402 s, 11.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:05:17,786::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913518::INFO::2013-11-20 04:05:18,042::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:05:18,043::task::495::TaskManager.Task::(_debug) Task fecd393a-bbd9-4ac7-afd9-394ea0dad7cf: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:05:18,046::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bca3e456-9d61-4422-a43f-03d504af027b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:05:18,047::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:05:18,049::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:05:18,049::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bca3e456-9d61-4422-a43f-03d504af027b`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:05:18,051::task::495::TaskManager.Task::(_debug) Task fecd393a-bbd9-4ac7-afd9-394ea0dad7cf: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:05:18,052::task::495::TaskManager.Task::(_debug) Task fecd393a-bbd9-4ac7-afd9-394ea0dad7cf: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:05:18,053::task::495::TaskManager.Task::(_debug) Task fecd393a-bbd9-4ac7-afd9-394ea0dad7cf: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:05:18,054::task::495::TaskManager.Task::(_debug) Task fecd393a-bbd9-4ac7-afd9-394ea0dad7cf: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:05:18,055::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:05:18,055::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:05:18,056::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:05:18,057::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:05:18,058::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:05:18,059::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:05:18,059::task::495::TaskManager.Task::(_debug) Task fecd393a-bbd9-4ac7-afd9-394ea0dad7cf: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:05:18,060::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:05:18,061::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:05:18,062::task::495::TaskManager.Task::(_debug) Task 202691fe-0150-44fb-a5e1-638ab13d5b6d: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:05:18,063::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2290e791-3152-4145-a903-ecdd2dc8df9c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:05:18,065::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:05:18,066::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:05:18,067::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2290e791-3152-4145-a903-ecdd2dc8df9c`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:05:18,068::task::495::TaskManager.Task::(_debug) Task 202691fe-0150-44fb-a5e1-638ab13d5b6d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:05:18,068::task::495::TaskManager.Task::(_debug) Task 202691fe-0150-44fb-a5e1-638ab13d5b6d: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:05:18,069::task::495::TaskManager.Task::(_debug) Task 202691fe-0150-44fb-a5e1-638ab13d5b6d: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:05:18,070::task::495::TaskManager.Task::(_debug) Task 202691fe-0150-44fb-a5e1-638ab13d5b6d: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:05:18,071::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:05:18,071::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:05:18,072::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:05:18,073::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:05:18,074::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:05:18,074::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:05:18,075::task::495::TaskManager.Task::(_debug) Task 202691fe-0150-44fb-a5e1-638ab13d5b6d: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:05:18,076::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:05:18,394::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.101942 s, 10.0 MB/s\n'; <rc> = 0
>Thread-42840::DEBUG::2013-11-20 04:05:20,014::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:05:20,416::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-42840::DEBUG::2013-11-20 04:05:20,818::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:05:21,090::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.112609 s, 9.1 MB/s\n'; <rc> = 0
>Thread-15915647::DEBUG::2013-11-20 04:05:22,405::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915647::INFO::2013-11-20 04:05:22,407::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915647::DEBUG::2013-11-20 04:05:22,408::task::495::TaskManager.Task::(_debug) Task 3e7fdac6-554e-4eb8-a544-29fcf81f6552: moving from state init -> state preparing
>Thread-15915647::DEBUG::2013-11-20 04:05:22,409::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915647::DEBUG::2013-11-20 04:05:22,410::task::495::TaskManager.Task::(_debug) Task 3e7fdac6-554e-4eb8-a544-29fcf81f6552: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915647::DEBUG::2013-11-20 04:05:22,411::task::495::TaskManager.Task::(_debug) Task 3e7fdac6-554e-4eb8-a544-29fcf81f6552: moving from state preparing -> state finished
>Thread-15915647::DEBUG::2013-11-20 04:05:22,412::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915647::DEBUG::2013-11-20 04:05:22,413::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915647::DEBUG::2013-11-20 04:05:22,413::task::495::TaskManager.Task::(_debug) Task 3e7fdac6-554e-4eb8-a544-29fcf81f6552: ref 0 aborting False
>Thread-15915647::INFO::2013-11-20 04:05:22,414::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915648::DEBUG::2013-11-20 04:05:22,420::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915648::INFO::2013-11-20 04:05:22,421::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915648::DEBUG::2013-11-20 04:05:22,422::task::495::TaskManager.Task::(_debug) Task 5ebb360e-1a1a-410d-a751-77d0d55206d0: moving from state init -> state preparing
>Thread-15915648::DEBUG::2013-11-20 04:05:22,424::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`02145164-6bbe-43ac-90b0-d37d1a4bb16f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915648::DEBUG::2013-11-20 04:05:22,425::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915648::DEBUG::2013-11-20 04:05:22,426::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915648::DEBUG::2013-11-20 04:05:22,427::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`02145164-6bbe-43ac-90b0-d37d1a4bb16f`::Granted request
>Thread-15915648::DEBUG::2013-11-20 04:05:22,428::task::495::TaskManager.Task::(_debug) Task 5ebb360e-1a1a-410d-a751-77d0d55206d0: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915648::DEBUG::2013-11-20 04:05:22,429::task::495::TaskManager.Task::(_debug) Task 5ebb360e-1a1a-410d-a751-77d0d55206d0: ref 1 aborting False
>Thread-15915648::WARNING::2013-11-20 04:05:22,431::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915648::DEBUG::2013-11-20 04:05:22,433::task::495::TaskManager.Task::(_debug) Task 5ebb360e-1a1a-410d-a751-77d0d55206d0: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915648::DEBUG::2013-11-20 04:05:22,434::task::495::TaskManager.Task::(_debug) Task 5ebb360e-1a1a-410d-a751-77d0d55206d0: moving from state preparing -> state finished
>Thread-15915648::DEBUG::2013-11-20 04:05:22,434::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915648::DEBUG::2013-11-20 04:05:22,435::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915648::DEBUG::2013-11-20 04:05:22,436::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915648::DEBUG::2013-11-20 04:05:22,437::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915648::DEBUG::2013-11-20 04:05:22,438::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915648::DEBUG::2013-11-20 04:05:22,439::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915648::DEBUG::2013-11-20 04:05:22,440::task::495::TaskManager.Task::(_debug) Task 5ebb360e-1a1a-410d-a751-77d0d55206d0: ref 0 aborting False
>Thread-15915648::INFO::2013-11-20 04:05:22,441::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:05:23,101::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:05:23,704::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.094179 s, 10.9 MB/s\n'; <rc> = 0
>Thread-15915650::INFO::2013-11-20 04:05:25,623::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915650::DEBUG::2013-11-20 04:05:25,624::task::495::TaskManager.Task::(_debug) Task 6bf8b82b-da5d-4dcd-bd15-a9bebd67f6bc: moving from state init -> state preparing
>Thread-15915650::DEBUG::2013-11-20 04:05:25,625::task::495::TaskManager.Task::(_debug) Task 6bf8b82b-da5d-4dcd-bd15-a9bebd67f6bc: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000664234161377', 'lastCheck': 1384920315.42646, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0168759822845', 'lastCheck': 1384920316.4261539, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.333788871765', 'lastCheck': 1384920323.3452711, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000448226928711', 'lastCheck': 1384920320.8217671, 'valid': True, 'code': 0}}
>Thread-15915650::DEBUG::2013-11-20 04:05:25,626::task::495::TaskManager.Task::(_debug) Task 6bf8b82b-da5d-4dcd-bd15-a9bebd67f6bc: moving from state preparing -> state finished
>Thread-15915650::DEBUG::2013-11-20 04:05:25,627::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915650::DEBUG::2013-11-20 04:05:25,627::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915650::DEBUG::2013-11-20 04:05:25,628::task::495::TaskManager.Task::(_debug) Task 6bf8b82b-da5d-4dcd-bd15-a9bebd67f6bc: ref 0 aborting False
>Thread-15915650::INFO::2013-11-20 04:05:25,629::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000664234161377', 'lastCheck': 1384920315.42646, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0168759822845', 'lastCheck': 1384920316.4261539, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.333788871765', 'lastCheck': 1384920323.3452711, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000448226928711', 'lastCheck': 1384920320.8217671, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:05:25,715::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:05:26,255::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0786291 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:05:28,271::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:05:28,798::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0828645 s, 12.4 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 04:05:30,708::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:05:30,711::task::495::TaskManager.Task::(_debug) Task db32b218-ee00-4bdd-9cf3-81340c637892: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:05:30,714::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d58c340a-62a8-492e-9ce0-0b700b055e79`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:05:30,717::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:05:30,719::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:05:30,720::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d58c340a-62a8-492e-9ce0-0b700b055e79`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:05:30,722::task::495::TaskManager.Task::(_debug) Task db32b218-ee00-4bdd-9cf3-81340c637892: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:05:30,723::task::495::TaskManager.Task::(_debug) Task db32b218-ee00-4bdd-9cf3-81340c637892: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:05:30,724::task::495::TaskManager.Task::(_debug) Task db32b218-ee00-4bdd-9cf3-81340c637892: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:05:30,725::task::495::TaskManager.Task::(_debug) Task db32b218-ee00-4bdd-9cf3-81340c637892: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:05:30,725::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:05:30,726::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:05:30,727::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:05:30,728::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:05:30,728::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:05:30,729::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:05:30,730::task::495::TaskManager.Task::(_debug) Task db32b218-ee00-4bdd-9cf3-81340c637892: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:05:30,731::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:05:30,812::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:05:31,343::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0777564 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15915654::DEBUG::2013-11-20 04:05:32,742::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915654::INFO::2013-11-20 04:05:32,744::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915654::DEBUG::2013-11-20 04:05:32,744::task::495::TaskManager.Task::(_debug) Task e83a71bd-20e1-4202-8b2d-a0da69277044: moving from state init -> state preparing
>Thread-15915654::DEBUG::2013-11-20 04:05:32,745::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915654::DEBUG::2013-11-20 04:05:32,746::task::495::TaskManager.Task::(_debug) Task e83a71bd-20e1-4202-8b2d-a0da69277044: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915654::DEBUG::2013-11-20 04:05:32,746::task::495::TaskManager.Task::(_debug) Task e83a71bd-20e1-4202-8b2d-a0da69277044: moving from state preparing -> state finished
>Thread-15915654::DEBUG::2013-11-20 04:05:32,747::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915654::DEBUG::2013-11-20 04:05:32,748::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915654::DEBUG::2013-11-20 04:05:32,748::task::495::TaskManager.Task::(_debug) Task e83a71bd-20e1-4202-8b2d-a0da69277044: ref 0 aborting False
>Thread-15915654::INFO::2013-11-20 04:05:32,749::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915655::DEBUG::2013-11-20 04:05:32,754::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915655::INFO::2013-11-20 04:05:32,755::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915655::DEBUG::2013-11-20 04:05:32,756::task::495::TaskManager.Task::(_debug) Task 74f83d79-ffe9-4a0b-b15f-922db5c24303: moving from state init -> state preparing
>Thread-15915655::DEBUG::2013-11-20 04:05:32,757::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`67742404-7e3a-43b4-bd0c-116be7aa89cb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915655::DEBUG::2013-11-20 04:05:32,758::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915655::DEBUG::2013-11-20 04:05:32,759::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915655::DEBUG::2013-11-20 04:05:32,760::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`67742404-7e3a-43b4-bd0c-116be7aa89cb`::Granted request
>Thread-15915655::DEBUG::2013-11-20 04:05:32,761::task::495::TaskManager.Task::(_debug) Task 74f83d79-ffe9-4a0b-b15f-922db5c24303: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915655::DEBUG::2013-11-20 04:05:32,762::task::495::TaskManager.Task::(_debug) Task 74f83d79-ffe9-4a0b-b15f-922db5c24303: ref 1 aborting False
>Thread-15915655::WARNING::2013-11-20 04:05:32,764::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915655::DEBUG::2013-11-20 04:05:32,766::task::495::TaskManager.Task::(_debug) Task 74f83d79-ffe9-4a0b-b15f-922db5c24303: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915655::DEBUG::2013-11-20 04:05:32,767::task::495::TaskManager.Task::(_debug) Task 74f83d79-ffe9-4a0b-b15f-922db5c24303: moving from state preparing -> state finished
>Thread-15915655::DEBUG::2013-11-20 04:05:32,767::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915655::DEBUG::2013-11-20 04:05:32,768::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915655::DEBUG::2013-11-20 04:05:32,769::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915655::DEBUG::2013-11-20 04:05:32,770::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915655::DEBUG::2013-11-20 04:05:32,770::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915655::DEBUG::2013-11-20 04:05:32,771::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915655::DEBUG::2013-11-20 04:05:32,771::task::495::TaskManager.Task::(_debug) Task 74f83d79-ffe9-4a0b-b15f-922db5c24303: ref 0 aborting False
>Thread-15915655::INFO::2013-11-20 04:05:32,772::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:05:33,358::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:05:33,874::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0712254 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:05:35,889::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:05:36,417::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0488588 s, 21.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:05:38,426::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:05:38,990::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0853719 s, 12.0 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 04:05:40,731::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:05:40,732::task::495::TaskManager.Task::(_debug) Task 04746b33-c605-4376-8707-c625144071f7: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:05:40,748::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f435d248-9844-4be8-b0bb-0354bac13235`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:05:40,749::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:05:40,750::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:05:40,751::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f435d248-9844-4be8-b0bb-0354bac13235`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:05:40,753::task::495::TaskManager.Task::(_debug) Task 04746b33-c605-4376-8707-c625144071f7: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:05:40,755::task::495::TaskManager.Task::(_debug) Task 04746b33-c605-4376-8707-c625144071f7: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:05:40,755::task::495::TaskManager.Task::(_debug) Task 04746b33-c605-4376-8707-c625144071f7: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:05:40,756::task::495::TaskManager.Task::(_debug) Task 04746b33-c605-4376-8707-c625144071f7: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:05:40,757::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:05:40,757::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:05:40,758::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:05:40,759::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:05:40,759::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:05:40,760::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:05:40,761::task::495::TaskManager.Task::(_debug) Task 04746b33-c605-4376-8707-c625144071f7: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:05:40,761::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:05:41,011::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915658::INFO::2013-11-20 04:05:41,275::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915658::DEBUG::2013-11-20 04:05:41,277::task::495::TaskManager.Task::(_debug) Task 905bb8f7-55ec-4402-9613-2744c97c4f99: moving from state init -> state preparing
>Thread-15915658::DEBUG::2013-11-20 04:05:41,279::task::495::TaskManager.Task::(_debug) Task 905bb8f7-55ec-4402-9613-2744c97c4f99: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000338077545166', 'lastCheck': 1384920335.7102871, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148296356201', 'lastCheck': 1384920336.713517, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00151586532593', 'lastCheck': 1384920333.609091, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0024299621582', 'lastCheck': 1384920341.2664161, 'valid': True, 'code': 0}}
>Thread-15915658::DEBUG::2013-11-20 04:05:41,280::task::495::TaskManager.Task::(_debug) Task 905bb8f7-55ec-4402-9613-2744c97c4f99: moving from state preparing -> state finished
>Thread-15915658::DEBUG::2013-11-20 04:05:41,281::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915658::DEBUG::2013-11-20 04:05:41,282::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915658::DEBUG::2013-11-20 04:05:41,283::task::495::TaskManager.Task::(_debug) Task 905bb8f7-55ec-4402-9613-2744c97c4f99: ref 0 aborting False
>Thread-15915658::INFO::2013-11-20 04:05:41,284::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000338077545166', 'lastCheck': 1384920335.7102871, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148296356201', 'lastCheck': 1384920336.713517, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00151586532593', 'lastCheck': 1384920333.609091, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0024299621582', 'lastCheck': 1384920341.2664161, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:05:41,586::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0737127 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15915660::DEBUG::2013-11-20 04:05:43,002::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915660::INFO::2013-11-20 04:05:43,004::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915660::DEBUG::2013-11-20 04:05:43,004::task::495::TaskManager.Task::(_debug) Task f148c273-4e25-4644-b892-4952ba1689a4: moving from state init -> state preparing
>Thread-15915660::DEBUG::2013-11-20 04:05:43,005::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915660::DEBUG::2013-11-20 04:05:43,005::task::495::TaskManager.Task::(_debug) Task f148c273-4e25-4644-b892-4952ba1689a4: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915660::DEBUG::2013-11-20 04:05:43,006::task::495::TaskManager.Task::(_debug) Task f148c273-4e25-4644-b892-4952ba1689a4: moving from state preparing -> state finished
>Thread-15915660::DEBUG::2013-11-20 04:05:43,007::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915660::DEBUG::2013-11-20 04:05:43,007::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915660::DEBUG::2013-11-20 04:05:43,008::task::495::TaskManager.Task::(_debug) Task f148c273-4e25-4644-b892-4952ba1689a4: ref 0 aborting False
>Thread-15915660::INFO::2013-11-20 04:05:43,009::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915661::DEBUG::2013-11-20 04:05:43,013::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915661::INFO::2013-11-20 04:05:43,014::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915661::DEBUG::2013-11-20 04:05:43,015::task::495::TaskManager.Task::(_debug) Task 9bd7bdbe-140b-41c4-9ed2-9f76ea31b8e9: moving from state init -> state preparing
>Thread-15915661::DEBUG::2013-11-20 04:05:43,016::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8ba1a294-cb81-47bd-bab7-aabf25745443`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915661::DEBUG::2013-11-20 04:05:43,016::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915661::DEBUG::2013-11-20 04:05:43,017::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915661::DEBUG::2013-11-20 04:05:43,018::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8ba1a294-cb81-47bd-bab7-aabf25745443`::Granted request
>Thread-15915661::DEBUG::2013-11-20 04:05:43,019::task::495::TaskManager.Task::(_debug) Task 9bd7bdbe-140b-41c4-9ed2-9f76ea31b8e9: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915661::DEBUG::2013-11-20 04:05:43,020::task::495::TaskManager.Task::(_debug) Task 9bd7bdbe-140b-41c4-9ed2-9f76ea31b8e9: ref 1 aborting False
>Thread-15915661::WARNING::2013-11-20 04:05:43,022::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915661::DEBUG::2013-11-20 04:05:43,024::task::495::TaskManager.Task::(_debug) Task 9bd7bdbe-140b-41c4-9ed2-9f76ea31b8e9: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915661::DEBUG::2013-11-20 04:05:43,024::task::495::TaskManager.Task::(_debug) Task 9bd7bdbe-140b-41c4-9ed2-9f76ea31b8e9: moving from state preparing -> state finished
>Thread-15915661::DEBUG::2013-11-20 04:05:43,025::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915661::DEBUG::2013-11-20 04:05:43,025::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915661::DEBUG::2013-11-20 04:05:43,026::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915661::DEBUG::2013-11-20 04:05:43,027::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915661::DEBUG::2013-11-20 04:05:43,027::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915661::DEBUG::2013-11-20 04:05:43,027::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915661::DEBUG::2013-11-20 04:05:43,028::task::495::TaskManager.Task::(_debug) Task 9bd7bdbe-140b-41c4-9ed2-9f76ea31b8e9: ref 0 aborting False
>Thread-15915661::INFO::2013-11-20 04:05:43,029::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:05:43,600::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 04:05:43,863::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:05:43,865::task::495::TaskManager.Task::(_debug) Task 1add489c-b9e2-4b36-97bf-6952ff327481: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:05:43,868::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`096d0469-781b-4483-a36c-17226605d5d2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:05:43,869::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:05:43,870::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:05:43,871::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`096d0469-781b-4483-a36c-17226605d5d2`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:05:43,872::task::495::TaskManager.Task::(_debug) Task 1add489c-b9e2-4b36-97bf-6952ff327481: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:05:43,872::task::495::TaskManager.Task::(_debug) Task 1add489c-b9e2-4b36-97bf-6952ff327481: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:05:43,873::task::495::TaskManager.Task::(_debug) Task 1add489c-b9e2-4b36-97bf-6952ff327481: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:05:43,874::task::495::TaskManager.Task::(_debug) Task 1add489c-b9e2-4b36-97bf-6952ff327481: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:05:43,875::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:05:43,875::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:05:43,876::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:05:43,877::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:05:43,877::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:05:43,878::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:05:43,879::task::495::TaskManager.Task::(_debug) Task 1add489c-b9e2-4b36-97bf-6952ff327481: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:05:43,879::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:05:44,200::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.081085 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15915085::INFO::2013-11-20 04:05:45,853::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=089b08fc-6544-4235-a061-fb55b09d2715 volUUID=66b599fc-b7b5-4934-a5d5-ef7283460029)
>Thread-15915085::DEBUG::2013-11-20 04:05:45,857::task::495::TaskManager.Task::(_debug) Task 78a7993d-b329-40f7-94cb-0c0d15900402: moving from state init -> state preparing
>Thread-15915085::DEBUG::2013-11-20 04:05:45,859::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e13cfed0-3a27-42b1-b52c-391968e95d09`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915085::DEBUG::2013-11-20 04:05:45,860::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915085::DEBUG::2013-11-20 04:05:45,861::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915085::DEBUG::2013-11-20 04:05:45,861::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e13cfed0-3a27-42b1-b52c-391968e95d09`::Granted request
>Thread-15915085::DEBUG::2013-11-20 04:05:45,863::task::495::TaskManager.Task::(_debug) Task 78a7993d-b329-40f7-94cb-0c0d15900402: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915085::DEBUG::2013-11-20 04:05:45,863::task::495::TaskManager.Task::(_debug) Task 78a7993d-b329-40f7-94cb-0c0d15900402: ref 1 aborting False
>Thread-15915085::DEBUG::2013-11-20 04:05:45,864::task::495::TaskManager.Task::(_debug) Task 78a7993d-b329-40f7-94cb-0c0d15900402: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15915085::DEBUG::2013-11-20 04:05:45,865::task::495::TaskManager.Task::(_debug) Task 78a7993d-b329-40f7-94cb-0c0d15900402: moving from state preparing -> state finished
>Thread-15915085::DEBUG::2013-11-20 04:05:45,865::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915085::DEBUG::2013-11-20 04:05:45,866::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915085::DEBUG::2013-11-20 04:05:45,867::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915085::DEBUG::2013-11-20 04:05:45,867::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915085::DEBUG::2013-11-20 04:05:45,868::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915085::DEBUG::2013-11-20 04:05:45,869::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915085::DEBUG::2013-11-20 04:05:45,869::task::495::TaskManager.Task::(_debug) Task 78a7993d-b329-40f7-94cb-0c0d15900402: ref 0 aborting False
>Thread-15915085::INFO::2013-11-20 04:05:45,870::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Dummy-15912651::DEBUG::2013-11-20 04:05:46,210::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:05:46,843::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.101701 s, 10.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:05:48,866::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:05:49,453::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.100658 s, 10.2 MB/s\n'; <rc> = 0
>Thread-15914453::INFO::2013-11-20 04:05:50,773::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:05:50,774::task::495::TaskManager.Task::(_debug) Task 7ff36b09-7b10-4529-a4c9-eb8d51e47385: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:05:50,788::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0eb1c2c4-9b83-4265-8658-ada3674d884d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:05:50,789::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:05:50,790::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:05:50,791::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0eb1c2c4-9b83-4265-8658-ada3674d884d`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:05:50,792::task::495::TaskManager.Task::(_debug) Task 7ff36b09-7b10-4529-a4c9-eb8d51e47385: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:05:50,793::task::495::TaskManager.Task::(_debug) Task 7ff36b09-7b10-4529-a4c9-eb8d51e47385: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:05:50,794::task::495::TaskManager.Task::(_debug) Task 7ff36b09-7b10-4529-a4c9-eb8d51e47385: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:05:50,795::task::495::TaskManager.Task::(_debug) Task 7ff36b09-7b10-4529-a4c9-eb8d51e47385: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:05:50,795::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:05:50,796::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:05:50,797::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:05:50,797::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:05:50,798::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:05:50,799::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:05:50,800::task::495::TaskManager.Task::(_debug) Task 7ff36b09-7b10-4529-a4c9-eb8d51e47385: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:05:50,800::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:05:51,467::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:05:51,990::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0610722 s, 16.8 MB/s\n'; <rc> = 0
>Thread-15915665::DEBUG::2013-11-20 04:05:53,443::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915665::INFO::2013-11-20 04:05:53,444::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915665::DEBUG::2013-11-20 04:05:53,445::task::495::TaskManager.Task::(_debug) Task a2c60bad-2dc0-4051-9996-e2c57b729b00: moving from state init -> state preparing
>Thread-15915665::DEBUG::2013-11-20 04:05:53,447::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915665::DEBUG::2013-11-20 04:05:53,448::task::495::TaskManager.Task::(_debug) Task a2c60bad-2dc0-4051-9996-e2c57b729b00: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915665::DEBUG::2013-11-20 04:05:53,448::task::495::TaskManager.Task::(_debug) Task a2c60bad-2dc0-4051-9996-e2c57b729b00: moving from state preparing -> state finished
>Thread-15915665::DEBUG::2013-11-20 04:05:53,450::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915665::DEBUG::2013-11-20 04:05:53,450::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915665::DEBUG::2013-11-20 04:05:53,451::task::495::TaskManager.Task::(_debug) Task a2c60bad-2dc0-4051-9996-e2c57b729b00: ref 0 aborting False
>Thread-15915665::INFO::2013-11-20 04:05:53,452::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915666::DEBUG::2013-11-20 04:05:53,458::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915666::INFO::2013-11-20 04:05:53,459::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915666::DEBUG::2013-11-20 04:05:53,460::task::495::TaskManager.Task::(_debug) Task db44caa4-13ee-465f-8181-bb3a5b03bcd1: moving from state init -> state preparing
>Thread-15915666::DEBUG::2013-11-20 04:05:53,461::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bbe863d5-f8b0-47be-968d-1dd9dd6c68b5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915666::DEBUG::2013-11-20 04:05:53,462::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915666::DEBUG::2013-11-20 04:05:53,463::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915666::DEBUG::2013-11-20 04:05:53,464::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bbe863d5-f8b0-47be-968d-1dd9dd6c68b5`::Granted request
>Thread-15915666::DEBUG::2013-11-20 04:05:53,465::task::495::TaskManager.Task::(_debug) Task db44caa4-13ee-465f-8181-bb3a5b03bcd1: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915666::DEBUG::2013-11-20 04:05:53,466::task::495::TaskManager.Task::(_debug) Task db44caa4-13ee-465f-8181-bb3a5b03bcd1: ref 1 aborting False
>Thread-15915666::WARNING::2013-11-20 04:05:53,469::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915666::DEBUG::2013-11-20 04:05:53,471::task::495::TaskManager.Task::(_debug) Task db44caa4-13ee-465f-8181-bb3a5b03bcd1: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915666::DEBUG::2013-11-20 04:05:53,472::task::495::TaskManager.Task::(_debug) Task db44caa4-13ee-465f-8181-bb3a5b03bcd1: moving from state preparing -> state finished
>Thread-15915666::DEBUG::2013-11-20 04:05:53,473::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915666::DEBUG::2013-11-20 04:05:53,474::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915666::DEBUG::2013-11-20 04:05:53,475::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915666::DEBUG::2013-11-20 04:05:53,475::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915666::DEBUG::2013-11-20 04:05:53,476::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915666::DEBUG::2013-11-20 04:05:53,477::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915666::DEBUG::2013-11-20 04:05:53,477::task::495::TaskManager.Task::(_debug) Task db44caa4-13ee-465f-8181-bb3a5b03bcd1: ref 0 aborting False
>Thread-15915666::INFO::2013-11-20 04:05:53,478::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:05:54,000::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:05:54,584::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0758424 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:05:56,599::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 04:05:56,836::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:05:56,837::task::495::TaskManager.Task::(_debug) Task 43d6c713-93b0-4c86-b754-48ccec1101c5: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:05:56,839::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1af66d3a-765f-4a4a-abef-59a63c38c95f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:05:56,840::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:05:56,841::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:05:56,842::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1af66d3a-765f-4a4a-abef-59a63c38c95f`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:05:56,843::task::495::TaskManager.Task::(_debug) Task 43d6c713-93b0-4c86-b754-48ccec1101c5: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:05:56,843::task::495::TaskManager.Task::(_debug) Task 43d6c713-93b0-4c86-b754-48ccec1101c5: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:05:56,844::task::495::TaskManager.Task::(_debug) Task 43d6c713-93b0-4c86-b754-48ccec1101c5: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:05:56,845::task::495::TaskManager.Task::(_debug) Task 43d6c713-93b0-4c86-b754-48ccec1101c5: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:05:56,845::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:05:56,846::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:05:56,847::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:05:56,847::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:05:56,848::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:05:56,848::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:05:56,849::task::495::TaskManager.Task::(_debug) Task 43d6c713-93b0-4c86-b754-48ccec1101c5: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:05:56,850::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:05:57,113::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0517064 s, 19.8 MB/s\n'; <rc> = 0
>Thread-15915668::INFO::2013-11-20 04:05:57,293::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915668::DEBUG::2013-11-20 04:05:57,294::task::495::TaskManager.Task::(_debug) Task 292fa32d-25bd-4f34-87b1-90c847fb6879: moving from state init -> state preparing
>Thread-15915668::DEBUG::2013-11-20 04:05:57,295::task::495::TaskManager.Task::(_debug) Task 292fa32d-25bd-4f34-87b1-90c847fb6879: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000321865081787', 'lastCheck': 1384920355.872612, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00151610374451', 'lastCheck': 1384920357.252507, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00299501419067', 'lastCheck': 1384920354.279458, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000431060791016', 'lastCheck': 1384920351.4666431, 'valid': True, 'code': 0}}
>Thread-15915668::DEBUG::2013-11-20 04:05:57,296::task::495::TaskManager.Task::(_debug) Task 292fa32d-25bd-4f34-87b1-90c847fb6879: moving from state preparing -> state finished
>Thread-15915668::DEBUG::2013-11-20 04:05:57,297::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915668::DEBUG::2013-11-20 04:05:57,298::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915668::DEBUG::2013-11-20 04:05:57,298::task::495::TaskManager.Task::(_debug) Task 292fa32d-25bd-4f34-87b1-90c847fb6879: ref 0 aborting False
>Thread-15915668::INFO::2013-11-20 04:05:57,299::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000321865081787', 'lastCheck': 1384920355.872612, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00151610374451', 'lastCheck': 1384920357.252507, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00299501419067', 'lastCheck': 1384920354.279458, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000431060791016', 'lastCheck': 1384920351.4666431, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:05:59,129::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 04:05:59,413::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:05:59,431::task::495::TaskManager.Task::(_debug) Task be489bde-b7e8-44a4-be30-e0d8e48d9d3c: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:05:59,457::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`557debae-c479-4d95-8547-0509caa166a8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:05:59,459::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:05:59,462::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:05:59,463::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`557debae-c479-4d95-8547-0509caa166a8`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:05:59,466::task::495::TaskManager.Task::(_debug) Task be489bde-b7e8-44a4-be30-e0d8e48d9d3c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 04:05:59,473::task::495::TaskManager.Task::(_debug) Task be489bde-b7e8-44a4-be30-e0d8e48d9d3c: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:05:59,474::task::495::TaskManager.Task::(_debug) Task be489bde-b7e8-44a4-be30-e0d8e48d9d3c: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 04:05:59,475::task::495::TaskManager.Task::(_debug) Task be489bde-b7e8-44a4-be30-e0d8e48d9d3c: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:05:59,477::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:05:59,478::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:05:59,480::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 04:05:59,482::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:05:59,483::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:05:59,484::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:05:59,485::task::495::TaskManager.Task::(_debug) Task be489bde-b7e8-44a4-be30-e0d8e48d9d3c: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 04:05:59,486::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 04:05:59,749::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0738368 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 04:06:00,828::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:06:00,831::task::495::TaskManager.Task::(_debug) Task 7c56712b-ff6d-4e7c-b7a3-a05319bc1294: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:06:00,833::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c30c67d7-4449-4663-a65f-6751acf11c29`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:06:00,836::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:06:00,837::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:06:00,838::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c30c67d7-4449-4663-a65f-6751acf11c29`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:06:00,839::task::495::TaskManager.Task::(_debug) Task 7c56712b-ff6d-4e7c-b7a3-a05319bc1294: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:06:00,840::task::495::TaskManager.Task::(_debug) Task 7c56712b-ff6d-4e7c-b7a3-a05319bc1294: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:06:00,841::task::495::TaskManager.Task::(_debug) Task 7c56712b-ff6d-4e7c-b7a3-a05319bc1294: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 04:06:00,841::task::495::TaskManager.Task::(_debug) Task 7c56712b-ff6d-4e7c-b7a3-a05319bc1294: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:06:00,842::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:06:00,843::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:06:00,844::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:06:00,844::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:06:00,845::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:06:00,846::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:06:00,846::task::495::TaskManager.Task::(_debug) Task 7c56712b-ff6d-4e7c-b7a3-a05319bc1294: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:06:00,847::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:06:00,848::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:06:00,848::task::495::TaskManager.Task::(_debug) Task 58ffc698-71e3-4c6a-a5f3-ec9f5e1b648c: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:06:00,850::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`00706a47-3210-4189-aaff-dd87892bb376`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:06:00,850::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:06:00,851::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:06:00,852::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`00706a47-3210-4189-aaff-dd87892bb376`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:06:00,861::task::495::TaskManager.Task::(_debug) Task 58ffc698-71e3-4c6a-a5f3-ec9f5e1b648c: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:06:00,862::task::495::TaskManager.Task::(_debug) Task 58ffc698-71e3-4c6a-a5f3-ec9f5e1b648c: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:06:00,863::task::495::TaskManager.Task::(_debug) Task 58ffc698-71e3-4c6a-a5f3-ec9f5e1b648c: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:06:00,864::task::495::TaskManager.Task::(_debug) Task 58ffc698-71e3-4c6a-a5f3-ec9f5e1b648c: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:06:00,864::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:06:00,865::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:06:00,866::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:06:00,867::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:06:00,870::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:06:00,870::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:06:00,871::task::495::TaskManager.Task::(_debug) Task 58ffc698-71e3-4c6a-a5f3-ec9f5e1b648c: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:06:00,871::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:06:00,872::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:06:00,873::task::495::TaskManager.Task::(_debug) Task d7f26430-4e48-46bc-bebc-504a6bef582e: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:06:00,874::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a3718cb1-4c12-402d-8d61-1669e70f7eda`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:06:00,874::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:06:00,875::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:06:00,875::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a3718cb1-4c12-402d-8d61-1669e70f7eda`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:06:00,876::task::495::TaskManager.Task::(_debug) Task d7f26430-4e48-46bc-bebc-504a6bef582e: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:06:00,877::task::495::TaskManager.Task::(_debug) Task d7f26430-4e48-46bc-bebc-504a6bef582e: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:06:00,877::task::495::TaskManager.Task::(_debug) Task d7f26430-4e48-46bc-bebc-504a6bef582e: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:06:00,878::task::495::TaskManager.Task::(_debug) Task d7f26430-4e48-46bc-bebc-504a6bef582e: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:06:00,878::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:06:00,878::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:06:00,879::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:06:00,879::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:06:00,879::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:06:00,880::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:06:00,880::task::495::TaskManager.Task::(_debug) Task d7f26430-4e48-46bc-bebc-504a6bef582e: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:06:00,880::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:06:00,881::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:06:00,881::task::495::TaskManager.Task::(_debug) Task a936c0f3-52cb-45e2-9bae-a984588825bb: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:06:00,882::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`4df7b51f-37bb-43dc-8897-f8b70ac34187`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:06:00,882::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:06:00,882::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:06:00,883::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`4df7b51f-37bb-43dc-8897-f8b70ac34187`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:06:00,883::task::495::TaskManager.Task::(_debug) Task a936c0f3-52cb-45e2-9bae-a984588825bb: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:06:00,884::task::495::TaskManager.Task::(_debug) Task a936c0f3-52cb-45e2-9bae-a984588825bb: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:06:00,884::task::495::TaskManager.Task::(_debug) Task a936c0f3-52cb-45e2-9bae-a984588825bb: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:06:00,884::task::495::TaskManager.Task::(_debug) Task a936c0f3-52cb-45e2-9bae-a984588825bb: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:06:00,885::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:06:00,885::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:06:00,885::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:06:00,886::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:06:00,886::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:06:00,886::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:06:00,887::task::495::TaskManager.Task::(_debug) Task a936c0f3-52cb-45e2-9bae-a984588825bb: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:06:00,888::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:06:01,765::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:06:02,403::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.094305 s, 10.9 MB/s\n'; <rc> = 0
>Thread-15915671::DEBUG::2013-11-20 04:06:03,817::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915671::INFO::2013-11-20 04:06:03,818::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915671::DEBUG::2013-11-20 04:06:03,819::task::495::TaskManager.Task::(_debug) Task 116829e7-7c36-4d40-873e-a6446071d909: moving from state init -> state preparing
>Thread-15915671::DEBUG::2013-11-20 04:06:03,820::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915671::DEBUG::2013-11-20 04:06:03,820::task::495::TaskManager.Task::(_debug) Task 116829e7-7c36-4d40-873e-a6446071d909: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915671::DEBUG::2013-11-20 04:06:03,821::task::495::TaskManager.Task::(_debug) Task 116829e7-7c36-4d40-873e-a6446071d909: moving from state preparing -> state finished
>Thread-15915671::DEBUG::2013-11-20 04:06:03,822::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915671::DEBUG::2013-11-20 04:06:03,822::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915671::DEBUG::2013-11-20 04:06:03,823::task::495::TaskManager.Task::(_debug) Task 116829e7-7c36-4d40-873e-a6446071d909: ref 0 aborting False
>Thread-15915671::INFO::2013-11-20 04:06:03,823::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915672::DEBUG::2013-11-20 04:06:03,829::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915672::INFO::2013-11-20 04:06:03,830::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915672::DEBUG::2013-11-20 04:06:03,830::task::495::TaskManager.Task::(_debug) Task e97d5516-5ff2-4038-bf10-2cc9d3d9c092: moving from state init -> state preparing
>Thread-15915672::DEBUG::2013-11-20 04:06:03,832::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2015c4a1-fa29-492a-a584-d24cf8bc1a9d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915672::DEBUG::2013-11-20 04:06:03,833::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915672::DEBUG::2013-11-20 04:06:03,833::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915672::DEBUG::2013-11-20 04:06:03,834::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2015c4a1-fa29-492a-a584-d24cf8bc1a9d`::Granted request
>Thread-15915672::DEBUG::2013-11-20 04:06:03,835::task::495::TaskManager.Task::(_debug) Task e97d5516-5ff2-4038-bf10-2cc9d3d9c092: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915672::DEBUG::2013-11-20 04:06:03,836::task::495::TaskManager.Task::(_debug) Task e97d5516-5ff2-4038-bf10-2cc9d3d9c092: ref 1 aborting False
>Thread-15915672::WARNING::2013-11-20 04:06:03,838::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915672::DEBUG::2013-11-20 04:06:03,839::task::495::TaskManager.Task::(_debug) Task e97d5516-5ff2-4038-bf10-2cc9d3d9c092: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915672::DEBUG::2013-11-20 04:06:03,840::task::495::TaskManager.Task::(_debug) Task e97d5516-5ff2-4038-bf10-2cc9d3d9c092: moving from state preparing -> state finished
>Thread-15915672::DEBUG::2013-11-20 04:06:03,840::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915672::DEBUG::2013-11-20 04:06:03,841::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915672::DEBUG::2013-11-20 04:06:03,842::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915672::DEBUG::2013-11-20 04:06:03,842::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915672::DEBUG::2013-11-20 04:06:03,843::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915672::DEBUG::2013-11-20 04:06:03,843::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915672::DEBUG::2013-11-20 04:06:03,844::task::495::TaskManager.Task::(_debug) Task e97d5516-5ff2-4038-bf10-2cc9d3d9c092: ref 0 aborting False
>Thread-15915672::INFO::2013-11-20 04:06:03,845::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:06:04,453::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:06:05,034::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0769162 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:06:07,088::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:06:07,704::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.070138 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:06:09,723::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:06:10,244::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0741391 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:06:12,260::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:06:12,802::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0705795 s, 14.5 MB/s\n'; <rc> = 0
>Thread-15915676::INFO::2013-11-20 04:06:12,940::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915676::DEBUG::2013-11-20 04:06:12,941::task::495::TaskManager.Task::(_debug) Task 1b568fef-2a42-4ccd-9213-b31afc2af2ad: moving from state init -> state preparing
>Thread-15915676::DEBUG::2013-11-20 04:06:12,942::task::495::TaskManager.Task::(_debug) Task 1b568fef-2a42-4ccd-9213-b31afc2af2ad: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000319004058838', 'lastCheck': 1384920366.0351989, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00275492668152', 'lastCheck': 1384920367.4036081, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00218415260315', 'lastCheck': 1384920364.4126959, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00032114982605', 'lastCheck': 1384920371.676698, 'valid': True, 'code': 0}}
>Thread-15915676::DEBUG::2013-11-20 04:06:12,943::task::495::TaskManager.Task::(_debug) Task 1b568fef-2a42-4ccd-9213-b31afc2af2ad: moving from state preparing -> state finished
>Thread-15915676::DEBUG::2013-11-20 04:06:12,944::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915676::DEBUG::2013-11-20 04:06:12,944::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915676::DEBUG::2013-11-20 04:06:12,945::task::495::TaskManager.Task::(_debug) Task 1b568fef-2a42-4ccd-9213-b31afc2af2ad: ref 0 aborting False
>Thread-15915676::INFO::2013-11-20 04:06:12,945::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000319004058838', 'lastCheck': 1384920366.0351989, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00275492668152', 'lastCheck': 1384920367.4036081, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00218415260315', 'lastCheck': 1384920364.4126959, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00032114982605', 'lastCheck': 1384920371.676698, 'valid': True, 'code': 0}}
>Thread-15915678::DEBUG::2013-11-20 04:06:14,125::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915678::INFO::2013-11-20 04:06:14,126::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915678::DEBUG::2013-11-20 04:06:14,127::task::495::TaskManager.Task::(_debug) Task 612e9234-6908-4433-962f-65f5a597197e: moving from state init -> state preparing
>Thread-15915678::DEBUG::2013-11-20 04:06:14,128::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915678::DEBUG::2013-11-20 04:06:14,129::task::495::TaskManager.Task::(_debug) Task 612e9234-6908-4433-962f-65f5a597197e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915678::DEBUG::2013-11-20 04:06:14,129::task::495::TaskManager.Task::(_debug) Task 612e9234-6908-4433-962f-65f5a597197e: moving from state preparing -> state finished
>Thread-15915678::DEBUG::2013-11-20 04:06:14,130::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915678::DEBUG::2013-11-20 04:06:14,131::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915678::DEBUG::2013-11-20 04:06:14,131::task::495::TaskManager.Task::(_debug) Task 612e9234-6908-4433-962f-65f5a597197e: ref 0 aborting False
>Thread-15915678::INFO::2013-11-20 04:06:14,132::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915679::DEBUG::2013-11-20 04:06:14,137::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915679::INFO::2013-11-20 04:06:14,138::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915679::DEBUG::2013-11-20 04:06:14,139::task::495::TaskManager.Task::(_debug) Task 241388f6-abac-42b2-90f0-600082f03594: moving from state init -> state preparing
>Thread-15915679::DEBUG::2013-11-20 04:06:14,140::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`cc22e2ca-6649-4d51-aca9-af838e5b2e89`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915679::DEBUG::2013-11-20 04:06:14,141::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915679::DEBUG::2013-11-20 04:06:14,142::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915679::DEBUG::2013-11-20 04:06:14,142::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`cc22e2ca-6649-4d51-aca9-af838e5b2e89`::Granted request
>Thread-15915679::DEBUG::2013-11-20 04:06:14,144::task::495::TaskManager.Task::(_debug) Task 241388f6-abac-42b2-90f0-600082f03594: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915679::DEBUG::2013-11-20 04:06:14,145::task::495::TaskManager.Task::(_debug) Task 241388f6-abac-42b2-90f0-600082f03594: ref 1 aborting False
>Thread-15915679::WARNING::2013-11-20 04:06:14,147::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915679::DEBUG::2013-11-20 04:06:14,148::task::495::TaskManager.Task::(_debug) Task 241388f6-abac-42b2-90f0-600082f03594: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915679::DEBUG::2013-11-20 04:06:14,149::task::495::TaskManager.Task::(_debug) Task 241388f6-abac-42b2-90f0-600082f03594: moving from state preparing -> state finished
>Thread-15915679::DEBUG::2013-11-20 04:06:14,150::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915679::DEBUG::2013-11-20 04:06:14,150::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915679::DEBUG::2013-11-20 04:06:14,151::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915679::DEBUG::2013-11-20 04:06:14,151::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915679::DEBUG::2013-11-20 04:06:14,152::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915679::DEBUG::2013-11-20 04:06:14,152::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915679::DEBUG::2013-11-20 04:06:14,153::task::495::TaskManager.Task::(_debug) Task 241388f6-abac-42b2-90f0-600082f03594: ref 0 aborting False
>Thread-15915679::INFO::2013-11-20 04:06:14,154::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:06:14,820::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:06:15,337::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0737676 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15913639::INFO::2013-11-20 04:06:16,016::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:06:16,018::task::495::TaskManager.Task::(_debug) Task 17593b34-9888-4c90-9c8e-02a8e1c5fa1b: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:06:16,032::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`25b45602-6460-4c9c-8747-88c2c9b91941`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:06:16,033::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:06:16,034::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:06:16,036::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`25b45602-6460-4c9c-8747-88c2c9b91941`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:06:16,039::task::495::TaskManager.Task::(_debug) Task 17593b34-9888-4c90-9c8e-02a8e1c5fa1b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:06:16,040::task::495::TaskManager.Task::(_debug) Task 17593b34-9888-4c90-9c8e-02a8e1c5fa1b: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:06:16,041::task::495::TaskManager.Task::(_debug) Task 17593b34-9888-4c90-9c8e-02a8e1c5fa1b: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:06:16,042::task::495::TaskManager.Task::(_debug) Task 17593b34-9888-4c90-9c8e-02a8e1c5fa1b: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:06:16,042::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:06:16,043::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:06:16,044::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:06:16,047::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:06:16,049::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:06:16,050::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:06:16,052::task::495::TaskManager.Task::(_debug) Task 17593b34-9888-4c90-9c8e-02a8e1c5fa1b: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:06:16,053::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:06:17,350::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:06:17,923::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0836158 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:06:19,941::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913518::INFO::2013-11-20 04:06:20,227::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:06:20,228::task::495::TaskManager.Task::(_debug) Task 64e41849-41a9-4396-bdbf-f18d6edfea47: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:06:20,231::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0459dca1-9c2d-41c9-879b-872148f059d8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:06:20,232::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:06:20,233::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:06:20,234::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0459dca1-9c2d-41c9-879b-872148f059d8`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:06:20,235::task::495::TaskManager.Task::(_debug) Task 64e41849-41a9-4396-bdbf-f18d6edfea47: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:06:20,236::task::495::TaskManager.Task::(_debug) Task 64e41849-41a9-4396-bdbf-f18d6edfea47: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:06:20,237::task::495::TaskManager.Task::(_debug) Task 64e41849-41a9-4396-bdbf-f18d6edfea47: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:06:20,238::task::495::TaskManager.Task::(_debug) Task 64e41849-41a9-4396-bdbf-f18d6edfea47: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:06:20,239::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:06:20,239::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:06:20,240::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:06:20,241::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:06:20,241::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:06:20,242::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:06:20,243::task::495::TaskManager.Task::(_debug) Task 64e41849-41a9-4396-bdbf-f18d6edfea47: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:06:20,243::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:06:20,244::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:06:20,245::task::495::TaskManager.Task::(_debug) Task 2e10da32-9716-4142-a73e-e401cdc7fc89: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:06:20,246::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ba95b90c-b0a3-4b9a-a83c-db9744c72cdc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:06:20,247::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:06:20,248::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:06:20,249::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ba95b90c-b0a3-4b9a-a83c-db9744c72cdc`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:06:20,250::task::495::TaskManager.Task::(_debug) Task 2e10da32-9716-4142-a73e-e401cdc7fc89: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:06:20,251::task::495::TaskManager.Task::(_debug) Task 2e10da32-9716-4142-a73e-e401cdc7fc89: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:06:20,252::task::495::TaskManager.Task::(_debug) Task 2e10da32-9716-4142-a73e-e401cdc7fc89: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:06:20,252::task::495::TaskManager.Task::(_debug) Task 2e10da32-9716-4142-a73e-e401cdc7fc89: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:06:20,253::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:06:20,254::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:06:20,255::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:06:20,255::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:06:20,256::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:06:20,257::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:06:20,258::task::495::TaskManager.Task::(_debug) Task 2e10da32-9716-4142-a73e-e401cdc7fc89: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:06:20,258::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:06:20,526::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0681608 s, 15.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:06:22,540::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:06:23,173::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0981178 s, 10.4 MB/s\n'; <rc> = 0
>Thread-15915683::DEBUG::2013-11-20 04:06:24,516::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915683::INFO::2013-11-20 04:06:24,517::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915683::DEBUG::2013-11-20 04:06:24,518::task::495::TaskManager.Task::(_debug) Task 217f575c-c5d5-43ef-826d-8065534178a7: moving from state init -> state preparing
>Thread-15915683::DEBUG::2013-11-20 04:06:24,519::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915683::DEBUG::2013-11-20 04:06:24,520::task::495::TaskManager.Task::(_debug) Task 217f575c-c5d5-43ef-826d-8065534178a7: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915683::DEBUG::2013-11-20 04:06:24,520::task::495::TaskManager.Task::(_debug) Task 217f575c-c5d5-43ef-826d-8065534178a7: moving from state preparing -> state finished
>Thread-15915683::DEBUG::2013-11-20 04:06:24,521::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915683::DEBUG::2013-11-20 04:06:24,521::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915683::DEBUG::2013-11-20 04:06:24,522::task::495::TaskManager.Task::(_debug) Task 217f575c-c5d5-43ef-826d-8065534178a7: ref 0 aborting False
>Thread-15915683::INFO::2013-11-20 04:06:24,523::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915684::DEBUG::2013-11-20 04:06:24,528::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915684::INFO::2013-11-20 04:06:24,529::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915684::DEBUG::2013-11-20 04:06:24,530::task::495::TaskManager.Task::(_debug) Task c37bb182-7e7d-4460-b224-018f52aa3305: moving from state init -> state preparing
>Thread-15915684::DEBUG::2013-11-20 04:06:24,532::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5e780b6c-ddb8-489d-9185-d7bf2e80a565`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915684::DEBUG::2013-11-20 04:06:24,533::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915684::DEBUG::2013-11-20 04:06:24,535::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915684::DEBUG::2013-11-20 04:06:24,536::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5e780b6c-ddb8-489d-9185-d7bf2e80a565`::Granted request
>Thread-15915684::DEBUG::2013-11-20 04:06:24,537::task::495::TaskManager.Task::(_debug) Task c37bb182-7e7d-4460-b224-018f52aa3305: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915684::DEBUG::2013-11-20 04:06:24,538::task::495::TaskManager.Task::(_debug) Task c37bb182-7e7d-4460-b224-018f52aa3305: ref 1 aborting False
>Thread-15915684::WARNING::2013-11-20 04:06:24,541::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915684::DEBUG::2013-11-20 04:06:24,543::task::495::TaskManager.Task::(_debug) Task c37bb182-7e7d-4460-b224-018f52aa3305: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915684::DEBUG::2013-11-20 04:06:24,544::task::495::TaskManager.Task::(_debug) Task c37bb182-7e7d-4460-b224-018f52aa3305: moving from state preparing -> state finished
>Thread-15915684::DEBUG::2013-11-20 04:06:24,545::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915684::DEBUG::2013-11-20 04:06:24,546::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915684::DEBUG::2013-11-20 04:06:24,547::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915684::DEBUG::2013-11-20 04:06:24,548::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915684::DEBUG::2013-11-20 04:06:24,548::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915684::DEBUG::2013-11-20 04:06:24,549::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915684::DEBUG::2013-11-20 04:06:24,550::task::495::TaskManager.Task::(_debug) Task c37bb182-7e7d-4460-b224-018f52aa3305: ref 0 aborting False
>Thread-15915684::INFO::2013-11-20 04:06:24,551::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:06:25,234::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:06:25,812::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0761417 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:06:27,828::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:06:28,363::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0769939 s, 13.3 MB/s\n'; <rc> = 0
>Thread-15915686::INFO::2013-11-20 04:06:28,573::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915686::DEBUG::2013-11-20 04:06:28,574::task::495::TaskManager.Task::(_debug) Task 8c2533bb-f5b2-47ef-99b8-e5458f103f3c: moving from state init -> state preparing
>Thread-15915686::DEBUG::2013-11-20 04:06:28,575::task::495::TaskManager.Task::(_debug) Task 8c2533bb-f5b2-47ef-99b8-e5458f103f3c: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000811100006104', 'lastCheck': 1384920386.1887159, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00147104263306', 'lastCheck': 1384920387.7764599, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00131106376648', 'lastCheck': 1384920384.7710919, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00039005279541', 'lastCheck': 1384920381.687479, 'valid': True, 'code': 0}}
>Thread-15915686::DEBUG::2013-11-20 04:06:28,576::task::495::TaskManager.Task::(_debug) Task 8c2533bb-f5b2-47ef-99b8-e5458f103f3c: moving from state preparing -> state finished
>Thread-15915686::DEBUG::2013-11-20 04:06:28,577::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915686::DEBUG::2013-11-20 04:06:28,578::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915686::DEBUG::2013-11-20 04:06:28,578::task::495::TaskManager.Task::(_debug) Task 8c2533bb-f5b2-47ef-99b8-e5458f103f3c: ref 0 aborting False
>Thread-15915686::INFO::2013-11-20 04:06:28,579::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000811100006104', 'lastCheck': 1384920386.1887159, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00147104263306', 'lastCheck': 1384920387.7764599, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00131106376648', 'lastCheck': 1384920384.7710919, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00039005279541', 'lastCheck': 1384920381.687479, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:06:30,378::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:06:30,954::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0790864 s, 12.9 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 04:06:31,075::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:06:31,077::task::495::TaskManager.Task::(_debug) Task 47f13fce-8fef-4577-ae49-c2321e7f1f95: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:06:31,087::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`39924c09-164f-4cdb-99ad-8627fde6edc5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:06:31,087::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:06:31,090::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:06:31,091::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`39924c09-164f-4cdb-99ad-8627fde6edc5`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:06:31,093::task::495::TaskManager.Task::(_debug) Task 47f13fce-8fef-4577-ae49-c2321e7f1f95: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:06:31,093::task::495::TaskManager.Task::(_debug) Task 47f13fce-8fef-4577-ae49-c2321e7f1f95: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:06:31,094::task::495::TaskManager.Task::(_debug) Task 47f13fce-8fef-4577-ae49-c2321e7f1f95: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:06:31,095::task::495::TaskManager.Task::(_debug) Task 47f13fce-8fef-4577-ae49-c2321e7f1f95: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:06:31,096::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:06:31,096::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:06:31,097::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:06:31,098::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:06:31,099::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:06:31,099::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:06:31,100::task::495::TaskManager.Task::(_debug) Task 47f13fce-8fef-4577-ae49-c2321e7f1f95: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:06:31,101::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:06:32,968::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:06:33,540::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0834309 s, 12.3 MB/s\n'; <rc> = 0
>Thread-15915689::DEBUG::2013-11-20 04:06:34,891::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915689::INFO::2013-11-20 04:06:34,893::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915689::DEBUG::2013-11-20 04:06:34,894::task::495::TaskManager.Task::(_debug) Task 44179b79-2ae5-417c-9b74-27ea1a4c0be0: moving from state init -> state preparing
>Thread-15915689::DEBUG::2013-11-20 04:06:34,894::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915689::DEBUG::2013-11-20 04:06:34,895::task::495::TaskManager.Task::(_debug) Task 44179b79-2ae5-417c-9b74-27ea1a4c0be0: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915689::DEBUG::2013-11-20 04:06:34,896::task::495::TaskManager.Task::(_debug) Task 44179b79-2ae5-417c-9b74-27ea1a4c0be0: moving from state preparing -> state finished
>Thread-15915689::DEBUG::2013-11-20 04:06:34,896::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915689::DEBUG::2013-11-20 04:06:34,897::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915689::DEBUG::2013-11-20 04:06:34,898::task::495::TaskManager.Task::(_debug) Task 44179b79-2ae5-417c-9b74-27ea1a4c0be0: ref 0 aborting False
>Thread-15915689::INFO::2013-11-20 04:06:34,898::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915690::DEBUG::2013-11-20 04:06:34,903::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915690::INFO::2013-11-20 04:06:34,904::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915690::DEBUG::2013-11-20 04:06:34,905::task::495::TaskManager.Task::(_debug) Task bcc70be8-078c-4096-a0df-f2d7e662c269: moving from state init -> state preparing
>Thread-15915690::DEBUG::2013-11-20 04:06:34,907::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a10a0b10-68bf-404b-9346-9241a830e81b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915690::DEBUG::2013-11-20 04:06:34,908::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915690::DEBUG::2013-11-20 04:06:34,909::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915690::DEBUG::2013-11-20 04:06:34,910::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a10a0b10-68bf-404b-9346-9241a830e81b`::Granted request
>Thread-15915690::DEBUG::2013-11-20 04:06:34,911::task::495::TaskManager.Task::(_debug) Task bcc70be8-078c-4096-a0df-f2d7e662c269: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915690::DEBUG::2013-11-20 04:06:34,912::task::495::TaskManager.Task::(_debug) Task bcc70be8-078c-4096-a0df-f2d7e662c269: ref 1 aborting False
>Thread-15915690::WARNING::2013-11-20 04:06:34,914::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915690::DEBUG::2013-11-20 04:06:34,915::task::495::TaskManager.Task::(_debug) Task bcc70be8-078c-4096-a0df-f2d7e662c269: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915690::DEBUG::2013-11-20 04:06:34,916::task::495::TaskManager.Task::(_debug) Task bcc70be8-078c-4096-a0df-f2d7e662c269: moving from state preparing -> state finished
>Thread-15915690::DEBUG::2013-11-20 04:06:34,916::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915690::DEBUG::2013-11-20 04:06:34,917::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915690::DEBUG::2013-11-20 04:06:34,918::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915690::DEBUG::2013-11-20 04:06:34,918::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915690::DEBUG::2013-11-20 04:06:34,919::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915690::DEBUG::2013-11-20 04:06:34,920::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915690::DEBUG::2013-11-20 04:06:34,920::task::495::TaskManager.Task::(_debug) Task bcc70be8-078c-4096-a0df-f2d7e662c269: ref 0 aborting False
>Thread-15915690::INFO::2013-11-20 04:06:34,921::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:06:35,556::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:06:36,148::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0814769 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:06:38,166::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:06:38,776::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.105881 s, 9.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:06:40,836::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 04:06:41,219::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:06:41,224::task::495::TaskManager.Task::(_debug) Task 719a4299-1436-4860-a76c-0b7404476fa8: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:06:41,228::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`42187b02-d950-49fa-ae3a-eb4d47e7ba10`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:06:41,230::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:06:41,232::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:06:41,232::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`42187b02-d950-49fa-ae3a-eb4d47e7ba10`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:06:41,235::task::495::TaskManager.Task::(_debug) Task 719a4299-1436-4860-a76c-0b7404476fa8: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:06:41,236::task::495::TaskManager.Task::(_debug) Task 719a4299-1436-4860-a76c-0b7404476fa8: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:06:41,237::task::495::TaskManager.Task::(_debug) Task 719a4299-1436-4860-a76c-0b7404476fa8: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:06:41,238::task::495::TaskManager.Task::(_debug) Task 719a4299-1436-4860-a76c-0b7404476fa8: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:06:41,239::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:06:41,240::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:06:41,241::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:06:41,242::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:06:41,243::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:06:41,244::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:06:41,245::task::495::TaskManager.Task::(_debug) Task 719a4299-1436-4860-a76c-0b7404476fa8: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:06:41,246::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:06:41,364::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0898129 s, 11.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:06:43,379::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:06:43,935::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0857799 s, 11.9 MB/s\n'; <rc> = 0
>Thread-15915694::INFO::2013-11-20 04:06:44,187::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915694::DEBUG::2013-11-20 04:06:44,188::task::495::TaskManager.Task::(_debug) Task 79d04bf2-7744-464c-b7da-8bf8b1bf63d4: moving from state init -> state preparing
>Thread-15915694::DEBUG::2013-11-20 04:06:44,189::task::495::TaskManager.Task::(_debug) Task 79d04bf2-7744-464c-b7da-8bf8b1bf63d4: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00047492980957', 'lastCheck': 1384920396.21804, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00122594833374', 'lastCheck': 1384920397.794867, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00148391723633', 'lastCheck': 1384920394.786979, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00110721588135', 'lastCheck': 1384920401.7970181, 'valid': True, 'code': 0}}
>Thread-15915694::DEBUG::2013-11-20 04:06:44,190::task::495::TaskManager.Task::(_debug) Task 79d04bf2-7744-464c-b7da-8bf8b1bf63d4: moving from state preparing -> state finished
>Thread-15915694::DEBUG::2013-11-20 04:06:44,191::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915694::DEBUG::2013-11-20 04:06:44,191::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915694::DEBUG::2013-11-20 04:06:44,192::task::495::TaskManager.Task::(_debug) Task 79d04bf2-7744-464c-b7da-8bf8b1bf63d4: ref 0 aborting False
>Thread-15915694::INFO::2013-11-20 04:06:44,192::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00047492980957', 'lastCheck': 1384920396.21804, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00122594833374', 'lastCheck': 1384920397.794867, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00148391723633', 'lastCheck': 1384920394.786979, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00110721588135', 'lastCheck': 1384920401.7970181, 'valid': True, 'code': 0}}
>Thread-15913695::INFO::2013-11-20 04:06:45,053::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:06:45,054::task::495::TaskManager.Task::(_debug) Task 366e4ba4-4f4e-4086-a8c5-e64eedce0345: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:06:45,055::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0eb96d52-3dc0-43de-a7cb-6d98bf9e13c2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:06:45,056::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:06:45,057::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:06:45,058::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0eb96d52-3dc0-43de-a7cb-6d98bf9e13c2`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:06:45,059::task::495::TaskManager.Task::(_debug) Task 366e4ba4-4f4e-4086-a8c5-e64eedce0345: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:06:45,059::task::495::TaskManager.Task::(_debug) Task 366e4ba4-4f4e-4086-a8c5-e64eedce0345: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:06:45,060::task::495::TaskManager.Task::(_debug) Task 366e4ba4-4f4e-4086-a8c5-e64eedce0345: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:06:45,061::task::495::TaskManager.Task::(_debug) Task 366e4ba4-4f4e-4086-a8c5-e64eedce0345: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:06:45,062::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:06:45,062::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:06:45,063::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:06:45,064::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:06:45,064::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:06:45,065::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:06:45,066::task::495::TaskManager.Task::(_debug) Task 366e4ba4-4f4e-4086-a8c5-e64eedce0345: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:06:45,066::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15915696::DEBUG::2013-11-20 04:06:45,158::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915696::INFO::2013-11-20 04:06:45,159::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915696::DEBUG::2013-11-20 04:06:45,160::task::495::TaskManager.Task::(_debug) Task 5cc36758-5898-4fef-b40d-236f92fc9e0b: moving from state init -> state preparing
>Thread-15915696::DEBUG::2013-11-20 04:06:45,161::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915696::DEBUG::2013-11-20 04:06:45,162::task::495::TaskManager.Task::(_debug) Task 5cc36758-5898-4fef-b40d-236f92fc9e0b: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915696::DEBUG::2013-11-20 04:06:45,162::task::495::TaskManager.Task::(_debug) Task 5cc36758-5898-4fef-b40d-236f92fc9e0b: moving from state preparing -> state finished
>Thread-15915696::DEBUG::2013-11-20 04:06:45,163::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915696::DEBUG::2013-11-20 04:06:45,163::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915696::DEBUG::2013-11-20 04:06:45,164::task::495::TaskManager.Task::(_debug) Task 5cc36758-5898-4fef-b40d-236f92fc9e0b: ref 0 aborting False
>Thread-15915696::INFO::2013-11-20 04:06:45,164::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915697::DEBUG::2013-11-20 04:06:45,169::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915697::INFO::2013-11-20 04:06:45,170::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915697::DEBUG::2013-11-20 04:06:45,171::task::495::TaskManager.Task::(_debug) Task 3c28afe4-bc7a-4edd-91fe-9f414495e906: moving from state init -> state preparing
>Thread-15915697::DEBUG::2013-11-20 04:06:45,172::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8cb7a58e-cdc7-48d2-bf39-35e1140f6948`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915697::DEBUG::2013-11-20 04:06:45,173::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915697::DEBUG::2013-11-20 04:06:45,174::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915697::DEBUG::2013-11-20 04:06:45,174::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8cb7a58e-cdc7-48d2-bf39-35e1140f6948`::Granted request
>Thread-15915697::DEBUG::2013-11-20 04:06:45,181::task::495::TaskManager.Task::(_debug) Task 3c28afe4-bc7a-4edd-91fe-9f414495e906: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915697::DEBUG::2013-11-20 04:06:45,182::task::495::TaskManager.Task::(_debug) Task 3c28afe4-bc7a-4edd-91fe-9f414495e906: ref 1 aborting False
>Thread-15915697::WARNING::2013-11-20 04:06:45,184::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915697::DEBUG::2013-11-20 04:06:45,185::task::495::TaskManager.Task::(_debug) Task 3c28afe4-bc7a-4edd-91fe-9f414495e906: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915697::DEBUG::2013-11-20 04:06:45,186::task::495::TaskManager.Task::(_debug) Task 3c28afe4-bc7a-4edd-91fe-9f414495e906: moving from state preparing -> state finished
>Thread-15915697::DEBUG::2013-11-20 04:06:45,187::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915697::DEBUG::2013-11-20 04:06:45,187::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915697::DEBUG::2013-11-20 04:06:45,188::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915697::DEBUG::2013-11-20 04:06:45,189::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915697::DEBUG::2013-11-20 04:06:45,189::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915697::DEBUG::2013-11-20 04:06:45,190::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915697::DEBUG::2013-11-20 04:06:45,191::task::495::TaskManager.Task::(_debug) Task 3c28afe4-bc7a-4edd-91fe-9f414495e906: ref 0 aborting False
>Thread-15915697::INFO::2013-11-20 04:06:45,191::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:06:45,950::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915085::INFO::2013-11-20 04:06:46,450::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=089b08fc-6544-4235-a061-fb55b09d2715 volUUID=66b599fc-b7b5-4934-a5d5-ef7283460029)
>Thread-15915085::DEBUG::2013-11-20 04:06:46,452::task::495::TaskManager.Task::(_debug) Task 4a93aa2c-1793-4e74-8b3e-f793c47088ca: moving from state init -> state preparing
>Thread-15915085::DEBUG::2013-11-20 04:06:46,456::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`62c89dca-f266-4d7c-8eed-538b4136ae61`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915085::DEBUG::2013-11-20 04:06:46,458::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915085::DEBUG::2013-11-20 04:06:46,462::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915085::DEBUG::2013-11-20 04:06:46,464::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`62c89dca-f266-4d7c-8eed-538b4136ae61`::Granted request
>Thread-15915085::DEBUG::2013-11-20 04:06:46,466::task::495::TaskManager.Task::(_debug) Task 4a93aa2c-1793-4e74-8b3e-f793c47088ca: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915085::DEBUG::2013-11-20 04:06:46,468::task::495::TaskManager.Task::(_debug) Task 4a93aa2c-1793-4e74-8b3e-f793c47088ca: ref 1 aborting False
>Thread-15915085::DEBUG::2013-11-20 04:06:46,470::task::495::TaskManager.Task::(_debug) Task 4a93aa2c-1793-4e74-8b3e-f793c47088ca: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15915085::DEBUG::2013-11-20 04:06:46,471::task::495::TaskManager.Task::(_debug) Task 4a93aa2c-1793-4e74-8b3e-f793c47088ca: moving from state preparing -> state finished
>Thread-15915085::DEBUG::2013-11-20 04:06:46,472::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915085::DEBUG::2013-11-20 04:06:46,474::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915085::DEBUG::2013-11-20 04:06:46,475::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915085::DEBUG::2013-11-20 04:06:46,477::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915085::DEBUG::2013-11-20 04:06:46,478::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915085::DEBUG::2013-11-20 04:06:46,480::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915085::DEBUG::2013-11-20 04:06:46,482::task::495::TaskManager.Task::(_debug) Task 4a93aa2c-1793-4e74-8b3e-f793c47088ca: ref 0 aborting False
>Thread-15915085::INFO::2013-11-20 04:06:46,483::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Dummy-15912651::DEBUG::2013-11-20 04:06:46,552::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0834827 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:06:48,567::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:06:49,131::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0860738 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:06:51,152::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 04:06:51,387::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:06:51,392::task::495::TaskManager.Task::(_debug) Task 0f1c9d43-29b2-4ac1-bb03-bca201278cd0: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:06:51,397::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d87e4372-1190-4c96-99f5-ff1b2cf55455`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:06:51,399::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:06:51,400::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:06:51,401::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d87e4372-1190-4c96-99f5-ff1b2cf55455`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:06:51,404::task::495::TaskManager.Task::(_debug) Task 0f1c9d43-29b2-4ac1-bb03-bca201278cd0: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:06:51,405::task::495::TaskManager.Task::(_debug) Task 0f1c9d43-29b2-4ac1-bb03-bca201278cd0: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:06:51,406::task::495::TaskManager.Task::(_debug) Task 0f1c9d43-29b2-4ac1-bb03-bca201278cd0: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:06:51,407::task::495::TaskManager.Task::(_debug) Task 0f1c9d43-29b2-4ac1-bb03-bca201278cd0: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:06:51,408::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:06:51,408::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:06:51,410::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:06:51,411::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:06:51,412::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:06:51,413::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:06:51,415::task::495::TaskManager.Task::(_debug) Task 0f1c9d43-29b2-4ac1-bb03-bca201278cd0: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:06:51,416::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:06:51,728::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0796383 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:06:53,744::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:06:54,324::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0816308 s, 12.5 MB/s\n'; <rc> = 0
>Thread-15915701::DEBUG::2013-11-20 04:06:55,343::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915701::INFO::2013-11-20 04:06:55,345::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915701::DEBUG::2013-11-20 04:06:55,345::task::495::TaskManager.Task::(_debug) Task b0c0ca0a-81d2-4d19-a31b-e6eb2f5743ce: moving from state init -> state preparing
>Thread-15915701::DEBUG::2013-11-20 04:06:55,346::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915701::DEBUG::2013-11-20 04:06:55,347::task::495::TaskManager.Task::(_debug) Task b0c0ca0a-81d2-4d19-a31b-e6eb2f5743ce: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915701::DEBUG::2013-11-20 04:06:55,347::task::495::TaskManager.Task::(_debug) Task b0c0ca0a-81d2-4d19-a31b-e6eb2f5743ce: moving from state preparing -> state finished
>Thread-15915701::DEBUG::2013-11-20 04:06:55,348::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915701::DEBUG::2013-11-20 04:06:55,349::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915701::DEBUG::2013-11-20 04:06:55,349::task::495::TaskManager.Task::(_debug) Task b0c0ca0a-81d2-4d19-a31b-e6eb2f5743ce: ref 0 aborting False
>Thread-15915701::INFO::2013-11-20 04:06:55,350::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915702::DEBUG::2013-11-20 04:06:55,356::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915702::INFO::2013-11-20 04:06:55,357::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915702::DEBUG::2013-11-20 04:06:55,358::task::495::TaskManager.Task::(_debug) Task 64e9ba34-2f62-4635-adfd-c0e7062efdbd: moving from state init -> state preparing
>Thread-15915702::DEBUG::2013-11-20 04:06:55,360::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`053eb8e6-9bc3-4739-8fc5-ea2ff08d96d9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915702::DEBUG::2013-11-20 04:06:55,361::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915702::DEBUG::2013-11-20 04:06:55,362::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915702::DEBUG::2013-11-20 04:06:55,363::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`053eb8e6-9bc3-4739-8fc5-ea2ff08d96d9`::Granted request
>Thread-15915702::DEBUG::2013-11-20 04:06:55,365::task::495::TaskManager.Task::(_debug) Task 64e9ba34-2f62-4635-adfd-c0e7062efdbd: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915702::DEBUG::2013-11-20 04:06:55,366::task::495::TaskManager.Task::(_debug) Task 64e9ba34-2f62-4635-adfd-c0e7062efdbd: ref 1 aborting False
>Thread-15915702::WARNING::2013-11-20 04:06:55,368::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915702::DEBUG::2013-11-20 04:06:55,369::task::495::TaskManager.Task::(_debug) Task 64e9ba34-2f62-4635-adfd-c0e7062efdbd: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915702::DEBUG::2013-11-20 04:06:55,370::task::495::TaskManager.Task::(_debug) Task 64e9ba34-2f62-4635-adfd-c0e7062efdbd: moving from state preparing -> state finished
>Thread-15915702::DEBUG::2013-11-20 04:06:55,371::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915702::DEBUG::2013-11-20 04:06:55,371::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915702::DEBUG::2013-11-20 04:06:55,372::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915702::DEBUG::2013-11-20 04:06:55,373::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915702::DEBUG::2013-11-20 04:06:55,374::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915702::DEBUG::2013-11-20 04:06:55,375::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915702::DEBUG::2013-11-20 04:06:55,376::task::495::TaskManager.Task::(_debug) Task 64e9ba34-2f62-4635-adfd-c0e7062efdbd: ref 0 aborting False
>Thread-15915702::INFO::2013-11-20 04:06:55,377::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:06:56,334::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:06:56,902::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.103406 s, 9.9 MB/s\n'; <rc> = 0
>Thread-15914435::INFO::2013-11-20 04:06:58,036::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:06:58,037::task::495::TaskManager.Task::(_debug) Task 17c42e2e-60d6-49ee-862c-b8fa445cdaab: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:06:58,040::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e9e64c03-de60-4865-8b7f-07e97bf0d638`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:06:58,041::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:06:58,043::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:06:58,043::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e9e64c03-de60-4865-8b7f-07e97bf0d638`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:06:58,046::task::495::TaskManager.Task::(_debug) Task 17c42e2e-60d6-49ee-862c-b8fa445cdaab: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:06:58,047::task::495::TaskManager.Task::(_debug) Task 17c42e2e-60d6-49ee-862c-b8fa445cdaab: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:06:58,048::task::495::TaskManager.Task::(_debug) Task 17c42e2e-60d6-49ee-862c-b8fa445cdaab: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:06:58,048::task::495::TaskManager.Task::(_debug) Task 17c42e2e-60d6-49ee-862c-b8fa445cdaab: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:06:58,049::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:06:58,050::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:06:58,051::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:06:58,052::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:06:58,053::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:06:58,053::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:06:58,054::task::495::TaskManager.Task::(_debug) Task 17c42e2e-60d6-49ee-862c-b8fa445cdaab: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:06:58,055::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:06:58,919::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:06:59,509::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0795566 s, 12.9 MB/s\n'; <rc> = 0
>Thread-15915704::INFO::2013-11-20 04:06:59,702::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915704::DEBUG::2013-11-20 04:06:59,703::task::495::TaskManager.Task::(_debug) Task e3009dea-e68a-4758-835d-40ee976d544f: moving from state init -> state preparing
>Thread-15915704::DEBUG::2013-11-20 04:06:59,704::task::495::TaskManager.Task::(_debug) Task e3009dea-e68a-4758-835d-40ee976d544f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000336885452271', 'lastCheck': 1384920416.98944, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00163984298706', 'lastCheck': 1384920417.994761, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00167417526245', 'lastCheck': 1384920414.9859991, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000364065170288', 'lastCheck': 1384920411.808356, 'valid': True, 'code': 0}}
>Thread-15915704::DEBUG::2013-11-20 04:06:59,705::task::495::TaskManager.Task::(_debug) Task e3009dea-e68a-4758-835d-40ee976d544f: moving from state preparing -> state finished
>Thread-15915704::DEBUG::2013-11-20 04:06:59,705::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915704::DEBUG::2013-11-20 04:06:59,706::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915704::DEBUG::2013-11-20 04:06:59,707::task::495::TaskManager.Task::(_debug) Task e3009dea-e68a-4758-835d-40ee976d544f: ref 0 aborting False
>Thread-15915704::INFO::2013-11-20 04:06:59,707::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000336885452271', 'lastCheck': 1384920416.98944, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00163984298706', 'lastCheck': 1384920417.994761, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00167417526245', 'lastCheck': 1384920414.9859991, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000364065170288', 'lastCheck': 1384920411.808356, 'valid': True, 'code': 0}}
>Thread-15914519::INFO::2013-11-20 04:07:00,005::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:07:00,008::task::495::TaskManager.Task::(_debug) Task ab5040c1-362d-40d2-8428-d5ba001cedf3: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:07:00,020::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9ade079e-2363-4c47-90ec-130e27cdf6fc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:07:00,025::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:07:00,040::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:07:00,041::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9ade079e-2363-4c47-90ec-130e27cdf6fc`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:07:00,043::task::495::TaskManager.Task::(_debug) Task ab5040c1-362d-40d2-8428-d5ba001cedf3: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 04:07:00,044::task::495::TaskManager.Task::(_debug) Task ab5040c1-362d-40d2-8428-d5ba001cedf3: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:07:00,045::task::495::TaskManager.Task::(_debug) Task ab5040c1-362d-40d2-8428-d5ba001cedf3: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 04:07:00,046::task::495::TaskManager.Task::(_debug) Task ab5040c1-362d-40d2-8428-d5ba001cedf3: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:07:00,047::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:07:00,048::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:07:00,049::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 04:07:00,050::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:07:00,051::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:07:00,052::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:07:00,053::task::495::TaskManager.Task::(_debug) Task ab5040c1-362d-40d2-8428-d5ba001cedf3: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 04:07:00,054::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 04:07:01,526::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 04:07:01,746::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:07:01,750::task::495::TaskManager.Task::(_debug) Task ca711a42-ac32-43f0-974c-8af4c23a3df1: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:07:01,755::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`376d7971-408c-49a7-973b-dda3d79cdd7e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:07:01,758::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:07:01,761::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:07:01,762::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`376d7971-408c-49a7-973b-dda3d79cdd7e`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:07:01,764::task::495::TaskManager.Task::(_debug) Task ca711a42-ac32-43f0-974c-8af4c23a3df1: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:07:01,764::task::495::TaskManager.Task::(_debug) Task ca711a42-ac32-43f0-974c-8af4c23a3df1: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:07:01,766::task::495::TaskManager.Task::(_debug) Task ca711a42-ac32-43f0-974c-8af4c23a3df1: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 04:07:01,771::task::495::TaskManager.Task::(_debug) Task ca711a42-ac32-43f0-974c-8af4c23a3df1: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:07:01,772::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:07:01,773::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:07:01,774::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:07:01,775::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:07:01,777::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:07:01,778::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:07:01,779::task::495::TaskManager.Task::(_debug) Task ca711a42-ac32-43f0-974c-8af4c23a3df1: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:07:01,780::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:07:01,781::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:07:01,782::task::495::TaskManager.Task::(_debug) Task 020b5bd0-6363-4a09-b723-a5a154546919: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:07:01,784::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`295862d3-b612-4907-b6bf-d5d4b8cedf31`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:07:01,784::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:07:01,785::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:07:01,786::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`295862d3-b612-4907-b6bf-d5d4b8cedf31`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:07:01,787::task::495::TaskManager.Task::(_debug) Task 020b5bd0-6363-4a09-b723-a5a154546919: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:07:01,788::task::495::TaskManager.Task::(_debug) Task 020b5bd0-6363-4a09-b723-a5a154546919: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:07:01,788::task::495::TaskManager.Task::(_debug) Task 020b5bd0-6363-4a09-b723-a5a154546919: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:07:01,789::task::495::TaskManager.Task::(_debug) Task 020b5bd0-6363-4a09-b723-a5a154546919: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:07:01,790::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:07:01,790::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:07:01,791::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:07:01,792::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:07:01,792::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:07:01,793::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:07:01,794::task::495::TaskManager.Task::(_debug) Task 020b5bd0-6363-4a09-b723-a5a154546919: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:07:01,794::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:07:01,795::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:07:01,796::task::495::TaskManager.Task::(_debug) Task 8d099e3f-da38-47b9-bdd2-3b3e469fc456: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:07:01,798::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`1cdd5da9-86b0-45ba-b8c6-6fc4ae153a89`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:07:01,799::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:07:01,800::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:07:01,800::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`1cdd5da9-86b0-45ba-b8c6-6fc4ae153a89`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:07:01,802::task::495::TaskManager.Task::(_debug) Task 8d099e3f-da38-47b9-bdd2-3b3e469fc456: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:07:01,803::task::495::TaskManager.Task::(_debug) Task 8d099e3f-da38-47b9-bdd2-3b3e469fc456: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:07:01,803::task::495::TaskManager.Task::(_debug) Task 8d099e3f-da38-47b9-bdd2-3b3e469fc456: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:07:01,804::task::495::TaskManager.Task::(_debug) Task 8d099e3f-da38-47b9-bdd2-3b3e469fc456: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:07:01,805::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:07:01,806::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:07:01,806::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:07:01,807::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:07:01,808::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:07:01,808::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:07:01,809::task::495::TaskManager.Task::(_debug) Task 8d099e3f-da38-47b9-bdd2-3b3e469fc456: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:07:01,809::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:07:01,810::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:07:01,811::task::495::TaskManager.Task::(_debug) Task 6f5f995f-3fb3-41fb-81ad-949a7f4dec31: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:07:01,812::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`ff11be4a-138f-4dcb-b232-e0d513f66d8d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:07:01,813::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:07:01,813::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:07:01,814::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`ff11be4a-138f-4dcb-b232-e0d513f66d8d`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:07:01,815::task::495::TaskManager.Task::(_debug) Task 6f5f995f-3fb3-41fb-81ad-949a7f4dec31: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:07:01,815::task::495::TaskManager.Task::(_debug) Task 6f5f995f-3fb3-41fb-81ad-949a7f4dec31: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:07:01,816::task::495::TaskManager.Task::(_debug) Task 6f5f995f-3fb3-41fb-81ad-949a7f4dec31: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:07:01,817::task::495::TaskManager.Task::(_debug) Task 6f5f995f-3fb3-41fb-81ad-949a7f4dec31: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:07:01,817::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:07:01,818::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:07:01,818::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:07:01,819::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:07:01,820::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:07:01,820::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:07:01,821::task::495::TaskManager.Task::(_debug) Task 6f5f995f-3fb3-41fb-81ad-949a7f4dec31: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:07:01,821::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:07:02,102::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.087789 s, 11.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:07:04,124::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:07:04,696::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0824479 s, 12.4 MB/s\n'; <rc> = 0
>Thread-15915707::DEBUG::2013-11-20 04:07:05,611::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915707::INFO::2013-11-20 04:07:05,617::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915707::DEBUG::2013-11-20 04:07:05,621::task::495::TaskManager.Task::(_debug) Task 73e57ffa-878d-48ed-8c3c-a5a26775793b: moving from state init -> state preparing
>Thread-15915707::DEBUG::2013-11-20 04:07:05,624::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915707::DEBUG::2013-11-20 04:07:05,626::task::495::TaskManager.Task::(_debug) Task 73e57ffa-878d-48ed-8c3c-a5a26775793b: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915707::DEBUG::2013-11-20 04:07:05,627::task::495::TaskManager.Task::(_debug) Task 73e57ffa-878d-48ed-8c3c-a5a26775793b: moving from state preparing -> state finished
>Thread-15915707::DEBUG::2013-11-20 04:07:05,627::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915707::DEBUG::2013-11-20 04:07:05,628::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915707::DEBUG::2013-11-20 04:07:05,629::task::495::TaskManager.Task::(_debug) Task 73e57ffa-878d-48ed-8c3c-a5a26775793b: ref 0 aborting False
>Thread-15915707::INFO::2013-11-20 04:07:05,629::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915708::DEBUG::2013-11-20 04:07:05,634::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915708::INFO::2013-11-20 04:07:05,635::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915708::DEBUG::2013-11-20 04:07:05,635::task::495::TaskManager.Task::(_debug) Task 935360b7-669c-4f96-82b1-dbd62192b857: moving from state init -> state preparing
>Thread-15915708::DEBUG::2013-11-20 04:07:05,637::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6614b6ea-0150-490b-86e5-947071b9eda1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915708::DEBUG::2013-11-20 04:07:05,637::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915708::DEBUG::2013-11-20 04:07:05,638::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915708::DEBUG::2013-11-20 04:07:05,639::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6614b6ea-0150-490b-86e5-947071b9eda1`::Granted request
>Thread-15915708::DEBUG::2013-11-20 04:07:05,640::task::495::TaskManager.Task::(_debug) Task 935360b7-669c-4f96-82b1-dbd62192b857: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915708::DEBUG::2013-11-20 04:07:05,641::task::495::TaskManager.Task::(_debug) Task 935360b7-669c-4f96-82b1-dbd62192b857: ref 1 aborting False
>Thread-15915708::WARNING::2013-11-20 04:07:05,643::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915708::DEBUG::2013-11-20 04:07:05,644::task::495::TaskManager.Task::(_debug) Task 935360b7-669c-4f96-82b1-dbd62192b857: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915708::DEBUG::2013-11-20 04:07:05,644::task::495::TaskManager.Task::(_debug) Task 935360b7-669c-4f96-82b1-dbd62192b857: moving from state preparing -> state finished
>Thread-15915708::DEBUG::2013-11-20 04:07:05,645::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915708::DEBUG::2013-11-20 04:07:05,646::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915708::DEBUG::2013-11-20 04:07:05,646::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915708::DEBUG::2013-11-20 04:07:05,647::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915708::DEBUG::2013-11-20 04:07:05,647::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915708::DEBUG::2013-11-20 04:07:05,648::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915708::DEBUG::2013-11-20 04:07:05,649::task::495::TaskManager.Task::(_debug) Task 935360b7-669c-4f96-82b1-dbd62192b857: ref 0 aborting False
>Thread-15915708::INFO::2013-11-20 04:07:05,649::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:07:06,708::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:07:07,315::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0873861 s, 11.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:07:09,331::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:07:09,873::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0737091 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:07:11,888::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:07:12,452::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.067168 s, 15.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:07:14,464::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:07:15,050::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0858056 s, 11.9 MB/s\n'; <rc> = 0
>Thread-15915712::INFO::2013-11-20 04:07:15,228::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915712::DEBUG::2013-11-20 04:07:15,229::task::495::TaskManager.Task::(_debug) Task 9de7cecb-0191-45c3-8d86-23a42ef075c4: moving from state init -> state preparing
>Thread-15915712::DEBUG::2013-11-20 04:07:15,230::task::495::TaskManager.Task::(_debug) Task 9de7cecb-0191-45c3-8d86-23a42ef075c4: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000335931777954', 'lastCheck': 1384920427.362411, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00152897834778', 'lastCheck': 1384920428.3629651, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0283589363098', 'lastCheck': 1384920425.3851941, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00131797790527', 'lastCheck': 1384920432.5490329, 'valid': True, 'code': 0}}
>Thread-15915712::DEBUG::2013-11-20 04:07:15,231::task::495::TaskManager.Task::(_debug) Task 9de7cecb-0191-45c3-8d86-23a42ef075c4: moving from state preparing -> state finished
>Thread-15915712::DEBUG::2013-11-20 04:07:15,231::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915712::DEBUG::2013-11-20 04:07:15,232::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915712::DEBUG::2013-11-20 04:07:15,233::task::495::TaskManager.Task::(_debug) Task 9de7cecb-0191-45c3-8d86-23a42ef075c4: ref 0 aborting False
>Thread-15915712::INFO::2013-11-20 04:07:15,233::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000335931777954', 'lastCheck': 1384920427.362411, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00152897834778', 'lastCheck': 1384920428.3629651, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0283589363098', 'lastCheck': 1384920425.3851941, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00131797790527', 'lastCheck': 1384920432.5490329, 'valid': True, 'code': 0}}
>Thread-15915714::DEBUG::2013-11-20 04:07:16,073::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915714::INFO::2013-11-20 04:07:16,074::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915714::DEBUG::2013-11-20 04:07:16,075::task::495::TaskManager.Task::(_debug) Task 44176d4f-34cf-476a-a0f6-d193e8f4219b: moving from state init -> state preparing
>Thread-15915714::DEBUG::2013-11-20 04:07:16,076::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915714::DEBUG::2013-11-20 04:07:16,077::task::495::TaskManager.Task::(_debug) Task 44176d4f-34cf-476a-a0f6-d193e8f4219b: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915714::DEBUG::2013-11-20 04:07:16,077::task::495::TaskManager.Task::(_debug) Task 44176d4f-34cf-476a-a0f6-d193e8f4219b: moving from state preparing -> state finished
>Thread-15915714::DEBUG::2013-11-20 04:07:16,078::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915714::DEBUG::2013-11-20 04:07:16,079::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915714::DEBUG::2013-11-20 04:07:16,079::task::495::TaskManager.Task::(_debug) Task 44176d4f-34cf-476a-a0f6-d193e8f4219b: ref 0 aborting False
>Thread-15915714::INFO::2013-11-20 04:07:16,080::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915715::DEBUG::2013-11-20 04:07:16,086::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915715::INFO::2013-11-20 04:07:16,087::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915715::DEBUG::2013-11-20 04:07:16,088::task::495::TaskManager.Task::(_debug) Task d6f0cf5e-0164-43b5-a17d-70473183a424: moving from state init -> state preparing
>Thread-15915715::DEBUG::2013-11-20 04:07:16,089::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`99d00f30-28f2-4634-9a29-ead0e4f36d18`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915715::DEBUG::2013-11-20 04:07:16,090::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915715::DEBUG::2013-11-20 04:07:16,091::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915715::DEBUG::2013-11-20 04:07:16,092::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`99d00f30-28f2-4634-9a29-ead0e4f36d18`::Granted request
>Thread-15915715::DEBUG::2013-11-20 04:07:16,093::task::495::TaskManager.Task::(_debug) Task d6f0cf5e-0164-43b5-a17d-70473183a424: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915715::DEBUG::2013-11-20 04:07:16,094::task::495::TaskManager.Task::(_debug) Task d6f0cf5e-0164-43b5-a17d-70473183a424: ref 1 aborting False
>Thread-15915715::WARNING::2013-11-20 04:07:16,096::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915715::DEBUG::2013-11-20 04:07:16,096::task::495::TaskManager.Task::(_debug) Task d6f0cf5e-0164-43b5-a17d-70473183a424: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915715::DEBUG::2013-11-20 04:07:16,097::task::495::TaskManager.Task::(_debug) Task d6f0cf5e-0164-43b5-a17d-70473183a424: moving from state preparing -> state finished
>Thread-15915715::DEBUG::2013-11-20 04:07:16,098::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915715::DEBUG::2013-11-20 04:07:16,098::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915715::DEBUG::2013-11-20 04:07:16,099::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915715::DEBUG::2013-11-20 04:07:16,100::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915715::DEBUG::2013-11-20 04:07:16,100::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915715::DEBUG::2013-11-20 04:07:16,101::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915715::DEBUG::2013-11-20 04:07:16,101::task::495::TaskManager.Task::(_debug) Task d6f0cf5e-0164-43b5-a17d-70473183a424: ref 0 aborting False
>Thread-15915715::INFO::2013-11-20 04:07:16,102::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:07:17,067::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:07:17,288::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:07:17,291::task::495::TaskManager.Task::(_debug) Task c22ac0d5-8f47-43e3-a43f-e0ad0e36a0fe: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:07:17,294::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4d8004b1-0773-4a2c-a162-bbd8cd53465f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:07:17,295::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:07:17,296::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:07:17,297::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4d8004b1-0773-4a2c-a162-bbd8cd53465f`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:07:17,298::task::495::TaskManager.Task::(_debug) Task c22ac0d5-8f47-43e3-a43f-e0ad0e36a0fe: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:07:17,298::task::495::TaskManager.Task::(_debug) Task c22ac0d5-8f47-43e3-a43f-e0ad0e36a0fe: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:07:17,299::task::495::TaskManager.Task::(_debug) Task c22ac0d5-8f47-43e3-a43f-e0ad0e36a0fe: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:07:17,299::task::495::TaskManager.Task::(_debug) Task c22ac0d5-8f47-43e3-a43f-e0ad0e36a0fe: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:07:17,300::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:07:17,300::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:07:17,301::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:07:17,301::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:07:17,302::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:07:17,302::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:07:17,303::task::495::TaskManager.Task::(_debug) Task c22ac0d5-8f47-43e3-a43f-e0ad0e36a0fe: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:07:17,303::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:07:17,642::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0905922 s, 11.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:07:19,659::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:07:20,233::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0791875 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:07:22,250::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913518::INFO::2013-11-20 04:07:22,513::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:07:22,515::task::495::TaskManager.Task::(_debug) Task 3fd70677-ea32-49d5-a789-8c35c7ae4da0: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:07:22,518::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b732b523-7c1c-45ea-b868-e76ff09d792f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:07:22,519::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:07:22,520::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:07:22,521::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b732b523-7c1c-45ea-b868-e76ff09d792f`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:07:22,522::task::495::TaskManager.Task::(_debug) Task 3fd70677-ea32-49d5-a789-8c35c7ae4da0: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:07:22,523::task::495::TaskManager.Task::(_debug) Task 3fd70677-ea32-49d5-a789-8c35c7ae4da0: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:07:22,524::task::495::TaskManager.Task::(_debug) Task 3fd70677-ea32-49d5-a789-8c35c7ae4da0: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:07:22,525::task::495::TaskManager.Task::(_debug) Task 3fd70677-ea32-49d5-a789-8c35c7ae4da0: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:07:22,526::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:07:22,527::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:07:22,528::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:07:22,528::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:07:22,529::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:07:22,529::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:07:22,530::task::495::TaskManager.Task::(_debug) Task 3fd70677-ea32-49d5-a789-8c35c7ae4da0: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:07:22,531::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:07:22,532::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:07:22,532::task::495::TaskManager.Task::(_debug) Task e488b1af-92f0-4259-8be3-635b9135f333: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:07:22,533::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`35553db1-b44b-48bb-9e80-592d47a38fe9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:07:22,534::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:07:22,535::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:07:22,536::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`35553db1-b44b-48bb-9e80-592d47a38fe9`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:07:22,537::task::495::TaskManager.Task::(_debug) Task e488b1af-92f0-4259-8be3-635b9135f333: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:07:22,537::task::495::TaskManager.Task::(_debug) Task e488b1af-92f0-4259-8be3-635b9135f333: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:07:22,538::task::495::TaskManager.Task::(_debug) Task e488b1af-92f0-4259-8be3-635b9135f333: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:07:22,539::task::495::TaskManager.Task::(_debug) Task e488b1af-92f0-4259-8be3-635b9135f333: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:07:22,539::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:07:22,540::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:07:22,541::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:07:22,541::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:07:22,542::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:07:22,542::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:07:22,543::task::495::TaskManager.Task::(_debug) Task e488b1af-92f0-4259-8be3-635b9135f333: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:07:22,544::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:07:22,861::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.100068 s, 10.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:07:24,879::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:07:25,450::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0866673 s, 11.8 MB/s\n'; <rc> = 0
>Thread-15915719::DEBUG::2013-11-20 04:07:27,289::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915719::INFO::2013-11-20 04:07:27,291::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915719::DEBUG::2013-11-20 04:07:27,291::task::495::TaskManager.Task::(_debug) Task fc78be87-51f8-4a29-8b29-79d9ca2472c8: moving from state init -> state preparing
>Thread-15915719::DEBUG::2013-11-20 04:07:27,292::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915719::DEBUG::2013-11-20 04:07:27,293::task::495::TaskManager.Task::(_debug) Task fc78be87-51f8-4a29-8b29-79d9ca2472c8: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915719::DEBUG::2013-11-20 04:07:27,294::task::495::TaskManager.Task::(_debug) Task fc78be87-51f8-4a29-8b29-79d9ca2472c8: moving from state preparing -> state finished
>Thread-15915719::DEBUG::2013-11-20 04:07:27,294::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915719::DEBUG::2013-11-20 04:07:27,295::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915719::DEBUG::2013-11-20 04:07:27,295::task::495::TaskManager.Task::(_debug) Task fc78be87-51f8-4a29-8b29-79d9ca2472c8: ref 0 aborting False
>Thread-15915719::INFO::2013-11-20 04:07:27,296::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915720::DEBUG::2013-11-20 04:07:27,301::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915720::INFO::2013-11-20 04:07:27,303::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915720::DEBUG::2013-11-20 04:07:27,303::task::495::TaskManager.Task::(_debug) Task ead03cd6-faa6-4da8-b32c-0e8d504b1991: moving from state init -> state preparing
>Thread-15915720::DEBUG::2013-11-20 04:07:27,305::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`43c85649-a842-4052-b1ac-e77603274372`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915720::DEBUG::2013-11-20 04:07:27,306::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915720::DEBUG::2013-11-20 04:07:27,307::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915720::DEBUG::2013-11-20 04:07:27,307::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`43c85649-a842-4052-b1ac-e77603274372`::Granted request
>Thread-15915720::DEBUG::2013-11-20 04:07:27,308::task::495::TaskManager.Task::(_debug) Task ead03cd6-faa6-4da8-b32c-0e8d504b1991: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915720::DEBUG::2013-11-20 04:07:27,309::task::495::TaskManager.Task::(_debug) Task ead03cd6-faa6-4da8-b32c-0e8d504b1991: ref 1 aborting False
>Thread-15915720::WARNING::2013-11-20 04:07:27,312::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915720::DEBUG::2013-11-20 04:07:27,313::task::495::TaskManager.Task::(_debug) Task ead03cd6-faa6-4da8-b32c-0e8d504b1991: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915720::DEBUG::2013-11-20 04:07:27,313::task::495::TaskManager.Task::(_debug) Task ead03cd6-faa6-4da8-b32c-0e8d504b1991: moving from state preparing -> state finished
>Thread-15915720::DEBUG::2013-11-20 04:07:27,314::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915720::DEBUG::2013-11-20 04:07:27,315::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915720::DEBUG::2013-11-20 04:07:27,315::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915720::DEBUG::2013-11-20 04:07:27,316::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915720::DEBUG::2013-11-20 04:07:27,317::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915720::DEBUG::2013-11-20 04:07:27,318::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915720::DEBUG::2013-11-20 04:07:27,319::task::495::TaskManager.Task::(_debug) Task ead03cd6-faa6-4da8-b32c-0e8d504b1991: ref 0 aborting False
>Thread-15915720::INFO::2013-11-20 04:07:27,320::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:07:27,465::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:07:28,058::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0829027 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:07:30,073::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:07:30,639::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0917821 s, 11.2 MB/s\n'; <rc> = 0
>Thread-15915722::INFO::2013-11-20 04:07:32,459::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915722::DEBUG::2013-11-20 04:07:32,460::task::495::TaskManager.Task::(_debug) Task 4c09a238-2d59-448a-b529-af66365c3337: moving from state init -> state preparing
>Thread-15915722::DEBUG::2013-11-20 04:07:32,461::task::495::TaskManager.Task::(_debug) Task 4c09a238-2d59-448a-b529-af66365c3337: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000384092330933', 'lastCheck': 1384920448.1140699, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00161504745483', 'lastCheck': 1384920449.1122561, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00146293640137', 'lastCheck': 1384920446.1021819, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000573873519897', 'lastCheck': 1384920442.8895309, 'valid': True, 'code': 0}}
>Thread-15915722::DEBUG::2013-11-20 04:07:32,462::task::495::TaskManager.Task::(_debug) Task 4c09a238-2d59-448a-b529-af66365c3337: moving from state preparing -> state finished
>Thread-15915722::DEBUG::2013-11-20 04:07:32,462::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915722::DEBUG::2013-11-20 04:07:32,463::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915722::DEBUG::2013-11-20 04:07:32,464::task::495::TaskManager.Task::(_debug) Task 4c09a238-2d59-448a-b529-af66365c3337: ref 0 aborting False
>Thread-15915722::INFO::2013-11-20 04:07:32,464::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000384092330933', 'lastCheck': 1384920448.1140699, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00161504745483', 'lastCheck': 1384920449.1122561, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00146293640137', 'lastCheck': 1384920446.1021819, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000573873519897', 'lastCheck': 1384920442.8895309, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:07:32,656::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 04:07:32,927::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:07:32,935::task::495::TaskManager.Task::(_debug) Task 6b65be4c-6fa4-4604-9afb-6eafe60e7a32: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:07:32,942::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`cdb94472-d2a5-4e63-8dc6-453946aeaed7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:07:32,943::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:07:32,946::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:07:32,946::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`cdb94472-d2a5-4e63-8dc6-453946aeaed7`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:07:32,948::task::495::TaskManager.Task::(_debug) Task 6b65be4c-6fa4-4604-9afb-6eafe60e7a32: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:07:32,949::task::495::TaskManager.Task::(_debug) Task 6b65be4c-6fa4-4604-9afb-6eafe60e7a32: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:07:32,949::task::495::TaskManager.Task::(_debug) Task 6b65be4c-6fa4-4604-9afb-6eafe60e7a32: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:07:32,950::task::495::TaskManager.Task::(_debug) Task 6b65be4c-6fa4-4604-9afb-6eafe60e7a32: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:07:32,951::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:07:32,952::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:07:32,953::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:07:32,954::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:07:32,955::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:07:32,956::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:07:32,957::task::495::TaskManager.Task::(_debug) Task 6b65be4c-6fa4-4604-9afb-6eafe60e7a32: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:07:32,958::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:07:33,252::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0861199 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:07:35,269::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:07:35,804::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0829104 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:07:37,821::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915725::DEBUG::2013-11-20 04:07:38,043::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915725::INFO::2013-11-20 04:07:38,048::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915725::DEBUG::2013-11-20 04:07:38,049::task::495::TaskManager.Task::(_debug) Task 1cb82512-721e-47ac-a976-d7f047f6cfb9: moving from state init -> state preparing
>Thread-15915725::DEBUG::2013-11-20 04:07:38,053::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915725::DEBUG::2013-11-20 04:07:38,053::task::495::TaskManager.Task::(_debug) Task 1cb82512-721e-47ac-a976-d7f047f6cfb9: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915725::DEBUG::2013-11-20 04:07:38,055::task::495::TaskManager.Task::(_debug) Task 1cb82512-721e-47ac-a976-d7f047f6cfb9: moving from state preparing -> state finished
>Thread-15915725::DEBUG::2013-11-20 04:07:38,056::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915725::DEBUG::2013-11-20 04:07:38,056::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915725::DEBUG::2013-11-20 04:07:38,057::task::495::TaskManager.Task::(_debug) Task 1cb82512-721e-47ac-a976-d7f047f6cfb9: ref 0 aborting False
>Thread-15915725::INFO::2013-11-20 04:07:38,058::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915726::DEBUG::2013-11-20 04:07:38,062::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915726::INFO::2013-11-20 04:07:38,063::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915726::DEBUG::2013-11-20 04:07:38,064::task::495::TaskManager.Task::(_debug) Task 04781f8a-5c80-4ca2-be18-de3237bcb640: moving from state init -> state preparing
>Thread-15915726::DEBUG::2013-11-20 04:07:38,066::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e213bf6f-c987-4d3a-ae14-9b8f25998d06`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915726::DEBUG::2013-11-20 04:07:38,067::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915726::DEBUG::2013-11-20 04:07:38,068::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915726::DEBUG::2013-11-20 04:07:38,068::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e213bf6f-c987-4d3a-ae14-9b8f25998d06`::Granted request
>Thread-15915726::DEBUG::2013-11-20 04:07:38,070::task::495::TaskManager.Task::(_debug) Task 04781f8a-5c80-4ca2-be18-de3237bcb640: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915726::DEBUG::2013-11-20 04:07:38,070::task::495::TaskManager.Task::(_debug) Task 04781f8a-5c80-4ca2-be18-de3237bcb640: ref 1 aborting False
>Thread-15915726::WARNING::2013-11-20 04:07:38,074::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915726::DEBUG::2013-11-20 04:07:38,075::task::495::TaskManager.Task::(_debug) Task 04781f8a-5c80-4ca2-be18-de3237bcb640: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915726::DEBUG::2013-11-20 04:07:38,076::task::495::TaskManager.Task::(_debug) Task 04781f8a-5c80-4ca2-be18-de3237bcb640: moving from state preparing -> state finished
>Thread-15915726::DEBUG::2013-11-20 04:07:38,076::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915726::DEBUG::2013-11-20 04:07:38,077::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915726::DEBUG::2013-11-20 04:07:38,078::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915726::DEBUG::2013-11-20 04:07:38,079::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915726::DEBUG::2013-11-20 04:07:38,079::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915726::DEBUG::2013-11-20 04:07:38,081::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915726::DEBUG::2013-11-20 04:07:38,081::task::495::TaskManager.Task::(_debug) Task 04781f8a-5c80-4ca2-be18-de3237bcb640: ref 0 aborting False
>Thread-15915726::INFO::2013-11-20 04:07:38,082::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:07:38,378::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0755258 s, 13.6 MB/s\n'; <rc> = 0
>Thread-42838::DEBUG::2013-11-20 04:07:38,472::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 04:07:39,214::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:07:40,398::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:07:40,950::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0813001 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:07:42,961::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 04:07:43,184::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:07:43,190::task::495::TaskManager.Task::(_debug) Task be04d6e6-6d14-43c3-a776-f6b09992fe35: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:07:43,194::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e261fd03-cf95-4a73-b5d3-bc10df0dd3f9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:07:43,196::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:07:43,198::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:07:43,199::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e261fd03-cf95-4a73-b5d3-bc10df0dd3f9`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:07:43,201::task::495::TaskManager.Task::(_debug) Task be04d6e6-6d14-43c3-a776-f6b09992fe35: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:07:43,201::task::495::TaskManager.Task::(_debug) Task be04d6e6-6d14-43c3-a776-f6b09992fe35: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:07:43,202::task::495::TaskManager.Task::(_debug) Task be04d6e6-6d14-43c3-a776-f6b09992fe35: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:07:43,203::task::495::TaskManager.Task::(_debug) Task be04d6e6-6d14-43c3-a776-f6b09992fe35: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:07:43,204::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:07:43,205::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:07:43,205::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:07:43,206::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:07:43,207::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:07:43,208::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:07:43,209::task::495::TaskManager.Task::(_debug) Task be04d6e6-6d14-43c3-a776-f6b09992fe35: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:07:43,209::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:07:43,508::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0871978 s, 11.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:07:45,522::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 04:07:45,801::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:07:45,803::task::495::TaskManager.Task::(_debug) Task 62a89760-a6c3-4483-b5f7-85e7e937270f: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:07:45,806::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ff83048d-36ef-4c84-9635-9285781631fb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:07:45,807::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:07:45,808::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:07:45,809::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ff83048d-36ef-4c84-9635-9285781631fb`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:07:45,810::task::495::TaskManager.Task::(_debug) Task 62a89760-a6c3-4483-b5f7-85e7e937270f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:07:45,811::task::495::TaskManager.Task::(_debug) Task 62a89760-a6c3-4483-b5f7-85e7e937270f: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:07:45,812::task::495::TaskManager.Task::(_debug) Task 62a89760-a6c3-4483-b5f7-85e7e937270f: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:07:45,813::task::495::TaskManager.Task::(_debug) Task 62a89760-a6c3-4483-b5f7-85e7e937270f: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:07:45,814::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:07:45,815::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:07:45,816::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:07:45,817::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:07:45,817::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:07:45,818::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:07:45,819::task::495::TaskManager.Task::(_debug) Task 62a89760-a6c3-4483-b5f7-85e7e937270f: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:07:45,820::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:07:46,128::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0884678 s, 11.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:07:48,145::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915085::INFO::2013-11-20 04:07:48,412::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=089b08fc-6544-4235-a061-fb55b09d2715 volUUID=66b599fc-b7b5-4934-a5d5-ef7283460029)
>Thread-15915085::DEBUG::2013-11-20 04:07:48,434::task::495::TaskManager.Task::(_debug) Task 1b7d0012-d99f-4eac-98b7-d1b9c7364c08: moving from state init -> state preparing
>Thread-15915730::INFO::2013-11-20 04:07:48,447::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915085::DEBUG::2013-11-20 04:07:48,449::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`529644c1-0139-4673-9634-5438c501cf3d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915730::DEBUG::2013-11-20 04:07:48,451::task::495::TaskManager.Task::(_debug) Task e48235cb-c242-452e-833a-e866061ad7f1: moving from state init -> state preparing
>Thread-15915085::DEBUG::2013-11-20 04:07:48,453::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915730::DEBUG::2013-11-20 04:07:48,455::task::495::TaskManager.Task::(_debug) Task e48235cb-c242-452e-833a-e866061ad7f1: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000346899032593', 'lastCheck': 1384920459.2203121, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00134491920471', 'lastCheck': 1384920459.4786329, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0012059211731', 'lastCheck': 1384920466.785439, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000678062438965', 'lastCheck': 1384920463.6102591, 'valid': True, 'code': 0}}
>Thread-15915731::DEBUG::2013-11-20 04:07:48,458::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915730::DEBUG::2013-11-20 04:07:48,459::task::495::TaskManager.Task::(_debug) Task e48235cb-c242-452e-833a-e866061ad7f1: moving from state preparing -> state finished
>Thread-15915731::INFO::2013-11-20 04:07:48,461::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915085::DEBUG::2013-11-20 04:07:48,462::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915730::DEBUG::2013-11-20 04:07:48,463::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915731::DEBUG::2013-11-20 04:07:48,464::task::495::TaskManager.Task::(_debug) Task cfc8160d-cf6d-4cac-9e4d-d03db101fd4b: moving from state init -> state preparing
>Thread-15915085::DEBUG::2013-11-20 04:07:48,465::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`529644c1-0139-4673-9634-5438c501cf3d`::Granted request
>Thread-15915730::DEBUG::2013-11-20 04:07:48,466::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915731::DEBUG::2013-11-20 04:07:48,467::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915085::DEBUG::2013-11-20 04:07:48,470::task::495::TaskManager.Task::(_debug) Task 1b7d0012-d99f-4eac-98b7-d1b9c7364c08: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915730::DEBUG::2013-11-20 04:07:48,470::task::495::TaskManager.Task::(_debug) Task e48235cb-c242-452e-833a-e866061ad7f1: ref 0 aborting False
>Thread-15915731::DEBUG::2013-11-20 04:07:48,471::task::495::TaskManager.Task::(_debug) Task cfc8160d-cf6d-4cac-9e4d-d03db101fd4b: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915085::DEBUG::2013-11-20 04:07:48,471::task::495::TaskManager.Task::(_debug) Task 1b7d0012-d99f-4eac-98b7-d1b9c7364c08: ref 1 aborting False
>Thread-15915730::INFO::2013-11-20 04:07:48,473::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000346899032593', 'lastCheck': 1384920459.2203121, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00134491920471', 'lastCheck': 1384920459.4786329, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0012059211731', 'lastCheck': 1384920466.785439, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000678062438965', 'lastCheck': 1384920463.6102591, 'valid': True, 'code': 0}}
>Thread-15915731::DEBUG::2013-11-20 04:07:48,473::task::495::TaskManager.Task::(_debug) Task cfc8160d-cf6d-4cac-9e4d-d03db101fd4b: moving from state preparing -> state finished
>Thread-15915085::DEBUG::2013-11-20 04:07:48,475::task::495::TaskManager.Task::(_debug) Task 1b7d0012-d99f-4eac-98b7-d1b9c7364c08: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15915731::DEBUG::2013-11-20 04:07:48,476::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915085::DEBUG::2013-11-20 04:07:48,477::task::495::TaskManager.Task::(_debug) Task 1b7d0012-d99f-4eac-98b7-d1b9c7364c08: moving from state preparing -> state finished
>Thread-15915731::DEBUG::2013-11-20 04:07:48,478::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915085::DEBUG::2013-11-20 04:07:48,479::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915731::DEBUG::2013-11-20 04:07:48,481::task::495::TaskManager.Task::(_debug) Task cfc8160d-cf6d-4cac-9e4d-d03db101fd4b: ref 0 aborting False
>Thread-15915085::DEBUG::2013-11-20 04:07:48,482::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915731::INFO::2013-11-20 04:07:48,482::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915085::DEBUG::2013-11-20 04:07:48,484::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915085::DEBUG::2013-11-20 04:07:48,488::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915085::DEBUG::2013-11-20 04:07:48,489::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915085::DEBUG::2013-11-20 04:07:48,491::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915085::DEBUG::2013-11-20 04:07:48,498::task::495::TaskManager.Task::(_debug) Task 1b7d0012-d99f-4eac-98b7-d1b9c7364c08: ref 0 aborting False
>Thread-15915732::DEBUG::2013-11-20 04:07:48,500::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915085::INFO::2013-11-20 04:07:48,501::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15915732::INFO::2013-11-20 04:07:48,503::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915732::DEBUG::2013-11-20 04:07:48,504::task::495::TaskManager.Task::(_debug) Task 199805a7-ad0a-46f7-837d-4b011b1344cc: moving from state init -> state preparing
>Thread-15915732::DEBUG::2013-11-20 04:07:48,505::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`16188db7-6caf-4fa7-ad97-9d080fd41358`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915732::DEBUG::2013-11-20 04:07:48,506::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915732::DEBUG::2013-11-20 04:07:48,508::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915732::DEBUG::2013-11-20 04:07:48,509::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`16188db7-6caf-4fa7-ad97-9d080fd41358`::Granted request
>Thread-15915732::DEBUG::2013-11-20 04:07:48,510::task::495::TaskManager.Task::(_debug) Task 199805a7-ad0a-46f7-837d-4b011b1344cc: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915732::DEBUG::2013-11-20 04:07:48,511::task::495::TaskManager.Task::(_debug) Task 199805a7-ad0a-46f7-837d-4b011b1344cc: ref 1 aborting False
>Thread-15915732::WARNING::2013-11-20 04:07:48,514::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915732::DEBUG::2013-11-20 04:07:48,517::task::495::TaskManager.Task::(_debug) Task 199805a7-ad0a-46f7-837d-4b011b1344cc: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915732::DEBUG::2013-11-20 04:07:48,534::task::495::TaskManager.Task::(_debug) Task 199805a7-ad0a-46f7-837d-4b011b1344cc: moving from state preparing -> state finished
>Thread-15915732::DEBUG::2013-11-20 04:07:48,550::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915732::DEBUG::2013-11-20 04:07:48,551::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915732::DEBUG::2013-11-20 04:07:48,553::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915732::DEBUG::2013-11-20 04:07:48,554::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915732::DEBUG::2013-11-20 04:07:48,554::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915732::DEBUG::2013-11-20 04:07:48,555::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915732::DEBUG::2013-11-20 04:07:48,556::task::495::TaskManager.Task::(_debug) Task 199805a7-ad0a-46f7-837d-4b011b1344cc: ref 0 aborting False
>Thread-15915732::INFO::2013-11-20 04:07:48,557::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:07:48,776::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0824308 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:07:50,807::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:07:51,339::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0776682 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:07:53,354::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 04:07:53,593::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:07:53,595::task::495::TaskManager.Task::(_debug) Task 7957914b-c867-4b70-bd53-324ca7bd96f5: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:07:53,600::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f18ee94f-0902-477d-bc7b-adb5d88a2027`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:07:53,602::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:07:53,604::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:07:53,604::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f18ee94f-0902-477d-bc7b-adb5d88a2027`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:07:53,605::task::495::TaskManager.Task::(_debug) Task 7957914b-c867-4b70-bd53-324ca7bd96f5: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:07:53,606::task::495::TaskManager.Task::(_debug) Task 7957914b-c867-4b70-bd53-324ca7bd96f5: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:07:53,607::task::495::TaskManager.Task::(_debug) Task 7957914b-c867-4b70-bd53-324ca7bd96f5: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:07:53,607::task::495::TaskManager.Task::(_debug) Task 7957914b-c867-4b70-bd53-324ca7bd96f5: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:07:53,607::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:07:53,608::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:07:53,609::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:07:53,609::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:07:53,610::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:07:53,610::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:07:53,611::task::495::TaskManager.Task::(_debug) Task 7957914b-c867-4b70-bd53-324ca7bd96f5: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:07:53,612::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:07:53,905::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0787838 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:07:55,919::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:07:56,431::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0803413 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:07:58,449::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 04:07:58,672::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:07:58,674::task::495::TaskManager.Task::(_debug) Task 933ccda3-24a8-4daa-b8db-281ce42eeaaa: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:07:58,677::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`40229919-9649-426f-991c-b3ae3c26e864`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:07:58,677::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:07:58,678::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:07:58,679::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`40229919-9649-426f-991c-b3ae3c26e864`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:07:58,680::task::495::TaskManager.Task::(_debug) Task 933ccda3-24a8-4daa-b8db-281ce42eeaaa: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:07:58,681::task::495::TaskManager.Task::(_debug) Task 933ccda3-24a8-4daa-b8db-281ce42eeaaa: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:07:58,682::task::495::TaskManager.Task::(_debug) Task 933ccda3-24a8-4daa-b8db-281ce42eeaaa: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:07:58,683::task::495::TaskManager.Task::(_debug) Task 933ccda3-24a8-4daa-b8db-281ce42eeaaa: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:07:58,684::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:07:58,684::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:07:58,685::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:07:58,686::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:07:58,686::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:07:58,687::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:07:58,688::task::495::TaskManager.Task::(_debug) Task 933ccda3-24a8-4daa-b8db-281ce42eeaaa: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:07:58,689::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:07:58,989::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0782472 s, 13.1 MB/s\n'; <rc> = 0
>Thread-15915737::DEBUG::2013-11-20 04:07:59,044::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915737::INFO::2013-11-20 04:07:59,045::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915737::DEBUG::2013-11-20 04:07:59,046::task::495::TaskManager.Task::(_debug) Task 126e2dd3-fded-4c54-9c65-245f998d037a: moving from state init -> state preparing
>Thread-15915737::DEBUG::2013-11-20 04:07:59,047::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915737::DEBUG::2013-11-20 04:07:59,047::task::495::TaskManager.Task::(_debug) Task 126e2dd3-fded-4c54-9c65-245f998d037a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915737::DEBUG::2013-11-20 04:07:59,048::task::495::TaskManager.Task::(_debug) Task 126e2dd3-fded-4c54-9c65-245f998d037a: moving from state preparing -> state finished
>Thread-15915737::DEBUG::2013-11-20 04:07:59,049::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915737::DEBUG::2013-11-20 04:07:59,049::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915737::DEBUG::2013-11-20 04:07:59,050::task::495::TaskManager.Task::(_debug) Task 126e2dd3-fded-4c54-9c65-245f998d037a: ref 0 aborting False
>Thread-15915737::INFO::2013-11-20 04:07:59,051::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915738::DEBUG::2013-11-20 04:07:59,056::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915738::INFO::2013-11-20 04:07:59,057::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915738::DEBUG::2013-11-20 04:07:59,058::task::495::TaskManager.Task::(_debug) Task 9669afc5-3f07-45f2-93ec-bbd2ae34443e: moving from state init -> state preparing
>Thread-15915738::DEBUG::2013-11-20 04:07:59,059::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`70492537-170a-48b4-9cdd-08e7ee77c106`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915738::DEBUG::2013-11-20 04:07:59,060::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915738::DEBUG::2013-11-20 04:07:59,061::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915738::DEBUG::2013-11-20 04:07:59,062::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`70492537-170a-48b4-9cdd-08e7ee77c106`::Granted request
>Thread-15915738::DEBUG::2013-11-20 04:07:59,063::task::495::TaskManager.Task::(_debug) Task 9669afc5-3f07-45f2-93ec-bbd2ae34443e: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915738::DEBUG::2013-11-20 04:07:59,064::task::495::TaskManager.Task::(_debug) Task 9669afc5-3f07-45f2-93ec-bbd2ae34443e: ref 1 aborting False
>Thread-15915738::WARNING::2013-11-20 04:07:59,066::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915738::DEBUG::2013-11-20 04:07:59,067::task::495::TaskManager.Task::(_debug) Task 9669afc5-3f07-45f2-93ec-bbd2ae34443e: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915738::DEBUG::2013-11-20 04:07:59,068::task::495::TaskManager.Task::(_debug) Task 9669afc5-3f07-45f2-93ec-bbd2ae34443e: moving from state preparing -> state finished
>Thread-15915738::DEBUG::2013-11-20 04:07:59,069::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915738::DEBUG::2013-11-20 04:07:59,069::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915738::DEBUG::2013-11-20 04:07:59,070::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915738::DEBUG::2013-11-20 04:07:59,071::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915738::DEBUG::2013-11-20 04:07:59,071::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915738::DEBUG::2013-11-20 04:07:59,072::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915738::DEBUG::2013-11-20 04:07:59,072::task::495::TaskManager.Task::(_debug) Task 9669afc5-3f07-45f2-93ec-bbd2ae34443e: ref 0 aborting False
>Thread-15915738::INFO::2013-11-20 04:07:59,073::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:08:01,004::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 04:08:01,241::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:08:01,254::task::495::TaskManager.Task::(_debug) Task e21ba9a1-1599-49c6-855d-5f405867319e: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:08:01,257::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`380dd30e-dc96-4f86-b6dd-bea2ccaa8253`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:08:01,263::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:08:01,264::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:08:01,265::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`380dd30e-dc96-4f86-b6dd-bea2ccaa8253`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:08:01,266::task::495::TaskManager.Task::(_debug) Task e21ba9a1-1599-49c6-855d-5f405867319e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 04:08:01,266::task::495::TaskManager.Task::(_debug) Task e21ba9a1-1599-49c6-855d-5f405867319e: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:08:01,267::task::495::TaskManager.Task::(_debug) Task e21ba9a1-1599-49c6-855d-5f405867319e: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 04:08:01,267::task::495::TaskManager.Task::(_debug) Task e21ba9a1-1599-49c6-855d-5f405867319e: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:08:01,267::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:08:01,268::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:08:01,268::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 04:08:01,269::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:08:01,269::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:08:01,270::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:08:01,270::task::495::TaskManager.Task::(_debug) Task e21ba9a1-1599-49c6-855d-5f405867319e: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 04:08:01,270::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 04:08:01,627::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0850045 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:08:03,642::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 04:08:03,897::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:08:03,902::task::495::TaskManager.Task::(_debug) Task 0cafa22a-3279-4075-a653-17931ad03a81: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:08:03,905::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6964e321-abcc-4fcc-b5e9-b840b0a3f8fd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:08:03,906::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:08:03,907::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:08:03,908::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6964e321-abcc-4fcc-b5e9-b840b0a3f8fd`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:08:03,908::task::495::TaskManager.Task::(_debug) Task 0cafa22a-3279-4075-a653-17931ad03a81: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:08:03,909::task::495::TaskManager.Task::(_debug) Task 0cafa22a-3279-4075-a653-17931ad03a81: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:08:03,909::task::495::TaskManager.Task::(_debug) Task 0cafa22a-3279-4075-a653-17931ad03a81: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 04:08:03,910::task::495::TaskManager.Task::(_debug) Task 0cafa22a-3279-4075-a653-17931ad03a81: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:08:03,910::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:08:03,910::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:08:03,911::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:08:03,911::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:08:03,912::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:08:03,912::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:08:03,914::task::495::TaskManager.Task::(_debug) Task 0cafa22a-3279-4075-a653-17931ad03a81: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:08:03,914::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:08:03,915::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:08:03,916::task::495::TaskManager.Task::(_debug) Task 6e21e2fa-ed02-48b3-a908-c3cd20ee8918: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:08:03,917::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`cfbd4b9f-d192-4029-ac38-ab8da142218c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:08:03,917::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:08:03,918::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:08:03,919::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`cfbd4b9f-d192-4029-ac38-ab8da142218c`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:08:03,920::task::495::TaskManager.Task::(_debug) Task 6e21e2fa-ed02-48b3-a908-c3cd20ee8918: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:08:03,920::task::495::TaskManager.Task::(_debug) Task 6e21e2fa-ed02-48b3-a908-c3cd20ee8918: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:08:03,921::task::495::TaskManager.Task::(_debug) Task 6e21e2fa-ed02-48b3-a908-c3cd20ee8918: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:08:03,921::task::495::TaskManager.Task::(_debug) Task 6e21e2fa-ed02-48b3-a908-c3cd20ee8918: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:08:03,921::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:08:03,922::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:08:03,922::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:08:03,923::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:08:03,923::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:08:03,924::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:08:03,924::task::495::TaskManager.Task::(_debug) Task 6e21e2fa-ed02-48b3-a908-c3cd20ee8918: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:08:03,924::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:08:03,925::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:08:03,925::task::495::TaskManager.Task::(_debug) Task 5929ef4a-ae21-461e-9946-0dcc0a19b5c1: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:08:03,926::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`0ae54405-4116-4474-81c9-68835f19d557`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:08:03,926::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:08:03,927::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:08:03,927::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`0ae54405-4116-4474-81c9-68835f19d557`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:08:03,928::task::495::TaskManager.Task::(_debug) Task 5929ef4a-ae21-461e-9946-0dcc0a19b5c1: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:08:03,928::task::495::TaskManager.Task::(_debug) Task 5929ef4a-ae21-461e-9946-0dcc0a19b5c1: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:08:03,929::task::495::TaskManager.Task::(_debug) Task 5929ef4a-ae21-461e-9946-0dcc0a19b5c1: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:08:03,929::task::495::TaskManager.Task::(_debug) Task 5929ef4a-ae21-461e-9946-0dcc0a19b5c1: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:08:03,930::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:08:03,930::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:08:03,931::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:08:03,931::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:08:03,931::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:08:03,932::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:08:03,932::task::495::TaskManager.Task::(_debug) Task 5929ef4a-ae21-461e-9946-0dcc0a19b5c1: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:08:03,933::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:08:03,933::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:08:03,934::task::495::TaskManager.Task::(_debug) Task 77970330-3933-43ee-885f-2f53f6f44611: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:08:03,934::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e0984cb3-afd1-46ab-ae35-69569bbd63f6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:08:03,935::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:08:03,935::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:08:03,936::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e0984cb3-afd1-46ab-ae35-69569bbd63f6`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:08:03,936::task::495::TaskManager.Task::(_debug) Task 77970330-3933-43ee-885f-2f53f6f44611: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:08:03,937::task::495::TaskManager.Task::(_debug) Task 77970330-3933-43ee-885f-2f53f6f44611: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:08:03,937::task::495::TaskManager.Task::(_debug) Task 77970330-3933-43ee-885f-2f53f6f44611: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:08:03,938::task::495::TaskManager.Task::(_debug) Task 77970330-3933-43ee-885f-2f53f6f44611: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:08:03,938::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:08:03,938::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:08:03,939::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:08:03,939::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:08:03,939::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:08:03,940::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:08:03,940::task::495::TaskManager.Task::(_debug) Task 77970330-3933-43ee-885f-2f53f6f44611: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:08:03,941::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:08:04,220::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0821373 s, 12.5 MB/s\n'; <rc> = 0
>Thread-15915740::INFO::2013-11-20 04:08:04,282::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915740::DEBUG::2013-11-20 04:08:04,283::task::495::TaskManager.Task::(_debug) Task 835399bb-6237-477f-b911-a5ccf7cf4c6f: moving from state init -> state preparing
>Thread-15915740::DEBUG::2013-11-20 04:08:04,284::task::495::TaskManager.Task::(_debug) Task 835399bb-6237-477f-b911-a5ccf7cf4c6f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000391006469727', 'lastCheck': 1384920479.659132, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00185203552246', 'lastCheck': 1384920479.802326, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00140595436096', 'lastCheck': 1384920476.802526, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00588989257812', 'lastCheck': 1384920483.901665, 'valid': True, 'code': 0}}
>Thread-15915740::DEBUG::2013-11-20 04:08:04,294::task::495::TaskManager.Task::(_debug) Task 835399bb-6237-477f-b911-a5ccf7cf4c6f: moving from state preparing -> state finished
>Thread-15915740::DEBUG::2013-11-20 04:08:04,295::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915740::DEBUG::2013-11-20 04:08:04,296::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915740::DEBUG::2013-11-20 04:08:04,296::task::495::TaskManager.Task::(_debug) Task 835399bb-6237-477f-b911-a5ccf7cf4c6f: ref 0 aborting False
>Thread-15915740::INFO::2013-11-20 04:08:04,297::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000391006469727', 'lastCheck': 1384920479.659132, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00185203552246', 'lastCheck': 1384920479.802326, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00140595436096', 'lastCheck': 1384920476.802526, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00588989257812', 'lastCheck': 1384920483.901665, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:08:06,248::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:08:06,819::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0792189 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:08:08,832::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:08:09,389::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0911787 s, 11.2 MB/s\n'; <rc> = 0
>Thread-15915743::DEBUG::2013-11-20 04:08:09,827::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915743::INFO::2013-11-20 04:08:09,828::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915743::DEBUG::2013-11-20 04:08:09,829::task::495::TaskManager.Task::(_debug) Task 03d432fd-3112-4907-a531-b4894c89b599: moving from state init -> state preparing
>Thread-15915743::DEBUG::2013-11-20 04:08:09,830::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915743::DEBUG::2013-11-20 04:08:09,831::task::495::TaskManager.Task::(_debug) Task 03d432fd-3112-4907-a531-b4894c89b599: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915743::DEBUG::2013-11-20 04:08:09,831::task::495::TaskManager.Task::(_debug) Task 03d432fd-3112-4907-a531-b4894c89b599: moving from state preparing -> state finished
>Thread-15915743::DEBUG::2013-11-20 04:08:09,832::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915743::DEBUG::2013-11-20 04:08:09,833::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915743::DEBUG::2013-11-20 04:08:09,833::task::495::TaskManager.Task::(_debug) Task 03d432fd-3112-4907-a531-b4894c89b599: ref 0 aborting False
>Thread-15915743::INFO::2013-11-20 04:08:09,834::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915744::DEBUG::2013-11-20 04:08:09,839::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915744::INFO::2013-11-20 04:08:09,841::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915744::DEBUG::2013-11-20 04:08:09,841::task::495::TaskManager.Task::(_debug) Task 6b0865b9-84a7-4b58-ae03-379009784f34: moving from state init -> state preparing
>Thread-15915744::DEBUG::2013-11-20 04:08:09,843::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d06c2681-52be-4d77-8ff6-005977e4df38`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915744::DEBUG::2013-11-20 04:08:09,844::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915744::DEBUG::2013-11-20 04:08:09,845::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915744::DEBUG::2013-11-20 04:08:09,846::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d06c2681-52be-4d77-8ff6-005977e4df38`::Granted request
>Thread-15915744::DEBUG::2013-11-20 04:08:09,847::task::495::TaskManager.Task::(_debug) Task 6b0865b9-84a7-4b58-ae03-379009784f34: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915744::DEBUG::2013-11-20 04:08:09,848::task::495::TaskManager.Task::(_debug) Task 6b0865b9-84a7-4b58-ae03-379009784f34: ref 1 aborting False
>Thread-15915744::WARNING::2013-11-20 04:08:09,850::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915744::DEBUG::2013-11-20 04:08:09,852::task::495::TaskManager.Task::(_debug) Task 6b0865b9-84a7-4b58-ae03-379009784f34: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915744::DEBUG::2013-11-20 04:08:09,853::task::495::TaskManager.Task::(_debug) Task 6b0865b9-84a7-4b58-ae03-379009784f34: moving from state preparing -> state finished
>Thread-15915744::DEBUG::2013-11-20 04:08:09,853::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915744::DEBUG::2013-11-20 04:08:09,854::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915744::DEBUG::2013-11-20 04:08:09,855::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915744::DEBUG::2013-11-20 04:08:09,856::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915744::DEBUG::2013-11-20 04:08:09,856::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915744::DEBUG::2013-11-20 04:08:09,857::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915744::DEBUG::2013-11-20 04:08:09,858::task::495::TaskManager.Task::(_debug) Task 6b0865b9-84a7-4b58-ae03-379009784f34: ref 0 aborting False
>Thread-15915744::INFO::2013-11-20 04:08:09,858::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:08:11,408::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:08:11,965::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0862509 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:08:13,979::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:08:14,435::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0474142 s, 21.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:08:16,446::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:08:17,022::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0782032 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:08:19,035::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:08:19,264::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:08:19,270::task::495::TaskManager.Task::(_debug) Task c98a8a74-2181-4f3a-b786-f4e76d492c52: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:08:19,277::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`89f4f934-05ab-4d91-aea1-24074183a7d5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:08:19,278::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:08:19,281::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:08:19,283::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`89f4f934-05ab-4d91-aea1-24074183a7d5`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:08:19,285::task::495::TaskManager.Task::(_debug) Task c98a8a74-2181-4f3a-b786-f4e76d492c52: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:08:19,286::task::495::TaskManager.Task::(_debug) Task c98a8a74-2181-4f3a-b786-f4e76d492c52: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:08:19,288::task::495::TaskManager.Task::(_debug) Task c98a8a74-2181-4f3a-b786-f4e76d492c52: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:08:19,289::task::495::TaskManager.Task::(_debug) Task c98a8a74-2181-4f3a-b786-f4e76d492c52: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:08:19,290::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:08:19,291::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:08:19,292::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:08:19,294::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:08:19,295::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:08:19,296::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:08:19,297::task::495::TaskManager.Task::(_debug) Task c98a8a74-2181-4f3a-b786-f4e76d492c52: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:08:19,298::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:08:19,632::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0900112 s, 11.4 MB/s\n'; <rc> = 0
>Thread-15915748::DEBUG::2013-11-20 04:08:20,015::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915748::INFO::2013-11-20 04:08:20,016::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915748::DEBUG::2013-11-20 04:08:20,017::task::495::TaskManager.Task::(_debug) Task 6c332716-a8c4-48e5-822d-c9a7aebe712d: moving from state init -> state preparing
>Thread-15915748::DEBUG::2013-11-20 04:08:20,018::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915748::DEBUG::2013-11-20 04:08:20,019::task::495::TaskManager.Task::(_debug) Task 6c332716-a8c4-48e5-822d-c9a7aebe712d: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915748::DEBUG::2013-11-20 04:08:20,019::task::495::TaskManager.Task::(_debug) Task 6c332716-a8c4-48e5-822d-c9a7aebe712d: moving from state preparing -> state finished
>Thread-15915748::DEBUG::2013-11-20 04:08:20,020::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915748::DEBUG::2013-11-20 04:08:20,021::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915748::DEBUG::2013-11-20 04:08:20,021::task::495::TaskManager.Task::(_debug) Task 6c332716-a8c4-48e5-822d-c9a7aebe712d: ref 0 aborting False
>Thread-15915748::INFO::2013-11-20 04:08:20,022::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915749::DEBUG::2013-11-20 04:08:20,026::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915749::INFO::2013-11-20 04:08:20,027::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915749::DEBUG::2013-11-20 04:08:20,028::task::495::TaskManager.Task::(_debug) Task 939ac504-3e99-4e11-bc4a-a64b3faa5362: moving from state init -> state preparing
>Thread-15915749::DEBUG::2013-11-20 04:08:20,029::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1c41578d-7765-45dc-900c-11cb30374392`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915749::DEBUG::2013-11-20 04:08:20,030::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915749::DEBUG::2013-11-20 04:08:20,031::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915749::DEBUG::2013-11-20 04:08:20,031::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1c41578d-7765-45dc-900c-11cb30374392`::Granted request
>Thread-15915749::DEBUG::2013-11-20 04:08:20,032::task::495::TaskManager.Task::(_debug) Task 939ac504-3e99-4e11-bc4a-a64b3faa5362: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915749::DEBUG::2013-11-20 04:08:20,033::task::495::TaskManager.Task::(_debug) Task 939ac504-3e99-4e11-bc4a-a64b3faa5362: ref 1 aborting False
>Thread-15915749::WARNING::2013-11-20 04:08:20,035::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915749::DEBUG::2013-11-20 04:08:20,037::task::495::TaskManager.Task::(_debug) Task 939ac504-3e99-4e11-bc4a-a64b3faa5362: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915749::DEBUG::2013-11-20 04:08:20,038::task::495::TaskManager.Task::(_debug) Task 939ac504-3e99-4e11-bc4a-a64b3faa5362: moving from state preparing -> state finished
>Thread-15915749::DEBUG::2013-11-20 04:08:20,038::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915749::DEBUG::2013-11-20 04:08:20,039::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915749::DEBUG::2013-11-20 04:08:20,040::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915749::DEBUG::2013-11-20 04:08:20,041::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915749::DEBUG::2013-11-20 04:08:20,042::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915749::DEBUG::2013-11-20 04:08:20,043::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915749::DEBUG::2013-11-20 04:08:20,044::task::495::TaskManager.Task::(_debug) Task 939ac504-3e99-4e11-bc4a-a64b3faa5362: ref 0 aborting False
>Thread-15915749::INFO::2013-11-20 04:08:20,045::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915750::INFO::2013-11-20 04:08:20,285::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915750::DEBUG::2013-11-20 04:08:20,286::task::495::TaskManager.Task::(_debug) Task b11a8213-6d29-4e87-935c-eb81195f728f: moving from state init -> state preparing
>Thread-15915750::DEBUG::2013-11-20 04:08:20,286::task::495::TaskManager.Task::(_debug) Task b11a8213-6d29-4e87-935c-eb81195f728f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000369071960449', 'lastCheck': 1384920500.2632179, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00163197517395', 'lastCheck': 1384920500.263371, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00137209892273', 'lastCheck': 1384920497.2233529, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00341105461121', 'lastCheck': 1384920494.2219801, 'valid': True, 'code': 0}}
>Thread-15915750::DEBUG::2013-11-20 04:08:20,295::task::495::TaskManager.Task::(_debug) Task b11a8213-6d29-4e87-935c-eb81195f728f: moving from state preparing -> state finished
>Thread-15915750::DEBUG::2013-11-20 04:08:20,296::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915750::DEBUG::2013-11-20 04:08:20,297::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915750::DEBUG::2013-11-20 04:08:20,298::task::495::TaskManager.Task::(_debug) Task b11a8213-6d29-4e87-935c-eb81195f728f: ref 0 aborting False
>Thread-15915750::INFO::2013-11-20 04:08:20,299::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000369071960449', 'lastCheck': 1384920500.2632179, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00163197517395', 'lastCheck': 1384920500.263371, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00137209892273', 'lastCheck': 1384920497.2233529, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00341105461121', 'lastCheck': 1384920494.2219801, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:08:21,650::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:08:22,195::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0840364 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:08:24,206::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913518::INFO::2013-11-20 04:08:24,449::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:08:24,450::task::495::TaskManager.Task::(_debug) Task 2738d187-c9f7-4f55-8a7d-38596d160438: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:08:24,453::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5716d0f3-5efd-4c32-948d-456152b92bb5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:08:24,454::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:08:24,455::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:08:24,456::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5716d0f3-5efd-4c32-948d-456152b92bb5`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:08:24,457::task::495::TaskManager.Task::(_debug) Task 2738d187-c9f7-4f55-8a7d-38596d160438: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:08:24,458::task::495::TaskManager.Task::(_debug) Task 2738d187-c9f7-4f55-8a7d-38596d160438: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:08:24,459::task::495::TaskManager.Task::(_debug) Task 2738d187-c9f7-4f55-8a7d-38596d160438: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:08:24,460::task::495::TaskManager.Task::(_debug) Task 2738d187-c9f7-4f55-8a7d-38596d160438: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:08:24,461::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:08:24,462::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:08:24,463::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:08:24,464::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:08:24,464::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:08:24,465::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:08:24,466::task::495::TaskManager.Task::(_debug) Task 2738d187-c9f7-4f55-8a7d-38596d160438: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:08:24,467::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:08:24,469::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:08:24,469::task::495::TaskManager.Task::(_debug) Task 8e3998be-e716-4bcb-9eb0-56e25f8b3d0c: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:08:24,471::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ba6598ce-e9e1-4f4a-9351-785cd4157f1f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:08:24,471::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:08:24,472::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:08:24,473::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ba6598ce-e9e1-4f4a-9351-785cd4157f1f`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:08:24,475::task::495::TaskManager.Task::(_debug) Task 8e3998be-e716-4bcb-9eb0-56e25f8b3d0c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:08:24,475::task::495::TaskManager.Task::(_debug) Task 8e3998be-e716-4bcb-9eb0-56e25f8b3d0c: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:08:24,476::task::495::TaskManager.Task::(_debug) Task 8e3998be-e716-4bcb-9eb0-56e25f8b3d0c: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:08:24,477::task::495::TaskManager.Task::(_debug) Task 8e3998be-e716-4bcb-9eb0-56e25f8b3d0c: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:08:24,477::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:08:24,478::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:08:24,478::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:08:24,479::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:08:24,480::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:08:24,480::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:08:24,481::task::495::TaskManager.Task::(_debug) Task 8e3998be-e716-4bcb-9eb0-56e25f8b3d0c: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:08:24,482::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:08:24,754::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0763612 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:08:26,773::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:08:27,315::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0723626 s, 14.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:08:29,332::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:08:29,888::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0732572 s, 14.0 MB/s\n'; <rc> = 0
>Thread-15915755::DEBUG::2013-11-20 04:08:30,342::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915755::INFO::2013-11-20 04:08:30,344::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915755::DEBUG::2013-11-20 04:08:30,345::task::495::TaskManager.Task::(_debug) Task 44b4c356-8ff9-4bb4-a421-0b4dd1143292: moving from state init -> state preparing
>Thread-15915755::DEBUG::2013-11-20 04:08:30,346::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915755::DEBUG::2013-11-20 04:08:30,347::task::495::TaskManager.Task::(_debug) Task 44b4c356-8ff9-4bb4-a421-0b4dd1143292: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915755::DEBUG::2013-11-20 04:08:30,348::task::495::TaskManager.Task::(_debug) Task 44b4c356-8ff9-4bb4-a421-0b4dd1143292: moving from state preparing -> state finished
>Thread-15915755::DEBUG::2013-11-20 04:08:30,349::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915755::DEBUG::2013-11-20 04:08:30,349::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915755::DEBUG::2013-11-20 04:08:30,350::task::495::TaskManager.Task::(_debug) Task 44b4c356-8ff9-4bb4-a421-0b4dd1143292: ref 0 aborting False
>Thread-15915755::INFO::2013-11-20 04:08:30,351::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915756::DEBUG::2013-11-20 04:08:30,357::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915756::INFO::2013-11-20 04:08:30,358::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915756::DEBUG::2013-11-20 04:08:30,359::task::495::TaskManager.Task::(_debug) Task ed10dcf5-a418-4584-993f-87d1a6553a60: moving from state init -> state preparing
>Thread-15915756::DEBUG::2013-11-20 04:08:30,360::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`87cdd98c-1c87-4434-8f5f-6d1b7f6cfe44`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915756::DEBUG::2013-11-20 04:08:30,361::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915756::DEBUG::2013-11-20 04:08:30,362::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915756::DEBUG::2013-11-20 04:08:30,363::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`87cdd98c-1c87-4434-8f5f-6d1b7f6cfe44`::Granted request
>Thread-15915756::DEBUG::2013-11-20 04:08:30,364::task::495::TaskManager.Task::(_debug) Task ed10dcf5-a418-4584-993f-87d1a6553a60: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915756::DEBUG::2013-11-20 04:08:30,365::task::495::TaskManager.Task::(_debug) Task ed10dcf5-a418-4584-993f-87d1a6553a60: ref 1 aborting False
>Thread-15915756::WARNING::2013-11-20 04:08:30,368::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915756::DEBUG::2013-11-20 04:08:30,369::task::495::TaskManager.Task::(_debug) Task ed10dcf5-a418-4584-993f-87d1a6553a60: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915756::DEBUG::2013-11-20 04:08:30,370::task::495::TaskManager.Task::(_debug) Task ed10dcf5-a418-4584-993f-87d1a6553a60: moving from state preparing -> state finished
>Thread-15915756::DEBUG::2013-11-20 04:08:30,371::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915756::DEBUG::2013-11-20 04:08:30,372::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915756::DEBUG::2013-11-20 04:08:30,373::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915756::DEBUG::2013-11-20 04:08:30,374::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915756::DEBUG::2013-11-20 04:08:30,375::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915756::DEBUG::2013-11-20 04:08:30,375::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915756::DEBUG::2013-11-20 04:08:30,376::task::495::TaskManager.Task::(_debug) Task ed10dcf5-a418-4584-993f-87d1a6553a60: ref 0 aborting False
>Thread-15915756::INFO::2013-11-20 04:08:30,377::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:08:31,905::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:08:32,485::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0776799 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:08:34,500::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 04:08:34,757::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:08:34,761::task::495::TaskManager.Task::(_debug) Task 1469ea70-d2ac-47b3-803a-56b366e33233: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:08:34,764::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`93e7bd93-fd0a-4a36-ab02-96d2976f008e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:08:34,766::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:08:34,769::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:08:34,770::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`93e7bd93-fd0a-4a36-ab02-96d2976f008e`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:08:34,772::task::495::TaskManager.Task::(_debug) Task 1469ea70-d2ac-47b3-803a-56b366e33233: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:08:34,773::task::495::TaskManager.Task::(_debug) Task 1469ea70-d2ac-47b3-803a-56b366e33233: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:08:34,775::task::495::TaskManager.Task::(_debug) Task 1469ea70-d2ac-47b3-803a-56b366e33233: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:08:34,775::task::495::TaskManager.Task::(_debug) Task 1469ea70-d2ac-47b3-803a-56b366e33233: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:08:34,776::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:08:34,777::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:08:34,778::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:08:34,778::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:08:34,779::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:08:34,780::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:08:34,780::task::495::TaskManager.Task::(_debug) Task 1469ea70-d2ac-47b3-803a-56b366e33233: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:08:34,781::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:08:35,081::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0755278 s, 13.6 MB/s\n'; <rc> = 0
>Thread-15915758::INFO::2013-11-20 04:08:36,215::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915758::DEBUG::2013-11-20 04:08:36,216::task::495::TaskManager.Task::(_debug) Task 213a551e-542f-498d-a7e2-e9a0c8cb3c05: moving from state init -> state preparing
>Thread-15915758::DEBUG::2013-11-20 04:08:36,217::task::495::TaskManager.Task::(_debug) Task 213a551e-542f-498d-a7e2-e9a0c8cb3c05: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000391006469727', 'lastCheck': 1384920510.5792911, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0448658466339', 'lastCheck': 1384920510.6194861, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00133991241455', 'lastCheck': 1384920507.4178231, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00828194618225', 'lastCheck': 1384920514.7600851, 'valid': True, 'code': 0}}
>Thread-15915758::DEBUG::2013-11-20 04:08:36,218::task::495::TaskManager.Task::(_debug) Task 213a551e-542f-498d-a7e2-e9a0c8cb3c05: moving from state preparing -> state finished
>Thread-15915758::DEBUG::2013-11-20 04:08:36,219::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915758::DEBUG::2013-11-20 04:08:36,220::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915758::DEBUG::2013-11-20 04:08:36,220::task::495::TaskManager.Task::(_debug) Task 213a551e-542f-498d-a7e2-e9a0c8cb3c05: ref 0 aborting False
>Thread-15915758::INFO::2013-11-20 04:08:36,221::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000391006469727', 'lastCheck': 1384920510.5792911, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0448658466339', 'lastCheck': 1384920510.6194861, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00133991241455', 'lastCheck': 1384920507.4178231, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00828194618225', 'lastCheck': 1384920514.7600851, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:08:37,094::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:08:37,721::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0951922 s, 10.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:08:39,736::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:08:40,298::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0639891 s, 16.0 MB/s\n'; <rc> = 0
>Thread-15915761::DEBUG::2013-11-20 04:08:41,320::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915761::INFO::2013-11-20 04:08:41,322::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915761::DEBUG::2013-11-20 04:08:41,322::task::495::TaskManager.Task::(_debug) Task 8e50403e-9859-47f9-9406-c30d02a6c115: moving from state init -> state preparing
>Thread-15915761::DEBUG::2013-11-20 04:08:41,324::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915761::DEBUG::2013-11-20 04:08:41,325::task::495::TaskManager.Task::(_debug) Task 8e50403e-9859-47f9-9406-c30d02a6c115: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915761::DEBUG::2013-11-20 04:08:41,326::task::495::TaskManager.Task::(_debug) Task 8e50403e-9859-47f9-9406-c30d02a6c115: moving from state preparing -> state finished
>Thread-15915761::DEBUG::2013-11-20 04:08:41,326::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915761::DEBUG::2013-11-20 04:08:41,327::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915761::DEBUG::2013-11-20 04:08:41,328::task::495::TaskManager.Task::(_debug) Task 8e50403e-9859-47f9-9406-c30d02a6c115: ref 0 aborting False
>Thread-15915761::INFO::2013-11-20 04:08:41,329::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915762::DEBUG::2013-11-20 04:08:41,334::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915762::INFO::2013-11-20 04:08:41,335::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915762::DEBUG::2013-11-20 04:08:41,336::task::495::TaskManager.Task::(_debug) Task 99dd7081-d314-40dc-a541-d95b03ced861: moving from state init -> state preparing
>Thread-15915762::DEBUG::2013-11-20 04:08:41,337::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`22e56962-346a-4d44-961b-3ce2d0890cc9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915762::DEBUG::2013-11-20 04:08:41,338::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915762::DEBUG::2013-11-20 04:08:41,339::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915762::DEBUG::2013-11-20 04:08:41,340::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`22e56962-346a-4d44-961b-3ce2d0890cc9`::Granted request
>Thread-15915762::DEBUG::2013-11-20 04:08:41,341::task::495::TaskManager.Task::(_debug) Task 99dd7081-d314-40dc-a541-d95b03ced861: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915762::DEBUG::2013-11-20 04:08:41,342::task::495::TaskManager.Task::(_debug) Task 99dd7081-d314-40dc-a541-d95b03ced861: ref 1 aborting False
>Thread-15915762::WARNING::2013-11-20 04:08:41,344::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915762::DEBUG::2013-11-20 04:08:41,345::task::495::TaskManager.Task::(_debug) Task 99dd7081-d314-40dc-a541-d95b03ced861: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915762::DEBUG::2013-11-20 04:08:41,346::task::495::TaskManager.Task::(_debug) Task 99dd7081-d314-40dc-a541-d95b03ced861: moving from state preparing -> state finished
>Thread-15915762::DEBUG::2013-11-20 04:08:41,347::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915762::DEBUG::2013-11-20 04:08:41,348::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915762::DEBUG::2013-11-20 04:08:41,348::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915762::DEBUG::2013-11-20 04:08:41,349::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915762::DEBUG::2013-11-20 04:08:41,350::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915762::DEBUG::2013-11-20 04:08:41,350::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915762::DEBUG::2013-11-20 04:08:41,351::task::495::TaskManager.Task::(_debug) Task 99dd7081-d314-40dc-a541-d95b03ced861: ref 0 aborting False
>Thread-15915762::INFO::2013-11-20 04:08:41,352::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:08:42,313::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:08:42,801::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0487588 s, 21.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:08:44,813::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 04:08:45,011::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:08:45,016::task::495::TaskManager.Task::(_debug) Task 7183835a-e545-4711-b3f9-55df2c196651: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:08:45,019::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0c5eb06e-82eb-4438-88da-68f0f4043353`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:08:45,021::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:08:45,024::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:08:45,024::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0c5eb06e-82eb-4438-88da-68f0f4043353`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:08:45,026::task::495::TaskManager.Task::(_debug) Task 7183835a-e545-4711-b3f9-55df2c196651: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:08:45,027::task::495::TaskManager.Task::(_debug) Task 7183835a-e545-4711-b3f9-55df2c196651: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:08:45,028::task::495::TaskManager.Task::(_debug) Task 7183835a-e545-4711-b3f9-55df2c196651: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:08:45,028::task::495::TaskManager.Task::(_debug) Task 7183835a-e545-4711-b3f9-55df2c196651: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:08:45,029::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:08:45,030::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:08:45,031::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:08:45,032::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:08:45,032::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:08:45,033::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:08:45,034::task::495::TaskManager.Task::(_debug) Task 7183835a-e545-4711-b3f9-55df2c196651: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:08:45,034::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:08:45,229::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.046572 s, 22.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:08:47,239::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 04:08:47,584::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:08:47,585::task::495::TaskManager.Task::(_debug) Task 1cdcc5de-f6e5-477c-916e-8015a9e0d810: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:08:47,588::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ba2c70c7-9cdd-4bcd-9576-2be4e0558c0b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:08:47,589::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:08:47,590::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:08:47,591::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ba2c70c7-9cdd-4bcd-9576-2be4e0558c0b`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:08:47,592::task::495::TaskManager.Task::(_debug) Task 1cdcc5de-f6e5-477c-916e-8015a9e0d810: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:08:47,593::task::495::TaskManager.Task::(_debug) Task 1cdcc5de-f6e5-477c-916e-8015a9e0d810: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:08:47,594::task::495::TaskManager.Task::(_debug) Task 1cdcc5de-f6e5-477c-916e-8015a9e0d810: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:08:47,595::task::495::TaskManager.Task::(_debug) Task 1cdcc5de-f6e5-477c-916e-8015a9e0d810: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:08:47,598::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:08:47,599::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:08:47,601::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:08:47,602::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:08:47,603::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:08:47,603::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:08:47,604::task::495::TaskManager.Task::(_debug) Task 1cdcc5de-f6e5-477c-916e-8015a9e0d810: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:08:47,605::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:08:47,819::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0800847 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:08:49,832::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915085::INFO::2013-11-20 04:08:50,083::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=089b08fc-6544-4235-a061-fb55b09d2715 volUUID=66b599fc-b7b5-4934-a5d5-ef7283460029)
>Thread-15915085::DEBUG::2013-11-20 04:08:50,085::task::495::TaskManager.Task::(_debug) Task c9086a59-7559-4a96-a4bb-6aea82b5933e: moving from state init -> state preparing
>Thread-15915085::DEBUG::2013-11-20 04:08:50,089::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`62ee2655-b46a-4a6a-a6c0-8d4fe4b728d2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915085::DEBUG::2013-11-20 04:08:50,091::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915085::DEBUG::2013-11-20 04:08:50,092::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915085::DEBUG::2013-11-20 04:08:50,092::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`62ee2655-b46a-4a6a-a6c0-8d4fe4b728d2`::Granted request
>Thread-15915085::DEBUG::2013-11-20 04:08:50,093::task::495::TaskManager.Task::(_debug) Task c9086a59-7559-4a96-a4bb-6aea82b5933e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915085::DEBUG::2013-11-20 04:08:50,094::task::495::TaskManager.Task::(_debug) Task c9086a59-7559-4a96-a4bb-6aea82b5933e: ref 1 aborting False
>Thread-15915085::DEBUG::2013-11-20 04:08:50,095::task::495::TaskManager.Task::(_debug) Task c9086a59-7559-4a96-a4bb-6aea82b5933e: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15915085::DEBUG::2013-11-20 04:08:50,097::task::495::TaskManager.Task::(_debug) Task c9086a59-7559-4a96-a4bb-6aea82b5933e: moving from state preparing -> state finished
>Thread-15915085::DEBUG::2013-11-20 04:08:50,098::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915085::DEBUG::2013-11-20 04:08:50,098::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915085::DEBUG::2013-11-20 04:08:50,099::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915085::DEBUG::2013-11-20 04:08:50,100::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915085::DEBUG::2013-11-20 04:08:50,100::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915085::DEBUG::2013-11-20 04:08:50,101::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915085::DEBUG::2013-11-20 04:08:50,102::task::495::TaskManager.Task::(_debug) Task c9086a59-7559-4a96-a4bb-6aea82b5933e: ref 0 aborting False
>Thread-15915085::INFO::2013-11-20 04:08:50,102::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Dummy-15912651::DEBUG::2013-11-20 04:08:50,391::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0831539 s, 12.3 MB/s\n'; <rc> = 0
>Thread-15915766::DEBUG::2013-11-20 04:08:51,830::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915766::INFO::2013-11-20 04:08:51,832::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915766::DEBUG::2013-11-20 04:08:51,833::task::495::TaskManager.Task::(_debug) Task 5bc0e4ea-3d95-40a9-b5da-f7c5bd60c973: moving from state init -> state preparing
>Thread-15915766::DEBUG::2013-11-20 04:08:51,834::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915766::DEBUG::2013-11-20 04:08:51,834::task::495::TaskManager.Task::(_debug) Task 5bc0e4ea-3d95-40a9-b5da-f7c5bd60c973: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915766::DEBUG::2013-11-20 04:08:51,835::task::495::TaskManager.Task::(_debug) Task 5bc0e4ea-3d95-40a9-b5da-f7c5bd60c973: moving from state preparing -> state finished
>Thread-15915766::DEBUG::2013-11-20 04:08:51,836::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915766::DEBUG::2013-11-20 04:08:51,837::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915766::DEBUG::2013-11-20 04:08:51,838::task::495::TaskManager.Task::(_debug) Task 5bc0e4ea-3d95-40a9-b5da-f7c5bd60c973: ref 0 aborting False
>Thread-15915766::INFO::2013-11-20 04:08:51,838::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915767::DEBUG::2013-11-20 04:08:51,844::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915767::INFO::2013-11-20 04:08:51,845::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915767::DEBUG::2013-11-20 04:08:51,846::task::495::TaskManager.Task::(_debug) Task 939a0deb-edc5-4460-b416-4562a7b303af: moving from state init -> state preparing
>Thread-15915767::DEBUG::2013-11-20 04:08:51,848::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b6a5e931-30a7-45bc-bd3a-26d0fc0a5c62`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915767::DEBUG::2013-11-20 04:08:51,848::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915767::DEBUG::2013-11-20 04:08:51,849::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915767::DEBUG::2013-11-20 04:08:51,850::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b6a5e931-30a7-45bc-bd3a-26d0fc0a5c62`::Granted request
>Thread-15915767::DEBUG::2013-11-20 04:08:51,857::task::495::TaskManager.Task::(_debug) Task 939a0deb-edc5-4460-b416-4562a7b303af: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915767::DEBUG::2013-11-20 04:08:51,858::task::495::TaskManager.Task::(_debug) Task 939a0deb-edc5-4460-b416-4562a7b303af: ref 1 aborting False
>Thread-15915767::WARNING::2013-11-20 04:08:51,860::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915767::DEBUG::2013-11-20 04:08:51,861::task::495::TaskManager.Task::(_debug) Task 939a0deb-edc5-4460-b416-4562a7b303af: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915767::DEBUG::2013-11-20 04:08:51,862::task::495::TaskManager.Task::(_debug) Task 939a0deb-edc5-4460-b416-4562a7b303af: moving from state preparing -> state finished
>Thread-15915767::DEBUG::2013-11-20 04:08:51,863::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915767::DEBUG::2013-11-20 04:08:51,863::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915767::DEBUG::2013-11-20 04:08:51,864::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915767::DEBUG::2013-11-20 04:08:51,865::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915767::DEBUG::2013-11-20 04:08:51,866::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915767::DEBUG::2013-11-20 04:08:51,866::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915767::DEBUG::2013-11-20 04:08:51,867::task::495::TaskManager.Task::(_debug) Task 939a0deb-edc5-4460-b416-4562a7b303af: ref 0 aborting False
>Thread-15915767::INFO::2013-11-20 04:08:51,868::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915768::INFO::2013-11-20 04:08:52,285::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915768::DEBUG::2013-11-20 04:08:52,286::task::495::TaskManager.Task::(_debug) Task efbca73e-e2bf-4836-852e-b842bbd23ebd: moving from state init -> state preparing
>Thread-15915768::DEBUG::2013-11-20 04:08:52,287::task::495::TaskManager.Task::(_debug) Task efbca73e-e2bf-4836-852e-b842bbd23ebd: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00154113769531', 'lastCheck': 1384920531.0880311, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00270509719849', 'lastCheck': 1384920531.0895109, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00139904022217', 'lastCheck': 1384920528.0120361, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00951790809631', 'lastCheck': 1384920525.0150731, 'valid': True, 'code': 0}}
>Thread-15915768::DEBUG::2013-11-20 04:08:52,289::task::495::TaskManager.Task::(_debug) Task efbca73e-e2bf-4836-852e-b842bbd23ebd: moving from state preparing -> state finished
>Thread-15915768::DEBUG::2013-11-20 04:08:52,290::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915768::DEBUG::2013-11-20 04:08:52,291::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915768::DEBUG::2013-11-20 04:08:52,292::task::495::TaskManager.Task::(_debug) Task efbca73e-e2bf-4836-852e-b842bbd23ebd: ref 0 aborting False
>Thread-15915768::INFO::2013-11-20 04:08:52,293::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00154113769531', 'lastCheck': 1384920531.0880311, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00270509719849', 'lastCheck': 1384920531.0895109, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00139904022217', 'lastCheck': 1384920528.0120361, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00951790809631', 'lastCheck': 1384920525.0150731, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:08:52,404::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:08:52,971::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0917851 s, 11.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:08:54,986::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 04:08:55,228::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:08:55,231::task::495::TaskManager.Task::(_debug) Task 1510b0d0-534a-4bbe-83e4-fc29a9a629aa: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:08:55,233::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`34b7ccdd-488f-4da2-9bda-22788bb431f5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:08:55,234::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:08:55,235::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:08:55,235::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`34b7ccdd-488f-4da2-9bda-22788bb431f5`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:08:55,236::task::495::TaskManager.Task::(_debug) Task 1510b0d0-534a-4bbe-83e4-fc29a9a629aa: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:08:55,236::task::495::TaskManager.Task::(_debug) Task 1510b0d0-534a-4bbe-83e4-fc29a9a629aa: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:08:55,237::task::495::TaskManager.Task::(_debug) Task 1510b0d0-534a-4bbe-83e4-fc29a9a629aa: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:08:55,237::task::495::TaskManager.Task::(_debug) Task 1510b0d0-534a-4bbe-83e4-fc29a9a629aa: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:08:55,238::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:08:55,238::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:08:55,239::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:08:55,239::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:08:55,240::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:08:55,240::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:08:55,240::task::495::TaskManager.Task::(_debug) Task 1510b0d0-534a-4bbe-83e4-fc29a9a629aa: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:08:55,241::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:08:55,567::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0811315 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:08:57,580::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:08:58,116::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.081086 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:09:00,132::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 04:09:00,355::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:09:00,357::task::495::TaskManager.Task::(_debug) Task 4b20b4a8-d08b-4d64-9d6c-176400d59f33: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:09:00,359::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`38f38e06-d6d7-48f4-8422-c6b366643899`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:09:00,360::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:09:00,361::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:09:00,361::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`38f38e06-d6d7-48f4-8422-c6b366643899`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:09:00,363::task::495::TaskManager.Task::(_debug) Task 4b20b4a8-d08b-4d64-9d6c-176400d59f33: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:09:00,363::task::495::TaskManager.Task::(_debug) Task 4b20b4a8-d08b-4d64-9d6c-176400d59f33: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:09:00,364::task::495::TaskManager.Task::(_debug) Task 4b20b4a8-d08b-4d64-9d6c-176400d59f33: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:09:00,365::task::495::TaskManager.Task::(_debug) Task 4b20b4a8-d08b-4d64-9d6c-176400d59f33: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:09:00,365::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:09:00,365::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:09:00,366::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:09:00,367::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:09:00,367::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:09:00,368::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:09:00,369::task::495::TaskManager.Task::(_debug) Task 4b20b4a8-d08b-4d64-9d6c-176400d59f33: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:09:00,369::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:09:00,670::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0869667 s, 11.8 MB/s\n'; <rc> = 0
>Thread-15915773::DEBUG::2013-11-20 04:09:02,115::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915773::INFO::2013-11-20 04:09:02,117::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915773::DEBUG::2013-11-20 04:09:02,117::task::495::TaskManager.Task::(_debug) Task 4996b771-c14a-4bc2-9ee7-36a77b5af6f3: moving from state init -> state preparing
>Thread-15915773::DEBUG::2013-11-20 04:09:02,118::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915773::DEBUG::2013-11-20 04:09:02,119::task::495::TaskManager.Task::(_debug) Task 4996b771-c14a-4bc2-9ee7-36a77b5af6f3: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915773::DEBUG::2013-11-20 04:09:02,119::task::495::TaskManager.Task::(_debug) Task 4996b771-c14a-4bc2-9ee7-36a77b5af6f3: moving from state preparing -> state finished
>Thread-15915773::DEBUG::2013-11-20 04:09:02,120::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915773::DEBUG::2013-11-20 04:09:02,120::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915773::DEBUG::2013-11-20 04:09:02,121::task::495::TaskManager.Task::(_debug) Task 4996b771-c14a-4bc2-9ee7-36a77b5af6f3: ref 0 aborting False
>Thread-15915773::INFO::2013-11-20 04:09:02,121::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915774::DEBUG::2013-11-20 04:09:02,126::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915774::INFO::2013-11-20 04:09:02,128::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915774::DEBUG::2013-11-20 04:09:02,129::task::495::TaskManager.Task::(_debug) Task 493855f4-659e-46d0-a5a0-5c67bbda6909: moving from state init -> state preparing
>Thread-15915774::DEBUG::2013-11-20 04:09:02,130::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`590126dd-655b-407d-871c-0a73012c1fea`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915774::DEBUG::2013-11-20 04:09:02,131::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915774::DEBUG::2013-11-20 04:09:02,131::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915774::DEBUG::2013-11-20 04:09:02,132::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`590126dd-655b-407d-871c-0a73012c1fea`::Granted request
>Thread-15915774::DEBUG::2013-11-20 04:09:02,133::task::495::TaskManager.Task::(_debug) Task 493855f4-659e-46d0-a5a0-5c67bbda6909: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915774::DEBUG::2013-11-20 04:09:02,134::task::495::TaskManager.Task::(_debug) Task 493855f4-659e-46d0-a5a0-5c67bbda6909: ref 1 aborting False
>Thread-15915774::WARNING::2013-11-20 04:09:02,136::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915774::DEBUG::2013-11-20 04:09:02,137::task::495::TaskManager.Task::(_debug) Task 493855f4-659e-46d0-a5a0-5c67bbda6909: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915774::DEBUG::2013-11-20 04:09:02,145::task::495::TaskManager.Task::(_debug) Task 493855f4-659e-46d0-a5a0-5c67bbda6909: moving from state preparing -> state finished
>Thread-15915774::DEBUG::2013-11-20 04:09:02,146::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915774::DEBUG::2013-11-20 04:09:02,147::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915774::DEBUG::2013-11-20 04:09:02,148::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915774::DEBUG::2013-11-20 04:09:02,148::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915774::DEBUG::2013-11-20 04:09:02,149::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915774::DEBUG::2013-11-20 04:09:02,150::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915774::DEBUG::2013-11-20 04:09:02,151::task::495::TaskManager.Task::(_debug) Task 493855f4-659e-46d0-a5a0-5c67bbda6909: ref 0 aborting False
>Thread-15915774::INFO::2013-11-20 04:09:02,152::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:09:02,683::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 04:09:02,888::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:09:02,897::task::495::TaskManager.Task::(_debug) Task 8021d798-4882-41e0-8139-bddb38b82767: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:09:02,900::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ec4babf8-1c3e-4eae-9909-4d320765a806`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:09:02,901::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:09:02,902::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:09:02,903::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ec4babf8-1c3e-4eae-9909-4d320765a806`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:09:02,910::task::495::TaskManager.Task::(_debug) Task 8021d798-4882-41e0-8139-bddb38b82767: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 04:09:02,911::task::495::TaskManager.Task::(_debug) Task 8021d798-4882-41e0-8139-bddb38b82767: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:09:02,912::task::495::TaskManager.Task::(_debug) Task 8021d798-4882-41e0-8139-bddb38b82767: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 04:09:02,912::task::495::TaskManager.Task::(_debug) Task 8021d798-4882-41e0-8139-bddb38b82767: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:09:02,913::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:09:02,914::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:09:02,915::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 04:09:02,916::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:09:02,917::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:09:02,917::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:09:02,918::task::495::TaskManager.Task::(_debug) Task 8021d798-4882-41e0-8139-bddb38b82767: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 04:09:02,919::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 04:09:03,229::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0844378 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:09:05,244::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 04:09:05,504::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:09:05,517::task::495::TaskManager.Task::(_debug) Task 8090dc51-0aa7-4571-8965-6486d9686024: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:09:05,522::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`17fe5ea9-b4d5-4e68-9301-071c6f6c47cd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:09:05,523::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:09:05,527::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:09:05,528::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`17fe5ea9-b4d5-4e68-9301-071c6f6c47cd`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:09:05,529::task::495::TaskManager.Task::(_debug) Task 8090dc51-0aa7-4571-8965-6486d9686024: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:09:05,530::task::495::TaskManager.Task::(_debug) Task 8090dc51-0aa7-4571-8965-6486d9686024: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:09:05,531::task::495::TaskManager.Task::(_debug) Task 8090dc51-0aa7-4571-8965-6486d9686024: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 04:09:05,532::task::495::TaskManager.Task::(_debug) Task 8090dc51-0aa7-4571-8965-6486d9686024: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:09:05,533::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:09:05,533::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:09:05,534::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:09:05,535::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:09:05,536::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:09:05,536::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:09:05,537::task::495::TaskManager.Task::(_debug) Task 8090dc51-0aa7-4571-8965-6486d9686024: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:09:05,538::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:09:05,538::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:09:05,539::task::495::TaskManager.Task::(_debug) Task b87b6d46-e561-4190-8b18-686a4ee35809: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:09:05,540::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`987c2c15-1c2b-4116-afc7-245f5dee22a4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:09:05,541::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:09:05,541::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:09:05,542::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`987c2c15-1c2b-4116-afc7-245f5dee22a4`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:09:05,543::task::495::TaskManager.Task::(_debug) Task b87b6d46-e561-4190-8b18-686a4ee35809: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:09:05,543::task::495::TaskManager.Task::(_debug) Task b87b6d46-e561-4190-8b18-686a4ee35809: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:09:05,544::task::495::TaskManager.Task::(_debug) Task b87b6d46-e561-4190-8b18-686a4ee35809: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:09:05,544::task::495::TaskManager.Task::(_debug) Task b87b6d46-e561-4190-8b18-686a4ee35809: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:09:05,545::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:09:05,546::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:09:05,546::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:09:05,547::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:09:05,547::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:09:05,548::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:09:05,550::task::495::TaskManager.Task::(_debug) Task b87b6d46-e561-4190-8b18-686a4ee35809: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:09:05,550::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:09:05,552::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:09:05,553::task::495::TaskManager.Task::(_debug) Task f2f49132-0442-4d3e-906a-8976cfef2055: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:09:05,554::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`83b6df54-7e02-46f1-8c53-cbf6e9e38e8b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:09:05,555::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:09:05,556::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:09:05,556::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`83b6df54-7e02-46f1-8c53-cbf6e9e38e8b`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:09:05,557::task::495::TaskManager.Task::(_debug) Task f2f49132-0442-4d3e-906a-8976cfef2055: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:09:05,558::task::495::TaskManager.Task::(_debug) Task f2f49132-0442-4d3e-906a-8976cfef2055: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:09:05,559::task::495::TaskManager.Task::(_debug) Task f2f49132-0442-4d3e-906a-8976cfef2055: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:09:05,560::task::495::TaskManager.Task::(_debug) Task f2f49132-0442-4d3e-906a-8976cfef2055: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:09:05,560::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:09:05,561::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:09:05,561::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:09:05,562::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:09:05,562::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:09:05,563::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:09:05,563::task::495::TaskManager.Task::(_debug) Task f2f49132-0442-4d3e-906a-8976cfef2055: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:09:05,564::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:09:05,564::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:09:05,565::task::495::TaskManager.Task::(_debug) Task ec1e1df0-3ed3-4543-bcd5-d236c00f709c: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:09:05,566::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`18a4102c-6f63-4a0d-b464-80f9a57c36b6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:09:05,566::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:09:05,567::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:09:05,567::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`18a4102c-6f63-4a0d-b464-80f9a57c36b6`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:09:05,568::task::495::TaskManager.Task::(_debug) Task ec1e1df0-3ed3-4543-bcd5-d236c00f709c: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:09:05,569::task::495::TaskManager.Task::(_debug) Task ec1e1df0-3ed3-4543-bcd5-d236c00f709c: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:09:05,569::task::495::TaskManager.Task::(_debug) Task ec1e1df0-3ed3-4543-bcd5-d236c00f709c: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:09:05,570::task::495::TaskManager.Task::(_debug) Task ec1e1df0-3ed3-4543-bcd5-d236c00f709c: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:09:05,571::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:09:05,571::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:09:05,572::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:09:05,572::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:09:05,573::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:09:05,574::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:09:05,574::task::495::TaskManager.Task::(_debug) Task ec1e1df0-3ed3-4543-bcd5-d236c00f709c: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:09:05,575::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:09:05,865::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0797116 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:09:07,880::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915776::INFO::2013-11-20 04:09:08,114::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915776::DEBUG::2013-11-20 04:09:08,117::task::495::TaskManager.Task::(_debug) Task 9fcebddc-c765-467e-9dcd-792560c768c6: moving from state init -> state preparing
>Thread-15915776::DEBUG::2013-11-20 04:09:08,118::task::495::TaskManager.Task::(_debug) Task 9fcebddc-c765-467e-9dcd-792560c768c6: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000302076339722', 'lastCheck': 1384920541.3422811, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.021842956543', 'lastCheck': 1384920541.361582, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00181889533997', 'lastCheck': 1384920538.219975, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00265002250671', 'lastCheck': 1384920545.503619, 'valid': True, 'code': 0}}
>Thread-15915776::DEBUG::2013-11-20 04:09:08,124::task::495::TaskManager.Task::(_debug) Task 9fcebddc-c765-467e-9dcd-792560c768c6: moving from state preparing -> state finished
>Thread-15915776::DEBUG::2013-11-20 04:09:08,125::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915776::DEBUG::2013-11-20 04:09:08,125::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915776::DEBUG::2013-11-20 04:09:08,126::task::495::TaskManager.Task::(_debug) Task 9fcebddc-c765-467e-9dcd-792560c768c6: ref 0 aborting False
>Thread-15915776::INFO::2013-11-20 04:09:08,126::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000302076339722', 'lastCheck': 1384920541.3422811, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.021842956543', 'lastCheck': 1384920541.361582, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00181889533997', 'lastCheck': 1384920538.219975, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00265002250671', 'lastCheck': 1384920545.503619, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:09:08,444::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0757964 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:09:10,460::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:09:10,999::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.077499 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15915779::DEBUG::2013-11-20 04:09:12,485::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915779::INFO::2013-11-20 04:09:12,487::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915779::DEBUG::2013-11-20 04:09:12,487::task::495::TaskManager.Task::(_debug) Task 10279a3c-cccc-4b5b-b4c0-c85760c3c5b6: moving from state init -> state preparing
>Thread-15915779::DEBUG::2013-11-20 04:09:12,488::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915779::DEBUG::2013-11-20 04:09:12,489::task::495::TaskManager.Task::(_debug) Task 10279a3c-cccc-4b5b-b4c0-c85760c3c5b6: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915779::DEBUG::2013-11-20 04:09:12,490::task::495::TaskManager.Task::(_debug) Task 10279a3c-cccc-4b5b-b4c0-c85760c3c5b6: moving from state preparing -> state finished
>Thread-15915779::DEBUG::2013-11-20 04:09:12,491::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915779::DEBUG::2013-11-20 04:09:12,491::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915779::DEBUG::2013-11-20 04:09:12,492::task::495::TaskManager.Task::(_debug) Task 10279a3c-cccc-4b5b-b4c0-c85760c3c5b6: ref 0 aborting False
>Thread-15915779::INFO::2013-11-20 04:09:12,493::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915780::DEBUG::2013-11-20 04:09:12,508::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915780::INFO::2013-11-20 04:09:12,509::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915780::DEBUG::2013-11-20 04:09:12,510::task::495::TaskManager.Task::(_debug) Task 7b00b4d6-48ca-457e-bb51-f1ef067be0b8: moving from state init -> state preparing
>Thread-15915780::DEBUG::2013-11-20 04:09:12,511::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8693435f-cca2-4471-8fb6-c57f805aa9a9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915780::DEBUG::2013-11-20 04:09:12,512::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915780::DEBUG::2013-11-20 04:09:12,514::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915780::DEBUG::2013-11-20 04:09:12,514::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8693435f-cca2-4471-8fb6-c57f805aa9a9`::Granted request
>Thread-15915780::DEBUG::2013-11-20 04:09:12,516::task::495::TaskManager.Task::(_debug) Task 7b00b4d6-48ca-457e-bb51-f1ef067be0b8: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915780::DEBUG::2013-11-20 04:09:12,517::task::495::TaskManager.Task::(_debug) Task 7b00b4d6-48ca-457e-bb51-f1ef067be0b8: ref 1 aborting False
>Thread-15915780::WARNING::2013-11-20 04:09:12,522::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915780::DEBUG::2013-11-20 04:09:12,524::task::495::TaskManager.Task::(_debug) Task 7b00b4d6-48ca-457e-bb51-f1ef067be0b8: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915780::DEBUG::2013-11-20 04:09:12,526::task::495::TaskManager.Task::(_debug) Task 7b00b4d6-48ca-457e-bb51-f1ef067be0b8: moving from state preparing -> state finished
>Thread-15915780::DEBUG::2013-11-20 04:09:12,527::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915780::DEBUG::2013-11-20 04:09:12,529::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915780::DEBUG::2013-11-20 04:09:12,530::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915780::DEBUG::2013-11-20 04:09:12,532::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915780::DEBUG::2013-11-20 04:09:12,533::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915780::DEBUG::2013-11-20 04:09:12,534::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915780::DEBUG::2013-11-20 04:09:12,537::task::495::TaskManager.Task::(_debug) Task 7b00b4d6-48ca-457e-bb51-f1ef067be0b8: ref 0 aborting False
>Thread-15915780::INFO::2013-11-20 04:09:12,538::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:09:13,013::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:09:13,603::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0883198 s, 11.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:09:15,620::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:09:16,223::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0841586 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:09:18,241::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:09:18,858::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0916923 s, 11.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:09:20,881::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:09:20,884::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:09:20,894::task::495::TaskManager.Task::(_debug) Task 847f8803-d11f-4d22-933e-559755da6811: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:09:21,161::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`87de1331-d223-4968-ad7c-0edb6212223f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:09:21,164::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:09:21,166::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:09:21,167::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`87de1331-d223-4968-ad7c-0edb6212223f`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:09:21,169::task::495::TaskManager.Task::(_debug) Task 847f8803-d11f-4d22-933e-559755da6811: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:09:21,170::task::495::TaskManager.Task::(_debug) Task 847f8803-d11f-4d22-933e-559755da6811: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:09:21,171::task::495::TaskManager.Task::(_debug) Task 847f8803-d11f-4d22-933e-559755da6811: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:09:21,172::task::495::TaskManager.Task::(_debug) Task 847f8803-d11f-4d22-933e-559755da6811: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:09:21,173::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:09:21,174::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:09:21,175::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:09:21,176::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:09:21,177::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:09:21,177::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:09:21,178::task::495::TaskManager.Task::(_debug) Task 847f8803-d11f-4d22-933e-559755da6811: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:09:21,179::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:09:21,478::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0913692 s, 11.2 MB/s\n'; <rc> = 0
>Thread-15915784::DEBUG::2013-11-20 04:09:22,843::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915784::INFO::2013-11-20 04:09:22,845::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915784::DEBUG::2013-11-20 04:09:22,846::task::495::TaskManager.Task::(_debug) Task e524832d-2ee7-4fe4-8e38-d2ac2d8a42bb: moving from state init -> state preparing
>Thread-15915784::DEBUG::2013-11-20 04:09:22,847::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915784::DEBUG::2013-11-20 04:09:22,848::task::495::TaskManager.Task::(_debug) Task e524832d-2ee7-4fe4-8e38-d2ac2d8a42bb: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915784::DEBUG::2013-11-20 04:09:22,849::task::495::TaskManager.Task::(_debug) Task e524832d-2ee7-4fe4-8e38-d2ac2d8a42bb: moving from state preparing -> state finished
>Thread-15915784::DEBUG::2013-11-20 04:09:22,850::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915784::DEBUG::2013-11-20 04:09:22,851::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915784::DEBUG::2013-11-20 04:09:22,851::task::495::TaskManager.Task::(_debug) Task e524832d-2ee7-4fe4-8e38-d2ac2d8a42bb: ref 0 aborting False
>Thread-15915784::INFO::2013-11-20 04:09:22,852::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915785::DEBUG::2013-11-20 04:09:22,857::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915785::INFO::2013-11-20 04:09:22,858::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915785::DEBUG::2013-11-20 04:09:22,859::task::495::TaskManager.Task::(_debug) Task ef58b1ba-74e7-4aa3-a7e1-34a714adbfc1: moving from state init -> state preparing
>Thread-15915785::DEBUG::2013-11-20 04:09:22,860::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`fce62863-9b62-4052-9cdb-2fdfd0550b1e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915785::DEBUG::2013-11-20 04:09:22,861::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915785::DEBUG::2013-11-20 04:09:22,862::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915785::DEBUG::2013-11-20 04:09:22,863::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`fce62863-9b62-4052-9cdb-2fdfd0550b1e`::Granted request
>Thread-15915785::DEBUG::2013-11-20 04:09:22,864::task::495::TaskManager.Task::(_debug) Task ef58b1ba-74e7-4aa3-a7e1-34a714adbfc1: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915785::DEBUG::2013-11-20 04:09:22,865::task::495::TaskManager.Task::(_debug) Task ef58b1ba-74e7-4aa3-a7e1-34a714adbfc1: ref 1 aborting False
>Thread-15915785::WARNING::2013-11-20 04:09:22,869::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915785::DEBUG::2013-11-20 04:09:22,870::task::495::TaskManager.Task::(_debug) Task ef58b1ba-74e7-4aa3-a7e1-34a714adbfc1: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915785::DEBUG::2013-11-20 04:09:22,871::task::495::TaskManager.Task::(_debug) Task ef58b1ba-74e7-4aa3-a7e1-34a714adbfc1: moving from state preparing -> state finished
>Thread-15915785::DEBUG::2013-11-20 04:09:22,872::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915785::DEBUG::2013-11-20 04:09:22,873::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915785::DEBUG::2013-11-20 04:09:22,874::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915785::DEBUG::2013-11-20 04:09:22,875::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915785::DEBUG::2013-11-20 04:09:22,876::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915785::DEBUG::2013-11-20 04:09:22,877::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915785::DEBUG::2013-11-20 04:09:22,878::task::495::TaskManager.Task::(_debug) Task ef58b1ba-74e7-4aa3-a7e1-34a714adbfc1: ref 0 aborting False
>Thread-15915785::INFO::2013-11-20 04:09:22,879::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:09:23,497::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915786::INFO::2013-11-20 04:09:23,883::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915786::DEBUG::2013-11-20 04:09:23,885::task::495::TaskManager.Task::(_debug) Task 115bd77a-3a62-4efc-92ad-d373c25394ec: moving from state init -> state preparing
>Thread-15915786::DEBUG::2013-11-20 04:09:23,886::task::495::TaskManager.Task::(_debug) Task 115bd77a-3a62-4efc-92ad-d373c25394ec: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000396013259888', 'lastCheck': 1384920561.8698521, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00159502029419', 'lastCheck': 1384920561.8703561, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00125789642334', 'lastCheck': 1384920558.884145, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00441193580627', 'lastCheck': 1384920555.8671911, 'valid': True, 'code': 0}}
>Thread-15915786::DEBUG::2013-11-20 04:09:23,887::task::495::TaskManager.Task::(_debug) Task 115bd77a-3a62-4efc-92ad-d373c25394ec: moving from state preparing -> state finished
>Thread-15915786::DEBUG::2013-11-20 04:09:23,887::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915786::DEBUG::2013-11-20 04:09:23,888::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915786::DEBUG::2013-11-20 04:09:23,889::task::495::TaskManager.Task::(_debug) Task 115bd77a-3a62-4efc-92ad-d373c25394ec: ref 0 aborting False
>Thread-15915786::INFO::2013-11-20 04:09:23,890::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000396013259888', 'lastCheck': 1384920561.8698521, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00159502029419', 'lastCheck': 1384920561.8703561, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00125789642334', 'lastCheck': 1384920558.884145, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00441193580627', 'lastCheck': 1384920555.8671911, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:09:24,105::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.102187 s, 10.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:09:26,129::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913518::INFO::2013-11-20 04:09:26,394::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:09:26,395::task::495::TaskManager.Task::(_debug) Task f44081e6-6ee0-4083-b081-0a83176a835f: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:09:26,398::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`90651077-9a1b-4002-b7a0-00446874aed6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:09:26,399::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:09:26,400::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:09:26,400::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`90651077-9a1b-4002-b7a0-00446874aed6`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:09:26,402::task::495::TaskManager.Task::(_debug) Task f44081e6-6ee0-4083-b081-0a83176a835f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:09:26,402::task::495::TaskManager.Task::(_debug) Task f44081e6-6ee0-4083-b081-0a83176a835f: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:09:26,403::task::495::TaskManager.Task::(_debug) Task f44081e6-6ee0-4083-b081-0a83176a835f: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:09:26,404::task::495::TaskManager.Task::(_debug) Task f44081e6-6ee0-4083-b081-0a83176a835f: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:09:26,405::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:09:26,406::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:09:26,407::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:09:26,408::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:09:26,409::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:09:26,409::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:09:26,410::task::495::TaskManager.Task::(_debug) Task f44081e6-6ee0-4083-b081-0a83176a835f: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:09:26,411::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:09:26,412::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:09:26,413::task::495::TaskManager.Task::(_debug) Task 62eb7b20-8282-468b-9616-c76e16d0e5bd: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:09:26,415::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`53c73b5d-55fd-4a81-9074-b4aed07baedb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:09:26,415::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:09:26,416::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:09:26,417::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`53c73b5d-55fd-4a81-9074-b4aed07baedb`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:09:26,418::task::495::TaskManager.Task::(_debug) Task 62eb7b20-8282-468b-9616-c76e16d0e5bd: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:09:26,418::task::495::TaskManager.Task::(_debug) Task 62eb7b20-8282-468b-9616-c76e16d0e5bd: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:09:26,419::task::495::TaskManager.Task::(_debug) Task 62eb7b20-8282-468b-9616-c76e16d0e5bd: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:09:26,420::task::495::TaskManager.Task::(_debug) Task 62eb7b20-8282-468b-9616-c76e16d0e5bd: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:09:26,420::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:09:26,421::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:09:26,422::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:09:26,423::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:09:26,423::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:09:26,424::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:09:26,425::task::495::TaskManager.Task::(_debug) Task 62eb7b20-8282-468b-9616-c76e16d0e5bd: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:09:26,425::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:09:26,691::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0770913 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:09:28,710::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:09:29,249::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0809866 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:09:31,266::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:09:31,816::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0881078 s, 11.6 MB/s\n'; <rc> = 0
>Thread-15915790::DEBUG::2013-11-20 04:09:33,163::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915790::INFO::2013-11-20 04:09:33,167::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915790::DEBUG::2013-11-20 04:09:33,169::task::495::TaskManager.Task::(_debug) Task 6e67d3ab-4435-4750-9ff1-3ff02e30c736: moving from state init -> state preparing
>Thread-15915790::DEBUG::2013-11-20 04:09:33,172::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915790::DEBUG::2013-11-20 04:09:33,175::task::495::TaskManager.Task::(_debug) Task 6e67d3ab-4435-4750-9ff1-3ff02e30c736: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915790::DEBUG::2013-11-20 04:09:33,177::task::495::TaskManager.Task::(_debug) Task 6e67d3ab-4435-4750-9ff1-3ff02e30c736: moving from state preparing -> state finished
>Thread-15915790::DEBUG::2013-11-20 04:09:33,178::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915790::DEBUG::2013-11-20 04:09:33,178::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915790::DEBUG::2013-11-20 04:09:33,179::task::495::TaskManager.Task::(_debug) Task 6e67d3ab-4435-4750-9ff1-3ff02e30c736: ref 0 aborting False
>Thread-15915790::INFO::2013-11-20 04:09:33,180::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915791::DEBUG::2013-11-20 04:09:33,185::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915791::INFO::2013-11-20 04:09:33,186::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915791::DEBUG::2013-11-20 04:09:33,187::task::495::TaskManager.Task::(_debug) Task 481ce5b9-602b-4373-92c8-ebe7e0a26a72: moving from state init -> state preparing
>Thread-15915791::DEBUG::2013-11-20 04:09:33,189::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6a281218-28a9-4c2e-97a8-dc6a0fe63494`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915791::DEBUG::2013-11-20 04:09:33,190::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915791::DEBUG::2013-11-20 04:09:33,191::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915791::DEBUG::2013-11-20 04:09:33,191::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6a281218-28a9-4c2e-97a8-dc6a0fe63494`::Granted request
>Thread-15915791::DEBUG::2013-11-20 04:09:33,193::task::495::TaskManager.Task::(_debug) Task 481ce5b9-602b-4373-92c8-ebe7e0a26a72: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915791::DEBUG::2013-11-20 04:09:33,193::task::495::TaskManager.Task::(_debug) Task 481ce5b9-602b-4373-92c8-ebe7e0a26a72: ref 1 aborting False
>Thread-15915791::WARNING::2013-11-20 04:09:33,196::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915791::DEBUG::2013-11-20 04:09:33,197::task::495::TaskManager.Task::(_debug) Task 481ce5b9-602b-4373-92c8-ebe7e0a26a72: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915791::DEBUG::2013-11-20 04:09:33,198::task::495::TaskManager.Task::(_debug) Task 481ce5b9-602b-4373-92c8-ebe7e0a26a72: moving from state preparing -> state finished
>Thread-15915791::DEBUG::2013-11-20 04:09:33,199::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915791::DEBUG::2013-11-20 04:09:33,205::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915791::DEBUG::2013-11-20 04:09:33,205::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915791::DEBUG::2013-11-20 04:09:33,206::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915791::DEBUG::2013-11-20 04:09:33,207::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915791::DEBUG::2013-11-20 04:09:33,208::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915791::DEBUG::2013-11-20 04:09:33,209::task::495::TaskManager.Task::(_debug) Task 481ce5b9-602b-4373-92c8-ebe7e0a26a72: ref 0 aborting False
>Thread-15915791::INFO::2013-11-20 04:09:33,209::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:09:33,829::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:09:34,379::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0945933 s, 10.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:09:36,395::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 04:09:36,654::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:09:36,666::task::495::TaskManager.Task::(_debug) Task a95403be-0427-40c5-bf89-a1e9b1ea7a76: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:09:36,678::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`157cf9a6-7259-421b-b66d-d888a4eed230`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:09:36,680::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:09:36,681::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:09:36,682::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`157cf9a6-7259-421b-b66d-d888a4eed230`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:09:36,685::task::495::TaskManager.Task::(_debug) Task a95403be-0427-40c5-bf89-a1e9b1ea7a76: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:09:36,686::task::495::TaskManager.Task::(_debug) Task a95403be-0427-40c5-bf89-a1e9b1ea7a76: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:09:36,687::task::495::TaskManager.Task::(_debug) Task a95403be-0427-40c5-bf89-a1e9b1ea7a76: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:09:36,687::task::495::TaskManager.Task::(_debug) Task a95403be-0427-40c5-bf89-a1e9b1ea7a76: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:09:36,688::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:09:36,689::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:09:36,690::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:09:36,691::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:09:36,691::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:09:36,692::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:09:36,693::task::495::TaskManager.Task::(_debug) Task a95403be-0427-40c5-bf89-a1e9b1ea7a76: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:09:36,694::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:09:36,966::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0911338 s, 11.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:09:38,983::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:09:39,522::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0786935 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15915794::INFO::2013-11-20 04:09:39,694::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915794::DEBUG::2013-11-20 04:09:39,696::task::495::TaskManager.Task::(_debug) Task d2888d1d-b9d6-4dde-b632-7902de6d19a8: moving from state init -> state preparing
>Thread-15915794::DEBUG::2013-11-20 04:09:39,697::task::495::TaskManager.Task::(_debug) Task d2888d1d-b9d6-4dde-b632-7902de6d19a8: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00052285194397', 'lastCheck': 1384920571.917428, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0108530521393', 'lastCheck': 1384920571.9236, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00126791000366', 'lastCheck': 1384920579.2692411, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000363826751709', 'lastCheck': 1384920576.0245931, 'valid': True, 'code': 0}}
>Thread-15915794::DEBUG::2013-11-20 04:09:39,698::task::495::TaskManager.Task::(_debug) Task d2888d1d-b9d6-4dde-b632-7902de6d19a8: moving from state preparing -> state finished
>Thread-15915794::DEBUG::2013-11-20 04:09:39,699::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915794::DEBUG::2013-11-20 04:09:39,699::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915794::DEBUG::2013-11-20 04:09:39,700::task::495::TaskManager.Task::(_debug) Task d2888d1d-b9d6-4dde-b632-7902de6d19a8: ref 0 aborting False
>Thread-15915794::INFO::2013-11-20 04:09:39,700::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00052285194397', 'lastCheck': 1384920571.917428, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0108530521393', 'lastCheck': 1384920571.9236, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00126791000366', 'lastCheck': 1384920579.2692411, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000363826751709', 'lastCheck': 1384920576.0245931, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:09:41,538::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:09:42,110::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0887092 s, 11.5 MB/s\n'; <rc> = 0
>Thread-15915797::DEBUG::2013-11-20 04:09:43,565::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915797::INFO::2013-11-20 04:09:43,566::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915797::DEBUG::2013-11-20 04:09:43,567::task::495::TaskManager.Task::(_debug) Task e99051b4-1bd8-4029-86d7-ecb1c4083679: moving from state init -> state preparing
>Thread-15915797::DEBUG::2013-11-20 04:09:43,568::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915797::DEBUG::2013-11-20 04:09:43,569::task::495::TaskManager.Task::(_debug) Task e99051b4-1bd8-4029-86d7-ecb1c4083679: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915797::DEBUG::2013-11-20 04:09:43,569::task::495::TaskManager.Task::(_debug) Task e99051b4-1bd8-4029-86d7-ecb1c4083679: moving from state preparing -> state finished
>Thread-15915797::DEBUG::2013-11-20 04:09:43,570::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915797::DEBUG::2013-11-20 04:09:43,571::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915797::DEBUG::2013-11-20 04:09:43,571::task::495::TaskManager.Task::(_debug) Task e99051b4-1bd8-4029-86d7-ecb1c4083679: ref 0 aborting False
>Thread-15915797::INFO::2013-11-20 04:09:43,572::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915798::DEBUG::2013-11-20 04:09:43,581::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915798::INFO::2013-11-20 04:09:43,582::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915798::DEBUG::2013-11-20 04:09:43,582::task::495::TaskManager.Task::(_debug) Task 4beae09d-5f1f-46e0-a3f6-afea14250bea: moving from state init -> state preparing
>Thread-15915798::DEBUG::2013-11-20 04:09:43,583::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9bc328e1-76bb-4437-8906-ede0390a36ff`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915798::DEBUG::2013-11-20 04:09:43,584::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915798::DEBUG::2013-11-20 04:09:43,585::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915798::DEBUG::2013-11-20 04:09:43,586::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9bc328e1-76bb-4437-8906-ede0390a36ff`::Granted request
>Thread-15915798::DEBUG::2013-11-20 04:09:43,587::task::495::TaskManager.Task::(_debug) Task 4beae09d-5f1f-46e0-a3f6-afea14250bea: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915798::DEBUG::2013-11-20 04:09:43,587::task::495::TaskManager.Task::(_debug) Task 4beae09d-5f1f-46e0-a3f6-afea14250bea: ref 1 aborting False
>Thread-15915798::WARNING::2013-11-20 04:09:43,589::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915798::DEBUG::2013-11-20 04:09:43,590::task::495::TaskManager.Task::(_debug) Task 4beae09d-5f1f-46e0-a3f6-afea14250bea: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915798::DEBUG::2013-11-20 04:09:43,591::task::495::TaskManager.Task::(_debug) Task 4beae09d-5f1f-46e0-a3f6-afea14250bea: moving from state preparing -> state finished
>Thread-15915798::DEBUG::2013-11-20 04:09:43,592::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915798::DEBUG::2013-11-20 04:09:43,593::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915798::DEBUG::2013-11-20 04:09:43,593::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915798::DEBUG::2013-11-20 04:09:43,594::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915798::DEBUG::2013-11-20 04:09:43,595::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915798::DEBUG::2013-11-20 04:09:43,595::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915798::DEBUG::2013-11-20 04:09:43,596::task::495::TaskManager.Task::(_debug) Task 4beae09d-5f1f-46e0-a3f6-afea14250bea: ref 0 aborting False
>Thread-15915798::INFO::2013-11-20 04:09:43,597::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:09:44,126::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:09:44,727::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0908557 s, 11.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:09:46,746::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 04:09:46,751::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:09:47,004::task::495::TaskManager.Task::(_debug) Task 6ac83c2f-9347-4eaf-a72c-13ae63c7f934: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:09:47,011::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`84230414-dc20-45db-95f4-fa0fdc1d3f24`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:09:47,012::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:09:47,013::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:09:47,014::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`84230414-dc20-45db-95f4-fa0fdc1d3f24`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:09:47,016::task::495::TaskManager.Task::(_debug) Task 6ac83c2f-9347-4eaf-a72c-13ae63c7f934: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:09:47,016::task::495::TaskManager.Task::(_debug) Task 6ac83c2f-9347-4eaf-a72c-13ae63c7f934: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:09:47,018::task::495::TaskManager.Task::(_debug) Task 6ac83c2f-9347-4eaf-a72c-13ae63c7f934: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:09:47,018::task::495::TaskManager.Task::(_debug) Task 6ac83c2f-9347-4eaf-a72c-13ae63c7f934: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:09:47,020::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:09:47,021::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:09:47,022::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:09:47,022::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:09:47,023::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:09:47,023::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:09:47,024::task::495::TaskManager.Task::(_debug) Task 6ac83c2f-9347-4eaf-a72c-13ae63c7f934: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:09:47,025::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:09:47,341::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0761719 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:09:49,355::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 04:09:49,596::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:09:49,598::task::495::TaskManager.Task::(_debug) Task f43f14e9-6e4d-4e6e-9388-faef7a55abd4: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:09:49,600::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0b6b5b17-c58e-4e6e-973f-d59eafb4cf1a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:09:49,601::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:09:49,602::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:09:49,602::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0b6b5b17-c58e-4e6e-973f-d59eafb4cf1a`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:09:49,604::task::495::TaskManager.Task::(_debug) Task f43f14e9-6e4d-4e6e-9388-faef7a55abd4: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:09:49,604::task::495::TaskManager.Task::(_debug) Task f43f14e9-6e4d-4e6e-9388-faef7a55abd4: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:09:49,605::task::495::TaskManager.Task::(_debug) Task f43f14e9-6e4d-4e6e-9388-faef7a55abd4: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:09:49,606::task::495::TaskManager.Task::(_debug) Task f43f14e9-6e4d-4e6e-9388-faef7a55abd4: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:09:49,606::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:09:49,607::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:09:49,608::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:09:49,608::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:09:49,609::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:09:49,609::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:09:49,610::task::495::TaskManager.Task::(_debug) Task f43f14e9-6e4d-4e6e-9388-faef7a55abd4: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:09:49,610::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:09:49,908::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0812696 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:09:51,923::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915085::INFO::2013-11-20 04:09:52,136::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=089b08fc-6544-4235-a061-fb55b09d2715 volUUID=66b599fc-b7b5-4934-a5d5-ef7283460029)
>Thread-15915085::DEBUG::2013-11-20 04:09:52,138::task::495::TaskManager.Task::(_debug) Task b1326845-b3f8-4e58-a02a-6c47e53c25c3: moving from state init -> state preparing
>Thread-15915085::DEBUG::2013-11-20 04:09:52,143::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`cb4898ad-8c4d-480d-8ac9-fe7a4ea9b1fc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915085::DEBUG::2013-11-20 04:09:52,145::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915085::DEBUG::2013-11-20 04:09:52,147::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915085::DEBUG::2013-11-20 04:09:52,148::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`cb4898ad-8c4d-480d-8ac9-fe7a4ea9b1fc`::Granted request
>Thread-15915085::DEBUG::2013-11-20 04:09:52,150::task::495::TaskManager.Task::(_debug) Task b1326845-b3f8-4e58-a02a-6c47e53c25c3: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915085::DEBUG::2013-11-20 04:09:52,151::task::495::TaskManager.Task::(_debug) Task b1326845-b3f8-4e58-a02a-6c47e53c25c3: ref 1 aborting False
>Thread-15915085::DEBUG::2013-11-20 04:09:52,152::task::495::TaskManager.Task::(_debug) Task b1326845-b3f8-4e58-a02a-6c47e53c25c3: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15915085::DEBUG::2013-11-20 04:09:52,153::task::495::TaskManager.Task::(_debug) Task b1326845-b3f8-4e58-a02a-6c47e53c25c3: moving from state preparing -> state finished
>Thread-15915085::DEBUG::2013-11-20 04:09:52,154::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915085::DEBUG::2013-11-20 04:09:52,155::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915085::DEBUG::2013-11-20 04:09:52,156::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915085::DEBUG::2013-11-20 04:09:52,157::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915085::DEBUG::2013-11-20 04:09:52,158::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915085::DEBUG::2013-11-20 04:09:52,159::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915085::DEBUG::2013-11-20 04:09:52,160::task::495::TaskManager.Task::(_debug) Task b1326845-b3f8-4e58-a02a-6c47e53c25c3: ref 0 aborting False
>Thread-15915085::INFO::2013-11-20 04:09:52,160::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Dummy-15912651::DEBUG::2013-11-20 04:09:52,475::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.085099 s, 12.0 MB/s\n'; <rc> = 0
>Thread-15915802::DEBUG::2013-11-20 04:09:53,978::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915802::INFO::2013-11-20 04:09:53,980::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915802::DEBUG::2013-11-20 04:09:53,980::task::495::TaskManager.Task::(_debug) Task 760e1383-4d89-4e24-b1ae-5cf013dcbb99: moving from state init -> state preparing
>Thread-15915802::DEBUG::2013-11-20 04:09:53,982::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915802::DEBUG::2013-11-20 04:09:53,982::task::495::TaskManager.Task::(_debug) Task 760e1383-4d89-4e24-b1ae-5cf013dcbb99: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915802::DEBUG::2013-11-20 04:09:53,983::task::495::TaskManager.Task::(_debug) Task 760e1383-4d89-4e24-b1ae-5cf013dcbb99: moving from state preparing -> state finished
>Thread-15915802::DEBUG::2013-11-20 04:09:53,984::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915802::DEBUG::2013-11-20 04:09:53,985::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915802::DEBUG::2013-11-20 04:09:53,985::task::495::TaskManager.Task::(_debug) Task 760e1383-4d89-4e24-b1ae-5cf013dcbb99: ref 0 aborting False
>Thread-15915802::INFO::2013-11-20 04:09:53,986::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915803::DEBUG::2013-11-20 04:09:53,994::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915803::INFO::2013-11-20 04:09:53,995::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915803::DEBUG::2013-11-20 04:09:53,996::task::495::TaskManager.Task::(_debug) Task 1f6d82d8-3e16-4831-b884-09849919f8dd: moving from state init -> state preparing
>Thread-15915803::DEBUG::2013-11-20 04:09:53,998::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`02de2082-8a4b-4519-b6ff-4f01e2f9c159`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915803::DEBUG::2013-11-20 04:09:53,998::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915803::DEBUG::2013-11-20 04:09:53,999::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915803::DEBUG::2013-11-20 04:09:54,000::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`02de2082-8a4b-4519-b6ff-4f01e2f9c159`::Granted request
>Thread-15915803::DEBUG::2013-11-20 04:09:54,001::task::495::TaskManager.Task::(_debug) Task 1f6d82d8-3e16-4831-b884-09849919f8dd: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915803::DEBUG::2013-11-20 04:09:54,002::task::495::TaskManager.Task::(_debug) Task 1f6d82d8-3e16-4831-b884-09849919f8dd: ref 1 aborting False
>Thread-15915803::WARNING::2013-11-20 04:09:54,006::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915803::DEBUG::2013-11-20 04:09:54,007::task::495::TaskManager.Task::(_debug) Task 1f6d82d8-3e16-4831-b884-09849919f8dd: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915803::DEBUG::2013-11-20 04:09:54,008::task::495::TaskManager.Task::(_debug) Task 1f6d82d8-3e16-4831-b884-09849919f8dd: moving from state preparing -> state finished
>Thread-15915803::DEBUG::2013-11-20 04:09:54,009::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915803::DEBUG::2013-11-20 04:09:54,009::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915803::DEBUG::2013-11-20 04:09:54,010::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915803::DEBUG::2013-11-20 04:09:54,011::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915803::DEBUG::2013-11-20 04:09:54,012::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915803::DEBUG::2013-11-20 04:09:54,012::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915803::DEBUG::2013-11-20 04:09:54,013::task::495::TaskManager.Task::(_debug) Task 1f6d82d8-3e16-4831-b884-09849919f8dd: ref 0 aborting False
>Thread-15915803::INFO::2013-11-20 04:09:54,014::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:09:54,489::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:09:55,086::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0800807 s, 12.8 MB/s\n'; <rc> = 0
>Thread-15915804::INFO::2013-11-20 04:09:55,259::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915804::DEBUG::2013-11-20 04:09:55,261::task::495::TaskManager.Task::(_debug) Task 7ee88136-c979-40bb-8687-35116b9c7859: moving from state init -> state preparing
>Thread-15915804::DEBUG::2013-11-20 04:09:55,262::task::495::TaskManager.Task::(_debug) Task 7ee88136-c979-40bb-8687-35116b9c7859: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000645875930786', 'lastCheck': 1384920592.3529079, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00171685218811', 'lastCheck': 1384920592.353209, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00137996673584', 'lastCheck': 1384920589.350733, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000334978103638', 'lastCheck': 1384920586.3397939, 'valid': True, 'code': 0}}
>Thread-15915804::DEBUG::2013-11-20 04:09:55,263::task::495::TaskManager.Task::(_debug) Task 7ee88136-c979-40bb-8687-35116b9c7859: moving from state preparing -> state finished
>Thread-15915804::DEBUG::2013-11-20 04:09:55,264::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915804::DEBUG::2013-11-20 04:09:55,264::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915804::DEBUG::2013-11-20 04:09:55,265::task::495::TaskManager.Task::(_debug) Task 7ee88136-c979-40bb-8687-35116b9c7859: ref 0 aborting False
>Thread-15915804::INFO::2013-11-20 04:09:55,266::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000645875930786', 'lastCheck': 1384920592.3529079, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00171685218811', 'lastCheck': 1384920592.353209, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00137996673584', 'lastCheck': 1384920589.350733, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000334978103638', 'lastCheck': 1384920586.3397939, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:09:57,101::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 04:09:57,391::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:09:57,394::task::495::TaskManager.Task::(_debug) Task 7ac2552b-ab34-414d-b8e4-76258aa5eee5: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:09:57,397::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bc57e485-1d64-4f77-a63f-02c2f1be9a1c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:09:57,397::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:09:57,399::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:09:57,400::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bc57e485-1d64-4f77-a63f-02c2f1be9a1c`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:09:57,401::task::495::TaskManager.Task::(_debug) Task 7ac2552b-ab34-414d-b8e4-76258aa5eee5: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:09:57,402::task::495::TaskManager.Task::(_debug) Task 7ac2552b-ab34-414d-b8e4-76258aa5eee5: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:09:57,403::task::495::TaskManager.Task::(_debug) Task 7ac2552b-ab34-414d-b8e4-76258aa5eee5: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:09:57,404::task::495::TaskManager.Task::(_debug) Task 7ac2552b-ab34-414d-b8e4-76258aa5eee5: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:09:57,405::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:09:57,406::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:09:57,406::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:09:57,407::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:09:57,407::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:09:57,408::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:09:57,409::task::495::TaskManager.Task::(_debug) Task 7ac2552b-ab34-414d-b8e4-76258aa5eee5: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:09:57,410::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:09:57,702::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0785298 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:09:59,712::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:10:00,277::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0956034 s, 10.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:10:02,293::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 04:10:02,532::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:10:02,533::task::495::TaskManager.Task::(_debug) Task 1f4b4412-5ee9-4c74-a948-cdce840ff6b0: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:10:02,544::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`56d18814-ff3b-4fbf-8826-ba66c1ad0f8c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:10:02,545::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:10:02,546::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:10:02,547::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`56d18814-ff3b-4fbf-8826-ba66c1ad0f8c`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:10:02,548::task::495::TaskManager.Task::(_debug) Task 1f4b4412-5ee9-4c74-a948-cdce840ff6b0: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:10:02,549::task::495::TaskManager.Task::(_debug) Task 1f4b4412-5ee9-4c74-a948-cdce840ff6b0: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:10:02,549::task::495::TaskManager.Task::(_debug) Task 1f4b4412-5ee9-4c74-a948-cdce840ff6b0: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:10:02,550::task::495::TaskManager.Task::(_debug) Task 1f4b4412-5ee9-4c74-a948-cdce840ff6b0: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:10:02,551::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:10:02,551::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:10:02,552::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:10:02,553::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:10:02,553::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:10:02,554::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:10:02,554::task::495::TaskManager.Task::(_debug) Task 1f4b4412-5ee9-4c74-a948-cdce840ff6b0: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:10:02,555::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:10:02,837::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0796369 s, 12.9 MB/s\n'; <rc> = 0
>Thread-15915808::DEBUG::2013-11-20 04:10:04,354::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915808::INFO::2013-11-20 04:10:04,356::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915808::DEBUG::2013-11-20 04:10:04,357::task::495::TaskManager.Task::(_debug) Task 2c4cc939-fd2d-4e19-9750-64737473ba3a: moving from state init -> state preparing
>Thread-15915808::DEBUG::2013-11-20 04:10:04,358::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915808::DEBUG::2013-11-20 04:10:04,358::task::495::TaskManager.Task::(_debug) Task 2c4cc939-fd2d-4e19-9750-64737473ba3a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915808::DEBUG::2013-11-20 04:10:04,359::task::495::TaskManager.Task::(_debug) Task 2c4cc939-fd2d-4e19-9750-64737473ba3a: moving from state preparing -> state finished
>Thread-15915808::DEBUG::2013-11-20 04:10:04,360::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915808::DEBUG::2013-11-20 04:10:04,360::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915808::DEBUG::2013-11-20 04:10:04,361::task::495::TaskManager.Task::(_debug) Task 2c4cc939-fd2d-4e19-9750-64737473ba3a: ref 0 aborting False
>Thread-15915808::INFO::2013-11-20 04:10:04,362::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915809::DEBUG::2013-11-20 04:10:04,367::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915809::INFO::2013-11-20 04:10:04,376::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915809::DEBUG::2013-11-20 04:10:04,377::task::495::TaskManager.Task::(_debug) Task 80f87c2c-42b3-4e11-be94-aa031a5c4d74: moving from state init -> state preparing
>Thread-15915809::DEBUG::2013-11-20 04:10:04,379::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9c355e29-3d08-48a7-a5e6-752bf94a7c57`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915809::DEBUG::2013-11-20 04:10:04,380::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915809::DEBUG::2013-11-20 04:10:04,381::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915809::DEBUG::2013-11-20 04:10:04,381::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9c355e29-3d08-48a7-a5e6-752bf94a7c57`::Granted request
>Thread-15915809::DEBUG::2013-11-20 04:10:04,383::task::495::TaskManager.Task::(_debug) Task 80f87c2c-42b3-4e11-be94-aa031a5c4d74: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915809::DEBUG::2013-11-20 04:10:04,384::task::495::TaskManager.Task::(_debug) Task 80f87c2c-42b3-4e11-be94-aa031a5c4d74: ref 1 aborting False
>Thread-15915809::WARNING::2013-11-20 04:10:04,386::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915809::DEBUG::2013-11-20 04:10:04,387::task::495::TaskManager.Task::(_debug) Task 80f87c2c-42b3-4e11-be94-aa031a5c4d74: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915809::DEBUG::2013-11-20 04:10:04,389::task::495::TaskManager.Task::(_debug) Task 80f87c2c-42b3-4e11-be94-aa031a5c4d74: moving from state preparing -> state finished
>Thread-15915809::DEBUG::2013-11-20 04:10:04,389::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915809::DEBUG::2013-11-20 04:10:04,390::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915809::DEBUG::2013-11-20 04:10:04,391::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915809::DEBUG::2013-11-20 04:10:04,392::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915809::DEBUG::2013-11-20 04:10:04,393::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915809::DEBUG::2013-11-20 04:10:04,394::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915809::DEBUG::2013-11-20 04:10:04,395::task::495::TaskManager.Task::(_debug) Task 80f87c2c-42b3-4e11-be94-aa031a5c4d74: ref 0 aborting False
>Thread-15915809::INFO::2013-11-20 04:10:04,395::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914519::INFO::2013-11-20 04:10:04,511::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:10:04,512::task::495::TaskManager.Task::(_debug) Task ab98ca33-e229-453c-a4c6-101401c8b2cb: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:10:04,513::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7911c322-a0f8-432e-be22-b2f1433b08f0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:10:04,514::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:10:04,515::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:10:04,516::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7911c322-a0f8-432e-be22-b2f1433b08f0`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:10:04,527::task::495::TaskManager.Task::(_debug) Task ab98ca33-e229-453c-a4c6-101401c8b2cb: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 04:10:04,528::task::495::TaskManager.Task::(_debug) Task ab98ca33-e229-453c-a4c6-101401c8b2cb: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:10:04,528::task::495::TaskManager.Task::(_debug) Task ab98ca33-e229-453c-a4c6-101401c8b2cb: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 04:10:04,529::task::495::TaskManager.Task::(_debug) Task ab98ca33-e229-453c-a4c6-101401c8b2cb: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:10:04,530::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:10:04,530::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:10:04,536::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 04:10:04,537::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:10:04,537::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:10:04,538::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:10:04,539::task::495::TaskManager.Task::(_debug) Task ab98ca33-e229-453c-a4c6-101401c8b2cb: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 04:10:04,540::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 04:10:04,853::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:10:05,439::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0908449 s, 11.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:10:07,454::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 04:10:07,654::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:10:07,656::task::495::TaskManager.Task::(_debug) Task f8cd61af-d941-467f-b338-f419ae0eb4de: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:10:07,659::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fe408018-d1b8-4fd9-8aef-d61866ce25cc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:10:07,660::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:10:07,661::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:10:07,662::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fe408018-d1b8-4fd9-8aef-d61866ce25cc`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:10:07,663::task::495::TaskManager.Task::(_debug) Task f8cd61af-d941-467f-b338-f419ae0eb4de: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:10:07,664::task::495::TaskManager.Task::(_debug) Task f8cd61af-d941-467f-b338-f419ae0eb4de: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:10:07,665::task::495::TaskManager.Task::(_debug) Task f8cd61af-d941-467f-b338-f419ae0eb4de: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 04:10:07,665::task::495::TaskManager.Task::(_debug) Task f8cd61af-d941-467f-b338-f419ae0eb4de: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:10:07,665::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:10:07,666::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:10:07,666::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:10:07,667::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:10:07,667::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:10:07,668::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:10:07,669::task::495::TaskManager.Task::(_debug) Task f8cd61af-d941-467f-b338-f419ae0eb4de: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:10:07,669::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:10:07,670::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:10:07,670::task::495::TaskManager.Task::(_debug) Task 5f6c4e10-04c7-4548-8707-6eaaba36bd24: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:10:07,672::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2103bae5-7b65-4dec-bd55-32b04edb9a7a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:10:07,672::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:10:07,673::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:10:07,673::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2103bae5-7b65-4dec-bd55-32b04edb9a7a`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:10:07,674::task::495::TaskManager.Task::(_debug) Task 5f6c4e10-04c7-4548-8707-6eaaba36bd24: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:10:07,674::task::495::TaskManager.Task::(_debug) Task 5f6c4e10-04c7-4548-8707-6eaaba36bd24: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:10:07,674::task::495::TaskManager.Task::(_debug) Task 5f6c4e10-04c7-4548-8707-6eaaba36bd24: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:10:07,675::task::495::TaskManager.Task::(_debug) Task 5f6c4e10-04c7-4548-8707-6eaaba36bd24: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:10:07,675::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:10:07,675::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:10:07,676::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:10:07,676::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:10:07,677::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:10:07,677::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:10:07,677::task::495::TaskManager.Task::(_debug) Task 5f6c4e10-04c7-4548-8707-6eaaba36bd24: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:10:07,678::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:10:07,678::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:10:07,679::task::495::TaskManager.Task::(_debug) Task 4dc234c2-3789-46fc-bd8b-d05207c43b61: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:10:07,679::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fcc649b6-603b-4314-9021-53e3a1ce4023`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:10:07,680::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:10:07,680::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:10:07,680::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fcc649b6-603b-4314-9021-53e3a1ce4023`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:10:07,681::task::495::TaskManager.Task::(_debug) Task 4dc234c2-3789-46fc-bd8b-d05207c43b61: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:10:07,681::task::495::TaskManager.Task::(_debug) Task 4dc234c2-3789-46fc-bd8b-d05207c43b61: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:10:07,682::task::495::TaskManager.Task::(_debug) Task 4dc234c2-3789-46fc-bd8b-d05207c43b61: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:10:07,682::task::495::TaskManager.Task::(_debug) Task 4dc234c2-3789-46fc-bd8b-d05207c43b61: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:10:07,682::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:10:07,683::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:10:07,683::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:10:07,684::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:10:07,684::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:10:07,684::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:10:07,684::task::495::TaskManager.Task::(_debug) Task 4dc234c2-3789-46fc-bd8b-d05207c43b61: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:10:07,685::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:10:07,685::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:10:07,686::task::495::TaskManager.Task::(_debug) Task a6a9e44d-b420-4149-9752-a04f1915fdfe: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:10:07,686::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d2f37a0d-64dc-4d83-8430-7f92e4835f4e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:10:07,687::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:10:07,687::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:10:07,687::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d2f37a0d-64dc-4d83-8430-7f92e4835f4e`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:10:07,688::task::495::TaskManager.Task::(_debug) Task a6a9e44d-b420-4149-9752-a04f1915fdfe: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:10:07,688::task::495::TaskManager.Task::(_debug) Task a6a9e44d-b420-4149-9752-a04f1915fdfe: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:10:07,689::task::495::TaskManager.Task::(_debug) Task a6a9e44d-b420-4149-9752-a04f1915fdfe: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:10:07,689::task::495::TaskManager.Task::(_debug) Task a6a9e44d-b420-4149-9752-a04f1915fdfe: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:10:07,689::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:10:07,690::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:10:07,690::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:10:07,691::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:10:07,691::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:10:07,691::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:10:07,692::task::495::TaskManager.Task::(_debug) Task a6a9e44d-b420-4149-9752-a04f1915fdfe: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:10:07,692::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:10:07,958::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0845279 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:10:09,976::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:10:10,574::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0834161 s, 12.3 MB/s\n'; <rc> = 0
>Thread-15915812::INFO::2013-11-20 04:10:10,719::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915812::DEBUG::2013-11-20 04:10:10,720::task::495::TaskManager.Task::(_debug) Task 9f0d73ec-b624-4e6e-8bd4-643403bea81a: moving from state init -> state preparing
>Thread-15915812::DEBUG::2013-11-20 04:10:10,721::task::495::TaskManager.Task::(_debug) Task 9f0d73ec-b624-4e6e-8bd4-643403bea81a: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000782012939453', 'lastCheck': 1384920602.529213, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0022759437561', 'lastCheck': 1384920602.5302451, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00157284736633', 'lastCheck': 1384920609.648814, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000293016433716', 'lastCheck': 1384920606.5100291, 'valid': True, 'code': 0}}
>Thread-15915812::DEBUG::2013-11-20 04:10:10,722::task::495::TaskManager.Task::(_debug) Task 9f0d73ec-b624-4e6e-8bd4-643403bea81a: moving from state preparing -> state finished
>Thread-15915812::DEBUG::2013-11-20 04:10:10,723::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915812::DEBUG::2013-11-20 04:10:10,724::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915812::DEBUG::2013-11-20 04:10:10,725::task::495::TaskManager.Task::(_debug) Task 9f0d73ec-b624-4e6e-8bd4-643403bea81a: ref 0 aborting False
>Thread-15915812::INFO::2013-11-20 04:10:10,725::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000782012939453', 'lastCheck': 1384920602.529213, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0022759437561', 'lastCheck': 1384920602.5302451, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00157284736633', 'lastCheck': 1384920609.648814, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000293016433716', 'lastCheck': 1384920606.5100291, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:10:12,583::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:10:13,165::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0540708 s, 18.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:10:15,177::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915815::DEBUG::2013-11-20 04:10:15,610::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915815::INFO::2013-11-20 04:10:15,613::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915815::DEBUG::2013-11-20 04:10:15,614::task::495::TaskManager.Task::(_debug) Task 667019ee-3da8-4ced-8ffb-1e956e1f3a76: moving from state init -> state preparing
>Thread-15915815::DEBUG::2013-11-20 04:10:15,615::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915815::DEBUG::2013-11-20 04:10:15,616::task::495::TaskManager.Task::(_debug) Task 667019ee-3da8-4ced-8ffb-1e956e1f3a76: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915815::DEBUG::2013-11-20 04:10:15,617::task::495::TaskManager.Task::(_debug) Task 667019ee-3da8-4ced-8ffb-1e956e1f3a76: moving from state preparing -> state finished
>Thread-15915815::DEBUG::2013-11-20 04:10:15,618::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915815::DEBUG::2013-11-20 04:10:15,619::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915815::DEBUG::2013-11-20 04:10:15,620::task::495::TaskManager.Task::(_debug) Task 667019ee-3da8-4ced-8ffb-1e956e1f3a76: ref 0 aborting False
>Thread-15915815::INFO::2013-11-20 04:10:15,621::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915816::DEBUG::2013-11-20 04:10:15,632::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915816::INFO::2013-11-20 04:10:15,633::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915816::DEBUG::2013-11-20 04:10:15,634::task::495::TaskManager.Task::(_debug) Task 2219121e-0251-4eaf-865c-ea0f4e0e1e25: moving from state init -> state preparing
>Thread-15915816::DEBUG::2013-11-20 04:10:15,636::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`95f6a7c5-fb10-4571-bd04-648f995ffd7b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915816::DEBUG::2013-11-20 04:10:15,637::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915816::DEBUG::2013-11-20 04:10:15,638::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915816::DEBUG::2013-11-20 04:10:15,640::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`95f6a7c5-fb10-4571-bd04-648f995ffd7b`::Granted request
>Thread-15915816::DEBUG::2013-11-20 04:10:15,642::task::495::TaskManager.Task::(_debug) Task 2219121e-0251-4eaf-865c-ea0f4e0e1e25: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915816::DEBUG::2013-11-20 04:10:15,643::task::495::TaskManager.Task::(_debug) Task 2219121e-0251-4eaf-865c-ea0f4e0e1e25: ref 1 aborting False
>Thread-15915816::WARNING::2013-11-20 04:10:15,646::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915816::DEBUG::2013-11-20 04:10:15,648::task::495::TaskManager.Task::(_debug) Task 2219121e-0251-4eaf-865c-ea0f4e0e1e25: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915816::DEBUG::2013-11-20 04:10:15,649::task::495::TaskManager.Task::(_debug) Task 2219121e-0251-4eaf-865c-ea0f4e0e1e25: moving from state preparing -> state finished
>Thread-15915816::DEBUG::2013-11-20 04:10:15,650::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915816::DEBUG::2013-11-20 04:10:15,651::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915816::DEBUG::2013-11-20 04:10:15,652::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915816::DEBUG::2013-11-20 04:10:15,653::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915816::DEBUG::2013-11-20 04:10:15,654::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915816::DEBUG::2013-11-20 04:10:15,655::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915816::DEBUG::2013-11-20 04:10:15,656::task::495::TaskManager.Task::(_debug) Task 2219121e-0251-4eaf-865c-ea0f4e0e1e25: ref 0 aborting False
>Thread-15915816::INFO::2013-11-20 04:10:15,657::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:10:15,772::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.117041 s, 8.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:10:17,787::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:10:18,466::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.100918 s, 10.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:10:20,483::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:10:21,093::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0890082 s, 11.5 MB/s\n'; <rc> = 0
>Thread-15913639::INFO::2013-11-20 04:10:23,030::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:10:23,035::task::495::TaskManager.Task::(_debug) Task c0c3403d-4510-47a5-927d-3a0c9a62fde3: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:10:23,038::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`07772d17-b4e2-45c5-9bb2-e3ec4e5cb002`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:10:23,040::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:10:23,042::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:10:23,043::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`07772d17-b4e2-45c5-9bb2-e3ec4e5cb002`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:10:23,045::task::495::TaskManager.Task::(_debug) Task c0c3403d-4510-47a5-927d-3a0c9a62fde3: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:10:23,046::task::495::TaskManager.Task::(_debug) Task c0c3403d-4510-47a5-927d-3a0c9a62fde3: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:10:23,047::task::495::TaskManager.Task::(_debug) Task c0c3403d-4510-47a5-927d-3a0c9a62fde3: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:10:23,048::task::495::TaskManager.Task::(_debug) Task c0c3403d-4510-47a5-927d-3a0c9a62fde3: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:10:23,049::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:10:23,049::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:10:23,050::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:10:23,051::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:10:23,052::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:10:23,052::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:10:23,053::task::495::TaskManager.Task::(_debug) Task c0c3403d-4510-47a5-927d-3a0c9a62fde3: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:10:23,054::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:10:23,106::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:10:23,673::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0830511 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:10:25,689::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915820::DEBUG::2013-11-20 04:10:26,099::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915820::INFO::2013-11-20 04:10:26,101::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915820::DEBUG::2013-11-20 04:10:26,102::task::495::TaskManager.Task::(_debug) Task 90495633-a020-4e05-b809-dd4bb946bf95: moving from state init -> state preparing
>Thread-15915820::DEBUG::2013-11-20 04:10:26,104::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915820::DEBUG::2013-11-20 04:10:26,105::task::495::TaskManager.Task::(_debug) Task 90495633-a020-4e05-b809-dd4bb946bf95: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915820::DEBUG::2013-11-20 04:10:26,106::task::495::TaskManager.Task::(_debug) Task 90495633-a020-4e05-b809-dd4bb946bf95: moving from state preparing -> state finished
>Thread-15915820::DEBUG::2013-11-20 04:10:26,107::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915820::DEBUG::2013-11-20 04:10:26,107::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915820::DEBUG::2013-11-20 04:10:26,108::task::495::TaskManager.Task::(_debug) Task 90495633-a020-4e05-b809-dd4bb946bf95: ref 0 aborting False
>Thread-15915820::INFO::2013-11-20 04:10:26,110::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915821::DEBUG::2013-11-20 04:10:26,116::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915821::INFO::2013-11-20 04:10:26,118::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915821::DEBUG::2013-11-20 04:10:26,118::task::495::TaskManager.Task::(_debug) Task e96e51ae-e1cb-4bc4-a544-7f2a748811af: moving from state init -> state preparing
>Thread-15915821::DEBUG::2013-11-20 04:10:26,120::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`153223a3-084f-420b-b492-0f725dc75e8f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915821::DEBUG::2013-11-20 04:10:26,121::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915821::DEBUG::2013-11-20 04:10:26,123::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915821::DEBUG::2013-11-20 04:10:26,123::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`153223a3-084f-420b-b492-0f725dc75e8f`::Granted request
>Thread-15915821::DEBUG::2013-11-20 04:10:26,126::task::495::TaskManager.Task::(_debug) Task e96e51ae-e1cb-4bc4-a544-7f2a748811af: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915821::DEBUG::2013-11-20 04:10:26,127::task::495::TaskManager.Task::(_debug) Task e96e51ae-e1cb-4bc4-a544-7f2a748811af: ref 1 aborting False
>Thread-15915821::WARNING::2013-11-20 04:10:26,131::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915821::DEBUG::2013-11-20 04:10:26,137::task::495::TaskManager.Task::(_debug) Task e96e51ae-e1cb-4bc4-a544-7f2a748811af: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915821::DEBUG::2013-11-20 04:10:26,138::task::495::TaskManager.Task::(_debug) Task e96e51ae-e1cb-4bc4-a544-7f2a748811af: moving from state preparing -> state finished
>Thread-15915821::DEBUG::2013-11-20 04:10:26,140::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915821::DEBUG::2013-11-20 04:10:26,141::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915821::DEBUG::2013-11-20 04:10:26,143::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915821::DEBUG::2013-11-20 04:10:26,144::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915821::DEBUG::2013-11-20 04:10:26,145::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915821::DEBUG::2013-11-20 04:10:26,147::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915821::DEBUG::2013-11-20 04:10:26,148::task::495::TaskManager.Task::(_debug) Task e96e51ae-e1cb-4bc4-a544-7f2a748811af: ref 0 aborting False
>Thread-15915821::INFO::2013-11-20 04:10:26,149::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:10:26,265::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.124721 s, 8.2 MB/s\n'; <rc> = 0
>Thread-15915822::INFO::2013-11-20 04:10:26,769::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915822::DEBUG::2013-11-20 04:10:26,770::task::495::TaskManager.Task::(_debug) Task 0f23e3f9-8df9-44aa-8909-3620f217952c: moving from state init -> state preparing
>Thread-15915822::DEBUG::2013-11-20 04:10:26,771::task::495::TaskManager.Task::(_debug) Task 0f23e3f9-8df9-44aa-8909-3620f217952c: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000313997268677', 'lastCheck': 1384920623.0267611, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00179219245911', 'lastCheck': 1384920623.0276361, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0014340877533', 'lastCheck': 1384920620.0338409, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000722169876099', 'lastCheck': 1384920616.8380461, 'valid': True, 'code': 0}}
>Thread-15915822::DEBUG::2013-11-20 04:10:26,772::task::495::TaskManager.Task::(_debug) Task 0f23e3f9-8df9-44aa-8909-3620f217952c: moving from state preparing -> state finished
>Thread-15915822::DEBUG::2013-11-20 04:10:26,773::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915822::DEBUG::2013-11-20 04:10:26,773::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915822::DEBUG::2013-11-20 04:10:26,774::task::495::TaskManager.Task::(_debug) Task 0f23e3f9-8df9-44aa-8909-3620f217952c: ref 0 aborting False
>Thread-15915822::INFO::2013-11-20 04:10:26,775::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000313997268677', 'lastCheck': 1384920623.0267611, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00179219245911', 'lastCheck': 1384920623.0276361, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0014340877533', 'lastCheck': 1384920620.0338409, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000722169876099', 'lastCheck': 1384920616.8380461, 'valid': True, 'code': 0}}
>Thread-42840::DEBUG::2013-11-20 04:10:27,029::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 04:10:27,847::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913518::INFO::2013-11-20 04:10:28,179::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:10:28,180::task::495::TaskManager.Task::(_debug) Task d29bdd87-4347-4d9f-b182-d8b1e8f4632c: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:10:28,182::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7eae033f-ec3a-4d14-b542-8cbef48bbfb3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:10:28,183::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:10:28,184::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:10:28,184::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7eae033f-ec3a-4d14-b542-8cbef48bbfb3`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:10:28,186::task::495::TaskManager.Task::(_debug) Task d29bdd87-4347-4d9f-b182-d8b1e8f4632c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:10:28,186::task::495::TaskManager.Task::(_debug) Task d29bdd87-4347-4d9f-b182-d8b1e8f4632c: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:10:28,187::task::495::TaskManager.Task::(_debug) Task d29bdd87-4347-4d9f-b182-d8b1e8f4632c: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:10:28,188::task::495::TaskManager.Task::(_debug) Task d29bdd87-4347-4d9f-b182-d8b1e8f4632c: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:10:28,189::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:10:28,189::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:10:28,190::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:10:28,191::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:10:28,191::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:10:28,192::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:10:28,193::task::495::TaskManager.Task::(_debug) Task d29bdd87-4347-4d9f-b182-d8b1e8f4632c: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:10:28,193::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:10:28,194::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:10:28,195::task::495::TaskManager.Task::(_debug) Task 4bb4f474-50b7-4000-9829-0c7ca075b81b: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:10:28,196::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b17de0b6-43d6-436d-929f-4951370983bb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:10:28,197::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:10:28,198::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:10:28,199::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b17de0b6-43d6-436d-929f-4951370983bb`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:10:28,200::task::495::TaskManager.Task::(_debug) Task 4bb4f474-50b7-4000-9829-0c7ca075b81b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:10:28,200::task::495::TaskManager.Task::(_debug) Task 4bb4f474-50b7-4000-9829-0c7ca075b81b: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:10:28,201::task::495::TaskManager.Task::(_debug) Task 4bb4f474-50b7-4000-9829-0c7ca075b81b: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:10:28,202::task::495::TaskManager.Task::(_debug) Task 4bb4f474-50b7-4000-9829-0c7ca075b81b: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:10:28,202::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:10:28,203::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:10:28,204::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:10:28,204::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:10:28,205::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:10:28,205::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:10:28,207::task::495::TaskManager.Task::(_debug) Task 4bb4f474-50b7-4000-9829-0c7ca075b81b: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:10:28,208::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:10:28,282::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:10:28,820::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0682202 s, 15.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:10:30,833::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:10:31,424::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.103739 s, 9.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:10:33,441::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:10:34,035::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0933911 s, 11.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:10:36,054::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915826::DEBUG::2013-11-20 04:10:36,608::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915826::INFO::2013-11-20 04:10:36,612::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915826::DEBUG::2013-11-20 04:10:36,614::task::495::TaskManager.Task::(_debug) Task 63d9c75a-a41b-4c74-898f-f03addc87fb3: moving from state init -> state preparing
>Thread-15915826::DEBUG::2013-11-20 04:10:36,616::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915826::DEBUG::2013-11-20 04:10:36,618::task::495::TaskManager.Task::(_debug) Task 63d9c75a-a41b-4c74-898f-f03addc87fb3: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915826::DEBUG::2013-11-20 04:10:36,619::task::495::TaskManager.Task::(_debug) Task 63d9c75a-a41b-4c74-898f-f03addc87fb3: moving from state preparing -> state finished
>Thread-15915826::DEBUG::2013-11-20 04:10:36,624::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915826::DEBUG::2013-11-20 04:10:36,627::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915826::DEBUG::2013-11-20 04:10:36,629::task::495::TaskManager.Task::(_debug) Task 63d9c75a-a41b-4c74-898f-f03addc87fb3: ref 0 aborting False
>Thread-15915826::INFO::2013-11-20 04:10:36,631::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915827::DEBUG::2013-11-20 04:10:36,640::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915827::INFO::2013-11-20 04:10:36,642::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915827::DEBUG::2013-11-20 04:10:36,643::task::495::TaskManager.Task::(_debug) Task d82a1468-758c-445f-8f89-d8e72ec09899: moving from state init -> state preparing
>Thread-15915827::DEBUG::2013-11-20 04:10:36,646::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2a138022-4a85-43cd-ba81-e87bf30df215`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915827::DEBUG::2013-11-20 04:10:36,649::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915827::DEBUG::2013-11-20 04:10:36,653::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915827::DEBUG::2013-11-20 04:10:36,655::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2a138022-4a85-43cd-ba81-e87bf30df215`::Granted request
>Thread-15915827::DEBUG::2013-11-20 04:10:36,659::task::495::TaskManager.Task::(_debug) Task d82a1468-758c-445f-8f89-d8e72ec09899: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915827::DEBUG::2013-11-20 04:10:36,661::task::495::TaskManager.Task::(_debug) Task d82a1468-758c-445f-8f89-d8e72ec09899: ref 1 aborting False
>Thread-15915827::WARNING::2013-11-20 04:10:36,665::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915827::DEBUG::2013-11-20 04:10:36,675::task::495::TaskManager.Task::(_debug) Task d82a1468-758c-445f-8f89-d8e72ec09899: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915827::DEBUG::2013-11-20 04:10:36,677::task::495::TaskManager.Task::(_debug) Task d82a1468-758c-445f-8f89-d8e72ec09899: moving from state preparing -> state finished
>Thread-15915827::DEBUG::2013-11-20 04:10:36,679::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915827::DEBUG::2013-11-20 04:10:36,681::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915827::DEBUG::2013-11-20 04:10:36,684::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915827::DEBUG::2013-11-20 04:10:36,687::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915827::DEBUG::2013-11-20 04:10:36,690::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915827::DEBUG::2013-11-20 04:10:36,692::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915827::DEBUG::2013-11-20 04:10:36,694::task::495::TaskManager.Task::(_debug) Task d82a1468-758c-445f-8f89-d8e72ec09899: ref 0 aborting False
>Thread-15915827::INFO::2013-11-20 04:10:36,696::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:10:36,727::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0956224 s, 10.7 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 04:10:38,271::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:10:38,273::task::495::TaskManager.Task::(_debug) Task 2747166b-7149-4500-a372-2e00aba430a0: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:10:38,275::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6a3f7ab8-625b-413a-8692-91f753badbe2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:10:38,276::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:10:38,277::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:10:38,278::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6a3f7ab8-625b-413a-8692-91f753badbe2`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:10:38,279::task::495::TaskManager.Task::(_debug) Task 2747166b-7149-4500-a372-2e00aba430a0: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:10:38,280::task::495::TaskManager.Task::(_debug) Task 2747166b-7149-4500-a372-2e00aba430a0: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:10:38,281::task::495::TaskManager.Task::(_debug) Task 2747166b-7149-4500-a372-2e00aba430a0: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:10:38,282::task::495::TaskManager.Task::(_debug) Task 2747166b-7149-4500-a372-2e00aba430a0: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:10:38,282::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:10:38,283::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:10:38,283::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:10:38,284::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:10:38,285::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:10:38,285::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:10:38,286::task::495::TaskManager.Task::(_debug) Task 2747166b-7149-4500-a372-2e00aba430a0: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:10:38,287::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:10:38,747::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:10:39,312::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0838332 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:10:41,331::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:10:41,915::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.082052 s, 12.5 MB/s\n'; <rc> = 0
>Thread-15915830::INFO::2013-11-20 04:10:42,859::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915830::DEBUG::2013-11-20 04:10:42,860::task::495::TaskManager.Task::(_debug) Task d90bb89d-4383-4a59-abd0-b9f7ebe66435: moving from state init -> state preparing
>Thread-15915830::DEBUG::2013-11-20 04:10:42,861::task::495::TaskManager.Task::(_debug) Task d90bb89d-4383-4a59-abd0-b9f7ebe66435: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000488042831421', 'lastCheck': 1384920633.2497189, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00151300430298', 'lastCheck': 1384920633.252743, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0112850666046', 'lastCheck': 1384920640.288347, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00048303604126', 'lastCheck': 1384920638.056478, 'valid': True, 'code': 0}}
>Thread-15915830::DEBUG::2013-11-20 04:10:42,863::task::495::TaskManager.Task::(_debug) Task d90bb89d-4383-4a59-abd0-b9f7ebe66435: moving from state preparing -> state finished
>Thread-15915830::DEBUG::2013-11-20 04:10:42,863::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915830::DEBUG::2013-11-20 04:10:42,864::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915830::DEBUG::2013-11-20 04:10:42,864::task::495::TaskManager.Task::(_debug) Task d90bb89d-4383-4a59-abd0-b9f7ebe66435: ref 0 aborting False
>Thread-15915830::INFO::2013-11-20 04:10:42,865::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000488042831421', 'lastCheck': 1384920633.2497189, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00151300430298', 'lastCheck': 1384920633.252743, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0112850666046', 'lastCheck': 1384920640.288347, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00048303604126', 'lastCheck': 1384920638.056478, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:10:43,930::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:10:44,488::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0941584 s, 10.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:10:46,503::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:10:47,073::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0822972 s, 12.4 MB/s\n'; <rc> = 0
>Thread-15915833::DEBUG::2013-11-20 04:10:47,183::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915833::INFO::2013-11-20 04:10:47,184::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915833::DEBUG::2013-11-20 04:10:47,185::task::495::TaskManager.Task::(_debug) Task 3aa3a641-2c42-40cf-a349-498b63bb9007: moving from state init -> state preparing
>Thread-15915833::DEBUG::2013-11-20 04:10:47,185::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915833::DEBUG::2013-11-20 04:10:47,186::task::495::TaskManager.Task::(_debug) Task 3aa3a641-2c42-40cf-a349-498b63bb9007: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915833::DEBUG::2013-11-20 04:10:47,186::task::495::TaskManager.Task::(_debug) Task 3aa3a641-2c42-40cf-a349-498b63bb9007: moving from state preparing -> state finished
>Thread-15915833::DEBUG::2013-11-20 04:10:47,187::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915833::DEBUG::2013-11-20 04:10:47,187::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915833::DEBUG::2013-11-20 04:10:47,188::task::495::TaskManager.Task::(_debug) Task 3aa3a641-2c42-40cf-a349-498b63bb9007: ref 0 aborting False
>Thread-15915833::INFO::2013-11-20 04:10:47,188::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915834::DEBUG::2013-11-20 04:10:47,193::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915834::INFO::2013-11-20 04:10:47,200::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915834::DEBUG::2013-11-20 04:10:47,201::task::495::TaskManager.Task::(_debug) Task 5a821b42-c499-4192-adab-a4e7218973d7: moving from state init -> state preparing
>Thread-15915834::DEBUG::2013-11-20 04:10:47,203::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`56b7633a-7dff-4bba-ab04-c2c412a148c9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915834::DEBUG::2013-11-20 04:10:47,204::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915834::DEBUG::2013-11-20 04:10:47,205::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915834::DEBUG::2013-11-20 04:10:47,206::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`56b7633a-7dff-4bba-ab04-c2c412a148c9`::Granted request
>Thread-15915834::DEBUG::2013-11-20 04:10:47,207::task::495::TaskManager.Task::(_debug) Task 5a821b42-c499-4192-adab-a4e7218973d7: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915834::DEBUG::2013-11-20 04:10:47,208::task::495::TaskManager.Task::(_debug) Task 5a821b42-c499-4192-adab-a4e7218973d7: ref 1 aborting False
>Thread-15915834::WARNING::2013-11-20 04:10:47,210::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915834::DEBUG::2013-11-20 04:10:47,211::task::495::TaskManager.Task::(_debug) Task 5a821b42-c499-4192-adab-a4e7218973d7: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915834::DEBUG::2013-11-20 04:10:47,212::task::495::TaskManager.Task::(_debug) Task 5a821b42-c499-4192-adab-a4e7218973d7: moving from state preparing -> state finished
>Thread-15915834::DEBUG::2013-11-20 04:10:47,213::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915834::DEBUG::2013-11-20 04:10:47,213::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915834::DEBUG::2013-11-20 04:10:47,214::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915834::DEBUG::2013-11-20 04:10:47,215::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915834::DEBUG::2013-11-20 04:10:47,215::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915834::DEBUG::2013-11-20 04:10:47,216::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915834::DEBUG::2013-11-20 04:10:47,217::task::495::TaskManager.Task::(_debug) Task 5a821b42-c499-4192-adab-a4e7218973d7: ref 0 aborting False
>Thread-15915834::INFO::2013-11-20 04:10:47,217::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912392::INFO::2013-11-20 04:10:48,289::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:10:48,291::task::495::TaskManager.Task::(_debug) Task f06f2012-3796-42ae-b29b-ea5d5b1f825a: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:10:48,293::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e8010f28-c2a8-4e8a-a0dd-7152a88174ad`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:10:48,294::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:10:48,296::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:10:48,297::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e8010f28-c2a8-4e8a-a0dd-7152a88174ad`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:10:48,298::task::495::TaskManager.Task::(_debug) Task f06f2012-3796-42ae-b29b-ea5d5b1f825a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:10:48,299::task::495::TaskManager.Task::(_debug) Task f06f2012-3796-42ae-b29b-ea5d5b1f825a: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:10:48,300::task::495::TaskManager.Task::(_debug) Task f06f2012-3796-42ae-b29b-ea5d5b1f825a: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:10:48,300::task::495::TaskManager.Task::(_debug) Task f06f2012-3796-42ae-b29b-ea5d5b1f825a: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:10:48,301::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:10:48,302::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:10:48,302::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:10:48,303::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:10:48,303::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:10:48,304::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:10:48,305::task::495::TaskManager.Task::(_debug) Task f06f2012-3796-42ae-b29b-ea5d5b1f825a: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:10:48,305::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:10:49,087::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:10:49,606::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.071396 s, 14.3 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 04:10:50,767::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:10:50,768::task::495::TaskManager.Task::(_debug) Task 37db3967-1442-4cf1-8f73-40d7f0425ffd: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:10:50,770::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3db65bdf-2cec-4798-bfc7-56448c04f24a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:10:50,770::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:10:50,771::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:10:50,772::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3db65bdf-2cec-4798-bfc7-56448c04f24a`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:10:50,773::task::495::TaskManager.Task::(_debug) Task 37db3967-1442-4cf1-8f73-40d7f0425ffd: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:10:50,773::task::495::TaskManager.Task::(_debug) Task 37db3967-1442-4cf1-8f73-40d7f0425ffd: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:10:50,774::task::495::TaskManager.Task::(_debug) Task 37db3967-1442-4cf1-8f73-40d7f0425ffd: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:10:50,774::task::495::TaskManager.Task::(_debug) Task 37db3967-1442-4cf1-8f73-40d7f0425ffd: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:10:50,775::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:10:50,776::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:10:50,776::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:10:50,777::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:10:50,777::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:10:50,778::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:10:50,778::task::495::TaskManager.Task::(_debug) Task 37db3967-1442-4cf1-8f73-40d7f0425ffd: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:10:50,779::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:10:51,616::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:10:52,196::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.101482 s, 10.1 MB/s\n'; <rc> = 0
>Thread-15915085::INFO::2013-11-20 04:10:53,334::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=089b08fc-6544-4235-a061-fb55b09d2715 volUUID=66b599fc-b7b5-4934-a5d5-ef7283460029)
>Thread-15915085::DEBUG::2013-11-20 04:10:53,337::task::495::TaskManager.Task::(_debug) Task d917b4d4-d654-44ea-a250-7bfa5ffbdddb: moving from state init -> state preparing
>Thread-15915085::DEBUG::2013-11-20 04:10:53,339::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f6a66d0e-17a5-43da-8d61-7de1a2cfcc0e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915085::DEBUG::2013-11-20 04:10:53,341::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915085::DEBUG::2013-11-20 04:10:53,342::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915085::DEBUG::2013-11-20 04:10:53,342::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f6a66d0e-17a5-43da-8d61-7de1a2cfcc0e`::Granted request
>Thread-15915085::DEBUG::2013-11-20 04:10:53,344::task::495::TaskManager.Task::(_debug) Task d917b4d4-d654-44ea-a250-7bfa5ffbdddb: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915085::DEBUG::2013-11-20 04:10:53,344::task::495::TaskManager.Task::(_debug) Task d917b4d4-d654-44ea-a250-7bfa5ffbdddb: ref 1 aborting False
>Thread-15915085::DEBUG::2013-11-20 04:10:53,345::task::495::TaskManager.Task::(_debug) Task d917b4d4-d654-44ea-a250-7bfa5ffbdddb: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15915085::DEBUG::2013-11-20 04:10:53,346::task::495::TaskManager.Task::(_debug) Task d917b4d4-d654-44ea-a250-7bfa5ffbdddb: moving from state preparing -> state finished
>Thread-15915085::DEBUG::2013-11-20 04:10:53,347::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915085::DEBUG::2013-11-20 04:10:53,348::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915085::DEBUG::2013-11-20 04:10:53,349::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915085::DEBUG::2013-11-20 04:10:53,350::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915085::DEBUG::2013-11-20 04:10:53,350::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915085::DEBUG::2013-11-20 04:10:53,351::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915085::DEBUG::2013-11-20 04:10:53,352::task::495::TaskManager.Task::(_debug) Task d917b4d4-d654-44ea-a250-7bfa5ffbdddb: ref 0 aborting False
>Thread-15915085::INFO::2013-11-20 04:10:53,352::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Dummy-15912651::DEBUG::2013-11-20 04:10:54,211::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:10:54,818::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0820149 s, 12.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:10:56,837::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:10:57,388::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0868838 s, 11.8 MB/s\n'; <rc> = 0
>Thread-15915838::DEBUG::2013-11-20 04:10:57,686::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915838::INFO::2013-11-20 04:10:57,687::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915838::DEBUG::2013-11-20 04:10:57,688::task::495::TaskManager.Task::(_debug) Task e6752bc4-6e76-4cb2-aa8c-025f2c5e5406: moving from state init -> state preparing
>Thread-15915838::DEBUG::2013-11-20 04:10:57,689::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915838::DEBUG::2013-11-20 04:10:57,690::task::495::TaskManager.Task::(_debug) Task e6752bc4-6e76-4cb2-aa8c-025f2c5e5406: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915838::DEBUG::2013-11-20 04:10:57,691::task::495::TaskManager.Task::(_debug) Task e6752bc4-6e76-4cb2-aa8c-025f2c5e5406: moving from state preparing -> state finished
>Thread-15915838::DEBUG::2013-11-20 04:10:57,691::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915838::DEBUG::2013-11-20 04:10:57,692::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915838::DEBUG::2013-11-20 04:10:57,693::task::495::TaskManager.Task::(_debug) Task e6752bc4-6e76-4cb2-aa8c-025f2c5e5406: ref 0 aborting False
>Thread-15915838::INFO::2013-11-20 04:10:57,694::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915839::DEBUG::2013-11-20 04:10:57,704::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915839::INFO::2013-11-20 04:10:57,705::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915839::DEBUG::2013-11-20 04:10:57,706::task::495::TaskManager.Task::(_debug) Task 9da367a4-ae9c-45e1-8ad2-702b17f9a8b9: moving from state init -> state preparing
>Thread-15915839::DEBUG::2013-11-20 04:10:57,708::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`954ac73c-25de-4202-85dc-f21d1bd91b37`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915839::DEBUG::2013-11-20 04:10:57,708::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915839::DEBUG::2013-11-20 04:10:57,709::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915839::DEBUG::2013-11-20 04:10:57,710::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`954ac73c-25de-4202-85dc-f21d1bd91b37`::Granted request
>Thread-15915839::DEBUG::2013-11-20 04:10:57,711::task::495::TaskManager.Task::(_debug) Task 9da367a4-ae9c-45e1-8ad2-702b17f9a8b9: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915839::DEBUG::2013-11-20 04:10:57,711::task::495::TaskManager.Task::(_debug) Task 9da367a4-ae9c-45e1-8ad2-702b17f9a8b9: ref 1 aborting False
>Thread-15915839::WARNING::2013-11-20 04:10:57,713::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915839::DEBUG::2013-11-20 04:10:57,714::task::495::TaskManager.Task::(_debug) Task 9da367a4-ae9c-45e1-8ad2-702b17f9a8b9: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915839::DEBUG::2013-11-20 04:10:57,715::task::495::TaskManager.Task::(_debug) Task 9da367a4-ae9c-45e1-8ad2-702b17f9a8b9: moving from state preparing -> state finished
>Thread-15915839::DEBUG::2013-11-20 04:10:57,716::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915839::DEBUG::2013-11-20 04:10:57,717::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915839::DEBUG::2013-11-20 04:10:57,717::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915839::DEBUG::2013-11-20 04:10:57,718::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915839::DEBUG::2013-11-20 04:10:57,719::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915839::DEBUG::2013-11-20 04:10:57,719::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915839::DEBUG::2013-11-20 04:10:57,720::task::495::TaskManager.Task::(_debug) Task 9da367a4-ae9c-45e1-8ad2-702b17f9a8b9: ref 0 aborting False
>Thread-15915839::INFO::2013-11-20 04:10:57,721::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914453::INFO::2013-11-20 04:10:58,476::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:10:58,477::task::495::TaskManager.Task::(_debug) Task 311c175c-34ce-4c76-9958-a22051e81577: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:10:58,479::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`43bdf912-56ee-46c7-ae2f-a96d4bfc705d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:10:58,480::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:10:58,481::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:10:58,482::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`43bdf912-56ee-46c7-ae2f-a96d4bfc705d`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:10:58,483::task::495::TaskManager.Task::(_debug) Task 311c175c-34ce-4c76-9958-a22051e81577: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:10:58,483::task::495::TaskManager.Task::(_debug) Task 311c175c-34ce-4c76-9958-a22051e81577: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:10:58,484::task::495::TaskManager.Task::(_debug) Task 311c175c-34ce-4c76-9958-a22051e81577: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:10:58,485::task::495::TaskManager.Task::(_debug) Task 311c175c-34ce-4c76-9958-a22051e81577: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:10:58,485::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:10:58,486::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:10:58,487::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:10:58,487::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:10:58,488::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:10:58,488::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:10:58,489::task::495::TaskManager.Task::(_debug) Task 311c175c-34ce-4c76-9958-a22051e81577: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:10:58,490::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15915840::INFO::2013-11-20 04:10:58,581::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915840::DEBUG::2013-11-20 04:10:58,582::task::495::TaskManager.Task::(_debug) Task 99516516-0336-4b4c-a084-e2211f86fb40: moving from state init -> state preparing
>Thread-15915840::DEBUG::2013-11-20 04:10:58,583::task::495::TaskManager.Task::(_debug) Task 99516516-0336-4b4c-a084-e2211f86fb40: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000669956207275', 'lastCheck': 1384920653.3374419, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00299692153931', 'lastCheck': 1384920653.340029, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00114703178406', 'lastCheck': 1384920650.3321681, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000444173812866', 'lastCheck': 1384920658.4486091, 'valid': True, 'code': 0}}
>Thread-15915840::DEBUG::2013-11-20 04:10:58,584::task::495::TaskManager.Task::(_debug) Task 99516516-0336-4b4c-a084-e2211f86fb40: moving from state preparing -> state finished
>Thread-15915840::DEBUG::2013-11-20 04:10:58,584::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915840::DEBUG::2013-11-20 04:10:58,585::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915840::DEBUG::2013-11-20 04:10:58,585::task::495::TaskManager.Task::(_debug) Task 99516516-0336-4b4c-a084-e2211f86fb40: ref 0 aborting False
>Thread-15915840::INFO::2013-11-20 04:10:58,586::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000669956207275', 'lastCheck': 1384920653.3374419, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00299692153931', 'lastCheck': 1384920653.340029, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00114703178406', 'lastCheck': 1384920650.3321681, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000444173812866', 'lastCheck': 1384920658.4486091, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:10:59,408::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:10:59,990::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0768731 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:11:02,005::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:11:02,445::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0766458 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15914435::INFO::2013-11-20 04:11:03,908::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:11:03,909::task::495::TaskManager.Task::(_debug) Task c84abe17-7345-4b85-856f-b280f04e324c: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:11:03,911::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f61dd5ae-1555-4601-a4ec-4cafac77c6f4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:11:03,911::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:11:03,913::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:11:03,913::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f61dd5ae-1555-4601-a4ec-4cafac77c6f4`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:11:03,915::task::495::TaskManager.Task::(_debug) Task c84abe17-7345-4b85-856f-b280f04e324c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:11:03,915::task::495::TaskManager.Task::(_debug) Task c84abe17-7345-4b85-856f-b280f04e324c: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:11:03,916::task::495::TaskManager.Task::(_debug) Task c84abe17-7345-4b85-856f-b280f04e324c: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:11:03,917::task::495::TaskManager.Task::(_debug) Task c84abe17-7345-4b85-856f-b280f04e324c: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:11:03,918::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:11:03,919::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:11:03,919::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:11:03,920::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:11:03,921::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:11:03,922::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:11:03,923::task::495::TaskManager.Task::(_debug) Task c84abe17-7345-4b85-856f-b280f04e324c: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:11:03,923::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:11:04,466::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 04:11:04,770::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:11:04,772::task::495::TaskManager.Task::(_debug) Task 0e282f5e-ac24-4462-8707-7ad57a376831: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:11:04,775::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c2758d07-7d79-4643-8e81-5af187b87013`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:11:04,776::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:11:04,778::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:11:04,779::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c2758d07-7d79-4643-8e81-5af187b87013`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:11:04,780::task::495::TaskManager.Task::(_debug) Task 0e282f5e-ac24-4462-8707-7ad57a376831: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 04:11:04,782::task::495::TaskManager.Task::(_debug) Task 0e282f5e-ac24-4462-8707-7ad57a376831: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:11:04,783::task::495::TaskManager.Task::(_debug) Task 0e282f5e-ac24-4462-8707-7ad57a376831: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 04:11:04,784::task::495::TaskManager.Task::(_debug) Task 0e282f5e-ac24-4462-8707-7ad57a376831: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:11:04,786::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:11:04,787::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:11:04,788::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 04:11:04,790::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:11:04,790::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:11:04,792::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:11:04,793::task::495::TaskManager.Task::(_debug) Task 0e282f5e-ac24-4462-8707-7ad57a376831: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 04:11:04,794::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 04:11:05,012::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0792726 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:11:07,028::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:11:07,561::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0802598 s, 12.8 MB/s\n'; <rc> = 0
>Thread-15915844::DEBUG::2013-11-20 04:11:08,102::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915844::INFO::2013-11-20 04:11:08,104::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915844::DEBUG::2013-11-20 04:11:08,105::task::495::TaskManager.Task::(_debug) Task d7597c4b-cf0d-43a7-b02b-f15ede2fb980: moving from state init -> state preparing
>Thread-15915844::DEBUG::2013-11-20 04:11:08,106::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915844::DEBUG::2013-11-20 04:11:08,107::task::495::TaskManager.Task::(_debug) Task d7597c4b-cf0d-43a7-b02b-f15ede2fb980: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915844::DEBUG::2013-11-20 04:11:08,108::task::495::TaskManager.Task::(_debug) Task d7597c4b-cf0d-43a7-b02b-f15ede2fb980: moving from state preparing -> state finished
>Thread-15915844::DEBUG::2013-11-20 04:11:08,109::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915844::DEBUG::2013-11-20 04:11:08,110::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915844::DEBUG::2013-11-20 04:11:08,110::task::495::TaskManager.Task::(_debug) Task d7597c4b-cf0d-43a7-b02b-f15ede2fb980: ref 0 aborting False
>Thread-15915844::INFO::2013-11-20 04:11:08,111::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915845::DEBUG::2013-11-20 04:11:08,120::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915845::INFO::2013-11-20 04:11:08,121::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915845::DEBUG::2013-11-20 04:11:08,122::task::495::TaskManager.Task::(_debug) Task 4dd9df95-b525-4faa-b834-d718aab92a44: moving from state init -> state preparing
>Thread-15915845::DEBUG::2013-11-20 04:11:08,124::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f61a7ecb-7ef3-4f1b-8d4b-f9cd048a6254`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915845::DEBUG::2013-11-20 04:11:08,125::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915845::DEBUG::2013-11-20 04:11:08,126::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915845::DEBUG::2013-11-20 04:11:08,126::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f61a7ecb-7ef3-4f1b-8d4b-f9cd048a6254`::Granted request
>Thread-15915845::DEBUG::2013-11-20 04:11:08,128::task::495::TaskManager.Task::(_debug) Task 4dd9df95-b525-4faa-b834-d718aab92a44: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915845::DEBUG::2013-11-20 04:11:08,128::task::495::TaskManager.Task::(_debug) Task 4dd9df95-b525-4faa-b834-d718aab92a44: ref 1 aborting False
>Thread-15915845::WARNING::2013-11-20 04:11:08,131::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915845::DEBUG::2013-11-20 04:11:08,132::task::495::TaskManager.Task::(_debug) Task 4dd9df95-b525-4faa-b834-d718aab92a44: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915845::DEBUG::2013-11-20 04:11:08,133::task::495::TaskManager.Task::(_debug) Task 4dd9df95-b525-4faa-b834-d718aab92a44: moving from state preparing -> state finished
>Thread-15915845::DEBUG::2013-11-20 04:11:08,134::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915845::DEBUG::2013-11-20 04:11:08,134::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915845::DEBUG::2013-11-20 04:11:08,135::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915845::DEBUG::2013-11-20 04:11:08,136::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915845::DEBUG::2013-11-20 04:11:08,137::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915845::DEBUG::2013-11-20 04:11:08,137::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915845::DEBUG::2013-11-20 04:11:08,138::task::495::TaskManager.Task::(_debug) Task 4dd9df95-b525-4faa-b834-d718aab92a44: ref 0 aborting False
>Thread-15915845::INFO::2013-11-20 04:11:08,139::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913546::INFO::2013-11-20 04:11:08,454::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:11:08,455::task::495::TaskManager.Task::(_debug) Task 44771a8e-11bf-4891-8437-98e181747ef4: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:11:08,470::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`59782d1b-3772-4ab6-b442-0c387fbca108`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:11:08,471::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:11:08,472::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:11:08,473::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`59782d1b-3772-4ab6-b442-0c387fbca108`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:11:08,474::task::495::TaskManager.Task::(_debug) Task 44771a8e-11bf-4891-8437-98e181747ef4: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:11:08,475::task::495::TaskManager.Task::(_debug) Task 44771a8e-11bf-4891-8437-98e181747ef4: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:11:08,476::task::495::TaskManager.Task::(_debug) Task 44771a8e-11bf-4891-8437-98e181747ef4: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 04:11:08,476::task::495::TaskManager.Task::(_debug) Task 44771a8e-11bf-4891-8437-98e181747ef4: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:11:08,477::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:11:08,478::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:11:08,479::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:11:08,479::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:11:08,480::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:11:08,481::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:11:08,481::task::495::TaskManager.Task::(_debug) Task 44771a8e-11bf-4891-8437-98e181747ef4: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:11:08,482::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:11:08,483::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:11:08,484::task::495::TaskManager.Task::(_debug) Task 6eaca066-241d-426b-84aa-4d49f3f0998f: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:11:08,485::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`4d5b3e19-546d-4ed2-902c-18f5dc732d85`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:11:08,486::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:11:08,487::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:11:08,488::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`4d5b3e19-546d-4ed2-902c-18f5dc732d85`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:11:08,489::task::495::TaskManager.Task::(_debug) Task 6eaca066-241d-426b-84aa-4d49f3f0998f: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:11:08,489::task::495::TaskManager.Task::(_debug) Task 6eaca066-241d-426b-84aa-4d49f3f0998f: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:11:08,490::task::495::TaskManager.Task::(_debug) Task 6eaca066-241d-426b-84aa-4d49f3f0998f: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:11:08,491::task::495::TaskManager.Task::(_debug) Task 6eaca066-241d-426b-84aa-4d49f3f0998f: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:11:08,492::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:11:08,492::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:11:08,493::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:11:08,494::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:11:08,495::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:11:08,495::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:11:08,496::task::495::TaskManager.Task::(_debug) Task 6eaca066-241d-426b-84aa-4d49f3f0998f: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:11:08,497::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:11:08,498::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:11:08,498::task::495::TaskManager.Task::(_debug) Task 44ddf8c3-093b-4378-b4d7-48d8c8c9f5b6: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:11:08,500::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`701c2c88-cf3a-4aeb-a67d-ca3257b56d44`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:11:08,500::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:11:08,501::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:11:08,502::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`701c2c88-cf3a-4aeb-a67d-ca3257b56d44`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:11:08,503::task::495::TaskManager.Task::(_debug) Task 44ddf8c3-093b-4378-b4d7-48d8c8c9f5b6: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:11:08,504::task::495::TaskManager.Task::(_debug) Task 44ddf8c3-093b-4378-b4d7-48d8c8c9f5b6: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:11:08,505::task::495::TaskManager.Task::(_debug) Task 44ddf8c3-093b-4378-b4d7-48d8c8c9f5b6: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:11:08,505::task::495::TaskManager.Task::(_debug) Task 44ddf8c3-093b-4378-b4d7-48d8c8c9f5b6: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:11:08,506::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:11:08,507::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:11:08,508::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:11:08,509::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:11:08,509::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:11:08,510::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:11:08,511::task::495::TaskManager.Task::(_debug) Task 44ddf8c3-093b-4378-b4d7-48d8c8c9f5b6: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:11:08,511::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:11:08,513::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:11:08,513::task::495::TaskManager.Task::(_debug) Task c4cbb2a4-7fba-496b-a82e-1c0d6d542d4c: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:11:08,515::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`59a33f7b-2441-4a8d-9b98-17a48ce86f83`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:11:08,515::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:11:08,516::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:11:08,517::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`59a33f7b-2441-4a8d-9b98-17a48ce86f83`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:11:08,518::task::495::TaskManager.Task::(_debug) Task c4cbb2a4-7fba-496b-a82e-1c0d6d542d4c: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:11:08,519::task::495::TaskManager.Task::(_debug) Task c4cbb2a4-7fba-496b-a82e-1c0d6d542d4c: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:11:08,520::task::495::TaskManager.Task::(_debug) Task c4cbb2a4-7fba-496b-a82e-1c0d6d542d4c: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:11:08,520::task::495::TaskManager.Task::(_debug) Task c4cbb2a4-7fba-496b-a82e-1c0d6d542d4c: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:11:08,521::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:11:08,522::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:11:08,523::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:11:08,524::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:11:08,524::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:11:08,525::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:11:08,526::task::495::TaskManager.Task::(_debug) Task c4cbb2a4-7fba-496b-a82e-1c0d6d542d4c: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:11:08,527::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:11:09,576::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:11:10,117::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0834989 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:11:12,134::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:11:12,646::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0736981 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15915848::INFO::2013-11-20 04:11:14,500::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915848::DEBUG::2013-11-20 04:11:14,501::task::495::TaskManager.Task::(_debug) Task e3d3c6c1-ecce-4c49-8772-d9235decfb00: moving from state init -> state preparing
>Thread-15915848::DEBUG::2013-11-20 04:11:14,502::task::495::TaskManager.Task::(_debug) Task e3d3c6c1-ecce-4c49-8772-d9235decfb00: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000372886657715', 'lastCheck': 1384920673.7840929, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0100159645081', 'lastCheck': 1384920673.793153, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00154399871826', 'lastCheck': 1384920670.780057, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000849008560181', 'lastCheck': 1384920668.674818, 'valid': True, 'code': 0}}
>Thread-15915848::DEBUG::2013-11-20 04:11:14,503::task::495::TaskManager.Task::(_debug) Task e3d3c6c1-ecce-4c49-8772-d9235decfb00: moving from state preparing -> state finished
>Thread-15915848::DEBUG::2013-11-20 04:11:14,510::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915848::DEBUG::2013-11-20 04:11:14,511::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915848::DEBUG::2013-11-20 04:11:14,512::task::495::TaskManager.Task::(_debug) Task e3d3c6c1-ecce-4c49-8772-d9235decfb00: ref 0 aborting False
>Thread-15915848::INFO::2013-11-20 04:11:14,513::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000372886657715', 'lastCheck': 1384920673.7840929, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0100159645081', 'lastCheck': 1384920673.793153, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00154399871826', 'lastCheck': 1384920670.780057, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000849008560181', 'lastCheck': 1384920668.674818, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:11:14,662::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:11:15,111::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0560524 s, 18.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:11:17,121::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:11:17,680::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0775369 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15915851::DEBUG::2013-11-20 04:11:18,487::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915851::INFO::2013-11-20 04:11:18,493::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915851::DEBUG::2013-11-20 04:11:18,496::task::495::TaskManager.Task::(_debug) Task 2c93809c-0679-4671-80fa-deaf80b50d53: moving from state init -> state preparing
>Thread-15915851::DEBUG::2013-11-20 04:11:18,501::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915851::DEBUG::2013-11-20 04:11:18,503::task::495::TaskManager.Task::(_debug) Task 2c93809c-0679-4671-80fa-deaf80b50d53: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915851::DEBUG::2013-11-20 04:11:18,505::task::495::TaskManager.Task::(_debug) Task 2c93809c-0679-4671-80fa-deaf80b50d53: moving from state preparing -> state finished
>Thread-15915851::DEBUG::2013-11-20 04:11:18,507::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915851::DEBUG::2013-11-20 04:11:18,508::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915851::DEBUG::2013-11-20 04:11:18,509::task::495::TaskManager.Task::(_debug) Task 2c93809c-0679-4671-80fa-deaf80b50d53: ref 0 aborting False
>Thread-15915851::INFO::2013-11-20 04:11:18,509::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915852::DEBUG::2013-11-20 04:11:18,515::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915852::INFO::2013-11-20 04:11:18,516::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915852::DEBUG::2013-11-20 04:11:18,516::task::495::TaskManager.Task::(_debug) Task 8e890bfb-f026-4c7d-a869-8fd8cd5c5bb2: moving from state init -> state preparing
>Thread-15915852::DEBUG::2013-11-20 04:11:18,518::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`763123ac-d5c1-46ef-aa7b-bcd375fb20be`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915852::DEBUG::2013-11-20 04:11:18,519::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915852::DEBUG::2013-11-20 04:11:18,519::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915852::DEBUG::2013-11-20 04:11:18,520::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`763123ac-d5c1-46ef-aa7b-bcd375fb20be`::Granted request
>Thread-15915852::DEBUG::2013-11-20 04:11:18,521::task::495::TaskManager.Task::(_debug) Task 8e890bfb-f026-4c7d-a869-8fd8cd5c5bb2: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915852::DEBUG::2013-11-20 04:11:18,522::task::495::TaskManager.Task::(_debug) Task 8e890bfb-f026-4c7d-a869-8fd8cd5c5bb2: ref 1 aborting False
>Thread-15915852::WARNING::2013-11-20 04:11:18,524::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915852::DEBUG::2013-11-20 04:11:18,525::task::495::TaskManager.Task::(_debug) Task 8e890bfb-f026-4c7d-a869-8fd8cd5c5bb2: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915852::DEBUG::2013-11-20 04:11:18,526::task::495::TaskManager.Task::(_debug) Task 8e890bfb-f026-4c7d-a869-8fd8cd5c5bb2: moving from state preparing -> state finished
>Thread-15915852::DEBUG::2013-11-20 04:11:18,526::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915852::DEBUG::2013-11-20 04:11:18,527::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915852::DEBUG::2013-11-20 04:11:18,528::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915852::DEBUG::2013-11-20 04:11:18,528::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915852::DEBUG::2013-11-20 04:11:18,529::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915852::DEBUG::2013-11-20 04:11:18,529::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915852::DEBUG::2013-11-20 04:11:18,530::task::495::TaskManager.Task::(_debug) Task 8e890bfb-f026-4c7d-a869-8fd8cd5c5bb2: ref 0 aborting False
>Thread-15915852::INFO::2013-11-20 04:11:18,531::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915853::DEBUG::2013-11-20 04:11:19,483::clientIF::54::vds::(wrapper) [192.168.100.135]::call create with ({'bridge': 'rhevm', 'acpiEnable': 'true', 'emulatedMachine': 'rhel6.2.0', 'vmId': '01c8c942-dab4-4e5e-aefa-d474a455143e', 'transparentHugePages': 'true', 'spiceSslCipherSuite': 'DEFAULT', 'cpuType': 'Westmere', 'smp': '2', 'macAddr': '00:1a:4a:a8:64:0f', 'boot': 'c', 'custom': {}, 'vmType': 'kvm', 'memSize': 4096, 'smpCoresPerSocket': '1', 'vmName': 'monaco', 'spiceMonitors': '1', 'nice': '0', 'drives': [{'index': '0', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'format': 'raw', 'volumeID': 'dfd07c26-472f-40dc-bff0-ed31eae9cb57', 'imageID': 'c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e', 'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'propagateErrors': 'off', 'if': 'ide'}], 'smartcardEnable': 'false', 'nicModel': 'pv', 'keyboardLayout': 'en-us', 'kvmEnable': 'true', 'pitReinjection': 'false', 'timeOffset': '0', 'spiceSecureChannels': 'smain,sinputs', 'display': 'qxl'},) {}
>Dummy-15912651::DEBUG::2013-11-20 04:11:19,706::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915853::INFO::2013-11-20 04:11:20,137::clientIF::523::vds::(_getNetworkIp) network None: using 0
>Thread-15915853::INFO::2013-11-20 04:11:20,138::clientIF::656::vds::(create) vmContainerLock acquired by vm 01c8c942-dab4-4e5e-aefa-d474a455143e
>Thread-15915854::DEBUG::2013-11-20 04:11:20,145::vm::342::vm.Vm::(_startUnderlyingVm) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::Start
>Thread-15915853::DEBUG::2013-11-20 04:11:20,149::clientIF::669::vds::(create) Total desktops after creation of 01c8c942-dab4-4e5e-aefa-d474a455143e is 11
>Thread-15915854::DEBUG::2013-11-20 04:11:20,150::vm::346::vm.Vm::(_startUnderlyingVm) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::_ongoingCreations acquired
>Thread-15915853::DEBUG::2013-11-20 04:11:20,151::clientIF::59::vds::(wrapper) return create with {'status': {'message': 'Done', 'code': 0}, 'vmList': {'status': 'WaitForLaunch', 'bridge': 'rhevm', 'acpiEnable': 'true', 'emulatedMachine': 'rhel6.2.0', 'vmId': '01c8c942-dab4-4e5e-aefa-d474a455143e', 'pid': '0', 'drives': [{'index': '0', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'format': 'raw', 'volumeID': 'dfd07c26-472f-40dc-bff0-ed31eae9cb57', 'imageID': 'c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e', 'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'propagateErrors': 'off', 'if': 'ide'}], 'transparentHugePages': 'true', 'keyboardLayout': 'en-us', 'displayPort': '-1', 'displaySecurePort': '-1', 'spiceSslCipherSuite': 'DEFAULT', 'cpuType': 'Westmere', 'custom': {}, 'clientIp': '', 'nicModel': 'pv', 'macAddr': '00:1a:4a:a8:64:0f', 'kvmEnable': 'true', 'pitReinjection': 'false', 'boot': 'c', 'smp': '2', 'vmType': 'kvm', 'smartcardEnable': 'false', 'timeOffset': '0', 'memSize': 4096, 'displayIp': '0', 'spiceSecureChannels': 'smain,sinputs', 'smpCoresPerSocket': '1', 'vmName': 'monaco', 'display': 'qxl', 'spiceMonitors': '1', 'nice': '0'}}
>Thread-15915854::INFO::2013-11-20 04:11:20,152::libvirtvm::1061::vm.Vm::(_run) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::VM wrapper has started
>Thread-15915854::INFO::2013-11-20 04:11:20,158::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915854::DEBUG::2013-11-20 04:11:20,160::task::495::TaskManager.Task::(_debug) Task 77c38c36-ab6d-410a-8758-9c8a8581783e: moving from state init -> state preparing
>Thread-15915854::DEBUG::2013-11-20 04:11:20,163::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a8a788a7-48db-4687-9b0f-e6d38f21a767`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1690' at 'public_prepareVolume'
>Thread-15915854::DEBUG::2013-11-20 04:11:20,164::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915854::DEBUG::2013-11-20 04:11:20,165::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915854::DEBUG::2013-11-20 04:11:20,166::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a8a788a7-48db-4687-9b0f-e6d38f21a767`::Granted request
>Thread-15915854::DEBUG::2013-11-20 04:11:20,167::task::495::TaskManager.Task::(_debug) Task 77c38c36-ab6d-410a-8758-9c8a8581783e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915854::DEBUG::2013-11-20 04:11:20,168::task::495::TaskManager.Task::(_debug) Task 77c38c36-ab6d-410a-8758-9c8a8581783e: ref 1 aborting False
>Thread-15915854::DEBUG::2013-11-20 04:11:20,170::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e`ReqID=`daa52740-51fa-4651-9074-d0e0d77e2741`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1695' at 'public_prepareVolume'
>Thread-15915854::DEBUG::2013-11-20 04:11:20,171::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e' for lock type 'exclusive'
>Thread-15915854::DEBUG::2013-11-20 04:11:20,172::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15915854::DEBUG::2013-11-20 04:11:20,174::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:11:20,620::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.131118 s, 7.8 MB/s\n'; <rc> = 0
>Thread-15915854::DEBUG::2013-11-20 04:11:21,228::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15915854::DEBUG::2013-11-20 04:11:21,412::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15915854::DEBUG::2013-11-20 04:11:21,414::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=26 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15915854::DEBUG::2013-11-20 04:11:21,761::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000303803 s, 1.7 MB/s\n'; <rc> = 0
>Thread-15915854::DEBUG::2013-11-20 04:11:21,762::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000303803 s, 1.7 MB/s'], size: 512
>Thread-15915854::DEBUG::2013-11-20 04:11:21,764::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=26 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15915854::DEBUG::2013-11-20 04:11:22,197::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000196165 s, 2.6 MB/s\n'; <rc> = 0
>Thread-15915854::DEBUG::2013-11-20 04:11:22,198::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000196165 s, 2.6 MB/s'], size: 512
>Thread-15915854::INFO::2013-11-20 04:11:22,199::image::340::Storage.Image::(getChain) sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e chain=[<storage.blockVolume.BlockVolume instance at 0x7f48ac1b1200>] 
>Thread-15915854::DEBUG::2013-11-20 04:11:22,201::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --autobackup n --available y 6b892067-ac7e-4039-a7be-5b7387525b16/dfd07c26-472f-40dc-bff0-ed31eae9cb57' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:11:22,636::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:11:23,182::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0809814 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15915854::DEBUG::2013-11-20 04:11:23,199::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15915854::DEBUG::2013-11-20 04:11:23,201::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15915854::DEBUG::2013-11-20 04:11:23,202::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15915854::DEBUG::2013-11-20 04:11:23,203::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.dfd07c26-472f-40dc-bff0-ed31eae9cb57`ReqID=`749fa9d9-efb3-4b1a-9280-8b656595328e`::Request was made in '/usr/share/vdsm/storage/resourceFactories.py' line '149' at '__getResourceCandidatesList'
>Thread-15915854::DEBUG::2013-11-20 04:11:23,204::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.dfd07c26-472f-40dc-bff0-ed31eae9cb57' for lock type 'exclusive'
>Thread-15915854::DEBUG::2013-11-20 04:11:23,205::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.dfd07c26-472f-40dc-bff0-ed31eae9cb57' is free. Now locking as 'exclusive' (1 active user)
>Thread-15915854::DEBUG::2013-11-20 04:11:23,205::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.dfd07c26-472f-40dc-bff0-ed31eae9cb57`ReqID=`749fa9d9-efb3-4b1a-9280-8b656595328e`::Granted request
>Thread-15915854::DEBUG::2013-11-20 04:11:23,206::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e' is free. Now locking as 'exclusive' (1 active user)
>Thread-15915854::DEBUG::2013-11-20 04:11:23,207::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e`ReqID=`daa52740-51fa-4651-9074-d0e0d77e2741`::Granted request
>Thread-15915854::DEBUG::2013-11-20 04:11:23,209::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15915854::DEBUG::2013-11-20 04:11:23,210::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:11:23,386::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:11:23,388::task::495::TaskManager.Task::(_debug) Task df59e168-958e-4228-ac0c-9cc82ba639bf: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:11:23,392::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f05ae88d-8560-4afc-b33c-de3547e2043e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:11:23,393::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:11:23,394::resourceManager::481::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' found in shared state and queue is empty, Joining current shared lock (2 active users)
>Thread-15913639::DEBUG::2013-11-20 04:11:23,394::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f05ae88d-8560-4afc-b33c-de3547e2043e`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:11:23,396::task::495::TaskManager.Task::(_debug) Task df59e168-958e-4228-ac0c-9cc82ba639bf: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:11:23,396::task::495::TaskManager.Task::(_debug) Task df59e168-958e-4228-ac0c-9cc82ba639bf: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:11:23,397::task::495::TaskManager.Task::(_debug) Task df59e168-958e-4228-ac0c-9cc82ba639bf: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:11:23,398::task::495::TaskManager.Task::(_debug) Task df59e168-958e-4228-ac0c-9cc82ba639bf: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:11:23,399::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:11:23,400::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:11:23,401::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:11:23,402::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (1 active users)
>Thread-15913639::DEBUG::2013-11-20 04:11:23,402::task::495::TaskManager.Task::(_debug) Task df59e168-958e-4228-ac0c-9cc82ba639bf: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:11:23,403::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15915854::DEBUG::2013-11-20 04:11:24,113::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15915854::DEBUG::2013-11-20 04:11:24,310::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15915854::DEBUG::2013-11-20 04:11:24,314::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --refresh 6b892067-ac7e-4039-a7be-5b7387525b16/dfd07c26-472f-40dc-bff0-ed31eae9cb57' (cwd None)
>Thread-15915854::DEBUG::2013-11-20 04:11:25,149::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15915854::DEBUG::2013-11-20 04:11:25,151::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15915854::DEBUG::2013-11-20 04:11:25,152::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15915854::INFO::2013-11-20 04:11:25,153::volume::517::Storage.Volume::(prepare) Volume: preparing volume 6b892067-ac7e-4039-a7be-5b7387525b16/dfd07c26-472f-40dc-bff0-ed31eae9cb57
>Thread-15915854::DEBUG::2013-11-20 04:11:25,154::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15915854::DEBUG::2013-11-20 04:11:25,155::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:11:25,372::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:11:26,151::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0952156 s, 10.8 MB/s\n'; <rc> = 0
>Thread-15915854::DEBUG::2013-11-20 04:11:26,301::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15915854::DEBUG::2013-11-20 04:11:26,570::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15915854::DEBUG::2013-11-20 04:11:26,572::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=26 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15915854::DEBUG::2013-11-20 04:11:26,956::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000162365 s, 3.2 MB/s\n'; <rc> = 0
>Thread-15915854::DEBUG::2013-11-20 04:11:26,957::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000162365 s, 3.2 MB/s'], size: 512
>Thread-15915854::DEBUG::2013-11-20 04:11:26,959::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=26 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15915854::DEBUG::2013-11-20 04:11:27,369::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.00016269 s, 3.1 MB/s\n'; <rc> = 0
>Thread-15915854::DEBUG::2013-11-20 04:11:27,371::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.00016269 s, 3.1 MB/s'], size: 512
>Thread-15915854::DEBUG::2013-11-20 04:11:27,372::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.dfd07c26-472f-40dc-bff0-ed31eae9cb57`ReqID=`8fb5d8a3-dcf2-4016-b088-339a38a69d41`::Request was made in '/usr/share/vdsm/storage/volume.py' line '533' at 'prepare'
>Thread-15915854::DEBUG::2013-11-20 04:11:27,373::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.dfd07c26-472f-40dc-bff0-ed31eae9cb57' for lock type 'exclusive'
>Thread-15915854::DEBUG::2013-11-20 04:11:27,374::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.dfd07c26-472f-40dc-bff0-ed31eae9cb57' is free. Now locking as 'exclusive' (1 active user)
>Thread-15915854::DEBUG::2013-11-20 04:11:27,375::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.dfd07c26-472f-40dc-bff0-ed31eae9cb57`ReqID=`8fb5d8a3-dcf2-4016-b088-339a38a69d41`::Granted request
>Thread-15915854::DEBUG::2013-11-20 04:11:27,376::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=26 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15915854::DEBUG::2013-11-20 04:11:27,780::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000217632 s, 2.4 MB/s\n'; <rc> = 0
>Thread-15915854::DEBUG::2013-11-20 04:11:27,782::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000217632 s, 2.4 MB/s'], size: 512
>Thread-15915854::DEBUG::2013-11-20 04:11:27,785::task::495::TaskManager.Task::(_debug) Task 77c38c36-ab6d-410a-8758-9c8a8581783e: finished: None
>Thread-15915854::DEBUG::2013-11-20 04:11:27,786::task::495::TaskManager.Task::(_debug) Task 77c38c36-ab6d-410a-8758-9c8a8581783e: moving from state preparing -> state finished
>Thread-15915854::DEBUG::2013-11-20 04:11:27,787::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915854::DEBUG::2013-11-20 04:11:27,787::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915854::DEBUG::2013-11-20 04:11:27,789::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915854::DEBUG::2013-11-20 04:11:27,789::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915854::DEBUG::2013-11-20 04:11:27,790::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915854::DEBUG::2013-11-20 04:11:27,791::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915854::DEBUG::2013-11-20 04:11:27,792::task::495::TaskManager.Task::(_debug) Task 77c38c36-ab6d-410a-8758-9c8a8581783e: ref 0 aborting False
>Thread-15915854::INFO::2013-11-20 04:11:27,793::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, Return response: {'status': {'message': 'OK', 'code': 0}}
>Thread-15915854::INFO::2013-11-20 04:11:27,794::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915854::DEBUG::2013-11-20 04:11:27,794::task::495::TaskManager.Task::(_debug) Task 0ae2dbb4-aae9-4b59-a8f5-8fc4c2e1d7cb: moving from state init -> state preparing
>Thread-15915854::DEBUG::2013-11-20 04:11:27,796::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`535dec1c-04c6-44af-b22d-7b0826535a1d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1666' at 'public_getVolumePath'
>Thread-15915854::DEBUG::2013-11-20 04:11:27,797::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915854::DEBUG::2013-11-20 04:11:27,797::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915854::DEBUG::2013-11-20 04:11:27,798::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`535dec1c-04c6-44af-b22d-7b0826535a1d`::Granted request
>Thread-15915854::DEBUG::2013-11-20 04:11:27,799::task::495::TaskManager.Task::(_debug) Task 0ae2dbb4-aae9-4b59-a8f5-8fc4c2e1d7cb: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915854::DEBUG::2013-11-20 04:11:27,800::task::495::TaskManager.Task::(_debug) Task 0ae2dbb4-aae9-4b59-a8f5-8fc4c2e1d7cb: ref 1 aborting False
>Thread-15915854::DEBUG::2013-11-20 04:11:27,801::task::495::TaskManager.Task::(_debug) Task 0ae2dbb4-aae9-4b59-a8f5-8fc4c2e1d7cb: finished: {'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e/dfd07c26-472f-40dc-bff0-ed31eae9cb57'}
>Thread-15915854::DEBUG::2013-11-20 04:11:27,802::task::495::TaskManager.Task::(_debug) Task 0ae2dbb4-aae9-4b59-a8f5-8fc4c2e1d7cb: moving from state preparing -> state finished
>Thread-15915854::DEBUG::2013-11-20 04:11:27,802::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915854::DEBUG::2013-11-20 04:11:27,803::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915854::DEBUG::2013-11-20 04:11:27,804::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915854::DEBUG::2013-11-20 04:11:27,804::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915854::DEBUG::2013-11-20 04:11:27,806::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915854::DEBUG::2013-11-20 04:11:27,806::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915854::DEBUG::2013-11-20 04:11:27,807::task::495::TaskManager.Task::(_debug) Task 0ae2dbb4-aae9-4b59-a8f5-8fc4c2e1d7cb: ref 0 aborting False
>Thread-15915854::INFO::2013-11-20 04:11:27,807::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, Return response: {'status': {'message': 'OK', 'code': 0}, 'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e/dfd07c26-472f-40dc-bff0-ed31eae9cb57'}
>Thread-15915854::INFO::2013-11-20 04:11:27,808::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915854::DEBUG::2013-11-20 04:11:27,809::task::495::TaskManager.Task::(_debug) Task 20cad266-0667-40c9-8bad-0477ac62b8da: moving from state init -> state preparing
>Thread-15915854::DEBUG::2013-11-20 04:11:27,810::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`65891dcf-738e-42a2-8e97-9ebead82c744`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915854::DEBUG::2013-11-20 04:11:27,811::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915854::DEBUG::2013-11-20 04:11:27,811::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915854::DEBUG::2013-11-20 04:11:27,812::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`65891dcf-738e-42a2-8e97-9ebead82c744`::Granted request
>Thread-15915854::DEBUG::2013-11-20 04:11:27,813::task::495::TaskManager.Task::(_debug) Task 20cad266-0667-40c9-8bad-0477ac62b8da: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915854::DEBUG::2013-11-20 04:11:27,814::task::495::TaskManager.Task::(_debug) Task 20cad266-0667-40c9-8bad-0477ac62b8da: ref 1 aborting False
>Thread-15915854::DEBUG::2013-11-20 04:11:27,815::task::495::TaskManager.Task::(_debug) Task 20cad266-0667-40c9-8bad-0477ac62b8da: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915854::DEBUG::2013-11-20 04:11:27,815::task::495::TaskManager.Task::(_debug) Task 20cad266-0667-40c9-8bad-0477ac62b8da: moving from state preparing -> state finished
>Thread-15915854::DEBUG::2013-11-20 04:11:27,816::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915854::DEBUG::2013-11-20 04:11:27,816::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915854::DEBUG::2013-11-20 04:11:27,817::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915854::DEBUG::2013-11-20 04:11:27,818::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915854::DEBUG::2013-11-20 04:11:27,818::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915854::DEBUG::2013-11-20 04:11:27,819::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915854::DEBUG::2013-11-20 04:11:27,819::task::495::TaskManager.Task::(_debug) Task 20cad266-0667-40c9-8bad-0477ac62b8da: ref 0 aborting False
>Thread-15915854::INFO::2013-11-20 04:11:27,820::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915854::INFO::2013-11-20 04:11:27,821::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16)
>Thread-15915854::DEBUG::2013-11-20 04:11:27,822::task::495::TaskManager.Task::(_debug) Task 3799178d-3bad-40fe-ad3e-dc93dfcc06be: moving from state init -> state preparing
>Thread-15915854::INFO::2013-11-20 04:11:27,823::blockSD::674::Storage.StorageDomain::(validate) sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16
>Thread-15915854::DEBUG::2013-11-20 04:11:27,824::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15915854::DEBUG::2013-11-20 04:11:27,824::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15915854::DEBUG::2013-11-20 04:11:27,825::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15915854::DEBUG::2013-11-20 04:11:27,827::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15915854::DEBUG::2013-11-20 04:11:27,828::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-15915854::DEBUG::2013-11-20 04:11:27,829::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:11:28,168::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915854::DEBUG::2013-11-20 04:11:28,554::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15915854::DEBUG::2013-11-20 04:11:28,561::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-15915854::DEBUG::2013-11-20 04:11:28,563::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'_SHA_CKSUM=716ea7492d9134a4b13de27ac61a336ad474727c', u'POOL_SPM_ID=2', u'POOL_SPM_LVER=287']
>Thread-15915854::DEBUG::2013-11-20 04:11:28,569::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c1b1ed40-ce52-481b-a692-b6e2b1bd281e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1297' at 'public_getStorageDomainInfo'
>Thread-15915854::DEBUG::2013-11-20 04:11:28,570::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915854::DEBUG::2013-11-20 04:11:28,573::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915854::DEBUG::2013-11-20 04:11:28,574::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c1b1ed40-ce52-481b-a692-b6e2b1bd281e`::Granted request
>Thread-15915854::DEBUG::2013-11-20 04:11:28,576::task::495::TaskManager.Task::(_debug) Task 3799178d-3bad-40fe-ad3e-dc93dfcc06be: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915854::DEBUG::2013-11-20 04:11:28,577::task::495::TaskManager.Task::(_debug) Task 3799178d-3bad-40fe-ad3e-dc93dfcc06be: ref 1 aborting False
>Thread-15915854::WARNING::2013-11-20 04:11:28,581::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915854::DEBUG::2013-11-20 04:11:28,583::task::495::TaskManager.Task::(_debug) Task 3799178d-3bad-40fe-ad3e-dc93dfcc06be: finished: {'info': {'uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'vguuid': 'aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', 'lver': 287, 'state': 'OK', 'version': '2', 'role': 'Master', 'master_ver': 1, 'spm_id': 2, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'OS'}}
>Thread-15915854::DEBUG::2013-11-20 04:11:28,584::task::495::TaskManager.Task::(_debug) Task 3799178d-3bad-40fe-ad3e-dc93dfcc06be: moving from state preparing -> state finished
>Thread-15915854::DEBUG::2013-11-20 04:11:28,585::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915854::DEBUG::2013-11-20 04:11:28,586::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915854::DEBUG::2013-11-20 04:11:28,588::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915854::DEBUG::2013-11-20 04:11:28,589::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915854::DEBUG::2013-11-20 04:11:28,590::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915854::DEBUG::2013-11-20 04:11:28,591::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915854::DEBUG::2013-11-20 04:11:28,592::task::495::TaskManager.Task::(_debug) Task 3799178d-3bad-40fe-ad3e-dc93dfcc06be: ref 0 aborting False
>Thread-15915854::INFO::2013-11-20 04:11:28,593::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'vguuid': 'aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', 'lver': 287, 'state': 'OK', 'version': '2', 'role': 'Master', 'master_ver': 1, 'spm_id': 2, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'OS'}}
>Thread-15915854::DEBUG::2013-11-20 04:11:28,615::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/libexec/vdsm/hooks/before_vm_start/50_vhostmd' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:11:29,027::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.391973 s, 2.6 MB/s\n'; <rc> = 0
>Thread-15915859::DEBUG::2013-11-20 04:11:29,059::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915859::INFO::2013-11-20 04:11:29,060::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915859::DEBUG::2013-11-20 04:11:29,061::task::495::TaskManager.Task::(_debug) Task 71a4581b-32bc-4dab-836f-d0edd3b2d41a: moving from state init -> state preparing
>Thread-15915859::DEBUG::2013-11-20 04:11:29,062::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915859::DEBUG::2013-11-20 04:11:29,062::task::495::TaskManager.Task::(_debug) Task 71a4581b-32bc-4dab-836f-d0edd3b2d41a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915859::DEBUG::2013-11-20 04:11:29,063::task::495::TaskManager.Task::(_debug) Task 71a4581b-32bc-4dab-836f-d0edd3b2d41a: moving from state preparing -> state finished
>Thread-15915859::DEBUG::2013-11-20 04:11:29,064::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915859::DEBUG::2013-11-20 04:11:29,064::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915859::DEBUG::2013-11-20 04:11:29,065::task::495::TaskManager.Task::(_debug) Task 71a4581b-32bc-4dab-836f-d0edd3b2d41a: ref 0 aborting False
>Thread-15915859::INFO::2013-11-20 04:11:29,065::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915860::DEBUG::2013-11-20 04:11:29,071::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915860::INFO::2013-11-20 04:11:29,073::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915860::DEBUG::2013-11-20 04:11:29,073::task::495::TaskManager.Task::(_debug) Task 85120320-08e5-4a6d-871a-72e62ca8d72e: moving from state init -> state preparing
>Thread-15915860::DEBUG::2013-11-20 04:11:29,075::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`63caa0c1-ae3d-4c04-9912-ee07c262a6c1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915860::DEBUG::2013-11-20 04:11:29,075::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915860::DEBUG::2013-11-20 04:11:29,077::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915860::DEBUG::2013-11-20 04:11:29,078::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`63caa0c1-ae3d-4c04-9912-ee07c262a6c1`::Granted request
>Thread-15915854::DEBUG::2013-11-20 04:11:29,079::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15915860::DEBUG::2013-11-20 04:11:29,081::task::495::TaskManager.Task::(_debug) Task 85120320-08e5-4a6d-871a-72e62ca8d72e: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915854::INFO::2013-11-20 04:11:29,082::hooks::51::root::(_runHooksDir) 
>Thread-15915860::DEBUG::2013-11-20 04:11:29,083::task::495::TaskManager.Task::(_debug) Task 85120320-08e5-4a6d-871a-72e62ca8d72e: ref 1 aborting False
>Thread-15915854::DEBUG::2013-11-20 04:11:29,083::libvirtvm::1077::vm.Vm::(_run) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::<?xml version="1.0" encoding="utf-8"?>
><domain type="kvm">
>	<name>monaco</name>
>	<uuid>01c8c942-dab4-4e5e-aefa-d474a455143e</uuid>
>	<memory>4194304</memory>
>	<currentMemory>4194304</currentMemory>
>	<vcpu>2</vcpu>
>	<devices>
>		<disk device="disk" type="block">
>			<source dev="/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e/dfd07c26-472f-40dc-bff0-ed31eae9cb57"/>
>			<target bus="ide" dev="hda"/>
>			<serial>2c-ab9b-7b7a7e3fa52e</serial>
>			<driver cache="none" error_policy="stop" io="native" name="qemu" type="raw"/>
>		</disk>
>		<disk device="cdrom" type="file">
>			<target bus="ide" dev="hdc"/>
>		</disk>
>		<controller index="0" ports="16" type="virtio-serial"/>
>		<channel type="unix">
>			<target name="com.redhat.rhevm.vdsm" type="virtio"/>
>			<source mode="bind" path="/var/lib/libvirt/qemu/channels/monaco.com.redhat.rhevm.vdsm"/>
>		</channel>
>		<memballoon model="none"/>
>		<interface type="bridge">
>			<mac address="00:1a:4a:a8:64:0f"/>
>			<model type="virtio"/>
>			<source bridge="rhevm"/>
>		</interface>
>		<input bus="ps2" type="mouse"/>
>		<video>
>			<model heads="1" type="qxl" vram="65536"/>
>		</video>
>		<channel type="spicevmc">
>			<target name="com.redhat.spice.0" type="virtio"/>
>		</channel>
>		<graphics autoport="yes" keymap="en-us" listen="0" passwd="*****" passwdValidTo="1970-01-01T00:00:01" port="-1" tlsPort="-1" type="spice">
>			<channel mode="secure" name="main"/>
>			<channel mode="secure" name="inputs"/>
>		</graphics>
>	</devices>
>	<os>
>		<type arch="x86_64" machine="rhel6.2.0">hvm</type>
>		<boot dev="hd"/>
>		<smbios mode="sysinfo"/>
>	</os>
>	<sysinfo type="smbios">
>		<system>
>			<entry name="manufacturer">Red Hat</entry>
>			<entry name="product">RHEV Hypervisor</entry>
>			<entry name="version">6.3-20121012.0.el6_3</entry>
>			<entry name="serial">30343536-3138-5A43-4A32-333630373442_AC:16:2D:76:EB:F0</entry>
>			<entry name="uuid">01c8c942-dab4-4e5e-aefa-d474a455143e</entry>
>		</system>
>	</sysinfo>
>	<clock adjustment="0" offset="variable">
>		<timer name="rtc" tickpolicy="catchup"/>
>	</clock>
>	<features>
>		<acpi/>
>	</features>
>	<cpu match="exact">
>		<model>Westmere</model>
>		<topology cores="1" sockets="2" threads="1"/>
>	</cpu>
></domain>
>
>Thread-15915860::WARNING::2013-11-20 04:11:29,087::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915860::DEBUG::2013-11-20 04:11:29,089::task::495::TaskManager.Task::(_debug) Task 85120320-08e5-4a6d-871a-72e62ca8d72e: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915860::DEBUG::2013-11-20 04:11:29,090::task::495::TaskManager.Task::(_debug) Task 85120320-08e5-4a6d-871a-72e62ca8d72e: moving from state preparing -> state finished
>Thread-15915860::DEBUG::2013-11-20 04:11:29,091::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915860::DEBUG::2013-11-20 04:11:29,091::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915860::DEBUG::2013-11-20 04:11:29,092::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915860::DEBUG::2013-11-20 04:11:29,093::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915860::DEBUG::2013-11-20 04:11:29,094::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915860::DEBUG::2013-11-20 04:11:29,094::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915860::DEBUG::2013-11-20 04:11:29,095::task::495::TaskManager.Task::(_debug) Task 85120320-08e5-4a6d-871a-72e62ca8d72e: ref 0 aborting False
>Thread-15915860::INFO::2013-11-20 04:11:29,096::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913518::INFO::2013-11-20 04:11:29,741::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:11:29,742::task::495::TaskManager.Task::(_debug) Task 63e293d0-9a5f-44a2-9081-2e658035eea3: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:11:29,744::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0eb024fb-4baa-4d2c-a108-435c6de13f96`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:11:29,745::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:11:29,747::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:11:29,757::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0eb024fb-4baa-4d2c-a108-435c6de13f96`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:11:29,758::task::495::TaskManager.Task::(_debug) Task 63e293d0-9a5f-44a2-9081-2e658035eea3: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:11:29,759::task::495::TaskManager.Task::(_debug) Task 63e293d0-9a5f-44a2-9081-2e658035eea3: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:11:29,760::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913518::DEBUG::2013-11-20 04:11:29,762::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15915861::INFO::2013-11-20 04:11:30,353::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915861::DEBUG::2013-11-20 04:11:30,354::task::495::TaskManager.Task::(_debug) Task 47aa45f0-f630-4947-bae8-636881c1f3a7: moving from state init -> state preparing
>Thread-15915861::DEBUG::2013-11-20 04:11:30,361::task::495::TaskManager.Task::(_debug) Task 47aa45f0-f630-4947-bae8-636881c1f3a7: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000645875930786', 'lastCheck': 1384920684.2439151, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0062370300293', 'lastCheck': 1384920684.2435939, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00142407417297', 'lastCheck': 1384920681.1264131, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000398874282837', 'lastCheck': 1384920689.1644621, 'valid': True, 'code': 0}}
>Thread-15915861::DEBUG::2013-11-20 04:11:30,362::task::495::TaskManager.Task::(_debug) Task 47aa45f0-f630-4947-bae8-636881c1f3a7: moving from state preparing -> state finished
>Thread-15915861::DEBUG::2013-11-20 04:11:30,362::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915861::DEBUG::2013-11-20 04:11:30,363::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915861::DEBUG::2013-11-20 04:11:30,363::task::495::TaskManager.Task::(_debug) Task 47aa45f0-f630-4947-bae8-636881c1f3a7: ref 0 aborting False
>Thread-15915861::INFO::2013-11-20 04:11:30,364::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000645875930786', 'lastCheck': 1384920684.2439151, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0062370300293', 'lastCheck': 1384920684.2435939, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00142407417297', 'lastCheck': 1384920681.1264131, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000398874282837', 'lastCheck': 1384920689.1644621, 'valid': True, 'code': 0}}
>Thread-15913518::DEBUG::2013-11-20 04:11:30,833::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-20 04:11:30,936::libvirtvm::1401::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::event Started detail 0 opaque None
>Dummy-15912651::DEBUG::2013-11-20 04:11:31,045::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913518::DEBUG::2013-11-20 04:11:31,328::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15915854::DEBUG::2013-11-20 04:11:31,352::utils::307::vm.Vm::(start) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::Start statistics collection
>Thread-15913518::DEBUG::2013-11-20 04:11:31,356::task::495::TaskManager.Task::(_debug) Task 63e293d0-9a5f-44a2-9081-2e658035eea3: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:11:31,359::task::495::TaskManager.Task::(_debug) Task 63e293d0-9a5f-44a2-9081-2e658035eea3: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:11:31,360::utils::336::vm.Vm::(run) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::Stats thread started
>Thread-15913518::DEBUG::2013-11-20 04:11:31,363::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915864::DEBUG::2013-11-20 04:11:31,364::guestIF::47::root::(_connect) Attempting connection to /var/lib/libvirt/qemu/channels/monaco.com.redhat.rhevm.vdsm
>Thread-15915863::INFO::2013-11-20 04:11:31,368::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15913518::DEBUG::2013-11-20 04:11:31,368::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915864::DEBUG::2013-11-20 04:11:31,370::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /usr/share/vdsm/prepare-vmchannel /var/lib/libvirt/qemu/channels/monaco.com.redhat.rhevm.vdsm' (cwd None)
>Thread-15915854::WARNING::2013-11-20 04:11:31,370::libvirtvm::1124::vm.Vm::(_readPauseCode) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::_readPauseCode unsupported by libvirt vm
>Thread-15915863::DEBUG::2013-11-20 04:11:31,371::task::495::TaskManager.Task::(_debug) Task c2c68c4f-0388-498d-aa7b-0a1b13bc7ee0: moving from state init -> state preparing
>Thread-15915854::DEBUG::2013-11-20 04:11:31,375::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /usr/share/vdsm/get-vm-pid monaco' (cwd None)
>Thread-15913518::DEBUG::2013-11-20 04:11:31,376::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:11:31,716::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`cea4e345-6534-4f93-9556-2e7a5c34c286`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:11:31,721::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:11:32,128::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:11:32,161::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:11:32,176::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915864::DEBUG::2013-11-20 04:11:32,178::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15915863::DEBUG::2013-11-20 04:11:32,180::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:11:32,182::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`cea4e345-6534-4f93-9556-2e7a5c34c286`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:11:32,184::task::495::TaskManager.Task::(_debug) Task 63e293d0-9a5f-44a2-9081-2e658035eea3: ref 0 aborting False
>Thread-15915864::DEBUG::2013-11-20 04:11:32,184::guestIF::53::root::(_connect) connected to /var/lib/libvirt/qemu/channels/monaco.com.redhat.rhevm.vdsm
>Thread-15913518::INFO::2013-11-20 04:11:32,186::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:11:32,189::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:11:32,190::task::495::TaskManager.Task::(_debug) Task e9f2703c-758b-4d04-b2b1-f9554872697a: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:11:32,190::task::495::TaskManager.Task::(_debug) Task c2c68c4f-0388-498d-aa7b-0a1b13bc7ee0: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:11:32,192::task::495::TaskManager.Task::(_debug) Task c2c68c4f-0388-498d-aa7b-0a1b13bc7ee0: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:11:32,193::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d4146a6e-a318-49ac-a8a3-8e574d750b8e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:11:32,194::task::495::TaskManager.Task::(_debug) Task c2c68c4f-0388-498d-aa7b-0a1b13bc7ee0: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15913518::DEBUG::2013-11-20 04:11:32,195::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:11:32,196::task::495::TaskManager.Task::(_debug) Task c2c68c4f-0388-498d-aa7b-0a1b13bc7ee0: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:11:32,198::resourceManager::481::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' found in shared state and queue is empty, Joining current shared lock (2 active users)
>Thread-15915863::DEBUG::2013-11-20 04:11:32,199::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:11:32,199::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d4146a6e-a318-49ac-a8a3-8e574d750b8e`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:11:32,201::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:11:32,203::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:11:32,204::task::495::TaskManager.Task::(_debug) Task e9f2703c-758b-4d04-b2b1-f9554872697a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:11:32,205::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (1 active users)
>Thread-15913518::DEBUG::2013-11-20 04:11:32,206::task::495::TaskManager.Task::(_debug) Task e9f2703c-758b-4d04-b2b1-f9554872697a: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:11:32,207::task::495::TaskManager.Task::(_debug) Task c2c68c4f-0388-498d-aa7b-0a1b13bc7ee0: ref 0 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:11:32,208::task::495::TaskManager.Task::(_debug) Task e9f2703c-758b-4d04-b2b1-f9554872697a: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15915863::INFO::2013-11-20 04:11:32,209::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15913518::DEBUG::2013-11-20 04:11:32,209::task::495::TaskManager.Task::(_debug) Task e9f2703c-758b-4d04-b2b1-f9554872697a: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:11:32,211::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:11:32,211::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:11:32,212::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:11:32,213::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:11:32,213::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:11:32,214::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:11:32,215::task::495::TaskManager.Task::(_debug) Task e9f2703c-758b-4d04-b2b1-f9554872697a: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:11:32,216::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15915854::DEBUG::2013-11-20 04:11:32,298::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15915854::DEBUG::2013-11-20 04:11:32,322::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /sbin/service ksmtuned retune' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:11:32,554::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.609675 s, 1.7 MB/s\n'; <rc> = 0
>Thread-15915854::DEBUG::2013-11-20 04:11:32,793::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15915854::DEBUG::2013-11-20 04:11:32,794::vm::358::vm.Vm::(_startUnderlyingVm) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::_ongoingCreations released
>Thread-15915865::DEBUG::2013-11-20 04:11:33,049::clientIF::54::vds::(wrapper) [192.168.100.135]::call setVmTicket with ('dab1628d-5a54-49ba-9685-47e4c156842c', 'tH0hTC21UcY1', '120') {}
>Thread-15915865::DEBUG::2013-11-20 04:11:33,087::clientIF::59::vds::(wrapper) return setVmTicket with {'status': {'message': 'Done', 'code': 0}}
>Thread-15915867::DEBUG::2013-11-20 04:11:34,049::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('01c8c942-dab4-4e5e-aefa-d474a455143e',) {}
>Thread-15915867::DEBUG::2013-11-20 04:11:34,050::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::CPU stats not available
>Thread-15915867::DEBUG::2013-11-20 04:11:34,051::libvirtvm::202::vm.Vm::(_getNetworkStats) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::Network stats not available
>Thread-15915867::DEBUG::2013-11-20 04:11:34,051::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::Disk stats not available
>Thread-15915867::DEBUG::2013-11-20 04:11:34,052::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::Disk latency not available
>Thread-15915867::DEBUG::2013-11-20 04:11:34,053::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '43977', 'displayIp': '0', 'displayPort': u'5920', 'session': 'Unknown', 'displaySecurePort': u'5921', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet11': {'state': 'unknown', 'macAddr': u'00:1a:4a:a8:64:0f', 'speed': '1000', 'name': u'vnet11'}}, 'vmId': '01c8c942-dab4-4e5e-aefa-d474a455143e', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '107776835584', 'writeLatency': '0', 'imageID': 'c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e', 'flushLatency': '0', 'truesize': '107776835584'}}, 'boot': 'c', 'statsAge': '2.69', 'cpuIdle': '100.00', 'elapsedTime': '13', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 04:11:34,570::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915868::DEBUG::2013-11-20 04:11:34,775::clientIF::54::vds::(wrapper) [192.168.100.135]::call setVmTicket with ('dab1628d-5a54-49ba-9685-47e4c156842c', 'Nw9P4ViRNwtR', '120') {}
>Thread-15915868::DEBUG::2013-11-20 04:11:34,804::clientIF::59::vds::(wrapper) return setVmTicket with {'status': {'message': 'Done', 'code': 0}}
>Thread-15915869::DEBUG::2013-11-20 04:11:34,877::clientIF::54::vds::(wrapper) [192.168.100.135]::call setVmTicket with ('dab1628d-5a54-49ba-9685-47e4c156842c', 'HTCwj5vuYBHY', '120') {}
>Thread-15915869::DEBUG::2013-11-20 04:11:34,897::clientIF::59::vds::(wrapper) return setVmTicket with {'status': {'message': 'Done', 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:11:35,119::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0909355 s, 11.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:11:37,139::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915871::DEBUG::2013-11-20 04:11:37,310::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('01c8c942-dab4-4e5e-aefa-d474a455143e',) {}
>Thread-15915871::DEBUG::2013-11-20 04:11:37,320::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::CPU stats not available
>Thread-15915871::DEBUG::2013-11-20 04:11:37,321::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::Disk stats not available
>Thread-15915871::DEBUG::2013-11-20 04:11:37,322::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::Disk latency not available
>Thread-15915871::DEBUG::2013-11-20 04:11:37,323::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '43977', 'displayIp': '0', 'displayPort': u'5920', 'session': 'Unknown', 'displaySecurePort': u'5921', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet11': {'macAddr': u'00:1a:4a:a8:64:0f', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet11'}}, 'vmId': '01c8c942-dab4-4e5e-aefa-d474a455143e', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '107776835584', 'writeLatency': '0', 'imageID': 'c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e', 'flushLatency': '0', 'truesize': '107776835584'}}, 'boot': 'c', 'statsAge': '0.02', 'cpuIdle': '100.00', 'elapsedTime': '17', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 04:11:37,659::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0956469 s, 10.7 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 04:11:38,922::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:11:38,924::task::495::TaskManager.Task::(_debug) Task eac8e6c8-fbbe-409f-a1cf-b25dfa7bc183: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:11:38,926::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b44ec239-59e8-4113-aae3-9b89f9693c09`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:11:38,927::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:11:38,928::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:11:38,928::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b44ec239-59e8-4113-aae3-9b89f9693c09`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:11:38,929::task::495::TaskManager.Task::(_debug) Task eac8e6c8-fbbe-409f-a1cf-b25dfa7bc183: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:11:38,930::task::495::TaskManager.Task::(_debug) Task eac8e6c8-fbbe-409f-a1cf-b25dfa7bc183: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:11:38,931::task::495::TaskManager.Task::(_debug) Task eac8e6c8-fbbe-409f-a1cf-b25dfa7bc183: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:11:38,932::task::495::TaskManager.Task::(_debug) Task eac8e6c8-fbbe-409f-a1cf-b25dfa7bc183: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:11:38,933::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:11:38,933::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:11:38,934::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:11:38,935::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:11:38,935::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:11:38,936::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:11:38,937::task::495::TaskManager.Task::(_debug) Task eac8e6c8-fbbe-409f-a1cf-b25dfa7bc183: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:11:38,938::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15915872::DEBUG::2013-11-20 04:11:39,621::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915872::INFO::2013-11-20 04:11:39,623::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915872::DEBUG::2013-11-20 04:11:39,629::task::495::TaskManager.Task::(_debug) Task c3351e36-6f58-4bdb-abb8-ea75f9480a52: moving from state init -> state preparing
>Thread-15915872::DEBUG::2013-11-20 04:11:39,633::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915872::DEBUG::2013-11-20 04:11:39,635::task::495::TaskManager.Task::(_debug) Task c3351e36-6f58-4bdb-abb8-ea75f9480a52: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915872::DEBUG::2013-11-20 04:11:39,636::task::495::TaskManager.Task::(_debug) Task c3351e36-6f58-4bdb-abb8-ea75f9480a52: moving from state preparing -> state finished
>Thread-15915872::DEBUG::2013-11-20 04:11:39,638::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915872::DEBUG::2013-11-20 04:11:39,640::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915872::DEBUG::2013-11-20 04:11:39,642::task::495::TaskManager.Task::(_debug) Task c3351e36-6f58-4bdb-abb8-ea75f9480a52: ref 0 aborting False
>Thread-15915872::INFO::2013-11-20 04:11:39,643::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915873::DEBUG::2013-11-20 04:11:39,649::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915873::INFO::2013-11-20 04:11:39,651::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915873::DEBUG::2013-11-20 04:11:39,651::task::495::TaskManager.Task::(_debug) Task 33d24bae-7bdb-4b67-ad87-679385a9fd4b: moving from state init -> state preparing
>Thread-15915873::DEBUG::2013-11-20 04:11:39,653::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`152e6599-0002-4af2-9b1f-adf6ad66a342`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915873::DEBUG::2013-11-20 04:11:39,653::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915873::DEBUG::2013-11-20 04:11:39,654::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915873::DEBUG::2013-11-20 04:11:39,655::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`152e6599-0002-4af2-9b1f-adf6ad66a342`::Granted request
>Thread-15915873::DEBUG::2013-11-20 04:11:39,656::task::495::TaskManager.Task::(_debug) Task 33d24bae-7bdb-4b67-ad87-679385a9fd4b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915873::DEBUG::2013-11-20 04:11:39,657::task::495::TaskManager.Task::(_debug) Task 33d24bae-7bdb-4b67-ad87-679385a9fd4b: ref 1 aborting False
>Thread-15915873::WARNING::2013-11-20 04:11:39,659::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915873::DEBUG::2013-11-20 04:11:39,660::task::495::TaskManager.Task::(_debug) Task 33d24bae-7bdb-4b67-ad87-679385a9fd4b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915873::DEBUG::2013-11-20 04:11:39,661::task::495::TaskManager.Task::(_debug) Task 33d24bae-7bdb-4b67-ad87-679385a9fd4b: moving from state preparing -> state finished
>Thread-15915873::DEBUG::2013-11-20 04:11:39,662::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915873::DEBUG::2013-11-20 04:11:39,662::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915873::DEBUG::2013-11-20 04:11:39,663::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915873::DEBUG::2013-11-20 04:11:39,664::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915873::DEBUG::2013-11-20 04:11:39,664::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915873::DEBUG::2013-11-20 04:11:39,665::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915873::DEBUG::2013-11-20 04:11:39,666::task::495::TaskManager.Task::(_debug) Task 33d24bae-7bdb-4b67-ad87-679385a9fd4b: ref 0 aborting False
>Thread-15915873::INFO::2013-11-20 04:11:39,667::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:11:39,676::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:11:40,308::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.105107 s, 9.7 MB/s\n'; <rc> = 0
>Thread-15915875::DEBUG::2013-11-20 04:11:40,348::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('01c8c942-dab4-4e5e-aefa-d474a455143e',) {}
>Thread-15915875::DEBUG::2013-11-20 04:11:40,349::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::CPU stats not available
>Thread-15915875::DEBUG::2013-11-20 04:11:40,350::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::Disk stats not available
>Thread-15915875::DEBUG::2013-11-20 04:11:40,351::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::Disk latency not available
>Thread-15915875::DEBUG::2013-11-20 04:11:40,352::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '43977', 'displayIp': '0', 'displayPort': u'5920', 'session': 'Unknown', 'displaySecurePort': u'5921', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet11': {'macAddr': u'00:1a:4a:a8:64:0f', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet11'}}, 'vmId': '01c8c942-dab4-4e5e-aefa-d474a455143e', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '107776835584', 'writeLatency': '0', 'imageID': 'c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e', 'flushLatency': '0', 'truesize': '107776835584'}}, 'boot': 'c', 'statsAge': '0.05', 'cpuIdle': '100.00', 'elapsedTime': '20', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 04:11:42,325::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:11:42,886::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0873379 s, 11.7 MB/s\n'; <rc> = 0
>Thread-15915876::DEBUG::2013-11-20 04:11:43,272::clientIF::54::vds::(wrapper) [192.168.100.135]::call shutdown with ('dab1628d-5a54-49ba-9685-47e4c156842c', '30', 'System Administrator has initiated shutdown of this Virtual Machine. Virtual Machine is shutting down.') {}
>Thread-15915876::DEBUG::2013-11-20 04:11:43,282::clientIF::59::vds::(wrapper) return shutdown with {'status': {'message': 'Machine shut down', 'code': 0}}
>Thread-15915878::DEBUG::2013-11-20 04:11:43,378::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('dab1628d-5a54-49ba-9685-47e4c156842c',) {}
>Thread-15915878::DEBUG::2013-11-20 04:11:43,380::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering down', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '22921', 'displayIp': '0', 'displayPort': u'5918', 'session': 'Unknown', 'displaySecurePort': u'5919', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/ubuntu-12.04.1-server-amd64.iso', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet10': {'macAddr': u'00:1a:4a:a8:64:1a', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '9.1', 'rxRate': '0.2', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet10'}}, 'vmId': 'dab1628d-5a54-49ba-9685-47e4c156842c', 'monitorResponse': '0', 'cpuUser': '135.12', 'disks': {u'vda': {'readLatency': '1804444', 'apparentsize': '107374182400', 'writeLatency': '1966966', 'imageID': '089b08fc-6544-4235-a061-fb55b09d2715', 'flushLatency': '260155', 'readRate': '62895427.63', 'truesize': '107374182400', 'writeRate': '321958.61'}}, 'boot': 'cd', 'statsAge': '1.24', 'cpuIdle': '0.00', 'elapsedTime': '1345', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': '0', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15915879::DEBUG::2013-11-20 04:11:43,387::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('01c8c942-dab4-4e5e-aefa-d474a455143e',) {}
>Thread-15915879::DEBUG::2013-11-20 04:11:43,388::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::CPU stats not available
>Thread-15915879::DEBUG::2013-11-20 04:11:43,389::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::Disk stats not available
>Thread-15915879::DEBUG::2013-11-20 04:11:43,389::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::Disk latency not available
>Thread-15915879::DEBUG::2013-11-20 04:11:43,390::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '43977', 'displayIp': '0', 'displayPort': u'5920', 'session': 'Unknown', 'displaySecurePort': u'5921', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet11': {'macAddr': u'00:1a:4a:a8:64:0f', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet11'}}, 'vmId': '01c8c942-dab4-4e5e-aefa-d474a455143e', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '107776835584', 'writeLatency': '0', 'imageID': 'c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e', 'flushLatency': '0', 'truesize': '107776835584'}}, 'boot': 'c', 'statsAge': '1.08', 'cpuIdle': '100.00', 'elapsedTime': '23', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>libvirtEventLoop::DEBUG::2013-11-20 04:11:43,858::vm::525::vm.Vm::(_rtcUpdate) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::new rtc offset -1
>Dummy-15912651::DEBUG::2013-11-20 04:11:44,909::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:11:45,506::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0779072 s, 13.1 MB/s\n'; <rc> = 0
>Thread-15915880::INFO::2013-11-20 04:11:46,667::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915880::DEBUG::2013-11-20 04:11:46,668::task::495::TaskManager.Task::(_debug) Task b3940d57-bffb-4146-9aac-05b499067273: moving from state init -> state preparing
>Thread-15915880::DEBUG::2013-11-20 04:11:46,669::task::495::TaskManager.Task::(_debug) Task b3940d57-bffb-4146-9aac-05b499067273: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000402927398682', 'lastCheck': 1384920704.9005051, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148797035217', 'lastCheck': 1384920704.900872, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0179698467255', 'lastCheck': 1384920701.554893, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00035285949707', 'lastCheck': 1384920699.3373611, 'valid': True, 'code': 0}}
>Thread-15915880::DEBUG::2013-11-20 04:11:46,670::task::495::TaskManager.Task::(_debug) Task b3940d57-bffb-4146-9aac-05b499067273: moving from state preparing -> state finished
>Thread-15915880::DEBUG::2013-11-20 04:11:46,671::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915880::DEBUG::2013-11-20 04:11:46,671::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915880::DEBUG::2013-11-20 04:11:46,672::task::495::TaskManager.Task::(_debug) Task b3940d57-bffb-4146-9aac-05b499067273: ref 0 aborting False
>Thread-15915880::INFO::2013-11-20 04:11:46,673::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000402927398682', 'lastCheck': 1384920704.9005051, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148797035217', 'lastCheck': 1384920704.900872, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0179698467255', 'lastCheck': 1384920701.554893, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00035285949707', 'lastCheck': 1384920699.3373611, 'valid': True, 'code': 0}}
>Thread-15915881::DEBUG::2013-11-20 04:11:46,696::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::CPU stats not available
>Thread-15915881::DEBUG::2013-11-20 04:11:46,698::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::Disk stats not available
>Thread-15915881::DEBUG::2013-11-20 04:11:46,698::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::Disk latency not available
>Dummy-15912651::DEBUG::2013-11-20 04:11:47,523::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:11:48,112::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0764696 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 04:11:48,778::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:11:48,779::task::495::TaskManager.Task::(_debug) Task 9ab24193-d674-4dcb-aac0-d72592ea3596: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:11:48,781::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f32b8f40-8bea-4922-bf08-2781db6a66f8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:11:48,782::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:11:48,783::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:11:48,783::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f32b8f40-8bea-4922-bf08-2781db6a66f8`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:11:48,784::task::495::TaskManager.Task::(_debug) Task 9ab24193-d674-4dcb-aac0-d72592ea3596: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:11:48,785::task::495::TaskManager.Task::(_debug) Task 9ab24193-d674-4dcb-aac0-d72592ea3596: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:11:48,786::task::495::TaskManager.Task::(_debug) Task 9ab24193-d674-4dcb-aac0-d72592ea3596: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:11:48,787::task::495::TaskManager.Task::(_debug) Task 9ab24193-d674-4dcb-aac0-d72592ea3596: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:11:48,787::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:11:48,788::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:11:48,789::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:11:48,790::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:11:48,791::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:11:48,791::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:11:48,792::task::495::TaskManager.Task::(_debug) Task 9ab24193-d674-4dcb-aac0-d72592ea3596: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:11:48,793::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15915883::DEBUG::2013-11-20 04:11:50,097::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('dab1628d-5a54-49ba-9685-47e4c156842c',) {}
>Thread-15915883::DEBUG::2013-11-20 04:11:50,099::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering down', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '22921', 'displayIp': '0', 'displayPort': u'5918', 'session': 'Unknown', 'displaySecurePort': u'5919', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/ubuntu-12.04.1-server-amd64.iso', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet10': {'macAddr': u'00:1a:4a:a8:64:1a', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '5.5', 'rxRate': '0.1', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet10'}}, 'vmId': 'dab1628d-5a54-49ba-9685-47e4c156842c', 'monitorResponse': '0', 'cpuUser': '135.12', 'disks': {u'vda': {'readLatency': '1804444', 'apparentsize': '107374182400', 'writeLatency': '1966966', 'imageID': '089b08fc-6544-4235-a061-fb55b09d2715', 'flushLatency': '260155', 'readRate': '62895427.63', 'truesize': '107374182400', 'writeRate': '321958.61'}}, 'boot': 'cd', 'statsAge': '0.95', 'cpuIdle': '0.00', 'elapsedTime': '1351', 'vmType': 'kvm', 'cpuSys': '0.00', 'timeOffset': -1L, 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15915884::DEBUG::2013-11-20 04:11:50,105::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('01c8c942-dab4-4e5e-aefa-d474a455143e',) {}
>Thread-15915884::DEBUG::2013-11-20 04:11:50,106::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::Disk stats not available
>Thread-15915884::DEBUG::2013-11-20 04:11:50,107::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::Disk latency not available
>Thread-15915884::DEBUG::2013-11-20 04:11:50,108::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '43977', 'displayIp': '0', 'displayPort': u'5920', 'session': 'Unknown', 'displaySecurePort': u'5921', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet11': {'macAddr': u'00:1a:4a:a8:64:0f', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet11'}}, 'vmId': '01c8c942-dab4-4e5e-aefa-d474a455143e', 'monitorResponse': '0', 'cpuUser': '59.62', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '107776835584', 'writeLatency': '0', 'imageID': 'c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e', 'flushLatency': '0', 'truesize': '107776835584'}}, 'boot': 'c', 'statsAge': '1.79', 'cpuIdle': '40.38', 'elapsedTime': '29', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 04:11:50,133::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915885::DEBUG::2013-11-20 04:11:50,473::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915885::INFO::2013-11-20 04:11:50,475::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915885::DEBUG::2013-11-20 04:11:50,476::task::495::TaskManager.Task::(_debug) Task dc1ab564-623d-42df-a0e3-53f33a5947e4: moving from state init -> state preparing
>Thread-15915885::DEBUG::2013-11-20 04:11:50,477::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915885::DEBUG::2013-11-20 04:11:50,478::task::495::TaskManager.Task::(_debug) Task dc1ab564-623d-42df-a0e3-53f33a5947e4: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915885::DEBUG::2013-11-20 04:11:50,479::task::495::TaskManager.Task::(_debug) Task dc1ab564-623d-42df-a0e3-53f33a5947e4: moving from state preparing -> state finished
>Thread-15915885::DEBUG::2013-11-20 04:11:50,479::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915885::DEBUG::2013-11-20 04:11:50,480::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915885::DEBUG::2013-11-20 04:11:50,481::task::495::TaskManager.Task::(_debug) Task dc1ab564-623d-42df-a0e3-53f33a5947e4: ref 0 aborting False
>Thread-15915885::INFO::2013-11-20 04:11:50,481::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915886::DEBUG::2013-11-20 04:11:50,487::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915886::INFO::2013-11-20 04:11:50,488::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915886::DEBUG::2013-11-20 04:11:50,489::task::495::TaskManager.Task::(_debug) Task 3bf6093a-5d83-4b54-a9b5-74d3e75ff4d1: moving from state init -> state preparing
>Thread-15915886::DEBUG::2013-11-20 04:11:50,491::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0747d483-0e7d-4c29-8600-fc72e1fbadf4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915886::DEBUG::2013-11-20 04:11:50,491::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915886::DEBUG::2013-11-20 04:11:50,492::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915886::DEBUG::2013-11-20 04:11:50,493::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0747d483-0e7d-4c29-8600-fc72e1fbadf4`::Granted request
>Thread-15915886::DEBUG::2013-11-20 04:11:50,494::task::495::TaskManager.Task::(_debug) Task 3bf6093a-5d83-4b54-a9b5-74d3e75ff4d1: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915886::DEBUG::2013-11-20 04:11:50,494::task::495::TaskManager.Task::(_debug) Task 3bf6093a-5d83-4b54-a9b5-74d3e75ff4d1: ref 1 aborting False
>Thread-15915886::WARNING::2013-11-20 04:11:50,497::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915886::DEBUG::2013-11-20 04:11:50,498::task::495::TaskManager.Task::(_debug) Task 3bf6093a-5d83-4b54-a9b5-74d3e75ff4d1: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915886::DEBUG::2013-11-20 04:11:50,499::task::495::TaskManager.Task::(_debug) Task 3bf6093a-5d83-4b54-a9b5-74d3e75ff4d1: moving from state preparing -> state finished
>Thread-15915886::DEBUG::2013-11-20 04:11:50,499::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915886::DEBUG::2013-11-20 04:11:50,500::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915886::DEBUG::2013-11-20 04:11:50,501::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915886::DEBUG::2013-11-20 04:11:50,501::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915886::DEBUG::2013-11-20 04:11:50,502::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915886::DEBUG::2013-11-20 04:11:50,502::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915886::DEBUG::2013-11-20 04:11:50,503::task::495::TaskManager.Task::(_debug) Task 3bf6093a-5d83-4b54-a9b5-74d3e75ff4d1: ref 0 aborting False
>Thread-15915886::INFO::2013-11-20 04:11:50,504::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:11:50,738::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.087762 s, 11.7 MB/s\n'; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-20 04:11:51,398::libvirtvm::1401::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::event Saved detail 0 opaque None
>Thread-15915086::ERROR::2013-11-20 04:11:51,734::guestIF::246::vm.Vm::(run) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Run exception: None
>Traceback (most recent call last):
>  File "/usr/share/vdsm/guestIF.py", line 238, in run
>  File "/usr/share/vdsm/guestIF.py", line 214, in _readLine
>  File "/usr/share/vdsm/guestIF.py", line 200, in _readBuffer
>error: [Errno 104] Connection reset by peer
>libvirtEventLoop::DEBUG::2013-11-20 04:11:52,263::libvirtvm::1401::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::event Stopped detail 0 opaque None
>libvirtEventLoop::INFO::2013-11-20 04:11:52,274::vm::445::vm.Vm::(_onQemuDeath) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::underlying process disconnected
>libvirtEventLoop::INFO::2013-11-20 04:11:52,275::libvirtvm::1304::vm.Vm::(releaseVm) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Release VM resources
>libvirtEventLoop::DEBUG::2013-11-20 04:11:52,280::utils::314::vm.Vm::(stop) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Stop statistics collection
>Thread-15915085::DEBUG::2013-11-20 04:11:52,282::utils::345::vm.Vm::(run) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Stats thread finished
>libvirtEventLoop::WARNING::2013-11-20 04:11:52,284::libvirtvm::1315::vm.Vm::(releaseVm) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Traceback (most recent call last):
>  File "/usr/share/vdsm/libvirtvm.py", line 1312, in releaseVm
>  File "/usr/share/vdsm/libvirtvm.py", line 445, in f
>  File "/usr/share/vdsm/libvirtconnection.py", line 63, in wrapper
>  File "/usr/lib64/python2.6/site-packages/libvirt.py", line 672, in destroy
>libvirtError: Domain not found: no domain with matching uuid 'dab1628d-5a54-49ba-9685-47e4c156842c'
>
>libvirtEventLoop::DEBUG::2013-11-20 04:11:52,285::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /sbin/service ksmtuned retune' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:11:52,755::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>libvirtEventLoop::DEBUG::2013-11-20 04:11:52,757::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-20 04:11:52,977::vm::718::vm.Vm::(_cleanup) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Drive /rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/ubuntu-12.04.1-server-amd64.iso cleanup
>libvirtEventLoop::DEBUG::2013-11-20 04:11:52,978::vm::718::vm.Vm::(_cleanup) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Drive /rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/089b08fc-6544-4235-a061-fb55b09d2715/66b599fc-b7b5-4934-a5d5-ef7283460029 cleanup
>libvirtEventLoop::INFO::2013-11-20 04:11:52,980::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: teardownVolume, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=089b08fc-6544-4235-a061-fb55b09d2715 volUUID=66b599fc-b7b5-4934-a5d5-ef7283460029)
>libvirtEventLoop::DEBUG::2013-11-20 04:11:52,980::task::495::TaskManager.Task::(_debug) Task d35f0d0e-a3d1-4801-8cf0-a4809166960d: moving from state init -> state preparing
>libvirtEventLoop::DEBUG::2013-11-20 04:11:52,982::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`57053c0e-35ab-4e4c-af64-e358ef6254ff`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1727' at 'public_teardownVolume'
>libvirtEventLoop::DEBUG::2013-11-20 04:11:52,982::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>libvirtEventLoop::DEBUG::2013-11-20 04:11:52,984::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>libvirtEventLoop::DEBUG::2013-11-20 04:11:52,984::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`57053c0e-35ab-4e4c-af64-e358ef6254ff`::Granted request
>libvirtEventLoop::DEBUG::2013-11-20 04:11:52,985::task::495::TaskManager.Task::(_debug) Task d35f0d0e-a3d1-4801-8cf0-a4809166960d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>libvirtEventLoop::DEBUG::2013-11-20 04:11:52,986::task::495::TaskManager.Task::(_debug) Task d35f0d0e-a3d1-4801-8cf0-a4809166960d: ref 1 aborting False
>libvirtEventLoop::DEBUG::2013-11-20 04:11:52,987::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.089b08fc-6544-4235-a061-fb55b09d2715'
>libvirtEventLoop::DEBUG::2013-11-20 04:11:52,988::resourceManager::530::ResourceManager::(releaseResource) Released resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.089b08fc-6544-4235-a061-fb55b09d2715' (0 active users)
>libvirtEventLoop::DEBUG::2013-11-20 04:11:52,988::resourceManager::535::ResourceManager::(releaseResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.089b08fc-6544-4235-a061-fb55b09d2715' is free, finding out if anyone is waiting for it.
>libvirtEventLoop::DEBUG::2013-11-20 04:11:52,988::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.66b599fc-b7b5-4934-a5d5-ef7283460029'
>libvirtEventLoop::DEBUG::2013-11-20 04:11:52,989::resourceManager::530::ResourceManager::(releaseResource) Released resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.66b599fc-b7b5-4934-a5d5-ef7283460029' (0 active users)
>libvirtEventLoop::DEBUG::2013-11-20 04:11:52,989::resourceManager::535::ResourceManager::(releaseResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.66b599fc-b7b5-4934-a5d5-ef7283460029' is free, finding out if anyone is waiting for it.
>libvirtEventLoop::DEBUG::2013-11-20 04:11:52,990::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.66b599fc-b7b5-4934-a5d5-ef7283460029', Clearing records.
>libvirtEventLoop::DEBUG::2013-11-20 04:11:52,990::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.089b08fc-6544-4235-a061-fb55b09d2715', Clearing records.
>libvirtEventLoop::INFO::2013-11-20 04:11:52,991::blockVolume::386::Storage.Volume::(teardown) Tearing down volume 6b892067-ac7e-4039-a7be-5b7387525b16/66b599fc-b7b5-4934-a5d5-ef7283460029 justme False
>libvirtEventLoop::DEBUG::2013-11-20 04:11:52,991::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.66b599fc-b7b5-4934-a5d5-ef7283460029'
>Thread-15913695::INFO::2013-11-20 04:11:52,993::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>libvirtEventLoop::DEBUG::2013-11-20 04:11:52,993::resourceManager::530::ResourceManager::(releaseResource) Released resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.66b599fc-b7b5-4934-a5d5-ef7283460029' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:11:52,993::task::495::TaskManager.Task::(_debug) Task 9dc9bbb5-7a84-42ec-b846-cd1f2a924f7c: moving from state init -> state preparing
>libvirtEventLoop::DEBUG::2013-11-20 04:11:52,994::resourceManager::535::ResourceManager::(releaseResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.66b599fc-b7b5-4934-a5d5-ef7283460029' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:11:52,995::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c72483ad-ab43-4d35-b9ea-7422828c688f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>libvirtEventLoop::DEBUG::2013-11-20 04:11:52,997::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --autobackup n --available n 6b892067-ac7e-4039-a7be-5b7387525b16/66b599fc-b7b5-4934-a5d5-ef7283460029' (cwd None)
>Thread-15913695::DEBUG::2013-11-20 04:11:52,997::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:11:52,999::resourceManager::481::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' found in shared state and queue is empty, Joining current shared lock (2 active users)
>Thread-15913695::DEBUG::2013-11-20 04:11:53,358::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c72483ad-ab43-4d35-b9ea-7422828c688f`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:11:53,365::task::495::TaskManager.Task::(_debug) Task 9dc9bbb5-7a84-42ec-b846-cd1f2a924f7c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:11:53,367::task::495::TaskManager.Task::(_debug) Task 9dc9bbb5-7a84-42ec-b846-cd1f2a924f7c: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:11:53,368::task::495::TaskManager.Task::(_debug) Task 9dc9bbb5-7a84-42ec-b846-cd1f2a924f7c: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:11:53,369::task::495::TaskManager.Task::(_debug) Task 9dc9bbb5-7a84-42ec-b846-cd1f2a924f7c: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:11:53,370::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:11:53,371::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:11:53,372::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:11:53,374::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (1 active users)
>Thread-15913695::DEBUG::2013-11-20 04:11:53,375::task::495::TaskManager.Task::(_debug) Task 9dc9bbb5-7a84-42ec-b846-cd1f2a924f7c: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:11:53,379::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15915888::DEBUG::2013-11-20 04:11:53,460::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('01c8c942-dab4-4e5e-aefa-d474a455143e',) {}
>Thread-15915888::DEBUG::2013-11-20 04:11:53,462::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::Disk stats not available
>Thread-15915888::DEBUG::2013-11-20 04:11:53,464::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::Disk latency not available
>Thread-15915888::DEBUG::2013-11-20 04:11:53,465::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '43977', 'displayIp': '0', 'displayPort': u'5920', 'session': 'Unknown', 'displaySecurePort': u'5921', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet11': {'macAddr': u'00:1a:4a:a8:64:0f', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet11'}}, 'vmId': '01c8c942-dab4-4e5e-aefa-d474a455143e', 'monitorResponse': '0', 'cpuUser': '59.62', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '107776835584', 'writeLatency': '0', 'imageID': 'c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e', 'flushLatency': '0', 'truesize': '107776835584'}}, 'boot': 'c', 'statsAge': '0.98', 'cpuIdle': '40.38', 'elapsedTime': '33', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 04:11:53,611::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.241121 s, 4.2 MB/s\n'; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-20 04:11:53,965::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-20 04:11:53,967::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>libvirtEventLoop::DEBUG::2013-11-20 04:11:53,967::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>libvirtEventLoop::DEBUG::2013-11-20 04:11:53,968::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.66b599fc-b7b5-4934-a5d5-ef7283460029', Clearing records.
>libvirtEventLoop::DEBUG::2013-11-20 04:11:53,969::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>libvirtEventLoop::DEBUG::2013-11-20 04:11:53,970::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>libvirtEventLoop::DEBUG::2013-11-20 04:11:54,633::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-20 04:11:54,887::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>libvirtEventLoop::DEBUG::2013-11-20 04:11:54,888::task::495::TaskManager.Task::(_debug) Task d35f0d0e-a3d1-4801-8cf0-a4809166960d: finished: None
>libvirtEventLoop::DEBUG::2013-11-20 04:11:54,888::task::495::TaskManager.Task::(_debug) Task d35f0d0e-a3d1-4801-8cf0-a4809166960d: moving from state preparing -> state finished
>libvirtEventLoop::DEBUG::2013-11-20 04:11:54,889::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>libvirtEventLoop::DEBUG::2013-11-20 04:11:54,889::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>libvirtEventLoop::DEBUG::2013-11-20 04:11:54,890::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>libvirtEventLoop::DEBUG::2013-11-20 04:11:54,890::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>libvirtEventLoop::DEBUG::2013-11-20 04:11:54,890::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>libvirtEventLoop::DEBUG::2013-11-20 04:11:54,891::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>libvirtEventLoop::DEBUG::2013-11-20 04:11:54,891::task::495::TaskManager.Task::(_debug) Task d35f0d0e-a3d1-4801-8cf0-a4809166960d: ref 0 aborting False
>libvirtEventLoop::INFO::2013-11-20 04:11:54,892::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: teardownVolume, Return response: {'status': {'message': 'OK', 'code': 0}}
>libvirtEventLoop::DEBUG::2013-11-20 04:11:54,893::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/libexec/vdsm/hooks/after_vm_destroy/50_vhostmd' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:11:55,626::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>libvirtEventLoop::DEBUG::2013-11-20 04:11:55,988::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>libvirtEventLoop::INFO::2013-11-20 04:11:55,991::hooks::51::root::(_runHooksDir) 
>libvirtEventLoop::DEBUG::2013-11-20 04:11:55,998::vm::742::vm.Vm::(setDownStatus) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Changed state to Down: User shut down
>libvirtEventLoop::DEBUG::2013-11-20 04:11:55,999::utils::314::vm.Vm::(stop) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Stop statistics collection
>Dummy-15912651::DEBUG::2013-11-20 04:11:56,226::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.118206 s, 8.7 MB/s\n'; <rc> = 0
>Thread-15915891::DEBUG::2013-11-20 04:11:56,506::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('dab1628d-5a54-49ba-9685-47e4c156842c',) {}
>Thread-15915891::DEBUG::2013-11-20 04:11:56,507::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Down', 'timeOffset': -1L, 'vmId': 'dab1628d-5a54-49ba-9685-47e4c156842c', 'exitMessage': 'User shut down', 'exitCode': 0}]}
>Thread-15915892::DEBUG::2013-11-20 04:11:56,513::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('01c8c942-dab4-4e5e-aefa-d474a455143e',) {}
>Thread-15915892::DEBUG::2013-11-20 04:11:56,514::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::Disk stats not available
>Thread-15915892::DEBUG::2013-11-20 04:11:56,515::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::Disk latency not available
>Thread-15915892::DEBUG::2013-11-20 04:11:56,516::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '43977', 'displayIp': '0', 'displayPort': u'5920', 'session': 'Unknown', 'displaySecurePort': u'5921', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet11': {'macAddr': u'00:1a:4a:a8:64:0f', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet11'}}, 'vmId': '01c8c942-dab4-4e5e-aefa-d474a455143e', 'monitorResponse': '0', 'cpuUser': '59.62', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '107776835584', 'writeLatency': '0', 'imageID': 'c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e', 'flushLatency': '0', 'truesize': '107776835584'}}, 'boot': 'c', 'statsAge': '0.03', 'cpuIdle': '40.38', 'elapsedTime': '36', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15915893::DEBUG::2013-11-20 04:11:56,708::clientIF::54::vds::(wrapper) [192.168.100.135]::call destroy with ('dab1628d-5a54-49ba-9685-47e4c156842c',) {}
>Thread-15915893::INFO::2013-11-20 04:11:56,709::clientIF::450::vds::(destroy) vmContainerLock acquired by vm dab1628d-5a54-49ba-9685-47e4c156842c
>Thread-15915893::DEBUG::2013-11-20 04:11:56,710::libvirtvm::1344::vm.Vm::(destroy) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::destroy Called
>Thread-15915893::DEBUG::2013-11-20 04:11:56,711::libvirtvm::1339::vm.Vm::(deleteVm) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Total desktops after destroy of dab1628d-5a54-49ba-9685-47e4c156842c is 10
>Thread-15915893::DEBUG::2013-11-20 04:11:56,712::clientIF::59::vds::(wrapper) return destroy with {'status': {'message': 'Machine destroyed', 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:11:58,240::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:11:58,742::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0781521 s, 13.1 MB/s\n'; <rc> = 0
>Thread-15914453::INFO::2013-11-20 04:11:59,358::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:11:59,361::task::495::TaskManager.Task::(_debug) Task 99998a17-5952-45d2-be8b-4d20cf2324b4: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:11:59,364::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c80548c1-a422-459b-bed7-51744dc0ca9f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:11:59,365::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:11:59,367::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:11:59,368::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c80548c1-a422-459b-bed7-51744dc0ca9f`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:11:59,369::task::495::TaskManager.Task::(_debug) Task 99998a17-5952-45d2-be8b-4d20cf2324b4: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:11:59,369::task::495::TaskManager.Task::(_debug) Task 99998a17-5952-45d2-be8b-4d20cf2324b4: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:11:59,370::task::495::TaskManager.Task::(_debug) Task 99998a17-5952-45d2-be8b-4d20cf2324b4: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:11:59,371::task::495::TaskManager.Task::(_debug) Task 99998a17-5952-45d2-be8b-4d20cf2324b4: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:11:59,371::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:11:59,372::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:11:59,373::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:11:59,373::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:11:59,374::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:11:59,374::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:11:59,375::task::495::TaskManager.Task::(_debug) Task 99998a17-5952-45d2-be8b-4d20cf2324b4: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:11:59,376::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15915895::DEBUG::2013-11-20 04:12:00,565::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('01c8c942-dab4-4e5e-aefa-d474a455143e',) {}
>Thread-15915895::DEBUG::2013-11-20 04:12:00,566::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::Disk stats not available
>Thread-15915895::DEBUG::2013-11-20 04:12:00,566::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::Disk latency not available
>Thread-15915895::DEBUG::2013-11-20 04:12:00,567::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '43977', 'displayIp': '0', 'displayPort': u'5920', 'session': 'Unknown', 'displaySecurePort': u'5921', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet11': {'macAddr': u'00:1a:4a:a8:64:0f', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet11'}}, 'vmId': '01c8c942-dab4-4e5e-aefa-d474a455143e', 'monitorResponse': '0', 'cpuUser': '59.62', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '107776835584', 'writeLatency': '0', 'imageID': 'c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e', 'flushLatency': '0', 'truesize': '107776835584'}}, 'boot': 'c', 'statsAge': '0.08', 'cpuIdle': '40.38', 'elapsedTime': '40', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 04:12:00,758::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915896::DEBUG::2013-11-20 04:12:00,974::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915896::INFO::2013-11-20 04:12:00,977::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915896::DEBUG::2013-11-20 04:12:00,978::task::495::TaskManager.Task::(_debug) Task 88aa0420-6a2e-44c4-bf12-e5c4873eb987: moving from state init -> state preparing
>Thread-15915896::DEBUG::2013-11-20 04:12:00,979::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915896::DEBUG::2013-11-20 04:12:00,980::task::495::TaskManager.Task::(_debug) Task 88aa0420-6a2e-44c4-bf12-e5c4873eb987: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915896::DEBUG::2013-11-20 04:12:00,980::task::495::TaskManager.Task::(_debug) Task 88aa0420-6a2e-44c4-bf12-e5c4873eb987: moving from state preparing -> state finished
>Thread-15915896::DEBUG::2013-11-20 04:12:00,981::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915896::DEBUG::2013-11-20 04:12:00,982::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915896::DEBUG::2013-11-20 04:12:00,982::task::495::TaskManager.Task::(_debug) Task 88aa0420-6a2e-44c4-bf12-e5c4873eb987: ref 0 aborting False
>Thread-15915896::INFO::2013-11-20 04:12:00,983::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915897::DEBUG::2013-11-20 04:12:00,988::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915897::INFO::2013-11-20 04:12:00,989::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915897::DEBUG::2013-11-20 04:12:00,990::task::495::TaskManager.Task::(_debug) Task 3af47d2c-674d-4423-b757-135382a2b81d: moving from state init -> state preparing
>Thread-15915897::DEBUG::2013-11-20 04:12:00,992::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3b65742b-6191-451e-bb11-2885c20bec6d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915897::DEBUG::2013-11-20 04:12:00,993::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915897::DEBUG::2013-11-20 04:12:00,994::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915897::DEBUG::2013-11-20 04:12:00,994::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3b65742b-6191-451e-bb11-2885c20bec6d`::Granted request
>Thread-15915897::DEBUG::2013-11-20 04:12:00,996::task::495::TaskManager.Task::(_debug) Task 3af47d2c-674d-4423-b757-135382a2b81d: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915897::DEBUG::2013-11-20 04:12:00,997::task::495::TaskManager.Task::(_debug) Task 3af47d2c-674d-4423-b757-135382a2b81d: ref 1 aborting False
>Thread-15915897::WARNING::2013-11-20 04:12:01,001::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915897::DEBUG::2013-11-20 04:12:01,003::task::495::TaskManager.Task::(_debug) Task 3af47d2c-674d-4423-b757-135382a2b81d: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915897::DEBUG::2013-11-20 04:12:01,004::task::495::TaskManager.Task::(_debug) Task 3af47d2c-674d-4423-b757-135382a2b81d: moving from state preparing -> state finished
>Thread-15915897::DEBUG::2013-11-20 04:12:01,004::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915897::DEBUG::2013-11-20 04:12:01,005::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915897::DEBUG::2013-11-20 04:12:01,006::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915897::DEBUG::2013-11-20 04:12:01,007::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915897::DEBUG::2013-11-20 04:12:01,008::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915897::DEBUG::2013-11-20 04:12:01,008::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915897::DEBUG::2013-11-20 04:12:01,009::task::495::TaskManager.Task::(_debug) Task 3af47d2c-674d-4423-b757-135382a2b81d: ref 0 aborting False
>Thread-15915897::INFO::2013-11-20 04:12:01,010::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:12:01,337::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0758488 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:12:03,352::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915898::INFO::2013-11-20 04:12:03,694::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915898::DEBUG::2013-11-20 04:12:03,695::task::495::TaskManager.Task::(_debug) Task d40a557e-f472-4454-87c1-a12efe2c7fad: moving from state init -> state preparing
>Thread-15915898::DEBUG::2013-11-20 04:12:03,703::task::495::TaskManager.Task::(_debug) Task d40a557e-f472-4454-87c1-a12efe2c7fad: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000445127487183', 'lastCheck': 1384920715.17435, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00965595245361', 'lastCheck': 1384920715.186913, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0164790153503', 'lastCheck': 1384920721.990217, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000416040420532', 'lastCheck': 1384920719.840275, 'valid': True, 'code': 0}}
>Thread-15915898::DEBUG::2013-11-20 04:12:03,704::task::495::TaskManager.Task::(_debug) Task d40a557e-f472-4454-87c1-a12efe2c7fad: moving from state preparing -> state finished
>Thread-15915898::DEBUG::2013-11-20 04:12:03,705::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915898::DEBUG::2013-11-20 04:12:03,705::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915898::DEBUG::2013-11-20 04:12:03,706::task::495::TaskManager.Task::(_debug) Task d40a557e-f472-4454-87c1-a12efe2c7fad: ref 0 aborting False
>Thread-15915898::INFO::2013-11-20 04:12:03,706::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000445127487183', 'lastCheck': 1384920715.17435, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00965595245361', 'lastCheck': 1384920715.186913, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0164790153503', 'lastCheck': 1384920721.990217, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000416040420532', 'lastCheck': 1384920719.840275, 'valid': True, 'code': 0}}
>Thread-15915899::DEBUG::2013-11-20 04:12:03,760::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::Disk stats not available
>Thread-15915899::DEBUG::2013-11-20 04:12:03,761::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::Disk latency not available
>Dummy-15912651::DEBUG::2013-11-20 04:12:03,989::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0823342 s, 12.4 MB/s\n'; <rc> = 0
>Thread-15914519::INFO::2013-11-20 04:12:05,567::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:12:05,568::task::495::TaskManager.Task::(_debug) Task 5718f3f4-0cd5-4b75-9811-e17b5b18df14: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:12:05,573::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`536a6f9b-95f8-4192-ba92-5fb290501f2d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:12:05,574::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:12:05,575::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:12:05,575::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`536a6f9b-95f8-4192-ba92-5fb290501f2d`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:12:05,577::task::495::TaskManager.Task::(_debug) Task 5718f3f4-0cd5-4b75-9811-e17b5b18df14: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 04:12:05,577::task::495::TaskManager.Task::(_debug) Task 5718f3f4-0cd5-4b75-9811-e17b5b18df14: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:12:05,578::task::495::TaskManager.Task::(_debug) Task 5718f3f4-0cd5-4b75-9811-e17b5b18df14: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 04:12:05,578::task::495::TaskManager.Task::(_debug) Task 5718f3f4-0cd5-4b75-9811-e17b5b18df14: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:12:05,579::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:12:05,579::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:12:05,583::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 04:12:05,584::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:12:05,584::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:12:05,585::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:12:05,585::task::495::TaskManager.Task::(_debug) Task 5718f3f4-0cd5-4b75-9811-e17b5b18df14: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 04:12:05,586::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914435::INFO::2013-11-20 04:12:05,587::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:12:05,588::task::495::TaskManager.Task::(_debug) Task 8e7139a0-6329-4489-ba0a-e2e5cc1681bd: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:12:05,590::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fc0a8ae4-4b4b-4739-a71b-8b12f4615ef7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:12:05,590::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:12:05,591::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:12:05,592::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fc0a8ae4-4b4b-4739-a71b-8b12f4615ef7`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:12:05,756::task::495::TaskManager.Task::(_debug) Task 8e7139a0-6329-4489-ba0a-e2e5cc1681bd: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:12:05,758::task::495::TaskManager.Task::(_debug) Task 8e7139a0-6329-4489-ba0a-e2e5cc1681bd: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:12:05,761::task::495::TaskManager.Task::(_debug) Task 8e7139a0-6329-4489-ba0a-e2e5cc1681bd: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:12:05,762::task::495::TaskManager.Task::(_debug) Task 8e7139a0-6329-4489-ba0a-e2e5cc1681bd: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:12:05,762::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:12:05,763::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:12:05,764::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:12:05,764::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:12:05,765::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:12:05,765::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:12:05,766::task::495::TaskManager.Task::(_debug) Task 8e7139a0-6329-4489-ba0a-e2e5cc1681bd: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:12:05,766::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:12:06,003::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:12:06,514::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0452121 s, 22.6 MB/s\n'; <rc> = 0
>Thread-15915901::DEBUG::2013-11-20 04:12:07,165::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('01c8c942-dab4-4e5e-aefa-d474a455143e',) {}
>Thread-15915901::DEBUG::2013-11-20 04:12:07,166::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::Disk stats not available
>Thread-15915901::DEBUG::2013-11-20 04:12:07,167::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::Disk latency not available
>Thread-15915901::DEBUG::2013-11-20 04:12:07,168::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '43977', 'displayIp': '0', 'displayPort': u'5920', 'session': 'Unknown', 'displaySecurePort': u'5921', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet11': {'macAddr': u'00:1a:4a:a8:64:0f', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet11'}}, 'vmId': '01c8c942-dab4-4e5e-aefa-d474a455143e', 'monitorResponse': '0', 'cpuUser': '98.81', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '107776835584', 'writeLatency': '0', 'imageID': 'c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e', 'flushLatency': '0', 'truesize': '107776835584'}}, 'boot': 'c', 'statsAge': '0.67', 'cpuIdle': '1.19', 'elapsedTime': '47', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 04:12:08,524::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:12:09,037::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0832477 s, 12.3 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 04:12:09,570::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:12:09,572::task::495::TaskManager.Task::(_debug) Task 2a865b48-f0ee-4b8a-bccd-70398293cd7e: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:12:09,573::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`93e706a5-c0b6-410b-9fee-9f5b2891dc15`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:12:09,574::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:12:09,575::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:12:09,575::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`93e706a5-c0b6-410b-9fee-9f5b2891dc15`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:12:09,577::task::495::TaskManager.Task::(_debug) Task 2a865b48-f0ee-4b8a-bccd-70398293cd7e: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:12:09,577::task::495::TaskManager.Task::(_debug) Task 2a865b48-f0ee-4b8a-bccd-70398293cd7e: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:12:09,578::task::495::TaskManager.Task::(_debug) Task 2a865b48-f0ee-4b8a-bccd-70398293cd7e: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 04:12:09,578::task::495::TaskManager.Task::(_debug) Task 2a865b48-f0ee-4b8a-bccd-70398293cd7e: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:12:09,579::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:12:09,580::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:12:09,580::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:12:09,581::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:12:09,581::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:12:09,582::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:12:09,583::task::495::TaskManager.Task::(_debug) Task 2a865b48-f0ee-4b8a-bccd-70398293cd7e: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:12:09,583::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:12:09,584::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:12:09,585::task::495::TaskManager.Task::(_debug) Task b8b21e3c-638c-4937-a67a-d751e3d294d8: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:12:09,587::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`af2ae9bb-062d-4ed3-bbc0-0c6402dc6f84`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:12:09,587::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:12:09,588::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:12:09,588::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`af2ae9bb-062d-4ed3-bbc0-0c6402dc6f84`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:12:09,589::task::495::TaskManager.Task::(_debug) Task b8b21e3c-638c-4937-a67a-d751e3d294d8: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:12:09,591::task::495::TaskManager.Task::(_debug) Task b8b21e3c-638c-4937-a67a-d751e3d294d8: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:12:09,591::task::495::TaskManager.Task::(_debug) Task b8b21e3c-638c-4937-a67a-d751e3d294d8: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:12:09,592::task::495::TaskManager.Task::(_debug) Task b8b21e3c-638c-4937-a67a-d751e3d294d8: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:12:09,593::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:12:09,594::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:12:09,596::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:12:09,597::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:12:09,597::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:12:09,597::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:12:09,598::task::495::TaskManager.Task::(_debug) Task b8b21e3c-638c-4937-a67a-d751e3d294d8: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:12:09,598::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:12:09,599::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:12:09,600::task::495::TaskManager.Task::(_debug) Task 5939522b-e9ab-4050-b2d6-5ecd0b856a95: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:12:09,600::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8562eea4-8ea0-4051-ad6f-a3c2f75a3368`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:12:09,601::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:12:09,601::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:12:09,602::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8562eea4-8ea0-4051-ad6f-a3c2f75a3368`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:12:09,603::task::495::TaskManager.Task::(_debug) Task 5939522b-e9ab-4050-b2d6-5ecd0b856a95: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:12:09,604::task::495::TaskManager.Task::(_debug) Task 5939522b-e9ab-4050-b2d6-5ecd0b856a95: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:12:09,605::task::495::TaskManager.Task::(_debug) Task 5939522b-e9ab-4050-b2d6-5ecd0b856a95: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:12:09,606::task::495::TaskManager.Task::(_debug) Task 5939522b-e9ab-4050-b2d6-5ecd0b856a95: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:12:09,607::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:12:09,607::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:12:09,608::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:12:09,609::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:12:09,609::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:12:09,609::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:12:09,611::task::495::TaskManager.Task::(_debug) Task 5939522b-e9ab-4050-b2d6-5ecd0b856a95: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:12:09,611::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:12:09,612::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:12:09,613::task::495::TaskManager.Task::(_debug) Task ce89c343-5668-459f-a46d-035535cbd7c7: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:12:09,614::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`cc0559c6-1eef-4a5e-b754-78d868190ab6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:12:09,615::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:12:09,615::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:12:09,616::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`cc0559c6-1eef-4a5e-b754-78d868190ab6`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:12:09,618::task::495::TaskManager.Task::(_debug) Task ce89c343-5668-459f-a46d-035535cbd7c7: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:12:09,618::task::495::TaskManager.Task::(_debug) Task ce89c343-5668-459f-a46d-035535cbd7c7: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:12:09,619::task::495::TaskManager.Task::(_debug) Task ce89c343-5668-459f-a46d-035535cbd7c7: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:12:09,619::task::495::TaskManager.Task::(_debug) Task ce89c343-5668-459f-a46d-035535cbd7c7: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:12:09,619::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:12:09,620::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:12:09,620::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:12:09,621::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:12:09,621::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:12:09,622::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:12:09,622::task::495::TaskManager.Task::(_debug) Task ce89c343-5668-459f-a46d-035535cbd7c7: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:12:09,623::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15915903::DEBUG::2013-11-20 04:12:10,194::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('01c8c942-dab4-4e5e-aefa-d474a455143e',) {}
>Thread-15915903::DEBUG::2013-11-20 04:12:10,195::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::Disk stats not available
>Thread-15915903::DEBUG::2013-11-20 04:12:10,196::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::Disk latency not available
>Thread-15915903::DEBUG::2013-11-20 04:12:10,197::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '43977', 'displayIp': '0', 'displayPort': u'5920', 'session': 'Unknown', 'displaySecurePort': u'5921', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet11': {'macAddr': u'00:1a:4a:a8:64:0f', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet11'}}, 'vmId': '01c8c942-dab4-4e5e-aefa-d474a455143e', 'monitorResponse': '0', 'cpuUser': '98.81', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '107776835584', 'writeLatency': '0', 'imageID': 'c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e', 'flushLatency': '0', 'truesize': '107776835584'}}, 'boot': 'c', 'statsAge': '1.70', 'cpuIdle': '1.19', 'elapsedTime': '50', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15915904::DEBUG::2013-11-20 04:12:10,682::clientIF::54::vds::(wrapper) [192.168.100.135]::call create with ({'bridge': 'rhevm,rhevm', 'acpiEnable': 'true', 'emulatedMachine': 'rhel6.2.0', 'vmId': '2afdc298-b10b-4bd1-8df2-2a401efd4f9c', 'transparentHugePages': 'true', 'spiceSslCipherSuite': 'DEFAULT', 'cpuType': 'Westmere', 'smp': '1', 'macAddr': '00:1a:4a:a8:64:11,00:1a:4a:a8:64:12', 'boot': 'c', 'custom': {}, 'vmType': 'kvm', 'memSize': 512, 'smpCoresPerSocket': '1', 'vmName': 'tripoli', 'spiceMonitors': '1', 'nice': '0', 'drives': [{'index': '0', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'format': 'raw', 'volumeID': 'cd4f9324-c91d-405b-983f-62afa4daa8ad', 'imageID': 'e0628852-41b1-4535-8dda-ebc96cb00952', 'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'propagateErrors': 'off', 'if': 'ide'}], 'smartcardEnable': 'false', 'nicModel': 'pv,rtl8139', 'keyboardLayout': 'en-us', 'kvmEnable': 'true', 'pitReinjection': 'false', 'timeOffset': '0', 'spiceSecureChannels': 'smain,sinputs', 'display': 'qxl'},) {}
>Dummy-15912651::DEBUG::2013-11-20 04:12:11,053::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915904::INFO::2013-11-20 04:12:11,422::clientIF::523::vds::(_getNetworkIp) network None: using 0
>Thread-15915904::INFO::2013-11-20 04:12:11,430::clientIF::656::vds::(create) vmContainerLock acquired by vm 2afdc298-b10b-4bd1-8df2-2a401efd4f9c
>Thread-15915905::DEBUG::2013-11-20 04:12:11,445::vm::342::vm.Vm::(_startUnderlyingVm) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Start
>Thread-15915904::DEBUG::2013-11-20 04:12:11,447::clientIF::669::vds::(create) Total desktops after creation of 2afdc298-b10b-4bd1-8df2-2a401efd4f9c is 11
>Thread-15915905::DEBUG::2013-11-20 04:12:11,450::vm::346::vm.Vm::(_startUnderlyingVm) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::_ongoingCreations acquired
>Thread-15915904::DEBUG::2013-11-20 04:12:11,451::clientIF::59::vds::(wrapper) return create with {'status': {'message': 'Done', 'code': 0}, 'vmList': {'status': 'WaitForLaunch', 'bridge': 'rhevm,rhevm', 'acpiEnable': 'true', 'emulatedMachine': 'rhel6.2.0', 'vmId': '2afdc298-b10b-4bd1-8df2-2a401efd4f9c', 'pid': '0', 'drives': [{'index': '0', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'format': 'raw', 'volumeID': 'cd4f9324-c91d-405b-983f-62afa4daa8ad', 'imageID': 'e0628852-41b1-4535-8dda-ebc96cb00952', 'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'propagateErrors': 'off', 'if': 'ide'}], 'transparentHugePages': 'true', 'keyboardLayout': 'en-us', 'displayPort': '-1', 'displaySecurePort': '-1', 'spiceSslCipherSuite': 'DEFAULT', 'cpuType': 'Westmere', 'custom': {}, 'clientIp': '', 'nicModel': 'pv,rtl8139', 'macAddr': '00:1a:4a:a8:64:11,00:1a:4a:a8:64:12', 'kvmEnable': 'true', 'pitReinjection': 'false', 'boot': 'c', 'smp': '1', 'vmType': 'kvm', 'smartcardEnable': 'false', 'timeOffset': '0', 'memSize': 512, 'displayIp': '0', 'spiceSecureChannels': 'smain,sinputs', 'smpCoresPerSocket': '1', 'vmName': 'tripoli', 'display': 'qxl', 'spiceMonitors': '1', 'nice': '0'}}
>Thread-15915905::INFO::2013-11-20 04:12:11,453::libvirtvm::1061::vm.Vm::(_run) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::VM wrapper has started
>Thread-15915905::INFO::2013-11-20 04:12:11,461::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915907::DEBUG::2013-11-20 04:12:11,467::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915905::DEBUG::2013-11-20 04:12:11,475::task::495::TaskManager.Task::(_debug) Task 8b5ddd9b-ddc1-45ae-95be-c3ca93d9953a: moving from state init -> state preparing
>Thread-15915907::INFO::2013-11-20 04:12:11,476::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915907::DEBUG::2013-11-20 04:12:11,479::task::495::TaskManager.Task::(_debug) Task 2cdbbb8c-02a0-4515-830e-d0dcfac61149: moving from state init -> state preparing
>Thread-15915905::DEBUG::2013-11-20 04:12:11,480::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`092a6b3a-640b-4926-b55b-dfeaba6582f9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1690' at 'public_prepareVolume'
>Thread-15915907::DEBUG::2013-11-20 04:12:11,481::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915905::DEBUG::2013-11-20 04:12:11,481::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915907::DEBUG::2013-11-20 04:12:11,482::task::495::TaskManager.Task::(_debug) Task 2cdbbb8c-02a0-4515-830e-d0dcfac61149: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915907::DEBUG::2013-11-20 04:12:11,483::task::495::TaskManager.Task::(_debug) Task 2cdbbb8c-02a0-4515-830e-d0dcfac61149: moving from state preparing -> state finished
>Thread-15915905::DEBUG::2013-11-20 04:12:11,483::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915907::DEBUG::2013-11-20 04:12:11,483::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915905::DEBUG::2013-11-20 04:12:11,484::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`092a6b3a-640b-4926-b55b-dfeaba6582f9`::Granted request
>Thread-15915907::DEBUG::2013-11-20 04:12:11,485::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915905::DEBUG::2013-11-20 04:12:11,486::task::495::TaskManager.Task::(_debug) Task 8b5ddd9b-ddc1-45ae-95be-c3ca93d9953a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915907::DEBUG::2013-11-20 04:12:11,486::task::495::TaskManager.Task::(_debug) Task 2cdbbb8c-02a0-4515-830e-d0dcfac61149: ref 0 aborting False
>Thread-15915905::DEBUG::2013-11-20 04:12:11,486::task::495::TaskManager.Task::(_debug) Task 8b5ddd9b-ddc1-45ae-95be-c3ca93d9953a: ref 1 aborting False
>Thread-15915907::INFO::2013-11-20 04:12:11,487::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915905::DEBUG::2013-11-20 04:12:11,488::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.e0628852-41b1-4535-8dda-ebc96cb00952`ReqID=`b8b349e2-d43f-4f6b-99ab-662b0149a58f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1695' at 'public_prepareVolume'
>Thread-15915905::DEBUG::2013-11-20 04:12:11,488::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.e0628852-41b1-4535-8dda-ebc96cb00952' for lock type 'exclusive'
>Thread-15915905::DEBUG::2013-11-20 04:12:11,490::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15915905::DEBUG::2013-11-20 04:12:11,498::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15915908::DEBUG::2013-11-20 04:12:11,790::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915908::INFO::2013-11-20 04:12:11,797::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915908::DEBUG::2013-11-20 04:12:11,800::task::495::TaskManager.Task::(_debug) Task e22de0a0-055b-44d1-b41f-9b5e83af34e5: moving from state init -> state preparing
>Thread-15915908::DEBUG::2013-11-20 04:12:11,803::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c43436fa-8df5-4f98-a153-41c21fe05c77`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915908::DEBUG::2013-11-20 04:12:11,804::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915908::DEBUG::2013-11-20 04:12:11,805::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915908::DEBUG::2013-11-20 04:12:11,806::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c43436fa-8df5-4f98-a153-41c21fe05c77`::Granted request
>Thread-15915908::DEBUG::2013-11-20 04:12:11,808::task::495::TaskManager.Task::(_debug) Task e22de0a0-055b-44d1-b41f-9b5e83af34e5: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915908::DEBUG::2013-11-20 04:12:11,809::task::495::TaskManager.Task::(_debug) Task e22de0a0-055b-44d1-b41f-9b5e83af34e5: ref 1 aborting False
>Thread-15915908::WARNING::2013-11-20 04:12:11,813::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915908::DEBUG::2013-11-20 04:12:11,815::task::495::TaskManager.Task::(_debug) Task e22de0a0-055b-44d1-b41f-9b5e83af34e5: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915908::DEBUG::2013-11-20 04:12:11,816::task::495::TaskManager.Task::(_debug) Task e22de0a0-055b-44d1-b41f-9b5e83af34e5: moving from state preparing -> state finished
>Thread-15915908::DEBUG::2013-11-20 04:12:11,818::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915908::DEBUG::2013-11-20 04:12:11,818::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915908::DEBUG::2013-11-20 04:12:11,819::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915908::DEBUG::2013-11-20 04:12:11,820::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915908::DEBUG::2013-11-20 04:12:11,821::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915908::DEBUG::2013-11-20 04:12:11,822::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915908::DEBUG::2013-11-20 04:12:11,823::task::495::TaskManager.Task::(_debug) Task e22de0a0-055b-44d1-b41f-9b5e83af34e5: ref 0 aborting False
>Thread-15915908::INFO::2013-11-20 04:12:11,824::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:12:11,974::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.170387 s, 6.0 MB/s\n'; <rc> = 0
>Thread-15915905::DEBUG::2013-11-20 04:12:12,464::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15915905::DEBUG::2013-11-20 04:12:13,680::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15915905::DEBUG::2013-11-20 04:12:13,688::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=28 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:12:13,996::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915905::DEBUG::2013-11-20 04:12:14,222::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.00013947 s, 3.7 MB/s\n'; <rc> = 0
>Thread-15915905::DEBUG::2013-11-20 04:12:14,224::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.00013947 s, 3.7 MB/s'], size: 512
>Thread-15915905::DEBUG::2013-11-20 04:12:14,227::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=28 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15915910::DEBUG::2013-11-20 04:12:14,228::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('01c8c942-dab4-4e5e-aefa-d474a455143e',) {}
>Thread-15915910::DEBUG::2013-11-20 04:12:14,615::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::Disk stats not available
>Thread-15915910::DEBUG::2013-11-20 04:12:14,632::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::Disk latency not available
>Thread-15915910::DEBUG::2013-11-20 04:12:14,635::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '43977', 'displayIp': '0', 'displayPort': u'5920', 'session': 'Unknown', 'displaySecurePort': u'5921', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet11': {'macAddr': u'00:1a:4a:a8:64:0f', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet11'}}, 'vmId': '01c8c942-dab4-4e5e-aefa-d474a455143e', 'monitorResponse': '0', 'cpuUser': '98.81', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '107776835584', 'writeLatency': '0', 'imageID': 'c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e', 'flushLatency': '0', 'truesize': '107776835584'}}, 'boot': 'c', 'statsAge': '1.96', 'cpuIdle': '1.19', 'elapsedTime': '54', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15915905::DEBUG::2013-11-20 04:12:14,765::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000194181 s, 2.6 MB/s\n'; <rc> = 0
>Thread-15915905::DEBUG::2013-11-20 04:12:14,768::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000194181 s, 2.6 MB/s'], size: 512
>Thread-15915905::INFO::2013-11-20 04:12:14,769::image::340::Storage.Image::(getChain) sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 chain=[<storage.blockVolume.BlockVolume instance at 0x7f48ac38fb48>] 
>Thread-15915905::DEBUG::2013-11-20 04:12:14,772::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --autobackup n --available y 6b892067-ac7e-4039-a7be-5b7387525b16/cd4f9324-c91d-405b-983f-62afa4daa8ad' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:12:15,006::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.176972 s, 5.8 MB/s\n'; <rc> = 0
>Thread-15915905::DEBUG::2013-11-20 04:12:15,797::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15915905::DEBUG::2013-11-20 04:12:15,799::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15915905::DEBUG::2013-11-20 04:12:15,800::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15915905::DEBUG::2013-11-20 04:12:15,802::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.cd4f9324-c91d-405b-983f-62afa4daa8ad`ReqID=`5179c534-8a58-4721-aa61-456a48841e6a`::Request was made in '/usr/share/vdsm/storage/resourceFactories.py' line '149' at '__getResourceCandidatesList'
>Thread-15915905::DEBUG::2013-11-20 04:12:15,803::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.cd4f9324-c91d-405b-983f-62afa4daa8ad' for lock type 'exclusive'
>Thread-15915905::DEBUG::2013-11-20 04:12:15,803::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.cd4f9324-c91d-405b-983f-62afa4daa8ad' is free. Now locking as 'exclusive' (1 active user)
>Thread-15915905::DEBUG::2013-11-20 04:12:15,804::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.cd4f9324-c91d-405b-983f-62afa4daa8ad`ReqID=`5179c534-8a58-4721-aa61-456a48841e6a`::Granted request
>Thread-15915905::DEBUG::2013-11-20 04:12:15,806::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.e0628852-41b1-4535-8dda-ebc96cb00952' is free. Now locking as 'exclusive' (1 active user)
>Thread-15915905::DEBUG::2013-11-20 04:12:15,806::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.e0628852-41b1-4535-8dda-ebc96cb00952`ReqID=`b8b349e2-d43f-4f6b-99ab-662b0149a58f`::Granted request
>Thread-15915905::DEBUG::2013-11-20 04:12:15,809::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15915905::DEBUG::2013-11-20 04:12:15,814::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15915905::DEBUG::2013-11-20 04:12:16,866::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:12:17,022::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915912::DEBUG::2013-11-20 04:12:18,187::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('01c8c942-dab4-4e5e-aefa-d474a455143e',) {}
>Thread-15915912::DEBUG::2013-11-20 04:12:18,212::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::Disk stats not available
>Thread-15915912::DEBUG::2013-11-20 04:12:18,244::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::Disk latency not available
>Thread-15915905::DEBUG::2013-11-20 04:12:18,258::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15915912::DEBUG::2013-11-20 04:12:18,259::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '43977', 'displayIp': '0', 'displayPort': u'5920', 'session': 'Unknown', 'displaySecurePort': u'5921', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet11': {'macAddr': u'00:1a:4a:a8:64:0f', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet11'}}, 'vmId': '01c8c942-dab4-4e5e-aefa-d474a455143e', 'monitorResponse': '0', 'cpuUser': '207.80', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '107776835584', 'writeLatency': '0', 'imageID': 'c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e', 'flushLatency': '0', 'truesize': '107776835584'}}, 'boot': 'c', 'statsAge': '0.43', 'cpuIdle': '0.00', 'elapsedTime': '58', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15915905::DEBUG::2013-11-20 04:12:18,264::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --refresh 6b892067-ac7e-4039-a7be-5b7387525b16/cd4f9324-c91d-405b-983f-62afa4daa8ad' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:12:18,570::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.645167 s, 1.6 MB/s\n'; <rc> = 0
>Thread-15915905::DEBUG::2013-11-20 04:12:19,257::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15915905::DEBUG::2013-11-20 04:12:19,258::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15915905::DEBUG::2013-11-20 04:12:19,259::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15915905::INFO::2013-11-20 04:12:19,259::volume::517::Storage.Volume::(prepare) Volume: preparing volume 6b892067-ac7e-4039-a7be-5b7387525b16/cd4f9324-c91d-405b-983f-62afa4daa8ad
>Thread-15915905::DEBUG::2013-11-20 04:12:19,260::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15915905::DEBUG::2013-11-20 04:12:19,260::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15915905::DEBUG::2013-11-20 04:12:19,991::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:12:20,754::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915905::DEBUG::2013-11-20 04:12:20,840::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15915905::DEBUG::2013-11-20 04:12:21,087::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=28 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15915913::INFO::2013-11-20 04:12:21,505::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915913::DEBUG::2013-11-20 04:12:21,507::task::495::TaskManager.Task::(_debug) Task f3794010-2959-461b-a17c-97149772fc3e: moving from state init -> state preparing
>Thread-15915913::DEBUG::2013-11-20 04:12:21,508::task::495::TaskManager.Task::(_debug) Task f3794010-2959-461b-a17c-97149772fc3e: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000357866287231', 'lastCheck': 1384920735.5650611, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00116515159607', 'lastCheck': 1384920735.5753, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00101399421692', 'lastCheck': 1384920732.007268, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0363399982452', 'lastCheck': 1384920740.318563, 'valid': True, 'code': 0}}
>Thread-15915913::DEBUG::2013-11-20 04:12:21,509::task::495::TaskManager.Task::(_debug) Task f3794010-2959-461b-a17c-97149772fc3e: moving from state preparing -> state finished
>Thread-15915913::DEBUG::2013-11-20 04:12:21,510::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915913::DEBUG::2013-11-20 04:12:21,511::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915913::DEBUG::2013-11-20 04:12:21,512::task::495::TaskManager.Task::(_debug) Task f3794010-2959-461b-a17c-97149772fc3e: ref 0 aborting False
>Thread-15915913::INFO::2013-11-20 04:12:21,513::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000357866287231', 'lastCheck': 1384920735.5650611, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00116515159607', 'lastCheck': 1384920735.5753, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00101399421692', 'lastCheck': 1384920732.007268, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0363399982452', 'lastCheck': 1384920740.318563, 'valid': True, 'code': 0}}
>Thread-15915914::DEBUG::2013-11-20 04:12:21,538::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::Disk stats not available
>Thread-15915914::DEBUG::2013-11-20 04:12:21,540::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::Disk latency not available
>Thread-15915905::DEBUG::2013-11-20 04:12:21,689::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000152662 s, 3.4 MB/s\n'; <rc> = 0
>Thread-15915905::DEBUG::2013-11-20 04:12:21,691::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000152662 s, 3.4 MB/s'], size: 512
>Thread-15915905::DEBUG::2013-11-20 04:12:21,697::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=28 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:12:21,919::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.16277 s, 6.3 MB/s\n'; <rc> = 0
>Thread-15915905::DEBUG::2013-11-20 04:12:22,127::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.00021088 s, 2.4 MB/s\n'; <rc> = 0
>Thread-15915905::DEBUG::2013-11-20 04:12:22,128::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.00021088 s, 2.4 MB/s'], size: 512
>Thread-15915905::DEBUG::2013-11-20 04:12:22,130::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.cd4f9324-c91d-405b-983f-62afa4daa8ad`ReqID=`352bc2cd-c72a-4be9-ba69-3d27805ba3c6`::Request was made in '/usr/share/vdsm/storage/volume.py' line '533' at 'prepare'
>Thread-15915905::DEBUG::2013-11-20 04:12:22,131::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.cd4f9324-c91d-405b-983f-62afa4daa8ad' for lock type 'exclusive'
>Thread-15915905::DEBUG::2013-11-20 04:12:22,132::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.cd4f9324-c91d-405b-983f-62afa4daa8ad' is free. Now locking as 'exclusive' (1 active user)
>Thread-15915905::DEBUG::2013-11-20 04:12:22,133::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.cd4f9324-c91d-405b-983f-62afa4daa8ad`ReqID=`352bc2cd-c72a-4be9-ba69-3d27805ba3c6`::Granted request
>Thread-15915905::DEBUG::2013-11-20 04:12:22,139::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=28 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15915915::DEBUG::2013-11-20 04:12:22,364::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915915::INFO::2013-11-20 04:12:22,366::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915915::DEBUG::2013-11-20 04:12:22,366::task::495::TaskManager.Task::(_debug) Task 85ec1850-ee5c-418f-8d72-653500be1a39: moving from state init -> state preparing
>Thread-15915915::DEBUG::2013-11-20 04:12:22,367::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915915::DEBUG::2013-11-20 04:12:22,368::task::495::TaskManager.Task::(_debug) Task 85ec1850-ee5c-418f-8d72-653500be1a39: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915915::DEBUG::2013-11-20 04:12:22,368::task::495::TaskManager.Task::(_debug) Task 85ec1850-ee5c-418f-8d72-653500be1a39: moving from state preparing -> state finished
>Thread-15915915::DEBUG::2013-11-20 04:12:22,369::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915915::DEBUG::2013-11-20 04:12:22,369::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915915::DEBUG::2013-11-20 04:12:22,370::task::495::TaskManager.Task::(_debug) Task 85ec1850-ee5c-418f-8d72-653500be1a39: ref 0 aborting False
>Thread-15915915::INFO::2013-11-20 04:12:22,370::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915916::DEBUG::2013-11-20 04:12:22,375::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915916::INFO::2013-11-20 04:12:22,377::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915916::DEBUG::2013-11-20 04:12:22,378::task::495::TaskManager.Task::(_debug) Task 54449f71-0a48-416d-b2b3-6f5deaa7afa8: moving from state init -> state preparing
>Thread-15915916::DEBUG::2013-11-20 04:12:22,380::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f1f1a7ad-4ee6-4c9d-8b1c-b0ae53225f21`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915916::DEBUG::2013-11-20 04:12:22,381::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915916::DEBUG::2013-11-20 04:12:22,382::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915916::DEBUG::2013-11-20 04:12:22,383::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f1f1a7ad-4ee6-4c9d-8b1c-b0ae53225f21`::Granted request
>Thread-15915916::DEBUG::2013-11-20 04:12:22,384::task::495::TaskManager.Task::(_debug) Task 54449f71-0a48-416d-b2b3-6f5deaa7afa8: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915916::DEBUG::2013-11-20 04:12:22,385::task::495::TaskManager.Task::(_debug) Task 54449f71-0a48-416d-b2b3-6f5deaa7afa8: ref 1 aborting False
>Thread-15915916::WARNING::2013-11-20 04:12:22,388::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915916::DEBUG::2013-11-20 04:12:22,389::task::495::TaskManager.Task::(_debug) Task 54449f71-0a48-416d-b2b3-6f5deaa7afa8: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915916::DEBUG::2013-11-20 04:12:22,390::task::495::TaskManager.Task::(_debug) Task 54449f71-0a48-416d-b2b3-6f5deaa7afa8: moving from state preparing -> state finished
>Thread-15915916::DEBUG::2013-11-20 04:12:22,391::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915916::DEBUG::2013-11-20 04:12:22,391::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915916::DEBUG::2013-11-20 04:12:22,392::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915916::DEBUG::2013-11-20 04:12:22,393::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915916::DEBUG::2013-11-20 04:12:22,394::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915916::DEBUG::2013-11-20 04:12:22,394::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915916::DEBUG::2013-11-20 04:12:22,395::task::495::TaskManager.Task::(_debug) Task 54449f71-0a48-416d-b2b3-6f5deaa7afa8: ref 0 aborting False
>Thread-15915916::INFO::2013-11-20 04:12:22,395::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915905::DEBUG::2013-11-20 04:12:22,565::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000178601 s, 2.9 MB/s\n'; <rc> = 0
>Thread-15915905::DEBUG::2013-11-20 04:12:22,566::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000178601 s, 2.9 MB/s'], size: 512
>Thread-15915905::DEBUG::2013-11-20 04:12:22,568::task::495::TaskManager.Task::(_debug) Task 8b5ddd9b-ddc1-45ae-95be-c3ca93d9953a: finished: None
>Thread-15915905::DEBUG::2013-11-20 04:12:22,568::task::495::TaskManager.Task::(_debug) Task 8b5ddd9b-ddc1-45ae-95be-c3ca93d9953a: moving from state preparing -> state finished
>Thread-15915905::DEBUG::2013-11-20 04:12:22,569::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915905::DEBUG::2013-11-20 04:12:22,570::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915905::DEBUG::2013-11-20 04:12:22,570::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915905::DEBUG::2013-11-20 04:12:22,571::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915905::DEBUG::2013-11-20 04:12:22,572::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915905::DEBUG::2013-11-20 04:12:22,572::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915905::DEBUG::2013-11-20 04:12:22,573::task::495::TaskManager.Task::(_debug) Task 8b5ddd9b-ddc1-45ae-95be-c3ca93d9953a: ref 0 aborting False
>Thread-15915905::INFO::2013-11-20 04:12:22,574::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, Return response: {'status': {'message': 'OK', 'code': 0}}
>Thread-15915905::INFO::2013-11-20 04:12:22,575::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915905::DEBUG::2013-11-20 04:12:22,575::task::495::TaskManager.Task::(_debug) Task 9201c645-f2aa-437b-abd4-ac7838ed188b: moving from state init -> state preparing
>Thread-15915905::DEBUG::2013-11-20 04:12:22,576::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`33668176-d3cc-4c94-8736-512e00c638a7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1666' at 'public_getVolumePath'
>Thread-15915905::DEBUG::2013-11-20 04:12:22,577::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915905::DEBUG::2013-11-20 04:12:22,578::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915905::DEBUG::2013-11-20 04:12:22,578::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`33668176-d3cc-4c94-8736-512e00c638a7`::Granted request
>Thread-15915905::DEBUG::2013-11-20 04:12:22,579::task::495::TaskManager.Task::(_debug) Task 9201c645-f2aa-437b-abd4-ac7838ed188b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915905::DEBUG::2013-11-20 04:12:22,580::task::495::TaskManager.Task::(_debug) Task 9201c645-f2aa-437b-abd4-ac7838ed188b: ref 1 aborting False
>Thread-15915905::DEBUG::2013-11-20 04:12:22,581::task::495::TaskManager.Task::(_debug) Task 9201c645-f2aa-437b-abd4-ac7838ed188b: finished: {'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/e0628852-41b1-4535-8dda-ebc96cb00952/cd4f9324-c91d-405b-983f-62afa4daa8ad'}
>Thread-15915905::DEBUG::2013-11-20 04:12:22,582::task::495::TaskManager.Task::(_debug) Task 9201c645-f2aa-437b-abd4-ac7838ed188b: moving from state preparing -> state finished
>Thread-15915905::DEBUG::2013-11-20 04:12:22,582::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915905::DEBUG::2013-11-20 04:12:22,583::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915905::DEBUG::2013-11-20 04:12:22,584::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915905::DEBUG::2013-11-20 04:12:22,584::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915905::DEBUG::2013-11-20 04:12:22,585::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915905::DEBUG::2013-11-20 04:12:22,585::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915905::DEBUG::2013-11-20 04:12:22,586::task::495::TaskManager.Task::(_debug) Task 9201c645-f2aa-437b-abd4-ac7838ed188b: ref 0 aborting False
>Thread-15915905::INFO::2013-11-20 04:12:22,587::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, Return response: {'status': {'message': 'OK', 'code': 0}, 'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/e0628852-41b1-4535-8dda-ebc96cb00952/cd4f9324-c91d-405b-983f-62afa4daa8ad'}
>Thread-15915905::INFO::2013-11-20 04:12:22,588::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915905::DEBUG::2013-11-20 04:12:22,588::task::495::TaskManager.Task::(_debug) Task 85cc61c9-7515-4938-b313-77ebd5d588f9: moving from state init -> state preparing
>Thread-15915905::DEBUG::2013-11-20 04:12:22,589::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ca121572-3133-4a69-80f4-5159b398f7a8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915905::DEBUG::2013-11-20 04:12:22,590::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915905::DEBUG::2013-11-20 04:12:22,590::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915905::DEBUG::2013-11-20 04:12:22,591::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ca121572-3133-4a69-80f4-5159b398f7a8`::Granted request
>Thread-15915905::DEBUG::2013-11-20 04:12:22,592::task::495::TaskManager.Task::(_debug) Task 85cc61c9-7515-4938-b313-77ebd5d588f9: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915905::DEBUG::2013-11-20 04:12:22,592::task::495::TaskManager.Task::(_debug) Task 85cc61c9-7515-4938-b313-77ebd5d588f9: ref 1 aborting False
>Thread-15915905::DEBUG::2013-11-20 04:12:22,593::task::495::TaskManager.Task::(_debug) Task 85cc61c9-7515-4938-b313-77ebd5d588f9: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915905::DEBUG::2013-11-20 04:12:22,594::task::495::TaskManager.Task::(_debug) Task 85cc61c9-7515-4938-b313-77ebd5d588f9: moving from state preparing -> state finished
>Thread-15915905::DEBUG::2013-11-20 04:12:22,594::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915905::DEBUG::2013-11-20 04:12:22,595::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915905::DEBUG::2013-11-20 04:12:22,595::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915905::DEBUG::2013-11-20 04:12:22,596::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915905::DEBUG::2013-11-20 04:12:22,597::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915905::DEBUG::2013-11-20 04:12:22,597::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915905::DEBUG::2013-11-20 04:12:22,598::task::495::TaskManager.Task::(_debug) Task 85cc61c9-7515-4938-b313-77ebd5d588f9: ref 0 aborting False
>Thread-15915905::INFO::2013-11-20 04:12:22,599::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915905::INFO::2013-11-20 04:12:22,600::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16)
>Thread-15915905::DEBUG::2013-11-20 04:12:22,600::task::495::TaskManager.Task::(_debug) Task 04261681-d8e5-46df-868a-dc54073a8fbb: moving from state init -> state preparing
>Thread-15915905::INFO::2013-11-20 04:12:22,601::blockSD::674::Storage.StorageDomain::(validate) sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16
>Thread-15915905::DEBUG::2013-11-20 04:12:22,602::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15915905::DEBUG::2013-11-20 04:12:22,603::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15915905::DEBUG::2013-11-20 04:12:22,603::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15915905::DEBUG::2013-11-20 04:12:22,605::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15915905::DEBUG::2013-11-20 04:12:22,606::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-15915905::DEBUG::2013-11-20 04:12:22,611::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15915905::DEBUG::2013-11-20 04:12:23,460::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15915905::DEBUG::2013-11-20 04:12:23,464::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-15915905::DEBUG::2013-11-20 04:12:23,465::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'_SHA_CKSUM=716ea7492d9134a4b13de27ac61a336ad474727c', u'POOL_SPM_ID=2', u'POOL_SPM_LVER=287']
>Thread-15915905::DEBUG::2013-11-20 04:12:23,468::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4e06ed70-0529-4757-afdb-702e10866970`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1297' at 'public_getStorageDomainInfo'
>Thread-15915905::DEBUG::2013-11-20 04:12:23,468::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915905::DEBUG::2013-11-20 04:12:23,469::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915905::DEBUG::2013-11-20 04:12:23,470::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4e06ed70-0529-4757-afdb-702e10866970`::Granted request
>Thread-15915905::DEBUG::2013-11-20 04:12:23,471::task::495::TaskManager.Task::(_debug) Task 04261681-d8e5-46df-868a-dc54073a8fbb: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915905::DEBUG::2013-11-20 04:12:23,472::task::495::TaskManager.Task::(_debug) Task 04261681-d8e5-46df-868a-dc54073a8fbb: ref 1 aborting False
>Thread-15915905::WARNING::2013-11-20 04:12:23,474::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915905::DEBUG::2013-11-20 04:12:23,475::task::495::TaskManager.Task::(_debug) Task 04261681-d8e5-46df-868a-dc54073a8fbb: finished: {'info': {'uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'vguuid': 'aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', 'lver': 287, 'state': 'OK', 'version': '2', 'role': 'Master', 'master_ver': 1, 'spm_id': 2, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'OS'}}
>Thread-15915905::DEBUG::2013-11-20 04:12:23,476::task::495::TaskManager.Task::(_debug) Task 04261681-d8e5-46df-868a-dc54073a8fbb: moving from state preparing -> state finished
>Thread-15915905::DEBUG::2013-11-20 04:12:23,477::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915905::DEBUG::2013-11-20 04:12:23,478::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915905::DEBUG::2013-11-20 04:12:23,479::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915905::DEBUG::2013-11-20 04:12:23,479::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915905::DEBUG::2013-11-20 04:12:23,480::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915905::DEBUG::2013-11-20 04:12:23,481::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915905::DEBUG::2013-11-20 04:12:23,482::task::495::TaskManager.Task::(_debug) Task 04261681-d8e5-46df-868a-dc54073a8fbb: ref 0 aborting False
>Thread-15915905::INFO::2013-11-20 04:12:23,482::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'vguuid': 'aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', 'lver': 287, 'state': 'OK', 'version': '2', 'role': 'Master', 'master_ver': 1, 'spm_id': 2, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'OS'}}
>Thread-15915905::DEBUG::2013-11-20 04:12:23,519::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/libexec/vdsm/hooks/before_vm_start/50_vhostmd' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:12:23,936::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915905::DEBUG::2013-11-20 04:12:24,133::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15915905::INFO::2013-11-20 04:12:24,135::hooks::51::root::(_runHooksDir) 
>Thread-15915905::DEBUG::2013-11-20 04:12:24,136::libvirtvm::1077::vm.Vm::(_run) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::<?xml version="1.0" encoding="utf-8"?>
><domain type="kvm">
>	<name>tripoli</name>
>	<uuid>2afdc298-b10b-4bd1-8df2-2a401efd4f9c</uuid>
>	<memory>524288</memory>
>	<currentMemory>524288</currentMemory>
>	<vcpu>1</vcpu>
>	<devices>
>		<disk device="disk" type="block">
>			<source dev="/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/e0628852-41b1-4535-8dda-ebc96cb00952/cd4f9324-c91d-405b-983f-62afa4daa8ad"/>
>			<target bus="ide" dev="hda"/>
>			<serial>35-8dda-ebc96cb00952</serial>
>			<driver cache="none" error_policy="stop" io="native" name="qemu" type="raw"/>
>		</disk>
>		<disk device="cdrom" type="file">
>			<target bus="ide" dev="hdc"/>
>		</disk>
>		<controller index="0" ports="16" type="virtio-serial"/>
>		<channel type="unix">
>			<target name="com.redhat.rhevm.vdsm" type="virtio"/>
>			<source mode="bind" path="/var/lib/libvirt/qemu/channels/tripoli.com.redhat.rhevm.vdsm"/>
>		</channel>
>		<memballoon model="none"/>
>		<interface type="bridge">
>			<mac address="00:1a:4a:a8:64:11"/>
>			<model type="virtio"/>
>			<source bridge="rhevm"/>
>		</interface>
>		<interface type="bridge">
>			<mac address="00:1a:4a:a8:64:12"/>
>			<model type="rtl8139"/>
>			<source bridge="rhevm"/>
>		</interface>
>		<input bus="ps2" type="mouse"/>
>		<video>
>			<model heads="1" type="qxl" vram="65536"/>
>		</video>
>		<channel type="spicevmc">
>			<target name="com.redhat.spice.0" type="virtio"/>
>		</channel>
>		<graphics autoport="yes" keymap="en-us" listen="0" passwd="*****" passwdValidTo="1970-01-01T00:00:01" port="-1" tlsPort="-1" type="spice">
>			<channel mode="secure" name="main"/>
>			<channel mode="secure" name="inputs"/>
>		</graphics>
>	</devices>
>	<os>
>		<type arch="x86_64" machine="rhel6.2.0">hvm</type>
>		<boot dev="hd"/>
>		<smbios mode="sysinfo"/>
>	</os>
>	<sysinfo type="smbios">
>		<system>
>			<entry name="manufacturer">Red Hat</entry>
>			<entry name="product">RHEV Hypervisor</entry>
>			<entry name="version">6.3-20121012.0.el6_3</entry>
>			<entry name="serial">30343536-3138-5A43-4A32-333630373442_AC:16:2D:76:EB:F0</entry>
>			<entry name="uuid">2afdc298-b10b-4bd1-8df2-2a401efd4f9c</entry>
>		</system>
>	</sysinfo>
>	<clock adjustment="0" offset="variable">
>		<timer name="rtc" tickpolicy="catchup"/>
>	</clock>
>	<features>
>		<acpi/>
>	</features>
>	<cpu match="exact">
>		<model>Westmere</model>
>		<topology cores="1" sockets="1" threads="1"/>
>	</cpu>
></domain>
>
>Dummy-15912651::DEBUG::2013-11-20 04:12:24,412::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0522296 s, 19.6 MB/s\n'; <rc> = 0
>Thread-15913639::INFO::2013-11-20 04:12:24,783::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:12:24,784::task::495::TaskManager.Task::(_debug) Task e640ef57-7a1c-432a-9b80-5f936fb4c7e6: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:12:24,792::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`60631fd7-f00f-42dc-8499-83cb42ce8bea`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:12:24,793::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:12:24,794::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:12:24,795::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`60631fd7-f00f-42dc-8499-83cb42ce8bea`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:12:24,796::task::495::TaskManager.Task::(_debug) Task e640ef57-7a1c-432a-9b80-5f936fb4c7e6: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:12:24,796::task::495::TaskManager.Task::(_debug) Task e640ef57-7a1c-432a-9b80-5f936fb4c7e6: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:12:24,797::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15913639::DEBUG::2013-11-20 04:12:24,798::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15913639::DEBUG::2013-11-20 04:12:25,877::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913639::DEBUG::2013-11-20 04:12:26,123::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15913639::DEBUG::2013-11-20 04:12:26,124::task::495::TaskManager.Task::(_debug) Task e640ef57-7a1c-432a-9b80-5f936fb4c7e6: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:12:26,125::task::495::TaskManager.Task::(_debug) Task e640ef57-7a1c-432a-9b80-5f936fb4c7e6: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:12:26,125::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:12:26,126::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:12:26,127::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:12:26,127::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:12:26,128::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:12:26,128::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:12:26,129::task::495::TaskManager.Task::(_debug) Task e640ef57-7a1c-432a-9b80-5f936fb4c7e6: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:12:26,129::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:12:26,428::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:12:27,148::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0795446 s, 12.9 MB/s\n'; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-20 04:12:27,675::libvirtvm::1401::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::event Started detail 0 opaque None
>Thread-15915905::DEBUG::2013-11-20 04:12:27,766::utils::307::vm.Vm::(start) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Start statistics collection
>Thread-15915918::DEBUG::2013-11-20 04:12:27,772::utils::336::vm.Vm::(run) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Stats thread started
>Thread-15915919::DEBUG::2013-11-20 04:12:27,774::guestIF::47::root::(_connect) Attempting connection to /var/lib/libvirt/qemu/channels/tripoli.com.redhat.rhevm.vdsm
>Thread-15915918::INFO::2013-11-20 04:12:27,777::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915919::DEBUG::2013-11-20 04:12:27,778::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /usr/share/vdsm/prepare-vmchannel /var/lib/libvirt/qemu/channels/tripoli.com.redhat.rhevm.vdsm' (cwd None)
>Thread-15915918::DEBUG::2013-11-20 04:12:27,780::task::495::TaskManager.Task::(_debug) Task ef8f0970-dc6f-4e63-8b36-c32147309d0e: moving from state init -> state preparing
>Thread-15915905::WARNING::2013-11-20 04:12:27,780::libvirtvm::1124::vm.Vm::(_readPauseCode) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::_readPauseCode unsupported by libvirt vm
>Thread-15915918::DEBUG::2013-11-20 04:12:27,785::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`96387293-6c31-46f8-ab4f-c060f26f44ea`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915905::DEBUG::2013-11-20 04:12:27,785::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /usr/share/vdsm/get-vm-pid tripoli' (cwd None)
>Thread-15915918::DEBUG::2013-11-20 04:12:27,993::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:12:28,266::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:12:28,268::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`96387293-6c31-46f8-ab4f-c060f26f44ea`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:12:28,270::task::495::TaskManager.Task::(_debug) Task ef8f0970-dc6f-4e63-8b36-c32147309d0e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:12:28,271::task::495::TaskManager.Task::(_debug) Task ef8f0970-dc6f-4e63-8b36-c32147309d0e: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:12:28,272::task::495::TaskManager.Task::(_debug) Task ef8f0970-dc6f-4e63-8b36-c32147309d0e: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:12:28,273::task::495::TaskManager.Task::(_debug) Task ef8f0970-dc6f-4e63-8b36-c32147309d0e: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:12:28,274::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:12:28,274::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:12:28,275::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:12:28,276::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:12:28,277::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:12:28,278::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:12:28,279::task::495::TaskManager.Task::(_debug) Task ef8f0970-dc6f-4e63-8b36-c32147309d0e: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:12:28,279::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915919::DEBUG::2013-11-20 04:12:28,339::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15915919::DEBUG::2013-11-20 04:12:28,341::guestIF::53::root::(_connect) connected to /var/lib/libvirt/qemu/channels/tripoli.com.redhat.rhevm.vdsm
>Thread-15915905::DEBUG::2013-11-20 04:12:28,500::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15915905::DEBUG::2013-11-20 04:12:28,547::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /sbin/service ksmtuned retune' (cwd None)
>Thread-15915905::DEBUG::2013-11-20 04:12:29,033::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15915905::DEBUG::2013-11-20 04:12:29,034::vm::358::vm.Vm::(_startUnderlyingVm) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::_ongoingCreations released
>Dummy-15912651::DEBUG::2013-11-20 04:12:29,158::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:12:29,694::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0752158 s, 13.6 MB/s\n'; <rc> = 0
>Thread-15915922::DEBUG::2013-11-20 04:12:31,497::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('2afdc298-b10b-4bd1-8df2-2a401efd4f9c',) {}
>Thread-15915922::DEBUG::2013-11-20 04:12:31,498::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::CPU stats not available
>Thread-15915922::DEBUG::2013-11-20 04:12:31,498::libvirtvm::202::vm.Vm::(_getNetworkStats) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Network stats not available
>Thread-15915922::DEBUG::2013-11-20 04:12:31,499::libvirtvm::202::vm.Vm::(_getNetworkStats) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Network stats not available
>Thread-15915922::DEBUG::2013-11-20 04:12:31,499::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Disk stats not available
>Thread-15915922::DEBUG::2013-11-20 04:12:31,499::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Disk latency not available
>Thread-15915922::DEBUG::2013-11-20 04:12:31,500::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '45042', 'displayIp': '0', 'displayPort': u'5918', 'session': 'Unknown', 'displaySecurePort': u'5919', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet12': {'state': 'unknown', 'macAddr': u'00:1a:4a:a8:64:12', 'speed': '100', 'name': u'vnet12'}, u'vnet10': {'state': 'unknown', 'macAddr': u'00:1a:4a:a8:64:11', 'speed': '1000', 'name': u'vnet10'}}, 'vmId': '2afdc298-b10b-4bd1-8df2-2a401efd4f9c', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '32346472448', 'writeLatency': '0', 'imageID': 'e0628852-41b1-4535-8dda-ebc96cb00952', 'flushLatency': '0', 'truesize': '32346472448'}}, 'boot': 'c', 'statsAge': '1.15', 'cpuIdle': '100.00', 'elapsedTime': '20', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 04:12:31,709::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:12:32,202::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.06273 s, 16.3 MB/s\n'; <rc> = 0
>Thread-15915923::DEBUG::2013-11-20 04:12:32,742::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915923::INFO::2013-11-20 04:12:32,743::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915923::DEBUG::2013-11-20 04:12:32,744::task::495::TaskManager.Task::(_debug) Task a2723f7c-8286-4bfe-b333-1e9179931127: moving from state init -> state preparing
>Thread-15915923::DEBUG::2013-11-20 04:12:32,745::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915923::DEBUG::2013-11-20 04:12:32,746::task::495::TaskManager.Task::(_debug) Task a2723f7c-8286-4bfe-b333-1e9179931127: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915923::DEBUG::2013-11-20 04:12:32,747::task::495::TaskManager.Task::(_debug) Task a2723f7c-8286-4bfe-b333-1e9179931127: moving from state preparing -> state finished
>Thread-15915923::DEBUG::2013-11-20 04:12:32,748::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915923::DEBUG::2013-11-20 04:12:32,748::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915923::DEBUG::2013-11-20 04:12:32,749::task::495::TaskManager.Task::(_debug) Task a2723f7c-8286-4bfe-b333-1e9179931127: ref 0 aborting False
>Thread-15915923::INFO::2013-11-20 04:12:32,750::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915924::DEBUG::2013-11-20 04:12:32,762::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915924::INFO::2013-11-20 04:12:32,763::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915924::DEBUG::2013-11-20 04:12:32,764::task::495::TaskManager.Task::(_debug) Task 8bf3c6eb-1f0f-4cdd-96e8-c516ebb07bac: moving from state init -> state preparing
>Thread-15915924::DEBUG::2013-11-20 04:12:32,765::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8192aba5-fde8-4b38-8b21-ee88caf7e8c1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915924::DEBUG::2013-11-20 04:12:32,766::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915924::DEBUG::2013-11-20 04:12:32,767::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915924::DEBUG::2013-11-20 04:12:32,767::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8192aba5-fde8-4b38-8b21-ee88caf7e8c1`::Granted request
>Thread-15915924::DEBUG::2013-11-20 04:12:32,768::task::495::TaskManager.Task::(_debug) Task 8bf3c6eb-1f0f-4cdd-96e8-c516ebb07bac: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915924::DEBUG::2013-11-20 04:12:32,769::task::495::TaskManager.Task::(_debug) Task 8bf3c6eb-1f0f-4cdd-96e8-c516ebb07bac: ref 1 aborting False
>Thread-15915924::WARNING::2013-11-20 04:12:32,771::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915924::DEBUG::2013-11-20 04:12:32,772::task::495::TaskManager.Task::(_debug) Task 8bf3c6eb-1f0f-4cdd-96e8-c516ebb07bac: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915924::DEBUG::2013-11-20 04:12:32,773::task::495::TaskManager.Task::(_debug) Task 8bf3c6eb-1f0f-4cdd-96e8-c516ebb07bac: moving from state preparing -> state finished
>Thread-15915924::DEBUG::2013-11-20 04:12:32,773::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915924::DEBUG::2013-11-20 04:12:32,774::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915924::DEBUG::2013-11-20 04:12:32,775::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915924::DEBUG::2013-11-20 04:12:32,775::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915924::DEBUG::2013-11-20 04:12:32,776::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915924::DEBUG::2013-11-20 04:12:32,776::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915924::DEBUG::2013-11-20 04:12:32,777::task::495::TaskManager.Task::(_debug) Task 8bf3c6eb-1f0f-4cdd-96e8-c516ebb07bac: ref 0 aborting False
>Thread-15915924::INFO::2013-11-20 04:12:32,778::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915863::INFO::2013-11-20 04:12:33,362::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:12:33,363::task::495::TaskManager.Task::(_debug) Task 3b4d0d97-0a79-4694-b13d-a4956b64f15b: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:12:33,364::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c07f4ae2-6a51-4549-a982-70926775d090`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:12:33,365::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:12:33,366::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:12:33,367::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c07f4ae2-6a51-4549-a982-70926775d090`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:12:33,369::task::495::TaskManager.Task::(_debug) Task 3b4d0d97-0a79-4694-b13d-a4956b64f15b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:12:33,369::task::495::TaskManager.Task::(_debug) Task 3b4d0d97-0a79-4694-b13d-a4956b64f15b: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:12:33,370::task::495::TaskManager.Task::(_debug) Task 3b4d0d97-0a79-4694-b13d-a4956b64f15b: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:12:33,371::task::495::TaskManager.Task::(_debug) Task 3b4d0d97-0a79-4694-b13d-a4956b64f15b: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:12:33,372::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:12:33,372::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:12:33,373::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:12:33,374::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:12:33,375::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:12:33,375::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:12:33,376::task::495::TaskManager.Task::(_debug) Task 3b4d0d97-0a79-4694-b13d-a4956b64f15b: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:12:33,377::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:12:34,216::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:12:34,812::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0841164 s, 12.2 MB/s\n'; <rc> = 0
>Thread-15915926::DEBUG::2013-11-20 04:12:35,437::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('2afdc298-b10b-4bd1-8df2-2a401efd4f9c',) {}
>Thread-15915926::DEBUG::2013-11-20 04:12:35,438::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::CPU stats not available
>Thread-15915926::DEBUG::2013-11-20 04:12:35,439::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Disk stats not available
>Thread-15915926::DEBUG::2013-11-20 04:12:35,439::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Disk latency not available
>Thread-15915926::DEBUG::2013-11-20 04:12:35,440::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '45042', 'displayIp': '0', 'displayPort': u'5918', 'session': 'Unknown', 'displaySecurePort': u'5919', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet12': {'macAddr': u'00:1a:4a:a8:64:12', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '100', 'name': u'vnet12'}, u'vnet10': {'macAddr': u'00:1a:4a:a8:64:11', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet10'}}, 'vmId': '2afdc298-b10b-4bd1-8df2-2a401efd4f9c', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '32346472448', 'writeLatency': '0', 'imageID': 'e0628852-41b1-4535-8dda-ebc96cb00952', 'flushLatency': '0', 'truesize': '32346472448'}}, 'boot': 'c', 'statsAge': '0.99', 'cpuIdle': '100.00', 'elapsedTime': '24', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15913518::INFO::2013-11-20 04:12:36,519::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:12:36,520::task::495::TaskManager.Task::(_debug) Task 01561276-5c56-4d3e-88a2-a0ef87103520: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:12:36,522::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ce022901-54c9-414f-99f2-8964ea01251a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:12:36,522::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:12:36,523::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:12:36,524::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ce022901-54c9-414f-99f2-8964ea01251a`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:12:36,525::task::495::TaskManager.Task::(_debug) Task 01561276-5c56-4d3e-88a2-a0ef87103520: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:12:36,526::task::495::TaskManager.Task::(_debug) Task 01561276-5c56-4d3e-88a2-a0ef87103520: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:12:36,526::task::495::TaskManager.Task::(_debug) Task 01561276-5c56-4d3e-88a2-a0ef87103520: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:12:36,527::task::495::TaskManager.Task::(_debug) Task 01561276-5c56-4d3e-88a2-a0ef87103520: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:12:36,528::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:12:36,529::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:12:36,529::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:12:36,530::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:12:36,531::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:12:36,531::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:12:36,532::task::495::TaskManager.Task::(_debug) Task 01561276-5c56-4d3e-88a2-a0ef87103520: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:12:36,533::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:12:36,534::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:12:36,535::task::495::TaskManager.Task::(_debug) Task 29cf4671-0fb0-4c20-8077-e4bf598c8ad0: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:12:36,536::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7013aaab-4c05-4e50-b933-7ecb6a79a625`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:12:36,536::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:12:36,537::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:12:36,538::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7013aaab-4c05-4e50-b933-7ecb6a79a625`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:12:36,538::task::495::TaskManager.Task::(_debug) Task 29cf4671-0fb0-4c20-8077-e4bf598c8ad0: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:12:36,539::task::495::TaskManager.Task::(_debug) Task 29cf4671-0fb0-4c20-8077-e4bf598c8ad0: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:12:36,540::task::495::TaskManager.Task::(_debug) Task 29cf4671-0fb0-4c20-8077-e4bf598c8ad0: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:12:36,540::task::495::TaskManager.Task::(_debug) Task 29cf4671-0fb0-4c20-8077-e4bf598c8ad0: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:12:36,541::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:12:36,541::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:12:36,542::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:12:36,543::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:12:36,543::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:12:36,544::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:12:36,544::task::495::TaskManager.Task::(_debug) Task 29cf4671-0fb0-4c20-8077-e4bf598c8ad0: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:12:36,545::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:12:36,827::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:12:37,348::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0753497 s, 13.6 MB/s\n'; <rc> = 0
>Thread-15915927::INFO::2013-11-20 04:12:38,465::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915927::DEBUG::2013-11-20 04:12:38,466::task::495::TaskManager.Task::(_debug) Task 62316206-7d0f-4e8c-aa3c-68cf5498254e: moving from state init -> state preparing
>Thread-15915927::DEBUG::2013-11-20 04:12:38,467::task::495::TaskManager.Task::(_debug) Task 62316206-7d0f-4e8c-aa3c-68cf5498254e: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000440835952759', 'lastCheck': 1384920756.4645319, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0130929946899', 'lastCheck': 1384920756.472512, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00216579437256', 'lastCheck': 1384920752.9456911, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000740051269531', 'lastCheck': 1384920750.7625539, 'valid': True, 'code': 0}}
>Thread-15915927::DEBUG::2013-11-20 04:12:38,468::task::495::TaskManager.Task::(_debug) Task 62316206-7d0f-4e8c-aa3c-68cf5498254e: moving from state preparing -> state finished
>Thread-15915927::DEBUG::2013-11-20 04:12:38,469::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915927::DEBUG::2013-11-20 04:12:38,470::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915927::DEBUG::2013-11-20 04:12:38,471::task::495::TaskManager.Task::(_debug) Task 62316206-7d0f-4e8c-aa3c-68cf5498254e: ref 0 aborting False
>Thread-15915927::INFO::2013-11-20 04:12:38,471::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000440835952759', 'lastCheck': 1384920756.4645319, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0130929946899', 'lastCheck': 1384920756.472512, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00216579437256', 'lastCheck': 1384920752.9456911, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000740051269531', 'lastCheck': 1384920750.7625539, 'valid': True, 'code': 0}}
>Thread-15915928::DEBUG::2013-11-20 04:12:38,519::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::CPU stats not available
>Thread-15915928::DEBUG::2013-11-20 04:12:38,520::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Disk stats not available
>Thread-15915928::DEBUG::2013-11-20 04:12:38,521::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Disk latency not available
>Dummy-15912651::DEBUG::2013-11-20 04:12:39,361::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:12:39,907::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0800806 s, 12.8 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 04:12:41,602::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:12:41,610::task::495::TaskManager.Task::(_debug) Task 56afc82b-5dc3-41dc-972d-42c013e8e71e: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:12:41,612::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`683f775a-57b8-4364-a622-f959944fdd33`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:12:41,613::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:12:41,614::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:12:41,614::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`683f775a-57b8-4364-a622-f959944fdd33`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:12:41,615::task::495::TaskManager.Task::(_debug) Task 56afc82b-5dc3-41dc-972d-42c013e8e71e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:12:41,616::task::495::TaskManager.Task::(_debug) Task 56afc82b-5dc3-41dc-972d-42c013e8e71e: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:12:41,617::task::495::TaskManager.Task::(_debug) Task 56afc82b-5dc3-41dc-972d-42c013e8e71e: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:12:41,617::task::495::TaskManager.Task::(_debug) Task 56afc82b-5dc3-41dc-972d-42c013e8e71e: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:12:41,618::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:12:41,619::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:12:41,619::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:12:41,620::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:12:41,620::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:12:41,621::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:12:41,622::task::495::TaskManager.Task::(_debug) Task 56afc82b-5dc3-41dc-972d-42c013e8e71e: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:12:41,622::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:12:41,923::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915930::DEBUG::2013-11-20 04:12:42,215::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('2afdc298-b10b-4bd1-8df2-2a401efd4f9c',) {}
>Thread-15915930::DEBUG::2013-11-20 04:12:42,216::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::CPU stats not available
>Thread-15915930::DEBUG::2013-11-20 04:12:42,218::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Disk stats not available
>Thread-15915930::DEBUG::2013-11-20 04:12:42,218::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Disk latency not available
>Thread-15915930::DEBUG::2013-11-20 04:12:42,220::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '45042', 'displayIp': '0', 'displayPort': u'5918', 'session': 'Unknown', 'displaySecurePort': u'5919', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet12': {'macAddr': u'00:1a:4a:a8:64:12', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '100', 'name': u'vnet12'}, u'vnet10': {'macAddr': u'00:1a:4a:a8:64:11', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet10'}}, 'vmId': '2afdc298-b10b-4bd1-8df2-2a401efd4f9c', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '32346472448', 'writeLatency': '0', 'imageID': 'e0628852-41b1-4535-8dda-ebc96cb00952', 'flushLatency': '0', 'truesize': '32346472448'}}, 'boot': 'c', 'statsAge': '1.76', 'cpuIdle': '100.00', 'elapsedTime': '30', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 04:12:42,552::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.10492 s, 9.8 MB/s\n'; <rc> = 0
>Thread-15915931::DEBUG::2013-11-20 04:12:43,552::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915931::INFO::2013-11-20 04:12:43,554::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915931::DEBUG::2013-11-20 04:12:43,554::task::495::TaskManager.Task::(_debug) Task 56d0ed44-73d1-4f3c-a4da-23435129021f: moving from state init -> state preparing
>Thread-15915931::DEBUG::2013-11-20 04:12:43,555::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915931::DEBUG::2013-11-20 04:12:43,556::task::495::TaskManager.Task::(_debug) Task 56d0ed44-73d1-4f3c-a4da-23435129021f: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915931::DEBUG::2013-11-20 04:12:43,557::task::495::TaskManager.Task::(_debug) Task 56d0ed44-73d1-4f3c-a4da-23435129021f: moving from state preparing -> state finished
>Thread-15915931::DEBUG::2013-11-20 04:12:43,557::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915931::DEBUG::2013-11-20 04:12:43,558::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915931::DEBUG::2013-11-20 04:12:43,559::task::495::TaskManager.Task::(_debug) Task 56d0ed44-73d1-4f3c-a4da-23435129021f: ref 0 aborting False
>Thread-15915931::INFO::2013-11-20 04:12:43,559::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915932::DEBUG::2013-11-20 04:12:43,566::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915932::INFO::2013-11-20 04:12:43,567::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915932::DEBUG::2013-11-20 04:12:43,567::task::495::TaskManager.Task::(_debug) Task 3ebba032-a59b-4483-855a-0e1fe6ecee9e: moving from state init -> state preparing
>Thread-15915932::DEBUG::2013-11-20 04:12:43,569::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`953ca9f7-6294-47bc-b983-5ed290cfb3a4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915932::DEBUG::2013-11-20 04:12:43,569::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915932::DEBUG::2013-11-20 04:12:43,570::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915932::DEBUG::2013-11-20 04:12:43,571::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`953ca9f7-6294-47bc-b983-5ed290cfb3a4`::Granted request
>Thread-15915932::DEBUG::2013-11-20 04:12:43,572::task::495::TaskManager.Task::(_debug) Task 3ebba032-a59b-4483-855a-0e1fe6ecee9e: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915932::DEBUG::2013-11-20 04:12:43,573::task::495::TaskManager.Task::(_debug) Task 3ebba032-a59b-4483-855a-0e1fe6ecee9e: ref 1 aborting False
>Thread-15915932::WARNING::2013-11-20 04:12:43,575::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915932::DEBUG::2013-11-20 04:12:43,576::task::495::TaskManager.Task::(_debug) Task 3ebba032-a59b-4483-855a-0e1fe6ecee9e: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915932::DEBUG::2013-11-20 04:12:43,577::task::495::TaskManager.Task::(_debug) Task 3ebba032-a59b-4483-855a-0e1fe6ecee9e: moving from state preparing -> state finished
>Thread-15915932::DEBUG::2013-11-20 04:12:43,578::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915932::DEBUG::2013-11-20 04:12:43,578::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915932::DEBUG::2013-11-20 04:12:43,579::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915932::DEBUG::2013-11-20 04:12:43,580::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915932::DEBUG::2013-11-20 04:12:43,580::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915932::DEBUG::2013-11-20 04:12:43,581::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915932::DEBUG::2013-11-20 04:12:43,582::task::495::TaskManager.Task::(_debug) Task 3ebba032-a59b-4483-855a-0e1fe6ecee9e: ref 0 aborting False
>Thread-15915932::INFO::2013-11-20 04:12:43,583::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:12:44,567::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:12:45,110::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0879485 s, 11.6 MB/s\n'; <rc> = 0
>Thread-15915934::DEBUG::2013-11-20 04:12:45,262::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('2afdc298-b10b-4bd1-8df2-2a401efd4f9c',) {}
>Thread-15915934::DEBUG::2013-11-20 04:12:45,263::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Disk stats not available
>Thread-15915934::DEBUG::2013-11-20 04:12:45,264::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Disk latency not available
>Thread-15915934::DEBUG::2013-11-20 04:12:45,265::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '45042', 'displayIp': '0', 'displayPort': u'5918', 'session': 'Unknown', 'displaySecurePort': u'5919', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet12': {'macAddr': u'00:1a:4a:a8:64:12', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '100', 'name': u'vnet12'}, u'vnet10': {'macAddr': u'00:1a:4a:a8:64:11', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet10'}}, 'vmId': '2afdc298-b10b-4bd1-8df2-2a401efd4f9c', 'monitorResponse': '0', 'cpuUser': '39.54', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '32346472448', 'writeLatency': '0', 'imageID': 'e0628852-41b1-4535-8dda-ebc96cb00952', 'flushLatency': '0', 'truesize': '32346472448'}}, 'boot': 'c', 'statsAge': '0.80', 'cpuIdle': '60.46', 'elapsedTime': '33', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-42838::DEBUG::2013-11-20 04:12:46,755::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:12:47,126::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-42838::DEBUG::2013-11-20 04:12:47,468::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:12:47,639::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.065473 s, 15.6 MB/s\n'; <rc> = 0
>Thread-15915936::DEBUG::2013-11-20 04:12:48,638::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('2afdc298-b10b-4bd1-8df2-2a401efd4f9c',) {}
>Thread-15915936::DEBUG::2013-11-20 04:12:48,640::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Disk stats not available
>Thread-15915936::DEBUG::2013-11-20 04:12:48,641::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Disk latency not available
>Thread-15915936::DEBUG::2013-11-20 04:12:48,642::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '45042', 'displayIp': '0', 'displayPort': u'5918', 'session': 'Unknown', 'displaySecurePort': u'5919', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet12': {'macAddr': u'00:1a:4a:a8:64:12', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '100', 'name': u'vnet12'}, u'vnet10': {'macAddr': u'00:1a:4a:a8:64:11', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet10'}}, 'vmId': '2afdc298-b10b-4bd1-8df2-2a401efd4f9c', 'monitorResponse': '0', 'cpuUser': '39.54', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '32346472448', 'writeLatency': '0', 'imageID': 'e0628852-41b1-4535-8dda-ebc96cb00952', 'flushLatency': '0', 'truesize': '32346472448'}}, 'boot': 'c', 'statsAge': '0.18', 'cpuIdle': '60.46', 'elapsedTime': '37', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 04:12:49,655::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:12:50,206::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0859261 s, 11.9 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 04:12:51,601::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:12:51,602::task::495::TaskManager.Task::(_debug) Task 3d9ccacb-44da-4a28-bdab-f087c6708dae: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:12:51,604::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`aae2e5ef-b18d-47c5-8031-7839245d2d21`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:12:51,605::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:12:51,605::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:12:51,605::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`aae2e5ef-b18d-47c5-8031-7839245d2d21`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:12:51,606::task::495::TaskManager.Task::(_debug) Task 3d9ccacb-44da-4a28-bdab-f087c6708dae: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:12:51,606::task::495::TaskManager.Task::(_debug) Task 3d9ccacb-44da-4a28-bdab-f087c6708dae: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:12:51,607::task::495::TaskManager.Task::(_debug) Task 3d9ccacb-44da-4a28-bdab-f087c6708dae: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:12:51,607::task::495::TaskManager.Task::(_debug) Task 3d9ccacb-44da-4a28-bdab-f087c6708dae: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:12:51,608::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:12:51,608::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:12:51,608::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:12:51,609::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:12:51,609::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:12:51,609::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:12:51,610::task::495::TaskManager.Task::(_debug) Task 3d9ccacb-44da-4a28-bdab-f087c6708dae: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:12:51,610::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15915938::DEBUG::2013-11-20 04:12:51,770::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('2afdc298-b10b-4bd1-8df2-2a401efd4f9c',) {}
>Thread-15915938::DEBUG::2013-11-20 04:12:51,771::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Disk stats not available
>Thread-15915938::DEBUG::2013-11-20 04:12:51,772::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Disk latency not available
>Thread-15915938::DEBUG::2013-11-20 04:12:51,773::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '45042', 'displayIp': '0', 'displayPort': u'5918', 'session': 'Unknown', 'displaySecurePort': u'5919', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet12': {'macAddr': u'00:1a:4a:a8:64:12', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '100', 'name': u'vnet12'}, u'vnet10': {'macAddr': u'00:1a:4a:a8:64:11', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet10'}}, 'vmId': '2afdc298-b10b-4bd1-8df2-2a401efd4f9c', 'monitorResponse': '0', 'cpuUser': '39.54', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '32346472448', 'writeLatency': '0', 'imageID': 'e0628852-41b1-4535-8dda-ebc96cb00952', 'flushLatency': '0', 'truesize': '32346472448'}}, 'boot': 'c', 'statsAge': '1.30', 'cpuIdle': '60.46', 'elapsedTime': '40', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 04:12:52,220::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:12:52,769::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0877013 s, 11.7 MB/s\n'; <rc> = 0
>Thread-15915939::DEBUG::2013-11-20 04:12:54,039::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915939::INFO::2013-11-20 04:12:54,040::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915939::DEBUG::2013-11-20 04:12:54,041::task::495::TaskManager.Task::(_debug) Task ba65bfb9-e3b5-4a97-8e79-f99889c136fe: moving from state init -> state preparing
>Thread-15915939::DEBUG::2013-11-20 04:12:54,042::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915939::DEBUG::2013-11-20 04:12:54,043::task::495::TaskManager.Task::(_debug) Task ba65bfb9-e3b5-4a97-8e79-f99889c136fe: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915939::DEBUG::2013-11-20 04:12:54,043::task::495::TaskManager.Task::(_debug) Task ba65bfb9-e3b5-4a97-8e79-f99889c136fe: moving from state preparing -> state finished
>Thread-15915939::DEBUG::2013-11-20 04:12:54,044::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915939::DEBUG::2013-11-20 04:12:54,044::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915939::DEBUG::2013-11-20 04:12:54,045::task::495::TaskManager.Task::(_debug) Task ba65bfb9-e3b5-4a97-8e79-f99889c136fe: ref 0 aborting False
>Thread-15915939::INFO::2013-11-20 04:12:54,046::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915940::DEBUG::2013-11-20 04:12:54,051::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915940::INFO::2013-11-20 04:12:54,052::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915940::DEBUG::2013-11-20 04:12:54,053::task::495::TaskManager.Task::(_debug) Task 07bc1a73-b416-43e5-822c-bd29ac40ae72: moving from state init -> state preparing
>Thread-15915940::DEBUG::2013-11-20 04:12:54,054::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c604a8a2-6469-441f-b0b8-747c31028a76`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915940::DEBUG::2013-11-20 04:12:54,055::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915940::DEBUG::2013-11-20 04:12:54,056::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915940::DEBUG::2013-11-20 04:12:54,056::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c604a8a2-6469-441f-b0b8-747c31028a76`::Granted request
>Thread-15915940::DEBUG::2013-11-20 04:12:54,057::task::495::TaskManager.Task::(_debug) Task 07bc1a73-b416-43e5-822c-bd29ac40ae72: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915940::DEBUG::2013-11-20 04:12:54,058::task::495::TaskManager.Task::(_debug) Task 07bc1a73-b416-43e5-822c-bd29ac40ae72: ref 1 aborting False
>Thread-15915940::WARNING::2013-11-20 04:12:54,060::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915940::DEBUG::2013-11-20 04:12:54,061::task::495::TaskManager.Task::(_debug) Task 07bc1a73-b416-43e5-822c-bd29ac40ae72: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915940::DEBUG::2013-11-20 04:12:54,062::task::495::TaskManager.Task::(_debug) Task 07bc1a73-b416-43e5-822c-bd29ac40ae72: moving from state preparing -> state finished
>Thread-15915940::DEBUG::2013-11-20 04:12:54,063::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915940::DEBUG::2013-11-20 04:12:54,063::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915940::DEBUG::2013-11-20 04:12:54,064::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915940::DEBUG::2013-11-20 04:12:54,065::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915940::DEBUG::2013-11-20 04:12:54,065::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915940::DEBUG::2013-11-20 04:12:54,066::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915940::DEBUG::2013-11-20 04:12:54,067::task::495::TaskManager.Task::(_debug) Task 07bc1a73-b416-43e5-822c-bd29ac40ae72: ref 0 aborting False
>Thread-15915940::INFO::2013-11-20 04:12:54,067::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:12:54,786::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915941::INFO::2013-11-20 04:12:54,993::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915941::DEBUG::2013-11-20 04:12:54,995::task::495::TaskManager.Task::(_debug) Task 4b34521a-e138-445f-8f2a-2cd071b4d103: moving from state init -> state preparing
>Thread-15915941::DEBUG::2013-11-20 04:12:54,996::task::495::TaskManager.Task::(_debug) Task 4b34521a-e138-445f-8f2a-2cd071b4d103: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000453948974609', 'lastCheck': 1384920767.4737871, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00132203102112', 'lastCheck': 1384920766.9675579, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00134611129761', 'lastCheck': 1384920773.4639921, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000377178192139', 'lastCheck': 1384920771.3264811, 'valid': True, 'code': 0}}
>Thread-15915941::DEBUG::2013-11-20 04:12:54,997::task::495::TaskManager.Task::(_debug) Task 4b34521a-e138-445f-8f2a-2cd071b4d103: moving from state preparing -> state finished
>Thread-15915941::DEBUG::2013-11-20 04:12:54,998::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915941::DEBUG::2013-11-20 04:12:54,998::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915941::DEBUG::2013-11-20 04:12:54,999::task::495::TaskManager.Task::(_debug) Task 4b34521a-e138-445f-8f2a-2cd071b4d103: ref 0 aborting False
>Thread-15915941::INFO::2013-11-20 04:12:55,000::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000453948974609', 'lastCheck': 1384920767.4737871, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00132203102112', 'lastCheck': 1384920766.9675579, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00134611129761', 'lastCheck': 1384920773.4639921, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000377178192139', 'lastCheck': 1384920771.3264811, 'valid': True, 'code': 0}}
>Thread-15915942::DEBUG::2013-11-20 04:12:55,155::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Disk stats not available
>Thread-15915942::DEBUG::2013-11-20 04:12:55,156::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Disk latency not available
>Dummy-15912651::DEBUG::2013-11-20 04:12:55,305::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0518296 s, 19.8 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 04:12:56,602::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:12:56,603::task::495::TaskManager.Task::(_debug) Task ab68392d-3ae8-4a27-a15e-5915f3da988e: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:12:56,604::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ccecc09b-00c7-484f-a03c-00188c344834`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:12:56,605::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:12:56,606::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:12:56,607::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ccecc09b-00c7-484f-a03c-00188c344834`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:12:56,608::task::495::TaskManager.Task::(_debug) Task ab68392d-3ae8-4a27-a15e-5915f3da988e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:12:56,609::task::495::TaskManager.Task::(_debug) Task ab68392d-3ae8-4a27-a15e-5915f3da988e: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:12:56,610::task::495::TaskManager.Task::(_debug) Task ab68392d-3ae8-4a27-a15e-5915f3da988e: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:12:56,610::task::495::TaskManager.Task::(_debug) Task ab68392d-3ae8-4a27-a15e-5915f3da988e: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:12:56,611::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:12:56,612::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:12:56,612::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:12:56,613::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:12:56,614::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:12:56,615::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:12:56,616::task::495::TaskManager.Task::(_debug) Task ab68392d-3ae8-4a27-a15e-5915f3da988e: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:12:56,616::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:12:57,319::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:12:57,796::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0558512 s, 18.3 MB/s\n'; <rc> = 0
>Thread-15915944::DEBUG::2013-11-20 04:12:58,993::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('2afdc298-b10b-4bd1-8df2-2a401efd4f9c',) {}
>Thread-15915944::DEBUG::2013-11-20 04:12:58,995::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Disk stats not available
>Thread-15915944::DEBUG::2013-11-20 04:12:58,995::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Disk latency not available
>Thread-15915944::DEBUG::2013-11-20 04:12:58,996::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '45042', 'displayIp': '0', 'displayPort': u'5918', 'session': 'Unknown', 'displaySecurePort': u'5919', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet12': {'macAddr': u'00:1a:4a:a8:64:12', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '100', 'name': u'vnet12'}, u'vnet10': {'macAddr': u'00:1a:4a:a8:64:11', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet10'}}, 'vmId': '2afdc298-b10b-4bd1-8df2-2a401efd4f9c', 'monitorResponse': '0', 'cpuUser': '59.92', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '32346472448', 'writeLatency': '0', 'imageID': 'e0628852-41b1-4535-8dda-ebc96cb00952', 'flushLatency': '0', 'truesize': '32346472448'}}, 'boot': 'c', 'statsAge': '0.52', 'cpuIdle': '40.08', 'elapsedTime': '47', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 04:12:59,810::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:13:00,346::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0858729 s, 11.9 MB/s\n'; <rc> = 0
>Thread-15914453::INFO::2013-11-20 04:13:01,626::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:13:01,627::task::495::TaskManager.Task::(_debug) Task 3d5c2d2c-181e-4c94-b992-1dbc65ea075d: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:13:01,629::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1360978c-6094-435b-8e73-69d0d0575f20`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:13:01,630::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:13:01,631::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:13:01,631::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1360978c-6094-435b-8e73-69d0d0575f20`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:13:01,633::task::495::TaskManager.Task::(_debug) Task 3d5c2d2c-181e-4c94-b992-1dbc65ea075d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:13:01,633::task::495::TaskManager.Task::(_debug) Task 3d5c2d2c-181e-4c94-b992-1dbc65ea075d: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:13:01,634::task::495::TaskManager.Task::(_debug) Task 3d5c2d2c-181e-4c94-b992-1dbc65ea075d: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:13:01,635::task::495::TaskManager.Task::(_debug) Task 3d5c2d2c-181e-4c94-b992-1dbc65ea075d: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:13:01,635::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:13:01,636::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:13:01,637::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:13:01,637::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:13:01,638::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:13:01,639::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:13:01,639::task::495::TaskManager.Task::(_debug) Task 3d5c2d2c-181e-4c94-b992-1dbc65ea075d: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:13:01,640::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15915946::DEBUG::2013-11-20 04:13:02,061::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('2afdc298-b10b-4bd1-8df2-2a401efd4f9c',) {}
>Thread-15915946::DEBUG::2013-11-20 04:13:02,062::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Disk stats not available
>Thread-15915946::DEBUG::2013-11-20 04:13:02,063::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Disk latency not available
>Thread-15915946::DEBUG::2013-11-20 04:13:02,064::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '45042', 'displayIp': '0', 'displayPort': u'5918', 'session': 'Unknown', 'displaySecurePort': u'5919', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet12': {'macAddr': u'00:1a:4a:a8:64:12', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '100', 'name': u'vnet12'}, u'vnet10': {'macAddr': u'00:1a:4a:a8:64:11', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet10'}}, 'vmId': '2afdc298-b10b-4bd1-8df2-2a401efd4f9c', 'monitorResponse': '0', 'cpuUser': '59.92', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '32346472448', 'writeLatency': '0', 'imageID': 'e0628852-41b1-4535-8dda-ebc96cb00952', 'flushLatency': '0', 'truesize': '32346472448'}}, 'boot': 'c', 'statsAge': '1.58', 'cpuIdle': '40.08', 'elapsedTime': '50', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 04:13:02,360::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:13:02,934::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0831182 s, 12.3 MB/s\n'; <rc> = 0
>Thread-15915947::DEBUG::2013-11-20 04:13:04,497::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915947::INFO::2013-11-20 04:13:04,498::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915947::DEBUG::2013-11-20 04:13:04,499::task::495::TaskManager.Task::(_debug) Task 276ba894-0413-4cdb-8e4a-3b1c083c2c67: moving from state init -> state preparing
>Thread-15915947::DEBUG::2013-11-20 04:13:04,500::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915947::DEBUG::2013-11-20 04:13:04,500::task::495::TaskManager.Task::(_debug) Task 276ba894-0413-4cdb-8e4a-3b1c083c2c67: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915947::DEBUG::2013-11-20 04:13:04,501::task::495::TaskManager.Task::(_debug) Task 276ba894-0413-4cdb-8e4a-3b1c083c2c67: moving from state preparing -> state finished
>Thread-15915947::DEBUG::2013-11-20 04:13:04,503::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915947::DEBUG::2013-11-20 04:13:04,503::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915947::DEBUG::2013-11-20 04:13:04,504::task::495::TaskManager.Task::(_debug) Task 276ba894-0413-4cdb-8e4a-3b1c083c2c67: ref 0 aborting False
>Thread-15915947::INFO::2013-11-20 04:13:04,504::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915948::DEBUG::2013-11-20 04:13:04,518::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915948::INFO::2013-11-20 04:13:04,520::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915948::DEBUG::2013-11-20 04:13:04,520::task::495::TaskManager.Task::(_debug) Task 35452b0f-5c4e-4746-a703-9bd563858b39: moving from state init -> state preparing
>Thread-15915948::DEBUG::2013-11-20 04:13:04,522::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`99e3978d-bc69-4970-acce-2536489e4c6c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915948::DEBUG::2013-11-20 04:13:04,523::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915948::DEBUG::2013-11-20 04:13:04,524::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915948::DEBUG::2013-11-20 04:13:04,524::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`99e3978d-bc69-4970-acce-2536489e4c6c`::Granted request
>Thread-15915948::DEBUG::2013-11-20 04:13:04,526::task::495::TaskManager.Task::(_debug) Task 35452b0f-5c4e-4746-a703-9bd563858b39: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915948::DEBUG::2013-11-20 04:13:04,526::task::495::TaskManager.Task::(_debug) Task 35452b0f-5c4e-4746-a703-9bd563858b39: ref 1 aborting False
>Thread-15915948::WARNING::2013-11-20 04:13:04,529::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915948::DEBUG::2013-11-20 04:13:04,530::task::495::TaskManager.Task::(_debug) Task 35452b0f-5c4e-4746-a703-9bd563858b39: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915948::DEBUG::2013-11-20 04:13:04,531::task::495::TaskManager.Task::(_debug) Task 35452b0f-5c4e-4746-a703-9bd563858b39: moving from state preparing -> state finished
>Thread-15915948::DEBUG::2013-11-20 04:13:04,532::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915948::DEBUG::2013-11-20 04:13:04,533::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915948::DEBUG::2013-11-20 04:13:04,533::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915948::DEBUG::2013-11-20 04:13:04,534::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915948::DEBUG::2013-11-20 04:13:04,535::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915948::DEBUG::2013-11-20 04:13:04,535::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915948::DEBUG::2013-11-20 04:13:04,536::task::495::TaskManager.Task::(_debug) Task 35452b0f-5c4e-4746-a703-9bd563858b39: ref 0 aborting False
>Thread-15915948::INFO::2013-11-20 04:13:04,537::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:13:04,949::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915950::DEBUG::2013-11-20 04:13:05,143::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('2afdc298-b10b-4bd1-8df2-2a401efd4f9c',) {}
>Thread-15915950::DEBUG::2013-11-20 04:13:05,145::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Disk stats not available
>Thread-15915950::DEBUG::2013-11-20 04:13:05,146::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Disk latency not available
>Thread-15915950::DEBUG::2013-11-20 04:13:05,148::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '45042', 'displayIp': '0', 'displayPort': u'5918', 'session': 'Unknown', 'displaySecurePort': u'5919', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet12': {'macAddr': u'00:1a:4a:a8:64:12', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '100', 'name': u'vnet12'}, u'vnet10': {'macAddr': u'00:1a:4a:a8:64:11', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet10'}}, 'vmId': '2afdc298-b10b-4bd1-8df2-2a401efd4f9c', 'monitorResponse': '0', 'cpuUser': '59.92', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '32346472448', 'writeLatency': '0', 'imageID': 'e0628852-41b1-4535-8dda-ebc96cb00952', 'flushLatency': '0', 'truesize': '32346472448'}}, 'boot': 'c', 'statsAge': '0.55', 'cpuIdle': '40.08', 'elapsedTime': '53', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 04:13:05,479::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.081491 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:13:07,495::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 04:13:07,730::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:13:07,732::task::495::TaskManager.Task::(_debug) Task 314391b0-3d7d-496c-b23a-bebdc7cabfc7: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:13:07,739::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`aa5e13c0-9e0c-416e-8dbc-237e31f5e317`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:13:07,743::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:13:07,744::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:13:07,749::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`aa5e13c0-9e0c-416e-8dbc-237e31f5e317`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:13:07,755::task::495::TaskManager.Task::(_debug) Task 314391b0-3d7d-496c-b23a-bebdc7cabfc7: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 04:13:07,756::task::495::TaskManager.Task::(_debug) Task 314391b0-3d7d-496c-b23a-bebdc7cabfc7: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:13:07,761::task::495::TaskManager.Task::(_debug) Task 314391b0-3d7d-496c-b23a-bebdc7cabfc7: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 04:13:07,762::task::495::TaskManager.Task::(_debug) Task 314391b0-3d7d-496c-b23a-bebdc7cabfc7: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:13:07,762::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:13:07,763::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:13:07,769::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 04:13:07,770::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:13:07,775::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:13:07,775::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:13:07,776::task::495::TaskManager.Task::(_debug) Task 314391b0-3d7d-496c-b23a-bebdc7cabfc7: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 04:13:07,777::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 04:13:08,084::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0775396 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15915952::DEBUG::2013-11-20 04:13:08,249::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('2afdc298-b10b-4bd1-8df2-2a401efd4f9c',) {}
>Thread-15915952::DEBUG::2013-11-20 04:13:08,250::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Disk stats not available
>Thread-15915952::DEBUG::2013-11-20 04:13:08,251::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Disk latency not available
>Thread-15915952::DEBUG::2013-11-20 04:13:08,252::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '45042', 'displayIp': '0', 'displayPort': u'5918', 'session': 'Unknown', 'displaySecurePort': u'5919', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet12': {'macAddr': u'00:1a:4a:a8:64:12', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '100', 'name': u'vnet12'}, u'vnet10': {'macAddr': u'00:1a:4a:a8:64:11', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet10'}}, 'vmId': '2afdc298-b10b-4bd1-8df2-2a401efd4f9c', 'monitorResponse': '0', 'cpuUser': '59.92', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '32346472448', 'writeLatency': '0', 'imageID': 'e0628852-41b1-4535-8dda-ebc96cb00952', 'flushLatency': '0', 'truesize': '32346472448'}}, 'boot': 'c', 'statsAge': '1.65', 'cpuIdle': '40.08', 'elapsedTime': '56', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15914435::INFO::2013-11-20 04:13:08,668::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:13:08,669::task::495::TaskManager.Task::(_debug) Task 54f0d946-99e3-4b85-b06e-3cab4c86cae1: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:13:08,670::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`eae10b95-eafe-4315-b777-3b72f397f0d2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:13:08,671::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:13:08,672::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:13:08,673::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`eae10b95-eafe-4315-b777-3b72f397f0d2`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:13:08,674::task::495::TaskManager.Task::(_debug) Task 54f0d946-99e3-4b85-b06e-3cab4c86cae1: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:13:08,675::task::495::TaskManager.Task::(_debug) Task 54f0d946-99e3-4b85-b06e-3cab4c86cae1: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:13:08,675::task::495::TaskManager.Task::(_debug) Task 54f0d946-99e3-4b85-b06e-3cab4c86cae1: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:13:08,676::task::495::TaskManager.Task::(_debug) Task 54f0d946-99e3-4b85-b06e-3cab4c86cae1: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:13:08,677::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:13:08,677::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:13:08,678::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:13:08,679::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:13:08,680::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:13:08,680::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:13:08,681::task::495::TaskManager.Task::(_debug) Task 54f0d946-99e3-4b85-b06e-3cab4c86cae1: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:13:08,682::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:13:10,099::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:13:10,653::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0783909 s, 13.1 MB/s\n'; <rc> = 0
>Thread-15915953::INFO::2013-11-20 04:13:11,286::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915953::DEBUG::2013-11-20 04:13:11,287::task::495::TaskManager.Task::(_debug) Task 4ac46a4b-afb4-480f-a9ca-68f1c8e0361b: moving from state init -> state preparing
>Thread-15915953::DEBUG::2013-11-20 04:13:11,287::task::495::TaskManager.Task::(_debug) Task 4ac46a4b-afb4-480f-a9ca-68f1c8e0361b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000642061233521', 'lastCheck': 1384920787.733706, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00173807144165', 'lastCheck': 1384920787.13431, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0016918182373', 'lastCheck': 1384920783.5974121, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000488042831421', 'lastCheck': 1384920781.5287039, 'valid': True, 'code': 0}}
>Thread-15915953::DEBUG::2013-11-20 04:13:11,288::task::495::TaskManager.Task::(_debug) Task 4ac46a4b-afb4-480f-a9ca-68f1c8e0361b: moving from state preparing -> state finished
>Thread-15915953::DEBUG::2013-11-20 04:13:11,289::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915953::DEBUG::2013-11-20 04:13:11,289::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915953::DEBUG::2013-11-20 04:13:11,290::task::495::TaskManager.Task::(_debug) Task 4ac46a4b-afb4-480f-a9ca-68f1c8e0361b: ref 0 aborting False
>Thread-15915953::INFO::2013-11-20 04:13:11,291::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000642061233521', 'lastCheck': 1384920787.733706, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00173807144165', 'lastCheck': 1384920787.13431, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0016918182373', 'lastCheck': 1384920783.5974121, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000488042831421', 'lastCheck': 1384920781.5287039, 'valid': True, 'code': 0}}
>Thread-15915954::DEBUG::2013-11-20 04:13:11,343::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Disk stats not available
>Thread-15915954::DEBUG::2013-11-20 04:13:11,344::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Disk latency not available
>Thread-15913546::INFO::2013-11-20 04:13:11,721::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:13:11,731::task::495::TaskManager.Task::(_debug) Task 54123e96-91f9-414d-bc63-8a80c4fb6dae: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:13:11,733::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`878fecb2-975e-4f11-8538-ae5c31fa828c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:13:11,733::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:13:11,736::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:13:11,737::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`878fecb2-975e-4f11-8538-ae5c31fa828c`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:13:11,739::task::495::TaskManager.Task::(_debug) Task 54123e96-91f9-414d-bc63-8a80c4fb6dae: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:13:11,739::task::495::TaskManager.Task::(_debug) Task 54123e96-91f9-414d-bc63-8a80c4fb6dae: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:13:11,740::task::495::TaskManager.Task::(_debug) Task 54123e96-91f9-414d-bc63-8a80c4fb6dae: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 04:13:11,741::task::495::TaskManager.Task::(_debug) Task 54123e96-91f9-414d-bc63-8a80c4fb6dae: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:13:11,742::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:13:11,743::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:13:11,744::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:13:11,745::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:13:11,745::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:13:11,746::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:13:11,747::task::495::TaskManager.Task::(_debug) Task 54123e96-91f9-414d-bc63-8a80c4fb6dae: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:13:11,747::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:13:11,748::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:13:11,749::task::495::TaskManager.Task::(_debug) Task 3f803c45-12a1-475c-bf69-ded548aef738: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:13:11,750::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`04313d46-2765-43df-9285-d5753b528503`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:13:11,751::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:13:11,751::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:13:11,752::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`04313d46-2765-43df-9285-d5753b528503`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:13:11,762::task::495::TaskManager.Task::(_debug) Task 3f803c45-12a1-475c-bf69-ded548aef738: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:13:11,763::task::495::TaskManager.Task::(_debug) Task 3f803c45-12a1-475c-bf69-ded548aef738: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:13:11,764::task::495::TaskManager.Task::(_debug) Task 3f803c45-12a1-475c-bf69-ded548aef738: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:13:11,765::task::495::TaskManager.Task::(_debug) Task 3f803c45-12a1-475c-bf69-ded548aef738: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:13:11,766::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:13:11,766::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:13:11,767::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:13:11,768::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:13:11,768::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:13:11,769::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:13:11,769::task::495::TaskManager.Task::(_debug) Task 3f803c45-12a1-475c-bf69-ded548aef738: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:13:11,770::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:13:11,771::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:13:11,771::task::495::TaskManager.Task::(_debug) Task 1f34a68f-c8a2-4204-a41d-7508e756ed2c: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:13:11,773::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`70c92c98-3320-4520-aeec-c16ce1e50908`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:13:11,773::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:13:11,774::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:13:11,775::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`70c92c98-3320-4520-aeec-c16ce1e50908`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:13:11,776::task::495::TaskManager.Task::(_debug) Task 1f34a68f-c8a2-4204-a41d-7508e756ed2c: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:13:11,776::task::495::TaskManager.Task::(_debug) Task 1f34a68f-c8a2-4204-a41d-7508e756ed2c: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:13:11,777::task::495::TaskManager.Task::(_debug) Task 1f34a68f-c8a2-4204-a41d-7508e756ed2c: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:13:11,777::task::495::TaskManager.Task::(_debug) Task 1f34a68f-c8a2-4204-a41d-7508e756ed2c: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:13:11,778::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:13:11,779::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:13:11,779::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:13:11,780::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:13:11,781::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:13:11,781::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:13:11,782::task::495::TaskManager.Task::(_debug) Task 1f34a68f-c8a2-4204-a41d-7508e756ed2c: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:13:11,782::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:13:11,783::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:13:11,784::task::495::TaskManager.Task::(_debug) Task 1dfa4aa8-9386-4a6a-8d05-57cf2eaab965: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:13:11,785::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`67c441cc-de0a-42e2-8b2b-bc0deb45c4df`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:13:11,785::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:13:11,786::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:13:11,787::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`67c441cc-de0a-42e2-8b2b-bc0deb45c4df`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:13:11,788::task::495::TaskManager.Task::(_debug) Task 1dfa4aa8-9386-4a6a-8d05-57cf2eaab965: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:13:11,788::task::495::TaskManager.Task::(_debug) Task 1dfa4aa8-9386-4a6a-8d05-57cf2eaab965: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:13:11,789::task::495::TaskManager.Task::(_debug) Task 1dfa4aa8-9386-4a6a-8d05-57cf2eaab965: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:13:11,790::task::495::TaskManager.Task::(_debug) Task 1dfa4aa8-9386-4a6a-8d05-57cf2eaab965: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:13:11,790::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:13:11,791::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:13:11,791::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:13:11,792::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:13:11,793::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:13:11,793::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:13:11,794::task::495::TaskManager.Task::(_debug) Task 1dfa4aa8-9386-4a6a-8d05-57cf2eaab965: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:13:11,794::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:13:12,672::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:13:13,181::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0660211 s, 15.5 MB/s\n'; <rc> = 0
>Thread-15915955::DEBUG::2013-11-20 04:13:14,837::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915955::INFO::2013-11-20 04:13:14,838::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915955::DEBUG::2013-11-20 04:13:14,839::task::495::TaskManager.Task::(_debug) Task 8e4b902e-24c3-46fd-a919-82f611adf0a5: moving from state init -> state preparing
>Thread-15915955::DEBUG::2013-11-20 04:13:14,840::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915955::DEBUG::2013-11-20 04:13:14,841::task::495::TaskManager.Task::(_debug) Task 8e4b902e-24c3-46fd-a919-82f611adf0a5: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915955::DEBUG::2013-11-20 04:13:14,841::task::495::TaskManager.Task::(_debug) Task 8e4b902e-24c3-46fd-a919-82f611adf0a5: moving from state preparing -> state finished
>Thread-15915955::DEBUG::2013-11-20 04:13:14,842::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915955::DEBUG::2013-11-20 04:13:14,843::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915955::DEBUG::2013-11-20 04:13:14,843::task::495::TaskManager.Task::(_debug) Task 8e4b902e-24c3-46fd-a919-82f611adf0a5: ref 0 aborting False
>Thread-15915955::INFO::2013-11-20 04:13:14,844::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915956::DEBUG::2013-11-20 04:13:14,853::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915956::INFO::2013-11-20 04:13:14,854::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915956::DEBUG::2013-11-20 04:13:14,854::task::495::TaskManager.Task::(_debug) Task ef1ee532-6456-486b-8165-8d48a0ced363: moving from state init -> state preparing
>Thread-15915956::DEBUG::2013-11-20 04:13:14,856::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b9c8c7b2-92a1-4da1-a4be-16ace90b31cc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915956::DEBUG::2013-11-20 04:13:14,857::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915956::DEBUG::2013-11-20 04:13:14,858::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915956::DEBUG::2013-11-20 04:13:14,858::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b9c8c7b2-92a1-4da1-a4be-16ace90b31cc`::Granted request
>Thread-15915956::DEBUG::2013-11-20 04:13:14,860::task::495::TaskManager.Task::(_debug) Task ef1ee532-6456-486b-8165-8d48a0ced363: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915956::DEBUG::2013-11-20 04:13:14,860::task::495::TaskManager.Task::(_debug) Task ef1ee532-6456-486b-8165-8d48a0ced363: ref 1 aborting False
>Thread-15915956::WARNING::2013-11-20 04:13:14,863::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915956::DEBUG::2013-11-20 04:13:14,864::task::495::TaskManager.Task::(_debug) Task ef1ee532-6456-486b-8165-8d48a0ced363: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915956::DEBUG::2013-11-20 04:13:14,865::task::495::TaskManager.Task::(_debug) Task ef1ee532-6456-486b-8165-8d48a0ced363: moving from state preparing -> state finished
>Thread-15915956::DEBUG::2013-11-20 04:13:14,865::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915956::DEBUG::2013-11-20 04:13:14,866::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915956::DEBUG::2013-11-20 04:13:14,867::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915956::DEBUG::2013-11-20 04:13:14,868::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915956::DEBUG::2013-11-20 04:13:14,868::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915956::DEBUG::2013-11-20 04:13:14,869::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915956::DEBUG::2013-11-20 04:13:14,870::task::495::TaskManager.Task::(_debug) Task ef1ee532-6456-486b-8165-8d48a0ced363: ref 0 aborting False
>Thread-15915956::INFO::2013-11-20 04:13:14,870::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915958::DEBUG::2013-11-20 04:13:15,119::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('2afdc298-b10b-4bd1-8df2-2a401efd4f9c',) {}
>Thread-15915958::DEBUG::2013-11-20 04:13:15,121::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Disk stats not available
>Thread-15915958::DEBUG::2013-11-20 04:13:15,121::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Disk latency not available
>Thread-15915958::DEBUG::2013-11-20 04:13:15,123::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '45042', 'displayIp': '0', 'displayPort': u'5918', 'session': 'Unknown', 'displaySecurePort': u'5919', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet12': {'macAddr': u'00:1a:4a:a8:64:12', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '100', 'name': u'vnet12'}, u'vnet10': {'macAddr': u'00:1a:4a:a8:64:11', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet10'}}, 'vmId': '2afdc298-b10b-4bd1-8df2-2a401efd4f9c', 'monitorResponse': '0', 'cpuUser': '39.66', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '32346472448', 'writeLatency': '0', 'imageID': 'e0628852-41b1-4535-8dda-ebc96cb00952', 'flushLatency': '0', 'truesize': '32346472448'}}, 'boot': 'c', 'statsAge': '0.51', 'cpuIdle': '60.34', 'elapsedTime': '63', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 04:13:15,196::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:13:15,748::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.076592 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:13:17,764::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:13:18,358::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0802189 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:13:20,374::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:13:20,956::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0752767 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:13:22,965::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:13:23,426::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0613861 s, 16.7 MB/s\n'; <rc> = 0
>Thread-15915962::DEBUG::2013-11-20 04:13:25,086::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915962::INFO::2013-11-20 04:13:25,087::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915962::DEBUG::2013-11-20 04:13:25,088::task::495::TaskManager.Task::(_debug) Task adc30322-92ce-49d6-bf2d-6a20f4f42de7: moving from state init -> state preparing
>Thread-15915962::DEBUG::2013-11-20 04:13:25,089::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915962::DEBUG::2013-11-20 04:13:25,089::task::495::TaskManager.Task::(_debug) Task adc30322-92ce-49d6-bf2d-6a20f4f42de7: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915962::DEBUG::2013-11-20 04:13:25,090::task::495::TaskManager.Task::(_debug) Task adc30322-92ce-49d6-bf2d-6a20f4f42de7: moving from state preparing -> state finished
>Thread-15915962::DEBUG::2013-11-20 04:13:25,090::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915962::DEBUG::2013-11-20 04:13:25,091::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915962::DEBUG::2013-11-20 04:13:25,091::task::495::TaskManager.Task::(_debug) Task adc30322-92ce-49d6-bf2d-6a20f4f42de7: ref 0 aborting False
>Thread-15915962::INFO::2013-11-20 04:13:25,092::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915963::DEBUG::2013-11-20 04:13:25,105::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915963::INFO::2013-11-20 04:13:25,107::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915963::DEBUG::2013-11-20 04:13:25,107::task::495::TaskManager.Task::(_debug) Task 3f231cf7-4ec5-4401-a7a3-b785f8cbbe10: moving from state init -> state preparing
>Thread-15915963::DEBUG::2013-11-20 04:13:25,109::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c51283e4-7335-4c35-9bc5-c1d6a593b3df`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915963::DEBUG::2013-11-20 04:13:25,110::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915963::DEBUG::2013-11-20 04:13:25,111::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915963::DEBUG::2013-11-20 04:13:25,112::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c51283e4-7335-4c35-9bc5-c1d6a593b3df`::Granted request
>Thread-15915963::DEBUG::2013-11-20 04:13:25,113::task::495::TaskManager.Task::(_debug) Task 3f231cf7-4ec5-4401-a7a3-b785f8cbbe10: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915963::DEBUG::2013-11-20 04:13:25,113::task::495::TaskManager.Task::(_debug) Task 3f231cf7-4ec5-4401-a7a3-b785f8cbbe10: ref 1 aborting False
>Thread-15915963::WARNING::2013-11-20 04:13:25,115::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915963::DEBUG::2013-11-20 04:13:25,116::task::495::TaskManager.Task::(_debug) Task 3f231cf7-4ec5-4401-a7a3-b785f8cbbe10: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915963::DEBUG::2013-11-20 04:13:25,117::task::495::TaskManager.Task::(_debug) Task 3f231cf7-4ec5-4401-a7a3-b785f8cbbe10: moving from state preparing -> state finished
>Thread-15915963::DEBUG::2013-11-20 04:13:25,118::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915963::DEBUG::2013-11-20 04:13:25,118::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915963::DEBUG::2013-11-20 04:13:25,119::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915963::DEBUG::2013-11-20 04:13:25,120::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915963::DEBUG::2013-11-20 04:13:25,120::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915963::DEBUG::2013-11-20 04:13:25,121::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915963::DEBUG::2013-11-20 04:13:25,122::task::495::TaskManager.Task::(_debug) Task 3f231cf7-4ec5-4401-a7a3-b785f8cbbe10: ref 0 aborting False
>Thread-15915963::INFO::2013-11-20 04:13:25,122::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:13:25,440::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:13:25,999::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0854793 s, 12.0 MB/s\n'; <rc> = 0
>Thread-15913639::INFO::2013-11-20 04:13:26,610::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:13:26,611::task::495::TaskManager.Task::(_debug) Task cc4b3a84-1aed-4aba-9a2e-c5744e7edf5c: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:13:26,614::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5e0fe5f8-11cc-43c1-a89f-8764ab823ef9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:13:26,614::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:13:26,616::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:13:26,616::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5e0fe5f8-11cc-43c1-a89f-8764ab823ef9`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:13:26,618::task::495::TaskManager.Task::(_debug) Task cc4b3a84-1aed-4aba-9a2e-c5744e7edf5c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:13:26,618::task::495::TaskManager.Task::(_debug) Task cc4b3a84-1aed-4aba-9a2e-c5744e7edf5c: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:13:26,619::task::495::TaskManager.Task::(_debug) Task cc4b3a84-1aed-4aba-9a2e-c5744e7edf5c: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:13:26,620::task::495::TaskManager.Task::(_debug) Task cc4b3a84-1aed-4aba-9a2e-c5744e7edf5c: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:13:26,621::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:13:26,621::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:13:26,622::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:13:26,623::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:13:26,623::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:13:26,624::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:13:26,625::task::495::TaskManager.Task::(_debug) Task cc4b3a84-1aed-4aba-9a2e-c5744e7edf5c: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:13:26,626::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15915964::INFO::2013-11-20 04:13:27,650::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915964::DEBUG::2013-11-20 04:13:27,650::task::495::TaskManager.Task::(_debug) Task a1ddafb0-beb5-4a8d-a5ef-8e37bc998e45: moving from state init -> state preparing
>Thread-15915964::DEBUG::2013-11-20 04:13:27,651::task::495::TaskManager.Task::(_debug) Task a1ddafb0-beb5-4a8d-a5ef-8e37bc998e45: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000351905822754', 'lastCheck': 1384920798.0086141, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00823283195496', 'lastCheck': 1384920797.4500279, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0219578742981', 'lastCheck': 1384920804.1735311, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000458002090454', 'lastCheck': 1384920802.008306, 'valid': True, 'code': 0}}
>Thread-15915964::DEBUG::2013-11-20 04:13:27,653::task::495::TaskManager.Task::(_debug) Task a1ddafb0-beb5-4a8d-a5ef-8e37bc998e45: moving from state preparing -> state finished
>Thread-15915964::DEBUG::2013-11-20 04:13:27,654::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915964::DEBUG::2013-11-20 04:13:27,656::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915964::DEBUG::2013-11-20 04:13:27,657::task::495::TaskManager.Task::(_debug) Task a1ddafb0-beb5-4a8d-a5ef-8e37bc998e45: ref 0 aborting False
>Thread-15915964::INFO::2013-11-20 04:13:27,658::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000351905822754', 'lastCheck': 1384920798.0086141, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00823283195496', 'lastCheck': 1384920797.4500279, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0219578742981', 'lastCheck': 1384920804.1735311, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000458002090454', 'lastCheck': 1384920802.008306, 'valid': True, 'code': 0}}
>Thread-15915965::DEBUG::2013-11-20 04:13:27,707::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Disk stats not available
>Thread-15915965::DEBUG::2013-11-20 04:13:27,708::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`2afdc298-b10b-4bd1-8df2-2a401efd4f9c`::Disk latency not available
>Dummy-15912651::DEBUG::2013-11-20 04:13:28,016::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:13:28,490::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0591645 s, 17.3 MB/s\n'; <rc> = 0
>Thread-15915918::INFO::2013-11-20 04:13:28,625::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:13:28,626::task::495::TaskManager.Task::(_debug) Task f9067cea-a1c3-41b1-8c55-1bafb6399a2f: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:13:28,628::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a32192cd-2581-4c3d-b421-19a0e24b7c3e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:13:28,629::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:13:28,630::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:13:28,630::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a32192cd-2581-4c3d-b421-19a0e24b7c3e`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:13:28,631::task::495::TaskManager.Task::(_debug) Task f9067cea-a1c3-41b1-8c55-1bafb6399a2f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:13:28,632::task::495::TaskManager.Task::(_debug) Task f9067cea-a1c3-41b1-8c55-1bafb6399a2f: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:13:28,633::task::495::TaskManager.Task::(_debug) Task f9067cea-a1c3-41b1-8c55-1bafb6399a2f: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:13:28,634::task::495::TaskManager.Task::(_debug) Task f9067cea-a1c3-41b1-8c55-1bafb6399a2f: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:13:28,634::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:13:28,635::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:13:28,635::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:13:28,636::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:13:28,637::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:13:28,637::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:13:28,638::task::495::TaskManager.Task::(_debug) Task f9067cea-a1c3-41b1-8c55-1bafb6399a2f: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:13:28,639::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:13:30,502::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:13:31,111::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0899461 s, 11.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:13:33,127::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:13:33,714::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0815072 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15915863::INFO::2013-11-20 04:13:33,744::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:13:33,747::task::495::TaskManager.Task::(_debug) Task d35cf1be-ae49-46ec-8b0f-75e9208a28f4: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:13:33,754::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d6f7f358-40ab-41b5-badf-e46f52d22085`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:13:33,762::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:13:33,763::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:13:33,764::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d6f7f358-40ab-41b5-badf-e46f52d22085`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:13:33,766::task::495::TaskManager.Task::(_debug) Task d35cf1be-ae49-46ec-8b0f-75e9208a28f4: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:13:33,767::task::495::TaskManager.Task::(_debug) Task d35cf1be-ae49-46ec-8b0f-75e9208a28f4: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:13:33,769::task::495::TaskManager.Task::(_debug) Task d35cf1be-ae49-46ec-8b0f-75e9208a28f4: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:13:33,770::task::495::TaskManager.Task::(_debug) Task d35cf1be-ae49-46ec-8b0f-75e9208a28f4: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:13:33,770::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:13:33,771::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:13:33,772::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:13:33,772::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:13:33,773::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:13:33,773::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:13:33,775::task::495::TaskManager.Task::(_debug) Task d35cf1be-ae49-46ec-8b0f-75e9208a28f4: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:13:33,776::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915968::DEBUG::2013-11-20 04:13:35,684::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915968::INFO::2013-11-20 04:13:35,685::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915968::DEBUG::2013-11-20 04:13:35,686::task::495::TaskManager.Task::(_debug) Task 9c99ae04-f7ec-4b4e-9245-c6c389e1199c: moving from state init -> state preparing
>Thread-15915968::DEBUG::2013-11-20 04:13:35,687::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915968::DEBUG::2013-11-20 04:13:35,687::task::495::TaskManager.Task::(_debug) Task 9c99ae04-f7ec-4b4e-9245-c6c389e1199c: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915968::DEBUG::2013-11-20 04:13:35,688::task::495::TaskManager.Task::(_debug) Task 9c99ae04-f7ec-4b4e-9245-c6c389e1199c: moving from state preparing -> state finished
>Thread-15915968::DEBUG::2013-11-20 04:13:35,689::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915968::DEBUG::2013-11-20 04:13:35,689::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915968::DEBUG::2013-11-20 04:13:35,690::task::495::TaskManager.Task::(_debug) Task 9c99ae04-f7ec-4b4e-9245-c6c389e1199c: ref 0 aborting False
>Thread-15915968::INFO::2013-11-20 04:13:35,691::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915969::DEBUG::2013-11-20 04:13:35,695::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915969::INFO::2013-11-20 04:13:35,696::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915969::DEBUG::2013-11-20 04:13:35,697::task::495::TaskManager.Task::(_debug) Task 8b069388-bcf3-4f28-95be-304840cc9f41: moving from state init -> state preparing
>Thread-15915969::DEBUG::2013-11-20 04:13:35,698::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9c68c4b6-57e5-4fd3-a8fb-93ef1141db9c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915969::DEBUG::2013-11-20 04:13:35,699::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915969::DEBUG::2013-11-20 04:13:35,700::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915969::DEBUG::2013-11-20 04:13:35,700::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9c68c4b6-57e5-4fd3-a8fb-93ef1141db9c`::Granted request
>Thread-15915969::DEBUG::2013-11-20 04:13:35,701::task::495::TaskManager.Task::(_debug) Task 8b069388-bcf3-4f28-95be-304840cc9f41: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915969::DEBUG::2013-11-20 04:13:35,702::task::495::TaskManager.Task::(_debug) Task 8b069388-bcf3-4f28-95be-304840cc9f41: ref 1 aborting False
>Thread-15915969::WARNING::2013-11-20 04:13:35,704::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915969::DEBUG::2013-11-20 04:13:35,706::task::495::TaskManager.Task::(_debug) Task 8b069388-bcf3-4f28-95be-304840cc9f41: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915969::DEBUG::2013-11-20 04:13:35,707::task::495::TaskManager.Task::(_debug) Task 8b069388-bcf3-4f28-95be-304840cc9f41: moving from state preparing -> state finished
>Thread-15915969::DEBUG::2013-11-20 04:13:35,708::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915969::DEBUG::2013-11-20 04:13:35,709::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915969::DEBUG::2013-11-20 04:13:35,709::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915969::DEBUG::2013-11-20 04:13:35,710::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915969::DEBUG::2013-11-20 04:13:35,711::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915969::DEBUG::2013-11-20 04:13:35,711::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915969::DEBUG::2013-11-20 04:13:35,712::task::495::TaskManager.Task::(_debug) Task 8b069388-bcf3-4f28-95be-304840cc9f41: ref 0 aborting False
>Thread-15915969::INFO::2013-11-20 04:13:35,713::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:13:35,733::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:13:36,294::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0610152 s, 16.8 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 04:13:38,003::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:13:38,004::task::495::TaskManager.Task::(_debug) Task 23fd4880-11d9-493e-998d-3a1d8b4f2782: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:13:38,006::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`61b6b010-1499-4a58-8464-b906f96350d5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:13:38,006::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:13:38,007::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:13:38,008::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`61b6b010-1499-4a58-8464-b906f96350d5`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:13:38,009::task::495::TaskManager.Task::(_debug) Task 23fd4880-11d9-493e-998d-3a1d8b4f2782: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:13:38,009::task::495::TaskManager.Task::(_debug) Task 23fd4880-11d9-493e-998d-3a1d8b4f2782: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:13:38,010::task::495::TaskManager.Task::(_debug) Task 23fd4880-11d9-493e-998d-3a1d8b4f2782: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:13:38,011::task::495::TaskManager.Task::(_debug) Task 23fd4880-11d9-493e-998d-3a1d8b4f2782: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:13:38,011::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:13:38,012::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:13:38,013::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:13:38,014::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:13:38,014::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:13:38,015::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:13:38,015::task::495::TaskManager.Task::(_debug) Task 23fd4880-11d9-493e-998d-3a1d8b4f2782: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:13:38,016::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:13:38,017::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:13:38,018::task::495::TaskManager.Task::(_debug) Task 65bb1d27-9db0-477a-9ef7-8db735bf2567: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:13:38,019::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`16793342-47af-47e6-a21a-3858a3e8ff93`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:13:38,019::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:13:38,020::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:13:38,020::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`16793342-47af-47e6-a21a-3858a3e8ff93`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:13:38,021::task::495::TaskManager.Task::(_debug) Task 65bb1d27-9db0-477a-9ef7-8db735bf2567: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:13:38,022::task::495::TaskManager.Task::(_debug) Task 65bb1d27-9db0-477a-9ef7-8db735bf2567: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:13:38,022::task::495::TaskManager.Task::(_debug) Task 65bb1d27-9db0-477a-9ef7-8db735bf2567: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:13:38,023::task::495::TaskManager.Task::(_debug) Task 65bb1d27-9db0-477a-9ef7-8db735bf2567: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:13:38,024::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:13:38,024::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:13:38,025::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:13:38,025::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:13:38,026::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:13:38,026::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:13:38,027::task::495::TaskManager.Task::(_debug) Task 65bb1d27-9db0-477a-9ef7-8db735bf2567: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:13:38,028::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:13:38,308::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:13:38,860::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0802463 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:13:40,875::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:13:41,448::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0872532 s, 11.7 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 04:13:42,236::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:13:42,237::task::495::TaskManager.Task::(_debug) Task d53ecc58-84a6-49b0-9f94-7a94066d9574: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:13:42,239::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1d960041-5252-4f9b-92f5-19ca1b477968`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:13:42,240::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:13:42,240::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:13:42,241::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1d960041-5252-4f9b-92f5-19ca1b477968`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:13:42,242::task::495::TaskManager.Task::(_debug) Task d53ecc58-84a6-49b0-9f94-7a94066d9574: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:13:42,243::task::495::TaskManager.Task::(_debug) Task d53ecc58-84a6-49b0-9f94-7a94066d9574: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:13:42,243::task::495::TaskManager.Task::(_debug) Task d53ecc58-84a6-49b0-9f94-7a94066d9574: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:13:42,244::task::495::TaskManager.Task::(_debug) Task d53ecc58-84a6-49b0-9f94-7a94066d9574: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:13:42,245::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:13:42,245::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:13:42,246::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:13:42,247::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:13:42,247::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:13:42,248::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:13:42,249::task::495::TaskManager.Task::(_debug) Task d53ecc58-84a6-49b0-9f94-7a94066d9574: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:13:42,250::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:13:43,458::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:13:43,998::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0813446 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15915972::INFO::2013-11-20 04:13:44,132::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915972::DEBUG::2013-11-20 04:13:44,134::task::495::TaskManager.Task::(_debug) Task c8c96e62-f7c3-4ac8-98c6-0223de728aba: moving from state init -> state preparing
>Thread-15915972::DEBUG::2013-11-20 04:13:44,135::task::495::TaskManager.Task::(_debug) Task c8c96e62-f7c3-4ac8-98c6-0223de728aba: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000389099121094', 'lastCheck': 1384920818.5332839, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00118279457092', 'lastCheck': 1384920817.955539, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00135278701782', 'lastCheck': 1384920814.362396, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000461101531982', 'lastCheck': 1384920822.526475, 'valid': True, 'code': 0}}
>Thread-15915972::DEBUG::2013-11-20 04:13:44,135::task::495::TaskManager.Task::(_debug) Task c8c96e62-f7c3-4ac8-98c6-0223de728aba: moving from state preparing -> state finished
>Thread-15915972::DEBUG::2013-11-20 04:13:44,136::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915972::DEBUG::2013-11-20 04:13:44,137::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915972::DEBUG::2013-11-20 04:13:44,138::task::495::TaskManager.Task::(_debug) Task c8c96e62-f7c3-4ac8-98c6-0223de728aba: ref 0 aborting False
>Thread-15915972::INFO::2013-11-20 04:13:44,138::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000389099121094', 'lastCheck': 1384920818.5332839, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00118279457092', 'lastCheck': 1384920817.955539, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00135278701782', 'lastCheck': 1384920814.362396, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000461101531982', 'lastCheck': 1384920822.526475, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:13:46,014::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:13:46,576::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0702684 s, 14.6 MB/s\n'; <rc> = 0
>Thread-15915974::DEBUG::2013-11-20 04:13:46,647::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915974::INFO::2013-11-20 04:13:46,649::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915974::DEBUG::2013-11-20 04:13:46,650::task::495::TaskManager.Task::(_debug) Task f6248146-5e96-4401-95a4-3d3fa0ff0933: moving from state init -> state preparing
>Thread-15915974::DEBUG::2013-11-20 04:13:46,651::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915974::DEBUG::2013-11-20 04:13:46,652::task::495::TaskManager.Task::(_debug) Task f6248146-5e96-4401-95a4-3d3fa0ff0933: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915974::DEBUG::2013-11-20 04:13:46,653::task::495::TaskManager.Task::(_debug) Task f6248146-5e96-4401-95a4-3d3fa0ff0933: moving from state preparing -> state finished
>Thread-15915974::DEBUG::2013-11-20 04:13:46,653::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915974::DEBUG::2013-11-20 04:13:46,654::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915974::DEBUG::2013-11-20 04:13:46,655::task::495::TaskManager.Task::(_debug) Task f6248146-5e96-4401-95a4-3d3fa0ff0933: ref 0 aborting False
>Thread-15915974::INFO::2013-11-20 04:13:46,656::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915975::DEBUG::2013-11-20 04:13:46,661::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915975::INFO::2013-11-20 04:13:46,662::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915975::DEBUG::2013-11-20 04:13:46,663::task::495::TaskManager.Task::(_debug) Task 33d5a4c8-7996-4368-8ccf-b3ce6dbb6c9a: moving from state init -> state preparing
>Thread-15915975::DEBUG::2013-11-20 04:13:46,666::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9d30ff7c-af50-4bd5-b2e8-f9f253f23dc4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915975::DEBUG::2013-11-20 04:13:46,669::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915975::DEBUG::2013-11-20 04:13:46,670::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915975::DEBUG::2013-11-20 04:13:46,670::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9d30ff7c-af50-4bd5-b2e8-f9f253f23dc4`::Granted request
>Thread-15915975::DEBUG::2013-11-20 04:13:46,677::task::495::TaskManager.Task::(_debug) Task 33d5a4c8-7996-4368-8ccf-b3ce6dbb6c9a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915975::DEBUG::2013-11-20 04:13:46,678::task::495::TaskManager.Task::(_debug) Task 33d5a4c8-7996-4368-8ccf-b3ce6dbb6c9a: ref 1 aborting False
>Thread-15915975::WARNING::2013-11-20 04:13:46,681::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915975::DEBUG::2013-11-20 04:13:46,687::task::495::TaskManager.Task::(_debug) Task 33d5a4c8-7996-4368-8ccf-b3ce6dbb6c9a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915975::DEBUG::2013-11-20 04:13:46,689::task::495::TaskManager.Task::(_debug) Task 33d5a4c8-7996-4368-8ccf-b3ce6dbb6c9a: moving from state preparing -> state finished
>Thread-15915975::DEBUG::2013-11-20 04:13:46,692::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915975::DEBUG::2013-11-20 04:13:46,694::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915975::DEBUG::2013-11-20 04:13:46,696::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915975::DEBUG::2013-11-20 04:13:46,699::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915975::DEBUG::2013-11-20 04:13:46,699::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915975::DEBUG::2013-11-20 04:13:46,700::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915975::DEBUG::2013-11-20 04:13:46,701::task::495::TaskManager.Task::(_debug) Task 33d5a4c8-7996-4368-8ccf-b3ce6dbb6c9a: ref 0 aborting False
>Thread-15915975::INFO::2013-11-20 04:13:46,701::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:13:48,590::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:13:49,193::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0803698 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:13:51,206::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:13:51,749::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0804918 s, 12.7 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 04:13:51,979::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:13:51,980::task::495::TaskManager.Task::(_debug) Task 5c52d1bc-a7d2-4d2f-8a25-aedc35e7c6ea: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:13:51,982::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0947142d-6fad-4137-bd40-a8bd197e984c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:13:51,982::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:13:51,983::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:13:51,984::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0947142d-6fad-4137-bd40-a8bd197e984c`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:13:51,985::task::495::TaskManager.Task::(_debug) Task 5c52d1bc-a7d2-4d2f-8a25-aedc35e7c6ea: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:13:51,986::task::495::TaskManager.Task::(_debug) Task 5c52d1bc-a7d2-4d2f-8a25-aedc35e7c6ea: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:13:51,986::task::495::TaskManager.Task::(_debug) Task 5c52d1bc-a7d2-4d2f-8a25-aedc35e7c6ea: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:13:51,987::task::495::TaskManager.Task::(_debug) Task 5c52d1bc-a7d2-4d2f-8a25-aedc35e7c6ea: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:13:51,988::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:13:51,989::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:13:51,989::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:13:51,990::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:13:51,990::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:13:51,991::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:13:51,991::task::495::TaskManager.Task::(_debug) Task 5c52d1bc-a7d2-4d2f-8a25-aedc35e7c6ea: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:13:51,992::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:13:53,763::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:13:54,322::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0929504 s, 11.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:13:56,332::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:13:56,789::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.052391 s, 19.5 MB/s\n'; <rc> = 0
>Thread-15915979::DEBUG::2013-11-20 04:13:57,096::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915979::INFO::2013-11-20 04:13:57,097::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915979::DEBUG::2013-11-20 04:13:57,098::task::495::TaskManager.Task::(_debug) Task 5e6b6623-cc3f-4451-a6e5-9fa0e015eddc: moving from state init -> state preparing
>Thread-15915979::DEBUG::2013-11-20 04:13:57,099::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915979::DEBUG::2013-11-20 04:13:57,100::task::495::TaskManager.Task::(_debug) Task 5e6b6623-cc3f-4451-a6e5-9fa0e015eddc: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915979::DEBUG::2013-11-20 04:13:57,101::task::495::TaskManager.Task::(_debug) Task 5e6b6623-cc3f-4451-a6e5-9fa0e015eddc: moving from state preparing -> state finished
>Thread-15915979::DEBUG::2013-11-20 04:13:57,101::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915979::DEBUG::2013-11-20 04:13:57,102::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915979::DEBUG::2013-11-20 04:13:57,102::task::495::TaskManager.Task::(_debug) Task 5e6b6623-cc3f-4451-a6e5-9fa0e015eddc: ref 0 aborting False
>Thread-15915979::INFO::2013-11-20 04:13:57,103::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915980::DEBUG::2013-11-20 04:13:57,108::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915980::INFO::2013-11-20 04:13:57,109::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915980::DEBUG::2013-11-20 04:13:57,110::task::495::TaskManager.Task::(_debug) Task fd74e25f-6478-4440-baa8-ea8bf2927ece: moving from state init -> state preparing
>Thread-15915980::DEBUG::2013-11-20 04:13:57,111::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`88e230ff-1f5d-4ecb-aab1-0a54cd5e9b9c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915980::DEBUG::2013-11-20 04:13:57,112::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915980::DEBUG::2013-11-20 04:13:57,113::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915980::DEBUG::2013-11-20 04:13:57,113::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`88e230ff-1f5d-4ecb-aab1-0a54cd5e9b9c`::Granted request
>Thread-15915980::DEBUG::2013-11-20 04:13:57,114::task::495::TaskManager.Task::(_debug) Task fd74e25f-6478-4440-baa8-ea8bf2927ece: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915980::DEBUG::2013-11-20 04:13:57,115::task::495::TaskManager.Task::(_debug) Task fd74e25f-6478-4440-baa8-ea8bf2927ece: ref 1 aborting False
>Thread-15915980::WARNING::2013-11-20 04:13:57,117::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915980::DEBUG::2013-11-20 04:13:57,119::task::495::TaskManager.Task::(_debug) Task fd74e25f-6478-4440-baa8-ea8bf2927ece: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915980::DEBUG::2013-11-20 04:13:57,120::task::495::TaskManager.Task::(_debug) Task fd74e25f-6478-4440-baa8-ea8bf2927ece: moving from state preparing -> state finished
>Thread-15915980::DEBUG::2013-11-20 04:13:57,120::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915980::DEBUG::2013-11-20 04:13:57,121::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915980::DEBUG::2013-11-20 04:13:57,122::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915980::DEBUG::2013-11-20 04:13:57,122::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915980::DEBUG::2013-11-20 04:13:57,123::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915980::DEBUG::2013-11-20 04:13:57,124::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915980::DEBUG::2013-11-20 04:13:57,124::task::495::TaskManager.Task::(_debug) Task fd74e25f-6478-4440-baa8-ea8bf2927ece: ref 0 aborting False
>Thread-15915980::INFO::2013-11-20 04:13:57,125::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913695::INFO::2013-11-20 04:13:57,211::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:13:57,212::task::495::TaskManager.Task::(_debug) Task 86f3fd42-585a-4066-9db7-3a15a23dad8c: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:13:57,213::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ecc5cc08-e709-474f-a358-2ed2791f3608`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:13:57,214::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:13:57,214::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:13:57,215::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ecc5cc08-e709-474f-a358-2ed2791f3608`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:13:57,216::task::495::TaskManager.Task::(_debug) Task 86f3fd42-585a-4066-9db7-3a15a23dad8c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:13:57,216::task::495::TaskManager.Task::(_debug) Task 86f3fd42-585a-4066-9db7-3a15a23dad8c: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:13:57,217::task::495::TaskManager.Task::(_debug) Task 86f3fd42-585a-4066-9db7-3a15a23dad8c: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:13:57,218::task::495::TaskManager.Task::(_debug) Task 86f3fd42-585a-4066-9db7-3a15a23dad8c: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:13:57,219::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:13:57,219::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:13:57,220::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:13:57,220::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:13:57,220::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:13:57,221::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:13:57,221::task::495::TaskManager.Task::(_debug) Task 86f3fd42-585a-4066-9db7-3a15a23dad8c: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:13:57,222::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:13:58,802::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:13:59,250::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0561483 s, 18.2 MB/s\n'; <rc> = 0
>Thread-15915982::INFO::2013-11-20 04:14:00,518::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915982::DEBUG::2013-11-20 04:14:00,519::task::495::TaskManager.Task::(_debug) Task fcb0b658-4f47-4201-99cb-93d0e15121f2: moving from state init -> state preparing
>Thread-15915982::DEBUG::2013-11-20 04:14:00,520::task::495::TaskManager.Task::(_debug) Task fcb0b658-4f47-4201-99cb-93d0e15121f2: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000375032424927', 'lastCheck': 1384920839.0043199, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.10501408577', 'lastCheck': 1384920838.6442621, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00153112411499', 'lastCheck': 1384920834.9854021, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000475883483887', 'lastCheck': 1384920832.8574629, 'valid': True, 'code': 0}}
>Thread-15915982::DEBUG::2013-11-20 04:14:00,522::task::495::TaskManager.Task::(_debug) Task fcb0b658-4f47-4201-99cb-93d0e15121f2: moving from state preparing -> state finished
>Thread-15915982::DEBUG::2013-11-20 04:14:00,522::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915982::DEBUG::2013-11-20 04:14:00,523::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915982::DEBUG::2013-11-20 04:14:00,524::task::495::TaskManager.Task::(_debug) Task fcb0b658-4f47-4201-99cb-93d0e15121f2: ref 0 aborting False
>Thread-15915982::INFO::2013-11-20 04:14:00,525::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000375032424927', 'lastCheck': 1384920839.0043199, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.10501408577', 'lastCheck': 1384920838.6442621, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00153112411499', 'lastCheck': 1384920834.9854021, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000475883483887', 'lastCheck': 1384920832.8574629, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:14:01,277::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:14:01,863::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0884378 s, 11.6 MB/s\n'; <rc> = 0
>Thread-15914453::INFO::2013-11-20 04:14:01,981::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:14:01,983::task::495::TaskManager.Task::(_debug) Task 0acb5997-1c1f-4696-ba9d-83f0c1ca44b0: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:14:01,985::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`267fde7a-7b87-4041-886f-0c197dd32d08`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:14:01,985::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:14:01,986::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:14:01,987::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`267fde7a-7b87-4041-886f-0c197dd32d08`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:14:01,988::task::495::TaskManager.Task::(_debug) Task 0acb5997-1c1f-4696-ba9d-83f0c1ca44b0: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:14:01,989::task::495::TaskManager.Task::(_debug) Task 0acb5997-1c1f-4696-ba9d-83f0c1ca44b0: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:14:01,989::task::495::TaskManager.Task::(_debug) Task 0acb5997-1c1f-4696-ba9d-83f0c1ca44b0: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:14:01,990::task::495::TaskManager.Task::(_debug) Task 0acb5997-1c1f-4696-ba9d-83f0c1ca44b0: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:14:01,991::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:14:01,991::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:14:01,992::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:14:01,993::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:14:01,993::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:14:01,994::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:14:01,994::task::495::TaskManager.Task::(_debug) Task 0acb5997-1c1f-4696-ba9d-83f0c1ca44b0: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:14:01,995::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:14:03,880::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:14:04,371::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0895691 s, 11.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:14:06,386::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:14:06,860::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0616039 s, 16.6 MB/s\n'; <rc> = 0
>Thread-15915986::DEBUG::2013-11-20 04:14:07,336::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915986::INFO::2013-11-20 04:14:07,337::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915986::DEBUG::2013-11-20 04:14:07,338::task::495::TaskManager.Task::(_debug) Task 19f56f3a-4cb6-486a-94ca-4199e134738f: moving from state init -> state preparing
>Thread-15915986::DEBUG::2013-11-20 04:14:07,339::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915986::DEBUG::2013-11-20 04:14:07,340::task::495::TaskManager.Task::(_debug) Task 19f56f3a-4cb6-486a-94ca-4199e134738f: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915986::DEBUG::2013-11-20 04:14:07,340::task::495::TaskManager.Task::(_debug) Task 19f56f3a-4cb6-486a-94ca-4199e134738f: moving from state preparing -> state finished
>Thread-15915986::DEBUG::2013-11-20 04:14:07,341::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915986::DEBUG::2013-11-20 04:14:07,341::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915986::DEBUG::2013-11-20 04:14:07,342::task::495::TaskManager.Task::(_debug) Task 19f56f3a-4cb6-486a-94ca-4199e134738f: ref 0 aborting False
>Thread-15915986::INFO::2013-11-20 04:14:07,343::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915987::DEBUG::2013-11-20 04:14:07,351::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915987::INFO::2013-11-20 04:14:07,352::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915987::DEBUG::2013-11-20 04:14:07,352::task::495::TaskManager.Task::(_debug) Task 0bd99f32-e560-4395-af9a-da6aa4b2e3a6: moving from state init -> state preparing
>Thread-15915987::DEBUG::2013-11-20 04:14:07,354::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`646622c3-a18f-4920-a66a-17acfb3e817d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915987::DEBUG::2013-11-20 04:14:07,354::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915987::DEBUG::2013-11-20 04:14:07,355::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915987::DEBUG::2013-11-20 04:14:07,356::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`646622c3-a18f-4920-a66a-17acfb3e817d`::Granted request
>Thread-15915987::DEBUG::2013-11-20 04:14:07,357::task::495::TaskManager.Task::(_debug) Task 0bd99f32-e560-4395-af9a-da6aa4b2e3a6: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915987::DEBUG::2013-11-20 04:14:07,358::task::495::TaskManager.Task::(_debug) Task 0bd99f32-e560-4395-af9a-da6aa4b2e3a6: ref 1 aborting False
>Thread-15915987::WARNING::2013-11-20 04:14:07,360::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915987::DEBUG::2013-11-20 04:14:07,361::task::495::TaskManager.Task::(_debug) Task 0bd99f32-e560-4395-af9a-da6aa4b2e3a6: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915987::DEBUG::2013-11-20 04:14:07,362::task::495::TaskManager.Task::(_debug) Task 0bd99f32-e560-4395-af9a-da6aa4b2e3a6: moving from state preparing -> state finished
>Thread-15915987::DEBUG::2013-11-20 04:14:07,363::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915987::DEBUG::2013-11-20 04:14:07,363::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915987::DEBUG::2013-11-20 04:14:07,364::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915987::DEBUG::2013-11-20 04:14:07,365::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915987::DEBUG::2013-11-20 04:14:07,365::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915987::DEBUG::2013-11-20 04:14:07,366::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915987::DEBUG::2013-11-20 04:14:07,367::task::495::TaskManager.Task::(_debug) Task 0bd99f32-e560-4395-af9a-da6aa4b2e3a6: ref 0 aborting False
>Thread-15915987::INFO::2013-11-20 04:14:07,367::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:14:08,879::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 04:14:09,148::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:14:09,156::task::495::TaskManager.Task::(_debug) Task 7e3feab0-3434-4580-a738-b93ae9f3ab0a: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:14:09,162::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`62ffd624-0999-4488-8f7f-935aef22b5e5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:14:09,171::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:14:09,172::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:14:09,173::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`62ffd624-0999-4488-8f7f-935aef22b5e5`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:14:09,176::task::495::TaskManager.Task::(_debug) Task 7e3feab0-3434-4580-a738-b93ae9f3ab0a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 04:14:09,177::task::495::TaskManager.Task::(_debug) Task 7e3feab0-3434-4580-a738-b93ae9f3ab0a: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:14:09,178::task::495::TaskManager.Task::(_debug) Task 7e3feab0-3434-4580-a738-b93ae9f3ab0a: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 04:14:09,178::task::495::TaskManager.Task::(_debug) Task 7e3feab0-3434-4580-a738-b93ae9f3ab0a: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:14:09,179::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:14:09,180::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:14:09,181::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 04:14:09,182::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:14:09,182::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:14:09,183::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:14:09,184::task::495::TaskManager.Task::(_debug) Task 7e3feab0-3434-4580-a738-b93ae9f3ab0a: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 04:14:09,184::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914435::INFO::2013-11-20 04:14:09,283::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:14:09,284::task::495::TaskManager.Task::(_debug) Task e9581bb3-452f-4a01-933c-56d062126364: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:14:09,286::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`536329e5-83cd-4ee8-9e0d-729057300fd0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:14:09,287::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:14:09,287::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:14:09,288::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`536329e5-83cd-4ee8-9e0d-729057300fd0`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:14:09,289::task::495::TaskManager.Task::(_debug) Task e9581bb3-452f-4a01-933c-56d062126364: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:14:09,290::task::495::TaskManager.Task::(_debug) Task e9581bb3-452f-4a01-933c-56d062126364: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:14:09,291::task::495::TaskManager.Task::(_debug) Task e9581bb3-452f-4a01-933c-56d062126364: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:14:09,291::task::495::TaskManager.Task::(_debug) Task e9581bb3-452f-4a01-933c-56d062126364: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:14:09,292::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:14:09,292::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:14:09,293::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:14:09,294::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:14:09,295::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:14:09,295::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:14:09,296::task::495::TaskManager.Task::(_debug) Task e9581bb3-452f-4a01-933c-56d062126364: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:14:09,297::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:14:09,451::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0766984 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:14:11,466::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:14:11,990::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0836041 s, 12.2 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 04:14:12,251::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:14:12,252::task::495::TaskManager.Task::(_debug) Task f9b25438-24ff-46ba-af76-e254f269330e: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:14:12,254::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fc030c38-b970-4c00-96f1-0e7a049ca11b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:14:12,254::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:14:12,255::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:14:12,256::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fc030c38-b970-4c00-96f1-0e7a049ca11b`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:14:12,266::task::495::TaskManager.Task::(_debug) Task f9b25438-24ff-46ba-af76-e254f269330e: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:14:12,267::task::495::TaskManager.Task::(_debug) Task f9b25438-24ff-46ba-af76-e254f269330e: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:14:12,268::task::495::TaskManager.Task::(_debug) Task f9b25438-24ff-46ba-af76-e254f269330e: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 04:14:12,269::task::495::TaskManager.Task::(_debug) Task f9b25438-24ff-46ba-af76-e254f269330e: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:14:12,269::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:14:12,270::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:14:12,271::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:14:12,271::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:14:12,272::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:14:12,272::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:14:12,273::task::495::TaskManager.Task::(_debug) Task f9b25438-24ff-46ba-af76-e254f269330e: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:14:12,274::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:14:12,275::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:14:12,275::task::495::TaskManager.Task::(_debug) Task a0af7bff-52fa-4415-8019-7e28ac6958d7: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:14:12,276::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2facb68a-8ceb-4a3a-840c-d4912ef1511d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:14:12,277::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:14:12,278::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:14:12,278::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2facb68a-8ceb-4a3a-840c-d4912ef1511d`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:14:12,279::task::495::TaskManager.Task::(_debug) Task a0af7bff-52fa-4415-8019-7e28ac6958d7: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:14:12,280::task::495::TaskManager.Task::(_debug) Task a0af7bff-52fa-4415-8019-7e28ac6958d7: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:14:12,280::task::495::TaskManager.Task::(_debug) Task a0af7bff-52fa-4415-8019-7e28ac6958d7: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:14:12,281::task::495::TaskManager.Task::(_debug) Task a0af7bff-52fa-4415-8019-7e28ac6958d7: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:14:12,282::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:14:12,282::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:14:12,283::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:14:12,284::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:14:12,284::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:14:12,285::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:14:12,285::task::495::TaskManager.Task::(_debug) Task a0af7bff-52fa-4415-8019-7e28ac6958d7: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:14:12,286::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:14:12,287::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:14:12,287::task::495::TaskManager.Task::(_debug) Task d5d9fa2c-ae30-4057-88df-af23e5031243: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:14:12,288::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`636bfa21-6f2c-4f43-a37d-028c06da1906`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:14:12,289::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:14:12,289::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:14:12,290::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`636bfa21-6f2c-4f43-a37d-028c06da1906`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:14:12,291::task::495::TaskManager.Task::(_debug) Task d5d9fa2c-ae30-4057-88df-af23e5031243: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:14:12,291::task::495::TaskManager.Task::(_debug) Task d5d9fa2c-ae30-4057-88df-af23e5031243: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:14:12,292::task::495::TaskManager.Task::(_debug) Task d5d9fa2c-ae30-4057-88df-af23e5031243: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:14:12,292::task::495::TaskManager.Task::(_debug) Task d5d9fa2c-ae30-4057-88df-af23e5031243: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:14:12,293::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:14:12,293::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:14:12,294::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:14:12,295::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:14:12,295::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:14:12,296::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:14:12,296::task::495::TaskManager.Task::(_debug) Task d5d9fa2c-ae30-4057-88df-af23e5031243: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:14:12,297::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:14:12,298::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:14:12,298::task::495::TaskManager.Task::(_debug) Task 1f369c0d-608d-444f-8cb8-b01f11f84fd9: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:14:12,299::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`3bd62da0-565c-4d2d-a438-4682bbdd9fa9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:14:12,300::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:14:12,301::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:14:12,302::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`3bd62da0-565c-4d2d-a438-4682bbdd9fa9`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:14:12,303::task::495::TaskManager.Task::(_debug) Task 1f369c0d-608d-444f-8cb8-b01f11f84fd9: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:14:12,303::task::495::TaskManager.Task::(_debug) Task 1f369c0d-608d-444f-8cb8-b01f11f84fd9: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:14:12,304::task::495::TaskManager.Task::(_debug) Task 1f369c0d-608d-444f-8cb8-b01f11f84fd9: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:14:12,305::task::495::TaskManager.Task::(_debug) Task 1f369c0d-608d-444f-8cb8-b01f11f84fd9: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:14:12,305::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:14:12,306::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:14:12,307::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:14:12,307::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:14:12,308::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:14:12,308::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:14:12,309::task::495::TaskManager.Task::(_debug) Task 1f369c0d-608d-444f-8cb8-b01f11f84fd9: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:14:12,310::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:14:14,007::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:14:14,583::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0843418 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15915990::INFO::2013-11-20 04:14:16,390::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15915990::DEBUG::2013-11-20 04:14:16,391::task::495::TaskManager.Task::(_debug) Task a78c507e-bc43-4cae-a6bf-8ec4770dc702: moving from state init -> state preparing
>Thread-15915990::DEBUG::2013-11-20 04:14:16,392::task::495::TaskManager.Task::(_debug) Task a78c507e-bc43-4cae-a6bf-8ec4770dc702: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000626087188721', 'lastCheck': 1384920849.1634591, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00167798995972', 'lastCheck': 1384920848.6602371, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00156807899475', 'lastCheck': 1384920855.2410121, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000468015670776', 'lastCheck': 1384920853.130815, 'valid': True, 'code': 0}}
>Thread-15915990::DEBUG::2013-11-20 04:14:16,393::task::495::TaskManager.Task::(_debug) Task a78c507e-bc43-4cae-a6bf-8ec4770dc702: moving from state preparing -> state finished
>Thread-15915990::DEBUG::2013-11-20 04:14:16,394::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915990::DEBUG::2013-11-20 04:14:16,394::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915990::DEBUG::2013-11-20 04:14:16,395::task::495::TaskManager.Task::(_debug) Task a78c507e-bc43-4cae-a6bf-8ec4770dc702: ref 0 aborting False
>Thread-15915990::INFO::2013-11-20 04:14:16,396::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000626087188721', 'lastCheck': 1384920849.1634591, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00167798995972', 'lastCheck': 1384920848.6602371, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00156807899475', 'lastCheck': 1384920855.2410121, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000468015670776', 'lastCheck': 1384920853.130815, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:14:16,598::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:14:17,149::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0763154 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15915992::DEBUG::2013-11-20 04:14:17,801::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915992::INFO::2013-11-20 04:14:17,802::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915992::DEBUG::2013-11-20 04:14:17,803::task::495::TaskManager.Task::(_debug) Task e6ef302a-a79f-4d1b-9f10-44bb5d3694b2: moving from state init -> state preparing
>Thread-15915992::DEBUG::2013-11-20 04:14:17,804::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915992::DEBUG::2013-11-20 04:14:17,804::task::495::TaskManager.Task::(_debug) Task e6ef302a-a79f-4d1b-9f10-44bb5d3694b2: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915992::DEBUG::2013-11-20 04:14:17,805::task::495::TaskManager.Task::(_debug) Task e6ef302a-a79f-4d1b-9f10-44bb5d3694b2: moving from state preparing -> state finished
>Thread-15915992::DEBUG::2013-11-20 04:14:17,806::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915992::DEBUG::2013-11-20 04:14:17,806::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915992::DEBUG::2013-11-20 04:14:17,807::task::495::TaskManager.Task::(_debug) Task e6ef302a-a79f-4d1b-9f10-44bb5d3694b2: ref 0 aborting False
>Thread-15915992::INFO::2013-11-20 04:14:17,808::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915993::DEBUG::2013-11-20 04:14:17,820::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915993::INFO::2013-11-20 04:14:17,821::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915993::DEBUG::2013-11-20 04:14:17,822::task::495::TaskManager.Task::(_debug) Task bfb2f4a1-bdf6-4ba0-99a9-67293f349237: moving from state init -> state preparing
>Thread-15915993::DEBUG::2013-11-20 04:14:17,824::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`64f1d193-6d75-4be5-9f5e-b5140faf029f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915993::DEBUG::2013-11-20 04:14:17,824::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915993::DEBUG::2013-11-20 04:14:17,825::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915993::DEBUG::2013-11-20 04:14:17,826::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`64f1d193-6d75-4be5-9f5e-b5140faf029f`::Granted request
>Thread-15915993::DEBUG::2013-11-20 04:14:17,827::task::495::TaskManager.Task::(_debug) Task bfb2f4a1-bdf6-4ba0-99a9-67293f349237: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915993::DEBUG::2013-11-20 04:14:17,827::task::495::TaskManager.Task::(_debug) Task bfb2f4a1-bdf6-4ba0-99a9-67293f349237: ref 1 aborting False
>Thread-15915993::WARNING::2013-11-20 04:14:17,830::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915993::DEBUG::2013-11-20 04:14:17,831::task::495::TaskManager.Task::(_debug) Task bfb2f4a1-bdf6-4ba0-99a9-67293f349237: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915993::DEBUG::2013-11-20 04:14:17,838::task::495::TaskManager.Task::(_debug) Task bfb2f4a1-bdf6-4ba0-99a9-67293f349237: moving from state preparing -> state finished
>Thread-15915993::DEBUG::2013-11-20 04:14:17,839::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915993::DEBUG::2013-11-20 04:14:17,840::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915993::DEBUG::2013-11-20 04:14:17,840::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915993::DEBUG::2013-11-20 04:14:17,841::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915993::DEBUG::2013-11-20 04:14:17,842::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915993::DEBUG::2013-11-20 04:14:17,843::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915993::DEBUG::2013-11-20 04:14:17,844::task::495::TaskManager.Task::(_debug) Task bfb2f4a1-bdf6-4ba0-99a9-67293f349237: ref 0 aborting False
>Thread-15915993::INFO::2013-11-20 04:14:17,845::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:14:19,171::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:14:19,721::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0839435 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:14:21,738::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:14:22,301::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0901098 s, 11.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:14:24,318::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:14:24,885::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0856571 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:14:26,901::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:14:27,182::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:14:27,184::task::495::TaskManager.Task::(_debug) Task b60e4356-448a-4842-9b16-fb9016cd5b77: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:14:27,187::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ce029f2f-5bd5-431b-8cc8-ac5c13ec4a9e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:14:27,188::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:14:27,190::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:14:27,191::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ce029f2f-5bd5-431b-8cc8-ac5c13ec4a9e`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:14:27,192::task::495::TaskManager.Task::(_debug) Task b60e4356-448a-4842-9b16-fb9016cd5b77: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:14:27,194::task::495::TaskManager.Task::(_debug) Task b60e4356-448a-4842-9b16-fb9016cd5b77: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:14:27,195::task::495::TaskManager.Task::(_debug) Task b60e4356-448a-4842-9b16-fb9016cd5b77: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:14:27,196::task::495::TaskManager.Task::(_debug) Task b60e4356-448a-4842-9b16-fb9016cd5b77: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:14:27,197::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:14:27,198::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:14:27,200::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:14:27,201::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:14:27,201::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:14:27,202::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:14:27,203::task::495::TaskManager.Task::(_debug) Task b60e4356-448a-4842-9b16-fb9016cd5b77: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:14:27,204::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:14:27,477::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.084 s, 12.2 MB/s\n'; <rc> = 0
>Thread-15915997::DEBUG::2013-11-20 04:14:28,342::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915997::INFO::2013-11-20 04:14:28,344::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915997::DEBUG::2013-11-20 04:14:28,345::task::495::TaskManager.Task::(_debug) Task dfc421a6-3e48-4258-a4e4-6a02abc4b0ee: moving from state init -> state preparing
>Thread-15915997::DEBUG::2013-11-20 04:14:28,346::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15915997::DEBUG::2013-11-20 04:14:28,346::task::495::TaskManager.Task::(_debug) Task dfc421a6-3e48-4258-a4e4-6a02abc4b0ee: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915997::DEBUG::2013-11-20 04:14:28,347::task::495::TaskManager.Task::(_debug) Task dfc421a6-3e48-4258-a4e4-6a02abc4b0ee: moving from state preparing -> state finished
>Thread-15915997::DEBUG::2013-11-20 04:14:28,347::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915997::DEBUG::2013-11-20 04:14:28,348::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915997::DEBUG::2013-11-20 04:14:28,349::task::495::TaskManager.Task::(_debug) Task dfc421a6-3e48-4258-a4e4-6a02abc4b0ee: ref 0 aborting False
>Thread-15915997::INFO::2013-11-20 04:14:28,349::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15915998::DEBUG::2013-11-20 04:14:28,360::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15915998::INFO::2013-11-20 04:14:28,361::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15915998::DEBUG::2013-11-20 04:14:28,361::task::495::TaskManager.Task::(_debug) Task 24d850db-cdb6-4562-aa26-717eb6c82279: moving from state init -> state preparing
>Thread-15915998::DEBUG::2013-11-20 04:14:28,363::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f52333fc-dfc7-4bbe-9f8e-08565a74303a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15915998::DEBUG::2013-11-20 04:14:28,363::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15915998::DEBUG::2013-11-20 04:14:28,364::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15915998::DEBUG::2013-11-20 04:14:28,365::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f52333fc-dfc7-4bbe-9f8e-08565a74303a`::Granted request
>Thread-15915998::DEBUG::2013-11-20 04:14:28,365::task::495::TaskManager.Task::(_debug) Task 24d850db-cdb6-4562-aa26-717eb6c82279: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15915998::DEBUG::2013-11-20 04:14:28,366::task::495::TaskManager.Task::(_debug) Task 24d850db-cdb6-4562-aa26-717eb6c82279: ref 1 aborting False
>Thread-15915998::WARNING::2013-11-20 04:14:28,368::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15915998::DEBUG::2013-11-20 04:14:28,369::task::495::TaskManager.Task::(_debug) Task 24d850db-cdb6-4562-aa26-717eb6c82279: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915998::DEBUG::2013-11-20 04:14:28,370::task::495::TaskManager.Task::(_debug) Task 24d850db-cdb6-4562-aa26-717eb6c82279: moving from state preparing -> state finished
>Thread-15915998::DEBUG::2013-11-20 04:14:28,370::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15915998::DEBUG::2013-11-20 04:14:28,371::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915998::DEBUG::2013-11-20 04:14:28,372::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15915998::DEBUG::2013-11-20 04:14:28,372::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15915998::DEBUG::2013-11-20 04:14:28,373::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15915998::DEBUG::2013-11-20 04:14:28,373::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15915998::DEBUG::2013-11-20 04:14:28,374::task::495::TaskManager.Task::(_debug) Task 24d850db-cdb6-4562-aa26-717eb6c82279: ref 0 aborting False
>Thread-15915998::INFO::2013-11-20 04:14:28,375::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:14:29,493::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915918::INFO::2013-11-20 04:14:29,724::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:14:29,728::task::495::TaskManager.Task::(_debug) Task 5ad436c1-3abf-41cf-aea3-9e3160fcc1a9: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:14:29,734::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ac875d83-3e00-4f7e-b70c-62b36237b3a8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:14:29,735::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:14:29,736::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:14:29,737::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ac875d83-3e00-4f7e-b70c-62b36237b3a8`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:14:29,739::task::495::TaskManager.Task::(_debug) Task 5ad436c1-3abf-41cf-aea3-9e3160fcc1a9: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:14:29,739::task::495::TaskManager.Task::(_debug) Task 5ad436c1-3abf-41cf-aea3-9e3160fcc1a9: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:14:29,740::task::495::TaskManager.Task::(_debug) Task 5ad436c1-3abf-41cf-aea3-9e3160fcc1a9: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:14:29,741::task::495::TaskManager.Task::(_debug) Task 5ad436c1-3abf-41cf-aea3-9e3160fcc1a9: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:14:29,742::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:14:29,743::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:14:29,743::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:14:29,744::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:14:29,744::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:14:29,745::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:14:29,746::task::495::TaskManager.Task::(_debug) Task 5ad436c1-3abf-41cf-aea3-9e3160fcc1a9: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:14:29,747::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:14:30,074::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0822057 s, 12.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:14:32,085::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916000::INFO::2013-11-20 04:14:32,450::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916000::DEBUG::2013-11-20 04:14:32,451::task::495::TaskManager.Task::(_debug) Task 1a78fe90-996e-47de-a0dd-5cabea1d3678: moving from state init -> state preparing
>Thread-15916000::DEBUG::2013-11-20 04:14:32,453::task::495::TaskManager.Task::(_debug) Task 1a78fe90-996e-47de-a0dd-5cabea1d3678: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00204300880432', 'lastCheck': 1384920869.734715, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00138115882874', 'lastCheck': 1384920869.1135211, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0014328956604', 'lastCheck': 1384920865.5324659, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000530958175659', 'lastCheck': 1384920863.3838251, 'valid': True, 'code': 0}}
>Thread-15916000::DEBUG::2013-11-20 04:14:32,454::task::495::TaskManager.Task::(_debug) Task 1a78fe90-996e-47de-a0dd-5cabea1d3678: moving from state preparing -> state finished
>Thread-15916000::DEBUG::2013-11-20 04:14:32,455::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916000::DEBUG::2013-11-20 04:14:32,455::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916000::DEBUG::2013-11-20 04:14:32,456::task::495::TaskManager.Task::(_debug) Task 1a78fe90-996e-47de-a0dd-5cabea1d3678: ref 0 aborting False
>Thread-15916000::INFO::2013-11-20 04:14:32,457::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00204300880432', 'lastCheck': 1384920869.734715, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00138115882874', 'lastCheck': 1384920869.1135211, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0014328956604', 'lastCheck': 1384920865.5324659, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000530958175659', 'lastCheck': 1384920863.3838251, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:14:32,690::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0825272 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:14:34,703::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915863::INFO::2013-11-20 04:14:34,918::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:14:34,923::task::495::TaskManager.Task::(_debug) Task 3fd153f8-943e-46bf-b593-98ea3ce6dfed: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:14:34,927::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f1b710bf-2fde-4c80-961f-194c91f4024a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:14:34,928::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:14:34,930::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:14:34,931::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f1b710bf-2fde-4c80-961f-194c91f4024a`::Granted request
>Thread-15915864::DEBUG::2013-11-20 04:14:34,933::guestIF::97::vm.Vm::(_handleMessage) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::u'None'
>Thread-15915863::DEBUG::2013-11-20 04:14:34,935::task::495::TaskManager.Task::(_debug) Task 3fd153f8-943e-46bf-b593-98ea3ce6dfed: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:14:34,936::task::495::TaskManager.Task::(_debug) Task 3fd153f8-943e-46bf-b593-98ea3ce6dfed: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:14:34,937::task::495::TaskManager.Task::(_debug) Task 3fd153f8-943e-46bf-b593-98ea3ce6dfed: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:14:34,937::task::495::TaskManager.Task::(_debug) Task 3fd153f8-943e-46bf-b593-98ea3ce6dfed: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:14:34,938::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:14:34,938::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:14:34,939::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:14:34,939::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:14:34,940::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:14:34,940::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:14:34,941::task::495::TaskManager.Task::(_debug) Task 3fd153f8-943e-46bf-b593-98ea3ce6dfed: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:14:34,941::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:14:35,255::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0763357 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:14:37,271::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:14:37,826::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0757857 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15916004::DEBUG::2013-11-20 04:14:39,240::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916004::INFO::2013-11-20 04:14:39,242::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916004::DEBUG::2013-11-20 04:14:39,242::task::495::TaskManager.Task::(_debug) Task b2fe3a29-803d-4987-87c7-b085ad205467: moving from state init -> state preparing
>Thread-15916004::DEBUG::2013-11-20 04:14:39,244::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916004::DEBUG::2013-11-20 04:14:39,245::task::495::TaskManager.Task::(_debug) Task b2fe3a29-803d-4987-87c7-b085ad205467: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916004::DEBUG::2013-11-20 04:14:39,246::task::495::TaskManager.Task::(_debug) Task b2fe3a29-803d-4987-87c7-b085ad205467: moving from state preparing -> state finished
>Thread-15916004::DEBUG::2013-11-20 04:14:39,246::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916004::DEBUG::2013-11-20 04:14:39,247::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916004::DEBUG::2013-11-20 04:14:39,248::task::495::TaskManager.Task::(_debug) Task b2fe3a29-803d-4987-87c7-b085ad205467: ref 0 aborting False
>Thread-15916004::INFO::2013-11-20 04:14:39,248::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916005::DEBUG::2013-11-20 04:14:39,253::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916005::INFO::2013-11-20 04:14:39,256::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916005::DEBUG::2013-11-20 04:14:39,257::task::495::TaskManager.Task::(_debug) Task d8608922-96d4-4afd-a7ea-121d3aa10485: moving from state init -> state preparing
>Thread-15916005::DEBUG::2013-11-20 04:14:39,259::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b2befd69-3711-4387-97bb-cb5e2b33ec4a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916005::DEBUG::2013-11-20 04:14:39,259::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916005::DEBUG::2013-11-20 04:14:39,260::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916005::DEBUG::2013-11-20 04:14:39,261::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b2befd69-3711-4387-97bb-cb5e2b33ec4a`::Granted request
>Thread-15916005::DEBUG::2013-11-20 04:14:39,262::task::495::TaskManager.Task::(_debug) Task d8608922-96d4-4afd-a7ea-121d3aa10485: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916005::DEBUG::2013-11-20 04:14:39,263::task::495::TaskManager.Task::(_debug) Task d8608922-96d4-4afd-a7ea-121d3aa10485: ref 1 aborting False
>Thread-15913518::INFO::2013-11-20 04:14:39,267::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15916005::WARNING::2013-11-20 04:14:39,268::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913518::DEBUG::2013-11-20 04:14:39,269::task::495::TaskManager.Task::(_debug) Task a24f4493-19d0-47f8-8921-ff5ac442e7db: moving from state init -> state preparing
>Thread-15916005::DEBUG::2013-11-20 04:14:39,270::task::495::TaskManager.Task::(_debug) Task d8608922-96d4-4afd-a7ea-121d3aa10485: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913518::DEBUG::2013-11-20 04:14:39,272::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5874f717-7a6b-4583-b9a5-a7a65181f52c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15916005::DEBUG::2013-11-20 04:14:39,273::task::495::TaskManager.Task::(_debug) Task d8608922-96d4-4afd-a7ea-121d3aa10485: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:14:39,274::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15916005::DEBUG::2013-11-20 04:14:39,275::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:14:39,276::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15916005::DEBUG::2013-11-20 04:14:39,276::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:14:39,277::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5874f717-7a6b-4583-b9a5-a7a65181f52c`::Granted request
>Thread-15916005::DEBUG::2013-11-20 04:14:39,278::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916005::DEBUG::2013-11-20 04:14:39,280::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:14:39,280::task::495::TaskManager.Task::(_debug) Task a24f4493-19d0-47f8-8921-ff5ac442e7db: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15916005::DEBUG::2013-11-20 04:14:39,281::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:14:39,282::task::495::TaskManager.Task::(_debug) Task a24f4493-19d0-47f8-8921-ff5ac442e7db: ref 1 aborting False
>Thread-15916005::DEBUG::2013-11-20 04:14:39,282::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:14:39,283::task::495::TaskManager.Task::(_debug) Task a24f4493-19d0-47f8-8921-ff5ac442e7db: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15916005::DEBUG::2013-11-20 04:14:39,284::task::495::TaskManager.Task::(_debug) Task d8608922-96d4-4afd-a7ea-121d3aa10485: ref 0 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:14:39,285::task::495::TaskManager.Task::(_debug) Task a24f4493-19d0-47f8-8921-ff5ac442e7db: moving from state preparing -> state finished
>Thread-15916005::INFO::2013-11-20 04:14:39,286::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913518::DEBUG::2013-11-20 04:14:39,287::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:14:39,296::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:14:39,297::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:14:39,298::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:14:39,299::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:14:39,300::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:14:39,300::task::495::TaskManager.Task::(_debug) Task a24f4493-19d0-47f8-8921-ff5ac442e7db: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:14:39,301::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:14:39,302::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:14:39,303::task::495::TaskManager.Task::(_debug) Task f71e8106-a6f6-4c63-af7d-0ed97b791c6c: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:14:39,304::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7fc33eb2-6d70-496a-814a-ab803852456a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:14:39,305::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:14:39,306::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:14:39,307::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7fc33eb2-6d70-496a-814a-ab803852456a`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:14:39,308::task::495::TaskManager.Task::(_debug) Task f71e8106-a6f6-4c63-af7d-0ed97b791c6c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:14:39,308::task::495::TaskManager.Task::(_debug) Task f71e8106-a6f6-4c63-af7d-0ed97b791c6c: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:14:39,309::task::495::TaskManager.Task::(_debug) Task f71e8106-a6f6-4c63-af7d-0ed97b791c6c: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:14:39,310::task::495::TaskManager.Task::(_debug) Task f71e8106-a6f6-4c63-af7d-0ed97b791c6c: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:14:39,310::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:14:39,311::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:14:39,312::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:14:39,312::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:14:39,313::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:14:39,314::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:14:39,314::task::495::TaskManager.Task::(_debug) Task f71e8106-a6f6-4c63-af7d-0ed97b791c6c: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:14:39,315::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:14:39,841::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:14:40,421::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0805385 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:14:42,435::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 04:14:42,659::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:14:42,674::task::495::TaskManager.Task::(_debug) Task 4d82af15-3811-4cef-9692-b48bb8c2c3aa: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:14:42,689::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d235a001-5e2e-4643-b936-fce6fb96a830`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:14:42,690::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:14:42,692::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:14:42,693::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d235a001-5e2e-4643-b936-fce6fb96a830`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:14:42,695::task::495::TaskManager.Task::(_debug) Task 4d82af15-3811-4cef-9692-b48bb8c2c3aa: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:14:42,696::task::495::TaskManager.Task::(_debug) Task 4d82af15-3811-4cef-9692-b48bb8c2c3aa: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:14:42,696::task::495::TaskManager.Task::(_debug) Task 4d82af15-3811-4cef-9692-b48bb8c2c3aa: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:14:42,697::task::495::TaskManager.Task::(_debug) Task 4d82af15-3811-4cef-9692-b48bb8c2c3aa: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:14:42,698::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:14:42,698::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:14:42,699::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:14:42,700::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:14:42,700::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:14:42,701::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:14:42,701::task::495::TaskManager.Task::(_debug) Task 4d82af15-3811-4cef-9692-b48bb8c2c3aa: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:14:42,702::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:14:42,990::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0908411 s, 11.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:14:45,010::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:14:45,583::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0918852 s, 11.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:14:47,593::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:14:48,181::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.086622 s, 11.8 MB/s\n'; <rc> = 0
>Thread-15916008::INFO::2013-11-20 04:14:48,296::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916008::DEBUG::2013-11-20 04:14:48,297::task::495::TaskManager.Task::(_debug) Task 63d4855e-7820-4739-a450-6bde32f0606d: moving from state init -> state preparing
>Thread-15916008::DEBUG::2013-11-20 04:14:48,298::task::495::TaskManager.Task::(_debug) Task 63d4855e-7820-4739-a450-6bde32f0606d: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000398874282837', 'lastCheck': 1384920880.1044669, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0247299671173', 'lastCheck': 1384920879.519485, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0110020637512', 'lastCheck': 1384920886.2451849, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000415086746216', 'lastCheck': 1384920884.068476, 'valid': True, 'code': 0}}
>Thread-15916008::DEBUG::2013-11-20 04:14:48,299::task::495::TaskManager.Task::(_debug) Task 63d4855e-7820-4739-a450-6bde32f0606d: moving from state preparing -> state finished
>Thread-15916008::DEBUG::2013-11-20 04:14:48,300::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916008::DEBUG::2013-11-20 04:14:48,300::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916008::DEBUG::2013-11-20 04:14:48,301::task::495::TaskManager.Task::(_debug) Task 63d4855e-7820-4739-a450-6bde32f0606d: ref 0 aborting False
>Thread-15916008::INFO::2013-11-20 04:14:48,302::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000398874282837', 'lastCheck': 1384920880.1044669, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0247299671173', 'lastCheck': 1384920879.519485, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0110020637512', 'lastCheck': 1384920886.2451849, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000415086746216', 'lastCheck': 1384920884.068476, 'valid': True, 'code': 0}}
>Thread-15916010::DEBUG::2013-11-20 04:14:49,565::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916010::INFO::2013-11-20 04:14:49,566::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916010::DEBUG::2013-11-20 04:14:49,567::task::495::TaskManager.Task::(_debug) Task 189cdabc-fd20-483a-865f-01342bdd8c0c: moving from state init -> state preparing
>Thread-15916010::DEBUG::2013-11-20 04:14:49,568::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916010::DEBUG::2013-11-20 04:14:49,568::task::495::TaskManager.Task::(_debug) Task 189cdabc-fd20-483a-865f-01342bdd8c0c: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916010::DEBUG::2013-11-20 04:14:49,569::task::495::TaskManager.Task::(_debug) Task 189cdabc-fd20-483a-865f-01342bdd8c0c: moving from state preparing -> state finished
>Thread-15916010::DEBUG::2013-11-20 04:14:49,570::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916010::DEBUG::2013-11-20 04:14:49,571::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916010::DEBUG::2013-11-20 04:14:49,572::task::495::TaskManager.Task::(_debug) Task 189cdabc-fd20-483a-865f-01342bdd8c0c: ref 0 aborting False
>Thread-15916010::INFO::2013-11-20 04:14:49,573::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916011::DEBUG::2013-11-20 04:14:49,584::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916011::INFO::2013-11-20 04:14:49,586::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916011::DEBUG::2013-11-20 04:14:49,587::task::495::TaskManager.Task::(_debug) Task d26e311d-98ea-4448-89a9-b8384877e8b1: moving from state init -> state preparing
>Thread-15916011::DEBUG::2013-11-20 04:14:49,589::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5d5b549a-db7c-4d7d-aa91-34828de33487`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916011::DEBUG::2013-11-20 04:14:49,590::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916011::DEBUG::2013-11-20 04:14:49,591::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916011::DEBUG::2013-11-20 04:14:49,592::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5d5b549a-db7c-4d7d-aa91-34828de33487`::Granted request
>Thread-15916011::DEBUG::2013-11-20 04:14:49,593::task::495::TaskManager.Task::(_debug) Task d26e311d-98ea-4448-89a9-b8384877e8b1: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916011::DEBUG::2013-11-20 04:14:49,594::task::495::TaskManager.Task::(_debug) Task d26e311d-98ea-4448-89a9-b8384877e8b1: ref 1 aborting False
>Thread-15916011::WARNING::2013-11-20 04:14:49,596::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916011::DEBUG::2013-11-20 04:14:49,597::task::495::TaskManager.Task::(_debug) Task d26e311d-98ea-4448-89a9-b8384877e8b1: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916011::DEBUG::2013-11-20 04:14:49,598::task::495::TaskManager.Task::(_debug) Task d26e311d-98ea-4448-89a9-b8384877e8b1: moving from state preparing -> state finished
>Thread-15916011::DEBUG::2013-11-20 04:14:49,599::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916011::DEBUG::2013-11-20 04:14:49,599::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916011::DEBUG::2013-11-20 04:14:49,600::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916011::DEBUG::2013-11-20 04:14:49,601::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916011::DEBUG::2013-11-20 04:14:49,601::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916011::DEBUG::2013-11-20 04:14:49,602::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916011::DEBUG::2013-11-20 04:14:49,603::task::495::TaskManager.Task::(_debug) Task d26e311d-98ea-4448-89a9-b8384877e8b1: ref 0 aborting False
>Thread-15916011::INFO::2013-11-20 04:14:49,603::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:14:50,195::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:14:50,757::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0773057 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:14:52,772::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 04:14:53,016::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:14:53,025::task::495::TaskManager.Task::(_debug) Task 27822cb6-2673-4de5-a9fc-5cfef56675fb: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:14:53,032::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3d6f090a-67b1-4450-be76-ad56d4d606fb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:14:53,033::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:14:53,034::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:14:53,034::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3d6f090a-67b1-4450-be76-ad56d4d606fb`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:14:53,035::task::495::TaskManager.Task::(_debug) Task 27822cb6-2673-4de5-a9fc-5cfef56675fb: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:14:53,035::task::495::TaskManager.Task::(_debug) Task 27822cb6-2673-4de5-a9fc-5cfef56675fb: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:14:53,036::task::495::TaskManager.Task::(_debug) Task 27822cb6-2673-4de5-a9fc-5cfef56675fb: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:14:53,037::task::495::TaskManager.Task::(_debug) Task 27822cb6-2673-4de5-a9fc-5cfef56675fb: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:14:53,037::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:14:53,038::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:14:53,038::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:14:53,039::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:14:53,039::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:14:53,040::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:14:53,040::task::495::TaskManager.Task::(_debug) Task 27822cb6-2673-4de5-a9fc-5cfef56675fb: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:14:53,041::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:14:53,337::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0869108 s, 11.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:14:55,351::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:14:55,921::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0871933 s, 11.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:14:57,937::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 04:14:58,169::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:14:58,171::task::495::TaskManager.Task::(_debug) Task 52e29dd2-4c04-4a59-ba7c-76967daa7039: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:14:58,175::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0e1e42f9-deae-452c-96b8-a515bfb33ad6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:14:58,176::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:14:58,178::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:14:58,179::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0e1e42f9-deae-452c-96b8-a515bfb33ad6`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:14:58,180::task::495::TaskManager.Task::(_debug) Task 52e29dd2-4c04-4a59-ba7c-76967daa7039: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:14:58,181::task::495::TaskManager.Task::(_debug) Task 52e29dd2-4c04-4a59-ba7c-76967daa7039: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:14:58,181::task::495::TaskManager.Task::(_debug) Task 52e29dd2-4c04-4a59-ba7c-76967daa7039: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:14:58,182::task::495::TaskManager.Task::(_debug) Task 52e29dd2-4c04-4a59-ba7c-76967daa7039: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:14:58,183::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:14:58,183::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:14:58,184::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:14:58,185::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:14:58,185::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:14:58,186::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:14:58,186::task::495::TaskManager.Task::(_debug) Task 52e29dd2-4c04-4a59-ba7c-76967daa7039: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:14:58,187::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:14:58,472::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0859709 s, 11.9 MB/s\n'; <rc> = 0
>Thread-15916015::DEBUG::2013-11-20 04:15:00,068::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916015::INFO::2013-11-20 04:15:00,069::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916015::DEBUG::2013-11-20 04:15:00,070::task::495::TaskManager.Task::(_debug) Task d91126dd-e626-41c4-be91-0895b6f6efb9: moving from state init -> state preparing
>Thread-15916015::DEBUG::2013-11-20 04:15:00,071::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916015::DEBUG::2013-11-20 04:15:00,072::task::495::TaskManager.Task::(_debug) Task d91126dd-e626-41c4-be91-0895b6f6efb9: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916015::DEBUG::2013-11-20 04:15:00,072::task::495::TaskManager.Task::(_debug) Task d91126dd-e626-41c4-be91-0895b6f6efb9: moving from state preparing -> state finished
>Thread-15916015::DEBUG::2013-11-20 04:15:00,073::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916015::DEBUG::2013-11-20 04:15:00,074::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916015::DEBUG::2013-11-20 04:15:00,074::task::495::TaskManager.Task::(_debug) Task d91126dd-e626-41c4-be91-0895b6f6efb9: ref 0 aborting False
>Thread-15916015::INFO::2013-11-20 04:15:00,075::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916016::DEBUG::2013-11-20 04:15:00,087::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916016::INFO::2013-11-20 04:15:00,088::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916016::DEBUG::2013-11-20 04:15:00,089::task::495::TaskManager.Task::(_debug) Task 14dd5da1-b6b1-488b-8804-b83fff979f83: moving from state init -> state preparing
>Thread-15916016::DEBUG::2013-11-20 04:15:00,090::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`74910421-abd0-49e2-a972-90b15579a59e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916016::DEBUG::2013-11-20 04:15:00,090::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916016::DEBUG::2013-11-20 04:15:00,091::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916016::DEBUG::2013-11-20 04:15:00,092::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`74910421-abd0-49e2-a972-90b15579a59e`::Granted request
>Thread-15916016::DEBUG::2013-11-20 04:15:00,093::task::495::TaskManager.Task::(_debug) Task 14dd5da1-b6b1-488b-8804-b83fff979f83: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916016::DEBUG::2013-11-20 04:15:00,094::task::495::TaskManager.Task::(_debug) Task 14dd5da1-b6b1-488b-8804-b83fff979f83: ref 1 aborting False
>Thread-15916016::WARNING::2013-11-20 04:15:00,097::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916016::DEBUG::2013-11-20 04:15:00,098::task::495::TaskManager.Task::(_debug) Task 14dd5da1-b6b1-488b-8804-b83fff979f83: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916016::DEBUG::2013-11-20 04:15:00,100::task::495::TaskManager.Task::(_debug) Task 14dd5da1-b6b1-488b-8804-b83fff979f83: moving from state preparing -> state finished
>Thread-15916016::DEBUG::2013-11-20 04:15:00,100::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916016::DEBUG::2013-11-20 04:15:00,101::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916016::DEBUG::2013-11-20 04:15:00,102::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916016::DEBUG::2013-11-20 04:15:00,103::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916016::DEBUG::2013-11-20 04:15:00,103::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916016::DEBUG::2013-11-20 04:15:00,104::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916016::DEBUG::2013-11-20 04:15:00,104::task::495::TaskManager.Task::(_debug) Task 14dd5da1-b6b1-488b-8804-b83fff979f83: ref 0 aborting False
>Thread-15916016::INFO::2013-11-20 04:15:00,105::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:15:00,487::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:15:01,082::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0968032 s, 10.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:15:03,100::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 04:15:03,384::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:15:03,387::task::495::TaskManager.Task::(_debug) Task fef75dcd-4d82-4907-8015-6ddcda5d1b10: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:15:03,391::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`900451f5-c2d6-4170-8586-5c5aa664c7b1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:15:03,393::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:15:03,394::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:15:03,394::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`900451f5-c2d6-4170-8586-5c5aa664c7b1`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:15:03,396::task::495::TaskManager.Task::(_debug) Task fef75dcd-4d82-4907-8015-6ddcda5d1b10: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:15:03,396::task::495::TaskManager.Task::(_debug) Task fef75dcd-4d82-4907-8015-6ddcda5d1b10: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:15:03,397::task::495::TaskManager.Task::(_debug) Task fef75dcd-4d82-4907-8015-6ddcda5d1b10: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:15:03,398::task::495::TaskManager.Task::(_debug) Task fef75dcd-4d82-4907-8015-6ddcda5d1b10: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:15:03,399::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:15:03,399::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:15:03,400::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:15:03,401::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:15:03,402::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:15:03,402::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:15:03,403::task::495::TaskManager.Task::(_debug) Task fef75dcd-4d82-4907-8015-6ddcda5d1b10: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:15:03,404::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:15:03,707::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0902405 s, 11.3 MB/s\n'; <rc> = 0
>Thread-15916018::INFO::2013-11-20 04:15:03,983::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916018::DEBUG::2013-11-20 04:15:03,984::task::495::TaskManager.Task::(_debug) Task 8ba915ba-4361-4d1d-9ef3-74af6d545a89: moving from state init -> state preparing
>Thread-15916018::DEBUG::2013-11-20 04:15:03,984::task::495::TaskManager.Task::(_debug) Task 8ba915ba-4361-4d1d-9ef3-74af6d545a89: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000616073608398', 'lastCheck': 1384920900.764466, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00150108337402', 'lastCheck': 1384920900.1511481, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00164985656738', 'lastCheck': 1384920896.5545089, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000530004501343', 'lastCheck': 1384920894.4159589, 'valid': True, 'code': 0}}
>Thread-15916018::DEBUG::2013-11-20 04:15:03,985::task::495::TaskManager.Task::(_debug) Task 8ba915ba-4361-4d1d-9ef3-74af6d545a89: moving from state preparing -> state finished
>Thread-15916018::DEBUG::2013-11-20 04:15:03,986::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916018::DEBUG::2013-11-20 04:15:03,987::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916018::DEBUG::2013-11-20 04:15:03,987::task::495::TaskManager.Task::(_debug) Task 8ba915ba-4361-4d1d-9ef3-74af6d545a89: ref 0 aborting False
>Thread-15916018::INFO::2013-11-20 04:15:03,988::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000616073608398', 'lastCheck': 1384920900.764466, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00150108337402', 'lastCheck': 1384920900.1511481, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00164985656738', 'lastCheck': 1384920896.5545089, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000530004501343', 'lastCheck': 1384920894.4159589, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:15:05,723::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:15:06,301::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.076428 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:15:08,319::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:15:08,836::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.074559 s, 13.7 MB/s\n'; <rc> = 0
>libvirtEventLoop::DEBUG::2013-11-20 04:15:09,415::vm::525::vm.Vm::(_rtcUpdate) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::new rtc offset 0
>libvirtEventLoop::DEBUG::2013-11-20 04:15:09,417::vm::525::vm.Vm::(_rtcUpdate) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::new rtc offset 0
>libvirtEventLoop::DEBUG::2013-11-20 04:15:09,418::vm::525::vm.Vm::(_rtcUpdate) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::new rtc offset 3600
>libvirtEventLoop::DEBUG::2013-11-20 04:15:09,420::vm::525::vm.Vm::(_rtcUpdate) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::new rtc offset 3600
>libvirtEventLoop::DEBUG::2013-11-20 04:15:09,421::vm::525::vm.Vm::(_rtcUpdate) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::new rtc offset 3600
>libvirtEventLoop::DEBUG::2013-11-20 04:15:09,422::vm::525::vm.Vm::(_rtcUpdate) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::new rtc offset 3600
>libvirtEventLoop::DEBUG::2013-11-20 04:15:09,424::vm::525::vm.Vm::(_rtcUpdate) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::new rtc offset 3600
>libvirtEventLoop::DEBUG::2013-11-20 04:15:09,425::vm::525::vm.Vm::(_rtcUpdate) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::new rtc offset 3600
>libvirtEventLoop::DEBUG::2013-11-20 04:15:09,426::vm::525::vm.Vm::(_rtcUpdate) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::new rtc offset 3600
>libvirtEventLoop::DEBUG::2013-11-20 04:15:09,427::vm::525::vm.Vm::(_rtcUpdate) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::new rtc offset 3600
>libvirtEventLoop::DEBUG::2013-11-20 04:15:09,428::vm::525::vm.Vm::(_rtcUpdate) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::new rtc offset 3600
>libvirtEventLoop::DEBUG::2013-11-20 04:15:09,429::vm::525::vm.Vm::(_rtcUpdate) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::new rtc offset 3600
>libvirtEventLoop::DEBUG::2013-11-20 04:15:09,430::vm::525::vm.Vm::(_rtcUpdate) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::new rtc offset 3600
>libvirtEventLoop::DEBUG::2013-11-20 04:15:09,431::vm::525::vm.Vm::(_rtcUpdate) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::new rtc offset 3600
>Thread-15916021::DEBUG::2013-11-20 04:15:10,671::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916021::INFO::2013-11-20 04:15:10,673::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916021::DEBUG::2013-11-20 04:15:10,674::task::495::TaskManager.Task::(_debug) Task 757da6e8-3002-44be-99ef-eb8ddaedb42d: moving from state init -> state preparing
>Thread-15916021::DEBUG::2013-11-20 04:15:10,675::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916021::DEBUG::2013-11-20 04:15:10,675::task::495::TaskManager.Task::(_debug) Task 757da6e8-3002-44be-99ef-eb8ddaedb42d: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916021::DEBUG::2013-11-20 04:15:10,676::task::495::TaskManager.Task::(_debug) Task 757da6e8-3002-44be-99ef-eb8ddaedb42d: moving from state preparing -> state finished
>Thread-15916021::DEBUG::2013-11-20 04:15:10,677::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916021::DEBUG::2013-11-20 04:15:10,677::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916021::DEBUG::2013-11-20 04:15:10,678::task::495::TaskManager.Task::(_debug) Task 757da6e8-3002-44be-99ef-eb8ddaedb42d: ref 0 aborting False
>Thread-15916021::INFO::2013-11-20 04:15:10,679::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916022::DEBUG::2013-11-20 04:15:10,684::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916022::INFO::2013-11-20 04:15:10,685::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916022::DEBUG::2013-11-20 04:15:10,686::task::495::TaskManager.Task::(_debug) Task e7685de8-0488-4860-8eb0-5e94d5bfd1b1: moving from state init -> state preparing
>Thread-15916022::DEBUG::2013-11-20 04:15:10,687::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e464226b-eb88-4faf-8140-40ae725a09d3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916022::DEBUG::2013-11-20 04:15:10,688::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916022::DEBUG::2013-11-20 04:15:10,689::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916022::DEBUG::2013-11-20 04:15:10,689::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e464226b-eb88-4faf-8140-40ae725a09d3`::Granted request
>Thread-15916022::DEBUG::2013-11-20 04:15:10,690::task::495::TaskManager.Task::(_debug) Task e7685de8-0488-4860-8eb0-5e94d5bfd1b1: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916022::DEBUG::2013-11-20 04:15:10,691::task::495::TaskManager.Task::(_debug) Task e7685de8-0488-4860-8eb0-5e94d5bfd1b1: ref 1 aborting False
>Thread-15916022::WARNING::2013-11-20 04:15:10,693::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916022::DEBUG::2013-11-20 04:15:10,694::task::495::TaskManager.Task::(_debug) Task e7685de8-0488-4860-8eb0-5e94d5bfd1b1: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916022::DEBUG::2013-11-20 04:15:10,695::task::495::TaskManager.Task::(_debug) Task e7685de8-0488-4860-8eb0-5e94d5bfd1b1: moving from state preparing -> state finished
>Thread-15916022::DEBUG::2013-11-20 04:15:10,696::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916022::DEBUG::2013-11-20 04:15:10,696::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916022::DEBUG::2013-11-20 04:15:10,696::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916022::DEBUG::2013-11-20 04:15:10,697::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916022::DEBUG::2013-11-20 04:15:10,697::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916022::DEBUG::2013-11-20 04:15:10,697::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916022::DEBUG::2013-11-20 04:15:10,698::task::495::TaskManager.Task::(_debug) Task e7685de8-0488-4860-8eb0-5e94d5bfd1b1: ref 0 aborting False
>Thread-15916022::INFO::2013-11-20 04:15:10,698::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:15:10,852::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 04:15:11,116::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:15:11,129::task::495::TaskManager.Task::(_debug) Task 416399d8-f05e-4ba4-bc16-b374edfc5300: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:15:11,133::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4c87f5cd-e3fe-4b27-8bde-7064f2bffcc5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:15:11,134::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:15:11,135::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:15:11,140::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4c87f5cd-e3fe-4b27-8bde-7064f2bffcc5`::Granted request
>Thread-15914435::INFO::2013-11-20 04:15:11,143::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914519::DEBUG::2013-11-20 04:15:11,144::task::495::TaskManager.Task::(_debug) Task 416399d8-f05e-4ba4-bc16-b374edfc5300: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:15:11,144::task::495::TaskManager.Task::(_debug) Task 3185b060-0060-4030-bbbf-d6b3e0fe4ece: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:15:11,145::task::495::TaskManager.Task::(_debug) Task 416399d8-f05e-4ba4-bc16-b374edfc5300: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:15:11,148::task::495::TaskManager.Task::(_debug) Task 416399d8-f05e-4ba4-bc16-b374edfc5300: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914435::DEBUG::2013-11-20 04:15:11,149::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bd40132b-0c5f-4675-8222-3b05b092a8c6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:15:11,149::task::495::TaskManager.Task::(_debug) Task 416399d8-f05e-4ba4-bc16-b374edfc5300: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:15:11,150::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:15:11,151::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:15:11,152::resourceManager::481::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' found in shared state and queue is empty, Joining current shared lock (2 active users)
>Thread-15914519::DEBUG::2013-11-20 04:15:11,153::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:15:11,154::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bd40132b-0c5f-4675-8222-3b05b092a8c6`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:15:11,158::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 04:15:11,160::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (1 active users)
>Thread-15914435::DEBUG::2013-11-20 04:15:11,160::task::495::TaskManager.Task::(_debug) Task 3185b060-0060-4030-bbbf-d6b3e0fe4ece: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 04:15:11,161::task::495::TaskManager.Task::(_debug) Task 416399d8-f05e-4ba4-bc16-b374edfc5300: ref 0 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:15:11,162::task::495::TaskManager.Task::(_debug) Task 3185b060-0060-4030-bbbf-d6b3e0fe4ece: ref 1 aborting False
>Thread-15914519::INFO::2013-11-20 04:15:11,163::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914435::DEBUG::2013-11-20 04:15:11,164::task::495::TaskManager.Task::(_debug) Task 3185b060-0060-4030-bbbf-d6b3e0fe4ece: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:15:11,165::task::495::TaskManager.Task::(_debug) Task 3185b060-0060-4030-bbbf-d6b3e0fe4ece: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:15:11,166::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:15:11,167::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:15:11,168::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:15:11,168::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:15:11,169::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:15:11,170::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:15:11,171::task::495::TaskManager.Task::(_debug) Task 3185b060-0060-4030-bbbf-d6b3e0fe4ece: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:15:11,172::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:15:11,438::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0818694 s, 12.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:15:13,451::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 04:15:13,642::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:15:13,648::task::495::TaskManager.Task::(_debug) Task 1a4632a1-5b8e-45da-b012-f871c9be4d47: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:15:13,651::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`5820bf52-0c73-47c7-afb1-b12804dc7493`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:15:13,655::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:15:13,658::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:15:13,660::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`5820bf52-0c73-47c7-afb1-b12804dc7493`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:15:13,663::task::495::TaskManager.Task::(_debug) Task 1a4632a1-5b8e-45da-b012-f871c9be4d47: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:15:13,664::task::495::TaskManager.Task::(_debug) Task 1a4632a1-5b8e-45da-b012-f871c9be4d47: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:15:13,665::task::495::TaskManager.Task::(_debug) Task 1a4632a1-5b8e-45da-b012-f871c9be4d47: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 04:15:13,666::task::495::TaskManager.Task::(_debug) Task 1a4632a1-5b8e-45da-b012-f871c9be4d47: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:15:13,667::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:15:13,668::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:15:13,670::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:15:13,671::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:15:13,672::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:15:13,673::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:15:13,674::task::495::TaskManager.Task::(_debug) Task 1a4632a1-5b8e-45da-b012-f871c9be4d47: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:15:13,675::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:15:13,676::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:15:13,677::task::495::TaskManager.Task::(_debug) Task de2e5cf8-5cb0-4cf0-ad59-387c82605aea: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:15:13,678::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f423199d-3367-4288-a4fd-96f4e84bccea`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:15:13,679::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:15:13,680::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:15:13,680::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f423199d-3367-4288-a4fd-96f4e84bccea`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:15:13,682::task::495::TaskManager.Task::(_debug) Task de2e5cf8-5cb0-4cf0-ad59-387c82605aea: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:15:13,682::task::495::TaskManager.Task::(_debug) Task de2e5cf8-5cb0-4cf0-ad59-387c82605aea: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:15:13,683::task::495::TaskManager.Task::(_debug) Task de2e5cf8-5cb0-4cf0-ad59-387c82605aea: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:15:13,684::task::495::TaskManager.Task::(_debug) Task de2e5cf8-5cb0-4cf0-ad59-387c82605aea: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:15:13,684::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:15:13,685::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:15:13,686::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:15:13,687::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:15:13,688::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:15:13,689::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:15:13,689::task::495::TaskManager.Task::(_debug) Task de2e5cf8-5cb0-4cf0-ad59-387c82605aea: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:15:13,690::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:15:13,691::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:15:13,691::task::495::TaskManager.Task::(_debug) Task 770ea9ee-ae5e-447e-93a8-f75ce41573d6: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:15:13,693::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`7ed267f9-0473-457a-a39a-b159de993557`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:15:13,693::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:15:13,694::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:15:13,695::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`7ed267f9-0473-457a-a39a-b159de993557`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:15:13,696::task::495::TaskManager.Task::(_debug) Task 770ea9ee-ae5e-447e-93a8-f75ce41573d6: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:15:13,697::task::495::TaskManager.Task::(_debug) Task 770ea9ee-ae5e-447e-93a8-f75ce41573d6: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:15:13,698::task::495::TaskManager.Task::(_debug) Task 770ea9ee-ae5e-447e-93a8-f75ce41573d6: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:15:13,699::task::495::TaskManager.Task::(_debug) Task 770ea9ee-ae5e-447e-93a8-f75ce41573d6: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:15:13,699::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:15:13,700::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:15:13,700::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:15:13,701::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:15:13,702::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:15:13,702::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:15:13,703::task::495::TaskManager.Task::(_debug) Task 770ea9ee-ae5e-447e-93a8-f75ce41573d6: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:15:13,704::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:15:13,705::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:15:13,705::task::495::TaskManager.Task::(_debug) Task cd45d034-6488-4d23-b009-618c0971aa12: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:15:13,706::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`42d3ea57-2424-4a92-a226-57378151127a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:15:13,707::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:15:13,708::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:15:13,708::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`42d3ea57-2424-4a92-a226-57378151127a`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:15:13,709::task::495::TaskManager.Task::(_debug) Task cd45d034-6488-4d23-b009-618c0971aa12: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:15:13,710::task::495::TaskManager.Task::(_debug) Task cd45d034-6488-4d23-b009-618c0971aa12: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:15:13,711::task::495::TaskManager.Task::(_debug) Task cd45d034-6488-4d23-b009-618c0971aa12: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:15:13,711::task::495::TaskManager.Task::(_debug) Task cd45d034-6488-4d23-b009-618c0971aa12: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:15:13,712::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:15:13,712::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:15:13,713::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:15:13,714::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:15:13,714::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:15:13,715::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:15:13,716::task::495::TaskManager.Task::(_debug) Task cd45d034-6488-4d23-b009-618c0971aa12: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:15:13,716::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:15:13,978::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0767917 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:15:15,994::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:15:16,582::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0837899 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:15:18,597::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:15:19,133::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0860185 s, 11.9 MB/s\n'; <rc> = 0
>Thread-15916026::INFO::2013-11-20 04:15:19,893::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916026::DEBUG::2013-11-20 04:15:19,894::task::495::TaskManager.Task::(_debug) Task 212657d1-f89d-4c27-a4b5-d5d055d27cf3: moving from state init -> state preparing
>Thread-15916026::DEBUG::2013-11-20 04:15:19,895::task::495::TaskManager.Task::(_debug) Task 212657d1-f89d-4c27-a4b5-d5d055d27cf3: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000723838806152', 'lastCheck': 1384920911.1317649, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0020740032196', 'lastCheck': 1384920910.516335, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00137901306152', 'lastCheck': 1384920917.2283659, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000458955764771', 'lastCheck': 1384920915.1088419, 'valid': True, 'code': 0}}
>Thread-15916026::DEBUG::2013-11-20 04:15:19,896::task::495::TaskManager.Task::(_debug) Task 212657d1-f89d-4c27-a4b5-d5d055d27cf3: moving from state preparing -> state finished
>Thread-15916026::DEBUG::2013-11-20 04:15:19,897::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916026::DEBUG::2013-11-20 04:15:19,897::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916026::DEBUG::2013-11-20 04:15:19,898::task::495::TaskManager.Task::(_debug) Task 212657d1-f89d-4c27-a4b5-d5d055d27cf3: ref 0 aborting False
>Thread-15916026::INFO::2013-11-20 04:15:19,899::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000723838806152', 'lastCheck': 1384920911.1317649, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0020740032196', 'lastCheck': 1384920910.516335, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00137901306152', 'lastCheck': 1384920917.2283659, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000458955764771', 'lastCheck': 1384920915.1088419, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:15:21,149::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916028::DEBUG::2013-11-20 04:15:21,426::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916028::INFO::2013-11-20 04:15:21,430::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916028::DEBUG::2013-11-20 04:15:21,430::task::495::TaskManager.Task::(_debug) Task f8031b71-88dd-44c0-ad8f-3e977ba1fb80: moving from state init -> state preparing
>Thread-15916028::DEBUG::2013-11-20 04:15:21,431::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916028::DEBUG::2013-11-20 04:15:21,432::task::495::TaskManager.Task::(_debug) Task f8031b71-88dd-44c0-ad8f-3e977ba1fb80: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916028::DEBUG::2013-11-20 04:15:21,437::task::495::TaskManager.Task::(_debug) Task f8031b71-88dd-44c0-ad8f-3e977ba1fb80: moving from state preparing -> state finished
>Thread-15916028::DEBUG::2013-11-20 04:15:21,438::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916028::DEBUG::2013-11-20 04:15:21,439::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916028::DEBUG::2013-11-20 04:15:21,439::task::495::TaskManager.Task::(_debug) Task f8031b71-88dd-44c0-ad8f-3e977ba1fb80: ref 0 aborting False
>Thread-15916028::INFO::2013-11-20 04:15:21,440::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916029::DEBUG::2013-11-20 04:15:21,450::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916029::INFO::2013-11-20 04:15:21,451::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916029::DEBUG::2013-11-20 04:15:21,452::task::495::TaskManager.Task::(_debug) Task 44cd13d7-6c09-4731-baf5-3de6454627e4: moving from state init -> state preparing
>Thread-15916029::DEBUG::2013-11-20 04:15:21,454::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`91e00594-af5f-4952-84a5-7d0dc358d472`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916029::DEBUG::2013-11-20 04:15:21,455::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916029::DEBUG::2013-11-20 04:15:21,456::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916029::DEBUG::2013-11-20 04:15:21,456::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`91e00594-af5f-4952-84a5-7d0dc358d472`::Granted request
>Thread-15916029::DEBUG::2013-11-20 04:15:21,458::task::495::TaskManager.Task::(_debug) Task 44cd13d7-6c09-4731-baf5-3de6454627e4: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916029::DEBUG::2013-11-20 04:15:21,458::task::495::TaskManager.Task::(_debug) Task 44cd13d7-6c09-4731-baf5-3de6454627e4: ref 1 aborting False
>Thread-15916029::WARNING::2013-11-20 04:15:21,461::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916029::DEBUG::2013-11-20 04:15:21,462::task::495::TaskManager.Task::(_debug) Task 44cd13d7-6c09-4731-baf5-3de6454627e4: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916029::DEBUG::2013-11-20 04:15:21,463::task::495::TaskManager.Task::(_debug) Task 44cd13d7-6c09-4731-baf5-3de6454627e4: moving from state preparing -> state finished
>Thread-15916029::DEBUG::2013-11-20 04:15:21,464::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916029::DEBUG::2013-11-20 04:15:21,465::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916029::DEBUG::2013-11-20 04:15:21,465::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916029::DEBUG::2013-11-20 04:15:21,466::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916029::DEBUG::2013-11-20 04:15:21,467::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916029::DEBUG::2013-11-20 04:15:21,468::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916029::DEBUG::2013-11-20 04:15:21,469::task::495::TaskManager.Task::(_debug) Task 44cd13d7-6c09-4731-baf5-3de6454627e4: ref 0 aborting False
>Thread-15916029::INFO::2013-11-20 04:15:21,470::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:15:21,696::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0793034 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:15:23,712::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:15:24,296::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0816165 s, 12.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:15:26,313::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:15:26,838::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0600212 s, 17.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:15:28,851::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:15:29,043::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:15:29,046::task::495::TaskManager.Task::(_debug) Task a544a847-9add-432c-b114-1face04a2a14: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:15:29,050::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`00a94b6c-d41e-4951-89b1-a0404f2b5107`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:15:29,052::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:15:29,055::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:15:29,056::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`00a94b6c-d41e-4951-89b1-a0404f2b5107`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:15:29,057::task::495::TaskManager.Task::(_debug) Task a544a847-9add-432c-b114-1face04a2a14: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:15:29,058::task::495::TaskManager.Task::(_debug) Task a544a847-9add-432c-b114-1face04a2a14: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:15:29,059::task::495::TaskManager.Task::(_debug) Task a544a847-9add-432c-b114-1face04a2a14: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:15:29,060::task::495::TaskManager.Task::(_debug) Task a544a847-9add-432c-b114-1face04a2a14: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:15:29,061::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:15:29,061::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:15:29,062::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:15:29,063::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:15:29,064::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:15:29,064::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:15:29,065::task::495::TaskManager.Task::(_debug) Task a544a847-9add-432c-b114-1face04a2a14: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:15:29,066::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:15:29,405::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0973856 s, 10.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:15:31,422::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915918::INFO::2013-11-20 04:15:31,633::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:15:31,636::task::495::TaskManager.Task::(_debug) Task 820fb7e7-59c2-44e9-870e-93f7923ee682: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:15:31,640::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fa201b14-2012-428d-b86c-9aac61288118`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:15:31,641::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:15:31,642::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:15:31,643::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fa201b14-2012-428d-b86c-9aac61288118`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:15:31,645::task::495::TaskManager.Task::(_debug) Task 820fb7e7-59c2-44e9-870e-93f7923ee682: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:15:31,646::task::495::TaskManager.Task::(_debug) Task 820fb7e7-59c2-44e9-870e-93f7923ee682: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:15:31,647::task::495::TaskManager.Task::(_debug) Task 820fb7e7-59c2-44e9-870e-93f7923ee682: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:15:31,648::task::495::TaskManager.Task::(_debug) Task 820fb7e7-59c2-44e9-870e-93f7923ee682: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:15:31,648::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:15:31,649::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:15:31,650::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:15:31,650::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:15:31,651::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:15:31,651::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:15:31,652::task::495::TaskManager.Task::(_debug) Task 820fb7e7-59c2-44e9-870e-93f7923ee682: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:15:31,653::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15916033::DEBUG::2013-11-20 04:15:31,941::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916033::INFO::2013-11-20 04:15:31,944::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916033::DEBUG::2013-11-20 04:15:31,945::task::495::TaskManager.Task::(_debug) Task 638c586a-10ca-4840-8eb0-3f9311dd2958: moving from state init -> state preparing
>Thread-15916033::DEBUG::2013-11-20 04:15:31,947::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916033::DEBUG::2013-11-20 04:15:31,948::task::495::TaskManager.Task::(_debug) Task 638c586a-10ca-4840-8eb0-3f9311dd2958: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916033::DEBUG::2013-11-20 04:15:31,949::task::495::TaskManager.Task::(_debug) Task 638c586a-10ca-4840-8eb0-3f9311dd2958: moving from state preparing -> state finished
>Thread-15916033::DEBUG::2013-11-20 04:15:31,950::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916033::DEBUG::2013-11-20 04:15:31,951::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916033::DEBUG::2013-11-20 04:15:31,952::task::495::TaskManager.Task::(_debug) Task 638c586a-10ca-4840-8eb0-3f9311dd2958: ref 0 aborting False
>Thread-15916033::INFO::2013-11-20 04:15:31,954::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916034::DEBUG::2013-11-20 04:15:31,962::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916034::INFO::2013-11-20 04:15:31,964::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916034::DEBUG::2013-11-20 04:15:31,965::task::495::TaskManager.Task::(_debug) Task 01f837c8-7a5b-4e37-b4e3-e23878387cee: moving from state init -> state preparing
>Thread-15916034::DEBUG::2013-11-20 04:15:31,968::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`fa17d78b-bc76-4979-8ae3-3eb760e5f661`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916034::DEBUG::2013-11-20 04:15:31,969::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916034::DEBUG::2013-11-20 04:15:31,971::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916034::DEBUG::2013-11-20 04:15:31,972::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`fa17d78b-bc76-4979-8ae3-3eb760e5f661`::Granted request
>Thread-15916034::DEBUG::2013-11-20 04:15:31,973::task::495::TaskManager.Task::(_debug) Task 01f837c8-7a5b-4e37-b4e3-e23878387cee: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916034::DEBUG::2013-11-20 04:15:31,974::task::495::TaskManager.Task::(_debug) Task 01f837c8-7a5b-4e37-b4e3-e23878387cee: ref 1 aborting False
>Thread-15916034::WARNING::2013-11-20 04:15:31,980::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916034::DEBUG::2013-11-20 04:15:31,982::task::495::TaskManager.Task::(_debug) Task 01f837c8-7a5b-4e37-b4e3-e23878387cee: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916034::DEBUG::2013-11-20 04:15:31,985::task::495::TaskManager.Task::(_debug) Task 01f837c8-7a5b-4e37-b4e3-e23878387cee: moving from state preparing -> state finished
>Thread-15916034::DEBUG::2013-11-20 04:15:31,987::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916034::DEBUG::2013-11-20 04:15:31,988::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916034::DEBUG::2013-11-20 04:15:31,990::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916034::DEBUG::2013-11-20 04:15:31,991::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916034::DEBUG::2013-11-20 04:15:31,993::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916034::DEBUG::2013-11-20 04:15:31,994::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916034::DEBUG::2013-11-20 04:15:31,996::task::495::TaskManager.Task::(_debug) Task 01f837c8-7a5b-4e37-b4e3-e23878387cee: ref 0 aborting False
>Thread-15916034::INFO::2013-11-20 04:15:31,998::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:15:32,010::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0755367 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:15:34,028::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:15:34,548::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0797742 s, 12.8 MB/s\n'; <rc> = 0
>Thread-15916036::INFO::2013-11-20 04:15:35,604::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916036::DEBUG::2013-11-20 04:15:35,605::task::495::TaskManager.Task::(_debug) Task 0723c4a2-7f12-48d5-8267-7f43f18a19f7: moving from state init -> state preparing
>Thread-15916036::DEBUG::2013-11-20 04:15:35,617::task::495::TaskManager.Task::(_debug) Task 0723c4a2-7f12-48d5-8267-7f43f18a19f7: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.001051902771', 'lastCheck': 1384920931.637399, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00132489204407', 'lastCheck': 1384920931.043848, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00146794319153', 'lastCheck': 1384920927.5180149, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000458002090454', 'lastCheck': 1384920925.4146669, 'valid': True, 'code': 0}}
>Thread-15916036::DEBUG::2013-11-20 04:15:35,618::task::495::TaskManager.Task::(_debug) Task 0723c4a2-7f12-48d5-8267-7f43f18a19f7: moving from state preparing -> state finished
>Thread-15916036::DEBUG::2013-11-20 04:15:35,618::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916036::DEBUG::2013-11-20 04:15:35,619::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916036::DEBUG::2013-11-20 04:15:35,620::task::495::TaskManager.Task::(_debug) Task 0723c4a2-7f12-48d5-8267-7f43f18a19f7: ref 0 aborting False
>Thread-15916036::INFO::2013-11-20 04:15:35,620::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.001051902771', 'lastCheck': 1384920931.637399, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00132489204407', 'lastCheck': 1384920931.043848, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00146794319153', 'lastCheck': 1384920927.5180149, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000458002090454', 'lastCheck': 1384920925.4146669, 'valid': True, 'code': 0}}
>Thread-42840::DEBUG::2013-11-20 04:15:35,630::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 04:15:36,459::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:15:36,563::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915863::INFO::2013-11-20 04:15:36,773::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:15:36,775::task::495::TaskManager.Task::(_debug) Task f4298689-18ea-4c4a-940c-72a220f49f42: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:15:36,779::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`989db52d-39ed-4b31-9aba-a8f481dade93`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:15:36,780::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:15:36,782::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:15:36,783::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`989db52d-39ed-4b31-9aba-a8f481dade93`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:15:36,784::task::495::TaskManager.Task::(_debug) Task f4298689-18ea-4c4a-940c-72a220f49f42: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:15:36,785::task::495::TaskManager.Task::(_debug) Task f4298689-18ea-4c4a-940c-72a220f49f42: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:15:36,787::task::495::TaskManager.Task::(_debug) Task f4298689-18ea-4c4a-940c-72a220f49f42: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:15:36,789::task::495::TaskManager.Task::(_debug) Task f4298689-18ea-4c4a-940c-72a220f49f42: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:15:36,790::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:15:36,791::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:15:36,792::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:15:36,793::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:15:36,796::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:15:36,798::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:15:36,800::task::495::TaskManager.Task::(_debug) Task f4298689-18ea-4c4a-940c-72a220f49f42: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:15:36,801::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915864::DEBUG::2013-11-20 04:15:37,011::guestIF::97::vm.Vm::(_handleMessage) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::u'BRD-KLEE\\administrator'
>Dummy-15912651::DEBUG::2013-11-20 04:15:37,133::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0841569 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:15:39,149::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:15:39,717::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0781961 s, 13.1 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 04:15:40,957::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:15:40,958::task::495::TaskManager.Task::(_debug) Task a7fd02d1-8596-49ba-a17f-1f4e5f396ce9: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:15:40,960::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`452199ca-d5e5-4d9a-b0b4-3af2f971f93e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:15:40,961::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:15:40,962::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:15:40,962::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`452199ca-d5e5-4d9a-b0b4-3af2f971f93e`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:15:40,964::task::495::TaskManager.Task::(_debug) Task a7fd02d1-8596-49ba-a17f-1f4e5f396ce9: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:15:40,964::task::495::TaskManager.Task::(_debug) Task a7fd02d1-8596-49ba-a17f-1f4e5f396ce9: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:15:40,965::task::495::TaskManager.Task::(_debug) Task a7fd02d1-8596-49ba-a17f-1f4e5f396ce9: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:15:40,966::task::495::TaskManager.Task::(_debug) Task a7fd02d1-8596-49ba-a17f-1f4e5f396ce9: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:15:40,967::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:15:40,967::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:15:40,968::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:15:40,969::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:15:40,970::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:15:40,971::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:15:40,972::task::495::TaskManager.Task::(_debug) Task a7fd02d1-8596-49ba-a17f-1f4e5f396ce9: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:15:40,972::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:15:40,973::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:15:40,974::task::495::TaskManager.Task::(_debug) Task 6659e3df-eae5-46e8-a2c1-62289185d8a1: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:15:40,975::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`18053dc8-d552-44f8-b2b5-264e67c97ff6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:15:40,976::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:15:40,977::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:15:40,978::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`18053dc8-d552-44f8-b2b5-264e67c97ff6`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:15:40,979::task::495::TaskManager.Task::(_debug) Task 6659e3df-eae5-46e8-a2c1-62289185d8a1: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:15:40,979::task::495::TaskManager.Task::(_debug) Task 6659e3df-eae5-46e8-a2c1-62289185d8a1: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:15:40,980::task::495::TaskManager.Task::(_debug) Task 6659e3df-eae5-46e8-a2c1-62289185d8a1: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:15:40,981::task::495::TaskManager.Task::(_debug) Task 6659e3df-eae5-46e8-a2c1-62289185d8a1: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:15:40,982::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:15:40,982::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:15:40,983::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:15:40,984::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:15:40,984::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:15:40,985::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:15:40,986::task::495::TaskManager.Task::(_debug) Task 6659e3df-eae5-46e8-a2c1-62289185d8a1: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:15:40,986::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:15:41,732::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:15:42,343::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0835839 s, 12.3 MB/s\n'; <rc> = 0
>Thread-15916039::DEBUG::2013-11-20 04:15:42,469::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916039::INFO::2013-11-20 04:15:42,471::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916039::DEBUG::2013-11-20 04:15:42,471::task::495::TaskManager.Task::(_debug) Task 789bf782-d096-4ec0-84bf-5982390ac82a: moving from state init -> state preparing
>Thread-15916039::DEBUG::2013-11-20 04:15:42,473::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916039::DEBUG::2013-11-20 04:15:42,474::task::495::TaskManager.Task::(_debug) Task 789bf782-d096-4ec0-84bf-5982390ac82a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916039::DEBUG::2013-11-20 04:15:42,475::task::495::TaskManager.Task::(_debug) Task 789bf782-d096-4ec0-84bf-5982390ac82a: moving from state preparing -> state finished
>Thread-15916039::DEBUG::2013-11-20 04:15:42,476::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916039::DEBUG::2013-11-20 04:15:42,476::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916039::DEBUG::2013-11-20 04:15:42,477::task::495::TaskManager.Task::(_debug) Task 789bf782-d096-4ec0-84bf-5982390ac82a: ref 0 aborting False
>Thread-15916039::INFO::2013-11-20 04:15:42,478::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916040::DEBUG::2013-11-20 04:15:42,484::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916040::INFO::2013-11-20 04:15:42,486::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916040::DEBUG::2013-11-20 04:15:42,486::task::495::TaskManager.Task::(_debug) Task 66edbf7e-32eb-4024-bf66-6a61dbd0ba0c: moving from state init -> state preparing
>Thread-15916040::DEBUG::2013-11-20 04:15:42,488::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`11182036-b7c6-46a7-aaca-28c027e70f3c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916040::DEBUG::2013-11-20 04:15:42,488::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916040::DEBUG::2013-11-20 04:15:42,489::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916040::DEBUG::2013-11-20 04:15:42,490::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`11182036-b7c6-46a7-aaca-28c027e70f3c`::Granted request
>Thread-15916040::DEBUG::2013-11-20 04:15:42,491::task::495::TaskManager.Task::(_debug) Task 66edbf7e-32eb-4024-bf66-6a61dbd0ba0c: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916040::DEBUG::2013-11-20 04:15:42,492::task::495::TaskManager.Task::(_debug) Task 66edbf7e-32eb-4024-bf66-6a61dbd0ba0c: ref 1 aborting False
>Thread-15916040::WARNING::2013-11-20 04:15:42,495::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916040::DEBUG::2013-11-20 04:15:42,496::task::495::TaskManager.Task::(_debug) Task 66edbf7e-32eb-4024-bf66-6a61dbd0ba0c: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916040::DEBUG::2013-11-20 04:15:42,497::task::495::TaskManager.Task::(_debug) Task 66edbf7e-32eb-4024-bf66-6a61dbd0ba0c: moving from state preparing -> state finished
>Thread-15916040::DEBUG::2013-11-20 04:15:42,498::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916040::DEBUG::2013-11-20 04:15:42,498::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916040::DEBUG::2013-11-20 04:15:42,499::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916040::DEBUG::2013-11-20 04:15:42,500::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916040::DEBUG::2013-11-20 04:15:42,501::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916040::DEBUG::2013-11-20 04:15:42,501::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916040::DEBUG::2013-11-20 04:15:42,502::task::495::TaskManager.Task::(_debug) Task 66edbf7e-32eb-4024-bf66-6a61dbd0ba0c: ref 0 aborting False
>Thread-15916040::INFO::2013-11-20 04:15:42,503::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:15:44,357::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 04:15:44,581::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:15:44,583::task::495::TaskManager.Task::(_debug) Task 1b415792-efb6-4105-870e-04200adcf40c: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:15:44,587::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fbc6430d-70e4-49e9-a9fb-ce36b8e7adc9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:15:44,588::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:15:44,589::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:15:44,590::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fbc6430d-70e4-49e9-a9fb-ce36b8e7adc9`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:15:44,591::task::495::TaskManager.Task::(_debug) Task 1b415792-efb6-4105-870e-04200adcf40c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:15:44,592::task::495::TaskManager.Task::(_debug) Task 1b415792-efb6-4105-870e-04200adcf40c: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:15:44,593::task::495::TaskManager.Task::(_debug) Task 1b415792-efb6-4105-870e-04200adcf40c: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:15:44,594::task::495::TaskManager.Task::(_debug) Task 1b415792-efb6-4105-870e-04200adcf40c: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:15:44,594::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:15:44,595::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:15:44,596::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:15:44,596::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:15:44,597::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:15:44,597::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:15:44,598::task::495::TaskManager.Task::(_debug) Task 1b415792-efb6-4105-870e-04200adcf40c: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:15:44,599::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:15:44,801::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0512659 s, 20.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:15:46,813::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:15:47,280::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0553992 s, 18.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:15:49,291::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:15:49,878::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0881257 s, 11.6 MB/s\n'; <rc> = 0
>Thread-15916044::INFO::2013-11-20 04:15:51,602::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916044::DEBUG::2013-11-20 04:15:51,603::task::495::TaskManager.Task::(_debug) Task 44d8fa48-c217-40f3-8f00-69a23486753b: moving from state init -> state preparing
>Thread-15916044::DEBUG::2013-11-20 04:15:51,604::task::495::TaskManager.Task::(_debug) Task 44d8fa48-c217-40f3-8f00-69a23486753b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000843048095703', 'lastCheck': 1384920941.9698241, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00168800354004', 'lastCheck': 1384920951.5834529, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00365400314331', 'lastCheck': 1384920948.027334, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000432014465332', 'lastCheck': 1384920946.57196, 'valid': True, 'code': 0}}
>Thread-15916044::DEBUG::2013-11-20 04:15:51,605::task::495::TaskManager.Task::(_debug) Task 44d8fa48-c217-40f3-8f00-69a23486753b: moving from state preparing -> state finished
>Thread-15916044::DEBUG::2013-11-20 04:15:51,606::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916044::DEBUG::2013-11-20 04:15:51,607::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916044::DEBUG::2013-11-20 04:15:51,607::task::495::TaskManager.Task::(_debug) Task 44d8fa48-c217-40f3-8f00-69a23486753b: ref 0 aborting False
>Thread-15916044::INFO::2013-11-20 04:15:51,608::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000843048095703', 'lastCheck': 1384920941.9698241, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00168800354004', 'lastCheck': 1384920951.5834529, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00365400314331', 'lastCheck': 1384920948.027334, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000432014465332', 'lastCheck': 1384920946.57196, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:15:51,894::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:15:52,446::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0761597 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15916046::DEBUG::2013-11-20 04:15:52,796::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916046::INFO::2013-11-20 04:15:52,798::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916046::DEBUG::2013-11-20 04:15:52,799::task::495::TaskManager.Task::(_debug) Task dab3219c-2a24-48f8-878a-cb6ea8bb1939: moving from state init -> state preparing
>Thread-15916046::DEBUG::2013-11-20 04:15:52,800::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916046::DEBUG::2013-11-20 04:15:52,801::task::495::TaskManager.Task::(_debug) Task dab3219c-2a24-48f8-878a-cb6ea8bb1939: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916046::DEBUG::2013-11-20 04:15:52,801::task::495::TaskManager.Task::(_debug) Task dab3219c-2a24-48f8-878a-cb6ea8bb1939: moving from state preparing -> state finished
>Thread-15916046::DEBUG::2013-11-20 04:15:52,802::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916046::DEBUG::2013-11-20 04:15:52,803::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916046::DEBUG::2013-11-20 04:15:52,803::task::495::TaskManager.Task::(_debug) Task dab3219c-2a24-48f8-878a-cb6ea8bb1939: ref 0 aborting False
>Thread-15916046::INFO::2013-11-20 04:15:52,804::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916047::DEBUG::2013-11-20 04:15:52,809::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916047::INFO::2013-11-20 04:15:52,810::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916047::DEBUG::2013-11-20 04:15:52,810::task::495::TaskManager.Task::(_debug) Task 33a87927-1ca7-4fc1-bb54-092fcbe05733: moving from state init -> state preparing
>Thread-15916047::DEBUG::2013-11-20 04:15:52,812::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b01e68f5-f53f-481f-be13-911e3fe5fdf0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916047::DEBUG::2013-11-20 04:15:52,813::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916047::DEBUG::2013-11-20 04:15:52,815::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916047::DEBUG::2013-11-20 04:15:52,815::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b01e68f5-f53f-481f-be13-911e3fe5fdf0`::Granted request
>Thread-15916047::DEBUG::2013-11-20 04:15:52,817::task::495::TaskManager.Task::(_debug) Task 33a87927-1ca7-4fc1-bb54-092fcbe05733: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916047::DEBUG::2013-11-20 04:15:52,818::task::495::TaskManager.Task::(_debug) Task 33a87927-1ca7-4fc1-bb54-092fcbe05733: ref 1 aborting False
>Thread-15916047::WARNING::2013-11-20 04:15:52,820::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916047::DEBUG::2013-11-20 04:15:52,821::task::495::TaskManager.Task::(_debug) Task 33a87927-1ca7-4fc1-bb54-092fcbe05733: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916047::DEBUG::2013-11-20 04:15:52,822::task::495::TaskManager.Task::(_debug) Task 33a87927-1ca7-4fc1-bb54-092fcbe05733: moving from state preparing -> state finished
>Thread-15916047::DEBUG::2013-11-20 04:15:52,822::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916047::DEBUG::2013-11-20 04:15:52,823::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916047::DEBUG::2013-11-20 04:15:52,823::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916047::DEBUG::2013-11-20 04:15:52,831::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916047::DEBUG::2013-11-20 04:15:52,831::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916047::DEBUG::2013-11-20 04:15:52,832::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916047::DEBUG::2013-11-20 04:15:52,832::task::495::TaskManager.Task::(_debug) Task 33a87927-1ca7-4fc1-bb54-092fcbe05733: ref 0 aborting False
>Thread-15916047::INFO::2013-11-20 04:15:52,833::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:15:54,460::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 04:15:54,670::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:15:54,674::task::495::TaskManager.Task::(_debug) Task d8c3a472-f998-4e56-a1c5-ec1378e91ff0: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:15:54,677::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f95ad3ea-c654-49d8-ba15-f0c3efe0b1c1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:15:54,678::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:15:54,679::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:15:54,680::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f95ad3ea-c654-49d8-ba15-f0c3efe0b1c1`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:15:54,682::task::495::TaskManager.Task::(_debug) Task d8c3a472-f998-4e56-a1c5-ec1378e91ff0: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:15:54,682::task::495::TaskManager.Task::(_debug) Task d8c3a472-f998-4e56-a1c5-ec1378e91ff0: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:15:54,683::task::495::TaskManager.Task::(_debug) Task d8c3a472-f998-4e56-a1c5-ec1378e91ff0: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:15:54,684::task::495::TaskManager.Task::(_debug) Task d8c3a472-f998-4e56-a1c5-ec1378e91ff0: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:15:54,685::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:15:54,685::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:15:54,686::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:15:54,687::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:15:54,688::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:15:54,688::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:15:54,689::task::495::TaskManager.Task::(_debug) Task d8c3a472-f998-4e56-a1c5-ec1378e91ff0: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:15:54,690::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:15:55,037::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0992486 s, 10.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:15:57,054::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:15:57,519::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0601604 s, 17.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:15:59,531::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 04:15:59,757::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:15:59,759::task::495::TaskManager.Task::(_debug) Task d328d93a-b318-4f6a-886a-749b2abe7e71: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:15:59,762::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f26116b7-ce79-4942-a87d-11f2545609c2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:15:59,763::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:15:59,765::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:15:59,765::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f26116b7-ce79-4942-a87d-11f2545609c2`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:15:59,767::task::495::TaskManager.Task::(_debug) Task d328d93a-b318-4f6a-886a-749b2abe7e71: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:15:59,768::task::495::TaskManager.Task::(_debug) Task d328d93a-b318-4f6a-886a-749b2abe7e71: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:15:59,769::task::495::TaskManager.Task::(_debug) Task d328d93a-b318-4f6a-886a-749b2abe7e71: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:15:59,769::task::495::TaskManager.Task::(_debug) Task d328d93a-b318-4f6a-886a-749b2abe7e71: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:15:59,770::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:15:59,771::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:15:59,772::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:15:59,773::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:15:59,773::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:15:59,774::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:15:59,775::task::495::TaskManager.Task::(_debug) Task d328d93a-b318-4f6a-886a-749b2abe7e71: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:15:59,776::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:16:00,077::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0782888 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:16:02,091::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:16:02,657::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0911596 s, 11.2 MB/s\n'; <rc> = 0
>Thread-15916051::DEBUG::2013-11-20 04:16:03,098::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916051::INFO::2013-11-20 04:16:03,100::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916051::DEBUG::2013-11-20 04:16:03,101::task::495::TaskManager.Task::(_debug) Task 3510728b-98bf-4e07-84bc-fd4a45331d13: moving from state init -> state preparing
>Thread-15916051::DEBUG::2013-11-20 04:16:03,102::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916051::DEBUG::2013-11-20 04:16:03,102::task::495::TaskManager.Task::(_debug) Task 3510728b-98bf-4e07-84bc-fd4a45331d13: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916051::DEBUG::2013-11-20 04:16:03,103::task::495::TaskManager.Task::(_debug) Task 3510728b-98bf-4e07-84bc-fd4a45331d13: moving from state preparing -> state finished
>Thread-15916051::DEBUG::2013-11-20 04:16:03,104::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916051::DEBUG::2013-11-20 04:16:03,104::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916051::DEBUG::2013-11-20 04:16:03,105::task::495::TaskManager.Task::(_debug) Task 3510728b-98bf-4e07-84bc-fd4a45331d13: ref 0 aborting False
>Thread-15916051::INFO::2013-11-20 04:16:03,105::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916052::DEBUG::2013-11-20 04:16:03,110::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916052::INFO::2013-11-20 04:16:03,112::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916052::DEBUG::2013-11-20 04:16:03,112::task::495::TaskManager.Task::(_debug) Task c60c9490-c946-4e04-9ec2-6c6eced0bb30: moving from state init -> state preparing
>Thread-15916052::DEBUG::2013-11-20 04:16:03,113::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`067f875e-3061-4ed9-ab4f-e7c4706b5253`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916052::DEBUG::2013-11-20 04:16:03,114::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916052::DEBUG::2013-11-20 04:16:03,115::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916052::DEBUG::2013-11-20 04:16:03,115::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`067f875e-3061-4ed9-ab4f-e7c4706b5253`::Granted request
>Thread-15916052::DEBUG::2013-11-20 04:16:03,116::task::495::TaskManager.Task::(_debug) Task c60c9490-c946-4e04-9ec2-6c6eced0bb30: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916052::DEBUG::2013-11-20 04:16:03,117::task::495::TaskManager.Task::(_debug) Task c60c9490-c946-4e04-9ec2-6c6eced0bb30: ref 1 aborting False
>Thread-15916052::WARNING::2013-11-20 04:16:03,119::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916052::DEBUG::2013-11-20 04:16:03,120::task::495::TaskManager.Task::(_debug) Task c60c9490-c946-4e04-9ec2-6c6eced0bb30: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916052::DEBUG::2013-11-20 04:16:03,121::task::495::TaskManager.Task::(_debug) Task c60c9490-c946-4e04-9ec2-6c6eced0bb30: moving from state preparing -> state finished
>Thread-15916052::DEBUG::2013-11-20 04:16:03,121::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916052::DEBUG::2013-11-20 04:16:03,122::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916052::DEBUG::2013-11-20 04:16:03,123::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916052::DEBUG::2013-11-20 04:16:03,123::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916052::DEBUG::2013-11-20 04:16:03,124::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916052::DEBUG::2013-11-20 04:16:03,124::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916052::DEBUG::2013-11-20 04:16:03,125::task::495::TaskManager.Task::(_debug) Task c60c9490-c946-4e04-9ec2-6c6eced0bb30: ref 0 aborting False
>Thread-15916052::INFO::2013-11-20 04:16:03,126::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:16:04,672::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 04:16:04,890::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:16:04,893::task::495::TaskManager.Task::(_debug) Task 023f0895-ba62-4dde-a6ef-3616bf62280a: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:16:04,897::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`72d77d92-8743-45ab-9010-e6cd10faaaa7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:16:04,898::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:16:04,899::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:16:04,900::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`72d77d92-8743-45ab-9010-e6cd10faaaa7`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:16:04,901::task::495::TaskManager.Task::(_debug) Task 023f0895-ba62-4dde-a6ef-3616bf62280a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:16:04,902::task::495::TaskManager.Task::(_debug) Task 023f0895-ba62-4dde-a6ef-3616bf62280a: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:16:04,903::task::495::TaskManager.Task::(_debug) Task 023f0895-ba62-4dde-a6ef-3616bf62280a: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:16:04,904::task::495::TaskManager.Task::(_debug) Task 023f0895-ba62-4dde-a6ef-3616bf62280a: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:16:04,904::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:16:04,905::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:16:04,906::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:16:04,907::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:16:04,907::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:16:04,908::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:16:04,909::task::495::TaskManager.Task::(_debug) Task 023f0895-ba62-4dde-a6ef-3616bf62280a: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:16:04,910::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:16:05,230::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0812511 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:16:07,247::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916054::INFO::2013-11-20 04:16:07,489::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916054::DEBUG::2013-11-20 04:16:07,490::task::495::TaskManager.Task::(_debug) Task 11a56e3a-dba4-497b-a1d1-9b2c4c5a2a99: moving from state init -> state preparing
>Thread-15916054::DEBUG::2013-11-20 04:16:07,491::task::495::TaskManager.Task::(_debug) Task 11a56e3a-dba4-497b-a1d1-9b2c4c5a2a99: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000764846801758', 'lastCheck': 1384920962.3296821, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0292229652405', 'lastCheck': 1384920961.790957, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00130319595337', 'lastCheck': 1384920958.2715509, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000471115112305', 'lastCheck': 1384920966.888988, 'valid': True, 'code': 0}}
>Thread-15916054::DEBUG::2013-11-20 04:16:07,493::task::495::TaskManager.Task::(_debug) Task 11a56e3a-dba4-497b-a1d1-9b2c4c5a2a99: moving from state preparing -> state finished
>Thread-15916054::DEBUG::2013-11-20 04:16:07,493::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916054::DEBUG::2013-11-20 04:16:07,494::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916054::DEBUG::2013-11-20 04:16:07,494::task::495::TaskManager.Task::(_debug) Task 11a56e3a-dba4-497b-a1d1-9b2c4c5a2a99: ref 0 aborting False
>Thread-15916054::INFO::2013-11-20 04:16:07,495::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000764846801758', 'lastCheck': 1384920962.3296821, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0292229652405', 'lastCheck': 1384920961.790957, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00130319595337', 'lastCheck': 1384920958.2715509, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000471115112305', 'lastCheck': 1384920966.888988, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:16:07,774::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0587522 s, 17.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:16:09,782::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:16:10,373::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0984969 s, 10.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:16:12,390::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 04:16:12,654::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914435::INFO::2013-11-20 04:16:12,657::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914519::DEBUG::2013-11-20 04:16:12,659::task::495::TaskManager.Task::(_debug) Task c1b4318e-5977-4d02-8ea5-c0c3c20f3f0f: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:16:12,660::task::495::TaskManager.Task::(_debug) Task bb44c80a-800b-4091-ad6f-2458d32b341b: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:16:12,664::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4480e94a-4453-4b08-9f9c-4d3d0d6a9589`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:16:12,666::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fcc68508-65b7-4ab2-9658-b69f6da539e3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:16:12,667::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:16:12,668::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:16:12,672::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:16:12,673::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4480e94a-4453-4b08-9f9c-4d3d0d6a9589`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:16:12,675::task::495::TaskManager.Task::(_debug) Task c1b4318e-5977-4d02-8ea5-c0c3c20f3f0f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:16:12,675::resourceManager::481::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' found in shared state and queue is empty, Joining current shared lock (2 active users)
>Thread-15914519::DEBUG::2013-11-20 04:16:12,680::task::495::TaskManager.Task::(_debug) Task c1b4318e-5977-4d02-8ea5-c0c3c20f3f0f: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:16:12,680::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fcc68508-65b7-4ab2-9658-b69f6da539e3`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:16:12,686::task::495::TaskManager.Task::(_debug) Task c1b4318e-5977-4d02-8ea5-c0c3c20f3f0f: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 04:16:12,688::task::495::TaskManager.Task::(_debug) Task c1b4318e-5977-4d02-8ea5-c0c3c20f3f0f: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:16:12,689::task::495::TaskManager.Task::(_debug) Task bb44c80a-800b-4091-ad6f-2458d32b341b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 04:16:12,690::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:16:12,690::task::495::TaskManager.Task::(_debug) Task bb44c80a-800b-4091-ad6f-2458d32b341b: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:16:12,691::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:16:12,692::task::495::TaskManager.Task::(_debug) Task bb44c80a-800b-4091-ad6f-2458d32b341b: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914519::DEBUG::2013-11-20 04:16:12,693::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:16:12,693::task::495::TaskManager.Task::(_debug) Task bb44c80a-800b-4091-ad6f-2458d32b341b: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:16:12,694::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (1 active users)
>Thread-15914435::DEBUG::2013-11-20 04:16:12,695::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:16:12,702::task::495::TaskManager.Task::(_debug) Task c1b4318e-5977-4d02-8ea5-c0c3c20f3f0f: ref 0 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:16:12,703::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::INFO::2013-11-20 04:16:12,704::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914435::DEBUG::2013-11-20 04:16:12,705::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:16:12,706::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:16:12,707::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:16:12,708::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:16:12,709::task::495::TaskManager.Task::(_debug) Task bb44c80a-800b-4091-ad6f-2458d32b341b: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:16:12,710::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:16:12,941::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0808725 s, 12.7 MB/s\n'; <rc> = 0
>Thread-15916057::DEBUG::2013-11-20 04:16:13,636::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916057::INFO::2013-11-20 04:16:13,638::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916057::DEBUG::2013-11-20 04:16:13,638::task::495::TaskManager.Task::(_debug) Task 3072ffe1-2235-4294-a2a3-8be76d05443f: moving from state init -> state preparing
>Thread-15916057::DEBUG::2013-11-20 04:16:13,639::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916057::DEBUG::2013-11-20 04:16:13,640::task::495::TaskManager.Task::(_debug) Task 3072ffe1-2235-4294-a2a3-8be76d05443f: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916057::DEBUG::2013-11-20 04:16:13,641::task::495::TaskManager.Task::(_debug) Task 3072ffe1-2235-4294-a2a3-8be76d05443f: moving from state preparing -> state finished
>Thread-15916057::DEBUG::2013-11-20 04:16:13,642::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916057::DEBUG::2013-11-20 04:16:13,642::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916057::DEBUG::2013-11-20 04:16:13,643::task::495::TaskManager.Task::(_debug) Task 3072ffe1-2235-4294-a2a3-8be76d05443f: ref 0 aborting False
>Thread-15916057::INFO::2013-11-20 04:16:13,644::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916058::DEBUG::2013-11-20 04:16:13,649::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916058::INFO::2013-11-20 04:16:13,650::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916058::DEBUG::2013-11-20 04:16:13,651::task::495::TaskManager.Task::(_debug) Task 7107da60-50cc-4c66-a57e-34ce8d95682c: moving from state init -> state preparing
>Thread-15916058::DEBUG::2013-11-20 04:16:13,652::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7782840d-bc5e-4561-adae-659c1a8ce3a1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916058::DEBUG::2013-11-20 04:16:13,653::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916058::DEBUG::2013-11-20 04:16:13,653::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916058::DEBUG::2013-11-20 04:16:13,654::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7782840d-bc5e-4561-adae-659c1a8ce3a1`::Granted request
>Thread-15916058::DEBUG::2013-11-20 04:16:13,655::task::495::TaskManager.Task::(_debug) Task 7107da60-50cc-4c66-a57e-34ce8d95682c: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916058::DEBUG::2013-11-20 04:16:13,656::task::495::TaskManager.Task::(_debug) Task 7107da60-50cc-4c66-a57e-34ce8d95682c: ref 1 aborting False
>Thread-15916058::WARNING::2013-11-20 04:16:13,658::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916058::DEBUG::2013-11-20 04:16:13,659::task::495::TaskManager.Task::(_debug) Task 7107da60-50cc-4c66-a57e-34ce8d95682c: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916058::DEBUG::2013-11-20 04:16:13,660::task::495::TaskManager.Task::(_debug) Task 7107da60-50cc-4c66-a57e-34ce8d95682c: moving from state preparing -> state finished
>Thread-15916058::DEBUG::2013-11-20 04:16:13,661::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916058::DEBUG::2013-11-20 04:16:13,661::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916058::DEBUG::2013-11-20 04:16:13,662::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916058::DEBUG::2013-11-20 04:16:13,663::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916058::DEBUG::2013-11-20 04:16:13,664::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916058::DEBUG::2013-11-20 04:16:13,664::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916058::DEBUG::2013-11-20 04:16:13,665::task::495::TaskManager.Task::(_debug) Task 7107da60-50cc-4c66-a57e-34ce8d95682c: ref 0 aborting False
>Thread-15916058::INFO::2013-11-20 04:16:13,666::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:16:14,953::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 04:16:15,161::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:16:15,165::task::495::TaskManager.Task::(_debug) Task aa006f92-c785-4fa0-8cc6-6116360b24c4: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:16:15,169::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`94f11eca-6705-42e3-8062-2838d8558262`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:16:15,170::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:16:15,172::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:16:15,173::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`94f11eca-6705-42e3-8062-2838d8558262`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:16:15,174::task::495::TaskManager.Task::(_debug) Task aa006f92-c785-4fa0-8cc6-6116360b24c4: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:16:15,174::task::495::TaskManager.Task::(_debug) Task aa006f92-c785-4fa0-8cc6-6116360b24c4: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:16:15,175::task::495::TaskManager.Task::(_debug) Task aa006f92-c785-4fa0-8cc6-6116360b24c4: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 04:16:15,176::task::495::TaskManager.Task::(_debug) Task aa006f92-c785-4fa0-8cc6-6116360b24c4: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:16:15,177::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:16:15,177::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:16:15,178::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:16:15,179::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:16:15,179::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:16:15,180::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:16:15,180::task::495::TaskManager.Task::(_debug) Task aa006f92-c785-4fa0-8cc6-6116360b24c4: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:16:15,181::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:16:15,182::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:16:15,183::task::495::TaskManager.Task::(_debug) Task a592f09e-de9e-4b2c-8d8e-6e1925b12aaa: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:16:15,184::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`4251b592-bf9c-43ad-9fb7-4f7044c9aca0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:16:15,184::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:16:15,185::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:16:15,186::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`4251b592-bf9c-43ad-9fb7-4f7044c9aca0`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:16:15,187::task::495::TaskManager.Task::(_debug) Task a592f09e-de9e-4b2c-8d8e-6e1925b12aaa: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:16:15,187::task::495::TaskManager.Task::(_debug) Task a592f09e-de9e-4b2c-8d8e-6e1925b12aaa: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:16:15,188::task::495::TaskManager.Task::(_debug) Task a592f09e-de9e-4b2c-8d8e-6e1925b12aaa: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:16:15,189::task::495::TaskManager.Task::(_debug) Task a592f09e-de9e-4b2c-8d8e-6e1925b12aaa: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:16:15,189::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:16:15,190::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:16:15,191::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:16:15,191::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:16:15,192::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:16:15,192::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:16:15,193::task::495::TaskManager.Task::(_debug) Task a592f09e-de9e-4b2c-8d8e-6e1925b12aaa: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:16:15,193::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:16:15,194::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:16:15,195::task::495::TaskManager.Task::(_debug) Task 7c02f13c-73d7-4809-906a-ea015d98aaac: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:16:15,196::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fa4bb57b-df8a-4fdf-9c77-5732257a276e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:16:15,197::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:16:15,198::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:16:15,198::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fa4bb57b-df8a-4fdf-9c77-5732257a276e`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:16:15,199::task::495::TaskManager.Task::(_debug) Task 7c02f13c-73d7-4809-906a-ea015d98aaac: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:16:15,200::task::495::TaskManager.Task::(_debug) Task 7c02f13c-73d7-4809-906a-ea015d98aaac: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:16:15,201::task::495::TaskManager.Task::(_debug) Task 7c02f13c-73d7-4809-906a-ea015d98aaac: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:16:15,202::task::495::TaskManager.Task::(_debug) Task 7c02f13c-73d7-4809-906a-ea015d98aaac: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:16:15,202::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:16:15,203::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:16:15,204::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:16:15,204::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:16:15,205::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:16:15,205::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:16:15,206::task::495::TaskManager.Task::(_debug) Task 7c02f13c-73d7-4809-906a-ea015d98aaac: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:16:15,207::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:16:15,208::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:16:15,208::task::495::TaskManager.Task::(_debug) Task c2e62141-1528-45c5-a486-84813059a5b0: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:16:15,209::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`bca42552-83ef-4fab-83c3-3465c2b28942`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:16:15,210::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:16:15,211::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:16:15,212::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`bca42552-83ef-4fab-83c3-3465c2b28942`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:16:15,213::task::495::TaskManager.Task::(_debug) Task c2e62141-1528-45c5-a486-84813059a5b0: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:16:15,214::task::495::TaskManager.Task::(_debug) Task c2e62141-1528-45c5-a486-84813059a5b0: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:16:15,214::task::495::TaskManager.Task::(_debug) Task c2e62141-1528-45c5-a486-84813059a5b0: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:16:15,215::task::495::TaskManager.Task::(_debug) Task c2e62141-1528-45c5-a486-84813059a5b0: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:16:15,216::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:16:15,216::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:16:15,217::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:16:15,218::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:16:15,219::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:16:15,219::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:16:15,220::task::495::TaskManager.Task::(_debug) Task c2e62141-1528-45c5-a486-84813059a5b0: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:16:15,220::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:16:15,430::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0538642 s, 19.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:16:17,445::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:16:17,979::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0763963 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:16:19,997::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:16:20,455::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.075004 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:16:22,469::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:16:23,044::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.096562 s, 10.6 MB/s\n'; <rc> = 0
>Thread-15916062::INFO::2013-11-20 04:16:23,475::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916062::DEBUG::2013-11-20 04:16:23,476::task::495::TaskManager.Task::(_debug) Task 9ed8ac12-3cb6-4842-9bd4-7e7b5f6e6013: moving from state init -> state preparing
>Thread-15916062::DEBUG::2013-11-20 04:16:23,477::task::495::TaskManager.Task::(_debug) Task 9ed8ac12-3cb6-4842-9bd4-7e7b5f6e6013: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000530958175659', 'lastCheck': 1384920982.698787, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00122904777527', 'lastCheck': 1384920982.1669419, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00165796279907', 'lastCheck': 1384920978.6631491, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000431060791016', 'lastCheck': 1384920977.158391, 'valid': True, 'code': 0}}
>Thread-15916062::DEBUG::2013-11-20 04:16:23,478::task::495::TaskManager.Task::(_debug) Task 9ed8ac12-3cb6-4842-9bd4-7e7b5f6e6013: moving from state preparing -> state finished
>Thread-15916062::DEBUG::2013-11-20 04:16:23,478::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916062::DEBUG::2013-11-20 04:16:23,479::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916062::DEBUG::2013-11-20 04:16:23,480::task::495::TaskManager.Task::(_debug) Task 9ed8ac12-3cb6-4842-9bd4-7e7b5f6e6013: ref 0 aborting False
>Thread-15916062::INFO::2013-11-20 04:16:23,480::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000530958175659', 'lastCheck': 1384920982.698787, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00122904777527', 'lastCheck': 1384920982.1669419, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00165796279907', 'lastCheck': 1384920978.6631491, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000431060791016', 'lastCheck': 1384920977.158391, 'valid': True, 'code': 0}}
>Thread-15916064::DEBUG::2013-11-20 04:16:24,181::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916064::INFO::2013-11-20 04:16:24,183::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916064::DEBUG::2013-11-20 04:16:24,183::task::495::TaskManager.Task::(_debug) Task de90302b-3ca9-4145-8e79-6003474bdfe0: moving from state init -> state preparing
>Thread-15916064::DEBUG::2013-11-20 04:16:24,185::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916064::DEBUG::2013-11-20 04:16:24,185::task::495::TaskManager.Task::(_debug) Task de90302b-3ca9-4145-8e79-6003474bdfe0: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916064::DEBUG::2013-11-20 04:16:24,186::task::495::TaskManager.Task::(_debug) Task de90302b-3ca9-4145-8e79-6003474bdfe0: moving from state preparing -> state finished
>Thread-15916064::DEBUG::2013-11-20 04:16:24,186::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916064::DEBUG::2013-11-20 04:16:24,187::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916064::DEBUG::2013-11-20 04:16:24,188::task::495::TaskManager.Task::(_debug) Task de90302b-3ca9-4145-8e79-6003474bdfe0: ref 0 aborting False
>Thread-15916064::INFO::2013-11-20 04:16:24,188::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916065::DEBUG::2013-11-20 04:16:24,250::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916065::INFO::2013-11-20 04:16:24,251::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916065::DEBUG::2013-11-20 04:16:24,252::task::495::TaskManager.Task::(_debug) Task 9ea68cdd-b6a3-4e6d-927d-0bc6a6f49983: moving from state init -> state preparing
>Thread-15916065::DEBUG::2013-11-20 04:16:24,253::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`940f8133-cabf-4140-8082-e150a16b9f65`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916065::DEBUG::2013-11-20 04:16:24,254::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916065::DEBUG::2013-11-20 04:16:24,254::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916065::DEBUG::2013-11-20 04:16:24,255::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`940f8133-cabf-4140-8082-e150a16b9f65`::Granted request
>Thread-15916065::DEBUG::2013-11-20 04:16:24,256::task::495::TaskManager.Task::(_debug) Task 9ea68cdd-b6a3-4e6d-927d-0bc6a6f49983: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916065::DEBUG::2013-11-20 04:16:24,257::task::495::TaskManager.Task::(_debug) Task 9ea68cdd-b6a3-4e6d-927d-0bc6a6f49983: ref 1 aborting False
>Thread-15916065::WARNING::2013-11-20 04:16:24,259::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916065::DEBUG::2013-11-20 04:16:24,260::task::495::TaskManager.Task::(_debug) Task 9ea68cdd-b6a3-4e6d-927d-0bc6a6f49983: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916065::DEBUG::2013-11-20 04:16:24,261::task::495::TaskManager.Task::(_debug) Task 9ea68cdd-b6a3-4e6d-927d-0bc6a6f49983: moving from state preparing -> state finished
>Thread-15916065::DEBUG::2013-11-20 04:16:24,261::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916065::DEBUG::2013-11-20 04:16:24,262::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916065::DEBUG::2013-11-20 04:16:24,263::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916065::DEBUG::2013-11-20 04:16:24,263::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916065::DEBUG::2013-11-20 04:16:24,264::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916065::DEBUG::2013-11-20 04:16:24,265::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916065::DEBUG::2013-11-20 04:16:24,265::task::495::TaskManager.Task::(_debug) Task 9ea68cdd-b6a3-4e6d-927d-0bc6a6f49983: ref 0 aborting False
>Thread-15916065::INFO::2013-11-20 04:16:24,266::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915864::DEBUG::2013-11-20 04:16:24,662::guestIF::97::vm.Vm::(_handleMessage) vmId=`01c8c942-dab4-4e5e-aefa-d474a455143e`::u'None'
>Dummy-15912651::DEBUG::2013-11-20 04:16:25,062::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:16:25,615::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0828607 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:16:27,630::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:16:28,206::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.08338 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:16:30,216::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:16:30,460::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:16:30,462::task::495::TaskManager.Task::(_debug) Task 146e36a8-138f-4761-9953-fad9451ff0ad: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:16:30,467::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c0d33cb4-2bde-4c4f-b3d4-afaf30aaa5cd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:16:30,468::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:16:30,471::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:16:30,471::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c0d33cb4-2bde-4c4f-b3d4-afaf30aaa5cd`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:16:30,473::task::495::TaskManager.Task::(_debug) Task 146e36a8-138f-4761-9953-fad9451ff0ad: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:16:30,474::task::495::TaskManager.Task::(_debug) Task 146e36a8-138f-4761-9953-fad9451ff0ad: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:16:30,474::task::495::TaskManager.Task::(_debug) Task 146e36a8-138f-4761-9953-fad9451ff0ad: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:16:30,475::task::495::TaskManager.Task::(_debug) Task 146e36a8-138f-4761-9953-fad9451ff0ad: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:16:30,475::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:16:30,476::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:16:30,476::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:16:30,477::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:16:30,478::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:16:30,478::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:16:30,479::task::495::TaskManager.Task::(_debug) Task 146e36a8-138f-4761-9953-fad9451ff0ad: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:16:30,479::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:16:30,815::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0878673 s, 11.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:16:32,829::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915918::INFO::2013-11-20 04:16:33,035::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:16:33,041::task::495::TaskManager.Task::(_debug) Task 3fd18501-79f1-4270-8b10-c9d30eb0a96d: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:16:33,044::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a605be23-5277-4ab1-b827-d61c167ca7bc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:16:33,045::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:16:33,046::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:16:33,047::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a605be23-5277-4ab1-b827-d61c167ca7bc`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:16:33,048::task::495::TaskManager.Task::(_debug) Task 3fd18501-79f1-4270-8b10-c9d30eb0a96d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:16:33,049::task::495::TaskManager.Task::(_debug) Task 3fd18501-79f1-4270-8b10-c9d30eb0a96d: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:16:33,050::task::495::TaskManager.Task::(_debug) Task 3fd18501-79f1-4270-8b10-c9d30eb0a96d: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:16:33,050::task::495::TaskManager.Task::(_debug) Task 3fd18501-79f1-4270-8b10-c9d30eb0a96d: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:16:33,051::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:16:33,051::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:16:33,053::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:16:33,053::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:16:33,054::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:16:33,054::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:16:33,055::task::495::TaskManager.Task::(_debug) Task 3fd18501-79f1-4270-8b10-c9d30eb0a96d: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:16:33,056::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:16:33,369::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0765535 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15916069::DEBUG::2013-11-20 04:16:34,709::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916069::INFO::2013-11-20 04:16:34,710::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916069::DEBUG::2013-11-20 04:16:34,711::task::495::TaskManager.Task::(_debug) Task f00ed699-e9fe-42c1-bbab-230730c97aa6: moving from state init -> state preparing
>Thread-15916069::DEBUG::2013-11-20 04:16:34,712::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916069::DEBUG::2013-11-20 04:16:34,713::task::495::TaskManager.Task::(_debug) Task f00ed699-e9fe-42c1-bbab-230730c97aa6: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916069::DEBUG::2013-11-20 04:16:34,713::task::495::TaskManager.Task::(_debug) Task f00ed699-e9fe-42c1-bbab-230730c97aa6: moving from state preparing -> state finished
>Thread-15916069::DEBUG::2013-11-20 04:16:34,714::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916069::DEBUG::2013-11-20 04:16:34,715::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916069::DEBUG::2013-11-20 04:16:34,715::task::495::TaskManager.Task::(_debug) Task f00ed699-e9fe-42c1-bbab-230730c97aa6: ref 0 aborting False
>Thread-15916069::INFO::2013-11-20 04:16:34,716::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916070::DEBUG::2013-11-20 04:16:34,751::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916070::INFO::2013-11-20 04:16:34,753::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916070::DEBUG::2013-11-20 04:16:34,753::task::495::TaskManager.Task::(_debug) Task afa93fbc-91ce-4fa3-ad01-ad8d34d54dfe: moving from state init -> state preparing
>Thread-15916070::DEBUG::2013-11-20 04:16:34,755::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bdba93d4-2fb4-4edb-a328-8163a0e71f14`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916070::DEBUG::2013-11-20 04:16:34,755::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916070::DEBUG::2013-11-20 04:16:34,756::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916070::DEBUG::2013-11-20 04:16:34,757::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bdba93d4-2fb4-4edb-a328-8163a0e71f14`::Granted request
>Thread-15916070::DEBUG::2013-11-20 04:16:34,758::task::495::TaskManager.Task::(_debug) Task afa93fbc-91ce-4fa3-ad01-ad8d34d54dfe: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916070::DEBUG::2013-11-20 04:16:34,759::task::495::TaskManager.Task::(_debug) Task afa93fbc-91ce-4fa3-ad01-ad8d34d54dfe: ref 1 aborting False
>Thread-15916070::WARNING::2013-11-20 04:16:34,762::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916070::DEBUG::2013-11-20 04:16:34,762::task::495::TaskManager.Task::(_debug) Task afa93fbc-91ce-4fa3-ad01-ad8d34d54dfe: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916070::DEBUG::2013-11-20 04:16:34,763::task::495::TaskManager.Task::(_debug) Task afa93fbc-91ce-4fa3-ad01-ad8d34d54dfe: moving from state preparing -> state finished
>Thread-15916070::DEBUG::2013-11-20 04:16:34,764::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916070::DEBUG::2013-11-20 04:16:34,764::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916070::DEBUG::2013-11-20 04:16:34,765::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916070::DEBUG::2013-11-20 04:16:34,766::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916070::DEBUG::2013-11-20 04:16:34,766::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916070::DEBUG::2013-11-20 04:16:34,767::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916070::DEBUG::2013-11-20 04:16:34,768::task::495::TaskManager.Task::(_debug) Task afa93fbc-91ce-4fa3-ad01-ad8d34d54dfe: ref 0 aborting False
>Thread-15916070::INFO::2013-11-20 04:16:34,768::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:16:35,384::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:16:35,916::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0647884 s, 15.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:16:37,928::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915863::INFO::2013-11-20 04:16:38,141::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:16:38,146::task::495::TaskManager.Task::(_debug) Task 327fe5a4-9cc8-453f-83b0-b4580d6818fd: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:16:38,149::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c3c8091a-ff72-482e-b2fc-bf96a16e35e1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:16:38,150::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:16:38,153::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:16:38,154::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c3c8091a-ff72-482e-b2fc-bf96a16e35e1`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:16:38,155::task::495::TaskManager.Task::(_debug) Task 327fe5a4-9cc8-453f-83b0-b4580d6818fd: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:16:38,156::task::495::TaskManager.Task::(_debug) Task 327fe5a4-9cc8-453f-83b0-b4580d6818fd: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:16:38,157::task::495::TaskManager.Task::(_debug) Task 327fe5a4-9cc8-453f-83b0-b4580d6818fd: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:16:38,157::task::495::TaskManager.Task::(_debug) Task 327fe5a4-9cc8-453f-83b0-b4580d6818fd: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:16:38,158::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:16:38,159::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:16:38,161::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:16:38,161::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:16:38,162::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:16:38,162::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:16:38,163::task::495::TaskManager.Task::(_debug) Task 327fe5a4-9cc8-453f-83b0-b4580d6818fd: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:16:38,164::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:16:38,472::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0744234 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15916072::INFO::2013-11-20 04:16:39,898::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916072::DEBUG::2013-11-20 04:16:39,899::task::495::TaskManager.Task::(_debug) Task c9c4c6ed-0de1-44bc-8c34-101f39f4720c: moving from state init -> state preparing
>Thread-15916072::DEBUG::2013-11-20 04:16:39,900::task::495::TaskManager.Task::(_debug) Task c9c4c6ed-0de1-44bc-8c34-101f39f4720c: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000469923019409', 'lastCheck': 1384920993.04058, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.026957988739', 'lastCheck': 1384920992.4888761, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0277709960938', 'lastCheck': 1384920999.1671381, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000552892684937', 'lastCheck': 1384920997.6049149, 'valid': True, 'code': 0}}
>Thread-15916072::DEBUG::2013-11-20 04:16:39,901::task::495::TaskManager.Task::(_debug) Task c9c4c6ed-0de1-44bc-8c34-101f39f4720c: moving from state preparing -> state finished
>Thread-15916072::DEBUG::2013-11-20 04:16:39,902::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916072::DEBUG::2013-11-20 04:16:39,903::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916072::DEBUG::2013-11-20 04:16:39,903::task::495::TaskManager.Task::(_debug) Task c9c4c6ed-0de1-44bc-8c34-101f39f4720c: ref 0 aborting False
>Thread-15916072::INFO::2013-11-20 04:16:39,904::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000469923019409', 'lastCheck': 1384920993.04058, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.026957988739', 'lastCheck': 1384920992.4888761, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0277709960938', 'lastCheck': 1384920999.1671381, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000552892684937', 'lastCheck': 1384920997.6049149, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:16:40,487::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:16:41,041::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0787633 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 04:16:42,212::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:16:42,213::task::495::TaskManager.Task::(_debug) Task 04e156d5-2c28-4a6c-8965-8e42c45b3e54: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:16:42,214::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c7696bbb-5c00-457c-b9ec-a19baadfb4d5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:16:42,215::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:16:42,216::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:16:42,216::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c7696bbb-5c00-457c-b9ec-a19baadfb4d5`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:16:42,217::task::495::TaskManager.Task::(_debug) Task 04e156d5-2c28-4a6c-8965-8e42c45b3e54: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:16:42,218::task::495::TaskManager.Task::(_debug) Task 04e156d5-2c28-4a6c-8965-8e42c45b3e54: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:16:42,219::task::495::TaskManager.Task::(_debug) Task 04e156d5-2c28-4a6c-8965-8e42c45b3e54: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:16:42,220::task::495::TaskManager.Task::(_debug) Task 04e156d5-2c28-4a6c-8965-8e42c45b3e54: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:16:42,220::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:16:42,221::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:16:42,221::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:16:42,222::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:16:42,223::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:16:42,223::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:16:42,224::task::495::TaskManager.Task::(_debug) Task 04e156d5-2c28-4a6c-8965-8e42c45b3e54: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:16:42,224::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:16:42,225::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:16:42,226::task::495::TaskManager.Task::(_debug) Task 27f91c12-4a5c-4702-8b6b-1ccd34e9347f: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:16:42,227::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`46931140-b22b-4685-ab0e-edd814ac22b5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:16:42,227::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:16:42,228::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:16:42,228::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`46931140-b22b-4685-ab0e-edd814ac22b5`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:16:42,229::task::495::TaskManager.Task::(_debug) Task 27f91c12-4a5c-4702-8b6b-1ccd34e9347f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:16:42,230::task::495::TaskManager.Task::(_debug) Task 27f91c12-4a5c-4702-8b6b-1ccd34e9347f: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:16:42,230::task::495::TaskManager.Task::(_debug) Task 27f91c12-4a5c-4702-8b6b-1ccd34e9347f: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:16:42,231::task::495::TaskManager.Task::(_debug) Task 27f91c12-4a5c-4702-8b6b-1ccd34e9347f: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:16:42,232::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:16:42,232::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:16:42,233::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:16:42,233::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:16:42,234::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:16:42,235::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:16:42,235::task::495::TaskManager.Task::(_debug) Task 27f91c12-4a5c-4702-8b6b-1ccd34e9347f: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:16:42,236::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:16:43,057::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:16:43,593::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0800985 s, 12.8 MB/s\n'; <rc> = 0
>Thread-15916075::DEBUG::2013-11-20 04:16:45,477::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916075::INFO::2013-11-20 04:16:45,479::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916075::DEBUG::2013-11-20 04:16:45,479::task::495::TaskManager.Task::(_debug) Task 19a5e9fa-9dd3-42f7-b9ff-8e5cbdf686f2: moving from state init -> state preparing
>Thread-15916075::DEBUG::2013-11-20 04:16:45,480::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916075::DEBUG::2013-11-20 04:16:45,481::task::495::TaskManager.Task::(_debug) Task 19a5e9fa-9dd3-42f7-b9ff-8e5cbdf686f2: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916075::DEBUG::2013-11-20 04:16:45,482::task::495::TaskManager.Task::(_debug) Task 19a5e9fa-9dd3-42f7-b9ff-8e5cbdf686f2: moving from state preparing -> state finished
>Thread-15916075::DEBUG::2013-11-20 04:16:45,483::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916075::DEBUG::2013-11-20 04:16:45,483::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916075::DEBUG::2013-11-20 04:16:45,484::task::495::TaskManager.Task::(_debug) Task 19a5e9fa-9dd3-42f7-b9ff-8e5cbdf686f2: ref 0 aborting False
>Thread-15916075::INFO::2013-11-20 04:16:45,485::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916076::DEBUG::2013-11-20 04:16:45,489::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916076::INFO::2013-11-20 04:16:45,490::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916076::DEBUG::2013-11-20 04:16:45,491::task::495::TaskManager.Task::(_debug) Task 4fc5dd2e-8b30-41ec-8e96-42bdcc7ee226: moving from state init -> state preparing
>Thread-15916076::DEBUG::2013-11-20 04:16:45,492::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`49cc00d5-3b4b-412b-ae7e-fc82214c69a9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916076::DEBUG::2013-11-20 04:16:45,493::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916076::DEBUG::2013-11-20 04:16:45,494::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916076::DEBUG::2013-11-20 04:16:45,494::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`49cc00d5-3b4b-412b-ae7e-fc82214c69a9`::Granted request
>Thread-15916076::DEBUG::2013-11-20 04:16:45,503::task::495::TaskManager.Task::(_debug) Task 4fc5dd2e-8b30-41ec-8e96-42bdcc7ee226: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916076::DEBUG::2013-11-20 04:16:45,504::task::495::TaskManager.Task::(_debug) Task 4fc5dd2e-8b30-41ec-8e96-42bdcc7ee226: ref 1 aborting False
>Thread-15916076::WARNING::2013-11-20 04:16:45,507::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916076::DEBUG::2013-11-20 04:16:45,508::task::495::TaskManager.Task::(_debug) Task 4fc5dd2e-8b30-41ec-8e96-42bdcc7ee226: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916076::DEBUG::2013-11-20 04:16:45,509::task::495::TaskManager.Task::(_debug) Task 4fc5dd2e-8b30-41ec-8e96-42bdcc7ee226: moving from state preparing -> state finished
>Thread-15916076::DEBUG::2013-11-20 04:16:45,510::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916076::DEBUG::2013-11-20 04:16:45,511::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916076::DEBUG::2013-11-20 04:16:45,511::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916076::DEBUG::2013-11-20 04:16:45,512::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916076::DEBUG::2013-11-20 04:16:45,513::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916076::DEBUG::2013-11-20 04:16:45,514::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916076::DEBUG::2013-11-20 04:16:45,514::task::495::TaskManager.Task::(_debug) Task 4fc5dd2e-8b30-41ec-8e96-42bdcc7ee226: ref 0 aborting False
>Thread-15916076::INFO::2013-11-20 04:16:45,515::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:16:45,606::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 04:16:45,849::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:16:45,851::task::495::TaskManager.Task::(_debug) Task 9e1f23a8-575b-41cf-8962-be13732b4a9c: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:16:45,857::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9ed58927-15ff-41c7-a565-cbc4090a0df1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:16:45,858::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:16:45,860::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:16:45,861::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9ed58927-15ff-41c7-a565-cbc4090a0df1`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:16:45,862::task::495::TaskManager.Task::(_debug) Task 9e1f23a8-575b-41cf-8962-be13732b4a9c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:16:45,863::task::495::TaskManager.Task::(_debug) Task 9e1f23a8-575b-41cf-8962-be13732b4a9c: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:16:45,864::task::495::TaskManager.Task::(_debug) Task 9e1f23a8-575b-41cf-8962-be13732b4a9c: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:16:45,865::task::495::TaskManager.Task::(_debug) Task 9e1f23a8-575b-41cf-8962-be13732b4a9c: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:16:45,865::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:16:45,866::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:16:45,867::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:16:45,868::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:16:45,869::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:16:45,870::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:16:45,871::task::495::TaskManager.Task::(_debug) Task 9e1f23a8-575b-41cf-8962-be13732b4a9c: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:16:45,872::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:16:46,191::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.109857 s, 9.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:16:48,201::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:16:48,773::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.109561 s, 9.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:16:50,788::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:16:51,342::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.076332 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:16:53,359::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:16:53,917::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0780088 s, 13.1 MB/s\n'; <rc> = 0
>Thread-15916080::DEBUG::2013-11-20 04:16:55,913::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916080::INFO::2013-11-20 04:16:55,914::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916080::DEBUG::2013-11-20 04:16:55,915::task::495::TaskManager.Task::(_debug) Task c8fd946f-2833-4574-bb8c-caaeff808bb8: moving from state init -> state preparing
>Thread-15916080::DEBUG::2013-11-20 04:16:55,916::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916080::DEBUG::2013-11-20 04:16:55,916::task::495::TaskManager.Task::(_debug) Task c8fd946f-2833-4574-bb8c-caaeff808bb8: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916080::DEBUG::2013-11-20 04:16:55,917::task::495::TaskManager.Task::(_debug) Task c8fd946f-2833-4574-bb8c-caaeff808bb8: moving from state preparing -> state finished
>Thread-15916080::DEBUG::2013-11-20 04:16:55,918::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916080::DEBUG::2013-11-20 04:16:55,918::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916080::DEBUG::2013-11-20 04:16:55,919::task::495::TaskManager.Task::(_debug) Task c8fd946f-2833-4574-bb8c-caaeff808bb8: ref 0 aborting False
>Thread-15916080::INFO::2013-11-20 04:16:55,920::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916081::DEBUG::2013-11-20 04:16:55,925::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916081::INFO::2013-11-20 04:16:55,926::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916081::DEBUG::2013-11-20 04:16:55,927::task::495::TaskManager.Task::(_debug) Task 52d9218a-af30-45a5-a2f9-d05ca22ab762: moving from state init -> state preparing
>Thread-15916081::DEBUG::2013-11-20 04:16:55,928::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bb25d284-fe0f-4e69-88fd-c3fa44c6cb8a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916081::DEBUG::2013-11-20 04:16:55,929::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916081::DEBUG::2013-11-20 04:16:55,930::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916081::DEBUG::2013-11-20 04:16:55,930::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bb25d284-fe0f-4e69-88fd-c3fa44c6cb8a`::Granted request
>Thread-15916081::DEBUG::2013-11-20 04:16:55,931::task::495::TaskManager.Task::(_debug) Task 52d9218a-af30-45a5-a2f9-d05ca22ab762: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916081::DEBUG::2013-11-20 04:16:55,932::task::495::TaskManager.Task::(_debug) Task 52d9218a-af30-45a5-a2f9-d05ca22ab762: ref 1 aborting False
>Dummy-15912651::DEBUG::2013-11-20 04:16:55,934::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916081::WARNING::2013-11-20 04:16:55,935::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15912392::INFO::2013-11-20 04:16:56,140::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15916081::DEBUG::2013-11-20 04:16:56,141::task::495::TaskManager.Task::(_debug) Task 52d9218a-af30-45a5-a2f9-d05ca22ab762: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15912392::DEBUG::2013-11-20 04:16:56,143::task::495::TaskManager.Task::(_debug) Task e16a471b-c8b7-4590-a077-0c7294ec512d: moving from state init -> state preparing
>Thread-15916081::DEBUG::2013-11-20 04:16:56,145::task::495::TaskManager.Task::(_debug) Task 52d9218a-af30-45a5-a2f9-d05ca22ab762: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:16:56,147::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0b2f836d-e1f4-4ecf-8e3d-e49a6f3b363a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15916081::DEBUG::2013-11-20 04:16:56,148::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:16:56,150::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15916081::DEBUG::2013-11-20 04:16:56,150::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:16:56,152::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15916081::DEBUG::2013-11-20 04:16:56,152::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15912392::DEBUG::2013-11-20 04:16:56,152::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0b2f836d-e1f4-4ecf-8e3d-e49a6f3b363a`::Granted request
>Thread-15916081::DEBUG::2013-11-20 04:16:56,154::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916081::DEBUG::2013-11-20 04:16:56,156::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:16:56,156::task::495::TaskManager.Task::(_debug) Task e16a471b-c8b7-4590-a077-0c7294ec512d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15916081::DEBUG::2013-11-20 04:16:56,157::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:16:56,158::task::495::TaskManager.Task::(_debug) Task e16a471b-c8b7-4590-a077-0c7294ec512d: ref 1 aborting False
>Thread-15916081::DEBUG::2013-11-20 04:16:56,159::task::495::TaskManager.Task::(_debug) Task 52d9218a-af30-45a5-a2f9-d05ca22ab762: ref 0 aborting False
>Thread-15916082::INFO::2013-11-20 04:16:56,162::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15912392::DEBUG::2013-11-20 04:16:56,162::task::495::TaskManager.Task::(_debug) Task e16a471b-c8b7-4590-a077-0c7294ec512d: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15916081::INFO::2013-11-20 04:16:56,163::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916082::DEBUG::2013-11-20 04:16:56,164::task::495::TaskManager.Task::(_debug) Task d533136c-266d-43b4-ad78-25fdb4e609ee: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:16:56,164::task::495::TaskManager.Task::(_debug) Task e16a471b-c8b7-4590-a077-0c7294ec512d: moving from state preparing -> state finished
>Thread-15916082::DEBUG::2013-11-20 04:16:56,167::task::495::TaskManager.Task::(_debug) Task d533136c-266d-43b4-ad78-25fdb4e609ee: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000636100769043', 'lastCheck': 1384921013.5817051, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00137996673584', 'lastCheck': 1384921013.006531, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00284600257874', 'lastCheck': 1384921009.4010091, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000527143478394', 'lastCheck': 1384921007.8405991, 'valid': True, 'code': 0}}
>Thread-15912392::DEBUG::2013-11-20 04:16:56,168::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15916082::DEBUG::2013-11-20 04:16:56,169::task::495::TaskManager.Task::(_debug) Task d533136c-266d-43b4-ad78-25fdb4e609ee: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:16:56,170::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916082::DEBUG::2013-11-20 04:16:56,171::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912392::DEBUG::2013-11-20 04:16:56,171::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15916082::DEBUG::2013-11-20 04:16:56,172::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:16:56,173::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15916082::DEBUG::2013-11-20 04:16:56,173::task::495::TaskManager.Task::(_debug) Task d533136c-266d-43b4-ad78-25fdb4e609ee: ref 0 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:16:56,174::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15916082::INFO::2013-11-20 04:16:56,174::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000636100769043', 'lastCheck': 1384921013.5817051, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00137996673584', 'lastCheck': 1384921013.006531, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00284600257874', 'lastCheck': 1384921009.4010091, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000527143478394', 'lastCheck': 1384921007.8405991, 'valid': True, 'code': 0}}
>Thread-15912392::DEBUG::2013-11-20 04:16:56,175::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:16:56,177::task::495::TaskManager.Task::(_debug) Task e16a471b-c8b7-4590-a077-0c7294ec512d: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:16:56,178::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:16:56,480::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0779509 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:16:58,510::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:16:59,074::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0803089 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:17:01,088::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 04:17:01,354::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:17:01,355::task::495::TaskManager.Task::(_debug) Task d768207e-9154-486f-9c7b-0d6f4009be94: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:17:01,357::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`446decc5-b94b-42df-85cc-68240cba8a51`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:17:01,357::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:17:01,358::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:17:01,358::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`446decc5-b94b-42df-85cc-68240cba8a51`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:17:01,359::task::495::TaskManager.Task::(_debug) Task d768207e-9154-486f-9c7b-0d6f4009be94: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:17:01,359::task::495::TaskManager.Task::(_debug) Task d768207e-9154-486f-9c7b-0d6f4009be94: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:17:01,360::task::495::TaskManager.Task::(_debug) Task d768207e-9154-486f-9c7b-0d6f4009be94: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:17:01,360::task::495::TaskManager.Task::(_debug) Task d768207e-9154-486f-9c7b-0d6f4009be94: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:17:01,361::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:17:01,361::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:17:01,361::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:17:01,362::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:17:01,363::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:17:01,363::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:17:01,364::task::495::TaskManager.Task::(_debug) Task d768207e-9154-486f-9c7b-0d6f4009be94: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:17:01,365::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:17:01,649::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0937034 s, 10.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:17:03,664::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:17:04,239::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0851087 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:17:06,255::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 04:17:06,490::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:17:06,492::task::495::TaskManager.Task::(_debug) Task 74f4f419-f7d6-4914-8100-53dcaacb267e: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:17:06,496::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0be6eef7-58bb-4c35-8074-de8e5cf6737d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:17:06,498::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:17:06,499::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:17:06,500::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0be6eef7-58bb-4c35-8074-de8e5cf6737d`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:17:06,501::task::495::TaskManager.Task::(_debug) Task 74f4f419-f7d6-4914-8100-53dcaacb267e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:17:06,502::task::495::TaskManager.Task::(_debug) Task 74f4f419-f7d6-4914-8100-53dcaacb267e: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:17:06,503::task::495::TaskManager.Task::(_debug) Task 74f4f419-f7d6-4914-8100-53dcaacb267e: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:17:06,503::task::495::TaskManager.Task::(_debug) Task 74f4f419-f7d6-4914-8100-53dcaacb267e: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:17:06,504::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:17:06,505::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:17:06,505::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:17:06,506::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:17:06,507::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:17:06,507::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:17:06,508::task::495::TaskManager.Task::(_debug) Task 74f4f419-f7d6-4914-8100-53dcaacb267e: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:17:06,508::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15916087::DEBUG::2013-11-20 04:17:06,619::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916087::INFO::2013-11-20 04:17:06,620::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916087::DEBUG::2013-11-20 04:17:06,621::task::495::TaskManager.Task::(_debug) Task 48a40b28-6229-44ff-9415-f3bc9c1ded7b: moving from state init -> state preparing
>Thread-15916087::DEBUG::2013-11-20 04:17:06,622::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916087::DEBUG::2013-11-20 04:17:06,622::task::495::TaskManager.Task::(_debug) Task 48a40b28-6229-44ff-9415-f3bc9c1ded7b: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916087::DEBUG::2013-11-20 04:17:06,623::task::495::TaskManager.Task::(_debug) Task 48a40b28-6229-44ff-9415-f3bc9c1ded7b: moving from state preparing -> state finished
>Thread-15916087::DEBUG::2013-11-20 04:17:06,624::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916087::DEBUG::2013-11-20 04:17:06,624::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916087::DEBUG::2013-11-20 04:17:06,625::task::495::TaskManager.Task::(_debug) Task 48a40b28-6229-44ff-9415-f3bc9c1ded7b: ref 0 aborting False
>Thread-15916087::INFO::2013-11-20 04:17:06,626::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916088::DEBUG::2013-11-20 04:17:06,638::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916088::INFO::2013-11-20 04:17:06,639::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916088::DEBUG::2013-11-20 04:17:06,640::task::495::TaskManager.Task::(_debug) Task c799e946-519f-48d4-a8a9-29de70ccabd6: moving from state init -> state preparing
>Thread-15916088::DEBUG::2013-11-20 04:17:06,642::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`adf075ed-0a9e-41b7-9369-d6af9ef5647a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916088::DEBUG::2013-11-20 04:17:06,642::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916088::DEBUG::2013-11-20 04:17:06,643::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916088::DEBUG::2013-11-20 04:17:06,644::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`adf075ed-0a9e-41b7-9369-d6af9ef5647a`::Granted request
>Thread-15916088::DEBUG::2013-11-20 04:17:06,645::task::495::TaskManager.Task::(_debug) Task c799e946-519f-48d4-a8a9-29de70ccabd6: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916088::DEBUG::2013-11-20 04:17:06,646::task::495::TaskManager.Task::(_debug) Task c799e946-519f-48d4-a8a9-29de70ccabd6: ref 1 aborting False
>Thread-15916088::WARNING::2013-11-20 04:17:06,649::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916088::DEBUG::2013-11-20 04:17:06,650::task::495::TaskManager.Task::(_debug) Task c799e946-519f-48d4-a8a9-29de70ccabd6: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916088::DEBUG::2013-11-20 04:17:06,651::task::495::TaskManager.Task::(_debug) Task c799e946-519f-48d4-a8a9-29de70ccabd6: moving from state preparing -> state finished
>Thread-15916088::DEBUG::2013-11-20 04:17:06,652::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916088::DEBUG::2013-11-20 04:17:06,653::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916088::DEBUG::2013-11-20 04:17:06,654::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916088::DEBUG::2013-11-20 04:17:06,654::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916088::DEBUG::2013-11-20 04:17:06,655::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916088::DEBUG::2013-11-20 04:17:06,656::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916088::DEBUG::2013-11-20 04:17:06,657::task::495::TaskManager.Task::(_debug) Task c799e946-519f-48d4-a8a9-29de70ccabd6: ref 0 aborting False
>Thread-15916088::INFO::2013-11-20 04:17:06,658::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:17:06,804::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0670432 s, 15.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:17:08,818::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:17:09,349::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0864506 s, 11.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:17:11,363::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:17:11,863::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0571095 s, 17.9 MB/s\n'; <rc> = 0
>Thread-15916090::INFO::2013-11-20 04:17:12,044::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916090::DEBUG::2013-11-20 04:17:12,045::task::495::TaskManager.Task::(_debug) Task f69b007b-77f2-42dd-9149-54d51db13ef2: moving from state init -> state preparing
>Thread-15916090::DEBUG::2013-11-20 04:17:12,046::task::495::TaskManager.Task::(_debug) Task f69b007b-77f2-42dd-9149-54d51db13ef2: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000442028045654', 'lastCheck': 1384921023.9257569, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0354599952698', 'lastCheck': 1384921023.374738, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0360178947449', 'lastCheck': 1384921030.0498459, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000380992889404', 'lastCheck': 1384921028.4848261, 'valid': True, 'code': 0}}
>Thread-15916090::DEBUG::2013-11-20 04:17:12,047::task::495::TaskManager.Task::(_debug) Task f69b007b-77f2-42dd-9149-54d51db13ef2: moving from state preparing -> state finished
>Thread-15916090::DEBUG::2013-11-20 04:17:12,048::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916090::DEBUG::2013-11-20 04:17:12,048::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916090::DEBUG::2013-11-20 04:17:12,049::task::495::TaskManager.Task::(_debug) Task f69b007b-77f2-42dd-9149-54d51db13ef2: ref 0 aborting False
>Thread-15916090::INFO::2013-11-20 04:17:12,050::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000442028045654', 'lastCheck': 1384921023.9257569, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0354599952698', 'lastCheck': 1384921023.374738, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0360178947449', 'lastCheck': 1384921030.0498459, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000380992889404', 'lastCheck': 1384921028.4848261, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:17:13,875::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 04:17:14,101::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:17:14,107::task::495::TaskManager.Task::(_debug) Task 6c5befa9-a718-4701-bed6-c84e07d95d33: moving from state init -> state preparing
>Thread-15914435::INFO::2013-11-20 04:17:14,110::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:17:14,121::task::495::TaskManager.Task::(_debug) Task 2f2ff4b9-7653-41f2-a28b-cbe39ca8fa26: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:17:14,122::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d08d9641-6066-469d-847b-172044e7b112`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:17:14,125::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0da8b385-3aab-4eab-a422-8ddf6ad6f57c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:17:14,127::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:17:14,128::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:17:14,130::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:17:14,134::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d08d9641-6066-469d-847b-172044e7b112`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:17:14,136::resourceManager::481::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' found in shared state and queue is empty, Joining current shared lock (2 active users)
>Thread-15914519::DEBUG::2013-11-20 04:17:14,136::task::495::TaskManager.Task::(_debug) Task 6c5befa9-a718-4701-bed6-c84e07d95d33: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:17:14,138::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0da8b385-3aab-4eab-a422-8ddf6ad6f57c`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:17:14,138::task::495::TaskManager.Task::(_debug) Task 6c5befa9-a718-4701-bed6-c84e07d95d33: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:17:14,139::task::495::TaskManager.Task::(_debug) Task 2f2ff4b9-7653-41f2-a28b-cbe39ca8fa26: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 04:17:14,146::task::495::TaskManager.Task::(_debug) Task 6c5befa9-a718-4701-bed6-c84e07d95d33: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914435::DEBUG::2013-11-20 04:17:14,146::task::495::TaskManager.Task::(_debug) Task 2f2ff4b9-7653-41f2-a28b-cbe39ca8fa26: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:17:14,147::task::495::TaskManager.Task::(_debug) Task 6c5befa9-a718-4701-bed6-c84e07d95d33: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:17:14,148::task::495::TaskManager.Task::(_debug) Task 2f2ff4b9-7653-41f2-a28b-cbe39ca8fa26: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914519::DEBUG::2013-11-20 04:17:14,149::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:17:14,150::task::495::TaskManager.Task::(_debug) Task 2f2ff4b9-7653-41f2-a28b-cbe39ca8fa26: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:17:14,150::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:17:14,151::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:17:14,152::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:17:14,153::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:17:14,154::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (1 active users)
>Thread-15914435::DEBUG::2013-11-20 04:17:14,155::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 04:17:14,157::task::495::TaskManager.Task::(_debug) Task 6c5befa9-a718-4701-bed6-c84e07d95d33: ref 0 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:17:14,158::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::INFO::2013-11-20 04:17:14,158::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914435::DEBUG::2013-11-20 04:17:14,159::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:17:14,160::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:17:14,161::task::495::TaskManager.Task::(_debug) Task 2f2ff4b9-7653-41f2-a28b-cbe39ca8fa26: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:17:14,162::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:17:14,435::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.107864 s, 9.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:17:16,453::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 04:17:16,687::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:17:16,692::task::495::TaskManager.Task::(_debug) Task 40ea1587-9149-4e0d-b6d2-935d564abaf5: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:17:16,696::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`314ce0cc-9276-468f-af15-be7eb74007c5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:17:16,698::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:17:16,700::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:17:16,701::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`314ce0cc-9276-468f-af15-be7eb74007c5`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:17:16,704::task::495::TaskManager.Task::(_debug) Task 40ea1587-9149-4e0d-b6d2-935d564abaf5: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:17:16,705::task::495::TaskManager.Task::(_debug) Task 40ea1587-9149-4e0d-b6d2-935d564abaf5: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:17:16,706::task::495::TaskManager.Task::(_debug) Task 40ea1587-9149-4e0d-b6d2-935d564abaf5: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 04:17:16,706::task::495::TaskManager.Task::(_debug) Task 40ea1587-9149-4e0d-b6d2-935d564abaf5: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:17:16,707::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:17:16,708::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:17:16,708::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:17:16,709::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:17:16,710::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:17:16,710::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:17:16,711::task::495::TaskManager.Task::(_debug) Task 40ea1587-9149-4e0d-b6d2-935d564abaf5: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:17:16,712::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:17:16,712::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:17:16,713::task::495::TaskManager.Task::(_debug) Task a9f67325-e159-4a18-9f9a-e0c73347f9ca: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:17:16,715::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`39b11fa7-1a82-4769-a724-a72501c53d4c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:17:16,716::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:17:16,717::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:17:16,718::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`39b11fa7-1a82-4769-a724-a72501c53d4c`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:17:16,719::task::495::TaskManager.Task::(_debug) Task a9f67325-e159-4a18-9f9a-e0c73347f9ca: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:17:16,720::task::495::TaskManager.Task::(_debug) Task a9f67325-e159-4a18-9f9a-e0c73347f9ca: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:17:16,721::task::495::TaskManager.Task::(_debug) Task a9f67325-e159-4a18-9f9a-e0c73347f9ca: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:17:16,722::task::495::TaskManager.Task::(_debug) Task a9f67325-e159-4a18-9f9a-e0c73347f9ca: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:17:16,723::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:17:16,723::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:17:16,725::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:17:16,725::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:17:16,726::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:17:16,727::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:17:16,728::task::495::TaskManager.Task::(_debug) Task a9f67325-e159-4a18-9f9a-e0c73347f9ca: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:17:16,729::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:17:16,730::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:17:16,732::task::495::TaskManager.Task::(_debug) Task 9016f8f7-abf4-4ccc-8a09-b92ec0cd998a: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:17:16,738::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2d6b3e5d-b663-471c-a825-2a2b6f5f2608`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:17:16,739::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:17:16,740::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:17:16,740::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2d6b3e5d-b663-471c-a825-2a2b6f5f2608`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:17:16,741::task::495::TaskManager.Task::(_debug) Task 9016f8f7-abf4-4ccc-8a09-b92ec0cd998a: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:17:16,742::task::495::TaskManager.Task::(_debug) Task 9016f8f7-abf4-4ccc-8a09-b92ec0cd998a: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:17:16,743::task::495::TaskManager.Task::(_debug) Task 9016f8f7-abf4-4ccc-8a09-b92ec0cd998a: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:17:16,743::task::495::TaskManager.Task::(_debug) Task 9016f8f7-abf4-4ccc-8a09-b92ec0cd998a: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:17:16,744::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:17:16,744::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:17:16,745::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:17:16,746::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:17:16,746::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:17:16,747::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:17:16,747::task::495::TaskManager.Task::(_debug) Task 9016f8f7-abf4-4ccc-8a09-b92ec0cd998a: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:17:16,748::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:17:16,749::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:17:16,749::task::495::TaskManager.Task::(_debug) Task 36072165-9e4c-400b-94c7-16fe2f10261f: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:17:16,750::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`5e6e283c-4380-4258-a838-a87062e8a34c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:17:16,751::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:17:16,751::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:17:16,752::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`5e6e283c-4380-4258-a838-a87062e8a34c`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:17:16,753::task::495::TaskManager.Task::(_debug) Task 36072165-9e4c-400b-94c7-16fe2f10261f: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:17:16,753::task::495::TaskManager.Task::(_debug) Task 36072165-9e4c-400b-94c7-16fe2f10261f: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:17:16,754::task::495::TaskManager.Task::(_debug) Task 36072165-9e4c-400b-94c7-16fe2f10261f: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:17:16,754::task::495::TaskManager.Task::(_debug) Task 36072165-9e4c-400b-94c7-16fe2f10261f: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:17:16,755::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:17:16,755::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:17:16,756::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:17:16,757::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:17:16,757::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:17:16,758::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:17:16,758::task::495::TaskManager.Task::(_debug) Task 36072165-9e4c-400b-94c7-16fe2f10261f: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:17:16,759::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15916093::DEBUG::2013-11-20 04:17:17,003::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916093::INFO::2013-11-20 04:17:17,005::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916093::DEBUG::2013-11-20 04:17:17,007::task::495::TaskManager.Task::(_debug) Task 3ede2197-b13d-4919-86e6-8d1dda8105d3: moving from state init -> state preparing
>Thread-15916093::DEBUG::2013-11-20 04:17:17,008::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916093::DEBUG::2013-11-20 04:17:17,010::task::495::TaskManager.Task::(_debug) Task 3ede2197-b13d-4919-86e6-8d1dda8105d3: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916093::DEBUG::2013-11-20 04:17:17,011::task::495::TaskManager.Task::(_debug) Task 3ede2197-b13d-4919-86e6-8d1dda8105d3: moving from state preparing -> state finished
>Thread-15916093::DEBUG::2013-11-20 04:17:17,012::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916093::DEBUG::2013-11-20 04:17:17,013::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916093::DEBUG::2013-11-20 04:17:17,014::task::495::TaskManager.Task::(_debug) Task 3ede2197-b13d-4919-86e6-8d1dda8105d3: ref 0 aborting False
>Thread-15916093::INFO::2013-11-20 04:17:17,015::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916094::DEBUG::2013-11-20 04:17:17,022::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916094::INFO::2013-11-20 04:17:17,024::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916094::DEBUG::2013-11-20 04:17:17,025::task::495::TaskManager.Task::(_debug) Task 31d5f69d-b77b-45d6-ae02-f4d0dbceea1d: moving from state init -> state preparing
>Thread-15916094::DEBUG::2013-11-20 04:17:17,028::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`13f06ee6-83df-4461-b29d-4044a84072db`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916094::DEBUG::2013-11-20 04:17:17,029::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916094::DEBUG::2013-11-20 04:17:17,030::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916094::DEBUG::2013-11-20 04:17:17,032::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`13f06ee6-83df-4461-b29d-4044a84072db`::Granted request
>Thread-15916094::DEBUG::2013-11-20 04:17:17,035::task::495::TaskManager.Task::(_debug) Task 31d5f69d-b77b-45d6-ae02-f4d0dbceea1d: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916094::DEBUG::2013-11-20 04:17:17,036::task::495::TaskManager.Task::(_debug) Task 31d5f69d-b77b-45d6-ae02-f4d0dbceea1d: ref 1 aborting False
>Thread-15916094::WARNING::2013-11-20 04:17:17,041::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916094::DEBUG::2013-11-20 04:17:17,042::task::495::TaskManager.Task::(_debug) Task 31d5f69d-b77b-45d6-ae02-f4d0dbceea1d: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916094::DEBUG::2013-11-20 04:17:17,044::task::495::TaskManager.Task::(_debug) Task 31d5f69d-b77b-45d6-ae02-f4d0dbceea1d: moving from state preparing -> state finished
>Thread-15916094::DEBUG::2013-11-20 04:17:17,044::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916094::DEBUG::2013-11-20 04:17:17,045::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916094::DEBUG::2013-11-20 04:17:17,046::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916094::DEBUG::2013-11-20 04:17:17,048::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916094::DEBUG::2013-11-20 04:17:17,048::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916094::DEBUG::2013-11-20 04:17:17,049::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916094::DEBUG::2013-11-20 04:17:17,051::task::495::TaskManager.Task::(_debug) Task 31d5f69d-b77b-45d6-ae02-f4d0dbceea1d: ref 0 aborting False
>Thread-15916094::INFO::2013-11-20 04:17:17,051::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:17:17,062::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0840963 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:17:19,077::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:17:19,647::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0770709 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:17:21,662::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:17:22,222::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0782606 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:17:24,237::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:17:24,792::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.075781 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:17:26,807::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:17:27,339::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0787444 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15916098::DEBUG::2013-11-20 04:17:27,647::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916098::INFO::2013-11-20 04:17:27,649::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916098::DEBUG::2013-11-20 04:17:27,650::task::495::TaskManager.Task::(_debug) Task 3fdb3cb5-405f-48c2-833e-d90735443544: moving from state init -> state preparing
>Thread-15916098::DEBUG::2013-11-20 04:17:27,651::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916098::DEBUG::2013-11-20 04:17:27,651::task::495::TaskManager.Task::(_debug) Task 3fdb3cb5-405f-48c2-833e-d90735443544: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916098::DEBUG::2013-11-20 04:17:27,652::task::495::TaskManager.Task::(_debug) Task 3fdb3cb5-405f-48c2-833e-d90735443544: moving from state preparing -> state finished
>Thread-15916098::DEBUG::2013-11-20 04:17:27,653::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916098::DEBUG::2013-11-20 04:17:27,653::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916098::DEBUG::2013-11-20 04:17:27,654::task::495::TaskManager.Task::(_debug) Task 3fdb3cb5-405f-48c2-833e-d90735443544: ref 0 aborting False
>Thread-15916098::INFO::2013-11-20 04:17:27,654::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916099::DEBUG::2013-11-20 04:17:27,665::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916099::INFO::2013-11-20 04:17:27,667::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916099::DEBUG::2013-11-20 04:17:27,667::task::495::TaskManager.Task::(_debug) Task b715a62c-bb93-4357-9a22-f609c44faf7b: moving from state init -> state preparing
>Thread-15916099::DEBUG::2013-11-20 04:17:27,669::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c06f3714-b682-419a-b141-c038554ab081`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916099::DEBUG::2013-11-20 04:17:27,669::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916099::DEBUG::2013-11-20 04:17:27,670::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916099::DEBUG::2013-11-20 04:17:27,671::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c06f3714-b682-419a-b141-c038554ab081`::Granted request
>Thread-15916099::DEBUG::2013-11-20 04:17:27,672::task::495::TaskManager.Task::(_debug) Task b715a62c-bb93-4357-9a22-f609c44faf7b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916099::DEBUG::2013-11-20 04:17:27,673::task::495::TaskManager.Task::(_debug) Task b715a62c-bb93-4357-9a22-f609c44faf7b: ref 1 aborting False
>Thread-15916099::WARNING::2013-11-20 04:17:27,675::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916099::DEBUG::2013-11-20 04:17:27,676::task::495::TaskManager.Task::(_debug) Task b715a62c-bb93-4357-9a22-f609c44faf7b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916099::DEBUG::2013-11-20 04:17:27,677::task::495::TaskManager.Task::(_debug) Task b715a62c-bb93-4357-9a22-f609c44faf7b: moving from state preparing -> state finished
>Thread-15916099::DEBUG::2013-11-20 04:17:27,677::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916099::DEBUG::2013-11-20 04:17:27,678::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916099::DEBUG::2013-11-20 04:17:27,679::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916099::DEBUG::2013-11-20 04:17:27,679::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916099::DEBUG::2013-11-20 04:17:27,680::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916099::DEBUG::2013-11-20 04:17:27,680::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916099::DEBUG::2013-11-20 04:17:27,681::task::495::TaskManager.Task::(_debug) Task b715a62c-bb93-4357-9a22-f609c44faf7b: ref 0 aborting False
>Thread-15916099::INFO::2013-11-20 04:17:27,682::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916100::INFO::2013-11-20 04:17:27,970::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916100::DEBUG::2013-11-20 04:17:27,971::task::495::TaskManager.Task::(_debug) Task 97502307-a9ce-4a4e-8b7d-fe5057bb6438: moving from state init -> state preparing
>Thread-15916100::DEBUG::2013-11-20 04:17:27,971::task::495::TaskManager.Task::(_debug) Task 97502307-a9ce-4a4e-8b7d-fe5057bb6438: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000427007675171', 'lastCheck': 1384921044.460191, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00134897232056', 'lastCheck': 1384921043.8828471, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00153303146362', 'lastCheck': 1384921040.3536961, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000416994094849', 'lastCheck': 1384921038.693639, 'valid': True, 'code': 0}}
>Thread-15916100::DEBUG::2013-11-20 04:17:27,972::task::495::TaskManager.Task::(_debug) Task 97502307-a9ce-4a4e-8b7d-fe5057bb6438: moving from state preparing -> state finished
>Thread-15916100::DEBUG::2013-11-20 04:17:27,973::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916100::DEBUG::2013-11-20 04:17:27,973::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916100::DEBUG::2013-11-20 04:17:27,974::task::495::TaskManager.Task::(_debug) Task 97502307-a9ce-4a4e-8b7d-fe5057bb6438: ref 0 aborting False
>Thread-15916100::INFO::2013-11-20 04:17:27,974::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000427007675171', 'lastCheck': 1384921044.460191, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00134897232056', 'lastCheck': 1384921043.8828471, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00153303146362', 'lastCheck': 1384921040.3536961, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000416994094849', 'lastCheck': 1384921038.693639, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:17:29,355::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:17:29,914::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0845122 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:17:31,930::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:17:32,153::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:17:32,155::task::495::TaskManager.Task::(_debug) Task f3599c7a-dba4-4ff4-93f0-f0a57cbc4bbb: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:17:32,160::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`27c16e17-8029-41f0-9279-d5eabe3f28a4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:17:32,161::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:17:32,162::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:17:32,163::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`27c16e17-8029-41f0-9279-d5eabe3f28a4`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:17:32,164::task::495::TaskManager.Task::(_debug) Task f3599c7a-dba4-4ff4-93f0-f0a57cbc4bbb: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:17:32,165::task::495::TaskManager.Task::(_debug) Task f3599c7a-dba4-4ff4-93f0-f0a57cbc4bbb: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:17:32,165::task::495::TaskManager.Task::(_debug) Task f3599c7a-dba4-4ff4-93f0-f0a57cbc4bbb: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:17:32,166::task::495::TaskManager.Task::(_debug) Task f3599c7a-dba4-4ff4-93f0-f0a57cbc4bbb: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:17:32,167::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:17:32,167::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:17:32,168::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:17:32,168::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:17:32,169::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:17:32,169::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:17:32,170::task::495::TaskManager.Task::(_debug) Task f3599c7a-dba4-4ff4-93f0-f0a57cbc4bbb: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:17:32,170::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:17:32,427::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0497122 s, 20.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:17:34,438::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915918::INFO::2013-11-20 04:17:34,629::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:17:34,632::task::495::TaskManager.Task::(_debug) Task fc6a4059-3eca-4e3a-879a-ada5720ad389: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:17:34,635::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`00472bcf-4848-44d3-8445-fbbf38796cd2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:17:34,637::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:17:34,639::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:17:34,639::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`00472bcf-4848-44d3-8445-fbbf38796cd2`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:17:34,641::task::495::TaskManager.Task::(_debug) Task fc6a4059-3eca-4e3a-879a-ada5720ad389: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:17:34,642::task::495::TaskManager.Task::(_debug) Task fc6a4059-3eca-4e3a-879a-ada5720ad389: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:17:34,643::task::495::TaskManager.Task::(_debug) Task fc6a4059-3eca-4e3a-879a-ada5720ad389: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:17:34,643::task::495::TaskManager.Task::(_debug) Task fc6a4059-3eca-4e3a-879a-ada5720ad389: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:17:34,644::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:17:34,645::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:17:34,645::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:17:34,646::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:17:34,647::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:17:34,647::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:17:34,648::task::495::TaskManager.Task::(_debug) Task fc6a4059-3eca-4e3a-879a-ada5720ad389: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:17:34,649::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:17:34,942::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0844029 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:17:36,957::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:17:37,506::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0849264 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15916105::DEBUG::2013-11-20 04:17:38,033::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916105::INFO::2013-11-20 04:17:38,035::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916105::DEBUG::2013-11-20 04:17:38,036::task::495::TaskManager.Task::(_debug) Task d048e007-b80a-4fff-bf5d-d108cfbf93b4: moving from state init -> state preparing
>Thread-15916105::DEBUG::2013-11-20 04:17:38,037::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916105::DEBUG::2013-11-20 04:17:38,038::task::495::TaskManager.Task::(_debug) Task d048e007-b80a-4fff-bf5d-d108cfbf93b4: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916105::DEBUG::2013-11-20 04:17:38,038::task::495::TaskManager.Task::(_debug) Task d048e007-b80a-4fff-bf5d-d108cfbf93b4: moving from state preparing -> state finished
>Thread-15916105::DEBUG::2013-11-20 04:17:38,039::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916105::DEBUG::2013-11-20 04:17:38,040::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916105::DEBUG::2013-11-20 04:17:38,041::task::495::TaskManager.Task::(_debug) Task d048e007-b80a-4fff-bf5d-d108cfbf93b4: ref 0 aborting False
>Thread-15916105::INFO::2013-11-20 04:17:38,042::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916106::DEBUG::2013-11-20 04:17:38,047::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916106::INFO::2013-11-20 04:17:38,048::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916106::DEBUG::2013-11-20 04:17:38,049::task::495::TaskManager.Task::(_debug) Task f8f978b9-0a57-411a-8ee7-60852f4f6adc: moving from state init -> state preparing
>Thread-15916106::DEBUG::2013-11-20 04:17:38,050::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4f8c5ac0-14b3-4a63-952d-5e0eb484c459`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916106::DEBUG::2013-11-20 04:17:38,051::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916106::DEBUG::2013-11-20 04:17:38,052::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916106::DEBUG::2013-11-20 04:17:38,052::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4f8c5ac0-14b3-4a63-952d-5e0eb484c459`::Granted request
>Thread-15916106::DEBUG::2013-11-20 04:17:38,053::task::495::TaskManager.Task::(_debug) Task f8f978b9-0a57-411a-8ee7-60852f4f6adc: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916106::DEBUG::2013-11-20 04:17:38,054::task::495::TaskManager.Task::(_debug) Task f8f978b9-0a57-411a-8ee7-60852f4f6adc: ref 1 aborting False
>Thread-15916106::WARNING::2013-11-20 04:17:38,056::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916106::DEBUG::2013-11-20 04:17:38,057::task::495::TaskManager.Task::(_debug) Task f8f978b9-0a57-411a-8ee7-60852f4f6adc: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916106::DEBUG::2013-11-20 04:17:38,058::task::495::TaskManager.Task::(_debug) Task f8f978b9-0a57-411a-8ee7-60852f4f6adc: moving from state preparing -> state finished
>Thread-15916106::DEBUG::2013-11-20 04:17:38,059::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916106::DEBUG::2013-11-20 04:17:38,060::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916106::DEBUG::2013-11-20 04:17:38,060::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916106::DEBUG::2013-11-20 04:17:38,061::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916106::DEBUG::2013-11-20 04:17:38,062::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916106::DEBUG::2013-11-20 04:17:38,063::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916106::DEBUG::2013-11-20 04:17:38,063::task::495::TaskManager.Task::(_debug) Task f8f978b9-0a57-411a-8ee7-60852f4f6adc: ref 0 aborting False
>Thread-15916106::INFO::2013-11-20 04:17:38,064::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:17:39,520::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915863::INFO::2013-11-20 04:17:39,747::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:17:39,749::task::495::TaskManager.Task::(_debug) Task c62a126e-e85a-4834-a4c5-837a9f63f53d: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:17:39,752::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f3b4e24e-4ff8-4ebd-9f2b-4ee0db53e993`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:17:39,753::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:17:39,755::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:17:39,756::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f3b4e24e-4ff8-4ebd-9f2b-4ee0db53e993`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:17:39,758::task::495::TaskManager.Task::(_debug) Task c62a126e-e85a-4834-a4c5-837a9f63f53d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:17:39,758::task::495::TaskManager.Task::(_debug) Task c62a126e-e85a-4834-a4c5-837a9f63f53d: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:17:39,760::task::495::TaskManager.Task::(_debug) Task c62a126e-e85a-4834-a4c5-837a9f63f53d: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:17:39,761::task::495::TaskManager.Task::(_debug) Task c62a126e-e85a-4834-a4c5-837a9f63f53d: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:17:39,762::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:17:39,763::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:17:39,764::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:17:39,765::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:17:39,767::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:17:39,767::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:17:39,768::task::495::TaskManager.Task::(_debug) Task c62a126e-e85a-4834-a4c5-837a9f63f53d: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:17:39,769::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:17:40,071::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0958825 s, 10.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:17:42,082::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:17:42,635::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0888949 s, 11.5 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 04:17:43,823::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:17:43,824::task::495::TaskManager.Task::(_debug) Task 05c83fc7-9432-4e00-860f-ef533c02a7c7: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:17:43,826::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c7cb0ea1-9bee-4029-9304-2fb4f1fa8031`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:17:43,827::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:17:43,829::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:17:43,829::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c7cb0ea1-9bee-4029-9304-2fb4f1fa8031`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:17:43,831::task::495::TaskManager.Task::(_debug) Task 05c83fc7-9432-4e00-860f-ef533c02a7c7: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:17:43,832::task::495::TaskManager.Task::(_debug) Task 05c83fc7-9432-4e00-860f-ef533c02a7c7: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:17:43,833::task::495::TaskManager.Task::(_debug) Task 05c83fc7-9432-4e00-860f-ef533c02a7c7: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:17:43,834::task::495::TaskManager.Task::(_debug) Task 05c83fc7-9432-4e00-860f-ef533c02a7c7: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:17:43,835::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:17:43,835::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:17:43,837::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:17:43,837::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:17:43,838::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:17:43,839::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:17:43,840::task::495::TaskManager.Task::(_debug) Task 05c83fc7-9432-4e00-860f-ef533c02a7c7: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:17:43,841::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:17:43,842::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:17:43,843::task::495::TaskManager.Task::(_debug) Task 65341bfa-a3eb-459b-a851-367bcba24aa7: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:17:43,844::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`844029c8-fde5-41c0-b130-4c65ff4adf22`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:17:43,845::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:17:43,845::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:17:43,846::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`844029c8-fde5-41c0-b130-4c65ff4adf22`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:17:43,847::task::495::TaskManager.Task::(_debug) Task 65341bfa-a3eb-459b-a851-367bcba24aa7: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:17:43,847::task::495::TaskManager.Task::(_debug) Task 65341bfa-a3eb-459b-a851-367bcba24aa7: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:17:43,848::task::495::TaskManager.Task::(_debug) Task 65341bfa-a3eb-459b-a851-367bcba24aa7: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:17:43,848::task::495::TaskManager.Task::(_debug) Task 65341bfa-a3eb-459b-a851-367bcba24aa7: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:17:43,849::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:17:43,850::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:17:43,850::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:17:43,851::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:17:43,851::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:17:43,852::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:17:43,852::task::495::TaskManager.Task::(_debug) Task 65341bfa-a3eb-459b-a851-367bcba24aa7: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:17:43,853::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15916108::INFO::2013-11-20 04:17:43,971::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916108::DEBUG::2013-11-20 04:17:43,972::task::495::TaskManager.Task::(_debug) Task 12afd80e-b545-473c-b1f5-0c17d64a946c: moving from state init -> state preparing
>Thread-15916108::DEBUG::2013-11-20 04:17:43,973::task::495::TaskManager.Task::(_debug) Task 12afd80e-b545-473c-b1f5-0c17d64a946c: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000356197357178', 'lastCheck': 1384921054.632154, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142598152161', 'lastCheck': 1384921054.153038, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00144290924072', 'lastCheck': 1384921060.7353089, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000497817993164', 'lastCheck': 1384921059.1876471, 'valid': True, 'code': 0}}
>Thread-15916108::DEBUG::2013-11-20 04:17:43,974::task::495::TaskManager.Task::(_debug) Task 12afd80e-b545-473c-b1f5-0c17d64a946c: moving from state preparing -> state finished
>Thread-15916108::DEBUG::2013-11-20 04:17:43,975::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916108::DEBUG::2013-11-20 04:17:43,975::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916108::DEBUG::2013-11-20 04:17:43,976::task::495::TaskManager.Task::(_debug) Task 12afd80e-b545-473c-b1f5-0c17d64a946c: ref 0 aborting False
>Thread-15916108::INFO::2013-11-20 04:17:43,977::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000356197357178', 'lastCheck': 1384921054.632154, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00142598152161', 'lastCheck': 1384921054.153038, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00144290924072', 'lastCheck': 1384921060.7353089, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000497817993164', 'lastCheck': 1384921059.1876471, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:17:44,652::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:17:45,215::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0799791 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:17:47,232::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 04:17:47,463::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:17:47,469::task::495::TaskManager.Task::(_debug) Task 22a5e87a-e795-4e95-8d27-e370ce103e60: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:17:47,472::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e29c8e34-f7be-46dd-88e7-826b35740b4b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:17:47,473::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:17:47,475::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:17:47,476::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e29c8e34-f7be-46dd-88e7-826b35740b4b`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:17:47,477::task::495::TaskManager.Task::(_debug) Task 22a5e87a-e795-4e95-8d27-e370ce103e60: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:17:47,477::task::495::TaskManager.Task::(_debug) Task 22a5e87a-e795-4e95-8d27-e370ce103e60: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:17:47,478::task::495::TaskManager.Task::(_debug) Task 22a5e87a-e795-4e95-8d27-e370ce103e60: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:17:47,479::task::495::TaskManager.Task::(_debug) Task 22a5e87a-e795-4e95-8d27-e370ce103e60: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:17:47,479::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:17:47,479::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:17:47,480::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:17:47,481::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:17:47,481::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:17:47,481::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:17:47,482::task::495::TaskManager.Task::(_debug) Task 22a5e87a-e795-4e95-8d27-e370ce103e60: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:17:47,483::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:17:47,775::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0844089 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15916111::DEBUG::2013-11-20 04:17:48,621::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916111::INFO::2013-11-20 04:17:48,622::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916111::DEBUG::2013-11-20 04:17:48,623::task::495::TaskManager.Task::(_debug) Task d9b42ee0-6331-4253-b53e-c2dbd8d0d540: moving from state init -> state preparing
>Thread-15916111::DEBUG::2013-11-20 04:17:48,624::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916111::DEBUG::2013-11-20 04:17:48,625::task::495::TaskManager.Task::(_debug) Task d9b42ee0-6331-4253-b53e-c2dbd8d0d540: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916111::DEBUG::2013-11-20 04:17:48,626::task::495::TaskManager.Task::(_debug) Task d9b42ee0-6331-4253-b53e-c2dbd8d0d540: moving from state preparing -> state finished
>Thread-15916111::DEBUG::2013-11-20 04:17:48,627::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916111::DEBUG::2013-11-20 04:17:48,627::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916111::DEBUG::2013-11-20 04:17:48,628::task::495::TaskManager.Task::(_debug) Task d9b42ee0-6331-4253-b53e-c2dbd8d0d540: ref 0 aborting False
>Thread-15916111::INFO::2013-11-20 04:17:48,629::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916112::DEBUG::2013-11-20 04:17:48,640::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916112::INFO::2013-11-20 04:17:48,641::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916112::DEBUG::2013-11-20 04:17:48,642::task::495::TaskManager.Task::(_debug) Task 01e3c754-304b-449e-957f-f747769e3118: moving from state init -> state preparing
>Thread-15916112::DEBUG::2013-11-20 04:17:48,652::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`52bb9d44-7d21-47de-9b3d-a055f47cf450`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916112::DEBUG::2013-11-20 04:17:48,653::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916112::DEBUG::2013-11-20 04:17:48,654::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916112::DEBUG::2013-11-20 04:17:48,655::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`52bb9d44-7d21-47de-9b3d-a055f47cf450`::Granted request
>Thread-15916112::DEBUG::2013-11-20 04:17:48,656::task::495::TaskManager.Task::(_debug) Task 01e3c754-304b-449e-957f-f747769e3118: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916112::DEBUG::2013-11-20 04:17:48,656::task::495::TaskManager.Task::(_debug) Task 01e3c754-304b-449e-957f-f747769e3118: ref 1 aborting False
>Thread-15916112::WARNING::2013-11-20 04:17:48,659::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916112::DEBUG::2013-11-20 04:17:48,660::task::495::TaskManager.Task::(_debug) Task 01e3c754-304b-449e-957f-f747769e3118: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916112::DEBUG::2013-11-20 04:17:48,661::task::495::TaskManager.Task::(_debug) Task 01e3c754-304b-449e-957f-f747769e3118: moving from state preparing -> state finished
>Thread-15916112::DEBUG::2013-11-20 04:17:48,661::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916112::DEBUG::2013-11-20 04:17:48,662::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916112::DEBUG::2013-11-20 04:17:48,663::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916112::DEBUG::2013-11-20 04:17:48,663::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916112::DEBUG::2013-11-20 04:17:48,664::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916112::DEBUG::2013-11-20 04:17:48,665::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916112::DEBUG::2013-11-20 04:17:48,665::task::495::TaskManager.Task::(_debug) Task 01e3c754-304b-449e-957f-f747769e3118: ref 0 aborting False
>Thread-15916112::INFO::2013-11-20 04:17:48,666::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:17:49,790::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:17:50,273::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0746932 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:17:52,289::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:17:52,870::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0968693 s, 10.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:17:54,886::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-42838::DEBUG::2013-11-20 04:17:55,110::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:17:55,672::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.189173 s, 5.4 MB/s\n'; <rc> = 0
>Thread-42838::DEBUG::2013-11-20 04:17:56,059::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15912392::INFO::2013-11-20 04:17:57,519::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:17:57,520::task::495::TaskManager.Task::(_debug) Task d31179c9-f454-4f70-9351-ad8beb547cac: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:17:57,523::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fdc27ed8-ebe9-45c3-9452-861c1dae28d9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:17:57,525::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:17:57,526::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:17:57,527::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fdc27ed8-ebe9-45c3-9452-861c1dae28d9`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:17:57,528::task::495::TaskManager.Task::(_debug) Task d31179c9-f454-4f70-9351-ad8beb547cac: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:17:57,529::task::495::TaskManager.Task::(_debug) Task d31179c9-f454-4f70-9351-ad8beb547cac: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:17:57,530::task::495::TaskManager.Task::(_debug) Task d31179c9-f454-4f70-9351-ad8beb547cac: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:17:57,531::task::495::TaskManager.Task::(_debug) Task d31179c9-f454-4f70-9351-ad8beb547cac: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:17:57,531::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:17:57,532::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:17:57,533::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:17:57,533::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:17:57,534::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:17:57,535::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:17:57,535::task::495::TaskManager.Task::(_debug) Task d31179c9-f454-4f70-9351-ad8beb547cac: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:17:57,536::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:17:57,692::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:17:58,269::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.114617 s, 8.9 MB/s\n'; <rc> = 0
>Thread-15916116::DEBUG::2013-11-20 04:17:59,112::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916116::INFO::2013-11-20 04:17:59,114::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916116::DEBUG::2013-11-20 04:17:59,114::task::495::TaskManager.Task::(_debug) Task 96448d25-f15b-4027-b8ee-158303c08b54: moving from state init -> state preparing
>Thread-15916116::DEBUG::2013-11-20 04:17:59,116::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916116::DEBUG::2013-11-20 04:17:59,116::task::495::TaskManager.Task::(_debug) Task 96448d25-f15b-4027-b8ee-158303c08b54: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916116::DEBUG::2013-11-20 04:17:59,117::task::495::TaskManager.Task::(_debug) Task 96448d25-f15b-4027-b8ee-158303c08b54: moving from state preparing -> state finished
>Thread-15916116::DEBUG::2013-11-20 04:17:59,117::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916116::DEBUG::2013-11-20 04:17:59,118::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916116::DEBUG::2013-11-20 04:17:59,119::task::495::TaskManager.Task::(_debug) Task 96448d25-f15b-4027-b8ee-158303c08b54: ref 0 aborting False
>Thread-15916116::INFO::2013-11-20 04:17:59,119::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916117::DEBUG::2013-11-20 04:17:59,131::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916117::INFO::2013-11-20 04:17:59,132::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916117::DEBUG::2013-11-20 04:17:59,132::task::495::TaskManager.Task::(_debug) Task 94dd2516-92af-4a2a-a31a-4094a3901748: moving from state init -> state preparing
>Thread-15916117::DEBUG::2013-11-20 04:17:59,134::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1c2371bb-b176-49f5-a39a-27fc975a72c1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916117::DEBUG::2013-11-20 04:17:59,134::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916117::DEBUG::2013-11-20 04:17:59,135::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916117::DEBUG::2013-11-20 04:17:59,136::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1c2371bb-b176-49f5-a39a-27fc975a72c1`::Granted request
>Thread-15916117::DEBUG::2013-11-20 04:17:59,137::task::495::TaskManager.Task::(_debug) Task 94dd2516-92af-4a2a-a31a-4094a3901748: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916117::DEBUG::2013-11-20 04:17:59,138::task::495::TaskManager.Task::(_debug) Task 94dd2516-92af-4a2a-a31a-4094a3901748: ref 1 aborting False
>Thread-15916117::WARNING::2013-11-20 04:17:59,140::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916117::DEBUG::2013-11-20 04:17:59,141::task::495::TaskManager.Task::(_debug) Task 94dd2516-92af-4a2a-a31a-4094a3901748: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916117::DEBUG::2013-11-20 04:17:59,142::task::495::TaskManager.Task::(_debug) Task 94dd2516-92af-4a2a-a31a-4094a3901748: moving from state preparing -> state finished
>Thread-15916117::DEBUG::2013-11-20 04:17:59,143::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916117::DEBUG::2013-11-20 04:17:59,144::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916117::DEBUG::2013-11-20 04:17:59,144::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916117::DEBUG::2013-11-20 04:17:59,145::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916117::DEBUG::2013-11-20 04:17:59,146::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916117::DEBUG::2013-11-20 04:17:59,146::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916117::DEBUG::2013-11-20 04:17:59,147::task::495::TaskManager.Task::(_debug) Task 94dd2516-92af-4a2a-a31a-4094a3901748: ref 0 aborting False
>Thread-15916117::INFO::2013-11-20 04:17:59,148::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916118::INFO::2013-11-20 04:17:59,950::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916118::DEBUG::2013-11-20 04:17:59,951::task::495::TaskManager.Task::(_debug) Task c1d130f6-9eff-4822-9681-9a5bbbfba52d: moving from state init -> state preparing
>Thread-15916118::DEBUG::2013-11-20 04:17:59,952::task::495::TaskManager.Task::(_debug) Task c1d130f6-9eff-4822-9681-9a5bbbfba52d: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000393867492676', 'lastCheck': 1384921076.0651159, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00138878822327', 'lastCheck': 1384921074.505234, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00197410583496', 'lastCheck': 1384921071.001157, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000464916229248', 'lastCheck': 1384921079.5002589, 'valid': True, 'code': 0}}
>Thread-15916118::DEBUG::2013-11-20 04:17:59,953::task::495::TaskManager.Task::(_debug) Task c1d130f6-9eff-4822-9681-9a5bbbfba52d: moving from state preparing -> state finished
>Thread-15916118::DEBUG::2013-11-20 04:17:59,954::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916118::DEBUG::2013-11-20 04:17:59,954::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916118::DEBUG::2013-11-20 04:17:59,955::task::495::TaskManager.Task::(_debug) Task c1d130f6-9eff-4822-9681-9a5bbbfba52d: ref 0 aborting False
>Thread-15916118::INFO::2013-11-20 04:17:59,956::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000393867492676', 'lastCheck': 1384921076.0651159, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00138878822327', 'lastCheck': 1384921074.505234, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00197410583496', 'lastCheck': 1384921071.001157, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000464916229248', 'lastCheck': 1384921079.5002589, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:18:00,286::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:18:00,885::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0906064 s, 11.3 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 04:18:02,570::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:18:02,571::task::495::TaskManager.Task::(_debug) Task b91d3c9e-eef4-4daf-9cee-8473bd71ed48: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:18:02,581::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`225f8d56-cfdd-4ba7-abb5-0df7ef4f22bb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:18:02,582::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:18:02,583::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:18:02,584::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`225f8d56-cfdd-4ba7-abb5-0df7ef4f22bb`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:18:02,585::task::495::TaskManager.Task::(_debug) Task b91d3c9e-eef4-4daf-9cee-8473bd71ed48: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:18:02,586::task::495::TaskManager.Task::(_debug) Task b91d3c9e-eef4-4daf-9cee-8473bd71ed48: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:18:02,587::task::495::TaskManager.Task::(_debug) Task b91d3c9e-eef4-4daf-9cee-8473bd71ed48: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:18:02,587::task::495::TaskManager.Task::(_debug) Task b91d3c9e-eef4-4daf-9cee-8473bd71ed48: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:18:02,588::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:18:02,589::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:18:02,590::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:18:02,590::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:18:02,591::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:18:02,591::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:18:02,592::task::495::TaskManager.Task::(_debug) Task b91d3c9e-eef4-4daf-9cee-8473bd71ed48: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:18:02,593::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:18:02,901::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:18:03,482::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0840136 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:18:05,496::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:18:06,080::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0848212 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15914453::INFO::2013-11-20 04:18:07,733::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:18:07,736::task::495::TaskManager.Task::(_debug) Task 740baf1c-1cb9-442e-9dbe-b024255e5b3a: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:18:07,738::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`438756fb-3336-4c0f-9e36-18cf1fae55f4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:18:07,740::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:18:07,743::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:18:07,744::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`438756fb-3336-4c0f-9e36-18cf1fae55f4`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:18:07,745::task::495::TaskManager.Task::(_debug) Task 740baf1c-1cb9-442e-9dbe-b024255e5b3a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:18:07,746::task::495::TaskManager.Task::(_debug) Task 740baf1c-1cb9-442e-9dbe-b024255e5b3a: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:18:07,747::task::495::TaskManager.Task::(_debug) Task 740baf1c-1cb9-442e-9dbe-b024255e5b3a: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:18:07,748::task::495::TaskManager.Task::(_debug) Task 740baf1c-1cb9-442e-9dbe-b024255e5b3a: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:18:07,749::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:18:07,749::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:18:07,750::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:18:07,751::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:18:07,751::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:18:07,752::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:18:07,753::task::495::TaskManager.Task::(_debug) Task 740baf1c-1cb9-442e-9dbe-b024255e5b3a: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:18:07,754::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:18:08,095::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:18:08,679::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0789117 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15916122::DEBUG::2013-11-20 04:18:09,581::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916122::INFO::2013-11-20 04:18:09,582::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916122::DEBUG::2013-11-20 04:18:09,583::task::495::TaskManager.Task::(_debug) Task bd92bec9-2c03-49fd-90f0-c98fe8d4c7b8: moving from state init -> state preparing
>Thread-15916122::DEBUG::2013-11-20 04:18:09,584::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916122::DEBUG::2013-11-20 04:18:09,585::task::495::TaskManager.Task::(_debug) Task bd92bec9-2c03-49fd-90f0-c98fe8d4c7b8: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916122::DEBUG::2013-11-20 04:18:09,586::task::495::TaskManager.Task::(_debug) Task bd92bec9-2c03-49fd-90f0-c98fe8d4c7b8: moving from state preparing -> state finished
>Thread-15916122::DEBUG::2013-11-20 04:18:09,586::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916122::DEBUG::2013-11-20 04:18:09,587::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916122::DEBUG::2013-11-20 04:18:09,588::task::495::TaskManager.Task::(_debug) Task bd92bec9-2c03-49fd-90f0-c98fe8d4c7b8: ref 0 aborting False
>Thread-15916122::INFO::2013-11-20 04:18:09,589::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916123::DEBUG::2013-11-20 04:18:09,602::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916123::INFO::2013-11-20 04:18:09,603::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916123::DEBUG::2013-11-20 04:18:09,604::task::495::TaskManager.Task::(_debug) Task 42c534e0-a421-4c28-a9ce-4c87206e38d5: moving from state init -> state preparing
>Thread-15916123::DEBUG::2013-11-20 04:18:09,605::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`64f7a6dc-9323-4850-896d-273b4b45c1e8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916123::DEBUG::2013-11-20 04:18:09,606::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916123::DEBUG::2013-11-20 04:18:09,607::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916123::DEBUG::2013-11-20 04:18:09,608::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`64f7a6dc-9323-4850-896d-273b4b45c1e8`::Granted request
>Thread-15916123::DEBUG::2013-11-20 04:18:09,609::task::495::TaskManager.Task::(_debug) Task 42c534e0-a421-4c28-a9ce-4c87206e38d5: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916123::DEBUG::2013-11-20 04:18:09,610::task::495::TaskManager.Task::(_debug) Task 42c534e0-a421-4c28-a9ce-4c87206e38d5: ref 1 aborting False
>Thread-15916123::WARNING::2013-11-20 04:18:09,612::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916123::DEBUG::2013-11-20 04:18:09,613::task::495::TaskManager.Task::(_debug) Task 42c534e0-a421-4c28-a9ce-4c87206e38d5: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916123::DEBUG::2013-11-20 04:18:09,614::task::495::TaskManager.Task::(_debug) Task 42c534e0-a421-4c28-a9ce-4c87206e38d5: moving from state preparing -> state finished
>Thread-15916123::DEBUG::2013-11-20 04:18:09,615::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916123::DEBUG::2013-11-20 04:18:09,615::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916123::DEBUG::2013-11-20 04:18:09,616::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916123::DEBUG::2013-11-20 04:18:09,617::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916123::DEBUG::2013-11-20 04:18:09,617::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916123::DEBUG::2013-11-20 04:18:09,618::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916123::DEBUG::2013-11-20 04:18:09,618::task::495::TaskManager.Task::(_debug) Task 42c534e0-a421-4c28-a9ce-4c87206e38d5: ref 0 aborting False
>Thread-15916123::INFO::2013-11-20 04:18:09,619::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:18:10,695::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:18:11,247::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0913741 s, 11.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:18:13,266::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:18:13,857::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0846459 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15914519::INFO::2013-11-20 04:18:15,171::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:18:15,181::task::495::TaskManager.Task::(_debug) Task b441a772-e57a-4ba9-9fb3-6e19405413cc: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:18:15,182::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`764049ae-8ad2-4e38-89db-17c458c0215d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:18:15,183::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:18:15,184::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:18:15,185::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`764049ae-8ad2-4e38-89db-17c458c0215d`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:18:15,192::task::495::TaskManager.Task::(_debug) Task b441a772-e57a-4ba9-9fb3-6e19405413cc: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 04:18:15,193::task::495::TaskManager.Task::(_debug) Task b441a772-e57a-4ba9-9fb3-6e19405413cc: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:18:15,194::task::495::TaskManager.Task::(_debug) Task b441a772-e57a-4ba9-9fb3-6e19405413cc: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 04:18:15,195::task::495::TaskManager.Task::(_debug) Task b441a772-e57a-4ba9-9fb3-6e19405413cc: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:18:15,195::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:18:15,196::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:18:15,197::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 04:18:15,198::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:18:15,198::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:18:15,199::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:18:15,200::task::495::TaskManager.Task::(_debug) Task b441a772-e57a-4ba9-9fb3-6e19405413cc: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 04:18:15,201::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15916126::INFO::2013-11-20 04:18:15,811::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916126::DEBUG::2013-11-20 04:18:15,812::task::495::TaskManager.Task::(_debug) Task 4366a258-47de-430d-9947-27a9d6acf55f: moving from state init -> state preparing
>Thread-15916126::DEBUG::2013-11-20 04:18:15,821::task::495::TaskManager.Task::(_debug) Task 4366a258-47de-430d-9947-27a9d6acf55f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000355005264282', 'lastCheck': 1384921086.1397951, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00205397605896', 'lastCheck': 1384921094.8979249, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00164818763733', 'lastCheck': 1384921091.3460629, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000385999679565', 'lastCheck': 1384921089.747905, 'valid': True, 'code': 0}}
>Thread-15916126::DEBUG::2013-11-20 04:18:15,822::task::495::TaskManager.Task::(_debug) Task 4366a258-47de-430d-9947-27a9d6acf55f: moving from state preparing -> state finished
>Thread-15916126::DEBUG::2013-11-20 04:18:15,823::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916126::DEBUG::2013-11-20 04:18:15,824::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916126::DEBUG::2013-11-20 04:18:15,825::task::495::TaskManager.Task::(_debug) Task 4366a258-47de-430d-9947-27a9d6acf55f: ref 0 aborting False
>Thread-15916126::INFO::2013-11-20 04:18:15,825::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000355005264282', 'lastCheck': 1384921086.1397951, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00205397605896', 'lastCheck': 1384921094.8979249, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00164818763733', 'lastCheck': 1384921091.3460629, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000385999679565', 'lastCheck': 1384921089.747905, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:18:15,883::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 04:18:16,111::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:18:16,112::task::495::TaskManager.Task::(_debug) Task e94cbef4-c616-452c-a82c-d1f1201ba699: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:18:16,114::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4ad66974-36e0-4f7f-80fe-25349d8157ba`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:18:16,114::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:18:16,115::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:18:16,115::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4ad66974-36e0-4f7f-80fe-25349d8157ba`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:18:16,116::task::495::TaskManager.Task::(_debug) Task e94cbef4-c616-452c-a82c-d1f1201ba699: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:18:16,116::task::495::TaskManager.Task::(_debug) Task e94cbef4-c616-452c-a82c-d1f1201ba699: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:18:16,117::task::495::TaskManager.Task::(_debug) Task e94cbef4-c616-452c-a82c-d1f1201ba699: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:18:16,117::task::495::TaskManager.Task::(_debug) Task e94cbef4-c616-452c-a82c-d1f1201ba699: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:18:16,119::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:18:16,120::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:18:16,120::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:18:16,121::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:18:16,121::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:18:16,122::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:18:16,124::task::495::TaskManager.Task::(_debug) Task e94cbef4-c616-452c-a82c-d1f1201ba699: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:18:16,125::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:18:16,465::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0858235 s, 11.9 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 04:18:17,744::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:18:17,746::task::495::TaskManager.Task::(_debug) Task 4e8d3528-5814-42ec-9729-70a472cb46b0: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:18:17,747::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`dda9396d-7505-4e18-833f-9d9bb72c60ea`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:18:17,748::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:18:17,749::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:18:17,749::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`dda9396d-7505-4e18-833f-9d9bb72c60ea`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:18:17,751::task::495::TaskManager.Task::(_debug) Task 4e8d3528-5814-42ec-9729-70a472cb46b0: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:18:17,752::task::495::TaskManager.Task::(_debug) Task 4e8d3528-5814-42ec-9729-70a472cb46b0: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:18:17,753::task::495::TaskManager.Task::(_debug) Task 4e8d3528-5814-42ec-9729-70a472cb46b0: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 04:18:17,753::task::495::TaskManager.Task::(_debug) Task 4e8d3528-5814-42ec-9729-70a472cb46b0: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:18:17,754::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:18:17,754::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:18:17,755::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:18:17,756::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:18:17,756::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:18:17,757::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:18:17,757::task::495::TaskManager.Task::(_debug) Task 4e8d3528-5814-42ec-9729-70a472cb46b0: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:18:17,758::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:18:17,759::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:18:17,760::task::495::TaskManager.Task::(_debug) Task 1005844b-3163-4efd-865b-7c12680c23f5: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:18:17,761::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a1042ae9-7085-453c-848c-44746d6b8638`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:18:17,761::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:18:17,762::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:18:17,762::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a1042ae9-7085-453c-848c-44746d6b8638`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:18:17,763::task::495::TaskManager.Task::(_debug) Task 1005844b-3163-4efd-865b-7c12680c23f5: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:18:17,764::task::495::TaskManager.Task::(_debug) Task 1005844b-3163-4efd-865b-7c12680c23f5: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:18:17,764::task::495::TaskManager.Task::(_debug) Task 1005844b-3163-4efd-865b-7c12680c23f5: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:18:17,765::task::495::TaskManager.Task::(_debug) Task 1005844b-3163-4efd-865b-7c12680c23f5: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:18:17,766::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:18:17,766::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:18:17,767::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:18:17,768::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:18:17,768::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:18:17,769::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:18:17,769::task::495::TaskManager.Task::(_debug) Task 1005844b-3163-4efd-865b-7c12680c23f5: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:18:17,770::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:18:17,771::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:18:17,771::task::495::TaskManager.Task::(_debug) Task 146d3940-e4f6-4a63-aec9-e8911966cda4: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:18:17,772::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a5e2fa2c-04f7-4557-8c04-57bddb61e4c9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:18:17,773::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:18:17,773::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:18:17,774::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a5e2fa2c-04f7-4557-8c04-57bddb61e4c9`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:18:17,775::task::495::TaskManager.Task::(_debug) Task 146d3940-e4f6-4a63-aec9-e8911966cda4: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:18:17,775::task::495::TaskManager.Task::(_debug) Task 146d3940-e4f6-4a63-aec9-e8911966cda4: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:18:17,776::task::495::TaskManager.Task::(_debug) Task 146d3940-e4f6-4a63-aec9-e8911966cda4: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:18:17,777::task::495::TaskManager.Task::(_debug) Task 146d3940-e4f6-4a63-aec9-e8911966cda4: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:18:17,777::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:18:17,778::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:18:17,779::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:18:17,779::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:18:17,780::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:18:17,780::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:18:17,781::task::495::TaskManager.Task::(_debug) Task 146d3940-e4f6-4a63-aec9-e8911966cda4: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:18:17,782::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:18:17,783::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:18:17,783::task::495::TaskManager.Task::(_debug) Task fba659d4-6f9d-4aca-97c7-fe7d0f29eb10: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:18:17,784::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6fabc9d3-3fdb-4086-abd5-ec5fac4c8d97`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:18:17,785::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:18:17,785::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:18:17,786::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6fabc9d3-3fdb-4086-abd5-ec5fac4c8d97`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:18:17,787::task::495::TaskManager.Task::(_debug) Task fba659d4-6f9d-4aca-97c7-fe7d0f29eb10: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:18:17,787::task::495::TaskManager.Task::(_debug) Task fba659d4-6f9d-4aca-97c7-fe7d0f29eb10: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:18:17,788::task::495::TaskManager.Task::(_debug) Task fba659d4-6f9d-4aca-97c7-fe7d0f29eb10: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:18:17,788::task::495::TaskManager.Task::(_debug) Task fba659d4-6f9d-4aca-97c7-fe7d0f29eb10: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:18:17,789::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:18:17,790::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:18:17,790::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:18:17,791::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:18:17,792::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:18:17,792::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:18:17,793::task::495::TaskManager.Task::(_debug) Task fba659d4-6f9d-4aca-97c7-fe7d0f29eb10: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:18:17,794::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:18:18,533::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:18:19,172::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0812539 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15916129::DEBUG::2013-11-20 04:18:20,094::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916129::INFO::2013-11-20 04:18:20,096::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916129::DEBUG::2013-11-20 04:18:20,097::task::495::TaskManager.Task::(_debug) Task 30d97c77-51bb-413c-918f-0f0dafe12bb2: moving from state init -> state preparing
>Thread-15916129::DEBUG::2013-11-20 04:18:20,098::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916129::DEBUG::2013-11-20 04:18:20,099::task::495::TaskManager.Task::(_debug) Task 30d97c77-51bb-413c-918f-0f0dafe12bb2: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916129::DEBUG::2013-11-20 04:18:20,100::task::495::TaskManager.Task::(_debug) Task 30d97c77-51bb-413c-918f-0f0dafe12bb2: moving from state preparing -> state finished
>Thread-15916129::DEBUG::2013-11-20 04:18:20,101::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916129::DEBUG::2013-11-20 04:18:20,101::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916129::DEBUG::2013-11-20 04:18:20,102::task::495::TaskManager.Task::(_debug) Task 30d97c77-51bb-413c-918f-0f0dafe12bb2: ref 0 aborting False
>Thread-15916129::INFO::2013-11-20 04:18:20,103::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916130::DEBUG::2013-11-20 04:18:20,114::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916130::INFO::2013-11-20 04:18:20,115::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916130::DEBUG::2013-11-20 04:18:20,116::task::495::TaskManager.Task::(_debug) Task c755507b-9569-4151-a96e-1ec1a945e7d1: moving from state init -> state preparing
>Thread-15916130::DEBUG::2013-11-20 04:18:20,117::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`00ae9f41-0424-4320-a085-7f3c9db69425`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916130::DEBUG::2013-11-20 04:18:20,118::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916130::DEBUG::2013-11-20 04:18:20,119::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916130::DEBUG::2013-11-20 04:18:20,120::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`00ae9f41-0424-4320-a085-7f3c9db69425`::Granted request
>Thread-15916130::DEBUG::2013-11-20 04:18:20,121::task::495::TaskManager.Task::(_debug) Task c755507b-9569-4151-a96e-1ec1a945e7d1: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916130::DEBUG::2013-11-20 04:18:20,122::task::495::TaskManager.Task::(_debug) Task c755507b-9569-4151-a96e-1ec1a945e7d1: ref 1 aborting False
>Thread-15916130::WARNING::2013-11-20 04:18:20,125::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916130::DEBUG::2013-11-20 04:18:20,126::task::495::TaskManager.Task::(_debug) Task c755507b-9569-4151-a96e-1ec1a945e7d1: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916130::DEBUG::2013-11-20 04:18:20,127::task::495::TaskManager.Task::(_debug) Task c755507b-9569-4151-a96e-1ec1a945e7d1: moving from state preparing -> state finished
>Thread-15916130::DEBUG::2013-11-20 04:18:20,128::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916130::DEBUG::2013-11-20 04:18:20,129::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916130::DEBUG::2013-11-20 04:18:20,130::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916130::DEBUG::2013-11-20 04:18:20,130::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916130::DEBUG::2013-11-20 04:18:20,131::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916130::DEBUG::2013-11-20 04:18:20,132::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916130::DEBUG::2013-11-20 04:18:20,133::task::495::TaskManager.Task::(_debug) Task c755507b-9569-4151-a96e-1ec1a945e7d1: ref 0 aborting False
>Thread-15916130::INFO::2013-11-20 04:18:20,134::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:18:21,187::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:18:21,711::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0734045 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:18:23,725::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:18:24,325::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0880482 s, 11.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:18:26,335::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:18:26,932::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0901925 s, 11.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:18:28,948::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:18:29,504::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0930085 s, 11.0 MB/s\n'; <rc> = 0
>Thread-15916134::DEBUG::2013-11-20 04:18:30,404::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916134::INFO::2013-11-20 04:18:30,406::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916134::DEBUG::2013-11-20 04:18:30,407::task::495::TaskManager.Task::(_debug) Task d72da27a-7296-4cf8-8c90-4e4a86951ded: moving from state init -> state preparing
>Thread-15916134::DEBUG::2013-11-20 04:18:30,408::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916134::DEBUG::2013-11-20 04:18:30,408::task::495::TaskManager.Task::(_debug) Task d72da27a-7296-4cf8-8c90-4e4a86951ded: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916134::DEBUG::2013-11-20 04:18:30,409::task::495::TaskManager.Task::(_debug) Task d72da27a-7296-4cf8-8c90-4e4a86951ded: moving from state preparing -> state finished
>Thread-15916134::DEBUG::2013-11-20 04:18:30,410::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916134::DEBUG::2013-11-20 04:18:30,411::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916134::DEBUG::2013-11-20 04:18:30,411::task::495::TaskManager.Task::(_debug) Task d72da27a-7296-4cf8-8c90-4e4a86951ded: ref 0 aborting False
>Thread-15916134::INFO::2013-11-20 04:18:30,412::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916135::DEBUG::2013-11-20 04:18:30,418::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916135::INFO::2013-11-20 04:18:30,419::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916135::DEBUG::2013-11-20 04:18:30,420::task::495::TaskManager.Task::(_debug) Task 604494ea-6605-4cd6-ba7f-cdee6027ecac: moving from state init -> state preparing
>Thread-15916135::DEBUG::2013-11-20 04:18:30,422::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6434644c-f76a-4c03-bbab-344bd4830517`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916135::DEBUG::2013-11-20 04:18:30,422::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916135::DEBUG::2013-11-20 04:18:30,423::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916135::DEBUG::2013-11-20 04:18:30,424::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6434644c-f76a-4c03-bbab-344bd4830517`::Granted request
>Thread-15916135::DEBUG::2013-11-20 04:18:30,425::task::495::TaskManager.Task::(_debug) Task 604494ea-6605-4cd6-ba7f-cdee6027ecac: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916135::DEBUG::2013-11-20 04:18:30,426::task::495::TaskManager.Task::(_debug) Task 604494ea-6605-4cd6-ba7f-cdee6027ecac: ref 1 aborting False
>Thread-15916135::WARNING::2013-11-20 04:18:30,428::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916135::DEBUG::2013-11-20 04:18:30,429::task::495::TaskManager.Task::(_debug) Task 604494ea-6605-4cd6-ba7f-cdee6027ecac: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916135::DEBUG::2013-11-20 04:18:30,431::task::495::TaskManager.Task::(_debug) Task 604494ea-6605-4cd6-ba7f-cdee6027ecac: moving from state preparing -> state finished
>Thread-15916135::DEBUG::2013-11-20 04:18:30,431::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916135::DEBUG::2013-11-20 04:18:30,432::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916135::DEBUG::2013-11-20 04:18:30,433::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916135::DEBUG::2013-11-20 04:18:30,433::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916135::DEBUG::2013-11-20 04:18:30,434::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916135::DEBUG::2013-11-20 04:18:30,434::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916135::DEBUG::2013-11-20 04:18:30,435::task::495::TaskManager.Task::(_debug) Task 604494ea-6605-4cd6-ba7f-cdee6027ecac: ref 0 aborting False
>Thread-15916135::INFO::2013-11-20 04:18:30,436::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:18:31,521::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:18:32,065::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0794164 s, 12.9 MB/s\n'; <rc> = 0
>Thread-15916136::INFO::2013-11-20 04:18:32,266::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916136::DEBUG::2013-11-20 04:18:32,267::task::495::TaskManager.Task::(_debug) Task f3c79869-877c-444f-8af5-60b14e7cef96: moving from state init -> state preparing
>Thread-15916136::DEBUG::2013-11-20 04:18:32,267::task::495::TaskManager.Task::(_debug) Task f3c79869-877c-444f-8af5-60b14e7cef96: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000473022460938', 'lastCheck': 1384921106.6005931, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00126099586487', 'lastCheck': 1384921104.962765, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00139403343201', 'lastCheck': 1384921111.7577331, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000499963760376', 'lastCheck': 1384921110.1589439, 'valid': True, 'code': 0}}
>Thread-15916136::DEBUG::2013-11-20 04:18:32,268::task::495::TaskManager.Task::(_debug) Task f3c79869-877c-444f-8af5-60b14e7cef96: moving from state preparing -> state finished
>Thread-15916136::DEBUG::2013-11-20 04:18:32,269::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916136::DEBUG::2013-11-20 04:18:32,269::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916136::DEBUG::2013-11-20 04:18:32,270::task::495::TaskManager.Task::(_debug) Task f3c79869-877c-444f-8af5-60b14e7cef96: ref 0 aborting False
>Thread-15916136::INFO::2013-11-20 04:18:32,271::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000473022460938', 'lastCheck': 1384921106.6005931, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00126099586487', 'lastCheck': 1384921104.962765, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00139403343201', 'lastCheck': 1384921111.7577331, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000499963760376', 'lastCheck': 1384921110.1589439, 'valid': True, 'code': 0}}
>Thread-15913639::INFO::2013-11-20 04:18:32,751::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:18:32,752::task::495::TaskManager.Task::(_debug) Task 02446e15-00d3-4a5f-94e9-e1d57b820bad: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:18:32,754::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6cd75405-6b3e-4c8c-a3b4-f1562322b2d6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:18:32,755::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:18:32,756::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:18:32,756::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6cd75405-6b3e-4c8c-a3b4-f1562322b2d6`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:18:32,757::task::495::TaskManager.Task::(_debug) Task 02446e15-00d3-4a5f-94e9-e1d57b820bad: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:18:32,758::task::495::TaskManager.Task::(_debug) Task 02446e15-00d3-4a5f-94e9-e1d57b820bad: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:18:32,759::task::495::TaskManager.Task::(_debug) Task 02446e15-00d3-4a5f-94e9-e1d57b820bad: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:18:32,760::task::495::TaskManager.Task::(_debug) Task 02446e15-00d3-4a5f-94e9-e1d57b820bad: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:18:32,760::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:18:32,761::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:18:32,762::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:18:32,762::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:18:32,763::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:18:32,763::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:18:32,764::task::495::TaskManager.Task::(_debug) Task 02446e15-00d3-4a5f-94e9-e1d57b820bad: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:18:32,765::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:18:34,079::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:18:34,635::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0930313 s, 11.0 MB/s\n'; <rc> = 0
>Thread-15915918::INFO::2013-11-20 04:18:36,325::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:18:36,326::task::495::TaskManager.Task::(_debug) Task d76226af-6653-49bb-8b50-03c7eb1b222c: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:18:36,328::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`013e4d76-7402-40a1-a49b-a0fe6e1a5fc2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:18:36,329::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:18:36,330::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:18:36,331::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`013e4d76-7402-40a1-a49b-a0fe6e1a5fc2`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:18:36,332::task::495::TaskManager.Task::(_debug) Task d76226af-6653-49bb-8b50-03c7eb1b222c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:18:36,333::task::495::TaskManager.Task::(_debug) Task d76226af-6653-49bb-8b50-03c7eb1b222c: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:18:36,334::task::495::TaskManager.Task::(_debug) Task d76226af-6653-49bb-8b50-03c7eb1b222c: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:18:36,334::task::495::TaskManager.Task::(_debug) Task d76226af-6653-49bb-8b50-03c7eb1b222c: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:18:36,335::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:18:36,336::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:18:36,337::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:18:36,338::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:18:36,338::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:18:36,339::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:18:36,340::task::495::TaskManager.Task::(_debug) Task d76226af-6653-49bb-8b50-03c7eb1b222c: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:18:36,340::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:18:36,656::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:18:37,183::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0787735 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:18:39,197::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:18:39,736::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0829216 s, 12.3 MB/s\n'; <rc> = 0
>Thread-15916140::DEBUG::2013-11-20 04:18:41,091::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916140::INFO::2013-11-20 04:18:41,093::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916140::DEBUG::2013-11-20 04:18:41,093::task::495::TaskManager.Task::(_debug) Task f4cf536e-3d54-4b76-88c0-73c6bb76c408: moving from state init -> state preparing
>Thread-15916140::DEBUG::2013-11-20 04:18:41,094::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916140::DEBUG::2013-11-20 04:18:41,095::task::495::TaskManager.Task::(_debug) Task f4cf536e-3d54-4b76-88c0-73c6bb76c408: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916140::DEBUG::2013-11-20 04:18:41,096::task::495::TaskManager.Task::(_debug) Task f4cf536e-3d54-4b76-88c0-73c6bb76c408: moving from state preparing -> state finished
>Thread-15916140::DEBUG::2013-11-20 04:18:41,096::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916140::DEBUG::2013-11-20 04:18:41,097::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916140::DEBUG::2013-11-20 04:18:41,097::task::495::TaskManager.Task::(_debug) Task f4cf536e-3d54-4b76-88c0-73c6bb76c408: ref 0 aborting False
>Thread-15916140::INFO::2013-11-20 04:18:41,098::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916141::DEBUG::2013-11-20 04:18:41,112::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916141::INFO::2013-11-20 04:18:41,113::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916141::DEBUG::2013-11-20 04:18:41,113::task::495::TaskManager.Task::(_debug) Task 9ba71df2-c67d-4b97-b6af-f99bbb632ad8: moving from state init -> state preparing
>Thread-15916141::DEBUG::2013-11-20 04:18:41,115::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1a322066-b4ea-472a-96f8-eb275b7c715a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916141::DEBUG::2013-11-20 04:18:41,115::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916141::DEBUG::2013-11-20 04:18:41,116::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916141::DEBUG::2013-11-20 04:18:41,117::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1a322066-b4ea-472a-96f8-eb275b7c715a`::Granted request
>Thread-15916141::DEBUG::2013-11-20 04:18:41,124::task::495::TaskManager.Task::(_debug) Task 9ba71df2-c67d-4b97-b6af-f99bbb632ad8: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916141::DEBUG::2013-11-20 04:18:41,125::task::495::TaskManager.Task::(_debug) Task 9ba71df2-c67d-4b97-b6af-f99bbb632ad8: ref 1 aborting False
>Thread-15916141::WARNING::2013-11-20 04:18:41,127::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916141::DEBUG::2013-11-20 04:18:41,128::task::495::TaskManager.Task::(_debug) Task 9ba71df2-c67d-4b97-b6af-f99bbb632ad8: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916141::DEBUG::2013-11-20 04:18:41,129::task::495::TaskManager.Task::(_debug) Task 9ba71df2-c67d-4b97-b6af-f99bbb632ad8: moving from state preparing -> state finished
>Thread-15916141::DEBUG::2013-11-20 04:18:41,130::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916141::DEBUG::2013-11-20 04:18:41,130::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916141::DEBUG::2013-11-20 04:18:41,131::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916141::DEBUG::2013-11-20 04:18:41,132::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916141::DEBUG::2013-11-20 04:18:41,132::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916141::DEBUG::2013-11-20 04:18:41,133::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916141::DEBUG::2013-11-20 04:18:41,133::task::495::TaskManager.Task::(_debug) Task 9ba71df2-c67d-4b97-b6af-f99bbb632ad8: ref 0 aborting False
>Thread-15916141::INFO::2013-11-20 04:18:41,134::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915863::INFO::2013-11-20 04:18:41,169::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:18:41,170::task::495::TaskManager.Task::(_debug) Task 868add3a-b61b-450d-b91f-ce3e8422832f: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:18:41,172::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`99776530-eeeb-4cda-bb3d-454689d81a12`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:18:41,173::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:18:41,174::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:18:41,174::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`99776530-eeeb-4cda-bb3d-454689d81a12`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:18:41,175::task::495::TaskManager.Task::(_debug) Task 868add3a-b61b-450d-b91f-ce3e8422832f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:18:41,176::task::495::TaskManager.Task::(_debug) Task 868add3a-b61b-450d-b91f-ce3e8422832f: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:18:41,177::task::495::TaskManager.Task::(_debug) Task 868add3a-b61b-450d-b91f-ce3e8422832f: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:18:41,178::task::495::TaskManager.Task::(_debug) Task 868add3a-b61b-450d-b91f-ce3e8422832f: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:18:41,182::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:18:41,183::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:18:41,183::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:18:41,184::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:18:41,185::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:18:41,186::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:18:41,187::task::495::TaskManager.Task::(_debug) Task 868add3a-b61b-450d-b91f-ce3e8422832f: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:18:41,188::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:18:41,751::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:18:42,315::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0763436 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:18:44,325::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:18:44,821::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0765139 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 04:18:44,959::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:18:44,960::task::495::TaskManager.Task::(_debug) Task 6797aa7b-bcca-4095-91d6-896ae5a565d2: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:18:44,962::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`275d5c4d-441c-4df0-9b1f-b6fb31a7bddb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:18:44,962::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:18:44,963::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:18:44,964::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`275d5c4d-441c-4df0-9b1f-b6fb31a7bddb`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:18:44,965::task::495::TaskManager.Task::(_debug) Task 6797aa7b-bcca-4095-91d6-896ae5a565d2: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:18:44,965::task::495::TaskManager.Task::(_debug) Task 6797aa7b-bcca-4095-91d6-896ae5a565d2: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:18:44,966::task::495::TaskManager.Task::(_debug) Task 6797aa7b-bcca-4095-91d6-896ae5a565d2: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:18:44,967::task::495::TaskManager.Task::(_debug) Task 6797aa7b-bcca-4095-91d6-896ae5a565d2: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:18:44,967::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:18:44,968::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:18:44,969::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:18:44,969::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:18:44,970::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:18:44,970::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:18:44,971::task::495::TaskManager.Task::(_debug) Task 6797aa7b-bcca-4095-91d6-896ae5a565d2: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:18:44,972::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:18:44,972::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:18:44,973::task::495::TaskManager.Task::(_debug) Task 900b2c84-1304-4dd1-bbb4-64a1db9f305f: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:18:44,974::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`552cd7f4-df07-4f6b-8014-872bacaf89e0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:18:44,975::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:18:44,975::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:18:44,976::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`552cd7f4-df07-4f6b-8014-872bacaf89e0`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:18:44,977::task::495::TaskManager.Task::(_debug) Task 900b2c84-1304-4dd1-bbb4-64a1db9f305f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:18:44,977::task::495::TaskManager.Task::(_debug) Task 900b2c84-1304-4dd1-bbb4-64a1db9f305f: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:18:44,978::task::495::TaskManager.Task::(_debug) Task 900b2c84-1304-4dd1-bbb4-64a1db9f305f: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:18:44,979::task::495::TaskManager.Task::(_debug) Task 900b2c84-1304-4dd1-bbb4-64a1db9f305f: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:18:44,979::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:18:44,980::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:18:44,980::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:18:44,981::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:18:44,981::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:18:44,982::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:18:44,983::task::495::TaskManager.Task::(_debug) Task 900b2c84-1304-4dd1-bbb4-64a1db9f305f: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:18:44,983::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:18:46,836::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:18:47,361::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0757916 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 04:18:47,987::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:18:47,989::task::495::TaskManager.Task::(_debug) Task 8840ac85-fb54-4a28-87d2-23165ff52365: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:18:47,991::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d4e90889-d95e-4594-a6f0-b3dd7bfda85a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:18:47,993::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:18:47,994::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:18:47,994::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d4e90889-d95e-4594-a6f0-b3dd7bfda85a`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:18:47,995::task::495::TaskManager.Task::(_debug) Task 8840ac85-fb54-4a28-87d2-23165ff52365: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:18:47,996::task::495::TaskManager.Task::(_debug) Task 8840ac85-fb54-4a28-87d2-23165ff52365: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:18:47,997::task::495::TaskManager.Task::(_debug) Task 8840ac85-fb54-4a28-87d2-23165ff52365: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:18:47,997::task::495::TaskManager.Task::(_debug) Task 8840ac85-fb54-4a28-87d2-23165ff52365: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:18:47,998::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:18:47,998::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:18:47,999::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:18:48,000::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:18:48,001::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:18:48,001::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:18:48,002::task::495::TaskManager.Task::(_debug) Task 8840ac85-fb54-4a28-87d2-23165ff52365: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:18:48,003::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15916144::INFO::2013-11-20 04:18:48,537::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916144::DEBUG::2013-11-20 04:18:48,538::task::495::TaskManager.Task::(_debug) Task 0c50eea1-e33e-46cd-b5a2-63d877e9efe1: moving from state init -> state preparing
>Thread-15916144::DEBUG::2013-11-20 04:18:48,539::task::495::TaskManager.Task::(_debug) Task 0c50eea1-e33e-46cd-b5a2-63d877e9efe1: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000624179840088', 'lastCheck': 1384921127.0655439, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00149917602539', 'lastCheck': 1384921125.5138869, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00141501426697', 'lastCheck': 1384921121.9892049, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000554084777832', 'lastCheck': 1384921120.3881509, 'valid': True, 'code': 0}}
>Thread-15916144::DEBUG::2013-11-20 04:18:48,540::task::495::TaskManager.Task::(_debug) Task 0c50eea1-e33e-46cd-b5a2-63d877e9efe1: moving from state preparing -> state finished
>Thread-15916144::DEBUG::2013-11-20 04:18:48,541::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916144::DEBUG::2013-11-20 04:18:48,541::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916144::DEBUG::2013-11-20 04:18:48,542::task::495::TaskManager.Task::(_debug) Task 0c50eea1-e33e-46cd-b5a2-63d877e9efe1: ref 0 aborting False
>Thread-15916144::INFO::2013-11-20 04:18:48,543::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000624179840088', 'lastCheck': 1384921127.0655439, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00149917602539', 'lastCheck': 1384921125.5138869, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00141501426697', 'lastCheck': 1384921121.9892049, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000554084777832', 'lastCheck': 1384921120.3881509, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:18:49,375::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:18:49,949::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.088033 s, 11.6 MB/s\n'; <rc> = 0
>Thread-15916146::DEBUG::2013-11-20 04:18:51,693::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916146::INFO::2013-11-20 04:18:51,695::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916146::DEBUG::2013-11-20 04:18:51,696::task::495::TaskManager.Task::(_debug) Task bbaa2bcb-8911-483f-8f70-a3a0263083fc: moving from state init -> state preparing
>Thread-15916146::DEBUG::2013-11-20 04:18:51,697::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916146::DEBUG::2013-11-20 04:18:51,698::task::495::TaskManager.Task::(_debug) Task bbaa2bcb-8911-483f-8f70-a3a0263083fc: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916146::DEBUG::2013-11-20 04:18:51,698::task::495::TaskManager.Task::(_debug) Task bbaa2bcb-8911-483f-8f70-a3a0263083fc: moving from state preparing -> state finished
>Thread-15916146::DEBUG::2013-11-20 04:18:51,699::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916146::DEBUG::2013-11-20 04:18:51,699::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916146::DEBUG::2013-11-20 04:18:51,700::task::495::TaskManager.Task::(_debug) Task bbaa2bcb-8911-483f-8f70-a3a0263083fc: ref 0 aborting False
>Thread-15916146::INFO::2013-11-20 04:18:51,701::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916147::DEBUG::2013-11-20 04:18:51,706::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916147::INFO::2013-11-20 04:18:51,707::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916147::DEBUG::2013-11-20 04:18:51,707::task::495::TaskManager.Task::(_debug) Task 0e7f2194-8800-4f07-bc1f-c79a8eeb2288: moving from state init -> state preparing
>Thread-15916147::DEBUG::2013-11-20 04:18:51,709::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`07a00edf-fe50-4165-b430-f5e5161385ac`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916147::DEBUG::2013-11-20 04:18:51,709::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916147::DEBUG::2013-11-20 04:18:51,710::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916147::DEBUG::2013-11-20 04:18:51,711::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`07a00edf-fe50-4165-b430-f5e5161385ac`::Granted request
>Thread-15916147::DEBUG::2013-11-20 04:18:51,712::task::495::TaskManager.Task::(_debug) Task 0e7f2194-8800-4f07-bc1f-c79a8eeb2288: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916147::DEBUG::2013-11-20 04:18:51,713::task::495::TaskManager.Task::(_debug) Task 0e7f2194-8800-4f07-bc1f-c79a8eeb2288: ref 1 aborting False
>Thread-15916147::WARNING::2013-11-20 04:18:51,715::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916147::DEBUG::2013-11-20 04:18:51,716::task::495::TaskManager.Task::(_debug) Task 0e7f2194-8800-4f07-bc1f-c79a8eeb2288: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916147::DEBUG::2013-11-20 04:18:51,717::task::495::TaskManager.Task::(_debug) Task 0e7f2194-8800-4f07-bc1f-c79a8eeb2288: moving from state preparing -> state finished
>Thread-15916147::DEBUG::2013-11-20 04:18:51,717::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916147::DEBUG::2013-11-20 04:18:51,718::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916147::DEBUG::2013-11-20 04:18:51,719::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916147::DEBUG::2013-11-20 04:18:51,720::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916147::DEBUG::2013-11-20 04:18:51,720::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916147::DEBUG::2013-11-20 04:18:51,721::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916147::DEBUG::2013-11-20 04:18:51,721::task::495::TaskManager.Task::(_debug) Task 0e7f2194-8800-4f07-bc1f-c79a8eeb2288: ref 0 aborting False
>Thread-15916147::INFO::2013-11-20 04:18:51,722::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:18:51,960::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:18:52,533::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0751548 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:18:54,548::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:18:55,095::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0809443 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:18:57,117::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:18:57,697::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0849355 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 04:18:58,209::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:18:58,211::task::495::TaskManager.Task::(_debug) Task 537c3012-9280-4460-83c1-98dca1bae336: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:18:58,214::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`85d21cc9-0ff6-4a86-8a46-b72597ee6f9d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:18:58,216::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:18:58,217::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:18:58,217::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`85d21cc9-0ff6-4a86-8a46-b72597ee6f9d`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:18:58,218::task::495::TaskManager.Task::(_debug) Task 537c3012-9280-4460-83c1-98dca1bae336: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:18:58,219::task::495::TaskManager.Task::(_debug) Task 537c3012-9280-4460-83c1-98dca1bae336: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:18:58,220::task::495::TaskManager.Task::(_debug) Task 537c3012-9280-4460-83c1-98dca1bae336: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:18:58,220::task::495::TaskManager.Task::(_debug) Task 537c3012-9280-4460-83c1-98dca1bae336: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:18:58,221::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:18:58,221::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:18:58,222::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:18:58,223::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:18:58,223::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:18:58,224::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:18:58,224::task::495::TaskManager.Task::(_debug) Task 537c3012-9280-4460-83c1-98dca1bae336: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:18:58,225::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:18:59,708::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:19:00,250::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.075753 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:19:02,266::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916152::DEBUG::2013-11-20 04:19:02,478::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916152::INFO::2013-11-20 04:19:02,482::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916152::DEBUG::2013-11-20 04:19:02,483::task::495::TaskManager.Task::(_debug) Task c1f904ce-c7fa-4033-9d1f-d00346f64180: moving from state init -> state preparing
>Thread-15916152::DEBUG::2013-11-20 04:19:02,484::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916152::DEBUG::2013-11-20 04:19:02,485::task::495::TaskManager.Task::(_debug) Task c1f904ce-c7fa-4033-9d1f-d00346f64180: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916152::DEBUG::2013-11-20 04:19:02,486::task::495::TaskManager.Task::(_debug) Task c1f904ce-c7fa-4033-9d1f-d00346f64180: moving from state preparing -> state finished
>Thread-15916152::DEBUG::2013-11-20 04:19:02,486::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916152::DEBUG::2013-11-20 04:19:02,487::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916152::DEBUG::2013-11-20 04:19:02,488::task::495::TaskManager.Task::(_debug) Task c1f904ce-c7fa-4033-9d1f-d00346f64180: ref 0 aborting False
>Thread-15916152::INFO::2013-11-20 04:19:02,488::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916153::DEBUG::2013-11-20 04:19:02,493::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916153::INFO::2013-11-20 04:19:02,494::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916153::DEBUG::2013-11-20 04:19:02,495::task::495::TaskManager.Task::(_debug) Task 10dbb5b8-ffde-4015-8b0b-138336110431: moving from state init -> state preparing
>Thread-15916153::DEBUG::2013-11-20 04:19:02,496::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6dec0e7f-9f97-44fb-9717-aa7f1e6a9f37`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916153::DEBUG::2013-11-20 04:19:02,497::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916153::DEBUG::2013-11-20 04:19:02,498::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916153::DEBUG::2013-11-20 04:19:02,499::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6dec0e7f-9f97-44fb-9717-aa7f1e6a9f37`::Granted request
>Thread-15916153::DEBUG::2013-11-20 04:19:02,500::task::495::TaskManager.Task::(_debug) Task 10dbb5b8-ffde-4015-8b0b-138336110431: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916153::DEBUG::2013-11-20 04:19:02,501::task::495::TaskManager.Task::(_debug) Task 10dbb5b8-ffde-4015-8b0b-138336110431: ref 1 aborting False
>Thread-15916153::WARNING::2013-11-20 04:19:02,503::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916153::DEBUG::2013-11-20 04:19:02,504::task::495::TaskManager.Task::(_debug) Task 10dbb5b8-ffde-4015-8b0b-138336110431: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916153::DEBUG::2013-11-20 04:19:02,505::task::495::TaskManager.Task::(_debug) Task 10dbb5b8-ffde-4015-8b0b-138336110431: moving from state preparing -> state finished
>Thread-15916153::DEBUG::2013-11-20 04:19:02,506::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916153::DEBUG::2013-11-20 04:19:02,507::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916153::DEBUG::2013-11-20 04:19:02,508::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916153::DEBUG::2013-11-20 04:19:02,508::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916153::DEBUG::2013-11-20 04:19:02,509::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916153::DEBUG::2013-11-20 04:19:02,510::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916153::DEBUG::2013-11-20 04:19:02,510::task::495::TaskManager.Task::(_debug) Task 10dbb5b8-ffde-4015-8b0b-138336110431: ref 0 aborting False
>Thread-15916153::INFO::2013-11-20 04:19:02,511::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:19:02,775::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0613666 s, 16.7 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 04:19:03,383::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:19:03,384::task::495::TaskManager.Task::(_debug) Task 0eb8ac36-da47-4d73-a7bb-54a9062ff1e8: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:19:03,385::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ca10c91e-c35e-4846-b7de-1b0f0c7673a2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:19:03,385::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:19:03,385::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:19:03,386::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ca10c91e-c35e-4846-b7de-1b0f0c7673a2`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:19:03,386::task::495::TaskManager.Task::(_debug) Task 0eb8ac36-da47-4d73-a7bb-54a9062ff1e8: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:19:03,387::task::495::TaskManager.Task::(_debug) Task 0eb8ac36-da47-4d73-a7bb-54a9062ff1e8: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:19:03,387::task::495::TaskManager.Task::(_debug) Task 0eb8ac36-da47-4d73-a7bb-54a9062ff1e8: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:19:03,388::task::495::TaskManager.Task::(_debug) Task 0eb8ac36-da47-4d73-a7bb-54a9062ff1e8: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:19:03,388::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:19:03,388::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:19:03,389::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:19:03,389::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:19:03,389::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:19:03,390::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:19:03,390::task::495::TaskManager.Task::(_debug) Task 0eb8ac36-da47-4d73-a7bb-54a9062ff1e8: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:19:03,391::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15916154::INFO::2013-11-20 04:19:04,493::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916154::DEBUG::2013-11-20 04:19:04,494::task::495::TaskManager.Task::(_debug) Task 53a80926-ff54-431a-9b48-13691e70513f: moving from state init -> state preparing
>Thread-15916154::DEBUG::2013-11-20 04:19:04,494::task::495::TaskManager.Task::(_debug) Task 53a80926-ff54-431a-9b48-13691e70513f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00055193901062', 'lastCheck': 1384921137.3526731, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00149488449097', 'lastCheck': 1384921135.7564261, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0015549659729', 'lastCheck': 1384921142.484185, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000483989715576', 'lastCheck': 1384921140.923511, 'valid': True, 'code': 0}}
>Thread-15916154::DEBUG::2013-11-20 04:19:04,495::task::495::TaskManager.Task::(_debug) Task 53a80926-ff54-431a-9b48-13691e70513f: moving from state preparing -> state finished
>Thread-15916154::DEBUG::2013-11-20 04:19:04,495::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916154::DEBUG::2013-11-20 04:19:04,495::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916154::DEBUG::2013-11-20 04:19:04,496::task::495::TaskManager.Task::(_debug) Task 53a80926-ff54-431a-9b48-13691e70513f: ref 0 aborting False
>Thread-15916154::INFO::2013-11-20 04:19:04,496::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00055193901062', 'lastCheck': 1384921137.3526731, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00149488449097', 'lastCheck': 1384921135.7564261, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0015549659729', 'lastCheck': 1384921142.484185, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000483989715576', 'lastCheck': 1384921140.923511, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:19:04,788::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:19:05,329::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0862005 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:19:07,344::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:19:07,881::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.076381 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15914453::INFO::2013-11-20 04:19:08,224::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:19:08,226::task::495::TaskManager.Task::(_debug) Task 426bd5c6-8008-42f8-a973-2c197d5733d2: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:19:08,229::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`689d29cf-524a-4485-ad05-483c88713eee`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:19:08,230::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:19:08,232::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:19:08,233::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`689d29cf-524a-4485-ad05-483c88713eee`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:19:08,234::task::495::TaskManager.Task::(_debug) Task 426bd5c6-8008-42f8-a973-2c197d5733d2: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:19:08,234::task::495::TaskManager.Task::(_debug) Task 426bd5c6-8008-42f8-a973-2c197d5733d2: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:19:08,235::task::495::TaskManager.Task::(_debug) Task 426bd5c6-8008-42f8-a973-2c197d5733d2: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:19:08,236::task::495::TaskManager.Task::(_debug) Task 426bd5c6-8008-42f8-a973-2c197d5733d2: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:19:08,236::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:19:08,237::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:19:08,238::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:19:08,238::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:19:08,239::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:19:08,239::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:19:08,240::task::495::TaskManager.Task::(_debug) Task 426bd5c6-8008-42f8-a973-2c197d5733d2: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:19:08,241::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:19:09,898::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:19:10,459::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.103679 s, 9.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:19:12,475::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916158::DEBUG::2013-11-20 04:19:12,798::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916158::INFO::2013-11-20 04:19:12,800::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916158::DEBUG::2013-11-20 04:19:12,801::task::495::TaskManager.Task::(_debug) Task 04742772-0403-4d9d-bd3a-7bb40898f53a: moving from state init -> state preparing
>Thread-15916158::DEBUG::2013-11-20 04:19:12,802::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916158::DEBUG::2013-11-20 04:19:12,803::task::495::TaskManager.Task::(_debug) Task 04742772-0403-4d9d-bd3a-7bb40898f53a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916158::DEBUG::2013-11-20 04:19:12,803::task::495::TaskManager.Task::(_debug) Task 04742772-0403-4d9d-bd3a-7bb40898f53a: moving from state preparing -> state finished
>Thread-15916158::DEBUG::2013-11-20 04:19:12,804::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916158::DEBUG::2013-11-20 04:19:12,804::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916158::DEBUG::2013-11-20 04:19:12,805::task::495::TaskManager.Task::(_debug) Task 04742772-0403-4d9d-bd3a-7bb40898f53a: ref 0 aborting False
>Thread-15916158::INFO::2013-11-20 04:19:12,806::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916159::DEBUG::2013-11-20 04:19:12,811::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916159::INFO::2013-11-20 04:19:12,812::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916159::DEBUG::2013-11-20 04:19:12,812::task::495::TaskManager.Task::(_debug) Task 8cb3128b-f3b3-4c42-8bc9-3637fc0efdb4: moving from state init -> state preparing
>Thread-15916159::DEBUG::2013-11-20 04:19:12,814::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d1a3ac60-1a79-4817-a937-0f54f2886271`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916159::DEBUG::2013-11-20 04:19:12,814::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916159::DEBUG::2013-11-20 04:19:12,815::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916159::DEBUG::2013-11-20 04:19:12,816::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d1a3ac60-1a79-4817-a937-0f54f2886271`::Granted request
>Thread-15916159::DEBUG::2013-11-20 04:19:12,817::task::495::TaskManager.Task::(_debug) Task 8cb3128b-f3b3-4c42-8bc9-3637fc0efdb4: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916159::DEBUG::2013-11-20 04:19:12,817::task::495::TaskManager.Task::(_debug) Task 8cb3128b-f3b3-4c42-8bc9-3637fc0efdb4: ref 1 aborting False
>Thread-15916159::WARNING::2013-11-20 04:19:12,820::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916159::DEBUG::2013-11-20 04:19:12,821::task::495::TaskManager.Task::(_debug) Task 8cb3128b-f3b3-4c42-8bc9-3637fc0efdb4: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916159::DEBUG::2013-11-20 04:19:12,822::task::495::TaskManager.Task::(_debug) Task 8cb3128b-f3b3-4c42-8bc9-3637fc0efdb4: moving from state preparing -> state finished
>Thread-15916159::DEBUG::2013-11-20 04:19:12,823::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916159::DEBUG::2013-11-20 04:19:12,824::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916159::DEBUG::2013-11-20 04:19:12,825::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916159::DEBUG::2013-11-20 04:19:12,825::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916159::DEBUG::2013-11-20 04:19:12,826::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916159::DEBUG::2013-11-20 04:19:12,827::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916159::DEBUG::2013-11-20 04:19:12,828::task::495::TaskManager.Task::(_debug) Task 8cb3128b-f3b3-4c42-8bc9-3637fc0efdb4: ref 0 aborting False
>Thread-15916159::INFO::2013-11-20 04:19:12,829::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:19:13,012::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0741325 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:19:15,027::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:19:15,546::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0789193 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15914519::INFO::2013-11-20 04:19:15,655::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:19:15,656::task::495::TaskManager.Task::(_debug) Task 469dc3b5-fba7-4623-a3ba-3f805976cc98: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:19:15,658::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fc7a2a28-2982-4178-a032-62a197dc2f1b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:19:15,659::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:19:15,660::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:19:15,661::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fc7a2a28-2982-4178-a032-62a197dc2f1b`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:19:15,668::task::495::TaskManager.Task::(_debug) Task 469dc3b5-fba7-4623-a3ba-3f805976cc98: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 04:19:15,669::task::495::TaskManager.Task::(_debug) Task 469dc3b5-fba7-4623-a3ba-3f805976cc98: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:19:15,670::task::495::TaskManager.Task::(_debug) Task 469dc3b5-fba7-4623-a3ba-3f805976cc98: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 04:19:15,671::task::495::TaskManager.Task::(_debug) Task 469dc3b5-fba7-4623-a3ba-3f805976cc98: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:19:15,672::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:19:15,672::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:19:15,673::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 04:19:15,674::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:19:15,674::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:19:15,675::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:19:15,676::task::495::TaskManager.Task::(_debug) Task 469dc3b5-fba7-4623-a3ba-3f805976cc98: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 04:19:15,677::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 04:19:17,561::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 04:19:17,786::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:19:17,788::task::495::TaskManager.Task::(_debug) Task 60c0d194-bd10-4bae-b056-c15794c0fde5: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:19:17,790::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a8062b87-cb7d-49fc-a842-27e708133e8f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:19:17,791::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:19:17,792::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:19:17,792::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a8062b87-cb7d-49fc-a842-27e708133e8f`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:19:17,794::task::495::TaskManager.Task::(_debug) Task 60c0d194-bd10-4bae-b056-c15794c0fde5: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:19:17,794::task::495::TaskManager.Task::(_debug) Task 60c0d194-bd10-4bae-b056-c15794c0fde5: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:19:17,795::task::495::TaskManager.Task::(_debug) Task 60c0d194-bd10-4bae-b056-c15794c0fde5: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:19:17,795::task::495::TaskManager.Task::(_debug) Task 60c0d194-bd10-4bae-b056-c15794c0fde5: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:19:17,796::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:19:17,797::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:19:17,798::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:19:17,799::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:19:17,800::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:19:17,801::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:19:17,802::task::495::TaskManager.Task::(_debug) Task 60c0d194-bd10-4bae-b056-c15794c0fde5: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:19:17,803::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:19:18,083::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0730138 s, 14.0 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 04:19:18,238::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:19:18,240::task::495::TaskManager.Task::(_debug) Task 1bea437e-bc8f-47c6-93f7-c83556d4d4e4: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:19:18,242::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8a96830a-53df-4b44-bfd9-38dc1bdd489f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:19:18,242::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:19:18,244::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:19:18,244::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8a96830a-53df-4b44-bfd9-38dc1bdd489f`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:19:18,246::task::495::TaskManager.Task::(_debug) Task 1bea437e-bc8f-47c6-93f7-c83556d4d4e4: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:19:18,246::task::495::TaskManager.Task::(_debug) Task 1bea437e-bc8f-47c6-93f7-c83556d4d4e4: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:19:18,247::task::495::TaskManager.Task::(_debug) Task 1bea437e-bc8f-47c6-93f7-c83556d4d4e4: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 04:19:18,248::task::495::TaskManager.Task::(_debug) Task 1bea437e-bc8f-47c6-93f7-c83556d4d4e4: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:19:18,249::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:19:18,249::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:19:18,250::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:19:18,251::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:19:18,252::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:19:18,252::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:19:18,253::task::495::TaskManager.Task::(_debug) Task 1bea437e-bc8f-47c6-93f7-c83556d4d4e4: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:19:18,254::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:19:18,255::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:19:18,255::task::495::TaskManager.Task::(_debug) Task aeefa402-3840-492a-b503-b01de24434d7: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:19:18,257::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2c3fad7d-eeff-4c46-ab7b-e82debbec10a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:19:18,257::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:19:18,258::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:19:18,259::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2c3fad7d-eeff-4c46-ab7b-e82debbec10a`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:19:18,260::task::495::TaskManager.Task::(_debug) Task aeefa402-3840-492a-b503-b01de24434d7: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:19:18,261::task::495::TaskManager.Task::(_debug) Task aeefa402-3840-492a-b503-b01de24434d7: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:19:18,262::task::495::TaskManager.Task::(_debug) Task aeefa402-3840-492a-b503-b01de24434d7: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:19:18,262::task::495::TaskManager.Task::(_debug) Task aeefa402-3840-492a-b503-b01de24434d7: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:19:18,263::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:19:18,264::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:19:18,264::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:19:18,265::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:19:18,266::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:19:18,266::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:19:18,268::task::495::TaskManager.Task::(_debug) Task aeefa402-3840-492a-b503-b01de24434d7: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:19:18,268::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:19:18,269::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:19:18,270::task::495::TaskManager.Task::(_debug) Task d9fd99b4-9a1e-4a1f-85ff-017f1ac43aa1: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:19:18,271::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`89bc5493-6e95-489c-8b01-084644634c70`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:19:18,272::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:19:18,273::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:19:18,273::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`89bc5493-6e95-489c-8b01-084644634c70`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:19:18,274::task::495::TaskManager.Task::(_debug) Task d9fd99b4-9a1e-4a1f-85ff-017f1ac43aa1: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:19:18,276::task::495::TaskManager.Task::(_debug) Task d9fd99b4-9a1e-4a1f-85ff-017f1ac43aa1: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:19:18,277::task::495::TaskManager.Task::(_debug) Task d9fd99b4-9a1e-4a1f-85ff-017f1ac43aa1: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:19:18,278::task::495::TaskManager.Task::(_debug) Task d9fd99b4-9a1e-4a1f-85ff-017f1ac43aa1: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:19:18,279::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:19:18,280::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:19:18,281::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:19:18,282::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:19:18,282::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:19:18,283::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:19:18,284::task::495::TaskManager.Task::(_debug) Task d9fd99b4-9a1e-4a1f-85ff-017f1ac43aa1: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:19:18,284::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:19:18,285::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:19:18,286::task::495::TaskManager.Task::(_debug) Task b67f203c-5f2b-4d62-8b96-836ad8001763: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:19:18,287::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`7002fd62-8919-453f-8b89-92a4aac774f2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:19:18,288::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:19:18,289::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:19:18,290::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`7002fd62-8919-453f-8b89-92a4aac774f2`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:19:18,291::task::495::TaskManager.Task::(_debug) Task b67f203c-5f2b-4d62-8b96-836ad8001763: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:19:18,292::task::495::TaskManager.Task::(_debug) Task b67f203c-5f2b-4d62-8b96-836ad8001763: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:19:18,292::task::495::TaskManager.Task::(_debug) Task b67f203c-5f2b-4d62-8b96-836ad8001763: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:19:18,293::task::495::TaskManager.Task::(_debug) Task b67f203c-5f2b-4d62-8b96-836ad8001763: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:19:18,294::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:19:18,294::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:19:18,295::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:19:18,296::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:19:18,296::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:19:18,297::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:19:18,298::task::495::TaskManager.Task::(_debug) Task b67f203c-5f2b-4d62-8b96-836ad8001763: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:19:18,298::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:19:20,097::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916162::INFO::2013-11-20 04:19:20,494::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916162::DEBUG::2013-11-20 04:19:20,496::task::495::TaskManager.Task::(_debug) Task 474bf0eb-699f-4646-9273-486c378c2983: moving from state init -> state preparing
>Thread-15916162::DEBUG::2013-11-20 04:19:20,497::task::495::TaskManager.Task::(_debug) Task 474bf0eb-699f-4646-9273-486c378c2983: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00110197067261', 'lastCheck': 1384921157.7888579, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00188612937927', 'lastCheck': 1384921156.2282679, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00147604942322', 'lastCheck': 1384921152.7033269, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000684022903442', 'lastCheck': 1384921151.0990901, 'valid': True, 'code': 0}}
>Thread-15916162::DEBUG::2013-11-20 04:19:20,498::task::495::TaskManager.Task::(_debug) Task 474bf0eb-699f-4646-9273-486c378c2983: moving from state preparing -> state finished
>Thread-15916162::DEBUG::2013-11-20 04:19:20,499::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916162::DEBUG::2013-11-20 04:19:20,499::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916162::DEBUG::2013-11-20 04:19:20,500::task::495::TaskManager.Task::(_debug) Task 474bf0eb-699f-4646-9273-486c378c2983: ref 0 aborting False
>Thread-15916162::INFO::2013-11-20 04:19:20,501::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00110197067261', 'lastCheck': 1384921157.7888579, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00188612937927', 'lastCheck': 1384921156.2282679, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00147604942322', 'lastCheck': 1384921152.7033269, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000684022903442', 'lastCheck': 1384921151.0990901, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:19:20,733::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.132981 s, 7.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:19:22,746::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916164::DEBUG::2013-11-20 04:19:23,275::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916164::INFO::2013-11-20 04:19:23,280::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916164::DEBUG::2013-11-20 04:19:23,284::task::495::TaskManager.Task::(_debug) Task 4cbbd526-44aa-4a0f-b4f9-1ff0426f228a: moving from state init -> state preparing
>Thread-15916164::DEBUG::2013-11-20 04:19:23,286::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916164::DEBUG::2013-11-20 04:19:23,286::task::495::TaskManager.Task::(_debug) Task 4cbbd526-44aa-4a0f-b4f9-1ff0426f228a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916164::DEBUG::2013-11-20 04:19:23,288::task::495::TaskManager.Task::(_debug) Task 4cbbd526-44aa-4a0f-b4f9-1ff0426f228a: moving from state preparing -> state finished
>Thread-15916164::DEBUG::2013-11-20 04:19:23,289::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916164::DEBUG::2013-11-20 04:19:23,291::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916164::DEBUG::2013-11-20 04:19:23,292::task::495::TaskManager.Task::(_debug) Task 4cbbd526-44aa-4a0f-b4f9-1ff0426f228a: ref 0 aborting False
>Thread-15916164::INFO::2013-11-20 04:19:23,294::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916165::DEBUG::2013-11-20 04:19:23,303::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916165::INFO::2013-11-20 04:19:23,307::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916165::DEBUG::2013-11-20 04:19:23,308::task::495::TaskManager.Task::(_debug) Task dda6d510-06f6-4106-8905-86bbbdc81312: moving from state init -> state preparing
>Thread-15916165::DEBUG::2013-11-20 04:19:23,310::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4e1282b4-658a-4143-b680-faf345043240`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916165::DEBUG::2013-11-20 04:19:23,313::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916165::DEBUG::2013-11-20 04:19:23,322::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916165::DEBUG::2013-11-20 04:19:23,325::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4e1282b4-658a-4143-b680-faf345043240`::Granted request
>Thread-15916165::DEBUG::2013-11-20 04:19:23,330::task::495::TaskManager.Task::(_debug) Task dda6d510-06f6-4106-8905-86bbbdc81312: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916165::DEBUG::2013-11-20 04:19:23,334::task::495::TaskManager.Task::(_debug) Task dda6d510-06f6-4106-8905-86bbbdc81312: ref 1 aborting False
>Thread-15916165::WARNING::2013-11-20 04:19:23,341::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916165::DEBUG::2013-11-20 04:19:23,344::task::495::TaskManager.Task::(_debug) Task dda6d510-06f6-4106-8905-86bbbdc81312: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916165::DEBUG::2013-11-20 04:19:23,347::task::495::TaskManager.Task::(_debug) Task dda6d510-06f6-4106-8905-86bbbdc81312: moving from state preparing -> state finished
>Thread-15916165::DEBUG::2013-11-20 04:19:23,348::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916165::DEBUG::2013-11-20 04:19:23,349::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916165::DEBUG::2013-11-20 04:19:23,352::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916165::DEBUG::2013-11-20 04:19:23,354::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916165::DEBUG::2013-11-20 04:19:23,356::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916165::DEBUG::2013-11-20 04:19:23,358::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916165::DEBUG::2013-11-20 04:19:23,359::task::495::TaskManager.Task::(_debug) Task dda6d510-06f6-4106-8905-86bbbdc81312: ref 0 aborting False
>Thread-15916165::INFO::2013-11-20 04:19:23,360::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:19:23,390::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0784631 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:19:25,405::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:19:25,944::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0774642 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:19:27,969::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:19:28,516::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.086216 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:19:30,532::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:19:31,093::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0678621 s, 15.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:19:33,110::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:19:33,341::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:19:33,345::task::495::TaskManager.Task::(_debug) Task 0149a699-19f9-4afa-a250-8f8ede188e04: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:19:33,348::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d9891b0d-c974-4ffd-987c-3e5daa2fb672`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:19:33,350::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:19:33,353::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:19:33,353::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d9891b0d-c974-4ffd-987c-3e5daa2fb672`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:19:33,355::task::495::TaskManager.Task::(_debug) Task 0149a699-19f9-4afa-a250-8f8ede188e04: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:19:33,356::task::495::TaskManager.Task::(_debug) Task 0149a699-19f9-4afa-a250-8f8ede188e04: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:19:33,357::task::495::TaskManager.Task::(_debug) Task 0149a699-19f9-4afa-a250-8f8ede188e04: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:19:33,358::task::495::TaskManager.Task::(_debug) Task 0149a699-19f9-4afa-a250-8f8ede188e04: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:19:33,359::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:19:33,359::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:19:33,360::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:19:33,360::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:19:33,361::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:19:33,361::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:19:33,362::task::495::TaskManager.Task::(_debug) Task 0149a699-19f9-4afa-a250-8f8ede188e04: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:19:33,362::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:19:33,666::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0836348 s, 12.2 MB/s\n'; <rc> = 0
>Thread-15916170::DEBUG::2013-11-20 04:19:33,697::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916170::INFO::2013-11-20 04:19:33,698::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916170::DEBUG::2013-11-20 04:19:33,699::task::495::TaskManager.Task::(_debug) Task a7e50455-689d-4cef-9997-05964ea797cc: moving from state init -> state preparing
>Thread-15916170::DEBUG::2013-11-20 04:19:33,700::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916170::DEBUG::2013-11-20 04:19:33,700::task::495::TaskManager.Task::(_debug) Task a7e50455-689d-4cef-9997-05964ea797cc: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916170::DEBUG::2013-11-20 04:19:33,701::task::495::TaskManager.Task::(_debug) Task a7e50455-689d-4cef-9997-05964ea797cc: moving from state preparing -> state finished
>Thread-15916170::DEBUG::2013-11-20 04:19:33,702::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916170::DEBUG::2013-11-20 04:19:33,702::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916170::DEBUG::2013-11-20 04:19:33,703::task::495::TaskManager.Task::(_debug) Task a7e50455-689d-4cef-9997-05964ea797cc: ref 0 aborting False
>Thread-15916170::INFO::2013-11-20 04:19:33,704::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916171::DEBUG::2013-11-20 04:19:33,709::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916171::INFO::2013-11-20 04:19:33,710::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916171::DEBUG::2013-11-20 04:19:33,711::task::495::TaskManager.Task::(_debug) Task 52c51a74-3737-4072-af15-581a1f2c0d42: moving from state init -> state preparing
>Thread-15916171::DEBUG::2013-11-20 04:19:33,712::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0682da0b-d22b-4d7c-84c1-e7a630468189`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916171::DEBUG::2013-11-20 04:19:33,713::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916171::DEBUG::2013-11-20 04:19:33,713::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916171::DEBUG::2013-11-20 04:19:33,714::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0682da0b-d22b-4d7c-84c1-e7a630468189`::Granted request
>Thread-15916171::DEBUG::2013-11-20 04:19:33,715::task::495::TaskManager.Task::(_debug) Task 52c51a74-3737-4072-af15-581a1f2c0d42: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916171::DEBUG::2013-11-20 04:19:33,716::task::495::TaskManager.Task::(_debug) Task 52c51a74-3737-4072-af15-581a1f2c0d42: ref 1 aborting False
>Thread-15916171::WARNING::2013-11-20 04:19:33,718::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916171::DEBUG::2013-11-20 04:19:33,720::task::495::TaskManager.Task::(_debug) Task 52c51a74-3737-4072-af15-581a1f2c0d42: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916171::DEBUG::2013-11-20 04:19:33,721::task::495::TaskManager.Task::(_debug) Task 52c51a74-3737-4072-af15-581a1f2c0d42: moving from state preparing -> state finished
>Thread-15916171::DEBUG::2013-11-20 04:19:33,722::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916171::DEBUG::2013-11-20 04:19:33,722::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916171::DEBUG::2013-11-20 04:19:33,723::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916171::DEBUG::2013-11-20 04:19:33,724::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916171::DEBUG::2013-11-20 04:19:33,725::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916171::DEBUG::2013-11-20 04:19:33,725::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916171::DEBUG::2013-11-20 04:19:33,726::task::495::TaskManager.Task::(_debug) Task 52c51a74-3737-4072-af15-581a1f2c0d42: ref 0 aborting False
>Thread-15916171::INFO::2013-11-20 04:19:33,727::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:19:35,682::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:19:36,228::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0801257 s, 12.8 MB/s\n'; <rc> = 0
>Thread-15916172::INFO::2013-11-20 04:19:36,605::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916172::DEBUG::2013-11-20 04:19:36,606::task::495::TaskManager.Task::(_debug) Task 59a95add-6391-4b6c-b6f3-c42bee9e922b: moving from state init -> state preparing
>Thread-15916172::DEBUG::2013-11-20 04:19:36,607::task::495::TaskManager.Task::(_debug) Task 59a95add-6391-4b6c-b6f3-c42bee9e922b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000555038452148', 'lastCheck': 1384921167.971101, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00689601898193', 'lastCheck': 1384921176.3825369, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00186491012573', 'lastCheck': 1384921173.354053, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000497102737427', 'lastCheck': 1384921171.3469679, 'valid': True, 'code': 0}}
>Thread-15916172::DEBUG::2013-11-20 04:19:36,608::task::495::TaskManager.Task::(_debug) Task 59a95add-6391-4b6c-b6f3-c42bee9e922b: moving from state preparing -> state finished
>Thread-15916172::DEBUG::2013-11-20 04:19:36,608::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916172::DEBUG::2013-11-20 04:19:36,609::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916172::DEBUG::2013-11-20 04:19:36,610::task::495::TaskManager.Task::(_debug) Task 59a95add-6391-4b6c-b6f3-c42bee9e922b: ref 0 aborting False
>Thread-15916172::INFO::2013-11-20 04:19:36,610::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000555038452148', 'lastCheck': 1384921167.971101, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00689601898193', 'lastCheck': 1384921176.3825369, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00186491012573', 'lastCheck': 1384921173.354053, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000497102737427', 'lastCheck': 1384921171.3469679, 'valid': True, 'code': 0}}
>Thread-15915918::INFO::2013-11-20 04:19:36,773::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:19:36,775::task::495::TaskManager.Task::(_debug) Task 14bc309d-385e-49dc-b12d-23693dbb1ab5: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:19:36,788::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`78b5e945-a117-4fe3-a6eb-59a3d747cfff`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:19:36,788::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:19:36,789::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:19:36,790::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`78b5e945-a117-4fe3-a6eb-59a3d747cfff`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:19:36,791::task::495::TaskManager.Task::(_debug) Task 14bc309d-385e-49dc-b12d-23693dbb1ab5: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:19:36,792::task::495::TaskManager.Task::(_debug) Task 14bc309d-385e-49dc-b12d-23693dbb1ab5: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:19:36,793::task::495::TaskManager.Task::(_debug) Task 14bc309d-385e-49dc-b12d-23693dbb1ab5: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:19:36,793::task::495::TaskManager.Task::(_debug) Task 14bc309d-385e-49dc-b12d-23693dbb1ab5: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:19:36,794::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:19:36,795::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:19:36,795::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:19:36,796::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:19:36,797::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:19:36,797::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:19:36,798::task::495::TaskManager.Task::(_debug) Task 14bc309d-385e-49dc-b12d-23693dbb1ab5: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:19:36,799::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:19:38,244::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:19:38,804::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.083698 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:19:40,821::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:19:41,367::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0766231 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15915863::INFO::2013-11-20 04:19:41,910::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:19:41,913::task::495::TaskManager.Task::(_debug) Task c9e365ed-1c09-4339-9433-13b47d9871b6: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:19:41,930::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ed58f87a-2737-42ed-8225-2d7eb9d91aa0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:19:41,931::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:19:41,932::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:19:41,933::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ed58f87a-2737-42ed-8225-2d7eb9d91aa0`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:19:41,936::task::495::TaskManager.Task::(_debug) Task c9e365ed-1c09-4339-9433-13b47d9871b6: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:19:41,937::task::495::TaskManager.Task::(_debug) Task c9e365ed-1c09-4339-9433-13b47d9871b6: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:19:41,939::task::495::TaskManager.Task::(_debug) Task c9e365ed-1c09-4339-9433-13b47d9871b6: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:19:41,940::task::495::TaskManager.Task::(_debug) Task c9e365ed-1c09-4339-9433-13b47d9871b6: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:19:41,941::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:19:41,942::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:19:41,944::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:19:41,944::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:19:41,945::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:19:41,946::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:19:41,947::task::495::TaskManager.Task::(_debug) Task c9e365ed-1c09-4339-9433-13b47d9871b6: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:19:41,947::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:19:43,382::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:19:43,927::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0856669 s, 12.0 MB/s\n'; <rc> = 0
>Thread-15916176::DEBUG::2013-11-20 04:19:44,182::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916176::INFO::2013-11-20 04:19:44,184::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916176::DEBUG::2013-11-20 04:19:44,185::task::495::TaskManager.Task::(_debug) Task b38dfd9f-7864-4f31-afe5-ea1353ee37aa: moving from state init -> state preparing
>Thread-15916176::DEBUG::2013-11-20 04:19:44,186::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916176::DEBUG::2013-11-20 04:19:44,187::task::495::TaskManager.Task::(_debug) Task b38dfd9f-7864-4f31-afe5-ea1353ee37aa: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916176::DEBUG::2013-11-20 04:19:44,188::task::495::TaskManager.Task::(_debug) Task b38dfd9f-7864-4f31-afe5-ea1353ee37aa: moving from state preparing -> state finished
>Thread-15916176::DEBUG::2013-11-20 04:19:44,189::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916176::DEBUG::2013-11-20 04:19:44,190::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916176::DEBUG::2013-11-20 04:19:44,191::task::495::TaskManager.Task::(_debug) Task b38dfd9f-7864-4f31-afe5-ea1353ee37aa: ref 0 aborting False
>Thread-15916176::INFO::2013-11-20 04:19:44,191::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916177::DEBUG::2013-11-20 04:19:44,196::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916177::INFO::2013-11-20 04:19:44,197::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916177::DEBUG::2013-11-20 04:19:44,198::task::495::TaskManager.Task::(_debug) Task 103c08b6-86c7-4808-845a-0ce02a319682: moving from state init -> state preparing
>Thread-15916177::DEBUG::2013-11-20 04:19:44,199::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`74b100f4-357e-4a4e-beca-3b0ca308e975`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916177::DEBUG::2013-11-20 04:19:44,200::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916177::DEBUG::2013-11-20 04:19:44,201::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916177::DEBUG::2013-11-20 04:19:44,202::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`74b100f4-357e-4a4e-beca-3b0ca308e975`::Granted request
>Thread-15916177::DEBUG::2013-11-20 04:19:44,203::task::495::TaskManager.Task::(_debug) Task 103c08b6-86c7-4808-845a-0ce02a319682: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916177::DEBUG::2013-11-20 04:19:44,204::task::495::TaskManager.Task::(_debug) Task 103c08b6-86c7-4808-845a-0ce02a319682: ref 1 aborting False
>Thread-15916177::WARNING::2013-11-20 04:19:44,206::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916177::DEBUG::2013-11-20 04:19:44,207::task::495::TaskManager.Task::(_debug) Task 103c08b6-86c7-4808-845a-0ce02a319682: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916177::DEBUG::2013-11-20 04:19:44,208::task::495::TaskManager.Task::(_debug) Task 103c08b6-86c7-4808-845a-0ce02a319682: moving from state preparing -> state finished
>Thread-15916177::DEBUG::2013-11-20 04:19:44,209::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916177::DEBUG::2013-11-20 04:19:44,209::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916177::DEBUG::2013-11-20 04:19:44,210::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916177::DEBUG::2013-11-20 04:19:44,211::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916177::DEBUG::2013-11-20 04:19:44,211::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916177::DEBUG::2013-11-20 04:19:44,212::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916177::DEBUG::2013-11-20 04:19:44,213::task::495::TaskManager.Task::(_debug) Task 103c08b6-86c7-4808-845a-0ce02a319682: ref 0 aborting False
>Thread-15916177::INFO::2013-11-20 04:19:44,214::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:19:45,946::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913518::INFO::2013-11-20 04:19:46,163::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:19:46,164::task::495::TaskManager.Task::(_debug) Task 0557e94a-e3b3-4983-801f-48965f92cb35: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:19:46,166::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`34ffc10e-060f-45ae-8019-c8356eaf8af3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:19:46,167::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:19:46,167::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:19:46,168::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`34ffc10e-060f-45ae-8019-c8356eaf8af3`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:19:46,169::task::495::TaskManager.Task::(_debug) Task 0557e94a-e3b3-4983-801f-48965f92cb35: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:19:46,169::task::495::TaskManager.Task::(_debug) Task 0557e94a-e3b3-4983-801f-48965f92cb35: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:19:46,170::task::495::TaskManager.Task::(_debug) Task 0557e94a-e3b3-4983-801f-48965f92cb35: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:19:46,170::task::495::TaskManager.Task::(_debug) Task 0557e94a-e3b3-4983-801f-48965f92cb35: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:19:46,171::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:19:46,171::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:19:46,172::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:19:46,172::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:19:46,172::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:19:46,173::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:19:46,173::task::495::TaskManager.Task::(_debug) Task 0557e94a-e3b3-4983-801f-48965f92cb35: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:19:46,174::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:19:46,174::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:19:46,175::task::495::TaskManager.Task::(_debug) Task 06aacfe8-d400-4b8c-a543-e155b85809f5: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:19:46,175::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f7deb674-7371-4bf3-8d03-82d728a3e196`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:19:46,176::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:19:46,176::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:19:46,177::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f7deb674-7371-4bf3-8d03-82d728a3e196`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:19:46,177::task::495::TaskManager.Task::(_debug) Task 06aacfe8-d400-4b8c-a543-e155b85809f5: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:19:46,178::task::495::TaskManager.Task::(_debug) Task 06aacfe8-d400-4b8c-a543-e155b85809f5: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:19:46,178::task::495::TaskManager.Task::(_debug) Task 06aacfe8-d400-4b8c-a543-e155b85809f5: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:19:46,179::task::495::TaskManager.Task::(_debug) Task 06aacfe8-d400-4b8c-a543-e155b85809f5: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:19:46,179::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:19:46,179::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:19:46,180::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:19:46,180::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:19:46,181::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:19:46,181::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:19:46,181::task::495::TaskManager.Task::(_debug) Task 06aacfe8-d400-4b8c-a543-e155b85809f5: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:19:46,182::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:19:46,465::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.077817 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:19:48,480::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 04:19:48,702::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:19:48,705::task::495::TaskManager.Task::(_debug) Task a2a3949a-240a-4b77-a9a8-239c0a7884e3: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:19:48,708::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`321e0b79-e70e-4d17-bd46-f61a817b1410`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:19:48,709::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:19:48,710::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:19:48,710::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`321e0b79-e70e-4d17-bd46-f61a817b1410`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:19:48,711::task::495::TaskManager.Task::(_debug) Task a2a3949a-240a-4b77-a9a8-239c0a7884e3: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:19:48,712::task::495::TaskManager.Task::(_debug) Task a2a3949a-240a-4b77-a9a8-239c0a7884e3: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:19:48,712::task::495::TaskManager.Task::(_debug) Task a2a3949a-240a-4b77-a9a8-239c0a7884e3: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:19:48,713::task::495::TaskManager.Task::(_debug) Task a2a3949a-240a-4b77-a9a8-239c0a7884e3: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:19:48,713::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:19:48,714::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:19:48,714::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:19:48,714::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:19:48,715::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:19:48,715::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:19:48,716::task::495::TaskManager.Task::(_debug) Task a2a3949a-240a-4b77-a9a8-239c0a7884e3: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:19:48,716::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:19:49,027::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0842348 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:19:51,042::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:19:51,439::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0458539 s, 22.3 MB/s\n'; <rc> = 0
>Thread-15916180::INFO::2013-11-20 04:19:52,922::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916180::DEBUG::2013-11-20 04:19:52,924::task::495::TaskManager.Task::(_debug) Task b2a6b622-12ab-4491-8823-900a88136d45: moving from state init -> state preparing
>Thread-15916180::DEBUG::2013-11-20 04:19:52,924::task::495::TaskManager.Task::(_debug) Task b2a6b622-12ab-4491-8823-900a88136d45: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000319004058838', 'lastCheck': 1384921188.1473789, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00108504295349', 'lastCheck': 1384921186.5849421, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0190200805664', 'lastCheck': 1384921183.611449, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000407934188843', 'lastCheck': 1384921191.7006011, 'valid': True, 'code': 0}}
>Thread-15916180::DEBUG::2013-11-20 04:19:52,925::task::495::TaskManager.Task::(_debug) Task b2a6b622-12ab-4491-8823-900a88136d45: moving from state preparing -> state finished
>Thread-15916180::DEBUG::2013-11-20 04:19:52,926::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916180::DEBUG::2013-11-20 04:19:52,927::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916180::DEBUG::2013-11-20 04:19:52,927::task::495::TaskManager.Task::(_debug) Task b2a6b622-12ab-4491-8823-900a88136d45: ref 0 aborting False
>Thread-15916180::INFO::2013-11-20 04:19:52,928::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000319004058838', 'lastCheck': 1384921188.1473789, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00108504295349', 'lastCheck': 1384921186.5849421, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0190200805664', 'lastCheck': 1384921183.611449, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000407934188843', 'lastCheck': 1384921191.7006011, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:19:53,449::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:19:54,026::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.105599 s, 9.7 MB/s\n'; <rc> = 0
>Thread-15916182::DEBUG::2013-11-20 04:19:54,727::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916182::INFO::2013-11-20 04:19:54,729::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916182::DEBUG::2013-11-20 04:19:54,729::task::495::TaskManager.Task::(_debug) Task e72b0028-d319-418d-9927-5f345df4a20d: moving from state init -> state preparing
>Thread-15916182::DEBUG::2013-11-20 04:19:54,730::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916182::DEBUG::2013-11-20 04:19:54,731::task::495::TaskManager.Task::(_debug) Task e72b0028-d319-418d-9927-5f345df4a20d: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916182::DEBUG::2013-11-20 04:19:54,731::task::495::TaskManager.Task::(_debug) Task e72b0028-d319-418d-9927-5f345df4a20d: moving from state preparing -> state finished
>Thread-15916182::DEBUG::2013-11-20 04:19:54,732::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916182::DEBUG::2013-11-20 04:19:54,733::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916182::DEBUG::2013-11-20 04:19:54,733::task::495::TaskManager.Task::(_debug) Task e72b0028-d319-418d-9927-5f345df4a20d: ref 0 aborting False
>Thread-15916182::INFO::2013-11-20 04:19:54,734::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916183::DEBUG::2013-11-20 04:19:54,739::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916183::INFO::2013-11-20 04:19:54,740::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916183::DEBUG::2013-11-20 04:19:54,740::task::495::TaskManager.Task::(_debug) Task 80804e7b-6207-4fcf-9023-a1a18e6a1149: moving from state init -> state preparing
>Thread-15916183::DEBUG::2013-11-20 04:19:54,742::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0a57877f-deda-4e98-8204-84d804357091`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916183::DEBUG::2013-11-20 04:19:54,742::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916183::DEBUG::2013-11-20 04:19:54,743::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916183::DEBUG::2013-11-20 04:19:54,743::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0a57877f-deda-4e98-8204-84d804357091`::Granted request
>Thread-15916183::DEBUG::2013-11-20 04:19:54,744::task::495::TaskManager.Task::(_debug) Task 80804e7b-6207-4fcf-9023-a1a18e6a1149: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916183::DEBUG::2013-11-20 04:19:54,745::task::495::TaskManager.Task::(_debug) Task 80804e7b-6207-4fcf-9023-a1a18e6a1149: ref 1 aborting False
>Thread-15916183::WARNING::2013-11-20 04:19:54,748::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916183::DEBUG::2013-11-20 04:19:54,748::task::495::TaskManager.Task::(_debug) Task 80804e7b-6207-4fcf-9023-a1a18e6a1149: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916183::DEBUG::2013-11-20 04:19:54,749::task::495::TaskManager.Task::(_debug) Task 80804e7b-6207-4fcf-9023-a1a18e6a1149: moving from state preparing -> state finished
>Thread-15916183::DEBUG::2013-11-20 04:19:54,750::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916183::DEBUG::2013-11-20 04:19:54,751::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916183::DEBUG::2013-11-20 04:19:54,752::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916183::DEBUG::2013-11-20 04:19:54,753::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916183::DEBUG::2013-11-20 04:19:54,753::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916183::DEBUG::2013-11-20 04:19:54,754::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916183::DEBUG::2013-11-20 04:19:54,754::task::495::TaskManager.Task::(_debug) Task 80804e7b-6207-4fcf-9023-a1a18e6a1149: ref 0 aborting False
>Thread-15916183::INFO::2013-11-20 04:19:54,755::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:19:56,042::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:19:56,594::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0784435 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:19:58,610::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 04:19:58,865::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:19:58,883::task::495::TaskManager.Task::(_debug) Task 776392f2-47d2-42b2-86d6-8abe5f096df9: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:19:58,891::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c1554f9e-9a02-4abc-af9b-00d328d15314`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:19:58,893::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:19:58,894::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:19:58,894::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c1554f9e-9a02-4abc-af9b-00d328d15314`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:19:58,895::task::495::TaskManager.Task::(_debug) Task 776392f2-47d2-42b2-86d6-8abe5f096df9: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:19:58,895::task::495::TaskManager.Task::(_debug) Task 776392f2-47d2-42b2-86d6-8abe5f096df9: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:19:58,896::task::495::TaskManager.Task::(_debug) Task 776392f2-47d2-42b2-86d6-8abe5f096df9: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:19:58,896::task::495::TaskManager.Task::(_debug) Task 776392f2-47d2-42b2-86d6-8abe5f096df9: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:19:58,897::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:19:58,897::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:19:58,897::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:19:58,898::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:19:58,898::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:19:58,898::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:19:58,899::task::495::TaskManager.Task::(_debug) Task 776392f2-47d2-42b2-86d6-8abe5f096df9: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:19:58,899::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:19:59,183::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0808675 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:20:01,197::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:20:01,762::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0805866 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:20:03,778::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 04:20:04,324::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:20:04,326::task::495::TaskManager.Task::(_debug) Task 210d3fa8-8f5d-4dde-85f5-af945fefa31a: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:20:04,331::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`cde3421f-aba9-4594-9798-b9a5fc12a471`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:20:04,332::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:20:04,336::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:20:04,337::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`cde3421f-aba9-4594-9798-b9a5fc12a471`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:20:04,341::task::495::TaskManager.Task::(_debug) Task 210d3fa8-8f5d-4dde-85f5-af945fefa31a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:20:04,343::task::495::TaskManager.Task::(_debug) Task 210d3fa8-8f5d-4dde-85f5-af945fefa31a: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:20:04,345::task::495::TaskManager.Task::(_debug) Task 210d3fa8-8f5d-4dde-85f5-af945fefa31a: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:20:04,347::task::495::TaskManager.Task::(_debug) Task 210d3fa8-8f5d-4dde-85f5-af945fefa31a: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:20:04,348::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:20:04,349::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:20:04,351::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:20:04,353::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:20:04,354::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:20:04,356::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:20:04,358::task::495::TaskManager.Task::(_debug) Task 210d3fa8-8f5d-4dde-85f5-af945fefa31a: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:20:04,360::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:20:04,410::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0882394 s, 11.6 MB/s\n'; <rc> = 0
>Thread-15916187::DEBUG::2013-11-20 04:20:05,124::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916187::INFO::2013-11-20 04:20:05,126::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916187::DEBUG::2013-11-20 04:20:05,126::task::495::TaskManager.Task::(_debug) Task 904ec397-1729-4ceb-9858-af34c0d40c36: moving from state init -> state preparing
>Thread-15916187::DEBUG::2013-11-20 04:20:05,127::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916187::DEBUG::2013-11-20 04:20:05,128::task::495::TaskManager.Task::(_debug) Task 904ec397-1729-4ceb-9858-af34c0d40c36: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916187::DEBUG::2013-11-20 04:20:05,128::task::495::TaskManager.Task::(_debug) Task 904ec397-1729-4ceb-9858-af34c0d40c36: moving from state preparing -> state finished
>Thread-15916187::DEBUG::2013-11-20 04:20:05,129::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916187::DEBUG::2013-11-20 04:20:05,130::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916187::DEBUG::2013-11-20 04:20:05,130::task::495::TaskManager.Task::(_debug) Task 904ec397-1729-4ceb-9858-af34c0d40c36: ref 0 aborting False
>Thread-15916187::INFO::2013-11-20 04:20:05,131::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916188::DEBUG::2013-11-20 04:20:05,136::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916188::INFO::2013-11-20 04:20:05,137::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916188::DEBUG::2013-11-20 04:20:05,138::task::495::TaskManager.Task::(_debug) Task f8663161-95b1-4b22-b215-52fb85435e80: moving from state init -> state preparing
>Thread-15916188::DEBUG::2013-11-20 04:20:05,139::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b98c8e36-2a35-4d26-95f8-11e5738f5dfc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916188::DEBUG::2013-11-20 04:20:05,140::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916188::DEBUG::2013-11-20 04:20:05,141::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916188::DEBUG::2013-11-20 04:20:05,142::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b98c8e36-2a35-4d26-95f8-11e5738f5dfc`::Granted request
>Thread-15916188::DEBUG::2013-11-20 04:20:05,143::task::495::TaskManager.Task::(_debug) Task f8663161-95b1-4b22-b215-52fb85435e80: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916188::DEBUG::2013-11-20 04:20:05,143::task::495::TaskManager.Task::(_debug) Task f8663161-95b1-4b22-b215-52fb85435e80: ref 1 aborting False
>Thread-15916188::WARNING::2013-11-20 04:20:05,145::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916188::DEBUG::2013-11-20 04:20:05,146::task::495::TaskManager.Task::(_debug) Task f8663161-95b1-4b22-b215-52fb85435e80: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916188::DEBUG::2013-11-20 04:20:05,147::task::495::TaskManager.Task::(_debug) Task f8663161-95b1-4b22-b215-52fb85435e80: moving from state preparing -> state finished
>Thread-15916188::DEBUG::2013-11-20 04:20:05,148::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916188::DEBUG::2013-11-20 04:20:05,148::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916188::DEBUG::2013-11-20 04:20:05,149::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916188::DEBUG::2013-11-20 04:20:05,150::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916188::DEBUG::2013-11-20 04:20:05,151::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916188::DEBUG::2013-11-20 04:20:05,151::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916188::DEBUG::2013-11-20 04:20:05,152::task::495::TaskManager.Task::(_debug) Task f8663161-95b1-4b22-b215-52fb85435e80: ref 0 aborting False
>Thread-15916188::INFO::2013-11-20 04:20:05,153::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:20:06,433::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:20:06,946::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0747132 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15916190::INFO::2013-11-20 04:20:08,735::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916190::DEBUG::2013-11-20 04:20:08,736::task::495::TaskManager.Task::(_debug) Task 702bbf5e-e810-4eb7-b449-e76c2038931b: moving from state init -> state preparing
>Thread-15916190::DEBUG::2013-11-20 04:20:08,737::task::495::TaskManager.Task::(_debug) Task 702bbf5e-e810-4eb7-b449-e76c2038931b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000463008880615', 'lastCheck': 1384921208.4391921, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00131893157959', 'lastCheck': 1384921206.991369, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00231003761292', 'lastCheck': 1384921204.0016899, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000348091125488', 'lastCheck': 1384921201.847039, 'valid': True, 'code': 0}}
>Thread-15916190::DEBUG::2013-11-20 04:20:08,738::task::495::TaskManager.Task::(_debug) Task 702bbf5e-e810-4eb7-b449-e76c2038931b: moving from state preparing -> state finished
>Thread-15916190::DEBUG::2013-11-20 04:20:08,738::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916190::DEBUG::2013-11-20 04:20:08,739::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916190::DEBUG::2013-11-20 04:20:08,740::task::495::TaskManager.Task::(_debug) Task 702bbf5e-e810-4eb7-b449-e76c2038931b: ref 0 aborting False
>Thread-15916190::INFO::2013-11-20 04:20:08,740::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000463008880615', 'lastCheck': 1384921208.4391921, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00131893157959', 'lastCheck': 1384921206.991369, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00231003761292', 'lastCheck': 1384921204.0016899, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000348091125488', 'lastCheck': 1384921201.847039, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:20:08,961::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 04:20:09,182::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:20:09,185::task::495::TaskManager.Task::(_debug) Task 1f116991-b81a-4f47-b7a2-7c19f85f291b: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:20:09,188::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`87847cc9-e103-4408-8588-a13d57d19b0e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:20:09,189::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:20:09,191::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:20:09,192::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`87847cc9-e103-4408-8588-a13d57d19b0e`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:20:09,194::task::495::TaskManager.Task::(_debug) Task 1f116991-b81a-4f47-b7a2-7c19f85f291b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:20:09,194::task::495::TaskManager.Task::(_debug) Task 1f116991-b81a-4f47-b7a2-7c19f85f291b: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:20:09,195::task::495::TaskManager.Task::(_debug) Task 1f116991-b81a-4f47-b7a2-7c19f85f291b: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:20:09,196::task::495::TaskManager.Task::(_debug) Task 1f116991-b81a-4f47-b7a2-7c19f85f291b: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:20:09,196::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:20:09,197::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:20:09,198::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:20:09,198::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:20:09,199::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:20:09,199::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:20:09,200::task::495::TaskManager.Task::(_debug) Task 1f116991-b81a-4f47-b7a2-7c19f85f291b: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:20:09,201::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:20:09,493::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0644907 s, 15.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:20:11,507::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:20:12,106::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0774885 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:20:14,121::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:20:14,687::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0765783 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15916194::DEBUG::2013-11-20 04:20:15,421::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916194::INFO::2013-11-20 04:20:15,423::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916194::DEBUG::2013-11-20 04:20:15,424::task::495::TaskManager.Task::(_debug) Task d2cd9c7e-ef8f-459b-b001-9194194b68a3: moving from state init -> state preparing
>Thread-15916194::DEBUG::2013-11-20 04:20:15,425::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916194::DEBUG::2013-11-20 04:20:15,426::task::495::TaskManager.Task::(_debug) Task d2cd9c7e-ef8f-459b-b001-9194194b68a3: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916194::DEBUG::2013-11-20 04:20:15,427::task::495::TaskManager.Task::(_debug) Task d2cd9c7e-ef8f-459b-b001-9194194b68a3: moving from state preparing -> state finished
>Thread-15916194::DEBUG::2013-11-20 04:20:15,427::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916194::DEBUG::2013-11-20 04:20:15,428::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916194::DEBUG::2013-11-20 04:20:15,429::task::495::TaskManager.Task::(_debug) Task d2cd9c7e-ef8f-459b-b001-9194194b68a3: ref 0 aborting False
>Thread-15916194::INFO::2013-11-20 04:20:15,429::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916195::DEBUG::2013-11-20 04:20:15,434::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916195::INFO::2013-11-20 04:20:15,435::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916195::DEBUG::2013-11-20 04:20:15,436::task::495::TaskManager.Task::(_debug) Task 3750596e-2e53-488f-881b-a676d0435320: moving from state init -> state preparing
>Thread-15916195::DEBUG::2013-11-20 04:20:15,438::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c32d04e2-5fcc-4013-9ffa-7ae73d8c4276`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916195::DEBUG::2013-11-20 04:20:15,439::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916195::DEBUG::2013-11-20 04:20:15,440::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916195::DEBUG::2013-11-20 04:20:15,441::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c32d04e2-5fcc-4013-9ffa-7ae73d8c4276`::Granted request
>Thread-15916195::DEBUG::2013-11-20 04:20:15,442::task::495::TaskManager.Task::(_debug) Task 3750596e-2e53-488f-881b-a676d0435320: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916195::DEBUG::2013-11-20 04:20:15,443::task::495::TaskManager.Task::(_debug) Task 3750596e-2e53-488f-881b-a676d0435320: ref 1 aborting False
>Thread-15916195::WARNING::2013-11-20 04:20:15,446::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916195::DEBUG::2013-11-20 04:20:15,447::task::495::TaskManager.Task::(_debug) Task 3750596e-2e53-488f-881b-a676d0435320: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916195::DEBUG::2013-11-20 04:20:15,448::task::495::TaskManager.Task::(_debug) Task 3750596e-2e53-488f-881b-a676d0435320: moving from state preparing -> state finished
>Thread-15916195::DEBUG::2013-11-20 04:20:15,449::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916195::DEBUG::2013-11-20 04:20:15,450::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916195::DEBUG::2013-11-20 04:20:15,451::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916195::DEBUG::2013-11-20 04:20:15,453::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916195::DEBUG::2013-11-20 04:20:15,453::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916195::DEBUG::2013-11-20 04:20:15,454::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916195::DEBUG::2013-11-20 04:20:15,455::task::495::TaskManager.Task::(_debug) Task 3750596e-2e53-488f-881b-a676d0435320: ref 0 aborting False
>Thread-15916195::INFO::2013-11-20 04:20:15,455::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:20:16,700::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 04:20:16,992::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:20:16,997::task::495::TaskManager.Task::(_debug) Task 28e9be7d-cf19-4d94-bb09-b3a5b8277cdf: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:20:17,001::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0bd66892-9969-47e1-95ac-8039806323dc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:20:17,002::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:20:17,003::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:20:17,003::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0bd66892-9969-47e1-95ac-8039806323dc`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:20:17,013::task::495::TaskManager.Task::(_debug) Task 28e9be7d-cf19-4d94-bb09-b3a5b8277cdf: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 04:20:17,014::task::495::TaskManager.Task::(_debug) Task 28e9be7d-cf19-4d94-bb09-b3a5b8277cdf: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:20:17,015::task::495::TaskManager.Task::(_debug) Task 28e9be7d-cf19-4d94-bb09-b3a5b8277cdf: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 04:20:17,017::task::495::TaskManager.Task::(_debug) Task 28e9be7d-cf19-4d94-bb09-b3a5b8277cdf: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:20:17,017::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:20:17,018::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:20:17,020::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 04:20:17,020::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:20:17,021::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:20:17,022::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:20:17,023::task::495::TaskManager.Task::(_debug) Task 28e9be7d-cf19-4d94-bb09-b3a5b8277cdf: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 04:20:17,023::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 04:20:17,232::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0614861 s, 16.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:20:19,247::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 04:20:19,502::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:20:19,507::task::495::TaskManager.Task::(_debug) Task a15ce9c0-44a4-48c6-94ab-828502a6f0f7: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:20:19,510::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`213aa07f-68a7-4873-9ba3-a187775fd761`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:20:19,511::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:20:19,512::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:20:19,512::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`213aa07f-68a7-4873-9ba3-a187775fd761`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:20:19,513::task::495::TaskManager.Task::(_debug) Task a15ce9c0-44a4-48c6-94ab-828502a6f0f7: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:20:19,514::task::495::TaskManager.Task::(_debug) Task a15ce9c0-44a4-48c6-94ab-828502a6f0f7: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:20:19,515::task::495::TaskManager.Task::(_debug) Task a15ce9c0-44a4-48c6-94ab-828502a6f0f7: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 04:20:19,515::task::495::TaskManager.Task::(_debug) Task a15ce9c0-44a4-48c6-94ab-828502a6f0f7: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:20:19,517::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15914435::INFO::2013-11-20 04:20:19,518::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15913546::DEBUG::2013-11-20 04:20:19,519::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:20:19,520::task::495::TaskManager.Task::(_debug) Task 7b604f34-6541-4708-b663-1b817682163b: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:20:19,521::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15914435::DEBUG::2013-11-20 04:20:19,523::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`98ba780d-1e5a-4560-9958-5dbcf3a7c645`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:20:19,524::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:20:19,524::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:20:19,525::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:20:19,526::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:20:19,527::task::495::TaskManager.Task::(_debug) Task a15ce9c0-44a4-48c6-94ab-828502a6f0f7: ref 0 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:20:19,527::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::INFO::2013-11-20 04:20:19,528::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914435::DEBUG::2013-11-20 04:20:19,529::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`98ba780d-1e5a-4560-9958-5dbcf3a7c645`::Granted request
>Thread-15913546::INFO::2013-11-20 04:20:19,530::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15914435::DEBUG::2013-11-20 04:20:19,532::task::495::TaskManager.Task::(_debug) Task 7b604f34-6541-4708-b663-1b817682163b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913546::DEBUG::2013-11-20 04:20:19,532::task::495::TaskManager.Task::(_debug) Task 870b1bdf-bafe-439b-972b-09a54465418b: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:20:19,533::task::495::TaskManager.Task::(_debug) Task 7b604f34-6541-4708-b663-1b817682163b: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:20:19,536::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2e2439ec-2a83-4a2a-a6e9-c622e21d9fa0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:20:19,537::task::495::TaskManager.Task::(_debug) Task 7b604f34-6541-4708-b663-1b817682163b: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15913546::DEBUG::2013-11-20 04:20:19,537::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:20:19,538::task::495::TaskManager.Task::(_debug) Task 7b604f34-6541-4708-b663-1b817682163b: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:20:19,539::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:20:19,540::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:20:19,540::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2e2439ec-2a83-4a2a-a6e9-c622e21d9fa0`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:20:19,541::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:20:19,542::task::495::TaskManager.Task::(_debug) Task 870b1bdf-bafe-439b-972b-09a54465418b: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15914435::DEBUG::2013-11-20 04:20:19,543::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913546::DEBUG::2013-11-20 04:20:19,544::task::495::TaskManager.Task::(_debug) Task 870b1bdf-bafe-439b-972b-09a54465418b: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:20:19,545::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:20:19,545::task::495::TaskManager.Task::(_debug) Task 870b1bdf-bafe-439b-972b-09a54465418b: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15914435::DEBUG::2013-11-20 04:20:19,546::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:20:19,547::task::495::TaskManager.Task::(_debug) Task 870b1bdf-bafe-439b-972b-09a54465418b: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:20:19,548::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:20:19,548::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:20:19,549::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:20:19,549::task::495::TaskManager.Task::(_debug) Task 7b604f34-6541-4708-b663-1b817682163b: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:20:19,550::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15914435::INFO::2013-11-20 04:20:19,551::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15913546::DEBUG::2013-11-20 04:20:19,551::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:20:19,552::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:20:19,553::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:20:19,554::task::495::TaskManager.Task::(_debug) Task 870b1bdf-bafe-439b-972b-09a54465418b: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:20:19,585::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:20:19,594::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:20:19,595::task::495::TaskManager.Task::(_debug) Task df3b534b-2220-489c-9422-f69659f01496: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:20:19,597::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e7790fa4-9c80-4e36-a731-e252bd17255c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:20:19,598::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:20:19,598::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:20:19,599::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e7790fa4-9c80-4e36-a731-e252bd17255c`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:20:19,600::task::495::TaskManager.Task::(_debug) Task df3b534b-2220-489c-9422-f69659f01496: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:20:19,601::task::495::TaskManager.Task::(_debug) Task df3b534b-2220-489c-9422-f69659f01496: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:20:19,602::task::495::TaskManager.Task::(_debug) Task df3b534b-2220-489c-9422-f69659f01496: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:20:19,602::task::495::TaskManager.Task::(_debug) Task df3b534b-2220-489c-9422-f69659f01496: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:20:19,603::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:20:19,604::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:20:19,604::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:20:19,605::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:20:19,605::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:20:19,606::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:20:19,606::task::495::TaskManager.Task::(_debug) Task df3b534b-2220-489c-9422-f69659f01496: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:20:19,607::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:20:19,608::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:20:19,613::task::495::TaskManager.Task::(_debug) Task a8a614e4-20d7-49e8-941d-c0980e86a925: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:20:19,615::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`142c77ca-703f-42c6-bd3d-8ccda3e792fd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:20:19,616::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:20:19,617::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:20:19,617::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`142c77ca-703f-42c6-bd3d-8ccda3e792fd`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:20:19,618::task::495::TaskManager.Task::(_debug) Task a8a614e4-20d7-49e8-941d-c0980e86a925: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:20:19,619::task::495::TaskManager.Task::(_debug) Task a8a614e4-20d7-49e8-941d-c0980e86a925: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:20:19,620::task::495::TaskManager.Task::(_debug) Task a8a614e4-20d7-49e8-941d-c0980e86a925: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:20:19,620::task::495::TaskManager.Task::(_debug) Task a8a614e4-20d7-49e8-941d-c0980e86a925: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:20:19,621::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:20:19,621::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:20:19,622::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:20:19,622::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:20:19,623::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:20:19,623::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:20:19,624::task::495::TaskManager.Task::(_debug) Task a8a614e4-20d7-49e8-941d-c0980e86a925: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:20:19,625::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:20:19,840::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0885519 s, 11.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:20:21,854::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:20:22,420::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0793764 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:20:24,442::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916198::INFO::2013-11-20 04:20:24,673::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916198::DEBUG::2013-11-20 04:20:24,678::task::495::TaskManager.Task::(_debug) Task 6d2c20c6-6f91-4b22-8259-a659e139d82c: moving from state init -> state preparing
>Thread-15916198::DEBUG::2013-11-20 04:20:24,680::task::495::TaskManager.Task::(_debug) Task 6d2c20c6-6f91-4b22-8259-a659e139d82c: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000353097915649', 'lastCheck': 1384921218.4530449, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0102939605713', 'lastCheck': 1384921217.396698, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00266695022583', 'lastCheck': 1384921224.6794701, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000745058059692', 'lastCheck': 1384921222.475513, 'valid': True, 'code': 0}}
>Thread-15916198::DEBUG::2013-11-20 04:20:24,681::task::495::TaskManager.Task::(_debug) Task 6d2c20c6-6f91-4b22-8259-a659e139d82c: moving from state preparing -> state finished
>Thread-15916198::DEBUG::2013-11-20 04:20:24,682::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916198::DEBUG::2013-11-20 04:20:24,683::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916198::DEBUG::2013-11-20 04:20:24,684::task::495::TaskManager.Task::(_debug) Task 6d2c20c6-6f91-4b22-8259-a659e139d82c: ref 0 aborting False
>Thread-15916198::INFO::2013-11-20 04:20:24,685::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000353097915649', 'lastCheck': 1384921218.4530449, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0102939605713', 'lastCheck': 1384921217.396698, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00266695022583', 'lastCheck': 1384921224.6794701, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000745058059692', 'lastCheck': 1384921222.475513, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:20:24,994::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.078047 s, 13.1 MB/s\n'; <rc> = 0
>Thread-15916200::DEBUG::2013-11-20 04:20:26,188::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916200::INFO::2013-11-20 04:20:26,190::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916200::DEBUG::2013-11-20 04:20:26,191::task::495::TaskManager.Task::(_debug) Task b1ec66e6-de1e-41ce-a78c-5527f1ede0de: moving from state init -> state preparing
>Thread-15916200::DEBUG::2013-11-20 04:20:26,192::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916200::DEBUG::2013-11-20 04:20:26,192::task::495::TaskManager.Task::(_debug) Task b1ec66e6-de1e-41ce-a78c-5527f1ede0de: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916200::DEBUG::2013-11-20 04:20:26,193::task::495::TaskManager.Task::(_debug) Task b1ec66e6-de1e-41ce-a78c-5527f1ede0de: moving from state preparing -> state finished
>Thread-15916200::DEBUG::2013-11-20 04:20:26,194::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916200::DEBUG::2013-11-20 04:20:26,194::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916200::DEBUG::2013-11-20 04:20:26,195::task::495::TaskManager.Task::(_debug) Task b1ec66e6-de1e-41ce-a78c-5527f1ede0de: ref 0 aborting False
>Thread-15916200::INFO::2013-11-20 04:20:26,195::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916201::DEBUG::2013-11-20 04:20:26,200::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916201::INFO::2013-11-20 04:20:26,202::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916201::DEBUG::2013-11-20 04:20:26,203::task::495::TaskManager.Task::(_debug) Task 39be4bb9-9d5b-42da-95ad-4c217a674774: moving from state init -> state preparing
>Thread-15916201::DEBUG::2013-11-20 04:20:26,204::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e84a8433-6510-4cd4-84af-01d5e598b423`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916201::DEBUG::2013-11-20 04:20:26,205::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916201::DEBUG::2013-11-20 04:20:26,206::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916201::DEBUG::2013-11-20 04:20:26,207::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e84a8433-6510-4cd4-84af-01d5e598b423`::Granted request
>Thread-15916201::DEBUG::2013-11-20 04:20:26,207::task::495::TaskManager.Task::(_debug) Task 39be4bb9-9d5b-42da-95ad-4c217a674774: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916201::DEBUG::2013-11-20 04:20:26,208::task::495::TaskManager.Task::(_debug) Task 39be4bb9-9d5b-42da-95ad-4c217a674774: ref 1 aborting False
>Thread-15916201::WARNING::2013-11-20 04:20:26,210::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916201::DEBUG::2013-11-20 04:20:26,211::task::495::TaskManager.Task::(_debug) Task 39be4bb9-9d5b-42da-95ad-4c217a674774: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916201::DEBUG::2013-11-20 04:20:26,212::task::495::TaskManager.Task::(_debug) Task 39be4bb9-9d5b-42da-95ad-4c217a674774: moving from state preparing -> state finished
>Thread-15916201::DEBUG::2013-11-20 04:20:26,213::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916201::DEBUG::2013-11-20 04:20:26,214::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916201::DEBUG::2013-11-20 04:20:26,214::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916201::DEBUG::2013-11-20 04:20:26,215::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916201::DEBUG::2013-11-20 04:20:26,216::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916201::DEBUG::2013-11-20 04:20:26,216::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916201::DEBUG::2013-11-20 04:20:26,217::task::495::TaskManager.Task::(_debug) Task 39be4bb9-9d5b-42da-95ad-4c217a674774: ref 0 aborting False
>Thread-15916201::INFO::2013-11-20 04:20:26,217::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:20:27,009::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:20:27,548::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0853264 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:20:29,563::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:20:30,096::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0793453 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:20:32,112::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:20:32,643::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0762914 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:20:34,657::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:20:34,893::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:20:34,910::task::495::TaskManager.Task::(_debug) Task c10cd4a5-8e94-486f-8606-385917f66f2a: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:20:34,916::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`86b9c4ec-2d6e-4619-b2f7-dccdcaee2ee6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:20:34,918::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:20:34,921::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:20:34,922::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`86b9c4ec-2d6e-4619-b2f7-dccdcaee2ee6`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:20:34,925::task::495::TaskManager.Task::(_debug) Task c10cd4a5-8e94-486f-8606-385917f66f2a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:20:34,927::task::495::TaskManager.Task::(_debug) Task c10cd4a5-8e94-486f-8606-385917f66f2a: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:20:34,928::task::495::TaskManager.Task::(_debug) Task c10cd4a5-8e94-486f-8606-385917f66f2a: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:20:34,929::task::495::TaskManager.Task::(_debug) Task c10cd4a5-8e94-486f-8606-385917f66f2a: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:20:34,930::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:20:34,931::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:20:34,933::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:20:34,934::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:20:34,935::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:20:34,936::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:20:34,937::task::495::TaskManager.Task::(_debug) Task c10cd4a5-8e94-486f-8606-385917f66f2a: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:20:34,938::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:20:35,231::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0786576 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15916205::DEBUG::2013-11-20 04:20:36,630::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916205::INFO::2013-11-20 04:20:36,631::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916205::DEBUG::2013-11-20 04:20:36,632::task::495::TaskManager.Task::(_debug) Task 2552d914-ab43-4349-859b-217b05a0c4ce: moving from state init -> state preparing
>Thread-15916205::DEBUG::2013-11-20 04:20:36,633::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916205::DEBUG::2013-11-20 04:20:36,633::task::495::TaskManager.Task::(_debug) Task 2552d914-ab43-4349-859b-217b05a0c4ce: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916205::DEBUG::2013-11-20 04:20:36,634::task::495::TaskManager.Task::(_debug) Task 2552d914-ab43-4349-859b-217b05a0c4ce: moving from state preparing -> state finished
>Thread-15916205::DEBUG::2013-11-20 04:20:36,634::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916205::DEBUG::2013-11-20 04:20:36,635::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916205::DEBUG::2013-11-20 04:20:36,635::task::495::TaskManager.Task::(_debug) Task 2552d914-ab43-4349-859b-217b05a0c4ce: ref 0 aborting False
>Thread-15916205::INFO::2013-11-20 04:20:36,636::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916206::DEBUG::2013-11-20 04:20:36,640::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916206::INFO::2013-11-20 04:20:36,641::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916206::DEBUG::2013-11-20 04:20:36,642::task::495::TaskManager.Task::(_debug) Task 1a6ab193-96b6-4973-aa1c-a08f01821485: moving from state init -> state preparing
>Thread-15916206::DEBUG::2013-11-20 04:20:36,643::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`50c5eed9-bc90-434d-949d-fac36609ca37`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916206::DEBUG::2013-11-20 04:20:36,644::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916206::DEBUG::2013-11-20 04:20:36,645::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916206::DEBUG::2013-11-20 04:20:36,646::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`50c5eed9-bc90-434d-949d-fac36609ca37`::Granted request
>Thread-15916206::DEBUG::2013-11-20 04:20:36,648::task::495::TaskManager.Task::(_debug) Task 1a6ab193-96b6-4973-aa1c-a08f01821485: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916206::DEBUG::2013-11-20 04:20:36,649::task::495::TaskManager.Task::(_debug) Task 1a6ab193-96b6-4973-aa1c-a08f01821485: ref 1 aborting False
>Thread-15916206::WARNING::2013-11-20 04:20:36,652::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916206::DEBUG::2013-11-20 04:20:36,654::task::495::TaskManager.Task::(_debug) Task 1a6ab193-96b6-4973-aa1c-a08f01821485: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916206::DEBUG::2013-11-20 04:20:36,655::task::495::TaskManager.Task::(_debug) Task 1a6ab193-96b6-4973-aa1c-a08f01821485: moving from state preparing -> state finished
>Thread-15916206::DEBUG::2013-11-20 04:20:36,656::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916206::DEBUG::2013-11-20 04:20:36,657::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916206::DEBUG::2013-11-20 04:20:36,658::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916206::DEBUG::2013-11-20 04:20:36,659::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916206::DEBUG::2013-11-20 04:20:36,660::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916206::DEBUG::2013-11-20 04:20:36,661::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916206::DEBUG::2013-11-20 04:20:36,662::task::495::TaskManager.Task::(_debug) Task 1a6ab193-96b6-4973-aa1c-a08f01821485: ref 0 aborting False
>Thread-15916206::INFO::2013-11-20 04:20:36,662::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:20:37,251::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915918::INFO::2013-11-20 04:20:37,535::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:20:37,536::task::495::TaskManager.Task::(_debug) Task 049f8244-1172-4a26-b40c-e74c78c28f03: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:20:37,539::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f68e765c-a875-4155-bead-880f2cf9e532`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:20:37,540::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:20:37,542::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:20:37,542::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f68e765c-a875-4155-bead-880f2cf9e532`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:20:37,544::task::495::TaskManager.Task::(_debug) Task 049f8244-1172-4a26-b40c-e74c78c28f03: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:20:37,545::task::495::TaskManager.Task::(_debug) Task 049f8244-1172-4a26-b40c-e74c78c28f03: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:20:37,546::task::495::TaskManager.Task::(_debug) Task 049f8244-1172-4a26-b40c-e74c78c28f03: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:20:37,547::task::495::TaskManager.Task::(_debug) Task 049f8244-1172-4a26-b40c-e74c78c28f03: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:20:37,548::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:20:37,549::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:20:37,550::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:20:37,551::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:20:37,551::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:20:37,552::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:20:37,553::task::495::TaskManager.Task::(_debug) Task 049f8244-1172-4a26-b40c-e74c78c28f03: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:20:37,554::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:20:37,820::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0771451 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:20:39,833::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:20:40,359::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0712878 s, 14.4 MB/s\n'; <rc> = 0
>Thread-15916208::INFO::2013-11-20 04:20:40,521::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916208::DEBUG::2013-11-20 04:20:40,522::task::495::TaskManager.Task::(_debug) Task 1a5f49f6-bf9a-40ea-ba58-f2cf2dddbb87: moving from state init -> state preparing
>Thread-15916208::DEBUG::2013-11-20 04:20:40,523::task::495::TaskManager.Task::(_debug) Task 1a5f49f6-bf9a-40ea-ba58-f2cf2dddbb87: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000329971313477', 'lastCheck': 1384921238.8849299, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00271701812744', 'lastCheck': 1384921237.887264, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0123319625854', 'lastCheck': 1384921234.926729, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00484299659729', 'lastCheck': 1384921232.7700961, 'valid': True, 'code': 0}}
>Thread-15916208::DEBUG::2013-11-20 04:20:40,524::task::495::TaskManager.Task::(_debug) Task 1a5f49f6-bf9a-40ea-ba58-f2cf2dddbb87: moving from state preparing -> state finished
>Thread-15916208::DEBUG::2013-11-20 04:20:40,525::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916208::DEBUG::2013-11-20 04:20:40,526::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916208::DEBUG::2013-11-20 04:20:40,526::task::495::TaskManager.Task::(_debug) Task 1a5f49f6-bf9a-40ea-ba58-f2cf2dddbb87: ref 0 aborting False
>Thread-15916208::INFO::2013-11-20 04:20:40,527::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000329971313477', 'lastCheck': 1384921238.8849299, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00271701812744', 'lastCheck': 1384921237.887264, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0123319625854', 'lastCheck': 1384921234.926729, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00484299659729', 'lastCheck': 1384921232.7700961, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:20:42,373::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915863::INFO::2013-11-20 04:20:42,596::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:20:42,610::task::495::TaskManager.Task::(_debug) Task 50d63aed-43de-4606-b454-d1e121bb254f: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:20:42,614::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7b03a02e-d813-4b6c-a3a8-7f302abaa296`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:20:42,616::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:20:42,620::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:20:42,621::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7b03a02e-d813-4b6c-a3a8-7f302abaa296`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:20:42,623::task::495::TaskManager.Task::(_debug) Task 50d63aed-43de-4606-b454-d1e121bb254f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:20:42,624::task::495::TaskManager.Task::(_debug) Task 50d63aed-43de-4606-b454-d1e121bb254f: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:20:42,625::task::495::TaskManager.Task::(_debug) Task 50d63aed-43de-4606-b454-d1e121bb254f: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:20:42,625::task::495::TaskManager.Task::(_debug) Task 50d63aed-43de-4606-b454-d1e121bb254f: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:20:42,626::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:20:42,627::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:20:42,627::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:20:42,628::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:20:42,629::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:20:42,629::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:20:42,630::task::495::TaskManager.Task::(_debug) Task 50d63aed-43de-4606-b454-d1e121bb254f: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:20:42,631::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:20:42,978::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0834223 s, 12.3 MB/s\n'; <rc> = 0
>Thread-42840::DEBUG::2013-11-20 04:20:43,043::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 04:20:44,072::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:20:44,998::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:20:45,510::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0824562 s, 12.4 MB/s\n'; <rc> = 0
>Thread-15916211::DEBUG::2013-11-20 04:20:47,094::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916211::INFO::2013-11-20 04:20:47,095::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916211::DEBUG::2013-11-20 04:20:47,096::task::495::TaskManager.Task::(_debug) Task 6a3549b0-4062-4555-a7f7-00334cdfa63c: moving from state init -> state preparing
>Thread-15916211::DEBUG::2013-11-20 04:20:47,097::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916211::DEBUG::2013-11-20 04:20:47,097::task::495::TaskManager.Task::(_debug) Task 6a3549b0-4062-4555-a7f7-00334cdfa63c: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916211::DEBUG::2013-11-20 04:20:47,098::task::495::TaskManager.Task::(_debug) Task 6a3549b0-4062-4555-a7f7-00334cdfa63c: moving from state preparing -> state finished
>Thread-15916211::DEBUG::2013-11-20 04:20:47,098::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916211::DEBUG::2013-11-20 04:20:47,099::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916211::DEBUG::2013-11-20 04:20:47,100::task::495::TaskManager.Task::(_debug) Task 6a3549b0-4062-4555-a7f7-00334cdfa63c: ref 0 aborting False
>Thread-15916211::INFO::2013-11-20 04:20:47,100::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916212::DEBUG::2013-11-20 04:20:47,105::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916212::INFO::2013-11-20 04:20:47,106::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916212::DEBUG::2013-11-20 04:20:47,106::task::495::TaskManager.Task::(_debug) Task 21556066-47a2-4baa-aceb-57dc14f881b9: moving from state init -> state preparing
>Thread-15916212::DEBUG::2013-11-20 04:20:47,108::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b85f1a87-0d94-40c2-a1bc-7b5c74ad1189`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916212::DEBUG::2013-11-20 04:20:47,109::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916212::DEBUG::2013-11-20 04:20:47,109::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916212::DEBUG::2013-11-20 04:20:47,110::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b85f1a87-0d94-40c2-a1bc-7b5c74ad1189`::Granted request
>Thread-15916212::DEBUG::2013-11-20 04:20:47,112::task::495::TaskManager.Task::(_debug) Task 21556066-47a2-4baa-aceb-57dc14f881b9: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916212::DEBUG::2013-11-20 04:20:47,112::task::495::TaskManager.Task::(_debug) Task 21556066-47a2-4baa-aceb-57dc14f881b9: ref 1 aborting False
>Thread-15916212::WARNING::2013-11-20 04:20:47,115::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916212::DEBUG::2013-11-20 04:20:47,116::task::495::TaskManager.Task::(_debug) Task 21556066-47a2-4baa-aceb-57dc14f881b9: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916212::DEBUG::2013-11-20 04:20:47,117::task::495::TaskManager.Task::(_debug) Task 21556066-47a2-4baa-aceb-57dc14f881b9: moving from state preparing -> state finished
>Thread-15916212::DEBUG::2013-11-20 04:20:47,118::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916212::DEBUG::2013-11-20 04:20:47,119::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916212::DEBUG::2013-11-20 04:20:47,120::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916212::DEBUG::2013-11-20 04:20:47,120::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916212::DEBUG::2013-11-20 04:20:47,121::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916212::DEBUG::2013-11-20 04:20:47,122::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916212::DEBUG::2013-11-20 04:20:47,123::task::495::TaskManager.Task::(_debug) Task 21556066-47a2-4baa-aceb-57dc14f881b9: ref 0 aborting False
>Thread-15916212::INFO::2013-11-20 04:20:47,123::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:20:47,526::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913518::INFO::2013-11-20 04:20:47,773::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:20:47,774::task::495::TaskManager.Task::(_debug) Task 419e898a-e20d-4196-a3ed-5e45fa674490: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:20:47,776::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`051cdb6e-878c-44d1-a909-20860952ad08`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:20:47,777::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:20:47,778::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:20:47,779::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`051cdb6e-878c-44d1-a909-20860952ad08`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:20:47,780::task::495::TaskManager.Task::(_debug) Task 419e898a-e20d-4196-a3ed-5e45fa674490: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:20:47,781::task::495::TaskManager.Task::(_debug) Task 419e898a-e20d-4196-a3ed-5e45fa674490: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:20:47,781::task::495::TaskManager.Task::(_debug) Task 419e898a-e20d-4196-a3ed-5e45fa674490: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:20:47,782::task::495::TaskManager.Task::(_debug) Task 419e898a-e20d-4196-a3ed-5e45fa674490: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:20:47,783::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:20:47,783::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:20:47,784::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:20:47,785::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:20:47,785::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:20:47,786::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:20:47,786::task::495::TaskManager.Task::(_debug) Task 419e898a-e20d-4196-a3ed-5e45fa674490: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:20:47,786::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:20:47,787::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:20:47,787::task::495::TaskManager.Task::(_debug) Task 79dc15f4-194f-4e70-9acb-74608c330ca2: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:20:47,788::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3fd9f969-d549-434a-b357-af2a16c998e6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:20:47,788::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:20:47,789::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:20:47,789::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3fd9f969-d549-434a-b357-af2a16c998e6`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:20:47,790::task::495::TaskManager.Task::(_debug) Task 79dc15f4-194f-4e70-9acb-74608c330ca2: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:20:47,790::task::495::TaskManager.Task::(_debug) Task 79dc15f4-194f-4e70-9acb-74608c330ca2: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:20:47,790::task::495::TaskManager.Task::(_debug) Task 79dc15f4-194f-4e70-9acb-74608c330ca2: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:20:47,791::task::495::TaskManager.Task::(_debug) Task 79dc15f4-194f-4e70-9acb-74608c330ca2: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:20:47,791::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:20:47,791::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:20:47,792::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:20:47,792::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:20:47,792::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:20:47,793::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:20:47,793::task::495::TaskManager.Task::(_debug) Task 79dc15f4-194f-4e70-9acb-74608c330ca2: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:20:47,794::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:20:48,073::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0904961 s, 11.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:20:50,082::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 04:20:50,311::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:20:50,316::task::495::TaskManager.Task::(_debug) Task 1981f5b1-ec5d-4b94-bf0c-4dcb11e81855: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:20:50,320::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5c4be155-fa6a-494d-985c-ce340c14db32`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:20:50,322::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:20:50,324::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:20:50,324::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5c4be155-fa6a-494d-985c-ce340c14db32`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:20:50,327::task::495::TaskManager.Task::(_debug) Task 1981f5b1-ec5d-4b94-bf0c-4dcb11e81855: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:20:50,328::task::495::TaskManager.Task::(_debug) Task 1981f5b1-ec5d-4b94-bf0c-4dcb11e81855: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:20:50,329::task::495::TaskManager.Task::(_debug) Task 1981f5b1-ec5d-4b94-bf0c-4dcb11e81855: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:20:50,330::task::495::TaskManager.Task::(_debug) Task 1981f5b1-ec5d-4b94-bf0c-4dcb11e81855: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:20:50,331::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:20:50,332::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:20:50,333::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:20:50,334::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:20:50,334::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:20:50,335::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:20:50,336::task::495::TaskManager.Task::(_debug) Task 1981f5b1-ec5d-4b94-bf0c-4dcb11e81855: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:20:50,336::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:20:50,662::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0879978 s, 11.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:20:52,686::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:20:53,218::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0879714 s, 11.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:20:55,236::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:20:55,793::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0874582 s, 11.7 MB/s\n'; <rc> = 0
>Thread-15916216::INFO::2013-11-20 04:20:56,816::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916216::DEBUG::2013-11-20 04:20:56,817::task::495::TaskManager.Task::(_debug) Task 7df7bcc0-289b-4b65-be58-2e5edb6f548b: moving from state init -> state preparing
>Thread-15916216::DEBUG::2013-11-20 04:20:56,818::task::495::TaskManager.Task::(_debug) Task 7df7bcc0-289b-4b65-be58-2e5edb6f548b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000399827957153', 'lastCheck': 1384921249.2036331, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00134205818176', 'lastCheck': 1384921248.2067831, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00167798995972', 'lastCheck': 1384921255.468631, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000418186187744', 'lastCheck': 1384921254.308481, 'valid': True, 'code': 0}}
>Thread-15916216::DEBUG::2013-11-20 04:20:56,819::task::495::TaskManager.Task::(_debug) Task 7df7bcc0-289b-4b65-be58-2e5edb6f548b: moving from state preparing -> state finished
>Thread-15916216::DEBUG::2013-11-20 04:20:56,820::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916216::DEBUG::2013-11-20 04:20:56,820::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916216::DEBUG::2013-11-20 04:20:56,821::task::495::TaskManager.Task::(_debug) Task 7df7bcc0-289b-4b65-be58-2e5edb6f548b: ref 0 aborting False
>Thread-15916216::INFO::2013-11-20 04:20:56,821::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000399827957153', 'lastCheck': 1384921249.2036331, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00134205818176', 'lastCheck': 1384921248.2067831, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00167798995972', 'lastCheck': 1384921255.468631, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000418186187744', 'lastCheck': 1384921254.308481, 'valid': True, 'code': 0}}
>Thread-15916218::DEBUG::2013-11-20 04:20:57,405::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916218::INFO::2013-11-20 04:20:57,406::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916218::DEBUG::2013-11-20 04:20:57,407::task::495::TaskManager.Task::(_debug) Task 59f287f5-e027-4436-b143-00d71bd8f51a: moving from state init -> state preparing
>Thread-15916218::DEBUG::2013-11-20 04:20:57,408::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916218::DEBUG::2013-11-20 04:20:57,409::task::495::TaskManager.Task::(_debug) Task 59f287f5-e027-4436-b143-00d71bd8f51a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916218::DEBUG::2013-11-20 04:20:57,409::task::495::TaskManager.Task::(_debug) Task 59f287f5-e027-4436-b143-00d71bd8f51a: moving from state preparing -> state finished
>Thread-15916218::DEBUG::2013-11-20 04:20:57,410::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916218::DEBUG::2013-11-20 04:20:57,411::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916218::DEBUG::2013-11-20 04:20:57,411::task::495::TaskManager.Task::(_debug) Task 59f287f5-e027-4436-b143-00d71bd8f51a: ref 0 aborting False
>Thread-15916218::INFO::2013-11-20 04:20:57,412::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916219::DEBUG::2013-11-20 04:20:57,417::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916219::INFO::2013-11-20 04:20:57,419::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916219::DEBUG::2013-11-20 04:20:57,419::task::495::TaskManager.Task::(_debug) Task b8ee9672-d450-47a1-8be6-8cda6f30de20: moving from state init -> state preparing
>Thread-15916219::DEBUG::2013-11-20 04:20:57,421::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b510e356-611a-445b-b96f-337383534f6c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916219::DEBUG::2013-11-20 04:20:57,421::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916219::DEBUG::2013-11-20 04:20:57,422::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916219::DEBUG::2013-11-20 04:20:57,423::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b510e356-611a-445b-b96f-337383534f6c`::Granted request
>Thread-15916219::DEBUG::2013-11-20 04:20:57,424::task::495::TaskManager.Task::(_debug) Task b8ee9672-d450-47a1-8be6-8cda6f30de20: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916219::DEBUG::2013-11-20 04:20:57,424::task::495::TaskManager.Task::(_debug) Task b8ee9672-d450-47a1-8be6-8cda6f30de20: ref 1 aborting False
>Thread-15916219::WARNING::2013-11-20 04:20:57,426::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916219::DEBUG::2013-11-20 04:20:57,427::task::495::TaskManager.Task::(_debug) Task b8ee9672-d450-47a1-8be6-8cda6f30de20: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916219::DEBUG::2013-11-20 04:20:57,428::task::495::TaskManager.Task::(_debug) Task b8ee9672-d450-47a1-8be6-8cda6f30de20: moving from state preparing -> state finished
>Thread-15916219::DEBUG::2013-11-20 04:20:57,429::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916219::DEBUG::2013-11-20 04:20:57,429::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916219::DEBUG::2013-11-20 04:20:57,430::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916219::DEBUG::2013-11-20 04:20:57,430::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916219::DEBUG::2013-11-20 04:20:57,431::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916219::DEBUG::2013-11-20 04:20:57,431::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916219::DEBUG::2013-11-20 04:20:57,432::task::495::TaskManager.Task::(_debug) Task b8ee9672-d450-47a1-8be6-8cda6f30de20: ref 0 aborting False
>Thread-15916219::INFO::2013-11-20 04:20:57,433::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:20:57,810::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:20:58,321::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0736174 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:21:00,335::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 04:21:00,563::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:21:00,566::task::495::TaskManager.Task::(_debug) Task 12e91f3b-bce8-46b2-8d5c-59077dc834b4: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:21:00,569::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`124173da-402f-4efc-b3b3-4cd3c56d6f60`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:21:00,570::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:21:00,571::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:21:00,572::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`124173da-402f-4efc-b3b3-4cd3c56d6f60`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:21:00,573::task::495::TaskManager.Task::(_debug) Task 12e91f3b-bce8-46b2-8d5c-59077dc834b4: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:21:00,573::task::495::TaskManager.Task::(_debug) Task 12e91f3b-bce8-46b2-8d5c-59077dc834b4: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:21:00,574::task::495::TaskManager.Task::(_debug) Task 12e91f3b-bce8-46b2-8d5c-59077dc834b4: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:21:00,574::task::495::TaskManager.Task::(_debug) Task 12e91f3b-bce8-46b2-8d5c-59077dc834b4: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:21:00,574::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:21:00,575::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:21:00,575::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:21:00,575::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:21:00,576::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:21:00,576::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:21:00,576::task::495::TaskManager.Task::(_debug) Task 12e91f3b-bce8-46b2-8d5c-59077dc834b4: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:21:00,577::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:21:00,833::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0534278 s, 19.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:21:02,844::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:21:03,418::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0820858 s, 12.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:21:05,433::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 04:21:05,649::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:21:05,651::task::495::TaskManager.Task::(_debug) Task ea66a920-7a84-4e68-b7e3-51a32b7b7bfe: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:21:05,658::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ae7a7ca9-b49b-4d37-8b59-0f8b6a69cc48`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:21:05,659::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:21:05,661::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:21:05,661::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ae7a7ca9-b49b-4d37-8b59-0f8b6a69cc48`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:21:05,663::task::495::TaskManager.Task::(_debug) Task ea66a920-7a84-4e68-b7e3-51a32b7b7bfe: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:21:05,664::task::495::TaskManager.Task::(_debug) Task ea66a920-7a84-4e68-b7e3-51a32b7b7bfe: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:21:05,665::task::495::TaskManager.Task::(_debug) Task ea66a920-7a84-4e68-b7e3-51a32b7b7bfe: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:21:05,666::task::495::TaskManager.Task::(_debug) Task ea66a920-7a84-4e68-b7e3-51a32b7b7bfe: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:21:05,667::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:21:05,668::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:21:05,669::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:21:05,669::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:21:05,670::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:21:05,671::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:21:05,672::task::495::TaskManager.Task::(_debug) Task ea66a920-7a84-4e68-b7e3-51a32b7b7bfe: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:21:05,673::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:21:05,966::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0738107 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:21:07,980::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916223::DEBUG::2013-11-20 04:21:08,236::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916223::INFO::2013-11-20 04:21:08,238::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916223::DEBUG::2013-11-20 04:21:08,238::task::495::TaskManager.Task::(_debug) Task 41477e93-d5dd-46ba-9a3f-8856897d0471: moving from state init -> state preparing
>Thread-15916223::DEBUG::2013-11-20 04:21:08,240::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916223::DEBUG::2013-11-20 04:21:08,241::task::495::TaskManager.Task::(_debug) Task 41477e93-d5dd-46ba-9a3f-8856897d0471: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916223::DEBUG::2013-11-20 04:21:08,242::task::495::TaskManager.Task::(_debug) Task 41477e93-d5dd-46ba-9a3f-8856897d0471: moving from state preparing -> state finished
>Thread-15916223::DEBUG::2013-11-20 04:21:08,243::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916223::DEBUG::2013-11-20 04:21:08,244::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916223::DEBUG::2013-11-20 04:21:08,244::task::495::TaskManager.Task::(_debug) Task 41477e93-d5dd-46ba-9a3f-8856897d0471: ref 0 aborting False
>Thread-15916223::INFO::2013-11-20 04:21:08,245::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916224::DEBUG::2013-11-20 04:21:08,250::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916224::INFO::2013-11-20 04:21:08,251::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916224::DEBUG::2013-11-20 04:21:08,252::task::495::TaskManager.Task::(_debug) Task 1ac5b12d-c7f9-4659-b7a5-40ea8ddd9872: moving from state init -> state preparing
>Thread-15916224::DEBUG::2013-11-20 04:21:08,254::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`82dbe8b0-23db-4261-8b40-33be4a771c26`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916224::DEBUG::2013-11-20 04:21:08,254::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916224::DEBUG::2013-11-20 04:21:08,256::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916224::DEBUG::2013-11-20 04:21:08,256::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`82dbe8b0-23db-4261-8b40-33be4a771c26`::Granted request
>Thread-15916224::DEBUG::2013-11-20 04:21:08,258::task::495::TaskManager.Task::(_debug) Task 1ac5b12d-c7f9-4659-b7a5-40ea8ddd9872: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916224::DEBUG::2013-11-20 04:21:08,258::task::495::TaskManager.Task::(_debug) Task 1ac5b12d-c7f9-4659-b7a5-40ea8ddd9872: ref 1 aborting False
>Thread-15916224::WARNING::2013-11-20 04:21:08,261::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916224::DEBUG::2013-11-20 04:21:08,262::task::495::TaskManager.Task::(_debug) Task 1ac5b12d-c7f9-4659-b7a5-40ea8ddd9872: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916224::DEBUG::2013-11-20 04:21:08,263::task::495::TaskManager.Task::(_debug) Task 1ac5b12d-c7f9-4659-b7a5-40ea8ddd9872: moving from state preparing -> state finished
>Thread-15916224::DEBUG::2013-11-20 04:21:08,264::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916224::DEBUG::2013-11-20 04:21:08,264::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916224::DEBUG::2013-11-20 04:21:08,265::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916224::DEBUG::2013-11-20 04:21:08,266::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916224::DEBUG::2013-11-20 04:21:08,267::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916224::DEBUG::2013-11-20 04:21:08,267::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916224::DEBUG::2013-11-20 04:21:08,268::task::495::TaskManager.Task::(_debug) Task 1ac5b12d-c7f9-4659-b7a5-40ea8ddd9872: ref 0 aborting False
>Thread-15916224::INFO::2013-11-20 04:21:08,269::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:21:08,474::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0554087 s, 18.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:21:10,484::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 04:21:10,740::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:21:10,743::task::495::TaskManager.Task::(_debug) Task ba7f25b7-c77f-44a2-8740-fdade4587689: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:21:10,746::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`92752ee7-2690-4c02-a711-f37e0c1ba70f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:21:10,747::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:21:10,748::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:21:10,749::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`92752ee7-2690-4c02-a711-f37e0c1ba70f`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:21:10,751::task::495::TaskManager.Task::(_debug) Task ba7f25b7-c77f-44a2-8740-fdade4587689: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:21:10,752::task::495::TaskManager.Task::(_debug) Task ba7f25b7-c77f-44a2-8740-fdade4587689: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:21:10,752::task::495::TaskManager.Task::(_debug) Task ba7f25b7-c77f-44a2-8740-fdade4587689: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:21:10,753::task::495::TaskManager.Task::(_debug) Task ba7f25b7-c77f-44a2-8740-fdade4587689: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:21:10,754::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:21:10,754::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:21:10,755::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:21:10,756::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:21:10,756::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:21:10,757::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:21:10,758::task::495::TaskManager.Task::(_debug) Task ba7f25b7-c77f-44a2-8740-fdade4587689: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:21:10,758::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:21:11,051::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0882898 s, 11.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:21:13,067::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916226::INFO::2013-11-20 04:21:13,430::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916226::DEBUG::2013-11-20 04:21:13,431::task::495::TaskManager.Task::(_debug) Task a51ebe4d-9f79-41b1-af14-b2c06b21da20: moving from state init -> state preparing
>Thread-15916226::DEBUG::2013-11-20 04:21:13,432::task::495::TaskManager.Task::(_debug) Task a51ebe4d-9f79-41b1-af14-b2c06b21da20: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000533103942871', 'lastCheck': 1384921269.6416321, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00138592720032', 'lastCheck': 1384921268.6407261, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00441789627075', 'lastCheck': 1384921265.658664, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000436067581177', 'lastCheck': 1384921264.5583129, 'valid': True, 'code': 0}}
>Thread-15916226::DEBUG::2013-11-20 04:21:13,433::task::495::TaskManager.Task::(_debug) Task a51ebe4d-9f79-41b1-af14-b2c06b21da20: moving from state preparing -> state finished
>Thread-15916226::DEBUG::2013-11-20 04:21:13,433::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916226::DEBUG::2013-11-20 04:21:13,434::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916226::DEBUG::2013-11-20 04:21:13,434::task::495::TaskManager.Task::(_debug) Task a51ebe4d-9f79-41b1-af14-b2c06b21da20: ref 0 aborting False
>Thread-15916226::INFO::2013-11-20 04:21:13,435::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000533103942871', 'lastCheck': 1384921269.6416321, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00138592720032', 'lastCheck': 1384921268.6407261, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00441789627075', 'lastCheck': 1384921265.658664, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000436067581177', 'lastCheck': 1384921264.5583129, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:21:13,624::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.114869 s, 8.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:21:15,639::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:21:16,171::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0804432 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:21:18,187::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 04:21:18,388::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:21:18,398::task::495::TaskManager.Task::(_debug) Task 8a41570c-96bd-4123-91dd-57ba04551873: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:21:18,404::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`98fa7dac-fe84-4a21-8e3e-dc96e71f3c99`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:21:18,406::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:21:18,407::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:21:18,408::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`98fa7dac-fe84-4a21-8e3e-dc96e71f3c99`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:21:18,410::task::495::TaskManager.Task::(_debug) Task 8a41570c-96bd-4123-91dd-57ba04551873: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 04:21:18,411::task::495::TaskManager.Task::(_debug) Task 8a41570c-96bd-4123-91dd-57ba04551873: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:21:18,412::task::495::TaskManager.Task::(_debug) Task 8a41570c-96bd-4123-91dd-57ba04551873: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 04:21:18,412::task::495::TaskManager.Task::(_debug) Task 8a41570c-96bd-4123-91dd-57ba04551873: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:21:18,413::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:21:18,414::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:21:18,415::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 04:21:18,416::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:21:18,417::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:21:18,417::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:21:18,419::task::495::TaskManager.Task::(_debug) Task 8a41570c-96bd-4123-91dd-57ba04551873: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 04:21:18,420::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15916229::DEBUG::2013-11-20 04:21:18,591::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916229::INFO::2013-11-20 04:21:18,593::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916229::DEBUG::2013-11-20 04:21:18,594::task::495::TaskManager.Task::(_debug) Task 1e63d334-7021-4e18-a314-4ad7831eaf4b: moving from state init -> state preparing
>Thread-15916229::DEBUG::2013-11-20 04:21:18,595::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916229::DEBUG::2013-11-20 04:21:18,596::task::495::TaskManager.Task::(_debug) Task 1e63d334-7021-4e18-a314-4ad7831eaf4b: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916229::DEBUG::2013-11-20 04:21:18,597::task::495::TaskManager.Task::(_debug) Task 1e63d334-7021-4e18-a314-4ad7831eaf4b: moving from state preparing -> state finished
>Thread-15916229::DEBUG::2013-11-20 04:21:18,598::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916229::DEBUG::2013-11-20 04:21:18,599::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916229::DEBUG::2013-11-20 04:21:18,600::task::495::TaskManager.Task::(_debug) Task 1e63d334-7021-4e18-a314-4ad7831eaf4b: ref 0 aborting False
>Thread-15916229::INFO::2013-11-20 04:21:18,601::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916230::DEBUG::2013-11-20 04:21:18,606::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916230::INFO::2013-11-20 04:21:18,607::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916230::DEBUG::2013-11-20 04:21:18,608::task::495::TaskManager.Task::(_debug) Task c6c391b8-bbbb-46fb-ae03-4a3292f39ee4: moving from state init -> state preparing
>Thread-15916230::DEBUG::2013-11-20 04:21:18,611::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ee8b171e-06fd-4ac5-aa70-c02204b72cbe`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916230::DEBUG::2013-11-20 04:21:18,612::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916230::DEBUG::2013-11-20 04:21:18,613::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916230::DEBUG::2013-11-20 04:21:18,614::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ee8b171e-06fd-4ac5-aa70-c02204b72cbe`::Granted request
>Thread-15916230::DEBUG::2013-11-20 04:21:18,615::task::495::TaskManager.Task::(_debug) Task c6c391b8-bbbb-46fb-ae03-4a3292f39ee4: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916230::DEBUG::2013-11-20 04:21:18,616::task::495::TaskManager.Task::(_debug) Task c6c391b8-bbbb-46fb-ae03-4a3292f39ee4: ref 1 aborting False
>Thread-15916230::WARNING::2013-11-20 04:21:18,620::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916230::DEBUG::2013-11-20 04:21:18,622::task::495::TaskManager.Task::(_debug) Task c6c391b8-bbbb-46fb-ae03-4a3292f39ee4: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916230::DEBUG::2013-11-20 04:21:18,623::task::495::TaskManager.Task::(_debug) Task c6c391b8-bbbb-46fb-ae03-4a3292f39ee4: moving from state preparing -> state finished
>Thread-15916230::DEBUG::2013-11-20 04:21:18,624::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916230::DEBUG::2013-11-20 04:21:18,625::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916230::DEBUG::2013-11-20 04:21:18,627::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916230::DEBUG::2013-11-20 04:21:18,628::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916230::DEBUG::2013-11-20 04:21:18,629::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916230::DEBUG::2013-11-20 04:21:18,630::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916230::DEBUG::2013-11-20 04:21:18,631::task::495::TaskManager.Task::(_debug) Task c6c391b8-bbbb-46fb-ae03-4a3292f39ee4: ref 0 aborting False
>Thread-15916230::INFO::2013-11-20 04:21:18,633::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:21:18,760::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.118693 s, 8.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:21:20,777::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 04:21:20,978::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:21:20,980::task::495::TaskManager.Task::(_debug) Task b095c279-0b5a-4768-be37-09dfad18163c: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:21:20,985::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`80bd4785-d430-41e6-b653-2570fb41094a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:21:20,987::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15914435::INFO::2013-11-20 04:21:20,988::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:21:20,990::task::495::TaskManager.Task::(_debug) Task d6507fb2-5570-425e-b026-d0bd186f87ba: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:21:20,990::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:21:20,992::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`80bd4785-d430-41e6-b653-2570fb41094a`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:21:20,993::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`def8d683-048e-43a1-938f-c8c1f11d1b2f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:21:20,994::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:21:20,995::task::495::TaskManager.Task::(_debug) Task b095c279-0b5a-4768-be37-09dfad18163c: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15914435::DEBUG::2013-11-20 04:21:20,996::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:21:20,996::task::495::TaskManager.Task::(_debug) Task b095c279-0b5a-4768-be37-09dfad18163c: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:21:20,997::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`def8d683-048e-43a1-938f-c8c1f11d1b2f`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:21:20,998::task::495::TaskManager.Task::(_debug) Task b095c279-0b5a-4768-be37-09dfad18163c: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914435::DEBUG::2013-11-20 04:21:20,999::task::495::TaskManager.Task::(_debug) Task d6507fb2-5570-425e-b026-d0bd186f87ba: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913546::DEBUG::2013-11-20 04:21:21,000::task::495::TaskManager.Task::(_debug) Task b095c279-0b5a-4768-be37-09dfad18163c: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:21:21,000::task::495::TaskManager.Task::(_debug) Task d6507fb2-5570-425e-b026-d0bd186f87ba: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:21:21,001::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:21:21,002::task::495::TaskManager.Task::(_debug) Task d6507fb2-5570-425e-b026-d0bd186f87ba: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15913546::DEBUG::2013-11-20 04:21:21,003::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:21:21,004::task::495::TaskManager.Task::(_debug) Task d6507fb2-5570-425e-b026-d0bd186f87ba: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:21:21,005::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15914435::DEBUG::2013-11-20 04:21:21,006::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:21:21,007::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:21:21,007::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:21:21,008::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:21:21,009::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913546::DEBUG::2013-11-20 04:21:21,010::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:21:21,011::task::495::TaskManager.Task::(_debug) Task b095c279-0b5a-4768-be37-09dfad18163c: ref 0 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:21:21,012::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913546::INFO::2013-11-20 04:21:21,012::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914435::DEBUG::2013-11-20 04:21:21,013::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913546::INFO::2013-11-20 04:21:21,014::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15914435::DEBUG::2013-11-20 04:21:21,014::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:21:21,015::task::495::TaskManager.Task::(_debug) Task ab3c2008-11f5-4258-b350-13bfe37dfde0: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:21:21,016::task::495::TaskManager.Task::(_debug) Task d6507fb2-5570-425e-b026-d0bd186f87ba: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:21:21,018::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b54d9fb0-c108-4887-8708-0e4095512f9e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::INFO::2013-11-20 04:21:21,019::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15913546::DEBUG::2013-11-20 04:21:21,019::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:21:21,021::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:21:21,021::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b54d9fb0-c108-4887-8708-0e4095512f9e`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:21:21,022::task::495::TaskManager.Task::(_debug) Task ab3c2008-11f5-4258-b350-13bfe37dfde0: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:21:21,023::task::495::TaskManager.Task::(_debug) Task ab3c2008-11f5-4258-b350-13bfe37dfde0: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:21:21,024::task::495::TaskManager.Task::(_debug) Task ab3c2008-11f5-4258-b350-13bfe37dfde0: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:21:21,024::task::495::TaskManager.Task::(_debug) Task ab3c2008-11f5-4258-b350-13bfe37dfde0: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:21:21,025::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:21:21,026::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:21:21,026::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:21:21,027::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:21:21,027::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:21:21,028::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:21:21,029::task::495::TaskManager.Task::(_debug) Task ab3c2008-11f5-4258-b350-13bfe37dfde0: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:21:21,029::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:21:21,030::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:21:21,031::task::495::TaskManager.Task::(_debug) Task ea570488-595c-460d-8052-ff5cb26f55d6: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:21:21,033::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e8ad61c3-831e-45cf-8d67-2a9884a7304c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:21:21,033::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:21:21,034::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:21:21,035::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e8ad61c3-831e-45cf-8d67-2a9884a7304c`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:21:21,036::task::495::TaskManager.Task::(_debug) Task ea570488-595c-460d-8052-ff5cb26f55d6: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:21:21,036::task::495::TaskManager.Task::(_debug) Task ea570488-595c-460d-8052-ff5cb26f55d6: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:21:21,037::task::495::TaskManager.Task::(_debug) Task ea570488-595c-460d-8052-ff5cb26f55d6: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:21:21,038::task::495::TaskManager.Task::(_debug) Task ea570488-595c-460d-8052-ff5cb26f55d6: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:21:21,038::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:21:21,039::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:21:21,039::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:21:21,040::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:21:21,041::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:21:21,045::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:21:21,046::task::495::TaskManager.Task::(_debug) Task ea570488-595c-460d-8052-ff5cb26f55d6: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:21:21,046::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:21:21,048::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:21:21,048::task::495::TaskManager.Task::(_debug) Task 8afcc010-9e77-4f64-ab65-fd5515642148: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:21:21,049::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`838e93d5-c7a3-4fdd-9074-1301f8738266`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:21:21,050::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:21:21,051::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:21:21,051::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`838e93d5-c7a3-4fdd-9074-1301f8738266`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:21:21,052::task::495::TaskManager.Task::(_debug) Task 8afcc010-9e77-4f64-ab65-fd5515642148: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:21:21,053::task::495::TaskManager.Task::(_debug) Task 8afcc010-9e77-4f64-ab65-fd5515642148: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:21:21,053::task::495::TaskManager.Task::(_debug) Task 8afcc010-9e77-4f64-ab65-fd5515642148: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:21:21,054::task::495::TaskManager.Task::(_debug) Task 8afcc010-9e77-4f64-ab65-fd5515642148: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:21:21,055::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:21:21,055::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:21:21,056::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:21:21,056::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:21:21,057::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:21:21,057::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:21:21,058::task::495::TaskManager.Task::(_debug) Task 8afcc010-9e77-4f64-ab65-fd5515642148: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:21:21,059::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:21:21,320::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0855978 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:21:23,330::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:21:23,883::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0743719 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:21:25,905::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:21:26,402::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0489209 s, 20.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:21:28,412::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:21:28,912::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0822241 s, 12.5 MB/s\n'; <rc> = 0
>Thread-15916234::DEBUG::2013-11-20 04:21:29,089::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916234::INFO::2013-11-20 04:21:29,091::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916234::DEBUG::2013-11-20 04:21:29,091::task::495::TaskManager.Task::(_debug) Task 960ef4ee-0808-445e-a5eb-0e43fc279c98: moving from state init -> state preparing
>Thread-15916234::DEBUG::2013-11-20 04:21:29,092::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916234::DEBUG::2013-11-20 04:21:29,093::task::495::TaskManager.Task::(_debug) Task 960ef4ee-0808-445e-a5eb-0e43fc279c98: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916234::DEBUG::2013-11-20 04:21:29,093::task::495::TaskManager.Task::(_debug) Task 960ef4ee-0808-445e-a5eb-0e43fc279c98: moving from state preparing -> state finished
>Thread-15916234::DEBUG::2013-11-20 04:21:29,094::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916234::DEBUG::2013-11-20 04:21:29,095::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916234::DEBUG::2013-11-20 04:21:29,095::task::495::TaskManager.Task::(_debug) Task 960ef4ee-0808-445e-a5eb-0e43fc279c98: ref 0 aborting False
>Thread-15916234::INFO::2013-11-20 04:21:29,096::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916235::DEBUG::2013-11-20 04:21:29,175::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916235::INFO::2013-11-20 04:21:29,177::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916235::DEBUG::2013-11-20 04:21:29,177::task::495::TaskManager.Task::(_debug) Task 985dead0-6397-4519-8acb-d515d47d0a31: moving from state init -> state preparing
>Thread-15916235::DEBUG::2013-11-20 04:21:29,179::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2173e46c-4b35-40e7-8abc-5be5e0199996`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916235::DEBUG::2013-11-20 04:21:29,179::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916235::DEBUG::2013-11-20 04:21:29,180::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916235::DEBUG::2013-11-20 04:21:29,181::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2173e46c-4b35-40e7-8abc-5be5e0199996`::Granted request
>Thread-15916235::DEBUG::2013-11-20 04:21:29,182::task::495::TaskManager.Task::(_debug) Task 985dead0-6397-4519-8acb-d515d47d0a31: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916235::DEBUG::2013-11-20 04:21:29,182::task::495::TaskManager.Task::(_debug) Task 985dead0-6397-4519-8acb-d515d47d0a31: ref 1 aborting False
>Thread-15916235::WARNING::2013-11-20 04:21:29,184::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916235::DEBUG::2013-11-20 04:21:29,185::task::495::TaskManager.Task::(_debug) Task 985dead0-6397-4519-8acb-d515d47d0a31: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916235::DEBUG::2013-11-20 04:21:29,186::task::495::TaskManager.Task::(_debug) Task 985dead0-6397-4519-8acb-d515d47d0a31: moving from state preparing -> state finished
>Thread-15916235::DEBUG::2013-11-20 04:21:29,187::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916235::DEBUG::2013-11-20 04:21:29,187::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916235::DEBUG::2013-11-20 04:21:29,188::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916235::DEBUG::2013-11-20 04:21:29,189::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916235::DEBUG::2013-11-20 04:21:29,189::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916235::DEBUG::2013-11-20 04:21:29,190::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916235::DEBUG::2013-11-20 04:21:29,191::task::495::TaskManager.Task::(_debug) Task 985dead0-6397-4519-8acb-d515d47d0a31: ref 0 aborting False
>Thread-15916235::INFO::2013-11-20 04:21:29,192::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916236::INFO::2013-11-20 04:21:29,506::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916236::DEBUG::2013-11-20 04:21:29,508::task::495::TaskManager.Task::(_debug) Task 493f6efd-aafa-4662-8a8a-22b149f44d34: moving from state init -> state preparing
>Thread-15916236::DEBUG::2013-11-20 04:21:29,508::task::495::TaskManager.Task::(_debug) Task 493f6efd-aafa-4662-8a8a-22b149f44d34: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000480890274048', 'lastCheck': 1384921279.845403, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00234389305115', 'lastCheck': 1384921289.1205771, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.028254032135', 'lastCheck': 1384921286.159646, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000440835952759', 'lastCheck': 1384921284.97488, 'valid': True, 'code': 0}}
>Thread-15916236::DEBUG::2013-11-20 04:21:29,509::task::495::TaskManager.Task::(_debug) Task 493f6efd-aafa-4662-8a8a-22b149f44d34: moving from state preparing -> state finished
>Thread-15916236::DEBUG::2013-11-20 04:21:29,510::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916236::DEBUG::2013-11-20 04:21:29,511::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916236::DEBUG::2013-11-20 04:21:29,511::task::495::TaskManager.Task::(_debug) Task 493f6efd-aafa-4662-8a8a-22b149f44d34: ref 0 aborting False
>Thread-15916236::INFO::2013-11-20 04:21:29,512::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000480890274048', 'lastCheck': 1384921279.845403, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00234389305115', 'lastCheck': 1384921289.1205771, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.028254032135', 'lastCheck': 1384921286.159646, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000440835952759', 'lastCheck': 1384921284.97488, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:21:30,927::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:21:31,473::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0787187 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:21:33,488::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:21:34,043::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0916782 s, 11.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:21:36,062::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:21:36,297::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:21:36,302::task::495::TaskManager.Task::(_debug) Task 71bdbc80-669b-46ba-8b31-c2427ce88061: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:21:36,311::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`38beaa56-3a53-402b-8baa-e26fcc8026da`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:21:36,313::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:21:36,317::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:21:36,319::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`38beaa56-3a53-402b-8baa-e26fcc8026da`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:21:36,321::task::495::TaskManager.Task::(_debug) Task 71bdbc80-669b-46ba-8b31-c2427ce88061: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:21:36,322::task::495::TaskManager.Task::(_debug) Task 71bdbc80-669b-46ba-8b31-c2427ce88061: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:21:36,323::task::495::TaskManager.Task::(_debug) Task 71bdbc80-669b-46ba-8b31-c2427ce88061: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:21:36,324::task::495::TaskManager.Task::(_debug) Task 71bdbc80-669b-46ba-8b31-c2427ce88061: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:21:36,325::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:21:36,326::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:21:36,327::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:21:36,328::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:21:36,328::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:21:36,329::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:21:36,330::task::495::TaskManager.Task::(_debug) Task 71bdbc80-669b-46ba-8b31-c2427ce88061: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:21:36,331::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:21:36,595::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0672796 s, 15.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:21:38,609::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915918::INFO::2013-11-20 04:21:38,854::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:21:38,860::task::495::TaskManager.Task::(_debug) Task 44d08c4b-fc22-4b3c-b22b-146ee696dc28: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:21:38,866::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`76b6061b-9634-41fe-9115-00ffaf82fc0a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:21:38,867::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:21:38,868::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:21:38,869::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`76b6061b-9634-41fe-9115-00ffaf82fc0a`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:21:38,872::task::495::TaskManager.Task::(_debug) Task 44d08c4b-fc22-4b3c-b22b-146ee696dc28: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:21:38,873::task::495::TaskManager.Task::(_debug) Task 44d08c4b-fc22-4b3c-b22b-146ee696dc28: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:21:38,874::task::495::TaskManager.Task::(_debug) Task 44d08c4b-fc22-4b3c-b22b-146ee696dc28: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:21:38,875::task::495::TaskManager.Task::(_debug) Task 44d08c4b-fc22-4b3c-b22b-146ee696dc28: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:21:38,875::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:21:38,876::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:21:38,877::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:21:38,878::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:21:38,878::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:21:38,879::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:21:38,879::task::495::TaskManager.Task::(_debug) Task 44d08c4b-fc22-4b3c-b22b-146ee696dc28: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:21:38,880::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:21:39,201::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0909043 s, 11.3 MB/s\n'; <rc> = 0
>Thread-15916241::DEBUG::2013-11-20 04:21:39,516::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916241::INFO::2013-11-20 04:21:39,517::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916241::DEBUG::2013-11-20 04:21:39,518::task::495::TaskManager.Task::(_debug) Task 2142bed5-19ef-4165-b6db-4034d74858c8: moving from state init -> state preparing
>Thread-15916241::DEBUG::2013-11-20 04:21:39,519::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916241::DEBUG::2013-11-20 04:21:39,519::task::495::TaskManager.Task::(_debug) Task 2142bed5-19ef-4165-b6db-4034d74858c8: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916241::DEBUG::2013-11-20 04:21:39,520::task::495::TaskManager.Task::(_debug) Task 2142bed5-19ef-4165-b6db-4034d74858c8: moving from state preparing -> state finished
>Thread-15916241::DEBUG::2013-11-20 04:21:39,520::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916241::DEBUG::2013-11-20 04:21:39,521::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916241::DEBUG::2013-11-20 04:21:39,522::task::495::TaskManager.Task::(_debug) Task 2142bed5-19ef-4165-b6db-4034d74858c8: ref 0 aborting False
>Thread-15916241::INFO::2013-11-20 04:21:39,522::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916242::DEBUG::2013-11-20 04:21:39,526::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916242::INFO::2013-11-20 04:21:39,527::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916242::DEBUG::2013-11-20 04:21:39,528::task::495::TaskManager.Task::(_debug) Task e267e895-83c5-4baf-88ea-402cb2270360: moving from state init -> state preparing
>Thread-15916242::DEBUG::2013-11-20 04:21:39,529::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2e3736ed-d8e9-4a96-8292-783fe725b09a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916242::DEBUG::2013-11-20 04:21:39,530::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916242::DEBUG::2013-11-20 04:21:39,531::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916242::DEBUG::2013-11-20 04:21:39,532::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2e3736ed-d8e9-4a96-8292-783fe725b09a`::Granted request
>Thread-15916242::DEBUG::2013-11-20 04:21:39,533::task::495::TaskManager.Task::(_debug) Task e267e895-83c5-4baf-88ea-402cb2270360: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916242::DEBUG::2013-11-20 04:21:39,534::task::495::TaskManager.Task::(_debug) Task e267e895-83c5-4baf-88ea-402cb2270360: ref 1 aborting False
>Thread-15916242::WARNING::2013-11-20 04:21:39,535::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916242::DEBUG::2013-11-20 04:21:39,536::task::495::TaskManager.Task::(_debug) Task e267e895-83c5-4baf-88ea-402cb2270360: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916242::DEBUG::2013-11-20 04:21:39,537::task::495::TaskManager.Task::(_debug) Task e267e895-83c5-4baf-88ea-402cb2270360: moving from state preparing -> state finished
>Thread-15916242::DEBUG::2013-11-20 04:21:39,538::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916242::DEBUG::2013-11-20 04:21:39,538::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916242::DEBUG::2013-11-20 04:21:39,539::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916242::DEBUG::2013-11-20 04:21:39,540::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916242::DEBUG::2013-11-20 04:21:39,540::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916242::DEBUG::2013-11-20 04:21:39,541::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916242::DEBUG::2013-11-20 04:21:39,541::task::495::TaskManager.Task::(_debug) Task e267e895-83c5-4baf-88ea-402cb2270360: ref 0 aborting False
>Thread-15916242::INFO::2013-11-20 04:21:39,542::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:21:41,213::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:21:41,794::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0774018 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:21:43,811::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915863::INFO::2013-11-20 04:21:44,005::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:21:44,007::task::495::TaskManager.Task::(_debug) Task 94fb8a66-2122-41cd-8784-23c453f34a42: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:21:44,010::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`468a711a-0af5-4b0f-bf87-3490b341cf21`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:21:44,011::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:21:44,013::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:21:44,013::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`468a711a-0af5-4b0f-bf87-3490b341cf21`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:21:44,017::task::495::TaskManager.Task::(_debug) Task 94fb8a66-2122-41cd-8784-23c453f34a42: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:21:44,019::task::495::TaskManager.Task::(_debug) Task 94fb8a66-2122-41cd-8784-23c453f34a42: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:21:44,020::task::495::TaskManager.Task::(_debug) Task 94fb8a66-2122-41cd-8784-23c453f34a42: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:21:44,022::task::495::TaskManager.Task::(_debug) Task 94fb8a66-2122-41cd-8784-23c453f34a42: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:21:44,022::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:21:44,024::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:21:44,025::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:21:44,025::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:21:44,026::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:21:44,027::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:21:44,027::task::495::TaskManager.Task::(_debug) Task 94fb8a66-2122-41cd-8784-23c453f34a42: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:21:44,028::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:21:44,336::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0853328 s, 12.0 MB/s\n'; <rc> = 0
>Thread-15916244::INFO::2013-11-20 04:21:45,407::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916244::DEBUG::2013-11-20 04:21:45,408::task::495::TaskManager.Task::(_debug) Task 3634b06e-db2a-4076-8c1c-b0e0a86d710f: moving from state init -> state preparing
>Thread-15916244::DEBUG::2013-11-20 04:21:45,409::task::495::TaskManager.Task::(_debug) Task 3634b06e-db2a-4076-8c1c-b0e0a86d710f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000356197357178', 'lastCheck': 1384921300.272012, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00373506546021', 'lastCheck': 1384921299.27649, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00227785110474', 'lastCheck': 1384921296.316931, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000350952148438', 'lastCheck': 1384921295.1385479, 'valid': True, 'code': 0}}
>Thread-15916244::DEBUG::2013-11-20 04:21:45,410::task::495::TaskManager.Task::(_debug) Task 3634b06e-db2a-4076-8c1c-b0e0a86d710f: moving from state preparing -> state finished
>Thread-15916244::DEBUG::2013-11-20 04:21:45,411::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916244::DEBUG::2013-11-20 04:21:45,411::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916244::DEBUG::2013-11-20 04:21:45,412::task::495::TaskManager.Task::(_debug) Task 3634b06e-db2a-4076-8c1c-b0e0a86d710f: ref 0 aborting False
>Thread-15916244::INFO::2013-11-20 04:21:45,413::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000356197357178', 'lastCheck': 1384921300.272012, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00373506546021', 'lastCheck': 1384921299.27649, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00227785110474', 'lastCheck': 1384921296.316931, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000350952148438', 'lastCheck': 1384921295.1385479, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:21:46,351::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:21:46,893::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0812726 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:21:48,909::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913518::INFO::2013-11-20 04:21:49,131::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:21:49,132::task::495::TaskManager.Task::(_debug) Task 198abb3e-36f1-438b-894d-fd92d61f508f: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:21:49,134::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ca11615c-5617-4826-bd39-d3d0a159bf3b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:21:49,135::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:21:49,136::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:21:49,136::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ca11615c-5617-4826-bd39-d3d0a159bf3b`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:21:49,137::task::495::TaskManager.Task::(_debug) Task 198abb3e-36f1-438b-894d-fd92d61f508f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:21:49,137::task::495::TaskManager.Task::(_debug) Task 198abb3e-36f1-438b-894d-fd92d61f508f: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:21:49,138::task::495::TaskManager.Task::(_debug) Task 198abb3e-36f1-438b-894d-fd92d61f508f: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:21:49,138::task::495::TaskManager.Task::(_debug) Task 198abb3e-36f1-438b-894d-fd92d61f508f: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:21:49,139::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:21:49,139::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:21:49,139::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:21:49,140::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:21:49,140::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:21:49,140::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:21:49,141::task::495::TaskManager.Task::(_debug) Task 198abb3e-36f1-438b-894d-fd92d61f508f: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:21:49,141::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:21:49,142::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:21:49,142::task::495::TaskManager.Task::(_debug) Task f8a5e109-8d7d-4c35-96a2-f26413ec840a: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:21:49,143::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`eb0afbcc-d1ad-4fb2-85a4-2894f2b62f70`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:21:49,143::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:21:49,144::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:21:49,144::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`eb0afbcc-d1ad-4fb2-85a4-2894f2b62f70`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:21:49,145::task::495::TaskManager.Task::(_debug) Task f8a5e109-8d7d-4c35-96a2-f26413ec840a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:21:49,145::task::495::TaskManager.Task::(_debug) Task f8a5e109-8d7d-4c35-96a2-f26413ec840a: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:21:49,145::task::495::TaskManager.Task::(_debug) Task f8a5e109-8d7d-4c35-96a2-f26413ec840a: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:21:49,146::task::495::TaskManager.Task::(_debug) Task f8a5e109-8d7d-4c35-96a2-f26413ec840a: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:21:49,146::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:21:49,146::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:21:49,147::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:21:49,148::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:21:49,148::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:21:49,149::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:21:49,149::task::495::TaskManager.Task::(_debug) Task f8a5e109-8d7d-4c35-96a2-f26413ec840a: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:21:49,150::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:21:49,441::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0886276 s, 11.6 MB/s\n'; <rc> = 0
>Thread-15916247::DEBUG::2013-11-20 04:21:49,836::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916247::INFO::2013-11-20 04:21:49,838::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916247::DEBUG::2013-11-20 04:21:49,839::task::495::TaskManager.Task::(_debug) Task 5f25bdad-566a-40c9-9e34-1530dd014c04: moving from state init -> state preparing
>Thread-15916247::DEBUG::2013-11-20 04:21:49,840::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916247::DEBUG::2013-11-20 04:21:49,840::task::495::TaskManager.Task::(_debug) Task 5f25bdad-566a-40c9-9e34-1530dd014c04: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916247::DEBUG::2013-11-20 04:21:49,841::task::495::TaskManager.Task::(_debug) Task 5f25bdad-566a-40c9-9e34-1530dd014c04: moving from state preparing -> state finished
>Thread-15916247::DEBUG::2013-11-20 04:21:49,841::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916247::DEBUG::2013-11-20 04:21:49,842::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916247::DEBUG::2013-11-20 04:21:49,842::task::495::TaskManager.Task::(_debug) Task 5f25bdad-566a-40c9-9e34-1530dd014c04: ref 0 aborting False
>Thread-15916247::INFO::2013-11-20 04:21:49,843::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916248::DEBUG::2013-11-20 04:21:49,848::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916248::INFO::2013-11-20 04:21:49,849::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916248::DEBUG::2013-11-20 04:21:49,850::task::495::TaskManager.Task::(_debug) Task 24bd7ac2-f1e4-455e-b0b6-fb622aa194a6: moving from state init -> state preparing
>Thread-15916248::DEBUG::2013-11-20 04:21:49,851::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`414c94da-9fb1-4391-a03e-afc53b0d15db`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916248::DEBUG::2013-11-20 04:21:49,852::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916248::DEBUG::2013-11-20 04:21:49,853::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916248::DEBUG::2013-11-20 04:21:49,853::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`414c94da-9fb1-4391-a03e-afc53b0d15db`::Granted request
>Thread-15916248::DEBUG::2013-11-20 04:21:49,854::task::495::TaskManager.Task::(_debug) Task 24bd7ac2-f1e4-455e-b0b6-fb622aa194a6: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916248::DEBUG::2013-11-20 04:21:49,855::task::495::TaskManager.Task::(_debug) Task 24bd7ac2-f1e4-455e-b0b6-fb622aa194a6: ref 1 aborting False
>Thread-15916248::WARNING::2013-11-20 04:21:49,857::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916248::DEBUG::2013-11-20 04:21:49,858::task::495::TaskManager.Task::(_debug) Task 24bd7ac2-f1e4-455e-b0b6-fb622aa194a6: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916248::DEBUG::2013-11-20 04:21:49,859::task::495::TaskManager.Task::(_debug) Task 24bd7ac2-f1e4-455e-b0b6-fb622aa194a6: moving from state preparing -> state finished
>Thread-15916248::DEBUG::2013-11-20 04:21:49,860::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916248::DEBUG::2013-11-20 04:21:49,860::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916248::DEBUG::2013-11-20 04:21:49,861::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916248::DEBUG::2013-11-20 04:21:49,862::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916248::DEBUG::2013-11-20 04:21:49,862::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916248::DEBUG::2013-11-20 04:21:49,863::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916248::DEBUG::2013-11-20 04:21:49,864::task::495::TaskManager.Task::(_debug) Task 24bd7ac2-f1e4-455e-b0b6-fb622aa194a6: ref 0 aborting False
>Thread-15916248::INFO::2013-11-20 04:21:49,864::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:21:51,452::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 04:21:51,660::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:21:51,664::task::495::TaskManager.Task::(_debug) Task 8cc98512-1860-4e2a-a47d-65cfd481333c: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:21:51,668::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7b894c22-90e4-4450-b0ea-1abfe5b8c0e7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:21:51,668::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:21:51,670::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:21:51,670::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7b894c22-90e4-4450-b0ea-1abfe5b8c0e7`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:21:51,672::task::495::TaskManager.Task::(_debug) Task 8cc98512-1860-4e2a-a47d-65cfd481333c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:21:51,673::task::495::TaskManager.Task::(_debug) Task 8cc98512-1860-4e2a-a47d-65cfd481333c: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:21:51,674::task::495::TaskManager.Task::(_debug) Task 8cc98512-1860-4e2a-a47d-65cfd481333c: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:21:51,675::task::495::TaskManager.Task::(_debug) Task 8cc98512-1860-4e2a-a47d-65cfd481333c: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:21:51,676::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:21:51,676::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:21:51,677::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:21:51,678::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:21:51,679::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:21:51,680::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:21:51,681::task::495::TaskManager.Task::(_debug) Task 8cc98512-1860-4e2a-a47d-65cfd481333c: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:21:51,681::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:21:51,988::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0829899 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:21:54,003::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:21:54,537::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0803416 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:21:56,553::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:21:57,090::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0737932 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:21:59,105::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:21:59,691::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0842158 s, 12.2 MB/s\n'; <rc> = 0
>Thread-15916252::DEBUG::2013-11-20 04:22:00,222::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916252::INFO::2013-11-20 04:22:00,223::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916252::DEBUG::2013-11-20 04:22:00,224::task::495::TaskManager.Task::(_debug) Task b0b1687e-4f65-4eea-b505-5c66141f0d90: moving from state init -> state preparing
>Thread-15916252::DEBUG::2013-11-20 04:22:00,225::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916252::DEBUG::2013-11-20 04:22:00,226::task::495::TaskManager.Task::(_debug) Task b0b1687e-4f65-4eea-b505-5c66141f0d90: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916252::DEBUG::2013-11-20 04:22:00,227::task::495::TaskManager.Task::(_debug) Task b0b1687e-4f65-4eea-b505-5c66141f0d90: moving from state preparing -> state finished
>Thread-15916252::DEBUG::2013-11-20 04:22:00,227::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916252::DEBUG::2013-11-20 04:22:00,228::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916252::DEBUG::2013-11-20 04:22:00,229::task::495::TaskManager.Task::(_debug) Task b0b1687e-4f65-4eea-b505-5c66141f0d90: ref 0 aborting False
>Thread-15916252::INFO::2013-11-20 04:22:00,229::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916253::DEBUG::2013-11-20 04:22:00,234::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916253::INFO::2013-11-20 04:22:00,235::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916253::DEBUG::2013-11-20 04:22:00,236::task::495::TaskManager.Task::(_debug) Task 9dafdeaa-3808-4c14-8f24-c31eb23b83aa: moving from state init -> state preparing
>Thread-15916253::DEBUG::2013-11-20 04:22:00,237::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`de99f1fc-85bf-4498-9229-4911915f99f1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916253::DEBUG::2013-11-20 04:22:00,238::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916253::DEBUG::2013-11-20 04:22:00,239::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916253::DEBUG::2013-11-20 04:22:00,240::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`de99f1fc-85bf-4498-9229-4911915f99f1`::Granted request
>Thread-15916253::DEBUG::2013-11-20 04:22:00,241::task::495::TaskManager.Task::(_debug) Task 9dafdeaa-3808-4c14-8f24-c31eb23b83aa: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916253::DEBUG::2013-11-20 04:22:00,241::task::495::TaskManager.Task::(_debug) Task 9dafdeaa-3808-4c14-8f24-c31eb23b83aa: ref 1 aborting False
>Thread-15916253::WARNING::2013-11-20 04:22:00,244::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916253::DEBUG::2013-11-20 04:22:00,245::task::495::TaskManager.Task::(_debug) Task 9dafdeaa-3808-4c14-8f24-c31eb23b83aa: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916253::DEBUG::2013-11-20 04:22:00,246::task::495::TaskManager.Task::(_debug) Task 9dafdeaa-3808-4c14-8f24-c31eb23b83aa: moving from state preparing -> state finished
>Thread-15916253::DEBUG::2013-11-20 04:22:00,247::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916253::DEBUG::2013-11-20 04:22:00,247::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916253::DEBUG::2013-11-20 04:22:00,248::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916253::DEBUG::2013-11-20 04:22:00,249::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916253::DEBUG::2013-11-20 04:22:00,250::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916253::DEBUG::2013-11-20 04:22:00,250::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916253::DEBUG::2013-11-20 04:22:00,251::task::495::TaskManager.Task::(_debug) Task 9dafdeaa-3808-4c14-8f24-c31eb23b83aa: ref 0 aborting False
>Thread-15916253::INFO::2013-11-20 04:22:00,252::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916254::INFO::2013-11-20 04:22:01,212::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916254::DEBUG::2013-11-20 04:22:01,213::task::495::TaskManager.Task::(_debug) Task c3b81c36-99dd-4203-8219-9929f421d2bb: moving from state init -> state preparing
>Thread-15916254::DEBUG::2013-11-20 04:22:01,213::task::495::TaskManager.Task::(_debug) Task c3b81c36-99dd-4203-8219-9929f421d2bb: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000359058380127', 'lastCheck': 1384921320.767175, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00624203681946', 'lastCheck': 1384921319.776423, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00171804428101', 'lastCheck': 1384921316.7899849, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000449180603027', 'lastCheck': 1384921315.658911, 'valid': True, 'code': 0}}
>Thread-15916254::DEBUG::2013-11-20 04:22:01,214::task::495::TaskManager.Task::(_debug) Task c3b81c36-99dd-4203-8219-9929f421d2bb: moving from state preparing -> state finished
>Thread-15916254::DEBUG::2013-11-20 04:22:01,215::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916254::DEBUG::2013-11-20 04:22:01,215::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916254::DEBUG::2013-11-20 04:22:01,216::task::495::TaskManager.Task::(_debug) Task c3b81c36-99dd-4203-8219-9929f421d2bb: ref 0 aborting False
>Thread-15916254::INFO::2013-11-20 04:22:01,217::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000359058380127', 'lastCheck': 1384921320.767175, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00624203681946', 'lastCheck': 1384921319.776423, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00171804428101', 'lastCheck': 1384921316.7899849, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000449180603027', 'lastCheck': 1384921315.658911, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:22:01,703::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 04:22:01,982::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:22:01,985::task::495::TaskManager.Task::(_debug) Task 0b7a45fe-495a-47e7-83e9-56c57fc06d6a: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:22:01,988::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ec57340b-669a-47d7-b809-7960743f66d8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:22:01,989::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:22:01,990::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:22:01,991::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ec57340b-669a-47d7-b809-7960743f66d8`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:22:01,992::task::495::TaskManager.Task::(_debug) Task 0b7a45fe-495a-47e7-83e9-56c57fc06d6a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:22:01,992::task::495::TaskManager.Task::(_debug) Task 0b7a45fe-495a-47e7-83e9-56c57fc06d6a: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:22:01,993::task::495::TaskManager.Task::(_debug) Task 0b7a45fe-495a-47e7-83e9-56c57fc06d6a: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:22:01,994::task::495::TaskManager.Task::(_debug) Task 0b7a45fe-495a-47e7-83e9-56c57fc06d6a: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:22:01,994::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:22:01,995::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:22:01,995::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:22:01,996::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:22:01,997::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:22:01,997::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:22:01,998::task::495::TaskManager.Task::(_debug) Task 0b7a45fe-495a-47e7-83e9-56c57fc06d6a: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:22:01,998::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:22:02,299::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0807405 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:22:04,323::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:22:04,770::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0788327 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:22:06,786::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 04:22:07,024::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:22:07,026::task::495::TaskManager.Task::(_debug) Task 8df1ee41-77d7-4973-866d-12454c4aece6: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:22:07,036::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5139ad1d-d8ac-4b34-958d-638e3c602e31`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:22:07,037::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:22:07,038::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:22:07,039::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5139ad1d-d8ac-4b34-958d-638e3c602e31`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:22:07,040::task::495::TaskManager.Task::(_debug) Task 8df1ee41-77d7-4973-866d-12454c4aece6: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:22:07,041::task::495::TaskManager.Task::(_debug) Task 8df1ee41-77d7-4973-866d-12454c4aece6: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:22:07,041::task::495::TaskManager.Task::(_debug) Task 8df1ee41-77d7-4973-866d-12454c4aece6: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:22:07,042::task::495::TaskManager.Task::(_debug) Task 8df1ee41-77d7-4973-866d-12454c4aece6: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:22:07,043::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:22:07,043::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:22:07,044::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:22:07,045::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:22:07,046::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:22:07,046::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:22:07,047::task::495::TaskManager.Task::(_debug) Task 8df1ee41-77d7-4973-866d-12454c4aece6: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:22:07,048::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:22:07,330::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.077093 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:22:09,346::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:22:09,922::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0879182 s, 11.6 MB/s\n'; <rc> = 0
>Thread-15916258::DEBUG::2013-11-20 04:22:10,650::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916258::INFO::2013-11-20 04:22:10,652::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916258::DEBUG::2013-11-20 04:22:10,653::task::495::TaskManager.Task::(_debug) Task 82108bc9-aeb8-46b3-a9cb-6ba9bf7f1095: moving from state init -> state preparing
>Thread-15916258::DEBUG::2013-11-20 04:22:10,653::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916258::DEBUG::2013-11-20 04:22:10,654::task::495::TaskManager.Task::(_debug) Task 82108bc9-aeb8-46b3-a9cb-6ba9bf7f1095: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916258::DEBUG::2013-11-20 04:22:10,655::task::495::TaskManager.Task::(_debug) Task 82108bc9-aeb8-46b3-a9cb-6ba9bf7f1095: moving from state preparing -> state finished
>Thread-15916258::DEBUG::2013-11-20 04:22:10,655::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916258::DEBUG::2013-11-20 04:22:10,656::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916258::DEBUG::2013-11-20 04:22:10,657::task::495::TaskManager.Task::(_debug) Task 82108bc9-aeb8-46b3-a9cb-6ba9bf7f1095: ref 0 aborting False
>Thread-15916258::INFO::2013-11-20 04:22:10,657::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916259::DEBUG::2013-11-20 04:22:10,662::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916259::INFO::2013-11-20 04:22:10,662::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916259::DEBUG::2013-11-20 04:22:10,663::task::495::TaskManager.Task::(_debug) Task d4396e6a-9e7e-47ac-b2c4-5547fc6b3999: moving from state init -> state preparing
>Thread-15916259::DEBUG::2013-11-20 04:22:10,664::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`09d4a622-6c42-4e00-ba2a-8e9baacfcfd7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916259::DEBUG::2013-11-20 04:22:10,665::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916259::DEBUG::2013-11-20 04:22:10,666::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916259::DEBUG::2013-11-20 04:22:10,666::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`09d4a622-6c42-4e00-ba2a-8e9baacfcfd7`::Granted request
>Thread-15916259::DEBUG::2013-11-20 04:22:10,667::task::495::TaskManager.Task::(_debug) Task d4396e6a-9e7e-47ac-b2c4-5547fc6b3999: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916259::DEBUG::2013-11-20 04:22:10,668::task::495::TaskManager.Task::(_debug) Task d4396e6a-9e7e-47ac-b2c4-5547fc6b3999: ref 1 aborting False
>Thread-15916259::WARNING::2013-11-20 04:22:10,670::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916259::DEBUG::2013-11-20 04:22:10,671::task::495::TaskManager.Task::(_debug) Task d4396e6a-9e7e-47ac-b2c4-5547fc6b3999: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916259::DEBUG::2013-11-20 04:22:10,671::task::495::TaskManager.Task::(_debug) Task d4396e6a-9e7e-47ac-b2c4-5547fc6b3999: moving from state preparing -> state finished
>Thread-15916259::DEBUG::2013-11-20 04:22:10,672::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916259::DEBUG::2013-11-20 04:22:10,673::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916259::DEBUG::2013-11-20 04:22:10,676::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916259::DEBUG::2013-11-20 04:22:10,677::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916259::DEBUG::2013-11-20 04:22:10,677::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916259::DEBUG::2013-11-20 04:22:10,678::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916259::DEBUG::2013-11-20 04:22:10,679::task::495::TaskManager.Task::(_debug) Task d4396e6a-9e7e-47ac-b2c4-5547fc6b3999: ref 0 aborting False
>Thread-15916259::INFO::2013-11-20 04:22:10,680::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:22:11,938::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 04:22:12,167::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:22:12,169::task::495::TaskManager.Task::(_debug) Task 7a437393-9684-4b7f-aa71-d88df46e15fd: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:22:12,173::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b7e0d623-1189-454b-805c-7702534b838b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:22:12,175::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:22:12,176::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:22:12,177::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b7e0d623-1189-454b-805c-7702534b838b`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:22:12,178::task::495::TaskManager.Task::(_debug) Task 7a437393-9684-4b7f-aa71-d88df46e15fd: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:22:12,179::task::495::TaskManager.Task::(_debug) Task 7a437393-9684-4b7f-aa71-d88df46e15fd: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:22:12,180::task::495::TaskManager.Task::(_debug) Task 7a437393-9684-4b7f-aa71-d88df46e15fd: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:22:12,180::task::495::TaskManager.Task::(_debug) Task 7a437393-9684-4b7f-aa71-d88df46e15fd: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:22:12,181::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:22:12,182::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:22:12,182::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:22:12,183::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:22:12,184::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:22:12,184::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:22:12,185::task::495::TaskManager.Task::(_debug) Task 7a437393-9684-4b7f-aa71-d88df46e15fd: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:22:12,185::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:22:12,510::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.082258 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:22:14,530::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:22:15,092::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0827169 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:22:17,109::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:22:17,635::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0785513 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15916262::INFO::2013-11-20 04:22:18,003::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916262::DEBUG::2013-11-20 04:22:18,004::task::495::TaskManager.Task::(_debug) Task 1e8fd9fd-8fb7-4d9d-8895-94c5bc6576b7: moving from state init -> state preparing
>Thread-15916262::DEBUG::2013-11-20 04:22:18,005::task::495::TaskManager.Task::(_debug) Task 1e8fd9fd-8fb7-4d9d-8895-94c5bc6576b7: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000371932983398', 'lastCheck': 1384921331.0023911, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00127410888672', 'lastCheck': 1384921330.0038249, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0141038894653', 'lastCheck': 1384921337.339771, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000475883483887', 'lastCheck': 1384921336.162719, 'valid': True, 'code': 0}}
>Thread-15916262::DEBUG::2013-11-20 04:22:18,005::task::495::TaskManager.Task::(_debug) Task 1e8fd9fd-8fb7-4d9d-8895-94c5bc6576b7: moving from state preparing -> state finished
>Thread-15916262::DEBUG::2013-11-20 04:22:18,006::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916262::DEBUG::2013-11-20 04:22:18,007::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916262::DEBUG::2013-11-20 04:22:18,007::task::495::TaskManager.Task::(_debug) Task 1e8fd9fd-8fb7-4d9d-8895-94c5bc6576b7: ref 0 aborting False
>Thread-15916262::INFO::2013-11-20 04:22:18,008::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000371932983398', 'lastCheck': 1384921331.0023911, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00127410888672', 'lastCheck': 1384921330.0038249, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0141038894653', 'lastCheck': 1384921337.339771, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000475883483887', 'lastCheck': 1384921336.162719, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:22:19,651::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 04:22:19,862::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:22:19,865::task::495::TaskManager.Task::(_debug) Task 1eb84a69-6d7b-40b6-81e3-aaed501f2797: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:22:19,876::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`836d880c-44c9-416a-a7d6-92a06f1f3255`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:22:19,877::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:22:19,879::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:22:19,880::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`836d880c-44c9-416a-a7d6-92a06f1f3255`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:22:19,882::task::495::TaskManager.Task::(_debug) Task 1eb84a69-6d7b-40b6-81e3-aaed501f2797: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 04:22:19,883::task::495::TaskManager.Task::(_debug) Task 1eb84a69-6d7b-40b6-81e3-aaed501f2797: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:22:19,885::task::495::TaskManager.Task::(_debug) Task 1eb84a69-6d7b-40b6-81e3-aaed501f2797: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 04:22:19,887::task::495::TaskManager.Task::(_debug) Task 1eb84a69-6d7b-40b6-81e3-aaed501f2797: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:22:19,888::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:22:19,889::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:22:19,891::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 04:22:19,893::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:22:19,893::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:22:19,894::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:22:19,895::task::495::TaskManager.Task::(_debug) Task 1eb84a69-6d7b-40b6-81e3-aaed501f2797: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 04:22:19,896::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 04:22:20,208::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0788298 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15916264::DEBUG::2013-11-20 04:22:21,071::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916264::INFO::2013-11-20 04:22:21,073::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916264::DEBUG::2013-11-20 04:22:21,073::task::495::TaskManager.Task::(_debug) Task 58076077-9a27-466b-9135-c2f904674349: moving from state init -> state preparing
>Thread-15916264::DEBUG::2013-11-20 04:22:21,074::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916264::DEBUG::2013-11-20 04:22:21,075::task::495::TaskManager.Task::(_debug) Task 58076077-9a27-466b-9135-c2f904674349: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916264::DEBUG::2013-11-20 04:22:21,076::task::495::TaskManager.Task::(_debug) Task 58076077-9a27-466b-9135-c2f904674349: moving from state preparing -> state finished
>Thread-15916264::DEBUG::2013-11-20 04:22:21,076::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916264::DEBUG::2013-11-20 04:22:21,077::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916264::DEBUG::2013-11-20 04:22:21,078::task::495::TaskManager.Task::(_debug) Task 58076077-9a27-466b-9135-c2f904674349: ref 0 aborting False
>Thread-15916264::INFO::2013-11-20 04:22:21,078::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916265::DEBUG::2013-11-20 04:22:21,083::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916265::INFO::2013-11-20 04:22:21,084::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916265::DEBUG::2013-11-20 04:22:21,085::task::495::TaskManager.Task::(_debug) Task a942359c-6861-470d-8a59-92c272b7e257: moving from state init -> state preparing
>Thread-15916265::DEBUG::2013-11-20 04:22:21,086::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e2063b6d-38b1-4215-b8e9-e1da93cb2c46`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916265::DEBUG::2013-11-20 04:22:21,087::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916265::DEBUG::2013-11-20 04:22:21,087::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916265::DEBUG::2013-11-20 04:22:21,088::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e2063b6d-38b1-4215-b8e9-e1da93cb2c46`::Granted request
>Thread-15916265::DEBUG::2013-11-20 04:22:21,089::task::495::TaskManager.Task::(_debug) Task a942359c-6861-470d-8a59-92c272b7e257: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916265::DEBUG::2013-11-20 04:22:21,090::task::495::TaskManager.Task::(_debug) Task a942359c-6861-470d-8a59-92c272b7e257: ref 1 aborting False
>Thread-15916265::WARNING::2013-11-20 04:22:21,092::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916265::DEBUG::2013-11-20 04:22:21,093::task::495::TaskManager.Task::(_debug) Task a942359c-6861-470d-8a59-92c272b7e257: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916265::DEBUG::2013-11-20 04:22:21,094::task::495::TaskManager.Task::(_debug) Task a942359c-6861-470d-8a59-92c272b7e257: moving from state preparing -> state finished
>Thread-15916265::DEBUG::2013-11-20 04:22:21,095::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916265::DEBUG::2013-11-20 04:22:21,095::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916265::DEBUG::2013-11-20 04:22:21,096::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916265::DEBUG::2013-11-20 04:22:21,097::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916265::DEBUG::2013-11-20 04:22:21,097::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916265::DEBUG::2013-11-20 04:22:21,098::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916265::DEBUG::2013-11-20 04:22:21,098::task::495::TaskManager.Task::(_debug) Task a942359c-6861-470d-8a59-92c272b7e257: ref 0 aborting False
>Thread-15916265::INFO::2013-11-20 04:22:21,099::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:22:22,222::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 04:22:22,457::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:22:22,460::task::495::TaskManager.Task::(_debug) Task 83147603-901a-478e-9acd-eebe89e31476: moving from state init -> state preparing
>Thread-15914435::INFO::2013-11-20 04:22:22,461::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15913546::DEBUG::2013-11-20 04:22:22,464::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`ea693444-9a4d-43a1-b3ce-7f1cbfb4c263`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:22:22,465::task::495::TaskManager.Task::(_debug) Task f702cba4-7b56-493c-a203-177bb6d43fdc: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:22:22,465::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:22:22,467::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ae1977d9-a5c9-4c32-b1d8-85ee59671b4b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:22:22,468::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:22:22,469::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:22:22,471::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`ea693444-9a4d-43a1-b3ce-7f1cbfb4c263`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:22:22,473::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:22:22,473::task::495::TaskManager.Task::(_debug) Task 83147603-901a-478e-9acd-eebe89e31476: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15914435::DEBUG::2013-11-20 04:22:22,474::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ae1977d9-a5c9-4c32-b1d8-85ee59671b4b`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:22:22,475::task::495::TaskManager.Task::(_debug) Task 83147603-901a-478e-9acd-eebe89e31476: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:22:22,477::task::495::TaskManager.Task::(_debug) Task f702cba4-7b56-493c-a203-177bb6d43fdc: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913546::DEBUG::2013-11-20 04:22:22,478::task::495::TaskManager.Task::(_debug) Task 83147603-901a-478e-9acd-eebe89e31476: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914435::DEBUG::2013-11-20 04:22:22,479::task::495::TaskManager.Task::(_debug) Task f702cba4-7b56-493c-a203-177bb6d43fdc: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:22:22,480::task::495::TaskManager.Task::(_debug) Task 83147603-901a-478e-9acd-eebe89e31476: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:22:22,480::task::495::TaskManager.Task::(_debug) Task f702cba4-7b56-493c-a203-177bb6d43fdc: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15913546::DEBUG::2013-11-20 04:22:22,482::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:22:22,483::task::495::TaskManager.Task::(_debug) Task f702cba4-7b56-493c-a203-177bb6d43fdc: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:22:22,483::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:22:22,484::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:22:22,486::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15914435::DEBUG::2013-11-20 04:22:22,486::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:22:22,488::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913546::DEBUG::2013-11-20 04:22:22,488::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:22:22,489::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:22:22,490::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:22:22,491::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:22:22,491::task::495::TaskManager.Task::(_debug) Task 83147603-901a-478e-9acd-eebe89e31476: ref 0 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:22:22,492::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913546::INFO::2013-11-20 04:22:22,493::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914435::DEBUG::2013-11-20 04:22:22,493::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913546::INFO::2013-11-20 04:22:22,494::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15914435::DEBUG::2013-11-20 04:22:22,495::task::495::TaskManager.Task::(_debug) Task f702cba4-7b56-493c-a203-177bb6d43fdc: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:22:22,496::task::495::TaskManager.Task::(_debug) Task d831a8d8-6cab-43f6-b10b-cac3d0e80f8b: moving from state init -> state preparing
>Thread-15914435::INFO::2013-11-20 04:22:22,497::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15913546::DEBUG::2013-11-20 04:22:22,499::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e78abfc8-40f7-479c-9dcb-2bc276cc48f4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:22:22,499::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:22:22,500::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:22:22,502::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e78abfc8-40f7-479c-9dcb-2bc276cc48f4`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:22:22,503::task::495::TaskManager.Task::(_debug) Task d831a8d8-6cab-43f6-b10b-cac3d0e80f8b: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:22:22,503::task::495::TaskManager.Task::(_debug) Task d831a8d8-6cab-43f6-b10b-cac3d0e80f8b: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:22:22,504::task::495::TaskManager.Task::(_debug) Task d831a8d8-6cab-43f6-b10b-cac3d0e80f8b: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:22:22,505::task::495::TaskManager.Task::(_debug) Task d831a8d8-6cab-43f6-b10b-cac3d0e80f8b: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:22:22,505::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:22:22,506::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:22:22,507::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:22:22,507::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:22:22,508::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:22:22,508::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:22:22,510::task::495::TaskManager.Task::(_debug) Task d831a8d8-6cab-43f6-b10b-cac3d0e80f8b: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:22:22,510::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:22:22,511::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:22:22,512::task::495::TaskManager.Task::(_debug) Task dc342536-3981-446e-9acd-abd3f97e97f2: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:22:22,513::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e02d4057-4354-4b1a-89a4-582551343d47`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:22:22,514::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:22:22,514::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:22:22,515::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e02d4057-4354-4b1a-89a4-582551343d47`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:22:22,516::task::495::TaskManager.Task::(_debug) Task dc342536-3981-446e-9acd-abd3f97e97f2: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:22:22,517::task::495::TaskManager.Task::(_debug) Task dc342536-3981-446e-9acd-abd3f97e97f2: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:22:22,517::task::495::TaskManager.Task::(_debug) Task dc342536-3981-446e-9acd-abd3f97e97f2: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:22:22,518::task::495::TaskManager.Task::(_debug) Task dc342536-3981-446e-9acd-abd3f97e97f2: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:22:22,519::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:22:22,519::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:22:22,520::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:22:22,520::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:22:22,521::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:22:22,525::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:22:22,526::task::495::TaskManager.Task::(_debug) Task dc342536-3981-446e-9acd-abd3f97e97f2: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:22:22,526::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:22:22,527::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:22:22,528::task::495::TaskManager.Task::(_debug) Task dd6487b4-bd23-4b72-bff0-a98c38774adf: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:22:22,529::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`3b704af2-fb34-4f48-8f51-c625a26ca32d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:22:22,530::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:22:22,530::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:22:22,531::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`3b704af2-fb34-4f48-8f51-c625a26ca32d`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:22:22,532::task::495::TaskManager.Task::(_debug) Task dd6487b4-bd23-4b72-bff0-a98c38774adf: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:22:22,533::task::495::TaskManager.Task::(_debug) Task dd6487b4-bd23-4b72-bff0-a98c38774adf: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:22:22,533::task::495::TaskManager.Task::(_debug) Task dd6487b4-bd23-4b72-bff0-a98c38774adf: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:22:22,534::task::495::TaskManager.Task::(_debug) Task dd6487b4-bd23-4b72-bff0-a98c38774adf: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:22:22,534::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:22:22,535::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:22:22,536::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:22:22,536::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:22:22,537::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:22:22,537::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:22:22,538::task::495::TaskManager.Task::(_debug) Task dd6487b4-bd23-4b72-bff0-a98c38774adf: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:22:22,539::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:22:22,773::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0792745 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:22:24,790::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:22:25,329::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0811694 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:22:27,344::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:22:27,925::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0925852 s, 11.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:22:29,942::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:22:30,477::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0758449 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15916270::DEBUG::2013-11-20 04:22:31,474::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916270::INFO::2013-11-20 04:22:31,475::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916270::DEBUG::2013-11-20 04:22:31,476::task::495::TaskManager.Task::(_debug) Task a3f2a9a1-d845-4d0a-bc34-39c41698ee72: moving from state init -> state preparing
>Thread-15916270::DEBUG::2013-11-20 04:22:31,477::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916270::DEBUG::2013-11-20 04:22:31,478::task::495::TaskManager.Task::(_debug) Task a3f2a9a1-d845-4d0a-bc34-39c41698ee72: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916270::DEBUG::2013-11-20 04:22:31,478::task::495::TaskManager.Task::(_debug) Task a3f2a9a1-d845-4d0a-bc34-39c41698ee72: moving from state preparing -> state finished
>Thread-15916270::DEBUG::2013-11-20 04:22:31,479::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916270::DEBUG::2013-11-20 04:22:31,480::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916270::DEBUG::2013-11-20 04:22:31,480::task::495::TaskManager.Task::(_debug) Task a3f2a9a1-d845-4d0a-bc34-39c41698ee72: ref 0 aborting False
>Thread-15916270::INFO::2013-11-20 04:22:31,481::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916271::DEBUG::2013-11-20 04:22:31,486::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916271::INFO::2013-11-20 04:22:31,488::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916271::DEBUG::2013-11-20 04:22:31,488::task::495::TaskManager.Task::(_debug) Task 0c937eed-316f-45bf-9f00-53a9536fa576: moving from state init -> state preparing
>Thread-15916271::DEBUG::2013-11-20 04:22:31,490::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7659bfb2-7aec-4736-b72d-2332a5ab62a6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916271::DEBUG::2013-11-20 04:22:31,491::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916271::DEBUG::2013-11-20 04:22:31,492::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916271::DEBUG::2013-11-20 04:22:31,492::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7659bfb2-7aec-4736-b72d-2332a5ab62a6`::Granted request
>Thread-15916271::DEBUG::2013-11-20 04:22:31,494::task::495::TaskManager.Task::(_debug) Task 0c937eed-316f-45bf-9f00-53a9536fa576: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916271::DEBUG::2013-11-20 04:22:31,494::task::495::TaskManager.Task::(_debug) Task 0c937eed-316f-45bf-9f00-53a9536fa576: ref 1 aborting False
>Thread-15916271::WARNING::2013-11-20 04:22:31,497::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916271::DEBUG::2013-11-20 04:22:31,498::task::495::TaskManager.Task::(_debug) Task 0c937eed-316f-45bf-9f00-53a9536fa576: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916271::DEBUG::2013-11-20 04:22:31,499::task::495::TaskManager.Task::(_debug) Task 0c937eed-316f-45bf-9f00-53a9536fa576: moving from state preparing -> state finished
>Thread-15916271::DEBUG::2013-11-20 04:22:31,500::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916271::DEBUG::2013-11-20 04:22:31,500::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916271::DEBUG::2013-11-20 04:22:31,501::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916271::DEBUG::2013-11-20 04:22:31,502::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916271::DEBUG::2013-11-20 04:22:31,503::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916271::DEBUG::2013-11-20 04:22:31,503::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916271::DEBUG::2013-11-20 04:22:31,504::task::495::TaskManager.Task::(_debug) Task 0c937eed-316f-45bf-9f00-53a9536fa576: ref 0 aborting False
>Thread-15916271::INFO::2013-11-20 04:22:31,505::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:22:32,490::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:22:33,069::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0851451 s, 12.0 MB/s\n'; <rc> = 0
>Thread-15916272::INFO::2013-11-20 04:22:33,812::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916272::DEBUG::2013-11-20 04:22:33,813::task::495::TaskManager.Task::(_debug) Task ca52d022-b8e1-4ab5-95d6-1c88d3272464: moving from state init -> state preparing
>Thread-15916272::DEBUG::2013-11-20 04:22:33,814::task::495::TaskManager.Task::(_debug) Task ca52d022-b8e1-4ab5-95d6-1c88d3272464: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00032114982605', 'lastCheck': 1384921351.5700891, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00123286247253', 'lastCheck': 1384921350.5715899, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00190186500549', 'lastCheck': 1384921347.578542, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00042986869812', 'lastCheck': 1384921346.44877, 'valid': True, 'code': 0}}
>Thread-15916272::DEBUG::2013-11-20 04:22:33,815::task::495::TaskManager.Task::(_debug) Task ca52d022-b8e1-4ab5-95d6-1c88d3272464: moving from state preparing -> state finished
>Thread-15916272::DEBUG::2013-11-20 04:22:33,816::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916272::DEBUG::2013-11-20 04:22:33,816::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916272::DEBUG::2013-11-20 04:22:33,818::task::495::TaskManager.Task::(_debug) Task ca52d022-b8e1-4ab5-95d6-1c88d3272464: ref 0 aborting False
>Thread-15916272::INFO::2013-11-20 04:22:33,818::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00032114982605', 'lastCheck': 1384921351.5700891, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00123286247253', 'lastCheck': 1384921350.5715899, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00190186500549', 'lastCheck': 1384921347.578542, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00042986869812', 'lastCheck': 1384921346.44877, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:22:35,079::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:22:35,638::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0748915 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:22:37,655::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:22:37,884::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:22:37,892::task::495::TaskManager.Task::(_debug) Task 462ab8f6-ffbf-465c-ae8c-bb77c36bc4b4: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:22:37,896::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`89471d6f-57f0-429e-a975-ccb468660639`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:22:37,898::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:22:37,901::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:22:37,902::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`89471d6f-57f0-429e-a975-ccb468660639`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:22:37,903::task::495::TaskManager.Task::(_debug) Task 462ab8f6-ffbf-465c-ae8c-bb77c36bc4b4: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:22:37,905::task::495::TaskManager.Task::(_debug) Task 462ab8f6-ffbf-465c-ae8c-bb77c36bc4b4: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:22:37,906::task::495::TaskManager.Task::(_debug) Task 462ab8f6-ffbf-465c-ae8c-bb77c36bc4b4: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:22:37,907::task::495::TaskManager.Task::(_debug) Task 462ab8f6-ffbf-465c-ae8c-bb77c36bc4b4: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:22:37,908::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:22:37,909::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:22:37,910::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:22:37,911::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:22:37,911::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:22:37,912::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:22:37,912::task::495::TaskManager.Task::(_debug) Task 462ab8f6-ffbf-465c-ae8c-bb77c36bc4b4: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:22:37,913::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:22:38,223::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0835874 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:22:40,236::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915918::INFO::2013-11-20 04:22:40,468::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:22:40,472::task::495::TaskManager.Task::(_debug) Task 548274cd-f256-430a-90c0-11888d2d1e11: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:22:40,477::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`cbee2909-7888-4b9b-a61a-40f029f16127`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:22:40,481::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:22:40,484::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:22:40,484::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`cbee2909-7888-4b9b-a61a-40f029f16127`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:22:40,486::task::495::TaskManager.Task::(_debug) Task 548274cd-f256-430a-90c0-11888d2d1e11: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:22:40,487::task::495::TaskManager.Task::(_debug) Task 548274cd-f256-430a-90c0-11888d2d1e11: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:22:40,488::task::495::TaskManager.Task::(_debug) Task 548274cd-f256-430a-90c0-11888d2d1e11: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:22:40,489::task::495::TaskManager.Task::(_debug) Task 548274cd-f256-430a-90c0-11888d2d1e11: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:22:40,490::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:22:40,490::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:22:40,491::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:22:40,492::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:22:40,492::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:22:40,493::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:22:40,494::task::495::TaskManager.Task::(_debug) Task 548274cd-f256-430a-90c0-11888d2d1e11: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:22:40,495::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:22:40,807::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0832889 s, 12.3 MB/s\n'; <rc> = 0
>Thread-15916276::DEBUG::2013-11-20 04:22:42,080::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916276::INFO::2013-11-20 04:22:42,081::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916276::DEBUG::2013-11-20 04:22:42,082::task::495::TaskManager.Task::(_debug) Task 3a0eeaa6-be85-48bd-a539-b8c4d87035e8: moving from state init -> state preparing
>Thread-15916276::DEBUG::2013-11-20 04:22:42,083::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916276::DEBUG::2013-11-20 04:22:42,084::task::495::TaskManager.Task::(_debug) Task 3a0eeaa6-be85-48bd-a539-b8c4d87035e8: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916276::DEBUG::2013-11-20 04:22:42,084::task::495::TaskManager.Task::(_debug) Task 3a0eeaa6-be85-48bd-a539-b8c4d87035e8: moving from state preparing -> state finished
>Thread-15916276::DEBUG::2013-11-20 04:22:42,085::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916276::DEBUG::2013-11-20 04:22:42,086::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916276::DEBUG::2013-11-20 04:22:42,086::task::495::TaskManager.Task::(_debug) Task 3a0eeaa6-be85-48bd-a539-b8c4d87035e8: ref 0 aborting False
>Thread-15916276::INFO::2013-11-20 04:22:42,087::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916277::DEBUG::2013-11-20 04:22:42,092::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916277::INFO::2013-11-20 04:22:42,093::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916277::DEBUG::2013-11-20 04:22:42,093::task::495::TaskManager.Task::(_debug) Task eeef898d-59fe-4880-9de1-fae97e04c473: moving from state init -> state preparing
>Thread-15916277::DEBUG::2013-11-20 04:22:42,095::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8392a206-261d-4a19-b0df-36194d268e65`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916277::DEBUG::2013-11-20 04:22:42,095::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916277::DEBUG::2013-11-20 04:22:42,096::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916277::DEBUG::2013-11-20 04:22:42,097::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8392a206-261d-4a19-b0df-36194d268e65`::Granted request
>Thread-15916277::DEBUG::2013-11-20 04:22:42,098::task::495::TaskManager.Task::(_debug) Task eeef898d-59fe-4880-9de1-fae97e04c473: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916277::DEBUG::2013-11-20 04:22:42,098::task::495::TaskManager.Task::(_debug) Task eeef898d-59fe-4880-9de1-fae97e04c473: ref 1 aborting False
>Thread-15916277::WARNING::2013-11-20 04:22:42,100::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916277::DEBUG::2013-11-20 04:22:42,101::task::495::TaskManager.Task::(_debug) Task eeef898d-59fe-4880-9de1-fae97e04c473: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916277::DEBUG::2013-11-20 04:22:42,102::task::495::TaskManager.Task::(_debug) Task eeef898d-59fe-4880-9de1-fae97e04c473: moving from state preparing -> state finished
>Thread-15916277::DEBUG::2013-11-20 04:22:42,102::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916277::DEBUG::2013-11-20 04:22:42,103::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916277::DEBUG::2013-11-20 04:22:42,104::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916277::DEBUG::2013-11-20 04:22:42,104::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916277::DEBUG::2013-11-20 04:22:42,105::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916277::DEBUG::2013-11-20 04:22:42,105::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916277::DEBUG::2013-11-20 04:22:42,106::task::495::TaskManager.Task::(_debug) Task eeef898d-59fe-4880-9de1-fae97e04c473: ref 0 aborting False
>Thread-15916277::INFO::2013-11-20 04:22:42,107::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:22:42,823::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:22:43,335::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0817594 s, 12.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:22:45,352::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915863::INFO::2013-11-20 04:22:45,590::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:22:45,593::task::495::TaskManager.Task::(_debug) Task 53b9a457-1af2-4834-b233-51d75c2c242b: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:22:45,598::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f5c78712-3b96-44af-b3b3-d0810a06b02c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:22:45,600::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:22:45,602::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:22:45,603::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f5c78712-3b96-44af-b3b3-d0810a06b02c`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:22:45,605::task::495::TaskManager.Task::(_debug) Task 53b9a457-1af2-4834-b233-51d75c2c242b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:22:45,607::task::495::TaskManager.Task::(_debug) Task 53b9a457-1af2-4834-b233-51d75c2c242b: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:22:45,608::task::495::TaskManager.Task::(_debug) Task 53b9a457-1af2-4834-b233-51d75c2c242b: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:22:45,609::task::495::TaskManager.Task::(_debug) Task 53b9a457-1af2-4834-b233-51d75c2c242b: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:22:45,611::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:22:45,612::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:22:45,613::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:22:45,615::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:22:45,616::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:22:45,617::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:22:45,618::task::495::TaskManager.Task::(_debug) Task 53b9a457-1af2-4834-b233-51d75c2c242b: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:22:45,619::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:22:45,917::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0847072 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:22:47,934::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:22:48,430::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0490739 s, 20.9 MB/s\n'; <rc> = 0
>Thread-15916280::INFO::2013-11-20 04:22:49,558::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916280::DEBUG::2013-11-20 04:22:49,559::task::495::TaskManager.Task::(_debug) Task 26f243da-050b-4f86-8d1d-62f0b10bc4c4: moving from state init -> state preparing
>Thread-15916280::DEBUG::2013-11-20 04:22:49,559::task::495::TaskManager.Task::(_debug) Task 26f243da-050b-4f86-8d1d-62f0b10bc4c4: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000341176986694', 'lastCheck': 1384921361.8881669, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0166900157928', 'lastCheck': 1384921360.901305, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000895977020264', 'lastCheck': 1384921368.152174, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000429153442383', 'lastCheck': 1384921367.011416, 'valid': True, 'code': 0}}
>Thread-15916280::DEBUG::2013-11-20 04:22:49,560::task::495::TaskManager.Task::(_debug) Task 26f243da-050b-4f86-8d1d-62f0b10bc4c4: moving from state preparing -> state finished
>Thread-15916280::DEBUG::2013-11-20 04:22:49,561::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916280::DEBUG::2013-11-20 04:22:49,561::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916280::DEBUG::2013-11-20 04:22:49,562::task::495::TaskManager.Task::(_debug) Task 26f243da-050b-4f86-8d1d-62f0b10bc4c4: ref 0 aborting False
>Thread-15916280::INFO::2013-11-20 04:22:49,563::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000341176986694', 'lastCheck': 1384921361.8881669, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0166900157928', 'lastCheck': 1384921360.901305, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.000895977020264', 'lastCheck': 1384921368.152174, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000429153442383', 'lastCheck': 1384921367.011416, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:22:50,440::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913518::INFO::2013-11-20 04:22:50,675::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:22:50,676::task::495::TaskManager.Task::(_debug) Task ac1d386f-b779-4ebd-b668-fb337e2c4295: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:22:50,677::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7d786f0e-f1ab-4bdd-94e0-b6caf03f62a8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:22:50,678::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:22:50,678::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:22:50,679::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7d786f0e-f1ab-4bdd-94e0-b6caf03f62a8`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:22:50,679::task::495::TaskManager.Task::(_debug) Task ac1d386f-b779-4ebd-b668-fb337e2c4295: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:22:50,680::task::495::TaskManager.Task::(_debug) Task ac1d386f-b779-4ebd-b668-fb337e2c4295: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:22:50,680::task::495::TaskManager.Task::(_debug) Task ac1d386f-b779-4ebd-b668-fb337e2c4295: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:22:50,681::task::495::TaskManager.Task::(_debug) Task ac1d386f-b779-4ebd-b668-fb337e2c4295: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:22:50,681::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:22:50,681::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:22:50,682::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:22:50,682::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:22:50,683::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:22:50,683::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:22:50,683::task::495::TaskManager.Task::(_debug) Task ac1d386f-b779-4ebd-b668-fb337e2c4295: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:22:50,684::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:22:50,684::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:22:50,685::task::495::TaskManager.Task::(_debug) Task 1c53cf25-ea88-4238-9b4b-62624c7f3180: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:22:50,685::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6af22b9c-bcc5-40bc-8d5f-833713857453`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:22:50,686::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:22:50,686::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:22:50,687::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6af22b9c-bcc5-40bc-8d5f-833713857453`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:22:50,687::task::495::TaskManager.Task::(_debug) Task 1c53cf25-ea88-4238-9b4b-62624c7f3180: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:22:50,687::task::495::TaskManager.Task::(_debug) Task 1c53cf25-ea88-4238-9b4b-62624c7f3180: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:22:50,688::task::495::TaskManager.Task::(_debug) Task 1c53cf25-ea88-4238-9b4b-62624c7f3180: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:22:50,688::task::495::TaskManager.Task::(_debug) Task 1c53cf25-ea88-4238-9b4b-62624c7f3180: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:22:50,689::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:22:50,689::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:22:50,689::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:22:50,690::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:22:50,690::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:22:50,690::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:22:50,691::task::495::TaskManager.Task::(_debug) Task 1c53cf25-ea88-4238-9b4b-62624c7f3180: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:22:50,691::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:22:50,977::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0842354 s, 12.2 MB/s\n'; <rc> = 0
>Thread-15916282::DEBUG::2013-11-20 04:22:52,986::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916282::INFO::2013-11-20 04:22:52,991::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916282::DEBUG::2013-11-20 04:22:52,992::task::495::TaskManager.Task::(_debug) Task 407f8cbc-f03d-4813-8f32-5d0aee70e468: moving from state init -> state preparing
>Dummy-15912651::DEBUG::2013-11-20 04:22:52,993::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916282::DEBUG::2013-11-20 04:22:52,996::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916282::DEBUG::2013-11-20 04:22:53,214::task::495::TaskManager.Task::(_debug) Task 407f8cbc-f03d-4813-8f32-5d0aee70e468: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913658::INFO::2013-11-20 04:22:53,219::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15916282::DEBUG::2013-11-20 04:22:53,224::task::495::TaskManager.Task::(_debug) Task 407f8cbc-f03d-4813-8f32-5d0aee70e468: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:22:53,226::task::495::TaskManager.Task::(_debug) Task 0a80844a-8398-42f1-bfed-b527311c5225: moving from state init -> state preparing
>Thread-15916282::DEBUG::2013-11-20 04:22:53,228::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913658::DEBUG::2013-11-20 04:22:53,236::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`81ce6a35-dd73-47e0-a848-c029bb24482e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15916282::DEBUG::2013-11-20 04:22:53,237::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:22:53,239::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15916282::DEBUG::2013-11-20 04:22:53,239::task::495::TaskManager.Task::(_debug) Task 407f8cbc-f03d-4813-8f32-5d0aee70e468: ref 0 aborting False
>Thread-15916282::INFO::2013-11-20 04:22:53,242::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913658::DEBUG::2013-11-20 04:22:53,242::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:22:53,245::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`81ce6a35-dd73-47e0-a848-c029bb24482e`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:22:53,249::task::495::TaskManager.Task::(_debug) Task 0a80844a-8398-42f1-bfed-b527311c5225: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:22:53,250::task::495::TaskManager.Task::(_debug) Task 0a80844a-8398-42f1-bfed-b527311c5225: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:22:53,251::task::495::TaskManager.Task::(_debug) Task 0a80844a-8398-42f1-bfed-b527311c5225: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15916284::DEBUG::2013-11-20 04:22:53,253::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913658::DEBUG::2013-11-20 04:22:53,253::task::495::TaskManager.Task::(_debug) Task 0a80844a-8398-42f1-bfed-b527311c5225: moving from state preparing -> state finished
>Thread-15916284::INFO::2013-11-20 04:22:53,254::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913658::DEBUG::2013-11-20 04:22:53,255::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15916284::DEBUG::2013-11-20 04:22:53,256::task::495::TaskManager.Task::(_debug) Task 83ccbece-3a2e-4c02-b087-d1419bd69bb9: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:22:53,256::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916284::DEBUG::2013-11-20 04:22:53,258::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d2bf1a08-fc21-45cb-adc8-c9b7cd6d7a67`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913658::DEBUG::2013-11-20 04:22:53,259::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15916284::DEBUG::2013-11-20 04:22:53,259::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:22:53,260::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:22:53,261::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:22:53,262::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15916284::DEBUG::2013-11-20 04:22:53,263::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:22:53,263::task::495::TaskManager.Task::(_debug) Task 0a80844a-8398-42f1-bfed-b527311c5225: ref 0 aborting False
>Thread-15916284::DEBUG::2013-11-20 04:22:53,264::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d2bf1a08-fc21-45cb-adc8-c9b7cd6d7a67`::Granted request
>Thread-15913658::INFO::2013-11-20 04:22:53,265::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15916284::DEBUG::2013-11-20 04:22:53,267::task::495::TaskManager.Task::(_debug) Task 83ccbece-3a2e-4c02-b087-d1419bd69bb9: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916284::DEBUG::2013-11-20 04:22:53,268::task::495::TaskManager.Task::(_debug) Task 83ccbece-3a2e-4c02-b087-d1419bd69bb9: ref 1 aborting False
>Thread-15916284::WARNING::2013-11-20 04:22:53,272::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916284::DEBUG::2013-11-20 04:22:53,273::task::495::TaskManager.Task::(_debug) Task 83ccbece-3a2e-4c02-b087-d1419bd69bb9: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916284::DEBUG::2013-11-20 04:22:53,274::task::495::TaskManager.Task::(_debug) Task 83ccbece-3a2e-4c02-b087-d1419bd69bb9: moving from state preparing -> state finished
>Thread-15916284::DEBUG::2013-11-20 04:22:53,275::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916284::DEBUG::2013-11-20 04:22:53,276::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916284::DEBUG::2013-11-20 04:22:53,277::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916284::DEBUG::2013-11-20 04:22:53,278::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916284::DEBUG::2013-11-20 04:22:53,279::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916284::DEBUG::2013-11-20 04:22:53,280::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916284::DEBUG::2013-11-20 04:22:53,281::task::495::TaskManager.Task::(_debug) Task 83ccbece-3a2e-4c02-b087-d1419bd69bb9: ref 0 aborting False
>Thread-15916284::INFO::2013-11-20 04:22:53,282::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:22:53,538::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.077501 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:22:55,553::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:22:56,117::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0995196 s, 10.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:22:58,134::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:22:58,658::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0482179 s, 21.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:23:00,667::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:23:01,208::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0806267 s, 12.7 MB/s\n'; <rc> = 0
>Thread-42838::DEBUG::2013-11-20 04:23:02,353::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 04:23:03,133::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:23:03,223::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 04:23:03,494::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:23:03,500::task::495::TaskManager.Task::(_debug) Task 05277759-dc89-4f44-aa50-c3d5a870e469: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:23:03,501::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`907dbbe9-a7b4-41ed-b34f-328256420a7e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:23:03,502::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:23:03,503::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:23:03,503::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`907dbbe9-a7b4-41ed-b34f-328256420a7e`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:23:03,504::task::495::TaskManager.Task::(_debug) Task 05277759-dc89-4f44-aa50-c3d5a870e469: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:23:03,504::task::495::TaskManager.Task::(_debug) Task 05277759-dc89-4f44-aa50-c3d5a870e469: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:23:03,505::task::495::TaskManager.Task::(_debug) Task 05277759-dc89-4f44-aa50-c3d5a870e469: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:23:03,506::task::495::TaskManager.Task::(_debug) Task 05277759-dc89-4f44-aa50-c3d5a870e469: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:23:03,506::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:23:03,507::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:23:03,508::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:23:03,508::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:23:03,509::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:23:03,509::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:23:03,510::task::495::TaskManager.Task::(_debug) Task 05277759-dc89-4f44-aa50-c3d5a870e469: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:23:03,511::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15916288::DEBUG::2013-11-20 04:23:03,601::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916288::INFO::2013-11-20 04:23:03,602::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916288::DEBUG::2013-11-20 04:23:03,603::task::495::TaskManager.Task::(_debug) Task 7e4b3f70-c42c-46f9-bfbb-b3e3a6d6c624: moving from state init -> state preparing
>Thread-15916288::DEBUG::2013-11-20 04:23:03,604::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916288::DEBUG::2013-11-20 04:23:03,605::task::495::TaskManager.Task::(_debug) Task 7e4b3f70-c42c-46f9-bfbb-b3e3a6d6c624: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916288::DEBUG::2013-11-20 04:23:03,606::task::495::TaskManager.Task::(_debug) Task 7e4b3f70-c42c-46f9-bfbb-b3e3a6d6c624: moving from state preparing -> state finished
>Thread-15916288::DEBUG::2013-11-20 04:23:03,607::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916288::DEBUG::2013-11-20 04:23:03,607::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916288::DEBUG::2013-11-20 04:23:03,608::task::495::TaskManager.Task::(_debug) Task 7e4b3f70-c42c-46f9-bfbb-b3e3a6d6c624: ref 0 aborting False
>Thread-15916288::INFO::2013-11-20 04:23:03,609::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916289::DEBUG::2013-11-20 04:23:03,615::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916289::INFO::2013-11-20 04:23:03,616::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916289::DEBUG::2013-11-20 04:23:03,617::task::495::TaskManager.Task::(_debug) Task 537cbdcd-0a92-474e-a9f9-549a24eaf69f: moving from state init -> state preparing
>Thread-15916289::DEBUG::2013-11-20 04:23:03,618::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b595508f-6842-4741-a038-a3f94989b3c4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916289::DEBUG::2013-11-20 04:23:03,618::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916289::DEBUG::2013-11-20 04:23:03,619::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916289::DEBUG::2013-11-20 04:23:03,620::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b595508f-6842-4741-a038-a3f94989b3c4`::Granted request
>Thread-15916289::DEBUG::2013-11-20 04:23:03,621::task::495::TaskManager.Task::(_debug) Task 537cbdcd-0a92-474e-a9f9-549a24eaf69f: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916289::DEBUG::2013-11-20 04:23:03,621::task::495::TaskManager.Task::(_debug) Task 537cbdcd-0a92-474e-a9f9-549a24eaf69f: ref 1 aborting False
>Thread-15916289::WARNING::2013-11-20 04:23:03,624::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916289::DEBUG::2013-11-20 04:23:03,625::task::495::TaskManager.Task::(_debug) Task 537cbdcd-0a92-474e-a9f9-549a24eaf69f: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916289::DEBUG::2013-11-20 04:23:03,626::task::495::TaskManager.Task::(_debug) Task 537cbdcd-0a92-474e-a9f9-549a24eaf69f: moving from state preparing -> state finished
>Thread-15916289::DEBUG::2013-11-20 04:23:03,627::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916289::DEBUG::2013-11-20 04:23:03,627::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916289::DEBUG::2013-11-20 04:23:03,628::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916289::DEBUG::2013-11-20 04:23:03,629::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916289::DEBUG::2013-11-20 04:23:03,630::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916289::DEBUG::2013-11-20 04:23:03,631::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916289::DEBUG::2013-11-20 04:23:03,631::task::495::TaskManager.Task::(_debug) Task 537cbdcd-0a92-474e-a9f9-549a24eaf69f: ref 0 aborting False
>Thread-15916289::INFO::2013-11-20 04:23:03,632::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:23:03,832::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0817486 s, 12.5 MB/s\n'; <rc> = 0
>Thread-15916290::INFO::2013-11-20 04:23:05,360::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916290::DEBUG::2013-11-20 04:23:05,361::task::495::TaskManager.Task::(_debug) Task 49ed061b-f17f-4fa8-9a68-80763c2cbf06: moving from state init -> state preparing
>Thread-15916290::DEBUG::2013-11-20 04:23:05,362::task::495::TaskManager.Task::(_debug) Task 49ed061b-f17f-4fa8-9a68-80763c2cbf06: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000375032424927', 'lastCheck': 1384921383.1386731, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00160503387451', 'lastCheck': 1384921381.3529179, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00176501274109', 'lastCheck': 1384921378.3610649, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000567197799683', 'lastCheck': 1384921377.225589, 'valid': True, 'code': 0}}
>Thread-15916290::DEBUG::2013-11-20 04:23:05,363::task::495::TaskManager.Task::(_debug) Task 49ed061b-f17f-4fa8-9a68-80763c2cbf06: moving from state preparing -> state finished
>Thread-15916290::DEBUG::2013-11-20 04:23:05,363::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916290::DEBUG::2013-11-20 04:23:05,364::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916290::DEBUG::2013-11-20 04:23:05,364::task::495::TaskManager.Task::(_debug) Task 49ed061b-f17f-4fa8-9a68-80763c2cbf06: ref 0 aborting False
>Thread-15916290::INFO::2013-11-20 04:23:05,365::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000375032424927', 'lastCheck': 1384921383.1386731, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00160503387451', 'lastCheck': 1384921381.3529179, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00176501274109', 'lastCheck': 1384921378.3610649, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000567197799683', 'lastCheck': 1384921377.225589, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:23:05,848::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:23:06,384::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0632956 s, 16.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:23:08,394::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 04:23:08,610::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:23:08,611::task::495::TaskManager.Task::(_debug) Task 5bd764a5-ccb8-462d-aeee-3f6408d32784: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:23:08,614::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`17021e0b-f7bd-4556-b9a1-f7b6c8786207`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:23:08,615::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:23:08,616::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:23:08,617::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`17021e0b-f7bd-4556-b9a1-f7b6c8786207`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:23:08,618::task::495::TaskManager.Task::(_debug) Task 5bd764a5-ccb8-462d-aeee-3f6408d32784: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:23:08,619::task::495::TaskManager.Task::(_debug) Task 5bd764a5-ccb8-462d-aeee-3f6408d32784: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:23:08,620::task::495::TaskManager.Task::(_debug) Task 5bd764a5-ccb8-462d-aeee-3f6408d32784: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:23:08,621::task::495::TaskManager.Task::(_debug) Task 5bd764a5-ccb8-462d-aeee-3f6408d32784: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:23:08,621::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:23:08,622::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:23:08,623::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:23:08,624::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:23:08,624::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:23:08,625::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:23:08,626::task::495::TaskManager.Task::(_debug) Task 5bd764a5-ccb8-462d-aeee-3f6408d32784: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:23:08,627::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:23:08,877::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0869737 s, 11.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:23:10,898::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:23:11,455::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0840412 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:23:13,469::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 04:23:13,695::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:23:13,702::task::495::TaskManager.Task::(_debug) Task e3183305-ad1a-4a69-b198-cf43d80718ae: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:23:13,705::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2fc35bd0-16d7-43de-8bf5-10df34f121ec`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:23:13,708::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:23:13,709::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:23:13,710::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2fc35bd0-16d7-43de-8bf5-10df34f121ec`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:23:13,711::task::495::TaskManager.Task::(_debug) Task e3183305-ad1a-4a69-b198-cf43d80718ae: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:23:13,712::task::495::TaskManager.Task::(_debug) Task e3183305-ad1a-4a69-b198-cf43d80718ae: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:23:13,712::task::495::TaskManager.Task::(_debug) Task e3183305-ad1a-4a69-b198-cf43d80718ae: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:23:13,714::task::495::TaskManager.Task::(_debug) Task e3183305-ad1a-4a69-b198-cf43d80718ae: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:23:13,714::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:23:13,715::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:23:13,716::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:23:13,717::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:23:13,717::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:23:13,718::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:23:13,719::task::495::TaskManager.Task::(_debug) Task e3183305-ad1a-4a69-b198-cf43d80718ae: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:23:13,719::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15916294::DEBUG::2013-11-20 04:23:13,985::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916294::INFO::2013-11-20 04:23:13,986::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916294::DEBUG::2013-11-20 04:23:13,988::task::495::TaskManager.Task::(_debug) Task 7dba50fa-80ee-4d30-afd9-078aed27cd79: moving from state init -> state preparing
>Thread-15916294::DEBUG::2013-11-20 04:23:13,994::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916294::DEBUG::2013-11-20 04:23:13,996::task::495::TaskManager.Task::(_debug) Task 7dba50fa-80ee-4d30-afd9-078aed27cd79: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916294::DEBUG::2013-11-20 04:23:13,998::task::495::TaskManager.Task::(_debug) Task 7dba50fa-80ee-4d30-afd9-078aed27cd79: moving from state preparing -> state finished
>Thread-15916294::DEBUG::2013-11-20 04:23:14,002::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916294::DEBUG::2013-11-20 04:23:14,005::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916294::DEBUG::2013-11-20 04:23:14,007::task::495::TaskManager.Task::(_debug) Task 7dba50fa-80ee-4d30-afd9-078aed27cd79: ref 0 aborting False
>Thread-15916294::INFO::2013-11-20 04:23:14,010::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916295::DEBUG::2013-11-20 04:23:14,020::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916295::INFO::2013-11-20 04:23:14,022::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916295::DEBUG::2013-11-20 04:23:14,023::task::495::TaskManager.Task::(_debug) Task 44901b50-d8db-4416-ba25-fce8ad800a7e: moving from state init -> state preparing
>Thread-15916295::DEBUG::2013-11-20 04:23:14,026::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2fb668db-63e8-485f-a035-9facccfcbff7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916295::DEBUG::2013-11-20 04:23:14,029::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916295::DEBUG::2013-11-20 04:23:14,031::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916295::DEBUG::2013-11-20 04:23:14,032::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2fb668db-63e8-485f-a035-9facccfcbff7`::Granted request
>Thread-15916295::DEBUG::2013-11-20 04:23:14,034::task::495::TaskManager.Task::(_debug) Task 44901b50-d8db-4416-ba25-fce8ad800a7e: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916295::DEBUG::2013-11-20 04:23:14,036::task::495::TaskManager.Task::(_debug) Task 44901b50-d8db-4416-ba25-fce8ad800a7e: ref 1 aborting False
>Thread-15916295::WARNING::2013-11-20 04:23:14,042::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916295::DEBUG::2013-11-20 04:23:14,045::task::495::TaskManager.Task::(_debug) Task 44901b50-d8db-4416-ba25-fce8ad800a7e: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916295::DEBUG::2013-11-20 04:23:14,047::task::495::TaskManager.Task::(_debug) Task 44901b50-d8db-4416-ba25-fce8ad800a7e: moving from state preparing -> state finished
>Thread-15916295::DEBUG::2013-11-20 04:23:14,049::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916295::DEBUG::2013-11-20 04:23:14,050::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916295::DEBUG::2013-11-20 04:23:14,052::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916295::DEBUG::2013-11-20 04:23:14,054::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916295::DEBUG::2013-11-20 04:23:14,055::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916295::DEBUG::2013-11-20 04:23:14,056::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916295::DEBUG::2013-11-20 04:23:14,058::task::495::TaskManager.Task::(_debug) Task 44901b50-d8db-4416-ba25-fce8ad800a7e: ref 0 aborting False
>Thread-15916295::INFO::2013-11-20 04:23:14,059::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:23:14,092::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0813581 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:23:16,107::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:23:16,668::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0802375 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:23:18,683::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:23:19,276::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0998482 s, 10.3 MB/s\n'; <rc> = 0
>Thread-15916298::INFO::2013-11-20 04:23:20,954::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916298::DEBUG::2013-11-20 04:23:20,955::task::495::TaskManager.Task::(_debug) Task 32de965e-d26f-4963-9bfd-b8c63fcb9741: moving from state init -> state preparing
>Thread-15916298::DEBUG::2013-11-20 04:23:20,956::task::495::TaskManager.Task::(_debug) Task 32de965e-d26f-4963-9bfd-b8c63fcb9741: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000422954559326', 'lastCheck': 1384921393.152513, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00164818763733', 'lastCheck': 1384921391.5628681, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0631861686707', 'lastCheck': 1384921398.9603851, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000433921813965', 'lastCheck': 1384921397.6949689, 'valid': True, 'code': 0}}
>Thread-15916298::DEBUG::2013-11-20 04:23:20,957::task::495::TaskManager.Task::(_debug) Task 32de965e-d26f-4963-9bfd-b8c63fcb9741: moving from state preparing -> state finished
>Thread-15916298::DEBUG::2013-11-20 04:23:20,957::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916298::DEBUG::2013-11-20 04:23:20,958::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916298::DEBUG::2013-11-20 04:23:20,959::task::495::TaskManager.Task::(_debug) Task 32de965e-d26f-4963-9bfd-b8c63fcb9741: ref 0 aborting False
>Thread-15916298::INFO::2013-11-20 04:23:20,959::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000422954559326', 'lastCheck': 1384921393.152513, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00164818763733', 'lastCheck': 1384921391.5628681, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0631861686707', 'lastCheck': 1384921398.9603851, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000433921813965', 'lastCheck': 1384921397.6949689, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:23:21,295::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 04:23:21,523::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:23:21,535::task::495::TaskManager.Task::(_debug) Task 2e66b18e-7911-4b3e-9f9e-3779d739a76f: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:23:21,539::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`06987204-8d7a-4b16-bfaa-46160684cfbc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:23:21,540::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:23:21,540::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:23:21,541::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`06987204-8d7a-4b16-bfaa-46160684cfbc`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:23:21,542::task::495::TaskManager.Task::(_debug) Task 2e66b18e-7911-4b3e-9f9e-3779d739a76f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 04:23:21,543::task::495::TaskManager.Task::(_debug) Task 2e66b18e-7911-4b3e-9f9e-3779d739a76f: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:23:21,544::task::495::TaskManager.Task::(_debug) Task 2e66b18e-7911-4b3e-9f9e-3779d739a76f: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 04:23:21,545::task::495::TaskManager.Task::(_debug) Task 2e66b18e-7911-4b3e-9f9e-3779d739a76f: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:23:21,546::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:23:21,547::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:23:21,548::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 04:23:21,548::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:23:21,549::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:23:21,549::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:23:21,550::task::495::TaskManager.Task::(_debug) Task 2e66b18e-7911-4b3e-9f9e-3779d739a76f: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 04:23:21,551::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 04:23:21,831::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0750937 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:23:23,847::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 04:23:24,074::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:23:24,084::task::495::TaskManager.Task::(_debug) Task 1422e479-fe24-43fc-9526-f5faa296f528: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:23:24,089::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f9c094a5-3d1c-4e95-90b3-b1cfebd93ce0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:23:24,090::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:23:24,093::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:23:24,094::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f9c094a5-3d1c-4e95-90b3-b1cfebd93ce0`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:23:24,095::task::495::TaskManager.Task::(_debug) Task 1422e479-fe24-43fc-9526-f5faa296f528: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:23:24,096::task::495::TaskManager.Task::(_debug) Task 1422e479-fe24-43fc-9526-f5faa296f528: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:23:24,097::task::495::TaskManager.Task::(_debug) Task 1422e479-fe24-43fc-9526-f5faa296f528: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 04:23:24,098::task::495::TaskManager.Task::(_debug) Task 1422e479-fe24-43fc-9526-f5faa296f528: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:23:24,099::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15914435::INFO::2013-11-20 04:23:24,100::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15913546::DEBUG::2013-11-20 04:23:24,101::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:23:24,102::task::495::TaskManager.Task::(_debug) Task 7e50bd98-4701-48d2-a813-25e10eca4a81: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:23:24,103::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15914435::DEBUG::2013-11-20 04:23:24,105::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e0f3e519-f1d9-4c16-b234-0f05005cd672`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:23:24,105::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:23:24,106::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:23:24,107::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:23:24,108::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:23:24,109::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:23:24,110::task::495::TaskManager.Task::(_debug) Task 1422e479-fe24-43fc-9526-f5faa296f528: ref 0 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:23:24,110::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e0f3e519-f1d9-4c16-b234-0f05005cd672`::Granted request
>Thread-15913546::INFO::2013-11-20 04:23:24,111::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914435::DEBUG::2013-11-20 04:23:24,113::task::495::TaskManager.Task::(_debug) Task 7e50bd98-4701-48d2-a813-25e10eca4a81: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913546::INFO::2013-11-20 04:23:24,114::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15914435::DEBUG::2013-11-20 04:23:24,115::task::495::TaskManager.Task::(_debug) Task 7e50bd98-4701-48d2-a813-25e10eca4a81: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:23:24,115::task::495::TaskManager.Task::(_debug) Task ca53d195-8f3f-4fa2-bb6d-6ffe13623f7b: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:23:24,116::task::495::TaskManager.Task::(_debug) Task 7e50bd98-4701-48d2-a813-25e10eca4a81: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15913546::DEBUG::2013-11-20 04:23:24,118::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d442960f-35d8-4dac-875a-e8d8683c7ba2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:23:24,119::task::495::TaskManager.Task::(_debug) Task 7e50bd98-4701-48d2-a813-25e10eca4a81: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:23:24,119::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:23:24,120::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:23:24,121::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:23:24,122::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:23:24,123::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913546::DEBUG::2013-11-20 04:23:24,123::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d442960f-35d8-4dac-875a-e8d8683c7ba2`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:23:24,125::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:23:24,126::task::495::TaskManager.Task::(_debug) Task ca53d195-8f3f-4fa2-bb6d-6ffe13623f7b: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15914435::DEBUG::2013-11-20 04:23:24,126::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:23:24,127::task::495::TaskManager.Task::(_debug) Task ca53d195-8f3f-4fa2-bb6d-6ffe13623f7b: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:23:24,128::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:23:24,129::task::495::TaskManager.Task::(_debug) Task ca53d195-8f3f-4fa2-bb6d-6ffe13623f7b: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15914435::DEBUG::2013-11-20 04:23:24,130::task::495::TaskManager.Task::(_debug) Task 7e50bd98-4701-48d2-a813-25e10eca4a81: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:23:24,131::task::495::TaskManager.Task::(_debug) Task ca53d195-8f3f-4fa2-bb6d-6ffe13623f7b: moving from state preparing -> state finished
>Thread-15914435::INFO::2013-11-20 04:23:24,131::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15913546::DEBUG::2013-11-20 04:23:24,132::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:23:24,133::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:23:24,135::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:23:24,136::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:23:24,137::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:23:24,137::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:23:24,138::task::495::TaskManager.Task::(_debug) Task ca53d195-8f3f-4fa2-bb6d-6ffe13623f7b: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:23:24,139::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:23:24,140::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:23:24,141::task::495::TaskManager.Task::(_debug) Task 19655bce-44e6-4923-8d26-8f0bace7aa55: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:23:24,142::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`26f86bca-895b-4cab-ab11-4a0d4bcbadf0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:23:24,143::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:23:24,143::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:23:24,144::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`26f86bca-895b-4cab-ab11-4a0d4bcbadf0`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:23:24,146::task::495::TaskManager.Task::(_debug) Task 19655bce-44e6-4923-8d26-8f0bace7aa55: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:23:24,147::task::495::TaskManager.Task::(_debug) Task 19655bce-44e6-4923-8d26-8f0bace7aa55: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:23:24,148::task::495::TaskManager.Task::(_debug) Task 19655bce-44e6-4923-8d26-8f0bace7aa55: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:23:24,149::task::495::TaskManager.Task::(_debug) Task 19655bce-44e6-4923-8d26-8f0bace7aa55: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:23:24,149::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:23:24,150::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:23:24,151::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:23:24,152::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:23:24,152::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:23:24,153::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:23:24,158::task::495::TaskManager.Task::(_debug) Task 19655bce-44e6-4923-8d26-8f0bace7aa55: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:23:24,159::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:23:24,160::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:23:24,160::task::495::TaskManager.Task::(_debug) Task 3b452cdb-5f27-462d-80ce-d2896c3f538e: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:23:24,162::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`55416429-c716-43bb-90ae-a755d3e28fd2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:23:24,162::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:23:24,163::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:23:24,164::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`55416429-c716-43bb-90ae-a755d3e28fd2`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:23:24,165::task::495::TaskManager.Task::(_debug) Task 3b452cdb-5f27-462d-80ce-d2896c3f538e: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:23:24,165::task::495::TaskManager.Task::(_debug) Task 3b452cdb-5f27-462d-80ce-d2896c3f538e: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:23:24,166::task::495::TaskManager.Task::(_debug) Task 3b452cdb-5f27-462d-80ce-d2896c3f538e: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:23:24,166::task::495::TaskManager.Task::(_debug) Task 3b452cdb-5f27-462d-80ce-d2896c3f538e: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:23:24,167::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:23:24,168::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:23:24,168::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:23:24,169::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:23:24,169::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:23:24,170::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:23:24,170::task::495::TaskManager.Task::(_debug) Task 3b452cdb-5f27-462d-80ce-d2896c3f538e: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:23:24,171::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15916300::DEBUG::2013-11-20 04:23:24,332::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916300::INFO::2013-11-20 04:23:24,334::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916300::DEBUG::2013-11-20 04:23:24,334::task::495::TaskManager.Task::(_debug) Task d0d66ddd-220f-46fa-9993-7d1fa5172847: moving from state init -> state preparing
>Thread-15916300::DEBUG::2013-11-20 04:23:24,336::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916300::DEBUG::2013-11-20 04:23:24,336::task::495::TaskManager.Task::(_debug) Task d0d66ddd-220f-46fa-9993-7d1fa5172847: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916300::DEBUG::2013-11-20 04:23:24,337::task::495::TaskManager.Task::(_debug) Task d0d66ddd-220f-46fa-9993-7d1fa5172847: moving from state preparing -> state finished
>Thread-15916300::DEBUG::2013-11-20 04:23:24,338::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916300::DEBUG::2013-11-20 04:23:24,339::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916300::DEBUG::2013-11-20 04:23:24,340::task::495::TaskManager.Task::(_debug) Task d0d66ddd-220f-46fa-9993-7d1fa5172847: ref 0 aborting False
>Thread-15916300::INFO::2013-11-20 04:23:24,341::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916301::DEBUG::2013-11-20 04:23:24,347::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916301::INFO::2013-11-20 04:23:24,348::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916301::DEBUG::2013-11-20 04:23:24,349::task::495::TaskManager.Task::(_debug) Task 097403ab-7282-4a44-aa40-cbccc71f67ce: moving from state init -> state preparing
>Thread-15916301::DEBUG::2013-11-20 04:23:24,366::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1a1f26b3-b0b3-493b-88be-8ea1e54adf46`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916301::DEBUG::2013-11-20 04:23:24,367::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916301::DEBUG::2013-11-20 04:23:24,369::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916301::DEBUG::2013-11-20 04:23:24,370::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1a1f26b3-b0b3-493b-88be-8ea1e54adf46`::Granted request
>Thread-15916301::DEBUG::2013-11-20 04:23:24,372::task::495::TaskManager.Task::(_debug) Task 097403ab-7282-4a44-aa40-cbccc71f67ce: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916301::DEBUG::2013-11-20 04:23:24,373::task::495::TaskManager.Task::(_debug) Task 097403ab-7282-4a44-aa40-cbccc71f67ce: ref 1 aborting False
>Thread-15916301::WARNING::2013-11-20 04:23:24,376::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916301::DEBUG::2013-11-20 04:23:24,377::task::495::TaskManager.Task::(_debug) Task 097403ab-7282-4a44-aa40-cbccc71f67ce: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916301::DEBUG::2013-11-20 04:23:24,378::task::495::TaskManager.Task::(_debug) Task 097403ab-7282-4a44-aa40-cbccc71f67ce: moving from state preparing -> state finished
>Thread-15916301::DEBUG::2013-11-20 04:23:24,379::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916301::DEBUG::2013-11-20 04:23:24,380::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916301::DEBUG::2013-11-20 04:23:24,381::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916301::DEBUG::2013-11-20 04:23:24,383::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916301::DEBUG::2013-11-20 04:23:24,383::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916301::DEBUG::2013-11-20 04:23:24,384::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916301::DEBUG::2013-11-20 04:23:24,385::task::495::TaskManager.Task::(_debug) Task 097403ab-7282-4a44-aa40-cbccc71f67ce: ref 0 aborting False
>Thread-15916301::INFO::2013-11-20 04:23:24,386::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:23:24,465::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.127322 s, 8.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:23:26,480::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:23:26,989::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0672918 s, 15.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:23:28,999::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:23:29,549::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0765332 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:23:31,564::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:23:32,103::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0840024 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:23:34,125::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916306::DEBUG::2013-11-20 04:23:34,690::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916306::INFO::2013-11-20 04:23:34,692::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916306::DEBUG::2013-11-20 04:23:34,693::task::495::TaskManager.Task::(_debug) Task 5519ad33-0473-4064-adf5-1d7f185ff3fd: moving from state init -> state preparing
>Thread-15916306::DEBUG::2013-11-20 04:23:34,697::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916306::DEBUG::2013-11-20 04:23:34,699::task::495::TaskManager.Task::(_debug) Task 5519ad33-0473-4064-adf5-1d7f185ff3fd: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916306::DEBUG::2013-11-20 04:23:34,700::task::495::TaskManager.Task::(_debug) Task 5519ad33-0473-4064-adf5-1d7f185ff3fd: moving from state preparing -> state finished
>Thread-15916306::DEBUG::2013-11-20 04:23:34,701::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916306::DEBUG::2013-11-20 04:23:34,703::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916306::DEBUG::2013-11-20 04:23:34,704::task::495::TaskManager.Task::(_debug) Task 5519ad33-0473-4064-adf5-1d7f185ff3fd: ref 0 aborting False
>Thread-15916306::INFO::2013-11-20 04:23:34,707::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916307::DEBUG::2013-11-20 04:23:34,715::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916307::INFO::2013-11-20 04:23:34,717::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916307::DEBUG::2013-11-20 04:23:34,718::task::495::TaskManager.Task::(_debug) Task 9f92c9a4-6f45-497f-ad53-0f4b3af8b6e9: moving from state init -> state preparing
>Thread-15916307::DEBUG::2013-11-20 04:23:34,722::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`319809b2-a4b6-4785-9472-3423ca02f8aa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916307::DEBUG::2013-11-20 04:23:34,724::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916307::DEBUG::2013-11-20 04:23:34,727::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916307::DEBUG::2013-11-20 04:23:34,729::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`319809b2-a4b6-4785-9472-3423ca02f8aa`::Granted request
>Thread-15916307::DEBUG::2013-11-20 04:23:34,732::task::495::TaskManager.Task::(_debug) Task 9f92c9a4-6f45-497f-ad53-0f4b3af8b6e9: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916307::DEBUG::2013-11-20 04:23:34,734::task::495::TaskManager.Task::(_debug) Task 9f92c9a4-6f45-497f-ad53-0f4b3af8b6e9: ref 1 aborting False
>Thread-15916307::WARNING::2013-11-20 04:23:34,740::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916307::DEBUG::2013-11-20 04:23:34,743::task::495::TaskManager.Task::(_debug) Task 9f92c9a4-6f45-497f-ad53-0f4b3af8b6e9: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916307::DEBUG::2013-11-20 04:23:34,745::task::495::TaskManager.Task::(_debug) Task 9f92c9a4-6f45-497f-ad53-0f4b3af8b6e9: moving from state preparing -> state finished
>Thread-15916307::DEBUG::2013-11-20 04:23:34,746::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916307::DEBUG::2013-11-20 04:23:34,747::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916307::DEBUG::2013-11-20 04:23:34,748::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916307::DEBUG::2013-11-20 04:23:34,752::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916307::DEBUG::2013-11-20 04:23:34,753::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916307::DEBUG::2013-11-20 04:23:34,755::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916307::DEBUG::2013-11-20 04:23:34,761::task::495::TaskManager.Task::(_debug) Task 9f92c9a4-6f45-497f-ad53-0f4b3af8b6e9: ref 0 aborting False
>Thread-15916307::INFO::2013-11-20 04:23:34,764::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:23:34,801::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0927308 s, 11.0 MB/s\n'; <rc> = 0
>Thread-15916308::INFO::2013-11-20 04:23:36,631::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916308::DEBUG::2013-11-20 04:23:36,633::task::495::TaskManager.Task::(_debug) Task 9e3e312c-83b5-4dd3-81be-b39282c77fc0: moving from state init -> state preparing
>Thread-15916308::DEBUG::2013-11-20 04:23:36,633::task::495::TaskManager.Task::(_debug) Task 9e3e312c-83b5-4dd3-81be-b39282c77fc0: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00039005279541', 'lastCheck': 1384921413.7574451, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00147700309753', 'lastCheck': 1384921412.221024, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00195908546448', 'lastCheck': 1384921409.2601631, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00042986869812', 'lastCheck': 1384921408.078835, 'valid': True, 'code': 0}}
>Thread-15916308::DEBUG::2013-11-20 04:23:36,635::task::495::TaskManager.Task::(_debug) Task 9e3e312c-83b5-4dd3-81be-b39282c77fc0: moving from state preparing -> state finished
>Thread-15916308::DEBUG::2013-11-20 04:23:36,635::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916308::DEBUG::2013-11-20 04:23:36,636::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916308::DEBUG::2013-11-20 04:23:36,636::task::495::TaskManager.Task::(_debug) Task 9e3e312c-83b5-4dd3-81be-b39282c77fc0: ref 0 aborting False
>Thread-15916308::INFO::2013-11-20 04:23:36,637::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00039005279541', 'lastCheck': 1384921413.7574451, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00147700309753', 'lastCheck': 1384921412.221024, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00195908546448', 'lastCheck': 1384921409.2601631, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00042986869812', 'lastCheck': 1384921408.078835, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:23:36,820::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:23:37,389::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0910415 s, 11.2 MB/s\n'; <rc> = 0
>Thread-15913639::INFO::2013-11-20 04:23:39,361::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:23:39,363::task::495::TaskManager.Task::(_debug) Task ffe68d29-0ae9-4a75-aa1c-308407d56d4d: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:23:39,377::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8fcabd24-d3ba-42f1-a07f-b3413ce0026b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:23:39,378::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:23:39,380::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:23:39,382::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8fcabd24-d3ba-42f1-a07f-b3413ce0026b`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:23:39,383::task::495::TaskManager.Task::(_debug) Task ffe68d29-0ae9-4a75-aa1c-308407d56d4d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:23:39,390::task::495::TaskManager.Task::(_debug) Task ffe68d29-0ae9-4a75-aa1c-308407d56d4d: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:23:39,392::task::495::TaskManager.Task::(_debug) Task ffe68d29-0ae9-4a75-aa1c-308407d56d4d: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:23:39,392::task::495::TaskManager.Task::(_debug) Task ffe68d29-0ae9-4a75-aa1c-308407d56d4d: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:23:39,393::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:23:39,394::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:23:39,394::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:23:39,395::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:23:39,396::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:23:39,396::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:23:39,397::task::495::TaskManager.Task::(_debug) Task ffe68d29-0ae9-4a75-aa1c-308407d56d4d: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:23:39,398::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:23:39,404::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:23:39,956::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0797947 s, 12.8 MB/s\n'; <rc> = 0
>Thread-15915918::INFO::2013-11-20 04:23:41,775::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:23:41,776::task::495::TaskManager.Task::(_debug) Task 6ee97d6d-0ad4-42ce-9be7-b76c87d766cf: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:23:41,780::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b3fbe3f9-1cb6-49a7-88a6-b7f2249a3f14`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:23:41,781::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:23:41,782::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:23:41,784::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b3fbe3f9-1cb6-49a7-88a6-b7f2249a3f14`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:23:41,785::task::495::TaskManager.Task::(_debug) Task 6ee97d6d-0ad4-42ce-9be7-b76c87d766cf: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:23:41,786::task::495::TaskManager.Task::(_debug) Task 6ee97d6d-0ad4-42ce-9be7-b76c87d766cf: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:23:41,787::task::495::TaskManager.Task::(_debug) Task 6ee97d6d-0ad4-42ce-9be7-b76c87d766cf: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:23:41,788::task::495::TaskManager.Task::(_debug) Task 6ee97d6d-0ad4-42ce-9be7-b76c87d766cf: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:23:41,791::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:23:41,792::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:23:41,794::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:23:41,796::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:23:41,797::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:23:41,798::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:23:41,798::task::495::TaskManager.Task::(_debug) Task 6ee97d6d-0ad4-42ce-9be7-b76c87d766cf: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:23:41,799::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:23:41,972::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:23:42,559::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0893995 s, 11.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:23:44,574::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:23:45,119::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0765089 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15916312::DEBUG::2013-11-20 04:23:45,216::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916312::INFO::2013-11-20 04:23:45,218::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916312::DEBUG::2013-11-20 04:23:45,219::task::495::TaskManager.Task::(_debug) Task c8816ba3-514b-4093-897c-82ada7f9a2a0: moving from state init -> state preparing
>Thread-15916312::DEBUG::2013-11-20 04:23:45,220::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916312::DEBUG::2013-11-20 04:23:45,221::task::495::TaskManager.Task::(_debug) Task c8816ba3-514b-4093-897c-82ada7f9a2a0: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916312::DEBUG::2013-11-20 04:23:45,222::task::495::TaskManager.Task::(_debug) Task c8816ba3-514b-4093-897c-82ada7f9a2a0: moving from state preparing -> state finished
>Thread-15916312::DEBUG::2013-11-20 04:23:45,222::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916312::DEBUG::2013-11-20 04:23:45,223::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916312::DEBUG::2013-11-20 04:23:45,223::task::495::TaskManager.Task::(_debug) Task c8816ba3-514b-4093-897c-82ada7f9a2a0: ref 0 aborting False
>Thread-15916312::INFO::2013-11-20 04:23:45,224::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916313::DEBUG::2013-11-20 04:23:45,229::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916313::INFO::2013-11-20 04:23:45,230::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916313::DEBUG::2013-11-20 04:23:45,231::task::495::TaskManager.Task::(_debug) Task d1e439bc-14ea-4459-ba50-947f07219dd8: moving from state init -> state preparing
>Thread-15916313::DEBUG::2013-11-20 04:23:45,232::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6633fb5f-1800-41b9-ac92-ec802942430a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916313::DEBUG::2013-11-20 04:23:45,233::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916313::DEBUG::2013-11-20 04:23:45,234::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916313::DEBUG::2013-11-20 04:23:45,235::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6633fb5f-1800-41b9-ac92-ec802942430a`::Granted request
>Thread-15916313::DEBUG::2013-11-20 04:23:45,236::task::495::TaskManager.Task::(_debug) Task d1e439bc-14ea-4459-ba50-947f07219dd8: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916313::DEBUG::2013-11-20 04:23:45,237::task::495::TaskManager.Task::(_debug) Task d1e439bc-14ea-4459-ba50-947f07219dd8: ref 1 aborting False
>Thread-15916313::WARNING::2013-11-20 04:23:45,239::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916313::DEBUG::2013-11-20 04:23:45,240::task::495::TaskManager.Task::(_debug) Task d1e439bc-14ea-4459-ba50-947f07219dd8: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916313::DEBUG::2013-11-20 04:23:45,241::task::495::TaskManager.Task::(_debug) Task d1e439bc-14ea-4459-ba50-947f07219dd8: moving from state preparing -> state finished
>Thread-15916313::DEBUG::2013-11-20 04:23:45,242::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916313::DEBUG::2013-11-20 04:23:45,242::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916313::DEBUG::2013-11-20 04:23:45,243::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916313::DEBUG::2013-11-20 04:23:45,244::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916313::DEBUG::2013-11-20 04:23:45,244::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916313::DEBUG::2013-11-20 04:23:45,245::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916313::DEBUG::2013-11-20 04:23:45,246::task::495::TaskManager.Task::(_debug) Task d1e439bc-14ea-4459-ba50-947f07219dd8: ref 0 aborting False
>Thread-15916313::INFO::2013-11-20 04:23:45,246::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15915863::INFO::2013-11-20 04:23:46,792::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:23:46,793::task::495::TaskManager.Task::(_debug) Task 38129950-e964-4aba-bacd-7707f88b7c04: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:23:46,795::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`674880c0-fcaf-4928-98a1-d777c9daff72`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:23:46,795::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:23:46,796::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:23:46,797::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`674880c0-fcaf-4928-98a1-d777c9daff72`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:23:46,798::task::495::TaskManager.Task::(_debug) Task 38129950-e964-4aba-bacd-7707f88b7c04: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:23:46,799::task::495::TaskManager.Task::(_debug) Task 38129950-e964-4aba-bacd-7707f88b7c04: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:23:46,800::task::495::TaskManager.Task::(_debug) Task 38129950-e964-4aba-bacd-7707f88b7c04: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:23:46,801::task::495::TaskManager.Task::(_debug) Task 38129950-e964-4aba-bacd-7707f88b7c04: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:23:46,801::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:23:46,802::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:23:46,803::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:23:46,804::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:23:46,804::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:23:46,805::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:23:46,806::task::495::TaskManager.Task::(_debug) Task 38129950-e964-4aba-bacd-7707f88b7c04: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:23:46,806::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:23:47,133::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:23:47,659::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0746424 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:23:49,675::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:23:50,212::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0735247 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:23:52,227::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913518::INFO::2013-11-20 04:23:52,470::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:23:52,471::task::495::TaskManager.Task::(_debug) Task 97aa21ec-2da5-4ca2-bb2c-8d104a23f0d7: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:23:52,475::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6dde8b55-5162-46b5-8957-03df4f3107c0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:23:52,476::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:23:52,478::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:23:52,478::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6dde8b55-5162-46b5-8957-03df4f3107c0`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:23:52,480::task::495::TaskManager.Task::(_debug) Task 97aa21ec-2da5-4ca2-bb2c-8d104a23f0d7: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:23:52,481::task::495::TaskManager.Task::(_debug) Task 97aa21ec-2da5-4ca2-bb2c-8d104a23f0d7: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:23:52,482::task::495::TaskManager.Task::(_debug) Task 97aa21ec-2da5-4ca2-bb2c-8d104a23f0d7: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:23:52,482::task::495::TaskManager.Task::(_debug) Task 97aa21ec-2da5-4ca2-bb2c-8d104a23f0d7: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:23:52,483::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:23:52,484::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:23:52,485::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:23:52,485::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:23:52,486::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:23:52,487::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:23:52,487::task::495::TaskManager.Task::(_debug) Task 97aa21ec-2da5-4ca2-bb2c-8d104a23f0d7: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:23:52,488::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:23:52,489::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:23:52,490::task::495::TaskManager.Task::(_debug) Task 84e9e4fd-5734-465a-896a-55296a06a261: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:23:52,491::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5038ff0d-46a8-4c3b-adc3-5ed8028122a7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:23:52,492::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:23:52,492::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:23:52,493::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5038ff0d-46a8-4c3b-adc3-5ed8028122a7`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:23:52,494::task::495::TaskManager.Task::(_debug) Task 84e9e4fd-5734-465a-896a-55296a06a261: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:23:52,495::task::495::TaskManager.Task::(_debug) Task 84e9e4fd-5734-465a-896a-55296a06a261: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:23:52,496::task::495::TaskManager.Task::(_debug) Task 84e9e4fd-5734-465a-896a-55296a06a261: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:23:52,496::task::495::TaskManager.Task::(_debug) Task 84e9e4fd-5734-465a-896a-55296a06a261: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:23:52,497::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:23:52,498::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:23:52,498::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:23:52,499::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:23:52,499::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:23:52,500::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:23:52,501::task::495::TaskManager.Task::(_debug) Task 84e9e4fd-5734-465a-896a-55296a06a261: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:23:52,501::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:23:52,804::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0872922 s, 11.7 MB/s\n'; <rc> = 0
>Thread-15916316::INFO::2013-11-20 04:23:52,896::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916316::DEBUG::2013-11-20 04:23:52,897::task::495::TaskManager.Task::(_debug) Task 6d6896d7-24a1-4f94-b84f-486e8f8f207e: moving from state init -> state preparing
>Thread-15916316::DEBUG::2013-11-20 04:23:52,898::task::495::TaskManager.Task::(_debug) Task 6d6896d7-24a1-4f94-b84f-486e8f8f207e: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000303983688354', 'lastCheck': 1384921423.770129, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00169086456299', 'lastCheck': 1384921432.450609, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00150895118713', 'lastCheck': 1384921429.3724101, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000610828399658', 'lastCheck': 1384921428.3717189, 'valid': True, 'code': 0}}
>Thread-15916316::DEBUG::2013-11-20 04:23:52,899::task::495::TaskManager.Task::(_debug) Task 6d6896d7-24a1-4f94-b84f-486e8f8f207e: moving from state preparing -> state finished
>Thread-15916316::DEBUG::2013-11-20 04:23:52,900::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916316::DEBUG::2013-11-20 04:23:52,900::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916316::DEBUG::2013-11-20 04:23:52,901::task::495::TaskManager.Task::(_debug) Task 6d6896d7-24a1-4f94-b84f-486e8f8f207e: ref 0 aborting False
>Thread-15916316::INFO::2013-11-20 04:23:52,902::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000303983688354', 'lastCheck': 1384921423.770129, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00169086456299', 'lastCheck': 1384921432.450609, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00150895118713', 'lastCheck': 1384921429.3724101, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000610828399658', 'lastCheck': 1384921428.3717189, 'valid': True, 'code': 0}}
>Thread-15913658::INFO::2013-11-20 04:23:54,438::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:23:54,439::task::495::TaskManager.Task::(_debug) Task 37fd06e1-6304-4cfb-a341-0cd34d977391: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:23:54,449::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d280c10c-108b-495f-b211-52f81f975c3e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:23:54,450::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:23:54,451::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:23:54,452::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d280c10c-108b-495f-b211-52f81f975c3e`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:23:54,455::task::495::TaskManager.Task::(_debug) Task 37fd06e1-6304-4cfb-a341-0cd34d977391: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:23:54,456::task::495::TaskManager.Task::(_debug) Task 37fd06e1-6304-4cfb-a341-0cd34d977391: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:23:54,457::task::495::TaskManager.Task::(_debug) Task 37fd06e1-6304-4cfb-a341-0cd34d977391: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:23:54,458::task::495::TaskManager.Task::(_debug) Task 37fd06e1-6304-4cfb-a341-0cd34d977391: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:23:54,459::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:23:54,460::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:23:54,461::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:23:54,462::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:23:54,462::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:23:54,463::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:23:54,464::task::495::TaskManager.Task::(_debug) Task 37fd06e1-6304-4cfb-a341-0cd34d977391: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:23:54,465::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:23:54,822::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:23:55,329::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.08893 s, 11.5 MB/s\n'; <rc> = 0
>Thread-15916318::DEBUG::2013-11-20 04:23:55,467::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916318::INFO::2013-11-20 04:23:55,468::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916318::DEBUG::2013-11-20 04:23:55,469::task::495::TaskManager.Task::(_debug) Task 85de5f6d-d3c5-4fdf-90f0-31f666e406d4: moving from state init -> state preparing
>Thread-15916318::DEBUG::2013-11-20 04:23:55,470::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916318::DEBUG::2013-11-20 04:23:55,470::task::495::TaskManager.Task::(_debug) Task 85de5f6d-d3c5-4fdf-90f0-31f666e406d4: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916318::DEBUG::2013-11-20 04:23:55,471::task::495::TaskManager.Task::(_debug) Task 85de5f6d-d3c5-4fdf-90f0-31f666e406d4: moving from state preparing -> state finished
>Thread-15916318::DEBUG::2013-11-20 04:23:55,472::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916318::DEBUG::2013-11-20 04:23:55,472::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916318::DEBUG::2013-11-20 04:23:55,473::task::495::TaskManager.Task::(_debug) Task 85de5f6d-d3c5-4fdf-90f0-31f666e406d4: ref 0 aborting False
>Thread-15916318::INFO::2013-11-20 04:23:55,473::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916319::DEBUG::2013-11-20 04:23:55,478::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916319::INFO::2013-11-20 04:23:55,479::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916319::DEBUG::2013-11-20 04:23:55,480::task::495::TaskManager.Task::(_debug) Task fcc1b3d4-5209-499f-bb49-75990a88dfa0: moving from state init -> state preparing
>Thread-15916319::DEBUG::2013-11-20 04:23:55,482::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f7a28dce-93ed-4e85-a5bb-d5fa12cdd66c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916319::DEBUG::2013-11-20 04:23:55,483::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916319::DEBUG::2013-11-20 04:23:55,484::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916319::DEBUG::2013-11-20 04:23:55,484::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f7a28dce-93ed-4e85-a5bb-d5fa12cdd66c`::Granted request
>Thread-15916319::DEBUG::2013-11-20 04:23:55,485::task::495::TaskManager.Task::(_debug) Task fcc1b3d4-5209-499f-bb49-75990a88dfa0: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916319::DEBUG::2013-11-20 04:23:55,486::task::495::TaskManager.Task::(_debug) Task fcc1b3d4-5209-499f-bb49-75990a88dfa0: ref 1 aborting False
>Thread-15916319::WARNING::2013-11-20 04:23:55,488::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916319::DEBUG::2013-11-20 04:23:55,489::task::495::TaskManager.Task::(_debug) Task fcc1b3d4-5209-499f-bb49-75990a88dfa0: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916319::DEBUG::2013-11-20 04:23:55,491::task::495::TaskManager.Task::(_debug) Task fcc1b3d4-5209-499f-bb49-75990a88dfa0: moving from state preparing -> state finished
>Thread-15916319::DEBUG::2013-11-20 04:23:55,492::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916319::DEBUG::2013-11-20 04:23:55,492::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916319::DEBUG::2013-11-20 04:23:55,493::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916319::DEBUG::2013-11-20 04:23:55,494::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916319::DEBUG::2013-11-20 04:23:55,494::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916319::DEBUG::2013-11-20 04:23:55,495::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916319::DEBUG::2013-11-20 04:23:55,495::task::495::TaskManager.Task::(_debug) Task fcc1b3d4-5209-499f-bb49-75990a88dfa0: ref 0 aborting False
>Thread-15916319::INFO::2013-11-20 04:23:55,496::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:23:57,343::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:23:57,931::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0824297 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:23:59,945::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:24:00,450::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0756301 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:24:02,465::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:24:02,981::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0657754 s, 15.6 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 04:24:04,469::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:24:04,487::task::495::TaskManager.Task::(_debug) Task 308ad42f-e3bc-4e4c-b16c-148b0b6268c3: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:24:04,489::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3b390744-17a7-4ad0-8f6e-9782f22e62c4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:24:04,491::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:24:04,492::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:24:04,492::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3b390744-17a7-4ad0-8f6e-9782f22e62c4`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:24:04,493::task::495::TaskManager.Task::(_debug) Task 308ad42f-e3bc-4e4c-b16c-148b0b6268c3: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:24:04,494::task::495::TaskManager.Task::(_debug) Task 308ad42f-e3bc-4e4c-b16c-148b0b6268c3: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:24:04,495::task::495::TaskManager.Task::(_debug) Task 308ad42f-e3bc-4e4c-b16c-148b0b6268c3: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:24:04,496::task::495::TaskManager.Task::(_debug) Task 308ad42f-e3bc-4e4c-b16c-148b0b6268c3: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:24:04,496::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:24:04,497::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:24:04,498::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:24:04,498::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:24:04,499::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:24:04,500::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:24:04,501::task::495::TaskManager.Task::(_debug) Task 308ad42f-e3bc-4e4c-b16c-148b0b6268c3: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:24:04,501::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:24:04,997::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:24:05,574::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0782278 s, 13.1 MB/s\n'; <rc> = 0
>Thread-15916323::DEBUG::2013-11-20 04:24:05,785::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916323::INFO::2013-11-20 04:24:05,787::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916323::DEBUG::2013-11-20 04:24:05,789::task::495::TaskManager.Task::(_debug) Task 3d2e9719-996e-4cf4-8ea0-35862d346e95: moving from state init -> state preparing
>Thread-15916323::DEBUG::2013-11-20 04:24:05,790::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916323::DEBUG::2013-11-20 04:24:05,791::task::495::TaskManager.Task::(_debug) Task 3d2e9719-996e-4cf4-8ea0-35862d346e95: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916323::DEBUG::2013-11-20 04:24:05,792::task::495::TaskManager.Task::(_debug) Task 3d2e9719-996e-4cf4-8ea0-35862d346e95: moving from state preparing -> state finished
>Thread-15916323::DEBUG::2013-11-20 04:24:05,793::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916323::DEBUG::2013-11-20 04:24:05,793::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916323::DEBUG::2013-11-20 04:24:05,794::task::495::TaskManager.Task::(_debug) Task 3d2e9719-996e-4cf4-8ea0-35862d346e95: ref 0 aborting False
>Thread-15916323::INFO::2013-11-20 04:24:05,795::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916324::DEBUG::2013-11-20 04:24:05,801::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916324::INFO::2013-11-20 04:24:05,803::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916324::DEBUG::2013-11-20 04:24:05,804::task::495::TaskManager.Task::(_debug) Task fbf20c1c-3660-42c6-a7f9-bad38c470c9d: moving from state init -> state preparing
>Thread-15916324::DEBUG::2013-11-20 04:24:05,807::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6eb3cdd9-64d3-4940-97a0-5cf90cccc028`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916324::DEBUG::2013-11-20 04:24:05,809::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916324::DEBUG::2013-11-20 04:24:05,811::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916324::DEBUG::2013-11-20 04:24:05,812::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6eb3cdd9-64d3-4940-97a0-5cf90cccc028`::Granted request
>Thread-15916324::DEBUG::2013-11-20 04:24:05,820::task::495::TaskManager.Task::(_debug) Task fbf20c1c-3660-42c6-a7f9-bad38c470c9d: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916324::DEBUG::2013-11-20 04:24:05,830::task::495::TaskManager.Task::(_debug) Task fbf20c1c-3660-42c6-a7f9-bad38c470c9d: ref 1 aborting False
>Thread-15916324::WARNING::2013-11-20 04:24:05,839::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916324::DEBUG::2013-11-20 04:24:05,844::task::495::TaskManager.Task::(_debug) Task fbf20c1c-3660-42c6-a7f9-bad38c470c9d: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916324::DEBUG::2013-11-20 04:24:05,845::task::495::TaskManager.Task::(_debug) Task fbf20c1c-3660-42c6-a7f9-bad38c470c9d: moving from state preparing -> state finished
>Thread-15916324::DEBUG::2013-11-20 04:24:05,846::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916324::DEBUG::2013-11-20 04:24:05,847::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916324::DEBUG::2013-11-20 04:24:05,850::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916324::DEBUG::2013-11-20 04:24:05,852::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916324::DEBUG::2013-11-20 04:24:05,853::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916324::DEBUG::2013-11-20 04:24:05,854::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916324::DEBUG::2013-11-20 04:24:05,855::task::495::TaskManager.Task::(_debug) Task fbf20c1c-3660-42c6-a7f9-bad38c470c9d: ref 0 aborting False
>Thread-15916324::INFO::2013-11-20 04:24:05,856::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:24:07,590::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:24:08,135::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0790119 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15916326::INFO::2013-11-20 04:24:09,285::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916326::DEBUG::2013-11-20 04:24:09,286::task::495::TaskManager.Task::(_debug) Task 9e9b70e2-5f13-4a21-b955-60568982e44c: moving from state init -> state preparing
>Thread-15916326::DEBUG::2013-11-20 04:24:09,287::task::495::TaskManager.Task::(_debug) Task 9e9b70e2-5f13-4a21-b955-60568982e44c: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000339031219482', 'lastCheck': 1384921444.1369309, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00205588340759', 'lastCheck': 1384921442.691879, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00183510780334', 'lastCheck': 1384921439.5685239, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000715017318726', 'lastCheck': 1384921448.811331, 'valid': True, 'code': 0}}
>Thread-15916326::DEBUG::2013-11-20 04:24:09,288::task::495::TaskManager.Task::(_debug) Task 9e9b70e2-5f13-4a21-b955-60568982e44c: moving from state preparing -> state finished
>Thread-15916326::DEBUG::2013-11-20 04:24:09,288::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916326::DEBUG::2013-11-20 04:24:09,289::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916326::DEBUG::2013-11-20 04:24:09,290::task::495::TaskManager.Task::(_debug) Task 9e9b70e2-5f13-4a21-b955-60568982e44c: ref 0 aborting False
>Thread-15916326::INFO::2013-11-20 04:24:09,290::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000339031219482', 'lastCheck': 1384921444.1369309, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00205588340759', 'lastCheck': 1384921442.691879, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00183510780334', 'lastCheck': 1384921439.5685239, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000715017318726', 'lastCheck': 1384921448.811331, 'valid': True, 'code': 0}}
>Thread-15913695::INFO::2013-11-20 04:24:09,828::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:24:09,829::task::495::TaskManager.Task::(_debug) Task 0e31cde2-1e5e-4934-bc88-72d7d1d73531: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:24:09,831::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5f629297-1725-4eaf-b856-aa837f99aa80`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:24:09,831::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:24:09,832::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:24:09,833::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5f629297-1725-4eaf-b856-aa837f99aa80`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:24:09,834::task::495::TaskManager.Task::(_debug) Task 0e31cde2-1e5e-4934-bc88-72d7d1d73531: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:24:09,834::task::495::TaskManager.Task::(_debug) Task 0e31cde2-1e5e-4934-bc88-72d7d1d73531: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:24:09,835::task::495::TaskManager.Task::(_debug) Task 0e31cde2-1e5e-4934-bc88-72d7d1d73531: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:24:09,836::task::495::TaskManager.Task::(_debug) Task 0e31cde2-1e5e-4934-bc88-72d7d1d73531: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:24:09,837::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:24:09,837::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:24:09,838::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:24:09,839::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:24:09,840::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:24:09,840::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:24:09,841::task::495::TaskManager.Task::(_debug) Task 0e31cde2-1e5e-4934-bc88-72d7d1d73531: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:24:09,842::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:24:10,154::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:24:10,700::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0757163 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:24:12,715::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:24:13,278::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0836051 s, 12.2 MB/s\n'; <rc> = 0
>Thread-15914453::INFO::2013-11-20 04:24:14,494::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:24:14,495::task::495::TaskManager.Task::(_debug) Task 71ebca85-ec80-490d-bbe1-53f495e9dc39: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:24:14,503::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ada659d4-7b7f-4e4b-9bdf-0160315381d3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:24:14,503::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:24:14,504::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:24:14,505::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ada659d4-7b7f-4e4b-9bdf-0160315381d3`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:24:14,515::task::495::TaskManager.Task::(_debug) Task 71ebca85-ec80-490d-bbe1-53f495e9dc39: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:24:14,515::task::495::TaskManager.Task::(_debug) Task 71ebca85-ec80-490d-bbe1-53f495e9dc39: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:24:14,516::task::495::TaskManager.Task::(_debug) Task 71ebca85-ec80-490d-bbe1-53f495e9dc39: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:24:14,518::task::495::TaskManager.Task::(_debug) Task 71ebca85-ec80-490d-bbe1-53f495e9dc39: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:24:14,518::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:24:14,519::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:24:14,520::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:24:14,521::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:24:14,521::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:24:14,522::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:24:14,522::task::495::TaskManager.Task::(_debug) Task 71ebca85-ec80-490d-bbe1-53f495e9dc39: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:24:14,523::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:24:15,293::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:24:15,868::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0856704 s, 12.0 MB/s\n'; <rc> = 0
>Thread-15916330::DEBUG::2013-11-20 04:24:16,428::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916330::INFO::2013-11-20 04:24:16,430::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916330::DEBUG::2013-11-20 04:24:16,430::task::495::TaskManager.Task::(_debug) Task b9956db9-b22f-45ac-a56f-440bae9ee699: moving from state init -> state preparing
>Thread-15916330::DEBUG::2013-11-20 04:24:16,431::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916330::DEBUG::2013-11-20 04:24:16,432::task::495::TaskManager.Task::(_debug) Task b9956db9-b22f-45ac-a56f-440bae9ee699: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916330::DEBUG::2013-11-20 04:24:16,433::task::495::TaskManager.Task::(_debug) Task b9956db9-b22f-45ac-a56f-440bae9ee699: moving from state preparing -> state finished
>Thread-15916330::DEBUG::2013-11-20 04:24:16,433::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916330::DEBUG::2013-11-20 04:24:16,434::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916330::DEBUG::2013-11-20 04:24:16,435::task::495::TaskManager.Task::(_debug) Task b9956db9-b22f-45ac-a56f-440bae9ee699: ref 0 aborting False
>Thread-15916330::INFO::2013-11-20 04:24:16,435::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916331::DEBUG::2013-11-20 04:24:16,440::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916331::INFO::2013-11-20 04:24:16,441::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916331::DEBUG::2013-11-20 04:24:16,442::task::495::TaskManager.Task::(_debug) Task 5f5d750f-ea9a-46f9-bb2e-bb1a5c1970c9: moving from state init -> state preparing
>Thread-15916331::DEBUG::2013-11-20 04:24:16,444::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4da17e0d-acbc-44f0-81b2-21bc7a1ad720`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916331::DEBUG::2013-11-20 04:24:16,444::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916331::DEBUG::2013-11-20 04:24:16,445::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916331::DEBUG::2013-11-20 04:24:16,446::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4da17e0d-acbc-44f0-81b2-21bc7a1ad720`::Granted request
>Thread-15916331::DEBUG::2013-11-20 04:24:16,448::task::495::TaskManager.Task::(_debug) Task 5f5d750f-ea9a-46f9-bb2e-bb1a5c1970c9: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916331::DEBUG::2013-11-20 04:24:16,449::task::495::TaskManager.Task::(_debug) Task 5f5d750f-ea9a-46f9-bb2e-bb1a5c1970c9: ref 1 aborting False
>Thread-15916331::WARNING::2013-11-20 04:24:16,451::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916331::DEBUG::2013-11-20 04:24:16,452::task::495::TaskManager.Task::(_debug) Task 5f5d750f-ea9a-46f9-bb2e-bb1a5c1970c9: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916331::DEBUG::2013-11-20 04:24:16,454::task::495::TaskManager.Task::(_debug) Task 5f5d750f-ea9a-46f9-bb2e-bb1a5c1970c9: moving from state preparing -> state finished
>Thread-15916331::DEBUG::2013-11-20 04:24:16,454::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916331::DEBUG::2013-11-20 04:24:16,455::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916331::DEBUG::2013-11-20 04:24:16,456::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916331::DEBUG::2013-11-20 04:24:16,457::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916331::DEBUG::2013-11-20 04:24:16,457::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916331::DEBUG::2013-11-20 04:24:16,458::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916331::DEBUG::2013-11-20 04:24:16,459::task::495::TaskManager.Task::(_debug) Task 5f5d750f-ea9a-46f9-bb2e-bb1a5c1970c9: ref 0 aborting False
>Thread-15916331::INFO::2013-11-20 04:24:16,459::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:24:17,887::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:24:18,470::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0861349 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:24:20,484::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:24:21,030::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0587792 s, 17.4 MB/s\n'; <rc> = 0
>Thread-15914519::INFO::2013-11-20 04:24:22,106::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:24:22,107::task::495::TaskManager.Task::(_debug) Task 7aa37448-3d1b-4245-a32d-5c26c2791663: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:24:22,118::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1af0f53b-5274-40f9-9bcb-c10713495249`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:24:22,119::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:24:22,120::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:24:22,121::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1af0f53b-5274-40f9-9bcb-c10713495249`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:24:22,129::task::495::TaskManager.Task::(_debug) Task 7aa37448-3d1b-4245-a32d-5c26c2791663: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 04:24:22,130::task::495::TaskManager.Task::(_debug) Task 7aa37448-3d1b-4245-a32d-5c26c2791663: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:24:22,131::task::495::TaskManager.Task::(_debug) Task 7aa37448-3d1b-4245-a32d-5c26c2791663: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 04:24:22,131::task::495::TaskManager.Task::(_debug) Task 7aa37448-3d1b-4245-a32d-5c26c2791663: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:24:22,132::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:24:22,133::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:24:22,133::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 04:24:22,134::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:24:22,134::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:24:22,135::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:24:22,136::task::495::TaskManager.Task::(_debug) Task 7aa37448-3d1b-4245-a32d-5c26c2791663: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 04:24:22,136::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 04:24:23,039::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:24:23,595::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0917975 s, 11.2 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 04:24:24,732::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:24:24,733::task::495::TaskManager.Task::(_debug) Task 83bb7b8e-15bb-4d71-a0df-f77fe3e0af6c: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:24:24,734::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a383fb45-00ee-43ee-9074-ab8bf592ef3a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:24:24,735::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:24:24,736::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:24:24,737::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a383fb45-00ee-43ee-9074-ab8bf592ef3a`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:24:24,738::task::495::TaskManager.Task::(_debug) Task 83bb7b8e-15bb-4d71-a0df-f77fe3e0af6c: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:24:24,738::task::495::TaskManager.Task::(_debug) Task 83bb7b8e-15bb-4d71-a0df-f77fe3e0af6c: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:24:24,739::task::495::TaskManager.Task::(_debug) Task 83bb7b8e-15bb-4d71-a0df-f77fe3e0af6c: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 04:24:24,740::task::495::TaskManager.Task::(_debug) Task 83bb7b8e-15bb-4d71-a0df-f77fe3e0af6c: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:24:24,741::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:24:24,741::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:24:24,742::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:24:24,743::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:24:24,743::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:24:24,744::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:24:24,744::task::495::TaskManager.Task::(_debug) Task 83bb7b8e-15bb-4d71-a0df-f77fe3e0af6c: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:24:24,745::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:24:24,746::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:24:24,746::task::495::TaskManager.Task::(_debug) Task e7e9c4ee-3b99-42c1-a378-ce75491385ae: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:24:24,748::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`1b4d7119-2779-49c8-9e88-33c15b013441`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:24:24,749::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:24:24,749::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:24:24,750::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`1b4d7119-2779-49c8-9e88-33c15b013441`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:24:24,752::task::495::TaskManager.Task::(_debug) Task e7e9c4ee-3b99-42c1-a378-ce75491385ae: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:24:24,753::task::495::TaskManager.Task::(_debug) Task e7e9c4ee-3b99-42c1-a378-ce75491385ae: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:24:24,754::task::495::TaskManager.Task::(_debug) Task e7e9c4ee-3b99-42c1-a378-ce75491385ae: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:24:24,754::task::495::TaskManager.Task::(_debug) Task e7e9c4ee-3b99-42c1-a378-ce75491385ae: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:24:24,755::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:24:24,755::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:24:24,756::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:24:24,757::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:24:24,757::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:24:24,758::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:24:24,759::task::495::TaskManager.Task::(_debug) Task e7e9c4ee-3b99-42c1-a378-ce75491385ae: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:24:24,759::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:24:24,760::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:24:24,761::task::495::TaskManager.Task::(_debug) Task 0ae48661-c74f-4d48-9a8e-b39263da8a8f: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:24:24,762::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`96a600bf-11a1-4308-aebf-2f12d054c26e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:24:24,763::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:24:24,763::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:24:24,764::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`96a600bf-11a1-4308-aebf-2f12d054c26e`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:24:24,765::task::495::TaskManager.Task::(_debug) Task 0ae48661-c74f-4d48-9a8e-b39263da8a8f: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:24:24,766::task::495::TaskManager.Task::(_debug) Task 0ae48661-c74f-4d48-9a8e-b39263da8a8f: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:24:24,766::task::495::TaskManager.Task::(_debug) Task 0ae48661-c74f-4d48-9a8e-b39263da8a8f: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:24:24,767::task::495::TaskManager.Task::(_debug) Task 0ae48661-c74f-4d48-9a8e-b39263da8a8f: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:24:24,768::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:24:24,768::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:24:24,769::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:24:24,769::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:24:24,770::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:24:24,770::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:24:24,771::task::495::TaskManager.Task::(_debug) Task 0ae48661-c74f-4d48-9a8e-b39263da8a8f: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:24:24,772::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:24:24,773::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:24:24,773::task::495::TaskManager.Task::(_debug) Task 4352879c-1b29-4f46-bb5c-1f20c2d2c416: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:24:24,774::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`aa22a1ae-6315-45eb-8aa2-aa00fc4094c3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:24:24,775::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:24:24,776::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:24:24,776::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`aa22a1ae-6315-45eb-8aa2-aa00fc4094c3`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:24:24,777::task::495::TaskManager.Task::(_debug) Task 4352879c-1b29-4f46-bb5c-1f20c2d2c416: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:24:24,778::task::495::TaskManager.Task::(_debug) Task 4352879c-1b29-4f46-bb5c-1f20c2d2c416: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:24:24,778::task::495::TaskManager.Task::(_debug) Task 4352879c-1b29-4f46-bb5c-1f20c2d2c416: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:24:24,779::task::495::TaskManager.Task::(_debug) Task 4352879c-1b29-4f46-bb5c-1f20c2d2c416: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:24:24,780::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:24:24,780::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:24:24,781::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:24:24,782::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:24:24,782::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:24:24,783::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:24:24,784::task::495::TaskManager.Task::(_debug) Task 4352879c-1b29-4f46-bb5c-1f20c2d2c416: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:24:24,784::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15914435::INFO::2013-11-20 04:24:25,022::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:24:25,023::task::495::TaskManager.Task::(_debug) Task 457f1e83-5d18-4bc8-bb34-084a44c94400: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:24:25,024::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`43d9988a-25b3-4563-a581-07b2e7fc9401`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:24:25,026::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:24:25,027::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:24:25,028::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`43d9988a-25b3-4563-a581-07b2e7fc9401`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:24:25,029::task::495::TaskManager.Task::(_debug) Task 457f1e83-5d18-4bc8-bb34-084a44c94400: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:24:25,029::task::495::TaskManager.Task::(_debug) Task 457f1e83-5d18-4bc8-bb34-084a44c94400: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:24:25,030::task::495::TaskManager.Task::(_debug) Task 457f1e83-5d18-4bc8-bb34-084a44c94400: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:24:25,031::task::495::TaskManager.Task::(_debug) Task 457f1e83-5d18-4bc8-bb34-084a44c94400: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:24:25,032::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:24:25,033::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:24:25,033::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:24:25,034::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:24:25,035::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:24:25,035::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:24:25,036::task::495::TaskManager.Task::(_debug) Task 457f1e83-5d18-4bc8-bb34-084a44c94400: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:24:25,036::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15916334::INFO::2013-11-20 04:24:25,437::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916334::DEBUG::2013-11-20 04:24:25,438::task::495::TaskManager.Task::(_debug) Task 2e545b69-1e22-43ee-b254-bdb64fa794bc: moving from state init -> state preparing
>Thread-15916334::DEBUG::2013-11-20 04:24:25,439::task::495::TaskManager.Task::(_debug) Task 2e545b69-1e22-43ee-b254-bdb64fa794bc: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000382900238037', 'lastCheck': 1384921464.7193301, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00226712226868', 'lastCheck': 1384921463.2596669, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00175809860229', 'lastCheck': 1384921460.1131649, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000529050827026', 'lastCheck': 1384921459.1039791, 'valid': True, 'code': 0}}
>Thread-15916334::DEBUG::2013-11-20 04:24:25,440::task::495::TaskManager.Task::(_debug) Task 2e545b69-1e22-43ee-b254-bdb64fa794bc: moving from state preparing -> state finished
>Thread-15916334::DEBUG::2013-11-20 04:24:25,441::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916334::DEBUG::2013-11-20 04:24:25,442::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916334::DEBUG::2013-11-20 04:24:25,442::task::495::TaskManager.Task::(_debug) Task 2e545b69-1e22-43ee-b254-bdb64fa794bc: ref 0 aborting False
>Thread-15916334::INFO::2013-11-20 04:24:25,443::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000382900238037', 'lastCheck': 1384921464.7193301, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00226712226868', 'lastCheck': 1384921463.2596669, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00175809860229', 'lastCheck': 1384921460.1131649, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000529050827026', 'lastCheck': 1384921459.1039791, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:24:25,610::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:24:26,170::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0764517 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15916336::DEBUG::2013-11-20 04:24:27,053::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916336::INFO::2013-11-20 04:24:27,055::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916336::DEBUG::2013-11-20 04:24:27,056::task::495::TaskManager.Task::(_debug) Task 0ddd4da0-51df-49b5-b5e3-0d3fef4bd054: moving from state init -> state preparing
>Thread-15916336::DEBUG::2013-11-20 04:24:27,057::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916336::DEBUG::2013-11-20 04:24:27,057::task::495::TaskManager.Task::(_debug) Task 0ddd4da0-51df-49b5-b5e3-0d3fef4bd054: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916336::DEBUG::2013-11-20 04:24:27,058::task::495::TaskManager.Task::(_debug) Task 0ddd4da0-51df-49b5-b5e3-0d3fef4bd054: moving from state preparing -> state finished
>Thread-15916336::DEBUG::2013-11-20 04:24:27,059::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916336::DEBUG::2013-11-20 04:24:27,059::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916336::DEBUG::2013-11-20 04:24:27,060::task::495::TaskManager.Task::(_debug) Task 0ddd4da0-51df-49b5-b5e3-0d3fef4bd054: ref 0 aborting False
>Thread-15916336::INFO::2013-11-20 04:24:27,061::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916337::DEBUG::2013-11-20 04:24:27,071::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916337::INFO::2013-11-20 04:24:27,073::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916337::DEBUG::2013-11-20 04:24:27,073::task::495::TaskManager.Task::(_debug) Task cb158bcb-9c5e-423f-bfd4-2b5f90a4272a: moving from state init -> state preparing
>Thread-15916337::DEBUG::2013-11-20 04:24:27,075::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a2eb060a-1e2f-4b17-99e4-f164aec1aabf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916337::DEBUG::2013-11-20 04:24:27,076::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916337::DEBUG::2013-11-20 04:24:27,076::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916337::DEBUG::2013-11-20 04:24:27,077::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a2eb060a-1e2f-4b17-99e4-f164aec1aabf`::Granted request
>Thread-15916337::DEBUG::2013-11-20 04:24:27,078::task::495::TaskManager.Task::(_debug) Task cb158bcb-9c5e-423f-bfd4-2b5f90a4272a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916337::DEBUG::2013-11-20 04:24:27,079::task::495::TaskManager.Task::(_debug) Task cb158bcb-9c5e-423f-bfd4-2b5f90a4272a: ref 1 aborting False
>Thread-15916337::WARNING::2013-11-20 04:24:27,081::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916337::DEBUG::2013-11-20 04:24:27,082::task::495::TaskManager.Task::(_debug) Task cb158bcb-9c5e-423f-bfd4-2b5f90a4272a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916337::DEBUG::2013-11-20 04:24:27,083::task::495::TaskManager.Task::(_debug) Task cb158bcb-9c5e-423f-bfd4-2b5f90a4272a: moving from state preparing -> state finished
>Thread-15916337::DEBUG::2013-11-20 04:24:27,084::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916337::DEBUG::2013-11-20 04:24:27,085::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916337::DEBUG::2013-11-20 04:24:27,085::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916337::DEBUG::2013-11-20 04:24:27,086::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916337::DEBUG::2013-11-20 04:24:27,087::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916337::DEBUG::2013-11-20 04:24:27,087::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916337::DEBUG::2013-11-20 04:24:27,088::task::495::TaskManager.Task::(_debug) Task cb158bcb-9c5e-423f-bfd4-2b5f90a4272a: ref 0 aborting False
>Thread-15916337::INFO::2013-11-20 04:24:27,089::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:24:28,186::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:24:28,666::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0791843 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:24:30,679::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:24:31,168::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0522328 s, 19.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:24:33,179::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:24:33,749::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.103257 s, 9.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:24:35,765::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:24:36,332::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0798782 s, 12.8 MB/s\n'; <rc> = 0
>Thread-15916341::DEBUG::2013-11-20 04:24:37,334::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916341::INFO::2013-11-20 04:24:37,335::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916341::DEBUG::2013-11-20 04:24:37,335::task::495::TaskManager.Task::(_debug) Task 43fe7b66-d32d-409d-8e44-e7a52e3aec23: moving from state init -> state preparing
>Thread-15916341::DEBUG::2013-11-20 04:24:37,336::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916341::DEBUG::2013-11-20 04:24:37,337::task::495::TaskManager.Task::(_debug) Task 43fe7b66-d32d-409d-8e44-e7a52e3aec23: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916341::DEBUG::2013-11-20 04:24:37,337::task::495::TaskManager.Task::(_debug) Task 43fe7b66-d32d-409d-8e44-e7a52e3aec23: moving from state preparing -> state finished
>Thread-15916341::DEBUG::2013-11-20 04:24:37,338::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916341::DEBUG::2013-11-20 04:24:37,338::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916341::DEBUG::2013-11-20 04:24:37,338::task::495::TaskManager.Task::(_debug) Task 43fe7b66-d32d-409d-8e44-e7a52e3aec23: ref 0 aborting False
>Thread-15916341::INFO::2013-11-20 04:24:37,339::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916342::DEBUG::2013-11-20 04:24:37,343::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916342::INFO::2013-11-20 04:24:37,345::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916342::DEBUG::2013-11-20 04:24:37,346::task::495::TaskManager.Task::(_debug) Task 4f80126d-27ba-4e11-9bbb-10bc186c6c8a: moving from state init -> state preparing
>Thread-15916342::DEBUG::2013-11-20 04:24:37,347::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bd9a19a6-d79c-486c-90b9-bd24940c5342`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916342::DEBUG::2013-11-20 04:24:37,348::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916342::DEBUG::2013-11-20 04:24:37,349::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916342::DEBUG::2013-11-20 04:24:37,349::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bd9a19a6-d79c-486c-90b9-bd24940c5342`::Granted request
>Thread-15916342::DEBUG::2013-11-20 04:24:37,350::task::495::TaskManager.Task::(_debug) Task 4f80126d-27ba-4e11-9bbb-10bc186c6c8a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916342::DEBUG::2013-11-20 04:24:37,351::task::495::TaskManager.Task::(_debug) Task 4f80126d-27ba-4e11-9bbb-10bc186c6c8a: ref 1 aborting False
>Thread-15916342::WARNING::2013-11-20 04:24:37,353::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916342::DEBUG::2013-11-20 04:24:37,354::task::495::TaskManager.Task::(_debug) Task 4f80126d-27ba-4e11-9bbb-10bc186c6c8a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916342::DEBUG::2013-11-20 04:24:37,355::task::495::TaskManager.Task::(_debug) Task 4f80126d-27ba-4e11-9bbb-10bc186c6c8a: moving from state preparing -> state finished
>Thread-15916342::DEBUG::2013-11-20 04:24:37,355::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916342::DEBUG::2013-11-20 04:24:37,356::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916342::DEBUG::2013-11-20 04:24:37,357::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916342::DEBUG::2013-11-20 04:24:37,357::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916342::DEBUG::2013-11-20 04:24:37,358::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916342::DEBUG::2013-11-20 04:24:37,358::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916342::DEBUG::2013-11-20 04:24:37,359::task::495::TaskManager.Task::(_debug) Task 4f80126d-27ba-4e11-9bbb-10bc186c6c8a: ref 0 aborting False
>Thread-15916342::INFO::2013-11-20 04:24:37,360::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:24:38,348::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:24:38,880::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0778272 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:24:40,896::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:24:41,102::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:24:41,104::task::495::TaskManager.Task::(_debug) Task 30bfe068-0c33-43a5-ad85-1261490a128f: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:24:41,107::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`92855574-8241-4b8f-b8c6-ffe3f35671f8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:24:41,108::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:24:41,110::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:24:41,111::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`92855574-8241-4b8f-b8c6-ffe3f35671f8`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:24:41,112::task::495::TaskManager.Task::(_debug) Task 30bfe068-0c33-43a5-ad85-1261490a128f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:24:41,113::task::495::TaskManager.Task::(_debug) Task 30bfe068-0c33-43a5-ad85-1261490a128f: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:24:41,114::task::495::TaskManager.Task::(_debug) Task 30bfe068-0c33-43a5-ad85-1261490a128f: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:24:41,114::task::495::TaskManager.Task::(_debug) Task 30bfe068-0c33-43a5-ad85-1261490a128f: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:24:41,115::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:24:41,116::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:24:41,117::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:24:41,117::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:24:41,118::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:24:41,118::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:24:41,119::task::495::TaskManager.Task::(_debug) Task 30bfe068-0c33-43a5-ad85-1261490a128f: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:24:41,120::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15916344::INFO::2013-11-20 04:24:41,263::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916344::DEBUG::2013-11-20 04:24:41,264::task::495::TaskManager.Task::(_debug) Task 7be70484-3539-4f34-ada5-f158d880ce46: moving from state init -> state preparing
>Thread-15916344::DEBUG::2013-11-20 04:24:41,265::task::495::TaskManager.Task::(_debug) Task 7be70484-3539-4f34-ada5-f158d880ce46: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000634908676147', 'lastCheck': 1384921474.901032, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0125098228455', 'lastCheck': 1384921473.4021809, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00140500068665', 'lastCheck': 1384921480.5862739, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000691175460815', 'lastCheck': 1384921479.578779, 'valid': True, 'code': 0}}
>Thread-15916344::DEBUG::2013-11-20 04:24:41,266::task::495::TaskManager.Task::(_debug) Task 7be70484-3539-4f34-ada5-f158d880ce46: moving from state preparing -> state finished
>Thread-15916344::DEBUG::2013-11-20 04:24:41,267::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916344::DEBUG::2013-11-20 04:24:41,267::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916344::DEBUG::2013-11-20 04:24:41,268::task::495::TaskManager.Task::(_debug) Task 7be70484-3539-4f34-ada5-f158d880ce46: ref 0 aborting False
>Thread-15916344::INFO::2013-11-20 04:24:41,268::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000634908676147', 'lastCheck': 1384921474.901032, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0125098228455', 'lastCheck': 1384921473.4021809, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00140500068665', 'lastCheck': 1384921480.5862739, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000691175460815', 'lastCheck': 1384921479.578779, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:24:41,467::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.120586 s, 8.5 MB/s\n'; <rc> = 0
>Thread-15915918::INFO::2013-11-20 04:24:42,137::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:24:42,137::task::495::TaskManager.Task::(_debug) Task 94fa483e-738c-4259-82fd-fc8cc228383e: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:24:42,139::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9b521611-b887-464a-b8ef-a845053f01fa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:24:42,139::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:24:42,140::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:24:42,141::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9b521611-b887-464a-b8ef-a845053f01fa`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:24:42,142::task::495::TaskManager.Task::(_debug) Task 94fa483e-738c-4259-82fd-fc8cc228383e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:24:42,142::task::495::TaskManager.Task::(_debug) Task 94fa483e-738c-4259-82fd-fc8cc228383e: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:24:42,143::task::495::TaskManager.Task::(_debug) Task 94fa483e-738c-4259-82fd-fc8cc228383e: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:24:42,144::task::495::TaskManager.Task::(_debug) Task 94fa483e-738c-4259-82fd-fc8cc228383e: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:24:42,144::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:24:42,145::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:24:42,145::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:24:42,146::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:24:42,147::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:24:42,147::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:24:42,148::task::495::TaskManager.Task::(_debug) Task 94fa483e-738c-4259-82fd-fc8cc228383e: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:24:42,148::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:24:43,482::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:24:44,033::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0904402 s, 11.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:24:46,048::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:24:46,545::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.076471 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15915863::INFO::2013-11-20 04:24:47,393::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:24:47,394::task::495::TaskManager.Task::(_debug) Task 4ab1ef8b-ad7d-4cc5-aa85-80d5b966effc: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:24:47,396::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`23c1abb7-e42e-4fa0-a54f-5cfcdea3601c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:24:47,396::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:24:47,398::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:24:47,398::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`23c1abb7-e42e-4fa0-a54f-5cfcdea3601c`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:24:47,399::task::495::TaskManager.Task::(_debug) Task 4ab1ef8b-ad7d-4cc5-aa85-80d5b966effc: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:24:47,400::task::495::TaskManager.Task::(_debug) Task 4ab1ef8b-ad7d-4cc5-aa85-80d5b966effc: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:24:47,401::task::495::TaskManager.Task::(_debug) Task 4ab1ef8b-ad7d-4cc5-aa85-80d5b966effc: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:24:47,401::task::495::TaskManager.Task::(_debug) Task 4ab1ef8b-ad7d-4cc5-aa85-80d5b966effc: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:24:47,402::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:24:47,403::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:24:47,404::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:24:47,405::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:24:47,405::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:24:47,406::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:24:47,407::task::495::TaskManager.Task::(_debug) Task 4ab1ef8b-ad7d-4cc5-aa85-80d5b966effc: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:24:47,407::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15916347::DEBUG::2013-11-20 04:24:47,862::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916347::INFO::2013-11-20 04:24:47,863::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916347::DEBUG::2013-11-20 04:24:47,864::task::495::TaskManager.Task::(_debug) Task a84772c2-6d00-4444-ad8b-a8c094da3643: moving from state init -> state preparing
>Thread-15916347::DEBUG::2013-11-20 04:24:47,865::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916347::DEBUG::2013-11-20 04:24:47,865::task::495::TaskManager.Task::(_debug) Task a84772c2-6d00-4444-ad8b-a8c094da3643: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916347::DEBUG::2013-11-20 04:24:47,866::task::495::TaskManager.Task::(_debug) Task a84772c2-6d00-4444-ad8b-a8c094da3643: moving from state preparing -> state finished
>Thread-15916347::DEBUG::2013-11-20 04:24:47,867::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916347::DEBUG::2013-11-20 04:24:47,867::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916347::DEBUG::2013-11-20 04:24:47,868::task::495::TaskManager.Task::(_debug) Task a84772c2-6d00-4444-ad8b-a8c094da3643: ref 0 aborting False
>Thread-15916347::INFO::2013-11-20 04:24:47,868::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916348::DEBUG::2013-11-20 04:24:47,874::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916348::INFO::2013-11-20 04:24:47,875::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916348::DEBUG::2013-11-20 04:24:47,876::task::495::TaskManager.Task::(_debug) Task 7c930f58-0511-4435-94c4-c496f3aa627b: moving from state init -> state preparing
>Thread-15916348::DEBUG::2013-11-20 04:24:47,878::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`45c87d50-27fc-40e8-9c67-fddf09d2e62e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916348::DEBUG::2013-11-20 04:24:47,878::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916348::DEBUG::2013-11-20 04:24:47,879::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916348::DEBUG::2013-11-20 04:24:47,880::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`45c87d50-27fc-40e8-9c67-fddf09d2e62e`::Granted request
>Thread-15916348::DEBUG::2013-11-20 04:24:47,881::task::495::TaskManager.Task::(_debug) Task 7c930f58-0511-4435-94c4-c496f3aa627b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916348::DEBUG::2013-11-20 04:24:47,882::task::495::TaskManager.Task::(_debug) Task 7c930f58-0511-4435-94c4-c496f3aa627b: ref 1 aborting False
>Thread-15916348::WARNING::2013-11-20 04:24:47,884::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916348::DEBUG::2013-11-20 04:24:47,885::task::495::TaskManager.Task::(_debug) Task 7c930f58-0511-4435-94c4-c496f3aa627b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916348::DEBUG::2013-11-20 04:24:47,886::task::495::TaskManager.Task::(_debug) Task 7c930f58-0511-4435-94c4-c496f3aa627b: moving from state preparing -> state finished
>Thread-15916348::DEBUG::2013-11-20 04:24:47,887::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916348::DEBUG::2013-11-20 04:24:47,887::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916348::DEBUG::2013-11-20 04:24:47,888::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916348::DEBUG::2013-11-20 04:24:47,889::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916348::DEBUG::2013-11-20 04:24:47,889::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916348::DEBUG::2013-11-20 04:24:47,890::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916348::DEBUG::2013-11-20 04:24:47,891::task::495::TaskManager.Task::(_debug) Task 7c930f58-0511-4435-94c4-c496f3aa627b: ref 0 aborting False
>Thread-15916348::INFO::2013-11-20 04:24:47,892::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:24:48,560::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:24:49,126::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0836974 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:24:51,144::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:24:51,703::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0771909 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:24:53,716::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913518::INFO::2013-11-20 04:24:53,961::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:24:53,962::task::495::TaskManager.Task::(_debug) Task 66548fce-32ae-4257-aa45-b6f8f8ed2c4a: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:24:53,965::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0e869593-e9f0-4f0e-9a6b-5b44ea0d86c1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:24:53,965::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:24:53,967::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:24:53,967::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0e869593-e9f0-4f0e-9a6b-5b44ea0d86c1`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:24:53,969::task::495::TaskManager.Task::(_debug) Task 66548fce-32ae-4257-aa45-b6f8f8ed2c4a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:24:53,969::task::495::TaskManager.Task::(_debug) Task 66548fce-32ae-4257-aa45-b6f8f8ed2c4a: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:24:53,970::task::495::TaskManager.Task::(_debug) Task 66548fce-32ae-4257-aa45-b6f8f8ed2c4a: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:24:53,971::task::495::TaskManager.Task::(_debug) Task 66548fce-32ae-4257-aa45-b6f8f8ed2c4a: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:24:53,972::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:24:53,973::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:24:53,974::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:24:53,975::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:24:53,975::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:24:53,976::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:24:53,977::task::495::TaskManager.Task::(_debug) Task 66548fce-32ae-4257-aa45-b6f8f8ed2c4a: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:24:53,978::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:24:53,979::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:24:53,979::task::495::TaskManager.Task::(_debug) Task 69601fa0-d0d3-4ad2-8e48-faa6667d8e46: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:24:53,981::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d0332bfc-8599-4611-9ee3-a368b9affe07`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:24:53,982::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:24:53,982::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:24:53,983::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d0332bfc-8599-4611-9ee3-a368b9affe07`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:24:53,984::task::495::TaskManager.Task::(_debug) Task 69601fa0-d0d3-4ad2-8e48-faa6667d8e46: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:24:53,985::task::495::TaskManager.Task::(_debug) Task 69601fa0-d0d3-4ad2-8e48-faa6667d8e46: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:24:53,986::task::495::TaskManager.Task::(_debug) Task 69601fa0-d0d3-4ad2-8e48-faa6667d8e46: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:24:53,986::task::495::TaskManager.Task::(_debug) Task 69601fa0-d0d3-4ad2-8e48-faa6667d8e46: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:24:53,987::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:24:53,988::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:24:53,989::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:24:53,990::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:24:53,990::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:24:53,991::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:24:53,992::task::495::TaskManager.Task::(_debug) Task 69601fa0-d0d3-4ad2-8e48-faa6667d8e46: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:24:53,993::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:24:54,292::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.08111 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 04:24:55,136::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:24:55,144::task::495::TaskManager.Task::(_debug) Task 692ede5d-d263-4881-957e-f59d1c855708: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:24:55,146::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4cb05140-9b57-4a86-b6c6-2a22052b9f79`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:24:55,147::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:24:55,148::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:24:55,149::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4cb05140-9b57-4a86-b6c6-2a22052b9f79`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:24:55,150::task::495::TaskManager.Task::(_debug) Task 692ede5d-d263-4881-957e-f59d1c855708: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:24:55,151::task::495::TaskManager.Task::(_debug) Task 692ede5d-d263-4881-957e-f59d1c855708: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:24:55,151::task::495::TaskManager.Task::(_debug) Task 692ede5d-d263-4881-957e-f59d1c855708: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:24:55,152::task::495::TaskManager.Task::(_debug) Task 692ede5d-d263-4881-957e-f59d1c855708: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:24:55,152::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:24:55,153::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:24:55,154::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:24:55,154::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:24:55,155::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:24:55,155::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:24:55,156::task::495::TaskManager.Task::(_debug) Task 692ede5d-d263-4881-957e-f59d1c855708: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:24:55,156::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:24:56,309::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:24:56,865::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0847636 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15916352::INFO::2013-11-20 04:24:57,051::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916352::DEBUG::2013-11-20 04:24:57,052::task::495::TaskManager.Task::(_debug) Task 17de15f1-e86c-40c0-ae6b-7cb58f6a8567: moving from state init -> state preparing
>Thread-15916352::DEBUG::2013-11-20 04:24:57,053::task::495::TaskManager.Task::(_debug) Task 17de15f1-e86c-40c0-ae6b-7cb58f6a8567: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000730037689209', 'lastCheck': 1384921495.367166, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00186586380005', 'lastCheck': 1384921493.9424739, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00144004821777', 'lastCheck': 1384921490.785208, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000525951385498', 'lastCheck': 1384921489.7869611, 'valid': True, 'code': 0}}
>Thread-15916352::DEBUG::2013-11-20 04:24:57,054::task::495::TaskManager.Task::(_debug) Task 17de15f1-e86c-40c0-ae6b-7cb58f6a8567: moving from state preparing -> state finished
>Thread-15916352::DEBUG::2013-11-20 04:24:57,055::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916352::DEBUG::2013-11-20 04:24:57,056::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916352::DEBUG::2013-11-20 04:24:57,058::task::495::TaskManager.Task::(_debug) Task 17de15f1-e86c-40c0-ae6b-7cb58f6a8567: ref 0 aborting False
>Thread-15916352::INFO::2013-11-20 04:24:57,059::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000730037689209', 'lastCheck': 1384921495.367166, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00186586380005', 'lastCheck': 1384921493.9424739, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00144004821777', 'lastCheck': 1384921490.785208, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000525951385498', 'lastCheck': 1384921489.7869611, 'valid': True, 'code': 0}}
>Thread-15916354::DEBUG::2013-11-20 04:24:58,225::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916354::INFO::2013-11-20 04:24:58,226::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916354::DEBUG::2013-11-20 04:24:58,227::task::495::TaskManager.Task::(_debug) Task c30b998a-415e-4b6f-aaf1-d5b391f54b2c: moving from state init -> state preparing
>Thread-15916354::DEBUG::2013-11-20 04:24:58,228::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916354::DEBUG::2013-11-20 04:24:58,228::task::495::TaskManager.Task::(_debug) Task c30b998a-415e-4b6f-aaf1-d5b391f54b2c: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916354::DEBUG::2013-11-20 04:24:58,229::task::495::TaskManager.Task::(_debug) Task c30b998a-415e-4b6f-aaf1-d5b391f54b2c: moving from state preparing -> state finished
>Thread-15916354::DEBUG::2013-11-20 04:24:58,230::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916354::DEBUG::2013-11-20 04:24:58,230::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916354::DEBUG::2013-11-20 04:24:58,231::task::495::TaskManager.Task::(_debug) Task c30b998a-415e-4b6f-aaf1-d5b391f54b2c: ref 0 aborting False
>Thread-15916354::INFO::2013-11-20 04:24:58,232::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916355::DEBUG::2013-11-20 04:24:58,237::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916355::INFO::2013-11-20 04:24:58,238::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916355::DEBUG::2013-11-20 04:24:58,239::task::495::TaskManager.Task::(_debug) Task 49155176-1c5e-4568-a72f-dc7a4f5f48e1: moving from state init -> state preparing
>Thread-15916355::DEBUG::2013-11-20 04:24:58,240::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3cbb990f-e003-4ce5-9daf-c5c552925a43`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916355::DEBUG::2013-11-20 04:24:58,241::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916355::DEBUG::2013-11-20 04:24:58,241::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916355::DEBUG::2013-11-20 04:24:58,242::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3cbb990f-e003-4ce5-9daf-c5c552925a43`::Granted request
>Thread-15916355::DEBUG::2013-11-20 04:24:58,243::task::495::TaskManager.Task::(_debug) Task 49155176-1c5e-4568-a72f-dc7a4f5f48e1: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916355::DEBUG::2013-11-20 04:24:58,244::task::495::TaskManager.Task::(_debug) Task 49155176-1c5e-4568-a72f-dc7a4f5f48e1: ref 1 aborting False
>Thread-15916355::WARNING::2013-11-20 04:24:58,246::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916355::DEBUG::2013-11-20 04:24:58,246::task::495::TaskManager.Task::(_debug) Task 49155176-1c5e-4568-a72f-dc7a4f5f48e1: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916355::DEBUG::2013-11-20 04:24:58,247::task::495::TaskManager.Task::(_debug) Task 49155176-1c5e-4568-a72f-dc7a4f5f48e1: moving from state preparing -> state finished
>Thread-15916355::DEBUG::2013-11-20 04:24:58,248::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916355::DEBUG::2013-11-20 04:24:58,249::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916355::DEBUG::2013-11-20 04:24:58,249::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916355::DEBUG::2013-11-20 04:24:58,250::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916355::DEBUG::2013-11-20 04:24:58,251::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916355::DEBUG::2013-11-20 04:24:58,251::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916355::DEBUG::2013-11-20 04:24:58,252::task::495::TaskManager.Task::(_debug) Task 49155176-1c5e-4568-a72f-dc7a4f5f48e1: ref 0 aborting False
>Thread-15916355::INFO::2013-11-20 04:24:58,253::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:24:58,881::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:24:59,459::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0806651 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:25:01,469::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:25:01,954::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0548158 s, 18.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:25:03,963::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:25:04,497::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0749662 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 04:25:05,164::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:25:05,180::task::495::TaskManager.Task::(_debug) Task 6a20f47e-fc45-4f67-8b5f-af80a6a4bd4e: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:25:05,182::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e14a994f-8bc5-4322-a61a-638ed02a0b87`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:25:05,184::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:25:05,185::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:25:05,186::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e14a994f-8bc5-4322-a61a-638ed02a0b87`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:25:05,188::task::495::TaskManager.Task::(_debug) Task 6a20f47e-fc45-4f67-8b5f-af80a6a4bd4e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:25:05,188::task::495::TaskManager.Task::(_debug) Task 6a20f47e-fc45-4f67-8b5f-af80a6a4bd4e: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:25:05,189::task::495::TaskManager.Task::(_debug) Task 6a20f47e-fc45-4f67-8b5f-af80a6a4bd4e: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:25:05,190::task::495::TaskManager.Task::(_debug) Task 6a20f47e-fc45-4f67-8b5f-af80a6a4bd4e: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:25:05,190::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:25:05,191::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:25:05,192::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:25:05,193::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:25:05,193::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:25:05,194::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:25:05,194::task::495::TaskManager.Task::(_debug) Task 6a20f47e-fc45-4f67-8b5f-af80a6a4bd4e: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:25:05,195::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:25:06,512::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:25:07,013::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0550177 s, 18.6 MB/s\n'; <rc> = 0
>Thread-15916359::DEBUG::2013-11-20 04:25:08,634::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916359::INFO::2013-11-20 04:25:08,635::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916359::DEBUG::2013-11-20 04:25:08,636::task::495::TaskManager.Task::(_debug) Task 9f784bb7-b034-42ea-a5dd-4e943554f494: moving from state init -> state preparing
>Thread-15916359::DEBUG::2013-11-20 04:25:08,637::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916359::DEBUG::2013-11-20 04:25:08,638::task::495::TaskManager.Task::(_debug) Task 9f784bb7-b034-42ea-a5dd-4e943554f494: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916359::DEBUG::2013-11-20 04:25:08,638::task::495::TaskManager.Task::(_debug) Task 9f784bb7-b034-42ea-a5dd-4e943554f494: moving from state preparing -> state finished
>Thread-15916359::DEBUG::2013-11-20 04:25:08,639::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916359::DEBUG::2013-11-20 04:25:08,640::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916359::DEBUG::2013-11-20 04:25:08,640::task::495::TaskManager.Task::(_debug) Task 9f784bb7-b034-42ea-a5dd-4e943554f494: ref 0 aborting False
>Thread-15916359::INFO::2013-11-20 04:25:08,641::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916360::DEBUG::2013-11-20 04:25:08,646::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916360::INFO::2013-11-20 04:25:08,656::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916360::DEBUG::2013-11-20 04:25:08,657::task::495::TaskManager.Task::(_debug) Task 7a81680e-1f2c-47e7-9472-f716fcea41d6: moving from state init -> state preparing
>Thread-15916360::DEBUG::2013-11-20 04:25:08,658::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ee06e1e1-6be9-41f2-9859-74636caa5a4e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916360::DEBUG::2013-11-20 04:25:08,659::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916360::DEBUG::2013-11-20 04:25:08,660::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916360::DEBUG::2013-11-20 04:25:08,660::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ee06e1e1-6be9-41f2-9859-74636caa5a4e`::Granted request
>Thread-15916360::DEBUG::2013-11-20 04:25:08,662::task::495::TaskManager.Task::(_debug) Task 7a81680e-1f2c-47e7-9472-f716fcea41d6: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916360::DEBUG::2013-11-20 04:25:08,662::task::495::TaskManager.Task::(_debug) Task 7a81680e-1f2c-47e7-9472-f716fcea41d6: ref 1 aborting False
>Thread-15916360::WARNING::2013-11-20 04:25:08,665::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916360::DEBUG::2013-11-20 04:25:08,666::task::495::TaskManager.Task::(_debug) Task 7a81680e-1f2c-47e7-9472-f716fcea41d6: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916360::DEBUG::2013-11-20 04:25:08,667::task::495::TaskManager.Task::(_debug) Task 7a81680e-1f2c-47e7-9472-f716fcea41d6: moving from state preparing -> state finished
>Thread-15916360::DEBUG::2013-11-20 04:25:08,667::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916360::DEBUG::2013-11-20 04:25:08,668::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916360::DEBUG::2013-11-20 04:25:08,669::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916360::DEBUG::2013-11-20 04:25:08,670::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916360::DEBUG::2013-11-20 04:25:08,670::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916360::DEBUG::2013-11-20 04:25:08,671::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916360::DEBUG::2013-11-20 04:25:08,671::task::495::TaskManager.Task::(_debug) Task 7a81680e-1f2c-47e7-9472-f716fcea41d6: ref 0 aborting False
>Thread-15916360::INFO::2013-11-20 04:25:08,672::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:25:09,023::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:25:09,580::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0909221 s, 11.3 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 04:25:10,748::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:25:10,749::task::495::TaskManager.Task::(_debug) Task f1a1a909-dc59-4a88-be1e-59fd897c309f: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:25:10,750::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`259de2c1-1291-4792-8d6d-61388a2bfeea`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:25:10,751::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:25:10,752::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:25:10,752::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`259de2c1-1291-4792-8d6d-61388a2bfeea`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:25:10,753::task::495::TaskManager.Task::(_debug) Task f1a1a909-dc59-4a88-be1e-59fd897c309f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:25:10,754::task::495::TaskManager.Task::(_debug) Task f1a1a909-dc59-4a88-be1e-59fd897c309f: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:25:10,755::task::495::TaskManager.Task::(_debug) Task f1a1a909-dc59-4a88-be1e-59fd897c309f: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:25:10,755::task::495::TaskManager.Task::(_debug) Task f1a1a909-dc59-4a88-be1e-59fd897c309f: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:25:10,756::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:25:10,756::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:25:10,757::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:25:10,758::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:25:10,758::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:25:10,759::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:25:10,759::task::495::TaskManager.Task::(_debug) Task f1a1a909-dc59-4a88-be1e-59fd897c309f: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:25:10,760::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:25:11,596::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:25:12,135::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0896452 s, 11.4 MB/s\n'; <rc> = 0
>Thread-15916362::INFO::2013-11-20 04:25:12,769::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916362::DEBUG::2013-11-20 04:25:12,770::task::495::TaskManager.Task::(_debug) Task 7df65786-72ee-43a2-99aa-6ebb1040662f: moving from state init -> state preparing
>Thread-15916362::DEBUG::2013-11-20 04:25:12,771::task::495::TaskManager.Task::(_debug) Task 7df65786-72ee-43a2-99aa-6ebb1040662f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000341892242432', 'lastCheck': 1384921505.681844, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0015709400177', 'lastCheck': 1384921504.1802671, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00141382217407', 'lastCheck': 1384921511.242204, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000498056411743', 'lastCheck': 1384921510.235795, 'valid': True, 'code': 0}}
>Thread-15916362::DEBUG::2013-11-20 04:25:12,772::task::495::TaskManager.Task::(_debug) Task 7df65786-72ee-43a2-99aa-6ebb1040662f: moving from state preparing -> state finished
>Thread-15916362::DEBUG::2013-11-20 04:25:12,773::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916362::DEBUG::2013-11-20 04:25:12,773::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916362::DEBUG::2013-11-20 04:25:12,774::task::495::TaskManager.Task::(_debug) Task 7df65786-72ee-43a2-99aa-6ebb1040662f: ref 0 aborting False
>Thread-15916362::INFO::2013-11-20 04:25:12,775::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000341892242432', 'lastCheck': 1384921505.681844, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0015709400177', 'lastCheck': 1384921504.1802671, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00141382217407', 'lastCheck': 1384921511.242204, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000498056411743', 'lastCheck': 1384921510.235795, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:25:14,152::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:25:14,659::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0934836 s, 11.0 MB/s\n'; <rc> = 0
>Thread-15914453::INFO::2013-11-20 04:25:15,260::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:25:15,270::task::495::TaskManager.Task::(_debug) Task 462be483-bbb0-43b2-9e31-021dddcdcc2e: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:25:15,282::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`cd7b07d8-37d3-4595-868c-0e920b3e11bd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:25:15,288::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:25:15,293::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:25:15,294::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`cd7b07d8-37d3-4595-868c-0e920b3e11bd`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:25:15,297::task::495::TaskManager.Task::(_debug) Task 462be483-bbb0-43b2-9e31-021dddcdcc2e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:25:15,298::task::495::TaskManager.Task::(_debug) Task 462be483-bbb0-43b2-9e31-021dddcdcc2e: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:25:15,299::task::495::TaskManager.Task::(_debug) Task 462be483-bbb0-43b2-9e31-021dddcdcc2e: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:25:15,299::task::495::TaskManager.Task::(_debug) Task 462be483-bbb0-43b2-9e31-021dddcdcc2e: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:25:15,300::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:25:15,300::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:25:15,301::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:25:15,302::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:25:15,302::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:25:15,303::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:25:15,304::task::495::TaskManager.Task::(_debug) Task 462be483-bbb0-43b2-9e31-021dddcdcc2e: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:25:15,304::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:25:16,674::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:25:17,214::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0714909 s, 14.3 MB/s\n'; <rc> = 0
>Thread-15916365::DEBUG::2013-11-20 04:25:19,021::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916365::INFO::2013-11-20 04:25:19,022::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916365::DEBUG::2013-11-20 04:25:19,023::task::495::TaskManager.Task::(_debug) Task adc0c246-4b9f-4b15-a9f5-287892abadc7: moving from state init -> state preparing
>Thread-15916365::DEBUG::2013-11-20 04:25:19,024::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916365::DEBUG::2013-11-20 04:25:19,025::task::495::TaskManager.Task::(_debug) Task adc0c246-4b9f-4b15-a9f5-287892abadc7: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916365::DEBUG::2013-11-20 04:25:19,025::task::495::TaskManager.Task::(_debug) Task adc0c246-4b9f-4b15-a9f5-287892abadc7: moving from state preparing -> state finished
>Thread-15916365::DEBUG::2013-11-20 04:25:19,026::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916365::DEBUG::2013-11-20 04:25:19,026::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916365::DEBUG::2013-11-20 04:25:19,027::task::495::TaskManager.Task::(_debug) Task adc0c246-4b9f-4b15-a9f5-287892abadc7: ref 0 aborting False
>Thread-15916365::INFO::2013-11-20 04:25:19,028::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916366::DEBUG::2013-11-20 04:25:19,043::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916366::INFO::2013-11-20 04:25:19,045::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916366::DEBUG::2013-11-20 04:25:19,046::task::495::TaskManager.Task::(_debug) Task 0ab55bc5-5758-431c-9593-5474aea6e698: moving from state init -> state preparing
>Thread-15916366::DEBUG::2013-11-20 04:25:19,047::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7a5b232b-3fc1-4803-a976-37f87e960a98`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916366::DEBUG::2013-11-20 04:25:19,048::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916366::DEBUG::2013-11-20 04:25:19,049::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916366::DEBUG::2013-11-20 04:25:19,050::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7a5b232b-3fc1-4803-a976-37f87e960a98`::Granted request
>Thread-15916366::DEBUG::2013-11-20 04:25:19,051::task::495::TaskManager.Task::(_debug) Task 0ab55bc5-5758-431c-9593-5474aea6e698: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916366::DEBUG::2013-11-20 04:25:19,052::task::495::TaskManager.Task::(_debug) Task 0ab55bc5-5758-431c-9593-5474aea6e698: ref 1 aborting False
>Thread-15916366::WARNING::2013-11-20 04:25:19,055::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916366::DEBUG::2013-11-20 04:25:19,056::task::495::TaskManager.Task::(_debug) Task 0ab55bc5-5758-431c-9593-5474aea6e698: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916366::DEBUG::2013-11-20 04:25:19,057::task::495::TaskManager.Task::(_debug) Task 0ab55bc5-5758-431c-9593-5474aea6e698: moving from state preparing -> state finished
>Thread-15916366::DEBUG::2013-11-20 04:25:19,058::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916366::DEBUG::2013-11-20 04:25:19,059::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916366::DEBUG::2013-11-20 04:25:19,060::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916366::DEBUG::2013-11-20 04:25:19,060::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916366::DEBUG::2013-11-20 04:25:19,061::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916366::DEBUG::2013-11-20 04:25:19,062::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916366::DEBUG::2013-11-20 04:25:19,063::task::495::TaskManager.Task::(_debug) Task 0ab55bc5-5758-431c-9593-5474aea6e698: ref 0 aborting False
>Thread-15916366::INFO::2013-11-20 04:25:19,064::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:25:19,229::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:25:19,798::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0823559 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:25:21,813::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 04:25:22,319::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:25:22,321::task::495::TaskManager.Task::(_debug) Task feed18c4-b069-40c9-a634-6fe1051194da: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:25:22,338::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4a05d38f-384b-4e58-83cd-7223dc583ec0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:25:22,340::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:25:22,343::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:25:22,344::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4a05d38f-384b-4e58-83cd-7223dc583ec0`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:25:22,352::task::495::TaskManager.Task::(_debug) Task feed18c4-b069-40c9-a634-6fe1051194da: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 04:25:22,354::task::495::TaskManager.Task::(_debug) Task feed18c4-b069-40c9-a634-6fe1051194da: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:25:22,355::task::495::TaskManager.Task::(_debug) Task feed18c4-b069-40c9-a634-6fe1051194da: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 04:25:22,356::task::495::TaskManager.Task::(_debug) Task feed18c4-b069-40c9-a634-6fe1051194da: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:25:22,357::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:25:22,358::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:25:22,363::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 04:25:22,365::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:25:22,366::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:25:22,367::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Dummy-15912651::DEBUG::2013-11-20 04:25:22,371::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0735937 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15914519::DEBUG::2013-11-20 04:25:22,372::task::495::TaskManager.Task::(_debug) Task feed18c4-b069-40c9-a634-6fe1051194da: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 04:25:22,387::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 04:25:24,392::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:25:24,957::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0973797 s, 10.5 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 04:25:25,469::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:25:25,473::task::495::TaskManager.Task::(_debug) Task 464a6bf0-04b2-4bef-9e10-903e71031933: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:25:25,475::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8d26c1fb-c6e4-440a-b9ca-bc08f85995d4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:25:25,476::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:25:25,477::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:25:25,478::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8d26c1fb-c6e4-440a-b9ca-bc08f85995d4`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:25:25,480::task::495::TaskManager.Task::(_debug) Task 464a6bf0-04b2-4bef-9e10-903e71031933: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:25:25,481::task::495::TaskManager.Task::(_debug) Task 464a6bf0-04b2-4bef-9e10-903e71031933: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:25:25,482::task::495::TaskManager.Task::(_debug) Task 464a6bf0-04b2-4bef-9e10-903e71031933: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 04:25:25,483::task::495::TaskManager.Task::(_debug) Task 464a6bf0-04b2-4bef-9e10-903e71031933: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:25:25,484::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:25:25,485::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:25:25,486::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:25:25,486::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:25:25,487::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:25:25,488::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:25:25,489::task::495::TaskManager.Task::(_debug) Task 464a6bf0-04b2-4bef-9e10-903e71031933: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:25:25,489::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:25:25,490::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:25:25,490::task::495::TaskManager.Task::(_debug) Task 97c27d2b-bdeb-4805-be1d-57b44a0db190: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:25:25,492::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`586c43ea-3b37-4a74-8c1b-ae5aa66fb2c9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:25:25,492::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:25:25,494::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:25:25,494::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`586c43ea-3b37-4a74-8c1b-ae5aa66fb2c9`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:25:25,496::task::495::TaskManager.Task::(_debug) Task 97c27d2b-bdeb-4805-be1d-57b44a0db190: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:25:25,504::task::495::TaskManager.Task::(_debug) Task 97c27d2b-bdeb-4805-be1d-57b44a0db190: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:25:25,506::task::495::TaskManager.Task::(_debug) Task 97c27d2b-bdeb-4805-be1d-57b44a0db190: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:25:25,507::task::495::TaskManager.Task::(_debug) Task 97c27d2b-bdeb-4805-be1d-57b44a0db190: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:25:25,508::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:25:25,508::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:25:25,509::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:25:25,510::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:25:25,510::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:25:25,511::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:25:25,512::task::495::TaskManager.Task::(_debug) Task 97c27d2b-bdeb-4805-be1d-57b44a0db190: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:25:25,512::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:25:25,513::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:25:25,514::task::495::TaskManager.Task::(_debug) Task c5029d28-9bdc-442e-a565-0025f79aca9f: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:25:25,515::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8d3aabcf-51d3-4612-a8c9-7af1967df72f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:25:25,516::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:25:25,516::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:25:25,517::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8d3aabcf-51d3-4612-a8c9-7af1967df72f`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:25:25,518::task::495::TaskManager.Task::(_debug) Task c5029d28-9bdc-442e-a565-0025f79aca9f: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:25:25,519::task::495::TaskManager.Task::(_debug) Task c5029d28-9bdc-442e-a565-0025f79aca9f: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:25:25,520::task::495::TaskManager.Task::(_debug) Task c5029d28-9bdc-442e-a565-0025f79aca9f: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:25:25,520::task::495::TaskManager.Task::(_debug) Task c5029d28-9bdc-442e-a565-0025f79aca9f: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:25:25,521::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:25:25,522::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:25:25,523::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:25:25,523::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:25:25,524::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:25:25,525::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:25:25,526::task::495::TaskManager.Task::(_debug) Task c5029d28-9bdc-442e-a565-0025f79aca9f: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:25:25,526::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:25:25,527::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:25:25,528::task::495::TaskManager.Task::(_debug) Task 55e3896d-7499-494f-8f12-4d590bf5dcd3: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:25:25,529::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`917d9a98-9d3d-4457-8171-5e0e2f94ff3b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:25:25,530::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:25:25,531::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:25:25,532::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`917d9a98-9d3d-4457-8171-5e0e2f94ff3b`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:25:25,534::task::495::TaskManager.Task::(_debug) Task 55e3896d-7499-494f-8f12-4d590bf5dcd3: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:25:25,535::task::495::TaskManager.Task::(_debug) Task 55e3896d-7499-494f-8f12-4d590bf5dcd3: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:25:25,536::task::495::TaskManager.Task::(_debug) Task 55e3896d-7499-494f-8f12-4d590bf5dcd3: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:25:25,537::task::495::TaskManager.Task::(_debug) Task 55e3896d-7499-494f-8f12-4d590bf5dcd3: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:25:25,537::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:25:25,538::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:25:25,540::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:25:25,541::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:25:25,542::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:25:25,543::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:25:25,544::task::495::TaskManager.Task::(_debug) Task 55e3896d-7499-494f-8f12-4d590bf5dcd3: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:25:25,544::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15914435::INFO::2013-11-20 04:25:25,555::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:25:25,556::task::495::TaskManager.Task::(_debug) Task 230e77f4-d283-4a32-ad54-48daa3c4f20a: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:25:25,558::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`03d9d5fe-9cce-48d5-b953-1b803513614d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:25:25,558::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:25:25,559::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:25:25,560::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`03d9d5fe-9cce-48d5-b953-1b803513614d`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:25:25,562::task::495::TaskManager.Task::(_debug) Task 230e77f4-d283-4a32-ad54-48daa3c4f20a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:25:25,563::task::495::TaskManager.Task::(_debug) Task 230e77f4-d283-4a32-ad54-48daa3c4f20a: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:25:25,564::task::495::TaskManager.Task::(_debug) Task 230e77f4-d283-4a32-ad54-48daa3c4f20a: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:25:25,565::task::495::TaskManager.Task::(_debug) Task 230e77f4-d283-4a32-ad54-48daa3c4f20a: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:25:25,565::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:25:25,566::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:25:25,567::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:25:25,568::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:25:25,569::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:25:25,569::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:25:25,570::task::495::TaskManager.Task::(_debug) Task 230e77f4-d283-4a32-ad54-48daa3c4f20a: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:25:25,571::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:25:26,974::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:25:27,542::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0940726 s, 10.9 MB/s\n'; <rc> = 0
>Thread-15916370::INFO::2013-11-20 04:25:28,525::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916370::DEBUG::2013-11-20 04:25:28,526::task::495::TaskManager.Task::(_debug) Task 9227e877-0fda-4ad2-92d5-13562c649599: moving from state init -> state preparing
>Thread-15916370::DEBUG::2013-11-20 04:25:28,527::task::495::TaskManager.Task::(_debug) Task 9227e877-0fda-4ad2-92d5-13562c649599: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000342845916748', 'lastCheck': 1384921526.007669, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00914883613586', 'lastCheck': 1384921524.606138, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00138998031616', 'lastCheck': 1384921521.4515431, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000632047653198', 'lastCheck': 1384921520.4523561, 'valid': True, 'code': 0}}
>Thread-15916370::DEBUG::2013-11-20 04:25:28,528::task::495::TaskManager.Task::(_debug) Task 9227e877-0fda-4ad2-92d5-13562c649599: moving from state preparing -> state finished
>Thread-15916370::DEBUG::2013-11-20 04:25:28,528::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916370::DEBUG::2013-11-20 04:25:28,529::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916370::DEBUG::2013-11-20 04:25:28,530::task::495::TaskManager.Task::(_debug) Task 9227e877-0fda-4ad2-92d5-13562c649599: ref 0 aborting False
>Thread-15916370::INFO::2013-11-20 04:25:28,531::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000342845916748', 'lastCheck': 1384921526.007669, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00914883613586', 'lastCheck': 1384921524.606138, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00138998031616', 'lastCheck': 1384921521.4515431, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000632047653198', 'lastCheck': 1384921520.4523561, 'valid': True, 'code': 0}}
>Thread-15916372::DEBUG::2013-11-20 04:25:29,303::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916372::INFO::2013-11-20 04:25:29,305::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916372::DEBUG::2013-11-20 04:25:29,306::task::495::TaskManager.Task::(_debug) Task 4dad2e16-66ce-4dba-b8ed-344006769334: moving from state init -> state preparing
>Thread-15916372::DEBUG::2013-11-20 04:25:29,307::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916372::DEBUG::2013-11-20 04:25:29,308::task::495::TaskManager.Task::(_debug) Task 4dad2e16-66ce-4dba-b8ed-344006769334: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916372::DEBUG::2013-11-20 04:25:29,309::task::495::TaskManager.Task::(_debug) Task 4dad2e16-66ce-4dba-b8ed-344006769334: moving from state preparing -> state finished
>Thread-15916372::DEBUG::2013-11-20 04:25:29,309::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916372::DEBUG::2013-11-20 04:25:29,310::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916372::DEBUG::2013-11-20 04:25:29,311::task::495::TaskManager.Task::(_debug) Task 4dad2e16-66ce-4dba-b8ed-344006769334: ref 0 aborting False
>Thread-15916372::INFO::2013-11-20 04:25:29,311::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916373::DEBUG::2013-11-20 04:25:29,316::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916373::INFO::2013-11-20 04:25:29,317::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916373::DEBUG::2013-11-20 04:25:29,318::task::495::TaskManager.Task::(_debug) Task f1fbfd68-b4dd-4a7a-8012-8d9723cd2220: moving from state init -> state preparing
>Thread-15916373::DEBUG::2013-11-20 04:25:29,319::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2fc2bada-c04f-41ae-98f0-4f6196360308`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916373::DEBUG::2013-11-20 04:25:29,320::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916373::DEBUG::2013-11-20 04:25:29,320::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916373::DEBUG::2013-11-20 04:25:29,321::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2fc2bada-c04f-41ae-98f0-4f6196360308`::Granted request
>Thread-15916373::DEBUG::2013-11-20 04:25:29,322::task::495::TaskManager.Task::(_debug) Task f1fbfd68-b4dd-4a7a-8012-8d9723cd2220: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916373::DEBUG::2013-11-20 04:25:29,323::task::495::TaskManager.Task::(_debug) Task f1fbfd68-b4dd-4a7a-8012-8d9723cd2220: ref 1 aborting False
>Thread-15916373::WARNING::2013-11-20 04:25:29,325::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916373::DEBUG::2013-11-20 04:25:29,325::task::495::TaskManager.Task::(_debug) Task f1fbfd68-b4dd-4a7a-8012-8d9723cd2220: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916373::DEBUG::2013-11-20 04:25:29,326::task::495::TaskManager.Task::(_debug) Task f1fbfd68-b4dd-4a7a-8012-8d9723cd2220: moving from state preparing -> state finished
>Thread-15916373::DEBUG::2013-11-20 04:25:29,327::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916373::DEBUG::2013-11-20 04:25:29,328::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916373::DEBUG::2013-11-20 04:25:29,329::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916373::DEBUG::2013-11-20 04:25:29,329::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916373::DEBUG::2013-11-20 04:25:29,330::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916373::DEBUG::2013-11-20 04:25:29,330::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916373::DEBUG::2013-11-20 04:25:29,331::task::495::TaskManager.Task::(_debug) Task f1fbfd68-b4dd-4a7a-8012-8d9723cd2220: ref 0 aborting False
>Thread-15916373::INFO::2013-11-20 04:25:29,332::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:25:29,561::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:25:30,097::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0794242 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:25:32,113::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:25:32,678::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0886682 s, 11.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:25:34,694::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:25:35,218::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0806519 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:25:37,235::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:25:37,826::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0819432 s, 12.5 MB/s\n'; <rc> = 0
>Thread-15916377::DEBUG::2013-11-20 04:25:39,651::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916377::INFO::2013-11-20 04:25:39,653::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916377::DEBUG::2013-11-20 04:25:39,654::task::495::TaskManager.Task::(_debug) Task 7117f1fb-ea63-41e3-8b15-b83ea5b5e9f4: moving from state init -> state preparing
>Thread-15916377::DEBUG::2013-11-20 04:25:39,655::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916377::DEBUG::2013-11-20 04:25:39,655::task::495::TaskManager.Task::(_debug) Task 7117f1fb-ea63-41e3-8b15-b83ea5b5e9f4: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916377::DEBUG::2013-11-20 04:25:39,656::task::495::TaskManager.Task::(_debug) Task 7117f1fb-ea63-41e3-8b15-b83ea5b5e9f4: moving from state preparing -> state finished
>Thread-15916377::DEBUG::2013-11-20 04:25:39,657::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916377::DEBUG::2013-11-20 04:25:39,658::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916377::DEBUG::2013-11-20 04:25:39,658::task::495::TaskManager.Task::(_debug) Task 7117f1fb-ea63-41e3-8b15-b83ea5b5e9f4: ref 0 aborting False
>Thread-15916377::INFO::2013-11-20 04:25:39,659::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916378::DEBUG::2013-11-20 04:25:39,664::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916378::INFO::2013-11-20 04:25:39,665::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916378::DEBUG::2013-11-20 04:25:39,665::task::495::TaskManager.Task::(_debug) Task 7f63282a-2e0e-41c4-a0e8-50e590558061: moving from state init -> state preparing
>Thread-15916378::DEBUG::2013-11-20 04:25:39,667::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d273c6eb-9f4d-4ff5-b5d8-859f8fd2431b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916378::DEBUG::2013-11-20 04:25:39,667::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916378::DEBUG::2013-11-20 04:25:39,668::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916378::DEBUG::2013-11-20 04:25:39,669::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d273c6eb-9f4d-4ff5-b5d8-859f8fd2431b`::Granted request
>Thread-15916378::DEBUG::2013-11-20 04:25:39,670::task::495::TaskManager.Task::(_debug) Task 7f63282a-2e0e-41c4-a0e8-50e590558061: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916378::DEBUG::2013-11-20 04:25:39,671::task::495::TaskManager.Task::(_debug) Task 7f63282a-2e0e-41c4-a0e8-50e590558061: ref 1 aborting False
>Thread-15916378::WARNING::2013-11-20 04:25:39,673::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916378::DEBUG::2013-11-20 04:25:39,675::task::495::TaskManager.Task::(_debug) Task 7f63282a-2e0e-41c4-a0e8-50e590558061: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916378::DEBUG::2013-11-20 04:25:39,676::task::495::TaskManager.Task::(_debug) Task 7f63282a-2e0e-41c4-a0e8-50e590558061: moving from state preparing -> state finished
>Thread-15916378::DEBUG::2013-11-20 04:25:39,677::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916378::DEBUG::2013-11-20 04:25:39,677::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916378::DEBUG::2013-11-20 04:25:39,678::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916378::DEBUG::2013-11-20 04:25:39,679::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916378::DEBUG::2013-11-20 04:25:39,679::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916378::DEBUG::2013-11-20 04:25:39,680::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916378::DEBUG::2013-11-20 04:25:39,681::task::495::TaskManager.Task::(_debug) Task 7f63282a-2e0e-41c4-a0e8-50e590558061: ref 0 aborting False
>Thread-15916378::INFO::2013-11-20 04:25:39,681::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:25:39,844::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:25:40,413::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0786898 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:25:42,429::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915918::INFO::2013-11-20 04:25:42,621::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15913639::INFO::2013-11-20 04:25:42,624::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15915918::DEBUG::2013-11-20 04:25:42,625::task::495::TaskManager.Task::(_debug) Task b90eb401-ff06-4923-b5ed-0d15599da77b: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:25:42,627::task::495::TaskManager.Task::(_debug) Task e1085262-7fad-45dd-8e59-04e28a012c58: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:25:42,631::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1350a43f-c748-4d05-bd41-9f7ba75b475d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:25:42,632::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a4fc46c1-e09b-4937-9979-d29d085071fa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:25:42,633::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:25:42,634::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:25:42,635::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:25:42,636::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1350a43f-c748-4d05-bd41-9f7ba75b475d`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:25:42,637::resourceManager::481::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' found in shared state and queue is empty, Joining current shared lock (2 active users)
>Thread-15915918::DEBUG::2013-11-20 04:25:42,638::task::495::TaskManager.Task::(_debug) Task b90eb401-ff06-4923-b5ed-0d15599da77b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:25:42,639::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a4fc46c1-e09b-4937-9979-d29d085071fa`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:25:42,639::task::495::TaskManager.Task::(_debug) Task b90eb401-ff06-4923-b5ed-0d15599da77b: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:25:42,641::task::495::TaskManager.Task::(_debug) Task e1085262-7fad-45dd-8e59-04e28a012c58: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:25:42,641::task::495::TaskManager.Task::(_debug) Task b90eb401-ff06-4923-b5ed-0d15599da77b: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15913639::DEBUG::2013-11-20 04:25:42,642::task::495::TaskManager.Task::(_debug) Task e1085262-7fad-45dd-8e59-04e28a012c58: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:25:42,642::task::495::TaskManager.Task::(_debug) Task b90eb401-ff06-4923-b5ed-0d15599da77b: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:25:42,643::task::495::TaskManager.Task::(_debug) Task e1085262-7fad-45dd-8e59-04e28a012c58: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15915918::DEBUG::2013-11-20 04:25:42,644::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:25:42,644::task::495::TaskManager.Task::(_debug) Task e1085262-7fad-45dd-8e59-04e28a012c58: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:25:42,645::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:25:42,645::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:25:42,646::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:25:42,647::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:25:42,648::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (1 active users)
>Thread-15913639::DEBUG::2013-11-20 04:25:42,648::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:25:42,649::task::495::TaskManager.Task::(_debug) Task b90eb401-ff06-4923-b5ed-0d15599da77b: ref 0 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:25:42,649::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::INFO::2013-11-20 04:25:42,650::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15913639::DEBUG::2013-11-20 04:25:42,650::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:25:42,651::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:25:42,652::task::495::TaskManager.Task::(_debug) Task e1085262-7fad-45dd-8e59-04e28a012c58: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:25:42,652::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:25:42,961::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0799022 s, 12.8 MB/s\n'; <rc> = 0
>Thread-15916380::INFO::2013-11-20 04:25:44,564::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916380::DEBUG::2013-11-20 04:25:44,565::task::495::TaskManager.Task::(_debug) Task 2c9484a8-ce53-4b0a-806c-266bd07818db: moving from state init -> state preparing
>Thread-15916380::DEBUG::2013-11-20 04:25:44,566::task::495::TaskManager.Task::(_debug) Task 2c9484a8-ce53-4b0a-806c-266bd07818db: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000452041625977', 'lastCheck': 1384921536.3325429, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00117111206055', 'lastCheck': 1384921534.8874969, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00140309333801', 'lastCheck': 1384921542.085505, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000487089157104', 'lastCheck': 1384921541.0821071, 'valid': True, 'code': 0}}
>Thread-15916380::DEBUG::2013-11-20 04:25:44,567::task::495::TaskManager.Task::(_debug) Task 2c9484a8-ce53-4b0a-806c-266bd07818db: moving from state preparing -> state finished
>Thread-15916380::DEBUG::2013-11-20 04:25:44,568::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916380::DEBUG::2013-11-20 04:25:44,568::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916380::DEBUG::2013-11-20 04:25:44,569::task::495::TaskManager.Task::(_debug) Task 2c9484a8-ce53-4b0a-806c-266bd07818db: ref 0 aborting False
>Thread-15916380::INFO::2013-11-20 04:25:44,570::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000452041625977', 'lastCheck': 1384921536.3325429, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00117111206055', 'lastCheck': 1384921534.8874969, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00140309333801', 'lastCheck': 1384921542.085505, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000487089157104', 'lastCheck': 1384921541.0821071, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:25:44,978::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:25:45,508::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0791324 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:25:47,525::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915863::INFO::2013-11-20 04:25:47,745::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:25:47,751::task::495::TaskManager.Task::(_debug) Task e796b2cc-b0f5-4fd8-ba3b-86ae0d59a397: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:25:47,755::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f386befa-a3fb-43b2-b77c-8cf28e4efdf9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:25:47,757::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:25:47,760::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:25:47,762::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f386befa-a3fb-43b2-b77c-8cf28e4efdf9`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:25:47,767::task::495::TaskManager.Task::(_debug) Task e796b2cc-b0f5-4fd8-ba3b-86ae0d59a397: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:25:47,781::task::495::TaskManager.Task::(_debug) Task e796b2cc-b0f5-4fd8-ba3b-86ae0d59a397: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:25:47,786::task::495::TaskManager.Task::(_debug) Task e796b2cc-b0f5-4fd8-ba3b-86ae0d59a397: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:25:47,787::task::495::TaskManager.Task::(_debug) Task e796b2cc-b0f5-4fd8-ba3b-86ae0d59a397: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:25:47,788::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:25:47,789::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:25:47,790::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:25:47,791::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:25:47,792::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:25:47,792::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:25:47,794::task::495::TaskManager.Task::(_debug) Task e796b2cc-b0f5-4fd8-ba3b-86ae0d59a397: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:25:47,795::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:25:48,084::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0787092 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15916383::DEBUG::2013-11-20 04:25:49,997::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916383::INFO::2013-11-20 04:25:49,999::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916383::DEBUG::2013-11-20 04:25:49,999::task::495::TaskManager.Task::(_debug) Task aa45478e-943f-492b-9e4f-b6282e61e2da: moving from state init -> state preparing
>Thread-15916383::DEBUG::2013-11-20 04:25:50,000::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916383::DEBUG::2013-11-20 04:25:50,001::task::495::TaskManager.Task::(_debug) Task aa45478e-943f-492b-9e4f-b6282e61e2da: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916383::DEBUG::2013-11-20 04:25:50,001::task::495::TaskManager.Task::(_debug) Task aa45478e-943f-492b-9e4f-b6282e61e2da: moving from state preparing -> state finished
>Thread-15916383::DEBUG::2013-11-20 04:25:50,002::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916383::DEBUG::2013-11-20 04:25:50,002::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916383::DEBUG::2013-11-20 04:25:50,003::task::495::TaskManager.Task::(_debug) Task aa45478e-943f-492b-9e4f-b6282e61e2da: ref 0 aborting False
>Thread-15916383::INFO::2013-11-20 04:25:50,004::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916384::DEBUG::2013-11-20 04:25:50,017::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916384::INFO::2013-11-20 04:25:50,018::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916384::DEBUG::2013-11-20 04:25:50,018::task::495::TaskManager.Task::(_debug) Task a0ae6405-8473-40c8-a13a-6f3801922be6: moving from state init -> state preparing
>Thread-15916384::DEBUG::2013-11-20 04:25:50,020::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a9869133-051f-471b-8c10-b80fc0a1cfc8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916384::DEBUG::2013-11-20 04:25:50,020::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916384::DEBUG::2013-11-20 04:25:50,021::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916384::DEBUG::2013-11-20 04:25:50,022::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a9869133-051f-471b-8c10-b80fc0a1cfc8`::Granted request
>Thread-15916384::DEBUG::2013-11-20 04:25:50,023::task::495::TaskManager.Task::(_debug) Task a0ae6405-8473-40c8-a13a-6f3801922be6: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916384::DEBUG::2013-11-20 04:25:50,024::task::495::TaskManager.Task::(_debug) Task a0ae6405-8473-40c8-a13a-6f3801922be6: ref 1 aborting False
>Thread-15916384::WARNING::2013-11-20 04:25:50,026::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916384::DEBUG::2013-11-20 04:25:50,027::task::495::TaskManager.Task::(_debug) Task a0ae6405-8473-40c8-a13a-6f3801922be6: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916384::DEBUG::2013-11-20 04:25:50,028::task::495::TaskManager.Task::(_debug) Task a0ae6405-8473-40c8-a13a-6f3801922be6: moving from state preparing -> state finished
>Thread-15916384::DEBUG::2013-11-20 04:25:50,028::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916384::DEBUG::2013-11-20 04:25:50,029::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916384::DEBUG::2013-11-20 04:25:50,030::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916384::DEBUG::2013-11-20 04:25:50,031::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916384::DEBUG::2013-11-20 04:25:50,031::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916384::DEBUG::2013-11-20 04:25:50,032::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916384::DEBUG::2013-11-20 04:25:50,033::task::495::TaskManager.Task::(_debug) Task a0ae6405-8473-40c8-a13a-6f3801922be6: ref 0 aborting False
>Thread-15916384::INFO::2013-11-20 04:25:50,033::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:25:50,098::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:25:50,696::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0935478 s, 10.9 MB/s\n'; <rc> = 0
>Thread-42840::DEBUG::2013-11-20 04:25:51,328::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 04:25:52,069::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:25:52,706::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:25:53,272::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0825156 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:25:55,288::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913518::INFO::2013-11-20 04:25:55,603::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:25:55,605::task::495::TaskManager.Task::(_debug) Task bcd64456-b1f3-4cbc-a929-74542777ad77: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:25:55,607::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`15e44980-d4ca-4c57-a579-f23f59bdc9f6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:25:55,608::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:25:55,609::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:25:55,610::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`15e44980-d4ca-4c57-a579-f23f59bdc9f6`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:25:55,611::task::495::TaskManager.Task::(_debug) Task bcd64456-b1f3-4cbc-a929-74542777ad77: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:25:55,612::task::495::TaskManager.Task::(_debug) Task bcd64456-b1f3-4cbc-a929-74542777ad77: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:25:55,613::task::495::TaskManager.Task::(_debug) Task bcd64456-b1f3-4cbc-a929-74542777ad77: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:25:55,613::task::495::TaskManager.Task::(_debug) Task bcd64456-b1f3-4cbc-a929-74542777ad77: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:25:55,614::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:25:55,615::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:25:55,615::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:25:55,616::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:25:55,616::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:25:55,617::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:25:55,618::task::495::TaskManager.Task::(_debug) Task bcd64456-b1f3-4cbc-a929-74542777ad77: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:25:55,618::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:25:55,619::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:25:55,620::task::495::TaskManager.Task::(_debug) Task 541eb35f-6c0b-461f-9333-841f837d183a: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:25:55,621::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`290b80ae-57a1-4b08-a292-8494c9823de9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:25:55,621::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:25:55,622::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:25:55,623::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`290b80ae-57a1-4b08-a292-8494c9823de9`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:25:55,624::task::495::TaskManager.Task::(_debug) Task 541eb35f-6c0b-461f-9333-841f837d183a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:25:55,624::task::495::TaskManager.Task::(_debug) Task 541eb35f-6c0b-461f-9333-841f837d183a: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:25:55,625::task::495::TaskManager.Task::(_debug) Task 541eb35f-6c0b-461f-9333-841f837d183a: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:25:55,626::task::495::TaskManager.Task::(_debug) Task 541eb35f-6c0b-461f-9333-841f837d183a: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:25:55,626::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:25:55,627::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:25:55,627::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:25:55,628::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:25:55,629::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:25:55,629::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:25:55,630::task::495::TaskManager.Task::(_debug) Task 541eb35f-6c0b-461f-9333-841f837d183a: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:25:55,630::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913658::INFO::2013-11-20 04:25:55,754::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:25:55,755::task::495::TaskManager.Task::(_debug) Task 4dbc3b77-7803-4605-a32b-c3c1f03de818: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:25:55,757::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7d753c1a-8429-440e-a6c9-73249409cc58`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:25:55,758::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:25:55,759::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:25:55,760::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7d753c1a-8429-440e-a6c9-73249409cc58`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:25:55,761::task::495::TaskManager.Task::(_debug) Task 4dbc3b77-7803-4605-a32b-c3c1f03de818: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:25:55,762::task::495::TaskManager.Task::(_debug) Task 4dbc3b77-7803-4605-a32b-c3c1f03de818: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:25:55,763::task::495::TaskManager.Task::(_debug) Task 4dbc3b77-7803-4605-a32b-c3c1f03de818: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:25:55,764::task::495::TaskManager.Task::(_debug) Task 4dbc3b77-7803-4605-a32b-c3c1f03de818: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:25:55,764::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:25:55,765::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:25:55,766::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:25:55,767::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:25:55,768::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:25:55,769::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:25:55,770::task::495::TaskManager.Task::(_debug) Task 4dbc3b77-7803-4605-a32b-c3c1f03de818: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:25:55,770::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:25:55,848::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0903836 s, 11.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:25:57,864::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:25:58,425::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0820472 s, 12.5 MB/s\n'; <rc> = 0
>Thread-15916388::DEBUG::2013-11-20 04:26:00,307::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916388::INFO::2013-11-20 04:26:00,309::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916388::DEBUG::2013-11-20 04:26:00,309::task::495::TaskManager.Task::(_debug) Task 1d35047f-d2ab-4da4-bdea-55d00ad8ad61: moving from state init -> state preparing
>Thread-15916388::DEBUG::2013-11-20 04:26:00,310::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916388::DEBUG::2013-11-20 04:26:00,311::task::495::TaskManager.Task::(_debug) Task 1d35047f-d2ab-4da4-bdea-55d00ad8ad61: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916388::DEBUG::2013-11-20 04:26:00,312::task::495::TaskManager.Task::(_debug) Task 1d35047f-d2ab-4da4-bdea-55d00ad8ad61: moving from state preparing -> state finished
>Thread-15916388::DEBUG::2013-11-20 04:26:00,313::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916388::DEBUG::2013-11-20 04:26:00,313::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916388::DEBUG::2013-11-20 04:26:00,314::task::495::TaskManager.Task::(_debug) Task 1d35047f-d2ab-4da4-bdea-55d00ad8ad61: ref 0 aborting False
>Thread-15916388::INFO::2013-11-20 04:26:00,315::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916389::DEBUG::2013-11-20 04:26:00,320::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916389::INFO::2013-11-20 04:26:00,322::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916389::DEBUG::2013-11-20 04:26:00,322::task::495::TaskManager.Task::(_debug) Task 5c252418-6472-485e-be36-3cd2cf978f2a: moving from state init -> state preparing
>Thread-15916389::DEBUG::2013-11-20 04:26:00,324::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0255fa85-8274-413b-848d-dbfb2dd29c8f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916389::DEBUG::2013-11-20 04:26:00,324::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916389::DEBUG::2013-11-20 04:26:00,325::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916389::DEBUG::2013-11-20 04:26:00,326::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0255fa85-8274-413b-848d-dbfb2dd29c8f`::Granted request
>Thread-15916389::DEBUG::2013-11-20 04:26:00,327::task::495::TaskManager.Task::(_debug) Task 5c252418-6472-485e-be36-3cd2cf978f2a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916389::DEBUG::2013-11-20 04:26:00,328::task::495::TaskManager.Task::(_debug) Task 5c252418-6472-485e-be36-3cd2cf978f2a: ref 1 aborting False
>Thread-15916389::WARNING::2013-11-20 04:26:00,330::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916389::DEBUG::2013-11-20 04:26:00,332::task::495::TaskManager.Task::(_debug) Task 5c252418-6472-485e-be36-3cd2cf978f2a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916389::DEBUG::2013-11-20 04:26:00,333::task::495::TaskManager.Task::(_debug) Task 5c252418-6472-485e-be36-3cd2cf978f2a: moving from state preparing -> state finished
>Thread-15916389::DEBUG::2013-11-20 04:26:00,333::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916389::DEBUG::2013-11-20 04:26:00,334::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916389::DEBUG::2013-11-20 04:26:00,335::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916389::DEBUG::2013-11-20 04:26:00,336::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916389::DEBUG::2013-11-20 04:26:00,336::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916389::DEBUG::2013-11-20 04:26:00,337::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916389::DEBUG::2013-11-20 04:26:00,338::task::495::TaskManager.Task::(_debug) Task 5c252418-6472-485e-be36-3cd2cf978f2a: ref 0 aborting False
>Thread-15916389::INFO::2013-11-20 04:26:00,338::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916390::INFO::2013-11-20 04:26:00,346::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916390::DEBUG::2013-11-20 04:26:00,347::task::495::TaskManager.Task::(_debug) Task 89f10c3e-8f20-4e77-9006-c7c68f56daa8: moving from state init -> state preparing
>Thread-15916390::DEBUG::2013-11-20 04:26:00,348::task::495::TaskManager.Task::(_debug) Task 89f10c3e-8f20-4e77-9006-c7c68f56daa8: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000435829162598', 'lastCheck': 1384921556.942241, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00205612182617', 'lastCheck': 1384921555.487859, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0136790275574', 'lastCheck': 1384921552.3469419, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000331163406372', 'lastCheck': 1384921552.071975, 'valid': True, 'code': 0}}
>Thread-15916390::DEBUG::2013-11-20 04:26:00,349::task::495::TaskManager.Task::(_debug) Task 89f10c3e-8f20-4e77-9006-c7c68f56daa8: moving from state preparing -> state finished
>Thread-15916390::DEBUG::2013-11-20 04:26:00,350::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916390::DEBUG::2013-11-20 04:26:00,350::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916390::DEBUG::2013-11-20 04:26:00,351::task::495::TaskManager.Task::(_debug) Task 89f10c3e-8f20-4e77-9006-c7c68f56daa8: ref 0 aborting False
>Thread-15916390::INFO::2013-11-20 04:26:00,352::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000435829162598', 'lastCheck': 1384921556.942241, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00205612182617', 'lastCheck': 1384921555.487859, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0136790275574', 'lastCheck': 1384921552.3469419, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000331163406372', 'lastCheck': 1384921552.071975, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:26:00,441::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:26:00,983::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0808651 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:26:02,998::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:26:03,549::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0792333 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:26:05,566::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 04:26:05,770::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:26:05,778::task::495::TaskManager.Task::(_debug) Task 6ec9bba2-b207-4553-9cd5-e988d4070c48: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:26:05,790::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`df1d8e7c-b656-404a-be1b-6a828e996871`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:26:05,791::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:26:05,792::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:26:05,793::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`df1d8e7c-b656-404a-be1b-6a828e996871`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:26:05,794::task::495::TaskManager.Task::(_debug) Task 6ec9bba2-b207-4553-9cd5-e988d4070c48: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:26:05,795::task::495::TaskManager.Task::(_debug) Task 6ec9bba2-b207-4553-9cd5-e988d4070c48: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:26:05,796::task::495::TaskManager.Task::(_debug) Task 6ec9bba2-b207-4553-9cd5-e988d4070c48: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:26:05,797::task::495::TaskManager.Task::(_debug) Task 6ec9bba2-b207-4553-9cd5-e988d4070c48: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:26:05,798::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:26:05,798::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:26:05,799::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:26:05,800::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:26:05,801::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:26:05,801::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:26:05,802::task::495::TaskManager.Task::(_debug) Task 6ec9bba2-b207-4553-9cd5-e988d4070c48: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:26:05,803::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:26:06,118::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0825621 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:26:08,134::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:26:08,701::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0797497 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:26:10,718::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916395::DEBUG::2013-11-20 04:26:11,098::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916395::INFO::2013-11-20 04:26:11,100::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916395::DEBUG::2013-11-20 04:26:11,101::task::495::TaskManager.Task::(_debug) Task 7daee276-bcc4-4493-8e61-969d9427ad2b: moving from state init -> state preparing
>Thread-15916395::DEBUG::2013-11-20 04:26:11,102::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916395::DEBUG::2013-11-20 04:26:11,102::task::495::TaskManager.Task::(_debug) Task 7daee276-bcc4-4493-8e61-969d9427ad2b: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916395::DEBUG::2013-11-20 04:26:11,103::task::495::TaskManager.Task::(_debug) Task 7daee276-bcc4-4493-8e61-969d9427ad2b: moving from state preparing -> state finished
>Thread-15916395::DEBUG::2013-11-20 04:26:11,104::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916395::DEBUG::2013-11-20 04:26:11,104::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916395::DEBUG::2013-11-20 04:26:11,105::task::495::TaskManager.Task::(_debug) Task 7daee276-bcc4-4493-8e61-969d9427ad2b: ref 0 aborting False
>Thread-15916395::INFO::2013-11-20 04:26:11,106::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916396::DEBUG::2013-11-20 04:26:11,110::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916396::INFO::2013-11-20 04:26:11,111::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916396::DEBUG::2013-11-20 04:26:11,111::task::495::TaskManager.Task::(_debug) Task a1c64de8-03b1-4c95-9a9d-3aa7b67ac0e8: moving from state init -> state preparing
>Thread-15916396::DEBUG::2013-11-20 04:26:11,113::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`60254a4c-69c5-45c8-bae8-66a5c2da3a36`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916396::DEBUG::2013-11-20 04:26:11,114::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916396::DEBUG::2013-11-20 04:26:11,115::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916396::DEBUG::2013-11-20 04:26:11,115::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`60254a4c-69c5-45c8-bae8-66a5c2da3a36`::Granted request
>Thread-15916396::DEBUG::2013-11-20 04:26:11,116::task::495::TaskManager.Task::(_debug) Task a1c64de8-03b1-4c95-9a9d-3aa7b67ac0e8: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916396::DEBUG::2013-11-20 04:26:11,117::task::495::TaskManager.Task::(_debug) Task a1c64de8-03b1-4c95-9a9d-3aa7b67ac0e8: ref 1 aborting False
>Thread-15916396::WARNING::2013-11-20 04:26:11,119::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916396::DEBUG::2013-11-20 04:26:11,121::task::495::TaskManager.Task::(_debug) Task a1c64de8-03b1-4c95-9a9d-3aa7b67ac0e8: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916396::DEBUG::2013-11-20 04:26:11,122::task::495::TaskManager.Task::(_debug) Task a1c64de8-03b1-4c95-9a9d-3aa7b67ac0e8: moving from state preparing -> state finished
>Thread-15916396::DEBUG::2013-11-20 04:26:11,123::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916396::DEBUG::2013-11-20 04:26:11,124::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916396::DEBUG::2013-11-20 04:26:11,126::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916396::DEBUG::2013-11-20 04:26:11,128::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916396::DEBUG::2013-11-20 04:26:11,129::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916396::DEBUG::2013-11-20 04:26:11,130::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916396::DEBUG::2013-11-20 04:26:11,131::task::495::TaskManager.Task::(_debug) Task a1c64de8-03b1-4c95-9a9d-3aa7b67ac0e8: ref 0 aborting False
>Thread-15916396::INFO::2013-11-20 04:26:11,132::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:26:11,267::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0894354 s, 11.4 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 04:26:11,346::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:26:11,347::task::495::TaskManager.Task::(_debug) Task 13da259c-55ab-4e04-8df3-91f77802567b: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:26:11,348::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4483575b-aa86-4696-b2ea-2a664ca4791f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:26:11,349::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:26:11,350::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:26:11,350::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4483575b-aa86-4696-b2ea-2a664ca4791f`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:26:11,351::task::495::TaskManager.Task::(_debug) Task 13da259c-55ab-4e04-8df3-91f77802567b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:26:11,352::task::495::TaskManager.Task::(_debug) Task 13da259c-55ab-4e04-8df3-91f77802567b: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:26:11,353::task::495::TaskManager.Task::(_debug) Task 13da259c-55ab-4e04-8df3-91f77802567b: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:26:11,353::task::495::TaskManager.Task::(_debug) Task 13da259c-55ab-4e04-8df3-91f77802567b: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:26:11,354::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:26:11,354::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:26:11,355::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:26:11,356::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:26:11,356::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:26:11,357::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:26:11,358::task::495::TaskManager.Task::(_debug) Task 13da259c-55ab-4e04-8df3-91f77802567b: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:26:11,359::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:26:13,283::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:26:13,856::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0767409 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:26:15,872::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 04:26:16,095::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:26:16,102::task::495::TaskManager.Task::(_debug) Task 6d058af4-a719-49d3-a0a1-d6d450f51fc0: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:26:16,106::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3ff26a15-9597-4205-a933-ea5d85bcb4fc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:26:16,108::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:26:16,109::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:26:16,110::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3ff26a15-9597-4205-a933-ea5d85bcb4fc`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:26:16,112::task::495::TaskManager.Task::(_debug) Task 6d058af4-a719-49d3-a0a1-d6d450f51fc0: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:26:16,113::task::495::TaskManager.Task::(_debug) Task 6d058af4-a719-49d3-a0a1-d6d450f51fc0: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:26:16,113::task::495::TaskManager.Task::(_debug) Task 6d058af4-a719-49d3-a0a1-d6d450f51fc0: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:26:16,114::task::495::TaskManager.Task::(_debug) Task 6d058af4-a719-49d3-a0a1-d6d450f51fc0: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:26:16,114::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:26:16,114::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:26:16,115::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:26:16,115::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:26:16,116::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:26:16,116::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:26:16,116::task::495::TaskManager.Task::(_debug) Task 6d058af4-a719-49d3-a0a1-d6d450f51fc0: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:26:16,117::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:26:16,426::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0801463 s, 12.8 MB/s\n'; <rc> = 0
>Thread-15916398::INFO::2013-11-20 04:26:17,000::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916398::DEBUG::2013-11-20 04:26:17,001::task::495::TaskManager.Task::(_debug) Task 8a037f98-52df-4ef5-861e-65575b0a5b47: moving from state init -> state preparing
>Thread-15916398::DEBUG::2013-11-20 04:26:17,002::task::495::TaskManager.Task::(_debug) Task 8a037f98-52df-4ef5-861e-65575b0a5b47: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000319957733154', 'lastCheck': 1384921567.2157011, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00178909301758', 'lastCheck': 1384921576.1093409, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00153708457947', 'lastCheck': 1384921572.93946, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000514030456543', 'lastCheck': 1384921572.3588099, 'valid': True, 'code': 0}}
>Thread-15916398::DEBUG::2013-11-20 04:26:17,003::task::495::TaskManager.Task::(_debug) Task 8a037f98-52df-4ef5-861e-65575b0a5b47: moving from state preparing -> state finished
>Thread-15916398::DEBUG::2013-11-20 04:26:17,003::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916398::DEBUG::2013-11-20 04:26:17,004::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916398::DEBUG::2013-11-20 04:26:17,004::task::495::TaskManager.Task::(_debug) Task 8a037f98-52df-4ef5-861e-65575b0a5b47: ref 0 aborting False
>Thread-15916398::INFO::2013-11-20 04:26:17,005::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000319957733154', 'lastCheck': 1384921567.2157011, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00178909301758', 'lastCheck': 1384921576.1093409, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00153708457947', 'lastCheck': 1384921572.93946, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000514030456543', 'lastCheck': 1384921572.3588099, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:26:18,441::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:26:18,993::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0916287 s, 11.2 MB/s\n'; <rc> = 0
>Thread-15916400::DEBUG::2013-11-20 04:26:19,162::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Thread-15916400::DEBUG::2013-11-20 04:26:19,839::clientIF::1219::vds::(fenceNode) rc 0 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = On
>Done
> err 
>Thread-15916401::DEBUG::2013-11-20 04:26:19,846::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Thread-15916401::DEBUG::2013-11-20 04:26:20,502::clientIF::1219::vds::(fenceNode) rc 0 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = On
>Done
> err 
>Dummy-15912651::DEBUG::2013-11-20 04:26:21,009::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916403::DEBUG::2013-11-20 04:26:21,424::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916403::INFO::2013-11-20 04:26:21,429::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916403::DEBUG::2013-11-20 04:26:21,431::task::495::TaskManager.Task::(_debug) Task 17d55a39-1a14-4908-9853-9d68fc08982e: moving from state init -> state preparing
>Thread-15916403::DEBUG::2013-11-20 04:26:21,433::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916403::DEBUG::2013-11-20 04:26:21,433::task::495::TaskManager.Task::(_debug) Task 17d55a39-1a14-4908-9853-9d68fc08982e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916403::DEBUG::2013-11-20 04:26:21,434::task::495::TaskManager.Task::(_debug) Task 17d55a39-1a14-4908-9853-9d68fc08982e: moving from state preparing -> state finished
>Thread-15916403::DEBUG::2013-11-20 04:26:21,435::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916403::DEBUG::2013-11-20 04:26:21,436::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916403::DEBUG::2013-11-20 04:26:21,438::task::495::TaskManager.Task::(_debug) Task 17d55a39-1a14-4908-9853-9d68fc08982e: ref 0 aborting False
>Thread-15916403::INFO::2013-11-20 04:26:21,439::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916404::DEBUG::2013-11-20 04:26:21,451::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916404::INFO::2013-11-20 04:26:21,453::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916404::DEBUG::2013-11-20 04:26:21,454::task::495::TaskManager.Task::(_debug) Task 05eeedfd-90dc-4dbd-a8f4-a98cedd40b5e: moving from state init -> state preparing
>Thread-15916404::DEBUG::2013-11-20 04:26:21,457::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`07feb051-7279-417e-a52d-063f1c2fbcd4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916404::DEBUG::2013-11-20 04:26:21,458::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916404::DEBUG::2013-11-20 04:26:21,459::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916404::DEBUG::2013-11-20 04:26:21,460::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`07feb051-7279-417e-a52d-063f1c2fbcd4`::Granted request
>Thread-15916404::DEBUG::2013-11-20 04:26:21,461::task::495::TaskManager.Task::(_debug) Task 05eeedfd-90dc-4dbd-a8f4-a98cedd40b5e: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916404::DEBUG::2013-11-20 04:26:21,462::task::495::TaskManager.Task::(_debug) Task 05eeedfd-90dc-4dbd-a8f4-a98cedd40b5e: ref 1 aborting False
>Thread-15916404::WARNING::2013-11-20 04:26:21,465::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916404::DEBUG::2013-11-20 04:26:21,467::task::495::TaskManager.Task::(_debug) Task 05eeedfd-90dc-4dbd-a8f4-a98cedd40b5e: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916404::DEBUG::2013-11-20 04:26:21,468::task::495::TaskManager.Task::(_debug) Task 05eeedfd-90dc-4dbd-a8f4-a98cedd40b5e: moving from state preparing -> state finished
>Thread-15916404::DEBUG::2013-11-20 04:26:21,469::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916404::DEBUG::2013-11-20 04:26:21,470::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916404::DEBUG::2013-11-20 04:26:21,471::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916404::DEBUG::2013-11-20 04:26:21,472::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916404::DEBUG::2013-11-20 04:26:21,473::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916404::DEBUG::2013-11-20 04:26:21,473::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916404::DEBUG::2013-11-20 04:26:21,474::task::495::TaskManager.Task::(_debug) Task 05eeedfd-90dc-4dbd-a8f4-a98cedd40b5e: ref 0 aborting False
>Thread-15916404::INFO::2013-11-20 04:26:21,475::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:26:21,588::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.106139 s, 9.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:26:23,602::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 04:26:23,835::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:26:23,844::task::495::TaskManager.Task::(_debug) Task bf79a453-d6c4-465a-989d-2ce621a9fd05: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:26:23,852::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7c72e250-91e6-47dc-8328-54977fe26fbf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:26:23,853::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:26:23,855::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:26:23,856::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7c72e250-91e6-47dc-8328-54977fe26fbf`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:26:23,857::task::495::TaskManager.Task::(_debug) Task bf79a453-d6c4-465a-989d-2ce621a9fd05: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 04:26:23,858::task::495::TaskManager.Task::(_debug) Task bf79a453-d6c4-465a-989d-2ce621a9fd05: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:26:23,859::task::495::TaskManager.Task::(_debug) Task bf79a453-d6c4-465a-989d-2ce621a9fd05: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 04:26:23,860::task::495::TaskManager.Task::(_debug) Task bf79a453-d6c4-465a-989d-2ce621a9fd05: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:26:23,861::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:26:23,862::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:26:23,869::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 04:26:23,870::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:26:23,870::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:26:23,871::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:26:23,872::task::495::TaskManager.Task::(_debug) Task bf79a453-d6c4-465a-989d-2ce621a9fd05: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 04:26:23,872::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 04:26:24,126::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0784414 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:26:26,141::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 04:26:26,350::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:26:26,354::task::495::TaskManager.Task::(_debug) Task d0691d86-8b05-4ea0-805d-e080b5b27aa9: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:26:26,358::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`4d641901-c16b-4de7-b95a-2740cc509842`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:26:26,360::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:26:26,361::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:26:26,361::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`4d641901-c16b-4de7-b95a-2740cc509842`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:26:26,363::task::495::TaskManager.Task::(_debug) Task d0691d86-8b05-4ea0-805d-e080b5b27aa9: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:26:26,363::task::495::TaskManager.Task::(_debug) Task d0691d86-8b05-4ea0-805d-e080b5b27aa9: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:26:26,364::task::495::TaskManager.Task::(_debug) Task d0691d86-8b05-4ea0-805d-e080b5b27aa9: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 04:26:26,365::task::495::TaskManager.Task::(_debug) Task d0691d86-8b05-4ea0-805d-e080b5b27aa9: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:26:26,365::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:26:26,366::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:26:26,367::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:26:26,367::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:26:26,368::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:26:26,369::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:26:26,370::task::495::TaskManager.Task::(_debug) Task d0691d86-8b05-4ea0-805d-e080b5b27aa9: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:26:26,371::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:26:26,372::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:26:26,373::task::495::TaskManager.Task::(_debug) Task bb930405-cee1-4d93-8375-305f353b61ac: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:26:26,374::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a6c35315-51e2-4831-892f-4a33c7b6fab4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:26:26,375::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:26:26,376::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:26:26,377::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a6c35315-51e2-4831-892f-4a33c7b6fab4`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:26:26,378::task::495::TaskManager.Task::(_debug) Task bb930405-cee1-4d93-8375-305f353b61ac: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:26:26,378::task::495::TaskManager.Task::(_debug) Task bb930405-cee1-4d93-8375-305f353b61ac: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:26:26,379::task::495::TaskManager.Task::(_debug) Task bb930405-cee1-4d93-8375-305f353b61ac: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:26:26,380::task::495::TaskManager.Task::(_debug) Task bb930405-cee1-4d93-8375-305f353b61ac: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:26:26,380::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:26:26,381::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:26:26,382::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:26:26,382::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:26:26,383::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:26:26,383::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:26:26,384::task::495::TaskManager.Task::(_debug) Task bb930405-cee1-4d93-8375-305f353b61ac: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:26:26,384::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:26:26,385::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:26:26,386::task::495::TaskManager.Task::(_debug) Task 7cb4c3fe-fc9d-454e-8acc-257fadcf6418: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:26:26,387::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8c51588f-08b6-4a4a-87e1-530bcd770d29`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:26:26,387::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:26:26,388::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:26:26,389::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8c51588f-08b6-4a4a-87e1-530bcd770d29`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:26:26,390::task::495::TaskManager.Task::(_debug) Task 7cb4c3fe-fc9d-454e-8acc-257fadcf6418: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:26:26,390::task::495::TaskManager.Task::(_debug) Task 7cb4c3fe-fc9d-454e-8acc-257fadcf6418: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:26:26,391::task::495::TaskManager.Task::(_debug) Task 7cb4c3fe-fc9d-454e-8acc-257fadcf6418: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:26:26,392::task::495::TaskManager.Task::(_debug) Task 7cb4c3fe-fc9d-454e-8acc-257fadcf6418: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:26:26,392::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:26:26,393::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:26:26,394::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:26:26,394::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:26:26,395::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:26:26,395::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:26:26,396::task::495::TaskManager.Task::(_debug) Task 7cb4c3fe-fc9d-454e-8acc-257fadcf6418: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:26:26,396::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:26:26,397::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:26:26,398::task::495::TaskManager.Task::(_debug) Task f3993c49-b21f-4900-8af6-29251db8de0a: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:26:26,399::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2c513a81-7308-4800-9799-f46708d1d575`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:26:26,399::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:26:26,400::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:26:26,400::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2c513a81-7308-4800-9799-f46708d1d575`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:26:26,401::task::495::TaskManager.Task::(_debug) Task f3993c49-b21f-4900-8af6-29251db8de0a: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:26:26,402::task::495::TaskManager.Task::(_debug) Task f3993c49-b21f-4900-8af6-29251db8de0a: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:26:26,403::task::495::TaskManager.Task::(_debug) Task f3993c49-b21f-4900-8af6-29251db8de0a: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:26:26,403::task::495::TaskManager.Task::(_debug) Task f3993c49-b21f-4900-8af6-29251db8de0a: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:26:26,404::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:26:26,404::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:26:26,405::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:26:26,405::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:26:26,406::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:26:26,407::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:26:26,407::task::495::TaskManager.Task::(_debug) Task f3993c49-b21f-4900-8af6-29251db8de0a: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:26:26,408::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:26:26,681::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0929757 s, 11.0 MB/s\n'; <rc> = 0
>Thread-15914435::INFO::2013-11-20 04:26:26,791::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:26:26,793::task::495::TaskManager.Task::(_debug) Task ded67467-3968-4b42-96ca-7807ba3756ef: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:26:26,794::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`29dbe962-6162-4b57-8c6e-0852825965de`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:26:26,795::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:26:26,796::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:26:26,796::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`29dbe962-6162-4b57-8c6e-0852825965de`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:26:26,798::task::495::TaskManager.Task::(_debug) Task ded67467-3968-4b42-96ca-7807ba3756ef: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:26:26,798::task::495::TaskManager.Task::(_debug) Task ded67467-3968-4b42-96ca-7807ba3756ef: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:26:26,799::task::495::TaskManager.Task::(_debug) Task ded67467-3968-4b42-96ca-7807ba3756ef: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:26:26,800::task::495::TaskManager.Task::(_debug) Task ded67467-3968-4b42-96ca-7807ba3756ef: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:26:26,801::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:26:26,802::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:26:26,803::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:26:26,803::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:26:26,804::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:26:26,805::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:26:26,806::task::495::TaskManager.Task::(_debug) Task ded67467-3968-4b42-96ca-7807ba3756ef: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:26:26,807::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:26:28,700::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:26:29,271::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0836869 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:26:31,289::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916408::DEBUG::2013-11-20 04:26:31,691::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916408::INFO::2013-11-20 04:26:31,692::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916408::DEBUG::2013-11-20 04:26:31,693::task::495::TaskManager.Task::(_debug) Task 476ada1d-931e-4bb9-8a81-c22c8e285263: moving from state init -> state preparing
>Thread-15916408::DEBUG::2013-11-20 04:26:31,693::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916408::DEBUG::2013-11-20 04:26:31,694::task::495::TaskManager.Task::(_debug) Task 476ada1d-931e-4bb9-8a81-c22c8e285263: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916408::DEBUG::2013-11-20 04:26:31,694::task::495::TaskManager.Task::(_debug) Task 476ada1d-931e-4bb9-8a81-c22c8e285263: moving from state preparing -> state finished
>Thread-15916408::DEBUG::2013-11-20 04:26:31,695::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916408::DEBUG::2013-11-20 04:26:31,695::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916408::DEBUG::2013-11-20 04:26:31,695::task::495::TaskManager.Task::(_debug) Task 476ada1d-931e-4bb9-8a81-c22c8e285263: ref 0 aborting False
>Thread-15916408::INFO::2013-11-20 04:26:31,696::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916409::DEBUG::2013-11-20 04:26:31,700::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916409::INFO::2013-11-20 04:26:31,702::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916409::DEBUG::2013-11-20 04:26:31,703::task::495::TaskManager.Task::(_debug) Task e8d26b74-5f1f-4780-91f6-e60b4b3b976a: moving from state init -> state preparing
>Thread-15916409::DEBUG::2013-11-20 04:26:31,705::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1eeea447-93c2-4fb1-9836-0b503ba4ff83`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916409::DEBUG::2013-11-20 04:26:31,706::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916409::DEBUG::2013-11-20 04:26:31,707::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916409::DEBUG::2013-11-20 04:26:31,708::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1eeea447-93c2-4fb1-9836-0b503ba4ff83`::Granted request
>Thread-15916409::DEBUG::2013-11-20 04:26:31,710::task::495::TaskManager.Task::(_debug) Task e8d26b74-5f1f-4780-91f6-e60b4b3b976a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916409::DEBUG::2013-11-20 04:26:31,711::task::495::TaskManager.Task::(_debug) Task e8d26b74-5f1f-4780-91f6-e60b4b3b976a: ref 1 aborting False
>Thread-15916409::WARNING::2013-11-20 04:26:31,714::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916409::DEBUG::2013-11-20 04:26:31,716::task::495::TaskManager.Task::(_debug) Task e8d26b74-5f1f-4780-91f6-e60b4b3b976a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916409::DEBUG::2013-11-20 04:26:31,718::task::495::TaskManager.Task::(_debug) Task e8d26b74-5f1f-4780-91f6-e60b4b3b976a: moving from state preparing -> state finished
>Thread-15916409::DEBUG::2013-11-20 04:26:31,719::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916409::DEBUG::2013-11-20 04:26:31,720::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916409::DEBUG::2013-11-20 04:26:31,722::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916409::DEBUG::2013-11-20 04:26:31,723::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916409::DEBUG::2013-11-20 04:26:31,724::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916409::DEBUG::2013-11-20 04:26:31,726::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916409::DEBUG::2013-11-20 04:26:31,727::task::495::TaskManager.Task::(_debug) Task e8d26b74-5f1f-4780-91f6-e60b4b3b976a: ref 0 aborting False
>Thread-15916409::INFO::2013-11-20 04:26:31,728::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:26:31,807::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0734069 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15916410::INFO::2013-11-20 04:26:32,587::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916410::DEBUG::2013-11-20 04:26:32,588::task::495::TaskManager.Task::(_debug) Task f6c932fc-49ad-4415-a20f-4a783dfa1c32: moving from state init -> state preparing
>Thread-15916410::DEBUG::2013-11-20 04:26:32,589::task::495::TaskManager.Task::(_debug) Task f6c932fc-49ad-4415-a20f-4a783dfa1c32: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000409841537476', 'lastCheck': 1384921587.8287671, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00436305999756', 'lastCheck': 1384921586.3592501, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00141406059265', 'lastCheck': 1384921583.2150121, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00053882598877', 'lastCheck': 1384921582.6364641, 'valid': True, 'code': 0}}
>Thread-15916410::DEBUG::2013-11-20 04:26:32,590::task::495::TaskManager.Task::(_debug) Task f6c932fc-49ad-4415-a20f-4a783dfa1c32: moving from state preparing -> state finished
>Thread-15916410::DEBUG::2013-11-20 04:26:32,591::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916410::DEBUG::2013-11-20 04:26:32,592::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916410::DEBUG::2013-11-20 04:26:32,593::task::495::TaskManager.Task::(_debug) Task f6c932fc-49ad-4415-a20f-4a783dfa1c32: ref 0 aborting False
>Thread-15916410::INFO::2013-11-20 04:26:32,594::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000409841537476', 'lastCheck': 1384921587.8287671, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00436305999756', 'lastCheck': 1384921586.3592501, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00141406059265', 'lastCheck': 1384921583.2150121, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00053882598877', 'lastCheck': 1384921582.6364641, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:26:33,818::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:26:34,345::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.075726 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:26:36,359::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:26:36,881::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.104222 s, 9.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:26:38,897::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:26:39,368::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0732264 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:26:41,377::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:26:41,910::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0712397 s, 14.4 MB/s\n'; <rc> = 0
>Thread-15916414::DEBUG::2013-11-20 04:26:42,131::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916414::INFO::2013-11-20 04:26:42,132::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916414::DEBUG::2013-11-20 04:26:42,133::task::495::TaskManager.Task::(_debug) Task c4eebc6c-21de-4556-8082-a6b5e9abbe00: moving from state init -> state preparing
>Thread-15916414::DEBUG::2013-11-20 04:26:42,133::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916414::DEBUG::2013-11-20 04:26:42,134::task::495::TaskManager.Task::(_debug) Task c4eebc6c-21de-4556-8082-a6b5e9abbe00: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916414::DEBUG::2013-11-20 04:26:42,134::task::495::TaskManager.Task::(_debug) Task c4eebc6c-21de-4556-8082-a6b5e9abbe00: moving from state preparing -> state finished
>Thread-15916414::DEBUG::2013-11-20 04:26:42,134::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916414::DEBUG::2013-11-20 04:26:42,135::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916414::DEBUG::2013-11-20 04:26:42,135::task::495::TaskManager.Task::(_debug) Task c4eebc6c-21de-4556-8082-a6b5e9abbe00: ref 0 aborting False
>Thread-15916414::INFO::2013-11-20 04:26:42,136::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916415::DEBUG::2013-11-20 04:26:42,139::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916415::INFO::2013-11-20 04:26:42,140::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916415::DEBUG::2013-11-20 04:26:42,140::task::495::TaskManager.Task::(_debug) Task d08f5ff2-022f-4249-8511-5dd0265cc498: moving from state init -> state preparing
>Thread-15916415::DEBUG::2013-11-20 04:26:42,141::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`37a4d18f-91f8-4480-87b9-5f4b3300007f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916415::DEBUG::2013-11-20 04:26:42,141::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916415::DEBUG::2013-11-20 04:26:42,142::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916415::DEBUG::2013-11-20 04:26:42,142::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`37a4d18f-91f8-4480-87b9-5f4b3300007f`::Granted request
>Thread-15916415::DEBUG::2013-11-20 04:26:42,143::task::495::TaskManager.Task::(_debug) Task d08f5ff2-022f-4249-8511-5dd0265cc498: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916415::DEBUG::2013-11-20 04:26:42,143::task::495::TaskManager.Task::(_debug) Task d08f5ff2-022f-4249-8511-5dd0265cc498: ref 1 aborting False
>Thread-15916415::WARNING::2013-11-20 04:26:42,144::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916415::DEBUG::2013-11-20 04:26:42,145::task::495::TaskManager.Task::(_debug) Task d08f5ff2-022f-4249-8511-5dd0265cc498: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916415::DEBUG::2013-11-20 04:26:42,146::task::495::TaskManager.Task::(_debug) Task d08f5ff2-022f-4249-8511-5dd0265cc498: moving from state preparing -> state finished
>Thread-15916415::DEBUG::2013-11-20 04:26:42,146::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916415::DEBUG::2013-11-20 04:26:42,146::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916415::DEBUG::2013-11-20 04:26:42,147::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916415::DEBUG::2013-11-20 04:26:42,147::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916415::DEBUG::2013-11-20 04:26:42,148::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916415::DEBUG::2013-11-20 04:26:42,148::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916415::DEBUG::2013-11-20 04:26:42,148::task::495::TaskManager.Task::(_debug) Task d08f5ff2-022f-4249-8511-5dd0265cc498: ref 0 aborting False
>Thread-15916415::INFO::2013-11-20 04:26:42,149::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:26:43,923::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:26:44,104::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15915918::INFO::2013-11-20 04:26:44,107::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15913639::DEBUG::2013-11-20 04:26:44,108::task::495::TaskManager.Task::(_debug) Task 056c3827-8a24-48ee-9ce6-7f5681c226bc: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:26:44,110::task::495::TaskManager.Task::(_debug) Task 0e693f3a-aadc-48d3-829b-759dd6bc57f5: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:26:44,112::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3003adf1-830e-45ab-b4de-9f9d6dcb9e9e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:26:44,118::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`39c1cfd4-961e-4632-b932-955d6eb35973`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:26:44,118::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:26:44,119::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:26:44,121::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:26:44,122::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3003adf1-830e-45ab-b4de-9f9d6dcb9e9e`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:26:44,123::resourceManager::481::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' found in shared state and queue is empty, Joining current shared lock (2 active users)
>Thread-15913639::DEBUG::2013-11-20 04:26:44,123::task::495::TaskManager.Task::(_debug) Task 056c3827-8a24-48ee-9ce6-7f5681c226bc: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:26:44,124::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`39c1cfd4-961e-4632-b932-955d6eb35973`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:26:44,124::task::495::TaskManager.Task::(_debug) Task 056c3827-8a24-48ee-9ce6-7f5681c226bc: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:26:44,126::task::495::TaskManager.Task::(_debug) Task 0e693f3a-aadc-48d3-829b-759dd6bc57f5: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:26:44,127::task::495::TaskManager.Task::(_debug) Task 056c3827-8a24-48ee-9ce6-7f5681c226bc: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15915918::DEBUG::2013-11-20 04:26:44,127::task::495::TaskManager.Task::(_debug) Task 0e693f3a-aadc-48d3-829b-759dd6bc57f5: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:26:44,128::task::495::TaskManager.Task::(_debug) Task 056c3827-8a24-48ee-9ce6-7f5681c226bc: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:26:44,129::task::495::TaskManager.Task::(_debug) Task 0e693f3a-aadc-48d3-829b-759dd6bc57f5: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15913639::DEBUG::2013-11-20 04:26:44,129::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:26:44,130::task::495::TaskManager.Task::(_debug) Task 0e693f3a-aadc-48d3-829b-759dd6bc57f5: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:26:44,130::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:26:44,131::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:26:44,132::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:26:44,132::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:26:44,133::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (1 active users)
>Thread-15915918::DEBUG::2013-11-20 04:26:44,134::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:26:44,134::task::495::TaskManager.Task::(_debug) Task 056c3827-8a24-48ee-9ce6-7f5681c226bc: ref 0 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:26:44,135::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::INFO::2013-11-20 04:26:44,136::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15915918::DEBUG::2013-11-20 04:26:44,136::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:26:44,137::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:26:44,138::task::495::TaskManager.Task::(_debug) Task 0e693f3a-aadc-48d3-829b-759dd6bc57f5: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:26:44,139::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:26:44,369::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0802984 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:26:46,385::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:26:46,833::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0584389 s, 17.5 MB/s\n'; <rc> = 0
>Thread-15916418::INFO::2013-11-20 04:26:48,309::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916418::DEBUG::2013-11-20 04:26:48,310::task::495::TaskManager.Task::(_debug) Task 0ef29e01-afe0-4cbf-b26a-379acb1aae32: moving from state init -> state preparing
>Thread-15916418::DEBUG::2013-11-20 04:26:48,312::task::495::TaskManager.Task::(_debug) Task 0ef29e01-afe0-4cbf-b26a-379acb1aae32: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000570774078369', 'lastCheck': 1384921608.0995059, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0020899772644', 'lastCheck': 1384921606.610044, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0212831497192', 'lastCheck': 1384921603.6233039, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00054407119751', 'lastCheck': 1384921603.0894279, 'valid': True, 'code': 0}}
>Thread-15916418::DEBUG::2013-11-20 04:26:48,313::task::495::TaskManager.Task::(_debug) Task 0ef29e01-afe0-4cbf-b26a-379acb1aae32: moving from state preparing -> state finished
>Thread-15916418::DEBUG::2013-11-20 04:26:48,314::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916418::DEBUG::2013-11-20 04:26:48,314::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916418::DEBUG::2013-11-20 04:26:48,315::task::495::TaskManager.Task::(_debug) Task 0ef29e01-afe0-4cbf-b26a-379acb1aae32: ref 0 aborting False
>Thread-15916418::INFO::2013-11-20 04:26:48,316::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000570774078369', 'lastCheck': 1384921608.0995059, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0020899772644', 'lastCheck': 1384921606.610044, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0212831497192', 'lastCheck': 1384921603.6233039, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00054407119751', 'lastCheck': 1384921603.0894279, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:26:48,846::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915863::INFO::2013-11-20 04:26:49,113::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:26:49,115::task::495::TaskManager.Task::(_debug) Task 5ab4b9ad-1429-49dc-9907-3967b236ecd7: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:26:49,124::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`29997d56-730a-4cd2-91a5-960b3e77214c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:26:49,125::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:26:49,126::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:26:49,127::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`29997d56-730a-4cd2-91a5-960b3e77214c`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:26:49,128::task::495::TaskManager.Task::(_debug) Task 5ab4b9ad-1429-49dc-9907-3967b236ecd7: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:26:49,129::task::495::TaskManager.Task::(_debug) Task 5ab4b9ad-1429-49dc-9907-3967b236ecd7: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:26:49,131::task::495::TaskManager.Task::(_debug) Task 5ab4b9ad-1429-49dc-9907-3967b236ecd7: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:26:49,132::task::495::TaskManager.Task::(_debug) Task 5ab4b9ad-1429-49dc-9907-3967b236ecd7: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:26:49,132::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:26:49,133::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:26:49,134::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:26:49,134::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:26:49,135::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:26:49,136::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:26:49,136::task::495::TaskManager.Task::(_debug) Task 5ab4b9ad-1429-49dc-9907-3967b236ecd7: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:26:49,137::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:26:49,416::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0750124 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:26:51,431::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:26:51,878::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0604046 s, 17.0 MB/s\n'; <rc> = 0
>Thread-15916421::DEBUG::2013-11-20 04:26:52,609::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916421::INFO::2013-11-20 04:26:52,611::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916421::DEBUG::2013-11-20 04:26:52,612::task::495::TaskManager.Task::(_debug) Task c3e5b669-0311-49d0-97d7-1e53f67121ec: moving from state init -> state preparing
>Thread-15916421::DEBUG::2013-11-20 04:26:52,613::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916421::DEBUG::2013-11-20 04:26:52,613::task::495::TaskManager.Task::(_debug) Task c3e5b669-0311-49d0-97d7-1e53f67121ec: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916421::DEBUG::2013-11-20 04:26:52,614::task::495::TaskManager.Task::(_debug) Task c3e5b669-0311-49d0-97d7-1e53f67121ec: moving from state preparing -> state finished
>Thread-15916421::DEBUG::2013-11-20 04:26:52,615::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916421::DEBUG::2013-11-20 04:26:52,615::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916421::DEBUG::2013-11-20 04:26:52,616::task::495::TaskManager.Task::(_debug) Task c3e5b669-0311-49d0-97d7-1e53f67121ec: ref 0 aborting False
>Thread-15916421::INFO::2013-11-20 04:26:52,616::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916422::DEBUG::2013-11-20 04:26:52,621::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916422::INFO::2013-11-20 04:26:52,622::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916422::DEBUG::2013-11-20 04:26:52,623::task::495::TaskManager.Task::(_debug) Task 6a91d1b6-e875-4ef4-aef6-5d188154913b: moving from state init -> state preparing
>Thread-15916422::DEBUG::2013-11-20 04:26:52,625::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`677760ef-357a-4067-a0fe-5b3724292737`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916422::DEBUG::2013-11-20 04:26:52,626::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916422::DEBUG::2013-11-20 04:26:52,627::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916422::DEBUG::2013-11-20 04:26:52,627::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`677760ef-357a-4067-a0fe-5b3724292737`::Granted request
>Thread-15916422::DEBUG::2013-11-20 04:26:52,629::task::495::TaskManager.Task::(_debug) Task 6a91d1b6-e875-4ef4-aef6-5d188154913b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916422::DEBUG::2013-11-20 04:26:52,630::task::495::TaskManager.Task::(_debug) Task 6a91d1b6-e875-4ef4-aef6-5d188154913b: ref 1 aborting False
>Thread-15916422::WARNING::2013-11-20 04:26:52,635::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916422::DEBUG::2013-11-20 04:26:52,637::task::495::TaskManager.Task::(_debug) Task 6a91d1b6-e875-4ef4-aef6-5d188154913b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916422::DEBUG::2013-11-20 04:26:52,638::task::495::TaskManager.Task::(_debug) Task 6a91d1b6-e875-4ef4-aef6-5d188154913b: moving from state preparing -> state finished
>Thread-15916422::DEBUG::2013-11-20 04:26:52,639::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916422::DEBUG::2013-11-20 04:26:52,640::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916422::DEBUG::2013-11-20 04:26:52,641::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916422::DEBUG::2013-11-20 04:26:52,642::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916422::DEBUG::2013-11-20 04:26:52,643::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916422::DEBUG::2013-11-20 04:26:52,643::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916422::DEBUG::2013-11-20 04:26:52,644::task::495::TaskManager.Task::(_debug) Task 6a91d1b6-e875-4ef4-aef6-5d188154913b: ref 0 aborting False
>Thread-15916422::INFO::2013-11-20 04:26:52,644::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:26:53,894::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:26:54,444::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.119418 s, 8.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:26:56,455::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 04:26:56,688::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:26:56,693::task::495::TaskManager.Task::(_debug) Task f60b214f-cf61-475b-b01f-624ef542eb5c: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:26:56,696::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ecc09f58-5383-49f7-866e-576775a888b8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:26:56,697::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:26:56,699::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:26:56,699::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ecc09f58-5383-49f7-866e-576775a888b8`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:26:56,700::task::495::TaskManager.Task::(_debug) Task f60b214f-cf61-475b-b01f-624ef542eb5c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:26:56,701::task::495::TaskManager.Task::(_debug) Task f60b214f-cf61-475b-b01f-624ef542eb5c: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:26:56,701::task::495::TaskManager.Task::(_debug) Task f60b214f-cf61-475b-b01f-624ef542eb5c: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:26:56,702::task::495::TaskManager.Task::(_debug) Task f60b214f-cf61-475b-b01f-624ef542eb5c: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:26:56,703::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:26:56,703::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:26:56,704::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:26:56,704::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:26:56,705::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:26:56,705::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:26:56,706::task::495::TaskManager.Task::(_debug) Task f60b214f-cf61-475b-b01f-624ef542eb5c: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:26:56,707::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913518::INFO::2013-11-20 04:26:56,777::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:26:56,778::task::495::TaskManager.Task::(_debug) Task 6e625961-8dfe-4cdc-aaf5-3cc77ec5165e: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:26:56,779::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f5e2077a-db01-4b6e-9c8d-cbe97fafd69e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:26:56,780::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:26:56,781::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:26:56,782::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f5e2077a-db01-4b6e-9c8d-cbe97fafd69e`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:26:56,783::task::495::TaskManager.Task::(_debug) Task 6e625961-8dfe-4cdc-aaf5-3cc77ec5165e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:26:56,784::task::495::TaskManager.Task::(_debug) Task 6e625961-8dfe-4cdc-aaf5-3cc77ec5165e: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:26:56,784::task::495::TaskManager.Task::(_debug) Task 6e625961-8dfe-4cdc-aaf5-3cc77ec5165e: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:26:56,785::task::495::TaskManager.Task::(_debug) Task 6e625961-8dfe-4cdc-aaf5-3cc77ec5165e: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:26:56,786::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:26:56,787::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:26:56,787::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:26:56,788::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:26:56,789::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:26:56,789::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:26:56,790::task::495::TaskManager.Task::(_debug) Task 6e625961-8dfe-4cdc-aaf5-3cc77ec5165e: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:26:56,791::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:26:56,792::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:26:56,793::task::495::TaskManager.Task::(_debug) Task 99cd9aed-c09e-496e-a542-aad2e350626e: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:26:56,794::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a4e5c3f3-a1e4-45a4-9fe9-6e1fb7afaffd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:26:56,795::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:26:56,795::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:26:56,796::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a4e5c3f3-a1e4-45a4-9fe9-6e1fb7afaffd`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:26:56,797::task::495::TaskManager.Task::(_debug) Task 99cd9aed-c09e-496e-a542-aad2e350626e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:26:56,798::task::495::TaskManager.Task::(_debug) Task 99cd9aed-c09e-496e-a542-aad2e350626e: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:26:56,799::task::495::TaskManager.Task::(_debug) Task 99cd9aed-c09e-496e-a542-aad2e350626e: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:26:56,799::task::495::TaskManager.Task::(_debug) Task 99cd9aed-c09e-496e-a542-aad2e350626e: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:26:56,800::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:26:56,801::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:26:56,801::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:26:56,802::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:26:56,803::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:26:56,803::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:26:56,804::task::495::TaskManager.Task::(_debug) Task 99cd9aed-c09e-496e-a542-aad2e350626e: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:26:56,804::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:26:56,997::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0744535 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:26:59,013::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:26:59,516::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0782312 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:27:01,532::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:27:02,067::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0821766 s, 12.5 MB/s\n'; <rc> = 0
>Thread-15916426::DEBUG::2013-11-20 04:27:03,055::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916426::INFO::2013-11-20 04:27:03,057::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916426::DEBUG::2013-11-20 04:27:03,057::task::495::TaskManager.Task::(_debug) Task 9d99adfd-d98b-41cb-8912-e06f7a2e353f: moving from state init -> state preparing
>Thread-15916426::DEBUG::2013-11-20 04:27:03,058::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916426::DEBUG::2013-11-20 04:27:03,059::task::495::TaskManager.Task::(_debug) Task 9d99adfd-d98b-41cb-8912-e06f7a2e353f: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916426::DEBUG::2013-11-20 04:27:03,060::task::495::TaskManager.Task::(_debug) Task 9d99adfd-d98b-41cb-8912-e06f7a2e353f: moving from state preparing -> state finished
>Thread-15916426::DEBUG::2013-11-20 04:27:03,060::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916426::DEBUG::2013-11-20 04:27:03,061::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916426::DEBUG::2013-11-20 04:27:03,062::task::495::TaskManager.Task::(_debug) Task 9d99adfd-d98b-41cb-8912-e06f7a2e353f: ref 0 aborting False
>Thread-15916426::INFO::2013-11-20 04:27:03,062::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916427::DEBUG::2013-11-20 04:27:03,067::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916427::INFO::2013-11-20 04:27:03,068::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916427::DEBUG::2013-11-20 04:27:03,069::task::495::TaskManager.Task::(_debug) Task bf2ddec7-c68e-4be9-af52-e68b3cdea9f6: moving from state init -> state preparing
>Thread-15916427::DEBUG::2013-11-20 04:27:03,070::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9f179bcc-ae2e-45b1-b0d1-cb84a2c2293b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916427::DEBUG::2013-11-20 04:27:03,071::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916427::DEBUG::2013-11-20 04:27:03,071::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916427::DEBUG::2013-11-20 04:27:03,072::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9f179bcc-ae2e-45b1-b0d1-cb84a2c2293b`::Granted request
>Thread-15916427::DEBUG::2013-11-20 04:27:03,073::task::495::TaskManager.Task::(_debug) Task bf2ddec7-c68e-4be9-af52-e68b3cdea9f6: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916427::DEBUG::2013-11-20 04:27:03,074::task::495::TaskManager.Task::(_debug) Task bf2ddec7-c68e-4be9-af52-e68b3cdea9f6: ref 1 aborting False
>Thread-15916427::WARNING::2013-11-20 04:27:03,076::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916427::DEBUG::2013-11-20 04:27:03,076::task::495::TaskManager.Task::(_debug) Task bf2ddec7-c68e-4be9-af52-e68b3cdea9f6: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916427::DEBUG::2013-11-20 04:27:03,077::task::495::TaskManager.Task::(_debug) Task bf2ddec7-c68e-4be9-af52-e68b3cdea9f6: moving from state preparing -> state finished
>Thread-15916427::DEBUG::2013-11-20 04:27:03,078::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916427::DEBUG::2013-11-20 04:27:03,079::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916427::DEBUG::2013-11-20 04:27:03,079::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916427::DEBUG::2013-11-20 04:27:03,080::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916427::DEBUG::2013-11-20 04:27:03,081::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916427::DEBUG::2013-11-20 04:27:03,081::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916427::DEBUG::2013-11-20 04:27:03,082::task::495::TaskManager.Task::(_debug) Task bf2ddec7-c68e-4be9-af52-e68b3cdea9f6: ref 0 aborting False
>Thread-15916427::INFO::2013-11-20 04:27:03,082::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916428::INFO::2013-11-20 04:27:03,991::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916428::DEBUG::2013-11-20 04:27:03,992::task::495::TaskManager.Task::(_debug) Task 004b7f2d-162a-43cd-9fa1-9a1257900027: moving from state init -> state preparing
>Thread-15916428::DEBUG::2013-11-20 04:27:03,993::task::495::TaskManager.Task::(_debug) Task 004b7f2d-162a-43cd-9fa1-9a1257900027: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00043797492981', 'lastCheck': 1384921618.1111879, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00174689292908', 'lastCheck': 1384921616.698756, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00169920921326', 'lastCheck': 1384921623.7327819, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000699996948242', 'lastCheck': 1384921623.2313621, 'valid': True, 'code': 0}}
>Thread-15916428::DEBUG::2013-11-20 04:27:03,994::task::495::TaskManager.Task::(_debug) Task 004b7f2d-162a-43cd-9fa1-9a1257900027: moving from state preparing -> state finished
>Thread-15916428::DEBUG::2013-11-20 04:27:03,995::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916428::DEBUG::2013-11-20 04:27:03,995::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916428::DEBUG::2013-11-20 04:27:03,996::task::495::TaskManager.Task::(_debug) Task 004b7f2d-162a-43cd-9fa1-9a1257900027: ref 0 aborting False
>Thread-15916428::INFO::2013-11-20 04:27:03,997::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00043797492981', 'lastCheck': 1384921618.1111879, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00174689292908', 'lastCheck': 1384921616.698756, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00169920921326', 'lastCheck': 1384921623.7327819, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000699996948242', 'lastCheck': 1384921623.2313621, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:27:04,077::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:27:04,634::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0812606 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:27:06,649::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 04:27:06,869::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:27:06,887::task::495::TaskManager.Task::(_debug) Task 7377fed2-5486-4426-99fc-efaf648337e0: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:27:06,891::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`dc223b1c-3418-408a-895e-e076eaa4928e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:27:06,893::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:27:06,894::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:27:06,894::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`dc223b1c-3418-408a-895e-e076eaa4928e`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:27:06,895::task::495::TaskManager.Task::(_debug) Task 7377fed2-5486-4426-99fc-efaf648337e0: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:27:06,896::task::495::TaskManager.Task::(_debug) Task 7377fed2-5486-4426-99fc-efaf648337e0: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:27:06,897::task::495::TaskManager.Task::(_debug) Task 7377fed2-5486-4426-99fc-efaf648337e0: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:27:06,897::task::495::TaskManager.Task::(_debug) Task 7377fed2-5486-4426-99fc-efaf648337e0: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:27:06,898::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:27:06,898::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:27:06,899::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:27:06,899::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:27:06,900::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:27:06,900::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:27:06,901::task::495::TaskManager.Task::(_debug) Task 7377fed2-5486-4426-99fc-efaf648337e0: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:27:06,902::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:27:07,199::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0816164 s, 12.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:27:09,214::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:27:09,789::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0870654 s, 11.8 MB/s\n'; <rc> = 0
>Thread-15916432::DEBUG::2013-11-20 04:27:11,361::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Dummy-15912651::DEBUG::2013-11-20 04:27:11,805::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916432::DEBUG::2013-11-20 04:27:12,034::clientIF::1219::vds::(fenceNode) rc 0 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = On
>Done
> err 
>Thread-15913695::INFO::2013-11-20 04:27:12,052::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:27:12,054::task::495::TaskManager.Task::(_debug) Task c2b4763a-aff5-43b5-9ab0-380db584c32f: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:27:12,058::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9745bd27-3ef4-4a34-8470-2b0a771e1503`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:27:12,060::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:27:12,063::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:27:12,063::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9745bd27-3ef4-4a34-8470-2b0a771e1503`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:27:12,064::task::495::TaskManager.Task::(_debug) Task c2b4763a-aff5-43b5-9ab0-380db584c32f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:27:12,065::task::495::TaskManager.Task::(_debug) Task c2b4763a-aff5-43b5-9ab0-380db584c32f: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:27:12,065::task::495::TaskManager.Task::(_debug) Task c2b4763a-aff5-43b5-9ab0-380db584c32f: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:27:12,066::task::495::TaskManager.Task::(_debug) Task c2b4763a-aff5-43b5-9ab0-380db584c32f: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:27:12,066::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:27:12,067::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:27:12,068::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:27:12,068::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:27:12,069::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:27:12,069::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:27:12,070::task::495::TaskManager.Task::(_debug) Task c2b4763a-aff5-43b5-9ab0-380db584c32f: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:27:12,071::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15916433::DEBUG::2013-11-20 04:27:12,095::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Dummy-15912651::DEBUG::2013-11-20 04:27:12,607::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.258894 s, 4.0 MB/s\n'; <rc> = 0
>Thread-15916433::DEBUG::2013-11-20 04:27:12,804::clientIF::1219::vds::(fenceNode) rc 0 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = On
>Done
> err 
>Thread-15916434::DEBUG::2013-11-20 04:27:13,510::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916434::INFO::2013-11-20 04:27:13,513::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916434::DEBUG::2013-11-20 04:27:13,514::task::495::TaskManager.Task::(_debug) Task e48501e1-0822-4807-8906-dccff72a43e6: moving from state init -> state preparing
>Thread-15916434::DEBUG::2013-11-20 04:27:13,515::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916434::DEBUG::2013-11-20 04:27:13,516::task::495::TaskManager.Task::(_debug) Task e48501e1-0822-4807-8906-dccff72a43e6: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916434::DEBUG::2013-11-20 04:27:13,516::task::495::TaskManager.Task::(_debug) Task e48501e1-0822-4807-8906-dccff72a43e6: moving from state preparing -> state finished
>Thread-15916434::DEBUG::2013-11-20 04:27:13,517::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916434::DEBUG::2013-11-20 04:27:13,517::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916434::DEBUG::2013-11-20 04:27:13,518::task::495::TaskManager.Task::(_debug) Task e48501e1-0822-4807-8906-dccff72a43e6: ref 0 aborting False
>Thread-15916434::INFO::2013-11-20 04:27:13,519::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916436::DEBUG::2013-11-20 04:27:13,549::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916436::INFO::2013-11-20 04:27:13,550::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916436::DEBUG::2013-11-20 04:27:13,550::task::495::TaskManager.Task::(_debug) Task cc754e89-0a1a-4d61-8b41-92aa16879332: moving from state init -> state preparing
>Thread-15916436::DEBUG::2013-11-20 04:27:13,552::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`fdf082e0-3ab6-44fc-96e6-9917a113f10d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916436::DEBUG::2013-11-20 04:27:13,553::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916436::DEBUG::2013-11-20 04:27:13,554::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916436::DEBUG::2013-11-20 04:27:13,554::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`fdf082e0-3ab6-44fc-96e6-9917a113f10d`::Granted request
>Thread-15916436::DEBUG::2013-11-20 04:27:13,555::task::495::TaskManager.Task::(_debug) Task cc754e89-0a1a-4d61-8b41-92aa16879332: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916436::DEBUG::2013-11-20 04:27:13,556::task::495::TaskManager.Task::(_debug) Task cc754e89-0a1a-4d61-8b41-92aa16879332: ref 1 aborting False
>Thread-15916436::WARNING::2013-11-20 04:27:13,558::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916436::DEBUG::2013-11-20 04:27:13,567::task::495::TaskManager.Task::(_debug) Task cc754e89-0a1a-4d61-8b41-92aa16879332: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916436::DEBUG::2013-11-20 04:27:13,568::task::495::TaskManager.Task::(_debug) Task cc754e89-0a1a-4d61-8b41-92aa16879332: moving from state preparing -> state finished
>Thread-15916436::DEBUG::2013-11-20 04:27:13,569::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916436::DEBUG::2013-11-20 04:27:13,570::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916436::DEBUG::2013-11-20 04:27:13,571::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916436::DEBUG::2013-11-20 04:27:13,571::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916436::DEBUG::2013-11-20 04:27:13,572::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916436::DEBUG::2013-11-20 04:27:13,572::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916436::DEBUG::2013-11-20 04:27:13,573::task::495::TaskManager.Task::(_debug) Task cc754e89-0a1a-4d61-8b41-92aa16879332: ref 0 aborting False
>Thread-15916436::INFO::2013-11-20 04:27:13,574::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:27:14,626::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:27:15,190::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0986865 s, 10.4 MB/s\n'; <rc> = 0
>Thread-15914453::INFO::2013-11-20 04:27:17,055::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:27:17,056::task::495::TaskManager.Task::(_debug) Task dd94d7e7-0a21-494f-b087-0fa750723395: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:27:17,057::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`df9aeb67-efaf-4f84-97cb-67a1652a026e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:27:17,057::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:27:17,058::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:27:17,058::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`df9aeb67-efaf-4f84-97cb-67a1652a026e`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:27:17,059::task::495::TaskManager.Task::(_debug) Task dd94d7e7-0a21-494f-b087-0fa750723395: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:27:17,059::task::495::TaskManager.Task::(_debug) Task dd94d7e7-0a21-494f-b087-0fa750723395: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:27:17,060::task::495::TaskManager.Task::(_debug) Task dd94d7e7-0a21-494f-b087-0fa750723395: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:27:17,060::task::495::TaskManager.Task::(_debug) Task dd94d7e7-0a21-494f-b087-0fa750723395: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:27:17,061::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:27:17,062::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:27:17,063::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:27:17,063::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:27:17,063::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:27:17,064::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:27:17,065::task::495::TaskManager.Task::(_debug) Task dd94d7e7-0a21-494f-b087-0fa750723395: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:27:17,065::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:27:17,203::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:27:17,682::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0871287 s, 11.8 MB/s\n'; <rc> = 0
>Thread-15916438::INFO::2013-11-20 04:27:19,582::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916438::DEBUG::2013-11-20 04:27:19,583::task::495::TaskManager.Task::(_debug) Task c1becebd-f8fb-4360-b4ad-e95694ab3afd: moving from state init -> state preparing
>Thread-15916438::DEBUG::2013-11-20 04:27:19,584::task::495::TaskManager.Task::(_debug) Task c1becebd-f8fb-4360-b4ad-e95694ab3afd: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000383138656616', 'lastCheck': 1384921638.5948901, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00151205062866', 'lastCheck': 1384921637.0218971, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00152611732483', 'lastCheck': 1384921634.0173421, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000379800796509', 'lastCheck': 1384921633.5898299, 'valid': True, 'code': 0}}
>Thread-15916438::DEBUG::2013-11-20 04:27:19,595::task::495::TaskManager.Task::(_debug) Task c1becebd-f8fb-4360-b4ad-e95694ab3afd: moving from state preparing -> state finished
>Thread-15916438::DEBUG::2013-11-20 04:27:19,598::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916438::DEBUG::2013-11-20 04:27:19,599::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916438::DEBUG::2013-11-20 04:27:19,600::task::495::TaskManager.Task::(_debug) Task c1becebd-f8fb-4360-b4ad-e95694ab3afd: ref 0 aborting False
>Thread-15916438::INFO::2013-11-20 04:27:19,606::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000383138656616', 'lastCheck': 1384921638.5948901, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00151205062866', 'lastCheck': 1384921637.0218971, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00152611732483', 'lastCheck': 1384921634.0173421, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000379800796509', 'lastCheck': 1384921633.5898299, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:27:19,700::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:27:20,240::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0808558 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:27:22,255::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:27:22,818::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0837407 s, 12.2 MB/s\n'; <rc> = 0
>Thread-15916441::DEBUG::2013-11-20 04:27:24,008::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916441::INFO::2013-11-20 04:27:24,010::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916441::DEBUG::2013-11-20 04:27:24,011::task::495::TaskManager.Task::(_debug) Task e0659593-55af-4e7f-8ce2-88ed2cef86c8: moving from state init -> state preparing
>Thread-15916441::DEBUG::2013-11-20 04:27:24,013::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916441::DEBUG::2013-11-20 04:27:24,013::task::495::TaskManager.Task::(_debug) Task e0659593-55af-4e7f-8ce2-88ed2cef86c8: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916441::DEBUG::2013-11-20 04:27:24,015::task::495::TaskManager.Task::(_debug) Task e0659593-55af-4e7f-8ce2-88ed2cef86c8: moving from state preparing -> state finished
>Thread-15916441::DEBUG::2013-11-20 04:27:24,015::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916441::DEBUG::2013-11-20 04:27:24,016::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916441::DEBUG::2013-11-20 04:27:24,017::task::495::TaskManager.Task::(_debug) Task e0659593-55af-4e7f-8ce2-88ed2cef86c8: ref 0 aborting False
>Thread-15916441::INFO::2013-11-20 04:27:24,018::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916442::DEBUG::2013-11-20 04:27:24,045::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916442::INFO::2013-11-20 04:27:24,052::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916442::DEBUG::2013-11-20 04:27:24,054::task::495::TaskManager.Task::(_debug) Task faeeb859-bafc-4fde-850e-111fda516aac: moving from state init -> state preparing
>Thread-15916442::DEBUG::2013-11-20 04:27:24,057::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7a1cc040-4827-44e4-b698-13dc95083508`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916442::DEBUG::2013-11-20 04:27:24,060::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916442::DEBUG::2013-11-20 04:27:24,063::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916442::DEBUG::2013-11-20 04:27:24,065::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7a1cc040-4827-44e4-b698-13dc95083508`::Granted request
>Thread-15916442::DEBUG::2013-11-20 04:27:24,066::task::495::TaskManager.Task::(_debug) Task faeeb859-bafc-4fde-850e-111fda516aac: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916442::DEBUG::2013-11-20 04:27:24,067::task::495::TaskManager.Task::(_debug) Task faeeb859-bafc-4fde-850e-111fda516aac: ref 1 aborting False
>Thread-15916442::WARNING::2013-11-20 04:27:24,069::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916442::DEBUG::2013-11-20 04:27:24,070::task::495::TaskManager.Task::(_debug) Task faeeb859-bafc-4fde-850e-111fda516aac: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916442::DEBUG::2013-11-20 04:27:24,071::task::495::TaskManager.Task::(_debug) Task faeeb859-bafc-4fde-850e-111fda516aac: moving from state preparing -> state finished
>Thread-15916442::DEBUG::2013-11-20 04:27:24,072::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916442::DEBUG::2013-11-20 04:27:24,073::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916442::DEBUG::2013-11-20 04:27:24,074::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916442::DEBUG::2013-11-20 04:27:24,074::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916442::DEBUG::2013-11-20 04:27:24,075::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916442::DEBUG::2013-11-20 04:27:24,076::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916442::DEBUG::2013-11-20 04:27:24,076::task::495::TaskManager.Task::(_debug) Task faeeb859-bafc-4fde-850e-111fda516aac: ref 0 aborting False
>Thread-15916442::INFO::2013-11-20 04:27:24,077::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914519::INFO::2013-11-20 04:27:24,559::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:27:24,568::task::495::TaskManager.Task::(_debug) Task 0d78faf2-9d22-4d0f-bfcd-52026554d255: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:27:24,577::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`76e2a9e1-db1e-4816-af32-ae9c5f33572e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:27:24,579::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:27:24,580::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:27:24,581::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`76e2a9e1-db1e-4816-af32-ae9c5f33572e`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:27:24,582::task::495::TaskManager.Task::(_debug) Task 0d78faf2-9d22-4d0f-bfcd-52026554d255: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 04:27:24,585::task::495::TaskManager.Task::(_debug) Task 0d78faf2-9d22-4d0f-bfcd-52026554d255: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:27:24,586::task::495::TaskManager.Task::(_debug) Task 0d78faf2-9d22-4d0f-bfcd-52026554d255: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 04:27:24,587::task::495::TaskManager.Task::(_debug) Task 0d78faf2-9d22-4d0f-bfcd-52026554d255: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:27:24,588::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:27:24,588::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:27:24,592::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 04:27:24,593::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:27:24,594::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:27:24,594::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:27:24,595::task::495::TaskManager.Task::(_debug) Task 0d78faf2-9d22-4d0f-bfcd-52026554d255: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 04:27:24,596::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 04:27:24,828::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:27:25,436::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0941938 s, 10.9 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 04:27:27,091::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:27:27,092::task::495::TaskManager.Task::(_debug) Task 6d0d132e-fe45-42e0-abc7-db2686e0a046: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:27:27,094::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a49b5760-87f7-435b-85d7-6e8cc3feb76e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:27:27,095::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:27:27,096::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:27:27,097::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a49b5760-87f7-435b-85d7-6e8cc3feb76e`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:27:27,099::task::495::TaskManager.Task::(_debug) Task 6d0d132e-fe45-42e0-abc7-db2686e0a046: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:27:27,099::task::495::TaskManager.Task::(_debug) Task 6d0d132e-fe45-42e0-abc7-db2686e0a046: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:27:27,100::task::495::TaskManager.Task::(_debug) Task 6d0d132e-fe45-42e0-abc7-db2686e0a046: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 04:27:27,101::task::495::TaskManager.Task::(_debug) Task 6d0d132e-fe45-42e0-abc7-db2686e0a046: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:27:27,101::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:27:27,102::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:27:27,103::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:27:27,104::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:27:27,105::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:27:27,105::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:27:27,106::task::495::TaskManager.Task::(_debug) Task 6d0d132e-fe45-42e0-abc7-db2686e0a046: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:27:27,107::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:27:27,108::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:27:27,108::task::495::TaskManager.Task::(_debug) Task bf7e2392-2161-447e-acd6-f185594884fa: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:27:27,110::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`4bd9b9b0-6537-4a29-a3bf-561dc041727a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:27:27,110::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:27:27,111::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:27:27,111::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`4bd9b9b0-6537-4a29-a3bf-561dc041727a`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:27:27,112::task::495::TaskManager.Task::(_debug) Task bf7e2392-2161-447e-acd6-f185594884fa: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:27:27,113::task::495::TaskManager.Task::(_debug) Task bf7e2392-2161-447e-acd6-f185594884fa: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:27:27,113::task::495::TaskManager.Task::(_debug) Task bf7e2392-2161-447e-acd6-f185594884fa: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:27:27,114::task::495::TaskManager.Task::(_debug) Task bf7e2392-2161-447e-acd6-f185594884fa: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:27:27,115::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:27:27,115::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:27:27,116::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:27:27,116::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:27:27,117::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:27:27,117::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:27:27,118::task::495::TaskManager.Task::(_debug) Task bf7e2392-2161-447e-acd6-f185594884fa: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:27:27,119::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:27:27,119::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:27:27,120::task::495::TaskManager.Task::(_debug) Task c0880e33-4464-4a58-9fcc-feccc3611cdf: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:27:27,121::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8d3a760e-fa53-4c41-aa66-32fb0431a663`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:27:27,122::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:27:27,122::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:27:27,123::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8d3a760e-fa53-4c41-aa66-32fb0431a663`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:27:27,124::task::495::TaskManager.Task::(_debug) Task c0880e33-4464-4a58-9fcc-feccc3611cdf: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:27:27,124::task::495::TaskManager.Task::(_debug) Task c0880e33-4464-4a58-9fcc-feccc3611cdf: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:27:27,125::task::495::TaskManager.Task::(_debug) Task c0880e33-4464-4a58-9fcc-feccc3611cdf: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:27:27,126::task::495::TaskManager.Task::(_debug) Task c0880e33-4464-4a58-9fcc-feccc3611cdf: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:27:27,126::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:27:27,127::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:27:27,128::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:27:27,128::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:27:27,129::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:27:27,130::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:27:27,130::task::495::TaskManager.Task::(_debug) Task c0880e33-4464-4a58-9fcc-feccc3611cdf: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:27:27,131::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:27:27,132::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:27:27,132::task::495::TaskManager.Task::(_debug) Task f9d0ea18-cece-4702-a264-3d2a6d4afed5: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:27:27,133::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`9b38e70b-e12f-4b88-92fa-a4537858b6dd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:27:27,134::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:27:27,135::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:27:27,135::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`9b38e70b-e12f-4b88-92fa-a4537858b6dd`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:27:27,136::task::495::TaskManager.Task::(_debug) Task f9d0ea18-cece-4702-a264-3d2a6d4afed5: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:27:27,137::task::495::TaskManager.Task::(_debug) Task f9d0ea18-cece-4702-a264-3d2a6d4afed5: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:27:27,137::task::495::TaskManager.Task::(_debug) Task f9d0ea18-cece-4702-a264-3d2a6d4afed5: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:27:27,138::task::495::TaskManager.Task::(_debug) Task f9d0ea18-cece-4702-a264-3d2a6d4afed5: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:27:27,138::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:27:27,139::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:27:27,140::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:27:27,141::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:27:27,141::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:27:27,142::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:27:27,143::task::495::TaskManager.Task::(_debug) Task f9d0ea18-cece-4702-a264-3d2a6d4afed5: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:27:27,143::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:27:27,452::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 04:27:27,703::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:27:27,705::task::495::TaskManager.Task::(_debug) Task 408a21e3-44a2-4e17-a4b5-b5fc6e395ab6: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:27:27,707::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`dd1ccaf7-88ea-43f4-aa6e-5a9777e6317d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:27:27,708::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:27:27,709::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:27:27,710::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`dd1ccaf7-88ea-43f4-aa6e-5a9777e6317d`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:27:27,711::task::495::TaskManager.Task::(_debug) Task 408a21e3-44a2-4e17-a4b5-b5fc6e395ab6: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:27:27,711::task::495::TaskManager.Task::(_debug) Task 408a21e3-44a2-4e17-a4b5-b5fc6e395ab6: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:27:27,712::task::495::TaskManager.Task::(_debug) Task 408a21e3-44a2-4e17-a4b5-b5fc6e395ab6: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:27:27,713::task::495::TaskManager.Task::(_debug) Task 408a21e3-44a2-4e17-a4b5-b5fc6e395ab6: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:27:27,714::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:27:27,714::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:27:27,715::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:27:27,715::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:27:27,716::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:27:27,716::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:27:27,717::task::495::TaskManager.Task::(_debug) Task 408a21e3-44a2-4e17-a4b5-b5fc6e395ab6: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:27:27,718::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:27:28,005::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0850541 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:27:30,021::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:27:30,551::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0764887 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:27:32,566::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:27:33,112::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0890334 s, 11.5 MB/s\n'; <rc> = 0
>Thread-15916446::DEBUG::2013-11-20 04:27:34,483::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916446::INFO::2013-11-20 04:27:34,484::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916446::DEBUG::2013-11-20 04:27:34,485::task::495::TaskManager.Task::(_debug) Task 2d5b071e-653e-45ce-bb7b-633bed8d2b6c: moving from state init -> state preparing
>Thread-15916446::DEBUG::2013-11-20 04:27:34,495::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916446::DEBUG::2013-11-20 04:27:34,496::task::495::TaskManager.Task::(_debug) Task 2d5b071e-653e-45ce-bb7b-633bed8d2b6c: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916446::DEBUG::2013-11-20 04:27:34,496::task::495::TaskManager.Task::(_debug) Task 2d5b071e-653e-45ce-bb7b-633bed8d2b6c: moving from state preparing -> state finished
>Thread-15916446::DEBUG::2013-11-20 04:27:34,497::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916446::DEBUG::2013-11-20 04:27:34,497::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916446::DEBUG::2013-11-20 04:27:34,498::task::495::TaskManager.Task::(_debug) Task 2d5b071e-653e-45ce-bb7b-633bed8d2b6c: ref 0 aborting False
>Thread-15916446::INFO::2013-11-20 04:27:34,499::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916447::DEBUG::2013-11-20 04:27:34,503::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916447::INFO::2013-11-20 04:27:34,504::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916447::DEBUG::2013-11-20 04:27:34,505::task::495::TaskManager.Task::(_debug) Task e3aa5e05-271f-400e-b95c-e457f321b540: moving from state init -> state preparing
>Thread-15916447::DEBUG::2013-11-20 04:27:34,513::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4b8c8a42-f872-47b1-ae6f-d101898301dc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916447::DEBUG::2013-11-20 04:27:34,514::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916447::DEBUG::2013-11-20 04:27:34,515::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916447::DEBUG::2013-11-20 04:27:34,515::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4b8c8a42-f872-47b1-ae6f-d101898301dc`::Granted request
>Thread-15916447::DEBUG::2013-11-20 04:27:34,517::task::495::TaskManager.Task::(_debug) Task e3aa5e05-271f-400e-b95c-e457f321b540: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916447::DEBUG::2013-11-20 04:27:34,518::task::495::TaskManager.Task::(_debug) Task e3aa5e05-271f-400e-b95c-e457f321b540: ref 1 aborting False
>Thread-15916447::WARNING::2013-11-20 04:27:34,520::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916447::DEBUG::2013-11-20 04:27:34,521::task::495::TaskManager.Task::(_debug) Task e3aa5e05-271f-400e-b95c-e457f321b540: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916447::DEBUG::2013-11-20 04:27:34,522::task::495::TaskManager.Task::(_debug) Task e3aa5e05-271f-400e-b95c-e457f321b540: moving from state preparing -> state finished
>Thread-15916447::DEBUG::2013-11-20 04:27:34,523::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916447::DEBUG::2013-11-20 04:27:34,523::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916447::DEBUG::2013-11-20 04:27:34,524::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916447::DEBUG::2013-11-20 04:27:34,525::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916447::DEBUG::2013-11-20 04:27:34,525::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916447::DEBUG::2013-11-20 04:27:34,526::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916447::DEBUG::2013-11-20 04:27:34,527::task::495::TaskManager.Task::(_debug) Task e3aa5e05-271f-400e-b95c-e457f321b540: ref 0 aborting False
>Thread-15916447::INFO::2013-11-20 04:27:34,527::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:27:35,127::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916448::INFO::2013-11-20 04:27:35,393::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916448::DEBUG::2013-11-20 04:27:35,394::task::495::TaskManager.Task::(_debug) Task f09e3e93-83f3-4bc6-bd91-1678c7bbbdaf: moving from state init -> state preparing
>Thread-15916448::DEBUG::2013-11-20 04:27:35,396::task::495::TaskManager.Task::(_debug) Task f09e3e93-83f3-4bc6-bd91-1678c7bbbdaf: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000374794006348', 'lastCheck': 1384921648.6418791, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00190806388855', 'lastCheck': 1384921647.0807209, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00172400474548', 'lastCheck': 1384921654.24876, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000450849533081', 'lastCheck': 1384921653.762162, 'valid': True, 'code': 0}}
>Thread-15916448::DEBUG::2013-11-20 04:27:35,397::task::495::TaskManager.Task::(_debug) Task f09e3e93-83f3-4bc6-bd91-1678c7bbbdaf: moving from state preparing -> state finished
>Thread-15916448::DEBUG::2013-11-20 04:27:35,398::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916448::DEBUG::2013-11-20 04:27:35,398::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916448::DEBUG::2013-11-20 04:27:35,399::task::495::TaskManager.Task::(_debug) Task f09e3e93-83f3-4bc6-bd91-1678c7bbbdaf: ref 0 aborting False
>Thread-15916448::INFO::2013-11-20 04:27:35,400::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000374794006348', 'lastCheck': 1384921648.6418791, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00190806388855', 'lastCheck': 1384921647.0807209, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00172400474548', 'lastCheck': 1384921654.24876, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000450849533081', 'lastCheck': 1384921653.762162, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:27:35,665::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.086279 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:27:37,684::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:27:38,192::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0772937 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:27:40,207::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:27:40,748::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0770447 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:27:42,765::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:27:43,320::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0773842 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15913639::INFO::2013-11-20 04:27:44,977::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15915918::INFO::2013-11-20 04:27:44,978::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15913639::DEBUG::2013-11-20 04:27:44,979::task::495::TaskManager.Task::(_debug) Task 51b4dc74-6336-4080-a686-151a16f8bd7a: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:27:44,980::task::495::TaskManager.Task::(_debug) Task 2f8b481f-5285-490c-b1b7-75248ca01e33: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:27:44,984::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e2366850-8384-4d3a-bd0a-1ea6f33eea94`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:27:44,985::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bf4f5812-36d9-4e7f-84da-f01c128f099e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:27:44,986::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:27:44,987::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:27:44,989::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:27:44,990::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e2366850-8384-4d3a-bd0a-1ea6f33eea94`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:27:44,991::resourceManager::481::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' found in shared state and queue is empty, Joining current shared lock (2 active users)
>Thread-15915918::DEBUG::2013-11-20 04:27:44,992::task::495::TaskManager.Task::(_debug) Task 2f8b481f-5285-490c-b1b7-75248ca01e33: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:27:44,993::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bf4f5812-36d9-4e7f-84da-f01c128f099e`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:27:44,995::task::495::TaskManager.Task::(_debug) Task 2f8b481f-5285-490c-b1b7-75248ca01e33: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:27:44,996::task::495::TaskManager.Task::(_debug) Task 51b4dc74-6336-4080-a686-151a16f8bd7a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:27:44,997::task::495::TaskManager.Task::(_debug) Task 2f8b481f-5285-490c-b1b7-75248ca01e33: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15916453::DEBUG::2013-11-20 04:27:44,999::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913639::DEBUG::2013-11-20 04:27:45,000::task::495::TaskManager.Task::(_debug) Task 51b4dc74-6336-4080-a686-151a16f8bd7a: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:27:45,001::task::495::TaskManager.Task::(_debug) Task 2f8b481f-5285-490c-b1b7-75248ca01e33: moving from state preparing -> state finished
>Thread-15916453::INFO::2013-11-20 04:27:45,001::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913639::DEBUG::2013-11-20 04:27:45,002::task::495::TaskManager.Task::(_debug) Task 51b4dc74-6336-4080-a686-151a16f8bd7a: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15915918::DEBUG::2013-11-20 04:27:45,003::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15916453::DEBUG::2013-11-20 04:27:45,004::task::495::TaskManager.Task::(_debug) Task 7ace8c5c-a613-4f2e-bda4-180d7feec62f: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:27:45,005::task::495::TaskManager.Task::(_debug) Task 51b4dc74-6336-4080-a686-151a16f8bd7a: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:27:45,005::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916453::DEBUG::2013-11-20 04:27:45,006::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15913639::DEBUG::2013-11-20 04:27:45,007::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:27:45,009::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15916453::DEBUG::2013-11-20 04:27:45,009::task::495::TaskManager.Task::(_debug) Task 7ace8c5c-a613-4f2e-bda4-180d7feec62f: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913639::DEBUG::2013-11-20 04:27:45,010::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:27:45,012::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (1 active users)
>Thread-15916453::DEBUG::2013-11-20 04:27:45,013::task::495::TaskManager.Task::(_debug) Task 7ace8c5c-a613-4f2e-bda4-180d7feec62f: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:27:45,014::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:27:45,016::task::495::TaskManager.Task::(_debug) Task 2f8b481f-5285-490c-b1b7-75248ca01e33: ref 0 aborting False
>Thread-15916453::DEBUG::2013-11-20 04:27:45,017::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913639::DEBUG::2013-11-20 04:27:45,018::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::INFO::2013-11-20 04:27:45,019::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15916453::DEBUG::2013-11-20 04:27:45,020::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:27:45,021::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15916453::DEBUG::2013-11-20 04:27:45,023::task::495::TaskManager.Task::(_debug) Task 7ace8c5c-a613-4f2e-bda4-180d7feec62f: ref 0 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:27:45,024::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15916453::INFO::2013-11-20 04:27:45,025::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913639::DEBUG::2013-11-20 04:27:45,026::task::495::TaskManager.Task::(_debug) Task 51b4dc74-6336-4080-a686-151a16f8bd7a: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:27:45,029::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15916454::DEBUG::2013-11-20 04:27:45,033::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916454::INFO::2013-11-20 04:27:45,035::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916454::DEBUG::2013-11-20 04:27:45,036::task::495::TaskManager.Task::(_debug) Task adacdf2b-a6a8-4e6d-9171-4cf5647ed38f: moving from state init -> state preparing
>Thread-15916454::DEBUG::2013-11-20 04:27:45,037::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`744ac508-f229-4f9a-bdcf-f82a388f2820`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916454::DEBUG::2013-11-20 04:27:45,038::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916454::DEBUG::2013-11-20 04:27:45,039::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916454::DEBUG::2013-11-20 04:27:45,040::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`744ac508-f229-4f9a-bdcf-f82a388f2820`::Granted request
>Thread-15916454::DEBUG::2013-11-20 04:27:45,041::task::495::TaskManager.Task::(_debug) Task adacdf2b-a6a8-4e6d-9171-4cf5647ed38f: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916454::DEBUG::2013-11-20 04:27:45,042::task::495::TaskManager.Task::(_debug) Task adacdf2b-a6a8-4e6d-9171-4cf5647ed38f: ref 1 aborting False
>Thread-15916454::WARNING::2013-11-20 04:27:45,044::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916454::DEBUG::2013-11-20 04:27:45,049::task::495::TaskManager.Task::(_debug) Task adacdf2b-a6a8-4e6d-9171-4cf5647ed38f: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916454::DEBUG::2013-11-20 04:27:45,050::task::495::TaskManager.Task::(_debug) Task adacdf2b-a6a8-4e6d-9171-4cf5647ed38f: moving from state preparing -> state finished
>Thread-15916454::DEBUG::2013-11-20 04:27:45,050::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916454::DEBUG::2013-11-20 04:27:45,051::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916454::DEBUG::2013-11-20 04:27:45,052::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916454::DEBUG::2013-11-20 04:27:45,053::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916454::DEBUG::2013-11-20 04:27:45,054::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916454::DEBUG::2013-11-20 04:27:45,055::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916454::DEBUG::2013-11-20 04:27:45,056::task::495::TaskManager.Task::(_debug) Task adacdf2b-a6a8-4e6d-9171-4cf5647ed38f: ref 0 aborting False
>Thread-15916454::INFO::2013-11-20 04:27:45,057::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:27:45,334::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:27:45,882::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0838631 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:27:47,901::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:27:48,450::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0751041 s, 13.6 MB/s\n'; <rc> = 0
>Thread-15915863::INFO::2013-11-20 04:27:49,902::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:27:49,903::task::495::TaskManager.Task::(_debug) Task 44a75e1e-6b74-48b5-bcaa-240b222790fb: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:27:49,904::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3d338419-fdf1-4b9f-8db5-2b0cd9c209ae`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:27:49,905::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:27:49,906::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:27:49,907::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3d338419-fdf1-4b9f-8db5-2b0cd9c209ae`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:27:49,909::task::495::TaskManager.Task::(_debug) Task 44a75e1e-6b74-48b5-bcaa-240b222790fb: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:27:49,910::task::495::TaskManager.Task::(_debug) Task 44a75e1e-6b74-48b5-bcaa-240b222790fb: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:27:49,910::task::495::TaskManager.Task::(_debug) Task 44a75e1e-6b74-48b5-bcaa-240b222790fb: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:27:49,911::task::495::TaskManager.Task::(_debug) Task 44a75e1e-6b74-48b5-bcaa-240b222790fb: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:27:49,912::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:27:49,912::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:27:49,913::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:27:49,914::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:27:49,915::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:27:49,915::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:27:49,916::task::495::TaskManager.Task::(_debug) Task 44a75e1e-6b74-48b5-bcaa-240b222790fb: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:27:49,917::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:27:50,464::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:27:51,008::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0833228 s, 12.3 MB/s\n'; <rc> = 0
>Thread-15916456::INFO::2013-11-20 04:27:51,146::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916456::DEBUG::2013-11-20 04:27:51,147::task::495::TaskManager.Task::(_debug) Task ecf62a2d-7f15-42cb-b670-4972ecfeffaa: moving from state init -> state preparing
>Thread-15916456::DEBUG::2013-11-20 04:27:51,148::task::495::TaskManager.Task::(_debug) Task ecf62a2d-7f15-42cb-b670-4972ecfeffaa: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00034499168396', 'lastCheck': 1384921669.1182261, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.020012140274', 'lastCheck': 1384921667.5832641, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00124311447144', 'lastCheck': 1384921664.4395599, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000458955764771', 'lastCheck': 1384921663.9783881, 'valid': True, 'code': 0}}
>Thread-15916456::DEBUG::2013-11-20 04:27:51,149::task::495::TaskManager.Task::(_debug) Task ecf62a2d-7f15-42cb-b670-4972ecfeffaa: moving from state preparing -> state finished
>Thread-15916456::DEBUG::2013-11-20 04:27:51,149::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916456::DEBUG::2013-11-20 04:27:51,150::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916456::DEBUG::2013-11-20 04:27:51,151::task::495::TaskManager.Task::(_debug) Task ecf62a2d-7f15-42cb-b670-4972ecfeffaa: ref 0 aborting False
>Thread-15916456::INFO::2013-11-20 04:27:51,151::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00034499168396', 'lastCheck': 1384921669.1182261, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.020012140274', 'lastCheck': 1384921667.5832641, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00124311447144', 'lastCheck': 1384921664.4395599, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000458955764771', 'lastCheck': 1384921663.9783881, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:27:53,024::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:27:53,560::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0854426 s, 12.0 MB/s\n'; <rc> = 0
>Thread-15916459::DEBUG::2013-11-20 04:27:55,514::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916459::INFO::2013-11-20 04:27:55,516::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916459::DEBUG::2013-11-20 04:27:55,516::task::495::TaskManager.Task::(_debug) Task 12348426-4927-44f0-a6f5-897464e317ed: moving from state init -> state preparing
>Thread-15916459::DEBUG::2013-11-20 04:27:55,517::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916459::DEBUG::2013-11-20 04:27:55,518::task::495::TaskManager.Task::(_debug) Task 12348426-4927-44f0-a6f5-897464e317ed: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916459::DEBUG::2013-11-20 04:27:55,518::task::495::TaskManager.Task::(_debug) Task 12348426-4927-44f0-a6f5-897464e317ed: moving from state preparing -> state finished
>Thread-15916459::DEBUG::2013-11-20 04:27:55,519::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916459::DEBUG::2013-11-20 04:27:55,519::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916459::DEBUG::2013-11-20 04:27:55,520::task::495::TaskManager.Task::(_debug) Task 12348426-4927-44f0-a6f5-897464e317ed: ref 0 aborting False
>Thread-15916459::INFO::2013-11-20 04:27:55,521::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916460::DEBUG::2013-11-20 04:27:55,525::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916460::INFO::2013-11-20 04:27:55,527::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916460::DEBUG::2013-11-20 04:27:55,527::task::495::TaskManager.Task::(_debug) Task 1311fa49-879b-4d64-9306-bf14d59818ca: moving from state init -> state preparing
>Thread-15916460::DEBUG::2013-11-20 04:27:55,529::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b7112abc-1e66-485b-88e3-3916f1cf2a48`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916460::DEBUG::2013-11-20 04:27:55,529::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916460::DEBUG::2013-11-20 04:27:55,530::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916460::DEBUG::2013-11-20 04:27:55,531::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b7112abc-1e66-485b-88e3-3916f1cf2a48`::Granted request
>Thread-15916460::DEBUG::2013-11-20 04:27:55,532::task::495::TaskManager.Task::(_debug) Task 1311fa49-879b-4d64-9306-bf14d59818ca: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916460::DEBUG::2013-11-20 04:27:55,532::task::495::TaskManager.Task::(_debug) Task 1311fa49-879b-4d64-9306-bf14d59818ca: ref 1 aborting False
>Thread-15916460::WARNING::2013-11-20 04:27:55,535::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916460::DEBUG::2013-11-20 04:27:55,536::task::495::TaskManager.Task::(_debug) Task 1311fa49-879b-4d64-9306-bf14d59818ca: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916460::DEBUG::2013-11-20 04:27:55,537::task::495::TaskManager.Task::(_debug) Task 1311fa49-879b-4d64-9306-bf14d59818ca: moving from state preparing -> state finished
>Thread-15916460::DEBUG::2013-11-20 04:27:55,537::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916460::DEBUG::2013-11-20 04:27:55,538::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916460::DEBUG::2013-11-20 04:27:55,539::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916460::DEBUG::2013-11-20 04:27:55,539::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916460::DEBUG::2013-11-20 04:27:55,540::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916460::DEBUG::2013-11-20 04:27:55,540::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916460::DEBUG::2013-11-20 04:27:55,541::task::495::TaskManager.Task::(_debug) Task 1311fa49-879b-4d64-9306-bf14d59818ca: ref 0 aborting False
>Thread-15916460::INFO::2013-11-20 04:27:55,542::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:27:55,573::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:27:56,112::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0810103 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 04:27:57,220::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:27:57,221::task::495::TaskManager.Task::(_debug) Task b5e86508-d74c-4795-81f8-6438f3ee7e24: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:27:57,225::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c2159c15-abae-4d3c-be80-d9eb4fb98c9b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:27:57,226::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:27:57,228::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:27:57,229::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c2159c15-abae-4d3c-be80-d9eb4fb98c9b`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:27:57,230::task::495::TaskManager.Task::(_debug) Task b5e86508-d74c-4795-81f8-6438f3ee7e24: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:27:57,231::task::495::TaskManager.Task::(_debug) Task b5e86508-d74c-4795-81f8-6438f3ee7e24: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:27:57,232::task::495::TaskManager.Task::(_debug) Task b5e86508-d74c-4795-81f8-6438f3ee7e24: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:27:57,232::task::495::TaskManager.Task::(_debug) Task b5e86508-d74c-4795-81f8-6438f3ee7e24: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:27:57,233::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:27:57,234::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:27:57,235::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:27:57,236::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:27:57,237::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:27:57,238::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:27:57,239::task::495::TaskManager.Task::(_debug) Task b5e86508-d74c-4795-81f8-6438f3ee7e24: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:27:57,240::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913518::INFO::2013-11-20 04:27:57,830::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:27:57,831::task::495::TaskManager.Task::(_debug) Task 6156be10-bacc-4db3-8bf9-93bd442ecff4: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:27:57,832::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`85338a80-e9aa-4a51-aeb9-8f0bba7e27c5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:27:57,832::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:27:57,833::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:27:57,833::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`85338a80-e9aa-4a51-aeb9-8f0bba7e27c5`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:27:57,834::task::495::TaskManager.Task::(_debug) Task 6156be10-bacc-4db3-8bf9-93bd442ecff4: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:27:57,834::task::495::TaskManager.Task::(_debug) Task 6156be10-bacc-4db3-8bf9-93bd442ecff4: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:27:57,835::task::495::TaskManager.Task::(_debug) Task 6156be10-bacc-4db3-8bf9-93bd442ecff4: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:27:57,835::task::495::TaskManager.Task::(_debug) Task 6156be10-bacc-4db3-8bf9-93bd442ecff4: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:27:57,835::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:27:57,836::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:27:57,836::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:27:57,836::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:27:57,837::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:27:57,837::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:27:57,838::task::495::TaskManager.Task::(_debug) Task 6156be10-bacc-4db3-8bf9-93bd442ecff4: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:27:57,838::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:27:57,839::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:27:57,839::task::495::TaskManager.Task::(_debug) Task f253b32f-87d3-4efa-8384-b9069e47f88d: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:27:57,840::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c9b740ad-0f0e-48e3-a4f0-ed51f5c757d8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:27:57,840::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:27:57,841::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:27:57,841::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c9b740ad-0f0e-48e3-a4f0-ed51f5c757d8`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:27:57,842::task::495::TaskManager.Task::(_debug) Task f253b32f-87d3-4efa-8384-b9069e47f88d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:27:57,842::task::495::TaskManager.Task::(_debug) Task f253b32f-87d3-4efa-8384-b9069e47f88d: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:27:57,842::task::495::TaskManager.Task::(_debug) Task f253b32f-87d3-4efa-8384-b9069e47f88d: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:27:57,843::task::495::TaskManager.Task::(_debug) Task f253b32f-87d3-4efa-8384-b9069e47f88d: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:27:57,843::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:27:57,843::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:27:57,844::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:27:57,845::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:27:57,845::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:27:57,846::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:27:57,846::task::495::TaskManager.Task::(_debug) Task f253b32f-87d3-4efa-8384-b9069e47f88d: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:27:57,847::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:27:58,126::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:27:58,664::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0625941 s, 16.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:28:00,679::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:28:01,215::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0972126 s, 10.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:28:03,232::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:28:03,799::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.101639 s, 10.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:28:05,816::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916464::DEBUG::2013-11-20 04:28:06,013::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916464::INFO::2013-11-20 04:28:06,018::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916464::DEBUG::2013-11-20 04:28:06,019::task::495::TaskManager.Task::(_debug) Task c78ac6a0-ed8d-45cf-83ae-238b07f3eb7b: moving from state init -> state preparing
>Thread-15916464::DEBUG::2013-11-20 04:28:06,020::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916464::DEBUG::2013-11-20 04:28:06,021::task::495::TaskManager.Task::(_debug) Task c78ac6a0-ed8d-45cf-83ae-238b07f3eb7b: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916464::DEBUG::2013-11-20 04:28:06,022::task::495::TaskManager.Task::(_debug) Task c78ac6a0-ed8d-45cf-83ae-238b07f3eb7b: moving from state preparing -> state finished
>Thread-15916464::DEBUG::2013-11-20 04:28:06,022::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916464::DEBUG::2013-11-20 04:28:06,023::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916464::DEBUG::2013-11-20 04:28:06,024::task::495::TaskManager.Task::(_debug) Task c78ac6a0-ed8d-45cf-83ae-238b07f3eb7b: ref 0 aborting False
>Thread-15916464::INFO::2013-11-20 04:28:06,025::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916465::DEBUG::2013-11-20 04:28:06,030::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916465::INFO::2013-11-20 04:28:06,032::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916465::DEBUG::2013-11-20 04:28:06,032::task::495::TaskManager.Task::(_debug) Task 1818b23d-72ce-4fda-8f02-1e93ec9b61e8: moving from state init -> state preparing
>Thread-15916465::DEBUG::2013-11-20 04:28:06,034::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1b6caba7-be3e-46cc-89ed-f3fdfacf7f46`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916465::DEBUG::2013-11-20 04:28:06,035::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916465::DEBUG::2013-11-20 04:28:06,036::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916465::DEBUG::2013-11-20 04:28:06,036::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1b6caba7-be3e-46cc-89ed-f3fdfacf7f46`::Granted request
>Thread-15916465::DEBUG::2013-11-20 04:28:06,037::task::495::TaskManager.Task::(_debug) Task 1818b23d-72ce-4fda-8f02-1e93ec9b61e8: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916465::DEBUG::2013-11-20 04:28:06,038::task::495::TaskManager.Task::(_debug) Task 1818b23d-72ce-4fda-8f02-1e93ec9b61e8: ref 1 aborting False
>Thread-15916465::WARNING::2013-11-20 04:28:06,041::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916465::DEBUG::2013-11-20 04:28:06,042::task::495::TaskManager.Task::(_debug) Task 1818b23d-72ce-4fda-8f02-1e93ec9b61e8: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916465::DEBUG::2013-11-20 04:28:06,043::task::495::TaskManager.Task::(_debug) Task 1818b23d-72ce-4fda-8f02-1e93ec9b61e8: moving from state preparing -> state finished
>Thread-15916465::DEBUG::2013-11-20 04:28:06,044::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916465::DEBUG::2013-11-20 04:28:06,044::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916465::DEBUG::2013-11-20 04:28:06,045::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916465::DEBUG::2013-11-20 04:28:06,046::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916465::DEBUG::2013-11-20 04:28:06,046::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916465::DEBUG::2013-11-20 04:28:06,047::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916465::DEBUG::2013-11-20 04:28:06,048::task::495::TaskManager.Task::(_debug) Task 1818b23d-72ce-4fda-8f02-1e93ec9b61e8: ref 0 aborting False
>Thread-15916465::INFO::2013-11-20 04:28:06,048::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:28:06,354::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0809718 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15916466::INFO::2013-11-20 04:28:07,115::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916466::DEBUG::2013-11-20 04:28:07,116::task::495::TaskManager.Task::(_debug) Task 862583f6-ff0a-4fbd-8781-9d6c7e2310c0: moving from state init -> state preparing
>Thread-15916466::DEBUG::2013-11-20 04:28:07,117::task::495::TaskManager.Task::(_debug) Task 862583f6-ff0a-4fbd-8781-9d6c7e2310c0: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000328063964844', 'lastCheck': 1384921679.3554859, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00134682655334', 'lastCheck': 1384921677.7981, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0196948051453', 'lastCheck': 1384921684.903749, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000420093536377', 'lastCheck': 1384921684.45049, 'valid': True, 'code': 0}}
>Thread-15916466::DEBUG::2013-11-20 04:28:07,118::task::495::TaskManager.Task::(_debug) Task 862583f6-ff0a-4fbd-8781-9d6c7e2310c0: moving from state preparing -> state finished
>Thread-15916466::DEBUG::2013-11-20 04:28:07,118::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916466::DEBUG::2013-11-20 04:28:07,119::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916466::DEBUG::2013-11-20 04:28:07,120::task::495::TaskManager.Task::(_debug) Task 862583f6-ff0a-4fbd-8781-9d6c7e2310c0: ref 0 aborting False
>Thread-15916466::INFO::2013-11-20 04:28:07,120::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000328063964844', 'lastCheck': 1384921679.3554859, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00134682655334', 'lastCheck': 1384921677.7981, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0196948051453', 'lastCheck': 1384921684.903749, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000420093536377', 'lastCheck': 1384921684.45049, 'valid': True, 'code': 0}}
>Thread-15912392::INFO::2013-11-20 04:28:07,457::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:28:07,458::task::495::TaskManager.Task::(_debug) Task 45242e0d-893f-4ffd-9a1c-0b072b2dd681: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:28:07,459::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4c53bb41-ac9a-466b-bb03-4e3953bd1f29`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:28:07,460::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:28:07,461::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:28:07,461::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4c53bb41-ac9a-466b-bb03-4e3953bd1f29`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:28:07,462::task::495::TaskManager.Task::(_debug) Task 45242e0d-893f-4ffd-9a1c-0b072b2dd681: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:28:07,463::task::495::TaskManager.Task::(_debug) Task 45242e0d-893f-4ffd-9a1c-0b072b2dd681: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:28:07,464::task::495::TaskManager.Task::(_debug) Task 45242e0d-893f-4ffd-9a1c-0b072b2dd681: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:28:07,464::task::495::TaskManager.Task::(_debug) Task 45242e0d-893f-4ffd-9a1c-0b072b2dd681: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:28:07,465::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:28:07,467::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:28:07,469::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:28:07,471::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:28:07,471::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:28:07,472::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:28:07,473::task::495::TaskManager.Task::(_debug) Task 45242e0d-893f-4ffd-9a1c-0b072b2dd681: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:28:07,474::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:28:08,369::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:28:08,888::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0710909 s, 14.4 MB/s\n'; <rc> = 0
>Thread-42838::DEBUG::2013-11-20 04:28:09,582::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 04:28:10,309::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:28:10,899::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:28:11,455::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0810668 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:28:13,470::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 04:28:13,819::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:28:13,820::task::495::TaskManager.Task::(_debug) Task 55f1248d-00bc-47eb-a194-e360d8756942: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:28:13,822::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fef7e6d7-525f-44f3-ab83-e7e4c832410b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:28:13,822::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:28:13,823::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:28:13,824::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fef7e6d7-525f-44f3-ab83-e7e4c832410b`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:28:13,824::task::495::TaskManager.Task::(_debug) Task 55f1248d-00bc-47eb-a194-e360d8756942: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:28:13,825::task::495::TaskManager.Task::(_debug) Task 55f1248d-00bc-47eb-a194-e360d8756942: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:28:13,825::task::495::TaskManager.Task::(_debug) Task 55f1248d-00bc-47eb-a194-e360d8756942: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:28:13,826::task::495::TaskManager.Task::(_debug) Task 55f1248d-00bc-47eb-a194-e360d8756942: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:28:13,826::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:28:13,827::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:28:13,827::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:28:13,828::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:28:13,828::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:28:13,828::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:28:13,829::task::495::TaskManager.Task::(_debug) Task 55f1248d-00bc-47eb-a194-e360d8756942: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:28:13,829::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:28:14,013::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0831319 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:28:16,029::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916470::DEBUG::2013-11-20 04:28:16,535::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916470::INFO::2013-11-20 04:28:16,538::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916470::DEBUG::2013-11-20 04:28:16,538::task::495::TaskManager.Task::(_debug) Task 25db2500-4ec3-402b-8bad-1d9c2ab50af3: moving from state init -> state preparing
>Thread-15916470::DEBUG::2013-11-20 04:28:16,540::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916470::DEBUG::2013-11-20 04:28:16,541::task::495::TaskManager.Task::(_debug) Task 25db2500-4ec3-402b-8bad-1d9c2ab50af3: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916470::DEBUG::2013-11-20 04:28:16,542::task::495::TaskManager.Task::(_debug) Task 25db2500-4ec3-402b-8bad-1d9c2ab50af3: moving from state preparing -> state finished
>Thread-15916470::DEBUG::2013-11-20 04:28:16,543::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916470::DEBUG::2013-11-20 04:28:16,543::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916470::DEBUG::2013-11-20 04:28:16,544::task::495::TaskManager.Task::(_debug) Task 25db2500-4ec3-402b-8bad-1d9c2ab50af3: ref 0 aborting False
>Thread-15916470::INFO::2013-11-20 04:28:16,545::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916471::DEBUG::2013-11-20 04:28:16,550::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916471::INFO::2013-11-20 04:28:16,552::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916471::DEBUG::2013-11-20 04:28:16,553::task::495::TaskManager.Task::(_debug) Task 9f1bb4c8-4649-4cfb-b782-39232b36e6c2: moving from state init -> state preparing
>Thread-15916471::DEBUG::2013-11-20 04:28:16,555::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f04fe195-bc27-45ea-9748-2e8552ad19fc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916471::DEBUG::2013-11-20 04:28:16,555::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916471::DEBUG::2013-11-20 04:28:16,557::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916471::DEBUG::2013-11-20 04:28:16,557::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f04fe195-bc27-45ea-9748-2e8552ad19fc`::Granted request
>Thread-15916471::DEBUG::2013-11-20 04:28:16,559::task::495::TaskManager.Task::(_debug) Task 9f1bb4c8-4649-4cfb-b782-39232b36e6c2: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916471::DEBUG::2013-11-20 04:28:16,560::task::495::TaskManager.Task::(_debug) Task 9f1bb4c8-4649-4cfb-b782-39232b36e6c2: ref 1 aborting False
>Thread-15916471::WARNING::2013-11-20 04:28:16,563::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916471::DEBUG::2013-11-20 04:28:16,564::task::495::TaskManager.Task::(_debug) Task 9f1bb4c8-4649-4cfb-b782-39232b36e6c2: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916471::DEBUG::2013-11-20 04:28:16,566::task::495::TaskManager.Task::(_debug) Task 9f1bb4c8-4649-4cfb-b782-39232b36e6c2: moving from state preparing -> state finished
>Thread-15916471::DEBUG::2013-11-20 04:28:16,567::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916471::DEBUG::2013-11-20 04:28:16,568::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916471::DEBUG::2013-11-20 04:28:16,569::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916471::DEBUG::2013-11-20 04:28:16,569::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916471::DEBUG::2013-11-20 04:28:16,570::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916471::DEBUG::2013-11-20 04:28:16,571::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916471::DEBUG::2013-11-20 04:28:16,572::task::495::TaskManager.Task::(_debug) Task 9f1bb4c8-4649-4cfb-b782-39232b36e6c2: ref 0 aborting False
>Thread-15916471::INFO::2013-11-20 04:28:16,573::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:28:16,646::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.114554 s, 8.9 MB/s\n'; <rc> = 0
>Thread-15914453::INFO::2013-11-20 04:28:17,692::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:28:17,693::task::495::TaskManager.Task::(_debug) Task a3179d50-c199-45c8-a915-89f8a2f2a930: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:28:17,696::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`aff49d68-abbf-4a96-951a-1dc512331115`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:28:17,697::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:28:17,699::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:28:17,700::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`aff49d68-abbf-4a96-951a-1dc512331115`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:28:17,701::task::495::TaskManager.Task::(_debug) Task a3179d50-c199-45c8-a915-89f8a2f2a930: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:28:17,702::task::495::TaskManager.Task::(_debug) Task a3179d50-c199-45c8-a915-89f8a2f2a930: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:28:17,703::task::495::TaskManager.Task::(_debug) Task a3179d50-c199-45c8-a915-89f8a2f2a930: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:28:17,704::task::495::TaskManager.Task::(_debug) Task a3179d50-c199-45c8-a915-89f8a2f2a930: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:28:17,705::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:28:17,705::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:28:17,706::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:28:17,707::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:28:17,707::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:28:17,708::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:28:17,709::task::495::TaskManager.Task::(_debug) Task a3179d50-c199-45c8-a915-89f8a2f2a930: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:28:17,709::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:28:18,667::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:28:19,224::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0793004 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:28:21,245::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:28:21,821::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0833814 s, 12.3 MB/s\n'; <rc> = 0
>Thread-15916474::INFO::2013-11-20 04:28:23,201::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916474::DEBUG::2013-11-20 04:28:23,202::task::495::TaskManager.Task::(_debug) Task 452202cd-e2f5-4889-990e-da06c9d2ba65: moving from state init -> state preparing
>Thread-15916474::DEBUG::2013-11-20 04:28:23,202::task::495::TaskManager.Task::(_debug) Task 452202cd-e2f5-4889-990e-da06c9d2ba65: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000369071960449', 'lastCheck': 1384921700.329037, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00145292282104', 'lastCheck': 1384921698.2409921, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00125002861023', 'lastCheck': 1384921695.1228969, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000566005706787', 'lastCheck': 1384921694.692132, 'valid': True, 'code': 0}}
>Thread-15916474::DEBUG::2013-11-20 04:28:23,203::task::495::TaskManager.Task::(_debug) Task 452202cd-e2f5-4889-990e-da06c9d2ba65: moving from state preparing -> state finished
>Thread-15916474::DEBUG::2013-11-20 04:28:23,204::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916474::DEBUG::2013-11-20 04:28:23,205::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916474::DEBUG::2013-11-20 04:28:23,206::task::495::TaskManager.Task::(_debug) Task 452202cd-e2f5-4889-990e-da06c9d2ba65: ref 0 aborting False
>Thread-15916474::INFO::2013-11-20 04:28:23,207::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000369071960449', 'lastCheck': 1384921700.329037, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00145292282104', 'lastCheck': 1384921698.2409921, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00125002861023', 'lastCheck': 1384921695.1228969, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000566005706787', 'lastCheck': 1384921694.692132, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:28:23,835::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:28:24,372::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0713921 s, 14.3 MB/s\n'; <rc> = 0
>Thread-15914519::INFO::2013-11-20 04:28:25,129::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:28:25,130::task::495::TaskManager.Task::(_debug) Task 59cb429b-5d76-4b99-a9d5-fc7044c9ec4d: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:28:25,132::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`da7e7e02-600d-4396-a2a0-0e79902ac16a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:28:25,132::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:28:25,133::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:28:25,134::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`da7e7e02-600d-4396-a2a0-0e79902ac16a`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:28:25,135::task::495::TaskManager.Task::(_debug) Task 59cb429b-5d76-4b99-a9d5-fc7044c9ec4d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 04:28:25,135::task::495::TaskManager.Task::(_debug) Task 59cb429b-5d76-4b99-a9d5-fc7044c9ec4d: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:28:25,136::task::495::TaskManager.Task::(_debug) Task 59cb429b-5d76-4b99-a9d5-fc7044c9ec4d: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 04:28:25,137::task::495::TaskManager.Task::(_debug) Task 59cb429b-5d76-4b99-a9d5-fc7044c9ec4d: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:28:25,137::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:28:25,138::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:28:25,139::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 04:28:25,139::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:28:25,140::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:28:25,140::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:28:25,141::task::495::TaskManager.Task::(_debug) Task 59cb429b-5d76-4b99-a9d5-fc7044c9ec4d: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 04:28:25,142::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 04:28:26,391::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:28:26,933::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0792797 s, 12.9 MB/s\n'; <rc> = 0
>Thread-15916477::DEBUG::2013-11-20 04:28:27,007::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916477::INFO::2013-11-20 04:28:27,007::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916477::DEBUG::2013-11-20 04:28:27,008::task::495::TaskManager.Task::(_debug) Task 7bafe11d-a327-4973-bdf7-efcb0569b049: moving from state init -> state preparing
>Thread-15916477::DEBUG::2013-11-20 04:28:27,008::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916477::DEBUG::2013-11-20 04:28:27,009::task::495::TaskManager.Task::(_debug) Task 7bafe11d-a327-4973-bdf7-efcb0569b049: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916477::DEBUG::2013-11-20 04:28:27,009::task::495::TaskManager.Task::(_debug) Task 7bafe11d-a327-4973-bdf7-efcb0569b049: moving from state preparing -> state finished
>Thread-15916477::DEBUG::2013-11-20 04:28:27,009::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916477::DEBUG::2013-11-20 04:28:27,010::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916477::DEBUG::2013-11-20 04:28:27,010::task::495::TaskManager.Task::(_debug) Task 7bafe11d-a327-4973-bdf7-efcb0569b049: ref 0 aborting False
>Thread-15916477::INFO::2013-11-20 04:28:27,010::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916478::DEBUG::2013-11-20 04:28:27,014::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916478::INFO::2013-11-20 04:28:27,015::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916478::DEBUG::2013-11-20 04:28:27,016::task::495::TaskManager.Task::(_debug) Task e6b73849-6418-4d48-9791-863395609f1a: moving from state init -> state preparing
>Thread-15916478::DEBUG::2013-11-20 04:28:27,017::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3b36f2ee-a94f-412b-a349-faec44f10928`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916478::DEBUG::2013-11-20 04:28:27,017::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916478::DEBUG::2013-11-20 04:28:27,018::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916478::DEBUG::2013-11-20 04:28:27,019::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3b36f2ee-a94f-412b-a349-faec44f10928`::Granted request
>Thread-15916478::DEBUG::2013-11-20 04:28:27,020::task::495::TaskManager.Task::(_debug) Task e6b73849-6418-4d48-9791-863395609f1a: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916478::DEBUG::2013-11-20 04:28:27,020::task::495::TaskManager.Task::(_debug) Task e6b73849-6418-4d48-9791-863395609f1a: ref 1 aborting False
>Thread-15916478::WARNING::2013-11-20 04:28:27,022::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916478::DEBUG::2013-11-20 04:28:27,023::task::495::TaskManager.Task::(_debug) Task e6b73849-6418-4d48-9791-863395609f1a: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916478::DEBUG::2013-11-20 04:28:27,024::task::495::TaskManager.Task::(_debug) Task e6b73849-6418-4d48-9791-863395609f1a: moving from state preparing -> state finished
>Thread-15916478::DEBUG::2013-11-20 04:28:27,024::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916478::DEBUG::2013-11-20 04:28:27,024::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916478::DEBUG::2013-11-20 04:28:27,025::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916478::DEBUG::2013-11-20 04:28:27,025::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916478::DEBUG::2013-11-20 04:28:27,025::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916478::DEBUG::2013-11-20 04:28:27,026::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916478::DEBUG::2013-11-20 04:28:27,026::task::495::TaskManager.Task::(_debug) Task e6b73849-6418-4d48-9791-863395609f1a: ref 0 aborting False
>Thread-15916478::INFO::2013-11-20 04:28:27,027::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913546::INFO::2013-11-20 04:28:27,707::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:28:27,709::task::495::TaskManager.Task::(_debug) Task e27912da-97ed-4ebe-8079-07b326e85b92: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:28:27,710::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`22fd0cab-70a6-47c3-b921-99fd7fa300f5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:28:27,712::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:28:27,713::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:28:27,714::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`22fd0cab-70a6-47c3-b921-99fd7fa300f5`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:28:27,715::task::495::TaskManager.Task::(_debug) Task e27912da-97ed-4ebe-8079-07b326e85b92: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:28:27,716::task::495::TaskManager.Task::(_debug) Task e27912da-97ed-4ebe-8079-07b326e85b92: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:28:27,717::task::495::TaskManager.Task::(_debug) Task e27912da-97ed-4ebe-8079-07b326e85b92: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 04:28:27,717::task::495::TaskManager.Task::(_debug) Task e27912da-97ed-4ebe-8079-07b326e85b92: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:28:27,718::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:28:27,718::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:28:27,719::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:28:27,720::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:28:27,720::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:28:27,721::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:28:27,722::task::495::TaskManager.Task::(_debug) Task e27912da-97ed-4ebe-8079-07b326e85b92: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:28:27,722::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:28:27,723::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:28:27,724::task::495::TaskManager.Task::(_debug) Task f479b245-bd4d-419c-b979-c88261483900: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:28:27,725::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`ec9d3f7b-d0df-4155-bcb8-1c381b540f23`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:28:27,725::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:28:27,726::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:28:27,727::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`ec9d3f7b-d0df-4155-bcb8-1c381b540f23`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:28:27,728::task::495::TaskManager.Task::(_debug) Task f479b245-bd4d-419c-b979-c88261483900: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:28:27,729::task::495::TaskManager.Task::(_debug) Task f479b245-bd4d-419c-b979-c88261483900: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:28:27,729::task::495::TaskManager.Task::(_debug) Task f479b245-bd4d-419c-b979-c88261483900: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:28:27,730::task::495::TaskManager.Task::(_debug) Task f479b245-bd4d-419c-b979-c88261483900: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:28:27,731::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:28:27,731::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:28:27,732::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:28:27,732::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:28:27,733::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:28:27,734::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:28:27,735::task::495::TaskManager.Task::(_debug) Task f479b245-bd4d-419c-b979-c88261483900: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:28:27,736::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:28:27,737::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:28:27,738::task::495::TaskManager.Task::(_debug) Task f3818ffb-2ce4-4be0-8e3d-6f6ca86753ea: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:28:27,739::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b838c4d8-7fe6-4a3a-b0f7-65cbb3ec2264`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:28:27,740::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:28:27,741::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:28:27,741::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b838c4d8-7fe6-4a3a-b0f7-65cbb3ec2264`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:28:27,742::task::495::TaskManager.Task::(_debug) Task f3818ffb-2ce4-4be0-8e3d-6f6ca86753ea: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:28:27,743::task::495::TaskManager.Task::(_debug) Task f3818ffb-2ce4-4be0-8e3d-6f6ca86753ea: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:28:27,744::task::495::TaskManager.Task::(_debug) Task f3818ffb-2ce4-4be0-8e3d-6f6ca86753ea: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:28:27,745::task::495::TaskManager.Task::(_debug) Task f3818ffb-2ce4-4be0-8e3d-6f6ca86753ea: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:28:27,745::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:28:27,746::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:28:27,747::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:28:27,747::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:28:27,748::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:28:27,749::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:28:27,750::task::495::TaskManager.Task::(_debug) Task f3818ffb-2ce4-4be0-8e3d-6f6ca86753ea: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:28:27,750::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:28:27,751::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:28:27,752::task::495::TaskManager.Task::(_debug) Task 6df09087-84f6-4838-96dd-68d75a882a27: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:28:27,753::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e26075fe-c3c6-4f1c-b775-2e9e1d7c853a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:28:27,754::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:28:27,754::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:28:27,755::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e26075fe-c3c6-4f1c-b775-2e9e1d7c853a`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:28:27,756::task::495::TaskManager.Task::(_debug) Task 6df09087-84f6-4838-96dd-68d75a882a27: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:28:27,756::task::495::TaskManager.Task::(_debug) Task 6df09087-84f6-4838-96dd-68d75a882a27: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:28:27,757::task::495::TaskManager.Task::(_debug) Task 6df09087-84f6-4838-96dd-68d75a882a27: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:28:27,758::task::495::TaskManager.Task::(_debug) Task 6df09087-84f6-4838-96dd-68d75a882a27: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:28:27,758::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:28:27,759::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:28:27,759::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:28:27,760::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:28:27,760::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:28:27,761::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:28:27,762::task::495::TaskManager.Task::(_debug) Task 6df09087-84f6-4838-96dd-68d75a882a27: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:28:27,762::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:28:28,945::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 04:28:29,146::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:28:29,147::task::495::TaskManager.Task::(_debug) Task e5d5d240-75db-4fac-9338-040208b0b059: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:28:29,149::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`78687e1a-7d1c-47e9-adc8-ef4b7437b681`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:28:29,149::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:28:29,150::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:28:29,150::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`78687e1a-7d1c-47e9-adc8-ef4b7437b681`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:28:29,151::task::495::TaskManager.Task::(_debug) Task e5d5d240-75db-4fac-9338-040208b0b059: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:28:29,151::task::495::TaskManager.Task::(_debug) Task e5d5d240-75db-4fac-9338-040208b0b059: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:28:29,152::task::495::TaskManager.Task::(_debug) Task e5d5d240-75db-4fac-9338-040208b0b059: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:28:29,152::task::495::TaskManager.Task::(_debug) Task e5d5d240-75db-4fac-9338-040208b0b059: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:28:29,153::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:28:29,153::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:28:29,153::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:28:29,154::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:28:29,154::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:28:29,155::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:28:29,155::task::495::TaskManager.Task::(_debug) Task e5d5d240-75db-4fac-9338-040208b0b059: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:28:29,155::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:28:29,483::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0879842 s, 11.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:28:31,499::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:28:32,099::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0814286 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:28:34,119::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:28:34,694::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0831666 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:28:36,707::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:28:37,247::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0832308 s, 12.3 MB/s\n'; <rc> = 0
>Thread-15916482::DEBUG::2013-11-20 04:28:37,557::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916482::INFO::2013-11-20 04:28:37,559::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916482::DEBUG::2013-11-20 04:28:37,559::task::495::TaskManager.Task::(_debug) Task 167c7100-8840-4cde-ba3c-b274191b0c07: moving from state init -> state preparing
>Thread-15916482::DEBUG::2013-11-20 04:28:37,561::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916482::DEBUG::2013-11-20 04:28:37,561::task::495::TaskManager.Task::(_debug) Task 167c7100-8840-4cde-ba3c-b274191b0c07: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916482::DEBUG::2013-11-20 04:28:37,562::task::495::TaskManager.Task::(_debug) Task 167c7100-8840-4cde-ba3c-b274191b0c07: moving from state preparing -> state finished
>Thread-15916482::DEBUG::2013-11-20 04:28:37,563::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916482::DEBUG::2013-11-20 04:28:37,563::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916482::DEBUG::2013-11-20 04:28:37,564::task::495::TaskManager.Task::(_debug) Task 167c7100-8840-4cde-ba3c-b274191b0c07: ref 0 aborting False
>Thread-15916482::INFO::2013-11-20 04:28:37,565::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916483::DEBUG::2013-11-20 04:28:37,570::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916483::INFO::2013-11-20 04:28:37,571::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916483::DEBUG::2013-11-20 04:28:37,572::task::495::TaskManager.Task::(_debug) Task 9a53d02c-cf41-49cb-a091-4576be595140: moving from state init -> state preparing
>Thread-15916483::DEBUG::2013-11-20 04:28:37,573::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`dc904724-897a-4d0e-9d4f-7f8aeb5ec095`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916483::DEBUG::2013-11-20 04:28:37,574::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916483::DEBUG::2013-11-20 04:28:37,575::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916483::DEBUG::2013-11-20 04:28:37,575::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`dc904724-897a-4d0e-9d4f-7f8aeb5ec095`::Granted request
>Thread-15916483::DEBUG::2013-11-20 04:28:37,584::task::495::TaskManager.Task::(_debug) Task 9a53d02c-cf41-49cb-a091-4576be595140: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916483::DEBUG::2013-11-20 04:28:37,584::task::495::TaskManager.Task::(_debug) Task 9a53d02c-cf41-49cb-a091-4576be595140: ref 1 aborting False
>Thread-15916483::WARNING::2013-11-20 04:28:37,586::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916483::DEBUG::2013-11-20 04:28:37,587::task::495::TaskManager.Task::(_debug) Task 9a53d02c-cf41-49cb-a091-4576be595140: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916483::DEBUG::2013-11-20 04:28:37,588::task::495::TaskManager.Task::(_debug) Task 9a53d02c-cf41-49cb-a091-4576be595140: moving from state preparing -> state finished
>Thread-15916483::DEBUG::2013-11-20 04:28:37,589::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916483::DEBUG::2013-11-20 04:28:37,589::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916483::DEBUG::2013-11-20 04:28:37,590::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916483::DEBUG::2013-11-20 04:28:37,590::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916483::DEBUG::2013-11-20 04:28:37,591::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916483::DEBUG::2013-11-20 04:28:37,591::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916483::DEBUG::2013-11-20 04:28:37,592::task::495::TaskManager.Task::(_debug) Task 9a53d02c-cf41-49cb-a091-4576be595140: ref 0 aborting False
>Thread-15916483::INFO::2013-11-20 04:28:37,593::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916484::INFO::2013-11-20 04:28:39,103::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916484::DEBUG::2013-11-20 04:28:39,104::task::495::TaskManager.Task::(_debug) Task 9777cdbc-e5c0-450a-923c-5b136803631e: moving from state init -> state preparing
>Thread-15916484::DEBUG::2013-11-20 04:28:39,105::task::495::TaskManager.Task::(_debug) Task 9777cdbc-e5c0-450a-923c-5b136803631e: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000530958175659', 'lastCheck': 1384921710.6084809, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0217761993408', 'lastCheck': 1384921718.3630121, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00205302238464', 'lastCheck': 1384921715.337208, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000435829162598', 'lastCheck': 1384921715.337815, 'valid': True, 'code': 0}}
>Thread-15916484::DEBUG::2013-11-20 04:28:39,106::task::495::TaskManager.Task::(_debug) Task 9777cdbc-e5c0-450a-923c-5b136803631e: moving from state preparing -> state finished
>Thread-15916484::DEBUG::2013-11-20 04:28:39,107::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916484::DEBUG::2013-11-20 04:28:39,107::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916484::DEBUG::2013-11-20 04:28:39,108::task::495::TaskManager.Task::(_debug) Task 9777cdbc-e5c0-450a-923c-5b136803631e: ref 0 aborting False
>Thread-15916484::INFO::2013-11-20 04:28:39,108::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000530958175659', 'lastCheck': 1384921710.6084809, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0217761993408', 'lastCheck': 1384921718.3630121, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00205302238464', 'lastCheck': 1384921715.337208, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000435829162598', 'lastCheck': 1384921715.337815, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:28:39,263::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:28:39,809::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0850512 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:28:41,823::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:28:42,336::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0746589 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:28:44,350::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:28:44,903::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0775834 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15913639::INFO::2013-11-20 04:28:45,750::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:28:45,751::task::495::TaskManager.Task::(_debug) Task 2b037c6c-71d1-41a7-b849-d6c2e09d516d: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:28:45,753::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1cc1f40d-7176-4084-81da-99a7193d69ba`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:28:45,755::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:28:45,756::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:28:45,757::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1cc1f40d-7176-4084-81da-99a7193d69ba`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:28:45,759::task::495::TaskManager.Task::(_debug) Task 2b037c6c-71d1-41a7-b849-d6c2e09d516d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:28:45,759::task::495::TaskManager.Task::(_debug) Task 2b037c6c-71d1-41a7-b849-d6c2e09d516d: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:28:45,760::task::495::TaskManager.Task::(_debug) Task 2b037c6c-71d1-41a7-b849-d6c2e09d516d: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:28:45,761::task::495::TaskManager.Task::(_debug) Task 2b037c6c-71d1-41a7-b849-d6c2e09d516d: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:28:45,761::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:28:45,762::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:28:45,769::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:28:45,771::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::INFO::2013-11-20 04:28:45,772::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15913639::DEBUG::2013-11-20 04:28:45,772::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:28:45,773::task::495::TaskManager.Task::(_debug) Task 485bffc0-3ae2-459d-a3e9-327596c43067: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:28:45,774::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:28:45,776::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4f8a0848-44b8-42cb-badf-8cd72d72a4c3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:28:45,777::task::495::TaskManager.Task::(_debug) Task 2b037c6c-71d1-41a7-b849-d6c2e09d516d: ref 0 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:28:45,777::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::INFO::2013-11-20 04:28:45,778::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15915918::DEBUG::2013-11-20 04:28:45,779::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:28:45,780::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4f8a0848-44b8-42cb-badf-8cd72d72a4c3`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:28:45,782::task::495::TaskManager.Task::(_debug) Task 485bffc0-3ae2-459d-a3e9-327596c43067: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:28:45,784::task::495::TaskManager.Task::(_debug) Task 485bffc0-3ae2-459d-a3e9-327596c43067: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:28:45,785::task::495::TaskManager.Task::(_debug) Task 485bffc0-3ae2-459d-a3e9-327596c43067: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:28:45,786::task::495::TaskManager.Task::(_debug) Task 485bffc0-3ae2-459d-a3e9-327596c43067: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:28:45,787::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:28:45,787::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:28:45,788::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:28:45,789::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:28:45,790::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:28:45,791::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:28:45,792::task::495::TaskManager.Task::(_debug) Task 485bffc0-3ae2-459d-a3e9-327596c43067: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:28:45,793::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:28:46,918::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:28:47,499::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0900945 s, 11.4 MB/s\n'; <rc> = 0
>Thread-15916488::DEBUG::2013-11-20 04:28:47,975::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916488::INFO::2013-11-20 04:28:47,976::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916488::DEBUG::2013-11-20 04:28:47,977::task::495::TaskManager.Task::(_debug) Task 80a7bee0-8d95-4d46-a26d-1a21db1d1473: moving from state init -> state preparing
>Thread-15916488::DEBUG::2013-11-20 04:28:47,978::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916488::DEBUG::2013-11-20 04:28:47,978::task::495::TaskManager.Task::(_debug) Task 80a7bee0-8d95-4d46-a26d-1a21db1d1473: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916488::DEBUG::2013-11-20 04:28:47,979::task::495::TaskManager.Task::(_debug) Task 80a7bee0-8d95-4d46-a26d-1a21db1d1473: moving from state preparing -> state finished
>Thread-15916488::DEBUG::2013-11-20 04:28:47,980::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916488::DEBUG::2013-11-20 04:28:47,980::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916488::DEBUG::2013-11-20 04:28:47,981::task::495::TaskManager.Task::(_debug) Task 80a7bee0-8d95-4d46-a26d-1a21db1d1473: ref 0 aborting False
>Thread-15916488::INFO::2013-11-20 04:28:47,982::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916489::DEBUG::2013-11-20 04:28:47,987::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916489::INFO::2013-11-20 04:28:47,989::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916489::DEBUG::2013-11-20 04:28:47,990::task::495::TaskManager.Task::(_debug) Task 01a7a839-3b7e-4f19-bf00-4d323139f7a8: moving from state init -> state preparing
>Thread-15916489::DEBUG::2013-11-20 04:28:47,992::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4a56da53-fb3d-45eb-97f2-2d0a39b9ad25`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916489::DEBUG::2013-11-20 04:28:47,992::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916489::DEBUG::2013-11-20 04:28:47,994::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916489::DEBUG::2013-11-20 04:28:47,994::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4a56da53-fb3d-45eb-97f2-2d0a39b9ad25`::Granted request
>Thread-15916489::DEBUG::2013-11-20 04:28:47,996::task::495::TaskManager.Task::(_debug) Task 01a7a839-3b7e-4f19-bf00-4d323139f7a8: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916489::DEBUG::2013-11-20 04:28:47,996::task::495::TaskManager.Task::(_debug) Task 01a7a839-3b7e-4f19-bf00-4d323139f7a8: ref 1 aborting False
>Thread-15916489::WARNING::2013-11-20 04:28:47,999::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916489::DEBUG::2013-11-20 04:28:48,000::task::495::TaskManager.Task::(_debug) Task 01a7a839-3b7e-4f19-bf00-4d323139f7a8: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916489::DEBUG::2013-11-20 04:28:48,001::task::495::TaskManager.Task::(_debug) Task 01a7a839-3b7e-4f19-bf00-4d323139f7a8: moving from state preparing -> state finished
>Thread-15916489::DEBUG::2013-11-20 04:28:48,002::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916489::DEBUG::2013-11-20 04:28:48,003::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916489::DEBUG::2013-11-20 04:28:48,004::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916489::DEBUG::2013-11-20 04:28:48,004::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916489::DEBUG::2013-11-20 04:28:48,005::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916489::DEBUG::2013-11-20 04:28:48,005::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916489::DEBUG::2013-11-20 04:28:48,006::task::495::TaskManager.Task::(_debug) Task 01a7a839-3b7e-4f19-bf00-4d323139f7a8: ref 0 aborting False
>Thread-15916489::INFO::2013-11-20 04:28:48,007::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:28:49,519::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:28:50,086::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0801925 s, 12.8 MB/s\n'; <rc> = 0
>Thread-15915863::INFO::2013-11-20 04:28:50,089::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:28:50,097::task::495::TaskManager.Task::(_debug) Task cb6e0a3c-c89d-4082-8bfe-024ab827eb24: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:28:50,115::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f116717b-f24b-405e-b6ae-795c08c2e604`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:28:50,116::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:28:50,117::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:28:50,118::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f116717b-f24b-405e-b6ae-795c08c2e604`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:28:50,119::task::495::TaskManager.Task::(_debug) Task cb6e0a3c-c89d-4082-8bfe-024ab827eb24: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:28:50,120::task::495::TaskManager.Task::(_debug) Task cb6e0a3c-c89d-4082-8bfe-024ab827eb24: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:28:50,121::task::495::TaskManager.Task::(_debug) Task cb6e0a3c-c89d-4082-8bfe-024ab827eb24: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:28:50,122::task::495::TaskManager.Task::(_debug) Task cb6e0a3c-c89d-4082-8bfe-024ab827eb24: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:28:50,122::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:28:50,123::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:28:50,124::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:28:50,125::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:28:50,126::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:28:50,126::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:28:50,127::task::495::TaskManager.Task::(_debug) Task cb6e0a3c-c89d-4082-8bfe-024ab827eb24: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:28:50,128::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:28:52,118::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:28:52,688::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0943416 s, 10.9 MB/s\n'; <rc> = 0
>Thread-15916492::INFO::2013-11-20 04:28:54,650::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916492::DEBUG::2013-11-20 04:28:54,651::task::495::TaskManager.Task::(_debug) Task d69c46c8-e841-4fd3-abfd-72634ced7d3c: moving from state init -> state preparing
>Thread-15916492::DEBUG::2013-11-20 04:28:54,652::task::495::TaskManager.Task::(_debug) Task d69c46c8-e841-4fd3-abfd-72634ced7d3c: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000417947769165', 'lastCheck': 1384921731.128675, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00164580345154', 'lastCheck': 1384921728.5732119, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00131487846375', 'lastCheck': 1384921725.5710521, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000463962554932', 'lastCheck': 1384921725.56796, 'valid': True, 'code': 0}}
>Thread-15916492::DEBUG::2013-11-20 04:28:54,653::task::495::TaskManager.Task::(_debug) Task d69c46c8-e841-4fd3-abfd-72634ced7d3c: moving from state preparing -> state finished
>Thread-15916492::DEBUG::2013-11-20 04:28:54,654::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916492::DEBUG::2013-11-20 04:28:54,655::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916492::DEBUG::2013-11-20 04:28:54,655::task::495::TaskManager.Task::(_debug) Task d69c46c8-e841-4fd3-abfd-72634ced7d3c: ref 0 aborting False
>Thread-15916492::INFO::2013-11-20 04:28:54,656::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000417947769165', 'lastCheck': 1384921731.128675, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00164580345154', 'lastCheck': 1384921728.5732119, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00131487846375', 'lastCheck': 1384921725.5710521, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000463962554932', 'lastCheck': 1384921725.56796, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:28:54,700::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:28:55,274::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0740207 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:28:57,289::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 04:28:57,768::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:28:57,772::task::495::TaskManager.Task::(_debug) Task bedfb7ee-7e5e-41d9-8589-bec813798eb3: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:28:57,775::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e2398198-87f4-4456-b818-809ea5c3c1e8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:28:57,776::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:28:57,777::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:28:57,778::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e2398198-87f4-4456-b818-809ea5c3c1e8`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:28:57,779::task::495::TaskManager.Task::(_debug) Task bedfb7ee-7e5e-41d9-8589-bec813798eb3: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:28:57,780::task::495::TaskManager.Task::(_debug) Task bedfb7ee-7e5e-41d9-8589-bec813798eb3: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:28:57,781::task::495::TaskManager.Task::(_debug) Task bedfb7ee-7e5e-41d9-8589-bec813798eb3: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:28:57,782::task::495::TaskManager.Task::(_debug) Task bedfb7ee-7e5e-41d9-8589-bec813798eb3: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:28:57,783::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:28:57,784::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:28:57,785::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:28:57,786::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:28:57,786::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:28:57,787::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:28:57,788::task::495::TaskManager.Task::(_debug) Task bedfb7ee-7e5e-41d9-8589-bec813798eb3: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:28:57,789::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:28:57,863::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0986896 s, 10.4 MB/s\n'; <rc> = 0
>Thread-15916494::DEBUG::2013-11-20 04:28:58,330::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916494::INFO::2013-11-20 04:28:58,331::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916494::DEBUG::2013-11-20 04:28:58,332::task::495::TaskManager.Task::(_debug) Task 04b5f4ee-c9f7-431d-87d7-02a6dc1a88c7: moving from state init -> state preparing
>Thread-15916494::DEBUG::2013-11-20 04:28:58,333::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916494::DEBUG::2013-11-20 04:28:58,333::task::495::TaskManager.Task::(_debug) Task 04b5f4ee-c9f7-431d-87d7-02a6dc1a88c7: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916494::DEBUG::2013-11-20 04:28:58,334::task::495::TaskManager.Task::(_debug) Task 04b5f4ee-c9f7-431d-87d7-02a6dc1a88c7: moving from state preparing -> state finished
>Thread-15916494::DEBUG::2013-11-20 04:28:58,335::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916494::DEBUG::2013-11-20 04:28:58,335::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916494::DEBUG::2013-11-20 04:28:58,336::task::495::TaskManager.Task::(_debug) Task 04b5f4ee-c9f7-431d-87d7-02a6dc1a88c7: ref 0 aborting False
>Thread-15916494::INFO::2013-11-20 04:28:58,336::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916495::DEBUG::2013-11-20 04:28:58,341::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916495::INFO::2013-11-20 04:28:58,341::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916495::DEBUG::2013-11-20 04:28:58,342::task::495::TaskManager.Task::(_debug) Task e9679f81-851e-465d-a851-1787527bb8b4: moving from state init -> state preparing
>Thread-15916495::DEBUG::2013-11-20 04:28:58,343::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`612d9ac4-a876-48b9-834f-f2ca0372fa53`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916495::DEBUG::2013-11-20 04:28:58,344::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916495::DEBUG::2013-11-20 04:28:58,345::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916495::DEBUG::2013-11-20 04:28:58,345::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`612d9ac4-a876-48b9-834f-f2ca0372fa53`::Granted request
>Thread-15916495::DEBUG::2013-11-20 04:28:58,346::task::495::TaskManager.Task::(_debug) Task e9679f81-851e-465d-a851-1787527bb8b4: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916495::DEBUG::2013-11-20 04:28:58,347::task::495::TaskManager.Task::(_debug) Task e9679f81-851e-465d-a851-1787527bb8b4: ref 1 aborting False
>Thread-15916495::WARNING::2013-11-20 04:28:58,351::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916495::DEBUG::2013-11-20 04:28:58,352::task::495::TaskManager.Task::(_debug) Task e9679f81-851e-465d-a851-1787527bb8b4: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916495::DEBUG::2013-11-20 04:28:58,354::task::495::TaskManager.Task::(_debug) Task e9679f81-851e-465d-a851-1787527bb8b4: moving from state preparing -> state finished
>Thread-15916495::DEBUG::2013-11-20 04:28:58,354::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916495::DEBUG::2013-11-20 04:28:58,355::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916495::DEBUG::2013-11-20 04:28:58,357::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916495::DEBUG::2013-11-20 04:28:58,358::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916495::DEBUG::2013-11-20 04:28:58,360::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916495::DEBUG::2013-11-20 04:28:58,361::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916495::DEBUG::2013-11-20 04:28:58,362::task::495::TaskManager.Task::(_debug) Task e9679f81-851e-465d-a851-1787527bb8b4: ref 0 aborting False
>Thread-15916495::INFO::2013-11-20 04:28:58,364::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913518::INFO::2013-11-20 04:28:58,983::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:28:58,984::task::495::TaskManager.Task::(_debug) Task 0b5e43b5-b6c8-428a-bfcf-b8f62c72bb1b: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:28:58,986::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d0441aff-5a34-464c-afcb-7ed40961ed95`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:28:58,986::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:28:58,987::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:28:58,988::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d0441aff-5a34-464c-afcb-7ed40961ed95`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:28:58,989::task::495::TaskManager.Task::(_debug) Task 0b5e43b5-b6c8-428a-bfcf-b8f62c72bb1b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:28:58,990::task::495::TaskManager.Task::(_debug) Task 0b5e43b5-b6c8-428a-bfcf-b8f62c72bb1b: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:28:58,990::task::495::TaskManager.Task::(_debug) Task 0b5e43b5-b6c8-428a-bfcf-b8f62c72bb1b: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:28:58,991::task::495::TaskManager.Task::(_debug) Task 0b5e43b5-b6c8-428a-bfcf-b8f62c72bb1b: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:28:58,992::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:28:58,992::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:28:58,993::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:28:58,994::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:28:58,994::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:28:58,995::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:28:58,996::task::495::TaskManager.Task::(_debug) Task 0b5e43b5-b6c8-428a-bfcf-b8f62c72bb1b: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:28:58,996::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:28:58,997::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:28:58,998::task::495::TaskManager.Task::(_debug) Task 358a614e-eb93-4b55-ad04-3f4ffce6a3cb: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:28:58,999::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5a174c03-2e6d-4e0a-ae85-b8bf1b7833d8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:28:58,999::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:28:59,000::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:28:59,001::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5a174c03-2e6d-4e0a-ae85-b8bf1b7833d8`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:28:59,002::task::495::TaskManager.Task::(_debug) Task 358a614e-eb93-4b55-ad04-3f4ffce6a3cb: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:28:59,003::task::495::TaskManager.Task::(_debug) Task 358a614e-eb93-4b55-ad04-3f4ffce6a3cb: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:28:59,003::task::495::TaskManager.Task::(_debug) Task 358a614e-eb93-4b55-ad04-3f4ffce6a3cb: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:28:59,004::task::495::TaskManager.Task::(_debug) Task 358a614e-eb93-4b55-ad04-3f4ffce6a3cb: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:28:59,005::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:28:59,005::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:28:59,006::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:28:59,006::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:28:59,007::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:28:59,007::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:28:59,008::task::495::TaskManager.Task::(_debug) Task 358a614e-eb93-4b55-ad04-3f4ffce6a3cb: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:28:59,009::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:28:59,879::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:29:00,415::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.090947 s, 11.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:29:02,431::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:29:02,976::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.087261 s, 11.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:29:04,991::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:29:05,506::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0562738 s, 18.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:29:07,516::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 04:29:07,784::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:29:07,787::task::495::TaskManager.Task::(_debug) Task de7d41ff-5abf-4ffe-978d-a1eddff817c6: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:29:07,790::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`94829f88-0653-4443-ab8e-cf763cc7b87e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:29:07,791::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:29:07,792::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:29:07,793::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`94829f88-0653-4443-ab8e-cf763cc7b87e`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:29:07,794::task::495::TaskManager.Task::(_debug) Task de7d41ff-5abf-4ffe-978d-a1eddff817c6: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:29:07,795::task::495::TaskManager.Task::(_debug) Task de7d41ff-5abf-4ffe-978d-a1eddff817c6: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:29:07,796::task::495::TaskManager.Task::(_debug) Task de7d41ff-5abf-4ffe-978d-a1eddff817c6: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:29:07,796::task::495::TaskManager.Task::(_debug) Task de7d41ff-5abf-4ffe-978d-a1eddff817c6: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:29:07,797::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:29:07,798::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:29:07,799::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:29:07,799::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:29:07,800::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:29:07,800::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:29:07,801::task::495::TaskManager.Task::(_debug) Task de7d41ff-5abf-4ffe-978d-a1eddff817c6: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:29:07,802::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:29:08,066::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0898706 s, 11.4 MB/s\n'; <rc> = 0
>Thread-15916500::DEBUG::2013-11-20 04:29:08,794::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916500::INFO::2013-11-20 04:29:08,795::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916500::DEBUG::2013-11-20 04:29:08,795::task::495::TaskManager.Task::(_debug) Task 9703658c-8261-4211-adb7-2a6b41f2bb44: moving from state init -> state preparing
>Thread-15916500::DEBUG::2013-11-20 04:29:08,796::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916500::DEBUG::2013-11-20 04:29:08,797::task::495::TaskManager.Task::(_debug) Task 9703658c-8261-4211-adb7-2a6b41f2bb44: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916500::DEBUG::2013-11-20 04:29:08,798::task::495::TaskManager.Task::(_debug) Task 9703658c-8261-4211-adb7-2a6b41f2bb44: moving from state preparing -> state finished
>Thread-15916500::DEBUG::2013-11-20 04:29:08,798::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916500::DEBUG::2013-11-20 04:29:08,799::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916500::DEBUG::2013-11-20 04:29:08,800::task::495::TaskManager.Task::(_debug) Task 9703658c-8261-4211-adb7-2a6b41f2bb44: ref 0 aborting False
>Thread-15916500::INFO::2013-11-20 04:29:08,800::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916501::DEBUG::2013-11-20 04:29:08,805::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916501::INFO::2013-11-20 04:29:08,806::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916501::DEBUG::2013-11-20 04:29:08,807::task::495::TaskManager.Task::(_debug) Task 27fb55d2-f784-4924-94fb-4742976fd68d: moving from state init -> state preparing
>Thread-15916501::DEBUG::2013-11-20 04:29:08,809::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1bbd4d76-80f7-41c3-a03d-cdf0ff4ba380`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916501::DEBUG::2013-11-20 04:29:08,810::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916501::DEBUG::2013-11-20 04:29:08,811::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916501::DEBUG::2013-11-20 04:29:08,812::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1bbd4d76-80f7-41c3-a03d-cdf0ff4ba380`::Granted request
>Thread-15916501::DEBUG::2013-11-20 04:29:08,813::task::495::TaskManager.Task::(_debug) Task 27fb55d2-f784-4924-94fb-4742976fd68d: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916501::DEBUG::2013-11-20 04:29:08,814::task::495::TaskManager.Task::(_debug) Task 27fb55d2-f784-4924-94fb-4742976fd68d: ref 1 aborting False
>Thread-15916501::WARNING::2013-11-20 04:29:08,816::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916501::DEBUG::2013-11-20 04:29:08,817::task::495::TaskManager.Task::(_debug) Task 27fb55d2-f784-4924-94fb-4742976fd68d: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916501::DEBUG::2013-11-20 04:29:08,818::task::495::TaskManager.Task::(_debug) Task 27fb55d2-f784-4924-94fb-4742976fd68d: moving from state preparing -> state finished
>Thread-15916501::DEBUG::2013-11-20 04:29:08,818::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916501::DEBUG::2013-11-20 04:29:08,819::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916501::DEBUG::2013-11-20 04:29:08,820::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916501::DEBUG::2013-11-20 04:29:08,820::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916501::DEBUG::2013-11-20 04:29:08,821::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916501::DEBUG::2013-11-20 04:29:08,821::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916501::DEBUG::2013-11-20 04:29:08,822::task::495::TaskManager.Task::(_debug) Task 27fb55d2-f784-4924-94fb-4742976fd68d: ref 0 aborting False
>Thread-15916501::INFO::2013-11-20 04:29:08,823::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:29:10,077::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:29:10,626::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0798981 s, 12.8 MB/s\n'; <rc> = 0
>Thread-15916502::INFO::2013-11-20 04:29:11,268::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916502::DEBUG::2013-11-20 04:29:11,269::task::495::TaskManager.Task::(_debug) Task 407ce234-1968-4bdf-b399-5c8c2c3349ad: moving from state init -> state preparing
>Thread-15916502::DEBUG::2013-11-20 04:29:11,269::task::495::TaskManager.Task::(_debug) Task 407ce234-1968-4bdf-b399-5c8c2c3349ad: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00049614906311', 'lastCheck': 1384921741.504401, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0192899703979', 'lastCheck': 1384921749.2417021, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00157594680786', 'lastCheck': 1384921746.2218871, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000550031661987', 'lastCheck': 1384921746.2162211, 'valid': True, 'code': 0}}
>Thread-15916502::DEBUG::2013-11-20 04:29:11,270::task::495::TaskManager.Task::(_debug) Task 407ce234-1968-4bdf-b399-5c8c2c3349ad: moving from state preparing -> state finished
>Thread-15916502::DEBUG::2013-11-20 04:29:11,271::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916502::DEBUG::2013-11-20 04:29:11,271::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916502::DEBUG::2013-11-20 04:29:11,272::task::495::TaskManager.Task::(_debug) Task 407ce234-1968-4bdf-b399-5c8c2c3349ad: ref 0 aborting False
>Thread-15916502::INFO::2013-11-20 04:29:11,273::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00049614906311', 'lastCheck': 1384921741.504401, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0192899703979', 'lastCheck': 1384921749.2417021, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00157594680786', 'lastCheck': 1384921746.2218871, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000550031661987', 'lastCheck': 1384921746.2162211, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:29:12,640::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:29:13,193::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0782574 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:29:15,209::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 04:29:15,452::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:29:15,453::task::495::TaskManager.Task::(_debug) Task 0630ac0e-2f74-429f-9ca1-23b82a6fb750: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:29:15,456::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`48412a04-c318-4559-8aba-0e47c5f435a1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:29:15,457::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:29:15,458::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:29:15,458::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`48412a04-c318-4559-8aba-0e47c5f435a1`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:29:15,460::task::495::TaskManager.Task::(_debug) Task 0630ac0e-2f74-429f-9ca1-23b82a6fb750: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:29:15,460::task::495::TaskManager.Task::(_debug) Task 0630ac0e-2f74-429f-9ca1-23b82a6fb750: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:29:15,461::task::495::TaskManager.Task::(_debug) Task 0630ac0e-2f74-429f-9ca1-23b82a6fb750: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:29:15,462::task::495::TaskManager.Task::(_debug) Task 0630ac0e-2f74-429f-9ca1-23b82a6fb750: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:29:15,462::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:29:15,463::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:29:15,464::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:29:15,465::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:29:15,465::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:29:15,466::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:29:15,467::task::495::TaskManager.Task::(_debug) Task 0630ac0e-2f74-429f-9ca1-23b82a6fb750: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:29:15,467::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:29:15,770::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0776654 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:29:17,788::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 04:29:18,008::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:29:18,011::task::495::TaskManager.Task::(_debug) Task 4b4de89c-f989-4c56-99fd-6a88100648e4: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:29:18,014::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3b3282bf-9e08-4afa-8013-c2d8344f480a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:29:18,018::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:29:18,020::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:29:18,020::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3b3282bf-9e08-4afa-8013-c2d8344f480a`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:29:18,021::task::495::TaskManager.Task::(_debug) Task 4b4de89c-f989-4c56-99fd-6a88100648e4: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:29:18,022::task::495::TaskManager.Task::(_debug) Task 4b4de89c-f989-4c56-99fd-6a88100648e4: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:29:18,023::task::495::TaskManager.Task::(_debug) Task 4b4de89c-f989-4c56-99fd-6a88100648e4: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:29:18,024::task::495::TaskManager.Task::(_debug) Task 4b4de89c-f989-4c56-99fd-6a88100648e4: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:29:18,024::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:29:18,025::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:29:18,025::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:29:18,026::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:29:18,027::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:29:18,027::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:29:18,028::task::495::TaskManager.Task::(_debug) Task 4b4de89c-f989-4c56-99fd-6a88100648e4: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:29:18,028::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:29:18,338::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0739657 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15916506::DEBUG::2013-11-20 04:29:19,366::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916506::INFO::2013-11-20 04:29:19,368::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916506::DEBUG::2013-11-20 04:29:19,369::task::495::TaskManager.Task::(_debug) Task cb62ff9f-c35e-46c0-804b-fab70f0e924e: moving from state init -> state preparing
>Thread-15916506::DEBUG::2013-11-20 04:29:19,371::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916506::DEBUG::2013-11-20 04:29:19,373::task::495::TaskManager.Task::(_debug) Task cb62ff9f-c35e-46c0-804b-fab70f0e924e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916506::DEBUG::2013-11-20 04:29:19,374::task::495::TaskManager.Task::(_debug) Task cb62ff9f-c35e-46c0-804b-fab70f0e924e: moving from state preparing -> state finished
>Thread-15916506::DEBUG::2013-11-20 04:29:19,375::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916506::DEBUG::2013-11-20 04:29:19,376::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916506::DEBUG::2013-11-20 04:29:19,377::task::495::TaskManager.Task::(_debug) Task cb62ff9f-c35e-46c0-804b-fab70f0e924e: ref 0 aborting False
>Thread-15916506::INFO::2013-11-20 04:29:19,378::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916507::DEBUG::2013-11-20 04:29:19,383::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916507::INFO::2013-11-20 04:29:19,384::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916507::DEBUG::2013-11-20 04:29:19,385::task::495::TaskManager.Task::(_debug) Task 951795b9-dabf-41bd-b1a7-b288845ff6d1: moving from state init -> state preparing
>Thread-15916507::DEBUG::2013-11-20 04:29:19,386::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ab91376f-aaee-4341-9bae-b8f32020ad9a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916507::DEBUG::2013-11-20 04:29:19,387::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916507::DEBUG::2013-11-20 04:29:19,388::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916507::DEBUG::2013-11-20 04:29:19,388::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ab91376f-aaee-4341-9bae-b8f32020ad9a`::Granted request
>Thread-15916507::DEBUG::2013-11-20 04:29:19,389::task::495::TaskManager.Task::(_debug) Task 951795b9-dabf-41bd-b1a7-b288845ff6d1: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916507::DEBUG::2013-11-20 04:29:19,390::task::495::TaskManager.Task::(_debug) Task 951795b9-dabf-41bd-b1a7-b288845ff6d1: ref 1 aborting False
>Thread-15916507::WARNING::2013-11-20 04:29:19,392::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916507::DEBUG::2013-11-20 04:29:19,394::task::495::TaskManager.Task::(_debug) Task 951795b9-dabf-41bd-b1a7-b288845ff6d1: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916507::DEBUG::2013-11-20 04:29:19,394::task::495::TaskManager.Task::(_debug) Task 951795b9-dabf-41bd-b1a7-b288845ff6d1: moving from state preparing -> state finished
>Thread-15916507::DEBUG::2013-11-20 04:29:19,395::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916507::DEBUG::2013-11-20 04:29:19,396::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916507::DEBUG::2013-11-20 04:29:19,397::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916507::DEBUG::2013-11-20 04:29:19,397::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916507::DEBUG::2013-11-20 04:29:19,398::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916507::DEBUG::2013-11-20 04:29:19,398::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916507::DEBUG::2013-11-20 04:29:19,399::task::495::TaskManager.Task::(_debug) Task 951795b9-dabf-41bd-b1a7-b288845ff6d1: ref 0 aborting False
>Thread-15916507::INFO::2013-11-20 04:29:19,400::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:29:20,355::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:29:20,769::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0554134 s, 18.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:29:22,780::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:29:23,345::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0898675 s, 11.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:29:25,360::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 04:29:25,596::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:29:25,607::task::495::TaskManager.Task::(_debug) Task e4292592-3da9-4078-a56f-cfcac647e2b3: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:29:25,611::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8601f07f-483d-400d-8510-2ad06fbc4a95`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:29:25,613::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:29:25,615::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:29:25,616::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8601f07f-483d-400d-8510-2ad06fbc4a95`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:29:25,619::task::495::TaskManager.Task::(_debug) Task e4292592-3da9-4078-a56f-cfcac647e2b3: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 04:29:25,620::task::495::TaskManager.Task::(_debug) Task e4292592-3da9-4078-a56f-cfcac647e2b3: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:29:25,621::task::495::TaskManager.Task::(_debug) Task e4292592-3da9-4078-a56f-cfcac647e2b3: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 04:29:25,623::task::495::TaskManager.Task::(_debug) Task e4292592-3da9-4078-a56f-cfcac647e2b3: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:29:25,624::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:29:25,625::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:29:25,629::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 04:29:25,630::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:29:25,631::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:29:25,632::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:29:25,633::task::495::TaskManager.Task::(_debug) Task e4292592-3da9-4078-a56f-cfcac647e2b3: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 04:29:25,634::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 04:29:25,938::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0878339 s, 11.7 MB/s\n'; <rc> = 0
>Thread-15916510::INFO::2013-11-20 04:29:27,081::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916510::DEBUG::2013-11-20 04:29:27,082::task::495::TaskManager.Task::(_debug) Task 8f85bc7d-2022-4200-81df-7fbb14512230: moving from state init -> state preparing
>Thread-15916510::DEBUG::2013-11-20 04:29:27,083::task::495::TaskManager.Task::(_debug) Task 8f85bc7d-2022-4200-81df-7fbb14512230: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000340938568115', 'lastCheck': 1384921762.017092, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00164604187012', 'lastCheck': 1384921759.4436879, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.017303943634', 'lastCheck': 1384921766.5967181, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00054407119751', 'lastCheck': 1384921766.575304, 'valid': True, 'code': 0}}
>Thread-15916510::DEBUG::2013-11-20 04:29:27,084::task::495::TaskManager.Task::(_debug) Task 8f85bc7d-2022-4200-81df-7fbb14512230: moving from state preparing -> state finished
>Thread-15916510::DEBUG::2013-11-20 04:29:27,085::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916510::DEBUG::2013-11-20 04:29:27,085::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916510::DEBUG::2013-11-20 04:29:27,086::task::495::TaskManager.Task::(_debug) Task 8f85bc7d-2022-4200-81df-7fbb14512230: ref 0 aborting False
>Thread-15916510::INFO::2013-11-20 04:29:27,086::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000340938568115', 'lastCheck': 1384921762.017092, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00164604187012', 'lastCheck': 1384921759.4436879, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.017303943634', 'lastCheck': 1384921766.5967181, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00054407119751', 'lastCheck': 1384921766.575304, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:29:27,949::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 04:29:28,321::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:29:28,322::task::495::TaskManager.Task::(_debug) Task 03c428e0-e27b-4c0b-b6b5-5a743792091f: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:29:28,324::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`19676d55-482d-4a9f-af5f-466470af391f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:29:28,324::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:29:28,325::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:29:28,325::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`19676d55-482d-4a9f-af5f-466470af391f`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:29:28,326::task::495::TaskManager.Task::(_debug) Task 03c428e0-e27b-4c0b-b6b5-5a743792091f: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:29:28,327::task::495::TaskManager.Task::(_debug) Task 03c428e0-e27b-4c0b-b6b5-5a743792091f: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:29:28,327::task::495::TaskManager.Task::(_debug) Task 03c428e0-e27b-4c0b-b6b5-5a743792091f: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 04:29:28,328::task::495::TaskManager.Task::(_debug) Task 03c428e0-e27b-4c0b-b6b5-5a743792091f: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:29:28,328::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:29:28,329::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:29:28,329::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:29:28,329::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:29:28,330::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:29:28,330::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:29:28,331::task::495::TaskManager.Task::(_debug) Task 03c428e0-e27b-4c0b-b6b5-5a743792091f: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:29:28,331::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:29:28,331::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:29:28,332::task::495::TaskManager.Task::(_debug) Task 68864478-fc7a-4547-a6c9-c70b638980cd: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:29:28,332::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fa7b8c57-adb0-4b57-a6a2-c23db9368d54`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:29:28,333::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:29:28,333::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:29:28,334::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fa7b8c57-adb0-4b57-a6a2-c23db9368d54`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:29:28,334::task::495::TaskManager.Task::(_debug) Task 68864478-fc7a-4547-a6c9-c70b638980cd: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:29:28,334::task::495::TaskManager.Task::(_debug) Task 68864478-fc7a-4547-a6c9-c70b638980cd: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:29:28,335::task::495::TaskManager.Task::(_debug) Task 68864478-fc7a-4547-a6c9-c70b638980cd: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:29:28,335::task::495::TaskManager.Task::(_debug) Task 68864478-fc7a-4547-a6c9-c70b638980cd: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:29:28,336::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:29:28,336::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:29:28,336::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:29:28,337::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:29:28,337::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:29:28,337::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:29:28,338::task::495::TaskManager.Task::(_debug) Task 68864478-fc7a-4547-a6c9-c70b638980cd: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:29:28,338::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:29:28,339::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:29:28,339::task::495::TaskManager.Task::(_debug) Task d1917a79-b34b-4c6a-b154-d5dc982cbb0b: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:29:28,340::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d67479b8-8cd4-4c58-aee6-bc84803be843`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:29:28,340::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:29:28,340::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:29:28,341::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d67479b8-8cd4-4c58-aee6-bc84803be843`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:29:28,341::task::495::TaskManager.Task::(_debug) Task d1917a79-b34b-4c6a-b154-d5dc982cbb0b: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:29:28,342::task::495::TaskManager.Task::(_debug) Task d1917a79-b34b-4c6a-b154-d5dc982cbb0b: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:29:28,342::task::495::TaskManager.Task::(_debug) Task d1917a79-b34b-4c6a-b154-d5dc982cbb0b: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:29:28,342::task::495::TaskManager.Task::(_debug) Task d1917a79-b34b-4c6a-b154-d5dc982cbb0b: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:29:28,343::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:29:28,343::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:29:28,343::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:29:28,344::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:29:28,344::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:29:28,345::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:29:28,345::task::495::TaskManager.Task::(_debug) Task d1917a79-b34b-4c6a-b154-d5dc982cbb0b: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:29:28,345::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:29:28,346::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:29:28,346::task::495::TaskManager.Task::(_debug) Task 27f6ea89-50fc-4d4a-964d-164b69a64a48: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:29:28,347::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c0f0820f-8e5f-4890-a8f1-bfe5327df742`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:29:28,347::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:29:28,348::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:29:28,348::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c0f0820f-8e5f-4890-a8f1-bfe5327df742`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:29:28,349::task::495::TaskManager.Task::(_debug) Task 27f6ea89-50fc-4d4a-964d-164b69a64a48: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:29:28,349::task::495::TaskManager.Task::(_debug) Task 27f6ea89-50fc-4d4a-964d-164b69a64a48: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:29:28,349::task::495::TaskManager.Task::(_debug) Task 27f6ea89-50fc-4d4a-964d-164b69a64a48: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:29:28,350::task::495::TaskManager.Task::(_debug) Task 27f6ea89-50fc-4d4a-964d-164b69a64a48: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:29:28,351::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:29:28,351::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:29:28,351::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:29:28,352::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:29:28,353::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:29:28,353::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:29:28,354::task::495::TaskManager.Task::(_debug) Task 27f6ea89-50fc-4d4a-964d-164b69a64a48: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:29:28,354::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:29:28,527::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0867184 s, 11.8 MB/s\n'; <rc> = 0
>Thread-15916512::DEBUG::2013-11-20 04:29:30,057::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916512::INFO::2013-11-20 04:29:30,058::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916512::DEBUG::2013-11-20 04:29:30,059::task::495::TaskManager.Task::(_debug) Task 331763ea-d0bf-4dd1-af25-c86bb51e9929: moving from state init -> state preparing
>Thread-15916512::DEBUG::2013-11-20 04:29:30,060::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916512::DEBUG::2013-11-20 04:29:30,060::task::495::TaskManager.Task::(_debug) Task 331763ea-d0bf-4dd1-af25-c86bb51e9929: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916512::DEBUG::2013-11-20 04:29:30,061::task::495::TaskManager.Task::(_debug) Task 331763ea-d0bf-4dd1-af25-c86bb51e9929: moving from state preparing -> state finished
>Thread-15916512::DEBUG::2013-11-20 04:29:30,062::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916512::DEBUG::2013-11-20 04:29:30,062::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916512::DEBUG::2013-11-20 04:29:30,063::task::495::TaskManager.Task::(_debug) Task 331763ea-d0bf-4dd1-af25-c86bb51e9929: ref 0 aborting False
>Thread-15916512::INFO::2013-11-20 04:29:30,064::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916513::DEBUG::2013-11-20 04:29:30,069::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916513::INFO::2013-11-20 04:29:30,070::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916513::DEBUG::2013-11-20 04:29:30,070::task::495::TaskManager.Task::(_debug) Task 574ad03b-6e86-4aa2-b9f0-93f7e813561d: moving from state init -> state preparing
>Thread-15916513::DEBUG::2013-11-20 04:29:30,071::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`52e9a0e4-c8e5-46c6-b3e1-5f2f895228fe`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916513::DEBUG::2013-11-20 04:29:30,072::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916513::DEBUG::2013-11-20 04:29:30,073::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916513::DEBUG::2013-11-20 04:29:30,073::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`52e9a0e4-c8e5-46c6-b3e1-5f2f895228fe`::Granted request
>Thread-15916513::DEBUG::2013-11-20 04:29:30,084::task::495::TaskManager.Task::(_debug) Task 574ad03b-6e86-4aa2-b9f0-93f7e813561d: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916513::DEBUG::2013-11-20 04:29:30,085::task::495::TaskManager.Task::(_debug) Task 574ad03b-6e86-4aa2-b9f0-93f7e813561d: ref 1 aborting False
>Thread-15916513::WARNING::2013-11-20 04:29:30,087::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916513::DEBUG::2013-11-20 04:29:30,088::task::495::TaskManager.Task::(_debug) Task 574ad03b-6e86-4aa2-b9f0-93f7e813561d: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916513::DEBUG::2013-11-20 04:29:30,089::task::495::TaskManager.Task::(_debug) Task 574ad03b-6e86-4aa2-b9f0-93f7e813561d: moving from state preparing -> state finished
>Thread-15916513::DEBUG::2013-11-20 04:29:30,089::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916513::DEBUG::2013-11-20 04:29:30,090::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916513::DEBUG::2013-11-20 04:29:30,091::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916513::DEBUG::2013-11-20 04:29:30,091::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916513::DEBUG::2013-11-20 04:29:30,092::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916513::DEBUG::2013-11-20 04:29:30,092::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916513::DEBUG::2013-11-20 04:29:30,093::task::495::TaskManager.Task::(_debug) Task 574ad03b-6e86-4aa2-b9f0-93f7e813561d: ref 0 aborting False
>Thread-15916513::INFO::2013-11-20 04:29:30,094::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:29:30,542::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 04:29:30,767::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:29:30,769::task::495::TaskManager.Task::(_debug) Task fce07a6c-46ca-4f5b-b636-742eabf97a0b: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:29:30,775::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ed1e47a3-2aff-4159-ba92-610bb2eeb22a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:29:30,776::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:29:30,777::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:29:30,778::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ed1e47a3-2aff-4159-ba92-610bb2eeb22a`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:29:30,779::task::495::TaskManager.Task::(_debug) Task fce07a6c-46ca-4f5b-b636-742eabf97a0b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:29:30,780::task::495::TaskManager.Task::(_debug) Task fce07a6c-46ca-4f5b-b636-742eabf97a0b: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:29:30,780::task::495::TaskManager.Task::(_debug) Task fce07a6c-46ca-4f5b-b636-742eabf97a0b: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:29:30,781::task::495::TaskManager.Task::(_debug) Task fce07a6c-46ca-4f5b-b636-742eabf97a0b: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:29:30,782::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:29:30,783::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:29:30,783::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:29:30,784::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:29:30,784::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:29:30,784::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:29:30,785::task::495::TaskManager.Task::(_debug) Task fce07a6c-46ca-4f5b-b636-742eabf97a0b: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:29:30,785::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:29:31,099::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0767931 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:29:33,115::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:29:33,702::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0809263 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:29:35,713::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:29:36,187::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0662517 s, 15.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:29:38,200::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:29:38,751::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.081753 s, 12.5 MB/s\n'; <rc> = 0
>Thread-15916518::DEBUG::2013-11-20 04:29:40,730::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916518::INFO::2013-11-20 04:29:40,732::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916518::DEBUG::2013-11-20 04:29:40,733::task::495::TaskManager.Task::(_debug) Task 8d0da883-3342-496d-9127-f8363198f9fc: moving from state init -> state preparing
>Thread-15916518::DEBUG::2013-11-20 04:29:40,733::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916518::DEBUG::2013-11-20 04:29:40,734::task::495::TaskManager.Task::(_debug) Task 8d0da883-3342-496d-9127-f8363198f9fc: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916518::DEBUG::2013-11-20 04:29:40,735::task::495::TaskManager.Task::(_debug) Task 8d0da883-3342-496d-9127-f8363198f9fc: moving from state preparing -> state finished
>Thread-15916518::DEBUG::2013-11-20 04:29:40,735::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916518::DEBUG::2013-11-20 04:29:40,736::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916518::DEBUG::2013-11-20 04:29:40,737::task::495::TaskManager.Task::(_debug) Task 8d0da883-3342-496d-9127-f8363198f9fc: ref 0 aborting False
>Thread-15916518::INFO::2013-11-20 04:29:40,737::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916519::DEBUG::2013-11-20 04:29:40,743::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916519::INFO::2013-11-20 04:29:40,744::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916519::DEBUG::2013-11-20 04:29:40,745::task::495::TaskManager.Task::(_debug) Task f9040fe4-044e-485b-9438-e91cbff10f18: moving from state init -> state preparing
>Thread-15916519::DEBUG::2013-11-20 04:29:40,746::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6d4091d3-9345-49da-99a9-af34168e71b3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916519::DEBUG::2013-11-20 04:29:40,747::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916519::DEBUG::2013-11-20 04:29:40,748::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916519::DEBUG::2013-11-20 04:29:40,748::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6d4091d3-9345-49da-99a9-af34168e71b3`::Granted request
>Thread-15916519::DEBUG::2013-11-20 04:29:40,749::task::495::TaskManager.Task::(_debug) Task f9040fe4-044e-485b-9438-e91cbff10f18: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916519::DEBUG::2013-11-20 04:29:40,750::task::495::TaskManager.Task::(_debug) Task f9040fe4-044e-485b-9438-e91cbff10f18: ref 1 aborting False
>Thread-15916519::WARNING::2013-11-20 04:29:40,752::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916519::DEBUG::2013-11-20 04:29:40,753::task::495::TaskManager.Task::(_debug) Task f9040fe4-044e-485b-9438-e91cbff10f18: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916519::DEBUG::2013-11-20 04:29:40,754::task::495::TaskManager.Task::(_debug) Task f9040fe4-044e-485b-9438-e91cbff10f18: moving from state preparing -> state finished
>Thread-15916519::DEBUG::2013-11-20 04:29:40,754::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916519::DEBUG::2013-11-20 04:29:40,755::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916519::DEBUG::2013-11-20 04:29:40,756::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916519::DEBUG::2013-11-20 04:29:40,756::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916519::DEBUG::2013-11-20 04:29:40,757::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916519::DEBUG::2013-11-20 04:29:40,757::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916519::DEBUG::2013-11-20 04:29:40,758::task::495::TaskManager.Task::(_debug) Task f9040fe4-044e-485b-9438-e91cbff10f18: ref 0 aborting False
>Thread-15916519::INFO::2013-11-20 04:29:40,759::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:29:40,766::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:29:41,304::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0755978 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15916520::INFO::2013-11-20 04:29:42,884::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916520::DEBUG::2013-11-20 04:29:42,885::task::495::TaskManager.Task::(_debug) Task 6f977328-9174-4450-9fad-bf9536c60d97: moving from state init -> state preparing
>Thread-15916520::DEBUG::2013-11-20 04:29:42,886::task::495::TaskManager.Task::(_debug) Task 6f977328-9174-4450-9fad-bf9536c60d97: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000385046005249', 'lastCheck': 1384921782.4276669, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.029372215271', 'lastCheck': 1384921779.962513, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00170683860779', 'lastCheck': 1384921776.935648, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000612020492554', 'lastCheck': 1384921776.9324651, 'valid': True, 'code': 0}}
>Thread-15916520::DEBUG::2013-11-20 04:29:42,886::task::495::TaskManager.Task::(_debug) Task 6f977328-9174-4450-9fad-bf9536c60d97: moving from state preparing -> state finished
>Thread-15916520::DEBUG::2013-11-20 04:29:42,887::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916520::DEBUG::2013-11-20 04:29:42,888::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916520::DEBUG::2013-11-20 04:29:42,888::task::495::TaskManager.Task::(_debug) Task 6f977328-9174-4450-9fad-bf9536c60d97: ref 0 aborting False
>Thread-15916520::INFO::2013-11-20 04:29:42,889::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000385046005249', 'lastCheck': 1384921782.4276669, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.029372215271', 'lastCheck': 1384921779.962513, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00170683860779', 'lastCheck': 1384921776.935648, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000612020492554', 'lastCheck': 1384921776.9324651, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:29:43,317::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:29:43,889::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0863383 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:29:45,912::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915918::INFO::2013-11-20 04:29:46,439::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:29:46,441::task::495::TaskManager.Task::(_debug) Task 66e22683-484a-4f55-b18d-fb8b47edc111: moving from state init -> state preparing
>Thread-15913639::INFO::2013-11-20 04:29:46,445::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:29:46,451::task::495::TaskManager.Task::(_debug) Task 189b3095-c3b8-4373-a82f-b9a0e1b9ccbd: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:29:46,455::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`716995bf-0ea6-4a0b-ba98-fe96e3100ca8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:29:46,456::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e86c6294-ff5d-42bb-8c01-48b9579bca69`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:29:46,458::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:29:46,460::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:29:46,463::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:29:46,464::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`716995bf-0ea6-4a0b-ba98-fe96e3100ca8`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:29:46,466::resourceManager::481::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' found in shared state and queue is empty, Joining current shared lock (2 active users)
>Thread-15915918::DEBUG::2013-11-20 04:29:46,469::task::495::TaskManager.Task::(_debug) Task 66e22683-484a-4f55-b18d-fb8b47edc111: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:29:46,470::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e86c6294-ff5d-42bb-8c01-48b9579bca69`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:29:46,472::task::495::TaskManager.Task::(_debug) Task 66e22683-484a-4f55-b18d-fb8b47edc111: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:29:46,474::task::495::TaskManager.Task::(_debug) Task 66e22683-484a-4f55-b18d-fb8b47edc111: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15913639::DEBUG::2013-11-20 04:29:46,474::task::495::TaskManager.Task::(_debug) Task 189b3095-c3b8-4373-a82f-b9a0e1b9ccbd: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:29:46,476::task::495::TaskManager.Task::(_debug) Task 66e22683-484a-4f55-b18d-fb8b47edc111: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:29:46,478::task::495::TaskManager.Task::(_debug) Task 189b3095-c3b8-4373-a82f-b9a0e1b9ccbd: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:29:46,479::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:29:46,480::task::495::TaskManager.Task::(_debug) Task 189b3095-c3b8-4373-a82f-b9a0e1b9ccbd: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15915918::DEBUG::2013-11-20 04:29:46,481::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:29:46,482::task::495::TaskManager.Task::(_debug) Task 189b3095-c3b8-4373-a82f-b9a0e1b9ccbd: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:29:46,484::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:29:46,484::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:29:46,486::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:29:46,487::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (1 active users)
>Thread-15913639::DEBUG::2013-11-20 04:29:46,489::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:29:46,491::task::495::TaskManager.Task::(_debug) Task 66e22683-484a-4f55-b18d-fb8b47edc111: ref 0 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:29:46,492::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::INFO::2013-11-20 04:29:46,493::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15913639::DEBUG::2013-11-20 04:29:46,494::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:29:46,497::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:29:46,499::task::495::TaskManager.Task::(_debug) Task 189b3095-c3b8-4373-a82f-b9a0e1b9ccbd: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:29:46,500::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:29:46,508::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0793658 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:29:48,526::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:29:49,079::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0717665 s, 14.3 MB/s\n'; <rc> = 0
>Thread-15916524::DEBUG::2013-11-20 04:29:50,999::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916524::INFO::2013-11-20 04:29:51,001::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916524::DEBUG::2013-11-20 04:29:51,001::task::495::TaskManager.Task::(_debug) Task 2ae3118f-3653-4217-9af0-7c26641283c0: moving from state init -> state preparing
>Thread-15916524::DEBUG::2013-11-20 04:29:51,003::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916524::DEBUG::2013-11-20 04:29:51,003::task::495::TaskManager.Task::(_debug) Task 2ae3118f-3653-4217-9af0-7c26641283c0: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916524::DEBUG::2013-11-20 04:29:51,004::task::495::TaskManager.Task::(_debug) Task 2ae3118f-3653-4217-9af0-7c26641283c0: moving from state preparing -> state finished
>Thread-15916524::DEBUG::2013-11-20 04:29:51,005::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916524::DEBUG::2013-11-20 04:29:51,005::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916524::DEBUG::2013-11-20 04:29:51,006::task::495::TaskManager.Task::(_debug) Task 2ae3118f-3653-4217-9af0-7c26641283c0: ref 0 aborting False
>Thread-15916524::INFO::2013-11-20 04:29:51,007::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916525::DEBUG::2013-11-20 04:29:51,011::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916525::INFO::2013-11-20 04:29:51,013::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916525::DEBUG::2013-11-20 04:29:51,013::task::495::TaskManager.Task::(_debug) Task dd4cb198-ef72-4960-b726-a499aceaf3a7: moving from state init -> state preparing
>Thread-15916525::DEBUG::2013-11-20 04:29:51,015::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`97b4c787-e4fe-450d-946a-d3dbe0f7a380`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916525::DEBUG::2013-11-20 04:29:51,015::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916525::DEBUG::2013-11-20 04:29:51,016::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916525::DEBUG::2013-11-20 04:29:51,017::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`97b4c787-e4fe-450d-946a-d3dbe0f7a380`::Granted request
>Thread-15916525::DEBUG::2013-11-20 04:29:51,018::task::495::TaskManager.Task::(_debug) Task dd4cb198-ef72-4960-b726-a499aceaf3a7: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916525::DEBUG::2013-11-20 04:29:51,019::task::495::TaskManager.Task::(_debug) Task dd4cb198-ef72-4960-b726-a499aceaf3a7: ref 1 aborting False
>Thread-15916525::WARNING::2013-11-20 04:29:51,021::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916525::DEBUG::2013-11-20 04:29:51,022::task::495::TaskManager.Task::(_debug) Task dd4cb198-ef72-4960-b726-a499aceaf3a7: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916525::DEBUG::2013-11-20 04:29:51,023::task::495::TaskManager.Task::(_debug) Task dd4cb198-ef72-4960-b726-a499aceaf3a7: moving from state preparing -> state finished
>Thread-15916525::DEBUG::2013-11-20 04:29:51,023::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916525::DEBUG::2013-11-20 04:29:51,024::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916525::DEBUG::2013-11-20 04:29:51,025::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916525::DEBUG::2013-11-20 04:29:51,026::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916525::DEBUG::2013-11-20 04:29:51,026::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916525::DEBUG::2013-11-20 04:29:51,027::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916525::DEBUG::2013-11-20 04:29:51,028::task::495::TaskManager.Task::(_debug) Task dd4cb198-ef72-4960-b726-a499aceaf3a7: ref 0 aborting False
>Thread-15916525::INFO::2013-11-20 04:29:51,028::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:29:51,098::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915863::INFO::2013-11-20 04:29:51,327::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:29:51,329::task::495::TaskManager.Task::(_debug) Task 0c48193d-3cbf-4dd5-ae45-8853c901747a: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:29:51,331::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0067f858-dc5a-4a23-9555-46e83fab80ea`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:29:51,331::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:29:51,332::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:29:51,333::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0067f858-dc5a-4a23-9555-46e83fab80ea`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:29:51,334::task::495::TaskManager.Task::(_debug) Task 0c48193d-3cbf-4dd5-ae45-8853c901747a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:29:51,334::task::495::TaskManager.Task::(_debug) Task 0c48193d-3cbf-4dd5-ae45-8853c901747a: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:29:51,335::task::495::TaskManager.Task::(_debug) Task 0c48193d-3cbf-4dd5-ae45-8853c901747a: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:29:51,335::task::495::TaskManager.Task::(_debug) Task 0c48193d-3cbf-4dd5-ae45-8853c901747a: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:29:51,336::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:29:51,336::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:29:51,337::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:29:51,337::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:29:51,338::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:29:51,338::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:29:51,339::task::495::TaskManager.Task::(_debug) Task 0c48193d-3cbf-4dd5-ae45-8853c901747a: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:29:51,339::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:29:51,669::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0871817 s, 11.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:29:53,686::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:29:54,194::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0649799 s, 15.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:29:56,206::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:29:56,773::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0924408 s, 11.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:29:58,796::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 04:29:59,000::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:29:59,004::task::495::TaskManager.Task::(_debug) Task 921294c7-3082-4393-a9b5-c5f30a1af9bb: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:29:59,006::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a1f9df12-7ebb-434c-aa44-66be9d4ea1d4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:29:59,007::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:29:59,009::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:29:59,009::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a1f9df12-7ebb-434c-aa44-66be9d4ea1d4`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:29:59,010::task::495::TaskManager.Task::(_debug) Task 921294c7-3082-4393-a9b5-c5f30a1af9bb: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:29:59,011::task::495::TaskManager.Task::(_debug) Task 921294c7-3082-4393-a9b5-c5f30a1af9bb: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:29:59,012::task::495::TaskManager.Task::(_debug) Task 921294c7-3082-4393-a9b5-c5f30a1af9bb: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:29:59,012::task::495::TaskManager.Task::(_debug) Task 921294c7-3082-4393-a9b5-c5f30a1af9bb: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:29:59,013::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:29:59,014::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:29:59,014::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:29:59,015::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:29:59,015::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:29:59,016::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:29:59,016::task::495::TaskManager.Task::(_debug) Task 921294c7-3082-4393-a9b5-c5f30a1af9bb: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:29:59,017::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15916528::INFO::2013-11-20 04:29:59,160::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916528::DEBUG::2013-11-20 04:29:59,161::task::495::TaskManager.Task::(_debug) Task 6d88268a-a3ba-42d0-8991-d7c82472fe74: moving from state init -> state preparing
>Thread-15916528::DEBUG::2013-11-20 04:29:59,162::task::495::TaskManager.Task::(_debug) Task 6d88268a-a3ba-42d0-8991-d7c82472fe74: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00042986869812', 'lastCheck': 1384921792.7524559, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00146794319153', 'lastCheck': 1384921790.131933, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0335340499878', 'lastCheck': 1384921797.4756, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000509977340698', 'lastCheck': 1384921797.436621, 'valid': True, 'code': 0}}
>Thread-15916528::DEBUG::2013-11-20 04:29:59,163::task::495::TaskManager.Task::(_debug) Task 6d88268a-a3ba-42d0-8991-d7c82472fe74: moving from state preparing -> state finished
>Thread-15916528::DEBUG::2013-11-20 04:29:59,164::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916528::DEBUG::2013-11-20 04:29:59,164::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916528::DEBUG::2013-11-20 04:29:59,165::task::495::TaskManager.Task::(_debug) Task 6d88268a-a3ba-42d0-8991-d7c82472fe74: ref 0 aborting False
>Thread-15916528::INFO::2013-11-20 04:29:59,166::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00042986869812', 'lastCheck': 1384921792.7524559, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00146794319153', 'lastCheck': 1384921790.131933, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0335340499878', 'lastCheck': 1384921797.4756, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000509977340698', 'lastCheck': 1384921797.436621, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:29:59,330::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0858336 s, 11.9 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 04:30:00,513::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:30:00,514::task::495::TaskManager.Task::(_debug) Task 94c38761-496b-4b32-a686-80208e515a17: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:30:00,515::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3b73c032-3351-41cc-92f8-4f0282c0cafd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:30:00,516::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:30:00,517::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:30:00,517::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3b73c032-3351-41cc-92f8-4f0282c0cafd`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:30:00,521::task::495::TaskManager.Task::(_debug) Task 94c38761-496b-4b32-a686-80208e515a17: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:30:00,524::task::495::TaskManager.Task::(_debug) Task 94c38761-496b-4b32-a686-80208e515a17: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:30:00,527::task::495::TaskManager.Task::(_debug) Task 94c38761-496b-4b32-a686-80208e515a17: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:30:00,528::task::495::TaskManager.Task::(_debug) Task 94c38761-496b-4b32-a686-80208e515a17: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:30:00,529::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:30:00,531::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:30:00,532::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:30:00,532::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:30:00,533::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:30:00,533::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:30:00,534::task::495::TaskManager.Task::(_debug) Task 94c38761-496b-4b32-a686-80208e515a17: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:30:00,534::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:30:00,535::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:30:00,535::task::495::TaskManager.Task::(_debug) Task 9cd053a5-bbb1-49b0-bc01-fcd4fedc07f6: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:30:00,536::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b7b5f5f0-82bf-4f52-a4cb-aaddb45f5a31`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:30:00,536::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:30:00,537::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:30:00,538::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b7b5f5f0-82bf-4f52-a4cb-aaddb45f5a31`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:30:00,539::task::495::TaskManager.Task::(_debug) Task 9cd053a5-bbb1-49b0-bc01-fcd4fedc07f6: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:30:00,540::task::495::TaskManager.Task::(_debug) Task 9cd053a5-bbb1-49b0-bc01-fcd4fedc07f6: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:30:00,540::task::495::TaskManager.Task::(_debug) Task 9cd053a5-bbb1-49b0-bc01-fcd4fedc07f6: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:30:00,541::task::495::TaskManager.Task::(_debug) Task 9cd053a5-bbb1-49b0-bc01-fcd4fedc07f6: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:30:00,542::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:30:00,542::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:30:00,543::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:30:00,544::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:30:00,544::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:30:00,545::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:30:00,546::task::495::TaskManager.Task::(_debug) Task 9cd053a5-bbb1-49b0-bc01-fcd4fedc07f6: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:30:00,546::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:30:01,342::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:30:01,809::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0842833 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15916530::DEBUG::2013-11-20 04:30:02,008::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916530::INFO::2013-11-20 04:30:02,009::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916530::DEBUG::2013-11-20 04:30:02,009::task::495::TaskManager.Task::(_debug) Task d6ca3f78-349f-48c0-8f41-da96060ff4e7: moving from state init -> state preparing
>Thread-15916530::DEBUG::2013-11-20 04:30:02,010::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916530::DEBUG::2013-11-20 04:30:02,010::task::495::TaskManager.Task::(_debug) Task d6ca3f78-349f-48c0-8f41-da96060ff4e7: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916530::DEBUG::2013-11-20 04:30:02,011::task::495::TaskManager.Task::(_debug) Task d6ca3f78-349f-48c0-8f41-da96060ff4e7: moving from state preparing -> state finished
>Thread-15916530::DEBUG::2013-11-20 04:30:02,011::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916530::DEBUG::2013-11-20 04:30:02,011::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916530::DEBUG::2013-11-20 04:30:02,012::task::495::TaskManager.Task::(_debug) Task d6ca3f78-349f-48c0-8f41-da96060ff4e7: ref 0 aborting False
>Thread-15916530::INFO::2013-11-20 04:30:02,012::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916531::DEBUG::2013-11-20 04:30:02,017::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916531::INFO::2013-11-20 04:30:02,018::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916531::DEBUG::2013-11-20 04:30:02,018::task::495::TaskManager.Task::(_debug) Task 6a01ff22-1e9e-4485-90fa-c8f948047b2f: moving from state init -> state preparing
>Thread-15916531::DEBUG::2013-11-20 04:30:02,019::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`250612ff-532d-4f83-9200-318506b0b163`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916531::DEBUG::2013-11-20 04:30:02,020::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916531::DEBUG::2013-11-20 04:30:02,020::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916531::DEBUG::2013-11-20 04:30:02,021::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`250612ff-532d-4f83-9200-318506b0b163`::Granted request
>Thread-15916531::DEBUG::2013-11-20 04:30:02,022::task::495::TaskManager.Task::(_debug) Task 6a01ff22-1e9e-4485-90fa-c8f948047b2f: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916531::DEBUG::2013-11-20 04:30:02,022::task::495::TaskManager.Task::(_debug) Task 6a01ff22-1e9e-4485-90fa-c8f948047b2f: ref 1 aborting False
>Thread-15916531::WARNING::2013-11-20 04:30:02,025::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916531::DEBUG::2013-11-20 04:30:02,026::task::495::TaskManager.Task::(_debug) Task 6a01ff22-1e9e-4485-90fa-c8f948047b2f: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916531::DEBUG::2013-11-20 04:30:02,027::task::495::TaskManager.Task::(_debug) Task 6a01ff22-1e9e-4485-90fa-c8f948047b2f: moving from state preparing -> state finished
>Thread-15916531::DEBUG::2013-11-20 04:30:02,027::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916531::DEBUG::2013-11-20 04:30:02,028::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916531::DEBUG::2013-11-20 04:30:02,028::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916531::DEBUG::2013-11-20 04:30:02,029::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916531::DEBUG::2013-11-20 04:30:02,029::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916531::DEBUG::2013-11-20 04:30:02,030::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916531::DEBUG::2013-11-20 04:30:02,031::task::495::TaskManager.Task::(_debug) Task 6a01ff22-1e9e-4485-90fa-c8f948047b2f: ref 0 aborting False
>Thread-15916531::INFO::2013-11-20 04:30:02,031::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:30:03,842::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:30:04,408::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0856165 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:30:06,423::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:30:06,973::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0793842 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:30:08,988::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 04:30:09,220::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:30:09,224::task::495::TaskManager.Task::(_debug) Task 27c1436c-5b0e-450a-b2bf-34fe1b6925dd: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:30:09,229::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`326a5fb7-74d8-41a8-ad95-13dbfc2bb745`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:30:09,230::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:30:09,231::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:30:09,232::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`326a5fb7-74d8-41a8-ad95-13dbfc2bb745`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:30:09,233::task::495::TaskManager.Task::(_debug) Task 27c1436c-5b0e-450a-b2bf-34fe1b6925dd: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:30:09,234::task::495::TaskManager.Task::(_debug) Task 27c1436c-5b0e-450a-b2bf-34fe1b6925dd: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:30:09,235::task::495::TaskManager.Task::(_debug) Task 27c1436c-5b0e-450a-b2bf-34fe1b6925dd: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:30:09,236::task::495::TaskManager.Task::(_debug) Task 27c1436c-5b0e-450a-b2bf-34fe1b6925dd: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:30:09,237::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:30:09,237::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:30:09,238::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:30:09,238::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:30:09,239::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:30:09,239::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:30:09,240::task::495::TaskManager.Task::(_debug) Task 27c1436c-5b0e-450a-b2bf-34fe1b6925dd: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:30:09,241::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:30:09,572::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0848295 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:30:11,586::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916535::DEBUG::2013-11-20 04:30:11,817::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Dummy-15912651::DEBUG::2013-11-20 04:30:12,298::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.12994 s, 7.9 MB/s\n'; <rc> = 0
>Thread-15916537::DEBUG::2013-11-20 04:30:12,463::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916537::INFO::2013-11-20 04:30:12,465::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916537::DEBUG::2013-11-20 04:30:12,465::task::495::TaskManager.Task::(_debug) Task 5c735faa-380f-44a6-8dd7-43156e584cac: moving from state init -> state preparing
>Thread-15916537::DEBUG::2013-11-20 04:30:12,466::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916537::DEBUG::2013-11-20 04:30:12,467::task::495::TaskManager.Task::(_debug) Task 5c735faa-380f-44a6-8dd7-43156e584cac: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916537::DEBUG::2013-11-20 04:30:12,468::task::495::TaskManager.Task::(_debug) Task 5c735faa-380f-44a6-8dd7-43156e584cac: moving from state preparing -> state finished
>Thread-15916537::DEBUG::2013-11-20 04:30:12,468::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916537::DEBUG::2013-11-20 04:30:12,469::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916537::DEBUG::2013-11-20 04:30:12,470::task::495::TaskManager.Task::(_debug) Task 5c735faa-380f-44a6-8dd7-43156e584cac: ref 0 aborting False
>Thread-15916537::INFO::2013-11-20 04:30:12,470::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916538::DEBUG::2013-11-20 04:30:12,498::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916538::INFO::2013-11-20 04:30:12,501::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916538::DEBUG::2013-11-20 04:30:12,502::task::495::TaskManager.Task::(_debug) Task 38ddb061-9875-4db9-b03a-67efad79c18e: moving from state init -> state preparing
>Thread-15916538::DEBUG::2013-11-20 04:30:12,506::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`97269072-0bdb-4607-8eac-c726ec4b675b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916538::DEBUG::2013-11-20 04:30:12,506::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916538::DEBUG::2013-11-20 04:30:12,507::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916538::DEBUG::2013-11-20 04:30:12,508::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`97269072-0bdb-4607-8eac-c726ec4b675b`::Granted request
>Thread-15916538::DEBUG::2013-11-20 04:30:12,509::task::495::TaskManager.Task::(_debug) Task 38ddb061-9875-4db9-b03a-67efad79c18e: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916538::DEBUG::2013-11-20 04:30:12,509::task::495::TaskManager.Task::(_debug) Task 38ddb061-9875-4db9-b03a-67efad79c18e: ref 1 aborting False
>Thread-15916538::WARNING::2013-11-20 04:30:12,512::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916538::DEBUG::2013-11-20 04:30:12,517::task::495::TaskManager.Task::(_debug) Task 38ddb061-9875-4db9-b03a-67efad79c18e: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916538::DEBUG::2013-11-20 04:30:12,524::task::495::TaskManager.Task::(_debug) Task 38ddb061-9875-4db9-b03a-67efad79c18e: moving from state preparing -> state finished
>Thread-15916538::DEBUG::2013-11-20 04:30:12,524::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916538::DEBUG::2013-11-20 04:30:12,525::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916538::DEBUG::2013-11-20 04:30:12,526::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916538::DEBUG::2013-11-20 04:30:12,526::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916538::DEBUG::2013-11-20 04:30:12,527::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916538::DEBUG::2013-11-20 04:30:12,527::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916538::DEBUG::2013-11-20 04:30:12,528::task::495::TaskManager.Task::(_debug) Task 38ddb061-9875-4db9-b03a-67efad79c18e: ref 0 aborting False
>Thread-15916538::INFO::2013-11-20 04:30:12,532::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916535::DEBUG::2013-11-20 04:30:12,546::clientIF::1219::vds::(fenceNode) rc 0 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = On
>Done
> err 
>Thread-15916539::DEBUG::2013-11-20 04:30:12,551::clientIF::1193::vds::(fenceNode) fenceNode(addr=192.168.100.119,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=,options=power_wait=4
>lanplus=1)
>Thread-15916539::DEBUG::2013-11-20 04:30:13,124::clientIF::1219::vds::(fenceNode) rc 0 in agent=fence_ipmilan
>ipaddr=192.168.100.119
>login=Administrator
>option=status
>passwd=XXXX
>power_wait=4
>lanplus=1 out Getting status of IPMI:192.168.100.119...Chassis power = On
>Done
> err 
>Dummy-15912651::DEBUG::2013-11-20 04:30:14,314::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:30:14,875::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0864345 s, 11.8 MB/s\n'; <rc> = 0
>Thread-15916540::INFO::2013-11-20 04:30:15,284::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916540::DEBUG::2013-11-20 04:30:15,285::task::495::TaskManager.Task::(_debug) Task a46e71bb-4bbb-4c07-981b-f9f2f67718ed: moving from state init -> state preparing
>Thread-15916540::DEBUG::2013-11-20 04:30:15,286::task::495::TaskManager.Task::(_debug) Task a46e71bb-4bbb-4c07-981b-f9f2f67718ed: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.0016450881958', 'lastCheck': 1384921813.227685, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0163979530334', 'lastCheck': 1384921810.649812, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00169014930725', 'lastCheck': 1384921807.6432171, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000427961349487', 'lastCheck': 1384921807.6430409, 'valid': True, 'code': 0}}
>Thread-15916540::DEBUG::2013-11-20 04:30:15,287::task::495::TaskManager.Task::(_debug) Task a46e71bb-4bbb-4c07-981b-f9f2f67718ed: moving from state preparing -> state finished
>Thread-15916540::DEBUG::2013-11-20 04:30:15,288::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916540::DEBUG::2013-11-20 04:30:15,288::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916540::DEBUG::2013-11-20 04:30:15,289::task::495::TaskManager.Task::(_debug) Task a46e71bb-4bbb-4c07-981b-f9f2f67718ed: ref 0 aborting False
>Thread-15916540::INFO::2013-11-20 04:30:15,290::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.0016450881958', 'lastCheck': 1384921813.227685, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0163979530334', 'lastCheck': 1384921810.649812, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00169014930725', 'lastCheck': 1384921807.6432171, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000427961349487', 'lastCheck': 1384921807.6430409, 'valid': True, 'code': 0}}
>Thread-15913695::INFO::2013-11-20 04:30:16,841::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:30:16,842::task::495::TaskManager.Task::(_debug) Task ef7a674b-3d6e-4c7a-8f13-ead00d446dd4: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:30:16,845::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`93f711d0-2b18-4771-9067-a79a4dbcbde2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:30:16,845::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:30:16,846::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:30:16,847::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`93f711d0-2b18-4771-9067-a79a4dbcbde2`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:30:16,848::task::495::TaskManager.Task::(_debug) Task ef7a674b-3d6e-4c7a-8f13-ead00d446dd4: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:30:16,849::task::495::TaskManager.Task::(_debug) Task ef7a674b-3d6e-4c7a-8f13-ead00d446dd4: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:30:16,850::task::495::TaskManager.Task::(_debug) Task ef7a674b-3d6e-4c7a-8f13-ead00d446dd4: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:30:16,850::task::495::TaskManager.Task::(_debug) Task ef7a674b-3d6e-4c7a-8f13-ead00d446dd4: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:30:16,851::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:30:16,852::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:30:16,853::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:30:16,854::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:30:16,855::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:30:16,855::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:30:16,856::task::495::TaskManager.Task::(_debug) Task ef7a674b-3d6e-4c7a-8f13-ead00d446dd4: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:30:16,856::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:30:16,891::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:30:17,444::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0866595 s, 11.8 MB/s\n'; <rc> = 0
>Thread-15914453::INFO::2013-11-20 04:30:19,239::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:30:19,250::task::495::TaskManager.Task::(_debug) Task 2867c884-0321-4eba-856a-15a5fa3c25e3: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:30:19,259::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d227fc62-0378-442f-8bdc-10f88ebf8e3b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:30:19,263::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:30:19,266::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:30:19,267::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d227fc62-0378-442f-8bdc-10f88ebf8e3b`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:30:19,268::task::495::TaskManager.Task::(_debug) Task 2867c884-0321-4eba-856a-15a5fa3c25e3: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:30:19,269::task::495::TaskManager.Task::(_debug) Task 2867c884-0321-4eba-856a-15a5fa3c25e3: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:30:19,270::task::495::TaskManager.Task::(_debug) Task 2867c884-0321-4eba-856a-15a5fa3c25e3: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:30:19,271::task::495::TaskManager.Task::(_debug) Task 2867c884-0321-4eba-856a-15a5fa3c25e3: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:30:19,271::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:30:19,272::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:30:19,273::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:30:19,273::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:30:19,274::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:30:19,275::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:30:19,275::task::495::TaskManager.Task::(_debug) Task 2867c884-0321-4eba-856a-15a5fa3c25e3: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:30:19,276::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:30:19,455::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:30:20,001::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0767517 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:30:22,015::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:30:22,567::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0861509 s, 11.9 MB/s\n'; <rc> = 0
>Thread-15916544::DEBUG::2013-11-20 04:30:22,817::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916544::INFO::2013-11-20 04:30:22,819::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916544::DEBUG::2013-11-20 04:30:22,820::task::495::TaskManager.Task::(_debug) Task 448507de-3e7b-45bb-ad4b-b71a7f0e1410: moving from state init -> state preparing
>Thread-15916544::DEBUG::2013-11-20 04:30:22,821::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916544::DEBUG::2013-11-20 04:30:22,821::task::495::TaskManager.Task::(_debug) Task 448507de-3e7b-45bb-ad4b-b71a7f0e1410: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916544::DEBUG::2013-11-20 04:30:22,822::task::495::TaskManager.Task::(_debug) Task 448507de-3e7b-45bb-ad4b-b71a7f0e1410: moving from state preparing -> state finished
>Thread-15916544::DEBUG::2013-11-20 04:30:22,822::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916544::DEBUG::2013-11-20 04:30:22,823::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916544::DEBUG::2013-11-20 04:30:22,824::task::495::TaskManager.Task::(_debug) Task 448507de-3e7b-45bb-ad4b-b71a7f0e1410: ref 0 aborting False
>Thread-15916544::INFO::2013-11-20 04:30:22,824::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916545::DEBUG::2013-11-20 04:30:22,829::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916545::INFO::2013-11-20 04:30:22,830::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916545::DEBUG::2013-11-20 04:30:22,831::task::495::TaskManager.Task::(_debug) Task 489e4f8d-313e-46e6-9050-86018e68b0f6: moving from state init -> state preparing
>Thread-15916545::DEBUG::2013-11-20 04:30:22,832::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4263adec-3118-4811-bc4a-37757482f323`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916545::DEBUG::2013-11-20 04:30:22,833::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916545::DEBUG::2013-11-20 04:30:22,834::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916545::DEBUG::2013-11-20 04:30:22,834::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4263adec-3118-4811-bc4a-37757482f323`::Granted request
>Thread-15916545::DEBUG::2013-11-20 04:30:22,835::task::495::TaskManager.Task::(_debug) Task 489e4f8d-313e-46e6-9050-86018e68b0f6: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916545::DEBUG::2013-11-20 04:30:22,836::task::495::TaskManager.Task::(_debug) Task 489e4f8d-313e-46e6-9050-86018e68b0f6: ref 1 aborting False
>Thread-15916545::WARNING::2013-11-20 04:30:22,838::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916545::DEBUG::2013-11-20 04:30:22,838::task::495::TaskManager.Task::(_debug) Task 489e4f8d-313e-46e6-9050-86018e68b0f6: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916545::DEBUG::2013-11-20 04:30:22,839::task::495::TaskManager.Task::(_debug) Task 489e4f8d-313e-46e6-9050-86018e68b0f6: moving from state preparing -> state finished
>Thread-15916545::DEBUG::2013-11-20 04:30:22,840::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916545::DEBUG::2013-11-20 04:30:22,840::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916545::DEBUG::2013-11-20 04:30:22,841::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916545::DEBUG::2013-11-20 04:30:22,842::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916545::DEBUG::2013-11-20 04:30:22,842::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916545::DEBUG::2013-11-20 04:30:22,843::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916545::DEBUG::2013-11-20 04:30:22,844::task::495::TaskManager.Task::(_debug) Task 489e4f8d-313e-46e6-9050-86018e68b0f6: ref 0 aborting False
>Thread-15916545::INFO::2013-11-20 04:30:22,844::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:30:24,578::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:30:25,183::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0825003 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:30:27,196::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 04:30:27,396::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:30:27,405::task::495::TaskManager.Task::(_debug) Task d5cc8e71-b927-425d-b816-7b24e2d21b85: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:30:27,409::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4568922f-b854-46c0-be56-bfef382f59e7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:30:27,410::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:30:27,412::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:30:27,413::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4568922f-b854-46c0-be56-bfef382f59e7`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:30:27,414::task::495::TaskManager.Task::(_debug) Task d5cc8e71-b927-425d-b816-7b24e2d21b85: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 04:30:27,415::task::495::TaskManager.Task::(_debug) Task d5cc8e71-b927-425d-b816-7b24e2d21b85: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:30:27,417::task::495::TaskManager.Task::(_debug) Task d5cc8e71-b927-425d-b816-7b24e2d21b85: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 04:30:27,417::task::495::TaskManager.Task::(_debug) Task d5cc8e71-b927-425d-b816-7b24e2d21b85: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:30:27,418::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:30:27,419::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:30:27,420::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 04:30:27,421::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:30:27,422::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:30:27,423::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:30:27,424::task::495::TaskManager.Task::(_debug) Task d5cc8e71-b927-425d-b816-7b24e2d21b85: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 04:30:27,425::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 04:30:27,684::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0495773 s, 20.7 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 04:30:29,394::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:30:29,397::task::495::TaskManager.Task::(_debug) Task 79b47fca-cbc5-4f96-b56d-2029a6d425b2: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:30:29,398::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`7807538e-35af-4ce1-8fe4-a81d27499be7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:30:29,400::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:30:29,401::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:30:29,402::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`7807538e-35af-4ce1-8fe4-a81d27499be7`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:30:29,403::task::495::TaskManager.Task::(_debug) Task 79b47fca-cbc5-4f96-b56d-2029a6d425b2: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:30:29,404::task::495::TaskManager.Task::(_debug) Task 79b47fca-cbc5-4f96-b56d-2029a6d425b2: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:30:29,405::task::495::TaskManager.Task::(_debug) Task 79b47fca-cbc5-4f96-b56d-2029a6d425b2: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 04:30:29,406::task::495::TaskManager.Task::(_debug) Task 79b47fca-cbc5-4f96-b56d-2029a6d425b2: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:30:29,407::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:30:29,407::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:30:29,408::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:30:29,409::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:30:29,409::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:30:29,410::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:30:29,411::task::495::TaskManager.Task::(_debug) Task 79b47fca-cbc5-4f96-b56d-2029a6d425b2: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:30:29,411::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:30:29,412::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:30:29,413::task::495::TaskManager.Task::(_debug) Task 1ef68f27-5be6-4c3a-9e12-64aaba018bdd: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:30:29,414::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`84e93b2d-8eeb-44d6-ad4f-657cea2b80ea`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:30:29,414::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:30:29,415::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:30:29,415::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`84e93b2d-8eeb-44d6-ad4f-657cea2b80ea`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:30:29,416::task::495::TaskManager.Task::(_debug) Task 1ef68f27-5be6-4c3a-9e12-64aaba018bdd: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:30:29,417::task::495::TaskManager.Task::(_debug) Task 1ef68f27-5be6-4c3a-9e12-64aaba018bdd: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:30:29,417::task::495::TaskManager.Task::(_debug) Task 1ef68f27-5be6-4c3a-9e12-64aaba018bdd: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:30:29,418::task::495::TaskManager.Task::(_debug) Task 1ef68f27-5be6-4c3a-9e12-64aaba018bdd: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:30:29,419::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:30:29,419::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:30:29,420::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:30:29,420::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:30:29,421::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:30:29,421::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:30:29,422::task::495::TaskManager.Task::(_debug) Task 1ef68f27-5be6-4c3a-9e12-64aaba018bdd: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:30:29,423::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:30:29,423::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:30:29,424::task::495::TaskManager.Task::(_debug) Task ae7d6c9e-83fb-433a-ad45-dbae9fbe94a9: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:30:29,425::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`397e29f5-af6e-4ee1-b698-5b3b60a0e29d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:30:29,426::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:30:29,426::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:30:29,427::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`397e29f5-af6e-4ee1-b698-5b3b60a0e29d`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:30:29,428::task::495::TaskManager.Task::(_debug) Task ae7d6c9e-83fb-433a-ad45-dbae9fbe94a9: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:30:29,429::task::495::TaskManager.Task::(_debug) Task ae7d6c9e-83fb-433a-ad45-dbae9fbe94a9: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:30:29,429::task::495::TaskManager.Task::(_debug) Task ae7d6c9e-83fb-433a-ad45-dbae9fbe94a9: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:30:29,430::task::495::TaskManager.Task::(_debug) Task ae7d6c9e-83fb-433a-ad45-dbae9fbe94a9: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:30:29,430::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:30:29,431::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:30:29,432::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:30:29,432::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:30:29,433::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:30:29,433::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:30:29,434::task::495::TaskManager.Task::(_debug) Task ae7d6c9e-83fb-433a-ad45-dbae9fbe94a9: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:30:29,435::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:30:29,435::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:30:29,436::task::495::TaskManager.Task::(_debug) Task 2126433b-5cb8-47c5-9373-ac03ed647d18: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:30:29,437::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`bab28a1d-46e4-4a1b-9931-8b88e603e3c3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:30:29,438::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:30:29,439::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:30:29,440::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`bab28a1d-46e4-4a1b-9931-8b88e603e3c3`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:30:29,441::task::495::TaskManager.Task::(_debug) Task 2126433b-5cb8-47c5-9373-ac03ed647d18: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:30:29,441::task::495::TaskManager.Task::(_debug) Task 2126433b-5cb8-47c5-9373-ac03ed647d18: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:30:29,442::task::495::TaskManager.Task::(_debug) Task 2126433b-5cb8-47c5-9373-ac03ed647d18: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:30:29,442::task::495::TaskManager.Task::(_debug) Task 2126433b-5cb8-47c5-9373-ac03ed647d18: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:30:29,443::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:30:29,444::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:30:29,444::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:30:29,445::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:30:29,445::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:30:29,446::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:30:29,447::task::495::TaskManager.Task::(_debug) Task 2126433b-5cb8-47c5-9373-ac03ed647d18: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:30:29,447::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:30:29,694::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:30:30,229::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0816085 s, 12.5 MB/s\n'; <rc> = 0
>Thread-15916548::INFO::2013-11-20 04:30:31,006::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916548::DEBUG::2013-11-20 04:30:31,007::task::495::TaskManager.Task::(_debug) Task de9cb55e-d1fd-4309-8c36-29f551dafd9e: moving from state init -> state preparing
>Thread-15916548::DEBUG::2013-11-20 04:30:31,018::task::495::TaskManager.Task::(_debug) Task de9cb55e-d1fd-4309-8c36-29f551dafd9e: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000354051589966', 'lastCheck': 1384921823.257585, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.002357006073', 'lastCheck': 1384921830.901942, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0208740234375', 'lastCheck': 1384921827.8595459, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000396966934204', 'lastCheck': 1384921827.842171, 'valid': True, 'code': 0}}
>Thread-15916548::DEBUG::2013-11-20 04:30:31,019::task::495::TaskManager.Task::(_debug) Task de9cb55e-d1fd-4309-8c36-29f551dafd9e: moving from state preparing -> state finished
>Thread-15916548::DEBUG::2013-11-20 04:30:31,020::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916548::DEBUG::2013-11-20 04:30:31,021::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916548::DEBUG::2013-11-20 04:30:31,022::task::495::TaskManager.Task::(_debug) Task de9cb55e-d1fd-4309-8c36-29f551dafd9e: ref 0 aborting False
>Thread-15916548::INFO::2013-11-20 04:30:31,023::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000354051589966', 'lastCheck': 1384921823.257585, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.002357006073', 'lastCheck': 1384921830.901942, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0208740234375', 'lastCheck': 1384921827.8595459, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000396966934204', 'lastCheck': 1384921827.842171, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:30:32,245::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 04:30:32,486::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:30:32,488::task::495::TaskManager.Task::(_debug) Task 7f4d9603-9dba-4a9a-bc77-4bd15025a063: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:30:32,490::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ec4ffa99-2415-43fa-8a22-1decd0cc343b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:30:32,491::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:30:32,492::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:30:32,493::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ec4ffa99-2415-43fa-8a22-1decd0cc343b`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:30:32,494::task::495::TaskManager.Task::(_debug) Task 7f4d9603-9dba-4a9a-bc77-4bd15025a063: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:30:32,494::task::495::TaskManager.Task::(_debug) Task 7f4d9603-9dba-4a9a-bc77-4bd15025a063: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:30:32,495::task::495::TaskManager.Task::(_debug) Task 7f4d9603-9dba-4a9a-bc77-4bd15025a063: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:30:32,496::task::495::TaskManager.Task::(_debug) Task 7f4d9603-9dba-4a9a-bc77-4bd15025a063: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:30:32,497::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:30:32,497::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:30:32,498::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:30:32,499::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:30:32,499::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:30:32,500::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:30:32,501::task::495::TaskManager.Task::(_debug) Task 7f4d9603-9dba-4a9a-bc77-4bd15025a063: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:30:32,501::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:30:32,809::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0763546 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15916550::DEBUG::2013-11-20 04:30:33,160::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916550::INFO::2013-11-20 04:30:33,162::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916550::DEBUG::2013-11-20 04:30:33,163::task::495::TaskManager.Task::(_debug) Task 6ae70dba-e416-44cf-a093-dda819aa0755: moving from state init -> state preparing
>Thread-15916550::DEBUG::2013-11-20 04:30:33,164::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916550::DEBUG::2013-11-20 04:30:33,164::task::495::TaskManager.Task::(_debug) Task 6ae70dba-e416-44cf-a093-dda819aa0755: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916550::DEBUG::2013-11-20 04:30:33,165::task::495::TaskManager.Task::(_debug) Task 6ae70dba-e416-44cf-a093-dda819aa0755: moving from state preparing -> state finished
>Thread-15916550::DEBUG::2013-11-20 04:30:33,166::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916550::DEBUG::2013-11-20 04:30:33,166::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916550::DEBUG::2013-11-20 04:30:33,167::task::495::TaskManager.Task::(_debug) Task 6ae70dba-e416-44cf-a093-dda819aa0755: ref 0 aborting False
>Thread-15916550::INFO::2013-11-20 04:30:33,168::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916551::DEBUG::2013-11-20 04:30:33,173::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916551::INFO::2013-11-20 04:30:33,174::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916551::DEBUG::2013-11-20 04:30:33,174::task::495::TaskManager.Task::(_debug) Task 722ee41f-7595-4548-8337-be6f7baa3c6b: moving from state init -> state preparing
>Thread-15916551::DEBUG::2013-11-20 04:30:33,176::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7cfd465b-edf4-4350-b43f-007b8c906e14`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916551::DEBUG::2013-11-20 04:30:33,176::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916551::DEBUG::2013-11-20 04:30:33,177::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916551::DEBUG::2013-11-20 04:30:33,177::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`7cfd465b-edf4-4350-b43f-007b8c906e14`::Granted request
>Thread-15916551::DEBUG::2013-11-20 04:30:33,178::task::495::TaskManager.Task::(_debug) Task 722ee41f-7595-4548-8337-be6f7baa3c6b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916551::DEBUG::2013-11-20 04:30:33,179::task::495::TaskManager.Task::(_debug) Task 722ee41f-7595-4548-8337-be6f7baa3c6b: ref 1 aborting False
>Thread-15916551::WARNING::2013-11-20 04:30:33,181::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916551::DEBUG::2013-11-20 04:30:33,182::task::495::TaskManager.Task::(_debug) Task 722ee41f-7595-4548-8337-be6f7baa3c6b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916551::DEBUG::2013-11-20 04:30:33,183::task::495::TaskManager.Task::(_debug) Task 722ee41f-7595-4548-8337-be6f7baa3c6b: moving from state preparing -> state finished
>Thread-15916551::DEBUG::2013-11-20 04:30:33,184::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916551::DEBUG::2013-11-20 04:30:33,184::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916551::DEBUG::2013-11-20 04:30:33,185::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916551::DEBUG::2013-11-20 04:30:33,185::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916551::DEBUG::2013-11-20 04:30:33,186::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916551::DEBUG::2013-11-20 04:30:33,186::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916551::DEBUG::2013-11-20 04:30:33,187::task::495::TaskManager.Task::(_debug) Task 722ee41f-7595-4548-8337-be6f7baa3c6b: ref 0 aborting False
>Thread-15916551::INFO::2013-11-20 04:30:33,188::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:30:34,824::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:30:35,363::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0729806 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:30:37,378::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:30:37,942::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.074929 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:30:39,956::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:30:40,493::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0773098 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:30:42,515::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:30:43,072::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0770223 s, 13.3 MB/s\n'; <rc> = 0
>Thread-15916555::DEBUG::2013-11-20 04:30:43,574::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916555::INFO::2013-11-20 04:30:43,576::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916555::DEBUG::2013-11-20 04:30:43,576::task::495::TaskManager.Task::(_debug) Task b837f5af-ea94-4db0-be3d-62aefb02a4cb: moving from state init -> state preparing
>Thread-15916555::DEBUG::2013-11-20 04:30:43,577::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916555::DEBUG::2013-11-20 04:30:43,578::task::495::TaskManager.Task::(_debug) Task b837f5af-ea94-4db0-be3d-62aefb02a4cb: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916555::DEBUG::2013-11-20 04:30:43,579::task::495::TaskManager.Task::(_debug) Task b837f5af-ea94-4db0-be3d-62aefb02a4cb: moving from state preparing -> state finished
>Thread-15916555::DEBUG::2013-11-20 04:30:43,580::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916555::DEBUG::2013-11-20 04:30:43,580::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916555::DEBUG::2013-11-20 04:30:43,581::task::495::TaskManager.Task::(_debug) Task b837f5af-ea94-4db0-be3d-62aefb02a4cb: ref 0 aborting False
>Thread-15916555::INFO::2013-11-20 04:30:43,582::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916556::DEBUG::2013-11-20 04:30:43,586::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916556::INFO::2013-11-20 04:30:43,587::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916556::DEBUG::2013-11-20 04:30:43,588::task::495::TaskManager.Task::(_debug) Task b6f24245-7347-4fa8-bacb-43ee99bd72ca: moving from state init -> state preparing
>Thread-15916556::DEBUG::2013-11-20 04:30:43,589::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d9ea37e3-334c-4e4c-ad57-b008f14cc93b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916556::DEBUG::2013-11-20 04:30:43,590::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916556::DEBUG::2013-11-20 04:30:43,591::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916556::DEBUG::2013-11-20 04:30:43,592::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d9ea37e3-334c-4e4c-ad57-b008f14cc93b`::Granted request
>Thread-15916556::DEBUG::2013-11-20 04:30:43,593::task::495::TaskManager.Task::(_debug) Task b6f24245-7347-4fa8-bacb-43ee99bd72ca: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916556::DEBUG::2013-11-20 04:30:43,594::task::495::TaskManager.Task::(_debug) Task b6f24245-7347-4fa8-bacb-43ee99bd72ca: ref 1 aborting False
>Thread-15916556::WARNING::2013-11-20 04:30:43,596::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916556::DEBUG::2013-11-20 04:30:43,597::task::495::TaskManager.Task::(_debug) Task b6f24245-7347-4fa8-bacb-43ee99bd72ca: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916556::DEBUG::2013-11-20 04:30:43,598::task::495::TaskManager.Task::(_debug) Task b6f24245-7347-4fa8-bacb-43ee99bd72ca: moving from state preparing -> state finished
>Thread-15916556::DEBUG::2013-11-20 04:30:43,599::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916556::DEBUG::2013-11-20 04:30:43,599::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916556::DEBUG::2013-11-20 04:30:43,600::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916556::DEBUG::2013-11-20 04:30:43,601::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916556::DEBUG::2013-11-20 04:30:43,602::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916556::DEBUG::2013-11-20 04:30:43,602::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916556::DEBUG::2013-11-20 04:30:43,603::task::495::TaskManager.Task::(_debug) Task b6f24245-7347-4fa8-bacb-43ee99bd72ca: ref 0 aborting False
>Thread-15916556::INFO::2013-11-20 04:30:43,604::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:30:45,086::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:30:45,582::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0631591 s, 16.2 MB/s\n'; <rc> = 0
>Thread-15916558::INFO::2013-11-20 04:30:46,794::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916558::DEBUG::2013-11-20 04:30:46,795::task::495::TaskManager.Task::(_debug) Task 38b3e1dc-1fd9-4c55-9cf0-121651059d1f: moving from state init -> state preparing
>Thread-15916558::DEBUG::2013-11-20 04:30:46,795::task::495::TaskManager.Task::(_debug) Task 38b3e1dc-1fd9-4c55-9cf0-121651059d1f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000443935394287', 'lastCheck': 1384921843.7364421, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0131030082703', 'lastCheck': 1384921841.18729, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00196099281311', 'lastCheck': 1384921838.0475371, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000481128692627', 'lastCheck': 1384921838.053194, 'valid': True, 'code': 0}}
>Thread-15916558::DEBUG::2013-11-20 04:30:46,796::task::495::TaskManager.Task::(_debug) Task 38b3e1dc-1fd9-4c55-9cf0-121651059d1f: moving from state preparing -> state finished
>Thread-15916558::DEBUG::2013-11-20 04:30:46,796::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916558::DEBUG::2013-11-20 04:30:46,797::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916558::DEBUG::2013-11-20 04:30:46,797::task::495::TaskManager.Task::(_debug) Task 38b3e1dc-1fd9-4c55-9cf0-121651059d1f: ref 0 aborting False
>Thread-15916558::INFO::2013-11-20 04:30:46,797::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000443935394287', 'lastCheck': 1384921843.7364421, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0131030082703', 'lastCheck': 1384921841.18729, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00196099281311', 'lastCheck': 1384921838.0475371, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000481128692627', 'lastCheck': 1384921838.053194, 'valid': True, 'code': 0}}
>Thread-15913639::INFO::2013-11-20 04:30:47,066::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:30:47,067::task::495::TaskManager.Task::(_debug) Task 234da947-d681-4dd1-b901-2fa543a208a7: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:30:47,069::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ee96fadb-ef70-40e6-b73c-62ecf5c87987`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:30:47,070::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:30:47,071::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:30:47,072::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ee96fadb-ef70-40e6-b73c-62ecf5c87987`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:30:47,073::task::495::TaskManager.Task::(_debug) Task 234da947-d681-4dd1-b901-2fa543a208a7: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:30:47,074::task::495::TaskManager.Task::(_debug) Task 234da947-d681-4dd1-b901-2fa543a208a7: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:30:47,075::task::495::TaskManager.Task::(_debug) Task 234da947-d681-4dd1-b901-2fa543a208a7: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:30:47,076::task::495::TaskManager.Task::(_debug) Task 234da947-d681-4dd1-b901-2fa543a208a7: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:30:47,077::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:30:47,077::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:30:47,078::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:30:47,079::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:30:47,080::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:30:47,080::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:30:47,081::task::495::TaskManager.Task::(_debug) Task 234da947-d681-4dd1-b901-2fa543a208a7: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:30:47,082::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:30:47,597::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915918::INFO::2013-11-20 04:30:47,765::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:30:47,767::task::495::TaskManager.Task::(_debug) Task ce3c771c-d31a-49c6-a080-7bcc4580c450: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:30:47,770::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b7d71b63-8692-48b7-9327-5b8cfc4c9683`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:30:47,771::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:30:47,772::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:30:47,773::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b7d71b63-8692-48b7-9327-5b8cfc4c9683`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:30:47,774::task::495::TaskManager.Task::(_debug) Task ce3c771c-d31a-49c6-a080-7bcc4580c450: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:30:47,774::task::495::TaskManager.Task::(_debug) Task ce3c771c-d31a-49c6-a080-7bcc4580c450: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:30:47,775::task::495::TaskManager.Task::(_debug) Task ce3c771c-d31a-49c6-a080-7bcc4580c450: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:30:47,775::task::495::TaskManager.Task::(_debug) Task ce3c771c-d31a-49c6-a080-7bcc4580c450: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:30:47,776::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:30:47,776::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:30:47,777::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:30:47,777::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:30:47,777::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:30:47,778::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:30:47,778::task::495::TaskManager.Task::(_debug) Task ce3c771c-d31a-49c6-a080-7bcc4580c450: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:30:47,779::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:30:48,140::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0988125 s, 10.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:30:50,159::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:30:50,751::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0843241 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:30:52,772::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915863::INFO::2013-11-20 04:30:52,774::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:30:52,973::task::495::TaskManager.Task::(_debug) Task 37d0b3b5-5608-492a-9f58-2936a8493160: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:30:52,983::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`89d6d77a-3ba6-4805-b4b8-f747580a0f11`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:30:52,985::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:30:52,988::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:30:52,989::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`89d6d77a-3ba6-4805-b4b8-f747580a0f11`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:30:52,992::task::495::TaskManager.Task::(_debug) Task 37d0b3b5-5608-492a-9f58-2936a8493160: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:30:52,993::task::495::TaskManager.Task::(_debug) Task 37d0b3b5-5608-492a-9f58-2936a8493160: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:30:52,994::task::495::TaskManager.Task::(_debug) Task 37d0b3b5-5608-492a-9f58-2936a8493160: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:30:52,996::task::495::TaskManager.Task::(_debug) Task 37d0b3b5-5608-492a-9f58-2936a8493160: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:30:52,997::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:30:52,998::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:30:52,999::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:30:53,000::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:30:53,001::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:30:53,004::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:30:53,005::task::495::TaskManager.Task::(_debug) Task 37d0b3b5-5608-492a-9f58-2936a8493160: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:30:53,006::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:30:53,314::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0729358 s, 14.0 MB/s\n'; <rc> = 0
>Thread-15916562::DEBUG::2013-11-20 04:30:54,236::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916562::INFO::2013-11-20 04:30:54,237::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916562::DEBUG::2013-11-20 04:30:54,238::task::495::TaskManager.Task::(_debug) Task 634ff3ae-927e-4347-876e-cc47461bedf8: moving from state init -> state preparing
>Thread-15916562::DEBUG::2013-11-20 04:30:54,239::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916562::DEBUG::2013-11-20 04:30:54,239::task::495::TaskManager.Task::(_debug) Task 634ff3ae-927e-4347-876e-cc47461bedf8: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916562::DEBUG::2013-11-20 04:30:54,240::task::495::TaskManager.Task::(_debug) Task 634ff3ae-927e-4347-876e-cc47461bedf8: moving from state preparing -> state finished
>Thread-15916562::DEBUG::2013-11-20 04:30:54,241::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916562::DEBUG::2013-11-20 04:30:54,241::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916562::DEBUG::2013-11-20 04:30:54,242::task::495::TaskManager.Task::(_debug) Task 634ff3ae-927e-4347-876e-cc47461bedf8: ref 0 aborting False
>Thread-15916562::INFO::2013-11-20 04:30:54,242::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916563::DEBUG::2013-11-20 04:30:54,247::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916563::INFO::2013-11-20 04:30:54,248::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916563::DEBUG::2013-11-20 04:30:54,249::task::495::TaskManager.Task::(_debug) Task caa3e6e8-f255-4982-96ae-0e97a6155eb5: moving from state init -> state preparing
>Thread-15916563::DEBUG::2013-11-20 04:30:54,251::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`eb9e6581-5593-4df9-b227-d7d0232362fb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916563::DEBUG::2013-11-20 04:30:54,251::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916563::DEBUG::2013-11-20 04:30:54,252::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916563::DEBUG::2013-11-20 04:30:54,253::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`eb9e6581-5593-4df9-b227-d7d0232362fb`::Granted request
>Thread-15916563::DEBUG::2013-11-20 04:30:54,254::task::495::TaskManager.Task::(_debug) Task caa3e6e8-f255-4982-96ae-0e97a6155eb5: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916563::DEBUG::2013-11-20 04:30:54,255::task::495::TaskManager.Task::(_debug) Task caa3e6e8-f255-4982-96ae-0e97a6155eb5: ref 1 aborting False
>Thread-15916563::WARNING::2013-11-20 04:30:54,257::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916563::DEBUG::2013-11-20 04:30:54,259::task::495::TaskManager.Task::(_debug) Task caa3e6e8-f255-4982-96ae-0e97a6155eb5: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916563::DEBUG::2013-11-20 04:30:54,260::task::495::TaskManager.Task::(_debug) Task caa3e6e8-f255-4982-96ae-0e97a6155eb5: moving from state preparing -> state finished
>Thread-15916563::DEBUG::2013-11-20 04:30:54,260::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916563::DEBUG::2013-11-20 04:30:54,261::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916563::DEBUG::2013-11-20 04:30:54,262::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916563::DEBUG::2013-11-20 04:30:54,262::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916563::DEBUG::2013-11-20 04:30:54,263::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916563::DEBUG::2013-11-20 04:30:54,264::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916563::DEBUG::2013-11-20 04:30:54,264::task::495::TaskManager.Task::(_debug) Task caa3e6e8-f255-4982-96ae-0e97a6155eb5: ref 0 aborting False
>Thread-15916563::INFO::2013-11-20 04:30:54,265::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:30:55,328::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:30:55,872::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0797113 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:30:57,887::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:30:58,437::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0704238 s, 14.5 MB/s\n'; <rc> = 0
>Thread-42840::DEBUG::2013-11-20 04:30:58,549::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 04:30:59,463::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913658::INFO::2013-11-20 04:30:59,792::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:30:59,796::task::495::TaskManager.Task::(_debug) Task 0d1a295f-a035-4c03-a0aa-951862448e09: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:30:59,799::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`18da889b-8e57-43ab-8259-b975dcf398ab`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:30:59,801::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:30:59,802::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:30:59,803::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`18da889b-8e57-43ab-8259-b975dcf398ab`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:30:59,804::task::495::TaskManager.Task::(_debug) Task 0d1a295f-a035-4c03-a0aa-951862448e09: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:30:59,805::task::495::TaskManager.Task::(_debug) Task 0d1a295f-a035-4c03-a0aa-951862448e09: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:30:59,806::task::495::TaskManager.Task::(_debug) Task 0d1a295f-a035-4c03-a0aa-951862448e09: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:30:59,807::task::495::TaskManager.Task::(_debug) Task 0d1a295f-a035-4c03-a0aa-951862448e09: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:30:59,808::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:30:59,809::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:30:59,810::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:30:59,811::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:30:59,812::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:30:59,812::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:30:59,814::task::495::TaskManager.Task::(_debug) Task 0d1a295f-a035-4c03-a0aa-951862448e09: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:30:59,815::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:31:00,451::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:31:01,032::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0776428 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 04:31:01,646::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:31:01,647::task::495::TaskManager.Task::(_debug) Task ddfb365d-f94c-4c99-b493-2488ac5f2f07: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:31:01,649::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`81021cfe-c163-41a9-b64c-e3ccc7e26160`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:31:01,650::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:31:01,651::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:31:01,651::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`81021cfe-c163-41a9-b64c-e3ccc7e26160`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:31:01,652::task::495::TaskManager.Task::(_debug) Task ddfb365d-f94c-4c99-b493-2488ac5f2f07: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:31:01,653::task::495::TaskManager.Task::(_debug) Task ddfb365d-f94c-4c99-b493-2488ac5f2f07: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:31:01,654::task::495::TaskManager.Task::(_debug) Task ddfb365d-f94c-4c99-b493-2488ac5f2f07: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:31:01,654::task::495::TaskManager.Task::(_debug) Task ddfb365d-f94c-4c99-b493-2488ac5f2f07: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:31:01,655::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:31:01,655::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:31:01,656::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:31:01,657::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:31:01,658::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:31:01,658::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:31:01,659::task::495::TaskManager.Task::(_debug) Task ddfb365d-f94c-4c99-b493-2488ac5f2f07: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:31:01,660::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:31:01,660::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:31:01,661::task::495::TaskManager.Task::(_debug) Task 2b714ca2-8184-4d5f-bfd8-f30f1b22055a: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:31:01,662::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`00d82d99-46fd-4963-aa1e-d647d7b34766`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:31:01,663::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:31:01,663::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:31:01,664::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`00d82d99-46fd-4963-aa1e-d647d7b34766`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:31:01,665::task::495::TaskManager.Task::(_debug) Task 2b714ca2-8184-4d5f-bfd8-f30f1b22055a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:31:01,666::task::495::TaskManager.Task::(_debug) Task 2b714ca2-8184-4d5f-bfd8-f30f1b22055a: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:31:01,666::task::495::TaskManager.Task::(_debug) Task 2b714ca2-8184-4d5f-bfd8-f30f1b22055a: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:31:01,667::task::495::TaskManager.Task::(_debug) Task 2b714ca2-8184-4d5f-bfd8-f30f1b22055a: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:31:01,668::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:31:01,668::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:31:01,669::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:31:01,670::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:31:01,670::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:31:01,671::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:31:01,671::task::495::TaskManager.Task::(_debug) Task 2b714ca2-8184-4d5f-bfd8-f30f1b22055a: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:31:01,672::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15916566::INFO::2013-11-20 04:31:02,664::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916566::DEBUG::2013-11-20 04:31:02,666::task::495::TaskManager.Task::(_debug) Task 5f60c46e-cbeb-49c8-be10-1b8cab63c25c: moving from state init -> state preparing
>Thread-15916566::DEBUG::2013-11-20 04:31:02,667::task::495::TaskManager.Task::(_debug) Task 5f60c46e-cbeb-49c8-be10-1b8cab63c25c: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000440835952759', 'lastCheck': 1384921853.7714789, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00517797470093', 'lastCheck': 1384921861.6839759, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0245630741119', 'lastCheck': 1384921858.7876511, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000367164611816', 'lastCheck': 1384921859.465512, 'valid': True, 'code': 0}}
>Thread-15916566::DEBUG::2013-11-20 04:31:02,668::task::495::TaskManager.Task::(_debug) Task 5f60c46e-cbeb-49c8-be10-1b8cab63c25c: moving from state preparing -> state finished
>Thread-15916566::DEBUG::2013-11-20 04:31:02,668::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916566::DEBUG::2013-11-20 04:31:02,669::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916566::DEBUG::2013-11-20 04:31:02,670::task::495::TaskManager.Task::(_debug) Task 5f60c46e-cbeb-49c8-be10-1b8cab63c25c: ref 0 aborting False
>Thread-15916566::INFO::2013-11-20 04:31:02,670::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000440835952759', 'lastCheck': 1384921853.7714789, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00517797470093', 'lastCheck': 1384921861.6839759, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0245630741119', 'lastCheck': 1384921858.7876511, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000367164611816', 'lastCheck': 1384921859.465512, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:31:03,053::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:31:03,586::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0813896 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15916568::DEBUG::2013-11-20 04:31:04,565::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916568::INFO::2013-11-20 04:31:04,566::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916568::DEBUG::2013-11-20 04:31:04,567::task::495::TaskManager.Task::(_debug) Task f8c5c1c7-07c3-4ff6-809b-554bb3db7192: moving from state init -> state preparing
>Thread-15916568::DEBUG::2013-11-20 04:31:04,568::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916568::DEBUG::2013-11-20 04:31:04,568::task::495::TaskManager.Task::(_debug) Task f8c5c1c7-07c3-4ff6-809b-554bb3db7192: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916568::DEBUG::2013-11-20 04:31:04,569::task::495::TaskManager.Task::(_debug) Task f8c5c1c7-07c3-4ff6-809b-554bb3db7192: moving from state preparing -> state finished
>Thread-15916568::DEBUG::2013-11-20 04:31:04,570::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916568::DEBUG::2013-11-20 04:31:04,570::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916568::DEBUG::2013-11-20 04:31:04,571::task::495::TaskManager.Task::(_debug) Task f8c5c1c7-07c3-4ff6-809b-554bb3db7192: ref 0 aborting False
>Thread-15916568::INFO::2013-11-20 04:31:04,571::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916569::DEBUG::2013-11-20 04:31:04,576::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916569::INFO::2013-11-20 04:31:04,577::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916569::DEBUG::2013-11-20 04:31:04,577::task::495::TaskManager.Task::(_debug) Task 197eb65b-ee61-4680-8c43-6ae536f99ace: moving from state init -> state preparing
>Thread-15916569::DEBUG::2013-11-20 04:31:04,579::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`002e8023-2ac7-4aa4-ab4a-92016cc70653`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916569::DEBUG::2013-11-20 04:31:04,579::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916569::DEBUG::2013-11-20 04:31:04,580::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916569::DEBUG::2013-11-20 04:31:04,581::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`002e8023-2ac7-4aa4-ab4a-92016cc70653`::Granted request
>Thread-15916569::DEBUG::2013-11-20 04:31:04,582::task::495::TaskManager.Task::(_debug) Task 197eb65b-ee61-4680-8c43-6ae536f99ace: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916569::DEBUG::2013-11-20 04:31:04,582::task::495::TaskManager.Task::(_debug) Task 197eb65b-ee61-4680-8c43-6ae536f99ace: ref 1 aborting False
>Thread-15916569::WARNING::2013-11-20 04:31:04,584::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916569::DEBUG::2013-11-20 04:31:04,585::task::495::TaskManager.Task::(_debug) Task 197eb65b-ee61-4680-8c43-6ae536f99ace: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916569::DEBUG::2013-11-20 04:31:04,586::task::495::TaskManager.Task::(_debug) Task 197eb65b-ee61-4680-8c43-6ae536f99ace: moving from state preparing -> state finished
>Thread-15916569::DEBUG::2013-11-20 04:31:04,587::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916569::DEBUG::2013-11-20 04:31:04,587::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916569::DEBUG::2013-11-20 04:31:04,588::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916569::DEBUG::2013-11-20 04:31:04,589::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916569::DEBUG::2013-11-20 04:31:04,590::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916569::DEBUG::2013-11-20 04:31:04,590::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916569::DEBUG::2013-11-20 04:31:04,591::task::495::TaskManager.Task::(_debug) Task 197eb65b-ee61-4680-8c43-6ae536f99ace: ref 0 aborting False
>Thread-15916569::INFO::2013-11-20 04:31:04,592::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:31:05,601::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:31:06,178::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0755236 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:31:08,194::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:31:08,736::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0754386 s, 13.6 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 04:31:09,854::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:31:09,860::task::495::TaskManager.Task::(_debug) Task 1850cd5a-facc-434c-91fc-51e7ed67d4ae: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:31:09,867::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`543a9614-b1b3-43bc-b524-6e99c1ed9fb8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:31:09,870::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:31:09,875::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:31:09,877::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`543a9614-b1b3-43bc-b524-6e99c1ed9fb8`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:31:09,879::task::495::TaskManager.Task::(_debug) Task 1850cd5a-facc-434c-91fc-51e7ed67d4ae: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:31:09,880::task::495::TaskManager.Task::(_debug) Task 1850cd5a-facc-434c-91fc-51e7ed67d4ae: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:31:09,881::task::495::TaskManager.Task::(_debug) Task 1850cd5a-facc-434c-91fc-51e7ed67d4ae: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:31:09,882::task::495::TaskManager.Task::(_debug) Task 1850cd5a-facc-434c-91fc-51e7ed67d4ae: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:31:09,883::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:31:09,883::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:31:09,884::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:31:09,885::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:31:09,886::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:31:09,886::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:31:09,887::task::495::TaskManager.Task::(_debug) Task 1850cd5a-facc-434c-91fc-51e7ed67d4ae: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:31:09,888::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:31:10,752::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:31:11,292::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0809355 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:31:13,313::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:31:13,836::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0850201 s, 12.0 MB/s\n'; <rc> = 0
>Thread-15916573::DEBUG::2013-11-20 04:31:15,191::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916573::INFO::2013-11-20 04:31:15,193::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916573::DEBUG::2013-11-20 04:31:15,194::task::495::TaskManager.Task::(_debug) Task 6b1e3923-6e19-4a36-8a9c-25e9997c1a05: moving from state init -> state preparing
>Thread-15916573::DEBUG::2013-11-20 04:31:15,195::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916573::DEBUG::2013-11-20 04:31:15,195::task::495::TaskManager.Task::(_debug) Task 6b1e3923-6e19-4a36-8a9c-25e9997c1a05: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916573::DEBUG::2013-11-20 04:31:15,196::task::495::TaskManager.Task::(_debug) Task 6b1e3923-6e19-4a36-8a9c-25e9997c1a05: moving from state preparing -> state finished
>Thread-15916573::DEBUG::2013-11-20 04:31:15,196::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916573::DEBUG::2013-11-20 04:31:15,197::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916573::DEBUG::2013-11-20 04:31:15,198::task::495::TaskManager.Task::(_debug) Task 6b1e3923-6e19-4a36-8a9c-25e9997c1a05: ref 0 aborting False
>Thread-15916573::INFO::2013-11-20 04:31:15,198::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916574::DEBUG::2013-11-20 04:31:15,203::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916574::INFO::2013-11-20 04:31:15,204::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916574::DEBUG::2013-11-20 04:31:15,205::task::495::TaskManager.Task::(_debug) Task 4469fbf4-0138-45c0-bfe3-ecdaf3c9408b: moving from state init -> state preparing
>Thread-15916574::DEBUG::2013-11-20 04:31:15,206::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f899937a-de98-4c00-a792-68d8c4558470`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916574::DEBUG::2013-11-20 04:31:15,206::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916574::DEBUG::2013-11-20 04:31:15,207::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916574::DEBUG::2013-11-20 04:31:15,208::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f899937a-de98-4c00-a792-68d8c4558470`::Granted request
>Thread-15916574::DEBUG::2013-11-20 04:31:15,209::task::495::TaskManager.Task::(_debug) Task 4469fbf4-0138-45c0-bfe3-ecdaf3c9408b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916574::DEBUG::2013-11-20 04:31:15,209::task::495::TaskManager.Task::(_debug) Task 4469fbf4-0138-45c0-bfe3-ecdaf3c9408b: ref 1 aborting False
>Thread-15916574::WARNING::2013-11-20 04:31:15,211::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916574::DEBUG::2013-11-20 04:31:15,212::task::495::TaskManager.Task::(_debug) Task 4469fbf4-0138-45c0-bfe3-ecdaf3c9408b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916574::DEBUG::2013-11-20 04:31:15,213::task::495::TaskManager.Task::(_debug) Task 4469fbf4-0138-45c0-bfe3-ecdaf3c9408b: moving from state preparing -> state finished
>Thread-15916574::DEBUG::2013-11-20 04:31:15,214::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916574::DEBUG::2013-11-20 04:31:15,214::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916574::DEBUG::2013-11-20 04:31:15,215::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916574::DEBUG::2013-11-20 04:31:15,215::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916574::DEBUG::2013-11-20 04:31:15,216::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916574::DEBUG::2013-11-20 04:31:15,216::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916574::DEBUG::2013-11-20 04:31:15,217::task::495::TaskManager.Task::(_debug) Task 4469fbf4-0138-45c0-bfe3-ecdaf3c9408b: ref 0 aborting False
>Thread-15916574::INFO::2013-11-20 04:31:15,218::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:31:15,852::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:31:16,414::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0823994 s, 12.4 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 04:31:18,093::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:31:18,094::task::495::TaskManager.Task::(_debug) Task e8ff2665-0efb-4c42-9ea5-16103fe8f68a: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:31:18,096::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3df6b56e-165e-426b-b5ad-ba1371e1dc09`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:31:18,096::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:31:18,097::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:31:18,098::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3df6b56e-165e-426b-b5ad-ba1371e1dc09`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:31:18,099::task::495::TaskManager.Task::(_debug) Task e8ff2665-0efb-4c42-9ea5-16103fe8f68a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:31:18,099::task::495::TaskManager.Task::(_debug) Task e8ff2665-0efb-4c42-9ea5-16103fe8f68a: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:31:18,100::task::495::TaskManager.Task::(_debug) Task e8ff2665-0efb-4c42-9ea5-16103fe8f68a: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:31:18,101::task::495::TaskManager.Task::(_debug) Task e8ff2665-0efb-4c42-9ea5-16103fe8f68a: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:31:18,102::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:31:18,102::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:31:18,103::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:31:18,103::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:31:18,104::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:31:18,104::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:31:18,105::task::495::TaskManager.Task::(_debug) Task e8ff2665-0efb-4c42-9ea5-16103fe8f68a: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:31:18,106::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:31:18,429::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916576::INFO::2013-11-20 04:31:18,626::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916576::DEBUG::2013-11-20 04:31:18,629::task::495::TaskManager.Task::(_debug) Task 36ac5dce-20d7-4a3d-ae48-fa65e017fc7b: moving from state init -> state preparing
>Thread-15916576::DEBUG::2013-11-20 04:31:18,631::task::495::TaskManager.Task::(_debug) Task 36ac5dce-20d7-4a3d-ae48-fa65e017fc7b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000345945358276', 'lastCheck': 1384921873.9667561, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0139031410217', 'lastCheck': 1384921871.9829161, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00149416923523', 'lastCheck': 1384921868.8544979, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0028030872345', 'lastCheck': 1384921869.8588109, 'valid': True, 'code': 0}}
>Thread-15916576::DEBUG::2013-11-20 04:31:18,631::task::495::TaskManager.Task::(_debug) Task 36ac5dce-20d7-4a3d-ae48-fa65e017fc7b: moving from state preparing -> state finished
>Thread-15916576::DEBUG::2013-11-20 04:31:18,633::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916576::DEBUG::2013-11-20 04:31:18,634::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916576::DEBUG::2013-11-20 04:31:18,634::task::495::TaskManager.Task::(_debug) Task 36ac5dce-20d7-4a3d-ae48-fa65e017fc7b: ref 0 aborting False
>Thread-15916576::INFO::2013-11-20 04:31:18,635::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000345945358276', 'lastCheck': 1384921873.9667561, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0139031410217', 'lastCheck': 1384921871.9829161, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00149416923523', 'lastCheck': 1384921868.8544979, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0028030872345', 'lastCheck': 1384921869.8588109, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:31:18,955::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0865538 s, 11.8 MB/s\n'; <rc> = 0
>Thread-15914453::INFO::2013-11-20 04:31:20,071::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:31:20,083::task::495::TaskManager.Task::(_debug) Task 7071ddb4-5778-4718-8425-9c851608cfff: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:31:20,085::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2d6f8e0e-294a-44a0-a002-52decabbda2e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:31:20,086::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:31:20,088::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:31:20,088::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2d6f8e0e-294a-44a0-a002-52decabbda2e`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:31:20,090::task::495::TaskManager.Task::(_debug) Task 7071ddb4-5778-4718-8425-9c851608cfff: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:31:20,090::task::495::TaskManager.Task::(_debug) Task 7071ddb4-5778-4718-8425-9c851608cfff: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:31:20,091::task::495::TaskManager.Task::(_debug) Task 7071ddb4-5778-4718-8425-9c851608cfff: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:31:20,092::task::495::TaskManager.Task::(_debug) Task 7071ddb4-5778-4718-8425-9c851608cfff: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:31:20,093::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:31:20,093::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:31:20,094::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:31:20,095::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:31:20,095::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:31:20,096::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:31:20,097::task::495::TaskManager.Task::(_debug) Task 7071ddb4-5778-4718-8425-9c851608cfff: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:31:20,097::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:31:20,971::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:31:21,486::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0819716 s, 12.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:31:23,500::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:31:23,996::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0850272 s, 12.0 MB/s\n'; <rc> = 0
>Thread-15916580::DEBUG::2013-11-20 04:31:25,746::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916580::INFO::2013-11-20 04:31:25,747::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916580::DEBUG::2013-11-20 04:31:25,748::task::495::TaskManager.Task::(_debug) Task 064dc2ae-ad06-4eab-8140-5d84df676d9a: moving from state init -> state preparing
>Thread-15916580::DEBUG::2013-11-20 04:31:25,749::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916580::DEBUG::2013-11-20 04:31:25,749::task::495::TaskManager.Task::(_debug) Task 064dc2ae-ad06-4eab-8140-5d84df676d9a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916580::DEBUG::2013-11-20 04:31:25,751::task::495::TaskManager.Task::(_debug) Task 064dc2ae-ad06-4eab-8140-5d84df676d9a: moving from state preparing -> state finished
>Thread-15916580::DEBUG::2013-11-20 04:31:25,752::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916580::DEBUG::2013-11-20 04:31:25,753::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916580::DEBUG::2013-11-20 04:31:25,753::task::495::TaskManager.Task::(_debug) Task 064dc2ae-ad06-4eab-8140-5d84df676d9a: ref 0 aborting False
>Thread-15916580::INFO::2013-11-20 04:31:25,755::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916581::DEBUG::2013-11-20 04:31:25,759::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916581::INFO::2013-11-20 04:31:25,760::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916581::DEBUG::2013-11-20 04:31:25,760::task::495::TaskManager.Task::(_debug) Task 7f88843c-7bb3-40a1-87df-2890d71e55e5: moving from state init -> state preparing
>Thread-15916581::DEBUG::2013-11-20 04:31:25,762::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`942d8dfe-2e84-4a07-8b62-f24202f78585`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916581::DEBUG::2013-11-20 04:31:25,762::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916581::DEBUG::2013-11-20 04:31:25,763::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916581::DEBUG::2013-11-20 04:31:25,764::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`942d8dfe-2e84-4a07-8b62-f24202f78585`::Granted request
>Thread-15916581::DEBUG::2013-11-20 04:31:25,765::task::495::TaskManager.Task::(_debug) Task 7f88843c-7bb3-40a1-87df-2890d71e55e5: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916581::DEBUG::2013-11-20 04:31:25,765::task::495::TaskManager.Task::(_debug) Task 7f88843c-7bb3-40a1-87df-2890d71e55e5: ref 1 aborting False
>Thread-15916581::WARNING::2013-11-20 04:31:25,767::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916581::DEBUG::2013-11-20 04:31:25,768::task::495::TaskManager.Task::(_debug) Task 7f88843c-7bb3-40a1-87df-2890d71e55e5: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916581::DEBUG::2013-11-20 04:31:25,769::task::495::TaskManager.Task::(_debug) Task 7f88843c-7bb3-40a1-87df-2890d71e55e5: moving from state preparing -> state finished
>Thread-15916581::DEBUG::2013-11-20 04:31:25,770::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916581::DEBUG::2013-11-20 04:31:25,770::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916581::DEBUG::2013-11-20 04:31:25,771::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916581::DEBUG::2013-11-20 04:31:25,772::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916581::DEBUG::2013-11-20 04:31:25,772::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916581::DEBUG::2013-11-20 04:31:25,773::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916581::DEBUG::2013-11-20 04:31:25,774::task::495::TaskManager.Task::(_debug) Task 7f88843c-7bb3-40a1-87df-2890d71e55e5: ref 0 aborting False
>Thread-15916581::INFO::2013-11-20 04:31:25,774::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:31:26,011::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:31:26,558::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0738039 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:31:28,573::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 04:31:28,796::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:31:28,803::task::495::TaskManager.Task::(_debug) Task 66c8c751-829e-497c-9ece-b2dbaad633e0: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:31:28,813::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e647a5ae-981e-4bd8-97b5-c69043da4068`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:31:28,814::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:31:28,815::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:31:28,816::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e647a5ae-981e-4bd8-97b5-c69043da4068`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:31:28,822::task::495::TaskManager.Task::(_debug) Task 66c8c751-829e-497c-9ece-b2dbaad633e0: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 04:31:28,826::task::495::TaskManager.Task::(_debug) Task 66c8c751-829e-497c-9ece-b2dbaad633e0: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:31:28,827::task::495::TaskManager.Task::(_debug) Task 66c8c751-829e-497c-9ece-b2dbaad633e0: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 04:31:28,828::task::495::TaskManager.Task::(_debug) Task 66c8c751-829e-497c-9ece-b2dbaad633e0: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:31:28,829::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:31:28,830::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:31:28,831::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 04:31:28,832::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:31:28,833::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:31:28,833::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:31:28,834::task::495::TaskManager.Task::(_debug) Task 66c8c751-829e-497c-9ece-b2dbaad633e0: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 04:31:28,835::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 04:31:29,109::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0879279 s, 11.6 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 04:31:30,227::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:31:30,229::task::495::TaskManager.Task::(_debug) Task 93b9311f-42bc-4576-8c3c-08e65b2f914c: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:31:30,231::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c5a5dc0f-32c3-44ee-8639-107d86d8cd49`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:31:30,232::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:31:30,233::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:31:30,233::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c5a5dc0f-32c3-44ee-8639-107d86d8cd49`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:31:30,234::task::495::TaskManager.Task::(_debug) Task 93b9311f-42bc-4576-8c3c-08e65b2f914c: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:31:30,235::task::495::TaskManager.Task::(_debug) Task 93b9311f-42bc-4576-8c3c-08e65b2f914c: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:31:30,236::task::495::TaskManager.Task::(_debug) Task 93b9311f-42bc-4576-8c3c-08e65b2f914c: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 04:31:30,236::task::495::TaskManager.Task::(_debug) Task 93b9311f-42bc-4576-8c3c-08e65b2f914c: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:31:30,237::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:31:30,237::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:31:30,238::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:31:30,239::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:31:30,239::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:31:30,240::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:31:30,240::task::495::TaskManager.Task::(_debug) Task 93b9311f-42bc-4576-8c3c-08e65b2f914c: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:31:30,241::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:31:30,242::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:31:30,242::task::495::TaskManager.Task::(_debug) Task ffce8838-8ac5-45d3-aa65-cd1e9cf132ac: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:31:30,243::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2cae2431-ec71-43b0-a5e8-cfcf6b076e37`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:31:30,244::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:31:30,244::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:31:30,245::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2cae2431-ec71-43b0-a5e8-cfcf6b076e37`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:31:30,246::task::495::TaskManager.Task::(_debug) Task ffce8838-8ac5-45d3-aa65-cd1e9cf132ac: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:31:30,246::task::495::TaskManager.Task::(_debug) Task ffce8838-8ac5-45d3-aa65-cd1e9cf132ac: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:31:30,247::task::495::TaskManager.Task::(_debug) Task ffce8838-8ac5-45d3-aa65-cd1e9cf132ac: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:31:30,247::task::495::TaskManager.Task::(_debug) Task ffce8838-8ac5-45d3-aa65-cd1e9cf132ac: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:31:30,248::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:31:30,249::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:31:30,250::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:31:30,250::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:31:30,251::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:31:30,251::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:31:30,252::task::495::TaskManager.Task::(_debug) Task ffce8838-8ac5-45d3-aa65-cd1e9cf132ac: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:31:30,253::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:31:30,254::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:31:30,254::task::495::TaskManager.Task::(_debug) Task b81125bc-4104-4a79-9908-8399a1c6b8aa: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:31:30,255::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e2f02b29-53ab-4b50-9ccb-06f6d52c1621`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:31:30,256::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:31:30,257::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:31:30,257::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e2f02b29-53ab-4b50-9ccb-06f6d52c1621`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:31:30,258::task::495::TaskManager.Task::(_debug) Task b81125bc-4104-4a79-9908-8399a1c6b8aa: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:31:30,259::task::495::TaskManager.Task::(_debug) Task b81125bc-4104-4a79-9908-8399a1c6b8aa: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:31:30,260::task::495::TaskManager.Task::(_debug) Task b81125bc-4104-4a79-9908-8399a1c6b8aa: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:31:30,260::task::495::TaskManager.Task::(_debug) Task b81125bc-4104-4a79-9908-8399a1c6b8aa: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:31:30,261::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:31:30,261::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:31:30,262::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:31:30,263::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:31:30,263::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:31:30,264::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:31:30,264::task::495::TaskManager.Task::(_debug) Task b81125bc-4104-4a79-9908-8399a1c6b8aa: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:31:30,265::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:31:30,266::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:31:30,266::task::495::TaskManager.Task::(_debug) Task 77c17485-ea15-4a1e-a6ef-e7dff6eed276: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:31:30,267::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`ef85099f-35ef-4700-8b93-eb985743ad96`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:31:30,268::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:31:30,269::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:31:30,269::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`ef85099f-35ef-4700-8b93-eb985743ad96`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:31:30,270::task::495::TaskManager.Task::(_debug) Task 77c17485-ea15-4a1e-a6ef-e7dff6eed276: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:31:30,271::task::495::TaskManager.Task::(_debug) Task 77c17485-ea15-4a1e-a6ef-e7dff6eed276: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:31:30,271::task::495::TaskManager.Task::(_debug) Task 77c17485-ea15-4a1e-a6ef-e7dff6eed276: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:31:30,272::task::495::TaskManager.Task::(_debug) Task 77c17485-ea15-4a1e-a6ef-e7dff6eed276: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:31:30,274::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:31:30,274::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:31:30,275::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:31:30,276::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:31:30,276::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:31:30,277::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:31:30,277::task::495::TaskManager.Task::(_debug) Task 77c17485-ea15-4a1e-a6ef-e7dff6eed276: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:31:30,278::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:31:31,122::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:31:31,671::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0751212 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:31:33,687::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 04:31:33,892::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:31:33,894::task::495::TaskManager.Task::(_debug) Task 980dea2a-dd57-4c17-94d3-7be0b02f3d3d: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:31:33,897::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9ff4eebd-972b-499d-98ce-bcef323946ae`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:31:33,897::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:31:33,898::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:31:33,899::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9ff4eebd-972b-499d-98ce-bcef323946ae`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:31:33,900::task::495::TaskManager.Task::(_debug) Task 980dea2a-dd57-4c17-94d3-7be0b02f3d3d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:31:33,901::task::495::TaskManager.Task::(_debug) Task 980dea2a-dd57-4c17-94d3-7be0b02f3d3d: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:31:33,902::task::495::TaskManager.Task::(_debug) Task 980dea2a-dd57-4c17-94d3-7be0b02f3d3d: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:31:33,902::task::495::TaskManager.Task::(_debug) Task 980dea2a-dd57-4c17-94d3-7be0b02f3d3d: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:31:33,903::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:31:33,904::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:31:33,905::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:31:33,905::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:31:33,906::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:31:33,906::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:31:33,908::task::495::TaskManager.Task::(_debug) Task 980dea2a-dd57-4c17-94d3-7be0b02f3d3d: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:31:33,909::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:31:34,215::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0791537 s, 12.9 MB/s\n'; <rc> = 0
>Thread-15916584::INFO::2013-11-20 04:31:34,927::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916584::DEBUG::2013-11-20 04:31:34,928::task::495::TaskManager.Task::(_debug) Task 305cc43a-c16a-46ea-9d9e-e01f2fb2d06e: moving from state init -> state preparing
>Thread-15916584::DEBUG::2013-11-20 04:31:34,928::task::495::TaskManager.Task::(_debug) Task 305cc43a-c16a-46ea-9d9e-e01f2fb2d06e: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00140285491943', 'lastCheck': 1384921894.3707311, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00176906585693', 'lastCheck': 1384921892.3453331, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0508351325989', 'lastCheck': 1384921889.27964, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000964879989624', 'lastCheck': 1384921890.22628, 'valid': True, 'code': 0}}
>Thread-15916584::DEBUG::2013-11-20 04:31:34,929::task::495::TaskManager.Task::(_debug) Task 305cc43a-c16a-46ea-9d9e-e01f2fb2d06e: moving from state preparing -> state finished
>Thread-15916584::DEBUG::2013-11-20 04:31:34,930::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916584::DEBUG::2013-11-20 04:31:34,931::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916584::DEBUG::2013-11-20 04:31:34,931::task::495::TaskManager.Task::(_debug) Task 305cc43a-c16a-46ea-9d9e-e01f2fb2d06e: ref 0 aborting False
>Thread-15916584::INFO::2013-11-20 04:31:34,932::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00140285491943', 'lastCheck': 1384921894.3707311, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00176906585693', 'lastCheck': 1384921892.3453331, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0508351325989', 'lastCheck': 1384921889.27964, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000964879989624', 'lastCheck': 1384921890.22628, 'valid': True, 'code': 0}}
>Thread-15916586::DEBUG::2013-11-20 04:31:36,093::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916586::INFO::2013-11-20 04:31:36,095::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916586::DEBUG::2013-11-20 04:31:36,095::task::495::TaskManager.Task::(_debug) Task 71e6beb5-3820-434b-ac6f-761c9dd74ec8: moving from state init -> state preparing
>Thread-15916586::DEBUG::2013-11-20 04:31:36,096::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916586::DEBUG::2013-11-20 04:31:36,096::task::495::TaskManager.Task::(_debug) Task 71e6beb5-3820-434b-ac6f-761c9dd74ec8: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916586::DEBUG::2013-11-20 04:31:36,097::task::495::TaskManager.Task::(_debug) Task 71e6beb5-3820-434b-ac6f-761c9dd74ec8: moving from state preparing -> state finished
>Thread-15916586::DEBUG::2013-11-20 04:31:36,098::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916586::DEBUG::2013-11-20 04:31:36,098::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916586::DEBUG::2013-11-20 04:31:36,099::task::495::TaskManager.Task::(_debug) Task 71e6beb5-3820-434b-ac6f-761c9dd74ec8: ref 0 aborting False
>Thread-15916586::INFO::2013-11-20 04:31:36,099::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916587::DEBUG::2013-11-20 04:31:36,104::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916587::INFO::2013-11-20 04:31:36,105::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916587::DEBUG::2013-11-20 04:31:36,106::task::495::TaskManager.Task::(_debug) Task 23db9140-ce39-4360-88cb-54bee2583e31: moving from state init -> state preparing
>Thread-15916587::DEBUG::2013-11-20 04:31:36,107::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f9e1f2a0-42b1-4c62-82f5-dfcfe0fe10f9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916587::DEBUG::2013-11-20 04:31:36,107::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916587::DEBUG::2013-11-20 04:31:36,108::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916587::DEBUG::2013-11-20 04:31:36,109::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f9e1f2a0-42b1-4c62-82f5-dfcfe0fe10f9`::Granted request
>Thread-15916587::DEBUG::2013-11-20 04:31:36,110::task::495::TaskManager.Task::(_debug) Task 23db9140-ce39-4360-88cb-54bee2583e31: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916587::DEBUG::2013-11-20 04:31:36,110::task::495::TaskManager.Task::(_debug) Task 23db9140-ce39-4360-88cb-54bee2583e31: ref 1 aborting False
>Thread-15916587::WARNING::2013-11-20 04:31:36,112::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916587::DEBUG::2013-11-20 04:31:36,113::task::495::TaskManager.Task::(_debug) Task 23db9140-ce39-4360-88cb-54bee2583e31: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916587::DEBUG::2013-11-20 04:31:36,114::task::495::TaskManager.Task::(_debug) Task 23db9140-ce39-4360-88cb-54bee2583e31: moving from state preparing -> state finished
>Thread-15916587::DEBUG::2013-11-20 04:31:36,115::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916587::DEBUG::2013-11-20 04:31:36,115::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916587::DEBUG::2013-11-20 04:31:36,116::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916587::DEBUG::2013-11-20 04:31:36,117::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916587::DEBUG::2013-11-20 04:31:36,117::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916587::DEBUG::2013-11-20 04:31:36,118::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916587::DEBUG::2013-11-20 04:31:36,118::task::495::TaskManager.Task::(_debug) Task 23db9140-ce39-4360-88cb-54bee2583e31: ref 0 aborting False
>Thread-15916587::INFO::2013-11-20 04:31:36,119::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:31:36,234::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:31:36,792::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0815519 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:31:38,806::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:31:39,289::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0601104 s, 17.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:31:41,301::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:31:41,826::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0778809 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:31:43,841::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:31:44,414::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0828809 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:31:46,430::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916591::DEBUG::2013-11-20 04:31:46,644::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916591::INFO::2013-11-20 04:31:46,647::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916591::DEBUG::2013-11-20 04:31:46,649::task::495::TaskManager.Task::(_debug) Task dea7c30e-c95f-4de0-baf7-841c3cd69eb6: moving from state init -> state preparing
>Thread-15916591::DEBUG::2013-11-20 04:31:46,650::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916591::DEBUG::2013-11-20 04:31:46,651::task::495::TaskManager.Task::(_debug) Task dea7c30e-c95f-4de0-baf7-841c3cd69eb6: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916591::DEBUG::2013-11-20 04:31:46,652::task::495::TaskManager.Task::(_debug) Task dea7c30e-c95f-4de0-baf7-841c3cd69eb6: moving from state preparing -> state finished
>Thread-15916591::DEBUG::2013-11-20 04:31:46,652::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916591::DEBUG::2013-11-20 04:31:46,653::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916591::DEBUG::2013-11-20 04:31:46,653::task::495::TaskManager.Task::(_debug) Task dea7c30e-c95f-4de0-baf7-841c3cd69eb6: ref 0 aborting False
>Thread-15916591::INFO::2013-11-20 04:31:46,654::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916592::DEBUG::2013-11-20 04:31:46,660::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916592::INFO::2013-11-20 04:31:46,661::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916592::DEBUG::2013-11-20 04:31:46,661::task::495::TaskManager.Task::(_debug) Task 2f39fcb3-d2b1-4a6d-8092-4bdf6ef29b0b: moving from state init -> state preparing
>Thread-15916592::DEBUG::2013-11-20 04:31:46,663::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2065a0e2-9632-4ed8-b3c5-06e08e051b8b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916592::DEBUG::2013-11-20 04:31:46,664::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916592::DEBUG::2013-11-20 04:31:46,665::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916592::DEBUG::2013-11-20 04:31:46,665::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2065a0e2-9632-4ed8-b3c5-06e08e051b8b`::Granted request
>Thread-15916592::DEBUG::2013-11-20 04:31:46,666::task::495::TaskManager.Task::(_debug) Task 2f39fcb3-d2b1-4a6d-8092-4bdf6ef29b0b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916592::DEBUG::2013-11-20 04:31:46,667::task::495::TaskManager.Task::(_debug) Task 2f39fcb3-d2b1-4a6d-8092-4bdf6ef29b0b: ref 1 aborting False
>Thread-15916592::WARNING::2013-11-20 04:31:46,669::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916592::DEBUG::2013-11-20 04:31:46,670::task::495::TaskManager.Task::(_debug) Task 2f39fcb3-d2b1-4a6d-8092-4bdf6ef29b0b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916592::DEBUG::2013-11-20 04:31:46,671::task::495::TaskManager.Task::(_debug) Task 2f39fcb3-d2b1-4a6d-8092-4bdf6ef29b0b: moving from state preparing -> state finished
>Thread-15916592::DEBUG::2013-11-20 04:31:46,672::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916592::DEBUG::2013-11-20 04:31:46,672::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916592::DEBUG::2013-11-20 04:31:46,673::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916592::DEBUG::2013-11-20 04:31:46,673::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916592::DEBUG::2013-11-20 04:31:46,673::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916592::DEBUG::2013-11-20 04:31:46,674::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916592::DEBUG::2013-11-20 04:31:46,675::task::495::TaskManager.Task::(_debug) Task 2f39fcb3-d2b1-4a6d-8092-4bdf6ef29b0b: ref 0 aborting False
>Thread-15916592::INFO::2013-11-20 04:31:46,675::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:31:46,959::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0737278 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15913639::INFO::2013-11-20 04:31:47,501::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:31:47,502::task::495::TaskManager.Task::(_debug) Task ff2371e7-240c-407b-8e68-ec8c5312aa6f: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:31:47,504::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2fac9e97-7de7-46dc-a7cc-d43f347c224e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:31:47,504::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:31:47,505::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:31:47,505::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2fac9e97-7de7-46dc-a7cc-d43f347c224e`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:31:47,506::task::495::TaskManager.Task::(_debug) Task ff2371e7-240c-407b-8e68-ec8c5312aa6f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:31:47,507::task::495::TaskManager.Task::(_debug) Task ff2371e7-240c-407b-8e68-ec8c5312aa6f: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:31:47,508::task::495::TaskManager.Task::(_debug) Task ff2371e7-240c-407b-8e68-ec8c5312aa6f: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:31:47,508::task::495::TaskManager.Task::(_debug) Task ff2371e7-240c-407b-8e68-ec8c5312aa6f: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:31:47,509::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:31:47,510::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:31:47,510::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:31:47,511::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:31:47,511::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:31:47,512::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:31:47,513::task::495::TaskManager.Task::(_debug) Task ff2371e7-240c-407b-8e68-ec8c5312aa6f: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:31:47,513::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:31:48,974::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915918::INFO::2013-11-20 04:31:49,204::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:31:49,214::task::495::TaskManager.Task::(_debug) Task e036a3fb-a717-4e2f-a6ff-5c3015fbc850: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:31:49,219::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fe417496-5196-4bf3-b395-b9087976ca28`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:31:49,221::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:31:49,222::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:31:49,223::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fe417496-5196-4bf3-b395-b9087976ca28`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:31:49,224::task::495::TaskManager.Task::(_debug) Task e036a3fb-a717-4e2f-a6ff-5c3015fbc850: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:31:49,225::task::495::TaskManager.Task::(_debug) Task e036a3fb-a717-4e2f-a6ff-5c3015fbc850: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:31:49,226::task::495::TaskManager.Task::(_debug) Task e036a3fb-a717-4e2f-a6ff-5c3015fbc850: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:31:49,227::task::495::TaskManager.Task::(_debug) Task e036a3fb-a717-4e2f-a6ff-5c3015fbc850: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:31:49,227::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:31:49,228::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:31:49,229::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:31:49,230::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:31:49,230::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:31:49,231::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:31:49,232::task::495::TaskManager.Task::(_debug) Task e036a3fb-a717-4e2f-a6ff-5c3015fbc850: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:31:49,232::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:31:49,529::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0822042 s, 12.5 MB/s\n'; <rc> = 0
>Thread-15916594::INFO::2013-11-20 04:31:50,808::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916594::DEBUG::2013-11-20 04:31:50,809::task::495::TaskManager.Task::(_debug) Task 690fda76-fed9-4f56-8a2c-40980c571343: moving from state init -> state preparing
>Thread-15916594::DEBUG::2013-11-20 04:31:50,810::task::495::TaskManager.Task::(_debug) Task 690fda76-fed9-4f56-8a2c-40980c571343: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000540018081665', 'lastCheck': 1384921904.501822, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0341439247131', 'lastCheck': 1384921902.533839, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00133681297302', 'lastCheck': 1384921909.644614, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000822067260742', 'lastCheck': 1384921910.641758, 'valid': True, 'code': 0}}
>Thread-15916594::DEBUG::2013-11-20 04:31:50,811::task::495::TaskManager.Task::(_debug) Task 690fda76-fed9-4f56-8a2c-40980c571343: moving from state preparing -> state finished
>Thread-15916594::DEBUG::2013-11-20 04:31:50,811::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916594::DEBUG::2013-11-20 04:31:50,812::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916594::DEBUG::2013-11-20 04:31:50,812::task::495::TaskManager.Task::(_debug) Task 690fda76-fed9-4f56-8a2c-40980c571343: ref 0 aborting False
>Thread-15916594::INFO::2013-11-20 04:31:50,813::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000540018081665', 'lastCheck': 1384921904.501822, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0341439247131', 'lastCheck': 1384921902.533839, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00133681297302', 'lastCheck': 1384921909.644614, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000822067260742', 'lastCheck': 1384921910.641758, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:31:51,545::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:31:52,075::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0740622 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:31:54,087::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915863::INFO::2013-11-20 04:31:54,296::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:31:54,299::task::495::TaskManager.Task::(_debug) Task ea32c127-8939-47a1-96ac-71f85e45a7a5: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:31:54,303::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b15a1a3b-176a-4e97-b24a-292048d11c79`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:31:54,305::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:31:54,307::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:31:54,309::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b15a1a3b-176a-4e97-b24a-292048d11c79`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:31:54,311::task::495::TaskManager.Task::(_debug) Task ea32c127-8939-47a1-96ac-71f85e45a7a5: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:31:54,312::task::495::TaskManager.Task::(_debug) Task ea32c127-8939-47a1-96ac-71f85e45a7a5: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:31:54,313::task::495::TaskManager.Task::(_debug) Task ea32c127-8939-47a1-96ac-71f85e45a7a5: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:31:54,315::task::495::TaskManager.Task::(_debug) Task ea32c127-8939-47a1-96ac-71f85e45a7a5: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:31:54,316::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:31:54,317::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:31:54,318::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:31:54,319::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:31:54,320::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:31:54,321::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:31:54,322::task::495::TaskManager.Task::(_debug) Task ea32c127-8939-47a1-96ac-71f85e45a7a5: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:31:54,323::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:31:54,609::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0740272 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:31:56,625::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916597::DEBUG::2013-11-20 04:31:57,099::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916597::INFO::2013-11-20 04:31:57,101::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916597::DEBUG::2013-11-20 04:31:57,102::task::495::TaskManager.Task::(_debug) Task df16c541-652c-48d9-b01c-f32cd2e82b03: moving from state init -> state preparing
>Thread-15916597::DEBUG::2013-11-20 04:31:57,104::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916597::DEBUG::2013-11-20 04:31:57,105::task::495::TaskManager.Task::(_debug) Task df16c541-652c-48d9-b01c-f32cd2e82b03: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916597::DEBUG::2013-11-20 04:31:57,106::task::495::TaskManager.Task::(_debug) Task df16c541-652c-48d9-b01c-f32cd2e82b03: moving from state preparing -> state finished
>Thread-15916597::DEBUG::2013-11-20 04:31:57,107::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916597::DEBUG::2013-11-20 04:31:57,108::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916597::DEBUG::2013-11-20 04:31:57,109::task::495::TaskManager.Task::(_debug) Task df16c541-652c-48d9-b01c-f32cd2e82b03: ref 0 aborting False
>Thread-15916597::INFO::2013-11-20 04:31:57,110::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916598::DEBUG::2013-11-20 04:31:57,116::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916598::INFO::2013-11-20 04:31:57,117::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916598::DEBUG::2013-11-20 04:31:57,118::task::495::TaskManager.Task::(_debug) Task e4e254bc-9d6c-46fa-baa6-5986372bd329: moving from state init -> state preparing
>Thread-15916598::DEBUG::2013-11-20 04:31:57,120::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c908c3da-ebbb-4906-a1c8-d40af7e3deed`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916598::DEBUG::2013-11-20 04:31:57,121::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916598::DEBUG::2013-11-20 04:31:57,122::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916598::DEBUG::2013-11-20 04:31:57,123::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c908c3da-ebbb-4906-a1c8-d40af7e3deed`::Granted request
>Thread-15916598::DEBUG::2013-11-20 04:31:57,125::task::495::TaskManager.Task::(_debug) Task e4e254bc-9d6c-46fa-baa6-5986372bd329: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916598::DEBUG::2013-11-20 04:31:57,126::task::495::TaskManager.Task::(_debug) Task e4e254bc-9d6c-46fa-baa6-5986372bd329: ref 1 aborting False
>Thread-15916598::WARNING::2013-11-20 04:31:57,129::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916598::DEBUG::2013-11-20 04:31:57,131::task::495::TaskManager.Task::(_debug) Task e4e254bc-9d6c-46fa-baa6-5986372bd329: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916598::DEBUG::2013-11-20 04:31:57,132::task::495::TaskManager.Task::(_debug) Task e4e254bc-9d6c-46fa-baa6-5986372bd329: moving from state preparing -> state finished
>Thread-15916598::DEBUG::2013-11-20 04:31:57,133::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916598::DEBUG::2013-11-20 04:31:57,134::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916598::DEBUG::2013-11-20 04:31:57,135::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916598::DEBUG::2013-11-20 04:31:57,137::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916598::DEBUG::2013-11-20 04:31:57,138::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916598::DEBUG::2013-11-20 04:31:57,139::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916598::DEBUG::2013-11-20 04:31:57,140::task::495::TaskManager.Task::(_debug) Task e4e254bc-9d6c-46fa-baa6-5986372bd329: ref 0 aborting False
>Thread-15916598::INFO::2013-11-20 04:31:57,141::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:31:57,204::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.119663 s, 8.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:31:59,219::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:31:59,795::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0829635 s, 12.3 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 04:32:00,821::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:32:00,822::task::495::TaskManager.Task::(_debug) Task bd72e6ee-6839-4241-8d44-2d0b48654441: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:32:00,832::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`aadf7229-23c9-4e05-95f1-2db5b46b390f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:32:00,833::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:32:00,835::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:32:00,836::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`aadf7229-23c9-4e05-95f1-2db5b46b390f`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:32:00,838::task::495::TaskManager.Task::(_debug) Task bd72e6ee-6839-4241-8d44-2d0b48654441: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:32:00,839::task::495::TaskManager.Task::(_debug) Task bd72e6ee-6839-4241-8d44-2d0b48654441: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:32:00,841::task::495::TaskManager.Task::(_debug) Task bd72e6ee-6839-4241-8d44-2d0b48654441: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:32:00,842::task::495::TaskManager.Task::(_debug) Task bd72e6ee-6839-4241-8d44-2d0b48654441: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:32:00,842::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:32:00,843::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:32:00,844::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:32:00,845::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:32:00,846::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:32:00,847::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:32:00,848::task::495::TaskManager.Task::(_debug) Task bd72e6ee-6839-4241-8d44-2d0b48654441: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:32:00,849::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:32:01,809::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:32:02,392::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0853826 s, 12.0 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 04:32:02,934::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:32:02,935::task::495::TaskManager.Task::(_debug) Task ec0a3c4b-a177-4666-8f18-a51fe2cfac58: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:32:02,946::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f1597df3-e3f7-4c0b-ab5c-cc70dbf0442b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:32:02,946::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:32:02,947::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:32:02,948::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f1597df3-e3f7-4c0b-ab5c-cc70dbf0442b`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:32:02,949::task::495::TaskManager.Task::(_debug) Task ec0a3c4b-a177-4666-8f18-a51fe2cfac58: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:32:02,950::task::495::TaskManager.Task::(_debug) Task ec0a3c4b-a177-4666-8f18-a51fe2cfac58: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:32:02,950::task::495::TaskManager.Task::(_debug) Task ec0a3c4b-a177-4666-8f18-a51fe2cfac58: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:32:02,951::task::495::TaskManager.Task::(_debug) Task ec0a3c4b-a177-4666-8f18-a51fe2cfac58: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:32:02,951::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:32:02,952::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:32:02,953::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:32:02,953::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:32:02,954::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:32:02,954::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:32:02,955::task::495::TaskManager.Task::(_debug) Task ec0a3c4b-a177-4666-8f18-a51fe2cfac58: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:32:02,956::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:32:02,956::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:32:02,957::task::495::TaskManager.Task::(_debug) Task 3bfec50f-2eac-42ff-a073-524bf17ac6ae: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:32:02,958::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5218c8ef-f217-4290-be48-dc2b7193b4a2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:32:02,959::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:32:02,960::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:32:02,960::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5218c8ef-f217-4290-be48-dc2b7193b4a2`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:32:02,962::task::495::TaskManager.Task::(_debug) Task 3bfec50f-2eac-42ff-a073-524bf17ac6ae: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:32:02,962::task::495::TaskManager.Task::(_debug) Task 3bfec50f-2eac-42ff-a073-524bf17ac6ae: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:32:02,963::task::495::TaskManager.Task::(_debug) Task 3bfec50f-2eac-42ff-a073-524bf17ac6ae: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:32:02,963::task::495::TaskManager.Task::(_debug) Task 3bfec50f-2eac-42ff-a073-524bf17ac6ae: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:32:02,964::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:32:02,965::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:32:02,965::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:32:02,966::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:32:02,966::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:32:02,967::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:32:02,968::task::495::TaskManager.Task::(_debug) Task 3bfec50f-2eac-42ff-a073-524bf17ac6ae: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:32:02,968::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:32:04,408::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:32:04,917::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0540316 s, 19.0 MB/s\n'; <rc> = 0
>Thread-15916602::INFO::2013-11-20 04:32:06,807::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916602::DEBUG::2013-11-20 04:32:06,808::task::495::TaskManager.Task::(_debug) Task 284fc3a5-c1c0-4bef-a6c8-f34d17c2e030: moving from state init -> state preparing
>Thread-15916602::DEBUG::2013-11-20 04:32:06,808::task::495::TaskManager.Task::(_debug) Task 284fc3a5-c1c0-4bef-a6c8-f34d17c2e030: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000377893447876', 'lastCheck': 1384921925.016633, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014009475708', 'lastCheck': 1384921923.015496, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0367870330811', 'lastCheck': 1384921919.867358, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00045919418335', 'lastCheck': 1384921920.8290341, 'valid': True, 'code': 0}}
>Thread-15916602::DEBUG::2013-11-20 04:32:06,809::task::495::TaskManager.Task::(_debug) Task 284fc3a5-c1c0-4bef-a6c8-f34d17c2e030: moving from state preparing -> state finished
>Thread-15916602::DEBUG::2013-11-20 04:32:06,810::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916602::DEBUG::2013-11-20 04:32:06,810::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916602::DEBUG::2013-11-20 04:32:06,811::task::495::TaskManager.Task::(_debug) Task 284fc3a5-c1c0-4bef-a6c8-f34d17c2e030: ref 0 aborting False
>Thread-15916602::INFO::2013-11-20 04:32:06,811::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000377893447876', 'lastCheck': 1384921925.016633, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014009475708', 'lastCheck': 1384921923.015496, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0367870330811', 'lastCheck': 1384921919.867358, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00045919418335', 'lastCheck': 1384921920.8290341, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:32:06,928::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916604::DEBUG::2013-11-20 04:32:07,483::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916604::INFO::2013-11-20 04:32:07,486::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916604::DEBUG::2013-11-20 04:32:07,486::task::495::TaskManager.Task::(_debug) Task 029b135c-e5ee-4ce1-835a-6fe3f981c2fc: moving from state init -> state preparing
>Thread-15916604::DEBUG::2013-11-20 04:32:07,488::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916604::DEBUG::2013-11-20 04:32:07,489::task::495::TaskManager.Task::(_debug) Task 029b135c-e5ee-4ce1-835a-6fe3f981c2fc: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916604::DEBUG::2013-11-20 04:32:07,490::task::495::TaskManager.Task::(_debug) Task 029b135c-e5ee-4ce1-835a-6fe3f981c2fc: moving from state preparing -> state finished
>Thread-15916604::DEBUG::2013-11-20 04:32:07,491::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916604::DEBUG::2013-11-20 04:32:07,492::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916604::DEBUG::2013-11-20 04:32:07,493::task::495::TaskManager.Task::(_debug) Task 029b135c-e5ee-4ce1-835a-6fe3f981c2fc: ref 0 aborting False
>Thread-15916604::INFO::2013-11-20 04:32:07,494::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916605::DEBUG::2013-11-20 04:32:07,501::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916605::INFO::2013-11-20 04:32:07,502::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916605::DEBUG::2013-11-20 04:32:07,503::task::495::TaskManager.Task::(_debug) Task 2bce4300-5e9c-4f6f-9c75-51abc405a3ac: moving from state init -> state preparing
>Thread-15916605::DEBUG::2013-11-20 04:32:07,505::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`edc3158b-75ed-40e7-b366-1ef09a2987df`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916605::DEBUG::2013-11-20 04:32:07,506::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916605::DEBUG::2013-11-20 04:32:07,509::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916605::DEBUG::2013-11-20 04:32:07,510::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`edc3158b-75ed-40e7-b366-1ef09a2987df`::Granted request
>Thread-15916605::DEBUG::2013-11-20 04:32:07,512::task::495::TaskManager.Task::(_debug) Task 2bce4300-5e9c-4f6f-9c75-51abc405a3ac: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916605::DEBUG::2013-11-20 04:32:07,513::task::495::TaskManager.Task::(_debug) Task 2bce4300-5e9c-4f6f-9c75-51abc405a3ac: ref 1 aborting False
>Thread-15916605::WARNING::2013-11-20 04:32:07,517::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916605::DEBUG::2013-11-20 04:32:07,520::task::495::TaskManager.Task::(_debug) Task 2bce4300-5e9c-4f6f-9c75-51abc405a3ac: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916605::DEBUG::2013-11-20 04:32:07,521::task::495::TaskManager.Task::(_debug) Task 2bce4300-5e9c-4f6f-9c75-51abc405a3ac: moving from state preparing -> state finished
>Thread-15916605::DEBUG::2013-11-20 04:32:07,523::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916605::DEBUG::2013-11-20 04:32:07,524::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916605::DEBUG::2013-11-20 04:32:07,526::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916605::DEBUG::2013-11-20 04:32:07,527::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916605::DEBUG::2013-11-20 04:32:07,528::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916605::DEBUG::2013-11-20 04:32:07,529::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916605::DEBUG::2013-11-20 04:32:07,531::task::495::TaskManager.Task::(_debug) Task 2bce4300-5e9c-4f6f-9c75-51abc405a3ac: ref 0 aborting False
>Thread-15916605::INFO::2013-11-20 04:32:07,534::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:32:07,552::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.121455 s, 8.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:32:09,571::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:32:10,116::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0717198 s, 14.3 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 04:32:10,855::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:32:10,864::task::495::TaskManager.Task::(_debug) Task bb842aa9-274a-44b7-9282-5a673bcbb7b6: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:32:10,868::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`30542119-4d79-4b13-9857-fec6194a6991`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:32:10,869::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:32:10,870::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:32:10,871::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`30542119-4d79-4b13-9857-fec6194a6991`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:32:10,872::task::495::TaskManager.Task::(_debug) Task bb842aa9-274a-44b7-9282-5a673bcbb7b6: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:32:10,873::task::495::TaskManager.Task::(_debug) Task bb842aa9-274a-44b7-9282-5a673bcbb7b6: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:32:10,874::task::495::TaskManager.Task::(_debug) Task bb842aa9-274a-44b7-9282-5a673bcbb7b6: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:32:10,874::task::495::TaskManager.Task::(_debug) Task bb842aa9-274a-44b7-9282-5a673bcbb7b6: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:32:10,875::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:32:10,876::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:32:10,876::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:32:10,877::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:32:10,878::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:32:10,880::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:32:10,881::task::495::TaskManager.Task::(_debug) Task bb842aa9-274a-44b7-9282-5a673bcbb7b6: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:32:10,882::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:32:12,133::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:32:12,661::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0767767 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:32:14,677::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:32:15,225::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0861189 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:32:17,241::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:32:17,777::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0774226 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15916609::DEBUG::2013-11-20 04:32:17,910::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916609::INFO::2013-11-20 04:32:17,912::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916609::DEBUG::2013-11-20 04:32:17,913::task::495::TaskManager.Task::(_debug) Task 00690cb2-25f5-4207-8c14-5f30a7aed8f4: moving from state init -> state preparing
>Thread-15916609::DEBUG::2013-11-20 04:32:17,914::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916609::DEBUG::2013-11-20 04:32:17,915::task::495::TaskManager.Task::(_debug) Task 00690cb2-25f5-4207-8c14-5f30a7aed8f4: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916609::DEBUG::2013-11-20 04:32:17,916::task::495::TaskManager.Task::(_debug) Task 00690cb2-25f5-4207-8c14-5f30a7aed8f4: moving from state preparing -> state finished
>Thread-15916609::DEBUG::2013-11-20 04:32:17,916::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916609::DEBUG::2013-11-20 04:32:17,917::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916609::DEBUG::2013-11-20 04:32:17,918::task::495::TaskManager.Task::(_debug) Task 00690cb2-25f5-4207-8c14-5f30a7aed8f4: ref 0 aborting False
>Thread-15916609::INFO::2013-11-20 04:32:17,918::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916610::DEBUG::2013-11-20 04:32:17,923::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916610::INFO::2013-11-20 04:32:17,924::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916610::DEBUG::2013-11-20 04:32:17,924::task::495::TaskManager.Task::(_debug) Task b9e3a802-5955-4fbd-97b5-85b400e6a0ab: moving from state init -> state preparing
>Thread-15916610::DEBUG::2013-11-20 04:32:17,926::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a27e5405-8590-4a5f-8b94-4e27ba4ff374`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916610::DEBUG::2013-11-20 04:32:17,926::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916610::DEBUG::2013-11-20 04:32:17,927::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916610::DEBUG::2013-11-20 04:32:17,928::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a27e5405-8590-4a5f-8b94-4e27ba4ff374`::Granted request
>Thread-15916610::DEBUG::2013-11-20 04:32:17,929::task::495::TaskManager.Task::(_debug) Task b9e3a802-5955-4fbd-97b5-85b400e6a0ab: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916610::DEBUG::2013-11-20 04:32:17,930::task::495::TaskManager.Task::(_debug) Task b9e3a802-5955-4fbd-97b5-85b400e6a0ab: ref 1 aborting False
>Thread-15916610::WARNING::2013-11-20 04:32:17,932::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916610::DEBUG::2013-11-20 04:32:17,933::task::495::TaskManager.Task::(_debug) Task b9e3a802-5955-4fbd-97b5-85b400e6a0ab: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916610::DEBUG::2013-11-20 04:32:17,934::task::495::TaskManager.Task::(_debug) Task b9e3a802-5955-4fbd-97b5-85b400e6a0ab: moving from state preparing -> state finished
>Thread-15916610::DEBUG::2013-11-20 04:32:17,935::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916610::DEBUG::2013-11-20 04:32:17,936::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916610::DEBUG::2013-11-20 04:32:17,936::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916610::DEBUG::2013-11-20 04:32:17,937::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916610::DEBUG::2013-11-20 04:32:17,938::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916610::DEBUG::2013-11-20 04:32:17,938::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916610::DEBUG::2013-11-20 04:32:17,939::task::495::TaskManager.Task::(_debug) Task b9e3a802-5955-4fbd-97b5-85b400e6a0ab: ref 0 aborting False
>Thread-15916610::INFO::2013-11-20 04:32:17,940::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913695::INFO::2013-11-20 04:32:18,973::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:32:18,974::task::495::TaskManager.Task::(_debug) Task ae2ee4c6-98be-446d-8ef9-b5731ed5921e: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:32:18,976::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6a0ec2c2-bb1e-4850-bb87-7e4577b23238`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:32:18,976::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:32:18,977::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:32:18,978::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6a0ec2c2-bb1e-4850-bb87-7e4577b23238`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:32:18,979::task::495::TaskManager.Task::(_debug) Task ae2ee4c6-98be-446d-8ef9-b5731ed5921e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:32:18,979::task::495::TaskManager.Task::(_debug) Task ae2ee4c6-98be-446d-8ef9-b5731ed5921e: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:32:18,980::task::495::TaskManager.Task::(_debug) Task ae2ee4c6-98be-446d-8ef9-b5731ed5921e: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:32:18,981::task::495::TaskManager.Task::(_debug) Task ae2ee4c6-98be-446d-8ef9-b5731ed5921e: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:32:18,981::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:32:18,982::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:32:18,983::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:32:18,983::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:32:18,984::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:32:18,984::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:32:18,985::task::495::TaskManager.Task::(_debug) Task ae2ee4c6-98be-446d-8ef9-b5731ed5921e: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:32:18,986::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:32:19,791::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:32:20,353::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0848937 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15914453::INFO::2013-11-20 04:32:20,887::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:32:20,888::task::495::TaskManager.Task::(_debug) Task 3a9dab42-c045-4305-8210-ad40bd8bfb1e: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:32:20,900::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`378f20c3-408c-4bbd-8dad-bd6c7eea2811`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:32:20,901::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:32:20,904::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:32:20,905::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`378f20c3-408c-4bbd-8dad-bd6c7eea2811`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:32:20,908::task::495::TaskManager.Task::(_debug) Task 3a9dab42-c045-4305-8210-ad40bd8bfb1e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:32:20,909::task::495::TaskManager.Task::(_debug) Task 3a9dab42-c045-4305-8210-ad40bd8bfb1e: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:32:20,910::task::495::TaskManager.Task::(_debug) Task 3a9dab42-c045-4305-8210-ad40bd8bfb1e: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:32:20,911::task::495::TaskManager.Task::(_debug) Task 3a9dab42-c045-4305-8210-ad40bd8bfb1e: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:32:20,911::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:32:20,912::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:32:20,913::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:32:20,914::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:32:20,914::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:32:20,915::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:32:20,916::task::495::TaskManager.Task::(_debug) Task 3a9dab42-c045-4305-8210-ad40bd8bfb1e: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:32:20,916::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:32:22,371::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:32:22,899::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.073098 s, 14.0 MB/s\n'; <rc> = 0
>Thread-15916612::INFO::2013-11-20 04:32:23,039::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916612::DEBUG::2013-11-20 04:32:23,040::task::495::TaskManager.Task::(_debug) Task aac3ef87-d3f5-48ba-98ff-8e168fbb7ea2: moving from state init -> state preparing
>Thread-15916612::DEBUG::2013-11-20 04:32:23,041::task::495::TaskManager.Task::(_debug) Task aac3ef87-d3f5-48ba-98ff-8e168fbb7ea2: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000432014465332', 'lastCheck': 1384921935.337837, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0213301181793', 'lastCheck': 1384921933.3578341, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00119614601135', 'lastCheck': 1384921940.4607639, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000746011734009', 'lastCheck': 1384921941.4552679, 'valid': True, 'code': 0}}
>Thread-15916612::DEBUG::2013-11-20 04:32:23,042::task::495::TaskManager.Task::(_debug) Task aac3ef87-d3f5-48ba-98ff-8e168fbb7ea2: moving from state preparing -> state finished
>Thread-15916612::DEBUG::2013-11-20 04:32:23,043::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916612::DEBUG::2013-11-20 04:32:23,044::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916612::DEBUG::2013-11-20 04:32:23,044::task::495::TaskManager.Task::(_debug) Task aac3ef87-d3f5-48ba-98ff-8e168fbb7ea2: ref 0 aborting False
>Thread-15916612::INFO::2013-11-20 04:32:23,045::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000432014465332', 'lastCheck': 1384921935.337837, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0213301181793', 'lastCheck': 1384921933.3578341, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00119614601135', 'lastCheck': 1384921940.4607639, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000746011734009', 'lastCheck': 1384921941.4552679, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:32:24,917::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:32:25,448::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0925564 s, 11.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:32:27,463::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:32:27,989::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0814364 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15916615::DEBUG::2013-11-20 04:32:28,674::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916615::INFO::2013-11-20 04:32:28,676::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916615::DEBUG::2013-11-20 04:32:28,677::task::495::TaskManager.Task::(_debug) Task 44f60b3d-fd3a-4ca8-9dfb-0134069993ae: moving from state init -> state preparing
>Thread-15916615::DEBUG::2013-11-20 04:32:28,678::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916615::DEBUG::2013-11-20 04:32:28,679::task::495::TaskManager.Task::(_debug) Task 44f60b3d-fd3a-4ca8-9dfb-0134069993ae: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916615::DEBUG::2013-11-20 04:32:28,679::task::495::TaskManager.Task::(_debug) Task 44f60b3d-fd3a-4ca8-9dfb-0134069993ae: moving from state preparing -> state finished
>Thread-15916615::DEBUG::2013-11-20 04:32:28,680::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916615::DEBUG::2013-11-20 04:32:28,681::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916615::DEBUG::2013-11-20 04:32:28,681::task::495::TaskManager.Task::(_debug) Task 44f60b3d-fd3a-4ca8-9dfb-0134069993ae: ref 0 aborting False
>Thread-15916615::INFO::2013-11-20 04:32:28,682::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916616::DEBUG::2013-11-20 04:32:28,687::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916616::INFO::2013-11-20 04:32:28,688::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916616::DEBUG::2013-11-20 04:32:28,689::task::495::TaskManager.Task::(_debug) Task 1bfaa73f-21e2-4304-ab7b-0da0608679e6: moving from state init -> state preparing
>Thread-15916616::DEBUG::2013-11-20 04:32:28,691::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8276f120-f7ff-46ca-889b-eaa7d9bb693c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916616::DEBUG::2013-11-20 04:32:28,692::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916616::DEBUG::2013-11-20 04:32:28,693::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916616::DEBUG::2013-11-20 04:32:28,694::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8276f120-f7ff-46ca-889b-eaa7d9bb693c`::Granted request
>Thread-15916616::DEBUG::2013-11-20 04:32:28,695::task::495::TaskManager.Task::(_debug) Task 1bfaa73f-21e2-4304-ab7b-0da0608679e6: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916616::DEBUG::2013-11-20 04:32:28,695::task::495::TaskManager.Task::(_debug) Task 1bfaa73f-21e2-4304-ab7b-0da0608679e6: ref 1 aborting False
>Thread-15916616::WARNING::2013-11-20 04:32:28,698::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916616::DEBUG::2013-11-20 04:32:28,699::task::495::TaskManager.Task::(_debug) Task 1bfaa73f-21e2-4304-ab7b-0da0608679e6: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916616::DEBUG::2013-11-20 04:32:28,700::task::495::TaskManager.Task::(_debug) Task 1bfaa73f-21e2-4304-ab7b-0da0608679e6: moving from state preparing -> state finished
>Thread-15916616::DEBUG::2013-11-20 04:32:28,701::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916616::DEBUG::2013-11-20 04:32:28,702::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916616::DEBUG::2013-11-20 04:32:28,703::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916616::DEBUG::2013-11-20 04:32:28,704::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916616::DEBUG::2013-11-20 04:32:28,705::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916616::DEBUG::2013-11-20 04:32:28,705::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916616::DEBUG::2013-11-20 04:32:28,706::task::495::TaskManager.Task::(_debug) Task 1bfaa73f-21e2-4304-ab7b-0da0608679e6: ref 0 aborting False
>Thread-15916616::INFO::2013-11-20 04:32:28,707::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:32:30,005::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 04:32:30,215::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:32:30,223::task::495::TaskManager.Task::(_debug) Task cef7f8af-ab87-4fa3-9213-a4409cc6881b: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:32:30,227::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e570d4c3-548c-4380-b922-97d1b2730aa3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:32:30,228::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:32:30,230::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:32:30,230::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e570d4c3-548c-4380-b922-97d1b2730aa3`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:32:30,239::task::495::TaskManager.Task::(_debug) Task cef7f8af-ab87-4fa3-9213-a4409cc6881b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914519::DEBUG::2013-11-20 04:32:30,240::task::495::TaskManager.Task::(_debug) Task cef7f8af-ab87-4fa3-9213-a4409cc6881b: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:32:30,240::task::495::TaskManager.Task::(_debug) Task cef7f8af-ab87-4fa3-9213-a4409cc6881b: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 04:32:30,241::task::495::TaskManager.Task::(_debug) Task cef7f8af-ab87-4fa3-9213-a4409cc6881b: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:32:30,242::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:32:30,242::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:32:30,243::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914519::DEBUG::2013-11-20 04:32:30,244::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:32:30,245::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:32:30,245::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:32:30,246::task::495::TaskManager.Task::(_debug) Task cef7f8af-ab87-4fa3-9213-a4409cc6881b: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 04:32:30,246::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Dummy-15912651::DEBUG::2013-11-20 04:32:30,526::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0795512 s, 12.9 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 04:32:30,931::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:32:30,943::task::495::TaskManager.Task::(_debug) Task 394fcc95-7b5e-4bae-b7e9-6198e11542a3: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:32:30,944::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c8782d52-9755-4e28-a8e3-1318c4c469a9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:32:30,946::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:32:30,947::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:32:30,947::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c8782d52-9755-4e28-a8e3-1318c4c469a9`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:32:30,949::task::495::TaskManager.Task::(_debug) Task 394fcc95-7b5e-4bae-b7e9-6198e11542a3: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:32:30,950::task::495::TaskManager.Task::(_debug) Task 394fcc95-7b5e-4bae-b7e9-6198e11542a3: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:32:30,950::task::495::TaskManager.Task::(_debug) Task 394fcc95-7b5e-4bae-b7e9-6198e11542a3: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 04:32:30,951::task::495::TaskManager.Task::(_debug) Task 394fcc95-7b5e-4bae-b7e9-6198e11542a3: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:32:30,952::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:32:30,952::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:32:30,953::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:32:30,954::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:32:30,955::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:32:30,955::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:32:30,956::task::495::TaskManager.Task::(_debug) Task 394fcc95-7b5e-4bae-b7e9-6198e11542a3: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:32:30,957::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:32:30,958::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:32:30,958::task::495::TaskManager.Task::(_debug) Task 5cca9ea1-d0ec-4b73-a671-118ed81643da: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:32:30,959::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`be74ff6a-a1bb-4541-b0a3-f1c1d652e550`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:32:30,960::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:32:30,961::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:32:30,961::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`be74ff6a-a1bb-4541-b0a3-f1c1d652e550`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:32:30,962::task::495::TaskManager.Task::(_debug) Task 5cca9ea1-d0ec-4b73-a671-118ed81643da: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:32:30,963::task::495::TaskManager.Task::(_debug) Task 5cca9ea1-d0ec-4b73-a671-118ed81643da: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:32:30,964::task::495::TaskManager.Task::(_debug) Task 5cca9ea1-d0ec-4b73-a671-118ed81643da: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:32:30,964::task::495::TaskManager.Task::(_debug) Task 5cca9ea1-d0ec-4b73-a671-118ed81643da: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:32:30,965::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:32:30,965::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:32:30,966::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:32:30,967::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:32:30,967::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:32:30,968::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:32:30,969::task::495::TaskManager.Task::(_debug) Task 5cca9ea1-d0ec-4b73-a671-118ed81643da: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:32:30,970::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:32:30,970::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:32:30,971::task::495::TaskManager.Task::(_debug) Task 28eafa3f-6cd2-4d73-abce-6426c97eb71b: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:32:30,972::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`4706b8f5-eb9e-4ef0-8c09-417faba2771d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:32:30,973::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:32:30,973::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:32:30,974::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`4706b8f5-eb9e-4ef0-8c09-417faba2771d`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:32:30,975::task::495::TaskManager.Task::(_debug) Task 28eafa3f-6cd2-4d73-abce-6426c97eb71b: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:32:30,975::task::495::TaskManager.Task::(_debug) Task 28eafa3f-6cd2-4d73-abce-6426c97eb71b: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:32:30,976::task::495::TaskManager.Task::(_debug) Task 28eafa3f-6cd2-4d73-abce-6426c97eb71b: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:32:30,977::task::495::TaskManager.Task::(_debug) Task 28eafa3f-6cd2-4d73-abce-6426c97eb71b: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:32:30,977::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:32:30,978::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:32:30,978::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:32:30,979::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:32:30,979::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:32:30,980::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:32:30,981::task::495::TaskManager.Task::(_debug) Task 28eafa3f-6cd2-4d73-abce-6426c97eb71b: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:32:30,981::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:32:30,982::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:32:30,982::task::495::TaskManager.Task::(_debug) Task 705b6917-706c-4715-b519-56e2c43e9a4e: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:32:30,983::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`05341dea-8724-44b8-bf9c-99c323524b48`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:32:30,984::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:32:30,985::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:32:30,985::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`05341dea-8724-44b8-bf9c-99c323524b48`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:32:30,986::task::495::TaskManager.Task::(_debug) Task 705b6917-706c-4715-b519-56e2c43e9a4e: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:32:30,987::task::495::TaskManager.Task::(_debug) Task 705b6917-706c-4715-b519-56e2c43e9a4e: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:32:30,987::task::495::TaskManager.Task::(_debug) Task 705b6917-706c-4715-b519-56e2c43e9a4e: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:32:30,988::task::495::TaskManager.Task::(_debug) Task 705b6917-706c-4715-b519-56e2c43e9a4e: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:32:30,988::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:32:30,989::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:32:30,989::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:32:30,990::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:32:30,991::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:32:30,991::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:32:30,992::task::495::TaskManager.Task::(_debug) Task 705b6917-706c-4715-b519-56e2c43e9a4e: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:32:30,992::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:32:32,541::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:32:33,083::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0854315 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:32:35,104::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 04:32:35,332::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:32:35,334::task::495::TaskManager.Task::(_debug) Task 02d9ff52-a73e-4f75-bd30-e324cf4464dc: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:32:35,337::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4092a052-8405-41da-8d29-ee9470dbd333`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:32:35,338::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:32:35,339::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:32:35,339::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4092a052-8405-41da-8d29-ee9470dbd333`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:32:35,341::task::495::TaskManager.Task::(_debug) Task 02d9ff52-a73e-4f75-bd30-e324cf4464dc: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:32:35,341::task::495::TaskManager.Task::(_debug) Task 02d9ff52-a73e-4f75-bd30-e324cf4464dc: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:32:35,342::task::495::TaskManager.Task::(_debug) Task 02d9ff52-a73e-4f75-bd30-e324cf4464dc: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:32:35,343::task::495::TaskManager.Task::(_debug) Task 02d9ff52-a73e-4f75-bd30-e324cf4464dc: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:32:35,344::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:32:35,344::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:32:35,345::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:32:35,346::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:32:35,347::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:32:35,347::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:32:35,348::task::495::TaskManager.Task::(_debug) Task 02d9ff52-a73e-4f75-bd30-e324cf4464dc: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:32:35,349::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:32:35,667::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0801242 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:32:37,683::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:32:38,212::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0856067 s, 12.0 MB/s\n'; <rc> = 0
>Thread-15916620::INFO::2013-11-20 04:32:39,218::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916620::DEBUG::2013-11-20 04:32:39,219::task::495::TaskManager.Task::(_debug) Task a8225db7-8db6-4ccd-93f8-6628ec976b03: moving from state init -> state preparing
>Thread-15916620::DEBUG::2013-11-20 04:32:39,220::task::495::TaskManager.Task::(_debug) Task a8225db7-8db6-4ccd-93f8-6628ec976b03: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000472068786621', 'lastCheck': 1384921955.738519, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00214695930481', 'lastCheck': 1384921953.7382939, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0380058288574', 'lastCheck': 1384921950.7247419, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00058388710022', 'lastCheck': 1384921951.6781809, 'valid': True, 'code': 0}}
>Thread-15916620::DEBUG::2013-11-20 04:32:39,221::task::495::TaskManager.Task::(_debug) Task a8225db7-8db6-4ccd-93f8-6628ec976b03: moving from state preparing -> state finished
>Thread-15916620::DEBUG::2013-11-20 04:32:39,221::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916620::DEBUG::2013-11-20 04:32:39,222::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916620::DEBUG::2013-11-20 04:32:39,223::task::495::TaskManager.Task::(_debug) Task a8225db7-8db6-4ccd-93f8-6628ec976b03: ref 0 aborting False
>Thread-15916620::INFO::2013-11-20 04:32:39,223::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000472068786621', 'lastCheck': 1384921955.738519, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00214695930481', 'lastCheck': 1384921953.7382939, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0380058288574', 'lastCheck': 1384921950.7247419, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00058388710022', 'lastCheck': 1384921951.6781809, 'valid': True, 'code': 0}}
>Thread-15916622::DEBUG::2013-11-20 04:32:39,434::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916622::INFO::2013-11-20 04:32:39,435::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916622::DEBUG::2013-11-20 04:32:39,436::task::495::TaskManager.Task::(_debug) Task 4b860b9f-8f39-4837-b086-5e8ddde24f3b: moving from state init -> state preparing
>Thread-15916622::DEBUG::2013-11-20 04:32:39,436::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916622::DEBUG::2013-11-20 04:32:39,437::task::495::TaskManager.Task::(_debug) Task 4b860b9f-8f39-4837-b086-5e8ddde24f3b: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916622::DEBUG::2013-11-20 04:32:39,438::task::495::TaskManager.Task::(_debug) Task 4b860b9f-8f39-4837-b086-5e8ddde24f3b: moving from state preparing -> state finished
>Thread-15916622::DEBUG::2013-11-20 04:32:39,439::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916622::DEBUG::2013-11-20 04:32:39,439::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916622::DEBUG::2013-11-20 04:32:39,440::task::495::TaskManager.Task::(_debug) Task 4b860b9f-8f39-4837-b086-5e8ddde24f3b: ref 0 aborting False
>Thread-15916622::INFO::2013-11-20 04:32:39,440::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916623::DEBUG::2013-11-20 04:32:39,444::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916623::INFO::2013-11-20 04:32:39,445::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916623::DEBUG::2013-11-20 04:32:39,446::task::495::TaskManager.Task::(_debug) Task 7a0a1925-2062-4377-ab6b-e09c8730bf83: moving from state init -> state preparing
>Thread-15916623::DEBUG::2013-11-20 04:32:39,447::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`73f6935a-ba3d-41f8-a95b-0553cc6c53ca`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916623::DEBUG::2013-11-20 04:32:39,448::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916623::DEBUG::2013-11-20 04:32:39,448::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916623::DEBUG::2013-11-20 04:32:39,449::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`73f6935a-ba3d-41f8-a95b-0553cc6c53ca`::Granted request
>Thread-15916623::DEBUG::2013-11-20 04:32:39,450::task::495::TaskManager.Task::(_debug) Task 7a0a1925-2062-4377-ab6b-e09c8730bf83: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916623::DEBUG::2013-11-20 04:32:39,450::task::495::TaskManager.Task::(_debug) Task 7a0a1925-2062-4377-ab6b-e09c8730bf83: ref 1 aborting False
>Thread-15916623::WARNING::2013-11-20 04:32:39,452::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916623::DEBUG::2013-11-20 04:32:39,453::task::495::TaskManager.Task::(_debug) Task 7a0a1925-2062-4377-ab6b-e09c8730bf83: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916623::DEBUG::2013-11-20 04:32:39,454::task::495::TaskManager.Task::(_debug) Task 7a0a1925-2062-4377-ab6b-e09c8730bf83: moving from state preparing -> state finished
>Thread-15916623::DEBUG::2013-11-20 04:32:39,455::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916623::DEBUG::2013-11-20 04:32:39,455::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916623::DEBUG::2013-11-20 04:32:39,456::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916623::DEBUG::2013-11-20 04:32:39,457::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916623::DEBUG::2013-11-20 04:32:39,458::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916623::DEBUG::2013-11-20 04:32:39,460::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916623::DEBUG::2013-11-20 04:32:39,460::task::495::TaskManager.Task::(_debug) Task 7a0a1925-2062-4377-ab6b-e09c8730bf83: ref 0 aborting False
>Thread-15916623::INFO::2013-11-20 04:32:39,461::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:32:40,227::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:32:40,803::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0870243 s, 11.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:32:42,818::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:32:43,351::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0935882 s, 10.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:32:45,370::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:32:45,938::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0788735 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:32:47,948::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:32:48,187::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:32:48,189::task::495::TaskManager.Task::(_debug) Task d57125b1-1912-4c85-8ed7-3fba21e711a6: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:32:48,192::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c939048b-8334-4c6b-afd2-60cd491d2c13`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:32:48,192::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:32:48,194::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:32:48,195::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c939048b-8334-4c6b-afd2-60cd491d2c13`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:32:48,196::task::495::TaskManager.Task::(_debug) Task d57125b1-1912-4c85-8ed7-3fba21e711a6: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:32:48,197::task::495::TaskManager.Task::(_debug) Task d57125b1-1912-4c85-8ed7-3fba21e711a6: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:32:48,198::task::495::TaskManager.Task::(_debug) Task d57125b1-1912-4c85-8ed7-3fba21e711a6: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:32:48,199::task::495::TaskManager.Task::(_debug) Task d57125b1-1912-4c85-8ed7-3fba21e711a6: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:32:48,199::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:32:48,200::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:32:48,201::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:32:48,202::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:32:48,202::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:32:48,203::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:32:48,204::task::495::TaskManager.Task::(_debug) Task d57125b1-1912-4c85-8ed7-3fba21e711a6: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:32:48,205::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:32:48,516::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0929854 s, 11.0 MB/s\n'; <rc> = 0
>Thread-15916627::DEBUG::2013-11-20 04:32:49,854::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916627::INFO::2013-11-20 04:32:49,856::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916627::DEBUG::2013-11-20 04:32:49,856::task::495::TaskManager.Task::(_debug) Task 99cecc18-c0b1-4772-9812-9a9607ff60d5: moving from state init -> state preparing
>Thread-15916627::DEBUG::2013-11-20 04:32:49,857::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916627::DEBUG::2013-11-20 04:32:49,858::task::495::TaskManager.Task::(_debug) Task 99cecc18-c0b1-4772-9812-9a9607ff60d5: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916627::DEBUG::2013-11-20 04:32:49,858::task::495::TaskManager.Task::(_debug) Task 99cecc18-c0b1-4772-9812-9a9607ff60d5: moving from state preparing -> state finished
>Thread-15916627::DEBUG::2013-11-20 04:32:49,859::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916627::DEBUG::2013-11-20 04:32:49,859::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916627::DEBUG::2013-11-20 04:32:49,860::task::495::TaskManager.Task::(_debug) Task 99cecc18-c0b1-4772-9812-9a9607ff60d5: ref 0 aborting False
>Thread-15916627::INFO::2013-11-20 04:32:49,861::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916628::DEBUG::2013-11-20 04:32:49,865::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916628::INFO::2013-11-20 04:32:49,867::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916628::DEBUG::2013-11-20 04:32:49,867::task::495::TaskManager.Task::(_debug) Task 055e058e-f749-44d7-b305-d8baaa2a5090: moving from state init -> state preparing
>Thread-15916628::DEBUG::2013-11-20 04:32:49,868::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`628fff67-47f7-4b13-8315-a0c6749cde72`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916628::DEBUG::2013-11-20 04:32:49,869::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916628::DEBUG::2013-11-20 04:32:49,870::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916628::DEBUG::2013-11-20 04:32:49,870::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`628fff67-47f7-4b13-8315-a0c6749cde72`::Granted request
>Thread-15916628::DEBUG::2013-11-20 04:32:49,871::task::495::TaskManager.Task::(_debug) Task 055e058e-f749-44d7-b305-d8baaa2a5090: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916628::DEBUG::2013-11-20 04:32:49,872::task::495::TaskManager.Task::(_debug) Task 055e058e-f749-44d7-b305-d8baaa2a5090: ref 1 aborting False
>Thread-15916628::WARNING::2013-11-20 04:32:49,874::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916628::DEBUG::2013-11-20 04:32:49,875::task::495::TaskManager.Task::(_debug) Task 055e058e-f749-44d7-b305-d8baaa2a5090: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916628::DEBUG::2013-11-20 04:32:49,876::task::495::TaskManager.Task::(_debug) Task 055e058e-f749-44d7-b305-d8baaa2a5090: moving from state preparing -> state finished
>Thread-15916628::DEBUG::2013-11-20 04:32:49,877::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916628::DEBUG::2013-11-20 04:32:49,877::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916628::DEBUG::2013-11-20 04:32:49,878::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916628::DEBUG::2013-11-20 04:32:49,879::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916628::DEBUG::2013-11-20 04:32:49,879::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916628::DEBUG::2013-11-20 04:32:49,880::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916628::DEBUG::2013-11-20 04:32:49,881::task::495::TaskManager.Task::(_debug) Task 055e058e-f749-44d7-b305-d8baaa2a5090: ref 0 aborting False
>Thread-15916628::INFO::2013-11-20 04:32:49,882::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:32:50,532::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915918::INFO::2013-11-20 04:32:50,743::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:32:50,745::task::495::TaskManager.Task::(_debug) Task 58d3ff26-69ce-4ea0-8b8c-631a28daa167: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:32:50,749::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`522bfce4-3146-4416-ae96-9243aa621c7d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:32:50,750::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:32:50,751::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:32:50,752::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`522bfce4-3146-4416-ae96-9243aa621c7d`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:32:50,753::task::495::TaskManager.Task::(_debug) Task 58d3ff26-69ce-4ea0-8b8c-631a28daa167: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:32:50,754::task::495::TaskManager.Task::(_debug) Task 58d3ff26-69ce-4ea0-8b8c-631a28daa167: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:32:50,754::task::495::TaskManager.Task::(_debug) Task 58d3ff26-69ce-4ea0-8b8c-631a28daa167: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:32:50,755::task::495::TaskManager.Task::(_debug) Task 58d3ff26-69ce-4ea0-8b8c-631a28daa167: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:32:50,756::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:32:50,756::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:32:50,757::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:32:50,758::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:32:50,758::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:32:50,759::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:32:50,760::task::495::TaskManager.Task::(_debug) Task 58d3ff26-69ce-4ea0-8b8c-631a28daa167: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:32:50,760::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:32:51,061::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0763738 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:32:53,076::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:32:53,606::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0760448 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15916630::INFO::2013-11-20 04:32:55,116::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916630::DEBUG::2013-11-20 04:32:55,117::task::495::TaskManager.Task::(_debug) Task 5a2aeefc-01cc-495e-96d2-c40a549b44c0: moving from state init -> state preparing
>Thread-15916630::DEBUG::2013-11-20 04:32:55,118::task::495::TaskManager.Task::(_debug) Task 5a2aeefc-01cc-495e-96d2-c40a549b44c0: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000480890274048', 'lastCheck': 1384921966.0271749, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00136208534241', 'lastCheck': 1384921974.3027251, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00189900398254', 'lastCheck': 1384921971.1902621, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00049090385437', 'lastCheck': 1384921972.185746, 'valid': True, 'code': 0}}
>Thread-15916630::DEBUG::2013-11-20 04:32:55,119::task::495::TaskManager.Task::(_debug) Task 5a2aeefc-01cc-495e-96d2-c40a549b44c0: moving from state preparing -> state finished
>Thread-15916630::DEBUG::2013-11-20 04:32:55,119::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916630::DEBUG::2013-11-20 04:32:55,120::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916630::DEBUG::2013-11-20 04:32:55,121::task::495::TaskManager.Task::(_debug) Task 5a2aeefc-01cc-495e-96d2-c40a549b44c0: ref 0 aborting False
>Thread-15916630::INFO::2013-11-20 04:32:55,121::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000480890274048', 'lastCheck': 1384921966.0271749, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00136208534241', 'lastCheck': 1384921974.3027251, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00189900398254', 'lastCheck': 1384921971.1902621, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00049090385437', 'lastCheck': 1384921972.185746, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:32:55,620::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915863::INFO::2013-11-20 04:32:55,861::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:32:55,865::task::495::TaskManager.Task::(_debug) Task c9b1cb2e-0a60-4614-a371-a936b466db02: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:32:55,869::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d29d9edd-6f1f-4d3e-94c1-c6acdaef53ab`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:32:55,870::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:32:55,870::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:32:55,871::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d29d9edd-6f1f-4d3e-94c1-c6acdaef53ab`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:32:55,872::task::495::TaskManager.Task::(_debug) Task c9b1cb2e-0a60-4614-a371-a936b466db02: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:32:55,873::task::495::TaskManager.Task::(_debug) Task c9b1cb2e-0a60-4614-a371-a936b466db02: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:32:55,874::task::495::TaskManager.Task::(_debug) Task c9b1cb2e-0a60-4614-a371-a936b466db02: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:32:55,875::task::495::TaskManager.Task::(_debug) Task c9b1cb2e-0a60-4614-a371-a936b466db02: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:32:55,875::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:32:55,876::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:32:55,877::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:32:55,878::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:32:55,878::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:32:55,879::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:32:55,880::task::495::TaskManager.Task::(_debug) Task c9b1cb2e-0a60-4614-a371-a936b466db02: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:32:55,881::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:32:56,136::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0535566 s, 19.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:32:58,145::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:32:58,701::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0629992 s, 16.3 MB/s\n'; <rc> = 0
>Thread-15916633::DEBUG::2013-11-20 04:33:00,362::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916633::INFO::2013-11-20 04:33:00,363::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916633::DEBUG::2013-11-20 04:33:00,364::task::495::TaskManager.Task::(_debug) Task c80b4671-f2e4-4212-9a2a-734161bd12f1: moving from state init -> state preparing
>Thread-15916633::DEBUG::2013-11-20 04:33:00,365::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916633::DEBUG::2013-11-20 04:33:00,365::task::495::TaskManager.Task::(_debug) Task c80b4671-f2e4-4212-9a2a-734161bd12f1: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916633::DEBUG::2013-11-20 04:33:00,366::task::495::TaskManager.Task::(_debug) Task c80b4671-f2e4-4212-9a2a-734161bd12f1: moving from state preparing -> state finished
>Thread-15916633::DEBUG::2013-11-20 04:33:00,367::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916633::DEBUG::2013-11-20 04:33:00,367::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916633::DEBUG::2013-11-20 04:33:00,368::task::495::TaskManager.Task::(_debug) Task c80b4671-f2e4-4212-9a2a-734161bd12f1: ref 0 aborting False
>Thread-15916633::INFO::2013-11-20 04:33:00,369::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916634::DEBUG::2013-11-20 04:33:00,373::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916634::INFO::2013-11-20 04:33:00,374::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916634::DEBUG::2013-11-20 04:33:00,375::task::495::TaskManager.Task::(_debug) Task c9171749-836f-470c-8f1d-32359236f470: moving from state init -> state preparing
>Thread-15916634::DEBUG::2013-11-20 04:33:00,376::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2edf394e-7f7f-4f81-a257-0d9436fe24c6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916634::DEBUG::2013-11-20 04:33:00,377::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916634::DEBUG::2013-11-20 04:33:00,377::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916634::DEBUG::2013-11-20 04:33:00,378::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2edf394e-7f7f-4f81-a257-0d9436fe24c6`::Granted request
>Thread-15916634::DEBUG::2013-11-20 04:33:00,379::task::495::TaskManager.Task::(_debug) Task c9171749-836f-470c-8f1d-32359236f470: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916634::DEBUG::2013-11-20 04:33:00,380::task::495::TaskManager.Task::(_debug) Task c9171749-836f-470c-8f1d-32359236f470: ref 1 aborting False
>Thread-15916634::WARNING::2013-11-20 04:33:00,382::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916634::DEBUG::2013-11-20 04:33:00,383::task::495::TaskManager.Task::(_debug) Task c9171749-836f-470c-8f1d-32359236f470: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916634::DEBUG::2013-11-20 04:33:00,384::task::495::TaskManager.Task::(_debug) Task c9171749-836f-470c-8f1d-32359236f470: moving from state preparing -> state finished
>Thread-15916634::DEBUG::2013-11-20 04:33:00,385::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916634::DEBUG::2013-11-20 04:33:00,385::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916634::DEBUG::2013-11-20 04:33:00,386::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916634::DEBUG::2013-11-20 04:33:00,387::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916634::DEBUG::2013-11-20 04:33:00,387::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916634::DEBUG::2013-11-20 04:33:00,388::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916634::DEBUG::2013-11-20 04:33:00,388::task::495::TaskManager.Task::(_debug) Task c9171749-836f-470c-8f1d-32359236f470: ref 0 aborting False
>Thread-15916634::INFO::2013-11-20 04:33:00,389::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:33:00,713::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 04:33:01,048::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:33:01,049::task::495::TaskManager.Task::(_debug) Task 6d08f039-9b86-4f33-b8aa-582922edf590: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:33:01,060::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`39383cdc-7143-4c33-93c7-62e91905b160`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:33:01,061::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:33:01,062::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:33:01,063::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`39383cdc-7143-4c33-93c7-62e91905b160`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:33:01,064::task::495::TaskManager.Task::(_debug) Task 6d08f039-9b86-4f33-b8aa-582922edf590: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:33:01,065::task::495::TaskManager.Task::(_debug) Task 6d08f039-9b86-4f33-b8aa-582922edf590: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:33:01,066::task::495::TaskManager.Task::(_debug) Task 6d08f039-9b86-4f33-b8aa-582922edf590: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:33:01,067::task::495::TaskManager.Task::(_debug) Task 6d08f039-9b86-4f33-b8aa-582922edf590: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:33:01,067::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:33:01,068::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:33:01,069::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:33:01,070::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:33:01,070::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:33:01,071::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:33:01,072::task::495::TaskManager.Task::(_debug) Task 6d08f039-9b86-4f33-b8aa-582922edf590: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:33:01,073::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:33:01,290::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0824157 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:33:03,308::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:33:03,795::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0832088 s, 12.3 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 04:33:04,477::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:33:04,478::task::495::TaskManager.Task::(_debug) Task 38bd9c3a-1e41-4842-ad01-ac8149064e8e: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:33:04,480::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8242af70-d7bc-40ac-84c3-aee21dc12567`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:33:04,481::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:33:04,481::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:33:04,482::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8242af70-d7bc-40ac-84c3-aee21dc12567`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:33:04,483::task::495::TaskManager.Task::(_debug) Task 38bd9c3a-1e41-4842-ad01-ac8149064e8e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:33:04,484::task::495::TaskManager.Task::(_debug) Task 38bd9c3a-1e41-4842-ad01-ac8149064e8e: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:33:04,485::task::495::TaskManager.Task::(_debug) Task 38bd9c3a-1e41-4842-ad01-ac8149064e8e: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:33:04,485::task::495::TaskManager.Task::(_debug) Task 38bd9c3a-1e41-4842-ad01-ac8149064e8e: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:33:04,486::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:33:04,487::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:33:04,487::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:33:04,488::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:33:04,489::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:33:04,489::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:33:04,490::task::495::TaskManager.Task::(_debug) Task 38bd9c3a-1e41-4842-ad01-ac8149064e8e: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:33:04,491::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:33:04,491::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:33:04,492::task::495::TaskManager.Task::(_debug) Task 4a665a21-71db-4163-add4-a5a62ae119ad: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:33:04,493::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`372c5b52-a291-4a65-b91b-1a72782ac0b2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:33:04,494::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:33:04,494::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:33:04,495::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`372c5b52-a291-4a65-b91b-1a72782ac0b2`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:33:04,496::task::495::TaskManager.Task::(_debug) Task 4a665a21-71db-4163-add4-a5a62ae119ad: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:33:04,497::task::495::TaskManager.Task::(_debug) Task 4a665a21-71db-4163-add4-a5a62ae119ad: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:33:04,497::task::495::TaskManager.Task::(_debug) Task 4a665a21-71db-4163-add4-a5a62ae119ad: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:33:04,498::task::495::TaskManager.Task::(_debug) Task 4a665a21-71db-4163-add4-a5a62ae119ad: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:33:04,498::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:33:04,499::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:33:04,500::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:33:04,500::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:33:04,501::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:33:04,501::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:33:04,502::task::495::TaskManager.Task::(_debug) Task 4a665a21-71db-4163-add4-a5a62ae119ad: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:33:04,503::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:33:05,810::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:33:06,357::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0757258 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:33:08,373::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:33:08,918::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.084958 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15916638::DEBUG::2013-11-20 04:33:10,769::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916638::INFO::2013-11-20 04:33:10,771::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916638::DEBUG::2013-11-20 04:33:10,771::task::495::TaskManager.Task::(_debug) Task 65ccff5c-acbb-416f-8d9c-76c63ce63d28: moving from state init -> state preparing
>Thread-15916638::DEBUG::2013-11-20 04:33:10,772::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916638::DEBUG::2013-11-20 04:33:10,773::task::495::TaskManager.Task::(_debug) Task 65ccff5c-acbb-416f-8d9c-76c63ce63d28: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916638::DEBUG::2013-11-20 04:33:10,773::task::495::TaskManager.Task::(_debug) Task 65ccff5c-acbb-416f-8d9c-76c63ce63d28: moving from state preparing -> state finished
>Thread-15916638::DEBUG::2013-11-20 04:33:10,774::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916638::DEBUG::2013-11-20 04:33:10,775::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916638::DEBUG::2013-11-20 04:33:10,775::task::495::TaskManager.Task::(_debug) Task 65ccff5c-acbb-416f-8d9c-76c63ce63d28: ref 0 aborting False
>Thread-15916638::INFO::2013-11-20 04:33:10,776::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916639::DEBUG::2013-11-20 04:33:10,780::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916639::INFO::2013-11-20 04:33:10,781::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916639::DEBUG::2013-11-20 04:33:10,782::task::495::TaskManager.Task::(_debug) Task 8267e0b6-70d4-49ef-9a75-0297bbbddb8c: moving from state init -> state preparing
>Thread-15916639::DEBUG::2013-11-20 04:33:10,784::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a47e948d-44ea-46d6-b22a-13bdf31d252a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916639::DEBUG::2013-11-20 04:33:10,784::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916639::DEBUG::2013-11-20 04:33:10,785::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916639::DEBUG::2013-11-20 04:33:10,786::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a47e948d-44ea-46d6-b22a-13bdf31d252a`::Granted request
>Thread-15916639::DEBUG::2013-11-20 04:33:10,787::task::495::TaskManager.Task::(_debug) Task 8267e0b6-70d4-49ef-9a75-0297bbbddb8c: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916639::DEBUG::2013-11-20 04:33:10,787::task::495::TaskManager.Task::(_debug) Task 8267e0b6-70d4-49ef-9a75-0297bbbddb8c: ref 1 aborting False
>Thread-15916639::WARNING::2013-11-20 04:33:10,789::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916639::DEBUG::2013-11-20 04:33:10,790::task::495::TaskManager.Task::(_debug) Task 8267e0b6-70d4-49ef-9a75-0297bbbddb8c: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916639::DEBUG::2013-11-20 04:33:10,791::task::495::TaskManager.Task::(_debug) Task 8267e0b6-70d4-49ef-9a75-0297bbbddb8c: moving from state preparing -> state finished
>Thread-15916639::DEBUG::2013-11-20 04:33:10,791::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916639::DEBUG::2013-11-20 04:33:10,792::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916639::DEBUG::2013-11-20 04:33:10,793::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916639::DEBUG::2013-11-20 04:33:10,793::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916639::DEBUG::2013-11-20 04:33:10,794::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916639::DEBUG::2013-11-20 04:33:10,794::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916639::DEBUG::2013-11-20 04:33:10,795::task::495::TaskManager.Task::(_debug) Task 8267e0b6-70d4-49ef-9a75-0297bbbddb8c: ref 0 aborting False
>Thread-15916639::INFO::2013-11-20 04:33:10,796::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:33:10,933::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 04:33:11,138::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:33:11,142::task::495::TaskManager.Task::(_debug) Task d8319417-cadb-4f47-a7c7-8a09153aeb75: moving from state init -> state preparing
>Thread-15916640::INFO::2013-11-20 04:33:11,145::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916640::DEBUG::2013-11-20 04:33:11,147::task::495::TaskManager.Task::(_debug) Task cc1fe5be-bfbe-422d-a44b-84d168f544c5: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:33:11,150::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3e6a416d-c495-4376-a7ff-38ad6b4c939b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15916640::DEBUG::2013-11-20 04:33:11,151::task::495::TaskManager.Task::(_debug) Task cc1fe5be-bfbe-422d-a44b-84d168f544c5: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000479936599731', 'lastCheck': 1384921986.4664381, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00225901603699', 'lastCheck': 1384921984.4663379, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00134706497192', 'lastCheck': 1384921981.3846681, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000430822372437', 'lastCheck': 1384921982.380192, 'valid': True, 'code': 0}}
>Thread-15912392::DEBUG::2013-11-20 04:33:11,151::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15916640::DEBUG::2013-11-20 04:33:11,152::task::495::TaskManager.Task::(_debug) Task cc1fe5be-bfbe-422d-a44b-84d168f544c5: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:33:11,155::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15916640::DEBUG::2013-11-20 04:33:11,155::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15912392::DEBUG::2013-11-20 04:33:11,156::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3e6a416d-c495-4376-a7ff-38ad6b4c939b`::Granted request
>Thread-15916640::DEBUG::2013-11-20 04:33:11,158::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916640::DEBUG::2013-11-20 04:33:11,163::task::495::TaskManager.Task::(_debug) Task cc1fe5be-bfbe-422d-a44b-84d168f544c5: ref 0 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:33:11,163::task::495::TaskManager.Task::(_debug) Task d8319417-cadb-4f47-a7c7-8a09153aeb75: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15916640::INFO::2013-11-20 04:33:11,164::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000479936599731', 'lastCheck': 1384921986.4664381, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00225901603699', 'lastCheck': 1384921984.4663379, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00134706497192', 'lastCheck': 1384921981.3846681, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000430822372437', 'lastCheck': 1384921982.380192, 'valid': True, 'code': 0}}
>Thread-15912392::DEBUG::2013-11-20 04:33:11,165::task::495::TaskManager.Task::(_debug) Task d8319417-cadb-4f47-a7c7-8a09153aeb75: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:33:11,168::task::495::TaskManager.Task::(_debug) Task d8319417-cadb-4f47-a7c7-8a09153aeb75: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:33:11,168::task::495::TaskManager.Task::(_debug) Task d8319417-cadb-4f47-a7c7-8a09153aeb75: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:33:11,170::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:33:11,170::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:33:11,172::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:33:11,173::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:33:11,174::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:33:11,175::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:33:11,177::task::495::TaskManager.Task::(_debug) Task d8319417-cadb-4f47-a7c7-8a09153aeb75: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:33:11,178::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:33:11,463::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0811174 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:33:13,485::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:33:14,042::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0816804 s, 12.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:33:16,058::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:33:16,557::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0733441 s, 14.0 MB/s\n'; <rc> = 0
>Thread-42838::DEBUG::2013-11-20 04:33:16,708::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 04:33:17,487::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:33:18,568::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:33:19,095::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.069582 s, 14.7 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 04:33:20,285::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:33:20,286::task::495::TaskManager.Task::(_debug) Task ff001d5c-0e0b-4594-889b-19b923853d99: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:33:20,288::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7c2f147c-c841-4151-8c29-c041682528c9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:33:20,288::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:33:20,290::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:33:20,291::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7c2f147c-c841-4151-8c29-c041682528c9`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:33:20,292::task::495::TaskManager.Task::(_debug) Task ff001d5c-0e0b-4594-889b-19b923853d99: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:33:20,293::task::495::TaskManager.Task::(_debug) Task ff001d5c-0e0b-4594-889b-19b923853d99: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:33:20,293::task::495::TaskManager.Task::(_debug) Task ff001d5c-0e0b-4594-889b-19b923853d99: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:33:20,294::task::495::TaskManager.Task::(_debug) Task ff001d5c-0e0b-4594-889b-19b923853d99: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:33:20,295::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:33:20,295::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:33:20,296::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:33:20,297::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:33:20,298::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:33:20,298::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:33:20,299::task::495::TaskManager.Task::(_debug) Task ff001d5c-0e0b-4594-889b-19b923853d99: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:33:20,300::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:33:21,113::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916645::DEBUG::2013-11-20 04:33:21,355::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916645::INFO::2013-11-20 04:33:21,357::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916645::DEBUG::2013-11-20 04:33:21,360::task::495::TaskManager.Task::(_debug) Task b514b4b9-68c6-4970-b45f-916d593400cb: moving from state init -> state preparing
>Thread-15916645::DEBUG::2013-11-20 04:33:21,362::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916645::DEBUG::2013-11-20 04:33:21,362::task::495::TaskManager.Task::(_debug) Task b514b4b9-68c6-4970-b45f-916d593400cb: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916645::DEBUG::2013-11-20 04:33:21,363::task::495::TaskManager.Task::(_debug) Task b514b4b9-68c6-4970-b45f-916d593400cb: moving from state preparing -> state finished
>Thread-15916645::DEBUG::2013-11-20 04:33:21,364::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916645::DEBUG::2013-11-20 04:33:21,365::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916645::DEBUG::2013-11-20 04:33:21,366::task::495::TaskManager.Task::(_debug) Task b514b4b9-68c6-4970-b45f-916d593400cb: ref 0 aborting False
>Thread-15916645::INFO::2013-11-20 04:33:21,367::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916646::DEBUG::2013-11-20 04:33:21,373::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916646::INFO::2013-11-20 04:33:21,374::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916646::DEBUG::2013-11-20 04:33:21,375::task::495::TaskManager.Task::(_debug) Task 62356aae-46ae-4bba-80aa-00325022ccde: moving from state init -> state preparing
>Thread-15916646::DEBUG::2013-11-20 04:33:21,377::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`27e47120-2901-4597-94ce-dec4ba22a2ad`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916646::DEBUG::2013-11-20 04:33:21,378::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916646::DEBUG::2013-11-20 04:33:21,379::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916646::DEBUG::2013-11-20 04:33:21,379::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`27e47120-2901-4597-94ce-dec4ba22a2ad`::Granted request
>Thread-15916646::DEBUG::2013-11-20 04:33:21,381::task::495::TaskManager.Task::(_debug) Task 62356aae-46ae-4bba-80aa-00325022ccde: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916646::DEBUG::2013-11-20 04:33:21,381::task::495::TaskManager.Task::(_debug) Task 62356aae-46ae-4bba-80aa-00325022ccde: ref 1 aborting False
>Thread-15916646::WARNING::2013-11-20 04:33:21,384::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916646::DEBUG::2013-11-20 04:33:21,385::task::495::TaskManager.Task::(_debug) Task 62356aae-46ae-4bba-80aa-00325022ccde: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916646::DEBUG::2013-11-20 04:33:21,386::task::495::TaskManager.Task::(_debug) Task 62356aae-46ae-4bba-80aa-00325022ccde: moving from state preparing -> state finished
>Thread-15916646::DEBUG::2013-11-20 04:33:21,387::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916646::DEBUG::2013-11-20 04:33:21,388::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916646::DEBUG::2013-11-20 04:33:21,389::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916646::DEBUG::2013-11-20 04:33:21,389::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916646::DEBUG::2013-11-20 04:33:21,390::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916646::DEBUG::2013-11-20 04:33:21,391::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916646::DEBUG::2013-11-20 04:33:21,392::task::495::TaskManager.Task::(_debug) Task 62356aae-46ae-4bba-80aa-00325022ccde: ref 0 aborting False
>Thread-15916646::INFO::2013-11-20 04:33:21,392::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914453::INFO::2013-11-20 04:33:21,493::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:33:21,495::task::495::TaskManager.Task::(_debug) Task 0b88b539-04c9-4c7a-96e8-1658419d17db: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:33:21,497::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f2c6cefd-11dd-4661-bb3c-691ddd00f29b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:33:21,498::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:33:21,499::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:33:21,499::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f2c6cefd-11dd-4661-bb3c-691ddd00f29b`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:33:21,500::task::495::TaskManager.Task::(_debug) Task 0b88b539-04c9-4c7a-96e8-1658419d17db: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:33:21,501::task::495::TaskManager.Task::(_debug) Task 0b88b539-04c9-4c7a-96e8-1658419d17db: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:33:21,502::task::495::TaskManager.Task::(_debug) Task 0b88b539-04c9-4c7a-96e8-1658419d17db: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:33:21,502::task::495::TaskManager.Task::(_debug) Task 0b88b539-04c9-4c7a-96e8-1658419d17db: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:33:21,503::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:33:21,504::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:33:21,504::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:33:21,505::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:33:21,505::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:33:21,506::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:33:21,507::task::495::TaskManager.Task::(_debug) Task 0b88b539-04c9-4c7a-96e8-1658419d17db: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:33:21,507::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:33:21,690::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.087058 s, 11.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:33:23,700::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:33:24,250::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0815999 s, 12.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:33:26,265::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:33:26,769::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.04802 s, 21.3 MB/s\n'; <rc> = 0
>Thread-15916648::INFO::2013-11-20 04:33:27,057::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916648::DEBUG::2013-11-20 04:33:27,058::task::495::TaskManager.Task::(_debug) Task 58c27b6e-9852-4f92-b934-800fbb24eddc: moving from state init -> state preparing
>Thread-15916648::DEBUG::2013-11-20 04:33:27,059::task::495::TaskManager.Task::(_debug) Task 58c27b6e-9852-4f92-b934-800fbb24eddc: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000410079956055', 'lastCheck': 1384921997.502425, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00144195556641', 'lastCheck': 1384922004.9569149, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00299406051636', 'lastCheck': 1384922001.9421871, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000494003295898', 'lastCheck': 1384922002.9572539, 'valid': True, 'code': 0}}
>Thread-15916648::DEBUG::2013-11-20 04:33:27,060::task::495::TaskManager.Task::(_debug) Task 58c27b6e-9852-4f92-b934-800fbb24eddc: moving from state preparing -> state finished
>Thread-15916648::DEBUG::2013-11-20 04:33:27,060::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916648::DEBUG::2013-11-20 04:33:27,061::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916648::DEBUG::2013-11-20 04:33:27,061::task::495::TaskManager.Task::(_debug) Task 58c27b6e-9852-4f92-b934-800fbb24eddc: ref 0 aborting False
>Thread-15916648::INFO::2013-11-20 04:33:27,062::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000410079956055', 'lastCheck': 1384921997.502425, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00144195556641', 'lastCheck': 1384922004.9569149, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00299406051636', 'lastCheck': 1384922001.9421871, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000494003295898', 'lastCheck': 1384922002.9572539, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:33:28,778::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:33:29,340::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0820773 s, 12.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:33:31,355::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 04:33:31,629::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15913546::INFO::2013-11-20 04:33:31,631::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15914519::DEBUG::2013-11-20 04:33:31,633::task::495::TaskManager.Task::(_debug) Task 164bb3d0-7719-4c7d-ad7f-d121a779e454: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:33:31,634::task::495::TaskManager.Task::(_debug) Task d82a45fa-f086-4f6b-981d-f333faaae82b: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:33:31,638::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7342ea33-9337-48b2-9360-b8bdc11f11f6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:33:31,640::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e843c6a6-47f1-476f-a7af-528675898c42`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:33:31,642::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:33:31,643::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:33:31,645::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:33:31,646::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7342ea33-9337-48b2-9360-b8bdc11f11f6`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:33:31,657::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:33:31,658::task::495::TaskManager.Task::(_debug) Task 164bb3d0-7719-4c7d-ad7f-d121a779e454: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913546::DEBUG::2013-11-20 04:33:31,659::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e843c6a6-47f1-476f-a7af-528675898c42`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:33:31,660::task::495::TaskManager.Task::(_debug) Task 164bb3d0-7719-4c7d-ad7f-d121a779e454: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:33:31,663::task::495::TaskManager.Task::(_debug) Task d82a45fa-f086-4f6b-981d-f333faaae82b: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15914519::DEBUG::2013-11-20 04:33:31,663::task::495::TaskManager.Task::(_debug) Task 164bb3d0-7719-4c7d-ad7f-d121a779e454: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:33:31,664::task::495::TaskManager.Task::(_debug) Task d82a45fa-f086-4f6b-981d-f333faaae82b: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:33:31,665::task::495::TaskManager.Task::(_debug) Task 164bb3d0-7719-4c7d-ad7f-d121a779e454: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:33:31,666::task::495::TaskManager.Task::(_debug) Task d82a45fa-f086-4f6b-981d-f333faaae82b: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:33:31,667::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:33:31,667::task::495::TaskManager.Task::(_debug) Task d82a45fa-f086-4f6b-981d-f333faaae82b: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:33:31,668::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:33:31,669::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:33:31,670::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913546::DEBUG::2013-11-20 04:33:31,670::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:33:31,672::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:33:31,673::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:33:31,674::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15914519::DEBUG::2013-11-20 04:33:31,675::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:33:31,676::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:33:31,677::task::495::TaskManager.Task::(_debug) Task 164bb3d0-7719-4c7d-ad7f-d121a779e454: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:33:31,678::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15914519::INFO::2013-11-20 04:33:31,679::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:33:31,679::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:33:31,681::task::495::TaskManager.Task::(_debug) Task d82a45fa-f086-4f6b-981d-f333faaae82b: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:33:31,681::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:33:31,682::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:33:31,682::task::495::TaskManager.Task::(_debug) Task 6f7a806b-05ac-45d9-8ac6-020b720dccaa: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:33:31,683::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8ae6dac1-cced-4efe-a56c-d63581e30b50`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:33:31,683::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:33:31,684::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:33:31,684::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8ae6dac1-cced-4efe-a56c-d63581e30b50`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:33:31,684::task::495::TaskManager.Task::(_debug) Task 6f7a806b-05ac-45d9-8ac6-020b720dccaa: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:33:31,685::task::495::TaskManager.Task::(_debug) Task 6f7a806b-05ac-45d9-8ac6-020b720dccaa: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:33:31,685::task::495::TaskManager.Task::(_debug) Task 6f7a806b-05ac-45d9-8ac6-020b720dccaa: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:33:31,686::task::495::TaskManager.Task::(_debug) Task 6f7a806b-05ac-45d9-8ac6-020b720dccaa: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:33:31,686::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:33:31,687::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:33:31,687::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:33:31,688::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:33:31,688::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:33:31,689::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:33:31,690::task::495::TaskManager.Task::(_debug) Task 6f7a806b-05ac-45d9-8ac6-020b720dccaa: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:33:31,690::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:33:31,691::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:33:31,692::task::495::TaskManager.Task::(_debug) Task fbd938cc-d231-4b6d-b82b-ab7401c222e7: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:33:31,693::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`ba98ea38-4683-41d3-b855-98cc6edcc698`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:33:31,693::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:33:31,694::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:33:31,695::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`ba98ea38-4683-41d3-b855-98cc6edcc698`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:33:31,695::task::495::TaskManager.Task::(_debug) Task fbd938cc-d231-4b6d-b82b-ab7401c222e7: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:33:31,696::task::495::TaskManager.Task::(_debug) Task fbd938cc-d231-4b6d-b82b-ab7401c222e7: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:33:31,697::task::495::TaskManager.Task::(_debug) Task fbd938cc-d231-4b6d-b82b-ab7401c222e7: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:33:31,698::task::495::TaskManager.Task::(_debug) Task fbd938cc-d231-4b6d-b82b-ab7401c222e7: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:33:31,698::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:33:31,699::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:33:31,699::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:33:31,700::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:33:31,700::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:33:31,701::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:33:31,702::task::495::TaskManager.Task::(_debug) Task fbd938cc-d231-4b6d-b82b-ab7401c222e7: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:33:31,703::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:33:31,705::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:33:31,705::task::495::TaskManager.Task::(_debug) Task 50fefaa4-bd04-4102-9692-5a04c967da68: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:33:31,707::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`cf8f234e-1e32-4dc8-bcbd-ea8a4813d97b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:33:31,707::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:33:31,708::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:33:31,709::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`cf8f234e-1e32-4dc8-bcbd-ea8a4813d97b`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:33:31,710::task::495::TaskManager.Task::(_debug) Task 50fefaa4-bd04-4102-9692-5a04c967da68: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:33:31,711::task::495::TaskManager.Task::(_debug) Task 50fefaa4-bd04-4102-9692-5a04c967da68: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:33:31,712::task::495::TaskManager.Task::(_debug) Task 50fefaa4-bd04-4102-9692-5a04c967da68: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:33:31,712::task::495::TaskManager.Task::(_debug) Task 50fefaa4-bd04-4102-9692-5a04c967da68: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:33:31,713::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:33:31,713::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:33:31,714::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:33:31,714::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:33:31,715::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:33:31,715::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:33:31,716::task::495::TaskManager.Task::(_debug) Task 50fefaa4-bd04-4102-9692-5a04c967da68: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:33:31,717::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:33:31,937::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0804292 s, 12.7 MB/s\n'; <rc> = 0
>Thread-15916651::DEBUG::2013-11-20 04:33:32,093::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916651::INFO::2013-11-20 04:33:32,094::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916651::DEBUG::2013-11-20 04:33:32,095::task::495::TaskManager.Task::(_debug) Task 778d19ad-7c9d-4ac9-84c4-a873da59574f: moving from state init -> state preparing
>Thread-15916651::DEBUG::2013-11-20 04:33:32,096::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916651::DEBUG::2013-11-20 04:33:32,096::task::495::TaskManager.Task::(_debug) Task 778d19ad-7c9d-4ac9-84c4-a873da59574f: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916651::DEBUG::2013-11-20 04:33:32,097::task::495::TaskManager.Task::(_debug) Task 778d19ad-7c9d-4ac9-84c4-a873da59574f: moving from state preparing -> state finished
>Thread-15916651::DEBUG::2013-11-20 04:33:32,098::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916651::DEBUG::2013-11-20 04:33:32,098::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916651::DEBUG::2013-11-20 04:33:32,099::task::495::TaskManager.Task::(_debug) Task 778d19ad-7c9d-4ac9-84c4-a873da59574f: ref 0 aborting False
>Thread-15916651::INFO::2013-11-20 04:33:32,099::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916652::DEBUG::2013-11-20 04:33:32,104::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916652::INFO::2013-11-20 04:33:32,105::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916652::DEBUG::2013-11-20 04:33:32,105::task::495::TaskManager.Task::(_debug) Task 0cded004-4b0b-4e37-b22d-008823c167ab: moving from state init -> state preparing
>Thread-15916652::DEBUG::2013-11-20 04:33:32,106::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e3b6f8d8-d90e-4635-8d8e-a40fff65e706`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916652::DEBUG::2013-11-20 04:33:32,107::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916652::DEBUG::2013-11-20 04:33:32,108::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916652::DEBUG::2013-11-20 04:33:32,108::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e3b6f8d8-d90e-4635-8d8e-a40fff65e706`::Granted request
>Thread-15916652::DEBUG::2013-11-20 04:33:32,109::task::495::TaskManager.Task::(_debug) Task 0cded004-4b0b-4e37-b22d-008823c167ab: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916652::DEBUG::2013-11-20 04:33:32,110::task::495::TaskManager.Task::(_debug) Task 0cded004-4b0b-4e37-b22d-008823c167ab: ref 1 aborting False
>Thread-15916652::WARNING::2013-11-20 04:33:32,112::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916652::DEBUG::2013-11-20 04:33:32,113::task::495::TaskManager.Task::(_debug) Task 0cded004-4b0b-4e37-b22d-008823c167ab: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916652::DEBUG::2013-11-20 04:33:32,114::task::495::TaskManager.Task::(_debug) Task 0cded004-4b0b-4e37-b22d-008823c167ab: moving from state preparing -> state finished
>Thread-15916652::DEBUG::2013-11-20 04:33:32,114::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916652::DEBUG::2013-11-20 04:33:32,115::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916652::DEBUG::2013-11-20 04:33:32,115::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916652::DEBUG::2013-11-20 04:33:32,116::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916652::DEBUG::2013-11-20 04:33:32,117::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916652::DEBUG::2013-11-20 04:33:32,117::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916652::DEBUG::2013-11-20 04:33:32,118::task::495::TaskManager.Task::(_debug) Task 0cded004-4b0b-4e37-b22d-008823c167ab: ref 0 aborting False
>Thread-15916652::INFO::2013-11-20 04:33:32,118::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:33:33,947::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:33:34,474::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0755164 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:33:36,490::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 04:33:36,724::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:33:36,725::task::495::TaskManager.Task::(_debug) Task 78e1c330-16ad-4a8e-948f-2eadff2524af: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:33:36,729::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0a319dc5-5907-4e85-8510-fcb3ba92f7be`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:33:36,730::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:33:36,732::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:33:36,732::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0a319dc5-5907-4e85-8510-fcb3ba92f7be`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:33:36,734::task::495::TaskManager.Task::(_debug) Task 78e1c330-16ad-4a8e-948f-2eadff2524af: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:33:36,734::task::495::TaskManager.Task::(_debug) Task 78e1c330-16ad-4a8e-948f-2eadff2524af: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:33:36,736::task::495::TaskManager.Task::(_debug) Task 78e1c330-16ad-4a8e-948f-2eadff2524af: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:33:36,736::task::495::TaskManager.Task::(_debug) Task 78e1c330-16ad-4a8e-948f-2eadff2524af: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:33:36,737::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:33:36,738::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:33:36,739::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:33:36,739::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:33:36,740::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:33:36,741::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:33:36,741::task::495::TaskManager.Task::(_debug) Task 78e1c330-16ad-4a8e-948f-2eadff2524af: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:33:36,742::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:33:37,064::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0808589 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:33:39,079::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:33:39,621::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0827362 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:33:41,637::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:33:42,178::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0847629 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15916656::DEBUG::2013-11-20 04:33:42,899::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916656::INFO::2013-11-20 04:33:42,900::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916656::DEBUG::2013-11-20 04:33:42,901::task::495::TaskManager.Task::(_debug) Task 4aa4d6e0-2d80-4c8b-816b-3facfb80887f: moving from state init -> state preparing
>Thread-15916656::DEBUG::2013-11-20 04:33:42,902::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916656::DEBUG::2013-11-20 04:33:42,903::task::495::TaskManager.Task::(_debug) Task 4aa4d6e0-2d80-4c8b-816b-3facfb80887f: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916656::DEBUG::2013-11-20 04:33:42,904::task::495::TaskManager.Task::(_debug) Task 4aa4d6e0-2d80-4c8b-816b-3facfb80887f: moving from state preparing -> state finished
>Thread-15916656::DEBUG::2013-11-20 04:33:42,904::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916656::DEBUG::2013-11-20 04:33:42,905::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916656::DEBUG::2013-11-20 04:33:42,906::task::495::TaskManager.Task::(_debug) Task 4aa4d6e0-2d80-4c8b-816b-3facfb80887f: ref 0 aborting False
>Thread-15916656::INFO::2013-11-20 04:33:42,907::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916657::DEBUG::2013-11-20 04:33:42,911::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916657::INFO::2013-11-20 04:33:42,912::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916657::DEBUG::2013-11-20 04:33:42,913::task::495::TaskManager.Task::(_debug) Task b147db96-e54f-439e-8a1b-b86fe2ffbb30: moving from state init -> state preparing
>Thread-15916657::DEBUG::2013-11-20 04:33:42,924::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f4224016-e10c-4788-a9e2-c108e23fa5e5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916657::DEBUG::2013-11-20 04:33:42,925::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916657::DEBUG::2013-11-20 04:33:42,926::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916657::DEBUG::2013-11-20 04:33:42,926::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f4224016-e10c-4788-a9e2-c108e23fa5e5`::Granted request
>Thread-15916657::DEBUG::2013-11-20 04:33:42,927::task::495::TaskManager.Task::(_debug) Task b147db96-e54f-439e-8a1b-b86fe2ffbb30: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916657::DEBUG::2013-11-20 04:33:42,928::task::495::TaskManager.Task::(_debug) Task b147db96-e54f-439e-8a1b-b86fe2ffbb30: ref 1 aborting False
>Thread-15916657::WARNING::2013-11-20 04:33:42,930::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916657::DEBUG::2013-11-20 04:33:42,931::task::495::TaskManager.Task::(_debug) Task b147db96-e54f-439e-8a1b-b86fe2ffbb30: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916657::DEBUG::2013-11-20 04:33:42,932::task::495::TaskManager.Task::(_debug) Task b147db96-e54f-439e-8a1b-b86fe2ffbb30: moving from state preparing -> state finished
>Thread-15916657::DEBUG::2013-11-20 04:33:42,932::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916657::DEBUG::2013-11-20 04:33:42,933::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916657::DEBUG::2013-11-20 04:33:42,934::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916657::DEBUG::2013-11-20 04:33:42,934::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916657::DEBUG::2013-11-20 04:33:42,935::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916657::DEBUG::2013-11-20 04:33:42,935::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916657::DEBUG::2013-11-20 04:33:42,936::task::495::TaskManager.Task::(_debug) Task b147db96-e54f-439e-8a1b-b86fe2ffbb30: ref 0 aborting False
>Thread-15916657::INFO::2013-11-20 04:33:42,937::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916658::INFO::2013-11-20 04:33:43,680::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916658::DEBUG::2013-11-20 04:33:43,681::task::495::TaskManager.Task::(_debug) Task 91fa54d3-db6a-487a-85f3-cfa457f52761: moving from state init -> state preparing
>Thread-15916658::DEBUG::2013-11-20 04:33:43,682::task::495::TaskManager.Task::(_debug) Task 91fa54d3-db6a-487a-85f3-cfa457f52761: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000463962554932', 'lastCheck': 1384922017.7064011, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00195002555847', 'lastCheck': 1384922015.173907, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00147891044617', 'lastCheck': 1384922022.30901, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000559091567993', 'lastCheck': 1384922023.3079171, 'valid': True, 'code': 0}}
>Thread-15916658::DEBUG::2013-11-20 04:33:43,683::task::495::TaskManager.Task::(_debug) Task 91fa54d3-db6a-487a-85f3-cfa457f52761: moving from state preparing -> state finished
>Thread-15916658::DEBUG::2013-11-20 04:33:43,684::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916658::DEBUG::2013-11-20 04:33:43,684::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916658::DEBUG::2013-11-20 04:33:43,685::task::495::TaskManager.Task::(_debug) Task 91fa54d3-db6a-487a-85f3-cfa457f52761: ref 0 aborting False
>Thread-15916658::INFO::2013-11-20 04:33:43,686::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000463962554932', 'lastCheck': 1384922017.7064011, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00195002555847', 'lastCheck': 1384922015.173907, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00147891044617', 'lastCheck': 1384922022.30901, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000559091567993', 'lastCheck': 1384922023.3079171, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:33:44,193::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:33:44,730::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0829055 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:33:46,745::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:33:47,298::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0807484 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:33:49,314::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:33:49,512::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:33:49,514::task::495::TaskManager.Task::(_debug) Task f790c36e-39b1-4462-abba-ef2d4e550c8e: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:33:49,517::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a8838d30-99d3-4a64-a57b-c2f86fa23168`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:33:49,518::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:33:49,519::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:33:49,520::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a8838d30-99d3-4a64-a57b-c2f86fa23168`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:33:49,521::task::495::TaskManager.Task::(_debug) Task f790c36e-39b1-4462-abba-ef2d4e550c8e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:33:49,522::task::495::TaskManager.Task::(_debug) Task f790c36e-39b1-4462-abba-ef2d4e550c8e: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:33:49,523::task::495::TaskManager.Task::(_debug) Task f790c36e-39b1-4462-abba-ef2d4e550c8e: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:33:49,524::task::495::TaskManager.Task::(_debug) Task f790c36e-39b1-4462-abba-ef2d4e550c8e: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:33:49,525::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:33:49,526::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:33:49,527::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:33:49,528::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:33:49,529::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:33:49,529::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:33:49,530::task::495::TaskManager.Task::(_debug) Task f790c36e-39b1-4462-abba-ef2d4e550c8e: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:33:49,531::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:33:49,846::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0761422 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:33:51,862::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915918::INFO::2013-11-20 04:33:52,087::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:33:52,092::task::495::TaskManager.Task::(_debug) Task 7fb14eb9-0673-4a80-b2da-d3256b286fe5: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:33:52,098::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f8867df6-cab6-4e57-b83f-1f7ce9d34bc5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:33:52,099::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:33:52,103::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:33:52,104::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f8867df6-cab6-4e57-b83f-1f7ce9d34bc5`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:33:52,107::task::495::TaskManager.Task::(_debug) Task 7fb14eb9-0673-4a80-b2da-d3256b286fe5: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:33:52,108::task::495::TaskManager.Task::(_debug) Task 7fb14eb9-0673-4a80-b2da-d3256b286fe5: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:33:52,109::task::495::TaskManager.Task::(_debug) Task 7fb14eb9-0673-4a80-b2da-d3256b286fe5: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:33:52,110::task::495::TaskManager.Task::(_debug) Task 7fb14eb9-0673-4a80-b2da-d3256b286fe5: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:33:52,110::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:33:52,111::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:33:52,112::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:33:52,112::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:33:52,113::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:33:52,113::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:33:52,114::task::495::TaskManager.Task::(_debug) Task 7fb14eb9-0673-4a80-b2da-d3256b286fe5: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:33:52,115::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:33:52,407::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0776383 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15916663::DEBUG::2013-11-20 04:33:54,290::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916663::INFO::2013-11-20 04:33:54,291::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916663::DEBUG::2013-11-20 04:33:54,292::task::495::TaskManager.Task::(_debug) Task 511dff74-ec1d-4bfe-8bf4-923a66ac1733: moving from state init -> state preparing
>Thread-15916663::DEBUG::2013-11-20 04:33:54,293::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916663::DEBUG::2013-11-20 04:33:54,293::task::495::TaskManager.Task::(_debug) Task 511dff74-ec1d-4bfe-8bf4-923a66ac1733: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916663::DEBUG::2013-11-20 04:33:54,294::task::495::TaskManager.Task::(_debug) Task 511dff74-ec1d-4bfe-8bf4-923a66ac1733: moving from state preparing -> state finished
>Thread-15916663::DEBUG::2013-11-20 04:33:54,295::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916663::DEBUG::2013-11-20 04:33:54,295::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916663::DEBUG::2013-11-20 04:33:54,296::task::495::TaskManager.Task::(_debug) Task 511dff74-ec1d-4bfe-8bf4-923a66ac1733: ref 0 aborting False
>Thread-15916663::INFO::2013-11-20 04:33:54,297::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916664::DEBUG::2013-11-20 04:33:54,301::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916664::INFO::2013-11-20 04:33:54,302::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916664::DEBUG::2013-11-20 04:33:54,303::task::495::TaskManager.Task::(_debug) Task 3c566b8f-2ef7-4d0f-8c04-1eb6da3acf0d: moving from state init -> state preparing
>Thread-15916664::DEBUG::2013-11-20 04:33:54,304::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ec1f5ea0-5280-4539-8a34-4040b0030122`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916664::DEBUG::2013-11-20 04:33:54,304::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916664::DEBUG::2013-11-20 04:33:54,305::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916664::DEBUG::2013-11-20 04:33:54,306::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ec1f5ea0-5280-4539-8a34-4040b0030122`::Granted request
>Thread-15916664::DEBUG::2013-11-20 04:33:54,307::task::495::TaskManager.Task::(_debug) Task 3c566b8f-2ef7-4d0f-8c04-1eb6da3acf0d: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916664::DEBUG::2013-11-20 04:33:54,307::task::495::TaskManager.Task::(_debug) Task 3c566b8f-2ef7-4d0f-8c04-1eb6da3acf0d: ref 1 aborting False
>Thread-15916664::WARNING::2013-11-20 04:33:54,310::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916664::DEBUG::2013-11-20 04:33:54,311::task::495::TaskManager.Task::(_debug) Task 3c566b8f-2ef7-4d0f-8c04-1eb6da3acf0d: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916664::DEBUG::2013-11-20 04:33:54,312::task::495::TaskManager.Task::(_debug) Task 3c566b8f-2ef7-4d0f-8c04-1eb6da3acf0d: moving from state preparing -> state finished
>Thread-15916664::DEBUG::2013-11-20 04:33:54,312::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916664::DEBUG::2013-11-20 04:33:54,313::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916664::DEBUG::2013-11-20 04:33:54,314::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916664::DEBUG::2013-11-20 04:33:54,315::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916664::DEBUG::2013-11-20 04:33:54,315::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916664::DEBUG::2013-11-20 04:33:54,316::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916664::DEBUG::2013-11-20 04:33:54,316::task::495::TaskManager.Task::(_debug) Task 3c566b8f-2ef7-4d0f-8c04-1eb6da3acf0d: ref 0 aborting False
>Thread-15916664::INFO::2013-11-20 04:33:54,317::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:33:54,422::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:33:54,934::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0707643 s, 14.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:33:56,949::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915863::INFO::2013-11-20 04:33:57,177::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:33:57,180::task::495::TaskManager.Task::(_debug) Task 8c31418d-9922-4ebb-b2d9-dd358f55a505: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:33:57,183::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`477a24c5-038b-4c89-9027-ae46b700d160`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:33:57,184::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:33:57,186::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:33:57,187::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`477a24c5-038b-4c89-9027-ae46b700d160`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:33:57,190::task::495::TaskManager.Task::(_debug) Task 8c31418d-9922-4ebb-b2d9-dd358f55a505: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:33:57,191::task::495::TaskManager.Task::(_debug) Task 8c31418d-9922-4ebb-b2d9-dd358f55a505: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:33:57,192::task::495::TaskManager.Task::(_debug) Task 8c31418d-9922-4ebb-b2d9-dd358f55a505: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:33:57,193::task::495::TaskManager.Task::(_debug) Task 8c31418d-9922-4ebb-b2d9-dd358f55a505: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:33:57,194::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:33:57,194::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:33:57,195::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:33:57,196::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:33:57,196::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:33:57,197::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:33:57,198::task::495::TaskManager.Task::(_debug) Task 8c31418d-9922-4ebb-b2d9-dd358f55a505: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:33:57,199::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:33:57,484::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0799572 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:33:59,499::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916666::INFO::2013-11-20 04:33:59,727::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916666::DEBUG::2013-11-20 04:33:59,730::task::495::TaskManager.Task::(_debug) Task 21575d87-a4bb-4b8f-9730-148838a00a08: moving from state init -> state preparing
>Thread-15916666::DEBUG::2013-11-20 04:33:59,731::task::495::TaskManager.Task::(_debug) Task 21575d87-a4bb-4b8f-9730-148838a00a08: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000391960144043', 'lastCheck': 1384922038.1721699, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0806660652161', 'lastCheck': 1384922035.6982231, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0120949745178', 'lastCheck': 1384922032.526273, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000900983810425', 'lastCheck': 1384922033.5135739, 'valid': True, 'code': 0}}
>Thread-15916666::DEBUG::2013-11-20 04:33:59,733::task::495::TaskManager.Task::(_debug) Task 21575d87-a4bb-4b8f-9730-148838a00a08: moving from state preparing -> state finished
>Thread-15916666::DEBUG::2013-11-20 04:33:59,734::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916666::DEBUG::2013-11-20 04:33:59,735::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916666::DEBUG::2013-11-20 04:33:59,735::task::495::TaskManager.Task::(_debug) Task 21575d87-a4bb-4b8f-9730-148838a00a08: ref 0 aborting False
>Thread-15916666::INFO::2013-11-20 04:33:59,736::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000391960144043', 'lastCheck': 1384922038.1721699, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0806660652161', 'lastCheck': 1384922035.6982231, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0120949745178', 'lastCheck': 1384922032.526273, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000900983810425', 'lastCheck': 1384922033.5135739, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:34:00,006::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0577694 s, 17.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:34:02,017::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 04:34:02,245::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:34:02,248::task::495::TaskManager.Task::(_debug) Task dfb12ce9-b6ba-4b79-af75-7ba90b482023: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:34:02,252::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`013b2b37-01a6-4574-b8c7-296bdad8d684`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:34:02,253::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:34:02,254::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:34:02,255::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`013b2b37-01a6-4574-b8c7-296bdad8d684`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:34:02,258::task::495::TaskManager.Task::(_debug) Task dfb12ce9-b6ba-4b79-af75-7ba90b482023: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:34:02,259::task::495::TaskManager.Task::(_debug) Task dfb12ce9-b6ba-4b79-af75-7ba90b482023: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:34:02,260::task::495::TaskManager.Task::(_debug) Task dfb12ce9-b6ba-4b79-af75-7ba90b482023: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:34:02,260::task::495::TaskManager.Task::(_debug) Task dfb12ce9-b6ba-4b79-af75-7ba90b482023: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:34:02,261::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:34:02,262::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:34:02,263::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:34:02,264::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:34:02,264::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:34:02,265::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:34:02,266::task::495::TaskManager.Task::(_debug) Task dfb12ce9-b6ba-4b79-af75-7ba90b482023: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:34:02,267::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:34:02,566::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0892076 s, 11.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:34:04,579::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916669::DEBUG::2013-11-20 04:34:04,903::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916669::INFO::2013-11-20 04:34:04,906::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916669::DEBUG::2013-11-20 04:34:04,906::task::495::TaskManager.Task::(_debug) Task 3d3a80f6-044e-4978-ae30-bb3f5bf5b4bf: moving from state init -> state preparing
>Thread-15916669::DEBUG::2013-11-20 04:34:04,908::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916669::DEBUG::2013-11-20 04:34:04,908::task::495::TaskManager.Task::(_debug) Task 3d3a80f6-044e-4978-ae30-bb3f5bf5b4bf: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916669::DEBUG::2013-11-20 04:34:04,909::task::495::TaskManager.Task::(_debug) Task 3d3a80f6-044e-4978-ae30-bb3f5bf5b4bf: moving from state preparing -> state finished
>Thread-15916669::DEBUG::2013-11-20 04:34:04,910::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916669::DEBUG::2013-11-20 04:34:04,910::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916669::DEBUG::2013-11-20 04:34:04,911::task::495::TaskManager.Task::(_debug) Task 3d3a80f6-044e-4978-ae30-bb3f5bf5b4bf: ref 0 aborting False
>Thread-15916669::INFO::2013-11-20 04:34:04,911::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916670::DEBUG::2013-11-20 04:34:04,917::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916670::INFO::2013-11-20 04:34:04,918::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916670::DEBUG::2013-11-20 04:34:04,919::task::495::TaskManager.Task::(_debug) Task 6249b2e5-8603-4425-a924-7f17d990e976: moving from state init -> state preparing
>Thread-15916670::DEBUG::2013-11-20 04:34:04,921::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5ed78374-f54e-49ad-ad7a-8c17784ecdfa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916670::DEBUG::2013-11-20 04:34:04,921::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916670::DEBUG::2013-11-20 04:34:04,923::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916670::DEBUG::2013-11-20 04:34:04,923::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`5ed78374-f54e-49ad-ad7a-8c17784ecdfa`::Granted request
>Thread-15916670::DEBUG::2013-11-20 04:34:04,925::task::495::TaskManager.Task::(_debug) Task 6249b2e5-8603-4425-a924-7f17d990e976: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916670::DEBUG::2013-11-20 04:34:04,925::task::495::TaskManager.Task::(_debug) Task 6249b2e5-8603-4425-a924-7f17d990e976: ref 1 aborting False
>Thread-15916670::WARNING::2013-11-20 04:34:04,928::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916670::DEBUG::2013-11-20 04:34:04,929::task::495::TaskManager.Task::(_debug) Task 6249b2e5-8603-4425-a924-7f17d990e976: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916670::DEBUG::2013-11-20 04:34:04,930::task::495::TaskManager.Task::(_debug) Task 6249b2e5-8603-4425-a924-7f17d990e976: moving from state preparing -> state finished
>Thread-15916670::DEBUG::2013-11-20 04:34:04,931::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916670::DEBUG::2013-11-20 04:34:04,931::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916670::DEBUG::2013-11-20 04:34:04,932::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916670::DEBUG::2013-11-20 04:34:04,933::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916670::DEBUG::2013-11-20 04:34:04,934::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916670::DEBUG::2013-11-20 04:34:04,935::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916670::DEBUG::2013-11-20 04:34:04,936::task::495::TaskManager.Task::(_debug) Task 6249b2e5-8603-4425-a924-7f17d990e976: ref 0 aborting False
>Thread-15916670::INFO::2013-11-20 04:34:04,936::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:34:05,137::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0946952 s, 10.8 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 04:34:05,812::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:34:05,813::task::495::TaskManager.Task::(_debug) Task 5dd6d9f0-d523-4e46-baf4-cd63feda3309: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:34:05,814::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8bfc0b3d-5553-4c36-bbc6-bbd4795d81f3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:34:05,815::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:34:05,816::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:34:05,816::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8bfc0b3d-5553-4c36-bbc6-bbd4795d81f3`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:34:05,817::task::495::TaskManager.Task::(_debug) Task 5dd6d9f0-d523-4e46-baf4-cd63feda3309: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:34:05,818::task::495::TaskManager.Task::(_debug) Task 5dd6d9f0-d523-4e46-baf4-cd63feda3309: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:34:05,819::task::495::TaskManager.Task::(_debug) Task 5dd6d9f0-d523-4e46-baf4-cd63feda3309: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:34:05,819::task::495::TaskManager.Task::(_debug) Task 5dd6d9f0-d523-4e46-baf4-cd63feda3309: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:34:05,820::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:34:05,820::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:34:05,821::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:34:05,822::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:34:05,822::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:34:05,823::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:34:05,823::task::495::TaskManager.Task::(_debug) Task 5dd6d9f0-d523-4e46-baf4-cd63feda3309: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:34:05,824::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:34:05,825::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:34:05,825::task::495::TaskManager.Task::(_debug) Task 1c2901ca-ca15-48f3-b6d8-c74911c750f9: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:34:05,826::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0ccf2a72-7386-4ce2-b829-826dfa7a1087`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:34:05,827::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:34:05,828::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:34:05,828::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0ccf2a72-7386-4ce2-b829-826dfa7a1087`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:34:05,829::task::495::TaskManager.Task::(_debug) Task 1c2901ca-ca15-48f3-b6d8-c74911c750f9: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:34:05,830::task::495::TaskManager.Task::(_debug) Task 1c2901ca-ca15-48f3-b6d8-c74911c750f9: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:34:05,830::task::495::TaskManager.Task::(_debug) Task 1c2901ca-ca15-48f3-b6d8-c74911c750f9: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:34:05,831::task::495::TaskManager.Task::(_debug) Task 1c2901ca-ca15-48f3-b6d8-c74911c750f9: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:34:05,831::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:34:05,832::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:34:05,833::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:34:05,833::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:34:05,834::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:34:05,834::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:34:05,835::task::495::TaskManager.Task::(_debug) Task 1c2901ca-ca15-48f3-b6d8-c74911c750f9: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:34:05,835::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:34:07,152::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:34:07,699::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0733432 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:34:09,715::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:34:10,285::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0835176 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:34:12,304::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 04:34:12,516::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:34:12,519::task::495::TaskManager.Task::(_debug) Task 86da98cf-7ebe-44ed-af43-4c582416312d: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:34:12,523::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`149bb466-4bc3-4240-88a0-daffc1da04e6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:34:12,526::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:34:12,529::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:34:12,530::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`149bb466-4bc3-4240-88a0-daffc1da04e6`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:34:12,532::task::495::TaskManager.Task::(_debug) Task 86da98cf-7ebe-44ed-af43-4c582416312d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:34:12,533::task::495::TaskManager.Task::(_debug) Task 86da98cf-7ebe-44ed-af43-4c582416312d: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:34:12,534::task::495::TaskManager.Task::(_debug) Task 86da98cf-7ebe-44ed-af43-4c582416312d: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:34:12,534::task::495::TaskManager.Task::(_debug) Task 86da98cf-7ebe-44ed-af43-4c582416312d: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:34:12,535::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:34:12,536::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:34:12,537::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:34:12,538::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:34:12,539::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:34:12,539::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:34:12,540::task::495::TaskManager.Task::(_debug) Task 86da98cf-7ebe-44ed-af43-4c582416312d: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:34:12,541::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:34:12,833::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0793879 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:34:14,849::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916674::DEBUG::2013-11-20 04:34:15,225::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916674::INFO::2013-11-20 04:34:15,227::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916674::DEBUG::2013-11-20 04:34:15,228::task::495::TaskManager.Task::(_debug) Task 048a7aa9-8211-434d-b5ab-e98699341418: moving from state init -> state preparing
>Thread-15916674::DEBUG::2013-11-20 04:34:15,230::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916674::DEBUG::2013-11-20 04:34:15,230::task::495::TaskManager.Task::(_debug) Task 048a7aa9-8211-434d-b5ab-e98699341418: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916674::DEBUG::2013-11-20 04:34:15,232::task::495::TaskManager.Task::(_debug) Task 048a7aa9-8211-434d-b5ab-e98699341418: moving from state preparing -> state finished
>Thread-15916674::DEBUG::2013-11-20 04:34:15,233::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916674::DEBUG::2013-11-20 04:34:15,233::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916674::DEBUG::2013-11-20 04:34:15,234::task::495::TaskManager.Task::(_debug) Task 048a7aa9-8211-434d-b5ab-e98699341418: ref 0 aborting False
>Thread-15916674::INFO::2013-11-20 04:34:15,235::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916675::DEBUG::2013-11-20 04:34:15,241::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916675::INFO::2013-11-20 04:34:15,242::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916675::DEBUG::2013-11-20 04:34:15,242::task::495::TaskManager.Task::(_debug) Task 288fd7b6-213a-44a4-bb82-bbc5f7403d7f: moving from state init -> state preparing
>Thread-15916675::DEBUG::2013-11-20 04:34:15,244::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`44318dbe-f30e-4b45-af76-173b52baf2c2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916675::DEBUG::2013-11-20 04:34:15,244::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916675::DEBUG::2013-11-20 04:34:15,245::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916675::DEBUG::2013-11-20 04:34:15,246::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`44318dbe-f30e-4b45-af76-173b52baf2c2`::Granted request
>Thread-15916675::DEBUG::2013-11-20 04:34:15,247::task::495::TaskManager.Task::(_debug) Task 288fd7b6-213a-44a4-bb82-bbc5f7403d7f: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916675::DEBUG::2013-11-20 04:34:15,248::task::495::TaskManager.Task::(_debug) Task 288fd7b6-213a-44a4-bb82-bbc5f7403d7f: ref 1 aborting False
>Thread-15916675::WARNING::2013-11-20 04:34:15,250::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916675::DEBUG::2013-11-20 04:34:15,251::task::495::TaskManager.Task::(_debug) Task 288fd7b6-213a-44a4-bb82-bbc5f7403d7f: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916675::DEBUG::2013-11-20 04:34:15,252::task::495::TaskManager.Task::(_debug) Task 288fd7b6-213a-44a4-bb82-bbc5f7403d7f: moving from state preparing -> state finished
>Thread-15916675::DEBUG::2013-11-20 04:34:15,253::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916675::DEBUG::2013-11-20 04:34:15,253::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916675::DEBUG::2013-11-20 04:34:15,254::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916675::DEBUG::2013-11-20 04:34:15,255::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916675::DEBUG::2013-11-20 04:34:15,255::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916675::DEBUG::2013-11-20 04:34:15,256::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916675::DEBUG::2013-11-20 04:34:15,257::task::495::TaskManager.Task::(_debug) Task 288fd7b6-213a-44a4-bb82-bbc5f7403d7f: ref 0 aborting False
>Thread-15916675::INFO::2013-11-20 04:34:15,257::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:34:15,404::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0818132 s, 12.5 MB/s\n'; <rc> = 0
>Thread-15916676::INFO::2013-11-20 04:34:16,095::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916676::DEBUG::2013-11-20 04:34:16,096::task::495::TaskManager.Task::(_debug) Task 4ba160d9-7a7c-4e97-a74c-05259d372d30: moving from state init -> state preparing
>Thread-15916676::DEBUG::2013-11-20 04:34:16,097::task::495::TaskManager.Task::(_debug) Task 4ba160d9-7a7c-4e97-a74c-05259d372d30: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000417947769165', 'lastCheck': 1384922048.38867, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0016770362854', 'lastCheck': 1384922056.065151, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00143885612488', 'lastCheck': 1384922052.9630899, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000452995300293', 'lastCheck': 1384922053.9515009, 'valid': True, 'code': 0}}
>Thread-15916676::DEBUG::2013-11-20 04:34:16,098::task::495::TaskManager.Task::(_debug) Task 4ba160d9-7a7c-4e97-a74c-05259d372d30: moving from state preparing -> state finished
>Thread-15916676::DEBUG::2013-11-20 04:34:16,098::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916676::DEBUG::2013-11-20 04:34:16,099::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916676::DEBUG::2013-11-20 04:34:16,099::task::495::TaskManager.Task::(_debug) Task 4ba160d9-7a7c-4e97-a74c-05259d372d30: ref 0 aborting False
>Thread-15916676::INFO::2013-11-20 04:34:16,100::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000417947769165', 'lastCheck': 1384922048.38867, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0016770362854', 'lastCheck': 1384922056.065151, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00143885612488', 'lastCheck': 1384922052.9630899, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000452995300293', 'lastCheck': 1384922053.9515009, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:34:17,418::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:34:17,965::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0775181 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:34:19,982::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:34:20,575::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0812715 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 04:34:21,664::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:34:21,665::task::495::TaskManager.Task::(_debug) Task cf41c463-bfc7-4d6b-89aa-490ca26b18d2: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:34:21,667::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3f5f15fc-be52-4f86-aa0c-7e57d7bc6fbc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:34:21,667::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:34:21,668::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:34:21,669::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3f5f15fc-be52-4f86-aa0c-7e57d7bc6fbc`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:34:21,670::task::495::TaskManager.Task::(_debug) Task cf41c463-bfc7-4d6b-89aa-490ca26b18d2: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:34:21,671::task::495::TaskManager.Task::(_debug) Task cf41c463-bfc7-4d6b-89aa-490ca26b18d2: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:34:21,672::task::495::TaskManager.Task::(_debug) Task cf41c463-bfc7-4d6b-89aa-490ca26b18d2: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:34:21,673::task::495::TaskManager.Task::(_debug) Task cf41c463-bfc7-4d6b-89aa-490ca26b18d2: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:34:21,674::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:34:21,674::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:34:21,675::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:34:21,676::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:34:21,676::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:34:21,677::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:34:21,677::task::495::TaskManager.Task::(_debug) Task cf41c463-bfc7-4d6b-89aa-490ca26b18d2: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:34:21,678::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:34:22,599::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 04:34:22,833::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:34:22,857::task::495::TaskManager.Task::(_debug) Task 66025296-52bf-4fff-b0a7-419ccc1ecb67: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:34:22,867::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b539ceea-2d76-4cd6-9444-7bd13f41948c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:34:22,869::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:34:22,873::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:34:22,875::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b539ceea-2d76-4cd6-9444-7bd13f41948c`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:34:22,877::task::495::TaskManager.Task::(_debug) Task 66025296-52bf-4fff-b0a7-419ccc1ecb67: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:34:22,879::task::495::TaskManager.Task::(_debug) Task 66025296-52bf-4fff-b0a7-419ccc1ecb67: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:34:22,880::task::495::TaskManager.Task::(_debug) Task 66025296-52bf-4fff-b0a7-419ccc1ecb67: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:34:22,881::task::495::TaskManager.Task::(_debug) Task 66025296-52bf-4fff-b0a7-419ccc1ecb67: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:34:22,882::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:34:22,882::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:34:22,884::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:34:22,885::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:34:22,885::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:34:22,886::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:34:22,887::task::495::TaskManager.Task::(_debug) Task 66025296-52bf-4fff-b0a7-419ccc1ecb67: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:34:22,888::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:34:23,175::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0849574 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15916680::DEBUG::2013-11-20 04:34:24,619::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916680::INFO::2013-11-20 04:34:24,621::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getFloppyList, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916680::DEBUG::2013-11-20 04:34:24,622::task::495::TaskManager.Task::(_debug) Task 31674703-8c5d-427b-b35c-0b9f29ef355e: moving from state init -> state preparing
>Thread-15916680::DEBUG::2013-11-20 04:34:24,624::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`aa79d157-8383-4d41-9e8e-76f03637d58e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1032' at 'public_getIsoList'
>Thread-15916680::DEBUG::2013-11-20 04:34:24,625::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916680::DEBUG::2013-11-20 04:34:24,625::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916680::DEBUG::2013-11-20 04:34:24,626::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`aa79d157-8383-4d41-9e8e-76f03637d58e`::Granted request
>Thread-15916680::DEBUG::2013-11-20 04:34:24,627::task::495::TaskManager.Task::(_debug) Task 31674703-8c5d-427b-b35c-0b9f29ef355e: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916680::DEBUG::2013-11-20 04:34:24,628::task::495::TaskManager.Task::(_debug) Task 31674703-8c5d-427b-b35c-0b9f29ef355e: ref 1 aborting False
>Thread-15916680::DEBUG::2013-11-20 04:34:24,644::task::495::TaskManager.Task::(_debug) Task 31674703-8c5d-427b-b35c-0b9f29ef355e: finished: {'isolist': ['virtio-win-1.5.3.vfd']}
>Thread-15916680::DEBUG::2013-11-20 04:34:24,645::task::495::TaskManager.Task::(_debug) Task 31674703-8c5d-427b-b35c-0b9f29ef355e: moving from state preparing -> state finished
>Thread-15916680::DEBUG::2013-11-20 04:34:24,646::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916680::DEBUG::2013-11-20 04:34:24,647::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916680::DEBUG::2013-11-20 04:34:24,647::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916680::DEBUG::2013-11-20 04:34:24,648::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916680::DEBUG::2013-11-20 04:34:24,649::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916680::DEBUG::2013-11-20 04:34:24,649::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916680::DEBUG::2013-11-20 04:34:24,650::task::495::TaskManager.Task::(_debug) Task 31674703-8c5d-427b-b35c-0b9f29ef355e: ref 0 aborting False
>Thread-15916680::INFO::2013-11-20 04:34:24,651::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getFloppyList, Return response: {'status': {'message': 'OK', 'code': 0}, 'isolist': ['virtio-win-1.5.3.vfd']}
>Dummy-15912651::DEBUG::2013-11-20 04:34:25,194::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916681::DEBUG::2013-11-20 04:34:25,406::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916681::INFO::2013-11-20 04:34:25,408::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getIsoList, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916681::DEBUG::2013-11-20 04:34:25,409::task::495::TaskManager.Task::(_debug) Task 0c6696bb-6f6e-494c-9215-492b5bdf62e9: moving from state init -> state preparing
>Thread-15916681::DEBUG::2013-11-20 04:34:25,411::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f700f9c7-59f0-4c6a-bd47-995bbd009c32`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1032' at 'public_getIsoList'
>Thread-15916681::DEBUG::2013-11-20 04:34:25,412::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916681::DEBUG::2013-11-20 04:34:25,413::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916681::DEBUG::2013-11-20 04:34:25,413::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f700f9c7-59f0-4c6a-bd47-995bbd009c32`::Granted request
>Thread-15916681::DEBUG::2013-11-20 04:34:25,415::task::495::TaskManager.Task::(_debug) Task 0c6696bb-6f6e-494c-9215-492b5bdf62e9: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916681::DEBUG::2013-11-20 04:34:25,416::task::495::TaskManager.Task::(_debug) Task 0c6696bb-6f6e-494c-9215-492b5bdf62e9: ref 1 aborting False
>Thread-15916681::DEBUG::2013-11-20 04:34:25,436::task::495::TaskManager.Task::(_debug) Task 0c6696bb-6f6e-494c-9215-492b5bdf62e9: finished: {'isolist': ['Acronis Boot Image 31-10-2013.iso', 'Windows.2000.SP3.iso', 'ubuntu-13.04-desktop-amd64.iso', 'virtio-win-1.5.3.iso', 'rhev-tools-setup.iso', 'rhev-hypervisor6-6.4-20131016.0.iso', 'ubuntu-12.04.1-server-amd64.iso', 'ubuntu-13.04-server-amd64.iso', 'ubuntu-12.04-mini.iso', 'SW_DVD5_Win_Pro_8_64BIT_English_MLF_X18-16134.ISO', 'RHEV-toolsSetup_3.0_39.iso', 'SW_DVD5_Windows_Svr_2008w_SP2_English__x86_DC_EE_SE_X15-41116.ISO', 'SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO']}
>Thread-15916681::DEBUG::2013-11-20 04:34:25,438::task::495::TaskManager.Task::(_debug) Task 0c6696bb-6f6e-494c-9215-492b5bdf62e9: moving from state preparing -> state finished
>Thread-15916681::DEBUG::2013-11-20 04:34:25,439::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916681::DEBUG::2013-11-20 04:34:25,440::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916681::DEBUG::2013-11-20 04:34:25,441::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916681::DEBUG::2013-11-20 04:34:25,441::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916681::DEBUG::2013-11-20 04:34:25,442::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916681::DEBUG::2013-11-20 04:34:25,443::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916681::DEBUG::2013-11-20 04:34:25,444::task::495::TaskManager.Task::(_debug) Task 0c6696bb-6f6e-494c-9215-492b5bdf62e9: ref 0 aborting False
>Thread-15916681::INFO::2013-11-20 04:34:25,445::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getIsoList, Return response: {'status': {'message': 'OK', 'code': 0}, 'isolist': ['Acronis Boot Image 31-10-2013.iso', 'Windows.2000.SP3.iso', 'ubuntu-13.04-desktop-amd64.iso', 'virtio-win-1.5.3.iso', 'rhev-tools-setup.iso', 'rhev-hypervisor6-6.4-20131016.0.iso', 'ubuntu-12.04.1-server-amd64.iso', 'ubuntu-13.04-server-amd64.iso', 'ubuntu-12.04-mini.iso', 'SW_DVD5_Win_Pro_8_64BIT_English_MLF_X18-16134.ISO', 'RHEV-toolsSetup_3.0_39.iso', 'SW_DVD5_Windows_Svr_2008w_SP2_English__x86_DC_EE_SE_X15-41116.ISO', 'SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588.ISO']}
>Dummy-15912651::DEBUG::2013-11-20 04:34:25,741::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0808304 s, 12.7 MB/s\n'; <rc> = 0
>Thread-15916682::DEBUG::2013-11-20 04:34:25,800::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916682::INFO::2013-11-20 04:34:25,801::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916682::DEBUG::2013-11-20 04:34:25,802::task::495::TaskManager.Task::(_debug) Task b844e40c-5de9-4b3a-98dd-f2afb484cdd1: moving from state init -> state preparing
>Thread-15916682::DEBUG::2013-11-20 04:34:25,803::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916682::DEBUG::2013-11-20 04:34:25,804::task::495::TaskManager.Task::(_debug) Task b844e40c-5de9-4b3a-98dd-f2afb484cdd1: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916682::DEBUG::2013-11-20 04:34:25,804::task::495::TaskManager.Task::(_debug) Task b844e40c-5de9-4b3a-98dd-f2afb484cdd1: moving from state preparing -> state finished
>Thread-15916682::DEBUG::2013-11-20 04:34:25,805::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916682::DEBUG::2013-11-20 04:34:25,806::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916682::DEBUG::2013-11-20 04:34:25,806::task::495::TaskManager.Task::(_debug) Task b844e40c-5de9-4b3a-98dd-f2afb484cdd1: ref 0 aborting False
>Thread-15916682::INFO::2013-11-20 04:34:25,807::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916683::DEBUG::2013-11-20 04:34:25,812::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916683::INFO::2013-11-20 04:34:25,814::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916683::DEBUG::2013-11-20 04:34:25,815::task::495::TaskManager.Task::(_debug) Task e9e5c248-e5fb-430f-ae2f-3526d8ba7dc0: moving from state init -> state preparing
>Thread-15916683::DEBUG::2013-11-20 04:34:25,816::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6cd55610-7aa2-4425-a5aa-33f66ae24561`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916683::DEBUG::2013-11-20 04:34:25,817::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916683::DEBUG::2013-11-20 04:34:25,817::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916683::DEBUG::2013-11-20 04:34:25,819::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6cd55610-7aa2-4425-a5aa-33f66ae24561`::Granted request
>Thread-15916683::DEBUG::2013-11-20 04:34:25,822::task::495::TaskManager.Task::(_debug) Task e9e5c248-e5fb-430f-ae2f-3526d8ba7dc0: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916683::DEBUG::2013-11-20 04:34:25,823::task::495::TaskManager.Task::(_debug) Task e9e5c248-e5fb-430f-ae2f-3526d8ba7dc0: ref 1 aborting False
>Thread-15916683::WARNING::2013-11-20 04:34:25,825::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916683::DEBUG::2013-11-20 04:34:25,826::task::495::TaskManager.Task::(_debug) Task e9e5c248-e5fb-430f-ae2f-3526d8ba7dc0: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916683::DEBUG::2013-11-20 04:34:25,827::task::495::TaskManager.Task::(_debug) Task e9e5c248-e5fb-430f-ae2f-3526d8ba7dc0: moving from state preparing -> state finished
>Thread-15916683::DEBUG::2013-11-20 04:34:25,829::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916683::DEBUG::2013-11-20 04:34:25,829::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916683::DEBUG::2013-11-20 04:34:25,830::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916683::DEBUG::2013-11-20 04:34:25,831::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916683::DEBUG::2013-11-20 04:34:25,832::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916683::DEBUG::2013-11-20 04:34:25,833::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916683::DEBUG::2013-11-20 04:34:25,834::task::495::TaskManager.Task::(_debug) Task e9e5c248-e5fb-430f-ae2f-3526d8ba7dc0: ref 0 aborting False
>Thread-15916683::INFO::2013-11-20 04:34:25,835::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:34:27,757::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:34:28,319::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.071953 s, 14.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:34:30,336::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:34:30,883::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0774532 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15916686::INFO::2013-11-20 04:34:32,448::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916686::DEBUG::2013-11-20 04:34:32,458::task::495::TaskManager.Task::(_debug) Task 86656a4c-ba6d-406f-abd6-54168c720693: moving from state init -> state preparing
>Thread-15916686::DEBUG::2013-11-20 04:34:32,459::task::495::TaskManager.Task::(_debug) Task 86656a4c-ba6d-406f-abd6-54168c720693: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000378847122192', 'lastCheck': 1384922068.981508, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0102889537811', 'lastCheck': 1384922066.4056139, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00198292732239', 'lastCheck': 1384922063.211421, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000491142272949', 'lastCheck': 1384922064.206598, 'valid': True, 'code': 0}}
>Thread-15916686::DEBUG::2013-11-20 04:34:32,460::task::495::TaskManager.Task::(_debug) Task 86656a4c-ba6d-406f-abd6-54168c720693: moving from state preparing -> state finished
>Thread-15916686::DEBUG::2013-11-20 04:34:32,460::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916686::DEBUG::2013-11-20 04:34:32,461::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916686::DEBUG::2013-11-20 04:34:32,462::task::495::TaskManager.Task::(_debug) Task 86656a4c-ba6d-406f-abd6-54168c720693: ref 0 aborting False
>Thread-15916686::INFO::2013-11-20 04:34:32,462::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000378847122192', 'lastCheck': 1384922068.981508, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0102889537811', 'lastCheck': 1384922066.4056139, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00198292732239', 'lastCheck': 1384922063.211421, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000491142272949', 'lastCheck': 1384922064.206598, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:34:32,898::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 04:34:33,116::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15914519::INFO::2013-11-20 04:34:33,117::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15913546::DEBUG::2013-11-20 04:34:33,122::task::495::TaskManager.Task::(_debug) Task 9f464c89-da4f-4d77-b438-a574f2e65e24: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:34:33,124::task::495::TaskManager.Task::(_debug) Task 7d0db423-4887-424a-9de2-29d2b80b7b87: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:34:33,127::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`313ba94c-532b-41bc-8351-a3565d4d8c32`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:34:33,130::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:34:33,130::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`104ed4e6-26c5-4e79-a108-f8bad1ce29dd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:34:33,132::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:34:33,133::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:34:33,134::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`313ba94c-532b-41bc-8351-a3565d4d8c32`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:34:33,135::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:34:33,137::task::495::TaskManager.Task::(_debug) Task 9f464c89-da4f-4d77-b438-a574f2e65e24: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15914519::DEBUG::2013-11-20 04:34:33,137::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`104ed4e6-26c5-4e79-a108-f8bad1ce29dd`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:34:33,138::task::495::TaskManager.Task::(_debug) Task 9f464c89-da4f-4d77-b438-a574f2e65e24: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:34:33,147::task::495::TaskManager.Task::(_debug) Task 7d0db423-4887-424a-9de2-29d2b80b7b87: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913546::DEBUG::2013-11-20 04:34:33,147::task::495::TaskManager.Task::(_debug) Task 9f464c89-da4f-4d77-b438-a574f2e65e24: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:34:33,148::task::495::TaskManager.Task::(_debug) Task 7d0db423-4887-424a-9de2-29d2b80b7b87: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:34:33,149::task::495::TaskManager.Task::(_debug) Task 9f464c89-da4f-4d77-b438-a574f2e65e24: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:34:33,150::task::495::TaskManager.Task::(_debug) Task 7d0db423-4887-424a-9de2-29d2b80b7b87: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:34:33,151::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:34:33,151::task::495::TaskManager.Task::(_debug) Task 7d0db423-4887-424a-9de2-29d2b80b7b87: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:34:33,152::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:34:33,153::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:34:33,155::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15914519::DEBUG::2013-11-20 04:34:33,156::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:34:33,158::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:34:33,158::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913546::DEBUG::2013-11-20 04:34:33,159::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:34:33,160::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:34:33,162::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:34:33,162::task::495::TaskManager.Task::(_debug) Task 9f464c89-da4f-4d77-b438-a574f2e65e24: ref 0 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:34:33,163::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913546::INFO::2013-11-20 04:34:33,164::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:34:33,164::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913546::INFO::2013-11-20 04:34:33,166::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:34:33,167::task::495::TaskManager.Task::(_debug) Task 86543f73-08c1-44df-bb65-17b1001c81da: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:34:33,168::task::495::TaskManager.Task::(_debug) Task 7d0db423-4887-424a-9de2-29d2b80b7b87: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:34:33,172::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`58ecbfe9-37f4-4125-9fcd-7221f1478097`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::INFO::2013-11-20 04:34:33,173::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:34:33,174::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:34:33,176::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:34:33,178::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`58ecbfe9-37f4-4125-9fcd-7221f1478097`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:34:33,180::task::495::TaskManager.Task::(_debug) Task 86543f73-08c1-44df-bb65-17b1001c81da: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:34:33,181::task::495::TaskManager.Task::(_debug) Task 86543f73-08c1-44df-bb65-17b1001c81da: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:34:33,181::task::495::TaskManager.Task::(_debug) Task 86543f73-08c1-44df-bb65-17b1001c81da: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:34:33,182::task::495::TaskManager.Task::(_debug) Task 86543f73-08c1-44df-bb65-17b1001c81da: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:34:33,182::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:34:33,183::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:34:33,184::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:34:33,184::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:34:33,185::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:34:33,186::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:34:33,187::task::495::TaskManager.Task::(_debug) Task 86543f73-08c1-44df-bb65-17b1001c81da: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:34:33,188::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:34:33,189::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:34:33,190::task::495::TaskManager.Task::(_debug) Task c6d70208-2f52-441e-9e4c-4b3dd4b4429f: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:34:33,191::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f0ffcac4-24c7-4689-9dd0-bd3fb3518786`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:34:33,192::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:34:33,193::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:34:33,193::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f0ffcac4-24c7-4689-9dd0-bd3fb3518786`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:34:33,194::task::495::TaskManager.Task::(_debug) Task c6d70208-2f52-441e-9e4c-4b3dd4b4429f: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:34:33,195::task::495::TaskManager.Task::(_debug) Task c6d70208-2f52-441e-9e4c-4b3dd4b4429f: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:34:33,196::task::495::TaskManager.Task::(_debug) Task c6d70208-2f52-441e-9e4c-4b3dd4b4429f: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:34:33,197::task::495::TaskManager.Task::(_debug) Task c6d70208-2f52-441e-9e4c-4b3dd4b4429f: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:34:33,198::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:34:33,198::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:34:33,199::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:34:33,200::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:34:33,201::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:34:33,201::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:34:33,202::task::495::TaskManager.Task::(_debug) Task c6d70208-2f52-441e-9e4c-4b3dd4b4429f: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:34:33,203::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:34:33,204::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:34:33,208::task::495::TaskManager.Task::(_debug) Task d9380ede-feb1-442b-b520-47287f780eeb: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:34:33,210::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`7a23fce2-fcc3-485f-a5a7-dc27d610fc80`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:34:33,210::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:34:33,211::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:34:33,211::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`7a23fce2-fcc3-485f-a5a7-dc27d610fc80`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:34:33,213::task::495::TaskManager.Task::(_debug) Task d9380ede-feb1-442b-b520-47287f780eeb: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:34:33,213::task::495::TaskManager.Task::(_debug) Task d9380ede-feb1-442b-b520-47287f780eeb: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:34:33,214::task::495::TaskManager.Task::(_debug) Task d9380ede-feb1-442b-b520-47287f780eeb: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:34:33,215::task::495::TaskManager.Task::(_debug) Task d9380ede-feb1-442b-b520-47287f780eeb: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:34:33,216::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:34:33,216::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:34:33,217::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:34:33,217::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:34:33,218::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:34:33,218::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:34:33,219::task::495::TaskManager.Task::(_debug) Task d9380ede-feb1-442b-b520-47287f780eeb: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:34:33,220::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:34:33,451::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0850383 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:34:35,465::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914436::DEBUG::2013-11-20 04:34:35,759::guestIF::97::vm.Vm::(_handleMessage) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::u'None'
>Dummy-15912651::DEBUG::2013-11-20 04:34:36,067::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0760963 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15916688::DEBUG::2013-11-20 04:34:36,257::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916688::INFO::2013-11-20 04:34:36,258::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916688::DEBUG::2013-11-20 04:34:36,259::task::495::TaskManager.Task::(_debug) Task a88c193e-d197-449f-a378-59349986aa4f: moving from state init -> state preparing
>Thread-15916688::DEBUG::2013-11-20 04:34:36,260::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916688::DEBUG::2013-11-20 04:34:36,261::task::495::TaskManager.Task::(_debug) Task a88c193e-d197-449f-a378-59349986aa4f: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916688::DEBUG::2013-11-20 04:34:36,262::task::495::TaskManager.Task::(_debug) Task a88c193e-d197-449f-a378-59349986aa4f: moving from state preparing -> state finished
>Thread-15916688::DEBUG::2013-11-20 04:34:36,262::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916688::DEBUG::2013-11-20 04:34:36,263::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916688::DEBUG::2013-11-20 04:34:36,264::task::495::TaskManager.Task::(_debug) Task a88c193e-d197-449f-a378-59349986aa4f: ref 0 aborting False
>Thread-15916688::INFO::2013-11-20 04:34:36,264::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916689::DEBUG::2013-11-20 04:34:36,269::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916689::INFO::2013-11-20 04:34:36,270::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916689::DEBUG::2013-11-20 04:34:36,271::task::495::TaskManager.Task::(_debug) Task 23ca3a25-12f7-4cda-90f3-fb091c139761: moving from state init -> state preparing
>Thread-15916689::DEBUG::2013-11-20 04:34:36,272::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f63cacaf-4f5e-48c0-9bb4-9627ff451e9f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916689::DEBUG::2013-11-20 04:34:36,273::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916689::DEBUG::2013-11-20 04:34:36,273::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916689::DEBUG::2013-11-20 04:34:36,274::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f63cacaf-4f5e-48c0-9bb4-9627ff451e9f`::Granted request
>Thread-15916689::DEBUG::2013-11-20 04:34:36,275::task::495::TaskManager.Task::(_debug) Task 23ca3a25-12f7-4cda-90f3-fb091c139761: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916689::DEBUG::2013-11-20 04:34:36,276::task::495::TaskManager.Task::(_debug) Task 23ca3a25-12f7-4cda-90f3-fb091c139761: ref 1 aborting False
>Thread-15916689::WARNING::2013-11-20 04:34:36,278::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916689::DEBUG::2013-11-20 04:34:36,279::task::495::TaskManager.Task::(_debug) Task 23ca3a25-12f7-4cda-90f3-fb091c139761: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916689::DEBUG::2013-11-20 04:34:36,280::task::495::TaskManager.Task::(_debug) Task 23ca3a25-12f7-4cda-90f3-fb091c139761: moving from state preparing -> state finished
>Thread-15916689::DEBUG::2013-11-20 04:34:36,280::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916689::DEBUG::2013-11-20 04:34:36,281::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916689::DEBUG::2013-11-20 04:34:36,282::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916689::DEBUG::2013-11-20 04:34:36,282::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916689::DEBUG::2013-11-20 04:34:36,283::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916689::DEBUG::2013-11-20 04:34:36,284::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916689::DEBUG::2013-11-20 04:34:36,284::task::495::TaskManager.Task::(_debug) Task 23ca3a25-12f7-4cda-90f3-fb091c139761: ref 0 aborting False
>Thread-15916689::INFO::2013-11-20 04:34:36,285::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:34:38,081::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 04:34:38,356::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:34:38,358::task::495::TaskManager.Task::(_debug) Task b70ec325-05f3-4052-82c7-5ed76a2afefa: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:34:38,361::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e48c9ddd-09d2-4038-a34d-fe7075ba3ede`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:34:38,362::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:34:38,363::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:34:38,364::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e48c9ddd-09d2-4038-a34d-fe7075ba3ede`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:34:38,365::task::495::TaskManager.Task::(_debug) Task b70ec325-05f3-4052-82c7-5ed76a2afefa: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:34:38,366::task::495::TaskManager.Task::(_debug) Task b70ec325-05f3-4052-82c7-5ed76a2afefa: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:34:38,367::task::495::TaskManager.Task::(_debug) Task b70ec325-05f3-4052-82c7-5ed76a2afefa: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:34:38,368::task::495::TaskManager.Task::(_debug) Task b70ec325-05f3-4052-82c7-5ed76a2afefa: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:34:38,369::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:34:38,369::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:34:38,370::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:34:38,371::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:34:38,372::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:34:38,372::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:34:38,373::task::495::TaskManager.Task::(_debug) Task b70ec325-05f3-4052-82c7-5ed76a2afefa: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:34:38,374::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:34:38,607::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0724187 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:34:40,621::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:34:41,163::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0821711 s, 12.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:34:43,179::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914436::DEBUG::2013-11-20 04:34:43,416::guestIF::97::vm.Vm::(_handleMessage) vmId=`5f587d5c-d4b4-4242-b049-ede331c7f8b4`::u'BRD-KLEE\\administrator'
>Dummy-15912651::DEBUG::2013-11-20 04:34:43,651::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0470131 s, 21.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:34:45,661::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:34:46,202::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0778004 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15916694::DEBUG::2013-11-20 04:34:46,931::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916694::INFO::2013-11-20 04:34:46,933::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916694::DEBUG::2013-11-20 04:34:46,933::task::495::TaskManager.Task::(_debug) Task b244d386-1ef8-455b-b0bb-3f0527709dab: moving from state init -> state preparing
>Thread-15916694::DEBUG::2013-11-20 04:34:46,934::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916694::DEBUG::2013-11-20 04:34:46,935::task::495::TaskManager.Task::(_debug) Task b244d386-1ef8-455b-b0bb-3f0527709dab: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916694::DEBUG::2013-11-20 04:34:46,936::task::495::TaskManager.Task::(_debug) Task b244d386-1ef8-455b-b0bb-3f0527709dab: moving from state preparing -> state finished
>Thread-15916694::DEBUG::2013-11-20 04:34:46,937::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916694::DEBUG::2013-11-20 04:34:46,938::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916694::DEBUG::2013-11-20 04:34:46,939::task::495::TaskManager.Task::(_debug) Task b244d386-1ef8-455b-b0bb-3f0527709dab: ref 0 aborting False
>Thread-15916694::INFO::2013-11-20 04:34:46,940::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916695::DEBUG::2013-11-20 04:34:46,945::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916695::INFO::2013-11-20 04:34:46,946::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916695::DEBUG::2013-11-20 04:34:46,947::task::495::TaskManager.Task::(_debug) Task c35377a7-d1dd-44c2-ba0f-5c68e0d5a32e: moving from state init -> state preparing
>Thread-15916695::DEBUG::2013-11-20 04:34:46,948::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6f74462d-bda5-4a9c-9fe5-611dde9d7ccc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916695::DEBUG::2013-11-20 04:34:46,949::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916695::DEBUG::2013-11-20 04:34:46,949::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916695::DEBUG::2013-11-20 04:34:46,950::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6f74462d-bda5-4a9c-9fe5-611dde9d7ccc`::Granted request
>Thread-15916695::DEBUG::2013-11-20 04:34:46,951::task::495::TaskManager.Task::(_debug) Task c35377a7-d1dd-44c2-ba0f-5c68e0d5a32e: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916695::DEBUG::2013-11-20 04:34:46,951::task::495::TaskManager.Task::(_debug) Task c35377a7-d1dd-44c2-ba0f-5c68e0d5a32e: ref 1 aborting False
>Thread-15916695::WARNING::2013-11-20 04:34:46,954::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916695::DEBUG::2013-11-20 04:34:46,955::task::495::TaskManager.Task::(_debug) Task c35377a7-d1dd-44c2-ba0f-5c68e0d5a32e: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916695::DEBUG::2013-11-20 04:34:46,956::task::495::TaskManager.Task::(_debug) Task c35377a7-d1dd-44c2-ba0f-5c68e0d5a32e: moving from state preparing -> state finished
>Thread-15916695::DEBUG::2013-11-20 04:34:46,956::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916695::DEBUG::2013-11-20 04:34:46,957::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916695::DEBUG::2013-11-20 04:34:46,957::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916695::DEBUG::2013-11-20 04:34:46,958::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916695::DEBUG::2013-11-20 04:34:46,959::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916695::DEBUG::2013-11-20 04:34:46,959::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916695::DEBUG::2013-11-20 04:34:46,960::task::495::TaskManager.Task::(_debug) Task c35377a7-d1dd-44c2-ba0f-5c68e0d5a32e: ref 0 aborting False
>Thread-15916695::INFO::2013-11-20 04:34:46,961::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:34:48,215::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916696::INFO::2013-11-20 04:34:48,685::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916696::DEBUG::2013-11-20 04:34:48,686::task::495::TaskManager.Task::(_debug) Task 1646c70b-0360-4aed-9349-8d505ee132f5: moving from state init -> state preparing
>Thread-15916696::DEBUG::2013-11-20 04:34:48,687::task::495::TaskManager.Task::(_debug) Task 1646c70b-0360-4aed-9349-8d505ee132f5: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000519037246704', 'lastCheck': 1384922079.322619, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00161290168762', 'lastCheck': 1384922086.8809969, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00205993652344', 'lastCheck': 1384922083.8700769, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00049901008606', 'lastCheck': 1384922084.8431361, 'valid': True, 'code': 0}}
>Thread-15916696::DEBUG::2013-11-20 04:34:48,688::task::495::TaskManager.Task::(_debug) Task 1646c70b-0360-4aed-9349-8d505ee132f5: moving from state preparing -> state finished
>Thread-15916696::DEBUG::2013-11-20 04:34:48,689::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916696::DEBUG::2013-11-20 04:34:48,691::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916696::DEBUG::2013-11-20 04:34:48,692::task::495::TaskManager.Task::(_debug) Task 1646c70b-0360-4aed-9349-8d505ee132f5: ref 0 aborting False
>Thread-15916696::INFO::2013-11-20 04:34:48,693::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000519037246704', 'lastCheck': 1384922079.322619, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00161290168762', 'lastCheck': 1384922086.8809969, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00205993652344', 'lastCheck': 1384922083.8700769, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00049901008606', 'lastCheck': 1384922084.8431361, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:34:48,795::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0912691 s, 11.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:34:50,810::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:34:50,999::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:34:51,001::task::495::TaskManager.Task::(_debug) Task fa553c64-eb7d-4d4c-81fc-a04b5acb6148: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:34:51,002::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a35277a7-e732-41a0-a6ea-39e1311e353c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:34:51,003::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:34:51,003::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:34:51,004::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a35277a7-e732-41a0-a6ea-39e1311e353c`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:34:51,005::task::495::TaskManager.Task::(_debug) Task fa553c64-eb7d-4d4c-81fc-a04b5acb6148: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:34:51,005::task::495::TaskManager.Task::(_debug) Task fa553c64-eb7d-4d4c-81fc-a04b5acb6148: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:34:51,006::task::495::TaskManager.Task::(_debug) Task fa553c64-eb7d-4d4c-81fc-a04b5acb6148: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:34:51,006::task::495::TaskManager.Task::(_debug) Task fa553c64-eb7d-4d4c-81fc-a04b5acb6148: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:34:51,006::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:34:51,007::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:34:51,007::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:34:51,008::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:34:51,009::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:34:51,010::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:34:51,010::task::495::TaskManager.Task::(_debug) Task fa553c64-eb7d-4d4c-81fc-a04b5acb6148: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:34:51,010::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:34:51,346::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0848974 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:34:53,362::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915918::INFO::2013-11-20 04:34:53,584::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:34:53,587::task::495::TaskManager.Task::(_debug) Task f97507db-0619-442b-bc18-733d1984eb15: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:34:53,592::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`412f9206-584e-492c-ac94-5620b8668f90`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:34:53,594::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:34:53,598::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:34:53,600::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`412f9206-584e-492c-ac94-5620b8668f90`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:34:53,603::task::495::TaskManager.Task::(_debug) Task f97507db-0619-442b-bc18-733d1984eb15: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:34:53,604::task::495::TaskManager.Task::(_debug) Task f97507db-0619-442b-bc18-733d1984eb15: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:34:53,605::task::495::TaskManager.Task::(_debug) Task f97507db-0619-442b-bc18-733d1984eb15: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:34:53,606::task::495::TaskManager.Task::(_debug) Task f97507db-0619-442b-bc18-733d1984eb15: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:34:53,606::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:34:53,607::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:34:53,608::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:34:53,610::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:34:53,611::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:34:53,611::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:34:53,612::task::495::TaskManager.Task::(_debug) Task f97507db-0619-442b-bc18-733d1984eb15: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:34:53,613::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:34:53,919::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.079957 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:34:55,938::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:34:56,497::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.113605 s, 9.0 MB/s\n'; <rc> = 0
>Thread-15916700::DEBUG::2013-11-20 04:34:57,367::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916700::INFO::2013-11-20 04:34:57,369::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916700::DEBUG::2013-11-20 04:34:57,370::task::495::TaskManager.Task::(_debug) Task 2cf01834-9a60-408b-baf4-a339249ce26e: moving from state init -> state preparing
>Thread-15916700::DEBUG::2013-11-20 04:34:57,371::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916700::DEBUG::2013-11-20 04:34:57,371::task::495::TaskManager.Task::(_debug) Task 2cf01834-9a60-408b-baf4-a339249ce26e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916700::DEBUG::2013-11-20 04:34:57,372::task::495::TaskManager.Task::(_debug) Task 2cf01834-9a60-408b-baf4-a339249ce26e: moving from state preparing -> state finished
>Thread-15916700::DEBUG::2013-11-20 04:34:57,372::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916700::DEBUG::2013-11-20 04:34:57,373::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916700::DEBUG::2013-11-20 04:34:57,374::task::495::TaskManager.Task::(_debug) Task 2cf01834-9a60-408b-baf4-a339249ce26e: ref 0 aborting False
>Thread-15916700::INFO::2013-11-20 04:34:57,374::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916701::DEBUG::2013-11-20 04:34:57,379::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916701::INFO::2013-11-20 04:34:57,380::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916701::DEBUG::2013-11-20 04:34:57,380::task::495::TaskManager.Task::(_debug) Task 1cc4afcb-b5b3-408a-aee7-7ae0aff54b30: moving from state init -> state preparing
>Thread-15916701::DEBUG::2013-11-20 04:34:57,382::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`47e7a8f0-d478-4179-914c-ade08fc462eb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916701::DEBUG::2013-11-20 04:34:57,382::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916701::DEBUG::2013-11-20 04:34:57,383::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916701::DEBUG::2013-11-20 04:34:57,384::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`47e7a8f0-d478-4179-914c-ade08fc462eb`::Granted request
>Thread-15916701::DEBUG::2013-11-20 04:34:57,385::task::495::TaskManager.Task::(_debug) Task 1cc4afcb-b5b3-408a-aee7-7ae0aff54b30: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916701::DEBUG::2013-11-20 04:34:57,385::task::495::TaskManager.Task::(_debug) Task 1cc4afcb-b5b3-408a-aee7-7ae0aff54b30: ref 1 aborting False
>Thread-15916701::WARNING::2013-11-20 04:34:57,387::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916701::DEBUG::2013-11-20 04:34:57,388::task::495::TaskManager.Task::(_debug) Task 1cc4afcb-b5b3-408a-aee7-7ae0aff54b30: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916701::DEBUG::2013-11-20 04:34:57,389::task::495::TaskManager.Task::(_debug) Task 1cc4afcb-b5b3-408a-aee7-7ae0aff54b30: moving from state preparing -> state finished
>Thread-15916701::DEBUG::2013-11-20 04:34:57,389::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916701::DEBUG::2013-11-20 04:34:57,390::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916701::DEBUG::2013-11-20 04:34:57,391::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916701::DEBUG::2013-11-20 04:34:57,391::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916701::DEBUG::2013-11-20 04:34:57,392::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916701::DEBUG::2013-11-20 04:34:57,392::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916701::DEBUG::2013-11-20 04:34:57,393::task::495::TaskManager.Task::(_debug) Task 1cc4afcb-b5b3-408a-aee7-7ae0aff54b30: ref 0 aborting False
>Thread-15916701::INFO::2013-11-20 04:34:57,394::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:34:58,514::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915863::INFO::2013-11-20 04:34:58,738::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:34:58,743::task::495::TaskManager.Task::(_debug) Task d59ce67d-6790-47a6-8c53-90a15f31275d: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:34:58,746::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3d5940eb-9fba-4ed6-aaa9-c8e1e42de8da`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:34:58,747::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:34:58,748::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:34:58,749::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3d5940eb-9fba-4ed6-aaa9-c8e1e42de8da`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:34:58,751::task::495::TaskManager.Task::(_debug) Task d59ce67d-6790-47a6-8c53-90a15f31275d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:34:58,752::task::495::TaskManager.Task::(_debug) Task d59ce67d-6790-47a6-8c53-90a15f31275d: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:34:58,753::task::495::TaskManager.Task::(_debug) Task d59ce67d-6790-47a6-8c53-90a15f31275d: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:34:58,755::task::495::TaskManager.Task::(_debug) Task d59ce67d-6790-47a6-8c53-90a15f31275d: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:34:58,756::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:34:58,757::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:34:58,759::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:34:58,760::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:34:58,761::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:34:58,762::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:34:58,762::task::495::TaskManager.Task::(_debug) Task d59ce67d-6790-47a6-8c53-90a15f31275d: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:34:58,763::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:34:59,075::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0786702 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:35:01,091::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:35:01,673::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0776451 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:35:03,687::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 04:35:03,897::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:35:03,901::task::495::TaskManager.Task::(_debug) Task e35bedda-2a51-4922-a884-6157bc822abe: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:35:03,905::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`948c5cc9-c48f-4c1a-ac57-d9b38ac0e59f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:35:03,905::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:35:03,910::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:35:03,911::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`948c5cc9-c48f-4c1a-ac57-d9b38ac0e59f`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:35:03,913::task::495::TaskManager.Task::(_debug) Task e35bedda-2a51-4922-a884-6157bc822abe: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:35:03,913::task::495::TaskManager.Task::(_debug) Task e35bedda-2a51-4922-a884-6157bc822abe: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:35:03,914::task::495::TaskManager.Task::(_debug) Task e35bedda-2a51-4922-a884-6157bc822abe: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:35:03,915::task::495::TaskManager.Task::(_debug) Task e35bedda-2a51-4922-a884-6157bc822abe: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:35:03,916::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:35:03,916::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:35:03,917::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:35:03,918::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:35:03,919::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:35:03,920::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:35:03,921::task::495::TaskManager.Task::(_debug) Task e35bedda-2a51-4922-a884-6157bc822abe: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:35:03,922::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:35:04,235::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0782719 s, 13.1 MB/s\n'; <rc> = 0
>Thread-15916704::INFO::2013-11-20 04:35:04,354::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916704::DEBUG::2013-11-20 04:35:04,355::task::495::TaskManager.Task::(_debug) Task 08fb9ee8-157a-4b30-889c-cabfe65984cc: moving from state init -> state preparing
>Thread-15916704::DEBUG::2013-11-20 04:35:04,356::task::495::TaskManager.Task::(_debug) Task 08fb9ee8-157a-4b30-889c-cabfe65984cc: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000460863113403', 'lastCheck': 1384922099.7422619, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0128259658813', 'lastCheck': 1384922097.147891, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0145401954651', 'lastCheck': 1384922104.3447509, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000532150268555', 'lastCheck': 1384922095.000308, 'valid': True, 'code': 0}}
>Thread-15916704::DEBUG::2013-11-20 04:35:04,356::task::495::TaskManager.Task::(_debug) Task 08fb9ee8-157a-4b30-889c-cabfe65984cc: moving from state preparing -> state finished
>Thread-15916704::DEBUG::2013-11-20 04:35:04,357::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916704::DEBUG::2013-11-20 04:35:04,358::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916704::DEBUG::2013-11-20 04:35:04,358::task::495::TaskManager.Task::(_debug) Task 08fb9ee8-157a-4b30-889c-cabfe65984cc: ref 0 aborting False
>Thread-15916704::INFO::2013-11-20 04:35:04,359::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000460863113403', 'lastCheck': 1384922099.7422619, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0128259658813', 'lastCheck': 1384922097.147891, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0145401954651', 'lastCheck': 1384922104.3447509, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000532150268555', 'lastCheck': 1384922095.000308, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:35:06,250::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:35:06,801::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0756026 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 04:35:07,417::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:35:07,418::task::495::TaskManager.Task::(_debug) Task 039bb11a-1307-4e13-ba89-bf2071927854: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:35:07,420::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`21859e84-6269-4a92-aef3-ddc88ecb818f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:35:07,421::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:35:07,422::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:35:07,422::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`21859e84-6269-4a92-aef3-ddc88ecb818f`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:35:07,424::task::495::TaskManager.Task::(_debug) Task 039bb11a-1307-4e13-ba89-bf2071927854: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:35:07,424::task::495::TaskManager.Task::(_debug) Task 039bb11a-1307-4e13-ba89-bf2071927854: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:35:07,425::task::495::TaskManager.Task::(_debug) Task 039bb11a-1307-4e13-ba89-bf2071927854: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:35:07,425::task::495::TaskManager.Task::(_debug) Task 039bb11a-1307-4e13-ba89-bf2071927854: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:35:07,426::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:35:07,427::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:35:07,427::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:35:07,428::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:35:07,429::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:35:07,429::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:35:07,430::task::495::TaskManager.Task::(_debug) Task 039bb11a-1307-4e13-ba89-bf2071927854: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:35:07,430::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:35:07,431::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:35:07,432::task::495::TaskManager.Task::(_debug) Task 993b264a-21ad-4d35-9dae-11fd68ce99c4: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:35:07,433::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`903241a7-fdb5-43e5-b351-e09651f8587b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:35:07,433::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:35:07,434::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:35:07,434::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`903241a7-fdb5-43e5-b351-e09651f8587b`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:35:07,435::task::495::TaskManager.Task::(_debug) Task 993b264a-21ad-4d35-9dae-11fd68ce99c4: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:35:07,436::task::495::TaskManager.Task::(_debug) Task 993b264a-21ad-4d35-9dae-11fd68ce99c4: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:35:07,436::task::495::TaskManager.Task::(_debug) Task 993b264a-21ad-4d35-9dae-11fd68ce99c4: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:35:07,437::task::495::TaskManager.Task::(_debug) Task 993b264a-21ad-4d35-9dae-11fd68ce99c4: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:35:07,438::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:35:07,439::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:35:07,439::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:35:07,440::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:35:07,440::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:35:07,441::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:35:07,442::task::495::TaskManager.Task::(_debug) Task 993b264a-21ad-4d35-9dae-11fd68ce99c4: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:35:07,442::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15916706::DEBUG::2013-11-20 04:35:07,609::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916706::INFO::2013-11-20 04:35:07,610::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916706::DEBUG::2013-11-20 04:35:07,611::task::495::TaskManager.Task::(_debug) Task d6f38125-baf2-45dd-ac65-2460fe220d15: moving from state init -> state preparing
>Thread-15916706::DEBUG::2013-11-20 04:35:07,612::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916706::DEBUG::2013-11-20 04:35:07,612::task::495::TaskManager.Task::(_debug) Task d6f38125-baf2-45dd-ac65-2460fe220d15: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916706::DEBUG::2013-11-20 04:35:07,613::task::495::TaskManager.Task::(_debug) Task d6f38125-baf2-45dd-ac65-2460fe220d15: moving from state preparing -> state finished
>Thread-15916706::DEBUG::2013-11-20 04:35:07,614::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916706::DEBUG::2013-11-20 04:35:07,614::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916706::DEBUG::2013-11-20 04:35:07,615::task::495::TaskManager.Task::(_debug) Task d6f38125-baf2-45dd-ac65-2460fe220d15: ref 0 aborting False
>Thread-15916706::INFO::2013-11-20 04:35:07,615::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916707::DEBUG::2013-11-20 04:35:07,620::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916707::INFO::2013-11-20 04:35:07,621::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916707::DEBUG::2013-11-20 04:35:07,621::task::495::TaskManager.Task::(_debug) Task 5973d968-8ac6-4073-81fd-5091772cd8e2: moving from state init -> state preparing
>Thread-15916707::DEBUG::2013-11-20 04:35:07,623::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`89f18e54-6135-411f-90b7-7597012c3ae1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916707::DEBUG::2013-11-20 04:35:07,623::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916707::DEBUG::2013-11-20 04:35:07,624::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916707::DEBUG::2013-11-20 04:35:07,625::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`89f18e54-6135-411f-90b7-7597012c3ae1`::Granted request
>Thread-15916707::DEBUG::2013-11-20 04:35:07,626::task::495::TaskManager.Task::(_debug) Task 5973d968-8ac6-4073-81fd-5091772cd8e2: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916707::DEBUG::2013-11-20 04:35:07,626::task::495::TaskManager.Task::(_debug) Task 5973d968-8ac6-4073-81fd-5091772cd8e2: ref 1 aborting False
>Thread-15916707::WARNING::2013-11-20 04:35:07,628::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916707::DEBUG::2013-11-20 04:35:07,629::task::495::TaskManager.Task::(_debug) Task 5973d968-8ac6-4073-81fd-5091772cd8e2: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916707::DEBUG::2013-11-20 04:35:07,630::task::495::TaskManager.Task::(_debug) Task 5973d968-8ac6-4073-81fd-5091772cd8e2: moving from state preparing -> state finished
>Thread-15916707::DEBUG::2013-11-20 04:35:07,631::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916707::DEBUG::2013-11-20 04:35:07,631::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916707::DEBUG::2013-11-20 04:35:07,632::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916707::DEBUG::2013-11-20 04:35:07,633::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916707::DEBUG::2013-11-20 04:35:07,633::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916707::DEBUG::2013-11-20 04:35:07,634::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916707::DEBUG::2013-11-20 04:35:07,635::task::495::TaskManager.Task::(_debug) Task 5973d968-8ac6-4073-81fd-5091772cd8e2: ref 0 aborting False
>Thread-15916707::INFO::2013-11-20 04:35:07,635::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:35:08,816::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:35:09,346::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0764034 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:35:11,363::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:35:11,922::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0842574 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:35:13,938::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 04:35:14,141::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:35:14,148::task::495::TaskManager.Task::(_debug) Task 5ac8aa9a-105e-4be4-83e9-c34b33276486: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:35:14,153::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b05907b0-939c-4ccb-a242-d8b2e2fcbf7f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:35:14,155::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:35:14,158::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:35:14,159::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b05907b0-939c-4ccb-a242-d8b2e2fcbf7f`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:35:14,161::task::495::TaskManager.Task::(_debug) Task 5ac8aa9a-105e-4be4-83e9-c34b33276486: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:35:14,162::task::495::TaskManager.Task::(_debug) Task 5ac8aa9a-105e-4be4-83e9-c34b33276486: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:35:14,163::task::495::TaskManager.Task::(_debug) Task 5ac8aa9a-105e-4be4-83e9-c34b33276486: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:35:14,164::task::495::TaskManager.Task::(_debug) Task 5ac8aa9a-105e-4be4-83e9-c34b33276486: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:35:14,165::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:35:14,166::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:35:14,166::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:35:14,167::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:35:14,168::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:35:14,168::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:35:14,169::task::495::TaskManager.Task::(_debug) Task 5ac8aa9a-105e-4be4-83e9-c34b33276486: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:35:14,170::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:35:14,491::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.087548 s, 11.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:35:16,507::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:35:17,019::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0475621 s, 21.5 MB/s\n'; <rc> = 0
>Thread-15916712::DEBUG::2013-11-20 04:35:17,829::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916712::INFO::2013-11-20 04:35:17,830::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916712::DEBUG::2013-11-20 04:35:17,831::task::495::TaskManager.Task::(_debug) Task 73f6808f-4477-4022-8ade-8faed39ad414: moving from state init -> state preparing
>Thread-15916712::DEBUG::2013-11-20 04:35:17,832::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916712::DEBUG::2013-11-20 04:35:17,832::task::495::TaskManager.Task::(_debug) Task 73f6808f-4477-4022-8ade-8faed39ad414: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916712::DEBUG::2013-11-20 04:35:17,833::task::495::TaskManager.Task::(_debug) Task 73f6808f-4477-4022-8ade-8faed39ad414: moving from state preparing -> state finished
>Thread-15916712::DEBUG::2013-11-20 04:35:17,833::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916712::DEBUG::2013-11-20 04:35:17,834::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916712::DEBUG::2013-11-20 04:35:17,835::task::495::TaskManager.Task::(_debug) Task 73f6808f-4477-4022-8ade-8faed39ad414: ref 0 aborting False
>Thread-15916712::INFO::2013-11-20 04:35:17,835::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916713::DEBUG::2013-11-20 04:35:17,840::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916713::INFO::2013-11-20 04:35:17,841::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916713::DEBUG::2013-11-20 04:35:17,842::task::495::TaskManager.Task::(_debug) Task 1cbcc482-7b4b-46ee-8d97-254e7a969de1: moving from state init -> state preparing
>Thread-15916713::DEBUG::2013-11-20 04:35:17,843::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f3f39ac8-c356-4d6a-989a-cdb91ed40c0c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916713::DEBUG::2013-11-20 04:35:17,844::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916713::DEBUG::2013-11-20 04:35:17,845::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916713::DEBUG::2013-11-20 04:35:17,845::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f3f39ac8-c356-4d6a-989a-cdb91ed40c0c`::Granted request
>Thread-15916713::DEBUG::2013-11-20 04:35:17,846::task::495::TaskManager.Task::(_debug) Task 1cbcc482-7b4b-46ee-8d97-254e7a969de1: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916713::DEBUG::2013-11-20 04:35:17,847::task::495::TaskManager.Task::(_debug) Task 1cbcc482-7b4b-46ee-8d97-254e7a969de1: ref 1 aborting False
>Thread-15916713::WARNING::2013-11-20 04:35:17,849::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916713::DEBUG::2013-11-20 04:35:17,850::task::495::TaskManager.Task::(_debug) Task 1cbcc482-7b4b-46ee-8d97-254e7a969de1: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916713::DEBUG::2013-11-20 04:35:17,851::task::495::TaskManager.Task::(_debug) Task 1cbcc482-7b4b-46ee-8d97-254e7a969de1: moving from state preparing -> state finished
>Thread-15916713::DEBUG::2013-11-20 04:35:17,851::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916713::DEBUG::2013-11-20 04:35:17,852::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916713::DEBUG::2013-11-20 04:35:17,853::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916713::DEBUG::2013-11-20 04:35:17,853::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916713::DEBUG::2013-11-20 04:35:17,854::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916713::DEBUG::2013-11-20 04:35:17,854::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916713::DEBUG::2013-11-20 04:35:17,855::task::495::TaskManager.Task::(_debug) Task 1cbcc482-7b4b-46ee-8d97-254e7a969de1: ref 0 aborting False
>Thread-15916713::INFO::2013-11-20 04:35:17,855::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:35:19,030::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:35:19,577::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0783638 s, 13.1 MB/s\n'; <rc> = 0
>Thread-15916714::INFO::2013-11-20 04:35:19,908::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916714::DEBUG::2013-11-20 04:35:19,909::task::495::TaskManager.Task::(_debug) Task 64e3d011-b9e4-4d5f-adf6-896856fc8e41: moving from state init -> state preparing
>Thread-15916714::DEBUG::2013-11-20 04:35:19,910::task::495::TaskManager.Task::(_debug) Task 64e3d011-b9e4-4d5f-adf6-896856fc8e41: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000816822052002', 'lastCheck': 1384922110.0178061, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0211210250854', 'lastCheck': 1384922117.7448659, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0232181549072', 'lastCheck': 1384922114.6182411, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000468969345093', 'lastCheck': 1384922115.5918331, 'valid': True, 'code': 0}}
>Thread-15916714::DEBUG::2013-11-20 04:35:19,911::task::495::TaskManager.Task::(_debug) Task 64e3d011-b9e4-4d5f-adf6-896856fc8e41: moving from state preparing -> state finished
>Thread-15916714::DEBUG::2013-11-20 04:35:19,912::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916714::DEBUG::2013-11-20 04:35:19,912::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916714::DEBUG::2013-11-20 04:35:19,913::task::495::TaskManager.Task::(_debug) Task 64e3d011-b9e4-4d5f-adf6-896856fc8e41: ref 0 aborting False
>Thread-15916714::INFO::2013-11-20 04:35:19,913::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000816822052002', 'lastCheck': 1384922110.0178061, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0211210250854', 'lastCheck': 1384922117.7448659, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0232181549072', 'lastCheck': 1384922114.6182411, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000468969345093', 'lastCheck': 1384922115.5918331, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:35:21,591::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:35:22,131::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0750823 s, 13.6 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 04:35:22,202::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:35:22,202::task::495::TaskManager.Task::(_debug) Task 6d462254-bb0b-4932-95e1-c3dd7999b879: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:35:22,204::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9a28c4ce-f8d7-4fc4-8f1c-5d7d7e322be0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:35:22,205::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:35:22,206::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:35:22,207::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9a28c4ce-f8d7-4fc4-8f1c-5d7d7e322be0`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:35:22,208::task::495::TaskManager.Task::(_debug) Task 6d462254-bb0b-4932-95e1-c3dd7999b879: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:35:22,208::task::495::TaskManager.Task::(_debug) Task 6d462254-bb0b-4932-95e1-c3dd7999b879: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:35:22,209::task::495::TaskManager.Task::(_debug) Task 6d462254-bb0b-4932-95e1-c3dd7999b879: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:35:22,210::task::495::TaskManager.Task::(_debug) Task 6d462254-bb0b-4932-95e1-c3dd7999b879: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:35:22,210::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:35:22,211::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:35:22,212::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:35:22,212::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:35:22,213::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:35:22,213::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:35:22,214::task::495::TaskManager.Task::(_debug) Task 6d462254-bb0b-4932-95e1-c3dd7999b879: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:35:22,215::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:35:24,146::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 04:35:24,361::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:35:24,365::task::495::TaskManager.Task::(_debug) Task 81f0821e-4875-4038-a0b5-587f28536680: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:35:24,369::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`34c55e38-6434-4ac5-bb3c-456001d02803`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:35:24,369::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:35:24,372::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:35:24,372::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`34c55e38-6434-4ac5-bb3c-456001d02803`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:35:24,374::task::495::TaskManager.Task::(_debug) Task 81f0821e-4875-4038-a0b5-587f28536680: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:35:24,375::task::495::TaskManager.Task::(_debug) Task 81f0821e-4875-4038-a0b5-587f28536680: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:35:24,377::task::495::TaskManager.Task::(_debug) Task 81f0821e-4875-4038-a0b5-587f28536680: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:35:24,379::task::495::TaskManager.Task::(_debug) Task 81f0821e-4875-4038-a0b5-587f28536680: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:35:24,380::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:35:24,380::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:35:24,381::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:35:24,382::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:35:24,382::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:35:24,383::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:35:24,384::task::495::TaskManager.Task::(_debug) Task 81f0821e-4875-4038-a0b5-587f28536680: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:35:24,385::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:35:24,701::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0774672 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:35:26,715::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:35:27,278::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0844483 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15916718::DEBUG::2013-11-20 04:35:28,704::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916718::INFO::2013-11-20 04:35:28,706::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916718::DEBUG::2013-11-20 04:35:28,706::task::495::TaskManager.Task::(_debug) Task 668f997c-f3ea-4b1f-9be0-ac878fa0fc35: moving from state init -> state preparing
>Thread-15916718::DEBUG::2013-11-20 04:35:28,707::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916718::DEBUG::2013-11-20 04:35:28,708::task::495::TaskManager.Task::(_debug) Task 668f997c-f3ea-4b1f-9be0-ac878fa0fc35: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916718::DEBUG::2013-11-20 04:35:28,708::task::495::TaskManager.Task::(_debug) Task 668f997c-f3ea-4b1f-9be0-ac878fa0fc35: moving from state preparing -> state finished
>Thread-15916718::DEBUG::2013-11-20 04:35:28,709::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916718::DEBUG::2013-11-20 04:35:28,710::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916718::DEBUG::2013-11-20 04:35:28,710::task::495::TaskManager.Task::(_debug) Task 668f997c-f3ea-4b1f-9be0-ac878fa0fc35: ref 0 aborting False
>Thread-15916718::INFO::2013-11-20 04:35:28,711::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916719::DEBUG::2013-11-20 04:35:28,716::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916719::INFO::2013-11-20 04:35:28,717::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916719::DEBUG::2013-11-20 04:35:28,717::task::495::TaskManager.Task::(_debug) Task cc0fe80d-0138-46a4-8378-75dd0f3e6c35: moving from state init -> state preparing
>Thread-15916719::DEBUG::2013-11-20 04:35:28,719::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4de2e8e0-c8d9-4aa6-9505-fc409ce04218`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916719::DEBUG::2013-11-20 04:35:28,720::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916719::DEBUG::2013-11-20 04:35:28,720::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916719::DEBUG::2013-11-20 04:35:28,721::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4de2e8e0-c8d9-4aa6-9505-fc409ce04218`::Granted request
>Thread-15916719::DEBUG::2013-11-20 04:35:28,722::task::495::TaskManager.Task::(_debug) Task cc0fe80d-0138-46a4-8378-75dd0f3e6c35: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916719::DEBUG::2013-11-20 04:35:28,723::task::495::TaskManager.Task::(_debug) Task cc0fe80d-0138-46a4-8378-75dd0f3e6c35: ref 1 aborting False
>Thread-15916719::WARNING::2013-11-20 04:35:28,725::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916719::DEBUG::2013-11-20 04:35:28,726::task::495::TaskManager.Task::(_debug) Task cc0fe80d-0138-46a4-8378-75dd0f3e6c35: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916719::DEBUG::2013-11-20 04:35:28,727::task::495::TaskManager.Task::(_debug) Task cc0fe80d-0138-46a4-8378-75dd0f3e6c35: moving from state preparing -> state finished
>Thread-15916719::DEBUG::2013-11-20 04:35:28,728::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916719::DEBUG::2013-11-20 04:35:28,728::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916719::DEBUG::2013-11-20 04:35:28,729::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916719::DEBUG::2013-11-20 04:35:28,730::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916719::DEBUG::2013-11-20 04:35:28,730::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916719::DEBUG::2013-11-20 04:35:28,731::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916719::DEBUG::2013-11-20 04:35:28,732::task::495::TaskManager.Task::(_debug) Task cc0fe80d-0138-46a4-8378-75dd0f3e6c35: ref 0 aborting False
>Thread-15916719::INFO::2013-11-20 04:35:28,732::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:35:29,294::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:35:29,829::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0815459 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:35:31,849::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:35:32,407::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0830064 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:35:34,424::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 04:35:34,624::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15914519::INFO::2013-11-20 04:35:34,625::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15913546::DEBUG::2013-11-20 04:35:34,627::task::495::TaskManager.Task::(_debug) Task afc14f1a-a8d6-43a4-8e2b-8ffc37dd7c51: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:35:34,628::task::495::TaskManager.Task::(_debug) Task 2dfad056-099c-4feb-9098-dd1d12dd1917: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:35:34,631::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`97547648-dab7-4497-a8e8-02478092779e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:35:34,634::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8191a7ee-d23b-48be-a17e-bb91538669e4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:35:34,635::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:35:34,635::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:35:34,638::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:35:34,638::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`97547648-dab7-4497-a8e8-02478092779e`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:35:34,640::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:35:34,640::task::495::TaskManager.Task::(_debug) Task afc14f1a-a8d6-43a4-8e2b-8ffc37dd7c51: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15914519::DEBUG::2013-11-20 04:35:34,641::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8191a7ee-d23b-48be-a17e-bb91538669e4`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:35:34,642::task::495::TaskManager.Task::(_debug) Task afc14f1a-a8d6-43a4-8e2b-8ffc37dd7c51: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:35:34,643::task::495::TaskManager.Task::(_debug) Task afc14f1a-a8d6-43a4-8e2b-8ffc37dd7c51: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:35:34,643::task::495::TaskManager.Task::(_debug) Task 2dfad056-099c-4feb-9098-dd1d12dd1917: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913546::DEBUG::2013-11-20 04:35:34,644::task::495::TaskManager.Task::(_debug) Task afc14f1a-a8d6-43a4-8e2b-8ffc37dd7c51: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:35:34,644::task::495::TaskManager.Task::(_debug) Task 2dfad056-099c-4feb-9098-dd1d12dd1917: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:35:34,645::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:35:34,645::task::495::TaskManager.Task::(_debug) Task 2dfad056-099c-4feb-9098-dd1d12dd1917: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:35:34,646::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:35:34,646::task::495::TaskManager.Task::(_debug) Task 2dfad056-099c-4feb-9098-dd1d12dd1917: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:35:34,647::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15914519::DEBUG::2013-11-20 04:35:34,647::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:35:34,648::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:35:34,648::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:35:34,649::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:35:34,649::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913546::DEBUG::2013-11-20 04:35:34,649::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:35:34,650::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:35:34,651::task::495::TaskManager.Task::(_debug) Task afc14f1a-a8d6-43a4-8e2b-8ffc37dd7c51: ref 0 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:35:34,651::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913546::INFO::2013-11-20 04:35:34,651::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:35:34,652::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913546::INFO::2013-11-20 04:35:34,652::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:35:34,653::task::495::TaskManager.Task::(_debug) Task f595fe18-3784-4374-9685-a92af9e40e3e: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:35:34,654::task::495::TaskManager.Task::(_debug) Task 2dfad056-099c-4feb-9098-dd1d12dd1917: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:35:34,655::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`721929b0-6e35-4bd2-9619-198b4fb91eda`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::INFO::2013-11-20 04:35:34,655::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:35:34,656::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:35:34,657::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:35:34,657::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`721929b0-6e35-4bd2-9619-198b4fb91eda`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:35:34,659::task::495::TaskManager.Task::(_debug) Task f595fe18-3784-4374-9685-a92af9e40e3e: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:35:34,660::task::495::TaskManager.Task::(_debug) Task f595fe18-3784-4374-9685-a92af9e40e3e: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:35:34,661::task::495::TaskManager.Task::(_debug) Task f595fe18-3784-4374-9685-a92af9e40e3e: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:35:34,661::task::495::TaskManager.Task::(_debug) Task f595fe18-3784-4374-9685-a92af9e40e3e: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:35:34,662::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:35:34,663::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:35:34,664::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:35:34,665::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:35:34,665::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:35:34,666::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:35:34,667::task::495::TaskManager.Task::(_debug) Task f595fe18-3784-4374-9685-a92af9e40e3e: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:35:34,667::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:35:34,668::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:35:34,668::task::495::TaskManager.Task::(_debug) Task 02fe60a5-0edf-41b3-93ef-184d89d59f8c: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:35:34,669::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6be8c642-99b8-4652-b40c-1187322661ac`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:35:34,669::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:35:34,670::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:35:34,670::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6be8c642-99b8-4652-b40c-1187322661ac`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:35:34,671::task::495::TaskManager.Task::(_debug) Task 02fe60a5-0edf-41b3-93ef-184d89d59f8c: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:35:34,671::task::495::TaskManager.Task::(_debug) Task 02fe60a5-0edf-41b3-93ef-184d89d59f8c: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:35:34,671::task::495::TaskManager.Task::(_debug) Task 02fe60a5-0edf-41b3-93ef-184d89d59f8c: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:35:34,672::task::495::TaskManager.Task::(_debug) Task 02fe60a5-0edf-41b3-93ef-184d89d59f8c: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:35:34,672::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:35:34,672::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:35:34,673::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:35:34,674::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:35:34,674::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:35:34,674::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:35:34,675::task::495::TaskManager.Task::(_debug) Task 02fe60a5-0edf-41b3-93ef-184d89d59f8c: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:35:34,675::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:35:34,676::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:35:34,676::task::495::TaskManager.Task::(_debug) Task fea3fae7-00bd-4505-a660-b0ff09304eed: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:35:34,677::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`3fbd36ac-6364-4d92-93e6-49c7f5fbdf90`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:35:34,677::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:35:34,677::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:35:34,678::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`3fbd36ac-6364-4d92-93e6-49c7f5fbdf90`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:35:34,679::task::495::TaskManager.Task::(_debug) Task fea3fae7-00bd-4505-a660-b0ff09304eed: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:35:34,682::task::495::TaskManager.Task::(_debug) Task fea3fae7-00bd-4505-a660-b0ff09304eed: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:35:34,683::task::495::TaskManager.Task::(_debug) Task fea3fae7-00bd-4505-a660-b0ff09304eed: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:35:34,683::task::495::TaskManager.Task::(_debug) Task fea3fae7-00bd-4505-a660-b0ff09304eed: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:35:34,683::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:35:34,683::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:35:34,684::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:35:34,685::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:35:34,685::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:35:34,685::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:35:34,686::task::495::TaskManager.Task::(_debug) Task fea3fae7-00bd-4505-a660-b0ff09304eed: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:35:34,686::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:35:34,953::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0919057 s, 11.1 MB/s\n'; <rc> = 0
>Thread-15916722::INFO::2013-11-20 04:35:35,656::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916722::DEBUG::2013-11-20 04:35:35,657::task::495::TaskManager.Task::(_debug) Task b02a3534-ab50-4b4f-84ff-7be215803ede: moving from state init -> state preparing
>Thread-15916722::DEBUG::2013-11-20 04:35:35,658::task::495::TaskManager.Task::(_debug) Task b02a3534-ab50-4b4f-84ff-7be215803ede: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000372886657715', 'lastCheck': 1384922130.5133121, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00902104377747', 'lastCheck': 1384922127.95525, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00949501991272', 'lastCheck': 1384922135.1199679, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000524997711182', 'lastCheck': 1384922125.8143251, 'valid': True, 'code': 0}}
>Thread-15916722::DEBUG::2013-11-20 04:35:35,659::task::495::TaskManager.Task::(_debug) Task b02a3534-ab50-4b4f-84ff-7be215803ede: moving from state preparing -> state finished
>Thread-15916722::DEBUG::2013-11-20 04:35:35,660::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916722::DEBUG::2013-11-20 04:35:35,660::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916722::DEBUG::2013-11-20 04:35:35,661::task::495::TaskManager.Task::(_debug) Task b02a3534-ab50-4b4f-84ff-7be215803ede: ref 0 aborting False
>Thread-15916722::INFO::2013-11-20 04:35:35,661::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000372886657715', 'lastCheck': 1384922130.5133121, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00902104377747', 'lastCheck': 1384922127.95525, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00949501991272', 'lastCheck': 1384922135.1199679, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000524997711182', 'lastCheck': 1384922125.8143251, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:35:36,970::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:35:37,482::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0720609 s, 14.2 MB/s\n'; <rc> = 0
>Thread-15916724::DEBUG::2013-11-20 04:35:39,303::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916724::INFO::2013-11-20 04:35:39,304::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916724::DEBUG::2013-11-20 04:35:39,305::task::495::TaskManager.Task::(_debug) Task cd73fe66-e1e2-4828-9fbf-31fc7d773dcc: moving from state init -> state preparing
>Thread-15916724::DEBUG::2013-11-20 04:35:39,306::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916724::DEBUG::2013-11-20 04:35:39,307::task::495::TaskManager.Task::(_debug) Task cd73fe66-e1e2-4828-9fbf-31fc7d773dcc: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916724::DEBUG::2013-11-20 04:35:39,308::task::495::TaskManager.Task::(_debug) Task cd73fe66-e1e2-4828-9fbf-31fc7d773dcc: moving from state preparing -> state finished
>Thread-15916724::DEBUG::2013-11-20 04:35:39,308::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916724::DEBUG::2013-11-20 04:35:39,309::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916724::DEBUG::2013-11-20 04:35:39,310::task::495::TaskManager.Task::(_debug) Task cd73fe66-e1e2-4828-9fbf-31fc7d773dcc: ref 0 aborting False
>Thread-15916724::INFO::2013-11-20 04:35:39,310::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916725::DEBUG::2013-11-20 04:35:39,315::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916725::INFO::2013-11-20 04:35:39,316::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916725::DEBUG::2013-11-20 04:35:39,316::task::495::TaskManager.Task::(_debug) Task 3e3ed97d-9fe9-4546-83d2-4415d3f782c7: moving from state init -> state preparing
>Thread-15916725::DEBUG::2013-11-20 04:35:39,318::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`fee7da68-5dd3-4e60-a2f7-fd26c1df1494`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916725::DEBUG::2013-11-20 04:35:39,318::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916725::DEBUG::2013-11-20 04:35:39,319::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916725::DEBUG::2013-11-20 04:35:39,320::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`fee7da68-5dd3-4e60-a2f7-fd26c1df1494`::Granted request
>Thread-15916725::DEBUG::2013-11-20 04:35:39,320::task::495::TaskManager.Task::(_debug) Task 3e3ed97d-9fe9-4546-83d2-4415d3f782c7: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916725::DEBUG::2013-11-20 04:35:39,321::task::495::TaskManager.Task::(_debug) Task 3e3ed97d-9fe9-4546-83d2-4415d3f782c7: ref 1 aborting False
>Thread-15916725::WARNING::2013-11-20 04:35:39,323::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916725::DEBUG::2013-11-20 04:35:39,324::task::495::TaskManager.Task::(_debug) Task 3e3ed97d-9fe9-4546-83d2-4415d3f782c7: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916725::DEBUG::2013-11-20 04:35:39,325::task::495::TaskManager.Task::(_debug) Task 3e3ed97d-9fe9-4546-83d2-4415d3f782c7: moving from state preparing -> state finished
>Thread-15916725::DEBUG::2013-11-20 04:35:39,325::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916725::DEBUG::2013-11-20 04:35:39,326::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916725::DEBUG::2013-11-20 04:35:39,327::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916725::DEBUG::2013-11-20 04:35:39,327::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916725::DEBUG::2013-11-20 04:35:39,328::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916725::DEBUG::2013-11-20 04:35:39,328::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916725::DEBUG::2013-11-20 04:35:39,329::task::495::TaskManager.Task::(_debug) Task 3e3ed97d-9fe9-4546-83d2-4415d3f782c7: ref 0 aborting False
>Thread-15916725::INFO::2013-11-20 04:35:39,330::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:35:39,495::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 04:35:39,709::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:35:39,711::task::495::TaskManager.Task::(_debug) Task 7c8a2161-1775-48b1-a4c6-fe0e56fd6626: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:35:39,716::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`397aca7c-4cb7-40e3-b3b3-be6b466f66c4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:35:39,717::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:35:39,718::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:35:39,719::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`397aca7c-4cb7-40e3-b3b3-be6b466f66c4`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:35:39,721::task::495::TaskManager.Task::(_debug) Task 7c8a2161-1775-48b1-a4c6-fe0e56fd6626: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:35:39,721::task::495::TaskManager.Task::(_debug) Task 7c8a2161-1775-48b1-a4c6-fe0e56fd6626: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:35:39,722::task::495::TaskManager.Task::(_debug) Task 7c8a2161-1775-48b1-a4c6-fe0e56fd6626: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:35:39,723::task::495::TaskManager.Task::(_debug) Task 7c8a2161-1775-48b1-a4c6-fe0e56fd6626: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:35:39,724::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:35:39,725::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:35:39,725::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:35:39,726::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:35:39,727::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:35:39,727::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:35:39,728::task::495::TaskManager.Task::(_debug) Task 7c8a2161-1775-48b1-a4c6-fe0e56fd6626: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:35:39,729::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:35:40,015::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0554858 s, 18.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:35:42,024::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:35:42,591::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0776669 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:35:44,607::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:35:45,152::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0806176 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:35:47,169::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:35:47,716::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0779454 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:35:49,732::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:35:50,274::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0812943 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15916730::DEBUG::2013-11-20 04:35:50,977::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916730::INFO::2013-11-20 04:35:50,979::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916730::DEBUG::2013-11-20 04:35:50,979::task::495::TaskManager.Task::(_debug) Task 5d5d100e-8518-49f8-902b-42cb0d8242d2: moving from state init -> state preparing
>Thread-15916730::DEBUG::2013-11-20 04:35:50,980::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916730::DEBUG::2013-11-20 04:35:50,981::task::495::TaskManager.Task::(_debug) Task 5d5d100e-8518-49f8-902b-42cb0d8242d2: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916730::DEBUG::2013-11-20 04:35:50,981::task::495::TaskManager.Task::(_debug) Task 5d5d100e-8518-49f8-902b-42cb0d8242d2: moving from state preparing -> state finished
>Thread-15916730::DEBUG::2013-11-20 04:35:50,982::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916730::DEBUG::2013-11-20 04:35:50,983::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916730::DEBUG::2013-11-20 04:35:50,983::task::495::TaskManager.Task::(_debug) Task 5d5d100e-8518-49f8-902b-42cb0d8242d2: ref 0 aborting False
>Thread-15916730::INFO::2013-11-20 04:35:50,984::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916731::DEBUG::2013-11-20 04:35:50,988::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916731::INFO::2013-11-20 04:35:50,989::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916731::DEBUG::2013-11-20 04:35:50,990::task::495::TaskManager.Task::(_debug) Task af6ecb1d-f2a3-465a-b1bd-41e64caef40b: moving from state init -> state preparing
>Thread-15916731::DEBUG::2013-11-20 04:35:50,991::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d467a890-9706-4930-8faa-6bfed04a2474`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916731::DEBUG::2013-11-20 04:35:50,992::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916731::DEBUG::2013-11-20 04:35:50,993::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916731::DEBUG::2013-11-20 04:35:50,993::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d467a890-9706-4930-8faa-6bfed04a2474`::Granted request
>Thread-15916731::DEBUG::2013-11-20 04:35:50,994::task::495::TaskManager.Task::(_debug) Task af6ecb1d-f2a3-465a-b1bd-41e64caef40b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916731::DEBUG::2013-11-20 04:35:50,995::task::495::TaskManager.Task::(_debug) Task af6ecb1d-f2a3-465a-b1bd-41e64caef40b: ref 1 aborting False
>Thread-15916731::WARNING::2013-11-20 04:35:50,997::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916731::DEBUG::2013-11-20 04:35:50,998::task::495::TaskManager.Task::(_debug) Task af6ecb1d-f2a3-465a-b1bd-41e64caef40b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916731::DEBUG::2013-11-20 04:35:50,998::task::495::TaskManager.Task::(_debug) Task af6ecb1d-f2a3-465a-b1bd-41e64caef40b: moving from state preparing -> state finished
>Thread-15916731::DEBUG::2013-11-20 04:35:50,999::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916731::DEBUG::2013-11-20 04:35:50,999::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916731::DEBUG::2013-11-20 04:35:51,000::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916731::DEBUG::2013-11-20 04:35:51,001::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916731::DEBUG::2013-11-20 04:35:51,001::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916731::DEBUG::2013-11-20 04:35:51,002::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916731::DEBUG::2013-11-20 04:35:51,003::task::495::TaskManager.Task::(_debug) Task af6ecb1d-f2a3-465a-b1bd-41e64caef40b: ref 0 aborting False
>Thread-15916731::INFO::2013-11-20 04:35:51,003::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916732::INFO::2013-11-20 04:35:52,118::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916732::DEBUG::2013-11-20 04:35:52,119::task::495::TaskManager.Task::(_debug) Task df44de2b-f11a-4c8d-a0f6-9dd77d0b0672: moving from state init -> state preparing
>Thread-15916732::DEBUG::2013-11-20 04:35:52,120::task::495::TaskManager.Task::(_debug) Task df44de2b-f11a-4c8d-a0f6-9dd77d0b0672: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000775098800659', 'lastCheck': 1384922150.9631569, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0342299938202', 'lastCheck': 1384922148.4155819, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0264699459076', 'lastCheck': 1384922145.285269, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000505924224854', 'lastCheck': 1384922146.2527859, 'valid': True, 'code': 0}}
>Thread-15916732::DEBUG::2013-11-20 04:35:52,121::task::495::TaskManager.Task::(_debug) Task df44de2b-f11a-4c8d-a0f6-9dd77d0b0672: moving from state preparing -> state finished
>Thread-15916732::DEBUG::2013-11-20 04:35:52,122::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916732::DEBUG::2013-11-20 04:35:52,122::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916732::DEBUG::2013-11-20 04:35:52,123::task::495::TaskManager.Task::(_debug) Task df44de2b-f11a-4c8d-a0f6-9dd77d0b0672: ref 0 aborting False
>Thread-15916732::INFO::2013-11-20 04:35:52,124::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000775098800659', 'lastCheck': 1384922150.9631569, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0342299938202', 'lastCheck': 1384922148.4155819, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0264699459076', 'lastCheck': 1384922145.285269, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000505924224854', 'lastCheck': 1384922146.2527859, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:35:52,297::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:35:52,496::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:35:52,500::task::495::TaskManager.Task::(_debug) Task 86e2d040-2cce-4937-8b11-782454a30b08: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:35:52,503::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`93de3aaa-89da-46ef-a29d-eaa9a36edb80`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:35:52,504::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:35:52,506::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:35:52,507::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`93de3aaa-89da-46ef-a29d-eaa9a36edb80`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:35:52,509::task::495::TaskManager.Task::(_debug) Task 86e2d040-2cce-4937-8b11-782454a30b08: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:35:52,510::task::495::TaskManager.Task::(_debug) Task 86e2d040-2cce-4937-8b11-782454a30b08: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:35:52,512::task::495::TaskManager.Task::(_debug) Task 86e2d040-2cce-4937-8b11-782454a30b08: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:35:52,512::task::495::TaskManager.Task::(_debug) Task 86e2d040-2cce-4937-8b11-782454a30b08: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:35:52,513::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:35:52,514::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:35:52,516::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:35:52,517::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:35:52,518::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:35:52,519::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:35:52,520::task::495::TaskManager.Task::(_debug) Task 86e2d040-2cce-4937-8b11-782454a30b08: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:35:52,521::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:35:52,826::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0764246 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:35:54,844::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915918::INFO::2013-11-20 04:35:55,076::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:35:55,083::task::495::TaskManager.Task::(_debug) Task c94e64b1-281c-4171-8f52-d3d8d150153d: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:35:55,088::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`72aef36b-2bcd-4d37-af03-470519a7c78b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:35:55,089::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:35:55,093::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:35:55,094::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`72aef36b-2bcd-4d37-af03-470519a7c78b`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:35:55,095::task::495::TaskManager.Task::(_debug) Task c94e64b1-281c-4171-8f52-d3d8d150153d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:35:55,096::task::495::TaskManager.Task::(_debug) Task c94e64b1-281c-4171-8f52-d3d8d150153d: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:35:55,097::task::495::TaskManager.Task::(_debug) Task c94e64b1-281c-4171-8f52-d3d8d150153d: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:35:55,098::task::495::TaskManager.Task::(_debug) Task c94e64b1-281c-4171-8f52-d3d8d150153d: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:35:55,099::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:35:55,100::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:35:55,100::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:35:55,101::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:35:55,102::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:35:55,102::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:35:55,103::task::495::TaskManager.Task::(_debug) Task c94e64b1-281c-4171-8f52-d3d8d150153d: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:35:55,104::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:35:55,414::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.082701 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:35:57,428::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:35:57,960::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0776458 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:35:59,977::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915863::INFO::2013-11-20 04:36:00,193::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:36:00,197::task::495::TaskManager.Task::(_debug) Task e50e069a-d322-46bb-ad74-1b7e1aace6b5: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:36:00,200::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1e3cb008-a0d3-4c74-b25e-1afa7699e5bb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:36:00,201::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:36:00,204::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:36:00,207::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1e3cb008-a0d3-4c74-b25e-1afa7699e5bb`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:36:00,209::task::495::TaskManager.Task::(_debug) Task e50e069a-d322-46bb-ad74-1b7e1aace6b5: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:36:00,210::task::495::TaskManager.Task::(_debug) Task e50e069a-d322-46bb-ad74-1b7e1aace6b5: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:36:00,211::task::495::TaskManager.Task::(_debug) Task e50e069a-d322-46bb-ad74-1b7e1aace6b5: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:36:00,212::task::495::TaskManager.Task::(_debug) Task e50e069a-d322-46bb-ad74-1b7e1aace6b5: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:36:00,213::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:36:00,214::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:36:00,214::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:36:00,215::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:36:00,216::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:36:00,216::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:36:00,217::task::495::TaskManager.Task::(_debug) Task e50e069a-d322-46bb-ad74-1b7e1aace6b5: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:36:00,218::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:36:00,524::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0820899 s, 12.5 MB/s\n'; <rc> = 0
>Thread-15916737::DEBUG::2013-11-20 04:36:02,071::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916737::INFO::2013-11-20 04:36:02,071::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916737::DEBUG::2013-11-20 04:36:02,072::task::495::TaskManager.Task::(_debug) Task d5d6eae9-1739-4d43-b1b8-1811f113c863: moving from state init -> state preparing
>Thread-15916737::DEBUG::2013-11-20 04:36:02,072::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916737::DEBUG::2013-11-20 04:36:02,073::task::495::TaskManager.Task::(_debug) Task d5d6eae9-1739-4d43-b1b8-1811f113c863: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916737::DEBUG::2013-11-20 04:36:02,073::task::495::TaskManager.Task::(_debug) Task d5d6eae9-1739-4d43-b1b8-1811f113c863: moving from state preparing -> state finished
>Thread-15916737::DEBUG::2013-11-20 04:36:02,073::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916737::DEBUG::2013-11-20 04:36:02,074::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916737::DEBUG::2013-11-20 04:36:02,074::task::495::TaskManager.Task::(_debug) Task d5d6eae9-1739-4d43-b1b8-1811f113c863: ref 0 aborting False
>Thread-15916737::INFO::2013-11-20 04:36:02,075::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916738::DEBUG::2013-11-20 04:36:02,079::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916738::INFO::2013-11-20 04:36:02,079::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916738::DEBUG::2013-11-20 04:36:02,079::task::495::TaskManager.Task::(_debug) Task 1ef2c871-9b10-43ed-a7af-6420c2be5cbc: moving from state init -> state preparing
>Thread-15916738::DEBUG::2013-11-20 04:36:02,080::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ec0c2aef-3d59-4285-9617-6666d79a10a6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916738::DEBUG::2013-11-20 04:36:02,081::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916738::DEBUG::2013-11-20 04:36:02,081::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916738::DEBUG::2013-11-20 04:36:02,082::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ec0c2aef-3d59-4285-9617-6666d79a10a6`::Granted request
>Thread-15916738::DEBUG::2013-11-20 04:36:02,082::task::495::TaskManager.Task::(_debug) Task 1ef2c871-9b10-43ed-a7af-6420c2be5cbc: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916738::DEBUG::2013-11-20 04:36:02,083::task::495::TaskManager.Task::(_debug) Task 1ef2c871-9b10-43ed-a7af-6420c2be5cbc: ref 1 aborting False
>Thread-15916738::WARNING::2013-11-20 04:36:02,084::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916738::DEBUG::2013-11-20 04:36:02,085::task::495::TaskManager.Task::(_debug) Task 1ef2c871-9b10-43ed-a7af-6420c2be5cbc: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916738::DEBUG::2013-11-20 04:36:02,086::task::495::TaskManager.Task::(_debug) Task 1ef2c871-9b10-43ed-a7af-6420c2be5cbc: moving from state preparing -> state finished
>Thread-15916738::DEBUG::2013-11-20 04:36:02,087::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916738::DEBUG::2013-11-20 04:36:02,087::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916738::DEBUG::2013-11-20 04:36:02,088::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916738::DEBUG::2013-11-20 04:36:02,088::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916738::DEBUG::2013-11-20 04:36:02,089::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916738::DEBUG::2013-11-20 04:36:02,089::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916738::DEBUG::2013-11-20 04:36:02,090::task::495::TaskManager.Task::(_debug) Task 1ef2c871-9b10-43ed-a7af-6420c2be5cbc: ref 0 aborting False
>Thread-15916738::INFO::2013-11-20 04:36:02,091::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:36:02,539::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:36:03,084::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0871711 s, 11.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:36:05,100::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 04:36:05,328::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:36:05,336::task::495::TaskManager.Task::(_debug) Task bf90d028-f5be-40e0-baeb-197a401238cf: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:36:05,347::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d629786e-9b07-4e66-a380-8afb72390494`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:36:05,348::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:36:05,349::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:36:05,350::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d629786e-9b07-4e66-a380-8afb72390494`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:36:05,352::task::495::TaskManager.Task::(_debug) Task bf90d028-f5be-40e0-baeb-197a401238cf: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:36:05,353::task::495::TaskManager.Task::(_debug) Task bf90d028-f5be-40e0-baeb-197a401238cf: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:36:05,354::task::495::TaskManager.Task::(_debug) Task bf90d028-f5be-40e0-baeb-197a401238cf: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:36:05,354::task::495::TaskManager.Task::(_debug) Task bf90d028-f5be-40e0-baeb-197a401238cf: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:36:05,355::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:36:05,356::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:36:05,356::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:36:05,357::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:36:05,358::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:36:05,358::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:36:05,359::task::495::TaskManager.Task::(_debug) Task bf90d028-f5be-40e0-baeb-197a401238cf: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:36:05,360::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:36:05,674::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0834608 s, 12.3 MB/s\n'; <rc> = 0
>Thread-42840::DEBUG::2013-11-20 04:36:06,749::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 04:36:07,442::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:36:07,690::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916740::INFO::2013-11-20 04:36:08,066::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916740::DEBUG::2013-11-20 04:36:08,067::task::495::TaskManager.Task::(_debug) Task 79e81511-6ef7-4bfb-ac22-3c5171d8cf42: moving from state init -> state preparing
>Thread-15916740::DEBUG::2013-11-20 04:36:08,068::task::495::TaskManager.Task::(_debug) Task 79e81511-6ef7-4bfb-ac22-3c5171d8cf42: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000558853149414', 'lastCheck': 1384922161.197649, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148105621338', 'lastCheck': 1384922158.6265111, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0177431106567', 'lastCheck': 1384922165.7776849, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000304222106934', 'lastCheck': 1384922167.4442401, 'valid': True, 'code': 0}}
>Thread-15916740::DEBUG::2013-11-20 04:36:08,069::task::495::TaskManager.Task::(_debug) Task 79e81511-6ef7-4bfb-ac22-3c5171d8cf42: moving from state preparing -> state finished
>Thread-15916740::DEBUG::2013-11-20 04:36:08,070::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916740::DEBUG::2013-11-20 04:36:08,070::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916740::DEBUG::2013-11-20 04:36:08,071::task::495::TaskManager.Task::(_debug) Task 79e81511-6ef7-4bfb-ac22-3c5171d8cf42: ref 0 aborting False
>Thread-15916740::INFO::2013-11-20 04:36:08,072::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000558853149414', 'lastCheck': 1384922161.197649, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148105621338', 'lastCheck': 1384922158.6265111, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0177431106567', 'lastCheck': 1384922165.7776849, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000304222106934', 'lastCheck': 1384922167.4442401, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:36:08,287::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.104369 s, 9.8 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 04:36:09,009::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:36:09,010::task::495::TaskManager.Task::(_debug) Task 5432b4ea-a3ad-43b6-beb4-32baabff1a2b: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:36:09,011::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`13c958c8-1700-4848-878c-145b21b93a2d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:36:09,012::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:36:09,013::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:36:09,013::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`13c958c8-1700-4848-878c-145b21b93a2d`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:36:09,014::task::495::TaskManager.Task::(_debug) Task 5432b4ea-a3ad-43b6-beb4-32baabff1a2b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:36:09,015::task::495::TaskManager.Task::(_debug) Task 5432b4ea-a3ad-43b6-beb4-32baabff1a2b: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:36:09,016::task::495::TaskManager.Task::(_debug) Task 5432b4ea-a3ad-43b6-beb4-32baabff1a2b: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:36:09,016::task::495::TaskManager.Task::(_debug) Task 5432b4ea-a3ad-43b6-beb4-32baabff1a2b: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:36:09,017::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:36:09,018::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:36:09,019::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:36:09,019::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:36:09,020::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:36:09,020::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:36:09,021::task::495::TaskManager.Task::(_debug) Task 5432b4ea-a3ad-43b6-beb4-32baabff1a2b: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:36:09,022::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:36:09,022::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:36:09,023::task::495::TaskManager.Task::(_debug) Task a554cb42-aa09-48d7-a4d1-e2d52309dc3c: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:36:09,024::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`05e76bf5-5676-40dc-a98d-13cb0b61a09a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:36:09,025::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:36:09,025::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:36:09,026::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`05e76bf5-5676-40dc-a98d-13cb0b61a09a`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:36:09,032::task::495::TaskManager.Task::(_debug) Task a554cb42-aa09-48d7-a4d1-e2d52309dc3c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:36:09,033::task::495::TaskManager.Task::(_debug) Task a554cb42-aa09-48d7-a4d1-e2d52309dc3c: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:36:09,034::task::495::TaskManager.Task::(_debug) Task a554cb42-aa09-48d7-a4d1-e2d52309dc3c: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:36:09,035::task::495::TaskManager.Task::(_debug) Task a554cb42-aa09-48d7-a4d1-e2d52309dc3c: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:36:09,035::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:36:09,036::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:36:09,037::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:36:09,037::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:36:09,038::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:36:09,038::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:36:09,039::task::495::TaskManager.Task::(_debug) Task a554cb42-aa09-48d7-a4d1-e2d52309dc3c: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:36:09,040::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:36:10,301::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:36:10,826::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0722585 s, 14.2 MB/s\n'; <rc> = 0
>Thread-15916743::DEBUG::2013-11-20 04:36:12,633::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916743::INFO::2013-11-20 04:36:12,635::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916743::DEBUG::2013-11-20 04:36:12,636::task::495::TaskManager.Task::(_debug) Task 0e04fd8c-2ab8-42f7-94fd-ee4b3afe0355: moving from state init -> state preparing
>Thread-15916743::DEBUG::2013-11-20 04:36:12,637::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916743::DEBUG::2013-11-20 04:36:12,637::task::495::TaskManager.Task::(_debug) Task 0e04fd8c-2ab8-42f7-94fd-ee4b3afe0355: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916743::DEBUG::2013-11-20 04:36:12,638::task::495::TaskManager.Task::(_debug) Task 0e04fd8c-2ab8-42f7-94fd-ee4b3afe0355: moving from state preparing -> state finished
>Thread-15916743::DEBUG::2013-11-20 04:36:12,638::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916743::DEBUG::2013-11-20 04:36:12,639::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916743::DEBUG::2013-11-20 04:36:12,640::task::495::TaskManager.Task::(_debug) Task 0e04fd8c-2ab8-42f7-94fd-ee4b3afe0355: ref 0 aborting False
>Thread-15916743::INFO::2013-11-20 04:36:12,640::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916744::DEBUG::2013-11-20 04:36:12,645::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916744::INFO::2013-11-20 04:36:12,646::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916744::DEBUG::2013-11-20 04:36:12,647::task::495::TaskManager.Task::(_debug) Task 2eb66e89-69da-4e1f-a84a-a88ebb159755: moving from state init -> state preparing
>Thread-15916744::DEBUG::2013-11-20 04:36:12,648::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f05aaa19-cd66-48e2-952e-96e028197216`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916744::DEBUG::2013-11-20 04:36:12,649::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916744::DEBUG::2013-11-20 04:36:12,650::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916744::DEBUG::2013-11-20 04:36:12,651::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f05aaa19-cd66-48e2-952e-96e028197216`::Granted request
>Thread-15916744::DEBUG::2013-11-20 04:36:12,652::task::495::TaskManager.Task::(_debug) Task 2eb66e89-69da-4e1f-a84a-a88ebb159755: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916744::DEBUG::2013-11-20 04:36:12,653::task::495::TaskManager.Task::(_debug) Task 2eb66e89-69da-4e1f-a84a-a88ebb159755: ref 1 aborting False
>Thread-15916744::WARNING::2013-11-20 04:36:12,656::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916744::DEBUG::2013-11-20 04:36:12,657::task::495::TaskManager.Task::(_debug) Task 2eb66e89-69da-4e1f-a84a-a88ebb159755: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916744::DEBUG::2013-11-20 04:36:12,658::task::495::TaskManager.Task::(_debug) Task 2eb66e89-69da-4e1f-a84a-a88ebb159755: moving from state preparing -> state finished
>Thread-15916744::DEBUG::2013-11-20 04:36:12,659::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916744::DEBUG::2013-11-20 04:36:12,659::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916744::DEBUG::2013-11-20 04:36:12,660::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916744::DEBUG::2013-11-20 04:36:12,661::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916744::DEBUG::2013-11-20 04:36:12,661::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916744::DEBUG::2013-11-20 04:36:12,662::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916744::DEBUG::2013-11-20 04:36:12,663::task::495::TaskManager.Task::(_debug) Task 2eb66e89-69da-4e1f-a84a-a88ebb159755: ref 0 aborting False
>Thread-15916744::INFO::2013-11-20 04:36:12,663::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:36:12,842::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:36:13,362::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0811984 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:36:15,378::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 04:36:15,588::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:36:15,592::task::495::TaskManager.Task::(_debug) Task b73ca259-c0b0-4e00-9aa4-872b9f3d9d50: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:36:15,594::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2f657baa-a15f-448d-9637-d12fb5810380`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:36:15,596::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:36:15,599::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:36:15,600::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2f657baa-a15f-448d-9637-d12fb5810380`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:36:15,602::task::495::TaskManager.Task::(_debug) Task b73ca259-c0b0-4e00-9aa4-872b9f3d9d50: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:36:15,602::task::495::TaskManager.Task::(_debug) Task b73ca259-c0b0-4e00-9aa4-872b9f3d9d50: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:36:15,604::task::495::TaskManager.Task::(_debug) Task b73ca259-c0b0-4e00-9aa4-872b9f3d9d50: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:36:15,604::task::495::TaskManager.Task::(_debug) Task b73ca259-c0b0-4e00-9aa4-872b9f3d9d50: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:36:15,605::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:36:15,605::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:36:15,606::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:36:15,607::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:36:15,607::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:36:15,608::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:36:15,608::task::495::TaskManager.Task::(_debug) Task b73ca259-c0b0-4e00-9aa4-872b9f3d9d50: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:36:15,609::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:36:15,913::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.08596 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:36:17,929::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:36:18,449::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0786016 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:36:20,464::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:36:21,017::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0895026 s, 11.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:36:23,034::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916748::DEBUG::2013-11-20 04:36:23,324::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916748::INFO::2013-11-20 04:36:23,326::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916748::DEBUG::2013-11-20 04:36:23,328::task::495::TaskManager.Task::(_debug) Task 5321b0e4-5b35-4739-9883-02816448e64d: moving from state init -> state preparing
>Thread-15916748::DEBUG::2013-11-20 04:36:23,329::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916748::DEBUG::2013-11-20 04:36:23,330::task::495::TaskManager.Task::(_debug) Task 5321b0e4-5b35-4739-9883-02816448e64d: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916748::DEBUG::2013-11-20 04:36:23,331::task::495::TaskManager.Task::(_debug) Task 5321b0e4-5b35-4739-9883-02816448e64d: moving from state preparing -> state finished
>Thread-15916748::DEBUG::2013-11-20 04:36:23,332::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916748::DEBUG::2013-11-20 04:36:23,333::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916748::DEBUG::2013-11-20 04:36:23,335::task::495::TaskManager.Task::(_debug) Task 5321b0e4-5b35-4739-9883-02816448e64d: ref 0 aborting False
>Thread-15916748::INFO::2013-11-20 04:36:23,337::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916749::DEBUG::2013-11-20 04:36:23,348::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916749::INFO::2013-11-20 04:36:23,350::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916749::DEBUG::2013-11-20 04:36:23,350::task::495::TaskManager.Task::(_debug) Task 41ad80f3-c1c6-4bd7-87c2-5fa7f8da6120: moving from state init -> state preparing
>Thread-15916749::DEBUG::2013-11-20 04:36:23,352::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e37b3285-cb34-4840-9124-dbbe14159048`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916749::DEBUG::2013-11-20 04:36:23,353::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916749::DEBUG::2013-11-20 04:36:23,354::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916749::DEBUG::2013-11-20 04:36:23,354::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e37b3285-cb34-4840-9124-dbbe14159048`::Granted request
>Thread-15916749::DEBUG::2013-11-20 04:36:23,356::task::495::TaskManager.Task::(_debug) Task 41ad80f3-c1c6-4bd7-87c2-5fa7f8da6120: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916749::DEBUG::2013-11-20 04:36:23,356::task::495::TaskManager.Task::(_debug) Task 41ad80f3-c1c6-4bd7-87c2-5fa7f8da6120: ref 1 aborting False
>Thread-15916749::WARNING::2013-11-20 04:36:23,359::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916749::DEBUG::2013-11-20 04:36:23,361::task::495::TaskManager.Task::(_debug) Task 41ad80f3-c1c6-4bd7-87c2-5fa7f8da6120: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916749::DEBUG::2013-11-20 04:36:23,361::task::495::TaskManager.Task::(_debug) Task 41ad80f3-c1c6-4bd7-87c2-5fa7f8da6120: moving from state preparing -> state finished
>Thread-15916749::DEBUG::2013-11-20 04:36:23,362::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916749::DEBUG::2013-11-20 04:36:23,363::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916749::DEBUG::2013-11-20 04:36:23,363::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916749::DEBUG::2013-11-20 04:36:23,364::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916749::DEBUG::2013-11-20 04:36:23,365::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916749::DEBUG::2013-11-20 04:36:23,365::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916749::DEBUG::2013-11-20 04:36:23,366::task::495::TaskManager.Task::(_debug) Task 41ad80f3-c1c6-4bd7-87c2-5fa7f8da6120: ref 0 aborting False
>Thread-15916749::INFO::2013-11-20 04:36:23,367::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:36:23,604::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0821073 s, 12.5 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 04:36:23,646::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:36:23,647::task::495::TaskManager.Task::(_debug) Task 93eefa85-b6f2-47bb-8830-d2892efcf17b: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:36:23,649::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d352681a-2f2c-4459-ac75-f5a90dce1aa0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:36:23,649::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:36:23,650::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:36:23,651::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d352681a-2f2c-4459-ac75-f5a90dce1aa0`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:36:23,652::task::495::TaskManager.Task::(_debug) Task 93eefa85-b6f2-47bb-8830-d2892efcf17b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:36:23,653::task::495::TaskManager.Task::(_debug) Task 93eefa85-b6f2-47bb-8830-d2892efcf17b: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:36:23,653::task::495::TaskManager.Task::(_debug) Task 93eefa85-b6f2-47bb-8830-d2892efcf17b: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:36:23,654::task::495::TaskManager.Task::(_debug) Task 93eefa85-b6f2-47bb-8830-d2892efcf17b: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:36:23,655::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:36:23,656::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:36:23,657::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:36:23,657::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:36:23,658::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:36:23,659::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:36:23,659::task::495::TaskManager.Task::(_debug) Task 93eefa85-b6f2-47bb-8830-d2892efcf17b: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:36:23,660::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15916750::INFO::2013-11-20 04:36:23,942::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916750::DEBUG::2013-11-20 04:36:23,943::task::495::TaskManager.Task::(_debug) Task 18919a5c-12a7-4f52-b0ea-e687f50c1779: moving from state init -> state preparing
>Thread-15916750::DEBUG::2013-11-20 04:36:23,944::task::495::TaskManager.Task::(_debug) Task 18919a5c-12a7-4f52-b0ea-e687f50c1779: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000613927841187', 'lastCheck': 1384922181.7030449, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00161004066467', 'lastCheck': 1384922179.1361971, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00235199928284', 'lastCheck': 1384922176.0567679, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000488996505737', 'lastCheck': 1384922177.5903261, 'valid': True, 'code': 0}}
>Thread-15916750::DEBUG::2013-11-20 04:36:23,944::task::495::TaskManager.Task::(_debug) Task 18919a5c-12a7-4f52-b0ea-e687f50c1779: moving from state preparing -> state finished
>Thread-15916750::DEBUG::2013-11-20 04:36:23,945::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916750::DEBUG::2013-11-20 04:36:23,945::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916750::DEBUG::2013-11-20 04:36:23,946::task::495::TaskManager.Task::(_debug) Task 18919a5c-12a7-4f52-b0ea-e687f50c1779: ref 0 aborting False
>Thread-15916750::INFO::2013-11-20 04:36:23,947::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000613927841187', 'lastCheck': 1384922181.7030449, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00161004066467', 'lastCheck': 1384922179.1361971, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00235199928284', 'lastCheck': 1384922176.0567679, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000488996505737', 'lastCheck': 1384922177.5903261, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:36:25,618::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 04:36:25,850::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:36:25,852::task::495::TaskManager.Task::(_debug) Task af0bdb1f-10c2-4c91-8f9b-6d307dcd3298: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:36:25,855::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`08897f0f-bdee-439f-a98d-afa639c1b346`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:36:25,856::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:36:25,858::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:36:25,861::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`08897f0f-bdee-439f-a98d-afa639c1b346`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:36:25,863::task::495::TaskManager.Task::(_debug) Task af0bdb1f-10c2-4c91-8f9b-6d307dcd3298: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:36:25,863::task::495::TaskManager.Task::(_debug) Task af0bdb1f-10c2-4c91-8f9b-6d307dcd3298: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:36:25,864::task::495::TaskManager.Task::(_debug) Task af0bdb1f-10c2-4c91-8f9b-6d307dcd3298: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:36:25,866::task::495::TaskManager.Task::(_debug) Task af0bdb1f-10c2-4c91-8f9b-6d307dcd3298: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:36:25,867::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:36:25,868::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:36:25,869::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:36:25,870::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:36:25,870::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:36:25,871::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:36:25,871::task::495::TaskManager.Task::(_debug) Task af0bdb1f-10c2-4c91-8f9b-6d307dcd3298: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:36:25,872::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:36:26,142::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0524211 s, 19.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:36:28,152::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:36:28,709::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0776406 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:36:30,725::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:36:31,285::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0841714 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:36:33,303::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:36:33,728::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0593804 s, 17.2 MB/s\n'; <rc> = 0
>Thread-15916755::DEBUG::2013-11-20 04:36:33,871::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916755::INFO::2013-11-20 04:36:33,872::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916755::DEBUG::2013-11-20 04:36:33,873::task::495::TaskManager.Task::(_debug) Task b12a4627-ff3d-4382-9640-3272cf33401f: moving from state init -> state preparing
>Thread-15916755::DEBUG::2013-11-20 04:36:33,874::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916755::DEBUG::2013-11-20 04:36:33,875::task::495::TaskManager.Task::(_debug) Task b12a4627-ff3d-4382-9640-3272cf33401f: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916755::DEBUG::2013-11-20 04:36:33,876::task::495::TaskManager.Task::(_debug) Task b12a4627-ff3d-4382-9640-3272cf33401f: moving from state preparing -> state finished
>Thread-15916755::DEBUG::2013-11-20 04:36:33,876::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916755::DEBUG::2013-11-20 04:36:33,877::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916755::DEBUG::2013-11-20 04:36:33,878::task::495::TaskManager.Task::(_debug) Task b12a4627-ff3d-4382-9640-3272cf33401f: ref 0 aborting False
>Thread-15916755::INFO::2013-11-20 04:36:33,878::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916756::DEBUG::2013-11-20 04:36:33,885::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916756::INFO::2013-11-20 04:36:33,886::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916756::DEBUG::2013-11-20 04:36:33,887::task::495::TaskManager.Task::(_debug) Task d81302e8-65f4-4adf-8f76-5a0c5398b15c: moving from state init -> state preparing
>Thread-15916756::DEBUG::2013-11-20 04:36:33,888::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`73d64e4a-6eaf-4467-afbd-fd4fccb16fa7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916756::DEBUG::2013-11-20 04:36:33,889::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916756::DEBUG::2013-11-20 04:36:33,890::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916756::DEBUG::2013-11-20 04:36:33,891::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`73d64e4a-6eaf-4467-afbd-fd4fccb16fa7`::Granted request
>Thread-15916756::DEBUG::2013-11-20 04:36:33,892::task::495::TaskManager.Task::(_debug) Task d81302e8-65f4-4adf-8f76-5a0c5398b15c: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916756::DEBUG::2013-11-20 04:36:33,893::task::495::TaskManager.Task::(_debug) Task d81302e8-65f4-4adf-8f76-5a0c5398b15c: ref 1 aborting False
>Thread-15916756::WARNING::2013-11-20 04:36:33,895::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916756::DEBUG::2013-11-20 04:36:33,896::task::495::TaskManager.Task::(_debug) Task d81302e8-65f4-4adf-8f76-5a0c5398b15c: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916756::DEBUG::2013-11-20 04:36:33,898::task::495::TaskManager.Task::(_debug) Task d81302e8-65f4-4adf-8f76-5a0c5398b15c: moving from state preparing -> state finished
>Thread-15916756::DEBUG::2013-11-20 04:36:33,898::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916756::DEBUG::2013-11-20 04:36:33,899::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916756::DEBUG::2013-11-20 04:36:33,900::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916756::DEBUG::2013-11-20 04:36:33,900::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916756::DEBUG::2013-11-20 04:36:33,901::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916756::DEBUG::2013-11-20 04:36:33,901::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916756::DEBUG::2013-11-20 04:36:33,903::task::495::TaskManager.Task::(_debug) Task d81302e8-65f4-4adf-8f76-5a0c5398b15c: ref 0 aborting False
>Thread-15916756::INFO::2013-11-20 04:36:33,904::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:36:35,740::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 04:36:35,975::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15914519::INFO::2013-11-20 04:36:35,978::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:36:35,981::task::495::TaskManager.Task::(_debug) Task d84579f1-17aa-4e48-950f-4a52f5243d81: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:36:35,981::task::495::TaskManager.Task::(_debug) Task c6125747-52ef-40e8-b6d4-fcff45b89888: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:36:35,986::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0ed3606f-7dfd-4a23-8651-09827c3de908`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:36:35,991::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:36:35,992::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e09e1985-6332-4c86-ace7-5feead0f9825`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:36:35,993::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:36:35,994::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:36:35,995::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0ed3606f-7dfd-4a23-8651-09827c3de908`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:36:36,000::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:36:36,000::task::495::TaskManager.Task::(_debug) Task d84579f1-17aa-4e48-950f-4a52f5243d81: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913546::DEBUG::2013-11-20 04:36:36,001::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e09e1985-6332-4c86-ace7-5feead0f9825`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:36:36,002::task::495::TaskManager.Task::(_debug) Task d84579f1-17aa-4e48-950f-4a52f5243d81: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:36:36,003::task::495::TaskManager.Task::(_debug) Task d84579f1-17aa-4e48-950f-4a52f5243d81: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:36:36,003::task::495::TaskManager.Task::(_debug) Task c6125747-52ef-40e8-b6d4-fcff45b89888: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15914519::DEBUG::2013-11-20 04:36:36,004::task::495::TaskManager.Task::(_debug) Task d84579f1-17aa-4e48-950f-4a52f5243d81: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:36:36,005::task::495::TaskManager.Task::(_debug) Task c6125747-52ef-40e8-b6d4-fcff45b89888: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:36:36,005::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:36:36,006::task::495::TaskManager.Task::(_debug) Task c6125747-52ef-40e8-b6d4-fcff45b89888: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:36:36,006::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:36:36,007::task::495::TaskManager.Task::(_debug) Task c6125747-52ef-40e8-b6d4-fcff45b89888: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:36:36,007::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913546::DEBUG::2013-11-20 04:36:36,008::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:36:36,008::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:36:36,009::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:36:36,009::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:36:36,010::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15914519::DEBUG::2013-11-20 04:36:36,010::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:36:36,011::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:36:36,011::task::495::TaskManager.Task::(_debug) Task d84579f1-17aa-4e48-950f-4a52f5243d81: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:36:36,012::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15914519::INFO::2013-11-20 04:36:36,012::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:36:36,013::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:36:36,013::task::495::TaskManager.Task::(_debug) Task c6125747-52ef-40e8-b6d4-fcff45b89888: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:36:36,014::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:36:36,015::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:36:36,016::task::495::TaskManager.Task::(_debug) Task 0b7f4a54-720d-4604-a96c-b72236bbad84: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:36:36,017::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e8eb1c40-3c6e-4690-9d78-10f50d0727f3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:36:36,018::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:36:36,019::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:36:36,020::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e8eb1c40-3c6e-4690-9d78-10f50d0727f3`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:36:36,021::task::495::TaskManager.Task::(_debug) Task 0b7f4a54-720d-4604-a96c-b72236bbad84: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:36:36,021::task::495::TaskManager.Task::(_debug) Task 0b7f4a54-720d-4604-a96c-b72236bbad84: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:36:36,022::task::495::TaskManager.Task::(_debug) Task 0b7f4a54-720d-4604-a96c-b72236bbad84: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:36:36,022::task::495::TaskManager.Task::(_debug) Task 0b7f4a54-720d-4604-a96c-b72236bbad84: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:36:36,024::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:36:36,024::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:36:36,025::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:36:36,026::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:36:36,026::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:36:36,027::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:36:36,027::task::495::TaskManager.Task::(_debug) Task 0b7f4a54-720d-4604-a96c-b72236bbad84: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:36:36,028::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:36:36,029::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:36:36,030::task::495::TaskManager.Task::(_debug) Task d6be66db-7241-455e-a530-c06d4f6c599a: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:36:36,031::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`37e861e1-d8b5-4451-9351-589bd7e63946`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:36:36,031::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:36:36,032::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:36:36,033::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`37e861e1-d8b5-4451-9351-589bd7e63946`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:36:36,033::task::495::TaskManager.Task::(_debug) Task d6be66db-7241-455e-a530-c06d4f6c599a: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:36:36,034::task::495::TaskManager.Task::(_debug) Task d6be66db-7241-455e-a530-c06d4f6c599a: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:36:36,035::task::495::TaskManager.Task::(_debug) Task d6be66db-7241-455e-a530-c06d4f6c599a: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:36:36,035::task::495::TaskManager.Task::(_debug) Task d6be66db-7241-455e-a530-c06d4f6c599a: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:36:36,036::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:36:36,036::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:36:36,037::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:36:36,038::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:36:36,038::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:36:36,039::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:36:36,040::task::495::TaskManager.Task::(_debug) Task d6be66db-7241-455e-a530-c06d4f6c599a: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:36:36,044::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:36:36,045::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:36:36,045::task::495::TaskManager.Task::(_debug) Task f343aa08-eebc-4b8c-8d36-574dae964521: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:36:36,047::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`ad945435-46f9-4a86-acbb-7a24a1282ee9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:36:36,047::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:36:36,048::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:36:36,049::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`ad945435-46f9-4a86-acbb-7a24a1282ee9`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:36:36,050::task::495::TaskManager.Task::(_debug) Task f343aa08-eebc-4b8c-8d36-574dae964521: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:36:36,051::task::495::TaskManager.Task::(_debug) Task f343aa08-eebc-4b8c-8d36-574dae964521: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:36:36,052::task::495::TaskManager.Task::(_debug) Task f343aa08-eebc-4b8c-8d36-574dae964521: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:36:36,052::task::495::TaskManager.Task::(_debug) Task f343aa08-eebc-4b8c-8d36-574dae964521: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:36:36,053::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:36:36,054::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:36:36,055::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:36:36,055::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:36:36,056::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:36:36,056::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:36:36,057::task::495::TaskManager.Task::(_debug) Task f343aa08-eebc-4b8c-8d36-574dae964521: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:36:36,058::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:36:36,286::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0889865 s, 11.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:36:38,301::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:36:38,835::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0959053 s, 10.7 MB/s\n'; <rc> = 0
>Thread-15916758::INFO::2013-11-20 04:36:39,870::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916758::DEBUG::2013-11-20 04:36:39,871::task::495::TaskManager.Task::(_debug) Task 28aa69be-01f7-4aaa-9d22-1e0ff917ab48: moving from state init -> state preparing
>Thread-15916758::DEBUG::2013-11-20 04:36:39,872::task::495::TaskManager.Task::(_debug) Task 28aa69be-01f7-4aaa-9d22-1e0ff917ab48: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000384092330933', 'lastCheck': 1384922191.9483011, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00209617614746', 'lastCheck': 1384922199.509306, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0115561485291', 'lastCheck': 1384922196.4780791, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000598192214966', 'lastCheck': 1384922197.9545431, 'valid': True, 'code': 0}}
>Thread-15916758::DEBUG::2013-11-20 04:36:39,873::task::495::TaskManager.Task::(_debug) Task 28aa69be-01f7-4aaa-9d22-1e0ff917ab48: moving from state preparing -> state finished
>Thread-15916758::DEBUG::2013-11-20 04:36:39,874::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916758::DEBUG::2013-11-20 04:36:39,874::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916758::DEBUG::2013-11-20 04:36:39,875::task::495::TaskManager.Task::(_debug) Task 28aa69be-01f7-4aaa-9d22-1e0ff917ab48: ref 0 aborting False
>Thread-15916758::INFO::2013-11-20 04:36:39,876::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000384092330933', 'lastCheck': 1384922191.9483011, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00209617614746', 'lastCheck': 1384922199.509306, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0115561485291', 'lastCheck': 1384922196.4780791, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000598192214966', 'lastCheck': 1384922197.9545431, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:36:40,850::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 04:36:41,077::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:36:41,080::task::495::TaskManager.Task::(_debug) Task de4d5ffe-dd86-424b-a17e-dde6e8a614d2: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:36:41,084::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e9b48463-a3ad-4624-a96e-af29bf3cca0c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:36:41,085::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:36:41,086::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:36:41,087::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e9b48463-a3ad-4624-a96e-af29bf3cca0c`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:36:41,089::task::495::TaskManager.Task::(_debug) Task de4d5ffe-dd86-424b-a17e-dde6e8a614d2: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:36:41,089::task::495::TaskManager.Task::(_debug) Task de4d5ffe-dd86-424b-a17e-dde6e8a614d2: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:36:41,090::task::495::TaskManager.Task::(_debug) Task de4d5ffe-dd86-424b-a17e-dde6e8a614d2: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:36:41,091::task::495::TaskManager.Task::(_debug) Task de4d5ffe-dd86-424b-a17e-dde6e8a614d2: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:36:41,092::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:36:41,093::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:36:41,093::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:36:41,094::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:36:41,095::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:36:41,095::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:36:41,096::task::495::TaskManager.Task::(_debug) Task de4d5ffe-dd86-424b-a17e-dde6e8a614d2: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:36:41,097::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:36:41,398::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0925011 s, 11.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:36:43,415::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:36:43,950::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.073742 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15916761::DEBUG::2013-11-20 04:36:44,479::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916761::INFO::2013-11-20 04:36:44,480::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916761::DEBUG::2013-11-20 04:36:44,481::task::495::TaskManager.Task::(_debug) Task b1f6ba5b-47fd-4e9f-803d-9a8e6a2071be: moving from state init -> state preparing
>Thread-15916761::DEBUG::2013-11-20 04:36:44,482::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916761::DEBUG::2013-11-20 04:36:44,482::task::495::TaskManager.Task::(_debug) Task b1f6ba5b-47fd-4e9f-803d-9a8e6a2071be: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916761::DEBUG::2013-11-20 04:36:44,483::task::495::TaskManager.Task::(_debug) Task b1f6ba5b-47fd-4e9f-803d-9a8e6a2071be: moving from state preparing -> state finished
>Thread-15916761::DEBUG::2013-11-20 04:36:44,484::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916761::DEBUG::2013-11-20 04:36:44,484::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916761::DEBUG::2013-11-20 04:36:44,485::task::495::TaskManager.Task::(_debug) Task b1f6ba5b-47fd-4e9f-803d-9a8e6a2071be: ref 0 aborting False
>Thread-15916761::INFO::2013-11-20 04:36:44,486::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916762::DEBUG::2013-11-20 04:36:44,490::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916762::INFO::2013-11-20 04:36:44,491::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916762::DEBUG::2013-11-20 04:36:44,491::task::495::TaskManager.Task::(_debug) Task df073b41-fd5f-4c14-921d-a237881010bb: moving from state init -> state preparing
>Thread-15916762::DEBUG::2013-11-20 04:36:44,493::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a1bb1ee0-3efd-4dab-b6aa-ccf176932484`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916762::DEBUG::2013-11-20 04:36:44,494::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916762::DEBUG::2013-11-20 04:36:44,494::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916762::DEBUG::2013-11-20 04:36:44,495::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a1bb1ee0-3efd-4dab-b6aa-ccf176932484`::Granted request
>Thread-15916762::DEBUG::2013-11-20 04:36:44,496::task::495::TaskManager.Task::(_debug) Task df073b41-fd5f-4c14-921d-a237881010bb: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916762::DEBUG::2013-11-20 04:36:44,496::task::495::TaskManager.Task::(_debug) Task df073b41-fd5f-4c14-921d-a237881010bb: ref 1 aborting False
>Thread-15916762::WARNING::2013-11-20 04:36:44,498::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916762::DEBUG::2013-11-20 04:36:44,500::task::495::TaskManager.Task::(_debug) Task df073b41-fd5f-4c14-921d-a237881010bb: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916762::DEBUG::2013-11-20 04:36:44,501::task::495::TaskManager.Task::(_debug) Task df073b41-fd5f-4c14-921d-a237881010bb: moving from state preparing -> state finished
>Thread-15916762::DEBUG::2013-11-20 04:36:44,501::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916762::DEBUG::2013-11-20 04:36:44,502::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916762::DEBUG::2013-11-20 04:36:44,504::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916762::DEBUG::2013-11-20 04:36:44,504::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916762::DEBUG::2013-11-20 04:36:44,505::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916762::DEBUG::2013-11-20 04:36:44,506::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916762::DEBUG::2013-11-20 04:36:44,506::task::495::TaskManager.Task::(_debug) Task df073b41-fd5f-4c14-921d-a237881010bb: ref 0 aborting False
>Thread-15916762::INFO::2013-11-20 04:36:44,508::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:36:45,965::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:36:46,555::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.088993 s, 11.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:36:48,567::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:36:49,094::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0731074 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:36:51,110::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:36:51,683::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0816918 s, 12.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:36:53,699::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:36:53,925::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:36:53,927::task::495::TaskManager.Task::(_debug) Task ddc1ddab-e8c4-4fa7-b8be-12b7c51b4b8a: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:36:53,930::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f7463239-d53b-4122-9ab6-6c4bb914d2c0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:36:53,932::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:36:53,934::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:36:53,935::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f7463239-d53b-4122-9ab6-6c4bb914d2c0`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:36:53,938::task::495::TaskManager.Task::(_debug) Task ddc1ddab-e8c4-4fa7-b8be-12b7c51b4b8a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:36:53,939::task::495::TaskManager.Task::(_debug) Task ddc1ddab-e8c4-4fa7-b8be-12b7c51b4b8a: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:36:53,940::task::495::TaskManager.Task::(_debug) Task ddc1ddab-e8c4-4fa7-b8be-12b7c51b4b8a: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:36:53,941::task::495::TaskManager.Task::(_debug) Task ddc1ddab-e8c4-4fa7-b8be-12b7c51b4b8a: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:36:53,942::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:36:53,943::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:36:53,945::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:36:53,946::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:36:53,947::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:36:53,947::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:36:53,949::task::495::TaskManager.Task::(_debug) Task ddc1ddab-e8c4-4fa7-b8be-12b7c51b4b8a: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:36:53,950::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:36:54,245::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0712734 s, 14.4 MB/s\n'; <rc> = 0
>Thread-15916766::DEBUG::2013-11-20 04:36:55,159::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916766::INFO::2013-11-20 04:36:55,161::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916766::DEBUG::2013-11-20 04:36:55,161::task::495::TaskManager.Task::(_debug) Task 162ac9f0-6b4c-4e9a-810a-4ab219cbc52f: moving from state init -> state preparing
>Thread-15916766::DEBUG::2013-11-20 04:36:55,162::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916766::DEBUG::2013-11-20 04:36:55,163::task::495::TaskManager.Task::(_debug) Task 162ac9f0-6b4c-4e9a-810a-4ab219cbc52f: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916766::DEBUG::2013-11-20 04:36:55,163::task::495::TaskManager.Task::(_debug) Task 162ac9f0-6b4c-4e9a-810a-4ab219cbc52f: moving from state preparing -> state finished
>Thread-15916766::DEBUG::2013-11-20 04:36:55,164::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916766::DEBUG::2013-11-20 04:36:55,165::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916766::DEBUG::2013-11-20 04:36:55,165::task::495::TaskManager.Task::(_debug) Task 162ac9f0-6b4c-4e9a-810a-4ab219cbc52f: ref 0 aborting False
>Thread-15916766::INFO::2013-11-20 04:36:55,166::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916767::DEBUG::2013-11-20 04:36:55,170::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916767::INFO::2013-11-20 04:36:55,171::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916767::DEBUG::2013-11-20 04:36:55,172::task::495::TaskManager.Task::(_debug) Task 8604f00a-108c-4212-886a-3e4f8e2f30a3: moving from state init -> state preparing
>Thread-15916767::DEBUG::2013-11-20 04:36:55,173::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`12b4862e-3b95-41b8-af6e-1a9717c66e15`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916767::DEBUG::2013-11-20 04:36:55,174::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916767::DEBUG::2013-11-20 04:36:55,174::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916767::DEBUG::2013-11-20 04:36:55,175::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`12b4862e-3b95-41b8-af6e-1a9717c66e15`::Granted request
>Thread-15916767::DEBUG::2013-11-20 04:36:55,176::task::495::TaskManager.Task::(_debug) Task 8604f00a-108c-4212-886a-3e4f8e2f30a3: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916767::DEBUG::2013-11-20 04:36:55,176::task::495::TaskManager.Task::(_debug) Task 8604f00a-108c-4212-886a-3e4f8e2f30a3: ref 1 aborting False
>Thread-15916767::WARNING::2013-11-20 04:36:55,178::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916767::DEBUG::2013-11-20 04:36:55,179::task::495::TaskManager.Task::(_debug) Task 8604f00a-108c-4212-886a-3e4f8e2f30a3: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916767::DEBUG::2013-11-20 04:36:55,180::task::495::TaskManager.Task::(_debug) Task 8604f00a-108c-4212-886a-3e4f8e2f30a3: moving from state preparing -> state finished
>Thread-15916767::DEBUG::2013-11-20 04:36:55,181::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916767::DEBUG::2013-11-20 04:36:55,181::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916767::DEBUG::2013-11-20 04:36:55,182::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916767::DEBUG::2013-11-20 04:36:55,182::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916767::DEBUG::2013-11-20 04:36:55,183::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916767::DEBUG::2013-11-20 04:36:55,183::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916767::DEBUG::2013-11-20 04:36:55,184::task::495::TaskManager.Task::(_debug) Task 8604f00a-108c-4212-886a-3e4f8e2f30a3: ref 0 aborting False
>Thread-15916767::INFO::2013-11-20 04:36:55,185::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:36:56,261::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915918::INFO::2013-11-20 04:36:56,493::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:36:56,497::task::495::TaskManager.Task::(_debug) Task 8fb56dd3-5841-4cb0-b82a-bd4b2e6544dd: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:36:56,501::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a270ce62-d114-4cc5-8156-7bdc5ec46744`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:36:56,503::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:36:56,506::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:36:56,507::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a270ce62-d114-4cc5-8156-7bdc5ec46744`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:36:56,512::task::495::TaskManager.Task::(_debug) Task 8fb56dd3-5841-4cb0-b82a-bd4b2e6544dd: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:36:56,513::task::495::TaskManager.Task::(_debug) Task 8fb56dd3-5841-4cb0-b82a-bd4b2e6544dd: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:36:56,514::task::495::TaskManager.Task::(_debug) Task 8fb56dd3-5841-4cb0-b82a-bd4b2e6544dd: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:36:56,516::task::495::TaskManager.Task::(_debug) Task 8fb56dd3-5841-4cb0-b82a-bd4b2e6544dd: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:36:56,518::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:36:56,521::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:36:56,524::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:36:56,525::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:36:56,526::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:36:56,526::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:36:56,527::task::495::TaskManager.Task::(_debug) Task 8fb56dd3-5841-4cb0-b82a-bd4b2e6544dd: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:36:56,528::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15916768::INFO::2013-11-20 04:36:56,602::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916768::DEBUG::2013-11-20 04:36:56,603::task::495::TaskManager.Task::(_debug) Task 3edf6a81-8212-43d3-a0dc-579433aaecee: moving from state init -> state preparing
>Thread-15916768::DEBUG::2013-11-20 04:36:56,604::task::495::TaskManager.Task::(_debug) Task 3edf6a81-8212-43d3-a0dc-579433aaecee: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000307083129883', 'lastCheck': 1384922212.3487711, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0180110931396', 'lastCheck': 1384922209.7855811, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00381708145142', 'lastCheck': 1384922206.6305349, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000377893447876', 'lastCheck': 1384922208.1989291, 'valid': True, 'code': 0}}
>Thread-15916768::DEBUG::2013-11-20 04:36:56,605::task::495::TaskManager.Task::(_debug) Task 3edf6a81-8212-43d3-a0dc-579433aaecee: moving from state preparing -> state finished
>Thread-15916768::DEBUG::2013-11-20 04:36:56,606::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916768::DEBUG::2013-11-20 04:36:56,606::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916768::DEBUG::2013-11-20 04:36:56,607::task::495::TaskManager.Task::(_debug) Task 3edf6a81-8212-43d3-a0dc-579433aaecee: ref 0 aborting False
>Thread-15916768::INFO::2013-11-20 04:36:56,608::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000307083129883', 'lastCheck': 1384922212.3487711, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0180110931396', 'lastCheck': 1384922209.7855811, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00381708145142', 'lastCheck': 1384922206.6305349, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000377893447876', 'lastCheck': 1384922208.1989291, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:36:56,819::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0920061 s, 11.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:36:58,834::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:36:59,376::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.082739 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:37:01,392::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915863::INFO::2013-11-20 04:37:01,605::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:37:01,608::task::495::TaskManager.Task::(_debug) Task 908969d0-215d-491f-be22-5ba048432f02: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:37:01,611::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a283a6e4-a5c1-4e9f-9d05-c20cbf6f76cd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:37:01,612::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:37:01,613::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:37:01,613::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a283a6e4-a5c1-4e9f-9d05-c20cbf6f76cd`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:37:01,614::task::495::TaskManager.Task::(_debug) Task 908969d0-215d-491f-be22-5ba048432f02: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:37:01,615::task::495::TaskManager.Task::(_debug) Task 908969d0-215d-491f-be22-5ba048432f02: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:37:01,616::task::495::TaskManager.Task::(_debug) Task 908969d0-215d-491f-be22-5ba048432f02: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:37:01,617::task::495::TaskManager.Task::(_debug) Task 908969d0-215d-491f-be22-5ba048432f02: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:37:01,617::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:37:01,619::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:37:01,620::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:37:01,621::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:37:01,621::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:37:01,622::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:37:01,622::task::495::TaskManager.Task::(_debug) Task 908969d0-215d-491f-be22-5ba048432f02: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:37:01,623::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:37:01,837::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.050998 s, 20.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:37:03,846::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:37:04,425::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0919172 s, 11.1 MB/s\n'; <rc> = 0
>Thread-15916771::DEBUG::2013-11-20 04:37:05,703::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916771::INFO::2013-11-20 04:37:05,706::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916771::DEBUG::2013-11-20 04:37:05,707::task::495::TaskManager.Task::(_debug) Task fae861a3-3ccb-4fe7-a01b-815b185718e7: moving from state init -> state preparing
>Thread-15916771::DEBUG::2013-11-20 04:37:05,708::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916771::DEBUG::2013-11-20 04:37:05,709::task::495::TaskManager.Task::(_debug) Task fae861a3-3ccb-4fe7-a01b-815b185718e7: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916771::DEBUG::2013-11-20 04:37:05,709::task::495::TaskManager.Task::(_debug) Task fae861a3-3ccb-4fe7-a01b-815b185718e7: moving from state preparing -> state finished
>Thread-15916771::DEBUG::2013-11-20 04:37:05,710::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916771::DEBUG::2013-11-20 04:37:05,711::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916771::DEBUG::2013-11-20 04:37:05,711::task::495::TaskManager.Task::(_debug) Task fae861a3-3ccb-4fe7-a01b-815b185718e7: ref 0 aborting False
>Thread-15916771::INFO::2013-11-20 04:37:05,712::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916772::DEBUG::2013-11-20 04:37:05,716::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916772::INFO::2013-11-20 04:37:05,718::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916772::DEBUG::2013-11-20 04:37:05,718::task::495::TaskManager.Task::(_debug) Task fbb0ae4c-f05d-43da-8104-55ea4c0d86b5: moving from state init -> state preparing
>Thread-15916772::DEBUG::2013-11-20 04:37:05,720::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b767b879-65e5-4128-8bce-49486b54af0e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916772::DEBUG::2013-11-20 04:37:05,720::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916772::DEBUG::2013-11-20 04:37:05,721::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916772::DEBUG::2013-11-20 04:37:05,722::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b767b879-65e5-4128-8bce-49486b54af0e`::Granted request
>Thread-15916772::DEBUG::2013-11-20 04:37:05,723::task::495::TaskManager.Task::(_debug) Task fbb0ae4c-f05d-43da-8104-55ea4c0d86b5: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916772::DEBUG::2013-11-20 04:37:05,724::task::495::TaskManager.Task::(_debug) Task fbb0ae4c-f05d-43da-8104-55ea4c0d86b5: ref 1 aborting False
>Thread-15916772::WARNING::2013-11-20 04:37:05,726::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916772::DEBUG::2013-11-20 04:37:05,727::task::495::TaskManager.Task::(_debug) Task fbb0ae4c-f05d-43da-8104-55ea4c0d86b5: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916772::DEBUG::2013-11-20 04:37:05,728::task::495::TaskManager.Task::(_debug) Task fbb0ae4c-f05d-43da-8104-55ea4c0d86b5: moving from state preparing -> state finished
>Thread-15916772::DEBUG::2013-11-20 04:37:05,729::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916772::DEBUG::2013-11-20 04:37:05,730::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916772::DEBUG::2013-11-20 04:37:05,731::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916772::DEBUG::2013-11-20 04:37:05,731::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916772::DEBUG::2013-11-20 04:37:05,732::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916772::DEBUG::2013-11-20 04:37:05,732::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916772::DEBUG::2013-11-20 04:37:05,733::task::495::TaskManager.Task::(_debug) Task fbb0ae4c-f05d-43da-8104-55ea4c0d86b5: ref 0 aborting False
>Thread-15916772::INFO::2013-11-20 04:37:05,734::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:37:06,440::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 04:37:06,648::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:37:06,650::task::495::TaskManager.Task::(_debug) Task e119bcde-1272-4ebb-aa53-2508214e2f1f: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:37:06,654::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8ac2fd88-638b-4d66-b1d1-59df44a0065b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:37:06,656::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:37:06,658::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:37:06,659::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8ac2fd88-638b-4d66-b1d1-59df44a0065b`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:37:06,660::task::495::TaskManager.Task::(_debug) Task e119bcde-1272-4ebb-aa53-2508214e2f1f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:37:06,661::task::495::TaskManager.Task::(_debug) Task e119bcde-1272-4ebb-aa53-2508214e2f1f: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:37:06,661::task::495::TaskManager.Task::(_debug) Task e119bcde-1272-4ebb-aa53-2508214e2f1f: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:37:06,662::task::495::TaskManager.Task::(_debug) Task e119bcde-1272-4ebb-aa53-2508214e2f1f: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:37:06,663::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:37:06,663::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:37:06,664::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:37:06,664::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:37:06,665::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:37:06,665::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:37:06,666::task::495::TaskManager.Task::(_debug) Task e119bcde-1272-4ebb-aa53-2508214e2f1f: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:37:06,667::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:37:06,975::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0779951 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:37:08,991::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:37:09,569::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0826451 s, 12.4 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 04:37:10,158::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:37:10,159::task::495::TaskManager.Task::(_debug) Task d9c6e6ac-4b69-4491-b7f2-ee97b3520197: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:37:10,160::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`12412eb0-ff52-4fe2-aaa5-6cad477c1aef`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:37:10,161::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:37:10,162::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:37:10,163::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`12412eb0-ff52-4fe2-aaa5-6cad477c1aef`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:37:10,164::task::495::TaskManager.Task::(_debug) Task d9c6e6ac-4b69-4491-b7f2-ee97b3520197: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:37:10,164::task::495::TaskManager.Task::(_debug) Task d9c6e6ac-4b69-4491-b7f2-ee97b3520197: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:37:10,165::task::495::TaskManager.Task::(_debug) Task d9c6e6ac-4b69-4491-b7f2-ee97b3520197: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:37:10,166::task::495::TaskManager.Task::(_debug) Task d9c6e6ac-4b69-4491-b7f2-ee97b3520197: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:37:10,167::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:37:10,167::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:37:10,168::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:37:10,169::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:37:10,169::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:37:10,170::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:37:10,170::task::495::TaskManager.Task::(_debug) Task d9c6e6ac-4b69-4491-b7f2-ee97b3520197: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:37:10,171::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:37:10,172::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:37:10,172::task::495::TaskManager.Task::(_debug) Task f794cd90-7282-4be1-a5c5-64f062337cbf: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:37:10,173::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0aa91bf0-38a5-43bb-981f-ed11d3daa2f1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:37:10,174::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:37:10,175::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:37:10,175::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0aa91bf0-38a5-43bb-981f-ed11d3daa2f1`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:37:10,176::task::495::TaskManager.Task::(_debug) Task f794cd90-7282-4be1-a5c5-64f062337cbf: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:37:10,177::task::495::TaskManager.Task::(_debug) Task f794cd90-7282-4be1-a5c5-64f062337cbf: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:37:10,178::task::495::TaskManager.Task::(_debug) Task f794cd90-7282-4be1-a5c5-64f062337cbf: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:37:10,178::task::495::TaskManager.Task::(_debug) Task f794cd90-7282-4be1-a5c5-64f062337cbf: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:37:10,179::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:37:10,179::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:37:10,180::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:37:10,181::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:37:10,181::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:37:10,182::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:37:10,182::task::495::TaskManager.Task::(_debug) Task f794cd90-7282-4be1-a5c5-64f062337cbf: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:37:10,183::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:37:11,582::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:37:12,137::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0799256 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:37:14,154::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:37:14,675::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0684274 s, 15.0 MB/s\n'; <rc> = 0
>Thread-15916776::INFO::2013-11-20 04:37:15,528::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916776::DEBUG::2013-11-20 04:37:15,529::task::495::TaskManager.Task::(_debug) Task 2e0a0ef6-1c07-4a86-9ab6-d9afa336b923: moving from state init -> state preparing
>Thread-15916776::DEBUG::2013-11-20 04:37:15,530::task::495::TaskManager.Task::(_debug) Task 2e0a0ef6-1c07-4a86-9ab6-d9afa336b923: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000423192977905', 'lastCheck': 1384922232.8081889, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00640320777893', 'lastCheck': 1384922230.220892, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00148296356201', 'lastCheck': 1384922227.074533, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000423908233643', 'lastCheck': 1384922228.638247, 'valid': True, 'code': 0}}
>Thread-15916776::DEBUG::2013-11-20 04:37:15,531::task::495::TaskManager.Task::(_debug) Task 2e0a0ef6-1c07-4a86-9ab6-d9afa336b923: moving from state preparing -> state finished
>Thread-15916776::DEBUG::2013-11-20 04:37:15,532::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916776::DEBUG::2013-11-20 04:37:15,532::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916776::DEBUG::2013-11-20 04:37:15,533::task::495::TaskManager.Task::(_debug) Task 2e0a0ef6-1c07-4a86-9ab6-d9afa336b923: ref 0 aborting False
>Thread-15916776::INFO::2013-11-20 04:37:15,534::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000423192977905', 'lastCheck': 1384922232.8081889, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00640320777893', 'lastCheck': 1384922230.220892, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00148296356201', 'lastCheck': 1384922227.074533, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000423908233643', 'lastCheck': 1384922228.638247, 'valid': True, 'code': 0}}
>Thread-15916778::DEBUG::2013-11-20 04:37:16,218::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916778::INFO::2013-11-20 04:37:16,220::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916778::DEBUG::2013-11-20 04:37:16,221::task::495::TaskManager.Task::(_debug) Task 53bb2e48-442e-480f-8c21-ee6addc55a40: moving from state init -> state preparing
>Thread-15916778::DEBUG::2013-11-20 04:37:16,221::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916778::DEBUG::2013-11-20 04:37:16,222::task::495::TaskManager.Task::(_debug) Task 53bb2e48-442e-480f-8c21-ee6addc55a40: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916778::DEBUG::2013-11-20 04:37:16,222::task::495::TaskManager.Task::(_debug) Task 53bb2e48-442e-480f-8c21-ee6addc55a40: moving from state preparing -> state finished
>Thread-15916778::DEBUG::2013-11-20 04:37:16,223::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916778::DEBUG::2013-11-20 04:37:16,224::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916778::DEBUG::2013-11-20 04:37:16,224::task::495::TaskManager.Task::(_debug) Task 53bb2e48-442e-480f-8c21-ee6addc55a40: ref 0 aborting False
>Thread-15916778::INFO::2013-11-20 04:37:16,225::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916779::DEBUG::2013-11-20 04:37:16,229::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916779::INFO::2013-11-20 04:37:16,230::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916779::DEBUG::2013-11-20 04:37:16,230::task::495::TaskManager.Task::(_debug) Task 01cceca4-bec4-4b25-929c-5938cb943f68: moving from state init -> state preparing
>Thread-15916779::DEBUG::2013-11-20 04:37:16,231::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a56379eb-63e4-412c-b0ce-7cc0780217ad`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916779::DEBUG::2013-11-20 04:37:16,232::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916779::DEBUG::2013-11-20 04:37:16,233::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916779::DEBUG::2013-11-20 04:37:16,233::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a56379eb-63e4-412c-b0ce-7cc0780217ad`::Granted request
>Thread-15916779::DEBUG::2013-11-20 04:37:16,234::task::495::TaskManager.Task::(_debug) Task 01cceca4-bec4-4b25-929c-5938cb943f68: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916779::DEBUG::2013-11-20 04:37:16,235::task::495::TaskManager.Task::(_debug) Task 01cceca4-bec4-4b25-929c-5938cb943f68: ref 1 aborting False
>Thread-15916779::WARNING::2013-11-20 04:37:16,237::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916779::DEBUG::2013-11-20 04:37:16,238::task::495::TaskManager.Task::(_debug) Task 01cceca4-bec4-4b25-929c-5938cb943f68: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916779::DEBUG::2013-11-20 04:37:16,239::task::495::TaskManager.Task::(_debug) Task 01cceca4-bec4-4b25-929c-5938cb943f68: moving from state preparing -> state finished
>Thread-15916779::DEBUG::2013-11-20 04:37:16,240::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916779::DEBUG::2013-11-20 04:37:16,240::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916779::DEBUG::2013-11-20 04:37:16,241::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916779::DEBUG::2013-11-20 04:37:16,242::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916779::DEBUG::2013-11-20 04:37:16,242::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916779::DEBUG::2013-11-20 04:37:16,243::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916779::DEBUG::2013-11-20 04:37:16,244::task::495::TaskManager.Task::(_debug) Task 01cceca4-bec4-4b25-929c-5938cb943f68: ref 0 aborting False
>Thread-15916779::INFO::2013-11-20 04:37:16,244::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:37:16,693::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 04:37:16,895::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:37:16,899::task::495::TaskManager.Task::(_debug) Task 0c6a09f5-544c-487b-99be-7c71463de4fc: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:37:16,904::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0c51f3c9-7d24-455a-87b0-7bdfd7024b43`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:37:16,905::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:37:16,906::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:37:16,907::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0c51f3c9-7d24-455a-87b0-7bdfd7024b43`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:37:16,908::task::495::TaskManager.Task::(_debug) Task 0c6a09f5-544c-487b-99be-7c71463de4fc: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:37:16,908::task::495::TaskManager.Task::(_debug) Task 0c6a09f5-544c-487b-99be-7c71463de4fc: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:37:16,909::task::495::TaskManager.Task::(_debug) Task 0c6a09f5-544c-487b-99be-7c71463de4fc: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:37:16,910::task::495::TaskManager.Task::(_debug) Task 0c6a09f5-544c-487b-99be-7c71463de4fc: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:37:16,910::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:37:16,911::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:37:16,912::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:37:16,912::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:37:16,913::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:37:16,913::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:37:16,914::task::495::TaskManager.Task::(_debug) Task 0c6a09f5-544c-487b-99be-7c71463de4fc: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:37:16,915::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:37:17,234::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0848929 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:37:19,249::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:37:19,813::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0805823 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:37:21,824::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:37:22,362::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0718032 s, 14.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:37:24,378::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 04:37:24,948::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:37:24,951::task::495::TaskManager.Task::(_debug) Task 7252c220-f7dc-48cb-bf5c-a2621821b325: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:37:24,955::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fdd95f40-271d-4d59-b9c9-10cd2a356de9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:37:24,956::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:37:24,959::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:37:24,960::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fdd95f40-271d-4d59-b9c9-10cd2a356de9`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:37:24,963::task::495::TaskManager.Task::(_debug) Task 7252c220-f7dc-48cb-bf5c-a2621821b325: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:37:24,964::task::495::TaskManager.Task::(_debug) Task 7252c220-f7dc-48cb-bf5c-a2621821b325: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:37:24,965::task::495::TaskManager.Task::(_debug) Task 7252c220-f7dc-48cb-bf5c-a2621821b325: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:37:24,967::task::495::TaskManager.Task::(_debug) Task 7252c220-f7dc-48cb-bf5c-a2621821b325: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:37:24,968::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:37:24,969::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:37:24,970::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:37:24,971::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:37:24,972::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:37:24,973::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:37:24,975::task::495::TaskManager.Task::(_debug) Task 7252c220-f7dc-48cb-bf5c-a2621821b325: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:37:24,975::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:37:24,981::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0851524 s, 12.0 MB/s\n'; <rc> = 0
>Thread-15916783::DEBUG::2013-11-20 04:37:26,625::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916783::INFO::2013-11-20 04:37:26,626::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916783::DEBUG::2013-11-20 04:37:26,627::task::495::TaskManager.Task::(_debug) Task d1131dc3-73f5-4dd6-b9bf-201da3482197: moving from state init -> state preparing
>Thread-15916783::DEBUG::2013-11-20 04:37:26,628::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916783::DEBUG::2013-11-20 04:37:26,629::task::495::TaskManager.Task::(_debug) Task d1131dc3-73f5-4dd6-b9bf-201da3482197: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916783::DEBUG::2013-11-20 04:37:26,629::task::495::TaskManager.Task::(_debug) Task d1131dc3-73f5-4dd6-b9bf-201da3482197: moving from state preparing -> state finished
>Thread-15916783::DEBUG::2013-11-20 04:37:26,630::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916783::DEBUG::2013-11-20 04:37:26,630::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916783::DEBUG::2013-11-20 04:37:26,631::task::495::TaskManager.Task::(_debug) Task d1131dc3-73f5-4dd6-b9bf-201da3482197: ref 0 aborting False
>Thread-15916783::INFO::2013-11-20 04:37:26,632::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916784::DEBUG::2013-11-20 04:37:26,637::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916784::INFO::2013-11-20 04:37:26,639::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916784::DEBUG::2013-11-20 04:37:26,639::task::495::TaskManager.Task::(_debug) Task c77f3d0f-5c92-4933-ab1a-5265684a1dbb: moving from state init -> state preparing
>Thread-15916784::DEBUG::2013-11-20 04:37:26,640::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`49399913-f9cd-4c28-ae98-864f838ca93a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916784::DEBUG::2013-11-20 04:37:26,641::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916784::DEBUG::2013-11-20 04:37:26,642::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916784::DEBUG::2013-11-20 04:37:26,642::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`49399913-f9cd-4c28-ae98-864f838ca93a`::Granted request
>Thread-15916784::DEBUG::2013-11-20 04:37:26,643::task::495::TaskManager.Task::(_debug) Task c77f3d0f-5c92-4933-ab1a-5265684a1dbb: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916784::DEBUG::2013-11-20 04:37:26,644::task::495::TaskManager.Task::(_debug) Task c77f3d0f-5c92-4933-ab1a-5265684a1dbb: ref 1 aborting False
>Thread-15916784::WARNING::2013-11-20 04:37:26,647::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916784::DEBUG::2013-11-20 04:37:26,648::task::495::TaskManager.Task::(_debug) Task c77f3d0f-5c92-4933-ab1a-5265684a1dbb: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916784::DEBUG::2013-11-20 04:37:26,649::task::495::TaskManager.Task::(_debug) Task c77f3d0f-5c92-4933-ab1a-5265684a1dbb: moving from state preparing -> state finished
>Thread-15916784::DEBUG::2013-11-20 04:37:26,650::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916784::DEBUG::2013-11-20 04:37:26,650::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916784::DEBUG::2013-11-20 04:37:26,651::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916784::DEBUG::2013-11-20 04:37:26,651::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916784::DEBUG::2013-11-20 04:37:26,652::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916784::DEBUG::2013-11-20 04:37:26,652::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916784::DEBUG::2013-11-20 04:37:26,653::task::495::TaskManager.Task::(_debug) Task c77f3d0f-5c92-4933-ab1a-5265684a1dbb: ref 0 aborting False
>Thread-15916784::INFO::2013-11-20 04:37:26,654::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:37:26,998::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 04:37:27,211::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:37:27,214::task::495::TaskManager.Task::(_debug) Task 9a124d8a-67e7-4a31-aed7-0b0dfb4736e1: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:37:27,221::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f8adc822-7512-430b-bf81-3057246c4329`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:37:27,223::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:37:27,225::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:37:27,226::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f8adc822-7512-430b-bf81-3057246c4329`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:37:27,228::task::495::TaskManager.Task::(_debug) Task 9a124d8a-67e7-4a31-aed7-0b0dfb4736e1: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:37:27,229::task::495::TaskManager.Task::(_debug) Task 9a124d8a-67e7-4a31-aed7-0b0dfb4736e1: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:37:27,229::task::495::TaskManager.Task::(_debug) Task 9a124d8a-67e7-4a31-aed7-0b0dfb4736e1: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:37:27,230::task::495::TaskManager.Task::(_debug) Task 9a124d8a-67e7-4a31-aed7-0b0dfb4736e1: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:37:27,231::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:37:27,231::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:37:27,232::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:37:27,233::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:37:27,233::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:37:27,234::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:37:27,234::task::495::TaskManager.Task::(_debug) Task 9a124d8a-67e7-4a31-aed7-0b0dfb4736e1: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:37:27,235::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:37:27,574::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0826606 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:37:29,591::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:37:30,126::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0605958 s, 16.9 MB/s\n'; <rc> = 0
>Thread-15916786::INFO::2013-11-20 04:37:31,571::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916786::DEBUG::2013-11-20 04:37:31,572::task::495::TaskManager.Task::(_debug) Task 6dd03a51-1f24-43e4-88e5-f50c1518f75a: moving from state init -> state preparing
>Thread-15916786::DEBUG::2013-11-20 04:37:31,573::task::495::TaskManager.Task::(_debug) Task 6dd03a51-1f24-43e4-88e5-f50c1518f75a: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000479936599731', 'lastCheck': 1384922243.0511069, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0231740474701', 'lastCheck': 1384922250.8512061, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00127291679382', 'lastCheck': 1384922247.6035421, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000456094741821', 'lastCheck': 1384922249.2016389, 'valid': True, 'code': 0}}
>Thread-15916786::DEBUG::2013-11-20 04:37:31,574::task::495::TaskManager.Task::(_debug) Task 6dd03a51-1f24-43e4-88e5-f50c1518f75a: moving from state preparing -> state finished
>Thread-15916786::DEBUG::2013-11-20 04:37:31,574::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916786::DEBUG::2013-11-20 04:37:31,575::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916786::DEBUG::2013-11-20 04:37:31,576::task::495::TaskManager.Task::(_debug) Task 6dd03a51-1f24-43e4-88e5-f50c1518f75a: ref 0 aborting False
>Thread-15916786::INFO::2013-11-20 04:37:31,576::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000479936599731', 'lastCheck': 1384922243.0511069, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0231740474701', 'lastCheck': 1384922250.8512061, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00127291679382', 'lastCheck': 1384922247.6035421, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000456094741821', 'lastCheck': 1384922249.2016389, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:37:32,135::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:37:32,547::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0494343 s, 20.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:37:34,559::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:37:35,124::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0759732 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:37:37,139::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 04:37:37,360::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15913546::INFO::2013-11-20 04:37:37,362::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15914519::DEBUG::2013-11-20 04:37:37,365::task::495::TaskManager.Task::(_debug) Task 34e712ac-a64a-42fb-93b1-6685c614240b: moving from state init -> state preparing
>Thread-15916789::DEBUG::2013-11-20 04:37:37,367::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913546::DEBUG::2013-11-20 04:37:37,368::task::495::TaskManager.Task::(_debug) Task 4c9b66e8-fbc4-4af3-81d1-3cf6d93af8ad: moving from state init -> state preparing
>Thread-15916789::INFO::2013-11-20 04:37:37,374::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914519::DEBUG::2013-11-20 04:37:37,375::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3e925e00-0af3-47fc-9133-d26391f6b561`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:37:37,378::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6432fbaa-de81-4541-9e8d-2d64f393e7b4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15916789::DEBUG::2013-11-20 04:37:37,378::task::495::TaskManager.Task::(_debug) Task ba959a23-9c08-4931-9ccc-dfba019a480b: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:37:37,380::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:37:37,381::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15916789::DEBUG::2013-11-20 04:37:37,382::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916789::DEBUG::2013-11-20 04:37:37,384::task::495::TaskManager.Task::(_debug) Task ba959a23-9c08-4931-9ccc-dfba019a480b: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914519::DEBUG::2013-11-20 04:37:37,385::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15916789::DEBUG::2013-11-20 04:37:37,386::task::495::TaskManager.Task::(_debug) Task ba959a23-9c08-4931-9ccc-dfba019a480b: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:37:37,386::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3e925e00-0af3-47fc-9133-d26391f6b561`::Granted request
>Thread-15916789::DEBUG::2013-11-20 04:37:37,387::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15913546::DEBUG::2013-11-20 04:37:37,388::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:37:37,389::task::495::TaskManager.Task::(_debug) Task 34e712ac-a64a-42fb-93b1-6685c614240b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15916789::DEBUG::2013-11-20 04:37:37,389::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:37:37,390::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6432fbaa-de81-4541-9e8d-2d64f393e7b4`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:37:37,391::task::495::TaskManager.Task::(_debug) Task 34e712ac-a64a-42fb-93b1-6685c614240b: ref 1 aborting False
>Thread-15916789::DEBUG::2013-11-20 04:37:37,392::task::495::TaskManager.Task::(_debug) Task ba959a23-9c08-4931-9ccc-dfba019a480b: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:37:37,394::task::495::TaskManager.Task::(_debug) Task 4c9b66e8-fbc4-4af3-81d1-3cf6d93af8ad: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15914519::DEBUG::2013-11-20 04:37:37,395::task::495::TaskManager.Task::(_debug) Task 34e712ac-a64a-42fb-93b1-6685c614240b: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15916789::INFO::2013-11-20 04:37:37,395::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913546::DEBUG::2013-11-20 04:37:37,396::task::495::TaskManager.Task::(_debug) Task 4c9b66e8-fbc4-4af3-81d1-3cf6d93af8ad: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:37:37,396::task::495::TaskManager.Task::(_debug) Task 34e712ac-a64a-42fb-93b1-6685c614240b: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:37:37,398::task::495::TaskManager.Task::(_debug) Task 4c9b66e8-fbc4-4af3-81d1-3cf6d93af8ad: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:37:37,399::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:37:37,401::task::495::TaskManager.Task::(_debug) Task 4c9b66e8-fbc4-4af3-81d1-3cf6d93af8ad: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:37:37,401::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:37:37,403::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:37:37,404::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913546::DEBUG::2013-11-20 04:37:37,405::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:37:37,405::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15916790::DEBUG::2013-11-20 04:37:37,407::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913546::DEBUG::2013-11-20 04:37:37,408::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15914519::DEBUG::2013-11-20 04:37:37,409::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15916790::INFO::2013-11-20 04:37:37,409::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914519::DEBUG::2013-11-20 04:37:37,411::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15916790::DEBUG::2013-11-20 04:37:37,412::task::495::TaskManager.Task::(_debug) Task 7dd0c9ea-0fb7-4fb6-bb3f-fe69421bf7c0: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:37:37,412::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:37:37,414::task::495::TaskManager.Task::(_debug) Task 34e712ac-a64a-42fb-93b1-6685c614240b: ref 0 aborting False
>Thread-15916790::DEBUG::2013-11-20 04:37:37,415::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ded42de3-380b-49c5-92ae-294c9ca3797c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913546::DEBUG::2013-11-20 04:37:37,415::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15914519::INFO::2013-11-20 04:37:37,416::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15916790::DEBUG::2013-11-20 04:37:37,416::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:37:37,417::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15916790::DEBUG::2013-11-20 04:37:37,419::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:37:37,420::task::495::TaskManager.Task::(_debug) Task 4c9b66e8-fbc4-4af3-81d1-3cf6d93af8ad: ref 0 aborting False
>Thread-15916790::DEBUG::2013-11-20 04:37:37,420::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ded42de3-380b-49c5-92ae-294c9ca3797c`::Granted request
>Thread-15913546::INFO::2013-11-20 04:37:37,422::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15916790::DEBUG::2013-11-20 04:37:37,424::task::495::TaskManager.Task::(_debug) Task 7dd0c9ea-0fb7-4fb6-bb3f-fe69421bf7c0: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913546::INFO::2013-11-20 04:37:37,425::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15916790::DEBUG::2013-11-20 04:37:37,425::task::495::TaskManager.Task::(_debug) Task 7dd0c9ea-0fb7-4fb6-bb3f-fe69421bf7c0: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:37:37,426::task::495::TaskManager.Task::(_debug) Task 4911d161-d88e-42bd-bcc4-18a90d2e74be: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:37:37,431::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`44cc442d-c1da-4a99-abdf-edf7d6209e96`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15916790::WARNING::2013-11-20 04:37:37,432::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913546::DEBUG::2013-11-20 04:37:37,433::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15916790::DEBUG::2013-11-20 04:37:37,435::task::495::TaskManager.Task::(_debug) Task 7dd0c9ea-0fb7-4fb6-bb3f-fe69421bf7c0: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913546::DEBUG::2013-11-20 04:37:37,436::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15916790::DEBUG::2013-11-20 04:37:37,438::task::495::TaskManager.Task::(_debug) Task 7dd0c9ea-0fb7-4fb6-bb3f-fe69421bf7c0: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:37:37,438::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`44cc442d-c1da-4a99-abdf-edf7d6209e96`::Granted request
>Thread-15916790::DEBUG::2013-11-20 04:37:37,439::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916790::DEBUG::2013-11-20 04:37:37,441::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:37:37,446::task::495::TaskManager.Task::(_debug) Task 4911d161-d88e-42bd-bcc4-18a90d2e74be: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:37:37,447::task::495::TaskManager.Task::(_debug) Task 4911d161-d88e-42bd-bcc4-18a90d2e74be: ref 1 aborting False
>Thread-15916790::DEBUG::2013-11-20 04:37:37,448::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15913546::DEBUG::2013-11-20 04:37:37,449::task::495::TaskManager.Task::(_debug) Task 4911d161-d88e-42bd-bcc4-18a90d2e74be: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15916790::DEBUG::2013-11-20 04:37:37,450::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:37:37,450::task::495::TaskManager.Task::(_debug) Task 4911d161-d88e-42bd-bcc4-18a90d2e74be: moving from state preparing -> state finished
>Thread-15916790::DEBUG::2013-11-20 04:37:37,451::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:37:37,452::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15916790::DEBUG::2013-11-20 04:37:37,453::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:37:37,454::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916790::DEBUG::2013-11-20 04:37:37,455::task::495::TaskManager.Task::(_debug) Task 7dd0c9ea-0fb7-4fb6-bb3f-fe69421bf7c0: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:37:37,456::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15916790::INFO::2013-11-20 04:37:37,457::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913546::DEBUG::2013-11-20 04:37:37,458::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:37:37,460::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:37:37,461::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:37:37,462::task::495::TaskManager.Task::(_debug) Task 4911d161-d88e-42bd-bcc4-18a90d2e74be: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:37:37,463::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:37:37,464::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:37:37,464::task::495::TaskManager.Task::(_debug) Task 5c229d2b-6091-47af-b499-02f14a015643: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:37:37,466::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`46d2d99d-3273-4686-8f47-7bb0fe67e81a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:37:37,466::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:37:37,467::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:37:37,468::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`46d2d99d-3273-4686-8f47-7bb0fe67e81a`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:37:37,469::task::495::TaskManager.Task::(_debug) Task 5c229d2b-6091-47af-b499-02f14a015643: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:37:37,470::task::495::TaskManager.Task::(_debug) Task 5c229d2b-6091-47af-b499-02f14a015643: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:37:37,471::task::495::TaskManager.Task::(_debug) Task 5c229d2b-6091-47af-b499-02f14a015643: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:37:37,472::task::495::TaskManager.Task::(_debug) Task 5c229d2b-6091-47af-b499-02f14a015643: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:37:37,472::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:37:37,473::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:37:37,474::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:37:37,475::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:37:37,476::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:37:37,477::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:37:37,477::task::495::TaskManager.Task::(_debug) Task 5c229d2b-6091-47af-b499-02f14a015643: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:37:37,479::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:37:37,480::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:37:37,483::task::495::TaskManager.Task::(_debug) Task d44f1400-e7c5-4248-82c9-a63f6e629503: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:37:37,485::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fe060082-9422-4bd1-9283-3f3757ee5cbc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:37:37,487::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:37:37,487::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:37:37,488::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`fe060082-9422-4bd1-9283-3f3757ee5cbc`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:37:37,489::task::495::TaskManager.Task::(_debug) Task d44f1400-e7c5-4248-82c9-a63f6e629503: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:37:37,489::task::495::TaskManager.Task::(_debug) Task d44f1400-e7c5-4248-82c9-a63f6e629503: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:37:37,490::task::495::TaskManager.Task::(_debug) Task d44f1400-e7c5-4248-82c9-a63f6e629503: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:37:37,491::task::495::TaskManager.Task::(_debug) Task d44f1400-e7c5-4248-82c9-a63f6e629503: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:37:37,491::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:37:37,492::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:37:37,492::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:37:37,493::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:37:37,493::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:37:37,494::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:37:37,495::task::495::TaskManager.Task::(_debug) Task d44f1400-e7c5-4248-82c9-a63f6e629503: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:37:37,495::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:37:37,693::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0884277 s, 11.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:37:39,707::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:37:40,319::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0831575 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:37:42,334::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 04:37:42,584::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:37:42,586::task::495::TaskManager.Task::(_debug) Task 8b9976c5-9e9c-428c-94dc-c53f12dbea74: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:37:42,588::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`33e8f225-384c-4ef4-8ee4-7b3b6d573a7e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:37:42,589::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:37:42,590::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:37:42,591::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`33e8f225-384c-4ef4-8ee4-7b3b6d573a7e`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:37:42,592::task::495::TaskManager.Task::(_debug) Task 8b9976c5-9e9c-428c-94dc-c53f12dbea74: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:37:42,593::task::495::TaskManager.Task::(_debug) Task 8b9976c5-9e9c-428c-94dc-c53f12dbea74: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:37:42,593::task::495::TaskManager.Task::(_debug) Task 8b9976c5-9e9c-428c-94dc-c53f12dbea74: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:37:42,594::task::495::TaskManager.Task::(_debug) Task 8b9976c5-9e9c-428c-94dc-c53f12dbea74: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:37:42,595::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:37:42,595::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:37:42,596::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:37:42,597::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:37:42,597::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:37:42,598::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:37:42,599::task::495::TaskManager.Task::(_debug) Task 8b9976c5-9e9c-428c-94dc-c53f12dbea74: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:37:42,599::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:37:42,824::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0570341 s, 18.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:37:44,840::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:37:45,381::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0822012 s, 12.5 MB/s\n'; <rc> = 0
>Thread-15916794::INFO::2013-11-20 04:37:47,263::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916794::DEBUG::2013-11-20 04:37:47,264::task::495::TaskManager.Task::(_debug) Task 6a0d3030-b743-451b-916d-70dc1c38226c: moving from state init -> state preparing
>Thread-15916794::DEBUG::2013-11-20 04:37:47,265::task::495::TaskManager.Task::(_debug) Task 6a0d3030-b743-451b-916d-70dc1c38226c: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000416994094849', 'lastCheck': 1384922263.564683, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00191593170166', 'lastCheck': 1384922260.9408989, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00202107429504', 'lastCheck': 1384922257.8345561, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000366926193237', 'lastCheck': 1384922259.3545749, 'valid': True, 'code': 0}}
>Thread-15916794::DEBUG::2013-11-20 04:37:47,267::task::495::TaskManager.Task::(_debug) Task 6a0d3030-b743-451b-916d-70dc1c38226c: moving from state preparing -> state finished
>Thread-15916794::DEBUG::2013-11-20 04:37:47,267::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916794::DEBUG::2013-11-20 04:37:47,268::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916794::DEBUG::2013-11-20 04:37:47,269::task::495::TaskManager.Task::(_debug) Task 6a0d3030-b743-451b-916d-70dc1c38226c: ref 0 aborting False
>Thread-15916794::INFO::2013-11-20 04:37:47,270::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000416994094849', 'lastCheck': 1384922263.564683, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00191593170166', 'lastCheck': 1384922260.9408989, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00202107429504', 'lastCheck': 1384922257.8345561, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000366926193237', 'lastCheck': 1384922259.3545749, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:37:47,397::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916796::DEBUG::2013-11-20 04:37:47,642::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916796::INFO::2013-11-20 04:37:47,645::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916796::DEBUG::2013-11-20 04:37:47,645::task::495::TaskManager.Task::(_debug) Task 18f3ef10-2bc6-42a1-a7f9-e3c247bf106b: moving from state init -> state preparing
>Thread-15916796::DEBUG::2013-11-20 04:37:47,647::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916796::DEBUG::2013-11-20 04:37:47,648::task::495::TaskManager.Task::(_debug) Task 18f3ef10-2bc6-42a1-a7f9-e3c247bf106b: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916796::DEBUG::2013-11-20 04:37:47,649::task::495::TaskManager.Task::(_debug) Task 18f3ef10-2bc6-42a1-a7f9-e3c247bf106b: moving from state preparing -> state finished
>Thread-15916796::DEBUG::2013-11-20 04:37:47,650::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916796::DEBUG::2013-11-20 04:37:47,651::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916796::DEBUG::2013-11-20 04:37:47,652::task::495::TaskManager.Task::(_debug) Task 18f3ef10-2bc6-42a1-a7f9-e3c247bf106b: ref 0 aborting False
>Thread-15916796::INFO::2013-11-20 04:37:47,652::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916797::DEBUG::2013-11-20 04:37:47,658::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916797::INFO::2013-11-20 04:37:47,659::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916797::DEBUG::2013-11-20 04:37:47,659::task::495::TaskManager.Task::(_debug) Task 74406a1a-693c-456c-a1e9-abc549a83dd5: moving from state init -> state preparing
>Thread-15916797::DEBUG::2013-11-20 04:37:47,660::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6c7b8081-6e21-4ae9-9377-b7e065dd6abd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916797::DEBUG::2013-11-20 04:37:47,660::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916797::DEBUG::2013-11-20 04:37:47,661::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916797::DEBUG::2013-11-20 04:37:47,661::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6c7b8081-6e21-4ae9-9377-b7e065dd6abd`::Granted request
>Thread-15916797::DEBUG::2013-11-20 04:37:47,662::task::495::TaskManager.Task::(_debug) Task 74406a1a-693c-456c-a1e9-abc549a83dd5: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916797::DEBUG::2013-11-20 04:37:47,663::task::495::TaskManager.Task::(_debug) Task 74406a1a-693c-456c-a1e9-abc549a83dd5: ref 1 aborting False
>Thread-15916797::WARNING::2013-11-20 04:37:47,664::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916797::DEBUG::2013-11-20 04:37:47,665::task::495::TaskManager.Task::(_debug) Task 74406a1a-693c-456c-a1e9-abc549a83dd5: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916797::DEBUG::2013-11-20 04:37:47,666::task::495::TaskManager.Task::(_debug) Task 74406a1a-693c-456c-a1e9-abc549a83dd5: moving from state preparing -> state finished
>Thread-15916797::DEBUG::2013-11-20 04:37:47,666::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916797::DEBUG::2013-11-20 04:37:47,667::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916797::DEBUG::2013-11-20 04:37:47,667::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916797::DEBUG::2013-11-20 04:37:47,668::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916797::DEBUG::2013-11-20 04:37:47,668::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916797::DEBUG::2013-11-20 04:37:47,669::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916797::DEBUG::2013-11-20 04:37:47,669::task::495::TaskManager.Task::(_debug) Task 74406a1a-693c-456c-a1e9-abc549a83dd5: ref 0 aborting False
>Thread-15916797::INFO::2013-11-20 04:37:47,670::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:37:47,972::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0919363 s, 11.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:37:49,988::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:37:50,567::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0860084 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:37:52,581::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:37:53,184::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.083954 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:37:55,198::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:37:55,437::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:37:55,439::task::495::TaskManager.Task::(_debug) Task 0736bc2c-5d6b-44e2-a6ce-b40c23365bde: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:37:55,441::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7a7ee644-8e74-49e6-a3ad-8d508b9c1574`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:37:55,442::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:37:55,443::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:37:55,444::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7a7ee644-8e74-49e6-a3ad-8d508b9c1574`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:37:55,445::task::495::TaskManager.Task::(_debug) Task 0736bc2c-5d6b-44e2-a6ce-b40c23365bde: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:37:55,446::task::495::TaskManager.Task::(_debug) Task 0736bc2c-5d6b-44e2-a6ce-b40c23365bde: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:37:55,447::task::495::TaskManager.Task::(_debug) Task 0736bc2c-5d6b-44e2-a6ce-b40c23365bde: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:37:55,448::task::495::TaskManager.Task::(_debug) Task 0736bc2c-5d6b-44e2-a6ce-b40c23365bde: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:37:55,448::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:37:55,449::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:37:55,450::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:37:55,451::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:37:55,452::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:37:55,452::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:37:55,453::task::495::TaskManager.Task::(_debug) Task 0736bc2c-5d6b-44e2-a6ce-b40c23365bde: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:37:55,454::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:37:55,751::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.082232 s, 12.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:37:57,767::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915918::INFO::2013-11-20 04:37:57,980::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:37:57,983::task::495::TaskManager.Task::(_debug) Task 7a2231b0-ef3b-4a12-b9e2-cef9b3bb9e31: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:37:57,989::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d16c2d27-2f0b-4d4d-8153-d7131322c8e4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:37:57,991::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:37:57,992::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:37:57,993::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d16c2d27-2f0b-4d4d-8153-d7131322c8e4`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:37:57,994::task::495::TaskManager.Task::(_debug) Task 7a2231b0-ef3b-4a12-b9e2-cef9b3bb9e31: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:37:57,994::task::495::TaskManager.Task::(_debug) Task 7a2231b0-ef3b-4a12-b9e2-cef9b3bb9e31: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:37:57,995::task::495::TaskManager.Task::(_debug) Task 7a2231b0-ef3b-4a12-b9e2-cef9b3bb9e31: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:37:57,996::task::495::TaskManager.Task::(_debug) Task 7a2231b0-ef3b-4a12-b9e2-cef9b3bb9e31: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:37:57,997::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:37:57,997::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:37:57,998::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:37:57,999::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:37:57,999::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:37:58,000::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:37:58,001::task::495::TaskManager.Task::(_debug) Task 7a2231b0-ef3b-4a12-b9e2-cef9b3bb9e31: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:37:58,001::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:37:58,296::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0746711 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:38:00,312::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:38:00,851::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0758993 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:38:02,867::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915863::INFO::2013-11-20 04:38:03,112::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:38:03,123::task::495::TaskManager.Task::(_debug) Task c816f1d4-d1f4-4e32-9512-14129a56b004: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:38:03,131::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e0381edc-9178-4b22-bc14-e38c055f4f59`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:38:03,132::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:38:03,134::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:38:03,135::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e0381edc-9178-4b22-bc14-e38c055f4f59`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:38:03,137::task::495::TaskManager.Task::(_debug) Task c816f1d4-d1f4-4e32-9512-14129a56b004: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:38:03,138::task::495::TaskManager.Task::(_debug) Task c816f1d4-d1f4-4e32-9512-14129a56b004: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:38:03,139::task::495::TaskManager.Task::(_debug) Task c816f1d4-d1f4-4e32-9512-14129a56b004: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:38:03,140::task::495::TaskManager.Task::(_debug) Task c816f1d4-d1f4-4e32-9512-14129a56b004: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:38:03,141::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:38:03,141::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:38:03,142::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:38:03,143::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:38:03,144::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:38:03,145::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:38:03,146::task::495::TaskManager.Task::(_debug) Task c816f1d4-d1f4-4e32-9512-14129a56b004: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:38:03,147::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:38:03,389::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0801834 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:38:05,404::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:38:05,949::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0848312 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:38:07,965::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 04:38:08,185::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:38:08,187::task::495::TaskManager.Task::(_debug) Task cae655f0-cbb7-4ed7-9861-8bd3922d288e: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:38:08,192::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fc61a09e-b74a-4637-915e-b3d08d9706cd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:38:08,193::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:38:08,196::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:38:08,197::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fc61a09e-b74a-4637-915e-b3d08d9706cd`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:38:08,199::task::495::TaskManager.Task::(_debug) Task cae655f0-cbb7-4ed7-9861-8bd3922d288e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:38:08,200::task::495::TaskManager.Task::(_debug) Task cae655f0-cbb7-4ed7-9861-8bd3922d288e: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:38:08,201::task::495::TaskManager.Task::(_debug) Task cae655f0-cbb7-4ed7-9861-8bd3922d288e: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:38:08,201::task::495::TaskManager.Task::(_debug) Task cae655f0-cbb7-4ed7-9861-8bd3922d288e: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:38:08,202::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:38:08,203::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:38:08,203::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:38:08,204::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:38:08,205::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:38:08,205::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:38:08,208::task::495::TaskManager.Task::(_debug) Task cae655f0-cbb7-4ed7-9861-8bd3922d288e: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:38:08,209::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:38:08,493::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0767782 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:38:10,508::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:38:11,077::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0785739 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 04:38:11,697::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:38:11,698::task::495::TaskManager.Task::(_debug) Task fd034970-290a-42e0-bb2c-7d88489e8a75: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:38:11,700::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9ae4d139-8393-4a16-8b8e-b91d9cc1c7bb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:38:11,700::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:38:11,701::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:38:11,702::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9ae4d139-8393-4a16-8b8e-b91d9cc1c7bb`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:38:11,703::task::495::TaskManager.Task::(_debug) Task fd034970-290a-42e0-bb2c-7d88489e8a75: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:38:11,704::task::495::TaskManager.Task::(_debug) Task fd034970-290a-42e0-bb2c-7d88489e8a75: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:38:11,704::task::495::TaskManager.Task::(_debug) Task fd034970-290a-42e0-bb2c-7d88489e8a75: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:38:11,705::task::495::TaskManager.Task::(_debug) Task fd034970-290a-42e0-bb2c-7d88489e8a75: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:38:11,706::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:38:11,706::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:38:11,707::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:38:11,708::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:38:11,708::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:38:11,709::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:38:11,709::task::495::TaskManager.Task::(_debug) Task fd034970-290a-42e0-bb2c-7d88489e8a75: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:38:11,710::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:38:11,711::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:38:11,711::task::495::TaskManager.Task::(_debug) Task c9ac7720-756d-4748-b76e-01cab67aff5c: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:38:11,712::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ef9655a6-38fe-4ab2-8b88-bc0322d5c50f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:38:11,713::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:38:11,713::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:38:11,714::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ef9655a6-38fe-4ab2-8b88-bc0322d5c50f`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:38:11,715::task::495::TaskManager.Task::(_debug) Task c9ac7720-756d-4748-b76e-01cab67aff5c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:38:11,715::task::495::TaskManager.Task::(_debug) Task c9ac7720-756d-4748-b76e-01cab67aff5c: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:38:11,716::task::495::TaskManager.Task::(_debug) Task c9ac7720-756d-4748-b76e-01cab67aff5c: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:38:11,717::task::495::TaskManager.Task::(_debug) Task c9ac7720-756d-4748-b76e-01cab67aff5c: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:38:11,717::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:38:11,718::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:38:11,718::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:38:11,719::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:38:11,720::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:38:11,720::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:38:11,721::task::495::TaskManager.Task::(_debug) Task c9ac7720-756d-4748-b76e-01cab67aff5c: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:38:11,721::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:38:13,094::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:38:13,662::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0805515 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:38:15,678::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:38:16,195::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0741285 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:38:18,210::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 04:38:18,436::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:38:18,441::task::495::TaskManager.Task::(_debug) Task cd4cbf2f-fc54-460e-a0ea-f43a0473362d: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:38:18,444::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`eb6ec9ca-122e-457d-b4e9-d83861af6095`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:38:18,445::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:38:18,446::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:38:18,447::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`eb6ec9ca-122e-457d-b4e9-d83861af6095`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:38:18,449::task::495::TaskManager.Task::(_debug) Task cd4cbf2f-fc54-460e-a0ea-f43a0473362d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:38:18,449::task::495::TaskManager.Task::(_debug) Task cd4cbf2f-fc54-460e-a0ea-f43a0473362d: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:38:18,450::task::495::TaskManager.Task::(_debug) Task cd4cbf2f-fc54-460e-a0ea-f43a0473362d: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:38:18,451::task::495::TaskManager.Task::(_debug) Task cd4cbf2f-fc54-460e-a0ea-f43a0473362d: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:38:18,451::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:38:18,452::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:38:18,453::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:38:18,453::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:38:18,454::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:38:18,454::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:38:18,455::task::495::TaskManager.Task::(_debug) Task cd4cbf2f-fc54-460e-a0ea-f43a0473362d: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:38:18,456::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:38:18,760::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0826694 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:38:20,776::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:38:21,321::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0974378 s, 10.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:38:23,332::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:38:23,874::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0813525 s, 12.6 MB/s\n'; <rc> = 0
>Thread-42838::DEBUG::2013-11-20 04:38:24,550::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 04:38:25,207::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:38:25,891::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 04:38:26,111::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:38:26,114::task::495::TaskManager.Task::(_debug) Task 4708b80e-ec5e-411d-b152-5616d2d361e6: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:38:26,117::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`224e63c4-f3ed-4b29-aa5f-35187ee2ea26`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:38:26,118::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:38:26,119::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:38:26,120::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`224e63c4-f3ed-4b29-aa5f-35187ee2ea26`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:38:26,121::task::495::TaskManager.Task::(_debug) Task 4708b80e-ec5e-411d-b152-5616d2d361e6: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:38:26,122::task::495::TaskManager.Task::(_debug) Task 4708b80e-ec5e-411d-b152-5616d2d361e6: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:38:26,123::task::495::TaskManager.Task::(_debug) Task 4708b80e-ec5e-411d-b152-5616d2d361e6: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:38:26,124::task::495::TaskManager.Task::(_debug) Task 4708b80e-ec5e-411d-b152-5616d2d361e6: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:38:26,125::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:38:26,126::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:38:26,126::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:38:26,127::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:38:26,128::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:38:26,128::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:38:26,129::task::495::TaskManager.Task::(_debug) Task 4708b80e-ec5e-411d-b152-5616d2d361e6: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:38:26,129::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:38:26,443::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.076205 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:38:28,458::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 04:38:28,756::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:38:28,758::task::495::TaskManager.Task::(_debug) Task e4d4d031-1627-4570-85a7-6b449df343b2: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:38:28,760::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a0ad7a50-4d02-4011-a0c1-6f198d486718`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:38:28,761::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:38:28,762::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:38:28,763::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a0ad7a50-4d02-4011-a0c1-6f198d486718`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:38:28,764::task::495::TaskManager.Task::(_debug) Task e4d4d031-1627-4570-85a7-6b449df343b2: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:38:28,765::task::495::TaskManager.Task::(_debug) Task e4d4d031-1627-4570-85a7-6b449df343b2: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:38:28,766::task::495::TaskManager.Task::(_debug) Task e4d4d031-1627-4570-85a7-6b449df343b2: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:38:28,766::task::495::TaskManager.Task::(_debug) Task e4d4d031-1627-4570-85a7-6b449df343b2: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:38:28,767::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:38:28,768::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:38:28,768::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:38:28,769::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:38:28,769::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:38:28,770::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:38:28,771::task::495::TaskManager.Task::(_debug) Task e4d4d031-1627-4570-85a7-6b449df343b2: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:38:28,771::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:38:29,029::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0830381 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:38:31,045::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:38:31,578::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0793252 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:38:33,597::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:38:34,188::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0871222 s, 11.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:38:36,198::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:38:36,714::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0569015 s, 18.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:38:38,726::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 04:38:38,941::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15914519::INFO::2013-11-20 04:38:38,945::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15913546::DEBUG::2013-11-20 04:38:38,947::task::495::TaskManager.Task::(_debug) Task 5a77f1a4-e533-4d4a-997c-bd4a21139d3b: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:38:38,953::task::495::TaskManager.Task::(_debug) Task ebc913f8-2769-489a-8e42-4eea25cf9ad6: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:38:38,961::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`59630f52-4bce-4ad3-8ce9-62badaee4fea`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:38:38,963::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`7dcbf9a6-8b26-4ce5-999b-24bc83f66abe`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:38:38,964::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:38:38,965::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:38:38,966::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:38:38,967::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`59630f52-4bce-4ad3-8ce9-62badaee4fea`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:38:38,968::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:38:38,968::task::495::TaskManager.Task::(_debug) Task ebc913f8-2769-489a-8e42-4eea25cf9ad6: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913546::DEBUG::2013-11-20 04:38:38,969::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`7dcbf9a6-8b26-4ce5-999b-24bc83f66abe`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:38:38,970::task::495::TaskManager.Task::(_debug) Task ebc913f8-2769-489a-8e42-4eea25cf9ad6: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:38:38,972::task::495::TaskManager.Task::(_debug) Task 5a77f1a4-e533-4d4a-997c-bd4a21139d3b: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15914519::DEBUG::2013-11-20 04:38:38,972::task::495::TaskManager.Task::(_debug) Task ebc913f8-2769-489a-8e42-4eea25cf9ad6: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:38:38,973::task::495::TaskManager.Task::(_debug) Task 5a77f1a4-e533-4d4a-997c-bd4a21139d3b: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:38:38,974::task::495::TaskManager.Task::(_debug) Task ebc913f8-2769-489a-8e42-4eea25cf9ad6: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:38:38,975::task::495::TaskManager.Task::(_debug) Task 5a77f1a4-e533-4d4a-997c-bd4a21139d3b: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:38:38,976::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:38:38,977::task::495::TaskManager.Task::(_debug) Task 5a77f1a4-e533-4d4a-997c-bd4a21139d3b: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:38:38,980::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:38:38,982::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:38:38,986::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:38:38,987::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913546::DEBUG::2013-11-20 04:38:38,992::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15914519::DEBUG::2013-11-20 04:38:38,994::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:38:38,995::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:38:38,996::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:38:38,996::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:38:38,997::task::495::TaskManager.Task::(_debug) Task ebc913f8-2769-489a-8e42-4eea25cf9ad6: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:38:38,997::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15914519::INFO::2013-11-20 04:38:38,998::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:38:38,998::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:38:38,999::task::495::TaskManager.Task::(_debug) Task 5a77f1a4-e533-4d4a-997c-bd4a21139d3b: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:38:39,000::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:38:39,001::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:38:39,001::task::495::TaskManager.Task::(_debug) Task b3e9da6b-d2e3-4daf-bbba-2189767360a9: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:38:39,003::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`3fc1fb40-b62f-4c78-a3f7-1602a91ca55f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:38:39,003::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:38:39,004::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:38:39,005::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`3fc1fb40-b62f-4c78-a3f7-1602a91ca55f`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:38:39,006::task::495::TaskManager.Task::(_debug) Task b3e9da6b-d2e3-4daf-bbba-2189767360a9: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:38:39,007::task::495::TaskManager.Task::(_debug) Task b3e9da6b-d2e3-4daf-bbba-2189767360a9: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:38:39,008::task::495::TaskManager.Task::(_debug) Task b3e9da6b-d2e3-4daf-bbba-2189767360a9: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:38:39,009::task::495::TaskManager.Task::(_debug) Task b3e9da6b-d2e3-4daf-bbba-2189767360a9: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:38:39,009::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:38:39,010::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:38:39,012::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:38:39,012::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:38:39,013::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:38:39,013::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:38:39,014::task::495::TaskManager.Task::(_debug) Task b3e9da6b-d2e3-4daf-bbba-2189767360a9: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:38:39,015::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:38:39,016::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:38:39,016::task::495::TaskManager.Task::(_debug) Task 18558901-8e93-497a-ae9c-fa89eef3d8c9: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:38:39,018::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`62726958-844b-458b-bdbf-733166da3588`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:38:39,018::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:38:39,019::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:38:39,020::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`62726958-844b-458b-bdbf-733166da3588`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:38:39,021::task::495::TaskManager.Task::(_debug) Task 18558901-8e93-497a-ae9c-fa89eef3d8c9: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:38:39,025::task::495::TaskManager.Task::(_debug) Task 18558901-8e93-497a-ae9c-fa89eef3d8c9: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:38:39,026::task::495::TaskManager.Task::(_debug) Task 18558901-8e93-497a-ae9c-fa89eef3d8c9: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:38:39,026::task::495::TaskManager.Task::(_debug) Task 18558901-8e93-497a-ae9c-fa89eef3d8c9: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:38:39,027::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:38:39,027::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:38:39,028::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:38:39,029::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:38:39,029::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:38:39,030::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:38:39,030::task::495::TaskManager.Task::(_debug) Task 18558901-8e93-497a-ae9c-fa89eef3d8c9: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:38:39,031::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:38:39,032::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:38:39,033::task::495::TaskManager.Task::(_debug) Task d3b42a00-f0f2-4fd8-8cff-fff3b990324a: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:38:39,034::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6768f5ed-013c-4c04-af4c-a8aff278c6a7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:38:39,035::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:38:39,035::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:38:39,036::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6768f5ed-013c-4c04-af4c-a8aff278c6a7`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:38:39,037::task::495::TaskManager.Task::(_debug) Task d3b42a00-f0f2-4fd8-8cff-fff3b990324a: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:38:39,038::task::495::TaskManager.Task::(_debug) Task d3b42a00-f0f2-4fd8-8cff-fff3b990324a: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:38:39,038::task::495::TaskManager.Task::(_debug) Task d3b42a00-f0f2-4fd8-8cff-fff3b990324a: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:38:39,039::task::495::TaskManager.Task::(_debug) Task d3b42a00-f0f2-4fd8-8cff-fff3b990324a: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:38:39,040::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:38:39,040::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:38:39,041::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:38:39,042::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:38:39,042::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:38:39,043::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:38:39,043::task::495::TaskManager.Task::(_debug) Task d3b42a00-f0f2-4fd8-8cff-fff3b990324a: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:38:39,044::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:38:39,286::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0905493 s, 11.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:38:41,301::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:38:41,851::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0781206 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:38:43,867::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 04:38:44,100::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:38:44,102::task::495::TaskManager.Task::(_debug) Task 6069b550-4a41-4432-9682-dbe4a36f09cf: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:38:44,104::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f03f6544-64d4-4d9c-ba07-73203090cfa1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:38:44,105::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:38:44,106::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:38:44,106::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f03f6544-64d4-4d9c-ba07-73203090cfa1`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:38:44,108::task::495::TaskManager.Task::(_debug) Task 6069b550-4a41-4432-9682-dbe4a36f09cf: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:38:44,108::task::495::TaskManager.Task::(_debug) Task 6069b550-4a41-4432-9682-dbe4a36f09cf: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:38:44,109::task::495::TaskManager.Task::(_debug) Task 6069b550-4a41-4432-9682-dbe4a36f09cf: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:38:44,110::task::495::TaskManager.Task::(_debug) Task 6069b550-4a41-4432-9682-dbe4a36f09cf: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:38:44,111::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:38:44,111::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:38:44,112::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:38:44,113::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:38:44,113::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:38:44,114::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:38:44,115::task::495::TaskManager.Task::(_debug) Task 6069b550-4a41-4432-9682-dbe4a36f09cf: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:38:44,115::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:38:44,418::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0763758 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:38:46,435::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:38:46,974::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0941391 s, 10.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:38:48,989::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:38:49,519::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0806484 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:38:51,534::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:38:52,073::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0797151 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:38:54,088::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:38:54,640::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0888328 s, 11.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:38:56,656::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:38:56,877::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:38:56,880::task::495::TaskManager.Task::(_debug) Task e1b8c473-f107-45de-81f9-c5b267a693cd: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:38:56,883::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5b54253c-cd62-46d6-8c9c-374adf60ea3f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:38:56,884::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:38:56,886::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:38:56,887::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5b54253c-cd62-46d6-8c9c-374adf60ea3f`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:38:56,889::task::495::TaskManager.Task::(_debug) Task e1b8c473-f107-45de-81f9-c5b267a693cd: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:38:56,891::task::495::TaskManager.Task::(_debug) Task e1b8c473-f107-45de-81f9-c5b267a693cd: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:38:56,892::task::495::TaskManager.Task::(_debug) Task e1b8c473-f107-45de-81f9-c5b267a693cd: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:38:56,893::task::495::TaskManager.Task::(_debug) Task e1b8c473-f107-45de-81f9-c5b267a693cd: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:38:56,894::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:38:56,895::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:38:56,895::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:38:56,896::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:38:56,897::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:38:56,898::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:38:56,899::task::495::TaskManager.Task::(_debug) Task e1b8c473-f107-45de-81f9-c5b267a693cd: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:38:56,899::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:38:57,195::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0736398 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:38:59,206::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915918::INFO::2013-11-20 04:38:59,446::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:38:59,448::task::495::TaskManager.Task::(_debug) Task 338ca682-6a93-4224-9ec4-4bb46d1a0875: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:38:59,452::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1e1a2dd9-d584-4b63-b5f3-3edd115ceaaf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:38:59,454::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:38:59,456::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:38:59,457::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1e1a2dd9-d584-4b63-b5f3-3edd115ceaaf`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:38:59,459::task::495::TaskManager.Task::(_debug) Task 338ca682-6a93-4224-9ec4-4bb46d1a0875: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:38:59,460::task::495::TaskManager.Task::(_debug) Task 338ca682-6a93-4224-9ec4-4bb46d1a0875: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:38:59,461::task::495::TaskManager.Task::(_debug) Task 338ca682-6a93-4224-9ec4-4bb46d1a0875: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:38:59,462::task::495::TaskManager.Task::(_debug) Task 338ca682-6a93-4224-9ec4-4bb46d1a0875: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:38:59,463::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:38:59,463::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:38:59,464::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:38:59,465::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:38:59,466::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:38:59,466::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:38:59,467::task::495::TaskManager.Task::(_debug) Task 338ca682-6a93-4224-9ec4-4bb46d1a0875: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:38:59,468::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:38:59,763::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.101041 s, 10.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:39:01,778::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:39:02,302::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.079821 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:39:04,317::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915863::INFO::2013-11-20 04:39:04,740::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:39:04,741::task::495::TaskManager.Task::(_debug) Task 971b535a-2d88-4e76-b781-78575709eb31: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:39:04,744::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a680851a-8610-48d0-815d-d5bf213191cf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:39:04,745::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:39:04,747::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:39:04,748::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a680851a-8610-48d0-815d-d5bf213191cf`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:39:04,750::task::495::TaskManager.Task::(_debug) Task 971b535a-2d88-4e76-b781-78575709eb31: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:39:04,751::task::495::TaskManager.Task::(_debug) Task 971b535a-2d88-4e76-b781-78575709eb31: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:39:04,752::task::495::TaskManager.Task::(_debug) Task 971b535a-2d88-4e76-b781-78575709eb31: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:39:04,755::task::495::TaskManager.Task::(_debug) Task 971b535a-2d88-4e76-b781-78575709eb31: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:39:04,756::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:39:04,757::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:39:04,758::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:39:04,759::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:39:04,760::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:39:04,762::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:39:04,763::task::495::TaskManager.Task::(_debug) Task 971b535a-2d88-4e76-b781-78575709eb31: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:39:04,764::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:39:04,875::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.118961 s, 8.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:39:06,890::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:39:07,459::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0795861 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:39:09,475::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 04:39:09,700::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:39:09,707::task::495::TaskManager.Task::(_debug) Task 476d6a9c-e512-45be-b3ad-3ee168d6bb71: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:39:09,711::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`62b7e647-7518-4051-aaeb-a9003502244c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:39:09,713::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:39:09,714::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:39:09,715::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`62b7e647-7518-4051-aaeb-a9003502244c`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:39:09,716::task::495::TaskManager.Task::(_debug) Task 476d6a9c-e512-45be-b3ad-3ee168d6bb71: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:39:09,717::task::495::TaskManager.Task::(_debug) Task 476d6a9c-e512-45be-b3ad-3ee168d6bb71: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:39:09,718::task::495::TaskManager.Task::(_debug) Task 476d6a9c-e512-45be-b3ad-3ee168d6bb71: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:39:09,718::task::495::TaskManager.Task::(_debug) Task 476d6a9c-e512-45be-b3ad-3ee168d6bb71: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:39:09,719::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:39:09,720::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:39:09,720::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:39:09,721::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:39:09,722::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:39:09,722::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:39:09,723::task::495::TaskManager.Task::(_debug) Task 476d6a9c-e512-45be-b3ad-3ee168d6bb71: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:39:09,724::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:39:10,019::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.083363 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:39:12,035::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:39:12,595::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0784971 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 04:39:13,216::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:39:13,217::task::495::TaskManager.Task::(_debug) Task f5bba0b4-cc2a-4465-a860-0d343da48556: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:39:13,219::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e400c77c-9b48-4153-9690-7817b7990c29`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:39:13,220::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:39:13,221::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:39:13,221::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e400c77c-9b48-4153-9690-7817b7990c29`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:39:13,222::task::495::TaskManager.Task::(_debug) Task f5bba0b4-cc2a-4465-a860-0d343da48556: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:39:13,223::task::495::TaskManager.Task::(_debug) Task f5bba0b4-cc2a-4465-a860-0d343da48556: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:39:13,224::task::495::TaskManager.Task::(_debug) Task f5bba0b4-cc2a-4465-a860-0d343da48556: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:39:13,225::task::495::TaskManager.Task::(_debug) Task f5bba0b4-cc2a-4465-a860-0d343da48556: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:39:13,225::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:39:13,226::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:39:13,227::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:39:13,227::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:39:13,228::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:39:13,229::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:39:13,229::task::495::TaskManager.Task::(_debug) Task f5bba0b4-cc2a-4465-a860-0d343da48556: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:39:13,230::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:39:13,231::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:39:13,231::task::495::TaskManager.Task::(_debug) Task 1193d5d2-2b21-4dd3-9e0e-f5cc2a918d04: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:39:13,232::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4ac2e373-c0c2-4fcd-a56e-37900d89812a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:39:13,233::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:39:13,234::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:39:13,234::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4ac2e373-c0c2-4fcd-a56e-37900d89812a`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:39:13,235::task::495::TaskManager.Task::(_debug) Task 1193d5d2-2b21-4dd3-9e0e-f5cc2a918d04: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:39:13,236::task::495::TaskManager.Task::(_debug) Task 1193d5d2-2b21-4dd3-9e0e-f5cc2a918d04: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:39:13,237::task::495::TaskManager.Task::(_debug) Task 1193d5d2-2b21-4dd3-9e0e-f5cc2a918d04: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:39:13,237::task::495::TaskManager.Task::(_debug) Task 1193d5d2-2b21-4dd3-9e0e-f5cc2a918d04: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:39:13,238::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:39:13,239::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:39:13,240::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:39:13,241::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:39:13,242::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:39:13,242::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:39:13,243::task::495::TaskManager.Task::(_debug) Task 1193d5d2-2b21-4dd3-9e0e-f5cc2a918d04: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:39:13,243::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:39:14,610::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:39:15,160::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0863845 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:39:17,176::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:39:17,696::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0775899 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:39:19,709::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 04:39:19,948::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:39:19,952::task::495::TaskManager.Task::(_debug) Task bb39f36a-1d7f-497e-be09-2c9e5d59d7f9: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:39:19,955::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d169d3e8-be0c-4fd3-8985-5645ff70eed1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:39:19,956::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:39:19,958::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:39:19,959::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d169d3e8-be0c-4fd3-8985-5645ff70eed1`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:39:19,961::task::495::TaskManager.Task::(_debug) Task bb39f36a-1d7f-497e-be09-2c9e5d59d7f9: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:39:19,961::task::495::TaskManager.Task::(_debug) Task bb39f36a-1d7f-497e-be09-2c9e5d59d7f9: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:39:19,962::task::495::TaskManager.Task::(_debug) Task bb39f36a-1d7f-497e-be09-2c9e5d59d7f9: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:39:19,963::task::495::TaskManager.Task::(_debug) Task bb39f36a-1d7f-497e-be09-2c9e5d59d7f9: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:39:19,964::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:39:19,964::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:39:19,965::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:39:19,966::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:39:19,966::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:39:19,967::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:39:19,967::task::495::TaskManager.Task::(_debug) Task bb39f36a-1d7f-497e-be09-2c9e5d59d7f9: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:39:19,968::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:39:20,266::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.080303 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:39:22,282::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:39:22,820::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.084106 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:39:24,834::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:39:25,346::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0735873 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:39:27,361::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 04:39:27,575::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:39:27,577::task::495::TaskManager.Task::(_debug) Task d542159a-b665-441d-8cfb-6a04575a432a: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:39:27,580::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1dfc02d7-478e-4544-ae04-6c5c5cc92cf4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:39:27,582::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:39:27,583::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:39:27,584::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1dfc02d7-478e-4544-ae04-6c5c5cc92cf4`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:39:27,585::task::495::TaskManager.Task::(_debug) Task d542159a-b665-441d-8cfb-6a04575a432a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:39:27,585::task::495::TaskManager.Task::(_debug) Task d542159a-b665-441d-8cfb-6a04575a432a: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:39:27,586::task::495::TaskManager.Task::(_debug) Task d542159a-b665-441d-8cfb-6a04575a432a: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:39:27,586::task::495::TaskManager.Task::(_debug) Task d542159a-b665-441d-8cfb-6a04575a432a: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:39:27,587::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:39:27,588::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:39:27,589::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:39:27,590::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:39:27,590::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:39:27,591::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:39:27,591::task::495::TaskManager.Task::(_debug) Task d542159a-b665-441d-8cfb-6a04575a432a: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:39:27,592::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:39:27,890::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0814372 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:39:29,905::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 04:39:30,121::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:39:30,124::task::495::TaskManager.Task::(_debug) Task 2f625674-082d-4d96-81aa-ecf308fb4d1f: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:39:30,128::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3cd20e03-81ff-4238-950b-8d7c793d9f6b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:39:30,130::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:39:30,132::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:39:30,132::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3cd20e03-81ff-4238-950b-8d7c793d9f6b`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:39:30,134::task::495::TaskManager.Task::(_debug) Task 2f625674-082d-4d96-81aa-ecf308fb4d1f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:39:30,135::task::495::TaskManager.Task::(_debug) Task 2f625674-082d-4d96-81aa-ecf308fb4d1f: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:39:30,136::task::495::TaskManager.Task::(_debug) Task 2f625674-082d-4d96-81aa-ecf308fb4d1f: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:39:30,137::task::495::TaskManager.Task::(_debug) Task 2f625674-082d-4d96-81aa-ecf308fb4d1f: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:39:30,138::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:39:30,138::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:39:30,139::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:39:30,140::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:39:30,141::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:39:30,141::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:39:30,142::task::495::TaskManager.Task::(_debug) Task 2f625674-082d-4d96-81aa-ecf308fb4d1f: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:39:30,143::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:39:30,449::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.103193 s, 9.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:39:32,464::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:39:32,988::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0788869 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:39:35,004::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:39:35,466::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0466194 s, 22.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:39:37,476::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:39:38,045::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0882451 s, 11.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:39:40,061::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 04:39:40,263::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15914519::INFO::2013-11-20 04:39:40,267::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15913546::DEBUG::2013-11-20 04:39:40,269::task::495::TaskManager.Task::(_debug) Task 9d12850c-72e7-4673-b8cc-86f7a250b997: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:39:40,273::task::495::TaskManager.Task::(_debug) Task c6249fa4-e774-4bfc-a5dd-911744e6444f: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:39:40,278::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a78a9864-f96f-42c4-b6d5-4f47dc750ef6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:39:40,280::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e9deef75-5f4a-4d5b-bbf8-d110b64946a4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:39:40,281::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:39:40,282::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:39:40,285::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:39:40,286::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a78a9864-f96f-42c4-b6d5-4f47dc750ef6`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:39:40,288::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:39:40,288::task::495::TaskManager.Task::(_debug) Task 9d12850c-72e7-4673-b8cc-86f7a250b997: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15914519::DEBUG::2013-11-20 04:39:40,289::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e9deef75-5f4a-4d5b-bbf8-d110b64946a4`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:39:40,290::task::495::TaskManager.Task::(_debug) Task 9d12850c-72e7-4673-b8cc-86f7a250b997: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:39:40,292::task::495::TaskManager.Task::(_debug) Task 9d12850c-72e7-4673-b8cc-86f7a250b997: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:39:40,292::task::495::TaskManager.Task::(_debug) Task c6249fa4-e774-4bfc-a5dd-911744e6444f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913546::DEBUG::2013-11-20 04:39:40,293::task::495::TaskManager.Task::(_debug) Task 9d12850c-72e7-4673-b8cc-86f7a250b997: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:39:40,294::task::495::TaskManager.Task::(_debug) Task c6249fa4-e774-4bfc-a5dd-911744e6444f: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:39:40,294::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:39:40,295::task::495::TaskManager.Task::(_debug) Task c6249fa4-e774-4bfc-a5dd-911744e6444f: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:39:40,296::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:39:40,296::task::495::TaskManager.Task::(_debug) Task c6249fa4-e774-4bfc-a5dd-911744e6444f: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:39:40,298::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15914519::DEBUG::2013-11-20 04:39:40,298::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:39:40,300::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:39:40,300::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:39:40,301::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913546::DEBUG::2013-11-20 04:39:40,302::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:39:40,303::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:39:40,304::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:39:40,304::task::495::TaskManager.Task::(_debug) Task 9d12850c-72e7-4673-b8cc-86f7a250b997: ref 0 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:39:40,305::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913546::INFO::2013-11-20 04:39:40,305::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:39:40,306::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913546::INFO::2013-11-20 04:39:40,307::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15914519::DEBUG::2013-11-20 04:39:40,308::task::495::TaskManager.Task::(_debug) Task c6249fa4-e774-4bfc-a5dd-911744e6444f: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:39:40,308::task::495::TaskManager.Task::(_debug) Task 5a588f78-a041-47e2-86b5-4e7727c91b16: moving from state init -> state preparing
>Thread-15914519::INFO::2013-11-20 04:39:40,309::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:39:40,311::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`0f9d7223-cd23-4b7c-b944-034b7fc7f8a2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:39:40,312::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:39:40,313::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:39:40,314::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`0f9d7223-cd23-4b7c-b944-034b7fc7f8a2`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:39:40,316::task::495::TaskManager.Task::(_debug) Task 5a588f78-a041-47e2-86b5-4e7727c91b16: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:39:40,316::task::495::TaskManager.Task::(_debug) Task 5a588f78-a041-47e2-86b5-4e7727c91b16: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:39:40,317::task::495::TaskManager.Task::(_debug) Task 5a588f78-a041-47e2-86b5-4e7727c91b16: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:39:40,318::task::495::TaskManager.Task::(_debug) Task 5a588f78-a041-47e2-86b5-4e7727c91b16: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:39:40,318::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:39:40,319::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:39:40,320::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:39:40,320::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:39:40,321::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:39:40,321::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:39:40,322::task::495::TaskManager.Task::(_debug) Task 5a588f78-a041-47e2-86b5-4e7727c91b16: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:39:40,323::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:39:40,324::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:39:40,325::task::495::TaskManager.Task::(_debug) Task ec509e29-a690-4469-95bd-442f48e9f837: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:39:40,327::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`9553d906-4d9b-4f08-81f7-2aa1d98caee6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:39:40,327::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:39:40,328::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:39:40,329::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`9553d906-4d9b-4f08-81f7-2aa1d98caee6`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:39:40,330::task::495::TaskManager.Task::(_debug) Task ec509e29-a690-4469-95bd-442f48e9f837: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:39:40,330::task::495::TaskManager.Task::(_debug) Task ec509e29-a690-4469-95bd-442f48e9f837: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:39:40,331::task::495::TaskManager.Task::(_debug) Task ec509e29-a690-4469-95bd-442f48e9f837: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:39:40,332::task::495::TaskManager.Task::(_debug) Task ec509e29-a690-4469-95bd-442f48e9f837: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:39:40,332::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:39:40,333::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:39:40,333::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:39:40,334::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:39:40,335::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:39:40,335::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:39:40,336::task::495::TaskManager.Task::(_debug) Task ec509e29-a690-4469-95bd-442f48e9f837: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:39:40,337::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:39:40,337::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:39:40,338::task::495::TaskManager.Task::(_debug) Task 014e54c2-ec10-4021-8298-1f19c8b5dc7d: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:39:40,344::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c93aa964-628b-4036-8675-bb4d2e551690`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:39:40,345::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:39:40,346::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:39:40,346::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c93aa964-628b-4036-8675-bb4d2e551690`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:39:40,347::task::495::TaskManager.Task::(_debug) Task 014e54c2-ec10-4021-8298-1f19c8b5dc7d: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:39:40,348::task::495::TaskManager.Task::(_debug) Task 014e54c2-ec10-4021-8298-1f19c8b5dc7d: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:39:40,349::task::495::TaskManager.Task::(_debug) Task 014e54c2-ec10-4021-8298-1f19c8b5dc7d: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:39:40,349::task::495::TaskManager.Task::(_debug) Task 014e54c2-ec10-4021-8298-1f19c8b5dc7d: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:39:40,350::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:39:40,351::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:39:40,352::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:39:40,352::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:39:40,353::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:39:40,353::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:39:40,354::task::495::TaskManager.Task::(_debug) Task 014e54c2-ec10-4021-8298-1f19c8b5dc7d: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:39:40,355::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:39:40,588::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0803481 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:39:42,604::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:39:43,175::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0869589 s, 11.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:39:45,191::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 04:39:45,433::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:39:45,437::task::495::TaskManager.Task::(_debug) Task 995484db-24e7-4786-b46e-29a7ebfbeaf5: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:39:45,440::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9df7487d-4215-4550-a816-67380284a697`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:39:45,441::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:39:45,442::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:39:45,443::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9df7487d-4215-4550-a816-67380284a697`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:39:45,444::task::495::TaskManager.Task::(_debug) Task 995484db-24e7-4786-b46e-29a7ebfbeaf5: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:39:45,445::task::495::TaskManager.Task::(_debug) Task 995484db-24e7-4786-b46e-29a7ebfbeaf5: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:39:45,446::task::495::TaskManager.Task::(_debug) Task 995484db-24e7-4786-b46e-29a7ebfbeaf5: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:39:45,447::task::495::TaskManager.Task::(_debug) Task 995484db-24e7-4786-b46e-29a7ebfbeaf5: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:39:45,447::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:39:45,448::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:39:45,449::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:39:45,450::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:39:45,450::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:39:45,451::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:39:45,452::task::495::TaskManager.Task::(_debug) Task 995484db-24e7-4786-b46e-29a7ebfbeaf5: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:39:45,453::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:39:45,722::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0835458 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:39:47,736::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:39:48,283::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.083414 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:39:50,297::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:39:50,818::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0759937 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:39:52,832::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:39:53,352::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0725579 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:39:55,368::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:39:55,919::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0879587 s, 11.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:39:57,936::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:39:58,145::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:39:58,148::task::495::TaskManager.Task::(_debug) Task 83b7d7d7-5ca0-4b23-9211-e0d8a9f27b0a: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:39:58,151::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`322e61ae-d492-4f0c-89a5-3603e31d0981`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:39:58,152::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:39:58,153::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:39:58,154::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`322e61ae-d492-4f0c-89a5-3603e31d0981`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:39:58,157::task::495::TaskManager.Task::(_debug) Task 83b7d7d7-5ca0-4b23-9211-e0d8a9f27b0a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:39:58,158::task::495::TaskManager.Task::(_debug) Task 83b7d7d7-5ca0-4b23-9211-e0d8a9f27b0a: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:39:58,159::task::495::TaskManager.Task::(_debug) Task 83b7d7d7-5ca0-4b23-9211-e0d8a9f27b0a: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:39:58,160::task::495::TaskManager.Task::(_debug) Task 83b7d7d7-5ca0-4b23-9211-e0d8a9f27b0a: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:39:58,161::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:39:58,162::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:39:58,163::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:39:58,163::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:39:58,164::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:39:58,165::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:39:58,166::task::495::TaskManager.Task::(_debug) Task 83b7d7d7-5ca0-4b23-9211-e0d8a9f27b0a: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:39:58,166::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:39:58,467::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0728639 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:40:00,483::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915918::INFO::2013-11-20 04:40:00,709::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:40:00,713::task::495::TaskManager.Task::(_debug) Task 9e8e23d8-ec08-4e80-876b-012a45b9f261: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:40:00,717::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0ad91fb9-f568-4789-b895-351ab124fefa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:40:00,718::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:40:00,720::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:40:00,721::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0ad91fb9-f568-4789-b895-351ab124fefa`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:40:00,722::task::495::TaskManager.Task::(_debug) Task 9e8e23d8-ec08-4e80-876b-012a45b9f261: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:40:00,723::task::495::TaskManager.Task::(_debug) Task 9e8e23d8-ec08-4e80-876b-012a45b9f261: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:40:00,724::task::495::TaskManager.Task::(_debug) Task 9e8e23d8-ec08-4e80-876b-012a45b9f261: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:40:00,725::task::495::TaskManager.Task::(_debug) Task 9e8e23d8-ec08-4e80-876b-012a45b9f261: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:40:00,726::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:40:00,726::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:40:00,727::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:40:00,728::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:40:00,729::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:40:00,729::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:40:00,730::task::495::TaskManager.Task::(_debug) Task 9e8e23d8-ec08-4e80-876b-012a45b9f261: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:40:00,731::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:40:01,059::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0884475 s, 11.6 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 04:40:02,801::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` (Operation Stuck)
>Thread-15916803::DEBUG::2013-11-20 04:40:02,813::misc::1156::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Emitting event
>Thread-15916803::DEBUG::2013-11-20 04:40:02,824::misc::1166::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Calling registered method `run`
>Thread-15916803::DEBUG::2013-11-20 04:40:02,829::misc::1174::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Event emitted
>Dummy-15912651::DEBUG::2013-11-20 04:40:03,072::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:40:03,615::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0663452 s, 15.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:40:05,630::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915863::INFO::2013-11-20 04:40:05,856::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:40:05,857::task::495::TaskManager.Task::(_debug) Task cc90f0ff-8046-40a4-92de-3f36e501aaf1: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:40:05,859::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3f8c0025-fa28-4bba-8444-22ed37823559`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:40:05,860::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:40:05,864::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:40:05,864::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3f8c0025-fa28-4bba-8444-22ed37823559`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:40:05,866::task::495::TaskManager.Task::(_debug) Task cc90f0ff-8046-40a4-92de-3f36e501aaf1: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:40:05,867::task::495::TaskManager.Task::(_debug) Task cc90f0ff-8046-40a4-92de-3f36e501aaf1: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:40:05,868::task::495::TaskManager.Task::(_debug) Task cc90f0ff-8046-40a4-92de-3f36e501aaf1: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:40:05,869::task::495::TaskManager.Task::(_debug) Task cc90f0ff-8046-40a4-92de-3f36e501aaf1: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:40:05,869::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:40:05,870::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:40:05,871::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:40:05,872::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:40:05,873::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:40:05,873::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:40:05,874::task::495::TaskManager.Task::(_debug) Task cc90f0ff-8046-40a4-92de-3f36e501aaf1: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:40:05,874::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-42839::ERROR::2013-11-20 04:40:05,878::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `45cdff8b-89c0-4926-a8c7-80354064bfa6` (Operation Stuck)
>Thread-15916806::DEBUG::2013-11-20 04:40:05,879::misc::1156::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Emitting event
>Thread-15916806::DEBUG::2013-11-20 04:40:05,880::misc::1166::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Calling registered method `run`
>Thread-15916806::DEBUG::2013-11-20 04:40:05,886::misc::1174::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Event emitted
>Dummy-15912651::DEBUG::2013-11-20 04:40:06,354::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0850899 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:40:08,374::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:40:08,939::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0965896 s, 10.6 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 04:40:10,866::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:40:10,883::task::495::TaskManager.Task::(_debug) Task a6b62600-68dd-44e9-8c1d-857d9d6db2fb: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:40:10,887::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0998b88e-b67f-44dd-bcbd-ff8c612da3ed`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:40:10,889::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:40:10,890::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:40:10,891::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0998b88e-b67f-44dd-bcbd-ff8c612da3ed`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:40:10,893::task::495::TaskManager.Task::(_debug) Task a6b62600-68dd-44e9-8c1d-857d9d6db2fb: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:40:10,894::task::495::TaskManager.Task::(_debug) Task a6b62600-68dd-44e9-8c1d-857d9d6db2fb: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:40:10,895::task::495::TaskManager.Task::(_debug) Task a6b62600-68dd-44e9-8c1d-857d9d6db2fb: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:40:10,895::task::495::TaskManager.Task::(_debug) Task a6b62600-68dd-44e9-8c1d-857d9d6db2fb: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:40:10,896::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:40:10,897::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:40:10,898::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:40:10,899::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:40:10,900::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:40:10,900::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:40:10,901::task::495::TaskManager.Task::(_debug) Task a6b62600-68dd-44e9-8c1d-857d9d6db2fb: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:40:10,902::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:40:10,948::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:40:11,542::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0911516 s, 11.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:40:13,565::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:40:14,079::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0797852 s, 12.8 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 04:40:14,390::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:40:14,392::task::495::TaskManager.Task::(_debug) Task 573a7728-cbc6-42f9-b229-2021b35dd1c5: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:40:14,393::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`12d7356c-0af5-454a-946f-41efc599ffdb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:40:14,394::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:40:14,395::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:40:14,396::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`12d7356c-0af5-454a-946f-41efc599ffdb`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:40:14,397::task::495::TaskManager.Task::(_debug) Task 573a7728-cbc6-42f9-b229-2021b35dd1c5: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:40:14,397::task::495::TaskManager.Task::(_debug) Task 573a7728-cbc6-42f9-b229-2021b35dd1c5: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:40:14,398::task::495::TaskManager.Task::(_debug) Task 573a7728-cbc6-42f9-b229-2021b35dd1c5: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:40:14,399::task::495::TaskManager.Task::(_debug) Task 573a7728-cbc6-42f9-b229-2021b35dd1c5: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:40:14,399::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:40:14,400::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:40:14,401::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:40:14,402::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:40:14,402::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:40:14,403::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:40:14,403::task::495::TaskManager.Task::(_debug) Task 573a7728-cbc6-42f9-b229-2021b35dd1c5: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:40:14,404::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:40:14,405::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:40:14,405::task::495::TaskManager.Task::(_debug) Task 3dccf4b4-f57f-4f3b-8ca6-568e23643c9f: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:40:14,407::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3e63652e-b4c8-4f65-be26-25b18d5d9d30`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:40:14,407::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:40:14,408::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:40:14,408::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3e63652e-b4c8-4f65-be26-25b18d5d9d30`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:40:14,409::task::495::TaskManager.Task::(_debug) Task 3dccf4b4-f57f-4f3b-8ca6-568e23643c9f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:40:14,410::task::495::TaskManager.Task::(_debug) Task 3dccf4b4-f57f-4f3b-8ca6-568e23643c9f: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:40:14,411::task::495::TaskManager.Task::(_debug) Task 3dccf4b4-f57f-4f3b-8ca6-568e23643c9f: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:40:14,411::task::495::TaskManager.Task::(_debug) Task 3dccf4b4-f57f-4f3b-8ca6-568e23643c9f: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:40:14,412::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:40:14,412::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:40:14,413::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:40:14,414::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:40:14,414::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:40:14,415::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:40:14,416::task::495::TaskManager.Task::(_debug) Task 3dccf4b4-f57f-4f3b-8ca6-568e23643c9f: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:40:14,417::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:40:16,095::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:40:16,657::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0841551 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:40:18,672::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:40:19,215::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0852147 s, 12.0 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 04:40:21,092::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:40:21,093::task::495::TaskManager.Task::(_debug) Task bbdbc4ab-b76c-4032-908f-ae30a4a34b05: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:40:21,095::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`547a3c7e-3941-4fe4-a940-1490dfb90adf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:40:21,096::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:40:21,097::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:40:21,098::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`547a3c7e-3941-4fe4-a940-1490dfb90adf`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:40:21,101::task::495::TaskManager.Task::(_debug) Task bbdbc4ab-b76c-4032-908f-ae30a4a34b05: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:40:21,103::task::495::TaskManager.Task::(_debug) Task bbdbc4ab-b76c-4032-908f-ae30a4a34b05: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:40:21,103::task::495::TaskManager.Task::(_debug) Task bbdbc4ab-b76c-4032-908f-ae30a4a34b05: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:40:21,104::task::495::TaskManager.Task::(_debug) Task bbdbc4ab-b76c-4032-908f-ae30a4a34b05: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:40:21,105::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:40:21,106::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:40:21,106::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:40:21,108::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:40:21,108::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:40:21,109::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:40:21,109::task::495::TaskManager.Task::(_debug) Task bbdbc4ab-b76c-4032-908f-ae30a4a34b05: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:40:21,110::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:40:21,230::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:40:21,827::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0822262 s, 12.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:40:23,841::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:40:24,419::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0815871 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:40:26,435::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:40:26,963::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0804899 s, 12.7 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 04:40:28,655::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:40:28,656::task::495::TaskManager.Task::(_debug) Task 39723b0b-3547-4378-8f28-8b29509c7927: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:40:28,658::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`dfeed507-f6db-4dd2-b7f0-73475466b440`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:40:28,658::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:40:28,659::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:40:28,660::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`dfeed507-f6db-4dd2-b7f0-73475466b440`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:40:28,661::task::495::TaskManager.Task::(_debug) Task 39723b0b-3547-4378-8f28-8b29509c7927: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:40:28,662::task::495::TaskManager.Task::(_debug) Task 39723b0b-3547-4378-8f28-8b29509c7927: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:40:28,662::task::495::TaskManager.Task::(_debug) Task 39723b0b-3547-4378-8f28-8b29509c7927: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:40:28,663::task::495::TaskManager.Task::(_debug) Task 39723b0b-3547-4378-8f28-8b29509c7927: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:40:28,664::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:40:28,664::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:40:28,665::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:40:28,666::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:40:28,667::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:40:28,667::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:40:28,668::task::495::TaskManager.Task::(_debug) Task 39723b0b-3547-4378-8f28-8b29509c7927: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:40:28,669::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:40:28,979::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:40:29,526::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0850781 s, 12.0 MB/s\n'; <rc> = 0
>Thread-15914453::INFO::2013-11-20 04:40:30,902::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:40:30,903::task::495::TaskManager.Task::(_debug) Task 2c962b7b-26bd-4f86-942b-6b81bda93987: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:40:30,905::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`67a7ebad-9d87-417f-9e57-f57a50523f82`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:40:30,906::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:40:30,907::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:40:30,908::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`67a7ebad-9d87-417f-9e57-f57a50523f82`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:40:30,909::task::495::TaskManager.Task::(_debug) Task 2c962b7b-26bd-4f86-942b-6b81bda93987: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:40:30,910::task::495::TaskManager.Task::(_debug) Task 2c962b7b-26bd-4f86-942b-6b81bda93987: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:40:30,910::task::495::TaskManager.Task::(_debug) Task 2c962b7b-26bd-4f86-942b-6b81bda93987: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:40:30,911::task::495::TaskManager.Task::(_debug) Task 2c962b7b-26bd-4f86-942b-6b81bda93987: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:40:30,912::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:40:30,913::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:40:30,913::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:40:30,914::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:40:30,915::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:40:30,915::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:40:30,916::task::495::TaskManager.Task::(_debug) Task 2c962b7b-26bd-4f86-942b-6b81bda93987: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:40:30,917::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:40:31,540::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:40:32,080::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0776545 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:40:34,093::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:40:34,677::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0812245 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:40:36,691::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:40:37,183::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0773802 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:40:39,197::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:40:39,724::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.072481 s, 14.1 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 04:40:41,420::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15914519::INFO::2013-11-20 04:40:41,421::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15913546::DEBUG::2013-11-20 04:40:41,422::task::495::TaskManager.Task::(_debug) Task 5402ec9f-dd8a-450d-b0d6-b55fa4799561: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:40:41,423::task::495::TaskManager.Task::(_debug) Task 1f2b5163-ae2e-46c7-8bcc-7aba050032ca: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:40:41,426::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`884e45e1-4454-4ea8-8722-fea48695d87b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:40:41,428::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bcc875a3-eeac-488c-bd36-0922d2f99dfd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:40:41,429::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:40:41,430::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:40:41,431::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:40:41,432::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`884e45e1-4454-4ea8-8722-fea48695d87b`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:40:41,433::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:40:41,434::task::495::TaskManager.Task::(_debug) Task 5402ec9f-dd8a-450d-b0d6-b55fa4799561: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15914519::DEBUG::2013-11-20 04:40:41,434::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bcc875a3-eeac-488c-bd36-0922d2f99dfd`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:40:41,435::task::495::TaskManager.Task::(_debug) Task 5402ec9f-dd8a-450d-b0d6-b55fa4799561: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:40:41,437::task::495::TaskManager.Task::(_debug) Task 5402ec9f-dd8a-450d-b0d6-b55fa4799561: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:40:41,438::task::495::TaskManager.Task::(_debug) Task 1f2b5163-ae2e-46c7-8bcc-7aba050032ca: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913546::DEBUG::2013-11-20 04:40:41,439::task::495::TaskManager.Task::(_debug) Task 5402ec9f-dd8a-450d-b0d6-b55fa4799561: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:40:41,440::task::495::TaskManager.Task::(_debug) Task 1f2b5163-ae2e-46c7-8bcc-7aba050032ca: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:40:41,441::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:40:41,442::task::495::TaskManager.Task::(_debug) Task 1f2b5163-ae2e-46c7-8bcc-7aba050032ca: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:40:41,442::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:40:41,443::task::495::TaskManager.Task::(_debug) Task 1f2b5163-ae2e-46c7-8bcc-7aba050032ca: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:40:41,444::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15914519::DEBUG::2013-11-20 04:40:41,445::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:40:41,446::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:40:41,446::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:40:41,447::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:40:41,448::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913546::DEBUG::2013-11-20 04:40:41,448::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:40:41,450::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:40:41,450::task::495::TaskManager.Task::(_debug) Task 5402ec9f-dd8a-450d-b0d6-b55fa4799561: ref 0 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:40:41,451::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913546::INFO::2013-11-20 04:40:41,451::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:40:41,452::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913546::INFO::2013-11-20 04:40:41,453::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15914519::DEBUG::2013-11-20 04:40:41,454::task::495::TaskManager.Task::(_debug) Task 1f2b5163-ae2e-46c7-8bcc-7aba050032ca: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:40:41,455::task::495::TaskManager.Task::(_debug) Task 1ec483e7-5a74-42cf-9c89-e0a72cfeaad1: moving from state init -> state preparing
>Thread-15914519::INFO::2013-11-20 04:40:41,456::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:40:41,457::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`15479f0a-7d4f-4867-8a91-e91df6d82e68`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:40:41,459::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:40:41,459::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:40:41,460::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`15479f0a-7d4f-4867-8a91-e91df6d82e68`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:40:41,462::task::495::TaskManager.Task::(_debug) Task 1ec483e7-5a74-42cf-9c89-e0a72cfeaad1: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:40:41,462::task::495::TaskManager.Task::(_debug) Task 1ec483e7-5a74-42cf-9c89-e0a72cfeaad1: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:40:41,463::task::495::TaskManager.Task::(_debug) Task 1ec483e7-5a74-42cf-9c89-e0a72cfeaad1: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:40:41,464::task::495::TaskManager.Task::(_debug) Task 1ec483e7-5a74-42cf-9c89-e0a72cfeaad1: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:40:41,464::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:40:41,465::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:40:41,466::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:40:41,467::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:40:41,467::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:40:41,468::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:40:41,468::task::495::TaskManager.Task::(_debug) Task 1ec483e7-5a74-42cf-9c89-e0a72cfeaad1: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:40:41,469::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:40:41,470::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:40:41,471::task::495::TaskManager.Task::(_debug) Task 47535d6f-7e44-4a0a-b17c-926c6bd1c54d: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:40:41,472::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`95bbbd47-fca5-471f-b9e0-3d621b1dd986`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:40:41,473::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:40:41,473::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:40:41,474::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`95bbbd47-fca5-471f-b9e0-3d621b1dd986`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:40:41,475::task::495::TaskManager.Task::(_debug) Task 47535d6f-7e44-4a0a-b17c-926c6bd1c54d: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:40:41,476::task::495::TaskManager.Task::(_debug) Task 47535d6f-7e44-4a0a-b17c-926c6bd1c54d: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:40:41,476::task::495::TaskManager.Task::(_debug) Task 47535d6f-7e44-4a0a-b17c-926c6bd1c54d: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:40:41,477::task::495::TaskManager.Task::(_debug) Task 47535d6f-7e44-4a0a-b17c-926c6bd1c54d: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:40:41,478::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:40:41,479::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:40:41,480::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:40:41,480::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:40:41,481::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:40:41,481::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:40:41,482::task::495::TaskManager.Task::(_debug) Task 47535d6f-7e44-4a0a-b17c-926c6bd1c54d: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:40:41,483::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:40:41,484::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:40:41,484::task::495::TaskManager.Task::(_debug) Task c5104a6d-3664-4413-b8b6-bb6f3ef6e2af: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:40:41,486::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`0afa15e0-bb0d-4b55-a5ab-7ddb8ca9e757`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:40:41,489::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:40:41,490::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:40:41,491::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`0afa15e0-bb0d-4b55-a5ab-7ddb8ca9e757`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:40:41,492::task::495::TaskManager.Task::(_debug) Task c5104a6d-3664-4413-b8b6-bb6f3ef6e2af: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:40:41,493::task::495::TaskManager.Task::(_debug) Task c5104a6d-3664-4413-b8b6-bb6f3ef6e2af: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:40:41,493::task::495::TaskManager.Task::(_debug) Task c5104a6d-3664-4413-b8b6-bb6f3ef6e2af: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:40:41,494::task::495::TaskManager.Task::(_debug) Task c5104a6d-3664-4413-b8b6-bb6f3ef6e2af: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:40:41,494::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:40:41,495::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:40:41,496::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:40:41,497::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:40:41,497::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:40:41,498::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:40:41,499::task::495::TaskManager.Task::(_debug) Task c5104a6d-3664-4413-b8b6-bb6f3ef6e2af: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:40:41,499::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:40:41,740::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:40:42,286::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0778115 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:40:44,301::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:40:44,831::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0805787 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:40:46,847::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 04:40:47,077::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:40:47,079::task::495::TaskManager.Task::(_debug) Task b372d0ae-4736-456f-a387-6d4bbc0b71a4: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:40:47,082::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c37f9dda-a9f6-490c-80a9-c551bd4ce09a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:40:47,082::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:40:47,084::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:40:47,084::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c37f9dda-a9f6-490c-80a9-c551bd4ce09a`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:40:47,086::task::495::TaskManager.Task::(_debug) Task b372d0ae-4736-456f-a387-6d4bbc0b71a4: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:40:47,086::task::495::TaskManager.Task::(_debug) Task b372d0ae-4736-456f-a387-6d4bbc0b71a4: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:40:47,087::task::495::TaskManager.Task::(_debug) Task b372d0ae-4736-456f-a387-6d4bbc0b71a4: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:40:47,089::task::495::TaskManager.Task::(_debug) Task b372d0ae-4736-456f-a387-6d4bbc0b71a4: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:40:47,089::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:40:47,090::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:40:47,091::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:40:47,091::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:40:47,092::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:40:47,092::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:40:47,093::task::495::TaskManager.Task::(_debug) Task b372d0ae-4736-456f-a387-6d4bbc0b71a4: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:40:47,094::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:40:47,378::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0536892 s, 19.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:40:49,387::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:40:49,980::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0833165 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:40:52,000::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:40:52,554::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0753174 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:40:54,565::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:40:55,071::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0733589 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:40:57,086::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:40:57,654::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.109701 s, 9.3 MB/s\n'; <rc> = 0
>Thread-15913639::INFO::2013-11-20 04:40:58,506::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:40:58,507::task::495::TaskManager.Task::(_debug) Task b0cea2fe-3708-42b4-9c22-1199a993bf3d: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:40:58,509::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`10f0b6c6-a67d-4fb3-97be-d07806b11cb4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:40:58,510::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:40:58,511::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:40:58,511::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`10f0b6c6-a67d-4fb3-97be-d07806b11cb4`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:40:58,513::task::495::TaskManager.Task::(_debug) Task b0cea2fe-3708-42b4-9c22-1199a993bf3d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:40:58,513::task::495::TaskManager.Task::(_debug) Task b0cea2fe-3708-42b4-9c22-1199a993bf3d: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:40:58,514::task::495::TaskManager.Task::(_debug) Task b0cea2fe-3708-42b4-9c22-1199a993bf3d: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:40:58,515::task::495::TaskManager.Task::(_debug) Task b0cea2fe-3708-42b4-9c22-1199a993bf3d: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:40:58,515::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:40:58,516::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:40:58,517::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:40:58,518::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:40:58,518::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:40:58,519::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:40:58,519::task::495::TaskManager.Task::(_debug) Task b0cea2fe-3708-42b4-9c22-1199a993bf3d: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:40:58,520::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:40:59,670::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:41:00,208::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0799332 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:41:02,224::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915918::INFO::2013-11-20 04:41:02,426::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:41:02,429::task::495::TaskManager.Task::(_debug) Task 037b74ae-be70-4e37-9c5d-ac2927526ce1: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:41:02,433::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b1d5496d-74bd-4d59-af99-14d8fad3fa94`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:41:02,435::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:41:02,437::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:41:02,437::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b1d5496d-74bd-4d59-af99-14d8fad3fa94`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:41:02,439::task::495::TaskManager.Task::(_debug) Task 037b74ae-be70-4e37-9c5d-ac2927526ce1: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:41:02,439::task::495::TaskManager.Task::(_debug) Task 037b74ae-be70-4e37-9c5d-ac2927526ce1: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:41:02,440::task::495::TaskManager.Task::(_debug) Task 037b74ae-be70-4e37-9c5d-ac2927526ce1: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:41:02,441::task::495::TaskManager.Task::(_debug) Task 037b74ae-be70-4e37-9c5d-ac2927526ce1: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:41:02,441::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:41:02,442::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:41:02,442::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:41:02,443::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:41:02,444::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:41:02,444::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:41:02,445::task::495::TaskManager.Task::(_debug) Task 037b74ae-be70-4e37-9c5d-ac2927526ce1: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:41:02,446::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:41:02,772::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0908789 s, 11.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:41:04,790::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:41:05,346::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0826592 s, 12.4 MB/s\n'; <rc> = 0
>Thread-15915863::INFO::2013-11-20 04:41:06,513::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:41:06,518::task::495::TaskManager.Task::(_debug) Task f9d72324-51cb-4c61-9420-ee4a98be2fac: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:41:06,524::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5c7e878f-487a-4bd8-899a-f4240a2ab85b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:41:06,526::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:41:06,528::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:41:06,528::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5c7e878f-487a-4bd8-899a-f4240a2ab85b`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:41:06,530::task::495::TaskManager.Task::(_debug) Task f9d72324-51cb-4c61-9420-ee4a98be2fac: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:41:06,530::task::495::TaskManager.Task::(_debug) Task f9d72324-51cb-4c61-9420-ee4a98be2fac: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:41:06,531::task::495::TaskManager.Task::(_debug) Task f9d72324-51cb-4c61-9420-ee4a98be2fac: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:41:06,532::task::495::TaskManager.Task::(_debug) Task f9d72324-51cb-4c61-9420-ee4a98be2fac: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:41:06,533::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:41:06,534::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:41:06,535::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:41:06,536::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:41:06,536::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:41:06,537::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:41:06,538::task::495::TaskManager.Task::(_debug) Task f9d72324-51cb-4c61-9420-ee4a98be2fac: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:41:06,539::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:41:07,363::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:41:07,922::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0894973 s, 11.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:41:09,939::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:41:10,522::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0906854 s, 11.3 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 04:41:11,632::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:41:11,635::task::495::TaskManager.Task::(_debug) Task db9b0468-ea89-400b-b03a-1ec90eeb333a: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:41:11,642::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0f3183bf-b4ee-47c6-878a-d58c3cc3d424`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:41:11,643::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:41:11,644::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:41:11,645::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0f3183bf-b4ee-47c6-878a-d58c3cc3d424`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:41:11,665::task::495::TaskManager.Task::(_debug) Task db9b0468-ea89-400b-b03a-1ec90eeb333a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:41:11,666::task::495::TaskManager.Task::(_debug) Task db9b0468-ea89-400b-b03a-1ec90eeb333a: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:41:11,667::task::495::TaskManager.Task::(_debug) Task db9b0468-ea89-400b-b03a-1ec90eeb333a: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:41:11,668::task::495::TaskManager.Task::(_debug) Task db9b0468-ea89-400b-b03a-1ec90eeb333a: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:41:11,669::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:41:11,669::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:41:11,670::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:41:11,671::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:41:11,671::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:41:11,672::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:41:11,673::task::495::TaskManager.Task::(_debug) Task db9b0468-ea89-400b-b03a-1ec90eeb333a: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:41:11,676::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:41:12,539::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:41:13,077::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0813617 s, 12.6 MB/s\n'; <rc> = 0
>Thread-42840::DEBUG::2013-11-20 04:41:14,140::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:41:15,208::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-42840::DEBUG::2013-11-20 04:41:15,212::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913518::INFO::2013-11-20 04:41:15,725::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:41:15,727::task::495::TaskManager.Task::(_debug) Task 38530202-8ded-4c64-91f6-fa852a3110fb: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:41:15,732::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`99f9e729-b555-4f8e-8eab-77151d9c6154`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:41:15,735::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:41:15,738::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:41:15,740::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`99f9e729-b555-4f8e-8eab-77151d9c6154`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:41:15,743::task::495::TaskManager.Task::(_debug) Task 38530202-8ded-4c64-91f6-fa852a3110fb: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:41:15,744::task::495::TaskManager.Task::(_debug) Task 38530202-8ded-4c64-91f6-fa852a3110fb: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:41:15,746::task::495::TaskManager.Task::(_debug) Task 38530202-8ded-4c64-91f6-fa852a3110fb: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:41:15,747::task::495::TaskManager.Task::(_debug) Task 38530202-8ded-4c64-91f6-fa852a3110fb: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:41:15,749::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:41:15,751::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:41:15,754::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:41:15,755::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:41:15,758::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:41:15,760::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:41:15,762::task::495::TaskManager.Task::(_debug) Task 38530202-8ded-4c64-91f6-fa852a3110fb: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:41:15,763::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:41:15,765::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:41:15,766::task::495::TaskManager.Task::(_debug) Task 3e866f1f-828b-470a-9b32-0edd217f4089: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:41:15,770::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`190dea5a-c6d0-47b7-8759-be01a3144a4e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:41:15,771::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:41:15,772::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:41:15,774::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`190dea5a-c6d0-47b7-8759-be01a3144a4e`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:41:15,778::task::495::TaskManager.Task::(_debug) Task 3e866f1f-828b-470a-9b32-0edd217f4089: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:41:15,779::task::495::TaskManager.Task::(_debug) Task 3e866f1f-828b-470a-9b32-0edd217f4089: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:41:15,781::task::495::TaskManager.Task::(_debug) Task 3e866f1f-828b-470a-9b32-0edd217f4089: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:41:15,782::task::495::TaskManager.Task::(_debug) Task 3e866f1f-828b-470a-9b32-0edd217f4089: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:41:15,784::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:41:15,785::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:41:15,787::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:41:15,788::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:41:15,790::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:41:15,791::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:41:15,792::task::495::TaskManager.Task::(_debug) Task 3e866f1f-828b-470a-9b32-0edd217f4089: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:41:15,795::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:41:15,807::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.086683 s, 11.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:41:17,823::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:41:18,791::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.316275 s, 3.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:41:20,807::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:41:21,341::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0893875 s, 11.5 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 04:41:21,671::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:41:21,673::task::495::TaskManager.Task::(_debug) Task 6b1affdb-6c23-42a7-8cc7-08d9013d9f9c: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:41:21,677::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`327a1ffb-4892-410b-8755-315360cad0d8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:41:21,677::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:41:21,678::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:41:21,679::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`327a1ffb-4892-410b-8755-315360cad0d8`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:41:21,680::task::495::TaskManager.Task::(_debug) Task 6b1affdb-6c23-42a7-8cc7-08d9013d9f9c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:41:21,680::task::495::TaskManager.Task::(_debug) Task 6b1affdb-6c23-42a7-8cc7-08d9013d9f9c: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:41:21,681::task::495::TaskManager.Task::(_debug) Task 6b1affdb-6c23-42a7-8cc7-08d9013d9f9c: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:41:21,682::task::495::TaskManager.Task::(_debug) Task 6b1affdb-6c23-42a7-8cc7-08d9013d9f9c: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:41:21,682::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:41:21,683::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:41:21,684::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:41:21,684::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:41:21,685::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:41:21,686::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:41:21,686::task::495::TaskManager.Task::(_debug) Task 6b1affdb-6c23-42a7-8cc7-08d9013d9f9c: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:41:21,687::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:41:23,356::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:41:23,928::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0840747 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:41:25,942::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:41:26,469::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0746662 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:41:28,480::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:41:29,048::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0735798 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 04:41:29,722::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:41:29,723::task::495::TaskManager.Task::(_debug) Task e886f1b0-95c7-4e92-8382-a32ce93aa635: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:41:29,725::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6fa91e7e-a4e2-4d28-a8b5-f7b13a93e6f4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:41:29,726::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:41:29,727::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:41:29,728::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6fa91e7e-a4e2-4d28-a8b5-f7b13a93e6f4`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:41:29,729::task::495::TaskManager.Task::(_debug) Task e886f1b0-95c7-4e92-8382-a32ce93aa635: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:41:29,730::task::495::TaskManager.Task::(_debug) Task e886f1b0-95c7-4e92-8382-a32ce93aa635: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:41:29,730::task::495::TaskManager.Task::(_debug) Task e886f1b0-95c7-4e92-8382-a32ce93aa635: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:41:29,731::task::495::TaskManager.Task::(_debug) Task e886f1b0-95c7-4e92-8382-a32ce93aa635: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:41:29,732::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:41:29,732::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:41:29,733::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:41:29,734::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:41:29,735::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:41:29,735::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:41:29,736::task::495::TaskManager.Task::(_debug) Task e886f1b0-95c7-4e92-8382-a32ce93aa635: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:41:29,737::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:41:31,063::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 04:41:31,260::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:41:31,262::task::495::TaskManager.Task::(_debug) Task 210475c0-0923-418e-af02-3d620d670d32: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:41:31,265::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`19d92b3b-de12-46fe-9dbe-51d23e0d91f6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:41:31,266::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:41:31,268::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:41:31,269::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`19d92b3b-de12-46fe-9dbe-51d23e0d91f6`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:41:31,270::task::495::TaskManager.Task::(_debug) Task 210475c0-0923-418e-af02-3d620d670d32: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:41:31,271::task::495::TaskManager.Task::(_debug) Task 210475c0-0923-418e-af02-3d620d670d32: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:41:31,272::task::495::TaskManager.Task::(_debug) Task 210475c0-0923-418e-af02-3d620d670d32: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:41:31,273::task::495::TaskManager.Task::(_debug) Task 210475c0-0923-418e-af02-3d620d670d32: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:41:31,273::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:41:31,274::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:41:31,275::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:41:31,275::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:41:31,276::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:41:31,276::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:41:31,277::task::495::TaskManager.Task::(_debug) Task 210475c0-0923-418e-af02-3d620d670d32: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:41:31,278::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:41:31,589::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0785198 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:41:33,605::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:41:34,154::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0769099 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:41:36,169::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:41:36,681::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0755275 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:41:38,698::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:41:39,199::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0769815 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:41:41,213::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:41:41,763::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0952691 s, 10.7 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 04:41:41,852::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:41:41,853::task::495::TaskManager.Task::(_debug) Task 005ddc0b-83b6-4042-84d6-85219d4b3cb0: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:41:41,854::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`41e8fe9f-5343-41da-a643-6ca48b22288c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:41:41,855::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:41:41,856::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:41:41,857::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`41e8fe9f-5343-41da-a643-6ca48b22288c`::Granted request
>Thread-15914519::INFO::2013-11-20 04:41:41,877::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15913546::DEBUG::2013-11-20 04:41:41,878::task::495::TaskManager.Task::(_debug) Task 005ddc0b-83b6-4042-84d6-85219d4b3cb0: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15914519::DEBUG::2013-11-20 04:41:41,879::task::495::TaskManager.Task::(_debug) Task af3b672b-c1d0-4112-9548-acb19c87a80c: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:41:41,880::task::495::TaskManager.Task::(_debug) Task 005ddc0b-83b6-4042-84d6-85219d4b3cb0: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:41:41,882::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`53b4cd72-bcfc-4858-b741-225cad833e92`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:41:41,883::task::495::TaskManager.Task::(_debug) Task 005ddc0b-83b6-4042-84d6-85219d4b3cb0: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:41:41,883::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:41:41,884::task::495::TaskManager.Task::(_debug) Task 005ddc0b-83b6-4042-84d6-85219d4b3cb0: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:41:41,885::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:41:41,887::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:41:41,887::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`53b4cd72-bcfc-4858-b741-225cad833e92`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:41:41,888::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:41:41,890::task::495::TaskManager.Task::(_debug) Task af3b672b-c1d0-4112-9548-acb19c87a80c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913546::DEBUG::2013-11-20 04:41:41,891::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15914519::DEBUG::2013-11-20 04:41:41,892::task::495::TaskManager.Task::(_debug) Task af3b672b-c1d0-4112-9548-acb19c87a80c: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:41:41,893::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:41:41,893::task::495::TaskManager.Task::(_debug) Task af3b672b-c1d0-4112-9548-acb19c87a80c: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:41:41,894::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:41:41,895::task::495::TaskManager.Task::(_debug) Task af3b672b-c1d0-4112-9548-acb19c87a80c: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:41:41,895::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:41:41,896::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:41:41,897::task::495::TaskManager.Task::(_debug) Task 005ddc0b-83b6-4042-84d6-85219d4b3cb0: ref 0 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:41:41,898::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::INFO::2013-11-20 04:41:41,899::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:41:41,900::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913546::INFO::2013-11-20 04:41:41,900::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15914519::DEBUG::2013-11-20 04:41:41,901::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:41:41,902::task::495::TaskManager.Task::(_debug) Task 6ca41894-04d7-4a3a-a2f4-03309f45ed19: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:41:41,902::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:41:41,904::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`7f21f36f-4b6b-43b3-8706-baa868ca00ce`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:41:41,905::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:41:41,905::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:41:41,906::task::495::TaskManager.Task::(_debug) Task af3b672b-c1d0-4112-9548-acb19c87a80c: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:41:41,907::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::INFO::2013-11-20 04:41:41,908::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:41:41,908::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`7f21f36f-4b6b-43b3-8706-baa868ca00ce`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:41:41,910::task::495::TaskManager.Task::(_debug) Task 6ca41894-04d7-4a3a-a2f4-03309f45ed19: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:41:41,911::task::495::TaskManager.Task::(_debug) Task 6ca41894-04d7-4a3a-a2f4-03309f45ed19: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:41:41,911::task::495::TaskManager.Task::(_debug) Task 6ca41894-04d7-4a3a-a2f4-03309f45ed19: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:41:41,912::task::495::TaskManager.Task::(_debug) Task 6ca41894-04d7-4a3a-a2f4-03309f45ed19: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:41:41,912::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:41:41,913::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:41:41,913::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:41:41,914::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:41:41,915::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:41:41,917::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:41:41,918::task::495::TaskManager.Task::(_debug) Task 6ca41894-04d7-4a3a-a2f4-03309f45ed19: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:41:41,919::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:41:41,919::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:41:41,920::task::495::TaskManager.Task::(_debug) Task 3b525c4d-a0b6-4bc9-98bd-b18d5efd4fc1: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:41:41,921::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f6aef9f6-82b6-4167-ae2d-e371970af9e3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:41:41,922::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:41:41,922::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:41:41,923::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f6aef9f6-82b6-4167-ae2d-e371970af9e3`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:41:41,924::task::495::TaskManager.Task::(_debug) Task 3b525c4d-a0b6-4bc9-98bd-b18d5efd4fc1: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:41:41,924::task::495::TaskManager.Task::(_debug) Task 3b525c4d-a0b6-4bc9-98bd-b18d5efd4fc1: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:41:41,925::task::495::TaskManager.Task::(_debug) Task 3b525c4d-a0b6-4bc9-98bd-b18d5efd4fc1: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:41:41,926::task::495::TaskManager.Task::(_debug) Task 3b525c4d-a0b6-4bc9-98bd-b18d5efd4fc1: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:41:41,927::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:41:41,927::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:41:41,928::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:41:41,929::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:41:41,929::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:41:41,930::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:41:41,930::task::495::TaskManager.Task::(_debug) Task 3b525c4d-a0b6-4bc9-98bd-b18d5efd4fc1: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:41:41,931::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:41:41,932::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:41:41,932::task::495::TaskManager.Task::(_debug) Task 9e0f5c00-52c0-41ce-86f9-f93cdaafd040: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:41:41,934::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`680918f5-1565-47d3-9872-03a897695eec`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:41:41,939::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:41:41,940::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:41:41,941::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`680918f5-1565-47d3-9872-03a897695eec`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:41:41,942::task::495::TaskManager.Task::(_debug) Task 9e0f5c00-52c0-41ce-86f9-f93cdaafd040: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:41:41,943::task::495::TaskManager.Task::(_debug) Task 9e0f5c00-52c0-41ce-86f9-f93cdaafd040: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:41:41,944::task::495::TaskManager.Task::(_debug) Task 9e0f5c00-52c0-41ce-86f9-f93cdaafd040: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:41:41,944::task::495::TaskManager.Task::(_debug) Task 9e0f5c00-52c0-41ce-86f9-f93cdaafd040: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:41:41,945::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:41:41,945::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:41:41,946::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:41:41,947::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:41:41,947::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:41:41,948::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:41:41,948::task::495::TaskManager.Task::(_debug) Task 9e0f5c00-52c0-41ce-86f9-f93cdaafd040: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:41:41,949::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:41:43,780::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:41:44,307::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0831533 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:41:46,319::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:41:46,855::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0874244 s, 11.7 MB/s\n'; <rc> = 0
>Thread-15914435::INFO::2013-11-20 04:41:48,001::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:41:48,002::task::495::TaskManager.Task::(_debug) Task 61889fb7-9367-463f-b13d-aeef0cf54c81: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:41:48,004::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`dd7943b3-9b62-45cb-88a5-f546a351b50d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:41:48,005::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:41:48,006::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:41:48,007::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`dd7943b3-9b62-45cb-88a5-f546a351b50d`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:41:48,008::task::495::TaskManager.Task::(_debug) Task 61889fb7-9367-463f-b13d-aeef0cf54c81: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:41:48,008::task::495::TaskManager.Task::(_debug) Task 61889fb7-9367-463f-b13d-aeef0cf54c81: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:41:48,009::task::495::TaskManager.Task::(_debug) Task 61889fb7-9367-463f-b13d-aeef0cf54c81: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:41:48,010::task::495::TaskManager.Task::(_debug) Task 61889fb7-9367-463f-b13d-aeef0cf54c81: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:41:48,010::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:41:48,011::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:41:48,012::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:41:48,012::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:41:48,013::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:41:48,014::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:41:48,014::task::495::TaskManager.Task::(_debug) Task 61889fb7-9367-463f-b13d-aeef0cf54c81: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:41:48,015::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:41:48,869::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:41:49,437::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0792058 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:41:51,446::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:41:52,010::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0852412 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:41:54,031::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:41:54,576::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0817771 s, 12.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:41:56,592::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:41:57,157::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0830232 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:41:59,175::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:41:59,369::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:41:59,372::task::495::TaskManager.Task::(_debug) Task e9136ff4-0014-419a-99c2-f2e664b0894d: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:41:59,375::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`43821a54-3100-406d-9521-137160462ada`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:41:59,376::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:41:59,379::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:41:59,379::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`43821a54-3100-406d-9521-137160462ada`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:41:59,381::task::495::TaskManager.Task::(_debug) Task e9136ff4-0014-419a-99c2-f2e664b0894d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:41:59,383::task::495::TaskManager.Task::(_debug) Task e9136ff4-0014-419a-99c2-f2e664b0894d: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:41:59,384::task::495::TaskManager.Task::(_debug) Task e9136ff4-0014-419a-99c2-f2e664b0894d: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:41:59,385::task::495::TaskManager.Task::(_debug) Task e9136ff4-0014-419a-99c2-f2e664b0894d: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:41:59,386::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:41:59,387::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:41:59,389::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:41:59,390::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:41:59,390::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:41:59,391::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:41:59,392::task::495::TaskManager.Task::(_debug) Task e9136ff4-0014-419a-99c2-f2e664b0894d: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:41:59,393::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:41:59,714::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0790908 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:42:01,730::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:42:02,311::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0762881 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15915918::INFO::2013-11-20 04:42:03,122::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:42:03,123::task::495::TaskManager.Task::(_debug) Task e701b7da-154c-46d8-98e9-1d7fb851c94d: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:42:03,125::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8a2315fd-1b13-4702-b44f-8d62508452e6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:42:03,126::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:42:03,127::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:42:03,127::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8a2315fd-1b13-4702-b44f-8d62508452e6`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:42:03,129::task::495::TaskManager.Task::(_debug) Task e701b7da-154c-46d8-98e9-1d7fb851c94d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:42:03,129::task::495::TaskManager.Task::(_debug) Task e701b7da-154c-46d8-98e9-1d7fb851c94d: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:42:03,130::task::495::TaskManager.Task::(_debug) Task e701b7da-154c-46d8-98e9-1d7fb851c94d: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:42:03,131::task::495::TaskManager.Task::(_debug) Task e701b7da-154c-46d8-98e9-1d7fb851c94d: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:42:03,132::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:42:03,132::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:42:03,133::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:42:03,134::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:42:03,134::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:42:03,135::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:42:03,136::task::495::TaskManager.Task::(_debug) Task e701b7da-154c-46d8-98e9-1d7fb851c94d: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:42:03,137::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:42:04,328::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:42:04,898::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0875491 s, 11.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:42:06,915::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915863::INFO::2013-11-20 04:42:07,149::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:42:07,155::task::495::TaskManager.Task::(_debug) Task abe1a99c-3af6-4ea4-b28f-c344b94a1c89: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:42:07,170::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`69c86359-a8aa-4182-a941-06cb4d233a2f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:42:07,175::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:42:07,176::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:42:07,177::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`69c86359-a8aa-4182-a941-06cb4d233a2f`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:42:07,179::task::495::TaskManager.Task::(_debug) Task abe1a99c-3af6-4ea4-b28f-c344b94a1c89: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:42:07,179::task::495::TaskManager.Task::(_debug) Task abe1a99c-3af6-4ea4-b28f-c344b94a1c89: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:42:07,180::task::495::TaskManager.Task::(_debug) Task abe1a99c-3af6-4ea4-b28f-c344b94a1c89: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:42:07,181::task::495::TaskManager.Task::(_debug) Task abe1a99c-3af6-4ea4-b28f-c344b94a1c89: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:42:07,182::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:42:07,182::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:42:07,183::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:42:07,184::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:42:07,184::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:42:07,185::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:42:07,186::task::495::TaskManager.Task::(_debug) Task abe1a99c-3af6-4ea4-b28f-c344b94a1c89: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:42:07,186::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:42:07,490::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0861275 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:42:09,508::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:42:10,064::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0719201 s, 14.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:42:12,078::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 04:42:12,333::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:42:12,336::task::495::TaskManager.Task::(_debug) Task 04c7d833-49bb-455f-aac5-4d268cc1fb98: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:42:12,340::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ab81c332-f14c-4576-90b3-20936dd5e8e1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:42:12,340::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:42:12,341::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:42:12,342::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ab81c332-f14c-4576-90b3-20936dd5e8e1`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:42:12,346::task::495::TaskManager.Task::(_debug) Task 04c7d833-49bb-455f-aac5-4d268cc1fb98: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:42:12,347::task::495::TaskManager.Task::(_debug) Task 04c7d833-49bb-455f-aac5-4d268cc1fb98: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:42:12,348::task::495::TaskManager.Task::(_debug) Task 04c7d833-49bb-455f-aac5-4d268cc1fb98: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:42:12,349::task::495::TaskManager.Task::(_debug) Task 04c7d833-49bb-455f-aac5-4d268cc1fb98: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:42:12,350::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:42:12,351::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:42:12,352::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:42:12,353::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:42:12,353::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:42:12,354::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:42:12,355::task::495::TaskManager.Task::(_debug) Task 04c7d833-49bb-455f-aac5-4d268cc1fb98: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:42:12,355::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:42:12,654::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0854811 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:42:14,669::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:42:15,199::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0783998 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:42:17,213::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-1024767::ERROR::2013-11-20 04:42:17,449::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` (Operation Stuck)
>Thread-15913518::INFO::2013-11-20 04:42:17,480::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:42:17,481::task::495::TaskManager.Task::(_debug) Task 954eb03f-b6e2-41eb-acc0-293855ef13ed: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:42:17,483::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a8fd30ed-e986-471e-ba46-8931a01da147`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:42:17,484::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:42:17,485::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:42:17,486::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a8fd30ed-e986-471e-ba46-8931a01da147`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:42:17,488::task::495::TaskManager.Task::(_debug) Task 954eb03f-b6e2-41eb-acc0-293855ef13ed: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:42:17,489::task::495::TaskManager.Task::(_debug) Task 954eb03f-b6e2-41eb-acc0-293855ef13ed: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:42:17,491::task::495::TaskManager.Task::(_debug) Task 954eb03f-b6e2-41eb-acc0-293855ef13ed: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:42:17,492::task::495::TaskManager.Task::(_debug) Task 954eb03f-b6e2-41eb-acc0-293855ef13ed: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:42:17,493::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:42:17,494::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:42:17,496::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:42:17,497::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:42:17,498::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:42:17,498::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:42:17,500::task::495::TaskManager.Task::(_debug) Task 954eb03f-b6e2-41eb-acc0-293855ef13ed: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:42:17,501::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:42:17,502::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:42:17,503::task::495::TaskManager.Task::(_debug) Task 5a6a66b5-6557-4184-9d80-c16302eb6be1: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:42:17,504::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`606df3da-c9b1-4f3a-8010-10489dff9e6a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:42:17,505::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:42:17,506::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:42:17,507::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`606df3da-c9b1-4f3a-8010-10489dff9e6a`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:42:17,509::task::495::TaskManager.Task::(_debug) Task 5a6a66b5-6557-4184-9d80-c16302eb6be1: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:42:17,510::task::495::TaskManager.Task::(_debug) Task 5a6a66b5-6557-4184-9d80-c16302eb6be1: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:42:17,511::task::495::TaskManager.Task::(_debug) Task 5a6a66b5-6557-4184-9d80-c16302eb6be1: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:42:17,511::task::495::TaskManager.Task::(_debug) Task 5a6a66b5-6557-4184-9d80-c16302eb6be1: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:42:17,512::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:42:17,512::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:42:17,513::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:42:17,514::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:42:17,514::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:42:17,515::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:42:17,516::task::495::TaskManager.Task::(_debug) Task 5a6a66b5-6557-4184-9d80-c16302eb6be1: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:42:17,516::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:42:17,920::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0479003 s, 21.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:42:19,936::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:42:20,476::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0785896 s, 13.0 MB/s\n'; <rc> = 0
>Thread-42839::ERROR::2013-11-20 04:42:20,605::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `45cdff8b-89c0-4926-a8c7-80354064bfa6` (Operation Stuck)
>Thread-15912392::INFO::2013-11-20 04:42:22,461::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:42:22,473::task::495::TaskManager.Task::(_debug) Task 7c064751-821f-4b16-a35f-ac29c10183cd: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:42:22,475::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e726f027-949c-43fd-97da-15c1a276b04d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:42:22,475::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:42:22,476::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:42:22,477::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e726f027-949c-43fd-97da-15c1a276b04d`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:42:22,478::task::495::TaskManager.Task::(_debug) Task 7c064751-821f-4b16-a35f-ac29c10183cd: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:42:22,479::task::495::TaskManager.Task::(_debug) Task 7c064751-821f-4b16-a35f-ac29c10183cd: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:42:22,479::task::495::TaskManager.Task::(_debug) Task 7c064751-821f-4b16-a35f-ac29c10183cd: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:42:22,480::task::495::TaskManager.Task::(_debug) Task 7c064751-821f-4b16-a35f-ac29c10183cd: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:42:22,481::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:42:22,481::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:42:22,482::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:42:22,483::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:42:22,483::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:42:22,484::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:42:22,486::task::495::TaskManager.Task::(_debug) Task 7c064751-821f-4b16-a35f-ac29c10183cd: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:42:22,487::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:42:22,494::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:42:23,038::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0837637 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:42:25,055::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:42:25,601::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0847405 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:42:27,619::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:42:28,175::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0847834 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:42:30,189::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 04:42:30,464::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:42:30,465::task::495::TaskManager.Task::(_debug) Task c9d84498-aae8-4c34-8527-01234a2193a2: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:42:30,468::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`04604223-125d-4eeb-b7de-d8c85e5c6aa6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:42:30,469::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:42:30,470::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:42:30,471::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`04604223-125d-4eeb-b7de-d8c85e5c6aa6`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:42:30,473::task::495::TaskManager.Task::(_debug) Task c9d84498-aae8-4c34-8527-01234a2193a2: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:42:30,473::task::495::TaskManager.Task::(_debug) Task c9d84498-aae8-4c34-8527-01234a2193a2: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:42:30,475::task::495::TaskManager.Task::(_debug) Task c9d84498-aae8-4c34-8527-01234a2193a2: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:42:30,475::task::495::TaskManager.Task::(_debug) Task c9d84498-aae8-4c34-8527-01234a2193a2: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:42:30,476::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:42:30,477::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:42:30,478::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:42:30,479::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:42:30,480::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:42:30,480::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:42:30,481::task::495::TaskManager.Task::(_debug) Task c9d84498-aae8-4c34-8527-01234a2193a2: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:42:30,482::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:42:30,723::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.08641 s, 11.9 MB/s\n'; <rc> = 0
>Thread-15914453::INFO::2013-11-20 04:42:32,510::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:42:32,515::task::495::TaskManager.Task::(_debug) Task 3fcbcb8b-a730-431b-baaf-a65e192f8983: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:42:32,517::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2843d1f4-61e1-4ecf-8bd7-a6eb15ed9093`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:42:32,518::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:42:32,528::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:42:32,529::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2843d1f4-61e1-4ecf-8bd7-a6eb15ed9093`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:42:32,530::task::495::TaskManager.Task::(_debug) Task 3fcbcb8b-a730-431b-baaf-a65e192f8983: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:42:32,531::task::495::TaskManager.Task::(_debug) Task 3fcbcb8b-a730-431b-baaf-a65e192f8983: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:42:32,532::task::495::TaskManager.Task::(_debug) Task 3fcbcb8b-a730-431b-baaf-a65e192f8983: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:42:32,533::task::495::TaskManager.Task::(_debug) Task 3fcbcb8b-a730-431b-baaf-a65e192f8983: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:42:32,534::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:42:32,535::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:42:32,536::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:42:32,537::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:42:32,537::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:42:32,538::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:42:32,539::task::495::TaskManager.Task::(_debug) Task 3fcbcb8b-a730-431b-baaf-a65e192f8983: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:42:32,540::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:42:32,737::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:42:33,319::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0835514 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:42:35,333::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:42:35,882::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0872443 s, 11.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:42:37,897::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:42:38,435::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0782962 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:42:40,448::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:42:40,981::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0767783 s, 13.3 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 04:42:42,674::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:42:42,676::task::495::TaskManager.Task::(_debug) Task 6ac5db12-f962-4a81-a8bd-99715a8937e2: moving from state init -> state preparing
>Thread-15914519::INFO::2013-11-20 04:42:42,677::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15913546::DEBUG::2013-11-20 04:42:42,680::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`96e70164-5a2d-42b0-b626-38d796805e25`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:42:42,681::task::495::TaskManager.Task::(_debug) Task 4e2d2b20-f4d3-4b40-a9dc-d9df87677aeb: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:42:42,689::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:42:42,691::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d44749dc-3bf0-4ff0-9649-e796831bb11b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:42:42,693::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:42:42,693::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:42:42,695::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`96e70164-5a2d-42b0-b626-38d796805e25`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:42:42,696::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:42:42,696::task::495::TaskManager.Task::(_debug) Task 6ac5db12-f962-4a81-a8bd-99715a8937e2: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15914519::DEBUG::2013-11-20 04:42:42,697::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d44749dc-3bf0-4ff0-9649-e796831bb11b`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:42:42,698::task::495::TaskManager.Task::(_debug) Task 6ac5db12-f962-4a81-a8bd-99715a8937e2: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:42:42,699::task::495::TaskManager.Task::(_debug) Task 4e2d2b20-f4d3-4b40-a9dc-d9df87677aeb: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913546::DEBUG::2013-11-20 04:42:42,700::task::495::TaskManager.Task::(_debug) Task 6ac5db12-f962-4a81-a8bd-99715a8937e2: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 04:42:42,701::task::495::TaskManager.Task::(_debug) Task 6ac5db12-f962-4a81-a8bd-99715a8937e2: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:42:42,702::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:42:42,702::task::495::TaskManager.Task::(_debug) Task 4e2d2b20-f4d3-4b40-a9dc-d9df87677aeb: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:42:42,704::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:42:42,704::task::495::TaskManager.Task::(_debug) Task 4e2d2b20-f4d3-4b40-a9dc-d9df87677aeb: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:42:42,706::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15914519::DEBUG::2013-11-20 04:42:42,706::task::495::TaskManager.Task::(_debug) Task 4e2d2b20-f4d3-4b40-a9dc-d9df87677aeb: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:42:42,707::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:42:42,708::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:42:42,709::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:42:42,709::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:42:42,710::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:42:42,711::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913546::DEBUG::2013-11-20 04:42:42,712::task::495::TaskManager.Task::(_debug) Task 6ac5db12-f962-4a81-a8bd-99715a8937e2: ref 0 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:42:42,713::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913546::INFO::2013-11-20 04:42:42,713::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:42:42,714::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913546::INFO::2013-11-20 04:42:42,715::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15914519::DEBUG::2013-11-20 04:42:42,716::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:42:42,716::task::495::TaskManager.Task::(_debug) Task 8908b108-4844-4e9c-9210-5a9b6d6602db: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:42:42,717::task::495::TaskManager.Task::(_debug) Task 4e2d2b20-f4d3-4b40-a9dc-d9df87677aeb: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:42:42,719::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`aa7094ab-50b1-47b2-9deb-8c69815bbda5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::INFO::2013-11-20 04:42:42,720::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:42:42,720::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:42:42,723::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:42:42,723::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`aa7094ab-50b1-47b2-9deb-8c69815bbda5`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:42:42,725::task::495::TaskManager.Task::(_debug) Task 8908b108-4844-4e9c-9210-5a9b6d6602db: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:42:42,726::task::495::TaskManager.Task::(_debug) Task 8908b108-4844-4e9c-9210-5a9b6d6602db: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:42:42,727::task::495::TaskManager.Task::(_debug) Task 8908b108-4844-4e9c-9210-5a9b6d6602db: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:42:42,727::task::495::TaskManager.Task::(_debug) Task 8908b108-4844-4e9c-9210-5a9b6d6602db: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:42:42,728::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:42:42,728::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:42:42,729::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:42:42,730::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:42:42,730::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:42:42,731::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:42:42,732::task::495::TaskManager.Task::(_debug) Task 8908b108-4844-4e9c-9210-5a9b6d6602db: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:42:42,732::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:42:42,733::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:42:42,734::task::495::TaskManager.Task::(_debug) Task 902f2175-d01f-4fda-9bba-5efb0e84c631: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:42:42,736::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`ed45d90d-4a4c-4b54-bb23-470347d8edeb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:42:42,736::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:42:42,737::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:42:42,738::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`ed45d90d-4a4c-4b54-bb23-470347d8edeb`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:42:42,739::task::495::TaskManager.Task::(_debug) Task 902f2175-d01f-4fda-9bba-5efb0e84c631: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:42:42,739::task::495::TaskManager.Task::(_debug) Task 902f2175-d01f-4fda-9bba-5efb0e84c631: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:42:42,740::task::495::TaskManager.Task::(_debug) Task 902f2175-d01f-4fda-9bba-5efb0e84c631: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:42:42,741::task::495::TaskManager.Task::(_debug) Task 902f2175-d01f-4fda-9bba-5efb0e84c631: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:42:42,741::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:42:42,742::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:42:42,742::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:42:42,743::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:42:42,744::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:42:42,744::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:42:42,749::task::495::TaskManager.Task::(_debug) Task 902f2175-d01f-4fda-9bba-5efb0e84c631: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:42:42,750::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:42:42,751::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:42:42,751::task::495::TaskManager.Task::(_debug) Task 475a1e59-3987-42ba-87eb-25590c1412d0: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:42:42,753::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`419a1e0c-05a1-447b-8d6f-fa464a5750e4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:42:42,753::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:42:42,754::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:42:42,755::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`419a1e0c-05a1-447b-8d6f-fa464a5750e4`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:42:42,755::task::495::TaskManager.Task::(_debug) Task 475a1e59-3987-42ba-87eb-25590c1412d0: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:42:42,756::task::495::TaskManager.Task::(_debug) Task 475a1e59-3987-42ba-87eb-25590c1412d0: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:42:42,757::task::495::TaskManager.Task::(_debug) Task 475a1e59-3987-42ba-87eb-25590c1412d0: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:42:42,757::task::495::TaskManager.Task::(_debug) Task 475a1e59-3987-42ba-87eb-25590c1412d0: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:42:42,758::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:42:42,758::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:42:42,759::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:42:42,760::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:42:42,760::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:42:42,761::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:42:42,761::task::495::TaskManager.Task::(_debug) Task 475a1e59-3987-42ba-87eb-25590c1412d0: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:42:42,762::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:42:42,994::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:42:43,576::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0878506 s, 11.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:42:45,591::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:42:46,120::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.079676 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:42:48,137::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:42:48,679::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0770317 s, 13.3 MB/s\n'; <rc> = 0
>Thread-15914435::INFO::2013-11-20 04:42:48,731::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:42:48,732::task::495::TaskManager.Task::(_debug) Task 6e1ef44b-f5f0-44ab-ab77-a27cb8065fe6: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:42:48,734::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`dddaa587-faac-453e-809a-3deb05560e7f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:42:48,735::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:42:48,736::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:42:48,737::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`dddaa587-faac-453e-809a-3deb05560e7f`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:42:48,738::task::495::TaskManager.Task::(_debug) Task 6e1ef44b-f5f0-44ab-ab77-a27cb8065fe6: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:42:48,739::task::495::TaskManager.Task::(_debug) Task 6e1ef44b-f5f0-44ab-ab77-a27cb8065fe6: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:42:48,740::task::495::TaskManager.Task::(_debug) Task 6e1ef44b-f5f0-44ab-ab77-a27cb8065fe6: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:42:48,740::task::495::TaskManager.Task::(_debug) Task 6e1ef44b-f5f0-44ab-ab77-a27cb8065fe6: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:42:48,741::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:42:48,742::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:42:48,743::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:42:48,743::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:42:48,744::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:42:48,745::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:42:48,745::task::495::TaskManager.Task::(_debug) Task 6e1ef44b-f5f0-44ab-ab77-a27cb8065fe6: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:42:48,746::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:42:50,692::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:42:51,224::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0773311 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:42:53,241::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:42:53,789::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0788584 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:42:55,804::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:42:56,346::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0795206 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:42:58,363::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:42:58,940::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0885042 s, 11.6 MB/s\n'; <rc> = 0
>Thread-15913639::INFO::2013-11-20 04:43:00,109::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:43:00,110::task::495::TaskManager.Task::(_debug) Task 291d3c3c-c401-4168-b864-f2e6bdb0e5af: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:43:00,112::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`657c5de7-cd94-40ed-8e51-0b9900e4fff3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:43:00,113::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:43:00,114::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:43:00,115::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`657c5de7-cd94-40ed-8e51-0b9900e4fff3`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:43:00,116::task::495::TaskManager.Task::(_debug) Task 291d3c3c-c401-4168-b864-f2e6bdb0e5af: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:43:00,117::task::495::TaskManager.Task::(_debug) Task 291d3c3c-c401-4168-b864-f2e6bdb0e5af: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:43:00,118::task::495::TaskManager.Task::(_debug) Task 291d3c3c-c401-4168-b864-f2e6bdb0e5af: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:43:00,119::task::495::TaskManager.Task::(_debug) Task 291d3c3c-c401-4168-b864-f2e6bdb0e5af: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:43:00,119::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:43:00,120::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:43:00,121::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:43:00,122::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:43:00,123::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:43:00,123::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:43:00,124::task::495::TaskManager.Task::(_debug) Task 291d3c3c-c401-4168-b864-f2e6bdb0e5af: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:43:00,125::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:43:00,953::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:43:01,497::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0757873 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:43:03,514::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915918::INFO::2013-11-20 04:43:03,727::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:43:03,730::task::495::TaskManager.Task::(_debug) Task 7fd6b795-fa9d-4bc5-8fbd-d222186abf08: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:43:03,733::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c0e2a604-1225-4bbe-96d8-08aa10afd7fd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:43:03,734::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:43:03,735::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:43:03,736::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c0e2a604-1225-4bbe-96d8-08aa10afd7fd`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:43:03,737::task::495::TaskManager.Task::(_debug) Task 7fd6b795-fa9d-4bc5-8fbd-d222186abf08: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:43:03,738::task::495::TaskManager.Task::(_debug) Task 7fd6b795-fa9d-4bc5-8fbd-d222186abf08: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:43:03,739::task::495::TaskManager.Task::(_debug) Task 7fd6b795-fa9d-4bc5-8fbd-d222186abf08: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:43:03,740::task::495::TaskManager.Task::(_debug) Task 7fd6b795-fa9d-4bc5-8fbd-d222186abf08: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:43:03,741::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:43:03,741::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:43:03,742::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:43:03,743::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:43:03,743::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:43:03,744::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:43:03,745::task::495::TaskManager.Task::(_debug) Task 7fd6b795-fa9d-4bc5-8fbd-d222186abf08: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:43:03,745::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:43:04,059::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0754261 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:43:06,075::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:43:06,612::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0788708 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15915863::INFO::2013-11-20 04:43:07,726::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:43:07,729::task::495::TaskManager.Task::(_debug) Task 700dbb44-97a2-4231-afde-adcc988f51c2: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:43:07,734::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`95fd0488-cc20-46a9-9a5d-e24711214702`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:43:07,738::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:43:07,744::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:43:07,744::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`95fd0488-cc20-46a9-9a5d-e24711214702`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:43:07,746::task::495::TaskManager.Task::(_debug) Task 700dbb44-97a2-4231-afde-adcc988f51c2: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:43:07,746::task::495::TaskManager.Task::(_debug) Task 700dbb44-97a2-4231-afde-adcc988f51c2: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:43:07,747::task::495::TaskManager.Task::(_debug) Task 700dbb44-97a2-4231-afde-adcc988f51c2: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:43:07,748::task::495::TaskManager.Task::(_debug) Task 700dbb44-97a2-4231-afde-adcc988f51c2: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:43:07,748::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:43:07,749::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:43:07,750::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:43:07,751::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:43:07,751::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:43:07,752::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:43:07,753::task::495::TaskManager.Task::(_debug) Task 700dbb44-97a2-4231-afde-adcc988f51c2: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:43:07,754::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:43:08,630::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:43:09,182::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0805776 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:43:11,194::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:43:11,739::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.075879 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 04:43:12,952::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:43:12,963::task::495::TaskManager.Task::(_debug) Task e014aacd-02ec-4829-b1b6-56d14100d0e9: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:43:12,966::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7d7029c2-8679-4304-8747-6cef02037f31`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:43:12,966::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:43:12,967::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:43:12,968::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7d7029c2-8679-4304-8747-6cef02037f31`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:43:12,969::task::495::TaskManager.Task::(_debug) Task e014aacd-02ec-4829-b1b6-56d14100d0e9: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:43:12,970::task::495::TaskManager.Task::(_debug) Task e014aacd-02ec-4829-b1b6-56d14100d0e9: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:43:12,970::task::495::TaskManager.Task::(_debug) Task e014aacd-02ec-4829-b1b6-56d14100d0e9: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:43:12,971::task::495::TaskManager.Task::(_debug) Task e014aacd-02ec-4829-b1b6-56d14100d0e9: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:43:12,972::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:43:12,973::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:43:12,974::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:43:12,975::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:43:12,976::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:43:12,976::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:43:12,977::task::495::TaskManager.Task::(_debug) Task e014aacd-02ec-4829-b1b6-56d14100d0e9: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:43:12,978::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:43:13,755::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:43:14,299::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.080631 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:43:16,312::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:43:16,896::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0822783 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:43:18,911::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913518::INFO::2013-11-20 04:43:19,142::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:43:19,143::task::495::TaskManager.Task::(_debug) Task d91b3ce9-50ec-4818-8767-f7f989864d09: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:43:19,146::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3bc377eb-0d41-475d-bcae-107283c8064a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:43:19,147::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:43:19,148::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:43:19,148::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3bc377eb-0d41-475d-bcae-107283c8064a`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:43:19,150::task::495::TaskManager.Task::(_debug) Task d91b3ce9-50ec-4818-8767-f7f989864d09: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:43:19,151::task::495::TaskManager.Task::(_debug) Task d91b3ce9-50ec-4818-8767-f7f989864d09: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:43:19,152::task::495::TaskManager.Task::(_debug) Task d91b3ce9-50ec-4818-8767-f7f989864d09: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:43:19,152::task::495::TaskManager.Task::(_debug) Task d91b3ce9-50ec-4818-8767-f7f989864d09: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:43:19,153::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:43:19,154::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:43:19,154::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:43:19,155::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:43:19,156::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:43:19,156::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:43:19,157::task::495::TaskManager.Task::(_debug) Task d91b3ce9-50ec-4818-8767-f7f989864d09: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:43:19,158::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:43:19,159::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:43:19,159::task::495::TaskManager.Task::(_debug) Task dfc0693b-e7f6-472a-9c6d-fd1e7f7e19aa: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:43:19,161::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f5e45791-15c7-42ad-bca7-7557d1cab070`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:43:19,161::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:43:19,162::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:43:19,163::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f5e45791-15c7-42ad-bca7-7557d1cab070`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:43:19,164::task::495::TaskManager.Task::(_debug) Task dfc0693b-e7f6-472a-9c6d-fd1e7f7e19aa: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:43:19,164::task::495::TaskManager.Task::(_debug) Task dfc0693b-e7f6-472a-9c6d-fd1e7f7e19aa: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:43:19,165::task::495::TaskManager.Task::(_debug) Task dfc0693b-e7f6-472a-9c6d-fd1e7f7e19aa: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:43:19,166::task::495::TaskManager.Task::(_debug) Task dfc0693b-e7f6-472a-9c6d-fd1e7f7e19aa: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:43:19,166::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:43:19,167::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:43:19,168::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:43:19,169::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:43:19,169::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:43:19,170::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:43:19,171::task::495::TaskManager.Task::(_debug) Task dfc0693b-e7f6-472a-9c6d-fd1e7f7e19aa: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:43:19,172::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:43:19,463::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.078987 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:43:21,485::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:43:22,034::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0749487 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:43:24,050::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 04:43:24,246::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:43:24,249::task::495::TaskManager.Task::(_debug) Task e2298882-98ff-47d9-b932-677a4f67b491: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:43:24,253::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`cec90abe-feb7-478b-987c-ad05501fb5ea`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:43:24,254::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:43:24,256::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:43:24,257::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`cec90abe-feb7-478b-987c-ad05501fb5ea`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:43:24,259::task::495::TaskManager.Task::(_debug) Task e2298882-98ff-47d9-b932-677a4f67b491: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:43:24,260::task::495::TaskManager.Task::(_debug) Task e2298882-98ff-47d9-b932-677a4f67b491: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:43:24,262::task::495::TaskManager.Task::(_debug) Task e2298882-98ff-47d9-b932-677a4f67b491: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:43:24,263::task::495::TaskManager.Task::(_debug) Task e2298882-98ff-47d9-b932-677a4f67b491: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:43:24,264::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:43:24,265::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:43:24,266::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:43:24,267::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:43:24,268::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:43:24,269::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:43:24,270::task::495::TaskManager.Task::(_debug) Task e2298882-98ff-47d9-b932-677a4f67b491: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:43:24,271::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:43:24,577::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.079778 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:43:26,593::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:43:27,165::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0784848 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:43:29,183::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:43:30,035::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.407061 s, 2.5 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 04:43:31,899::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:43:31,900::task::495::TaskManager.Task::(_debug) Task 9cd1fe01-4497-4cca-b701-6f013b900aef: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:43:31,903::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`68435eb7-261f-48ea-890a-6b4063b3fb55`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:43:31,904::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:43:31,905::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:43:31,906::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`68435eb7-261f-48ea-890a-6b4063b3fb55`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:43:31,907::task::495::TaskManager.Task::(_debug) Task 9cd1fe01-4497-4cca-b701-6f013b900aef: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:43:31,908::task::495::TaskManager.Task::(_debug) Task 9cd1fe01-4497-4cca-b701-6f013b900aef: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:43:31,909::task::495::TaskManager.Task::(_debug) Task 9cd1fe01-4497-4cca-b701-6f013b900aef: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:43:31,910::task::495::TaskManager.Task::(_debug) Task 9cd1fe01-4497-4cca-b701-6f013b900aef: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:43:31,911::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:43:31,911::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:43:31,912::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:43:31,913::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:43:31,913::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:43:31,914::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:43:31,914::task::495::TaskManager.Task::(_debug) Task 9cd1fe01-4497-4cca-b701-6f013b900aef: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:43:31,915::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:43:32,050::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:43:32,588::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0768327 s, 13.3 MB/s\n'; <rc> = 0
>Thread-42838::DEBUG::2013-11-20 04:43:32,873::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15914453::INFO::2013-11-20 04:43:33,716::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:43:33,720::task::495::TaskManager.Task::(_debug) Task 54b64b2d-8f07-480c-8d2b-b8dbee3c5d58: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:43:33,722::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5414840a-17b5-4b0e-870c-b7857754ed25`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:43:33,723::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:43:33,725::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:43:33,726::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5414840a-17b5-4b0e-870c-b7857754ed25`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:43:33,728::task::495::TaskManager.Task::(_debug) Task 54b64b2d-8f07-480c-8d2b-b8dbee3c5d58: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:43:33,728::task::495::TaskManager.Task::(_debug) Task 54b64b2d-8f07-480c-8d2b-b8dbee3c5d58: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:43:33,729::task::495::TaskManager.Task::(_debug) Task 54b64b2d-8f07-480c-8d2b-b8dbee3c5d58: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:43:33,729::task::495::TaskManager.Task::(_debug) Task 54b64b2d-8f07-480c-8d2b-b8dbee3c5d58: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:43:33,730::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:43:33,730::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:43:33,731::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:43:33,731::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:43:33,732::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:43:33,732::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:43:33,733::task::495::TaskManager.Task::(_debug) Task 54b64b2d-8f07-480c-8d2b-b8dbee3c5d58: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:43:33,734::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-42838::DEBUG::2013-11-20 04:43:34,085::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:43:34,602::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:43:35,180::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0841225 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:43:37,192::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:43:37,747::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0801008 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:43:39,769::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:43:40,291::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0783737 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:43:42,306::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:43:42,791::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0747629 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 04:43:43,745::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15914519::INFO::2013-11-20 04:43:43,746::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15913546::DEBUG::2013-11-20 04:43:43,753::task::495::TaskManager.Task::(_debug) Task 924ccd11-ae54-444b-a4d4-cacbd9311af8: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:43:43,754::task::495::TaskManager.Task::(_debug) Task 0e05dc92-150d-4b03-9b3b-5aed5343dcc3: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:43:43,757::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c4d09721-a81d-4ee7-b5d2-97b4bff7daf1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:43:43,759::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`98b84a70-6717-474a-b931-03f1b5ca55d3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:43:43,760::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:43:43,761::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:43:43,763::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:43:43,763::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`98b84a70-6717-474a-b931-03f1b5ca55d3`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:43:43,771::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:43:43,772::task::495::TaskManager.Task::(_debug) Task 0e05dc92-150d-4b03-9b3b-5aed5343dcc3: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913546::DEBUG::2013-11-20 04:43:43,773::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c4d09721-a81d-4ee7-b5d2-97b4bff7daf1`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:43:43,774::task::495::TaskManager.Task::(_debug) Task 0e05dc92-150d-4b03-9b3b-5aed5343dcc3: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:43:43,776::task::495::TaskManager.Task::(_debug) Task 924ccd11-ae54-444b-a4d4-cacbd9311af8: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15914519::DEBUG::2013-11-20 04:43:43,777::task::495::TaskManager.Task::(_debug) Task 0e05dc92-150d-4b03-9b3b-5aed5343dcc3: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:43:43,778::task::495::TaskManager.Task::(_debug) Task 924ccd11-ae54-444b-a4d4-cacbd9311af8: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:43:43,779::task::495::TaskManager.Task::(_debug) Task 0e05dc92-150d-4b03-9b3b-5aed5343dcc3: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:43:43,779::task::495::TaskManager.Task::(_debug) Task 924ccd11-ae54-444b-a4d4-cacbd9311af8: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:43:43,780::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:43:43,781::task::495::TaskManager.Task::(_debug) Task 924ccd11-ae54-444b-a4d4-cacbd9311af8: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:43:43,782::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:43:43,782::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:43:43,784::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913546::DEBUG::2013-11-20 04:43:43,784::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:43:43,785::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:43:43,786::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15914519::DEBUG::2013-11-20 04:43:43,787::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:43:43,788::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:43:43,789::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:43:43,789::task::495::TaskManager.Task::(_debug) Task 0e05dc92-150d-4b03-9b3b-5aed5343dcc3: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:43:43,790::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15914519::INFO::2013-11-20 04:43:43,790::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:43:43,791::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:43:43,792::task::495::TaskManager.Task::(_debug) Task 924ccd11-ae54-444b-a4d4-cacbd9311af8: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:43:43,793::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:43:43,795::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:43:43,795::task::495::TaskManager.Task::(_debug) Task 0559d7b7-0c5c-476f-b749-c1ac29e5620d: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:43:43,797::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2b2d1e5f-ed0b-4aac-b437-5620b936f92c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:43:43,797::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:43:43,798::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:43:43,799::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`2b2d1e5f-ed0b-4aac-b437-5620b936f92c`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:43:43,799::task::495::TaskManager.Task::(_debug) Task 0559d7b7-0c5c-476f-b749-c1ac29e5620d: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:43:43,800::task::495::TaskManager.Task::(_debug) Task 0559d7b7-0c5c-476f-b749-c1ac29e5620d: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:43:43,801::task::495::TaskManager.Task::(_debug) Task 0559d7b7-0c5c-476f-b749-c1ac29e5620d: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:43:43,801::task::495::TaskManager.Task::(_debug) Task 0559d7b7-0c5c-476f-b749-c1ac29e5620d: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:43:43,802::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:43:43,802::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:43:43,804::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:43:43,804::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:43:43,805::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:43:43,805::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:43:43,806::task::495::TaskManager.Task::(_debug) Task 0559d7b7-0c5c-476f-b749-c1ac29e5620d: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:43:43,806::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:43:43,807::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:43:43,808::task::495::TaskManager.Task::(_debug) Task 7d9e0867-8de7-4535-80cd-ce60b4bf56f8: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:43:43,809::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c3f30bff-9f74-40a8-a14c-0d087d550a6a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:43:43,810::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:43:43,811::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:43:43,811::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c3f30bff-9f74-40a8-a14c-0d087d550a6a`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:43:43,812::task::495::TaskManager.Task::(_debug) Task 7d9e0867-8de7-4535-80cd-ce60b4bf56f8: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:43:43,813::task::495::TaskManager.Task::(_debug) Task 7d9e0867-8de7-4535-80cd-ce60b4bf56f8: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:43:43,814::task::495::TaskManager.Task::(_debug) Task 7d9e0867-8de7-4535-80cd-ce60b4bf56f8: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:43:43,814::task::495::TaskManager.Task::(_debug) Task 7d9e0867-8de7-4535-80cd-ce60b4bf56f8: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:43:43,815::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:43:43,815::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:43:43,819::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:43:43,820::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:43:43,821::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:43:43,822::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:43:43,822::task::495::TaskManager.Task::(_debug) Task 7d9e0867-8de7-4535-80cd-ce60b4bf56f8: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:43:43,823::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:43:43,824::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:43:43,824::task::495::TaskManager.Task::(_debug) Task a705537a-bf48-4a31-ae0b-b878546c33fa: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:43:43,826::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`980a1690-449b-451c-9ce1-4af2cc01c40e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:43:43,826::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:43:43,827::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:43:43,827::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`980a1690-449b-451c-9ce1-4af2cc01c40e`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:43:43,828::task::495::TaskManager.Task::(_debug) Task a705537a-bf48-4a31-ae0b-b878546c33fa: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:43:43,829::task::495::TaskManager.Task::(_debug) Task a705537a-bf48-4a31-ae0b-b878546c33fa: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:43:43,830::task::495::TaskManager.Task::(_debug) Task a705537a-bf48-4a31-ae0b-b878546c33fa: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:43:43,830::task::495::TaskManager.Task::(_debug) Task a705537a-bf48-4a31-ae0b-b878546c33fa: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:43:43,831::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:43:43,832::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:43:43,832::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:43:43,833::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:43:43,833::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:43:43,834::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:43:43,835::task::495::TaskManager.Task::(_debug) Task a705537a-bf48-4a31-ae0b-b878546c33fa: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:43:43,835::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:43:44,806::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:43:45,273::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0748159 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15916814::DEBUG::2013-11-20 04:43:47,005::misc::1156::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Emitting event
>Thread-15916815::DEBUG::2013-11-20 04:43:47,007::misc::1156::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Emitting event
>Thread-15916814::DEBUG::2013-11-20 04:43:47,008::misc::1166::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Calling registered method `run`
>Thread-15916815::DEBUG::2013-11-20 04:43:47,011::misc::1166::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Calling registered method `run`
>Thread-15916814::DEBUG::2013-11-20 04:43:47,011::misc::1174::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Event emitted
>Thread-15916815::DEBUG::2013-11-20 04:43:47,013::misc::1174::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Event emitted
>Thread-15916817::DEBUG::2013-11-20 04:43:47,014::spm::365::Storage.SPM::(_upgradePoolDomain) Preparing to upgrade domain 5d7b5178-3790-472c-a13b-b481eb27a61e
>Thread-15916817::ERROR::2013-11-20 04:43:47,017::spm::373::Storage.SPM::(_upgradePoolDomain) Error while preparing domain `5d7b5178-3790-472c-a13b-b481eb27a61e` upgrade
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/spm.py", line 369, in _upgradePoolDomain
>IndexError: list index out of range
>Dummy-15912651::DEBUG::2013-11-20 04:43:47,288::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:43:47,806::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0749686 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:43:49,821::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 04:43:50,058::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:43:50,060::task::495::TaskManager.Task::(_debug) Task 67f3b37e-7bc6-4cb3-b469-94d69468dd09: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:43:50,063::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`871ffba0-cf28-4a62-8949-66841b1c7a94`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:43:50,064::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:43:50,065::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:43:50,065::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`871ffba0-cf28-4a62-8949-66841b1c7a94`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:43:50,067::task::495::TaskManager.Task::(_debug) Task 67f3b37e-7bc6-4cb3-b469-94d69468dd09: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:43:50,068::task::495::TaskManager.Task::(_debug) Task 67f3b37e-7bc6-4cb3-b469-94d69468dd09: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:43:50,070::task::495::TaskManager.Task::(_debug) Task 67f3b37e-7bc6-4cb3-b469-94d69468dd09: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:43:50,071::task::495::TaskManager.Task::(_debug) Task 67f3b37e-7bc6-4cb3-b469-94d69468dd09: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:43:50,071::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:43:50,072::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:43:50,073::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:43:50,074::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:43:50,075::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:43:50,076::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:43:50,077::task::495::TaskManager.Task::(_debug) Task 67f3b37e-7bc6-4cb3-b469-94d69468dd09: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:43:50,078::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:43:50,373::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0875112 s, 11.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:43:52,389::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:43:52,935::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0759753 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:43:54,950::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:43:55,471::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0786188 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:43:57,486::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:43:58,039::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0800358 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:44:00,058::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:44:00,588::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0821546 s, 12.5 MB/s\n'; <rc> = 0
>Thread-15913639::INFO::2013-11-20 04:44:00,637::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:44:00,639::task::495::TaskManager.Task::(_debug) Task 19f517b8-03db-43c3-9a1c-719faa2196d2: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:44:00,641::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`374ac48b-9ba3-4de4-b8f1-c395d730d742`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:44:00,642::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:44:00,643::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:44:00,644::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`374ac48b-9ba3-4de4-b8f1-c395d730d742`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:44:00,645::task::495::TaskManager.Task::(_debug) Task 19f517b8-03db-43c3-9a1c-719faa2196d2: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:44:00,647::task::495::TaskManager.Task::(_debug) Task 19f517b8-03db-43c3-9a1c-719faa2196d2: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:44:00,647::task::495::TaskManager.Task::(_debug) Task 19f517b8-03db-43c3-9a1c-719faa2196d2: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:44:00,648::task::495::TaskManager.Task::(_debug) Task 19f517b8-03db-43c3-9a1c-719faa2196d2: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:44:00,649::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:44:00,650::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:44:00,651::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:44:00,652::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:44:00,653::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:44:00,653::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:44:00,654::task::495::TaskManager.Task::(_debug) Task 19f517b8-03db-43c3-9a1c-719faa2196d2: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:44:00,655::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:44:02,606::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:44:03,095::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0794513 s, 12.9 MB/s\n'; <rc> = 0
>Thread-15915918::INFO::2013-11-20 04:44:04,804::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:44:04,805::task::495::TaskManager.Task::(_debug) Task 5bf451a2-8932-4211-8824-17e2eee26819: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:44:04,808::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1ec2ba31-00cb-4de7-a06c-6985b6620ea6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:44:04,809::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:44:04,811::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:44:04,811::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1ec2ba31-00cb-4de7-a06c-6985b6620ea6`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:44:04,825::task::495::TaskManager.Task::(_debug) Task 5bf451a2-8932-4211-8824-17e2eee26819: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:44:04,826::task::495::TaskManager.Task::(_debug) Task 5bf451a2-8932-4211-8824-17e2eee26819: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:44:04,827::task::495::TaskManager.Task::(_debug) Task 5bf451a2-8932-4211-8824-17e2eee26819: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:44:04,828::task::495::TaskManager.Task::(_debug) Task 5bf451a2-8932-4211-8824-17e2eee26819: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:44:04,830::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:44:04,830::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:44:04,832::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:44:04,833::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:44:04,834::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:44:04,835::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:44:04,836::task::495::TaskManager.Task::(_debug) Task 5bf451a2-8932-4211-8824-17e2eee26819: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:44:04,836::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:44:05,109::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:44:05,650::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0818102 s, 12.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:44:07,666::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:44:08,198::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0784893 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15915863::INFO::2013-11-20 04:44:09,318::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:44:09,319::task::495::TaskManager.Task::(_debug) Task 0a612f56-982e-43de-bb5e-e2381a6a6c00: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:44:09,321::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b553e631-8f15-4170-bb87-2f94d538b0e5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:44:09,322::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:44:09,323::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:44:09,324::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b553e631-8f15-4170-bb87-2f94d538b0e5`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:44:09,325::task::495::TaskManager.Task::(_debug) Task 0a612f56-982e-43de-bb5e-e2381a6a6c00: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:44:09,326::task::495::TaskManager.Task::(_debug) Task 0a612f56-982e-43de-bb5e-e2381a6a6c00: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:44:09,327::task::495::TaskManager.Task::(_debug) Task 0a612f56-982e-43de-bb5e-e2381a6a6c00: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:44:09,328::task::495::TaskManager.Task::(_debug) Task 0a612f56-982e-43de-bb5e-e2381a6a6c00: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:44:09,329::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:44:09,329::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:44:09,330::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:44:09,331::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:44:09,332::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:44:09,332::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:44:09,333::task::495::TaskManager.Task::(_debug) Task 0a612f56-982e-43de-bb5e-e2381a6a6c00: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:44:09,334::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:44:10,212::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:44:10,770::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.107996 s, 9.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:44:12,785::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:44:13,307::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0738903 s, 13.9 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 04:44:13,898::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:44:13,903::task::495::TaskManager.Task::(_debug) Task ac7404ab-c543-4c64-8c11-f5479ec3b8bd: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:44:13,906::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0854dd30-4b90-4c28-bf0d-bd8c90e23a06`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:44:13,913::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:44:13,916::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:44:13,916::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0854dd30-4b90-4c28-bf0d-bd8c90e23a06`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:44:13,917::task::495::TaskManager.Task::(_debug) Task ac7404ab-c543-4c64-8c11-f5479ec3b8bd: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:44:13,918::task::495::TaskManager.Task::(_debug) Task ac7404ab-c543-4c64-8c11-f5479ec3b8bd: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:44:13,919::task::495::TaskManager.Task::(_debug) Task ac7404ab-c543-4c64-8c11-f5479ec3b8bd: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:44:13,920::task::495::TaskManager.Task::(_debug) Task ac7404ab-c543-4c64-8c11-f5479ec3b8bd: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:44:13,920::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:44:13,921::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:44:13,922::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:44:13,922::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:44:13,923::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:44:13,924::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:44:13,924::task::495::TaskManager.Task::(_debug) Task ac7404ab-c543-4c64-8c11-f5479ec3b8bd: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:44:13,925::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:44:15,323::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:44:15,867::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0793714 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:44:17,882::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:44:18,463::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0833637 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:44:20,479::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913518::INFO::2013-11-20 04:44:20,725::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:44:20,726::task::495::TaskManager.Task::(_debug) Task 5e4d96bb-fd9a-4fd7-9a36-480871a845f6: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:44:20,739::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`10b922ff-83cb-403c-a306-063720a96c28`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:44:20,740::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:44:20,741::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:44:20,741::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`10b922ff-83cb-403c-a306-063720a96c28`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:44:20,743::task::495::TaskManager.Task::(_debug) Task 5e4d96bb-fd9a-4fd7-9a36-480871a845f6: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:44:20,743::task::495::TaskManager.Task::(_debug) Task 5e4d96bb-fd9a-4fd7-9a36-480871a845f6: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:44:20,744::task::495::TaskManager.Task::(_debug) Task 5e4d96bb-fd9a-4fd7-9a36-480871a845f6: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:44:20,745::task::495::TaskManager.Task::(_debug) Task 5e4d96bb-fd9a-4fd7-9a36-480871a845f6: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:44:20,746::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:44:20,746::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:44:20,747::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:44:20,748::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:44:20,748::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:44:20,749::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:44:20,750::task::495::TaskManager.Task::(_debug) Task 5e4d96bb-fd9a-4fd7-9a36-480871a845f6: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:44:20,751::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:44:20,751::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:44:20,752::task::495::TaskManager.Task::(_debug) Task 9bc5d13b-731f-4056-8916-3cee0164a206: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:44:20,753::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2b67dade-36df-422c-9600-1fc74653c59f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:44:20,754::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:44:20,754::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:44:20,755::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2b67dade-36df-422c-9600-1fc74653c59f`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:44:20,756::task::495::TaskManager.Task::(_debug) Task 9bc5d13b-731f-4056-8916-3cee0164a206: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:44:20,757::task::495::TaskManager.Task::(_debug) Task 9bc5d13b-731f-4056-8916-3cee0164a206: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:44:20,757::task::495::TaskManager.Task::(_debug) Task 9bc5d13b-731f-4056-8916-3cee0164a206: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:44:20,758::task::495::TaskManager.Task::(_debug) Task 9bc5d13b-731f-4056-8916-3cee0164a206: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:44:20,758::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:44:20,759::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:44:20,760::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:44:20,760::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:44:20,761::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:44:20,761::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:44:20,762::task::495::TaskManager.Task::(_debug) Task 9bc5d13b-731f-4056-8916-3cee0164a206: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:44:20,763::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:44:21,059::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0843134 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15916818::INFO::2013-11-20 04:44:22,751::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916818::DEBUG::2013-11-20 04:44:22,752::task::495::TaskManager.Task::(_debug) Task 8da1d069-4f29-4726-97ef-a29bdb90f445: moving from state init -> state preparing
>Thread-15916818::DEBUG::2013-11-20 04:44:22,753::task::495::TaskManager.Task::(_debug) Task 8da1d069-4f29-4726-97ef-a29bdb90f445: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00045108795166', 'lastCheck': 1384922654.4379001, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00126099586487', 'lastCheck': 1384922657.551379, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00159406661987', 'lastCheck': 1384922657.550519, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000589847564697', 'lastCheck': 1384922659.547255, 'valid': True, 'code': 0}}
>Thread-15916818::DEBUG::2013-11-20 04:44:22,759::task::495::TaskManager.Task::(_debug) Task 8da1d069-4f29-4726-97ef-a29bdb90f445: moving from state preparing -> state finished
>Thread-15916818::DEBUG::2013-11-20 04:44:22,760::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916818::DEBUG::2013-11-20 04:44:22,761::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916818::DEBUG::2013-11-20 04:44:22,762::task::495::TaskManager.Task::(_debug) Task 8da1d069-4f29-4726-97ef-a29bdb90f445: ref 0 aborting False
>Thread-15916818::INFO::2013-11-20 04:44:22,763::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00045108795166', 'lastCheck': 1384922654.4379001, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00126099586487', 'lastCheck': 1384922657.551379, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00159406661987', 'lastCheck': 1384922657.550519, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000589847564697', 'lastCheck': 1384922659.547255, 'valid': True, 'code': 0}}
>Thread-15916819::DEBUG::2013-11-20 04:44:22,970::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVdsCapabilities with () {}
>Dummy-15912651::DEBUG::2013-11-20 04:44:23,220::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916819::DEBUG::2013-11-20 04:44:23,608::utils::579::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-img-rhev' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:44:24,050::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.146491 s, 7.0 MB/s\n'; <rc> = 0
>Thread-15916819::DEBUG::2013-11-20 04:44:24,136::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15916819::DEBUG::2013-11-20 04:44:24,138::utils::579::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-kvm-rhev' (cwd None)
>Thread-15916819::DEBUG::2013-11-20 04:44:24,590::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15916819::DEBUG::2013-11-20 04:44:24,591::utils::579::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" vdsm' (cwd None)
>Thread-15916819::DEBUG::2013-11-20 04:44:25,057::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15916819::DEBUG::2013-11-20 04:44:25,058::utils::579::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" spice-server' (cwd None)
>Thread-15912392::INFO::2013-11-20 04:44:25,264::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:44:25,266::task::495::TaskManager.Task::(_debug) Task 7ef7da06-9207-48ff-89a3-5e64c3738ef7: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:44:25,277::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0f00ed61-c985-46f7-b602-40b275a5863b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:44:25,278::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:44:25,279::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:44:25,280::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0f00ed61-c985-46f7-b602-40b275a5863b`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:44:25,281::task::495::TaskManager.Task::(_debug) Task 7ef7da06-9207-48ff-89a3-5e64c3738ef7: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:44:25,282::task::495::TaskManager.Task::(_debug) Task 7ef7da06-9207-48ff-89a3-5e64c3738ef7: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:44:25,282::task::495::TaskManager.Task::(_debug) Task 7ef7da06-9207-48ff-89a3-5e64c3738ef7: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:44:25,283::task::495::TaskManager.Task::(_debug) Task 7ef7da06-9207-48ff-89a3-5e64c3738ef7: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:44:25,284::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:44:25,284::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:44:25,285::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:44:25,286::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:44:25,286::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:44:25,287::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:44:25,287::task::495::TaskManager.Task::(_debug) Task 7ef7da06-9207-48ff-89a3-5e64c3738ef7: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:44:25,288::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15916819::DEBUG::2013-11-20 04:44:25,480::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15916819::DEBUG::2013-11-20 04:44:25,481::utils::579::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" libvirt' (cwd None)
>Thread-15916819::DEBUG::2013-11-20 04:44:25,934::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15916819::DEBUG::2013-11-20 04:44:25,937::clientIF::59::vds::(wrapper) return getVdsCapabilities with {'status': {'message': 'Done', 'code': 0}, 'info': {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:c936ebb147a8'}], 'FC': []}, 'packages2': {'kernel': {'release': '279.11.1.el6.x86_64', 'buildtime': 1348297826.0, 'version': '2.6.32'}, 'spice-server': {'release': '10.el6', 'buildtime': '1337611492', 'version': '0.10.1'}, 'vdsm': {'release': '113.4.el6_3', 'buildtime': '1347534072', 'version': '4.9'}, 'qemu-kvm': {'release': '2.295.el6_3.2', 'buildtime': '1346673796', 'version': '0.12.1.2'}, 'libvirt': {'release': '21.el6_3.5', 'buildtime': '1348687577', 'version': '0.9.10'}, 'qemu-img': {'release': '2.295.el6_3.2', 'buildtime': '1346673796', 'version': '0.12.1.2'}}, 'cpuModel': 'Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz', 'hooks': {'before_vm_start': {'50_vhostmd': {'md5': '1635e45584995d9ad2629939cb0bcdbf'}}, 'after_vm_destroy': {'50_vhostmd': {'md5': '7e7509430304099f38e4a9de966af090'}}, 'before_vm_dehibernate': {'50_vhostmd': {'md5': '1635e45584995d9ad2629939cb0bcdbf'}}, 'before_vm_migrate_destination': {'50_vhostmd': {'md5': '1635e45584995d9ad2629939cb0bcdbf'}}}, 'vmTypes': ['kvm'], 'supportedProtocols': ['2.2', '2.3'], 'networks': {'rhevm': {'addr': '192.168.100.133', 'cfg': {'IPADDR': '192.168.100.133', 'GATEWAY': '192.168.100.1', 'DELAY': '0', 'NETMASK': '255.255.128.0', 'STP': 'no', 'DEVICE': 'rhevm', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'ports': ['vnet0', 'vnet1', 'vnet2', 'vnet3', 'vnet4', 'bond0', 'vnet5', 'vnet6', 'vnet7', 'vnet8', 'vnet9', 'vnet10', 'vnet11', 'vnet12'], 'netmask': '255.255.128.0', 'stp': 'off', 'gateway': '192.168.100.1'}, 'DMZ': {'addr': '172.16.1.133', 'cfg': {'IPADDR': '172.16.1.133', 'DELAY': '0', 'NETMASK': '255.255.255.0', 'STP': 'no', 'DEVICE': 'DMZ', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'ports': ['eth0'], 'netmask': '255.255.255.0', 'stp': 'off', 'gateway': '0.0.0.0'}}, 'uuid': '30343536-3138-5A43-4A32-333630373442_AC:16:2D:76:EB:F0', 'lastClientIface': 'rhevm', 'nics': {'eth3': {'hwaddr': 'AC:16:2D:76:EB:F2', 'netmask': '', 'speed': 1000, 'addr': '', 'permhwaddr': 'AC:16:2D:76:EB:F3'}, 'eth2': {'hwaddr': 'AC:16:2D:76:EB:F2', 'netmask': '', 'speed': 1000, 'addr': '', 'permhwaddr': 'AC:16:2D:76:EB:F2'}, 'eth1': {'hwaddr': 'AC:16:2D:76:EB:F1', 'netmask': '', 'speed': 0, 'addr': ''}, 'eth0': {'hwaddr': 'AC:16:2D:76:EB:F0', 'netmask': '', 'speed': 1000, 'addr': ''}}, 'software_revision': '113.4', 'management_ip': '192.168.100.133', 'clusterLevels': ['3.0'], 'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,constant_tsc,arch_perfmon,pebs,bts,rep_good,xtopology,nonstop_tsc,aperfmperf,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,dca,sse4_1,sse4_2,x2apic,popcnt,aes,xsave,avx,lahf_lm,ida,arat,epb,xsaveopt,pln,pts,dts,tpr_shadow,vnmi,flexpriority,ept,vpid,model_486,model_pentium,model_pentium2,model_pentium3,model_pentiumpro,model_qemu32,model_cpu64-rhel5,model_cpu64-rhel6,model_kvm32,model_coreduo,model_kvm64,model_core2duo,model_n270,model_Conroe,model_Penryn,model_Nehalem,model_Westmere,model_SandyBridge,model_Opteron_G1', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:c936ebb147a8', 'memSize': '64392', 'reservedMem': '256', 'bondings': {'bond4': {'hwaddr': '00:00:00:00:00:00', 'cfg': {}, 'netmask': '', 'addr': '', 'slaves': []}, 'bond0': {'hwaddr': 'AC:16:2D:76:EB:F2', 'cfg': {'DEVICE': 'bond0', 'BRIDGE': 'rhevm', 'BOOTPROTO': 'none', 'ONBOOT': 'yes', 'BONDING_OPTS': 'mode=1 miimon=100'}, 'netmask': '', 'addr': '', 'slaves': ['eth2', 'eth3']}, 'bond1': {'hwaddr': '00:00:00:00:00:00', 'cfg': {}, 'netmask': '', 'addr': '', 'slaves': []}, 'bond2': {'hwaddr': '00:00:00:00:00:00', 'cfg': {}, 'netmask': '', 'addr': '', 'slaves': []}, 'bond3': {'hwaddr': '00:00:00:00:00:00', 'cfg': {}, 'netmask': '', 'addr': '', 'slaves': []}}, 'software_version': '4.9', 'cpuSpeed': '1995.126', 'cpuSockets': '2', 'vlans': {}, 'cpuCores': '12', 'kvmEnabled': 'true', 'guestOverhead': '65', 'supportedRHEVMs': ['3.0'], 'version_name': 'Snow Man', 'emulatedMachines': [u'rhel6.3.0', u'pc', u'rhel6.2.0', u'rhel6.1.0', u'rhel6.0.0', u'rhel5.5.0', u'rhel5.4.4', u'rhel5.4.0'], 'operatingSystem': {'release': '20121012.0.el6_3', 'version': '6.3', 'name': 'RHEV Hypervisor'}, 'lastClient': '192.168.100.135'}}
>Dummy-15912651::DEBUG::2013-11-20 04:44:26,065::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:44:26,656::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.08658 s, 11.8 MB/s\n'; <rc> = 0
>Thread-15916821::DEBUG::2013-11-20 04:44:28,112::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916821::INFO::2013-11-20 04:44:28,113::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916821::DEBUG::2013-11-20 04:44:28,114::task::495::TaskManager.Task::(_debug) Task 6b8b1339-8dc3-4fab-9f46-e38a3a422b74: moving from state init -> state preparing
>Thread-15916821::DEBUG::2013-11-20 04:44:28,115::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916821::DEBUG::2013-11-20 04:44:28,116::task::495::TaskManager.Task::(_debug) Task 6b8b1339-8dc3-4fab-9f46-e38a3a422b74: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916821::DEBUG::2013-11-20 04:44:28,116::task::495::TaskManager.Task::(_debug) Task 6b8b1339-8dc3-4fab-9f46-e38a3a422b74: moving from state preparing -> state finished
>Thread-15916821::DEBUG::2013-11-20 04:44:28,117::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916821::DEBUG::2013-11-20 04:44:28,118::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916821::DEBUG::2013-11-20 04:44:28,118::task::495::TaskManager.Task::(_debug) Task 6b8b1339-8dc3-4fab-9f46-e38a3a422b74: ref 0 aborting False
>Thread-15916821::INFO::2013-11-20 04:44:28,119::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916822::DEBUG::2013-11-20 04:44:28,203::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916822::INFO::2013-11-20 04:44:28,204::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916822::DEBUG::2013-11-20 04:44:28,205::task::495::TaskManager.Task::(_debug) Task 4f9594d2-bc0e-43d2-b559-c0f55accff51: moving from state init -> state preparing
>Thread-15916822::DEBUG::2013-11-20 04:44:28,206::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e0978c7c-ae25-432f-b5a3-6aeca96b8c94`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916822::DEBUG::2013-11-20 04:44:28,207::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916822::DEBUG::2013-11-20 04:44:28,208::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916822::DEBUG::2013-11-20 04:44:28,209::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e0978c7c-ae25-432f-b5a3-6aeca96b8c94`::Granted request
>Thread-15916822::DEBUG::2013-11-20 04:44:28,210::task::495::TaskManager.Task::(_debug) Task 4f9594d2-bc0e-43d2-b559-c0f55accff51: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916822::DEBUG::2013-11-20 04:44:28,211::task::495::TaskManager.Task::(_debug) Task 4f9594d2-bc0e-43d2-b559-c0f55accff51: ref 1 aborting False
>Thread-15916822::WARNING::2013-11-20 04:44:28,213::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916822::DEBUG::2013-11-20 04:44:28,214::task::495::TaskManager.Task::(_debug) Task 4f9594d2-bc0e-43d2-b559-c0f55accff51: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916822::DEBUG::2013-11-20 04:44:28,222::task::495::TaskManager.Task::(_debug) Task 4f9594d2-bc0e-43d2-b559-c0f55accff51: moving from state preparing -> state finished
>Thread-15916822::DEBUG::2013-11-20 04:44:28,223::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916822::DEBUG::2013-11-20 04:44:28,224::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916822::DEBUG::2013-11-20 04:44:28,224::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916822::DEBUG::2013-11-20 04:44:28,225::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916822::DEBUG::2013-11-20 04:44:28,226::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916822::DEBUG::2013-11-20 04:44:28,226::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916822::DEBUG::2013-11-20 04:44:28,227::task::495::TaskManager.Task::(_debug) Task 4f9594d2-bc0e-43d2-b559-c0f55accff51: ref 0 aborting False
>Thread-15916822::INFO::2013-11-20 04:44:28,228::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916823::DEBUG::2013-11-20 04:44:28,349::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916823::INFO::2013-11-20 04:44:28,350::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksInfo, args: ()
>Thread-15916823::DEBUG::2013-11-20 04:44:28,351::task::495::TaskManager.Task::(_debug) Task d656131a-9772-49a6-af75-418171db7371: moving from state init -> state preparing
>Thread-15916823::DEBUG::2013-11-20 04:44:28,352::taskManager::168::TaskManager::(getAllTasksInfo) Entry.
>Thread-15916823::DEBUG::2013-11-20 04:44:28,353::taskManager::173::TaskManager::(getAllTasksInfo) Return. Response: {}
>Thread-15916823::DEBUG::2013-11-20 04:44:28,354::task::495::TaskManager.Task::(_debug) Task d656131a-9772-49a6-af75-418171db7371: finished: {'allTasksInfo': {}}
>Thread-15916823::DEBUG::2013-11-20 04:44:28,355::task::495::TaskManager.Task::(_debug) Task d656131a-9772-49a6-af75-418171db7371: moving from state preparing -> state finished
>Thread-15916823::DEBUG::2013-11-20 04:44:28,356::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916823::DEBUG::2013-11-20 04:44:28,359::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916823::DEBUG::2013-11-20 04:44:28,360::task::495::TaskManager.Task::(_debug) Task d656131a-9772-49a6-af75-418171db7371: ref 0 aborting False
>Thread-15916823::INFO::2013-11-20 04:44:28,360::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'allTasksInfo': {}}
>Dummy-15912651::DEBUG::2013-11-20 04:44:28,670::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:44:29,207::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0787484 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:44:31,223::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:44:31,767::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0857178 s, 11.9 MB/s\n'; <rc> = 0
>Thread-15916824::DEBUG::2013-11-20 04:44:32,298::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916824::INFO::2013-11-20 04:44:32,301::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916824::DEBUG::2013-11-20 04:44:32,302::task::495::TaskManager.Task::(_debug) Task db65c3c2-0351-4c86-b55a-12e4abd89a86: moving from state init -> state preparing
>Thread-15916824::DEBUG::2013-11-20 04:44:32,303::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916824::DEBUG::2013-11-20 04:44:32,304::task::495::TaskManager.Task::(_debug) Task db65c3c2-0351-4c86-b55a-12e4abd89a86: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916824::DEBUG::2013-11-20 04:44:32,305::task::495::TaskManager.Task::(_debug) Task db65c3c2-0351-4c86-b55a-12e4abd89a86: moving from state preparing -> state finished
>Thread-15916824::DEBUG::2013-11-20 04:44:32,305::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916824::DEBUG::2013-11-20 04:44:32,306::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916824::DEBUG::2013-11-20 04:44:32,307::task::495::TaskManager.Task::(_debug) Task db65c3c2-0351-4c86-b55a-12e4abd89a86: ref 0 aborting False
>Thread-15916824::INFO::2013-11-20 04:44:32,307::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916825::DEBUG::2013-11-20 04:44:32,321::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916825::INFO::2013-11-20 04:44:32,322::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916825::DEBUG::2013-11-20 04:44:32,328::task::495::TaskManager.Task::(_debug) Task 90ddaa23-dd35-48fd-ae6c-ba54622a9310: moving from state init -> state preparing
>Thread-15916825::DEBUG::2013-11-20 04:44:32,329::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`46fdce58-ea6e-4c9b-a30a-c88f856f230c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916825::DEBUG::2013-11-20 04:44:32,330::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916825::DEBUG::2013-11-20 04:44:32,331::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916825::DEBUG::2013-11-20 04:44:32,332::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`46fdce58-ea6e-4c9b-a30a-c88f856f230c`::Granted request
>Thread-15916825::DEBUG::2013-11-20 04:44:32,333::task::495::TaskManager.Task::(_debug) Task 90ddaa23-dd35-48fd-ae6c-ba54622a9310: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916825::DEBUG::2013-11-20 04:44:32,334::task::495::TaskManager.Task::(_debug) Task 90ddaa23-dd35-48fd-ae6c-ba54622a9310: ref 1 aborting False
>Thread-15916825::WARNING::2013-11-20 04:44:32,336::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916825::DEBUG::2013-11-20 04:44:32,346::task::495::TaskManager.Task::(_debug) Task 90ddaa23-dd35-48fd-ae6c-ba54622a9310: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916825::DEBUG::2013-11-20 04:44:32,352::task::495::TaskManager.Task::(_debug) Task 90ddaa23-dd35-48fd-ae6c-ba54622a9310: moving from state preparing -> state finished
>Thread-15916825::DEBUG::2013-11-20 04:44:32,353::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916825::DEBUG::2013-11-20 04:44:32,353::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916825::DEBUG::2013-11-20 04:44:32,354::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916825::DEBUG::2013-11-20 04:44:32,355::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916825::DEBUG::2013-11-20 04:44:32,356::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916825::DEBUG::2013-11-20 04:44:32,356::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916825::DEBUG::2013-11-20 04:44:32,357::task::495::TaskManager.Task::(_debug) Task 90ddaa23-dd35-48fd-ae6c-ba54622a9310: ref 0 aborting False
>Thread-15916825::INFO::2013-11-20 04:44:32,361::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913695::INFO::2013-11-20 04:44:32,958::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:44:32,959::task::495::TaskManager.Task::(_debug) Task 7971be86-56e9-4c4c-9526-a8ac76430850: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:44:32,960::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bed1f514-fdf9-4637-a366-8fc3dc65796a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:44:32,961::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:44:32,962::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:44:32,962::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bed1f514-fdf9-4637-a366-8fc3dc65796a`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:44:32,963::task::495::TaskManager.Task::(_debug) Task 7971be86-56e9-4c4c-9526-a8ac76430850: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:44:32,964::task::495::TaskManager.Task::(_debug) Task 7971be86-56e9-4c4c-9526-a8ac76430850: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:44:32,965::task::495::TaskManager.Task::(_debug) Task 7971be86-56e9-4c4c-9526-a8ac76430850: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:44:32,965::task::495::TaskManager.Task::(_debug) Task 7971be86-56e9-4c4c-9526-a8ac76430850: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:44:32,966::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:44:32,966::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:44:32,967::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:44:32,968::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:44:32,968::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:44:32,969::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:44:32,969::task::495::TaskManager.Task::(_debug) Task 7971be86-56e9-4c4c-9526-a8ac76430850: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:44:32,970::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:44:33,785::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 04:44:34,276::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:44:34,278::task::495::TaskManager.Task::(_debug) Task fc770353-348f-479a-b059-30393562ee57: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:44:34,282::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d8d72f6d-fa2e-4f02-9b1d-45968cc97937`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:44:34,283::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:44:34,285::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:44:34,286::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d8d72f6d-fa2e-4f02-9b1d-45968cc97937`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:44:34,288::task::495::TaskManager.Task::(_debug) Task fc770353-348f-479a-b059-30393562ee57: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:44:34,289::task::495::TaskManager.Task::(_debug) Task fc770353-348f-479a-b059-30393562ee57: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:44:34,290::task::495::TaskManager.Task::(_debug) Task fc770353-348f-479a-b059-30393562ee57: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:44:34,292::task::495::TaskManager.Task::(_debug) Task fc770353-348f-479a-b059-30393562ee57: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:44:34,293::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:44:34,294::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:44:34,295::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:44:34,296::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:44:34,297::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:44:34,298::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:44:34,300::task::495::TaskManager.Task::(_debug) Task fc770353-348f-479a-b059-30393562ee57: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:44:34,301::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:44:34,375::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.113266 s, 9.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:44:36,390::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:44:36,944::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0792727 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:44:38,960::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:44:39,506::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0869624 s, 11.8 MB/s\n'; <rc> = 0
>Thread-15916830::INFO::2013-11-20 04:44:41,252::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916830::DEBUG::2013-11-20 04:44:41,254::task::495::TaskManager.Task::(_debug) Task 1d0621a1-db27-45cc-95d8-fee0b39895df: moving from state init -> state preparing
>Thread-15916830::DEBUG::2013-11-20 04:44:41,255::task::495::TaskManager.Task::(_debug) Task 1d0621a1-db27-45cc-95d8-fee0b39895df: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000488996505737', 'lastCheck': 1384922674.9859719, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00217008590698', 'lastCheck': 1384922677.990953, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00170397758484', 'lastCheck': 1384922677.9917951, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000483989715576', 'lastCheck': 1384922680.2097809, 'valid': True, 'code': 0}}
>Thread-15916830::DEBUG::2013-11-20 04:44:41,264::task::495::TaskManager.Task::(_debug) Task 1d0621a1-db27-45cc-95d8-fee0b39895df: moving from state preparing -> state finished
>Thread-15916830::DEBUG::2013-11-20 04:44:41,267::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916830::DEBUG::2013-11-20 04:44:41,269::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916830::DEBUG::2013-11-20 04:44:41,270::task::495::TaskManager.Task::(_debug) Task 1d0621a1-db27-45cc-95d8-fee0b39895df: ref 0 aborting False
>Thread-15916830::INFO::2013-11-20 04:44:41,274::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000488996505737', 'lastCheck': 1384922674.9859719, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00217008590698', 'lastCheck': 1384922677.990953, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00170397758484', 'lastCheck': 1384922677.9917951, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000483989715576', 'lastCheck': 1384922680.2097809, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:44:41,522::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:44:42,079::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0772326 s, 13.3 MB/s\n'; <rc> = 0
>Thread-15916832::DEBUG::2013-11-20 04:44:43,117::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916832::INFO::2013-11-20 04:44:43,119::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916832::DEBUG::2013-11-20 04:44:43,120::task::495::TaskManager.Task::(_debug) Task 03a47515-379e-48aa-835c-73a322e2fa69: moving from state init -> state preparing
>Thread-15916832::DEBUG::2013-11-20 04:44:43,120::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916832::DEBUG::2013-11-20 04:44:43,121::task::495::TaskManager.Task::(_debug) Task 03a47515-379e-48aa-835c-73a322e2fa69: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916832::DEBUG::2013-11-20 04:44:43,122::task::495::TaskManager.Task::(_debug) Task 03a47515-379e-48aa-835c-73a322e2fa69: moving from state preparing -> state finished
>Thread-15916832::DEBUG::2013-11-20 04:44:43,122::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916832::DEBUG::2013-11-20 04:44:43,123::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916832::DEBUG::2013-11-20 04:44:43,124::task::495::TaskManager.Task::(_debug) Task 03a47515-379e-48aa-835c-73a322e2fa69: ref 0 aborting False
>Thread-15916832::INFO::2013-11-20 04:44:43,124::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916833::DEBUG::2013-11-20 04:44:43,139::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916833::INFO::2013-11-20 04:44:43,141::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916833::DEBUG::2013-11-20 04:44:43,141::task::495::TaskManager.Task::(_debug) Task 872ea8f8-8a01-4b61-b91e-1331dc294f10: moving from state init -> state preparing
>Thread-15916833::DEBUG::2013-11-20 04:44:43,146::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`34c8aab7-97c6-4115-b07c-7210af497a56`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916833::DEBUG::2013-11-20 04:44:43,147::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916833::DEBUG::2013-11-20 04:44:43,148::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916833::DEBUG::2013-11-20 04:44:43,148::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`34c8aab7-97c6-4115-b07c-7210af497a56`::Granted request
>Thread-15916833::DEBUG::2013-11-20 04:44:43,149::task::495::TaskManager.Task::(_debug) Task 872ea8f8-8a01-4b61-b91e-1331dc294f10: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916833::DEBUG::2013-11-20 04:44:43,150::task::495::TaskManager.Task::(_debug) Task 872ea8f8-8a01-4b61-b91e-1331dc294f10: ref 1 aborting False
>Thread-15916833::WARNING::2013-11-20 04:44:43,152::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916833::DEBUG::2013-11-20 04:44:43,157::task::495::TaskManager.Task::(_debug) Task 872ea8f8-8a01-4b61-b91e-1331dc294f10: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916833::DEBUG::2013-11-20 04:44:43,158::task::495::TaskManager.Task::(_debug) Task 872ea8f8-8a01-4b61-b91e-1331dc294f10: moving from state preparing -> state finished
>Thread-15916833::DEBUG::2013-11-20 04:44:43,159::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916833::DEBUG::2013-11-20 04:44:43,160::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916833::DEBUG::2013-11-20 04:44:43,161::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916833::DEBUG::2013-11-20 04:44:43,161::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916833::DEBUG::2013-11-20 04:44:43,162::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916833::DEBUG::2013-11-20 04:44:43,163::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916833::DEBUG::2013-11-20 04:44:43,164::task::495::TaskManager.Task::(_debug) Task 872ea8f8-8a01-4b61-b91e-1331dc294f10: ref 0 aborting False
>Thread-15916833::INFO::2013-11-20 04:44:43,169::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:44:44,094::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:44:44,572::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.048895 s, 20.9 MB/s\n'; <rc> = 0
>Thread-15914519::INFO::2013-11-20 04:44:44,603::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:44:44,610::task::495::TaskManager.Task::(_debug) Task 10b84146-af63-4350-92b3-de8e62e3d157: moving from state init -> state preparing
>Thread-15913546::INFO::2013-11-20 04:44:44,611::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15914519::DEBUG::2013-11-20 04:44:44,616::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2f33133f-6c28-4954-adbb-fd86be0310d0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:44:44,617::task::495::TaskManager.Task::(_debug) Task aa629379-ea92-471b-b1b4-e1c55a02a8e3: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:44:44,618::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:44:44,620::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`18c94655-87d5-42c1-aa3d-6beec3dbd035`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:44:44,621::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:44:44,622::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:44:44,623::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2f33133f-6c28-4954-adbb-fd86be0310d0`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:44:44,624::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:44:44,625::task::495::TaskManager.Task::(_debug) Task 10b84146-af63-4350-92b3-de8e62e3d157: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913546::DEBUG::2013-11-20 04:44:44,626::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`18c94655-87d5-42c1-aa3d-6beec3dbd035`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:44:44,626::task::495::TaskManager.Task::(_debug) Task 10b84146-af63-4350-92b3-de8e62e3d157: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:44:44,628::task::495::TaskManager.Task::(_debug) Task aa629379-ea92-471b-b1b4-e1c55a02a8e3: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15914519::DEBUG::2013-11-20 04:44:44,629::task::495::TaskManager.Task::(_debug) Task 10b84146-af63-4350-92b3-de8e62e3d157: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:44:44,630::task::495::TaskManager.Task::(_debug) Task aa629379-ea92-471b-b1b4-e1c55a02a8e3: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:44:44,631::task::495::TaskManager.Task::(_debug) Task 10b84146-af63-4350-92b3-de8e62e3d157: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:44:44,631::task::495::TaskManager.Task::(_debug) Task aa629379-ea92-471b-b1b4-e1c55a02a8e3: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:44:44,632::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:44:44,633::task::495::TaskManager.Task::(_debug) Task aa629379-ea92-471b-b1b4-e1c55a02a8e3: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:44:44,634::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:44:44,634::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:44:44,635::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913546::DEBUG::2013-11-20 04:44:44,636::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:44:44,637::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:44:44,638::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15914519::DEBUG::2013-11-20 04:44:44,638::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:44:44,639::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:44:44,640::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:44:44,641::task::495::TaskManager.Task::(_debug) Task 10b84146-af63-4350-92b3-de8e62e3d157: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:44:44,642::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15914519::INFO::2013-11-20 04:44:44,642::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:44:44,643::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:44:44,644::task::495::TaskManager.Task::(_debug) Task aa629379-ea92-471b-b1b4-e1c55a02a8e3: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:44:44,645::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:44:44,646::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:44:44,647::task::495::TaskManager.Task::(_debug) Task 196d73eb-02b7-40aa-a68a-6c44c4365a98: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:44:44,648::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`ddbda3fc-4783-4606-90ea-c205ca557149`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:44:44,649::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:44:44,649::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:44:44,650::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`ddbda3fc-4783-4606-90ea-c205ca557149`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:44:44,651::task::495::TaskManager.Task::(_debug) Task 196d73eb-02b7-40aa-a68a-6c44c4365a98: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:44:44,651::task::495::TaskManager.Task::(_debug) Task 196d73eb-02b7-40aa-a68a-6c44c4365a98: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:44:44,652::task::495::TaskManager.Task::(_debug) Task 196d73eb-02b7-40aa-a68a-6c44c4365a98: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:44:44,653::task::495::TaskManager.Task::(_debug) Task 196d73eb-02b7-40aa-a68a-6c44c4365a98: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:44:44,653::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:44:44,654::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:44:44,654::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:44:44,655::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:44:44,655::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:44:44,656::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:44:44,657::task::495::TaskManager.Task::(_debug) Task 196d73eb-02b7-40aa-a68a-6c44c4365a98: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:44:44,657::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:44:44,658::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:44:44,658::task::495::TaskManager.Task::(_debug) Task d60c695e-be3c-40e7-a1d3-b93f8f5758fe: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:44:44,660::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e6b53530-0091-46e2-aab2-80b043ebe61f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:44:44,660::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:44:44,661::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:44:44,661::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e6b53530-0091-46e2-aab2-80b043ebe61f`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:44:44,662::task::495::TaskManager.Task::(_debug) Task d60c695e-be3c-40e7-a1d3-b93f8f5758fe: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:44:44,663::task::495::TaskManager.Task::(_debug) Task d60c695e-be3c-40e7-a1d3-b93f8f5758fe: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:44:44,663::task::495::TaskManager.Task::(_debug) Task d60c695e-be3c-40e7-a1d3-b93f8f5758fe: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:44:44,664::task::495::TaskManager.Task::(_debug) Task d60c695e-be3c-40e7-a1d3-b93f8f5758fe: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:44:44,664::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:44:44,665::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:44:44,665::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:44:44,666::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:44:44,667::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:44:44,667::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:44:44,668::task::495::TaskManager.Task::(_debug) Task d60c695e-be3c-40e7-a1d3-b93f8f5758fe: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:44:44,668::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:44:44,669::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:44:44,669::task::495::TaskManager.Task::(_debug) Task c4cf9728-7daa-49b1-ac3a-aae3993f1880: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:44:44,671::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d5ade673-7bc2-4e69-8f12-646a694468d4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:44:44,671::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:44:44,672::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:44:44,672::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d5ade673-7bc2-4e69-8f12-646a694468d4`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:44:44,673::task::495::TaskManager.Task::(_debug) Task c4cf9728-7daa-49b1-ac3a-aae3993f1880: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:44:44,674::task::495::TaskManager.Task::(_debug) Task c4cf9728-7daa-49b1-ac3a-aae3993f1880: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:44:44,674::task::495::TaskManager.Task::(_debug) Task c4cf9728-7daa-49b1-ac3a-aae3993f1880: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:44:44,675::task::495::TaskManager.Task::(_debug) Task c4cf9728-7daa-49b1-ac3a-aae3993f1880: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:44:44,676::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:44:44,677::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:44:44,677::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:44:44,678::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:44:44,679::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:44:44,679::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:44:44,680::task::495::TaskManager.Task::(_debug) Task c4cf9728-7daa-49b1-ac3a-aae3993f1880: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:44:44,680::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:44:46,581::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:44:47,174::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0809068 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:44:49,188::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:44:49,721::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0800294 s, 12.8 MB/s\n'; <rc> = 0
>Thread-15914435::INFO::2013-11-20 04:44:51,438::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:44:51,439::task::495::TaskManager.Task::(_debug) Task 59c120c4-c10f-43b1-a738-5b6861041c83: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:44:51,442::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`85c8344f-50ff-4736-a273-b17175999c5d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:44:51,443::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:44:51,445::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:44:51,446::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`85c8344f-50ff-4736-a273-b17175999c5d`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:44:51,456::task::495::TaskManager.Task::(_debug) Task 59c120c4-c10f-43b1-a738-5b6861041c83: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:44:51,457::task::495::TaskManager.Task::(_debug) Task 59c120c4-c10f-43b1-a738-5b6861041c83: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:44:51,459::task::495::TaskManager.Task::(_debug) Task 59c120c4-c10f-43b1-a738-5b6861041c83: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:44:51,460::task::495::TaskManager.Task::(_debug) Task 59c120c4-c10f-43b1-a738-5b6861041c83: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:44:51,460::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:44:51,461::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:44:51,463::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:44:51,464::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:44:51,465::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:44:51,467::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:44:51,468::task::495::TaskManager.Task::(_debug) Task 59c120c4-c10f-43b1-a738-5b6861041c83: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:44:51,469::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:44:51,741::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916838::INFO::2013-11-20 04:44:51,964::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916838::DEBUG::2013-11-20 04:44:51,966::task::495::TaskManager.Task::(_debug) Task 9abd5b1d-c23e-4339-984d-309c82e7ef0f: moving from state init -> state preparing
>Thread-15916838::DEBUG::2013-11-20 04:44:51,967::task::495::TaskManager.Task::(_debug) Task 9abd5b1d-c23e-4339-984d-309c82e7ef0f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000427007675171', 'lastCheck': 1384922685.323349, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00132989883423', 'lastCheck': 1384922688.3231361, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00169396400452', 'lastCheck': 1384922688.322191, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000542163848877', 'lastCheck': 1384922690.3923559, 'valid': True, 'code': 0}}
>Thread-15916838::DEBUG::2013-11-20 04:44:51,973::task::495::TaskManager.Task::(_debug) Task 9abd5b1d-c23e-4339-984d-309c82e7ef0f: moving from state preparing -> state finished
>Thread-15916838::DEBUG::2013-11-20 04:44:51,974::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916838::DEBUG::2013-11-20 04:44:51,975::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916838::DEBUG::2013-11-20 04:44:51,975::task::495::TaskManager.Task::(_debug) Task 9abd5b1d-c23e-4339-984d-309c82e7ef0f: ref 0 aborting False
>Thread-15916838::INFO::2013-11-20 04:44:51,976::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000427007675171', 'lastCheck': 1384922685.323349, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00132989883423', 'lastCheck': 1384922688.3231361, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00169396400452', 'lastCheck': 1384922688.322191, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000542163848877', 'lastCheck': 1384922690.3923559, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:44:52,262::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0761932 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15916840::DEBUG::2013-11-20 04:44:53,474::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916840::INFO::2013-11-20 04:44:53,476::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916840::DEBUG::2013-11-20 04:44:53,477::task::495::TaskManager.Task::(_debug) Task 701d6650-805f-4d70-9c3c-91d4687abd09: moving from state init -> state preparing
>Thread-15916840::DEBUG::2013-11-20 04:44:53,478::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916840::DEBUG::2013-11-20 04:44:53,479::task::495::TaskManager.Task::(_debug) Task 701d6650-805f-4d70-9c3c-91d4687abd09: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916840::DEBUG::2013-11-20 04:44:53,480::task::495::TaskManager.Task::(_debug) Task 701d6650-805f-4d70-9c3c-91d4687abd09: moving from state preparing -> state finished
>Thread-15916840::DEBUG::2013-11-20 04:44:53,480::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916840::DEBUG::2013-11-20 04:44:53,481::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916840::DEBUG::2013-11-20 04:44:53,482::task::495::TaskManager.Task::(_debug) Task 701d6650-805f-4d70-9c3c-91d4687abd09: ref 0 aborting False
>Thread-15916840::INFO::2013-11-20 04:44:53,482::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916841::DEBUG::2013-11-20 04:44:53,503::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916841::INFO::2013-11-20 04:44:53,505::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916841::DEBUG::2013-11-20 04:44:53,506::task::495::TaskManager.Task::(_debug) Task 19d6bd07-122c-4e1e-812d-48039c011d00: moving from state init -> state preparing
>Thread-15916841::DEBUG::2013-11-20 04:44:53,508::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`dbf09da8-1074-4ce4-aa95-87b7710040c2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916841::DEBUG::2013-11-20 04:44:53,515::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916841::DEBUG::2013-11-20 04:44:53,516::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916841::DEBUG::2013-11-20 04:44:53,517::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`dbf09da8-1074-4ce4-aa95-87b7710040c2`::Granted request
>Thread-15916841::DEBUG::2013-11-20 04:44:53,518::task::495::TaskManager.Task::(_debug) Task 19d6bd07-122c-4e1e-812d-48039c011d00: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916841::DEBUG::2013-11-20 04:44:53,519::task::495::TaskManager.Task::(_debug) Task 19d6bd07-122c-4e1e-812d-48039c011d00: ref 1 aborting False
>Thread-15916841::WARNING::2013-11-20 04:44:53,521::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916841::DEBUG::2013-11-20 04:44:53,532::task::495::TaskManager.Task::(_debug) Task 19d6bd07-122c-4e1e-812d-48039c011d00: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916841::DEBUG::2013-11-20 04:44:53,533::task::495::TaskManager.Task::(_debug) Task 19d6bd07-122c-4e1e-812d-48039c011d00: moving from state preparing -> state finished
>Thread-15916841::DEBUG::2013-11-20 04:44:53,534::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916841::DEBUG::2013-11-20 04:44:53,535::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916841::DEBUG::2013-11-20 04:44:53,536::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916841::DEBUG::2013-11-20 04:44:53,537::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916841::DEBUG::2013-11-20 04:44:53,537::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916841::DEBUG::2013-11-20 04:44:53,538::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916841::DEBUG::2013-11-20 04:44:53,539::task::495::TaskManager.Task::(_debug) Task 19d6bd07-122c-4e1e-812d-48039c011d00: ref 0 aborting False
>Thread-15916841::INFO::2013-11-20 04:44:53,539::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:44:54,279::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:44:54,814::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0764214 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:44:56,836::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:44:57,360::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0719871 s, 14.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:44:59,375::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:44:59,930::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.086091 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:45:01,946::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:45:02,196::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:45:02,200::task::495::TaskManager.Task::(_debug) Task 4ab46740-eb11-4727-b95f-f62845b8e3e9: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:45:02,203::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`77035416-4ab4-431c-9961-fa85eeb9e2c8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:45:02,204::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:45:02,205::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:45:02,206::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`77035416-4ab4-431c-9961-fa85eeb9e2c8`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:45:02,208::task::495::TaskManager.Task::(_debug) Task 4ab46740-eb11-4727-b95f-f62845b8e3e9: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:45:02,209::task::495::TaskManager.Task::(_debug) Task 4ab46740-eb11-4727-b95f-f62845b8e3e9: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:45:02,210::task::495::TaskManager.Task::(_debug) Task 4ab46740-eb11-4727-b95f-f62845b8e3e9: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:45:02,211::task::495::TaskManager.Task::(_debug) Task 4ab46740-eb11-4727-b95f-f62845b8e3e9: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:45:02,211::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:45:02,212::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:45:02,213::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:45:02,214::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:45:02,215::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:45:02,216::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:45:02,217::task::495::TaskManager.Task::(_debug) Task 4ab46740-eb11-4727-b95f-f62845b8e3e9: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:45:02,218::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:45:02,519::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0793681 s, 12.9 MB/s\n'; <rc> = 0
>Thread-15916846::INFO::2013-11-20 04:45:02,689::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916846::DEBUG::2013-11-20 04:45:02,690::task::495::TaskManager.Task::(_debug) Task a3ffd3dc-8096-4539-9d43-bca92df091cf: moving from state init -> state preparing
>Thread-15916846::DEBUG::2013-11-20 04:45:02,690::task::495::TaskManager.Task::(_debug) Task a3ffd3dc-8096-4539-9d43-bca92df091cf: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000349998474121', 'lastCheck': 1384922695.497515, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00134682655334', 'lastCheck': 1384922698.497818, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00159096717834', 'lastCheck': 1384922698.4947369, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000530004501343', 'lastCheck': 1384922700.577009, 'valid': True, 'code': 0}}
>Thread-15916846::DEBUG::2013-11-20 04:45:02,691::task::495::TaskManager.Task::(_debug) Task a3ffd3dc-8096-4539-9d43-bca92df091cf: moving from state preparing -> state finished
>Thread-15916846::DEBUG::2013-11-20 04:45:02,691::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916846::DEBUG::2013-11-20 04:45:02,692::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916846::DEBUG::2013-11-20 04:45:02,692::task::495::TaskManager.Task::(_debug) Task a3ffd3dc-8096-4539-9d43-bca92df091cf: ref 0 aborting False
>Thread-15916846::INFO::2013-11-20 04:45:02,693::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000349998474121', 'lastCheck': 1384922695.497515, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00134682655334', 'lastCheck': 1384922698.497818, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00159096717834', 'lastCheck': 1384922698.4947369, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000530004501343', 'lastCheck': 1384922700.577009, 'valid': True, 'code': 0}}
>Thread-15916848::DEBUG::2013-11-20 04:45:03,938::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916848::INFO::2013-11-20 04:45:03,939::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916848::DEBUG::2013-11-20 04:45:03,940::task::495::TaskManager.Task::(_debug) Task 50ed9e0b-c8fc-483f-bd3c-3d16206879d8: moving from state init -> state preparing
>Thread-15916848::DEBUG::2013-11-20 04:45:03,941::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916848::DEBUG::2013-11-20 04:45:03,941::task::495::TaskManager.Task::(_debug) Task 50ed9e0b-c8fc-483f-bd3c-3d16206879d8: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916848::DEBUG::2013-11-20 04:45:03,942::task::495::TaskManager.Task::(_debug) Task 50ed9e0b-c8fc-483f-bd3c-3d16206879d8: moving from state preparing -> state finished
>Thread-15916848::DEBUG::2013-11-20 04:45:03,942::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916848::DEBUG::2013-11-20 04:45:03,943::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916848::DEBUG::2013-11-20 04:45:03,944::task::495::TaskManager.Task::(_debug) Task 50ed9e0b-c8fc-483f-bd3c-3d16206879d8: ref 0 aborting False
>Thread-15916848::INFO::2013-11-20 04:45:03,944::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916849::DEBUG::2013-11-20 04:45:03,954::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916849::INFO::2013-11-20 04:45:03,955::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916849::DEBUG::2013-11-20 04:45:03,956::task::495::TaskManager.Task::(_debug) Task 25f12f06-4657-4c41-b963-558fb691e450: moving from state init -> state preparing
>Thread-15916849::DEBUG::2013-11-20 04:45:03,957::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f0449f6c-6e2b-41a2-8976-8f934f87f858`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916849::DEBUG::2013-11-20 04:45:03,958::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916849::DEBUG::2013-11-20 04:45:03,959::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916849::DEBUG::2013-11-20 04:45:03,960::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`f0449f6c-6e2b-41a2-8976-8f934f87f858`::Granted request
>Thread-15916849::DEBUG::2013-11-20 04:45:03,961::task::495::TaskManager.Task::(_debug) Task 25f12f06-4657-4c41-b963-558fb691e450: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916849::DEBUG::2013-11-20 04:45:03,961::task::495::TaskManager.Task::(_debug) Task 25f12f06-4657-4c41-b963-558fb691e450: ref 1 aborting False
>Thread-15916849::WARNING::2013-11-20 04:45:03,963::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916849::DEBUG::2013-11-20 04:45:03,970::task::495::TaskManager.Task::(_debug) Task 25f12f06-4657-4c41-b963-558fb691e450: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916849::DEBUG::2013-11-20 04:45:03,972::task::495::TaskManager.Task::(_debug) Task 25f12f06-4657-4c41-b963-558fb691e450: moving from state preparing -> state finished
>Thread-15916849::DEBUG::2013-11-20 04:45:03,973::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916849::DEBUG::2013-11-20 04:45:03,974::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916849::DEBUG::2013-11-20 04:45:03,975::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916849::DEBUG::2013-11-20 04:45:03,976::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916849::DEBUG::2013-11-20 04:45:03,977::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916849::DEBUG::2013-11-20 04:45:03,978::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916849::DEBUG::2013-11-20 04:45:03,978::task::495::TaskManager.Task::(_debug) Task 25f12f06-4657-4c41-b963-558fb691e450: ref 0 aborting False
>Thread-15916849::INFO::2013-11-20 04:45:03,979::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:45:04,534::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:45:05,081::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0830522 s, 12.3 MB/s\n'; <rc> = 0
>Thread-15915918::INFO::2013-11-20 04:45:05,581::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:45:05,583::task::495::TaskManager.Task::(_debug) Task f7e51d39-e6ba-460f-a298-6812104ddb46: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:45:05,585::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`231b9c9f-c131-462c-a840-0c9d7986fca7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:45:05,586::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:45:05,587::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:45:05,588::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`231b9c9f-c131-462c-a840-0c9d7986fca7`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:45:05,589::task::495::TaskManager.Task::(_debug) Task f7e51d39-e6ba-460f-a298-6812104ddb46: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:45:05,590::task::495::TaskManager.Task::(_debug) Task f7e51d39-e6ba-460f-a298-6812104ddb46: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:45:05,591::task::495::TaskManager.Task::(_debug) Task f7e51d39-e6ba-460f-a298-6812104ddb46: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:45:05,591::task::495::TaskManager.Task::(_debug) Task f7e51d39-e6ba-460f-a298-6812104ddb46: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:45:05,592::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:45:05,592::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:45:05,593::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:45:05,594::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:45:05,595::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:45:05,595::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:45:05,596::task::495::TaskManager.Task::(_debug) Task f7e51d39-e6ba-460f-a298-6812104ddb46: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:45:05,596::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:45:07,097::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:45:07,626::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0734473 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:45:09,646::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915863::INFO::2013-11-20 04:45:10,193::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:45:10,195::task::495::TaskManager.Task::(_debug) Task 5c558b43-913b-44d5-9380-38ad345b2180: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:45:10,198::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`65c3ff0f-c29c-4eea-8f3a-f9ed82e8850a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:45:10,199::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:45:10,202::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:45:10,203::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`65c3ff0f-c29c-4eea-8f3a-f9ed82e8850a`::Granted request
>Dummy-15912651::DEBUG::2013-11-20 04:45:10,206::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0813872 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15915863::DEBUG::2013-11-20 04:45:10,208::task::495::TaskManager.Task::(_debug) Task 5c558b43-913b-44d5-9380-38ad345b2180: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:45:10,213::task::495::TaskManager.Task::(_debug) Task 5c558b43-913b-44d5-9380-38ad345b2180: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:45:10,225::task::495::TaskManager.Task::(_debug) Task 5c558b43-913b-44d5-9380-38ad345b2180: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:45:10,226::task::495::TaskManager.Task::(_debug) Task 5c558b43-913b-44d5-9380-38ad345b2180: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:45:10,227::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:45:10,228::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:45:10,229::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:45:10,229::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:45:10,230::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:45:10,231::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:45:10,232::task::495::TaskManager.Task::(_debug) Task 5c558b43-913b-44d5-9380-38ad345b2180: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:45:10,232::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:45:12,228::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:45:12,800::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0840274 s, 12.2 MB/s\n'; <rc> = 0
>Thread-15916854::INFO::2013-11-20 04:45:13,413::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916854::DEBUG::2013-11-20 04:45:13,414::task::495::TaskManager.Task::(_debug) Task acfb8642-39fa-4271-a3c5-4d0d56da4607: moving from state init -> state preparing
>Thread-15916854::DEBUG::2013-11-20 04:45:13,415::task::495::TaskManager.Task::(_debug) Task acfb8642-39fa-4271-a3c5-4d0d56da4607: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00133585929871', 'lastCheck': 1384922705.737771, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00137305259705', 'lastCheck': 1384922708.7453561, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00192499160767', 'lastCheck': 1384922708.744441, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000460863113403', 'lastCheck': 1384922710.8465719, 'valid': True, 'code': 0}}
>Thread-15916854::DEBUG::2013-11-20 04:45:13,425::task::495::TaskManager.Task::(_debug) Task acfb8642-39fa-4271-a3c5-4d0d56da4607: moving from state preparing -> state finished
>Thread-15916854::DEBUG::2013-11-20 04:45:13,425::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916854::DEBUG::2013-11-20 04:45:13,426::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916854::DEBUG::2013-11-20 04:45:13,427::task::495::TaskManager.Task::(_debug) Task acfb8642-39fa-4271-a3c5-4d0d56da4607: ref 0 aborting False
>Thread-15916854::INFO::2013-11-20 04:45:13,428::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00133585929871', 'lastCheck': 1384922705.737771, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00137305259705', 'lastCheck': 1384922708.7453561, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00192499160767', 'lastCheck': 1384922708.744441, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000460863113403', 'lastCheck': 1384922710.8465719, 'valid': True, 'code': 0}}
>Thread-15916856::DEBUG::2013-11-20 04:45:14,332::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916856::INFO::2013-11-20 04:45:14,338::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916856::DEBUG::2013-11-20 04:45:14,339::task::495::TaskManager.Task::(_debug) Task 2b9461c0-b1ca-4fa5-8f0a-47fa54a6edac: moving from state init -> state preparing
>Thread-15916856::DEBUG::2013-11-20 04:45:14,340::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916856::DEBUG::2013-11-20 04:45:14,341::task::495::TaskManager.Task::(_debug) Task 2b9461c0-b1ca-4fa5-8f0a-47fa54a6edac: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916856::DEBUG::2013-11-20 04:45:14,342::task::495::TaskManager.Task::(_debug) Task 2b9461c0-b1ca-4fa5-8f0a-47fa54a6edac: moving from state preparing -> state finished
>Thread-15916856::DEBUG::2013-11-20 04:45:14,342::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916856::DEBUG::2013-11-20 04:45:14,343::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916856::DEBUG::2013-11-20 04:45:14,344::task::495::TaskManager.Task::(_debug) Task 2b9461c0-b1ca-4fa5-8f0a-47fa54a6edac: ref 0 aborting False
>Thread-15916856::INFO::2013-11-20 04:45:14,345::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916857::DEBUG::2013-11-20 04:45:14,361::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916857::INFO::2013-11-20 04:45:14,362::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916857::DEBUG::2013-11-20 04:45:14,362::task::495::TaskManager.Task::(_debug) Task bb8a6d8b-eb5a-4608-87c0-b1dd1e18dca1: moving from state init -> state preparing
>Thread-15916857::DEBUG::2013-11-20 04:45:14,364::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`95e79de7-3e6f-4a45-b682-17bbc7fd525f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916857::DEBUG::2013-11-20 04:45:14,364::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916857::DEBUG::2013-11-20 04:45:14,365::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916857::DEBUG::2013-11-20 04:45:14,366::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`95e79de7-3e6f-4a45-b682-17bbc7fd525f`::Granted request
>Thread-15916857::DEBUG::2013-11-20 04:45:14,367::task::495::TaskManager.Task::(_debug) Task bb8a6d8b-eb5a-4608-87c0-b1dd1e18dca1: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916857::DEBUG::2013-11-20 04:45:14,368::task::495::TaskManager.Task::(_debug) Task bb8a6d8b-eb5a-4608-87c0-b1dd1e18dca1: ref 1 aborting False
>Thread-15916857::WARNING::2013-11-20 04:45:14,370::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916857::DEBUG::2013-11-20 04:45:14,376::task::495::TaskManager.Task::(_debug) Task bb8a6d8b-eb5a-4608-87c0-b1dd1e18dca1: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916857::DEBUG::2013-11-20 04:45:14,377::task::495::TaskManager.Task::(_debug) Task bb8a6d8b-eb5a-4608-87c0-b1dd1e18dca1: moving from state preparing -> state finished
>Thread-15916857::DEBUG::2013-11-20 04:45:14,378::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916857::DEBUG::2013-11-20 04:45:14,378::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916857::DEBUG::2013-11-20 04:45:14,379::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916857::DEBUG::2013-11-20 04:45:14,380::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916857::DEBUG::2013-11-20 04:45:14,380::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916857::DEBUG::2013-11-20 04:45:14,381::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916857::DEBUG::2013-11-20 04:45:14,381::task::495::TaskManager.Task::(_debug) Task bb8a6d8b-eb5a-4608-87c0-b1dd1e18dca1: ref 0 aborting False
>Thread-15916857::INFO::2013-11-20 04:45:14,382::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:45:14,813::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 04:45:15,026::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:45:15,034::task::495::TaskManager.Task::(_debug) Task d89a49b0-4403-4a95-852e-71f7ddf7babb: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:45:15,036::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c6a149e7-e2d7-44ef-9b4f-93ce96cb3e0f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:45:15,037::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:45:15,038::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:45:15,038::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c6a149e7-e2d7-44ef-9b4f-93ce96cb3e0f`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:45:15,040::task::495::TaskManager.Task::(_debug) Task d89a49b0-4403-4a95-852e-71f7ddf7babb: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:45:15,040::task::495::TaskManager.Task::(_debug) Task d89a49b0-4403-4a95-852e-71f7ddf7babb: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:45:15,041::task::495::TaskManager.Task::(_debug) Task d89a49b0-4403-4a95-852e-71f7ddf7babb: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:45:15,042::task::495::TaskManager.Task::(_debug) Task d89a49b0-4403-4a95-852e-71f7ddf7babb: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:45:15,043::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:45:15,043::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:45:15,044::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:45:15,045::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:45:15,047::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:45:15,048::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:45:15,049::task::495::TaskManager.Task::(_debug) Task d89a49b0-4403-4a95-852e-71f7ddf7babb: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:45:15,050::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:45:15,349::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0823801 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:45:17,363::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:45:17,935::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0891278 s, 11.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:45:19,950::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:45:20,491::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0749119 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 04:45:22,214::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:45:22,215::task::495::TaskManager.Task::(_debug) Task 82efb9c8-f245-40c0-ae97-e9c6366dd18b: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:45:22,216::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`99d28ada-7cdc-49af-ab2b-9338df1b706a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:45:22,217::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:45:22,218::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:45:22,218::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`99d28ada-7cdc-49af-ab2b-9338df1b706a`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:45:22,219::task::495::TaskManager.Task::(_debug) Task 82efb9c8-f245-40c0-ae97-e9c6366dd18b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:45:22,220::task::495::TaskManager.Task::(_debug) Task 82efb9c8-f245-40c0-ae97-e9c6366dd18b: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:45:22,221::task::495::TaskManager.Task::(_debug) Task 82efb9c8-f245-40c0-ae97-e9c6366dd18b: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:45:22,222::task::495::TaskManager.Task::(_debug) Task 82efb9c8-f245-40c0-ae97-e9c6366dd18b: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:45:22,222::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:45:22,223::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:45:22,224::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:45:22,225::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:45:22,225::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:45:22,226::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:45:22,227::task::495::TaskManager.Task::(_debug) Task 82efb9c8-f245-40c0-ae97-e9c6366dd18b: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:45:22,227::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:45:22,228::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:45:22,229::task::495::TaskManager.Task::(_debug) Task 2864cf76-32bb-41d8-813a-a2f10b83c74f: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:45:22,230::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a2808664-2c4b-4dee-86bc-5617d07e6247`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:45:22,230::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:45:22,231::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:45:22,232::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a2808664-2c4b-4dee-86bc-5617d07e6247`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:45:22,232::task::495::TaskManager.Task::(_debug) Task 2864cf76-32bb-41d8-813a-a2f10b83c74f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:45:22,233::task::495::TaskManager.Task::(_debug) Task 2864cf76-32bb-41d8-813a-a2f10b83c74f: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:45:22,234::task::495::TaskManager.Task::(_debug) Task 2864cf76-32bb-41d8-813a-a2f10b83c74f: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:45:22,234::task::495::TaskManager.Task::(_debug) Task 2864cf76-32bb-41d8-813a-a2f10b83c74f: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:45:22,235::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:45:22,236::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:45:22,236::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:45:22,237::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:45:22,237::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:45:22,238::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:45:22,239::task::495::TaskManager.Task::(_debug) Task 2864cf76-32bb-41d8-813a-a2f10b83c74f: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:45:22,239::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:45:22,513::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:45:23,078::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0932816 s, 11.0 MB/s\n'; <rc> = 0
>Thread-15916862::INFO::2013-11-20 04:45:23,961::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916862::DEBUG::2013-11-20 04:45:23,962::task::495::TaskManager.Task::(_debug) Task c131d730-15f4-42d5-b31c-20f90be62583: moving from state init -> state preparing
>Thread-15916862::DEBUG::2013-11-20 04:45:23,963::task::495::TaskManager.Task::(_debug) Task c131d730-15f4-42d5-b31c-20f90be62583: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000494956970215', 'lastCheck': 1384922716.0244701, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00165390968323', 'lastCheck': 1384922719.0308981, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00196290016174', 'lastCheck': 1384922719.030751, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000613927841187', 'lastCheck': 1384922721.167268, 'valid': True, 'code': 0}}
>Thread-15916862::DEBUG::2013-11-20 04:45:23,973::task::495::TaskManager.Task::(_debug) Task c131d730-15f4-42d5-b31c-20f90be62583: moving from state preparing -> state finished
>Thread-15916862::DEBUG::2013-11-20 04:45:23,974::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916862::DEBUG::2013-11-20 04:45:23,975::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916862::DEBUG::2013-11-20 04:45:23,976::task::495::TaskManager.Task::(_debug) Task c131d730-15f4-42d5-b31c-20f90be62583: ref 0 aborting False
>Thread-15916862::INFO::2013-11-20 04:45:23,977::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000494956970215', 'lastCheck': 1384922716.0244701, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00165390968323', 'lastCheck': 1384922719.0308981, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00196290016174', 'lastCheck': 1384922719.030751, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000613927841187', 'lastCheck': 1384922721.167268, 'valid': True, 'code': 0}}
>Thread-15916864::DEBUG::2013-11-20 04:45:24,677::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916864::INFO::2013-11-20 04:45:24,678::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916864::DEBUG::2013-11-20 04:45:24,679::task::495::TaskManager.Task::(_debug) Task dff17822-91ce-4e49-b0f8-91c148ae0035: moving from state init -> state preparing
>Thread-15916864::DEBUG::2013-11-20 04:45:24,680::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916864::DEBUG::2013-11-20 04:45:24,680::task::495::TaskManager.Task::(_debug) Task dff17822-91ce-4e49-b0f8-91c148ae0035: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916864::DEBUG::2013-11-20 04:45:24,681::task::495::TaskManager.Task::(_debug) Task dff17822-91ce-4e49-b0f8-91c148ae0035: moving from state preparing -> state finished
>Thread-15916864::DEBUG::2013-11-20 04:45:24,682::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916864::DEBUG::2013-11-20 04:45:24,682::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916864::DEBUG::2013-11-20 04:45:24,683::task::495::TaskManager.Task::(_debug) Task dff17822-91ce-4e49-b0f8-91c148ae0035: ref 0 aborting False
>Thread-15916864::INFO::2013-11-20 04:45:24,684::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916865::DEBUG::2013-11-20 04:45:24,702::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916865::INFO::2013-11-20 04:45:24,703::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916865::DEBUG::2013-11-20 04:45:24,704::task::495::TaskManager.Task::(_debug) Task f8ee404d-c80d-41f1-a634-358966077b0e: moving from state init -> state preparing
>Thread-15916865::DEBUG::2013-11-20 04:45:24,709::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0af0b7cc-d1b6-4bac-b5d4-9242670fedc8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916865::DEBUG::2013-11-20 04:45:24,710::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916865::DEBUG::2013-11-20 04:45:24,711::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916865::DEBUG::2013-11-20 04:45:24,711::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0af0b7cc-d1b6-4bac-b5d4-9242670fedc8`::Granted request
>Thread-15916865::DEBUG::2013-11-20 04:45:24,712::task::495::TaskManager.Task::(_debug) Task f8ee404d-c80d-41f1-a634-358966077b0e: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916865::DEBUG::2013-11-20 04:45:24,713::task::495::TaskManager.Task::(_debug) Task f8ee404d-c80d-41f1-a634-358966077b0e: ref 1 aborting False
>Thread-15916865::WARNING::2013-11-20 04:45:24,715::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916865::DEBUG::2013-11-20 04:45:24,722::task::495::TaskManager.Task::(_debug) Task f8ee404d-c80d-41f1-a634-358966077b0e: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916865::DEBUG::2013-11-20 04:45:24,723::task::495::TaskManager.Task::(_debug) Task f8ee404d-c80d-41f1-a634-358966077b0e: moving from state preparing -> state finished
>Thread-15916865::DEBUG::2013-11-20 04:45:24,724::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916865::DEBUG::2013-11-20 04:45:24,725::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916865::DEBUG::2013-11-20 04:45:24,726::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916865::DEBUG::2013-11-20 04:45:24,727::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916865::DEBUG::2013-11-20 04:45:24,728::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916865::DEBUG::2013-11-20 04:45:24,729::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916865::DEBUG::2013-11-20 04:45:24,730::task::495::TaskManager.Task::(_debug) Task f8ee404d-c80d-41f1-a634-358966077b0e: ref 0 aborting False
>Thread-15916865::INFO::2013-11-20 04:45:24,731::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:45:25,092::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 04:45:25,604::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:45:25,606::task::495::TaskManager.Task::(_debug) Task a594ae26-68dd-4c40-9088-522aca33524f: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:45:25,609::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e9c33393-626a-4af8-88a4-dda7d120a294`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:45:25,611::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:45:25,615::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:45:25,616::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e9c33393-626a-4af8-88a4-dda7d120a294`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:45:25,618::task::495::TaskManager.Task::(_debug) Task a594ae26-68dd-4c40-9088-522aca33524f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:45:25,622::task::495::TaskManager.Task::(_debug) Task a594ae26-68dd-4c40-9088-522aca33524f: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:45:25,623::task::495::TaskManager.Task::(_debug) Task a594ae26-68dd-4c40-9088-522aca33524f: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:45:25,624::task::495::TaskManager.Task::(_debug) Task a594ae26-68dd-4c40-9088-522aca33524f: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:45:25,626::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:45:25,627::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:45:25,630::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:45:25,631::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:45:25,633::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:45:25,634::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:45:25,635::task::495::TaskManager.Task::(_debug) Task a594ae26-68dd-4c40-9088-522aca33524f: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:45:25,636::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:45:25,648::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0782031 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:45:27,667::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:45:28,217::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0771308 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:45:30,231::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:45:30,784::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.076001 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:45:32,801::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:45:33,341::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0836156 s, 12.2 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 04:45:34,005::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:45:34,007::task::495::TaskManager.Task::(_debug) Task bb6e64e9-b9ed-401a-bbfd-f5543ab10270: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:45:34,009::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b6047c8d-4623-47d1-a3ff-a8d31d1ee082`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:45:34,009::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:45:34,010::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:45:34,011::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b6047c8d-4623-47d1-a3ff-a8d31d1ee082`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:45:34,012::task::495::TaskManager.Task::(_debug) Task bb6e64e9-b9ed-401a-bbfd-f5543ab10270: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:45:34,012::task::495::TaskManager.Task::(_debug) Task bb6e64e9-b9ed-401a-bbfd-f5543ab10270: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:45:34,013::task::495::TaskManager.Task::(_debug) Task bb6e64e9-b9ed-401a-bbfd-f5543ab10270: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:45:34,014::task::495::TaskManager.Task::(_debug) Task bb6e64e9-b9ed-401a-bbfd-f5543ab10270: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:45:34,015::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:45:34,015::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:45:34,016::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:45:34,017::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:45:34,017::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:45:34,018::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:45:34,018::task::495::TaskManager.Task::(_debug) Task bb6e64e9-b9ed-401a-bbfd-f5543ab10270: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:45:34,019::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15916870::DEBUG::2013-11-20 04:45:35,004::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916870::INFO::2013-11-20 04:45:35,006::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916870::DEBUG::2013-11-20 04:45:35,006::task::495::TaskManager.Task::(_debug) Task c8824d72-c727-46b9-a86c-712f5e652605: moving from state init -> state preparing
>Thread-15916870::DEBUG::2013-11-20 04:45:35,007::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916870::DEBUG::2013-11-20 04:45:35,008::task::495::TaskManager.Task::(_debug) Task c8824d72-c727-46b9-a86c-712f5e652605: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916870::DEBUG::2013-11-20 04:45:35,009::task::495::TaskManager.Task::(_debug) Task c8824d72-c727-46b9-a86c-712f5e652605: moving from state preparing -> state finished
>Thread-15916870::DEBUG::2013-11-20 04:45:35,009::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916870::DEBUG::2013-11-20 04:45:35,010::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916870::DEBUG::2013-11-20 04:45:35,011::task::495::TaskManager.Task::(_debug) Task c8824d72-c727-46b9-a86c-712f5e652605: ref 0 aborting False
>Thread-15916870::INFO::2013-11-20 04:45:35,011::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916871::DEBUG::2013-11-20 04:45:35,025::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916871::INFO::2013-11-20 04:45:35,026::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916871::DEBUG::2013-11-20 04:45:35,027::task::495::TaskManager.Task::(_debug) Task 0e5963e2-a0b2-449d-9dbf-f6e783ab8df1: moving from state init -> state preparing
>Thread-15916871::DEBUG::2013-11-20 04:45:35,028::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8501fe60-a882-4d08-9eef-e9b8b02ff63e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916871::DEBUG::2013-11-20 04:45:35,034::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916871::DEBUG::2013-11-20 04:45:35,035::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916871::DEBUG::2013-11-20 04:45:35,036::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8501fe60-a882-4d08-9eef-e9b8b02ff63e`::Granted request
>Thread-15916871::DEBUG::2013-11-20 04:45:35,037::task::495::TaskManager.Task::(_debug) Task 0e5963e2-a0b2-449d-9dbf-f6e783ab8df1: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916871::DEBUG::2013-11-20 04:45:35,037::task::495::TaskManager.Task::(_debug) Task 0e5963e2-a0b2-449d-9dbf-f6e783ab8df1: ref 1 aborting False
>Thread-15916871::WARNING::2013-11-20 04:45:35,039::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916871::DEBUG::2013-11-20 04:45:35,048::task::495::TaskManager.Task::(_debug) Task 0e5963e2-a0b2-449d-9dbf-f6e783ab8df1: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916871::DEBUG::2013-11-20 04:45:35,049::task::495::TaskManager.Task::(_debug) Task 0e5963e2-a0b2-449d-9dbf-f6e783ab8df1: moving from state preparing -> state finished
>Thread-15916871::DEBUG::2013-11-20 04:45:35,050::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916871::DEBUG::2013-11-20 04:45:35,051::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916872::INFO::2013-11-20 04:45:35,054::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916872::DEBUG::2013-11-20 04:45:35,055::task::495::TaskManager.Task::(_debug) Task f1d613b7-641e-43c8-b421-acbf26ef3751: moving from state init -> state preparing
>Thread-15916871::DEBUG::2013-11-20 04:45:35,055::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916872::DEBUG::2013-11-20 04:45:35,056::task::495::TaskManager.Task::(_debug) Task f1d613b7-641e-43c8-b421-acbf26ef3751: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000382900238037', 'lastCheck': 1384922726.311672, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00150609016418', 'lastCheck': 1384922729.3121769, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00196290016174', 'lastCheck': 1384922729.31355, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000370025634766', 'lastCheck': 1384922731.4494779, 'valid': True, 'code': 0}}
>Thread-15916871::DEBUG::2013-11-20 04:45:35,057::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916872::DEBUG::2013-11-20 04:45:35,058::task::495::TaskManager.Task::(_debug) Task f1d613b7-641e-43c8-b421-acbf26ef3751: moving from state preparing -> state finished
>Thread-15916871::DEBUG::2013-11-20 04:45:35,059::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916872::DEBUG::2013-11-20 04:45:35,059::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916871::DEBUG::2013-11-20 04:45:35,060::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916872::DEBUG::2013-11-20 04:45:35,061::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916871::DEBUG::2013-11-20 04:45:35,061::task::495::TaskManager.Task::(_debug) Task 0e5963e2-a0b2-449d-9dbf-f6e783ab8df1: ref 0 aborting False
>Thread-15916872::DEBUG::2013-11-20 04:45:35,062::task::495::TaskManager.Task::(_debug) Task f1d613b7-641e-43c8-b421-acbf26ef3751: ref 0 aborting False
>Thread-15916871::INFO::2013-11-20 04:45:35,063::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916872::INFO::2013-11-20 04:45:35,064::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000382900238037', 'lastCheck': 1384922726.311672, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00150609016418', 'lastCheck': 1384922729.3121769, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00196290016174', 'lastCheck': 1384922729.31355, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000370025634766', 'lastCheck': 1384922731.4494779, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:45:35,356::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 04:45:35,578::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:45:35,586::task::495::TaskManager.Task::(_debug) Task db88c2c7-425c-47a3-8aef-9d121a859c78: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:45:35,590::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b698cab2-7118-47f5-a6af-bd5f1da8fa97`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:45:35,592::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:45:35,593::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:45:35,594::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b698cab2-7118-47f5-a6af-bd5f1da8fa97`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:45:35,595::task::495::TaskManager.Task::(_debug) Task db88c2c7-425c-47a3-8aef-9d121a859c78: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:45:35,596::task::495::TaskManager.Task::(_debug) Task db88c2c7-425c-47a3-8aef-9d121a859c78: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:45:35,597::task::495::TaskManager.Task::(_debug) Task db88c2c7-425c-47a3-8aef-9d121a859c78: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:45:35,598::task::495::TaskManager.Task::(_debug) Task db88c2c7-425c-47a3-8aef-9d121a859c78: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:45:35,599::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:45:35,599::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:45:35,600::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:45:35,601::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:45:35,602::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:45:35,603::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:45:35,604::task::495::TaskManager.Task::(_debug) Task db88c2c7-425c-47a3-8aef-9d121a859c78: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:45:35,604::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:45:35,939::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.08625 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:45:37,949::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:45:38,481::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0736681 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:45:40,496::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:45:41,029::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.067959 s, 15.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:45:43,040::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:45:43,556::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0779887 s, 13.1 MB/s\n'; <rc> = 0
>Thread-15916878::DEBUG::2013-11-20 04:45:45,551::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916878::INFO::2013-11-20 04:45:45,553::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916878::DEBUG::2013-11-20 04:45:45,553::task::495::TaskManager.Task::(_debug) Task 17d57494-260a-4ed6-b206-3db846134aae: moving from state init -> state preparing
>Thread-15916878::DEBUG::2013-11-20 04:45:45,554::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916878::DEBUG::2013-11-20 04:45:45,555::task::495::TaskManager.Task::(_debug) Task 17d57494-260a-4ed6-b206-3db846134aae: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916878::DEBUG::2013-11-20 04:45:45,555::task::495::TaskManager.Task::(_debug) Task 17d57494-260a-4ed6-b206-3db846134aae: moving from state preparing -> state finished
>Thread-15916878::DEBUG::2013-11-20 04:45:45,556::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916878::DEBUG::2013-11-20 04:45:45,556::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916878::DEBUG::2013-11-20 04:45:45,557::task::495::TaskManager.Task::(_debug) Task 17d57494-260a-4ed6-b206-3db846134aae: ref 0 aborting False
>Thread-15916878::INFO::2013-11-20 04:45:45,557::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916879::DEBUG::2013-11-20 04:45:45,566::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916879::INFO::2013-11-20 04:45:45,567::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916879::DEBUG::2013-11-20 04:45:45,568::task::495::TaskManager.Task::(_debug) Task d65fdc48-2ca7-4b1c-8ebb-1b132222c175: moving from state init -> state preparing
>Thread-15916879::DEBUG::2013-11-20 04:45:45,572::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`13742346-85b6-4995-b491-7f1cd2aceccf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Dummy-15912651::DEBUG::2013-11-20 04:45:45,572::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916879::DEBUG::2013-11-20 04:45:45,573::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15913546::INFO::2013-11-20 04:45:45,747::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15914519::INFO::2013-11-20 04:45:45,750::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15916879::DEBUG::2013-11-20 04:45:45,750::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:45:45,752::task::495::TaskManager.Task::(_debug) Task bd615cdf-6a2f-4516-a1fd-274fa6d27385: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:45:45,753::task::495::TaskManager.Task::(_debug) Task 8448378a-f61b-4af7-869b-337ae8fa4d42: moving from state init -> state preparing
>Thread-15916880::INFO::2013-11-20 04:45:45,757::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916879::DEBUG::2013-11-20 04:45:45,758::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`13742346-85b6-4995-b491-7f1cd2aceccf`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:45:45,763::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f1b665ac-1f58-4da6-8225-460d5d75670e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15916880::DEBUG::2013-11-20 04:45:45,769::task::495::TaskManager.Task::(_debug) Task 3f150b4a-d17a-421f-bf9a-1b1326db7a08: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:45:45,770::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5ca83992-e9a6-44d2-aebe-31040c5927c0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:45:45,772::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15916880::DEBUG::2013-11-20 04:45:45,779::task::495::TaskManager.Task::(_debug) Task 3f150b4a-d17a-421f-bf9a-1b1326db7a08: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000382900238037', 'lastCheck': 1384922736.577661, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00205588340759', 'lastCheck': 1384922739.5844359, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00148296356201', 'lastCheck': 1384922739.5846031, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000516891479492', 'lastCheck': 1384922741.7228141, 'valid': True, 'code': 0}}
>Thread-15914519::DEBUG::2013-11-20 04:45:45,780::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15916879::DEBUG::2013-11-20 04:45:45,780::task::495::TaskManager.Task::(_debug) Task d65fdc48-2ca7-4b1c-8ebb-1b132222c175: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15913546::DEBUG::2013-11-20 04:45:45,785::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15916880::DEBUG::2013-11-20 04:45:45,793::task::495::TaskManager.Task::(_debug) Task 3f150b4a-d17a-421f-bf9a-1b1326db7a08: moving from state preparing -> state finished
>Thread-15916879::DEBUG::2013-11-20 04:45:45,795::task::495::TaskManager.Task::(_debug) Task d65fdc48-2ca7-4b1c-8ebb-1b132222c175: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:45:45,796::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f1b665ac-1f58-4da6-8225-460d5d75670e`::Granted request
>Thread-15916880::DEBUG::2013-11-20 04:45:45,796::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916879::WARNING::2013-11-20 04:45:45,800::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916880::DEBUG::2013-11-20 04:45:45,807::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:45:45,809::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15916879::DEBUG::2013-11-20 04:45:45,810::task::495::TaskManager.Task::(_debug) Task d65fdc48-2ca7-4b1c-8ebb-1b132222c175: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913546::DEBUG::2013-11-20 04:45:45,810::task::495::TaskManager.Task::(_debug) Task bd615cdf-6a2f-4516-a1fd-274fa6d27385: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15916880::DEBUG::2013-11-20 04:45:45,811::task::495::TaskManager.Task::(_debug) Task 3f150b4a-d17a-421f-bf9a-1b1326db7a08: ref 0 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:45:45,812::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5ca83992-e9a6-44d2-aebe-31040c5927c0`::Granted request
>Thread-15916879::DEBUG::2013-11-20 04:45:45,822::task::495::TaskManager.Task::(_debug) Task d65fdc48-2ca7-4b1c-8ebb-1b132222c175: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:45:45,824::task::495::TaskManager.Task::(_debug) Task bd615cdf-6a2f-4516-a1fd-274fa6d27385: ref 1 aborting False
>Thread-15916880::INFO::2013-11-20 04:45:45,825::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000382900238037', 'lastCheck': 1384922736.577661, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00205588340759', 'lastCheck': 1384922739.5844359, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00148296356201', 'lastCheck': 1384922739.5846031, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000516891479492', 'lastCheck': 1384922741.7228141, 'valid': True, 'code': 0}}
>Thread-15914519::DEBUG::2013-11-20 04:45:45,831::task::495::TaskManager.Task::(_debug) Task 8448378a-f61b-4af7-869b-337ae8fa4d42: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15916879::DEBUG::2013-11-20 04:45:45,832::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:45:45,833::task::495::TaskManager.Task::(_debug) Task bd615cdf-6a2f-4516-a1fd-274fa6d27385: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:45:45,835::task::495::TaskManager.Task::(_debug) Task 8448378a-f61b-4af7-869b-337ae8fa4d42: ref 1 aborting False
>Thread-15916879::DEBUG::2013-11-20 04:45:45,836::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:45:45,837::task::495::TaskManager.Task::(_debug) Task bd615cdf-6a2f-4516-a1fd-274fa6d27385: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:45:45,838::task::495::TaskManager.Task::(_debug) Task 8448378a-f61b-4af7-869b-337ae8fa4d42: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:45:45,840::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15916879::DEBUG::2013-11-20 04:45:45,841::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914519::DEBUG::2013-11-20 04:45:45,841::task::495::TaskManager.Task::(_debug) Task 8448378a-f61b-4af7-869b-337ae8fa4d42: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:45:45,842::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916879::DEBUG::2013-11-20 04:45:45,843::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:45:45,847::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15916879::DEBUG::2013-11-20 04:45:45,849::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:45:45,850::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:45:45,851::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15916879::DEBUG::2013-11-20 04:45:45,851::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:45:45,854::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913546::DEBUG::2013-11-20 04:45:45,854::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15916879::DEBUG::2013-11-20 04:45:45,855::task::495::TaskManager.Task::(_debug) Task d65fdc48-2ca7-4b1c-8ebb-1b132222c175: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:45:45,856::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15916879::INFO::2013-11-20 04:45:45,865::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913546::DEBUG::2013-11-20 04:45:45,866::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:45:45,880::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:45:45,882::task::495::TaskManager.Task::(_debug) Task bd615cdf-6a2f-4516-a1fd-274fa6d27385: ref 0 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:45:45,883::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913546::INFO::2013-11-20 04:45:45,884::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:45:45,885::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913546::INFO::2013-11-20 04:45:45,886::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:45:45,887::task::495::TaskManager.Task::(_debug) Task 4f94a87c-79e1-41a0-b29e-1cf2553cd2b1: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:45:45,888::task::495::TaskManager.Task::(_debug) Task 8448378a-f61b-4af7-869b-337ae8fa4d42: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:45:45,889::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`08fef820-234d-4458-9e9a-d5423031124f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::INFO::2013-11-20 04:45:45,890::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:45:45,891::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:45:45,922::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:45:45,923::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`08fef820-234d-4458-9e9a-d5423031124f`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:45:45,925::task::495::TaskManager.Task::(_debug) Task 4f94a87c-79e1-41a0-b29e-1cf2553cd2b1: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:45:45,925::task::495::TaskManager.Task::(_debug) Task 4f94a87c-79e1-41a0-b29e-1cf2553cd2b1: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:45:45,926::task::495::TaskManager.Task::(_debug) Task 4f94a87c-79e1-41a0-b29e-1cf2553cd2b1: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:45:45,926::task::495::TaskManager.Task::(_debug) Task 4f94a87c-79e1-41a0-b29e-1cf2553cd2b1: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:45:45,927::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:45:45,927::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:45:45,928::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:45:45,928::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:45:45,929::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:45:45,929::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:45:45,929::task::495::TaskManager.Task::(_debug) Task 4f94a87c-79e1-41a0-b29e-1cf2553cd2b1: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:45:45,930::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:45:45,930::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:45:45,930::task::495::TaskManager.Task::(_debug) Task 39778450-1e61-462d-b8ba-6452d627419b: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:45:45,932::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`63c14fde-a039-4212-a20c-7cb66704e78f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:45:45,932::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:45:45,932::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:45:45,933::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`63c14fde-a039-4212-a20c-7cb66704e78f`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:45:45,934::task::495::TaskManager.Task::(_debug) Task 39778450-1e61-462d-b8ba-6452d627419b: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:45:45,934::task::495::TaskManager.Task::(_debug) Task 39778450-1e61-462d-b8ba-6452d627419b: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:45:45,934::task::495::TaskManager.Task::(_debug) Task 39778450-1e61-462d-b8ba-6452d627419b: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:45:45,935::task::495::TaskManager.Task::(_debug) Task 39778450-1e61-462d-b8ba-6452d627419b: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:45:45,935::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:45:45,935::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:45:45,936::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:45:45,936::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:45:45,936::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:45:45,937::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:45:45,937::task::495::TaskManager.Task::(_debug) Task 39778450-1e61-462d-b8ba-6452d627419b: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:45:45,938::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:45:45,938::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:45:45,938::task::495::TaskManager.Task::(_debug) Task 2885f056-4592-442e-9f90-5cb061289a2c: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:45:45,939::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`701780d8-be8e-4f2a-9216-6055646fecda`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:45:45,940::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:45:45,940::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:45:45,943::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`701780d8-be8e-4f2a-9216-6055646fecda`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:45:45,955::task::495::TaskManager.Task::(_debug) Task 2885f056-4592-442e-9f90-5cb061289a2c: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:45:45,956::task::495::TaskManager.Task::(_debug) Task 2885f056-4592-442e-9f90-5cb061289a2c: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:45:45,956::task::495::TaskManager.Task::(_debug) Task 2885f056-4592-442e-9f90-5cb061289a2c: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:45:45,957::task::495::TaskManager.Task::(_debug) Task 2885f056-4592-442e-9f90-5cb061289a2c: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:45:45,957::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:45:45,958::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:45:45,958::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:45:45,959::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:45:45,960::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:45:45,961::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:45:45,961::task::495::TaskManager.Task::(_debug) Task 2885f056-4592-442e-9f90-5cb061289a2c: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:45:45,962::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:45:46,105::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.104996 s, 9.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:45:48,121::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:45:48,658::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0777041 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:45:50,675::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:45:51,204::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0828293 s, 12.4 MB/s\n'; <rc> = 0
>Thread-15914435::INFO::2013-11-20 04:45:52,382::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:45:52,383::task::495::TaskManager.Task::(_debug) Task e83b9573-e933-460b-8c13-120229caeb2d: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:45:52,385::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9a8d6a9b-5a49-4172-a688-e9c88d654541`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:45:52,385::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:45:52,386::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:45:52,387::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9a8d6a9b-5a49-4172-a688-e9c88d654541`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:45:52,388::task::495::TaskManager.Task::(_debug) Task e83b9573-e933-460b-8c13-120229caeb2d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:45:52,389::task::495::TaskManager.Task::(_debug) Task e83b9573-e933-460b-8c13-120229caeb2d: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:45:52,390::task::495::TaskManager.Task::(_debug) Task e83b9573-e933-460b-8c13-120229caeb2d: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:45:52,390::task::495::TaskManager.Task::(_debug) Task e83b9573-e933-460b-8c13-120229caeb2d: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:45:52,391::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:45:52,392::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:45:52,393::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:45:52,393::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:45:52,394::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:45:52,394::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:45:52,395::task::495::TaskManager.Task::(_debug) Task e83b9573-e933-460b-8c13-120229caeb2d: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:45:52,396::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:45:53,218::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:45:53,777::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0805426 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:45:55,791::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:45:56,332::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0810327 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15916886::DEBUG::2013-11-20 04:45:56,970::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916886::INFO::2013-11-20 04:45:56,971::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916886::DEBUG::2013-11-20 04:45:56,972::task::495::TaskManager.Task::(_debug) Task a08f067a-657b-4c45-bb2a-1702f98a2f9a: moving from state init -> state preparing
>Thread-15916886::DEBUG::2013-11-20 04:45:56,973::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916886::DEBUG::2013-11-20 04:45:56,974::task::495::TaskManager.Task::(_debug) Task a08f067a-657b-4c45-bb2a-1702f98a2f9a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916886::DEBUG::2013-11-20 04:45:56,974::task::495::TaskManager.Task::(_debug) Task a08f067a-657b-4c45-bb2a-1702f98a2f9a: moving from state preparing -> state finished
>Thread-15916886::DEBUG::2013-11-20 04:45:56,975::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916886::DEBUG::2013-11-20 04:45:56,976::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916886::DEBUG::2013-11-20 04:45:56,976::task::495::TaskManager.Task::(_debug) Task a08f067a-657b-4c45-bb2a-1702f98a2f9a: ref 0 aborting False
>Thread-15916886::INFO::2013-11-20 04:45:56,977::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916887::DEBUG::2013-11-20 04:45:56,992::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916887::INFO::2013-11-20 04:45:56,993::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916887::DEBUG::2013-11-20 04:45:56,994::task::495::TaskManager.Task::(_debug) Task 3abcd541-c37c-4d14-a294-9dfcc1068eff: moving from state init -> state preparing
>Thread-15916887::DEBUG::2013-11-20 04:45:56,995::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b8c61580-cb08-441e-a392-98904b2f9a88`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916887::DEBUG::2013-11-20 04:45:56,996::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916887::DEBUG::2013-11-20 04:45:56,997::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916887::DEBUG::2013-11-20 04:45:57,000::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b8c61580-cb08-441e-a392-98904b2f9a88`::Granted request
>Thread-15916887::DEBUG::2013-11-20 04:45:57,002::task::495::TaskManager.Task::(_debug) Task 3abcd541-c37c-4d14-a294-9dfcc1068eff: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916887::DEBUG::2013-11-20 04:45:57,003::task::495::TaskManager.Task::(_debug) Task 3abcd541-c37c-4d14-a294-9dfcc1068eff: ref 1 aborting False
>Thread-15916887::WARNING::2013-11-20 04:45:57,005::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916887::DEBUG::2013-11-20 04:45:57,012::task::495::TaskManager.Task::(_debug) Task 3abcd541-c37c-4d14-a294-9dfcc1068eff: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916887::DEBUG::2013-11-20 04:45:57,013::task::495::TaskManager.Task::(_debug) Task 3abcd541-c37c-4d14-a294-9dfcc1068eff: moving from state preparing -> state finished
>Thread-15916887::DEBUG::2013-11-20 04:45:57,013::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916887::DEBUG::2013-11-20 04:45:57,014::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916887::DEBUG::2013-11-20 04:45:57,015::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916887::DEBUG::2013-11-20 04:45:57,016::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916887::DEBUG::2013-11-20 04:45:57,016::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916887::DEBUG::2013-11-20 04:45:57,017::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916887::DEBUG::2013-11-20 04:45:57,018::task::495::TaskManager.Task::(_debug) Task 3abcd541-c37c-4d14-a294-9dfcc1068eff: ref 0 aborting False
>Thread-15916887::INFO::2013-11-20 04:45:57,019::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916888::INFO::2013-11-20 04:45:57,771::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916888::DEBUG::2013-11-20 04:45:57,772::task::495::TaskManager.Task::(_debug) Task 6244c050-02b9-4c4d-b20b-4ecd1b0795d1: moving from state init -> state preparing
>Thread-15916888::DEBUG::2013-11-20 04:45:57,772::task::495::TaskManager.Task::(_debug) Task 6244c050-02b9-4c4d-b20b-4ecd1b0795d1: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000508069992065', 'lastCheck': 1384922757.0033109, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00139403343201', 'lastCheck': 1384922749.7496879, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00117301940918', 'lastCheck': 1384922749.749964, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000548124313354', 'lastCheck': 1384922751.8617201, 'valid': True, 'code': 0}}
>Thread-15916888::DEBUG::2013-11-20 04:45:57,784::task::495::TaskManager.Task::(_debug) Task 6244c050-02b9-4c4d-b20b-4ecd1b0795d1: moving from state preparing -> state finished
>Thread-15916888::DEBUG::2013-11-20 04:45:57,785::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916888::DEBUG::2013-11-20 04:45:57,785::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916888::DEBUG::2013-11-20 04:45:57,786::task::495::TaskManager.Task::(_debug) Task 6244c050-02b9-4c4d-b20b-4ecd1b0795d1: ref 0 aborting False
>Thread-15916888::INFO::2013-11-20 04:45:57,787::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000508069992065', 'lastCheck': 1384922757.0033109, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00139403343201', 'lastCheck': 1384922749.7496879, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00117301940918', 'lastCheck': 1384922749.749964, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000548124313354', 'lastCheck': 1384922751.8617201, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:45:58,346::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:45:58,942::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0830746 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:46:00,952::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:46:01,499::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0813061 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:46:03,514::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:46:03,721::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:46:03,723::task::495::TaskManager.Task::(_debug) Task e35dbe44-6e85-4df2-8e6a-187387601522: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:46:03,726::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`50b50405-3973-4508-a6ae-19047a1ca89c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:46:03,726::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:46:03,728::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:46:03,728::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`50b50405-3973-4508-a6ae-19047a1ca89c`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:46:03,730::task::495::TaskManager.Task::(_debug) Task e35dbe44-6e85-4df2-8e6a-187387601522: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:46:03,731::task::495::TaskManager.Task::(_debug) Task e35dbe44-6e85-4df2-8e6a-187387601522: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:46:03,732::task::495::TaskManager.Task::(_debug) Task e35dbe44-6e85-4df2-8e6a-187387601522: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:46:03,732::task::495::TaskManager.Task::(_debug) Task e35dbe44-6e85-4df2-8e6a-187387601522: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:46:03,733::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:46:03,734::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:46:03,735::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:46:03,735::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:46:03,736::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:46:03,737::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:46:03,737::task::495::TaskManager.Task::(_debug) Task e35dbe44-6e85-4df2-8e6a-187387601522: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:46:03,738::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:46:04,068::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0807167 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:46:06,083::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915918::INFO::2013-11-20 04:46:06,315::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:46:06,327::task::495::TaskManager.Task::(_debug) Task cceeeed8-bd0a-4bad-923e-40efafcc5343: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:46:06,331::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`956c0bea-6470-422b-9ab8-7b8c291c9551`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:46:06,332::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:46:06,336::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:46:06,339::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`956c0bea-6470-422b-9ab8-7b8c291c9551`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:46:06,346::task::495::TaskManager.Task::(_debug) Task cceeeed8-bd0a-4bad-923e-40efafcc5343: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:46:06,347::task::495::TaskManager.Task::(_debug) Task cceeeed8-bd0a-4bad-923e-40efafcc5343: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:46:06,348::task::495::TaskManager.Task::(_debug) Task cceeeed8-bd0a-4bad-923e-40efafcc5343: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:46:06,348::task::495::TaskManager.Task::(_debug) Task cceeeed8-bd0a-4bad-923e-40efafcc5343: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:46:06,349::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:46:06,350::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:46:06,351::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:46:06,351::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:46:06,352::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:46:06,352::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:46:06,353::task::495::TaskManager.Task::(_debug) Task cceeeed8-bd0a-4bad-923e-40efafcc5343: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:46:06,354::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:46:06,667::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.08099 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:46:08,682::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:46:09,210::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0803216 s, 12.7 MB/s\n'; <rc> = 0
>Thread-15916894::DEBUG::2013-11-20 04:46:10,615::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916894::INFO::2013-11-20 04:46:10,616::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916894::DEBUG::2013-11-20 04:46:10,617::task::495::TaskManager.Task::(_debug) Task 9b4ae59f-403a-48c7-aabe-c141c750b8c2: moving from state init -> state preparing
>Thread-15916894::DEBUG::2013-11-20 04:46:10,618::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916894::DEBUG::2013-11-20 04:46:10,619::task::495::TaskManager.Task::(_debug) Task 9b4ae59f-403a-48c7-aabe-c141c750b8c2: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916894::DEBUG::2013-11-20 04:46:10,619::task::495::TaskManager.Task::(_debug) Task 9b4ae59f-403a-48c7-aabe-c141c750b8c2: moving from state preparing -> state finished
>Thread-15916894::DEBUG::2013-11-20 04:46:10,620::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916894::DEBUG::2013-11-20 04:46:10,621::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916894::DEBUG::2013-11-20 04:46:10,621::task::495::TaskManager.Task::(_debug) Task 9b4ae59f-403a-48c7-aabe-c141c750b8c2: ref 0 aborting False
>Thread-15916894::INFO::2013-11-20 04:46:10,622::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916895::DEBUG::2013-11-20 04:46:10,641::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916895::INFO::2013-11-20 04:46:10,642::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916895::DEBUG::2013-11-20 04:46:10,643::task::495::TaskManager.Task::(_debug) Task 8af49d3f-a0b3-4d5b-a2d6-6eb455321336: moving from state init -> state preparing
>Thread-15916895::DEBUG::2013-11-20 04:46:10,644::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a9745ea0-efa1-4dec-a2c5-633bdb9c51ad`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916895::DEBUG::2013-11-20 04:46:10,645::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916895::DEBUG::2013-11-20 04:46:10,646::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916895::DEBUG::2013-11-20 04:46:10,647::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a9745ea0-efa1-4dec-a2c5-633bdb9c51ad`::Granted request
>Thread-15916895::DEBUG::2013-11-20 04:46:10,649::task::495::TaskManager.Task::(_debug) Task 8af49d3f-a0b3-4d5b-a2d6-6eb455321336: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916895::DEBUG::2013-11-20 04:46:10,649::task::495::TaskManager.Task::(_debug) Task 8af49d3f-a0b3-4d5b-a2d6-6eb455321336: ref 1 aborting False
>Thread-15916895::WARNING::2013-11-20 04:46:10,653::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916895::DEBUG::2013-11-20 04:46:10,672::task::495::TaskManager.Task::(_debug) Task 8af49d3f-a0b3-4d5b-a2d6-6eb455321336: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916896::INFO::2013-11-20 04:46:10,673::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916895::DEBUG::2013-11-20 04:46:10,675::task::495::TaskManager.Task::(_debug) Task 8af49d3f-a0b3-4d5b-a2d6-6eb455321336: moving from state preparing -> state finished
>Thread-15916896::DEBUG::2013-11-20 04:46:10,675::task::495::TaskManager.Task::(_debug) Task 21e9e701-a68e-462f-b670-22d67426717f: moving from state init -> state preparing
>Thread-15916895::DEBUG::2013-11-20 04:46:10,676::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916896::DEBUG::2013-11-20 04:46:10,677::task::495::TaskManager.Task::(_debug) Task 21e9e701-a68e-462f-b670-22d67426717f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000375986099243', 'lastCheck': 1384922767.3160801, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00174808502197', 'lastCheck': 1384922770.320914, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00133204460144', 'lastCheck': 1384922770.321063, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000462055206299', 'lastCheck': 1384922762.170532, 'valid': True, 'code': 0}}
>Thread-15916895::DEBUG::2013-11-20 04:46:10,678::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916896::DEBUG::2013-11-20 04:46:10,679::task::495::TaskManager.Task::(_debug) Task 21e9e701-a68e-462f-b670-22d67426717f: moving from state preparing -> state finished
>Thread-15916895::DEBUG::2013-11-20 04:46:10,680::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916896::DEBUG::2013-11-20 04:46:10,681::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916895::DEBUG::2013-11-20 04:46:10,682::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916896::DEBUG::2013-11-20 04:46:10,683::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916895::DEBUG::2013-11-20 04:46:10,684::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916896::DEBUG::2013-11-20 04:46:10,684::task::495::TaskManager.Task::(_debug) Task 21e9e701-a68e-462f-b670-22d67426717f: ref 0 aborting False
>Thread-15916895::DEBUG::2013-11-20 04:46:10,685::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916896::INFO::2013-11-20 04:46:10,686::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000375986099243', 'lastCheck': 1384922767.3160801, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00174808502197', 'lastCheck': 1384922770.320914, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00133204460144', 'lastCheck': 1384922770.321063, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000462055206299', 'lastCheck': 1384922762.170532, 'valid': True, 'code': 0}}
>Thread-15916895::DEBUG::2013-11-20 04:46:10,687::task::495::TaskManager.Task::(_debug) Task 8af49d3f-a0b3-4d5b-a2d6-6eb455321336: ref 0 aborting False
>Thread-15916895::INFO::2013-11-20 04:46:10,689::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:46:11,226::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915863::INFO::2013-11-20 04:46:11,457::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:46:11,467::task::495::TaskManager.Task::(_debug) Task 17d49f04-0c5f-419f-8d8f-f56b6d9ef335: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:46:11,479::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6876f596-a84b-4ce8-a739-cf86f7bc95be`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:46:11,482::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:46:11,483::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:46:11,484::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6876f596-a84b-4ce8-a739-cf86f7bc95be`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:46:11,485::task::495::TaskManager.Task::(_debug) Task 17d49f04-0c5f-419f-8d8f-f56b6d9ef335: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:46:11,486::task::495::TaskManager.Task::(_debug) Task 17d49f04-0c5f-419f-8d8f-f56b6d9ef335: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:46:11,487::task::495::TaskManager.Task::(_debug) Task 17d49f04-0c5f-419f-8d8f-f56b6d9ef335: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:46:11,488::task::495::TaskManager.Task::(_debug) Task 17d49f04-0c5f-419f-8d8f-f56b6d9ef335: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:46:11,489::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:46:11,489::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:46:11,490::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:46:11,491::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:46:11,492::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:46:11,492::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:46:11,493::task::495::TaskManager.Task::(_debug) Task 17d49f04-0c5f-419f-8d8f-f56b6d9ef335: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:46:11,494::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:46:11,784::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0896599 s, 11.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:46:13,801::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:46:14,347::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0926732 s, 11.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:46:16,360::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 04:46:16,580::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:46:16,583::task::495::TaskManager.Task::(_debug) Task 6ab00af9-5b8b-4b7c-a253-2035eea8b822: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:46:16,585::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`391a5f4c-0638-4584-898c-64aff099fd73`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:46:16,586::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:46:16,588::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:46:16,588::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`391a5f4c-0638-4584-898c-64aff099fd73`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:46:16,589::task::495::TaskManager.Task::(_debug) Task 6ab00af9-5b8b-4b7c-a253-2035eea8b822: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:46:16,589::task::495::TaskManager.Task::(_debug) Task 6ab00af9-5b8b-4b7c-a253-2035eea8b822: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:46:16,590::task::495::TaskManager.Task::(_debug) Task 6ab00af9-5b8b-4b7c-a253-2035eea8b822: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:46:16,590::task::495::TaskManager.Task::(_debug) Task 6ab00af9-5b8b-4b7c-a253-2035eea8b822: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:46:16,590::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:46:16,591::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:46:16,591::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:46:16,591::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:46:16,592::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:46:16,592::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:46:16,593::task::495::TaskManager.Task::(_debug) Task 6ab00af9-5b8b-4b7c-a253-2035eea8b822: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:46:16,593::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:46:16,848::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0769955 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:46:18,864::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:46:19,414::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0789066 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:46:21,430::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:46:21,961::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0761805 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15916902::DEBUG::2013-11-20 04:46:22,004::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916902::INFO::2013-11-20 04:46:22,007::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916902::DEBUG::2013-11-20 04:46:22,008::task::495::TaskManager.Task::(_debug) Task 96e8131a-e521-4742-b38c-966412a5ac76: moving from state init -> state preparing
>Thread-15916902::DEBUG::2013-11-20 04:46:22,009::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916902::DEBUG::2013-11-20 04:46:22,010::task::495::TaskManager.Task::(_debug) Task 96e8131a-e521-4742-b38c-966412a5ac76: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916902::DEBUG::2013-11-20 04:46:22,010::task::495::TaskManager.Task::(_debug) Task 96e8131a-e521-4742-b38c-966412a5ac76: moving from state preparing -> state finished
>Thread-15916902::DEBUG::2013-11-20 04:46:22,011::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916902::DEBUG::2013-11-20 04:46:22,012::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916902::DEBUG::2013-11-20 04:46:22,012::task::495::TaskManager.Task::(_debug) Task 96e8131a-e521-4742-b38c-966412a5ac76: ref 0 aborting False
>Thread-15916902::INFO::2013-11-20 04:46:22,013::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916903::DEBUG::2013-11-20 04:46:22,054::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916903::INFO::2013-11-20 04:46:22,058::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916903::DEBUG::2013-11-20 04:46:22,059::task::495::TaskManager.Task::(_debug) Task 1454e625-6dba-4984-bb72-1a80cdbe97ee: moving from state init -> state preparing
>Thread-15916903::DEBUG::2013-11-20 04:46:22,062::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2130ce9b-825a-4ba7-b7dd-745e289091f9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916903::DEBUG::2013-11-20 04:46:22,063::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916903::DEBUG::2013-11-20 04:46:22,064::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916903::DEBUG::2013-11-20 04:46:22,065::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2130ce9b-825a-4ba7-b7dd-745e289091f9`::Granted request
>Thread-15916903::DEBUG::2013-11-20 04:46:22,066::task::495::TaskManager.Task::(_debug) Task 1454e625-6dba-4984-bb72-1a80cdbe97ee: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916903::DEBUG::2013-11-20 04:46:22,066::task::495::TaskManager.Task::(_debug) Task 1454e625-6dba-4984-bb72-1a80cdbe97ee: ref 1 aborting False
>Thread-15916903::WARNING::2013-11-20 04:46:22,069::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916903::DEBUG::2013-11-20 04:46:22,075::task::495::TaskManager.Task::(_debug) Task 1454e625-6dba-4984-bb72-1a80cdbe97ee: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916903::DEBUG::2013-11-20 04:46:22,083::task::495::TaskManager.Task::(_debug) Task 1454e625-6dba-4984-bb72-1a80cdbe97ee: moving from state preparing -> state finished
>Thread-15916903::DEBUG::2013-11-20 04:46:22,084::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916903::DEBUG::2013-11-20 04:46:22,084::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916903::DEBUG::2013-11-20 04:46:22,085::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916903::DEBUG::2013-11-20 04:46:22,086::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916903::DEBUG::2013-11-20 04:46:22,086::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916903::DEBUG::2013-11-20 04:46:22,087::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916903::DEBUG::2013-11-20 04:46:22,088::task::495::TaskManager.Task::(_debug) Task 1454e625-6dba-4984-bb72-1a80cdbe97ee: ref 0 aborting False
>Thread-15916903::INFO::2013-11-20 04:46:22,088::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916904::INFO::2013-11-20 04:46:22,310::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916904::DEBUG::2013-11-20 04:46:22,311::task::495::TaskManager.Task::(_debug) Task c9908588-51e5-4f03-8d6c-929825b7507a: moving from state init -> state preparing
>Thread-15916904::DEBUG::2013-11-20 04:46:22,312::task::495::TaskManager.Task::(_debug) Task c9908588-51e5-4f03-8d6c-929825b7507a: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000394105911255', 'lastCheck': 1384922777.5759261, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00208806991577', 'lastCheck': 1384922780.580986, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00145888328552', 'lastCheck': 1384922780.582036, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000409126281738', 'lastCheck': 1384922772.4442971, 'valid': True, 'code': 0}}
>Thread-15916904::DEBUG::2013-11-20 04:46:22,313::task::495::TaskManager.Task::(_debug) Task c9908588-51e5-4f03-8d6c-929825b7507a: moving from state preparing -> state finished
>Thread-15916904::DEBUG::2013-11-20 04:46:22,314::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916904::DEBUG::2013-11-20 04:46:22,314::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916904::DEBUG::2013-11-20 04:46:22,315::task::495::TaskManager.Task::(_debug) Task c9908588-51e5-4f03-8d6c-929825b7507a: ref 0 aborting False
>Thread-15916904::INFO::2013-11-20 04:46:22,316::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000394105911255', 'lastCheck': 1384922777.5759261, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00208806991577', 'lastCheck': 1384922780.580986, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00145888328552', 'lastCheck': 1384922780.582036, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000409126281738', 'lastCheck': 1384922772.4442971, 'valid': True, 'code': 0}}
>Thread-42840::DEBUG::2013-11-20 04:46:22,628::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 04:46:23,455::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913518::INFO::2013-11-20 04:46:23,693::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:46:23,694::task::495::TaskManager.Task::(_debug) Task 01120b60-7d8a-415b-b70f-609929e6265e: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:46:23,696::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8cdd72ad-733f-45e3-8d33-d19bf875b53b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:46:23,697::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:46:23,697::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:46:23,698::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8cdd72ad-733f-45e3-8d33-d19bf875b53b`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:46:23,699::task::495::TaskManager.Task::(_debug) Task 01120b60-7d8a-415b-b70f-609929e6265e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:46:23,700::task::495::TaskManager.Task::(_debug) Task 01120b60-7d8a-415b-b70f-609929e6265e: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:46:23,700::task::495::TaskManager.Task::(_debug) Task 01120b60-7d8a-415b-b70f-609929e6265e: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:46:23,701::task::495::TaskManager.Task::(_debug) Task 01120b60-7d8a-415b-b70f-609929e6265e: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:46:23,702::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:46:23,702::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:46:23,703::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:46:23,704::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:46:23,704::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:46:23,705::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:46:23,705::task::495::TaskManager.Task::(_debug) Task 01120b60-7d8a-415b-b70f-609929e6265e: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:46:23,706::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:46:23,707::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:46:23,707::task::495::TaskManager.Task::(_debug) Task a3dab835-8e5d-48a6-a245-3b1b75dbcbe9: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:46:23,708::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a62ea52b-4963-42de-a477-6c6b31c7498e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:46:23,709::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:46:23,710::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:46:23,710::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a62ea52b-4963-42de-a477-6c6b31c7498e`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:46:23,711::task::495::TaskManager.Task::(_debug) Task a3dab835-8e5d-48a6-a245-3b1b75dbcbe9: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:46:23,712::task::495::TaskManager.Task::(_debug) Task a3dab835-8e5d-48a6-a245-3b1b75dbcbe9: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:46:23,712::task::495::TaskManager.Task::(_debug) Task a3dab835-8e5d-48a6-a245-3b1b75dbcbe9: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:46:23,713::task::495::TaskManager.Task::(_debug) Task a3dab835-8e5d-48a6-a245-3b1b75dbcbe9: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:46:23,713::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:46:23,714::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:46:23,714::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:46:23,715::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:46:23,715::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:46:23,716::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:46:23,717::task::495::TaskManager.Task::(_debug) Task a3dab835-8e5d-48a6-a245-3b1b75dbcbe9: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:46:23,717::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:46:24,009::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:46:24,544::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.07861 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:46:26,561::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 04:46:26,838::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:46:26,840::task::495::TaskManager.Task::(_debug) Task 1470613a-78b7-4ec0-b60e-490a246d0dd4: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:46:26,842::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9f7f2e09-7d09-4e64-acc0-009fb1a4dd95`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:46:26,843::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:46:26,844::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:46:26,845::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9f7f2e09-7d09-4e64-acc0-009fb1a4dd95`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:46:26,848::task::495::TaskManager.Task::(_debug) Task 1470613a-78b7-4ec0-b60e-490a246d0dd4: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:46:26,849::task::495::TaskManager.Task::(_debug) Task 1470613a-78b7-4ec0-b60e-490a246d0dd4: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:46:26,852::task::495::TaskManager.Task::(_debug) Task 1470613a-78b7-4ec0-b60e-490a246d0dd4: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:46:26,853::task::495::TaskManager.Task::(_debug) Task 1470613a-78b7-4ec0-b60e-490a246d0dd4: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:46:26,854::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:46:26,855::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:46:26,856::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:46:26,857::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:46:26,857::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:46:26,858::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:46:26,859::task::495::TaskManager.Task::(_debug) Task 1470613a-78b7-4ec0-b60e-490a246d0dd4: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:46:26,860::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:46:27,092::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0736236 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:46:29,108::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:46:29,668::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0742182 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:46:31,684::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:46:32,226::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0835585 s, 12.3 MB/s\n'; <rc> = 0
>Thread-15916910::DEBUG::2013-11-20 04:46:33,206::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916910::INFO::2013-11-20 04:46:33,208::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916910::DEBUG::2013-11-20 04:46:33,208::task::495::TaskManager.Task::(_debug) Task bbfc1b2e-2033-4efd-a53a-65396a860686: moving from state init -> state preparing
>Thread-15916910::DEBUG::2013-11-20 04:46:33,209::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916910::DEBUG::2013-11-20 04:46:33,210::task::495::TaskManager.Task::(_debug) Task bbfc1b2e-2033-4efd-a53a-65396a860686: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916910::DEBUG::2013-11-20 04:46:33,211::task::495::TaskManager.Task::(_debug) Task bbfc1b2e-2033-4efd-a53a-65396a860686: moving from state preparing -> state finished
>Thread-15916910::DEBUG::2013-11-20 04:46:33,211::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916910::DEBUG::2013-11-20 04:46:33,212::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916910::DEBUG::2013-11-20 04:46:33,212::task::495::TaskManager.Task::(_debug) Task bbfc1b2e-2033-4efd-a53a-65396a860686: ref 0 aborting False
>Thread-15916910::INFO::2013-11-20 04:46:33,213::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916911::DEBUG::2013-11-20 04:46:33,222::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916911::INFO::2013-11-20 04:46:33,223::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916911::DEBUG::2013-11-20 04:46:33,223::task::495::TaskManager.Task::(_debug) Task c15845da-c48e-4b80-b1ee-ab555434a357: moving from state init -> state preparing
>Thread-15916911::DEBUG::2013-11-20 04:46:33,224::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8fc9507e-7557-47d8-a612-2f32a72e857b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916911::DEBUG::2013-11-20 04:46:33,225::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916911::DEBUG::2013-11-20 04:46:33,229::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916911::DEBUG::2013-11-20 04:46:33,230::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8fc9507e-7557-47d8-a612-2f32a72e857b`::Granted request
>Thread-15916911::DEBUG::2013-11-20 04:46:33,230::task::495::TaskManager.Task::(_debug) Task c15845da-c48e-4b80-b1ee-ab555434a357: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916911::DEBUG::2013-11-20 04:46:33,231::task::495::TaskManager.Task::(_debug) Task c15845da-c48e-4b80-b1ee-ab555434a357: ref 1 aborting False
>Thread-15916911::WARNING::2013-11-20 04:46:33,232::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916911::DEBUG::2013-11-20 04:46:33,238::task::495::TaskManager.Task::(_debug) Task c15845da-c48e-4b80-b1ee-ab555434a357: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916911::DEBUG::2013-11-20 04:46:33,249::task::495::TaskManager.Task::(_debug) Task c15845da-c48e-4b80-b1ee-ab555434a357: moving from state preparing -> state finished
>Thread-15916911::DEBUG::2013-11-20 04:46:33,249::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916911::DEBUG::2013-11-20 04:46:33,250::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916911::DEBUG::2013-11-20 04:46:33,251::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916911::DEBUG::2013-11-20 04:46:33,251::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916911::DEBUG::2013-11-20 04:46:33,252::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916911::DEBUG::2013-11-20 04:46:33,253::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916911::DEBUG::2013-11-20 04:46:33,253::task::495::TaskManager.Task::(_debug) Task c15845da-c48e-4b80-b1ee-ab555434a357: ref 0 aborting False
>Thread-15916911::INFO::2013-11-20 04:46:33,256::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916912::INFO::2013-11-20 04:46:33,993::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916912::DEBUG::2013-11-20 04:46:33,994::task::495::TaskManager.Task::(_debug) Task 3a90c125-13ee-46b7-a133-6fea4bdb5426: moving from state init -> state preparing
>Thread-15916912::DEBUG::2013-11-20 04:46:33,994::task::495::TaskManager.Task::(_debug) Task 3a90c125-13ee-46b7-a133-6fea4bdb5426: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00829005241394', 'lastCheck': 1384922787.7726331, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00182008743286', 'lastCheck': 1384922790.7828031, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00213599205017', 'lastCheck': 1384922790.783601, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000502824783325', 'lastCheck': 1384922793.4689431, 'valid': True, 'code': 0}}
>Thread-15916912::DEBUG::2013-11-20 04:46:33,995::task::495::TaskManager.Task::(_debug) Task 3a90c125-13ee-46b7-a133-6fea4bdb5426: moving from state preparing -> state finished
>Thread-15916912::DEBUG::2013-11-20 04:46:33,996::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916912::DEBUG::2013-11-20 04:46:33,997::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916912::DEBUG::2013-11-20 04:46:33,997::task::495::TaskManager.Task::(_debug) Task 3a90c125-13ee-46b7-a133-6fea4bdb5426: ref 0 aborting False
>Thread-15916912::INFO::2013-11-20 04:46:33,998::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00829005241394', 'lastCheck': 1384922787.7726331, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00182008743286', 'lastCheck': 1384922790.7828031, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00213599205017', 'lastCheck': 1384922790.783601, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000502824783325', 'lastCheck': 1384922793.4689431, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:46:34,242::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:46:34,788::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0795988 s, 12.9 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 04:46:34,902::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:46:34,903::task::495::TaskManager.Task::(_debug) Task 2b50e31e-02e5-4b1c-ab7b-a8285616df0a: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:46:34,906::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`74e09767-cef6-46da-87b3-4f0459d68041`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:46:34,906::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:46:34,907::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:46:34,908::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`74e09767-cef6-46da-87b3-4f0459d68041`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:46:34,909::task::495::TaskManager.Task::(_debug) Task 2b50e31e-02e5-4b1c-ab7b-a8285616df0a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:46:34,910::task::495::TaskManager.Task::(_debug) Task 2b50e31e-02e5-4b1c-ab7b-a8285616df0a: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:46:34,910::task::495::TaskManager.Task::(_debug) Task 2b50e31e-02e5-4b1c-ab7b-a8285616df0a: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:46:34,911::task::495::TaskManager.Task::(_debug) Task 2b50e31e-02e5-4b1c-ab7b-a8285616df0a: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:46:34,912::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:46:34,912::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:46:34,913::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:46:34,914::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:46:34,915::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:46:34,915::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:46:34,916::task::495::TaskManager.Task::(_debug) Task 2b50e31e-02e5-4b1c-ab7b-a8285616df0a: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:46:34,917::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:46:36,805::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 04:46:37,045::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:46:37,048::task::495::TaskManager.Task::(_debug) Task 5586e859-520c-48fe-ac9b-f492a999b6bc: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:46:37,053::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`23be026e-1d43-4e56-9d49-a513c7151909`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:46:37,054::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:46:37,056::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:46:37,056::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`23be026e-1d43-4e56-9d49-a513c7151909`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:46:37,059::task::495::TaskManager.Task::(_debug) Task 5586e859-520c-48fe-ac9b-f492a999b6bc: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:46:37,060::task::495::TaskManager.Task::(_debug) Task 5586e859-520c-48fe-ac9b-f492a999b6bc: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:46:37,061::task::495::TaskManager.Task::(_debug) Task 5586e859-520c-48fe-ac9b-f492a999b6bc: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:46:37,062::task::495::TaskManager.Task::(_debug) Task 5586e859-520c-48fe-ac9b-f492a999b6bc: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:46:37,063::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:46:37,063::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:46:37,064::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:46:37,065::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:46:37,066::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:46:37,066::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:46:37,067::task::495::TaskManager.Task::(_debug) Task 5586e859-520c-48fe-ac9b-f492a999b6bc: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:46:37,068::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:46:37,369::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0846876 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:46:39,386::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:46:39,959::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.101363 s, 10.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:46:41,975::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:46:42,541::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0882641 s, 11.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:46:44,556::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:46:45,081::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.080588 s, 12.7 MB/s\n'; <rc> = 0
>Thread-15916918::INFO::2013-11-20 04:46:45,959::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916918::DEBUG::2013-11-20 04:46:45,960::task::495::TaskManager.Task::(_debug) Task 8c6df3cf-b336-4467-bf3e-93629834da1d: moving from state init -> state preparing
>Thread-15916918::DEBUG::2013-11-20 04:46:45,961::task::495::TaskManager.Task::(_debug) Task 8c6df3cf-b336-4467-bf3e-93629834da1d: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00377702713013', 'lastCheck': 1384922798.049217, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.006422996521', 'lastCheck': 1384922801.0754721, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00438117980957', 'lastCheck': 1384922801.0752971, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000478982925415', 'lastCheck': 1384922803.601423, 'valid': True, 'code': 0}}
>Thread-15916918::DEBUG::2013-11-20 04:46:45,962::task::495::TaskManager.Task::(_debug) Task 8c6df3cf-b336-4467-bf3e-93629834da1d: moving from state preparing -> state finished
>Thread-15916918::DEBUG::2013-11-20 04:46:45,963::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916918::DEBUG::2013-11-20 04:46:45,963::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916918::DEBUG::2013-11-20 04:46:45,964::task::495::TaskManager.Task::(_debug) Task 8c6df3cf-b336-4467-bf3e-93629834da1d: ref 0 aborting False
>Thread-15916918::INFO::2013-11-20 04:46:45,965::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00377702713013', 'lastCheck': 1384922798.049217, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.006422996521', 'lastCheck': 1384922801.0754721, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00438117980957', 'lastCheck': 1384922801.0752971, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000478982925415', 'lastCheck': 1384922803.601423, 'valid': True, 'code': 0}}
>Thread-15916920::DEBUG::2013-11-20 04:46:47,089::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916920::INFO::2013-11-20 04:46:47,091::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916920::DEBUG::2013-11-20 04:46:47,092::task::495::TaskManager.Task::(_debug) Task 034b1bcb-ecc8-40da-9e73-18b6b959e756: moving from state init -> state preparing
>Thread-15916920::DEBUG::2013-11-20 04:46:47,093::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916920::DEBUG::2013-11-20 04:46:47,093::task::495::TaskManager.Task::(_debug) Task 034b1bcb-ecc8-40da-9e73-18b6b959e756: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916920::DEBUG::2013-11-20 04:46:47,094::task::495::TaskManager.Task::(_debug) Task 034b1bcb-ecc8-40da-9e73-18b6b959e756: moving from state preparing -> state finished
>Thread-15916920::DEBUG::2013-11-20 04:46:47,095::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916920::DEBUG::2013-11-20 04:46:47,095::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Dummy-15912651::DEBUG::2013-11-20 04:46:47,096::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916920::DEBUG::2013-11-20 04:46:47,098::task::495::TaskManager.Task::(_debug) Task 034b1bcb-ecc8-40da-9e73-18b6b959e756: ref 0 aborting False
>Thread-15916920::INFO::2013-11-20 04:46:47,322::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15913546::INFO::2013-11-20 04:46:47,339::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15914519::INFO::2013-11-20 04:46:47,344::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15913546::DEBUG::2013-11-20 04:46:47,351::task::495::TaskManager.Task::(_debug) Task becec105-29ae-4c58-a6e0-c751f8514555: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:46:47,356::task::495::TaskManager.Task::(_debug) Task fb748894-cb27-4e57-9964-b6ca9e0a569b: moving from state init -> state preparing
>Thread-15916921::DEBUG::2013-11-20 04:46:47,375::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15913546::DEBUG::2013-11-20 04:46:47,377::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b57b6943-0de6-4610-bb51-7a605b0646a2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:46:47,385::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`533d0e28-915e-49ac-96a5-001b8fe1ccaa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15916921::INFO::2013-11-20 04:46:47,386::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15913546::DEBUG::2013-11-20 04:46:47,387::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:46:47,388::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15916921::DEBUG::2013-11-20 04:46:47,388::task::495::TaskManager.Task::(_debug) Task 3d64c386-766d-449f-88f8-56f6dd725cbe: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:46:47,391::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15916921::DEBUG::2013-11-20 04:46:47,393::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0f0c69eb-86f9-417a-a6a2-290cb1060f1e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15913546::DEBUG::2013-11-20 04:46:47,393::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b57b6943-0de6-4610-bb51-7a605b0646a2`::Granted request
>Thread-15916921::DEBUG::2013-11-20 04:46:47,394::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:46:47,396::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:46:47,397::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`533d0e28-915e-49ac-96a5-001b8fe1ccaa`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:46:47,398::task::495::TaskManager.Task::(_debug) Task becec105-29ae-4c58-a6e0-c751f8514555: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15916921::DEBUG::2013-11-20 04:46:47,401::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:46:47,402::task::495::TaskManager.Task::(_debug) Task becec105-29ae-4c58-a6e0-c751f8514555: ref 1 aborting False
>Thread-15916921::DEBUG::2013-11-20 04:46:47,403::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0f0c69eb-86f9-417a-a6a2-290cb1060f1e`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:46:47,404::task::495::TaskManager.Task::(_debug) Task fb748894-cb27-4e57-9964-b6ca9e0a569b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913546::DEBUG::2013-11-20 04:46:47,405::task::495::TaskManager.Task::(_debug) Task becec105-29ae-4c58-a6e0-c751f8514555: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15916921::DEBUG::2013-11-20 04:46:47,407::task::495::TaskManager.Task::(_debug) Task 3d64c386-766d-449f-88f8-56f6dd725cbe: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914519::DEBUG::2013-11-20 04:46:47,407::task::495::TaskManager.Task::(_debug) Task fb748894-cb27-4e57-9964-b6ca9e0a569b: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:46:47,408::task::495::TaskManager.Task::(_debug) Task becec105-29ae-4c58-a6e0-c751f8514555: moving from state preparing -> state finished
>Thread-15916921::DEBUG::2013-11-20 04:46:47,409::task::495::TaskManager.Task::(_debug) Task 3d64c386-766d-449f-88f8-56f6dd725cbe: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:46:47,410::task::495::TaskManager.Task::(_debug) Task fb748894-cb27-4e57-9964-b6ca9e0a569b: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:46:47,411::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:46:47,413::task::495::TaskManager.Task::(_debug) Task fb748894-cb27-4e57-9964-b6ca9e0a569b: moving from state preparing -> state finished
>Thread-15916921::WARNING::2013-11-20 04:46:47,415::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15913546::DEBUG::2013-11-20 04:46:47,416::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:46:47,417::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15916921::DEBUG::2013-11-20 04:46:47,423::task::495::TaskManager.Task::(_debug) Task 3d64c386-766d-449f-88f8-56f6dd725cbe: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914519::DEBUG::2013-11-20 04:46:47,425::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:46:47,425::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15916921::DEBUG::2013-11-20 04:46:47,427::task::495::TaskManager.Task::(_debug) Task 3d64c386-766d-449f-88f8-56f6dd725cbe: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:46:47,428::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913546::DEBUG::2013-11-20 04:46:47,428::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15916921::DEBUG::2013-11-20 04:46:47,429::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:46:47,430::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15916921::DEBUG::2013-11-20 04:46:47,431::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:46:47,432::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15916921::DEBUG::2013-11-20 04:46:47,433::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15914519::DEBUG::2013-11-20 04:46:47,434::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:46:47,435::task::495::TaskManager.Task::(_debug) Task becec105-29ae-4c58-a6e0-c751f8514555: ref 0 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:46:47,436::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913546::INFO::2013-11-20 04:46:47,437::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:46:47,437::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913546::INFO::2013-11-20 04:46:47,438::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15916921::DEBUG::2013-11-20 04:46:47,439::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:46:47,440::task::495::TaskManager.Task::(_debug) Task fb748894-cb27-4e57-9964-b6ca9e0a569b: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:46:47,440::task::495::TaskManager.Task::(_debug) Task fdf859a6-7400-48e5-9dcf-d7096a0f29d1: moving from state init -> state preparing
>Thread-15916921::DEBUG::2013-11-20 04:46:47,441::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15914519::INFO::2013-11-20 04:46:47,442::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15916921::DEBUG::2013-11-20 04:46:47,444::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:46:47,445::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`83bdee9a-2cd5-4da3-ac01-cb9663fa1d57`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15916921::DEBUG::2013-11-20 04:46:47,449::task::495::TaskManager.Task::(_debug) Task 3d64c386-766d-449f-88f8-56f6dd725cbe: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:46:47,450::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15916921::INFO::2013-11-20 04:46:47,454::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15913546::DEBUG::2013-11-20 04:46:47,457::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:46:47,459::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`83bdee9a-2cd5-4da3-ac01-cb9663fa1d57`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:46:47,462::task::495::TaskManager.Task::(_debug) Task fdf859a6-7400-48e5-9dcf-d7096a0f29d1: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:46:47,463::task::495::TaskManager.Task::(_debug) Task fdf859a6-7400-48e5-9dcf-d7096a0f29d1: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:46:47,464::task::495::TaskManager.Task::(_debug) Task fdf859a6-7400-48e5-9dcf-d7096a0f29d1: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:46:47,466::task::495::TaskManager.Task::(_debug) Task fdf859a6-7400-48e5-9dcf-d7096a0f29d1: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:46:47,467::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:46:47,468::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:46:47,469::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:46:47,475::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:46:47,477::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:46:47,479::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:46:47,482::task::495::TaskManager.Task::(_debug) Task fdf859a6-7400-48e5-9dcf-d7096a0f29d1: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:46:47,483::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:46:47,485::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:46:47,486::task::495::TaskManager.Task::(_debug) Task 8bfcdb85-603a-4f7a-ac95-773e0c306be3: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:46:47,488::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f9092feb-5bb3-4c7f-a5ee-11b35dabf00f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:46:47,488::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:46:47,489::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:46:47,490::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`f9092feb-5bb3-4c7f-a5ee-11b35dabf00f`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:46:47,491::task::495::TaskManager.Task::(_debug) Task 8bfcdb85-603a-4f7a-ac95-773e0c306be3: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:46:47,492::task::495::TaskManager.Task::(_debug) Task 8bfcdb85-603a-4f7a-ac95-773e0c306be3: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:46:47,493::task::495::TaskManager.Task::(_debug) Task 8bfcdb85-603a-4f7a-ac95-773e0c306be3: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:46:47,494::task::495::TaskManager.Task::(_debug) Task 8bfcdb85-603a-4f7a-ac95-773e0c306be3: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:46:47,494::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:46:47,495::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:46:47,495::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:46:47,496::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:46:47,496::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:46:47,497::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:46:47,497::task::495::TaskManager.Task::(_debug) Task 8bfcdb85-603a-4f7a-ac95-773e0c306be3: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:46:47,498::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:46:47,498::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:46:47,505::task::495::TaskManager.Task::(_debug) Task 96429ccc-f11e-4aa1-8fb2-758a74c272f1: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:46:47,507::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`cdd243b6-2da5-456a-80d3-84dc4d006efb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:46:47,508::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:46:47,508::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:46:47,509::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`cdd243b6-2da5-456a-80d3-84dc4d006efb`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:46:47,510::task::495::TaskManager.Task::(_debug) Task 96429ccc-f11e-4aa1-8fb2-758a74c272f1: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:46:47,511::task::495::TaskManager.Task::(_debug) Task 96429ccc-f11e-4aa1-8fb2-758a74c272f1: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:46:47,511::task::495::TaskManager.Task::(_debug) Task 96429ccc-f11e-4aa1-8fb2-758a74c272f1: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:46:47,512::task::495::TaskManager.Task::(_debug) Task 96429ccc-f11e-4aa1-8fb2-758a74c272f1: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:46:47,513::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:46:47,513::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:46:47,514::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:46:47,516::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:46:47,517::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:46:47,518::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:46:47,519::task::495::TaskManager.Task::(_debug) Task 96429ccc-f11e-4aa1-8fb2-758a74c272f1: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:46:47,520::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:46:47,686::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0936371 s, 10.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:46:49,704::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:46:50,268::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0830709 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:46:52,285::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:46:52,825::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0746898 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15914435::INFO::2013-11-20 04:46:53,408::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:46:53,410::task::495::TaskManager.Task::(_debug) Task 156564df-2a14-40ae-a16c-88a9110ae9e4: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:46:53,412::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`60b537bc-2299-49ff-8c49-d03da67e2b42`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:46:53,412::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:46:53,413::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:46:53,414::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`60b537bc-2299-49ff-8c49-d03da67e2b42`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:46:53,415::task::495::TaskManager.Task::(_debug) Task 156564df-2a14-40ae-a16c-88a9110ae9e4: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:46:53,416::task::495::TaskManager.Task::(_debug) Task 156564df-2a14-40ae-a16c-88a9110ae9e4: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:46:53,416::task::495::TaskManager.Task::(_debug) Task 156564df-2a14-40ae-a16c-88a9110ae9e4: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:46:53,417::task::495::TaskManager.Task::(_debug) Task 156564df-2a14-40ae-a16c-88a9110ae9e4: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:46:53,418::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:46:53,418::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:46:53,419::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:46:53,420::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:46:53,420::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:46:53,421::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:46:53,421::task::495::TaskManager.Task::(_debug) Task 156564df-2a14-40ae-a16c-88a9110ae9e4: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:46:53,422::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:46:54,841::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:46:55,424::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0964466 s, 10.6 MB/s\n'; <rc> = 0
>Thread-15916926::INFO::2013-11-20 04:46:57,336::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916926::DEBUG::2013-11-20 04:46:57,337::task::495::TaskManager.Task::(_debug) Task 99e07a92-6e13-4c2d-a9b1-248c292ba07a: moving from state init -> state preparing
>Thread-15916926::DEBUG::2013-11-20 04:46:57,338::task::495::TaskManager.Task::(_debug) Task 99e07a92-6e13-4c2d-a9b1-248c292ba07a: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000402212142944', 'lastCheck': 1384922808.3257279, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0012800693512', 'lastCheck': 1384922811.3331709, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00158286094666', 'lastCheck': 1384922811.332994, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000503063201904', 'lastCheck': 1384922813.930547, 'valid': True, 'code': 0}}
>Thread-15916926::DEBUG::2013-11-20 04:46:57,353::task::495::TaskManager.Task::(_debug) Task 99e07a92-6e13-4c2d-a9b1-248c292ba07a: moving from state preparing -> state finished
>Thread-15916926::DEBUG::2013-11-20 04:46:57,360::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916926::DEBUG::2013-11-20 04:46:57,361::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916926::DEBUG::2013-11-20 04:46:57,362::task::495::TaskManager.Task::(_debug) Task 99e07a92-6e13-4c2d-a9b1-248c292ba07a: ref 0 aborting False
>Thread-15916926::INFO::2013-11-20 04:46:57,364::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000402212142944', 'lastCheck': 1384922808.3257279, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0012800693512', 'lastCheck': 1384922811.3331709, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00158286094666', 'lastCheck': 1384922811.332994, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000503063201904', 'lastCheck': 1384922813.930547, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:46:57,440::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:46:58,005::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0861883 s, 11.9 MB/s\n'; <rc> = 0
>Thread-15916928::DEBUG::2013-11-20 04:46:58,215::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916928::INFO::2013-11-20 04:46:58,217::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916928::DEBUG::2013-11-20 04:46:58,218::task::495::TaskManager.Task::(_debug) Task 11957088-bb75-431c-9285-6dc742692963: moving from state init -> state preparing
>Thread-15916928::DEBUG::2013-11-20 04:46:58,219::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916928::DEBUG::2013-11-20 04:46:58,219::task::495::TaskManager.Task::(_debug) Task 11957088-bb75-431c-9285-6dc742692963: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916928::DEBUG::2013-11-20 04:46:58,221::task::495::TaskManager.Task::(_debug) Task 11957088-bb75-431c-9285-6dc742692963: moving from state preparing -> state finished
>Thread-15916928::DEBUG::2013-11-20 04:46:58,221::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916928::DEBUG::2013-11-20 04:46:58,222::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916928::DEBUG::2013-11-20 04:46:58,222::task::495::TaskManager.Task::(_debug) Task 11957088-bb75-431c-9285-6dc742692963: ref 0 aborting False
>Thread-15916928::INFO::2013-11-20 04:46:58,223::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916929::DEBUG::2013-11-20 04:46:58,230::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916929::INFO::2013-11-20 04:46:58,232::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916929::DEBUG::2013-11-20 04:46:58,233::task::495::TaskManager.Task::(_debug) Task 44ee47a8-089d-4bf5-a48c-3168e1b6cb0d: moving from state init -> state preparing
>Thread-15916929::DEBUG::2013-11-20 04:46:58,234::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1163705b-8693-4fb3-8f51-c67672890924`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916929::DEBUG::2013-11-20 04:46:58,235::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916929::DEBUG::2013-11-20 04:46:58,236::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916929::DEBUG::2013-11-20 04:46:58,237::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1163705b-8693-4fb3-8f51-c67672890924`::Granted request
>Thread-15916929::DEBUG::2013-11-20 04:46:58,238::task::495::TaskManager.Task::(_debug) Task 44ee47a8-089d-4bf5-a48c-3168e1b6cb0d: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916929::DEBUG::2013-11-20 04:46:58,239::task::495::TaskManager.Task::(_debug) Task 44ee47a8-089d-4bf5-a48c-3168e1b6cb0d: ref 1 aborting False
>Thread-15916929::WARNING::2013-11-20 04:46:58,241::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916929::DEBUG::2013-11-20 04:46:58,251::task::495::TaskManager.Task::(_debug) Task 44ee47a8-089d-4bf5-a48c-3168e1b6cb0d: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916929::DEBUG::2013-11-20 04:46:58,257::task::495::TaskManager.Task::(_debug) Task 44ee47a8-089d-4bf5-a48c-3168e1b6cb0d: moving from state preparing -> state finished
>Thread-15916929::DEBUG::2013-11-20 04:46:58,258::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916929::DEBUG::2013-11-20 04:46:58,259::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916929::DEBUG::2013-11-20 04:46:58,260::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916929::DEBUG::2013-11-20 04:46:58,261::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916929::DEBUG::2013-11-20 04:46:58,261::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916929::DEBUG::2013-11-20 04:46:58,262::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916929::DEBUG::2013-11-20 04:46:58,263::task::495::TaskManager.Task::(_debug) Task 44ee47a8-089d-4bf5-a48c-3168e1b6cb0d: ref 0 aborting False
>Thread-15916929::INFO::2013-11-20 04:46:58,264::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:47:00,021::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:47:00,585::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0854069 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:47:02,600::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:47:03,141::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0730739 s, 14.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:47:05,156::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:47:05,375::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:47:05,383::task::495::TaskManager.Task::(_debug) Task 94df6b45-449d-46db-a325-74a57512e05b: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:47:05,386::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c067442e-a844-4fdc-b4de-1118510ede7a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:47:05,386::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:47:05,387::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:47:05,387::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c067442e-a844-4fdc-b4de-1118510ede7a`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:47:05,388::task::495::TaskManager.Task::(_debug) Task 94df6b45-449d-46db-a325-74a57512e05b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:47:05,388::task::495::TaskManager.Task::(_debug) Task 94df6b45-449d-46db-a325-74a57512e05b: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:47:05,389::task::495::TaskManager.Task::(_debug) Task 94df6b45-449d-46db-a325-74a57512e05b: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:47:05,389::task::495::TaskManager.Task::(_debug) Task 94df6b45-449d-46db-a325-74a57512e05b: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:47:05,390::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:47:05,390::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:47:05,391::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:47:05,391::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:47:05,391::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:47:05,392::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:47:05,392::task::495::TaskManager.Task::(_debug) Task 94df6b45-449d-46db-a325-74a57512e05b: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:47:05,392::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:47:05,642::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0479723 s, 21.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:47:07,653::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915918::INFO::2013-11-20 04:47:07,862::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:47:07,866::task::495::TaskManager.Task::(_debug) Task 5b213e59-820e-47bc-af60-65cfc1ced63c: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:47:07,869::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f7099e19-bd26-4eaf-8256-5eb8e996b8c3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:47:07,870::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:47:07,872::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:47:07,872::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f7099e19-bd26-4eaf-8256-5eb8e996b8c3`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:47:07,875::task::495::TaskManager.Task::(_debug) Task 5b213e59-820e-47bc-af60-65cfc1ced63c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:47:07,876::task::495::TaskManager.Task::(_debug) Task 5b213e59-820e-47bc-af60-65cfc1ced63c: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:47:07,879::task::495::TaskManager.Task::(_debug) Task 5b213e59-820e-47bc-af60-65cfc1ced63c: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:47:07,880::task::495::TaskManager.Task::(_debug) Task 5b213e59-820e-47bc-af60-65cfc1ced63c: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:47:07,882::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:47:07,883::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:47:07,884::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:47:07,886::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:47:07,886::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:47:07,887::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:47:07,889::task::495::TaskManager.Task::(_debug) Task 5b213e59-820e-47bc-af60-65cfc1ced63c: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:47:07,890::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:47:08,207::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0892961 s, 11.5 MB/s\n'; <rc> = 0
>Thread-15916934::INFO::2013-11-20 04:47:09,140::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916934::DEBUG::2013-11-20 04:47:09,141::task::495::TaskManager.Task::(_debug) Task a669f851-186b-4adb-8935-aa70bef82428: moving from state init -> state preparing
>Thread-15916934::DEBUG::2013-11-20 04:47:09,142::task::495::TaskManager.Task::(_debug) Task a669f851-186b-4adb-8935-aa70bef82428: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000396013259888', 'lastCheck': 1384922828.86164, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00162196159363', 'lastCheck': 1384922821.672539, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0016450881958', 'lastCheck': 1384922821.68366, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000476121902466', 'lastCheck': 1384922824.2378919, 'valid': True, 'code': 0}}
>Thread-15916934::DEBUG::2013-11-20 04:47:09,151::task::495::TaskManager.Task::(_debug) Task a669f851-186b-4adb-8935-aa70bef82428: moving from state preparing -> state finished
>Thread-15916934::DEBUG::2013-11-20 04:47:09,155::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916934::DEBUG::2013-11-20 04:47:09,156::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916934::DEBUG::2013-11-20 04:47:09,157::task::495::TaskManager.Task::(_debug) Task a669f851-186b-4adb-8935-aa70bef82428: ref 0 aborting False
>Thread-15916934::INFO::2013-11-20 04:47:09,158::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000396013259888', 'lastCheck': 1384922828.86164, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00162196159363', 'lastCheck': 1384922821.672539, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0016450881958', 'lastCheck': 1384922821.68366, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000476121902466', 'lastCheck': 1384922824.2378919, 'valid': True, 'code': 0}}
>Thread-15916935::DEBUG::2013-11-20 04:47:09,190::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916935::INFO::2013-11-20 04:47:09,191::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916935::DEBUG::2013-11-20 04:47:09,193::task::495::TaskManager.Task::(_debug) Task bea0792b-0bf6-4807-9d44-ff31839e49db: moving from state init -> state preparing
>Thread-15916935::DEBUG::2013-11-20 04:47:09,194::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916935::DEBUG::2013-11-20 04:47:09,194::task::495::TaskManager.Task::(_debug) Task bea0792b-0bf6-4807-9d44-ff31839e49db: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916935::DEBUG::2013-11-20 04:47:09,195::task::495::TaskManager.Task::(_debug) Task bea0792b-0bf6-4807-9d44-ff31839e49db: moving from state preparing -> state finished
>Thread-15916935::DEBUG::2013-11-20 04:47:09,196::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916935::DEBUG::2013-11-20 04:47:09,196::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916935::DEBUG::2013-11-20 04:47:09,197::task::495::TaskManager.Task::(_debug) Task bea0792b-0bf6-4807-9d44-ff31839e49db: ref 0 aborting False
>Thread-15916935::INFO::2013-11-20 04:47:09,197::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916936::DEBUG::2013-11-20 04:47:09,213::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916936::INFO::2013-11-20 04:47:09,215::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916936::DEBUG::2013-11-20 04:47:09,215::task::495::TaskManager.Task::(_debug) Task e53981c6-7dcc-42f1-a13a-25d368cc935b: moving from state init -> state preparing
>Thread-15916936::DEBUG::2013-11-20 04:47:09,217::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`83c6e207-8b1d-4654-9291-d23aaad74d5c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916936::DEBUG::2013-11-20 04:47:09,217::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916936::DEBUG::2013-11-20 04:47:09,218::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916936::DEBUG::2013-11-20 04:47:09,219::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`83c6e207-8b1d-4654-9291-d23aaad74d5c`::Granted request
>Thread-15916936::DEBUG::2013-11-20 04:47:09,220::task::495::TaskManager.Task::(_debug) Task e53981c6-7dcc-42f1-a13a-25d368cc935b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916936::DEBUG::2013-11-20 04:47:09,221::task::495::TaskManager.Task::(_debug) Task e53981c6-7dcc-42f1-a13a-25d368cc935b: ref 1 aborting False
>Thread-15916936::WARNING::2013-11-20 04:47:09,223::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916936::DEBUG::2013-11-20 04:47:09,234::task::495::TaskManager.Task::(_debug) Task e53981c6-7dcc-42f1-a13a-25d368cc935b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916936::DEBUG::2013-11-20 04:47:09,235::task::495::TaskManager.Task::(_debug) Task e53981c6-7dcc-42f1-a13a-25d368cc935b: moving from state preparing -> state finished
>Thread-15916936::DEBUG::2013-11-20 04:47:09,236::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916936::DEBUG::2013-11-20 04:47:09,236::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916936::DEBUG::2013-11-20 04:47:09,237::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916936::DEBUG::2013-11-20 04:47:09,238::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916936::DEBUG::2013-11-20 04:47:09,238::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916936::DEBUG::2013-11-20 04:47:09,239::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916936::DEBUG::2013-11-20 04:47:09,240::task::495::TaskManager.Task::(_debug) Task e53981c6-7dcc-42f1-a13a-25d368cc935b: ref 0 aborting False
>Thread-15916936::INFO::2013-11-20 04:47:09,241::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:47:10,222::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:47:10,778::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0783061 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:47:12,794::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915863::INFO::2013-11-20 04:47:13,009::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:47:13,012::task::495::TaskManager.Task::(_debug) Task cb4a3b67-1a58-43cc-bdf8-805ffc52a6f8: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:47:13,016::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`80a3d90e-b0a9-40bb-8baf-8c45ef2a1b56`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:47:13,018::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:47:13,019::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:47:13,020::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`80a3d90e-b0a9-40bb-8baf-8c45ef2a1b56`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:47:13,022::task::495::TaskManager.Task::(_debug) Task cb4a3b67-1a58-43cc-bdf8-805ffc52a6f8: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:47:13,023::task::495::TaskManager.Task::(_debug) Task cb4a3b67-1a58-43cc-bdf8-805ffc52a6f8: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:47:13,024::task::495::TaskManager.Task::(_debug) Task cb4a3b67-1a58-43cc-bdf8-805ffc52a6f8: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:47:13,025::task::495::TaskManager.Task::(_debug) Task cb4a3b67-1a58-43cc-bdf8-805ffc52a6f8: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:47:13,026::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:47:13,026::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:47:13,027::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:47:13,028::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:47:13,028::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:47:13,029::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:47:13,030::task::495::TaskManager.Task::(_debug) Task cb4a3b67-1a58-43cc-bdf8-805ffc52a6f8: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:47:13,030::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:47:13,334::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0778711 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:47:15,349::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:47:15,912::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0775244 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:47:17,934::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 04:47:18,132::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:47:18,138::task::495::TaskManager.Task::(_debug) Task 3fbb4c2f-c2df-4bef-b94d-361d4108c879: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:47:18,142::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`56457915-fc3b-41f7-bc48-80fea022ad4d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:47:18,145::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:47:18,146::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:47:18,147::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`56457915-fc3b-41f7-bc48-80fea022ad4d`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:47:18,149::task::495::TaskManager.Task::(_debug) Task 3fbb4c2f-c2df-4bef-b94d-361d4108c879: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:47:18,149::task::495::TaskManager.Task::(_debug) Task 3fbb4c2f-c2df-4bef-b94d-361d4108c879: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:47:18,150::task::495::TaskManager.Task::(_debug) Task 3fbb4c2f-c2df-4bef-b94d-361d4108c879: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:47:18,150::task::495::TaskManager.Task::(_debug) Task 3fbb4c2f-c2df-4bef-b94d-361d4108c879: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:47:18,151::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:47:18,151::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:47:18,152::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:47:18,153::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:47:18,153::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:47:18,153::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:47:18,154::task::495::TaskManager.Task::(_debug) Task 3fbb4c2f-c2df-4bef-b94d-361d4108c879: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:47:18,155::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:47:18,461::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0813175 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:47:20,477::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916942::DEBUG::2013-11-20 04:47:20,714::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916942::INFO::2013-11-20 04:47:20,717::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916942::DEBUG::2013-11-20 04:47:20,719::task::495::TaskManager.Task::(_debug) Task 5960adbc-43a2-406d-b77a-b193709ad937: moving from state init -> state preparing
>Thread-15916942::DEBUG::2013-11-20 04:47:20,720::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916942::DEBUG::2013-11-20 04:47:20,721::task::495::TaskManager.Task::(_debug) Task 5960adbc-43a2-406d-b77a-b193709ad937: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916942::DEBUG::2013-11-20 04:47:20,722::task::495::TaskManager.Task::(_debug) Task 5960adbc-43a2-406d-b77a-b193709ad937: moving from state preparing -> state finished
>Thread-15916942::DEBUG::2013-11-20 04:47:20,723::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916942::DEBUG::2013-11-20 04:47:20,723::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916942::DEBUG::2013-11-20 04:47:20,724::task::495::TaskManager.Task::(_debug) Task 5960adbc-43a2-406d-b77a-b193709ad937: ref 0 aborting False
>Thread-15916942::INFO::2013-11-20 04:47:20,725::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916943::DEBUG::2013-11-20 04:47:20,740::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916943::INFO::2013-11-20 04:47:20,742::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916943::DEBUG::2013-11-20 04:47:20,742::task::495::TaskManager.Task::(_debug) Task 4c8f1df9-375e-4214-a905-053fc2a9fcaf: moving from state init -> state preparing
>Thread-15916943::DEBUG::2013-11-20 04:47:20,744::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`eaa7d0d5-97c5-41fe-94e5-4c164418f0d8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916943::DEBUG::2013-11-20 04:47:20,745::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916943::DEBUG::2013-11-20 04:47:20,746::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916943::DEBUG::2013-11-20 04:47:20,747::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`eaa7d0d5-97c5-41fe-94e5-4c164418f0d8`::Granted request
>Thread-15916943::DEBUG::2013-11-20 04:47:20,748::task::495::TaskManager.Task::(_debug) Task 4c8f1df9-375e-4214-a905-053fc2a9fcaf: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916943::DEBUG::2013-11-20 04:47:20,749::task::495::TaskManager.Task::(_debug) Task 4c8f1df9-375e-4214-a905-053fc2a9fcaf: ref 1 aborting False
>Thread-15916943::WARNING::2013-11-20 04:47:20,752::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916943::DEBUG::2013-11-20 04:47:20,757::task::495::TaskManager.Task::(_debug) Task 4c8f1df9-375e-4214-a905-053fc2a9fcaf: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916943::DEBUG::2013-11-20 04:47:20,758::task::495::TaskManager.Task::(_debug) Task 4c8f1df9-375e-4214-a905-053fc2a9fcaf: moving from state preparing -> state finished
>Thread-15916943::DEBUG::2013-11-20 04:47:20,759::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916943::DEBUG::2013-11-20 04:47:20,760::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916943::DEBUG::2013-11-20 04:47:20,761::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916943::DEBUG::2013-11-20 04:47:20,762::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916943::DEBUG::2013-11-20 04:47:20,762::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916943::DEBUG::2013-11-20 04:47:20,767::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916943::DEBUG::2013-11-20 04:47:20,768::task::495::TaskManager.Task::(_debug) Task 4c8f1df9-375e-4214-a905-053fc2a9fcaf: ref 0 aborting False
>Thread-15916943::INFO::2013-11-20 04:47:20,769::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916944::INFO::2013-11-20 04:47:20,861::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916944::DEBUG::2013-11-20 04:47:20,862::task::495::TaskManager.Task::(_debug) Task 60fc0f38-1c42-487c-b349-2a88bd263d32: moving from state init -> state preparing
>Thread-15916944::DEBUG::2013-11-20 04:47:20,863::task::495::TaskManager.Task::(_debug) Task 60fc0f38-1c42-487c-b349-2a88bd263d32: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000458955764771', 'lastCheck': 1384922839.127516, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014328956604', 'lastCheck': 1384922831.8694301, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00151610374451', 'lastCheck': 1384922831.8687911, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000416994094849', 'lastCheck': 1384922834.4362669, 'valid': True, 'code': 0}}
>Thread-15916944::DEBUG::2013-11-20 04:47:20,876::task::495::TaskManager.Task::(_debug) Task 60fc0f38-1c42-487c-b349-2a88bd263d32: moving from state preparing -> state finished
>Thread-15916944::DEBUG::2013-11-20 04:47:20,879::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916944::DEBUG::2013-11-20 04:47:20,879::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916944::DEBUG::2013-11-20 04:47:20,880::task::495::TaskManager.Task::(_debug) Task 60fc0f38-1c42-487c-b349-2a88bd263d32: ref 0 aborting False
>Thread-15916944::INFO::2013-11-20 04:47:20,881::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000458955764771', 'lastCheck': 1384922839.127516, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014328956604', 'lastCheck': 1384922831.8694301, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00151610374451', 'lastCheck': 1384922831.8687911, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000416994094849', 'lastCheck': 1384922834.4362669, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:47:21,057::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.11429 s, 9.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:47:23,066::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:47:23,590::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0833061 s, 12.3 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 04:47:25,309::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:47:25,310::task::495::TaskManager.Task::(_debug) Task b773581e-9d10-4f40-a8de-e849033bcaca: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:47:25,311::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`436522cf-875c-4e2b-a62c-f4562fa741e7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:47:25,312::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:47:25,313::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:47:25,314::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`436522cf-875c-4e2b-a62c-f4562fa741e7`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:47:25,315::task::495::TaskManager.Task::(_debug) Task b773581e-9d10-4f40-a8de-e849033bcaca: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:47:25,316::task::495::TaskManager.Task::(_debug) Task b773581e-9d10-4f40-a8de-e849033bcaca: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:47:25,317::task::495::TaskManager.Task::(_debug) Task b773581e-9d10-4f40-a8de-e849033bcaca: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:47:25,318::task::495::TaskManager.Task::(_debug) Task b773581e-9d10-4f40-a8de-e849033bcaca: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:47:25,318::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:47:25,319::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:47:25,320::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:47:25,320::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:47:25,321::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:47:25,322::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:47:25,322::task::495::TaskManager.Task::(_debug) Task b773581e-9d10-4f40-a8de-e849033bcaca: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:47:25,323::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:47:25,324::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:47:25,324::task::495::TaskManager.Task::(_debug) Task 5d820a3b-9619-4095-81c6-dff630caa305: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:47:25,326::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e0274d59-3699-456e-b234-4d52298c432b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:47:25,327::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:47:25,327::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:47:25,328::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e0274d59-3699-456e-b234-4d52298c432b`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:47:25,329::task::495::TaskManager.Task::(_debug) Task 5d820a3b-9619-4095-81c6-dff630caa305: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:47:25,330::task::495::TaskManager.Task::(_debug) Task 5d820a3b-9619-4095-81c6-dff630caa305: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:47:25,330::task::495::TaskManager.Task::(_debug) Task 5d820a3b-9619-4095-81c6-dff630caa305: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:47:25,331::task::495::TaskManager.Task::(_debug) Task 5d820a3b-9619-4095-81c6-dff630caa305: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:47:25,331::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:47:25,332::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:47:25,333::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:47:25,334::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:47:25,334::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:47:25,335::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:47:25,336::task::495::TaskManager.Task::(_debug) Task 5d820a3b-9619-4095-81c6-dff630caa305: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:47:25,336::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:47:25,607::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:47:26,191::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0978762 s, 10.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:47:28,210::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 04:47:28,428::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:47:28,431::task::495::TaskManager.Task::(_debug) Task 50698551-d0c8-4ec2-b558-40bd3b806152: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:47:28,436::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`61db945a-d9a4-4ebb-ad14-8760b12c8c9f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:47:28,437::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:47:28,440::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:47:28,443::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`61db945a-d9a4-4ebb-ad14-8760b12c8c9f`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:47:28,446::task::495::TaskManager.Task::(_debug) Task 50698551-d0c8-4ec2-b558-40bd3b806152: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:47:28,447::task::495::TaskManager.Task::(_debug) Task 50698551-d0c8-4ec2-b558-40bd3b806152: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:47:28,449::task::495::TaskManager.Task::(_debug) Task 50698551-d0c8-4ec2-b558-40bd3b806152: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:47:28,450::task::495::TaskManager.Task::(_debug) Task 50698551-d0c8-4ec2-b558-40bd3b806152: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:47:28,451::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:47:28,452::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:47:28,453::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:47:28,454::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:47:28,455::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:47:28,456::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:47:28,457::task::495::TaskManager.Task::(_debug) Task 50698551-d0c8-4ec2-b558-40bd3b806152: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:47:28,458::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:47:28,751::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0772976 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:47:30,767::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:47:31,199::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0465799 s, 22.0 MB/s\n'; <rc> = 0
>Thread-15916950::DEBUG::2013-11-20 04:47:31,597::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916950::INFO::2013-11-20 04:47:31,598::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916950::DEBUG::2013-11-20 04:47:31,598::task::495::TaskManager.Task::(_debug) Task 98062ede-257f-473e-9b60-bee45c6d79fe: moving from state init -> state preparing
>Thread-15916950::DEBUG::2013-11-20 04:47:31,599::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916950::DEBUG::2013-11-20 04:47:31,599::task::495::TaskManager.Task::(_debug) Task 98062ede-257f-473e-9b60-bee45c6d79fe: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916950::DEBUG::2013-11-20 04:47:31,600::task::495::TaskManager.Task::(_debug) Task 98062ede-257f-473e-9b60-bee45c6d79fe: moving from state preparing -> state finished
>Thread-15916950::DEBUG::2013-11-20 04:47:31,600::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916950::DEBUG::2013-11-20 04:47:31,600::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916950::DEBUG::2013-11-20 04:47:31,601::task::495::TaskManager.Task::(_debug) Task 98062ede-257f-473e-9b60-bee45c6d79fe: ref 0 aborting False
>Thread-15916950::INFO::2013-11-20 04:47:31,601::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916951::DEBUG::2013-11-20 04:47:31,613::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916951::INFO::2013-11-20 04:47:31,614::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916951::DEBUG::2013-11-20 04:47:31,615::task::495::TaskManager.Task::(_debug) Task 66851302-f9db-43dc-8cee-d1f5fc0f2886: moving from state init -> state preparing
>Thread-15916951::DEBUG::2013-11-20 04:47:31,616::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`46273746-e539-417b-871a-731bba685e70`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916951::DEBUG::2013-11-20 04:47:31,616::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916951::DEBUG::2013-11-20 04:47:31,617::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916951::DEBUG::2013-11-20 04:47:31,617::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`46273746-e539-417b-871a-731bba685e70`::Granted request
>Thread-15916951::DEBUG::2013-11-20 04:47:31,618::task::495::TaskManager.Task::(_debug) Task 66851302-f9db-43dc-8cee-d1f5fc0f2886: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916951::DEBUG::2013-11-20 04:47:31,618::task::495::TaskManager.Task::(_debug) Task 66851302-f9db-43dc-8cee-d1f5fc0f2886: ref 1 aborting False
>Thread-15916951::WARNING::2013-11-20 04:47:31,620::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916951::DEBUG::2013-11-20 04:47:31,626::task::495::TaskManager.Task::(_debug) Task 66851302-f9db-43dc-8cee-d1f5fc0f2886: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916951::DEBUG::2013-11-20 04:47:31,626::task::495::TaskManager.Task::(_debug) Task 66851302-f9db-43dc-8cee-d1f5fc0f2886: moving from state preparing -> state finished
>Thread-15916951::DEBUG::2013-11-20 04:47:31,627::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916951::DEBUG::2013-11-20 04:47:31,627::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916951::DEBUG::2013-11-20 04:47:31,628::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916951::DEBUG::2013-11-20 04:47:31,628::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916951::DEBUG::2013-11-20 04:47:31,628::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916951::DEBUG::2013-11-20 04:47:31,629::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916951::DEBUG::2013-11-20 04:47:31,629::task::495::TaskManager.Task::(_debug) Task 66851302-f9db-43dc-8cee-d1f5fc0f2886: ref 0 aborting False
>Thread-15916951::INFO::2013-11-20 04:47:31,634::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916952::INFO::2013-11-20 04:47:31,869::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916952::DEBUG::2013-11-20 04:47:31,869::task::495::TaskManager.Task::(_debug) Task 74402acb-0477-4380-8451-0228cbfed0c1: moving from state init -> state preparing
>Thread-15916952::DEBUG::2013-11-20 04:47:31,870::task::495::TaskManager.Task::(_debug) Task 74402acb-0477-4380-8451-0228cbfed0c1: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000379085540771', 'lastCheck': 1384922849.4228301, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0011739730835', 'lastCheck': 1384922842.1371269, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00165605545044', 'lastCheck': 1384922842.1363549, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000524997711182', 'lastCheck': 1384922844.700346, 'valid': True, 'code': 0}}
>Thread-15916952::DEBUG::2013-11-20 04:47:31,870::task::495::TaskManager.Task::(_debug) Task 74402acb-0477-4380-8451-0228cbfed0c1: moving from state preparing -> state finished
>Thread-15916952::DEBUG::2013-11-20 04:47:31,871::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916952::DEBUG::2013-11-20 04:47:31,871::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916952::DEBUG::2013-11-20 04:47:31,872::task::495::TaskManager.Task::(_debug) Task 74402acb-0477-4380-8451-0228cbfed0c1: ref 0 aborting False
>Thread-15916952::INFO::2013-11-20 04:47:31,872::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000379085540771', 'lastCheck': 1384922849.4228301, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0011739730835', 'lastCheck': 1384922842.1371269, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00165605545044', 'lastCheck': 1384922842.1363549, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000524997711182', 'lastCheck': 1384922844.700346, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:47:33,210::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:47:33,797::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.090904 s, 11.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:47:35,811::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 04:47:36,017::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:47:36,018::task::495::TaskManager.Task::(_debug) Task a4aab61a-e0a7-4cff-8380-0ac45120009d: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:47:36,021::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6315c49b-6fac-4897-97da-e75e57aeb98c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:47:36,021::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:47:36,022::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:47:36,023::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6315c49b-6fac-4897-97da-e75e57aeb98c`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:47:36,024::task::495::TaskManager.Task::(_debug) Task a4aab61a-e0a7-4cff-8380-0ac45120009d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:47:36,025::task::495::TaskManager.Task::(_debug) Task a4aab61a-e0a7-4cff-8380-0ac45120009d: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:47:36,026::task::495::TaskManager.Task::(_debug) Task a4aab61a-e0a7-4cff-8380-0ac45120009d: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:47:36,026::task::495::TaskManager.Task::(_debug) Task a4aab61a-e0a7-4cff-8380-0ac45120009d: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:47:36,027::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:47:36,028::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:47:36,028::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:47:36,029::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:47:36,030::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:47:36,030::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:47:36,031::task::495::TaskManager.Task::(_debug) Task a4aab61a-e0a7-4cff-8380-0ac45120009d: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:47:36,032::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:47:36,339::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0828873 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:47:38,354::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 04:47:38,637::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:47:38,639::task::495::TaskManager.Task::(_debug) Task db069308-f856-419d-913f-68a7e91735fe: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:47:38,643::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b2d0f420-516c-446b-9cbb-58ae3b0a6f94`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:47:38,644::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:47:38,646::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:47:38,647::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b2d0f420-516c-446b-9cbb-58ae3b0a6f94`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:47:38,649::task::495::TaskManager.Task::(_debug) Task db069308-f856-419d-913f-68a7e91735fe: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:47:38,650::task::495::TaskManager.Task::(_debug) Task db069308-f856-419d-913f-68a7e91735fe: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:47:38,651::task::495::TaskManager.Task::(_debug) Task db069308-f856-419d-913f-68a7e91735fe: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:47:38,652::task::495::TaskManager.Task::(_debug) Task db069308-f856-419d-913f-68a7e91735fe: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:47:38,653::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:47:38,654::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:47:38,655::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:47:38,656::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:47:38,657::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:47:38,658::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:47:38,659::task::495::TaskManager.Task::(_debug) Task db069308-f856-419d-913f-68a7e91735fe: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:47:38,660::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:47:38,933::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0811076 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:47:40,944::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:47:41,471::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0797673 s, 12.8 MB/s\n'; <rc> = 0
>Thread-15916958::DEBUG::2013-11-20 04:47:42,016::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916958::INFO::2013-11-20 04:47:42,018::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916958::DEBUG::2013-11-20 04:47:42,018::task::495::TaskManager.Task::(_debug) Task ab6f1351-413b-405b-979e-2c9564a2776b: moving from state init -> state preparing
>Thread-15916958::DEBUG::2013-11-20 04:47:42,019::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916958::DEBUG::2013-11-20 04:47:42,020::task::495::TaskManager.Task::(_debug) Task ab6f1351-413b-405b-979e-2c9564a2776b: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916958::DEBUG::2013-11-20 04:47:42,021::task::495::TaskManager.Task::(_debug) Task ab6f1351-413b-405b-979e-2c9564a2776b: moving from state preparing -> state finished
>Thread-15916958::DEBUG::2013-11-20 04:47:42,021::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916958::DEBUG::2013-11-20 04:47:42,022::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916958::DEBUG::2013-11-20 04:47:42,022::task::495::TaskManager.Task::(_debug) Task ab6f1351-413b-405b-979e-2c9564a2776b: ref 0 aborting False
>Thread-15916958::INFO::2013-11-20 04:47:42,023::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916959::DEBUG::2013-11-20 04:47:42,032::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916959::INFO::2013-11-20 04:47:42,033::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916959::DEBUG::2013-11-20 04:47:42,033::task::495::TaskManager.Task::(_debug) Task 358ab045-bd52-4e4e-8ce9-ec46bc511849: moving from state init -> state preparing
>Thread-15916959::DEBUG::2013-11-20 04:47:42,043::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1d27a382-bdb4-4684-a95f-861f323c3831`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916959::DEBUG::2013-11-20 04:47:42,043::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916959::DEBUG::2013-11-20 04:47:42,044::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916959::DEBUG::2013-11-20 04:47:42,045::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`1d27a382-bdb4-4684-a95f-861f323c3831`::Granted request
>Thread-15916959::DEBUG::2013-11-20 04:47:42,046::task::495::TaskManager.Task::(_debug) Task 358ab045-bd52-4e4e-8ce9-ec46bc511849: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916959::DEBUG::2013-11-20 04:47:42,047::task::495::TaskManager.Task::(_debug) Task 358ab045-bd52-4e4e-8ce9-ec46bc511849: ref 1 aborting False
>Thread-15916959::WARNING::2013-11-20 04:47:42,050::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916959::DEBUG::2013-11-20 04:47:42,051::task::495::TaskManager.Task::(_debug) Task 358ab045-bd52-4e4e-8ce9-ec46bc511849: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916959::DEBUG::2013-11-20 04:47:42,052::task::495::TaskManager.Task::(_debug) Task 358ab045-bd52-4e4e-8ce9-ec46bc511849: moving from state preparing -> state finished
>Thread-15916959::DEBUG::2013-11-20 04:47:42,053::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916959::DEBUG::2013-11-20 04:47:42,054::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916959::DEBUG::2013-11-20 04:47:42,054::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916959::DEBUG::2013-11-20 04:47:42,055::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916959::DEBUG::2013-11-20 04:47:42,056::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916959::DEBUG::2013-11-20 04:47:42,056::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916959::DEBUG::2013-11-20 04:47:42,057::task::495::TaskManager.Task::(_debug) Task 358ab045-bd52-4e4e-8ce9-ec46bc511849: ref 0 aborting False
>Thread-15916959::INFO::2013-11-20 04:47:42,058::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916960::INFO::2013-11-20 04:47:42,252::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916960::DEBUG::2013-11-20 04:47:42,253::task::495::TaskManager.Task::(_debug) Task d1481487-faad-49db-bfe5-86bb3aa791bf: moving from state init -> state preparing
>Thread-15916960::DEBUG::2013-11-20 04:47:42,254::task::495::TaskManager.Task::(_debug) Task d1481487-faad-49db-bfe5-86bb3aa791bf: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000340938568115', 'lastCheck': 1384922859.621747, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0016028881073', 'lastCheck': 1384922852.434449, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00164389610291', 'lastCheck': 1384922852.426234, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000468015670776', 'lastCheck': 1384922854.944052, 'valid': True, 'code': 0}}
>Thread-15916960::DEBUG::2013-11-20 04:47:42,267::task::495::TaskManager.Task::(_debug) Task d1481487-faad-49db-bfe5-86bb3aa791bf: moving from state preparing -> state finished
>Thread-15916960::DEBUG::2013-11-20 04:47:42,268::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916960::DEBUG::2013-11-20 04:47:42,269::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916960::DEBUG::2013-11-20 04:47:42,270::task::495::TaskManager.Task::(_debug) Task d1481487-faad-49db-bfe5-86bb3aa791bf: ref 0 aborting False
>Thread-15916960::INFO::2013-11-20 04:47:42,276::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000340938568115', 'lastCheck': 1384922859.621747, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0016028881073', 'lastCheck': 1384922852.434449, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00164389610291', 'lastCheck': 1384922852.426234, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000468015670776', 'lastCheck': 1384922854.944052, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:47:43,487::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:47:44,057::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0837895 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:47:46,070::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:47:46,611::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0736428 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:47:48,628::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 04:47:48,859::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15914519::INFO::2013-11-20 04:47:48,862::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15913546::DEBUG::2013-11-20 04:47:48,863::task::495::TaskManager.Task::(_debug) Task f7e4a2e4-6de7-4a5f-aa2c-22b8bc7dd9f6: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:47:48,865::task::495::TaskManager.Task::(_debug) Task ab5ded07-9ef5-45fd-8181-877b1312ff37: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:47:48,884::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`14d4987c-af92-4064-a9ab-e85f3d3315ce`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:47:48,885::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b5913a63-0301-4103-a416-2f3b221db890`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:47:48,887::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:47:48,888::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:47:48,890::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:47:48,891::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`14d4987c-af92-4064-a9ab-e85f3d3315ce`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:47:48,893::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:47:48,893::task::495::TaskManager.Task::(_debug) Task ab5ded07-9ef5-45fd-8181-877b1312ff37: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913546::DEBUG::2013-11-20 04:47:48,894::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`b5913a63-0301-4103-a416-2f3b221db890`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:47:48,895::task::495::TaskManager.Task::(_debug) Task ab5ded07-9ef5-45fd-8181-877b1312ff37: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:47:48,898::task::495::TaskManager.Task::(_debug) Task f7e4a2e4-6de7-4a5f-aa2c-22b8bc7dd9f6: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15914519::DEBUG::2013-11-20 04:47:48,899::task::495::TaskManager.Task::(_debug) Task ab5ded07-9ef5-45fd-8181-877b1312ff37: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:47:48,899::task::495::TaskManager.Task::(_debug) Task f7e4a2e4-6de7-4a5f-aa2c-22b8bc7dd9f6: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:47:48,900::task::495::TaskManager.Task::(_debug) Task ab5ded07-9ef5-45fd-8181-877b1312ff37: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:47:48,901::task::495::TaskManager.Task::(_debug) Task f7e4a2e4-6de7-4a5f-aa2c-22b8bc7dd9f6: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:47:48,902::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:47:48,903::task::495::TaskManager.Task::(_debug) Task f7e4a2e4-6de7-4a5f-aa2c-22b8bc7dd9f6: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:47:48,904::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:47:48,904::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:47:48,905::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913546::DEBUG::2013-11-20 04:47:48,906::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:47:48,908::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:47:48,909::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15914519::DEBUG::2013-11-20 04:47:48,909::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:47:48,910::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:47:48,911::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:47:48,912::task::495::TaskManager.Task::(_debug) Task ab5ded07-9ef5-45fd-8181-877b1312ff37: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:47:48,913::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15914519::INFO::2013-11-20 04:47:48,914::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:47:48,915::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:47:48,916::task::495::TaskManager.Task::(_debug) Task f7e4a2e4-6de7-4a5f-aa2c-22b8bc7dd9f6: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:47:48,916::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:47:48,917::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:47:48,918::task::495::TaskManager.Task::(_debug) Task 28a5a36e-e9da-470a-ace8-f330b7f0209a: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:47:48,920::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`cc6f423a-5945-4a81-b6a2-57e5c2215c06`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:47:48,921::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:47:48,921::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:47:48,922::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`cc6f423a-5945-4a81-b6a2-57e5c2215c06`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:47:48,923::task::495::TaskManager.Task::(_debug) Task 28a5a36e-e9da-470a-ace8-f330b7f0209a: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:47:48,924::task::495::TaskManager.Task::(_debug) Task 28a5a36e-e9da-470a-ace8-f330b7f0209a: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:47:48,925::task::495::TaskManager.Task::(_debug) Task 28a5a36e-e9da-470a-ace8-f330b7f0209a: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:47:48,926::task::495::TaskManager.Task::(_debug) Task 28a5a36e-e9da-470a-ace8-f330b7f0209a: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:47:48,927::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:47:48,928::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:47:48,929::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:47:48,929::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:47:48,930::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:47:48,931::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:47:48,932::task::495::TaskManager.Task::(_debug) Task 28a5a36e-e9da-470a-ace8-f330b7f0209a: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:47:48,932::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:47:48,934::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:47:48,934::task::495::TaskManager.Task::(_debug) Task 7d1fdd4e-c10e-4fe6-8b22-26ad673d1f15: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:47:48,935::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c4abdee9-fd31-4c32-b456-45b0b82a8ea6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:47:48,936::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:47:48,941::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:47:48,942::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c4abdee9-fd31-4c32-b456-45b0b82a8ea6`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:47:48,943::task::495::TaskManager.Task::(_debug) Task 7d1fdd4e-c10e-4fe6-8b22-26ad673d1f15: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:47:48,944::task::495::TaskManager.Task::(_debug) Task 7d1fdd4e-c10e-4fe6-8b22-26ad673d1f15: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:47:48,945::task::495::TaskManager.Task::(_debug) Task 7d1fdd4e-c10e-4fe6-8b22-26ad673d1f15: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:47:48,945::task::495::TaskManager.Task::(_debug) Task 7d1fdd4e-c10e-4fe6-8b22-26ad673d1f15: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:47:48,946::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:47:48,946::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:47:48,947::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:47:48,948::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:47:48,948::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:47:48,949::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:47:48,949::task::495::TaskManager.Task::(_debug) Task 7d1fdd4e-c10e-4fe6-8b22-26ad673d1f15: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:47:48,950::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:47:48,951::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:47:48,951::task::495::TaskManager.Task::(_debug) Task 5e4534bf-4932-49db-b8c6-bce86ef8f34e: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:47:48,952::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e9710c09-9aba-4e52-b4d8-05d1eccf41a7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:47:48,953::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:47:48,953::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:47:48,954::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e9710c09-9aba-4e52-b4d8-05d1eccf41a7`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:47:48,955::task::495::TaskManager.Task::(_debug) Task 5e4534bf-4932-49db-b8c6-bce86ef8f34e: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:47:48,956::task::495::TaskManager.Task::(_debug) Task 5e4534bf-4932-49db-b8c6-bce86ef8f34e: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:47:48,956::task::495::TaskManager.Task::(_debug) Task 5e4534bf-4932-49db-b8c6-bce86ef8f34e: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:47:48,957::task::495::TaskManager.Task::(_debug) Task 5e4534bf-4932-49db-b8c6-bce86ef8f34e: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:47:48,957::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:47:48,958::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:47:48,958::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:47:48,959::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:47:48,960::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:47:48,960::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:47:48,961::task::495::TaskManager.Task::(_debug) Task 5e4534bf-4932-49db-b8c6-bce86ef8f34e: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:47:48,961::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:47:49,176::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0680136 s, 15.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:47:51,191::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:47:51,751::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0772798 s, 13.3 MB/s\n'; <rc> = 0
>Thread-15916966::DEBUG::2013-11-20 04:47:52,704::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916966::INFO::2013-11-20 04:47:52,708::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916966::DEBUG::2013-11-20 04:47:52,709::task::495::TaskManager.Task::(_debug) Task 63c33dc8-fa0e-43dc-85b0-59fb676e242e: moving from state init -> state preparing
>Thread-15916966::DEBUG::2013-11-20 04:47:52,711::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916966::DEBUG::2013-11-20 04:47:52,711::task::495::TaskManager.Task::(_debug) Task 63c33dc8-fa0e-43dc-85b0-59fb676e242e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916966::DEBUG::2013-11-20 04:47:52,713::task::495::TaskManager.Task::(_debug) Task 63c33dc8-fa0e-43dc-85b0-59fb676e242e: moving from state preparing -> state finished
>Thread-15916966::DEBUG::2013-11-20 04:47:52,714::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916966::DEBUG::2013-11-20 04:47:52,714::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916966::DEBUG::2013-11-20 04:47:52,715::task::495::TaskManager.Task::(_debug) Task 63c33dc8-fa0e-43dc-85b0-59fb676e242e: ref 0 aborting False
>Thread-15916966::INFO::2013-11-20 04:47:52,716::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916967::DEBUG::2013-11-20 04:47:52,732::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916967::INFO::2013-11-20 04:47:52,734::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916967::DEBUG::2013-11-20 04:47:52,734::task::495::TaskManager.Task::(_debug) Task 83264bde-8368-4a04-bf41-4230e48fb33e: moving from state init -> state preparing
>Thread-15916967::DEBUG::2013-11-20 04:47:52,736::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2bd2849a-4684-4dc2-9df3-2cb983db6be2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916967::DEBUG::2013-11-20 04:47:52,737::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916967::DEBUG::2013-11-20 04:47:52,738::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916967::DEBUG::2013-11-20 04:47:52,739::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2bd2849a-4684-4dc2-9df3-2cb983db6be2`::Granted request
>Thread-15916967::DEBUG::2013-11-20 04:47:52,740::task::495::TaskManager.Task::(_debug) Task 83264bde-8368-4a04-bf41-4230e48fb33e: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916967::DEBUG::2013-11-20 04:47:52,741::task::495::TaskManager.Task::(_debug) Task 83264bde-8368-4a04-bf41-4230e48fb33e: ref 1 aborting False
>Thread-15916967::WARNING::2013-11-20 04:47:52,744::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916967::DEBUG::2013-11-20 04:47:52,748::task::495::TaskManager.Task::(_debug) Task 83264bde-8368-4a04-bf41-4230e48fb33e: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916967::DEBUG::2013-11-20 04:47:52,749::task::495::TaskManager.Task::(_debug) Task 83264bde-8368-4a04-bf41-4230e48fb33e: moving from state preparing -> state finished
>Thread-15916967::DEBUG::2013-11-20 04:47:52,750::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916967::DEBUG::2013-11-20 04:47:52,751::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916967::DEBUG::2013-11-20 04:47:52,751::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916967::DEBUG::2013-11-20 04:47:52,752::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916967::DEBUG::2013-11-20 04:47:52,753::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916967::DEBUG::2013-11-20 04:47:52,753::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916967::DEBUG::2013-11-20 04:47:52,754::task::495::TaskManager.Task::(_debug) Task 83264bde-8368-4a04-bf41-4230e48fb33e: ref 0 aborting False
>Thread-15916967::INFO::2013-11-20 04:47:52,755::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916968::INFO::2013-11-20 04:47:52,907::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916968::DEBUG::2013-11-20 04:47:52,908::task::495::TaskManager.Task::(_debug) Task 1ba22869-fe0d-41fd-b754-fa0cc1a4571b: moving from state init -> state preparing
>Thread-15916968::DEBUG::2013-11-20 04:47:52,909::task::495::TaskManager.Task::(_debug) Task 1ba22869-fe0d-41fd-b754-fa0cc1a4571b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000359058380127', 'lastCheck': 1384922869.8566871, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00144910812378', 'lastCheck': 1384922872.8670361, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00746703147888', 'lastCheck': 1384922872.8529789, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00058388710022', 'lastCheck': 1384922865.164969, 'valid': True, 'code': 0}}
>Thread-15916968::DEBUG::2013-11-20 04:47:52,913::task::495::TaskManager.Task::(_debug) Task 1ba22869-fe0d-41fd-b754-fa0cc1a4571b: moving from state preparing -> state finished
>Thread-15916968::DEBUG::2013-11-20 04:47:52,913::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916968::DEBUG::2013-11-20 04:47:52,914::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916968::DEBUG::2013-11-20 04:47:52,915::task::495::TaskManager.Task::(_debug) Task 1ba22869-fe0d-41fd-b754-fa0cc1a4571b: ref 0 aborting False
>Thread-15916968::INFO::2013-11-20 04:47:52,916::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000359058380127', 'lastCheck': 1384922869.8566871, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00144910812378', 'lastCheck': 1384922872.8670361, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00746703147888', 'lastCheck': 1384922872.8529789, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00058388710022', 'lastCheck': 1384922865.164969, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:47:53,767::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:47:54,323::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0743556 s, 13.8 MB/s\n'; <rc> = 0
>Thread-15914435::INFO::2013-11-20 04:47:54,348::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:47:54,348::task::495::TaskManager.Task::(_debug) Task 258979ad-d9ef-4054-8f47-cad2e4e66ea9: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:47:54,350::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`60fb55ec-6dd0-466e-946b-e8d0724b7fbf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:47:54,351::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:47:54,352::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:47:54,352::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`60fb55ec-6dd0-466e-946b-e8d0724b7fbf`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:47:54,353::task::495::TaskManager.Task::(_debug) Task 258979ad-d9ef-4054-8f47-cad2e4e66ea9: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:47:54,354::task::495::TaskManager.Task::(_debug) Task 258979ad-d9ef-4054-8f47-cad2e4e66ea9: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:47:54,355::task::495::TaskManager.Task::(_debug) Task 258979ad-d9ef-4054-8f47-cad2e4e66ea9: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:47:54,356::task::495::TaskManager.Task::(_debug) Task 258979ad-d9ef-4054-8f47-cad2e4e66ea9: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:47:54,356::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:47:54,357::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:47:54,358::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:47:54,358::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:47:54,359::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:47:54,359::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:47:54,360::task::495::TaskManager.Task::(_debug) Task 258979ad-d9ef-4054-8f47-cad2e4e66ea9: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:47:54,361::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15916971::DEBUG::2013-11-20 04:47:56,033::clientIF::54::vds::(wrapper) [192.168.100.135]::call create with ({'bridge': 'rhevm', 'acpiEnable': 'true', 'emulatedMachine': 'rhel6.2.0', 'vmId': 'db6ed253-f76f-44bf-96e9-d268b0827271', 'transparentHugePages': 'true', 'spiceSslCipherSuite': 'DEFAULT', 'cpuType': 'Westmere', 'smp': '1', 'macAddr': '00:1a:4a:a8:64:10', 'boot': 'c', 'custom': {}, 'vmType': 'kvm', 'memSize': 1024, 'smpCoresPerSocket': '1', 'vmName': 'baku', 'spiceMonitors': '1', 'nice': '0', 'drives': [{'index': '0', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'format': 'raw', 'volumeID': 'f715faaa-081a-4598-b125-80a2d12af554', 'imageID': '240c0593-fe33-4f8b-a029-097379dbcb85', 'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'propagateErrors': 'off', 'if': 'ide'}], 'nicModel': 'pv', 'keyboardLayout': 'en-us', 'kvmEnable': 'true', 'pitReinjection': 'false', 'timeOffset': '0', 'spiceSecureChannels': 'smain,sinputs', 'display': 'qxl'},) {}
>Dummy-15912651::DEBUG::2013-11-20 04:47:56,341::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916971::INFO::2013-11-20 04:47:56,617::clientIF::523::vds::(_getNetworkIp) network None: using 0
>Thread-15916971::INFO::2013-11-20 04:47:56,622::clientIF::656::vds::(create) vmContainerLock acquired by vm db6ed253-f76f-44bf-96e9-d268b0827271
>Thread-15916972::DEBUG::2013-11-20 04:47:56,653::vm::342::vm.Vm::(_startUnderlyingVm) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Start
>Thread-15916971::DEBUG::2013-11-20 04:47:56,654::clientIF::669::vds::(create) Total desktops after creation of db6ed253-f76f-44bf-96e9-d268b0827271 is 12
>Thread-15916972::DEBUG::2013-11-20 04:47:56,655::vm::346::vm.Vm::(_startUnderlyingVm) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::_ongoingCreations acquired
>Thread-15916971::DEBUG::2013-11-20 04:47:56,656::clientIF::59::vds::(wrapper) return create with {'status': {'message': 'Done', 'code': 0}, 'vmList': {'status': 'WaitForLaunch', 'bridge': 'rhevm', 'acpiEnable': 'true', 'emulatedMachine': 'rhel6.2.0', 'vmId': 'db6ed253-f76f-44bf-96e9-d268b0827271', 'pid': '0', 'drives': [{'index': '0', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'format': 'raw', 'volumeID': 'f715faaa-081a-4598-b125-80a2d12af554', 'imageID': '240c0593-fe33-4f8b-a029-097379dbcb85', 'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'propagateErrors': 'off', 'if': 'ide'}], 'transparentHugePages': 'true', 'keyboardLayout': 'en-us', 'displayPort': '-1', 'displaySecurePort': '-1', 'spiceSslCipherSuite': 'DEFAULT', 'cpuType': 'Westmere', 'custom': {}, 'clientIp': '', 'nicModel': 'pv', 'macAddr': '00:1a:4a:a8:64:10', 'kvmEnable': 'true', 'pitReinjection': 'false', 'boot': 'c', 'smp': '1', 'vmType': 'kvm', 'timeOffset': '0', 'memSize': 1024, 'displayIp': '0', 'spiceSecureChannels': 'smain,sinputs', 'smpCoresPerSocket': '1', 'vmName': 'baku', 'display': 'qxl', 'spiceMonitors': '1', 'nice': '0'}}
>Thread-15916972::INFO::2013-11-20 04:47:56,657::libvirtvm::1061::vm.Vm::(_run) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::VM wrapper has started
>Thread-15916972::INFO::2013-11-20 04:47:56,662::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=240c0593-fe33-4f8b-a029-097379dbcb85 volUUID=f715faaa-081a-4598-b125-80a2d12af554)
>Thread-15916972::DEBUG::2013-11-20 04:47:56,663::task::495::TaskManager.Task::(_debug) Task 7754908c-ef38-44fb-8160-47c23cf89a39: moving from state init -> state preparing
>Thread-15916972::DEBUG::2013-11-20 04:47:56,666::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`295ba6bc-2eda-4f1d-982f-5a1cd665bfa3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1690' at 'public_prepareVolume'
>Thread-15916972::DEBUG::2013-11-20 04:47:56,666::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15916972::DEBUG::2013-11-20 04:47:56,667::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15916972::DEBUG::2013-11-20 04:47:56,668::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`295ba6bc-2eda-4f1d-982f-5a1cd665bfa3`::Granted request
>Thread-15916972::DEBUG::2013-11-20 04:47:56,669::task::495::TaskManager.Task::(_debug) Task 7754908c-ef38-44fb-8160-47c23cf89a39: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15916972::DEBUG::2013-11-20 04:47:56,670::task::495::TaskManager.Task::(_debug) Task 7754908c-ef38-44fb-8160-47c23cf89a39: ref 1 aborting False
>Thread-15916972::DEBUG::2013-11-20 04:47:56,671::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.240c0593-fe33-4f8b-a029-097379dbcb85`ReqID=`6557caf6-1838-4ce6-a329-503da5569595`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1695' at 'public_prepareVolume'
>Thread-15916972::DEBUG::2013-11-20 04:47:56,672::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.240c0593-fe33-4f8b-a029-097379dbcb85' for lock type 'exclusive'
>Thread-15916972::DEBUG::2013-11-20 04:47:56,673::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15916972::DEBUG::2013-11-20 04:47:56,675::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:47:57,078::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.149447 s, 6.9 MB/s\n'; <rc> = 0
>Thread-15916972::DEBUG::2013-11-20 04:47:57,917::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15916972::DEBUG::2013-11-20 04:47:58,741::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15916972::DEBUG::2013-11-20 04:47:58,748::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=27 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:47:59,092::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916972::DEBUG::2013-11-20 04:47:59,368::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.10829 s, 4.7 kB/s\n'; <rc> = 0
>Thread-15916972::DEBUG::2013-11-20 04:47:59,370::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.10829 s, 4.7 kB/s'], size: 512
>Thread-15916972::DEBUG::2013-11-20 04:47:59,372::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=27 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:47:59,882::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.158846 s, 6.4 MB/s\n'; <rc> = 0
>Thread-15916972::DEBUG::2013-11-20 04:47:59,923::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000221732 s, 2.3 MB/s\n'; <rc> = 0
>Thread-15916972::DEBUG::2013-11-20 04:47:59,924::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000221732 s, 2.3 MB/s'], size: 512
>Thread-15916972::INFO::2013-11-20 04:47:59,925::image::340::Storage.Image::(getChain) sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 imgUUID=240c0593-fe33-4f8b-a029-097379dbcb85 chain=[<storage.blockVolume.BlockVolume instance at 0x7f487c139128>] 
>Thread-15916972::DEBUG::2013-11-20 04:47:59,927::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --autobackup n --available y 6b892067-ac7e-4039-a7be-5b7387525b16/f715faaa-081a-4598-b125-80a2d12af554' (cwd None)
>Thread-15916972::DEBUG::2013-11-20 04:48:01,058::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15916972::DEBUG::2013-11-20 04:48:01,060::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15916972::DEBUG::2013-11-20 04:48:01,061::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15916972::DEBUG::2013-11-20 04:48:01,063::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.f715faaa-081a-4598-b125-80a2d12af554`ReqID=`7a3f6c15-a260-4ae7-b39a-d7417b4f543b`::Request was made in '/usr/share/vdsm/storage/resourceFactories.py' line '149' at '__getResourceCandidatesList'
>Thread-15916972::DEBUG::2013-11-20 04:48:01,064::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.f715faaa-081a-4598-b125-80a2d12af554' for lock type 'exclusive'
>Thread-15916972::DEBUG::2013-11-20 04:48:01,065::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.f715faaa-081a-4598-b125-80a2d12af554' is free. Now locking as 'exclusive' (1 active user)
>Thread-15916972::DEBUG::2013-11-20 04:48:01,065::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.f715faaa-081a-4598-b125-80a2d12af554`ReqID=`7a3f6c15-a260-4ae7-b39a-d7417b4f543b`::Granted request
>Thread-15916972::DEBUG::2013-11-20 04:48:01,075::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.240c0593-fe33-4f8b-a029-097379dbcb85' is free. Now locking as 'exclusive' (1 active user)
>Thread-15916972::DEBUG::2013-11-20 04:48:01,075::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.240c0593-fe33-4f8b-a029-097379dbcb85`ReqID=`6557caf6-1838-4ce6-a329-503da5569595`::Granted request
>Thread-15916972::DEBUG::2013-11-20 04:48:01,077::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15916972::DEBUG::2013-11-20 04:48:01,079::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:48:01,896::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916972::DEBUG::2013-11-20 04:48:02,199::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15916972::DEBUG::2013-11-20 04:48:03,220::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15916977::DEBUG::2013-11-20 04:48:03,228::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916977::INFO::2013-11-20 04:48:03,231::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916977::DEBUG::2013-11-20 04:48:03,236::task::495::TaskManager.Task::(_debug) Task 424eb814-e794-4999-80f6-8ab2897e8fbe: moving from state init -> state preparing
>Thread-15916972::DEBUG::2013-11-20 04:48:03,237::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --refresh 6b892067-ac7e-4039-a7be-5b7387525b16/f715faaa-081a-4598-b125-80a2d12af554' (cwd None)
>Thread-15916977::DEBUG::2013-11-20 04:48:03,239::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916977::DEBUG::2013-11-20 04:48:03,241::task::495::TaskManager.Task::(_debug) Task 424eb814-e794-4999-80f6-8ab2897e8fbe: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916977::DEBUG::2013-11-20 04:48:03,460::task::495::TaskManager.Task::(_debug) Task 424eb814-e794-4999-80f6-8ab2897e8fbe: moving from state preparing -> state finished
>Thread-15916977::DEBUG::2013-11-20 04:48:03,476::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916977::DEBUG::2013-11-20 04:48:03,480::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916977::DEBUG::2013-11-20 04:48:03,483::task::495::TaskManager.Task::(_debug) Task 424eb814-e794-4999-80f6-8ab2897e8fbe: ref 0 aborting False
>Thread-15916977::INFO::2013-11-20 04:48:03,484::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916978::DEBUG::2013-11-20 04:48:03,501::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916978::INFO::2013-11-20 04:48:03,504::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916978::DEBUG::2013-11-20 04:48:03,504::task::495::TaskManager.Task::(_debug) Task dad28883-6952-4070-bcf7-8891525c904d: moving from state init -> state preparing
>Thread-15916978::DEBUG::2013-11-20 04:48:03,506::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6fdfb70b-ad2d-4d03-ae7d-a99cf3f33e8d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916978::DEBUG::2013-11-20 04:48:03,507::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916978::DEBUG::2013-11-20 04:48:03,509::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916978::DEBUG::2013-11-20 04:48:03,510::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6fdfb70b-ad2d-4d03-ae7d-a99cf3f33e8d`::Granted request
>Thread-15916978::DEBUG::2013-11-20 04:48:03,512::task::495::TaskManager.Task::(_debug) Task dad28883-6952-4070-bcf7-8891525c904d: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916978::DEBUG::2013-11-20 04:48:03,516::task::495::TaskManager.Task::(_debug) Task dad28883-6952-4070-bcf7-8891525c904d: ref 1 aborting False
>Thread-15916978::WARNING::2013-11-20 04:48:03,519::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916978::DEBUG::2013-11-20 04:48:03,526::task::495::TaskManager.Task::(_debug) Task dad28883-6952-4070-bcf7-8891525c904d: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916978::DEBUG::2013-11-20 04:48:03,528::task::495::TaskManager.Task::(_debug) Task dad28883-6952-4070-bcf7-8891525c904d: moving from state preparing -> state finished
>Thread-15916978::DEBUG::2013-11-20 04:48:03,529::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916978::DEBUG::2013-11-20 04:48:03,530::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916978::DEBUG::2013-11-20 04:48:03,531::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916978::DEBUG::2013-11-20 04:48:03,532::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916978::DEBUG::2013-11-20 04:48:03,533::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916978::DEBUG::2013-11-20 04:48:03,533::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916978::DEBUG::2013-11-20 04:48:03,535::task::495::TaskManager.Task::(_debug) Task dad28883-6952-4070-bcf7-8891525c904d: ref 0 aborting False
>Thread-15916978::INFO::2013-11-20 04:48:03,536::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:48:03,677::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 1.30029 s, 788 kB/s\n'; <rc> = 0
>Thread-15916979::INFO::2013-11-20 04:48:03,787::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916979::DEBUG::2013-11-20 04:48:03,788::task::495::TaskManager.Task::(_debug) Task fb1c8c7a-b9ad-4659-828f-20e015b77994: moving from state init -> state preparing
>Thread-15916979::DEBUG::2013-11-20 04:48:03,789::task::495::TaskManager.Task::(_debug) Task fb1c8c7a-b9ad-4659-828f-20e015b77994: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000436067581177', 'lastCheck': 1384922880.1502471, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00313901901245', 'lastCheck': 1384922883.4818909, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00251913070679', 'lastCheck': 1384922883.4835491, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000487089157104', 'lastCheck': 1384922875.411468, 'valid': True, 'code': 0}}
>Thread-15916979::DEBUG::2013-11-20 04:48:03,790::task::495::TaskManager.Task::(_debug) Task fb1c8c7a-b9ad-4659-828f-20e015b77994: moving from state preparing -> state finished
>Thread-15916979::DEBUG::2013-11-20 04:48:03,791::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916979::DEBUG::2013-11-20 04:48:03,791::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916979::DEBUG::2013-11-20 04:48:03,792::task::495::TaskManager.Task::(_debug) Task fb1c8c7a-b9ad-4659-828f-20e015b77994: ref 0 aborting False
>Thread-15916979::INFO::2013-11-20 04:48:03,793::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000436067581177', 'lastCheck': 1384922880.1502471, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00313901901245', 'lastCheck': 1384922883.4818909, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00251913070679', 'lastCheck': 1384922883.4835491, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000487089157104', 'lastCheck': 1384922875.411468, 'valid': True, 'code': 0}}
>Thread-15916972::DEBUG::2013-11-20 04:48:04,207::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15916972::DEBUG::2013-11-20 04:48:04,208::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15916972::DEBUG::2013-11-20 04:48:04,209::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15916972::INFO::2013-11-20 04:48:04,210::volume::517::Storage.Volume::(prepare) Volume: preparing volume 6b892067-ac7e-4039-a7be-5b7387525b16/f715faaa-081a-4598-b125-80a2d12af554
>Thread-15916972::DEBUG::2013-11-20 04:48:04,211::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15916972::DEBUG::2013-11-20 04:48:04,212::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15916972::DEBUG::2013-11-20 04:48:05,053::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:48:05,741::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916972::DEBUG::2013-11-20 04:48:06,416::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15916972::DEBUG::2013-11-20 04:48:06,450::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=27 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:48:06,674::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:48:06,676::task::495::TaskManager.Task::(_debug) Task 02657f5e-ad8b-4be6-8c9f-6b17ed5c9ee7: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:48:06,680::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b0eb7793-3884-4710-8d24-ecac61c563f2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:48:06,681::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:48:06,684::resourceManager::481::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' found in shared state and queue is empty, Joining current shared lock (2 active users)
>Thread-15913639::DEBUG::2013-11-20 04:48:06,684::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b0eb7793-3884-4710-8d24-ecac61c563f2`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:48:06,687::task::495::TaskManager.Task::(_debug) Task 02657f5e-ad8b-4be6-8c9f-6b17ed5c9ee7: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:48:06,688::task::495::TaskManager.Task::(_debug) Task 02657f5e-ad8b-4be6-8c9f-6b17ed5c9ee7: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:48:06,690::task::495::TaskManager.Task::(_debug) Task 02657f5e-ad8b-4be6-8c9f-6b17ed5c9ee7: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:48:06,692::task::495::TaskManager.Task::(_debug) Task 02657f5e-ad8b-4be6-8c9f-6b17ed5c9ee7: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:48:06,698::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:48:06,699::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:48:06,700::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:48:06,701::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (1 active users)
>Thread-15913639::DEBUG::2013-11-20 04:48:06,703::task::495::TaskManager.Task::(_debug) Task 02657f5e-ad8b-4be6-8c9f-6b17ed5c9ee7: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:48:06,704::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:48:06,851::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.379824 s, 2.7 MB/s\n'; <rc> = 0
>Thread-15916972::DEBUG::2013-11-20 04:48:06,863::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000267608 s, 1.9 MB/s\n'; <rc> = 0
>Thread-15916972::DEBUG::2013-11-20 04:48:06,876::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000267608 s, 1.9 MB/s'], size: 512
>Thread-15916972::DEBUG::2013-11-20 04:48:06,878::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=27 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15916972::DEBUG::2013-11-20 04:48:07,298::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000255984 s, 2.0 MB/s\n'; <rc> = 0
>Thread-15916972::DEBUG::2013-11-20 04:48:07,299::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000255984 s, 2.0 MB/s'], size: 512
>Thread-15916972::DEBUG::2013-11-20 04:48:07,301::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.f715faaa-081a-4598-b125-80a2d12af554`ReqID=`4b828b2b-b64a-48c8-a034-b183e1925832`::Request was made in '/usr/share/vdsm/storage/volume.py' line '533' at 'prepare'
>Thread-15916972::DEBUG::2013-11-20 04:48:07,301::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.f715faaa-081a-4598-b125-80a2d12af554' for lock type 'exclusive'
>Thread-15916972::DEBUG::2013-11-20 04:48:07,303::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.f715faaa-081a-4598-b125-80a2d12af554' is free. Now locking as 'exclusive' (1 active user)
>Thread-15916972::DEBUG::2013-11-20 04:48:07,303::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.f715faaa-081a-4598-b125-80a2d12af554`ReqID=`4b828b2b-b64a-48c8-a034-b183e1925832`::Granted request
>Thread-15916972::DEBUG::2013-11-20 04:48:07,305::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=27 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15916972::DEBUG::2013-11-20 04:48:07,697::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.0003027 s, 1.7 MB/s\n'; <rc> = 0
>Thread-15916972::DEBUG::2013-11-20 04:48:07,699::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.0003027 s, 1.7 MB/s'], size: 512
>Thread-15916972::DEBUG::2013-11-20 04:48:07,701::task::495::TaskManager.Task::(_debug) Task 7754908c-ef38-44fb-8160-47c23cf89a39: finished: None
>Thread-15916972::DEBUG::2013-11-20 04:48:07,702::task::495::TaskManager.Task::(_debug) Task 7754908c-ef38-44fb-8160-47c23cf89a39: moving from state preparing -> state finished
>Thread-15916972::DEBUG::2013-11-20 04:48:07,703::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15916972::DEBUG::2013-11-20 04:48:07,704::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916972::DEBUG::2013-11-20 04:48:07,705::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15916972::DEBUG::2013-11-20 04:48:07,706::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15916972::DEBUG::2013-11-20 04:48:07,707::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15916972::DEBUG::2013-11-20 04:48:07,708::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15916972::DEBUG::2013-11-20 04:48:07,709::task::495::TaskManager.Task::(_debug) Task 7754908c-ef38-44fb-8160-47c23cf89a39: ref 0 aborting False
>Thread-15916972::INFO::2013-11-20 04:48:07,710::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, Return response: {'status': {'message': 'OK', 'code': 0}}
>Thread-15916972::INFO::2013-11-20 04:48:07,711::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=240c0593-fe33-4f8b-a029-097379dbcb85 volUUID=f715faaa-081a-4598-b125-80a2d12af554)
>Thread-15916972::DEBUG::2013-11-20 04:48:07,711::task::495::TaskManager.Task::(_debug) Task 9069a6c8-1f3c-4178-a726-a116aee456fc: moving from state init -> state preparing
>Thread-15916972::DEBUG::2013-11-20 04:48:07,713::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c7921d02-0d95-4e42-b99d-507997a04586`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1666' at 'public_getVolumePath'
>Thread-15916972::DEBUG::2013-11-20 04:48:07,713::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15916972::DEBUG::2013-11-20 04:48:07,714::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15916972::DEBUG::2013-11-20 04:48:07,714::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c7921d02-0d95-4e42-b99d-507997a04586`::Granted request
>Thread-15916972::DEBUG::2013-11-20 04:48:07,715::task::495::TaskManager.Task::(_debug) Task 9069a6c8-1f3c-4178-a726-a116aee456fc: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15916972::DEBUG::2013-11-20 04:48:07,716::task::495::TaskManager.Task::(_debug) Task 9069a6c8-1f3c-4178-a726-a116aee456fc: ref 1 aborting False
>Thread-15916972::DEBUG::2013-11-20 04:48:07,717::task::495::TaskManager.Task::(_debug) Task 9069a6c8-1f3c-4178-a726-a116aee456fc: finished: {'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/240c0593-fe33-4f8b-a029-097379dbcb85/f715faaa-081a-4598-b125-80a2d12af554'}
>Thread-15916972::DEBUG::2013-11-20 04:48:07,718::task::495::TaskManager.Task::(_debug) Task 9069a6c8-1f3c-4178-a726-a116aee456fc: moving from state preparing -> state finished
>Thread-15916972::DEBUG::2013-11-20 04:48:07,718::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15916972::DEBUG::2013-11-20 04:48:07,719::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916972::DEBUG::2013-11-20 04:48:07,719::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15916972::DEBUG::2013-11-20 04:48:07,720::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15916972::DEBUG::2013-11-20 04:48:07,721::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15916972::DEBUG::2013-11-20 04:48:07,721::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15916972::DEBUG::2013-11-20 04:48:07,722::task::495::TaskManager.Task::(_debug) Task 9069a6c8-1f3c-4178-a726-a116aee456fc: ref 0 aborting False
>Thread-15916972::INFO::2013-11-20 04:48:07,723::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, Return response: {'status': {'message': 'OK', 'code': 0}, 'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/240c0593-fe33-4f8b-a029-097379dbcb85/f715faaa-081a-4598-b125-80a2d12af554'}
>Thread-15916972::INFO::2013-11-20 04:48:07,724::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=240c0593-fe33-4f8b-a029-097379dbcb85 volUUID=f715faaa-081a-4598-b125-80a2d12af554)
>Thread-15916972::DEBUG::2013-11-20 04:48:07,724::task::495::TaskManager.Task::(_debug) Task 47c195c7-3dba-43a3-9121-8f5838cdb7c6: moving from state init -> state preparing
>Thread-15916972::DEBUG::2013-11-20 04:48:07,725::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c15c0742-bcca-4ddc-bb62-751c457bdb8b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15916972::DEBUG::2013-11-20 04:48:07,726::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15916972::DEBUG::2013-11-20 04:48:07,727::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15916972::DEBUG::2013-11-20 04:48:07,727::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c15c0742-bcca-4ddc-bb62-751c457bdb8b`::Granted request
>Thread-15916972::DEBUG::2013-11-20 04:48:07,728::task::495::TaskManager.Task::(_debug) Task 47c195c7-3dba-43a3-9121-8f5838cdb7c6: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15916972::DEBUG::2013-11-20 04:48:07,729::task::495::TaskManager.Task::(_debug) Task 47c195c7-3dba-43a3-9121-8f5838cdb7c6: ref 1 aborting False
>Thread-15916972::DEBUG::2013-11-20 04:48:07,729::task::495::TaskManager.Task::(_debug) Task 47c195c7-3dba-43a3-9121-8f5838cdb7c6: finished: {'truesize': '75296145408', 'apparentsize': '75296145408'}
>Thread-15916972::DEBUG::2013-11-20 04:48:07,730::task::495::TaskManager.Task::(_debug) Task 47c195c7-3dba-43a3-9121-8f5838cdb7c6: moving from state preparing -> state finished
>Thread-15916972::DEBUG::2013-11-20 04:48:07,731::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15916972::DEBUG::2013-11-20 04:48:07,731::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916972::DEBUG::2013-11-20 04:48:07,732::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15916972::DEBUG::2013-11-20 04:48:07,733::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15916972::DEBUG::2013-11-20 04:48:07,733::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15916972::DEBUG::2013-11-20 04:48:07,734::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15916972::DEBUG::2013-11-20 04:48:07,734::task::495::TaskManager.Task::(_debug) Task 47c195c7-3dba-43a3-9121-8f5838cdb7c6: ref 0 aborting False
>Thread-15916972::INFO::2013-11-20 04:48:07,735::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '75296145408', 'apparentsize': '75296145408'}
>Thread-15916972::INFO::2013-11-20 04:48:07,736::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16)
>Thread-15916972::DEBUG::2013-11-20 04:48:07,736::task::495::TaskManager.Task::(_debug) Task 2266be15-e8dd-42b9-8128-2a664674c2f0: moving from state init -> state preparing
>Thread-15916972::INFO::2013-11-20 04:48:07,737::blockSD::674::Storage.StorageDomain::(validate) sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16
>Thread-15916972::DEBUG::2013-11-20 04:48:07,738::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15916972::DEBUG::2013-11-20 04:48:07,738::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15916972::DEBUG::2013-11-20 04:48:07,739::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15916972::DEBUG::2013-11-20 04:48:07,741::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15916972::DEBUG::2013-11-20 04:48:07,743::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-15916972::DEBUG::2013-11-20 04:48:07,746::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15916972::DEBUG::2013-11-20 04:48:08,485::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15916972::DEBUG::2013-11-20 04:48:08,501::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-15916972::DEBUG::2013-11-20 04:48:08,505::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'_SHA_CKSUM=716ea7492d9134a4b13de27ac61a336ad474727c', u'POOL_SPM_ID=2', u'POOL_SPM_LVER=287']
>Thread-15916972::DEBUG::2013-11-20 04:48:08,521::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`87c280c2-0d82-43e8-b44a-736848d3bcf9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1297' at 'public_getStorageDomainInfo'
>Thread-15916972::DEBUG::2013-11-20 04:48:08,522::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15916972::DEBUG::2013-11-20 04:48:08,523::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15916972::DEBUG::2013-11-20 04:48:08,524::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`87c280c2-0d82-43e8-b44a-736848d3bcf9`::Granted request
>Thread-15916972::DEBUG::2013-11-20 04:48:08,525::task::495::TaskManager.Task::(_debug) Task 2266be15-e8dd-42b9-8128-2a664674c2f0: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15916972::DEBUG::2013-11-20 04:48:08,526::task::495::TaskManager.Task::(_debug) Task 2266be15-e8dd-42b9-8128-2a664674c2f0: ref 1 aborting False
>Thread-15916972::WARNING::2013-11-20 04:48:08,534::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916972::DEBUG::2013-11-20 04:48:08,538::task::495::TaskManager.Task::(_debug) Task 2266be15-e8dd-42b9-8128-2a664674c2f0: finished: {'info': {'uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'vguuid': 'aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', 'lver': 287, 'state': 'OK', 'version': '2', 'role': 'Master', 'master_ver': 1, 'spm_id': 2, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'OS'}}
>Thread-15916972::DEBUG::2013-11-20 04:48:08,540::task::495::TaskManager.Task::(_debug) Task 2266be15-e8dd-42b9-8128-2a664674c2f0: moving from state preparing -> state finished
>Thread-15916972::DEBUG::2013-11-20 04:48:08,541::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15916972::DEBUG::2013-11-20 04:48:08,541::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916972::DEBUG::2013-11-20 04:48:08,542::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15916972::DEBUG::2013-11-20 04:48:08,543::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15916972::DEBUG::2013-11-20 04:48:08,543::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15916972::DEBUG::2013-11-20 04:48:08,544::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15916972::DEBUG::2013-11-20 04:48:08,545::task::495::TaskManager.Task::(_debug) Task 2266be15-e8dd-42b9-8128-2a664674c2f0: ref 0 aborting False
>Thread-15916972::INFO::2013-11-20 04:48:08,545::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'vguuid': 'aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', 'lver': 287, 'state': 'OK', 'version': '2', 'role': 'Master', 'master_ver': 1, 'spm_id': 2, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'OS'}}
>Thread-15916972::DEBUG::2013-11-20 04:48:08,571::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/libexec/vdsm/hooks/before_vm_start/50_vhostmd' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:48:08,879::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915918::INFO::2013-11-20 04:48:09,175::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:48:09,184::task::495::TaskManager.Task::(_debug) Task c8025b50-4e66-4f86-af9c-79aae1457ed8: moving from state init -> state preparing
>Thread-15916972::DEBUG::2013-11-20 04:48:09,187::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15915918::DEBUG::2013-11-20 04:48:09,191::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c3784729-b15b-455b-bea5-67cbb4d87ee5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15916972::INFO::2013-11-20 04:48:09,191::hooks::51::root::(_runHooksDir) 
>Thread-15915918::DEBUG::2013-11-20 04:48:09,192::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15916972::DEBUG::2013-11-20 04:48:09,194::libvirtvm::1077::vm.Vm::(_run) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::<?xml version="1.0" encoding="utf-8"?>
><domain type="kvm">
>	<name>baku</name>
>	<uuid>db6ed253-f76f-44bf-96e9-d268b0827271</uuid>
>	<memory>1048576</memory>
>	<currentMemory>1048576</currentMemory>
>	<vcpu>1</vcpu>
>	<devices>
>		<disk device="disk" type="block">
>			<source dev="/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/240c0593-fe33-4f8b-a029-097379dbcb85/f715faaa-081a-4598-b125-80a2d12af554"/>
>			<target bus="ide" dev="hda"/>
>			<serial>8b-a029-097379dbcb85</serial>
>			<driver cache="none" error_policy="stop" io="native" name="qemu" type="raw"/>
>		</disk>
>		<disk device="cdrom" type="file">
>			<target bus="ide" dev="hdc"/>
>		</disk>
>		<controller index="0" ports="16" type="virtio-serial"/>
>		<channel type="unix">
>			<target name="com.redhat.rhevm.vdsm" type="virtio"/>
>			<source mode="bind" path="/var/lib/libvirt/qemu/channels/baku.com.redhat.rhevm.vdsm"/>
>		</channel>
>		<memballoon model="none"/>
>		<interface type="bridge">
>			<mac address="00:1a:4a:a8:64:10"/>
>			<model type="virtio"/>
>			<source bridge="rhevm"/>
>		</interface>
>		<input bus="ps2" type="mouse"/>
>		<video>
>			<model heads="1" type="qxl" vram="65536"/>
>		</video>
>		<channel type="spicevmc">
>			<target name="com.redhat.spice.0" type="virtio"/>
>		</channel>
>		<graphics autoport="yes" keymap="en-us" listen="0" passwd="*****" passwdValidTo="1970-01-01T00:00:01" port="-1" tlsPort="-1" type="spice">
>			<channel mode="secure" name="main"/>
>			<channel mode="secure" name="inputs"/>
>		</graphics>
>	</devices>
>	<os>
>		<type arch="x86_64" machine="rhel6.2.0">hvm</type>
>		<boot dev="hd"/>
>		<smbios mode="sysinfo"/>
>	</os>
>	<sysinfo type="smbios">
>		<system>
>			<entry name="manufacturer">Red Hat</entry>
>			<entry name="product">RHEV Hypervisor</entry>
>			<entry name="version">6.3-20121012.0.el6_3</entry>
>			<entry name="serial">30343536-3138-5A43-4A32-333630373442_AC:16:2D:76:EB:F0</entry>
>			<entry name="uuid">db6ed253-f76f-44bf-96e9-d268b0827271</entry>
>		</system>
>	</sysinfo>
>	<clock adjustment="0" offset="variable">
>		<timer name="rtc" tickpolicy="catchup"/>
>	</clock>
>	<features>
>		<acpi/>
>	</features>
>	<cpu match="exact">
>		<model>Westmere</model>
>		<topology cores="1" sockets="1" threads="1"/>
>	</cpu>
></domain>
>
>Thread-15915918::DEBUG::2013-11-20 04:48:09,195::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:48:09,196::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c3784729-b15b-455b-bea5-67cbb4d87ee5`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:48:09,198::task::495::TaskManager.Task::(_debug) Task c8025b50-4e66-4f86-af9c-79aae1457ed8: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:48:09,198::task::495::TaskManager.Task::(_debug) Task c8025b50-4e66-4f86-af9c-79aae1457ed8: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:48:09,200::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15915918::DEBUG::2013-11-20 04:48:09,201::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:48:09,675::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.167837 s, 6.1 MB/s\n'; <rc> = 0
>Thread-15915918::DEBUG::2013-11-20 04:48:09,993::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15915918::DEBUG::2013-11-20 04:48:10,238::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15915918::DEBUG::2013-11-20 04:48:10,239::task::495::TaskManager.Task::(_debug) Task c8025b50-4e66-4f86-af9c-79aae1457ed8: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:48:10,239::task::495::TaskManager.Task::(_debug) Task c8025b50-4e66-4f86-af9c-79aae1457ed8: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:48:10,240::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:48:10,240::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:48:10,241::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:48:10,241::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:48:10,242::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:48:10,242::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:48:10,242::task::495::TaskManager.Task::(_debug) Task c8025b50-4e66-4f86-af9c-79aae1457ed8: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:48:10,243::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>libvirtEventLoop::DEBUG::2013-11-20 04:48:10,441::libvirtvm::1401::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::event Started detail 0 opaque None
>Thread-15916972::DEBUG::2013-11-20 04:48:10,522::utils::307::vm.Vm::(start) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Start statistics collection
>Thread-15916984::DEBUG::2013-11-20 04:48:10,529::utils::336::vm.Vm::(run) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Stats thread started
>Thread-15916985::DEBUG::2013-11-20 04:48:10,532::guestIF::47::root::(_connect) Attempting connection to /var/lib/libvirt/qemu/channels/baku.com.redhat.rhevm.vdsm
>Thread-15916984::INFO::2013-11-20 04:48:10,534::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=240c0593-fe33-4f8b-a029-097379dbcb85 volUUID=f715faaa-081a-4598-b125-80a2d12af554)
>Thread-15916985::DEBUG::2013-11-20 04:48:10,535::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /usr/share/vdsm/prepare-vmchannel /var/lib/libvirt/qemu/channels/baku.com.redhat.rhevm.vdsm' (cwd None)
>Thread-15916984::DEBUG::2013-11-20 04:48:10,536::task::495::TaskManager.Task::(_debug) Task 6e5f4d56-41cc-44e9-9a4b-ee14a46f7295: moving from state init -> state preparing
>Thread-15916984::DEBUG::2013-11-20 04:48:10,732::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`83721d3f-c2db-458c-8b74-06a7d41eceec`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15916972::WARNING::2013-11-20 04:48:10,733::libvirtvm::1124::vm.Vm::(_readPauseCode) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::_readPauseCode unsupported by libvirt vm
>Thread-15916984::DEBUG::2013-11-20 04:48:10,739::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15916972::DEBUG::2013-11-20 04:48:10,745::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /usr/share/vdsm/get-vm-pid baku' (cwd None)
>Thread-15916984::DEBUG::2013-11-20 04:48:10,750::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15916984::DEBUG::2013-11-20 04:48:10,755::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`83721d3f-c2db-458c-8b74-06a7d41eceec`::Granted request
>Thread-15916985::DEBUG::2013-11-20 04:48:11,140::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15916984::DEBUG::2013-11-20 04:48:11,141::task::495::TaskManager.Task::(_debug) Task 6e5f4d56-41cc-44e9-9a4b-ee14a46f7295: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15916984::DEBUG::2013-11-20 04:48:11,144::task::495::TaskManager.Task::(_debug) Task 6e5f4d56-41cc-44e9-9a4b-ee14a46f7295: ref 1 aborting False
>Thread-15916985::DEBUG::2013-11-20 04:48:11,145::guestIF::53::root::(_connect) connected to /var/lib/libvirt/qemu/channels/baku.com.redhat.rhevm.vdsm
>Thread-15916984::DEBUG::2013-11-20 04:48:11,150::task::495::TaskManager.Task::(_debug) Task 6e5f4d56-41cc-44e9-9a4b-ee14a46f7295: finished: {'truesize': '75296145408', 'apparentsize': '75296145408'}
>Thread-15916984::DEBUG::2013-11-20 04:48:11,152::task::495::TaskManager.Task::(_debug) Task 6e5f4d56-41cc-44e9-9a4b-ee14a46f7295: moving from state preparing -> state finished
>Thread-15916984::DEBUG::2013-11-20 04:48:11,153::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15916984::DEBUG::2013-11-20 04:48:11,159::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916984::DEBUG::2013-11-20 04:48:11,166::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15916984::DEBUG::2013-11-20 04:48:11,167::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15916984::DEBUG::2013-11-20 04:48:11,167::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15916984::DEBUG::2013-11-20 04:48:11,173::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15916984::DEBUG::2013-11-20 04:48:11,179::task::495::TaskManager.Task::(_debug) Task 6e5f4d56-41cc-44e9-9a4b-ee14a46f7295: ref 0 aborting False
>Thread-15916984::INFO::2013-11-20 04:48:11,185::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '75296145408', 'apparentsize': '75296145408'}
>Thread-15916972::DEBUG::2013-11-20 04:48:11,333::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15916972::DEBUG::2013-11-20 04:48:11,351::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /sbin/service ksmtuned retune' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:48:11,689::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916972::DEBUG::2013-11-20 04:48:11,957::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15916972::DEBUG::2013-11-20 04:48:11,960::vm::358::vm.Vm::(_startUnderlyingVm) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::_ongoingCreations released
>Dummy-15912651::DEBUG::2013-11-20 04:48:12,270::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.104963 s, 9.8 MB/s\n'; <rc> = 0
>Thread-15916987::DEBUG::2013-11-20 04:48:12,500::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('db6ed253-f76f-44bf-96e9-d268b0827271',) {}
>Thread-15916987::DEBUG::2013-11-20 04:48:12,501::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::CPU stats not available
>Thread-15916987::DEBUG::2013-11-20 04:48:12,502::libvirtvm::202::vm.Vm::(_getNetworkStats) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Network stats not available
>Thread-15916987::DEBUG::2013-11-20 04:48:12,503::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk stats not available
>Thread-15916987::DEBUG::2013-11-20 04:48:12,503::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk latency not available
>Thread-15916987::DEBUG::2013-11-20 04:48:12,504::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '14264', 'displayIp': '0', 'displayPort': u'5922', 'session': 'Unknown', 'displaySecurePort': u'5923', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet13': {'state': 'unknown', 'macAddr': u'00:1a:4a:a8:64:10', 'speed': '1000', 'name': u'vnet13'}}, 'vmId': 'db6ed253-f76f-44bf-96e9-d268b0827271', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '75296145408', 'writeLatency': '0', 'imageID': '240c0593-fe33-4f8b-a029-097379dbcb85', 'flushLatency': '0', 'truesize': '75296145408'}}, 'boot': 'c', 'statsAge': '1.97', 'cpuIdle': '100.00', 'elapsedTime': '15', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15916988::DEBUG::2013-11-20 04:48:13,912::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916988::INFO::2013-11-20 04:48:13,913::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916988::DEBUG::2013-11-20 04:48:13,914::task::495::TaskManager.Task::(_debug) Task 01a42689-54db-4841-bac0-3cd08ef6cbe2: moving from state init -> state preparing
>Thread-15916988::DEBUG::2013-11-20 04:48:13,915::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916988::DEBUG::2013-11-20 04:48:13,915::task::495::TaskManager.Task::(_debug) Task 01a42689-54db-4841-bac0-3cd08ef6cbe2: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916988::DEBUG::2013-11-20 04:48:13,916::task::495::TaskManager.Task::(_debug) Task 01a42689-54db-4841-bac0-3cd08ef6cbe2: moving from state preparing -> state finished
>Thread-15916988::DEBUG::2013-11-20 04:48:13,917::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916988::DEBUG::2013-11-20 04:48:13,917::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916988::DEBUG::2013-11-20 04:48:13,918::task::495::TaskManager.Task::(_debug) Task 01a42689-54db-4841-bac0-3cd08ef6cbe2: ref 0 aborting False
>Thread-15916988::INFO::2013-11-20 04:48:13,919::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916989::DEBUG::2013-11-20 04:48:13,941::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916989::INFO::2013-11-20 04:48:13,952::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916989::DEBUG::2013-11-20 04:48:13,953::task::495::TaskManager.Task::(_debug) Task edba8d45-8f00-480f-bc71-604d944d655b: moving from state init -> state preparing
>Thread-15916989::DEBUG::2013-11-20 04:48:13,954::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b56d4601-d0c6-4e4a-a618-b10a722f76f3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15916989::DEBUG::2013-11-20 04:48:13,955::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15916989::DEBUG::2013-11-20 04:48:13,956::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15916989::DEBUG::2013-11-20 04:48:13,956::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b56d4601-d0c6-4e4a-a618-b10a722f76f3`::Granted request
>Thread-15916989::DEBUG::2013-11-20 04:48:13,957::task::495::TaskManager.Task::(_debug) Task edba8d45-8f00-480f-bc71-604d944d655b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15916989::DEBUG::2013-11-20 04:48:13,958::task::495::TaskManager.Task::(_debug) Task edba8d45-8f00-480f-bc71-604d944d655b: ref 1 aborting False
>Thread-15916989::WARNING::2013-11-20 04:48:13,960::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15916989::DEBUG::2013-11-20 04:48:13,962::task::495::TaskManager.Task::(_debug) Task edba8d45-8f00-480f-bc71-604d944d655b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15916989::DEBUG::2013-11-20 04:48:13,965::task::495::TaskManager.Task::(_debug) Task edba8d45-8f00-480f-bc71-604d944d655b: moving from state preparing -> state finished
>Thread-15916989::DEBUG::2013-11-20 04:48:13,965::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15916989::DEBUG::2013-11-20 04:48:13,966::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916989::DEBUG::2013-11-20 04:48:13,967::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15916989::DEBUG::2013-11-20 04:48:13,968::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15916989::DEBUG::2013-11-20 04:48:13,968::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15916989::DEBUG::2013-11-20 04:48:13,969::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15916989::DEBUG::2013-11-20 04:48:13,970::task::495::TaskManager.Task::(_debug) Task edba8d45-8f00-480f-bc71-604d944d655b: ref 0 aborting False
>Thread-15916989::INFO::2013-11-20 04:48:13,973::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:48:14,286::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916990::INFO::2013-11-20 04:48:14,630::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15916990::DEBUG::2013-11-20 04:48:14,632::task::495::TaskManager.Task::(_debug) Task 93780f9f-5da5-4cca-9114-e3689d32cb5d: moving from state init -> state preparing
>Thread-15916990::DEBUG::2013-11-20 04:48:14,633::task::495::TaskManager.Task::(_debug) Task 93780f9f-5da5-4cca-9114-e3689d32cb5d: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00276803970337', 'lastCheck': 1384922891.1436529, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00197196006775', 'lastCheck': 1384922894.1378059, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00169706344604', 'lastCheck': 1384922894.1381381, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.143919944763', 'lastCheck': 1384922886.0050271, 'valid': True, 'code': 0}}
>Thread-15916990::DEBUG::2013-11-20 04:48:14,634::task::495::TaskManager.Task::(_debug) Task 93780f9f-5da5-4cca-9114-e3689d32cb5d: moving from state preparing -> state finished
>Thread-15916990::DEBUG::2013-11-20 04:48:14,634::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15916990::DEBUG::2013-11-20 04:48:14,635::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916990::DEBUG::2013-11-20 04:48:14,636::task::495::TaskManager.Task::(_debug) Task 93780f9f-5da5-4cca-9114-e3689d32cb5d: ref 0 aborting False
>Thread-15916990::INFO::2013-11-20 04:48:14,636::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00276803970337', 'lastCheck': 1384922891.1436529, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00197196006775', 'lastCheck': 1384922894.1378059, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00169706344604', 'lastCheck': 1384922894.1381381, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.143919944763', 'lastCheck': 1384922886.0050271, 'valid': True, 'code': 0}}
>Thread-15916991::DEBUG::2013-11-20 04:48:14,663::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::CPU stats not available
>Thread-15916991::DEBUG::2013-11-20 04:48:14,664::libvirtvm::202::vm.Vm::(_getNetworkStats) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Network stats not available
>Thread-15916991::DEBUG::2013-11-20 04:48:14,665::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk stats not available
>Thread-15916991::DEBUG::2013-11-20 04:48:14,665::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk latency not available
>Dummy-15912651::DEBUG::2013-11-20 04:48:14,822::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0899731 s, 11.4 MB/s\n'; <rc> = 0
>Thread-15915863::INFO::2013-11-20 04:48:15,129::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:48:15,130::task::495::TaskManager.Task::(_debug) Task 697c136f-1eec-4d9b-a0de-5b693882208b: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:48:15,132::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`27cd3516-a6b6-46fd-9ebd-b8ab5a13d019`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:48:15,133::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:48:15,134::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:48:15,134::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`27cd3516-a6b6-46fd-9ebd-b8ab5a13d019`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:48:15,136::task::495::TaskManager.Task::(_debug) Task 697c136f-1eec-4d9b-a0de-5b693882208b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:48:15,137::task::495::TaskManager.Task::(_debug) Task 697c136f-1eec-4d9b-a0de-5b693882208b: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:48:15,138::task::495::TaskManager.Task::(_debug) Task 697c136f-1eec-4d9b-a0de-5b693882208b: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:48:15,140::task::495::TaskManager.Task::(_debug) Task 697c136f-1eec-4d9b-a0de-5b693882208b: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:48:15,141::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:48:15,142::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:48:15,144::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:48:15,145::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:48:15,146::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:48:15,147::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:48:15,149::task::495::TaskManager.Task::(_debug) Task 697c136f-1eec-4d9b-a0de-5b693882208b: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:48:15,150::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:48:16,837::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916993::DEBUG::2013-11-20 04:48:17,191::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('db6ed253-f76f-44bf-96e9-d268b0827271',) {}
>Thread-15916993::DEBUG::2013-11-20 04:48:17,201::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::CPU stats not available
>Thread-15916993::DEBUG::2013-11-20 04:48:17,222::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk stats not available
>Thread-15916993::DEBUG::2013-11-20 04:48:17,230::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk latency not available
>Thread-15916993::DEBUG::2013-11-20 04:48:17,248::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '14264', 'displayIp': '0', 'displayPort': u'5922', 'session': 'Unknown', 'displaySecurePort': u'5923', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet13': {'macAddr': u'00:1a:4a:a8:64:10', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet13'}}, 'vmId': 'db6ed253-f76f-44bf-96e9-d268b0827271', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '75296145408', 'writeLatency': '0', 'imageID': '240c0593-fe33-4f8b-a029-097379dbcb85', 'flushLatency': '0', 'truesize': '75296145408'}}, 'boot': 'c', 'statsAge': '0.99', 'cpuIdle': '100.00', 'elapsedTime': '20', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 04:48:17,440::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.152234 s, 6.7 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 04:48:19,197::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:48:19,198::task::495::TaskManager.Task::(_debug) Task 9ab847bc-403e-4cfb-8f50-48650deb4fee: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:48:19,200::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`476c9e36-181b-4064-8144-2a4df4d4acc1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:48:19,201::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:48:19,203::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:48:19,203::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`476c9e36-181b-4064-8144-2a4df4d4acc1`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:48:19,207::task::495::TaskManager.Task::(_debug) Task 9ab847bc-403e-4cfb-8f50-48650deb4fee: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:48:19,208::task::495::TaskManager.Task::(_debug) Task 9ab847bc-403e-4cfb-8f50-48650deb4fee: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:48:19,208::task::495::TaskManager.Task::(_debug) Task 9ab847bc-403e-4cfb-8f50-48650deb4fee: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:48:19,209::task::495::TaskManager.Task::(_debug) Task 9ab847bc-403e-4cfb-8f50-48650deb4fee: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:48:19,210::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:48:19,211::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:48:19,212::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:48:19,213::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:48:19,213::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:48:19,214::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:48:19,215::task::495::TaskManager.Task::(_debug) Task 9ab847bc-403e-4cfb-8f50-48650deb4fee: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:48:19,216::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:48:19,450::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916995::DEBUG::2013-11-20 04:48:19,786::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('db6ed253-f76f-44bf-96e9-d268b0827271',) {}
>Thread-15916995::DEBUG::2013-11-20 04:48:19,798::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::CPU stats not available
>Thread-15916995::DEBUG::2013-11-20 04:48:19,814::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk stats not available
>Thread-15916995::DEBUG::2013-11-20 04:48:19,822::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk latency not available
>Thread-15916995::DEBUG::2013-11-20 04:48:19,833::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '14264', 'displayIp': '0', 'displayPort': u'5922', 'session': 'Unknown', 'displaySecurePort': u'5923', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet13': {'macAddr': u'00:1a:4a:a8:64:10', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet13'}}, 'vmId': 'db6ed253-f76f-44bf-96e9-d268b0827271', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '75296145408', 'writeLatency': '0', 'imageID': '240c0593-fe33-4f8b-a029-097379dbcb85', 'flushLatency': '0', 'truesize': '75296145408'}}, 'boot': 'c', 'statsAge': '0.57', 'cpuIdle': '100.00', 'elapsedTime': '23', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 04:48:19,999::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0926846 s, 11.0 MB/s\n'; <rc> = 0
>Thread-15916997::DEBUG::2013-11-20 04:48:21,965::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('db6ed253-f76f-44bf-96e9-d268b0827271',) {}
>Thread-15916997::DEBUG::2013-11-20 04:48:21,969::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::CPU stats not available
>Thread-15916997::DEBUG::2013-11-20 04:48:21,976::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk stats not available
>Thread-15916997::DEBUG::2013-11-20 04:48:21,988::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk latency not available
>Thread-15916997::DEBUG::2013-11-20 04:48:21,997::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '14264', 'displayIp': '0', 'displayPort': u'5922', 'session': 'Unknown', 'displaySecurePort': u'5923', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet13': {'macAddr': u'00:1a:4a:a8:64:10', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet13'}}, 'vmId': 'db6ed253-f76f-44bf-96e9-d268b0827271', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '75296145408', 'writeLatency': '0', 'imageID': '240c0593-fe33-4f8b-a029-097379dbcb85', 'flushLatency': '0', 'truesize': '75296145408'}}, 'boot': 'c', 'statsAge': '0.74', 'cpuIdle': '100.00', 'elapsedTime': '25', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 04:48:22,019::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:48:22,564::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0868285 s, 11.8 MB/s\n'; <rc> = 0
>Thread-15916999::DEBUG::2013-11-20 04:48:24,231::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('db6ed253-f76f-44bf-96e9-d268b0827271',) {}
>Thread-15916999::DEBUG::2013-11-20 04:48:24,237::libvirtvm::171::vm.Vm::(_getCpuStats) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::CPU stats not available
>Thread-15916999::DEBUG::2013-11-20 04:48:24,239::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk stats not available
>Thread-15916999::DEBUG::2013-11-20 04:48:24,241::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk latency not available
>Thread-15916999::DEBUG::2013-11-20 04:48:24,242::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '14264', 'displayIp': '0', 'displayPort': u'5922', 'session': 'Unknown', 'displaySecurePort': u'5923', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet13': {'macAddr': u'00:1a:4a:a8:64:10', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet13'}}, 'vmId': 'db6ed253-f76f-44bf-96e9-d268b0827271', 'monitorResponse': '0', 'cpuUser': '0.00', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '75296145408', 'writeLatency': '0', 'imageID': '240c0593-fe33-4f8b-a029-097379dbcb85', 'flushLatency': '0', 'truesize': '75296145408'}}, 'boot': 'c', 'statsAge': '1.01', 'cpuIdle': '100.00', 'elapsedTime': '27', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15917000::DEBUG::2013-11-20 04:48:24,271::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917000::INFO::2013-11-20 04:48:24,284::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917000::DEBUG::2013-11-20 04:48:24,290::task::495::TaskManager.Task::(_debug) Task 794256dc-a9a4-49e3-ba3c-cef452824298: moving from state init -> state preparing
>Thread-15917000::DEBUG::2013-11-20 04:48:24,307::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917000::DEBUG::2013-11-20 04:48:24,308::task::495::TaskManager.Task::(_debug) Task 794256dc-a9a4-49e3-ba3c-cef452824298: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917000::DEBUG::2013-11-20 04:48:24,314::task::495::TaskManager.Task::(_debug) Task 794256dc-a9a4-49e3-ba3c-cef452824298: moving from state preparing -> state finished
>Thread-15917000::DEBUG::2013-11-20 04:48:24,329::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917000::DEBUG::2013-11-20 04:48:24,341::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917000::DEBUG::2013-11-20 04:48:24,341::task::495::TaskManager.Task::(_debug) Task 794256dc-a9a4-49e3-ba3c-cef452824298: ref 0 aborting False
>Thread-15917000::INFO::2013-11-20 04:48:24,364::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917001::DEBUG::2013-11-20 04:48:24,417::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917001::INFO::2013-11-20 04:48:24,445::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917001::DEBUG::2013-11-20 04:48:24,455::task::495::TaskManager.Task::(_debug) Task 84d36159-e5a7-42ae-b459-2bdf5db8cafe: moving from state init -> state preparing
>Thread-15917001::DEBUG::2013-11-20 04:48:24,471::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b744fc32-01d7-45a4-bbf2-043d447f1b64`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917001::DEBUG::2013-11-20 04:48:24,474::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917001::DEBUG::2013-11-20 04:48:24,486::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917001::DEBUG::2013-11-20 04:48:24,498::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`b744fc32-01d7-45a4-bbf2-043d447f1b64`::Granted request
>Thread-15917001::DEBUG::2013-11-20 04:48:24,512::task::495::TaskManager.Task::(_debug) Task 84d36159-e5a7-42ae-b459-2bdf5db8cafe: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917001::DEBUG::2013-11-20 04:48:24,518::task::495::TaskManager.Task::(_debug) Task 84d36159-e5a7-42ae-b459-2bdf5db8cafe: ref 1 aborting False
>Thread-15917001::WARNING::2013-11-20 04:48:24,532::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917001::DEBUG::2013-11-20 04:48:24,536::task::495::TaskManager.Task::(_debug) Task 84d36159-e5a7-42ae-b459-2bdf5db8cafe: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917001::DEBUG::2013-11-20 04:48:24,551::task::495::TaskManager.Task::(_debug) Task 84d36159-e5a7-42ae-b459-2bdf5db8cafe: moving from state preparing -> state finished
>Thread-15917001::DEBUG::2013-11-20 04:48:24,555::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917001::DEBUG::2013-11-20 04:48:24,556::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917001::DEBUG::2013-11-20 04:48:24,561::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917001::DEBUG::2013-11-20 04:48:24,563::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917001::DEBUG::2013-11-20 04:48:24,572::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917001::DEBUG::2013-11-20 04:48:24,573::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917001::DEBUG::2013-11-20 04:48:24,574::task::495::TaskManager.Task::(_debug) Task 84d36159-e5a7-42ae-b459-2bdf5db8cafe: ref 0 aborting False
>Dummy-15912651::DEBUG::2013-11-20 04:48:24,577::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15917001::INFO::2013-11-20 04:48:24,579::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:48:25,117::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0769409 s, 13.3 MB/s\n'; <rc> = 0
>Thread-15917002::INFO::2013-11-20 04:48:26,362::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15917002::DEBUG::2013-11-20 04:48:26,363::task::495::TaskManager.Task::(_debug) Task f5f7946b-e821-486f-bda1-83cc4e5ddf45: moving from state init -> state preparing
>Thread-15917002::DEBUG::2013-11-20 04:48:26,367::task::495::TaskManager.Task::(_debug) Task f5f7946b-e821-486f-bda1-83cc4e5ddf45: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000370025634766', 'lastCheck': 1384922901.2016411, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00185012817383', 'lastCheck': 1384922904.239979, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00494813919067', 'lastCheck': 1384922904.2456689, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000408887863159', 'lastCheck': 1384922906.2246921, 'valid': True, 'code': 0}}
>Thread-15917002::DEBUG::2013-11-20 04:48:26,368::task::495::TaskManager.Task::(_debug) Task f5f7946b-e821-486f-bda1-83cc4e5ddf45: moving from state preparing -> state finished
>Thread-15917002::DEBUG::2013-11-20 04:48:26,372::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917002::DEBUG::2013-11-20 04:48:26,380::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917002::DEBUG::2013-11-20 04:48:26,381::task::495::TaskManager.Task::(_debug) Task f5f7946b-e821-486f-bda1-83cc4e5ddf45: ref 0 aborting False
>Thread-15917002::INFO::2013-11-20 04:48:26,388::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000370025634766', 'lastCheck': 1384922901.2016411, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00185012817383', 'lastCheck': 1384922904.239979, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00494813919067', 'lastCheck': 1384922904.2456689, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000408887863159', 'lastCheck': 1384922906.2246921, 'valid': True, 'code': 0}}
>Thread-15917003::DEBUG::2013-11-20 04:48:26,528::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk stats not available
>Thread-15917003::DEBUG::2013-11-20 04:48:26,529::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk latency not available
>Dummy-15912651::DEBUG::2013-11-20 04:48:27,134::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913518::INFO::2013-11-20 04:48:27,406::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:48:27,408::task::495::TaskManager.Task::(_debug) Task 3c14ef16-54ae-4b2c-9159-e246936d030b: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:48:27,410::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`774cefd0-2ece-454b-b338-3aeb0ead5274`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:48:27,411::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:48:27,412::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:48:27,413::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`774cefd0-2ece-454b-b338-3aeb0ead5274`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:48:27,414::task::495::TaskManager.Task::(_debug) Task 3c14ef16-54ae-4b2c-9159-e246936d030b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:48:27,415::task::495::TaskManager.Task::(_debug) Task 3c14ef16-54ae-4b2c-9159-e246936d030b: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:48:27,416::task::495::TaskManager.Task::(_debug) Task 3c14ef16-54ae-4b2c-9159-e246936d030b: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:48:27,416::task::495::TaskManager.Task::(_debug) Task 3c14ef16-54ae-4b2c-9159-e246936d030b: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:48:27,417::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:48:27,418::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:48:27,418::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:48:27,419::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:48:27,419::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:48:27,420::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:48:27,421::task::495::TaskManager.Task::(_debug) Task 3c14ef16-54ae-4b2c-9159-e246936d030b: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:48:27,421::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:48:27,423::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:48:27,423::task::495::TaskManager.Task::(_debug) Task 4b151119-03a2-4d4a-8573-f00dce3ba1ed: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:48:27,424::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e7924b77-bcbd-4638-80f2-66ae9ec695f5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:48:27,425::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:48:27,426::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:48:27,426::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e7924b77-bcbd-4638-80f2-66ae9ec695f5`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:48:27,427::task::495::TaskManager.Task::(_debug) Task 4b151119-03a2-4d4a-8573-f00dce3ba1ed: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:48:27,428::task::495::TaskManager.Task::(_debug) Task 4b151119-03a2-4d4a-8573-f00dce3ba1ed: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:48:27,428::task::495::TaskManager.Task::(_debug) Task 4b151119-03a2-4d4a-8573-f00dce3ba1ed: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:48:27,429::task::495::TaskManager.Task::(_debug) Task 4b151119-03a2-4d4a-8573-f00dce3ba1ed: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:48:27,430::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:48:27,430::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:48:27,431::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:48:27,431::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:48:27,432::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:48:27,433::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:48:27,434::task::495::TaskManager.Task::(_debug) Task 4b151119-03a2-4d4a-8573-f00dce3ba1ed: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:48:27,434::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:48:27,696::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0756033 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15917005::DEBUG::2013-11-20 04:48:28,821::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('db6ed253-f76f-44bf-96e9-d268b0827271',) {}
>Thread-15917005::DEBUG::2013-11-20 04:48:28,833::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk stats not available
>Thread-15917005::DEBUG::2013-11-20 04:48:28,838::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk latency not available
>Thread-15917005::DEBUG::2013-11-20 04:48:28,843::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '14264', 'displayIp': '0', 'displayPort': u'5922', 'session': 'Unknown', 'displaySecurePort': u'5923', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet13': {'macAddr': u'00:1a:4a:a8:64:10', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet13'}}, 'vmId': 'db6ed253-f76f-44bf-96e9-d268b0827271', 'monitorResponse': '0', 'cpuUser': '53.19', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '75296145408', 'writeLatency': '0', 'imageID': '240c0593-fe33-4f8b-a029-097379dbcb85', 'flushLatency': '0', 'truesize': '75296145408'}}, 'boot': 'c', 'statsAge': '1.49', 'cpuIdle': '46.81', 'elapsedTime': '32', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15912392::INFO::2013-11-20 04:48:29,365::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:48:29,366::task::495::TaskManager.Task::(_debug) Task d5f8176c-d093-43bd-bec6-0b0d2518fd9d: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:48:29,367::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f03ddb8e-cd29-4d98-83f3-1146c8e780c8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:48:29,367::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:48:29,368::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:48:29,368::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f03ddb8e-cd29-4d98-83f3-1146c8e780c8`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:48:29,369::task::495::TaskManager.Task::(_debug) Task d5f8176c-d093-43bd-bec6-0b0d2518fd9d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:48:29,369::task::495::TaskManager.Task::(_debug) Task d5f8176c-d093-43bd-bec6-0b0d2518fd9d: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:48:29,370::task::495::TaskManager.Task::(_debug) Task d5f8176c-d093-43bd-bec6-0b0d2518fd9d: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:48:29,370::task::495::TaskManager.Task::(_debug) Task d5f8176c-d093-43bd-bec6-0b0d2518fd9d: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:48:29,370::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:48:29,371::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:48:29,371::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:48:29,371::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:48:29,372::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:48:29,372::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:48:29,372::task::495::TaskManager.Task::(_debug) Task d5f8176c-d093-43bd-bec6-0b0d2518fd9d: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:48:29,373::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:48:29,713::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:48:30,263::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0786985 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15917007::DEBUG::2013-11-20 04:48:30,926::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('db6ed253-f76f-44bf-96e9-d268b0827271',) {}
>Thread-15917007::DEBUG::2013-11-20 04:48:30,930::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk stats not available
>Thread-15917007::DEBUG::2013-11-20 04:48:30,930::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk latency not available
>Thread-15917007::DEBUG::2013-11-20 04:48:30,931::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '14264', 'displayIp': '0', 'displayPort': u'5922', 'session': 'Unknown', 'displaySecurePort': u'5923', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet13': {'macAddr': u'00:1a:4a:a8:64:10', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet13'}}, 'vmId': 'db6ed253-f76f-44bf-96e9-d268b0827271', 'monitorResponse': '0', 'cpuUser': '53.19', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '75296145408', 'writeLatency': '0', 'imageID': '240c0593-fe33-4f8b-a029-097379dbcb85', 'flushLatency': '0', 'truesize': '75296145408'}}, 'boot': 'c', 'statsAge': '1.59', 'cpuIdle': '46.81', 'elapsedTime': '34', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 04:48:32,281::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:48:32,811::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0746724 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15917009::DEBUG::2013-11-20 04:48:33,005::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('db6ed253-f76f-44bf-96e9-d268b0827271',) {}
>Thread-15917009::DEBUG::2013-11-20 04:48:33,013::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk stats not available
>Thread-15917009::DEBUG::2013-11-20 04:48:33,016::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk latency not available
>Thread-15917009::DEBUG::2013-11-20 04:48:33,020::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '14264', 'displayIp': '0', 'displayPort': u'5922', 'session': 'Unknown', 'displaySecurePort': u'5923', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet13': {'macAddr': u'00:1a:4a:a8:64:10', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet13'}}, 'vmId': 'db6ed253-f76f-44bf-96e9-d268b0827271', 'monitorResponse': '0', 'cpuUser': '53.19', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '75296145408', 'writeLatency': '0', 'imageID': '240c0593-fe33-4f8b-a029-097379dbcb85', 'flushLatency': '0', 'truesize': '75296145408'}}, 'boot': 'c', 'statsAge': '1.68', 'cpuIdle': '46.81', 'elapsedTime': '36', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 04:48:34,826::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15917010::DEBUG::2013-11-20 04:48:35,046::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917010::INFO::2013-11-20 04:48:35,054::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917010::DEBUG::2013-11-20 04:48:35,058::task::495::TaskManager.Task::(_debug) Task 1e8aa081-12b4-4416-8145-7f4af7644b6c: moving from state init -> state preparing
>Thread-15917010::DEBUG::2013-11-20 04:48:35,064::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917010::DEBUG::2013-11-20 04:48:35,069::task::495::TaskManager.Task::(_debug) Task 1e8aa081-12b4-4416-8145-7f4af7644b6c: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917010::DEBUG::2013-11-20 04:48:35,070::task::495::TaskManager.Task::(_debug) Task 1e8aa081-12b4-4416-8145-7f4af7644b6c: moving from state preparing -> state finished
>Thread-15917010::DEBUG::2013-11-20 04:48:35,072::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917010::DEBUG::2013-11-20 04:48:35,090::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917012::DEBUG::2013-11-20 04:48:35,096::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('db6ed253-f76f-44bf-96e9-d268b0827271',) {}
>Thread-15917010::DEBUG::2013-11-20 04:48:35,097::task::495::TaskManager.Task::(_debug) Task 1e8aa081-12b4-4416-8145-7f4af7644b6c: ref 0 aborting False
>Thread-15917012::DEBUG::2013-11-20 04:48:35,101::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk stats not available
>Thread-15917010::INFO::2013-11-20 04:48:35,102::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917012::DEBUG::2013-11-20 04:48:35,107::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk latency not available
>Thread-15917012::DEBUG::2013-11-20 04:48:35,112::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '14264', 'displayIp': '0', 'displayPort': u'5922', 'session': 'Unknown', 'displaySecurePort': u'5923', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet13': {'macAddr': u'00:1a:4a:a8:64:10', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet13'}}, 'vmId': 'db6ed253-f76f-44bf-96e9-d268b0827271', 'monitorResponse': '0', 'cpuUser': '53.19', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '75296145408', 'writeLatency': '0', 'imageID': '240c0593-fe33-4f8b-a029-097379dbcb85', 'flushLatency': '0', 'truesize': '75296145408'}}, 'boot': 'c', 'statsAge': '1.76', 'cpuIdle': '46.81', 'elapsedTime': '38', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15917013::DEBUG::2013-11-20 04:48:35,134::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917013::INFO::2013-11-20 04:48:35,136::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917013::DEBUG::2013-11-20 04:48:35,137::task::495::TaskManager.Task::(_debug) Task 29522bff-1691-4a16-b632-872fd008a298: moving from state init -> state preparing
>Thread-15917013::DEBUG::2013-11-20 04:48:35,138::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4cf39141-334c-43a3-b2f4-38efa3eb3bee`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917013::DEBUG::2013-11-20 04:48:35,139::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917013::DEBUG::2013-11-20 04:48:35,140::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917013::DEBUG::2013-11-20 04:48:35,141::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`4cf39141-334c-43a3-b2f4-38efa3eb3bee`::Granted request
>Thread-15917013::DEBUG::2013-11-20 04:48:35,142::task::495::TaskManager.Task::(_debug) Task 29522bff-1691-4a16-b632-872fd008a298: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917013::DEBUG::2013-11-20 04:48:35,143::task::495::TaskManager.Task::(_debug) Task 29522bff-1691-4a16-b632-872fd008a298: ref 1 aborting False
>Thread-15917013::WARNING::2013-11-20 04:48:35,146::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917013::DEBUG::2013-11-20 04:48:35,147::task::495::TaskManager.Task::(_debug) Task 29522bff-1691-4a16-b632-872fd008a298: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917013::DEBUG::2013-11-20 04:48:35,148::task::495::TaskManager.Task::(_debug) Task 29522bff-1691-4a16-b632-872fd008a298: moving from state preparing -> state finished
>Thread-15917013::DEBUG::2013-11-20 04:48:35,148::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917013::DEBUG::2013-11-20 04:48:35,150::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917013::DEBUG::2013-11-20 04:48:35,151::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917013::DEBUG::2013-11-20 04:48:35,151::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917013::DEBUG::2013-11-20 04:48:35,152::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917013::DEBUG::2013-11-20 04:48:35,152::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917013::DEBUG::2013-11-20 04:48:35,153::task::495::TaskManager.Task::(_debug) Task 29522bff-1691-4a16-b632-872fd008a298: ref 0 aborting False
>Thread-15917013::INFO::2013-11-20 04:48:35,154::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:48:35,308::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0483909 s, 21.2 MB/s\n'; <rc> = 0
>Thread-15917014::INFO::2013-11-20 04:48:37,179::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15917014::DEBUG::2013-11-20 04:48:37,180::task::495::TaskManager.Task::(_debug) Task b7e19c99-b2ae-4328-ab1e-72f9e820b521: moving from state init -> state preparing
>Thread-15917014::DEBUG::2013-11-20 04:48:37,181::task::495::TaskManager.Task::(_debug) Task b7e19c99-b2ae-4328-ab1e-72f9e820b521: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000324964523315', 'lastCheck': 1384922911.345649, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00157189369202', 'lastCheck': 1384922914.494199, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0015389919281', 'lastCheck': 1384922914.4948039, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000416040420532', 'lastCheck': 1384922916.489645, 'valid': True, 'code': 0}}
>Thread-15917014::DEBUG::2013-11-20 04:48:37,182::task::495::TaskManager.Task::(_debug) Task b7e19c99-b2ae-4328-ab1e-72f9e820b521: moving from state preparing -> state finished
>Thread-15917014::DEBUG::2013-11-20 04:48:37,183::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917014::DEBUG::2013-11-20 04:48:37,184::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917014::DEBUG::2013-11-20 04:48:37,184::task::495::TaskManager.Task::(_debug) Task b7e19c99-b2ae-4328-ab1e-72f9e820b521: ref 0 aborting False
>Thread-15917014::INFO::2013-11-20 04:48:37,185::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000324964523315', 'lastCheck': 1384922911.345649, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00157189369202', 'lastCheck': 1384922914.494199, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0015389919281', 'lastCheck': 1384922914.4948039, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000416040420532', 'lastCheck': 1384922916.489645, 'valid': True, 'code': 0}}
>Thread-15917015::DEBUG::2013-11-20 04:48:37,200::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk stats not available
>Thread-15917015::DEBUG::2013-11-20 04:48:37,201::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk latency not available
>Dummy-15912651::DEBUG::2013-11-20 04:48:37,318::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:48:37,834::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0579165 s, 17.7 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 04:48:38,530::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:48:38,531::task::495::TaskManager.Task::(_debug) Task b0978297-8f6b-48b9-ba63-fde32f60284b: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:48:38,533::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`46f6f7b1-1449-4b05-82ae-458fc01c5ca7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:48:38,534::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:48:38,535::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:48:38,536::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`46f6f7b1-1449-4b05-82ae-458fc01c5ca7`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:48:38,537::task::495::TaskManager.Task::(_debug) Task b0978297-8f6b-48b9-ba63-fde32f60284b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:48:38,538::task::495::TaskManager.Task::(_debug) Task b0978297-8f6b-48b9-ba63-fde32f60284b: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:48:38,538::task::495::TaskManager.Task::(_debug) Task b0978297-8f6b-48b9-ba63-fde32f60284b: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:48:38,539::task::495::TaskManager.Task::(_debug) Task b0978297-8f6b-48b9-ba63-fde32f60284b: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:48:38,540::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:48:38,540::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:48:38,541::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:48:38,542::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:48:38,542::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:48:38,543::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:48:38,544::task::495::TaskManager.Task::(_debug) Task b0978297-8f6b-48b9-ba63-fde32f60284b: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:48:38,544::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15917017::DEBUG::2013-11-20 04:48:39,443::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('db6ed253-f76f-44bf-96e9-d268b0827271',) {}
>Thread-15917017::DEBUG::2013-11-20 04:48:39,444::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk stats not available
>Thread-15917017::DEBUG::2013-11-20 04:48:39,445::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk latency not available
>Thread-15917017::DEBUG::2013-11-20 04:48:39,446::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '14264', 'displayIp': '0', 'displayPort': u'5922', 'session': 'Unknown', 'displaySecurePort': u'5923', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet13': {'macAddr': u'00:1a:4a:a8:64:10', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet13'}}, 'vmId': 'db6ed253-f76f-44bf-96e9-d268b0827271', 'monitorResponse': '0', 'cpuUser': '53.19', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '75296145408', 'writeLatency': '0', 'imageID': '240c0593-fe33-4f8b-a029-097379dbcb85', 'flushLatency': '0', 'truesize': '75296145408'}}, 'boot': 'c', 'statsAge': '1.90', 'cpuIdle': '46.81', 'elapsedTime': '42', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15914453::INFO::2013-11-20 04:48:39,547::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:48:39,548::task::495::TaskManager.Task::(_debug) Task 3e2b7762-dda8-493f-aa16-f5b9e0ebf198: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:48:39,551::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d6469270-2876-4edd-8700-aef345706325`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:48:39,552::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:48:39,553::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:48:39,553::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d6469270-2876-4edd-8700-aef345706325`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:48:39,554::task::495::TaskManager.Task::(_debug) Task 3e2b7762-dda8-493f-aa16-f5b9e0ebf198: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:48:39,555::task::495::TaskManager.Task::(_debug) Task 3e2b7762-dda8-493f-aa16-f5b9e0ebf198: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:48:39,555::task::495::TaskManager.Task::(_debug) Task 3e2b7762-dda8-493f-aa16-f5b9e0ebf198: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:48:39,556::task::495::TaskManager.Task::(_debug) Task 3e2b7762-dda8-493f-aa16-f5b9e0ebf198: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:48:39,557::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:48:39,559::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:48:39,560::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:48:39,561::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:48:39,562::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:48:39,563::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:48:39,564::task::495::TaskManager.Task::(_debug) Task 3e2b7762-dda8-493f-aa16-f5b9e0ebf198: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:48:39,565::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:48:39,843::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:48:40,411::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0814725 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15917019::DEBUG::2013-11-20 04:48:41,508::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('db6ed253-f76f-44bf-96e9-d268b0827271',) {}
>Thread-15917019::DEBUG::2013-11-20 04:48:41,509::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk stats not available
>Thread-15917019::DEBUG::2013-11-20 04:48:41,517::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk latency not available
>Thread-15917019::DEBUG::2013-11-20 04:48:41,518::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '14264', 'displayIp': '0', 'displayPort': u'5922', 'session': 'Unknown', 'displaySecurePort': u'5923', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet13': {'macAddr': u'00:1a:4a:a8:64:10', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet13'}}, 'vmId': 'db6ed253-f76f-44bf-96e9-d268b0827271', 'monitorResponse': '0', 'cpuUser': '53.19', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '75296145408', 'writeLatency': '0', 'imageID': '240c0593-fe33-4f8b-a029-097379dbcb85', 'flushLatency': '0', 'truesize': '75296145408'}}, 'boot': 'c', 'statsAge': '1.96', 'cpuIdle': '46.81', 'elapsedTime': '44', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-42838::DEBUG::2013-11-20 04:48:41,547::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:48:42,428::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-42838::DEBUG::2013-11-20 04:48:42,644::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:48:42,969::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0777707 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15917021::DEBUG::2013-11-20 04:48:43,547::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('db6ed253-f76f-44bf-96e9-d268b0827271',) {}
>Thread-15917021::DEBUG::2013-11-20 04:48:43,549::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk stats not available
>Thread-15917021::DEBUG::2013-11-20 04:48:43,550::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk latency not available
>Thread-15917021::DEBUG::2013-11-20 04:48:43,551::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '14264', 'displayIp': '0', 'displayPort': u'5922', 'session': 'Unknown', 'displaySecurePort': u'5923', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet13': {'macAddr': u'00:1a:4a:a8:64:10', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet13'}}, 'vmId': 'db6ed253-f76f-44bf-96e9-d268b0827271', 'monitorResponse': '0', 'cpuUser': '38.63', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '75296145408', 'writeLatency': '0', 'imageID': '240c0593-fe33-4f8b-a029-097379dbcb85', 'flushLatency': '0', 'truesize': '75296145408'}}, 'boot': 'c', 'statsAge': '2.00', 'cpuIdle': '61.37', 'elapsedTime': '46', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 04:48:44,984::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15917022::DEBUG::2013-11-20 04:48:45,251::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917022::INFO::2013-11-20 04:48:45,253::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917022::DEBUG::2013-11-20 04:48:45,253::task::495::TaskManager.Task::(_debug) Task 0478ab42-2471-42e5-a805-58a6f72faae1: moving from state init -> state preparing
>Thread-15917022::DEBUG::2013-11-20 04:48:45,254::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917022::DEBUG::2013-11-20 04:48:45,255::task::495::TaskManager.Task::(_debug) Task 0478ab42-2471-42e5-a805-58a6f72faae1: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917022::DEBUG::2013-11-20 04:48:45,256::task::495::TaskManager.Task::(_debug) Task 0478ab42-2471-42e5-a805-58a6f72faae1: moving from state preparing -> state finished
>Thread-15917022::DEBUG::2013-11-20 04:48:45,256::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917022::DEBUG::2013-11-20 04:48:45,257::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917022::DEBUG::2013-11-20 04:48:45,258::task::495::TaskManager.Task::(_debug) Task 0478ab42-2471-42e5-a805-58a6f72faae1: ref 0 aborting False
>Thread-15917022::INFO::2013-11-20 04:48:45,259::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917023::DEBUG::2013-11-20 04:48:45,265::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917023::INFO::2013-11-20 04:48:45,266::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917023::DEBUG::2013-11-20 04:48:45,267::task::495::TaskManager.Task::(_debug) Task 937da571-100c-4216-be5a-2cca4a6572fb: moving from state init -> state preparing
>Thread-15917023::DEBUG::2013-11-20 04:48:45,268::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d6d44aa8-889a-491a-8011-516129f2b33d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917023::DEBUG::2013-11-20 04:48:45,269::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917023::DEBUG::2013-11-20 04:48:45,270::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917023::DEBUG::2013-11-20 04:48:45,271::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d6d44aa8-889a-491a-8011-516129f2b33d`::Granted request
>Thread-15917023::DEBUG::2013-11-20 04:48:45,272::task::495::TaskManager.Task::(_debug) Task 937da571-100c-4216-be5a-2cca4a6572fb: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917023::DEBUG::2013-11-20 04:48:45,273::task::495::TaskManager.Task::(_debug) Task 937da571-100c-4216-be5a-2cca4a6572fb: ref 1 aborting False
>Thread-15917023::WARNING::2013-11-20 04:48:45,275::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917023::DEBUG::2013-11-20 04:48:45,276::task::495::TaskManager.Task::(_debug) Task 937da571-100c-4216-be5a-2cca4a6572fb: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917023::DEBUG::2013-11-20 04:48:45,277::task::495::TaskManager.Task::(_debug) Task 937da571-100c-4216-be5a-2cca4a6572fb: moving from state preparing -> state finished
>Thread-15917023::DEBUG::2013-11-20 04:48:45,278::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917023::DEBUG::2013-11-20 04:48:45,279::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917023::DEBUG::2013-11-20 04:48:45,279::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917023::DEBUG::2013-11-20 04:48:45,280::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917023::DEBUG::2013-11-20 04:48:45,281::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917023::DEBUG::2013-11-20 04:48:45,281::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917023::DEBUG::2013-11-20 04:48:45,282::task::495::TaskManager.Task::(_debug) Task 937da571-100c-4216-be5a-2cca4a6572fb: ref 0 aborting False
>Thread-15917023::INFO::2013-11-20 04:48:45,283::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:48:45,527::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0776488 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15917025::DEBUG::2013-11-20 04:48:45,610::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('db6ed253-f76f-44bf-96e9-d268b0827271',) {}
>Thread-15917025::DEBUG::2013-11-20 04:48:45,611::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk stats not available
>Thread-15917025::DEBUG::2013-11-20 04:48:45,612::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk latency not available
>Thread-15917025::DEBUG::2013-11-20 04:48:45,613::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '14264', 'displayIp': '0', 'displayPort': u'5922', 'session': 'Unknown', 'displaySecurePort': u'5923', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet13': {'macAddr': u'00:1a:4a:a8:64:10', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet13'}}, 'vmId': 'db6ed253-f76f-44bf-96e9-d268b0827271', 'monitorResponse': '0', 'cpuUser': '38.63', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '75296145408', 'writeLatency': '0', 'imageID': '240c0593-fe33-4f8b-a029-097379dbcb85', 'flushLatency': '0', 'truesize': '75296145408'}}, 'boot': 'c', 'statsAge': '1.84', 'cpuIdle': '61.37', 'elapsedTime': '48', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 04:48:47,547::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15917026::INFO::2013-11-20 04:48:47,788::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15917026::DEBUG::2013-11-20 04:48:47,789::task::495::TaskManager.Task::(_debug) Task 8c6df7f2-8117-481e-9553-cbb2168c1a1e: moving from state init -> state preparing
>Thread-15917026::DEBUG::2013-11-20 04:48:47,791::task::495::TaskManager.Task::(_debug) Task 8c6df7f2-8117-481e-9553-cbb2168c1a1e: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000349044799805', 'lastCheck': 1384922922.6547439, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00281286239624', 'lastCheck': 1384922924.63658, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00310206413269', 'lastCheck': 1384922924.7604849, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000473976135254', 'lastCheck': 1384922926.648649, 'valid': True, 'code': 0}}
>Thread-15917026::DEBUG::2013-11-20 04:48:47,792::task::495::TaskManager.Task::(_debug) Task 8c6df7f2-8117-481e-9553-cbb2168c1a1e: moving from state preparing -> state finished
>Thread-15917026::DEBUG::2013-11-20 04:48:47,792::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917026::DEBUG::2013-11-20 04:48:47,793::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917026::DEBUG::2013-11-20 04:48:47,794::task::495::TaskManager.Task::(_debug) Task 8c6df7f2-8117-481e-9553-cbb2168c1a1e: ref 0 aborting False
>Thread-15917026::INFO::2013-11-20 04:48:47,795::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000349044799805', 'lastCheck': 1384922922.6547439, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00281286239624', 'lastCheck': 1384922924.63658, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00310206413269', 'lastCheck': 1384922924.7604849, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000473976135254', 'lastCheck': 1384922926.648649, 'valid': True, 'code': 0}}
>Thread-15917027::DEBUG::2013-11-20 04:48:47,814::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk stats not available
>Thread-15917027::DEBUG::2013-11-20 04:48:47,815::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk latency not available
>Dummy-15912651::DEBUG::2013-11-20 04:48:48,106::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0786671 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 04:48:49,772::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:48:49,773::task::495::TaskManager.Task::(_debug) Task bce62b15-761d-442c-b97c-41812491a87f: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:48:49,775::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`3dbf8cfe-f698-403c-ae3a-9a1bc3abc86b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:48:49,775::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:48:49,776::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:48:49,777::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`3dbf8cfe-f698-403c-ae3a-9a1bc3abc86b`::Granted request
>Thread-15914519::INFO::2013-11-20 04:48:49,786::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:48:49,788::task::495::TaskManager.Task::(_debug) Task eba7c548-1cd1-488b-9db8-ca153ff0347d: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:48:49,798::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8d2fcd4b-0510-4d88-ad2b-1e7d2183e3fa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:48:49,800::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:48:49,801::task::495::TaskManager.Task::(_debug) Task bce62b15-761d-442c-b97c-41812491a87f: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15914519::DEBUG::2013-11-20 04:48:49,802::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:48:49,802::task::495::TaskManager.Task::(_debug) Task bce62b15-761d-442c-b97c-41812491a87f: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:48:49,803::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8d2fcd4b-0510-4d88-ad2b-1e7d2183e3fa`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:48:49,804::task::495::TaskManager.Task::(_debug) Task bce62b15-761d-442c-b97c-41812491a87f: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 04:48:49,806::task::495::TaskManager.Task::(_debug) Task bce62b15-761d-442c-b97c-41812491a87f: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:48:49,808::task::495::TaskManager.Task::(_debug) Task eba7c548-1cd1-488b-9db8-ca153ff0347d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913546::DEBUG::2013-11-20 04:48:49,808::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:48:49,809::task::495::TaskManager.Task::(_debug) Task eba7c548-1cd1-488b-9db8-ca153ff0347d: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:48:49,810::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:48:49,810::task::495::TaskManager.Task::(_debug) Task eba7c548-1cd1-488b-9db8-ca153ff0347d: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:48:49,812::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15914519::DEBUG::2013-11-20 04:48:49,812::task::495::TaskManager.Task::(_debug) Task eba7c548-1cd1-488b-9db8-ca153ff0347d: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:48:49,814::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:48:49,814::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:48:49,815::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:48:49,815::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:48:49,816::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:48:49,820::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913546::DEBUG::2013-11-20 04:48:49,821::task::495::TaskManager.Task::(_debug) Task bce62b15-761d-442c-b97c-41812491a87f: ref 0 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:48:49,821::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913546::INFO::2013-11-20 04:48:49,822::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:48:49,823::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913546::INFO::2013-11-20 04:48:49,824::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15914519::DEBUG::2013-11-20 04:48:49,824::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:48:49,825::task::495::TaskManager.Task::(_debug) Task 1f7f061e-8eec-44ef-9d45-f8fbd97a9f85: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:48:49,826::task::495::TaskManager.Task::(_debug) Task eba7c548-1cd1-488b-9db8-ca153ff0347d: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:48:49,827::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`4eec0f66-5289-4db2-809a-f83604fd5381`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::INFO::2013-11-20 04:48:49,828::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:48:49,829::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:48:49,831::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:48:49,832::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`4eec0f66-5289-4db2-809a-f83604fd5381`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:48:49,834::task::495::TaskManager.Task::(_debug) Task 1f7f061e-8eec-44ef-9d45-f8fbd97a9f85: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:48:49,835::task::495::TaskManager.Task::(_debug) Task 1f7f061e-8eec-44ef-9d45-f8fbd97a9f85: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:48:49,837::task::495::TaskManager.Task::(_debug) Task 1f7f061e-8eec-44ef-9d45-f8fbd97a9f85: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:48:49,837::task::495::TaskManager.Task::(_debug) Task 1f7f061e-8eec-44ef-9d45-f8fbd97a9f85: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:48:49,838::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:48:49,839::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:48:49,840::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:48:49,841::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:48:49,841::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:48:49,842::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:48:49,843::task::495::TaskManager.Task::(_debug) Task 1f7f061e-8eec-44ef-9d45-f8fbd97a9f85: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:48:49,844::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:48:49,845::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:48:49,845::task::495::TaskManager.Task::(_debug) Task e364c836-07e4-4db4-a155-767f1590d93e: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:48:49,847::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`dffb9173-e361-4d70-b7da-5ba84d570afc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:48:49,848::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:48:49,849::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:48:49,849::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`dffb9173-e361-4d70-b7da-5ba84d570afc`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:48:49,851::task::495::TaskManager.Task::(_debug) Task e364c836-07e4-4db4-a155-767f1590d93e: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:48:49,851::task::495::TaskManager.Task::(_debug) Task e364c836-07e4-4db4-a155-767f1590d93e: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:48:49,852::task::495::TaskManager.Task::(_debug) Task e364c836-07e4-4db4-a155-767f1590d93e: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:48:49,853::task::495::TaskManager.Task::(_debug) Task e364c836-07e4-4db4-a155-767f1590d93e: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:48:49,853::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:48:49,854::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:48:49,855::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:48:49,856::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:48:49,857::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:48:49,857::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:48:49,858::task::495::TaskManager.Task::(_debug) Task e364c836-07e4-4db4-a155-767f1590d93e: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:48:49,859::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:48:49,860::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:48:49,861::task::495::TaskManager.Task::(_debug) Task 15151f94-7710-4f90-b04a-093c80fd0b62: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:48:49,863::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a81b1b7e-b608-4e38-a38a-8febbde73d25`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:48:49,863::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:48:49,864::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:48:49,865::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a81b1b7e-b608-4e38-a38a-8febbde73d25`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:48:49,866::task::495::TaskManager.Task::(_debug) Task 15151f94-7710-4f90-b04a-093c80fd0b62: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:48:49,867::task::495::TaskManager.Task::(_debug) Task 15151f94-7710-4f90-b04a-093c80fd0b62: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:48:49,867::task::495::TaskManager.Task::(_debug) Task 15151f94-7710-4f90-b04a-093c80fd0b62: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:48:49,868::task::495::TaskManager.Task::(_debug) Task 15151f94-7710-4f90-b04a-093c80fd0b62: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:48:49,869::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:48:49,869::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:48:49,870::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:48:49,871::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:48:49,872::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:48:49,872::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:48:49,873::task::495::TaskManager.Task::(_debug) Task 15151f94-7710-4f90-b04a-093c80fd0b62: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:48:49,874::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:48:50,120::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15917029::DEBUG::2013-11-20 04:48:50,357::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('db6ed253-f76f-44bf-96e9-d268b0827271',) {}
>Thread-15917029::DEBUG::2013-11-20 04:48:50,359::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk stats not available
>Thread-15917029::DEBUG::2013-11-20 04:48:50,360::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk latency not available
>Thread-15917029::DEBUG::2013-11-20 04:48:50,361::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '14264', 'displayIp': '0', 'displayPort': u'5922', 'session': 'Unknown', 'displaySecurePort': u'5923', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet13': {'macAddr': u'00:1a:4a:a8:64:10', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet13'}}, 'vmId': 'db6ed253-f76f-44bf-96e9-d268b0827271', 'monitorResponse': '0', 'cpuUser': '38.63', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '75296145408', 'writeLatency': '0', 'imageID': '240c0593-fe33-4f8b-a029-097379dbcb85', 'flushLatency': '0', 'truesize': '75296145408'}}, 'boot': 'c', 'statsAge': '0.58', 'cpuIdle': '61.37', 'elapsedTime': '53', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 04:48:50,671::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0777785 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15917031::DEBUG::2013-11-20 04:48:52,423::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('db6ed253-f76f-44bf-96e9-d268b0827271',) {}
>Thread-15917031::DEBUG::2013-11-20 04:48:52,423::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk stats not available
>Thread-15917031::DEBUG::2013-11-20 04:48:52,424::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk latency not available
>Thread-15917031::DEBUG::2013-11-20 04:48:52,424::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '14264', 'displayIp': '0', 'displayPort': u'5922', 'session': 'Unknown', 'displaySecurePort': u'5923', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet13': {'macAddr': u'00:1a:4a:a8:64:10', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet13'}}, 'vmId': 'db6ed253-f76f-44bf-96e9-d268b0827271', 'monitorResponse': '0', 'cpuUser': '38.63', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '75296145408', 'writeLatency': '0', 'imageID': '240c0593-fe33-4f8b-a029-097379dbcb85', 'flushLatency': '0', 'truesize': '75296145408'}}, 'boot': 'c', 'statsAge': '0.64', 'cpuIdle': '61.37', 'elapsedTime': '55', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 04:48:52,686::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:48:53,161::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0614287 s, 16.7 MB/s\n'; <rc> = 0
>Thread-15917033::DEBUG::2013-11-20 04:48:54,515::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('db6ed253-f76f-44bf-96e9-d268b0827271',) {}
>Thread-15917033::DEBUG::2013-11-20 04:48:54,516::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk stats not available
>Thread-15917033::DEBUG::2013-11-20 04:48:54,517::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk latency not available
>Thread-15917033::DEBUG::2013-11-20 04:48:54,518::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '14264', 'displayIp': '0', 'displayPort': u'5922', 'session': 'Unknown', 'displaySecurePort': u'5923', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet13': {'macAddr': u'00:1a:4a:a8:64:10', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet13'}}, 'vmId': 'db6ed253-f76f-44bf-96e9-d268b0827271', 'monitorResponse': '0', 'cpuUser': '38.63', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '75296145408', 'writeLatency': '0', 'imageID': '240c0593-fe33-4f8b-a029-097379dbcb85', 'flushLatency': '0', 'truesize': '75296145408'}}, 'boot': 'c', 'statsAge': '0.73', 'cpuIdle': '61.37', 'elapsedTime': '57', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 04:48:55,172::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15917034::DEBUG::2013-11-20 04:48:55,409::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917034::INFO::2013-11-20 04:48:55,411::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917034::DEBUG::2013-11-20 04:48:55,412::task::495::TaskManager.Task::(_debug) Task b4076b89-d492-4b17-9adb-52b974ca40f5: moving from state init -> state preparing
>Thread-15917034::DEBUG::2013-11-20 04:48:55,413::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917034::DEBUG::2013-11-20 04:48:55,414::task::495::TaskManager.Task::(_debug) Task b4076b89-d492-4b17-9adb-52b974ca40f5: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917034::DEBUG::2013-11-20 04:48:55,415::task::495::TaskManager.Task::(_debug) Task b4076b89-d492-4b17-9adb-52b974ca40f5: moving from state preparing -> state finished
>Thread-15917034::DEBUG::2013-11-20 04:48:55,415::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917034::DEBUG::2013-11-20 04:48:55,416::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917034::DEBUG::2013-11-20 04:48:55,416::task::495::TaskManager.Task::(_debug) Task b4076b89-d492-4b17-9adb-52b974ca40f5: ref 0 aborting False
>Thread-15917034::INFO::2013-11-20 04:48:55,417::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917035::DEBUG::2013-11-20 04:48:55,423::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917035::INFO::2013-11-20 04:48:55,424::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917035::DEBUG::2013-11-20 04:48:55,424::task::495::TaskManager.Task::(_debug) Task 2b435291-6f3c-486d-8767-80e255ac7424: moving from state init -> state preparing
>Thread-15917035::DEBUG::2013-11-20 04:48:55,426::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2029e4d6-61f4-4cdf-9e67-7cebd92b208d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917035::DEBUG::2013-11-20 04:48:55,426::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917035::DEBUG::2013-11-20 04:48:55,427::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917035::DEBUG::2013-11-20 04:48:55,428::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2029e4d6-61f4-4cdf-9e67-7cebd92b208d`::Granted request
>Thread-15917035::DEBUG::2013-11-20 04:48:55,429::task::495::TaskManager.Task::(_debug) Task 2b435291-6f3c-486d-8767-80e255ac7424: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917035::DEBUG::2013-11-20 04:48:55,430::task::495::TaskManager.Task::(_debug) Task 2b435291-6f3c-486d-8767-80e255ac7424: ref 1 aborting False
>Thread-15917035::WARNING::2013-11-20 04:48:55,433::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917035::DEBUG::2013-11-20 04:48:55,434::task::495::TaskManager.Task::(_debug) Task 2b435291-6f3c-486d-8767-80e255ac7424: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917035::DEBUG::2013-11-20 04:48:55,436::task::495::TaskManager.Task::(_debug) Task 2b435291-6f3c-486d-8767-80e255ac7424: moving from state preparing -> state finished
>Thread-15917035::DEBUG::2013-11-20 04:48:55,436::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917035::DEBUG::2013-11-20 04:48:55,437::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917035::DEBUG::2013-11-20 04:48:55,438::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917035::DEBUG::2013-11-20 04:48:55,439::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917035::DEBUG::2013-11-20 04:48:55,439::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917035::DEBUG::2013-11-20 04:48:55,440::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917035::DEBUG::2013-11-20 04:48:55,441::task::495::TaskManager.Task::(_debug) Task 2b435291-6f3c-486d-8767-80e255ac7424: ref 0 aborting False
>Thread-15917035::INFO::2013-11-20 04:48:55,441::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '134046810112', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:48:55,710::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0753591 s, 13.6 MB/s\n'; <rc> = 0
>Thread-15917037::DEBUG::2013-11-20 04:48:56,554::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('db6ed253-f76f-44bf-96e9-d268b0827271',) {}
>Thread-15917037::DEBUG::2013-11-20 04:48:56,555::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk stats not available
>Thread-15917037::DEBUG::2013-11-20 04:48:56,556::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk latency not available
>Thread-15917037::DEBUG::2013-11-20 04:48:56,557::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'username': 'Unknown', 'memUsage': '0', 'acpiEnable': 'true', 'pid': '14264', 'displayIp': '0', 'displayPort': u'5922', 'session': 'Unknown', 'displaySecurePort': u'5923', 'timeOffset': '0', 'pauseCode': 'NOERR', 'clientIp': '', 'kvmEnable': 'true', 'network': {u'vnet13': {'macAddr': u'00:1a:4a:a8:64:10', 'rxDropped': '0', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet13'}}, 'vmId': 'db6ed253-f76f-44bf-96e9-d268b0827271', 'monitorResponse': '0', 'cpuUser': '38.63', 'disks': {u'hda': {'readLatency': '0', 'apparentsize': '75296145408', 'writeLatency': '0', 'imageID': '240c0593-fe33-4f8b-a029-097379dbcb85', 'flushLatency': '0', 'truesize': '75296145408'}}, 'boot': 'c', 'statsAge': '0.77', 'cpuIdle': '61.37', 'elapsedTime': '59', 'vmType': 'kvm', 'cpuSys': '0.00', 'appsList': [], 'guestIPs': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15914435::INFO::2013-11-20 04:48:56,929::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:48:56,931::task::495::TaskManager.Task::(_debug) Task 565c9988-488c-466a-ac04-c030dd4a129e: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:48:56,934::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`da6534a4-dacf-4e8a-a5c0-35b4e34a8a63`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:48:56,938::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:48:56,939::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:48:56,944::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`da6534a4-dacf-4e8a-a5c0-35b4e34a8a63`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:48:56,951::task::495::TaskManager.Task::(_debug) Task 565c9988-488c-466a-ac04-c030dd4a129e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:48:56,955::task::495::TaskManager.Task::(_debug) Task 565c9988-488c-466a-ac04-c030dd4a129e: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:48:56,957::task::495::TaskManager.Task::(_debug) Task 565c9988-488c-466a-ac04-c030dd4a129e: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:48:56,959::task::495::TaskManager.Task::(_debug) Task 565c9988-488c-466a-ac04-c030dd4a129e: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:48:56,964::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:48:56,967::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:48:56,972::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:48:56,975::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:48:56,975::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:48:56,976::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:48:56,977::task::495::TaskManager.Task::(_debug) Task 565c9988-488c-466a-ac04-c030dd4a129e: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:48:56,977::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:48:57,725::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:48:58,321::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0909743 s, 11.3 MB/s\n'; <rc> = 0
>Thread-15917038::INFO::2013-11-20 04:48:58,602::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15917038::DEBUG::2013-11-20 04:48:58,603::task::495::TaskManager.Task::(_debug) Task 56db6d39-23d7-486d-839c-7a0b296c2413: moving from state init -> state preparing
>Thread-15917038::DEBUG::2013-11-20 04:48:58,603::task::495::TaskManager.Task::(_debug) Task 56db6d39-23d7-486d-839c-7a0b296c2413: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000362873077393', 'lastCheck': 1384922932.9091461, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00140690803528', 'lastCheck': 1384922934.904623, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00120306015015', 'lastCheck': 1384922934.9049611, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0065610408783', 'lastCheck': 1384922936.9116211, 'valid': True, 'code': 0}}
>Thread-15917038::DEBUG::2013-11-20 04:48:58,604::task::495::TaskManager.Task::(_debug) Task 56db6d39-23d7-486d-839c-7a0b296c2413: moving from state preparing -> state finished
>Thread-15917038::DEBUG::2013-11-20 04:48:58,605::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917038::DEBUG::2013-11-20 04:48:58,605::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917038::DEBUG::2013-11-20 04:48:58,606::task::495::TaskManager.Task::(_debug) Task 56db6d39-23d7-486d-839c-7a0b296c2413: ref 0 aborting False
>Thread-15917038::INFO::2013-11-20 04:48:58,607::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000362873077393', 'lastCheck': 1384922932.9091461, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00140690803528', 'lastCheck': 1384922934.904623, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00120306015015', 'lastCheck': 1384922934.9049611, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0065610408783', 'lastCheck': 1384922936.9116211, 'valid': True, 'code': 0}}
>Thread-15917039::DEBUG::2013-11-20 04:48:58,656::libvirtvm::222::vm.Vm::(_getDiskStats) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk stats not available
>Thread-15917039::DEBUG::2013-11-20 04:48:58,657::libvirtvm::251::vm.Vm::(_getDiskLatency) vmId=`db6ed253-f76f-44bf-96e9-d268b0827271`::Disk latency not available
>Dummy-15912651::DEBUG::2013-11-20 04:49:00,335::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:49:00,944::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0981176 s, 10.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:49:02,970::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:49:03,539::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0801179 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:49:05,555::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:49:06,108::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.104537 s, 9.8 MB/s\n'; <rc> = 0
>Thread-15913639::INFO::2013-11-20 04:49:07,979::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:49:07,980::task::495::TaskManager.Task::(_debug) Task e414ece5-cbe1-4434-ab3f-e8618b4480dc: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:49:07,982::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2d349bb9-667e-422c-8900-c435fd36a9de`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:49:07,983::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:49:07,984::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:49:07,984::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2d349bb9-667e-422c-8900-c435fd36a9de`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:49:07,985::task::495::TaskManager.Task::(_debug) Task e414ece5-cbe1-4434-ab3f-e8618b4480dc: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:49:07,986::task::495::TaskManager.Task::(_debug) Task e414ece5-cbe1-4434-ab3f-e8618b4480dc: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:49:07,987::task::495::TaskManager.Task::(_debug) Task e414ece5-cbe1-4434-ab3f-e8618b4480dc: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:49:07,988::task::495::TaskManager.Task::(_debug) Task e414ece5-cbe1-4434-ab3f-e8618b4480dc: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:49:07,988::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:49:07,989::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:49:07,990::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:49:07,990::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:49:07,991::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:49:07,991::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:49:07,992::task::495::TaskManager.Task::(_debug) Task e414ece5-cbe1-4434-ab3f-e8618b4480dc: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:49:07,993::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:49:08,124::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:49:08,675::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0856837 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:49:10,689::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915918::INFO::2013-11-20 04:49:10,885::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:49:10,887::task::495::TaskManager.Task::(_debug) Task 0dd20469-e89b-4dd7-9b9e-9ccc962bf2d6: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:49:10,890::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7c250f19-0776-46a7-aebd-a940e89723f1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:49:10,891::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:49:10,892::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:49:10,893::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7c250f19-0776-46a7-aebd-a940e89723f1`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:49:10,895::task::495::TaskManager.Task::(_debug) Task 0dd20469-e89b-4dd7-9b9e-9ccc962bf2d6: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:49:10,895::task::495::TaskManager.Task::(_debug) Task 0dd20469-e89b-4dd7-9b9e-9ccc962bf2d6: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:49:10,896::task::495::TaskManager.Task::(_debug) Task 0dd20469-e89b-4dd7-9b9e-9ccc962bf2d6: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:49:10,897::task::495::TaskManager.Task::(_debug) Task 0dd20469-e89b-4dd7-9b9e-9ccc962bf2d6: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:49:10,900::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:49:10,902::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:49:10,903::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:49:10,904::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:49:10,904::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:49:10,905::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:49:10,906::task::495::TaskManager.Task::(_debug) Task 0dd20469-e89b-4dd7-9b9e-9ccc962bf2d6: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:49:10,906::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:49:11,227::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0753651 s, 13.6 MB/s\n'; <rc> = 0
>Thread-15916984::INFO::2013-11-20 04:49:11,964::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=240c0593-fe33-4f8b-a029-097379dbcb85 volUUID=f715faaa-081a-4598-b125-80a2d12af554)
>Thread-15916984::DEBUG::2013-11-20 04:49:11,974::task::495::TaskManager.Task::(_debug) Task 2843657c-1888-470e-a0f8-37feae56f7b7: moving from state init -> state preparing
>Thread-15916984::DEBUG::2013-11-20 04:49:11,980::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c2516176-d80b-4b9d-bdf9-ae4cce7244b6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15916984::DEBUG::2013-11-20 04:49:11,981::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15916984::DEBUG::2013-11-20 04:49:11,982::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15916984::DEBUG::2013-11-20 04:49:11,982::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c2516176-d80b-4b9d-bdf9-ae4cce7244b6`::Granted request
>Thread-15916984::DEBUG::2013-11-20 04:49:11,984::task::495::TaskManager.Task::(_debug) Task 2843657c-1888-470e-a0f8-37feae56f7b7: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15916984::DEBUG::2013-11-20 04:49:11,985::task::495::TaskManager.Task::(_debug) Task 2843657c-1888-470e-a0f8-37feae56f7b7: ref 1 aborting False
>Thread-15916984::DEBUG::2013-11-20 04:49:11,990::task::495::TaskManager.Task::(_debug) Task 2843657c-1888-470e-a0f8-37feae56f7b7: finished: {'truesize': '75296145408', 'apparentsize': '75296145408'}
>Thread-15916984::DEBUG::2013-11-20 04:49:11,991::task::495::TaskManager.Task::(_debug) Task 2843657c-1888-470e-a0f8-37feae56f7b7: moving from state preparing -> state finished
>Thread-15916984::DEBUG::2013-11-20 04:49:11,995::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15916984::DEBUG::2013-11-20 04:49:12,003::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916984::DEBUG::2013-11-20 04:49:12,006::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15916984::DEBUG::2013-11-20 04:49:12,009::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15916984::DEBUG::2013-11-20 04:49:12,011::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15916984::DEBUG::2013-11-20 04:49:12,012::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15916984::DEBUG::2013-11-20 04:49:12,013::task::495::TaskManager.Task::(_debug) Task 2843657c-1888-470e-a0f8-37feae56f7b7: ref 0 aborting False
>Thread-15916984::INFO::2013-11-20 04:49:12,013::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '75296145408', 'apparentsize': '75296145408'}
>Dummy-15912651::DEBUG::2013-11-20 04:49:13,243::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:49:13,768::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0749347 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:49:15,784::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915863::INFO::2013-11-20 04:49:16,000::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:49:16,001::task::495::TaskManager.Task::(_debug) Task b111e0aa-fb8e-4ca9-b664-d22d16f73f70: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:49:16,003::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a681b24f-c199-4627-a75a-dd69dbc1704e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:49:16,004::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:49:16,005::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:49:16,005::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a681b24f-c199-4627-a75a-dd69dbc1704e`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:49:16,006::task::495::TaskManager.Task::(_debug) Task b111e0aa-fb8e-4ca9-b664-d22d16f73f70: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:49:16,006::task::495::TaskManager.Task::(_debug) Task b111e0aa-fb8e-4ca9-b664-d22d16f73f70: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:49:16,007::task::495::TaskManager.Task::(_debug) Task b111e0aa-fb8e-4ca9-b664-d22d16f73f70: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:49:16,007::task::495::TaskManager.Task::(_debug) Task b111e0aa-fb8e-4ca9-b664-d22d16f73f70: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:49:16,008::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:49:16,008::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:49:16,009::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:49:16,009::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:49:16,010::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:49:16,010::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:49:16,010::task::495::TaskManager.Task::(_debug) Task b111e0aa-fb8e-4ca9-b664-d22d16f73f70: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:49:16,011::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:49:16,351::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0966568 s, 10.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:49:18,368::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:49:18,933::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0819661 s, 12.5 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 04:49:19,990::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:49:19,991::task::495::TaskManager.Task::(_debug) Task 67a0aee6-b48c-4e49-afc0-9fd9e67e5366: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:49:19,992::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`51a6a084-e499-4d8c-b145-43a2f85b059f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:49:19,993::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:49:19,994::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:49:19,995::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`51a6a084-e499-4d8c-b145-43a2f85b059f`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:49:19,997::task::495::TaskManager.Task::(_debug) Task 67a0aee6-b48c-4e49-afc0-9fd9e67e5366: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:49:19,999::task::495::TaskManager.Task::(_debug) Task 67a0aee6-b48c-4e49-afc0-9fd9e67e5366: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:49:20,001::task::495::TaskManager.Task::(_debug) Task 67a0aee6-b48c-4e49-afc0-9fd9e67e5366: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:49:20,003::task::495::TaskManager.Task::(_debug) Task 67a0aee6-b48c-4e49-afc0-9fd9e67e5366: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:49:20,004::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:49:20,005::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:49:20,005::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:49:20,006::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:49:20,007::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:49:20,007::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:49:20,008::task::495::TaskManager.Task::(_debug) Task 67a0aee6-b48c-4e49-afc0-9fd9e67e5366: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:49:20,009::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:49:20,943::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:49:21,491::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0801628 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:49:23,512::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:49:24,049::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0860625 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:49:26,070::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:49:26,611::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0863834 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:49:28,625::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913518::INFO::2013-11-20 04:49:28,858::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:49:28,859::task::495::TaskManager.Task::(_debug) Task ceb5b7f6-7b44-40c4-a6c0-fb46c504f89d: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:49:28,862::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6b2d9ccd-73c1-477e-97c1-90f99000e6d0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:49:28,863::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:49:28,864::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:49:28,865::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6b2d9ccd-73c1-477e-97c1-90f99000e6d0`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:49:28,866::task::495::TaskManager.Task::(_debug) Task ceb5b7f6-7b44-40c4-a6c0-fb46c504f89d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:49:28,867::task::495::TaskManager.Task::(_debug) Task ceb5b7f6-7b44-40c4-a6c0-fb46c504f89d: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:49:28,868::task::495::TaskManager.Task::(_debug) Task ceb5b7f6-7b44-40c4-a6c0-fb46c504f89d: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:49:28,868::task::495::TaskManager.Task::(_debug) Task ceb5b7f6-7b44-40c4-a6c0-fb46c504f89d: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:49:28,869::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:49:28,870::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:49:28,871::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:49:28,871::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:49:28,872::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:49:28,872::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:49:28,873::task::495::TaskManager.Task::(_debug) Task ceb5b7f6-7b44-40c4-a6c0-fb46c504f89d: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:49:28,874::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:49:28,875::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:49:28,876::task::495::TaskManager.Task::(_debug) Task a10e14fe-28bf-41c6-8fcc-50d09e22af09: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:49:28,877::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`94d15aea-b9a6-4fd6-94c8-a5ab56abd0ad`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:49:28,878::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:49:28,878::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:49:28,879::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`94d15aea-b9a6-4fd6-94c8-a5ab56abd0ad`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:49:28,880::task::495::TaskManager.Task::(_debug) Task a10e14fe-28bf-41c6-8fcc-50d09e22af09: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:49:28,880::task::495::TaskManager.Task::(_debug) Task a10e14fe-28bf-41c6-8fcc-50d09e22af09: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:49:28,881::task::495::TaskManager.Task::(_debug) Task a10e14fe-28bf-41c6-8fcc-50d09e22af09: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:49:28,882::task::495::TaskManager.Task::(_debug) Task a10e14fe-28bf-41c6-8fcc-50d09e22af09: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:49:28,882::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:49:28,883::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:49:28,883::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:49:28,884::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:49:28,884::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:49:28,885::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:49:28,886::task::495::TaskManager.Task::(_debug) Task a10e14fe-28bf-41c6-8fcc-50d09e22af09: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:49:28,886::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:49:29,175::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0809304 s, 12.7 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 04:49:30,003::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:49:30,004::task::495::TaskManager.Task::(_debug) Task 51385211-5278-4227-bc61-6c9f404250e8: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:49:30,006::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`79685f44-c093-4228-8aa8-720a9a41b428`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:49:30,006::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:49:30,007::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:49:30,008::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`79685f44-c093-4228-8aa8-720a9a41b428`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:49:30,010::task::495::TaskManager.Task::(_debug) Task 51385211-5278-4227-bc61-6c9f404250e8: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:49:30,010::task::495::TaskManager.Task::(_debug) Task 51385211-5278-4227-bc61-6c9f404250e8: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:49:30,011::task::495::TaskManager.Task::(_debug) Task 51385211-5278-4227-bc61-6c9f404250e8: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:49:30,012::task::495::TaskManager.Task::(_debug) Task 51385211-5278-4227-bc61-6c9f404250e8: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:49:30,013::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:49:30,013::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:49:30,014::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:49:30,014::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:49:30,015::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:49:30,016::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:49:30,016::task::495::TaskManager.Task::(_debug) Task 51385211-5278-4227-bc61-6c9f404250e8: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:49:30,017::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:49:31,188::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:49:31,718::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0759343 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:49:33,733::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:49:34,305::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0800343 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:49:36,317::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:49:36,847::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0763912 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:49:38,862::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:49:39,432::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0810156 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15913695::INFO::2013-11-20 04:49:40,005::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:49:40,006::task::495::TaskManager.Task::(_debug) Task 08e1fc5c-9eae-49c9-bdc7-c067a1ac75a8: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:49:40,008::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`70845252-87f9-4baf-be48-23441ff9ffc3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:49:40,009::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:49:40,010::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:49:40,010::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`70845252-87f9-4baf-be48-23441ff9ffc3`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:49:40,011::task::495::TaskManager.Task::(_debug) Task 08e1fc5c-9eae-49c9-bdc7-c067a1ac75a8: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:49:40,012::task::495::TaskManager.Task::(_debug) Task 08e1fc5c-9eae-49c9-bdc7-c067a1ac75a8: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:49:40,013::task::495::TaskManager.Task::(_debug) Task 08e1fc5c-9eae-49c9-bdc7-c067a1ac75a8: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:49:40,013::task::495::TaskManager.Task::(_debug) Task 08e1fc5c-9eae-49c9-bdc7-c067a1ac75a8: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:49:40,014::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:49:40,015::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:49:40,015::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:49:40,016::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:49:40,017::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:49:40,018::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:49:40,019::task::495::TaskManager.Task::(_debug) Task 08e1fc5c-9eae-49c9-bdc7-c067a1ac75a8: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:49:40,019::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15914453::INFO::2013-11-20 04:49:40,029::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:49:40,029::task::495::TaskManager.Task::(_debug) Task 710c5591-eba3-4362-acf3-feb4441af65c: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:49:40,031::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ee30e716-a975-49c1-a057-9d483700ca07`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:49:40,032::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:49:40,033::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:49:40,034::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ee30e716-a975-49c1-a057-9d483700ca07`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:49:40,035::task::495::TaskManager.Task::(_debug) Task 710c5591-eba3-4362-acf3-feb4441af65c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:49:40,035::task::495::TaskManager.Task::(_debug) Task 710c5591-eba3-4362-acf3-feb4441af65c: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:49:40,036::task::495::TaskManager.Task::(_debug) Task 710c5591-eba3-4362-acf3-feb4441af65c: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:49:40,037::task::495::TaskManager.Task::(_debug) Task 710c5591-eba3-4362-acf3-feb4441af65c: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:49:40,038::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:49:40,042::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:49:40,043::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:49:40,044::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:49:40,046::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:49:40,047::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:49:40,049::task::495::TaskManager.Task::(_debug) Task 710c5591-eba3-4362-acf3-feb4441af65c: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:49:40,051::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:49:41,448::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:49:41,993::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0823545 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:49:44,013::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:49:44,606::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0834259 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:49:46,621::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:49:47,175::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0786555 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:49:49,189::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:49:49,712::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0811857 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15914519::INFO::2013-11-20 04:49:50,222::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:49:50,223::task::495::TaskManager.Task::(_debug) Task fb14ebca-9479-45ce-8fd8-5a10a35e0ec5: moving from state init -> state preparing
>Thread-15913546::INFO::2013-11-20 04:49:50,228::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15914519::DEBUG::2013-11-20 04:49:50,229::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f58b456a-e8fb-4780-9185-506da59807bf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:49:50,230::task::495::TaskManager.Task::(_debug) Task eb64426e-159d-495d-8a60-8bc01bc116d0: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:49:50,233::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:49:50,236::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a7e73345-6d17-4749-a7cf-2b0e03f02a86`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:49:50,238::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:49:50,239::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:49:50,240::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f58b456a-e8fb-4780-9185-506da59807bf`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:49:50,241::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:49:50,242::task::495::TaskManager.Task::(_debug) Task fb14ebca-9479-45ce-8fd8-5a10a35e0ec5: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913546::DEBUG::2013-11-20 04:49:50,242::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a7e73345-6d17-4749-a7cf-2b0e03f02a86`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:49:50,243::task::495::TaskManager.Task::(_debug) Task fb14ebca-9479-45ce-8fd8-5a10a35e0ec5: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:49:50,244::task::495::TaskManager.Task::(_debug) Task eb64426e-159d-495d-8a60-8bc01bc116d0: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15914519::DEBUG::2013-11-20 04:49:50,245::task::495::TaskManager.Task::(_debug) Task fb14ebca-9479-45ce-8fd8-5a10a35e0ec5: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:49:50,246::task::495::TaskManager.Task::(_debug) Task eb64426e-159d-495d-8a60-8bc01bc116d0: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:49:50,247::task::495::TaskManager.Task::(_debug) Task fb14ebca-9479-45ce-8fd8-5a10a35e0ec5: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:49:50,248::task::495::TaskManager.Task::(_debug) Task eb64426e-159d-495d-8a60-8bc01bc116d0: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:49:50,249::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:49:50,250::task::495::TaskManager.Task::(_debug) Task eb64426e-159d-495d-8a60-8bc01bc116d0: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:49:50,251::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:49:50,252::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:49:50,254::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913546::DEBUG::2013-11-20 04:49:50,255::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:49:50,257::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:49:50,257::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15914519::DEBUG::2013-11-20 04:49:50,258::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:49:50,259::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:49:50,260::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:49:50,261::task::495::TaskManager.Task::(_debug) Task fb14ebca-9479-45ce-8fd8-5a10a35e0ec5: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:49:50,262::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15914519::INFO::2013-11-20 04:49:50,263::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:49:50,264::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:49:50,266::task::495::TaskManager.Task::(_debug) Task eb64426e-159d-495d-8a60-8bc01bc116d0: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:49:50,266::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:49:50,268::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:49:50,269::task::495::TaskManager.Task::(_debug) Task 3b9ea208-4040-46fd-b1fe-2260d13b0242: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:49:50,270::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`bf729f12-0629-40c5-b148-84ab4edd53cf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:49:50,271::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:49:50,272::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:49:50,273::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`bf729f12-0629-40c5-b148-84ab4edd53cf`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:49:50,274::task::495::TaskManager.Task::(_debug) Task 3b9ea208-4040-46fd-b1fe-2260d13b0242: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:49:50,274::task::495::TaskManager.Task::(_debug) Task 3b9ea208-4040-46fd-b1fe-2260d13b0242: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:49:50,275::task::495::TaskManager.Task::(_debug) Task 3b9ea208-4040-46fd-b1fe-2260d13b0242: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:49:50,275::task::495::TaskManager.Task::(_debug) Task 3b9ea208-4040-46fd-b1fe-2260d13b0242: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:49:50,276::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:49:50,277::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:49:50,278::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:49:50,279::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:49:50,279::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:49:50,280::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:49:50,280::task::495::TaskManager.Task::(_debug) Task 3b9ea208-4040-46fd-b1fe-2260d13b0242: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:49:50,281::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:49:50,282::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:49:50,283::task::495::TaskManager.Task::(_debug) Task 73ab6c65-298a-4c09-9730-364407db9d90: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:49:50,284::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`1d698118-bfda-4613-b98e-c8c87dbd8843`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:49:50,285::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:49:50,286::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:49:50,286::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`1d698118-bfda-4613-b98e-c8c87dbd8843`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:49:50,287::task::495::TaskManager.Task::(_debug) Task 73ab6c65-298a-4c09-9730-364407db9d90: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:49:50,288::task::495::TaskManager.Task::(_debug) Task 73ab6c65-298a-4c09-9730-364407db9d90: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:49:50,288::task::495::TaskManager.Task::(_debug) Task 73ab6c65-298a-4c09-9730-364407db9d90: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:49:50,293::task::495::TaskManager.Task::(_debug) Task 73ab6c65-298a-4c09-9730-364407db9d90: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:49:50,294::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:49:50,295::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:49:50,296::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:49:50,297::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:49:50,298::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:49:50,298::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:49:50,299::task::495::TaskManager.Task::(_debug) Task 73ab6c65-298a-4c09-9730-364407db9d90: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:49:50,300::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:49:50,301::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:49:50,302::task::495::TaskManager.Task::(_debug) Task 6df64c55-3f31-42df-bfac-c9c754f3f6fe: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:49:50,303::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`372e1c76-97d7-45b5-8ab9-1196496fd5ed`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:49:50,304::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:49:50,305::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:49:50,305::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`372e1c76-97d7-45b5-8ab9-1196496fd5ed`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:49:50,306::task::495::TaskManager.Task::(_debug) Task 6df64c55-3f31-42df-bfac-c9c754f3f6fe: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:49:50,307::task::495::TaskManager.Task::(_debug) Task 6df64c55-3f31-42df-bfac-c9c754f3f6fe: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:49:50,308::task::495::TaskManager.Task::(_debug) Task 6df64c55-3f31-42df-bfac-c9c754f3f6fe: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:49:50,308::task::495::TaskManager.Task::(_debug) Task 6df64c55-3f31-42df-bfac-c9c754f3f6fe: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:49:50,308::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:49:50,308::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:49:50,309::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:49:50,309::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:49:50,310::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:49:50,310::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:49:50,310::task::495::TaskManager.Task::(_debug) Task 6df64c55-3f31-42df-bfac-c9c754f3f6fe: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:49:50,311::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:49:51,728::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:49:52,159::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0526717 s, 19.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:49:54,167::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:49:54,768::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.122409 s, 8.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:49:56,784::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:49:57,332::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0848392 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15914435::INFO::2013-11-20 04:49:58,450::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:49:58,451::task::495::TaskManager.Task::(_debug) Task 4da98b43-cb2d-41d5-abb6-57b8c8ccbd31: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:49:58,453::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a4d33b0c-b0f7-44d1-8581-6014cff764aa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:49:58,454::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:49:58,455::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:49:58,456::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a4d33b0c-b0f7-44d1-8581-6014cff764aa`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:49:58,464::task::495::TaskManager.Task::(_debug) Task 4da98b43-cb2d-41d5-abb6-57b8c8ccbd31: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:49:58,465::task::495::TaskManager.Task::(_debug) Task 4da98b43-cb2d-41d5-abb6-57b8c8ccbd31: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:49:58,466::task::495::TaskManager.Task::(_debug) Task 4da98b43-cb2d-41d5-abb6-57b8c8ccbd31: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:49:58,467::task::495::TaskManager.Task::(_debug) Task 4da98b43-cb2d-41d5-abb6-57b8c8ccbd31: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:49:58,468::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:49:58,468::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:49:58,469::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:49:58,470::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:49:58,470::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:49:58,471::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:49:58,472::task::495::TaskManager.Task::(_debug) Task 4da98b43-cb2d-41d5-abb6-57b8c8ccbd31: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:49:58,472::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:49:59,348::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:49:59,903::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0831321 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:50:01,917::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:50:02,438::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0766488 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:50:04,457::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:50:05,027::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0811061 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:50:07,354::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:50:08,197::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0810109 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15913639::INFO::2013-11-20 04:50:08,420::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:50:08,421::task::495::TaskManager.Task::(_debug) Task 61a881d4-2d8d-4b33-8e45-16a584770ff0: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:50:08,441::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4db12732-8242-497a-a086-72247967954c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:50:08,442::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:50:08,443::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:50:08,445::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4db12732-8242-497a-a086-72247967954c`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:50:08,449::task::495::TaskManager.Task::(_debug) Task 61a881d4-2d8d-4b33-8e45-16a584770ff0: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:50:08,449::task::495::TaskManager.Task::(_debug) Task 61a881d4-2d8d-4b33-8e45-16a584770ff0: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:50:08,450::task::495::TaskManager.Task::(_debug) Task 61a881d4-2d8d-4b33-8e45-16a584770ff0: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:50:08,451::task::495::TaskManager.Task::(_debug) Task 61a881d4-2d8d-4b33-8e45-16a584770ff0: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:50:08,452::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:50:08,453::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:50:08,454::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:50:08,455::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:50:08,456::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:50:08,456::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:50:08,457::task::495::TaskManager.Task::(_debug) Task 61a881d4-2d8d-4b33-8e45-16a584770ff0: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:50:08,458::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:50:10,214::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:50:10,726::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0689958 s, 14.8 MB/s\n'; <rc> = 0
>Thread-15915918::INFO::2013-11-20 04:50:12,443::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:50:12,446::task::495::TaskManager.Task::(_debug) Task 22fec1ee-c549-41c8-b05c-1639c55b87f0: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:50:12,449::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d4b2f176-4e4a-428e-9241-29c4f0bebe77`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:50:12,449::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:50:12,450::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:50:12,451::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d4b2f176-4e4a-428e-9241-29c4f0bebe77`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:50:12,452::task::495::TaskManager.Task::(_debug) Task 22fec1ee-c549-41c8-b05c-1639c55b87f0: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:50:12,453::task::495::TaskManager.Task::(_debug) Task 22fec1ee-c549-41c8-b05c-1639c55b87f0: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:50:12,454::task::495::TaskManager.Task::(_debug) Task 22fec1ee-c549-41c8-b05c-1639c55b87f0: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:50:12,455::task::495::TaskManager.Task::(_debug) Task 22fec1ee-c549-41c8-b05c-1639c55b87f0: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:50:12,455::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:50:12,456::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:50:12,457::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:50:12,457::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:50:12,458::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:50:12,459::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:50:12,459::task::495::TaskManager.Task::(_debug) Task 22fec1ee-c549-41c8-b05c-1639c55b87f0: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:50:12,460::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:50:12,750::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916984::INFO::2013-11-20 04:50:12,961::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=240c0593-fe33-4f8b-a029-097379dbcb85 volUUID=f715faaa-081a-4598-b125-80a2d12af554)
>Thread-15916984::DEBUG::2013-11-20 04:50:12,963::task::495::TaskManager.Task::(_debug) Task ffdf170c-3158-4d90-97ad-758b7f5f2c6e: moving from state init -> state preparing
>Thread-15916984::DEBUG::2013-11-20 04:50:12,970::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9e036810-77df-4d89-9766-27b064993d5c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15916984::DEBUG::2013-11-20 04:50:12,972::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15916984::DEBUG::2013-11-20 04:50:12,973::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15916984::DEBUG::2013-11-20 04:50:12,974::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9e036810-77df-4d89-9766-27b064993d5c`::Granted request
>Thread-15916984::DEBUG::2013-11-20 04:50:12,978::task::495::TaskManager.Task::(_debug) Task ffdf170c-3158-4d90-97ad-758b7f5f2c6e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15916984::DEBUG::2013-11-20 04:50:12,981::task::495::TaskManager.Task::(_debug) Task ffdf170c-3158-4d90-97ad-758b7f5f2c6e: ref 1 aborting False
>Thread-15916984::DEBUG::2013-11-20 04:50:12,982::task::495::TaskManager.Task::(_debug) Task ffdf170c-3158-4d90-97ad-758b7f5f2c6e: finished: {'truesize': '75296145408', 'apparentsize': '75296145408'}
>Thread-15916984::DEBUG::2013-11-20 04:50:12,987::task::495::TaskManager.Task::(_debug) Task ffdf170c-3158-4d90-97ad-758b7f5f2c6e: moving from state preparing -> state finished
>Thread-15916984::DEBUG::2013-11-20 04:50:12,988::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15916984::DEBUG::2013-11-20 04:50:12,988::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916984::DEBUG::2013-11-20 04:50:12,990::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15916984::DEBUG::2013-11-20 04:50:12,992::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15916984::DEBUG::2013-11-20 04:50:13,004::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15916984::DEBUG::2013-11-20 04:50:13,008::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15916984::DEBUG::2013-11-20 04:50:13,009::task::495::TaskManager.Task::(_debug) Task ffdf170c-3158-4d90-97ad-758b7f5f2c6e: ref 0 aborting False
>Thread-15916984::INFO::2013-11-20 04:50:13,014::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '75296145408', 'apparentsize': '75296145408'}
>Dummy-15912651::DEBUG::2013-11-20 04:50:13,319::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0948753 s, 10.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:50:15,328::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:50:15,916::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0860381 s, 11.9 MB/s\n'; <rc> = 0
>Thread-15915863::INFO::2013-11-20 04:50:17,579::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:50:17,580::task::495::TaskManager.Task::(_debug) Task 56b00b97-31be-4733-a1f6-0942ae239b70: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:50:17,581::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f5ad5cb3-2846-4eaa-8486-10c5dbbbf61a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:50:17,582::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:50:17,582::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:50:17,583::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f5ad5cb3-2846-4eaa-8486-10c5dbbbf61a`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:50:17,584::task::495::TaskManager.Task::(_debug) Task 56b00b97-31be-4733-a1f6-0942ae239b70: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:50:17,584::task::495::TaskManager.Task::(_debug) Task 56b00b97-31be-4733-a1f6-0942ae239b70: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:50:17,585::task::495::TaskManager.Task::(_debug) Task 56b00b97-31be-4733-a1f6-0942ae239b70: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:50:17,585::task::495::TaskManager.Task::(_debug) Task 56b00b97-31be-4733-a1f6-0942ae239b70: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:50:17,585::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:50:17,586::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:50:17,586::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:50:17,587::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:50:17,587::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:50:17,588::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:50:17,588::task::495::TaskManager.Task::(_debug) Task 56b00b97-31be-4733-a1f6-0942ae239b70: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:50:17,589::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:50:17,933::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:50:18,455::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0689951 s, 14.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:50:20,469::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 04:50:20,699::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:50:20,702::task::495::TaskManager.Task::(_debug) Task e3c616d9-7b27-4f4e-b010-c003550ec8c2: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:50:20,707::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6de28623-cda6-40fc-a8a0-cc2a81a49802`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:50:20,708::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:50:20,710::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:50:20,710::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6de28623-cda6-40fc-a8a0-cc2a81a49802`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:50:20,711::task::495::TaskManager.Task::(_debug) Task e3c616d9-7b27-4f4e-b010-c003550ec8c2: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:50:20,712::task::495::TaskManager.Task::(_debug) Task e3c616d9-7b27-4f4e-b010-c003550ec8c2: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:50:20,713::task::495::TaskManager.Task::(_debug) Task e3c616d9-7b27-4f4e-b010-c003550ec8c2: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:50:20,714::task::495::TaskManager.Task::(_debug) Task e3c616d9-7b27-4f4e-b010-c003550ec8c2: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:50:20,714::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:50:20,715::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:50:20,716::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:50:20,716::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:50:20,717::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:50:20,717::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:50:20,718::task::495::TaskManager.Task::(_debug) Task e3c616d9-7b27-4f4e-b010-c003550ec8c2: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:50:20,719::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:50:21,028::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0539556 s, 19.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:50:23,037::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:50:23,586::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0715434 s, 14.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:50:25,600::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:50:25,989::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0516908 s, 19.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:50:28,000::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:50:28,542::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0889287 s, 11.5 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 04:50:30,302::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:50:30,303::task::495::TaskManager.Task::(_debug) Task 5bdc92ff-34d0-4494-abf0-d1a6d08c3681: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:50:30,305::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a7ffc0ae-c1ef-4360-b057-702346a82b3b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:50:30,306::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:50:30,307::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:50:30,308::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a7ffc0ae-c1ef-4360-b057-702346a82b3b`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:50:30,309::task::495::TaskManager.Task::(_debug) Task 5bdc92ff-34d0-4494-abf0-d1a6d08c3681: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:50:30,309::task::495::TaskManager.Task::(_debug) Task 5bdc92ff-34d0-4494-abf0-d1a6d08c3681: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:50:30,310::task::495::TaskManager.Task::(_debug) Task 5bdc92ff-34d0-4494-abf0-d1a6d08c3681: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:50:30,311::task::495::TaskManager.Task::(_debug) Task 5bdc92ff-34d0-4494-abf0-d1a6d08c3681: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:50:30,311::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:50:30,312::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:50:30,313::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:50:30,314::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:50:30,314::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:50:30,315::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:50:30,315::task::495::TaskManager.Task::(_debug) Task 5bdc92ff-34d0-4494-abf0-d1a6d08c3681: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:50:30,316::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:50:30,317::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:50:30,317::task::495::TaskManager.Task::(_debug) Task 870574ed-bea7-4e6e-8a2e-ccaa3cc018e4: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:50:30,318::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8dbc311e-aa1a-482d-9efd-5e3c99b4ad7c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:50:30,319::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:50:30,320::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:50:30,320::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8dbc311e-aa1a-482d-9efd-5e3c99b4ad7c`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:50:30,321::task::495::TaskManager.Task::(_debug) Task 870574ed-bea7-4e6e-8a2e-ccaa3cc018e4: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:50:30,322::task::495::TaskManager.Task::(_debug) Task 870574ed-bea7-4e6e-8a2e-ccaa3cc018e4: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:50:30,322::task::495::TaskManager.Task::(_debug) Task 870574ed-bea7-4e6e-8a2e-ccaa3cc018e4: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:50:30,323::task::495::TaskManager.Task::(_debug) Task 870574ed-bea7-4e6e-8a2e-ccaa3cc018e4: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:50:30,324::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:50:30,324::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:50:30,325::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:50:30,326::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:50:30,326::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:50:30,327::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:50:30,327::task::495::TaskManager.Task::(_debug) Task 870574ed-bea7-4e6e-8a2e-ccaa3cc018e4: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:50:30,328::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:50:30,560::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 04:50:30,813::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:50:30,820::task::495::TaskManager.Task::(_debug) Task 8c8e4048-9481-485e-8356-4b58545be53a: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:50:30,823::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d0b039b3-fea1-4857-931c-0399ecdfb0e8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:50:30,823::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:50:30,824::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:50:30,824::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d0b039b3-fea1-4857-931c-0399ecdfb0e8`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:50:30,825::task::495::TaskManager.Task::(_debug) Task 8c8e4048-9481-485e-8356-4b58545be53a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:50:30,825::task::495::TaskManager.Task::(_debug) Task 8c8e4048-9481-485e-8356-4b58545be53a: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:50:30,826::task::495::TaskManager.Task::(_debug) Task 8c8e4048-9481-485e-8356-4b58545be53a: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:50:30,826::task::495::TaskManager.Task::(_debug) Task 8c8e4048-9481-485e-8356-4b58545be53a: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:50:30,827::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:50:30,827::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:50:30,827::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:50:30,828::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:50:30,828::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:50:30,829::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:50:30,829::task::495::TaskManager.Task::(_debug) Task 8c8e4048-9481-485e-8356-4b58545be53a: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:50:30,829::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:50:31,118::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0787197 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:50:33,132::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:50:33,588::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0752416 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:50:35,601::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:50:36,184::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0869427 s, 11.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:50:38,195::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:50:38,737::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0755076 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:50:40,752::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 04:50:40,981::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:50:40,984::task::495::TaskManager.Task::(_debug) Task dd26cd4e-445d-43f2-9125-669b6a950eb2: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:50:40,988::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7da216c6-4de4-4896-a00e-05f4b6ad2f1f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:50:40,989::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:50:40,990::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:50:40,991::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7da216c6-4de4-4896-a00e-05f4b6ad2f1f`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:50:40,993::task::495::TaskManager.Task::(_debug) Task dd26cd4e-445d-43f2-9125-669b6a950eb2: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:50:40,994::task::495::TaskManager.Task::(_debug) Task dd26cd4e-445d-43f2-9125-669b6a950eb2: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:50:40,995::task::495::TaskManager.Task::(_debug) Task dd26cd4e-445d-43f2-9125-669b6a950eb2: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:50:40,996::task::495::TaskManager.Task::(_debug) Task dd26cd4e-445d-43f2-9125-669b6a950eb2: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:50:40,997::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:50:40,998::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:50:40,999::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:50:41,000::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:50:41,000::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:50:41,001::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:50:41,002::task::495::TaskManager.Task::(_debug) Task dd26cd4e-445d-43f2-9125-669b6a950eb2: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:50:41,003::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15913695::INFO::2013-11-20 04:50:41,103::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:50:41,104::task::495::TaskManager.Task::(_debug) Task 2dab1c66-7f9c-4451-a577-efe48aabf49f: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:50:41,106::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b324433a-6d95-4a36-88c2-10eece0cf714`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:50:41,106::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:50:41,107::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:50:41,108::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b324433a-6d95-4a36-88c2-10eece0cf714`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:50:41,109::task::495::TaskManager.Task::(_debug) Task 2dab1c66-7f9c-4451-a577-efe48aabf49f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:50:41,110::task::495::TaskManager.Task::(_debug) Task 2dab1c66-7f9c-4451-a577-efe48aabf49f: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:50:41,111::task::495::TaskManager.Task::(_debug) Task 2dab1c66-7f9c-4451-a577-efe48aabf49f: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:50:41,112::task::495::TaskManager.Task::(_debug) Task 2dab1c66-7f9c-4451-a577-efe48aabf49f: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:50:41,112::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:50:41,113::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:50:41,114::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:50:41,115::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:50:41,115::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:50:41,116::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:50:41,117::task::495::TaskManager.Task::(_debug) Task 2dab1c66-7f9c-4451-a577-efe48aabf49f: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:50:41,117::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:50:41,323::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0850489 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:50:43,335::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:50:43,868::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0762928 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:50:45,883::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:50:46,451::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0759193 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:50:48,466::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:50:48,976::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0591993 s, 17.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:50:50,991::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 04:50:51,237::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15914519::INFO::2013-11-20 04:50:51,239::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15913546::DEBUG::2013-11-20 04:50:51,242::task::495::TaskManager.Task::(_debug) Task a87ba3fa-42c0-4914-bb73-04b7a23dbf62: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:50:51,245::task::495::TaskManager.Task::(_debug) Task e24ffef0-26cd-408e-8a28-deecc69e0b2c: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:50:51,246::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`04406b55-8962-4026-bdb2-8b20eb2f38d5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:50:51,251::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:50:51,252::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`50dfd4a2-16a3-45a2-90c4-75fa768e22f6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:50:51,253::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:50:51,254::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:50:51,255::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`04406b55-8962-4026-bdb2-8b20eb2f38d5`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:50:51,256::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:50:51,256::task::495::TaskManager.Task::(_debug) Task a87ba3fa-42c0-4914-bb73-04b7a23dbf62: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15914519::DEBUG::2013-11-20 04:50:51,257::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`50dfd4a2-16a3-45a2-90c4-75fa768e22f6`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:50:51,257::task::495::TaskManager.Task::(_debug) Task a87ba3fa-42c0-4914-bb73-04b7a23dbf62: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:50:51,259::task::495::TaskManager.Task::(_debug) Task a87ba3fa-42c0-4914-bb73-04b7a23dbf62: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:50:51,259::task::495::TaskManager.Task::(_debug) Task e24ffef0-26cd-408e-8a28-deecc69e0b2c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913546::DEBUG::2013-11-20 04:50:51,260::task::495::TaskManager.Task::(_debug) Task a87ba3fa-42c0-4914-bb73-04b7a23dbf62: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:50:51,260::task::495::TaskManager.Task::(_debug) Task e24ffef0-26cd-408e-8a28-deecc69e0b2c: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:50:51,261::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:50:51,261::task::495::TaskManager.Task::(_debug) Task e24ffef0-26cd-408e-8a28-deecc69e0b2c: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:50:51,262::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:50:51,262::task::495::TaskManager.Task::(_debug) Task e24ffef0-26cd-408e-8a28-deecc69e0b2c: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:50:51,263::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15914519::DEBUG::2013-11-20 04:50:51,264::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:50:51,264::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:50:51,265::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:50:51,265::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:50:51,266::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:50:51,266::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913546::DEBUG::2013-11-20 04:50:51,268::task::495::TaskManager.Task::(_debug) Task a87ba3fa-42c0-4914-bb73-04b7a23dbf62: ref 0 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:50:51,268::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913546::INFO::2013-11-20 04:50:51,269::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:50:51,269::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913546::INFO::2013-11-20 04:50:51,270::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15914519::DEBUG::2013-11-20 04:50:51,270::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:50:51,271::task::495::TaskManager.Task::(_debug) Task 131957b8-636f-4fc8-82ba-179bc6e8a252: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:50:51,271::task::495::TaskManager.Task::(_debug) Task e24ffef0-26cd-408e-8a28-deecc69e0b2c: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:50:51,272::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`5f36af6c-c19a-40d0-b9ce-d11e3a594051`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::INFO::2013-11-20 04:50:51,273::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:50:51,273::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:50:51,274::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:50:51,274::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`5f36af6c-c19a-40d0-b9ce-d11e3a594051`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:50:51,275::task::495::TaskManager.Task::(_debug) Task 131957b8-636f-4fc8-82ba-179bc6e8a252: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:50:51,275::task::495::TaskManager.Task::(_debug) Task 131957b8-636f-4fc8-82ba-179bc6e8a252: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:50:51,276::task::495::TaskManager.Task::(_debug) Task 131957b8-636f-4fc8-82ba-179bc6e8a252: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:50:51,276::task::495::TaskManager.Task::(_debug) Task 131957b8-636f-4fc8-82ba-179bc6e8a252: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:50:51,277::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:50:51,277::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:50:51,278::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:50:51,278::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:50:51,278::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:50:51,279::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:50:51,279::task::495::TaskManager.Task::(_debug) Task 131957b8-636f-4fc8-82ba-179bc6e8a252: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:50:51,280::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:50:51,280::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:50:51,281::task::495::TaskManager.Task::(_debug) Task 14be92c6-3f89-453c-965d-c337b23fafca: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:50:51,281::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`abc0d7ec-0936-4271-87bc-ce12fc62057f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:50:51,282::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:50:51,282::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:50:51,283::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`abc0d7ec-0936-4271-87bc-ce12fc62057f`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:50:51,283::task::495::TaskManager.Task::(_debug) Task 14be92c6-3f89-453c-965d-c337b23fafca: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:50:51,283::task::495::TaskManager.Task::(_debug) Task 14be92c6-3f89-453c-965d-c337b23fafca: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:50:51,284::task::495::TaskManager.Task::(_debug) Task 14be92c6-3f89-453c-965d-c337b23fafca: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:50:51,284::task::495::TaskManager.Task::(_debug) Task 14be92c6-3f89-453c-965d-c337b23fafca: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:50:51,285::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:50:51,285::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:50:51,285::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:50:51,286::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:50:51,286::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:50:51,286::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:50:51,287::task::495::TaskManager.Task::(_debug) Task 14be92c6-3f89-453c-965d-c337b23fafca: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:50:51,287::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:50:51,288::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:50:51,288::task::495::TaskManager.Task::(_debug) Task 28f56d99-d3c2-4dbe-b467-9ee152a746f0: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:50:51,289::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`96721250-2a22-440c-8c73-cc7f6bb00f75`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:50:51,289::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:50:51,289::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:50:51,290::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`96721250-2a22-440c-8c73-cc7f6bb00f75`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:50:51,290::task::495::TaskManager.Task::(_debug) Task 28f56d99-d3c2-4dbe-b467-9ee152a746f0: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:50:51,291::task::495::TaskManager.Task::(_debug) Task 28f56d99-d3c2-4dbe-b467-9ee152a746f0: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:50:51,291::task::495::TaskManager.Task::(_debug) Task 28f56d99-d3c2-4dbe-b467-9ee152a746f0: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:50:51,291::task::495::TaskManager.Task::(_debug) Task 28f56d99-d3c2-4dbe-b467-9ee152a746f0: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:50:51,292::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:50:51,292::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:50:51,292::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:50:51,293::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:50:51,293::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:50:51,293::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:50:51,294::task::495::TaskManager.Task::(_debug) Task 28f56d99-d3c2-4dbe-b467-9ee152a746f0: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:50:51,294::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:50:51,478::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0544061 s, 18.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:50:53,488::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:50:54,068::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.079733 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:50:56,083::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:50:56,663::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0907859 s, 11.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:50:58,680::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:50:59,218::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0847668 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15914435::INFO::2013-11-20 04:50:59,298::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:50:59,299::task::495::TaskManager.Task::(_debug) Task 3e11959b-db9a-4506-b0be-c65b50f437ab: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:50:59,301::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9db8252b-65a0-41e2-b50b-29c511fafc8a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:50:59,301::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:50:59,302::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:50:59,303::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9db8252b-65a0-41e2-b50b-29c511fafc8a`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:50:59,304::task::495::TaskManager.Task::(_debug) Task 3e11959b-db9a-4506-b0be-c65b50f437ab: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:50:59,305::task::495::TaskManager.Task::(_debug) Task 3e11959b-db9a-4506-b0be-c65b50f437ab: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:50:59,305::task::495::TaskManager.Task::(_debug) Task 3e11959b-db9a-4506-b0be-c65b50f437ab: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:50:59,306::task::495::TaskManager.Task::(_debug) Task 3e11959b-db9a-4506-b0be-c65b50f437ab: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:50:59,307::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:50:59,307::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:50:59,308::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:50:59,308::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:50:59,308::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:50:59,309::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:50:59,309::task::495::TaskManager.Task::(_debug) Task 3e11959b-db9a-4506-b0be-c65b50f437ab: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:50:59,310::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:51:01,232::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:51:01,756::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0788294 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:51:03,771::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:51:04,243::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0724938 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:51:06,259::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:51:06,711::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0538924 s, 19.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:51:08,721::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:51:09,010::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:51:09,012::task::495::TaskManager.Task::(_debug) Task a6e162a3-8704-42ff-a7d7-267c23aa3780: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:51:09,015::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b0fa26a6-3535-4341-acba-6464ac842094`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:51:09,016::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:51:09,017::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:51:09,017::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b0fa26a6-3535-4341-acba-6464ac842094`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:51:09,018::task::495::TaskManager.Task::(_debug) Task a6e162a3-8704-42ff-a7d7-267c23aa3780: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:51:09,019::task::495::TaskManager.Task::(_debug) Task a6e162a3-8704-42ff-a7d7-267c23aa3780: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:51:09,020::task::495::TaskManager.Task::(_debug) Task a6e162a3-8704-42ff-a7d7-267c23aa3780: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:51:09,021::task::495::TaskManager.Task::(_debug) Task a6e162a3-8704-42ff-a7d7-267c23aa3780: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:51:09,021::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:51:09,022::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:51:09,022::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:51:09,023::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:51:09,024::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:51:09,024::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:51:09,025::task::495::TaskManager.Task::(_debug) Task a6e162a3-8704-42ff-a7d7-267c23aa3780: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:51:09,025::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:51:09,193::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.084747 s, 12.1 MB/s\n'; <rc> = 0
>Thread-1024767::ERROR::2013-11-20 04:51:09,415::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `5d7b5178-3790-472c-a13b-b481eb27a61e` (Operation Stuck)
>Thread-15917043::DEBUG::2013-11-20 04:51:09,417::misc::1156::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Emitting event
>Thread-15917043::DEBUG::2013-11-20 04:51:09,418::misc::1166::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Calling registered method `run`
>Thread-15917043::DEBUG::2013-11-20 04:51:09,419::misc::1174::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Event emitted
>Thread-42839::ERROR::2013-11-20 04:51:09,652::sp::113::Storage.StatsThread::(run) Could not figure out delay for domain `45cdff8b-89c0-4926-a8c7-80354064bfa6` (Operation Stuck)
>Thread-15917046::DEBUG::2013-11-20 04:51:09,653::misc::1156::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Emitting event
>Thread-15917046::DEBUG::2013-11-20 04:51:09,654::misc::1166::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Calling registered method `run`
>Thread-15917046::DEBUG::2013-11-20 04:51:09,655::misc::1174::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Event emitted
>Dummy-15912651::DEBUG::2013-11-20 04:51:11,207::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:51:11,658::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0565412 s, 18.1 MB/s\n'; <rc> = 0
>Thread-15915918::INFO::2013-11-20 04:51:13,047::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:51:13,056::task::495::TaskManager.Task::(_debug) Task ea826035-82a3-48c4-bddb-65e74f2e80b3: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:51:13,058::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4b5eba6e-44e8-4c83-84c8-5f170e9cd688`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:51:13,058::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:51:13,058::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:51:13,059::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4b5eba6e-44e8-4c83-84c8-5f170e9cd688`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:51:13,059::task::495::TaskManager.Task::(_debug) Task ea826035-82a3-48c4-bddb-65e74f2e80b3: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:51:13,060::task::495::TaskManager.Task::(_debug) Task ea826035-82a3-48c4-bddb-65e74f2e80b3: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:51:13,060::task::495::TaskManager.Task::(_debug) Task ea826035-82a3-48c4-bddb-65e74f2e80b3: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:51:13,061::task::495::TaskManager.Task::(_debug) Task ea826035-82a3-48c4-bddb-65e74f2e80b3: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:51:13,061::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:51:13,061::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:51:13,062::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:51:13,062::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:51:13,062::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:51:13,063::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:51:13,063::task::495::TaskManager.Task::(_debug) Task ea826035-82a3-48c4-bddb-65e74f2e80b3: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:51:13,064::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:51:13,669::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916984::INFO::2013-11-20 04:51:13,987::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=240c0593-fe33-4f8b-a029-097379dbcb85 volUUID=f715faaa-081a-4598-b125-80a2d12af554)
>Thread-15916984::DEBUG::2013-11-20 04:51:13,989::task::495::TaskManager.Task::(_debug) Task f2e1dd9d-e0ee-4c6a-8797-c39435cfedda: moving from state init -> state preparing
>Thread-15916984::DEBUG::2013-11-20 04:51:13,991::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d02ae34a-9713-4907-9cfc-28feafdac123`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15916984::DEBUG::2013-11-20 04:51:13,992::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15916984::DEBUG::2013-11-20 04:51:13,996::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15916984::DEBUG::2013-11-20 04:51:14,003::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d02ae34a-9713-4907-9cfc-28feafdac123`::Granted request
>Thread-15916984::DEBUG::2013-11-20 04:51:14,007::task::495::TaskManager.Task::(_debug) Task f2e1dd9d-e0ee-4c6a-8797-c39435cfedda: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15916984::DEBUG::2013-11-20 04:51:14,008::task::495::TaskManager.Task::(_debug) Task f2e1dd9d-e0ee-4c6a-8797-c39435cfedda: ref 1 aborting False
>Thread-15916984::DEBUG::2013-11-20 04:51:14,017::task::495::TaskManager.Task::(_debug) Task f2e1dd9d-e0ee-4c6a-8797-c39435cfedda: finished: {'truesize': '75296145408', 'apparentsize': '75296145408'}
>Thread-15916984::DEBUG::2013-11-20 04:51:14,021::task::495::TaskManager.Task::(_debug) Task f2e1dd9d-e0ee-4c6a-8797-c39435cfedda: moving from state preparing -> state finished
>Thread-15916984::DEBUG::2013-11-20 04:51:14,022::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15916984::DEBUG::2013-11-20 04:51:14,023::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916984::DEBUG::2013-11-20 04:51:14,031::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15916984::DEBUG::2013-11-20 04:51:14,034::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15916984::DEBUG::2013-11-20 04:51:14,041::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15916984::DEBUG::2013-11-20 04:51:14,045::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15916984::DEBUG::2013-11-20 04:51:14,046::task::495::TaskManager.Task::(_debug) Task f2e1dd9d-e0ee-4c6a-8797-c39435cfedda: ref 0 aborting False
>Thread-15916984::INFO::2013-11-20 04:51:14,047::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '75296145408', 'apparentsize': '75296145408'}
>Dummy-15912651::DEBUG::2013-11-20 04:51:14,103::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0614976 s, 16.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:51:16,114::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:51:16,620::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.084091 s, 12.2 MB/s\n'; <rc> = 0
>Thread-15915863::INFO::2013-11-20 04:51:17,985::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:51:17,986::task::495::TaskManager.Task::(_debug) Task ba75fdd6-a059-4861-9893-ed46ae8140e6: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:51:17,989::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`56cf405b-e081-47fc-87da-fb5f2488de6a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:51:17,990::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:51:17,991::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:51:17,992::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`56cf405b-e081-47fc-87da-fb5f2488de6a`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:51:17,994::task::495::TaskManager.Task::(_debug) Task ba75fdd6-a059-4861-9893-ed46ae8140e6: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:51:17,994::task::495::TaskManager.Task::(_debug) Task ba75fdd6-a059-4861-9893-ed46ae8140e6: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:51:17,996::task::495::TaskManager.Task::(_debug) Task ba75fdd6-a059-4861-9893-ed46ae8140e6: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:51:17,996::task::495::TaskManager.Task::(_debug) Task ba75fdd6-a059-4861-9893-ed46ae8140e6: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:51:17,997::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:51:17,998::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:51:17,999::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:51:18,000::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:51:18,001::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:51:18,001::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:51:18,002::task::495::TaskManager.Task::(_debug) Task ba75fdd6-a059-4861-9893-ed46ae8140e6: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:51:18,003::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:51:18,637::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:51:19,168::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0882516 s, 11.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:51:21,184::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:51:21,689::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0797711 s, 12.8 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 04:51:21,705::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:51:21,707::task::495::TaskManager.Task::(_debug) Task 7d0713fe-efd3-4b1c-b264-4dfba7a85e85: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:51:21,710::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`85a32472-1103-4158-800a-55b3452985e6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:51:21,711::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:51:21,712::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:51:21,712::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`85a32472-1103-4158-800a-55b3452985e6`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:51:21,713::task::495::TaskManager.Task::(_debug) Task 7d0713fe-efd3-4b1c-b264-4dfba7a85e85: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:51:21,713::task::495::TaskManager.Task::(_debug) Task 7d0713fe-efd3-4b1c-b264-4dfba7a85e85: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:51:21,714::task::495::TaskManager.Task::(_debug) Task 7d0713fe-efd3-4b1c-b264-4dfba7a85e85: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:51:21,714::task::495::TaskManager.Task::(_debug) Task 7d0713fe-efd3-4b1c-b264-4dfba7a85e85: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:51:21,715::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:51:21,715::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:51:21,716::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:51:21,716::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:51:21,717::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:51:21,717::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:51:21,718::task::495::TaskManager.Task::(_debug) Task 7d0713fe-efd3-4b1c-b264-4dfba7a85e85: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:51:21,719::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:51:23,708::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:51:24,225::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0836047 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:51:26,241::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:51:26,773::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0882335 s, 11.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:51:28,787::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:51:29,291::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0820043 s, 12.5 MB/s\n'; <rc> = 0
>Thread-42840::DEBUG::2013-11-20 04:51:29,499::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 04:51:30,232::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913518::INFO::2013-11-20 04:51:31,249::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:51:31,250::task::495::TaskManager.Task::(_debug) Task 9bbdfdb2-6d71-408d-8063-66fabeb43301: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:51:31,252::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fe0bd287-6f83-4e60-b7ed-2a5dd77cf37f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:51:31,252::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:51:31,253::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:51:31,254::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fe0bd287-6f83-4e60-b7ed-2a5dd77cf37f`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:51:31,255::task::495::TaskManager.Task::(_debug) Task 9bbdfdb2-6d71-408d-8063-66fabeb43301: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:51:31,256::task::495::TaskManager.Task::(_debug) Task 9bbdfdb2-6d71-408d-8063-66fabeb43301: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:51:31,256::task::495::TaskManager.Task::(_debug) Task 9bbdfdb2-6d71-408d-8063-66fabeb43301: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:51:31,257::task::495::TaskManager.Task::(_debug) Task 9bbdfdb2-6d71-408d-8063-66fabeb43301: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:51:31,257::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:51:31,258::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:51:31,259::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:51:31,259::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:51:31,260::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:51:31,260::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:51:31,261::task::495::TaskManager.Task::(_debug) Task 9bbdfdb2-6d71-408d-8063-66fabeb43301: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:51:31,262::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:51:31,262::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:51:31,263::task::495::TaskManager.Task::(_debug) Task a48e99e1-6da2-40ab-9774-4589f7d94a9c: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:51:31,264::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`23c783e7-b647-417d-90f0-5aabac62b2fb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:51:31,265::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:51:31,265::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:51:31,266::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`23c783e7-b647-417d-90f0-5aabac62b2fb`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:51:31,272::task::495::TaskManager.Task::(_debug) Task a48e99e1-6da2-40ab-9774-4589f7d94a9c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:51:31,273::task::495::TaskManager.Task::(_debug) Task a48e99e1-6da2-40ab-9774-4589f7d94a9c: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:51:31,273::task::495::TaskManager.Task::(_debug) Task a48e99e1-6da2-40ab-9774-4589f7d94a9c: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:51:31,274::task::495::TaskManager.Task::(_debug) Task a48e99e1-6da2-40ab-9774-4589f7d94a9c: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:51:31,274::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:51:31,275::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:51:31,275::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:51:31,276::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:51:31,276::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:51:31,276::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:51:31,277::task::495::TaskManager.Task::(_debug) Task a48e99e1-6da2-40ab-9774-4589f7d94a9c: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:51:31,278::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:51:31,310::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:51:31,804::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0788523 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 04:51:31,924::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:51:31,924::task::495::TaskManager.Task::(_debug) Task 85aa9d3c-d6b2-4f29-854f-e1dc6681ae0a: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:51:31,926::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`64b71a2d-46f1-438b-a7e5-26c43c6d3dc0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:51:31,926::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:51:31,927::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:51:31,928::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`64b71a2d-46f1-438b-a7e5-26c43c6d3dc0`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:51:31,930::task::495::TaskManager.Task::(_debug) Task 85aa9d3c-d6b2-4f29-854f-e1dc6681ae0a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:51:31,931::task::495::TaskManager.Task::(_debug) Task 85aa9d3c-d6b2-4f29-854f-e1dc6681ae0a: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:51:31,932::task::495::TaskManager.Task::(_debug) Task 85aa9d3c-d6b2-4f29-854f-e1dc6681ae0a: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:51:31,932::task::495::TaskManager.Task::(_debug) Task 85aa9d3c-d6b2-4f29-854f-e1dc6681ae0a: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:51:31,933::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:51:31,933::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:51:31,934::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:51:31,934::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:51:31,935::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:51:31,935::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:51:31,935::task::495::TaskManager.Task::(_debug) Task 85aa9d3c-d6b2-4f29-854f-e1dc6681ae0a: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:51:31,936::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:51:33,816::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:51:34,352::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.083725 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:51:36,368::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:51:36,911::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0776172 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:51:38,927::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:51:39,354::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0535247 s, 19.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:51:41,364::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:51:41,846::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0805848 s, 12.7 MB/s\n'; <rc> = 0
>Thread-15914453::INFO::2013-11-20 04:51:42,014::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:51:42,015::task::495::TaskManager.Task::(_debug) Task abf937c5-5874-4d3e-b8f9-115025c9dc4e: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:51:42,017::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3dff1423-83b9-4b6c-8a75-0b3d61790cae`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:51:42,018::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:51:42,019::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:51:42,020::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3dff1423-83b9-4b6c-8a75-0b3d61790cae`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:51:42,021::task::495::TaskManager.Task::(_debug) Task abf937c5-5874-4d3e-b8f9-115025c9dc4e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:51:42,021::task::495::TaskManager.Task::(_debug) Task abf937c5-5874-4d3e-b8f9-115025c9dc4e: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:51:42,022::task::495::TaskManager.Task::(_debug) Task abf937c5-5874-4d3e-b8f9-115025c9dc4e: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:51:42,023::task::495::TaskManager.Task::(_debug) Task abf937c5-5874-4d3e-b8f9-115025c9dc4e: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:51:42,023::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:51:42,024::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:51:42,025::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:51:42,026::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:51:42,026::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:51:42,027::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:51:42,028::task::495::TaskManager.Task::(_debug) Task abf937c5-5874-4d3e-b8f9-115025c9dc4e: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:51:42,029::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15913695::INFO::2013-11-20 04:51:42,112::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:51:42,113::task::495::TaskManager.Task::(_debug) Task 0e85c054-e9dd-45a2-bc76-5ad08c5c5d03: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:51:42,114::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5ba9d37b-13c6-4a9f-bdab-2ae822785b3d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:51:42,115::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:51:42,116::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:51:42,116::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5ba9d37b-13c6-4a9f-bdab-2ae822785b3d`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:51:42,117::task::495::TaskManager.Task::(_debug) Task 0e85c054-e9dd-45a2-bc76-5ad08c5c5d03: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:51:42,118::task::495::TaskManager.Task::(_debug) Task 0e85c054-e9dd-45a2-bc76-5ad08c5c5d03: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:51:42,119::task::495::TaskManager.Task::(_debug) Task 0e85c054-e9dd-45a2-bc76-5ad08c5c5d03: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:51:42,119::task::495::TaskManager.Task::(_debug) Task 0e85c054-e9dd-45a2-bc76-5ad08c5c5d03: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:51:42,120::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:51:42,120::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:51:42,122::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:51:42,122::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:51:42,124::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:51:42,124::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:51:42,125::task::495::TaskManager.Task::(_debug) Task 0e85c054-e9dd-45a2-bc76-5ad08c5c5d03: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:51:42,126::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:51:43,863::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:51:44,339::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0525803 s, 19.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:51:46,353::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:51:46,889::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0801891 s, 12.8 MB/s\n'; <rc> = 0
>Thread-15917048::DEBUG::2013-11-20 04:51:47,013::misc::1156::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Emitting event
>Thread-15917049::DEBUG::2013-11-20 04:51:47,015::misc::1156::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Emitting event
>Thread-15917048::DEBUG::2013-11-20 04:51:47,016::misc::1166::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Calling registered method `run`
>Thread-15917050::DEBUG::2013-11-20 04:51:47,023::spm::365::Storage.SPM::(_upgradePoolDomain) Preparing to upgrade domain 5d7b5178-3790-472c-a13b-b481eb27a61e
>Thread-15917048::DEBUG::2013-11-20 04:51:47,024::misc::1174::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Event emitted
>Thread-15917049::DEBUG::2013-11-20 04:51:47,025::misc::1166::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Calling registered method `run`
>Thread-15917050::ERROR::2013-11-20 04:51:47,026::spm::373::Storage.SPM::(_upgradePoolDomain) Error while preparing domain `5d7b5178-3790-472c-a13b-b481eb27a61e` upgrade
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/spm.py", line 369, in _upgradePoolDomain
>IndexError: list index out of range
>Thread-15917049::DEBUG::2013-11-20 04:51:47,033::misc::1174::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Event emitted
>Dummy-15912651::DEBUG::2013-11-20 04:51:48,906::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:51:49,406::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0477114 s, 21.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:51:51,416::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:51:51,939::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0698496 s, 14.7 MB/s\n'; <rc> = 0
>Thread-15914519::INFO::2013-11-20 04:51:52,103::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:51:52,105::task::495::TaskManager.Task::(_debug) Task ab26b082-7002-4810-8934-6824eede2e9d: moving from state init -> state preparing
>Thread-15913546::INFO::2013-11-20 04:51:52,105::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15914519::DEBUG::2013-11-20 04:51:52,118::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`928d884c-ac41-4760-8698-4fec370770aa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:51:52,119::task::495::TaskManager.Task::(_debug) Task c7bef505-8e75-4c88-9621-d4df364a47f2: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:51:52,121::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:51:52,124::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`11b6f0b0-b7a8-46ad-9d33-9ac31768a953`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:51:52,127::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:51:52,127::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:51:52,129::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`928d884c-ac41-4760-8698-4fec370770aa`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:51:52,144::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:51:52,148::task::495::TaskManager.Task::(_debug) Task ab26b082-7002-4810-8934-6824eede2e9d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913546::DEBUG::2013-11-20 04:51:52,149::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`11b6f0b0-b7a8-46ad-9d33-9ac31768a953`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:51:52,151::task::495::TaskManager.Task::(_debug) Task ab26b082-7002-4810-8934-6824eede2e9d: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:51:52,160::task::495::TaskManager.Task::(_debug) Task ab26b082-7002-4810-8934-6824eede2e9d: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15914519::DEBUG::2013-11-20 04:51:52,173::task::495::TaskManager.Task::(_debug) Task ab26b082-7002-4810-8934-6824eede2e9d: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:51:52,193::task::495::TaskManager.Task::(_debug) Task c7bef505-8e75-4c88-9621-d4df364a47f2: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15914519::DEBUG::2013-11-20 04:51:52,195::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:51:52,204::task::495::TaskManager.Task::(_debug) Task c7bef505-8e75-4c88-9621-d4df364a47f2: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:51:52,208::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:51:52,210::task::495::TaskManager.Task::(_debug) Task c7bef505-8e75-4c88-9621-d4df364a47f2: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:51:52,212::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913546::DEBUG::2013-11-20 04:51:52,213::task::495::TaskManager.Task::(_debug) Task c7bef505-8e75-4c88-9621-d4df364a47f2: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:51:52,214::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:51:52,215::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:51:52,215::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:51:52,216::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:51:52,217::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:51:52,218::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15914519::DEBUG::2013-11-20 04:51:52,220::task::495::TaskManager.Task::(_debug) Task ab26b082-7002-4810-8934-6824eede2e9d: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:51:52,220::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15914519::INFO::2013-11-20 04:51:52,221::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:51:52,222::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:51:52,223::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:51:52,224::task::495::TaskManager.Task::(_debug) Task c7bef505-8e75-4c88-9621-d4df364a47f2: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:51:52,225::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:51:52,227::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:51:52,228::task::495::TaskManager.Task::(_debug) Task b1462877-03eb-40e0-9a73-fc7f04bd80c5: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:51:52,230::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e77e9ae9-dca0-4597-9163-4e4b046dd023`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:51:52,230::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:51:52,232::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:51:52,233::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e77e9ae9-dca0-4597-9163-4e4b046dd023`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:51:52,234::task::495::TaskManager.Task::(_debug) Task b1462877-03eb-40e0-9a73-fc7f04bd80c5: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:51:52,235::task::495::TaskManager.Task::(_debug) Task b1462877-03eb-40e0-9a73-fc7f04bd80c5: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:51:52,236::task::495::TaskManager.Task::(_debug) Task b1462877-03eb-40e0-9a73-fc7f04bd80c5: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:51:52,237::task::495::TaskManager.Task::(_debug) Task b1462877-03eb-40e0-9a73-fc7f04bd80c5: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:51:52,237::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:51:52,238::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:51:52,239::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:51:52,239::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:51:52,240::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:51:52,240::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:51:52,241::task::495::TaskManager.Task::(_debug) Task b1462877-03eb-40e0-9a73-fc7f04bd80c5: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:51:52,242::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:51:52,243::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:51:52,243::task::495::TaskManager.Task::(_debug) Task 7e7fbb1e-fbc9-4440-8db1-f37c8e2b02a2: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:51:52,245::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`32e81e70-79cb-45fb-a9d3-769424f4ebad`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:51:52,245::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:51:52,246::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:51:52,246::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`32e81e70-79cb-45fb-a9d3-769424f4ebad`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:51:52,248::task::495::TaskManager.Task::(_debug) Task 7e7fbb1e-fbc9-4440-8db1-f37c8e2b02a2: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:51:52,248::task::495::TaskManager.Task::(_debug) Task 7e7fbb1e-fbc9-4440-8db1-f37c8e2b02a2: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:51:52,249::task::495::TaskManager.Task::(_debug) Task 7e7fbb1e-fbc9-4440-8db1-f37c8e2b02a2: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:51:52,250::task::495::TaskManager.Task::(_debug) Task 7e7fbb1e-fbc9-4440-8db1-f37c8e2b02a2: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:51:52,251::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:51:52,254::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:51:52,255::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:51:52,256::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:51:52,257::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:51:52,258::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:51:52,258::task::495::TaskManager.Task::(_debug) Task 7e7fbb1e-fbc9-4440-8db1-f37c8e2b02a2: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:51:52,259::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:51:52,260::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:51:52,260::task::495::TaskManager.Task::(_debug) Task 0c49d48e-ba28-4ff6-844d-68d3186bdd6c: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:51:52,262::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`56067904-7c60-4464-9b37-848e335ed062`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:51:52,262::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:51:52,263::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:51:52,264::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`56067904-7c60-4464-9b37-848e335ed062`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:51:52,265::task::495::TaskManager.Task::(_debug) Task 0c49d48e-ba28-4ff6-844d-68d3186bdd6c: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:51:52,265::task::495::TaskManager.Task::(_debug) Task 0c49d48e-ba28-4ff6-844d-68d3186bdd6c: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:51:52,266::task::495::TaskManager.Task::(_debug) Task 0c49d48e-ba28-4ff6-844d-68d3186bdd6c: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:51:52,266::task::495::TaskManager.Task::(_debug) Task 0c49d48e-ba28-4ff6-844d-68d3186bdd6c: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:51:52,267::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:51:52,268::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:51:52,268::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:51:52,269::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:51:52,269::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:51:52,270::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:51:52,271::task::495::TaskManager.Task::(_debug) Task 0c49d48e-ba28-4ff6-844d-68d3186bdd6c: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:51:52,271::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:51:53,954::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:51:54,493::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0786928 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:51:56,514::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:51:57,076::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0774045 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:51:59,090::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:51:59,617::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0782889 s, 13.1 MB/s\n'; <rc> = 0
>Thread-15914435::INFO::2013-11-20 04:52:00,232::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:52:00,233::task::495::TaskManager.Task::(_debug) Task a192d7bb-5c5b-4c67-8145-fe2d00dae8b6: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:52:00,236::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`043d3ddb-137e-487a-aaca-5c04725e6f63`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:52:00,237::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:52:00,238::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:52:00,239::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`043d3ddb-137e-487a-aaca-5c04725e6f63`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:52:00,240::task::495::TaskManager.Task::(_debug) Task a192d7bb-5c5b-4c67-8145-fe2d00dae8b6: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:52:00,241::task::495::TaskManager.Task::(_debug) Task a192d7bb-5c5b-4c67-8145-fe2d00dae8b6: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:52:00,242::task::495::TaskManager.Task::(_debug) Task a192d7bb-5c5b-4c67-8145-fe2d00dae8b6: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:52:00,243::task::495::TaskManager.Task::(_debug) Task a192d7bb-5c5b-4c67-8145-fe2d00dae8b6: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:52:00,243::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:52:00,244::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:52:00,245::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:52:00,246::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:52:00,246::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:52:00,247::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:52:00,248::task::495::TaskManager.Task::(_debug) Task a192d7bb-5c5b-4c67-8145-fe2d00dae8b6: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:52:00,250::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:52:01,633::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:52:02,190::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0833645 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:52:04,206::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:52:04,783::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0855747 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:52:06,798::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:52:07,373::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0862193 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:52:09,389::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:52:09,599::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:52:09,601::task::495::TaskManager.Task::(_debug) Task 1cb61bcf-24e5-4610-9b05-253e077111ce: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:52:09,605::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`29d1f503-06cb-4530-8898-3cee64ad792b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:52:09,606::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:52:09,608::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:52:09,609::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`29d1f503-06cb-4530-8898-3cee64ad792b`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:52:09,614::task::495::TaskManager.Task::(_debug) Task 1cb61bcf-24e5-4610-9b05-253e077111ce: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:52:09,618::task::495::TaskManager.Task::(_debug) Task 1cb61bcf-24e5-4610-9b05-253e077111ce: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:52:09,621::task::495::TaskManager.Task::(_debug) Task 1cb61bcf-24e5-4610-9b05-253e077111ce: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:52:09,626::task::495::TaskManager.Task::(_debug) Task 1cb61bcf-24e5-4610-9b05-253e077111ce: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:52:09,627::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:52:09,628::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:52:09,629::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:52:09,629::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:52:09,630::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:52:09,631::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:52:09,631::task::495::TaskManager.Task::(_debug) Task 1cb61bcf-24e5-4610-9b05-253e077111ce: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:52:09,632::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:52:09,959::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0856141 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:52:11,974::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:52:12,559::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0935967 s, 10.9 MB/s\n'; <rc> = 0
>Thread-15915918::INFO::2013-11-20 04:52:13,298::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:52:13,299::task::495::TaskManager.Task::(_debug) Task c8d49480-33c3-44d1-848e-5ea6bba9bc44: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:52:13,301::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ef134815-e9b0-4975-89ee-8d005a6cdd7b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:52:13,301::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:52:13,302::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:52:13,303::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ef134815-e9b0-4975-89ee-8d005a6cdd7b`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:52:13,304::task::495::TaskManager.Task::(_debug) Task c8d49480-33c3-44d1-848e-5ea6bba9bc44: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:52:13,305::task::495::TaskManager.Task::(_debug) Task c8d49480-33c3-44d1-848e-5ea6bba9bc44: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:52:13,305::task::495::TaskManager.Task::(_debug) Task c8d49480-33c3-44d1-848e-5ea6bba9bc44: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:52:13,306::task::495::TaskManager.Task::(_debug) Task c8d49480-33c3-44d1-848e-5ea6bba9bc44: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:52:13,307::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:52:13,308::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:52:13,308::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:52:13,309::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:52:13,309::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:52:13,310::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:52:13,311::task::495::TaskManager.Task::(_debug) Task c8d49480-33c3-44d1-848e-5ea6bba9bc44: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:52:13,311::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:52:14,575::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916984::INFO::2013-11-20 04:52:14,782::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=240c0593-fe33-4f8b-a029-097379dbcb85 volUUID=f715faaa-081a-4598-b125-80a2d12af554)
>Thread-15916984::DEBUG::2013-11-20 04:52:14,793::task::495::TaskManager.Task::(_debug) Task 011567b4-2c78-469f-8a32-7ea368e3126c: moving from state init -> state preparing
>Thread-15916984::DEBUG::2013-11-20 04:52:14,796::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2ff3a02b-d16b-4b73-bf6f-3b75a567efac`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15916984::DEBUG::2013-11-20 04:52:14,797::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15916984::DEBUG::2013-11-20 04:52:14,801::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15916984::DEBUG::2013-11-20 04:52:14,802::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2ff3a02b-d16b-4b73-bf6f-3b75a567efac`::Granted request
>Thread-15916984::DEBUG::2013-11-20 04:52:14,803::task::495::TaskManager.Task::(_debug) Task 011567b4-2c78-469f-8a32-7ea368e3126c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15916984::DEBUG::2013-11-20 04:52:14,804::task::495::TaskManager.Task::(_debug) Task 011567b4-2c78-469f-8a32-7ea368e3126c: ref 1 aborting False
>Thread-15916984::DEBUG::2013-11-20 04:52:14,810::task::495::TaskManager.Task::(_debug) Task 011567b4-2c78-469f-8a32-7ea368e3126c: finished: {'truesize': '75296145408', 'apparentsize': '75296145408'}
>Thread-15916984::DEBUG::2013-11-20 04:52:14,813::task::495::TaskManager.Task::(_debug) Task 011567b4-2c78-469f-8a32-7ea368e3126c: moving from state preparing -> state finished
>Thread-15916984::DEBUG::2013-11-20 04:52:14,814::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15916984::DEBUG::2013-11-20 04:52:14,817::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916984::DEBUG::2013-11-20 04:52:14,826::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15916984::DEBUG::2013-11-20 04:52:14,830::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15916984::DEBUG::2013-11-20 04:52:14,836::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15916984::DEBUG::2013-11-20 04:52:14,842::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15916984::DEBUG::2013-11-20 04:52:14,843::task::495::TaskManager.Task::(_debug) Task 011567b4-2c78-469f-8a32-7ea368e3126c: ref 0 aborting False
>Thread-15916984::INFO::2013-11-20 04:52:14,844::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '75296145408', 'apparentsize': '75296145408'}
>Dummy-15912651::DEBUG::2013-11-20 04:52:15,118::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0787675 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:52:17,141::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:52:17,642::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0783448 s, 13.1 MB/s\n'; <rc> = 0
>Thread-15915863::INFO::2013-11-20 04:52:18,199::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:52:18,200::task::495::TaskManager.Task::(_debug) Task a4caa552-310b-4dce-825c-f6728ab04ede: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:52:18,202::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bfd78f61-23b2-4277-8fb5-54751d05ba91`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:52:18,203::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:52:18,204::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:52:18,204::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`bfd78f61-23b2-4277-8fb5-54751d05ba91`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:52:18,205::task::495::TaskManager.Task::(_debug) Task a4caa552-310b-4dce-825c-f6728ab04ede: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:52:18,206::task::495::TaskManager.Task::(_debug) Task a4caa552-310b-4dce-825c-f6728ab04ede: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:52:18,207::task::495::TaskManager.Task::(_debug) Task a4caa552-310b-4dce-825c-f6728ab04ede: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:52:18,208::task::495::TaskManager.Task::(_debug) Task a4caa552-310b-4dce-825c-f6728ab04ede: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:52:18,208::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:52:18,209::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:52:18,210::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:52:18,210::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:52:18,211::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:52:18,212::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:52:18,212::task::495::TaskManager.Task::(_debug) Task a4caa552-310b-4dce-825c-f6728ab04ede: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:52:18,213::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:52:19,658::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:52:20,176::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0639769 s, 16.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:52:22,188::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 04:52:22,480::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:52:22,490::task::495::TaskManager.Task::(_debug) Task 1c13c933-e71d-4280-9419-a55af56c28ed: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:52:22,495::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ad00088e-2af9-4f29-a92c-f2bd4e1beefd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:52:22,496::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:52:22,497::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:52:22,498::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ad00088e-2af9-4f29-a92c-f2bd4e1beefd`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:52:22,499::task::495::TaskManager.Task::(_debug) Task 1c13c933-e71d-4280-9419-a55af56c28ed: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:52:22,500::task::495::TaskManager.Task::(_debug) Task 1c13c933-e71d-4280-9419-a55af56c28ed: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:52:22,501::task::495::TaskManager.Task::(_debug) Task 1c13c933-e71d-4280-9419-a55af56c28ed: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:52:22,501::task::495::TaskManager.Task::(_debug) Task 1c13c933-e71d-4280-9419-a55af56c28ed: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:52:22,502::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:52:22,502::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:52:22,503::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:52:22,504::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:52:22,504::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:52:22,505::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:52:22,505::task::495::TaskManager.Task::(_debug) Task 1c13c933-e71d-4280-9419-a55af56c28ed: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:52:22,506::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:52:22,783::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.073741 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:52:24,797::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:52:25,333::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0790535 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15917052::INFO::2013-11-20 04:52:26,861::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15917052::DEBUG::2013-11-20 04:52:26,862::task::495::TaskManager.Task::(_debug) Task d84eb1ef-146c-4d4f-ba5d-ba3e9a791f48: moving from state init -> state preparing
>Thread-15917052::DEBUG::2013-11-20 04:52:26,867::task::495::TaskManager.Task::(_debug) Task d84eb1ef-146c-4d4f-ba5d-ba3e9a791f48: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000317811965942', 'lastCheck': 1384923136.788502, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00162506103516', 'lastCheck': 1384923137.790252, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00204110145569', 'lastCheck': 1384923137.7896309, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00060510635376', 'lastCheck': 1384923140.852155, 'valid': True, 'code': 0}}
>Thread-15917052::DEBUG::2013-11-20 04:52:26,870::task::495::TaskManager.Task::(_debug) Task d84eb1ef-146c-4d4f-ba5d-ba3e9a791f48: moving from state preparing -> state finished
>Thread-15917052::DEBUG::2013-11-20 04:52:26,870::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917052::DEBUG::2013-11-20 04:52:26,876::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917052::DEBUG::2013-11-20 04:52:26,881::task::495::TaskManager.Task::(_debug) Task d84eb1ef-146c-4d4f-ba5d-ba3e9a791f48: ref 0 aborting False
>Thread-15917052::INFO::2013-11-20 04:52:26,886::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000317811965942', 'lastCheck': 1384923136.788502, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00162506103516', 'lastCheck': 1384923137.790252, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00204110145569', 'lastCheck': 1384923137.7896309, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00060510635376', 'lastCheck': 1384923140.852155, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:52:27,349::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15917053::DEBUG::2013-11-20 04:52:27,583::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVdsCapabilities with () {}
>Dummy-15912651::DEBUG::2013-11-20 04:52:28,067::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.167349 s, 6.1 MB/s\n'; <rc> = 0
>Thread-15917053::DEBUG::2013-11-20 04:52:28,217::utils::579::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-img-rhev' (cwd None)
>Thread-15917053::DEBUG::2013-11-20 04:52:28,660::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15917053::DEBUG::2013-11-20 04:52:28,661::utils::579::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-kvm-rhev' (cwd None)
>Thread-15917053::DEBUG::2013-11-20 04:52:29,097::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15917053::DEBUG::2013-11-20 04:52:29,101::utils::579::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" vdsm' (cwd None)
>Thread-15917053::DEBUG::2013-11-20 04:52:29,543::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15917053::DEBUG::2013-11-20 04:52:29,545::utils::579::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" spice-server' (cwd None)
>Thread-15917053::DEBUG::2013-11-20 04:52:29,963::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15917053::DEBUG::2013-11-20 04:52:29,965::utils::579::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" libvirt' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:52:30,189::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15917053::DEBUG::2013-11-20 04:52:30,550::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15917053::DEBUG::2013-11-20 04:52:30,559::clientIF::59::vds::(wrapper) return getVdsCapabilities with {'status': {'message': 'Done', 'code': 0}, 'info': {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:c936ebb147a8'}], 'FC': []}, 'packages2': {'kernel': {'release': '279.11.1.el6.x86_64', 'buildtime': 1348297826.0, 'version': '2.6.32'}, 'spice-server': {'release': '10.el6', 'buildtime': '1337611492', 'version': '0.10.1'}, 'vdsm': {'release': '113.4.el6_3', 'buildtime': '1347534072', 'version': '4.9'}, 'qemu-kvm': {'release': '2.295.el6_3.2', 'buildtime': '1346673796', 'version': '0.12.1.2'}, 'libvirt': {'release': '21.el6_3.5', 'buildtime': '1348687577', 'version': '0.9.10'}, 'qemu-img': {'release': '2.295.el6_3.2', 'buildtime': '1346673796', 'version': '0.12.1.2'}}, 'cpuModel': 'Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz', 'hooks': {'before_vm_start': {'50_vhostmd': {'md5': '1635e45584995d9ad2629939cb0bcdbf'}}, 'after_vm_destroy': {'50_vhostmd': {'md5': '7e7509430304099f38e4a9de966af090'}}, 'before_vm_dehibernate': {'50_vhostmd': {'md5': '1635e45584995d9ad2629939cb0bcdbf'}}, 'before_vm_migrate_destination': {'50_vhostmd': {'md5': '1635e45584995d9ad2629939cb0bcdbf'}}}, 'vmTypes': ['kvm'], 'supportedProtocols': ['2.2', '2.3'], 'networks': {'rhevm': {'addr': '192.168.100.133', 'cfg': {'IPADDR': '192.168.100.133', 'GATEWAY': '192.168.100.1', 'DELAY': '0', 'NETMASK': '255.255.128.0', 'STP': 'no', 'DEVICE': 'rhevm', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'ports': ['vnet0', 'vnet1', 'vnet2', 'vnet3', 'vnet4', 'bond0', 'vnet5', 'vnet6', 'vnet7', 'vnet8', 'vnet9', 'vnet10', 'vnet11', 'vnet12', 'vnet13'], 'netmask': '255.255.128.0', 'stp': 'off', 'gateway': '192.168.100.1'}, 'DMZ': {'addr': '172.16.1.133', 'cfg': {'IPADDR': '172.16.1.133', 'DELAY': '0', 'NETMASK': '255.255.255.0', 'STP': 'no', 'DEVICE': 'DMZ', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'ports': ['eth0'], 'netmask': '255.255.255.0', 'stp': 'off', 'gateway': '0.0.0.0'}}, 'uuid': '30343536-3138-5A43-4A32-333630373442_AC:16:2D:76:EB:F0', 'lastClientIface': 'rhevm', 'nics': {'eth3': {'hwaddr': 'AC:16:2D:76:EB:F2', 'netmask': '', 'speed': 1000, 'addr': '', 'permhwaddr': 'AC:16:2D:76:EB:F3'}, 'eth2': {'hwaddr': 'AC:16:2D:76:EB:F2', 'netmask': '', 'speed': 1000, 'addr': '', 'permhwaddr': 'AC:16:2D:76:EB:F2'}, 'eth1': {'hwaddr': 'AC:16:2D:76:EB:F1', 'netmask': '', 'speed': 0, 'addr': ''}, 'eth0': {'hwaddr': 'AC:16:2D:76:EB:F0', 'netmask': '', 'speed': 1000, 'addr': ''}}, 'software_revision': '113.4', 'management_ip': '192.168.100.133', 'clusterLevels': ['3.0'], 'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,constant_tsc,arch_perfmon,pebs,bts,rep_good,xtopology,nonstop_tsc,aperfmperf,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,dca,sse4_1,sse4_2,x2apic,popcnt,aes,xsave,avx,lahf_lm,ida,arat,epb,xsaveopt,pln,pts,dts,tpr_shadow,vnmi,flexpriority,ept,vpid,model_486,model_pentium,model_pentium2,model_pentium3,model_pentiumpro,model_qemu32,model_cpu64-rhel5,model_cpu64-rhel6,model_kvm32,model_coreduo,model_kvm64,model_core2duo,model_n270,model_Conroe,model_Penryn,model_Nehalem,model_Westmere,model_SandyBridge,model_Opteron_G1', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:c936ebb147a8', 'memSize': '64392', 'reservedMem': '256', 'bondings': {'bond4': {'hwaddr': '00:00:00:00:00:00', 'cfg': {}, 'netmask': '', 'addr': '', 'slaves': []}, 'bond0': {'hwaddr': 'AC:16:2D:76:EB:F2', 'cfg': {'DEVICE': 'bond0', 'BRIDGE': 'rhevm', 'BOOTPROTO': 'none', 'ONBOOT': 'yes', 'BONDING_OPTS': 'mode=1 miimon=100'}, 'netmask': '', 'addr': '', 'slaves': ['eth2', 'eth3']}, 'bond1': {'hwaddr': '00:00:00:00:00:00', 'cfg': {}, 'netmask': '', 'addr': '', 'slaves': []}, 'bond2': {'hwaddr': '00:00:00:00:00:00', 'cfg': {}, 'netmask': '', 'addr': '', 'slaves': []}, 'bond3': {'hwaddr': '00:00:00:00:00:00', 'cfg': {}, 'netmask': '', 'addr': '', 'slaves': []}}, 'software_version': '4.9', 'cpuSpeed': '1995.126', 'cpuSockets': '2', 'vlans': {}, 'cpuCores': '12', 'kvmEnabled': 'true', 'guestOverhead': '65', 'supportedRHEVMs': ['3.0'], 'version_name': 'Snow Man', 'emulatedMachines': [u'rhel6.3.0', u'pc', u'rhel6.2.0', u'rhel6.1.0', u'rhel6.0.0', u'rhel5.5.0', u'rhel5.4.4', u'rhel5.4.0'], 'operatingSystem': {'release': '20121012.0.el6_3', 'version': '6.3', 'name': 'RHEV Hypervisor'}, 'lastClient': '192.168.100.135'}}
>Dummy-15912651::DEBUG::2013-11-20 04:52:30,882::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0811776 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 04:52:32,577::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:52:32,579::task::495::TaskManager.Task::(_debug) Task 1cd7448f-a46b-4cf6-ae00-44e4b31b65fd: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:52:32,581::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ca25c1fd-cd80-47d1-b74b-541357f7ece4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:52:32,581::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:52:32,583::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:52:32,583::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ca25c1fd-cd80-47d1-b74b-541357f7ece4`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:52:32,585::task::495::TaskManager.Task::(_debug) Task 1cd7448f-a46b-4cf6-ae00-44e4b31b65fd: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:52:32,586::task::495::TaskManager.Task::(_debug) Task 1cd7448f-a46b-4cf6-ae00-44e4b31b65fd: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:52:32,586::task::495::TaskManager.Task::(_debug) Task 1cd7448f-a46b-4cf6-ae00-44e4b31b65fd: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:52:32,587::task::495::TaskManager.Task::(_debug) Task 1cd7448f-a46b-4cf6-ae00-44e4b31b65fd: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:52:32,588::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:52:32,589::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:52:32,590::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:52:32,590::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:52:32,591::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:52:32,592::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:52:32,592::task::495::TaskManager.Task::(_debug) Task 1cd7448f-a46b-4cf6-ae00-44e4b31b65fd: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:52:32,593::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913518::INFO::2013-11-20 04:52:32,630::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:52:32,631::task::495::TaskManager.Task::(_debug) Task 1fb3f5de-a13e-425d-92af-7c7373d8718a: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:52:32,632::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f8eb5c69-407d-4870-9d68-21fec418304e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:52:32,633::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:52:32,633::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:52:32,634::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f8eb5c69-407d-4870-9d68-21fec418304e`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:52:32,635::task::495::TaskManager.Task::(_debug) Task 1fb3f5de-a13e-425d-92af-7c7373d8718a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:52:32,635::task::495::TaskManager.Task::(_debug) Task 1fb3f5de-a13e-425d-92af-7c7373d8718a: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:52:32,635::task::495::TaskManager.Task::(_debug) Task 1fb3f5de-a13e-425d-92af-7c7373d8718a: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:52:32,636::task::495::TaskManager.Task::(_debug) Task 1fb3f5de-a13e-425d-92af-7c7373d8718a: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:52:32,636::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:52:32,637::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:52:32,637::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:52:32,637::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:52:32,638::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:52:32,638::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:52:32,639::task::495::TaskManager.Task::(_debug) Task 1fb3f5de-a13e-425d-92af-7c7373d8718a: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:52:32,639::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:52:32,639::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:52:32,640::task::495::TaskManager.Task::(_debug) Task 8f0f5d55-b874-42fa-9975-420dda825763: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:52:32,641::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`42a7f582-b00f-42a6-bc03-de22f0fcc5e3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:52:32,641::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:52:32,642::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:52:32,642::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`42a7f582-b00f-42a6-bc03-de22f0fcc5e3`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:52:32,643::task::495::TaskManager.Task::(_debug) Task 8f0f5d55-b874-42fa-9975-420dda825763: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:52:32,643::task::495::TaskManager.Task::(_debug) Task 8f0f5d55-b874-42fa-9975-420dda825763: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:52:32,643::task::495::TaskManager.Task::(_debug) Task 8f0f5d55-b874-42fa-9975-420dda825763: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:52:32,644::task::495::TaskManager.Task::(_debug) Task 8f0f5d55-b874-42fa-9975-420dda825763: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:52:32,644::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:52:32,645::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:52:32,645::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:52:32,645::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:52:32,646::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:52:32,646::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:52:32,646::task::495::TaskManager.Task::(_debug) Task 8f0f5d55-b874-42fa-9975-420dda825763: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:52:32,647::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:52:32,896::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:52:33,406::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0830018 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:52:35,441::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15917055::DEBUG::2013-11-20 04:52:35,871::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917055::INFO::2013-11-20 04:52:35,873::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917055::DEBUG::2013-11-20 04:52:35,874::task::495::TaskManager.Task::(_debug) Task a73e4b5c-040f-4d50-a8f9-a0ca233cf44d: moving from state init -> state preparing
>Thread-15917055::DEBUG::2013-11-20 04:52:35,875::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917055::DEBUG::2013-11-20 04:52:35,887::task::495::TaskManager.Task::(_debug) Task a73e4b5c-040f-4d50-a8f9-a0ca233cf44d: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917055::DEBUG::2013-11-20 04:52:35,888::task::495::TaskManager.Task::(_debug) Task a73e4b5c-040f-4d50-a8f9-a0ca233cf44d: moving from state preparing -> state finished
>Thread-15917055::DEBUG::2013-11-20 04:52:35,889::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917055::DEBUG::2013-11-20 04:52:35,889::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917055::DEBUG::2013-11-20 04:52:35,891::task::495::TaskManager.Task::(_debug) Task a73e4b5c-040f-4d50-a8f9-a0ca233cf44d: ref 0 aborting False
>Thread-15917055::INFO::2013-11-20 04:52:35,896::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Dummy-15912651::DEBUG::2013-11-20 04:52:35,989::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0996738 s, 10.3 MB/s\n'; <rc> = 0
>Thread-15917056::DEBUG::2013-11-20 04:52:36,016::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917056::INFO::2013-11-20 04:52:36,019::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917056::DEBUG::2013-11-20 04:52:36,025::task::495::TaskManager.Task::(_debug) Task 386e2815-372c-490d-ac40-ce07f9084316: moving from state init -> state preparing
>Thread-15917056::DEBUG::2013-11-20 04:52:36,026::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6dbe951b-d72c-4aa4-b3c0-4fc40e6ae457`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917056::DEBUG::2013-11-20 04:52:36,027::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917056::DEBUG::2013-11-20 04:52:36,028::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917056::DEBUG::2013-11-20 04:52:36,029::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6dbe951b-d72c-4aa4-b3c0-4fc40e6ae457`::Granted request
>Thread-15917056::DEBUG::2013-11-20 04:52:36,047::task::495::TaskManager.Task::(_debug) Task 386e2815-372c-490d-ac40-ce07f9084316: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917056::DEBUG::2013-11-20 04:52:36,048::task::495::TaskManager.Task::(_debug) Task 386e2815-372c-490d-ac40-ce07f9084316: ref 1 aborting False
>Thread-15917056::WARNING::2013-11-20 04:52:36,051::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917056::DEBUG::2013-11-20 04:52:36,052::task::495::TaskManager.Task::(_debug) Task 386e2815-372c-490d-ac40-ce07f9084316: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917056::DEBUG::2013-11-20 04:52:36,053::task::495::TaskManager.Task::(_debug) Task 386e2815-372c-490d-ac40-ce07f9084316: moving from state preparing -> state finished
>Thread-15917056::DEBUG::2013-11-20 04:52:36,054::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917056::DEBUG::2013-11-20 04:52:36,054::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917056::DEBUG::2013-11-20 04:52:36,055::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917056::DEBUG::2013-11-20 04:52:36,059::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917056::DEBUG::2013-11-20 04:52:36,060::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917056::DEBUG::2013-11-20 04:52:36,063::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917056::DEBUG::2013-11-20 04:52:36,064::task::495::TaskManager.Task::(_debug) Task 386e2815-372c-490d-ac40-ce07f9084316: ref 0 aborting False
>Thread-15917056::INFO::2013-11-20 04:52:36,068::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917057::DEBUG::2013-11-20 04:52:37,453::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917057::INFO::2013-11-20 04:52:37,464::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksInfo, args: ()
>Thread-15917057::DEBUG::2013-11-20 04:52:37,468::task::495::TaskManager.Task::(_debug) Task 8c5fa22d-3d2c-44d8-9e28-20dce100cdfa: moving from state init -> state preparing
>Thread-15917057::DEBUG::2013-11-20 04:52:37,472::taskManager::168::TaskManager::(getAllTasksInfo) Entry.
>Thread-15917057::DEBUG::2013-11-20 04:52:37,485::taskManager::173::TaskManager::(getAllTasksInfo) Return. Response: {}
>Thread-15917057::DEBUG::2013-11-20 04:52:37,489::task::495::TaskManager.Task::(_debug) Task 8c5fa22d-3d2c-44d8-9e28-20dce100cdfa: finished: {'allTasksInfo': {}}
>Thread-15917057::DEBUG::2013-11-20 04:52:37,493::task::495::TaskManager.Task::(_debug) Task 8c5fa22d-3d2c-44d8-9e28-20dce100cdfa: moving from state preparing -> state finished
>Thread-15917057::DEBUG::2013-11-20 04:52:37,498::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917057::DEBUG::2013-11-20 04:52:37,499::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917057::DEBUG::2013-11-20 04:52:37,500::task::495::TaskManager.Task::(_debug) Task 8c5fa22d-3d2c-44d8-9e28-20dce100cdfa: ref 0 aborting False
>Thread-15917057::INFO::2013-11-20 04:52:37,501::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'allTasksInfo': {}}
>Dummy-15912651::DEBUG::2013-11-20 04:52:38,005::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:52:38,598::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0809731 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:52:40,616::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:52:41,171::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.07504 s, 13.6 MB/s\n'; <rc> = 0
>Thread-15914453::INFO::2013-11-20 04:52:42,596::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:52:42,601::task::495::TaskManager.Task::(_debug) Task debe1fd0-42ae-4321-ae9a-0a88e30889d6: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:52:42,604::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1b62fa17-ea78-4f02-b00b-b53973c206e4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:52:42,604::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:52:42,605::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:52:42,606::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1b62fa17-ea78-4f02-b00b-b53973c206e4`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:52:42,608::task::495::TaskManager.Task::(_debug) Task debe1fd0-42ae-4321-ae9a-0a88e30889d6: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:52:42,608::task::495::TaskManager.Task::(_debug) Task debe1fd0-42ae-4321-ae9a-0a88e30889d6: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:52:42,609::task::495::TaskManager.Task::(_debug) Task debe1fd0-42ae-4321-ae9a-0a88e30889d6: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:52:42,610::task::495::TaskManager.Task::(_debug) Task debe1fd0-42ae-4321-ae9a-0a88e30889d6: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:52:42,610::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:52:42,611::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:52:42,612::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:52:42,612::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:52:42,613::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:52:42,613::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:52:42,614::task::495::TaskManager.Task::(_debug) Task debe1fd0-42ae-4321-ae9a-0a88e30889d6: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:52:42,615::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:52:43,185::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 04:52:43,397::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:52:43,399::task::495::TaskManager.Task::(_debug) Task 59ecdfc4-86da-44f1-a505-00694dbd54ed: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:52:43,401::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`43757510-d934-4a1a-838a-b46d6510ade3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:52:43,402::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:52:43,403::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:52:43,404::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`43757510-d934-4a1a-838a-b46d6510ade3`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:52:43,405::task::495::TaskManager.Task::(_debug) Task 59ecdfc4-86da-44f1-a505-00694dbd54ed: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:52:43,406::task::495::TaskManager.Task::(_debug) Task 59ecdfc4-86da-44f1-a505-00694dbd54ed: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:52:43,407::task::495::TaskManager.Task::(_debug) Task 59ecdfc4-86da-44f1-a505-00694dbd54ed: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:52:43,407::task::495::TaskManager.Task::(_debug) Task 59ecdfc4-86da-44f1-a505-00694dbd54ed: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:52:43,408::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:52:43,409::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:52:43,409::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:52:43,410::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:52:43,411::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:52:43,411::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:52:43,412::task::495::TaskManager.Task::(_debug) Task 59ecdfc4-86da-44f1-a505-00694dbd54ed: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:52:43,413::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:52:43,720::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0727169 s, 14.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:52:45,736::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:52:46,298::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.106028 s, 9.7 MB/s\n'; <rc> = 0
>Thread-15917061::DEBUG::2013-11-20 04:52:47,444::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917061::INFO::2013-11-20 04:52:47,449::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917061::DEBUG::2013-11-20 04:52:47,450::task::495::TaskManager.Task::(_debug) Task 4d37e71e-3c0a-4ef8-91db-89cf6e94b48e: moving from state init -> state preparing
>Thread-15917061::DEBUG::2013-11-20 04:52:47,451::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917061::DEBUG::2013-11-20 04:52:47,452::task::495::TaskManager.Task::(_debug) Task 4d37e71e-3c0a-4ef8-91db-89cf6e94b48e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917061::DEBUG::2013-11-20 04:52:47,456::task::495::TaskManager.Task::(_debug) Task 4d37e71e-3c0a-4ef8-91db-89cf6e94b48e: moving from state preparing -> state finished
>Thread-15917061::DEBUG::2013-11-20 04:52:47,457::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917061::DEBUG::2013-11-20 04:52:47,458::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917061::DEBUG::2013-11-20 04:52:47,459::task::495::TaskManager.Task::(_debug) Task 4d37e71e-3c0a-4ef8-91db-89cf6e94b48e: ref 0 aborting False
>Thread-15917061::INFO::2013-11-20 04:52:47,460::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917062::DEBUG::2013-11-20 04:52:47,469::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917062::INFO::2013-11-20 04:52:47,473::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917062::DEBUG::2013-11-20 04:52:47,474::task::495::TaskManager.Task::(_debug) Task 93f84fa9-2d1e-4103-b44a-5e746d609597: moving from state init -> state preparing
>Thread-15917062::DEBUG::2013-11-20 04:52:47,476::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c94bc2d4-7b56-4eb4-b0bb-52fd17fec7e2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917062::DEBUG::2013-11-20 04:52:47,477::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917062::DEBUG::2013-11-20 04:52:47,483::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917062::DEBUG::2013-11-20 04:52:47,484::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c94bc2d4-7b56-4eb4-b0bb-52fd17fec7e2`::Granted request
>Thread-15917062::DEBUG::2013-11-20 04:52:47,485::task::495::TaskManager.Task::(_debug) Task 93f84fa9-2d1e-4103-b44a-5e746d609597: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917062::DEBUG::2013-11-20 04:52:47,486::task::495::TaskManager.Task::(_debug) Task 93f84fa9-2d1e-4103-b44a-5e746d609597: ref 1 aborting False
>Thread-15917062::WARNING::2013-11-20 04:52:47,492::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917062::DEBUG::2013-11-20 04:52:47,493::task::495::TaskManager.Task::(_debug) Task 93f84fa9-2d1e-4103-b44a-5e746d609597: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917062::DEBUG::2013-11-20 04:52:47,494::task::495::TaskManager.Task::(_debug) Task 93f84fa9-2d1e-4103-b44a-5e746d609597: moving from state preparing -> state finished
>Thread-15917062::DEBUG::2013-11-20 04:52:47,495::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917062::DEBUG::2013-11-20 04:52:47,496::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917062::DEBUG::2013-11-20 04:52:47,497::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917062::DEBUG::2013-11-20 04:52:47,497::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917062::DEBUG::2013-11-20 04:52:47,498::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917062::DEBUG::2013-11-20 04:52:47,504::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917062::DEBUG::2013-11-20 04:52:47,505::task::495::TaskManager.Task::(_debug) Task 93f84fa9-2d1e-4103-b44a-5e746d609597: ref 0 aborting False
>Thread-15917062::INFO::2013-11-20 04:52:47,505::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:52:48,312::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:52:48,833::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0750871 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:52:50,848::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:52:51,435::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0804563 s, 12.7 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 04:52:52,584::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:52:52,585::task::495::TaskManager.Task::(_debug) Task fde1a89e-01c2-43f7-a366-a155548d3a0d: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:52:52,587::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`0d2a819f-229b-4451-98f5-da90b421d703`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:52:52,588::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:52:52,589::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:52:52,589::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`0d2a819f-229b-4451-98f5-da90b421d703`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:52:52,590::task::495::TaskManager.Task::(_debug) Task fde1a89e-01c2-43f7-a366-a155548d3a0d: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:52:52,591::task::495::TaskManager.Task::(_debug) Task fde1a89e-01c2-43f7-a366-a155548d3a0d: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:52:52,592::task::495::TaskManager.Task::(_debug) Task fde1a89e-01c2-43f7-a366-a155548d3a0d: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::DEBUG::2013-11-20 04:52:52,593::task::495::TaskManager.Task::(_debug) Task fde1a89e-01c2-43f7-a366-a155548d3a0d: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:52:52,593::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:52:52,594::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:52:52,595::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:52:52,595::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:52:52,596::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:52:52,596::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:52:52,597::task::495::TaskManager.Task::(_debug) Task fde1a89e-01c2-43f7-a366-a155548d3a0d: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:52:52,598::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:52:52,599::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:52:52,599::task::495::TaskManager.Task::(_debug) Task 480ef3ce-d62d-44ff-9bc9-e15ebd008905: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:52:52,600::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`9bb79f66-dfec-4589-9983-13838522b8c5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:52:52,601::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:52:52,601::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:52:52,602::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`9bb79f66-dfec-4589-9983-13838522b8c5`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:52:52,603::task::495::TaskManager.Task::(_debug) Task 480ef3ce-d62d-44ff-9bc9-e15ebd008905: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:52:52,603::task::495::TaskManager.Task::(_debug) Task 480ef3ce-d62d-44ff-9bc9-e15ebd008905: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:52:52,604::task::495::TaskManager.Task::(_debug) Task 480ef3ce-d62d-44ff-9bc9-e15ebd008905: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:52:52,605::task::495::TaskManager.Task::(_debug) Task 480ef3ce-d62d-44ff-9bc9-e15ebd008905: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:52:52,606::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:52:52,607::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:52:52,607::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:52:52,622::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:52:52,623::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:52:52,624::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:52:52,625::task::495::TaskManager.Task::(_debug) Task 480ef3ce-d62d-44ff-9bc9-e15ebd008905: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:52:52,626::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:52:52,627::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:52:52,627::task::495::TaskManager.Task::(_debug) Task b48cf9ee-b918-454d-95b1-0905e59e251f: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:52:52,629::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`55aa3a77-aa84-4f08-9b58-f102544c73e5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::INFO::2013-11-20 04:52:52,630::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15913546::DEBUG::2013-11-20 04:52:52,631::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:52:52,632::task::495::TaskManager.Task::(_debug) Task e42b997c-a94c-478d-b566-b27e0342c121: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:52:52,633::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:52:52,635::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`55aa3a77-aa84-4f08-9b58-f102544c73e5`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:52:52,636::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9be5568f-898c-4007-b10f-496965c62384`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:52:52,638::task::495::TaskManager.Task::(_debug) Task b48cf9ee-b918-454d-95b1-0905e59e251f: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15914519::DEBUG::2013-11-20 04:52:52,638::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:52:52,639::task::495::TaskManager.Task::(_debug) Task b48cf9ee-b918-454d-95b1-0905e59e251f: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:52:52,640::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:52:52,641::task::495::TaskManager.Task::(_debug) Task b48cf9ee-b918-454d-95b1-0905e59e251f: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15914519::DEBUG::2013-11-20 04:52:52,641::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9be5568f-898c-4007-b10f-496965c62384`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:52:52,642::task::495::TaskManager.Task::(_debug) Task b48cf9ee-b918-454d-95b1-0905e59e251f: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:52:52,649::task::495::TaskManager.Task::(_debug) Task e42b997c-a94c-478d-b566-b27e0342c121: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913546::DEBUG::2013-11-20 04:52:52,650::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:52:52,651::task::495::TaskManager.Task::(_debug) Task e42b997c-a94c-478d-b566-b27e0342c121: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:52:52,652::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:52:52,653::task::495::TaskManager.Task::(_debug) Task e42b997c-a94c-478d-b566-b27e0342c121: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:52:52,655::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15914519::DEBUG::2013-11-20 04:52:52,655::task::495::TaskManager.Task::(_debug) Task e42b997c-a94c-478d-b566-b27e0342c121: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:52:52,657::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:52:52,657::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:52:52,658::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:52:52,659::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:52:52,660::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:52:52,661::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913546::DEBUG::2013-11-20 04:52:52,662::task::495::TaskManager.Task::(_debug) Task b48cf9ee-b918-454d-95b1-0905e59e251f: ref 0 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:52:52,663::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913546::INFO::2013-11-20 04:52:52,664::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15914519::DEBUG::2013-11-20 04:52:52,664::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913546::INFO::2013-11-20 04:52:52,665::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15914519::DEBUG::2013-11-20 04:52:52,666::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:52:52,666::task::495::TaskManager.Task::(_debug) Task 2df07624-a945-4eca-94d7-740e8c0c6fd9: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:52:52,667::task::495::TaskManager.Task::(_debug) Task e42b997c-a94c-478d-b566-b27e0342c121: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:52:52,669::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`398e7d6f-3832-4903-9e34-e50a32406068`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::INFO::2013-11-20 04:52:52,670::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:52:52,670::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:52:52,671::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:52:52,672::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`398e7d6f-3832-4903-9e34-e50a32406068`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:52:52,673::task::495::TaskManager.Task::(_debug) Task 2df07624-a945-4eca-94d7-740e8c0c6fd9: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:52:52,674::task::495::TaskManager.Task::(_debug) Task 2df07624-a945-4eca-94d7-740e8c0c6fd9: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:52:52,675::task::495::TaskManager.Task::(_debug) Task 2df07624-a945-4eca-94d7-740e8c0c6fd9: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:52:52,675::task::495::TaskManager.Task::(_debug) Task 2df07624-a945-4eca-94d7-740e8c0c6fd9: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:52:52,676::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:52:52,676::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:52:52,677::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:52:52,678::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:52:52,678::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:52:52,679::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:52:52,679::task::495::TaskManager.Task::(_debug) Task 2df07624-a945-4eca-94d7-740e8c0c6fd9: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:52:52,680::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15917064::INFO::2013-11-20 04:52:52,823::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15917064::DEBUG::2013-11-20 04:52:52,829::task::495::TaskManager.Task::(_debug) Task 4562cadc-44a9-428e-a016-5705160d2ebd: moving from state init -> state preparing
>Thread-15917064::DEBUG::2013-11-20 04:52:52,832::task::495::TaskManager.Task::(_debug) Task 4562cadc-44a9-428e-a016-5705160d2ebd: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000362873077393', 'lastCheck': 1384923167.386796, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00199604034424', 'lastCheck': 1384923168.5194099, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00200915336609', 'lastCheck': 1384923168.5180659, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000523090362549', 'lastCheck': 1384923171.5076699, 'valid': True, 'code': 0}}
>Thread-15917064::DEBUG::2013-11-20 04:52:52,839::task::495::TaskManager.Task::(_debug) Task 4562cadc-44a9-428e-a016-5705160d2ebd: moving from state preparing -> state finished
>Thread-15917064::DEBUG::2013-11-20 04:52:52,839::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917064::DEBUG::2013-11-20 04:52:52,840::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917064::DEBUG::2013-11-20 04:52:52,840::task::495::TaskManager.Task::(_debug) Task 4562cadc-44a9-428e-a016-5705160d2ebd: ref 0 aborting False
>Thread-15917064::INFO::2013-11-20 04:52:52,841::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000362873077393', 'lastCheck': 1384923167.386796, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00199604034424', 'lastCheck': 1384923168.5194099, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00200915336609', 'lastCheck': 1384923168.5180659, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000523090362549', 'lastCheck': 1384923171.5076699, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:52:53,450::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:52:54,032::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0835767 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:52:56,046::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:52:56,577::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0766357 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15917067::DEBUG::2013-11-20 04:52:57,620::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917067::INFO::2013-11-20 04:52:57,621::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917067::DEBUG::2013-11-20 04:52:57,631::task::495::TaskManager.Task::(_debug) Task d44f4c67-0826-45cd-989b-8b124b07f533: moving from state init -> state preparing
>Thread-15917067::DEBUG::2013-11-20 04:52:57,633::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917067::DEBUG::2013-11-20 04:52:57,633::task::495::TaskManager.Task::(_debug) Task d44f4c67-0826-45cd-989b-8b124b07f533: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917067::DEBUG::2013-11-20 04:52:57,634::task::495::TaskManager.Task::(_debug) Task d44f4c67-0826-45cd-989b-8b124b07f533: moving from state preparing -> state finished
>Thread-15917067::DEBUG::2013-11-20 04:52:57,640::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917067::DEBUG::2013-11-20 04:52:57,642::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917067::DEBUG::2013-11-20 04:52:57,644::task::495::TaskManager.Task::(_debug) Task d44f4c67-0826-45cd-989b-8b124b07f533: ref 0 aborting False
>Thread-15917067::INFO::2013-11-20 04:52:57,645::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917068::DEBUG::2013-11-20 04:52:57,654::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917068::INFO::2013-11-20 04:52:57,656::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917068::DEBUG::2013-11-20 04:52:57,658::task::495::TaskManager.Task::(_debug) Task cb97843f-4365-437f-a882-8300ec5f251b: moving from state init -> state preparing
>Thread-15917068::DEBUG::2013-11-20 04:52:57,666::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6056c259-8026-4394-a33b-15de00010818`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917068::DEBUG::2013-11-20 04:52:57,670::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917068::DEBUG::2013-11-20 04:52:57,673::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917068::DEBUG::2013-11-20 04:52:57,677::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`6056c259-8026-4394-a33b-15de00010818`::Granted request
>Thread-15917068::DEBUG::2013-11-20 04:52:57,680::task::495::TaskManager.Task::(_debug) Task cb97843f-4365-437f-a882-8300ec5f251b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917068::DEBUG::2013-11-20 04:52:57,681::task::495::TaskManager.Task::(_debug) Task cb97843f-4365-437f-a882-8300ec5f251b: ref 1 aborting False
>Thread-15917068::WARNING::2013-11-20 04:52:57,682::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917068::DEBUG::2013-11-20 04:52:57,682::task::495::TaskManager.Task::(_debug) Task cb97843f-4365-437f-a882-8300ec5f251b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917068::DEBUG::2013-11-20 04:52:57,683::task::495::TaskManager.Task::(_debug) Task cb97843f-4365-437f-a882-8300ec5f251b: moving from state preparing -> state finished
>Thread-15917068::DEBUG::2013-11-20 04:52:57,683::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917068::DEBUG::2013-11-20 04:52:57,684::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917068::DEBUG::2013-11-20 04:52:57,684::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917068::DEBUG::2013-11-20 04:52:57,684::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917068::DEBUG::2013-11-20 04:52:57,685::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917068::DEBUG::2013-11-20 04:52:57,690::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917068::DEBUG::2013-11-20 04:52:57,691::task::495::TaskManager.Task::(_debug) Task cb97843f-4365-437f-a882-8300ec5f251b: ref 0 aborting False
>Thread-15917068::INFO::2013-11-20 04:52:57,691::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:52:58,599::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:52:59,190::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0915969 s, 11.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:53:01,200::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 04:53:01,463::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:53:01,466::task::495::TaskManager.Task::(_debug) Task 062c3ecf-ab94-4f0e-b5c4-c5e6d9e9b052: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:53:01,473::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`549e86fe-f80b-4fc2-b369-b2872166f777`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:53:01,479::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:53:01,482::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:53:01,484::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`549e86fe-f80b-4fc2-b369-b2872166f777`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:53:01,487::task::495::TaskManager.Task::(_debug) Task 062c3ecf-ab94-4f0e-b5c4-c5e6d9e9b052: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:53:01,489::task::495::TaskManager.Task::(_debug) Task 062c3ecf-ab94-4f0e-b5c4-c5e6d9e9b052: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:53:01,489::task::495::TaskManager.Task::(_debug) Task 062c3ecf-ab94-4f0e-b5c4-c5e6d9e9b052: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:53:01,490::task::495::TaskManager.Task::(_debug) Task 062c3ecf-ab94-4f0e-b5c4-c5e6d9e9b052: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:53:01,492::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:53:01,495::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:53:01,510::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:53:01,513::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:53:01,514::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:53:01,515::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:53:01,516::task::495::TaskManager.Task::(_debug) Task 062c3ecf-ab94-4f0e-b5c4-c5e6d9e9b052: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:53:01,517::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:53:01,762::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0521409 s, 19.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:53:03,772::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:53:04,321::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0998163 s, 10.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:53:06,333::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:53:06,883::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0819499 s, 12.5 MB/s\n'; <rc> = 0
>Thread-15917072::DEBUG::2013-11-20 04:53:07,794::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917072::INFO::2013-11-20 04:53:07,795::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917072::DEBUG::2013-11-20 04:53:07,796::task::495::TaskManager.Task::(_debug) Task 54f87bef-4c17-4c61-8dcb-581785382556: moving from state init -> state preparing
>Thread-15917072::DEBUG::2013-11-20 04:53:07,797::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917072::DEBUG::2013-11-20 04:53:07,798::task::495::TaskManager.Task::(_debug) Task 54f87bef-4c17-4c61-8dcb-581785382556: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917072::DEBUG::2013-11-20 04:53:07,798::task::495::TaskManager.Task::(_debug) Task 54f87bef-4c17-4c61-8dcb-581785382556: moving from state preparing -> state finished
>Thread-15917072::DEBUG::2013-11-20 04:53:07,799::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917072::DEBUG::2013-11-20 04:53:07,799::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917072::DEBUG::2013-11-20 04:53:07,800::task::495::TaskManager.Task::(_debug) Task 54f87bef-4c17-4c61-8dcb-581785382556: ref 0 aborting False
>Thread-15917072::INFO::2013-11-20 04:53:07,801::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917073::DEBUG::2013-11-20 04:53:07,807::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917073::INFO::2013-11-20 04:53:07,808::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917073::DEBUG::2013-11-20 04:53:07,809::task::495::TaskManager.Task::(_debug) Task 692b4d4b-e599-469c-85f9-3fc519d961a3: moving from state init -> state preparing
>Thread-15917073::DEBUG::2013-11-20 04:53:07,810::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8c84014a-2eff-404f-8900-2d35f9186320`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917073::DEBUG::2013-11-20 04:53:07,811::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917073::DEBUG::2013-11-20 04:53:07,812::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917073::DEBUG::2013-11-20 04:53:07,812::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8c84014a-2eff-404f-8900-2d35f9186320`::Granted request
>Thread-15917073::DEBUG::2013-11-20 04:53:07,813::task::495::TaskManager.Task::(_debug) Task 692b4d4b-e599-469c-85f9-3fc519d961a3: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917073::DEBUG::2013-11-20 04:53:07,826::task::495::TaskManager.Task::(_debug) Task 692b4d4b-e599-469c-85f9-3fc519d961a3: ref 1 aborting False
>Thread-15917073::WARNING::2013-11-20 04:53:07,831::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917073::DEBUG::2013-11-20 04:53:07,832::task::495::TaskManager.Task::(_debug) Task 692b4d4b-e599-469c-85f9-3fc519d961a3: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917073::DEBUG::2013-11-20 04:53:07,835::task::495::TaskManager.Task::(_debug) Task 692b4d4b-e599-469c-85f9-3fc519d961a3: moving from state preparing -> state finished
>Thread-15917073::DEBUG::2013-11-20 04:53:07,836::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917073::DEBUG::2013-11-20 04:53:07,836::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917073::DEBUG::2013-11-20 04:53:07,837::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917073::DEBUG::2013-11-20 04:53:07,838::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917073::DEBUG::2013-11-20 04:53:07,839::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917073::DEBUG::2013-11-20 04:53:07,839::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917073::DEBUG::2013-11-20 04:53:07,840::task::495::TaskManager.Task::(_debug) Task 692b4d4b-e599-469c-85f9-3fc519d961a3: ref 0 aborting False
>Thread-15917073::INFO::2013-11-20 04:53:07,841::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917074::INFO::2013-11-20 04:53:08,454::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15917074::DEBUG::2013-11-20 04:53:08,455::task::495::TaskManager.Task::(_debug) Task b6ca65c2-32a0-4fc2-ba7a-9d857add0bad: moving from state init -> state preparing
>Thread-15917074::DEBUG::2013-11-20 04:53:08,456::task::495::TaskManager.Task::(_debug) Task b6ca65c2-32a0-4fc2-ba7a-9d857add0bad: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000425815582275', 'lastCheck': 1384923187.986625, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00171995162964', 'lastCheck': 1384923178.8290341, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00223994255066', 'lastCheck': 1384923178.8277481, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00049901008606', 'lastCheck': 1384923181.8135419, 'valid': True, 'code': 0}}
>Thread-15917074::DEBUG::2013-11-20 04:53:08,457::task::495::TaskManager.Task::(_debug) Task b6ca65c2-32a0-4fc2-ba7a-9d857add0bad: moving from state preparing -> state finished
>Thread-15917074::DEBUG::2013-11-20 04:53:08,458::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917074::DEBUG::2013-11-20 04:53:08,459::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917074::DEBUG::2013-11-20 04:53:08,459::task::495::TaskManager.Task::(_debug) Task b6ca65c2-32a0-4fc2-ba7a-9d857add0bad: ref 0 aborting False
>Thread-15917074::INFO::2013-11-20 04:53:08,460::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000425815582275', 'lastCheck': 1384923187.986625, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00171995162964', 'lastCheck': 1384923178.8290341, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00223994255066', 'lastCheck': 1384923178.8277481, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00049901008606', 'lastCheck': 1384923181.8135419, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:53:08,900::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:53:09,467::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0722056 s, 14.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:53:11,483::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:53:11,704::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:53:11,706::task::495::TaskManager.Task::(_debug) Task 37d786a7-dc5b-4964-a4b1-444c30ca16b9: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:53:11,709::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`03da45dd-de0a-4376-a626-ad6db7ecbd2f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:53:11,710::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:53:11,711::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:53:11,712::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`03da45dd-de0a-4376-a626-ad6db7ecbd2f`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:53:11,713::task::495::TaskManager.Task::(_debug) Task 37d786a7-dc5b-4964-a4b1-444c30ca16b9: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:53:11,714::task::495::TaskManager.Task::(_debug) Task 37d786a7-dc5b-4964-a4b1-444c30ca16b9: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:53:11,715::task::495::TaskManager.Task::(_debug) Task 37d786a7-dc5b-4964-a4b1-444c30ca16b9: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:53:11,717::task::495::TaskManager.Task::(_debug) Task 37d786a7-dc5b-4964-a4b1-444c30ca16b9: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:53:11,718::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:53:11,719::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:53:11,720::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:53:11,721::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:53:11,721::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:53:11,722::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:53:11,723::task::495::TaskManager.Task::(_debug) Task 37d786a7-dc5b-4964-a4b1-444c30ca16b9: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:53:11,723::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:53:12,031::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.081946 s, 12.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:53:14,046::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915918::INFO::2013-11-20 04:53:14,248::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:53:14,251::task::495::TaskManager.Task::(_debug) Task abbc55d7-4098-4cea-9e30-58858d997afb: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:53:14,255::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`13a4bc94-6fff-4cf6-a73c-c3536d2acad3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:53:14,262::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:53:14,264::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:53:14,266::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`13a4bc94-6fff-4cf6-a73c-c3536d2acad3`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:53:14,269::task::495::TaskManager.Task::(_debug) Task abbc55d7-4098-4cea-9e30-58858d997afb: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:53:14,270::task::495::TaskManager.Task::(_debug) Task abbc55d7-4098-4cea-9e30-58858d997afb: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:53:14,271::task::495::TaskManager.Task::(_debug) Task abbc55d7-4098-4cea-9e30-58858d997afb: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:53:14,272::task::495::TaskManager.Task::(_debug) Task abbc55d7-4098-4cea-9e30-58858d997afb: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:53:14,272::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:53:14,273::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:53:14,274::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:53:14,275::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:53:14,275::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:53:14,276::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:53:14,276::task::495::TaskManager.Task::(_debug) Task abbc55d7-4098-4cea-9e30-58858d997afb: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:53:14,277::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:53:14,568::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0775004 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:53:16,582::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916984::INFO::2013-11-20 04:53:16,788::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=240c0593-fe33-4f8b-a029-097379dbcb85 volUUID=f715faaa-081a-4598-b125-80a2d12af554)
>Thread-15916984::DEBUG::2013-11-20 04:53:16,792::task::495::TaskManager.Task::(_debug) Task 62b6cb66-eda5-4e27-baac-531742a7b957: moving from state init -> state preparing
>Thread-15916984::DEBUG::2013-11-20 04:53:16,800::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`84e8c253-3264-43ed-8c70-6d5b7f1c8879`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15916984::DEBUG::2013-11-20 04:53:16,803::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15916984::DEBUG::2013-11-20 04:53:16,804::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15916984::DEBUG::2013-11-20 04:53:16,805::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`84e8c253-3264-43ed-8c70-6d5b7f1c8879`::Granted request
>Thread-15916984::DEBUG::2013-11-20 04:53:16,807::task::495::TaskManager.Task::(_debug) Task 62b6cb66-eda5-4e27-baac-531742a7b957: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15916984::DEBUG::2013-11-20 04:53:16,813::task::495::TaskManager.Task::(_debug) Task 62b6cb66-eda5-4e27-baac-531742a7b957: ref 1 aborting False
>Thread-15916984::DEBUG::2013-11-20 04:53:16,814::task::495::TaskManager.Task::(_debug) Task 62b6cb66-eda5-4e27-baac-531742a7b957: finished: {'truesize': '75296145408', 'apparentsize': '75296145408'}
>Thread-15916984::DEBUG::2013-11-20 04:53:16,815::task::495::TaskManager.Task::(_debug) Task 62b6cb66-eda5-4e27-baac-531742a7b957: moving from state preparing -> state finished
>Thread-15916984::DEBUG::2013-11-20 04:53:16,815::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15916984::DEBUG::2013-11-20 04:53:16,816::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916984::DEBUG::2013-11-20 04:53:16,824::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15916984::DEBUG::2013-11-20 04:53:16,827::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15916984::DEBUG::2013-11-20 04:53:16,827::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15916984::DEBUG::2013-11-20 04:53:16,830::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15916984::DEBUG::2013-11-20 04:53:16,833::task::495::TaskManager.Task::(_debug) Task 62b6cb66-eda5-4e27-baac-531742a7b957: ref 0 aborting False
>Thread-15916984::INFO::2013-11-20 04:53:16,833::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '75296145408', 'apparentsize': '75296145408'}
>Dummy-15912651::DEBUG::2013-11-20 04:53:17,102::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0773087 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15917078::DEBUG::2013-11-20 04:53:17,958::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917078::INFO::2013-11-20 04:53:17,960::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917078::DEBUG::2013-11-20 04:53:17,960::task::495::TaskManager.Task::(_debug) Task d8d34b00-bd80-4b1f-a9ac-244b9f788919: moving from state init -> state preparing
>Thread-15917078::DEBUG::2013-11-20 04:53:17,961::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917078::DEBUG::2013-11-20 04:53:17,962::task::495::TaskManager.Task::(_debug) Task d8d34b00-bd80-4b1f-a9ac-244b9f788919: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917078::DEBUG::2013-11-20 04:53:17,963::task::495::TaskManager.Task::(_debug) Task d8d34b00-bd80-4b1f-a9ac-244b9f788919: moving from state preparing -> state finished
>Thread-15917078::DEBUG::2013-11-20 04:53:17,963::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917078::DEBUG::2013-11-20 04:53:17,964::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917078::DEBUG::2013-11-20 04:53:17,964::task::495::TaskManager.Task::(_debug) Task d8d34b00-bd80-4b1f-a9ac-244b9f788919: ref 0 aborting False
>Thread-15917078::INFO::2013-11-20 04:53:17,965::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917080::DEBUG::2013-11-20 04:53:17,975::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917080::INFO::2013-11-20 04:53:17,976::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917080::DEBUG::2013-11-20 04:53:17,976::task::495::TaskManager.Task::(_debug) Task f913f3b6-9777-46ad-ba0d-401549b1b4e1: moving from state init -> state preparing
>Thread-15917080::DEBUG::2013-11-20 04:53:17,978::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`737101b4-cf1a-43df-a68c-1887e84c91a2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917080::DEBUG::2013-11-20 04:53:17,978::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917080::DEBUG::2013-11-20 04:53:17,979::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917080::DEBUG::2013-11-20 04:53:17,980::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`737101b4-cf1a-43df-a68c-1887e84c91a2`::Granted request
>Thread-15917080::DEBUG::2013-11-20 04:53:17,981::task::495::TaskManager.Task::(_debug) Task f913f3b6-9777-46ad-ba0d-401549b1b4e1: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917080::DEBUG::2013-11-20 04:53:17,981::task::495::TaskManager.Task::(_debug) Task f913f3b6-9777-46ad-ba0d-401549b1b4e1: ref 1 aborting False
>Thread-15917080::WARNING::2013-11-20 04:53:17,983::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917080::DEBUG::2013-11-20 04:53:17,985::task::495::TaskManager.Task::(_debug) Task f913f3b6-9777-46ad-ba0d-401549b1b4e1: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917080::DEBUG::2013-11-20 04:53:17,986::task::495::TaskManager.Task::(_debug) Task f913f3b6-9777-46ad-ba0d-401549b1b4e1: moving from state preparing -> state finished
>Thread-15917080::DEBUG::2013-11-20 04:53:17,986::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917080::DEBUG::2013-11-20 04:53:17,987::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917080::DEBUG::2013-11-20 04:53:17,988::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917080::DEBUG::2013-11-20 04:53:17,988::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917080::DEBUG::2013-11-20 04:53:17,989::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917080::DEBUG::2013-11-20 04:53:17,990::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917080::DEBUG::2013-11-20 04:53:17,990::task::495::TaskManager.Task::(_debug) Task f913f3b6-9777-46ad-ba0d-401549b1b4e1: ref 0 aborting False
>Thread-15917080::INFO::2013-11-20 04:53:17,991::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:53:19,117::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915863::INFO::2013-11-20 04:53:19,361::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:53:19,365::task::495::TaskManager.Task::(_debug) Task 08b03285-0c2f-4cfb-abc6-f3f236a93d20: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:53:19,369::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b6792ce1-c1ce-4ddf-8d9e-4e4d349a313c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:53:19,371::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:53:19,372::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:53:19,373::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b6792ce1-c1ce-4ddf-8d9e-4e4d349a313c`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:53:19,375::task::495::TaskManager.Task::(_debug) Task 08b03285-0c2f-4cfb-abc6-f3f236a93d20: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:53:19,376::task::495::TaskManager.Task::(_debug) Task 08b03285-0c2f-4cfb-abc6-f3f236a93d20: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:53:19,377::task::495::TaskManager.Task::(_debug) Task 08b03285-0c2f-4cfb-abc6-f3f236a93d20: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:53:19,378::task::495::TaskManager.Task::(_debug) Task 08b03285-0c2f-4cfb-abc6-f3f236a93d20: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:53:19,379::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:53:19,380::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:53:19,381::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:53:19,382::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:53:19,382::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:53:19,383::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:53:19,384::task::495::TaskManager.Task::(_debug) Task 08b03285-0c2f-4cfb-abc6-f3f236a93d20: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:53:19,384::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:53:19,651::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0736679 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:53:21,666::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:53:22,198::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0847396 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 04:53:22,865::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:53:22,873::task::495::TaskManager.Task::(_debug) Task d4e42bdc-5ba7-4df9-b85b-ede7ca275541: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:53:22,876::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`da1a395e-bd25-41b8-a80c-02bf3dd5dac7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:53:22,876::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:53:22,877::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:53:22,878::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`da1a395e-bd25-41b8-a80c-02bf3dd5dac7`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:53:22,879::task::495::TaskManager.Task::(_debug) Task d4e42bdc-5ba7-4df9-b85b-ede7ca275541: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:53:22,880::task::495::TaskManager.Task::(_debug) Task d4e42bdc-5ba7-4df9-b85b-ede7ca275541: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:53:22,880::task::495::TaskManager.Task::(_debug) Task d4e42bdc-5ba7-4df9-b85b-ede7ca275541: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:53:22,881::task::495::TaskManager.Task::(_debug) Task d4e42bdc-5ba7-4df9-b85b-ede7ca275541: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:53:22,882::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:53:22,883::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:53:22,884::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:53:22,884::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:53:22,886::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:53:22,887::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:53:22,888::task::495::TaskManager.Task::(_debug) Task d4e42bdc-5ba7-4df9-b85b-ede7ca275541: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:53:22,889::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15917082::INFO::2013-11-20 04:53:24,027::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15917082::DEBUG::2013-11-20 04:53:24,028::task::495::TaskManager.Task::(_debug) Task 5fc3200d-0825-470d-a017-b60927edf288: moving from state init -> state preparing
>Thread-15917082::DEBUG::2013-11-20 04:53:24,029::task::495::TaskManager.Task::(_debug) Task 5fc3200d-0825-470d-a017-b60927edf288: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000373125076294', 'lastCheck': 1384923198.251951, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00320196151733', 'lastCheck': 1384923199.3704281, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00395894050598', 'lastCheck': 1384923199.370549, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000534057617188', 'lastCheck': 1384923202.329881, 'valid': True, 'code': 0}}
>Thread-15917082::DEBUG::2013-11-20 04:53:24,030::task::495::TaskManager.Task::(_debug) Task 5fc3200d-0825-470d-a017-b60927edf288: moving from state preparing -> state finished
>Thread-15917082::DEBUG::2013-11-20 04:53:24,031::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917082::DEBUG::2013-11-20 04:53:24,032::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917082::DEBUG::2013-11-20 04:53:24,032::task::495::TaskManager.Task::(_debug) Task 5fc3200d-0825-470d-a017-b60927edf288: ref 0 aborting False
>Thread-15917082::INFO::2013-11-20 04:53:24,033::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000373125076294', 'lastCheck': 1384923198.251951, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00320196151733', 'lastCheck': 1384923199.3704281, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00395894050598', 'lastCheck': 1384923199.370549, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000534057617188', 'lastCheck': 1384923202.329881, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:53:24,213::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:53:24,784::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0929064 s, 11.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:53:26,800::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:53:27,349::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0774543 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15917085::DEBUG::2013-11-20 04:53:28,157::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917085::INFO::2013-11-20 04:53:28,159::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917085::DEBUG::2013-11-20 04:53:28,159::task::495::TaskManager.Task::(_debug) Task 8c8169ed-af2e-4c36-a3da-8002ca50bc5a: moving from state init -> state preparing
>Thread-15917085::DEBUG::2013-11-20 04:53:28,160::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917085::DEBUG::2013-11-20 04:53:28,161::task::495::TaskManager.Task::(_debug) Task 8c8169ed-af2e-4c36-a3da-8002ca50bc5a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917085::DEBUG::2013-11-20 04:53:28,161::task::495::TaskManager.Task::(_debug) Task 8c8169ed-af2e-4c36-a3da-8002ca50bc5a: moving from state preparing -> state finished
>Thread-15917085::DEBUG::2013-11-20 04:53:28,162::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917085::DEBUG::2013-11-20 04:53:28,163::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917085::DEBUG::2013-11-20 04:53:28,163::task::495::TaskManager.Task::(_debug) Task 8c8169ed-af2e-4c36-a3da-8002ca50bc5a: ref 0 aborting False
>Thread-15917085::INFO::2013-11-20 04:53:28,164::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917086::DEBUG::2013-11-20 04:53:28,170::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917086::INFO::2013-11-20 04:53:28,171::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917086::DEBUG::2013-11-20 04:53:28,171::task::495::TaskManager.Task::(_debug) Task 2aa6f9ba-17c1-4c22-a8e5-a1301f649c64: moving from state init -> state preparing
>Thread-15917086::DEBUG::2013-11-20 04:53:28,173::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`824ca5ad-bd83-4c1e-bc7b-8a95c09f1aca`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917086::DEBUG::2013-11-20 04:53:28,174::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917086::DEBUG::2013-11-20 04:53:28,175::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917086::DEBUG::2013-11-20 04:53:28,175::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`824ca5ad-bd83-4c1e-bc7b-8a95c09f1aca`::Granted request
>Thread-15917086::DEBUG::2013-11-20 04:53:28,176::task::495::TaskManager.Task::(_debug) Task 2aa6f9ba-17c1-4c22-a8e5-a1301f649c64: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917086::DEBUG::2013-11-20 04:53:28,177::task::495::TaskManager.Task::(_debug) Task 2aa6f9ba-17c1-4c22-a8e5-a1301f649c64: ref 1 aborting False
>Thread-15917086::WARNING::2013-11-20 04:53:28,179::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917086::DEBUG::2013-11-20 04:53:28,181::task::495::TaskManager.Task::(_debug) Task 2aa6f9ba-17c1-4c22-a8e5-a1301f649c64: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917086::DEBUG::2013-11-20 04:53:28,182::task::495::TaskManager.Task::(_debug) Task 2aa6f9ba-17c1-4c22-a8e5-a1301f649c64: moving from state preparing -> state finished
>Thread-15917086::DEBUG::2013-11-20 04:53:28,182::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917086::DEBUG::2013-11-20 04:53:28,183::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917086::DEBUG::2013-11-20 04:53:28,184::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917086::DEBUG::2013-11-20 04:53:28,184::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917086::DEBUG::2013-11-20 04:53:28,185::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917086::DEBUG::2013-11-20 04:53:28,185::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917086::DEBUG::2013-11-20 04:53:28,186::task::495::TaskManager.Task::(_debug) Task 2aa6f9ba-17c1-4c22-a8e5-a1301f649c64: ref 0 aborting False
>Thread-15917086::INFO::2013-11-20 04:53:28,187::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:53:29,365::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:53:29,920::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0760924 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:53:31,935::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:53:32,469::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0773006 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 04:53:33,017::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:53:33,029::task::495::TaskManager.Task::(_debug) Task eb40aa18-c3d6-4145-a26d-2665cc621ad8: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:53:33,031::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`97463c5c-89bf-4923-87d6-83e6ade6fc03`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:53:33,032::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:53:33,033::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:53:33,033::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`97463c5c-89bf-4923-87d6-83e6ade6fc03`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:53:33,034::task::495::TaskManager.Task::(_debug) Task eb40aa18-c3d6-4145-a26d-2665cc621ad8: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:53:33,035::task::495::TaskManager.Task::(_debug) Task eb40aa18-c3d6-4145-a26d-2665cc621ad8: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:53:33,036::task::495::TaskManager.Task::(_debug) Task eb40aa18-c3d6-4145-a26d-2665cc621ad8: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:53:33,036::task::495::TaskManager.Task::(_debug) Task eb40aa18-c3d6-4145-a26d-2665cc621ad8: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:53:33,037::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:53:33,039::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:53:33,039::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:53:33,040::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:53:33,041::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:53:33,041::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:53:33,042::task::495::TaskManager.Task::(_debug) Task eb40aa18-c3d6-4145-a26d-2665cc621ad8: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:53:33,043::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913518::INFO::2013-11-20 04:53:34,170::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:53:34,171::task::495::TaskManager.Task::(_debug) Task 6de8928a-a904-45e8-91fa-d6442ac1d7f8: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:53:34,172::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b6d98e08-9504-4f6e-8269-44355cfe2b87`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:53:34,173::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:53:34,174::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:53:34,174::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b6d98e08-9504-4f6e-8269-44355cfe2b87`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:53:34,175::task::495::TaskManager.Task::(_debug) Task 6de8928a-a904-45e8-91fa-d6442ac1d7f8: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:53:34,176::task::495::TaskManager.Task::(_debug) Task 6de8928a-a904-45e8-91fa-d6442ac1d7f8: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:53:34,176::task::495::TaskManager.Task::(_debug) Task 6de8928a-a904-45e8-91fa-d6442ac1d7f8: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:53:34,177::task::495::TaskManager.Task::(_debug) Task 6de8928a-a904-45e8-91fa-d6442ac1d7f8: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:53:34,178::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:53:34,178::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:53:34,179::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:53:34,179::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:53:34,180::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:53:34,181::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:53:34,181::task::495::TaskManager.Task::(_debug) Task 6de8928a-a904-45e8-91fa-d6442ac1d7f8: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:53:34,182::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:53:34,183::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:53:34,183::task::495::TaskManager.Task::(_debug) Task 9b04d426-6b3e-41e9-bf74-3e936514df47: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:53:34,184::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`eaa0a16a-dc7e-4c34-83fa-ad8b884085f1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:53:34,185::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:53:34,186::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:53:34,186::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`eaa0a16a-dc7e-4c34-83fa-ad8b884085f1`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:53:34,187::task::495::TaskManager.Task::(_debug) Task 9b04d426-6b3e-41e9-bf74-3e936514df47: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:53:34,188::task::495::TaskManager.Task::(_debug) Task 9b04d426-6b3e-41e9-bf74-3e936514df47: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:53:34,188::task::495::TaskManager.Task::(_debug) Task 9b04d426-6b3e-41e9-bf74-3e936514df47: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:53:34,189::task::495::TaskManager.Task::(_debug) Task 9b04d426-6b3e-41e9-bf74-3e936514df47: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:53:34,190::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:53:34,190::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:53:34,191::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:53:34,192::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:53:34,192::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:53:34,192::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:53:34,193::task::495::TaskManager.Task::(_debug) Task 9b04d426-6b3e-41e9-bf74-3e936514df47: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:53:34,194::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:53:34,483::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:53:35,004::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0544196 s, 18.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:53:37,015::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:53:37,570::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0934398 s, 11.0 MB/s\n'; <rc> = 0
>Thread-15917090::DEBUG::2013-11-20 04:53:38,298::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917090::INFO::2013-11-20 04:53:38,300::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917090::DEBUG::2013-11-20 04:53:38,301::task::495::TaskManager.Task::(_debug) Task a90b2310-b1e8-4bbe-98c7-3263efa96397: moving from state init -> state preparing
>Thread-15917090::DEBUG::2013-11-20 04:53:38,302::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917090::DEBUG::2013-11-20 04:53:38,302::task::495::TaskManager.Task::(_debug) Task a90b2310-b1e8-4bbe-98c7-3263efa96397: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917090::DEBUG::2013-11-20 04:53:38,303::task::495::TaskManager.Task::(_debug) Task a90b2310-b1e8-4bbe-98c7-3263efa96397: moving from state preparing -> state finished
>Thread-15917090::DEBUG::2013-11-20 04:53:38,304::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917090::DEBUG::2013-11-20 04:53:38,305::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917090::DEBUG::2013-11-20 04:53:38,306::task::495::TaskManager.Task::(_debug) Task a90b2310-b1e8-4bbe-98c7-3263efa96397: ref 0 aborting False
>Thread-15917090::INFO::2013-11-20 04:53:38,306::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917091::DEBUG::2013-11-20 04:53:38,313::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917091::INFO::2013-11-20 04:53:38,315::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917091::DEBUG::2013-11-20 04:53:38,315::task::495::TaskManager.Task::(_debug) Task fce08e4f-3f4e-4590-b489-a58e2088f030: moving from state init -> state preparing
>Thread-15917091::DEBUG::2013-11-20 04:53:38,324::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`01f75549-d4d0-4e66-8274-f5f4c899fe01`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917091::DEBUG::2013-11-20 04:53:38,324::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917091::DEBUG::2013-11-20 04:53:38,326::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917091::DEBUG::2013-11-20 04:53:38,326::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`01f75549-d4d0-4e66-8274-f5f4c899fe01`::Granted request
>Thread-15917091::DEBUG::2013-11-20 04:53:38,327::task::495::TaskManager.Task::(_debug) Task fce08e4f-3f4e-4590-b489-a58e2088f030: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917091::DEBUG::2013-11-20 04:53:38,328::task::495::TaskManager.Task::(_debug) Task fce08e4f-3f4e-4590-b489-a58e2088f030: ref 1 aborting False
>Thread-15917091::WARNING::2013-11-20 04:53:38,330::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917091::DEBUG::2013-11-20 04:53:38,331::task::495::TaskManager.Task::(_debug) Task fce08e4f-3f4e-4590-b489-a58e2088f030: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917091::DEBUG::2013-11-20 04:53:38,332::task::495::TaskManager.Task::(_debug) Task fce08e4f-3f4e-4590-b489-a58e2088f030: moving from state preparing -> state finished
>Thread-15917091::DEBUG::2013-11-20 04:53:38,333::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917091::DEBUG::2013-11-20 04:53:38,333::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917091::DEBUG::2013-11-20 04:53:38,334::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917091::DEBUG::2013-11-20 04:53:38,334::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917091::DEBUG::2013-11-20 04:53:38,335::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917091::DEBUG::2013-11-20 04:53:38,335::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917091::DEBUG::2013-11-20 04:53:38,336::task::495::TaskManager.Task::(_debug) Task fce08e4f-3f4e-4590-b489-a58e2088f030: ref 0 aborting False
>Thread-15917091::INFO::2013-11-20 04:53:38,337::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917092::INFO::2013-11-20 04:53:39,531::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15917092::DEBUG::2013-11-20 04:53:39,532::task::495::TaskManager.Task::(_debug) Task 9b15dce1-9c7c-44b4-9b7a-3103729d4b15: moving from state init -> state preparing
>Thread-15917092::DEBUG::2013-11-20 04:53:39,533::task::495::TaskManager.Task::(_debug) Task 9b15dce1-9c7c-44b4-9b7a-3103729d4b15: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00990796089172', 'lastCheck': 1384923218.747884, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00130605697632', 'lastCheck': 1384923209.5911019, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00188398361206', 'lastCheck': 1384923209.590399, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000549077987671', 'lastCheck': 1384923212.5799179, 'valid': True, 'code': 0}}
>Thread-15917092::DEBUG::2013-11-20 04:53:39,534::task::495::TaskManager.Task::(_debug) Task 9b15dce1-9c7c-44b4-9b7a-3103729d4b15: moving from state preparing -> state finished
>Thread-15917092::DEBUG::2013-11-20 04:53:39,534::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917092::DEBUG::2013-11-20 04:53:39,535::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917092::DEBUG::2013-11-20 04:53:39,535::task::495::TaskManager.Task::(_debug) Task 9b15dce1-9c7c-44b4-9b7a-3103729d4b15: ref 0 aborting False
>Thread-15917092::INFO::2013-11-20 04:53:39,536::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00990796089172', 'lastCheck': 1384923218.747884, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00130605697632', 'lastCheck': 1384923209.5911019, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00188398361206', 'lastCheck': 1384923209.590399, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000549077987671', 'lastCheck': 1384923212.5799179, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:53:39,585::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:53:40,112::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0747339 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:53:42,127::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:53:42,683::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0827202 s, 12.4 MB/s\n'; <rc> = 0
>Thread-15914453::INFO::2013-11-20 04:53:43,229::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:53:43,234::task::495::TaskManager.Task::(_debug) Task 91cfcd96-5ed7-4098-8aeb-1af2ba53c442: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:53:43,237::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5caf0cd9-b1a5-452b-aeb3-e348d95f8262`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:53:43,238::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:53:43,240::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:53:43,240::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5caf0cd9-b1a5-452b-aeb3-e348d95f8262`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:53:43,241::task::495::TaskManager.Task::(_debug) Task 91cfcd96-5ed7-4098-8aeb-1af2ba53c442: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:53:43,242::task::495::TaskManager.Task::(_debug) Task 91cfcd96-5ed7-4098-8aeb-1af2ba53c442: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:53:43,243::task::495::TaskManager.Task::(_debug) Task 91cfcd96-5ed7-4098-8aeb-1af2ba53c442: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:53:43,244::task::495::TaskManager.Task::(_debug) Task 91cfcd96-5ed7-4098-8aeb-1af2ba53c442: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:53:43,244::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:53:43,245::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:53:43,246::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:53:43,247::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:53:43,247::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:53:43,248::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:53:43,249::task::495::TaskManager.Task::(_debug) Task 91cfcd96-5ed7-4098-8aeb-1af2ba53c442: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:53:43,249::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:53:44,698::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 04:53:44,958::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:53:44,960::task::495::TaskManager.Task::(_debug) Task 432743c5-51e6-4b78-b64b-bc610a0f5e2e: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:53:44,962::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8a8948f5-746f-46cb-9c31-e3424de788cb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:53:44,963::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:53:44,964::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:53:44,965::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8a8948f5-746f-46cb-9c31-e3424de788cb`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:53:44,966::task::495::TaskManager.Task::(_debug) Task 432743c5-51e6-4b78-b64b-bc610a0f5e2e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:53:44,967::task::495::TaskManager.Task::(_debug) Task 432743c5-51e6-4b78-b64b-bc610a0f5e2e: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:53:44,968::task::495::TaskManager.Task::(_debug) Task 432743c5-51e6-4b78-b64b-bc610a0f5e2e: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:53:44,968::task::495::TaskManager.Task::(_debug) Task 432743c5-51e6-4b78-b64b-bc610a0f5e2e: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:53:44,969::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:53:44,970::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:53:44,970::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:53:44,971::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:53:44,971::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:53:44,972::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:53:44,973::task::495::TaskManager.Task::(_debug) Task 432743c5-51e6-4b78-b64b-bc610a0f5e2e: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:53:44,973::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:53:45,291::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0843783 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:53:47,305::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:53:47,825::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0844604 s, 12.1 MB/s\n'; <rc> = 0
>Thread-15917096::DEBUG::2013-11-20 04:53:48,456::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917096::INFO::2013-11-20 04:53:48,459::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917096::DEBUG::2013-11-20 04:53:48,460::task::495::TaskManager.Task::(_debug) Task a965c26d-d21e-41ea-950c-ac573c87eae7: moving from state init -> state preparing
>Thread-15917096::DEBUG::2013-11-20 04:53:48,461::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917096::DEBUG::2013-11-20 04:53:48,461::task::495::TaskManager.Task::(_debug) Task a965c26d-d21e-41ea-950c-ac573c87eae7: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917096::DEBUG::2013-11-20 04:53:48,462::task::495::TaskManager.Task::(_debug) Task a965c26d-d21e-41ea-950c-ac573c87eae7: moving from state preparing -> state finished
>Thread-15917096::DEBUG::2013-11-20 04:53:48,463::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917096::DEBUG::2013-11-20 04:53:48,463::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917096::DEBUG::2013-11-20 04:53:48,464::task::495::TaskManager.Task::(_debug) Task a965c26d-d21e-41ea-950c-ac573c87eae7: ref 0 aborting False
>Thread-15917096::INFO::2013-11-20 04:53:48,464::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917097::DEBUG::2013-11-20 04:53:48,470::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917097::INFO::2013-11-20 04:53:48,471::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917097::DEBUG::2013-11-20 04:53:48,472::task::495::TaskManager.Task::(_debug) Task f844d98c-c666-406f-826c-c92d89650af8: moving from state init -> state preparing
>Thread-15917097::DEBUG::2013-11-20 04:53:48,473::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e50a536f-89c7-483d-bc8c-a5cf796f6281`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917097::DEBUG::2013-11-20 04:53:48,473::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917097::DEBUG::2013-11-20 04:53:48,474::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917097::DEBUG::2013-11-20 04:53:48,475::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e50a536f-89c7-483d-bc8c-a5cf796f6281`::Granted request
>Thread-15917097::DEBUG::2013-11-20 04:53:48,476::task::495::TaskManager.Task::(_debug) Task f844d98c-c666-406f-826c-c92d89650af8: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917097::DEBUG::2013-11-20 04:53:48,476::task::495::TaskManager.Task::(_debug) Task f844d98c-c666-406f-826c-c92d89650af8: ref 1 aborting False
>Thread-15917097::WARNING::2013-11-20 04:53:48,479::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917097::DEBUG::2013-11-20 04:53:48,480::task::495::TaskManager.Task::(_debug) Task f844d98c-c666-406f-826c-c92d89650af8: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917097::DEBUG::2013-11-20 04:53:48,481::task::495::TaskManager.Task::(_debug) Task f844d98c-c666-406f-826c-c92d89650af8: moving from state preparing -> state finished
>Thread-15917097::DEBUG::2013-11-20 04:53:48,482::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917097::DEBUG::2013-11-20 04:53:48,483::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917097::DEBUG::2013-11-20 04:53:48,483::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917097::DEBUG::2013-11-20 04:53:48,484::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917097::DEBUG::2013-11-20 04:53:48,485::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917097::DEBUG::2013-11-20 04:53:48,486::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917097::DEBUG::2013-11-20 04:53:48,486::task::495::TaskManager.Task::(_debug) Task f844d98c-c666-406f-826c-c92d89650af8: ref 0 aborting False
>Thread-15917097::INFO::2013-11-20 04:53:48,487::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-42838::DEBUG::2013-11-20 04:53:48,950::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 04:53:49,823::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:53:49,839::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:53:50,374::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0778921 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:53:52,391::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:53:52,952::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0862555 s, 11.9 MB/s\n'; <rc> = 0
>Thread-15914519::INFO::2013-11-20 04:53:53,253::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15913546::INFO::2013-11-20 04:53:53,256::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15914519::DEBUG::2013-11-20 04:53:53,259::task::495::TaskManager.Task::(_debug) Task 8fcc24f5-d2b4-4237-8065-e7fcc7ffabb1: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:53:53,260::task::495::TaskManager.Task::(_debug) Task a7303d50-7867-4549-b7df-d4a9df78fa62: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:53:53,275::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0e223e9d-e8d5-4a6c-96d3-0096e56b3263`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:53:53,279::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:53:53,280::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c7cb3254-028c-4196-a5fb-a2c7f2ed50ac`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:53:53,282::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:53:53,282::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:53:53,283::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0e223e9d-e8d5-4a6c-96d3-0096e56b3263`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:53:53,285::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:53:53,286::task::495::TaskManager.Task::(_debug) Task 8fcc24f5-d2b4-4237-8065-e7fcc7ffabb1: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913546::DEBUG::2013-11-20 04:53:53,286::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c7cb3254-028c-4196-a5fb-a2c7f2ed50ac`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:53:53,287::task::495::TaskManager.Task::(_debug) Task 8fcc24f5-d2b4-4237-8065-e7fcc7ffabb1: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:53:53,289::task::495::TaskManager.Task::(_debug) Task a7303d50-7867-4549-b7df-d4a9df78fa62: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15914519::DEBUG::2013-11-20 04:53:53,290::task::495::TaskManager.Task::(_debug) Task 8fcc24f5-d2b4-4237-8065-e7fcc7ffabb1: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:53:53,291::task::495::TaskManager.Task::(_debug) Task a7303d50-7867-4549-b7df-d4a9df78fa62: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:53:53,292::task::495::TaskManager.Task::(_debug) Task 8fcc24f5-d2b4-4237-8065-e7fcc7ffabb1: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:53:53,293::task::495::TaskManager.Task::(_debug) Task a7303d50-7867-4549-b7df-d4a9df78fa62: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:53:53,294::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:53:53,295::task::495::TaskManager.Task::(_debug) Task a7303d50-7867-4549-b7df-d4a9df78fa62: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:53:53,295::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:53:53,296::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:53:53,303::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:53:53,303::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913546::DEBUG::2013-11-20 04:53:53,308::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15914519::DEBUG::2013-11-20 04:53:53,310::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:53:53,310::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:53:53,311::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:53:53,312::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:53:53,313::task::495::TaskManager.Task::(_debug) Task 8fcc24f5-d2b4-4237-8065-e7fcc7ffabb1: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:53:53,314::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15914519::INFO::2013-11-20 04:53:53,314::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:53:53,315::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:53:53,316::task::495::TaskManager.Task::(_debug) Task a7303d50-7867-4549-b7df-d4a9df78fa62: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:53:53,317::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15913546::INFO::2013-11-20 04:53:53,318::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:53:53,319::task::495::TaskManager.Task::(_debug) Task 1b71e77d-f945-429f-88d0-cbbd2d395285: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:53:53,320::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8970f69d-9ed8-4790-b398-453cae3eed34`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:53:53,321::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:53:53,321::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:53:53,322::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`8970f69d-9ed8-4790-b398-453cae3eed34`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:53:53,323::task::495::TaskManager.Task::(_debug) Task 1b71e77d-f945-429f-88d0-cbbd2d395285: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:53:53,324::task::495::TaskManager.Task::(_debug) Task 1b71e77d-f945-429f-88d0-cbbd2d395285: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:53:53,324::task::495::TaskManager.Task::(_debug) Task 1b71e77d-f945-429f-88d0-cbbd2d395285: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:53:53,325::task::495::TaskManager.Task::(_debug) Task 1b71e77d-f945-429f-88d0-cbbd2d395285: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:53:53,326::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:53:53,326::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:53:53,327::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:53:53,328::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:53:53,328::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:53:53,329::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:53:53,330::task::495::TaskManager.Task::(_debug) Task 1b71e77d-f945-429f-88d0-cbbd2d395285: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:53:53,331::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:53:53,331::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:53:53,332::task::495::TaskManager.Task::(_debug) Task d9175364-d72e-4e47-82c5-49497a9776e1: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:53:53,333::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`46775a7d-9388-48cd-a34b-6cd5b6e4ee91`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:53:53,334::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:53:53,335::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:53:53,336::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`46775a7d-9388-48cd-a34b-6cd5b6e4ee91`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:53:53,337::task::495::TaskManager.Task::(_debug) Task d9175364-d72e-4e47-82c5-49497a9776e1: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:53:53,338::task::495::TaskManager.Task::(_debug) Task d9175364-d72e-4e47-82c5-49497a9776e1: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:53:53,338::task::495::TaskManager.Task::(_debug) Task d9175364-d72e-4e47-82c5-49497a9776e1: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:53:53,339::task::495::TaskManager.Task::(_debug) Task d9175364-d72e-4e47-82c5-49497a9776e1: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:53:53,340::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:53:53,340::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:53:53,341::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:53:53,342::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:53:53,342::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:53:53,343::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:53:53,343::task::495::TaskManager.Task::(_debug) Task d9175364-d72e-4e47-82c5-49497a9776e1: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:53:53,344::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:53:53,345::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:53:53,346::task::495::TaskManager.Task::(_debug) Task a398a069-7186-44a8-9cba-7eb1b4e411ac: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:53:53,347::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e210e0b0-e4b4-418c-964e-e054ba8c9ca4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:53:53,348::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:53:53,348::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:53:53,349::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e210e0b0-e4b4-418c-964e-e054ba8c9ca4`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:53:53,350::task::495::TaskManager.Task::(_debug) Task a398a069-7186-44a8-9cba-7eb1b4e411ac: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:53:53,350::task::495::TaskManager.Task::(_debug) Task a398a069-7186-44a8-9cba-7eb1b4e411ac: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:53:53,351::task::495::TaskManager.Task::(_debug) Task a398a069-7186-44a8-9cba-7eb1b4e411ac: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:53:53,352::task::495::TaskManager.Task::(_debug) Task a398a069-7186-44a8-9cba-7eb1b4e411ac: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:53:53,352::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:53:53,353::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:53:53,353::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:53:53,354::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:53:53,354::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:53:53,355::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:53:53,356::task::495::TaskManager.Task::(_debug) Task a398a069-7186-44a8-9cba-7eb1b4e411ac: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:53:53,356::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:53:54,969::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15917100::INFO::2013-11-20 04:53:55,218::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15917100::DEBUG::2013-11-20 04:53:55,219::task::495::TaskManager.Task::(_debug) Task a6103e8e-4fb7-4577-8a1f-3a21563b9959: moving from state init -> state preparing
>Thread-15917100::DEBUG::2013-11-20 04:53:55,220::task::495::TaskManager.Task::(_debug) Task a6103e8e-4fb7-4577-8a1f-3a21563b9959: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000569105148315', 'lastCheck': 1384923229.8293419, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0020580291748', 'lastCheck': 1384923230.0595479, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.002201795578', 'lastCheck': 1384923230.059166, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000449180603027', 'lastCheck': 1384923233.048697, 'valid': True, 'code': 0}}
>Thread-15917100::DEBUG::2013-11-20 04:53:55,222::task::495::TaskManager.Task::(_debug) Task a6103e8e-4fb7-4577-8a1f-3a21563b9959: moving from state preparing -> state finished
>Thread-15917100::DEBUG::2013-11-20 04:53:55,222::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917100::DEBUG::2013-11-20 04:53:55,223::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917100::DEBUG::2013-11-20 04:53:55,224::task::495::TaskManager.Task::(_debug) Task a6103e8e-4fb7-4577-8a1f-3a21563b9959: ref 0 aborting False
>Thread-15917100::INFO::2013-11-20 04:53:55,224::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000569105148315', 'lastCheck': 1384923229.8293419, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0020580291748', 'lastCheck': 1384923230.0595479, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.002201795578', 'lastCheck': 1384923230.059166, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000449180603027', 'lastCheck': 1384923233.048697, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:53:55,507::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0527159 s, 19.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:53:57,518::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:53:57,985::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0520003 s, 19.7 MB/s\n'; <rc> = 0
>Thread-15917102::DEBUG::2013-11-20 04:53:58,604::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917102::INFO::2013-11-20 04:53:58,606::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917102::DEBUG::2013-11-20 04:53:58,608::task::495::TaskManager.Task::(_debug) Task 7a08d2b2-77da-4991-9ebb-68017389546f: moving from state init -> state preparing
>Thread-15917102::DEBUG::2013-11-20 04:53:58,609::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917102::DEBUG::2013-11-20 04:53:58,610::task::495::TaskManager.Task::(_debug) Task 7a08d2b2-77da-4991-9ebb-68017389546f: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917102::DEBUG::2013-11-20 04:53:58,611::task::495::TaskManager.Task::(_debug) Task 7a08d2b2-77da-4991-9ebb-68017389546f: moving from state preparing -> state finished
>Thread-15917102::DEBUG::2013-11-20 04:53:58,613::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917102::DEBUG::2013-11-20 04:53:58,613::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917102::DEBUG::2013-11-20 04:53:58,616::task::495::TaskManager.Task::(_debug) Task 7a08d2b2-77da-4991-9ebb-68017389546f: ref 0 aborting False
>Thread-15917102::INFO::2013-11-20 04:53:58,619::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917104::DEBUG::2013-11-20 04:53:58,640::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917104::INFO::2013-11-20 04:53:58,641::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917104::DEBUG::2013-11-20 04:53:58,642::task::495::TaskManager.Task::(_debug) Task 05a8059d-022d-4924-827f-e0d9327fda9f: moving from state init -> state preparing
>Thread-15917104::DEBUG::2013-11-20 04:53:58,643::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a995a925-6381-463e-ba3c-019dfcd06da9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917104::DEBUG::2013-11-20 04:53:58,643::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917104::DEBUG::2013-11-20 04:53:58,644::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917104::DEBUG::2013-11-20 04:53:58,645::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`a995a925-6381-463e-ba3c-019dfcd06da9`::Granted request
>Thread-15917104::DEBUG::2013-11-20 04:53:58,646::task::495::TaskManager.Task::(_debug) Task 05a8059d-022d-4924-827f-e0d9327fda9f: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917104::DEBUG::2013-11-20 04:53:58,647::task::495::TaskManager.Task::(_debug) Task 05a8059d-022d-4924-827f-e0d9327fda9f: ref 1 aborting False
>Thread-15917104::WARNING::2013-11-20 04:53:58,649::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917104::DEBUG::2013-11-20 04:53:58,651::task::495::TaskManager.Task::(_debug) Task 05a8059d-022d-4924-827f-e0d9327fda9f: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917104::DEBUG::2013-11-20 04:53:58,652::task::495::TaskManager.Task::(_debug) Task 05a8059d-022d-4924-827f-e0d9327fda9f: moving from state preparing -> state finished
>Thread-15917104::DEBUG::2013-11-20 04:53:58,652::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917104::DEBUG::2013-11-20 04:53:58,653::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917104::DEBUG::2013-11-20 04:53:58,654::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917104::DEBUG::2013-11-20 04:53:58,655::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917104::DEBUG::2013-11-20 04:53:58,655::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917104::DEBUG::2013-11-20 04:53:58,656::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917104::DEBUG::2013-11-20 04:53:58,657::task::495::TaskManager.Task::(_debug) Task 05a8059d-022d-4924-827f-e0d9327fda9f: ref 0 aborting False
>Thread-15917104::INFO::2013-11-20 04:53:58,658::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:53:59,994::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:54:00,457::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0556353 s, 18.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:54:02,472::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 04:54:02,775::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:54:02,776::task::495::TaskManager.Task::(_debug) Task b0046b38-d6c5-4ebd-b74d-1afb8042f916: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:54:02,779::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2722bc50-b63f-4447-9b84-1da2ca377444`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:54:02,780::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:54:02,781::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:54:02,782::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2722bc50-b63f-4447-9b84-1da2ca377444`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:54:02,784::task::495::TaskManager.Task::(_debug) Task b0046b38-d6c5-4ebd-b74d-1afb8042f916: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:54:02,785::task::495::TaskManager.Task::(_debug) Task b0046b38-d6c5-4ebd-b74d-1afb8042f916: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:54:02,786::task::495::TaskManager.Task::(_debug) Task b0046b38-d6c5-4ebd-b74d-1afb8042f916: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:54:02,787::task::495::TaskManager.Task::(_debug) Task b0046b38-d6c5-4ebd-b74d-1afb8042f916: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:54:02,788::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:54:02,788::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:54:02,789::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:54:02,790::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:54:02,791::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:54:02,791::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:54:02,793::task::495::TaskManager.Task::(_debug) Task b0046b38-d6c5-4ebd-b74d-1afb8042f916: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:54:02,793::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:54:03,021::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0763426 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:54:05,036::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:54:05,572::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0781809 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:54:07,587::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:54:08,100::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0630695 s, 16.2 MB/s\n'; <rc> = 0
>Thread-15917108::DEBUG::2013-11-20 04:54:08,727::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917108::INFO::2013-11-20 04:54:08,729::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917108::DEBUG::2013-11-20 04:54:08,729::task::495::TaskManager.Task::(_debug) Task aee0c5ad-9dd4-47db-a826-d43c6a77248a: moving from state init -> state preparing
>Thread-15917108::DEBUG::2013-11-20 04:54:08,730::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917108::DEBUG::2013-11-20 04:54:08,731::task::495::TaskManager.Task::(_debug) Task aee0c5ad-9dd4-47db-a826-d43c6a77248a: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917108::DEBUG::2013-11-20 04:54:08,732::task::495::TaskManager.Task::(_debug) Task aee0c5ad-9dd4-47db-a826-d43c6a77248a: moving from state preparing -> state finished
>Thread-15917108::DEBUG::2013-11-20 04:54:08,732::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917108::DEBUG::2013-11-20 04:54:08,733::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917108::DEBUG::2013-11-20 04:54:08,733::task::495::TaskManager.Task::(_debug) Task aee0c5ad-9dd4-47db-a826-d43c6a77248a: ref 0 aborting False
>Thread-15917108::INFO::2013-11-20 04:54:08,734::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917109::DEBUG::2013-11-20 04:54:08,740::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917109::INFO::2013-11-20 04:54:08,740::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917109::DEBUG::2013-11-20 04:54:08,741::task::495::TaskManager.Task::(_debug) Task 6dff6be6-cda7-4277-b29b-b43bfe5ba0d8: moving from state init -> state preparing
>Thread-15917109::DEBUG::2013-11-20 04:54:08,742::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9f9cf151-b6ed-458b-aae2-bf58101644f8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917109::DEBUG::2013-11-20 04:54:08,743::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917109::DEBUG::2013-11-20 04:54:08,744::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917109::DEBUG::2013-11-20 04:54:08,744::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9f9cf151-b6ed-458b-aae2-bf58101644f8`::Granted request
>Thread-15917109::DEBUG::2013-11-20 04:54:08,745::task::495::TaskManager.Task::(_debug) Task 6dff6be6-cda7-4277-b29b-b43bfe5ba0d8: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917109::DEBUG::2013-11-20 04:54:08,746::task::495::TaskManager.Task::(_debug) Task 6dff6be6-cda7-4277-b29b-b43bfe5ba0d8: ref 1 aborting False
>Thread-15917109::WARNING::2013-11-20 04:54:08,748::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917109::DEBUG::2013-11-20 04:54:08,749::task::495::TaskManager.Task::(_debug) Task 6dff6be6-cda7-4277-b29b-b43bfe5ba0d8: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917109::DEBUG::2013-11-20 04:54:08,750::task::495::TaskManager.Task::(_debug) Task 6dff6be6-cda7-4277-b29b-b43bfe5ba0d8: moving from state preparing -> state finished
>Thread-15917109::DEBUG::2013-11-20 04:54:08,751::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917109::DEBUG::2013-11-20 04:54:08,751::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917109::DEBUG::2013-11-20 04:54:08,752::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917109::DEBUG::2013-11-20 04:54:08,752::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917109::DEBUG::2013-11-20 04:54:08,753::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917109::DEBUG::2013-11-20 04:54:08,753::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917109::DEBUG::2013-11-20 04:54:08,754::task::495::TaskManager.Task::(_debug) Task 6dff6be6-cda7-4277-b29b-b43bfe5ba0d8: ref 0 aborting False
>Thread-15917109::INFO::2013-11-20 04:54:08,755::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:54:10,110::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:54:10,659::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0813589 s, 12.6 MB/s\n'; <rc> = 0
>Thread-15917110::INFO::2013-11-20 04:54:10,890::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15917110::DEBUG::2013-11-20 04:54:10,891::task::495::TaskManager.Task::(_debug) Task 6a05160f-ca94-4c19-95ea-76290faae5ca: moving from state init -> state preparing
>Thread-15917110::DEBUG::2013-11-20 04:54:10,893::task::495::TaskManager.Task::(_debug) Task 6a05160f-ca94-4c19-95ea-76290faae5ca: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000306129455566', 'lastCheck': 1384923249.856622, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00172996520996', 'lastCheck': 1384923250.377635, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0035297870636', 'lastCheck': 1384923250.37657, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000561952590942', 'lastCheck': 1384923243.2152359, 'valid': True, 'code': 0}}
>Thread-15917110::DEBUG::2013-11-20 04:54:10,894::task::495::TaskManager.Task::(_debug) Task 6a05160f-ca94-4c19-95ea-76290faae5ca: moving from state preparing -> state finished
>Thread-15917110::DEBUG::2013-11-20 04:54:10,894::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917110::DEBUG::2013-11-20 04:54:10,895::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917110::DEBUG::2013-11-20 04:54:10,896::task::495::TaskManager.Task::(_debug) Task 6a05160f-ca94-4c19-95ea-76290faae5ca: ref 0 aborting False
>Thread-15917110::INFO::2013-11-20 04:54:10,896::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000306129455566', 'lastCheck': 1384923249.856622, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00172996520996', 'lastCheck': 1384923250.377635, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.0035297870636', 'lastCheck': 1384923250.37657, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000561952590942', 'lastCheck': 1384923243.2152359, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:54:12,677::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:54:12,900::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:54:12,913::task::495::TaskManager.Task::(_debug) Task b8de840f-2f8c-4afc-b4d3-42af29416092: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:54:12,915::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f658455c-e3f3-44fa-b9ca-4622bbde5483`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:54:12,916::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:54:12,917::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:54:12,918::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f658455c-e3f3-44fa-b9ca-4622bbde5483`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:54:12,920::task::495::TaskManager.Task::(_debug) Task b8de840f-2f8c-4afc-b4d3-42af29416092: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:54:12,921::task::495::TaskManager.Task::(_debug) Task b8de840f-2f8c-4afc-b4d3-42af29416092: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:54:12,922::task::495::TaskManager.Task::(_debug) Task b8de840f-2f8c-4afc-b4d3-42af29416092: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:54:12,922::task::495::TaskManager.Task::(_debug) Task b8de840f-2f8c-4afc-b4d3-42af29416092: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:54:12,923::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:54:12,924::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:54:12,924::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:54:12,925::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:54:12,926::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:54:12,926::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:54:12,927::task::495::TaskManager.Task::(_debug) Task b8de840f-2f8c-4afc-b4d3-42af29416092: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:54:12,928::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:54:13,218::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0826855 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:54:15,234::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915918::INFO::2013-11-20 04:54:15,494::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:54:15,497::task::495::TaskManager.Task::(_debug) Task 0be24ed8-94df-4e9d-bf83-6d5a0f1b9467: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:54:15,500::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0b0ab20c-78c2-4c07-a9d8-63ca0019cb50`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:54:15,502::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:54:15,504::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:54:15,504::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0b0ab20c-78c2-4c07-a9d8-63ca0019cb50`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:54:15,506::task::495::TaskManager.Task::(_debug) Task 0be24ed8-94df-4e9d-bf83-6d5a0f1b9467: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:54:15,507::task::495::TaskManager.Task::(_debug) Task 0be24ed8-94df-4e9d-bf83-6d5a0f1b9467: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:54:15,508::task::495::TaskManager.Task::(_debug) Task 0be24ed8-94df-4e9d-bf83-6d5a0f1b9467: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:54:15,508::task::495::TaskManager.Task::(_debug) Task 0be24ed8-94df-4e9d-bf83-6d5a0f1b9467: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:54:15,509::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:54:15,510::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:54:15,511::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:54:15,512::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:54:15,512::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:54:15,513::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:54:15,514::task::495::TaskManager.Task::(_debug) Task 0be24ed8-94df-4e9d-bf83-6d5a0f1b9467: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:54:15,515::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:54:15,801::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0642527 s, 15.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:54:17,815::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916984::INFO::2013-11-20 04:54:18,059::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=240c0593-fe33-4f8b-a029-097379dbcb85 volUUID=f715faaa-081a-4598-b125-80a2d12af554)
>Thread-15916984::DEBUG::2013-11-20 04:54:18,061::task::495::TaskManager.Task::(_debug) Task 935798dd-418e-4dd5-a19d-a80eb4f2c0d4: moving from state init -> state preparing
>Thread-15916984::DEBUG::2013-11-20 04:54:18,064::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5e2f5dbe-2c2f-4fe5-a259-05a7ddc5766e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15916984::DEBUG::2013-11-20 04:54:18,064::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15916984::DEBUG::2013-11-20 04:54:18,068::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15916984::DEBUG::2013-11-20 04:54:18,068::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5e2f5dbe-2c2f-4fe5-a259-05a7ddc5766e`::Granted request
>Thread-15916984::DEBUG::2013-11-20 04:54:18,069::task::495::TaskManager.Task::(_debug) Task 935798dd-418e-4dd5-a19d-a80eb4f2c0d4: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15916984::DEBUG::2013-11-20 04:54:18,071::task::495::TaskManager.Task::(_debug) Task 935798dd-418e-4dd5-a19d-a80eb4f2c0d4: ref 1 aborting False
>Thread-15916984::DEBUG::2013-11-20 04:54:18,073::task::495::TaskManager.Task::(_debug) Task 935798dd-418e-4dd5-a19d-a80eb4f2c0d4: finished: {'truesize': '75296145408', 'apparentsize': '75296145408'}
>Thread-15916984::DEBUG::2013-11-20 04:54:18,074::task::495::TaskManager.Task::(_debug) Task 935798dd-418e-4dd5-a19d-a80eb4f2c0d4: moving from state preparing -> state finished
>Thread-15916984::DEBUG::2013-11-20 04:54:18,074::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15916984::DEBUG::2013-11-20 04:54:18,075::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916984::DEBUG::2013-11-20 04:54:18,081::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15916984::DEBUG::2013-11-20 04:54:18,082::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15916984::DEBUG::2013-11-20 04:54:18,084::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15916984::DEBUG::2013-11-20 04:54:18,088::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15916984::DEBUG::2013-11-20 04:54:18,089::task::495::TaskManager.Task::(_debug) Task 935798dd-418e-4dd5-a19d-a80eb4f2c0d4: ref 0 aborting False
>Thread-15916984::INFO::2013-11-20 04:54:18,092::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '75296145408', 'apparentsize': '75296145408'}
>Dummy-15912651::DEBUG::2013-11-20 04:54:18,438::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0898414 s, 11.4 MB/s\n'; <rc> = 0
>Thread-15917114::DEBUG::2013-11-20 04:54:18,869::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917114::INFO::2013-11-20 04:54:18,871::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917114::DEBUG::2013-11-20 04:54:18,871::task::495::TaskManager.Task::(_debug) Task 49f8e7d7-fdd8-4af5-8359-8e5c5c28e551: moving from state init -> state preparing
>Thread-15917114::DEBUG::2013-11-20 04:54:18,872::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917114::DEBUG::2013-11-20 04:54:18,873::task::495::TaskManager.Task::(_debug) Task 49f8e7d7-fdd8-4af5-8359-8e5c5c28e551: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917114::DEBUG::2013-11-20 04:54:18,873::task::495::TaskManager.Task::(_debug) Task 49f8e7d7-fdd8-4af5-8359-8e5c5c28e551: moving from state preparing -> state finished
>Thread-15917114::DEBUG::2013-11-20 04:54:18,874::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917114::DEBUG::2013-11-20 04:54:18,874::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917114::DEBUG::2013-11-20 04:54:18,875::task::495::TaskManager.Task::(_debug) Task 49f8e7d7-fdd8-4af5-8359-8e5c5c28e551: ref 0 aborting False
>Thread-15917114::INFO::2013-11-20 04:54:18,876::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917115::DEBUG::2013-11-20 04:54:18,881::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917115::INFO::2013-11-20 04:54:18,882::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917115::DEBUG::2013-11-20 04:54:18,883::task::495::TaskManager.Task::(_debug) Task a858ff36-ff48-46b6-a42b-a87340e0d126: moving from state init -> state preparing
>Thread-15917115::DEBUG::2013-11-20 04:54:18,884::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2bf72806-7c7b-481b-b48c-29469e5e7765`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917115::DEBUG::2013-11-20 04:54:18,885::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917115::DEBUG::2013-11-20 04:54:18,886::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917115::DEBUG::2013-11-20 04:54:18,887::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`2bf72806-7c7b-481b-b48c-29469e5e7765`::Granted request
>Thread-15917115::DEBUG::2013-11-20 04:54:18,888::task::495::TaskManager.Task::(_debug) Task a858ff36-ff48-46b6-a42b-a87340e0d126: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917115::DEBUG::2013-11-20 04:54:18,889::task::495::TaskManager.Task::(_debug) Task a858ff36-ff48-46b6-a42b-a87340e0d126: ref 1 aborting False
>Thread-15917115::WARNING::2013-11-20 04:54:18,892::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917115::DEBUG::2013-11-20 04:54:18,893::task::495::TaskManager.Task::(_debug) Task a858ff36-ff48-46b6-a42b-a87340e0d126: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917115::DEBUG::2013-11-20 04:54:18,896::task::495::TaskManager.Task::(_debug) Task a858ff36-ff48-46b6-a42b-a87340e0d126: moving from state preparing -> state finished
>Thread-15917115::DEBUG::2013-11-20 04:54:18,899::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917115::DEBUG::2013-11-20 04:54:18,900::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917115::DEBUG::2013-11-20 04:54:18,903::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917115::DEBUG::2013-11-20 04:54:18,904::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917115::DEBUG::2013-11-20 04:54:18,904::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917115::DEBUG::2013-11-20 04:54:18,905::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917115::DEBUG::2013-11-20 04:54:18,905::task::495::TaskManager.Task::(_debug) Task a858ff36-ff48-46b6-a42b-a87340e0d126: ref 0 aborting False
>Thread-15917115::INFO::2013-11-20 04:54:18,906::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:54:20,455::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915863::INFO::2013-11-20 04:54:20,663::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:54:20,671::task::495::TaskManager.Task::(_debug) Task 8965621d-9e93-4826-8bac-1601177b065d: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:54:20,678::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0d60b1be-29fd-41fe-9bcd-e2fc6225d4ef`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:54:20,679::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:54:20,681::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:54:20,681::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0d60b1be-29fd-41fe-9bcd-e2fc6225d4ef`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:54:20,683::task::495::TaskManager.Task::(_debug) Task 8965621d-9e93-4826-8bac-1601177b065d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:54:20,683::task::495::TaskManager.Task::(_debug) Task 8965621d-9e93-4826-8bac-1601177b065d: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:54:20,684::task::495::TaskManager.Task::(_debug) Task 8965621d-9e93-4826-8bac-1601177b065d: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:54:20,685::task::495::TaskManager.Task::(_debug) Task 8965621d-9e93-4826-8bac-1601177b065d: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:54:20,686::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:54:20,687::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:54:20,688::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:54:20,688::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:54:20,689::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:54:20,690::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:54:20,691::task::495::TaskManager.Task::(_debug) Task 8965621d-9e93-4826-8bac-1601177b065d: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:54:20,691::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:54:20,956::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0749697 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:54:22,966::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 04:54:23,500::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:54:23,503::task::495::TaskManager.Task::(_debug) Task ee9282f3-5072-430f-b448-b3984a0c735a: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:54:23,508::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`de122010-a2bb-467d-bb07-73e6b027023f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:54:23,510::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:54:23,512::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:54:23,513::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`de122010-a2bb-467d-bb07-73e6b027023f`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:54:23,515::task::495::TaskManager.Task::(_debug) Task ee9282f3-5072-430f-b448-b3984a0c735a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:54:23,516::task::495::TaskManager.Task::(_debug) Task ee9282f3-5072-430f-b448-b3984a0c735a: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:54:23,517::task::495::TaskManager.Task::(_debug) Task ee9282f3-5072-430f-b448-b3984a0c735a: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:54:23,518::task::495::TaskManager.Task::(_debug) Task ee9282f3-5072-430f-b448-b3984a0c735a: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:54:23,519::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:54:23,520::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:54:23,522::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:54:23,524::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:54:23,524::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:54:23,525::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:54:23,527::task::495::TaskManager.Task::(_debug) Task ee9282f3-5072-430f-b448-b3984a0c735a: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:54:23,527::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:54:23,561::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0841477 s, 12.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:54:25,577::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:54:26,112::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.075597 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15917118::INFO::2013-11-20 04:54:26,369::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15917118::DEBUG::2013-11-20 04:54:26,370::task::495::TaskManager.Task::(_debug) Task 6e747303-6138-4b63-8c41-e1899ef31f4b: moving from state init -> state preparing
>Thread-15917118::DEBUG::2013-11-20 04:54:26,371::task::495::TaskManager.Task::(_debug) Task 6e747303-6138-4b63-8c41-e1899ef31f4b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000432014465332', 'lastCheck': 1384923260.0615771, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00372886657715', 'lastCheck': 1384923260.679008, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00277304649353', 'lastCheck': 1384923260.6775041, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00059700012207', 'lastCheck': 1384923263.6593339, 'valid': True, 'code': 0}}
>Thread-15917118::DEBUG::2013-11-20 04:54:26,372::task::495::TaskManager.Task::(_debug) Task 6e747303-6138-4b63-8c41-e1899ef31f4b: moving from state preparing -> state finished
>Thread-15917118::DEBUG::2013-11-20 04:54:26,372::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917118::DEBUG::2013-11-20 04:54:26,373::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917118::DEBUG::2013-11-20 04:54:26,373::task::495::TaskManager.Task::(_debug) Task 6e747303-6138-4b63-8c41-e1899ef31f4b: ref 0 aborting False
>Thread-15917118::INFO::2013-11-20 04:54:26,374::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000432014465332', 'lastCheck': 1384923260.0615771, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00372886657715', 'lastCheck': 1384923260.679008, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00277304649353', 'lastCheck': 1384923260.6775041, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00059700012207', 'lastCheck': 1384923263.6593339, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:54:28,127::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:54:28,681::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0858084 s, 11.9 MB/s\n'; <rc> = 0
>Thread-15917120::DEBUG::2013-11-20 04:54:29,068::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917120::INFO::2013-11-20 04:54:29,070::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917120::DEBUG::2013-11-20 04:54:29,070::task::495::TaskManager.Task::(_debug) Task b1291b2b-69e6-4d6f-8a48-6c682fa29bf3: moving from state init -> state preparing
>Thread-15917120::DEBUG::2013-11-20 04:54:29,071::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917120::DEBUG::2013-11-20 04:54:29,072::task::495::TaskManager.Task::(_debug) Task b1291b2b-69e6-4d6f-8a48-6c682fa29bf3: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917120::DEBUG::2013-11-20 04:54:29,073::task::495::TaskManager.Task::(_debug) Task b1291b2b-69e6-4d6f-8a48-6c682fa29bf3: moving from state preparing -> state finished
>Thread-15917120::DEBUG::2013-11-20 04:54:29,073::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917120::DEBUG::2013-11-20 04:54:29,074::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917120::DEBUG::2013-11-20 04:54:29,074::task::495::TaskManager.Task::(_debug) Task b1291b2b-69e6-4d6f-8a48-6c682fa29bf3: ref 0 aborting False
>Thread-15917120::INFO::2013-11-20 04:54:29,075::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917121::DEBUG::2013-11-20 04:54:29,081::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917121::INFO::2013-11-20 04:54:29,082::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917121::DEBUG::2013-11-20 04:54:29,083::task::495::TaskManager.Task::(_debug) Task ce7ebccf-f4c9-4bce-9049-8b76ef9abf2b: moving from state init -> state preparing
>Thread-15917121::DEBUG::2013-11-20 04:54:29,084::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c41d8eaa-da02-4043-8923-d7c166005a07`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917121::DEBUG::2013-11-20 04:54:29,085::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917121::DEBUG::2013-11-20 04:54:29,085::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917121::DEBUG::2013-11-20 04:54:29,086::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c41d8eaa-da02-4043-8923-d7c166005a07`::Granted request
>Thread-15917121::DEBUG::2013-11-20 04:54:29,087::task::495::TaskManager.Task::(_debug) Task ce7ebccf-f4c9-4bce-9049-8b76ef9abf2b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917121::DEBUG::2013-11-20 04:54:29,088::task::495::TaskManager.Task::(_debug) Task ce7ebccf-f4c9-4bce-9049-8b76ef9abf2b: ref 1 aborting False
>Thread-15917121::WARNING::2013-11-20 04:54:29,089::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917121::DEBUG::2013-11-20 04:54:29,090::task::495::TaskManager.Task::(_debug) Task ce7ebccf-f4c9-4bce-9049-8b76ef9abf2b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917121::DEBUG::2013-11-20 04:54:29,091::task::495::TaskManager.Task::(_debug) Task ce7ebccf-f4c9-4bce-9049-8b76ef9abf2b: moving from state preparing -> state finished
>Thread-15917121::DEBUG::2013-11-20 04:54:29,092::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917121::DEBUG::2013-11-20 04:54:29,092::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917121::DEBUG::2013-11-20 04:54:29,093::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917121::DEBUG::2013-11-20 04:54:29,094::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917121::DEBUG::2013-11-20 04:54:29,094::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917121::DEBUG::2013-11-20 04:54:29,095::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917121::DEBUG::2013-11-20 04:54:29,096::task::495::TaskManager.Task::(_debug) Task ce7ebccf-f4c9-4bce-9049-8b76ef9abf2b: ref 0 aborting False
>Thread-15917121::INFO::2013-11-20 04:54:29,096::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:54:30,692::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:54:31,260::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.097332 s, 10.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:54:33,279::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 04:54:33,520::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:54:33,522::task::495::TaskManager.Task::(_debug) Task 8a668a96-a463-4cc4-8911-d2fe253ff39c: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:54:33,530::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2976d29e-d390-4bf6-83de-2af3204f2d46`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:54:33,532::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:54:33,533::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:54:33,534::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2976d29e-d390-4bf6-83de-2af3204f2d46`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:54:33,536::task::495::TaskManager.Task::(_debug) Task 8a668a96-a463-4cc4-8911-d2fe253ff39c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:54:33,536::task::495::TaskManager.Task::(_debug) Task 8a668a96-a463-4cc4-8911-d2fe253ff39c: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:54:33,537::task::495::TaskManager.Task::(_debug) Task 8a668a96-a463-4cc4-8911-d2fe253ff39c: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:54:33,538::task::495::TaskManager.Task::(_debug) Task 8a668a96-a463-4cc4-8911-d2fe253ff39c: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:54:33,539::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:54:33,540::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:54:33,540::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:54:33,541::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:54:33,542::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:54:33,543::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:54:33,543::task::495::TaskManager.Task::(_debug) Task 8a668a96-a463-4cc4-8911-d2fe253ff39c: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:54:33,544::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:54:33,818::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.077438 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 04:54:35,529::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:54:35,529::task::495::TaskManager.Task::(_debug) Task 86f719c4-2fbd-44fe-85d3-c1b19f8a4731: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:54:35,531::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`333e3f61-aec7-4687-946c-cfdb7cd9fb8f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:54:35,532::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:54:35,533::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:54:35,534::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`333e3f61-aec7-4687-946c-cfdb7cd9fb8f`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:54:35,536::task::495::TaskManager.Task::(_debug) Task 86f719c4-2fbd-44fe-85d3-c1b19f8a4731: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:54:35,536::task::495::TaskManager.Task::(_debug) Task 86f719c4-2fbd-44fe-85d3-c1b19f8a4731: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:54:35,537::task::495::TaskManager.Task::(_debug) Task 86f719c4-2fbd-44fe-85d3-c1b19f8a4731: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:54:35,538::task::495::TaskManager.Task::(_debug) Task 86f719c4-2fbd-44fe-85d3-c1b19f8a4731: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:54:35,538::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:54:35,539::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:54:35,540::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:54:35,540::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:54:35,541::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:54:35,541::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:54:35,542::task::495::TaskManager.Task::(_debug) Task 86f719c4-2fbd-44fe-85d3-c1b19f8a4731: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:54:35,542::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:54:35,543::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:54:35,544::task::495::TaskManager.Task::(_debug) Task 770de0a3-5dff-47fc-8b87-9dfc1a6f7132: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:54:35,545::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ef28a30c-bfa9-40d3-9569-4e0a67fd1215`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:54:35,545::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:54:35,546::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:54:35,547::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ef28a30c-bfa9-40d3-9569-4e0a67fd1215`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:54:35,548::task::495::TaskManager.Task::(_debug) Task 770de0a3-5dff-47fc-8b87-9dfc1a6f7132: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:54:35,548::task::495::TaskManager.Task::(_debug) Task 770de0a3-5dff-47fc-8b87-9dfc1a6f7132: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:54:35,549::task::495::TaskManager.Task::(_debug) Task 770de0a3-5dff-47fc-8b87-9dfc1a6f7132: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:54:35,550::task::495::TaskManager.Task::(_debug) Task 770de0a3-5dff-47fc-8b87-9dfc1a6f7132: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:54:35,550::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:54:35,551::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:54:35,551::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:54:35,552::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:54:35,552::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:54:35,553::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:54:35,554::task::495::TaskManager.Task::(_debug) Task 770de0a3-5dff-47fc-8b87-9dfc1a6f7132: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:54:35,554::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:54:35,833::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:54:36,368::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0834315 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:54:38,384::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:54:38,935::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0784746 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15917126::DEBUG::2013-11-20 04:54:39,249::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917126::INFO::2013-11-20 04:54:39,251::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917126::DEBUG::2013-11-20 04:54:39,252::task::495::TaskManager.Task::(_debug) Task 26f3a746-3f93-4776-803a-272c7ad0346e: moving from state init -> state preparing
>Thread-15917126::DEBUG::2013-11-20 04:54:39,253::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917126::DEBUG::2013-11-20 04:54:39,253::task::495::TaskManager.Task::(_debug) Task 26f3a746-3f93-4776-803a-272c7ad0346e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917126::DEBUG::2013-11-20 04:54:39,254::task::495::TaskManager.Task::(_debug) Task 26f3a746-3f93-4776-803a-272c7ad0346e: moving from state preparing -> state finished
>Thread-15917126::DEBUG::2013-11-20 04:54:39,254::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917126::DEBUG::2013-11-20 04:54:39,255::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917126::DEBUG::2013-11-20 04:54:39,255::task::495::TaskManager.Task::(_debug) Task 26f3a746-3f93-4776-803a-272c7ad0346e: ref 0 aborting False
>Thread-15917126::INFO::2013-11-20 04:54:39,256::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917127::DEBUG::2013-11-20 04:54:39,261::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917127::INFO::2013-11-20 04:54:39,262::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917127::DEBUG::2013-11-20 04:54:39,263::task::495::TaskManager.Task::(_debug) Task a035fc35-fe6b-4e84-ada4-f3651fdf2d84: moving from state init -> state preparing
>Thread-15917127::DEBUG::2013-11-20 04:54:39,264::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`df839004-297b-42e9-9843-35492c4b9512`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917127::DEBUG::2013-11-20 04:54:39,264::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917127::DEBUG::2013-11-20 04:54:39,265::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917127::DEBUG::2013-11-20 04:54:39,266::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`df839004-297b-42e9-9843-35492c4b9512`::Granted request
>Thread-15917127::DEBUG::2013-11-20 04:54:39,267::task::495::TaskManager.Task::(_debug) Task a035fc35-fe6b-4e84-ada4-f3651fdf2d84: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917127::DEBUG::2013-11-20 04:54:39,267::task::495::TaskManager.Task::(_debug) Task a035fc35-fe6b-4e84-ada4-f3651fdf2d84: ref 1 aborting False
>Thread-15917127::WARNING::2013-11-20 04:54:39,269::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917127::DEBUG::2013-11-20 04:54:39,270::task::495::TaskManager.Task::(_debug) Task a035fc35-fe6b-4e84-ada4-f3651fdf2d84: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917127::DEBUG::2013-11-20 04:54:39,271::task::495::TaskManager.Task::(_debug) Task a035fc35-fe6b-4e84-ada4-f3651fdf2d84: moving from state preparing -> state finished
>Thread-15917127::DEBUG::2013-11-20 04:54:39,272::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917127::DEBUG::2013-11-20 04:54:39,272::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917127::DEBUG::2013-11-20 04:54:39,273::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917127::DEBUG::2013-11-20 04:54:39,274::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917127::DEBUG::2013-11-20 04:54:39,274::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917127::DEBUG::2013-11-20 04:54:39,275::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917127::DEBUG::2013-11-20 04:54:39,275::task::495::TaskManager.Task::(_debug) Task a035fc35-fe6b-4e84-ada4-f3651fdf2d84: ref 0 aborting False
>Thread-15917127::INFO::2013-11-20 04:54:39,276::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:54:40,951::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:54:41,516::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0817875 s, 12.5 MB/s\n'; <rc> = 0
>Thread-15917128::INFO::2013-11-20 04:54:42,100::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15917128::DEBUG::2013-11-20 04:54:42,101::task::495::TaskManager.Task::(_debug) Task b1672498-acde-4221-ba15-7e1197c78f4a: moving from state init -> state preparing
>Thread-15917128::DEBUG::2013-11-20 04:54:42,102::task::495::TaskManager.Task::(_debug) Task b1672498-acde-4221-ba15-7e1197c78f4a: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000414133071899', 'lastCheck': 1384923280.6090529, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00151181221008', 'lastCheck': 1384923281.190428, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00184392929077', 'lastCheck': 1384923281.1896739, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000438928604126', 'lastCheck': 1384923273.915359, 'valid': True, 'code': 0}}
>Thread-15917128::DEBUG::2013-11-20 04:54:42,102::task::495::TaskManager.Task::(_debug) Task b1672498-acde-4221-ba15-7e1197c78f4a: moving from state preparing -> state finished
>Thread-15917128::DEBUG::2013-11-20 04:54:42,103::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917128::DEBUG::2013-11-20 04:54:42,104::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917128::DEBUG::2013-11-20 04:54:42,104::task::495::TaskManager.Task::(_debug) Task b1672498-acde-4221-ba15-7e1197c78f4a: ref 0 aborting False
>Thread-15917128::INFO::2013-11-20 04:54:42,105::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000414133071899', 'lastCheck': 1384923280.6090529, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00151181221008', 'lastCheck': 1384923281.190428, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00184392929077', 'lastCheck': 1384923281.1896739, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000438928604126', 'lastCheck': 1384923273.915359, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:54:43,540::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 04:54:43,756::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:54:43,760::task::495::TaskManager.Task::(_debug) Task 3226a397-b4a6-4f98-8480-b9d9d64076bc: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:54:43,764::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e82e2e6d-0d54-418d-9815-aabcfd0364ed`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:54:43,766::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:54:43,769::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:54:43,769::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e82e2e6d-0d54-418d-9815-aabcfd0364ed`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:54:43,771::task::495::TaskManager.Task::(_debug) Task 3226a397-b4a6-4f98-8480-b9d9d64076bc: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:54:43,772::task::495::TaskManager.Task::(_debug) Task 3226a397-b4a6-4f98-8480-b9d9d64076bc: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:54:43,773::task::495::TaskManager.Task::(_debug) Task 3226a397-b4a6-4f98-8480-b9d9d64076bc: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:54:43,774::task::495::TaskManager.Task::(_debug) Task 3226a397-b4a6-4f98-8480-b9d9d64076bc: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:54:43,775::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:54:43,775::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:54:43,776::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:54:43,777::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:54:43,777::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:54:43,778::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:54:43,778::task::495::TaskManager.Task::(_debug) Task 3226a397-b4a6-4f98-8480-b9d9d64076bc: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:54:43,779::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:54:44,099::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0772979 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:54:46,114::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 04:54:46,346::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:54:46,357::task::495::TaskManager.Task::(_debug) Task 59e338e9-64bd-43dc-9d09-0cc7548e7692: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:54:46,371::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`757a3e49-8729-473b-8cc1-fb3f53ae6c11`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:54:46,378::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:54:46,383::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:54:46,385::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`757a3e49-8729-473b-8cc1-fb3f53ae6c11`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:54:46,386::task::495::TaskManager.Task::(_debug) Task 59e338e9-64bd-43dc-9d09-0cc7548e7692: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:54:46,387::task::495::TaskManager.Task::(_debug) Task 59e338e9-64bd-43dc-9d09-0cc7548e7692: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:54:46,387::task::495::TaskManager.Task::(_debug) Task 59e338e9-64bd-43dc-9d09-0cc7548e7692: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:54:46,388::task::495::TaskManager.Task::(_debug) Task 59e338e9-64bd-43dc-9d09-0cc7548e7692: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:54:46,389::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:54:46,389::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:54:46,390::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:54:46,391::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:54:46,391::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:54:46,392::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:54:46,392::task::495::TaskManager.Task::(_debug) Task 59e338e9-64bd-43dc-9d09-0cc7548e7692: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:54:46,393::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:54:46,702::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.102977 s, 9.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:54:48,717::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:54:49,311::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0856786 s, 12.0 MB/s\n'; <rc> = 0
>Thread-15917132::DEBUG::2013-11-20 04:54:49,384::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917132::INFO::2013-11-20 04:54:49,387::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917132::DEBUG::2013-11-20 04:54:49,388::task::495::TaskManager.Task::(_debug) Task e37c18e3-0cca-43a6-a477-a1e2eaff1d49: moving from state init -> state preparing
>Thread-15917132::DEBUG::2013-11-20 04:54:49,389::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917132::DEBUG::2013-11-20 04:54:49,389::task::495::TaskManager.Task::(_debug) Task e37c18e3-0cca-43a6-a477-a1e2eaff1d49: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917132::DEBUG::2013-11-20 04:54:49,389::task::495::TaskManager.Task::(_debug) Task e37c18e3-0cca-43a6-a477-a1e2eaff1d49: moving from state preparing -> state finished
>Thread-15917132::DEBUG::2013-11-20 04:54:49,390::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917132::DEBUG::2013-11-20 04:54:49,390::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917132::DEBUG::2013-11-20 04:54:49,391::task::495::TaskManager.Task::(_debug) Task e37c18e3-0cca-43a6-a477-a1e2eaff1d49: ref 0 aborting False
>Thread-15917132::INFO::2013-11-20 04:54:49,391::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917133::DEBUG::2013-11-20 04:54:49,396::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917133::INFO::2013-11-20 04:54:49,397::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917133::DEBUG::2013-11-20 04:54:49,397::task::495::TaskManager.Task::(_debug) Task a3fcc2db-4b89-4bc0-8e10-d34da9ed47f2: moving from state init -> state preparing
>Thread-15917133::DEBUG::2013-11-20 04:54:49,398::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e533af9d-9020-4c51-aefa-7c09d0f27557`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917133::DEBUG::2013-11-20 04:54:49,398::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917133::DEBUG::2013-11-20 04:54:49,399::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917133::DEBUG::2013-11-20 04:54:49,399::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`e533af9d-9020-4c51-aefa-7c09d0f27557`::Granted request
>Thread-15917133::DEBUG::2013-11-20 04:54:49,400::task::495::TaskManager.Task::(_debug) Task a3fcc2db-4b89-4bc0-8e10-d34da9ed47f2: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917133::DEBUG::2013-11-20 04:54:49,401::task::495::TaskManager.Task::(_debug) Task a3fcc2db-4b89-4bc0-8e10-d34da9ed47f2: ref 1 aborting False
>Thread-15917133::WARNING::2013-11-20 04:54:49,402::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917133::DEBUG::2013-11-20 04:54:49,403::task::495::TaskManager.Task::(_debug) Task a3fcc2db-4b89-4bc0-8e10-d34da9ed47f2: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917133::DEBUG::2013-11-20 04:54:49,404::task::495::TaskManager.Task::(_debug) Task a3fcc2db-4b89-4bc0-8e10-d34da9ed47f2: moving from state preparing -> state finished
>Thread-15917133::DEBUG::2013-11-20 04:54:49,404::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917133::DEBUG::2013-11-20 04:54:49,405::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917133::DEBUG::2013-11-20 04:54:49,406::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917133::DEBUG::2013-11-20 04:54:49,406::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917133::DEBUG::2013-11-20 04:54:49,407::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917133::DEBUG::2013-11-20 04:54:49,407::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917133::DEBUG::2013-11-20 04:54:49,408::task::495::TaskManager.Task::(_debug) Task a3fcc2db-4b89-4bc0-8e10-d34da9ed47f2: ref 0 aborting False
>Thread-15917133::INFO::2013-11-20 04:54:49,409::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:54:51,322::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:54:51,844::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0876068 s, 11.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:54:53,862::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 04:54:54,075::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15913546::INFO::2013-11-20 04:54:54,080::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:54:54,083::task::495::TaskManager.Task::(_debug) Task c1c1d2be-af01-43e3-9ca9-0e81ceeef741: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:54:54,082::task::495::TaskManager.Task::(_debug) Task 933fefbd-6ab1-49ab-8bcb-b04d8c3adc77: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:54:54,089::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a2bac423-f179-4095-9cfc-8f2ae033b7a9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:54:54,090::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6d4c7c95-846a-46ec-97b8-1c7c294904b6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:54:54,091::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:54:54,092::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:54:54,095::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:54:54,096::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a2bac423-f179-4095-9cfc-8f2ae033b7a9`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:54:54,097::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:54:54,099::task::495::TaskManager.Task::(_debug) Task c1c1d2be-af01-43e3-9ca9-0e81ceeef741: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15914519::DEBUG::2013-11-20 04:54:54,099::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6d4c7c95-846a-46ec-97b8-1c7c294904b6`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:54:54,100::task::495::TaskManager.Task::(_debug) Task c1c1d2be-af01-43e3-9ca9-0e81ceeef741: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:54:54,102::task::495::TaskManager.Task::(_debug) Task c1c1d2be-af01-43e3-9ca9-0e81ceeef741: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:54:54,103::task::495::TaskManager.Task::(_debug) Task 933fefbd-6ab1-49ab-8bcb-b04d8c3adc77: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913546::DEBUG::2013-11-20 04:54:54,104::task::495::TaskManager.Task::(_debug) Task c1c1d2be-af01-43e3-9ca9-0e81ceeef741: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:54:54,104::task::495::TaskManager.Task::(_debug) Task 933fefbd-6ab1-49ab-8bcb-b04d8c3adc77: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:54:54,105::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:54:54,107::task::495::TaskManager.Task::(_debug) Task 933fefbd-6ab1-49ab-8bcb-b04d8c3adc77: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:54:54,107::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:54:54,108::task::495::TaskManager.Task::(_debug) Task 933fefbd-6ab1-49ab-8bcb-b04d8c3adc77: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:54:54,109::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15914519::DEBUG::2013-11-20 04:54:54,110::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:54:54,111::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:54:54,112::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:54:54,113::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:54:54,114::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913546::DEBUG::2013-11-20 04:54:54,115::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:54:54,116::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:54:54,117::task::495::TaskManager.Task::(_debug) Task c1c1d2be-af01-43e3-9ca9-0e81ceeef741: ref 0 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:54:54,117::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913546::INFO::2013-11-20 04:54:54,118::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:54:54,119::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913546::INFO::2013-11-20 04:54:54,120::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15914519::DEBUG::2013-11-20 04:54:54,121::task::495::TaskManager.Task::(_debug) Task 933fefbd-6ab1-49ab-8bcb-b04d8c3adc77: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:54:54,121::task::495::TaskManager.Task::(_debug) Task 698ab61f-bd81-4ac9-b2b0-03728b8f19af: moving from state init -> state preparing
>Thread-15914519::INFO::2013-11-20 04:54:54,122::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:54:54,124::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`80108665-d2b2-450b-b8f9-3b638f2774ff`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:54:54,125::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:54:54,125::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:54:54,126::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`80108665-d2b2-450b-b8f9-3b638f2774ff`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:54:54,127::task::495::TaskManager.Task::(_debug) Task 698ab61f-bd81-4ac9-b2b0-03728b8f19af: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:54:54,128::task::495::TaskManager.Task::(_debug) Task 698ab61f-bd81-4ac9-b2b0-03728b8f19af: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:54:54,128::task::495::TaskManager.Task::(_debug) Task 698ab61f-bd81-4ac9-b2b0-03728b8f19af: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:54:54,129::task::495::TaskManager.Task::(_debug) Task 698ab61f-bd81-4ac9-b2b0-03728b8f19af: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:54:54,130::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:54:54,130::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:54:54,132::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:54:54,132::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:54:54,133::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:54:54,133::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:54:54,134::task::495::TaskManager.Task::(_debug) Task 698ab61f-bd81-4ac9-b2b0-03728b8f19af: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:54:54,135::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:54:54,136::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:54:54,136::task::495::TaskManager.Task::(_debug) Task d688e752-e26d-40d1-8e31-25f4db9cd3d8: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:54:54,137::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6245aa63-500e-487c-9dae-8861e3b4a7f8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:54:54,138::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:54:54,139::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:54:54,139::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`6245aa63-500e-487c-9dae-8861e3b4a7f8`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:54:54,140::task::495::TaskManager.Task::(_debug) Task d688e752-e26d-40d1-8e31-25f4db9cd3d8: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:54:54,141::task::495::TaskManager.Task::(_debug) Task d688e752-e26d-40d1-8e31-25f4db9cd3d8: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:54:54,142::task::495::TaskManager.Task::(_debug) Task d688e752-e26d-40d1-8e31-25f4db9cd3d8: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:54:54,143::task::495::TaskManager.Task::(_debug) Task d688e752-e26d-40d1-8e31-25f4db9cd3d8: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:54:54,143::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:54:54,144::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:54:54,144::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:54:54,145::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:54:54,145::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:54:54,146::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:54:54,147::task::495::TaskManager.Task::(_debug) Task d688e752-e26d-40d1-8e31-25f4db9cd3d8: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:54:54,148::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:54:54,148::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:54:54,149::task::495::TaskManager.Task::(_debug) Task 3b59bf42-23ec-47fe-a3c9-2b80c1bf8aae: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:54:54,150::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`67330951-96c2-4bce-82c0-f2574318d153`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:54:54,151::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:54:54,151::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:54:54,152::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`67330951-96c2-4bce-82c0-f2574318d153`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:54:54,153::task::495::TaskManager.Task::(_debug) Task 3b59bf42-23ec-47fe-a3c9-2b80c1bf8aae: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:54:54,153::task::495::TaskManager.Task::(_debug) Task 3b59bf42-23ec-47fe-a3c9-2b80c1bf8aae: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:54:54,154::task::495::TaskManager.Task::(_debug) Task 3b59bf42-23ec-47fe-a3c9-2b80c1bf8aae: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:54:54,159::task::495::TaskManager.Task::(_debug) Task 3b59bf42-23ec-47fe-a3c9-2b80c1bf8aae: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:54:54,160::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:54:54,160::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:54:54,161::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:54:54,161::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:54:54,162::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:54:54,163::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:54:54,163::task::495::TaskManager.Task::(_debug) Task 3b59bf42-23ec-47fe-a3c9-2b80c1bf8aae: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:54:54,164::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:54:54,415::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0756005 s, 13.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:54:56,425::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:54:56,962::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0798226 s, 12.8 MB/s\n'; <rc> = 0
>Thread-15917136::INFO::2013-11-20 04:54:57,586::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15917136::DEBUG::2013-11-20 04:54:57,587::task::495::TaskManager.Task::(_debug) Task 3c34b6ff-d5b8-4a47-b143-3f76e688cd79: moving from state init -> state preparing
>Thread-15917136::DEBUG::2013-11-20 04:54:57,588::task::495::TaskManager.Task::(_debug) Task 3c34b6ff-d5b8-4a47-b143-3f76e688cd79: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000489950180054', 'lastCheck': 1384923290.9651959, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00253105163574', 'lastCheck': 1384923291.5401571, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00176191329956', 'lastCheck': 1384923291.541024, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000766038894653', 'lastCheck': 1384923294.5126481, 'valid': True, 'code': 0}}
>Thread-15917136::DEBUG::2013-11-20 04:54:57,589::task::495::TaskManager.Task::(_debug) Task 3c34b6ff-d5b8-4a47-b143-3f76e688cd79: moving from state preparing -> state finished
>Thread-15917136::DEBUG::2013-11-20 04:54:57,589::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917136::DEBUG::2013-11-20 04:54:57,590::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917136::DEBUG::2013-11-20 04:54:57,591::task::495::TaskManager.Task::(_debug) Task 3c34b6ff-d5b8-4a47-b143-3f76e688cd79: ref 0 aborting False
>Thread-15917136::INFO::2013-11-20 04:54:57,592::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000489950180054', 'lastCheck': 1384923290.9651959, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00253105163574', 'lastCheck': 1384923291.5401571, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00176191329956', 'lastCheck': 1384923291.541024, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000766038894653', 'lastCheck': 1384923294.5126481, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:54:58,978::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15917138::DEBUG::2013-11-20 04:54:59,542::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917138::INFO::2013-11-20 04:54:59,545::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917138::DEBUG::2013-11-20 04:54:59,546::task::495::TaskManager.Task::(_debug) Task b665097e-641b-46d7-9b00-7a61c6b38eae: moving from state init -> state preparing
>Thread-15917138::DEBUG::2013-11-20 04:54:59,548::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917138::DEBUG::2013-11-20 04:54:59,549::task::495::TaskManager.Task::(_debug) Task b665097e-641b-46d7-9b00-7a61c6b38eae: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917138::DEBUG::2013-11-20 04:54:59,550::task::495::TaskManager.Task::(_debug) Task b665097e-641b-46d7-9b00-7a61c6b38eae: moving from state preparing -> state finished
>Thread-15917138::DEBUG::2013-11-20 04:54:59,552::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917138::DEBUG::2013-11-20 04:54:59,553::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917138::DEBUG::2013-11-20 04:54:59,556::task::495::TaskManager.Task::(_debug) Task b665097e-641b-46d7-9b00-7a61c6b38eae: ref 0 aborting False
>Thread-15917138::INFO::2013-11-20 04:54:59,557::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917139::DEBUG::2013-11-20 04:54:59,564::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917139::INFO::2013-11-20 04:54:59,566::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917139::DEBUG::2013-11-20 04:54:59,566::task::495::TaskManager.Task::(_debug) Task 81735ff1-64f4-4e58-a2ac-dd8a0fa3c8ac: moving from state init -> state preparing
>Dummy-15912651::DEBUG::2013-11-20 04:54:59,569::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0771529 s, 13.3 MB/s\n'; <rc> = 0
>Thread-15917139::DEBUG::2013-11-20 04:54:59,569::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`afa68dc8-632f-4bbd-bb6e-c4f0a523f8e2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917139::DEBUG::2013-11-20 04:54:59,572::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917139::DEBUG::2013-11-20 04:54:59,581::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917139::DEBUG::2013-11-20 04:54:59,583::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`afa68dc8-632f-4bbd-bb6e-c4f0a523f8e2`::Granted request
>Thread-15917139::DEBUG::2013-11-20 04:54:59,584::task::495::TaskManager.Task::(_debug) Task 81735ff1-64f4-4e58-a2ac-dd8a0fa3c8ac: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917139::DEBUG::2013-11-20 04:54:59,585::task::495::TaskManager.Task::(_debug) Task 81735ff1-64f4-4e58-a2ac-dd8a0fa3c8ac: ref 1 aborting False
>Thread-15917139::WARNING::2013-11-20 04:54:59,587::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917139::DEBUG::2013-11-20 04:54:59,588::task::495::TaskManager.Task::(_debug) Task 81735ff1-64f4-4e58-a2ac-dd8a0fa3c8ac: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917139::DEBUG::2013-11-20 04:54:59,589::task::495::TaskManager.Task::(_debug) Task 81735ff1-64f4-4e58-a2ac-dd8a0fa3c8ac: moving from state preparing -> state finished
>Thread-15917139::DEBUG::2013-11-20 04:54:59,589::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917139::DEBUG::2013-11-20 04:54:59,590::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917139::DEBUG::2013-11-20 04:54:59,591::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917139::DEBUG::2013-11-20 04:54:59,592::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917139::DEBUG::2013-11-20 04:54:59,592::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917139::DEBUG::2013-11-20 04:54:59,593::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917139::DEBUG::2013-11-20 04:54:59,594::task::495::TaskManager.Task::(_debug) Task 81735ff1-64f4-4e58-a2ac-dd8a0fa3c8ac: ref 0 aborting False
>Thread-15917139::INFO::2013-11-20 04:54:59,595::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:55:01,585::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:55:02,176::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0911815 s, 11.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:55:04,195::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 04:55:04,476::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:55:04,478::task::495::TaskManager.Task::(_debug) Task 3b44053f-db9d-4c5c-be72-7352aa6934ad: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:55:04,481::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f6ee8964-c49d-4077-a105-dfbf1e775957`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:55:04,482::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:55:04,483::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:55:04,483::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f6ee8964-c49d-4077-a105-dfbf1e775957`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:55:04,484::task::495::TaskManager.Task::(_debug) Task 3b44053f-db9d-4c5c-be72-7352aa6934ad: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:55:04,485::task::495::TaskManager.Task::(_debug) Task 3b44053f-db9d-4c5c-be72-7352aa6934ad: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:55:04,486::task::495::TaskManager.Task::(_debug) Task 3b44053f-db9d-4c5c-be72-7352aa6934ad: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:55:04,486::task::495::TaskManager.Task::(_debug) Task 3b44053f-db9d-4c5c-be72-7352aa6934ad: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:55:04,487::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:55:04,487::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:55:04,488::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:55:04,489::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:55:04,489::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:55:04,490::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:55:04,490::task::495::TaskManager.Task::(_debug) Task 3b44053f-db9d-4c5c-be72-7352aa6934ad: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:55:04,491::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:55:04,756::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0589036 s, 17.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:55:06,771::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:55:07,342::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0811182 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:55:09,359::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15917143::DEBUG::2013-11-20 04:55:09,708::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917143::INFO::2013-11-20 04:55:09,710::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917143::DEBUG::2013-11-20 04:55:09,711::task::495::TaskManager.Task::(_debug) Task 492f9d66-b38d-4b36-9e89-f9f37d057143: moving from state init -> state preparing
>Thread-15917143::DEBUG::2013-11-20 04:55:09,712::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917143::DEBUG::2013-11-20 04:55:09,713::task::495::TaskManager.Task::(_debug) Task 492f9d66-b38d-4b36-9e89-f9f37d057143: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917143::DEBUG::2013-11-20 04:55:09,713::task::495::TaskManager.Task::(_debug) Task 492f9d66-b38d-4b36-9e89-f9f37d057143: moving from state preparing -> state finished
>Thread-15917143::DEBUG::2013-11-20 04:55:09,714::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917143::DEBUG::2013-11-20 04:55:09,715::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917143::DEBUG::2013-11-20 04:55:09,715::task::495::TaskManager.Task::(_debug) Task 492f9d66-b38d-4b36-9e89-f9f37d057143: ref 0 aborting False
>Thread-15917143::INFO::2013-11-20 04:55:09,716::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917144::DEBUG::2013-11-20 04:55:09,722::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917144::INFO::2013-11-20 04:55:09,723::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917144::DEBUG::2013-11-20 04:55:09,724::task::495::TaskManager.Task::(_debug) Task 36c537a9-5672-4d58-8115-f9f23a1fd6c0: moving from state init -> state preparing
>Thread-15917144::DEBUG::2013-11-20 04:55:09,725::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`37d8b335-bead-4f94-9c6a-8126f838fcc9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917144::DEBUG::2013-11-20 04:55:09,726::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917144::DEBUG::2013-11-20 04:55:09,728::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917144::DEBUG::2013-11-20 04:55:09,728::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`37d8b335-bead-4f94-9c6a-8126f838fcc9`::Granted request
>Thread-15917144::DEBUG::2013-11-20 04:55:09,730::task::495::TaskManager.Task::(_debug) Task 36c537a9-5672-4d58-8115-f9f23a1fd6c0: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917144::DEBUG::2013-11-20 04:55:09,732::task::495::TaskManager.Task::(_debug) Task 36c537a9-5672-4d58-8115-f9f23a1fd6c0: ref 1 aborting False
>Thread-15917144::WARNING::2013-11-20 04:55:09,736::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917144::DEBUG::2013-11-20 04:55:09,739::task::495::TaskManager.Task::(_debug) Task 36c537a9-5672-4d58-8115-f9f23a1fd6c0: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917144::DEBUG::2013-11-20 04:55:09,740::task::495::TaskManager.Task::(_debug) Task 36c537a9-5672-4d58-8115-f9f23a1fd6c0: moving from state preparing -> state finished
>Thread-15917144::DEBUG::2013-11-20 04:55:09,742::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917144::DEBUG::2013-11-20 04:55:09,743::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917144::DEBUG::2013-11-20 04:55:09,745::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917144::DEBUG::2013-11-20 04:55:09,746::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917144::DEBUG::2013-11-20 04:55:09,747::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917144::DEBUG::2013-11-20 04:55:09,748::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917144::DEBUG::2013-11-20 04:55:09,750::task::495::TaskManager.Task::(_debug) Task 36c537a9-5672-4d58-8115-f9f23a1fd6c0: ref 0 aborting False
>Thread-15917144::INFO::2013-11-20 04:55:09,751::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:55:09,898::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.104158 s, 9.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:55:11,913::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:55:12,448::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0780075 s, 13.1 MB/s\n'; <rc> = 0
>Thread-15917146::INFO::2013-11-20 04:55:13,060::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15917146::DEBUG::2013-11-20 04:55:13,062::task::495::TaskManager.Task::(_debug) Task f0694fde-1347-4bc6-8260-c87e7ce35e5b: moving from state init -> state preparing
>Thread-15917146::DEBUG::2013-11-20 04:55:13,062::task::495::TaskManager.Task::(_debug) Task f0694fde-1347-4bc6-8260-c87e7ce35e5b: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00036883354187', 'lastCheck': 1384923311.5841391, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00140190124512', 'lastCheck': 1384923312.1219361, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00201082229614', 'lastCheck': 1384923312.12204, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000492095947266', 'lastCheck': 1384923304.809463, 'valid': True, 'code': 0}}
>Thread-15917146::DEBUG::2013-11-20 04:55:13,063::task::495::TaskManager.Task::(_debug) Task f0694fde-1347-4bc6-8260-c87e7ce35e5b: moving from state preparing -> state finished
>Thread-15917146::DEBUG::2013-11-20 04:55:13,064::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917146::DEBUG::2013-11-20 04:55:13,065::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917146::DEBUG::2013-11-20 04:55:13,065::task::495::TaskManager.Task::(_debug) Task f0694fde-1347-4bc6-8260-c87e7ce35e5b: ref 0 aborting False
>Thread-15917146::INFO::2013-11-20 04:55:13,066::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00036883354187', 'lastCheck': 1384923311.5841391, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00140190124512', 'lastCheck': 1384923312.1219361, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00201082229614', 'lastCheck': 1384923312.12204, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000492095947266', 'lastCheck': 1384923304.809463, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:55:14,463::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:55:14,699::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:55:14,705::task::495::TaskManager.Task::(_debug) Task 0073541d-1145-4128-af93-d7ef1ecefe4e: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:55:14,707::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e459a613-ed3e-4b71-bded-e9fab68aa1a9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:55:14,708::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:55:14,709::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:55:14,710::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e459a613-ed3e-4b71-bded-e9fab68aa1a9`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:55:14,711::task::495::TaskManager.Task::(_debug) Task 0073541d-1145-4128-af93-d7ef1ecefe4e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:55:14,712::task::495::TaskManager.Task::(_debug) Task 0073541d-1145-4128-af93-d7ef1ecefe4e: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:55:14,713::task::495::TaskManager.Task::(_debug) Task 0073541d-1145-4128-af93-d7ef1ecefe4e: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:55:14,715::task::495::TaskManager.Task::(_debug) Task 0073541d-1145-4128-af93-d7ef1ecefe4e: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:55:14,716::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:55:14,717::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:55:14,718::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:55:14,719::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:55:14,720::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:55:14,721::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:55:14,722::task::495::TaskManager.Task::(_debug) Task 0073541d-1145-4128-af93-d7ef1ecefe4e: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:55:14,723::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:55:15,021::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0829605 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:55:17,037::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915918::INFO::2013-11-20 04:55:17,234::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:55:17,236::task::495::TaskManager.Task::(_debug) Task 78d932ba-9520-4317-94e9-2587e8d29949: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:55:17,239::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b07394a6-7e6a-4e2a-a6c4-c37659ed47c7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:55:17,240::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:55:17,242::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:55:17,243::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b07394a6-7e6a-4e2a-a6c4-c37659ed47c7`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:55:17,245::task::495::TaskManager.Task::(_debug) Task 78d932ba-9520-4317-94e9-2587e8d29949: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:55:17,245::task::495::TaskManager.Task::(_debug) Task 78d932ba-9520-4317-94e9-2587e8d29949: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:55:17,246::task::495::TaskManager.Task::(_debug) Task 78d932ba-9520-4317-94e9-2587e8d29949: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:55:17,247::task::495::TaskManager.Task::(_debug) Task 78d932ba-9520-4317-94e9-2587e8d29949: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:55:17,247::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:55:17,248::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:55:17,249::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:55:17,250::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:55:17,250::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:55:17,251::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:55:17,252::task::495::TaskManager.Task::(_debug) Task 78d932ba-9520-4317-94e9-2587e8d29949: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:55:17,252::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:55:17,572::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0829695 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:55:19,594::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916984::INFO::2013-11-20 04:55:19,824::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=240c0593-fe33-4f8b-a029-097379dbcb85 volUUID=f715faaa-081a-4598-b125-80a2d12af554)
>Thread-15916984::DEBUG::2013-11-20 04:55:19,829::task::495::TaskManager.Task::(_debug) Task a0f6f401-4edf-4cd9-9a7d-912c012b8d22: moving from state init -> state preparing
>Thread-15916984::DEBUG::2013-11-20 04:55:19,842::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1db27854-469b-4071-953a-60c38cbc4a5b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15916984::DEBUG::2013-11-20 04:55:19,851::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15916984::DEBUG::2013-11-20 04:55:19,859::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15916984::DEBUG::2013-11-20 04:55:19,862::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1db27854-469b-4071-953a-60c38cbc4a5b`::Granted request
>Thread-15916984::DEBUG::2013-11-20 04:55:19,870::task::495::TaskManager.Task::(_debug) Task a0f6f401-4edf-4cd9-9a7d-912c012b8d22: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15916984::DEBUG::2013-11-20 04:55:19,871::task::495::TaskManager.Task::(_debug) Task a0f6f401-4edf-4cd9-9a7d-912c012b8d22: ref 1 aborting False
>Thread-15916984::DEBUG::2013-11-20 04:55:19,876::task::495::TaskManager.Task::(_debug) Task a0f6f401-4edf-4cd9-9a7d-912c012b8d22: finished: {'truesize': '75296145408', 'apparentsize': '75296145408'}
>Thread-15916984::DEBUG::2013-11-20 04:55:19,884::task::495::TaskManager.Task::(_debug) Task a0f6f401-4edf-4cd9-9a7d-912c012b8d22: moving from state preparing -> state finished
>Thread-15916984::DEBUG::2013-11-20 04:55:19,887::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15916984::DEBUG::2013-11-20 04:55:19,890::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917150::DEBUG::2013-11-20 04:55:19,892::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15916984::DEBUG::2013-11-20 04:55:19,893::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15917150::INFO::2013-11-20 04:55:19,894::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15916984::DEBUG::2013-11-20 04:55:19,895::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15917150::DEBUG::2013-11-20 04:55:19,895::task::495::TaskManager.Task::(_debug) Task 0bc8dce5-b78c-4d8c-ab08-1f19bcdd10f5: moving from state init -> state preparing
>Thread-15916984::DEBUG::2013-11-20 04:55:19,899::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15917150::DEBUG::2013-11-20 04:55:19,900::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15916984::DEBUG::2013-11-20 04:55:19,907::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15917150::DEBUG::2013-11-20 04:55:19,907::task::495::TaskManager.Task::(_debug) Task 0bc8dce5-b78c-4d8c-ab08-1f19bcdd10f5: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15916984::DEBUG::2013-11-20 04:55:19,909::task::495::TaskManager.Task::(_debug) Task a0f6f401-4edf-4cd9-9a7d-912c012b8d22: ref 0 aborting False
>Thread-15917150::DEBUG::2013-11-20 04:55:19,909::task::495::TaskManager.Task::(_debug) Task 0bc8dce5-b78c-4d8c-ab08-1f19bcdd10f5: moving from state preparing -> state finished
>Thread-15916984::INFO::2013-11-20 04:55:19,910::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '75296145408', 'apparentsize': '75296145408'}
>Thread-15917150::DEBUG::2013-11-20 04:55:19,911::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917150::DEBUG::2013-11-20 04:55:19,912::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917150::DEBUG::2013-11-20 04:55:19,912::task::495::TaskManager.Task::(_debug) Task 0bc8dce5-b78c-4d8c-ab08-1f19bcdd10f5: ref 0 aborting False
>Thread-15917150::INFO::2013-11-20 04:55:19,912::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917151::DEBUG::2013-11-20 04:55:19,918::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917151::INFO::2013-11-20 04:55:19,920::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917151::DEBUG::2013-11-20 04:55:19,920::task::495::TaskManager.Task::(_debug) Task 6844c73e-b664-41b3-813f-3860903f80a8: moving from state init -> state preparing
>Thread-15917151::DEBUG::2013-11-20 04:55:19,922::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c22de528-4524-45d0-9963-975acc1db39d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917151::DEBUG::2013-11-20 04:55:19,923::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917151::DEBUG::2013-11-20 04:55:19,924::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917151::DEBUG::2013-11-20 04:55:19,925::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c22de528-4524-45d0-9963-975acc1db39d`::Granted request
>Thread-15917151::DEBUG::2013-11-20 04:55:19,926::task::495::TaskManager.Task::(_debug) Task 6844c73e-b664-41b3-813f-3860903f80a8: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917151::DEBUG::2013-11-20 04:55:19,927::task::495::TaskManager.Task::(_debug) Task 6844c73e-b664-41b3-813f-3860903f80a8: ref 1 aborting False
>Thread-15917151::WARNING::2013-11-20 04:55:19,929::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917151::DEBUG::2013-11-20 04:55:19,933::task::495::TaskManager.Task::(_debug) Task 6844c73e-b664-41b3-813f-3860903f80a8: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917151::DEBUG::2013-11-20 04:55:19,934::task::495::TaskManager.Task::(_debug) Task 6844c73e-b664-41b3-813f-3860903f80a8: moving from state preparing -> state finished
>Thread-15917151::DEBUG::2013-11-20 04:55:19,935::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917151::DEBUG::2013-11-20 04:55:19,936::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917151::DEBUG::2013-11-20 04:55:19,937::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917151::DEBUG::2013-11-20 04:55:19,937::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917151::DEBUG::2013-11-20 04:55:19,938::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917151::DEBUG::2013-11-20 04:55:19,939::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917151::DEBUG::2013-11-20 04:55:19,939::task::495::TaskManager.Task::(_debug) Task 6844c73e-b664-41b3-813f-3860903f80a8: ref 0 aborting False
>Thread-15917151::INFO::2013-11-20 04:55:19,940::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:55:20,179::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0911745 s, 11.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:55:22,188::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915863::INFO::2013-11-20 04:55:22,402::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:55:22,413::task::495::TaskManager.Task::(_debug) Task c9e7b8f7-d8a2-4e2f-ab2d-c42618dc7a27: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:55:22,418::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`399099ca-e226-483a-955f-c602786453e3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:55:22,420::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:55:22,424::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:55:22,427::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`399099ca-e226-483a-955f-c602786453e3`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:55:22,430::task::495::TaskManager.Task::(_debug) Task c9e7b8f7-d8a2-4e2f-ab2d-c42618dc7a27: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:55:22,431::task::495::TaskManager.Task::(_debug) Task c9e7b8f7-d8a2-4e2f-ab2d-c42618dc7a27: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:55:22,432::task::495::TaskManager.Task::(_debug) Task c9e7b8f7-d8a2-4e2f-ab2d-c42618dc7a27: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:55:22,433::task::495::TaskManager.Task::(_debug) Task c9e7b8f7-d8a2-4e2f-ab2d-c42618dc7a27: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:55:22,433::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:55:22,434::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:55:22,435::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:55:22,436::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:55:22,437::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:55:22,437::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:55:22,439::task::495::TaskManager.Task::(_debug) Task c9e7b8f7-d8a2-4e2f-ab2d-c42618dc7a27: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:55:22,440::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:55:22,775::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0876431 s, 11.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:55:24,797::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 04:55:25,025::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:55:25,029::task::495::TaskManager.Task::(_debug) Task 6b44a157-6c10-4c5f-8fb3-978144e3d1f9: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:55:25,033::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7eaeb2ce-db0f-4785-baa0-8bbae7438cb2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:55:25,034::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:55:25,035::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:55:25,036::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7eaeb2ce-db0f-4785-baa0-8bbae7438cb2`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:55:25,038::task::495::TaskManager.Task::(_debug) Task 6b44a157-6c10-4c5f-8fb3-978144e3d1f9: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:55:25,039::task::495::TaskManager.Task::(_debug) Task 6b44a157-6c10-4c5f-8fb3-978144e3d1f9: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:55:25,040::task::495::TaskManager.Task::(_debug) Task 6b44a157-6c10-4c5f-8fb3-978144e3d1f9: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:55:25,040::task::495::TaskManager.Task::(_debug) Task 6b44a157-6c10-4c5f-8fb3-978144e3d1f9: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:55:25,041::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:55:25,042::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:55:25,043::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:55:25,044::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:55:25,045::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:55:25,046::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:55:25,048::task::495::TaskManager.Task::(_debug) Task 6b44a157-6c10-4c5f-8fb3-978144e3d1f9: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:55:25,049::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:55:25,320::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0743879 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:55:27,335::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:55:27,902::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0855206 s, 12.0 MB/s\n'; <rc> = 0
>Thread-15917154::INFO::2013-11-20 04:55:28,581::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15917154::DEBUG::2013-11-20 04:55:28,582::task::495::TaskManager.Task::(_debug) Task 14105e77-6fc3-41df-a73f-10370c6c7228: moving from state init -> state preparing
>Thread-15917154::DEBUG::2013-11-20 04:55:28,583::task::495::TaskManager.Task::(_debug) Task 14105e77-6fc3-41df-a73f-10370c6c7228: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00043797492981', 'lastCheck': 1384923321.8297629, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00257205963135', 'lastCheck': 1384923322.422087, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00624203681946', 'lastCheck': 1384923322.420366, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000458955764771', 'lastCheck': 1384923325.4006231, 'valid': True, 'code': 0}}
>Thread-15917154::DEBUG::2013-11-20 04:55:28,583::task::495::TaskManager.Task::(_debug) Task 14105e77-6fc3-41df-a73f-10370c6c7228: moving from state preparing -> state finished
>Thread-15917154::DEBUG::2013-11-20 04:55:28,584::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917154::DEBUG::2013-11-20 04:55:28,585::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917154::DEBUG::2013-11-20 04:55:28,585::task::495::TaskManager.Task::(_debug) Task 14105e77-6fc3-41df-a73f-10370c6c7228: ref 0 aborting False
>Thread-15917154::INFO::2013-11-20 04:55:28,586::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00043797492981', 'lastCheck': 1384923321.8297629, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00257205963135', 'lastCheck': 1384923322.422087, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00624203681946', 'lastCheck': 1384923322.420366, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000458955764771', 'lastCheck': 1384923325.4006231, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:55:29,919::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15917156::DEBUG::2013-11-20 04:55:30,134::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917156::INFO::2013-11-20 04:55:30,138::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917156::DEBUG::2013-11-20 04:55:30,139::task::495::TaskManager.Task::(_debug) Task 7748f55d-26c7-457d-8c9e-db4053bdf4aa: moving from state init -> state preparing
>Thread-15917156::DEBUG::2013-11-20 04:55:30,140::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917156::DEBUG::2013-11-20 04:55:30,141::task::495::TaskManager.Task::(_debug) Task 7748f55d-26c7-457d-8c9e-db4053bdf4aa: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917156::DEBUG::2013-11-20 04:55:30,142::task::495::TaskManager.Task::(_debug) Task 7748f55d-26c7-457d-8c9e-db4053bdf4aa: moving from state preparing -> state finished
>Thread-15917156::DEBUG::2013-11-20 04:55:30,142::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917156::DEBUG::2013-11-20 04:55:30,143::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917156::DEBUG::2013-11-20 04:55:30,143::task::495::TaskManager.Task::(_debug) Task 7748f55d-26c7-457d-8c9e-db4053bdf4aa: ref 0 aborting False
>Thread-15917156::INFO::2013-11-20 04:55:30,144::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917157::DEBUG::2013-11-20 04:55:30,150::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917157::INFO::2013-11-20 04:55:30,151::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917157::DEBUG::2013-11-20 04:55:30,152::task::495::TaskManager.Task::(_debug) Task 4a8210fd-ba56-42e3-b5e0-490928faacde: moving from state init -> state preparing
>Thread-15917157::DEBUG::2013-11-20 04:55:30,154::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`70c06dd7-0028-40f5-b38a-c545e85b04b2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917157::DEBUG::2013-11-20 04:55:30,155::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917157::DEBUG::2013-11-20 04:55:30,156::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917157::DEBUG::2013-11-20 04:55:30,157::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`70c06dd7-0028-40f5-b38a-c545e85b04b2`::Granted request
>Thread-15917157::DEBUG::2013-11-20 04:55:30,158::task::495::TaskManager.Task::(_debug) Task 4a8210fd-ba56-42e3-b5e0-490928faacde: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917157::DEBUG::2013-11-20 04:55:30,159::task::495::TaskManager.Task::(_debug) Task 4a8210fd-ba56-42e3-b5e0-490928faacde: ref 1 aborting False
>Thread-15917157::WARNING::2013-11-20 04:55:30,161::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917157::DEBUG::2013-11-20 04:55:30,162::task::495::TaskManager.Task::(_debug) Task 4a8210fd-ba56-42e3-b5e0-490928faacde: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917157::DEBUG::2013-11-20 04:55:30,163::task::495::TaskManager.Task::(_debug) Task 4a8210fd-ba56-42e3-b5e0-490928faacde: moving from state preparing -> state finished
>Thread-15917157::DEBUG::2013-11-20 04:55:30,163::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917157::DEBUG::2013-11-20 04:55:30,164::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917157::DEBUG::2013-11-20 04:55:30,165::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917157::DEBUG::2013-11-20 04:55:30,165::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917157::DEBUG::2013-11-20 04:55:30,166::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917157::DEBUG::2013-11-20 04:55:30,166::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917157::DEBUG::2013-11-20 04:55:30,167::task::495::TaskManager.Task::(_debug) Task 4a8210fd-ba56-42e3-b5e0-490928faacde: ref 0 aborting False
>Thread-15917157::INFO::2013-11-20 04:55:30,168::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:55:30,462::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0877227 s, 11.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:55:32,478::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:55:33,032::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0789551 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:55:35,046::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 04:55:35,269::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:55:35,274::task::495::TaskManager.Task::(_debug) Task d4d7ae13-1b6e-4321-a73c-6dc4e1f47315: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:55:35,284::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fe2cbbb9-0d95-4ea7-8911-9242463f8581`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:55:35,289::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:55:35,294::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:55:35,295::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`fe2cbbb9-0d95-4ea7-8911-9242463f8581`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:55:35,298::task::495::TaskManager.Task::(_debug) Task d4d7ae13-1b6e-4321-a73c-6dc4e1f47315: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:55:35,299::task::495::TaskManager.Task::(_debug) Task d4d7ae13-1b6e-4321-a73c-6dc4e1f47315: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:55:35,301::task::495::TaskManager.Task::(_debug) Task d4d7ae13-1b6e-4321-a73c-6dc4e1f47315: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:55:35,302::task::495::TaskManager.Task::(_debug) Task d4d7ae13-1b6e-4321-a73c-6dc4e1f47315: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:55:35,303::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:55:35,304::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:55:35,305::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:55:35,306::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:55:35,307::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:55:35,307::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:55:35,308::task::495::TaskManager.Task::(_debug) Task d4d7ae13-1b6e-4321-a73c-6dc4e1f47315: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:55:35,308::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:55:35,608::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0871558 s, 11.7 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 04:55:37,327::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:55:37,328::task::495::TaskManager.Task::(_debug) Task 290e0cbc-1993-4e93-bf6d-7fb18a19db29: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:55:37,330::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`50e01cac-26bf-4de4-a4d7-1a56a6acc428`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:55:37,331::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:55:37,332::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:55:37,333::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`50e01cac-26bf-4de4-a4d7-1a56a6acc428`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:55:37,334::task::495::TaskManager.Task::(_debug) Task 290e0cbc-1993-4e93-bf6d-7fb18a19db29: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:55:37,335::task::495::TaskManager.Task::(_debug) Task 290e0cbc-1993-4e93-bf6d-7fb18a19db29: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:55:37,336::task::495::TaskManager.Task::(_debug) Task 290e0cbc-1993-4e93-bf6d-7fb18a19db29: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:55:37,337::task::495::TaskManager.Task::(_debug) Task 290e0cbc-1993-4e93-bf6d-7fb18a19db29: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:55:37,338::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:55:37,338::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:55:37,339::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:55:37,341::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:55:37,341::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:55:37,342::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:55:37,343::task::495::TaskManager.Task::(_debug) Task 290e0cbc-1993-4e93-bf6d-7fb18a19db29: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:55:37,344::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:55:37,345::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:55:37,346::task::495::TaskManager.Task::(_debug) Task e512594d-e115-442d-a848-9ad914f910a2: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:55:37,348::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`21dcad46-adbc-4353-8150-ec3b7862a086`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:55:37,348::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:55:37,349::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:55:37,350::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`21dcad46-adbc-4353-8150-ec3b7862a086`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:55:37,352::task::495::TaskManager.Task::(_debug) Task e512594d-e115-442d-a848-9ad914f910a2: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:55:37,352::task::495::TaskManager.Task::(_debug) Task e512594d-e115-442d-a848-9ad914f910a2: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:55:37,353::task::495::TaskManager.Task::(_debug) Task e512594d-e115-442d-a848-9ad914f910a2: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:55:37,354::task::495::TaskManager.Task::(_debug) Task e512594d-e115-442d-a848-9ad914f910a2: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:55:37,355::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:55:37,356::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:55:37,357::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:55:37,358::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:55:37,359::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:55:37,360::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:55:37,361::task::495::TaskManager.Task::(_debug) Task e512594d-e115-442d-a848-9ad914f910a2: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:55:37,361::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:55:37,624::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:55:38,174::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0714117 s, 14.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:55:40,187::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15917161::DEBUG::2013-11-20 04:55:40,412::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917161::INFO::2013-11-20 04:55:40,415::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917161::DEBUG::2013-11-20 04:55:40,418::task::495::TaskManager.Task::(_debug) Task 134b8a25-596b-4180-a0ce-915423ddb12c: moving from state init -> state preparing
>Thread-15917161::DEBUG::2013-11-20 04:55:40,420::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917161::DEBUG::2013-11-20 04:55:40,420::task::495::TaskManager.Task::(_debug) Task 134b8a25-596b-4180-a0ce-915423ddb12c: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917161::DEBUG::2013-11-20 04:55:40,421::task::495::TaskManager.Task::(_debug) Task 134b8a25-596b-4180-a0ce-915423ddb12c: moving from state preparing -> state finished
>Thread-15917161::DEBUG::2013-11-20 04:55:40,422::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917161::DEBUG::2013-11-20 04:55:40,423::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917161::DEBUG::2013-11-20 04:55:40,425::task::495::TaskManager.Task::(_debug) Task 134b8a25-596b-4180-a0ce-915423ddb12c: ref 0 aborting False
>Thread-15917161::INFO::2013-11-20 04:55:40,427::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917162::DEBUG::2013-11-20 04:55:40,436::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917162::INFO::2013-11-20 04:55:40,438::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917162::DEBUG::2013-11-20 04:55:40,439::task::495::TaskManager.Task::(_debug) Task 75a79a93-a4b5-48cc-8b88-e9841d1af799: moving from state init -> state preparing
>Thread-15917162::DEBUG::2013-11-20 04:55:40,442::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`83437234-979f-4217-932a-a16c18370f3d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917162::DEBUG::2013-11-20 04:55:40,443::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917162::DEBUG::2013-11-20 04:55:40,446::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917162::DEBUG::2013-11-20 04:55:40,447::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`83437234-979f-4217-932a-a16c18370f3d`::Granted request
>Thread-15917162::DEBUG::2013-11-20 04:55:40,449::task::495::TaskManager.Task::(_debug) Task 75a79a93-a4b5-48cc-8b88-e9841d1af799: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917162::DEBUG::2013-11-20 04:55:40,450::task::495::TaskManager.Task::(_debug) Task 75a79a93-a4b5-48cc-8b88-e9841d1af799: ref 1 aborting False
>Thread-15917162::WARNING::2013-11-20 04:55:40,463::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917162::DEBUG::2013-11-20 04:55:40,469::task::495::TaskManager.Task::(_debug) Task 75a79a93-a4b5-48cc-8b88-e9841d1af799: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917162::DEBUG::2013-11-20 04:55:40,471::task::495::TaskManager.Task::(_debug) Task 75a79a93-a4b5-48cc-8b88-e9841d1af799: moving from state preparing -> state finished
>Thread-15917162::DEBUG::2013-11-20 04:55:40,472::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917162::DEBUG::2013-11-20 04:55:40,473::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917162::DEBUG::2013-11-20 04:55:40,474::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917162::DEBUG::2013-11-20 04:55:40,477::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917162::DEBUG::2013-11-20 04:55:40,479::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917162::DEBUG::2013-11-20 04:55:40,484::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917162::DEBUG::2013-11-20 04:55:40,486::task::495::TaskManager.Task::(_debug) Task 75a79a93-a4b5-48cc-8b88-e9841d1af799: ref 0 aborting False
>Thread-15917162::INFO::2013-11-20 04:55:40,486::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:55:40,677::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.052097 s, 19.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:55:42,688::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:55:43,224::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0796802 s, 12.9 MB/s\n'; <rc> = 0
>Thread-15917164::INFO::2013-11-20 04:55:44,068::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15917164::DEBUG::2013-11-20 04:55:44,069::task::495::TaskManager.Task::(_debug) Task b76c98b2-3701-4e41-a64b-143fff315f94: moving from state init -> state preparing
>Thread-15917164::DEBUG::2013-11-20 04:55:44,070::task::495::TaskManager.Task::(_debug) Task b76c98b2-3701-4e41-a64b-143fff315f94: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000444889068604', 'lastCheck': 1384923342.4080739, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00157499313354', 'lastCheck': 1384923342.918937, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00138282775879', 'lastCheck': 1384923342.919513, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000396013259888', 'lastCheck': 1384923335.6963329, 'valid': True, 'code': 0}}
>Thread-15917164::DEBUG::2013-11-20 04:55:44,071::task::495::TaskManager.Task::(_debug) Task b76c98b2-3701-4e41-a64b-143fff315f94: moving from state preparing -> state finished
>Thread-15917164::DEBUG::2013-11-20 04:55:44,071::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917164::DEBUG::2013-11-20 04:55:44,072::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917164::DEBUG::2013-11-20 04:55:44,072::task::495::TaskManager.Task::(_debug) Task b76c98b2-3701-4e41-a64b-143fff315f94: ref 0 aborting False
>Thread-15917164::INFO::2013-11-20 04:55:44,073::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000444889068604', 'lastCheck': 1384923342.4080739, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00157499313354', 'lastCheck': 1384923342.918937, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00138282775879', 'lastCheck': 1384923342.919513, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000396013259888', 'lastCheck': 1384923335.6963329, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:55:45,240::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 04:55:45,461::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:55:45,464::task::495::TaskManager.Task::(_debug) Task 6ddb74fa-5ee7-4601-a153-8cbef43000aa: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:55:45,469::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b68e21f3-71b7-4995-b2a3-dac3cd4e7c40`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:55:45,471::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:55:45,474::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:55:45,476::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b68e21f3-71b7-4995-b2a3-dac3cd4e7c40`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:55:45,477::task::495::TaskManager.Task::(_debug) Task 6ddb74fa-5ee7-4601-a153-8cbef43000aa: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:55:45,478::task::495::TaskManager.Task::(_debug) Task 6ddb74fa-5ee7-4601-a153-8cbef43000aa: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:55:45,479::task::495::TaskManager.Task::(_debug) Task 6ddb74fa-5ee7-4601-a153-8cbef43000aa: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:55:45,480::task::495::TaskManager.Task::(_debug) Task 6ddb74fa-5ee7-4601-a153-8cbef43000aa: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:55:45,481::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:55:45,482::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:55:45,483::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:55:45,484::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:55:45,486::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:55:45,487::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:55:45,488::task::495::TaskManager.Task::(_debug) Task 6ddb74fa-5ee7-4601-a153-8cbef43000aa: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:55:45,489::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:55:45,790::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0831906 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:55:47,805::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 04:55:48,005::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:55:48,007::task::495::TaskManager.Task::(_debug) Task b21b422d-4843-4c60-8c9d-38f78fe930cf: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:55:48,011::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1a3333f4-3207-4010-bb1a-d905cd632f4b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:55:48,012::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:55:48,013::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:55:48,013::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1a3333f4-3207-4010-bb1a-d905cd632f4b`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:55:48,015::task::495::TaskManager.Task::(_debug) Task b21b422d-4843-4c60-8c9d-38f78fe930cf: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:55:48,015::task::495::TaskManager.Task::(_debug) Task b21b422d-4843-4c60-8c9d-38f78fe930cf: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:55:48,016::task::495::TaskManager.Task::(_debug) Task b21b422d-4843-4c60-8c9d-38f78fe930cf: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:55:48,017::task::495::TaskManager.Task::(_debug) Task b21b422d-4843-4c60-8c9d-38f78fe930cf: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:55:48,017::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:55:48,018::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:55:48,019::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:55:48,020::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:55:48,020::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:55:48,021::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:55:48,022::task::495::TaskManager.Task::(_debug) Task b21b422d-4843-4c60-8c9d-38f78fe930cf: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:55:48,022::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:55:48,328::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0789919 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:55:50,343::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15917168::DEBUG::2013-11-20 04:55:50,592::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917168::INFO::2013-11-20 04:55:50,594::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917168::DEBUG::2013-11-20 04:55:50,595::task::495::TaskManager.Task::(_debug) Task 2587a0cb-add6-4da1-899e-6abbc908ee07: moving from state init -> state preparing
>Thread-15917168::DEBUG::2013-11-20 04:55:50,596::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917168::DEBUG::2013-11-20 04:55:50,596::task::495::TaskManager.Task::(_debug) Task 2587a0cb-add6-4da1-899e-6abbc908ee07: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917168::DEBUG::2013-11-20 04:55:50,597::task::495::TaskManager.Task::(_debug) Task 2587a0cb-add6-4da1-899e-6abbc908ee07: moving from state preparing -> state finished
>Thread-15917168::DEBUG::2013-11-20 04:55:50,598::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917168::DEBUG::2013-11-20 04:55:50,598::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917168::DEBUG::2013-11-20 04:55:50,599::task::495::TaskManager.Task::(_debug) Task 2587a0cb-add6-4da1-899e-6abbc908ee07: ref 0 aborting False
>Thread-15917168::INFO::2013-11-20 04:55:50,599::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917169::DEBUG::2013-11-20 04:55:50,604::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917169::INFO::2013-11-20 04:55:50,605::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917169::DEBUG::2013-11-20 04:55:50,605::task::495::TaskManager.Task::(_debug) Task 08b99359-f5fe-40f3-9d0f-bb3b93f2ae68: moving from state init -> state preparing
>Thread-15917169::DEBUG::2013-11-20 04:55:50,606::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bd459f63-8064-40cc-9a98-add9a33e7c26`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917169::DEBUG::2013-11-20 04:55:50,607::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917169::DEBUG::2013-11-20 04:55:50,608::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917169::DEBUG::2013-11-20 04:55:50,608::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`bd459f63-8064-40cc-9a98-add9a33e7c26`::Granted request
>Thread-15917169::DEBUG::2013-11-20 04:55:50,609::task::495::TaskManager.Task::(_debug) Task 08b99359-f5fe-40f3-9d0f-bb3b93f2ae68: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917169::DEBUG::2013-11-20 04:55:50,609::task::495::TaskManager.Task::(_debug) Task 08b99359-f5fe-40f3-9d0f-bb3b93f2ae68: ref 1 aborting False
>Thread-15917169::WARNING::2013-11-20 04:55:50,612::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917169::DEBUG::2013-11-20 04:55:50,613::task::495::TaskManager.Task::(_debug) Task 08b99359-f5fe-40f3-9d0f-bb3b93f2ae68: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917169::DEBUG::2013-11-20 04:55:50,614::task::495::TaskManager.Task::(_debug) Task 08b99359-f5fe-40f3-9d0f-bb3b93f2ae68: moving from state preparing -> state finished
>Thread-15917169::DEBUG::2013-11-20 04:55:50,614::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917169::DEBUG::2013-11-20 04:55:50,614::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917169::DEBUG::2013-11-20 04:55:50,615::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917169::DEBUG::2013-11-20 04:55:50,616::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917169::DEBUG::2013-11-20 04:55:50,616::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917169::DEBUG::2013-11-20 04:55:50,617::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917169::DEBUG::2013-11-20 04:55:50,618::task::495::TaskManager.Task::(_debug) Task 08b99359-f5fe-40f3-9d0f-bb3b93f2ae68: ref 0 aborting False
>Thread-15917169::INFO::2013-11-20 04:55:50,618::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:55:50,855::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0495116 s, 20.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:55:52,870::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:55:53,450::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0825155 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:55:55,465::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 04:55:55,658::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:55:55,662::task::495::TaskManager.Task::(_debug) Task 4ec1d6f3-90f8-4868-b230-dceba03ac3c6: moving from state init -> state preparing
>Thread-15914519::INFO::2013-11-20 04:55:55,663::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:55:55,666::task::495::TaskManager.Task::(_debug) Task d69f8dfe-ce85-474a-afa4-c274bc22d06c: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:55:55,667::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`ac7830b4-c8d4-420c-a3b3-4c74bb51ca23`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:55:55,670::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ef604eb3-a62c-4831-bb33-3d8d28bd3f99`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:55:55,670::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:55:55,671::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:55:55,673::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:55:55,674::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`ac7830b4-c8d4-420c-a3b3-4c74bb51ca23`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:55:55,676::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:55:55,677::task::495::TaskManager.Task::(_debug) Task 4ec1d6f3-90f8-4868-b230-dceba03ac3c6: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15914519::DEBUG::2013-11-20 04:55:55,678::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ef604eb3-a62c-4831-bb33-3d8d28bd3f99`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:55:55,679::task::495::TaskManager.Task::(_debug) Task 4ec1d6f3-90f8-4868-b230-dceba03ac3c6: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:55:55,681::task::495::TaskManager.Task::(_debug) Task 4ec1d6f3-90f8-4868-b230-dceba03ac3c6: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:55:55,682::task::495::TaskManager.Task::(_debug) Task d69f8dfe-ce85-474a-afa4-c274bc22d06c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913546::DEBUG::2013-11-20 04:55:55,682::task::495::TaskManager.Task::(_debug) Task 4ec1d6f3-90f8-4868-b230-dceba03ac3c6: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:55:55,683::task::495::TaskManager.Task::(_debug) Task d69f8dfe-ce85-474a-afa4-c274bc22d06c: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:55:55,684::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:55:55,685::task::495::TaskManager.Task::(_debug) Task d69f8dfe-ce85-474a-afa4-c274bc22d06c: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:55:55,686::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:55:55,686::task::495::TaskManager.Task::(_debug) Task d69f8dfe-ce85-474a-afa4-c274bc22d06c: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:55:55,688::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15914519::DEBUG::2013-11-20 04:55:55,688::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:55:55,690::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:55:55,690::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:55:55,691::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15914519::DEBUG::2013-11-20 04:55:55,692::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913546::DEBUG::2013-11-20 04:55:55,693::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:55:55,694::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:55:55,696::task::495::TaskManager.Task::(_debug) Task 4ec1d6f3-90f8-4868-b230-dceba03ac3c6: ref 0 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:55:55,696::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913546::INFO::2013-11-20 04:55:55,697::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:55:55,698::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913546::INFO::2013-11-20 04:55:55,699::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15914519::DEBUG::2013-11-20 04:55:55,700::task::495::TaskManager.Task::(_debug) Task d69f8dfe-ce85-474a-afa4-c274bc22d06c: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:55:55,701::task::495::TaskManager.Task::(_debug) Task 18c66f20-00af-4f3c-849d-4ad19160a5b6: moving from state init -> state preparing
>Thread-15914519::INFO::2013-11-20 04:55:55,701::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:55:55,703::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`1d37b5fa-f75e-4608-b5b7-3eaa31734bf1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:55:55,704::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:55:55,705::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:55:55,706::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`1d37b5fa-f75e-4608-b5b7-3eaa31734bf1`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:55:55,708::task::495::TaskManager.Task::(_debug) Task 18c66f20-00af-4f3c-849d-4ad19160a5b6: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:55:55,708::task::495::TaskManager.Task::(_debug) Task 18c66f20-00af-4f3c-849d-4ad19160a5b6: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:55:55,709::task::495::TaskManager.Task::(_debug) Task 18c66f20-00af-4f3c-849d-4ad19160a5b6: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:55:55,710::task::495::TaskManager.Task::(_debug) Task 18c66f20-00af-4f3c-849d-4ad19160a5b6: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:55:55,710::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:55:55,711::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:55:55,712::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:55:55,712::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:55:55,713::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:55:55,714::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:55:55,715::task::495::TaskManager.Task::(_debug) Task 18c66f20-00af-4f3c-849d-4ad19160a5b6: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:55:55,716::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:55:55,717::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:55:55,717::task::495::TaskManager.Task::(_debug) Task c6212eb7-4f8a-4c84-87ce-83b8ee38dc2f: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:55:55,718::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`710902b5-9880-4317-b16c-89292fcbb0eb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:55:55,719::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:55:55,719::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:55:55,720::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`710902b5-9880-4317-b16c-89292fcbb0eb`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:55:55,720::task::495::TaskManager.Task::(_debug) Task c6212eb7-4f8a-4c84-87ce-83b8ee38dc2f: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:55:55,721::task::495::TaskManager.Task::(_debug) Task c6212eb7-4f8a-4c84-87ce-83b8ee38dc2f: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:55:55,722::task::495::TaskManager.Task::(_debug) Task c6212eb7-4f8a-4c84-87ce-83b8ee38dc2f: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:55:55,722::task::495::TaskManager.Task::(_debug) Task c6212eb7-4f8a-4c84-87ce-83b8ee38dc2f: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:55:55,723::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:55:55,723::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:55:55,724::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:55:55,724::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:55:55,725::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:55:55,725::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:55:55,726::task::495::TaskManager.Task::(_debug) Task c6212eb7-4f8a-4c84-87ce-83b8ee38dc2f: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:55:55,727::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:55:55,728::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:55:55,729::task::495::TaskManager.Task::(_debug) Task 66b8e464-b67e-48b1-88b2-3f83119050ef: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:55:55,730::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`bac2480c-bf36-4f53-bb69-fc1f73559e40`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:55:55,730::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:55:55,731::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:55:55,735::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`bac2480c-bf36-4f53-bb69-fc1f73559e40`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:55:55,737::task::495::TaskManager.Task::(_debug) Task 66b8e464-b67e-48b1-88b2-3f83119050ef: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:55:55,737::task::495::TaskManager.Task::(_debug) Task 66b8e464-b67e-48b1-88b2-3f83119050ef: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:55:55,739::task::495::TaskManager.Task::(_debug) Task 66b8e464-b67e-48b1-88b2-3f83119050ef: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:55:55,739::task::495::TaskManager.Task::(_debug) Task 66b8e464-b67e-48b1-88b2-3f83119050ef: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:55:55,740::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:55:55,741::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:55:55,741::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:55:55,742::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:55:55,743::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:55:55,743::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:55:55,744::task::495::TaskManager.Task::(_debug) Task 66b8e464-b67e-48b1-88b2-3f83119050ef: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:55:55,745::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:55:55,984::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0911691 s, 11.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:55:58,000::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:55:58,575::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0840527 s, 12.2 MB/s\n'; <rc> = 0
>Thread-15917172::INFO::2013-11-20 04:55:59,640::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15917172::DEBUG::2013-11-20 04:55:59,641::task::495::TaskManager.Task::(_debug) Task 15b298db-829e-475f-a0a7-17986c15bb6c: moving from state init -> state preparing
>Thread-15917172::DEBUG::2013-11-20 04:55:59,642::task::495::TaskManager.Task::(_debug) Task 15b298db-829e-475f-a0a7-17986c15bb6c: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00032901763916', 'lastCheck': 1384923352.5589781, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00206422805786', 'lastCheck': 1384923353.159498, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00252795219421', 'lastCheck': 1384923353.1585619, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00056004524231', 'lastCheck': 1384923356.1172659, 'valid': True, 'code': 0}}
>Thread-15917172::DEBUG::2013-11-20 04:55:59,643::task::495::TaskManager.Task::(_debug) Task 15b298db-829e-475f-a0a7-17986c15bb6c: moving from state preparing -> state finished
>Thread-15917172::DEBUG::2013-11-20 04:55:59,644::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917172::DEBUG::2013-11-20 04:55:59,645::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917172::DEBUG::2013-11-20 04:55:59,645::task::495::TaskManager.Task::(_debug) Task 15b298db-829e-475f-a0a7-17986c15bb6c: ref 0 aborting False
>Thread-15917172::INFO::2013-11-20 04:55:59,646::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00032901763916', 'lastCheck': 1384923352.5589781, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00206422805786', 'lastCheck': 1384923353.159498, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00252795219421', 'lastCheck': 1384923353.1585619, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00056004524231', 'lastCheck': 1384923356.1172659, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:56:00,591::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15917174::DEBUG::2013-11-20 04:56:00,833::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917174::INFO::2013-11-20 04:56:00,837::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917174::DEBUG::2013-11-20 04:56:00,839::task::495::TaskManager.Task::(_debug) Task 4b06d990-0ddf-4f34-bd76-4dadbf2732a7: moving from state init -> state preparing
>Thread-15917174::DEBUG::2013-11-20 04:56:00,841::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917174::DEBUG::2013-11-20 04:56:00,842::task::495::TaskManager.Task::(_debug) Task 4b06d990-0ddf-4f34-bd76-4dadbf2732a7: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917174::DEBUG::2013-11-20 04:56:00,843::task::495::TaskManager.Task::(_debug) Task 4b06d990-0ddf-4f34-bd76-4dadbf2732a7: moving from state preparing -> state finished
>Thread-15917174::DEBUG::2013-11-20 04:56:00,844::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917174::DEBUG::2013-11-20 04:56:00,845::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917174::DEBUG::2013-11-20 04:56:00,846::task::495::TaskManager.Task::(_debug) Task 4b06d990-0ddf-4f34-bd76-4dadbf2732a7: ref 0 aborting False
>Thread-15917174::INFO::2013-11-20 04:56:00,847::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917175::DEBUG::2013-11-20 04:56:00,853::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917175::INFO::2013-11-20 04:56:00,854::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917175::DEBUG::2013-11-20 04:56:00,855::task::495::TaskManager.Task::(_debug) Task f094f42f-b4aa-40ea-8051-6f79580f8d00: moving from state init -> state preparing
>Thread-15917175::DEBUG::2013-11-20 04:56:00,856::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`cb76d1ad-800b-4d59-a468-303fa4965a80`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917175::DEBUG::2013-11-20 04:56:00,857::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917175::DEBUG::2013-11-20 04:56:00,858::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917175::DEBUG::2013-11-20 04:56:00,858::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`cb76d1ad-800b-4d59-a468-303fa4965a80`::Granted request
>Thread-15917175::DEBUG::2013-11-20 04:56:00,860::task::495::TaskManager.Task::(_debug) Task f094f42f-b4aa-40ea-8051-6f79580f8d00: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917175::DEBUG::2013-11-20 04:56:00,861::task::495::TaskManager.Task::(_debug) Task f094f42f-b4aa-40ea-8051-6f79580f8d00: ref 1 aborting False
>Thread-15917175::WARNING::2013-11-20 04:56:00,865::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917175::DEBUG::2013-11-20 04:56:00,866::task::495::TaskManager.Task::(_debug) Task f094f42f-b4aa-40ea-8051-6f79580f8d00: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917175::DEBUG::2013-11-20 04:56:00,867::task::495::TaskManager.Task::(_debug) Task f094f42f-b4aa-40ea-8051-6f79580f8d00: moving from state preparing -> state finished
>Thread-15917175::DEBUG::2013-11-20 04:56:00,868::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917175::DEBUG::2013-11-20 04:56:00,869::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917175::DEBUG::2013-11-20 04:56:00,870::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917175::DEBUG::2013-11-20 04:56:00,871::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917175::DEBUG::2013-11-20 04:56:00,871::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917175::DEBUG::2013-11-20 04:56:00,872::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917175::DEBUG::2013-11-20 04:56:00,873::task::495::TaskManager.Task::(_debug) Task f094f42f-b4aa-40ea-8051-6f79580f8d00: ref 0 aborting False
>Thread-15917175::INFO::2013-11-20 04:56:00,873::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:56:01,144::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0753744 s, 13.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:56:03,159::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:56:03,725::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.091301 s, 11.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:56:05,745::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 04:56:05,989::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:56:05,990::task::495::TaskManager.Task::(_debug) Task 96e02b78-d016-4de6-abcb-d6c99e74723a: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:56:05,991::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7516e5ea-bdd7-449f-af7c-6d70dd8c1659`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:56:05,992::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:56:05,992::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:56:05,993::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7516e5ea-bdd7-449f-af7c-6d70dd8c1659`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:56:05,994::task::495::TaskManager.Task::(_debug) Task 96e02b78-d016-4de6-abcb-d6c99e74723a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:56:05,994::task::495::TaskManager.Task::(_debug) Task 96e02b78-d016-4de6-abcb-d6c99e74723a: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:56:05,995::task::495::TaskManager.Task::(_debug) Task 96e02b78-d016-4de6-abcb-d6c99e74723a: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:56:05,995::task::495::TaskManager.Task::(_debug) Task 96e02b78-d016-4de6-abcb-d6c99e74723a: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:56:05,995::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:56:05,996::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:56:05,997::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:56:05,998::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:56:05,999::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:56:05,999::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:56:06,000::task::495::TaskManager.Task::(_debug) Task 96e02b78-d016-4de6-abcb-d6c99e74723a: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:56:06,001::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:56:06,320::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0830816 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:56:08,338::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:56:08,868::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0610117 s, 16.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:56:10,879::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15917179::DEBUG::2013-11-20 04:56:11,108::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917179::INFO::2013-11-20 04:56:11,111::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917179::DEBUG::2013-11-20 04:56:11,113::task::495::TaskManager.Task::(_debug) Task e2ec3eeb-be08-4217-885a-8868399b368d: moving from state init -> state preparing
>Thread-15917179::DEBUG::2013-11-20 04:56:11,115::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917179::DEBUG::2013-11-20 04:56:11,116::task::495::TaskManager.Task::(_debug) Task e2ec3eeb-be08-4217-885a-8868399b368d: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917179::DEBUG::2013-11-20 04:56:11,117::task::495::TaskManager.Task::(_debug) Task e2ec3eeb-be08-4217-885a-8868399b368d: moving from state preparing -> state finished
>Thread-15917179::DEBUG::2013-11-20 04:56:11,118::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917179::DEBUG::2013-11-20 04:56:11,119::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917179::DEBUG::2013-11-20 04:56:11,119::task::495::TaskManager.Task::(_debug) Task e2ec3eeb-be08-4217-885a-8868399b368d: ref 0 aborting False
>Thread-15917179::INFO::2013-11-20 04:56:11,120::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917180::DEBUG::2013-11-20 04:56:11,129::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917180::INFO::2013-11-20 04:56:11,134::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917180::DEBUG::2013-11-20 04:56:11,138::task::495::TaskManager.Task::(_debug) Task 4f64cf1f-ae94-4dc0-8d49-c31e6d57c331: moving from state init -> state preparing
>Thread-15917180::DEBUG::2013-11-20 04:56:11,144::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`00616a57-ddec-44be-9708-5679c23f7c1f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917180::DEBUG::2013-11-20 04:56:11,149::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917180::DEBUG::2013-11-20 04:56:11,163::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917180::DEBUG::2013-11-20 04:56:11,163::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`00616a57-ddec-44be-9708-5679c23f7c1f`::Granted request
>Thread-15917180::DEBUG::2013-11-20 04:56:11,165::task::495::TaskManager.Task::(_debug) Task 4f64cf1f-ae94-4dc0-8d49-c31e6d57c331: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917180::DEBUG::2013-11-20 04:56:11,165::task::495::TaskManager.Task::(_debug) Task 4f64cf1f-ae94-4dc0-8d49-c31e6d57c331: ref 1 aborting False
>Thread-15917180::WARNING::2013-11-20 04:56:11,168::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917180::DEBUG::2013-11-20 04:56:11,169::task::495::TaskManager.Task::(_debug) Task 4f64cf1f-ae94-4dc0-8d49-c31e6d57c331: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917180::DEBUG::2013-11-20 04:56:11,170::task::495::TaskManager.Task::(_debug) Task 4f64cf1f-ae94-4dc0-8d49-c31e6d57c331: moving from state preparing -> state finished
>Thread-15917180::DEBUG::2013-11-20 04:56:11,170::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917180::DEBUG::2013-11-20 04:56:11,171::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917180::DEBUG::2013-11-20 04:56:11,171::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917180::DEBUG::2013-11-20 04:56:11,172::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917180::DEBUG::2013-11-20 04:56:11,173::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917180::DEBUG::2013-11-20 04:56:11,173::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917180::DEBUG::2013-11-20 04:56:11,174::task::495::TaskManager.Task::(_debug) Task 4f64cf1f-ae94-4dc0-8d49-c31e6d57c331: ref 0 aborting False
>Thread-15917180::INFO::2013-11-20 04:56:11,174::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115248988160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:56:11,440::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0801789 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:56:13,454::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:56:13,977::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0751336 s, 13.6 MB/s\n'; <rc> = 0
>Thread-15917182::INFO::2013-11-20 04:56:15,215::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15917182::DEBUG::2013-11-20 04:56:15,216::task::495::TaskManager.Task::(_debug) Task 8f3afeae-3638-4282-b2bf-e62510aaa7de: moving from state init -> state preparing
>Thread-15917182::DEBUG::2013-11-20 04:56:15,217::task::495::TaskManager.Task::(_debug) Task 8f3afeae-3638-4282-b2bf-e62510aaa7de: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000502824783325', 'lastCheck': 1384923373.1016841, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00190806388855', 'lastCheck': 1384923373.7202909, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00247287750244', 'lastCheck': 1384923373.721879, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000447988510132', 'lastCheck': 1384923366.3695221, 'valid': True, 'code': 0}}
>Thread-15917182::DEBUG::2013-11-20 04:56:15,218::task::495::TaskManager.Task::(_debug) Task 8f3afeae-3638-4282-b2bf-e62510aaa7de: moving from state preparing -> state finished
>Thread-15917182::DEBUG::2013-11-20 04:56:15,219::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917182::DEBUG::2013-11-20 04:56:15,220::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917182::DEBUG::2013-11-20 04:56:15,220::task::495::TaskManager.Task::(_debug) Task 8f3afeae-3638-4282-b2bf-e62510aaa7de: ref 0 aborting False
>Thread-15917182::INFO::2013-11-20 04:56:15,221::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000502824783325', 'lastCheck': 1384923373.1016841, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00190806388855', 'lastCheck': 1384923373.7202909, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00247287750244', 'lastCheck': 1384923373.721879, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000447988510132', 'lastCheck': 1384923366.3695221, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:56:15,993::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:56:16,202::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:56:16,204::task::495::TaskManager.Task::(_debug) Task 0895b23b-d32d-4a0f-ac22-284e0815c10e: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:56:16,210::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3ef73ec3-71e5-4f21-b613-de9ae7072437`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:56:16,211::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:56:16,214::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:56:16,215::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3ef73ec3-71e5-4f21-b613-de9ae7072437`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:56:16,220::task::495::TaskManager.Task::(_debug) Task 0895b23b-d32d-4a0f-ac22-284e0815c10e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:56:16,224::task::495::TaskManager.Task::(_debug) Task 0895b23b-d32d-4a0f-ac22-284e0815c10e: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:56:16,226::task::495::TaskManager.Task::(_debug) Task 0895b23b-d32d-4a0f-ac22-284e0815c10e: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:56:16,227::task::495::TaskManager.Task::(_debug) Task 0895b23b-d32d-4a0f-ac22-284e0815c10e: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:56:16,228::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:56:16,231::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:56:16,234::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:56:16,238::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:56:16,240::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:56:16,241::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:56:16,242::task::495::TaskManager.Task::(_debug) Task 0895b23b-d32d-4a0f-ac22-284e0815c10e: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:56:16,243::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:56:16,568::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0882458 s, 11.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:56:18,594::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915918::INFO::2013-11-20 04:56:18,824::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:56:18,826::task::495::TaskManager.Task::(_debug) Task f7689fad-49b5-4c25-95be-187e1889fa0c: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:56:18,829::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0631babb-8e99-4eb4-81f4-55661f392691`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:56:18,831::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:56:18,832::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:56:18,833::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0631babb-8e99-4eb4-81f4-55661f392691`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:56:18,834::task::495::TaskManager.Task::(_debug) Task f7689fad-49b5-4c25-95be-187e1889fa0c: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:56:18,835::task::495::TaskManager.Task::(_debug) Task f7689fad-49b5-4c25-95be-187e1889fa0c: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:56:18,836::task::495::TaskManager.Task::(_debug) Task f7689fad-49b5-4c25-95be-187e1889fa0c: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:56:18,838::task::495::TaskManager.Task::(_debug) Task f7689fad-49b5-4c25-95be-187e1889fa0c: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:56:18,838::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:56:18,839::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:56:18,841::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:56:18,842::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:56:18,843::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:56:18,844::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:56:18,845::task::495::TaskManager.Task::(_debug) Task f7689fad-49b5-4c25-95be-187e1889fa0c: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:56:18,846::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:56:19,194::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0898942 s, 11.4 MB/s\n'; <rc> = 0
>Thread-15916984::INFO::2013-11-20 04:56:21,209::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=240c0593-fe33-4f8b-a029-097379dbcb85 volUUID=f715faaa-081a-4598-b125-80a2d12af554)
>Dummy-15912651::DEBUG::2013-11-20 04:56:21,211::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916984::DEBUG::2013-11-20 04:56:21,216::task::495::TaskManager.Task::(_debug) Task f8cc9ec1-ada3-4de2-b41d-914c0bb1c3e3: moving from state init -> state preparing
>Thread-15916984::DEBUG::2013-11-20 04:56:21,442::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6faa6557-a979-4b98-8942-3873bc4a7892`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15916984::DEBUG::2013-11-20 04:56:21,444::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15916984::DEBUG::2013-11-20 04:56:21,445::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15916984::DEBUG::2013-11-20 04:56:21,450::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6faa6557-a979-4b98-8942-3873bc4a7892`::Granted request
>Thread-15916984::DEBUG::2013-11-20 04:56:21,451::task::495::TaskManager.Task::(_debug) Task f8cc9ec1-ada3-4de2-b41d-914c0bb1c3e3: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15916984::DEBUG::2013-11-20 04:56:21,452::task::495::TaskManager.Task::(_debug) Task f8cc9ec1-ada3-4de2-b41d-914c0bb1c3e3: ref 1 aborting False
>Thread-15916984::DEBUG::2013-11-20 04:56:21,455::task::495::TaskManager.Task::(_debug) Task f8cc9ec1-ada3-4de2-b41d-914c0bb1c3e3: finished: {'truesize': '75296145408', 'apparentsize': '75296145408'}
>Thread-15916984::DEBUG::2013-11-20 04:56:21,455::task::495::TaskManager.Task::(_debug) Task f8cc9ec1-ada3-4de2-b41d-914c0bb1c3e3: moving from state preparing -> state finished
>Thread-15916984::DEBUG::2013-11-20 04:56:21,461::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15916984::DEBUG::2013-11-20 04:56:21,462::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916984::DEBUG::2013-11-20 04:56:21,467::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15916984::DEBUG::2013-11-20 04:56:21,470::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15916984::DEBUG::2013-11-20 04:56:21,470::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15916984::DEBUG::2013-11-20 04:56:21,470::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15916984::DEBUG::2013-11-20 04:56:21,471::task::495::TaskManager.Task::(_debug) Task f8cc9ec1-ada3-4de2-b41d-914c0bb1c3e3: ref 0 aborting False
>Thread-15916984::INFO::2013-11-20 04:56:21,471::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '75296145408', 'apparentsize': '75296145408'}
>Thread-15917185::DEBUG::2013-11-20 04:56:21,494::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917185::INFO::2013-11-20 04:56:21,496::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917185::DEBUG::2013-11-20 04:56:21,497::task::495::TaskManager.Task::(_debug) Task 03da56b7-6d7c-4eca-93da-0b3abb877444: moving from state init -> state preparing
>Thread-15917185::DEBUG::2013-11-20 04:56:21,498::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917185::DEBUG::2013-11-20 04:56:21,498::task::495::TaskManager.Task::(_debug) Task 03da56b7-6d7c-4eca-93da-0b3abb877444: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917185::DEBUG::2013-11-20 04:56:21,499::task::495::TaskManager.Task::(_debug) Task 03da56b7-6d7c-4eca-93da-0b3abb877444: moving from state preparing -> state finished
>Thread-15917185::DEBUG::2013-11-20 04:56:21,500::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917185::DEBUG::2013-11-20 04:56:21,500::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917185::DEBUG::2013-11-20 04:56:21,501::task::495::TaskManager.Task::(_debug) Task 03da56b7-6d7c-4eca-93da-0b3abb877444: ref 0 aborting False
>Thread-15917185::INFO::2013-11-20 04:56:21,502::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917186::DEBUG::2013-11-20 04:56:21,510::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917186::INFO::2013-11-20 04:56:21,511::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917186::DEBUG::2013-11-20 04:56:21,512::task::495::TaskManager.Task::(_debug) Task d661bce3-5f4e-497e-82b3-9bc8951e3ef4: moving from state init -> state preparing
>Thread-15917186::DEBUG::2013-11-20 04:56:21,513::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`80b98a80-23df-4054-83c5-aa90a5d702a4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917186::DEBUG::2013-11-20 04:56:21,514::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917186::DEBUG::2013-11-20 04:56:21,514::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917186::DEBUG::2013-11-20 04:56:21,515::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`80b98a80-23df-4054-83c5-aa90a5d702a4`::Granted request
>Thread-15917186::DEBUG::2013-11-20 04:56:21,516::task::495::TaskManager.Task::(_debug) Task d661bce3-5f4e-497e-82b3-9bc8951e3ef4: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917186::DEBUG::2013-11-20 04:56:21,517::task::495::TaskManager.Task::(_debug) Task d661bce3-5f4e-497e-82b3-9bc8951e3ef4: ref 1 aborting False
>Thread-15917186::WARNING::2013-11-20 04:56:21,519::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917186::DEBUG::2013-11-20 04:56:21,521::task::495::TaskManager.Task::(_debug) Task d661bce3-5f4e-497e-82b3-9bc8951e3ef4: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917186::DEBUG::2013-11-20 04:56:21,522::task::495::TaskManager.Task::(_debug) Task d661bce3-5f4e-497e-82b3-9bc8951e3ef4: moving from state preparing -> state finished
>Thread-15917186::DEBUG::2013-11-20 04:56:21,522::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917186::DEBUG::2013-11-20 04:56:21,523::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917186::DEBUG::2013-11-20 04:56:21,524::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917186::DEBUG::2013-11-20 04:56:21,524::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917186::DEBUG::2013-11-20 04:56:21,525::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917186::DEBUG::2013-11-20 04:56:21,526::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917186::DEBUG::2013-11-20 04:56:21,526::task::495::TaskManager.Task::(_debug) Task d661bce3-5f4e-497e-82b3-9bc8951e3ef4: ref 0 aborting False
>Thread-15917186::INFO::2013-11-20 04:56:21,527::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115246891008', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:56:21,779::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0748766 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:56:23,795::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915863::INFO::2013-11-20 04:56:24,000::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:56:24,006::task::495::TaskManager.Task::(_debug) Task f7325d20-a6e6-4a63-8173-68615fbcb3a0: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:56:24,012::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5244de59-701d-48a5-bab7-33351ea2d2bd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:56:24,013::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:56:24,016::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:56:24,016::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5244de59-701d-48a5-bab7-33351ea2d2bd`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:56:24,018::task::495::TaskManager.Task::(_debug) Task f7325d20-a6e6-4a63-8173-68615fbcb3a0: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:56:24,018::task::495::TaskManager.Task::(_debug) Task f7325d20-a6e6-4a63-8173-68615fbcb3a0: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:56:24,019::task::495::TaskManager.Task::(_debug) Task f7325d20-a6e6-4a63-8173-68615fbcb3a0: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:56:24,020::task::495::TaskManager.Task::(_debug) Task f7325d20-a6e6-4a63-8173-68615fbcb3a0: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:56:24,021::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:56:24,021::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:56:24,022::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:56:24,023::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:56:24,023::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:56:24,024::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:56:24,024::task::495::TaskManager.Task::(_debug) Task f7325d20-a6e6-4a63-8173-68615fbcb3a0: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:56:24,025::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:56:24,341::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0888388 s, 11.5 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 04:56:26,212::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:56:26,213::task::495::TaskManager.Task::(_debug) Task 8ea9c46d-cdcc-4072-97e6-f6f4caf1560f: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:56:26,215::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8b41a3a4-9493-4594-9e01-a8583d0406df`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:56:26,216::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:56:26,217::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:56:26,218::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8b41a3a4-9493-4594-9e01-a8583d0406df`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:56:26,231::task::495::TaskManager.Task::(_debug) Task 8ea9c46d-cdcc-4072-97e6-f6f4caf1560f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:56:26,233::task::495::TaskManager.Task::(_debug) Task 8ea9c46d-cdcc-4072-97e6-f6f4caf1560f: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:56:26,234::task::495::TaskManager.Task::(_debug) Task 8ea9c46d-cdcc-4072-97e6-f6f4caf1560f: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:56:26,234::task::495::TaskManager.Task::(_debug) Task 8ea9c46d-cdcc-4072-97e6-f6f4caf1560f: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:56:26,235::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:56:26,236::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:56:26,236::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:56:26,237::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:56:26,238::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:56:26,238::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:56:26,239::task::495::TaskManager.Task::(_debug) Task 8ea9c46d-cdcc-4072-97e6-f6f4caf1560f: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:56:26,240::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:56:26,356::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:56:26,895::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0668656 s, 15.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:56:28,909::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:56:29,425::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0790131 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15917190::INFO::2013-11-20 04:56:31,158::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15917190::DEBUG::2013-11-20 04:56:31,159::task::495::TaskManager.Task::(_debug) Task 49b85490-aeee-43fd-a2b0-742e659a2ca3: moving from state init -> state preparing
>Thread-15917190::DEBUG::2013-11-20 04:56:31,160::task::495::TaskManager.Task::(_debug) Task 49b85490-aeee-43fd-a2b0-742e659a2ca3: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00106906890869', 'lastCheck': 1384923383.2043109, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00231099128723', 'lastCheck': 1384923384.0153019, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00192594528198', 'lastCheck': 1384923384.016433, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000521183013916', 'lastCheck': 1384923386.9974301, 'valid': True, 'code': 0}}
>Thread-15917190::DEBUG::2013-11-20 04:56:31,161::task::495::TaskManager.Task::(_debug) Task 49b85490-aeee-43fd-a2b0-742e659a2ca3: moving from state preparing -> state finished
>Thread-15917190::DEBUG::2013-11-20 04:56:31,161::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917190::DEBUG::2013-11-20 04:56:31,162::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917190::DEBUG::2013-11-20 04:56:31,163::task::495::TaskManager.Task::(_debug) Task 49b85490-aeee-43fd-a2b0-742e659a2ca3: ref 0 aborting False
>Thread-15917190::INFO::2013-11-20 04:56:31,163::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00106906890869', 'lastCheck': 1384923383.2043109, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00231099128723', 'lastCheck': 1384923384.0153019, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00192594528198', 'lastCheck': 1384923384.016433, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000521183013916', 'lastCheck': 1384923386.9974301, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:56:31,435::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15917192::DEBUG::2013-11-20 04:56:31,806::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917192::INFO::2013-11-20 04:56:31,808::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917192::DEBUG::2013-11-20 04:56:31,809::task::495::TaskManager.Task::(_debug) Task 35edd175-5af0-4e06-a9a3-1a51b3bc5a17: moving from state init -> state preparing
>Thread-15917192::DEBUG::2013-11-20 04:56:31,810::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917192::DEBUG::2013-11-20 04:56:31,810::task::495::TaskManager.Task::(_debug) Task 35edd175-5af0-4e06-a9a3-1a51b3bc5a17: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917192::DEBUG::2013-11-20 04:56:31,811::task::495::TaskManager.Task::(_debug) Task 35edd175-5af0-4e06-a9a3-1a51b3bc5a17: moving from state preparing -> state finished
>Thread-15917192::DEBUG::2013-11-20 04:56:31,812::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917192::DEBUG::2013-11-20 04:56:31,812::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917192::DEBUG::2013-11-20 04:56:31,813::task::495::TaskManager.Task::(_debug) Task 35edd175-5af0-4e06-a9a3-1a51b3bc5a17: ref 0 aborting False
>Thread-15917192::INFO::2013-11-20 04:56:31,814::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917193::DEBUG::2013-11-20 04:56:31,820::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917193::INFO::2013-11-20 04:56:31,821::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917193::DEBUG::2013-11-20 04:56:31,821::task::495::TaskManager.Task::(_debug) Task e2f016ac-a413-4af1-8450-951cac3e06a7: moving from state init -> state preparing
>Thread-15917193::DEBUG::2013-11-20 04:56:31,823::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ab9f7b78-aed1-457e-b38e-fcf7bd5df1cf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917193::DEBUG::2013-11-20 04:56:31,824::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917193::DEBUG::2013-11-20 04:56:31,825::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917193::DEBUG::2013-11-20 04:56:31,826::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`ab9f7b78-aed1-457e-b38e-fcf7bd5df1cf`::Granted request
>Thread-15917193::DEBUG::2013-11-20 04:56:31,827::task::495::TaskManager.Task::(_debug) Task e2f016ac-a413-4af1-8450-951cac3e06a7: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917193::DEBUG::2013-11-20 04:56:31,828::task::495::TaskManager.Task::(_debug) Task e2f016ac-a413-4af1-8450-951cac3e06a7: ref 1 aborting False
>Thread-15917193::WARNING::2013-11-20 04:56:31,830::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917193::DEBUG::2013-11-20 04:56:31,831::task::495::TaskManager.Task::(_debug) Task e2f016ac-a413-4af1-8450-951cac3e06a7: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115117916160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115117916160', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917193::DEBUG::2013-11-20 04:56:31,832::task::495::TaskManager.Task::(_debug) Task e2f016ac-a413-4af1-8450-951cac3e06a7: moving from state preparing -> state finished
>Thread-15917193::DEBUG::2013-11-20 04:56:31,833::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917193::DEBUG::2013-11-20 04:56:31,833::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917193::DEBUG::2013-11-20 04:56:31,834::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917193::DEBUG::2013-11-20 04:56:31,835::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917193::DEBUG::2013-11-20 04:56:31,835::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917193::DEBUG::2013-11-20 04:56:31,836::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917193::DEBUG::2013-11-20 04:56:31,837::task::495::TaskManager.Task::(_debug) Task e2f016ac-a413-4af1-8450-951cac3e06a7: ref 0 aborting False
>Thread-15917193::INFO::2013-11-20 04:56:31,838::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115117916160', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115117916160', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:56:32,018::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0863257 s, 11.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:56:34,032::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:56:34,559::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0777199 s, 13.2 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 04:56:36,241::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:56:36,242::task::495::TaskManager.Task::(_debug) Task 60d7b31e-517c-4d9a-8af3-b7f67e364d7d: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:56:36,243::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`35d620a7-bc22-4045-9307-76fc246235fa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:56:36,244::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:56:36,245::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:56:36,245::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`35d620a7-bc22-4045-9307-76fc246235fa`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:56:36,246::task::495::TaskManager.Task::(_debug) Task 60d7b31e-517c-4d9a-8af3-b7f67e364d7d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:56:36,246::task::495::TaskManager.Task::(_debug) Task 60d7b31e-517c-4d9a-8af3-b7f67e364d7d: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:56:36,247::task::495::TaskManager.Task::(_debug) Task 60d7b31e-517c-4d9a-8af3-b7f67e364d7d: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:56:36,247::task::495::TaskManager.Task::(_debug) Task 60d7b31e-517c-4d9a-8af3-b7f67e364d7d: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:56:36,248::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:56:36,248::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:56:36,248::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:56:36,249::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:56:36,249::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:56:36,249::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:56:36,250::task::495::TaskManager.Task::(_debug) Task 60d7b31e-517c-4d9a-8af3-b7f67e364d7d: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:56:36,250::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:56:36,572::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:56:37,103::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0680502 s, 15.0 MB/s\n'; <rc> = 0
>Thread-42840::DEBUG::2013-11-20 04:56:37,222::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 772472cb-2429-466d-96cd-00b3301ef78a' (cwd None)
>Thread-42840::DEBUG::2013-11-20 04:56:37,905::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15913518::INFO::2013-11-20 04:56:38,854::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:56:38,855::task::495::TaskManager.Task::(_debug) Task f397f7a3-605f-4259-ad50-64167ceaf379: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:56:38,861::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`63e77a4d-8c0e-4881-9025-1eda136da9ec`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:56:38,862::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:56:38,863::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:56:38,863::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`63e77a4d-8c0e-4881-9025-1eda136da9ec`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:56:38,865::task::495::TaskManager.Task::(_debug) Task f397f7a3-605f-4259-ad50-64167ceaf379: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:56:38,865::task::495::TaskManager.Task::(_debug) Task f397f7a3-605f-4259-ad50-64167ceaf379: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:56:38,865::task::495::TaskManager.Task::(_debug) Task f397f7a3-605f-4259-ad50-64167ceaf379: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:56:38,866::task::495::TaskManager.Task::(_debug) Task f397f7a3-605f-4259-ad50-64167ceaf379: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:56:38,866::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:56:38,867::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:56:38,867::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:56:38,867::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:56:38,868::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:56:38,868::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:56:38,869::task::495::TaskManager.Task::(_debug) Task f397f7a3-605f-4259-ad50-64167ceaf379: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:56:38,869::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:56:38,869::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:56:38,870::task::495::TaskManager.Task::(_debug) Task 8c38177f-b265-4b6e-9daf-a7b346c81102: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:56:38,870::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`337875ec-bbde-4852-a621-501e46ce62b8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:56:38,871::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:56:38,871::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:56:38,872::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`337875ec-bbde-4852-a621-501e46ce62b8`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:56:38,872::task::495::TaskManager.Task::(_debug) Task 8c38177f-b265-4b6e-9daf-a7b346c81102: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:56:38,873::task::495::TaskManager.Task::(_debug) Task 8c38177f-b265-4b6e-9daf-a7b346c81102: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:56:38,873::task::495::TaskManager.Task::(_debug) Task 8c38177f-b265-4b6e-9daf-a7b346c81102: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:56:38,873::task::495::TaskManager.Task::(_debug) Task 8c38177f-b265-4b6e-9daf-a7b346c81102: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:56:38,874::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:56:38,874::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:56:38,874::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:56:38,875::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:56:38,875::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:56:38,876::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:56:38,876::task::495::TaskManager.Task::(_debug) Task 8c38177f-b265-4b6e-9daf-a7b346c81102: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:56:38,876::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:56:39,114::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:56:39,681::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.071508 s, 14.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:56:41,695::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15917197::DEBUG::2013-11-20 04:56:42,094::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917197::INFO::2013-11-20 04:56:42,096::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917197::DEBUG::2013-11-20 04:56:42,097::task::495::TaskManager.Task::(_debug) Task a9573fce-3e89-492c-8490-475177d6f848: moving from state init -> state preparing
>Thread-15917197::DEBUG::2013-11-20 04:56:42,098::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917197::DEBUG::2013-11-20 04:56:42,099::task::495::TaskManager.Task::(_debug) Task a9573fce-3e89-492c-8490-475177d6f848: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917197::DEBUG::2013-11-20 04:56:42,099::task::495::TaskManager.Task::(_debug) Task a9573fce-3e89-492c-8490-475177d6f848: moving from state preparing -> state finished
>Thread-15917197::DEBUG::2013-11-20 04:56:42,100::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917197::DEBUG::2013-11-20 04:56:42,100::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917197::DEBUG::2013-11-20 04:56:42,101::task::495::TaskManager.Task::(_debug) Task a9573fce-3e89-492c-8490-475177d6f848: ref 0 aborting False
>Thread-15917197::INFO::2013-11-20 04:56:42,102::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917198::DEBUG::2013-11-20 04:56:42,106::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917198::INFO::2013-11-20 04:56:42,107::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917198::DEBUG::2013-11-20 04:56:42,108::task::495::TaskManager.Task::(_debug) Task a1c2f3ed-df07-443b-be90-3b5c048cda09: moving from state init -> state preparing
>Thread-15917198::DEBUG::2013-11-20 04:56:42,109::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c2e7983c-f866-4b59-81a5-49ffbe82eafb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917198::DEBUG::2013-11-20 04:56:42,110::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917198::DEBUG::2013-11-20 04:56:42,111::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917198::DEBUG::2013-11-20 04:56:42,111::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`c2e7983c-f866-4b59-81a5-49ffbe82eafb`::Granted request
>Thread-15917198::DEBUG::2013-11-20 04:56:42,112::task::495::TaskManager.Task::(_debug) Task a1c2f3ed-df07-443b-be90-3b5c048cda09: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917198::DEBUG::2013-11-20 04:56:42,113::task::495::TaskManager.Task::(_debug) Task a1c2f3ed-df07-443b-be90-3b5c048cda09: ref 1 aborting False
>Thread-15917198::WARNING::2013-11-20 04:56:42,115::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917198::DEBUG::2013-11-20 04:56:42,116::task::495::TaskManager.Task::(_debug) Task a1c2f3ed-df07-443b-be90-3b5c048cda09: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115101138944', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115101138944', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917198::DEBUG::2013-11-20 04:56:42,117::task::495::TaskManager.Task::(_debug) Task a1c2f3ed-df07-443b-be90-3b5c048cda09: moving from state preparing -> state finished
>Thread-15917198::DEBUG::2013-11-20 04:56:42,117::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917198::DEBUG::2013-11-20 04:56:42,118::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917198::DEBUG::2013-11-20 04:56:42,119::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917198::DEBUG::2013-11-20 04:56:42,119::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917198::DEBUG::2013-11-20 04:56:42,120::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917198::DEBUG::2013-11-20 04:56:42,120::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917198::DEBUG::2013-11-20 04:56:42,121::task::495::TaskManager.Task::(_debug) Task a1c2f3ed-df07-443b-be90-3b5c048cda09: ref 0 aborting False
>Thread-15917198::INFO::2013-11-20 04:56:42,122::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115101138944', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115101138944', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:56:42,257::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.074007 s, 13.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:56:44,273::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:56:44,816::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0747553 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15914453::INFO::2013-11-20 04:56:46,255::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:56:46,257::task::495::TaskManager.Task::(_debug) Task fdc71121-bfbc-4a7f-84b4-bb967eaa3c21: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:56:46,259::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`29e523bb-0c7f-4735-94ee-b9f2aabe8db8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:56:46,260::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:56:46,261::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:56:46,262::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`29e523bb-0c7f-4735-94ee-b9f2aabe8db8`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:56:46,263::task::495::TaskManager.Task::(_debug) Task fdc71121-bfbc-4a7f-84b4-bb967eaa3c21: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:56:46,264::task::495::TaskManager.Task::(_debug) Task fdc71121-bfbc-4a7f-84b4-bb967eaa3c21: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:56:46,264::task::495::TaskManager.Task::(_debug) Task fdc71121-bfbc-4a7f-84b4-bb967eaa3c21: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:56:46,265::task::495::TaskManager.Task::(_debug) Task fdc71121-bfbc-4a7f-84b4-bb967eaa3c21: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:56:46,266::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:56:46,266::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:56:46,267::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:56:46,267::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:56:46,268::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:56:46,269::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:56:46,269::task::495::TaskManager.Task::(_debug) Task fdc71121-bfbc-4a7f-84b4-bb967eaa3c21: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:56:46,270::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:56:46,830::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15917200::INFO::2013-11-20 04:56:47,072::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15917200::DEBUG::2013-11-20 04:56:47,073::task::495::TaskManager.Task::(_debug) Task 801f19d3-0e3c-4786-a7be-07391a462b16: moving from state init -> state preparing
>Thread-15917200::DEBUG::2013-11-20 04:56:47,074::task::495::TaskManager.Task::(_debug) Task 801f19d3-0e3c-4786-a7be-07391a462b16: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00204014778137', 'lastCheck': 1384923403.4398179, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0015389919281', 'lastCheck': 1384923404.5167029, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00182199478149', 'lastCheck': 1384923404.5157499, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00034499168396', 'lastCheck': 1384923397.9149621, 'valid': True, 'code': 0}}
>Thread-15917200::DEBUG::2013-11-20 04:56:47,075::task::495::TaskManager.Task::(_debug) Task 801f19d3-0e3c-4786-a7be-07391a462b16: moving from state preparing -> state finished
>Thread-15917200::DEBUG::2013-11-20 04:56:47,076::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917200::DEBUG::2013-11-20 04:56:47,076::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917200::DEBUG::2013-11-20 04:56:47,077::task::495::TaskManager.Task::(_debug) Task 801f19d3-0e3c-4786-a7be-07391a462b16: ref 0 aborting False
>Thread-15917200::INFO::2013-11-20 04:56:47,078::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.00204014778137', 'lastCheck': 1384923403.4398179, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0015389919281', 'lastCheck': 1384923404.5167029, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00182199478149', 'lastCheck': 1384923404.5157499, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00034499168396', 'lastCheck': 1384923397.9149621, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:56:47,355::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0768699 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:56:49,369::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 04:56:49,606::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:56:49,608::task::495::TaskManager.Task::(_debug) Task 9c09052e-500a-4b91-9414-2537c33af3ab: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:56:49,611::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b7e34c06-5ab6-4301-97c1-dc1345ed15b2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:56:49,612::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:56:49,614::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:56:49,614::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b7e34c06-5ab6-4301-97c1-dc1345ed15b2`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:56:49,616::task::495::TaskManager.Task::(_debug) Task 9c09052e-500a-4b91-9414-2537c33af3ab: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:56:49,616::task::495::TaskManager.Task::(_debug) Task 9c09052e-500a-4b91-9414-2537c33af3ab: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:56:49,617::task::495::TaskManager.Task::(_debug) Task 9c09052e-500a-4b91-9414-2537c33af3ab: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:56:49,618::task::495::TaskManager.Task::(_debug) Task 9c09052e-500a-4b91-9414-2537c33af3ab: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:56:49,619::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:56:49,619::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:56:49,620::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:56:49,621::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:56:49,621::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:56:49,622::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:56:49,623::task::495::TaskManager.Task::(_debug) Task 9c09052e-500a-4b91-9414-2537c33af3ab: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:56:49,623::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:56:49,904::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.079028 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:56:51,922::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:56:52,454::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0789308 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15917202::DEBUG::2013-11-20 04:56:52,560::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917202::INFO::2013-11-20 04:56:52,562::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917202::DEBUG::2013-11-20 04:56:52,563::task::495::TaskManager.Task::(_debug) Task 8ae53388-b79c-4e7a-b5f7-3522a1e4858e: moving from state init -> state preparing
>Thread-15917202::DEBUG::2013-11-20 04:56:52,564::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917202::DEBUG::2013-11-20 04:56:52,564::task::495::TaskManager.Task::(_debug) Task 8ae53388-b79c-4e7a-b5f7-3522a1e4858e: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917202::DEBUG::2013-11-20 04:56:52,565::task::495::TaskManager.Task::(_debug) Task 8ae53388-b79c-4e7a-b5f7-3522a1e4858e: moving from state preparing -> state finished
>Thread-15917202::DEBUG::2013-11-20 04:56:52,566::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917202::DEBUG::2013-11-20 04:56:52,566::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917202::DEBUG::2013-11-20 04:56:52,567::task::495::TaskManager.Task::(_debug) Task 8ae53388-b79c-4e7a-b5f7-3522a1e4858e: ref 0 aborting False
>Thread-15917202::INFO::2013-11-20 04:56:52,567::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917203::DEBUG::2013-11-20 04:56:52,573::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917203::INFO::2013-11-20 04:56:52,574::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917203::DEBUG::2013-11-20 04:56:52,575::task::495::TaskManager.Task::(_debug) Task cdd3ed5e-af73-458a-ab6a-a2a2fa15dfe5: moving from state init -> state preparing
>Thread-15917203::DEBUG::2013-11-20 04:56:52,576::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3446fa45-8236-4c6e-ae38-f19358e17bd4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917203::DEBUG::2013-11-20 04:56:52,577::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917203::DEBUG::2013-11-20 04:56:52,578::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917203::DEBUG::2013-11-20 04:56:52,580::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`3446fa45-8236-4c6e-ae38-f19358e17bd4`::Granted request
>Thread-15917203::DEBUG::2013-11-20 04:56:52,582::task::495::TaskManager.Task::(_debug) Task cdd3ed5e-af73-458a-ab6a-a2a2fa15dfe5: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917203::DEBUG::2013-11-20 04:56:52,583::task::495::TaskManager.Task::(_debug) Task cdd3ed5e-af73-458a-ab6a-a2a2fa15dfe5: ref 1 aborting False
>Thread-15917203::WARNING::2013-11-20 04:56:52,586::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917203::DEBUG::2013-11-20 04:56:52,587::task::495::TaskManager.Task::(_debug) Task cdd3ed5e-af73-458a-ab6a-a2a2fa15dfe5: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '114495062016', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '114495062016', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917203::DEBUG::2013-11-20 04:56:52,588::task::495::TaskManager.Task::(_debug) Task cdd3ed5e-af73-458a-ab6a-a2a2fa15dfe5: moving from state preparing -> state finished
>Thread-15917203::DEBUG::2013-11-20 04:56:52,589::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917203::DEBUG::2013-11-20 04:56:52,590::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917203::DEBUG::2013-11-20 04:56:52,591::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917203::DEBUG::2013-11-20 04:56:52,592::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917203::DEBUG::2013-11-20 04:56:52,593::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917203::DEBUG::2013-11-20 04:56:52,594::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917203::DEBUG::2013-11-20 04:56:52,595::task::495::TaskManager.Task::(_debug) Task cdd3ed5e-af73-458a-ab6a-a2a2fa15dfe5: ref 0 aborting False
>Thread-15917203::INFO::2013-11-20 04:56:52,597::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '114495062016', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '114495062016', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:56:54,469::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:56:55,032::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0839985 s, 12.2 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 04:56:56,266::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:56:56,267::task::495::TaskManager.Task::(_debug) Task 97736cf8-5c89-4308-9b9b-7989aae72163: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:56:56,269::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`552d24ee-8e51-470c-a1c5-8bf0217b037a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:56:56,269::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15914519::INFO::2013-11-20 04:56:56,271::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15913546::DEBUG::2013-11-20 04:56:56,272::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15914519::DEBUG::2013-11-20 04:56:56,273::task::495::TaskManager.Task::(_debug) Task 886f3ab0-f617-4203-9381-129208c36245: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:56:56,273::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`552d24ee-8e51-470c-a1c5-8bf0217b037a`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:56:56,275::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`79c4ea1c-0817-4473-8012-f6dde4185d33`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:56:56,276::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:56:56,277::task::495::TaskManager.Task::(_debug) Task 97736cf8-5c89-4308-9b9b-7989aae72163: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15914519::DEBUG::2013-11-20 04:56:56,278::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:56:56,279::task::495::TaskManager.Task::(_debug) Task 97736cf8-5c89-4308-9b9b-7989aae72163: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:56:56,279::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`79c4ea1c-0817-4473-8012-f6dde4185d33`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:56:56,280::task::495::TaskManager.Task::(_debug) Task 97736cf8-5c89-4308-9b9b-7989aae72163: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:56:56,288::task::495::TaskManager.Task::(_debug) Task 886f3ab0-f617-4203-9381-129208c36245: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913546::DEBUG::2013-11-20 04:56:56,289::task::495::TaskManager.Task::(_debug) Task 97736cf8-5c89-4308-9b9b-7989aae72163: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:56:56,290::task::495::TaskManager.Task::(_debug) Task 886f3ab0-f617-4203-9381-129208c36245: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:56:56,290::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:56:56,291::task::495::TaskManager.Task::(_debug) Task 886f3ab0-f617-4203-9381-129208c36245: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:56:56,292::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:56:56,293::task::495::TaskManager.Task::(_debug) Task 886f3ab0-f617-4203-9381-129208c36245: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:56:56,294::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15914519::DEBUG::2013-11-20 04:56:56,295::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:56:56,296::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:56:56,296::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:56:56,298::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:56:56,299::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:56:56,300::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913546::DEBUG::2013-11-20 04:56:56,301::task::495::TaskManager.Task::(_debug) Task 97736cf8-5c89-4308-9b9b-7989aae72163: ref 0 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:56:56,302::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913546::INFO::2013-11-20 04:56:56,303::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:56:56,303::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913546::INFO::2013-11-20 04:56:56,305::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15914519::DEBUG::2013-11-20 04:56:56,306::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:56:56,306::task::495::TaskManager.Task::(_debug) Task 3b02764a-00aa-4769-aac4-ce7e90cfd680: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:56:56,308::task::495::TaskManager.Task::(_debug) Task 886f3ab0-f617-4203-9381-129208c36245: ref 0 aborting False
>Thread-15914519::INFO::2013-11-20 04:56:56,310::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:56:56,311::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c8d1a803-063d-4c1b-921e-5f94be529d3a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:56:56,313::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:56:56,314::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:56:56,315::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`c8d1a803-063d-4c1b-921e-5f94be529d3a`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:56:56,316::task::495::TaskManager.Task::(_debug) Task 3b02764a-00aa-4769-aac4-ce7e90cfd680: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:56:56,317::task::495::TaskManager.Task::(_debug) Task 3b02764a-00aa-4769-aac4-ce7e90cfd680: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:56:56,317::task::495::TaskManager.Task::(_debug) Task 3b02764a-00aa-4769-aac4-ce7e90cfd680: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:56:56,318::task::495::TaskManager.Task::(_debug) Task 3b02764a-00aa-4769-aac4-ce7e90cfd680: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:56:56,319::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:56:56,320::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:56:56,320::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:56:56,321::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:56:56,322::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:56:56,323::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:56:56,324::task::495::TaskManager.Task::(_debug) Task 3b02764a-00aa-4769-aac4-ce7e90cfd680: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:56:56,324::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:56:56,325::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:56:56,326::task::495::TaskManager.Task::(_debug) Task e0034c57-edc0-4a4f-a9f2-78d9ca0bd891: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:56:56,327::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`749f4bdd-f215-4be5-bd13-26755cef9009`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:56:56,328::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:56:56,328::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:56:56,329::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`749f4bdd-f215-4be5-bd13-26755cef9009`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:56:56,330::task::495::TaskManager.Task::(_debug) Task e0034c57-edc0-4a4f-a9f2-78d9ca0bd891: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:56:56,330::task::495::TaskManager.Task::(_debug) Task e0034c57-edc0-4a4f-a9f2-78d9ca0bd891: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:56:56,331::task::495::TaskManager.Task::(_debug) Task e0034c57-edc0-4a4f-a9f2-78d9ca0bd891: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:56:56,332::task::495::TaskManager.Task::(_debug) Task e0034c57-edc0-4a4f-a9f2-78d9ca0bd891: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:56:56,332::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:56:56,333::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:56:56,334::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:56:56,335::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:56:56,335::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:56:56,336::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:56:56,337::task::495::TaskManager.Task::(_debug) Task e0034c57-edc0-4a4f-a9f2-78d9ca0bd891: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:56:56,337::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:56:56,338::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:56:56,339::task::495::TaskManager.Task::(_debug) Task d0bddc19-b816-44ea-bebd-25ec76a91b04: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:56:56,340::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a6d67729-7fcd-4e4a-a453-e3b0f45277ff`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:56:56,341::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:56:56,342::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:56:56,343::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a6d67729-7fcd-4e4a-a453-e3b0f45277ff`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:56:56,344::task::495::TaskManager.Task::(_debug) Task d0bddc19-b816-44ea-bebd-25ec76a91b04: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:56:56,344::task::495::TaskManager.Task::(_debug) Task d0bddc19-b816-44ea-bebd-25ec76a91b04: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:56:56,345::task::495::TaskManager.Task::(_debug) Task d0bddc19-b816-44ea-bebd-25ec76a91b04: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:56:56,346::task::495::TaskManager.Task::(_debug) Task d0bddc19-b816-44ea-bebd-25ec76a91b04: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:56:56,346::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:56:56,347::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:56:56,348::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:56:56,348::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:56:56,349::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:56:56,349::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:56:56,350::task::495::TaskManager.Task::(_debug) Task d0bddc19-b816-44ea-bebd-25ec76a91b04: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:56:56,351::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:56:57,047::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:56:57,583::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.07094 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:56:59,602::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:57:00,183::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0800008 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:57:02,200::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:57:02,778::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0791996 s, 12.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:57:04,796::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:57:05,344::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0801626 s, 12.8 MB/s\n'; <rc> = 0
>Thread-15917208::INFO::2013-11-20 04:57:06,815::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15917208::DEBUG::2013-11-20 04:57:06,817::task::495::TaskManager.Task::(_debug) Task 4c65809c-cbe7-46c1-a39d-32f14856db29: moving from state init -> state preparing
>Thread-15917208::DEBUG::2013-11-20 04:57:06,818::task::495::TaskManager.Task::(_debug) Task 4c65809c-cbe7-46c1-a39d-32f14856db29: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000447034835815', 'lastCheck': 1384923423.8386919, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148510932922', 'lastCheck': 1384923425.0287189, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00188994407654', 'lastCheck': 1384923425.0279191, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00042986869812', 'lastCheck': 1384923418.2302151, 'valid': True, 'code': 0}}
>Thread-15917208::DEBUG::2013-11-20 04:57:06,819::task::495::TaskManager.Task::(_debug) Task 4c65809c-cbe7-46c1-a39d-32f14856db29: moving from state preparing -> state finished
>Thread-15917208::DEBUG::2013-11-20 04:57:06,820::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917208::DEBUG::2013-11-20 04:57:06,820::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917208::DEBUG::2013-11-20 04:57:06,821::task::495::TaskManager.Task::(_debug) Task 4c65809c-cbe7-46c1-a39d-32f14856db29: ref 0 aborting False
>Thread-15917208::INFO::2013-11-20 04:57:06,822::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000447034835815', 'lastCheck': 1384923423.8386919, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00148510932922', 'lastCheck': 1384923425.0287189, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00188994407654', 'lastCheck': 1384923425.0279191, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00042986869812', 'lastCheck': 1384923418.2302151, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:57:07,361::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15917210::DEBUG::2013-11-20 04:57:07,567::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917210::INFO::2013-11-20 04:57:07,569::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917210::DEBUG::2013-11-20 04:57:07,570::task::495::TaskManager.Task::(_debug) Task c9c0af41-d17b-4414-bd3c-0f8ac9135e16: moving from state init -> state preparing
>Thread-15917210::DEBUG::2013-11-20 04:57:07,571::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917210::DEBUG::2013-11-20 04:57:07,572::task::495::TaskManager.Task::(_debug) Task c9c0af41-d17b-4414-bd3c-0f8ac9135e16: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914435::INFO::2013-11-20 04:57:07,573::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15917210::DEBUG::2013-11-20 04:57:07,573::task::495::TaskManager.Task::(_debug) Task c9c0af41-d17b-4414-bd3c-0f8ac9135e16: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:57:07,574::task::495::TaskManager.Task::(_debug) Task e282f4eb-4c2d-4cdc-a029-2ab6e5e483e7: moving from state init -> state preparing
>Thread-15917210::DEBUG::2013-11-20 04:57:07,575::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15914435::DEBUG::2013-11-20 04:57:07,576::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6c17c0aa-c9a4-4b9b-b573-268cfcb18fac`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15917210::DEBUG::2013-11-20 04:57:07,577::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:57:07,577::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15917210::DEBUG::2013-11-20 04:57:07,578::task::495::TaskManager.Task::(_debug) Task c9c0af41-d17b-4414-bd3c-0f8ac9135e16: ref 0 aborting False
>Thread-15917210::INFO::2013-11-20 04:57:07,580::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15914435::DEBUG::2013-11-20 04:57:07,580::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:57:07,586::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6c17c0aa-c9a4-4b9b-b573-268cfcb18fac`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:57:07,592::task::495::TaskManager.Task::(_debug) Task e282f4eb-4c2d-4cdc-a029-2ab6e5e483e7: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15917211::DEBUG::2013-11-20 04:57:07,595::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15914435::DEBUG::2013-11-20 04:57:07,595::task::495::TaskManager.Task::(_debug) Task e282f4eb-4c2d-4cdc-a029-2ab6e5e483e7: ref 1 aborting False
>Thread-15917211::INFO::2013-11-20 04:57:07,597::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15914435::DEBUG::2013-11-20 04:57:07,599::task::495::TaskManager.Task::(_debug) Task e282f4eb-4c2d-4cdc-a029-2ab6e5e483e7: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15917211::DEBUG::2013-11-20 04:57:07,600::task::495::TaskManager.Task::(_debug) Task 7285c5d7-b821-4be0-9b97-8e088fec0114: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:57:07,602::task::495::TaskManager.Task::(_debug) Task e282f4eb-4c2d-4cdc-a029-2ab6e5e483e7: moving from state preparing -> state finished
>Thread-15917211::DEBUG::2013-11-20 04:57:07,604::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`558aa06a-1fbb-4a1b-a3d1-c358d9d0ceec`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15914435::DEBUG::2013-11-20 04:57:07,604::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15917211::DEBUG::2013-11-20 04:57:07,611::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:57:07,612::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917211::DEBUG::2013-11-20 04:57:07,612::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:57:07,613::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15917211::DEBUG::2013-11-20 04:57:07,614::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`558aa06a-1fbb-4a1b-a3d1-c358d9d0ceec`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:57:07,615::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15917211::DEBUG::2013-11-20 04:57:07,616::task::495::TaskManager.Task::(_debug) Task 7285c5d7-b821-4be0-9b97-8e088fec0114: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:57:07,616::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15917211::DEBUG::2013-11-20 04:57:07,617::task::495::TaskManager.Task::(_debug) Task 7285c5d7-b821-4be0-9b97-8e088fec0114: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:57:07,617::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15917211::WARNING::2013-11-20 04:57:07,620::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15914435::DEBUG::2013-11-20 04:57:07,621::task::495::TaskManager.Task::(_debug) Task e282f4eb-4c2d-4cdc-a029-2ab6e5e483e7: ref 0 aborting False
>Thread-15917211::DEBUG::2013-11-20 04:57:07,622::task::495::TaskManager.Task::(_debug) Task 7285c5d7-b821-4be0-9b97-8e088fec0114: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '114499256320', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '114499256320', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15914435::INFO::2013-11-20 04:57:07,623::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15917211::DEBUG::2013-11-20 04:57:07,623::task::495::TaskManager.Task::(_debug) Task 7285c5d7-b821-4be0-9b97-8e088fec0114: moving from state preparing -> state finished
>Thread-15917211::DEBUG::2013-11-20 04:57:07,624::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917211::DEBUG::2013-11-20 04:57:07,625::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917211::DEBUG::2013-11-20 04:57:07,626::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917211::DEBUG::2013-11-20 04:57:07,627::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917211::DEBUG::2013-11-20 04:57:07,627::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917211::DEBUG::2013-11-20 04:57:07,628::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917211::DEBUG::2013-11-20 04:57:07,628::task::495::TaskManager.Task::(_debug) Task 7285c5d7-b821-4be0-9b97-8e088fec0114: ref 0 aborting False
>Thread-15917211::INFO::2013-11-20 04:57:07,629::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '114499256320', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '114499256320', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:57:07,896::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0797908 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:57:09,911::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:57:10,442::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0720614 s, 14.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:57:12,456::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:57:12,978::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0708788 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:57:14,993::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:57:15,534::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0803406 s, 12.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:57:17,549::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:57:17,761::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:57:17,763::task::495::TaskManager.Task::(_debug) Task 0a0cfd5d-7f58-4954-a4bc-df6b050235f6: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:57:17,765::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e4da9966-7c29-44da-be4a-b4b7c3ab4de7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:57:17,766::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:57:17,767::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:57:17,768::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`e4da9966-7c29-44da-be4a-b4b7c3ab4de7`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:57:17,768::task::495::TaskManager.Task::(_debug) Task 0a0cfd5d-7f58-4954-a4bc-df6b050235f6: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:57:17,769::task::495::TaskManager.Task::(_debug) Task 0a0cfd5d-7f58-4954-a4bc-df6b050235f6: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:57:17,770::task::495::TaskManager.Task::(_debug) Task 0a0cfd5d-7f58-4954-a4bc-df6b050235f6: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:57:17,770::task::495::TaskManager.Task::(_debug) Task 0a0cfd5d-7f58-4954-a4bc-df6b050235f6: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:57:17,770::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:57:17,771::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:57:17,771::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:57:17,772::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:57:17,772::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:57:17,772::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:57:17,773::task::495::TaskManager.Task::(_debug) Task 0a0cfd5d-7f58-4954-a4bc-df6b050235f6: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:57:17,773::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:57:18,079::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0737367 s, 13.9 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:57:20,094::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915918::INFO::2013-11-20 04:57:20,361::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:57:20,376::task::495::TaskManager.Task::(_debug) Task 4f7f1c89-25a4-4705-be03-71a8e14b4671: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:57:20,389::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a67ed21a-3e47-4373-8e37-646027cb7635`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:57:20,391::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:57:20,392::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:57:20,392::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a67ed21a-3e47-4373-8e37-646027cb7635`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:57:20,393::task::495::TaskManager.Task::(_debug) Task 4f7f1c89-25a4-4705-be03-71a8e14b4671: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:57:20,394::task::495::TaskManager.Task::(_debug) Task 4f7f1c89-25a4-4705-be03-71a8e14b4671: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:57:20,394::task::495::TaskManager.Task::(_debug) Task 4f7f1c89-25a4-4705-be03-71a8e14b4671: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:57:20,395::task::495::TaskManager.Task::(_debug) Task 4f7f1c89-25a4-4705-be03-71a8e14b4671: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:57:20,395::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:57:20,395::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:57:20,396::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:57:20,396::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:57:20,397::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:57:20,397::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:57:20,397::task::495::TaskManager.Task::(_debug) Task 4f7f1c89-25a4-4705-be03-71a8e14b4671: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:57:20,398::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:57:20,709::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.083532 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:57:22,724::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916984::INFO::2013-11-20 04:57:22,958::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=240c0593-fe33-4f8b-a029-097379dbcb85 volUUID=f715faaa-081a-4598-b125-80a2d12af554)
>Thread-15916984::DEBUG::2013-11-20 04:57:22,967::task::495::TaskManager.Task::(_debug) Task ec29843e-1724-4590-b280-3763ce3e1d90: moving from state init -> state preparing
>Thread-15916984::DEBUG::2013-11-20 04:57:22,971::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`697cdd1d-36bd-48c3-9108-66f778c652c0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15916984::DEBUG::2013-11-20 04:57:22,984::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15916984::DEBUG::2013-11-20 04:57:22,986::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15916984::DEBUG::2013-11-20 04:57:22,986::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`697cdd1d-36bd-48c3-9108-66f778c652c0`::Granted request
>Thread-15916984::DEBUG::2013-11-20 04:57:22,988::task::495::TaskManager.Task::(_debug) Task ec29843e-1724-4590-b280-3763ce3e1d90: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15916984::DEBUG::2013-11-20 04:57:22,988::task::495::TaskManager.Task::(_debug) Task ec29843e-1724-4590-b280-3763ce3e1d90: ref 1 aborting False
>Thread-15916984::DEBUG::2013-11-20 04:57:22,989::task::495::TaskManager.Task::(_debug) Task ec29843e-1724-4590-b280-3763ce3e1d90: finished: {'truesize': '75296145408', 'apparentsize': '75296145408'}
>Thread-15916984::DEBUG::2013-11-20 04:57:22,990::task::495::TaskManager.Task::(_debug) Task ec29843e-1724-4590-b280-3763ce3e1d90: moving from state preparing -> state finished
>Thread-15916984::DEBUG::2013-11-20 04:57:22,991::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15916984::DEBUG::2013-11-20 04:57:22,994::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916984::DEBUG::2013-11-20 04:57:22,995::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15916984::DEBUG::2013-11-20 04:57:22,996::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15916984::DEBUG::2013-11-20 04:57:22,999::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15916984::DEBUG::2013-11-20 04:57:22,999::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15916984::DEBUG::2013-11-20 04:57:23,000::task::495::TaskManager.Task::(_debug) Task ec29843e-1724-4590-b280-3763ce3e1d90: ref 0 aborting False
>Thread-15916984::INFO::2013-11-20 04:57:23,001::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '75296145408', 'apparentsize': '75296145408'}
>Dummy-15912651::DEBUG::2013-11-20 04:57:23,300::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0864581 s, 11.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:57:25,313::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915863::INFO::2013-11-20 04:57:25,519::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:57:25,530::task::495::TaskManager.Task::(_debug) Task 5820a976-dbd3-4a44-a9e3-4779ac21722d: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:57:25,535::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`db9d5e32-6801-4f1f-9cb4-5a36ca251004`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:57:25,538::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:57:25,540::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915863::DEBUG::2013-11-20 04:57:25,540::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`db9d5e32-6801-4f1f-9cb4-5a36ca251004`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:57:25,542::task::495::TaskManager.Task::(_debug) Task 5820a976-dbd3-4a44-a9e3-4779ac21722d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:57:25,543::task::495::TaskManager.Task::(_debug) Task 5820a976-dbd3-4a44-a9e3-4779ac21722d: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:57:25,544::task::495::TaskManager.Task::(_debug) Task 5820a976-dbd3-4a44-a9e3-4779ac21722d: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:57:25,544::task::495::TaskManager.Task::(_debug) Task 5820a976-dbd3-4a44-a9e3-4779ac21722d: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:57:25,545::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:57:25,546::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:57:25,546::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:57:25,547::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:57:25,548::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:57:25,549::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:57:25,549::task::495::TaskManager.Task::(_debug) Task 5820a976-dbd3-4a44-a9e3-4779ac21722d: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:57:25,550::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:57:25,857::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.100226 s, 10.2 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 04:57:26,954::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:57:26,955::task::495::TaskManager.Task::(_debug) Task 78eb0384-43a1-42f4-ad1a-0ee651de684a: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:57:26,957::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`50fd2175-46d3-4f72-b7d7-0baea035ee4e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:57:26,957::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:57:26,959::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:57:26,959::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`50fd2175-46d3-4f72-b7d7-0baea035ee4e`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:57:26,960::task::495::TaskManager.Task::(_debug) Task 78eb0384-43a1-42f4-ad1a-0ee651de684a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:57:26,961::task::495::TaskManager.Task::(_debug) Task 78eb0384-43a1-42f4-ad1a-0ee651de684a: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:57:26,961::task::495::TaskManager.Task::(_debug) Task 78eb0384-43a1-42f4-ad1a-0ee651de684a: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:57:26,962::task::495::TaskManager.Task::(_debug) Task 78eb0384-43a1-42f4-ad1a-0ee651de684a: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:57:26,963::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:57:26,964::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:57:26,964::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:57:26,965::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:57:26,965::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:57:26,966::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:57:26,967::task::495::TaskManager.Task::(_debug) Task 78eb0384-43a1-42f4-ad1a-0ee651de684a: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:57:26,967::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:57:27,877::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:57:28,415::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0760733 s, 13.5 MB/s\n'; <rc> = 0
>Thread-15917214::DEBUG::2013-11-20 04:57:29,348::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917214::INFO::2013-11-20 04:57:29,350::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917214::DEBUG::2013-11-20 04:57:29,351::task::495::TaskManager.Task::(_debug) Task 47ecf135-c5fa-46aa-ace1-f3eb2a4b0dbe: moving from state init -> state preparing
>Thread-15917214::DEBUG::2013-11-20 04:57:29,352::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917214::DEBUG::2013-11-20 04:57:29,352::task::495::TaskManager.Task::(_debug) Task 47ecf135-c5fa-46aa-ace1-f3eb2a4b0dbe: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917214::DEBUG::2013-11-20 04:57:29,353::task::495::TaskManager.Task::(_debug) Task 47ecf135-c5fa-46aa-ace1-f3eb2a4b0dbe: moving from state preparing -> state finished
>Thread-15917214::DEBUG::2013-11-20 04:57:29,354::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917214::DEBUG::2013-11-20 04:57:29,354::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917214::DEBUG::2013-11-20 04:57:29,355::task::495::TaskManager.Task::(_debug) Task 47ecf135-c5fa-46aa-ace1-f3eb2a4b0dbe: ref 0 aborting False
>Thread-15917214::INFO::2013-11-20 04:57:29,356::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917215::DEBUG::2013-11-20 04:57:29,362::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917215::INFO::2013-11-20 04:57:29,363::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917215::DEBUG::2013-11-20 04:57:29,364::task::495::TaskManager.Task::(_debug) Task 80ebbfa0-f804-480d-9808-af96bbb5529e: moving from state init -> state preparing
>Thread-15917215::DEBUG::2013-11-20 04:57:29,366::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`849066fc-2d75-414f-88f7-a402a00597b4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917215::DEBUG::2013-11-20 04:57:29,367::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917215::DEBUG::2013-11-20 04:57:29,370::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917215::DEBUG::2013-11-20 04:57:29,371::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`849066fc-2d75-414f-88f7-a402a00597b4`::Granted request
>Thread-15917215::DEBUG::2013-11-20 04:57:29,398::task::495::TaskManager.Task::(_debug) Task 80ebbfa0-f804-480d-9808-af96bbb5529e: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917215::DEBUG::2013-11-20 04:57:29,399::task::495::TaskManager.Task::(_debug) Task 80ebbfa0-f804-480d-9808-af96bbb5529e: ref 1 aborting False
>Thread-15917215::WARNING::2013-11-20 04:57:29,401::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917215::DEBUG::2013-11-20 04:57:29,402::task::495::TaskManager.Task::(_debug) Task 80ebbfa0-f804-480d-9808-af96bbb5529e: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '114499256320', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '114499256320', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917215::DEBUG::2013-11-20 04:57:29,403::task::495::TaskManager.Task::(_debug) Task 80ebbfa0-f804-480d-9808-af96bbb5529e: moving from state preparing -> state finished
>Thread-15917215::DEBUG::2013-11-20 04:57:29,404::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917215::DEBUG::2013-11-20 04:57:29,404::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917215::DEBUG::2013-11-20 04:57:29,405::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917215::DEBUG::2013-11-20 04:57:29,406::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917215::DEBUG::2013-11-20 04:57:29,406::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917215::DEBUG::2013-11-20 04:57:29,407::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917215::DEBUG::2013-11-20 04:57:29,408::task::495::TaskManager.Task::(_debug) Task 80ebbfa0-f804-480d-9808-af96bbb5529e: ref 0 aborting False
>Thread-15917215::INFO::2013-11-20 04:57:29,408::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '114499256320', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '114499256320', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:57:30,429::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:57:30,952::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0853539 s, 12.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:57:32,969::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:57:33,450::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0677218 s, 15.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:57:35,465::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:57:36,015::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0764626 s, 13.4 MB/s\n'; <rc> = 0
>Thread-15912392::INFO::2013-11-20 04:57:36,971::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:57:36,972::task::495::TaskManager.Task::(_debug) Task ec871c8c-3531-4559-84f3-a235d5b89b27: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:57:36,974::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`30219b39-5a5b-465c-8b9d-d9a110cfa387`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:57:36,974::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:57:36,975::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:57:36,976::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`30219b39-5a5b-465c-8b9d-d9a110cfa387`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:57:36,977::task::495::TaskManager.Task::(_debug) Task ec871c8c-3531-4559-84f3-a235d5b89b27: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:57:36,977::task::495::TaskManager.Task::(_debug) Task ec871c8c-3531-4559-84f3-a235d5b89b27: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:57:36,978::task::495::TaskManager.Task::(_debug) Task ec871c8c-3531-4559-84f3-a235d5b89b27: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:57:36,979::task::495::TaskManager.Task::(_debug) Task ec871c8c-3531-4559-84f3-a235d5b89b27: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:57:36,979::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:57:36,980::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:57:36,981::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:57:36,982::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:57:36,982::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:57:36,983::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:57:36,983::task::495::TaskManager.Task::(_debug) Task ec871c8c-3531-4559-84f3-a235d5b89b27: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:57:36,984::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15917218::INFO::2013-11-20 04:57:37,657::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15917218::DEBUG::2013-11-20 04:57:37,658::task::495::TaskManager.Task::(_debug) Task 698172b7-3db1-45d3-9f65-56042db74523: moving from state init -> state preparing
>Thread-15917218::DEBUG::2013-11-20 04:57:37,659::task::495::TaskManager.Task::(_debug) Task 698172b7-3db1-45d3-9f65-56042db74523: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000879049301147', 'lastCheck': 1384923454.6220491, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014591217041', 'lastCheck': 1384923455.6991141, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00319790840149', 'lastCheck': 1384923455.698509, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000509023666382', 'lastCheck': 1384923449.0833659, 'valid': True, 'code': 0}}
>Thread-15917218::DEBUG::2013-11-20 04:57:37,660::task::495::TaskManager.Task::(_debug) Task 698172b7-3db1-45d3-9f65-56042db74523: moving from state preparing -> state finished
>Thread-15917218::DEBUG::2013-11-20 04:57:37,661::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917218::DEBUG::2013-11-20 04:57:37,662::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917218::DEBUG::2013-11-20 04:57:37,662::task::495::TaskManager.Task::(_debug) Task 698172b7-3db1-45d3-9f65-56042db74523: ref 0 aborting False
>Thread-15917218::INFO::2013-11-20 04:57:37,663::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000879049301147', 'lastCheck': 1384923454.6220491, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.0014591217041', 'lastCheck': 1384923455.6991141, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00319790840149', 'lastCheck': 1384923455.698509, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000509023666382', 'lastCheck': 1384923449.0833659, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:57:38,031::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:57:38,598::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0805344 s, 12.7 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 04:57:40,306::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:57:40,307::task::495::TaskManager.Task::(_debug) Task 75523c21-e715-427a-a6ff-efe64de27f0e: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:57:40,309::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8e380e2e-4a0c-40f6-a146-81ee1b547044`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:57:40,310::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:57:40,311::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:57:40,311::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`8e380e2e-4a0c-40f6-a146-81ee1b547044`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:57:40,313::task::495::TaskManager.Task::(_debug) Task 75523c21-e715-427a-a6ff-efe64de27f0e: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:57:40,313::task::495::TaskManager.Task::(_debug) Task 75523c21-e715-427a-a6ff-efe64de27f0e: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:57:40,314::task::495::TaskManager.Task::(_debug) Task 75523c21-e715-427a-a6ff-efe64de27f0e: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:57:40,315::task::495::TaskManager.Task::(_debug) Task 75523c21-e715-427a-a6ff-efe64de27f0e: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:57:40,315::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:57:40,316::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:57:40,317::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:57:40,318::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:57:40,318::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:57:40,319::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:57:40,320::task::495::TaskManager.Task::(_debug) Task 75523c21-e715-427a-a6ff-efe64de27f0e: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:57:40,320::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:57:40,321::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:57:40,322::task::495::TaskManager.Task::(_debug) Task 194bb771-0fec-44e1-a20c-5ad643923f70: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:57:40,323::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7b147a9f-961f-4148-8eff-92ab8e00c576`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:57:40,323::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:57:40,324::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:57:40,325::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7b147a9f-961f-4148-8eff-92ab8e00c576`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:57:40,326::task::495::TaskManager.Task::(_debug) Task 194bb771-0fec-44e1-a20c-5ad643923f70: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:57:40,326::task::495::TaskManager.Task::(_debug) Task 194bb771-0fec-44e1-a20c-5ad643923f70: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:57:40,327::task::495::TaskManager.Task::(_debug) Task 194bb771-0fec-44e1-a20c-5ad643923f70: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:57:40,328::task::495::TaskManager.Task::(_debug) Task 194bb771-0fec-44e1-a20c-5ad643923f70: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:57:40,329::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:57:40,329::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:57:40,330::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:57:40,331::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:57:40,331::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:57:40,332::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:57:40,333::task::495::TaskManager.Task::(_debug) Task 194bb771-0fec-44e1-a20c-5ad643923f70: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:57:40,334::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:57:40,614::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:57:41,184::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0843035 s, 12.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:57:43,198::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:57:43,679::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0817645 s, 12.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:57:45,698::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:57:46,290::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0879136 s, 11.6 MB/s\n'; <rc> = 0
>Thread-15914453::INFO::2013-11-20 04:57:46,859::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:57:46,861::task::495::TaskManager.Task::(_debug) Task 6766ab42-7e33-4296-bbfa-36d32ede4a8a: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:57:46,874::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c472fbda-20ac-4e2f-8681-9ecb2e1f29a7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:57:46,881::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:57:46,883::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:57:46,883::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`c472fbda-20ac-4e2f-8681-9ecb2e1f29a7`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:57:46,885::task::495::TaskManager.Task::(_debug) Task 6766ab42-7e33-4296-bbfa-36d32ede4a8a: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:57:46,885::task::495::TaskManager.Task::(_debug) Task 6766ab42-7e33-4296-bbfa-36d32ede4a8a: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:57:46,886::task::495::TaskManager.Task::(_debug) Task 6766ab42-7e33-4296-bbfa-36d32ede4a8a: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:57:46,887::task::495::TaskManager.Task::(_debug) Task 6766ab42-7e33-4296-bbfa-36d32ede4a8a: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:57:46,888::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:57:46,888::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:57:46,889::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:57:46,890::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:57:46,890::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:57:46,891::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:57:46,892::task::495::TaskManager.Task::(_debug) Task 6766ab42-7e33-4296-bbfa-36d32ede4a8a: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:57:46,893::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15917222::DEBUG::2013-11-20 04:57:47,205::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917222::INFO::2013-11-20 04:57:47,206::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917222::DEBUG::2013-11-20 04:57:47,206::task::495::TaskManager.Task::(_debug) Task d2fd9a44-5987-40a3-91df-19ad12aa3cd4: moving from state init -> state preparing
>Thread-15917222::DEBUG::2013-11-20 04:57:47,207::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917222::DEBUG::2013-11-20 04:57:47,208::task::495::TaskManager.Task::(_debug) Task d2fd9a44-5987-40a3-91df-19ad12aa3cd4: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917222::DEBUG::2013-11-20 04:57:47,208::task::495::TaskManager.Task::(_debug) Task d2fd9a44-5987-40a3-91df-19ad12aa3cd4: moving from state preparing -> state finished
>Thread-15917222::DEBUG::2013-11-20 04:57:47,209::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917222::DEBUG::2013-11-20 04:57:47,209::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917222::DEBUG::2013-11-20 04:57:47,210::task::495::TaskManager.Task::(_debug) Task d2fd9a44-5987-40a3-91df-19ad12aa3cd4: ref 0 aborting False
>Thread-15917222::INFO::2013-11-20 04:57:47,211::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917223::DEBUG::2013-11-20 04:57:47,216::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917223::INFO::2013-11-20 04:57:47,217::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917223::DEBUG::2013-11-20 04:57:47,218::task::495::TaskManager.Task::(_debug) Task c640fcf1-d528-4c75-8933-7d50ff363e1b: moving from state init -> state preparing
>Thread-15917223::DEBUG::2013-11-20 04:57:47,219::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8b16f06b-7eda-4f0d-9870-1eea80307abf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917223::DEBUG::2013-11-20 04:57:47,220::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917223::DEBUG::2013-11-20 04:57:47,221::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917223::DEBUG::2013-11-20 04:57:47,221::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`8b16f06b-7eda-4f0d-9870-1eea80307abf`::Granted request
>Thread-15917223::DEBUG::2013-11-20 04:57:47,222::task::495::TaskManager.Task::(_debug) Task c640fcf1-d528-4c75-8933-7d50ff363e1b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917223::DEBUG::2013-11-20 04:57:47,223::task::495::TaskManager.Task::(_debug) Task c640fcf1-d528-4c75-8933-7d50ff363e1b: ref 1 aborting False
>Thread-15917223::WARNING::2013-11-20 04:57:47,225::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917223::DEBUG::2013-11-20 04:57:47,226::task::495::TaskManager.Task::(_debug) Task c640fcf1-d528-4c75-8933-7d50ff363e1b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917223::DEBUG::2013-11-20 04:57:47,227::task::495::TaskManager.Task::(_debug) Task c640fcf1-d528-4c75-8933-7d50ff363e1b: moving from state preparing -> state finished
>Thread-15917223::DEBUG::2013-11-20 04:57:47,228::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917223::DEBUG::2013-11-20 04:57:47,228::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917223::DEBUG::2013-11-20 04:57:47,229::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917223::DEBUG::2013-11-20 04:57:47,230::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917223::DEBUG::2013-11-20 04:57:47,230::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917223::DEBUG::2013-11-20 04:57:47,231::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917223::DEBUG::2013-11-20 04:57:47,232::task::495::TaskManager.Task::(_debug) Task c640fcf1-d528-4c75-8933-7d50ff363e1b: ref 0 aborting False
>Thread-15917223::INFO::2013-11-20 04:57:47,232::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:57:48,306::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:57:48,822::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0777638 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:57:50,837::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 04:57:51,077::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:57:51,078::task::495::TaskManager.Task::(_debug) Task 22aba95d-60c2-4899-b964-45fa7f8ced70: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:57:51,081::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9832c5e9-12fa-4f57-926b-23dbbd98a929`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:57:51,082::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:57:51,083::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:57:51,083::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9832c5e9-12fa-4f57-926b-23dbbd98a929`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:57:51,085::task::495::TaskManager.Task::(_debug) Task 22aba95d-60c2-4899-b964-45fa7f8ced70: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:57:51,085::task::495::TaskManager.Task::(_debug) Task 22aba95d-60c2-4899-b964-45fa7f8ced70: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:57:51,086::task::495::TaskManager.Task::(_debug) Task 22aba95d-60c2-4899-b964-45fa7f8ced70: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:57:51,087::task::495::TaskManager.Task::(_debug) Task 22aba95d-60c2-4899-b964-45fa7f8ced70: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:57:51,087::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:57:51,088::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:57:51,089::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:57:51,089::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:57:51,090::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:57:51,091::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:57:51,091::task::495::TaskManager.Task::(_debug) Task 22aba95d-60c2-4899-b964-45fa7f8ced70: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:57:51,092::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:57:51,435::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0854199 s, 12.0 MB/s\n'; <rc> = 0
>Thread-15917226::INFO::2013-11-20 04:57:53,363::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15917226::DEBUG::2013-11-20 04:57:53,364::task::495::TaskManager.Task::(_debug) Task 9fbc1765-54b2-4864-82ce-86320d309753: moving from state init -> state preparing
>Thread-15917226::DEBUG::2013-11-20 04:57:53,365::task::495::TaskManager.Task::(_debug) Task 9fbc1765-54b2-4864-82ce-86320d309753: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000410079956055', 'lastCheck': 1384923464.8521571, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00100612640381', 'lastCheck': 1384923465.961143, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00141000747681', 'lastCheck': 1384923465.9605141, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00056791305542', 'lastCheck': 1384923469.5008891, 'valid': True, 'code': 0}}
>Thread-15917226::DEBUG::2013-11-20 04:57:53,367::task::495::TaskManager.Task::(_debug) Task 9fbc1765-54b2-4864-82ce-86320d309753: moving from state preparing -> state finished
>Thread-15917226::DEBUG::2013-11-20 04:57:53,367::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917226::DEBUG::2013-11-20 04:57:53,368::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917226::DEBUG::2013-11-20 04:57:53,369::task::495::TaskManager.Task::(_debug) Task 9fbc1765-54b2-4864-82ce-86320d309753: ref 0 aborting False
>Thread-15917226::INFO::2013-11-20 04:57:53,369::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000410079956055', 'lastCheck': 1384923464.8521571, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00100612640381', 'lastCheck': 1384923465.961143, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00141000747681', 'lastCheck': 1384923465.9605141, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.00056791305542', 'lastCheck': 1384923469.5008891, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:57:53,446::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:57:53,977::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0787606 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:57:55,992::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:57:56,559::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0986059 s, 10.4 MB/s\n'; <rc> = 0
>Thread-15913546::INFO::2013-11-20 04:57:57,066::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15913546::DEBUG::2013-11-20 04:57:57,068::task::495::TaskManager.Task::(_debug) Task c49c8813-3d6c-4df0-baee-405a776de7d6: moving from state init -> state preparing
>Thread-15914519::INFO::2013-11-20 04:57:57,069::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15914519::DEBUG::2013-11-20 04:57:57,071::task::495::TaskManager.Task::(_debug) Task ce992ac1-ecdf-49b6-8251-9a05a56f0d6d: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:57:57,072::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e07d8272-8a07-474b-96fc-fc5274c18a6b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:57:57,081::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:57:57,081::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3800f17a-bbda-44c9-a64d-4ce8a34f9435`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:57:57,083::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:57:57,083::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:57:57,084::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e07d8272-8a07-474b-96fc-fc5274c18a6b`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:57:57,085::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:57:57,086::task::495::TaskManager.Task::(_debug) Task c49c8813-3d6c-4df0-baee-405a776de7d6: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15914519::DEBUG::2013-11-20 04:57:57,087::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3800f17a-bbda-44c9-a64d-4ce8a34f9435`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:57:57,087::task::495::TaskManager.Task::(_debug) Task c49c8813-3d6c-4df0-baee-405a776de7d6: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:57:57,096::task::495::TaskManager.Task::(_debug) Task ce992ac1-ecdf-49b6-8251-9a05a56f0d6d: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913546::DEBUG::2013-11-20 04:57:57,100::task::495::TaskManager.Task::(_debug) Task c49c8813-3d6c-4df0-baee-405a776de7d6: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:57:57,101::task::495::TaskManager.Task::(_debug) Task ce992ac1-ecdf-49b6-8251-9a05a56f0d6d: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:57:57,102::task::495::TaskManager.Task::(_debug) Task c49c8813-3d6c-4df0-baee-405a776de7d6: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:57:57,103::task::495::TaskManager.Task::(_debug) Task ce992ac1-ecdf-49b6-8251-9a05a56f0d6d: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:57:57,103::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:57:57,104::task::495::TaskManager.Task::(_debug) Task ce992ac1-ecdf-49b6-8251-9a05a56f0d6d: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:57:57,105::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:57:57,105::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:57:57,106::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15914519::DEBUG::2013-11-20 04:57:57,107::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:57:57,107::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:57:57,108::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913546::DEBUG::2013-11-20 04:57:57,109::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:57:57,110::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:57:57,111::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:57:57,111::task::495::TaskManager.Task::(_debug) Task c49c8813-3d6c-4df0-baee-405a776de7d6: ref 0 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:57:57,112::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913546::INFO::2013-11-20 04:57:57,113::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:57:57,113::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913546::INFO::2013-11-20 04:57:57,114::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15914519::DEBUG::2013-11-20 04:57:57,116::task::495::TaskManager.Task::(_debug) Task ce992ac1-ecdf-49b6-8251-9a05a56f0d6d: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:57:57,116::task::495::TaskManager.Task::(_debug) Task fa1642cf-1e74-4518-9425-8c075a85f29b: moving from state init -> state preparing
>Thread-15914519::INFO::2013-11-20 04:57:57,117::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:57:57,118::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`24fd27f1-517b-4724-abb7-cda6f0ffcd2c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:57:57,119::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:57:57,120::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:57:57,121::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`24fd27f1-517b-4724-abb7-cda6f0ffcd2c`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:57:57,123::task::495::TaskManager.Task::(_debug) Task fa1642cf-1e74-4518-9425-8c075a85f29b: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:57:57,124::task::495::TaskManager.Task::(_debug) Task fa1642cf-1e74-4518-9425-8c075a85f29b: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:57:57,125::task::495::TaskManager.Task::(_debug) Task fa1642cf-1e74-4518-9425-8c075a85f29b: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:57:57,125::task::495::TaskManager.Task::(_debug) Task fa1642cf-1e74-4518-9425-8c075a85f29b: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:57:57,126::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:57:57,127::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:57:57,127::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:57:57,128::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:57:57,129::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:57:57,129::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:57:57,130::task::495::TaskManager.Task::(_debug) Task fa1642cf-1e74-4518-9425-8c075a85f29b: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:57:57,130::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:57:57,131::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:57:57,132::task::495::TaskManager.Task::(_debug) Task a422477d-011e-46c2-9045-09ee21d1911b: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:57:57,134::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a01d10ae-6246-4171-8912-66ce0a5aca33`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:57:57,135::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:57:57,136::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:57:57,136::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`a01d10ae-6246-4171-8912-66ce0a5aca33`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:57:57,137::task::495::TaskManager.Task::(_debug) Task a422477d-011e-46c2-9045-09ee21d1911b: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:57:57,138::task::495::TaskManager.Task::(_debug) Task a422477d-011e-46c2-9045-09ee21d1911b: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:57:57,138::task::495::TaskManager.Task::(_debug) Task a422477d-011e-46c2-9045-09ee21d1911b: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:57:57,139::task::495::TaskManager.Task::(_debug) Task a422477d-011e-46c2-9045-09ee21d1911b: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:57:57,139::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:57:57,140::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:57:57,141::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:57:57,145::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:57:57,146::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:57:57,147::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:57:57,148::task::495::TaskManager.Task::(_debug) Task a422477d-011e-46c2-9045-09ee21d1911b: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:57:57,149::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:57:57,149::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:57:57,150::task::495::TaskManager.Task::(_debug) Task 8954edc9-12ae-4e8e-8f37-c504e79729c9: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:57:57,151::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`dd5ad1df-d2e4-42b9-9e96-73294ac7a834`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:57:57,152::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:57:57,152::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:57:57,153::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`dd5ad1df-d2e4-42b9-9e96-73294ac7a834`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:57:57,154::task::495::TaskManager.Task::(_debug) Task 8954edc9-12ae-4e8e-8f37-c504e79729c9: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:57:57,154::task::495::TaskManager.Task::(_debug) Task 8954edc9-12ae-4e8e-8f37-c504e79729c9: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:57:57,155::task::495::TaskManager.Task::(_debug) Task 8954edc9-12ae-4e8e-8f37-c504e79729c9: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:57:57,156::task::495::TaskManager.Task::(_debug) Task 8954edc9-12ae-4e8e-8f37-c504e79729c9: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:57:57,156::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:57:57,157::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:57:57,158::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:57:57,159::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:57:57,160::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:57:57,160::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:57:57,161::task::495::TaskManager.Task::(_debug) Task 8954edc9-12ae-4e8e-8f37-c504e79729c9: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:57:57,161::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15917229::DEBUG::2013-11-20 04:57:57,752::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917229::INFO::2013-11-20 04:57:57,754::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917229::DEBUG::2013-11-20 04:57:57,754::task::495::TaskManager.Task::(_debug) Task 4538ba50-b40e-486b-bfc4-ff81216ae8ae: moving from state init -> state preparing
>Thread-15917229::DEBUG::2013-11-20 04:57:57,755::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917229::DEBUG::2013-11-20 04:57:57,756::task::495::TaskManager.Task::(_debug) Task 4538ba50-b40e-486b-bfc4-ff81216ae8ae: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917229::DEBUG::2013-11-20 04:57:57,756::task::495::TaskManager.Task::(_debug) Task 4538ba50-b40e-486b-bfc4-ff81216ae8ae: moving from state preparing -> state finished
>Thread-15917229::DEBUG::2013-11-20 04:57:57,757::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917229::DEBUG::2013-11-20 04:57:57,757::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917229::DEBUG::2013-11-20 04:57:57,758::task::495::TaskManager.Task::(_debug) Task 4538ba50-b40e-486b-bfc4-ff81216ae8ae: ref 0 aborting False
>Thread-15917229::INFO::2013-11-20 04:57:57,759::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917230::DEBUG::2013-11-20 04:57:57,763::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917230::INFO::2013-11-20 04:57:57,764::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917230::DEBUG::2013-11-20 04:57:57,765::task::495::TaskManager.Task::(_debug) Task 72ea89d3-f171-4438-b120-183152e4a16c: moving from state init -> state preparing
>Thread-15917230::DEBUG::2013-11-20 04:57:57,766::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`87d86947-db8f-41a4-a38f-4ea7356ce9ae`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917230::DEBUG::2013-11-20 04:57:57,767::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917230::DEBUG::2013-11-20 04:57:57,768::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917230::DEBUG::2013-11-20 04:57:57,768::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`87d86947-db8f-41a4-a38f-4ea7356ce9ae`::Granted request
>Thread-15917230::DEBUG::2013-11-20 04:57:57,769::task::495::TaskManager.Task::(_debug) Task 72ea89d3-f171-4438-b120-183152e4a16c: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917230::DEBUG::2013-11-20 04:57:57,770::task::495::TaskManager.Task::(_debug) Task 72ea89d3-f171-4438-b120-183152e4a16c: ref 1 aborting False
>Thread-15917230::WARNING::2013-11-20 04:57:57,772::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917230::DEBUG::2013-11-20 04:57:57,773::task::495::TaskManager.Task::(_debug) Task 72ea89d3-f171-4438-b120-183152e4a16c: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917230::DEBUG::2013-11-20 04:57:57,774::task::495::TaskManager.Task::(_debug) Task 72ea89d3-f171-4438-b120-183152e4a16c: moving from state preparing -> state finished
>Thread-15917230::DEBUG::2013-11-20 04:57:57,774::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917230::DEBUG::2013-11-20 04:57:57,775::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917230::DEBUG::2013-11-20 04:57:57,775::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917230::DEBUG::2013-11-20 04:57:57,776::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917230::DEBUG::2013-11-20 04:57:57,777::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917230::DEBUG::2013-11-20 04:57:57,777::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917230::DEBUG::2013-11-20 04:57:57,778::task::495::TaskManager.Task::(_debug) Task 72ea89d3-f171-4438-b120-183152e4a16c: ref 0 aborting False
>Thread-15917230::INFO::2013-11-20 04:57:57,778::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:57:58,569::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:57:59,123::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0776599 s, 13.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:58:01,138::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:58:01,697::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0816358 s, 12.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:58:03,714::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:58:04,274::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0892709 s, 11.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:58:06,290::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:58:06,772::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0789098 s, 13.0 MB/s\n'; <rc> = 0
>Thread-15917234::DEBUG::2013-11-20 04:58:08,212::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917234::INFO::2013-11-20 04:58:08,214::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917234::DEBUG::2013-11-20 04:58:08,214::task::495::TaskManager.Task::(_debug) Task 80d1a51e-1f79-4697-97ac-606a968e6b2b: moving from state init -> state preparing
>Thread-15917234::DEBUG::2013-11-20 04:58:08,215::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917234::DEBUG::2013-11-20 04:58:08,216::task::495::TaskManager.Task::(_debug) Task 80d1a51e-1f79-4697-97ac-606a968e6b2b: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917234::DEBUG::2013-11-20 04:58:08,217::task::495::TaskManager.Task::(_debug) Task 80d1a51e-1f79-4697-97ac-606a968e6b2b: moving from state preparing -> state finished
>Thread-15917234::DEBUG::2013-11-20 04:58:08,217::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917234::DEBUG::2013-11-20 04:58:08,218::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917234::DEBUG::2013-11-20 04:58:08,219::task::495::TaskManager.Task::(_debug) Task 80d1a51e-1f79-4697-97ac-606a968e6b2b: ref 0 aborting False
>Thread-15917234::INFO::2013-11-20 04:58:08,219::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917235::DEBUG::2013-11-20 04:58:08,225::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917235::INFO::2013-11-20 04:58:08,226::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917235::DEBUG::2013-11-20 04:58:08,227::task::495::TaskManager.Task::(_debug) Task 804efe83-e389-4c1f-ae5e-e05d85dcb030: moving from state init -> state preparing
>Thread-15917235::DEBUG::2013-11-20 04:58:08,228::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d5c54018-98d7-4556-97cf-9e2b331c01bb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917235::DEBUG::2013-11-20 04:58:08,229::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917235::DEBUG::2013-11-20 04:58:08,230::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917235::DEBUG::2013-11-20 04:58:08,230::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`d5c54018-98d7-4556-97cf-9e2b331c01bb`::Granted request
>Thread-15917235::DEBUG::2013-11-20 04:58:08,231::task::495::TaskManager.Task::(_debug) Task 804efe83-e389-4c1f-ae5e-e05d85dcb030: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917235::DEBUG::2013-11-20 04:58:08,232::task::495::TaskManager.Task::(_debug) Task 804efe83-e389-4c1f-ae5e-e05d85dcb030: ref 1 aborting False
>Thread-15917235::WARNING::2013-11-20 04:58:08,234::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917235::DEBUG::2013-11-20 04:58:08,235::task::495::TaskManager.Task::(_debug) Task 804efe83-e389-4c1f-ae5e-e05d85dcb030: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917235::DEBUG::2013-11-20 04:58:08,236::task::495::TaskManager.Task::(_debug) Task 804efe83-e389-4c1f-ae5e-e05d85dcb030: moving from state preparing -> state finished
>Thread-15917235::DEBUG::2013-11-20 04:58:08,237::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917235::DEBUG::2013-11-20 04:58:08,237::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917235::DEBUG::2013-11-20 04:58:08,238::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917235::DEBUG::2013-11-20 04:58:08,239::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917235::DEBUG::2013-11-20 04:58:08,239::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917235::DEBUG::2013-11-20 04:58:08,240::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917235::DEBUG::2013-11-20 04:58:08,240::task::495::TaskManager.Task::(_debug) Task 804efe83-e389-4c1f-ae5e-e05d85dcb030: ref 0 aborting False
>Thread-15917235::INFO::2013-11-20 04:58:08,241::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:58:08,785::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 04:58:09,021::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:58:09,023::task::495::TaskManager.Task::(_debug) Task 52c1ab20-6f74-40e0-8bbb-a7a81c3df7d9: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:58:09,025::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`70423b66-7b9e-4f04-904d-32f6a144d357`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:58:09,026::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:58:09,027::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:58:09,028::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`70423b66-7b9e-4f04-904d-32f6a144d357`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:58:09,029::task::495::TaskManager.Task::(_debug) Task 52c1ab20-6f74-40e0-8bbb-a7a81c3df7d9: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:58:09,030::task::495::TaskManager.Task::(_debug) Task 52c1ab20-6f74-40e0-8bbb-a7a81c3df7d9: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:58:09,031::task::495::TaskManager.Task::(_debug) Task 52c1ab20-6f74-40e0-8bbb-a7a81c3df7d9: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:58:09,031::task::495::TaskManager.Task::(_debug) Task 52c1ab20-6f74-40e0-8bbb-a7a81c3df7d9: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:58:09,032::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:58:09,033::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:58:09,034::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:58:09,034::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:58:09,035::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:58:09,036::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:58:09,036::task::495::TaskManager.Task::(_debug) Task 52c1ab20-6f74-40e0-8bbb-a7a81c3df7d9: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:58:09,037::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15917236::INFO::2013-11-20 04:58:09,287::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15917236::DEBUG::2013-11-20 04:58:09,288::task::495::TaskManager.Task::(_debug) Task 253ee339-6a13-4fb6-adae-c0670df5d4b3: moving from state init -> state preparing
>Thread-15917236::DEBUG::2013-11-20 04:58:09,289::task::495::TaskManager.Task::(_debug) Task 253ee339-6a13-4fb6-adae-c0670df5d4b3: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000663042068481', 'lastCheck': 1384923485.346663, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00100612640381', 'lastCheck': 1384923465.961143, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00141000747681', 'lastCheck': 1384923465.9605141, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000547885894775', 'lastCheck': 1384923479.7949929, 'valid': True, 'code': 0}}
>Thread-15917236::DEBUG::2013-11-20 04:58:09,291::task::495::TaskManager.Task::(_debug) Task 253ee339-6a13-4fb6-adae-c0670df5d4b3: moving from state preparing -> state finished
>Thread-15917236::DEBUG::2013-11-20 04:58:09,292::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917236::DEBUG::2013-11-20 04:58:09,293::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917236::DEBUG::2013-11-20 04:58:09,294::task::495::TaskManager.Task::(_debug) Task 253ee339-6a13-4fb6-adae-c0670df5d4b3: ref 0 aborting False
>Thread-15917236::INFO::2013-11-20 04:58:09,295::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000663042068481', 'lastCheck': 1384923485.346663, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00100612640381', 'lastCheck': 1384923465.961143, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00141000747681', 'lastCheck': 1384923465.9605141, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000547885894775', 'lastCheck': 1384923479.7949929, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:58:09,405::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.096396 s, 10.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:58:11,420::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:58:11,956::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0768371 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:58:13,971::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:58:14,492::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0609528 s, 16.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:58:16,505::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:58:17,054::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.083388 s, 12.3 MB/s\n'; <rc> = 0
>Thread-15917240::DEBUG::2013-11-20 04:58:18,695::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917240::INFO::2013-11-20 04:58:18,697::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917240::DEBUG::2013-11-20 04:58:18,698::task::495::TaskManager.Task::(_debug) Task 32d0cd12-6bf7-4864-8c0a-7f6785167575: moving from state init -> state preparing
>Thread-15917240::DEBUG::2013-11-20 04:58:18,699::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917240::DEBUG::2013-11-20 04:58:18,699::task::495::TaskManager.Task::(_debug) Task 32d0cd12-6bf7-4864-8c0a-7f6785167575: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917240::DEBUG::2013-11-20 04:58:18,700::task::495::TaskManager.Task::(_debug) Task 32d0cd12-6bf7-4864-8c0a-7f6785167575: moving from state preparing -> state finished
>Thread-15917240::DEBUG::2013-11-20 04:58:18,701::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917240::DEBUG::2013-11-20 04:58:18,702::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917240::DEBUG::2013-11-20 04:58:18,702::task::495::TaskManager.Task::(_debug) Task 32d0cd12-6bf7-4864-8c0a-7f6785167575: ref 0 aborting False
>Thread-15917240::INFO::2013-11-20 04:58:18,703::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917241::DEBUG::2013-11-20 04:58:18,709::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917241::INFO::2013-11-20 04:58:18,710::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917241::DEBUG::2013-11-20 04:58:18,711::task::495::TaskManager.Task::(_debug) Task 82a17d2d-04bc-4478-b1f4-068686ce9418: moving from state init -> state preparing
>Thread-15917241::DEBUG::2013-11-20 04:58:18,712::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`86b03051-565d-4a08-9bdd-a9f57f2958b9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917241::DEBUG::2013-11-20 04:58:18,713::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917241::DEBUG::2013-11-20 04:58:18,714::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917241::DEBUG::2013-11-20 04:58:18,714::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`86b03051-565d-4a08-9bdd-a9f57f2958b9`::Granted request
>Thread-15917241::DEBUG::2013-11-20 04:58:18,715::task::495::TaskManager.Task::(_debug) Task 82a17d2d-04bc-4478-b1f4-068686ce9418: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917241::DEBUG::2013-11-20 04:58:18,716::task::495::TaskManager.Task::(_debug) Task 82a17d2d-04bc-4478-b1f4-068686ce9418: ref 1 aborting False
>Thread-15917241::WARNING::2013-11-20 04:58:18,718::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917241::DEBUG::2013-11-20 04:58:18,719::task::495::TaskManager.Task::(_debug) Task 82a17d2d-04bc-4478-b1f4-068686ce9418: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917241::DEBUG::2013-11-20 04:58:18,720::task::495::TaskManager.Task::(_debug) Task 82a17d2d-04bc-4478-b1f4-068686ce9418: moving from state preparing -> state finished
>Thread-15917241::DEBUG::2013-11-20 04:58:18,720::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917241::DEBUG::2013-11-20 04:58:18,721::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917241::DEBUG::2013-11-20 04:58:18,722::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917241::DEBUG::2013-11-20 04:58:18,722::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917241::DEBUG::2013-11-20 04:58:18,723::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917241::DEBUG::2013-11-20 04:58:18,723::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917241::DEBUG::2013-11-20 04:58:18,724::task::495::TaskManager.Task::(_debug) Task 82a17d2d-04bc-4478-b1f4-068686ce9418: ref 0 aborting False
>Thread-15917241::INFO::2013-11-20 04:58:18,725::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:58:19,062::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:58:19,244::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:58:19,246::task::495::TaskManager.Task::(_debug) Task b58b677c-f587-4606-8c97-3bd5c671e3d7: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:58:19,249::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`65d4c8ac-c46c-4217-98e5-e4e19d1ff1d2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:58:19,250::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:58:19,252::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:58:19,252::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`65d4c8ac-c46c-4217-98e5-e4e19d1ff1d2`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:58:19,254::task::495::TaskManager.Task::(_debug) Task b58b677c-f587-4606-8c97-3bd5c671e3d7: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:58:19,256::task::495::TaskManager.Task::(_debug) Task b58b677c-f587-4606-8c97-3bd5c671e3d7: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:58:19,257::task::495::TaskManager.Task::(_debug) Task b58b677c-f587-4606-8c97-3bd5c671e3d7: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:58:19,258::task::495::TaskManager.Task::(_debug) Task b58b677c-f587-4606-8c97-3bd5c671e3d7: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:58:19,259::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:58:19,259::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:58:19,260::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:58:19,261::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:58:19,262::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:58:19,262::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:58:19,263::task::495::TaskManager.Task::(_debug) Task b58b677c-f587-4606-8c97-3bd5c671e3d7: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:58:19,264::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:58:19,578::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.089123 s, 11.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:58:21,592::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915918::INFO::2013-11-20 04:58:21,828::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:58:21,835::task::495::TaskManager.Task::(_debug) Task 308ecc80-7bce-47c7-8932-68cbb98dc149: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:58:21,839::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2c058abb-5af5-4388-9c03-b1be493fbc5b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:58:21,840::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:58:21,842::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:58:21,844::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`2c058abb-5af5-4388-9c03-b1be493fbc5b`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:58:21,846::task::495::TaskManager.Task::(_debug) Task 308ecc80-7bce-47c7-8932-68cbb98dc149: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:58:21,847::task::495::TaskManager.Task::(_debug) Task 308ecc80-7bce-47c7-8932-68cbb98dc149: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:58:21,848::task::495::TaskManager.Task::(_debug) Task 308ecc80-7bce-47c7-8932-68cbb98dc149: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:58:21,849::task::495::TaskManager.Task::(_debug) Task 308ecc80-7bce-47c7-8932-68cbb98dc149: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:58:21,850::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:58:21,850::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:58:21,851::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:58:21,852::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:58:21,852::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:58:21,853::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:58:21,853::task::495::TaskManager.Task::(_debug) Task 308ecc80-7bce-47c7-8932-68cbb98dc149: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:58:21,854::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:58:22,159::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0826845 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:58:24,177::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916984::INFO::2013-11-20 04:58:24,393::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=240c0593-fe33-4f8b-a029-097379dbcb85 volUUID=f715faaa-081a-4598-b125-80a2d12af554)
>Thread-15916984::DEBUG::2013-11-20 04:58:24,396::task::495::TaskManager.Task::(_debug) Task d7c909cf-7345-42c1-aae2-6adfa0ca271b: moving from state init -> state preparing
>Thread-15916984::DEBUG::2013-11-20 04:58:24,400::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`089b60b5-b77d-4f7c-b82a-ac4cc48c88d9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15916984::DEBUG::2013-11-20 04:58:24,401::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15916984::DEBUG::2013-11-20 04:58:24,409::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15916984::DEBUG::2013-11-20 04:58:24,410::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`089b60b5-b77d-4f7c-b82a-ac4cc48c88d9`::Granted request
>Thread-15916984::DEBUG::2013-11-20 04:58:24,411::task::495::TaskManager.Task::(_debug) Task d7c909cf-7345-42c1-aae2-6adfa0ca271b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15916984::DEBUG::2013-11-20 04:58:24,412::task::495::TaskManager.Task::(_debug) Task d7c909cf-7345-42c1-aae2-6adfa0ca271b: ref 1 aborting False
>Thread-15916984::DEBUG::2013-11-20 04:58:24,418::task::495::TaskManager.Task::(_debug) Task d7c909cf-7345-42c1-aae2-6adfa0ca271b: finished: {'truesize': '75296145408', 'apparentsize': '75296145408'}
>Thread-15916984::DEBUG::2013-11-20 04:58:24,419::task::495::TaskManager.Task::(_debug) Task d7c909cf-7345-42c1-aae2-6adfa0ca271b: moving from state preparing -> state finished
>Thread-15916984::DEBUG::2013-11-20 04:58:24,419::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15916984::DEBUG::2013-11-20 04:58:24,420::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916984::DEBUG::2013-11-20 04:58:24,421::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15916984::DEBUG::2013-11-20 04:58:24,422::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15916984::DEBUG::2013-11-20 04:58:24,422::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15916984::DEBUG::2013-11-20 04:58:24,423::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15916984::DEBUG::2013-11-20 04:58:24,429::task::495::TaskManager.Task::(_debug) Task d7c909cf-7345-42c1-aae2-6adfa0ca271b: ref 0 aborting False
>Thread-15916984::INFO::2013-11-20 04:58:24,430::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '75296145408', 'apparentsize': '75296145408'}
>Dummy-15912651::DEBUG::2013-11-20 04:58:24,730::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0911173 s, 11.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:58:26,746::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915863::INFO::2013-11-20 04:58:26,956::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15915863::DEBUG::2013-11-20 04:58:26,960::task::495::TaskManager.Task::(_debug) Task 4430da74-1f96-4738-9eb7-cfcc12fa46e3: moving from state init -> state preparing
>Thread-15917244::INFO::2013-11-20 04:58:26,964::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15917244::DEBUG::2013-11-20 04:58:26,968::task::495::TaskManager.Task::(_debug) Task a663aee7-ed28-4b3e-9156-970d95c0222a: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:58:26,970::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`51593c21-2995-4bfa-8edd-61e0c13d658e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15917244::DEBUG::2013-11-20 04:58:26,972::task::495::TaskManager.Task::(_debug) Task a663aee7-ed28-4b3e-9156-970d95c0222a: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000385999679565', 'lastCheck': 1384923505.8307829, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00100612640381', 'lastCheck': 1384923465.961143, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00141000747681', 'lastCheck': 1384923465.9605141, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000490188598633', 'lastCheck': 1384923500.238615, 'valid': True, 'code': 0}}
>Thread-15915863::DEBUG::2013-11-20 04:58:26,973::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15917244::DEBUG::2013-11-20 04:58:26,974::task::495::TaskManager.Task::(_debug) Task a663aee7-ed28-4b3e-9156-970d95c0222a: moving from state preparing -> state finished
>Thread-15917244::DEBUG::2013-11-20 04:58:26,976::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15915863::DEBUG::2013-11-20 04:58:26,977::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15917244::DEBUG::2013-11-20 04:58:26,977::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:58:26,978::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`51593c21-2995-4bfa-8edd-61e0c13d658e`::Granted request
>Thread-15917244::DEBUG::2013-11-20 04:58:26,979::task::495::TaskManager.Task::(_debug) Task a663aee7-ed28-4b3e-9156-970d95c0222a: ref 0 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:58:26,981::task::495::TaskManager.Task::(_debug) Task 4430da74-1f96-4738-9eb7-cfcc12fa46e3: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15917244::INFO::2013-11-20 04:58:26,982::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000385999679565', 'lastCheck': 1384923505.8307829, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00100612640381', 'lastCheck': 1384923465.961143, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00141000747681', 'lastCheck': 1384923465.9605141, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000490188598633', 'lastCheck': 1384923500.238615, 'valid': True, 'code': 0}}
>Thread-15915863::DEBUG::2013-11-20 04:58:26,983::task::495::TaskManager.Task::(_debug) Task 4430da74-1f96-4738-9eb7-cfcc12fa46e3: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:58:26,985::task::495::TaskManager.Task::(_debug) Task 4430da74-1f96-4738-9eb7-cfcc12fa46e3: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15915863::DEBUG::2013-11-20 04:58:26,987::task::495::TaskManager.Task::(_debug) Task 4430da74-1f96-4738-9eb7-cfcc12fa46e3: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:58:26,988::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:58:26,989::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:58:26,991::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:58:26,992::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915863::DEBUG::2013-11-20 04:58:26,993::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:58:26,994::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:58:26,999::task::495::TaskManager.Task::(_debug) Task 4430da74-1f96-4738-9eb7-cfcc12fa46e3: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:58:27,000::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:58:27,309::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0899664 s, 11.4 MB/s\n'; <rc> = 0
>Thread-15913658::INFO::2013-11-20 04:58:27,370::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15913658::DEBUG::2013-11-20 04:58:27,371::task::495::TaskManager.Task::(_debug) Task 97b40bbf-656e-4ab7-87ad-0a091d223e78: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:58:27,373::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`58acedf4-c806-4d68-a4e1-3cab99abacd8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:58:27,373::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:58:27,374::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:58:27,374::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`58acedf4-c806-4d68-a4e1-3cab99abacd8`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:58:27,376::task::495::TaskManager.Task::(_debug) Task 97b40bbf-656e-4ab7-87ad-0a091d223e78: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:58:27,377::task::495::TaskManager.Task::(_debug) Task 97b40bbf-656e-4ab7-87ad-0a091d223e78: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:58:27,378::task::495::TaskManager.Task::(_debug) Task 97b40bbf-656e-4ab7-87ad-0a091d223e78: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913658::DEBUG::2013-11-20 04:58:27,379::task::495::TaskManager.Task::(_debug) Task 97b40bbf-656e-4ab7-87ad-0a091d223e78: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:58:27,379::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:58:27,380::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:58:27,380::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:58:27,381::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::DEBUG::2013-11-20 04:58:27,382::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913658::DEBUG::2013-11-20 04:58:27,382::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913658::DEBUG::2013-11-20 04:58:27,383::task::495::TaskManager.Task::(_debug) Task 97b40bbf-656e-4ab7-87ad-0a091d223e78: ref 0 aborting False
>Thread-15913658::INFO::2013-11-20 04:58:27,384::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:58:29,319::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:58:29,876::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0833399 s, 12.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:58:31,900::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:58:32,455::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0827166 s, 12.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:58:34,469::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:58:35,007::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0733831 s, 14.0 MB/s\n'; <rc> = 0
>Thread-15917246::DEBUG::2013-11-20 04:58:35,172::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917246::INFO::2013-11-20 04:58:35,173::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917246::DEBUG::2013-11-20 04:58:35,174::task::495::TaskManager.Task::(_debug) Task d986da73-f88f-4ec8-9474-5d83bf5ab3f8: moving from state init -> state preparing
>Thread-15917246::DEBUG::2013-11-20 04:58:35,175::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917246::DEBUG::2013-11-20 04:58:35,176::task::495::TaskManager.Task::(_debug) Task d986da73-f88f-4ec8-9474-5d83bf5ab3f8: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917246::DEBUG::2013-11-20 04:58:35,176::task::495::TaskManager.Task::(_debug) Task d986da73-f88f-4ec8-9474-5d83bf5ab3f8: moving from state preparing -> state finished
>Thread-15917246::DEBUG::2013-11-20 04:58:35,177::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917246::DEBUG::2013-11-20 04:58:35,177::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917246::DEBUG::2013-11-20 04:58:35,178::task::495::TaskManager.Task::(_debug) Task d986da73-f88f-4ec8-9474-5d83bf5ab3f8: ref 0 aborting False
>Thread-15917246::INFO::2013-11-20 04:58:35,179::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917247::DEBUG::2013-11-20 04:58:35,185::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917247::INFO::2013-11-20 04:58:35,186::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917247::DEBUG::2013-11-20 04:58:35,187::task::495::TaskManager.Task::(_debug) Task 482470e6-09a8-4521-bf58-09114e200311: moving from state init -> state preparing
>Thread-15917247::DEBUG::2013-11-20 04:58:35,188::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0eb5948a-f95c-4cda-aea3-a31b0a48e1da`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917247::DEBUG::2013-11-20 04:58:35,189::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917247::DEBUG::2013-11-20 04:58:35,190::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917247::DEBUG::2013-11-20 04:58:35,190::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`0eb5948a-f95c-4cda-aea3-a31b0a48e1da`::Granted request
>Thread-15917247::DEBUG::2013-11-20 04:58:35,191::task::495::TaskManager.Task::(_debug) Task 482470e6-09a8-4521-bf58-09114e200311: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917247::DEBUG::2013-11-20 04:58:35,192::task::495::TaskManager.Task::(_debug) Task 482470e6-09a8-4521-bf58-09114e200311: ref 1 aborting False
>Thread-15917247::WARNING::2013-11-20 04:58:35,194::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917247::DEBUG::2013-11-20 04:58:35,195::task::495::TaskManager.Task::(_debug) Task 482470e6-09a8-4521-bf58-09114e200311: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917247::DEBUG::2013-11-20 04:58:35,196::task::495::TaskManager.Task::(_debug) Task 482470e6-09a8-4521-bf58-09114e200311: moving from state preparing -> state finished
>Thread-15917247::DEBUG::2013-11-20 04:58:35,196::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917247::DEBUG::2013-11-20 04:58:35,197::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917247::DEBUG::2013-11-20 04:58:35,197::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917247::DEBUG::2013-11-20 04:58:35,198::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917247::DEBUG::2013-11-20 04:58:35,199::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917247::DEBUG::2013-11-20 04:58:35,199::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917247::DEBUG::2013-11-20 04:58:35,200::task::495::TaskManager.Task::(_debug) Task 482470e6-09a8-4521-bf58-09114e200311: ref 0 aborting False
>Thread-15917247::INFO::2013-11-20 04:58:35,200::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:58:37,024::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 04:58:37,261::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:58:37,262::task::495::TaskManager.Task::(_debug) Task 7605fbc6-486e-4597-9ef4-941b546dfbaa: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:58:37,264::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1a64576c-d5f5-4429-8cc0-a5f838af6b7f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:58:37,265::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:58:37,267::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:58:37,268::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1a64576c-d5f5-4429-8cc0-a5f838af6b7f`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:58:37,269::task::495::TaskManager.Task::(_debug) Task 7605fbc6-486e-4597-9ef4-941b546dfbaa: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:58:37,270::task::495::TaskManager.Task::(_debug) Task 7605fbc6-486e-4597-9ef4-941b546dfbaa: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:58:37,271::task::495::TaskManager.Task::(_debug) Task 7605fbc6-486e-4597-9ef4-941b546dfbaa: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:58:37,271::task::495::TaskManager.Task::(_debug) Task 7605fbc6-486e-4597-9ef4-941b546dfbaa: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:58:37,272::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:58:37,273::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:58:37,273::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:58:37,274::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:58:37,274::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:58:37,275::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:58:37,275::task::495::TaskManager.Task::(_debug) Task 7605fbc6-486e-4597-9ef4-941b546dfbaa: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:58:37,276::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:58:37,559::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0789058 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:58:39,567::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:58:40,115::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.079648 s, 12.9 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 04:58:41,829::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:58:41,830::task::495::TaskManager.Task::(_debug) Task cf4387aa-0448-43a9-abce-0a8d06fe49a0: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:58:41,832::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a698e808-466a-4bbf-a5de-ba690ca2d3bc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:58:41,832::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:58:41,833::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:58:41,834::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a698e808-466a-4bbf-a5de-ba690ca2d3bc`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:58:41,835::task::495::TaskManager.Task::(_debug) Task cf4387aa-0448-43a9-abce-0a8d06fe49a0: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:58:41,835::task::495::TaskManager.Task::(_debug) Task cf4387aa-0448-43a9-abce-0a8d06fe49a0: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:58:41,836::task::495::TaskManager.Task::(_debug) Task cf4387aa-0448-43a9-abce-0a8d06fe49a0: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:58:41,837::task::495::TaskManager.Task::(_debug) Task cf4387aa-0448-43a9-abce-0a8d06fe49a0: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:58:41,837::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:58:41,838::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:58:41,838::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:58:41,839::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:58:41,840::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:58:41,840::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:58:41,841::task::495::TaskManager.Task::(_debug) Task cf4387aa-0448-43a9-abce-0a8d06fe49a0: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:58:41,841::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:58:41,842::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:58:41,843::task::495::TaskManager.Task::(_debug) Task 20cf48b3-5b1d-4d09-b1c6-e1791a705aa1: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:58:41,844::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b4a6e753-51ac-4f7e-b813-1081230dabba`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:58:41,844::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:58:41,845::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:58:41,846::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`b4a6e753-51ac-4f7e-b813-1081230dabba`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:58:41,846::task::495::TaskManager.Task::(_debug) Task 20cf48b3-5b1d-4d09-b1c6-e1791a705aa1: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:58:41,847::task::495::TaskManager.Task::(_debug) Task 20cf48b3-5b1d-4d09-b1c6-e1791a705aa1: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:58:41,848::task::495::TaskManager.Task::(_debug) Task 20cf48b3-5b1d-4d09-b1c6-e1791a705aa1: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:58:41,848::task::495::TaskManager.Task::(_debug) Task 20cf48b3-5b1d-4d09-b1c6-e1791a705aa1: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:58:41,849::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:58:41,849::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:58:41,850::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:58:41,850::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:58:41,851::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:58:41,851::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:58:41,852::task::495::TaskManager.Task::(_debug) Task 20cf48b3-5b1d-4d09-b1c6-e1791a705aa1: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:58:41,853::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:58:42,131::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:58:42,686::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.076631 s, 13.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:58:44,701::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:58:45,221::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0789704 s, 13.0 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:58:47,237::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 04:58:47,448::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:58:47,451::task::495::TaskManager.Task::(_debug) Task 59dbb98a-e9ea-48ff-a4d8-849a71e70797: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:58:47,454::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1814ee65-241f-41e7-babb-c8ed51af7dba`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:58:47,456::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:58:47,458::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:58:47,458::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1814ee65-241f-41e7-babb-c8ed51af7dba`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:58:47,460::task::495::TaskManager.Task::(_debug) Task 59dbb98a-e9ea-48ff-a4d8-849a71e70797: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:58:47,461::task::495::TaskManager.Task::(_debug) Task 59dbb98a-e9ea-48ff-a4d8-849a71e70797: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:58:47,462::task::495::TaskManager.Task::(_debug) Task 59dbb98a-e9ea-48ff-a4d8-849a71e70797: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:58:47,462::task::495::TaskManager.Task::(_debug) Task 59dbb98a-e9ea-48ff-a4d8-849a71e70797: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:58:47,463::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:58:47,464::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:58:47,465::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:58:47,465::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:58:47,466::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:58:47,467::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:58:47,467::task::495::TaskManager.Task::(_debug) Task 59dbb98a-e9ea-48ff-a4d8-849a71e70797: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:58:47,468::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:58:47,760::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0892701 s, 11.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:58:49,776::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:58:50,288::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0840434 s, 12.2 MB/s\n'; <rc> = 0
>Thread-15917252::INFO::2013-11-20 04:58:51,515::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15917252::DEBUG::2013-11-20 04:58:51,516::task::495::TaskManager.Task::(_debug) Task 5219c3dd-bf10-41cd-8660-2f61e453c8fe: moving from state init -> state preparing
>Thread-15917252::DEBUG::2013-11-20 04:58:51,517::task::495::TaskManager.Task::(_debug) Task 5219c3dd-bf10-41cd-8660-2f61e453c8fe: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000444889068604', 'lastCheck': 1384923526.3639531, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00100612640381', 'lastCheck': 1384923465.961143, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00141000747681', 'lastCheck': 1384923465.9605141, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000581979751587', 'lastCheck': 1384923530.9939051, 'valid': True, 'code': 0}}
>Thread-15917252::DEBUG::2013-11-20 04:58:51,517::task::495::TaskManager.Task::(_debug) Task 5219c3dd-bf10-41cd-8660-2f61e453c8fe: moving from state preparing -> state finished
>Thread-15917252::DEBUG::2013-11-20 04:58:51,518::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917252::DEBUG::2013-11-20 04:58:51,519::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917252::DEBUG::2013-11-20 04:58:51,519::task::495::TaskManager.Task::(_debug) Task 5219c3dd-bf10-41cd-8660-2f61e453c8fe: ref 0 aborting False
>Thread-15917252::INFO::2013-11-20 04:58:51,520::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000444889068604', 'lastCheck': 1384923526.3639531, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00100612640381', 'lastCheck': 1384923465.961143, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00141000747681', 'lastCheck': 1384923465.9605141, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000581979751587', 'lastCheck': 1384923530.9939051, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:58:52,304::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 04:58:52,510::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:58:52,512::task::495::TaskManager.Task::(_debug) Task f1b6ee1b-70e0-4da8-a871-0dbd8e7092cd: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:58:52,514::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`dfdbbb66-cfb6-41f1-96e4-3d21dcec8724`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:58:52,515::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:58:52,516::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:58:52,517::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`dfdbbb66-cfb6-41f1-96e4-3d21dcec8724`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:58:52,518::task::495::TaskManager.Task::(_debug) Task f1b6ee1b-70e0-4da8-a871-0dbd8e7092cd: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:58:52,519::task::495::TaskManager.Task::(_debug) Task f1b6ee1b-70e0-4da8-a871-0dbd8e7092cd: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:58:52,519::task::495::TaskManager.Task::(_debug) Task f1b6ee1b-70e0-4da8-a871-0dbd8e7092cd: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:58:52,520::task::495::TaskManager.Task::(_debug) Task f1b6ee1b-70e0-4da8-a871-0dbd8e7092cd: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:58:52,521::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:58:52,521::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:58:52,522::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:58:52,523::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:58:52,524::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:58:52,524::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:58:52,525::task::495::TaskManager.Task::(_debug) Task f1b6ee1b-70e0-4da8-a871-0dbd8e7092cd: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:58:52,526::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Dummy-15912651::DEBUG::2013-11-20 04:58:52,851::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0810945 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:58:54,867::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:58:55,426::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0832928 s, 12.3 MB/s\n'; <rc> = 0
>Thread-42838::DEBUG::2013-11-20 04:58:56,498::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgck --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-42838::DEBUG::2013-11-20 04:58:57,258::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:58:57,438::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913546::INFO::2013-11-20 04:58:57,673::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15914519::INFO::2013-11-20 04:58:57,675::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15913546::DEBUG::2013-11-20 04:58:57,679::task::495::TaskManager.Task::(_debug) Task e49ae379-8698-4781-a498-8c61cb359953: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:58:57,680::task::495::TaskManager.Task::(_debug) Task 714c0743-4294-4664-9383-4b77ac8bf550: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:58:57,683::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d300d387-d2ad-4278-aee1-eb025d72eb81`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:58:57,692::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:58:57,693::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`74652dcd-ba5f-4610-9b79-097327a27411`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:58:57,694::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:58:57,695::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:58:57,696::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`d300d387-d2ad-4278-aee1-eb025d72eb81`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:58:57,697::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:58:57,698::task::495::TaskManager.Task::(_debug) Task e49ae379-8698-4781-a498-8c61cb359953: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15914519::DEBUG::2013-11-20 04:58:57,698::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`74652dcd-ba5f-4610-9b79-097327a27411`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:58:57,699::task::495::TaskManager.Task::(_debug) Task e49ae379-8698-4781-a498-8c61cb359953: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:58:57,701::task::495::TaskManager.Task::(_debug) Task 714c0743-4294-4664-9383-4b77ac8bf550: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913546::DEBUG::2013-11-20 04:58:57,701::task::495::TaskManager.Task::(_debug) Task e49ae379-8698-4781-a498-8c61cb359953: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:58:57,702::task::495::TaskManager.Task::(_debug) Task 714c0743-4294-4664-9383-4b77ac8bf550: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:58:57,702::task::495::TaskManager.Task::(_debug) Task e49ae379-8698-4781-a498-8c61cb359953: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:58:57,703::task::495::TaskManager.Task::(_debug) Task 714c0743-4294-4664-9383-4b77ac8bf550: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:58:57,704::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:58:57,705::task::495::TaskManager.Task::(_debug) Task 714c0743-4294-4664-9383-4b77ac8bf550: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:58:57,705::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:58:57,706::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:58:57,707::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15914519::DEBUG::2013-11-20 04:58:57,707::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:58:57,708::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15914519::DEBUG::2013-11-20 04:58:57,710::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913546::DEBUG::2013-11-20 04:58:57,710::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:58:57,711::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:58:57,712::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:58:57,712::task::495::TaskManager.Task::(_debug) Task e49ae379-8698-4781-a498-8c61cb359953: ref 0 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:58:57,713::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913546::INFO::2013-11-20 04:58:57,713::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:58:57,714::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913546::INFO::2013-11-20 04:58:57,715::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15913546::DEBUG::2013-11-20 04:58:57,716::task::495::TaskManager.Task::(_debug) Task 12d7ae4c-4f7d-4e2e-861f-4dc105bd83f8: moving from state init -> state preparing
>Thread-15914519::DEBUG::2013-11-20 04:58:57,716::task::495::TaskManager.Task::(_debug) Task 714c0743-4294-4664-9383-4b77ac8bf550: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:58:57,718::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`eaa81bfb-731d-4f00-91ea-deee003b0944`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::INFO::2013-11-20 04:58:57,718::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:58:57,719::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:58:57,721::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:58:57,721::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`eaa81bfb-731d-4f00-91ea-deee003b0944`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:58:57,722::task::495::TaskManager.Task::(_debug) Task 12d7ae4c-4f7d-4e2e-861f-4dc105bd83f8: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:58:57,723::task::495::TaskManager.Task::(_debug) Task 12d7ae4c-4f7d-4e2e-861f-4dc105bd83f8: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:58:57,724::task::495::TaskManager.Task::(_debug) Task 12d7ae4c-4f7d-4e2e-861f-4dc105bd83f8: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:58:57,724::task::495::TaskManager.Task::(_debug) Task 12d7ae4c-4f7d-4e2e-861f-4dc105bd83f8: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:58:57,725::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:58:57,726::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:58:57,726::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:58:57,727::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:58:57,727::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:58:57,728::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:58:57,729::task::495::TaskManager.Task::(_debug) Task 12d7ae4c-4f7d-4e2e-861f-4dc105bd83f8: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:58:57,729::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:58:57,730::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:58:57,731::task::495::TaskManager.Task::(_debug) Task 3bd471d6-00bb-4afa-af69-a45c95c72571: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:58:57,732::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`39465cfa-6f15-4199-b212-d3957a4ad7cc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:58:57,733::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:58:57,734::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:58:57,734::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`39465cfa-6f15-4199-b212-d3957a4ad7cc`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:58:57,735::task::495::TaskManager.Task::(_debug) Task 3bd471d6-00bb-4afa-af69-a45c95c72571: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:58:57,736::task::495::TaskManager.Task::(_debug) Task 3bd471d6-00bb-4afa-af69-a45c95c72571: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:58:57,737::task::495::TaskManager.Task::(_debug) Task 3bd471d6-00bb-4afa-af69-a45c95c72571: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:58:57,737::task::495::TaskManager.Task::(_debug) Task 3bd471d6-00bb-4afa-af69-a45c95c72571: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:58:57,738::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:58:57,739::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:58:57,740::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:58:57,741::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:58:57,746::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:58:57,746::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:58:57,747::task::495::TaskManager.Task::(_debug) Task 3bd471d6-00bb-4afa-af69-a45c95c72571: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:58:57,748::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:58:57,750::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:58:57,751::task::495::TaskManager.Task::(_debug) Task e865ddd4-b8b8-4666-8113-bb4fc293b1f9: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:58:57,753::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`81c16acc-26d9-4cbb-9bbf-d004a2ed46ab`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:58:57,753::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:58:57,754::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:58:57,755::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`81c16acc-26d9-4cbb-9bbf-d004a2ed46ab`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:58:57,756::task::495::TaskManager.Task::(_debug) Task e865ddd4-b8b8-4666-8113-bb4fc293b1f9: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:58:57,756::task::495::TaskManager.Task::(_debug) Task e865ddd4-b8b8-4666-8113-bb4fc293b1f9: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:58:57,757::task::495::TaskManager.Task::(_debug) Task e865ddd4-b8b8-4666-8113-bb4fc293b1f9: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:58:57,758::task::495::TaskManager.Task::(_debug) Task e865ddd4-b8b8-4666-8113-bb4fc293b1f9: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:58:57,758::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:58:57,759::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:58:57,760::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:58:57,760::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:58:57,761::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:58:57,761::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:58:57,762::task::495::TaskManager.Task::(_debug) Task e865ddd4-b8b8-4666-8113-bb4fc293b1f9: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:58:57,762::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:58:57,985::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.074927 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:59:00,001::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:59:00,580::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.091344 s, 11.2 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:59:02,597::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:59:03,163::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0798432 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:59:05,177::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:59:05,660::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0666449 s, 15.4 MB/s\n'; <rc> = 0
>Thread-15917259::DEBUG::2013-11-20 04:59:06,608::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917259::INFO::2013-11-20 04:59:06,610::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917259::DEBUG::2013-11-20 04:59:06,611::task::495::TaskManager.Task::(_debug) Task 12cc9110-e638-4fb8-a0b4-d0c969e73550: moving from state init -> state preparing
>Thread-15917259::DEBUG::2013-11-20 04:59:06,612::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917259::DEBUG::2013-11-20 04:59:06,612::task::495::TaskManager.Task::(_debug) Task 12cc9110-e638-4fb8-a0b4-d0c969e73550: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917259::DEBUG::2013-11-20 04:59:06,613::task::495::TaskManager.Task::(_debug) Task 12cc9110-e638-4fb8-a0b4-d0c969e73550: moving from state preparing -> state finished
>Thread-15917259::DEBUG::2013-11-20 04:59:06,614::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917259::DEBUG::2013-11-20 04:59:06,615::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917259::DEBUG::2013-11-20 04:59:06,615::task::495::TaskManager.Task::(_debug) Task 12cc9110-e638-4fb8-a0b4-d0c969e73550: ref 0 aborting False
>Thread-15917259::INFO::2013-11-20 04:59:06,616::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917260::DEBUG::2013-11-20 04:59:06,622::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917260::INFO::2013-11-20 04:59:06,623::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917260::DEBUG::2013-11-20 04:59:06,624::task::495::TaskManager.Task::(_debug) Task 23a2b463-be00-45cb-90bb-b1d847fdf61c: moving from state init -> state preparing
>Thread-15917260::DEBUG::2013-11-20 04:59:06,625::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`345d6cf2-b26a-4b74-8d3f-733f0f34a679`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917260::DEBUG::2013-11-20 04:59:06,626::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917260::DEBUG::2013-11-20 04:59:06,627::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917260::DEBUG::2013-11-20 04:59:06,627::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`345d6cf2-b26a-4b74-8d3f-733f0f34a679`::Granted request
>Thread-15917260::DEBUG::2013-11-20 04:59:06,628::task::495::TaskManager.Task::(_debug) Task 23a2b463-be00-45cb-90bb-b1d847fdf61c: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917260::DEBUG::2013-11-20 04:59:06,629::task::495::TaskManager.Task::(_debug) Task 23a2b463-be00-45cb-90bb-b1d847fdf61c: ref 1 aborting False
>Thread-15917260::WARNING::2013-11-20 04:59:06,632::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917260::DEBUG::2013-11-20 04:59:06,633::task::495::TaskManager.Task::(_debug) Task 23a2b463-be00-45cb-90bb-b1d847fdf61c: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917260::DEBUG::2013-11-20 04:59:06,634::task::495::TaskManager.Task::(_debug) Task 23a2b463-be00-45cb-90bb-b1d847fdf61c: moving from state preparing -> state finished
>Thread-15917260::DEBUG::2013-11-20 04:59:06,635::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917260::DEBUG::2013-11-20 04:59:06,635::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917260::DEBUG::2013-11-20 04:59:06,636::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917260::DEBUG::2013-11-20 04:59:06,637::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917260::DEBUG::2013-11-20 04:59:06,637::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917260::DEBUG::2013-11-20 04:59:06,638::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917260::DEBUG::2013-11-20 04:59:06,638::task::495::TaskManager.Task::(_debug) Task 23a2b463-be00-45cb-90bb-b1d847fdf61c: ref 0 aborting False
>Thread-15917260::INFO::2013-11-20 04:59:06,639::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:59:07,672::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:59:08,213::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0875227 s, 11.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:59:10,230::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914435::INFO::2013-11-20 04:59:10,461::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 04:59:10,462::task::495::TaskManager.Task::(_debug) Task d1cfcc65-e675-43b5-a2e1-bb281ee3fd99: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 04:59:10,465::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`45c78da3-853d-4402-9ffd-c802cf2496f9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 04:59:10,465::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 04:59:10,466::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914435::DEBUG::2013-11-20 04:59:10,467::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`45c78da3-853d-4402-9ffd-c802cf2496f9`::Granted request
>Thread-15914435::DEBUG::2013-11-20 04:59:10,468::task::495::TaskManager.Task::(_debug) Task d1cfcc65-e675-43b5-a2e1-bb281ee3fd99: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914435::DEBUG::2013-11-20 04:59:10,469::task::495::TaskManager.Task::(_debug) Task d1cfcc65-e675-43b5-a2e1-bb281ee3fd99: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 04:59:10,470::task::495::TaskManager.Task::(_debug) Task d1cfcc65-e675-43b5-a2e1-bb281ee3fd99: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15914435::DEBUG::2013-11-20 04:59:10,471::task::495::TaskManager.Task::(_debug) Task d1cfcc65-e675-43b5-a2e1-bb281ee3fd99: moving from state preparing -> state finished
>Thread-15914435::DEBUG::2013-11-20 04:59:10,472::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 04:59:10,473::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914435::DEBUG::2013-11-20 04:59:10,473::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914435::DEBUG::2013-11-20 04:59:10,474::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914435::DEBUG::2013-11-20 04:59:10,475::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914435::DEBUG::2013-11-20 04:59:10,475::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914435::DEBUG::2013-11-20 04:59:10,476::task::495::TaskManager.Task::(_debug) Task d1cfcc65-e675-43b5-a2e1-bb281ee3fd99: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 04:59:10,477::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Dummy-15912651::DEBUG::2013-11-20 04:59:10,724::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0666362 s, 15.4 MB/s\n'; <rc> = 0
>Thread-15917262::INFO::2013-11-20 04:59:11,952::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15917262::DEBUG::2013-11-20 04:59:11,954::task::495::TaskManager.Task::(_debug) Task ef14ff23-37af-4534-84f7-46cc5fb4ca10: moving from state init -> state preparing
>Thread-15917262::DEBUG::2013-11-20 04:59:11,954::task::495::TaskManager.Task::(_debug) Task ef14ff23-37af-4534-84f7-46cc5fb4ca10: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.0018630027771', 'lastCheck': 1384923547.874018, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00100612640381', 'lastCheck': 1384923465.961143, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00141000747681', 'lastCheck': 1384923465.9605141, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000490188598633', 'lastCheck': 1384923551.436218, 'valid': True, 'code': 0}}
>Thread-15917262::DEBUG::2013-11-20 04:59:11,955::task::495::TaskManager.Task::(_debug) Task ef14ff23-37af-4534-84f7-46cc5fb4ca10: moving from state preparing -> state finished
>Thread-15917262::DEBUG::2013-11-20 04:59:11,956::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917262::DEBUG::2013-11-20 04:59:11,956::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917262::DEBUG::2013-11-20 04:59:11,957::task::495::TaskManager.Task::(_debug) Task ef14ff23-37af-4534-84f7-46cc5fb4ca10: ref 0 aborting False
>Thread-15917262::INFO::2013-11-20 04:59:11,958::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.0018630027771', 'lastCheck': 1384923547.874018, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00100612640381', 'lastCheck': 1384923465.961143, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00141000747681', 'lastCheck': 1384923465.9605141, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000490188598633', 'lastCheck': 1384923551.436218, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:59:12,734::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:59:13,306::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0882596 s, 11.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:59:15,322::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:59:15,805::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0711539 s, 14.4 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:59:17,819::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:59:18,322::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0671368 s, 15.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:59:20,337::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913639::INFO::2013-11-20 04:59:20,578::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 04:59:20,579::task::495::TaskManager.Task::(_debug) Task 33ef2c47-07d8-42d8-919c-8bbea3ec28d0: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 04:59:20,582::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`eb5fdb2b-7e36-41b7-b580-bd8c82e06f0c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 04:59:20,583::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 04:59:20,585::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913639::DEBUG::2013-11-20 04:59:20,586::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`eb5fdb2b-7e36-41b7-b580-bd8c82e06f0c`::Granted request
>Thread-15913639::DEBUG::2013-11-20 04:59:20,589::task::495::TaskManager.Task::(_debug) Task 33ef2c47-07d8-42d8-919c-8bbea3ec28d0: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 04:59:20,589::task::495::TaskManager.Task::(_debug) Task 33ef2c47-07d8-42d8-919c-8bbea3ec28d0: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 04:59:20,590::task::495::TaskManager.Task::(_debug) Task 33ef2c47-07d8-42d8-919c-8bbea3ec28d0: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 04:59:20,591::task::495::TaskManager.Task::(_debug) Task 33ef2c47-07d8-42d8-919c-8bbea3ec28d0: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 04:59:20,592::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 04:59:20,593::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 04:59:20,594::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 04:59:20,595::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913639::DEBUG::2013-11-20 04:59:20,595::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913639::DEBUG::2013-11-20 04:59:20,596::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913639::DEBUG::2013-11-20 04:59:20,597::task::495::TaskManager.Task::(_debug) Task 33ef2c47-07d8-42d8-919c-8bbea3ec28d0: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 04:59:20,597::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Dummy-15912651::DEBUG::2013-11-20 04:59:20,911::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0817957 s, 12.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:59:22,927::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15915918::INFO::2013-11-20 04:59:23,171::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=e0628852-41b1-4535-8dda-ebc96cb00952 volUUID=cd4f9324-c91d-405b-983f-62afa4daa8ad)
>Thread-15915918::DEBUG::2013-11-20 04:59:23,184::task::495::TaskManager.Task::(_debug) Task 9c90e5d2-4f08-4229-8411-1e751f65e8fa: moving from state init -> state preparing
>Thread-15915918::DEBUG::2013-11-20 04:59:23,188::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4b0e2bf0-1dac-45ee-aeed-68e9f7b4e214`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915918::DEBUG::2013-11-20 04:59:23,190::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915918::DEBUG::2013-11-20 04:59:23,192::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15915918::DEBUG::2013-11-20 04:59:23,193::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4b0e2bf0-1dac-45ee-aeed-68e9f7b4e214`::Granted request
>Thread-15915918::DEBUG::2013-11-20 04:59:23,195::task::495::TaskManager.Task::(_debug) Task 9c90e5d2-4f08-4229-8411-1e751f65e8fa: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915918::DEBUG::2013-11-20 04:59:23,195::task::495::TaskManager.Task::(_debug) Task 9c90e5d2-4f08-4229-8411-1e751f65e8fa: ref 1 aborting False
>Thread-15915918::DEBUG::2013-11-20 04:59:23,196::task::495::TaskManager.Task::(_debug) Task 9c90e5d2-4f08-4229-8411-1e751f65e8fa: finished: {'truesize': '32346472448', 'apparentsize': '32346472448'}
>Thread-15915918::DEBUG::2013-11-20 04:59:23,198::task::495::TaskManager.Task::(_debug) Task 9c90e5d2-4f08-4229-8411-1e751f65e8fa: moving from state preparing -> state finished
>Thread-15915918::DEBUG::2013-11-20 04:59:23,199::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915918::DEBUG::2013-11-20 04:59:23,200::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915918::DEBUG::2013-11-20 04:59:23,201::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915918::DEBUG::2013-11-20 04:59:23,203::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15915918::DEBUG::2013-11-20 04:59:23,204::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915918::DEBUG::2013-11-20 04:59:23,205::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915918::DEBUG::2013-11-20 04:59:23,206::task::495::TaskManager.Task::(_debug) Task 9c90e5d2-4f08-4229-8411-1e751f65e8fa: ref 0 aborting False
>Thread-15915918::INFO::2013-11-20 04:59:23,207::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '32346472448', 'apparentsize': '32346472448'}
>Dummy-15912651::DEBUG::2013-11-20 04:59:23,450::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0746331 s, 13.7 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:59:25,465::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15916984::INFO::2013-11-20 04:59:25,677::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=240c0593-fe33-4f8b-a029-097379dbcb85 volUUID=f715faaa-081a-4598-b125-80a2d12af554)
>Thread-15916984::DEBUG::2013-11-20 04:59:25,680::task::495::TaskManager.Task::(_debug) Task 87d19982-e81c-4c67-88c7-80f26bb03d63: moving from state init -> state preparing
>Thread-15916984::DEBUG::2013-11-20 04:59:25,684::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3e34281d-8197-4017-b857-30318ee2574a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15916984::DEBUG::2013-11-20 04:59:25,689::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15916984::DEBUG::2013-11-20 04:59:25,690::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15916984::DEBUG::2013-11-20 04:59:25,691::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3e34281d-8197-4017-b857-30318ee2574a`::Granted request
>Thread-15916984::DEBUG::2013-11-20 04:59:25,692::task::495::TaskManager.Task::(_debug) Task 87d19982-e81c-4c67-88c7-80f26bb03d63: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15916984::DEBUG::2013-11-20 04:59:25,693::task::495::TaskManager.Task::(_debug) Task 87d19982-e81c-4c67-88c7-80f26bb03d63: ref 1 aborting False
>Thread-15916984::DEBUG::2013-11-20 04:59:25,696::task::495::TaskManager.Task::(_debug) Task 87d19982-e81c-4c67-88c7-80f26bb03d63: finished: {'truesize': '75296145408', 'apparentsize': '75296145408'}
>Thread-15916984::DEBUG::2013-11-20 04:59:25,697::task::495::TaskManager.Task::(_debug) Task 87d19982-e81c-4c67-88c7-80f26bb03d63: moving from state preparing -> state finished
>Thread-15916984::DEBUG::2013-11-20 04:59:25,698::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15916984::DEBUG::2013-11-20 04:59:25,698::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15916984::DEBUG::2013-11-20 04:59:25,699::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15916984::DEBUG::2013-11-20 04:59:25,700::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15916984::DEBUG::2013-11-20 04:59:25,705::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15916984::DEBUG::2013-11-20 04:59:25,710::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15916984::DEBUG::2013-11-20 04:59:25,711::task::495::TaskManager.Task::(_debug) Task 87d19982-e81c-4c67-88c7-80f26bb03d63: ref 0 aborting False
>Thread-15916984::INFO::2013-11-20 04:59:25,711::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '75296145408', 'apparentsize': '75296145408'}
>Dummy-15912651::DEBUG::2013-11-20 04:59:26,010::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.08665 s, 11.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:59:28,031::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913658::INFO::2013-11-20 04:59:28,260::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b74dd33b-748d-48df-83f2-ebcc3523e117 volUUID=8506c15b-61ae-4830-84ed-c830955c25a3)
>Thread-15915863::INFO::2013-11-20 04:59:28,261::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4ba94a2-ce52-412c-ab9b-7b7a7e3fa52e volUUID=dfd07c26-472f-40dc-bff0-ed31eae9cb57)
>Thread-15913658::DEBUG::2013-11-20 04:59:28,268::task::495::TaskManager.Task::(_debug) Task c70654c0-6b10-42dd-88a9-99b11e7ce7a7: moving from state init -> state preparing
>Thread-15915863::DEBUG::2013-11-20 04:59:28,270::task::495::TaskManager.Task::(_debug) Task 515ef25e-8009-42d6-ab1b-0122ff9687f6: moving from state init -> state preparing
>Thread-15913658::DEBUG::2013-11-20 04:59:28,274::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9ff50e90-c2de-46b3-a22d-20c75585bfc7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15915863::DEBUG::2013-11-20 04:59:28,277::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a2ae3481-bdc1-4850-a695-fb07041276df`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913658::DEBUG::2013-11-20 04:59:28,279::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15915863::DEBUG::2013-11-20 04:59:28,279::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913658::DEBUG::2013-11-20 04:59:28,280::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913658::DEBUG::2013-11-20 04:59:28,281::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`9ff50e90-c2de-46b3-a22d-20c75585bfc7`::Granted request
>Thread-15915863::DEBUG::2013-11-20 04:59:28,282::resourceManager::481::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' found in shared state and queue is empty, Joining current shared lock (2 active users)
>Thread-15913658::DEBUG::2013-11-20 04:59:28,282::task::495::TaskManager.Task::(_debug) Task c70654c0-6b10-42dd-88a9-99b11e7ce7a7: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15915863::DEBUG::2013-11-20 04:59:28,283::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`a2ae3481-bdc1-4850-a695-fb07041276df`::Granted request
>Thread-15913658::DEBUG::2013-11-20 04:59:28,283::task::495::TaskManager.Task::(_debug) Task c70654c0-6b10-42dd-88a9-99b11e7ce7a7: ref 1 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:59:28,284::task::495::TaskManager.Task::(_debug) Task 515ef25e-8009-42d6-ab1b-0122ff9687f6: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913658::DEBUG::2013-11-20 04:59:28,285::task::495::TaskManager.Task::(_debug) Task c70654c0-6b10-42dd-88a9-99b11e7ce7a7: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15915863::DEBUG::2013-11-20 04:59:28,285::task::495::TaskManager.Task::(_debug) Task 515ef25e-8009-42d6-ab1b-0122ff9687f6: ref 1 aborting False
>Thread-15913658::DEBUG::2013-11-20 04:59:28,285::task::495::TaskManager.Task::(_debug) Task c70654c0-6b10-42dd-88a9-99b11e7ce7a7: moving from state preparing -> state finished
>Thread-15915863::DEBUG::2013-11-20 04:59:28,286::task::495::TaskManager.Task::(_debug) Task 515ef25e-8009-42d6-ab1b-0122ff9687f6: finished: {'truesize': '107776835584', 'apparentsize': '107776835584'}
>Thread-15913658::DEBUG::2013-11-20 04:59:28,286::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15915863::DEBUG::2013-11-20 04:59:28,287::task::495::TaskManager.Task::(_debug) Task 515ef25e-8009-42d6-ab1b-0122ff9687f6: moving from state preparing -> state finished
>Thread-15913658::DEBUG::2013-11-20 04:59:28,287::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15915863::DEBUG::2013-11-20 04:59:28,288::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913658::DEBUG::2013-11-20 04:59:28,289::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15915863::DEBUG::2013-11-20 04:59:28,289::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913658::DEBUG::2013-11-20 04:59:28,290::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (1 active users)
>Thread-15915863::DEBUG::2013-11-20 04:59:28,291::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913658::DEBUG::2013-11-20 04:59:28,292::task::495::TaskManager.Task::(_debug) Task c70654c0-6b10-42dd-88a9-99b11e7ce7a7: ref 0 aborting False
>Thread-15915863::DEBUG::2013-11-20 04:59:28,304::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913658::INFO::2013-11-20 04:59:28,304::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15915863::DEBUG::2013-11-20 04:59:28,305::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15915863::DEBUG::2013-11-20 04:59:28,306::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15915863::DEBUG::2013-11-20 04:59:28,307::task::495::TaskManager.Task::(_debug) Task 515ef25e-8009-42d6-ab1b-0122ff9687f6: ref 0 aborting False
>Thread-15915863::INFO::2013-11-20 04:59:28,308::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107776835584', 'apparentsize': '107776835584'}
>Dummy-15912651::DEBUG::2013-11-20 04:59:28,585::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0840062 s, 12.2 MB/s\n'; <rc> = 0
>Thread-15917264::DEBUG::2013-11-20 04:59:30,561::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917264::INFO::2013-11-20 04:59:30,563::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917264::DEBUG::2013-11-20 04:59:30,564::task::495::TaskManager.Task::(_debug) Task abbf310a-7931-47d8-bb72-21610b452b56: moving from state init -> state preparing
>Thread-15917264::DEBUG::2013-11-20 04:59:30,565::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917264::DEBUG::2013-11-20 04:59:30,565::task::495::TaskManager.Task::(_debug) Task abbf310a-7931-47d8-bb72-21610b452b56: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917264::DEBUG::2013-11-20 04:59:30,566::task::495::TaskManager.Task::(_debug) Task abbf310a-7931-47d8-bb72-21610b452b56: moving from state preparing -> state finished
>Thread-15917264::DEBUG::2013-11-20 04:59:30,566::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917264::DEBUG::2013-11-20 04:59:30,567::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917264::DEBUG::2013-11-20 04:59:30,568::task::495::TaskManager.Task::(_debug) Task abbf310a-7931-47d8-bb72-21610b452b56: ref 0 aborting False
>Thread-15917264::INFO::2013-11-20 04:59:30,568::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917265::DEBUG::2013-11-20 04:59:30,573::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917265::INFO::2013-11-20 04:59:30,574::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917265::DEBUG::2013-11-20 04:59:30,575::task::495::TaskManager.Task::(_debug) Task 00bba451-6f5b-45ac-8c95-c8c0d23c5d38: moving from state init -> state preparing
>Thread-15917265::DEBUG::2013-11-20 04:59:30,576::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`853b47e0-10cd-4696-908d-be0eee34a272`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917265::DEBUG::2013-11-20 04:59:30,577::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917265::DEBUG::2013-11-20 04:59:30,578::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917265::DEBUG::2013-11-20 04:59:30,579::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`853b47e0-10cd-4696-908d-be0eee34a272`::Granted request
>Thread-15917265::DEBUG::2013-11-20 04:59:30,580::task::495::TaskManager.Task::(_debug) Task 00bba451-6f5b-45ac-8c95-c8c0d23c5d38: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917265::DEBUG::2013-11-20 04:59:30,580::task::495::TaskManager.Task::(_debug) Task 00bba451-6f5b-45ac-8c95-c8c0d23c5d38: ref 1 aborting False
>Thread-15917265::WARNING::2013-11-20 04:59:30,582::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917265::DEBUG::2013-11-20 04:59:30,584::task::495::TaskManager.Task::(_debug) Task 00bba451-6f5b-45ac-8c95-c8c0d23c5d38: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917265::DEBUG::2013-11-20 04:59:30,585::task::495::TaskManager.Task::(_debug) Task 00bba451-6f5b-45ac-8c95-c8c0d23c5d38: moving from state preparing -> state finished
>Thread-15917265::DEBUG::2013-11-20 04:59:30,585::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917265::DEBUG::2013-11-20 04:59:30,586::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917265::DEBUG::2013-11-20 04:59:30,587::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917265::DEBUG::2013-11-20 04:59:30,587::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917265::DEBUG::2013-11-20 04:59:30,588::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917265::DEBUG::2013-11-20 04:59:30,589::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917265::DEBUG::2013-11-20 04:59:30,589::task::495::TaskManager.Task::(_debug) Task 00bba451-6f5b-45ac-8c95-c8c0d23c5d38: ref 0 aborting False
>Thread-15917265::INFO::2013-11-20 04:59:30,590::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:59:30,600::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:59:31,174::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0882059 s, 11.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:59:33,187::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:59:33,771::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0796899 s, 12.8 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:59:35,788::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:59:36,334::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.081682 s, 12.5 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:59:38,349::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15912392::INFO::2013-11-20 04:59:38,529::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d8be6e76-2f7f-4b56-9ada-c3fd543a9794 volUUID=038d75ec-9421-4d79-9865-0dbebd5271d9)
>Thread-15912392::DEBUG::2013-11-20 04:59:38,531::task::495::TaskManager.Task::(_debug) Task 757433f5-92f2-4a48-9401-38e39e5dcf3b: moving from state init -> state preparing
>Thread-15912392::DEBUG::2013-11-20 04:59:38,536::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6fb0eff1-b969-4a32-8b96-b9e18c859be7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15912392::DEBUG::2013-11-20 04:59:38,537::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15912392::DEBUG::2013-11-20 04:59:38,539::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15912392::DEBUG::2013-11-20 04:59:38,540::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`6fb0eff1-b969-4a32-8b96-b9e18c859be7`::Granted request
>Thread-15912392::DEBUG::2013-11-20 04:59:38,541::task::495::TaskManager.Task::(_debug) Task 757433f5-92f2-4a48-9401-38e39e5dcf3b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15912392::DEBUG::2013-11-20 04:59:38,542::task::495::TaskManager.Task::(_debug) Task 757433f5-92f2-4a48-9401-38e39e5dcf3b: ref 1 aborting False
>Thread-15912392::DEBUG::2013-11-20 04:59:38,543::task::495::TaskManager.Task::(_debug) Task 757433f5-92f2-4a48-9401-38e39e5dcf3b: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15912392::DEBUG::2013-11-20 04:59:38,544::task::495::TaskManager.Task::(_debug) Task 757433f5-92f2-4a48-9401-38e39e5dcf3b: moving from state preparing -> state finished
>Thread-15912392::DEBUG::2013-11-20 04:59:38,545::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15912392::DEBUG::2013-11-20 04:59:38,545::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15912392::DEBUG::2013-11-20 04:59:38,546::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15912392::DEBUG::2013-11-20 04:59:38,547::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15912392::DEBUG::2013-11-20 04:59:38,547::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15912392::DEBUG::2013-11-20 04:59:38,548::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15912392::DEBUG::2013-11-20 04:59:38,549::task::495::TaskManager.Task::(_debug) Task 757433f5-92f2-4a48-9401-38e39e5dcf3b: ref 0 aborting False
>Thread-15912392::INFO::2013-11-20 04:59:38,549::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Dummy-15912651::DEBUG::2013-11-20 04:59:38,755::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0508465 s, 20.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:59:40,768::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:59:41,321::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0872764 s, 11.7 MB/s\n'; <rc> = 0
>Thread-15913518::INFO::2013-11-20 04:59:43,017::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=b0980185-7de1-4e3b-bd76-4495169ab08b volUUID=0ecd1db7-48b0-4663-a038-e40e891b9424)
>Thread-15913518::DEBUG::2013-11-20 04:59:43,018::task::495::TaskManager.Task::(_debug) Task d8edbf37-045f-4fad-b9c9-0ed85714ef95: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:59:43,020::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3e627809-b855-4672-b4d2-60242a663415`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:59:43,020::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:59:43,022::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:59:43,022::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`3e627809-b855-4672-b4d2-60242a663415`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:59:43,023::task::495::TaskManager.Task::(_debug) Task d8edbf37-045f-4fad-b9c9-0ed85714ef95: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:59:43,024::task::495::TaskManager.Task::(_debug) Task d8edbf37-045f-4fad-b9c9-0ed85714ef95: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:59:43,025::task::495::TaskManager.Task::(_debug) Task d8edbf37-045f-4fad-b9c9-0ed85714ef95: finished: {'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::DEBUG::2013-11-20 04:59:43,025::task::495::TaskManager.Task::(_debug) Task d8edbf37-045f-4fad-b9c9-0ed85714ef95: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:59:43,026::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:59:43,027::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:59:43,027::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:59:43,028::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:59:43,028::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:59:43,029::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:59:43,030::task::495::TaskManager.Task::(_debug) Task d8edbf37-045f-4fad-b9c9-0ed85714ef95: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:59:43,031::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '30064771072', 'apparentsize': '30064771072'}
>Thread-15913518::INFO::2013-11-20 04:59:43,032::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=0b89347c-4654-42bb-a8d7-1a627130b7bd volUUID=6d82661d-9502-4cb3-a0a0-91cd82dd2a80)
>Thread-15913518::DEBUG::2013-11-20 04:59:43,032::task::495::TaskManager.Task::(_debug) Task 58ae86e0-de66-4278-8c11-176222ab153f: moving from state init -> state preparing
>Thread-15913518::DEBUG::2013-11-20 04:59:43,034::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d3cf1c23-29b6-4db7-917d-5fc35594e025`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913518::DEBUG::2013-11-20 04:59:43,034::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913518::DEBUG::2013-11-20 04:59:43,035::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913518::DEBUG::2013-11-20 04:59:43,035::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`d3cf1c23-29b6-4db7-917d-5fc35594e025`::Granted request
>Thread-15913518::DEBUG::2013-11-20 04:59:43,036::task::495::TaskManager.Task::(_debug) Task 58ae86e0-de66-4278-8c11-176222ab153f: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913518::DEBUG::2013-11-20 04:59:43,037::task::495::TaskManager.Task::(_debug) Task 58ae86e0-de66-4278-8c11-176222ab153f: ref 1 aborting False
>Thread-15913518::DEBUG::2013-11-20 04:59:43,038::task::495::TaskManager.Task::(_debug) Task 58ae86e0-de66-4278-8c11-176222ab153f: finished: {'truesize': '826781204480', 'apparentsize': '826781204480'}
>Thread-15913518::DEBUG::2013-11-20 04:59:43,038::task::495::TaskManager.Task::(_debug) Task 58ae86e0-de66-4278-8c11-176222ab153f: moving from state preparing -> state finished
>Thread-15913518::DEBUG::2013-11-20 04:59:43,039::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913518::DEBUG::2013-11-20 04:59:43,040::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913518::DEBUG::2013-11-20 04:59:43,040::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913518::DEBUG::2013-11-20 04:59:43,041::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913518::DEBUG::2013-11-20 04:59:43,041::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913518::DEBUG::2013-11-20 04:59:43,042::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913518::DEBUG::2013-11-20 04:59:43,043::task::495::TaskManager.Task::(_debug) Task 58ae86e0-de66-4278-8c11-176222ab153f: ref 0 aborting False
>Thread-15913518::INFO::2013-11-20 04:59:43,043::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '826781204480', 'apparentsize': '826781204480'}
>Dummy-15912651::DEBUG::2013-11-20 04:59:43,335::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15917269::DEBUG::2013-11-20 04:59:43,569::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917269::INFO::2013-11-20 04:59:43,573::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917269::DEBUG::2013-11-20 04:59:43,575::task::495::TaskManager.Task::(_debug) Task 62eeb942-0a0f-42e8-9e6a-ce9188fb9cb0: moving from state init -> state preparing
>Thread-15917269::DEBUG::2013-11-20 04:59:43,578::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917269::DEBUG::2013-11-20 04:59:43,580::task::495::TaskManager.Task::(_debug) Task 62eeb942-0a0f-42e8-9e6a-ce9188fb9cb0: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917269::DEBUG::2013-11-20 04:59:43,583::task::495::TaskManager.Task::(_debug) Task 62eeb942-0a0f-42e8-9e6a-ce9188fb9cb0: moving from state preparing -> state finished
>Thread-15917269::DEBUG::2013-11-20 04:59:43,584::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917269::DEBUG::2013-11-20 04:59:43,585::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917269::DEBUG::2013-11-20 04:59:43,585::task::495::TaskManager.Task::(_debug) Task 62eeb942-0a0f-42e8-9e6a-ce9188fb9cb0: ref 0 aborting False
>Thread-15917269::INFO::2013-11-20 04:59:43,586::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917270::DEBUG::2013-11-20 04:59:43,594::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917270::INFO::2013-11-20 04:59:43,595::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917270::DEBUG::2013-11-20 04:59:43,596::task::495::TaskManager.Task::(_debug) Task c63ca780-6a8b-46fe-aa92-92553fd46df3: moving from state init -> state preparing
>Thread-15917270::DEBUG::2013-11-20 04:59:43,598::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`261f0275-017f-4894-8596-764d7be57b17`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917270::DEBUG::2013-11-20 04:59:43,599::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917270::DEBUG::2013-11-20 04:59:43,600::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917270::DEBUG::2013-11-20 04:59:43,601::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`261f0275-017f-4894-8596-764d7be57b17`::Granted request
>Thread-15917270::DEBUG::2013-11-20 04:59:43,602::task::495::TaskManager.Task::(_debug) Task c63ca780-6a8b-46fe-aa92-92553fd46df3: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917270::DEBUG::2013-11-20 04:59:43,603::task::495::TaskManager.Task::(_debug) Task c63ca780-6a8b-46fe-aa92-92553fd46df3: ref 1 aborting False
>Thread-15917270::WARNING::2013-11-20 04:59:43,606::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917270::DEBUG::2013-11-20 04:59:43,607::task::495::TaskManager.Task::(_debug) Task c63ca780-6a8b-46fe-aa92-92553fd46df3: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917270::DEBUG::2013-11-20 04:59:43,608::task::495::TaskManager.Task::(_debug) Task c63ca780-6a8b-46fe-aa92-92553fd46df3: moving from state preparing -> state finished
>Thread-15917270::DEBUG::2013-11-20 04:59:43,609::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917270::DEBUG::2013-11-20 04:59:43,609::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917270::DEBUG::2013-11-20 04:59:43,610::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917270::DEBUG::2013-11-20 04:59:43,611::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917270::DEBUG::2013-11-20 04:59:43,612::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917270::DEBUG::2013-11-20 04:59:43,612::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917270::DEBUG::2013-11-20 04:59:43,613::task::495::TaskManager.Task::(_debug) Task c63ca780-6a8b-46fe-aa92-92553fd46df3: ref 0 aborting False
>Thread-15917270::INFO::2013-11-20 04:59:43,614::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:59:43,887::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0769383 s, 13.3 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:59:45,902::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:59:46,437::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0724763 s, 14.1 MB/s\n'; <rc> = 0
>Thread-15917272::INFO::2013-11-20 04:59:47,861::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15917272::DEBUG::2013-11-20 04:59:47,862::task::495::TaskManager.Task::(_debug) Task 4956478f-9042-4b38-8a9c-6ea66af77d08: moving from state init -> state preparing
>Thread-15917272::DEBUG::2013-11-20 04:59:47,863::task::495::TaskManager.Task::(_debug) Task 4956478f-9042-4b38-8a9c-6ea66af77d08: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000967025756836', 'lastCheck': 1384923578.5338459, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00100612640381', 'lastCheck': 1384923465.961143, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00141000747681', 'lastCheck': 1384923465.9605141, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000463008880615', 'lastCheck': 1384923581.9983211, 'valid': True, 'code': 0}}
>Thread-15917272::DEBUG::2013-11-20 04:59:47,863::task::495::TaskManager.Task::(_debug) Task 4956478f-9042-4b38-8a9c-6ea66af77d08: moving from state preparing -> state finished
>Thread-15917272::DEBUG::2013-11-20 04:59:47,864::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917272::DEBUG::2013-11-20 04:59:47,865::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917272::DEBUG::2013-11-20 04:59:47,865::task::495::TaskManager.Task::(_debug) Task 4956478f-9042-4b38-8a9c-6ea66af77d08: ref 0 aborting False
>Thread-15917272::INFO::2013-11-20 04:59:47,866::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000967025756836', 'lastCheck': 1384923578.5338459, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00100612640381', 'lastCheck': 1384923465.961143, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00141000747681', 'lastCheck': 1384923465.9605141, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000463008880615', 'lastCheck': 1384923581.9983211, 'valid': True, 'code': 0}}
>Dummy-15912651::DEBUG::2013-11-20 04:59:48,452::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914453::INFO::2013-11-20 04:59:48,683::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5f930ef3-763f-4294-a62a-6d52df220f8c volUUID=b9cffdbe-152e-41ce-ae5d-08db7c08454e)
>Thread-15914453::DEBUG::2013-11-20 04:59:48,689::task::495::TaskManager.Task::(_debug) Task 4ce05b85-056d-4e3d-b88d-bd0dd44083a4: moving from state init -> state preparing
>Thread-15914453::DEBUG::2013-11-20 04:59:48,695::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`cb11bef6-3aee-4cbd-ab6d-234ac21c4e97`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914453::DEBUG::2013-11-20 04:59:48,698::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914453::DEBUG::2013-11-20 04:59:48,700::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15914453::DEBUG::2013-11-20 04:59:48,701::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`cb11bef6-3aee-4cbd-ab6d-234ac21c4e97`::Granted request
>Thread-15914453::DEBUG::2013-11-20 04:59:48,702::task::495::TaskManager.Task::(_debug) Task 4ce05b85-056d-4e3d-b88d-bd0dd44083a4: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15914453::DEBUG::2013-11-20 04:59:48,703::task::495::TaskManager.Task::(_debug) Task 4ce05b85-056d-4e3d-b88d-bd0dd44083a4: ref 1 aborting False
>Thread-15914453::DEBUG::2013-11-20 04:59:48,704::task::495::TaskManager.Task::(_debug) Task 4ce05b85-056d-4e3d-b88d-bd0dd44083a4: finished: {'truesize': '53821308928', 'apparentsize': '53821308928'}
>Thread-15914453::DEBUG::2013-11-20 04:59:48,705::task::495::TaskManager.Task::(_debug) Task 4ce05b85-056d-4e3d-b88d-bd0dd44083a4: moving from state preparing -> state finished
>Thread-15914453::DEBUG::2013-11-20 04:59:48,706::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914453::DEBUG::2013-11-20 04:59:48,706::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914453::DEBUG::2013-11-20 04:59:48,707::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15914453::DEBUG::2013-11-20 04:59:48,708::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15914453::DEBUG::2013-11-20 04:59:48,708::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15914453::DEBUG::2013-11-20 04:59:48,709::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15914453::DEBUG::2013-11-20 04:59:48,709::task::495::TaskManager.Task::(_debug) Task 4ce05b85-056d-4e3d-b88d-bd0dd44083a4: ref 0 aborting False
>Thread-15914453::INFO::2013-11-20 04:59:48,710::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '53821308928', 'apparentsize': '53821308928'}
>Dummy-15912651::DEBUG::2013-11-20 04:59:49,045::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0815898 s, 12.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:59:51,059::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:59:51,635::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0783911 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:59:53,657::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15913695::INFO::2013-11-20 04:59:53,877::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=cc863eeb-33db-4227-b4eb-9f4347299cf3 volUUID=ff5e3845-0d0d-4037-a0de-a3f584a4aef5)
>Thread-15913695::DEBUG::2013-11-20 04:59:53,879::task::495::TaskManager.Task::(_debug) Task 5aef7eee-31d6-45a8-ada0-85a0467fe7f9: moving from state init -> state preparing
>Thread-15913695::DEBUG::2013-11-20 04:59:53,883::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ef8dc14a-cccd-4482-9159-ca7083cf3857`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913695::DEBUG::2013-11-20 04:59:53,884::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913695::DEBUG::2013-11-20 04:59:53,885::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913695::DEBUG::2013-11-20 04:59:53,886::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`ef8dc14a-cccd-4482-9159-ca7083cf3857`::Granted request
>Thread-15913695::DEBUG::2013-11-20 04:59:53,888::task::495::TaskManager.Task::(_debug) Task 5aef7eee-31d6-45a8-ada0-85a0467fe7f9: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913695::DEBUG::2013-11-20 04:59:53,888::task::495::TaskManager.Task::(_debug) Task 5aef7eee-31d6-45a8-ada0-85a0467fe7f9: ref 1 aborting False
>Thread-15913695::DEBUG::2013-11-20 04:59:53,889::task::495::TaskManager.Task::(_debug) Task 5aef7eee-31d6-45a8-ada0-85a0467fe7f9: finished: {'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15913695::DEBUG::2013-11-20 04:59:53,890::task::495::TaskManager.Task::(_debug) Task 5aef7eee-31d6-45a8-ada0-85a0467fe7f9: moving from state preparing -> state finished
>Thread-15913695::DEBUG::2013-11-20 04:59:53,891::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913695::DEBUG::2013-11-20 04:59:53,892::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913695::DEBUG::2013-11-20 04:59:53,892::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913695::DEBUG::2013-11-20 04:59:53,893::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913695::DEBUG::2013-11-20 04:59:53,894::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913695::DEBUG::2013-11-20 04:59:53,894::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913695::DEBUG::2013-11-20 04:59:53,895::task::495::TaskManager.Task::(_debug) Task 5aef7eee-31d6-45a8-ada0-85a0467fe7f9: ref 0 aborting False
>Thread-15913695::INFO::2013-11-20 04:59:53,896::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '20401094656', 'apparentsize': '20401094656'}
>Thread-15917275::DEBUG::2013-11-20 04:59:53,965::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917275::INFO::2013-11-20 04:59:53,967::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917275::DEBUG::2013-11-20 04:59:53,968::task::495::TaskManager.Task::(_debug) Task 99bd97a9-8aed-47de-90ef-d9d8b9554c4d: moving from state init -> state preparing
>Thread-15917275::DEBUG::2013-11-20 04:59:53,969::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917275::DEBUG::2013-11-20 04:59:53,969::task::495::TaskManager.Task::(_debug) Task 99bd97a9-8aed-47de-90ef-d9d8b9554c4d: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917275::DEBUG::2013-11-20 04:59:53,970::task::495::TaskManager.Task::(_debug) Task 99bd97a9-8aed-47de-90ef-d9d8b9554c4d: moving from state preparing -> state finished
>Thread-15917275::DEBUG::2013-11-20 04:59:53,971::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917275::DEBUG::2013-11-20 04:59:53,972::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917275::DEBUG::2013-11-20 04:59:53,972::task::495::TaskManager.Task::(_debug) Task 99bd97a9-8aed-47de-90ef-d9d8b9554c4d: ref 0 aborting False
>Thread-15917275::INFO::2013-11-20 04:59:53,973::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917276::DEBUG::2013-11-20 04:59:53,979::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917276::INFO::2013-11-20 04:59:53,980::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917276::DEBUG::2013-11-20 04:59:53,981::task::495::TaskManager.Task::(_debug) Task e94efea7-8657-4392-bf7b-0d241b920d8b: moving from state init -> state preparing
>Thread-15917276::DEBUG::2013-11-20 04:59:53,982::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`34358711-174c-442a-8a9c-f4cc072eb16d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917276::DEBUG::2013-11-20 04:59:53,983::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917276::DEBUG::2013-11-20 04:59:53,984::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917276::DEBUG::2013-11-20 04:59:53,984::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`34358711-174c-442a-8a9c-f4cc072eb16d`::Granted request
>Thread-15917276::DEBUG::2013-11-20 04:59:53,986::task::495::TaskManager.Task::(_debug) Task e94efea7-8657-4392-bf7b-0d241b920d8b: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917276::DEBUG::2013-11-20 04:59:53,986::task::495::TaskManager.Task::(_debug) Task e94efea7-8657-4392-bf7b-0d241b920d8b: ref 1 aborting False
>Thread-15917276::WARNING::2013-11-20 04:59:53,989::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917276::DEBUG::2013-11-20 04:59:53,990::task::495::TaskManager.Task::(_debug) Task e94efea7-8657-4392-bf7b-0d241b920d8b: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917276::DEBUG::2013-11-20 04:59:53,991::task::495::TaskManager.Task::(_debug) Task e94efea7-8657-4392-bf7b-0d241b920d8b: moving from state preparing -> state finished
>Thread-15917276::DEBUG::2013-11-20 04:59:53,992::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917276::DEBUG::2013-11-20 04:59:53,993::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917276::DEBUG::2013-11-20 04:59:53,993::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917276::DEBUG::2013-11-20 04:59:53,994::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917276::DEBUG::2013-11-20 04:59:53,995::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917276::DEBUG::2013-11-20 04:59:53,995::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917276::DEBUG::2013-11-20 04:59:53,996::task::495::TaskManager.Task::(_debug) Task e94efea7-8657-4392-bf7b-0d241b920d8b: ref 0 aborting False
>Thread-15917276::INFO::2013-11-20 04:59:53,997::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '115018301440', 'alerts': [], 'disktotal': '240964861952'}}}
>Dummy-15912651::DEBUG::2013-11-20 04:59:54,193::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0782611 s, 13.1 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 04:59:56,205::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 04:59:56,756::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0746973 s, 13.7 MB/s\n'; <rc> = 0
>Thread-15917279::DEBUG::2013-11-20 04:59:58,688::misc::1156::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Emitting event
>Thread-15917279::DEBUG::2013-11-20 04:59:58,690::misc::1166::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Calling registered method `run`
>Thread-15917279::DEBUG::2013-11-20 04:59:58,691::misc::1174::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Event emitted
>Dummy-15912651::DEBUG::2013-11-20 04:59:58,772::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15914519::INFO::2013-11-20 04:59:58,987::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=5af747d8-95b1-4b7e-af53-cb77e4fcc896 volUUID=8d8f8dca-fbb0-4f15-9a32-28fc0176bd7f)
>Thread-15913546::INFO::2013-11-20 04:59:58,988::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=c4460480-32d3-469f-b0f4-c03bb6d74e41 volUUID=e6984064-e09e-4c0c-8b51-a8a1711994cf)
>Thread-15914519::DEBUG::2013-11-20 04:59:58,994::task::495::TaskManager.Task::(_debug) Task a810492f-c703-4da1-a7a5-a4362633260b: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:59:58,996::task::495::TaskManager.Task::(_debug) Task 07804611-5103-4d6f-96aa-61711be5a184: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:59:59,007::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`00d15ac0-751a-45a3-8366-75bab3441bab`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914519::DEBUG::2013-11-20 04:59:59,007::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0adea8eb-a8d1-46d9-b0d2-51f12f4b5155`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:59:59,009::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15914519::DEBUG::2013-11-20 04:59:59,009::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:59:59,011::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:59:59,012::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`00d15ac0-751a-45a3-8366-75bab3441bab`::Granted request
>Thread-15914519::DEBUG::2013-11-20 04:59:59,013::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:59:59,014::task::495::TaskManager.Task::(_debug) Task 07804611-5103-4d6f-96aa-61711be5a184: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15914519::DEBUG::2013-11-20 04:59:59,014::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`0adea8eb-a8d1-46d9-b0d2-51f12f4b5155`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:59:59,015::task::495::TaskManager.Task::(_debug) Task 07804611-5103-4d6f-96aa-61711be5a184: ref 1 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:59:59,016::task::495::TaskManager.Task::(_debug) Task a810492f-c703-4da1-a7a5-a4362633260b: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913546::DEBUG::2013-11-20 04:59:59,017::task::495::TaskManager.Task::(_debug) Task 07804611-5103-4d6f-96aa-61711be5a184: finished: {'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:59:59,018::task::495::TaskManager.Task::(_debug) Task a810492f-c703-4da1-a7a5-a4362633260b: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:59:59,018::task::495::TaskManager.Task::(_debug) Task 07804611-5103-4d6f-96aa-61711be5a184: moving from state preparing -> state finished
>Thread-15914519::DEBUG::2013-11-20 04:59:59,019::task::495::TaskManager.Task::(_debug) Task a810492f-c703-4da1-a7a5-a4362633260b: finished: {'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:59:59,020::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:59:59,020::task::495::TaskManager.Task::(_debug) Task a810492f-c703-4da1-a7a5-a4362633260b: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:59:59,021::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15914519::DEBUG::2013-11-20 04:59:59,021::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914519::DEBUG::2013-11-20 04:59:59,023::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:59:59,023::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15914519::DEBUG::2013-11-20 04:59:59,028::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913546::DEBUG::2013-11-20 04:59:59,028::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:59:59,030::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:59:59,030::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15914519::DEBUG::2013-11-20 04:59:59,031::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:59:59,032::task::495::TaskManager.Task::(_debug) Task 07804611-5103-4d6f-96aa-61711be5a184: ref 0 aborting False
>Thread-15914519::DEBUG::2013-11-20 04:59:59,033::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15913546::INFO::2013-11-20 04:59:59,034::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '64424509440', 'apparentsize': '64424509440'}
>Thread-15914519::DEBUG::2013-11-20 04:59:59,034::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15913546::INFO::2013-11-20 04:59:59,035::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=df8b790c-c762-4feb-9378-d64a321ce43c volUUID=2e903e43-1016-4339-a7f3-c5723a995725)
>Thread-15914519::DEBUG::2013-11-20 04:59:59,036::task::495::TaskManager.Task::(_debug) Task a810492f-c703-4da1-a7a5-a4362633260b: ref 0 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:59:59,037::task::495::TaskManager.Task::(_debug) Task 857927ac-6b66-4b97-965a-66e7e0b796c8: moving from state init -> state preparing
>Thread-15914519::INFO::2013-11-20 04:59:59,038::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '86033563648', 'apparentsize': '86033563648'}
>Thread-15913546::DEBUG::2013-11-20 04:59:59,040::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`3fcebbb4-a85c-4577-a53b-b0fbe59c7164`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:59:59,041::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15917281::DEBUG::2013-11-20 04:59:59,042::misc::1156::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Emitting event
>Thread-15917281::DEBUG::2013-11-20 04:59:59,044::misc::1166::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Calling registered method `run`
>Thread-15913546::DEBUG::2013-11-20 04:59:59,044::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15917281::DEBUG::2013-11-20 04:59:59,046::misc::1174::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Event emitted
>Thread-15913546::DEBUG::2013-11-20 04:59:59,047::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`3fcebbb4-a85c-4577-a53b-b0fbe59c7164`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:59:59,049::task::495::TaskManager.Task::(_debug) Task 857927ac-6b66-4b97-965a-66e7e0b796c8: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:59:59,050::task::495::TaskManager.Task::(_debug) Task 857927ac-6b66-4b97-965a-66e7e0b796c8: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:59:59,051::task::495::TaskManager.Task::(_debug) Task 857927ac-6b66-4b97-965a-66e7e0b796c8: finished: {'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::DEBUG::2013-11-20 04:59:59,051::task::495::TaskManager.Task::(_debug) Task 857927ac-6b66-4b97-965a-66e7e0b796c8: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:59:59,052::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:59:59,053::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:59:59,054::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:59:59,054::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:59:59,055::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:59:59,055::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:59:59,058::task::495::TaskManager.Task::(_debug) Task 857927ac-6b66-4b97-965a-66e7e0b796c8: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:59:59,059::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '150323855360', 'apparentsize': '150323855360'}
>Thread-15913546::INFO::2013-11-20 04:59:59,060::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=13efb45b-f4d2-4c94-bbdb-91ce9b9b7321 volUUID=088640f7-316c-4e2b-bfdc-0f64cf2913c8)
>Thread-15913546::DEBUG::2013-11-20 04:59:59,061::task::495::TaskManager.Task::(_debug) Task 0abc7b8e-a8d1-4b03-8b77-665b392e0035: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:59:59,062::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`06a5f577-128e-4ce9-a314-9e4e1af3ed7a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:59:59,063::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:59:59,063::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:59:59,064::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`06a5f577-128e-4ce9-a314-9e4e1af3ed7a`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:59:59,065::task::495::TaskManager.Task::(_debug) Task 0abc7b8e-a8d1-4b03-8b77-665b392e0035: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:59:59,065::task::495::TaskManager.Task::(_debug) Task 0abc7b8e-a8d1-4b03-8b77-665b392e0035: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:59:59,066::task::495::TaskManager.Task::(_debug) Task 0abc7b8e-a8d1-4b03-8b77-665b392e0035: finished: {'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::DEBUG::2013-11-20 04:59:59,070::task::495::TaskManager.Task::(_debug) Task 0abc7b8e-a8d1-4b03-8b77-665b392e0035: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:59:59,071::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:59:59,072::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:59:59,072::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:59:59,073::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:59:59,074::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:59:59,074::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:59:59,075::task::495::TaskManager.Task::(_debug) Task 0abc7b8e-a8d1-4b03-8b77-665b392e0035: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:59:59,076::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '107374182400', 'apparentsize': '107374182400'}
>Thread-15913546::INFO::2013-11-20 04:59:59,077::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=772472cb-2429-466d-96cd-00b3301ef78a spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=d368307e-425e-423a-85b7-266c1e4a99ec volUUID=fdc91117-476f-4b82-8fc4-d682fa283ecc)
>Thread-15913546::DEBUG::2013-11-20 04:59:59,077::task::495::TaskManager.Task::(_debug) Task 5466d700-c653-471d-9db5-758d1715cc20: moving from state init -> state preparing
>Thread-15913546::DEBUG::2013-11-20 04:59:59,079::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e6621294-dd46-42ed-9da4-04309ee5f78a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913546::DEBUG::2013-11-20 04:59:59,080::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' for lock type 'shared'
>Thread-15913546::DEBUG::2013-11-20 04:59:59,080::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free. Now locking as 'shared' (1 active user)
>Thread-15913546::DEBUG::2013-11-20 04:59:59,081::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.772472cb-2429-466d-96cd-00b3301ef78a`ReqID=`e6621294-dd46-42ed-9da4-04309ee5f78a`::Granted request
>Thread-15913546::DEBUG::2013-11-20 04:59:59,082::task::495::TaskManager.Task::(_debug) Task 5466d700-c653-471d-9db5-758d1715cc20: _resourcesAcquired: Storage.772472cb-2429-466d-96cd-00b3301ef78a (shared)
>Thread-15913546::DEBUG::2013-11-20 04:59:59,083::task::495::TaskManager.Task::(_debug) Task 5466d700-c653-471d-9db5-758d1715cc20: ref 1 aborting False
>Thread-15913546::DEBUG::2013-11-20 04:59:59,083::task::495::TaskManager.Task::(_debug) Task 5466d700-c653-471d-9db5-758d1715cc20: finished: {'truesize': '429496729600', 'apparentsize': '429496729600'}
>Thread-15913546::DEBUG::2013-11-20 04:59:59,084::task::495::TaskManager.Task::(_debug) Task 5466d700-c653-471d-9db5-758d1715cc20: moving from state preparing -> state finished
>Thread-15913546::DEBUG::2013-11-20 04:59:59,084::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.772472cb-2429-466d-96cd-00b3301ef78a': < ResourceRef 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', isValid: 'True' obj: 'None'>}
>Thread-15913546::DEBUG::2013-11-20 04:59:59,085::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913546::DEBUG::2013-11-20 04:59:59,086::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a'
>Thread-15913546::DEBUG::2013-11-20 04:59:59,087::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' (0 active users)
>Thread-15913546::DEBUG::2013-11-20 04:59:59,087::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a' is free, finding out if anyone is waiting for it.
>Thread-15913546::DEBUG::2013-11-20 04:59:59,088::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.772472cb-2429-466d-96cd-00b3301ef78a', Clearing records.
>Thread-15913546::DEBUG::2013-11-20 04:59:59,088::task::495::TaskManager.Task::(_debug) Task 5466d700-c653-471d-9db5-758d1715cc20: ref 0 aborting False
>Thread-15913546::INFO::2013-11-20 04:59:59,089::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '429496729600', 'apparentsize': '429496729600'}
>Dummy-15912651::DEBUG::2013-11-20 04:59:59,329::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0699293 s, 14.6 MB/s\n'; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 05:00:01,344::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 05:00:01,902::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0873843 s, 11.7 MB/s\n'; <rc> = 0
>Thread-15917284::DEBUG::2013-11-20 05:00:03,131::clientIF::54::vds::(wrapper) [192.168.100.134]::call getVmStats with ('7feb052b-c76b-45f2-bdea-005f3caf9b75',) {}
>Thread-15917284::DEBUG::2013-11-20 05:00:03,157::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Virtual machine does not exist', 'code': 1}}
>Thread-15917285::DEBUG::2013-11-20 05:00:03,243::clientIF::54::vds::(wrapper) [192.168.100.134]::call migrationCreate with ({'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/ubuntu-12.04.1-server-amd64.iso', 'bridge': 'DMZ', 'acpiEnable': 'true', 'emulatedMachine': 'rhel6.2.0', 'afterMigrationStatus': 'Up', 'vmId': '7feb052b-c76b-45f2-bdea-005f3caf9b75', 'transparentHugePages': 'true', 'displaySecurePort': '5911', 'spiceSslCipherSuite': 'DEFAULT', 'cpuType': 'Westmere', 'smp': '2', 'pauseCode': 'NOERR', 'migrationDest': 'libvirt', 'macAddr': '00:1a:4a:a8:64:1b', 'boot': 'cd', 'custom': {}, 'vmType': 'kvm', '_srcDomXML': "<domain type='kvm' id='6'>\n  <name>Nuuk</name>\n  <uuid>7feb052b-c76b-45f2-bdea-005f3caf9b75</uuid>\n  <memory unit='KiB'>6291456</memory>\n  <currentMemory unit='KiB'>6291456</currentMemory>\n  <vcpu placement='static'>2</vcpu>\n  <cputune>\n    <shares>1020</shares>\n  </cputune>\n  <sysinfo type='smbios'>\n    <system>\n      <entry name='manufacturer'>Red Hat</entry>\n      <entry name='product'>RHEV Hypervisor</entry>\n      <entry name='version'>6.4-20131016.0.el6</entry>\n      <entry name='serial'>30343536-3138-5A43-4A32-333630373443</entry>\n      <entry name='uuid'>7feb052b-c76b-45f2-bdea-005f3caf9b75</entry>\n    </system>\n  </sysinfo>\n  <os>\n    <type arch='x86_64' machine='rhel6.2.0'>hvm</type>\n    <boot dev='hd'/>\n    <boot dev='cdrom'/>\n    <smbios mode='sysinfo'/>\n  </os>\n  <features>\n    <acpi/>\n  </features>\n  <cpu mode='custom' match='exact'>\n    <model fallback='allow'>Westmere</model>\n    <topology sockets='2' cores='1' threads='1'/>\n  </cpu>\n  <clock offset='variable' adjustment='1' basis='utc'>\n    <timer name='rtc' tickpolicy='catchup'/>\n  </clock>\n  <on_poweroff>destroy</on_poweroff>\n  <on_reboot>restart</on_reboot>\n  <on_crash>destroy</on_crash>\n  <devices>\n    <emulator>/usr/libexec/qemu-kvm</emulator>\n    <disk type='block' device='disk' snapshot='no'>\n      <driver name='qemu' type='raw' cache='none' error_policy='stop' io='native'/>\n      <source dev='/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/af8a32e4-b3f3-4ec5-84aa-02c1972ac96c/d8ad65aa-4f0b-49f3-ba3f-c9c2c11e6a25'/>\n      <target dev='vda' bus='virtio'/>\n      <serial>af8a32e4-b3f3-4ec5-84aa-02c1972ac96c</serial>\n      <alias name='virtio-disk0'/>\n      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>\n    </disk>\n    <disk type='file' device='cdrom'>\n      <driver name='qemu' type='raw'/>\n      <source file='/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/ubuntu-12.04.1-server-amd64.iso' startupPolicy='optional'>\n        <seclabel model='selinux' relabel='no'/>\n      </source>\n      <target dev='hdc' bus='ide'/>\n      <readonly/>\n      <serial></serial>\n      <alias name='ide0-1-0'/>\n      <address type='drive' controller='0' bus='1' target='0' unit='0'/>\n    </disk>\n    <controller type='virtio-serial' index='0' ports='16'>\n      <alias name='virtio-serial0'/>\n      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>\n    </controller>\n    <controller type='usb' index='0'>\n      <alias name='usb0'/>\n      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>\n    </controller>\n    <controller type='ide' index='0'>\n      <alias name='ide0'/>\n      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>\n    </controller>\n    <interface type='bridge'>\n      <mac address='00:1a:4a:a8:64:1b'/>\n      <source bridge='DMZ'/>\n      <target dev='vnet5'/>\n      <model type='virtio'/>\n      <alias name='net0'/>\n      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>\n    </interface>\n    <channel type='unix'>\n      <source mode='bind' path='/var/lib/libvirt/qemu/channels/Nuuk.com.redhat.rhevm.vdsm'/>\n      <target type='virtio' name='com.redhat.rhevm.vdsm'/>\n      <alias name='channel0'/>\n      <address type='virtio-serial' controller='0' bus='0' port='1'/>\n    </channel>\n    <channel type='unix'>\n      <source mode='bind' path='/var/lib/libvirt/qemu/channels/Nuuk.org.qemu.guest_agent.0'/>\n      <target type='virtio' name='org.qemu.guest_agent.0'/>\n      <alias name='channel1'/>\n      <address type='virtio-serial' controller='0' bus='0' port='2'/>\n    </channel>\n    <channel type='spicevmc'>\n      <target type='virtio' name='com.redhat.spice.0'/>\n      <alias name='channel2'/>\n      <address type='virtio-serial' controller='0' bus='0' port='3'/>\n    </channel>\n    <input type='mouse' bus='ps2'/>\n    <graphics type='spice' port='5910' tlsPort='5911' autoport='yes' listen='0' keymap='en-us' passwdValidTo='1970-01-01T00:00:01'>\n      <listen type='address' address='0'/>\n      <channel name='main' mode='secure'/>\n      <channel name='inputs' mode='secure'/>\n    </graphics>\n    <video>\n      <model type='qxl' ram='65536' vram='65536' heads='1'/>\n      <alias name='video0'/>\n      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>\n    </video>\n    <memballoon model='none'>\n      <alias name='balloon0'/>\n    </memballoon>\n  </devices>\n  <seclabel type='dynamic' model='selinux' relabel='yes'>\n    <label>system_u:system_r:svirt_t:s0:c18,c836</label>\n    <imagelabel>system_u:object_r:svirt_image_t:s0:c18,c836</imagelabel>\n  </seclabel>\n</domain>\n", 'memSize': 6144, 'smpCoresPerSocket': '1', 'vmName': 'Nuuk', 'spiceMonitors': '1', 'nice': '0', 'status': 'Up', 'clientIp': '', 'displayIp': '0', 'drives': [{'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'if': 'virtio', 'index': '0', 'iface': 'virtio', 'apparentsize': '42949672960', 'boot': 'true', 'imageID': 'af8a32e4-b3f3-4ec5-84aa-02c1972ac96c', 'readonly': 'False', 'shared': False, 'truesize': '42949672960', 'type': 'disk', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'reqsize': '0', 'format': 'raw', 'bus': '0', 'address': {'slot': '0x05', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'disk', 'path': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/af8a32e4-b3f3-4ec5-84aa-02c1972ac96c/d8ad65aa-4f0b-49f3-ba3f-c9c2c11e6a25', 'propagateErrors': 'off', 'name': 'vda', 'volumeID': 'd8ad65aa-4f0b-49f3-ba3f-c9c2c11e6a25', 'alias': 'virtio-disk0', 'volumeChain': [{'path': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/af8a32e4-b3f3-4ec5-84aa-02c1972ac96c/d8ad65aa-4f0b-49f3-ba3f-c9c2c11e6a25', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'volumeID': 'd8ad65aa-4f0b-49f3-ba3f-c9c2c11e6a25', 'imageID': 'af8a32e4-b3f3-4ec5-84aa-02c1972ac96c'}]}], 'keyboardLayout': 'en-us', 'displayPort': '5910', 'spiceSecureChannels': 'smain,sinputs', 'nicModel': 'pv', 'elapsedTimeOffset': 758.75372385978699, 'kvmEnable': 'true', 'pitReinjection': 'false', 'devices': [{'device': 'memballoon', 'specParams': {'model': 'none'}, 'type': 'balloon'}, {'device': 'virtio-serial', 'alias': 'virtio-serial0', 'type': 'controller', 'address': {'slot': '0x04', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}}, {'device': 'qxl', 'specParams': {'vram': '65536'}, 'address': {'slot': '0x02', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'type': 'video', 'alias': 'video0'}, {'nicModel': 'virtio', 'macAddr': '00:1a:4a:a8:64:1b', 'linkActive': True, 'network': 'DMZ', 'alias': 'net0', 'address': {'slot': '0x03', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'bridge', 'type': 'interface', 'name': 'vnet5'}, {'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'if': 'virtio', 'index': '0', 'iface': 'virtio', 'apparentsize': '42949672960', 'boot': 'true', 'imageID': 'af8a32e4-b3f3-4ec5-84aa-02c1972ac96c', 'readonly': 'False', 'shared': False, 'truesize': '42949672960', 'type': 'disk', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'reqsize': '0', 'format': 'raw', 'bus': '0', 'address': {'slot': '0x05', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'disk', 'path': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/af8a32e4-b3f3-4ec5-84aa-02c1972ac96c/d8ad65aa-4f0b-49f3-ba3f-c9c2c11e6a25', 'propagateErrors': 'off', 'name': 'vda', 'volumeID': 'd8ad65aa-4f0b-49f3-ba3f-c9c2c11e6a25', 'alias': 'virtio-disk0', 'volumeChain': [{'path': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/af8a32e4-b3f3-4ec5-84aa-02c1972ac96c/d8ad65aa-4f0b-49f3-ba3f-c9c2c11e6a25', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'volumeID': 'd8ad65aa-4f0b-49f3-ba3f-c9c2c11e6a25', 'imageID': 'af8a32e4-b3f3-4ec5-84aa-02c1972ac96c'}]}, {'index': 2, 'iface': 'ide', 'name': 'hdc', 'format': 'raw', 'alias': 'ide0-1-0', 'readonly': 'True', 'truesize': 0, 'address': {'bus': '1', 'controller': '0', 'type': 'drive', 'target': '0', 'unit': '0'}, 'device': 'cdrom', 'shared': False, 'path': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/ubuntu-12.04.1-server-amd64.iso', 'propagateErrors': 'off', 'type': 'disk'}, {'device': 'usb', 'alias': 'usb0', 'type': 'controller', 'address': {'slot': '0x01', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x2'}}, {'device': 'ide', 'alias': 'ide0', 'type': 'controller', 'address': {'slot': '0x01', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x1'}}, {'device': 'unix', 'alias': 'channel0', 'type': 'channel', 'address': {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '1'}}, {'device': 'unix', 'alias': 'channel1', 'type': 'channel', 'address': {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '2'}}, {'device': 'spicevmc', 'alias': 'channel2', 'type': 'channel', 'address': {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '3'}}], 'username': 'Unknown', 'timeOffset': '1', 'guestIPs': '', 'display': 'qxl'},) {}
>Thread-15917285::DEBUG::2013-11-20 05:00:03,244::clientIF::770::vds::(migrationCreate) Migration create
>Thread-15917286::DEBUG::2013-11-20 05:00:03,516::clientIF::54::vds::(wrapper) [192.168.100.134]::call getVmStats with ('199a4bf8-ec57-49ec-93e6-2c9c32169418',) {}
>Thread-15917286::DEBUG::2013-11-20 05:00:03,517::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Virtual machine does not exist', 'code': 1}}
>Thread-15917287::DEBUG::2013-11-20 05:00:03,607::clientIF::54::vds::(wrapper) [192.168.100.134]::call migrationCreate with ({'bridge': 'DMZ', 'acpiEnable': 'true', 'emulatedMachine': 'rhel6.2.0', 'afterMigrationStatus': 'Up', 'vmId': '199a4bf8-ec57-49ec-93e6-2c9c32169418', 'transparentHugePages': 'true', 'displayPort': '5904', 'displaySecurePort': '5905', 'spiceSslCipherSuite': 'DEFAULT', 'cpuType': 'Westmere', 'smp': '4', 'pauseCode': 'NOERR', 'migrationDest': 'libvirt', 'macAddr': '00:1a:4a:a8:64:20', 'boot': 'cd', 'custom': {}, 'vmType': 'kvm', '_srcDomXML': "<domain type='kvm' id='3'>\n  <name>Yaren</name>\n  <uuid>199a4bf8-ec57-49ec-93e6-2c9c32169418</uuid>\n  <memory unit='KiB'>6291456</memory>\n  <currentMemory unit='KiB'>6291456</currentMemory>\n  <vcpu placement='static'>4</vcpu>\n  <cputune>\n    <shares>1020</shares>\n  </cputune>\n  <sysinfo type='smbios'>\n    <system>\n      <entry name='manufacturer'>Red Hat</entry>\n      <entry name='product'>RHEV Hypervisor</entry>\n      <entry name='version'>6.4-20131016.0.el6</entry>\n      <entry name='serial'>30343536-3138-5A43-4A32-333630373443</entry>\n      <entry name='uuid'>199a4bf8-ec57-49ec-93e6-2c9c32169418</entry>\n    </system>\n  </sysinfo>\n  <os>\n    <type arch='x86_64' machine='rhel6.2.0'>hvm</type>\n    <boot dev='hd'/>\n    <boot dev='cdrom'/>\n    <smbios mode='sysinfo'/>\n  </os>\n  <features>\n    <acpi/>\n  </features>\n  <cpu mode='custom' match='exact'>\n    <model fallback='allow'>Westmere</model>\n    <topology sockets='1' cores='4' threads='1'/>\n  </cpu>\n  <clock offset='variable' adjustment='0' basis='utc'>\n    <timer name='rtc' tickpolicy='catchup'/>\n  </clock>\n  <on_poweroff>destroy</on_poweroff>\n  <on_reboot>restart</on_reboot>\n  <on_crash>destroy</on_crash>\n  <devices>\n    <emulator>/usr/libexec/qemu-kvm</emulator>\n    <disk type='block' device='disk' snapshot='no'>\n      <driver name='qemu' type='raw' cache='none' error_policy='stop' io='native'/>\n      <source dev='/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/9a7c3c64-cd9f-4d1f-b3b6-e7cd16bdf4a2/a2b4cb5a-09da-4e66-aed3-c105832f9185'/>\n      <target dev='vda' bus='virtio'/>\n      <serial>9a7c3c64-cd9f-4d1f-b3b6-e7cd16bdf4a2</serial>\n      <alias name='virtio-disk0'/>\n      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>\n    </disk>\n    <disk type='file' device='cdrom'>\n      <driver name='qemu' type='raw'/>\n      <source startupPolicy='optional'/>\n      <target dev='hdc' bus='ide'/>\n      <readonly/>\n      <serial></serial>\n      <alias name='ide0-1-0'/>\n      <address type='drive' controller='0' bus='1' target='0' unit='0'/>\n    </disk>\n    <controller type='virtio-serial' index='0' ports='16'>\n      <alias name='virtio-serial0'/>\n      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>\n    </controller>\n    <controller type='usb' index='0'>\n      <alias name='usb0'/>\n      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>\n    </controller>\n    <controller type='ide' index='0'>\n      <alias name='ide0'/>\n      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>\n    </controller>\n    <interface type='bridge'>\n      <mac address='00:1a:4a:a8:64:20'/>\n      <source bridge='DMZ'/>\n      <target dev='vnet2'/>\n      <model type='virtio'/>\n      <alias name='net0'/>\n      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>\n    </interface>\n    <channel type='unix'>\n      <source mode='bind' path='/var/lib/libvirt/qemu/channels/Yaren.com.redhat.rhevm.vdsm'/>\n      <target type='virtio' name='com.redhat.rhevm.vdsm'/>\n      <alias name='channel0'/>\n      <address type='virtio-serial' controller='0' bus='0' port='1'/>\n    </channel>\n    <channel type='unix'>\n      <source mode='bind' path='/var/lib/libvirt/qemu/channels/Yaren.org.qemu.guest_agent.0'/>\n      <target type='virtio' name='org.qemu.guest_agent.0'/>\n      <alias name='channel1'/>\n      <address type='virtio-serial' controller='0' bus='0' port='2'/>\n    </channel>\n    <channel type='spicevmc'>\n      <target type='virtio' name='com.redhat.spice.0'/>\n      <alias name='channel2'/>\n      <address type='virtio-serial' controller='0' bus='0' port='3'/>\n    </channel>\n    <input type='mouse' bus='ps2'/>\n    <graphics type='spice' port='5904' tlsPort='5905' autoport='yes' listen='0' keymap='en-us' passwdValidTo='1970-01-01T00:00:01'>\n      <listen type='address' address='0'/>\n      <channel name='main' mode='secure'/>\n      <channel name='inputs' mode='secure'/>\n    </graphics>\n    <video>\n      <model type='qxl' ram='65536' vram='65536' heads='1'/>\n      <alias name='video0'/>\n      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>\n    </video>\n    <memballoon model='none'>\n      <alias name='balloon0'/>\n    </memballoon>\n  </devices>\n  <seclabel type='dynamic' model='selinux' relabel='yes'>\n    <label>system_u:system_r:svirt_t:s0:c16,c981</label>\n    <imagelabel>system_u:object_r:svirt_image_t:s0:c16,c981</imagelabel>\n  </seclabel>\n</domain>\n", 'memSize': 6144, 'smpCoresPerSocket': '4', 'vmName': 'Yaren', 'spiceMonitors': '1', 'nice': '0', 'status': 'Up', 'clientIp': '', 'displayIp': '0', 'drives': [{'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'if': 'virtio', 'index': '0', 'iface': 'virtio', 'apparentsize': '37580963840', 'boot': 'true', 'imageID': '9a7c3c64-cd9f-4d1f-b3b6-e7cd16bdf4a2', 'readonly': 'False', 'shared': False, 'truesize': '37580963840', 'type': 'disk', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'reqsize': '0', 'format': 'raw', 'bus': '0', 'address': {'slot': '0x05', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'disk', 'path': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/9a7c3c64-cd9f-4d1f-b3b6-e7cd16bdf4a2/a2b4cb5a-09da-4e66-aed3-c105832f9185', 'propagateErrors': 'off', 'name': 'vda', 'volumeID': 'a2b4cb5a-09da-4e66-aed3-c105832f9185', 'alias': 'virtio-disk0', 'volumeChain': [{'path': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/9a7c3c64-cd9f-4d1f-b3b6-e7cd16bdf4a2/a2b4cb5a-09da-4e66-aed3-c105832f9185', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'volumeID': 'a2b4cb5a-09da-4e66-aed3-c105832f9185', 'imageID': '9a7c3c64-cd9f-4d1f-b3b6-e7cd16bdf4a2'}]}], 'keyboardLayout': 'en-us', 'cdrom': '', 'spiceSecureChannels': 'smain,sinputs', 'nicModel': 'pv', 'elapsedTimeOffset': 933.71078300476074, 'kvmEnable': 'true', 'pitReinjection': 'false', 'devices': [{'device': 'memballoon', 'specParams': {'model': 'none'}, 'type': 'balloon'}, {'device': 'virtio-serial', 'alias': 'virtio-serial0', 'type': 'controller', 'address': {'slot': '0x04', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}}, {'device': 'qxl', 'specParams': {'vram': '65536'}, 'address': {'slot': '0x02', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'type': 'video', 'alias': 'video0'}, {'nicModel': 'virtio', 'macAddr': '00:1a:4a:a8:64:20', 'linkActive': True, 'network': 'DMZ', 'alias': 'net0', 'address': {'slot': '0x03', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'bridge', 'type': 'interface', 'name': 'vnet2'}, {'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'if': 'virtio', 'index': '0', 'iface': 'virtio', 'apparentsize': '37580963840', 'boot': 'true', 'imageID': '9a7c3c64-cd9f-4d1f-b3b6-e7cd16bdf4a2', 'readonly': 'False', 'shared': False, 'truesize': '37580963840', 'type': 'disk', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'reqsize': '0', 'format': 'raw', 'bus': '0', 'address': {'slot': '0x05', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'disk', 'path': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/9a7c3c64-cd9f-4d1f-b3b6-e7cd16bdf4a2/a2b4cb5a-09da-4e66-aed3-c105832f9185', 'propagateErrors': 'off', 'name': 'vda', 'volumeID': 'a2b4cb5a-09da-4e66-aed3-c105832f9185', 'alias': 'virtio-disk0', 'volumeChain': [{'path': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/9a7c3c64-cd9f-4d1f-b3b6-e7cd16bdf4a2/a2b4cb5a-09da-4e66-aed3-c105832f9185', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'volumeID': 'a2b4cb5a-09da-4e66-aed3-c105832f9185', 'imageID': '9a7c3c64-cd9f-4d1f-b3b6-e7cd16bdf4a2'}]}, {'index': 2, 'iface': 'ide', 'name': 'hdc', 'format': 'raw', 'alias': 'ide0-1-0', 'readonly': 'True', 'truesize': 0, 'address': {'bus': '1', 'controller': '0', 'type': 'drive', 'target': '0', 'unit': '0'}, 'device': 'cdrom', 'shared': False, 'path': '', 'propagateErrors': 'off', 'type': 'disk'}, {'device': 'usb', 'alias': 'usb0', 'type': 'controller', 'address': {'slot': '0x01', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x2'}}, {'device': 'ide', 'alias': 'ide0', 'type': 'controller', 'address': {'slot': '0x01', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x1'}}, {'device': 'unix', 'alias': 'channel0', 'type': 'channel', 'address': {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '1'}}, {'device': 'unix', 'alias': 'channel1', 'type': 'channel', 'address': {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '2'}}, {'device': 'spicevmc', 'alias': 'channel2', 'type': 'channel', 'address': {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '3'}}], 'username': 'Unknown', 'timeOffset': '0', 'guestIPs': '', 'display': 'qxl'},) {}
>Thread-15917287::DEBUG::2013-11-20 05:00:03,610::clientIF::770::vds::(migrationCreate) Migration create
>Thread-15917285::INFO::2013-11-20 05:00:03,834::clientIF::523::vds::(_getNetworkIp) network None: using 0
>Thread-15917285::INFO::2013-11-20 05:00:03,836::clientIF::656::vds::(create) vmContainerLock acquired by vm 7feb052b-c76b-45f2-bdea-005f3caf9b75
>Thread-15917288::DEBUG::2013-11-20 05:00:03,875::vm::342::vm.Vm::(_startUnderlyingVm) vmId=`7feb052b-c76b-45f2-bdea-005f3caf9b75`::Start
>Thread-15917285::DEBUG::2013-11-20 05:00:03,876::clientIF::669::vds::(create) Total desktops after creation of 7feb052b-c76b-45f2-bdea-005f3caf9b75 is 13
>Thread-15917288::DEBUG::2013-11-20 05:00:03,878::vm::346::vm.Vm::(_startUnderlyingVm) vmId=`7feb052b-c76b-45f2-bdea-005f3caf9b75`::_ongoingCreations acquired
>Thread-15917285::DEBUG::2013-11-20 05:00:03,879::libvirtvm::1430::vm.Vm::(waitForMigrationDestinationPrepare) vmId=`7feb052b-c76b-45f2-bdea-005f3caf9b75`::migration destination: waiting 49s for path preparation
>Thread-15917288::INFO::2013-11-20 05:00:03,880::libvirtvm::1061::vm.Vm::(_run) vmId=`7feb052b-c76b-45f2-bdea-005f3caf9b75`::VM wrapper has started
>Thread-15917288::INFO::2013-11-20 05:00:03,882::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=af8a32e4-b3f3-4ec5-84aa-02c1972ac96c volUUID=d8ad65aa-4f0b-49f3-ba3f-c9c2c11e6a25)
>Thread-15917288::DEBUG::2013-11-20 05:00:03,883::task::495::TaskManager.Task::(_debug) Task bddf3746-25fa-402d-855d-3fa3e8bb4605: moving from state init -> state preparing
>Thread-15917288::DEBUG::2013-11-20 05:00:03,887::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4cedc280-708b-46e3-b3bf-cf82dec3ae56`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1690' at 'public_prepareVolume'
>Thread-15917288::DEBUG::2013-11-20 05:00:03,888::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15917288::DEBUG::2013-11-20 05:00:03,889::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15917288::DEBUG::2013-11-20 05:00:03,890::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`4cedc280-708b-46e3-b3bf-cf82dec3ae56`::Granted request
>Thread-15917288::DEBUG::2013-11-20 05:00:03,892::task::495::TaskManager.Task::(_debug) Task bddf3746-25fa-402d-855d-3fa3e8bb4605: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15917288::DEBUG::2013-11-20 05:00:03,893::task::495::TaskManager.Task::(_debug) Task bddf3746-25fa-402d-855d-3fa3e8bb4605: ref 1 aborting False
>Thread-15917288::DEBUG::2013-11-20 05:00:03,895::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.af8a32e4-b3f3-4ec5-84aa-02c1972ac96c`ReqID=`a4f34cb0-c627-480f-99d8-fa65056d96be`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1695' at 'public_prepareVolume'
>Thread-15917288::DEBUG::2013-11-20 05:00:03,895::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.af8a32e4-b3f3-4ec5-84aa-02c1972ac96c' for lock type 'exclusive'
>Thread-15917288::DEBUG::2013-11-20 05:00:03,897::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15917288::DEBUG::2013-11-20 05:00:03,904::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 05:00:04,173::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15917287::INFO::2013-11-20 05:00:04,184::clientIF::523::vds::(_getNetworkIp) network None: using 0
>Thread-15917287::INFO::2013-11-20 05:00:04,187::clientIF::656::vds::(create) vmContainerLock acquired by vm 199a4bf8-ec57-49ec-93e6-2c9c32169418
>Thread-15917290::INFO::2013-11-20 05:00:04,570::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15917291::DEBUG::2013-11-20 05:00:04,581::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917290::DEBUG::2013-11-20 05:00:04,582::task::495::TaskManager.Task::(_debug) Task 9cf763a2-117d-4259-aeb0-2ea0ff3a7c4f: moving from state init -> state preparing
>Thread-15917291::INFO::2013-11-20 05:00:04,584::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917290::DEBUG::2013-11-20 05:00:04,586::task::495::TaskManager.Task::(_debug) Task 9cf763a2-117d-4259-aeb0-2ea0ff3a7c4f: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000938892364502', 'lastCheck': 1384923598.9974079, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '60.3919250965', 'lastCheck': 1384923599.041863, 'valid': False, 'code': 379}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '60.4595360756', 'lastCheck': 1384923598.6878049, 'valid': False, 'code': 379}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000308036804199', 'lastCheck': 1384923602.5587969, 'valid': True, 'code': 0}}
>Thread-15917291::DEBUG::2013-11-20 05:00:04,586::task::495::TaskManager.Task::(_debug) Task cae0082a-392b-4208-bff5-27d15272cecc: moving from state init -> state preparing
>Thread-15917290::DEBUG::2013-11-20 05:00:04,587::task::495::TaskManager.Task::(_debug) Task 9cf763a2-117d-4259-aeb0-2ea0ff3a7c4f: moving from state preparing -> state finished
>Thread-15917291::DEBUG::2013-11-20 05:00:04,589::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917290::DEBUG::2013-11-20 05:00:04,590::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917291::DEBUG::2013-11-20 05:00:04,590::task::495::TaskManager.Task::(_debug) Task cae0082a-392b-4208-bff5-27d15272cecc: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917290::DEBUG::2013-11-20 05:00:04,591::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917291::DEBUG::2013-11-20 05:00:04,592::task::495::TaskManager.Task::(_debug) Task cae0082a-392b-4208-bff5-27d15272cecc: moving from state preparing -> state finished
>Thread-15917290::DEBUG::2013-11-20 05:00:04,593::task::495::TaskManager.Task::(_debug) Task 9cf763a2-117d-4259-aeb0-2ea0ff3a7c4f: ref 0 aborting False
>Thread-15917291::DEBUG::2013-11-20 05:00:04,593::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917290::INFO::2013-11-20 05:00:04,594::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000938892364502', 'lastCheck': 1384923598.9974079, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '60.3919250965', 'lastCheck': 1384923599.041863, 'valid': False, 'code': 379}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '60.4595360756', 'lastCheck': 1384923598.6878049, 'valid': False, 'code': 379}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.000308036804199', 'lastCheck': 1384923602.5587969, 'valid': True, 'code': 0}}
>Thread-15917291::DEBUG::2013-11-20 05:00:04,595::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917291::DEBUG::2013-11-20 05:00:04,596::task::495::TaskManager.Task::(_debug) Task cae0082a-392b-4208-bff5-27d15272cecc: ref 0 aborting False
>Thread-15917291::INFO::2013-11-20 05:00:04,597::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917292::DEBUG::2013-11-20 05:00:04,616::vm::342::vm.Vm::(_startUnderlyingVm) vmId=`199a4bf8-ec57-49ec-93e6-2c9c32169418`::Start
>Thread-15917287::DEBUG::2013-11-20 05:00:04,616::clientIF::669::vds::(create) Total desktops after creation of 199a4bf8-ec57-49ec-93e6-2c9c32169418 is 14
>Thread-15917294::DEBUG::2013-11-20 05:00:04,617::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917287::DEBUG::2013-11-20 05:00:04,618::libvirtvm::1430::vm.Vm::(waitForMigrationDestinationPrepare) vmId=`199a4bf8-ec57-49ec-93e6-2c9c32169418`::migration destination: waiting 51s for path preparation
>Thread-15917294::INFO::2013-11-20 05:00:04,619::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917294::DEBUG::2013-11-20 05:00:04,619::task::495::TaskManager.Task::(_debug) Task 8bb307b8-4830-4615-9446-13bbd7ad15c8: moving from state init -> state preparing
>Thread-15917294::DEBUG::2013-11-20 05:00:04,620::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9fffc092-45e2-4def-afe2-43615499ce8b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917294::DEBUG::2013-11-20 05:00:04,621::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917294::DEBUG::2013-11-20 05:00:04,621::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917294::DEBUG::2013-11-20 05:00:04,622::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`9fffc092-45e2-4def-afe2-43615499ce8b`::Granted request
>Thread-15917294::DEBUG::2013-11-20 05:00:04,623::task::495::TaskManager.Task::(_debug) Task 8bb307b8-4830-4615-9446-13bbd7ad15c8: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917294::DEBUG::2013-11-20 05:00:04,623::task::495::TaskManager.Task::(_debug) Task 8bb307b8-4830-4615-9446-13bbd7ad15c8: ref 1 aborting False
>Thread-15917294::WARNING::2013-11-20 05:00:04,624::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917294::DEBUG::2013-11-20 05:00:04,625::task::495::TaskManager.Task::(_debug) Task 8bb307b8-4830-4615-9446-13bbd7ad15c8: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'alerts': []}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Thread-15917294::DEBUG::2013-11-20 05:00:04,626::task::495::TaskManager.Task::(_debug) Task 8bb307b8-4830-4615-9446-13bbd7ad15c8: moving from state preparing -> state finished
>Thread-15917294::DEBUG::2013-11-20 05:00:04,626::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917294::DEBUG::2013-11-20 05:00:04,626::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917294::DEBUG::2013-11-20 05:00:04,627::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917294::DEBUG::2013-11-20 05:00:04,627::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917294::DEBUG::2013-11-20 05:00:04,628::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917294::DEBUG::2013-11-20 05:00:04,628::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917294::DEBUG::2013-11-20 05:00:04,629::task::495::TaskManager.Task::(_debug) Task 8bb307b8-4830-4615-9446-13bbd7ad15c8: ref 0 aborting False
>Thread-15917294::INFO::2013-11-20 05:00:04,629::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'alerts': []}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'alerts': []}}}
>Dummy-15912651::DEBUG::2013-11-20 05:00:04,863::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.086801 s, 11.8 MB/s\n'; <rc> = 0
>Thread-15917296::DEBUG::2013-11-20 05:00:05,056::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.133]
>Thread-15917296::INFO::2013-11-20 05:00:05,058::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getConnectedStoragePoolsList, args: ()
>Thread-15917296::DEBUG::2013-11-20 05:00:05,058::task::495::TaskManager.Task::(_debug) Task 3dfcaedd-b207-4d0a-9a40-a3aa0e853663: moving from state init -> state preparing
>Thread-15917296::DEBUG::2013-11-20 05:00:05,058::task::495::TaskManager.Task::(_debug) Task 3dfcaedd-b207-4d0a-9a40-a3aa0e853663: finished: {'poollist': ['c4e5889a-348d-11e2-83fe-9f72369eab30']}
>Thread-15917296::DEBUG::2013-11-20 05:00:05,059::task::495::TaskManager.Task::(_debug) Task 3dfcaedd-b207-4d0a-9a40-a3aa0e853663: moving from state preparing -> state finished
>Thread-15917296::DEBUG::2013-11-20 05:00:05,059::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917296::DEBUG::2013-11-20 05:00:05,059::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917296::DEBUG::2013-11-20 05:00:05,060::task::495::TaskManager.Task::(_debug) Task 3dfcaedd-b207-4d0a-9a40-a3aa0e853663: ref 0 aborting False
>Thread-15917296::INFO::2013-11-20 05:00:05,060::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getConnectedStoragePoolsList, Return response: {'status': {'message': 'OK', 'code': 0}, 'poollist': ['c4e5889a-348d-11e2-83fe-9f72369eab30']}
>Thread-15917288::DEBUG::2013-11-20 05:00:05,207::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15917288::DEBUG::2013-11-20 05:00:06,398::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15917288::DEBUG::2013-11-20 05:00:06,409::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=5 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15917288::DEBUG::2013-11-20 05:00:06,826::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000241637 s, 2.1 MB/s\n'; <rc> = 0
>Thread-15917288::DEBUG::2013-11-20 05:00:06,828::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000241637 s, 2.1 MB/s'], size: 512
>Thread-15917288::DEBUG::2013-11-20 05:00:06,834::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=5 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 05:00:07,052::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15917288::DEBUG::2013-11-20 05:00:07,454::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000176806 s, 2.9 MB/s\n'; <rc> = 0
>Thread-15917288::DEBUG::2013-11-20 05:00:07,455::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000176806 s, 2.9 MB/s'], size: 512
>Thread-15917288::INFO::2013-11-20 05:00:07,456::image::340::Storage.Image::(getChain) sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 imgUUID=af8a32e4-b3f3-4ec5-84aa-02c1972ac96c chain=[<storage.blockVolume.BlockVolume instance at 0x7f48ac37dd40>] 
>Thread-15917288::DEBUG::2013-11-20 05:00:07,457::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --autobackup n --available y 6b892067-ac7e-4039-a7be-5b7387525b16/d8ad65aa-4f0b-49f3-ba3f-c9c2c11e6a25' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 05:00:07,966::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0864926 s, 11.8 MB/s\n'; <rc> = 0
>Thread-15917298::DEBUG::2013-11-20 05:00:08,152::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('199a4bf8-ec57-49ec-93e6-2c9c32169418',) {}
>Thread-15917298::DEBUG::2013-11-20 05:00:08,153::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Migration Destination', 'acpiEnable': 'true', 'pid': '0', 'displayIp': '0', 'displayPort': '-1', 'displaySecurePort': '-1', 'cdrom': '', 'pauseCode': 'NOERR', 'kvmEnable': 'true', 'network': {}, 'vmId': '199a4bf8-ec57-49ec-93e6-2c9c32169418', 'monitorResponse': '0', 'disks': {}, 'boot': 'cd', 'elapsedTime': '937', 'vmType': 'kvm', 'timeOffset': '0', 'clientIp': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15917299::DEBUG::2013-11-20 05:00:08,161::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('7feb052b-c76b-45f2-bdea-005f3caf9b75',) {}
>Thread-15917299::DEBUG::2013-11-20 05:00:08,162::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Migration Destination', 'acpiEnable': 'true', 'pid': '0', 'displayIp': '0', 'displayPort': '-1', 'displaySecurePort': '-1', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/ubuntu-12.04.1-server-amd64.iso', 'pauseCode': 'NOERR', 'kvmEnable': 'true', 'network': {}, 'vmId': '7feb052b-c76b-45f2-bdea-005f3caf9b75', 'monitorResponse': '0', 'disks': {}, 'boot': 'cd', 'elapsedTime': '763', 'vmType': 'kvm', 'timeOffset': '1', 'clientIp': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15917288::DEBUG::2013-11-20 05:00:08,624::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15917288::DEBUG::2013-11-20 05:00:08,626::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15917288::DEBUG::2013-11-20 05:00:08,626::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15917288::DEBUG::2013-11-20 05:00:08,629::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.d8ad65aa-4f0b-49f3-ba3f-c9c2c11e6a25`ReqID=`ae894ae3-41eb-4533-9888-2579e2f66b34`::Request was made in '/usr/share/vdsm/storage/resourceFactories.py' line '149' at '__getResourceCandidatesList'
>Thread-15917288::DEBUG::2013-11-20 05:00:08,631::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.d8ad65aa-4f0b-49f3-ba3f-c9c2c11e6a25' for lock type 'exclusive'
>Thread-15917288::DEBUG::2013-11-20 05:00:08,632::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.d8ad65aa-4f0b-49f3-ba3f-c9c2c11e6a25' is free. Now locking as 'exclusive' (1 active user)
>Thread-15917288::DEBUG::2013-11-20 05:00:08,633::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.d8ad65aa-4f0b-49f3-ba3f-c9c2c11e6a25`ReqID=`ae894ae3-41eb-4533-9888-2579e2f66b34`::Granted request
>Thread-15917288::DEBUG::2013-11-20 05:00:08,634::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.af8a32e4-b3f3-4ec5-84aa-02c1972ac96c' is free. Now locking as 'exclusive' (1 active user)
>Thread-15917288::DEBUG::2013-11-20 05:00:08,635::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.af8a32e4-b3f3-4ec5-84aa-02c1972ac96c`ReqID=`a4f34cb0-c627-480f-99d8-fa65056d96be`::Granted request
>Thread-15917288::DEBUG::2013-11-20 05:00:08,636::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15917288::DEBUG::2013-11-20 05:00:08,642::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15917300::DEBUG::2013-11-20 05:00:09,064::misc::1156::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Emitting event
>Thread-15917300::DEBUG::2013-11-20 05:00:09,065::misc::1166::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Calling registered method `run`
>Thread-15917300::DEBUG::2013-11-20 05:00:09,076::misc::1174::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Event emitted
>Thread-15917301::DEBUG::2013-11-20 05:00:09,066::spm::365::Storage.SPM::(_upgradePoolDomain) Preparing to upgrade domain 5d7b5178-3790-472c-a13b-b481eb27a61e
>Thread-15917301::ERROR::2013-11-20 05:00:09,079::spm::373::Storage.SPM::(_upgradePoolDomain) Error while preparing domain `5d7b5178-3790-472c-a13b-b481eb27a61e` upgrade
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/spm.py", line 369, in _upgradePoolDomain
>IndexError: list index out of range
>Thread-15917302::DEBUG::2013-11-20 05:00:09,456::misc::1156::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Emitting event
>Thread-15917302::DEBUG::2013-11-20 05:00:09,462::misc::1166::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Calling registered method `run`
>Thread-15917302::DEBUG::2013-11-20 05:00:09,463::misc::1174::Event.StatsThread.onDomainConnectivityStateChange::(_emit) Event emitted
>Thread-15917288::DEBUG::2013-11-20 05:00:09,815::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Dummy-15912651::DEBUG::2013-11-20 05:00:10,081::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15917288::DEBUG::2013-11-20 05:00:11,123::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15917288::DEBUG::2013-11-20 05:00:11,129::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --refresh 6b892067-ac7e-4039-a7be-5b7387525b16/d8ad65aa-4f0b-49f3-ba3f-c9c2c11e6a25' (cwd None)
>Thread-15917305::DEBUG::2013-11-20 05:00:11,417::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('199a4bf8-ec57-49ec-93e6-2c9c32169418',) {}
>Thread-15917305::DEBUG::2013-11-20 05:00:11,419::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Migration Destination', 'acpiEnable': 'true', 'pid': '0', 'displayIp': '0', 'displayPort': '-1', 'displaySecurePort': '-1', 'cdrom': '', 'pauseCode': 'NOERR', 'kvmEnable': 'true', 'network': {}, 'vmId': '199a4bf8-ec57-49ec-93e6-2c9c32169418', 'monitorResponse': '0', 'disks': {}, 'boot': 'cd', 'elapsedTime': '940', 'vmType': 'kvm', 'timeOffset': '0', 'clientIp': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15917306::DEBUG::2013-11-20 05:00:11,426::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('7feb052b-c76b-45f2-bdea-005f3caf9b75',) {}
>Thread-15917306::DEBUG::2013-11-20 05:00:11,427::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Migration Destination', 'acpiEnable': 'true', 'pid': '0', 'displayIp': '0', 'displayPort': '-1', 'displaySecurePort': '-1', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/ubuntu-12.04.1-server-amd64.iso', 'pauseCode': 'NOERR', 'kvmEnable': 'true', 'network': {}, 'vmId': '7feb052b-c76b-45f2-bdea-005f3caf9b75', 'monitorResponse': '0', 'disks': {}, 'boot': 'cd', 'elapsedTime': '766', 'vmType': 'kvm', 'timeOffset': '1', 'clientIp': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 05:00:11,557::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.634745 s, 1.6 MB/s\n'; <rc> = 0
>Thread-15917288::DEBUG::2013-11-20 05:00:12,045::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15917288::DEBUG::2013-11-20 05:00:12,049::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15917288::DEBUG::2013-11-20 05:00:12,050::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15917288::INFO::2013-11-20 05:00:12,051::volume::517::Storage.Volume::(prepare) Volume: preparing volume 6b892067-ac7e-4039-a7be-5b7387525b16/d8ad65aa-4f0b-49f3-ba3f-c9c2c11e6a25
>Thread-15917288::DEBUG::2013-11-20 05:00:12,052::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15917288::DEBUG::2013-11-20 05:00:12,058::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15917288::DEBUG::2013-11-20 05:00:12,815::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15914435::INFO::2013-11-20 05:00:13,253::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=143cb26d-de08-4d30-89cf-aee10245b66a volUUID=d7028f7c-0551-4bbe-a907-6f804cc9bd54)
>Thread-15914435::DEBUG::2013-11-20 05:00:13,373::task::495::TaskManager.Task::(_debug) Task 11fccb18-e168-4830-a9ec-82ac1a72b905: moving from state init -> state preparing
>Thread-15914435::DEBUG::2013-11-20 05:00:13,450::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1fa8e3b3-570b-4acd-93ec-7d54cd64327d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15914435::DEBUG::2013-11-20 05:00:13,479::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15914435::DEBUG::2013-11-20 05:00:13,515::resourceManager::481::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' found in shared state and queue is empty, Joining current shared lock (2 active users)
>Thread-15914435::DEBUG::2013-11-20 05:00:13,575::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`1fa8e3b3-570b-4acd-93ec-7d54cd64327d`::Granted request
>Dummy-15912651::DEBUG::2013-11-20 05:00:13,576::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15917288::DEBUG::2013-11-20 05:00:14,037::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15914435::DEBUG::2013-11-20 05:00:14,047::task::495::TaskManager.Task::(_debug) Task 11fccb18-e168-4830-a9ec-82ac1a72b905: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15917288::DEBUG::2013-11-20 05:00:14,049::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=5 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15914435::DEBUG::2013-11-20 05:00:14,050::task::495::TaskManager.Task::(_debug) Task 11fccb18-e168-4830-a9ec-82ac1a72b905: ref 1 aborting False
>Thread-15914435::DEBUG::2013-11-20 05:00:14,051::task::495::TaskManager.Task::(_debug) Task 11fccb18-e168-4830-a9ec-82ac1a72b905: finished: {'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15917307::DEBUG::2013-11-20 05:00:14,357::clientIF::54::vds::(wrapper) [192.168.100.134]::call getVmStats with ('dab1628d-5a54-49ba-9685-47e4c156842c',) {}
>Thread-15917308::DEBUG::2013-11-20 05:00:14,358::clientIF::54::vds::(wrapper) [192.168.100.134]::call getVmStats with ('aeb21bdb-eba2-472b-ae43-1590668054b5',) {}
>Thread-15914435::DEBUG::2013-11-20 05:00:14,364::task::495::TaskManager.Task::(_debug) Task 11fccb18-e168-4830-a9ec-82ac1a72b905: moving from state preparing -> state finished
>Thread-15917307::DEBUG::2013-11-20 05:00:14,369::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Virtual machine does not exist', 'code': 1}}
>Thread-15917308::DEBUG::2013-11-20 05:00:14,370::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Virtual machine does not exist', 'code': 1}}
>Thread-15914435::DEBUG::2013-11-20 05:00:14,374::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15914435::DEBUG::2013-11-20 05:00:14,403::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917310::DEBUG::2013-11-20 05:00:14,423::clientIF::54::vds::(wrapper) [192.168.100.134]::call getVmStats with ('866bb15c-0a0c-464c-9cf3-4fbcf302832f',) {}
>Thread-15917310::DEBUG::2013-11-20 05:00:14,426::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Virtual machine does not exist', 'code': 1}}
>Thread-15914435::DEBUG::2013-11-20 05:00:14,427::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15917309::DEBUG::2013-11-20 05:00:14,429::clientIF::54::vds::(wrapper) [192.168.100.134]::call getVmStats with ('6fc0379e-5f19-4d16-bc07-574a75cf8356',) {}
>Thread-15917309::DEBUG::2013-11-20 05:00:14,432::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Virtual machine does not exist', 'code': 1}}
>Thread-15914435::DEBUG::2013-11-20 05:00:14,433::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (1 active users)
>Thread-15914435::DEBUG::2013-11-20 05:00:14,437::task::495::TaskManager.Task::(_debug) Task 11fccb18-e168-4830-a9ec-82ac1a72b905: ref 0 aborting False
>Thread-15914435::INFO::2013-11-20 05:00:14,439::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '56908316672', 'apparentsize': '56908316672'}
>Thread-15917312::DEBUG::2013-11-20 05:00:14,462::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('199a4bf8-ec57-49ec-93e6-2c9c32169418',) {}
>Thread-15917312::DEBUG::2013-11-20 05:00:14,464::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Migration Destination', 'acpiEnable': 'true', 'pid': '0', 'displayIp': '0', 'displayPort': '-1', 'displaySecurePort': '-1', 'cdrom': '', 'pauseCode': 'NOERR', 'kvmEnable': 'true', 'network': {}, 'vmId': '199a4bf8-ec57-49ec-93e6-2c9c32169418', 'monitorResponse': '0', 'disks': {}, 'boot': 'cd', 'elapsedTime': '943', 'vmType': 'kvm', 'timeOffset': '0', 'clientIp': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15917313::DEBUG::2013-11-20 05:00:14,477::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('7feb052b-c76b-45f2-bdea-005f3caf9b75',) {}
>Thread-15917313::DEBUG::2013-11-20 05:00:14,478::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Migration Destination', 'acpiEnable': 'true', 'pid': '0', 'displayIp': '0', 'displayPort': '-1', 'displaySecurePort': '-1', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/ubuntu-12.04.1-server-amd64.iso', 'pauseCode': 'NOERR', 'kvmEnable': 'true', 'network': {}, 'vmId': '7feb052b-c76b-45f2-bdea-005f3caf9b75', 'monitorResponse': '0', 'disks': {}, 'boot': 'cd', 'elapsedTime': '769', 'vmType': 'kvm', 'timeOffset': '1', 'clientIp': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15917288::DEBUG::2013-11-20 05:00:14,594::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000191538 s, 2.7 MB/s\n'; <rc> = 0
>Thread-15917288::DEBUG::2013-11-20 05:00:14,597::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000191538 s, 2.7 MB/s'], size: 512
>Thread-15917288::DEBUG::2013-11-20 05:00:14,602::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=5 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15917314::DEBUG::2013-11-20 05:00:14,836::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917314::INFO::2013-11-20 05:00:14,840::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917314::DEBUG::2013-11-20 05:00:14,842::task::495::TaskManager.Task::(_debug) Task f32b6d6e-f13b-4b1d-9ef1-1f4890624c8c: moving from state init -> state preparing
>Thread-15917314::DEBUG::2013-11-20 05:00:14,852::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30: spmStatus=SPM spmLver=287 spmId=2
>Thread-15917314::DEBUG::2013-11-20 05:00:14,861::task::495::TaskManager.Task::(_debug) Task f32b6d6e-f13b-4b1d-9ef1-1f4890624c8c: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917315::DEBUG::2013-11-20 05:00:14,884::clientIF::54::vds::(wrapper) [192.168.100.134]::call migrationCreate with ({'bridge': 'DMZ', 'acpiEnable': 'true', 'emulatedMachine': 'rhel6.2.0', 'afterMigrationStatus': 'Up', 'vmId': 'dab1628d-5a54-49ba-9685-47e4c156842c', 'transparentHugePages': 'true', 'displayPort': '5908', 'displaySecurePort': '5909', 'spiceSslCipherSuite': 'DEFAULT', 'cpuType': 'Westmere', 'smp': '4', 'pauseCode': 'NOERR', 'migrationDest': 'libvirt', 'macAddr': '00:1a:4a:a8:64:1a', 'boot': 'cd', 'custom': {}, 'vmType': 'kvm', '_srcDomXML': "<domain type='kvm' id='5'>\n  <name>Dili</name>\n  <uuid>dab1628d-5a54-49ba-9685-47e4c156842c</uuid>\n  <memory unit='KiB'>6291456</memory>\n  <currentMemory unit='KiB'>6291456</currentMemory>\n  <vcpu placement='static'>4</vcpu>\n  <cputune>\n    <shares>1020</shares>\n  </cputune>\n  <sysinfo type='smbios'>\n    <system>\n      <entry name='manufacturer'>Red Hat</entry>\n      <entry name='product'>RHEV Hypervisor</entry>\n      <entry name='version'>6.4-20131016.0.el6</entry>\n      <entry name='serial'>30343536-3138-5A43-4A32-333630373443</entry>\n      <entry name='uuid'>dab1628d-5a54-49ba-9685-47e4c156842c</entry>\n    </system>\n  </sysinfo>\n  <os>\n    <type arch='x86_64' machine='rhel6.2.0'>hvm</type>\n    <boot dev='hd'/>\n    <boot dev='cdrom'/>\n    <smbios mode='sysinfo'/>\n  </os>\n  <features>\n    <acpi/>\n  </features>\n  <cpu mode='custom' match='exact'>\n    <model fallback='allow'>Westmere</model>\n    <topology sockets='2' cores='2' threads='1'/>\n  </cpu>\n  <clock offset='variable' adjustment='0' basis='utc'>\n    <timer name='rtc' tickpolicy='catchup'/>\n  </clock>\n  <on_poweroff>destroy</on_poweroff>\n  <on_reboot>restart</on_reboot>\n  <on_crash>destroy</on_crash>\n  <devices>\n    <emulator>/usr/libexec/qemu-kvm</emulator>\n    <disk type='block' device='disk' snapshot='no'>\n      <driver name='qemu' type='raw' cache='none' error_policy='stop' io='native'/>\n      <source dev='/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/089b08fc-6544-4235-a061-fb55b09d2715/66b599fc-b7b5-4934-a5d5-ef7283460029'/>\n      <target dev='vda' bus='virtio'/>\n      <serial>089b08fc-6544-4235-a061-fb55b09d2715</serial>\n      <alias name='virtio-disk0'/>\n      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>\n    </disk>\n    <disk type='file' device='cdrom'>\n      <driver name='qemu' type='raw'/>\n      <source startupPolicy='optional'/>\n      <target dev='hdc' bus='ide'/>\n      <readonly/>\n      <serial></serial>\n      <alias name='ide0-1-0'/>\n      <address type='drive' controller='0' bus='1' target='0' unit='0'/>\n    </disk>\n    <controller type='virtio-serial' index='0' ports='16'>\n      <alias name='virtio-serial0'/>\n      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>\n    </controller>\n    <controller type='usb' index='0'>\n      <alias name='usb0'/>\n      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>\n    </controller>\n    <controller type='ide' index='0'>\n      <alias name='ide0'/>\n      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>\n    </controller>\n    <interface type='bridge'>\n      <mac address='00:1a:4a:a8:64:1a'/>\n      <source bridge='DMZ'/>\n      <target dev='vnet4'/>\n      <model type='virtio'/>\n      <alias name='net0'/>\n      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>\n    </interface>\n    <channel type='unix'>\n      <source mode='bind' path='/var/lib/libvirt/qemu/channels/Dili.com.redhat.rhevm.vdsm'/>\n      <target type='virtio' name='com.redhat.rhevm.vdsm'/>\n      <alias name='channel0'/>\n      <address type='virtio-serial' controller='0' bus='0' port='1'/>\n    </channel>\n    <channel type='unix'>\n      <source mode='bind' path='/var/lib/libvirt/qemu/channels/Dili.org.qemu.guest_agent.0'/>\n      <target type='virtio' name='org.qemu.guest_agent.0'/>\n      <alias name='channel1'/>\n      <address type='virtio-serial' controller='0' bus='0' port='2'/>\n    </channel>\n    <channel type='spicevmc'>\n      <target type='virtio' name='com.redhat.spice.0'/>\n      <alias name='channel2'/>\n      <address type='virtio-serial' controller='0' bus='0' port='3'/>\n    </channel>\n    <input type='mouse' bus='ps2'/>\n    <graphics type='spice' port='5908' tlsPort='5909' autoport='yes' listen='0' keymap='en-us' passwdValidTo='1970-01-01T00:00:01'>\n      <listen type='address' address='0'/>\n      <channel name='main' mode='secure'/>\n      <channel name='inputs' mode='secure'/>\n    </graphics>\n    <video>\n      <model type='qxl' ram='65536' vram='65536' heads='1'/>\n      <alias name='video0'/>\n      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>\n    </video>\n    <memballoon model='none'>\n      <alias name='balloon0'/>\n    </memballoon>\n  </devices>\n  <seclabel type='dynamic' model='selinux' relabel='yes'>\n    <label>system_u:system_r:svirt_t:s0:c79,c678</label>\n    <imagelabel>system_u:object_r:svirt_image_t:s0:c79,c678</imagelabel>\n  </seclabel>\n</domain>\n", 'memSize': 6144, 'smpCoresPerSocket': '2', 'vmName': 'Dili', 'spiceMonitors': '1', 'nice': '0', 'status': 'Up', 'clientIp': '', 'displayIp': '0', 'drives': [{'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'if': 'virtio', 'index': '0', 'iface': 'virtio', 'apparentsize': '107374182400', 'boot': 'true', 'imageID': '089b08fc-6544-4235-a061-fb55b09d2715', 'readonly': 'False', 'shared': False, 'truesize': '107374182400', 'type': 'disk', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'reqsize': '0', 'format': 'raw', 'bus': '0', 'address': {'slot': '0x05', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'disk', 'path': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/089b08fc-6544-4235-a061-fb55b09d2715/66b599fc-b7b5-4934-a5d5-ef7283460029', 'propagateErrors': 'off', 'name': 'vda', 'volumeID': '66b599fc-b7b5-4934-a5d5-ef7283460029', 'alias': 'virtio-disk0', 'volumeChain': [{'path': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/089b08fc-6544-4235-a061-fb55b09d2715/66b599fc-b7b5-4934-a5d5-ef7283460029', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'volumeID': '66b599fc-b7b5-4934-a5d5-ef7283460029', 'imageID': '089b08fc-6544-4235-a061-fb55b09d2715'}]}], 'keyboardLayout': 'en-us', 'cdrom': '', 'spiceSecureChannels': 'smain,sinputs', 'nicModel': 'pv', 'elapsedTimeOffset': 943.89679098129272, 'kvmEnable': 'true', 'pitReinjection': 'false', 'devices': [{'device': 'memballoon', 'specParams': {'model': 'none'}, 'type': 'balloon'}, {'device': 'virtio-serial', 'alias': 'virtio-serial0', 'type': 'controller', 'address': {'slot': '0x04', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}}, {'device': 'qxl', 'specParams': {'vram': '65536'}, 'address': {'slot': '0x02', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'type': 'video', 'alias': 'video0'}, {'nicModel': 'virtio', 'macAddr': '00:1a:4a:a8:64:1a', 'linkActive': True, 'network': 'DMZ', 'alias': 'net0', 'address': {'slot': '0x03', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'bridge', 'type': 'interface', 'name': 'vnet4'}, {'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'if': 'virtio', 'index': '0', 'iface': 'virtio', 'apparentsize': '107374182400', 'boot': 'true', 'imageID': '089b08fc-6544-4235-a061-fb55b09d2715', 'readonly': 'False', 'shared': False, 'truesize': '107374182400', 'type': 'disk', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'reqsize': '0', 'format': 'raw', 'bus': '0', 'address': {'slot': '0x05', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'disk', 'path': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/089b08fc-6544-4235-a061-fb55b09d2715/66b599fc-b7b5-4934-a5d5-ef7283460029', 'propagateErrors': 'off', 'name': 'vda', 'volumeID': '66b599fc-b7b5-4934-a5d5-ef7283460029', 'alias': 'virtio-disk0', 'volumeChain': [{'path': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/089b08fc-6544-4235-a061-fb55b09d2715/66b599fc-b7b5-4934-a5d5-ef7283460029', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'volumeID': '66b599fc-b7b5-4934-a5d5-ef7283460029', 'imageID': '089b08fc-6544-4235-a061-fb55b09d2715'}]}, {'index': 2, 'iface': 'ide', 'name': 'hdc', 'format': 'raw', 'alias': 'ide0-1-0', 'readonly': 'True', 'truesize': 0, 'address': {'bus': '1', 'controller': '0', 'type': 'drive', 'target': '0', 'unit': '0'}, 'device': 'cdrom', 'shared': False, 'path': '', 'propagateErrors': 'off', 'type': 'disk'}, {'device': 'usb', 'alias': 'usb0', 'type': 'controller', 'address': {'slot': '0x01', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x2'}}, {'device': 'ide', 'alias': 'ide0', 'type': 'controller', 'address': {'slot': '0x01', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x1'}}, {'device': 'unix', 'alias': 'channel0', 'type': 'channel', 'address': {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '1'}}, {'device': 'unix', 'alias': 'channel1', 'type': 'channel', 'address': {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '2'}}, {'device': 'spicevmc', 'alias': 'channel2', 'type': 'channel', 'address': {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '3'}}], 'username': 'Unknown', 'timeOffset': '0', 'guestIPs': '', 'display': 'qxl'},) {}
>Thread-15917314::DEBUG::2013-11-20 05:00:14,885::task::495::TaskManager.Task::(_debug) Task f32b6d6e-f13b-4b1d-9ef1-1f4890624c8c: moving from state preparing -> state finished
>Thread-15917315::DEBUG::2013-11-20 05:00:14,891::clientIF::770::vds::(migrationCreate) Migration create
>Thread-15917314::DEBUG::2013-11-20 05:00:14,893::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Dummy-15912651::DEBUG::2013-11-20 05:00:14,896::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.251819 s, 4.1 MB/s\n'; <rc> = 0
>Thread-15917314::DEBUG::2013-11-20 05:00:14,898::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917314::DEBUG::2013-11-20 05:00:15,142::task::495::TaskManager.Task::(_debug) Task f32b6d6e-f13b-4b1d-9ef1-1f4890624c8c: ref 0 aborting False
>Thread-15917314::INFO::2013-11-20 05:00:15,148::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 287}}
>Thread-15917288::DEBUG::2013-11-20 05:00:15,165::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.00022912 s, 2.2 MB/s\n'; <rc> = 0
>Thread-15917288::DEBUG::2013-11-20 05:00:15,170::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.00022912 s, 2.2 MB/s'], size: 512
>Thread-15917316::DEBUG::2013-11-20 05:00:15,173::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [192.168.100.135]
>Thread-15917288::DEBUG::2013-11-20 05:00:15,176::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.d8ad65aa-4f0b-49f3-ba3f-c9c2c11e6a25`ReqID=`da9e7b7c-e999-40d0-a36b-cf05167c2749`::Request was made in '/usr/share/vdsm/storage/volume.py' line '533' at 'prepare'
>Thread-15917316::INFO::2013-11-20 05:00:15,177::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, args: ( spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30)
>Thread-15917288::DEBUG::2013-11-20 05:00:15,178::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.d8ad65aa-4f0b-49f3-ba3f-c9c2c11e6a25' for lock type 'exclusive'
>Thread-15917316::DEBUG::2013-11-20 05:00:15,179::task::495::TaskManager.Task::(_debug) Task 3e0de7c4-72ab-447a-ab1e-b22177dfe1a4: moving from state init -> state preparing
>Thread-15917316::DEBUG::2013-11-20 05:00:15,181::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`00bc9253-8ad3-4987-8608-a1bd7538003b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1162' at 'public_getStoragePoolInfo'
>Thread-15917288::DEBUG::2013-11-20 05:00:15,182::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.d8ad65aa-4f0b-49f3-ba3f-c9c2c11e6a25' is free. Now locking as 'exclusive' (1 active user)
>Thread-15917316::DEBUG::2013-11-20 05:00:15,183::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' for lock type 'shared'
>Thread-15917288::DEBUG::2013-11-20 05:00:15,183::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_lvmActivationNS.d8ad65aa-4f0b-49f3-ba3f-c9c2c11e6a25`ReqID=`da9e7b7c-e999-40d0-a36b-cf05167c2749`::Granted request
>Thread-15917316::DEBUG::2013-11-20 05:00:15,184::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free. Now locking as 'shared' (1 active user)
>Thread-15917288::DEBUG::2013-11-20 05:00:15,205::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=5 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15917316::DEBUG::2013-11-20 05:00:15,206::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.c4e5889a-348d-11e2-83fe-9f72369eab30`ReqID=`00bc9253-8ad3-4987-8608-a1bd7538003b`::Granted request
>Thread-15917316::DEBUG::2013-11-20 05:00:15,533::task::495::TaskManager.Task::(_debug) Task 3e0de7c4-72ab-447a-ab1e-b22177dfe1a4: _resourcesAcquired: Storage.c4e5889a-348d-11e2-83fe-9f72369eab30 (shared)
>Thread-15917315::INFO::2013-11-20 05:00:15,537::clientIF::523::vds::(_getNetworkIp) network None: using 0
>Thread-15917316::DEBUG::2013-11-20 05:00:15,538::task::495::TaskManager.Task::(_debug) Task 3e0de7c4-72ab-447a-ab1e-b22177dfe1a4: ref 1 aborting False
>Thread-15917315::INFO::2013-11-20 05:00:15,538::clientIF::656::vds::(create) vmContainerLock acquired by vm dab1628d-5a54-49ba-9685-47e4c156842c
>Thread-15917316::WARNING::2013-11-20 05:00:15,543::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917316::DEBUG::2013-11-20 05:00:15,550::task::495::TaskManager.Task::(_debug) Task 3e0de7c4-72ab-447a-ab1e-b22177dfe1a4: finished: {'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '114997329920', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '114997329920', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917316::DEBUG::2013-11-20 05:00:15,552::task::495::TaskManager.Task::(_debug) Task 3e0de7c4-72ab-447a-ab1e-b22177dfe1a4: moving from state preparing -> state finished
>Thread-15917316::DEBUG::2013-11-20 05:00:15,553::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30': < ResourceRef 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', isValid: 'True' obj: 'None'>}
>Thread-15917316::DEBUG::2013-11-20 05:00:15,555::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917316::DEBUG::2013-11-20 05:00:15,556::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30'
>Thread-15917316::DEBUG::2013-11-20 05:00:15,557::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' (0 active users)
>Thread-15917316::DEBUG::2013-11-20 05:00:15,558::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30' is free, finding out if anyone is waiting for it.
>Thread-15917316::DEBUG::2013-11-20 05:00:15,559::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.c4e5889a-348d-11e2-83fe-9f72369eab30', Clearing records.
>Thread-15917316::DEBUG::2013-11-20 05:00:15,560::task::495::TaskManager.Task::(_debug) Task 3e0de7c4-72ab-447a-ab1e-b22177dfe1a4: ref 0 aborting False
>Thread-15917316::INFO::2013-11-20 05:00:15,561::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'name': 'Default', 'version': '2', 'domains': u'6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', 'pool_status': 'connected', 'isoprefix': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111', 'type': 'FCP', 'master_ver': 1, 'lver': 287}, 'dominfo': {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'status': u'Active', 'diskfree': '1142595518464', 'alerts': [], 'disktotal': '3595827150848'}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'status': u'Active', 'diskfree': '114997329920', 'alerts': [], 'disktotal': '240964861952'}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'status': u'Active', 'diskfree': '442650066944', 'alerts': [], 'disktotal': '1198430093312'}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'status': u'Active', 'diskfree': '114997329920', 'alerts': [], 'disktotal': '240964861952'}}}
>Thread-15917317::DEBUG::2013-11-20 05:00:15,568::vm::342::vm.Vm::(_startUnderlyingVm) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::Start
>Thread-15917315::DEBUG::2013-11-20 05:00:15,569::clientIF::669::vds::(create) Total desktops after creation of dab1628d-5a54-49ba-9685-47e4c156842c is 15
>Thread-15917315::DEBUG::2013-11-20 05:00:15,570::libvirtvm::1430::vm.Vm::(waitForMigrationDestinationPrepare) vmId=`dab1628d-5a54-49ba-9685-47e4c156842c`::migration destination: waiting 52s for path preparation
>Thread-15917288::DEBUG::2013-11-20 05:00:15,726::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000233216 s, 2.2 MB/s\n'; <rc> = 0
>Thread-15917288::DEBUG::2013-11-20 05:00:15,727::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000233216 s, 2.2 MB/s'], size: 512
>Thread-15917288::DEBUG::2013-11-20 05:00:15,729::task::495::TaskManager.Task::(_debug) Task bddf3746-25fa-402d-855d-3fa3e8bb4605: finished: None
>Thread-15917288::DEBUG::2013-11-20 05:00:15,729::task::495::TaskManager.Task::(_debug) Task bddf3746-25fa-402d-855d-3fa3e8bb4605: moving from state preparing -> state finished
>Thread-15917288::DEBUG::2013-11-20 05:00:15,730::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15917288::DEBUG::2013-11-20 05:00:15,730::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917288::DEBUG::2013-11-20 05:00:15,731::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15917288::DEBUG::2013-11-20 05:00:15,732::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15917288::DEBUG::2013-11-20 05:00:15,732::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15917288::DEBUG::2013-11-20 05:00:15,733::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15917288::DEBUG::2013-11-20 05:00:15,734::task::495::TaskManager.Task::(_debug) Task bddf3746-25fa-402d-855d-3fa3e8bb4605: ref 0 aborting False
>Thread-15917288::INFO::2013-11-20 05:00:15,734::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, Return response: {'status': {'message': 'OK', 'code': 0}}
>Thread-15917288::INFO::2013-11-20 05:00:15,735::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=af8a32e4-b3f3-4ec5-84aa-02c1972ac96c volUUID=d8ad65aa-4f0b-49f3-ba3f-c9c2c11e6a25)
>Thread-15917288::DEBUG::2013-11-20 05:00:15,736::task::495::TaskManager.Task::(_debug) Task 0f5ce87a-ef1a-430f-9c5f-5996d8a7b2b7: moving from state init -> state preparing
>Thread-15917288::DEBUG::2013-11-20 05:00:15,737::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f163d7c9-9cdc-447e-8ac3-bf4c7bae612c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1666' at 'public_getVolumePath'
>Thread-15917288::DEBUG::2013-11-20 05:00:15,738::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15917288::DEBUG::2013-11-20 05:00:15,738::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15917288::DEBUG::2013-11-20 05:00:15,739::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`f163d7c9-9cdc-447e-8ac3-bf4c7bae612c`::Granted request
>Thread-15917288::DEBUG::2013-11-20 05:00:15,740::task::495::TaskManager.Task::(_debug) Task 0f5ce87a-ef1a-430f-9c5f-5996d8a7b2b7: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15917288::DEBUG::2013-11-20 05:00:15,741::task::495::TaskManager.Task::(_debug) Task 0f5ce87a-ef1a-430f-9c5f-5996d8a7b2b7: ref 1 aborting False
>Thread-15917288::DEBUG::2013-11-20 05:00:15,742::task::495::TaskManager.Task::(_debug) Task 0f5ce87a-ef1a-430f-9c5f-5996d8a7b2b7: finished: {'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/af8a32e4-b3f3-4ec5-84aa-02c1972ac96c/d8ad65aa-4f0b-49f3-ba3f-c9c2c11e6a25'}
>Thread-15917288::DEBUG::2013-11-20 05:00:15,742::task::495::TaskManager.Task::(_debug) Task 0f5ce87a-ef1a-430f-9c5f-5996d8a7b2b7: moving from state preparing -> state finished
>Thread-15917288::DEBUG::2013-11-20 05:00:15,743::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15917288::DEBUG::2013-11-20 05:00:15,743::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917288::DEBUG::2013-11-20 05:00:15,744::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15917288::DEBUG::2013-11-20 05:00:15,745::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15917288::DEBUG::2013-11-20 05:00:15,745::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15917288::DEBUG::2013-11-20 05:00:15,746::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15917288::DEBUG::2013-11-20 05:00:15,746::task::495::TaskManager.Task::(_debug) Task 0f5ce87a-ef1a-430f-9c5f-5996d8a7b2b7: ref 0 aborting False
>Thread-15917288::INFO::2013-11-20 05:00:15,747::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumePath, Return response: {'status': {'message': 'OK', 'code': 0}, 'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/af8a32e4-b3f3-4ec5-84aa-02c1972ac96c/d8ad65aa-4f0b-49f3-ba3f-c9c2c11e6a25'}
>Thread-15917288::INFO::2013-11-20 05:00:15,788::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=af8a32e4-b3f3-4ec5-84aa-02c1972ac96c volUUID=d8ad65aa-4f0b-49f3-ba3f-c9c2c11e6a25)
>Thread-15917288::DEBUG::2013-11-20 05:00:15,789::task::495::TaskManager.Task::(_debug) Task 7216c3fa-5b32-40e7-9257-2402f87fb4ed: moving from state init -> state preparing
>Thread-15917288::DEBUG::2013-11-20 05:00:15,790::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7470d74f-aa2d-4d58-9964-1e78e816ad48`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15917288::DEBUG::2013-11-20 05:00:15,790::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15917288::DEBUG::2013-11-20 05:00:15,791::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15917288::DEBUG::2013-11-20 05:00:15,791::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7470d74f-aa2d-4d58-9964-1e78e816ad48`::Granted request
>Thread-15917288::DEBUG::2013-11-20 05:00:15,792::task::495::TaskManager.Task::(_debug) Task 7216c3fa-5b32-40e7-9257-2402f87fb4ed: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15917288::DEBUG::2013-11-20 05:00:15,793::task::495::TaskManager.Task::(_debug) Task 7216c3fa-5b32-40e7-9257-2402f87fb4ed: ref 1 aborting False
>Thread-15917288::DEBUG::2013-11-20 05:00:15,793::task::495::TaskManager.Task::(_debug) Task 7216c3fa-5b32-40e7-9257-2402f87fb4ed: finished: {'truesize': '42949672960', 'apparentsize': '42949672960'}
>Thread-15917288::DEBUG::2013-11-20 05:00:15,794::task::495::TaskManager.Task::(_debug) Task 7216c3fa-5b32-40e7-9257-2402f87fb4ed: moving from state preparing -> state finished
>Thread-15917288::DEBUG::2013-11-20 05:00:15,794::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15917288::DEBUG::2013-11-20 05:00:15,794::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917288::DEBUG::2013-11-20 05:00:15,795::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15917288::DEBUG::2013-11-20 05:00:15,795::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15917288::DEBUG::2013-11-20 05:00:15,796::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15917288::DEBUG::2013-11-20 05:00:15,796::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15917288::DEBUG::2013-11-20 05:00:15,797::task::495::TaskManager.Task::(_debug) Task 7216c3fa-5b32-40e7-9257-2402f87fb4ed: ref 0 aborting False
>Thread-15917288::INFO::2013-11-20 05:00:15,797::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '42949672960', 'apparentsize': '42949672960'}
>Thread-15917288::INFO::2013-11-20 05:00:15,798::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16)
>Thread-15917288::DEBUG::2013-11-20 05:00:15,799::task::495::TaskManager.Task::(_debug) Task 7424383f-14fb-45e3-a85b-dd22eefa2115: moving from state init -> state preparing
>Thread-15917288::INFO::2013-11-20 05:00:15,799::blockSD::674::Storage.StorageDomain::(validate) sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16
>Thread-15917288::DEBUG::2013-11-20 05:00:15,800::lvm::554::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15917288::DEBUG::2013-11-20 05:00:15,800::lvm::556::OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15917288::DEBUG::2013-11-20 05:00:15,801::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15917288::DEBUG::2013-11-20 05:00:15,802::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15917288::DEBUG::2013-11-20 05:00:15,802::lvm::439::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex
>Thread-15917288::DEBUG::2013-11-20 05:00:15,808::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15917288::DEBUG::2013-11-20 05:00:16,498::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15917288::DEBUG::2013-11-20 05:00:16,512::lvm::466::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex
>Thread-15917288::DEBUG::2013-11-20 05:00:16,513::persistentDict::204::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[u'VERSION=2', u'LOCKPOLICY=', u'DESCRIPTION=OS', u'VGUUID=aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', u'LOGBLKSIZE=512', u'LEASERETRIES=3', u'PV0=pv:3600c0ff000152111c050605001000000,uuid:V0Qu2g-uNB7-OL9h-ogfd-i1xl-RRZk-CpgKaU,pestart:0,pecount:26791,mapoffset:0', u'LOCKRENEWALINTERVALSEC=5', u'PHYBLKSIZE=512', u'SDUUID=6b892067-ac7e-4039-a7be-5b7387525b16', u'CLASS=Data', u'TYPE=FCP', u'LEASETIMESEC=60', u'IOOPTIMEOUTSEC=10', u'POOL_UUID=c4e5889a-348d-11e2-83fe-9f72369eab30', u'POOL_DESCRIPTION=Default', u'ROLE=Master', u'MASTER_VERSION=1', u'POOL_DOMAINS=6b892067-ac7e-4039-a7be-5b7387525b16:Active,45cdff8b-89c0-4926-a8c7-80354064bfa6:Active,772472cb-2429-466d-96cd-00b3301ef78a:Active,5d7b5178-3790-472c-a13b-b481eb27a61e:Active', u'_SHA_CKSUM=716ea7492d9134a4b13de27ac61a336ad474727c', u'POOL_SPM_ID=2', u'POOL_SPM_LVER=287']
>Thread-15917288::DEBUG::2013-11-20 05:00:16,529::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7c2a4a36-ca76-43d0-b3ec-4fedee193c8d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1297' at 'public_getStorageDomainInfo'
>Thread-15917288::DEBUG::2013-11-20 05:00:16,531::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15917288::DEBUG::2013-11-20 05:00:16,532::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15917288::DEBUG::2013-11-20 05:00:16,533::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`7c2a4a36-ca76-43d0-b3ec-4fedee193c8d`::Granted request
>Thread-15917288::DEBUG::2013-11-20 05:00:16,534::task::495::TaskManager.Task::(_debug) Task 7424383f-14fb-45e3-a85b-dd22eefa2115: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15917288::DEBUG::2013-11-20 05:00:16,535::task::495::TaskManager.Task::(_debug) Task 7424383f-14fb-45e3-a85b-dd22eefa2115: ref 1 aborting False
>Thread-15917288::WARNING::2013-11-20 05:00:16,539::sp::1065::Storage.StoragePool::(getInfo) Could not get full domain information, it is probably unavailable
>Traceback (most recent call last):
>  File "/usr/share/vdsm/storage/sp.py", line 1061, in getInfo
>  File "/usr/share/vdsm/storage/sd.py", line 663, in isISO
>  File "/usr/share/vdsm/storage/sd.py", line 620, in getMetaParam
>  File "/usr/share/vdsm/storage/persistentDict.py", line 63, in __getitem__
>  File "/usr/share/vdsm/storage/persistentDict.py", line 173, in __getitem__
>KeyError: 'CLASS'
>Thread-15917288::DEBUG::2013-11-20 05:00:16,541::task::495::TaskManager.Task::(_debug) Task 7424383f-14fb-45e3-a85b-dd22eefa2115: finished: {'info': {'uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'vguuid': 'aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', 'lver': 287, 'state': 'OK', 'version': '2', 'role': 'Master', 'master_ver': 1, 'spm_id': 2, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'OS'}}
>Thread-15917288::DEBUG::2013-11-20 05:00:16,542::task::495::TaskManager.Task::(_debug) Task 7424383f-14fb-45e3-a85b-dd22eefa2115: moving from state preparing -> state finished
>Thread-15917288::DEBUG::2013-11-20 05:00:16,543::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15917288::DEBUG::2013-11-20 05:00:16,543::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917288::DEBUG::2013-11-20 05:00:16,544::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15917288::DEBUG::2013-11-20 05:00:16,545::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (0 active users)
>Thread-15917288::DEBUG::2013-11-20 05:00:16,545::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free, finding out if anyone is waiting for it.
>Thread-15917288::DEBUG::2013-11-20 05:00:16,546::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', Clearing records.
>Thread-15917288::DEBUG::2013-11-20 05:00:16,547::task::495::TaskManager.Task::(_debug) Task 7424383f-14fb-45e3-a85b-dd22eefa2115: ref 0 aborting False
>Thread-15917288::INFO::2013-11-20 05:00:16,548::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStorageDomainInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'uuid': '6b892067-ac7e-4039-a7be-5b7387525b16', 'vguuid': 'aK7xIW-K1tt-PB18-Inq2-4CkH-gEEx-jxRPYn', 'lver': 287, 'state': 'OK', 'version': '2', 'role': 'Master', 'master_ver': 1, 'spm_id': 2, 'type': 'FCP', 'class': 'Data', 'pool': [u'c4e5889a-348d-11e2-83fe-9f72369eab30'], 'name': 'OS'}}
>Thread-15917288::DEBUG::2013-11-20 05:00:16,549::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/bin/sudo -n /sbin/service ksmtuned retune' (cwd None)
>Thread-15917285::DEBUG::2013-11-20 05:00:16,731::utils::579::Storage.Misc.excCmd::(execCmd) '/usr/libexec/vdsm/hooks/before_vm_migrate_destination/50_vhostmd' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 05:00:17,174::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>Thread-15917288::DEBUG::2013-11-20 05:00:17,520::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15917288::DEBUG::2013-11-20 05:00:17,524::vm::358::vm.Vm::(_startUnderlyingVm) vmId=`7feb052b-c76b-45f2-bdea-005f3caf9b75`::_ongoingCreations released
>Thread-15917292::DEBUG::2013-11-20 05:00:17,525::vm::346::vm.Vm::(_startUnderlyingVm) vmId=`199a4bf8-ec57-49ec-93e6-2c9c32169418`::_ongoingCreations acquired
>Thread-15917288::DEBUG::2013-11-20 05:00:17,526::libvirtvm::1143::vm.Vm::(_waitForIncomingMigrationFinish) vmId=`7feb052b-c76b-45f2-bdea-005f3caf9b75`::Waiting 300 seconds for end of migration
>Thread-15917292::INFO::2013-11-20 05:00:17,526::libvirtvm::1061::vm.Vm::(_run) vmId=`199a4bf8-ec57-49ec-93e6-2c9c32169418`::VM wrapper has started
>Thread-15917292::INFO::2013-11-20 05:00:17,528::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: prepareVolume, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=9a7c3c64-cd9f-4d1f-b3b6-e7cd16bdf4a2 volUUID=a2b4cb5a-09da-4e66-aed3-c105832f9185)
>Thread-15917292::DEBUG::2013-11-20 05:00:17,528::task::495::TaskManager.Task::(_debug) Task 9bdbeb7d-8520-4176-923b-9f6baf64a434: moving from state init -> state preparing
>Thread-15917292::DEBUG::2013-11-20 05:00:17,530::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`96c8b1a0-4c1a-4e71-9e22-6cbac6674169`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1690' at 'public_prepareVolume'
>Thread-15917292::DEBUG::2013-11-20 05:00:17,531::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15917292::DEBUG::2013-11-20 05:00:17,532::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' is free. Now locking as 'shared' (1 active user)
>Thread-15917292::DEBUG::2013-11-20 05:00:17,532::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`96c8b1a0-4c1a-4e71-9e22-6cbac6674169`::Granted request
>Thread-15917292::DEBUG::2013-11-20 05:00:17,533::task::495::TaskManager.Task::(_debug) Task 9bdbeb7d-8520-4176-923b-9f6baf64a434: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15917292::DEBUG::2013-11-20 05:00:17,533::task::495::TaskManager.Task::(_debug) Task 9bdbeb7d-8520-4176-923b-9f6baf64a434: ref 1 aborting False
>Thread-15917292::DEBUG::2013-11-20 05:00:17,534::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.9a7c3c64-cd9f-4d1f-b3b6-e7cd16bdf4a2`ReqID=`f6c84c61-bf3a-454d-acfd-6f7485c1153d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1695' at 'public_prepareVolume'
>Thread-15917292::DEBUG::2013-11-20 05:00:17,534::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.9a7c3c64-cd9f-4d1f-b3b6-e7cd16bdf4a2' for lock type 'exclusive'
>Thread-15917292::DEBUG::2013-11-20 05:00:17,535::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15917292::DEBUG::2013-11-20 05:00:17,536::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Thread-15917285::DEBUG::2013-11-20 05:00:17,926::utils::579::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15917285::INFO::2013-11-20 05:00:17,938::hooks::51::root::(_runHooksDir) 
>Thread-15917320::DEBUG::2013-11-20 05:00:17,940::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('dab1628d-5a54-49ba-9685-47e4c156842c',) {}
>Thread-15917320::DEBUG::2013-11-20 05:00:17,942::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Migration Destination', 'acpiEnable': 'true', 'pid': '0', 'displayIp': '0', 'displayPort': '-1', 'displaySecurePort': '-1', 'cdrom': '', 'pauseCode': 'NOERR', 'kvmEnable': 'true', 'network': {}, 'vmId': 'dab1628d-5a54-49ba-9685-47e4c156842c', 'monitorResponse': '0', 'disks': {}, 'boot': 'cd', 'elapsedTime': '946', 'vmType': 'kvm', 'timeOffset': '0', 'clientIp': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15917285::DEBUG::2013-11-20 05:00:17,944::clientIF::786::vds::(migrationCreate) Destination VM creation succeeded
>Thread-15917285::DEBUG::2013-11-20 05:00:17,945::clientIF::59::vds::(wrapper) return migrationCreate with {'status': {'message': 'Done', 'code': 0}, 'migrationPort': 0, 'params': {'bridge': 'DMZ', 'acpiEnable': 'true', 'emulatedMachine': 'rhel6.2.0', 'afterMigrationStatus': 'Up', 'pid': '0', 'transparentHugePages': 'true', 'displayPort': '-1', 'displaySecurePort': '-1', 'spiceSslCipherSuite': 'DEFAULT', 'cpuType': 'Westmere', 'custom': {}, 'pauseCode': 'NOERR', 'migrationDest': 'libvirt', 'macAddr': '00:1a:4a:a8:64:1b', 'boot': 'cd', 'smp': '2', 'vmType': 'kvm', 'memSize': 6144, 'smpCoresPerSocket': '1', 'vmName': 'Nuuk', 'spiceMonitors': '1', 'nice': '0', 'status': 'Migration Destination', 'vmId': '7feb052b-c76b-45f2-bdea-005f3caf9b75', 'displayIp': '0', 'drives': [{'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'blockDev': True, 'serial': 'c5-84aa-02c1972ac96c', 'if': 'virtio', 'index': '0', 'iface': 'virtio', 'apparentsize': '42949672960', 'boot': 'true', 'imageID': 'af8a32e4-b3f3-4ec5-84aa-02c1972ac96c', 'readonly': 'False', 'shared': False, 'truesize': '42949672960', 'type': 'disk', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'reqsize': '0', 'format': 'raw', 'bus': '0', 'address': {'slot': '0x05', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'disk', 'path': u'/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/af8a32e4-b3f3-4ec5-84aa-02c1972ac96c/d8ad65aa-4f0b-49f3-ba3f-c9c2c11e6a25', 'propagateErrors': 'off', 'name': 'vda', 'volumeID': 'd8ad65aa-4f0b-49f3-ba3f-c9c2c11e6a25', 'alias': 'virtio-disk0', 'volumeChain': [{'path': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/af8a32e4-b3f3-4ec5-84aa-02c1972ac96c/d8ad65aa-4f0b-49f3-ba3f-c9c2c11e6a25', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'volumeID': 'd8ad65aa-4f0b-49f3-ba3f-c9c2c11e6a25', 'imageID': 'af8a32e4-b3f3-4ec5-84aa-02c1972ac96c'}]}], 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/ubuntu-12.04.1-server-amd64.iso', 'guestIPs': '', 'spiceSecureChannels': 'smain,sinputs', 'nicModel': 'pv', 'keyboardLayout': 'en-us', 'kvmEnable': 'true', 'pitReinjection': 'false', 'devices': [{'device': 'memballoon', 'specParams': {'model': 'none'}, 'type': 'balloon'}, {'device': 'virtio-serial', 'alias': 'virtio-serial0', 'type': 'controller', 'address': {'slot': '0x04', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}}, {'device': 'qxl', 'specParams': {'vram': '65536'}, 'address': {'slot': '0x02', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'type': 'video', 'alias': 'video0'}, {'nicModel': 'virtio', 'macAddr': '00:1a:4a:a8:64:1b', 'linkActive': True, 'network': 'DMZ', 'alias': 'net0', 'address': {'slot': '0x03', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'bridge', 'type': 'interface', 'name': 'vnet5'}, {'poolID': 'c4e5889a-348d-11e2-83fe-9f72369eab30', 'if': 'virtio', 'index': '0', 'iface': 'virtio', 'apparentsize': '42949672960', 'boot': 'true', 'imageID': 'af8a32e4-b3f3-4ec5-84aa-02c1972ac96c', 'readonly': 'False', 'shared': False, 'truesize': '42949672960', 'type': 'disk', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'reqsize': '0', 'format': 'raw', 'bus': '0', 'address': {'slot': '0x05', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'disk', 'path': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/af8a32e4-b3f3-4ec5-84aa-02c1972ac96c/d8ad65aa-4f0b-49f3-ba3f-c9c2c11e6a25', 'propagateErrors': 'off', 'name': 'vda', 'volumeID': 'd8ad65aa-4f0b-49f3-ba3f-c9c2c11e6a25', 'alias': 'virtio-disk0', 'volumeChain': [{'path': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/6b892067-ac7e-4039-a7be-5b7387525b16/images/af8a32e4-b3f3-4ec5-84aa-02c1972ac96c/d8ad65aa-4f0b-49f3-ba3f-c9c2c11e6a25', 'domainID': '6b892067-ac7e-4039-a7be-5b7387525b16', 'volumeID': 'd8ad65aa-4f0b-49f3-ba3f-c9c2c11e6a25', 'imageID': 'af8a32e4-b3f3-4ec5-84aa-02c1972ac96c'}]}, {'index': 2, 'iface': 'ide', 'name': 'hdc', 'format': 'raw', 'alias': 'ide0-1-0', 'readonly': 'True', 'truesize': 0, 'address': {'bus': '1', 'controller': '0', 'type': 'drive', 'target': '0', 'unit': '0'}, 'device': 'cdrom', 'shared': False, 'path': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/ubuntu-12.04.1-server-amd64.iso', 'propagateErrors': 'off', 'type': 'disk'}, {'device': 'usb', 'alias': 'usb0', 'type': 'controller', 'address': {'slot': '0x01', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x2'}}, {'device': 'ide', 'alias': 'ide0', 'type': 'controller', 'address': {'slot': '0x01', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x1'}}, {'device': 'unix', 'alias': 'channel0', 'type': 'channel', 'address': {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '1'}}, {'device': 'unix', 'alias': 'channel1', 'type': 'channel', 'address': {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '2'}}, {'device': 'spicevmc', 'alias': 'channel2', 'type': 'channel', 'address': {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '3'}}], 'username': 'Unknown', 'timeOffset': '1', 'clientIp': '', 'display': 'qxl'}}
>Thread-15917321::DEBUG::2013-11-20 05:00:17,956::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('199a4bf8-ec57-49ec-93e6-2c9c32169418',) {}
>Thread-15917321::DEBUG::2013-11-20 05:00:17,957::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Migration Destination', 'acpiEnable': 'true', 'pid': '0', 'displayIp': '0', 'displayPort': '-1', 'displaySecurePort': '-1', 'cdrom': '', 'pauseCode': 'NOERR', 'kvmEnable': 'true', 'network': {}, 'vmId': '199a4bf8-ec57-49ec-93e6-2c9c32169418', 'monitorResponse': '0', 'disks': {}, 'boot': 'cd', 'elapsedTime': '947', 'vmType': 'kvm', 'timeOffset': '0', 'clientIp': '', 'displayType': 'qxl', 'nice': ''}]}
>Thread-15917322::DEBUG::2013-11-20 05:00:17,965::clientIF::54::vds::(wrapper) [192.168.100.135]::call getVmStats with ('7feb052b-c76b-45f2-bdea-005f3caf9b75',) {}
>Thread-15917322::DEBUG::2013-11-20 05:00:17,966::clientIF::59::vds::(wrapper) return getVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Migration Destination', 'acpiEnable': 'true', 'pid': '0', 'displayIp': '0', 'displayPort': '-1', 'displaySecurePort': '-1', 'cdrom': '/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/45cdff8b-89c0-4926-a8c7-80354064bfa6/images/11111111-1111-1111-1111-111111111111/ubuntu-12.04.1-server-amd64.iso', 'pauseCode': 'NOERR', 'kvmEnable': 'true', 'network': {}, 'vmId': '7feb052b-c76b-45f2-bdea-005f3caf9b75', 'monitorResponse': '0', 'disks': {}, 'boot': 'cd', 'elapsedTime': '772', 'vmType': 'kvm', 'timeOffset': '1', 'clientIp': '', 'displayType': 'qxl', 'nice': ''}]}
>Dummy-15912651::DEBUG::2013-11-20 05:00:18,120::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.238536 s, 4.3 MB/s\n'; <rc> = 0
>Thread-15917292::DEBUG::2013-11-20 05:00:18,491::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15917292::DEBUG::2013-11-20 05:00:18,838::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15917292::DEBUG::2013-11-20 05:00:18,841::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=7 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15917292::DEBUG::2013-11-20 05:00:19,244::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.000249762 s, 2.0 MB/s\n'; <rc> = 0
>Thread-15917292::DEBUG::2013-11-20 05:00:19,254::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.000249762 s, 2.0 MB/s'], size: 512
>Thread-15917292::DEBUG::2013-11-20 05:00:19,256::blockVolume::587::Storage.Misc.excCmd::(getMetadata) '/bin/dd iflag=direct skip=7 bs=512 if=/dev/6b892067-ac7e-4039-a7be-5b7387525b16/metadata count=1' (cwd None)
>Thread-15917292::DEBUG::2013-11-20 05:00:19,697::blockVolume::587::Storage.Misc.excCmd::(getMetadata) SUCCESS: <err> = '1+0 records in\n1+0 records out\n512 bytes (512 B) copied, 0.00023659 s, 2.2 MB/s\n'; <rc> = 0
>Thread-15917292::DEBUG::2013-11-20 05:00:19,698::misc::306::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '512 bytes (512 B) copied, 0.00023659 s, 2.2 MB/s'], size: 512
>Thread-15917292::INFO::2013-11-20 05:00:19,700::image::340::Storage.Image::(getChain) sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 imgUUID=9a7c3c64-cd9f-4d1f-b3b6-e7cd16bdf4a2 chain=[<storage.blockVolume.BlockVolume instance at 0x7f48ac602e18>] 
>Thread-15917292::DEBUG::2013-11-20 05:00:19,701::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --autobackup n --available y 6b892067-ac7e-4039-a7be-5b7387525b16/a2b4cb5a-09da-4e66-aed3-c105832f9185' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 05:00:20,141::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) 'dd if=/rhev/data-center/c4e5889a-348d-11e2-83fe-9f72369eab30/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000' (cwd None)
>libvirtEventLoop::DEBUG::2013-11-20 05:00:20,543::libvirtvm::1401::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`7feb052b-c76b-45f2-bdea-005f3caf9b75`::event Started detail 1 opaque None
>Thread-15917292::DEBUG::2013-11-20 05:00:20,688::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15917292::DEBUG::2013-11-20 05:00:20,692::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15917292::DEBUG::2013-11-20 05:00:20,693::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15917292::DEBUG::2013-11-20 05:00:20,696::resourceManager::155::ResourceManager.Request::(__init__) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.a2b4cb5a-09da-4e66-aed3-c105832f9185`ReqID=`de39c061-5a6f-4d40-8fa6-b8a37744282d`::Request was made in '/usr/share/vdsm/storage/resourceFactories.py' line '149' at '__getResourceCandidatesList'
>Thread-15917292::DEBUG::2013-11-20 05:00:20,698::resourceManager::463::ResourceManager::(registerResource) Trying to register resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.a2b4cb5a-09da-4e66-aed3-c105832f9185' for lock type 'exclusive'
>Thread-15917292::DEBUG::2013-11-20 05:00:20,700::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.a2b4cb5a-09da-4e66-aed3-c105832f9185' is free. Now locking as 'exclusive' (1 active user)
>Thread-15917292::DEBUG::2013-11-20 05:00:20,702::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_volumeNS.a2b4cb5a-09da-4e66-aed3-c105832f9185`ReqID=`de39c061-5a6f-4d40-8fa6-b8a37744282d`::Granted request
>Thread-15917292::DEBUG::2013-11-20 05:00:20,704::resourceManager::505::ResourceManager::(registerResource) Resource '6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.9a7c3c64-cd9f-4d1f-b3b6-e7cd16bdf4a2' is free. Now locking as 'exclusive' (1 active user)
>Thread-15917292::DEBUG::2013-11-20 05:00:20,706::resourceManager::192::ResourceManager.Request::(grant) ResName=`6b892067-ac7e-4039-a7be-5b7387525b16_imageNS.9a7c3c64-cd9f-4d1f-b3b6-e7cd16bdf4a2`ReqID=`f6c84c61-bf3a-454d-acfd-6f7485c1153d`::Granted request
>Thread-15917292::DEBUG::2013-11-20 05:00:20,712::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15917292::DEBUG::2013-11-20 05:00:20,714::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)
>Dummy-15912651::DEBUG::2013-11-20 05:00:20,991::storage_mailbox::623::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0955684 s, 10.7 MB/s\n'; <rc> = 0
>Thread-15917323::INFO::2013-11-20 05:00:21,012::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
>Thread-15917323::DEBUG::2013-11-20 05:00:21,013::task::495::TaskManager.Task::(_debug) Task 121ae440-5445-44b8-a443-4033f246f746: moving from state init -> state preparing
>Thread-15917323::DEBUG::2013-11-20 05:00:21,014::task::495::TaskManager.Task::(_debug) Task 121ae440-5445-44b8-a443-4033f246f746: finished: {u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000736951828003', 'lastCheck': 1384923619.9274671, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00182795524597', 'lastCheck': 1384923619.9269249, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00145697593689', 'lastCheck': 1384923619.9277699, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0823199748993', 'lastCheck': 1384923613.2330511, 'valid': True, 'code': 0}}
>Thread-15917323::DEBUG::2013-11-20 05:00:21,015::task::495::TaskManager.Task::(_debug) Task 121ae440-5445-44b8-a443-4033f246f746: moving from state preparing -> state finished
>Thread-15917323::DEBUG::2013-11-20 05:00:21,015::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
>Thread-15917323::DEBUG::2013-11-20 05:00:21,016::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15917323::DEBUG::2013-11-20 05:00:21,017::task::495::TaskManager.Task::(_debug) Task 121ae440-5445-44b8-a443-4033f246f746: ref 0 aborting False
>Thread-15917323::INFO::2013-11-20 05:00:21,017::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, u'6b892067-ac7e-4039-a7be-5b7387525b16': {'delay': '0.000736951828003', 'lastCheck': 1384923619.9274671, 'valid': True, 'code': 0}, u'45cdff8b-89c0-4926-a8c7-80354064bfa6': {'delay': '0.00182795524597', 'lastCheck': 1384923619.9269249, 'valid': True, 'code': 0}, u'5d7b5178-3790-472c-a13b-b481eb27a61e': {'delay': '0.00145697593689', 'lastCheck': 1384923619.9277699, 'valid': True, 'code': 0}, u'772472cb-2429-466d-96cd-00b3301ef78a': {'delay': '0.0823199748993', 'lastCheck': 1384923613.2330511, 'valid': True, 'code': 0}}
>Thread-15917292::DEBUG::2013-11-20 05:00:21,436::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15917292::DEBUG::2013-11-20 05:00:21,669::lvm::509::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex
>Thread-15917292::DEBUG::2013-11-20 05:00:21,678::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvchange --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --refresh 6b892067-ac7e-4039-a7be-5b7387525b16/a2b4cb5a-09da-4e66-aed3-c105832f9185' (cwd None)
>Thread-15913639::INFO::2013-11-20 05:00:22,364::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, args: ( sdUUID=6b892067-ac7e-4039-a7be-5b7387525b16 spUUID=c4e5889a-348d-11e2-83fe-9f72369eab30 imgUUID=6f97dfa5-ee35-4f37-8495-1e6b3136b92c volUUID=5eaa8dde-10d8-45b5-9e10-5417c297c4be)
>Thread-15913639::DEBUG::2013-11-20 05:00:22,365::task::495::TaskManager.Task::(_debug) Task 4f54fe31-ae26-4f01-a195-87cedc3d7dfa: moving from state init -> state preparing
>Thread-15913639::DEBUG::2013-11-20 05:00:22,368::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5cb23d2e-4aa3-4036-b55d-e6fdf16292c7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1615' at 'public_getVolumeSize'
>Thread-15913639::DEBUG::2013-11-20 05:00:22,368::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' for lock type 'shared'
>Thread-15913639::DEBUG::2013-11-20 05:00:22,369::resourceManager::481::ResourceManager::(registerResource) Resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' found in shared state and queue is empty, Joining current shared lock (2 active users)
>Thread-15913639::DEBUG::2013-11-20 05:00:22,370::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.6b892067-ac7e-4039-a7be-5b7387525b16`ReqID=`5cb23d2e-4aa3-4036-b55d-e6fdf16292c7`::Granted request
>Thread-15913639::DEBUG::2013-11-20 05:00:22,371::task::495::TaskManager.Task::(_debug) Task 4f54fe31-ae26-4f01-a195-87cedc3d7dfa: _resourcesAcquired: Storage.6b892067-ac7e-4039-a7be-5b7387525b16 (shared)
>Thread-15913639::DEBUG::2013-11-20 05:00:22,372::task::495::TaskManager.Task::(_debug) Task 4f54fe31-ae26-4f01-a195-87cedc3d7dfa: ref 1 aborting False
>Thread-15913639::DEBUG::2013-11-20 05:00:22,373::task::495::TaskManager.Task::(_debug) Task 4f54fe31-ae26-4f01-a195-87cedc3d7dfa: finished: {'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15913639::DEBUG::2013-11-20 05:00:22,374::task::495::TaskManager.Task::(_debug) Task 4f54fe31-ae26-4f01-a195-87cedc3d7dfa: moving from state preparing -> state finished
>Thread-15913639::DEBUG::2013-11-20 05:00:22,375::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.6b892067-ac7e-4039-a7be-5b7387525b16': < ResourceRef 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16', isValid: 'True' obj: 'None'>}
>Thread-15913639::DEBUG::2013-11-20 05:00:22,375::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
>Thread-15913639::DEBUG::2013-11-20 05:00:22,376::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16'
>Thread-15913639::DEBUG::2013-11-20 05:00:22,376::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.6b892067-ac7e-4039-a7be-5b7387525b16' (1 active users)
>Thread-15913639::DEBUG::2013-11-20 05:00:22,377::task::495::TaskManager.Task::(_debug) Task 4f54fe31-ae26-4f01-a195-87cedc3d7dfa: ref 0 aborting False
>Thread-15913639::INFO::2013-11-20 05:00:22,377::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getVolumeSize, Return response: {'status': {'message': 'OK', 'code': 0}, 'truesize': '85899345920', 'apparentsize': '85899345920'}
>Thread-15917292::DEBUG::2013-11-20 05:00:22,577::lvm::374::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0
>Thread-15917292::DEBUG::2013-11-20 05:00:22,579::lvm::565::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex
>Thread-15917292::DEBUG::2013-11-20 05:00:22,580::lvm::577::OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex
>Thread-15917292::INFO::2013-11-20 05:00:22,581::volume::517::Storage.Volume::(prepare) Volume: preparing volume 6b892067-ac7e-4039-a7be-5b7387525b16/a2b4cb5a-09da-4e66-aed3-c105832f9185
>Thread-15917292::DEBUG::2013-11-20 05:00:22,582::lvm::480::OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex
>Thread-15917292::DEBUG::2013-11-20 05:00:22,583::lvm::374::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm lvs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \\"a%3600508b1001c7159da8b28c482e8a07c|3600c0ff0001520c7414f605001000000|3600c0ff000152111c050605001000000%\\", \\"r%.*%\\" ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1 }  backup {  retain_min = 50  retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 6b892067-ac7e-4039-a7be-5b7387525b16' (cwd None)

Actions: View

Attachments on bug 1032598: 827386 | 829107 | 829108 | 832144